Abstract

In complex task environments in both nature and human society, structural violations of expectation (VoE) occur with non-trivialfrequency. Agents that are designed to operate in such environments must be capable of open-world learning (OWL), definedas the ability to detect and accommodate out-of-distribution inputs, as well as more complex structural VoEs, without requiringextensive and offline re-training. Until recently, OWL research was relatively constrained and limited to areas such as anomalydetection and concept drift. More recently, agent-based OWL research has witnessed much interest from across the community. To support this research, not just for developing OWL algorithms, but also evaluating them, there is a need for multi-agentenvironments where structural VoEs can be generated, and controlled experiments can be run with relative ease. To addressthis need, we propose a resource called PokerOWL, a platform that is supported on the Gymnasium infrastructure (formerlysupported by OpenAI), which is extensively used in the reinforcement learning and AI gameplaying communities. PokerOWLsupports both a rich VoE generator and a graphical interface for facilitating development and evaluation of OWL methods. Usingan extensive set of experiments and a Poker-playing agent based on Deep Q-Networks, we use PokerOWL to demonstratehow even state-of-the-art agents can struggle to generalize to novel situations without additional OWL capabilities.

Metadata

publication

year

2024

publication date

2024/7/26

authors

Min-Hsueh Chiu, Mayank Kejriwal

link

https://www.researchsquare.com/article/rs-4677706/latest

resource_link

https://www.researchsquare.com/article/rs-4677706/latest.pdf