Event Details

Speaker: Shunyu Yao, Princeton University (will present Remotely. Virtual Talk will be broadcast in CR#689)

Conference Rm Location: ISI-MDR #689 in-person attendance will be permitted for USC/ISI faculty, staff, students only. Open to the public virtually via Zoom

REMINDER:

Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom.

If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance and let you in.

In-person attendance will be permitted for USC/ISI faculty, staff, students only. Open to the public virtually via the zoom link.

For more information on the NL Seminar series and upcoming talks, please visit:

https://nlg.isi.edu/nl-seminar/

Language agents are AI systems that use large language models (LLMs) to interact with the world. While various methods have been developed, it is often hard to systematically understand or evaluate them. In this talk, we present Cognitive Architectures for Language Agents (CoALA), a theoretical framework grounded in the classical research of cognitive architectures to make sense of existing agents and shed light into future directions. We also present three benchmarks (WebShop, InterCode, Collie) to develop and evaluate language agents using web, code, and grammar respectively. Notably, all three are scalable and practical, with simple and faithful evaluation metics that do not rely on human preference labeling or LLM scoring.

Speaker Bio

Shunyu Yao is a final year Phd student with Karthik Narasimhan at Princeton NLP Group. His research focuses on language agents, and is supported by the Harold W. Dodds Fellowship from Princeton. Homepage: https://ysymyth.github.io/

If speaker approves to be recorded for this NL Seminar talk, it will be posted on our USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI.

Subscribe here to learn more about upcoming seminars: https://www-staging.isi.edu/events/

Hosts: Jon May and Justin Cho