Abstract

Computational narrative understanding studies the identification, description, and interaction of the elements of a narrative: characters, attributes, events, and relations. Narrative research has given considerable attention to defining and classifying character types. However, these character-type taxonomies do not generalize well because they are small, too simple, or specific to a domain. We require robust and reliable benchmarks to test whether narrative models truly understand the nuances of the character's development in the story. Our work addresses this by curating the Chatter dataset that labels whether a character portrays some attribute for 88148 character-attribute pairs, encompassing 2998 characters, 13324 attributes and 660 movies. We validate a subset of Chatter, called ChatterEval, using human annotations to serve as an evaluation benchmark for the character attribution task in movie scripts. ChatterEval assesses narrative understanding and the long-context modeling capacity of language models.

Metadata

publication

arXiv preprint arXiv:2411.05227, 2024

year

2024

publication date

2024/11/7

authors

Sabyasachee Baruah, Shrikanth Narayanan

link

https://arxiv.org/abs/2411.05227

resource_link

https://arxiv.org/pdf/2411.05227

journal

arXiv preprint arXiv:2411.05227