Abstract

Video summarization remains a huge challenge in computer vision due to the size of the input videos to be summarized. We propose an efficient, language-only video summarizer that achieves competitive accuracy with high data efficiency. Using only textual captions obtained via a zero-shot approach, we train a language transformer model and forego image representations. This method allows us to perform filtration amongst the representative text vectors and condense the sequence. With our approach, we gain explainability with natural language that comes easily for human interpretation and textual summaries of the videos. An ablation study that focuses on modality and data compression shows that leveraging text modality only effectively reduces input data processing while retaining comparable results.

Metadata

publication

ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and …, 2024

year

2024

publication date

2024/4/14

authors

Yoonsoo Nam, Adam Lehavi, Daniel Yang, Digbalay Bose, Swabha Swayamdipta, Shrikanth Narayanan

link

https://ieeexplore.ieee.org/abstract/document/10445931/

resource_link

https://arxiv.org/pdf/2309.09405

conference

ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

pages

8396-8400

publisher

IEEE