Abstract

Today’s high resolution, high frame rate cameras in autonomous vehicles generate a large volume of data that needs to be transferred and processed by a downstream processor or machine learning (ML) accelerator to enable intelligent computing tasks, such as multi-object detection and tracking. The massive amount of data transfer incurs significant energy, latency, and bandwidth bottlenecks, which hinders real-time processing. To mitigate this problem, we propose an algorithm-hardware co-design framework called Processing-in-Pixel-in-Memory-based object Detection and Tracking (P 2 M-DeTrack). P 2 M-DeTrack is based on a custom faster R-CNN-based model that is distributed partly inside the pixel array (front-end) and partly in a separate FPGA/ASIC (back-end). The proposed front-end in-pixel processing down-samples the input feature maps significantly with judiciously optimized strided convolution …

Metadata

publication

2022 IFIP/IEEE 30th International Conference on Very Large Scale Integration …, 2022

year

2022

publication date

2022/10/3

authors

Gourav Datta, Souvik Kundu, Zihan Yin, Joe Mathai, Zeyu Liu, Zixu Wang, Mulin Tian, Shunlin Lu, Ravi Teja Lakkireddy, Andrew Schmidt, Wael Abd-Almageed, Ajey Jacob, Akhilesh Jaiswal, Peter Beerel

link

https://ieeexplore.ieee.org/abstract/document/9939582/

resource_link

https://arxiv.org/pdf/2205.14285

conference

2022 IFIP/IEEE 30th International Conference on Very Large Scale Integration (VLSI-SoC)

pages

1-6

publisher

IEEE