Abstract

In contemporary scientific research, data acquisition and analysis platforms have grown increasingly complex, often spanning multiple facilities with diverse internal structures. Efficiently managing the interactions between job scheduling, resource allocation, and networking across these distributed systems requires a robust simulation framework. However, existing simulators fall short in capturing the detailed interactions necessary for comprehensive analysis of large-scale distributed environments. To address this gap, we introduce DISTRI, a versatile framework specifically designed for the development and testing of distributed multi-facility workflows. DISTRI allows for customizable facility configurations and includes built-in support for distributed, resilient scheduling and resource management, alongside detailed network simulation for data communication. Key features of DISTRI encompass inter- and intra …

Metadata

publication

2024 IEEE International Conference on Big Data (BigData), 4167-4177, 2024

year

2024

publication date

2024/12/15

authors

Imtiaz Mahmud, Pawel Zuk, Cong Wang, Mariam Kiran, Kesheng Wu, Komal Thareja, Krishnan Raghavan, Anirban Mandal, Ewa Deelman

link

https://ieeexplore.ieee.org/abstract/document/10825783/

conference

2024 IEEE International Conference on Big Data (BigData)

pages

4167-4177

publisher

IEEE