Abstract

Scientific workflows have enabled large-scale scientific computations and data analysis, and lowered the entry barrier for performing computations in distributed heterogeneous platforms (e.g., HTC and HPC). In spite of impressive achievements to date, large-scale modeling, simulation, and data analytics in the long-tail still face several challenges such as efficient scheduling and execution of large-scale workflows (O(106)) with very short-running tasks (few seconds). While the current trend to support next-generation workflows on leadership class machines have gained much attention in the past years, at the other end of the spectrum scientific workflows from the long-tail science have become larger and require processing massive volumes of data. In this paper, we report on our experience in designing and implementing an HTC workflow for agroecosystem modeling. We leverage well-known (task clustering and …

Metadata

publication

2019 IEEE International Conference on Big Data (Big Data), 4545-4552, 2019

year

2019

publication date

2019/12/9

authors

Rafael Ferreira Da Silva, Rajiv Mayani, Yuning Shi, Armen R Kemanian, Mats Rynge, Ewa Deelman

link

https://ieeexplore.ieee.org/abstract/document/9006107/

resource_link

https://par.nsf.gov/servlets/purl/10142732

conference

2019 IEEE International Conference on Big Data (Big Data)

pages

4545-4552

publisher

IEEE