Abstract

Scientific workflows are one of the well-established pillars of modern large-scale computational science. More recently, scientists have started to leverage machine learning (ML) capabilities in their workflows, leading to a new category of scientific workflows, denoted as scientific ML workflows. ML is not only about training and inference, modern ML workflows also involve complex data processing steps before the training can start, which are not often accounted for in most performance studies. In this work, we consider scientific ML workflows, from data pre-processing to training, inference, and model evaluation. We aim to explore (i) how scientific ML workflows differ from more traditional scientific workflows and; (ii) how we can characterize ML workflows both in terms of execution time and data movements when executing on a target cloud platform. We select three representative workflows, ranging from image …

Date

2021

Authors

Patrycja Krawczuk, George Papadimitriou, Ryan Tanaka, Tu Mai Anh Do, Srujana Subramanya, Shubham Nagarkar, Aditi Jain, Kelsie Lam, Anirban Mandal, Loïc Pottier, Ewa Deelman

Conference

2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS)

Pages

58-65

Publisher

IEEE