Abstract

Experimentation is an essential tool for developing networked and distributed systems. However, it is inherently complex due to the concurrent, asynchronous, heterogeneous, and prototype-based systems that must be integrated into representative scenarios to conduct valid evaluations. This paper offers a retrospective on the development and use of MAGI, an orchestration tool, that translates an experiment specification into an execution on an emulation-based testbed with high-level directives for message passing, remote process execution, and failure tracking, for conducting large and complex experiments. The MAGI tool has been used for more than seven years in a variety of experiments, including undergraduate education, anonymous communication, cyber-physical systems, and attacker-defender games on the DETER testbed. We hope the insights and takeaways learned from using our tool will aid in developing the next-generation experiment management tools.

Metadata

publication

13th USENIX Workshop on Cyber Security Experimentation and Test (CSET 20), 2020

year

2020

publication date

2020

authors

Alefiya Hussain, Prateek Jaipuria, Geoff Lawler, Stephen Schwab, Terry Benzel

link

https://www.usenix.org/conference/cset20/presentation/hussain

resource_link

https://www.usenix.org/system/files/cset20-paper-hussain.pdf

conference

13th USENIX Workshop on Cyber Security Experimentation and Test (CSET 20)