March 2026

Conference Paper

Towards Cross-Facility Workflows Orchestration through Distributed Automation

By:
Skluzacek, Tyler J; Santos Souza, Renan Francisco ; Coletti, Mark A; Suter, Frederic ; Ferreira Da Silva, Rafael
Page Number:
1-4
Book Title:
Practice and Experience in Advanced Research Computing 2024: Human Powered Computing
Publication Date:
March 12, 2026
Publisher Location:
Association for Computing Machinery, New York, New York, United States of America
Conference Name:
PEARC: Practice and Experience in Advanced Research Computing 2024
Conference Location:
Providence, Rhode Island, United States of America
Conference Sponsor:
ACM
View DOI Listing:
https://doi.org/10.1145/3626203.3670606

Abstract

Modern science relies on end-to-end workflows that incorporate experimental instruments and utilize edge, cloud, or high-performance computing and storage resources. These components are geographically dispersed across various user facilities and interconnected through high-speed networks. In this paper, we present Zambeze, an automated distributed framework designed to facilitate this new class of cross-facility workflows. Utilizing swarm intelligence principles, Zambeze orchestrates science campaigns by managing distributed autonomous agents. These agents can offer a suite of services, including computing, storage, and data management. We demonstrate the feasibility of Zambeze through a real-world application involving electron microscopy, enhanced with Artificial Intelligence capabilities.