OLCF’s Advanced Computing Ecosystem (ACE): FY24 Efforts for the DOE Integrated Research Infrastructure (IRI) Program

Abstract

This report highlights significant strides made by Oak Ridge National Laboratory’s Oak Ridge Leadership Computing Facility (OLCF) in advancing computational research and infrastructure. Through the Advanced Computing Ecosystem (ACE) strategic initiative, OLCF has been successfully integrated with DOE’s Integrated Research Infrastructure (IRI) program, establishing itself as a critical framework for enhancing scientific computing capabilities across various domains. This report outlines the activities, accomplishments, and future directions of ACE, emphasizing its role in developing cutting-edge technologies, supporting science pilots, and fostering collaborations that drive scientific innovation. One of the key achievements of ACE in FY24 has been the establishment of a robust testbed environment that facilitates the development and validation of new computing technologies. This testbed, integrated into the larger IRI ecosystem through Energy Sciences Network (ESnet), allows for seamless collaboration among DOE’s Advanced Scientific Computing Research (ASCR) facilities, including the Argonne Leadership Computing Facility (ALCF), the National Energy Research Scientific Computing Center (NERSC), and Jefferson Lab (JLab). The testbed has been instrumental in enabling science pilots to experiment with innovative methodologies in a controlled environment, thereby accelerating the translation of research into practical applications. ACE has been instrumental in advancing a series of science pilots using computational resources from OLCF. These pilots span a wide range of scientific domains, from basic energy sciences to biological and environmental research, demonstrating the versatility and impact of the ACE framework. By integrating cutting-edge technologies, such as AI-driven analytics and real-time data processing, with HPC, these pilots have enabled significant advancements in scientific workflows. For instance, projects like LCLStream and DELERIA have showcased the potential of ACE in optimizing experimental designs, accelerating data analysis, and driving innovations that are critical for addressing some of the most complex scientific challenges of IRI pathfinder projects. ACE has also made significant progress in foundational technology development, particularly in areas such as data movement, interface design, and scheduling. These advancements have been key in addressing the complex requirements of modern scientific workflows, ensuring that OLCF remains at the forefront of high-performance computing (HPC). The report details several ongoing projects, including the development of the OLCF Facility API, Data Streaming technologies, and the Zambeze distributed orchestration system, which together provide a secure and flexible framework for managing computational resources across a distributed infrastructure. In addition to technical achievements, ACE has played a central role in outreach and engagement activi- ties. Through participation in conferences, hackathons, and training sessions, the ACE team has actively contributed to the broader scientific community. These efforts have not only facilitated the exchange of knowledge but have also strengthened collaborations with other DOE facilities and research institutions, further enhancing the impact of the ACE program. Looking ahead, the report outlines strategic plans for FY25, which include continued support for science pilots, further integration with ASCR facilities, and the exploration of new technologies. These initiatives are designed to ensure that ACE remains a dynamic and adaptable platform, capable of meeting the evolving needs of the scientific community. The ongoing collaboration with ORNL’s INTERSECT initiative and the planned technology refreshes underscore ACE’s commitment to maintaining its leadership in HPC.