- By:
- Wang, Teng; Oral, Hakki S; Wang, Yandong; Settlemyer, Bradley W; Atchley, Edward S; Yu, Weikuan
- Page Number:
- 1
- Issue Number:
- N/A
- Book Title:
- 2014 IEEE International Conference on Big Data (Big Data)
- Publication Date:
- April 1, 2024
- Publisher Location:
- IEEE, New Jersey, United States of America
- Conference Name:
- IEEE BigData
- Conference Location:
- Washington DC, District of Columbia, United States of America
- View DOI Listing:
- https://doi.org/10.1109/BigData.2014.7004215
Abstract
The growth of computing power on large-scale sys- tems requires commensurate high-bandwidth I/O system. Many parallel file systems are designed to provide fast sustainable I/O in response to applications’ soaring requirements. To meet this need, a novel system is imperative to temporarily buffer the bursty I/O and gradually flush datasets to long-term parallel file systems. In this paper, we introduce the design of BurstMem, a high- performance burst buffer system. BurstMem provides a storage framework with efficient storage and communication manage- ment strategies. Our experiments demonstrate that BurstMem is able to speed up the I/O performance of scientific applications by up to 8.5× on leadership computer systems.
