- By:
- Tsaris, Aristeidis ; Hinkle, Jacob D; Lunga, Wadzanai D; Ambrozio Dias, Philipe
- Page Number:
- 27-33
- Book Title:
- 2021 IEEE/ACM Redefining Scalability for Diversely Heterogeneous Architectures Workshop (RSDHA)
- Publication Date:
- March 12, 2026
- Conference Name:
- International Conference for High Performance Computing, Networking, Storage and Analysis (SC '21), Redefining Scalability for Diversely Heterogeneous Architectures Workshop (RSDHA 21)
- Conference Location:
- St. Louis, Missouri, United States of America
- Conference Sponsor:
- Various
- View DOI Listing:
- https://doi.org/10.1109/RSDHA54838.2021.00009
Abstract
In this work we developed two Pytorch libraries using the PyTorch RPC interface for distributed deep learning approaches on high resolution images. The spatial decomposition library allows for distributed training on very large images, which otherwise wouldn’t be possible on a single GPU. The domain parallelism library allows for distributed training across multiple domain unlabeled data, by leveraging the domain separation architecture. Both of those libraries where tested on the Summit supercomputer at Oak Ridge National Laboratory at a moderate scale.
