April 2022

Conference Paper

A Lakehouse Architecture for the Management and Analysis of Heterogeneous Data for Biomedical Research and Mega-biobanks

By:
Begoli, Edmon ; Goethert, Ian D; Knight, Kathryn E
Page Number:
4643-4651
Issue Number:
1
Book Title:
2021 IEEE International Conference on Big Data (Big Data)
Publication Date:
April 29, 2022
Publisher Location:
IEEE, New Jersey, United States of America
View DOI Listing:
https://doi.org/10.1109/BigData52589.2021.9671534

Abstract

Data Lakehouse is a new paradigm in data architectures that embodies and integrates already established concepts for the systematic management of disparate, large-scale data – a data lake for heterogeneous data management, use of open standards for high-performance querying, and systematic maintenance of the data "freshness". In addition to being a new concept, the data lakehouse is also still a conceptual construct. Many projects that use the lakehouse require maturing, empirical studies, and specific implementations. In this paper, we present our implementation of the data lakehouse concept in a biomedical research and health data analytics domain, and we discuss the implementation of some unique and novel features such as support for specialized access controls in support of HIPAA regulation and IRB protocols, and support for the FAIR standard.