This paper is published in Volume-10, Issue-5, 2024
Area
Software Engineering
Author
Ramla Suhra
Org/Univ
H-E-B Digital, Texas, USA
Pub. Date
25 October, 2024
Paper ID
V10I5-1324
Publisher
Keywords
Artificial Intelligence, Big Data, Data Lake, Data Lake House, Heterogeneous Data, Data Science, Data Warehouse, Machine Learning, Heterogeneous Computing.

Citationsacebook

IEEE
Ramla Suhra. Building a Comprehensive Enterprise Data Lake Architecture, International Journal of Advance Research, Ideas and Innovations in Technology, www.IJARIIT.com.

APA
Ramla Suhra (2024). Building a Comprehensive Enterprise Data Lake Architecture. International Journal of Advance Research, Ideas and Innovations in Technology, 10(5) www.IJARIIT.com.

MLA
Ramla Suhra. "Building a Comprehensive Enterprise Data Lake Architecture." International Journal of Advance Research, Ideas and Innovations in Technology 10.5 (2024). www.IJARIIT.com.

Abstract

Organizations need to be driven by “data” more than ever to stay ahead of the curve and be competitive. With the tremendous data growth of data both by volume as well as variety, it is no longer sustainable to store the data in traditional data warehouses as they are not designed to be scalable. Data lake architecture which is typically built on top of cheap hardware is the most economically viable solution for this problem as they are elastic and can scale up based on the increasing data needs of an organization. While the solution might seemingly look straightforward there are many nuances associated with this shift in paradigm and a very careful and thoroughly thought through design is necessary when building an enterprise data lake architecture. This white paper explores various aspects related to setting up a comprehensive enterprise data lake which can steer towards the success of the organization. It also touches up on the pit falls and opportunities based on the research and case studies relevant in this area. Finally, a summary and outlook on data lake management is presented to the readers.