Scribd
Upload
11 views2 pages

Data Engineering

The document discusses data architecture concepts including datalakes, swamps, data warehouses, and datamarts. Datalakes facilitate comprehensive data collection and democratization for skilled users, but require organization to manage data effectively and comply with legal regulations regarding data retention. It also highlights potential costs associated with data egress in cloud services like AWS S3.

Uploaded by

madhavi.mangalara2516
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views2 pages

Data Engineering

The document discusses data architecture concepts including datalakes, swamps, data warehouses, and datamarts. Datalakes facilitate comprehensive data collection and democratization for skilled users, but require organization to manage data effectively and comply with legal regulations regarding data retention. It also highlights potential costs associated with data egress in cloud services like AWS S3.

Uploaded by

madhavi.mangalara2516
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Data Engineering

SAMPLE DATA ARCHITECTURE

DATALAKES, SWAMPS, DATA WAREHOUSES and DATAMARTS


Datalake is like a dumping ground for data. Enables data democratization meaning – people such as Data
Analysts and Data scientists who have the right set of skills can go and get the needed data themselves.

Facilitates – comprehensive data collection

Cost-effective but watch out and the data egress (data leaving the cloud infrastructure )may cost a lot
depending on the cloud service provider used for allowing access to data and the cloud infrastructure.

Datalake: AWS S3 – Blob solution of AWS allowing any data dump there

If using a data lake, recommend to have some sort of organization rather than not have an idea of which
file is where and how, and allowing more people to dump data with no organization at all.

For legal purposes as well, sometimes your organization may not be allowed to delete data for seven
years, so good to have a plan as to how and where you will be storing this data or delete it to comply
with certain regulations and legislations

You might also like