Data Lake vs Data Warehouse: What Each One Is For

Data Lake vs Data Warehouse is a choice between keeping data raw and keeping it ready. A lake stores data in its original shape. A warehouse stores cleaned, modeled tables. Both are useful, and most teams end up with some of each.

Gouache painting split in two: on the left a calm mountain lake with leaves, branches and a small boat floating on its surface; on the right a tidy warehouse shelf of identical plain boxes in straight rows, one box vermilion.

The Data Warehouse: Cleaned and Modeled

A data warehouse is a database built for questions, not for taking orders. Data from many systems is cleaned, matched and stored in tables with fixed columns and types. Many follow a star layout. A central fact table holds events such as sales. Dimension tables around it hold things such as products and stores.

The schema is defined before the data loads. That’s called schema on write. A row with text in a date column is rejected at the door. The users are analysts, report builders and finance teams. They ask repeatable questions, such as revenue by region for last month, and they expect the same answer every time.

The Data Lake: Raw and Open

A data lake is a large store of files, usually in cloud object storage. Its raw zone keeps them in their original form. A JSON feed, a CSV export, a log, a photo and an audio recording can sit side by side. Nothing’s cleaned on the way in.

In the raw zone, the structure is applied when you read the data. That’s called schema on read. The lake accepts everything, and you’ll learn what a file contains when a query opens it. Many lakes add cleaned zones with schemas next to the raw one. The users are data engineers and data scientists. They explore, build inputs for models and keep history in case a new question appears.

Schema on Write or Schema on Read

The difference is when you pay for structure. In a warehouse you pay at load time. Someone designs the table, writes the transform and fixes the bad rows before anyone queries. Queries are then fast and trustworthy.

In a lake you pay at query time. Loading is easy, but every reader has to understand the files and handle the bad rows. Skipping the cost doesn’t remove it. It moves the cost to every query and every reader.

The Two Side by Side

The diagram below compares the two row by row. For the data kept, the lake holds raw files of every shape, and the warehouse holds cleaned, modeled tables. For the schema, the lake applies it on read, and the warehouse applies it on write.

For the users, the lake serves engineers and data scientists, and the warehouse serves analysts and report builders. For the questions, the lake explores what the data could teach us. The warehouse reports what last month’s sales were. At the bottom, the lakehouse joins both: open files with table features on top.

Diagram of a data lake vs a data warehouse compared row by row: raw files versus modeled tables, schema on read versus schema on write, engineers and data scientists versus analysts, open versus repeatable questions, with the lakehouse joining both below.

Why Many Teams Combine Them

Take a bookstore chain. Its lake holds raw daily files from the tills, click logs from the website and scanned supplier invoices. Its warehouse holds a clean sales table with date, store, book and quantity, plus a table of stores. Staff explore the lake. Managers read the warehouse.

The two aren’t rivals. A common design lands raw data in the lake first, as ELT does. Then it cleans the data in steps and publishes modeled tables for reporting. ETL vs ELT: How Big Data Changed the Way We Load Data covers how the data gets in.

The line has blurred from both sides. Warehouses now store JSON and read files in the lake, and lakes have curated tables of their own.

A lakehouse goes one step further. It keeps data in open files, such as Parquet, in object storage. A table format then adds features on top. Examples are transactions, schema checks and reading an earlier version of a table. Delta Lake and Apache Iceberg are two such formats. Open Table Formats: Delta Lake and Apache Iceberg Explained shows how they work.

How to Choose

Three questions decide it. Who will use the data, and how clean must their answer be? How fast does the shape of the data change? Do you need data that nobody has modeled yet?

Finance reports point to a warehouse. Unknown sources and model training point to a lake. If you need both, a lakehouse lets one copy of the files serve both groups.

Skills matter too. A warehouse needs people who can model data and write careful SQL. A lake also needs people who can work with files, partitions and code outside SQL. Pick the design your team can run, not only the one that looks good on a slide.

Is the Lakehouse the End of the Debate?

One objection is that the lakehouse makes Data Lake vs Data Warehouse pointless. One platform now does both jobs. It narrows the gap, but the two jobs stay different. Someone still has to model, clean and own the tables that finance trusts. A platform doesn’t do that work for you.

The reverse holds too. A lake with no owner and no catalog turns into a swamp, where nobody can find or trust anything. The label on the platform matters less than the discipline around it.

What to Remember

A warehouse stores cleaned, modeled tables and applies the schema when data loads. A lake stores raw files of any shape and applies the schema when data is read. The Data Lake vs Data Warehouse choice follows who the users are and how clean their answers must be.

When I review a design, I ask who owns the trusted tables. If the answer is nobody, that’s the first thing I fix.

A data lake is not a cheaper warehouse, it is a different promise: keep everything now, decide the shape later.

Published by Pinal Dave on SQLAuthority. More of my work at pinaldave.com.


Discover more from SQL Authority with Pinal Dave

Subscribe to get the latest posts sent to your email.

Data Warehousing, Database, ETL, SQL Data Storage
Previous Post
ETL vs ELT: How Big Data Changed the Way We Load Data
Next Post
The 5 Vs of Big Data: Volume, Velocity, Variety, Veracity and Value

Related Posts

20 Comments. Leave new

  • Jagdish Rathore
    October 1, 2013 9:25 am

    Nice Start !!!!!!!!!!!

    Reply
  • Thank you very much for that series.

    Reply
  • Thanks Pinal It’s a wonderful time to have discussion about BIgData…

    Reply
  • Michael Heindel
    October 1, 2013 4:39 pm

    It is not so much about data, but what you do with the data. To me “Big Data” is just a term to describe ALL of the data. It could be in the data warehouse, CRM database, text files, web logs, the list goes on.

    How we use the data to make better decisions, improve processes and reduce risks is what makes “Big Data” valuable.

    Just like the various definitions for Big Data there are numerous tools to process that data. A Hadoop cluster won’t work to alert me of a potential server failure, but it can crunch millions of rows from various sources very efficiently to identify a fraudulent claim. A Python script can alert me to a potential server failure, but would not be efficient at identifying a fraudulent claim.

    Looking forward to following along on this journey as Big Data progresses into better decisions.

    Michael Heindel
    @BeachBum

    Reply
  • Hi Pinal, great initiative.

    Reply
  • Thanks Pinal, Today I started to look for it and I got your email … my good luck :)

    Reply
  • Interesting First day. Thanks!

    Reply
  • Imran Mohammed
    October 2, 2013 5:33 am

    Looking forward to this series. Thanks Pinal, appreciate your time like always.

    I know its too early to ask this question, but whats your recommendation as to when an organization should decide they should move to NOSQL from RDBMS, like when data size reaches 500 GB/ 1 TB ?

    Reply
  • Geetanjali Agarwal
    October 2, 2013 9:26 am

    Thanks Pinal..I searched about big data a lot but did not get any good article on that..hope this 31 days series worked out… Please incude some real world examples as well. Thanks.

    Reply
  • Nice Series Pinal

    Reply
  • good….:-)

    Reply
  • nice start for a new person who want to take a step towards BigData

    Reply
  • Amit Deshmukh
    April 15, 2015 9:55 am

    Nice start pinal for journey towards big data. I really interested to do the study on BIG DATA

    Reply
  • Neelima Bansal
    April 29, 2015 11:24 am

    Nice.. I am really interested to study Big Data and really like the way you have started…

    Reply
  • Girivardhana Reddy
    June 19, 2015 7:19 pm

    Good start Pinal. Thanks Much..

    Reply
  • Pradipta Dasgupta
    June 29, 2016 3:28 pm

    This is a great article and looks very interesting to start up learning about Big Data. Thanks Pinal to set it up for guys like us who want to start learning Big Data

    Reply
  • It’s good to start

    Reply
  • oh it is impressing

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.