How Apache Spark works comes down to a driver that plans the job, executors that run it, and data cut into partitions. This post follows one sales total through two stages and a shuffle, and compares it with a parallel plan in SQL Server.
Big Data Architecture Today: The Lakehouse and Medallion Layers
Big Data Architecture today is a pipeline with four jobs: ingest, store, process and serve, with governance underneath. This post explains the lakehouse, the bronze, silver and gold layers, batch and streaming, and where SQL Server 2025 fits.
Evolution of Big Data: From Flat Files to the Lakehouse
Evolution of Big Data follows one pattern: data grew, the old tool strained, and someone built the next one. This post walks from flat files and the relational model through warehouses and the cloud to the lakehouse.
The 5 Vs of Big Data: Volume, Velocity, Variety, Veracity and Value
The 5 Vs of Big Data are volume, velocity, variety, veracity and value. This post walks a grocery chain through each one, so you can ask the same five questions about your own data before you choose a tool.
Data Lake vs Data Warehouse: What Each One Is For
Data Lake vs Data Warehouse is a choice between raw files and cleaned, modeled tables. This post compares schema on read with schema on write, who uses each, and why many teams combine them in a lakehouse.
ETL vs ELT: How Big Data Changed the Way We Load Data
ETL vs ELT comes down to when you transform: before the load on a separate engine, or after it inside the platform. This post compares the two and runs a small ELT pipeline in T-SQL on JSON orders.
Table Partitioning Is Not Sharding: What Partitioning Really Gives You
See why partitioning is not sharding, build monthly partitions, inspect elimination, and switch a month out for simpler data management.