Data Ingestion is how data moves from the systems that create it into the platform that analyzes it. This post compares batch loads, change data capture and event streams, and shows the captured rows from a small SQL Server table.
Big Data Architecture Today: The Lakehouse and Medallion Layers
Big Data Architecture today is a pipeline with four jobs: ingest, store, process and serve, with governance underneath. This post explains the lakehouse, the bronze, silver and gold layers, batch and streaming, and where SQL Server 2025 fits.
Running SQL Code: Execute, Batches and GO in SSMS
Running SQL code in SSMS starts with F5 or the Execute button. This post covers running only a selection, what a batch is, why GO is not T-SQL, why variables disappear after GO, and how GO with a number repeats a batch.
GO Is Not T-SQL: Batches, Variable Scope and GO With a Count
Learn why GO is not T-SQL, how batches reset variable scope, what GO 3 repeats, and how applications should send scripts safely.
Updating a Large Table in Batches With a Moving Key Range
Walk the clustered key a slice at a time, log each batch, and learn why stopping at the first empty batch quietly leaves rows behind.
Splitting a Big Table Into Equal Key Ranges for Batch Work
Equal-width key ranges fail when ids have gaps. NTILE cuts by row count instead, so every worker gets a fair share of the table.
Fast Bulk Inserts From Python
Bulk inserts from Python work best with parameterized executemany, measured batch sizes, a tested fast_executemany path, and clear restart rules.