Big Data Architecture today is a pipeline with four jobs: ingest, store, process and serve, with governance underneath. This post explains the lakehouse, the bronze, silver and gold layers, batch and streaming, and where SQL Server 2025 fits.
Change Data Capture or Change Tracking
Change data capture or change tracking serves different sync needs; compare captured detail, retention, cleanup, and operating cost before choosing.
Watermarks for Incremental Extracts
Watermarks for incremental extracts need clear boundaries, durable checkpoints, and a plan for late updates, deletes, and missed rows.
Data Integration Patterns Without Special Tools
Data integration patterns in SQL Server cover full reloads, incremental pulls, change capture, and file drops with clear restart rules.
TechEd Sri Lanka 2010: Two Sessions and a Room Full of Questions
TechEd Sri Lanka ran at Waters Edge in Colombo from February 8 to 10, 2010, and I presented two sessions there. This post looks back at the index session, rated 8.9, and the change data capture session, rated 7.8, and what each one still teaches. It also shares attendee feedback and photos of the people who made the event.
CDC Capture Lag: Read sys.dm_cdc_log_scan_sessions
CDC capture lag measures work committed at the source that the capture process has not yet exposed. Compare known markers, scan sessions and captured LSNs, and distinguish capture progress from a downstream consumer reading the changes.
SQL SERVER – CDC and TRUNCATE – Cannot truncate table because it is published for replication or enabled for Change Data Capture
Does CDC capture a TRUNCATE? No. On a table enabled for Change Data Capture you get Msg 4711: Cannot truncate table. I recreate the error step by step.