Introducing WalShadow: Sub-second Postgres replication to ClickHouse from physical WAL | ClickHouseSkip to contentCollapse the terminalCopy logo as SVGDownload full logoDownload logomarkOpen searchOpen region selectorEnglishJapaneseKoreanChineseFrenchSpanishPortugueseArabicOpen menuProductsProductsClickHouse CloudThe best way to use ClickHouse.Available on AWS, GCP, and Azure.Bring Your Own CloudA fully managed ClickHouse service,deployed in your own AWS, GCP or Azure account.ClickHouse Managed PostgresUnified data stack for transactionsand analytics.Managed ClickStackManaged observability with high-performancequeries and long-term retention.Langfuse CloudLLM observability and evaluations for reliable AI applications and agents.Open sourceClickHouseFast open-source OLAP database forreal-time analytics.ClickStackOpen-source observability stack for logs,metrics, traces, and session replays.Agentic Data StackBuild AI-powered applicationswith ClickHouse.chDBIn-process SQL Engine powered byClickHouse, with a Pandas-compatible APISolutionsUse casesReal-time analyticsObservabilityData warehousingMachine learning and GenAIAll use cases ->IndustriesFinancial servicesCybersecurityGaming and entertainmentE-commerce and retailAutomotiveEnergyAll industries ->DocsResourcesCompany resourcesUser storiesBlogEventsNewsLearning and certificationPartnersVideosDemosComparisonsBenchmark hubCostBenchBigQueryPostgreSQLRedshiftSnowflakeElastic ObservabilitySplunkDatadogOpenSearchFor observabilityDatabricksPricingContact usOpen searchToggle the web terminalOpen region selectorEnglishJapaneseKoreanChineseFrenchSpanishPortugueseArabic49.9kSign inGet Started->Scroll to top<-BackBlog/ProductCopy pageCopied!More actionsView as Markdown Open this page in MarkdownOpen in ChatGPT Ask questions about this pageOpen in Claude Ask questions about this pageOpen in v0 Ask questions about this pageIntroducing WalShadow: Sub-second Postgres replication to ClickHouse from physical WALSai SrirampurSep 10, 2026 · 7 minutes readToday, we’re announcing WalShadow, an open-source engine that replicates Postgres data to ClickHouse directly from physical WAL.
In our benchmarks, transactions committed in Postgres became visible in ClickHouse in around 200 ms, while WalShadow sustained 289K rows/sec, effectively keeping pace with the source Postgres instance.
Unlike traditional CDC based systems, WalShadow doesn’t use Postgres logical replication. It consumes the same physical WAL stream used by Postgres replicas, decodes it outside the source database, and writes ClickHouse-native blocks directly into ClickHouse. The result is a replication architecture that gets close to the latency and throughput of a Postgres physical standby, while making the data immediately available for analytics in ClickHouse.
WalShadow supports the complete replication lifecycle, including initial load, continuous replication, schema evolution, restart recovery, and planned source switchovers.
By consuming physical WAL directly, WalShadow eliminates the need for logical replication slots, removes much of the operational overhead associated with logical replication, and significantly reduces resource consumption on the source Postgres instance. It also supports complex schema changes such as ADD COLUMN, RENAME COLUMN, DROP COLUMN, and CREATE TABLE.
WalShadow is fully open source and available today on GitHub.
Bringing WalShadow to ClickHouse Managed Postgres #
For a fully managed experience, we’re also launching WalShadow in private preview for ClickHouse Managed Postgres.

Physical WAL is key to WalShadow’s architecture, but most managed Postgres services don’t expose it to customers, making it impossible to use WalShadow. ClickHouse Managed Postgres manages both sides of the stack, allowing us to integrate WalShadow directly into the Postgres replication layer and provide a native path from Postgres WAL to ClickHouse.
Sign up for the private preview of WalShadow on ClickHouse Managed Postgres.
Architecture: From physical WAL to ClickHouse-native blocks #

WalShadow takes a new approach to Postgres-to-ClickHouse replication, effectively turning ClickHouse into an analytical physical standby.

WalShadow consumes the same WAL stream Postgres generates for physical replication and recovery. Instead of asking the source database to decode changes into logical events, WalShadow processes the WAL outside the source through four stages:

Track the live schema. WalShadow filters catalog WAL records and replays them into a schema-only shadow Postgres instance. This maintains an up-to-date catalog of tables, columns, and Postgres types as the source schema evolves.
Decode data changes in parallel. Heap records are distributed across a pool of Rust-based decoders without passing through the shadow Postgres instance.
Build ClickHouse-native blocks. A batcher groups decoded rows by table into complete ClickHouse-native blocks, preserving data fidelity without intermediate format conversions.
Insert in parallel. A separate pool of inserters writes multiple blocks to ClickHouse concurrently, allowing decoding and insertion to scale independently.

This creates a direct path from Postgres to ClickHouse: no logical decoding output plugin, no Kafka, and no JSON serialization or separate normalization step.
Because WalShadow processes blocks in parallel, they may arrive in ClickHouse out of order. WalShadow preserves correctness by attaching the source WAL position (_lsn) to every row, allowing ClickHouse to retain the latest version for each key. Operations that require strict ordering, such as schema changes and truncations, introduce barriers that wait for all preceding data to become durable before they are applied.

The source only needs to ship physical WAL, resulting in a load profile similar to a physical standby while allowing changes to reach ClickHouse within a second.

For a detailed overview of architecture, see the architecture documentation. To understand various available tuning settings affecting performance and functionality, see the configuration guide.
Performance benchmarks #
We benchmarked WalShadow against PeerDB, our state-of-the-art Postgres-to-ClickHouse CDC tool powered by logical replication which powers ClickPipes.
For me, this comparison is bittersweet. We’re proud that PeerDB, which we created, serves thousands of customers. WalShadow carries that journey forward by reimagining replication directly from physical WAL and moving us closer to a unified Postgres and ClickHouse stack.
The benchmark replicated a continuous stream of changes from a single table, with Postgres, ClickHouse, and each replication tool running in the same region. We used modest c8i.2xlarge instances with 8 vCPUs each. Performance will vary by workload, but these results offer a useful indication of what WalShadow can offer.
Around 200 ms commit-to-visible latency #

Commit-to-visible latency measures the time from a transaction committing in Postgres to its rows becoming visible in ClickHouse. A native Postgres physical replica established a practical baseline of around 50 ms. WalShadow achieved approximately 200 ms, compared with around 10 seconds for PeerDB, about 50x lower latency in this benchmark.
289,000 rows per second sustained throughput #

The source Postgres instance sustained approximately 290,000 inserted rows per second, establishing the maximum rate the replication pipeline could process. WalShadow replicated 289,000 rows per second, effectively matching the source without becoming the bottleneck. PeerDB sustained approximately 120,000 rows per second, or around 40% of the source rate.
These results show that low latency does not have to come at the cost of throughput: WalShadow keeps data real-time while operating at nearly the full speed of the source.
Conclusion and vision #
At ClickHouse, we have taken several major steps to bring Postgres and ClickHouse closer together: acquisition of PeerDB, launching Postgres CDC in ClickPipes, and introducing ClickHouse Managed Postgres, an enterprise-grade Postgres service built on local NVMe storage for fast OLTP and natively integrated with ClickHouse for fast OLAP.
These efforts share one goal: to give developers a unified data stack that combines Postgres for transactions and ClickHouse for analytics, without added complexity.
WalShadow represents a major milestone toward that vision. By replicating directly from physical WAL into ClickHouse-native blocks, it delivers sub-second analytics, removes much of the operational overhead of logical replication, and supports advanced schema changes that are difficult to handle through conventional logical-decoding pipelines.
Over the coming months, we will work closely with customers to harden WalShadow across real-world workloads. We are already collaborating with an initial group of design partners and are now ready to expand access.
Sign up for the private preview, and we’ll get you access within a day or two.Request access to WalShadowSign up for the private preview of WalShadow on ClickHouse Managed Postgres.WalShadow is one of several initiatives underway to make Postgres and ClickHouse work seamlessly together. Stay tuned for more.Get started with ClickHouse Managed Postgres todayInterested in seeing how ClickHouse Managed Postgres works on your data? Get started with ClickHouse Cloud in minutes and receive $300 in free credits.Sign upGet started todayInterested in seeing how ClickHouse works on your data? Get started with ClickHouse Cloud in minutes and receive $300 in free credits.Sign upShare this postCopy URLSubscribe to our newsletterStay informed on feature releases, product roadmap, support, and cloud offerings!Recent postsView all BlogsProductClickHouse is now available on the dbt platformAditya Chidurala, José Muñoz and Alex Francoeur · Sep 16, 2026ProductIntroducing ClickHouse's new TimeSeries Engine: Your drop-In Prometheus replacementJames Cunningham · Sep 15, 2026ProductReplica-aware routing public betaAmy Chen and Jan Mensch · Sep 15, 2026ProductAI Functions in ClickHouse: Upgrade your SQL to the AI ageAndriy Yakovlev and George Larionov · Sep 11, 2026View all BlogsFollow usProductClickHouse CloudBring Your Own CloudClickHouse Managed PostgresManaged ClickStackClickHouseClickStackAgentic Data StackClickHouse GovernmentClickHouse KeeperClickPipesIntegrationschDBPricingResourcesDocumentationTrust centerTrainingSupportBenchmarksUse casesVideosDemosPresentationsReal-time data warehouseClickHouse for data lakesClickStack for AI SREsEngineering resourcesCompanyBlogOur storyCareersContact usEventsNewsMediaJoin our communityClickHouse CommunityGitHubSlackLinkedInXBlueskyTelegramMeetupComparisonsBigQueryPostgreSQLRedshiftSnowflakeElasticSplunkDatadogOpenSearchPartnersAWSAzureGoogle CloudStay informed on feature releases, product roadmap, support, and cloud offerings!Star us on Github© 2026 ClickHouse, Inc. HQ in the Bay Area, CA and Amsterdam, NL.TrademarkPrivacySecurityLegalCookie policyYour privacy choices

WalShadow is an open-source engine designed to replicate data from PostgreSQL to ClickHouse by directly consuming the physical Write-Ahead Log (WAL) stream, representing a novel approach to achieving sub-second data replication for analytical purposes. This methodology contrasts with traditional Change Data Capture (CDC) systems that rely on logical replication, as WalShadow processes the same physical WAL stream outside the source database, thereby eliminating the need for logical replication slots and significantly reducing the operational overhead associated with them. The system achieves performance benchmarks demonstrating low latency and high throughput, making data immediately available for analytics in ClickHouse.

The core architecture of WalShadow involves processing the physical WAL stream through a parallelized, multi-stage pipeline. First, the system tracks the live schema by filtering catalog WAL records and replaying them into a shadow PostgreSQL instance, ensuring the catalog remains synchronized as the source schema evolves. Second, data changes are decoded in parallel across a pool of Rust-based decoders, avoiding bottlenecks associated with intermediate processing layers. Third, these decoded rows are grouped by table into complete ClickHouse-native blocks, maintaining data fidelity without requiring intermediate format conversions. Finally, a separate pool of inserters concurrently writes these blocks to ClickHouse, allowing the decoding and insertion processes to scale independently. This architecture establishes a direct link from Postgres to ClickHouse, bypassing logical decoding plugins, Kafka, and separate normalization steps. Correctness is maintained by attaching the source WAL position to every row, allowing ClickHouse to retain the latest version of data, although operations requiring strict ordering, such as schema changes, introduce barriers that enforce durability before application.

For operational deployment, WalShadow supports the entire replication lifecycle, encompassing initial data loading, continuous replication, schema evolution, restart recovery, and planned source switchovers. Furthermore, the innovation is extended through integration with ClickHouse Managed Postgres, where WalShadow is implemented to access the physical WAL directly, a feature otherwise inaccessible in most managed services. This integration allows the system to operate natively within the Postgres replication layer, providing a direct path from WAL to ClickHouse blocks.

Performance evaluations against existing CDC tools indicate significant advantages. Benchmarks demonstrated that WalShadow achieved a commit-to-visible latency of approximately 200 milliseconds, significantly lower than the around 50 milliseconds achievable by a native physical replica and about 10 seconds observed with PeerDB, resulting in approximately fifty times lower latency. Sustained throughput was also high, achieving a replication rate of 289,000 rows per second, effectively matching the source Postgres instance's maximum insertion rate while avoiding becoming a bottleneck, whereas conventional logical replication tools sometimes operate at only a fraction of the source rate.

Ultimately, WalShadow represents a step toward a unified data stack by enabling seamless integration between PostgreSQL for transactional integrity and ClickHouse for high-speed analytical processing. By utilizing physical WAL replication, it delivers real-time analytics with minimal latency and reduced operational complexity, supporting advanced schema alterations that are often challenging for conventional logical decoding pipelines. The vision is to provide an efficient, low-overhead path for data movement that supports the demands of modern data architectures.