展示 HN：带有原生 ClickHouse 集成的托管 Postgres
你好，HN，我是来自 ClickHouse 的 Sai 和 Kaushik。今天，我们推出了一项与 ClickHouse 原生集成的 Postgres 托管服务。这项服务是与 Ubicloud（YC W24）共同开发的。
<p>简而言之：基于 NVMe 的 Postgres + 内置的 CDC 到 ClickHouse + pg_clickhouse，让你在运行 ClickHouse 进行分析的同时，保持应用程序以 Postgres 为主。</p>
试用（私人预览）：<a href="https://clickhouse.com/cloud/postgres" rel="nofollow">https://clickhouse.com/cloud/postgres</a>
博客及实时演示：<a href="https://clickhouse.com/blog/postgres-managed-by-clickhouse" rel="nofollow">https://clickhouse.com/blog/postgres-managed-by-clickhouse</a>
<p>问题</p>
在许多快速增长的使用 Postgres 的公司中，随着业务的发展，性能和可扩展性常常成为挑战。这对事务和分析工作负载都是如此。在 OLTP 方面，常见问题包括数据摄取速度变慢（尤其是更新和插入更新）、清理速度变慢、长时间运行的事务导致 WAL 峰值等。在大多数情况下，这些问题源于有限的磁盘 IOPS 和不理想的磁盘延迟。如果不需要配置或限制 IOPS，Postgres 可以做得远比现在更好。
在分析方面，许多限制源于 Postgres 主要是为 OLTP 设计的，缺乏分析数据库随着时间发展而具备的多个特性，例如向量化执行、支持多种数据摄取格式等。我们越来越多地看到 GitLab、Ramp、Cloudflare 等许多公司通过 ClickHouse 来补充 Postgres，以卸载分析工作。这种架构使团队能够采用两个专门构建的开源数据库。
<p>解决方案</p>
在 OLTP 方面，我们认为基于 NVMe 的 Postgres 是合适的选择，可以显著提高性能。NVMe 存储与计算物理上共置，能够实现比网络附加存储更低的磁盘延迟和更高的 IOPS，后者在访问磁盘时需要网络往返。这对磁盘受限的工作负载有利，并能显著（最高可达 10 倍）加快包括更新、插入更新、清理、检查点等操作。我们正在撰写一篇详细的博客，研究在慢 I/O 上 WAL fsync、缓冲区读取和检查点的主导作用，以及在 NVMe 上的显著减少。敬请关注！
在 OLAP 方面，Postgres 服务包括原生的 CDC 到 ClickHouse 和通过 pg_clickhouse 的统一查询能力。目前，CDC 是由 ClickPipes/PeerDB 提供支持，基于逻辑复制。我们正在努力通过支持逻辑复制 v2 来加快和简化这一过程，以便流式传输进行中的事务，推出新的逻辑解码插件以解决现有的逻辑复制限制，朝着亚秒级复制等目标迈进。
每个 Postgres 都配备了 pg_clickhouse 扩展，减少了将 ClickHouse 驱动的分析添加到 Postgres 应用程序所需的工作量。它允许你直接从 Postgres 查询 ClickHouse，使 Postgres 同时支持事务和分析。pg_clickhouse 支持全面的查询下推以进行分析，我们计划持续扩展这一功能（<a href="https://news.ycombinator.com/item?id=46249462">https://news.ycombinator.com/item?id=46249462</a>）。
<p>愿景</p>
总而言之，我们的愿景是提供一个统一的数据栈，将 Postgres 用于事务与 ClickHouse 用于分析结合在一起，为你提供基于开源基础的最佳性能和可扩展性。
<p>开始使用</p>
我们正在积极与用户合作，将他们引入 Postgres 服务。由于这是私人预览，目前是免费的。如果你感兴趣，请在这里注册。<a href="https://clickhouse.com/cloud/postgres" rel="nofollow">https://clickhouse.com/cloud/postgres</a>
我们非常希望听到你对我们论点的反馈，以及任何其他想法，这对我们在构建这一服务时将非常有帮助！
查看原文
Hello HN, this is Sai and Kaushik from ClickHouse. Today we are launching a Postgres managed service that is natively integrated with ClickHouse. It is built together with Ubicloud (YC W24).<p>TL;DR: NVMe-backed Postgres + built-in CDC into ClickHouse + pg_clickhouse so you can keep your app Postgres-first while running analytics in ClickHouse.<p>Try it (private preview): <a href="https://clickhouse.com/cloud/postgres" rel="nofollow">https://clickhouse.com/cloud/postgres</a>
Blog w/ live demo: <a href="https://clickhouse.com/blog/postgres-managed-by-clickhouse" rel="nofollow">https://clickhouse.com/blog/postgres-managed-by-clickhouse</a><p>Problem<p>Across many fast-growing companies using Postgres, performance and scalability commonly emerge as challenges as they grow. This is for both transactional and analytical workloads. On the OLTP side, common issues include slower ingestion (especially updates, upserts), slower vacuums, long-running transactions incurring WAL spikes, among others. In most cases, these problems stem from limited disk IOPS and suboptimal disk latency. Without the need to provision or cap IOPS, Postgres could do far more than it does today.<p>On the analytics side, many limitations stem from the fact that Postgres was designed primarily for OLTP and lacks several features that analytical databases have developed over time, for example vectorized execution, support for a wide variety of ingest formats, etc. We’re increasingly seeing a common pattern where many companies like GitLab, Ramp, Cloudflare etc. complement Postgres with ClickHouse to offload analytics. This architecture enables teams to adopt two purpose-built open-source databases.<p>That said, if you’re running a Postgres based application, adopting ClickHouse isn’t straightforward. You typically end up building a CDC pipeline, handling backfills, and dealing with schema changes and updating your application code to be aware of a second database for analytics.<p>Solution<p>On the OLTP side, we believe that NVMe-based Postgres is the right fit and can drastically improve performance. NVMe storage is physically colocated with compute, enabling significantly lower disk latency and higher IOPS than network-attached storage, which requires a network round trip for disk access. This benefits disk-throttled workloads and can significantly (up to 10x) speed up operations incl. updates, upserts, vacuums, checkpointing, etc. We are working on a detailed blog examining how WAL fsyncs, buffer reads, and checkpoints dominate on slow I/O and are significantly reduced on NVMe. Stay tuned!<p>On the OLAP side, the Postgres service includes native CDC to ClickHouse and unified query capabilities through pg_clickhouse. Today, CDC is powered by ClickPipes/PeerDB under the hood, which is based on logical replication. We are working to make this faster and easier by supporting logical replication v2 for streaming in-progress transactions, a new logical decoding plugin to address existing limitations of logical replication, working toward sub-second replication, and more.<p>Every Postgres comes packaged with the pg_clickhouse extension, which reduces the effort required to add ClickHouse-powered analytics to a Postgres application. It allows you to query ClickHouse directly from Postgres, enabling Postgres for both transactions and analytics. pg_clickhouse supports comprehensive query pushdown for analytics, and we plan to continuously expand this further (<a href="https://news.ycombinator.com/item?id=46249462">https://news.ycombinator.com/item?id=46249462</a>).<p>Vision<p>To sum it up - Our vision is to provide a unified data stack that combines Postgres for transactions with ClickHouse for analytics, giving you best-in-class performance and scalability on an open-source foundation.<p>Get Started<p>We are actively working with users to onboard them to the Postgres service. Since this is a private preview, it is currently free of cost.If you’re interested, please sign up here. <a href="https://clickhouse.com/cloud/postgres" rel="nofollow">https://clickhouse.com/cloud/postgres</a><p>We’d love to hear your feedback on our thesis and anything else that comes to mind, it would be super helpful to us as we build this out!