Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .clang-tidy
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
# clang-tidy for pg_clickhouse
# clang-tidy for pg_chdb
#
# PostgreSQL extensions use palloc/pfree, longjmp-based error handling,
# and C89-ish idioms — checks are tuned accordingly.
Expand Down
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.gitignore export-ignore
.gitattributes export-ignore
.github export-ignore
.gitmodules export-ignore
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@ All notable changes to this project will be documented in this file. It uses the
[Semantic Versioning]: https://semver.org/spec/v2.0.0.html
"Semantic Versioning 2.0.0"

## [v0.1.1] — Unreleased
## [v0.1.1] — 2026-09-06

This release makes binary-only changes. Once installed and loaded, any
existing use of chdb and chdb_hook v0.1 will get the v0.1.1 benefits without
needing to `ALTER EXTENSION UPDATE`.

### ⚡ Improvements

Expand Down Expand Up @@ -41,6 +45,8 @@ All notable changes to this project will be documented in this file. It uses the
document the ISO-8601 format of plain text exports of `timestamp` and
`timestamptz` values, as well as the impact of the `timezone` setting on
exported and imported values.
* Added a [benchmark](dev/benchmark/) and documented its results in the
README.

[v0.1.1]: https://github.com/clickhouse/pg_chdb/compare/v0.1.0...v0.1.1
[#76]: https://github.com/clickhouse/pg_chdb/issues/76
Expand Down
9 changes: 6 additions & 3 deletions META.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@
"name": "chdb",
"abstract": "Execute queries in chDB.",
"description": "Execute queries in chDB.",
"version": "0.1.0",
"version": "0.1.1",
"maintainer": "David E. Wheeler <david.wheeler@clickhouse.com>",
"license": "apache_2_0",
"provides": {
"chdb": {
"abstract": "Execute queries in chDB.",
"file": "chdb.control",
"docfile": "doc/chdb.md",
"version": "0.1.0"
"version": "0.1.1"
},
"chdb_hook": {
"abstract": "Hook into PostgreSQL commands to execute chDB queries.",
"file": "src/hook/hook.c",
"docfile": "doc/chdb_hook.md",
"version": "0.1.0"
"version": "0.1.1"
}
},
"prereqs": {
Expand All @@ -26,6 +26,9 @@
}
}
},
"no_index": {
"directory": ["dev"]
},
"resources": {
"bugtracker": {
"web": "http://github.com/ClickHouse/pg_chdb/issues/"
Expand Down
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -180,5 +180,7 @@ release-notes: CHANGELOG.md
$(EXTENSION)-$(DISTVERSION).zip:
git archive-all -v --prefix "$(EXTENSION)-$(DISTVERSION)/" --force-submodules $(EXTENSION)-$(DISTVERSION).zip

zip: $(EXTENSION)-$(DISTVERSION).zip

# Run make print-VARIABLE_NAME to print VARIABLE_NAME's flavor and value.
print-% : ; $(info $* is $(flavor $*) variable set to "$($*)") @true
54 changes: 50 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,43 @@ CREATE TABLE reviews () WITH (

See the [chdb_hook documentation](doc/chdb_hook.md) for details.

Benchmarking Formats
--------------------

The contents of [dev/benchmark](dev/benchmark/) compare the performance of
[chdb_hook] `COPY` to that of [aws_s3], [pg_duckdb], and [pg_lake] for ca. 1m
rows of [NYC Taxi dataset] in a variety of formats.

<picture>
<source media="(prefers-color-scheme: dark)" srcset="dev/benchmark/taxi-bench-dark.png">
<img alt="NYC Taxi Data Benchmark" src="dev/benchmark/taxi-bench.png">
</picture>

Of the four extensions, [chdb] exhibits the most consistent performance.
[pg_duckdb] and [pg_lake], both backed by [DuckDB], take around 2-3x as long
to import data from CSV, JSON, and Parquet. Only [aws_s3] approaches
[chdb_hook]'s performance, but it supports a much more limited array of data
formats:

| Extension | Compression | Data Formats
| --------- | ------------------------------------ | -------------------------------- |
| aws_s3 | none | Text (TSV), CSV, Postgres Binary |
| pg_lake | gzip, zstd, snappy (Parquet only) | CSV, JSON, Parquet |
| pg_duckdb | gzip, zstd, snappy (Parquet only) | CSV, JSON, Parquet |
| chdb | tzip, zstd, lz4, bz2, snappy, brotli | TSV, CSV, JSON, BSON, Prometheus, Protobuf, Avro, Parquet, Arrow, XML, CapnProto, Markdown, MsgPack, ORC, and [more][formats]! |

Additional benchmarking demonstrates relatively consistent performance
importing the [NYC Taxi dataset] in a variety of formats:

<picture>
<source media="(prefers-color-scheme: dark)" srcset="dev/benchmark/chdb-bench-dark.png">
<img alt="Import Benchmark" src="dev/benchmark/chdb-bench.png">
</picture>

The benchmark uses the [JSONCompact] format for compatibility with the other
extensions; Other JSON formats, such as [JSONCompactEachRow], will more
closely approximate the performance of the other formats.

Architecture
------------

Expand All @@ -114,10 +151,10 @@ A helper that dies triggers an error only in the backend that started it,
leaving other sessions untouched.

> [!IMPORTANT]
> For each query, the helper connects to a new in-memory database to execute
> it. As a consequence, each query currently runs in complete isolation from
> all other queries. Don't create a table and expect to query it in a
> subsequent query.
> For each query, the helper connects to a temporary chDB database on disk to
> execute it. As a consequence, each query currently runs in complete
> isolation from all other queries. Don't create a table and expect to query
> it in a subsequent query.

Dependencies
------------
Expand Down Expand Up @@ -246,3 +283,12 @@ Copyright (c) 2026, ClickHouse
"Postgres Docs: CREATE TABLE"
[lib.chdb.io]: https://lib.chdb.io "curl -sL https://lib.chdb.io | bash"
[`postgresql.conf` parameters]: https://www.postgresql.org/docs/devel/runtime-config-client.html#RUNTIME-CONFIG-CLIENT-OTHER
[chdb_hook]: https://pgxn.org/dist/chdb/doc/chdb_hook.html "chdb_hook Docs on PGXN"
[aws_s3]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.S3Import.html
"Importing data from Amazon S3 into an RDS for PostgreSQL DB instance"
[pg_duckdb]: https://github.com/duckdb/pg_duckdb "DuckDB-powered Postgres for high performance apps & analytics"
[pg_lake]: https://github.com/Snowflake-Labs/pg_lake "pg_lake: Postgres with Iceberg and data lake access"
[formats]: https://clickhouse.com/docs/reference/formats/index
"ClickHouse Docs: Formats for input and output data"
[NYC Taxi dataset]: https://clickhouse.com/docs/get-started/quickstarts/tutorial
"ClickHouse Docs: Advanced tutorial"
15 changes: 12 additions & 3 deletions dev/benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,24 @@ Test configurations:

### NYC Taxi Data

![NYC Taxi Data Benchmark](taxi-bench.png)
<picture>
<source media="(prefers-color-scheme: dark)" srcset="taxi-bench-dark.png">
<img alt="NYC Taxi Data Benchmark" src="taxi-bench.png">
</picture>

### "Logs" Data

!["Logs" Data Benchmark](logs-bench.png)
<picture>
<source media="(prefers-color-scheme: dark)" srcset="logs-bench-dark.png">
<img alt="“Logs” Taxi Data Benchmark" src="logs-bench.png">
</picture>

### chdb Data Formats

![chdb Data Formats](chdb-bench.png)
<picture>
<source media="(prefers-color-scheme: dark)" srcset="chdb-bench-dark.png">
<img alt="Import Benchmark" src="chdb-bench.png">
</picture>

## Data Sets

Expand Down
Binary file added dev/benchmark/chdb-bench-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified dev/benchmark/chdb-bench.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added dev/benchmark/logs-bench-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified dev/benchmark/logs-bench.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added dev/benchmark/taxi-bench-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified dev/benchmark/taxi-bench.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 6 additions & 6 deletions doc/chdb.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
chdb 0.1.0
===========
chdb 0.1.1
==========

## Synopsis

Expand Down Expand Up @@ -153,17 +153,17 @@ Once installed, PostgreSQL tracks two variations of the version:
function.
* The extension version (defined in the control file) includes only the
major and minor versions, visible in the `pg_catalog.pg_extension` table,
the output of the `pg_available_extension_versions()` function, and `\dx
pg_clickhouse`.
the output of the `pg_available_extension_versions()` function, and
`\dx chdb`.

In practice this means that a release that increments the patch version, e.g.
from `v0.1.0` to `v0.1.1`, benefits all databases that have loaded `v0.1` and
do not need to run `ALTER EXTENSION` to benefit from the upgrade.

A release that increments the minor or major versions, on the other hand, will
be accompanied by SQL upgrade scripts, and all existing database that contain
the extension must run `ALTER EXTENSION pg_clickhouse UPDATE` to benefit from
the upgrade.
the extension must run `ALTER EXTENSION chdb UPDATE` to benefit from the
upgrade.

## Authors

Expand Down
8 changes: 4 additions & 4 deletions doc/chdb_hook.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
chdb_hook 0.1.0
chdb_hook 0.1.1
===============

## Synopsis
Expand Down Expand Up @@ -152,7 +152,7 @@ s3://{bucket}/{path}
Or of an object URL:

```
s3://{bucket}.{region}.amazonaws.com/{path}
s3://{bucket}.s3.{region}.amazonaws.com/{path}
```

#### GCS
Expand Down Expand Up @@ -689,11 +689,11 @@ chdb_hook adheres to [Semantic Versioning] for its public releases.
* The minor version increments for backward compatible SQL changes
* The patch version increments for binary-only changes

Once installed, PostgreSQL the version via the the Postgres
Once installed, PostgreSQL the version via the the Postgres 18
[`pg_get_loaded_modules()`] function.

```sql
SELECT version FROM pg_get_loaded_modules() WHERE module_name = 'chdb';
SELECT version FROM pg_get_loaded_modules() WHERE module_name = 'chdb_hook';
```

## Authors
Expand Down
2 changes: 1 addition & 1 deletion src/hook/copy.c
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ make_ch_query(chdbCopyContext* ctx, StringInfo query, char** names, char** value
if (ctx->scheme == s3_scheme) {
if (slash && slash - uri >= strlen(AWS_HOST) &&
!pg_strncasecmp(slash - strlen(AWS_HOST), AWS_HOST, strlen(AWS_HOST))) {
/* s3://{bucket}.{region}.amazonaws.com/{path} */
/* s3://{bucket}.s3.{region}.amazonaws.com/{path} */
PARAM("{url:String}", "url", psprintf("https://%s", uri));
} else {
/* s3://{bucket}/{path} */
Expand Down
Loading