@clickhouse/client 1.18.4 → 1.18.5-head.50d1ed2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/dist/version.js.map +1 -1
- package/package.json +2 -2
- package/skills/clickhouse-js-node-coding/SKILL.md +146 -0
- package/skills/clickhouse-js-node-coding/evals/evals.json +103 -0
- package/skills/clickhouse-js-node-coding/reference/async-insert.md +103 -0
- package/skills/clickhouse-js-node-coding/reference/client-configuration.md +159 -0
- package/skills/clickhouse-js-node-coding/reference/custom-json.md +149 -0
- package/skills/clickhouse-js-node-coding/reference/data-types.md +169 -0
- package/skills/clickhouse-js-node-coding/reference/insert-columns.md +113 -0
- package/skills/clickhouse-js-node-coding/reference/insert-formats.md +145 -0
- package/skills/clickhouse-js-node-coding/reference/insert-values.md +141 -0
- package/skills/clickhouse-js-node-coding/reference/ping.md +120 -0
- package/skills/clickhouse-js-node-coding/reference/query-parameters.md +152 -0
- package/skills/clickhouse-js-node-coding/reference/select-formats.md +111 -0
- package/skills/clickhouse-js-node-coding/reference/sessions.md +152 -0
package/dist/version.d.ts
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
declare const _default: "1.18.
|
|
1
|
+
declare const _default: "1.18.5-head.50d1ed2.1";
|
|
2
2
|
export default _default;
|
package/dist/version.js
CHANGED
package/dist/version.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"version.js","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":";;AAAA,kBAAe,
|
|
1
|
+
{"version":3,"file":"version.js","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":";;AAAA,kBAAe,uBAAuB,CAAA"}
|
package/package.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"name": "@clickhouse/client",
|
|
3
3
|
"description": "Official JS client for ClickHouse DB - Node.js implementation",
|
|
4
4
|
"homepage": "https://clickhouse.com",
|
|
5
|
-
"version": "1.18.
|
|
5
|
+
"version": "1.18.5-head.50d1ed2.1",
|
|
6
6
|
"license": "Apache-2.0",
|
|
7
7
|
"keywords": [
|
|
8
8
|
"clickhouse",
|
|
@@ -40,7 +40,7 @@
|
|
|
40
40
|
"build": "rm -rf dist; tsc"
|
|
41
41
|
},
|
|
42
42
|
"dependencies": {
|
|
43
|
-
"@clickhouse/client-common": "1.18.
|
|
43
|
+
"@clickhouse/client-common": "1.18.5-head.50d1ed2.1"
|
|
44
44
|
},
|
|
45
45
|
"devDependencies": {
|
|
46
46
|
"simdjson": "^0.9.2"
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: clickhouse-js-node-coding
|
|
3
|
+
description: >
|
|
4
|
+
Write idiomatic application code with the ClickHouse Node.js client
|
|
5
|
+
(`@clickhouse/client`). Use this skill whenever a user is *building* against
|
|
6
|
+
the Node.js client — configuring the client, pinging, inserting rows in JSON
|
|
7
|
+
or raw formats, selecting and parsing results, binding query parameters,
|
|
8
|
+
managing sessions and temporary tables, working with data types like
|
|
9
|
+
`Date`/`DateTime`/`Decimal`/`Time`/`Time64`/`Dynamic`/`Variant`/`JSON`, or
|
|
10
|
+
customizing JSON parsing. Trigger on phrases like "how do I insert…", "how
|
|
11
|
+
do I select…", "what format should I use…", "how do I parameterize…", "how
|
|
12
|
+
do I configure the client…". Do NOT use for browser/Web client code, for
|
|
13
|
+
performance/streaming/Parquet questions (see `examples/node/performance/`),
|
|
14
|
+
or for diagnosing errors and unexpected behavior (see
|
|
15
|
+
clickhouse-js-node-troubleshooting).
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
# ClickHouse Node.js Client — Coding
|
|
19
|
+
|
|
20
|
+
Reference: https://clickhouse.com/docs/integrations/javascript
|
|
21
|
+
|
|
22
|
+
> **⚠️ Node.js runtime only.** This skill covers the `@clickhouse/client`
|
|
23
|
+
> package running in a **Node.js runtime** exclusively — including **Next.js
|
|
24
|
+
> Node runtime** API routes, React Server Components, Server Actions, and
|
|
25
|
+
> standard Node.js processes. Do **not** apply this skill to browser client
|
|
26
|
+
> components, Web Workers, **Next.js Edge runtime**, Cloudflare Workers, or
|
|
27
|
+
> any usage of `@clickhouse/client-web`. For browser/edge environments, the
|
|
28
|
+
> correct package is `@clickhouse/client-web`.
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## How to Use This Skill
|
|
33
|
+
|
|
34
|
+
1. **Match the user's intent** to a row in the Task Index below and read the
|
|
35
|
+
corresponding reference file before writing code. After reading it, scan any
|
|
36
|
+
**Answer checklist** in that reference and make sure the final answer covers
|
|
37
|
+
each relevant item; those checklists capture details users usually need but
|
|
38
|
+
are easy to omit in short answers.
|
|
39
|
+
2. **Always import from `@clickhouse/client`** (never `@clickhouse/client-web`)
|
|
40
|
+
and create a single client with `createClient({ url })` or rely on
|
|
41
|
+
supported defaults when appropriate. Close it with `await client.close()`
|
|
42
|
+
during graceful shutdown.
|
|
43
|
+
3. **Prefer `JSONEachRow` for typical row inserts/selects** unless the user
|
|
44
|
+
has already chosen another format or is streaming raw bytes (CSV / TSV /
|
|
45
|
+
Parquet — see `examples/node/performance/`).
|
|
46
|
+
**Note on `clickhouse_settings`:** settings passed to `createClient` are
|
|
47
|
+
defaults for every request; they can be overridden per-call by passing
|
|
48
|
+
`clickhouse_settings` directly to `insert()`, `query()`, or `command()`.
|
|
49
|
+
Always mention this when the user configures settings at the client level.
|
|
50
|
+
4. **Always use `query_params` for user-supplied values** — never template-
|
|
51
|
+
literal-interpolate them into SQL. See `reference/query-parameters.md`.
|
|
52
|
+
5. **Pick the right method for the job:**
|
|
53
|
+
- `client.insert()` — write rows.
|
|
54
|
+
- `client.query()` + `resultSet.json()` / `.text()` / `.stream()` — read
|
|
55
|
+
rows that return data.
|
|
56
|
+
- `client.command()` — DDL and other statements that don't return rows
|
|
57
|
+
(`CREATE`, `DROP`, `TRUNCATE`, `ALTER`, `SET` in a session, etc.).
|
|
58
|
+
- `client.exec()` — when you need the raw response stream of an arbitrary
|
|
59
|
+
statement (rare in coding scenarios).
|
|
60
|
+
- `client.ping()` — health check; returns `{ success, error? }`, never
|
|
61
|
+
throws on connection failure.
|
|
62
|
+
6. **Note version constraints** when relevant. Examples:
|
|
63
|
+
- `pathname` config option: client `>= 1.0.0`.
|
|
64
|
+
- `BigInt` values in `query_params`: client `>= 1.15.0`.
|
|
65
|
+
- `TupleParam` and JS `Map` in `query_params`: client `>= 1.9.0`.
|
|
66
|
+
- Configurable `json.parse` / `json.stringify`: client `>= 1.14.0`.
|
|
67
|
+
- `Time` / `Time64` data types: ClickHouse server `>= 25.6`.
|
|
68
|
+
- `Dynamic` / `Variant` / new `JSON` types: ClickHouse server `>= 24.1` /
|
|
69
|
+
`24.5` / `24.8` (no longer experimental since `25.3`).
|
|
70
|
+
7. **Show a runnable snippet**, not pseudo-code. The examples in
|
|
71
|
+
[`examples/node/coding/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/coding)
|
|
72
|
+
are all self-contained and runnable against the repo's `docker-compose up`
|
|
73
|
+
setup — pattern your snippet after them.
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Task Index
|
|
78
|
+
|
|
79
|
+
Identify the user's task and read the matching reference file.
|
|
80
|
+
|
|
81
|
+
| Task | Triggers / symptoms | Reference file |
|
|
82
|
+
| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------- |
|
|
83
|
+
| **Configure / connect the client** | Building a `createClient` call, URL parameters, `clickhouse_settings`, default format, custom HTTP headers | `reference/client-configuration.md` |
|
|
84
|
+
| **Ping the server** | Health checks, readiness probes, "is ClickHouse up?" | `reference/ping.md` |
|
|
85
|
+
| **Choose an insert format** | "Which format should I use to insert?", JSON vs raw, `JSONEachRow` vs `JSON` vs `JSONObjectEachRow` | `reference/insert-formats.md` |
|
|
86
|
+
| **Insert into a subset of columns / different database** | `insert({ columns })`, excluding columns, ephemeral columns, cross-DB inserts | `reference/insert-columns.md` |
|
|
87
|
+
| **Insert values, expressions, dates, decimals** | `INSERT … VALUES` with SQL functions, `Date`/`DateTime` from JS, `Decimal` precision, `INSERT … SELECT` | `reference/insert-values.md` |
|
|
88
|
+
| **Async inserts (server-side batching)** | `async_insert=1`, fire-and-forget vs wait-for-ack | `reference/async-insert.md` |
|
|
89
|
+
| **Select and parse results** | `JSONEachRow` reads, `JSON` with metadata, picking a select format | `reference/select-formats.md` |
|
|
90
|
+
| **Parameterize queries** | Binding values, special characters / escaping, "SQL injection?", `{name: Type}` syntax | `reference/query-parameters.md` |
|
|
91
|
+
| **Sessions & temporary tables** | `session_id`, `CREATE TEMPORARY TABLE`, per-session `SET` commands | `reference/sessions.md` |
|
|
92
|
+
| **Modern data types** | `Dynamic`, `Variant`, `JSON` (object), `Time`, `Time64` | `reference/data-types.md` |
|
|
93
|
+
| **Custom JSON parse/stringify** | Plug in `JSONBig` / `safe-stable-stringify` / a `BigInt`-aware serializer | `reference/custom-json.md` |
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Conventions used in answers
|
|
98
|
+
|
|
99
|
+
- Always show `import { createClient } from '@clickhouse/client'` (Node, never
|
|
100
|
+
Web). For things that require a runtime API, prefer `node:` built-ins
|
|
101
|
+
(e.g., `import * as crypto from 'node:crypto'`).
|
|
102
|
+
- Always `await client.close()` at the end of self-contained snippets; in
|
|
103
|
+
long-running services, close on graceful shutdown.
|
|
104
|
+
- Prefer top-level `await` in snippets to match the style of
|
|
105
|
+
`examples/node/coding/*.ts`.
|
|
106
|
+
- For inserts, prefer `format: 'JSONEachRow'` and `values: [...]` unless the
|
|
107
|
+
user's scenario requires otherwise.
|
|
108
|
+
- For selects, prefer `await (await client.query({...})).json<RowType>()` for
|
|
109
|
+
small / medium result sets; for streaming, see `examples/node/performance/`.
|
|
110
|
+
- When showing parameter binding, use ClickHouse's native `{name: Type}`
|
|
111
|
+
syntax — never `$1`, `?`, or `:name`.
|
|
112
|
+
- For DDL inside a cluster or behind a load balancer, set
|
|
113
|
+
`clickhouse_settings: { wait_end_of_query: 1 }` on the `command()` call so
|
|
114
|
+
the server only acknowledges after the change is applied. See
|
|
115
|
+
https://clickhouse.com/docs/en/interfaces/http/#response-buffering.
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Out of scope
|
|
120
|
+
|
|
121
|
+
This skill covers day-to-day coding against `@clickhouse/client` (Node).
|
|
122
|
+
The following topics are intentionally **not** covered here:
|
|
123
|
+
|
|
124
|
+
- **Errors, hangs, type mismatches, proxy pathname surprises, log silence,
|
|
125
|
+
socket hang-ups, `ECONNRESET`** → use the
|
|
126
|
+
`clickhouse-js-node-troubleshooting` skill.
|
|
127
|
+
- **Streaming, Parquet, file streams, server-side bulk moves, progress
|
|
128
|
+
streaming, async-insert throughput tuning** — see
|
|
129
|
+
[`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance).
|
|
130
|
+
- **TLS, RBAC / read-only users, deeper SQL-injection guidance** — see
|
|
131
|
+
[`examples/node/security/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/security).
|
|
132
|
+
- **`CREATE TABLE` patterns, deployment-shaped connection strings,
|
|
133
|
+
replication / sharding choices** — see
|
|
134
|
+
[`examples/node/schema-and-deployments/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/schema-and-deployments).
|
|
135
|
+
- **Browser, Web Worker, Next.js Edge, Cloudflare Workers** — use
|
|
136
|
+
`@clickhouse/client-web` and see
|
|
137
|
+
[`examples/web/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/web).
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Still Stuck?
|
|
142
|
+
|
|
143
|
+
- [`examples/node/coding/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/coding) — the runnable corpus this skill is built on.
|
|
144
|
+
- [ClickHouse JS client docs](https://clickhouse.com/docs/integrations/javascript)
|
|
145
|
+
- [ClickHouse supported formats](https://clickhouse.com/docs/interfaces/formats)
|
|
146
|
+
- [ClickHouse data types](https://clickhouse.com/docs/sql-reference/data-types)
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skill_name": "clickhouse-js-node-coding",
|
|
3
|
+
"evals": [
|
|
4
|
+
{
|
|
5
|
+
"id": 0,
|
|
6
|
+
"prompt": "I'm setting up clickhouse client in a Node service. I want to point it at https://my.host:8124, use database 'analytics', user 'bob' / password 'secret', set application name 'my_app', and turn on async_insert without waiting for ack. What's the cleanest way to express that?",
|
|
7
|
+
"expected_output": "A createClient call (Node, not Web) that sets url, username/password (or embeds them in the URL), database, application, and clickhouse_settings: { async_insert: 1, wait_for_async_insert: 0 }. Optionally mentions the equivalent URL-parameter form (ch_async_insert=1&ch_wait_for_async_insert=0) and that URL params override the config object.",
|
|
8
|
+
"files": [],
|
|
9
|
+
"expectations": [
|
|
10
|
+
"Uses createClient from @clickhouse/client (Node, not Web).",
|
|
11
|
+
"Either passes a single URL string with auth + ?ch_async_insert=1&ch_wait_for_async_insert=0, or a config object with database, username, password, application, clickhouse_settings.",
|
|
12
|
+
"Mentions or implies that URL parameters override the config object if both are provided.",
|
|
13
|
+
"Does not suggest using URL parameters in code and instead suggests that the URL should be read from environment variables or a config file.",
|
|
14
|
+
"Does not construct the URL with parameters directly in code neither using string concatenation nor query objects.",
|
|
15
|
+
"Suggests using `await client.close()` during graceful shutdown.",
|
|
16
|
+
"Suggests that settings in `clickhouse_settings` can be overridden per-query by passing them inside the individual `insert()` or `query()` call for finer control."
|
|
17
|
+
]
|
|
18
|
+
},
|
|
19
|
+
{
|
|
20
|
+
"id": 1,
|
|
21
|
+
"prompt": "How do I do a health check against ClickHouse from Node? I want to return 200/503 from an Express endpoint based on whether ClickHouse is reachable.",
|
|
22
|
+
"expected_output": "Use await client.ping(), branch on { success } (no try/catch needed) — return 200 on success and 503 on failure, optionally surfacing the error.",
|
|
23
|
+
"files": [],
|
|
24
|
+
"expectations": [
|
|
25
|
+
"Uses await client.ping() and reads { success, error } directly — does NOT wrap it in try/catch as the only check.",
|
|
26
|
+
"Maps success === true to 200 and success === false to 503.",
|
|
27
|
+
"Suggests lowering request_timeout to make the probe fail fast.",
|
|
28
|
+
"Explains the difference between `client.ping()` (checks connectivity only and ignores credentials by default) and `client.ping({ select: true })` (lightweight query that also checks auth and query processing) and when to use each."
|
|
29
|
+
]
|
|
30
|
+
},
|
|
31
|
+
{
|
|
32
|
+
"id": 2,
|
|
33
|
+
"prompt": "I have an array of about 10k plain JS objects I want to insert into a MergeTree table. What's the right format and call?",
|
|
34
|
+
"expected_output": "client.insert with format: 'JSONEachRow' and values: <array>. No streaming / Parquet needed for this size.",
|
|
35
|
+
"files": [],
|
|
36
|
+
"expectations": [
|
|
37
|
+
"Uses client.insert({ table, values, format: 'JSONEachRow' }).",
|
|
38
|
+
"Notes that the array can be passed directly to values — no need to stringify or stream for a few thousand rows.",
|
|
39
|
+
"Does NOT recommend a streaming/Parquet flow for this size.",
|
|
40
|
+
"Mentions `JSONCompact*` formats as an alternative for bigger payloads"
|
|
41
|
+
]
|
|
42
|
+
},
|
|
43
|
+
{
|
|
44
|
+
"id": 3,
|
|
45
|
+
"prompt": "My table has columns (id, name, created_at, internal_hash) but the rows I have only contain id and name. How do I insert just those two columns?",
|
|
46
|
+
"expected_output": "Use the columns option on client.insert: either columns: ['id', 'name'] (allowlist) or columns: { except: ['created_at', 'internal_hash'] } (excludelist). Omitted columns get their declared defaults.",
|
|
47
|
+
"files": [],
|
|
48
|
+
"expectations": [
|
|
49
|
+
"Uses client.insert with the columns: ['id', 'name'] option.",
|
|
50
|
+
"Mentions the alternative columns: { except: [...] } form.",
|
|
51
|
+
"Notes that omitted columns will receive their server-side defaults."
|
|
52
|
+
]
|
|
53
|
+
},
|
|
54
|
+
{
|
|
55
|
+
"id": 4,
|
|
56
|
+
"prompt": "I want to call: SELECT * FROM users WHERE country = '<user input>' AND signup_date > '<user input>'. How should I pass those values from JS?",
|
|
57
|
+
"expected_output": "Use parameterized queries with the ClickHouse {name: Type} syntax and query_params: { country: ..., signup_date: ... }. Explicitly warn against template-literal interpolation (SQL injection).",
|
|
58
|
+
"files": [],
|
|
59
|
+
"expectations": [
|
|
60
|
+
"Uses {name: Type} parameter syntax (e.g., {country: String}, {signup_date: Date}) and query_params.",
|
|
61
|
+
"Explicitly warns against template-literal interpolation as a SQL injection risk.",
|
|
62
|
+
"Does NOT suggest $1/?/:name placeholders."
|
|
63
|
+
]
|
|
64
|
+
},
|
|
65
|
+
{
|
|
66
|
+
"id": 5,
|
|
67
|
+
"prompt": "I'm doing CREATE TEMPORARY TABLE and then SELECT from it in a follow-up call. They keep disappearing between calls. What am I missing?",
|
|
68
|
+
"expected_output": "Temporary tables are scoped to a session — set a stable session_id (e.g., crypto.randomUUID()) on the client (or per-call) so consecutive requests share server-side state. Also flag the load-balancer/Cloud caveat (replica-aware routing).",
|
|
69
|
+
"files": [],
|
|
70
|
+
"expectations": [
|
|
71
|
+
"Explains that temporary tables are scoped to a session and require a stable session_id across calls.",
|
|
72
|
+
"Shows setting session_id either on createClient or via per-call session_id.",
|
|
73
|
+
"Mentions the load-balancer / ClickHouse Cloud caveat (sessions are pinned to a node; recommend replica-aware routing or a single-node connection).",
|
|
74
|
+
"Explicitly explains that parallel calls with the same session_id will result in an error as ClickHouse does not allow concurrent queries within the same session_id.",
|
|
75
|
+
"Explicitly advises against using session_id in client configuration for a global / module static client",
|
|
76
|
+
"When session_id is used as a client option it should suggest configuring the maximum number of connections to 1 to minimize concurrency issues at the client level."
|
|
77
|
+
]
|
|
78
|
+
},
|
|
79
|
+
{
|
|
80
|
+
"id": 6,
|
|
81
|
+
"prompt": "I'm running 25.x ClickHouse. I want to store a JSON object per row and read it back as a real JS object, not a JSON string. How do I do that with the Node client?",
|
|
82
|
+
"expected_output": "Use the new JSON column type (>= 24.8, no longer experimental since 25.3). CREATE TABLE with a JSON column, insert with format: 'JSONEachRow' passing JS objects, select with JSONEachRow — values come back as parsed JS objects with no manual JSON.parse.",
|
|
83
|
+
"files": [],
|
|
84
|
+
"expectations": [
|
|
85
|
+
"Uses the JSON column type and format: 'JSONEachRow' for both insert and select.",
|
|
86
|
+
"Inserts a real JS object as the column value (no JSON.stringify) and shows it returns as a parsed object.",
|
|
87
|
+
"Mentions the relevant ClickHouse server version (>= 24.8 introduced; non-experimental since 25.3) and, if needed on older servers, allow_experimental_json_type."
|
|
88
|
+
]
|
|
89
|
+
},
|
|
90
|
+
{
|
|
91
|
+
"id": 7,
|
|
92
|
+
"prompt": "Our IDs are UInt64 and we don't want them coming back as strings or losing precision.",
|
|
93
|
+
"expected_output": "Yes — pass a custom { parse, stringify } via the json config option (>= 1.14.0). Show wiring up json-bigint (or similar) so 64-bit integers are parsed as BigInt. Mention output_format_json_quote_64bit_integers: 0 so the server emits unquoted ints. Note that switching to native Number would lose precision and is the wrong fix.",
|
|
94
|
+
"files": [],
|
|
95
|
+
"expectations": [
|
|
96
|
+
"Shows passing custom { parse, stringify } via the json config option on createClient.",
|
|
97
|
+
"Notes the >= 1.14.0 client requirement for the json option.",
|
|
98
|
+
"Mentions output_format_json_quote_64bit_integers: 0 (default since 25.8) so 64-bit integers come back unquoted and parseable as BigInt.",
|
|
99
|
+
"Warns that switching to native Number would lose precision and is the wrong fix."
|
|
100
|
+
]
|
|
101
|
+
}
|
|
102
|
+
]
|
|
103
|
+
}
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# Async Inserts
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all client versions; the relevant settings are server-side.
|
|
4
|
+
> See https://clickhouse.com/docs/en/optimize/asynchronous-inserts.
|
|
5
|
+
|
|
6
|
+
Backing example:
|
|
7
|
+
[`examples/node/coding/async_insert.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/async_insert.ts).
|
|
8
|
+
|
|
9
|
+
> **When to use async inserts:** when many small inserts arrive concurrently
|
|
10
|
+
> (e.g., one per HTTP request) and you don't want to maintain a client-side
|
|
11
|
+
> batching layer. ClickHouse will batch them server-side. This is also the
|
|
12
|
+
> recommended ingestion pattern for **ClickHouse Cloud**.
|
|
13
|
+
|
|
14
|
+
> **When _not_ to use async inserts:** when you already build large batches
|
|
15
|
+
> client-side (e.g., from a stream). Plain inserts are simpler and lower
|
|
16
|
+
> latency. For raw throughput tuning of large async-insert workloads, see
|
|
17
|
+
> [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance).
|
|
18
|
+
|
|
19
|
+
## Setup
|
|
20
|
+
|
|
21
|
+
Enable on the client (or per-request) via `clickhouse_settings`:
|
|
22
|
+
|
|
23
|
+
```ts
|
|
24
|
+
import { createClient, ClickHouseError } from '@clickhouse/client'
|
|
25
|
+
|
|
26
|
+
const client = createClient({
|
|
27
|
+
url: process.env.CLICKHOUSE_URL,
|
|
28
|
+
password: process.env.CLICKHOUSE_PASSWORD,
|
|
29
|
+
max_open_connections: 10,
|
|
30
|
+
clickhouse_settings: {
|
|
31
|
+
async_insert: 1,
|
|
32
|
+
wait_for_async_insert: 1, // wait for ack from server
|
|
33
|
+
async_insert_max_data_size: '1000000',
|
|
34
|
+
async_insert_busy_timeout_ms: 1000,
|
|
35
|
+
},
|
|
36
|
+
})
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## Concurrent small inserts
|
|
40
|
+
|
|
41
|
+
Each call still uses the client's normal `insert()` API — the server merges
|
|
42
|
+
the batches.
|
|
43
|
+
|
|
44
|
+
```ts
|
|
45
|
+
const promises = [...new Array(10)].map(async () => {
|
|
46
|
+
const values = [...new Array(1000).keys()].map(() => ({
|
|
47
|
+
id: Math.floor(Math.random() * 100_000) + 1,
|
|
48
|
+
data: Math.random().toString(36).slice(2),
|
|
49
|
+
}))
|
|
50
|
+
|
|
51
|
+
await client
|
|
52
|
+
.insert({ table: 'async_insert_example', values, format: 'JSONEachRow' })
|
|
53
|
+
.catch((err) => {
|
|
54
|
+
if (err instanceof ClickHouseError) {
|
|
55
|
+
// err.code matches a row in system.errors
|
|
56
|
+
console.error(`ClickHouse error ${err.code}:`, err)
|
|
57
|
+
return
|
|
58
|
+
}
|
|
59
|
+
console.error('Insert failed:', err)
|
|
60
|
+
})
|
|
61
|
+
})
|
|
62
|
+
|
|
63
|
+
await Promise.all(promises)
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## `wait_for_async_insert` — fire-and-forget vs ack
|
|
67
|
+
|
|
68
|
+
| `wait_for_async_insert` | Promise resolves when… | Trade-off |
|
|
69
|
+
| ----------------------- | ------------------------------------------------- | ------------------------------------------------------------------- |
|
|
70
|
+
| `1` (default) | Server has flushed the batch to the table | Slower per call; insert errors surface to the client |
|
|
71
|
+
| `0` | Server accepted the row into its in-memory buffer | Faster; flush errors won't surface — only validation/parsing errors |
|
|
72
|
+
|
|
73
|
+
With `wait_for_async_insert: 1`, expect each insert call to take roughly
|
|
74
|
+
`async_insert_busy_timeout_ms` to resolve when traffic is light, because the
|
|
75
|
+
server waits for more rows or for the timer to fire before flushing.
|
|
76
|
+
|
|
77
|
+
## Combining DDL with async inserts
|
|
78
|
+
|
|
79
|
+
When creating tables in scripts that immediately insert, ack the DDL with
|
|
80
|
+
`wait_end_of_query: 1` so the table is ready before the first insert:
|
|
81
|
+
|
|
82
|
+
```ts
|
|
83
|
+
await client.command({
|
|
84
|
+
query: `
|
|
85
|
+
CREATE OR REPLACE TABLE async_insert_example (id Int32, data String)
|
|
86
|
+
ENGINE MergeTree ORDER BY id
|
|
87
|
+
`,
|
|
88
|
+
clickhouse_settings: { wait_end_of_query: 1 },
|
|
89
|
+
})
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## Common pitfalls
|
|
93
|
+
|
|
94
|
+
- **Setting `async_insert` per call but expecting client-side batching.**
|
|
95
|
+
The client still issues each `insert()` as a separate HTTP request — the
|
|
96
|
+
batching happens on the server.
|
|
97
|
+
- **Confusing `wait_for_async_insert` (async-insert ack) with
|
|
98
|
+
`wait_end_of_query` (DDL ack).** They are unrelated.
|
|
99
|
+
- **Treating a resolved insert under `wait_for_async_insert: 0` as
|
|
100
|
+
durably written.** It only means the server accepted the bytes; flush
|
|
101
|
+
failures will not surface to the client.
|
|
102
|
+
- **Not handling `ClickHouseError`.** It exposes `err.code`, which maps to
|
|
103
|
+
rows in the `system.errors` table — use it to decide whether to retry.
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# Client Configuration
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions, with these notable additions:
|
|
4
|
+
>
|
|
5
|
+
> - `pathname` config option: client `>= 1.0.0`.
|
|
6
|
+
> - `clickhouse_setting_*` / `ch_*` URL parameters: client `>= 1.0.0`.
|
|
7
|
+
> - `keep_alive.idle_socket_ttl` (Node-only): client `>= 1.0.0`.
|
|
8
|
+
|
|
9
|
+
Backing examples:
|
|
10
|
+
[`examples/node/coding/url_configuration.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/url_configuration.ts),
|
|
11
|
+
[`examples/node/coding/clickhouse_settings.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/clickhouse_settings.ts),
|
|
12
|
+
[`examples/node/coding/default_format_setting.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/default_format_setting.ts).
|
|
13
|
+
|
|
14
|
+
## Answer checklist
|
|
15
|
+
|
|
16
|
+
When answering configuration questions, include the relevant points:
|
|
17
|
+
|
|
18
|
+
- Show `createClient` from `@clickhouse/client` with explicit fields when the
|
|
19
|
+
user is writing code; this is easier to read and review than encoding
|
|
20
|
+
everything into a URL string.
|
|
21
|
+
- When mentioning the URL form for environment variables / DSNs: show a **Bash**
|
|
22
|
+
`export` with the literal URL value, and `createClient({ url: process.env.CLICKHOUSE_URL })`
|
|
23
|
+
in the Node code. **Never construct a URL in application code** — no string
|
|
24
|
+
concatenation, no template literals, no query-string builders.
|
|
25
|
+
- If URL parameters and object fields both set the same option, URL parameters
|
|
26
|
+
override the rest of the configuration object.
|
|
27
|
+
- If `clickhouse_settings` appear on `createClient`, explain that they are
|
|
28
|
+
defaults for every request and can be overridden on individual `query()`,
|
|
29
|
+
`insert()`, `command()`, or `exec()` calls.
|
|
30
|
+
- Remind long-running services to close the client during graceful shutdown.
|
|
31
|
+
- The `application` field sets the name that appears in `system.query_log`.
|
|
32
|
+
Do **not** mention any specific HTTP header name — the client handles header
|
|
33
|
+
mapping internally and the header names are an implementation detail.
|
|
34
|
+
|
|
35
|
+
## Minimal client
|
|
36
|
+
|
|
37
|
+
```ts
|
|
38
|
+
import { createClient } from '@clickhouse/client'
|
|
39
|
+
|
|
40
|
+
const client = createClient({
|
|
41
|
+
url: process.env.CLICKHOUSE_URL, // defaults to 'http://localhost:8123'
|
|
42
|
+
username: process.env.CLICKHOUSE_USER, // defaults to 'default'
|
|
43
|
+
password: process.env.CLICKHOUSE_PASSWORD, // defaults to ''
|
|
44
|
+
database: 'analytics', // defaults to 'default'
|
|
45
|
+
})
|
|
46
|
+
// ... your queries ...
|
|
47
|
+
await client.close()
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
`url` accepts a string or a `URL` object. The accepted string format is:
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
http[s]://[username:password@]hostname:port[/database][?param1=value1¶m2=value2]
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Configuration via URL parameters
|
|
57
|
+
|
|
58
|
+
A fixed allowlist of config fields can be set as URL query parameters
|
|
59
|
+
(plus any key prefixed with `clickhouse_setting_` / `ch_` / `http_header_`).
|
|
60
|
+
**Supported URL parameters override the corresponding values in the rest of
|
|
61
|
+
the configuration object** — when they do, the client logs a warning.
|
|
62
|
+
Unknown URL parameters cause `createClient` to throw
|
|
63
|
+
`Unknown URL parameters: ...`
|
|
64
|
+
(see [`packages/client-common/src/config.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/packages/client-common/src/config.ts) for the shared allowlist, and [`packages/client-node/src/config.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/packages/client-node/src/config.ts) for Node-specific URL parameters).
|
|
65
|
+
|
|
66
|
+
Supported non-prefixed keys parsed by `client-common`: `application`,
|
|
67
|
+
`session_id`, `pathname`, `access_token`, `request_timeout`,
|
|
68
|
+
`max_open_connections`, `compression_request`, `compression_response`,
|
|
69
|
+
`log_level`, `keep_alive_enabled`. Additionally, Node supports
|
|
70
|
+
`keep_alive_idle_socket_ttl` via the Node-specific config implementation.
|
|
71
|
+
Anything else must be passed via the config object on `createClient`.
|
|
72
|
+
|
|
73
|
+
Prefer explicit object fields in application code. Use the URL form when the
|
|
74
|
+
application receives one connection string from an environment variable, secret
|
|
75
|
+
manager, or config file. The URL value belongs in the environment, not in the
|
|
76
|
+
source code — show it as a shell export and read it in Node:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
# In your shell environment / deployment config (e.g. .env, Kubernetes secret):
|
|
80
|
+
export CLICKHOUSE_URL='https://bob:secret@my.host:8124/analytics?application=my_analytics_app&ch_async_insert=1&ch_wait_for_async_insert=0'
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
```ts
|
|
84
|
+
// In your Node.js code — no URL construction needed:
|
|
85
|
+
const client = createClient({ url: process.env.CLICKHOUSE_URL })
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
The same connection can also be expressed as an explicit config object (useful when you want to document each field individually):
|
|
89
|
+
|
|
90
|
+
```ts
|
|
91
|
+
import { createClient } from '@clickhouse/client'
|
|
92
|
+
|
|
93
|
+
createClient({
|
|
94
|
+
url: 'https://my.host:8124',
|
|
95
|
+
username: 'bob',
|
|
96
|
+
password: 'secret',
|
|
97
|
+
database: 'analytics',
|
|
98
|
+
application: 'my_analytics_app',
|
|
99
|
+
clickhouse_settings: { async_insert: 1, wait_for_async_insert: 0 },
|
|
100
|
+
})
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Per-client vs per-request `clickhouse_settings` ⭐
|
|
104
|
+
|
|
105
|
+
> **Always mention this when discussing `clickhouse_settings`:** settings set
|
|
106
|
+
> on `createClient` are defaults; any individual call can override them.
|
|
107
|
+
|
|
108
|
+
Settings on `createClient` apply to every request. Settings on a single
|
|
109
|
+
operation (`query`, `insert`, `command`, `exec`) override the client defaults
|
|
110
|
+
for **that call only**.
|
|
111
|
+
|
|
112
|
+
```ts
|
|
113
|
+
const client = createClient({
|
|
114
|
+
clickhouse_settings: {
|
|
115
|
+
date_time_input_format: 'best_effort', // applied to every request
|
|
116
|
+
},
|
|
117
|
+
})
|
|
118
|
+
|
|
119
|
+
const rows = await client.query({
|
|
120
|
+
query: 'SELECT number FROM system.numbers LIMIT 2',
|
|
121
|
+
format: 'JSONEachRow',
|
|
122
|
+
clickhouse_settings: {
|
|
123
|
+
output_format_json_quote_64bit_integers: 1, // overrides client default for this call
|
|
124
|
+
},
|
|
125
|
+
})
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
## `default_format` for `exec()`
|
|
129
|
+
|
|
130
|
+
`client.exec()` runs an arbitrary statement and returns a stream. If your
|
|
131
|
+
query has no trailing `FORMAT …` clause, set `default_format` so the server
|
|
132
|
+
knows what to send back, then wrap the response in a `ResultSet`:
|
|
133
|
+
|
|
134
|
+
```ts
|
|
135
|
+
import { createClient, ResultSet } from '@clickhouse/client'
|
|
136
|
+
|
|
137
|
+
const client = createClient()
|
|
138
|
+
const format = 'JSONCompactEachRowWithNamesAndTypes'
|
|
139
|
+
const { stream, query_id } = await client.exec({
|
|
140
|
+
query: 'SELECT database, name, engine FROM system.tables LIMIT 5',
|
|
141
|
+
clickhouse_settings: { default_format: format },
|
|
142
|
+
})
|
|
143
|
+
const rs = new ResultSet(stream, format, query_id)
|
|
144
|
+
console.log(await rs.json())
|
|
145
|
+
await client.close()
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
For ordinary `SELECT`s prefer `client.query({ format })` — `default_format` is
|
|
149
|
+
only needed for raw `exec()`.
|
|
150
|
+
|
|
151
|
+
## Common pitfalls
|
|
152
|
+
|
|
153
|
+
- **Don't put a path in `url` and expect it to be the database name when
|
|
154
|
+
you're behind a proxy.** Use `pathname` for the proxy path and `database`
|
|
155
|
+
for the DB. (Symptom: "wrong database selected.") See the
|
|
156
|
+
troubleshooting skill for diagnosis.
|
|
157
|
+
- **Don't create a client per request.** `createClient` opens a connection
|
|
158
|
+
pool; share one client across the process and `close()` on shutdown.
|
|
159
|
+
- **`max_open_connections` must be `>= 1`** when set explicitly.
|