@clickhouse/client 1.18.4 → 1.18.5-head.50d1ed2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/dist/version.js.map +1 -1
- package/package.json +2 -2
- package/skills/clickhouse-js-node-coding/SKILL.md +146 -0
- package/skills/clickhouse-js-node-coding/evals/evals.json +103 -0
- package/skills/clickhouse-js-node-coding/reference/async-insert.md +103 -0
- package/skills/clickhouse-js-node-coding/reference/client-configuration.md +159 -0
- package/skills/clickhouse-js-node-coding/reference/custom-json.md +149 -0
- package/skills/clickhouse-js-node-coding/reference/data-types.md +169 -0
- package/skills/clickhouse-js-node-coding/reference/insert-columns.md +113 -0
- package/skills/clickhouse-js-node-coding/reference/insert-formats.md +145 -0
- package/skills/clickhouse-js-node-coding/reference/insert-values.md +141 -0
- package/skills/clickhouse-js-node-coding/reference/ping.md +120 -0
- package/skills/clickhouse-js-node-coding/reference/query-parameters.md +152 -0
- package/skills/clickhouse-js-node-coding/reference/select-formats.md +111 -0
- package/skills/clickhouse-js-node-coding/reference/sessions.md +152 -0
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
# Insert Values, SQL Expressions, Dates, Decimals
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `wait_end_of_query: 1` is a server-side
|
|
4
|
+
> setting available on every supported ClickHouse version.
|
|
5
|
+
|
|
6
|
+
Backing examples:
|
|
7
|
+
[`examples/node/coding/insert_from_select.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_from_select.ts),
|
|
8
|
+
[`examples/node/coding/insert_values_and_functions.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_values_and_functions.ts),
|
|
9
|
+
[`examples/node/coding/insert_js_dates.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_js_dates.ts),
|
|
10
|
+
[`examples/node/coding/insert_decimals.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_decimals.ts).
|
|
11
|
+
|
|
12
|
+
## `INSERT … SELECT` (no values payload)
|
|
13
|
+
|
|
14
|
+
When the data already lives in ClickHouse, use `client.command()` with a raw
|
|
15
|
+
`INSERT … SELECT`:
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
await client.command({
|
|
19
|
+
query: `
|
|
20
|
+
INSERT INTO target
|
|
21
|
+
SELECT '42', quantilesBFloat16State(0.5)(arrayJoin([toFloat32(10), toFloat32(20)]))
|
|
22
|
+
`,
|
|
23
|
+
})
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Use `command()` (not `insert()`) — there is no row payload to send.
|
|
27
|
+
|
|
28
|
+
## `INSERT … VALUES` with SQL functions
|
|
29
|
+
|
|
30
|
+
When you need `unhex(...)`, `toUUID(...)`, `now()`, or any other SQL
|
|
31
|
+
function around a value, keep the SQL shape static and pass values with
|
|
32
|
+
ClickHouse `{name: Type}` parameters. Run it via `command()` and set
|
|
33
|
+
`wait_end_of_query: 1` for safety in clustered setups.
|
|
34
|
+
|
|
35
|
+
```ts
|
|
36
|
+
await client.command({
|
|
37
|
+
query: `
|
|
38
|
+
INSERT INTO events (id, timestamp, email, name)
|
|
39
|
+
VALUES (
|
|
40
|
+
unhex({id: String}),
|
|
41
|
+
{timestamp: DateTime},
|
|
42
|
+
{email: String},
|
|
43
|
+
{name: Nullable(String)}
|
|
44
|
+
)
|
|
45
|
+
`,
|
|
46
|
+
query_params: {
|
|
47
|
+
id: '00112233445566778899aabbccddeeff',
|
|
48
|
+
timestamp: '2026-05-06 12:34:56',
|
|
49
|
+
email: 'alice@example.com',
|
|
50
|
+
name: 'Alice',
|
|
51
|
+
},
|
|
52
|
+
clickhouse_settings: { wait_end_of_query: 1 },
|
|
53
|
+
})
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Do not build `VALUES` rows with string interpolation or manual escaping. If
|
|
57
|
+
you need to insert many ordinary JS rows, prefer `client.insert()` with
|
|
58
|
+
`format: 'JSONEachRow'`; use this `command()` pattern when the SQL itself needs
|
|
59
|
+
functions or expressions around the values.
|
|
60
|
+
|
|
61
|
+
## Inserting JS `Date` objects
|
|
62
|
+
|
|
63
|
+
JS `Date` objects work for `DateTime` and `DateTime64` columns once the
|
|
64
|
+
server is set to accept ISO-8601 strings. Either set
|
|
65
|
+
`date_time_input_format: 'best_effort'` per request, on the client, or
|
|
66
|
+
session-wide.
|
|
67
|
+
|
|
68
|
+
```ts
|
|
69
|
+
await client.insert({
|
|
70
|
+
table: 'events',
|
|
71
|
+
format: 'JSONEachRow',
|
|
72
|
+
values: [{ id: '42', dt: new Date() }],
|
|
73
|
+
clickhouse_settings: {
|
|
74
|
+
date_time_input_format: 'best_effort',
|
|
75
|
+
},
|
|
76
|
+
})
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
> JS `Date` objects do **not** work for the `Date` type (date-only) — pass
|
|
80
|
+
> `'YYYY-MM-DD'` strings for that.
|
|
81
|
+
|
|
82
|
+
## Inserting `Decimal*` values
|
|
83
|
+
|
|
84
|
+
Decimals must be passed as **strings** in JSON formats to avoid precision
|
|
85
|
+
loss in JavaScript:
|
|
86
|
+
|
|
87
|
+
```ts
|
|
88
|
+
await client.command({
|
|
89
|
+
query: `
|
|
90
|
+
CREATE OR REPLACE TABLE prices (
|
|
91
|
+
id UInt32,
|
|
92
|
+
dec32 Decimal(9, 2),
|
|
93
|
+
dec64 Decimal(18, 3),
|
|
94
|
+
dec128 Decimal(38, 10),
|
|
95
|
+
dec256 Decimal(76, 20)
|
|
96
|
+
)
|
|
97
|
+
ENGINE MergeTree ORDER BY id
|
|
98
|
+
`,
|
|
99
|
+
})
|
|
100
|
+
|
|
101
|
+
await client.insert({
|
|
102
|
+
table: 'prices',
|
|
103
|
+
format: 'JSONEachRow',
|
|
104
|
+
values: [
|
|
105
|
+
{
|
|
106
|
+
id: 1,
|
|
107
|
+
dec32: '1234567.89',
|
|
108
|
+
dec64: '123456789123456.789',
|
|
109
|
+
dec128: '1234567891234567891234567891.1234567891',
|
|
110
|
+
dec256:
|
|
111
|
+
'12345678912345678912345678911234567891234567891234567891.12345678911234567891',
|
|
112
|
+
},
|
|
113
|
+
],
|
|
114
|
+
})
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
When reading them back, cast to string in the SELECT to avoid the same
|
|
118
|
+
precision loss:
|
|
119
|
+
|
|
120
|
+
```ts
|
|
121
|
+
const rs = await client.query({
|
|
122
|
+
query: `
|
|
123
|
+
SELECT toString(dec64) AS decimal64,
|
|
124
|
+
toString(dec128) AS decimal128
|
|
125
|
+
FROM prices
|
|
126
|
+
`,
|
|
127
|
+
format: 'JSONEachRow',
|
|
128
|
+
})
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
## Common pitfalls
|
|
132
|
+
|
|
133
|
+
- **Passing decimals as JS `number`s.** Anything beyond `Number.MAX_SAFE_INTEGER`
|
|
134
|
+
silently loses precision before it ever reaches the server.
|
|
135
|
+
- **Using `client.insert()` for `INSERT … SELECT`.** There's nothing to
|
|
136
|
+
upload — use `client.command()` with the full SQL.
|
|
137
|
+
- **Forgetting `date_time_input_format: 'best_effort'`** when inserting
|
|
138
|
+
`Date` objects (or ISO strings). The default input format does not accept
|
|
139
|
+
ISO-8601 with the `T`/`Z` separators.
|
|
140
|
+
- **Hand-building `VALUES` with user input.** Always parameterize user data;
|
|
141
|
+
see `reference/query-parameters.md`.
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
# Ping the Server
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `ping()` returns a discriminated
|
|
4
|
+
> `PingResult = { success: true } | { success: false, error: Error }` —
|
|
5
|
+
> it does **not** throw on connection failures.
|
|
6
|
+
|
|
7
|
+
Backing examples:
|
|
8
|
+
[`examples/node/coding/ping_existing_host.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/ping_existing_host.ts),
|
|
9
|
+
[`examples/node/coding/ping_non_existing_host.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/ping_non_existing_host.ts).
|
|
10
|
+
|
|
11
|
+
## Successful ping
|
|
12
|
+
|
|
13
|
+
```ts
|
|
14
|
+
import { createClient } from '@clickhouse/client'
|
|
15
|
+
|
|
16
|
+
const client = createClient({
|
|
17
|
+
url: process.env.CLICKHOUSE_URL,
|
|
18
|
+
password: process.env.CLICKHOUSE_PASSWORD,
|
|
19
|
+
})
|
|
20
|
+
|
|
21
|
+
const pingResult = await client.ping()
|
|
22
|
+
if (pingResult.success) {
|
|
23
|
+
console.info('ClickHouse is reachable')
|
|
24
|
+
} else {
|
|
25
|
+
console.error('Ping failed:', pingResult.error)
|
|
26
|
+
}
|
|
27
|
+
await client.close()
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Use `ping()` to:
|
|
31
|
+
|
|
32
|
+
- Probe ClickHouse at application startup.
|
|
33
|
+
- Wake up a ClickHouse Cloud instance that may be idling (a ping is enough to
|
|
34
|
+
bring it out of sleep).
|
|
35
|
+
- Implement a `/healthz` / readiness endpoint.
|
|
36
|
+
|
|
37
|
+
## Failure: host unreachable
|
|
38
|
+
|
|
39
|
+
`ping()` does **not** throw — it resolves with
|
|
40
|
+
`{ success: false, error: Error }`, so you can branch without `try/catch`:
|
|
41
|
+
|
|
42
|
+
```ts
|
|
43
|
+
import type { PingResult } from '@clickhouse/client'
|
|
44
|
+
import { createClient } from '@clickhouse/client'
|
|
45
|
+
|
|
46
|
+
const client = createClient({
|
|
47
|
+
url: 'http://localhost:8100', // non-existing host
|
|
48
|
+
request_timeout: 50, // keep failure fast
|
|
49
|
+
})
|
|
50
|
+
|
|
51
|
+
const pingResult = await client.ping()
|
|
52
|
+
if (hasConnectionRefusedError(pingResult)) {
|
|
53
|
+
console.info('Connection refused, as expected')
|
|
54
|
+
} else {
|
|
55
|
+
console.error('Ping expected ECONNREFUSED, got:', pingResult)
|
|
56
|
+
}
|
|
57
|
+
await client.close()
|
|
58
|
+
|
|
59
|
+
function hasConnectionRefusedError(
|
|
60
|
+
pingResult: PingResult,
|
|
61
|
+
): pingResult is PingResult & { error: { code: 'ECONNREFUSED' } } {
|
|
62
|
+
return (
|
|
63
|
+
!pingResult.success &&
|
|
64
|
+
'code' in pingResult.error &&
|
|
65
|
+
pingResult.error.code === 'ECONNREFUSED'
|
|
66
|
+
)
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Mapping to an HTTP health endpoint
|
|
71
|
+
|
|
72
|
+
```ts
|
|
73
|
+
app.get('/healthz', async (_req, res) => {
|
|
74
|
+
const r = await client.ping()
|
|
75
|
+
if (r.success) {
|
|
76
|
+
res.status(200).json({ ok: true })
|
|
77
|
+
} else {
|
|
78
|
+
res.status(503).json({ ok: false, error: String(r.error) })
|
|
79
|
+
}
|
|
80
|
+
})
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
## `ping()` vs `ping({ select: true })`
|
|
84
|
+
|
|
85
|
+
The default `ping()` hits ClickHouse's `/ping` HTTP endpoint — it verifies
|
|
86
|
+
network connectivity but **does not check credentials or query processing**.
|
|
87
|
+
A server that is reachable but has a bad password (or a broken query
|
|
88
|
+
pipeline) will still return `{ success: true }` from a plain `ping()`.
|
|
89
|
+
|
|
90
|
+
Pass `{ select: true }` to run a lightweight `SELECT 1` instead:
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
const r = await client.ping({ select: true })
|
|
94
|
+
// success only if the server is reachable AND auth is correct AND it can run queries
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
| | `client.ping()` | `client.ping({ select: true })` |
|
|
98
|
+
| ----------------------- | --------------- | ------------------------------- |
|
|
99
|
+
| Endpoint | `/ping` (HTTP) | `SELECT 1` query |
|
|
100
|
+
| Checks auth | **No** | Yes |
|
|
101
|
+
| Checks query processing | No | **Yes** |
|
|
102
|
+
| Overhead | Minimal | Slightly higher |
|
|
103
|
+
|
|
104
|
+
**When to use which:**
|
|
105
|
+
|
|
106
|
+
- **Liveness probe** (is the process alive?) — plain `ping()` is fine.
|
|
107
|
+
- **Readiness probe** (can it serve traffic?) — use `ping({ select: true })`
|
|
108
|
+
so the probe fails if credentials are wrong or the query layer is broken.
|
|
109
|
+
- **Waking a ClickHouse Cloud idle instance** — plain `ping()` is enough.
|
|
110
|
+
|
|
111
|
+
## Common pitfalls
|
|
112
|
+
|
|
113
|
+
- **Do not wrap `ping()` in `try/catch` as your only check.** It resolves on
|
|
114
|
+
failure; the `success` boolean is the source of truth.
|
|
115
|
+
- **Lower `request_timeout` if you want pings to fail fast** (the example
|
|
116
|
+
above uses `50` ms). The default is high enough to be unsuitable for
|
|
117
|
+
liveness probes.
|
|
118
|
+
- **Plain `ping()` does not check credentials.** If auth is part of what you
|
|
119
|
+
want to verify, use `ping({ select: true })`.
|
|
120
|
+
- For ping that times out specifically, see the troubleshooting skill.
|
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# Query Parameter Binding
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. NULL parameter binding fixed in `0.0.16`.
|
|
4
|
+
> Special-character (tab/newline/quote/backslash) binding `>= 0.3.1`.
|
|
5
|
+
> `TupleParam` and JS `Map` parameters `>= 1.9.0`. Boolean formatting in
|
|
6
|
+
> `Array`/`Tuple`/`Map` parameters fixed in `>= 1.13.0`. `BigInt` query
|
|
7
|
+
> parameters `>= 1.15.0`.
|
|
8
|
+
|
|
9
|
+
Backing examples:
|
|
10
|
+
[`examples/node/coding/query_with_parameter_binding.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/query_with_parameter_binding.ts),
|
|
11
|
+
[`examples/node/coding/query_with_parameter_binding_special_chars.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/query_with_parameter_binding_special_chars.ts).
|
|
12
|
+
|
|
13
|
+
## Answer checklist
|
|
14
|
+
|
|
15
|
+
When the user passes user-controlled values into SQL:
|
|
16
|
+
|
|
17
|
+
- Use ClickHouse `{name: Type}` placeholders and a `query_params` object.
|
|
18
|
+
- Explicitly call template-literal/string interpolation of user input a
|
|
19
|
+
**SQL injection risk**.
|
|
20
|
+
- Do not suggest PostgreSQL/MySQL-style `$1`, `?`, or `:name` placeholders.
|
|
21
|
+
- Pick the placeholder type to match the ClickHouse column type (`String`,
|
|
22
|
+
`Date`, `DateTime`, `Nullable(T)`, etc.).
|
|
23
|
+
|
|
24
|
+
## Syntax: `{name: Type}`
|
|
25
|
+
|
|
26
|
+
ClickHouse uses `{name: Type}` placeholders — **not** `$1`, `?`, or `:name`.
|
|
27
|
+
|
|
28
|
+
```ts
|
|
29
|
+
await client.query({
|
|
30
|
+
query: 'SELECT plus({a: Int32}, {b: Int32})',
|
|
31
|
+
format: 'JSONEachRow',
|
|
32
|
+
query_params: { a: 10, b: 20 },
|
|
33
|
+
})
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The `Type` must be a valid ClickHouse type (`Int32`, `String`, `Date`,
|
|
37
|
+
`Array(UInt32)`, `Tuple(Int32, String)`, `Map(K, V)`, `Nullable(T)`, etc.).
|
|
38
|
+
|
|
39
|
+
## ⚠️ Never use template literals for user values
|
|
40
|
+
|
|
41
|
+
Interpolating user input into the SQL string bypasses server-side escaping
|
|
42
|
+
and opens the door to SQL injection:
|
|
43
|
+
|
|
44
|
+
```ts
|
|
45
|
+
// ❌ Dangerous — never do this with user-controlled values
|
|
46
|
+
const userId = req.params.id
|
|
47
|
+
await client.query({ query: `SELECT * FROM users WHERE id = ${userId}` })
|
|
48
|
+
|
|
49
|
+
// ✓ Safe — parameterized
|
|
50
|
+
await client.query({
|
|
51
|
+
query: 'SELECT * FROM users WHERE id = {id: UInt32}',
|
|
52
|
+
query_params: { id: userId },
|
|
53
|
+
})
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
This is the most common mistake for users coming from PostgreSQL/MySQL. Call
|
|
57
|
+
it out explicitly when the user shows template-literal interpolation.
|
|
58
|
+
|
|
59
|
+
## Common types
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
import { TupleParam } from '@clickhouse/client'
|
|
63
|
+
|
|
64
|
+
await client.query({
|
|
65
|
+
query: `
|
|
66
|
+
SELECT
|
|
67
|
+
{var_int: Int32} AS var_int,
|
|
68
|
+
{var_float: Float32} AS var_float,
|
|
69
|
+
{var_str: String} AS var_str,
|
|
70
|
+
{var_array: Array(Int32)} AS var_array,
|
|
71
|
+
{var_tuple: Tuple(Int32, String)} AS var_tuple,
|
|
72
|
+
{var_map: Map(Int, Array(String))} AS var_map,
|
|
73
|
+
{var_date: Date} AS var_date,
|
|
74
|
+
{var_datetime: DateTime} AS var_datetime,
|
|
75
|
+
{var_datetime64_3: DateTime64(3)} AS var_datetime64_3,
|
|
76
|
+
{var_datetime64_9: DateTime64(9)} AS var_datetime64_9,
|
|
77
|
+
{var_decimal: Decimal(9, 2)} AS var_decimal,
|
|
78
|
+
{var_uuid: UUID} AS var_uuid,
|
|
79
|
+
{var_ipv4: IPv4} AS var_ipv4,
|
|
80
|
+
{var_null: Nullable(String)} AS var_null
|
|
81
|
+
`,
|
|
82
|
+
format: 'JSONEachRow',
|
|
83
|
+
query_params: {
|
|
84
|
+
var_int: 10,
|
|
85
|
+
var_float: '10.557',
|
|
86
|
+
var_str: 'hello',
|
|
87
|
+
var_array: [42, 144],
|
|
88
|
+
var_tuple: new TupleParam([42, 'foo']), // >= 1.9.0
|
|
89
|
+
var_map: new Map([
|
|
90
|
+
[42, ['a', 'b']],
|
|
91
|
+
[144, ['c', 'd']],
|
|
92
|
+
]), // >= 1.9.0
|
|
93
|
+
var_date: '2022-01-01',
|
|
94
|
+
var_datetime: '2022-01-01 12:34:56', // or a Date
|
|
95
|
+
var_datetime64_3: '2022-01-01 12:34:56.789', // or a Date
|
|
96
|
+
var_datetime64_9: '2022-01-01 12:34:56.123456789', // string for ns precision
|
|
97
|
+
var_decimal: '123.45', // string to avoid precision loss
|
|
98
|
+
var_uuid: '01234567-89ab-cdef-0123-456789abcdef',
|
|
99
|
+
var_ipv4: '192.168.0.1',
|
|
100
|
+
var_null: null, // fixed in 0.0.16
|
|
101
|
+
},
|
|
102
|
+
})
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### Type-by-type tips
|
|
106
|
+
|
|
107
|
+
- **Decimals** — pass as strings to avoid JS number precision loss.
|
|
108
|
+
- **`DateTime64(>3)`** — pass as a string; JS `Date` only has millisecond
|
|
109
|
+
precision and will lose sub-millisecond digits.
|
|
110
|
+
- **`DateTime64`** — strings can also be UNIX timestamps, including
|
|
111
|
+
fractional ones (e.g., `'1651490755.123456789'`).
|
|
112
|
+
- **`BigInt`** — supported in `query_params` since `>= 1.15.0`. On older
|
|
113
|
+
clients, pass as a string.
|
|
114
|
+
- **`Tuple(...)`** — wrap in `new TupleParam([...])` (`>= 1.9.0`); on older
|
|
115
|
+
clients, build the literal manually as a string.
|
|
116
|
+
- **`Map(K, V)`** — pass a JS `Map` (`>= 1.9.0`); on older clients, build
|
|
117
|
+
it manually.
|
|
118
|
+
- **`Nullable(T)`** — pass `null` directly (`>= 0.0.16`).
|
|
119
|
+
|
|
120
|
+
## Special characters in string parameters (`>= 0.3.1`)
|
|
121
|
+
|
|
122
|
+
Tabs, newlines, carriage returns, single quotes, and backslashes are
|
|
123
|
+
escaped automatically by the client — just pass the JS string as-is:
|
|
124
|
+
|
|
125
|
+
```ts
|
|
126
|
+
await client.query({
|
|
127
|
+
query: `
|
|
128
|
+
SELECT
|
|
129
|
+
'foo_\t_bar' = {tab: String} AS has_tab,
|
|
130
|
+
'foo_\n_bar' = {newline: String} AS has_newline,
|
|
131
|
+
'foo_\\'_bar' = {single_quote: String} AS has_single_quote,
|
|
132
|
+
'foo_\\_bar' = {backslash: String} AS has_backslash
|
|
133
|
+
`,
|
|
134
|
+
format: 'JSONEachRow',
|
|
135
|
+
query_params: {
|
|
136
|
+
tab: 'foo_\t_bar',
|
|
137
|
+
newline: 'foo_\n_bar',
|
|
138
|
+
single_quote: "foo_'_bar",
|
|
139
|
+
backslash: 'foo_\\_bar',
|
|
140
|
+
},
|
|
141
|
+
})
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
## Common pitfalls
|
|
145
|
+
|
|
146
|
+
- **`$1` / `?` / `:name` placeholders.** None work — use `{name: Type}`.
|
|
147
|
+
- **Forgetting the type in the placeholder.** `{id}` is a syntax error;
|
|
148
|
+
it must be `{id: UInt32}`.
|
|
149
|
+
- **Stringifying tuples/maps manually on `>= 1.9.0`.** Use `TupleParam`
|
|
150
|
+
and `Map` — both serialize correctly and respect special characters.
|
|
151
|
+
- **Boolean array/tuple/map elements before `1.13.0`.** Boolean formatting
|
|
152
|
+
was fixed in 1.13.0 — earlier versions may misformat them.
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Select Data Formats
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `JSONEachRowWithProgress` requires client
|
|
4
|
+
> `>= 1.7.0`; see the in-repo performance examples under
|
|
5
|
+
> `examples/node/performance/`.
|
|
6
|
+
|
|
7
|
+
Backing examples:
|
|
8
|
+
[`examples/node/coding/select_json_each_row.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/select_json_each_row.ts),
|
|
9
|
+
[`examples/node/coding/select_data_formats_overview.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/select_data_formats_overview.ts),
|
|
10
|
+
[`examples/node/coding/select_json_with_metadata.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/select_json_with_metadata.ts).
|
|
11
|
+
|
|
12
|
+
## Default choice: `JSONEachRow` → `.json<T>()`
|
|
13
|
+
|
|
14
|
+
Right answer for ~90% of selects when the result fits in memory.
|
|
15
|
+
|
|
16
|
+
```ts
|
|
17
|
+
import { createClient } from '@clickhouse/client'
|
|
18
|
+
|
|
19
|
+
interface Row {
|
|
20
|
+
number: string
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
const client = createClient()
|
|
24
|
+
const rows = await client.query({
|
|
25
|
+
query: 'SELECT number FROM system.numbers LIMIT 5',
|
|
26
|
+
format: 'JSONEachRow',
|
|
27
|
+
})
|
|
28
|
+
const result = await rows.json<Row>() // Row[]
|
|
29
|
+
result.forEach((r) => console.log(r))
|
|
30
|
+
await client.close()
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
`UInt64`/`Int64` and other 64-bit integers are returned as **strings**
|
|
34
|
+
when `output_format_json_quote_64bit_integers=1`, to avoid JS precision
|
|
35
|
+
loss. If that setting is `0`, they may be returned as unquoted JSON
|
|
36
|
+
numbers instead. Note that in ClickHouse `>= 25.8`, this setting can
|
|
37
|
+
default to `0`; see the troubleshooting skill for ways to control that.
|
|
38
|
+
|
|
39
|
+
## Single-document `JSON` format with metadata
|
|
40
|
+
|
|
41
|
+
Use `JSON` (or `JSONCompact`) when you need ClickHouse's response envelope
|
|
42
|
+
(rows + meta + statistics + row count). Type the result with
|
|
43
|
+
`ResponseJSON<T>`:
|
|
44
|
+
|
|
45
|
+
```ts
|
|
46
|
+
import { createClient, type ResponseJSON } from '@clickhouse/client'
|
|
47
|
+
|
|
48
|
+
const client = createClient()
|
|
49
|
+
const rows = await client.query({
|
|
50
|
+
query: 'SELECT number FROM system.numbers LIMIT 2',
|
|
51
|
+
format: 'JSON',
|
|
52
|
+
})
|
|
53
|
+
const result = await rows.json<ResponseJSON<{ number: string }>>()
|
|
54
|
+
console.info(result.meta, result.data, result.rows, result.statistics)
|
|
55
|
+
await client.close()
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
> `JSON`, `JSONCompact`, `JSONStrings`, `JSONCompactStrings`,
|
|
59
|
+
> `JSONColumnsWithMetadata`, `JSONObjectEachRow` are **single-document**
|
|
60
|
+
> formats — they cannot be streamed. Use a `*EachRow` variant if you want
|
|
61
|
+
> to stream.
|
|
62
|
+
|
|
63
|
+
## Selecting raw text (CSV / TSV / CustomSeparated)
|
|
64
|
+
|
|
65
|
+
Use `.text()` (not `.json()`) for raw textual formats:
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
const rs = await client.query({
|
|
69
|
+
query: 'SELECT number, number * 2 AS doubled FROM system.numbers LIMIT 3',
|
|
70
|
+
format: 'CSVWithNames',
|
|
71
|
+
})
|
|
72
|
+
console.log(await rs.text())
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Streaming raw text/Parquet line-by-line belongs in
|
|
76
|
+
[`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)
|
|
77
|
+
— in particular, Parquet exports use `client.exec()` and pipe the raw
|
|
78
|
+
response stream rather than `ResultSet.stream()` (see
|
|
79
|
+
[`select_parquet_as_file.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/performance/select_parquet_as_file.ts)).
|
|
80
|
+
|
|
81
|
+
## Format chooser
|
|
82
|
+
|
|
83
|
+
| Use case | Format |
|
|
84
|
+
| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
85
|
+
| Read rows as JS objects | `JSONEachRow` _(default)_ |
|
|
86
|
+
| Read rows as positional tuples (smaller payload) | `JSONCompactEachRow` |
|
|
87
|
+
| Need `meta` / `statistics` / `rows` envelope | `JSON` or `JSONCompact` + `ResponseJSON<T>` |
|
|
88
|
+
| Read all values as strings (avoid number-precision loss) | `JSONStringsEachRow` / `JSONCompactStringsEachRow` |
|
|
89
|
+
| Stream very large result | `JSONEachRow` / `JSONCompactEachRow` (see [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)) |
|
|
90
|
+
| Export to CSV/TSV/Parquet | `CSV*`, `TabSeparated*`, `Parquet` (see [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)) |
|
|
91
|
+
|
|
92
|
+
## ResultSet methods
|
|
93
|
+
|
|
94
|
+
| Method | Returns | Notes |
|
|
95
|
+
| -------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
96
|
+
| `await rs.json<T>()` | `T[]` for `*EachRow`, single-doc shape otherwise | Buffers the full response |
|
|
97
|
+
| `await rs.text()` | `string` | Buffers the full response — for textual formats only (CSV/TSV/etc.) |
|
|
98
|
+
| `rs.stream()` | Node `Readable` of `Row[]` chunks | Use for large newline-delimited results (`JSONEachRow`/`JSONCompactEachRow`/`CSV`/`TSV`); **not** suitable for binary formats like `Parquet` — for those, use `client.exec()` and pipe the raw response stream (see [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)) |
|
|
99
|
+
| `rs.close()` | `void` (synchronous) | Always call if you obtained `stream()` and stop reading early |
|
|
100
|
+
|
|
101
|
+
## Common pitfalls
|
|
102
|
+
|
|
103
|
+
- **Calling `.json()` on a `JSON` (single-doc) result and expecting an
|
|
104
|
+
array.** You get a `ResponseJSON<T>` object; the rows are under
|
|
105
|
+
`.data`. Use `JSONEachRow` if you want a flat array.
|
|
106
|
+
- **Leaving a `stream()` half-consumed.** This is a top cause of
|
|
107
|
+
`ECONNRESET` on the _next_ request — fully iterate the stream or call
|
|
108
|
+
`resultSet.close()` (synchronous — no `await`). (Diagnosis details live in the
|
|
109
|
+
troubleshooting skill.)
|
|
110
|
+
- **Reaching for `.json()` on a CSV/TSV result.** Use `.text()` (or
|
|
111
|
+
`.stream()` for large results).
|
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# Sessions and Temporary Tables
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `session_id` is a server-level concept; the
|
|
4
|
+
> client just forwards it on every request that names it.
|
|
5
|
+
|
|
6
|
+
Backing examples:
|
|
7
|
+
[`examples/node/coding/session_id_and_temporary_tables.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/session_id_and_temporary_tables.ts),
|
|
8
|
+
[`examples/node/coding/session_level_commands.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/session_level_commands.ts).
|
|
9
|
+
|
|
10
|
+
## When you need a session
|
|
11
|
+
|
|
12
|
+
Use a `session_id` whenever multiple calls must share **server-side state**:
|
|
13
|
+
|
|
14
|
+
- `CREATE TEMPORARY TABLE` (the table only exists within its session).
|
|
15
|
+
- `SET <setting> = <value>` to apply for subsequent queries on the same
|
|
16
|
+
session.
|
|
17
|
+
- Any other server feature scoped per session (e.g., session-scoped
|
|
18
|
+
variables in newer ClickHouse versions).
|
|
19
|
+
|
|
20
|
+
## ⚠️ `session_id` and concurrency
|
|
21
|
+
|
|
22
|
+
ClickHouse **rejects concurrent queries within the same session** — if two
|
|
23
|
+
requests arrive at the server at the same time sharing the same `session_id`,
|
|
24
|
+
the second one gets an error like
|
|
25
|
+
`"Session is locked by a concurrent client"`. This has two practical
|
|
26
|
+
implications:
|
|
27
|
+
|
|
28
|
+
1. **Do not set `session_id` on a global / module-static client** that handles
|
|
29
|
+
concurrent requests (e.g., an Express app's shared client). Every
|
|
30
|
+
in-flight request would share the same session and collide under load.
|
|
31
|
+
2. **If you do set `session_id` on a client**, restrict its concurrency:
|
|
32
|
+
set `max_open_connections: 1` so at most one request is in flight at a
|
|
33
|
+
time, turning the pool into a serial queue. This is fine for a
|
|
34
|
+
dedicated per-workflow client but wrong for a shared application client.
|
|
35
|
+
|
|
36
|
+
The right pattern for application code: create a **short-lived client** (or
|
|
37
|
+
use per-request `session_id`) scoped to a single logical workflow, not to
|
|
38
|
+
the entire process.
|
|
39
|
+
|
|
40
|
+
## Per-client `session_id`
|
|
41
|
+
|
|
42
|
+
Appropriate when **one client handles exactly one sequential workflow** (a
|
|
43
|
+
script, a background job, a single user's session that you've already
|
|
44
|
+
serialized).
|
|
45
|
+
|
|
46
|
+
```ts
|
|
47
|
+
import { createClient } from '@clickhouse/client'
|
|
48
|
+
import * as crypto from 'node:crypto'
|
|
49
|
+
|
|
50
|
+
const client = createClient({
|
|
51
|
+
session_id: crypto.randomUUID(),
|
|
52
|
+
max_open_connections: 1, // prevent concurrent-session errors
|
|
53
|
+
})
|
|
54
|
+
|
|
55
|
+
await client.command({
|
|
56
|
+
query: 'CREATE TEMPORARY TABLE temporary_example (i Int32)',
|
|
57
|
+
})
|
|
58
|
+
|
|
59
|
+
await client.insert({
|
|
60
|
+
table: 'temporary_example',
|
|
61
|
+
values: [{ i: 42 }, { i: 144 }],
|
|
62
|
+
format: 'JSONEachRow',
|
|
63
|
+
})
|
|
64
|
+
|
|
65
|
+
const rs = await client.query({
|
|
66
|
+
query: 'SELECT * FROM temporary_example',
|
|
67
|
+
format: 'JSONEachRow',
|
|
68
|
+
})
|
|
69
|
+
console.info(await rs.json())
|
|
70
|
+
await client.close()
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
## Session-level `SET` commands
|
|
74
|
+
|
|
75
|
+
`SET` only persists within a session. With `session_id` defined on the
|
|
76
|
+
client, every subsequent call inherits the change.
|
|
77
|
+
|
|
78
|
+
```ts
|
|
79
|
+
import { createClient } from '@clickhouse/client'
|
|
80
|
+
import * as crypto from 'node:crypto'
|
|
81
|
+
|
|
82
|
+
const client = createClient({
|
|
83
|
+
session_id: crypto.randomUUID(),
|
|
84
|
+
max_open_connections: 1, // prevent concurrent-session errors
|
|
85
|
+
})
|
|
86
|
+
|
|
87
|
+
await client.command({
|
|
88
|
+
query: 'SET output_format_json_quote_64bit_integers = 0',
|
|
89
|
+
clickhouse_settings: { wait_end_of_query: 1 }, // ack before next call
|
|
90
|
+
})
|
|
91
|
+
|
|
92
|
+
const rs1 = await client.query({
|
|
93
|
+
query: 'SELECT toInt64(42)',
|
|
94
|
+
format: 'JSONEachRow',
|
|
95
|
+
})
|
|
96
|
+
// → 64-bit integers come back as numbers in this query
|
|
97
|
+
|
|
98
|
+
await client.command({
|
|
99
|
+
query: 'SET output_format_json_quote_64bit_integers = 1',
|
|
100
|
+
clickhouse_settings: { wait_end_of_query: 1 },
|
|
101
|
+
})
|
|
102
|
+
|
|
103
|
+
const rs2 = await client.query({
|
|
104
|
+
query: 'SELECT toInt64(144)',
|
|
105
|
+
format: 'JSONEachRow',
|
|
106
|
+
})
|
|
107
|
+
// → 64-bit integers come back as strings again
|
|
108
|
+
|
|
109
|
+
await client.close()
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
> **`wait_end_of_query: 1` matters here.** Without it, a `SET` on one
|
|
113
|
+
> connection in the pool may not yet be applied when the next query lands
|
|
114
|
+
> on the same socket.
|
|
115
|
+
|
|
116
|
+
## Per-request `session_id`
|
|
117
|
+
|
|
118
|
+
You can also pass `session_id` on a single `query()` / `insert()` /
|
|
119
|
+
`command()` call to override (or set) it for that one request.
|
|
120
|
+
|
|
121
|
+
## ⚠️ Sessions and load balancers / ClickHouse Cloud
|
|
122
|
+
|
|
123
|
+
Sessions are bound to a **specific ClickHouse node**. If a load balancer in
|
|
124
|
+
front of ClickHouse routes consecutive requests to different nodes, the
|
|
125
|
+
temporary table / `SET` won't be visible — you'll get
|
|
126
|
+
`UNKNOWN_TABLE` / surprising results.
|
|
127
|
+
|
|
128
|
+
Mitigations:
|
|
129
|
+
|
|
130
|
+
- Talk to a single node directly.
|
|
131
|
+
- For ClickHouse Cloud, use [replica-aware
|
|
132
|
+
routing](https://clickhouse.com/docs/manage/replica-aware-routing).
|
|
133
|
+
- Avoid sessions for cross-node workflows; persist intermediate state in a
|
|
134
|
+
regular (non-temporary) table instead.
|
|
135
|
+
|
|
136
|
+
## Common pitfalls
|
|
137
|
+
|
|
138
|
+
- **Forgetting `session_id` and being surprised that
|
|
139
|
+
`CREATE TEMPORARY TABLE` "disappears."** Without a session, every request
|
|
140
|
+
may land on a different connection / server context.
|
|
141
|
+
- **Setting `session_id` on a shared application client.** Under concurrent
|
|
142
|
+
load, two in-flight requests will share the same session and one will fail
|
|
143
|
+
with `"Session is locked by a concurrent client"`. Use per-request
|
|
144
|
+
`session_id` or a dedicated short-lived client instead.
|
|
145
|
+
- **Reusing the same `session_id` across unrelated workflows.** A second
|
|
146
|
+
session-using consumer will trip over your temporary tables and `SET`
|
|
147
|
+
values. Generate a fresh UUID per logical session.
|
|
148
|
+
- **Leaving session state pinned for the lifetime of the process.** If
|
|
149
|
+
long-lived clients accumulate `SET` / temp-table state, consider creating
|
|
150
|
+
a short-lived sub-client with its own `session_id` for the unit of work.
|
|
151
|
+
- **Skipping `wait_end_of_query: 1` on `SET`** — race conditions between
|
|
152
|
+
`SET` and the next query can show up under load.
|