@clickhouse/client 1.18.4 → 1.18.5-head.09b78f0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/config.d.ts +17 -0
- package/dist/config.js +1 -0
- package/dist/config.js.map +1 -1
- package/dist/connection/create_connection.d.ts +2 -1
- package/dist/connection/create_connection.js +4 -1
- package/dist/connection/create_connection.js.map +1 -1
- package/dist/connection/node_base_connection.d.ts +9 -0
- package/dist/connection/node_base_connection.js.map +1 -1
- package/dist/connection/node_custom_agent_connection.js +3 -0
- package/dist/connection/node_custom_agent_connection.js.map +1 -1
- package/dist/connection/node_http_connection.js +3 -0
- package/dist/connection/node_http_connection.js.map +1 -1
- package/dist/connection/node_https_connection.js +3 -0
- package/dist/connection/node_https_connection.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/dist/version.js.map +1 -1
- package/package.json +6 -2
- package/skills/clickhouse-js-node-coding/SKILL.md +146 -0
- package/skills/clickhouse-js-node-coding/evals/evals.json +103 -0
- package/skills/clickhouse-js-node-coding/reference/async-insert.md +103 -0
- package/skills/clickhouse-js-node-coding/reference/client-configuration.md +159 -0
- package/skills/clickhouse-js-node-coding/reference/custom-json.md +149 -0
- package/skills/clickhouse-js-node-coding/reference/data-types.md +169 -0
- package/skills/clickhouse-js-node-coding/reference/insert-columns.md +113 -0
- package/skills/clickhouse-js-node-coding/reference/insert-formats.md +145 -0
- package/skills/clickhouse-js-node-coding/reference/insert-values.md +141 -0
- package/skills/clickhouse-js-node-coding/reference/ping.md +120 -0
- package/skills/clickhouse-js-node-coding/reference/query-parameters.md +152 -0
- package/skills/clickhouse-js-node-coding/reference/select-formats.md +111 -0
- package/skills/clickhouse-js-node-coding/reference/sessions.md +152 -0
- package/skills/clickhouse-js-node-troubleshooting/reference/socket-hangup.md +15 -3
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
# Insert Data Formats
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. The `JSON` type column / new JSON family is a
|
|
4
|
+
> ClickHouse feature; the JSON _formats_ listed here are universally supported
|
|
5
|
+
> by the client.
|
|
6
|
+
|
|
7
|
+
Backing examples:
|
|
8
|
+
[`examples/node/coding/array_json_each_row.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/array_json_each_row.ts),
|
|
9
|
+
[`examples/node/coding/insert_data_formats_overview.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_data_formats_overview.ts).
|
|
10
|
+
|
|
11
|
+
> **Raw / binary formats (CSV, TSV, CustomSeparated, Parquet) require a Node
|
|
12
|
+
> stream as input.** See
|
|
13
|
+
> [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)
|
|
14
|
+
> — defer if the user wants to insert from a file or `Readable`.
|
|
15
|
+
|
|
16
|
+
## Answer checklist
|
|
17
|
+
|
|
18
|
+
When answering "what format/call should I use for an array of JS objects?":
|
|
19
|
+
|
|
20
|
+
- Use `client.insert({ table, values, format: 'JSONEachRow' })`.
|
|
21
|
+
- Say the array of plain objects can be passed directly as `values` for
|
|
22
|
+
ordinary in-memory batches such as a few thousand or tens of thousands of
|
|
23
|
+
rows.
|
|
24
|
+
- Do not steer the user to streaming, Parquet, or file APIs unless their input
|
|
25
|
+
is already a stream/file or the task is explicitly about throughput.
|
|
26
|
+
- Warn not to wrap `JSONEachRow` rows in a `{ data: [...] }` envelope; that
|
|
27
|
+
shape belongs to single-document formats.
|
|
28
|
+
- Mention `JSONCompactEachRow*` as a denser alternative for larger payloads
|
|
29
|
+
when the caller can provide positional arrays or explicit names/types.
|
|
30
|
+
|
|
31
|
+
## Default choice: `JSONEachRow` with an array of objects
|
|
32
|
+
|
|
33
|
+
This is the right answer for ~90% of inserts.
|
|
34
|
+
|
|
35
|
+
```ts
|
|
36
|
+
import { createClient } from '@clickhouse/client'
|
|
37
|
+
|
|
38
|
+
const client = createClient()
|
|
39
|
+
|
|
40
|
+
await client.insert({
|
|
41
|
+
table: 'events',
|
|
42
|
+
format: 'JSONEachRow',
|
|
43
|
+
values: [
|
|
44
|
+
{ id: 42, name: 'foo' },
|
|
45
|
+
{ id: 43, name: 'bar' },
|
|
46
|
+
],
|
|
47
|
+
})
|
|
48
|
+
|
|
49
|
+
await client.close()
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
The shape of `values` must match the chosen format.
|
|
53
|
+
|
|
54
|
+
## Streamable JSON formats (pass an array)
|
|
55
|
+
|
|
56
|
+
| Format | `values` shape |
|
|
57
|
+
| -------------------------------------------- | --------------------------------------------------- |
|
|
58
|
+
| `JSONEachRow` | `Array<{ col: value, ... }>` |
|
|
59
|
+
| `JSONStringsEachRow` | `Array<{ col: stringifiedValue, ... }>` |
|
|
60
|
+
| `JSONCompactEachRow` | `Array<[v1, v2, ...]>` |
|
|
61
|
+
| `JSONCompactStringsEachRow` | `Array<[stringV1, stringV2, ...]>` |
|
|
62
|
+
| `JSONCompactEachRowWithNames` | First row = column names, then data rows |
|
|
63
|
+
| `JSONCompactEachRowWithNamesAndTypes` | Row 1 = names, row 2 = types, then data |
|
|
64
|
+
| `JSONCompactStringsEachRowWithNames` | First row = names, then stringified data rows |
|
|
65
|
+
| `JSONCompactStringsEachRowWithNamesAndTypes` | Row 1 = names, row 2 = types, then stringified data |
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
await client.insert({
|
|
69
|
+
table: 'events',
|
|
70
|
+
format: 'JSONCompactEachRowWithNamesAndTypes',
|
|
71
|
+
values: [
|
|
72
|
+
['id', 'name', 'sku'],
|
|
73
|
+
['UInt32', 'String', 'Array(UInt32)'],
|
|
74
|
+
[11, 'foo', [1, 2, 3]],
|
|
75
|
+
[12, 'bar', [4, 5, 6]],
|
|
76
|
+
],
|
|
77
|
+
})
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
These formats can be **streamed** — pass a Node stream of rows instead of an
|
|
81
|
+
array. See
|
|
82
|
+
[`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)
|
|
83
|
+
for streaming guidance.
|
|
84
|
+
|
|
85
|
+
## Single-document JSON formats (pass an object)
|
|
86
|
+
|
|
87
|
+
These cannot be streamed — the entire body is sent in one shot.
|
|
88
|
+
|
|
89
|
+
| Format | `values` shape (typed via `InputJSON<T>` / `InputJSONObjectEachRow<T>`) |
|
|
90
|
+
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
|
91
|
+
| `JSON` | `{ meta: [], data: Array<{ col: value, ... }> }` — for TypeScript/client usage, pass `meta: []` if metadata is not needed |
|
|
92
|
+
| `JSONCompact` | `{ meta: [{ name, type }, ...], data: Array<[v1, v2, ...]> }` |
|
|
93
|
+
| `JSONColumnsWithMetadata` | `{ meta: [...], data: { col1: [v, ...], col2: [v, ...] } }` |
|
|
94
|
+
| `JSONObjectEachRow` | `Record<string, { col: value, ... }>` (the record key labels each row but is not stored) |
|
|
95
|
+
|
|
96
|
+
```ts
|
|
97
|
+
import type { InputJSON, InputJSONObjectEachRow } from '@clickhouse/client'
|
|
98
|
+
|
|
99
|
+
const meta: InputJSON['meta'] = [
|
|
100
|
+
{ name: 'id', type: 'UInt32' },
|
|
101
|
+
{ name: 'name', type: 'String' },
|
|
102
|
+
]
|
|
103
|
+
|
|
104
|
+
await client.insert({
|
|
105
|
+
table: 'events',
|
|
106
|
+
format: 'JSONCompact',
|
|
107
|
+
values: {
|
|
108
|
+
meta,
|
|
109
|
+
data: [
|
|
110
|
+
[19, 'foo'],
|
|
111
|
+
[20, 'bar'],
|
|
112
|
+
],
|
|
113
|
+
},
|
|
114
|
+
})
|
|
115
|
+
|
|
116
|
+
await client.insert({
|
|
117
|
+
table: 'events',
|
|
118
|
+
format: 'JSONObjectEachRow',
|
|
119
|
+
values: {
|
|
120
|
+
row_1: { id: 23, name: 'foo' },
|
|
121
|
+
row_2: { id: 24, name: 'bar' },
|
|
122
|
+
} satisfies InputJSONObjectEachRow<{ id: number; name: string }>,
|
|
123
|
+
})
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## Quick chooser
|
|
127
|
+
|
|
128
|
+
| Use case | Format |
|
|
129
|
+
| -------------------------------------------- | ------------------------------------------------- |
|
|
130
|
+
| Insert plain JS objects | `JSONEachRow` _(default)_ |
|
|
131
|
+
| Insert tuples / column-positional rows | `JSONCompactEachRow` |
|
|
132
|
+
| Insert with explicit column ordering / types | `JSONCompactEachRow*WithNames…` |
|
|
133
|
+
| Insert a single document with metadata | `JSON`, `JSONCompact` |
|
|
134
|
+
| Insert from a CSV / TSV / Parquet file | Raw format + Node stream → `examples/node/performance/` |
|
|
135
|
+
|
|
136
|
+
## Common pitfalls
|
|
137
|
+
|
|
138
|
+
- **Wrong shape for the format.** The most common cause of insert failures —
|
|
139
|
+
e.g., passing `Array<{...}>` to `JSONCompact` (which expects
|
|
140
|
+
`{ meta, data }`).
|
|
141
|
+
- **Don't wrap a `JSONEachRow` array in a `{ data: [...] }` envelope.** That
|
|
142
|
+
envelope only belongs to single-document formats (`JSON` / `JSONCompact` /
|
|
143
|
+
`JSONColumnsWithMetadata`).
|
|
144
|
+
- For type guidance (`Decimal` strings, `Date` objects, `BigInt`), see
|
|
145
|
+
`insert-values.md` and `custom-json.md`.
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
# Insert Values, SQL Expressions, Dates, Decimals
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `wait_end_of_query: 1` is a server-side
|
|
4
|
+
> setting available on every supported ClickHouse version.
|
|
5
|
+
|
|
6
|
+
Backing examples:
|
|
7
|
+
[`examples/node/coding/insert_from_select.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_from_select.ts),
|
|
8
|
+
[`examples/node/coding/insert_values_and_functions.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_values_and_functions.ts),
|
|
9
|
+
[`examples/node/coding/insert_js_dates.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_js_dates.ts),
|
|
10
|
+
[`examples/node/coding/insert_decimals.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/insert_decimals.ts).
|
|
11
|
+
|
|
12
|
+
## `INSERT … SELECT` (no values payload)
|
|
13
|
+
|
|
14
|
+
When the data already lives in ClickHouse, use `client.command()` with a raw
|
|
15
|
+
`INSERT … SELECT`:
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
await client.command({
|
|
19
|
+
query: `
|
|
20
|
+
INSERT INTO target
|
|
21
|
+
SELECT '42', quantilesBFloat16State(0.5)(arrayJoin([toFloat32(10), toFloat32(20)]))
|
|
22
|
+
`,
|
|
23
|
+
})
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Use `command()` (not `insert()`) — there is no row payload to send.
|
|
27
|
+
|
|
28
|
+
## `INSERT … VALUES` with SQL functions
|
|
29
|
+
|
|
30
|
+
When you need `unhex(...)`, `toUUID(...)`, `now()`, or any other SQL
|
|
31
|
+
function around a value, keep the SQL shape static and pass values with
|
|
32
|
+
ClickHouse `{name: Type}` parameters. Run it via `command()` and set
|
|
33
|
+
`wait_end_of_query: 1` for safety in clustered setups.
|
|
34
|
+
|
|
35
|
+
```ts
|
|
36
|
+
await client.command({
|
|
37
|
+
query: `
|
|
38
|
+
INSERT INTO events (id, timestamp, email, name)
|
|
39
|
+
VALUES (
|
|
40
|
+
unhex({id: String}),
|
|
41
|
+
{timestamp: DateTime},
|
|
42
|
+
{email: String},
|
|
43
|
+
{name: Nullable(String)}
|
|
44
|
+
)
|
|
45
|
+
`,
|
|
46
|
+
query_params: {
|
|
47
|
+
id: '00112233445566778899aabbccddeeff',
|
|
48
|
+
timestamp: '2026-05-06 12:34:56',
|
|
49
|
+
email: 'alice@example.com',
|
|
50
|
+
name: 'Alice',
|
|
51
|
+
},
|
|
52
|
+
clickhouse_settings: { wait_end_of_query: 1 },
|
|
53
|
+
})
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Do not build `VALUES` rows with string interpolation or manual escaping. If
|
|
57
|
+
you need to insert many ordinary JS rows, prefer `client.insert()` with
|
|
58
|
+
`format: 'JSONEachRow'`; use this `command()` pattern when the SQL itself needs
|
|
59
|
+
functions or expressions around the values.
|
|
60
|
+
|
|
61
|
+
## Inserting JS `Date` objects
|
|
62
|
+
|
|
63
|
+
JS `Date` objects work for `DateTime` and `DateTime64` columns once the
|
|
64
|
+
server is set to accept ISO-8601 strings. Either set
|
|
65
|
+
`date_time_input_format: 'best_effort'` per request, on the client, or
|
|
66
|
+
session-wide.
|
|
67
|
+
|
|
68
|
+
```ts
|
|
69
|
+
await client.insert({
|
|
70
|
+
table: 'events',
|
|
71
|
+
format: 'JSONEachRow',
|
|
72
|
+
values: [{ id: '42', dt: new Date() }],
|
|
73
|
+
clickhouse_settings: {
|
|
74
|
+
date_time_input_format: 'best_effort',
|
|
75
|
+
},
|
|
76
|
+
})
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
> JS `Date` objects do **not** work for the `Date` type (date-only) — pass
|
|
80
|
+
> `'YYYY-MM-DD'` strings for that.
|
|
81
|
+
|
|
82
|
+
## Inserting `Decimal*` values
|
|
83
|
+
|
|
84
|
+
Decimals must be passed as **strings** in JSON formats to avoid precision
|
|
85
|
+
loss in JavaScript:
|
|
86
|
+
|
|
87
|
+
```ts
|
|
88
|
+
await client.command({
|
|
89
|
+
query: `
|
|
90
|
+
CREATE OR REPLACE TABLE prices (
|
|
91
|
+
id UInt32,
|
|
92
|
+
dec32 Decimal(9, 2),
|
|
93
|
+
dec64 Decimal(18, 3),
|
|
94
|
+
dec128 Decimal(38, 10),
|
|
95
|
+
dec256 Decimal(76, 20)
|
|
96
|
+
)
|
|
97
|
+
ENGINE MergeTree ORDER BY id
|
|
98
|
+
`,
|
|
99
|
+
})
|
|
100
|
+
|
|
101
|
+
await client.insert({
|
|
102
|
+
table: 'prices',
|
|
103
|
+
format: 'JSONEachRow',
|
|
104
|
+
values: [
|
|
105
|
+
{
|
|
106
|
+
id: 1,
|
|
107
|
+
dec32: '1234567.89',
|
|
108
|
+
dec64: '123456789123456.789',
|
|
109
|
+
dec128: '1234567891234567891234567891.1234567891',
|
|
110
|
+
dec256:
|
|
111
|
+
'12345678912345678912345678911234567891234567891234567891.12345678911234567891',
|
|
112
|
+
},
|
|
113
|
+
],
|
|
114
|
+
})
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
When reading them back, cast to string in the SELECT to avoid the same
|
|
118
|
+
precision loss:
|
|
119
|
+
|
|
120
|
+
```ts
|
|
121
|
+
const rs = await client.query({
|
|
122
|
+
query: `
|
|
123
|
+
SELECT toString(dec64) AS decimal64,
|
|
124
|
+
toString(dec128) AS decimal128
|
|
125
|
+
FROM prices
|
|
126
|
+
`,
|
|
127
|
+
format: 'JSONEachRow',
|
|
128
|
+
})
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
## Common pitfalls
|
|
132
|
+
|
|
133
|
+
- **Passing decimals as JS `number`s.** Anything beyond `Number.MAX_SAFE_INTEGER`
|
|
134
|
+
silently loses precision before it ever reaches the server.
|
|
135
|
+
- **Using `client.insert()` for `INSERT … SELECT`.** There's nothing to
|
|
136
|
+
upload — use `client.command()` with the full SQL.
|
|
137
|
+
- **Forgetting `date_time_input_format: 'best_effort'`** when inserting
|
|
138
|
+
`Date` objects (or ISO strings). The default input format does not accept
|
|
139
|
+
ISO-8601 with the `T`/`Z` separators.
|
|
140
|
+
- **Hand-building `VALUES` with user input.** Always parameterize user data;
|
|
141
|
+
see `reference/query-parameters.md`.
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
# Ping the Server
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `ping()` returns a discriminated
|
|
4
|
+
> `PingResult = { success: true } | { success: false, error: Error }` —
|
|
5
|
+
> it does **not** throw on connection failures.
|
|
6
|
+
|
|
7
|
+
Backing examples:
|
|
8
|
+
[`examples/node/coding/ping_existing_host.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/ping_existing_host.ts),
|
|
9
|
+
[`examples/node/coding/ping_non_existing_host.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/ping_non_existing_host.ts).
|
|
10
|
+
|
|
11
|
+
## Successful ping
|
|
12
|
+
|
|
13
|
+
```ts
|
|
14
|
+
import { createClient } from '@clickhouse/client'
|
|
15
|
+
|
|
16
|
+
const client = createClient({
|
|
17
|
+
url: process.env.CLICKHOUSE_URL,
|
|
18
|
+
password: process.env.CLICKHOUSE_PASSWORD,
|
|
19
|
+
})
|
|
20
|
+
|
|
21
|
+
const pingResult = await client.ping()
|
|
22
|
+
if (pingResult.success) {
|
|
23
|
+
console.info('ClickHouse is reachable')
|
|
24
|
+
} else {
|
|
25
|
+
console.error('Ping failed:', pingResult.error)
|
|
26
|
+
}
|
|
27
|
+
await client.close()
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Use `ping()` to:
|
|
31
|
+
|
|
32
|
+
- Probe ClickHouse at application startup.
|
|
33
|
+
- Wake up a ClickHouse Cloud instance that may be idling (a ping is enough to
|
|
34
|
+
bring it out of sleep).
|
|
35
|
+
- Implement a `/healthz` / readiness endpoint.
|
|
36
|
+
|
|
37
|
+
## Failure: host unreachable
|
|
38
|
+
|
|
39
|
+
`ping()` does **not** throw — it resolves with
|
|
40
|
+
`{ success: false, error: Error }`, so you can branch without `try/catch`:
|
|
41
|
+
|
|
42
|
+
```ts
|
|
43
|
+
import type { PingResult } from '@clickhouse/client'
|
|
44
|
+
import { createClient } from '@clickhouse/client'
|
|
45
|
+
|
|
46
|
+
const client = createClient({
|
|
47
|
+
url: 'http://localhost:8100', // non-existing host
|
|
48
|
+
request_timeout: 50, // keep failure fast
|
|
49
|
+
})
|
|
50
|
+
|
|
51
|
+
const pingResult = await client.ping()
|
|
52
|
+
if (hasConnectionRefusedError(pingResult)) {
|
|
53
|
+
console.info('Connection refused, as expected')
|
|
54
|
+
} else {
|
|
55
|
+
console.error('Ping expected ECONNREFUSED, got:', pingResult)
|
|
56
|
+
}
|
|
57
|
+
await client.close()
|
|
58
|
+
|
|
59
|
+
function hasConnectionRefusedError(
|
|
60
|
+
pingResult: PingResult,
|
|
61
|
+
): pingResult is PingResult & { error: { code: 'ECONNREFUSED' } } {
|
|
62
|
+
return (
|
|
63
|
+
!pingResult.success &&
|
|
64
|
+
'code' in pingResult.error &&
|
|
65
|
+
pingResult.error.code === 'ECONNREFUSED'
|
|
66
|
+
)
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Mapping to an HTTP health endpoint
|
|
71
|
+
|
|
72
|
+
```ts
|
|
73
|
+
app.get('/healthz', async (_req, res) => {
|
|
74
|
+
const r = await client.ping()
|
|
75
|
+
if (r.success) {
|
|
76
|
+
res.status(200).json({ ok: true })
|
|
77
|
+
} else {
|
|
78
|
+
res.status(503).json({ ok: false, error: String(r.error) })
|
|
79
|
+
}
|
|
80
|
+
})
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
## `ping()` vs `ping({ select: true })`
|
|
84
|
+
|
|
85
|
+
The default `ping()` hits ClickHouse's `/ping` HTTP endpoint — it verifies
|
|
86
|
+
network connectivity but **does not check credentials or query processing**.
|
|
87
|
+
A server that is reachable but has a bad password (or a broken query
|
|
88
|
+
pipeline) will still return `{ success: true }` from a plain `ping()`.
|
|
89
|
+
|
|
90
|
+
Pass `{ select: true }` to run a lightweight `SELECT 1` instead:
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
const r = await client.ping({ select: true })
|
|
94
|
+
// success only if the server is reachable AND auth is correct AND it can run queries
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
| | `client.ping()` | `client.ping({ select: true })` |
|
|
98
|
+
| ----------------------- | --------------- | ------------------------------- |
|
|
99
|
+
| Endpoint | `/ping` (HTTP) | `SELECT 1` query |
|
|
100
|
+
| Checks auth | **No** | Yes |
|
|
101
|
+
| Checks query processing | No | **Yes** |
|
|
102
|
+
| Overhead | Minimal | Slightly higher |
|
|
103
|
+
|
|
104
|
+
**When to use which:**
|
|
105
|
+
|
|
106
|
+
- **Liveness probe** (is the process alive?) — plain `ping()` is fine.
|
|
107
|
+
- **Readiness probe** (can it serve traffic?) — use `ping({ select: true })`
|
|
108
|
+
so the probe fails if credentials are wrong or the query layer is broken.
|
|
109
|
+
- **Waking a ClickHouse Cloud idle instance** — plain `ping()` is enough.
|
|
110
|
+
|
|
111
|
+
## Common pitfalls
|
|
112
|
+
|
|
113
|
+
- **Do not wrap `ping()` in `try/catch` as your only check.** It resolves on
|
|
114
|
+
failure; the `success` boolean is the source of truth.
|
|
115
|
+
- **Lower `request_timeout` if you want pings to fail fast** (the example
|
|
116
|
+
above uses `50` ms). The default is high enough to be unsuitable for
|
|
117
|
+
liveness probes.
|
|
118
|
+
- **Plain `ping()` does not check credentials.** If auth is part of what you
|
|
119
|
+
want to verify, use `ping({ select: true })`.
|
|
120
|
+
- For ping that times out specifically, see the troubleshooting skill.
|
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# Query Parameter Binding
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. NULL parameter binding fixed in `0.0.16`.
|
|
4
|
+
> Special-character (tab/newline/quote/backslash) binding `>= 0.3.1`.
|
|
5
|
+
> `TupleParam` and JS `Map` parameters `>= 1.9.0`. Boolean formatting in
|
|
6
|
+
> `Array`/`Tuple`/`Map` parameters fixed in `>= 1.13.0`. `BigInt` query
|
|
7
|
+
> parameters `>= 1.15.0`.
|
|
8
|
+
|
|
9
|
+
Backing examples:
|
|
10
|
+
[`examples/node/coding/query_with_parameter_binding.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/query_with_parameter_binding.ts),
|
|
11
|
+
[`examples/node/coding/query_with_parameter_binding_special_chars.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/query_with_parameter_binding_special_chars.ts).
|
|
12
|
+
|
|
13
|
+
## Answer checklist
|
|
14
|
+
|
|
15
|
+
When the user passes user-controlled values into SQL:
|
|
16
|
+
|
|
17
|
+
- Use ClickHouse `{name: Type}` placeholders and a `query_params` object.
|
|
18
|
+
- Explicitly call template-literal/string interpolation of user input a
|
|
19
|
+
**SQL injection risk**.
|
|
20
|
+
- Do not suggest PostgreSQL/MySQL-style `$1`, `?`, or `:name` placeholders.
|
|
21
|
+
- Pick the placeholder type to match the ClickHouse column type (`String`,
|
|
22
|
+
`Date`, `DateTime`, `Nullable(T)`, etc.).
|
|
23
|
+
|
|
24
|
+
## Syntax: `{name: Type}`
|
|
25
|
+
|
|
26
|
+
ClickHouse uses `{name: Type}` placeholders — **not** `$1`, `?`, or `:name`.
|
|
27
|
+
|
|
28
|
+
```ts
|
|
29
|
+
await client.query({
|
|
30
|
+
query: 'SELECT plus({a: Int32}, {b: Int32})',
|
|
31
|
+
format: 'JSONEachRow',
|
|
32
|
+
query_params: { a: 10, b: 20 },
|
|
33
|
+
})
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The `Type` must be a valid ClickHouse type (`Int32`, `String`, `Date`,
|
|
37
|
+
`Array(UInt32)`, `Tuple(Int32, String)`, `Map(K, V)`, `Nullable(T)`, etc.).
|
|
38
|
+
|
|
39
|
+
## ⚠️ Never use template literals for user values
|
|
40
|
+
|
|
41
|
+
Interpolating user input into the SQL string bypasses server-side escaping
|
|
42
|
+
and opens the door to SQL injection:
|
|
43
|
+
|
|
44
|
+
```ts
|
|
45
|
+
// ❌ Dangerous — never do this with user-controlled values
|
|
46
|
+
const userId = req.params.id
|
|
47
|
+
await client.query({ query: `SELECT * FROM users WHERE id = ${userId}` })
|
|
48
|
+
|
|
49
|
+
// ✓ Safe — parameterized
|
|
50
|
+
await client.query({
|
|
51
|
+
query: 'SELECT * FROM users WHERE id = {id: UInt32}',
|
|
52
|
+
query_params: { id: userId },
|
|
53
|
+
})
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
This is the most common mistake for users coming from PostgreSQL/MySQL. Call
|
|
57
|
+
it out explicitly when the user shows template-literal interpolation.
|
|
58
|
+
|
|
59
|
+
## Common types
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
import { TupleParam } from '@clickhouse/client'
|
|
63
|
+
|
|
64
|
+
await client.query({
|
|
65
|
+
query: `
|
|
66
|
+
SELECT
|
|
67
|
+
{var_int: Int32} AS var_int,
|
|
68
|
+
{var_float: Float32} AS var_float,
|
|
69
|
+
{var_str: String} AS var_str,
|
|
70
|
+
{var_array: Array(Int32)} AS var_array,
|
|
71
|
+
{var_tuple: Tuple(Int32, String)} AS var_tuple,
|
|
72
|
+
{var_map: Map(Int, Array(String))} AS var_map,
|
|
73
|
+
{var_date: Date} AS var_date,
|
|
74
|
+
{var_datetime: DateTime} AS var_datetime,
|
|
75
|
+
{var_datetime64_3: DateTime64(3)} AS var_datetime64_3,
|
|
76
|
+
{var_datetime64_9: DateTime64(9)} AS var_datetime64_9,
|
|
77
|
+
{var_decimal: Decimal(9, 2)} AS var_decimal,
|
|
78
|
+
{var_uuid: UUID} AS var_uuid,
|
|
79
|
+
{var_ipv4: IPv4} AS var_ipv4,
|
|
80
|
+
{var_null: Nullable(String)} AS var_null
|
|
81
|
+
`,
|
|
82
|
+
format: 'JSONEachRow',
|
|
83
|
+
query_params: {
|
|
84
|
+
var_int: 10,
|
|
85
|
+
var_float: '10.557',
|
|
86
|
+
var_str: 'hello',
|
|
87
|
+
var_array: [42, 144],
|
|
88
|
+
var_tuple: new TupleParam([42, 'foo']), // >= 1.9.0
|
|
89
|
+
var_map: new Map([
|
|
90
|
+
[42, ['a', 'b']],
|
|
91
|
+
[144, ['c', 'd']],
|
|
92
|
+
]), // >= 1.9.0
|
|
93
|
+
var_date: '2022-01-01',
|
|
94
|
+
var_datetime: '2022-01-01 12:34:56', // or a Date
|
|
95
|
+
var_datetime64_3: '2022-01-01 12:34:56.789', // or a Date
|
|
96
|
+
var_datetime64_9: '2022-01-01 12:34:56.123456789', // string for ns precision
|
|
97
|
+
var_decimal: '123.45', // string to avoid precision loss
|
|
98
|
+
var_uuid: '01234567-89ab-cdef-0123-456789abcdef',
|
|
99
|
+
var_ipv4: '192.168.0.1',
|
|
100
|
+
var_null: null, // fixed in 0.0.16
|
|
101
|
+
},
|
|
102
|
+
})
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### Type-by-type tips
|
|
106
|
+
|
|
107
|
+
- **Decimals** — pass as strings to avoid JS number precision loss.
|
|
108
|
+
- **`DateTime64(>3)`** — pass as a string; JS `Date` only has millisecond
|
|
109
|
+
precision and will lose sub-millisecond digits.
|
|
110
|
+
- **`DateTime64`** — strings can also be UNIX timestamps, including
|
|
111
|
+
fractional ones (e.g., `'1651490755.123456789'`).
|
|
112
|
+
- **`BigInt`** — supported in `query_params` since `>= 1.15.0`. On older
|
|
113
|
+
clients, pass as a string.
|
|
114
|
+
- **`Tuple(...)`** — wrap in `new TupleParam([...])` (`>= 1.9.0`); on older
|
|
115
|
+
clients, build the literal manually as a string.
|
|
116
|
+
- **`Map(K, V)`** — pass a JS `Map` (`>= 1.9.0`); on older clients, build
|
|
117
|
+
it manually.
|
|
118
|
+
- **`Nullable(T)`** — pass `null` directly (`>= 0.0.16`).
|
|
119
|
+
|
|
120
|
+
## Special characters in string parameters (`>= 0.3.1`)
|
|
121
|
+
|
|
122
|
+
Tabs, newlines, carriage returns, single quotes, and backslashes are
|
|
123
|
+
escaped automatically by the client — just pass the JS string as-is:
|
|
124
|
+
|
|
125
|
+
```ts
|
|
126
|
+
await client.query({
|
|
127
|
+
query: `
|
|
128
|
+
SELECT
|
|
129
|
+
'foo_\t_bar' = {tab: String} AS has_tab,
|
|
130
|
+
'foo_\n_bar' = {newline: String} AS has_newline,
|
|
131
|
+
'foo_\\'_bar' = {single_quote: String} AS has_single_quote,
|
|
132
|
+
'foo_\\_bar' = {backslash: String} AS has_backslash
|
|
133
|
+
`,
|
|
134
|
+
format: 'JSONEachRow',
|
|
135
|
+
query_params: {
|
|
136
|
+
tab: 'foo_\t_bar',
|
|
137
|
+
newline: 'foo_\n_bar',
|
|
138
|
+
single_quote: "foo_'_bar",
|
|
139
|
+
backslash: 'foo_\\_bar',
|
|
140
|
+
},
|
|
141
|
+
})
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
## Common pitfalls
|
|
145
|
+
|
|
146
|
+
- **`$1` / `?` / `:name` placeholders.** None work — use `{name: Type}`.
|
|
147
|
+
- **Forgetting the type in the placeholder.** `{id}` is a syntax error;
|
|
148
|
+
it must be `{id: UInt32}`.
|
|
149
|
+
- **Stringifying tuples/maps manually on `>= 1.9.0`.** Use `TupleParam`
|
|
150
|
+
and `Map` — both serialize correctly and respect special characters.
|
|
151
|
+
- **Boolean array/tuple/map elements before `1.13.0`.** Boolean formatting
|
|
152
|
+
was fixed in 1.13.0 — earlier versions may misformat them.
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Select Data Formats
|
|
2
|
+
|
|
3
|
+
> **Applies to:** all versions. `JSONEachRowWithProgress` requires client
|
|
4
|
+
> `>= 1.7.0`; see the in-repo performance examples under
|
|
5
|
+
> `examples/node/performance/`.
|
|
6
|
+
|
|
7
|
+
Backing examples:
|
|
8
|
+
[`examples/node/coding/select_json_each_row.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/select_json_each_row.ts),
|
|
9
|
+
[`examples/node/coding/select_data_formats_overview.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/select_data_formats_overview.ts),
|
|
10
|
+
[`examples/node/coding/select_json_with_metadata.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/coding/select_json_with_metadata.ts).
|
|
11
|
+
|
|
12
|
+
## Default choice: `JSONEachRow` → `.json<T>()`
|
|
13
|
+
|
|
14
|
+
Right answer for ~90% of selects when the result fits in memory.
|
|
15
|
+
|
|
16
|
+
```ts
|
|
17
|
+
import { createClient } from '@clickhouse/client'
|
|
18
|
+
|
|
19
|
+
interface Row {
|
|
20
|
+
number: string
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
const client = createClient()
|
|
24
|
+
const rows = await client.query({
|
|
25
|
+
query: 'SELECT number FROM system.numbers LIMIT 5',
|
|
26
|
+
format: 'JSONEachRow',
|
|
27
|
+
})
|
|
28
|
+
const result = await rows.json<Row>() // Row[]
|
|
29
|
+
result.forEach((r) => console.log(r))
|
|
30
|
+
await client.close()
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
`UInt64`/`Int64` and other 64-bit integers are returned as **strings**
|
|
34
|
+
when `output_format_json_quote_64bit_integers=1`, to avoid JS precision
|
|
35
|
+
loss. If that setting is `0`, they may be returned as unquoted JSON
|
|
36
|
+
numbers instead. Note that in ClickHouse `>= 25.8`, this setting can
|
|
37
|
+
default to `0`; see the troubleshooting skill for ways to control that.
|
|
38
|
+
|
|
39
|
+
## Single-document `JSON` format with metadata
|
|
40
|
+
|
|
41
|
+
Use `JSON` (or `JSONCompact`) when you need ClickHouse's response envelope
|
|
42
|
+
(rows + meta + statistics + row count). Type the result with
|
|
43
|
+
`ResponseJSON<T>`:
|
|
44
|
+
|
|
45
|
+
```ts
|
|
46
|
+
import { createClient, type ResponseJSON } from '@clickhouse/client'
|
|
47
|
+
|
|
48
|
+
const client = createClient()
|
|
49
|
+
const rows = await client.query({
|
|
50
|
+
query: 'SELECT number FROM system.numbers LIMIT 2',
|
|
51
|
+
format: 'JSON',
|
|
52
|
+
})
|
|
53
|
+
const result = await rows.json<ResponseJSON<{ number: string }>>()
|
|
54
|
+
console.info(result.meta, result.data, result.rows, result.statistics)
|
|
55
|
+
await client.close()
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
> `JSON`, `JSONCompact`, `JSONStrings`, `JSONCompactStrings`,
|
|
59
|
+
> `JSONColumnsWithMetadata`, `JSONObjectEachRow` are **single-document**
|
|
60
|
+
> formats — they cannot be streamed. Use a `*EachRow` variant if you want
|
|
61
|
+
> to stream.
|
|
62
|
+
|
|
63
|
+
## Selecting raw text (CSV / TSV / CustomSeparated)
|
|
64
|
+
|
|
65
|
+
Use `.text()` (not `.json()`) for raw textual formats:
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
const rs = await client.query({
|
|
69
|
+
query: 'SELECT number, number * 2 AS doubled FROM system.numbers LIMIT 3',
|
|
70
|
+
format: 'CSVWithNames',
|
|
71
|
+
})
|
|
72
|
+
console.log(await rs.text())
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Streaming raw text/Parquet line-by-line belongs in
|
|
76
|
+
[`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)
|
|
77
|
+
— in particular, Parquet exports use `client.exec()` and pipe the raw
|
|
78
|
+
response stream rather than `ResultSet.stream()` (see
|
|
79
|
+
[`select_parquet_as_file.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/performance/select_parquet_as_file.ts)).
|
|
80
|
+
|
|
81
|
+
## Format chooser
|
|
82
|
+
|
|
83
|
+
| Use case | Format |
|
|
84
|
+
| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
85
|
+
| Read rows as JS objects | `JSONEachRow` _(default)_ |
|
|
86
|
+
| Read rows as positional tuples (smaller payload) | `JSONCompactEachRow` |
|
|
87
|
+
| Need `meta` / `statistics` / `rows` envelope | `JSON` or `JSONCompact` + `ResponseJSON<T>` |
|
|
88
|
+
| Read all values as strings (avoid number-precision loss) | `JSONStringsEachRow` / `JSONCompactStringsEachRow` |
|
|
89
|
+
| Stream very large result | `JSONEachRow` / `JSONCompactEachRow` (see [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)) |
|
|
90
|
+
| Export to CSV/TSV/Parquet | `CSV*`, `TabSeparated*`, `Parquet` (see [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)) |
|
|
91
|
+
|
|
92
|
+
## ResultSet methods
|
|
93
|
+
|
|
94
|
+
| Method | Returns | Notes |
|
|
95
|
+
| -------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
96
|
+
| `await rs.json<T>()` | `T[]` for `*EachRow`, single-doc shape otherwise | Buffers the full response |
|
|
97
|
+
| `await rs.text()` | `string` | Buffers the full response — for textual formats only (CSV/TSV/etc.) |
|
|
98
|
+
| `rs.stream()` | Node `Readable` of `Row[]` chunks | Use for large newline-delimited results (`JSONEachRow`/`JSONCompactEachRow`/`CSV`/`TSV`); **not** suitable for binary formats like `Parquet` — for those, use `client.exec()` and pipe the raw response stream (see [`examples/node/performance/`](https://github.com/ClickHouse/clickhouse-js/tree/main/examples/node/performance)) |
|
|
99
|
+
| `rs.close()` | `void` (synchronous) | Always call if you obtained `stream()` and stop reading early |
|
|
100
|
+
|
|
101
|
+
## Common pitfalls
|
|
102
|
+
|
|
103
|
+
- **Calling `.json()` on a `JSON` (single-doc) result and expecting an
|
|
104
|
+
array.** You get a `ResponseJSON<T>` object; the rows are under
|
|
105
|
+
`.data`. Use `JSONEachRow` if you want a flat array.
|
|
106
|
+
- **Leaving a `stream()` half-consumed.** This is a top cause of
|
|
107
|
+
`ECONNRESET` on the _next_ request — fully iterate the stream or call
|
|
108
|
+
`resultSet.close()` (synchronous — no `await`). (Diagnosis details live in the
|
|
109
|
+
troubleshooting skill.)
|
|
110
|
+
- **Reaching for `.json()` on a CSV/TSV result.** Use `.text()` (or
|
|
111
|
+
`.stream()` for large results).
|