@elyracode/db-tools 0.9.58 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +17 -0
- package/README.md +6 -5
- package/extensions/index.ts +549 -290
- package/package.json +9 -3
- package/test/clickhouse-connect.test.ts +0 -48
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,22 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [1.0.1] - 2026-11-22
|
|
4
|
+
|
|
5
|
+
### Changed
|
|
6
|
+
|
|
7
|
+
- Peer dependency on `@elyracode/coding-agent` is now `>=1.0.0`, the first version with the APIs this package uses
|
|
8
|
+
|
|
9
|
+
## [1.0.0] - 2026-11-22
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
- `query_sqlite` and SQLite schema discovery built `sqlite3` shell command strings, so `$(...)` or backticks in the SQL, the `table` name or the database path (including `DB_DATABASE` from a repository `.env`) ran on the host. `sqlite3` is now spawned without a shell with an argument array, always with `-safe` (blocks `writefile()`, `readfile()`, `edit()`, `load_extension()`, `ATTACH` and file/shell dot-commands) and with `-readonly` unless `ELYRA_DB_ALLOW_WRITES` is set. Dot-commands are rejected. sqlite3 3.37 or newer is now required; older versions fail with an explicit upgrade message
|
|
13
|
+
- The read-only check only looked at the first keyword, so `SELECT 1; DROP TABLE x` and `WITH a AS (SELECT 1) DELETE FROM u` passed. It now scans the statement outside string literals and comments (per-dialect quoting, MySQL `#` and `/*! */`, ClickHouse nested comments and heredocs), rejects multiple statements, data-modifying keywords anywhere (INSERT, UPDATE, DELETE, DROP, ALTER, CREATE, TRUNCATE, ATTACH, PRAGMA, INTO, ...), SQLite file/extension functions and MySQL `LOAD_FILE()`
|
|
14
|
+
- Read-only mode is now also enforced by the engines: MySQL queries run after `SET SESSION TRANSACTION READ ONLY`, ClickHouse queries and schema lookups run with `readonly=1` (which also disables table functions such as `url()` and `file()`)
|
|
15
|
+
- `get_database_schema` interpolated the `table` argument into `SHOW INDEX FROM`, `DESCRIBE TABLE` and SQLite `PRAGMA` statements, and the database name into the ClickHouse `system.tables` query. Table names are now validated (`name` or `schema.name`, letters, digits, `_`, `$`) and quoted; the ClickHouse database name is passed as a query parameter
|
|
16
|
+
- `get_database_schema` with `engine: "sqlite"` used `database` as the file path, so passing a database name ignored the auto-detected file. SQLite now takes the file from the new `db_path` parameter (or auto-detection) and treats `database` as the schema name (default `main`)
|
|
17
|
+
- Tool errors (not configured, blocked query, driver errors) are now thrown so they are reported as tool errors instead of successful results
|
|
18
|
+
- The project `.env` was copied into `process.env` when the extension loaded, in every project, so a repository could set `DB_HOST` and receive `ELYRA_MYSQL_PASSWORD` from your environment, and every command the agent ran inherited the repository's database credentials. The `.env` is now read when a tool runs, only in trusted projects, and never copied into the environment. Each engine takes all its settings from one source: the environment when it sets `ELYRA_MYSQL_HOST`/`ELYRA_CLICKHOUSE_HOST`, otherwise the project `.env`. SQLite auto-detection from `DB_DATABASE` in `.env` also requires a trusted project
|
|
19
|
+
|
|
3
20
|
## [0.9.58] - 2026-11-18
|
|
4
21
|
|
|
5
22
|
## [0.9.57] - 2026-11-10
|
package/README.md
CHANGED
|
@@ -12,7 +12,7 @@ elyra install npm:@elyracode/db-tools
|
|
|
12
12
|
|
|
13
13
|
### Automatic: .env file (recommended)
|
|
14
14
|
|
|
15
|
-
If your project has a `.env` file (e.g. Laravel), db-tools reads it
|
|
15
|
+
If your project has a `.env` file (e.g. Laravel), db-tools reads it when a tool runs, in [trusted projects](../coding-agent/docs/project-trust.md) only. It is never copied into the environment, so commands the agent runs do not see it. Standard key names are mapped:
|
|
16
16
|
|
|
17
17
|
| .env key | Maps to | Used by |
|
|
18
18
|
|----------|---------|--------|
|
|
@@ -34,7 +34,7 @@ For a typical Laravel project, your existing `.env` already has `DB_HOST`, `DB_D
|
|
|
34
34
|
|
|
35
35
|
### Manual: environment variables
|
|
36
36
|
|
|
37
|
-
You can also set `ELYRA_*` env vars directly.
|
|
37
|
+
You can also set `ELYRA_*` env vars directly. Each engine takes its settings from one source: if `ELYRA_MYSQL_HOST` is set in the environment, every MySQL setting comes from the environment and the `.env` is ignored for MySQL (likewise `ELYRA_CLICKHOUSE_HOST` for ClickHouse). Values are never mixed, so a repository's `.env` cannot point your password at its own host.
|
|
38
38
|
|
|
39
39
|
```
|
|
40
40
|
# MySQL
|
|
@@ -67,15 +67,16 @@ export ELYRA_CLICKHOUSE_TIMEOUT_QUERY=30000
|
|
|
67
67
|
### SQLite
|
|
68
68
|
|
|
69
69
|
SQLite is auto-detected from your project:
|
|
70
|
-
1. If `.env` has `DB_CONNECTION=sqlite`, uses `DB_DATABASE` path
|
|
70
|
+
1. If `.env` has `DB_CONNECTION=sqlite`, uses `DB_DATABASE` path (trusted projects only)
|
|
71
71
|
2. Falls back to `database/database.sqlite` (Laravel default)
|
|
72
72
|
3. Or pass an explicit path: the agent uses `db_path` parameter
|
|
73
73
|
|
|
74
|
-
No configuration needed for most Laravel projects.
|
|
74
|
+
No configuration needed for most Laravel projects. Requires the `sqlite3` CLI, version 3.37 or newer (for `-safe`).
|
|
75
75
|
|
|
76
76
|
## Security
|
|
77
77
|
|
|
78
|
-
- **Read-only by default**: Only SELECT, SHOW, DESCRIBE, and
|
|
78
|
+
- **Read-only by default**: Only single SELECT, SHOW, DESCRIBE, EXPLAIN, and WITH ... SELECT statements are allowed. The tool rejects stacked statements and data-modifying keywords, and the engines enforce it as well: MySQL sessions run with `SET SESSION TRANSACTION READ ONLY`, ClickHouse queries with `readonly=1` (table functions such as `url()` and `file()` are disabled), and `sqlite3` runs with `-readonly`.
|
|
79
|
+
- **SQLite safe mode**: `sqlite3` always runs with `-safe`, also when writes are enabled, so `writefile()`, `readfile()`, `load_extension()`, `ATTACH`, and file/shell dot-commands are refused.
|
|
79
80
|
- **Write access**: Set `ELYRA_DB_ALLOW_WRITES=true` to enable INSERT, UPDATE, DELETE.
|
|
80
81
|
- **Recommendation**: Use a database user with minimal privileges (SELECT only).
|
|
81
82
|
|
package/extensions/index.ts
CHANGED
|
@@ -1,7 +1,9 @@
|
|
|
1
|
+
import { execFile } from "node:child_process";
|
|
1
2
|
import { existsSync, readFileSync } from "node:fs";
|
|
2
|
-
import { join } from "node:path";
|
|
3
|
+
import { isAbsolute, join } from "node:path";
|
|
4
|
+
import { promisify } from "node:util";
|
|
3
5
|
import type * as ClickHouse from "@clickhouse/client";
|
|
4
|
-
import type { AgentToolResult, ExtensionAPI } from "@elyracode/coding-agent";
|
|
6
|
+
import type { AgentToolResult, ExtensionAPI, ExtensionContext } from "@elyracode/coding-agent";
|
|
5
7
|
import type * as MySql from "mysql2/promise";
|
|
6
8
|
import { Type } from "typebox";
|
|
7
9
|
|
|
@@ -15,11 +17,27 @@ let clickhouseDriver: Promise<typeof ClickHouse> | undefined;
|
|
|
15
17
|
const loadMysql = (): Promise<typeof MySql> => (mysqlDriver ??= import("mysql2/promise"));
|
|
16
18
|
const loadClickHouse = (): Promise<typeof ClickHouse> => (clickhouseDriver ??= import("@clickhouse/client"));
|
|
17
19
|
|
|
20
|
+
const execFileAsync = promisify(execFile);
|
|
21
|
+
|
|
18
22
|
/** Details reported by the query tools. */
|
|
19
23
|
interface QueryDetails {
|
|
20
24
|
rowCount?: number;
|
|
21
25
|
}
|
|
22
26
|
|
|
27
|
+
type SchemaResult = AgentToolResult<Record<string, unknown>>;
|
|
28
|
+
|
|
29
|
+
const MAX_RESULT_CHARS = 50000;
|
|
30
|
+
|
|
31
|
+
function truncate(text: string): string {
|
|
32
|
+
return text.length > MAX_RESULT_CHARS
|
|
33
|
+
? `${text.slice(0, MAX_RESULT_CHARS)}\n\n... (truncated, ${text.length} chars total)`
|
|
34
|
+
: text;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
function errorMessage(error: unknown): string {
|
|
38
|
+
return error instanceof Error ? error.message : String(error);
|
|
39
|
+
}
|
|
40
|
+
|
|
23
41
|
/**
|
|
24
42
|
* Create a ClickHouse client and make sure the server answers within `timeoutConnect`.
|
|
25
43
|
* The client has no connect-timeout option of its own (only `request_timeout` for the whole
|
|
@@ -64,18 +82,36 @@ async function openClickHouse(config: ClickHouseConfig): Promise<ClickHouse.Clic
|
|
|
64
82
|
}
|
|
65
83
|
}
|
|
66
84
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
85
|
+
async function openMysql(config: MysqlConfig): Promise<MySql.Connection> {
|
|
86
|
+
return (await loadMysql()).createConnection({
|
|
87
|
+
host: config.host,
|
|
88
|
+
port: config.port,
|
|
89
|
+
user: config.user,
|
|
90
|
+
password: config.password,
|
|
91
|
+
database: config.database,
|
|
92
|
+
connectTimeout: 10000,
|
|
93
|
+
});
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
/**
|
|
97
|
+
* ClickHouse settings for read-only mode. `readonly=1` makes the server reject writes, DDL,
|
|
98
|
+
* setting changes and table functions (url(), file(), s3(), remote(), ...) for this request.
|
|
99
|
+
*/
|
|
100
|
+
const CLICKHOUSE_READONLY_SETTINGS: ClickHouse.ClickHouseSettings = { readonly: "1" };
|
|
73
101
|
|
|
102
|
+
function blockedMessage(reason: string): string {
|
|
103
|
+
return `Blocked: ${reason}. Only single read-only statements (SELECT, SHOW, DESCRIBE, EXPLAIN, WITH ... SELECT) are allowed in read-only mode. Set ELYRA_DB_ALLOW_WRITES=true to enable writes.`;
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
export default function (elyra: ExtensionAPI) {
|
|
74
107
|
// ── Configuration ──
|
|
75
|
-
// Connection config is resolved
|
|
76
|
-
//
|
|
77
|
-
//
|
|
108
|
+
// Connection config is resolved on every call (see resolveConnectionEnv), per engine, from
|
|
109
|
+
// exactly one source:
|
|
110
|
+
// 1. ELYRA_MYSQL_* / ELYRA_CLICKHOUSE_* environment variables, when the engine's host is set there
|
|
111
|
+
// 2. Otherwise standard names from the project's .env, only in trusted projects:
|
|
112
|
+
// DB_HOST, DB_PORT, DB_USERNAME, DB_PASSWORD, DB_DATABASE (MySQL)
|
|
78
113
|
// CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_DATABASE, CLICKHOUSE_USERNAME, etc. (ClickHouse)
|
|
114
|
+
// The project's .env is never copied into process.env, so commands the agent runs do not inherit it.
|
|
79
115
|
|
|
80
116
|
// ── Tool: query_mysql ──
|
|
81
117
|
elyra.registerTool({
|
|
@@ -84,70 +120,40 @@ export default function (elyra: ExtensionAPI) {
|
|
|
84
120
|
description:
|
|
85
121
|
"Execute a SQL query against the configured MySQL database. Returns results as JSON. " +
|
|
86
122
|
"Use get_database_schema first to understand the database structure. " +
|
|
87
|
-
"By default, only SELECT queries are allowed (set ELYRA_DB_ALLOW_WRITES=true to allow writes).",
|
|
123
|
+
"By default, only single SELECT queries are allowed (set ELYRA_DB_ALLOW_WRITES=true to allow writes).",
|
|
88
124
|
parameters: Type.Object({
|
|
89
125
|
sql: Type.String({ description: "The SQL query to execute" }),
|
|
90
126
|
database: Type.Optional(Type.String({ description: "Override the default database (optional)" })),
|
|
91
127
|
}),
|
|
92
|
-
execute: async (_toolCallId, params): Promise<AgentToolResult<QueryDetails>> => {
|
|
93
|
-
const config = getMysqlConfig(params.database);
|
|
94
|
-
if (!config)
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
},
|
|
101
|
-
],
|
|
102
|
-
details: {},
|
|
103
|
-
};
|
|
104
|
-
}
|
|
105
|
-
|
|
106
|
-
// Security: block non-SELECT queries unless ELYRA_DB_ALLOW_WRITES=true
|
|
107
|
-
if (!isReadOnly(params.sql) && !isWriteEnabled()) {
|
|
108
|
-
return {
|
|
109
|
-
content: [
|
|
110
|
-
{
|
|
111
|
-
type: "text",
|
|
112
|
-
text: `Blocked: Only SELECT queries are allowed in read-only mode. Set ELYRA_DB_ALLOW_WRITES=true to enable writes.\n\nQuery: ${params.sql}`,
|
|
113
|
-
},
|
|
114
|
-
],
|
|
115
|
-
details: {},
|
|
116
|
-
};
|
|
128
|
+
execute: async (_toolCallId, params, _signal, _onUpdate, ctx): Promise<AgentToolResult<QueryDetails>> => {
|
|
129
|
+
const config = getMysqlConfig(getConnectionEnv(ctx), params.database);
|
|
130
|
+
if (!config) throw new Error(MYSQL_NOT_CONFIGURED);
|
|
131
|
+
|
|
132
|
+
const readOnly = !isWriteEnabled();
|
|
133
|
+
if (readOnly) {
|
|
134
|
+
const violation = readOnlyViolation(params.sql, "mysql");
|
|
135
|
+
if (violation) throw new Error(blockedMessage(violation));
|
|
117
136
|
}
|
|
118
137
|
|
|
138
|
+
let rows: unknown;
|
|
119
139
|
try {
|
|
120
|
-
const connection = await (
|
|
121
|
-
host: config.host,
|
|
122
|
-
port: config.port,
|
|
123
|
-
user: config.user,
|
|
124
|
-
password: config.password,
|
|
125
|
-
database: config.database,
|
|
126
|
-
connectTimeout: 10000,
|
|
127
|
-
});
|
|
128
|
-
|
|
140
|
+
const connection = await openMysql(config);
|
|
129
141
|
try {
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
? `${resultText.slice(0, 50000)}\n\n... (truncated, ${resultText.length} chars total)`
|
|
135
|
-
: resultText;
|
|
136
|
-
|
|
137
|
-
return {
|
|
138
|
-
content: [{ type: "text", text: truncated }],
|
|
139
|
-
details: { rowCount: Array.isArray(rows) ? rows.length : 0 },
|
|
140
|
-
};
|
|
142
|
+
// Engine-level guard: the server rejects writes and DDL for every statement on
|
|
143
|
+
// this session. mysql2 sends one statement per execute() (multipleStatements is off).
|
|
144
|
+
if (readOnly) await connection.query("SET SESSION TRANSACTION READ ONLY");
|
|
145
|
+
[rows] = await connection.execute(params.sql);
|
|
141
146
|
} finally {
|
|
142
147
|
await connection.end();
|
|
143
148
|
}
|
|
144
149
|
} catch (error) {
|
|
145
|
-
|
|
146
|
-
return {
|
|
147
|
-
content: [{ type: "text", text: `MySQL error: ${msg}` }],
|
|
148
|
-
details: {},
|
|
149
|
-
};
|
|
150
|
+
throw new Error(`MySQL error: ${errorMessage(error)}`);
|
|
150
151
|
}
|
|
152
|
+
|
|
153
|
+
return {
|
|
154
|
+
content: [{ type: "text", text: truncate(JSON.stringify(rows, null, 2)) }],
|
|
155
|
+
details: { rowCount: Array.isArray(rows) ? rows.length : 0 },
|
|
156
|
+
};
|
|
151
157
|
},
|
|
152
158
|
});
|
|
153
159
|
|
|
@@ -158,67 +164,43 @@ export default function (elyra: ExtensionAPI) {
|
|
|
158
164
|
description:
|
|
159
165
|
"Execute a SQL query against the configured ClickHouse database. Returns results as JSON. " +
|
|
160
166
|
"ClickHouse is optimized for analytical queries on large datasets. " +
|
|
161
|
-
"Use get_database_schema first to understand the database structure. By default, only SELECT queries are allowed
|
|
167
|
+
"Use get_database_schema first to understand the database structure. By default, only single SELECT queries are allowed, " +
|
|
168
|
+
"and table functions are disabled.",
|
|
162
169
|
parameters: Type.Object({
|
|
163
170
|
sql: Type.String({ description: "The SQL query to execute" }),
|
|
164
171
|
database: Type.Optional(Type.String({ description: "Override the default database (optional)" })),
|
|
165
172
|
}),
|
|
166
|
-
execute: async (_toolCallId, params): Promise<AgentToolResult<QueryDetails>> => {
|
|
167
|
-
const config = getClickHouseConfig(params.database);
|
|
168
|
-
if (!config)
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
},
|
|
175
|
-
],
|
|
176
|
-
details: {},
|
|
177
|
-
};
|
|
178
|
-
}
|
|
179
|
-
|
|
180
|
-
// Security: block non-SELECT queries unless writes enabled
|
|
181
|
-
if (!isReadOnly(params.sql) && !isWriteEnabled()) {
|
|
182
|
-
return {
|
|
183
|
-
content: [
|
|
184
|
-
{
|
|
185
|
-
type: "text",
|
|
186
|
-
text: `Blocked: Only SELECT queries are allowed in read-only mode. Set ELYRA_DB_ALLOW_WRITES=true to enable writes.\n\nQuery: ${params.sql}`,
|
|
187
|
-
},
|
|
188
|
-
],
|
|
189
|
-
details: {},
|
|
190
|
-
};
|
|
173
|
+
execute: async (_toolCallId, params, _signal, _onUpdate, ctx): Promise<AgentToolResult<QueryDetails>> => {
|
|
174
|
+
const config = getClickHouseConfig(getConnectionEnv(ctx), params.database);
|
|
175
|
+
if (!config) throw new Error(CLICKHOUSE_NOT_CONFIGURED);
|
|
176
|
+
|
|
177
|
+
const readOnly = !isWriteEnabled();
|
|
178
|
+
if (readOnly) {
|
|
179
|
+
const violation = readOnlyViolation(params.sql, "clickhouse");
|
|
180
|
+
if (violation) throw new Error(blockedMessage(violation));
|
|
191
181
|
}
|
|
192
182
|
|
|
183
|
+
let rows: unknown;
|
|
193
184
|
try {
|
|
194
185
|
const client = await openClickHouse(config);
|
|
195
|
-
|
|
196
186
|
try {
|
|
197
187
|
const result = await client.query({
|
|
198
188
|
query: params.sql,
|
|
199
189
|
format: "JSONEachRow",
|
|
190
|
+
clickhouse_settings: readOnly ? CLICKHOUSE_READONLY_SETTINGS : undefined,
|
|
200
191
|
});
|
|
201
|
-
|
|
202
|
-
const resultText = JSON.stringify(rows, null, 2);
|
|
203
|
-
const truncated =
|
|
204
|
-
resultText.length > 50000
|
|
205
|
-
? `${resultText.slice(0, 50000)}\n\n... (truncated, ${resultText.length} chars total)`
|
|
206
|
-
: resultText;
|
|
207
|
-
|
|
208
|
-
return {
|
|
209
|
-
content: [{ type: "text", text: truncated }],
|
|
210
|
-
details: { rowCount: Array.isArray(rows) ? rows.length : 0 },
|
|
211
|
-
};
|
|
192
|
+
rows = await result.json();
|
|
212
193
|
} finally {
|
|
213
194
|
await client.close();
|
|
214
195
|
}
|
|
215
196
|
} catch (error) {
|
|
216
|
-
|
|
217
|
-
return {
|
|
218
|
-
content: [{ type: "text", text: `ClickHouse error: ${msg}` }],
|
|
219
|
-
details: {},
|
|
220
|
-
};
|
|
197
|
+
throw new Error(`ClickHouse error: ${errorMessage(error)}`);
|
|
221
198
|
}
|
|
199
|
+
|
|
200
|
+
return {
|
|
201
|
+
content: [{ type: "text", text: truncate(JSON.stringify(rows, null, 2)) }],
|
|
202
|
+
details: { rowCount: Array.isArray(rows) ? rows.length : 0 },
|
|
203
|
+
};
|
|
222
204
|
},
|
|
223
205
|
});
|
|
224
206
|
|
|
@@ -229,8 +211,8 @@ export default function (elyra: ExtensionAPI) {
|
|
|
229
211
|
description:
|
|
230
212
|
"Execute a SQL query against an SQLite database file. Returns results as JSON. " +
|
|
231
213
|
"Use get_database_schema first to understand the database structure. " +
|
|
232
|
-
"By default, only SELECT queries are allowed. " +
|
|
233
|
-
"Auto-detects Laravel SQLite databases from .env (DB_CONNECTION=sqlite).",
|
|
214
|
+
"By default, only single SELECT queries are allowed. " +
|
|
215
|
+
"Auto-detects Laravel SQLite databases from .env (DB_CONNECTION=sqlite). Requires the sqlite3 CLI (>= 3.37).",
|
|
234
216
|
parameters: Type.Object({
|
|
235
217
|
sql: Type.String({ description: "The SQL query to execute" }),
|
|
236
218
|
db_path: Type.Optional(
|
|
@@ -240,64 +222,32 @@ export default function (elyra: ExtensionAPI) {
|
|
|
240
222
|
}),
|
|
241
223
|
),
|
|
242
224
|
}),
|
|
243
|
-
execute: async (_toolCallId, params): Promise<AgentToolResult<QueryDetails>> => {
|
|
244
|
-
const dbPath = resolveSqlitePath(params.db_path);
|
|
225
|
+
execute: async (_toolCallId, params, signal, _onUpdate, ctx): Promise<AgentToolResult<QueryDetails>> => {
|
|
226
|
+
const dbPath = resolveSqlitePath(ctx, params.db_path);
|
|
245
227
|
if (!dbPath) {
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
type: "text",
|
|
250
|
-
text: "SQLite database not found. Provide db_path or set DB_CONNECTION=sqlite in .env.",
|
|
251
|
-
},
|
|
252
|
-
],
|
|
253
|
-
details: {},
|
|
254
|
-
};
|
|
228
|
+
throw new Error(
|
|
229
|
+
"SQLite database not found. Provide db_path, or set DB_CONNECTION=sqlite in the project's .env (read only in trusted projects).",
|
|
230
|
+
);
|
|
255
231
|
}
|
|
256
232
|
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
type: "text",
|
|
262
|
-
text: `Blocked: Only SELECT queries are allowed in read-only mode. Set ELYRA_DB_ALLOW_WRITES=true to enable writes.\n\nQuery: ${params.sql}`,
|
|
263
|
-
},
|
|
264
|
-
],
|
|
265
|
-
details: {},
|
|
266
|
-
};
|
|
233
|
+
const readOnly = !isWriteEnabled();
|
|
234
|
+
if (readOnly) {
|
|
235
|
+
const violation = readOnlyViolation(params.sql, "sqlite");
|
|
236
|
+
if (violation) throw new Error(blockedMessage(violation));
|
|
267
237
|
}
|
|
268
238
|
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
const result = execSync(`sqlite3 -json "${dbPath}" "${params.sql.replace(/"/g, '\\"')}"`, {
|
|
272
|
-
timeout: 30000,
|
|
273
|
-
encoding: "utf-8",
|
|
274
|
-
stdio: ["pipe", "pipe", "pipe"],
|
|
275
|
-
});
|
|
276
|
-
|
|
277
|
-
const trimmed = result.trim();
|
|
278
|
-
if (!trimmed) {
|
|
279
|
-
return {
|
|
280
|
-
content: [{ type: "text", text: "Query executed successfully (no results)." }],
|
|
281
|
-
details: { rowCount: 0 },
|
|
282
|
-
};
|
|
283
|
-
}
|
|
284
|
-
|
|
285
|
-
const truncated =
|
|
286
|
-
trimmed.length > 50000
|
|
287
|
-
? `${trimmed.slice(0, 50000)}\n\n... (truncated, ${trimmed.length} chars total)`
|
|
288
|
-
: trimmed;
|
|
289
|
-
|
|
290
|
-
return {
|
|
291
|
-
content: [{ type: "text", text: truncated }],
|
|
292
|
-
details: {},
|
|
293
|
-
};
|
|
294
|
-
} catch (error) {
|
|
295
|
-
const msg = error instanceof Error ? error.message : String(error);
|
|
239
|
+
const trimmed = (await runSqlite(dbPath, params.sql, { readOnly, timeout: 30000, signal })).trim();
|
|
240
|
+
if (!trimmed) {
|
|
296
241
|
return {
|
|
297
|
-
content: [{ type: "text", text:
|
|
298
|
-
details: {},
|
|
242
|
+
content: [{ type: "text", text: "Query executed successfully (no results)." }],
|
|
243
|
+
details: { rowCount: 0 },
|
|
299
244
|
};
|
|
300
245
|
}
|
|
246
|
+
|
|
247
|
+
return {
|
|
248
|
+
content: [{ type: "text", text: truncate(trimmed) }],
|
|
249
|
+
details: {},
|
|
250
|
+
};
|
|
301
251
|
},
|
|
302
252
|
});
|
|
303
253
|
|
|
@@ -313,21 +263,32 @@ export default function (elyra: ExtensionAPI) {
|
|
|
313
263
|
engine: Type.Union([Type.Literal("mysql"), Type.Literal("clickhouse"), Type.Literal("sqlite")], {
|
|
314
264
|
description: "Which database engine to query",
|
|
315
265
|
}),
|
|
316
|
-
database: Type.Optional(
|
|
266
|
+
database: Type.Optional(
|
|
267
|
+
Type.String({
|
|
268
|
+
description:
|
|
269
|
+
"MySQL/ClickHouse: override the default database. SQLite: schema name inside the file (default: main). Optional.",
|
|
270
|
+
}),
|
|
271
|
+
),
|
|
272
|
+
db_path: Type.Optional(
|
|
273
|
+
Type.String({
|
|
274
|
+
description:
|
|
275
|
+
"SQLite only: path to the database file (default: auto-detect from .env or database/database.sqlite)",
|
|
276
|
+
}),
|
|
277
|
+
),
|
|
317
278
|
table: Type.Optional(
|
|
318
279
|
Type.String({
|
|
319
280
|
description: "Get detailed schema for a specific table (optional, omit for overview of all tables)",
|
|
320
281
|
}),
|
|
321
282
|
),
|
|
322
283
|
}),
|
|
323
|
-
execute: async (_toolCallId, params) => {
|
|
284
|
+
execute: async (_toolCallId, params, signal, _onUpdate, ctx): Promise<SchemaResult> => {
|
|
324
285
|
if (params.engine === "mysql") {
|
|
325
|
-
return getMysqlSchema(params.database, params.table);
|
|
286
|
+
return getMysqlSchema(getConnectionEnv(ctx), params.database, params.table);
|
|
326
287
|
}
|
|
327
288
|
if (params.engine === "sqlite") {
|
|
328
|
-
return getSqliteSchema(params.database, params.table);
|
|
289
|
+
return getSqliteSchema(ctx, params.db_path, params.database, params.table, signal);
|
|
329
290
|
}
|
|
330
|
-
return getClickHouseSchema(params.database, params.table);
|
|
291
|
+
return getClickHouseSchema(getConnectionEnv(ctx), params.database, params.table);
|
|
331
292
|
},
|
|
332
293
|
});
|
|
333
294
|
}
|
|
@@ -353,35 +314,45 @@ interface ClickHouseConfig {
|
|
|
353
314
|
timeoutQuery: number;
|
|
354
315
|
}
|
|
355
316
|
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
317
|
+
const MYSQL_NOT_CONFIGURED =
|
|
318
|
+
"MySQL not configured. Set ELYRA_MYSQL_HOST, ELYRA_MYSQL_USER, ELYRA_MYSQL_PASSWORD, and ELYRA_MYSQL_DATABASE in the environment, " +
|
|
319
|
+
"or DB_HOST, DB_USERNAME, DB_PASSWORD, and DB_DATABASE in the project's .env (read only in trusted projects; run /trust).";
|
|
320
|
+
const CLICKHOUSE_NOT_CONFIGURED =
|
|
321
|
+
"ClickHouse not configured. Set ELYRA_CLICKHOUSE_HOST and ELYRA_CLICKHOUSE_DATABASE in the environment, " +
|
|
322
|
+
"or CLICKHOUSE_HOST and CLICKHOUSE_DATABASE in the project's .env (read only in trusted projects; run /trust).";
|
|
323
|
+
|
|
324
|
+
/** Connection variables (ELYRA_MYSQL_*, ELYRA_CLICKHOUSE_*) for one tool call. */
|
|
325
|
+
type ConnectionEnv = Readonly<Record<string, string | undefined>>;
|
|
326
|
+
|
|
327
|
+
function getMysqlConfig(env: ConnectionEnv, databaseOverride?: string): MysqlConfig | undefined {
|
|
328
|
+
const host = env.ELYRA_MYSQL_HOST;
|
|
329
|
+
const user = env.ELYRA_MYSQL_USER;
|
|
330
|
+
const password = env.ELYRA_MYSQL_PASSWORD;
|
|
331
|
+
const database = databaseOverride || env.ELYRA_MYSQL_DATABASE;
|
|
361
332
|
if (!host || !user || !database) return undefined;
|
|
362
333
|
return {
|
|
363
334
|
host,
|
|
364
|
-
port: Number.parseInt(
|
|
335
|
+
port: Number.parseInt(env.ELYRA_MYSQL_PORT || "3306", 10),
|
|
365
336
|
user,
|
|
366
337
|
password: password || "",
|
|
367
338
|
database,
|
|
368
339
|
};
|
|
369
340
|
}
|
|
370
341
|
|
|
371
|
-
function getClickHouseConfig(databaseOverride?: string): ClickHouseConfig | undefined {
|
|
372
|
-
const host =
|
|
373
|
-
const database = databaseOverride ||
|
|
342
|
+
function getClickHouseConfig(env: ConnectionEnv, databaseOverride?: string): ClickHouseConfig | undefined {
|
|
343
|
+
const host = env.ELYRA_CLICKHOUSE_HOST;
|
|
344
|
+
const database = databaseOverride || env.ELYRA_CLICKHOUSE_DATABASE;
|
|
374
345
|
if (!host || !database) return undefined;
|
|
375
|
-
const port = Number.parseInt(
|
|
376
|
-
const https =
|
|
377
|
-
const timeoutConnect = Number.parseInt(
|
|
378
|
-
const timeoutQuery = Number.parseInt(
|
|
346
|
+
const port = Number.parseInt(env.ELYRA_CLICKHOUSE_PORT || "8123", 10);
|
|
347
|
+
const https = env.ELYRA_CLICKHOUSE_HTTPS === "true";
|
|
348
|
+
const timeoutConnect = Number.parseInt(env.ELYRA_CLICKHOUSE_TIMEOUT_CONNECT || "10000", 10);
|
|
349
|
+
const timeoutQuery = Number.parseInt(env.ELYRA_CLICKHOUSE_TIMEOUT_QUERY || "30000", 10);
|
|
379
350
|
return {
|
|
380
351
|
host,
|
|
381
352
|
port,
|
|
382
353
|
https,
|
|
383
|
-
user:
|
|
384
|
-
password:
|
|
354
|
+
user: env.ELYRA_CLICKHOUSE_USERNAME || "default",
|
|
355
|
+
password: env.ELYRA_CLICKHOUSE_PASSWORD || "",
|
|
385
356
|
database,
|
|
386
357
|
timeoutConnect,
|
|
387
358
|
timeoutQuery,
|
|
@@ -393,10 +364,242 @@ function isWriteEnabled(): boolean {
|
|
|
393
364
|
return v === "true" || v === "1" || v === "yes";
|
|
394
365
|
}
|
|
395
366
|
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
367
|
+
// ── Read-only SQL check ──
|
|
368
|
+
//
|
|
369
|
+
// Defense in depth only. The real guards are engine-level: SET SESSION TRANSACTION READ ONLY
|
|
370
|
+
// (MySQL), readonly=1 (ClickHouse) and sqlite3 -safe -readonly (SQLite).
|
|
371
|
+
|
|
372
|
+
export type SqlDialect = "mysql" | "clickhouse" | "sqlite";
|
|
373
|
+
|
|
374
|
+
/** Statements allowed in read-only mode, by first keyword. */
|
|
375
|
+
const READ_STATEMENT_START = /^(SELECT|SHOW|DESCRIBE|DESC|EXPLAIN|WITH)\b/;
|
|
376
|
+
|
|
377
|
+
/** Keywords that modify data, schema, privileges or server state. Rejected anywhere outside literals. */
|
|
378
|
+
const FORBIDDEN_KEYWORDS = [
|
|
379
|
+
"INSERT",
|
|
380
|
+
"UPDATE",
|
|
381
|
+
"DELETE",
|
|
382
|
+
"REPLACE",
|
|
383
|
+
"MERGE",
|
|
384
|
+
"UPSERT",
|
|
385
|
+
"DROP",
|
|
386
|
+
"ALTER",
|
|
387
|
+
"CREATE",
|
|
388
|
+
"TRUNCATE",
|
|
389
|
+
"RENAME",
|
|
390
|
+
"ATTACH",
|
|
391
|
+
"DETACH",
|
|
392
|
+
"PRAGMA",
|
|
393
|
+
"VACUUM",
|
|
394
|
+
"REINDEX",
|
|
395
|
+
"GRANT",
|
|
396
|
+
"REVOKE",
|
|
397
|
+
"OPTIMIZE",
|
|
398
|
+
"KILL",
|
|
399
|
+
"CALL",
|
|
400
|
+
"LOAD",
|
|
401
|
+
"HANDLER",
|
|
402
|
+
"INTO",
|
|
403
|
+
];
|
|
404
|
+
|
|
405
|
+
/** Forbidden keywords that are also ordinary functions (REPLACE(s, a, b), TRUNCATE(x, 2), merge(...)). */
|
|
406
|
+
const KEYWORDS_ALLOWED_AS_FUNCTIONS = new Set(["INSERT", "REPLACE", "MERGE", "TRUNCATE"]);
|
|
407
|
+
|
|
408
|
+
/** Functions that read or write host files or load native code. Rejected even when written as a quoted name. */
|
|
409
|
+
const FORBIDDEN_FUNCTIONS: Record<SqlDialect, string[]> = {
|
|
410
|
+
sqlite: ["WRITEFILE", "READFILE", "EDIT", "LOAD_EXTENSION", "FTS3_TOKENIZER"],
|
|
411
|
+
mysql: ["LOAD_FILE"],
|
|
412
|
+
clickhouse: [],
|
|
413
|
+
};
|
|
414
|
+
|
|
415
|
+
const FORBIDDEN_KEYWORD_PATTERN = new RegExp(`\\b(${FORBIDDEN_KEYWORDS.join("|")})\\b(\\s*\\()?`, "g");
|
|
416
|
+
|
|
417
|
+
interface ScannedSql {
|
|
418
|
+
/** The statement with comments removed, string literals replaced by `0` and quoted identifiers by `Q_<name>`. */
|
|
419
|
+
code: string;
|
|
420
|
+
error?: string;
|
|
421
|
+
}
|
|
422
|
+
|
|
423
|
+
/**
|
|
424
|
+
* Strip comments and literals so keyword checks only see SQL code. Follows each dialect's
|
|
425
|
+
* quoting rules closely enough that text the engine parses as code is never hidden inside
|
|
426
|
+
* what the scanner believes is a literal or comment.
|
|
427
|
+
*/
|
|
428
|
+
function scanSql(sql: string, dialect: SqlDialect): ScannedSql {
|
|
429
|
+
const backslashEscapes = dialect !== "sqlite";
|
|
430
|
+
let code = "";
|
|
431
|
+
let i = 0;
|
|
432
|
+
const n = sql.length;
|
|
433
|
+
|
|
434
|
+
const readQuoted = (close: string, escapes: boolean): string | undefined => {
|
|
435
|
+
let content = "";
|
|
436
|
+
let j = i + 1;
|
|
437
|
+
while (j < n) {
|
|
438
|
+
const ch = sql[j];
|
|
439
|
+
if (escapes && ch === "\\") {
|
|
440
|
+
content += sql.slice(j, j + 2);
|
|
441
|
+
j += 2;
|
|
442
|
+
continue;
|
|
443
|
+
}
|
|
444
|
+
if (ch === close) {
|
|
445
|
+
if (sql[j + 1] === close) {
|
|
446
|
+
content += close;
|
|
447
|
+
j += 2;
|
|
448
|
+
continue;
|
|
449
|
+
}
|
|
450
|
+
i = j + 1;
|
|
451
|
+
return content;
|
|
452
|
+
}
|
|
453
|
+
content += ch;
|
|
454
|
+
j++;
|
|
455
|
+
}
|
|
456
|
+
return undefined;
|
|
457
|
+
};
|
|
458
|
+
|
|
459
|
+
while (i < n) {
|
|
460
|
+
const ch = sql[i];
|
|
461
|
+
const next = sql[i + 1];
|
|
462
|
+
|
|
463
|
+
// Line comments: `--` (MySQL needs whitespace after it), `#` (MySQL; ClickHouse `# ` and `#!`)
|
|
464
|
+
const dashComment = ch === "-" && next === "-" && (dialect !== "mysql" || i + 2 >= n || /\s/.test(sql[i + 2]));
|
|
465
|
+
const hashComment =
|
|
466
|
+
ch === "#" && (dialect === "mysql" || (dialect === "clickhouse" && (next === " " || next === "!")));
|
|
467
|
+
if (dashComment || hashComment) {
|
|
468
|
+
const end = sql.indexOf("\n", i);
|
|
469
|
+
i = end === -1 ? n : end + 1;
|
|
470
|
+
code += " ";
|
|
471
|
+
continue;
|
|
472
|
+
}
|
|
473
|
+
|
|
474
|
+
if (ch === "/" && next === "*") {
|
|
475
|
+
if (dialect === "mysql" && sql[i + 2] === "!") {
|
|
476
|
+
return { code, error: "MySQL executable comments (/*! ... */) are not allowed" };
|
|
477
|
+
}
|
|
478
|
+
// ClickHouse block comments nest; MySQL and SQLite ones do not.
|
|
479
|
+
let depth = 1;
|
|
480
|
+
let j = i + 2;
|
|
481
|
+
while (j < n && depth > 0) {
|
|
482
|
+
if (sql[j] === "*" && sql[j + 1] === "/") {
|
|
483
|
+
depth--;
|
|
484
|
+
j += 2;
|
|
485
|
+
} else if (dialect === "clickhouse" && sql[j] === "/" && sql[j + 1] === "*") {
|
|
486
|
+
depth++;
|
|
487
|
+
j += 2;
|
|
488
|
+
} else {
|
|
489
|
+
j++;
|
|
490
|
+
}
|
|
491
|
+
}
|
|
492
|
+
// An unterminated block comment runs to the end of input.
|
|
493
|
+
i = j;
|
|
494
|
+
code += " ";
|
|
495
|
+
continue;
|
|
496
|
+
}
|
|
497
|
+
|
|
498
|
+
if (ch === "'") {
|
|
499
|
+
if (readQuoted("'", backslashEscapes) === undefined) return { code, error: "unterminated string literal" };
|
|
500
|
+
code += " 0 ";
|
|
501
|
+
continue;
|
|
502
|
+
}
|
|
503
|
+
|
|
504
|
+
const identifierClose =
|
|
505
|
+
ch === '"' ? '"' : ch === "`" ? "`" : ch === "[" && dialect === "sqlite" ? "]" : undefined;
|
|
506
|
+
if (identifierClose) {
|
|
507
|
+
// MySQL backtick identifiers have no backslash escapes; everything else follows the string rules.
|
|
508
|
+
const escapes = backslashEscapes && !(dialect === "mysql" && ch === "`");
|
|
509
|
+
const content = readQuoted(identifierClose, escapes);
|
|
510
|
+
if (content === undefined) return { code, error: "unterminated quoted identifier" };
|
|
511
|
+
code += ` Q_${content.replace(/\W/g, "_")} `;
|
|
512
|
+
continue;
|
|
513
|
+
}
|
|
514
|
+
|
|
515
|
+
// ClickHouse heredoc string literals: $tag$ ... $tag$
|
|
516
|
+
if (dialect === "clickhouse" && ch === "$") {
|
|
517
|
+
const tag = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/.exec(sql.slice(i))?.[0];
|
|
518
|
+
if (tag) {
|
|
519
|
+
const end = sql.indexOf(tag, i + tag.length);
|
|
520
|
+
if (end === -1) return { code, error: "unterminated heredoc literal" };
|
|
521
|
+
i = end + tag.length;
|
|
522
|
+
code += " 0 ";
|
|
523
|
+
continue;
|
|
524
|
+
}
|
|
525
|
+
}
|
|
526
|
+
|
|
527
|
+
code += ch;
|
|
528
|
+
i++;
|
|
529
|
+
}
|
|
530
|
+
|
|
531
|
+
return { code };
|
|
532
|
+
}
|
|
533
|
+
|
|
534
|
+
/**
|
|
535
|
+
* Returns why `sql` is not a single read-only statement, or undefined when it is.
|
|
536
|
+
* Rejects multiple statements, data-modifying keywords anywhere outside literals and
|
|
537
|
+
* comments, and functions that touch the host file system.
|
|
538
|
+
*/
|
|
539
|
+
export function readOnlyViolation(sql: string, dialect: SqlDialect): string | undefined {
|
|
540
|
+
const scanned = scanSql(sql, dialect);
|
|
541
|
+
if (scanned.error) return scanned.error;
|
|
542
|
+
const code = scanned.code.toUpperCase();
|
|
543
|
+
|
|
544
|
+
const [first, ...rest] = code.split(";");
|
|
545
|
+
if (rest.some((statement) => statement.trim() !== "")) return "multiple statements are not allowed";
|
|
546
|
+
|
|
547
|
+
if (!READ_STATEMENT_START.test(first.trim()))
|
|
548
|
+
return "only SELECT, SHOW, DESCRIBE, EXPLAIN and WITH queries are allowed";
|
|
549
|
+
|
|
550
|
+
for (const match of first.matchAll(FORBIDDEN_KEYWORD_PATTERN)) {
|
|
551
|
+
const keyword = match[1];
|
|
552
|
+
const calledAsFunction = match[2] !== undefined;
|
|
553
|
+
if (calledAsFunction && KEYWORDS_ALLOWED_AS_FUNCTIONS.has(keyword)) continue;
|
|
554
|
+
return `${keyword} is not allowed`;
|
|
555
|
+
}
|
|
556
|
+
|
|
557
|
+
const forbiddenFunctions = FORBIDDEN_FUNCTIONS[dialect];
|
|
558
|
+
if (forbiddenFunctions.length > 0) {
|
|
559
|
+
const pattern = new RegExp(`\\b(?:Q_)?(${forbiddenFunctions.join("|")})\\s*\\(`);
|
|
560
|
+
const match = pattern.exec(first);
|
|
561
|
+
if (match) return `${match[1].toLowerCase()}() is not allowed`;
|
|
562
|
+
}
|
|
563
|
+
|
|
564
|
+
return undefined;
|
|
565
|
+
}
|
|
566
|
+
|
|
567
|
+
/** True when `sql` is a single read-only statement (see {@link readOnlyViolation}). */
|
|
568
|
+
export function isReadOnly(sql: string, dialect: SqlDialect): boolean {
|
|
569
|
+
return readOnlyViolation(sql, dialect) === undefined;
|
|
570
|
+
}
|
|
571
|
+
|
|
572
|
+
// ── Identifiers ──
|
|
573
|
+
|
|
574
|
+
const IDENTIFIER_PART = /^[A-Za-z_][A-Za-z0-9_$]*$/;
|
|
575
|
+
|
|
576
|
+
/**
|
|
577
|
+
* Split a table name such as `users` or `schema.users` into validated parts.
|
|
578
|
+
* Throws for anything else, so the parts can be quoted and interpolated into SQL safely.
|
|
579
|
+
*/
|
|
580
|
+
export function parseQualifiedIdentifier(name: string): string[] {
|
|
581
|
+
const parts = name.split(".");
|
|
582
|
+
if (parts.length > 2 || !parts.every((part) => IDENTIFIER_PART.test(part))) {
|
|
583
|
+
throw new Error(
|
|
584
|
+
`Invalid identifier "${name}": use letters, digits, _ and $ (not starting with a digit), optionally as schema.table`,
|
|
585
|
+
);
|
|
586
|
+
}
|
|
587
|
+
return parts;
|
|
588
|
+
}
|
|
589
|
+
|
|
590
|
+
/** Validate a single unqualified name (no schema prefix) and quote it. */
|
|
591
|
+
export function quoteIdentifierPart(name: string, quote: "`" | '"'): string {
|
|
592
|
+
if (!IDENTIFIER_PART.test(name)) {
|
|
593
|
+
throw new Error(`Invalid identifier "${name}": use letters, digits, _ and $ (not starting with a digit)`);
|
|
594
|
+
}
|
|
595
|
+
return `${quote}${name}${quote}`;
|
|
596
|
+
}
|
|
597
|
+
|
|
598
|
+
/** Validate `name` and quote each part with `quote` (backtick for MySQL/ClickHouse, double quote for SQLite). */
|
|
599
|
+
export function quoteIdentifier(name: string, quote: "`" | '"'): string {
|
|
600
|
+
return parseQualifiedIdentifier(name)
|
|
601
|
+
.map((part) => `${quote}${part}${quote}`)
|
|
602
|
+
.join(".");
|
|
400
603
|
}
|
|
401
604
|
|
|
402
605
|
// ── .env Loading ──
|
|
@@ -420,10 +623,7 @@ function parseEnvFile(content: string): Record<string, string> {
|
|
|
420
623
|
return result;
|
|
421
624
|
}
|
|
422
625
|
|
|
423
|
-
/**
|
|
424
|
-
* Mapping from standard .env keys (Laravel, Docker, etc.) to ELYRA_ env vars.
|
|
425
|
-
* Only sets the ELYRA_ var if it's not already set (explicit env vars win).
|
|
426
|
-
*/
|
|
626
|
+
/** Mapping from standard .env keys (Laravel, Docker, etc.) to ELYRA_ env vars. */
|
|
427
627
|
const ENV_KEY_MAP: Array<[string, string]> = [
|
|
428
628
|
// MySQL (Laravel DB_* convention)
|
|
429
629
|
["DB_HOST", "ELYRA_MYSQL_HOST"],
|
|
@@ -442,61 +642,87 @@ const ENV_KEY_MAP: Array<[string, string]> = [
|
|
|
442
642
|
["CLICKHOUSE_TIMEOUT_QUERY", "ELYRA_CLICKHOUSE_TIMEOUT_QUERY"],
|
|
443
643
|
];
|
|
444
644
|
|
|
445
|
-
/**
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
if (!existsSync(envPath)) return;
|
|
451
|
-
|
|
452
|
-
const parsed = parseEnvFile(readFileSync(envPath, "utf-8"));
|
|
645
|
+
/** Per engine: the host variable that selects the source, and the prefix of every variable it owns. */
|
|
646
|
+
const CONNECTION_GROUPS = [
|
|
647
|
+
{ hostKey: "ELYRA_MYSQL_HOST", prefix: "ELYRA_MYSQL_" },
|
|
648
|
+
{ hostKey: "ELYRA_CLICKHOUSE_HOST", prefix: "ELYRA_CLICKHOUSE_" },
|
|
649
|
+
] as const;
|
|
453
650
|
|
|
651
|
+
/**
|
|
652
|
+
* Resolve connection variables per engine from exactly one source. When the environment sets
|
|
653
|
+
* the engine's host, every variable of that engine comes from the environment; otherwise all of
|
|
654
|
+
* them come from the project's .env (mapped to ELYRA_ names). Mixing would let a repository's
|
|
655
|
+
* .env set DB_HOST and receive a password from the user's environment.
|
|
656
|
+
* `projectDotEnv` is the parsed project .env, or undefined when the project is not trusted.
|
|
657
|
+
*/
|
|
658
|
+
export function resolveConnectionEnv(
|
|
659
|
+
processEnv: ConnectionEnv,
|
|
660
|
+
projectDotEnv: ConnectionEnv | undefined,
|
|
661
|
+
): Record<string, string> {
|
|
662
|
+
const mapped: Record<string, string> = {};
|
|
663
|
+
if (projectDotEnv) {
|
|
454
664
|
for (const [sourceKey, targetKey] of ENV_KEY_MAP) {
|
|
455
|
-
const value =
|
|
456
|
-
if (value !== undefined
|
|
457
|
-
process.env[targetKey] = value;
|
|
458
|
-
}
|
|
665
|
+
const value = projectDotEnv[sourceKey];
|
|
666
|
+
if (value !== undefined) mapped[targetKey] = value;
|
|
459
667
|
}
|
|
668
|
+
}
|
|
669
|
+
const result: Record<string, string> = {};
|
|
670
|
+
for (const group of CONNECTION_GROUPS) {
|
|
671
|
+
const source: ConnectionEnv | undefined = processEnv[group.hostKey]
|
|
672
|
+
? processEnv
|
|
673
|
+
: mapped[group.hostKey]
|
|
674
|
+
? mapped
|
|
675
|
+
: undefined;
|
|
676
|
+
if (!source) continue;
|
|
677
|
+
for (const [key, value] of Object.entries(source)) {
|
|
678
|
+
if (key.startsWith(group.prefix) && value !== undefined) result[key] = value;
|
|
679
|
+
}
|
|
680
|
+
}
|
|
681
|
+
return result;
|
|
682
|
+
}
|
|
683
|
+
|
|
684
|
+
/** The project's parsed .env, read on demand and only in trusted projects. */
|
|
685
|
+
function readTrustedProjectDotEnv(ctx: ExtensionContext | undefined): Record<string, string> | undefined {
|
|
686
|
+
if (!ctx?.isProjectTrusted()) return undefined;
|
|
687
|
+
try {
|
|
688
|
+
const envPath = join(ctx.cwd, ".env");
|
|
689
|
+
if (!existsSync(envPath)) return undefined;
|
|
690
|
+
return parseEnvFile(readFileSync(envPath, "utf-8"));
|
|
460
691
|
} catch {
|
|
461
|
-
// Never fail on .env parse errors
|
|
692
|
+
// Never fail on .env read or parse errors.
|
|
693
|
+
return undefined;
|
|
462
694
|
}
|
|
463
695
|
}
|
|
464
696
|
|
|
697
|
+
function getConnectionEnv(ctx: ExtensionContext | undefined): ConnectionEnv {
|
|
698
|
+
return resolveConnectionEnv(process.env, readTrustedProjectDotEnv(ctx));
|
|
699
|
+
}
|
|
700
|
+
|
|
465
701
|
// ── Schema Discovery ──
|
|
466
702
|
|
|
467
|
-
async function getMysqlSchema(
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
return {
|
|
474
|
-
content: [{ type: "text", text: "MySQL not configured." }],
|
|
475
|
-
details: {},
|
|
476
|
-
};
|
|
477
|
-
}
|
|
703
|
+
async function getMysqlSchema(env: ConnectionEnv, databaseOverride?: string, table?: string): Promise<SchemaResult> {
|
|
704
|
+
const config = getMysqlConfig(env, databaseOverride);
|
|
705
|
+
if (!config) throw new Error("MySQL not configured.");
|
|
706
|
+
|
|
707
|
+
// Validate before connecting; the name is interpolated into SHOW INDEX.
|
|
708
|
+
const tableParts = table ? parseQualifiedIdentifier(table) : undefined;
|
|
478
709
|
|
|
479
710
|
try {
|
|
480
|
-
const connection = await (
|
|
481
|
-
host: config.host,
|
|
482
|
-
port: config.port,
|
|
483
|
-
user: config.user,
|
|
484
|
-
password: config.password,
|
|
485
|
-
database: config.database,
|
|
486
|
-
connectTimeout: 10000,
|
|
487
|
-
});
|
|
711
|
+
const connection = await openMysql(config);
|
|
488
712
|
|
|
489
713
|
try {
|
|
490
|
-
if (table) {
|
|
714
|
+
if (table && tableParts) {
|
|
715
|
+
const schema = tableParts.length === 2 ? tableParts[0] : config.database;
|
|
716
|
+
const tableName = tableParts[tableParts.length - 1];
|
|
491
717
|
// Detailed schema for a specific table
|
|
492
718
|
const [columns] = await connection.execute(
|
|
493
719
|
`SELECT COLUMN_NAME, COLUMN_TYPE, IS_NULLABLE, COLUMN_KEY, COLUMN_DEFAULT, EXTRA
|
|
494
720
|
FROM INFORMATION_SCHEMA.COLUMNS
|
|
495
721
|
WHERE TABLE_SCHEMA = ? AND TABLE_NAME = ?
|
|
496
722
|
ORDER BY ORDINAL_POSITION`,
|
|
497
|
-
[
|
|
723
|
+
[schema, tableName],
|
|
498
724
|
);
|
|
499
|
-
const [indexes] = await connection.execute(`SHOW INDEX FROM
|
|
725
|
+
const [indexes] = await connection.execute(`SHOW INDEX FROM ${quoteIdentifier(table, "`")}`);
|
|
500
726
|
return {
|
|
501
727
|
content: [
|
|
502
728
|
{
|
|
@@ -529,34 +755,30 @@ async function getMysqlSchema(
|
|
|
529
755
|
await connection.end();
|
|
530
756
|
}
|
|
531
757
|
} catch (error) {
|
|
532
|
-
|
|
533
|
-
return {
|
|
534
|
-
content: [{ type: "text", text: `MySQL schema error: ${msg}` }],
|
|
535
|
-
details: {},
|
|
536
|
-
};
|
|
758
|
+
throw new Error(`MySQL schema error: ${errorMessage(error)}`);
|
|
537
759
|
}
|
|
538
760
|
}
|
|
539
761
|
|
|
540
762
|
async function getClickHouseSchema(
|
|
763
|
+
env: ConnectionEnv,
|
|
541
764
|
databaseOverride?: string,
|
|
542
765
|
table?: string,
|
|
543
|
-
): Promise<
|
|
544
|
-
const config = getClickHouseConfig(databaseOverride);
|
|
545
|
-
if (!config)
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
};
|
|
550
|
-
}
|
|
766
|
+
): Promise<SchemaResult> {
|
|
767
|
+
const config = getClickHouseConfig(env, databaseOverride);
|
|
768
|
+
if (!config) throw new Error("ClickHouse not configured.");
|
|
769
|
+
|
|
770
|
+
// Validate before connecting; the name is interpolated into DESCRIBE TABLE.
|
|
771
|
+
const quotedTable = table ? quoteIdentifier(table, "`") : undefined;
|
|
551
772
|
|
|
552
773
|
try {
|
|
553
774
|
const client = await openClickHouse(config);
|
|
554
775
|
|
|
555
776
|
try {
|
|
556
|
-
if (table) {
|
|
777
|
+
if (table && quotedTable) {
|
|
557
778
|
const result = await client.query({
|
|
558
|
-
query: `DESCRIBE TABLE ${
|
|
779
|
+
query: `DESCRIBE TABLE ${quotedTable}`,
|
|
559
780
|
format: "JSONEachRow",
|
|
781
|
+
clickhouse_settings: CLICKHOUSE_READONLY_SETTINGS,
|
|
560
782
|
});
|
|
561
783
|
const columns = await result.json();
|
|
562
784
|
return {
|
|
@@ -573,9 +795,11 @@ async function getClickHouseSchema(
|
|
|
573
795
|
const result = await client.query({
|
|
574
796
|
query: `SELECT name, engine, total_rows, total_bytes
|
|
575
797
|
FROM system.tables
|
|
576
|
-
WHERE database =
|
|
798
|
+
WHERE database = {database:String}
|
|
577
799
|
ORDER BY name`,
|
|
800
|
+
query_params: { database: config.database },
|
|
578
801
|
format: "JSONEachRow",
|
|
802
|
+
clickhouse_settings: CLICKHOUSE_READONLY_SETTINGS,
|
|
579
803
|
});
|
|
580
804
|
const tables = await result.json();
|
|
581
805
|
return {
|
|
@@ -591,31 +815,26 @@ async function getClickHouseSchema(
|
|
|
591
815
|
await client.close();
|
|
592
816
|
}
|
|
593
817
|
} catch (error) {
|
|
594
|
-
|
|
595
|
-
return {
|
|
596
|
-
content: [{ type: "text", text: `ClickHouse schema error: ${msg}` }],
|
|
597
|
-
details: {},
|
|
598
|
-
};
|
|
818
|
+
throw new Error(`ClickHouse schema error: ${errorMessage(error)}`);
|
|
599
819
|
}
|
|
600
820
|
}
|
|
601
821
|
|
|
602
822
|
// ── SQLite Helpers ──
|
|
603
823
|
|
|
604
|
-
function resolveSqlitePath(explicitPath?: string): string | undefined {
|
|
824
|
+
function resolveSqlitePath(ctx: ExtensionContext, explicitPath?: string): string | undefined {
|
|
825
|
+
const cwd = ctx.cwd;
|
|
605
826
|
if (explicitPath) {
|
|
606
|
-
const resolved = explicitPath
|
|
827
|
+
const resolved = isAbsolute(explicitPath) ? explicitPath : join(cwd, explicitPath);
|
|
607
828
|
return existsSync(resolved) ? resolved : undefined;
|
|
608
829
|
}
|
|
609
830
|
|
|
610
|
-
// Auto-detect from .env
|
|
611
|
-
const
|
|
612
|
-
|
|
613
|
-
if (existsSync(envPath)) {
|
|
614
|
-
const parsed = parseEnvFile(readFileSync(envPath, "utf-8"));
|
|
831
|
+
// Auto-detect from .env (trusted projects only: the path is repository-controlled)
|
|
832
|
+
const parsed = readTrustedProjectDotEnv(ctx);
|
|
833
|
+
if (parsed) {
|
|
615
834
|
if (parsed.DB_CONNECTION === "sqlite") {
|
|
616
835
|
const dbDatabase = parsed.DB_DATABASE;
|
|
617
836
|
if (dbDatabase) {
|
|
618
|
-
const resolved = dbDatabase
|
|
837
|
+
const resolved = isAbsolute(dbDatabase) ? dbDatabase : join(cwd, dbDatabase);
|
|
619
838
|
if (existsSync(resolved)) return resolved;
|
|
620
839
|
}
|
|
621
840
|
}
|
|
@@ -628,59 +847,99 @@ function resolveSqlitePath(explicitPath?: string): string | undefined {
|
|
|
628
847
|
return undefined;
|
|
629
848
|
}
|
|
630
849
|
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
|
|
635
|
-
|
|
636
|
-
if (!dbPath) {
|
|
637
|
-
return {
|
|
638
|
-
content: [{ type: "text", text: "SQLite database not found." }],
|
|
639
|
-
details: {},
|
|
640
|
-
};
|
|
641
|
-
}
|
|
850
|
+
interface SqliteRunOptions {
|
|
851
|
+
readOnly: boolean;
|
|
852
|
+
timeout: number;
|
|
853
|
+
signal?: AbortSignal;
|
|
854
|
+
}
|
|
642
855
|
|
|
856
|
+
/**
|
|
857
|
+
* Run one SQL string through the sqlite3 CLI without a shell.
|
|
858
|
+
*
|
|
859
|
+
* `-safe` (sqlite3 >= 3.37) blocks writefile(), readfile(), edit(), load_extension(), ATTACH and the
|
|
860
|
+
* dot-commands that touch files or run programs. It is always on, also when writes are enabled.
|
|
861
|
+
* `-readonly` opens the file read-only, but on its own does not stop `SELECT writefile(...)`.
|
|
862
|
+
*/
|
|
863
|
+
async function runSqlite(dbPath: string, sql: string, options: SqliteRunOptions): Promise<string> {
|
|
864
|
+
if (sql.trimStart().startsWith(".")) {
|
|
865
|
+
throw new Error("sqlite3 dot-commands are not allowed; send SQL statements only.");
|
|
866
|
+
}
|
|
867
|
+
// sqlite3 treats any argument starting with "-" as an option and a command argument starting
|
|
868
|
+
// with "." as a dot-command. The leading space keeps the SQL a plain SQL argument. dbPath is
|
|
869
|
+
// always absolute (see resolveSqlitePath), so it cannot be read as an option.
|
|
870
|
+
const args = ["-safe", ...(options.readOnly ? ["-readonly"] : []), "-json", dbPath, ` ${sql}`];
|
|
643
871
|
try {
|
|
644
|
-
const {
|
|
872
|
+
const { stdout } = await execFileAsync("sqlite3", args, {
|
|
873
|
+
encoding: "utf-8",
|
|
874
|
+
timeout: options.timeout,
|
|
875
|
+
signal: options.signal,
|
|
876
|
+
maxBuffer: 16 * 1024 * 1024,
|
|
877
|
+
});
|
|
878
|
+
return stdout;
|
|
879
|
+
} catch (error) {
|
|
880
|
+
throw new Error(describeSqliteFailure(error, options.timeout));
|
|
881
|
+
}
|
|
882
|
+
}
|
|
645
883
|
|
|
646
|
-
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
|
|
661
|
-
],
|
|
662
|
-
details: {},
|
|
663
|
-
};
|
|
664
|
-
}
|
|
884
|
+
function describeSqliteFailure(error: unknown, timeout: number): string {
|
|
885
|
+
if (!(error instanceof Error)) return `SQLite error: ${String(error)}`;
|
|
886
|
+
const code = "code" in error ? error.code : undefined;
|
|
887
|
+
const stderr = "stderr" in error && typeof error.stderr === "string" ? error.stderr.trim() : "";
|
|
888
|
+
if (code === "ENOENT") {
|
|
889
|
+
return "SQLite error: the sqlite3 CLI was not found on PATH. Install sqlite3 3.37 or newer.";
|
|
890
|
+
}
|
|
891
|
+
if (/unknown option:\s*-safe/.test(stderr)) {
|
|
892
|
+
return "SQLite error: the installed sqlite3 CLI is too old to support -safe (needs 3.37 or newer). -safe is required to block writefile() and load_extension(); upgrade sqlite3.";
|
|
893
|
+
}
|
|
894
|
+
if ("killed" in error && error.killed === true && error.name !== "AbortError") {
|
|
895
|
+
return `SQLite error: query timed out after ${timeout} ms`;
|
|
896
|
+
}
|
|
897
|
+
return `SQLite error: ${stderr || error.message}`;
|
|
898
|
+
}
|
|
665
899
|
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
900
|
+
async function getSqliteSchema(
|
|
901
|
+
ctx: ExtensionContext,
|
|
902
|
+
dbPathOverride: string | undefined,
|
|
903
|
+
schemaName: string | undefined,
|
|
904
|
+
table: string | undefined,
|
|
905
|
+
signal: AbortSignal | undefined,
|
|
906
|
+
): Promise<SchemaResult> {
|
|
907
|
+
const dbPath = resolveSqlitePath(ctx, dbPathOverride);
|
|
908
|
+
if (!dbPath)
|
|
909
|
+
throw new Error(
|
|
910
|
+
"SQLite database not found. Provide db_path, or set DB_CONNECTION=sqlite in the project's .env (read only in trusted projects).",
|
|
669
911
|
);
|
|
912
|
+
|
|
913
|
+
const run = (sql: string) => runSqlite(dbPath, sql, { readOnly: true, timeout: 10000, signal });
|
|
914
|
+
|
|
915
|
+
if (table) {
|
|
916
|
+
const parts = parseQualifiedIdentifier(table);
|
|
917
|
+
const tableName = parts[parts.length - 1];
|
|
918
|
+
const schema = quoteIdentifierPart(parts.length === 2 ? parts[0] : (schemaName ?? "main"), '"');
|
|
919
|
+
const columns = await run(`PRAGMA ${schema}.table_info("${tableName}")`);
|
|
920
|
+
const indexes = await run(`PRAGMA ${schema}.index_list("${tableName}")`);
|
|
670
921
|
return {
|
|
671
922
|
content: [
|
|
672
923
|
{
|
|
673
924
|
type: "text",
|
|
674
|
-
text: `
|
|
925
|
+
text: `Table: ${table}\n\nColumns:\n${columns.trim()}\n\nIndexes:\n${indexes.trim()}`,
|
|
675
926
|
},
|
|
676
927
|
],
|
|
677
928
|
details: {},
|
|
678
929
|
};
|
|
679
|
-
} catch (error) {
|
|
680
|
-
const msg = error instanceof Error ? error.message : String(error);
|
|
681
|
-
return {
|
|
682
|
-
content: [{ type: "text", text: `SQLite schema error: ${msg}` }],
|
|
683
|
-
details: {},
|
|
684
|
-
};
|
|
685
930
|
}
|
|
931
|
+
|
|
932
|
+
const schema = quoteIdentifierPart(schemaName ?? "main", '"');
|
|
933
|
+
const tables = await run(
|
|
934
|
+
`SELECT name, type FROM ${schema}.sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name`,
|
|
935
|
+
);
|
|
936
|
+
return {
|
|
937
|
+
content: [
|
|
938
|
+
{
|
|
939
|
+
type: "text",
|
|
940
|
+
text: `Database: ${dbPath}\n\nTables:\n${tables.trim()}`,
|
|
941
|
+
},
|
|
942
|
+
],
|
|
943
|
+
details: {},
|
|
944
|
+
};
|
|
686
945
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@elyracode/db-tools",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "1.0.1",
|
|
4
4
|
"description": "Database tools for Elyra - query MySQL and ClickHouse with schema awareness and safety guardrails",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"keywords": [
|
|
@@ -22,17 +22,23 @@
|
|
|
22
22
|
"./extensions/index.ts"
|
|
23
23
|
]
|
|
24
24
|
},
|
|
25
|
+
"files": [
|
|
26
|
+
"extensions",
|
|
27
|
+
"README.md",
|
|
28
|
+
"CHANGELOG.md"
|
|
29
|
+
],
|
|
25
30
|
"dependencies": {
|
|
26
31
|
"mysql2": "^3.14.0",
|
|
27
32
|
"@clickhouse/client": "^1.8.0"
|
|
28
33
|
},
|
|
29
34
|
"peerDependencies": {
|
|
30
|
-
"@elyracode/coding-agent": "
|
|
35
|
+
"@elyracode/coding-agent": ">=1.0.0",
|
|
31
36
|
"typebox": "*"
|
|
32
37
|
},
|
|
33
38
|
"scripts": {
|
|
34
39
|
"clean": "echo 'nothing to clean'",
|
|
35
40
|
"build": "echo 'nothing to build'",
|
|
36
|
-
"check": "echo 'nothing to check'"
|
|
41
|
+
"check": "echo 'nothing to check'",
|
|
42
|
+
"test": "vitest --run"
|
|
37
43
|
}
|
|
38
44
|
}
|
|
@@ -1,48 +0,0 @@
|
|
|
1
|
-
import type { AddressInfo } from "node:net";
|
|
2
|
-
import { createServer, type Server, type Socket } from "node:net";
|
|
3
|
-
import { afterEach, describe, expect, it } from "vitest";
|
|
4
|
-
import dbTools from "../extensions/index.js";
|
|
5
|
-
|
|
6
|
-
type Tool = { name: string; execute: (...args: unknown[]) => Promise<{ content: Array<{ text?: string }> }> };
|
|
7
|
-
|
|
8
|
-
function loadTools(): Map<string, Tool> {
|
|
9
|
-
const tools = new Map<string, Tool>();
|
|
10
|
-
const api = new Proxy(
|
|
11
|
-
{ registerTool: (tool: Tool) => tools.set(tool.name, tool) },
|
|
12
|
-
{ get: (target, prop) => (prop in target ? target[prop as keyof typeof target] : () => {}) },
|
|
13
|
-
);
|
|
14
|
-
dbTools(api as never);
|
|
15
|
-
return tools;
|
|
16
|
-
}
|
|
17
|
-
|
|
18
|
-
describe("ClickHouse connect timeout", () => {
|
|
19
|
-
let server: Server | undefined;
|
|
20
|
-
const sockets: Socket[] = [];
|
|
21
|
-
const saved = { ...process.env };
|
|
22
|
-
|
|
23
|
-
afterEach(() => {
|
|
24
|
-
for (const s of sockets.splice(0)) s.destroy();
|
|
25
|
-
server?.close();
|
|
26
|
-
process.env = { ...saved };
|
|
27
|
-
});
|
|
28
|
-
|
|
29
|
-
it("fails fast with a clear error when the server accepts but never answers", async () => {
|
|
30
|
-
// Accept connections and never respond, like a firewall or a hung server.
|
|
31
|
-
server = createServer((socket) => sockets.push(socket));
|
|
32
|
-
await new Promise<void>((resolve) => server!.listen(0, "127.0.0.1", resolve));
|
|
33
|
-
const port = (server.address() as AddressInfo).port;
|
|
34
|
-
process.env.ELYRA_CLICKHOUSE_HOST = "127.0.0.1";
|
|
35
|
-
process.env.ELYRA_CLICKHOUSE_PORT = String(port);
|
|
36
|
-
process.env.ELYRA_CLICKHOUSE_DATABASE = "default";
|
|
37
|
-
process.env.ELYRA_CLICKHOUSE_TIMEOUT_CONNECT = "300";
|
|
38
|
-
process.env.ELYRA_CLICKHOUSE_TIMEOUT_QUERY = "30000";
|
|
39
|
-
|
|
40
|
-
const tool = loadTools().get("query_clickhouse");
|
|
41
|
-
expect(tool).toBeDefined();
|
|
42
|
-
const started = Date.now();
|
|
43
|
-
const result = await tool!.execute("call-1", { sql: "SELECT 1" });
|
|
44
|
-
const elapsed = Date.now() - started;
|
|
45
|
-
expect(result.content[0].text).toContain("within 300 ms");
|
|
46
|
-
expect(elapsed).toBeLessThan(3000);
|
|
47
|
-
});
|
|
48
|
-
});
|