@keb-org/consol 0.1.1 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +37 -41
- package/dist/consol.d.ts +2 -2
- package/dist/errors.d.ts +1 -3
- package/dist/index.d.ts +2 -2
- package/dist/index.js +130 -416
- package/dist/native.d.ts +94 -0
- package/dist/transport.d.ts +3 -2
- package/dist/types.d.ts +1 -6
- package/dist/vault.d.ts +9 -6
- package/package.json +8 -8
- package/dist/stdio.d.ts +0 -120
package/README.md
CHANGED
|
@@ -2,27 +2,27 @@
|
|
|
2
2
|
|
|
3
3
|
TypeScript SDK for the consol memory engine, for three uses:
|
|
4
4
|
|
|
5
|
-
1. **Local**: a vault on disk (or S3) in your own process
|
|
5
|
+
1. **Local**: a vault on disk (or S3), with the engine running in your own process as a native addon.
|
|
6
6
|
2. **Hosted client**: the same `Vault` methods against a backend over HTTP.
|
|
7
7
|
3. **Backend**: `Consol.handle` behind your own HTTP route, so clients of any language can use it.
|
|
8
8
|
|
|
9
9
|
Node >= 22 or Bun, no dependencies; `import` and `require` share one implementation. The behaviour
|
|
10
10
|
is the same in every mode.
|
|
11
11
|
|
|
12
|
-
## The
|
|
12
|
+
## The engine addon and database
|
|
13
13
|
|
|
14
|
-
The
|
|
14
|
+
The engine is the native addon `consol.node`, loaded the first time you start a vault or a
|
|
15
|
+
`Consol` (importing the SDK loads nothing). The SDK uses the first of these:
|
|
15
16
|
|
|
16
|
-
1. the `
|
|
17
|
-
2. the `
|
|
18
|
-
3. the platform package `@keb-org/consol-<platform>-<arch>`, resolved from the SDK's own location
|
|
19
|
-
4. `consol` (`consol.exe` on Windows) on `PATH`.
|
|
17
|
+
1. the `addonPath` option,
|
|
18
|
+
2. the `CONSOL_ADDON` environment variable,
|
|
19
|
+
3. the platform package `@keb-org/consol-<platform>-<arch>`, resolved from the SDK's own location.
|
|
20
20
|
|
|
21
|
-
|
|
22
|
-
of falling through to the next place.
|
|
23
|
-
`
|
|
21
|
+
An `addonPath` or `CONSOL_ADDON` that points nowhere, or at a file that does not load, fails at
|
|
22
|
+
once with `addon_not_found` instead of falling through to the next place. The platform package
|
|
23
|
+
also ships the `consol` CLI as `bin/consol`.
|
|
24
24
|
|
|
25
|
-
The engine needs a database, PostgreSQL with pgvector or CockroachDB: pass `db` or set
|
|
25
|
+
The engine still needs a database, PostgreSQL with pgvector or CockroachDB: pass `db` or set
|
|
26
26
|
`CONSOL_DB_URL` in your process (the one `CONSOL_*` variable the SDK takes from its host).
|
|
27
27
|
|
|
28
28
|
## 1. Local
|
|
@@ -44,8 +44,8 @@ try {
|
|
|
44
44
|
```
|
|
45
45
|
|
|
46
46
|
`createVault` fails with `vault_exists` when the directory already holds a vault, `openVault`
|
|
47
|
-
(same options) with `vault_not_found` when it does not. Both
|
|
48
|
-
version matches the SDK's major.minor before they return.
|
|
47
|
+
(same options) with `vault_not_found` when it does not. Both open the engine and check that the
|
|
48
|
+
addon's version matches the SDK's major.minor before they return.
|
|
49
49
|
|
|
50
50
|
## 2. Hosted client
|
|
51
51
|
|
|
@@ -109,7 +109,7 @@ Each method takes one object with the tool's wire argument names, plus optional
|
|
|
109
109
|
|
|
110
110
|
| Method | Arguments | Returns |
|
|
111
111
|
|---|---|---|
|
|
112
|
-
| `recall` | `{ query, include_history?, limit?, agent? }` | `RecallResult`: `text` is the progressive disclosure to give a model (cards by tier, intent block, hints); `candidates` are the same cards as data. Cards are leads: read a note before relying on it |
|
|
112
|
+
| `recall` | `{ query, include_history?, limit?, intents?, agent? }` | `RecallResult`: `text` is the progressive disclosure to give a model (cards by tier, intent block, hints); `candidates` are the same cards as data. Cards are leads: read a note before relying on it |
|
|
113
113
|
| `read` | `{ id, continuation?, agent? }` | `ReadResult`; `text` is the exact file |
|
|
114
114
|
| `remember` | `{ id, content?, expected_version?, summary?, entities?, facets?, scope?, team?, valid_from?, valid_to?, reflection?, agent? }` | `RememberResult` |
|
|
115
115
|
| `remove` | `{ id, expected_version, reflection?, agent? }` | `RemoveResult`: the note is archived, its body stays readable |
|
|
@@ -189,8 +189,8 @@ try {
|
|
|
189
189
|
`content_too_large`, `unsupported`, `work_not_found`, `basis_mismatch`, `already_decided`,
|
|
190
190
|
`unresolved_writes`, `postcondition_failed`, `index_unavailable`, `storage_unavailable`, `vault_busy`, `internal`.
|
|
191
191
|
Vault codes: `vault_exists`, `vault_not_found`.
|
|
192
|
-
- SDK codes: `request_failed` (`status?`), `
|
|
193
|
-
major.minor differs from the SDK's), `
|
|
192
|
+
- SDK codes: `request_failed` (`status?`), `addon_not_found`, `incompatible_server` (addon or
|
|
193
|
+
server major.minor differs from the SDK's), `timeout`, `aborted`, `closed`.
|
|
194
194
|
- `retryable: true` (`vault_busy`, `index_unavailable`, `storage_unavailable`, `request_failed` on a connect error or
|
|
195
195
|
502/503/504) means the identical request may succeed later. The SDK never retries by itself.
|
|
196
196
|
|
|
@@ -198,22 +198,17 @@ try {
|
|
|
198
198
|
|
|
199
199
|
- `close()` is final and idempotent: in-flight and later calls fail with `closed`, and concurrent
|
|
200
200
|
calls share one completion. `await using vault = await openVault(...)` closes it for you.
|
|
201
|
-
- An idle local vault does not keep Node or Bun alive: a script that forgets `close()` still
|
|
202
|
-
exits, and its server exits with it.
|
|
203
201
|
- Every call has a deadline: 300 s by default, `timeoutMs` per vault or per call, `0` disables it.
|
|
204
202
|
`signal` aborts a call. Both only stop waiting: a write may still complete, so read the note
|
|
205
203
|
before you retry.
|
|
206
|
-
-
|
|
207
|
-
|
|
208
|
-
- Servers on one vault take turns on its lock, a lease in the database; a wait over 5 s fails with
|
|
209
|
-
`vault_busy`.
|
|
204
|
+
- Engines on one vault, in this process or another, take turns on its lock, a lease in the
|
|
205
|
+
database; a wait over 5 s fails with `vault_busy`.
|
|
210
206
|
|
|
211
207
|
## Configuration and logging
|
|
212
208
|
|
|
213
|
-
Options of `createVault`, `openVault` (local) and `Consol.start` become the
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
vault.
|
|
209
|
+
Options of `createVault`, `openVault` (local) and `Consol.start` become the engine's settings (the
|
|
210
|
+
`CONSOL_*` names). Your process's own `CONSOL_*` variables are not read (except `CONSOL_DB_URL` as
|
|
211
|
+
the default for `db`), so a stray host variable cannot change the agent or the vault.
|
|
217
212
|
|
|
218
213
|
| Option | Meaning |
|
|
219
214
|
|---|---|
|
|
@@ -223,14 +218,14 @@ vault.
|
|
|
223
218
|
| `db` | database URL (`CONSOL_DB_URL`): PostgreSQL with pgvector, or CockroachDB |
|
|
224
219
|
| `embedding` | `{ url, model, apiKey?, dims?, queryPrefix?, batchSize?, timeoutMs? }` (`CONSOL_EMBED_*`); without it recall is lexical-only |
|
|
225
220
|
| `jev` | `{ url?, apiKey?, model?, auto? }` (`CONSOL_JEV_*`); experimental advice |
|
|
226
|
-
| `env` | any other
|
|
227
|
-
| `
|
|
221
|
+
| `env` | any other engine setting (a `CONSOL_*` name), applied last |
|
|
222
|
+
| `addonPath` | the `consol.node` to load, see above |
|
|
228
223
|
| `timeoutMs` | default per-call deadline in ms |
|
|
229
|
-
| `onLog` | `(line) => void`, receives each
|
|
224
|
+
| `onLog` | `(line) => void`, receives each engine log line |
|
|
225
|
+
| `logLevel` | which lines reach `onLog`, such as `info` or `consol_engine=debug` (default `warn`) |
|
|
230
226
|
|
|
231
|
-
The
|
|
232
|
-
|
|
233
|
-
last lines in its message and in `stderrTail`.
|
|
227
|
+
The addon has one log sink per process: every engine's `onLog` receives the same lines at the
|
|
228
|
+
latest `logLevel` set, and the sink is released when the last of them closes.
|
|
234
229
|
|
|
235
230
|
## Types and stability
|
|
236
231
|
|
|
@@ -246,21 +241,22 @@ decisions, and every `jev`, `jev_triage` and `jev_advisory` field (typed as plai
|
|
|
246
241
|
bun install
|
|
247
242
|
bun run build # dist/index.js (CJS), dist/index.mjs (ESM wrapper), dist/*.d.ts
|
|
248
243
|
bun run typecheck # tsc over src
|
|
249
|
-
#
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
bun run test
|
|
253
|
-
|
|
244
|
+
bun run test:offline # pure checks, hosted-client stub, fake-addon transport checks, strict tsc incl. conformance
|
|
245
|
+
# the rest needs the real addon and a database:
|
|
246
|
+
export CONSOL_ADDON=/abs/consol.node CONSOL_DB_URL=postgres://...
|
|
247
|
+
bun run test # the offline checks plus the real-addon suite on Node
|
|
248
|
+
CONSOL_BIN=/abs/consol bun run test:consumers # packed tarballs in Bun, Node ESM, Node CJS and platform-package projects
|
|
249
|
+
node test/capture.mjs # recapture test/fixtures from the real engine (without CONSOL_ADDON it only regenerates the types check)
|
|
254
250
|
```
|
|
255
251
|
|
|
256
252
|
`bun run test`, not `bun test` (that is Bun's own runner). Consumer logs go to
|
|
257
253
|
`consumers/<project>.log`.
|
|
258
254
|
|
|
259
|
-
## Publishing platform
|
|
255
|
+
## Publishing platform packages
|
|
260
256
|
|
|
261
257
|
1. `bun run build`
|
|
262
|
-
2. `bun scripts/make-platform-package.mjs <binary> <platform> <arch> <outDir>` per target
|
|
263
|
-
(win32-x64, darwin-arm64, darwin-x64, linux-x64, linux-arm64); the binary is stored as
|
|
264
|
-
`bin/consol[.exe]` whatever
|
|
258
|
+
2. `bun scripts/make-platform-package.mjs <binary> <addon> <platform> <arch> <outDir>` per target
|
|
259
|
+
(win32-x64, darwin-arm64, darwin-x64, linux-x64, linux-arm64); the CLI binary is stored as
|
|
260
|
+
`bin/consol[.exe]` and the addon as `consol.node`, whatever their file names
|
|
265
261
|
3. Publish each platform package, then `bun scripts/inject-optional-deps.mjs <publish-copy-of-package.json>`
|
|
266
262
|
4. `bun publish` the main package (`publishConfig.access` is `public`)
|
package/dist/consol.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { type EngineOptions } from "./
|
|
1
|
+
import { type EngineOptions } from "./native";
|
|
2
2
|
import type { CallOptions, Context } from "./transport";
|
|
3
3
|
import type { Reply } from "./types";
|
|
4
4
|
/** Exactly one of `vaultRoot` / `vault`, plus the engine settings (`db`, `embedding`, `jev`, ...). */
|
|
@@ -10,7 +10,7 @@ export interface ConsolOptions extends EngineOptions {
|
|
|
10
10
|
}
|
|
11
11
|
export declare function httpStatus(reply: Reply): number;
|
|
12
12
|
/**
|
|
13
|
-
* The engine in your own backend:
|
|
13
|
+
* The engine in your own backend: the in-process engine behind `handle`. A backend authorizes
|
|
14
14
|
* the request itself, then passes the result as `ctx`.
|
|
15
15
|
*
|
|
16
16
|
* SECURITY: `ctx` (the vaults the caller may use, the agent they are) must come from YOUR auth,
|
package/dist/errors.d.ts
CHANGED
|
@@ -1,13 +1,12 @@
|
|
|
1
1
|
import type { Reply } from "./types";
|
|
2
2
|
/** Branchable error codes: wire tool codes (contract section 4) plus the SDK codes of section 6. */
|
|
3
|
-
export type ConsolErrorCode = "invalid_argument" | "not_found" | "already_exists" | "version_conflict" | "team_not_attached" | "secret_rejected" | "content_too_large" | "unsupported" | "work_not_found" | "basis_mismatch" | "already_decided" | "unresolved_writes" | "postcondition_failed" | "index_unavailable" | "storage_unavailable" | "vault_busy" | "forbidden" | "vault_exists" | "vault_not_found" | "internal" | "request_failed" | "
|
|
3
|
+
export type ConsolErrorCode = "invalid_argument" | "not_found" | "already_exists" | "version_conflict" | "team_not_attached" | "secret_rejected" | "content_too_large" | "unsupported" | "work_not_found" | "basis_mismatch" | "already_decided" | "unresolved_writes" | "postcondition_failed" | "index_unavailable" | "storage_unavailable" | "vault_busy" | "forbidden" | "vault_exists" | "vault_not_found" | "internal" | "request_failed" | "addon_not_found" | "incompatible_server" | "timeout" | "aborted" | "closed" | (string & {});
|
|
4
4
|
export interface ConsolErrorDetails {
|
|
5
5
|
retryable?: boolean;
|
|
6
6
|
field?: string;
|
|
7
7
|
tool?: string;
|
|
8
8
|
/** HTTP status of a `request_failed` / `forbidden` that came from the transport, not the engine. */
|
|
9
9
|
status?: number;
|
|
10
|
-
stderrTail?: string;
|
|
11
10
|
result?: Reply;
|
|
12
11
|
cause?: unknown;
|
|
13
12
|
}
|
|
@@ -18,7 +17,6 @@ export declare class ConsolError extends Error {
|
|
|
18
17
|
readonly field?: string;
|
|
19
18
|
readonly tool?: string;
|
|
20
19
|
readonly status?: number;
|
|
21
|
-
readonly stderrTail?: string;
|
|
22
20
|
/** Raw reply of a failed typed call: durable write state (`result.result`) survives the throw. */
|
|
23
21
|
readonly result?: Reply;
|
|
24
22
|
constructor(code: ConsolErrorCode, message: string, details?: ConsolErrorDetails);
|
package/dist/index.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
export { ConsolError, type ConsolErrorCode } from "./errors";
|
|
2
2
|
export { Consol, httpStatus, type ConsolOptions } from "./consol";
|
|
3
3
|
export type { CallOptions, Context } from "./transport";
|
|
4
|
-
export type { EmbeddingOptions, EngineOptions, JevOptions } from "./
|
|
5
|
-
export { Vault, createVault, openVault, sliceUtf8, type VaultOptions, type HostedOptions, type RecallArgs, type ReadArgs, type RememberArgs, type RemoveArgs, type ResumeArgs, type ReflectArgs, type ReflectionContext, type Decision, type InspectedSource, type InspectedSpan, type TemporalAssessment, type PairAssertion, } from "./vault";
|
|
4
|
+
export type { EmbeddingOptions, EngineOptions, JevOptions } from "./native";
|
|
5
|
+
export { Vault, createVault, openVault, sliceUtf8, type VaultOptions, type HostedOptions, type Intent, type RecallArgs, type ReadArgs, type RememberArgs, type RemoveArgs, type ResumeArgs, type ReflectArgs, type ReflectionContext, type Decision, type InspectedSource, type InspectedSpan, type TemporalAssessment, type PairAssertion, } from "./vault";
|
|
6
6
|
export type { Reply, ToolError, Unavailable, Status, ByteRange, Scope, SourceRef, RelationshipKind, Relationship, RecallRetrieval, SourceExcerpt, RecallCard, RecallResult, ReadResult, RememberResult, RemoveResult, ReflectResult, AffectedDoc, EntityAffectedGroup, WriteState, WriteFailureCode, WriteReceipt, WriteResolution, WriteOutcome, ReflectionWork, SourceWork, PairWork, HubWork, PromotionWork, DeliveredSource, CompleteDelivery, FailedDelivery, ReflectDelivery, DecisionReceipt, SourceDisposition, PairDisposition, } from "./types";
|
package/dist/index.js
CHANGED
|
@@ -1,5 +1,3 @@
|
|
|
1
|
-
var __create = Object.create;
|
|
2
|
-
var __getProtoOf = Object.getPrototypeOf;
|
|
3
1
|
var __defProp = Object.defineProperty;
|
|
4
2
|
var __getOwnPropNames = Object.getOwnPropertyNames;
|
|
5
3
|
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
|
|
@@ -7,30 +5,6 @@ var __hasOwnProp = Object.prototype.hasOwnProperty;
|
|
|
7
5
|
function __accessProp(key) {
|
|
8
6
|
return this[key];
|
|
9
7
|
}
|
|
10
|
-
var __toESMCache_node;
|
|
11
|
-
var __toESMCache_esm;
|
|
12
|
-
var __toESM = (mod, isNodeMode, target) => {
|
|
13
|
-
var canCache = mod != null && typeof mod === "object";
|
|
14
|
-
if (canCache) {
|
|
15
|
-
var cache = isNodeMode ? __toESMCache_node ??= new WeakMap : __toESMCache_esm ??= new WeakMap;
|
|
16
|
-
var cached = cache.get(mod);
|
|
17
|
-
if (cached)
|
|
18
|
-
return cached;
|
|
19
|
-
}
|
|
20
|
-
target = mod != null ? __create(__getProtoOf(mod)) : {};
|
|
21
|
-
const to = isNodeMode || !mod || !mod.__esModule || !__hasOwnProp.call(mod, "default") ? __defProp(target, "default", { value: mod, enumerable: true }) : target;
|
|
22
|
-
if (mod && typeof mod === "object" || typeof mod === "function") {
|
|
23
|
-
for (let key of __getOwnPropNames(mod))
|
|
24
|
-
if (!__hasOwnProp.call(to, key))
|
|
25
|
-
__defProp(to, key, {
|
|
26
|
-
get: __accessProp.bind(mod, key),
|
|
27
|
-
enumerable: true
|
|
28
|
-
});
|
|
29
|
-
}
|
|
30
|
-
if (canCache)
|
|
31
|
-
cache.set(mod, to);
|
|
32
|
-
return to;
|
|
33
|
-
};
|
|
34
8
|
var __toCommonJS = (from) => {
|
|
35
9
|
var entry = (__moduleCache ??= new WeakMap).get(from), desc;
|
|
36
10
|
if (entry)
|
|
@@ -84,7 +58,6 @@ class ConsolError extends Error {
|
|
|
84
58
|
field;
|
|
85
59
|
tool;
|
|
86
60
|
status;
|
|
87
|
-
stderrTail;
|
|
88
61
|
result;
|
|
89
62
|
constructor(code, message, details = {}) {
|
|
90
63
|
super(message, details.cause === undefined ? undefined : { cause: details.cause });
|
|
@@ -97,8 +70,6 @@ class ConsolError extends Error {
|
|
|
97
70
|
this.tool = details.tool;
|
|
98
71
|
if (details.status !== undefined)
|
|
99
72
|
this.status = details.status;
|
|
100
|
-
if (details.stderrTail !== undefined)
|
|
101
|
-
this.stderrTail = details.stderrTail;
|
|
102
73
|
if (details.result !== undefined)
|
|
103
74
|
this.result = details.result;
|
|
104
75
|
Object.defineProperty(this, BRAND, { value: true });
|
|
@@ -118,11 +89,8 @@ function replyError(reply, tool) {
|
|
|
118
89
|
result: reply
|
|
119
90
|
});
|
|
120
91
|
}
|
|
121
|
-
// src/
|
|
122
|
-
var
|
|
123
|
-
var import_node_fs = require("node:fs");
|
|
124
|
-
var readline = __toESM(require("node:readline"));
|
|
125
|
-
var import_node_string_decoder = require("node:string_decoder");
|
|
92
|
+
// src/native.ts
|
|
93
|
+
var import_node_path = require("node:path");
|
|
126
94
|
|
|
127
95
|
// src/transport.ts
|
|
128
96
|
var DEFAULT_TIMEOUT_MS = 300000;
|
|
@@ -211,409 +179,168 @@ class HttpTransport {
|
|
|
211
179
|
}
|
|
212
180
|
}
|
|
213
181
|
|
|
214
|
-
// src/
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
var KILL_WAIT_MS = 4000;
|
|
218
|
-
function engineEnv(o) {
|
|
219
|
-
const env = {};
|
|
182
|
+
// src/native.ts
|
|
183
|
+
function engineSettings(o) {
|
|
184
|
+
const s = {};
|
|
220
185
|
const db = o.db ?? process.env.CONSOL_DB_URL;
|
|
221
186
|
if (db !== undefined)
|
|
222
|
-
|
|
187
|
+
s.CONSOL_DB_URL = db;
|
|
223
188
|
const e = o.embedding;
|
|
224
189
|
if (e !== undefined) {
|
|
225
|
-
|
|
190
|
+
s.CONSOL_EMBED_URL = e.url;
|
|
226
191
|
if (e.apiKey !== undefined)
|
|
227
|
-
|
|
228
|
-
|
|
192
|
+
s.CONSOL_EMBED_API_KEY = e.apiKey;
|
|
193
|
+
s.CONSOL_EMBED_MODEL = e.model;
|
|
229
194
|
if (e.dims !== undefined)
|
|
230
|
-
|
|
195
|
+
s.CONSOL_EMBED_DIMS = String(e.dims);
|
|
231
196
|
if (e.queryPrefix !== undefined)
|
|
232
|
-
|
|
197
|
+
s.CONSOL_EMBED_QUERY_PREFIX = e.queryPrefix;
|
|
233
198
|
if (e.batchSize !== undefined)
|
|
234
|
-
|
|
199
|
+
s.CONSOL_EMBED_BATCH = String(e.batchSize);
|
|
235
200
|
if (e.timeoutMs !== undefined)
|
|
236
|
-
|
|
201
|
+
s.CONSOL_EMBED_TIMEOUT_MS = String(e.timeoutMs);
|
|
237
202
|
}
|
|
238
203
|
const j = o.jev;
|
|
239
204
|
if (j !== undefined) {
|
|
240
205
|
if (j.url !== undefined)
|
|
241
|
-
|
|
206
|
+
s.CONSOL_JEV_URL = j.url;
|
|
242
207
|
if (j.apiKey !== undefined)
|
|
243
|
-
|
|
208
|
+
s.CONSOL_JEV_API_KEY = j.apiKey;
|
|
244
209
|
if (j.model !== undefined)
|
|
245
|
-
|
|
210
|
+
s.CONSOL_JEV_MODEL = j.model;
|
|
246
211
|
if (j.auto === false)
|
|
247
|
-
|
|
212
|
+
s.CONSOL_JEV_AUTO = "0";
|
|
248
213
|
}
|
|
249
|
-
return env;
|
|
250
|
-
}
|
|
251
|
-
function platformExe() {
|
|
252
|
-
return process.platform === "win32" ? "consol.exe" : "consol";
|
|
214
|
+
return { ...s, ...o.env };
|
|
253
215
|
}
|
|
254
|
-
function
|
|
216
|
+
function loadAddon(explicit) {
|
|
217
|
+
const fromEnv = process.env.CONSOL_ADDON;
|
|
218
|
+
const spec = `@keb-org/consol-${process.platform}-${process.arch}/consol.node`;
|
|
219
|
+
const [attempt, target] = explicit ? [`addonPath:${explicit}`, import_node_path.resolve(explicit)] : fromEnv ? [`CONSOL_ADDON:${fromEnv}`, import_node_path.resolve(fromEnv)] : [`package:${spec}`, spec];
|
|
255
220
|
try {
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
221
|
+
const sdkRequire = require;
|
|
222
|
+
return sdkRequire(target);
|
|
223
|
+
} catch (cause) {
|
|
224
|
+
throw new ConsolError("addon_not_found", `addon not found: ${attempt} (${String(cause?.message).split(`
|
|
225
|
+
`)[0]})`, { cause });
|
|
259
226
|
}
|
|
260
227
|
}
|
|
261
|
-
|
|
262
|
-
|
|
228
|
+
var logSinks = new Set;
|
|
229
|
+
var fanOut = (line) => {
|
|
230
|
+
for (const sink of logSinks)
|
|
231
|
+
sink(line);
|
|
232
|
+
};
|
|
233
|
+
function subscribe(addon, onLog, level) {
|
|
234
|
+
const sink = (line) => {
|
|
235
|
+
try {
|
|
236
|
+
onLog(line);
|
|
237
|
+
} catch {}
|
|
238
|
+
};
|
|
239
|
+
logSinks.add(sink);
|
|
240
|
+
addon.setLogSink(fanOut, level);
|
|
241
|
+
return () => {
|
|
242
|
+
if (logSinks.delete(sink) && logSinks.size === 0)
|
|
243
|
+
addon.setLogSink(null);
|
|
244
|
+
};
|
|
263
245
|
}
|
|
264
|
-
function
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
throw missingBinary(`explicit:${explicit}`);
|
|
269
|
-
return { path: explicit, attempts: [`explicit:${explicit}`] };
|
|
270
|
-
}
|
|
271
|
-
const fromEnv = process.env.CONSOL_BIN;
|
|
272
|
-
if (fromEnv) {
|
|
273
|
-
if (!isFile(fromEnv))
|
|
274
|
-
throw missingBinary(`CONSOL_BIN:${fromEnv}`);
|
|
275
|
-
return { path: fromEnv, attempts: [`CONSOL_BIN:${fromEnv}`] };
|
|
276
|
-
}
|
|
277
|
-
const spec = `@keb-org/consol-${process.platform}-${process.arch}/bin/${exe}`;
|
|
246
|
+
function engineError(error) {
|
|
247
|
+
if (ConsolError.is(error))
|
|
248
|
+
return error;
|
|
249
|
+
const text = error instanceof Error ? error.message : String(error);
|
|
278
250
|
try {
|
|
279
|
-
const
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
}
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
const name = process.platform === "win32" ? key.toUpperCase() : key;
|
|
289
|
-
if (value === undefined || name.startsWith("CONSOL_") || name === "RUST_LOG")
|
|
290
|
-
continue;
|
|
291
|
-
env[key] = value;
|
|
292
|
-
}
|
|
293
|
-
return { ...env, ...base, ...extra };
|
|
251
|
+
const { code, message, field } = JSON.parse(text);
|
|
252
|
+
if (typeof code === "string") {
|
|
253
|
+
return new ConsolError(code, typeof message === "string" ? message : code, {
|
|
254
|
+
field: typeof field === "string" ? field : undefined,
|
|
255
|
+
cause: error
|
|
256
|
+
});
|
|
257
|
+
}
|
|
258
|
+
} catch {}
|
|
259
|
+
return new ConsolError("internal", text, { cause: error });
|
|
294
260
|
}
|
|
295
|
-
function
|
|
296
|
-
const
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
261
|
+
function race(work, label, ms, closed, user) {
|
|
262
|
+
const timer = ms > 0 ? AbortSignal.timeout(ms) : undefined;
|
|
263
|
+
const stop = anySignal([closed, user, timer].filter((s) => s !== undefined));
|
|
264
|
+
return new Promise((resolve, reject) => {
|
|
265
|
+
const onStop = () => reject(closed?.aborted ? new ConsolError("closed", "vault is closed") : user?.aborted ? new ConsolError("aborted", `${label} aborted`, { cause: user.reason }) : new ConsolError("timeout", `${label} timed out after ${ms}ms`));
|
|
266
|
+
stop.addEventListener("abort", onStop, { once: true });
|
|
267
|
+
work.then(resolve, reject).finally(() => stop.removeEventListener("abort", onStop));
|
|
268
|
+
});
|
|
303
269
|
}
|
|
304
|
-
var READY = 0;
|
|
305
270
|
|
|
306
|
-
class
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
binary;
|
|
271
|
+
class NativeTransport {
|
|
272
|
+
engine;
|
|
273
|
+
version;
|
|
310
274
|
timeoutMs;
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
generation = 0;
|
|
314
|
-
nextId = 1;
|
|
315
|
-
pending = new Map;
|
|
316
|
-
starting = null;
|
|
317
|
-
started = false;
|
|
318
|
-
closed = false;
|
|
275
|
+
unsubscribe;
|
|
276
|
+
closed = new AbortController;
|
|
319
277
|
closing = null;
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
this.
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
const lines = (this.stderrPartial + this.decoder.write(chunk)).split(/\r?\n/);
|
|
334
|
-
this.stderrPartial = lastBytes(lines.pop() ?? "");
|
|
335
|
-
for (const line of lines)
|
|
336
|
-
this.pushLine(line);
|
|
337
|
-
}
|
|
338
|
-
flushStderr() {
|
|
339
|
-
const rest = (this.stderrPartial + this.decoder.end()).trim();
|
|
340
|
-
this.stderrPartial = "";
|
|
341
|
-
this.decoder = new import_node_string_decoder.StringDecoder("utf8");
|
|
342
|
-
if (rest)
|
|
343
|
-
this.pushLine(rest);
|
|
344
|
-
}
|
|
345
|
-
pushLine(line) {
|
|
346
|
-
const text = lastBytes(line.trim().replace(/\x1b\[[0-9;]*[A-Za-z]/g, ""));
|
|
347
|
-
if (!text)
|
|
348
|
-
return;
|
|
349
|
-
this.stderrTail.push(text);
|
|
350
|
-
this.stderrTailBytes += Buffer.byteLength(text, "utf8");
|
|
351
|
-
while (this.stderrTailBytes > STDERR_TAIL_BYTES && this.stderrTail.length > 1) {
|
|
352
|
-
const dropped = this.stderrTail.shift();
|
|
353
|
-
this.stderrTailBytes -= Buffer.byteLength(dropped, "utf8");
|
|
354
|
-
}
|
|
355
|
-
try {
|
|
356
|
-
this.opts.onLog?.(text);
|
|
357
|
-
} catch {}
|
|
358
|
-
}
|
|
359
|
-
tailText() {
|
|
360
|
-
return this.stderrTail.join(`
|
|
361
|
-
`);
|
|
362
|
-
}
|
|
363
|
-
updateRef() {
|
|
364
|
-
const child = this.child;
|
|
365
|
-
if (!child || child.exitCode !== null || child.signalCode !== null)
|
|
366
|
-
return;
|
|
367
|
-
try {
|
|
368
|
-
if (this.pending.size > 0 || this.starting !== null)
|
|
369
|
-
child.ref();
|
|
370
|
-
else
|
|
371
|
-
child.unref();
|
|
372
|
-
} catch {}
|
|
373
|
-
}
|
|
374
|
-
onDeath(error) {
|
|
375
|
-
const child = this.child;
|
|
376
|
-
this.child = null;
|
|
377
|
-
this.reader?.close();
|
|
378
|
-
this.reader = null;
|
|
278
|
+
constructor(engine, version, timeoutMs, unsubscribe) {
|
|
279
|
+
this.engine = engine;
|
|
280
|
+
this.version = version;
|
|
281
|
+
this.timeoutMs = timeoutMs;
|
|
282
|
+
this.unsubscribe = unsubscribe;
|
|
283
|
+
}
|
|
284
|
+
static async start(opts) {
|
|
285
|
+
const timeoutMs = deadline(opts.timeoutMs, DEFAULT_TIMEOUT_MS);
|
|
286
|
+
const addon = loadAddon(opts.addonPath);
|
|
287
|
+
const version = checkVersion(addon.version(), opts.sdkVersion);
|
|
288
|
+
const unsubscribe = opts.onLog ? subscribe(addon, opts.onLog, opts.logLevel) : () => {};
|
|
289
|
+
let opening;
|
|
290
|
+
let engine;
|
|
379
291
|
try {
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
292
|
+
opening = addon.openEngine(JSON.stringify(opts.location), engineSettings(opts));
|
|
293
|
+
engine = await race(opening, "start", timeoutMs);
|
|
294
|
+
} catch (error) {
|
|
295
|
+
unsubscribe();
|
|
296
|
+
opening?.then((late) => late.close()).catch(() => {});
|
|
297
|
+
throw engineError(error);
|
|
298
|
+
}
|
|
299
|
+
const transport = new NativeTransport(engine, version, timeoutMs, unsubscribe);
|
|
300
|
+
const { boot } = opts;
|
|
301
|
+
if (boot) {
|
|
302
|
+
try {
|
|
303
|
+
const error = replyError(await transport.vaultOp(boot.create ? "create_vault" : "open_vault", boot.vault));
|
|
304
|
+
if (error)
|
|
305
|
+
throw error;
|
|
306
|
+
} catch (error) {
|
|
307
|
+
await transport.close().catch(() => {});
|
|
308
|
+
throw error;
|
|
309
|
+
}
|
|
310
|
+
}
|
|
311
|
+
return transport;
|
|
387
312
|
}
|
|
388
313
|
close() {
|
|
389
314
|
this.closing ??= this.shutdown();
|
|
390
315
|
return this.closing;
|
|
391
316
|
}
|
|
392
317
|
async shutdown() {
|
|
393
|
-
this.closed
|
|
394
|
-
|
|
395
|
-
this.child = null;
|
|
396
|
-
this.reader?.close();
|
|
397
|
-
this.reader = null;
|
|
398
|
-
const failure = new ConsolError("closed", "vault is closed");
|
|
399
|
-
for (const pending of this.pending.values())
|
|
400
|
-
pending.reject(failure);
|
|
401
|
-
this.pending.clear();
|
|
402
|
-
if (!child)
|
|
403
|
-
return;
|
|
404
|
-
await new Promise((resolve) => {
|
|
405
|
-
if (!child.pid || child.exitCode !== null || child.signalCode !== null) {
|
|
406
|
-
resolve();
|
|
407
|
-
return;
|
|
408
|
-
}
|
|
409
|
-
const terminate = setTimeout(() => {
|
|
410
|
-
try {
|
|
411
|
-
child.kill();
|
|
412
|
-
} catch {}
|
|
413
|
-
}, EXIT_WAIT_MS);
|
|
414
|
-
const force = setTimeout(() => {
|
|
415
|
-
try {
|
|
416
|
-
child.kill("SIGKILL");
|
|
417
|
-
} catch {}
|
|
418
|
-
}, KILL_WAIT_MS);
|
|
419
|
-
child.once("close", () => {
|
|
420
|
-
clearTimeout(terminate);
|
|
421
|
-
clearTimeout(force);
|
|
422
|
-
resolve();
|
|
423
|
-
});
|
|
424
|
-
try {
|
|
425
|
-
child.stdin?.end();
|
|
426
|
-
} catch {
|
|
427
|
-
resolve();
|
|
428
|
-
}
|
|
429
|
-
});
|
|
430
|
-
}
|
|
431
|
-
async start() {
|
|
432
|
-
if (this.closed)
|
|
433
|
-
throw new ConsolError("closed", "vault is closed");
|
|
434
|
-
if (!this.starting && !this.child)
|
|
435
|
-
this.starting = this.spawn();
|
|
436
|
-
if (this.starting) {
|
|
437
|
-
try {
|
|
438
|
-
await this.starting;
|
|
439
|
-
} finally {
|
|
440
|
-
this.starting = null;
|
|
441
|
-
this.updateRef();
|
|
442
|
-
}
|
|
443
|
-
}
|
|
444
|
-
if (this.closed)
|
|
445
|
-
throw new ConsolError("closed", "vault is closed");
|
|
446
|
-
}
|
|
447
|
-
async spawn() {
|
|
448
|
-
const generation = ++this.generation;
|
|
449
|
-
const child = import_node_child_process.spawn(this.binary, ["stdio", ...this.opts.location], {
|
|
450
|
-
stdio: ["pipe", "pipe", "pipe"],
|
|
451
|
-
env: childEnv(engineEnv(this.opts), this.opts.env)
|
|
452
|
-
});
|
|
453
|
-
this.child = child;
|
|
454
|
-
for (const stream of [child.stdin, child.stdout, child.stderr])
|
|
455
|
-
stream?.unref?.();
|
|
456
|
-
this.updateRef();
|
|
457
|
-
child.stderr?.on("data", (chunk) => this.onStderr(chunk));
|
|
458
|
-
child.stderr?.on("error", () => {});
|
|
459
|
-
const dead = (reason, cause) => {
|
|
460
|
-
if (this.child !== child)
|
|
461
|
-
return;
|
|
462
|
-
this.flushStderr();
|
|
463
|
-
const tail = this.stderrTail.slice(-3).join(" | ");
|
|
464
|
-
this.onDeath(new ConsolError("server_exited", `${reason}${tail ? `: ${tail}` : ""}`, {
|
|
465
|
-
stderrTail: this.tailText() || undefined,
|
|
466
|
-
...cause === undefined ? {} : { cause }
|
|
467
|
-
}));
|
|
468
|
-
};
|
|
469
|
-
child.once("error", (error) => {
|
|
470
|
-
if (error?.code === "ENOENT") {
|
|
471
|
-
if (this.child !== child)
|
|
472
|
-
return;
|
|
473
|
-
this.onDeath(new ConsolError("binary_not_found", `binary not found: ${this.binary}`, {
|
|
474
|
-
stderrTail: this.attempts.join(`
|
|
475
|
-
`),
|
|
476
|
-
cause: error
|
|
477
|
-
}));
|
|
478
|
-
} else {
|
|
479
|
-
dead(`server failed to start: ${error.message}`, error);
|
|
480
|
-
}
|
|
481
|
-
});
|
|
482
|
-
child.once("exit", (code, signal) => {
|
|
483
|
-
dead(`server exited (${signal ?? code})`);
|
|
484
|
-
});
|
|
485
|
-
child.stdin?.on("error", () => {
|
|
486
|
-
dead("server stdin failed");
|
|
487
|
-
});
|
|
488
|
-
const reader = readline.createInterface({ input: child.stdout, crlfDelay: Infinity });
|
|
489
|
-
this.reader = reader;
|
|
490
|
-
reader.once("close", () => {
|
|
491
|
-
dead("server closed output");
|
|
492
|
-
});
|
|
493
|
-
reader.on("line", (line) => {
|
|
494
|
-
if (this.child === child && generation === this.generation)
|
|
495
|
-
this.onLine(line);
|
|
496
|
-
});
|
|
318
|
+
this.closed.abort();
|
|
319
|
+
this.unsubscribe();
|
|
497
320
|
try {
|
|
498
|
-
|
|
499
|
-
const boot = this.opts.boot;
|
|
500
|
-
if (boot) {
|
|
501
|
-
const op = boot.create && !this.started ? "create_vault" : "open_vault";
|
|
502
|
-
const reply = await this.call(op, this.nextId++, { op, vault: boot.vault }, {});
|
|
503
|
-
const error = replyError(reply);
|
|
504
|
-
if (error)
|
|
505
|
-
throw error;
|
|
506
|
-
}
|
|
507
|
-
this.started = true;
|
|
508
|
-
this.stderrTail = [];
|
|
509
|
-
this.stderrTailBytes = 0;
|
|
321
|
+
await this.engine.close();
|
|
510
322
|
} catch (error) {
|
|
511
|
-
|
|
512
|
-
this.child = null;
|
|
513
|
-
this.reader?.close();
|
|
514
|
-
this.reader = null;
|
|
515
|
-
try {
|
|
516
|
-
child.kill();
|
|
517
|
-
} catch {}
|
|
518
|
-
}
|
|
519
|
-
throw error instanceof ConsolError ? error : new ConsolError("internal", String(error), { cause: error });
|
|
323
|
+
throw engineError(error);
|
|
520
324
|
}
|
|
521
325
|
}
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
326
|
+
async send(frame, options) {
|
|
327
|
+
const { signal } = options;
|
|
328
|
+
if (this.closed.signal.aborted)
|
|
329
|
+
throw new ConsolError("closed", "vault is closed");
|
|
330
|
+
if (signal?.aborted)
|
|
331
|
+
throw new ConsolError("aborted", `${frame.op} aborted before dispatch`, { cause: signal.reason });
|
|
332
|
+
const ms = deadline(options.timeoutMs, this.timeoutMs);
|
|
333
|
+
let text;
|
|
526
334
|
try {
|
|
527
|
-
|
|
528
|
-
} catch {
|
|
529
|
-
|
|
530
|
-
stderrTail: this.tailText() || undefined
|
|
531
|
-
}));
|
|
532
|
-
return;
|
|
533
|
-
}
|
|
534
|
-
const id = msg?.ready === true ? READY : msg?.id;
|
|
535
|
-
if (typeof id !== "number")
|
|
536
|
-
return;
|
|
537
|
-
const pending = this.pending.get(id);
|
|
538
|
-
if (!pending || pending.settled)
|
|
539
|
-
return;
|
|
540
|
-
this.pending.delete(id);
|
|
541
|
-
if (id === READY) {
|
|
542
|
-
pending.resolve(msg.version);
|
|
543
|
-
} else {
|
|
544
|
-
try {
|
|
545
|
-
this.version = checkVersion(msg.reply?.version, this.opts.sdkVersion);
|
|
546
|
-
pending.resolve(msg.reply);
|
|
547
|
-
} catch (error) {
|
|
548
|
-
pending.reject(error);
|
|
549
|
-
}
|
|
335
|
+
text = JSON.stringify(frame);
|
|
336
|
+
} catch (cause) {
|
|
337
|
+
throw new ConsolError("invalid_argument", `${frame.op}: body is not JSON`, { cause });
|
|
550
338
|
}
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
const signal = options.signal;
|
|
556
|
-
if (signal?.aborted) {
|
|
557
|
-
reject(new ConsolError("aborted", `${label} aborted before dispatch`, { cause: signal.reason }));
|
|
558
|
-
return;
|
|
559
|
-
}
|
|
560
|
-
let line = null;
|
|
561
|
-
let timeoutMs;
|
|
562
|
-
try {
|
|
563
|
-
timeoutMs = deadline(options.timeoutMs, this.timeoutMs);
|
|
564
|
-
line = frame ? JSON.stringify({ id, ...frame }) + `
|
|
565
|
-
` : null;
|
|
566
|
-
} catch (error) {
|
|
567
|
-
reject(ConsolError.is(error) ? error : new ConsolError("invalid_argument", `${label}: body is not JSON`, { cause: error }));
|
|
568
|
-
return;
|
|
569
|
-
}
|
|
570
|
-
const input = this.child?.stdin;
|
|
571
|
-
if (line && !input?.writable) {
|
|
572
|
-
reject(new ConsolError(this.closed ? "closed" : "server_exited", "server is not running"));
|
|
573
|
-
return;
|
|
574
|
-
}
|
|
575
|
-
const pending = { resolve, reject, settled: false };
|
|
576
|
-
const done = (error, value) => {
|
|
577
|
-
if (pending.settled)
|
|
578
|
-
return;
|
|
579
|
-
pending.settled = true;
|
|
580
|
-
if (pending.timer !== undefined)
|
|
581
|
-
clearTimeout(pending.timer);
|
|
582
|
-
if (pending.onAbort && signal)
|
|
583
|
-
signal.removeEventListener("abort", pending.onAbort);
|
|
584
|
-
this.pending.delete(id);
|
|
585
|
-
if (error)
|
|
586
|
-
reject(error);
|
|
587
|
-
else
|
|
588
|
-
resolve(value);
|
|
589
|
-
this.updateRef();
|
|
590
|
-
};
|
|
591
|
-
pending.resolve = (value) => done(null, value);
|
|
592
|
-
pending.reject = (error) => done(error);
|
|
593
|
-
if (timeoutMs > 0) {
|
|
594
|
-
pending.timer = setTimeout(() => done(new ConsolError("timeout", `${label} timed out after ${timeoutMs}ms`)), timeoutMs);
|
|
595
|
-
pending.timer.unref?.();
|
|
596
|
-
}
|
|
597
|
-
if (signal) {
|
|
598
|
-
pending.onAbort = () => done(new ConsolError("aborted", `${label} aborted`, { cause: signal.reason }));
|
|
599
|
-
signal.addEventListener("abort", pending.onAbort, { once: true });
|
|
600
|
-
}
|
|
601
|
-
this.pending.set(id, pending);
|
|
602
|
-
this.updateRef();
|
|
603
|
-
if (line) {
|
|
604
|
-
input.write(line, (error) => {
|
|
605
|
-
if (error)
|
|
606
|
-
done(new ConsolError("server_exited", "server stdin failed", { cause: error }));
|
|
607
|
-
});
|
|
608
|
-
}
|
|
609
|
-
});
|
|
610
|
-
}
|
|
611
|
-
async send(frame, options) {
|
|
612
|
-
if (options.signal?.aborted) {
|
|
613
|
-
throw new ConsolError("aborted", `${frame.op} aborted before dispatch`, { cause: options.signal.reason });
|
|
339
|
+
try {
|
|
340
|
+
return JSON.parse(await race(this.engine.request(text), frame.op, ms, this.closed.signal, signal));
|
|
341
|
+
} catch (error) {
|
|
342
|
+
throw engineError(error);
|
|
614
343
|
}
|
|
615
|
-
await this.start();
|
|
616
|
-
return await this.call(frame.op, this.nextId++, frame, options);
|
|
617
344
|
}
|
|
618
345
|
handle(body, ctx, options = {}) {
|
|
619
346
|
return this.send({ op: "handle", ctx: { vaults: ctx.vaults, agent: ctx.agent ?? null }, body }, options);
|
|
@@ -624,7 +351,7 @@ class StdioTransport {
|
|
|
624
351
|
}
|
|
625
352
|
|
|
626
353
|
// src/vault.ts
|
|
627
|
-
var
|
|
354
|
+
var import_node_path2 = require("node:path");
|
|
628
355
|
|
|
629
356
|
// src/types.ts
|
|
630
357
|
function sliceUtf8(text, start, end) {
|
|
@@ -646,7 +373,7 @@ function sliceUtf8(text, start, end) {
|
|
|
646
373
|
}
|
|
647
374
|
|
|
648
375
|
// src/vault.ts
|
|
649
|
-
var CLIENT_INFO = { name: "consol-js-client", version: "0.
|
|
376
|
+
var CLIENT_INFO = { name: "consol-js-client", version: "0.2.1" };
|
|
650
377
|
var LOCAL = "default";
|
|
651
378
|
function payload(tool, reply) {
|
|
652
379
|
const error = replyError(reply, tool);
|
|
@@ -709,19 +436,13 @@ function required(options, fields) {
|
|
|
709
436
|
}
|
|
710
437
|
async function local(options, create) {
|
|
711
438
|
required(options, ["path", "agent"]);
|
|
712
|
-
const path = options.path.startsWith("s3://") ? options.path :
|
|
713
|
-
const transport =
|
|
439
|
+
const path = options.path.startsWith("s3://") ? options.path : import_node_path2.resolve(options.path);
|
|
440
|
+
const transport = await NativeTransport.start({
|
|
714
441
|
...options,
|
|
715
|
-
location:
|
|
442
|
+
location: { vault: path },
|
|
716
443
|
boot: { vault: LOCAL, create },
|
|
717
444
|
sdkVersion: CLIENT_INFO.version
|
|
718
445
|
});
|
|
719
|
-
try {
|
|
720
|
-
await transport.start();
|
|
721
|
-
} catch (error) {
|
|
722
|
-
await transport.close();
|
|
723
|
-
throw error;
|
|
724
|
-
}
|
|
725
446
|
return construct(transport, LOCAL, options.agent);
|
|
726
447
|
}
|
|
727
448
|
function createVault(options) {
|
|
@@ -777,18 +498,11 @@ class Consol {
|
|
|
777
498
|
if (vaultRoot === undefined === (vault === undefined)) {
|
|
778
499
|
throw new ConsolError("invalid_argument", "give exactly one of vaultRoot or vault", { field: "vaultRoot" });
|
|
779
500
|
}
|
|
780
|
-
|
|
501
|
+
return new Consol(await NativeTransport.start({
|
|
781
502
|
...engine,
|
|
782
|
-
location: vaultRoot === undefined ?
|
|
503
|
+
location: vaultRoot === undefined ? { vault } : { vaultRoot },
|
|
783
504
|
sdkVersion: CLIENT_INFO.version
|
|
784
|
-
});
|
|
785
|
-
try {
|
|
786
|
-
await transport.start();
|
|
787
|
-
} catch (error) {
|
|
788
|
-
await transport.close();
|
|
789
|
-
throw error;
|
|
790
|
-
}
|
|
791
|
-
return new Consol(transport);
|
|
505
|
+
}));
|
|
792
506
|
}
|
|
793
507
|
async handle(body, ctx, options) {
|
|
794
508
|
if (ctx === null || typeof ctx !== "object" || ctx.vaults !== null && !Array.isArray(ctx.vaults)) {
|
package/dist/native.d.ts
ADDED
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
import { type CallOptions, type Context, type Transport } from "./transport";
|
|
2
|
+
import type { Reply } from "./types";
|
|
3
|
+
/** Any OpenAI-shaped `/embeddings` endpoint; without it recall stays lexical-only. */
|
|
4
|
+
export interface EmbeddingOptions {
|
|
5
|
+
url: string;
|
|
6
|
+
/** Omit for endpoints that take no key. */
|
|
7
|
+
apiKey?: string;
|
|
8
|
+
model: string;
|
|
9
|
+
dims?: number;
|
|
10
|
+
queryPrefix?: string;
|
|
11
|
+
batchSize?: number;
|
|
12
|
+
timeoutMs?: number;
|
|
13
|
+
}
|
|
14
|
+
/** Jev advice (experimental). */
|
|
15
|
+
export interface JevOptions {
|
|
16
|
+
url?: string;
|
|
17
|
+
apiKey?: string;
|
|
18
|
+
model?: string;
|
|
19
|
+
/** `false` turns automatic triage off. */
|
|
20
|
+
auto?: boolean;
|
|
21
|
+
}
|
|
22
|
+
/**
|
|
23
|
+
* What starts the in-process engine. Options become its settings (the CONSOL_* map); this process's
|
|
24
|
+
* own CONSOL_* variables are never read, except CONSOL_DB_URL as the default database.
|
|
25
|
+
*/
|
|
26
|
+
export interface EngineOptions {
|
|
27
|
+
/** Database URL: PostgreSQL with pgvector, or CockroachDB. Without it this process's `CONSOL_DB_URL` applies; the engine needs one. */
|
|
28
|
+
db?: string;
|
|
29
|
+
embedding?: EmbeddingOptions;
|
|
30
|
+
jev?: JevOptions;
|
|
31
|
+
/** Explicit `consol.node`; otherwise CONSOL_ADDON, then the platform package. */
|
|
32
|
+
addonPath?: string;
|
|
33
|
+
/** Any other engine setting (a CONSOL_* name), applied last. */
|
|
34
|
+
env?: Record<string, string>;
|
|
35
|
+
/** Default per-call deadline in ms (300000); 0 disables it. */
|
|
36
|
+
timeoutMs?: number;
|
|
37
|
+
/** Receives each engine log line. One sink is shared by every engine in the process. */
|
|
38
|
+
onLog?: (line: string) => void;
|
|
39
|
+
/** Which engine lines reach `onLog`: an EnvFilter directive such as `info` or `consol_engine=debug` (default `warn`). Process-wide like the sink: the latest one set wins. */
|
|
40
|
+
logLevel?: string;
|
|
41
|
+
}
|
|
42
|
+
export declare function engineSettings(o: EngineOptions): Record<string, string>;
|
|
43
|
+
/** One opened engine of `consol.node`. A frame is the JSON of `{op, ...}`; the answer is the Reply JSON. */
|
|
44
|
+
export interface NativeEngine {
|
|
45
|
+
request(frame: string): Promise<string>;
|
|
46
|
+
close(): Promise<void>;
|
|
47
|
+
}
|
|
48
|
+
/** What the SDK uses of `consol.node`. Failures reject with an Error whose message is JSON `{code, message, field?}`. */
|
|
49
|
+
export interface Addon {
|
|
50
|
+
version(): string;
|
|
51
|
+
/** `location` is the JSON `{"vaultRoot"}` or `{"vault"}`. */
|
|
52
|
+
openEngine(location: string, settings: Record<string, string>): Promise<NativeEngine>;
|
|
53
|
+
setLogSink(sink: ((line: string) => void) | null, level?: string): void;
|
|
54
|
+
}
|
|
55
|
+
/**
|
|
56
|
+
* Explicit path authoritative (a missing file fails here), then CONSOL_ADDON authoritative, then the
|
|
57
|
+
* platform package resolved relative to this SDK file. CONSOL_ADDON is read only here. Loaded on
|
|
58
|
+
* first use, so importing the SDK never fails.
|
|
59
|
+
*/
|
|
60
|
+
export declare function loadAddon(explicit?: string): Addon;
|
|
61
|
+
export interface NativeOptions extends EngineOptions {
|
|
62
|
+
location: {
|
|
63
|
+
vaultRoot: string;
|
|
64
|
+
} | {
|
|
65
|
+
vault: string;
|
|
66
|
+
};
|
|
67
|
+
/** After opening the engine: create or open this vault. */
|
|
68
|
+
boot?: {
|
|
69
|
+
vault: string;
|
|
70
|
+
create: boolean;
|
|
71
|
+
};
|
|
72
|
+
/** The SDK version the addon's major.minor must match. */
|
|
73
|
+
sdkVersion: string;
|
|
74
|
+
}
|
|
75
|
+
/** The engine of `consol.node` in this process: one frame per call, answered by a Reply JSON, in any order. */
|
|
76
|
+
export declare class NativeTransport implements Transport {
|
|
77
|
+
private readonly engine;
|
|
78
|
+
readonly version: string;
|
|
79
|
+
private readonly timeoutMs;
|
|
80
|
+
private readonly unsubscribe;
|
|
81
|
+
private readonly closed;
|
|
82
|
+
private closing;
|
|
83
|
+
private constructor();
|
|
84
|
+
/** Loads the addon, checks its version, opens the engine and runs the boot step; a failure leaves nothing open. */
|
|
85
|
+
static start(opts: NativeOptions): Promise<NativeTransport>;
|
|
86
|
+
/** Final and idempotent: in-flight and later calls fail with `closed`. Every close() returns the same promise. */
|
|
87
|
+
close(): Promise<void>;
|
|
88
|
+
private shutdown;
|
|
89
|
+
private send;
|
|
90
|
+
/** Runs one envelope under `ctx`. The reply is returned untouched, tool errors included. */
|
|
91
|
+
handle(body: unknown, ctx: Context, options?: CallOptions): Promise<Reply>;
|
|
92
|
+
/** Create or open a vault by name; a refusal arrives as the reply's `vault_exists` / `vault_not_found`. */
|
|
93
|
+
vaultOp(op: "create_vault" | "open_vault", vault: string, options?: CallOptions): Promise<Reply>;
|
|
94
|
+
}
|
package/dist/transport.d.ts
CHANGED
|
@@ -12,9 +12,9 @@ export interface Context {
|
|
|
12
12
|
/** Pins the agent; the request cannot name another one. */
|
|
13
13
|
agent?: string | null;
|
|
14
14
|
}
|
|
15
|
-
/** One way to move an envelope:
|
|
15
|
+
/** One way to move an envelope: the in-process engine or HTTP. Internal: only Vault and Consol are public. */
|
|
16
16
|
export interface Transport {
|
|
17
|
-
/**
|
|
17
|
+
/** Engine version (known at start) or, over HTTP, that of the last reply; null before the first. */
|
|
18
18
|
readonly version: string | null;
|
|
19
19
|
handle(body: unknown, ctx: Context, options?: CallOptions): Promise<Reply>;
|
|
20
20
|
/** Final and idempotent. */
|
|
@@ -24,6 +24,7 @@ export interface Transport {
|
|
|
24
24
|
export declare function deadline(ms: number | undefined, fallback: number): number;
|
|
25
25
|
/** Server major.minor must equal the SDK's; returns the server version. */
|
|
26
26
|
export declare function checkVersion(got: unknown, sdk: string): string;
|
|
27
|
+
export declare const anySignal: (signals: AbortSignal[]) => AbortSignal;
|
|
27
28
|
/** POSTs the envelope to `url`; the caller's headers are the backend's auth. Stateless. */
|
|
28
29
|
export declare class HttpTransport implements Transport {
|
|
29
30
|
private readonly url;
|
package/dist/types.d.ts
CHANGED
|
@@ -208,11 +208,6 @@ export interface SharedFacetLead {
|
|
|
208
208
|
route: "shared_facet";
|
|
209
209
|
value: string;
|
|
210
210
|
}
|
|
211
|
-
/** Pair found through a shared numeric value. */
|
|
212
|
-
export interface SharedNumericLead {
|
|
213
|
-
route: "shared_numeric";
|
|
214
|
-
value: string;
|
|
215
|
-
}
|
|
216
211
|
/** Pair found through embedding similarity (embeddings on only). */
|
|
217
212
|
export interface SemanticLead {
|
|
218
213
|
route: "semantic";
|
|
@@ -224,7 +219,7 @@ export interface SemanticLead {
|
|
|
224
219
|
score: number;
|
|
225
220
|
}
|
|
226
221
|
/** Discovery witness of a pair; one witness, not an exhaustive list or a truth judgment. */
|
|
227
|
-
export type PairLead = AuthoredLead | SharedFacetLead |
|
|
222
|
+
export type PairLead = AuthoredLead | SharedFacetLead | SemanticLead;
|
|
228
223
|
/** Hub witness; `target` is null when a new hub would be formed. Unstable: checked against Rust source only. */
|
|
229
224
|
export interface HubLead {
|
|
230
225
|
route: "hub";
|
package/dist/vault.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { type EngineOptions } from "./
|
|
1
|
+
import { type EngineOptions } from "./native";
|
|
2
2
|
import { type CallOptions } from "./transport";
|
|
3
3
|
import type { ConsolidationDisposition, PairDisposition, ReadResult, RecallResult, Reply, ReflectResult, RememberResult, RemoveResult, Scope, SourceDisposition } from "./types";
|
|
4
4
|
export { sliceUtf8 } from "./types";
|
|
@@ -6,7 +6,7 @@ export declare const CLIENT_INFO: {
|
|
|
6
6
|
name: string;
|
|
7
7
|
version: string;
|
|
8
8
|
};
|
|
9
|
-
/** Local vault: the
|
|
9
|
+
/** Local vault: the engine runs in this process. */
|
|
10
10
|
export interface VaultOptions extends EngineOptions {
|
|
11
11
|
/** Local directory, or `s3://bucket[/prefix]/<name>` (needs `db`; AWS_* from the environment). */
|
|
12
12
|
path: string;
|
|
@@ -97,12 +97,15 @@ export type Decision = {
|
|
|
97
97
|
inspected: InspectedSource | null;
|
|
98
98
|
reason: string;
|
|
99
99
|
};
|
|
100
|
+
/** What a recall question asks for beyond its terms; the caller declares it, in any language. */
|
|
101
|
+
export type Intent = "breadth" | "chronology" | "latest";
|
|
100
102
|
/** Flat wire args for recall. */
|
|
101
103
|
export interface RecallArgs {
|
|
102
104
|
query: string;
|
|
103
105
|
agent?: string;
|
|
104
106
|
include_history?: boolean;
|
|
105
107
|
limit?: number;
|
|
108
|
+
intents?: Intent[];
|
|
106
109
|
}
|
|
107
110
|
/** Flat wire args for read. */
|
|
108
111
|
export interface ReadArgs {
|
|
@@ -151,13 +154,13 @@ export interface ReflectArgs {
|
|
|
151
154
|
limit?: number;
|
|
152
155
|
decisions?: Decision[];
|
|
153
156
|
}
|
|
154
|
-
/** One vault behind a transport (
|
|
157
|
+
/** One vault behind a transport (the in-process engine or a hosted backend). Build it with createVault or openVault; close() is final. */
|
|
155
158
|
export declare class Vault {
|
|
156
159
|
private readonly transport;
|
|
157
160
|
private readonly vault;
|
|
158
161
|
private readonly agent;
|
|
159
162
|
private constructor();
|
|
160
|
-
/**
|
|
163
|
+
/** Engine version (known once opened); a hosted vault's is that of the last reply, null before the first. */
|
|
161
164
|
get serverVersion(): string | null;
|
|
162
165
|
/** Final and idempotent; concurrent calls share one completion. Later calls fail with `closed`. */
|
|
163
166
|
close(): Promise<void>;
|
|
@@ -171,10 +174,10 @@ export declare class Vault {
|
|
|
171
174
|
remove(args: RemoveArgs | ResumeArgs, options?: CallOptions): Promise<RemoveResult>;
|
|
172
175
|
reflect(args?: ReflectArgs, options?: CallOptions): Promise<ReflectResult>;
|
|
173
176
|
}
|
|
174
|
-
/** Creates the vault and
|
|
177
|
+
/** Creates the vault and opens its engine; fails with `vault_exists` when a vault marker exists. */
|
|
175
178
|
export declare function createVault(options: VaultOptions): Promise<Vault>;
|
|
176
179
|
/**
|
|
177
|
-
* Local:
|
|
180
|
+
* Local: opens the engine of an existing vault (`vault_not_found` when no marker exists). Hosted
|
|
178
181
|
* (`url`): no round trip here, a missing vault is `vault_not_found` on the first call.
|
|
179
182
|
*/
|
|
180
183
|
export declare function openVault(options: VaultOptions | HostedOptions): Promise<Vault>;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@keb-org/consol",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.1",
|
|
4
4
|
"description": "Consol memory engine SDK: a local vault, a hosted vault client, and the Consol backend class",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"mcp",
|
|
@@ -39,8 +39,8 @@
|
|
|
39
39
|
"scripts": {
|
|
40
40
|
"build": "node -e \"require('node:fs').rmSync('dist',{recursive:true,force:true})\" && bun build src/index.ts --outfile dist/index.js --target node --format cjs && cp src/index.mjs dist/index.mjs && tsc --emitDeclarationOnly --pretty false",
|
|
41
41
|
"typecheck": "tsc --noEmit --pretty false",
|
|
42
|
-
"test": "bun run build && bun test/pure.ts && node test/http.ts && node test/sdk.ts && tsc --noEmit --pretty false -p test",
|
|
43
|
-
"test:offline": "bun run build && bun test/pure.ts && node test/http.ts && tsc --noEmit --pretty false -p test",
|
|
42
|
+
"test": "bun run build && bun test/pure.ts && node test/http.ts && node test/native.ts && node test/sdk.ts && tsc --noEmit --pretty false -p test",
|
|
43
|
+
"test:offline": "bun run build && bun test/pure.ts && node test/http.ts && node test/native.ts && tsc --noEmit --pretty false -p test",
|
|
44
44
|
"test:consumers": "bun run build && bash consumers/run.sh"
|
|
45
45
|
},
|
|
46
46
|
"devDependencies": {
|
|
@@ -51,10 +51,10 @@
|
|
|
51
51
|
"access": "public"
|
|
52
52
|
},
|
|
53
53
|
"optionalDependencies": {
|
|
54
|
-
"@keb-org/consol-win32-x64": "0.
|
|
55
|
-
"@keb-org/consol-darwin-arm64": "0.
|
|
56
|
-
"@keb-org/consol-darwin-x64": "0.
|
|
57
|
-
"@keb-org/consol-linux-x64": "0.
|
|
58
|
-
"@keb-org/consol-linux-arm64": "0.
|
|
54
|
+
"@keb-org/consol-win32-x64": "0.2.1",
|
|
55
|
+
"@keb-org/consol-darwin-arm64": "0.2.1",
|
|
56
|
+
"@keb-org/consol-darwin-x64": "0.2.1",
|
|
57
|
+
"@keb-org/consol-linux-x64": "0.2.1",
|
|
58
|
+
"@keb-org/consol-linux-arm64": "0.2.1"
|
|
59
59
|
}
|
|
60
60
|
}
|
package/dist/stdio.d.ts
DELETED
|
@@ -1,120 +0,0 @@
|
|
|
1
|
-
import { type CallOptions, type Context, type Transport } from "./transport";
|
|
2
|
-
import type { Reply } from "./types";
|
|
3
|
-
/** Bound of the kept stderr tail and of any one line, in UTF-8 bytes. */
|
|
4
|
-
export declare const STDERR_TAIL_BYTES = 4096;
|
|
5
|
-
/** Any OpenAI-shaped `/embeddings` endpoint; without it recall stays lexical-only. */
|
|
6
|
-
export interface EmbeddingOptions {
|
|
7
|
-
url: string;
|
|
8
|
-
/** Omit for endpoints that take no key. */
|
|
9
|
-
apiKey?: string;
|
|
10
|
-
model: string;
|
|
11
|
-
dims?: number;
|
|
12
|
-
queryPrefix?: string;
|
|
13
|
-
batchSize?: number;
|
|
14
|
-
timeoutMs?: number;
|
|
15
|
-
}
|
|
16
|
-
/** Jev advice (experimental). */
|
|
17
|
-
export interface JevOptions {
|
|
18
|
-
url?: string;
|
|
19
|
-
apiKey?: string;
|
|
20
|
-
model?: string;
|
|
21
|
-
/** `false` turns automatic triage off. */
|
|
22
|
-
auto?: boolean;
|
|
23
|
-
}
|
|
24
|
-
/**
|
|
25
|
-
* What starts the `consol stdio` child. Options become the child's environment (never argv);
|
|
26
|
-
* inherited CONSOL_* and RUST_LOG are stripped first and `env` is applied last.
|
|
27
|
-
*/
|
|
28
|
-
export interface EngineOptions {
|
|
29
|
-
/** Database URL: PostgreSQL with pgvector, or CockroachDB. Without it this process's `CONSOL_DB_URL` applies; the child needs one. */
|
|
30
|
-
db?: string;
|
|
31
|
-
embedding?: EmbeddingOptions;
|
|
32
|
-
jev?: JevOptions;
|
|
33
|
-
/** Explicit binary; otherwise CONSOL_BIN, the platform package, then PATH. */
|
|
34
|
-
binaryPath?: string;
|
|
35
|
-
env?: Record<string, string>;
|
|
36
|
-
/** Default per-call deadline in ms (300000); 0 disables it. */
|
|
37
|
-
timeoutMs?: number;
|
|
38
|
-
/** Receives each child stderr line. Without it lines are only kept for error messages. */
|
|
39
|
-
onLog?: (line: string) => void;
|
|
40
|
-
}
|
|
41
|
-
export declare function engineEnv(o: EngineOptions): Record<string, string>;
|
|
42
|
-
export interface StdioOptions extends EngineOptions {
|
|
43
|
-
/** `--vault PATH` or `--vault-root ROOT`. */
|
|
44
|
-
location: ["--vault" | "--vault-root", string];
|
|
45
|
-
/** After the ready line: create (first start only) or open this vault. A restart always opens. */
|
|
46
|
-
boot?: {
|
|
47
|
-
vault: string;
|
|
48
|
-
create: boolean;
|
|
49
|
-
};
|
|
50
|
-
/** The SDK version the child's major.minor must match. */
|
|
51
|
-
sdkVersion: string;
|
|
52
|
-
}
|
|
53
|
-
export interface BinaryResolution {
|
|
54
|
-
path: string;
|
|
55
|
-
attempts: string[];
|
|
56
|
-
}
|
|
57
|
-
/**
|
|
58
|
-
* Explicit path authoritative (missing file fails here), then CONSOL_BIN authoritative, then the
|
|
59
|
-
* platform package resolved relative to this SDK file, then PATH. CONSOL_BIN is read only here,
|
|
60
|
-
* never sent to the child.
|
|
61
|
-
*/
|
|
62
|
-
export declare function resolveBinary(explicit?: string): BinaryResolution;
|
|
63
|
-
/** Child env: host minus ALL CONSOL_* and RUST_LOG, then options, then explicit env last. */
|
|
64
|
-
export declare function childEnv(base?: Record<string, string>, extra?: Record<string, string>): Record<string, string>;
|
|
65
|
-
/** The last STDERR_TAIL_BYTES bytes of `text`, cut on a character boundary. */
|
|
66
|
-
export declare function lastBytes(text: string): string;
|
|
67
|
-
/**
|
|
68
|
-
* One `consol stdio` child speaking JSON lines (docs W-wire): a ready line, then `{id, op, ...}`
|
|
69
|
-
* frames answered by `{id, reply}` lines in any order. Stdin EOF ends the child gracefully.
|
|
70
|
-
*/
|
|
71
|
-
export declare class StdioTransport implements Transport {
|
|
72
|
-
private readonly opts;
|
|
73
|
-
private readonly attempts;
|
|
74
|
-
private readonly binary;
|
|
75
|
-
private readonly timeoutMs;
|
|
76
|
-
private child;
|
|
77
|
-
private reader;
|
|
78
|
-
private generation;
|
|
79
|
-
private nextId;
|
|
80
|
-
private pending;
|
|
81
|
-
private starting;
|
|
82
|
-
/** After the first successful start, every restart opens the vault instead of creating it. */
|
|
83
|
-
private started;
|
|
84
|
-
private closed;
|
|
85
|
-
private closing;
|
|
86
|
-
private stderrTail;
|
|
87
|
-
private stderrTailBytes;
|
|
88
|
-
version: string | null;
|
|
89
|
-
constructor(opts: StdioOptions);
|
|
90
|
-
private decoder;
|
|
91
|
-
private stderrPartial;
|
|
92
|
-
private onStderr;
|
|
93
|
-
private flushStderr;
|
|
94
|
-
private pushLine;
|
|
95
|
-
private tailText;
|
|
96
|
-
/**
|
|
97
|
-
* Idle children are unref'd so a forgotten close() never keeps Node or Bun alive. Only the process
|
|
98
|
-
* handle toggles (its stdio is unref'd once at spawn): Bun cannot unref a stream again once ref'd.
|
|
99
|
-
*/
|
|
100
|
-
private updateRef;
|
|
101
|
-
private onDeath;
|
|
102
|
-
/**
|
|
103
|
-
* Final and idempotent: in-flight calls reject now, later calls with `closed`. Every close()
|
|
104
|
-
* returns the same promise, settled once the child has exited.
|
|
105
|
-
*/
|
|
106
|
-
close(): Promise<void>;
|
|
107
|
-
private shutdown;
|
|
108
|
-
/** Starts the child and waits for its ready line, unless one is running or starting already. */
|
|
109
|
-
start(): Promise<void>;
|
|
110
|
-
private spawn;
|
|
111
|
-
private onLine;
|
|
112
|
-
/** Waits for the answer to `id`, writing `frame` first unless it is null (the ready line). */
|
|
113
|
-
private call;
|
|
114
|
-
/** One frame; after a crash the next call starts the child once more. Never retried. */
|
|
115
|
-
private send;
|
|
116
|
-
/** Runs one envelope under `ctx`. The reply is returned untouched, tool errors included. */
|
|
117
|
-
handle(body: unknown, ctx: Context, options?: CallOptions): Promise<Reply>;
|
|
118
|
-
/** Create or open a vault by name; a refusal arrives as the reply's `vault_exists` / `vault_not_found`. */
|
|
119
|
-
vaultOp(op: "create_vault" | "open_vault", vault: string, options?: CallOptions): Promise<Reply>;
|
|
120
|
-
}
|