ds4-context-engine 0.2.0-beta.1 → 0.2.0-beta.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -5
- package/docs/ARCHITECTURE.md +11 -2
- package/docs/CONTEXT_MANIFEST.md +1 -1
- package/docs/LOCAL_KV_REUSE.md +131 -0
- package/docs/PORTABLE_CORE.md +8 -5
- package/docs/ROADMAP_0.2.0.md +4 -2
- package/docs/RUNTIME_ADAPTER_KIT.md +6 -4
- package/docs/STORAGE.md +2 -0
- package/package.json +2 -2
- package/src/pi-adapter/version.ts +1 -1
package/README.md
CHANGED
|
@@ -16,7 +16,7 @@ bounded active context with provenance
|
|
|
16
16
|
Pi provider
|
|
17
17
|
```
|
|
18
18
|
|
|
19
|
-
> **Project status:** M0–
|
|
19
|
+
> **Project status:** M0–M20 are implemented on `main`. Prerelease `0.2.0-beta.2` includes M14–M20: quality measurement, structural/hybrid retrieval, cross-session project memory, learned-ranking shadow evaluation, the runtime adapter kit, and opt-in local KV reuse. Stable `0.1.2` remains available; all lines target Pi `0.84.3`.
|
|
20
20
|
|
|
21
21
|
## Why DS4
|
|
22
22
|
|
|
@@ -39,6 +39,7 @@ It provides:
|
|
|
39
39
|
- opt-in metadata-only context-quality metrics and deterministic replay comparisons;
|
|
40
40
|
- checksummed metadata-only learned ranking with shadow mode, canonical classified feedback and static fallback;
|
|
41
41
|
- a versioned runtime adapter contract, reusable conformance kit and non-Pi callback/JSONL reference adapter;
|
|
42
|
+
- opt-in exact-prefix local KV reuse for capable local runtime adapters, with volatile handles and full-replay fallback;
|
|
42
43
|
- an inspectable Context Manifest explaining included and excluded material;
|
|
43
44
|
- fail-open recovery to Pi's native context path for operational failures.
|
|
44
45
|
|
|
@@ -359,7 +360,9 @@ Native continuation is also disabled by default. Enabling it requires both expli
|
|
|
359
360
|
|
|
360
361
|
Eligible OpenAI Responses requests then set `store: true`. Review the provider's retention policy before enabling this option. DS4 keeps response handles only in volatile memory, verifies exact managed prefixes before reuse and retries once with a full managed replay when recognized continuation state is stale.
|
|
361
362
|
|
|
362
|
-
|
|
363
|
+
Local KV reuse is separately disabled by default through `localKvReuse.enabled`. It also requires a local runtime adapter with a versioned `local-kv-reuse` capability and a volatile runtime port. Pi exposes no such handles and remains unsupported even if configuration is enabled.
|
|
364
|
+
|
|
365
|
+
See [`docs/PRIVACY.md`](docs/PRIVACY.md), [`docs/NATIVE_CONTINUATION.md`](docs/NATIVE_CONTINUATION.md), and [`docs/LOCAL_KV_REUSE.md`](docs/LOCAL_KV_REUSE.md).
|
|
363
366
|
|
|
364
367
|
## Storage and recovery
|
|
365
368
|
|
|
@@ -399,7 +402,7 @@ npm pack --dry-run --workspace ds4-context-core
|
|
|
399
402
|
npm pack --dry-run --workspace ds4-context-reference-adapter
|
|
400
403
|
```
|
|
401
404
|
|
|
402
|
-
The test suite covers configuration, migrations, canonical JSONL projection, planning, atomic tool groups, retrieval, compaction, project knowledge, artifacts, memory, privacy, model awareness, continuation, runtime-adapter conformance, the portable-core dependency boundary and Pi extension lifecycle behavior. The package check builds all three tarballs, installs them in a clean temporary consumer, reruns compiled reference-adapter conformance and starts the packaged Pi extension with isolated RPC state.
|
|
405
|
+
The test suite covers configuration, migrations, canonical JSONL projection, planning, atomic tool groups, retrieval, compaction, project knowledge, artifacts, memory, privacy, model awareness, continuation, local-KV eligibility/replay, runtime-adapter conformance, the portable-core dependency boundary and Pi extension lifecycle behavior. The package check builds all three tarballs, installs them in a clean temporary consumer, reruns compiled reference-adapter conformance and starts the packaged Pi extension with isolated RPC state.
|
|
403
406
|
|
|
404
407
|
### Portable core
|
|
405
408
|
|
|
@@ -436,6 +439,7 @@ scripts package and release-readiness checks
|
|
|
436
439
|
- [Native continuation](docs/NATIVE_CONTINUATION.md)
|
|
437
440
|
- [Portable core](docs/PORTABLE_CORE.md)
|
|
438
441
|
- [Runtime adapter kit](docs/RUNTIME_ADAPTER_KIT.md)
|
|
442
|
+
- [Local KV reuse](docs/LOCAL_KV_REUSE.md)
|
|
439
443
|
- [Storage](docs/STORAGE.md)
|
|
440
444
|
- [Roadmap 0.2.0](docs/ROADMAP_0.2.0.md)
|
|
441
445
|
- [Release process](docs/RELEASING.md)
|
|
@@ -444,9 +448,9 @@ scripts package and release-readiness checks
|
|
|
444
448
|
|
|
445
449
|
## Roadmap
|
|
446
450
|
|
|
447
|
-
The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation
|
|
451
|
+
The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation, M19's runtime adapter/conformance kit, and M20 opt-in local KV eligibility/replay are implemented on `main`. Learned active ranking remains promotion-gated, Pi reports local KV as unsupported, and static ranking/native completion stay authoritative on every failure.
|
|
448
452
|
|
|
449
|
-
The
|
|
453
|
+
The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) now proceeds to release-candidate hardening. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
|
|
450
454
|
|
|
451
455
|
## Contributing
|
|
452
456
|
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -66,6 +66,15 @@ optional OpenAI Responses provider wrapper
|
|
|
66
66
|
-> retry a rejected stale handle once through the complete managed replay before exposing stream events
|
|
67
67
|
-> record metadata-only mode/item counts/retry/invalidation diagnostics; never record the provider handle
|
|
68
68
|
|
|
69
|
+
optional local runtime KV port (non-Pi adapters)
|
|
70
|
+
-> require opt-in configuration plus a negotiated versioned local-KV capability
|
|
71
|
+
-> apply current privacy policy before runtime-specific prefix/options extraction
|
|
72
|
+
-> hash exact prefix bytes, provider/model/revision, options, privacy policy, runtime revision and capability version
|
|
73
|
+
-> let the runtime port map only the fingerprint to its volatile native handle
|
|
74
|
+
-> replay the complete sanitized payload after miss, stale rejection, unavailable state or runtime restart
|
|
75
|
+
-> report aggregate hits/misses/saved-prefill/replay latency separately from context occupancy
|
|
76
|
+
-> never place handles, prefixes or payloads in canonical history, manifests, SQLite or diagnostics
|
|
77
|
+
|
|
69
78
|
assistant message_end
|
|
70
79
|
-> attach uncached input plus cache read/write usage to the pending manifest
|
|
71
80
|
-> append one exact provider/model calibration sample
|
|
@@ -162,7 +171,7 @@ ds4-context-engine ds4-context-reference-adapter
|
|
|
162
171
|
|
|
163
172
|
`ds4-context-core` is compiled ESM and has no dependency on Pi. Its workspace contains:
|
|
164
173
|
|
|
165
|
-
- `packages/core/src/adapter`: versioned runtime contract, canonical tool-group validation, isolated capability negotiation and framework-neutral conformance runner;
|
|
174
|
+
- `packages/core/src/adapter`: versioned runtime contract, canonical tool-group validation, isolated capability negotiation, exact local-KV eligibility/replay orchestration and framework-neutral conformance runner;
|
|
166
175
|
- `packages/core/src/core`: portable canonical messages, model profiles, robust calibration, adaptive category limits, budgets and token-estimation policy;
|
|
167
176
|
- `packages/core/src/continuation`: hashed-prefix continuation decisions without provider transport or response APIs;
|
|
168
177
|
- `packages/core/src/config`: runtime-neutral configuration model and filesystem loader;
|
|
@@ -178,7 +187,7 @@ ds4-context-engine ds4-context-reference-adapter
|
|
|
178
187
|
- `packages/core/src/persistence`: rebuildable session/project/vector/memory/pin/quality SQLite state, repositories, FTS5, event replay and transactional migrations;
|
|
179
188
|
- `packages/core/src/manifest` and `packages/core/src/shared`: runtime-neutral projections, provenance, hashing, stable serialization and logging.
|
|
180
189
|
|
|
181
|
-
The `packages/reference-adapter` workspace is the non-Pi reference adapter: it reads bounded append-only canonical JSONL, injects completion through a host callback, enforces privacy at that callback boundary, rebuilds disposable snapshots and explicitly disables unsupported native features.
|
|
190
|
+
The `packages/reference-adapter` workspace is the non-Pi reference adapter: it reads bounded append-only canonical JSONL, injects completion through a host callback, enforces privacy at that callback boundary, rebuilds disposable snapshots and explicitly disables unsupported native features. A local host can inject a handle-free `LocalKvRuntimePort`; the port alone retains native handles and transport while core receives only exact prefix bytes transiently for hashing.
|
|
182
191
|
|
|
183
192
|
The root `ds4-context-engine` package is the Pi adapter:
|
|
184
193
|
|
package/docs/CONTEXT_MANIFEST.md
CHANGED
|
@@ -27,7 +27,7 @@ A Context Manifest explains the context visible at DS4's Pi `context` hook witho
|
|
|
27
27
|
- finalized uncached input, cache-read, cache-write, total provider input, and cache shares when available;
|
|
28
28
|
- optional native-continuation eligibility, storage-consent state, request mode, full/sent/omitted input-item counts, state age, generic fallback/invalidation reason, and managed-replay retry outcome.
|
|
29
29
|
|
|
30
|
-
The manifest does **not** contain system instructions, message text, classified spans, pin content, memory claims, project snippets, artifact content/excerpts, learned-ranking feature vectors/labels/candidate IDs/model weights, tool arguments/results, image data, provider payloads, provider response/conversation IDs, API keys, or headers.
|
|
30
|
+
The manifest does **not** contain system instructions, message text, classified spans, pin content, memory claims, project snippets, artifact content/excerpts, learned-ranking feature vectors/labels/candidate IDs/model weights, local-KV prefixes/fingerprints/handles, tool arguments/results, image data, provider payloads, provider response/conversation IDs, API keys, or headers. Local-KV hit/miss/prefill counters are volatile adapter diagnostics and are not copied into Pi manifests.
|
|
31
31
|
|
|
32
32
|
## Provenance mapping
|
|
33
33
|
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# Local KV Reuse
|
|
2
|
+
|
|
3
|
+
M20 adds an opt-in, runtime-neutral prefix/KV capability for local inference runtimes. KV state is only an inference optimization: it is not canonical history, memory, retrieval evidence, a Context Manifest source, or a SQLite projection.
|
|
4
|
+
|
|
5
|
+
Pi does not expose native KV handles and continues to report `local-kv-reuse` as unsupported. The callback/JSONL reference adapter can exercise the capability when its host injects a `LocalKvRuntimePort`; no provider SDK is added to core or the reference package.
|
|
6
|
+
|
|
7
|
+
## Boundary
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
runtime/provider payload
|
|
11
|
+
↓
|
|
12
|
+
adapter privacy enforcement
|
|
13
|
+
↓
|
|
14
|
+
runtime-specific prefix/options extraction from sanitized payload
|
|
15
|
+
↓
|
|
16
|
+
core exact-byte eligibility hashes
|
|
17
|
+
↓
|
|
18
|
+
volatile LocalKvRuntimePort lookup + transport
|
|
19
|
+
├─ exact hit ───────────────> cached completion
|
|
20
|
+
└─ miss/rejected/unavailable -> full prompt replay
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Core receives the provider-ready prefix only long enough to hash it. It returns metadata-only component hashes and an aggregate fingerprint. A runtime port maps that fingerprint to its own volatile handle. Handles never cross the port, and core has no API for serializing them.
|
|
24
|
+
|
|
25
|
+
## Configuration and negotiation
|
|
26
|
+
|
|
27
|
+
Configuration is disabled by default:
|
|
28
|
+
|
|
29
|
+
```json
|
|
30
|
+
{
|
|
31
|
+
"localKvReuse": {
|
|
32
|
+
"enabled": false
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Enabling configuration is not sufficient. The active adapter must independently declare a versioned `local-kv-reuse` capability and provide a `LocalKvRuntimePort`. Unsupported or malformed declarations disable KV reuse without affecting history, privacy, completion fallback, or any other capability.
|
|
38
|
+
|
|
39
|
+
A reference-adapter host opts in explicitly:
|
|
40
|
+
|
|
41
|
+
```ts
|
|
42
|
+
const adapter = new JsonlReferenceRuntimeAdapter({
|
|
43
|
+
// canonical history, model, privacy, and ordinary transport options...
|
|
44
|
+
localKv: {
|
|
45
|
+
enabled: true,
|
|
46
|
+
port: myVolatileKvPort,
|
|
47
|
+
runtimeRevision: "llama-runtime-build-42",
|
|
48
|
+
modelRevision: "model-checksum-abc",
|
|
49
|
+
prepare(sanitizedPayload) {
|
|
50
|
+
return {
|
|
51
|
+
promptPrefix: exactProviderPrefixBytes(sanitizedPayload),
|
|
52
|
+
systemOptions: providerSystemOptions(sanitizedPayload),
|
|
53
|
+
toolOptions: providerToolOptions(sanitizedPayload),
|
|
54
|
+
prefixTokenCount: 32_000,
|
|
55
|
+
contextTokenCount: 40_000,
|
|
56
|
+
};
|
|
57
|
+
},
|
|
58
|
+
},
|
|
59
|
+
});
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
`prepare()` receives only the payload returned by adapter privacy enforcement. Remote destinations never enter the local KV path.
|
|
63
|
+
|
|
64
|
+
## Exact eligibility
|
|
65
|
+
|
|
66
|
+
`deriveLocalKvEligibility()` uses `local-kv-eligibility-v1`. An eligible fingerprint includes independent SHA-256 components for:
|
|
67
|
+
|
|
68
|
+
- exact provider, model, and model revision;
|
|
69
|
+
- exact UTF-8/string or byte prefix;
|
|
70
|
+
- deterministic JSON-compatible system and tool options;
|
|
71
|
+
- privacy policy version and local destination;
|
|
72
|
+
- runtime identity, runtime revision, and capability implementation version.
|
|
73
|
+
|
|
74
|
+
Identifiers are not case-folded or wildcard-matched. Prefix bytes are not normalized. Option object keys are ordered only for deterministic serialization; values, array order, missing/undefined values, and system/tool separation remain significant. Functions, symbols, binary option values, accessors, custom prototypes, circular values, and non-finite numbers are ineligible instead of being hashed ambiguously.
|
|
75
|
+
|
|
76
|
+
Any changed prefix byte, provider/model identity, model revision, system/tool option, privacy version, runtime revision, or capability version produces a different fingerprint. Disabled configuration, unsupported capability, remote destination, invalid identifiers, an empty prefix, or invalid token counts bypass reuse.
|
|
77
|
+
|
|
78
|
+
## Runtime port and replay
|
|
79
|
+
|
|
80
|
+
`LocalKvRuntimePort` deliberately exposes no handle:
|
|
81
|
+
|
|
82
|
+
```ts
|
|
83
|
+
interface LocalKvRuntimePort {
|
|
84
|
+
tryReuse(request): Promise<
|
|
85
|
+
| { status: "hit"; output: unknown; savedPrefillTokens: number; prefillLatencyMs: number }
|
|
86
|
+
| { status: "miss" | "rejected" | "unavailable" }
|
|
87
|
+
>;
|
|
88
|
+
fullReplay(request): Promise<{
|
|
89
|
+
output: unknown;
|
|
90
|
+
prefillTokens: number;
|
|
91
|
+
prefillLatencyMs: number;
|
|
92
|
+
}>;
|
|
93
|
+
shutdown?(): Promise<void>;
|
|
94
|
+
}
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
The runtime owns fingerprint-to-handle lookup, eviction, provider transport, and restart cleanup. `LocalKvReuseController` treats thrown lookup errors and malformed lookup results as unavailable state. Every miss, stale rejection, unavailable state, or runtime restart invokes `fullReplay()` with the complete sanitized payload. If the injected KV full replay itself fails, the reference adapter retries the complete sanitized payload through its ordinary native transport.
|
|
98
|
+
|
|
99
|
+
Adapter shutdown calls optional port shutdown once. Cache deletion, eviction, process loss, or runtime restart therefore changes performance only; it cannot change canonical state or prevent a full replay.
|
|
100
|
+
|
|
101
|
+
## Aggregate diagnostics
|
|
102
|
+
|
|
103
|
+
`local-kv-diagnostics-v1` contains numbers only:
|
|
104
|
+
|
|
105
|
+
- requests, eligible requests, and bypasses;
|
|
106
|
+
- hits, misses, rejected states, unavailable states, full replays, and transport failures;
|
|
107
|
+
- reusable prefix tokens and total context-occupancy tokens;
|
|
108
|
+
- saved prefill tokens;
|
|
109
|
+
- full-replay prefill tokens and runtime-reported prefill latency.
|
|
110
|
+
|
|
111
|
+
Diagnostics never include provider/model text, prefix or payload content, component fingerprints, runtime handles, credentials, provider response IDs, or output. Per-request completion metadata keeps `contextTokens`, `savedPrefillTokens`, `replayPrefillTokens`, and `prefillLatencyMs` as separate fields; context occupancy is never reported as cache savings.
|
|
112
|
+
|
|
113
|
+
## Privacy and persistence
|
|
114
|
+
|
|
115
|
+
Privacy enforcement precedes runtime-specific prefix extraction and hash verification. A remote payload is sanitized and sent through ordinary transport without consulting local KV state. An enabled local policy is still applied before extraction. A prior cached state cannot authorize material excluded by the current privacy policy because the policy version and sanitized exact prefix are both part of eligibility.
|
|
116
|
+
|
|
117
|
+
No KV handle or prefix copy is written to:
|
|
118
|
+
|
|
119
|
+
- Pi JSONL or reference canonical JSONL;
|
|
120
|
+
- Context Manifests;
|
|
121
|
+
- DS4 SQLite;
|
|
122
|
+
- learned-ranking artifacts;
|
|
123
|
+
- adapter/core diagnostics.
|
|
124
|
+
|
|
125
|
+
There is no M20 database migration. Deleting all volatile runtime KV state requires no DS4 rebuild and no canonical-history migration.
|
|
126
|
+
|
|
127
|
+
## Verification and benchmark
|
|
128
|
+
|
|
129
|
+
Unit and reference-adapter integration tests cover exact hits, changed bytes/options, provider/model/privacy/runtime invalidation, cache loss, stale rejection, unavailable state, full replay, privacy-before-verification, aggregate-only diagnostics, idempotent shutdown, and unchanged canonical JSONL.
|
|
130
|
+
|
|
131
|
+
`tests/benchmarks/local-kv.bench.ts` hashes a synthetic 32,000-token reusable prefix inside a 40,000-token context and separately benchmarks a warm completion whose simulated runtime metadata reports 32,000 saved prefill tokens and `0.5 ms` prefill latency. On the development host, exact eligibility averaged about `0.259 ms` and the warm controller path about `0.248 ms`. These are observational, not portable provider-latency guarantees; the benchmark intentionally reports runtime prefill latency/token savings separately from context occupancy.
|
package/docs/PORTABLE_CORE.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Portable Core
|
|
2
2
|
|
|
3
|
-
M13 extracts the runtime-neutral implementation into the independently buildable `ds4-context-core` workspace package. The root `ds4-context-engine` package remains the Pi integration. M19 adds the versioned adapter contract/conformance kit in core and a separately compiled non-Pi callback/JSONL reference adapter.
|
|
3
|
+
M13 extracts the runtime-neutral implementation into the independently buildable `ds4-context-core` workspace package. The root `ds4-context-engine` package remains the Pi integration. M19 adds the versioned adapter contract/conformance kit in core and a separately compiled non-Pi callback/JSONL reference adapter. M20 adds exact local-KV eligibility and replay orchestration while leaving every provider SDK and native cache handle in the runtime adapter.
|
|
4
4
|
|
|
5
5
|
## Dependency rule
|
|
6
6
|
|
|
@@ -30,6 +30,7 @@ The core may use Node.js standard-library facilities such as `node:sqlite`, file
|
|
|
30
30
|
- memory/pin materialization;
|
|
31
31
|
- privacy classification and provider policy;
|
|
32
32
|
- native-continuation eligibility and hash state;
|
|
33
|
+
- exact local-KV eligibility, handle-free runtime port types, replay policy and aggregate diagnostics;
|
|
33
34
|
- rebuildable SQLite repositories;
|
|
34
35
|
- stable serialization, hashing and logging.
|
|
35
36
|
|
|
@@ -54,10 +55,11 @@ Every `runtime-adapter-v1` implementation must:
|
|
|
54
55
|
6. append memory/pin mutations to canonical runtime history before materializing derived state;
|
|
55
56
|
7. invoke model completion at the adapter boundary for generated summaries;
|
|
56
57
|
8. enforce privacy immediately before provider transport;
|
|
57
|
-
9.
|
|
58
|
-
10.
|
|
58
|
+
9. retain optional local-KV handles inside its volatile runtime port and replay the full sanitized payload after every non-hit;
|
|
59
|
+
10. discard or rebuild SQLite and artifact projections safely;
|
|
60
|
+
11. fall back to native runtime behavior when operational integration fails.
|
|
59
61
|
|
|
60
|
-
The executable contract, independent capability negotiation and framework-neutral seven-case conformance runner are exported from `ds4-context-core/adapter/runtime-adapter` and `ds4-context-core/adapter/conformance`. See [Runtime Adapter Kit](RUNTIME_ADAPTER_KIT.md).
|
|
62
|
+
The executable contract, independent capability negotiation and framework-neutral seven-case conformance runner are exported from `ds4-context-core/adapter/runtime-adapter` and `ds4-context-core/adapter/conformance`. M20's handle-free port and controller are exported from `ds4-context-core/adapter/local-kv`. See [Runtime Adapter Kit](RUNTIME_ADAPTER_KIT.md) and [Local KV Reuse](LOCAL_KV_REUSE.md).
|
|
61
63
|
|
|
62
64
|
## Build and exports
|
|
63
65
|
|
|
@@ -91,6 +93,7 @@ Extraction does not change DS4 state semantics:
|
|
|
91
93
|
- Pi JSONL remains canonical for Pi sessions;
|
|
92
94
|
- SQLite remains disposable and rebuildable;
|
|
93
95
|
- compaction remains non-destructive and strictly validated;
|
|
94
|
-
- provider continuation handles remain volatile;
|
|
96
|
+
- provider continuation and local-KV handles remain volatile and adapter-owned;
|
|
97
|
+
- local-KV cache loss changes only prefill performance and transparently replays the full sanitized payload;
|
|
95
98
|
- planner, retrieval, compaction and persistence failures still fail open at the adapter boundary;
|
|
96
99
|
- enabled privacy enforcement still fails closed before remote transport.
|
package/docs/ROADMAP_0.2.0.md
CHANGED
|
@@ -163,7 +163,7 @@ If the gate is not met, 0.2.0 ships shadow mode but keeps static ranking active.
|
|
|
163
163
|
|
|
164
164
|
## M19 — Runtime Adapter Kit
|
|
165
165
|
|
|
166
|
-
Status: **implemented on `main`
|
|
166
|
+
Status: **implemented on `main` and released in `0.2.0-beta.1`**. Core ships `runtime-adapter-v1` plus a framework-neutral conformance runner; Pi exposes contract capability diagnostics, and the separately packaged callback/JSONL reference adapter passes all seven conformance cases.
|
|
167
167
|
|
|
168
168
|
### Deliverables
|
|
169
169
|
|
|
@@ -182,6 +182,8 @@ Status: **implemented on `main` for `0.2.0-beta.2`**. Core ships `runtime-adapte
|
|
|
182
182
|
|
|
183
183
|
## M20 — Local KV Capability
|
|
184
184
|
|
|
185
|
+
Status: **implemented on `main` and released in `0.2.0-beta.2` behind `localKvReuse.enabled`**. Core derives exact metadata-only eligibility fingerprints and guarantees full replay on every non-hit; injected runtime ports retain all volatile handles and transport. Pi remains explicitly unsupported.
|
|
186
|
+
|
|
185
187
|
### Deliverables
|
|
186
188
|
|
|
187
189
|
- An optional adapter capability for local inference runtimes that expose reusable prefix/KV state.
|
|
@@ -231,10 +233,10 @@ SQLite schema changes use forward migrations plus complete rebuild tests from ca
|
|
|
231
233
|
|
|
232
234
|
- M18 learned ranker in shadow mode.
|
|
233
235
|
- Privacy, rebuild, corruption and performance hardening for M14–M18.
|
|
236
|
+
- M19 adapter contract, conformance kit and reference adapter.
|
|
234
237
|
|
|
235
238
|
### `0.2.0-beta.2`
|
|
236
239
|
|
|
237
|
-
- M19 adapter contract, conformance kit and reference adapter.
|
|
238
240
|
- M20 local KV capability for runtimes that support it.
|
|
239
241
|
|
|
240
242
|
### `0.2.0-rc.1`
|
|
@@ -41,6 +41,8 @@ Applying a DS4 context plan must not write provider-facing synthetic messages ba
|
|
|
41
41
|
|
|
42
42
|
Completion output remains runtime-owned and is not automatically canonical. An adapter must append a final runtime-native message through its ordinary canonical history API if the host chooses to persist it.
|
|
43
43
|
|
|
44
|
+
M20-capable adapters may additionally expose `localKvPort` and aggregate `localKvDiagnostics()`. Both are optional because most runtimes, including Pi, expose no native KV state. The port interface carries eligibility hashes and complete sanitized replay payloads but never returns or accepts a cache handle.
|
|
45
|
+
|
|
44
46
|
## Tool atomicity
|
|
45
47
|
|
|
46
48
|
`buildCanonicalToolAtomicGroups()` connects every canonical `toolCall` block to all matching `toolResult` blocks. Calls sharing one assistant message form one component, so parallel tool batches cannot be split. `validateRuntimeHistorySnapshot()` rejects missing, overlapping, unknown, or mismatched groups. An incomplete live call is represented by a group with `complete: false`; it cannot be treated as a complete selectable exchange.
|
|
@@ -62,7 +64,7 @@ A supported capability requires a non-empty implementation version. Unsupported
|
|
|
62
64
|
- a duplicate, missing, or unversioned declaration is malformed and disabled;
|
|
63
65
|
- no failure in one capability disables canonical history, privacy, completion fallback, or another valid capability.
|
|
64
66
|
|
|
65
|
-
M19
|
|
67
|
+
M19 introduced negotiation for `local-kv-reuse`; M20 adds `local-kv-eligibility-v1`, `LocalKvReuseController`, and the handle-free `LocalKvRuntimePort`. KV handles are never part of canonical history, core state, manifests, or diagnostics. See [Local KV Reuse](LOCAL_KV_REUSE.md).
|
|
66
68
|
|
|
67
69
|
Pi advertises versioned compaction, OpenAI Responses continuation, and embedding-port support. It explicitly reports local KV reuse as unavailable. `/context adapter` shows the complete negotiation; `/context status` shows aggregate enabled/disabled counts.
|
|
68
70
|
|
|
@@ -113,7 +115,7 @@ The callback JSONL target was selected because it covers the full contract while
|
|
|
113
115
|
|
|
114
116
|
Its `ds4-runtime-session-v1` file is canonical and append-only. The header binds runtime ID, session ID, and canonical project root. Message records contain DS4 canonical messages. `createReferenceHistory()` refuses overwrite, `appendReferenceHistoryMessage()` checks provenance before append, and `rebuildDerivedState()` discards only the in-memory snapshot. File size and message count are bounded. Files are mode `0600` where supported.
|
|
115
117
|
|
|
116
|
-
The reference adapter provides a callback completion transport, enabled fail-closed privacy by default, optional `EmbeddingPort
|
|
118
|
+
The reference adapter provides a callback completion transport, enabled fail-closed privacy by default, and optional `EmbeddingPort`. Native compaction and provider continuation remain explicitly unsupported. Local KV reuse is unsupported by default but becomes a versioned supported capability when the host injects a `LocalKvRuntimePort`; configuration remains disabled until explicitly enabled.
|
|
117
119
|
|
|
118
120
|
## Packaging rules
|
|
119
121
|
|
|
@@ -138,6 +140,6 @@ npm pack --dry-run
|
|
|
138
140
|
|
|
139
141
|
## Limitations and rollback
|
|
140
142
|
|
|
141
|
-
The reference adapter is intentionally synchronous-history/callback-completion infrastructure: it does not implement streaming, native compaction, branch editing, provider continuation, or KV
|
|
143
|
+
The reference adapter is intentionally synchronous-history/callback-completion infrastructure: it does not implement streaming, native compaction, branch editing, provider continuation, or a built-in provider-specific KV cache. Its optional KV path requires a host-owned port that retains handles and transport. Unsupported features remain disabled and visible rather than emulated.
|
|
142
144
|
|
|
143
|
-
Removing the reference package does not affect Pi or core. A runtime can roll back its adapter by stopping it and returning to native history/context behavior; no canonical migration or SQLite downgrade is required. Deleting adapter caches is safe. Never delete the runtime-owned JSONL file as part of DS4 rollback.
|
|
145
|
+
Removing the reference package does not affect Pi or core. A runtime can roll back its adapter by stopping it and returning to native history/context behavior; no canonical migration or SQLite downgrade is required. Deleting adapter caches is safe and produces a transparent full replay. Never delete the runtime-owned JSONL file as part of DS4 rollback.
|
package/docs/STORAGE.md
CHANGED
|
@@ -12,6 +12,8 @@ The extension never edits or rewrites Pi JSONL or project source files. Manual m
|
|
|
12
12
|
|
|
13
13
|
The M19 non-Pi reference adapter owns a separate `ds4-runtime-session-v1` JSONL source selected by its host runtime. Its header binds runtime/session identity and the exact canonical project root; following records contain provenance-checked canonical messages. DS4 snapshots and capability diagnostics are disposable. `createReferenceHistory()` refuses overwrite, append uses a dedicated provenance-checked operation, files are mode `0600` where supported, and rebuild never edits this runtime-owned canonical file. Reference JSONL is not imported into Pi or `context.db`.
|
|
14
14
|
|
|
15
|
+
M20 local KV state is entirely runtime-owned and volatile. Core returns only an in-memory eligibility fingerprint to the runtime port; it has no cache-handle field or serialization API. Prefixes, fingerprints, handles and provider outputs are absent from Pi/reference JSONL, Context Manifests, ranking artifacts and every SQLite table. Aggregate hit/miss/prefill counters live only on the adapter controller, and a restart safely resets them with the runtime cache. M20 adds no database migration.
|
|
16
|
+
|
|
15
17
|
## Session index
|
|
16
18
|
|
|
17
19
|
Each parsed entry stores:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ds4-context-engine",
|
|
3
|
-
"version": "0.2.0-beta.
|
|
3
|
+
"version": "0.2.0-beta.2",
|
|
4
4
|
"description": "Non-destructive, provider-independent context management for Pi.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -55,7 +55,7 @@
|
|
|
55
55
|
]
|
|
56
56
|
},
|
|
57
57
|
"dependencies": {
|
|
58
|
-
"ds4-context-core": "0.2.0-beta.
|
|
58
|
+
"ds4-context-core": "0.2.0-beta.2"
|
|
59
59
|
},
|
|
60
60
|
"peerDependencies": {
|
|
61
61
|
"@earendil-works/pi-ai": "0.84.3",
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
export const EXTENSION_VERSION = "0.2.0-beta.
|
|
1
|
+
export const EXTENSION_VERSION = "0.2.0-beta.2";
|
|
2
2
|
export const SUPPORTED_PI_VERSION = "0.84.3";
|
|
3
3
|
export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
|
|
4
4
|
export const PLANNER_VERSION = "managed-learned-ranking-v1";
|