@weftlet/toolkit 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. package/README.md +15 -0
  2. package/dist/application/application.js +7 -4
  3. package/dist/application/cache-adoption.d.ts +43 -0
  4. package/dist/application/cache-adoption.js +129 -0
  5. package/dist/application/extraction-identity.d.ts +6 -0
  6. package/dist/application/extraction-identity.js +15 -0
  7. package/dist/application/hydration-failure.d.ts +41 -0
  8. package/dist/application/hydration-failure.js +43 -0
  9. package/dist/application/hydration-plan.d.ts +4 -7
  10. package/dist/application/hydration-plan.js +7 -15
  11. package/dist/application/hydrator.d.ts +3 -5
  12. package/dist/application/hydrator.js +23 -48
  13. package/dist/application/overlay.js +10 -3
  14. package/dist/application/saved-evidence.d.ts +6 -3
  15. package/dist/application/saved-evidence.js +7 -1
  16. package/dist/application/semantic-artifact-store.d.ts +4 -0
  17. package/dist/application/semantic-artifact-store.js +7 -0
  18. package/dist/application/semantic-run.d.ts +29 -0
  19. package/dist/application/semantic-run.js +111 -0
  20. package/dist/application/semantic-scope.d.ts +13 -0
  21. package/dist/application/semantic-scope.js +12 -0
  22. package/dist/application/snapshot-checks.d.ts +2 -1
  23. package/dist/application/snapshot-checks.js +8 -1
  24. package/dist/configuration/cache.d.ts +6 -0
  25. package/dist/configuration/cache.js +18 -0
  26. package/dist/extraction/document.js +1 -1
  27. package/dist/extraction/providers.js +10 -3
  28. package/dist/extraction/repository-nodes.d.ts +2 -3
  29. package/dist/extraction/repository-nodes.js +5 -11
  30. package/dist/extraction/routing.d.ts +1 -1
  31. package/dist/extraction/routing.js +3 -6
  32. package/dist/extraction/semantic-cache.d.ts +1 -0
  33. package/dist/extraction/semantic-cache.js +9 -2
  34. package/dist/extraction/semantic-diagnostics.d.ts +9 -0
  35. package/dist/extraction/semantic-diagnostics.js +15 -0
  36. package/dist/extraction/semantic.d.ts +5 -1
  37. package/dist/extraction/semantic.js +39 -6
  38. package/dist/extraction/static.d.ts +13 -5
  39. package/dist/extraction/static.js +8 -15
  40. package/dist/governance/governance.js +11 -7
  41. package/dist/graph/query.js +2 -0
  42. package/dist/index.d.ts +3 -0
  43. package/dist/index.js +3 -0
  44. package/dist/interfaces/cli.js +44 -21
  45. package/dist/storage/storage.d.ts +7 -1
  46. package/dist/storage/storage.js +15 -3
  47. package/dist/visualization/visualization.js +1 -0
  48. package/docs/ADR-016-hydration-preview.md +1 -1
  49. package/docs/ADR-017-shacl-validation-union.md +1 -1
  50. package/docs/ADR-018-operational-semantic-cache.md +100 -0
  51. package/docs/ADR-019-semantic-failure-diagnostics.md +82 -0
  52. package/docs/ADR-020-canonical-physical-file-producers.md +97 -0
  53. package/docs/PROVIDERS.md +51 -1
  54. package/docs/RELEASE-1.0-PLAN.md +70 -21
  55. package/docs/RELEASING.md +4 -2
  56. package/docs/ROADMAP.md +31 -12
  57. package/docs/VALIDATION.md +212 -4
  58. package/docs/extraction-manifest.json +42 -21
  59. package/package.json +2 -2
@@ -0,0 +1,82 @@
1
+ # ADR-019: Safe semantic failure diagnostics and settled progress
2
+
3
+ Status: implemented in development source after 0.3.0; validation and publication are recorded
4
+ separately in [validation](VALIDATION.md). This decision does not change extraction identity.
5
+
6
+ ## Problem
7
+
8
+ A generic timeout does not identify which source failed, how much work survived, or how to
9
+ resume. Concurrent workers can also cancel each other: reporting the last error can hide the
10
+ initiating timeout, and reporting before active writes finish can undercount retained artifacts.
11
+ Arbitrary provider exceptions and abort reasons may contain private input.
12
+
13
+ ## Decision
14
+
15
+ Keep semantic-run scheduling and reporting in `application/semantic-run.ts`, separate from graph
16
+ assembly. Record the first source failure before aborting siblings, stop scheduling queued sources,
17
+ and wait for active workers to settle before constructing `SemanticHydrationError.report`.
18
+ The report has a versioned kind, safe code/category, phase, committed SHA, configuration digest,
19
+ elapsed time and scope, effective timeout and its scope, final counts, and recovery guidance.
20
+
21
+ Completion means a validated artifact was retained or successfully reused. A write already in
22
+ progress can finish after cancellation and remains a completion. A provider result returned after
23
+ abort is rejected before validation/publication. Late storage failures remain failures, not sibling
24
+ cancellations. Cancellation between sources has phase `semantic-scheduling`, no failing source ID,
25
+ and run elapsed time; it does not relabel successfully retained artifacts as cancelled.
26
+
27
+ Counts have these meanings:
28
+
29
+ - `completed = extracted + reused`.
30
+ - `total = completed + failed + cancelled + notStarted` after active workers settle.
31
+ - `remaining = total - completed`; this includes failures, cancellations and queued work. It is
32
+ not a promise about the number of model calls required by a future run.
33
+
34
+ Known provider timeouts, cancellations, unavailable infrastructure, process exits and invalid
35
+ output remain distinct. Configured unresolved-mapping and confidence rejections retain separate
36
+ `unresolved` and `low-confidence` codes and remain extraction gaps. Their acceptance criteria are
37
+ unchanged. Unknown extraction exceptions become `provider-failure`, without an
38
+ invented diagnosis. Cache read/write failures have a separate code; invalid extraction validation
39
+ is an extraction gap. Module loading/export/identity problems are unavailable infrastructure.
40
+ No raw exception, cause, provider output, prompt, credential or source path is copied into the
41
+ report. These diagnostics are not approved intent or evidence that a product rule was violated.
42
+
43
+ The CLI prints safe errors and progress on stderr, leaving stdout for successful results.
44
+ `hydrate --json --quiet` produces a JSON semantic failure report on stderr. The JSON contract
45
+ covers the semantic-source run; configuration, repository, indexing and graph validation errors
46
+ retain their existing interfaces. SIGINT listeners are removed on both success and failure.
47
+ The saved-evidence MCP interface still performs no hydration and writes only protocol messages
48
+ to stdout. Trusted module code must itself avoid logging private inputs; this is not a sandbox.
49
+
50
+ ## Source correlation and timeout meaning
51
+
52
+ A domain-separated SHA-256 of repository ID and stable semantic ID gives each source an opaque
53
+ `sourceId`. `hydrate --plan` explicitly maps that ID to its local path. The ID is stable across
54
+ retries and source edits when the stable semantic ID stays the same; renames without a declared
55
+ stable ID may change it. It is a correlation aid, not anonymization, encryption or an access
56
+ boundary. Do not publish the local preview without reviewing its paths and other metadata.
57
+ This ID never enters artifact keys, prompts, facts or snapshot configuration digests.
58
+
59
+ For source failures, elapsed time includes cache checks, authentication, provider attempts and
60
+ retries through failure. Codex's configured limit is per attempt; authentication has a separate
61
+ bounded status check. Module execution has one deadline covering import, factory and extraction.
62
+ The report distinguishes these scopes rather than presenting the limit as an end-to-end deadline.
63
+ No timeout default, retry policy, extraction contract or cache version changes here.
64
+
65
+ ## Recovery and limits
66
+
67
+ Failures do not reach graph publication. Prior snapshots and completed artifacts remain intact.
68
+ Use the same configuration, committed ref and scope in the no-model preview to inspect current
69
+ reuse before retrying without `--force-semantic`. T03 preserves compatible built-in Codex artifacts
70
+ across supported operational changes; legacy recovery requires the exact previous configuration.
71
+ Custom modules retain conservative full-configuration identity. Completed forced revisions are
72
+ retained but are not automatically adopted by an ordinary rerun or a new forced revision.
73
+
74
+ The toolkit waits for built-in subprocess termination or the module wrapper's bounded promise.
75
+ It cannot forcibly terminate arbitrary trusted in-process module code. Cancellation before source
76
+ selection and outside semantic processing uses the existing safe cancellation error without
77
+ per-source counts. Phase boundaries remain the progress signal; no synchronous SHACL heartbeat,
78
+ incremental validator or invented percentage is introduced by this milestone.
79
+
80
+ Public deterministic tests cover failures, cancellation races, durable-write completion, counts,
81
+ privacy, reuse, unchanged snapshot heads, source mapping and installed CLI behavior. They do not
82
+ claim private consumer acceptance or live-provider semantic quality.
@@ -0,0 +1,97 @@
1
+ # ADR-020: One physical-file policy for deterministic producers and adapters
2
+
3
+ Status: implemented in development source after 0.3.0; release/consumer acceptance is recorded
4
+ separately in [validation](VALIDATION.md).
5
+
6
+ ## Reported defect and ownership
7
+
8
+ The consumer's bounded replay of installed 0.3.0 extractors reported 120 conflicting file IDs:
9
+ 34 from the neutral contract extractor alone, 57 from its local framework adapter alone, and
10
+ 29 from both. Actual file/SCIP extraction classified spec/test files as `Evidence`; contract
11
+ consumers and local framework nodes could classify those same physical IDs as `Component`.
12
+ The consumer isolated a neutral file-plus-contract pair that failed strict SHACL single-kind
13
+ validation. This is an extraction identity defect, not conflicting product intent.
14
+
15
+ Upstream owns the neutral policy and producers. The consumer owns its framework call sites and
16
+ their tests. This work uses public fixtures; it does not reproduce all private IDs, modify the
17
+ consumer repository, patch installed packages or claim successful consumer hydration.
18
+
19
+ ## Public contract
20
+
21
+ Move the existing pure `repositoryFileNode(path)` implementation into `@weftlet/core` and export
22
+ it publicly. Its module imports only the `GraphNode` type; core gains no runtime package
23
+ dependencies. Keep the toolkit's internal re-export for compatibility, but adapters should use:
24
+
25
+ ```ts
26
+ import { repositoryFileNode } from '@weftlet/core';
27
+
28
+ const file = {
29
+ ...repositoryFileNode(path),
30
+ attributes: { /* adapter-specific evidence metadata */ },
31
+ };
32
+ ```
33
+
34
+ The caller supplies a repository-relative path in its canonical spelling. Existing mappings stay
35
+ unchanged: `.md`, `.mdx` and `.txt` use `document:<path>`/`Document`; `.spec.<extension>` and
36
+ `.test.<extension>` use `component:file:<path>`/`Evidence`; other physical files use that file
37
+ prefix and `Component`. Matching ignores extension case but preserves path spelling. This is not
38
+ filesystem resolution, universal test detection, proof of existence or approved intent. Do not
39
+ rewrite business concepts, contract identities or architectural modules into physical-file IDs.
40
+
41
+ The producer audit is explicit:
42
+
43
+ | Producer | Change |
44
+ | --- | --- |
45
+ | Neutral file extractor | Replace duplicated kind logic; preserve source/index attributes |
46
+ | SCIP symbols | Replace duplicated kind logic; preserve symbols, sites and citations |
47
+ | Contract consumers | Replace hard-coded Component; retain conceptual contract and evidence IDs |
48
+ | Document linker | Import the public factory; existing owner/reference behavior stays intact |
49
+ | Routing | Replace duplicated mapping; document endpoints now join document identities |
50
+ | Governance | Use the factory for dependency and membership files and query lookup; retain module/layer concepts |
51
+ | Semantic normalization | Use the shared factory for its existing document-only normalization; see the boundary below |
52
+
53
+ Facts are retained, including test files with mocked client calls, DI and decorators. We do not
54
+ suppress test evidence, rename conflicting IDs, weaken kind cardinality or change source/assertion
55
+ authority. Graphs and all working overlays require strict SHACL before publication; overlay
56
+ validation no longer depends on routing being configured.
57
+
58
+ ## Identity, saved evidence and artifacts
59
+
60
+ Contract extraction advances from 1.1.0 to 1.2.0; routing and SCIP fact extraction from 1.0.0 to
61
+ 1.1.0; governance analyzer from 2 to 3. The SCIP indexer/tool version is unchanged. Committed
62
+ assembly becomes `standalone-graph-v7`, with `graph-v7`, contract and SCIP fact identity included
63
+ in snapshot IDs. Overlay IDs/metadata also bind assembly, contract and SCIP fact identity,
64
+ so rebuilding identical captured bytes cannot replace an older overlay. Governance records its
65
+ analyzer identity in both snapshot digest inputs and metadata.
66
+
67
+ Current validation requires matching assembly, contract and SCIP fact versions, plus routing version when
68
+ configured. Missing metadata is not a pass. Final checks reject old versions without initiating
69
+ hydration. Historical saved queries still return the original facts, with extraction compatibility
70
+ reported separately from commit/configuration/working-overlay freshness in JSON, Markdown and
71
+ the viewer. Saved queries remain ineligible to validate a current diff.
72
+
73
+ The physical policy itself was already correct. Its semantic cache identity stays at version 1;
74
+ provider/prompt/artifact keys and original artifact bytes remain unchanged. The original snapshot
75
+ and any compatible semantic artifacts remain available. A fresh graph must still be assembled,
76
+ indexed and validated. Reuse does not imply that an old graph passes current validation.
77
+
78
+ ## Semantic and consumer boundary
79
+
80
+ The reported 120 conflicts were reproduced entirely by deterministic producers. This fix does
81
+ not rewrite semantic caches or force paid re-extraction to hide that defect. Existing semantic
82
+ normalization continues correcting repository document IDs/kinds only. Non-document model kinds
83
+ remain asserted, inferred and unreviewed; an incompatible model assertion still fails strict
84
+ SHACL. A broader interpretation migration would need separate provenance/OML parity tests and
85
+ an explicit cache/assembly decision. It is not silently bundled into this deterministic fix.
86
+
87
+ After upgrading both packages to the reviewed public release, the consumer must change every
88
+ physical-file node emitted by its framework adapter to the public factory and version that
89
+ adapter's extractor deliberately. Add actual adapter regressions for Prisma/DI/decorator-bearing
90
+ spec/test files composed with neutral file facts. Do not change semantic provider/prompt policy
91
+ versions merely to perform this deterministic adapter migration. Preview compatible artifact reuse,
92
+ then run full hydration and repeat acceptance against the committed adapter update.
93
+
94
+ Public upstream tests compose actual committed-source extractors, real TypeScript indexing,
95
+ document and routing links and a public adapter-shaped fixture. They assert strict SHACL,
96
+ provenance, confidence, authority, stable conceptual identities, saved-byte preservation and
97
+ version guards. They do not establish the private adapter's acceptance or semantic quality.
package/docs/PROVIDERS.md CHANGED
@@ -83,7 +83,7 @@ continue using their ChatGPT login; this change does not provision or copy accou
83
83
  ## Limits and reuse
84
84
 
85
85
  Defaults are at most 8 documents per hydration, 64 KiB per source, a 120-second timeout per
86
- request, no retries, and the configured concurrency (validated at 1–4). Source budgets are checked
86
+ Codex attempt (or module execution), no retries, and the configured concurrency (validated at 1–4). Source budgets are checked
87
87
  before any model call. `maxDocuments`, `maxSourceBytes`, `timeoutMs`, and `retries` may be configured
88
88
  within validated limits. Review scope and cost before raising them; these byte/document limits
89
89
  are not dollar or token estimates. Ask the operator before substantial paid runs.
@@ -96,6 +96,56 @@ digest/stable ID, extractor/adapter/ontology versions, confidence threshold and
96
96
  Unchanged artifacts are reused; forced revisions are separate. Failures never replace a valid
97
97
  snapshot. Progress shows phase and counts on stderr, with no document text or paths.
98
98
 
99
+ Starting with 0.4.0, T04 adds opaque source IDs, source elapsed times and structured semantic
100
+ failure reports. With that version or newer installed, retain the same committed ref and any
101
+ agent scope options as the failed run:
102
+
103
+ ```sh
104
+ pnpm exec weftlet hydrate --config /path/to/project/context-graph.config.json --ref <commit> --plan --json
105
+ pnpm exec weftlet hydrate --config /path/to/project/context-graph.config.json --ref <commit> --json --quiet
106
+ ```
107
+
108
+ The first command makes no model calls and maps each `sourceId` to its local document path. Review
109
+ that preview before sharing it. The second performs hydration: semantic failures exit nonzero,
110
+ leave stdout empty and emit JSON on stderr. Omit `--quiet` for progress; omit `--json` for a safe
111
+ human failure summary. Successful hydration still prints its result on stdout. JSON failure
112
+ reporting covers semantic processing, not every configuration/indexing/validation failure.
113
+
114
+ Reports distinguish timeout, cancellation, unavailable infrastructure, provider failure, invalid
115
+ output, unresolved mappings, low confidence and cache problems. Completed/extracted/reused counts include active writes that finished
116
+ before workers settled. `remaining` includes failed, cancelled and not-started sources; consult
117
+ preview for future extraction estimates. Cancellation between sources has no failed `sourceId`.
118
+ Source elapsed time includes cache checks and retries and may exceed the per-attempt limit;
119
+ module timeout includes import/factory/extraction. No defaults or retry rules change.
120
+
121
+ For a timeout, review `semantic.timeoutMs` and preview again before retrying without
122
+ `--force-semantic`. Completed forced revisions remain stored but are not automatically reused by
123
+ an ordinary rerun or a new forced run. Custom provider configuration changes remain conservative.
124
+ See [ADR 019](ADR-019-semantic-failure-diagnostics.md) for reporting and privacy boundaries.
125
+
126
+ Starting with 0.4.0, built-in Codex semantic artifacts are separate from user
127
+ snapshot profiles. Changing `timeoutMs`, `retries`, `concurrency`, `maxDocuments` or
128
+ `maxSourceBytes` can reuse completed artifacts; selection budgets are still enforced. Snapshot
129
+ configuration digests remain complete and saved snapshots are not accepted as current merely
130
+ because their artifacts were reused. Workspace cache paths and custom-provider identities remain
131
+ unchanged. See [ADR 018](ADR-018-operational-semantic-cache.md) for exact isolation and recovery rules.
132
+
133
+ If an older user-cache run was interrupted and its configuration has already changed, retain an
134
+ exact copy of the previous configuration in its original directory. Its original `repositoryRoot`
135
+ must still resolve to the same checkout; editing that field changes the old profile identity.
136
+ With 0.4.0 or newer installed:
137
+
138
+ ```sh
139
+ pnpm exec weftlet cache adopt --config /path/to/project/context-graph.config.json --from-config /path/to/project/previous-context-graph.config.json --ref HEAD --dry-run
140
+ pnpm exec weftlet cache adopt --config /path/to/project/context-graph.config.json --from-config /path/to/project/previous-context-graph.config.json --ref HEAD
141
+ ```
142
+
143
+ For an agent-scoped run, supply the same `--owner-doc`, `--task` or `--capability` selection used
144
+ for hydration. The JSON report shows what was retained and what still needs extraction. Adoption
145
+ makes no model calls and changes no snapshots or source files. Incompatible configurations are
146
+ refused; unrelated cache profiles are never scanned. Corrupt/conflicting artifacts are reported
147
+ and retained. This command is not present in published 0.3.0.
148
+
99
149
  For a trusted local provider module, set `provider: "module"`, `providerModule` to a local path
100
150
  (relative to the repository root) or installed package, and `providerIdentity` to its exact
101
151
  declared version. Packages resolve from the consumer repository using ESM import conditions.
@@ -1,10 +1,12 @@
1
1
  # Consumer hardening and the 1.0 release plan
2
2
 
3
- Status: maintainer-directed work; source checkpoint on 2026-10-26 before the planned 0.3.0 release.
3
+ Status: maintainer-directed work; release checkpoint on 2026-10-26 after verified 0.3.0 publication.
4
+ **New adoption blocker: T08, deterministic physical-file kind conflicts.** This takes release
5
+ priority over the remaining planned increments; T04 was completed first as requested.
4
6
  The report concerned `@weftlet/toolkit@0.2.0`. **T01's upstream patch shipped in 0.2.1**;
5
- genuine consumer acceptance remains pending. T02 is merged and awaits release. T05's focused
6
- cold-validation optimization is implemented in development source, with validation/release pending;
7
- its broader follow-ups and T03, T04, T06 and T07 remain open. These tasks improve setup, reuse, diagnostics and document coverage
7
+ genuine consumer acceptance remains pending. T02 and T05's focused cold-validation optimization
8
+ shipped in 0.3.0. T03 and T04 are implemented in development source after that release. Broader T05
9
+ follow-ups and T06 and T07 remain open. These tasks improve setup, reuse, diagnostics and document coverage
8
10
  without expanding the consumer's migration. An official 1.0
9
11
  follows this work and genuine consumer acceptance, with its exact release scope reviewed when
10
12
  the evidence is available.
@@ -34,20 +36,40 @@ integration. Consumer acceptance is reported separately from upstream determinis
34
36
 
35
37
  | ID | Task | Priority/status | Dependencies and release boundary |
36
38
  | --- | --- | --- | --- |
39
+ | **T08** | Canonical physical-file kinds and public adapter factory | **Merged; local and hosted Ubuntu/Windows checks passed; release pending** | Neutral contract consumers conflict with test Evidence; consumer framework migration remains separately required |
37
40
  | **T01** | Coherent repository-document identity and kind | **Upstream patch shipped in 0.2.1; consumer acceptance pending** | SHACL and assertions preserved; genuine consumer re-run is still required |
38
- | **T02** | No-model hydration/setup preview | Merged in PR #18; not published in 0.2.1 | Shared selection/cache inspection, human/JSON plans; extend with T07 traversal coverage |
39
- | **T03** | Artifact reuse across operational configuration changes | Planned efficiency work | Preserve T01's semantic identity separation; keep snapshot/config identity distinct from reusable artifact identity |
40
- | **T04** | Actionable, privacy-safe timeout diagnostics | Planned diagnostics work | Use stable source identifiers and run progress; document recovery with T02/T03 |
41
- | **T05** | Early identity checks and validation progress/profiling | **Adoption blocker; cold-validation fix in source** | Validate and release union-projection optimization now; early checks, detailed progress and validation reuse remain open |
41
+ | **T02** | No-model hydration/setup preview | Shipped in 0.3.0 | Shared selection/cache inspection, human/JSON plans; extend with T07 traversal coverage |
42
+ | **T03** | Artifact reuse across operational configuration changes | Merged; local and hosted Ubuntu/Windows checks passed; unreleased | Built-in Codex user artifacts share five operational changes; custom providers remain conservative; exact legacy recovery without model calls |
43
+ | **T04** | Actionable, privacy-safe timeout diagnostics | Merged; local and hosted Ubuntu/Windows checks passed; unreleased | Safe source IDs, settled counts, failure classification and T02/T03 recovery; no extraction/cache identity change |
44
+ | **T05** | Early identity checks and validation progress/profiling | **Cold-validation fix shipped in 0.3.0; consumer acceptance pending** | Full validation preserved; early checks, detailed progress and validation reuse remain open |
42
45
  | **T06** | Packaged release-documentation consistency | Open; README version pins removed in 0.2.1 | Automated archive/version consistency check remains to be implemented |
43
46
  | **T07** | Coverage-aware traversal from SOT/roadmap entry points | Planned bounded feature work | Use canonical identity from T01, preview from T02 and incremental reuse from T03; split selection/classification/extraction changes |
44
47
 
45
- T02 passed the local publication checks and hosted Ubuntu/Windows checks before
46
- [PR #18](https://github.com/jchewitt/weftlet/pull/18) merged. Prioritize the focused T05 performance
47
- release following the maintainer's reported timeout; do not wait for the remaining 1.0 features.
48
- T03, T04, T06 and T07 remain independently useful increments. Keep them
49
- separate from the optional platform work. Publication follows the normal release process; no new
50
- npm release is claimed for T02. Normal release checks apply to every shipped increment.
48
+ T02 and the focused T05 fix passed local publication checks and hosted Ubuntu/Windows checks,
49
+ then shipped in [0.3.0](https://github.com/jchewitt/weftlet/releases/tag/%40weftlet/toolkit%400.3.0).
50
+ Registry archives match the independently reviewed export; source commit, provenance and release
51
+ tags are recorded in [validation](VALIDATION.md). T03, T04, T06 and T07 remain independently
52
+ useful increments, separate from the optional platform work. A genuine consumer hydration and
53
+ repeat against the current release remain required; upstream fixtures do not establish that result.
54
+
55
+ ## T08 — Resolve deterministic physical-file kind conflicts
56
+
57
+ The consumer reports 120 IDs reproduced by actual 0.3.0 deterministic producers: 34 neutral
58
+ contract-only, 57 local-framework-only and 29 overlapping. This is separate from T01's repository
59
+ document identity defect and does not diagnose semantic-cache corruption or business-rule drift.
60
+
61
+ [ADR 020](ADR-020-canonical-physical-file-producers.md) records the public core factory, producer
62
+ audit, deterministic version changes and preserved semantic-cache boundary. The upstream fix
63
+ routes physical-file producers through one policy, preserves strict SHACL/provenance/authority,
64
+ and distinguishes historical graph versions from current validation. Public tests include real
65
+ SCIP and actual extractor composition, plus negative overlay validation without routing.
66
+
67
+ Release requires local publication checks, hosted Ubuntu/Windows checks, reviewed archives and
68
+ fresh installed-package verification. Consumer acceptance then requires the published packages
69
+ plus its own framework adapter migration and composed Prisma/DI/decorator spec/test regressions,
70
+ followed by full hydration and repeat reuse. Neither an upstream-only release nor a local-only
71
+ adapter patch closes both ownership boundaries. Do not rewrite semantic caches or force model
72
+ work to work around the proven deterministic defect.
51
73
 
52
74
  ## T01 — Resolve repository-document identity conflicts
53
75
 
@@ -87,13 +109,14 @@ Upstream completion evidence:
87
109
  snapshot identity/content on artifact reuse, without live model calls. See the detailed
88
110
  [validation record](VALIDATION.md) for commits, checks and result boundaries.
89
111
 
90
- The private consumer has not yet supplied a successful full hydration/repeat against 0.2.1.
112
+ The private consumer has not yet supplied a successful full hydration/repeat with the identity
113
+ patch. Use the current 0.3.0 release, which also includes the cold-validation improvement.
91
114
  Upstream publication and regression success do not close that acceptance gate or diagnose the
92
115
  separate full-graph validation latency observation.
93
116
 
94
117
  ## T02 — Preview scope and work without calling a model
95
118
 
96
- Implemented in development source: `hydrate --plan` renders a human summary and `--json` returns
119
+ Shipped in 0.3.0: `hydrate --plan` renders a human summary and `--json` returns
97
120
  the structured plan. This feature is not in published 0.2.1. Shared source selection and identity
98
121
  logic report selected documents and sizes, inclusion reasons, configured role/source approval,
99
122
  unreviewed assertion status, artifact reuse/extraction estimates and document/per-source byte limits.
@@ -120,6 +143,20 @@ Acceptance:
120
143
 
121
144
  ## T03 — Reuse valid artifacts after operational changes
122
145
 
146
+ Merged in [PR 22](https://github.com/jchewitt/weftlet/pull/22) after 0.3.0, with local publication
147
+ and hosted Ubuntu/Windows checks passing. This source checkpoint does not claim npm publication.
148
+ [ADR 018](ADR-018-operational-semantic-cache.md) separates built-in Codex user artifacts from
149
+ full snapshot profiles. Timeout, retry, concurrency and document/byte limits can change without
150
+ hiding completed artifacts. Model/policy/authentication descriptors and unknown fields remain
151
+ identity inputs; custom providers retain their full configuration contract. Workspace cache
152
+ paths and existing 0.2.1/0.3.0 artifact identities remain unchanged.
153
+
154
+ Planning reads the exact compatible legacy profile without writing. Hydration can retain a
155
+ validated copy atomically. Explicit `cache adopt --from-config` and its read-only `--dry-run`
156
+ recover expected committed-source keys after settings were already changed; no unrelated profiles
157
+ are scanned and no providers or snapshots are initialized. Agent scope and budget/coverage
158
+ reporting match the hydration preview. Current-diff checks still reject old-configuration snapshots.
159
+
123
160
  The consumer reported that changing only a timeout selected a different cache profile and made
124
161
  completed extractions unavailable for reuse. Investigate both storage-location selection and
125
162
  artifact key construction. Separate operational controls from semantic-output identity, while
@@ -136,10 +173,22 @@ Acceptance:
136
173
  - Preserve force-extraction semantics, concurrent-writer safety, isolation between consumers,
137
174
  credentials and private cache contents. Record reuse and reasons for a miss without secrets.
138
175
  - Test interrupted work followed by a changed timeout: valid completed artifacts are reused,
139
- incomplete/invalid ones are re-extracted, and source provenance remains correct.
176
+ missing/incomplete artifacts are extracted, and source provenance remains correct. Corrupt
177
+ complete artifacts stay intact and require explicit forced refresh or operator repair;
178
+ they are not silently overwritten or treated as misses.
140
179
 
141
180
  ## T04 — Explain timeouts and recovery
142
181
 
182
+ Merged in [PR 25](https://github.com/jchewitt/weftlet/pull/25) after 0.3.0, with local publication
183
+ and hosted Ubuntu/Windows checks passing. This development checkpoint does not claim npm publication.
184
+ [ADR 019](ADR-019-semantic-failure-diagnostics.md) records opaque source IDs and explicit local
185
+ preview mapping, final counts after active workers settle, safe typed errors and recovery.
186
+ Concurrent sibling cancellation preserves the initiating failure. Completed writes stay counted
187
+ as completed; cancellation between sources reports no failing source. The CLI supports semantic
188
+ failure JSON on stderr with `hydrate --json --quiet`, leaving protocol/success stdout untouched.
189
+ Provider loading problems and unknown provider errors are distinct. Timeout defaults, extraction
190
+ identities, cache formulas, saved queries and full graph validation remain unchanged.
191
+
143
192
  Report a privacy-safe source identifier, phase, elapsed time, configured limit and completed/
144
193
  remaining counts when extraction times out. Explain which completed artifacts can be retained
145
194
  and how to inspect the plan or adjust a bound before retrying. The documented two-minute default
@@ -162,11 +211,11 @@ entire assembled graph, including when semantic artifacts were reused. There is
162
211
  GraphDB backend or incremental validation. Querying a saved snapshot does not rerun SHACL.
163
212
 
164
213
  The maintainer reports that validation pushes the consumer's overall generation beyond its
165
- 30-minute timeout, preventing snapshot publication and making adoption unusable. This is now
166
- a release blocker, not an optional observability improvement. Upstream has measured a specific
214
+ 30-minute timeout, preventing snapshot publication and making adoption unusable. That report made
215
+ cold validation a release priority ahead of the remaining 1.0 work. Upstream has measured a specific
167
216
  named-graph lookup cost in public synthetic fixtures, but has not measured that consumer graph or
168
- diagnosed its complete timeout. Ship the focused performance release once its correctness and
169
- improvement are demonstrated, ahead of the remaining 1.0 tasks.
217
+ diagnosed its complete timeout. The focused cold-validation fix shipped in 0.3.0, ahead of the
218
+ remaining 1.0 tasks. Consumer acceptance remains pending; broader T05 items stay open.
170
219
 
171
220
  [ADR 017](ADR-017-shacl-validation-union.md) records the implemented cold-validation optimization:
172
221
  build a temporary union of every SPO triple for the same validator/shapes, while keeping original
package/docs/RELEASING.md CHANGED
@@ -1,7 +1,9 @@
1
1
  # Weftlet npm releases
2
2
 
3
- `@weftlet/core@0.2.1` and `@weftlet/toolkit@0.2.1` are published through GitHub OIDC.
4
- [Run 40382864083](https://github.com/jchewitt/weftlet/actions/runs/40382864083) completed
3
+ `@weftlet/core` and `@weftlet/toolkit` are published through GitHub OIDC. The latest verified
4
+ release checkpoint is recorded in [validation](VALIDATION.md); the following are historical
5
+ publication records. For release 0.2.1,
6
+ [run 40382864083](https://github.com/jchewitt/weftlet/actions/runs/40382864083) completed
5
7
  preparation, publication and release records on its first attempt at commit
6
8
  `5d1db91bf19ed493a20849b2d0f92d3aeac35dfe`. Registry archives match the reviewed archives;
7
9
  provenance and package tags identify that commit. A fresh public-registry installation passes
package/docs/ROADMAP.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # Delivery roadmap
2
2
 
3
- Source checkpoint: 2026-10-26, before the planned 0.3.0 release. Publication evidence is
4
- recorded separately in [validation](VALIDATION.md); this roadmap is not a live registry status.
3
+ Release checkpoint: 2026-10-26, after verified 0.3.0 publication. Evidence is recorded in
4
+ [validation](VALIDATION.md); this roadmap is not a live registry status.
5
5
 
6
6
  Project: `jchewitt/weftlet`. Extraction provenance is recorded in the
7
7
  [source inventory](SOURCE-INVENTORY.md).
@@ -9,18 +9,19 @@ Project: `jchewitt/weftlet`. Extraction provenance is recorded in the
9
9
  Forward direction is captured in the [Weftlet 2.0 product plan](WEFTLET-2.0-PLAN.md):
10
10
  consumer hardening, cited code-derived drafts, scoped approvals and reconciliation, an optional
11
11
  review website, source publication, managed AI and guided setup. “2.0” is a planning label;
12
- the published packages are 0.2.1. The received consumer findings are tracked in the
12
+ the verified release at this checkpoint is 0.3.0. The received consumer findings are tracked in the
13
13
  [consumer hardening and 1.0 task list](RELEASE-1.0-PLAN.md): the document-identity patch is
14
14
  published and passes upstream checks; genuine consumer acceptance remains pending. T02's
15
- no-model preview is implemented in development source but is not in published 0.2.1. Cache reuse,
16
- diagnostics and document coverage remain open before official 1.0 acceptance.
15
+ no-model preview and T05's cold-validation improvement shipped in 0.3.0. Operational cache reuse
16
+ and timeout diagnostics are development increments; document coverage and other acceptance work
17
+ remain open before official 1.0.
17
18
  The 2.0 plan distinguishes proposed work from delivered capability.
18
19
 
19
- T05 now blocks adoption: the maintainer reports SHACL pushing generation past a 30-minute
20
- timeout before graph publication. T02 merged in [PR #18](https://github.com/jchewitt/weftlet/pull/18)
21
- after local and hosted Ubuntu/Windows checks. The focused T05 cold-validation optimization is
22
- implemented in development source, awaiting validation/release; the other 1.0 features must not
23
- delay it. Neither increment is in the published 0.2.1 packages.
20
+ The maintainer reported SHACL pushing generation past a 30-minute timeout before graph
21
+ publication. T02 and the focused T05 optimization shipped together in 0.3.0 through
22
+ [release PR #19](https://github.com/jchewitt/weftlet/pull/19), following local and hosted
23
+ Ubuntu/Windows checks and independent source/archive review. A genuine consumer re-run is
24
+ still required; the remaining 1.0 features did not delay this performance release.
24
25
 
25
26
  [Hydration planning](ADR-016-hydration-preview.md) shares committed selection and semantic cache
26
27
  identity with execution, reports human/JSON scope and work estimates, and preserves source versus
@@ -29,10 +30,28 @@ still checks the complete in-memory N3 graph on each hydration; saved queries do
29
30
  validation. [The T05 union optimization](ADR-017-shacl-validation-union.md) indexes all SPO triples
30
31
  in one temporary validation graph, preserving the same shapes and canonical named evidence.
31
32
  Bounded public experiments measured substantial cold-path improvement, alongside about 1.3 GiB
32
- RSS at 10,000 unique-node facts. They do not reproduce the private consumer's overall timeout.
33
+ RSS in the prototype at 10,000 unique-node facts; the production harness including serialization
34
+ peaked near 1.6 GiB. They do not reproduce the private consumer's overall timeout.
33
35
  Early checks, in-phase progress, result reuse and incremental validation remain open; no Rust
34
36
  engine, graph database or validation-result cache is introduced by this focused release.
35
37
 
38
+ [T03 operational cache reuse](ADR-018-operational-semantic-cache.md) is merged with local and
39
+ hosted checks passing and is not part of published 0.3.0. Built-in Codex user
40
+ artifacts can survive changes to timeout/retry/concurrency/document/byte limits while snapshots
41
+ retain their full configuration identity. Exact legacy reads and explicit no-model adoption
42
+ preserve prior artifacts; custom providers remain conservative.
43
+ [T04 diagnostics](ADR-019-semantic-failure-diagnostics.md) add safe failure reports, opaque source
44
+ mapping and accurate settled progress without changing extraction/cache identity. T04 is merged
45
+ with local and hosted Ubuntu/Windows checks passing; release remains pending. Broader T05 and
46
+ T06/T07 work remain open.
47
+
48
+ A subsequent 0.3.0 consumer report makes [T08 physical-file kind consistency](ADR-020-canonical-physical-file-producers.md)
49
+ the next release priority. Its neutral contract consumer fix and public core factory are merged
50
+ with deterministic producer adoption, version guards and passing local/hosted Ubuntu/Windows
51
+ checks. Publication is pending; the consumer's framework adapter migration and full acceptance
52
+ remain consumer-owned.
53
+ This fix preserves semantic artifacts and does not reinterpret model assertions to hide conflicts.
54
+
36
55
  | Milestone | Delivered and checked | Remaining limits |
37
56
  | --- | --- | --- |
38
57
  | 1. Standalone | Explicit committed inventory, core/toolkit boundary, public examples, build/tests, packed independent consumer and real TypeScript indexing | GitHub exported; both @launch-echo packages published at 0.1.0 and public installation verified |
@@ -119,7 +138,7 @@ preparation/publication under `@launch-echo`; the maintainer completed the 0.1.0
119
138
  source-application reintegration remain outside the authorized scope. See [release instructions](RELEASING.md).
120
139
 
121
140
  The maintainer authorized the Weftlet migration on 2026-10-21. Both `@weftlet/core` and
122
- `@weftlet/toolkit` are published at 0.2.1 through GitHub OIDC; the publication and verification
141
+ `@weftlet/toolkit` are published at 0.3.0 through GitHub OIDC; the publication and verification
123
142
  record is in [validation](VALIDATION.md). Changesets prepares reviewed version PRs for the fixed package group;
124
143
  merging a release plan triggers publication and package-specific tags/GitHub Releases.
125
144
  See [ADR 009](ADR-009-release-automation.md) and [release instructions](RELEASING.md).