@fraczak/k 6.8.0 → 6.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/CHANGELOG.md +40 -0
  2. package/CLAUDE.md +1 -1
  3. package/DOCS/BIG_PICTURE.md +3 -5
  4. package/DOCS/CODECS.md +80 -181
  5. package/DOCS/CODEC_ARCHITECTURE.md +194 -0
  6. package/DOCS/REPL.md +69 -51
  7. package/Examples/arithmetics.k +1 -0
  8. package/Examples/poly.k +46 -0
  9. package/backends/llvm/runtime/krt.c +4 -0
  10. package/backends/llvm/src/kvm2llvm.mjs +2 -2
  11. package/backends/llvm/tests/test-llvm.mjs +19 -0
  12. package/backends/wasm/src/kvm2wasm.mjs +4 -0
  13. package/backends/wasm/tests/test-wasm.mjs +4 -0
  14. package/browser/repl-browser.mjs +131 -779
  15. package/browser/shims/process.mjs +7 -0
  16. package/browser/tree-view.mjs +5 -2
  17. package/browser/vfs-data.mjs +14 -14
  18. package/codecs/README.md +7 -0
  19. package/codecs/ieee.mjs +50 -75
  20. package/codecs/int.mjs +83 -62
  21. package/codecs/json-codec.mjs +5 -77
  22. package/codecs/json.mjs +12 -56
  23. package/codecs/k-parse.mjs +3 -19
  24. package/codecs/runtime/codec-sdk.mjs +94 -0
  25. package/codecs/runtime/pattern-k.mjs +45 -0
  26. package/codecs/string-codec.mjs +48 -74
  27. package/codecs/unit.mjs +13 -61
  28. package/codecs/utf16.mjs +13 -56
  29. package/codecs/utf8.mjs +11 -55
  30. package/package.json +1 -1
  31. package/patterns/from-k.mjs +2 -22
  32. package/repl-codecs.mjs +69 -196
  33. package/repl.mjs +177 -281
  34. package/scripts/build-repl-html.mjs +114 -385
  35. package/scripts/run-tests.mjs +1 -0
  36. package/tests/test-codecs.mjs +186 -0
  37. package/tests/test-poly.mjs +44 -0
  38. package/tests/test-repl-html.mjs +163 -68
  39. package/tests/test-repl.mjs +210 -156
  40. package/tests/test-structural-values.mjs +17 -0
  41. package/tests/test-tree-view.mjs +3 -0
  42. package/codecs/example-pipeline.mjs +0 -163
  43. package/codecs/runtime/ieee-pattern.mjs +0 -30
  44. package/codecs/runtime/test-codec.mjs +0 -89
  45. package/codecs/runtime/typeFromValue.mjs +0 -34
package/CHANGELOG.md CHANGED
@@ -5,6 +5,46 @@ All notable changes to this project will be documented here.
5
5
  The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
6
6
  This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [6.10.1] — 2026-10-02 — Fix Compaction Mark Reset and False Tail Calls in LLVM & WASM Backends
9
+
10
+ ### LLVM & WebAssembly Backends
11
+
12
+ - **Compaction Mark Reset**: Fixed an issue in both LLVM (`krt.c`) and WebAssembly (`kvm2wasm.mjs`) where tail-loop compaction never reset the baseline arena mark after compacting live objects. Once the threshold was reached, compaction was erroneously triggered on every subsequent iteration, turning $O(N)$ execution into $O(N^2)$ and causing call-stack overflow on large inputs.
13
+ - **Product Branch Tail-Call Fix**: Fixed LLVM lowering in `kvm2llvm.mjs` where recursive calls inside product constructor fields were incorrectly classified as self-tail calls.
14
+
15
+ ---
16
+
17
+ ## [6.10.0] — 2026-10-02 — Dual Evaluation Engines (WASM & JS) and REPL Command Modernization
18
+
19
+ ### REPL & Evaluation Engines
20
+
21
+ - **Dual Evaluation Engines**: The REPL now supports two evaluation modes:
22
+ - `wasm` (default): Lowers expressions via polymorphic kVM to WebAssembly AOT bytecode for near-native performance, tail-call optimization, and stack safety.
23
+ - `js`: Direct tree-walking JavaScript interpreter evaluating parsed relations via `run_rel` with pattern constraints for instant interpretation without compilation latency.
24
+ - **Engine Switching & CLI**:
25
+ - REPL commands `:engine [wasm|js]`, `:wasm`, and `:js` dynamically inspect or switch the active evaluation engine.
26
+ - CLI options `--engine=wasm`, `--engine=js`, `--wasm`, and `--js` added to `k-repl`.
27
+ - Active engine displayed in execution timing annotations (`/* comp: Xms, exec: Yms (wasm) */` or `(js)`).
28
+ - **Web REPL (`repl.html`) Integration**:
29
+ - Added clickable navbar engine badge (`wasm-in-process` <-> `js-in-process`) for one-click engine toggling in the browser.
30
+ - Exported `getEngine()` and `setEngine()` on `window.kRepl`.
31
+ - **Command Modernization**:
32
+ - Removed deprecated REPL commands: `:run expr`, `:eval expr`, `:time expr`, and `:t name`.
33
+ - Renamed `:codes` to `:types` (listing type aliases).
34
+ - Renamed `:d name` to `:rel name` (displaying relation definitions).
35
+
36
+ ---
37
+
38
+ ## [6.9.0] — 2026-10-01 — Reusable Codecs, Dynamic Pattern Derivation & Unified SDK
39
+
40
+ ### Codecs & Type System
41
+
42
+ - **Dynamic Pattern Derivation**: All codecs derive their closed pattern graphs through `k`'s type system from filter and type expressions via `patternFromFilter`. Handwritten property-list JSON arrays have been completely eliminated from codecs.
43
+ - **Unified Codec SDK**: Added `codecs/runtime/codec-sdk.mjs` exporting a standard CLI harness (`runCodecCLI`), `Value` API, wire codecs, and `patternFromFilter`.
44
+ - **Recipe-Based REPL Codecs**: Refactored the REPL to use explicit recipe-driven input syntax (`:input <codec.mjs> [text]`) and multi-codec output formatting. Removed universal catch-all fallback and ungrounded types.
45
+ - **Self-Contained Web REPL VFS**: The browser REPL compiler (`scripts/build-repl-html.mjs`) bundles verified codecs into zero-import ES modules for the virtual file system, delegating `patternFromFilter` dynamically to the browser's bundled `k` engine.
46
+ - **Dead Code Clean-Up**: Removed dead files (`ieee-pattern.mjs`, `test-codec.mjs`, `typeFromValue.mjs`, `example-pipeline.mjs`).
47
+
8
48
  ---
9
49
 
10
50
  ## [6.5.0] — 2026-09-22 — Tail-loop arena compaction & OOM diagnostics
package/CLAUDE.md CHANGED
@@ -61,4 +61,4 @@ npm test # Run comprehensive test suite (unit tests, derivation tests, shell te
61
61
  - No external runtime dependencies (only jiwson for parser generation)
62
62
  - Uses ES6 modules (.mjs files) throughout
63
63
  - Extensive test suite in `tests/code-derivation/` for type system validation
64
- - REPL supports commands like `:C name` (show canonical type), `:t name` (show relation type), and `:d name` (show relation definition)
64
+ - REPL supports commands like `:rel name` (show relation definition), `:type name` (show type definition), `:types`, `:rels`, etc.
@@ -252,13 +252,11 @@ car = list? /cons .car;
252
252
  cdr = list? /cons .cdr;
253
253
  ```
254
254
 
255
- For example, the filter annotation for `car` is:
255
+ For example, the definition for `car` is:
256
256
 
257
257
  ```text
258
- > :t car
259
- car : ?<{X1 car, X0 cdr} cons, $KL nil>=X0 --> ?X1 (@...)
260
- > :C KL
261
- $ KL = {}; -- @...
258
+ > :rel car
259
+ car = list? /cons .car; -- @...
262
260
  ```
263
261
 
264
262
  ## Universal Schema Registry
package/DOCS/CODECS.md CHANGED
@@ -1,113 +1,86 @@
1
1
  # Writing Codecs
2
2
 
3
- A codec is a small ES module that translates between some external text format
4
- and k `Value` objects. The same module may serve two entry points:
3
+ A codec is an ES module that translates between an external domain format and enveloped **k** `Value` objects `(P, v)`. The same module serves two entry points:
5
4
 
6
- - a command-line codec such as `k-int --parse` or `k-int --print`,
7
- - a REPL codec loaded with `:codec load file`.
5
+ - a standalone command-line filter supporting `--parse`, `--print`, and `--help`,
6
+ - a REPL module loaded with `:codec load <file.mjs>`.
8
7
 
9
8
  The command-line entry point reads or writes the binary pattern+value stream.
10
- The REPL entry point exports `name`, type metadata, `parse`, and `print`
11
- functions.
9
+ The REPL entry point provides interactive parsing (`:input <codec.mjs>`) and multi-codec output formatting.
12
10
 
13
- ## Choose The Shape
11
+ ## Architecture & Mental Model
14
12
 
15
- Use a type-specific codec when the external format belongs to one k type. The
16
- module exports either `codes` or `patterns`.
13
+ A codec maps between external text and enveloped *k* values:
17
14
 
18
- ```js
19
- export const name = "unit";
20
- export const patterns = [[
21
- ["closed-product", []]
22
- ]];
23
- ```
15
+ - `parse(text)` translates external text to an enveloped *k* value (a `Value` with an associated closed pattern envelope `(P, v)`).
16
+ - `print(value)` translates an enveloped *k* value into formatted external text, throwing an `Error` if the value is not representable.
17
+ - `doc` (optional string) documents the accepted syntax and semantic representation.
24
18
 
25
- Use a universal codec when the external format can describe many k types. A
26
- universal codec is available for any type named by `:input`; the REPL validates
27
- the parsed value against that requested type before accepting it.
19
+ Codecs are identified directly by their file name. They do not export artificial identifiers like `name` or pattern registries like `patterns = [...]`.
28
20
 
29
- ```js
30
- export const name = "json";
31
- export const universal = true;
32
- ```
21
+ In the REPL:
22
+ - **Input**: The user explicitly chooses which codec to parse with:
23
+ - Interactive mode: `:input <codec.mjs>` sets the prompt to `<codec.mjs>> `, and the next entered line is parsed with that codec.
24
+ - One-line mode: `:input <codec.mjs> <text>` immediately parses `<text>` using that codec.
25
+ - **Output**: Any evaluated *k* value is displayed in its standard *k* envelope representation, followed by formatting from **all loaded codecs** that can represent it:
26
+ ```text
27
+ > 42 int
28
+ {}|_|0|1|0|1|0|1|+ ?<{} _, ...>
29
+ int.mjs: 42
30
+ ```
31
+ Every loaded codec attempts `codec.print(value)`. If it succeeds, the REPL prints `<codec.mjs>: <formatted>`. If it throws, the codec is silently ignored for that value.
33
32
 
34
- Prefer `patterns` for codecs that live in the source tree because the REPL
35
- recomputes the canonical code hash. Use `codes` only when the codec is tied to a
36
- known canonical hash:
37
-
38
- ```js
39
- export const codes = ["@..."];
40
- ```
33
+ ## Module Interface
41
34
 
42
- ## Implement The REPL API
43
-
44
- A REPL codec exports:
35
+ A codec module exports:
45
36
 
46
37
  ```js
47
- export const name = "mycodec";
48
- export const patterns = [MY_PATTERN]; // or codes = ["@..."], or universal = true
38
+ export const doc = "Description of syntax and usage.";
49
39
 
50
- export function parse(text, context) {
51
- // text is the line entered after :input
52
- // return a k Value
40
+ export function parse(text) {
41
+ // parse text into an enveloped k Value (P, v) using withPattern(value, pattern)
53
42
  }
54
43
 
55
- export function print(value, context) {
56
- // return text shown under normal REPL output
44
+ export function print(value) {
45
+ // return formatted text
46
+ // throw Error if value is not representable
57
47
  }
58
48
  ```
59
49
 
60
- `context` contains:
61
-
62
- - `codeHash`: the canonical code hash selected by `:input`,
63
- - `pattern`: the closed property-list pattern for that type,
64
- - `state`: the current REPL state.
65
-
66
- Most codecs do not need `context`. Universal codecs can use it when the same
67
- text syntax needs to adapt to the requested type.
68
-
69
- `parse` should throw an `Error` for invalid external text. `print` should throw
70
- when the value is not representable by the external format. For universal
71
- codecs, REPL output suppresses `print` errors so a generic codec does not add
72
- noise to every value.
50
+ `parse` should throw an `Error` for invalid external text. `print` should throw when the value is not representable by the external format. REPL output suppresses `print` errors so loaded codecs only output when they recognize the value.
73
51
 
74
- ## Build Values
52
+ ## The Codec SDK & Runner
75
53
 
76
- Use the structural `Value` API:
54
+ To eliminate CLI stream boilerplate, codecs import from `./runtime/codec-sdk.mjs` and invoke `runCodecCLI`:
77
55
 
78
56
  ```js
79
- import { Value, isProduct, isVariant } from "../Value.mjs";
80
-
81
- const unit = Value.product({});
82
- const yes = Value.variant("true", unit);
83
- const point = Value.product({
84
- x: Value.variant("+", unit),
85
- y: Value.variant("-", unit)
86
- });
57
+ import { Value, isVariant, withPattern, runCodecCLI } from "./runtime/codec-sdk.mjs";
87
58
  ```
88
59
 
89
- Products are JavaScript objects whose keys are k field labels. Variants have a
90
- string tag and a payload value.
60
+ `runCodecCLI(import.meta.url, { parse, print, doc })` automatically handles:
61
+ 1. Entrypoint detection (no-op when loaded as a module or in browsers).
62
+ 2. `-h` and `--help` CLI flag handling.
63
+ 3. `--parse` and `--print` stream piping and wire encoding/decoding.
91
64
 
92
- ## Minimal Type-Specific Codec
65
+ ## Minimal Type-Specific Codec Example
93
66
 
94
- This codec accepts `yes` and `no` for the type `<{} true, {} false>`.
67
+ This codec accepts `yes` and `no` for the type `<{} false, {} true>`. Patterns are derived by `k` from filter expressions via `patternFromFilter`:
95
68
 
96
69
  ```js
97
- import { Value, isVariant } from "../Value.mjs";
70
+ #!/usr/bin/env node
71
+
72
+ import { Value, isVariant, withPattern, patternFromFilter, runCodecCLI } from "./runtime/codec-sdk.mjs";
98
73
 
99
- const BOOL_PATTERN = [
100
- ["closed-union", [["false", 1], ["true", 1]]],
101
- ["closed-product", []]
102
- ];
74
+ const BOOL_PATTERN = patternFromFilter("?< {} false, {} true >");
103
75
 
104
- export const name = "yesno";
105
- export const patterns = [BOOL_PATTERN];
76
+ export const doc = `
77
+ Boolean codec: accepts 'yes' and 'no'.
78
+ `;
106
79
 
107
80
  export function parse(text) {
108
81
  const word = text.trim();
109
- if (word === "yes") return Value.variant("true", Value.product({}));
110
- if (word === "no") return Value.variant("false", Value.product({}));
82
+ if (word === "yes") return withPattern(Value.variant("true", Value.product({})), BOOL_PATTERN);
83
+ if (word === "no") return withPattern(Value.variant("false", Value.product({})), BOOL_PATTERN);
111
84
  throw new Error("expected yes or no");
112
85
  }
113
86
 
@@ -117,106 +90,30 @@ export function print(value) {
117
90
  if (value.tag === "false") return "no";
118
91
  throw new Error("expected true or false");
119
92
  }
93
+
94
+ runCodecCLI(import.meta.url, { parse, print, doc });
120
95
  ```
121
96
 
122
- Load it in the REPL:
97
+ ### Using in the REPL:
123
98
 
124
99
  ```text
125
100
  > :codec load ./codecs/yesno.mjs
126
- loaded codec yesno for @...
127
- > :input <{} true, {} false> yesno
128
- input @... using yesno: enter value text
129
- yesno> yes
101
+ loaded codec yesno.mjs
102
+ > :input yesno.mjs yes
130
103
  {}|true ?<{} false, {} true>
131
- yesno: yes
132
- ```
133
-
134
- ## Add The CLI Boundary
135
-
136
- To make the same module usable as an installed command, add a `main` function
137
- that supports `--parse`, `--print`, and `--help`.
138
-
139
- ```js
140
- #!/usr/bin/env node
141
-
142
- import { stdin, stdout, argv, exit } from "node:process";
143
- import { decodeWire, encodeToWire } from "./runtime/prefix-codec.mjs";
144
- import { isMainEntrypoint } from "./runtime/cli-entry.mjs";
145
-
146
- function usage(stream = console.error) {
147
- stream(`Usage: ${argv[1]} --parse | --print`);
148
- stream(" --parse Read yes/no text, write binary pattern+value stream.");
149
- stream(" --print Read binary pattern+value stream, write yes/no text.");
150
- stream(" -h, --help Show this help.");
151
- }
152
-
153
- function readAll(stream) {
154
- return new Promise((resolve, reject) => {
155
- const chunks = [];
156
- stream.on("data", c => chunks.push(Buffer.isBuffer(c) ? c : Buffer.from(c)));
157
- stream.on("end", () => resolve(Buffer.concat(chunks)));
158
- stream.on("error", reject);
159
- });
160
- }
161
-
162
- async function main() {
163
- const args = argv.slice(2);
164
- if (args.includes("-h") || args.includes("--help")) {
165
- usage(console.log);
166
- exit(0);
167
- }
168
- if (args.length !== 1 || (args[0] !== "--parse" && args[0] !== "--print")) {
169
- usage();
170
- exit(1);
171
- }
172
-
173
- const input = await readAll(stdin);
174
- if (args[0] === "--parse") {
175
- stdout.write(encodeToWire(parse(input.toString("utf8")), BOOL_PATTERN));
176
- } else {
177
- stdout.write(`${print(decodeWire(input).value)}\n`);
178
- }
179
- }
180
-
181
- if (isMainEntrypoint(import.meta.url, argv[1])) {
182
- main().catch(error => {
183
- console.error(error.message || String(error));
184
- exit(1);
185
- });
186
- }
104
+ yesno.mjs: yes
187
105
  ```
188
106
 
189
- The command-line interface must write only the binary stream to stdout in
190
- `--parse` mode. Diagnostics belong on stderr.
191
-
192
- Installed codec binaries use the `k-` prefix plus the source basename without
193
- `.mjs`. For example, `codecs/yesno.mjs` installs as `k-yesno` when added to
194
- `package.json`.
107
+ ### Using as a CLI Pipeline Filter:
195
108
 
196
- ## Universal Codecs
197
-
198
- Universal codecs omit `codes` and `patterns`:
199
-
200
- ```js
201
- export const name = "json";
202
- export const universal = true;
203
-
204
- export function parse(text) {
205
- // Convert external text to a Value.
206
- }
207
-
208
- export function print(value) {
209
- // Convert a Value to external text.
210
- }
109
+ ```sh
110
+ printf 'yes\n' | ./codecs/yesno.mjs --parse | ./codecs/yesno.mjs --print
111
+ # Output: yes
211
112
  ```
212
113
 
213
- The built-in [`../codecs/json.mjs`](../codecs/json.mjs) codec is the reference
214
- example. It derives a pattern for command-line `--parse`, but in the REPL the
215
- requested `:input` type supplies the target type.
216
-
217
- ## Test A Codec
114
+ ## Testing a Codec
218
115
 
219
- For CLI codecs, test both directions:
116
+ For CLI codecs, test both directions in a pipeline:
220
117
 
221
118
  ```sh
222
119
  printf 'yes\n' | node codecs/yesno.mjs --parse | node codecs/yesno.mjs --print
@@ -227,17 +124,13 @@ For REPL codecs, add a focused case to [`../tests/test-repl.mjs`](../tests/test-
227
124
  ```js
228
125
  const state = createState();
229
126
  let output = await evaluateInput(":codec load ./codecs/yesno.mjs", state);
230
- assert.match(output[0], /^loaded codec yesno for @/);
231
- output = await evaluateInput(":input <{} true, {} false> yesno", state);
232
- assert.match(output[0], /^input @[^ ]+ using yesno: enter value text$/);
127
+ assert.match(output[0], /^loaded codec yesno\.mjs/);
128
+ output = await evaluateInput(":input yesno.mjs", state);
129
+ assert.equal(promptForState(state), "yesno.mjs> ");
233
130
  output = await evaluateInput("yes", state);
234
- assert.match(output[0], /yesno: yes/);
235
- ```
236
-
237
- Run the targeted test first:
238
-
239
- ```sh
240
- node tests/test-repl.mjs
131
+ assert.match(output[0], /yesno\.mjs: yes/);
132
+ output = await evaluateInput(":input yesno.mjs no", state);
133
+ assert.match(output[0], /yesno\.mjs: no/);
241
134
  ```
242
135
 
243
136
  Then run the full suite before committing:
@@ -246,13 +139,19 @@ Then run the full suite before committing:
246
139
  npm test
247
140
  ```
248
141
 
142
+ ## Web REPL (VFS) Bundling
143
+
144
+ When `repl.html` is generated (`npm run build:repl-html`):
145
+ - Codecs (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`) are bundled into 100% self-contained ES modules with zero imports.
146
+ - `runCodecCLI` is shimmed as a no-op, and `Value` helpers are directly inlined.
147
+ - In the browser, the VFS serves these self-contained modules directly to native dynamic `import(blobUrl)`. No runtime helper files (`codecs/runtime/*`) need to be included in the VFS or exposed as globals.
148
+
249
149
  ## Checklist
250
150
 
251
- - Export a stable `name`.
252
- - Export exactly one dispatch shape: `patterns`, `codes`, or `universal = true`.
253
- - Keep `parse` and `print` deterministic.
254
- - Return structural `Value` objects from `parse`.
255
- - Validate shapes in `print` and throw clear errors.
256
- - Keep command-line `--parse` stdout binary-only.
257
- - Add `-h` and `--help` for installed commands.
258
- - Add REPL coverage for `:codec load` and `:input`.
151
+ - Return an enveloped `Value` from `parse` using `withPattern(val, pattern)`.
152
+ - Keep `parse` and `print` deterministic and free of side effects.
153
+ - Validate shapes in `print` and throw clear errors when not representable.
154
+ - Use `runCodecCLI(import.meta.url, { parse, print, doc })` for CLI execution.
155
+ - No artificial `name` or `patterns` exports required.
156
+ - Add `-h` and `--help` support via `runCodecCLI`.
157
+
@@ -0,0 +1,194 @@
1
+ # Reusable Codec Architecture
2
+
3
+ ## 1. Overview & Objectives
4
+
5
+ In **k**, a **codec** is a two-way translator between external domain data (e.g. decimal integers, JSON, UTF-8 text, IEEE 754 floats) and **k** enveloped values (a structural `Value` tree paired with a closed pattern graph).
6
+
7
+ Every codec in the **k** ecosystem is designed to serve three unified objectives from a single module definition:
8
+
9
+ 1. **Document the Codec**: Formally specify the external syntax and its semantics as an enveloped **k** value `(P, v)`.
10
+ 2. **REPL Module**: Provide interactive input parsing (`:input <codec.mjs>`) and multi-codec output inspection (`<codec.mjs>: <formatted>`) in both Node.js and browser REPLs.
11
+ 3. **Standalone Shell Executable**: Function as a first-class Unix filter (`--parse`, `--print`, `--help`) for composition in command-line pipelines.
12
+
13
+ ```text
14
+ +---------------------------------------+
15
+ | External Domain Data |
16
+ | (JSON, Decimals, UTF-8, IEEE floats) |
17
+ +---------------------------------------+
18
+ | ^
19
+ parse(text) | | print(value)
20
+ v |
21
+ +---------------------------------------+
22
+ | k Codec |
23
+ | (Syntax & Value Mapper) |
24
+ +---------------------------------------+
25
+ | ^
26
+ CLI: --parse | | CLI: --print
27
+ REPL: :input v | REPL: output
28
+ +---------------------------------------+
29
+ | Enveloped k Value |
30
+ | (pattern graph P, value tree v) |
31
+ | Binary Wire Stream (P, v) |
32
+ +---------------------------------------+
33
+ ```
34
+
35
+ ---
36
+
37
+ ## 2. Objective 1: Documentation & Specification
38
+
39
+ Every codec defines what external text it accepts and how that text maps to an enveloped **k** value.
40
+
41
+ ### 2.1 External Syntax
42
+ The syntax specification defines the valid textual forms accepted by the codec:
43
+ - **Grammar & Rules**: Defines the accepted language. While regular expressions offer simple examples, the syntax can be an arbitrary formal language (context-free, recursive, JSON RFC 8259, etc.).
44
+ - **Normalization**: Rules for whitespace, casing, or escaping.
45
+ - **Rejection Criteria**: Explicit errors when invalid input is encountered.
46
+
47
+ ### 2.2 Semantics as an Enveloped k Value
48
+ Parsing external text produces an **enveloped value** `(P, v)`:
49
+ - **Pattern Graph ($P$)**: A closed pattern graph specifying the algebraic structure.
50
+ - **Value Tree ($v$)**: A concrete tree matching $P$, constructed from products (`{...}`) and variants (`<...>`).
51
+
52
+ A codec does not produce an ungrounded type; it performs a concrete transformation from text to an enveloped value `(P, v)`.
53
+
54
+ ### 2.3 Pattern Derivation via k Type System (`patternFromFilter`)
55
+ Pattern graphs ($P$) must never be handwritten as raw property-list JSON arrays (e.g. `[["closed-union", ...]]`). Instead, codecs derive pattern graphs directly through the **k** type system from filter or type expressions using `patternFromFilter`:
56
+
57
+ ```js
58
+ import { patternFromFilter } from "./runtime/codec-sdk.mjs";
59
+
60
+ // Derived from k filter expression
61
+ const INT_PATTERN = patternFromFilter('?< <bits 0, bits 1, {} _>=bits "+", bits "-">');
62
+ ```
63
+
64
+ This guarantees:
65
+ 1. All patterns are verified, normalized, and canonicalized by **k**'s type derivation and graph unification engine.
66
+ 2. Codecs remain concise, declarative, and maintainable.
67
+ 3. Codecs run identically in Node.js and in the Web REPL (via the browser runtime shim).
68
+
69
+ ### 2.4 Documentation Contract
70
+ A codec is identified solely by the file loaded—there is no artificial `name` export or `patterns` list export. A codec may optionally export a `doc` string:
71
+ - Describes accepted syntax, options, and representation.
72
+ - Displayed by `--help` in shell invocations.
73
+ - Displayed by `:help <codec.mjs>` or documentation commands in REPLs.
74
+
75
+ ---
76
+
77
+ ## 3. Objective 2: REPL Module Interface
78
+
79
+ In the REPL, codecs are loaded dynamically on demand:
80
+ ```text
81
+ > :codec load int.mjs
82
+ ```
83
+
84
+ ### 3.1 Module Exports
85
+ A codec module exports:
86
+
87
+ | Export | Type | Required | Description |
88
+ | :--- | :--- | :--- | :--- |
89
+ | `parse` | `(text: string) => EnvelopedValue \| Value` | **Yes** | Parses external text into an enveloped `Value`. Throws on syntax error. |
90
+ | `print` | `(value: Value) => string` | **Yes** | Formats a `Value` into external text. Throws if value is not representable. |
91
+ | `doc` | `string` | Optional | Documentation of syntax and usage for CLI help and REPL inspections. |
92
+
93
+ ### 3.2 REPL Input Protocol
94
+ The user specifies the codec to parse an input payload:
95
+ - **Interactive Mode**:
96
+ ```text
97
+ > :input int.mjs
98
+ int.mjs> 42
99
+ {}|_|0|1|0|1|0|1|+ ?<{} _, ...>
100
+ int.mjs: 42
101
+ ```
102
+ - **One-Line Mode**:
103
+ ```text
104
+ > :input int.mjs 42
105
+ {}|_|0|1|0|1|0|1|+ ?<{} _, ...>
106
+ int.mjs: 42
107
+ ```
108
+
109
+ ### 3.3 REPL Output Protocol (Multi-Codec Inspection)
110
+ Whenever an evaluation produces a resulting **k** `Value`:
111
+ 1. The primary representation is the standard **k** value envelope.
112
+ 2. The REPL iterates over loaded codecs and calls `codec.print(value)`.
113
+ 3. If `codec.print(value)` succeeds, the REPL displays `<codec>: <formattedString>`.
114
+ 4. If `codec.print(value)` throws, the codec is silently ignored for that value.
115
+
116
+ ---
117
+
118
+ ## 4. Objective 3: Standalone Shell Executable Contract
119
+
120
+ Every codec file is directly executable from the command line:
121
+ ```bash
122
+ chmod +x codecs/int.mjs
123
+ ```
124
+
125
+ ### 4.1 CLI Invocations
126
+
127
+ 1. **Parse Mode (`--parse`)**:
128
+ - Reads external text from `stdin`.
129
+ - Parses it into an enveloped value `(P, v)`.
130
+ - Writes the binary wire encoding of `(P, v)` to `stdout`.
131
+ - Diagnostics and errors are written to `stderr`.
132
+ - Exit code: `0` on success, non-zero on error.
133
+
134
+ 2. **Print Mode (`--print`)**:
135
+ - Reads binary wire bytes `(P, v)` from `stdin`.
136
+ - Decodes the wire stream and calls `print(value)`.
137
+ - Writes formatted text to `stdout`.
138
+ - Exit code: `0` on success, non-zero if not representable under this codec.
139
+
140
+ 3. **Help Mode (`-h`, `--help`)**:
141
+ - Outputs CLI usage and the codec's `doc` string to `stdout`.
142
+
143
+ ---
144
+
145
+ ## 5. Implementation: Codec CLI Runner
146
+
147
+ To avoid duplicating stdin/stdout stream handling and CLI flag parsing across codecs, a helper is provided:
148
+
149
+ ```javascript
150
+ runCodecCLI(import.meta.url, { parse, print, doc });
151
+ ```
152
+
153
+ ### Responsibilities of `runCodecCLI`:
154
+ 1. **Entrypoint Detection**: Checks if the module is invoked directly from the CLI (e.g. `import.meta.url === pathToFileURL(process.argv[1]).href`). If imported as a module or running in a browser, it is a no-op.
155
+ 2. **Flag Parsing**: Handles `--parse`, `--print`, and `-h` / `--help`.
156
+ 3. **Stream Piping**:
157
+ - For `--parse`: reads `stdin` to string, calls `parse(text)`, encodes enveloped value `(P, v)` to binary wire format on `stdout`.
158
+ - For `--print`: decodes wire format from `stdin` into `(P, v)`, calls `print(value)`, writes formatted text to `stdout`.
159
+
160
+ ---
161
+
162
+ ## 6. Directory Layout
163
+
164
+ To make `codecs/` intuitive and self-contained:
165
+
166
+ ```text
167
+ codecs/
168
+ ├── int.mjs # Integer & list codec
169
+ ├── json.mjs # JSON codec
170
+ ├── utf8.mjs # UTF-8 string codec
171
+ ├── utf16.mjs # UTF-16 string codec
172
+ ├── ieee.mjs # IEEE 754 float codec
173
+ ├── unit.mjs # Unit {} codec
174
+ ├── k-parse.mjs # Textual k value {...} -> binary wire stream
175
+ ├── k-print.mjs # Binary wire stream -> textual k value
176
+ ├── show.mjs # Wire stream pass-through inspector
177
+ └── runtime/ # Shared codec infrastructure
178
+ ├── codec-sdk.mjs # runCodecCLI, Value, encodeToWire, decodeWire
179
+ ├── prefix-codec.mjs # Low-level bitstream wire serialization
180
+ ├── codec.mjs # Abstract pattern graph algorithms
181
+ └── cli-entry.mjs # Entrypoint detection
182
+ ```
183
+
184
+ Each codec file directly under `codecs/` is both an executable CLI script and an importable module.
185
+
186
+ ---
187
+
188
+ ## 7. Web REPL & VFS Bundling Architecture
189
+
190
+ In the browser REPL (`repl.html`):
191
+ - **Self-Contained Modules**: At build time (`scripts/build-repl-html.mjs`), verified codecs in `codecs/` are compiled via `esbuild` into 100% self-contained ES modules with **zero imports**.
192
+ - **No Runtime Files in VFS**: `codecs/runtime/*` is not included in the browser VFS. The VFS only contains verified, self-contained user-facing files (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`).
193
+ - **Native Browser Loading**: In `repl-codecs.mjs`, the browser reads the codec source from VFS, wraps it in a Blob, and invokes native `await import(blobUrl)`. No regex manipulation, import stripping, or global environment injection is required.
194
+