@fraczak/k 6.10.1 → 6.10.3

package/DOCS/CODEC_ARCHITECTURE.md

@@ -2,13 +2,28 @@
 
 ## 1. Overview & Objectives
 
-In **k**, a **codec** is a two-way translator between external domain data (e.g. decimal integers, JSON, UTF-8 text, IEEE 754 floats) and **k** enveloped values (a structural `Value` tree paired with a closed pattern graph).
-
-Every codec in the **k** ecosystem is designed to serve three unified objectives from a single module definition:
-
-1. **Document the Codec**: Formally specify the external syntax and its semantics as an enveloped **k** value `(P, v)`.
-2. **REPL Module**: Provide interactive input parsing (`:input <codec.mjs>`) and multi-codec output inspection (`<codec.mjs>: <formatted>`) in both Node.js and browser REPLs.
-3. **Standalone Shell Executable**: Function as a first-class Unix filter (`--parse`, `--print`, `--help`) for composition in command-line pipelines.
+In **k**, a **codec** is a two-way translator between external domain
+data (e.g. decimal integers, JSON, UTF-8 text, IEEE 754 floats) and **k**
+enveloped values (a structural `Value` tree paired with a closed pattern
+graph).
+
+In **k**, there are no primitive numbers or built-in numerical types.
+Numbers are nullary relations mapping the unit to bit trees
+(`${} -> $bits`), and arithmetic relations operate on signed
+integer variants (`$int`). Codecs like `int.mjs` bridge human
+decimal representations and these algebraic structures.
+
+Every codec in the **k** ecosystem is designed to serve three unified
+objectives from a single module definition:
+
+1. **Document the Codec**: Formally specify the external syntax and its
+   semantics as an enveloped **k** value `(P, v)`.
+2. **REPL Module**: Provide interactive input parsing (`:input
+   <codec.mjs>`) and multi-codec output inspection (`<codec.mjs>:
+   <formatted>`) in both Node.js and browser REPLs.
+3. **Standalone Shell Executable**: Function as a first-class Unix filter
+   (`--parse`, `--print`, `--help`) for composition in command-line
+   pipelines.
 
 ```text
                      +---------------------------------------+
@@ -36,38 +51,55 @@ Every codec in the **k** ecosystem is designed to serve three unified objectives
 
 ## 2. Objective 1: Documentation & Specification
 
-Every codec defines what external text it accepts and how that text maps to an enveloped **k** value.
+Every codec defines what external text it accepts and how that text
+maps to an enveloped **k** value.
 
 ### 2.1 External Syntax
-The syntax specification defines the valid textual forms accepted by the codec:
-- **Grammar & Rules**: Defines the accepted language. While regular expressions offer simple examples, the syntax can be an arbitrary formal language (context-free, recursive, JSON RFC 8259, etc.).
+The syntax specification defines the valid textual forms accepted by
+the codec:
+- **Grammar & Rules**: Defines the accepted language. While regular
+  expressions offer simple examples, the syntax can be an arbitrary
+  formal language (context-free, recursive, JSON RFC 8259, etc.).
 - **Normalization**: Rules for whitespace, casing, or escaping.
-- **Rejection Criteria**: Explicit errors when invalid input is encountered.
+- **Rejection Criteria**: Explicit errors when invalid input is
+  encountered.
 
 ### 2.2 Semantics as an Enveloped k Value
 Parsing external text produces an **enveloped value** `(P, v)`:
-- **Pattern Graph ($P$)**: A closed pattern graph specifying the algebraic structure.
-- **Value Tree ($v$)**: A concrete tree matching $P$, constructed from products (`{...}`) and variants (`<...>`).
+- **Pattern Graph ($P$)**: A closed pattern graph specifying the
+  algebraic structure.
+- **Value Tree ($v$)**: A concrete tree matching $P$, constructed from
+  products (`{...}`) and variants (`<...>`).
 
-A codec does not produce an ungrounded type; it performs a concrete transformation from text to an enveloped value `(P, v)`.
+A codec does not produce an ungrounded type; it performs a concrete
+transformation from text to an enveloped value `(P, v)`.
 
 ### 2.3 Pattern Derivation via k Type System (`patternFromFilter`)
-Pattern graphs ($P$) must never be handwritten as raw property-list JSON arrays (e.g. `[["closed-union", ...]]`). Instead, codecs derive pattern graphs directly through the **k** type system from filter or type expressions using `patternFromFilter`:
+Pattern graphs ($P$) must never be handwritten as raw property-list
+JSON arrays (e.g. `[["closed-union", ...]]`). Instead, codecs derive
+pattern graphs directly through the **k** type system from filter or
+type expressions using `patternFromFilter`:
 
 ```js
 import { patternFromFilter } from "./runtime/codec-sdk.mjs";
 
 // Derived from k filter expression
-const INT_PATTERN = patternFromFilter('?< <bits 0, bits 1, {} _>=bits "+", bits "-">');
+const INT_PATTERN = patternFromFilter(
+  '?< <bits 0, bits 1, {} _>=bits "+", bits "-">'
+);
 ```
 
 This guarantees:
-1. All patterns are verified, normalized, and canonicalized by **k**'s type derivation and graph unification engine.
+1. All patterns are verified, normalized, and canonicalized by **k**'s
+   type derivation and graph unification engine.
 2. Codecs remain concise, declarative, and maintainable.
-3. Codecs run identically in Node.js and in the Web REPL (via the browser runtime shim).
+3. Codecs run identically in Node.js and in the Web REPL (via the browser
+   runtime shim).
 
 ### 2.4 Documentation Contract
-A codec is identified solely by the file loaded—there is no artificial `name` export or `patterns` list export. A codec may optionally export a `doc` string:
+A codec is identified solely by the file loaded—there is no artificial
+`name` export or `patterns` list export. A codec may optionally export
+a `doc` string:
 - Describes accepted syntax, options, and representation.
 - Displayed by `--help` in shell invocations.
 - Displayed by `:help <codec.mjs>` or documentation commands in REPLs.
@@ -110,8 +142,10 @@ The user specifies the codec to parse an input payload:
 Whenever an evaluation produces a resulting **k** `Value`:
 1. The primary representation is the standard **k** value envelope.
 2. The REPL iterates over loaded codecs and calls `codec.print(value)`.
-3. If `codec.print(value)` succeeds, the REPL displays `<codec>: <formattedString>`.
-4. If `codec.print(value)` throws, the codec is silently ignored for that value.
+3. If `codec.print(value)` succeeds, the REPL displays
+   `<codec>: <formattedString>`.
+4. If `codec.print(value)` throws, the codec is silently ignored for
+   that value.
 
 ---
 
@@ -135,7 +169,8 @@ chmod +x codecs/int.mjs
    - Reads binary wire bytes `(P, v)` from `stdin`.
    - Decodes the wire stream and calls `print(value)`.
    - Writes formatted text to `stdout`.
-   - Exit code: `0` on success, non-zero if not representable under this codec.
+   - Exit code: `0` on success, non-zero if not representable under this
+     codec.
 
 3. **Help Mode (`-h`, `--help`)**:
    - Outputs CLI usage and the codec's `doc` string to `stdout`.
@@ -144,18 +179,24 @@ chmod +x codecs/int.mjs
 
 ## 5. Implementation: Codec CLI Runner
 
-To avoid duplicating stdin/stdout stream handling and CLI flag parsing across codecs, a helper is provided:
+To avoid duplicating stdin/stdout stream handling and CLI flag parsing
+across codecs, a helper is provided:
 
 ```javascript
 runCodecCLI(import.meta.url, { parse, print, doc });
 ```
 
 ### Responsibilities of `runCodecCLI`:
-1. **Entrypoint Detection**: Checks if the module is invoked directly from the CLI (e.g. `import.meta.url === pathToFileURL(process.argv[1]).href`). If imported as a module or running in a browser, it is a no-op.
+1. **Entrypoint Detection**: Checks if the module is invoked directly
+   from the CLI (e.g. `import.meta.url ===
+   pathToFileURL(process.argv[1]).href`). If imported as a module or
+   running in a browser, it is a no-op.
 2. **Flag Parsing**: Handles `--parse`, `--print`, and `-h` / `--help`.
 3. **Stream Piping**:
-   - For `--parse`: reads `stdin` to string, calls `parse(text)`, encodes enveloped value `(P, v)` to binary wire format on `stdout`.
-   - For `--print`: decodes wire format from `stdin` into `(P, v)`, calls `print(value)`, writes formatted text to `stdout`.
+   - For `--parse`: reads `stdin` to string, calls `parse(text)`, encodes
+     enveloped value `(P, v)` to binary wire format on `stdout`.
+   - For `--print`: decodes wire format from `stdin` into `(P, v)`, calls
+     `print(value)`, writes formatted text to `stdout`.
 
 ---
 
@@ -181,14 +222,29 @@ codecs/
     └── cli-entry.mjs          # Entrypoint detection
 ```
 
-Each codec file directly under `codecs/` is both an executable CLI script and an importable module.
+Each codec file directly under `codecs/` is both an executable CLI
+script and an importable module.
 
 ---
 
 ## 7. Web REPL & VFS Bundling Architecture
 
 In the browser REPL (`repl.html`):
-- **Self-Contained Modules**: At build time (`scripts/build-repl-html.mjs`), verified codecs in `codecs/` are compiled via `esbuild` into 100% self-contained ES modules with **zero imports**.
-- **No Runtime Files in VFS**: `codecs/runtime/*` is not included in the browser VFS. The VFS only contains verified, self-contained user-facing files (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`).
-- **Native Browser Loading**: In `repl-codecs.mjs`, the browser reads the codec source from VFS, wraps it in a Blob, and invokes native `await import(blobUrl)`. No regex manipulation, import stripping, or global environment injection is required.
-
+- **Self-Contained Modules**: At build time
+  (`scripts/build-repl-html.mjs`), verified codecs in `codecs/` are
+  compiled via `esbuild` into 100% self-contained ES modules with **zero
+  imports**.
+- **No Runtime Files in VFS**: `codecs/runtime/*` is not included in the
+  browser VFS. The VFS only contains verified, self-contained user-facing
+  files (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`).
+- **Native Browser Loading**: In `repl-codecs.mjs`, the browser reads the
+  codec source from VFS, wraps it in a Blob, and invokes native `await
+  import(blobUrl)`. No regex manipulation, import stripping, or global
+  environment injection is required.
+
+## Further Reading
+
+- [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to
+  .k, .ko, .klib, and .kvm formats
+- [DOCS/CODECS.md](./CODECS.md) — guide to writing codecs
+- [DOCS/DICTIONARY.md](./DICTIONARY.md) — concept names and terminology

package/DOCS/CONVERGENCE.md

@@ -1,6 +1,7 @@
 # Convergence Strategies
 
-This note describes the current structure of the JavaScript type-derivation engine and the places intended for experimentation.
+This note describes the current structure of the JavaScript
+type-derivation engine and the places intended for experimentation.
 
 ## Current default
 
@@ -12,9 +13,11 @@ Type derivation operates on the SCC DAG of relation dependencies.
   - compact once
 - Recursive SCC:
   - use `fixed_point`
-  - iterate until signatures stabilize or the iteration budget is exhausted
+  - iterate until signatures stabilize or the iteration budget is
+    exhausted
 
-This split keeps acyclic modules fast while preserving the old behavior for recursive polymorphic definitions.
+This split keeps acyclic modules fast while preserving the old
+behavior for recursive polymorphic definitions.
 
 ## Public tuning hook
 
@@ -51,22 +54,37 @@ This is the supported inspection surface for convergence experiments.
 
 ## Why the current fast path works
 
-For an acyclic singleton SCC, every callee has already been fully derived by the time the current relation is processed. There is no need for repeated fixed-point propagation because:
+For an acyclic singleton SCC, every callee has already been fully
+derived by the time the current relation is processed. There is no
+need for repeated fixed-point propagation because:
 
 1. Callee boundary patterns are stable.
 2. The current relation only needs those stable input/output boundaries.
 3. One propagation pass plus one compaction is sufficient.
 
-`Examples/ieee.k` is exactly this shape: a large acyclic dependency graph. The major speedup comes from avoiding unnecessary fixed-point iteration there.
+`Examples/ieee.k` is exactly this shape: a large acyclic dependency
+graph. The major speedup comes from avoiding unnecessary fixed-point
+iteration there.
 
 ## Current implementation choices
 
 Two implementation changes matter for performance:
 
-1. Compaction now starts from the quotient graph of live representatives instead of cloning the full historical union-find forest.
-2. Singleton-pattern registration is incremental and no longer re-finalizes the entire global code repository on every compaction.
+1. Compaction now starts from the quotient graph of live
+   representatives instead of cloning the full historical union-find
+   forest.
+2. Singleton-pattern registration is incremental and no longer
+   re-finalizes the entire global code repository on every compaction.
 
-These two changes are what make `Examples/ieee.k` practical to type-check.
+These two changes are what make `Examples/ieee.k` practical to
+type-check.
+
+## Role in Ahead-Of-Time (AOT) Compilation
+
+Convergence strategies dictate the time spent during Ahead-Of-Time
+(AOT) type checking. Because compilation resolves and converges these
+dependency SCCs beforehand into `.ko`, `.klib`, and `.kvm` artifacts,
+runtime execution does not incur any convergence overhead.
 
 ## Where to experiment next
 
@@ -86,7 +104,20 @@ If you want to continue convergence work, these are the main seams:
 
 Promising next experiments:
 
-- Replace signature-string stabilization in `fixed_point` with structural change tracking.
+- Replace signature-string stabilization in `fixed_point` with
+  structural change tracking.
 - Add divergence heuristics for recursive polymorphic SCCs.
-- Cache cloned callee boundary graphs for repeated call sites within one relation.
-- Compare the current partition refinement in `compression.mjs` with a hash-consed canonical-form approach.
+- Cache cloned callee boundary graphs for repeated call sites within
+  one relation.
+- Compare the current partition refinement in `compression.mjs` with
+  a hash-consed canonical-form approach.
+
+## Further Reading
+
+- [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to
+  .k, .ko, .klib, and .kvm formats
+- [DOCS/TYPE_DERIVATION.md](./TYPE_DERIVATION.md) — type derivation
+  algorithm and rules
+- [DOCS/OBJECT_FILE_AND_PATTERN.md](./OBJECT_FILE_AND_PATTERN.md) —
+  object format and pattern encoding
+- [DOCS/DICTIONARY.md](./DICTIONARY.md) — concept names and terminology

package/DOCS/DICTIONARY.md

@@ -69,6 +69,34 @@ pattern and otherwise behaves like identity.
 Source syntax may present this operation as a filter expression. The semantic
 operation is a pattern guard.
 
+### Canonical Content-Addressed Hash (`@hash`)
+
+A `canonical content-addressed hash` is the immutable, unique identifier
+derived from the normalized AST of a relation or the minimal deterministic
+tree automaton of a type.
+
+In `k`, there is no global symbol namespace. Every compiled relation and
+type code is identified globally and uniquely by its `@hash`.
+
+### Local Alias
+
+A `local alias` is a human-readable identifier (such as `plus`, `int`,
+or `5`) mapped to a canonical hash in object metadata (`relAlias`,
+`typeAliases`).
+
+Aliases are local to their compilation scope. When loading libraries
+with `--lib`, canonical definitions are available by their `@hash`, but
+local aliases are not automatically exported into the consumer's scope
+unless explicitly mapped with `--export`.
+
+### Relations as Values (No Built-in Numbers)
+
+In `k`, there are no primitive numerical types or built-in integer
+literals. Numerical identifiers such as `5` or `10` are nullary
+relations (constants) mapping the unit `${}` to bit trees of type
+`${} -> $bits`. Operations on signed integers (`$int`) require
+converting bit trees via the `int` constructor (`5 int`).
+
 ## Values, Types, And Patterns
 
 ### Value Tree
@@ -235,6 +263,16 @@ The key product is still a k-shaped core expression graph, now annotated by
 type-pattern facts. This stage is a semantic artifact; it does not imply a
 particular serialized format.
 
+### Ahead-Of-Time (AOT) Type Checking
+
+`Ahead-of-time (AOT) type checking` refers to the compilation phase
+where parsing, relation expansion, structural type derivation, and
+fixed-point constraint convergence are solved in advance.
+
+The resulting compiled artifacts (`.ko`, `.klib`, `.kvm`) store fully
+derived principal type graphs and interned type codes, allowing downstream
+execution or linking to skip cold-start type derivation entirely.
+
 ### k Object
 
 A `k object` is a compiled semantic object.
@@ -242,8 +280,37 @@ A `k object` is a compiled semantic object.
 It stores canonical type codes, partial functions, aliases, metadata,
 type-pattern graphs, and type-derivation status.
 
-An executable object has a main partial function. A library object contains
-reusable definitions but no selected main partial function.
+An executable object (`.ko`) has an execution entrypoint (`main:
+"__main__"`). A library object (`.klib`) contains reusable definitions
+but no selected main relation (`main: null`).
+
+### File Formats in the K Universe
+
+The `k` toolchain defines four primary file formats (for comprehensive
+details and workflows, see [`DOCS/FILE_FORMATS.md`](FILE_FORMATS.md)):
+
+- **Source File (`.k`):** Plain UTF-8 source text defining algebraic data
+  shapes, partial functions, and expressions.
+- **Executable Object (`.ko`):** AOT type-checked executable artifact
+  with binary header `KOBJ\n` and JSON payload containing `codes`, `rels`,
+  and entrypoint `main: "__main__"`. Executable directly via `k.mjs`,
+  WebAssembly, or LLVM.
+- **Library Object (`.klib`):** AOT type-checked library container stored
+  as plain UTF-8 JSON without an entrypoint (`main: null`). Provides
+  reusable pre-converged relations and type graphs for downstream linking.
+- **Polymorphic kVM Template (`.kvm`):** Register-IR bytecode template
+  (`format: "k-vm"`, `layer: "KVM-P"`) carrying principal pattern graphs.
+  Specialized on-the-fly at runtime (`specializeKVM`) against concrete
+  input envelopes $(v, P_v)$ without re-running type inference.
+
+### Stream Transformer vs. Closed Constant Relation
+
+- A **stream transformer** is a relation with an open domain (e.g. `{() x,
+  5 int y} plus`) that consumes external data arriving from a binary
+  stream or standard input.
+- A **closed constant relation** is a relation with no free inputs (such
+  as `{10 int x, 5 int y} plus`). Its domain is the unit type `${}`,
+  requiring an empty unit `{}` when executed as a standalone script.
 
 ### KIR-P