@fraczak/k 6.10.2 → 6.10.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,36 @@ All notable changes to this project will be documented here.
5
5
  The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
6
6
  This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [6.10.3] — 2026-10-04 — Unified File Formats Specification and Documentation Modernization
9
+
10
+ ### Documentation & Core Architecture
11
+
12
+ - **Unified File Formats Specification (`DOCS/FILE_FORMATS.md`)**:
13
+ - Added comprehensive specification for all primary file formats: `.k`
14
+ (source), `.ko` (executable object), `.klib` (library object), and
15
+ `.kvm` (polymorphic register-IR template), plus target artifacts
16
+ (`.wasm`, native binary).
17
+ - Added Mermaid transformation flowchart mapping compilation, linking,
18
+ specialization, and execution workflows across CLI tools.
19
+ - **AOT Type Checking & Canonical Content-Addressing**:
20
+ - Formally specified that AOT type checking performs relation expansion,
21
+ structural type derivation, and constraint convergence in advance.
22
+ - Clarified canonical content-addressed hashes (`@hash`) vs. local aliases,
23
+ confirming there is no global symbol namespace, and documented explicit
24
+ `--export` semantics for `--lib`.
25
+ - **Relations as Values & Absence of Built-in Numbers**:
26
+ - Documented that numerical literals do not exist in `k`; names like `5`
27
+ or `10` are nullary relations mapping `${} -> $bits`. Arithmetic relations
28
+ operate on signed integer variants (`$int`), converting via `5 int`.
29
+ - Clarified stream transformers (open domain) vs. closed constant relations
30
+ (unit input domain `${}`).
31
+ - **Documentation Reformatting**:
32
+ - Systematically updated and wrapped prose across `README.md` and all
33
+ documents in `DOCS/`, `codecs/`, `objects/`, and `tests/` to <= 74
34
+ characters per line.
35
+
36
+ ---
37
+
8
38
  ## [6.10.2] — 2026-10-03 — Stack-Safe Value Serializer for REPL and Web REPL
9
39
 
10
40
  ### REPL & Codecs
@@ -1,28 +1,32 @@
1
1
  # Big Picture
2
2
 
3
- `Kernel-code` is an experiment aiming to define a "perfect programming language".
3
+ `Kernel-code` is an experiment aiming to define a "perfect programming
4
+ language".
4
5
 
5
6
  ## Basics
6
7
 
7
- The language describes "partial first-order functions" on "printable" values.
8
- The printable values can be seen as rooted, edge-labeled trees, intuitively similar to an
9
- XML or a JSON document. The values are always typed; in the current codec-aware
10
- runtime, a value may carry a pattern describing the family of types compatible
11
- with that tree.
8
+ The language describes "partial first-order functions" on "printable"
9
+ values. The printable values can be seen as rooted, edge-labeled trees,
10
+ intuitively similar to an XML or a JSON document. The values are always
11
+ typed; in the current codec-aware runtime, a value may carry a pattern
12
+ describing the family of types compatible with that tree.
12
13
 
13
14
  ### Types (codes)
14
15
 
15
- Formally, a _type_ is a finite tree automaton. The type defines a set of values,
16
- which corresponds to the set of trees accepted by the automaton.
17
- Therefore, every type admits a unique normal form in terms of its minimal deterministic tree automaton.
16
+ Formally, a _type_ is a finite tree automaton. The type defines a set of
17
+ values, which corresponds to the set of trees accepted by the
18
+ automaton. Therefore, every type admits a unique normal form in terms of
19
+ its minimal deterministic tree automaton.
18
20
 
19
21
  Types are defined by two constructs:
20
22
 
21
23
  1. Product
22
24
  2. Union (also called tagged or disjoint union)
23
25
 
24
- > Notation: We use native k-like notation as canonical — products `{ T l, ... }` and unions `< T t, ... >`.
25
- > For convenience, JSON-like forms `{ l: T, ... }` and `< t: T, ... >` are also supported and equivalent in meaning.
26
+ > Notation: We use native k-like notation as canonical — products
27
+ > `{ T l, ... }` and unions `< T t, ... >`. For convenience, JSON-like
28
+ > forms `{ l: T, ... }` and `< t: T, ... >` are also supported and
29
+ > equivalent in meaning.
26
30
 
27
31
  ```bnf
28
32
  type ::= name | '{' type_label_list '}' | '<' type_label_list '>'
@@ -31,18 +35,23 @@ name ::= IDENTIFIER
31
35
  label ::= STRING
32
36
  ```
33
37
 
34
- Given a finite (possibly empty) set of types `t1, t2, ..., tn` and a set of pairwise different tags (labels) `l1, l2, ..., ln`,
38
+ Given a finite (possibly empty) set of types `t1, t2, ..., tn` and a
39
+ set of pairwise different tags (labels) `l1, l2, ..., ln`,
35
40
 
36
- - by `{ t1 l1, t2 l2, ..., tn ln }`, we define a _product type_ with exactly `n`
37
- _projection_ functions: `.l1`, `.l2`, `...`, `.ln`.
38
- Each projection function maps a value of type `{t1 l1, t2 l2, ..., tn ln}`
39
- into a value of type `t1`, `t2`, `...`, `tn`, respectively.
40
- - by `< t1 l1, t2 l2, ..., tn ln >`, we define a _union type_ with exactly `n` projection functions `/l1`, `/l2`, `...`, `/ln`,
41
- mapping a value of type `<t1 l1, t2 l2, ..., tn ln>` into a value of type `t1`, `t2`, `...`, `tn`, respectively.
41
+ - by `{ t1 l1, t2 l2, ..., tn ln }`, we define a _product type_ with
42
+ exactly `n` _projection_ functions: `.l1`, `.l2`, `...`, `.ln`.
43
+ Each projection function maps a value of type
44
+ `{t1 l1, t2 l2, ..., tn ln}` into a value of type `t1`, `t2`,
45
+ `...`, `tn`, respectively.
46
+ - by `< t1 l1, t2 l2, ..., tn ln >`, we define a _union type_ with
47
+ exactly `n` projection functions `/l1`, `/l2`, `...`, `/ln`,
48
+ mapping a value of type `<t1 l1, t2 l2, ..., tn ln>` into a value
49
+ of type `t1`, `t2`, `...`, `tn`, respectively.
42
50
 
43
- Intuitively, the projection functions on a product type are all total (meaning, they are defined for every value of the
44
- input product type), whereas in the case of a union type, exactly one projection function is defined
45
- for a given value of the union type.
51
+ Intuitively, the projection functions on a product type are all total
52
+ (meaning, they are defined for every value of the input product type),
53
+ whereas in the case of a union type, exactly one projection function
54
+ is defined for a given value of the union type.
46
55
 
47
56
  Some properties of these types:
48
57
 
@@ -51,26 +60,32 @@ Some properties of these types:
51
60
 
52
61
  #### Variant (union) value literal convention
53
62
 
54
- - Variant values are written using "|" (pipe) followed by the variant tag.
63
+ - Variant values are written using "|" (pipe) followed by the variant
64
+ tag.
55
65
  - Unit variant `` `{}` `` | tag (e.g., `{} | nil`, `{} | zero`)
56
66
  - Variant with payload `v` at label `tag`: ` v | tag `
57
67
  - Example (list): with `$list = < {} nil, {X car, list cdr} cons >`:
58
68
  - Empty list []: `{}|nil`
59
69
  - Singleton `[v]`: `{ v car, {}|nil cdr } | cons`
60
- - Angle brackets are for type definitions and merge expressions; use "`| tag`" for a variant construction.
70
+ - Angle brackets are for type definitions and merge expressions; use
71
+ "`| tag`" for a variant construction.
61
72
 
62
73
  #### Equivalence of types (bisimilarity)
63
74
 
64
- Two (possibly recursive) codes are equivalent iff they are bisimilar over their definition graphs. Concretely, there exists a relation B such that `(t1, t2) ∈ B` if and only if:
75
+ Two (possibly recursive) codes are equivalent iff they are bisimilar
76
+ over their definition graphs. Concretely, there exists a relation B
77
+ such that `(t1, t2) ∈ B` if and only if:
65
78
  - t1 and t2 have exactly the same set of labels/tags;
66
79
  - t1 and t2 are simultaneously products or simultaneously unions;
67
- - for each label/tag `ℓ` in the set, the subcodes under `ℓ` are again related by B.
80
+ - for each label/tag `ℓ` in the set, the subcodes under `ℓ` are again
81
+ related by B.
68
82
 
69
83
  ### Filters (patterns)
70
84
 
71
- A _filter_ is a way of representing a _pattern_, i.e., a set of types sharing some structure.
72
- For example, if I wanted to refer to all product types that have a field `f` of type `T`,
73
- I would write: `?{ T f, ... }`. In general, a filter is defined by:
85
+ A _filter_ is a way of representing a _pattern_, i.e., a set of types
86
+ sharing some structure. For example, if I wanted to refer to all product
87
+ types that have a field `f` of type `T`, I would write:
88
+ `?{ T f, ... }`. In general, a filter is defined by:
74
89
 
75
90
  ```bnf
76
91
  filter ::= name | '$' type_expr | '{' filter_label_list '}'
@@ -92,29 +107,36 @@ Examples:
92
107
 
93
108
  ### Partial functions
94
109
 
95
- A _partial function_ is a function that is not defined for every value of its domain.
110
+ A _partial function_ is a function that is not defined for every value
111
+ of its domain.
96
112
 
97
113
  There are three ways of combining partial functions:
98
114
 
99
- 1. __Composition__: we write a composition of `f` and `g` as `(f g)`,
100
- meaning that if `f` is defined on value `x` producing `y`,
101
- and `g` is defined on value `y` producing `z`, then `(f g)` is defined on value `x` producing `z`.
102
- 2. __Union__: we write `< f1, f2 >` for a union of two functions `f1` and `f2`, meaning that
103
- if `f1` is defined on value `x` producing `y`, then `< f1, f2 >`
104
- is defined on value `x` producing `y`. Otherwise, i.e., when `f1` is not defined for `x`, `< f1, f2 >` acts as `f2`.
105
- 3. __Product__: we write `{f1 lab1, f2 lab2 }` for a product of two functions `f1` and `f2`,
106
- meaning that if `f1` and `f2` are both defined on value `x` producing `y1` and `y2`,
107
- respectively, then `{f1 lab1, f2 lab2 }` is defined on value `x` producing
108
- `{y1 lab1, y2 lab2}`.
109
- Otherwise, `{f1 lab1, f2 lab2 }` is not defined for `x`.
110
-
111
- Actually, composition, union, and product combine any number of partial functions, not only two. For example,
112
- empty composition, `()`, defines the identity function.
115
+ 1. __Composition__: we write a composition of `f` and `g` as `(f g)`,
116
+ meaning that if `f` is defined on value `x` producing `y`, and `g`
117
+ is defined on value `y` producing `z`, then `(f g)` is defined on
118
+ value `x` producing `z`.
119
+ 2. __Union__: we write `< f1, f2 >` for a union of two functions `f1`
120
+ and `f2`, meaning that if `f1` is defined on value `x` producing
121
+ `y`, then `< f1, f2 >` is defined on value `x` producing `y`.
122
+ Otherwise, i.e., when `f1` is not defined for `x`, `< f1, f2 >`
123
+ acts as `f2`.
124
+ 3. __Product__: we write `{f1 lab1, f2 lab2 }` for a product of two
125
+ functions `f1` and `f2`, meaning that if `f1` and `f2` are both
126
+ defined on value `x` producing `y1` and `y2`, respectively, then
127
+ `{f1 lab1, f2 lab2 }` is defined on value `x` producing
128
+ `{y1 lab1, y2 lab2}`. Otherwise, `{f1 lab1, f2 lab2 }` is not
129
+ defined for `x`.
130
+
131
+ Actually, composition, union, and product combine any number of
132
+ partial functions, not only two. For example, empty composition,
133
+ `()`, defines the identity function.
113
134
 
114
135
  ### Program
115
136
 
116
- A `kernel-core` program is a set of definitions of types and partial functions, followed by an _expression_,
117
- which is the partial function the program defines.
137
+ A `kernel-core` program is a set of definitions of types and partial
138
+ functions, followed by an _expression_, which is the partial function
139
+ the program defines.
118
140
 
119
141
  The language syntax is defined by:
120
142
 
@@ -132,47 +154,65 @@ expr_label_list ::= /* empty */
132
154
  | (expression label_name ',')* expression label
133
155
  ```
134
156
 
135
- We assume that in a `kernel-core` program the names (for types, functions, and labels) are identifiers (strings) such that:
157
+ We assume that in a `kernel-core` program the names (for types,
158
+ functions, and labels) are identifiers (strings) such that:
136
159
 
137
160
  1. All defined type names are distinct.
138
161
  2. All defined function names are distinct.
139
- 3. All label names within a type definition or a `product` expression are locally distinct.
162
+ 3. All label names within a type definition or a `product` expression
163
+ are locally distinct.
140
164
 
141
- Filters and type expressions act as a type annotation, i.e., an identity
142
- function defined only for the values of the corresponding types.
165
+ Filters and type expressions act as a type annotation, i.e., an
166
+ identity function defined only for the values of the corresponding
167
+ types.
143
168
 
144
169
  That's it.
145
170
 
146
- No `builtin` types, no `if` statement, no `loop`, no `throw`, no _closure_, no _annotations_, and no macros.
147
- Just _types_, _filters_, and _functions_.
171
+ No `builtin` types, no `if` statement, no `loop`, no `throw`, no
172
+ _closure_, no _annotations_, and no macros. Just _types_, _filters_,
173
+ and _functions_.
148
174
 
149
- ### Normalization
175
+ ### Normalization and Content-Addressing
150
176
 
151
- All type names are replaced by the hash of their canonical representation.
177
+ All type names are replaced by the canonical hash (`@hash`) of their
178
+ minimal deterministic tree automaton representation.
152
179
 
153
- We replace the function name by the hash of the normalized definition of the function as provided by the program defining the function.
154
- Obviously, a function may have many different definitions so the hash-based names are not canonical. By using hash-based function names we avoid the problem of name clashes when bringing together many `kernel-core` programs.
180
+ Every relation is identified canonically by the content-addressed hash
181
+ of its normalized definition AST.
155
182
 
156
- This approach solves the problem of modules, imports, etc., and opens the door to a universal
157
- registry of types (schemas) and functions.
183
+ There is no global symbol namespace. Human-readable names (such as
184
+ `plus`, `int`, or `5`) are local aliases recorded in object metadata
185
+ (`relAlias`, `typeAliases`). When libraries are loaded with `--lib`,
186
+ definitions are identified by their canonical `@hash`, but local aliases
187
+ are not implicitly exported into the importing scope unless explicitly
188
+ mapped via `--export`.
189
+
190
+ This approach completely solves the problem of modules, name collisions,
191
+ and unversioned imports, opening the door to a universal registry of
192
+ types (schemas) and relations.
158
193
 
159
194
  ## Typing and Polymorphism (codes and patterns)
160
195
 
161
- The abstract syntax tree of a program consists of a dictionary of _type definitions_,
162
- a dictionary of _function definitions_, and the final _main expression_.
196
+ The abstract syntax tree of a program consists of a dictionary of
197
+ _type definitions_, a dictionary of _function definitions_, and the
198
+ final _main expression_.
163
199
 
164
- In principle, the initial form of the Abstract Syntax Tree, _raw_ AST, is enough to evaluate
165
- the program on an input.
200
+ In principle, the initial form of the Abstract Syntax Tree, _raw_ AST,
201
+ is enough to evaluate the program on an input.
166
202
 
167
- Our objective, however, is to compute canonical representations for types and normalized
168
- representations for functions so they can be reused.
203
+ Our objective, however, is to compute canonical representations for
204
+ types and normalized representations for functions so they can be
205
+ reused and compiled ahead-of-time.
169
206
 
170
207
  The normalization steps are:
171
208
 
172
- 1. Build the _type graph_ from all types used by the program, i.e., used in _type definitions_
173
- as well as type expressions used in _function definitions_ and the _main expression_.
174
- 2. Annotate the expressions with filters; every node of the AST is annotated with a pair of filters.
175
- 3. Turn the _singleton_ filters into types and add them to the _type graph_. Go to (2) unless no change.
209
+ 1. Build the _type graph_ from all types used by the program, i.e.,
210
+ used in _type definitions_ as well as type expressions used in
211
+ _function definitions_ and the _main expression_.
212
+ 2. Annotate the expressions with filters; every node of the AST is
213
+ annotated with a pair of filters.
214
+ 3. Turn the _singleton_ filters into types and add them to the _type
215
+ graph_. Go to (2) unless no change.
176
216
 
177
217
  ### Examples
178
218
 
@@ -213,21 +253,25 @@ inc3
213
253
  In the above program, we define type `bit` as union of two unit types,
214
254
  and type `byte` as product, for simplicity, of four `bit`s.
215
255
 
216
- Functions `bit0`, `bit1`, and `zero` are "constant polymorphic functions", meaning:
256
+ Functions `bit0`, `bit1`, and `zero` are "constant polymorphic
257
+ functions", meaning:
217
258
 
218
259
  - constant: if defined, they always return exactly the same value;
219
- - polymorphic: they are defined for more than one pair of input and output types: functions
220
- `bit0` and `bit1` are of type `?X -> $bit`, and `zero` is of type `?X -> $byte`, where `?X`
221
- denotes an unconstrained type pattern, also denoted as `?(...)`.
260
+ - polymorphic: they are defined for more than one pair of input and
261
+ output types: functions `bit0` and `bit1` are of type `?X -> $bit`,
262
+ and `zero` is of type `?X -> $byte`, where `?X` denotes an
263
+ unconstrained type pattern, also denoted as `?(...)`.
222
264
 
223
265
  Functions `inc` and `inc3` are (non-polymorphic) functions of types
224
266
  `$byte -> ${ bit overflown, byte byte }` and
225
- `${ bit overflown, byte byte } -> ${ bit overflown, byte byte }`, respectively.
267
+ `${ bit overflown, byte byte } -> ${ bit overflown, byte byte }`,
268
+ respectively.
226
269
 
227
- Function `inc_o` is a polymorphic function of type patterns: `?X -> ${ byte byte, bit overflown }`
228
- with the following constraints:
270
+ Function `inc_o` is a polymorphic function of type patterns:
271
+ `?X -> ${ byte byte, bit overflown }` with the following constraints:
229
272
 
230
- - `?X` is a product type with at least two fields: `byte` and `overflown`, denoted by:
273
+ - `?X` is a product type with at least two fields: `byte` and
274
+ `overflown`, denoted by:
231
275
  > `?{ $byte byte, Z overflown, ...}`;
232
276
  - `?Z` is a union type with field `i` denoted by:
233
277
  > `?< V i, ...>`;
@@ -235,8 +279,9 @@ with the following constraints:
235
279
 
236
280
  We can write it as: `?{$byte byte, <(...) i, ...> overflown, ...}`.
237
281
 
238
- The target type `${ byte byte, bit overflown }` corresponds to pattern `?{ $byte byte, $bit overflown }`.
239
- Such a pattern is called _singleton pattern_, as only one type fits the pattern.
282
+ The target type `${ byte byte, bit overflown }` corresponds to pattern
283
+ `?{ $byte byte, $bit overflown }`. Such a pattern is called _singleton
284
+ pattern_, as only one type fits the pattern.
240
285
 
241
286
  #### Example 2
242
287
 
@@ -261,43 +306,61 @@ car = list? /cons .car; -- @...
261
306
 
262
307
  ## Universal Schema Registry
263
308
 
264
- Since the normalization process for types is fast and deterministic, we can build a universal
265
- schema registry that will store all invented types.
266
- The registry will be a key-value store, where the key is the hash of the normalized type,
267
- and the value is the normalized type itself.
309
+ Since the normalization process for types is fast and deterministic,
310
+ we can build a universal schema registry that will store all invented
311
+ types. The registry will be a key-value store, where the key is the hash
312
+ of the normalized type, and the value is the normalized type itself.
268
313
 
269
- All functions (non-polymorphic and polymorphic) will be renamed by the hash of their normalized
270
- definition and stored in a similar key-value store.
314
+ All functions (non-polymorphic and polymorphic) are named by the hash
315
+ of their normalized definition and stored in a similar key-value store.
271
316
 
272
- Non-polymorphic functions can be easily indexed by the hashes of their input and output types
273
- so that we can quickly find the function we need.
317
+ Non-polymorphic functions can be easily indexed by the hashes of their
318
+ input and output types so that we can quickly find the function we need.
274
319
  Indexing polymorphic functions seems complicated (TODO).
275
320
 
276
321
  ## Serialization and Compilation
277
322
 
278
- The language is designed to transform _codes_, i.e., serialized typed values, i.e., trees
279
- accepted by a tree automaton, into other _codes_.
280
- The transformation is done by partial functions as defined in the language.
281
- The non-recursive and tail-recursive (and even some non-tail-recursive) functions can
282
- be compiled into deterministic finite (pushdown) transducers.
323
+ The language is designed to transform _codes_, i.e., serialized typed
324
+ values, i.e., trees accepted by a tree automaton, into other _codes_.
325
+ The transformation is done by partial functions as defined in the
326
+ language. The non-recursive and tail-recursive (and even some
327
+ non-tail-recursive) functions can be compiled into deterministic finite
328
+ (pushdown) transducers.
329
+
330
+ Ahead-Of-Time (AOT) compilation produces four primary file formats (see
331
+ [`DOCS/FILE_FORMATS.md`](./FILE_FORMATS.md)):
332
+ - `.k`: Plain source text.
333
+ - `.ko`: AOT type-checked executable object container (`KOBJ\n` + JSON).
334
+ - `.klib`: AOT type-checked library container (JSON).
335
+ - `.kvm`: Polymorphic register-IR template (`layer: "KVM-P"`).
336
+
337
+ Downstream backends compile or specialize these into WebAssembly
338
+ (`.wasm`) and native binaries.
283
339
 
284
340
  ### AND-OR graphs for encoding and decoding codes
285
341
 
286
- This could be a generic and compact way to encode and decode typed values.
287
- The idea is to use _prefix codes_ (a class of languages such that no word is a prefix of
288
- another word) to encode the values.
342
+ This could be a generic and compact way to encode and decode typed
343
+ values. The idea is to use _prefix codes_ (a class of languages such
344
+ that no word is a prefix of another word) to encode the values.
289
345
 
290
346
  ### Rust data structures
291
347
 
292
- A type can be translated into a Rust data structure, and a function can be translated into a Rust function.
348
+ A type can be translated into a Rust data structure, and a function
349
+ can be translated into a Rust function.
293
350
 
294
351
  ## Linking with Other Languages
295
352
 
296
353
  ## Streams and Arrays (Map-Reduce)
297
354
 
298
- ## Bits
355
+ ## Bits and Absence of Built-in Numbers
356
+
357
+ In `k`, there are no primitive numbers or built-in integer literals.
358
+ Numerical identifiers such as `5` or `10` are nullary relations
359
+ (constants) mapping the unit `${}` to bit trees of type `${} -> $bits`.
360
+ Arithmetic relations operate on signed integers (`$int`), requiring
361
+ conversion via `5 int`.
299
362
 
300
- Extending `@string` into `@bits`.
363
+ Bit trees are defined algebraically:
301
364
 
302
365
  ```text
303
366
  $ bits = < {} _, bits 0, bits 1 >;
@@ -318,17 +381,28 @@ $ bits = < {} _, bits 0, bits 1 >;
318
381
 
319
382
  #### Eat `/`
320
383
 
321
- - `$@bits / @bits` --- division, e.g., `0b1011 / 0b10` = `0b11`, `"abc" / "a"` = `"bc"`
384
+ - `$@bits / @bits` --- division, e.g., `0b1011 / 0b10` = `0b11`,
385
+ `"abc" / "a"` = `"bc"`
322
386
 
323
387
  #### Prepend `\`
324
388
 
325
- - `$@bits \ @bits` --- multiplication, e.g., `0b11 \ 0b10` = `0b1011`, `"bc" \ "a"` = `"abc"`
389
+ - `$@bits \ @bits` --- multiplication, e.g., `0b11 \ 0b10` = `0b1011`,
390
+ `"bc" \ "a"` = `"abc"`
326
391
 
327
- Empty `bits` can be checked by
392
+ Empty `bits` can be checked by:
328
393
 
329
394
  ```k
330
-
331
395
  kind = $@bits
332
396
  < {/ 0b0 \ 0b0 starts_with_0}, {/ 0b1 \ 0b1 starts_with_1}, {() empty} >
333
397
  $< @bits starts_with_0, @bits starts_with_1, @bits empty >;
334
398
  ```
399
+
400
+ ## Further Reading
401
+
402
+ - [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to
403
+ .k, .ko, .klib, and .kvm formats
404
+ - [DOCS/DICTIONARY.md](./DICTIONARY.md) — concept names and terminology
405
+ - [DOCS/TYPE_DERIVATION.md](./TYPE_DERIVATION.md) — type derivation
406
+ algorithm
407
+ - [DOCS/OBJECT_FILE_AND_PATTERN.md](./OBJECT_FILE_AND_PATTERN.md) —
408
+ object format and pattern encoding
package/DOCS/CODECS.md CHANGED
@@ -1,34 +1,52 @@
1
1
  # Writing Codecs
2
2
 
3
- A codec is an ES module that translates between an external domain format and enveloped **k** `Value` objects `(P, v)`. The same module serves two entry points:
3
+ A codec is an ES module that translates between an external domain format
4
+ and enveloped **k** `Value` objects `(P, v)`. The same module serves two
5
+ entry points:
4
6
 
5
- - a standalone command-line filter supporting `--parse`, `--print`, and `--help`,
7
+ - a standalone command-line filter supporting `--parse`, `--print`,
8
+ and `--help`,
6
9
  - a REPL module loaded with `:codec load <file.mjs>`.
7
10
 
8
- The command-line entry point reads or writes the binary pattern+value stream.
9
- The REPL entry point provides interactive parsing (`:input <codec.mjs>`) and multi-codec output formatting.
11
+ The command-line entry point reads or writes the binary pattern+value
12
+ stream. The REPL entry point provides interactive parsing
13
+ (`:input <codec.mjs>`) and multi-codec output formatting.
10
14
 
11
15
  ## Architecture & Mental Model
12
16
 
13
- A codec maps between external text and enveloped *k* values:
17
+ In `k`, there are no primitive numbers, strings, or built-in numerical
18
+ literals. All data is represented as algebraic shapes (e.g. bits, units,
19
+ products, and tagged unions). A codec bridges external formats with
20
+ these structural *k* values:
14
21
 
15
- - `parse(text)` translates external text to an enveloped *k* value (a `Value` with an associated closed pattern envelope `(P, v)`).
16
- - `print(value)` translates an enveloped *k* value into formatted external text, throwing an `Error` if the value is not representable.
17
- - `doc` (optional string) documents the accepted syntax and semantic representation.
22
+ - `parse(text)` translates external text to an enveloped *k* value (a
23
+ `Value` with an associated closed pattern envelope `(P, v)`).
24
+ - `print(value)` translates an enveloped *k* value into formatted
25
+ external text, throwing an `Error` if the value is not representable.
26
+ - `doc` (optional string) documents the accepted syntax and semantic
27
+ representation.
18
28
 
19
- Codecs are identified directly by their file name. They do not export artificial identifiers like `name` or pattern registries like `patterns = [...]`.
29
+ Codecs are identified directly by their file name. They do not export
30
+ artificial identifiers like `name` or pattern registries like
31
+ `patterns = [...]`.
20
32
 
21
33
  In the REPL:
22
34
  - **Input**: The user explicitly chooses which codec to parse with:
23
- - Interactive mode: `:input <codec.mjs>` sets the prompt to `<codec.mjs>> `, and the next entered line is parsed with that codec.
24
- - One-line mode: `:input <codec.mjs> <text>` immediately parses `<text>` using that codec.
25
- - **Output**: Any evaluated *k* value is displayed in its standard *k* envelope representation, followed by formatting from **all loaded codecs** that can represent it:
35
+ - Interactive mode: `:input <codec.mjs>` sets the prompt to
36
+ `<codec.mjs>> `, and the next entered line is parsed with that codec.
37
+ - One-line mode: `:input <codec.mjs> <text>` immediately parses
38
+ `<text>` using that codec.
39
+ - **Output**: Any evaluated *k* value is displayed in its standard *k*
40
+ envelope representation, followed by formatting from **all loaded
41
+ codecs** that can represent it:
26
42
  ```text
27
- > 42 int
43
+ > :input int.mjs 42
28
44
  {}|_|0|1|0|1|0|1|+ ?<{} _, ...>
29
45
  int.mjs: 42
30
46
  ```
31
- Every loaded codec attempts `codec.print(value)`. If it succeeds, the REPL prints `<codec.mjs>: <formatted>`. If it throws, the codec is silently ignored for that value.
47
+ Every loaded codec attempts `codec.print(value)`. If it succeeds, the
48
+ REPL prints `<codec.mjs>: <formatted>`. If it throws, the codec is
49
+ silently ignored for that value.
32
50
 
33
51
  ## Module Interface
34
52
 
@@ -38,7 +56,8 @@ A codec module exports:
38
56
  export const doc = "Description of syntax and usage.";
39
57
 
40
58
  export function parse(text) {
41
- // parse text into an enveloped k Value (P, v) using withPattern(value, pattern)
59
+ // parse text into an enveloped k Value (P, v) using
60
+ // withPattern(value, pattern)
42
61
  }
43
62
 
44
63
  export function print(value) {
@@ -47,24 +66,31 @@ export function print(value) {
47
66
  }
48
67
  ```
49
68
 
50
- `parse` should throw an `Error` for invalid external text. `print` should throw when the value is not representable by the external format. REPL output suppresses `print` errors so loaded codecs only output when they recognize the value.
69
+ `parse` should throw an `Error` for invalid external text. `print` should
70
+ throw when the value is not representable by the external format. REPL
71
+ output suppresses `print` errors so loaded codecs only output when they
72
+ recognize the value.
51
73
 
52
74
  ## The Codec SDK & Runner
53
75
 
54
- To eliminate CLI stream boilerplate, codecs import from `./runtime/codec-sdk.mjs` and invoke `runCodecCLI`:
76
+ To eliminate CLI stream boilerplate, codecs import from
77
+ `./runtime/codec-sdk.mjs` and invoke `runCodecCLI`:
55
78
 
56
79
  ```js
57
80
  import { Value, isVariant, withPattern, runCodecCLI } from "./runtime/codec-sdk.mjs";
58
81
  ```
59
82
 
60
- `runCodecCLI(import.meta.url, { parse, print, doc })` automatically handles:
83
+ `runCodecCLI(import.meta.url, { parse, print, doc })` automatically
84
+ handles:
61
85
  1. Entrypoint detection (no-op when loaded as a module or in browsers).
62
86
  2. `-h` and `--help` CLI flag handling.
63
87
  3. `--parse` and `--print` stream piping and wire encoding/decoding.
64
88
 
65
89
  ## Minimal Type-Specific Codec Example
66
90
 
67
- This codec accepts `yes` and `no` for the type `<{} false, {} true>`. Patterns are derived by `k` from filter expressions via `patternFromFilter`:
91
+ This codec accepts `yes` and `no` for the type `<{} false, {} true>`.
92
+ Patterns are derived by `k` from filter expressions via
93
+ `patternFromFilter`:
68
94
 
69
95
  ```js
70
96
  #!/usr/bin/env node
@@ -79,8 +105,12 @@ Boolean codec: accepts 'yes' and 'no'.
79
105
 
80
106
  export function parse(text) {
81
107
  const word = text.trim();
82
- if (word === "yes") return withPattern(Value.variant("true", Value.product({})), BOOL_PATTERN);
83
- if (word === "no") return withPattern(Value.variant("false", Value.product({})), BOOL_PATTERN);
108
+ if (word === "yes") {
109
+ return withPattern(Value.variant("true", Value.product({})), BOOL_PATTERN);
110
+ }
111
+ if (word === "no") {
112
+ return withPattern(Value.variant("false", Value.product({})), BOOL_PATTERN);
113
+ }
84
114
  throw new Error("expected yes or no");
85
115
  }
86
116
 
@@ -116,10 +146,12 @@ printf 'yes\n' | ./codecs/yesno.mjs --parse | ./codecs/yesno.mjs --print
116
146
  For CLI codecs, test both directions in a pipeline:
117
147
 
118
148
  ```sh
119
- printf 'yes\n' | node codecs/yesno.mjs --parse | node codecs/yesno.mjs --print
149
+ printf 'yes\n' | node codecs/yesno.mjs --parse | \
150
+ node codecs/yesno.mjs --print
120
151
  ```
121
152
 
122
- For REPL codecs, add a focused case to [`../tests/test-repl.mjs`](../tests/test-repl.mjs):
153
+ For REPL codecs, add a focused case to
154
+ [`../tests/test-repl.mjs`](../tests/test-repl.mjs):
123
155
 
124
156
  ```js
125
157
  const state = createState();
@@ -142,16 +174,29 @@ npm test
142
174
  ## Web REPL (VFS) Bundling
143
175
 
144
176
  When `repl.html` is generated (`npm run build:repl-html`):
145
- - Codecs (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`) are bundled into 100% self-contained ES modules with zero imports.
146
- - `runCodecCLI` is shimmed as a no-op, and `Value` helpers are directly inlined.
147
- - In the browser, the VFS serves these self-contained modules directly to native dynamic `import(blobUrl)`. No runtime helper files (`codecs/runtime/*`) need to be included in the VFS or exposed as globals.
177
+ - Codecs (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`) are
178
+ bundled into 100% self-contained ES modules with zero imports.
179
+ - `runCodecCLI` is shimmed as a no-op, and `Value` helpers are directly
180
+ inlined.
181
+ - In the browser, the VFS serves these self-contained modules directly to
182
+ native dynamic `import(blobUrl)`. No runtime helper files
183
+ (`codecs/runtime/*`) need to be included in the VFS or exposed as
184
+ globals.
148
185
 
149
186
  ## Checklist
150
187
 
151
- - Return an enveloped `Value` from `parse` using `withPattern(val, pattern)`.
188
+ - Return an enveloped `Value` from `parse` using
189
+ `withPattern(val, pattern)`.
152
190
  - Keep `parse` and `print` deterministic and free of side effects.
153
- - Validate shapes in `print` and throw clear errors when not representable.
154
- - Use `runCodecCLI(import.meta.url, { parse, print, doc })` for CLI execution.
191
+ - Validate shapes in `print` and throw clear errors when not
192
+ representable.
193
+ - Use `runCodecCLI(import.meta.url, { parse, print, doc })` for CLI
194
+ execution.
155
195
  - No artificial `name` or `patterns` exports required.
156
196
  - Add `-h` and `--help` support via `runCodecCLI`.
157
197
 
198
+ - [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) —
199
+ guide to .k, .ko, .klib, and .kvm formats
200
+ - [DOCS/CODEC_ARCHITECTURE.md](./CODEC_ARCHITECTURE.md) —
201
+ codec architectural design
202
+ - [DOCS/REPL.md](./REPL.md) — interactive interpreter and codec commands