@fraczak/k 6.10.1 → 6.10.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -0
- package/DOCS/BIG_PICTURE.md +174 -100
- package/DOCS/CODECS.md +74 -29
- package/DOCS/CODEC_ARCHITECTURE.md +87 -31
- package/DOCS/CONVERGENCE.md +42 -11
- package/DOCS/DICTIONARY.md +69 -2
- package/DOCS/FILE_FORMATS.md +385 -0
- package/DOCS/KIR_V1.md +53 -42
- package/DOCS/KVM_EXECUTION_MODEL.md +189 -147
- package/DOCS/OBJECT_FILE_AND_PATTERN.md +35 -17
- package/DOCS/PATTERNS.md +10 -6
- package/DOCS/REPL.md +100 -60
- package/DOCS/TEXTUAL_VALUES.md +24 -19
- package/DOCS/TYPE_DERIVATION.md +68 -22
- package/README.md +92 -58
- package/codecs/README.md +78 -67
- package/codecs/runtime/show-value.mjs +68 -16
- package/objects/README.md +57 -37
- package/package.json +1 -1
- package/tests/README.md +28 -17
- package/tests/test-repl-html.mjs +19 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,45 @@ All notable changes to this project will be documented here.
|
|
|
5
5
|
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
6
6
|
This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [6.10.3] — 2026-10-04 — Unified File Formats Specification and Documentation Modernization
|
|
9
|
+
|
|
10
|
+
### Documentation & Core Architecture
|
|
11
|
+
|
|
12
|
+
- **Unified File Formats Specification (`DOCS/FILE_FORMATS.md`)**:
|
|
13
|
+
- Added comprehensive specification for all primary file formats: `.k`
|
|
14
|
+
(source), `.ko` (executable object), `.klib` (library object), and
|
|
15
|
+
`.kvm` (polymorphic register-IR template), plus target artifacts
|
|
16
|
+
(`.wasm`, native binary).
|
|
17
|
+
- Added Mermaid transformation flowchart mapping compilation, linking,
|
|
18
|
+
specialization, and execution workflows across CLI tools.
|
|
19
|
+
- **AOT Type Checking & Canonical Content-Addressing**:
|
|
20
|
+
- Formally specified that AOT type checking performs relation expansion,
|
|
21
|
+
structural type derivation, and constraint convergence in advance.
|
|
22
|
+
- Clarified canonical content-addressed hashes (`@hash`) vs. local aliases,
|
|
23
|
+
confirming there is no global symbol namespace, and documented explicit
|
|
24
|
+
`--export` semantics for `--lib`.
|
|
25
|
+
- **Relations as Values & Absence of Built-in Numbers**:
|
|
26
|
+
- Documented that numerical literals do not exist in `k`; names like `5`
|
|
27
|
+
or `10` are nullary relations mapping `${} -> $bits`. Arithmetic relations
|
|
28
|
+
operate on signed integer variants (`$int`), converting via `5 int`.
|
|
29
|
+
- Clarified stream transformers (open domain) vs. closed constant relations
|
|
30
|
+
(unit input domain `${}`).
|
|
31
|
+
- **Documentation Reformatting**:
|
|
32
|
+
- Systematically updated and wrapped prose across `README.md` and all
|
|
33
|
+
documents in `DOCS/`, `codecs/`, `objects/`, and `tests/` to <= 74
|
|
34
|
+
characters per line.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## [6.10.2] — 2026-10-03 — Stack-Safe Value Serializer for REPL and Web REPL
|
|
39
|
+
|
|
40
|
+
### REPL & Codecs
|
|
41
|
+
|
|
42
|
+
- **Stack-Safe `valueToK()` Serializer**: Rewrote `valueToK()` in `codecs/runtime/show-value.mjs` using an explicit iterative worklist stack instead of JavaScript call-stack recursion. This prevents `RangeError: Maximum call stack size exceeded` crashes when displaying or previewing large, deep data structures (such as long lists) in the terminal REPL and Web REPL (`repl.html`).
|
|
43
|
+
- **Web REPL Tests**: Added end-to-end browser tests in `tests/test-repl-html.mjs` verifying list sorting (`poly.k`) and 10,000-element tail-loop compaction in `repl.html` under the `wasm` engine.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
8
47
|
## [6.10.1] — 2026-10-02 — Fix Compaction Mark Reset and False Tail Calls in LLVM & WASM Backends
|
|
9
48
|
|
|
10
49
|
### LLVM & WebAssembly Backends
|
package/DOCS/BIG_PICTURE.md
CHANGED
|
@@ -1,28 +1,32 @@
|
|
|
1
1
|
# Big Picture
|
|
2
2
|
|
|
3
|
-
`Kernel-code` is an experiment aiming to define a "perfect programming
|
|
3
|
+
`Kernel-code` is an experiment aiming to define a "perfect programming
|
|
4
|
+
language".
|
|
4
5
|
|
|
5
6
|
## Basics
|
|
6
7
|
|
|
7
|
-
The language describes "partial first-order functions" on "printable"
|
|
8
|
-
The printable values can be seen as rooted, edge-labeled trees,
|
|
9
|
-
XML or a JSON document. The values are always
|
|
10
|
-
runtime, a value may carry a pattern
|
|
11
|
-
with that tree.
|
|
8
|
+
The language describes "partial first-order functions" on "printable"
|
|
9
|
+
values. The printable values can be seen as rooted, edge-labeled trees,
|
|
10
|
+
intuitively similar to an XML or a JSON document. The values are always
|
|
11
|
+
typed; in the current codec-aware runtime, a value may carry a pattern
|
|
12
|
+
describing the family of types compatible with that tree.
|
|
12
13
|
|
|
13
14
|
### Types (codes)
|
|
14
15
|
|
|
15
|
-
Formally, a _type_ is a finite tree automaton. The type defines a set of
|
|
16
|
-
which corresponds to the set of trees accepted by the
|
|
17
|
-
Therefore, every type admits a unique normal form in terms of
|
|
16
|
+
Formally, a _type_ is a finite tree automaton. The type defines a set of
|
|
17
|
+
values, which corresponds to the set of trees accepted by the
|
|
18
|
+
automaton. Therefore, every type admits a unique normal form in terms of
|
|
19
|
+
its minimal deterministic tree automaton.
|
|
18
20
|
|
|
19
21
|
Types are defined by two constructs:
|
|
20
22
|
|
|
21
23
|
1. Product
|
|
22
24
|
2. Union (also called tagged or disjoint union)
|
|
23
25
|
|
|
24
|
-
> Notation: We use native k-like notation as canonical — products
|
|
25
|
-
>
|
|
26
|
+
> Notation: We use native k-like notation as canonical — products
|
|
27
|
+
> `{ T l, ... }` and unions `< T t, ... >`. For convenience, JSON-like
|
|
28
|
+
> forms `{ l: T, ... }` and `< t: T, ... >` are also supported and
|
|
29
|
+
> equivalent in meaning.
|
|
26
30
|
|
|
27
31
|
```bnf
|
|
28
32
|
type ::= name | '{' type_label_list '}' | '<' type_label_list '>'
|
|
@@ -31,18 +35,23 @@ name ::= IDENTIFIER
|
|
|
31
35
|
label ::= STRING
|
|
32
36
|
```
|
|
33
37
|
|
|
34
|
-
Given a finite (possibly empty) set of types `t1, t2, ..., tn` and a
|
|
38
|
+
Given a finite (possibly empty) set of types `t1, t2, ..., tn` and a
|
|
39
|
+
set of pairwise different tags (labels) `l1, l2, ..., ln`,
|
|
35
40
|
|
|
36
|
-
- by `{ t1 l1, t2 l2, ..., tn ln }`, we define a _product type_ with
|
|
37
|
-
_projection_ functions: `.l1`, `.l2`, `...`, `.ln`.
|
|
38
|
-
Each projection function maps a value of type
|
|
39
|
-
into a value of type `t1`, `t2`,
|
|
40
|
-
|
|
41
|
-
|
|
41
|
+
- by `{ t1 l1, t2 l2, ..., tn ln }`, we define a _product type_ with
|
|
42
|
+
exactly `n` _projection_ functions: `.l1`, `.l2`, `...`, `.ln`.
|
|
43
|
+
Each projection function maps a value of type
|
|
44
|
+
`{t1 l1, t2 l2, ..., tn ln}` into a value of type `t1`, `t2`,
|
|
45
|
+
`...`, `tn`, respectively.
|
|
46
|
+
- by `< t1 l1, t2 l2, ..., tn ln >`, we define a _union type_ with
|
|
47
|
+
exactly `n` projection functions `/l1`, `/l2`, `...`, `/ln`,
|
|
48
|
+
mapping a value of type `<t1 l1, t2 l2, ..., tn ln>` into a value
|
|
49
|
+
of type `t1`, `t2`, `...`, `tn`, respectively.
|
|
42
50
|
|
|
43
|
-
Intuitively, the projection functions on a product type are all total
|
|
44
|
-
|
|
45
|
-
|
|
51
|
+
Intuitively, the projection functions on a product type are all total
|
|
52
|
+
(meaning, they are defined for every value of the input product type),
|
|
53
|
+
whereas in the case of a union type, exactly one projection function
|
|
54
|
+
is defined for a given value of the union type.
|
|
46
55
|
|
|
47
56
|
Some properties of these types:
|
|
48
57
|
|
|
@@ -51,26 +60,32 @@ Some properties of these types:
|
|
|
51
60
|
|
|
52
61
|
#### Variant (union) value literal convention
|
|
53
62
|
|
|
54
|
-
- Variant values are written using "|" (pipe) followed by the variant
|
|
63
|
+
- Variant values are written using "|" (pipe) followed by the variant
|
|
64
|
+
tag.
|
|
55
65
|
- Unit variant `` `{}` `` | tag (e.g., `{} | nil`, `{} | zero`)
|
|
56
66
|
- Variant with payload `v` at label `tag`: ` v | tag `
|
|
57
67
|
- Example (list): with `$list = < {} nil, {X car, list cdr} cons >`:
|
|
58
68
|
- Empty list []: `{}|nil`
|
|
59
69
|
- Singleton `[v]`: `{ v car, {}|nil cdr } | cons`
|
|
60
|
-
- Angle brackets are for type definitions and merge expressions; use
|
|
70
|
+
- Angle brackets are for type definitions and merge expressions; use
|
|
71
|
+
"`| tag`" for a variant construction.
|
|
61
72
|
|
|
62
73
|
#### Equivalence of types (bisimilarity)
|
|
63
74
|
|
|
64
|
-
Two (possibly recursive) codes are equivalent iff they are bisimilar
|
|
75
|
+
Two (possibly recursive) codes are equivalent iff they are bisimilar
|
|
76
|
+
over their definition graphs. Concretely, there exists a relation B
|
|
77
|
+
such that `(t1, t2) ∈ B` if and only if:
|
|
65
78
|
- t1 and t2 have exactly the same set of labels/tags;
|
|
66
79
|
- t1 and t2 are simultaneously products or simultaneously unions;
|
|
67
|
-
- for each label/tag `ℓ` in the set, the subcodes under `ℓ` are again
|
|
80
|
+
- for each label/tag `ℓ` in the set, the subcodes under `ℓ` are again
|
|
81
|
+
related by B.
|
|
68
82
|
|
|
69
83
|
### Filters (patterns)
|
|
70
84
|
|
|
71
|
-
A _filter_ is a way of representing a _pattern_, i.e., a set of types
|
|
72
|
-
For example, if I wanted to refer to all product
|
|
73
|
-
|
|
85
|
+
A _filter_ is a way of representing a _pattern_, i.e., a set of types
|
|
86
|
+
sharing some structure. For example, if I wanted to refer to all product
|
|
87
|
+
types that have a field `f` of type `T`, I would write:
|
|
88
|
+
`?{ T f, ... }`. In general, a filter is defined by:
|
|
74
89
|
|
|
75
90
|
```bnf
|
|
76
91
|
filter ::= name | '$' type_expr | '{' filter_label_list '}'
|
|
@@ -92,29 +107,36 @@ Examples:
|
|
|
92
107
|
|
|
93
108
|
### Partial functions
|
|
94
109
|
|
|
95
|
-
A _partial function_ is a function that is not defined for every value
|
|
110
|
+
A _partial function_ is a function that is not defined for every value
|
|
111
|
+
of its domain.
|
|
96
112
|
|
|
97
113
|
There are three ways of combining partial functions:
|
|
98
114
|
|
|
99
|
-
1. __Composition__: we write a composition of `f` and `g` as `(f
|
|
100
|
-
meaning that if `f` is defined on value `x` producing `y`,
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
115
|
+
1. __Composition__: we write a composition of `f` and `g` as `(f g)`,
|
|
116
|
+
meaning that if `f` is defined on value `x` producing `y`, and `g`
|
|
117
|
+
is defined on value `y` producing `z`, then `(f g)` is defined on
|
|
118
|
+
value `x` producing `z`.
|
|
119
|
+
2. __Union__: we write `< f1, f2 >` for a union of two functions `f1`
|
|
120
|
+
and `f2`, meaning that if `f1` is defined on value `x` producing
|
|
121
|
+
`y`, then `< f1, f2 >` is defined on value `x` producing `y`.
|
|
122
|
+
Otherwise, i.e., when `f1` is not defined for `x`, `< f1, f2 >`
|
|
123
|
+
acts as `f2`.
|
|
124
|
+
3. __Product__: we write `{f1 lab1, f2 lab2 }` for a product of two
|
|
125
|
+
functions `f1` and `f2`, meaning that if `f1` and `f2` are both
|
|
126
|
+
defined on value `x` producing `y1` and `y2`, respectively, then
|
|
127
|
+
`{f1 lab1, f2 lab2 }` is defined on value `x` producing
|
|
128
|
+
`{y1 lab1, y2 lab2}`. Otherwise, `{f1 lab1, f2 lab2 }` is not
|
|
129
|
+
defined for `x`.
|
|
130
|
+
|
|
131
|
+
Actually, composition, union, and product combine any number of
|
|
132
|
+
partial functions, not only two. For example, empty composition,
|
|
133
|
+
`()`, defines the identity function.
|
|
113
134
|
|
|
114
135
|
### Program
|
|
115
136
|
|
|
116
|
-
A `kernel-core` program is a set of definitions of types and partial
|
|
117
|
-
which is the partial function
|
|
137
|
+
A `kernel-core` program is a set of definitions of types and partial
|
|
138
|
+
functions, followed by an _expression_, which is the partial function
|
|
139
|
+
the program defines.
|
|
118
140
|
|
|
119
141
|
The language syntax is defined by:
|
|
120
142
|
|
|
@@ -132,47 +154,65 @@ expr_label_list ::= /* empty */
|
|
|
132
154
|
| (expression label_name ',')* expression label
|
|
133
155
|
```
|
|
134
156
|
|
|
135
|
-
We assume that in a `kernel-core` program the names (for types,
|
|
157
|
+
We assume that in a `kernel-core` program the names (for types,
|
|
158
|
+
functions, and labels) are identifiers (strings) such that:
|
|
136
159
|
|
|
137
160
|
1. All defined type names are distinct.
|
|
138
161
|
2. All defined function names are distinct.
|
|
139
|
-
3. All label names within a type definition or a `product` expression
|
|
162
|
+
3. All label names within a type definition or a `product` expression
|
|
163
|
+
are locally distinct.
|
|
140
164
|
|
|
141
|
-
Filters and type expressions act as a type annotation, i.e., an
|
|
142
|
-
function defined only for the values of the corresponding
|
|
165
|
+
Filters and type expressions act as a type annotation, i.e., an
|
|
166
|
+
identity function defined only for the values of the corresponding
|
|
167
|
+
types.
|
|
143
168
|
|
|
144
169
|
That's it.
|
|
145
170
|
|
|
146
|
-
No `builtin` types, no `if` statement, no `loop`, no `throw`, no
|
|
147
|
-
Just _types_, _filters_,
|
|
171
|
+
No `builtin` types, no `if` statement, no `loop`, no `throw`, no
|
|
172
|
+
_closure_, no _annotations_, and no macros. Just _types_, _filters_,
|
|
173
|
+
and _functions_.
|
|
148
174
|
|
|
149
|
-
### Normalization
|
|
175
|
+
### Normalization and Content-Addressing
|
|
150
176
|
|
|
151
|
-
All type names are replaced by the hash of their
|
|
177
|
+
All type names are replaced by the canonical hash (`@hash`) of their
|
|
178
|
+
minimal deterministic tree automaton representation.
|
|
152
179
|
|
|
153
|
-
|
|
154
|
-
|
|
180
|
+
Every relation is identified canonically by the content-addressed hash
|
|
181
|
+
of its normalized definition AST.
|
|
155
182
|
|
|
156
|
-
|
|
157
|
-
|
|
183
|
+
There is no global symbol namespace. Human-readable names (such as
|
|
184
|
+
`plus`, `int`, or `5`) are local aliases recorded in object metadata
|
|
185
|
+
(`relAlias`, `typeAliases`). When libraries are loaded with `--lib`,
|
|
186
|
+
definitions are identified by their canonical `@hash`, but local aliases
|
|
187
|
+
are not implicitly exported into the importing scope unless explicitly
|
|
188
|
+
mapped via `--export`.
|
|
189
|
+
|
|
190
|
+
This approach completely solves the problem of modules, name collisions,
|
|
191
|
+
and unversioned imports, opening the door to a universal registry of
|
|
192
|
+
types (schemas) and relations.
|
|
158
193
|
|
|
159
194
|
## Typing and Polymorphism (codes and patterns)
|
|
160
195
|
|
|
161
|
-
The abstract syntax tree of a program consists of a dictionary of
|
|
162
|
-
a dictionary of _function definitions_, and the
|
|
196
|
+
The abstract syntax tree of a program consists of a dictionary of
|
|
197
|
+
_type definitions_, a dictionary of _function definitions_, and the
|
|
198
|
+
final _main expression_.
|
|
163
199
|
|
|
164
|
-
In principle, the initial form of the Abstract Syntax Tree, _raw_ AST,
|
|
165
|
-
the program on an input.
|
|
200
|
+
In principle, the initial form of the Abstract Syntax Tree, _raw_ AST,
|
|
201
|
+
is enough to evaluate the program on an input.
|
|
166
202
|
|
|
167
|
-
Our objective, however, is to compute canonical representations for
|
|
168
|
-
representations for functions so they can be
|
|
203
|
+
Our objective, however, is to compute canonical representations for
|
|
204
|
+
types and normalized representations for functions so they can be
|
|
205
|
+
reused and compiled ahead-of-time.
|
|
169
206
|
|
|
170
207
|
The normalization steps are:
|
|
171
208
|
|
|
172
|
-
1. Build the _type graph_ from all types used by the program, i.e.,
|
|
173
|
-
as well as type expressions used in
|
|
174
|
-
|
|
175
|
-
|
|
209
|
+
1. Build the _type graph_ from all types used by the program, i.e.,
|
|
210
|
+
used in _type definitions_ as well as type expressions used in
|
|
211
|
+
_function definitions_ and the _main expression_.
|
|
212
|
+
2. Annotate the expressions with filters; every node of the AST is
|
|
213
|
+
annotated with a pair of filters.
|
|
214
|
+
3. Turn the _singleton_ filters into types and add them to the _type
|
|
215
|
+
graph_. Go to (2) unless no change.
|
|
176
216
|
|
|
177
217
|
### Examples
|
|
178
218
|
|
|
@@ -213,21 +253,25 @@ inc3
|
|
|
213
253
|
In the above program, we define type `bit` as union of two unit types,
|
|
214
254
|
and type `byte` as product, for simplicity, of four `bit`s.
|
|
215
255
|
|
|
216
|
-
Functions `bit0`, `bit1`, and `zero` are "constant polymorphic
|
|
256
|
+
Functions `bit0`, `bit1`, and `zero` are "constant polymorphic
|
|
257
|
+
functions", meaning:
|
|
217
258
|
|
|
218
259
|
- constant: if defined, they always return exactly the same value;
|
|
219
|
-
- polymorphic: they are defined for more than one pair of input and
|
|
220
|
-
`bit0` and `bit1` are of type `?X -> $bit`,
|
|
221
|
-
|
|
260
|
+
- polymorphic: they are defined for more than one pair of input and
|
|
261
|
+
output types: functions `bit0` and `bit1` are of type `?X -> $bit`,
|
|
262
|
+
and `zero` is of type `?X -> $byte`, where `?X` denotes an
|
|
263
|
+
unconstrained type pattern, also denoted as `?(...)`.
|
|
222
264
|
|
|
223
265
|
Functions `inc` and `inc3` are (non-polymorphic) functions of types
|
|
224
266
|
`$byte -> ${ bit overflown, byte byte }` and
|
|
225
|
-
`${ bit overflown, byte byte } -> ${ bit overflown, byte byte }`,
|
|
267
|
+
`${ bit overflown, byte byte } -> ${ bit overflown, byte byte }`,
|
|
268
|
+
respectively.
|
|
226
269
|
|
|
227
|
-
Function `inc_o` is a polymorphic function of type patterns:
|
|
228
|
-
with the following constraints:
|
|
270
|
+
Function `inc_o` is a polymorphic function of type patterns:
|
|
271
|
+
`?X -> ${ byte byte, bit overflown }` with the following constraints:
|
|
229
272
|
|
|
230
|
-
- `?X` is a product type with at least two fields: `byte` and
|
|
273
|
+
- `?X` is a product type with at least two fields: `byte` and
|
|
274
|
+
`overflown`, denoted by:
|
|
231
275
|
> `?{ $byte byte, Z overflown, ...}`;
|
|
232
276
|
- `?Z` is a union type with field `i` denoted by:
|
|
233
277
|
> `?< V i, ...>`;
|
|
@@ -235,8 +279,9 @@ with the following constraints:
|
|
|
235
279
|
|
|
236
280
|
We can write it as: `?{$byte byte, <(...) i, ...> overflown, ...}`.
|
|
237
281
|
|
|
238
|
-
The target type `${ byte byte, bit overflown }` corresponds to pattern
|
|
239
|
-
Such a pattern is called _singleton
|
|
282
|
+
The target type `${ byte byte, bit overflown }` corresponds to pattern
|
|
283
|
+
`?{ $byte byte, $bit overflown }`. Such a pattern is called _singleton
|
|
284
|
+
pattern_, as only one type fits the pattern.
|
|
240
285
|
|
|
241
286
|
#### Example 2
|
|
242
287
|
|
|
@@ -261,43 +306,61 @@ car = list? /cons .car; -- @...
|
|
|
261
306
|
|
|
262
307
|
## Universal Schema Registry
|
|
263
308
|
|
|
264
|
-
Since the normalization process for types is fast and deterministic,
|
|
265
|
-
schema registry that will store all invented
|
|
266
|
-
The registry will be a key-value store, where the key is the hash
|
|
267
|
-
and the value is the normalized type itself.
|
|
309
|
+
Since the normalization process for types is fast and deterministic,
|
|
310
|
+
we can build a universal schema registry that will store all invented
|
|
311
|
+
types. The registry will be a key-value store, where the key is the hash
|
|
312
|
+
of the normalized type, and the value is the normalized type itself.
|
|
268
313
|
|
|
269
|
-
All functions (non-polymorphic and polymorphic)
|
|
270
|
-
definition and stored in a similar key-value store.
|
|
314
|
+
All functions (non-polymorphic and polymorphic) are named by the hash
|
|
315
|
+
of their normalized definition and stored in a similar key-value store.
|
|
271
316
|
|
|
272
|
-
Non-polymorphic functions can be easily indexed by the hashes of their
|
|
273
|
-
so that we can quickly find the function we need.
|
|
317
|
+
Non-polymorphic functions can be easily indexed by the hashes of their
|
|
318
|
+
input and output types so that we can quickly find the function we need.
|
|
274
319
|
Indexing polymorphic functions seems complicated (TODO).
|
|
275
320
|
|
|
276
321
|
## Serialization and Compilation
|
|
277
322
|
|
|
278
|
-
The language is designed to transform _codes_, i.e., serialized typed
|
|
279
|
-
accepted by a tree automaton, into other _codes_.
|
|
280
|
-
The transformation is done by partial functions as defined in the
|
|
281
|
-
The non-recursive and tail-recursive (and even some
|
|
282
|
-
be compiled into deterministic finite
|
|
323
|
+
The language is designed to transform _codes_, i.e., serialized typed
|
|
324
|
+
values, i.e., trees accepted by a tree automaton, into other _codes_.
|
|
325
|
+
The transformation is done by partial functions as defined in the
|
|
326
|
+
language. The non-recursive and tail-recursive (and even some
|
|
327
|
+
non-tail-recursive) functions can be compiled into deterministic finite
|
|
328
|
+
(pushdown) transducers.
|
|
329
|
+
|
|
330
|
+
Ahead-Of-Time (AOT) compilation produces four primary file formats (see
|
|
331
|
+
[`DOCS/FILE_FORMATS.md`](./FILE_FORMATS.md)):
|
|
332
|
+
- `.k`: Plain source text.
|
|
333
|
+
- `.ko`: AOT type-checked executable object container (`KOBJ\n` + JSON).
|
|
334
|
+
- `.klib`: AOT type-checked library container (JSON).
|
|
335
|
+
- `.kvm`: Polymorphic register-IR template (`layer: "KVM-P"`).
|
|
336
|
+
|
|
337
|
+
Downstream backends compile or specialize these into WebAssembly
|
|
338
|
+
(`.wasm`) and native binaries.
|
|
283
339
|
|
|
284
340
|
### AND-OR graphs for encoding and decoding codes
|
|
285
341
|
|
|
286
|
-
This could be a generic and compact way to encode and decode typed
|
|
287
|
-
The idea is to use _prefix codes_ (a class of languages such
|
|
288
|
-
another word) to encode the values.
|
|
342
|
+
This could be a generic and compact way to encode and decode typed
|
|
343
|
+
values. The idea is to use _prefix codes_ (a class of languages such
|
|
344
|
+
that no word is a prefix of another word) to encode the values.
|
|
289
345
|
|
|
290
346
|
### Rust data structures
|
|
291
347
|
|
|
292
|
-
A type can be translated into a Rust data structure, and a function
|
|
348
|
+
A type can be translated into a Rust data structure, and a function
|
|
349
|
+
can be translated into a Rust function.
|
|
293
350
|
|
|
294
351
|
## Linking with Other Languages
|
|
295
352
|
|
|
296
353
|
## Streams and Arrays (Map-Reduce)
|
|
297
354
|
|
|
298
|
-
## Bits
|
|
355
|
+
## Bits and Absence of Built-in Numbers
|
|
356
|
+
|
|
357
|
+
In `k`, there are no primitive numbers or built-in integer literals.
|
|
358
|
+
Numerical identifiers such as `5` or `10` are nullary relations
|
|
359
|
+
(constants) mapping the unit `${}` to bit trees of type `${} -> $bits`.
|
|
360
|
+
Arithmetic relations operate on signed integers (`$int`), requiring
|
|
361
|
+
conversion via `5 int`.
|
|
299
362
|
|
|
300
|
-
|
|
363
|
+
Bit trees are defined algebraically:
|
|
301
364
|
|
|
302
365
|
```text
|
|
303
366
|
$ bits = < {} _, bits 0, bits 1 >;
|
|
@@ -318,17 +381,28 @@ $ bits = < {} _, bits 0, bits 1 >;
|
|
|
318
381
|
|
|
319
382
|
#### Eat `/`
|
|
320
383
|
|
|
321
|
-
- `$@bits / @bits` --- division, e.g., `0b1011 / 0b10` = `0b11`,
|
|
384
|
+
- `$@bits / @bits` --- division, e.g., `0b1011 / 0b10` = `0b11`,
|
|
385
|
+
`"abc" / "a"` = `"bc"`
|
|
322
386
|
|
|
323
387
|
#### Prepend `\`
|
|
324
388
|
|
|
325
|
-
- `$@bits \ @bits` --- multiplication, e.g., `0b11 \ 0b10` = `0b1011`,
|
|
389
|
+
- `$@bits \ @bits` --- multiplication, e.g., `0b11 \ 0b10` = `0b1011`,
|
|
390
|
+
`"bc" \ "a"` = `"abc"`
|
|
326
391
|
|
|
327
|
-
Empty `bits` can be checked by
|
|
392
|
+
Empty `bits` can be checked by:
|
|
328
393
|
|
|
329
394
|
```k
|
|
330
|
-
|
|
331
395
|
kind = $@bits
|
|
332
396
|
< {/ 0b0 \ 0b0 starts_with_0}, {/ 0b1 \ 0b1 starts_with_1}, {() empty} >
|
|
333
397
|
$< @bits starts_with_0, @bits starts_with_1, @bits empty >;
|
|
334
398
|
```
|
|
399
|
+
|
|
400
|
+
## Further Reading
|
|
401
|
+
|
|
402
|
+
- [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to
|
|
403
|
+
.k, .ko, .klib, and .kvm formats
|
|
404
|
+
- [DOCS/DICTIONARY.md](./DICTIONARY.md) — concept names and terminology
|
|
405
|
+
- [DOCS/TYPE_DERIVATION.md](./TYPE_DERIVATION.md) — type derivation
|
|
406
|
+
algorithm
|
|
407
|
+
- [DOCS/OBJECT_FILE_AND_PATTERN.md](./OBJECT_FILE_AND_PATTERN.md) —
|
|
408
|
+
object format and pattern encoding
|
package/DOCS/CODECS.md
CHANGED
|
@@ -1,34 +1,52 @@
|
|
|
1
1
|
# Writing Codecs
|
|
2
2
|
|
|
3
|
-
A codec is an ES module that translates between an external domain format
|
|
3
|
+
A codec is an ES module that translates between an external domain format
|
|
4
|
+
and enveloped **k** `Value` objects `(P, v)`. The same module serves two
|
|
5
|
+
entry points:
|
|
4
6
|
|
|
5
|
-
- a standalone command-line filter supporting `--parse`, `--print`,
|
|
7
|
+
- a standalone command-line filter supporting `--parse`, `--print`,
|
|
8
|
+
and `--help`,
|
|
6
9
|
- a REPL module loaded with `:codec load <file.mjs>`.
|
|
7
10
|
|
|
8
|
-
The command-line entry point reads or writes the binary pattern+value
|
|
9
|
-
The REPL entry point provides interactive parsing
|
|
11
|
+
The command-line entry point reads or writes the binary pattern+value
|
|
12
|
+
stream. The REPL entry point provides interactive parsing
|
|
13
|
+
(`:input <codec.mjs>`) and multi-codec output formatting.
|
|
10
14
|
|
|
11
15
|
## Architecture & Mental Model
|
|
12
16
|
|
|
13
|
-
|
|
17
|
+
In `k`, there are no primitive numbers, strings, or built-in numerical
|
|
18
|
+
literals. All data is represented as algebraic shapes (e.g. bits, units,
|
|
19
|
+
products, and tagged unions). A codec bridges external formats with
|
|
20
|
+
these structural *k* values:
|
|
14
21
|
|
|
15
|
-
- `parse(text)` translates external text to an enveloped *k* value (a
|
|
16
|
-
|
|
17
|
-
- `
|
|
22
|
+
- `parse(text)` translates external text to an enveloped *k* value (a
|
|
23
|
+
`Value` with an associated closed pattern envelope `(P, v)`).
|
|
24
|
+
- `print(value)` translates an enveloped *k* value into formatted
|
|
25
|
+
external text, throwing an `Error` if the value is not representable.
|
|
26
|
+
- `doc` (optional string) documents the accepted syntax and semantic
|
|
27
|
+
representation.
|
|
18
28
|
|
|
19
|
-
Codecs are identified directly by their file name. They do not export
|
|
29
|
+
Codecs are identified directly by their file name. They do not export
|
|
30
|
+
artificial identifiers like `name` or pattern registries like
|
|
31
|
+
`patterns = [...]`.
|
|
20
32
|
|
|
21
33
|
In the REPL:
|
|
22
34
|
- **Input**: The user explicitly chooses which codec to parse with:
|
|
23
|
-
- Interactive mode: `:input <codec.mjs>` sets the prompt to
|
|
24
|
-
|
|
25
|
-
-
|
|
35
|
+
- Interactive mode: `:input <codec.mjs>` sets the prompt to
|
|
36
|
+
`<codec.mjs>> `, and the next entered line is parsed with that codec.
|
|
37
|
+
- One-line mode: `:input <codec.mjs> <text>` immediately parses
|
|
38
|
+
`<text>` using that codec.
|
|
39
|
+
- **Output**: Any evaluated *k* value is displayed in its standard *k*
|
|
40
|
+
envelope representation, followed by formatting from **all loaded
|
|
41
|
+
codecs** that can represent it:
|
|
26
42
|
```text
|
|
27
|
-
>
|
|
43
|
+
> :input int.mjs 42
|
|
28
44
|
{}|_|0|1|0|1|0|1|+ ?<{} _, ...>
|
|
29
45
|
int.mjs: 42
|
|
30
46
|
```
|
|
31
|
-
Every loaded codec attempts `codec.print(value)`. If it succeeds, the
|
|
47
|
+
Every loaded codec attempts `codec.print(value)`. If it succeeds, the
|
|
48
|
+
REPL prints `<codec.mjs>: <formatted>`. If it throws, the codec is
|
|
49
|
+
silently ignored for that value.
|
|
32
50
|
|
|
33
51
|
## Module Interface
|
|
34
52
|
|
|
@@ -38,7 +56,8 @@ A codec module exports:
|
|
|
38
56
|
export const doc = "Description of syntax and usage.";
|
|
39
57
|
|
|
40
58
|
export function parse(text) {
|
|
41
|
-
// parse text into an enveloped k Value (P, v) using
|
|
59
|
+
// parse text into an enveloped k Value (P, v) using
|
|
60
|
+
// withPattern(value, pattern)
|
|
42
61
|
}
|
|
43
62
|
|
|
44
63
|
export function print(value) {
|
|
@@ -47,24 +66,31 @@ export function print(value) {
|
|
|
47
66
|
}
|
|
48
67
|
```
|
|
49
68
|
|
|
50
|
-
`parse` should throw an `Error` for invalid external text. `print` should
|
|
69
|
+
`parse` should throw an `Error` for invalid external text. `print` should
|
|
70
|
+
throw when the value is not representable by the external format. REPL
|
|
71
|
+
output suppresses `print` errors so loaded codecs only output when they
|
|
72
|
+
recognize the value.
|
|
51
73
|
|
|
52
74
|
## The Codec SDK & Runner
|
|
53
75
|
|
|
54
|
-
To eliminate CLI stream boilerplate, codecs import from
|
|
76
|
+
To eliminate CLI stream boilerplate, codecs import from
|
|
77
|
+
`./runtime/codec-sdk.mjs` and invoke `runCodecCLI`:
|
|
55
78
|
|
|
56
79
|
```js
|
|
57
80
|
import { Value, isVariant, withPattern, runCodecCLI } from "./runtime/codec-sdk.mjs";
|
|
58
81
|
```
|
|
59
82
|
|
|
60
|
-
`runCodecCLI(import.meta.url, { parse, print, doc })` automatically
|
|
83
|
+
`runCodecCLI(import.meta.url, { parse, print, doc })` automatically
|
|
84
|
+
handles:
|
|
61
85
|
1. Entrypoint detection (no-op when loaded as a module or in browsers).
|
|
62
86
|
2. `-h` and `--help` CLI flag handling.
|
|
63
87
|
3. `--parse` and `--print` stream piping and wire encoding/decoding.
|
|
64
88
|
|
|
65
89
|
## Minimal Type-Specific Codec Example
|
|
66
90
|
|
|
67
|
-
This codec accepts `yes` and `no` for the type `<{} false, {} true>`.
|
|
91
|
+
This codec accepts `yes` and `no` for the type `<{} false, {} true>`.
|
|
92
|
+
Patterns are derived by `k` from filter expressions via
|
|
93
|
+
`patternFromFilter`:
|
|
68
94
|
|
|
69
95
|
```js
|
|
70
96
|
#!/usr/bin/env node
|
|
@@ -79,8 +105,12 @@ Boolean codec: accepts 'yes' and 'no'.
|
|
|
79
105
|
|
|
80
106
|
export function parse(text) {
|
|
81
107
|
const word = text.trim();
|
|
82
|
-
if (word === "yes")
|
|
83
|
-
|
|
108
|
+
if (word === "yes") {
|
|
109
|
+
return withPattern(Value.variant("true", Value.product({})), BOOL_PATTERN);
|
|
110
|
+
}
|
|
111
|
+
if (word === "no") {
|
|
112
|
+
return withPattern(Value.variant("false", Value.product({})), BOOL_PATTERN);
|
|
113
|
+
}
|
|
84
114
|
throw new Error("expected yes or no");
|
|
85
115
|
}
|
|
86
116
|
|
|
@@ -116,10 +146,12 @@ printf 'yes\n' | ./codecs/yesno.mjs --parse | ./codecs/yesno.mjs --print
|
|
|
116
146
|
For CLI codecs, test both directions in a pipeline:
|
|
117
147
|
|
|
118
148
|
```sh
|
|
119
|
-
printf 'yes\n' | node codecs/yesno.mjs --parse |
|
|
149
|
+
printf 'yes\n' | node codecs/yesno.mjs --parse | \
|
|
150
|
+
node codecs/yesno.mjs --print
|
|
120
151
|
```
|
|
121
152
|
|
|
122
|
-
For REPL codecs, add a focused case to
|
|
153
|
+
For REPL codecs, add a focused case to
|
|
154
|
+
[`../tests/test-repl.mjs`](../tests/test-repl.mjs):
|
|
123
155
|
|
|
124
156
|
```js
|
|
125
157
|
const state = createState();
|
|
@@ -142,16 +174,29 @@ npm test
|
|
|
142
174
|
## Web REPL (VFS) Bundling
|
|
143
175
|
|
|
144
176
|
When `repl.html` is generated (`npm run build:repl-html`):
|
|
145
|
-
- Codecs (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`) are
|
|
146
|
-
|
|
147
|
-
-
|
|
177
|
+
- Codecs (`int.mjs`, `json.mjs`, `utf8.mjs`, `unit.mjs`, `ieee.mjs`) are
|
|
178
|
+
bundled into 100% self-contained ES modules with zero imports.
|
|
179
|
+
- `runCodecCLI` is shimmed as a no-op, and `Value` helpers are directly
|
|
180
|
+
inlined.
|
|
181
|
+
- In the browser, the VFS serves these self-contained modules directly to
|
|
182
|
+
native dynamic `import(blobUrl)`. No runtime helper files
|
|
183
|
+
(`codecs/runtime/*`) need to be included in the VFS or exposed as
|
|
184
|
+
globals.
|
|
148
185
|
|
|
149
186
|
## Checklist
|
|
150
187
|
|
|
151
|
-
- Return an enveloped `Value` from `parse` using
|
|
188
|
+
- Return an enveloped `Value` from `parse` using
|
|
189
|
+
`withPattern(val, pattern)`.
|
|
152
190
|
- Keep `parse` and `print` deterministic and free of side effects.
|
|
153
|
-
- Validate shapes in `print` and throw clear errors when not
|
|
154
|
-
|
|
191
|
+
- Validate shapes in `print` and throw clear errors when not
|
|
192
|
+
representable.
|
|
193
|
+
- Use `runCodecCLI(import.meta.url, { parse, print, doc })` for CLI
|
|
194
|
+
execution.
|
|
155
195
|
- No artificial `name` or `patterns` exports required.
|
|
156
196
|
- Add `-h` and `--help` support via `runCodecCLI`.
|
|
157
197
|
|
|
198
|
+
- [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) —
|
|
199
|
+
guide to .k, .ko, .klib, and .kvm formats
|
|
200
|
+
- [DOCS/CODEC_ARCHITECTURE.md](./CODEC_ARCHITECTURE.md) —
|
|
201
|
+
codec architectural design
|
|
202
|
+
- [DOCS/REPL.md](./REPL.md) — interactive interpreter and codec commands
|