@fraczak/k 6.10.5 → 7.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +70 -0
- package/CLAUDE.md +1 -1
- package/DOCS/BIG_PICTURE.md +156 -172
- package/DOCS/BINARY_CODEC_STATUS_AND_TODO.md +3 -3
- package/DOCS/CODEC_ARCHITECTURE.md +2 -2
- package/DOCS/DICTIONARY.md +243 -250
- package/DOCS/DIVERGENCE_DETECTION.md +0 -1
- package/DOCS/FILE_FORMATS.md +5 -5
- package/DOCS/KIR_SPECIALIZATION_DESIGN.md +1 -1
- package/DOCS/OBJECT_FILE_AND_PATTERN.md +12 -12
- package/DOCS/PATTERNS.md +17 -17
- package/DOCS/PATTERNS_AND_TYPED_VALUES.md +1 -1
- package/DOCS/REPL.md +9 -19
- package/DOCS/TYPE_DERIVATION.md +2 -8
- package/DOCS/WEB_REPL_REQUIREMENTS.md +8 -8
- package/DOCS/book/01-introduction.md +8 -11
- package/DOCS/book/02-syntax-and-values.md +19 -20
- package/DOCS/book/03-types-as-finite-automata.md +42 -43
- package/DOCS/book/04-partial-functions-and-composition.md +2 -3
- package/DOCS/book/05-typing-filters-and-normalization.md +44 -46
- package/DOCS/book/06-values-in-memory.md +10 -10
- package/DOCS/book/08-operational-semantics-and-execution.md +21 -26
- package/DOCS/book/09-compiler-architecture.md +11 -11
- package/DOCS/book/10-from-ast-to-intermediate-representation.md +6 -7
- package/DOCS/book/14-canonical-serialization.md +2 -2
- package/DOCS/book/appendix-b-incorporating-external-predefined-types-and-functions.md +17 -17
- package/DOCS/book.md +5 -5
- package/Examples/arithmetics.k +49 -153
- package/Examples/bnat.k +13 -13
- package/Examples/byte.k +2 -50
- package/Examples/ieee.k +408 -423
- package/Examples/karatsuba-mult.k +30 -34
- package/Examples/nat.k +5 -6
- package/README.md +54 -49
- package/TODO.md +13 -18
- package/browser/repl-browser.mjs +40 -22
- package/browser/shims/fs.mjs +7 -10
- package/browser/vfs-data.mjs +12 -12
- package/codecs/BINARY_FORMAT.md +4 -4
- package/codecs/POLYMORPHIC_BINARY_FORMAT.md +4 -4
- package/codecs/README.md +6 -6
- package/codecs/ieee.mjs +15 -11
- package/codecs/int.mjs +9 -5
- package/codecs/string-codec.mjs +46 -47
- package/core.k +124 -175
- package/object.mjs +0 -7
- package/package.json +5 -2
- package/parser.jison +2 -41
- package/parser.mjs +55 -95
- package/pretty.mjs +11 -4
- package/repl.mjs +8 -63
- package/scripts/build-repl-html.mjs +18 -49
- package/tests/code-derivation/blowup.mjs +2 -4
- package/tests/code-derivation/bnat.mjs +2 -3
- package/tests/code-derivation/dot.mjs +5 -11
- package/tests/code-derivation/nat.mjs +2 -2
- package/tests/code-derivation/not-not.mjs +6 -6
- package/tests/code-derivation/pattern-envelope.mjs +4 -6
- package/tests/code-derivation/product.mjs +3 -6
- package/tests/code-derivation/rlz.mjs +4 -5
- package/tests/test-codecs.mjs +1 -1
- package/tests/test-fingerprint.mjs +36 -41
- package/tests/test-hash-normalization.mjs +4 -6
- package/tests/test-ieee-arithmetic.mjs +6 -7
- package/tests/test-k-object.mjs +11 -15
- package/tests/test-kir.mjs +0 -1
- package/tests/test-kvm-polymorphic.mjs +1 -2
- package/tests/test-kvm.mjs +8 -11
- package/tests/test-repl-html.mjs +86 -9
- package/tests/test-repl.mjs +38 -53
- package/tests/test.mjs +2 -3
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,76 @@ All notable changes to this project will be documented here.
|
|
|
5
5
|
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
6
6
|
This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [7.0.0] — 2026-10-10 — Unified Schemas, Relational Contracts, and Pure Syntax
|
|
9
|
+
|
|
10
|
+
### Breaking Language & Syntax Changes
|
|
11
|
+
|
|
12
|
+
- **Pure Relation Syntax (Removal of `$`)**:
|
|
13
|
+
- Removed type declaration syntax (`$name = ...;`) and type expressions
|
|
14
|
+
from the grammar (`parser.jison`).
|
|
15
|
+
- All definitions in `.k` source files are now uniformly defined as
|
|
16
|
+
relations: `name = relExpr;`.
|
|
17
|
+
- Data shapes and types are defined as algebraic schemas via filter
|
|
18
|
+
expressions (`?schema`) acting as partial identity relations.
|
|
19
|
+
- Removed `:type` and `:types` REPL commands in favor of `:rel`,
|
|
20
|
+
`:show`, and `:env`.
|
|
21
|
+
|
|
22
|
+
### Conceptual Architecture & Narrative
|
|
23
|
+
|
|
24
|
+
- **Schemas, Contracts, and Framed Payloads**:
|
|
25
|
+
- **Schema**: Algebraic finite tree automata defining acceptable labeled
|
|
26
|
+
tree payloads (replacing "codes" and "type patterns").
|
|
27
|
+
- **Contract**: Connected $(input, output)$ schema graphs with shared
|
|
28
|
+
vertices ($=X$) proving structure preservation across relations.
|
|
29
|
+
- **Filter**: Syntactic operator `?schema` acting as a partial identity
|
|
30
|
+
relation and inductive termination witness.
|
|
31
|
+
- **Framed Payload `(schema, payload)`**: Universal self-describing
|
|
32
|
+
wire framing representation.
|
|
33
|
+
- **Canonical Schema**: Deterministic normal form of schema automata;
|
|
34
|
+
hashes (`@hash`) serve as fixed-width content-addressable digests in
|
|
35
|
+
the Schema Registry.
|
|
36
|
+
|
|
37
|
+
### Standard Library & Documentation
|
|
38
|
+
|
|
39
|
+
- **Library & Codec Migration**:
|
|
40
|
+
- Updated `core.k`, `Examples/`, and test suites to pure relational
|
|
41
|
+
syntax.
|
|
42
|
+
- Updated string codecs and Web REPL VFS data.
|
|
43
|
+
- **Documentation Overhaul**:
|
|
44
|
+
- Overhauled `README.md`, `DOCS/BIG_PICTURE.md`, `DOCS/DICTIONARY.md`,
|
|
45
|
+
`DOCS/PATTERNS.md`, `DOCS/book.md`, and *The k Book* chapters 1–5.
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## [6.10.6] — 2026-10-08 — Minimal Core Schema and REPL VFS Uploads
|
|
50
|
+
|
|
51
|
+
### Core Schema & Language
|
|
52
|
+
|
|
53
|
+
- **Minimal `core.k` Schema**:
|
|
54
|
+
- Streamlined `core.k` to focus strictly on structural type schemas
|
|
55
|
+
and pattern foundations.
|
|
56
|
+
- Removed legacy bit arithmetic (`succ`, `plus`, `times`, constants
|
|
57
|
+
`2`..`10`, and helpers `inv`, `concat`), achieving a 39% reduction
|
|
58
|
+
in file size.
|
|
59
|
+
- Retained `$bit`, `$bits`, `_`, `0`, `1`, `$string`, `$unicode`, and
|
|
60
|
+
`$pattern` definitions intact for type derivation and codecs.
|
|
61
|
+
|
|
62
|
+
### Web REPL & VFS
|
|
63
|
+
|
|
64
|
+
- **VFS Namespace Flattening**:
|
|
65
|
+
- Flattened VFS back to root filenames (`int.mjs`, `json.mjs`,
|
|
66
|
+
`utf8.mjs`, `unit.mjs`, `ieee.mjs`), ensuring consistent resolution
|
|
67
|
+
across browser and CLI environments.
|
|
68
|
+
- **Upload File Directly to VFS**:
|
|
69
|
+
- Updated "Upload" action to strictly add uploaded files into the VFS
|
|
70
|
+
without auto-loading or evaluating them as K code.
|
|
71
|
+
- Enables uploading JavaScript codecs, JSON payloads, and test vectors.
|
|
72
|
+
- Expanded file picker `accept` filter to allow `.mjs`, `.js`, and
|
|
73
|
+
`.json` files.
|
|
74
|
+
- Supported `:codec load` for `.js` codecs in the VFS explorer modal.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
8
78
|
## [6.10.5] — 2026-10-08 — Adaptive Division and VFS Codecs
|
|
9
79
|
|
|
10
80
|
### Arithmetic & Algorithms
|
package/CLAUDE.md
CHANGED
|
@@ -61,4 +61,4 @@ npm test # Run comprehensive test suite (unit tests, derivation tests, shell te
|
|
|
61
61
|
- No external runtime dependencies (only jiwson for parser generation)
|
|
62
62
|
- Uses ES6 modules (.mjs files) throughout
|
|
63
63
|
- Extensive test suite in `tests/code-derivation/` for type system validation
|
|
64
|
-
- REPL supports commands like `:rel name` (show relation definition), `:
|
|
64
|
+
- REPL supports commands like `:rel name` (show relation definition), `:rels`, etc.
|
package/DOCS/BIG_PICTURE.md
CHANGED
|
@@ -1,94 +1,85 @@
|
|
|
1
1
|
# Big Picture
|
|
2
2
|
|
|
3
|
-
`
|
|
4
|
-
|
|
3
|
+
`k` is a minimal, unified calculus of **content-addressable schemas** and
|
|
4
|
+
**relational data transformations**.
|
|
5
5
|
|
|
6
6
|
## Basics
|
|
7
7
|
|
|
8
|
-
The language describes
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
describing the family of types compatible with that tree.
|
|
8
|
+
The language describes first-order partial relations on labeled tree values.
|
|
9
|
+
Values are rooted, edge-labeled trees, intuitively similar to an XML or a JSON
|
|
10
|
+
document. Every value is self-describing at boundaries as a pair:
|
|
11
|
+
`(schema, payload)`.
|
|
13
12
|
|
|
14
|
-
###
|
|
13
|
+
### Schemas
|
|
15
14
|
|
|
16
|
-
Formally, a
|
|
17
|
-
values, which corresponds to the
|
|
18
|
-
automaton. Therefore, every
|
|
19
|
-
its minimal deterministic tree automaton.
|
|
15
|
+
Formally, a _schema_ is a finite tree automaton. The schema defines a set of
|
|
16
|
+
values, which corresponds to the language of trees accepted by the
|
|
17
|
+
automaton. Therefore, every schema admits a unique canonical normal form in
|
|
18
|
+
terms of its minimal deterministic tree automaton.
|
|
20
19
|
|
|
21
|
-
|
|
20
|
+
Schemas are built from two structural constructors:
|
|
22
21
|
|
|
23
|
-
1. Product
|
|
24
|
-
2. Union (
|
|
22
|
+
1. **Product** (`{ ... }`)
|
|
23
|
+
2. **Union** (`< ... >`) (tagged or disjoint union of variants)
|
|
25
24
|
|
|
26
|
-
> Notation:
|
|
27
|
-
>
|
|
28
|
-
>
|
|
29
|
-
> equivalent in meaning.
|
|
25
|
+
> Notation: Native k notation uses products `{ S l, ... }` and unions
|
|
26
|
+
> `< S t, ... >`. The empty product `{}` has a single value (the _unit_).
|
|
27
|
+
> The empty union `<>` has no values.
|
|
30
28
|
|
|
31
29
|
```bnf
|
|
32
|
-
|
|
33
|
-
|
|
30
|
+
schema ::= name | '{' schema_label_list '}' | '<' schema_label_list '>'
|
|
31
|
+
schema_label_list ::= /* empty */ | ( schema label ',' )* schema label
|
|
34
32
|
name ::= IDENTIFIER
|
|
35
33
|
label ::= STRING
|
|
36
34
|
```
|
|
37
35
|
|
|
38
|
-
Given a finite (possibly empty) set of
|
|
39
|
-
set of pairwise
|
|
36
|
+
Given a finite (possibly empty) set of schemas `s1, s2, ..., sn` and a
|
|
37
|
+
set of pairwise distinct labels `l1, l2, ..., ln`:
|
|
40
38
|
|
|
41
|
-
-
|
|
42
|
-
|
|
43
|
-
Each projection
|
|
44
|
-
`
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
mapping a value of type `<t1 l1, t2 l2, ..., tn ln>` into a value
|
|
49
|
-
of type `t1`, `t2`, `...`, `tn`, respectively.
|
|
39
|
+
- `{ s1 l1, s2 l2, ..., sn ln }` defines a _product schema_ with
|
|
40
|
+
`n` projection relations: `.l1`, `.l2`, `...`, `.ln`.
|
|
41
|
+
Each projection maps a value matching the product schema into a value
|
|
42
|
+
matching `s1`, `s2`, `...`, `sn`, respectively.
|
|
43
|
+
- `< s1 l1, s2 l2, ..., sn ln >` defines a _union schema_ with
|
|
44
|
+
`n` projection relations `/l1`, `/l2`, `...`, `/ln`,
|
|
45
|
+
mapping a variant into its payload matching `s1`, `s2`, `...`, `sn`.
|
|
50
46
|
|
|
51
|
-
|
|
52
|
-
(meaning, they are defined for every value of the input product type),
|
|
53
|
-
whereas in the case of a union type, exactly one projection function
|
|
54
|
-
is defined for a given value of the union type.
|
|
47
|
+
#### Variant (Union) Values
|
|
55
48
|
|
|
56
|
-
|
|
49
|
+
- Variant values are constructed using "|" (pipe) followed by the tag:
|
|
50
|
+
- Unit variant: `{} | nil`, `{} | zero`
|
|
51
|
+
- Variant with payload `v` at tag: `v | tag`
|
|
52
|
+
- Example (list): with `list = ?< {} nil, { X car, L cdr } cons > = L;`:
|
|
53
|
+
- Empty list `[]`: `{} | nil`
|
|
54
|
+
- Singleton `[v]`: `{ v car, {} | nil cdr } | cons`
|
|
55
|
+
- Angle brackets `< ... >` denote ordered choice (union expressions); use
|
|
56
|
+
`| tag` for variant construction.
|
|
57
57
|
|
|
58
|
-
|
|
59
|
-
2. The empty union type, `<>`, has no values.
|
|
58
|
+
#### Equivalence of Schemas (Bisimilarity)
|
|
60
59
|
|
|
61
|
-
|
|
60
|
+
Two (possibly recursive) schemas are equivalent iff they are bisimilar
|
|
61
|
+
over their definition graphs. Concretely, there exists a relation B
|
|
62
|
+
such that `(s1, s2) ∈ B` if and only if:
|
|
63
|
+
- `s1` and `s2` have exactly the same set of labels/tags;
|
|
64
|
+
- `s1` and `s2` are simultaneously products or simultaneously unions;
|
|
65
|
+
- for each label/tag `ℓ`, the subschemas under `ℓ` are again related by B.
|
|
62
66
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
- Unit variant `` `{}` `` | tag (e.g., `{} | nil`, `{} | zero`)
|
|
66
|
-
- Variant with payload `v` at label `tag`: ` v | tag `
|
|
67
|
-
- Example (list): with `$list = < {} nil, {X car, list cdr} cons >`:
|
|
68
|
-
- Empty list []: `{}|nil`
|
|
69
|
-
- Singleton `[v]`: `{ v car, {}|nil cdr } | cons`
|
|
70
|
-
- Angle brackets are for type definitions and merge expressions; use
|
|
71
|
-
"`| tag`" for a variant construction.
|
|
67
|
+
Canonicalization erases local variable names and deterministically orders
|
|
68
|
+
states, so any two structurally equivalent schemas yield identical canonical text.
|
|
72
69
|
|
|
73
|
-
|
|
70
|
+
### Filters
|
|
74
71
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
- t1 and t2 have exactly the same set of labels/tags;
|
|
79
|
-
- t1 and t2 are simultaneously products or simultaneously unions;
|
|
80
|
-
- for each label/tag `ℓ` in the set, the subcodes under `ℓ` are again
|
|
81
|
-
related by B.
|
|
72
|
+
A _filter_ is a syntactic expression `?schema` that acts as a partial identity
|
|
73
|
+
relation on values conforming to `schema`. If the input value matches the
|
|
74
|
+
schema, it passes through unchanged; otherwise, the relation is undefined.
|
|
82
75
|
|
|
83
|
-
|
|
76
|
+
In recursive relations, filters also serve as **inductive termination witnesses**,
|
|
77
|
+
ensuring type and contract derivation converges.
|
|
84
78
|
|
|
85
|
-
|
|
86
|
-
sharing some structure. For example, if I wanted to refer to all product
|
|
87
|
-
types that have a field `f` of type `T`, I would write:
|
|
88
|
-
`?{ T f, ... }`. In general, a filter is defined by:
|
|
79
|
+
In general, a filter schema is defined by:
|
|
89
80
|
|
|
90
81
|
```bnf
|
|
91
|
-
filter ::= name | '
|
|
82
|
+
filter ::= name | '{' filter_label_list '}'
|
|
92
83
|
| '<' filter_label_list '>' | '(' filter_label_list ')'
|
|
93
84
|
| filter '=' name
|
|
94
85
|
filter_label_list ::= /* empty */ | '...'
|
|
@@ -98,11 +89,11 @@ filter_label_list ::= /* empty */ | '...'
|
|
|
98
89
|
Examples:
|
|
99
90
|
|
|
100
91
|
```text
|
|
101
|
-
X --
|
|
102
|
-
( ... ) --
|
|
103
|
-
{ ... } -- any product
|
|
104
|
-
( X x, X y ) -- any
|
|
105
|
-
< X x, ... > = X -- a
|
|
92
|
+
X -- unconstrained schema variable
|
|
93
|
+
( ... ) -- unconstrained schema
|
|
94
|
+
{ ... } -- any product schema
|
|
95
|
+
( X x, X y ) -- any product schema with two fields of the same schema X
|
|
96
|
+
< X x, ... > = X -- a recursive variant schema X with variant x of schema X
|
|
106
97
|
```
|
|
107
98
|
|
|
108
99
|
### Partial functions
|
|
@@ -134,104 +125,96 @@ partial functions, not only two. For example, empty composition,
|
|
|
134
125
|
|
|
135
126
|
### Program
|
|
136
127
|
|
|
137
|
-
A `kernel-core` program is a set of definitions of
|
|
138
|
-
functions, followed by an _expression_, which is the
|
|
139
|
-
|
|
128
|
+
A `kernel-core` program is a set of definitions of relations (partial
|
|
129
|
+
functions), followed by an _expression_, which is the relation the
|
|
130
|
+
program evaluates.
|
|
140
131
|
|
|
141
132
|
The language syntax is defined by:
|
|
142
133
|
|
|
143
134
|
```bnf
|
|
144
|
-
program ::=
|
|
145
|
-
type_definition ::= '$' type_name '=' type ';'
|
|
135
|
+
program ::= function_definition* expression
|
|
146
136
|
function_definition ::= name '=' expression ';'
|
|
147
|
-
expression ::= name | '
|
|
148
|
-
| projection | composition | union | product
|
|
137
|
+
expression ::= name | '?' filter
|
|
138
|
+
| projection | constructor | composition | union | product
|
|
149
139
|
projection ::= '.' label | '/' label
|
|
140
|
+
constructor ::= '|' label
|
|
150
141
|
composition ::= '(' expression * ')'
|
|
151
|
-
union ::= '<' (expression ',')
|
|
142
|
+
union ::= '<' (expression ',') * expression '>'
|
|
152
143
|
product ::= '{' expr_label_list '}'
|
|
153
144
|
expr_label_list ::= /* empty */
|
|
154
145
|
| (expression label_name ',')* expression label
|
|
155
146
|
```
|
|
156
147
|
|
|
157
|
-
We assume that in a `kernel-core` program the names (for
|
|
158
|
-
|
|
148
|
+
We assume that in a `kernel-core` program the names (for relations
|
|
149
|
+
and labels) are identifiers (strings) such that:
|
|
159
150
|
|
|
160
|
-
1. All defined
|
|
161
|
-
2. All
|
|
162
|
-
3. All label names within a type definition or a `product` expression
|
|
163
|
-
are locally distinct.
|
|
151
|
+
1. All defined relation names are distinct.
|
|
152
|
+
2. All label names within a `product` expression are locally distinct.
|
|
164
153
|
|
|
165
|
-
Filters
|
|
166
|
-
|
|
167
|
-
|
|
154
|
+
Filters act as partial identity relations defined only for the values
|
|
155
|
+
matching the corresponding schema, while also serving as inductive witnesses
|
|
156
|
+
for recursive relation termination.
|
|
168
157
|
|
|
169
158
|
That's it.
|
|
170
159
|
|
|
171
160
|
No `builtin` types, no `if` statement, no `loop`, no `throw`, no
|
|
172
|
-
_closure_, no _annotations_, and no macros. Just
|
|
173
|
-
and
|
|
161
|
+
_closure_, no _annotations_, and no macros. Just _schemas_, _filters_,
|
|
162
|
+
and _relations_.
|
|
174
163
|
|
|
175
164
|
### Normalization and Content-Addressing
|
|
176
165
|
|
|
177
|
-
All
|
|
178
|
-
minimal deterministic tree automaton representation
|
|
166
|
+
All schemas are normalized to the canonical text representation of their
|
|
167
|
+
minimal deterministic tree automaton. That canonical representation is their
|
|
168
|
+
true identity; `@hash` is an immutable, fixed-width index digest.
|
|
179
169
|
|
|
180
170
|
Every relation is identified canonically by the content-addressed hash
|
|
181
171
|
of its normalized definition AST.
|
|
182
172
|
|
|
183
173
|
There is no global symbol namespace. Human-readable names (such as
|
|
184
174
|
`plus`, `int`, or `5`) are local aliases recorded in object metadata
|
|
185
|
-
(`relAlias
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
175
|
+
(`relAlias`). When libraries are loaded with `--lib`, definitions are
|
|
176
|
+
identified by their canonical `@hash`, but local aliases are not
|
|
177
|
+
implicitly exported into the importing scope unless explicitly mapped
|
|
178
|
+
via `--export`.
|
|
189
179
|
|
|
190
180
|
This approach completely solves the problem of modules, name collisions,
|
|
191
|
-
and unversioned imports,
|
|
192
|
-
|
|
181
|
+
and unversioned imports, establishing a universal registry of
|
|
182
|
+
schemas and relations.
|
|
193
183
|
|
|
194
|
-
##
|
|
184
|
+
## Schemas, Contracts, and Derivation
|
|
195
185
|
|
|
196
186
|
The abstract syntax tree of a program consists of a dictionary of
|
|
197
|
-
|
|
198
|
-
final _main expression_.
|
|
199
|
-
|
|
200
|
-
In principle, the initial form of the Abstract Syntax Tree, _raw_ AST,
|
|
201
|
-
is enough to evaluate the program on an input.
|
|
187
|
+
relation definitions and an optional entry expression.
|
|
202
188
|
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
189
|
+
The compiler computes canonical representations for schemas and derives
|
|
190
|
+
a **contract** for every relation: a connected $(input, output)$ schema
|
|
191
|
+
graph where shared vertices ($=X$) prove that input substructures are
|
|
192
|
+
preserved through to the output.
|
|
206
193
|
|
|
207
|
-
The
|
|
194
|
+
The contract derivation steps are:
|
|
208
195
|
|
|
209
|
-
1.
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
2. Annotate the expressions with filters; every node of the AST is
|
|
213
|
-
annotated with a pair of filters.
|
|
214
|
-
3. Turn the _singleton_ filters into types and add them to the _type
|
|
215
|
-
graph_. Go to (2) unless no change.
|
|
196
|
+
1. Initialize the schema graph from primitive operations (projections, constructors, products) and explicit filter annotations.
|
|
197
|
+
2. Propagate schemas forward and backward through compositions, products, and choices until reaching a fixed point.
|
|
198
|
+
3. Validate inductive convergence for recursive relations using filter witnesses.
|
|
216
199
|
|
|
217
200
|
### Examples
|
|
218
201
|
|
|
219
202
|
#### Example 1
|
|
220
203
|
|
|
221
204
|
```text
|
|
222
|
-
|
|
223
|
-
bit0 = | o
|
|
224
|
-
bit1 = | i
|
|
205
|
+
bit = ?< {} o, {} i >;
|
|
206
|
+
bit0 = | o;
|
|
207
|
+
bit1 = | i;
|
|
225
208
|
|
|
226
|
-
|
|
227
|
-
zero = bit0 { () b0, () b1, () b2, () b3 }
|
|
209
|
+
byte = ?{ bit b0, bit b1, bit b2, bit b3 };
|
|
210
|
+
zero = bit0 { () b0, () b1, () b2, () b3 };
|
|
228
211
|
|
|
229
|
-
inc =
|
|
212
|
+
inc = byte
|
|
230
213
|
<
|
|
231
|
-
{ bit0 overflown, { .b0/o bit1 b0, .b1 b1, .b2 b2, .b3 b3 } byte },
|
|
232
|
-
{ bit0 overflown, { bit0 b0, .b1/o bit1 b1, .b2 b2, .b3 b3 } byte },
|
|
233
|
-
{ bit0 overflown, { bit0 b0, bit0 b1, .b2/o bit1 b2, .b3 b3 } byte },
|
|
234
|
-
{ bit0 overflown, { bit0 b0, bit0 b1, bit0 b2,
|
|
214
|
+
{ bit0 overflown, { .b0 /o bit1 b0, .b1 b1, .b2 b2, .b3 b3 } byte },
|
|
215
|
+
{ bit0 overflown, { bit0 b0, .b1 /o bit1 b1, .b2 b2, .b3 b3 } byte },
|
|
216
|
+
{ bit0 overflown, { bit0 b0, bit0 b1, .b2 /o bit1 b2, .b3 b3 } byte },
|
|
217
|
+
{ bit0 overflown, { bit0 b0, bit0 b1, bit0 b2, .b3 /o bit1 b3 } byte },
|
|
235
218
|
{ bit1 overflown, zero byte }
|
|
236
219
|
>
|
|
237
220
|
;
|
|
@@ -250,42 +233,42 @@ inc3 = inc inc_o inc_o;
|
|
|
250
233
|
inc3
|
|
251
234
|
```
|
|
252
235
|
|
|
253
|
-
In the above program, we define
|
|
254
|
-
and
|
|
236
|
+
In the above program, we define relation `bit` as a union of two unit variants,
|
|
237
|
+
and relation `byte` as a product of four `bit`s.
|
|
255
238
|
|
|
256
|
-
|
|
257
|
-
|
|
239
|
+
Relations `bit0`, `bit1`, and `zero` are "constant polymorphic
|
|
240
|
+
relations", meaning:
|
|
258
241
|
|
|
259
242
|
- constant: if defined, they always return exactly the same value;
|
|
260
243
|
- polymorphic: they are defined for more than one pair of input and
|
|
261
|
-
output
|
|
262
|
-
and `zero`
|
|
263
|
-
unconstrained
|
|
244
|
+
output schemas: relations `bit0` and `bit1` have contract `?X -> bit`,
|
|
245
|
+
and `zero` has contract `?X -> byte`, where `?X` denotes an
|
|
246
|
+
unconstrained open schema, also denoted as `?(...)`.
|
|
264
247
|
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
248
|
+
Relations `inc` and `inc3` are relations with contracts
|
|
249
|
+
`byte -> { bit overflown, byte byte }` and
|
|
250
|
+
`{ bit overflown, byte byte } -> { bit overflown, byte byte }`,
|
|
268
251
|
respectively.
|
|
269
252
|
|
|
270
|
-
|
|
271
|
-
`?X ->
|
|
253
|
+
Relation `inc_o` is a polymorphic relation with contract:
|
|
254
|
+
`?X -> { byte byte, bit overflown }` with the following input schema constraints:
|
|
272
255
|
|
|
273
|
-
- `?X` is a product
|
|
256
|
+
- `?X` is a product with at least two fields: `byte` and
|
|
274
257
|
`overflown`, denoted by:
|
|
275
|
-
> `?{
|
|
276
|
-
- `?Z` is a union
|
|
258
|
+
> `?{ byte byte, Z overflown, ...}`;
|
|
259
|
+
- `?Z` is a union with field `i` denoted by:
|
|
277
260
|
> `?< V i, ...>`;
|
|
278
261
|
- `V` is unconstrained, denoted by `(...)`;
|
|
279
262
|
|
|
280
|
-
We can write it as: `?{
|
|
263
|
+
We can write it as: `?{ byte byte, <(...) i, ...> overflown, ...}`.
|
|
281
264
|
|
|
282
|
-
The target
|
|
283
|
-
`?{
|
|
284
|
-
|
|
265
|
+
The target schema `{ byte byte, bit overflown }` corresponds to filter
|
|
266
|
+
`?{ byte byte, bit overflown }`. Such a schema is called a _closed
|
|
267
|
+
schema_, as all constructors are fully determined.
|
|
285
268
|
|
|
286
269
|
#### Example 2
|
|
287
270
|
|
|
288
|
-
Polymorphic list
|
|
271
|
+
Polymorphic list relations:
|
|
289
272
|
|
|
290
273
|
```text
|
|
291
274
|
list? = ?< {} nil, {X car, Y cdr} cons > = Y;
|
|
@@ -306,46 +289,47 @@ car = list? /cons .car; -- @...
|
|
|
306
289
|
|
|
307
290
|
## Universal Schema Registry
|
|
308
291
|
|
|
309
|
-
Since the normalization process for
|
|
310
|
-
we can build a universal schema registry that will store all
|
|
311
|
-
|
|
312
|
-
|
|
292
|
+
Since the normalization process for schemas is fast and deterministic,
|
|
293
|
+
we can build a universal schema registry that will store all discovered
|
|
294
|
+
schemas. The registry is a key-value store, where the primary identity is
|
|
295
|
+
the canonical textual definition (or fixed-width hash) of the normalized
|
|
296
|
+
schema, and the value is the normalized schema automaton itself.
|
|
313
297
|
|
|
314
|
-
All
|
|
298
|
+
All relations (monomorphic and polymorphic) are named by the hash
|
|
315
299
|
of their normalized definition and stored in a similar key-value store.
|
|
316
300
|
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
301
|
+
Relations can be indexed by their derived contracts—the connected $(input, output)$
|
|
302
|
+
schema graph—enabling instant lookup of transformations that map between specific schemas
|
|
303
|
+
or preserve specific subtrees ($=X$).
|
|
320
304
|
|
|
321
305
|
## Serialization and Compilation
|
|
322
306
|
|
|
323
|
-
The language is designed to transform
|
|
324
|
-
|
|
325
|
-
The transformation is
|
|
326
|
-
language.
|
|
327
|
-
non-tail-recursive)
|
|
307
|
+
The language is designed to transform self-describing binary streams of
|
|
308
|
+
`(schema, payload)`, where the payload is a labeled tree conforming to the
|
|
309
|
+
accompanying schema. The transformation is executed by partial relations
|
|
310
|
+
defined in the language. Non-recursive and tail-recursive (and even some
|
|
311
|
+
non-tail-recursive) relations can be compiled into deterministic finite
|
|
328
312
|
(pushdown) transducers.
|
|
329
313
|
|
|
330
314
|
Ahead-Of-Time (AOT) compilation produces four primary file formats (see
|
|
331
315
|
[`DOCS/FILE_FORMATS.md`](./FILE_FORMATS.md)):
|
|
332
|
-
- `.k`: Plain source text.
|
|
333
|
-
- `.ko`: AOT
|
|
334
|
-
- `.klib`: AOT
|
|
316
|
+
- `.k`: Plain source text defining schemas and relations.
|
|
317
|
+
- `.ko`: AOT contract-checked executable object container (`KOBJ\n` + JSON).
|
|
318
|
+
- `.klib`: AOT contract-checked library container (JSON).
|
|
335
319
|
- `.kvm`: Polymorphic register-IR template (`layer: "KVM-P"`).
|
|
336
320
|
|
|
337
321
|
Downstream backends compile or specialize these into WebAssembly
|
|
338
322
|
(`.wasm`) and native binaries.
|
|
339
323
|
|
|
340
|
-
### AND-OR graphs for encoding and decoding
|
|
324
|
+
### AND-OR graphs for encoding and decoding payloads
|
|
341
325
|
|
|
342
|
-
This
|
|
343
|
-
values. The idea is to use _prefix codes_ (a class of languages such
|
|
344
|
-
that no word is a prefix of another word) to encode
|
|
326
|
+
This provides a generic and compact way to encode and decode payload
|
|
327
|
+
values under an algebraic schema. The idea is to use _prefix codes_ (a class of languages such
|
|
328
|
+
that no word is a prefix of another word) to encode variant payloads deterministically.
|
|
345
329
|
|
|
346
330
|
### Rust data structures
|
|
347
331
|
|
|
348
|
-
A
|
|
332
|
+
A schema can be translated into a Rust data structure, and a relation
|
|
349
333
|
can be translated into a Rust function.
|
|
350
334
|
|
|
351
335
|
## Linking with Other Languages
|
|
@@ -356,14 +340,14 @@ can be translated into a Rust function.
|
|
|
356
340
|
|
|
357
341
|
In `k`, there are no primitive numbers or built-in integer literals.
|
|
358
342
|
Numerical identifiers such as `5` or `10` are nullary relations
|
|
359
|
-
(constants) mapping the unit
|
|
360
|
-
Arithmetic relations operate on signed integers (
|
|
343
|
+
(constants) mapping the unit `{}` to bit trees of contract `{} -> bits`.
|
|
344
|
+
Arithmetic relations operate on signed integers (`int`), requiring
|
|
361
345
|
conversion via `5 int`.
|
|
362
346
|
|
|
363
347
|
Bit trees are defined algebraically:
|
|
364
348
|
|
|
365
349
|
```text
|
|
366
|
-
|
|
350
|
+
bits = ?< {} _, B 0, B 1 > = B;
|
|
367
351
|
```
|
|
368
352
|
|
|
369
353
|
### Literals for `@bits` are:
|
|
@@ -377,24 +361,24 @@ $ bits = < {} _, bits 0, bits 1 >;
|
|
|
377
361
|
- "" for 0 bits
|
|
378
362
|
- "ala" for 24 bits
|
|
379
363
|
|
|
380
|
-
### Two operations on
|
|
364
|
+
### Two operations on `bits`:
|
|
381
365
|
|
|
382
366
|
#### Eat `/`
|
|
383
367
|
|
|
384
|
-
-
|
|
368
|
+
- `bits / bits` --- division, e.g., `0b1011 / 0b10` = `0b11`,
|
|
385
369
|
`"abc" / "a"` = `"bc"`
|
|
386
370
|
|
|
387
371
|
#### Prepend `\`
|
|
388
372
|
|
|
389
|
-
-
|
|
373
|
+
- `bits \ bits` --- multiplication, e.g., `0b11 \ 0b10` = `0b1011`,
|
|
390
374
|
`"bc" \ "a"` = `"abc"`
|
|
391
375
|
|
|
392
376
|
Empty `bits` can be checked by:
|
|
393
377
|
|
|
394
378
|
```k
|
|
395
|
-
kind =
|
|
379
|
+
kind = bits
|
|
396
380
|
< {/ 0b0 \ 0b0 starts_with_0}, {/ 0b1 \ 0b1 starts_with_1}, {() empty} >
|
|
397
|
-
|
|
381
|
+
< bits starts_with_0, bits starts_with_1, bits empty >;
|
|
398
382
|
```
|
|
399
383
|
|
|
400
384
|
## Further Reading
|
|
@@ -405,4 +389,4 @@ kind = $@bits
|
|
|
405
389
|
- [DOCS/TYPE_DERIVATION.md](./TYPE_DERIVATION.md) — type derivation
|
|
406
390
|
algorithm
|
|
407
391
|
- [DOCS/OBJECT_FILE_AND_PATTERN.md](./OBJECT_FILE_AND_PATTERN.md) —
|
|
408
|
-
object format and
|
|
392
|
+
object format and schema encoding
|
|
@@ -8,13 +8,13 @@ to self-hosted binary pattern framing.
|
|
|
8
8
|
The only wire format is:
|
|
9
9
|
|
|
10
10
|
```text
|
|
11
|
-
encode(
|
|
11
|
+
encode(pattern_value : pattern) encode(value : decoded_pattern)
|
|
12
12
|
```
|
|
13
13
|
|
|
14
14
|
where:
|
|
15
15
|
|
|
16
|
-
- the first segment is a normal k value of
|
|
17
|
-
fixed singleton pattern for
|
|
16
|
+
- the first segment is a normal k value of pattern `pattern`, encoded under the
|
|
17
|
+
fixed singleton pattern for `pattern` from [`core.k`](../core.k),
|
|
18
18
|
- that decoded pattern becomes the structural authority for the second segment,
|
|
19
19
|
- the second segment is the prefix-free value payload interpreted by that
|
|
20
20
|
pattern.
|
|
@@ -9,8 +9,8 @@ graph).
|
|
|
9
9
|
|
|
10
10
|
In **k**, there are no primitive numbers or built-in numerical types.
|
|
11
11
|
Numbers are nullary relations mapping the unit to bit trees
|
|
12
|
-
(
|
|
13
|
-
integer variants (
|
|
12
|
+
(`{} -> bits`), and arithmetic relations operate on signed
|
|
13
|
+
integer variants (`int`). Codecs like `int.mjs` bridge human
|
|
14
14
|
decimal representations and these algebraic structures.
|
|
15
15
|
|
|
16
16
|
Every codec in the **k** ecosystem is designed to serve three unified
|