@fraczak/k 6.10.2 → 6.10.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +73 -0
- package/DOCS/BIG_PICTURE.md +174 -100
- package/DOCS/CODECS.md +74 -29
- package/DOCS/CODEC_ARCHITECTURE.md +87 -31
- package/DOCS/CONVERGENCE.md +42 -11
- package/DOCS/DICTIONARY.md +69 -2
- package/DOCS/FILE_FORMATS.md +420 -0
- package/DOCS/KIR_V1.md +53 -42
- package/DOCS/KVM_EXECUTION_MODEL.md +189 -147
- package/DOCS/OBJECT_FILE_AND_PATTERN.md +35 -17
- package/DOCS/PATTERNS.md +10 -6
- package/DOCS/REPL.md +100 -60
- package/DOCS/TEXTUAL_VALUES.md +24 -19
- package/DOCS/TYPE_DERIVATION.md +68 -22
- package/Examples/karatsuba-mult.k +215 -0
- package/README.md +92 -58
- package/backends/arm64/scripts/run-tests.mjs +16 -0
- package/backends/llvm/bin/k-llvm-build.mjs +40 -14
- package/backends/llvm/bin/k-llvm-compile.mjs +29 -12
- package/backends/llvm/runtime/krt.c +10 -10
- package/backends/llvm/runtime/krt.h +6 -1
- package/backends/llvm/scripts/run-tests.mjs +16 -0
- package/backends/llvm/src/cli.mjs +1 -1
- package/backends/llvm/src/executable.mjs +36 -0
- package/backends/llvm/src/kvm2llvm.mjs +10 -4
- package/backends/llvm/tests/integration.sh +11 -0
- package/backends/llvm/tests/test-llvm.mjs +15 -1
- package/backends/wasm/scripts/run-tests.mjs +16 -0
- package/browser/vfs-data.mjs +1 -1
- package/codecs/README.md +78 -67
- package/core.k +5 -5
- package/objects/README.md +57 -37
- package/package.json +2 -1
- package/scripts/benchmark-idempotency.mjs +27 -0
- package/scripts/benchmark.mjs +51 -0
- package/scripts/build-repl-html.mjs +17 -0
- package/scripts/generate-dependency-graph.mjs +25 -1
- package/scripts/perf-ieee.mjs +75 -0
- package/scripts/perf-int.mjs +67 -0
- package/scripts/perf-multiplication.mjs +644 -0
- package/scripts/perf-poly.mjs +67 -0
- package/scripts/render-benchmark.mjs +25 -0
- package/scripts/run-tests.mjs +19 -0
- package/tests/README.md +28 -17
- package/tests/test-karatsuba.mjs +133 -0
|
@@ -1,23 +1,29 @@
|
|
|
1
1
|
# kVM Execution Model
|
|
2
2
|
|
|
3
|
-
This note sketches a low-level execution model for `k`. The goal is to
|
|
4
|
-
one abstraction exactly between the current JavaScript evaluator
|
|
5
|
-
backends such as LLVM and WebAssembly.
|
|
3
|
+
This note sketches a low-level execution model for `k`. The goal is to
|
|
4
|
+
place one abstraction exactly between the current JavaScript evaluator
|
|
5
|
+
and concrete backends such as LLVM and WebAssembly.
|
|
6
6
|
|
|
7
|
-
The working name is **kVM**. It is not primarily a bytecode format. It
|
|
8
|
-
execution contract: a small value model, a small instruction set,
|
|
9
|
-
rules for partial failure, products, unions, calls, and safe
|
|
7
|
+
The working name is **kVM**. It is not primarily a bytecode format. It
|
|
8
|
+
is an execution contract: a small value model, a small instruction set,
|
|
9
|
+
and precise rules for partial failure, products, unions, calls, and safe
|
|
10
|
+
scheduling.
|
|
10
11
|
|
|
11
12
|
## Position In The Pipeline
|
|
12
13
|
|
|
13
14
|
The current design already points at three layers:
|
|
14
15
|
|
|
15
16
|
- KIR-P: portable, polymorphic object IR.
|
|
16
|
-
- Polymorphic kVM Template: unspecialized kVM instruction streams with
|
|
17
|
-
|
|
18
|
-
-
|
|
17
|
+
- Polymorphic kVM Template: unspecialized kVM instruction streams with
|
|
18
|
+
principal pattern signatures.
|
|
19
|
+
- Envelope-Specialized kVM: kVM functions instantiated for a concrete
|
|
20
|
+
input envelope, with dead branches pruned and redundant guards
|
|
21
|
+
eliminated.
|
|
22
|
+
- Target Lowering: direct emission from specialized kVM to WebAssembly
|
|
23
|
+
or native machine code.
|
|
19
24
|
|
|
20
|
-
kVM bridges high-level relational expressions and concrete execution
|
|
25
|
+
kVM bridges high-level relational expressions and concrete execution
|
|
26
|
+
targets:
|
|
21
27
|
|
|
22
28
|
```text
|
|
23
29
|
[AOT Compilation Phase]
|
|
@@ -29,16 +35,20 @@ k source
|
|
|
29
35
|
|
|
30
36
|
[Request / JIT Phase: Enveloped Value V = (v, Pv) arrives]
|
|
31
37
|
kVM template + input envelope Pv
|
|
32
|
-
-> boundary type compatibility check (validates Pv against Pin;
|
|
33
|
-
|
|
38
|
+
-> boundary type compatibility check (validates Pv against Pin;
|
|
39
|
+
raises Type Error on contradiction)
|
|
40
|
+
-> fast kVM specialization (specializeKVM: guard folding, dead
|
|
41
|
+
branch pruning, output pattern derivation)
|
|
34
42
|
-> specialized kVM function
|
|
35
43
|
-> Tier 0: direct interpretation in kVM runtime (< 1 ms cold start)
|
|
36
|
-
-> Tier 1: single-pass lowering to WebAssembly (kvm2wasm) or native
|
|
44
|
+
-> Tier 1: single-pass lowering to WebAssembly (kvm2wasm) or native
|
|
45
|
+
machine code (~17 ms peak speed)
|
|
37
46
|
```
|
|
38
47
|
|
|
39
|
-
KIR-P remains the portable semantic object format. The kVM lowerer
|
|
40
|
-
polymorphic kVM templates ahead of time. When
|
|
41
|
-
|
|
48
|
+
KIR-P remains the portable semantic object format. The kVM lowerer
|
|
49
|
+
compiles KIR-P into polymorphic kVM templates ahead of time. When
|
|
50
|
+
concrete input envelopes arrive at runtime, `specializeKVM` instantiates
|
|
51
|
+
and optimizes the kVM instruction stream in microseconds.
|
|
42
52
|
|
|
43
53
|
## Design Center
|
|
44
54
|
|
|
@@ -53,20 +63,22 @@ where:
|
|
|
53
63
|
- `KRef` is a reference to an immutable runtime value;
|
|
54
64
|
- `KResult` is either `ok(value)` or `fail`;
|
|
55
65
|
- failure is ordinary partial-function undefinedness, not an exception;
|
|
56
|
-
- user-level computation has no mutable environment and no observable
|
|
66
|
+
- user-level computation has no mutable environment and no observable
|
|
67
|
+
effects.
|
|
57
68
|
|
|
58
69
|
This purity is central. It means kVM can expose parallel product and
|
|
59
|
-
speculative union execution while preserving exactly the same
|
|
60
|
-
semantics as the interpreter.
|
|
70
|
+
speculative union execution while preserving exactly the same
|
|
71
|
+
deterministic semantics as the interpreter.
|
|
61
72
|
|
|
62
|
-
Current debug or host operations such as `_log!` do not fit the pure
|
|
63
|
-
contract. They should either be excluded from kVM backend
|
|
64
|
-
as effectful intrinsics that create ordering
|
|
73
|
+
Current debug or host operations such as `_log!` do not fit the pure
|
|
74
|
+
backend contract. They should either be excluded from kVM backend
|
|
75
|
+
eligibility or marked as effectful intrinsics that create ordering
|
|
76
|
+
barriers.
|
|
65
77
|
|
|
66
78
|
## Value Model
|
|
67
79
|
|
|
68
|
-
A kVM value reference should be more abstract than an LLVM pointer but
|
|
69
|
-
explicit than the current JavaScript `Value` object.
|
|
80
|
+
A kVM value reference should be more abstract than an LLVM pointer but
|
|
81
|
+
more explicit than the current JavaScript `Value` object.
|
|
70
82
|
|
|
71
83
|
```text
|
|
72
84
|
KRef {
|
|
@@ -79,33 +91,35 @@ KRef {
|
|
|
79
91
|
}
|
|
80
92
|
```
|
|
81
93
|
|
|
82
|
-
The important property is immutability. A `KRef` may point into the
|
|
83
|
-
binary input stream, a materialized node, a constructed result,
|
|
84
|
-
host value. Projections can often produce new `KRef`s
|
|
85
|
-
Construction creates new immutable values.
|
|
94
|
+
The important property is immutability. A `KRef` may point into the
|
|
95
|
+
original binary input stream, a materialized node, a constructed result,
|
|
96
|
+
or an external host value. Projections can often produce new `KRef`s
|
|
97
|
+
without allocation. Construction creates new immutable values.
|
|
86
98
|
|
|
87
99
|
There are two valid execution modes:
|
|
88
100
|
|
|
89
|
-
- **Envelope-aware mode**: each `KRef` may carry a runtime pattern,
|
|
90
|
-
current interpreter and codec model.
|
|
91
|
-
- **Envelope-free mode**: envelope specialization has proven the input
|
|
92
|
-
every call site, so inner operations use
|
|
93
|
-
output pattern only at the
|
|
101
|
+
- **Envelope-aware mode**: each `KRef` may carry a runtime pattern,
|
|
102
|
+
matching the current interpreter and codec model.
|
|
103
|
+
- **Envelope-free mode**: envelope specialization has proven the input
|
|
104
|
+
and output patterns for every call site, so inner operations use
|
|
105
|
+
static layouts and attach the derived output pattern only at the
|
|
106
|
+
boundary.
|
|
94
107
|
|
|
95
|
-
The first mode is the reference path. The second mode is the main
|
|
96
|
-
performance path.
|
|
108
|
+
The first mode is the reference path. The second mode is the main
|
|
109
|
+
LLVM/Wasm performance path.
|
|
97
110
|
|
|
98
111
|
## Function Shape
|
|
99
112
|
|
|
100
|
-
A kVM relation instance is specialized by relation hash plus input
|
|
113
|
+
A kVM relation instance is specialized by relation hash plus input
|
|
114
|
+
pattern hash:
|
|
101
115
|
|
|
102
116
|
```text
|
|
103
117
|
fn @rel_hash__input_pattern_hash(%input: KRef) -> KResult<KRef>
|
|
104
118
|
```
|
|
105
119
|
|
|
106
|
-
Recursive relations compile to mutually recursive kVM functions. A
|
|
107
|
-
source relation may produce several kVM functions when called
|
|
108
|
-
input patterns.
|
|
120
|
+
Recursive relations compile to mutually recursive kVM functions. A
|
|
121
|
+
single source relation may produce several kVM functions when called
|
|
122
|
+
under different input patterns.
|
|
109
123
|
|
|
110
124
|
Each kVM function records:
|
|
111
125
|
|
|
@@ -117,13 +131,14 @@ Each kVM function records:
|
|
|
117
131
|
- called relation instances;
|
|
118
132
|
- required intrinsics.
|
|
119
133
|
|
|
120
|
-
LLVM and Wasm backends should reject functions whose relation derivation
|
|
121
|
-
`converged`, unless they intentionally target the envelope-aware
|
|
134
|
+
LLVM and Wasm backends should reject functions whose relation derivation
|
|
135
|
+
is not `converged`, unless they intentionally target the envelope-aware
|
|
136
|
+
reference mode.
|
|
122
137
|
|
|
123
138
|
## Core Instructions
|
|
124
139
|
|
|
125
|
-
kVM should be register-based and block-oriented. Backends can lower it
|
|
126
|
-
SSA, Wasm locals and blocks, C temporaries, or a JS interpreter.
|
|
140
|
+
kVM should be register-based and block-oriented. Backends can lower it
|
|
141
|
+
to LLVM SSA, Wasm locals and blocks, C temporaries, or a JS interpreter.
|
|
127
142
|
|
|
128
143
|
Primitive instructions:
|
|
129
144
|
|
|
@@ -141,24 +156,25 @@ call_intrinsic @symbol, %value -> ok(%result) | fail
|
|
|
141
156
|
return %value -> ok(%value)
|
|
142
157
|
```
|
|
143
158
|
|
|
144
|
-
Composition is ordinary control flow: run the next instruction only
|
|
145
|
-
previous one has returned `ok`.
|
|
159
|
+
Composition is ordinary control flow: run the next instruction only
|
|
160
|
+
after the previous one has returned `ok`.
|
|
146
161
|
|
|
147
|
-
Filters lower either to `guard_pattern` or disappear after envelope
|
|
148
|
-
the guard is redundant. Type/code
|
|
149
|
-
|
|
150
|
-
product region plus
|
|
162
|
+
Filters lower either to `guard_pattern` or disappear after envelope
|
|
163
|
+
specialization proves that the guard is redundant. Type/code
|
|
164
|
+
expressions lower to guards. Variant introduction lowers to
|
|
165
|
+
`make_variant`. Product construction lowers to a product region plus
|
|
166
|
+
`make_product`.
|
|
151
167
|
|
|
152
168
|
## Structured Regions
|
|
153
169
|
|
|
154
|
-
The two important high-level operations should survive lowering into kVM
|
|
155
|
-
structured regions. They carry semantic information that is easy to
|
|
156
|
-
everything is flattened too early.
|
|
170
|
+
The two important high-level operations should survive lowering into kVM
|
|
171
|
+
as structured regions. They carry semantic information that is easy to
|
|
172
|
+
lose if everything is flattened too early.
|
|
157
173
|
|
|
158
174
|
### Product Region
|
|
159
175
|
|
|
160
|
-
A product expression applies several functions to the same input and
|
|
161
|
-
only if all of them succeed.
|
|
176
|
+
A product expression applies several functions to the same input and
|
|
177
|
+
succeeds only if all of them succeed.
|
|
162
178
|
|
|
163
179
|
```text
|
|
164
180
|
%p = product %input [
|
|
@@ -170,21 +186,23 @@ only if all of them succeed.
|
|
|
170
186
|
Semantics:
|
|
171
187
|
|
|
172
188
|
1. Every field branch receives the same input `KRef`.
|
|
173
|
-
2. If every branch succeeds, the join constructs a product in canonical
|
|
174
|
-
order.
|
|
189
|
+
2. If every branch succeeds, the join constructs a product in canonical
|
|
190
|
+
field order.
|
|
175
191
|
3. If any branch fails, the whole product fails.
|
|
176
|
-
4. For terminating pure branches, branch evaluation order is not
|
|
192
|
+
4. For terminating pure branches, branch evaluation order is not
|
|
193
|
+
observable.
|
|
177
194
|
|
|
178
195
|
Scheduling:
|
|
179
196
|
|
|
180
197
|
- A sequential backend may run branches left to right.
|
|
181
198
|
- A parallel backend may fork all branches.
|
|
182
|
-
- A result-oriented runtime may fail fast and cancel unfinished branches
|
|
183
|
-
any failure, because failure has no observable payload in
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
199
|
+
- A result-oriented runtime may fail fast and cancel unfinished branches
|
|
200
|
+
after any failure, because failure has no observable payload in
|
|
201
|
+
release semantics.
|
|
202
|
+
- A strict operational runtime should preserve the chosen product
|
|
203
|
+
evaluation order for divergence-sensitive conformance. It may still
|
|
204
|
+
run later branches speculatively, but it should not expose a later
|
|
205
|
+
failure while an earlier branch is still pending.
|
|
188
206
|
- A debug runtime may optionally wait for all branches to report better
|
|
189
207
|
diagnostics.
|
|
190
208
|
|
|
@@ -192,8 +210,8 @@ This is the direct kVM form of parallel composition.
|
|
|
192
210
|
|
|
193
211
|
### Union Region
|
|
194
212
|
|
|
195
|
-
A union expression tries several functions and returns the first
|
|
196
|
-
branch in source order.
|
|
213
|
+
A union expression tries several functions and returns the first
|
|
214
|
+
successful branch in source order.
|
|
197
215
|
|
|
198
216
|
```text
|
|
199
217
|
%u = union %input [
|
|
@@ -207,42 +225,47 @@ Semantics:
|
|
|
207
225
|
|
|
208
226
|
1. Every branch receives the same input `KRef`.
|
|
209
227
|
2. If no branch succeeds, the union fails.
|
|
210
|
-
3. If one or more branches succeed, the result is the successful branch
|
|
211
|
-
the lowest index.
|
|
228
|
+
3. If one or more branches succeed, the result is the successful branch
|
|
229
|
+
with the lowest index.
|
|
212
230
|
|
|
213
231
|
Scheduling:
|
|
214
232
|
|
|
215
|
-
- A sequential backend may run branches in source order and stop at the
|
|
216
|
-
success.
|
|
233
|
+
- A sequential backend may run branches in source order and stop at the
|
|
234
|
+
first success.
|
|
217
235
|
- A parallel backend may speculatively run branches concurrently.
|
|
218
|
-
- A parallel backend may return branch `j` only after branch `j` has
|
|
219
|
-
and every branch `i < j` has failed.
|
|
220
|
-
- Once branch `j` is known to be selected, branches `i > j` may be
|
|
221
|
-
|
|
236
|
+
- A parallel backend may return branch `j` only after branch `j` has
|
|
237
|
+
succeeded and every branch `i < j` has failed.
|
|
238
|
+
- Once branch `j` is known to be selected, branches `i > j` may be
|
|
239
|
+
cancelled.
|
|
240
|
+
- If any earlier branch is still pending, a later success remains
|
|
241
|
+
provisional.
|
|
222
242
|
|
|
223
|
-
This gives concurrency without nondeterminism. The fastest branch does
|
|
224
|
-
the lowest-index successful branch wins.
|
|
243
|
+
This gives concurrency without nondeterminism. The fastest branch does
|
|
244
|
+
not win; the lowest-index successful branch wins.
|
|
225
245
|
|
|
226
246
|
## Partiality And Cancellation
|
|
227
247
|
|
|
228
|
-
kVM failure is pure and unobservable except through product and union
|
|
229
|
-
That gives simple cancellation laws:
|
|
248
|
+
kVM failure is pure and unobservable except through product and union
|
|
249
|
+
joins. That gives simple cancellation laws:
|
|
230
250
|
|
|
231
251
|
- In a product, one failure is enough to fail the product.
|
|
232
|
-
- In a union, a later success is provisional until all earlier branches
|
|
252
|
+
- In a union, a later success is provisional until all earlier branches
|
|
253
|
+
fail.
|
|
233
254
|
- Cancelling an unfinished pure branch cannot change the result.
|
|
234
|
-
- Effectful intrinsics are not cancellable unless they declare that
|
|
255
|
+
- Effectful intrinsics are not cancellable unless they declare that
|
|
256
|
+
property.
|
|
235
257
|
|
|
236
|
-
These laws are the bridge between mathematical `k` semantics and
|
|
237
|
-
schedulers.
|
|
258
|
+
These laws are the bridge between mathematical `k` semantics and
|
|
259
|
+
practical task schedulers.
|
|
238
260
|
|
|
239
|
-
There is one important caveat: purity does not by itself erase
|
|
240
|
-
backend must preserve exact operational behavior in
|
|
241
|
-
nontermination, it should use the strict scheduling
|
|
242
|
-
is always valid, and speculative parallel
|
|
243
|
-
failure that an earlier unresolved
|
|
244
|
-
contract is only result equivalence
|
|
245
|
-
fail-fast and broader cancellation
|
|
261
|
+
There is one important caveat: purity does not by itself erase
|
|
262
|
+
divergence. If a backend must preserve exact operational behavior in
|
|
263
|
+
the presence of nontermination, it should use the strict scheduling
|
|
264
|
+
profile: sequential lowering is always valid, and speculative parallel
|
|
265
|
+
work may not commit a result or failure that an earlier unresolved
|
|
266
|
+
branch could have prevented. If the contract is only result equivalence
|
|
267
|
+
for terminating computations, product fail-fast and broader cancellation
|
|
268
|
+
are valid optimizations.
|
|
246
269
|
|
|
247
270
|
## Lowering From Envelope-Specialized KIR-P
|
|
248
271
|
|
|
@@ -262,51 +285,55 @@ Suggested lowering rules:
|
|
|
262
285
|
| `product` | `product` region plus `make_product` |
|
|
263
286
|
| `union` | `union` region with `first_success_by_index` join |
|
|
264
287
|
|
|
265
|
-
The lowerer should preserve labels and tags as table ids, not strings in
|
|
266
|
-
instructions. Debug metadata may retain source labels.
|
|
288
|
+
The lowerer should preserve labels and tags as table ids, not strings in
|
|
289
|
+
hot instructions. Debug metadata may retain source labels.
|
|
267
290
|
|
|
268
291
|
## Polymorphic kVM and JIT Specialization
|
|
269
292
|
|
|
270
|
-
A `polymorphic kVM function` is an instruction stream derived from a
|
|
271
|
-
KIR-P relation before a concrete input envelope is known.
|
|
272
|
-
patterns contain pattern variables (open products,
|
|
273
|
-
variables).
|
|
293
|
+
A `polymorphic kVM function` is an instruction stream derived from a
|
|
294
|
+
polymorphic KIR-P relation before a concrete input envelope is known.
|
|
295
|
+
Its input and output patterns contain pattern variables (open products,
|
|
296
|
+
open unions, or type variables).
|
|
274
297
|
|
|
275
298
|
### Specialization Contract (`specializeKVM`)
|
|
276
299
|
|
|
277
|
-
When an `input enveloped value` $(v, P_V)$ arrives at runtime,
|
|
300
|
+
When an `input enveloped value` $(v, P_V)$ arrives at runtime,
|
|
301
|
+
`specializeKVM`:
|
|
278
302
|
|
|
279
|
-
1. **Validates Type Compatibility**:
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
2. **Substitutes Pattern Variables**:
|
|
284
|
-
|
|
285
|
-
|
|
303
|
+
1. **Validates Type Compatibility**: Checks whether $P_V$ conforms to
|
|
304
|
+
the function's principal input pattern. If $P_V$ is disjoint or
|
|
305
|
+
contradicts the input pattern, execution immediately signals a **Type
|
|
306
|
+
Error** (never partial failure) without compiling.
|
|
307
|
+
2. **Substitutes Pattern Variables**: Binds the open pattern variables
|
|
308
|
+
of the principal input pattern to the concrete subpatterns carried in
|
|
309
|
+
$P_V$, deriving the concrete output envelope $P_D$.
|
|
286
310
|
3. **Optimizes Guards and Eliminates Dead Code**:
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
311
|
+
`guard_pattern` instructions proven satisfied by $P_V$ are rewritten
|
|
312
|
+
to `id` or erased.
|
|
313
|
+
`guard_pattern` instructions proven unsatisfiable by $P_V$ are
|
|
314
|
+
rewritten to `fail`.
|
|
315
|
+
In `union` regions, branches that unconditionally fail are pruned. If
|
|
316
|
+
an earlier branch is proven to always succeed, later branches are
|
|
317
|
+
discarded.
|
|
318
|
+
4. **Binds Concrete Layouts**: Product field labels and variant tags are
|
|
319
|
+
mapped to concrete memory offsets matching $P_V$, enabling envelope-
|
|
320
|
+
free execution in lower tiers.
|
|
295
321
|
|
|
296
322
|
## Layout Tables
|
|
297
323
|
|
|
298
|
-
kVM should not hard-code one memory layout. It should refer to layout
|
|
299
|
-
generated from envelope-specialized patterns and canonical codes.
|
|
324
|
+
kVM should not hard-code one memory layout. It should refer to layout
|
|
325
|
+
tables generated from envelope-specialized patterns and canonical codes.
|
|
300
326
|
|
|
301
327
|
Useful tables:
|
|
302
328
|
|
|
303
329
|
- label table: label string -> label id;
|
|
304
330
|
- tag table: tag string -> tag id;
|
|
305
331
|
- pattern table: canonical pattern graph ids;
|
|
306
|
-
- layout table: product field offsets and union tag dispatch for each
|
|
307
|
-
pattern/layout;
|
|
332
|
+
- layout table: product field offsets and union tag dispatch for each
|
|
333
|
+
static pattern/layout;
|
|
308
334
|
- intrinsic table: pure host functions and external types;
|
|
309
|
-
- relation instance table: `relation hash + input pattern hash ->
|
|
335
|
+
- relation instance table: `relation hash + input pattern hash ->
|
|
336
|
+
function id`.
|
|
310
337
|
|
|
311
338
|
This keeps kVM stable while allowing experiments with serialized, lazy,
|
|
312
339
|
materialized, and external value representations.
|
|
@@ -315,10 +342,10 @@ materialized, and external value representations.
|
|
|
315
342
|
|
|
316
343
|
### LLVM
|
|
317
344
|
|
|
318
|
-
LLVM can lower each kVM function to a native function returning a
|
|
319
|
-
struct. kVM registers map naturally to LLVM SSA values.
|
|
320
|
-
regions can lower first to ordinary blocks, then later
|
|
321
|
-
LLVM parallelism experiments.
|
|
345
|
+
LLVM can lower each kVM function to a native function returning a
|
|
346
|
+
`KResult` struct. kVM registers map naturally to LLVM SSA values.
|
|
347
|
+
Product and union regions can lower first to ordinary blocks, then later
|
|
348
|
+
to runtime task calls or LLVM parallelism experiments.
|
|
322
349
|
|
|
323
350
|
Baseline product lowering:
|
|
324
351
|
|
|
@@ -345,19 +372,22 @@ Parallel lowering is an optimization, not a semantic requirement.
|
|
|
345
372
|
|
|
346
373
|
### WebAssembly
|
|
347
374
|
|
|
348
|
-
Wasm can use the same kVM control-flow shape with locals and blocks.
|
|
349
|
-
path should be sequential. Wasm threads, shared memory, or host
|
|
350
|
-
later implement product and union regions without changing
|
|
375
|
+
Wasm can use the same kVM control-flow shape with locals and blocks.
|
|
376
|
+
The MVP path should be sequential. Wasm threads, shared memory, or host
|
|
377
|
+
promises can later implement product and union regions without changing
|
|
378
|
+
kVM semantics.
|
|
351
379
|
|
|
352
|
-
The key Wasm constraint is that the kVM instruction set should avoid
|
|
353
|
-
arbitrary pointer arithmetic in the IR itself.
|
|
354
|
-
live behind runtime imports or explicit
|
|
380
|
+
The key Wasm constraint is that the kVM instruction set should avoid
|
|
381
|
+
relying on arbitrary pointer arithmetic in the IR itself.
|
|
382
|
+
Pointer/layout details should live behind runtime imports or explicit
|
|
383
|
+
memory-layout helpers.
|
|
355
384
|
|
|
356
385
|
### JS kVM Interpreter
|
|
357
386
|
|
|
358
|
-
A JS kVM interpreter is valuable as a conformance oracle. It can start
|
|
359
|
-
sequential product and union execution, compare results against
|
|
360
|
-
`run_converged`, then add optional parallel scheduling
|
|
387
|
+
A JS kVM interpreter is valuable as a conformance oracle. It can start
|
|
388
|
+
with sequential product and union execution, compare results against
|
|
389
|
+
`run.mjs` and `run_converged`, then add optional parallel scheduling
|
|
390
|
+
later.
|
|
361
391
|
|
|
362
392
|
## Validation Rules
|
|
363
393
|
|
|
@@ -368,10 +398,11 @@ A valid kVM function should satisfy:
|
|
|
368
398
|
3. Every failure path is explicit.
|
|
369
399
|
4. Product field labels are unique and canonical.
|
|
370
400
|
5. Union branch order is preserved.
|
|
371
|
-
6. Calls target concrete relation instances, not unspecialized source
|
|
401
|
+
6. Calls target concrete relation instances, not unspecialized source
|
|
402
|
+
names.
|
|
372
403
|
7. Layout hints match the input pattern expected by the instruction.
|
|
373
|
-
8. Envelope-free functions are allowed only when every reachable
|
|
374
|
-
instance has converged type derivation.
|
|
404
|
+
8. Envelope-free functions are allowed only when every reachable
|
|
405
|
+
relation instance has converged type derivation.
|
|
375
406
|
9. Parallel scheduling is allowed only through pure/cancellable regions.
|
|
376
407
|
|
|
377
408
|
## Example
|
|
@@ -417,21 +448,22 @@ fn @choice(%in):
|
|
|
417
448
|
return %out
|
|
418
449
|
```
|
|
419
450
|
|
|
420
|
-
The runtime may evaluate `f`, `g`, and `h` concurrently, but it may
|
|
421
|
-
only after `f` has failed.
|
|
451
|
+
The runtime may evaluate `f`, `g`, and `h` concurrently, but it may
|
|
452
|
+
return `g` only after `f` has failed.
|
|
422
453
|
|
|
423
454
|
## Open Questions
|
|
424
455
|
|
|
425
|
-
- Should the first persisted kVM encoding be textual for inspection,
|
|
426
|
-
binary for embedding, or both?
|
|
427
|
-
- Should product fail-fast be the only release behavior, or should there
|
|
428
|
-
standard diagnostic mode that collects all failed fields?
|
|
429
|
-
- How should effectful debug intrinsics be represented so they cannot
|
|
430
|
-
enter parallel backend execution?
|
|
431
|
-
- Is `guard_code` needed in the final envelope-free backend path, or
|
|
432
|
-
code/type checks be resolved into layout-specific
|
|
433
|
-
|
|
434
|
-
|
|
456
|
+
- Should the first persisted kVM encoding be textual for inspection,
|
|
457
|
+
compact binary for embedding, or both?
|
|
458
|
+
- Should product fail-fast be the only release behavior, or should there
|
|
459
|
+
be a standard diagnostic mode that collects all failed fields?
|
|
460
|
+
- How should effectful debug intrinsics be represented so they cannot
|
|
461
|
+
silently enter parallel backend execution?
|
|
462
|
+
- Is `guard_code` needed in the final envelope-free backend path, or
|
|
463
|
+
should all code/type checks be resolved into layout-specific
|
|
464
|
+
projections and guards?
|
|
465
|
+
- Should kVM expose a generic `match` instruction for union dispatch, or
|
|
466
|
+
are `project_variant` plus union regions enough?
|
|
435
467
|
|
|
436
468
|
## Implementation Order
|
|
437
469
|
|
|
@@ -439,12 +471,22 @@ only after `f` has failed.
|
|
|
439
471
|
2. Define a kVM artifact schema for specialized relation instances.
|
|
440
472
|
3. Lower KIR-P relation ops to kVM in envelope-aware mode.
|
|
441
473
|
4. Add a sequential JS kVM interpreter and compare it against `run.mjs`.
|
|
442
|
-
5. Lower envelope-specialized and converged relation instances to
|
|
474
|
+
5. Lower envelope-specialized and converged relation instances to
|
|
475
|
+
envelope-free kVM.
|
|
443
476
|
6. Add kVM validation and conformance fixtures.
|
|
444
477
|
7. Add a minimal C or Wasm backend for sequential kVM.
|
|
445
478
|
8. Add LLVM lowering.
|
|
446
479
|
9. Add optional runtime scheduling for product and union regions.
|
|
447
480
|
|
|
448
|
-
The important first milestone is not parallel speed.
|
|
449
|
-
contract that makes sequential LLVM and Wasm generation
|
|
450
|
-
the parallel structure that `k` gets from
|
|
481
|
+
The important first milestone is not parallel speed. It is a stable
|
|
482
|
+
execution contract that makes sequential LLVM and Wasm generation
|
|
483
|
+
boring, while preserving the parallel structure that `k` gets from
|
|
484
|
+
purity.
|
|
485
|
+
|
|
486
|
+
## Further Reading
|
|
487
|
+
|
|
488
|
+
- [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to .k, .ko, .klib,
|
|
489
|
+
and .kvm formats
|
|
490
|
+
- [DOCS/KIR_V1.md](./KIR_V1.md) — KIR-P inspection/export contract for
|
|
491
|
+
backends
|
|
492
|
+
- [DOCS/DICTIONARY.md](./DICTIONARY.md) — concept names and terminology
|
|
@@ -88,7 +88,22 @@ implementation changes.
|
|
|
88
88
|
|
|
89
89
|
## Object And Library Containers
|
|
90
90
|
|
|
91
|
-
|
|
91
|
+
For a comprehensive specification of all file types, see
|
|
92
|
+
[`DOCS/FILE_FORMATS.md`](./FILE_FORMATS.md).
|
|
93
|
+
|
|
94
|
+
Both `.ko` and `.klib` are ahead-of-time (AOT) type-checked artifacts.
|
|
95
|
+
In `k`, ahead-of-time type checking means parsing, relation expansion,
|
|
96
|
+
structural type derivation, and constraint convergence are performed
|
|
97
|
+
beforehand during compilation. The resulting artifact stores fully
|
|
98
|
+
converged principal pattern graphs, allowing downstream execution to
|
|
99
|
+
bypass cold-start type derivation.
|
|
100
|
+
|
|
101
|
+
All relations and types are identified canonically by immutable
|
|
102
|
+
content-addressed hashes (`@hash`). There is no global symbol namespace;
|
|
103
|
+
human-readable names are local aliases recorded in `relAlias` and
|
|
104
|
+
`typeAliases` metadata.
|
|
105
|
+
|
|
106
|
+
The executable `.ko` format is a small binary container:
|
|
92
107
|
|
|
93
108
|
```text
|
|
94
109
|
"KOBJ" 0x0a
|
|
@@ -98,9 +113,9 @@ utf8-json-payload
|
|
|
98
113
|
|
|
99
114
|
Library files (`.klib`) are plain UTF-8 JSON with no binary header.
|
|
100
115
|
|
|
101
|
-
Both containers use the same JSON payload shape. Executable `.ko` files
|
|
102
|
-
`main` to the entry relation name
|
|
103
|
-
There is no payload version field.
|
|
116
|
+
Both containers use the same JSON payload shape. Executable `.ko` files
|
|
117
|
+
set `main` to the entry relation name (usually `"__main__"`). Library
|
|
118
|
+
`.klib` files set `main` to `null`. There is no payload version field.
|
|
104
119
|
|
|
105
120
|
The payload stores:
|
|
106
121
|
|
|
@@ -116,43 +131,46 @@ The payload stores:
|
|
|
116
131
|
Relation `def` values do not store source `start` / `end` markers. Source
|
|
117
132
|
locations live on `meta[hash].origins[]` entries. Each metadata entry has
|
|
118
133
|
`type: "code"` or `type: "rel"`. Origin entries hold `source`, `name`,
|
|
119
|
-
`compiledAt`, and optional `start` / `end`; origin entries do not have
|
|
134
|
+
`compiledAt`, and optional `start` / `end`; origin entries do not have
|
|
135
|
+
`kind`.
|
|
120
136
|
|
|
121
|
-
Relation `def` values also do not store
|
|
137
|
+
Relation `def` values also do not store generated input/output filters.
|
|
122
138
|
Those filters are derived from `def.patterns` and `typePatternGraph` when
|
|
123
139
|
printing or enforcing relation boundaries.
|
|
124
140
|
|
|
125
141
|
Codes/types do not carry `typeDerivation`. Type derivation is a relation
|
|
126
|
-
property, and the persisted `typeDerivation` object currently contains
|
|
127
|
-
`status`.
|
|
142
|
+
property, and the persisted `typeDerivation` object currently contains
|
|
143
|
+
only `status`.
|
|
128
144
|
|
|
129
145
|
Loading an object hydrates the type-pattern graphs back into runtime
|
|
130
146
|
`TypePatternGraph` instances and executes relation boundaries through
|
|
131
|
-
`run.mjs`. This avoids source parsing and type derivation on the
|
|
147
|
+
`run.mjs`. This avoids source parsing and type derivation on the
|
|
148
|
+
execution path.
|
|
132
149
|
|
|
133
150
|
Object execution through `k.mjs`:
|
|
134
151
|
|
|
135
152
|
```sh
|
|
136
|
-
echo "..." | some-codec --parse |
|
|
153
|
+
echo "..." | some-codec --parse | \
|
|
154
|
+
k path/to/program.ko | some-codec --print
|
|
137
155
|
```
|
|
138
156
|
|
|
139
|
-
When the first positional argument names an existing file, `k.mjs`
|
|
140
|
-
either an object file or a source `.k` file. It tries to load
|
|
141
|
-
object first, then falls back to source parsing and
|
|
157
|
+
When the first positional argument names an existing file, `k.mjs`
|
|
158
|
+
loads it as either an object file or a source `.k` file. It tries to load
|
|
159
|
+
the file as an object first, then falls back to source parsing and
|
|
160
|
+
compilation.
|
|
142
161
|
|
|
143
162
|
Standalone helpers live in `objects/`:
|
|
144
163
|
|
|
145
164
|
```sh
|
|
146
165
|
k-compile path/to/program.k path/to/program.ko
|
|
147
166
|
k-compile path/to/library.k path/to/library.klib
|
|
148
|
-
k-compile --
|
|
149
|
-
k-compile --input-type '$ bit = <{} off, {} on>; $bit' path/to/program.k path/to/program.kvm
|
|
167
|
+
k-compile --format kvm path/to/program.k path/to/program.kvm
|
|
150
168
|
k-decompile path/to/program.ko path/to/program.decompiled.k
|
|
151
169
|
k-extract-aliases path/to/library.klib path/to/aliases.k
|
|
152
170
|
```
|
|
153
171
|
|
|
154
|
-
`k-compile` also accepts inline source snippets. With no arguments,
|
|
155
|
-
read from stdin and write to stdout:
|
|
172
|
+
`k-compile` also accepts inline source snippets. With no arguments,
|
|
173
|
+
the helpers read from stdin and write to stdout:
|
|
156
174
|
|
|
157
175
|
```sh
|
|
158
176
|
k-compile 'x = |x; x x' > path/to/program.ko
|
package/DOCS/PATTERNS.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Patterns
|
|
2
2
|
|
|
3
|
-
A pattern describes the shape of values that may flow through a k
|
|
4
|
-
Patterns are used by:
|
|
3
|
+
A pattern describes the shape of values that may flow through a k
|
|
4
|
+
expression. Patterns are used by:
|
|
5
5
|
|
|
6
6
|
- the type-derivation engine
|
|
7
7
|
- the binary codec
|
|
@@ -102,14 +102,18 @@ The active wire format is:
|
|
|
102
102
|
encode($pattern_value : $pattern) encode(value : decoded_pattern)
|
|
103
103
|
```
|
|
104
104
|
|
|
105
|
-
So the property-list array printed by `patterns/from-k.mjs` is a
|
|
106
|
-
externalization of the same pattern information carried in the
|
|
105
|
+
So the property-list array printed by `patterns/from-k.mjs` is a
|
|
106
|
+
readable externalization of the same pattern information carried in the
|
|
107
|
+
wire stream.
|
|
107
108
|
|
|
108
|
-
`k-parse --input-pattern ...` and `k-parse --input-type ...` both
|
|
109
|
-
produce the same kind of root pattern used to interpret
|
|
109
|
+
`k-parse --input-pattern ...` and `k-parse --input-type ...` both
|
|
110
|
+
ultimately produce the same kind of root pattern used to interpret
|
|
111
|
+
textual values.
|
|
110
112
|
|
|
111
113
|
## See Also
|
|
112
114
|
|
|
115
|
+
- [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to
|
|
116
|
+
.k, .ko, .klib, and .kvm formats
|
|
113
117
|
- [DOCS/TEXTUAL_VALUES.md](./TEXTUAL_VALUES.md)
|
|
114
118
|
- [codecs/README.md](../codecs/README.md)
|
|
115
119
|
- [DOCS/OBJECT_FILE_AND_PATTERN.md](./OBJECT_FILE_AND_PATTERN.md)
|