@fraczak/k 6.10.2 → 6.10.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/CHANGELOG.md +73 -0
  2. package/DOCS/BIG_PICTURE.md +174 -100
  3. package/DOCS/CODECS.md +74 -29
  4. package/DOCS/CODEC_ARCHITECTURE.md +87 -31
  5. package/DOCS/CONVERGENCE.md +42 -11
  6. package/DOCS/DICTIONARY.md +69 -2
  7. package/DOCS/FILE_FORMATS.md +420 -0
  8. package/DOCS/KIR_V1.md +53 -42
  9. package/DOCS/KVM_EXECUTION_MODEL.md +189 -147
  10. package/DOCS/OBJECT_FILE_AND_PATTERN.md +35 -17
  11. package/DOCS/PATTERNS.md +10 -6
  12. package/DOCS/REPL.md +100 -60
  13. package/DOCS/TEXTUAL_VALUES.md +24 -19
  14. package/DOCS/TYPE_DERIVATION.md +68 -22
  15. package/Examples/karatsuba-mult.k +215 -0
  16. package/README.md +92 -58
  17. package/backends/arm64/scripts/run-tests.mjs +16 -0
  18. package/backends/llvm/bin/k-llvm-build.mjs +40 -14
  19. package/backends/llvm/bin/k-llvm-compile.mjs +29 -12
  20. package/backends/llvm/runtime/krt.c +10 -10
  21. package/backends/llvm/runtime/krt.h +6 -1
  22. package/backends/llvm/scripts/run-tests.mjs +16 -0
  23. package/backends/llvm/src/cli.mjs +1 -1
  24. package/backends/llvm/src/executable.mjs +36 -0
  25. package/backends/llvm/src/kvm2llvm.mjs +10 -4
  26. package/backends/llvm/tests/integration.sh +11 -0
  27. package/backends/llvm/tests/test-llvm.mjs +15 -1
  28. package/backends/wasm/scripts/run-tests.mjs +16 -0
  29. package/browser/vfs-data.mjs +1 -1
  30. package/codecs/README.md +78 -67
  31. package/core.k +5 -5
  32. package/objects/README.md +57 -37
  33. package/package.json +2 -1
  34. package/scripts/benchmark-idempotency.mjs +27 -0
  35. package/scripts/benchmark.mjs +51 -0
  36. package/scripts/build-repl-html.mjs +17 -0
  37. package/scripts/generate-dependency-graph.mjs +25 -1
  38. package/scripts/perf-ieee.mjs +75 -0
  39. package/scripts/perf-int.mjs +67 -0
  40. package/scripts/perf-multiplication.mjs +644 -0
  41. package/scripts/perf-poly.mjs +67 -0
  42. package/scripts/render-benchmark.mjs +25 -0
  43. package/scripts/run-tests.mjs +19 -0
  44. package/tests/README.md +28 -17
  45. package/tests/test-karatsuba.mjs +133 -0
@@ -1,23 +1,29 @@
1
1
  # kVM Execution Model
2
2
 
3
- This note sketches a low-level execution model for `k`. The goal is to place
4
- one abstraction exactly between the current JavaScript evaluator and concrete
5
- backends such as LLVM and WebAssembly.
3
+ This note sketches a low-level execution model for `k`. The goal is to
4
+ place one abstraction exactly between the current JavaScript evaluator
5
+ and concrete backends such as LLVM and WebAssembly.
6
6
 
7
- The working name is **kVM**. It is not primarily a bytecode format. It is an
8
- execution contract: a small value model, a small instruction set, and precise
9
- rules for partial failure, products, unions, calls, and safe scheduling.
7
+ The working name is **kVM**. It is not primarily a bytecode format. It
8
+ is an execution contract: a small value model, a small instruction set,
9
+ and precise rules for partial failure, products, unions, calls, and safe
10
+ scheduling.
10
11
 
11
12
  ## Position In The Pipeline
12
13
 
13
14
  The current design already points at three layers:
14
15
 
15
16
  - KIR-P: portable, polymorphic object IR.
16
- - Polymorphic kVM Template: unspecialized kVM instruction streams with principal pattern signatures.
17
- - Envelope-Specialized kVM: kVM functions instantiated for a concrete input envelope, with dead branches pruned and redundant guards eliminated.
18
- - Target Lowering: direct emission from specialized kVM to WebAssembly or native machine code.
17
+ - Polymorphic kVM Template: unspecialized kVM instruction streams with
18
+ principal pattern signatures.
19
+ - Envelope-Specialized kVM: kVM functions instantiated for a concrete
20
+ input envelope, with dead branches pruned and redundant guards
21
+ eliminated.
22
+ - Target Lowering: direct emission from specialized kVM to WebAssembly
23
+ or native machine code.
19
24
 
20
- kVM bridges high-level relational expressions and concrete execution targets:
25
+ kVM bridges high-level relational expressions and concrete execution
26
+ targets:
21
27
 
22
28
  ```text
23
29
  [AOT Compilation Phase]
@@ -29,16 +35,20 @@ k source
29
35
 
30
36
  [Request / JIT Phase: Enveloped Value V = (v, Pv) arrives]
31
37
  kVM template + input envelope Pv
32
- -> boundary type compatibility check (validates Pv against Pin; raises Type Error on contradiction)
33
- -> fast kVM specialization (specializeKVM: guard folding, dead branch pruning, output pattern derivation)
38
+ -> boundary type compatibility check (validates Pv against Pin;
39
+ raises Type Error on contradiction)
40
+ -> fast kVM specialization (specializeKVM: guard folding, dead
41
+ branch pruning, output pattern derivation)
34
42
  -> specialized kVM function
35
43
  -> Tier 0: direct interpretation in kVM runtime (< 1 ms cold start)
36
- -> Tier 1: single-pass lowering to WebAssembly (kvm2wasm) or native machine code (~17 ms peak speed)
44
+ -> Tier 1: single-pass lowering to WebAssembly (kvm2wasm) or native
45
+ machine code (~17 ms peak speed)
37
46
  ```
38
47
 
39
- KIR-P remains the portable semantic object format. The kVM lowerer compiles KIR-P into
40
- polymorphic kVM templates ahead of time. When concrete input envelopes arrive at runtime,
41
- `specializeKVM` instantiates and optimizes the kVM instruction stream in microseconds.
48
+ KIR-P remains the portable semantic object format. The kVM lowerer
49
+ compiles KIR-P into polymorphic kVM templates ahead of time. When
50
+ concrete input envelopes arrive at runtime, `specializeKVM` instantiates
51
+ and optimizes the kVM instruction stream in microseconds.
42
52
 
43
53
  ## Design Center
44
54
 
@@ -53,20 +63,22 @@ where:
53
63
  - `KRef` is a reference to an immutable runtime value;
54
64
  - `KResult` is either `ok(value)` or `fail`;
55
65
  - failure is ordinary partial-function undefinedness, not an exception;
56
- - user-level computation has no mutable environment and no observable effects.
66
+ - user-level computation has no mutable environment and no observable
67
+ effects.
57
68
 
58
69
  This purity is central. It means kVM can expose parallel product and
59
- speculative union execution while preserving exactly the same deterministic
60
- semantics as the interpreter.
70
+ speculative union execution while preserving exactly the same
71
+ deterministic semantics as the interpreter.
61
72
 
62
- Current debug or host operations such as `_log!` do not fit the pure backend
63
- contract. They should either be excluded from kVM backend eligibility or marked
64
- as effectful intrinsics that create ordering barriers.
73
+ Current debug or host operations such as `_log!` do not fit the pure
74
+ backend contract. They should either be excluded from kVM backend
75
+ eligibility or marked as effectful intrinsics that create ordering
76
+ barriers.
65
77
 
66
78
  ## Value Model
67
79
 
68
- A kVM value reference should be more abstract than an LLVM pointer but more
69
- explicit than the current JavaScript `Value` object.
80
+ A kVM value reference should be more abstract than an LLVM pointer but
81
+ more explicit than the current JavaScript `Value` object.
70
82
 
71
83
  ```text
72
84
  KRef {
@@ -79,33 +91,35 @@ KRef {
79
91
  }
80
92
  ```
81
93
 
82
- The important property is immutability. A `KRef` may point into the original
83
- binary input stream, a materialized node, a constructed result, or an external
84
- host value. Projections can often produce new `KRef`s without allocation.
85
- Construction creates new immutable values.
94
+ The important property is immutability. A `KRef` may point into the
95
+ original binary input stream, a materialized node, a constructed result,
96
+ or an external host value. Projections can often produce new `KRef`s
97
+ without allocation. Construction creates new immutable values.
86
98
 
87
99
  There are two valid execution modes:
88
100
 
89
- - **Envelope-aware mode**: each `KRef` may carry a runtime pattern, matching the
90
- current interpreter and codec model.
91
- - **Envelope-free mode**: envelope specialization has proven the input and output patterns for
92
- every call site, so inner operations use static layouts and attach the derived
93
- output pattern only at the boundary.
101
+ - **Envelope-aware mode**: each `KRef` may carry a runtime pattern,
102
+ matching the current interpreter and codec model.
103
+ - **Envelope-free mode**: envelope specialization has proven the input
104
+ and output patterns for every call site, so inner operations use
105
+ static layouts and attach the derived output pattern only at the
106
+ boundary.
94
107
 
95
- The first mode is the reference path. The second mode is the main LLVM/Wasm
96
- performance path.
108
+ The first mode is the reference path. The second mode is the main
109
+ LLVM/Wasm performance path.
97
110
 
98
111
  ## Function Shape
99
112
 
100
- A kVM relation instance is specialized by relation hash plus input pattern hash:
113
+ A kVM relation instance is specialized by relation hash plus input
114
+ pattern hash:
101
115
 
102
116
  ```text
103
117
  fn @rel_hash__input_pattern_hash(%input: KRef) -> KResult<KRef>
104
118
  ```
105
119
 
106
- Recursive relations compile to mutually recursive kVM functions. A single
107
- source relation may produce several kVM functions when called under different
108
- input patterns.
120
+ Recursive relations compile to mutually recursive kVM functions. A
121
+ single source relation may produce several kVM functions when called
122
+ under different input patterns.
109
123
 
110
124
  Each kVM function records:
111
125
 
@@ -117,13 +131,14 @@ Each kVM function records:
117
131
  - called relation instances;
118
132
  - required intrinsics.
119
133
 
120
- LLVM and Wasm backends should reject functions whose relation derivation is not
121
- `converged`, unless they intentionally target the envelope-aware reference mode.
134
+ LLVM and Wasm backends should reject functions whose relation derivation
135
+ is not `converged`, unless they intentionally target the envelope-aware
136
+ reference mode.
122
137
 
123
138
  ## Core Instructions
124
139
 
125
- kVM should be register-based and block-oriented. Backends can lower it to LLVM
126
- SSA, Wasm locals and blocks, C temporaries, or a JS interpreter.
140
+ kVM should be register-based and block-oriented. Backends can lower it
141
+ to LLVM SSA, Wasm locals and blocks, C temporaries, or a JS interpreter.
127
142
 
128
143
  Primitive instructions:
129
144
 
@@ -141,24 +156,25 @@ call_intrinsic @symbol, %value -> ok(%result) | fail
141
156
  return %value -> ok(%value)
142
157
  ```
143
158
 
144
- Composition is ordinary control flow: run the next instruction only after the
145
- previous one has returned `ok`.
159
+ Composition is ordinary control flow: run the next instruction only
160
+ after the previous one has returned `ok`.
146
161
 
147
- Filters lower either to `guard_pattern` or disappear after envelope specialization proves that
148
- the guard is redundant. Type/code expressions lower to guards. Variant
149
- introduction lowers to `make_variant`. Product construction lowers to a
150
- product region plus `make_product`.
162
+ Filters lower either to `guard_pattern` or disappear after envelope
163
+ specialization proves that the guard is redundant. Type/code
164
+ expressions lower to guards. Variant introduction lowers to
165
+ `make_variant`. Product construction lowers to a product region plus
166
+ `make_product`.
151
167
 
152
168
  ## Structured Regions
153
169
 
154
- The two important high-level operations should survive lowering into kVM as
155
- structured regions. They carry semantic information that is easy to lose if
156
- everything is flattened too early.
170
+ The two important high-level operations should survive lowering into kVM
171
+ as structured regions. They carry semantic information that is easy to
172
+ lose if everything is flattened too early.
157
173
 
158
174
  ### Product Region
159
175
 
160
- A product expression applies several functions to the same input and succeeds
161
- only if all of them succeed.
176
+ A product expression applies several functions to the same input and
177
+ succeeds only if all of them succeed.
162
178
 
163
179
  ```text
164
180
  %p = product %input [
@@ -170,21 +186,23 @@ only if all of them succeed.
170
186
  Semantics:
171
187
 
172
188
  1. Every field branch receives the same input `KRef`.
173
- 2. If every branch succeeds, the join constructs a product in canonical field
174
- order.
189
+ 2. If every branch succeeds, the join constructs a product in canonical
190
+ field order.
175
191
  3. If any branch fails, the whole product fails.
176
- 4. For terminating pure branches, branch evaluation order is not observable.
192
+ 4. For terminating pure branches, branch evaluation order is not
193
+ observable.
177
194
 
178
195
  Scheduling:
179
196
 
180
197
  - A sequential backend may run branches left to right.
181
198
  - A parallel backend may fork all branches.
182
- - A result-oriented runtime may fail fast and cancel unfinished branches after
183
- any failure, because failure has no observable payload in release semantics.
184
- - A strict operational runtime should preserve the chosen product evaluation
185
- order for divergence-sensitive conformance. It may still run later branches
186
- speculatively, but it should not expose a later failure while an earlier
187
- branch is still pending.
199
+ - A result-oriented runtime may fail fast and cancel unfinished branches
200
+ after any failure, because failure has no observable payload in
201
+ release semantics.
202
+ - A strict operational runtime should preserve the chosen product
203
+ evaluation order for divergence-sensitive conformance. It may still
204
+ run later branches speculatively, but it should not expose a later
205
+ failure while an earlier branch is still pending.
188
206
  - A debug runtime may optionally wait for all branches to report better
189
207
  diagnostics.
190
208
 
@@ -192,8 +210,8 @@ This is the direct kVM form of parallel composition.
192
210
 
193
211
  ### Union Region
194
212
 
195
- A union expression tries several functions and returns the first successful
196
- branch in source order.
213
+ A union expression tries several functions and returns the first
214
+ successful branch in source order.
197
215
 
198
216
  ```text
199
217
  %u = union %input [
@@ -207,42 +225,47 @@ Semantics:
207
225
 
208
226
  1. Every branch receives the same input `KRef`.
209
227
  2. If no branch succeeds, the union fails.
210
- 3. If one or more branches succeed, the result is the successful branch with
211
- the lowest index.
228
+ 3. If one or more branches succeed, the result is the successful branch
229
+ with the lowest index.
212
230
 
213
231
  Scheduling:
214
232
 
215
- - A sequential backend may run branches in source order and stop at the first
216
- success.
233
+ - A sequential backend may run branches in source order and stop at the
234
+ first success.
217
235
  - A parallel backend may speculatively run branches concurrently.
218
- - A parallel backend may return branch `j` only after branch `j` has succeeded
219
- and every branch `i < j` has failed.
220
- - Once branch `j` is known to be selected, branches `i > j` may be cancelled.
221
- - If any earlier branch is still pending, a later success remains provisional.
236
+ - A parallel backend may return branch `j` only after branch `j` has
237
+ succeeded and every branch `i < j` has failed.
238
+ - Once branch `j` is known to be selected, branches `i > j` may be
239
+ cancelled.
240
+ - If any earlier branch is still pending, a later success remains
241
+ provisional.
222
242
 
223
- This gives concurrency without nondeterminism. The fastest branch does not win;
224
- the lowest-index successful branch wins.
243
+ This gives concurrency without nondeterminism. The fastest branch does
244
+ not win; the lowest-index successful branch wins.
225
245
 
226
246
  ## Partiality And Cancellation
227
247
 
228
- kVM failure is pure and unobservable except through product and union joins.
229
- That gives simple cancellation laws:
248
+ kVM failure is pure and unobservable except through product and union
249
+ joins. That gives simple cancellation laws:
230
250
 
231
251
  - In a product, one failure is enough to fail the product.
232
- - In a union, a later success is provisional until all earlier branches fail.
252
+ - In a union, a later success is provisional until all earlier branches
253
+ fail.
233
254
  - Cancelling an unfinished pure branch cannot change the result.
234
- - Effectful intrinsics are not cancellable unless they declare that property.
255
+ - Effectful intrinsics are not cancellable unless they declare that
256
+ property.
235
257
 
236
- These laws are the bridge between mathematical `k` semantics and practical task
237
- schedulers.
258
+ These laws are the bridge between mathematical `k` semantics and
259
+ practical task schedulers.
238
260
 
239
- There is one important caveat: purity does not by itself erase divergence. If a
240
- backend must preserve exact operational behavior in the presence of
241
- nontermination, it should use the strict scheduling profile: sequential lowering
242
- is always valid, and speculative parallel work may not commit a result or
243
- failure that an earlier unresolved branch could have prevented. If the
244
- contract is only result equivalence for terminating computations, product
245
- fail-fast and broader cancellation are valid optimizations.
261
+ There is one important caveat: purity does not by itself erase
262
+ divergence. If a backend must preserve exact operational behavior in
263
+ the presence of nontermination, it should use the strict scheduling
264
+ profile: sequential lowering is always valid, and speculative parallel
265
+ work may not commit a result or failure that an earlier unresolved
266
+ branch could have prevented. If the contract is only result equivalence
267
+ for terminating computations, product fail-fast and broader cancellation
268
+ are valid optimizations.
246
269
 
247
270
  ## Lowering From Envelope-Specialized KIR-P
248
271
 
@@ -262,51 +285,55 @@ Suggested lowering rules:
262
285
  | `product` | `product` region plus `make_product` |
263
286
  | `union` | `union` region with `first_success_by_index` join |
264
287
 
265
- The lowerer should preserve labels and tags as table ids, not strings in hot
266
- instructions. Debug metadata may retain source labels.
288
+ The lowerer should preserve labels and tags as table ids, not strings in
289
+ hot instructions. Debug metadata may retain source labels.
267
290
 
268
291
  ## Polymorphic kVM and JIT Specialization
269
292
 
270
- A `polymorphic kVM function` is an instruction stream derived from a polymorphic
271
- KIR-P relation before a concrete input envelope is known. Its input and output
272
- patterns contain pattern variables (open products, open unions, or type
273
- variables).
293
+ A `polymorphic kVM function` is an instruction stream derived from a
294
+ polymorphic KIR-P relation before a concrete input envelope is known.
295
+ Its input and output patterns contain pattern variables (open products,
296
+ open unions, or type variables).
274
297
 
275
298
  ### Specialization Contract (`specializeKVM`)
276
299
 
277
- When an `input enveloped value` $(v, P_V)$ arrives at runtime, `specializeKVM`:
300
+ When an `input enveloped value` $(v, P_V)$ arrives at runtime,
301
+ `specializeKVM`:
278
302
 
279
- 1. **Validates Type Compatibility**:
280
- Checks whether $P_V$ conforms to the function's principal input pattern.
281
- If $P_V$ is disjoint or contradicts the input pattern, execution immediately
282
- signals a **Type Error** (never partial failure) without compiling.
283
- 2. **Substitutes Pattern Variables**:
284
- Binds the open pattern variables of the principal input pattern to the concrete
285
- subpatterns carried in $P_V$, deriving the concrete output envelope $P_D$.
303
+ 1. **Validates Type Compatibility**: Checks whether $P_V$ conforms to
304
+ the function's principal input pattern. If $P_V$ is disjoint or
305
+ contradicts the input pattern, execution immediately signals a **Type
306
+ Error** (never partial failure) without compiling.
307
+ 2. **Substitutes Pattern Variables**: Binds the open pattern variables
308
+ of the principal input pattern to the concrete subpatterns carried in
309
+ $P_V$, deriving the concrete output envelope $P_D$.
286
310
  3. **Optimizes Guards and Eliminates Dead Code**:
287
- - `guard_pattern` instructions proven satisfied by $P_V$ are rewritten to `id`
288
- or erased.
289
- - `guard_pattern` instructions proven unsatisfiable by $P_V$ are rewritten to `fail`.
290
- - In `union` regions, branches that unconditionally fail are pruned. If an
291
- earlier branch is proven to always succeed, later branches are discarded.
292
- 4. **Binds Concrete Layouts**:
293
- Product field labels and variant tags are mapped to concrete memory offsets
294
- matching $P_V$, enabling envelope-free execution in lower tiers.
311
+ `guard_pattern` instructions proven satisfied by $P_V$ are rewritten
312
+ to `id` or erased.
313
+ `guard_pattern` instructions proven unsatisfiable by $P_V$ are
314
+ rewritten to `fail`.
315
+ In `union` regions, branches that unconditionally fail are pruned. If
316
+ an earlier branch is proven to always succeed, later branches are
317
+ discarded.
318
+ 4. **Binds Concrete Layouts**: Product field labels and variant tags are
319
+ mapped to concrete memory offsets matching $P_V$, enabling envelope-
320
+ free execution in lower tiers.
295
321
 
296
322
  ## Layout Tables
297
323
 
298
- kVM should not hard-code one memory layout. It should refer to layout tables
299
- generated from envelope-specialized patterns and canonical codes.
324
+ kVM should not hard-code one memory layout. It should refer to layout
325
+ tables generated from envelope-specialized patterns and canonical codes.
300
326
 
301
327
  Useful tables:
302
328
 
303
329
  - label table: label string -> label id;
304
330
  - tag table: tag string -> tag id;
305
331
  - pattern table: canonical pattern graph ids;
306
- - layout table: product field offsets and union tag dispatch for each static
307
- pattern/layout;
332
+ - layout table: product field offsets and union tag dispatch for each
333
+ static pattern/layout;
308
334
  - intrinsic table: pure host functions and external types;
309
- - relation instance table: `relation hash + input pattern hash -> function id`.
335
+ - relation instance table: `relation hash + input pattern hash ->
336
+ function id`.
310
337
 
311
338
  This keeps kVM stable while allowing experiments with serialized, lazy,
312
339
  materialized, and external value representations.
@@ -315,10 +342,10 @@ materialized, and external value representations.
315
342
 
316
343
  ### LLVM
317
344
 
318
- LLVM can lower each kVM function to a native function returning a `KResult`
319
- struct. kVM registers map naturally to LLVM SSA values. Product and union
320
- regions can lower first to ordinary blocks, then later to runtime task calls or
321
- LLVM parallelism experiments.
345
+ LLVM can lower each kVM function to a native function returning a
346
+ `KResult` struct. kVM registers map naturally to LLVM SSA values.
347
+ Product and union regions can lower first to ordinary blocks, then later
348
+ to runtime task calls or LLVM parallelism experiments.
322
349
 
323
350
  Baseline product lowering:
324
351
 
@@ -345,19 +372,22 @@ Parallel lowering is an optimization, not a semantic requirement.
345
372
 
346
373
  ### WebAssembly
347
374
 
348
- Wasm can use the same kVM control-flow shape with locals and blocks. The MVP
349
- path should be sequential. Wasm threads, shared memory, or host promises can
350
- later implement product and union regions without changing kVM semantics.
375
+ Wasm can use the same kVM control-flow shape with locals and blocks.
376
+ The MVP path should be sequential. Wasm threads, shared memory, or host
377
+ promises can later implement product and union regions without changing
378
+ kVM semantics.
351
379
 
352
- The key Wasm constraint is that the kVM instruction set should avoid relying on
353
- arbitrary pointer arithmetic in the IR itself. Pointer/layout details should
354
- live behind runtime imports or explicit memory-layout helpers.
380
+ The key Wasm constraint is that the kVM instruction set should avoid
381
+ relying on arbitrary pointer arithmetic in the IR itself.
382
+ Pointer/layout details should live behind runtime imports or explicit
383
+ memory-layout helpers.
355
384
 
356
385
  ### JS kVM Interpreter
357
386
 
358
- A JS kVM interpreter is valuable as a conformance oracle. It can start with
359
- sequential product and union execution, compare results against `run.mjs` and
360
- `run_converged`, then add optional parallel scheduling later.
387
+ A JS kVM interpreter is valuable as a conformance oracle. It can start
388
+ with sequential product and union execution, compare results against
389
+ `run.mjs` and `run_converged`, then add optional parallel scheduling
390
+ later.
361
391
 
362
392
  ## Validation Rules
363
393
 
@@ -368,10 +398,11 @@ A valid kVM function should satisfy:
368
398
  3. Every failure path is explicit.
369
399
  4. Product field labels are unique and canonical.
370
400
  5. Union branch order is preserved.
371
- 6. Calls target concrete relation instances, not unspecialized source names.
401
+ 6. Calls target concrete relation instances, not unspecialized source
402
+ names.
372
403
  7. Layout hints match the input pattern expected by the instruction.
373
- 8. Envelope-free functions are allowed only when every reachable relation
374
- instance has converged type derivation.
404
+ 8. Envelope-free functions are allowed only when every reachable
405
+ relation instance has converged type derivation.
375
406
  9. Parallel scheduling is allowed only through pure/cancellable regions.
376
407
 
377
408
  ## Example
@@ -417,21 +448,22 @@ fn @choice(%in):
417
448
  return %out
418
449
  ```
419
450
 
420
- The runtime may evaluate `f`, `g`, and `h` concurrently, but it may return `g`
421
- only after `f` has failed.
451
+ The runtime may evaluate `f`, `g`, and `h` concurrently, but it may
452
+ return `g` only after `f` has failed.
422
453
 
423
454
  ## Open Questions
424
455
 
425
- - Should the first persisted kVM encoding be textual for inspection, compact
426
- binary for embedding, or both?
427
- - Should product fail-fast be the only release behavior, or should there be a
428
- standard diagnostic mode that collects all failed fields?
429
- - How should effectful debug intrinsics be represented so they cannot silently
430
- enter parallel backend execution?
431
- - Is `guard_code` needed in the final envelope-free backend path, or should all
432
- code/type checks be resolved into layout-specific projections and guards?
433
- - Should kVM expose a generic `match` instruction for union dispatch, or are
434
- `project_variant` plus union regions enough?
456
+ - Should the first persisted kVM encoding be textual for inspection,
457
+ compact binary for embedding, or both?
458
+ - Should product fail-fast be the only release behavior, or should there
459
+ be a standard diagnostic mode that collects all failed fields?
460
+ - How should effectful debug intrinsics be represented so they cannot
461
+ silently enter parallel backend execution?
462
+ - Is `guard_code` needed in the final envelope-free backend path, or
463
+ should all code/type checks be resolved into layout-specific
464
+ projections and guards?
465
+ - Should kVM expose a generic `match` instruction for union dispatch, or
466
+ are `project_variant` plus union regions enough?
435
467
 
436
468
  ## Implementation Order
437
469
 
@@ -439,12 +471,22 @@ only after `f` has failed.
439
471
  2. Define a kVM artifact schema for specialized relation instances.
440
472
  3. Lower KIR-P relation ops to kVM in envelope-aware mode.
441
473
  4. Add a sequential JS kVM interpreter and compare it against `run.mjs`.
442
- 5. Lower envelope-specialized and converged relation instances to envelope-free kVM.
474
+ 5. Lower envelope-specialized and converged relation instances to
475
+ envelope-free kVM.
443
476
  6. Add kVM validation and conformance fixtures.
444
477
  7. Add a minimal C or Wasm backend for sequential kVM.
445
478
  8. Add LLVM lowering.
446
479
  9. Add optional runtime scheduling for product and union regions.
447
480
 
448
- The important first milestone is not parallel speed. It is a stable execution
449
- contract that makes sequential LLVM and Wasm generation boring, while preserving
450
- the parallel structure that `k` gets from purity.
481
+ The important first milestone is not parallel speed. It is a stable
482
+ execution contract that makes sequential LLVM and Wasm generation
483
+ boring, while preserving the parallel structure that `k` gets from
484
+ purity.
485
+
486
+ ## Further Reading
487
+
488
+ - [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to .k, .ko, .klib,
489
+ and .kvm formats
490
+ - [DOCS/KIR_V1.md](./KIR_V1.md) — KIR-P inspection/export contract for
491
+ backends
492
+ - [DOCS/DICTIONARY.md](./DICTIONARY.md) — concept names and terminology
@@ -88,7 +88,22 @@ implementation changes.
88
88
 
89
89
  ## Object And Library Containers
90
90
 
91
- The current executable `.ko` format is a small binary container:
91
+ For a comprehensive specification of all file types, see
92
+ [`DOCS/FILE_FORMATS.md`](./FILE_FORMATS.md).
93
+
94
+ Both `.ko` and `.klib` are ahead-of-time (AOT) type-checked artifacts.
95
+ In `k`, ahead-of-time type checking means parsing, relation expansion,
96
+ structural type derivation, and constraint convergence are performed
97
+ beforehand during compilation. The resulting artifact stores fully
98
+ converged principal pattern graphs, allowing downstream execution to
99
+ bypass cold-start type derivation.
100
+
101
+ All relations and types are identified canonically by immutable
102
+ content-addressed hashes (`@hash`). There is no global symbol namespace;
103
+ human-readable names are local aliases recorded in `relAlias` and
104
+ `typeAliases` metadata.
105
+
106
+ The executable `.ko` format is a small binary container:
92
107
 
93
108
  ```text
94
109
  "KOBJ" 0x0a
@@ -98,9 +113,9 @@ utf8-json-payload
98
113
 
99
114
  Library files (`.klib`) are plain UTF-8 JSON with no binary header.
100
115
 
101
- Both containers use the same JSON payload shape. Executable `.ko` files set
102
- `main` to the entry relation name. Library `.klib` files set `main` to `null`.
103
- There is no payload version field.
116
+ Both containers use the same JSON payload shape. Executable `.ko` files
117
+ set `main` to the entry relation name (usually `"__main__"`). Library
118
+ `.klib` files set `main` to `null`. There is no payload version field.
104
119
 
105
120
  The payload stores:
106
121
 
@@ -116,43 +131,46 @@ The payload stores:
116
131
  Relation `def` values do not store source `start` / `end` markers. Source
117
132
  locations live on `meta[hash].origins[]` entries. Each metadata entry has
118
133
  `type: "code"` or `type: "rel"`. Origin entries hold `source`, `name`,
119
- `compiledAt`, and optional `start` / `end`; origin entries do not have `kind`.
134
+ `compiledAt`, and optional `start` / `end`; origin entries do not have
135
+ `kind`.
120
136
 
121
- Relation `def` values also do not store the generated input/output filters.
137
+ Relation `def` values also do not store generated input/output filters.
122
138
  Those filters are derived from `def.patterns` and `typePatternGraph` when
123
139
  printing or enforcing relation boundaries.
124
140
 
125
141
  Codes/types do not carry `typeDerivation`. Type derivation is a relation
126
- property, and the persisted `typeDerivation` object currently contains only
127
- `status`.
142
+ property, and the persisted `typeDerivation` object currently contains
143
+ only `status`.
128
144
 
129
145
  Loading an object hydrates the type-pattern graphs back into runtime
130
146
  `TypePatternGraph` instances and executes relation boundaries through
131
- `run.mjs`. This avoids source parsing and type derivation on the execution path.
147
+ `run.mjs`. This avoids source parsing and type derivation on the
148
+ execution path.
132
149
 
133
150
  Object execution through `k.mjs`:
134
151
 
135
152
  ```sh
136
- echo "..." | some-codec --parse | k path/to/program.ko | some-codec --print
153
+ echo "..." | some-codec --parse | \
154
+ k path/to/program.ko | some-codec --print
137
155
  ```
138
156
 
139
- When the first positional argument names an existing file, `k.mjs` loads it as
140
- either an object file or a source `.k` file. It tries to load the file as an
141
- object first, then falls back to source parsing and compilation.
157
+ When the first positional argument names an existing file, `k.mjs`
158
+ loads it as either an object file or a source `.k` file. It tries to load
159
+ the file as an object first, then falls back to source parsing and
160
+ compilation.
142
161
 
143
162
  Standalone helpers live in `objects/`:
144
163
 
145
164
  ```sh
146
165
  k-compile path/to/program.k path/to/program.ko
147
166
  k-compile path/to/library.k path/to/library.klib
148
- k-compile --input-pattern '[["closed-product",[]]]' path/to/program.k path/to/program.kvm
149
- k-compile --input-type '$ bit = <{} off, {} on>; $bit' path/to/program.k path/to/program.kvm
167
+ k-compile --format kvm path/to/program.k path/to/program.kvm
150
168
  k-decompile path/to/program.ko path/to/program.decompiled.k
151
169
  k-extract-aliases path/to/library.klib path/to/aliases.k
152
170
  ```
153
171
 
154
- `k-compile` also accepts inline source snippets. With no arguments, the helpers
155
- read from stdin and write to stdout:
172
+ `k-compile` also accepts inline source snippets. With no arguments,
173
+ the helpers read from stdin and write to stdout:
156
174
 
157
175
  ```sh
158
176
  k-compile 'x = |x; x x' > path/to/program.ko
package/DOCS/PATTERNS.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # Patterns
2
2
 
3
- A pattern describes the shape of values that may flow through a k expression.
4
- Patterns are used by:
3
+ A pattern describes the shape of values that may flow through a k
4
+ expression. Patterns are used by:
5
5
 
6
6
  - the type-derivation engine
7
7
  - the binary codec
@@ -102,14 +102,18 @@ The active wire format is:
102
102
  encode($pattern_value : $pattern) encode(value : decoded_pattern)
103
103
  ```
104
104
 
105
- So the property-list array printed by `patterns/from-k.mjs` is a readable
106
- externalization of the same pattern information carried in the wire stream.
105
+ So the property-list array printed by `patterns/from-k.mjs` is a
106
+ readable externalization of the same pattern information carried in the
107
+ wire stream.
107
108
 
108
- `k-parse --input-pattern ...` and `k-parse --input-type ...` both ultimately
109
- produce the same kind of root pattern used to interpret textual values.
109
+ `k-parse --input-pattern ...` and `k-parse --input-type ...` both
110
+ ultimately produce the same kind of root pattern used to interpret
111
+ textual values.
110
112
 
111
113
  ## See Also
112
114
 
115
+ - [DOCS/FILE_FORMATS.md](./FILE_FORMATS.md) — guide to
116
+ .k, .ko, .klib, and .kvm formats
113
117
  - [DOCS/TEXTUAL_VALUES.md](./TEXTUAL_VALUES.md)
114
118
  - [codecs/README.md](../codecs/README.md)
115
119
  - [DOCS/OBJECT_FILE_AND_PATTERN.md](./OBJECT_FILE_AND_PATTERN.md)