@fraczak/k 6.0.1 → 6.0.3

package/DOCS/book/07-the-partial-function-abi.md

@@ -0,0 +1,143 @@
+# Chapter 7 — The Partial Function ABI
+
+## **7.1  Purpose**
+
+The **application binary interface (ABI)** defines how partial functions in `k` are represented and invoked at runtime.
+Its goal is to make all functions - whether user-defined or compiled - compatible with the same calling convention and data layout.
+
+Every compiled `k` function receives a single argument (a runtime value) and may either produce a result or remain undefined.
+The ABI provides a uniform way to express both outcomes.
+
+Conceptually, a runtime value is a materialized product/union tree plus its
+current pattern context. The older bare-node ABI below describes the tree part;
+a pattern-aware ABI must carry the pattern beside the node pointer.
+
+---
+
+## **7.2  Function result representation**
+
+The result of every function call is a pair of fields:
+
+```c
+struct KOpt {
+    bool ok;     // 1 if function is defined for the given input
+    struct KNode* val;  // valid only if ok == true
+    struct KPattern* pattern;  // optional root pattern for val
+};
+```
+
+If `ok == 1`, the function succeeded and `val` points to the root node of the resulting value.
+If `ok == 0`, the function is undefined for the input, and `val` is meaningless.
+
+This structure carries both the success flag and the value pointer, allowing partiality to be expressed directly in generated code.
+
+---
+
+## **7.3  Function parameter representation**
+
+Each function takes exactly one parameter:
+
+```c
+struct KValue* input;
+```
+
+The parameter points to the input runtime value, including both the root node
+and any carried pattern.
+Since all values are immutable, the function must not modify this structure.
+
+---
+
+## **7.4  Calling convention**
+
+Every `k` function compiled to machine code follows this C-style signature:
+
+```c
+struct KOpt k_function(struct KValue* input);
+```
+
+The return value indicates success or failure.
+The convention is uniform for all functions, regardless of the specific types involved.
+
+---
+
+## **7.5  Core runtime operations**
+
+The runtime provides a small set of primitive functions implementing the fundamental operations of `k`:
+
+| Operation                              | Purpose                       | Result                                            |
+| -------------------------------------- | ----------------------------- | ------------------------------------------------- |
+| `k_project(input, label_id)`           | projection `.label` or `/label` | defined if the field or variant `label_id` exists; result carries the selected subpattern when available |
+| `k_make_product(children[], labels[], n)` | construct a product value      | always defined; result carries a closed product pattern when child patterns are known |
+| `k_make_union(tag, child)`             | construct a union value        | always defined; result carries a closed union pattern for `tag` when the child pattern is known |
+| `k_fail()`                             | represent undefined result    | returns `{ ok = 0 }`                              |
+| `k_unit()`                             | return the constant unit node | returns `{ ok = 1, val = &unit_node }`            |
+
+All user-defined functions can be compiled entirely in terms of these operations.
+
+---
+
+## **7.6  Metadata tables**
+
+To interpret or construct nodes correctly, the runtime holds constant tables derived from canonical type definitions:
+
+* `state_kind[state]` — `0` for product, `1` for union,
+* `state_arity[state]` — number of child fields for products (always 1 for unions),
+* `field_index[state][label_id]` — index of field in product (or `-1` if absent),
+* `variant_index[state][label_id]` — index of variant in union (or `-1` if absent).
+
+These tables are read-only and known at compile time for each canonical type.
+
+---
+
+## **7.7  Example: projection**
+
+To implement the projection `.x`:
+
+```c
+struct KOpt k_project_x(struct KValue* input) {
+    struct KNode* node = input->root;
+    int state = node->state;
+    int kind  = state_kind[state];
+
+    if (kind == PRODUCT) {
+        int idx = field_index[state]["x"];
+        if (idx < 0 || idx >= node->arity)
+            return (struct KOpt){0, NULL, NULL};
+        return (struct KOpt){1, node->child[idx], project_pattern(input->pattern, "x")};
+    }
+
+    if (kind == UNION) {
+        int tag  = variant_index[state]["x"];
+        if (tag < 0 || node->tag != tag)
+            return (struct KOpt){0, NULL, NULL};
+        return (struct KOpt){1, node->child[0], project_pattern(input->pattern, "x")};
+    }
+
+    return (struct KOpt){0, NULL, NULL};
+}
+```
+
+This code demonstrates the generic principle:
+the projection is defined if and only if the input has a field or variant named `x`.
+
+---
+
+## **7.8  Error propagation**
+
+When one function calls another, the success flag `ok` propagates forward.
+If a subcall returns `ok == 0`, the caller must return `{0, NULL}` immediately.
+This makes undefinedness explicit and local—no exceptions, no global flags.
+
+---
+
+## **7.9  Summary**
+
+* Every function has the form `KOpt f(KValue*)`.
+* Partiality is expressed by the `ok` flag.
+* Runtime operations handle projection and construction.
+* Runtime values carry pattern context when available.
+* Metadata tables describe field positions and variant indices.
+* By inspecting `arity`, `state`, and `tag`, the runtime can interpret any value correctly.
+* This ABI ensures all generated and runtime functions can interoperate safely.
+
+---

package/DOCS/book/08-operational-semantics-and-execution.md

@@ -0,0 +1,201 @@
+# Chapter 8 — Operational Semantics and Execution
+
+## **8.1  Purpose**
+
+The **operational semantics** of `k` describe how expressions are evaluated step by step on runtime values.
+It defines when a function is *defined* for a particular input and what value it returns.
+All execution—interpretive or compiled—follows these rules.
+
+In the current runtime, a value is a materialized tree together with an optional
+root pattern. The tree determines the ordinary product/union computation; the
+pattern records the polymorphic type context used by the codec stream and is
+propagated by structural operations.
+
+---
+
+## **8.2  Evaluation relation**
+
+Evaluation is written as:
+
+```text
+⟨ e , v ⟩ ⇓ r
+```
+
+meaning that expression `e` applied to value `v` yields result `r`.
+If `e` is undefined for `v`, the relation does not hold.
+
+`r` is a runtime value in memory, represented as described in Chapter 6.
+Undefined results are expressed by the absence of any rule that produces `r`.
+
+---
+
+## **8.3  Rules for base expressions**
+
+### **Identity**
+
+```text
+⟨ () , v ⟩ ⇓ v
+```
+
+The empty composition `()` returns its argument unchanged.
+
+### **Projection**
+
+Let `label` be a field or variant name.
+If `v` has a child under `label`,
+
+```text
+⟨ .label , v ⟩ ⇓ v.label
+```
+
+Otherwise the projection is undefined.
+
+If `v` carries a pattern, the result carries the subpattern reached by `label`.
+For a union projection `/label`, the same rule applies to the selected tag's
+payload.
+
+### **Type expression**
+
+For a type `T`,
+
+```text
+⟨ $T , v ⟩ ⇓ v     if v ∈ T
+```
+
+and undefined otherwise.
+Type expressions thus act as identity functions restricted to their type.
+
+---
+
+## **8.4  Rules for composition**
+
+### **Sequential composition**
+
+```text
+⟨ (f g) , v ⟩ ⇓ r
+```
+
+if there exists `u` such that
+`⟨ f , v ⟩ ⇓ u` and `⟨ g , u ⟩ ⇓ r`.
+
+If either step is undefined, the composition is undefined.
+Because composition is associative,
+
+```text
+(f (g h))  ≡  ((f g) h)  ≡  (f g h)
+```
+
+Parentheses are needed only for `()`.
+
+---
+
+## **8.5  Rules for product composition**
+
+```text
+⟨ { f₁ l₁ , f₂ l₂ , … , fₙ lₙ } , v ⟩ ⇓ { r₁ l₁ , r₂ l₂ , … , rₙ lₙ }
+```
+
+if and only if all subfunctions `fᵢ` are defined on `v` and yield `rᵢ`.
+If any subfunction is undefined, the whole product composition is undefined.
+
+A product composition constructs a new product node;
+each result `rᵢ` becomes one child in canonical field order.
+If the component results carry patterns, the constructed value carries the
+closed product pattern with field `lᵢ` pointing to the corresponding result
+pattern.
+
+---
+
+## **8.6  Rules for union composition**
+
+```text
+⟨ < f₁ , f₂ , … , fₙ > , v ⟩ ⇓ rⱼ
+```
+
+if there exists the smallest index `j` such that
+`⟨ fⱼ , v ⟩ ⇓ rⱼ`.
+
+If no subfunction is defined, the union composition is undefined.
+
+---
+
+## **8.7  Rules for filters**
+
+If a filter `F` matches a single type `T`
+
+```text
+⟨ ?F , v ⟩ ⇓ ⟨ $T , v ⟩ 
+```
+
+otherwise it is an identity
+
+```text
+⟨ ?F , v ⟩ ⇓ v
+```
+
+Filters therefore act as compile time annotations for type checking only.
+As operations they are identities at run time, unless they lead to fully typed
+expressions (i.e., can be replaced by a type). Their pattern information may
+still be present on the runtime value because values can carry codec patterns.
+
+---
+
+## **8.8  Evaluation order**
+
+Evaluation proceeds left to right.
+In products, all subfunctions receive the same input;
+in unions, later functions are evaluated only if earlier ones fail.
+
+The semantics are deterministic:
+for any given input, at most one result tree can be produced.
+
+---
+
+## **8.9  Example**
+
+Given:
+
+```k-lang
+$bool = < {} true, {} false >;
+neg = $bool < .true {{ } false}, .false {{ } true} > $bool;
+```
+
+and input value `{ {} true }` of type `bool`,
+evaluation steps are:
+
+1. `⟨ $bool , { {} true } ⟩ ⇓ { {} true }`
+2. `⟨ < .true {{ } false}, .false {{ } true} > , { {} true } ⟩ ⇓ {{ } false}`
+3. `⟨ $bool , {{ } false} ⟩ ⇓ {{ } false}`
+
+Final result: `{{ } false}`.
+If the input were of another type, step 1 would be undefined.
+
+---
+
+## **8.10  Implementation correspondence**
+
+The evaluation rules map directly onto the runtime ABI:
+
+| Semantic rule       | Runtime operation                     |
+| ------------------- | ------------------------------------- |
+| Projection          | `k_project`                           |
+| Product composition | multiple subcalls + `k_make_product`  |
+| Union composition   | sequential subcalls with early return |
+| Type                | runtime check of `state`              |
+| Composition         | function call chain                   |
+
+In compiled form, the `ok` flag of `KOpt` represents whether a rule applies;
+the node pointer represents the result value.
+
+---
+
+## **8.11  Summary**
+
+* Execution follows deterministic, left-to-right rules.
+* All expressions denote partial functions on runtime values.
+* Runtime values may carry patterns, and structural operations propagate them.
+* Type expressions act as restricted identities.
+* Composition is associative; undefined propagates automatically.
+* Runtime semantics match the formal evaluation relation exactly.
+
+---

package/DOCS/book/09-compiler-architecture.md

@@ -0,0 +1,160 @@
+# Chapter 9 — Compiler Architecture
+
+## **9.1  What the compiler does**
+
+A *compiler* for the `k` language is a program that reads another program (written in `k`) and produces a lower-level version of it.
+The lower-level version can then be executed efficiently by a computer.
+
+For `k`, the compiler’s task is simple in concept:
+
+1. **Read** the definitions of types and functions.
+2. **Understand** their structure and the relationships between them.
+3. **Translate** them into equivalent instructions that follow the runtime conventions described in Chapters 6–8.
+
+The compiler must ensure that every function in the source program becomes a machine-level function that behaves in exactly the same way:
+it takes a value as input, may or may not be defined for it, and, if defined, produces another value.
+
+---
+
+## **9.2  The main stages**
+
+A `k` compiler is organized as a sequence of well-defined stages.
+Each stage transforms one representation of the program into another, simpler one.
+
+1. **Reading and parsing** — The compiler converts the source text into a tree of syntactic objects.
+   This tree is called the *abstract syntax tree* (AST).
+   It records the structure of expressions such as `{ .x a, .y b }` or `< .x, .y >`.
+
+2. **Type analysis and normalization** —
+   The compiler expands type names, checks their correctness, and replaces all filters and meta-variables by concrete type information, following the procedure from Chapter 5.
+   The result is a fully typed AST.
+
+3. **Preparation for execution** —
+   The compiler builds a *type table* for all canonical type states (C0, C1, …).
+   For each type, it records whether it is a product or a union and the number and names of its fields or variants.
+   These tables become the metadata used at runtime.
+
+4. **Code generation** —
+   Finally, the compiler walks through the typed AST and produces an equivalent description in a lower-level form that follows the runtime conventions: functions returning a success flag and a value pointer.
+   This form can be expressed directly in C or in the *LLVM intermediate representation*, which is a platform-independent format understood by many compilers.
+
+---
+
+## **9.3  Internal structure**
+
+A minimal compiler can be viewed as three connected modules:
+
+* **Front end** — reads source code, builds the AST, performs type analysis.
+* **Middle** — resolves all references, normalizes types, and simplifies expressions.
+* **Back end** — produces the executable form.
+
+Although these terms are traditional, the distinction is simple:
+the front end *understands* the program, the back end *emits* it, and the middle part connects the two.
+
+---
+
+## **9.4  Data kept by the compiler**
+
+During translation the compiler maintains several tables:
+
+| Name               | Purpose                                                           |
+| ------------------ | ----------------------------------------------------------------- |
+| **Type table**     | Maps type names to canonical forms and assigns numeric state IDs. |
+| **Field table**    | For each product type, records the order and index of its fields. |
+| **Variant table**  | For each union type, records the tags and their numeric indices.  |
+| **Function table** | Associates function names with their input and output types.      |
+
+These tables ensure that generated code can find fields and variants by number rather than by name, which simplifies execution.
+
+---
+
+## **9.5  Example of transformation**
+
+Consider a simple k program:
+
+```k-lang
+$bool = < {} true, {} false >;
+neg = $bool < /true | false, /false | true > $bool;
+```
+
+1. **Parsing:**
+   The compiler recognizes a type definition `$bool` and a function `neg`.
+
+2. **Type analysis:**
+   Both the input and output of `neg` are of type `bool`.
+
+3. **Internal representation:**
+   The compiler constructs an internal tree describing that
+   `< /true f₁, /false f₂ >` means:
+   try the projection `/true`, then `/false`, each producing a constant value.
+
+4. **Generated form:**
+   In the target language, `neg` becomes a small function that:
+
+   * calls `k_project` to check which variant the input has,
+   * creates a new node with the opposite variant using `k_make_union`, and
+   * returns the result with `ok = 1`.
+
+This sequence of steps is fully automatic and follows directly from the semantic rules in Chapter 8.
+
+---
+
+## **9.6  Intermediate representation**
+
+Before producing final code, many compilers use an *intermediate representation* (IR).
+It is a language designed to be easy to generate and easy to translate further into real machine code.
+For `k`, the IR mirrors the structure of partial functions:
+
+| Concept in k          | IR operation                     |
+| --------------------- | -------------------------------- |
+| Projection `.x` '/x'  | `PROJECT label_id`               |
+| Composition `(f g)`   | `CALL f; CALL g`                 |
+| Union `<f,g>`         | `TRY f; IF undefined THEN TRY g` |
+| Product `{.x f, .y g}` | `PRODUCT_CALL (f, g); COMBINE`   |
+| Type restriction `$T` | `CHECK_TYPE state_id`            |
+
+The compiler converts each AST node into one or more IR instructions.
+The `PRODUCT_CALL(f, g)` instruction is a higher-level concept that implies applying both `f` and `g` to the same input value.
+The low-level implementation of this would use stack operations to duplicate the input, but the IR itself remains more abstract.
+Later, these instructions are translated into LLVM or C code using the runtime ABI.
+
+---
+
+## **9.7  Linking with the runtime**
+
+The generated code does not contain memory allocation or type tables itself.
+Instead, it relies on the runtime library (`krt`), which provides:
+
+* functions such as `k_project` and `k_make_union`,
+* metadata tables for each canonical type, and
+* the unit value `{}` shared by all programs.
+
+When the compiler finishes, it outputs a text file in the target language plus references to this runtime library.
+The two together form an executable program.
+
+---
+
+## **9.8  Simplicity of the model**
+
+Because every expression in `k` takes exactly one input and may return one output or be undefined,
+the compiler never has to deal with variable environments, loops, or side effects.
+This makes the structure of the generated program very regular:
+
+```text
+input → [series of calls and checks] → output or undefined
+```
+
+Each compiled function is independent and stateless.
+This regularity is what allows `k` to map cleanly to low-level code.
+
+---
+
+## **9.9  Summary**
+
+* The compiler transforms `k` programs into executable form through a sequence of simple, deterministic steps.
+* Type information and structure are resolved before code generation.
+* All generated functions follow the same calling convention (`ok`, `val`).
+* The runtime library provides the shared operations needed by every program.
+* The entire process mirrors the formal semantics defined earlier but expresses it as concrete machine operations.
+
+---

package/DOCS/book/10-from-ast-to-intermediate-representation.md

@@ -0,0 +1,167 @@
+# Chapter 10 — From AST to Intermediate Representation
+
+## **10.1  The purpose of an intermediate form**
+
+The abstract syntax tree (AST) of a `k` program directly reflects how the source code was written.
+It is easy to read but not convenient for generating machine code.
+Before code can be produced, the compiler rewrites this tree into a **simpler, regular structure** called the *intermediate representation* (IR).
+
+The IR is close to what the machine will execute, but it still looks like a structured program rather than raw binary instructions.
+Each node in the IR corresponds to a single, well-defined operation such as *project a field*, *apply a function*, or *combine results*.
+
+Using an intermediate form helps in three ways:
+
+1. It separates understanding of the language from machine details.
+2. It allows the compiler to verify that every function is well formed.
+3. It provides a uniform surface for optimization and for later translation into C or LLVM.
+
+---
+
+## **10.2  Shape of the intermediate representation**
+
+Each IR instruction corresponds to one semantic rule from Chapter 8.
+The IR does not need to represent filters or meta-variables, since those are resolved earlier.
+Only concrete actions remain.
+
+| Operation                     | Meaning                                                          |
+| ----------------------------- | ---------------------------------------------------------------- |
+| **PROJECT label_id**          | Apply `.label` to the input value.                               |
+| **TYPECHECK state_id**        | Accept the input only if its root has the given canonical state. |
+| **CALL function_id**          | Invoke another function following the ABI.                       |
+| **UNION [f₁, f₂, …]**         | Try several functions in order until one succeeds.               |
+| **PRODUCT [l₁=f₁, l₂=f₂, …]** | Apply each function to the same input and combine results.       |
+| **FAIL**                      | Always undefined.                                                |
+| **SEQ [f₁, f₂, …]**           | Apply functions in sequence (composition).                       |
+
+Each function in the program becomes a small list or tree of such IR instructions.
+
+---
+
+## **10.3  Example**
+
+Source program:
+
+```k-lang
+$bool = < {} true, {} false >;
+neg = $bool < .true {{ } false}, .false {{ } true} > $bool;
+```
+
+The IR for `neg` can be written informally as:
+
+```text
+neg:
+  TYPECHECK bool
+  UNION [
+    SEQ [ PROJECT true, CONST false ],
+    SEQ [ PROJECT false, CONST true ]
+  ]
+  TYPECHECK bool
+```
+
+This IR expresses the same logic as the source:
+check the type, then try the first projection `.true`, otherwise the second, and finally ensure the result is again of type `$bool`.
+
+---
+
+## **10.4  Relation to evaluation**
+
+Every IR operation corresponds to a runtime action:
+
+| IR operation | Runtime function or behavior                        |
+| ------------ | --------------------------------------------------- |
+| PROJECT      | `k_project(input, label_id)`                        |
+| TYPECHECK    | verify `input->state == state_id`                   |
+| CALL         | call another generated function                     |
+| UNION        | sequential `if`–`else` test on success flags        |
+| PRODUCT      | multiple subcalls + `k_make_product`                |
+| FAIL         | return `{ ok = 0 }` immediately                     |
+| SEQ          | connect the output of one step as input to the next |
+
+By describing computation in these terms, the compiler can generate executable code simply by replacing each IR instruction with its runtime equivalent.
+
+---
+
+## **10.5  Building the IR**
+
+The process of converting the AST to IR follows a recursive pattern:
+
+1. **Projections** become single instructions `PROJECT`.
+2. **Compositions** `(f g h)` become a `SEQ` list of their components.
+3. **Unions** `<f,g>` become a `UNION` list.
+4. **Products** `{f l₁, g l₂}` become a `PRODUCT` list with labeled entries.
+5. **Type expressions** `$T` become `TYPECHECK` instructions.
+6. **Filters** disappear; their meaning is already captured by typechecks.
+
+Each transformation step produces IR nodes with fixed behavior and explicit order.
+
+---
+
+## **10.6  Verifying the IR**
+
+Because k programs are defined by simple composition, a small set of checks ensures correctness:
+
+1. Each `UNION` list must have at least one element.
+2. Every `PROJECT` must have a valid label.
+3. Input and output states of consecutive steps must match.
+4. A function must end in an operation that produces a result (`PROJECT`, `CALL`, or combination).
+
+After verification, the IR is guaranteed to represent a valid partial function according to the semantics in Chapter 8.
+
+---
+
+## **10.7  Intermediate form as a small language**
+
+The IR can be viewed as a minimal language on its own.
+It has:
+
+* one data type (`KNode*`),
+* one universal value format (trees),
+* and a fixed set of operators.
+
+Its execution model is the same as the runtime ABI.
+This means an interpreter for the IR can serve as a reference implementation of the k semantics before any machine code generation is added.
+
+---
+
+## **10.8  Example: product composition**
+
+For input function `{ .x a, .y b }`,
+the IR is:
+
+```text
+PRODUCT [
+  a = PROJECT x,
+  b = PROJECT y
+]
+```
+
+The runtime sequence is:
+
+1. Apply each projection.
+2. If both succeed, build a new node with two children in order `(a,b)`.
+
+This pattern covers all record-building operations in the language.
+
+---
+
+## **10.9  Why this representation is useful**
+
+This intermediate form provides three practical benefits:
+
+1. **Clarity** – every step is explicit and local.
+2. **Verifiability** – type consistency and definedness can be checked mechanically.
+3. **Flexibility** – the same IR can be translated to many targets (LLVM IR, C, or even interpreted directly).
+
+Because of these advantages, most compilers for declarative languages use an intermediate form of this kind.
+
+---
+
+## **10.10  Summary**
+
+* The abstract syntax tree is rewritten into a simpler, machine-oriented form.
+* Each IR instruction represents one operation defined by the `k` semantics.
+* Type and filter information appear as explicit checks.
+* The IR is easy to verify and to translate into executable code.
+* The IR can also serve as a precise description of how `k` functions behave step by step.
+
+---