@jhlagado/azm 0.2.8 → 0.2.9

package/docs/codebase/03-assembly-and-z80-emission.md

@@ -0,0 +1,251 @@
+---
+layout: default
+title: 'Chapter 3 - Assembly and Z80 Emission'
+parent: 'AZM Engineering Manual'
+nav_order: 3
+---
+
+[<- Source Loading and Parsing](02-source-loading-and-parsing.md) | [Ops and Register Contracts ->](04-ops-and-register-contracts.md)
+
+# Chapter 3 - Assembly and Z80 Emission
+
+Assembly turns source items into facts, bytes, fixups and source segments. This
+chapter combines the assembler-time fact model with the Z80 emission path
+because those two stages are tightly coupled: planning decides addresses and
+sizes, then emission writes bytes using those facts.
+
+The central files are:
+
+- `src/assembly/address-planning.ts`
+- `src/assembly/address-symbols.ts`
+- `src/assembly/placement.ts`
+- `src/assembly/program-emission.ts`
+- `src/assembly/fixup-emission.ts`
+- `src/assembly/assemble-program.ts`
+- `src/semantics/expression-evaluation.ts`
+- `src/semantics/constant-operators.ts`
+- `src/semantics/layout-evaluation.ts`
+- `src/z80/parse-instruction.ts`
+- `src/z80/instruction.ts`
+- `src/z80/encode.ts`
+- `src/z80/effects.ts`
+
+## Assembly Orchestration
+
+`src/assembly/assemble-program.ts` is the entry point:
+
+```ts
+export function assembleProgram(items: readonly SourceItem[]): AssembleProgramResult;
+```
+
+It builds the address state, emits the program image and returns diagnostics,
+symbols, byte maps and source segments. The function is intentionally small.
+Detailed decisions live in the modules below it.
+
+## Address Planning
+
+`src/assembly/address-planning.ts` walks source items and builds the facts
+needed to place and evaluate the program:
+
+- labels and their addresses
+- `.equ` constants and enum members
+- record and union layouts
+- type aliases
+- `.org` placement state
+- sizes for data, storage and instructions
+
+The returned `AddressState` contains symbol facts, equate records, layout
+records, enum names, diagnostics and placement information. This is where parsed
+syntax becomes assembler knowledge. A label item becomes an address. A type
+declaration becomes a layout record. A `.ds` directive becomes a byte count.
+
+## Placement
+
+`src/assembly/placement.ts` owns placement state. `.org` sets the active
+address. Instructions advance by encoded size. `.db` advances by emitted byte
+count. `.dw` advances by two bytes per expression. `.ds` advances by calculated
+storage size. `.align` advances to the next aligned address.
+
+Placement is separate from byte emission so the assembler can resolve symbols
+before every byte has been written. A branch to a later label can be encoded
+after address planning has seen the label definition.
+
+## Symbols and Layouts
+
+Labels define addresses. `.equ` declarations define assembler-time values.
+Enums define qualified constants. `src/assembly/address-symbols.ts` contains
+the symbol helpers that define labels, equates and enum members, enforce
+duplicate rules and record spans for diagnostics and output metadata.
+
+Record and union declarations become layout records. A record field advances the
+offset by its byte size. A union field starts at offset zero and the union size
+is the largest field size.
+
+For a record:
+
+```asm
+Sprite .type
+x      .field byte
+y      .field byte
+tile   .field byte
+flags  .field byte
+       .endtype
+```
+
+the layout record stores `x = 0`, `y = 1`, `tile = 2`, `flags = 3` and total
+size `4`.
+
+Type aliases bind a name to another type expression:
+
+```asm
+SpriteArray .typealias Sprite[16]
+```
+
+`sizeof(SpriteArray)` resolves through the alias to the size of `Sprite[16]`,
+with the same field paths as the underlying array expression.
+
+## Expression Evaluation
+
+`src/semantics/expression-evaluation.ts` evaluates expression trees against the
+assembler-time environment. It coordinates literal arithmetic, symbol lookup,
+constant operators, byte functions, layout functions and layout casts that fold
+to constant addresses.
+
+The operator and layout rules live in focused modules:
+
+| File                    | Role                                                                |
+| ----------------------- | ------------------------------------------------------------------- |
+| `constant-operators.ts` | Binary and unary constant-operator dispatch.                        |
+| `binary-operators.ts`   | Binary arithmetic, bitwise, comparison and logical operators.       |
+| `unary-operators.ts`    | Unary numeric operators.                                            |
+| `byte-functions.ts`     | `LSB(...)`, `MSB(...)` and related byte extraction helpers.         |
+| `layout-evaluation.ts`  | `sizeof(...)`, `offset(...)` and layout type expression evaluation. |
+| `layout-path.ts`        | Field and array path resolution for layout casts.                   |
+| `layout-format.ts`      | Human-readable layout names for diagnostics.                        |
+| `diagnostics.ts`        | Shared semantic diagnostic construction.                            |
+
+Expression evaluation is context-sensitive. A symbol in an instruction operand
+may be a label or constant. A type expression in `.ds Sprite[4]` resolves to a
+byte count. A layout cast such as `<SpriteArray>Sprites[3].flags` resolves to an
+address when the base address, type alias, index and field path are all known at
+assembly time.
+
+## Data and Storage Size
+
+Address planning needs the byte length of each directive. It contains size
+helpers for `.db`, `.dw`, `.ds`, strings and alignment. The same string
+directive byte rules are reused during emission so planning and output stay
+aligned.
+
+Initialized data writes bytes through `.db`, `.dw`, `.cstr`, `.pstr` and
+`.istr`. `.ds` reserves space calculated from numbers or layout type
+expressions. Layout type expressions are byte-size expressions in storage and
+field-size positions.
+
+## Program Emission
+
+`src/assembly/program-emission.ts` owns byte writing. It walks source items in
+order and writes emitted bytes to a map keyed by absolute address. It also
+records source segments for the D8 map writer.
+
+The emitted program shape is:
+
+```ts
+export interface EmittedProgram {
+  readonly image: ReadonlyMap<number, number>;
+  readonly sourceSegments: readonly EmittedSourceSegment[];
+  readonly initializedAddresses: readonly number[];
+}
+```
+
+The byte map stores final addresses. If source emits one byte at `$0100` and
+another at `$8000`, the map has two entries. A sparse map fits Z80 programs that
+place code, data and ROM vectors at separate origins.
+
+## Emission Walkthrough
+
+For this source:
+
+```asm
+        .org $0100
+Start:
+        ld      a,42
+        jp      Start
+```
+
+address planning records `Start = $0100`. The instruction parser represents
+`ld a,42` as an immediate load and `jp Start` as an absolute branch. The encoder
+returns literal opcode bytes plus a 16-bit expression fragment for `Start`.
+Fixup emission evaluates `Start` to `$0100` and writes the little-endian operand
+bytes.
+
+The final byte map contains:
+
+```text
+$0100: 3E
+$0101: 2A
+$0102: C3
+$0103: 00
+$0104: 01
+```
+
+The D8 source segment records that those bytes came from the source lines that
+emitted them.
+
+## Z80 Instruction Model
+
+`src/z80/instruction.ts` defines the instruction and operand types.
+`src/z80/parse-instruction.ts` dispatches instruction text into parser families.
+`src/z80/encode.ts` dispatches typed instructions into encoder families.
+`src/z80/effects.ts` describes register and flag effects for register contracts.
+
+The parser and encoder work as a pair:
+
+| File                                                                                           | Question                                                                                     |
+| ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
+| `parse-instruction.ts`                                                                         | Which parser family should handle this instruction head?                                     |
+| `parse-basic.ts`, `parse-branch.ts`, `parse-exchange.ts`, `parse-io-control.ts`, `parse-ld.ts` | Parse specific instruction families.                                                         |
+| `parse-operands.ts`, `parse-conditions.ts`, `operand-split.ts`                                 | Split operands and classify conditions, registers, indexed operands and expression operands. |
+| `instruction.ts`                                                                               | How is that form represented as TypeScript data?                                             |
+| `encode.ts`, `encode-core.ts`, `encode-ld.ts`, `encode-ld-helpers.ts`                          | Which bytes and fixup fragments represent that form?                                         |
+| `effects.ts`                                                                                   | Which registers and flags does that form read or write?                                      |
+
+The instruction model keeps overloaded Z80 mnemonics manageable. `ld` has many
+forms, but the parser classifies operands before encoding. The encoder then
+selects a specific opcode family from typed operands.
+
+## Fixups and Relative Branches
+
+The Z80 encoder returns fragments: literal bytes, 8-bit immediates, 16-bit
+immediates and relative branch targets. `src/assembly/fixup-emission.ts`
+evaluates the expression attached to each fragment and writes the final byte or
+word.
+
+Relative branches such as `jr` and `djnz` emit an 8-bit displacement from the
+next instruction. The parser recognises `jr nz,Loop`. The encoder knows the
+opcode and that the operand is relative. Fixup emission knows the current
+address and final target address.
+
+The same principle applies to absolute branches and immediate operands. The
+encoder chooses the fragment width. Fixup emission evaluates the expression and
+checks that the value fits that width.
+
+## Source Segments
+
+Every emitted byte range can carry source provenance. `program-emission.ts`
+adds `EmittedSourceSegment` records with start address, end address, source
+file, line and segment kind. `writeD8m()` later groups these segments by file
+for Debug80.
+
+Source segments classify emitted ranges as code, data, directive output, label
+context or unknown output. Debug80 uses this metadata to connect source lines to
+addresses. A byte-perfect assembler can still give a poor debugging experience
+when source segments are too broad, absent or attached to a different file.
+
+## Changing Assembly or Encoding
+
+Instruction changes usually touch `parse-instruction.ts`, `instruction.ts`,
+`encode.ts` and `effects.ts`. Assembly changes usually touch
+`address-planning.ts`, `program-emission.ts`, `fixup-emission.ts` or
+`placement.ts`, depending on whether the change affects facts, emitted bytes,
+symbolic references or address movement.

package/docs/codebase/04-ops-and-register-contracts.md

@@ -0,0 +1,237 @@
+---
+layout: default
+title: 'Chapter 4 - Ops and Register Contracts'
+parent: 'AZM Engineering Manual'
+nav_order: 4
+---
+
+[<- Assembly and Z80 Emission](03-assembly-and-z80-emission.md) | [Interfaces and Output Artifacts ->](05-interfaces-and-output-artifacts.md)
+
+# Chapter 4 - Ops and Register Contracts
+
+Ops and register contracts are the two AZM-specific subsystems that sit above plain
+Z80 instruction assembly. Ops expand source into visible inline assembly.
+Register contract analysis checks the resulting routines and calls.
+
+These features belong together in the codebase tour because they meet at the
+same boundary: parsed source items. Ops produce source items. Register contract analysis
+reads source items.
+
+## Ops as Visible Expansion
+
+Ops are named inline instruction idioms. They let source define a small
+operation once and expand it visibly at each use site. The implementation lives
+in `src/expansion/`. `op-expansion.ts` coordinates the subsystem. Operand
+splitting, overload selection, selected expansion, instruction instantiation
+and local-label rewriting live in focused helper modules.
+
+An op is closer to a typed inline template than to a text macro. The op parser
+understands operands, chooses an overload and parses the expanded body back
+through the normal AZM parser. The result is visible assembly with the same
+diagnostic and register contract behaviour as handwritten source.
+
+For example:
+
+```asm
+op clear(reg8 r)
+        xor     r
+end
+
+        clear a
+```
+
+The expansion stage matches `a` as a register operand, substitutes it into the
+template and emits the source item for `xor a`. Address planning and emission
+then treat that instruction exactly like a line written directly in the source.
+
+## Op Collection and Invocation
+
+`collectOps()` scans logical lines before normal parsing. It finds top-level
+`op` blocks, parses their parameter lists, records the body template and marks
+the source lines that belong to the definition body.
+
+An op declaration has:
+
+- a name
+- a parameter list
+- matcher information for overload selection
+- a body template
+- source location metadata for diagnostics
+
+The registry is complete before invocation parsing starts. `parseOpInvocation()`
+checks whether a source line could be an op call. If the name matches a
+collected op, `expandOpInvocation()` selects an overload and instantiates the
+body.
+
+The parser handles op invocations before `parseLogicalLine()`. An op head can
+look like an instruction head at the source level. The expansion stage resolves
+it before ordinary line parsing.
+
+## Overloads and Templates
+
+Ops support overloads. `op-selection.ts` compares invocation operands against
+each candidate signature. It prefers the most specific matching overload and
+emits diagnostics for arity errors, unsupported operands, ambiguous matches and
+invalid expansions.
+
+The matcher vocabulary recognises fixed tokens, registers, register pairs,
+immediates, conditions, ports and indexed operands. It stays close to the Z80
+operand model, so op dispatch and instruction parsing describe operands in the
+same terms.
+
+An op body template is parsed into template items. During expansion, operands
+from the call site are substituted into the template by
+`op-instruction-instantiation.ts`. The result is formatted as ordinary source
+text and parsed through the same line parser used for top-level source.
+
+Local label rewriting lives in `op-local-labels.ts`. A local label in an op
+expansion becomes unique at the use site so each expansion receives its own
+generated label. Once the rewritten labels become source items, address planning
+defines and resolves them through the ordinary symbol path.
+
+## Op Diagnostics and Register Contracts
+
+Op diagnostics point at the call site while explaining the definition that
+matched or failed. Invalid expanded instructions are reported as op expansion
+failures with the underlying Z80 parser diagnostic included.
+
+Ops expand before register contract analysis builds routines. AZM sees the
+expanded instructions. An op is visible inline assembly, so its register effects
+belong to the caller.
+
+## Register Contract Analysis
+
+Register contract analysis checks how routines use Z80 registers. It reads routine
+boundaries, instruction effects and AZMDoc contract comments, then reports
+conflicts where a caller still needs a register value that a callee may change.
+
+The implementation lives in `src/register-contracts/`. The public analysis entry
+point is `analyzeRegisterContracts()` in `src/register-contracts/analyze.ts`.
+
+Register contract analysis is a data-flow analysis over assembled source
+structure. It works with routines, calls, instruction effects and contracts. It
+analyses the source structure to find values live across a call and callee
+summaries that can change those values.
+
+## A Register Contract Conflict
+
+This source shape captures the problem:
+
+```asm
+@Caller:
+        ld      b,8
+Loop:
+        call    Worker
+        djnz    Loop
+        ret
+
+;! clobbers B
+@Worker:
+        ld      b,0
+        ret
+```
+
+`Caller` uses `B` as the `djnz` counter. `Worker` declares that it clobbers
+`B`. Liveness sees that `B` is still needed after the call because `djnz Loop`
+reads it. The register contract conflict is at the call site: `Caller` passes
+through a routine boundary that may change a live unit.
+
+The programmer can preserve `B`, choose a different counter register, change
+`Worker` so it leaves `B` unchanged or update the calling sequence. Register
+contract analysis identifies the conflict and the source location where the
+caller crosses the boundary.
+
+## Routine Model and Contracts
+
+`src/register-contracts/programModel.ts` builds the program model from parsed source
+items. Routine-specific extraction is split into
+`programModel-boundaries.ts` and `programModel-routines.ts`. Together they find
+routine boundaries, direct calls, labels and instructions. Routine entry labels
+use `@` in source and become callable public routine names after the marker is
+removed.
+
+`src/register-contracts/smartComments.ts` reads AZMDoc comments from the comment maps
+captured during loading. Comment-block splitting and token parsing live in
+`smartCommentBlocks.ts` and `smartCommentParsing.ts`. External `.asmi`
+contracts are parsed in `interfaceContracts.ts`.
+
+Contracts can describe:
+
+- inputs
+- outputs
+- clobbered registers
+- preserved registers
+- expected outputs at call sites
+
+Source comments and external interfaces describe the same kind of fact: a
+routine contract. Source comments attach to routines in the current program.
+`.asmi` entries attach to routines whose source is assembled elsewhere.
+
+## Effects, Summaries and Liveness
+
+Register contract analysis depends on `src/z80/effects.ts`. Effects describe which registers
+and flags an instruction reads, writes or preserves. `instruction-head.ts`,
+`instruction-operands.ts`, `instruction-predicates.ts` and
+`operand-register-name.ts` translate between Z80 instruction shapes and
+register contract units such as `A`, `HL`, `carry` and register pairs.
+
+`src/register-contracts/summary.ts` infers a summary for a single routine. Boundary,
+contract, result, state and token-transfer logic now lives in
+`summary-boundary.ts`, `summary-contract.ts`, `summary-result.ts`,
+`summary-state.ts` and `summary-token-transfer.ts`. `routine-summaries.ts` and
+`summaries.ts` combine routine summaries, external contracts and profile
+summaries into lookup tables. A summary records the observable contract of a
+routine: the units it reads, writes, preserves, clobbers and returns as outputs.
+
+`src/register-contracts/liveness.ts` performs the caller-side analysis. It works
+backwards through each routine. At a call, it compares the live-after set with
+the callee summary. A live unit that the callee clobbers becomes a conflict. A
+unit produced by the callee and read by the caller becomes an output candidate.
+
+Stack behaviour is part of routine summaries. `summary.ts` tracks push, pop,
+exchange-top and unknown stack effects. `routine-summaries.ts` infers summaries
+to a fixed point so internal routine calls can see optimistic boundary
+summaries before the final pass. Strict mode uses `stackBalanced` and
+`hasUnknownStackEffect` to distinguish balanced stack use from a routine whose
+boundary may leave the stack in an unknown state.
+
+## Reports, Interfaces and Tooling
+
+`report.ts` renders human-readable `.regcontracts.txt` reports and `.asmi` interface
+metadata. `annotate.ts`, `annotations.ts`, `fix.ts` and `sourceText.ts` support
+source updates for generated AZMDoc comments and conservative fixes.
+
+The CLI can request these behaviours through:
+
+- `--reg-report`
+- `--reg-interface`
+- `--contracts`
+- `--fix`
+- `--accept-out`
+
+`src/register-contracts/tooling.ts` exposes editor-friendly diagnostics and code
+actions through `analyzeRegisterContractsForTools()`. Tooling diagnostics carry
+file, line, column, message, fixability and optional text edits. An editor can
+show the same register contract information that the CLI reports while using
+normal editor actions for accepted fixes.
+
+## Changing Ops or Register Contracts
+
+Op changes belong in `src/expansion/`, with tests under
+`test/unit/expansion/` and integration tests for source-level behaviour.
+Register contract changes usually begin in one of these files:
+
+- Routine boundaries and calls: `programModel.ts`
+- AZMDoc parsing: `smartComments.ts`, `smartCommentBlocks.ts`,
+  `smartCommentParsing.ts`, `interfaceContracts.ts`
+- Instruction effects: `z80/effects.ts`, `instruction-head.ts`,
+  `instruction-operands.ts`, `instruction-predicates.ts`
+- Summary inference: `summary.ts`
+- Caller liveness: `liveness.ts`
+- Output text: `report.ts`
+- Source edits: `annotate.ts`, `fix.ts`, `annotations.ts`
+- Tooling surface: `tooling.ts`
+
+Run unit tests under `test/unit/register-contracts/`, integration tests under
+`test/integration/register-contracts/` and CLI tests in
+`test/cli/register_contracts_cli.test.ts`.