chipgraph 0.1.0__py3-none-any.whl

chipgraph/packs/dv/skills/dv/cocotb.md

@@ -0,0 +1,72 @@
+---
+id: dv/cocotb
+description: How to write a cocotb 2.1 test module for a block from its spec and interface, never its RTL.
+roles: [tb-author]
+version: 0.1.0
+---
+Write one cocotb 2.1 test module (`outputs[0]`) that checks the block **against its
+spec**. You see the spec, its requirements and the interface; you never see the RTL, and
+must not guess it.
+
+What to use from the context:
+
+- `interface.ports`: the only signals you may touch, as `dut.<name>`. Drive inputs
+  with `dut.<name>.value = ...`, sample outputs with `int(dut.<name>.value)` (see the
+  table below: a one-bit port is not a vector).
+  A port marked `rtl differs` or `missing in rtl` disagrees with the design: test what
+  the spec says, and say so in `assumptions`.
+- `interface.clock` and `interface.reset` (with `reset_active_low`): start the clock with
+  `cocotb.start_soon(Clock(dut.<clock>, 10, unit="ns").start())`, hold reset active for a
+  few cycles with every input at 0, then release it.
+- The register bus: the spec's interface table and register map (`registers` in the
+  block's model input, the register-map table in the spec text) give the address,
+  write-strobe and data ports and each register's offset, access and reset value. Write
+  small `write(dut, offset, value)` and `read(dut, offset)` helpers on those ports, and
+  name each offset as a constant from the register map.
+- `requirements`: every id must be cited at least once.
+
+cocotb 2.x API (checked against cocotb 2.1.0; a name that is not in this table is a
+guess: look for it here first):
+
+| You want | Write | Not this |
+|---|---|---|
+| Read any port as a number | `int(dut.x.value)` (raises `ValueError` if a bit is `X`/`Z`) | `dut.x.value.integer` (deprecated) |
+| Read a multi-bit port, explicit | `dut.x.value.to_unsigned()`, `.to_signed()` (a `LogicArray`) | |
+| Read a **one-bit** port | `int(dut.irq.value)` or `dut.irq.value == 1` (its value is a `Logic`) | `dut.irq.value.to_unsigned()`: `Logic` has no such method (`AttributeError`) |
+| Check for `X`/`Z` | `dut.x.value.is_resolvable` | comparing to a string |
+| Compare | `dut.x.value == 0xA5`, `dut.irq.value == 1` | |
+| Write a port | `dut.x.value = 5`; takes effect at the end of the current time step, so read the effect after the next `await RisingEdge(dut.clk)` | writing and reading in the same step |
+| One bit, a slice of a vector | `dut.x.value[0]` is a `Logic`; `dut.x.value[7:4]` is a `LogicArray` | |
+| Build a vector | `from cocotb.types import LogicArray, Range`; `LogicArray.from_unsigned(5, 8)`, `LogicArray(0xA5, Range(7, 0))` | `BinaryValue` / `cocotb.binary` (removed) |
+| Clock | `Clock(dut.clk, 10, unit="ns")`, started with `cocotb.start_soon(clock.start())` | `units=` (deprecated), `cocotb.fork`, `@cocotb.coroutine` (removed) |
+| Wait | `await RisingEdge(dut.clk)`, `await ClockCycles(dut.clk, n)`, `await Timer(10, unit="ns")` | `yield` |
+| Start a task | `task = cocotb.start_soon(coro())`; `await cocotb.start(coro())` runs it now; `task.cancel()` | `cocotb.fork` |
+| Fail a test | a plain `assert cond, "message"` | raising `TestFailure` |
+| Time limit for a test | `@cocotb.test(timeout_time=50, timeout_unit="us")` | a hand-made counter loop |
+
+How to write it:
+
+- `import cocotb`, `from cocotb.clock import Clock`, `from cocotb.triggers import
+  ClockCycles, ReadOnly, RisingEdge, Timer`. Nothing that reads files or runs programs:
+  no `open`, `os`, `pathlib`, `glob`, `subprocess`, no path strings. Use no `dut.<name>`
+  that is not in `interface.ports`: no internal signal, no `dut._id(...)`, no
+  `getattr(dut, ...)` on a computed name.
+- One `@cocotb.test()` `async def` per requirement group (the requirements of one spec
+  subsection, or ones that need the same setup). Put `# verifies: REQ-...` (comma
+  separated) on the line after the `def`, and name the ids in its docstring.
+- Every `assert` has a message that quotes the spec value it checks, e.g.
+  `assert got == 0x3C, f"REQ-GPIO-002: DATA_IN reads pin_in; expected 0x3c, got {got:#x}"`.
+- Reset values: check each register's spec reset value right after reset.
+- Keep each test short and deterministic: fixed stimulus, or `random.Random(<seed>)`
+  with a fixed seed; wait a stated number of cycles, never "long enough".
+- Test only behaviour the spec states. Do not assume internals (pipeline depth, state
+  names, encodings) the spec does not give.
+
+When the spec is ambiguous or silent on something a test needs (a latency, what a
+reserved offset reads, which edge), do not guess: write the tests you can, and report
+`needs_human` with the question.
+
+If a redo says a failure in the design was hidden, the test failed against the RTL in a
+way you cannot see. Change the test only where it does not follow the spec; if it does
+follow the spec, report `needs_human` and say which requirement the design seems to
+break.

chipgraph/packs/spec_core/__init__.py

@@ -0,0 +1,9 @@
+"""The ``spec-core`` pack: deterministic extractors for text specifications.
+
+This pack ships the ``mas-markdown`` extractor (:mod:`chipgraph.packs.spec_core.extract.mas`),
+which reads QNSC-style Micro-Architecture Specification (MAS) Markdown into the Design
+Model: requirements (declared or inferred, D37), interface ports, register maps with
+fields, and open items. The extractor is registered through the
+``chipgraph.adapters.extractor`` entry point; the pack directory itself is discovered by
+:func:`chipgraph.app.build.builtin_packs_dir`.
+"""

chipgraph/packs/spec_core/extract/__init__.py

@@ -0,0 +1,12 @@
+"""Deterministic extractors for the ``spec-core`` pack.
+
+Currently the MAS Markdown extractor, :class:`~chipgraph.packs.spec_core.extract.mas.MasExtractor`.
+"""
+
+from chipgraph.packs.spec_core.extract.mas import (
+    MasDiagnostic,
+    MasExtractor,
+    MasTemplate,
+)
+
+__all__ = ["MasDiagnostic", "MasExtractor", "MasTemplate"]

chipgraph/packs/spec_core/extract/_markdown.py

@@ -0,0 +1,371 @@
+"""A small, line-exact Markdown scanner for MAS documents.
+
+The scanner keeps every source line with its 1-based number and classifies it so the
+extractor never has to rescan raw text. It skips YAML front matter, HTML comments
+(multi-line; the QNSC template is full of them) and fenced code blocks, but keeps line
+numbers exact so a diagnostic can always cite the real source line.
+
+``<!-- gen:... -->`` and ``<!-- /gen -->`` markers are ordinary comments and are skipped,
+but the table between them is content and is scanned normally.
+
+Nothing here knows about ports, registers or requirements: it produces headings, tables,
+list items and paragraphs, and the MAS extractor gives them meaning.
+"""
+
+from __future__ import annotations
+
+import re
+from collections.abc import Iterator, Sequence
+from dataclasses import dataclass, field
+
+_HEADING = re.compile(r"^(#{1,6})\s+(.*?)\s*#*\s*$")
+_HEADING_NUMBER = re.compile(r"^\s*\d+(?:\.\d+)*\.?\s+")
+_LIST_ITEM = re.compile(r"^(\s*)(?:[-*+]|(\d+)[.)])\s+(.*)$")
+_TABLE_DIVIDER = re.compile(r"^\s*\|?\s*:?-{1,}:?\s*(?:\|\s*:?-{1,}:?\s*)*\|?\s*$")
+_FRONT_MATTER = "---"
+_FENCE = re.compile(r"^\s*(`{3,}|~{3,})")
+
+
+@dataclass(frozen=True, slots=True)
+class Heading:
+    """A section heading: its raw title, the number-stripped title, and its level."""
+
+    line: int
+    level: int
+    raw_title: str
+    title: str
+    """The title with any leading section number (``5.``, ``7.1``) stripped."""
+
+
+@dataclass(frozen=True, slots=True)
+class ListItem:
+    """A top-level or nested list item, with its continuation lines already joined."""
+
+    line: int
+    indent: int
+    ordered: bool
+    number: int | None
+    text: str
+    """The item body (marker removed), continuation lines joined with single spaces."""
+
+
+@dataclass(frozen=True, slots=True)
+class Paragraph:
+    """A run of non-list, non-table prose lines."""
+
+    line: int
+    text: str
+
+
+@dataclass(frozen=True, slots=True)
+class TableRow:
+    """One row of a pipe table: its cells (outer pipes stripped) and its source line."""
+
+    line: int
+    cells: tuple[str, ...]
+
+
+@dataclass(frozen=True, slots=True)
+class Table:
+    """A GitHub-style pipe table: a header row, its column names, and body rows."""
+
+    header_line: int
+    headers: tuple[str, ...]
+    header_cells_raw: tuple[str, ...]
+    rows: tuple[TableRow, ...]
+
+
+@dataclass(slots=True)
+class Section:
+    """A heading and every block that belongs to it, until the next same-or-higher heading."""
+
+    heading: Heading | None
+    tables: list[Table] = field(default_factory=list)
+    list_items: list[ListItem] = field(default_factory=list)
+    paragraphs: list[Paragraph] = field(default_factory=list)
+
+
+@dataclass(frozen=True, slots=True)
+class Document:
+    """A scanned MAS document: its sections and every declared block, in source order."""
+
+    sections: tuple[Section, ...]
+    all_tables: tuple[Table, ...]
+    all_list_items: tuple[ListItem, ...]
+
+
+def strip_heading_number(title: str) -> str:
+    """Remove a leading section number (``5.``, ``7.1``, ``11.``) from a heading title."""
+    return _HEADING_NUMBER.sub("", title).strip()
+
+
+def split_row(line: str) -> tuple[str, ...]:
+    """Split a pipe-table row into trimmed cells, honouring escaped ``\\|``.
+
+    Outer pipes are stripped. ``\\|`` inside a cell is kept as a literal ``|``.
+    """
+    body = line.strip()
+    if body.startswith("|"):
+        body = body[1:]
+    if body.endswith("|"):
+        body = body[:-1]
+    cells: list[str] = []
+    current: list[str] = []
+    escaped = False
+    for ch in body:
+        if escaped:
+            current.append(ch)
+            escaped = False
+        elif ch == "\\":
+            escaped = True
+            current.append(ch)
+        elif ch == "|":
+            cells.append("".join(current).strip())
+            current = []
+        else:
+            current.append(ch)
+    cells.append("".join(current).strip())
+    return tuple(cells)
+
+
+class _LineKind:
+    """Internal per-line classification produced by the pre-pass."""
+
+    __slots__ = ("kind", "line", "text")
+
+    def __init__(self, line: int, kind: str, text: str) -> None:
+        self.line = line
+        self.kind = kind  # "content" or "skip"
+        self.text = text
+
+
+def _prepass(text: str) -> list[_LineKind]:
+    """Classify every physical line as content or skipped, keeping numbers exact."""
+    lines = text.splitlines()
+    result: list[_LineKind] = []
+    in_front_matter = False
+    front_matter_possible = True
+    in_comment = False
+    fence: str | None = None
+
+    for index, raw in enumerate(lines):
+        number = index + 1
+        stripped = raw.strip()
+
+        # YAML front matter: only a leading '---' on line 1 opens it.
+        if front_matter_possible and number == 1 and stripped == _FRONT_MATTER:
+            in_front_matter = True
+            front_matter_possible = False
+            result.append(_LineKind(number, "skip", raw))
+            continue
+        front_matter_possible = False
+        if in_front_matter:
+            result.append(_LineKind(number, "skip", raw))
+            if stripped == _FRONT_MATTER:
+                in_front_matter = False
+            continue
+
+        # Fenced code blocks.
+        if fence is not None:
+            result.append(_LineKind(number, "skip", raw))
+            if stripped.startswith(fence):
+                fence = None
+            continue
+
+        # HTML comments (may open and close on the same line, or span many lines).
+        if in_comment:
+            result.append(_LineKind(number, "skip", raw))
+            if "-->" in raw:
+                in_comment = False
+            continue
+
+        working = raw
+        if "<!--" in working:
+            # A comment may sit inline; skip the whole physical line for simplicity, and
+            # track whether it stays open past this line.
+            if "-->" in working[working.index("<!--") :]:
+                # Opens and closes on this line: treat the line as skipped content.
+                result.append(_LineKind(number, "skip", raw))
+                continue
+            in_comment = True
+            result.append(_LineKind(number, "skip", raw))
+            continue
+
+        fence_match = _FENCE.match(raw)
+        if fence_match:
+            fence = fence_match.group(1)[0] * 3
+            result.append(_LineKind(number, "skip", raw))
+            continue
+
+        result.append(_LineKind(number, "content", raw))
+
+    return result
+
+
+def _looks_like_table_header(lines: Sequence[_LineKind], i: int) -> bool:
+    """True when line ``i`` is a header row followed by a divider row."""
+    if "|" not in lines[i].text:
+        return False
+    if i + 1 >= len(lines):
+        return False
+    nxt = lines[i + 1]
+    return nxt.kind == "content" and bool(_TABLE_DIVIDER.match(nxt.text))
+
+
+def scan(text: str) -> Document:
+    """Scan MAS Markdown into a :class:`Document`."""
+    lines = _prepass(text)
+    sections: list[Section] = []
+    current = Section(heading=None)
+    sections.append(current)
+    all_tables: list[Table] = []
+    all_items: list[ListItem] = []
+
+    i = 0
+    n = len(lines)
+    while i < n:
+        entry = lines[i]
+        if entry.kind == "skip":
+            i += 1
+            continue
+        raw = entry.text
+        stripped = raw.strip()
+
+        if not stripped:
+            i += 1
+            continue
+
+        heading_match = _HEADING.match(raw)
+        if heading_match:
+            level = len(heading_match.group(1))
+            raw_title = heading_match.group(2).strip()
+            heading = Heading(
+                line=entry.line,
+                level=level,
+                raw_title=raw_title,
+                title=strip_heading_number(raw_title),
+            )
+            current = Section(heading=heading)
+            sections.append(current)
+            i += 1
+            continue
+
+        if _looks_like_table_header(lines, i):
+            table, consumed = _consume_table(lines, i)
+            current.tables.append(table)
+            all_tables.append(table)
+            i += consumed
+            continue
+
+        item_match = _LIST_ITEM.match(raw)
+        if item_match:
+            item, consumed = _consume_list_item(lines, i)
+            current.list_items.append(item)
+            all_items.append(item)
+            i += consumed
+            continue
+
+        paragraph, consumed = _consume_paragraph(lines, i)
+        current.paragraphs.append(paragraph)
+        i += consumed
+
+    return Document(
+        sections=tuple(sections),
+        all_tables=tuple(all_tables),
+        all_list_items=tuple(all_items),
+    )
+
+
+def _consume_table(lines: Sequence[_LineKind], start: int) -> tuple[Table, int]:
+    """Consume a pipe table beginning at ``start`` (header + divider + body rows)."""
+    header = lines[start]
+    header_cells = split_row(header.text)
+    headers = tuple(c.strip() for c in header_cells)
+    rows: list[TableRow] = []
+    i = start + 2  # skip header and divider
+    n = len(lines)
+    while i < n:
+        entry = lines[i]
+        if entry.kind == "skip":
+            # A comment or fence inside a table ends it (tables are contiguous).
+            break
+        if "|" not in entry.text or not entry.text.strip():
+            break
+        rows.append(TableRow(line=entry.line, cells=split_row(entry.text)))
+        i += 1
+    table = Table(
+        header_line=header.line,
+        headers=headers,
+        header_cells_raw=header_cells,
+        rows=tuple(rows),
+    )
+    return table, i - start
+
+
+def _consume_list_item(lines: Sequence[_LineKind], start: int) -> tuple[ListItem, int]:
+    """Consume a list item and its continuation lines beginning at ``start``."""
+    match = _LIST_ITEM.match(lines[start].text)
+    assert match is not None
+    indent = len(match.group(1))
+    number = int(match.group(2)) if match.group(2) else None
+    ordered = number is not None
+    parts = [match.group(3).strip()]
+    i = start + 1
+    n = len(lines)
+    while i < n:
+        entry = lines[i]
+        if entry.kind == "skip":
+            break
+        raw = entry.text
+        if not raw.strip():
+            break
+        if _LIST_ITEM.match(raw):
+            break
+        if _HEADING.match(raw):
+            break
+        if "|" in raw and _looks_like_table_header(lines, i):
+            break
+        # A continuation line is indented under the item.
+        leading = len(raw) - len(raw.lstrip())
+        if leading <= indent and not raw.startswith(" "):
+            break
+        parts.append(raw.strip())
+        i += 1
+    text = " ".join(p for p in parts if p)
+    item = ListItem(
+        line=lines[start].line,
+        indent=indent,
+        ordered=ordered,
+        number=number,
+        text=text,
+    )
+    return item, i - start
+
+
+def _consume_paragraph(lines: Sequence[_LineKind], start: int) -> tuple[Paragraph, int]:
+    """Consume a run of prose lines beginning at ``start``."""
+    parts = [lines[start].text.strip()]
+    i = start + 1
+    n = len(lines)
+    while i < n:
+        entry = lines[i]
+        if entry.kind == "skip":
+            break
+        raw = entry.text
+        if not raw.strip():
+            break
+        if _HEADING.match(raw) or _LIST_ITEM.match(raw):
+            break
+        if "|" in raw and _looks_like_table_header(lines, i):
+            break
+        parts.append(raw.strip())
+        i += 1
+    return Paragraph(line=lines[start].line, text=" ".join(parts)), i - start
+
+
+def iter_sections_by_title(doc: Document, title: str) -> Iterator[Section]:
+    """Yield every section whose number-stripped title equals ``title`` (case-insensitive)."""
+    wanted = title.strip().casefold()
+    for section in doc.sections:
+        if section.heading is not None and section.heading.title.casefold() == wanted:
+            yield section

chipgraph/packs/spec_core/extract/_text.py

@@ -0,0 +1,116 @@
+"""Text helpers shared by the MAS extractor: normalization, hashing, cell parsing.
+
+Kept separate from the scanner and the extractor so the rules for turning a cell or an
+item into a value are all in one place and easy to test.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import re
+
+_WHITESPACE = re.compile(r"\s+")
+_LEADING_NUMBER = re.compile(r"^\s*\d+(?:\.\d+)*[.)]?\s+")
+_INT_LITERAL = re.compile(r"^[+-]?(?:0[xX][0-9a-fA-F_]+|0[bB][01_]+|[0-9][0-9_]*)$")
+_IDENTIFIER = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
+
+
+def collapse_whitespace(text: str) -> str:
+    """Collapse every run of whitespace to a single space and strip the ends."""
+    return _WHITESPACE.sub(" ", text).strip()
+
+
+def strip_leading_number(text: str) -> str:
+    """Remove a leading ordered-list number (``1.``, ``2)``) from item text."""
+    return _LEADING_NUMBER.sub("", text, count=1)
+
+
+def normalize_item_text(text: str) -> str:
+    """Normalize a list item for hashing: drop its number, collapse whitespace.
+
+    Inline markup (backticks, ``**``) is intentionally kept: two items that differ only
+    in emphasis are different requirements.
+    """
+    return collapse_whitespace(strip_leading_number(text))
+
+
+def content_hash8(text: str) -> str:
+    """The first 8 hex chars of the SHA-256 of ``text``, for a stable inferred key."""
+    return hashlib.sha256(text.encode("utf-8")).hexdigest()[:8]
+
+
+def strip_markup_token(token: str) -> str:
+    """Strip surrounding backticks and ``**`` bold markers from a single token."""
+    value = token.strip()
+    changed = True
+    while changed:
+        changed = False
+        if value.startswith("**") and value.endswith("**") and len(value) >= 4:
+            value = value[2:-2]
+            changed = True
+        if len(value) >= 2 and value.startswith("`") and value.endswith("`"):
+            value = value[1:-1]
+            changed = True
+    return value.strip()
+
+
+def strip_cell_markup(cell: str) -> str:
+    """Remove backticks and bold markers throughout a table cell, then collapse spaces."""
+    without = cell.replace("`", "").replace("**", "")
+    return collapse_whitespace(without)
+
+
+def first_token(text: str) -> str:
+    """Return the first whitespace-separated token of ``text`` (empty if none)."""
+    stripped = text.strip()
+    if not stripped:
+        return ""
+    return stripped.split()[0]
+
+
+def parse_int_maybe(text: str) -> int | str | None:
+    """Parse ``text`` as an int (hex/bin/decimal, ``_`` separators allowed).
+
+    Returns the int on success, the collapsed string if it is non-empty but not an int,
+    and ``None`` if empty.
+    """
+    value = strip_cell_markup(text)
+    if not value:
+        return None
+    compact = value.replace(" ", "")
+    if not _INT_LITERAL.match(compact):
+        return value
+    cleaned = compact.replace("_", "")
+    lowered = cleaned.lstrip("+-").lower()
+    try:
+        if lowered.startswith("0x"):
+            return int(cleaned, 16)
+        if lowered.startswith("0b"):
+            return int(cleaned, 2)
+        return int(cleaned, 10)
+    except ValueError:  # pragma: no cover - guarded by the regex
+        return value
+
+
+def is_identifier(text: str) -> bool:
+    """True when ``text`` (after markup removal) is a single simple identifier."""
+    return bool(_IDENTIFIER.match(strip_cell_markup(text)))
+
+
+def parse_bits(text: str) -> tuple[int, int] | None:
+    """Parse a Bits cell (``15:8`` or ``0``) into ``(msb, lsb)``.
+
+    Returns ``None`` when the cell is not a plain bit or bit range (e.g. ``bits 0-15``).
+    """
+    value = strip_cell_markup(text)
+    if not value:
+        return None
+    if ":" in value:
+        hi, _, lo = value.partition(":")
+        if hi.strip().isdigit() and lo.strip().isdigit():
+            return int(hi), int(lo)
+        return None
+    if value.isdigit():
+        n = int(value)
+        return n, n
+    return None