@opengsd/gsd-path 1.4.0 → 1.6.0

package/skills/gsd-path-plan/references/spec-probes.md

@@ -0,0 +1,150 @@
+# Spec-reach probes
+
+Define runs two probes over the success criteria before approval: the edge
+probe (what each criterion must do at its edges) and the prohibition probe
+(what each criterion must never silently become). Both leave a table in
+INTENT.md. Plan, build review, and final review read those tables; the
+`check_handoffs.py intent`, `plan`, and `final` gates enforce them.
+
+## Why define owns this
+
+A reviewer can only check what the intent states. An edge nobody wrote down
+is never checked: final review covers exactly the INTENT success criteria,
+and a reviewer passes any reasonable reading of an ambiguous one. Model
+reviewers are also confidently wrong on exactly these gaps. On three tasks
+whose correct behavior the spec left unstated, LLM verifier verdicts caught
+the defect 0 times in 12 while a held-out test caught all three; in a larger
+replication, stating the surfaced edge in the spec raised the verifier's
+catch rate to 98% ([Verifier Reach Is Spec Reach](https://github.com/davesienkowski/verifier-reach-is-spec-reach)).
+A smarter reviewer does not close that gap; a wider intent does. The user is
+present at define, so define is where the edge gets its ruling.
+
+## Edge probe
+
+Walk every success criterion:
+
+1. Name the data shapes the criterion handles, then raise only the
+   categories those shapes reach:
+
+   | Shape | Categories |
+   |-------|------------|
+   | numeric range, dates, money, counts | boundary, precision |
+   | list, set, rows, intervals | adjacency, empty, ordering |
+   | text, names, identifiers | empty, encoding |
+   | state that persists or repeats | idempotency, concurrency |
+   | files, network, processes | concurrency |
+
+2. For each raised category, write one concrete candidate edge and the
+   ruling you recommend:
+
+   | Category | Probe question |
+   |----------|----------------|
+   | boundary | What happens exactly at each minimum, maximum, or threshold, and one step either side? |
+   | adjacency | When two items are equal or just touch, do they merge, collide, or stay separate? |
+   | empty | What is the result for empty, single-item, or missing input? |
+   | encoding | Whose length or equality applies: bytes, code points, grapheme clusters, normalized form, case? |
+   | ordering | When items compare equal, is output order specified and stable? |
+   | precision | Where can rounding, overflow, or truncation happen, and which rule applies? |
+   | idempotency | What happens when this runs twice on the same input? |
+   | concurrency | If interrupted or run in parallel, what is guaranteed? |
+
+3. Ask the user in one small round, all criteria together, recommended
+   ruling first. Never rule on an edge's behavior yourself: the correct
+   answer is exactly what the criteria did not say, so a guess here is the
+   defect this probe exists to prevent. Under pre-approval, an edge whose
+   behavior the user has not stated is still a required answer: stop and
+   ask instead of ruling.
+
+4. Record each ruling as one row:
+   - `criterion SCn` when the answer is observable where the user sees the
+     work. Add or amend that success criterion (define owns criteria), then
+     point the row at it. An existing criterion that already states the
+     edge may be named directly.
+   - `held-out` when the answer is a precise rule or a set of examples that
+     belongs in a test rather than a user-facing criterion (a normalization
+     form, a rounding rule, an inclusive bound on an internal API). Write the
+     ruling or the user's examples verbatim in Detail. The plan must schedule
+     a named test that pins it.
+   - `dismissed` when the edge cannot occur for this criterion. Detail says
+     why ("input is a closed enum"). "Not important" is not a reason.
+
+Every criterion gets at least one row. A criterion with no data shape gets
+one row with category `none`, dismissed with its reason.
+
+## Prohibition probe
+
+Walk every success criterion with one adversarial question: what could this
+silently become that the user would not want, although nothing in INTENT.md
+forbids it? Generate candidates privately, then keep only values, safety,
+privacy, fairness, and transparency items: "must not email unsubscribed
+users", "must not store raw card numbers in logs", "must not use guilt
+wording". Drop routine engineering items (mutating input, throwing on empty
+input, leaking handles); those are edges or ordinary review. Do not mint
+generic security or compliance canon (injection, path traversal, data
+retention law); record a real compliance need under Constraints instead.
+Expect two or three kept items for work people touch, and often none for
+internals.
+
+Present the kept items with a recommended ruling and record each as a row:
+
+- `criterion SCn` when a test or command can check it. State it as a
+  success criterion and point the row at it.
+- `judgment` when it is real but not mechanically checkable. Detail says
+  what the final reviewer looks for.
+- `dismissed` when the user rules it out. Detail records the reason.
+
+Work that affects no person directly records a single row instead, and no
+other: `| N1 | all | none | dismissed | <why no person is affected> |`.
+
+## INTENT.md tables
+
+```markdown
+## Edge coverage
+
+| Edge | Criterion | Category | Disposition | Detail |
+|------|-----------|----------|-------------|--------|
+| E1 | SC1 | boundary | criterion SC4 | |
+| E2 | SC1 | empty | dismissed | the command always receives at least one file |
+| E3 | SC2 | encoding | held-out | names compare by NFC code points, case-sensitive |
+
+## Prohibitions
+
+| Prohibition | Criterion | Must not | Disposition | Detail |
+|-------------|-----------|----------|-------------|--------|
+| N1 | SC2 | send a reminder to an unsubscribed user | criterion SC5 | |
+| N2 | SC2 | use guilt or shame wording | judgment | reminder copy is neutral and states only the due item |
+```
+
+Ids are contiguous from `E1` and `N1`. Detail is required for every row that
+is not `criterion SCn`. The headings are exact: `## Edge coverage` and
+`## Prohibitions`. An INTENT.md without them predates the probes and is not
+rechecked; a near-miss heading fails the gate rather than switching it off.
+
+## Downstream
+
+- **Plan.** Every `held-out` edge gets one PLAN.md `## Held-out checks` row
+  naming the task that owns the edge's criterion and the test file (a
+  `path::test_name` node id is fine). That file is listed in the task's
+  `files`, the task's Verify runs it, and the task's Acceptance criteria
+  state the edge id and its ruling. A `criterion SCn` row needs nothing extra: SCn is
+  already covered like any criterion. A `judgment` prohibition belongs in
+  the owning task's Approach so the coder knows the line.
+- **Review.** A row pins the reading of its criterion. A reviewer does not
+  pass a reading that contradicts a ruling in either table, and cites each
+  held-out edge and judgment prohibition id with its evidence.
+- **Final review.** A criterion that carries a held-out edge or a judgment
+  prohibition is `met` only when its Check, Observed, or Reference cites
+  every such id with the evidence: the held-out test's recorded passing
+  output, or the observed behavior judged against Detail. When that
+  evidence is missing, did not run, or cannot be judged, record
+  `unverifiable` with `Finding: insufficient spec evidence: <id> ...`;
+  never `met` on inference. The trigger is the tag, not reviewer doubt:
+  criteria without a tag are graded as usual and never abstained.
+
+## What the gates can and cannot prove
+
+`check_handoffs.py intent` (before approval and again when define records
+approval), `plan`, and `final` prove the tables are complete, every held-out
+edge has a scheduled test, and every tagged `met` cites its tags. They cannot
+prove who made a ruling or that a cited result says what the verdict claims.
+Those stay with the define conversation and the reviewer's recorded evidence.

package/skills/gsd-path-plan/scripts/_common.py

@@ -25,7 +25,8 @@ import signal
 import subprocess
 import time
 from pathlib import Path
-from typing import BinaryIO, Iterator, List, Optional, Sequence
+from typing import BinaryIO, Iterable, Iterator, List, Optional, Sequence, Tuple
+
 
 # Chosen by what imports, not os.name, so tests can emulate Windows on POSIX.
 try:
@@ -65,6 +66,49 @@ VERIFY_RESULTS = ("pass", "fail")
 VERIFY_LEDGER_SCHEMA = "gsd-path/verify-ledger/v2"
 VERIFY_BLOCK_PATTERN = re.compile(r"```bash[ \t]*\n(?P<block>.*?)```", re.DOTALL)
 
+REVIEW_BULLET_HEADING_PATTERN = re.compile(r"^#{2,3} ")
+REVIEW_BULLET_FIELD_LABEL_PATTERN = re.compile(r"^- \*\*[^*]+\*\*:")
+
+
+def review_bullet_field_label_pattern(field: str) -> re.Pattern[str]:
+    return re.compile(rf"^- \*\*{re.escape(field)}\*\*:[ \t]*(.*)$")
+
+
+def _review_bullet_field_value(lines: Sequence[str], index: int, inline: str) -> str:
+    if inline:
+        return inline
+    collected: list[str] = []
+    for line in lines[index + 1 :]:
+        if REVIEW_BULLET_FIELD_LABEL_PATTERN.match(line) or REVIEW_BULLET_HEADING_PATTERN.match(
+            line
+        ):
+            break
+        if line and not line[0].isspace():
+            break
+        stripped = line.strip()
+        if stripped:
+            collected.append(stripped)
+    return "\n".join(collected)
+
+
+def find_review_bullet_field_values(lines: Sequence[str], field: str) -> list[str]:
+    pattern = review_bullet_field_label_pattern(field)
+    values: list[str] = []
+    for index, line in enumerate(lines):
+        match = pattern.fullmatch(line)
+        if match is None:
+            continue
+        values.append(_review_bullet_field_value(lines, index, match.group(1)))
+    return values
+
+
+def parse_review_bullet_field(lines: Sequence[str], field: str) -> str:
+    """Parse one `- **Field**:` review bullet, including indented continuations."""
+    values = find_review_bullet_field_values(lines, field)
+    if len(values) != 1:
+        raise ValueError(field)
+    return values[0]
+
 
 # Sample paths the pipeline must commit. A product rule such as an unanchored
 # `build/` would otherwise drop them silently from every commit.
@@ -118,6 +162,33 @@ def project_ignore_error(repo: Path) -> Optional[str]:
             + ". Anchor the product rule (for example `/build/`) in a task that owns it.")
 
 
+_DEFAULT_BRANCH_RE = re.compile(
+    r"[^\s#\"'\\~^:?*\[\x00-\x1f\x7f-][^\s#\"'\\~^:?*\[\x00-\x1f\x7f]*"
+)
+_RESERVED_BRANCH_PREFIXES = ("gsd-path/", "gsd-path-integrate/", "gsd-path-task/")
+
+
+def valid_default_branch(name: str) -> bool:
+    """A git branch name that STATE frontmatter can hold and Path does not reserve."""
+    return (
+        bool(_DEFAULT_BRANCH_RE.fullmatch(name))
+        and name != "null"
+        and not name.startswith(_RESERVED_BRANCH_PREFIXES)
+        and not name.endswith(("/", ".", ".lock"))
+        and not any(part in name for part in ("..", "//", "/.", "@{"))
+    )
+
+
+def origin_default_branch(repo: Path) -> str:
+    """The origin default branch that initialization records; an unset origin/HEAD means main."""
+    result = run_git(repo, "symbolic-ref", "--quiet", "refs/remotes/origin/HEAD")
+    prefix = "refs/remotes/origin/"
+    remote_ref = result.stdout.strip()
+    if result.returncode != 0 or not remote_ref.startswith(prefix):
+        return "main"
+    return remote_ref.removeprefix(prefix)
+
+
 def section_body(text: str, heading: str) -> Optional[str]:
     """The text under `## <heading>` up to the next `## `, or None when absent."""
     match = re.search(
@@ -126,6 +197,79 @@ def section_body(text: str, heading: str) -> Optional[str]:
     return match.group("body") if match else None
 
 
+def overlapping_paths(left: Sequence[str], right: Sequence[str]) -> List[str]:
+    """Return the sorted paths that two task `files` lists both cover.
+
+    An entry covers itself and the paths below it. For a declared directory
+    and a path below it, the result names the lower path. The callers have no
+    task base, so this does not ask if an entry is a file: the task-brief
+    lint rejects a path below a file at the base.
+    """
+    return sorted(
+        {
+            max(one, other, key=len)
+            for one in left
+            for other in right
+            if one == other or one.startswith(other + "/") or other.startswith(one + "/")
+        }
+    )
+
+
+# The `- ` of an indented sub-bullet line. A line that would then read as a
+# numbered item (`- 2. text`) keeps its marker, so both readings of a source
+# have the same criteria.
+SUB_BULLET_MARKER = re.compile(r"(?m)^([ \t]+)-[ \t]+(?![ \t]*\d+\.\s)")
+
+
+def without_sub_bullet_markers(source: str) -> str:
+    """`source` with the `- ` marker of each indented sub-bullet line removed."""
+    return SUB_BULLET_MARKER.sub(r"\1", source)
+
+
+class Criterion(str):
+    """A criterion as its parser joins it, with the `- ` markers of its sub-bullet lines.
+
+    `bare` is the same criterion parsed from `without_sub_bullet_markers(source)`.
+    The joined text cannot tell a marker from a minus sign; the source lines can.
+    """
+
+    bare: str
+
+    def __new__(cls, text: str, bare: str) -> "Criterion":
+        criterion = super().__new__(cls, text)
+        criterion.bare = bare
+        return criterion
+
+
+def quotes_criterion(quote: str, criterion: str) -> bool:
+    """Whether a review quote is `criterion`, for every reader.
+
+    Only whitespace is collapsed. The quote must equal the criterion with all
+    the markers of its sub-bullet lines or with none of them; every other
+    character, a hyphen inside a line too, is compared.
+    """
+    forms = (criterion, getattr(criterion, "bare", criterion))
+    return " ".join(quote.split()) in {" ".join(form.split()) for form in forms}
+
+
+def quoted_criterion(item: str, criteria: Sequence[str]) -> Optional[Tuple[int, int]]:
+    """Index of the criterion quoted at the start of `item`, and where the quote ends.
+
+    A criterion can contain ` — ` itself, so each separator and the end of
+    `item` is a candidate, the longest quote first. A criterion as written
+    wins over any criterion read without its sub-bullet markers.
+    """
+    ends = [match.start() for match in re.finditer("(?= —(?: |$))|$", item)][::-1]
+    for bare in (False, True):
+        for end in ends:
+            quote = " ".join(item[:end].split())
+            for index, criterion in enumerate(criteria):
+                form = getattr(criterion, "bare", criterion) if bare else criterion
+                if quote == " ".join(form.split()):
+                    return index, end
+    return None
+
+
 def task_verify_command(task_text: str) -> str:
     """The Verify shell text, excluding the closing fence's separator newline."""
     body = section_body(task_text, "Verify")
@@ -133,6 +277,11 @@ def task_verify_command(task_text: str) -> str:
     return block.group("block").removesuffix("\n") if block else ""
 
 
+def fix_verify(commands: Iterable[str]) -> str:
+    """The Verify of a fix task: each distinct source Verify in its own subshell, under `set -e`."""
+    return "set -e\n" + "\n".join(f"(\n{command}\n)" for command in dict.fromkeys(commands))
+
+
 def latest_verify_entry(entries: list, command: str, commit: str,
                         repo: Optional[str] = None) -> Optional[dict]:
     """Rows are keyed by (command, repo, commit); a row without `repo` is a coordinator row."""
@@ -150,6 +299,11 @@ def verify_ledger_entries(path: Path) -> list:
     return parse_verify_ledger(path.read_text(encoding="utf-8"))
 
 
+def read_user_text(path: Path) -> str:
+    """Read a user-edited text file with CRLF normalized to LF."""
+    return path.read_text(encoding="utf-8").replace("\r\n", "\n").replace("\r", "\n")
+
+
 def parse_verify_ledger(text: str) -> list:
     entries = []
     for number, line in enumerate(text.splitlines(), start=1):
@@ -204,7 +358,43 @@ def run_command(
 def run_git(
     repo: Path, *arguments: str, input: Optional[str] = None
 ) -> subprocess.CompletedProcess[str]:
-    return _run_exact(("git", "-C", str(repo), *arguments), input=input)
+    return _run_exact(git_command(repo, *arguments), input=input)
+
+
+def git_command(repo: Path, *arguments: str) -> tuple[str, ...]:
+    """Git argv for repo, including WSL drvfs workarounds when needed."""
+    return ("git", "-C", str(repo), *git_drvfs_config_flags(repo), *arguments)
+
+
+def git_drvfs_config_flags(repo: Path) -> tuple[str, ...]:
+    """Disable index preload on 9p/drvfs mounts where git commit races on index.lock (#207)."""
+    if os.name != "posix":
+        return ()
+    try:
+        root = repo.resolve()
+        mounts = Path("/proc/mounts").read_text(encoding="utf-8", errors="replace").splitlines()
+    except OSError:
+        return ()
+    best = ""
+    fstype = ""
+    for line in mounts:
+        parts = line.split()
+        if len(parts) < 3:
+            continue
+        mount_point, mount_type = parts[1], parts[2]
+        if mount_type not in {"9p", "drvfs"}:
+            continue
+        if root == Path(mount_point) or root.is_relative_to(mount_point):
+            if len(mount_point) >= len(best):
+                best, fstype = mount_point, mount_type
+    if not fstype:
+        return ()
+    return ("-c", "core.preloadindex=false", "-c", "index.threads=1")
+
+
+def git_index_lock_retryable(stderr: str) -> bool:
+    text = stderr.casefold()
+    return "index.lock" in text and "file exists" in text
 
 
 # OS files that are never pipeline artifacts when git ignores them.
@@ -227,6 +417,70 @@ def git_visible_entries(directory: Path) -> set:
     return {path.split("/", 1)[0] for path in result.stdout.split("\0") if path}
 
 
+RUNTIME_PIN_PATH = ".gsd-path/runtime.json"
+RUNTIME_PIN_ALLOWANCE = (
+    "only runtime-pin commits (.gsd-path/runtime.json) may follow the reviewed HEAD"
+)
+
+
+def reviewed_head_covers(repo: Path, reviewed: str, head: str) -> bool:
+    """True when a review of `reviewed` still covers `head`.
+
+    That is the same commit, or `reviewed` is an ancestor of `head` and every
+    commit on the first-parent path in ``reviewed..head`` is a non-merge commit
+    whose only change is ``.gsd-path/runtime.json``. An empty commit does not
+    qualify. Any other path, any merge commit, an unresolvable SHA, or a
+    non-ancestor returns False.
+    """
+    if not reviewed or not head:
+        return False
+    resolved_repo = Path(repo)
+    resolved: List[str] = []
+    for value in (reviewed, head):
+        result = run_git(
+            resolved_repo, "rev-parse", "--verify", "--quiet", f"{value}^{{commit}}"
+        )
+        if result.returncode != 0 or not result.stdout.strip():
+            return False
+        resolved.append(result.stdout.strip())
+    reviewed_sha, head_sha = resolved
+    if reviewed_sha == head_sha:
+        return True
+    if (
+        run_git(resolved_repo, "merge-base", "--is-ancestor", reviewed_sha, head_sha).returncode
+        != 0
+    ):
+        return False
+    listing = run_git(
+        resolved_repo,
+        "rev-list",
+        "--first-parent",
+        "--parents",
+        f"{reviewed_sha}..{head_sha}",
+    )
+    if listing.returncode != 0:
+        return False
+    for line in listing.stdout.splitlines():
+        fields = line.split()
+        if len(fields) != 2:
+            return False
+        commit_sha, parent_sha = fields[0], fields[1]
+        changed = run_git(
+            resolved_repo,
+            "diff",
+            "--name-only",
+            "--no-renames",
+            "--no-ext-diff",
+            parent_sha,
+            commit_sha,
+        )
+        if changed.returncode != 0:
+            return False
+        if {path for path in changed.stdout.splitlines() if path} != {RUNTIME_PIN_PATH}:
+            return False
+    return True
+
+
 def atomic_replace(path: Path, temporary_path: Path, content: str) -> None:
     temporary_path.parent.mkdir(parents=True, exist_ok=True)
     descriptor = None

package/skills/gsd-path-plan/scripts/archive_milestone.py

@@ -426,10 +426,10 @@ def frontmatter_value(content: str, key: str) -> Optional[str]:
     for line in lines[1:]:
         if line == "---":
             break
-        match = re.match(rf"^{re.escape(key)}:\s*([^#]*?)\s*(?:#.*)?$", line)
+        match = re.match(rf"^{re.escape(key)}:(.*)$", line)
         if match:
-            value = match.group(1).strip().strip('"').strip("'")
-            return value
+            value = _common.strip_yaml_comment(match.group(1))
+            return value.strip().strip('"').strip("'")
     return None
 
 
@@ -586,14 +586,11 @@ def resolved_archive_target(
     expected_number = milestone_number(branch)
     configured = parsed_state.archive
     entries = archive_sequence_entries(project)
-    missing_prior = [
-        number for number in range(1, expected_number) if number not in entries
-    ]
     later = sorted(number for number in entries if number > expected_number)
-    if missing_prior or later:
+    if later:
         raise ArchiveError(
             f"archive sequence does not match bound branch {branch}: "
-            f"missing={missing_prior}, later={later}"
+            f"later={later}"
         )
 
     if configured is not None:
@@ -1065,9 +1062,10 @@ def prepare_locked(project: Path, active_root: Path, slug: str) -> dict:
         final = active_root / "review" / "FINAL.md"
         reviewed = re.search(r"(?m)^Reviewed HEAD:\s*(\S+)", final.read_text(encoding="utf-8")) if final.is_file() else None
         head = require_git_success(run_git(project, "rev-parse", "HEAD"), "resolve HEAD").strip()
-        if reviewed and reviewed.group(1) != head:
+        if reviewed and not _common.reviewed_head_covers(project, reviewed.group(1), head):
             raise ArchiveError(
                 f"FINAL.md Reviewed HEAD {reviewed.group(1)} does not match current HEAD {head}; "
+                f"{_common.RUNTIME_PIN_ALLOWANCE}; "
                 "re-run the final review on the current commit before archiving"
             )
         target = resolved_archive_target(project, slug, parsed_state)
@@ -1680,6 +1678,7 @@ def abandon_locked(
 
 
 run_git = _common.run_git
+reviewed_head_covers = _common.reviewed_head_covers
 run_command = _common.run_command
 
 
@@ -1899,9 +1898,10 @@ def preflight(repo: Path) -> dict:
     require_stageable_carry_forward(project, archive)
     require_clean_older_archives(project, configured)
     head = require_git_success(run_git(project, "rev-parse", "HEAD"), "resolve preflight HEAD")
-    if reviewed_head != head:
+    if not reviewed_head_covers(project, reviewed_head, head):
         raise ArchiveError(
-            f"FINAL.md Reviewed HEAD {reviewed_head} does not match current HEAD {head}"
+            f"FINAL.md Reviewed HEAD {reviewed_head} does not match current HEAD {head}; "
+            f"{_common.RUNTIME_PIN_ALLOWANCE}"
         )
     return {
         "archive": configured,
@@ -2093,9 +2093,10 @@ def validate(repo: Path, *, historical: bool = False) -> dict:
 
     members = member_ship_rows(project, archive, ship_commit)
     ship_parent = require_git_success(run_git(project, "rev-parse", f"{ship_commit}^"), "resolve ship parent")
-    if reviewed_head != ship_parent:
+    if not reviewed_head_covers(project, reviewed_head, ship_parent):
         raise ArchiveError(
-            f"FINAL.md Reviewed HEAD {reviewed_head} does not match ship parent {ship_parent}"
+            f"FINAL.md Reviewed HEAD {reviewed_head} does not match ship parent {ship_parent}; "
+            f"{_common.RUNTIME_PIN_ALLOWANCE}"
         )
     require_canonical_commit_body(
         project,