@hoanghainh/komagu 1.8.0 → 2.0.1

package/assess/lsa/knowledge/severity-and-cwe.md

@@ -43,7 +43,7 @@ Finding id prefixes: `BOLA-`, `INJ-`, `BFLA-`, `MASS-`.
 
 `fingerprint` is optional. When present it has the form `kfp1:<64 lowercase hex>`
 and must not depend on line numbers, so the same issue keeps the same value
-across scans. Exporters copy it to SARIF `partialFingerprints["komainu/v1"]`.
+across scans. Exporters copy it to SARIF `partialFingerprints["komagu/v1"]`.
 
 ## SARIF level mapping
 

package/assess/lsa/prompts/35-privilege-auditor.md

@@ -69,7 +69,9 @@ privileged, and a hinted route can be safe.
 2. **Find the model's fields.** Read the model / entity / table. List the
    privileged fields it holds (role, admin flag, permissions, owner, user,
    tenant, price, amount, balance, status, verified, approved, plan).
-   No privileged field → `safe`, reason "no privileged field".
+   No privileged field → `safe`, reason "no privileged field". If you cannot
+   find or read the model's definition, you do not know its fields: answer
+   `needs_review` (reason "model fields not visible"), never `safe`.
 3. **Look for what limits the fields**, with `file:line`: explicit field
    copy, allow-list (`$fillable` without privileged fields, `permit(...)`,
    `[Bind(Include=...)]`, DTO with only editable fields), validation that

package/assess/lsa/scripts/baseline.py

@@ -35,7 +35,9 @@ from trend import append_history, mttr_by_severity, snapshot  # noqa: E402
 
 SEVERITY_RANK = {"info": 0, "low": 1, "medium": 2, "high": 3, "critical": 4}
 FAIL_ON = ("none", "new", "any")
-SARIF_FINGERPRINT_KEY = "komainu/v1"
+SARIF_FINGERPRINT_KEY = "komagu/v1"
+# SARIF baselines written before 2.0 use the old key; still read for one major release (#312)
+LEGACY_SARIF_FINGERPRINT_KEY = "komainu/v1"
 
 
 class BaselineError(Exception):
@@ -57,7 +59,8 @@ def _entries_from_sarif(doc: Dict) -> Dict[str, Dict]:
     entries: Dict[str, Dict] = {}
     for run in doc.get("runs") or []:
         for result in run.get("results") or []:
-            fp = (result.get("partialFingerprints") or {}).get(SARIF_FINGERPRINT_KEY)
+            fingerprints = result.get("partialFingerprints") or {}
+            fp = fingerprints.get(SARIF_FINGERPRINT_KEY) or fingerprints.get(LEGACY_SARIF_FINGERPRINT_KEY)
             if not fp:
                 continue
             props = result.get("properties") or {}

package/assess/lsa/scripts/findings_to_sarif.py

@@ -132,6 +132,28 @@ OWASP_TAGS = {
 }
 
 
+# Attack-vector keys that `komagu test --from-scan` and `--export-poc` need to replay a
+# finding from SARIF. suggested_payload is left out so attack strings stay out of code scanning.
+REPLAY_KEYS = ("method", "path", "fields", "readback", "target_param")
+
+
+def replay_properties(f: Dict[str, Any]) -> Dict[str, Any]:
+    """Result properties that let a SARIF result be replayed like a findings.json entry."""
+    props: Dict[str, Any] = {"finding_id": f.get("finding_id"), "category": f.get("category", "bola")}
+    ep = f.get("endpoint")
+    if isinstance(ep, dict) and ep.get("path"):
+        props["endpoint"] = {k: ep[k] for k in ("method", "path") if ep.get(k)}
+    av = f.get("attack_vector")
+    if isinstance(av, dict):
+        kept = {k: av[k] for k in REPLAY_KEYS if av.get(k) not in (None, "", {})}
+        if kept:
+            props["attack_vector"] = kept
+    cons = f.get("consensus")
+    if isinstance(cons, dict) and isinstance(cons.get("score"), (int, float)):
+        props["consensus_score"] = cons["score"]
+    return props
+
+
 def normalize_cwe_id(cwe_val: Any) -> str:
     """Normalize numeric or prefixed CWE ID to standard 'CWE-XXX' format."""
     s = str(cwe_val).strip()
@@ -319,7 +341,7 @@ def to_sarif(
 
         fingerprints = {"lsaFindingId": f.get("finding_id", f"LSA-{len(results)+1}")}
         if f.get("fingerprint"):
-            fingerprints["komainu/v1"] = f["fingerprint"]
+            fingerprints["komagu/v1"] = f["fingerprint"]
 
         result_taxa = []
         for c in norm_cwes:
@@ -352,6 +374,7 @@ def to_sarif(
                 "confidence": f.get("confidence", "high"),
                 "status": f.get("status", "unverified"),
                 "cwe": norm_cwes,
+                **replay_properties(f),
             },
         }
         state = f.get("baseline_state")

package/assess/lsa/tests/EVAL-RESULTS.md

@@ -152,3 +152,159 @@ Based on Qwen 3.5's findings, the following concrete improvements were implement
 | **False Positives on Safe Code** | 0 | 0 | 0 on safe markers | **0** |
 | **Precision (matched / active)** | 0.965 (55 / 57) | 0.965 (55 / 57) | 0.966 | **1.000 (57 / 57)** |
 | **Legacy Stack Shims** | None | None | Recommended | **4 Stacks (`remediate/legacy_shims/`)** |
+
+---
+
+## Run 4 — 2026-10-31, kit 0.3.1 (BFLA & mass assignment Tier 2, local Ollama)
+
+Issue #286. First end-to-end measurement of phase 4c (`35-privilege-auditor`) with real models, and a
+re-check of BOLA / injection on the same harness. All runs are **local only**: fixture code never left
+the machine, and no third-party LLM was used.
+
+### Setup
+
+| Item | Value |
+|---|---|
+| Eval repo | `tests/build_eval_repo.py` (10 fixture apps, markers stripped); ground truth 117 items: 25 BOLA, 34 SQL/NoSQL injection, 10 BFLA, 15 mass assignment, plus 33 SAFE markers |
+| Pipeline | `komagu scan <repo> --no-cache --headless-ai http://localhost:11434/v1 --model <model> --ai-timeout 3600` (`operate/ai/runner.py`, Tier 1 pre-scan + phases 2–6) |
+| Hosting | Ollama 0.40.13 on an Apple M4 Pro, 24 GB RAM, 100% GPU; context 32,768 tokens |
+| Models | `gpt-oss:20b` (MXFP4, reasoning effort default `medium`), `qwen3.5:9b` |
+| Scoring | `tests/score_findings.py --verdicts` (tolerance 3 lines); archived per run in [`assess/lsa/examples/eval-run-4/`](../examples/eval-run-4/) |
+
+**This is a floor, not a ceiling.** A 9–20B model on a laptop is much weaker than the models used in
+Runs 1–2. The numbers below say what a fully local deployment achieves today, not what the prompts can
+do with a stronger model.
+
+**Not comparable with Run 3.** Run 3 does not record its model, and the archived Run 1 / Run 2 / Qwen
+numbers were produced through agent runtimes that can open files, not through the headless runner.
+The BOLA / injection columns below are therefore a new baseline for the headless harness, not a
+regression check against Run 3.
+
+### Harness fixes found by this run
+
+Measuring with real models exposed defects in the headless runner that inflated or lost results.
+They were fixed before the numbers were trusted, in two iterations (the limit set in #286):
+
+| Iteration | Fix | Effect seen |
+|---|---|---|
+| 0 → 1 | Verifier no longer marks a whole batch `confirmed` when its reply cannot be parsed (it used to, with a made-up "consensus" note); such findings stay `unverified` | Results stopped being inflated by broken replies |
+| 0 → 1 | A failed call (timeout) is recorded per batch in `ai-run-stats.json` instead of aborting the run | A 25-candidate injection batch took > 15 min and used to kill the run |
+| 0 → 1 | Phase 4c gets the same JSON output template as 4a / 4b; sanitizer joins list `impact` values and moves a verdict written into `subtype` to `status`; unparseable replies saved to `OUT/ai-raw/` | Mass assignment findings stopped failing schema validation |
+| 1 → 2 | Verifier now receives the code it judges (finding site, file header, evidence lines, server-side includes) and the auth model, and may only cite controls that are visible; otherwise `likely` + `code_not_visible` | In iteration 1 it rejected 10 of 11 real BFLA findings by inventing an admin base class and a role check in a login-only `auth.php` |
+| 1 → 2 | Verifier batches of 5 instead of 10; sanitizer fills a `null` evidence line and a missing `status` | No verifier parse failures in iteration 2 |
+
+### Results after verification
+
+"Marker" columns use the normal location match. "Verdict" columns (BFLA / mass assignment only) count a
+truth item as found when the privilege auditor judged the route or candidate that contains it
+`vulnerable` or `needs_review`, whatever path and line the model then wrote into the finding.
+
+| Run | BOLA | Injection | BFLA marker | Mass marker | BFLA verdict | Mass verdict | FP on safe code | Precision | Rejected | Unverified | Parse fail | Min · calls · tokens |
+|---|---|---|---|---|---|---|---|---|---|---|---|---|
+| iter 0, gpt-oss A | 19/25 | 20/34 | 8/10 | 2/15 | 10/10 | 12/15 | 0 | 0.689 | 10 | 11 | 3 | 81 · 22 · 311K |
+| iter 1, gpt-oss A | 21/25 | 34/34 | 1/10 | 2/15 | 10/10 | 14/15 | 0 | 0.765 | 19 | 17 | 2 | 75 · 21 · 288K |
+| iter 1, gpt-oss B | 19/25 | 28/34 | 5/10 | 5/15 | 10/10 | 14/15 | 1 | 0.667 | 4 | 8 | 3 | 80 · 22 · 273K |
+| iter 1, qwen3.5 (incomplete) | 23/25 | 33/34 | 0/10 | 0/15 | 0/10 | 8/15 | 1 | 0.778 | 0 | 52 | — | stopped after 2 h 50 min |
+| **iter 2, gpt-oss A** | **24/25** | **34/34** | **8/10** | **6/15** | **10/10** | **10/15** | **0** | 0.729 | 1 | 0 | 0 | 57 · 32 · 309K |
+| **iter 2, gpt-oss B** | **11/25** | **32/34** | **5/10** | **4/15** | **10/10** | **9/15** | **0** | 0.693 | 1 | 0 | 1 | 56 · 29 · 318K |
+
+Final harness = iteration 2. Stated plainly for `gpt-oss:20b` on it:
+
+- **BFLA:** the auditor flagged the right route for **10 / 10** BFLA markers in both runs, with no SAFE-BFLA
+  route flagged. Only **8 / 10 and 5 / 10** survive as correctly located findings, because the model
+  writes wrong or non-existent paths into the finding (`dotnet-mvc/Controllers/AdminController.cs`,
+  `routes.js`).
+- **Mass assignment:** **10 / 15 and 9 / 15** at verdict level, **6 / 15 and 4 / 15** as located findings.
+  Five candidates in .NET (`CustomerController.cs`) and Java (`AccountController.java`,
+  `ProfileServlet.java`) were judged `safe` in both runs: real model misses.
+- **False positives on SAFE markers:** **0** in both iteration-2 runs, for all four categories.
+- **BOLA / injection:** 24 / 25 and 34 / 34 in run A. In run B, 11 of its 14 missed BOLA items sit in
+  one batch whose reply was empty (`routes-001`: `gpt-oss` returned no content, so 15 routes got no
+  verdict); the other 3 are real misses (`perl-cgi/cgi-bin/order.cgi:19` is missed in both runs).
+- **Run-to-run variance is large.** Two runs with the same model, harness and temperature 0.1 differ by
+  up to 13 BOLA items and 3 BFLA findings. Single-run numbers from a local model should not be quoted
+  as product capability.
+- **Precision ~0.7** is measured against markers only. Of the ~25 unmatched findings per run, most are BOLA
+  findings in files that carry BFLA / mass-assignment markers but no BOLA markers
+  (`CustomerController.cs`, `AccountController.java`, `AdminController.cs`), plus the misplaced BFLA /
+  mass findings above. They have not been triaged by a person and are not counted as false positives.
+
+**qwen3.5:9b** could not be measured on this machine: with a 32K context the MLX runner used 22 GB of the
+24 GB RAM, the system swapped (29.7 / 30 GB), and two calls hit the 1-hour timeout (the privilege batch
+holding all BFLA routes, and a verifier batch). The run was stopped after 2 h 50 min with 52 findings
+unverified. Its partial output is archived as `iter1-qwen-a-incomplete/` and says nothing about the
+model's ability.
+
+### Known gaps (follow-up: #319)
+
+1. **Misplaced findings.** Take `file` / `line` of BFLA and mass assignment findings from the Tier 1 route
+   or candidate the auditor judged (`route_id` / `candidate_id`), not from the model's text. Expected to
+   close most of the gap between the marker and verdict columns.
+2. **Empty replies.** Retry a batch once when the reply is empty or unparseable (seen with `gpt-oss`).
+3. **Mass assignment on .NET / Java binding.** Review `knowledge/mass-assignment-patterns.md` for
+   `UpdateModel` / `TryUpdateModel` and Spring `@ModelAttribute` binding into entities.
+4. **Re-run qwen3.5:9b** with 16K context or on a machine with more memory.
+5. **Triage unmatched BOLA findings** and add markers to the fixtures where they are real.
+
+---
+
+## Run 5 — 2026-10-31, kit 0.3.1 (Run 4 follow-up: anchored locations, retries, model context)
+
+Issue #319. Same eval repo, ground truth, hosting and model as Run 4 (`gpt-oss:20b`, Ollama 0.40.13,
+M4 Pro 24 GB, 32K context, local only). Outputs are archived in
+[`assess/lsa/examples/eval-run-5/`](../examples/eval-run-5/).
+
+### Changes since Run 4 iteration 2
+
+| Change | Why (from Run 4) |
+|---|---|
+| Finding `file` / `line` come from the Tier 1 route handler span or candidate the finding references (`operate/ai/anchoring.py`); evidence is re-pointed or dropped; unknown references get `confidence: low` | The auditor judged the right route for 10 / 10 BFLA markers, but wrote paths such as `routes.js`, `app.py`, `dotnet-mvc/...` |
+| An empty or unparseable batch reply is retried once with a JSON-only reminder | One empty reply cost 15 BOLA routes |
+| Each mass assignment candidate carries the bound model's definition (same app and language); an unknown model means `needs_review`, never `safe` (prompt `35-privilege-auditor.md` and the headless 4c prompt) | Five .NET / Java candidates were judged `safe` in both runs because the auditor never saw `Account.java` (`role`, `balance`) |
+
+### Results after verification
+
+| Run | BOLA | Injection | BFLA marker | Mass marker | BFLA verdict | Mass verdict | FP on safe code | Precision | Rejected | Unverified | Retries · parse fail | Min · calls · tokens |
+|---|---|---|---|---|---|---|---|---|---|---|---|---|
+| Run 4 iter 2, A | 24/25 | 34/34 | 8/10 | 6/15 | 10/10 | 10/15 | 0 | 0.729 | 1 | 0 | — · 0 | 57 · 32 · 309K |
+| Run 4 iter 2, B | 11/25 | 32/34 | 5/10 | 4/15 | 10/10 | 9/15 | 0 | 0.693 | 1 | 0 | — · 1 | 56 · 29 · 318K |
+| **Run 5, A** | **24/25** | **34/34** | **10/10** | **10/15** | **10/10** | **15/15** | **0** | **0.818** | 2 | 0 | 0 · 0 | 83 · 32 · 328K |
+| **Run 5, B** | **12/25** | **32/34** | **9/10** | **14/15** | **10/10** | **15/15** | **0** | **0.779** | 3 | 0 | 1 · 1 | 86 · 31 · 369K |
+
+- **BFLA:** 10 / 10 and 9 / 10 as located findings (from 8 and 5). Every anchored finding pointed at a
+  known route or candidate (0 unanchored).
+- **Mass assignment:** the auditor flagged **15 / 15** candidates in both runs (from 10 and 9), including the
+  five .NET / Java bindings. As located findings 10 / 15 and 14 / 15: in run A the model gave a
+  `vulnerable` verdict for four candidates (`php-plain/profile.php` ×2, `CustomerController.cs` ×2)
+  without writing a finding for them.
+- **`php-plain/register.php:4` (MASS marker)** was rejected by the verifier in both runs: `extract($_POST)`
+  is followed by an INSERT of only `email` and `name`. The argument is sound for this fixture as written,
+  so the marker should be reviewed rather than counted as a verifier error.
+- **BOLA in run B (12 / 25)** again lost batch `routes-001` (15 routes). This time the reply was not empty
+  but invalid JSON: the model put code with unescaped `"` into `recommendation.code_before`; the retry
+  failed the same way. The same batch failed in Run 4 run B.
+- **No false positives** on SAFE markers in either run; precision rose to 0.82 / 0.78.
+- Runs are ~45% slower than Run 4 iteration 2 (83–86 min): the privilege and verifier prompts are longer
+  and there are more findings to verify.
+
+### Unmatched BOLA findings (triage, not yet reflected in fixtures)
+
+About 20 BOLA findings per run sit in files with no BOLA marker. By route:
+
+| Route(s) | Assessment |
+|---|---|
+| `Customer/EditName`, `account/rename`, `python-flask` `/users/<uid>/name`, `account/profile` | Likely **real BOLA** not marked: the record ID comes from the request and is not bound to the caller |
+| `Customer/Edit`, `Customer/Refresh`, `account/save`, `account/api`, `/users/<uid>` | Mass-assignment fixtures that also take the ID from the request: plausibly BOLA as well |
+| `Admin/DeleteUser`, `Admin/Export`, `admin/users/{id}/delete`, `delete_user.php`, `promote.php`, `manage_users.php` | **BFLA routes also reported as BOLA**: same flaw seen from a second category; duplicate rather than wrong |
+
+Fixture markers were left unchanged so Run 4 and Run 5 share the same ground truth.
+
+### Remaining gaps
+
+1. **Invalid JSON from code fields:** drop `code_before` / `code_after` from the 4a / 4b output templates
+   (the sanitizer already reduces `recommendation` to a string), or repair unescaped quotes before parsing.
+2. **Verdict without finding:** write a minimal finding for every `vulnerable` / `needs_review` verdict
+   the model did not turn into one.
+3. **Fixtures:** review the `register.php` MASS marker; add BOLA markers for the routes judged real above.
+4. **qwen3.5:9b** still unmeasured: 32K context does not fit in 24 GB next to the OS; needs 16K context or
+   a larger machine.

package/assess/lsa/tests/score_findings.py

@@ -7,13 +7,20 @@ A vulnerable truth item is found (TP) when an active finding (status != rejected
 same category (sql_injection and nosql_injection are treated as one group), and the truth
 line is within --tolerance lines of the finding location or of any evidence line in the same file.
 A safe truth item hit by an active finding counts as a false positive (FP-safe).
+
+--verdicts adds a verdict-level score for bfla and mass_assignment: a truth item counts as found when
+the privilege auditor judged the route or candidate that contains it "vulnerable" or "needs_review",
+whatever file and line the model then wrote into the finding. Routes cover their handler line up to
+the next inventoried route in the same file; mass assignment candidates match their exact line.
 """
 from __future__ import annotations
 
 import argparse
+import glob
 import json
 import sys
 from pathlib import Path
+from typing import Dict, List, Optional, Tuple
 
 
 def group(cat: str) -> str:
@@ -32,11 +39,60 @@ def hits(finding: dict, item: dict, tol: int) -> bool:
     return False
 
 
+FLAGGED = ("vulnerable", "needs_review")
+
+
+def _same_file(a: str, b: str) -> bool:
+    return bool(a) and bool(b) and (a.endswith(b) or b.endswith(a))
+
+
+def _route_for(item: dict, routes: List[Tuple[str, int, str]]) -> Optional[str]:
+    """route_id whose handler span (handler line .. next route in the file) contains the truth line."""
+    in_file = sorted((line, rid) for f, line, rid in routes if _same_file(f, item["file"]) and line <= item["line"])
+    return in_file[-1][1] if in_file else None
+
+
+def verdict_scores(out: Path, truth: List[dict]) -> Dict[str, dict]:
+    routes: List[Tuple[str, int, str]] = []
+    for p in glob.glob(str(out / "routes" / "*.json")):
+        for r in json.loads(Path(p).read_text()).get("routes", []):
+            h = r.get("handler") or {}
+            if h.get("file") and isinstance(h.get("line"), int):
+                routes.append((h["file"], h["line"], r.get("route_id")))
+    cands_path = out / "mass-assignment.candidates.json"
+    cands = json.loads(cands_path.read_text()).get("candidates", []) if cands_path.is_file() else []
+    route_v: Dict[str, str] = {}
+    cand_v: Dict[str, str] = {}
+    for p in glob.glob(str(out / "findings" / "priv-*.json")):
+        doc = json.loads(Path(p).read_text())
+        route_v.update({v.get("route_id"): v.get("verdict") for v in doc.get("privilege_route_verdicts", []) if isinstance(v, dict)})
+        cand_v.update({v.get("candidate_id"): v.get("verdict") for v in doc.get("candidate_verdicts", []) if isinstance(v, dict)})
+
+    def flagged(item: dict) -> bool:
+        if item["category"] == "bfla":
+            return route_v.get(_route_for(item, routes)) in FLAGGED
+        return any(_same_file(c.get("file", ""), item["file"]) and c.get("line") == item["line"]
+                   and cand_v.get(c.get("candidate_id")) in FLAGGED for c in cands)
+
+    rows = {}
+    for cat in ("bfla", "mass_assignment"):
+        vul = [t for t in truth if t["category"] == cat and t["vulnerable"]]
+        safe = [t for t in truth if t["category"] == cat and not t["vulnerable"]]
+        if not vul:
+            continue
+        found = [t for t in vul if flagged(t)]
+        rows[cat] = {"truth_vulnerable": len(vul), "found": len(found), "recall": round(len(found) / len(vul), 3),
+                     "flagged_safe": [f"{t['file']}:{t['line']}" for t in safe if flagged(t)],
+                     "missed": [f"{t['file']}:{t['line']}" for t in vul if t not in found]}
+    return rows
+
+
 def main() -> int:
     ap = argparse.ArgumentParser(description=__doc__)
     ap.add_argument("ground_truth")
     ap.add_argument("out")
     ap.add_argument("--tolerance", type=int, default=3)
+    ap.add_argument("--verdicts", action="store_true", help="also score bfla / mass_assignment by auditor verdicts")
     a = ap.parse_args()
     truth = json.loads(Path(a.ground_truth).read_text())["items"]
     # fixture paths are relative to tests/fixtures; eval repo keeps the same layout
@@ -69,6 +125,8 @@ def main() -> int:
     report = {"per_category": rows, "active_findings": len(g_findings), "matched_findings": len(matched_findings),
               "precision": precision, "unmatched_findings": unmatched,
               "needs_review_not_scored": needs_review}
+    if a.verdicts:
+        report["verdict_level"] = verdict_scores(Path(a.out), truth)
     print(json.dumps(report, indent=2))
     return 0
 

package/assess/lsa/tests/test_adapters.py

@@ -93,6 +93,17 @@ class TestCodexAdapter(unittest.TestCase):
         self.assertEqual(text.count(AGENTS_MD_BEGIN), 1)
         self.assertEqual(text.count(AGENTS_MD_END), 1)
 
+    def test_pre_2_0_markers_are_replaced_in_place(self) -> None:
+        old = (f"# Notes\n\n{build_adapters.LEGACY_AGENTS_MD_BEGIN}\nold block\n"
+               f"{build_adapters.LEGACY_AGENTS_MD_END}\n\n# Footer\n")
+        (self.dest / "AGENTS.md").write_text(old, encoding="utf-8")
+        build("codex", self.dest)
+        text = (self.dest / "AGENTS.md").read_text(encoding="utf-8")
+        self.assertNotIn("komainu-lsa", text)
+        self.assertNotIn("old block", text)
+        self.assertEqual(text.count(AGENTS_MD_BEGIN), 1)
+        self.assertTrue(text.startswith("# Notes\n\n") and text.endswith("# Footer\n"))
+
     def test_unbalanced_markers_fail_before_touching_workspace(self) -> None:
         user = f"# Notes\n{AGENTS_MD_BEGIN}\nimportant user text\n"
         (self.dest / "AGENTS.md").write_text(user, encoding="utf-8")
@@ -187,7 +198,7 @@ class TestGenericAdapterGuard(unittest.TestCase):
 
     def test_contract_carries_generator_marker(self) -> None:
         build("generic", self.dest)
-        self.assertEqual(json.loads(self.contract.read_text(encoding="utf-8"))["generator"], "komainu-lsa")
+        self.assertEqual(json.loads(self.contract.read_text(encoding="utf-8"))["generator"], "komagu-lsa")
 
     def test_rebuild_refreshes_generated_contract(self) -> None:
         build("generic", self.dest)
@@ -197,6 +208,14 @@ class TestGenericAdapterGuard(unittest.TestCase):
         build("generic", self.dest)
         self.assertEqual(json.loads(self.contract.read_text(encoding="utf-8"))["kit_version"], build_adapters.VERSION)
 
+    def test_rebuild_accepts_pre_2_0_generator(self) -> None:
+        build("generic", self.dest)
+        data = json.loads(self.contract.read_text(encoding="utf-8"))
+        data["generator"] = "komainu-lsa"
+        self.contract.write_text(json.dumps(data), encoding="utf-8")
+        build("generic", self.dest)
+        self.assertEqual(json.loads(self.contract.read_text(encoding="utf-8"))["generator"], "komagu-lsa")
+
     def test_foreign_contract_fails_before_touching_workspace(self) -> None:
         self.contract.write_text('{"name": "my-project-contract"}', encoding="utf-8")
         with self.assertRaisesRegex(ValueError, "contract.json"):
@@ -211,7 +230,7 @@ class TestGenericAdapterGuard(unittest.TestCase):
         self.assert_untouched("contract.json")
 
     def test_non_object_contract_fails(self) -> None:
-        self.contract.write_text('["komainu-lsa"]', encoding="utf-8")
+        self.contract.write_text('["komagu-lsa"]', encoding="utf-8")
         with self.assertRaises(ValueError):
             build("generic", self.dest)
 

package/assess/lsa/tests/test_baseline.py

@@ -198,6 +198,14 @@ class TestLoadBaseline(unittest.TestCase):
         entries = load_baseline(self.dir / "prev.sarif")
         self.assertEqual(entries["kfp1:" + "e" * 64]["state"], "open")
 
+    def test_loads_pre_2_0_sarif_fingerprint_key(self) -> None:
+        sarif = to_sarif({"findings": [{**_finding(), "fingerprint": "kfp1:" + "f" * 64}]})
+        for result in sarif["runs"][0]["results"]:
+            fps = result["partialFingerprints"]
+            fps["komainu/v1"] = fps.pop("komagu/v1")
+        write_json(self.dir / "old.sarif", sarif)
+        self.assertIn("kfp1:" + "f" * 64, load_baseline(self.dir / "old.sarif"))
+
     def test_invalid_file_raises(self) -> None:
         (self.dir / "bad.json").write_text("{not json", encoding="utf-8")
         with self.assertRaises(BaselineError):

package/assess/lsa/tests/test_contract_groundwork.py

@@ -156,9 +156,9 @@ class TestLsaSarifExport(unittest.TestCase):
         by_rule = {r["ruleId"]: r for r in self.run["results"]}
         mass_fp = by_rule["lsa/mass_assignment/whole_object_binding"]["partialFingerprints"]
         bfla_fp = by_rule["lsa/bfla/missing_role_check"]["partialFingerprints"]
-        self.assertEqual(mass_fp["komainu/v1"], FINGERPRINT)
+        self.assertEqual(mass_fp["komagu/v1"], FINGERPRINT)
         self.assertEqual(mass_fp["lsaFindingId"], "MASS-R-0002-1")
-        self.assertNotIn("komainu/v1", bfla_fp)
+        self.assertNotIn("komagu/v1", bfla_fp)
 
 
 class TestOperateSarifExport(unittest.TestCase):
@@ -166,16 +166,16 @@ class TestOperateSarifExport(unittest.TestCase):
         sarif = convert_findings_to_sarif([_finding(cwe=285), {**_mass_assignment(), "cwe": 915}])
         run = sarif["runs"][0]
         rule_ids = {r["id"] for r in run["tool"]["driver"]["rules"]}
-        self.assertEqual(rule_ids, {"KOMAINU-BFLA-CWE285", "KOMAINU-MASS-CWE915"})
-        mass = next(r for r in run["results"] if r["ruleId"] == "KOMAINU-MASS-CWE915")
-        self.assertEqual(mass["partialFingerprints"], {"komainu/v1": FINGERPRINT})
-        bfla = next(r for r in run["results"] if r["ruleId"] == "KOMAINU-BFLA-CWE285")
+        self.assertEqual(rule_ids, {"KOMAGU-BFLA-CWE285", "KOMAGU-MASS-CWE915"})
+        mass = next(r for r in run["results"] if r["ruleId"] == "KOMAGU-MASS-CWE915")
+        self.assertEqual(mass["partialFingerprints"], {"komagu/v1": FINGERPRINT})
+        bfla = next(r for r in run["results"] if r["ruleId"] == "KOMAGU-BFLA-CWE285")
         self.assertNotIn("partialFingerprints", bfla)
 
     def test_default_cwe_for_new_categories(self) -> None:
         sarif = convert_findings_to_sarif([_finding(cwe=None), {**_mass_assignment(), "cwe": None}])
         rule_ids = {r["id"] for r in sarif["runs"][0]["tool"]["driver"]["rules"]}
-        self.assertEqual(rule_ids, {"KOMAINU-BFLA-CWE285", "KOMAINU-MASS-CWE915"})
+        self.assertEqual(rule_ids, {"KOMAGU-BFLA-CWE285", "KOMAGU-MASS-CWE915"})
 
 
 class TestSummaryAndReports(unittest.TestCase):

package/assess/lsa/tests/test_eval_run_4.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""Keeps the archived Evaluation Run 4 numbers (Issue #286) in step with the scorer and EVAL-RESULTS.md."""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import sys
+import unittest
+from pathlib import Path
+
+TESTS_DIR = Path(__file__).resolve().parent
+LSA_ROOT = TESTS_DIR.parent
+EVAL_RUN_4_DIR = LSA_ROOT / "examples" / "eval-run-4"
+GROUND_TRUTH = EVAL_RUN_4_DIR / "ground-truth.json"
+RUNS = ("iter0-gptoss-a", "iter1-gptoss-a", "iter1-gptoss-b", "iter1-qwen-a-incomplete", "iter2-gptoss-a", "iter2-gptoss-b")
+# (BOLA, injection, BFLA, mass assignment) found by location, then BFLA / mass found by auditor verdict
+EXPECTED = {
+    "iter2-gptoss-a": {"marker": (24, 34, 8, 6), "verdict": (10, 10)},
+    "iter2-gptoss-b": {"marker": (11, 32, 5, 4), "verdict": (10, 9)},
+}
+
+
+def score(run: str) -> dict:
+    proc = subprocess.run(
+        [sys.executable, str(TESTS_DIR / "score_findings.py"), "--verdicts", str(GROUND_TRUTH), str(EVAL_RUN_4_DIR / run)],
+        capture_output=True, text=True, check=True,
+    )
+    return json.loads(proc.stdout)
+
+
+class TestEvaluationRun4(unittest.TestCase):
+    def test_artifacts_exist(self) -> None:
+        self.assertTrue(GROUND_TRUTH.is_file())
+        for run in RUNS:
+            for name in ("findings.json", "coverage.json", "score.json"):
+                self.assertTrue((EVAL_RUN_4_DIR / run / name).is_file(), f"{run}/{name}")
+
+    def test_archived_scores_match_the_scorer(self) -> None:
+        for run in RUNS:
+            with self.subTest(run=run):
+                archived = json.loads((EVAL_RUN_4_DIR / run / "score.json").read_text(encoding="utf-8"))
+                self.assertEqual(score(run), archived)
+
+    def test_final_harness_numbers_match_eval_results(self) -> None:
+        for run, want in EXPECTED.items():
+            with self.subTest(run=run):
+                report = score(run)
+                pc, vl = report["per_category"], report["verdict_level"]
+                got = tuple(pc[c]["found"] for c in ("bola", "injection", "bfla", "mass_assignment"))
+                self.assertEqual(got, want["marker"])
+                self.assertEqual((vl["bfla"]["found"], vl["mass_assignment"]["found"]), want["verdict"])
+
+    def test_final_harness_has_no_false_positive_on_safe_code(self) -> None:
+        for run in EXPECTED:
+            with self.subTest(run=run):
+                report = score(run)
+                for cat, row in report["per_category"].items():
+                    self.assertEqual(row["false_positive_on_safe_code"], [], cat)
+                for cat, row in report["verdict_level"].items():
+                    self.assertEqual(row["flagged_safe"], [], cat)
+
+    def test_final_harness_records_run_stats(self) -> None:
+        for run in EXPECTED:
+            with self.subTest(run=run):
+                stats = json.loads((EVAL_RUN_4_DIR / run / "ai-run-stats.json").read_text(encoding="utf-8"))
+                self.assertEqual(stats["model"], "gpt-oss:20b")
+                self.assertGreater(stats["totals"]["prompt_tokens"], 0)
+                self.assertEqual(stats["call_errors"], {})
+
+
+if __name__ == "__main__":
+    unittest.main()

package/assess/lsa/tests/test_eval_run_5.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""Keeps the archived Evaluation Run 5 numbers (Issue #319) in step with the scorer and EVAL-RESULTS.md."""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import sys
+import unittest
+from pathlib import Path
+
+TESTS_DIR = Path(__file__).resolve().parent
+LSA_ROOT = TESTS_DIR.parent
+EVAL_RUN_5_DIR = LSA_ROOT / "examples" / "eval-run-5"
+GROUND_TRUTH = EVAL_RUN_5_DIR / "ground-truth.json"
+RUNS = ("gptoss-a", "gptoss-b")
+# (BOLA, injection, BFLA, mass assignment) found by location, then BFLA / mass found by auditor verdict
+EXPECTED = {
+    "gptoss-a": {"marker": (24, 34, 10, 10), "verdict": (10, 15)},
+    "gptoss-b": {"marker": (12, 32, 9, 14), "verdict": (10, 15)},
+}
+
+def score(run: str) -> dict:
+    proc = subprocess.run(
+        [sys.executable, str(TESTS_DIR / "score_findings.py"), "--verdicts", str(GROUND_TRUTH), str(EVAL_RUN_5_DIR / run)],
+        capture_output=True, text=True, check=True,
+    )
+    return json.loads(proc.stdout)
+
+
+class TestEvaluationRun5(unittest.TestCase):
+    def test_artifacts_exist(self) -> None:
+        self.assertTrue(GROUND_TRUTH.is_file())
+        for run in RUNS:
+            for name in ("findings.json", "coverage.json", "score.json"):
+                self.assertTrue((EVAL_RUN_5_DIR / run / name).is_file(), f"{run}/{name}")
+
+    def test_archived_scores_match_the_scorer(self) -> None:
+        for run in RUNS:
+            with self.subTest(run=run):
+                archived = json.loads((EVAL_RUN_5_DIR / run / "score.json").read_text(encoding="utf-8"))
+                self.assertEqual(score(run), archived)
+
+    def test_final_harness_numbers_match_eval_results(self) -> None:
+        for run, want in EXPECTED.items():
+            with self.subTest(run=run):
+                report = score(run)
+                pc, vl = report["per_category"], report["verdict_level"]
+                got = tuple(pc[c]["found"] for c in ("bola", "injection", "bfla", "mass_assignment"))
+                self.assertEqual(got, want["marker"])
+                self.assertEqual((vl["bfla"]["found"], vl["mass_assignment"]["found"]), want["verdict"])
+
+    def test_final_harness_has_no_false_positive_on_safe_code(self) -> None:
+        for run in EXPECTED:
+            with self.subTest(run=run):
+                report = score(run)
+                for cat, row in report["per_category"].items():
+                    self.assertEqual(row["false_positive_on_safe_code"], [], cat)
+                for cat, row in report["verdict_level"].items():
+                    self.assertEqual(row["flagged_safe"], [], cat)
+
+    def test_final_harness_records_run_stats(self) -> None:
+        for run in EXPECTED:
+            with self.subTest(run=run):
+                stats = json.loads((EVAL_RUN_5_DIR / run / "ai-run-stats.json").read_text(encoding="utf-8"))
+                self.assertEqual(stats["model"], "gpt-oss:20b")
+                self.assertGreater(stats["totals"]["prompt_tokens"], 0)
+                self.assertEqual(stats["call_errors"], {})
+                self.assertEqual(stats["anchoring"]["unanchored"], [])
+
+
+if __name__ == "__main__":
+    unittest.main()

package/assess/lsa/tests/test_privilege_tier2.py

@@ -242,6 +242,36 @@ class TestScoring(unittest.TestCase):
             self.assertEqual(report["per_category"]["mass_assignment"]["recall"], 0.0)
             self.assertNotIn("bola", report["per_category"])
 
+    def test_verdict_level_ignores_misplaced_findings(self) -> None:
+        """--verdicts credits the auditor's route / candidate verdict even when the finding has a wrong path (#286)."""
+        with tempfile.TemporaryDirectory() as tmp:
+            truth = Path(tmp) / "gt.json"
+            write_json(truth, {"items": [
+                {"file": "admin.js", "line": 9, "category": "bfla", "vulnerable": True, "label": "BFLA"},
+                {"file": "admin.js", "line": 13, "category": "bfla", "vulnerable": False, "label": "SAFE-BFLA"},
+                {"file": "p.php", "line": 6, "category": "mass_assignment", "vulnerable": True, "label": "MASS"},
+                {"file": "p.php", "line": 9, "category": "mass_assignment", "vulnerable": False, "label": "SAFE-MASS"},
+            ]})
+            out = Path(tmp) / "out"
+            write_json(out / "findings.json", {"findings": [
+                {"finding_id": "BFLA-R-1-1", "category": "bfla", "status": "confirmed",
+                 "location": {"file": "routes.js", "start_line": 1}, "evidence": []}]})
+            write_json(out / "routes" / "routes-001.json", {"routes": [
+                {"route_id": "R-1", "handler": {"file": "admin.js", "line": 8}},
+                {"route_id": "R-2", "handler": {"file": "admin.js", "line": 12}}]})
+            write_json(out / "mass-assignment.candidates.json", {"candidates": [
+                {"candidate_id": "M-1", "file": "p.php", "line": 6}]})
+            write_json(out / "findings" / "priv-privilege-001.json", {
+                "privilege_route_verdicts": [{"route_id": "R-1", "verdict": "vulnerable"}],
+                "candidate_verdicts": [{"candidate_id": "M-1", "verdict": "needs_review"}], "findings": []})
+            res = subprocess.run([sys.executable, str(SCORE), "--verdicts", str(truth), str(out)], capture_output=True, text=True)
+            report = json.loads(res.stdout)
+            self.assertEqual(report["per_category"]["bfla"]["found"], 0)
+            self.assertEqual(report["verdict_level"]["bfla"], {"truth_vulnerable": 1, "found": 1, "recall": 1.0,
+                                                              "flagged_safe": [], "missed": []})
+            self.assertEqual(report["verdict_level"]["mass_assignment"]["found"], 1)
+            self.assertEqual(report["verdict_level"]["mass_assignment"]["flagged_safe"], [])
+
 
 if __name__ == "__main__":
     unittest.main()