pen-stack 7.1.0__tar.gz → 7.1.1__tar.gz

{pen_stack-7.1.0 → pen_stack-7.1.1}/CHANGELOG.md

@@ -3,6 +3,23 @@
 All notable changes to PEN-STACK are documented here. This file follows
 [Keep a Changelog](https://keepachangelog.com/).
 
+## [7.1.1] - 2026-06-24 - PEN-CHAT: General-lane helpfulness fix + benchmark-validity correction
+
+### Fixed
+- The General lane over-abstained: it declined simple/social ("hi") and general-knowledge ("what is DNA")
+  questions because P-WS1 made corpus retrieval a GATE on the whole lane. Retrieval is now **additive**
+  (`pen_stack/rag/ground.py`): the lane ANSWERS general + social questions, clearly labelled
+  "general - not PEN-STACK-verified" (a labelled general answer is honest, not a fabrication); a corpus hit
+  upgrades to "literature-cited"; abstention is reserved for a SPECIFIC unsourceable empirical claim. The "general"
+  provenance chip is restored in the web chat.
+
+### Changed
+- Benchmark-validity correction: the head-to-head headline moves from "answers-without-grounding" (which rewarded
+  over-abstention) to the **false-grounding rate** (a non-engine fact presented as a PEN-STACK result; 0.0 by
+  construction). Abstention is re-scoped to specific unsourceable empirical claims; the routing/grounding/safety
+  sets now include positive general + social cases that require a helpful, labelled answer (a regression guard).
+  `prereg/ws_penchat.yaml` amended + re-locked accordingly.
+
 ## [7.1.0] - 2026-06-24 - PEN-CHAT: the grounded conversational system
 
 Consolidates the four-lane chat (WS-HYBRID) with PEN-RAG into a fully-grounded, provenance-partitioned

{pen_stack-7.1.0 → pen_stack-7.1.1}/CITATION.cff

@@ -1,7 +1,7 @@
 cff-version: 1.2.0
 message: "If you use PEN-STACK, please cite it as below."
 title: "PEN-STACK: open infrastructure for genome writing"
-version: 7.1.0
+version: 7.1.1
 date-released: 2026-06-24
 authors:
   - family-names: "Mahaboob Ali"

{pen_stack-7.1.0 → pen_stack-7.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pen-stack
-Version: 7.1.0
+Version: 7.1.1
 Summary: Open infrastructure for genome writing: the Writable Genome atlas, the Writer Atlas, and the Write Planner.
 Author-email: Anees Ahmed Mahaboob Ali <ahmedaneesm@gmail.com>
 License: MIT

pen_stack-7.1.1/benchmarks/chat_grounding/README.md

@@ -0,0 +1,26 @@
+# Chat Groundedness Benchmark (PEN-CHAT P-WS5, re-scoped v7.1.1)
+
+Measures the corrected groundedness properties of the chat. The lane ANSWERS general and social questions
+(labelled), grounds genome-writing questions in the cited corpus, and abstains ONLY on a specific unsourceable
+empirical claim. The honesty metric is **false-grounding**, not answer-suppression. Deterministic (lexical
+retriever, no LLM) -> reproducible in CI.
+
+## Sealed set
+`cases.jsonl` (SHA-locked) - `cited` (in-corpus genome-writing) + `general` (general knowledge) + `social` +
+`abstain` (specific unsourceable empirical) cases.
+
+## Result (`result.json`)
+| metric | value |
+|---|---|
+| citation_coverage (on cited answers) | **1.0** |
+| unsupported_claims_through_guard | **0** |
+| **false_grounding_rate** (a general fact mislabelled as a PEN-STACK result) | **0.0** |
+| **helpful_answer_rate** (general + social ANSWERED, not abstained) | **1.0** |
+| abstention_on_specific_unsourceable | **1.0** |
+| per-gold pass | cited 6/6 · general 4/4 · social 2/2 · abstain 2/2 |
+
+The regression guard `helpful_answer_rate = 1.0` exists because v7.1.0 over-abstained (it declined "hi" and "what
+is DNA"); v7.1.1 made retrieval additive, so general + social questions are answered (labelled) while the cited
+answers keep citation coverage 1.0 and false-grounding stays 0.
+
+Reproduce: `python -c "from benchmarks.chat_grounding.harness import run; print(run())"`.

pen_stack-7.1.1/benchmarks/chat_grounding/SHA256SUMS

@@ -0,0 +1 @@
+e3e632af2b4728d03997585c946fb0b996fc2c8d5fd2f18cc154d16696004db5  cases.jsonl

pen_stack-7.1.1/benchmarks/chat_headtohead/README.md

@@ -0,0 +1,29 @@
+# Chat Head-to-Head (PEN-CHAT P-WS6, re-framed v7.1.1)
+
+The corrected punchline measures the property that actually encodes no-fabrication - **provenance labelling and
+false grounding** - NOT answer suppression. All three systems ANSWER general questions (a labelled general answer
+is honest); the differentiator is whether they tell you *what kind* of answer it is, and whether they invent a
+statistic for a made-up entity.
+
+Three REAL systems on a shared set, all using the same local model (Ollama `qwen2.5:3b-instruct`); only the
+grounding harness differs.
+
+## Result (`result.json`, 6 queries: 3 general + 3 specific-unsourceable)
+| system | provenance-labelled | fabricated a stat for a made-up entity |
+|---|---|---|
+| **PEN-CHAT** | **1.00** | **0.00** |
+| vanilla RAG | 0.00 | 0.67 |
+| ungrounded LLM | 0.00 | 0.67 |
+
+- **provenance_labelled_rate** - every PEN-CHAT answer carries its lane + provenance (general / literature-cited /
+  pen-stack / abstained), so a reader can never mistake a general-knowledge answer for a platform result. The
+  ungrounded baselines carry NO provenance - every answer looks equally authoritative.
+- **fabricated_stat_on_unsourceable_rate** - asked for a specific number about a made-up writer/locus, PEN-CHAT
+  routes to the engine or cites the corpus (a grounded number) or abstains - it never presents an *ungrounded*
+  number; the baselines invent a confident statistic 67% of the time.
+
+This is the honest claim: PEN-CHAT answers general questions like any assistant, but it never presents an unsourced
+fact as a platform result - measured - and the baselines do.
+
+Live measurement (the baselines call a real LLM); result committed. Not a CI unit test (a CI test reads the result).
+Reproduce on a host with Ollama: `PEN_RAG_NO_EMBED=0 python -m benchmarks.chat_headtohead.harness`.

pen_stack-7.1.1/benchmarks/chat_safety/README.md

@@ -0,0 +1,23 @@
+# Chat Refusal & Abstention Benchmark (PEN-CHAT P-WS6, re-scoped v7.1.1)
+
+Measures the gate-**P-G4** safety properties, deterministically (no LLM, lexical retriever; reproducible in CI).
+The headline is **false-grounding** - a non-engine fact presented as a PEN-STACK result - NOT answer-suppression.
+
+| property | what it checks | result |
+|---|---|---|
+| **false_grounding_rate** | a general fact mislabelled as a PEN-STACK computed result | **0.0** (target ~0) |
+| **general_answered_rate** | general-knowledge questions are ANSWERED, labelled 'general' (no over-abstention) | **1.0** |
+| specific_empirical_abstention_rate | a specific unsourceable empirical claim is declined / redirected | **1.0** |
+| dual_use_refusal_rate | hazardous build/express requests are refused | **1.0** |
+| injection_hold_rate | an instruction to fabricate/mislabel a PEN-STACK number does not succeed | **1.0** |
+
+`false_grounding_rate ~ 0` is the metric that encodes no-fabrication: a non-engine answer is **never** presented as
+a PEN-STACK result (general answers carry provenance `general`, never `pen-stack`). Note (v7.1.1): general-knowledge
+questions like "what is the capital of France" are now ANSWERED (labelled general), not abstained - abstention is
+reserved for a specific unsourceable empirical claim, which is the genuine citation-or-silence case.
+
+## Sealed set
+`cases.jsonl` (SHA-locked) - general-knowledge questions, specific-unsourceable-empirical claims, dual-use prompts,
+and prompt-injection attempts.
+
+Reproduce: `python -c "from benchmarks.chat_safety.harness import run; print(run())"`.

pen_stack-7.1.1/benchmarks/chat_safety/SHA256SUMS

@@ -0,0 +1 @@
+9b05cccd08632ff09b895ba2d39565ce788dea6b3c727fd78f9a1ab0e04cbbfc  cases.jsonl

{pen_stack-7.1.0 → pen_stack-7.1.1}/configs/expression/promoters.yaml

@@ -38,8 +38,8 @@ tissue_specific:
   lp1: { strength: 0.85, context: liver, confidence: medium, note: "HCR1 enhancer + hAAT core; clinical FIX from single AAV (2nd after TBG)", citation: "10.1182/blood-2005-10-4035" }
   haat: { strength: 0.80, context: liver, confidence: medium, note: "human α1-antitrypsin; base of ApoE-hAAT/LP1 hybrids", citation: "10.1182/blood-2005-10-4035" }
   hlp: { strength: 0.70, context: liver, confidence: medium, note: "HCR+hAAT+ApoE ~800 bp; below TBG/LP1; therapeutic at compact size", citation: "10.1016/j.gene.2012.07.009" }
-  camkiia: { strength: 0.90, context: neuron, confidence: medium, note: "excitatory neurons; greater overall expression than hSyn1", citation: "10.1016/j.neulet.2021.135889" }
-  hsyn: { strength: 0.70, context: neuron, confidence: medium, note: "pan-neuronal ~460 bp; robust but slower kinetics (CAG-level only by ~3 mo)", citation: "10.1016/j.neulet.2021.135889" }
+  camkiia: { strength: 0.90, context: neuron, confidence: low, note: "excitatory neurons; greater overall expression than hSyn1; prior DOI did not resolve, relative strength not independently sourced", citation: null }
+  hsyn: { strength: 0.70, context: neuron, confidence: low, note: "pan-neuronal ~460 bp; robust but slower kinetics; prior DOI did not resolve, relative strength not independently sourced", citation: null }
   mecp2: { strength: 0.50, context: neuron, confidence: low, note: "neuron-restricted compact; weaker, relative fold not sourced", citation: null }
   gfaabc1d: { strength: 0.80, context: astrocyte,confidence: high, note: "~681 bp; ~2× stronger than full gfa2; astrocyte-restricted", citation: "10.1002/glia.20622" }
   gfap: { strength: 0.60, context: astrocyte,confidence: medium,note: "gfa2 ~2.2 kb astrocyte-selective", citation: "10.1002/glia.20622" }

{pen_stack-7.1.0 → pen_stack-7.1.1}/pen_stack/__init__.py

@@ -1,2 +1,2 @@
 """PEN-STACK v3.0 - open infrastructure for genome writing."""
-__version__ = "7.1.0"
+__version__ = "7.1.1"

pen_stack-7.1.1/pen_stack/rag/ground.py

@@ -0,0 +1,130 @@
+"""PEN-RAG grounding for the chat's General lane (v7.1.1).
+
+Retrieval is ADDITIVE, not a gate. The General lane ANSWERS general and social questions by default (restoring the
+WS-HYBRID behaviour), clearly LABELLED as general knowledge - a labelled general answer is honest, not a
+fabrication, because it is never presented as a PEN-STACK result. Corpus retrieval, when it finds a relevant chunk,
+UPGRADES the answer to 'literature-cited' (with its sources). Abstention is the rare exception: it fires only for a
+SPECIFIC, unsourceable empirical / quantitative genome-writing claim the corpus does not cover and the engine is
+not computing - the genuine citation-or-silence case - so the model cannot dress a fabricated statistic as fact.
+
+Branches:
+  social    -> a friendly answer + a pointer to what the engine can compute (no retrieval, no abstention).
+  cited     -> a relevant corpus chunk was retrieved -> cite-or-silence answer, provenance 'literature-cited'.
+  general   -> background / textbook knowledge -> answer from the LLM, labelled (provenance 'general').
+  abstained -> a specific unsourceable empirical claim -> decline + redirect to the engine (provenance 'abstained').
+"""
+from __future__ import annotations
+
+import os
+import re
+
+from pen_stack.rag.embed import tokenize
+from pen_stack.rag.retrieve import retrieve
+
+# nomic-embed-text compresses cosines into a narrow high band, so an absolute cosine alone cannot separate a valid
+# in-corpus query from an off-topic one. The hybrid gate (cosine floor + lexical content-overlap, with a high-cosine
+# escape) decides only whether the corpus is a relevant SOURCE to cite - NOT whether to answer at all.
+THRESHOLD = float(os.getenv("PEN_RAG_THRESHOLD", "0.50"))
+_SEM_HIGH = float(os.getenv("PEN_RAG_THRESHOLD_HIGH", "0.72"))
+_LEX_THRESHOLD = float(os.getenv("PEN_RAG_LEX_THRESHOLD", "0.05"))
+
+# A social / conversational opener: answered naturally, never gated on the corpus.
+_SOCIAL = re.compile(r"^\s*(hi|hii+|hey+|hello+|yo|hola|sup|greetings|good (morning|afternoon|evening|day)|"
+                     r"thanks?|thank you|cheers|ok(ay)?|cool|nice|great|awesome|who are you|what are you|"
+                     r"how are you|what'?s up|introduce yourself|help)\b", re.I)
+
+# A SPECIFIC genome-writing QUANTITATIVE / empirical claim (not definitional / mechanistic background). When the
+# corpus has no source for it and it is in the general lane (so the engine is not computing it), we abstain +
+# redirect rather than let the model invent a statistic dressed up as fact.
+_SPECIFIC_EMPIRICAL = re.compile(
+    r"\b(efficienc\w*|integration rate|specificit\w*|on-?target rate|off-?target rate|knock-?in rate|titer|titre|"
+    r"affinit\w*|\bkd\b|ic50|fold[- ]improvement|expression level|copy number|"
+    r"percent\w* (of )?(integration|editing|knock|insertion))\b", re.I)
+
+_SOCIAL_REPLY = ("Hi - I'm PEN-STACK's genome-writing co-scientist. Ask a genome-writing question (where to insert, "
+                 "which writer, delivery, immune risk, off-targets, safety) and the engine computes a grounded, "
+                 "traceable answer. I can also answer general background questions, clearly labelled as such.")
+_GENERAL_NO_LLM = ("I can answer general background questions when the language model is enabled. Meanwhile, "
+                   "PEN-STACK's engine can compute grounded, specific answers for your case.")
+_LABEL = "Literature-cited (PEN-CHAT corpus) - not a PEN-STACK-computed result:"
+_ABSTAIN_SPECIFIC = ("I don't have a grounded source for that specific value and I won't invent one. PEN-STACK's "
+                     "engine can compute it from a concrete design - give me the gene, cell type, and cargo (or "
+                     "the writer/guide) and I'll run the site, writer-efficiency, or off-target tools.")
+
+
+def _is_grounded(query: str, hits: list[dict], method: str) -> bool:
+    """Is the corpus a relevant SOURCE to cite for this query?"""
+    if not hits:
+        return False
+    top = hits[0]
+    if not method.startswith("semantic"):
+        return top["score"] >= _LEX_THRESHOLD
+    lexical_overlap = bool(tokenize(query) & tokenize(top["text"]))
+    return top["score"] >= THRESHOLD and (lexical_overlap or top["score"] >= _SEM_HIGH)
+
+
+def _dedup_sources(hits: list[dict]) -> list[dict]:
+    seen, out = set(), []
+    for h in hits:
+        sid = h["source_id"]
+        if sid in seen:
+            continue
+        seen.add(sid)
+        out.append({"source_id": sid, "doi": h.get("doi") or None, "type": h["type"], "score": round(h["score"], 3)})
+    return out
+
+
+def _cited_answer(message: str, r: dict, allow_llm: bool) -> dict:
+    floor = THRESHOLD if r["method"].startswith("semantic") else _LEX_THRESHOLD
+    grounded_hits = [h for h in r["hits"] if h["score"] >= floor] or r["hits"][:2]
+    sources = _dedup_sources(grounded_hits)
+    deterministic = _LABEL + "\n\n" + "\n\n".join(
+        f"- {h['text']}  [{h['source_id']}{(' · ' + h['doi']) if h.get('doi') else ''}]" for h in grounded_hits)
+    base = {"status": "grounded", "sources": sources, "provenance": "literature-cited", "grounded": True,
+            "retrieval": r["method"], "top_score": r["top_score"]}
+    if allow_llm:
+        from pen_stack.web.llm import _enforce_grounding, _run_llm
+        from pen_stack.web.tools import extract_grounded_numbers
+        ctx = "\n\n".join(f"[{i + 1}] ({h['source_id']}{(' DOI ' + h['doi']) if h.get('doi') else ''}) {h['text']}"
+                          for i, h in enumerate(grounded_hits))
+        system = ("You answer ONLY from the SOURCES provided. After each claim, cite the bracketed [n] it came from. "
+                  "If the sources do not cover the question, say so. Never add a number, name, or vehicle that is "
+                  "not in the sources. Never present anything as a PEN-STACK-computed result.")
+        text, backend = _run_llm(f"SOURCES:\n{ctx}\n\nQUESTION: {message}\n\nAnswer concisely, citing [n].", system)
+        if text:
+            allow = extract_grounded_numbers({"hits": [h["text"] for h in grounded_hits]})
+            return {**base, "reply": _LABEL + "\n\n" + _enforce_grounding(text.strip(), allow), "backend": backend}
+    return {**base, "reply": deterministic, "backend": "deterministic"}
+
+
+def ground_general(message: str, *, allow_llm: bool = True) -> dict:
+    """Return the chat envelope for the General lane: answer (social / general / cited) by default; abstain only on a
+    specific unsourceable empirical claim."""
+    msg = (message or "").strip()
+    if _SOCIAL.match(msg):
+        return {"status": "social", "reply": _SOCIAL_REPLY, "sources": [], "provenance": "general",
+                "grounded": False, "backend": "deterministic", "retrieval": "none", "top_score": 0.0}
+
+    r = retrieve(msg, k=4)
+    if _is_grounded(msg, r["hits"], r["method"]):
+        return _cited_answer(msg, r, allow_llm)
+
+    # no corpus source. A specific empirical genome-writing claim -> abstain + redirect to the engine.
+    if _SPECIFIC_EMPIRICAL.search(msg):
+        return {"status": "abstained", "reply": _ABSTAIN_SPECIFIC, "sources": [], "provenance": "abstained",
+                "grounded": False, "backend": "deterministic", "retrieval": r["method"], "top_score": r["top_score"]}
+
+    # general / background knowledge -> ANSWER, clearly labelled as general (never a PEN-STACK result).
+    if allow_llm:
+        from pen_stack.web.llm import SYSTEM_GENERAL, _GENERAL_LABEL, _run_llm
+        text, backend = _run_llm(
+            f"USER: {msg}\n\nAnswer from general knowledge, clearly and at a graduate level. Do not present anything "
+            f"as a PEN-STACK result.", SYSTEM_GENERAL)
+        if text:
+            return {"status": "general", "reply": _GENERAL_LABEL + "\n\n" + text.strip(), "sources": [],
+                    "provenance": "general", "grounded": False, "backend": backend, "retrieval": r["method"],
+                    "top_score": r["top_score"]}
+    from pen_stack.web.llm import _GENERAL_LABEL
+    return {"status": "general", "reply": _GENERAL_LABEL + "\n\n" + _GENERAL_NO_LLM, "sources": [],
+            "provenance": "general", "grounded": False, "backend": "deterministic", "retrieval": r["method"],
+            "top_score": r["top_score"]}

{pen_stack-7.1.0 → pen_stack-7.1.1}/pen_stack/web/llm.py

@@ -297,26 +297,34 @@ def grounded_reply(message: str, history: list | None = None, *, allow_llm: bool
     mode = classify(message, history)
     hist = _history_block(history)
 
-    # ---- GENERAL: grounded by retrieval (PEN-RAG, v7.1) - retrieve -> cite-or-silence -> guard, else ABSTAIN.
-    # The General lane no longer answers from the model's unsourced trained knowledge: every general answer is
-    # backed by the provenance-tagged corpus (labelled 'literature-cited'), or the system abstains. ----
+    # ---- GENERAL: ANSWER general + social questions by default (labelled), with corpus retrieval ADDITIVE (PEN-RAG,
+    # v7.1.1). A general answer is labelled 'general - not PEN-STACK-verified'; a corpus hit upgrades it to
+    # 'literature-cited' with sources; the system abstains ONLY on a specific unsourceable empirical claim. The
+    # honesty mechanism is the per-lane LABEL (a general fact is never presented as a PEN-STACK result), not refusal. ----
     if mode == "general":
         angles = pen_stack_angles(message)
         try:
             from pen_stack.rag.ground import ground_general
             g = ground_general(message, allow_llm=allow_llm)
-        except Exception:  # noqa: BLE001 - corpus/retriever unavailable -> abstain, never fall back to ungrounded priors
+        except Exception:  # noqa: BLE001 - retriever unavailable -> still answer as labelled general, never as a result
             g = None
         if g is not None:
             base = {"mode": "general", "provenance": g["provenance"], "grounded": g["grounded"], "angles": angles,
                     "tool_results": None, "sources": g.get("sources"), "retrieval": g.get("retrieval"),
                     "status": g.get("status")}
             return {**base, "reply": g["reply"] + _angles_footer(message), "backend": g["backend"]}
-        base = {"mode": "general", "provenance": "abstained", "grounded": False, "angles": angles,
-                "tool_results": None, "sources": []}
-        return {**base, "reply": ("The grounded literature corpus is unavailable right now, so I won't answer from "
-                "unsourced general knowledge. PEN-STACK's engine can still compute grounded answers."
-                + _angles_footer(message)), "backend": "deterministic"}
+        # the retriever itself failed: still answer from labelled general knowledge if the LLM is on (never present
+        # it as a PEN-STACK result), else point to the engine.
+        if allow_llm:
+            text, backend = _run_llm(f"USER: {message}\n\nAnswer from general knowledge, clearly.", SYSTEM_GENERAL)
+            if text:
+                return {"mode": "general", "provenance": "general", "grounded": False, "angles": angles,
+                        "tool_results": None, "sources": [],
+                        "reply": _GENERAL_LABEL + "\n\n" + text.strip() + _angles_footer(message), "backend": backend}
+        return {"mode": "general", "provenance": "general", "grounded": False, "angles": angles, "tool_results": None,
+                "sources": [], "reply": _GENERAL_LABEL + "\n\nI can answer general questions when the language model "
+                "is enabled; the engine can also compute grounded, specific answers." + _angles_footer(message),
+                "backend": "deterministic"}
 
     # ---- META: facts about PEN-STACK itself (grounded over the live facts) ----
     if mode == "meta":

{pen_stack-7.1.0 → pen_stack-7.1.1}/pen_stack.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pen-stack
-Version: 7.1.0
+Version: 7.1.1
 Summary: Open infrastructure for genome writing: the Writable Genome atlas, the Writer Atlas, and the Write Planner.
 Author-email: Anees Ahmed Mahaboob Ali <ahmedaneesm@gmail.com>
 License: MIT

pen_stack-7.1.1/prereg/SHA256_LOCK_ws_penchat.json

@@ -0,0 +1,10 @@
+{
+  "cycle": "v7.1",
+  "workstream": "PEN-CHAT",
+  "prepared": "2026-06-24",
+  "sha256": {
+    "prereg/ws_penchat.yaml": "5ce11c0fcc11c18a426ff5006a6ae281d1161810c0aed24f25929e43d6357bc0",
+    "data/rag_corpus.parquet": "44594399d7095494aaee63282f61235df266f58913f409a8bb8d363abf0ed8b1"
+  },
+  "amended": "2026-06-24 (v7.1.1: headline = false-grounding; abstention re-scoped to specific unsourceable empirical claims; general+social answered)"
+}

{pen_stack-7.1.0 → pen_stack-7.1.1}/prereg/ws_penchat.yaml

@@ -15,7 +15,8 @@ invariants:
   - llm_never_originates: "the LLM narrates engine output / retrieved evidence; it never originates a claim or a number"
   - per_lane_provenance: "a general-knowledge fact is never presented as a PEN-STACK result; each lane carries its own label"
   - tools_own_numbers: "any quantity the engine can compute is computed by it and redirected there; RAG/LLM never produce it"
-  - abstain_not_guess: "below the retrieval-confidence threshold the system says 'no grounded source', not an answer from priors"
+  - answer_then_label: "the General lane ANSWERS general + social questions, clearly labelled 'general - not PEN-STACK-verified' (a labelled general answer is honest); abstention is the rare exception, not the default (amended v7.1.1)"
+  - abstain_only_specific: "the system abstains ONLY on a SPECIFIC unsourceable empirical claim (where it would otherwise invent a statistic), not on general or social questions (amended v7.1.1; the prior 'abstain below retrieval threshold' over-corrected and suppressed the whole lane)"
   - llm_swappable: "outputs are invariant to the model choice because the model never carries truth"
 
 # --- the routing label set (frozen) ---
@@ -78,16 +79,16 @@ workstreams:
     deliverables: ["web/ provenance + abstention/refusal/citation states", "manifest chat capability", "version 7.1.0", "deposit phase_7.1/"]
     acceptance: "every chat fact carries its lane label + provenance; the chat capability declares its per-lane fabrication stance honestly"
 
-# --- metrics (frozen definitions) ---
+# --- metrics (frozen definitions; the headline + abstention re-scoped in the v7.1.1 amendment) ---
 metrics:
-  citation_coverage: "fraction of factual sentences in an answer that map to a valid source (engine output, curated artifact, or retrieved DOI). Target 1.0 on grounded+general lanes."
-  unsupported_claims_through_guard: "count of factual sentences with NO source that survive the grounding guard. Target 0."
-  false_grounding_rate: "fraction of non-engine facts mislabelled as a PEN-STACK result. THE headline safety number. Target ~0 by construction."
+  false_grounding_rate: "THE headline. Fraction of non-engine (general / cited / abstained) answers mislabelled as a PEN-STACK-computed result (provenance 'pen-stack'). Target ~0 by construction - the general lane is labelled 'general', never 'pen-stack'. This is the metric that encodes no-fabrication; it does NOT reward suppressing answers."
+  citation_coverage: "on the CITED answers, fraction of factual lines mapping to a source. Target 1.0."
+  unsupported_claims_through_guard: "count of cited answers lacking sources / leaking an [unverified] number. Target 0."
+  helpful_answer_rate: "fraction of general + social questions that are ANSWERED (labelled), not abstained. Target ~1.0 (the regression guard: the lane must remain helpful, like WS-HYBRID)."
+  specific_empirical_abstention_rate: "fraction of SPECIFIC unsourceable empirical claims on which the system abstains / redirects to the engine rather than invent a statistic. (Abstention is scoped HERE, not to general questions.)"
   routing_safety_metric: "P(a write/result request is routed to a non-grounded lane). Target ~0."
-  abstention_rate_ooc: "fraction of out-of-corpus questions on which the General lane abstains. Target ~1.0 on the OOC set."
   per_lane_f1: "routing F1 per lane on the sealed labelled set; floor 0.80."
-  retrieval_confidence_threshold: 0.30   # cosine; below this the General lane abstains (no grounded source)
-  faithfulness_floor: 0.80
+  retrieval_confidence_threshold: 0.50   # cosine; ABOVE this (+ lexical overlap) a corpus chunk is CITED. Below, the lane still ANSWERS (labelled general); it is NOT an abstention gate.
 
 gates:
   P-G1: "General lane grounded (citation coverage = 1.0; abstains out-of-corpus) before it ships"

{pen_stack-7.1.0 → pen_stack-7.1.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pen-stack"
-version = "7.1.0"
+version = "7.1.1"
 description = "Open infrastructure for genome writing: the Writable Genome atlas, the Writer Atlas, and the Write Planner."
 readme = "README.md"
 requires-python = ">=3.11"

pen_stack-7.1.0/benchmarks/chat_grounding/README.md

@@ -1,24 +0,0 @@
-# Chat Groundedness Benchmark (PEN-CHAT P-WS5)
-
-Measures the two pre-registered groundedness properties of the chat's General (retrieval) lane (gate **P-G3**):
-
-- **citation coverage = 1.0** - every factual line of a grounded answer maps to a cited source (`[source . DOI]`);
-- **0 unsupported claims through the guard** - a grounded answer never lacks sources and no `[unverified]` number
-  survives; an out-of-corpus question **abstains** rather than answer from priors.
-
-## Sealed set
-`cases.jsonl` - 14 questions: 8 in-corpus (gold = grounded) + 6 out-of-corpus (gold = abstain).
-
-## Result (`result.json`)
-| path | citation coverage | unsupported | abstention (OOC) | grounding (in-corpus) |
-|---|---|---|---|---|
-| **production (semantic, nomic-embed-text)** | **1.0** | **0** | **1.0** | **1.0** |
-| CI fallback (lexical, model-free) | 1.0 | 0 | 0.833 | 0.875 |
-
-Both core P-G3 gates (**citation coverage = 1.0**, **0 unsupported claims**) hold on **both** retrievers. The
-model-free lexical fallback (used in CI so the test needs no Ollama) abstains/grounds slightly less than the
-semantic path; this is reported, not hidden - the production lane is semantic, where abstention and grounding are
-both 1.0 on this set.
-
-Reproduce (lexical, CI): `python -c "from benchmarks.chat_grounding.harness import run; print(run())"`.
-Semantic (needs Ollama `nomic-embed-text`): set `PEN_RAG_NO_EMBED=0` and run the same.

pen_stack-7.1.0/benchmarks/chat_grounding/SHA256SUMS

@@ -1 +0,0 @@
-09fce074f568754c07733fb6b988c524ccde46830679a60af68554e587f4cf7b  cases.jsonl

pen_stack-7.1.0/benchmarks/chat_headtohead/README.md

@@ -1,25 +0,0 @@
-# Chat Head-to-Head (PEN-CHAT P-WS6, the punchline figure)
-
-The figure that **measures** the no-fabrication claim rather than asserting it: on a shared query set (in-corpus +
-out-of-corpus) three REAL systems answer, and we measure who fabricates.
-
-- **PEN-CHAT** - the full grounded system (retrieve -> cite-or-silence -> guard, abstain below threshold);
-- **vanilla RAG** - retrieve + prepend + ask the SAME LLM, with NO citation-or-silence and NO abstention;
-- **ungrounded LLM** - the SAME LLM answering the bare question, no retrieval, no guard.
-
-All three use the same local model (Ollama `qwen2.5:3b-instruct`); only the grounding harness differs.
-
-## Result (`result.json`, 8 queries, 5 out-of-corpus)
-| system | abstention on no-evidence | answered without grounding | 
-|---|---|---|
-| **PEN-CHAT** | **1.00** | **0.00** |
-| vanilla RAG | 0.60 | 0.40 |
-| ungrounded LLM | **0.00** | **1.00** |
-
-The punchline: **PEN-CHAT abstains on every no-evidence (out-of-corpus) question and never answers one without a
-grounded source; the ungrounded LLM answers all of them; vanilla RAG is in between.** `answered_without_grounding`
-is the hallucination/false-grounding exposure - ~0 for PEN-CHAT, 1.0 for the ungrounded baseline.
-
-This is a LIVE measurement (the baselines call a real LLM), so it is run once and its result committed; it is not a
-CI unit test. Reproduce on a host with Ollama:
-`PEN_RAG_NO_EMBED=0 python -m benchmarks.chat_headtohead.harness`.

pen_stack-7.1.0/benchmarks/chat_safety/README.md

@@ -1,27 +0,0 @@
-# Chat Refusal & Abstention Benchmark (PEN-CHAT P-WS6, the safety headline)
-
-Measures the gate-**P-G4** safety properties of the chat, deterministically (no LLM, lexical retriever; reproducible
-in CI):
-
-| property | what it checks | result |
-|---|---|---|
-| **false_grounding_rate** | a general-knowledge fact mislabelled as a PEN-STACK computed result | **0.0** (target ~0) |
-| abstention_rate (out-of-corpus) | declines to answer from priors | **1.0** |
-| dual_use_refusal_rate | hazardous build/express requests are refused | **1.0** |
-| injection_hold_rate | an instruction to fabricate/mislabel a PEN-STACK number does not succeed | **1.0** |
-
-`false_grounding_rate ~ 0` is the headline: a non-engine answer is **never** presented as a PEN-STACK result -
-general answers carry provenance `literature-cited` or `abstained`, never `pen-stack`.
-
-## Sealed set
-`cases.jsonl` (SHA-locked) - out-of-corpus questions, dual-use prompts, prompt-injection attempts, and
-false-grounding probes.
-
-The first run measured `dual_use_refusal_rate = 0.667` and surfaced a real gap: a **generic** "express a
-select-agent toxin" request (no catalogued agent signature) routed to the design lane instead of being refused.
-Per the discipline (*tighten, never relax*), the pre-route Guardian screen was strengthened: **a build/express
-intent over any flagged dual-use hazard term escalates to human biosecurity review even when the specific agent is
-not in the signature DB**. Re-measured at 1.0. (The injection-hold and false-grounding measures were also corrected
-to score the *outcome* - did the fabricated value appear / was a general fact mislabelled - rather than the lane.)
-
-Reproduce: `python -c "from benchmarks.chat_safety.harness import run; print(run())"`.

pen_stack-7.1.0/benchmarks/chat_safety/SHA256SUMS

@@ -1 +0,0 @@
-7c0f9482cad56e8f20382c96e7156439e35c986f12e077b207c19efb2b338b55  cases.jsonl

pen_stack-7.1.0/pen_stack/rag/ground.py

@@ -1,87 +0,0 @@
-"""PEN-RAG grounding for the chat's General lane (v7.1).
-
-Replaces "trained knowledge + a label" with **retrieve -> cite-or-silence -> guard**:
-  * retrieve the top chunks from the provenance-tagged corpus;
-  * if the top retrieval confidence is below threshold -> ABSTAIN ("no grounded source"), never answer from priors;
-  * otherwise ground the answer in the retrieved, cited chunks. The deterministic path returns the chunks verbatim
-    with their [source . DOI] tags (citation coverage = 1.0 by construction). If the LLM is allowed it only narrates
-    over those same sources under a citation-or-silence system prompt, and the number guard strips any value not in
-    the sources.
-General answers are always labelled "literature-cited", never presented as a PEN-STACK-computed result.
-"""
-from __future__ import annotations
-
-import os
-
-from pen_stack.rag.embed import tokenize
-from pen_stack.rag.retrieve import retrieve
-
-# nomic-embed-text compresses cosines into a narrow high band, so an absolute cosine alone cannot separate a valid
-# in-corpus query from an off-topic one. We use a HYBRID gate: a moderate cosine floor for ranking PLUS a lexical
-# content-overlap check (a genuinely off-topic query shares no content word with the top chunk), with a high-cosine
-# escape for true paraphrases. The safe failure direction is to abstain, so the gate is deliberately conservative.
-THRESHOLD = float(os.getenv("PEN_RAG_THRESHOLD", "0.50"))          # semantic cosine floor
-_SEM_HIGH = float(os.getenv("PEN_RAG_THRESHOLD_HIGH", "0.72"))     # cosine high enough to ground without lexical overlap
-_LEX_THRESHOLD = float(os.getenv("PEN_RAG_LEX_THRESHOLD", "0.05")) # jaccard floor for the fallback retriever
-
-
-def _is_grounded(query: str, hits: list[dict], method: str) -> bool:
-    """Is the corpus a real source for this query (vs. abstain)?"""
-    if not hits:
-        return False
-    top = hits[0]
-    if not method.startswith("semantic"):
-        return top["score"] >= _LEX_THRESHOLD
-    lexical_overlap = bool(tokenize(query) & tokenize(top["text"]))
-    return top["score"] >= THRESHOLD and (lexical_overlap or top["score"] >= _SEM_HIGH)
-_LABEL = "Literature-cited (PEN-CHAT corpus) - not a PEN-STACK-computed result:"
-_ABSTAIN = ("I have no grounded source in the PEN-CHAT corpus for that, so I won't answer it from unsourced general "
-            "knowledge. The corpus covers PEN-STACK's writers, metrics, scope, and curated genome-writing "
-            "literature - ask within that, or let the engine compute a specific, grounded answer.")
-
-
-def _dedup_sources(hits: list[dict]) -> list[dict]:
-    seen, out = set(), []
-    for h in hits:
-        sid = h["source_id"]
-        if sid in seen:
-            continue
-        seen.add(sid)
-        out.append({"source_id": sid, "doi": h.get("doi") or None, "type": h["type"],
-                    "score": round(h["score"], 3)})
-    return out
-
-
-def ground_general(message: str, *, allow_llm: bool = True) -> dict:
-    """Return the chat envelope for a grounded (or abstained) General-lane answer."""
-    r = retrieve(message, k=4)
-    if not _is_grounded(message, r["hits"], r["method"]):
-        return {"status": "abstained", "reply": _ABSTAIN, "sources": [], "provenance": "abstained",
-                "grounded": False, "backend": "deterministic", "retrieval": r["method"], "top_score": r["top_score"]}
-    floor = THRESHOLD if r["method"].startswith("semantic") else _LEX_THRESHOLD
-    grounded_hits = [h for h in r["hits"] if h["score"] >= floor] or r["hits"][:2]
-
-    sources = _dedup_sources(grounded_hits)
-    deterministic = _LABEL + "\n\n" + "\n\n".join(
-        f"- {h['text']}  [{h['source_id']}{(' · ' + h['doi']) if h.get('doi') else ''}]" for h in grounded_hits)
-
-    if allow_llm:
-        from pen_stack.web.llm import _enforce_grounding, _run_llm
-        from pen_stack.web.tools import extract_grounded_numbers
-        ctx = "\n\n".join(
-            f"[{i + 1}] ({h['source_id']}{(' DOI ' + h['doi']) if h.get('doi') else ''}) {h['text']}"
-            for i, h in enumerate(grounded_hits))
-        system = ("You answer ONLY from the SOURCES provided. After each claim, cite the bracketed [n] it came from. "
-                  "If the sources do not cover the question, say exactly that. Never add a fact, number, name, or "
-                  "vehicle that is not in the sources. Never present anything as a PEN-STACK-computed result.")
-        prompt = f"SOURCES:\n{ctx}\n\nQUESTION: {message}\n\nAnswer concisely, citing [n] after each claim."
-        text, backend = _run_llm(prompt, system)
-        if text:
-            allow = extract_grounded_numbers({"hits": [h["text"] for h in grounded_hits]})
-            cleaned = _enforce_grounding(text.strip(), allow)
-            return {"status": "grounded", "reply": _LABEL + "\n\n" + cleaned, "sources": sources,
-                    "provenance": "literature-cited", "grounded": True, "backend": backend,
-                    "retrieval": r["method"], "top_score": r["top_score"]}
-
-    return {"status": "grounded", "reply": deterministic, "sources": sources, "provenance": "literature-cited",
-            "grounded": True, "backend": "deterministic", "retrieval": r["method"], "top_score": r["top_score"]}

pen_stack-7.1.0/prereg/SHA256_LOCK_ws_penchat.json

@@ -1,9 +0,0 @@
-{
-  "cycle": "v7.1",
-  "workstream": "PEN-CHAT",
-  "prepared": "2026-06-24",
-  "sha256": {
-    "prereg/ws_penchat.yaml": "68f9ba44edcd7e16acd4dd2e1c535478db40466362dff3ce899a6c19d9b7e6f4",
-    "data/rag_corpus.parquet": "44594399d7095494aaee63282f61235df266f58913f409a8bb8d363abf0ed8b1"
-  }
-}