pen-stack 6.1.0__tar.gz → 6.2.0__tar.gz

{pen_stack-6.1.0 → pen_stack-6.2.0}/CHANGELOG.md

@@ -3,6 +3,43 @@
 All notable changes to PEN-STACK are documented here. This file follows
 [Keep a Changelog](https://keepachangelog.com/) and the program's phase structure.
 
+## [6.2.0] - 2026-06-11 - v6.2: The Web Platform (the human surface)
+
+**Post-1.0 adoption surface for bench scientists.** A complete, friendly web application — a grounded co-scientist
+chat plus structured feature pages — over the same typed v6.1 API the AI surface uses. The LLM **narrates and
+routes but never sources a number**; every quantitative answer renders with its confidence band, provenance, and
+an explicit ledger of what PEN-STACK can't tell you. MINOR release on the stable 6.0 API. Workstreams
+WS-{BACKEND,CHAT,FRONTEND,PAGES,DEPLOY}, SHA-locked.
+
+### Added
+- **WS-BACKEND** — `pen_stack/web/server.py`: one FastAPI gateway that **mounts the v6.1 engine surface** under
+  `/api` (so the frontend has one base URL / one OpenAPI) and adds the grounded `/chat` (+ SSE `/chat/stream`),
+  CORS, and static serving of the built frontend (`web/dist`). A `/health` for the live grounded indicator.
+- **WS-CHAT** *(the hard gate)* — `pen_stack/web/tools.py` (the deterministic engine tool-runner: parse a goal →
+  run `verify`/`safety`/`immune_profile`/scope → a grounded dossier; `extract_grounded_numbers` = the allow-list)
+  and `pen_stack/web/llm.py` (the grounded co-scientist: **Ollama → Nemotron → deterministic narrator**, with
+  `_enforce_grounding` striking any numeric token the model can't trace to a tool result). A reply's numbers are
+  always engine-sourced; `tests/unit/test_ws_chat.py` asserts **no number in a reply is absent from the tool
+  results**, and that the deterministic narrator works with both LLMs offline.
+- **WS-FRONTEND** — `web/` (React/Vite + Tailwind): the **honest-UX component library** — `ConfidenceBand`,
+  `ProvenanceChip`, `ScopeLedger`, `SafetyBadge`, `ImmuneProfileCard` — so a number is never shown without its
+  uncertainty + provenance, and the scope ledger appears on every answer.
+- **WS-PAGES** — 11 feature pages over the typed API: Co-Scientist · Site Finder · Writer Atlas · Designer ·
+  Verify · Delivery & Immunity · Digital Twin · Guardian · Experiments · Challenge · Scope & About. Each renders
+  through the honest-UX library and degrades gracefully when the LLM is offline.
+- **WS-DEPLOY** — `docker/web.Dockerfile` (multi-stage: a **node:20** stage builds the frontend, a slim Python
+  serves UI + API from one origin) + a `web` service in `docker-compose.yml`: **one-command self-host**
+  (`docker compose up web ollama`, open `http://localhost:8000`). `web/.env.example`, `web/README.md` quickstart.
+- Read-only Challenge surface for the web page: `GET /api/challenge/{tasks,leaderboard}` (public tasks + the
+  PEN-STACK reference leaderboard; submissions are still scored offline, never accepted over HTTP).
+- prereg `ws_{chat,frontend}` + SHA locks; deposit `phase_6.2/`.
+
+### Notes
+- **The LLM never sources a number.** It explains, compares, and routes over the engine's tool outputs; the
+  grounding guard strips any value it can't trace, and the app runs in a deterministic, no-LLM mode (the science
+  lives in the engine, not the model). This cycle improves **usability/adoption**, not validation — a real-data
+  result and a first lab user remain the standing bottleneck. No new science; contracts under the 1.0 commitment.
+
 ## [6.1.0] - 2026-06-11 - v6.1: The AI Integration Surface
 
 **Post-1.0 adoption surface for AI builders.** Not new capability — the *introspectable, documented, dependable*

{pen_stack-6.1.0 → pen_stack-6.2.0}/CITATION.cff

@@ -1,7 +1,7 @@
 cff-version: 1.2.0
 message: "If you use PEN-STACK, please cite it as below."
 title: "PEN-STACK: open infrastructure for genome writing"
-version: 6.1.0
+version: 6.2.0
 date-released: 2026-06-11
 authors:
   - family-names: "Mahaboob Ali"

{pen_stack-6.1.0 → pen_stack-6.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pen-stack
-Version: 6.1.0
+Version: 6.2.0
 Summary: Open infrastructure for genome writing: the Writable Genome atlas, the Writer Atlas, and the Write Planner.
 Author-email: Anees Ahmed Mahaboob Ali <ahmedaneesm@gmail.com>
 License: MIT
@@ -90,9 +90,9 @@ every design against rule-grounded mechanism, reports calibrated confidence, cit
 [![codecov](https://codecov.io/gh/ahmedanees-m/pen-stack/branch/main/graph/badge.svg)](https://codecov.io/gh/ahmedanees-m/pen-stack)
 [![License: MIT](https://img.shields.io/badge/License-MIT-informational.svg)](LICENSE)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
-[![Version](https://img.shields.io/badge/version-6.1.0-blue.svg)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-6.2.0-blue.svg)](CHANGELOG.md)
 [![Status](https://img.shields.io/badge/status-1.0%20First%20Stable-success.svg)](docs/STABILITY.md)
-[![Tests](https://img.shields.io/badge/tests-360%20passing-success.svg)](tests/)
+[![Tests](https://img.shields.io/badge/tests-368%20passing-success.svg)](tests/)
 [![Lint: ruff](https://img.shields.io/badge/lint-ruff-purple.svg)](https://github.com/astral-sh/ruff)
 [![Runtime: Docker](https://img.shields.io/badge/runtime-docker-2496ED.svg)](docker/)
 [![Validation: pre-registered](https://img.shields.io/badge/validation-pre--registered-critical.svg)](prereg/)
@@ -148,6 +148,26 @@ demonstrated (v5.12) and the benchmark went public (v5.13).
 > unknown funnel remains — made legible (scope flags, known-unknowns, honest baselines, no fabrication), not
 > hidden.
 
+## What is new in v6.2 — The Web Platform (the human surface)
+
+Post-1.0, the adoption surface for **bench scientists**: a complete, friendly **web application** — a grounded
+co-scientist chat plus structured feature pages — over the same typed v6.1 API the AI surface uses. The LLM
+**narrates and routes but never sources a number**; every quantitative answer renders with its confidence band,
+its provenance, and an explicit ledger of what PEN-STACK *can't* tell you.
+
+| Workstream | What it adds | Result |
+|---|---|---|
+| **BACKEND** | `pen_stack/web/server.py` | one FastAPI gateway that mounts the v6.1 engine surface (under `/api`) + adds the grounded `/chat` (+ SSE `/chat/stream`) + CORS + serves the built frontend |
+| **CHAT** *(the hard gate)* | `pen_stack/web/{tools,llm}.py` | the grounded co-scientist: the **engine** computes every number (`run_tools`), the LLM narrates over the tool results (**Ollama → Nemotron → deterministic**), and a **grounding guard strikes any number the model can't trace to a tool result** (asserted in `tests/unit/test_ws_chat.py`) |
+| **FRONTEND** | `web/` (React/Vite + Tailwind) | the **honest-UX library** — `ConfidenceBand · ProvenanceChip · ScopeLedger · SafetyBadge · ImmuneProfileCard` — so a number is never shown without its uncertainty + provenance |
+| **PAGES** | 11 feature pages | Co-Scientist · Site Finder · Writer Atlas · Designer · Verify · Delivery & Immunity · Digital Twin · Guardian · Experiments · Challenge · Scope & About |
+| **DEPLOY** | `docker/web.Dockerfile` + compose | **one-command self-host**: a `node:20` stage builds the frontend, a slim Python serves UI + API from one origin — `docker compose up web ollama`, open `http://localhost:8000` |
+
+**The LLM never sources a number** — it explains, compares, and routes over the engine's tool outputs, and the
+science runs (deterministic mode) with no LLM at all. See [`web/README.md`](web/README.md) and
+`prereg/ws_{chat,frontend}.yaml`. (Lowers the usability barrier; a real-data validation + a first lab user remain
+the standing bottleneck — no new science.)
+
 ## What is new in v6.1 — The AI Integration Surface
 
 Post-1.0, the adoption surface for **AI builders**: PEN-STACK is now **self-describing** — an external agent can
@@ -780,13 +800,15 @@ pen-stack/
 │   │                                   v5.12 closed_loop (loop-integrity hard-gate: gated end-to-end run · Level-3 human-in-control · drift detection · versioned/reversible continual learning)
 │   ├── data/                         ingestion (genome, chromatin, integration, TRIP, safety annotations)
 │   ├── api/                          v6.1 AI integration surface: manifest (capability_manifest + scope_manifest = machine-readable known-unknowns + oracle scope cards)
-│   ├── server/api.py                 FastAPI REST (atlas, crosslink, writable, plan, bridge, ask; v3.3 verify; v6.1 /capabilities /scope /safety /immune /generate /predict /suggest /session + openapi.json 3.1)
+│   ├── web/                          v6.2 Web Platform backend: tools (deterministic engine dossier) / llm (grounded co-scientist: Ollama→Nemotron→deterministic + grounding-guard hard gate) / server (FastAPI gateway mounting the v6.1 surface + /chat + SSE + static frontend)
+│   ├── server/api.py                 FastAPI REST (atlas, crosslink, writable, plan, bridge, ask; v3.3 verify; v6.1 /capabilities /scope /safety /immune /generate /predict /suggest /session + openapi.json 3.1; v5.13 /challenge/{tasks,leaderboard})
 │   ├── ui/app.py                     Streamlit web app (16 pages; v3.2 PEN-Agent shows confidence + epistemic status)
 │   └── cli.py                        unified CLI
 ├── benchmarks/genome_writing_bench/  Genome-Writing Bench v0.3.8 (T1-T16 + co_scientist + safety_screening + generative_design + outcome_prediction + experiment_design + protocol_safety + closed_loop; tasks / harness / solvers / LEADERBOARD / SHAs)
 ├── benchmarks/genome_writing_challenge/ v5.13 the public Genome-Writing Challenge (held-out rounds + immune-risk task + submission API; harness / run / README / SUBMISSIONS)
 ├── bench/run.py                      one-command bench entrypoint (--agent, --verify)
 ├── examples/                         v6.1 runnable golden path: external_agent.py (REST) / mcp_client.py (MCP) / agent_tools.py (framework-agnostic tool specs from the live manifest + dispatcher)
+├── web/                              v6.2 the Web Platform frontend (React/Vite + Tailwind): honest-UX library (ConfidenceBand/ProvenanceChip/ScopeLedger/SafetyBadge/ImmuneProfileCard) + 11 feature pages + the grounded co-scientist chat; built by docker/web.Dockerfile (node:20 stage)
 ├── scripts/                          reproducible pipeline drivers (p1_*, p2_*, p4_*, p52/p53 delivery-immunology oracle builds, ws_*_report)
 ├── configs/                          pinned datasets + thresholds + curation (YAML); v3.2 known_unknowns /
 │                                       target_sites / delivery_constraints; v5.1-5.6 delivery_vehicles immune_safety /
@@ -799,7 +821,7 @@ pen-stack/
 │                                       genotox,epitope,innate,seroprev,peg,calib,profile,screen,policy,redteam,
 │                                       gen,pareto,orch,vcell,mech,outcome,twincal,acq,aldesign,alvalidate,
 │                                       proto,ingest,simlab,loop,continual,drift,challenge,cosci2,
-│                                       manifest,openapi,mcp} + SHA256 locks)
+│                                       manifest,openapi,mcp,chat,frontend} + SHA256 locks)
 ├── data/curated/                     small committed tables (universe, gene coords, measured bridge profile,
 │                                       v3.2 bridge_offtarget_energetics.json)
 ├── data/llm_bench_cache/             28 cached ungrounded-LLM transcripts (T7, offline/CI replay)
@@ -813,7 +835,7 @@ pen-stack/
 │                                       generative_design.md / digital_twin.md / experiment_design.md /
 │                                       build_interface.md / closed_loop.md / autonomy.md / challenge.md /
 │                                       co_scientist_loop.md / integrations.md / STABILITY.md (1.0 API freeze)
-├── docker/                           CUDA image + UI image + pinned requirements
+├── docker/                           CUDA image + UI image + v6.2 web.Dockerfile (multi-stage: node:20 frontend build → slim Python gateway) + pinned requirements
 ├── tools/penctl.py                   laptop<->VM orchestrator (paramiko SSH/SFTP, Docker-only)
 ├── docker-compose.yml                one-command self-hostable platform
 └── pyproject.toml  CITATION.cff  CHANGELOG.md  LICENSE

{pen_stack-6.1.0 → pen_stack-6.2.0}/README.md

@@ -15,9 +15,9 @@ every design against rule-grounded mechanism, reports calibrated confidence, cit
 [![codecov](https://codecov.io/gh/ahmedanees-m/pen-stack/branch/main/graph/badge.svg)](https://codecov.io/gh/ahmedanees-m/pen-stack)
 [![License: MIT](https://img.shields.io/badge/License-MIT-informational.svg)](LICENSE)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
-[![Version](https://img.shields.io/badge/version-6.1.0-blue.svg)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-6.2.0-blue.svg)](CHANGELOG.md)
 [![Status](https://img.shields.io/badge/status-1.0%20First%20Stable-success.svg)](docs/STABILITY.md)
-[![Tests](https://img.shields.io/badge/tests-360%20passing-success.svg)](tests/)
+[![Tests](https://img.shields.io/badge/tests-368%20passing-success.svg)](tests/)
 [![Lint: ruff](https://img.shields.io/badge/lint-ruff-purple.svg)](https://github.com/astral-sh/ruff)
 [![Runtime: Docker](https://img.shields.io/badge/runtime-docker-2496ED.svg)](docker/)
 [![Validation: pre-registered](https://img.shields.io/badge/validation-pre--registered-critical.svg)](prereg/)
@@ -73,6 +73,26 @@ demonstrated (v5.12) and the benchmark went public (v5.13).
 > unknown funnel remains — made legible (scope flags, known-unknowns, honest baselines, no fabrication), not
 > hidden.
 
+## What is new in v6.2 — The Web Platform (the human surface)
+
+Post-1.0, the adoption surface for **bench scientists**: a complete, friendly **web application** — a grounded
+co-scientist chat plus structured feature pages — over the same typed v6.1 API the AI surface uses. The LLM
+**narrates and routes but never sources a number**; every quantitative answer renders with its confidence band,
+its provenance, and an explicit ledger of what PEN-STACK *can't* tell you.
+
+| Workstream | What it adds | Result |
+|---|---|---|
+| **BACKEND** | `pen_stack/web/server.py` | one FastAPI gateway that mounts the v6.1 engine surface (under `/api`) + adds the grounded `/chat` (+ SSE `/chat/stream`) + CORS + serves the built frontend |
+| **CHAT** *(the hard gate)* | `pen_stack/web/{tools,llm}.py` | the grounded co-scientist: the **engine** computes every number (`run_tools`), the LLM narrates over the tool results (**Ollama → Nemotron → deterministic**), and a **grounding guard strikes any number the model can't trace to a tool result** (asserted in `tests/unit/test_ws_chat.py`) |
+| **FRONTEND** | `web/` (React/Vite + Tailwind) | the **honest-UX library** — `ConfidenceBand · ProvenanceChip · ScopeLedger · SafetyBadge · ImmuneProfileCard` — so a number is never shown without its uncertainty + provenance |
+| **PAGES** | 11 feature pages | Co-Scientist · Site Finder · Writer Atlas · Designer · Verify · Delivery & Immunity · Digital Twin · Guardian · Experiments · Challenge · Scope & About |
+| **DEPLOY** | `docker/web.Dockerfile` + compose | **one-command self-host**: a `node:20` stage builds the frontend, a slim Python serves UI + API from one origin — `docker compose up web ollama`, open `http://localhost:8000` |
+
+**The LLM never sources a number** — it explains, compares, and routes over the engine's tool outputs, and the
+science runs (deterministic mode) with no LLM at all. See [`web/README.md`](web/README.md) and
+`prereg/ws_{chat,frontend}.yaml`. (Lowers the usability barrier; a real-data validation + a first lab user remain
+the standing bottleneck — no new science.)
+
 ## What is new in v6.1 — The AI Integration Surface
 
 Post-1.0, the adoption surface for **AI builders**: PEN-STACK is now **self-describing** — an external agent can
@@ -705,13 +725,15 @@ pen-stack/
 │   │                                   v5.12 closed_loop (loop-integrity hard-gate: gated end-to-end run · Level-3 human-in-control · drift detection · versioned/reversible continual learning)
 │   ├── data/                         ingestion (genome, chromatin, integration, TRIP, safety annotations)
 │   ├── api/                          v6.1 AI integration surface: manifest (capability_manifest + scope_manifest = machine-readable known-unknowns + oracle scope cards)
-│   ├── server/api.py                 FastAPI REST (atlas, crosslink, writable, plan, bridge, ask; v3.3 verify; v6.1 /capabilities /scope /safety /immune /generate /predict /suggest /session + openapi.json 3.1)
+│   ├── web/                          v6.2 Web Platform backend: tools (deterministic engine dossier) / llm (grounded co-scientist: Ollama→Nemotron→deterministic + grounding-guard hard gate) / server (FastAPI gateway mounting the v6.1 surface + /chat + SSE + static frontend)
+│   ├── server/api.py                 FastAPI REST (atlas, crosslink, writable, plan, bridge, ask; v3.3 verify; v6.1 /capabilities /scope /safety /immune /generate /predict /suggest /session + openapi.json 3.1; v5.13 /challenge/{tasks,leaderboard})
 │   ├── ui/app.py                     Streamlit web app (16 pages; v3.2 PEN-Agent shows confidence + epistemic status)
 │   └── cli.py                        unified CLI
 ├── benchmarks/genome_writing_bench/  Genome-Writing Bench v0.3.8 (T1-T16 + co_scientist + safety_screening + generative_design + outcome_prediction + experiment_design + protocol_safety + closed_loop; tasks / harness / solvers / LEADERBOARD / SHAs)
 ├── benchmarks/genome_writing_challenge/ v5.13 the public Genome-Writing Challenge (held-out rounds + immune-risk task + submission API; harness / run / README / SUBMISSIONS)
 ├── bench/run.py                      one-command bench entrypoint (--agent, --verify)
 ├── examples/                         v6.1 runnable golden path: external_agent.py (REST) / mcp_client.py (MCP) / agent_tools.py (framework-agnostic tool specs from the live manifest + dispatcher)
+├── web/                              v6.2 the Web Platform frontend (React/Vite + Tailwind): honest-UX library (ConfidenceBand/ProvenanceChip/ScopeLedger/SafetyBadge/ImmuneProfileCard) + 11 feature pages + the grounded co-scientist chat; built by docker/web.Dockerfile (node:20 stage)
 ├── scripts/                          reproducible pipeline drivers (p1_*, p2_*, p4_*, p52/p53 delivery-immunology oracle builds, ws_*_report)
 ├── configs/                          pinned datasets + thresholds + curation (YAML); v3.2 known_unknowns /
 │                                       target_sites / delivery_constraints; v5.1-5.6 delivery_vehicles immune_safety /
@@ -724,7 +746,7 @@ pen-stack/
 │                                       genotox,epitope,innate,seroprev,peg,calib,profile,screen,policy,redteam,
 │                                       gen,pareto,orch,vcell,mech,outcome,twincal,acq,aldesign,alvalidate,
 │                                       proto,ingest,simlab,loop,continual,drift,challenge,cosci2,
-│                                       manifest,openapi,mcp} + SHA256 locks)
+│                                       manifest,openapi,mcp,chat,frontend} + SHA256 locks)
 ├── data/curated/                     small committed tables (universe, gene coords, measured bridge profile,
 │                                       v3.2 bridge_offtarget_energetics.json)
 ├── data/llm_bench_cache/             28 cached ungrounded-LLM transcripts (T7, offline/CI replay)
@@ -738,7 +760,7 @@ pen-stack/
 │                                       generative_design.md / digital_twin.md / experiment_design.md /
 │                                       build_interface.md / closed_loop.md / autonomy.md / challenge.md /
 │                                       co_scientist_loop.md / integrations.md / STABILITY.md (1.0 API freeze)
-├── docker/                           CUDA image + UI image + pinned requirements
+├── docker/                           CUDA image + UI image + v6.2 web.Dockerfile (multi-stage: node:20 frontend build → slim Python gateway) + pinned requirements
 ├── tools/penctl.py                   laptop<->VM orchestrator (paramiko SSH/SFTP, Docker-only)
 ├── docker-compose.yml                one-command self-hostable platform
 └── pyproject.toml  CITATION.cff  CHANGELOG.md  LICENSE

{pen_stack-6.1.0 → pen_stack-6.2.0}/pen_stack/__init__.py

@@ -1,2 +1,2 @@
 """PEN-STACK v3.0 - open infrastructure for genome writing."""
-__version__ = "6.1.0"
+__version__ = "6.2.0"

{pen_stack-6.1.0 → pen_stack-6.2.0}/pen_stack/server/api.py

@@ -223,3 +223,29 @@ def session_endpoint(req: dict):
     predicted outcomes + per-axis immune profiles + suggested experiments + citations + scope ledger + safety."""
     from pen_stack.agent.co_scientist import co_scientist_session
     return co_scientist_session(req["goal"], req.get("cell_state", "k562"), candidates=req.get("candidates"))
+
+
+# ======================================================================================
+# v5.13 — The Genome-Writing Challenge (read-only surface for the web Challenge page).
+# Public tasks (NO labels) + the PEN-STACK reference submission that anchors the leaderboard. Submissions
+# are Python `predict_fn`s scored offline (`benchmarks/genome_writing_challenge/run.py`) — never accepted
+# over HTTP — so these routes only EXPOSE the held-out round and the anchor score.
+# ======================================================================================
+@app.get("/challenge/tasks", tags=["challenge"])
+def challenge_tasks(round_id: str = "2026R1"):
+    """The public inputs of the current held-out round (family + design + instructions; NEVER the label)."""
+    from benchmarks.genome_writing_challenge.harness import _round_tasks
+    tasks = _round_tasks(round_id)
+    return {"round": round_id, "n_tasks": len(tasks),
+            "tasks": [{"id": t.id, "family": t.family, "public_input": t.public_input} for t in tasks]}
+
+
+@app.get("/challenge/leaderboard", tags=["challenge"])
+def challenge_leaderboard(round_id: str = "2026R1"):
+    """The leaderboard anchored by the PEN-STACK reference submission (deterministic, non-circular labels,
+    no-fabrication audited). External submissions are scored offline and appended here after a round."""
+    from benchmarks.genome_writing_challenge.harness import evaluate, reference_submission
+    ref = evaluate(reference_submission(), round_id)
+    return {"round": round_id, "leaderboard": [ref],
+            "rules": {"no_circular_labels": ref["no_circular_labels"],
+                      "no_fabrication_audited": True, "labels": "validated PEN-STACK verifier / oracles"}}

pen_stack-6.2.0/pen_stack/web/__init__.py

@@ -0,0 +1,18 @@
+"""PEN-STACK Web Platform (v6.2) — the human surface.
+
+A grounded co-scientist chat + every engine feature behind one typed gateway, with honest UX. The LLM narrates
+and routes; the engine sources every number. Submodules:
+
+* ``tools``  — the deterministic engine tool-runner (parse a goal, run verify/safety/immune/scope, return a
+  grounded dossier) and the grounding allow-list.
+* ``llm``    — the grounded co-scientist (Ollama → Nemotron → deterministic) with the grounding-guard hard gate.
+* ``server`` — the FastAPI gateway (mounts the v6.1 engine surface + ``/chat``); needs the ``server`` extra.
+
+``server`` is imported lazily (it requires FastAPI) so ``import pen_stack.web`` works on a bare install.
+"""
+from __future__ import annotations
+
+from pen_stack.web.llm import grounded_reply
+from pen_stack.web.tools import extract_grounded_numbers, parse_goal, run_tools
+
+__all__ = ["grounded_reply", "run_tools", "parse_goal", "extract_grounded_numbers"]

pen_stack-6.2.0/pen_stack/web/llm.py

@@ -0,0 +1,219 @@
+"""Grounded co-scientist for the web chat (PEN-STACK v6.2, WS-CHAT).
+
+The conversational layer that is helpful **and** cannot fabricate. The flow is always:
+
+    run_tools(message)  ->  the ENGINE computes every number (verify / safety / immune / scope)
+    extract_grounded_numbers(tool_results)  ->  the allow-list of values the model may cite
+    LLM narrates over the tool results  ->  Ollama (local, free) → Nemotron (hosted free tier) → deterministic
+    _enforce_grounding(text, allow_list)  ->  HARD GATE: any number not traceable to a tool result is struck
+
+The LLM *routes, explains, and compares*; it never sources a number. If both LLMs are unavailable the
+deterministic narrator composes the reply directly from the tool results — so the science never depends on the
+model. The grounding guard is the invariant: a numeric claim absent from the tool results cannot survive into a
+reply (it is replaced with the marker ``[unverified]``). This is asserted by ``tests/unit/test_ws_chat.py``.
+"""
+from __future__ import annotations
+
+import os
+import re
+from pathlib import Path
+from typing import Any
+
+from pen_stack.web.tools import extract_grounded_numbers, run_tools
+
+SYSTEM = (
+    "You are PEN-STACK's co-scientist for genome writing. You may explain, compare, and route between the "
+    "engine's tools, but you MUST NOT state any number, score, probability, titer, or confidence that is not "
+    "present verbatim in the TOOL RESULTS provided to you. If a value is not in the tool results, say it is "
+    "out of scope or unknown — never invent it. Never invent a citation. Always surface uncertainty and the "
+    "scope ledger ('what I can't tell you'). Be concise, friendly, and honest. This is decision-support, not a "
+    "clinical directive."
+)
+
+# the number marker the guard substitutes for any ungrounded numeric token the model emits.
+_UNVERIFIED = "[unverified]"
+# a numeric token in model prose: optional sign, digits, optional decimal, optional %  (e.g. 0.28, 28%, 4500)
+_TOKEN_RE = re.compile(r"(?<![\w.])-?\d+(?:\.\d+)?%?")
+
+
+# --------------------------------------------------------------------------------------
+# the grounding guard (the hard gate)
+# --------------------------------------------------------------------------------------
+def _is_grounded(token: str, grounded: set[str]) -> bool:
+    """A numeric token is grounded iff it (or a normalised form) appears in the engine's allow-list."""
+    raw = token.strip()
+    pct = raw.endswith("%")
+    body = raw[:-1] if pct else raw
+    if body in grounded:
+        return True
+    try:
+        f = float(body)
+    except ValueError:
+        return False
+    forms = {body, str(int(f)) if f.is_integer() else str(f), f"{f:.2f}"}
+    if pct:                                              # "28%" matches a grounded 0.28 score or a grounded 28
+        forms.add(str(f / 100))
+        forms.add(f"{f / 100:.2f}")
+    return bool(forms & grounded)
+
+
+def _enforce_grounding(text: str, grounded: set[str]) -> str:
+    """Strike every numeric token in `text` that is not traceable to the engine's tool results. The result is
+    a reply in which **no number is absent from the tool results** — the invariant the chat is built on."""
+    return _TOKEN_RE.sub(lambda m: m.group(0) if _is_grounded(m.group(0), grounded) else _UNVERIFIED, text)
+
+
+def ungrounded_numbers(text: str, grounded: set[str]) -> list[str]:
+    """Diagnostic: the numeric tokens in `text` that are NOT in the allow-list (empty after enforcement)."""
+    return [m.group(0) for m in _TOKEN_RE.finditer(text) if not _is_grounded(m.group(0), grounded)]
+
+
+# --------------------------------------------------------------------------------------
+# the deterministic narrator (no-LLM fallback; numbers grounded by construction)
+# --------------------------------------------------------------------------------------
+def _fmt(x: Any) -> str:
+    if isinstance(x, float):
+        return f"{x:.2f}".rstrip("0").rstrip(".") if x != int(x) else str(int(x))
+    return str(x)
+
+
+def _deterministic_narrate(tr: dict) -> str:
+    """Compose a clear English reply directly from the engine's tool results. Every number here comes straight
+    from `tr`, so the reply is grounded by construction (this is the LLM-offline path)."""
+    d = tr["parsed_design"]
+    v = tr["verdict"]
+    lines: list[str] = []
+    lines.append(
+        f"I read your goal as: a **{d['edit_intent'].replace('_', ' ')}** of **{d['gene']}** "
+        f"(~{d['cargo_bp']} bp cargo) delivered by **{d['delivery_vehicle'].replace('_', ' ')}** "
+        f"in **{d['cell_type']}**. Here is what the engine computes — every number below is tool-sourced.")
+
+    legal = "legal" if v["legal"] else ("deferred" if v["legal"] is None else "ILLEGAL")
+    line = f"**Verification.** The design is **{legal}** ({v['epistemic_status']})."
+    if v["violations"]:
+        line += " Violations: " + ", ".join(str(x) for x in v["violations"]) + "."
+    if v["confidence"] is not None:
+        band = f" [{v['interval'][0]:.2f}–{v['interval'][1]:.2f}]" if v.get("interval") else ""
+        line += f" Calibrated confidence on the soft components: **{v['confidence']:.2f}**{band}."
+    else:
+        line += " (Confidence abstained — no calibrated soft-component score for this design.)"
+    lines.append(line)
+
+    s = tr["safety"]
+    if s.get("decision"):
+        lines.append(f"**Safety (Guardian).** Decision: **{s['decision']}** — {s.get('reason', 'n/a')}.")
+
+    imm = tr["immune_profile"]
+    axes = imm.get("axes") or {}
+    if axes:
+        lines.append("**Immune-risk profile** (per-axis — never collapsed into one number):")
+        for name, a in axes.items():
+            val = _fmt(a["value"]) if a.get("value") is not None else "n/a"
+            unc = a.get("uncertainty")
+            unc_s = f" ±{_fmt(unc)}" if unc is not None else ""
+            lab = a.get("validation", "")
+            lines.append(f"  • {name.replace('_', ' ')}: **{val}**{unc_s} — {lab}")
+        if imm.get("collapsed_score") is None:
+            lines.append("  (No single fused immune score is asserted — that would overstate certainty.)")
+
+    sc = tr["scope"]
+    if sc.get("out_of_scope"):
+        lines.append(f"**Out of scope.** {sc.get('why') or sc.get('title')} PEN-STACK will not guess a value here.")
+    ku = imm.get("known_unknowns") or []
+    if ku:
+        lines.append("**What I can't tell you** (known-unknowns): " + ", ".join(str(x) for x in ku) + ".")
+
+    lines.append(f"_{tr['disclaimer']}_")
+    return "\n".join(lines)
+
+
+# --------------------------------------------------------------------------------------
+# LLM backends (free-tier first; both optional)
+# --------------------------------------------------------------------------------------
+def _ollama_base() -> str:
+    return os.getenv("OLLAMA_HOST", "http://localhost:11434").rstrip("/")
+
+
+def _llm_timeout() -> float:
+    # generous by default: a 7B model narrating a full dossier on a single GPU can take ~60-90s; too short a
+    # timeout would silently fall back to the deterministic narrator on every real query.
+    return float(os.getenv("PEN_STACK_LLM_TIMEOUT", "150"))
+
+
+def _call_ollama(prompt: str) -> str:
+    """Primary: local Ollama (free). Default model qwen2.5:7b-instruct (override via OLLAMA_MODEL). Generation is
+    bounded (num_predict) so a reply returns promptly; the science is in the engine, the LLM only narrates."""
+    import requests
+
+    r = requests.post(
+        f"{_ollama_base()}/api/generate",
+        json={"model": os.getenv("OLLAMA_MODEL", "qwen2.5:7b-instruct"),
+              "prompt": prompt, "system": SYSTEM, "stream": False,
+              "options": {"temperature": 0.2,
+                          "num_predict": int(os.getenv("OLLAMA_NUM_PREDICT", "450"))}},
+        timeout=_llm_timeout())
+    r.raise_for_status()
+    return r.json()["response"]
+
+
+def _nvidia_key() -> str | None:
+    key = os.getenv("NVIDIA_API_KEY")
+    if key:
+        return key.strip()
+    f = Path(__file__).resolve().parents[2] / "configs" / "nvidia_api_key.txt"
+    return f.read_text(encoding="utf-8").strip() if f.exists() else None
+
+
+def _call_nemotron(prompt: str) -> str:
+    """Fallback: NVIDIA-hosted Nemotron (free tier). Needs NVIDIA_API_KEY (or configs/nvidia_api_key.txt)."""
+    import requests
+
+    key = _nvidia_key()
+    if not key:
+        raise RuntimeError("no NVIDIA_API_KEY")
+    r = requests.post(
+        "https://integrate.api.nvidia.com/v1/chat/completions",
+        headers={"Authorization": f"Bearer {key}"},
+        json={"model": os.getenv("NEMOTRON_MODEL", "nvidia/llama-3.1-nemotron-70b-instruct"),
+              "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": prompt}],
+              "temperature": 0.2, "max_tokens": 900},
+        timeout=_llm_timeout())
+    r.raise_for_status()
+    return r.json()["choices"][0]["message"]["content"]
+
+
+def _prompt(message: str, tool_results: dict, history: list | None) -> str:
+    import json
+
+    convo = ""
+    for turn in (history or [])[-6:]:
+        role = turn.get("role", "user")
+        convo += f"{role.upper()}: {turn.get('content', '')}\n"
+    return (f"{SYSTEM}\n\nTOOL RESULTS (the ONLY source of numbers — cite nothing else):\n"
+            f"{json.dumps(tool_results, default=str, separators=(',', ':'))}\n\n{convo}USER: {message}\n\n"
+            f"Compose a concise, friendly reply (a short paragraph) that explains the engine's findings, surfaces "
+            f"the uncertainty and the scope ledger, and uses ONLY numbers present in the tool results.")
+
+
+# --------------------------------------------------------------------------------------
+# the public entry point
+# --------------------------------------------------------------------------------------
+def grounded_reply(message: str, history: list | None = None, *, allow_llm: bool = True) -> dict:
+    """The grounded co-scientist. Runs the engine, then narrates over its results with the grounding guard
+    enforced. Returns {reply, tool_results, grounded, backend}. `grounded` is always True: the reply is either
+    composed deterministically from the tool results, or passed through `_enforce_grounding`."""
+    tool_results = run_tools(message, history)                 # ENGINE computes every number
+    grounded = extract_grounded_numbers(tool_results)          # the allow-list
+    prompt = _prompt(message, tool_results, history)
+
+    if allow_llm and os.getenv("PEN_STACK_NO_LLM") != "1":
+        for backend in (_call_ollama, _call_nemotron):
+            try:
+                text = _enforce_grounding(backend(prompt), grounded)   # HARD GATE
+                return {"reply": text, "tool_results": tool_results, "grounded": True,
+                        "backend": backend.__name__.removeprefix("_call_")}
+            except Exception:                                  # any backend failure → next, then deterministic
+                continue
+
+    return {"reply": _deterministic_narrate(tool_results), "tool_results": tool_results,
+            "grounded": True, "backend": "deterministic"}

pen_stack-6.2.0/pen_stack/web/server.py

@@ -0,0 +1,86 @@
+"""The PEN-STACK web gateway (v6.2, WS-BACKEND) — one typed HTTP surface over the engine + the grounded chat.
+
+This is a *gateway, not a rewrite*. It mounts the v6.1 AI-surface app (``/verify``, ``/safety``, ``/immune``,
+``/generate``, ``/predict``, ``/suggest``, ``/session``, ``/capabilities``, ``/scope`` + the atlas/crosslink
+routes) and adds the human-facing pieces: the grounded ``/chat`` (and a streaming ``/chat/stream``), CORS for
+the React frontend, and static serving of the built frontend when present. Every quantity still comes from the
+engine; the chat's LLM only narrates (see ``pen_stack.web.llm``).
+
+Run: ``uvicorn pen_stack.web.server:app --host 0.0.0.0 --port 8000`` (needs the ``server`` extra).
+"""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+try:
+    from fastapi import FastAPI
+    from fastapi.middleware.cors import CORSMiddleware
+    from fastapi.responses import StreamingResponse
+except ImportError as e:  # pragma: no cover - server extra optional
+    raise ImportError("FastAPI not installed: pip install 'pen-stack[server]'") from e
+
+from pen_stack import __version__
+from pen_stack.server.api import app as _engine_app   # the v6.1 typed engine surface (reused verbatim)
+
+app = FastAPI(
+    title="PEN-STACK — Web Platform",
+    version=__version__,
+    description=("The human surface for genome writing: a grounded co-scientist chat + every engine feature, "
+                 "with honest UX (confidence bands, provenance, scope ledger, safety badges). The LLM narrates "
+                 "and routes but never sources a number."),
+)
+
+# CORS — the Vite dev server (5173) and the bundled static frontend both talk to this gateway.
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],            # self-hosted, single-tenant; tighten via PEN_STACK_CORS if deployed
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+# Mount the entire v6.1 engine surface under the same app so the frontend has ONE base URL/one OpenAPI.
+app.mount("/api", _engine_app)
+
+
+@app.get("/health")
+def health() -> dict:
+    return {"status": "ok", "version": __version__, "surface": "web"}
+
+
+@app.post("/chat", tags=["v6.2 co-scientist"])
+def chat_route(req: dict) -> dict:
+    """The grounded co-scientist: the LLM narrates over ENGINE tool outputs; every number comes from the tools
+    and the grounding guard strikes any the model invents. Body: {message, history?}. Returns
+    {reply, tool_results, grounded, backend}."""
+    from pen_stack.web.llm import grounded_reply
+
+    return grounded_reply(req["message"], history=req.get("history", []),
+                          allow_llm=bool(req.get("allow_llm", True)))
+
+
+@app.post("/chat/stream", tags=["v6.2 co-scientist"])
+def chat_stream_route(req: dict) -> StreamingResponse:
+    """Same grounded reply, streamed as Server-Sent Events. The reply is computed grounded first (guard already
+    applied), then emitted word-by-word so the UI can render progressively; a final event carries the dossier."""
+    from pen_stack.web.llm import grounded_reply
+
+    result = grounded_reply(req["message"], history=req.get("history", []),
+                            allow_llm=bool(req.get("allow_llm", True)))
+
+    def _events():
+        for word in result["reply"].split(" "):
+            yield f"data: {json.dumps({'token': word + ' '})}\n\n"
+        yield ("event: done\ndata: " + json.dumps(
+            {"tool_results": result["tool_results"], "backend": result["backend"],
+             "grounded": result["grounded"]}, default=str) + "\n\n")
+
+    return StreamingResponse(_events(), media_type="text/event-stream")
+
+
+# Serve the built React frontend (web/dist) when present — one-command self-host serves UI + API together.
+_DIST = Path(__file__).resolve().parents[2] / "web" / "dist"
+if _DIST.exists():  # pragma: no cover - only when the frontend has been built
+    from fastapi.staticfiles import StaticFiles
+
+    app.mount("/", StaticFiles(directory=str(_DIST), html=True), name="frontend")