@pharmatools/opengate 0.2.0 → 0.2.1

package/README.md

@@ -124,7 +124,7 @@ Scorers never talk to a system directly — they go through an adapter, injected
 OPENGATE_ADAPTER=./adapters/my-system.mjs npm run eval:online
 ```
 
-An adapter is one file with four required exports — `splitClaims(text)`, `analyzeBatch(payload)`, `onlineAvailable()`, `onlineConfigHint()` — plus optional timing/token/model-label hooks that unlock latency and cost columns in the scorecard. Adapters are validated at load: a malformed one fails fast with a message listing every missing export.
+An adapter is one file: two base exports — `onlineAvailable()`, `onlineConfigHint()` — plus at least one complete **capability**: `qa` (`splitClaims` + `analyzeBatch`) or `redaction` (`redact`). Scorers check `adapter.capabilities` and skip cleanly across the boundary. Optional timing/token/model-label hooks unlock latency and cost columns in the scorecard. Adapters are validated at load: a malformed one fails fast with a message naming every missing export and incomplete capability.
 
 For REST-backed systems there's a no-code path: the bundled **generic HTTP adapter** (`src/adapters/http.mjs`) reads endpoint paths and headers from `opengate.http.json` (see `opengate.http.example.json`), with `${ENV}` interpolation and built-in latency/token capture.
 
@@ -137,7 +137,7 @@ Full contract, minimal skeleton, and verdict-mapping notes: **[ADAPTERS.md](ADAP
 Use OpenGATE as a drop-in regression gate in any repository. Keep your gold set and committed `baseline.json` in your own tree; any metric that drops fails the build:
 
 ```yaml
-- uses: nickjlamb/opengate@main
+- uses: nickjlamb/opengate@v0
   with:
     datasets: ./evals/datasets      # your cases/ + fixtures/
     results: ./evals/results        # where baseline.json lives
@@ -169,7 +169,7 @@ npm install --no-save @pharmatools/redacta
 node src/runner.mjs --online --adapter ./src/adapters/redacta.mjs
 ```
 
-On its first run against the new gold set, the eval found two real engine bugs — a relative name directly followed by a parenthesis escapes the name pattern, and apostrophe surnames are dropped from titled-name capture — both now documented as tracked known gaps in the case files. Current scorecard: **100% recall on 23 gold identifiers, 0 leaks, 3 tracked gaps**.
+On its first run against the new gold set, the eval found two real engine bugs — relation phrases like "Next of kin:" swallowed nested name matches, and apostrophe surnames (O'Brien) were dropped from name capture. Both were fixed in `@pharmatools/redacta` 1.2.1 and confirmed by the eval (`knownGap_closed: 2`), then promoted to gold. Street-line address detection followed in 1.3.0, closing the last tracked gap. Current scorecard: **100% recall on 25 gold identifiers, 0 leaks, no open gaps**.
 
 ## Layout
 
@@ -191,7 +191,7 @@ opengate/
 
 ## Roadmap
 
-- **Redacta engine fixes** — two bugs found by the redaction eval (paren-adjacent relative names; apostrophe surnames in titled-name capture) are tracked as known gaps; fixing them in `@pharmatools/redacta` flips `knownGap_closed` and promotes the cases to gold
+
 - **Third adapter** — Patiently AI (faithfulness evaluation for patient-language simplification)
 - **Author-year in RefCheckr production** — `detectAuthorYear()` now lands "Smith 2020"-style keys in the reference implementation; adopting them in RefCheckr's numeric-keyed citation mapping is tracked separately
 - **Number-adjacent superscript** — `week 24.1` is genuinely ambiguous with decimals; remains a tracked known gap

package/datasets/cases/redact-cardio-clinic.json

@@ -1,7 +1,7 @@
 {
   "id": "redact-cardio-clinic",
   "kind": "redaction",
-  "description": "Cardiology clinic letter — titled patient name, NHS number, DOB, phone, email, postcode. SYNTHETIC: all identifiers are test-range values; no real persons.",
+  "description": "Cardiology clinic letter — titled patient name, NHS number, DOB, phone, email, postcode. SYNTHETIC: all identifiers are test-range values; no real persons. The street address was a tracked engine gap; street-line detection landed in @pharmatools/redacta 1.3.0 and the entity was promoted to gold.",
   "text": "Mr John Smith (NHS 999 000 0018, DOB 12/03/1958) attended cardiology clinic today. He was seen by Dr Patel for review of atrial fibrillation. Blood pressure was 132/84 mmHg and apixaban was continued at 5 mg BD. Contact him on 07700 900123 or john.smith58@example.com. Address on file: 42 Maple Road, Oxford OX2 6JX. Follow-up in 12 weeks.",
   "goldEntities": [
     {
@@ -27,12 +27,11 @@
     {
       "type": "POSTCODE",
       "value": "OX2 6JX"
-    }
-  ],
-  "knownGapEntities": [
+    },
     {
       "type": "STREET_ADDRESS",
       "value": "42 Maple Road"
     }
-  ]
-}
+  ],
+  "knownGapEntities": []
+}

package/datasets/cases/redact-discharge-summary.json

@@ -1,7 +1,7 @@
 {
   "id": "redact-discharge-summary",
   "kind": "redaction",
-  "description": "Discharge summary with a relative as next of kin. SYNTHETIC.",
+  "description": "Discharge summary with a relative as next of kin. SYNTHETIC. The relative name was a tracked engine gap (a 'Next of kin:' phrase swallowed the inner 'daughter Anita' match — not the parenthesis, as first suspected); fixed in @pharmatools/redacta >1.2.0 and promoted to gold.",
   "text": "Discharge summary for Mrs Priya Sharma, NHS 999 000 0026, DOB 04/11/1947. Admitted with community-acquired pneumonia; completed 5 days of co-amoxiclav and improved steadily. Next of kin: her daughter Anita (07700 900456) was updated by phone. Discharged to home address, postcode LS8 2QT, with district nurse follow-up.",
   "goldEntities": [
     {
@@ -23,13 +23,11 @@
     {
       "type": "POSTCODE",
       "value": "LS8 2QT"
-    }
-  ],
-  "knownGapEntities": [
+    },
     {
       "type": "RELATIVE_NAME",
-      "value": "Anita",
-      "comment": "Engine bug (tracked): a relative name directly followed by an opening parenthesis escapes the relative-name pattern — 'her daughter Anita (07700...' leaks, while 'her daughter Anita was' is caught. The phone match also consumes the opening paren, leaving an orphan ')'."
+      "value": "Anita"
     }
-  ]
+  ],
+  "knownGapEntities": []
 }

package/datasets/cases/redact-ward-round.json

@@ -1,7 +1,7 @@
 {
   "id": "redact-ward-round",
   "kind": "redaction",
-  "description": "Ward round entry — apostrophe surname, NHS number without spaces, carer mention. SYNTHETIC.",
+  "description": "Ward round entry — apostrophe surname, NHS number without spaces, carer mention. SYNTHETIC. The apostrophe surname was a tracked engine gap (NAME pattern required a lowercase letter after the initial capital); fixed in @pharmatools/redacta >1.2.0.",
   "text": "Ward round: Mrs Eileen O'Brien, NHS 9990000034, remains stable post hip hemiarthroplasty. Mobilising with frame. Her son Daniel visited and raised concerns about home support; referred to social work. Plan: continue enoxaparin, orthogeriatric review tomorrow. Contact for family updates: 07700 900321.",
   "goldEntities": [
     {
@@ -21,11 +21,5 @@
       "value": "07700 900321"
     }
   ],
-  "knownGapEntities": [
-    {
-      "type": "PATIENT_SURNAME",
-      "value": "O'Brien",
-      "comment": "Engine bug (tracked): apostrophe surnames are dropped from titled-name capture — 'Mrs Eileen O'Brien' tokenises as 'Mrs Eileen' only, leaving the surname in the text."
-    }
-  ]
+  "knownGapEntities": []
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pharmatools/opengate",
-  "version": "0.2.0",
+  "version": "0.2.1",
   "type": "module",
   "description": "OpenGATE — Open Grounded AI Testing & Evaluation. An open-source framework for evaluating evidence-grounded AI systems.",
   "license": "MIT",