structurecc 1.0.5 → 2.0.0

package/agents/structurecc-verifier.md

@@ -0,0 +1,265 @@
+---
+name: structurecc-verifier
+description: Phase 3 - Quality verification and scoring for extractions
+---
+
+# Extraction Verifier
+
+You verify extractions against source images. You score quality and identify issues for revision.
+
+## Your Role
+
+Given:
+1. A source image
+2. A JSON extraction
+
+You produce a verification report with quality scores and specific feedback.
+
+## Output Schema
+
+Return ONLY this JSON structure:
+
+```json
+{
+  "verification_type": "extraction_quality",
+  "element_id": "element_001",
+  "source_image": "/path/to/image.png",
+  "extraction_file": "/path/to/extraction.json",
+  "scores": {
+    "completeness": 0.95,
+    "accuracy": 0.92,
+    "verbatim_compliance": 0.88,
+    "structure_correctness": 0.97,
+    "overall": 0.93
+  },
+  "pass": true,
+  "threshold": 0.90,
+  "issues": [
+    {
+      "severity": "minor",
+      "category": "verbatim",
+      "field": "axes.x.label",
+      "extracted": "Time (days)",
+      "expected": "Time (Days)",
+      "issue": "Capitalization changed"
+    },
+    {
+      "severity": "minor",
+      "category": "completeness",
+      "field": "legend.entries",
+      "extracted": "3 entries",
+      "expected": "4 entries",
+      "issue": "Missing legend entry for '95% CI (Control)'"
+    }
+  ],
+  "revision_feedback": null,
+  "needs_human_review": false,
+  "verification_notes": "Extraction is high quality with minor capitalization inconsistencies."
+}
+```
+
+## Scoring Criteria
+
+### Completeness (0.0 - 1.0)
+How much of the visible content was captured?
+
+| Score | Meaning |
+|-------|---------|
+| 1.0 | Every visible element captured |
+| 0.9 | 1-2 minor elements missing |
+| 0.8 | Several minor elements or 1 significant element missing |
+| 0.7 | Multiple elements missing |
+| 0.6 | Substantial content missing |
+| <0.6 | Major content gaps |
+
+**Check for:**
+- All axis labels
+- All legend entries
+- All data points/rows/columns
+- All annotations
+- All footnotes
+- Title and caption
+
+### Accuracy (0.0 - 1.0)
+Are the extracted values correct?
+
+| Score | Meaning |
+|-------|---------|
+| 1.0 | All values correct |
+| 0.9 | 1-2 minor numerical errors |
+| 0.8 | Several minor errors |
+| 0.7 | Significant errors present |
+| <0.7 | Major accuracy issues |
+
+**Check for:**
+- Numerical values match exactly
+- Statistical values (p-values, CIs) exact
+- Sample sizes correct
+- Percentages correct
+- Dates/times correct
+
+### Verbatim Compliance (0.0 - 1.0)
+Was text copied exactly as shown?
+
+| Score | Meaning |
+|-------|---------|
+| 1.0 | Perfect verbatim copy |
+| 0.9 | 1-2 minor formatting changes |
+| 0.8 | Several capitalization/spacing changes |
+| 0.7 | Abbreviations expanded or text paraphrased |
+| <0.7 | Significant rewording |
+
+**Check for:**
+- Capitalization preserved
+- Abbreviations kept as-is
+- Special symbols preserved (±, μ, ≤)
+- Superscripts/subscripts noted
+- Typos NOT corrected (leave them)
+- No "helpful" expansions
+
+### Structure Correctness (0.0 - 1.0)
+Is the JSON structure valid and appropriate?
+
+| Score | Meaning |
+|-------|---------|
+| 1.0 | Perfect structure |
+| 0.9 | Minor structural issues |
+| 0.8 | Some misplaced fields |
+| 0.7 | Structure partially matches schema |
+| <0.7 | Major structural problems |
+
+**Check for:**
+- JSON is valid
+- Required fields present
+- Correct schema for element type
+- Arrays where arrays expected
+- Nested structures correct
+
+### Overall Score
+```
+overall = (completeness * 0.35) + (accuracy * 0.30) + (verbatim_compliance * 0.25) + (structure_correctness * 0.10)
+```
+
+## Pass/Fail Decision
+
+```json
+{
+  "pass": overall >= 0.90,
+  "threshold": 0.90
+}
+```
+
+## Issue Severity Levels
+
+| Severity | Impact | Examples |
+|----------|--------|----------|
+| `critical` | Data integrity compromised | Wrong numbers, missing tables, fabricated data |
+| `major` | Significant content affected | Missing legend, wrong axis labels, incomplete rows |
+| `minor` | Small inaccuracies | Capitalization, spacing, minor formatting |
+| `cosmetic` | Non-data formatting | JSON formatting, field ordering |
+
+## Revision Feedback
+
+When `pass: false`, provide specific revision instructions:
+
+```json
+{
+  "revision_feedback": {
+    "revision_number": 1,
+    "max_revisions": 2,
+    "specific_fixes": [
+      "Add missing legend entry: 'Control: 95% CI' with color 'light orange' and style 'shaded'",
+      "Fix axis.x.label capitalization: use 'Time (Days)' not 'Time (days)'",
+      "Add missing risk table row for timepoint 7000"
+    ],
+    "re_extract_sections": ["legend", "risk_table"],
+    "preserve_sections": ["axes", "data_series", "annotations"]
+  }
+}
+```
+
+## Human Review Triggers
+
+Set `needs_human_review: true` when:
+
+1. **Revision limit reached:** Already revised twice, still failing
+2. **Unreadable content:** Image quality too poor
+3. **Complex ambiguity:** Multiple valid interpretations exist
+4. **Unusual format:** Element doesn't fit any schema
+5. **Confidence too low:** Verifier cannot reliably assess
+
+```json
+{
+  "needs_human_review": true,
+  "human_review_reason": "Extraction has been revised twice but legend colors cannot be reliably determined from image quality. Score: 0.85"
+}
+```
+
+## Verification Process
+
+1. **Load source image** - View the original
+2. **Load extraction JSON** - Parse the extraction
+3. **Systematic comparison:**
+   - Title/caption → exact match?
+   - Axes → all labels, all ticks?
+   - Legend → all entries, colors, styles?
+   - Data → all values present and correct?
+   - Annotations → all text captured?
+4. **Score each dimension**
+5. **List all issues found**
+6. **Calculate overall score**
+7. **Determine pass/fail**
+8. **Generate revision feedback if needed**
+
+## Example Verification
+
+**Source:** Kaplan-Meier curve with risk table
+**Extraction:** JSON with missing 95% CI legend entry
+
+```json
+{
+  "verification_type": "extraction_quality",
+  "element_id": "element_004",
+  "source_image": "/output/images/p8_img1.png",
+  "extraction_file": "/output/extractions/element_004.json",
+  "scores": {
+    "completeness": 0.85,
+    "accuracy": 0.98,
+    "verbatim_compliance": 0.92,
+    "structure_correctness": 1.0,
+    "overall": 0.91
+  },
+  "pass": true,
+  "threshold": 0.90,
+  "issues": [
+    {
+      "severity": "major",
+      "category": "completeness",
+      "field": "legend.entries",
+      "extracted": "2 entries (HSV line, Control line)",
+      "expected": "4 entries (lines + CI shading)",
+      "issue": "Missing confidence interval legend entries"
+    },
+    {
+      "severity": "minor",
+      "category": "verbatim",
+      "field": "axes.y.label",
+      "extracted": "Cumulative Risk",
+      "expected": "Cumulative Risk of Dementia",
+      "issue": "Truncated axis label"
+    }
+  ],
+  "revision_feedback": null,
+  "needs_human_review": false,
+  "verification_notes": "Extraction meets threshold despite missing CI legend entries. Minor label truncation noted but overall quality acceptable."
+}
+```
+
+## Output Rules
+
+1. Return ONLY the JSON object
+2. No markdown code fences
+3. No explanatory text
+4. Be specific in issue descriptions
+5. Provide actionable revision feedback
+6. Always include all score dimensions

package/bin/install.js

@@ -4,9 +4,21 @@ const fs = require('fs');
 const path = require('path');
 const os = require('os');
 
-const VERSION = '1.0.5';
+const VERSION = '2.0.0';
 const PACKAGE_NAME = 'structurecc';
 
+// Agent files in v2.0
+const AGENT_FILES = [
+  'structurecc-classifier.md',
+  'structurecc-extract-table.md',
+  'structurecc-extract-chart.md',
+  'structurecc-extract-heatmap.md',
+  'structurecc-extract-diagram.md',
+  'structurecc-extract-multipanel.md',
+  'structurecc-extract-generic.md',
+  'structurecc-verifier.md'
+];
+
 // Colors
 const colors = {
   reset: '\x1b[0m',
@@ -29,19 +41,17 @@ function banner() {
   console.log('');
   console.log(c.cyan + '  ┌─────────────────────────────────────────────────────┐' + c.reset);
   console.log(c.cyan + '  │                                                     │' + c.reset);
-  console.log(c.cyan + '  │  ' + c.bright + 'S T R U C T U R E' + c.reset + c.cyan + '                              │' + c.reset);
+  console.log(c.cyan + '  │  ' + c.bright + 'S T R U C T U R E   v2.0' + c.reset + c.cyan + '                       │' + c.reset);
   console.log(c.cyan + '  │                                                     │' + c.reset);
   console.log(c.cyan + '  │  ' + c.reset + 'Agentic Document Structuring' + c.cyan + '                   │' + c.reset);
-  console.log(c.cyan + '  │  ' + c.dim + 'One command. Every figure. Every table.' + c.reset + c.cyan + '        │' + c.reset);
+  console.log(c.cyan + '  │  ' + c.dim + 'Verbatim extraction. Quality verified.' + c.reset + c.cyan + '         │' + c.reset);
   console.log(c.cyan + '  │                                                     │' + c.reset);
   console.log(c.cyan + '  ├─────────────────────────────────────────────────────┤' + c.reset);
   console.log(c.cyan + '  │                                                     │' + c.reset);
-  console.log(c.cyan + '  │  ' + c.yellow + 'PDF' + c.reset + ' ──▶ ' + c.green + '[Agent 1]' + c.reset + ' ──┐                        ' + c.cyan + '│' + c.reset);
-  console.log(c.cyan + '  │          ' + c.green + '[Agent 2]' + c.reset + ' ──┤                        ' + c.cyan + '│' + c.reset);
-  console.log(c.cyan + '  │          ' + c.green + '[Agent 3]' + c.reset + ' ──┼──▶ ' + c.magenta + 'STRUCTURED.md' + c.reset + '    ' + c.cyan + '│' + c.reset);
-  console.log(c.cyan + '  │          ' + c.green + '[Agent N]' + c.reset + ' ──┘                        ' + c.cyan + '│' + c.reset);
+  console.log(c.cyan + '  │  ' + c.yellow + 'PDF' + c.reset + ' ──▶ ' + c.magenta + '[Classify]' + c.reset + ' ──▶ ' + c.green + '[Extract]' + c.reset + ' ──▶ ' + c.cyan + '[Verify]' + c.reset + ' ' + c.cyan + '│' + c.reset);
+  console.log(c.cyan + '  │                              ↑_______↻_______↓      │' + c.reset);
   console.log(c.cyan + '  │                                                     │' + c.reset);
-  console.log(c.cyan + '  │  ' + c.white + 'Unstructured in. Structured out.' + c.reset + c.cyan + '                │' + c.reset);
+  console.log(c.cyan + '  │  ' + c.white + '3-phase pipeline with quality scoring' + c.reset + c.cyan + '         │' + c.reset);
   console.log(c.cyan + '  │                                                     │' + c.reset);
   console.log(c.cyan + '  └─────────────────────────────────────────────────────┘' + c.reset);
   console.log('');
@@ -83,33 +93,59 @@ function install() {
   const srcCommandsDir = path.join(packageDir, 'commands', 'structure');
   const srcAgentsDir = path.join(packageDir, 'agents');
 
-  log('Installing structurecc...', colors.yellow);
+  log('Installing structurecc v2.0...', colors.yellow);
   log('');
 
   // Install command
   if (fs.existsSync(srcCommandsDir)) {
     copyDir(srcCommandsDir, commandsDir);
-    log('  ✓ Installed /structure command', colors.green);
+    log('  ✓ Installed /structure command (3-phase pipeline)', colors.green);
   }
 
   // Install agents
   if (fs.existsSync(srcAgentsDir)) {
-    const agentFiles = fs.readdirSync(srcAgentsDir);
     ensureDir(agentsDir);
-    for (const file of agentFiles) {
-      if (file.startsWith('structurecc-')) {
-        fs.copyFileSync(
-          path.join(srcAgentsDir, file),
-          path.join(agentsDir, file)
-        );
-        log(`  ✓ Installed ${file.replace('.md', '')}`, colors.green);
+    let installed = 0;
+    let skipped = 0;
+
+    for (const file of AGENT_FILES) {
+      const srcPath = path.join(srcAgentsDir, file);
+      const destPath = path.join(agentsDir, file);
+
+      if (fs.existsSync(srcPath)) {
+        fs.copyFileSync(srcPath, destPath);
+        const agentName = file.replace('.md', '');
+        log(`  ✓ Installed ${agentName}`, colors.green);
+        installed++;
+      } else {
+        log(`  ⚠ Missing ${file}`, colors.yellow);
+        skipped++;
       }
     }
+
+    // Remove old extractor if present
+    const oldExtractor = path.join(agentsDir, 'structurecc-extractor.md');
+    if (fs.existsSync(oldExtractor)) {
+      fs.unlinkSync(oldExtractor);
+      log('  ✓ Removed legacy structurecc-extractor', colors.dim);
+    }
+
+    log('');
+    log(`  Agents installed: ${installed}`, colors.dim);
+    if (skipped > 0) {
+      log(`  Agents skipped: ${skipped}`, colors.yellow);
+    }
   }
 
   log('');
   log(`${colors.green}Done!${colors.reset}`);
   log('');
+  log(`${colors.bright}What's new in v2.0:${colors.reset}`);
+  log(`  • 3-phase pipeline: Classify → Extract → Verify`, colors.dim);
+  log(`  • 7 specialized extractors (tables, charts, heatmaps, etc.)`, colors.dim);
+  log(`  • Verbatim extraction with quality scoring`, colors.dim);
+  log(`  • Auto-revision loop for failed extractions`, colors.dim);
+  log('');
   log(`Run in Claude Code:`, colors.bright);
   log(`  /structure path/to/document.pdf`, colors.cyan);
   log('');
@@ -130,13 +166,17 @@ function uninstall() {
   }
 
   if (fs.existsSync(agentsDir)) {
+    let removed = 0;
+    // Remove all structurecc agents (both old and new)
     const agentFiles = fs.readdirSync(agentsDir);
     for (const file of agentFiles) {
       if (file.startsWith('structurecc-')) {
         fs.unlinkSync(path.join(agentsDir, file));
         log(`  ✓ Removed ${file}`, colors.green);
+        removed++;
       }
     }
+    log(`  Total agents removed: ${removed}`, colors.dim);
   }
 
   log('');
@@ -144,6 +184,14 @@ function uninstall() {
   log('');
 }
 
+function showVersion() {
+  log(`structurecc v${VERSION}`, colors.bright);
+  log('');
+  log('Pipeline: 3-phase with verification', colors.dim);
+  log('Agents: 8 (classifier + 6 extractors + verifier)', colors.dim);
+  log('');
+}
+
 // Main
 const args = process.argv.slice(2);
 
@@ -151,17 +199,33 @@ banner();
 
 if (args.includes('--uninstall') || args.includes('-u')) {
   uninstall();
+} else if (args.includes('--version') || args.includes('-v')) {
+  showVersion();
 } else if (args.includes('--help') || args.includes('-h')) {
   log('Usage: npx structurecc [options]', colors.bright);
   log('');
   log('Options:', colors.bright);
   log('  --help, -h       Show this help', colors.dim);
+  log('  --version, -v    Show version info', colors.dim);
   log('  --uninstall, -u  Remove from Claude Code', colors.dim);
   log('');
   log('After install, use in Claude Code:', colors.bright);
   log('  /structure path/to/document.pdf', colors.cyan);
   log('  /structure path/to/document.docx', colors.cyan);
   log('');
+  log('Pipeline:', colors.bright);
+  log('  Phase 1: Classification (haiku - fast triage)', colors.dim);
+  log('  Phase 2: Specialized extraction (opus - quality)', colors.dim);
+  log('  Phase 3: Verification (sonnet - balance)', colors.dim);
+  log('');
+  log('Extractors:', colors.bright);
+  log('  • structurecc-extract-table    - Tables with cell-by-cell accuracy', colors.dim);
+  log('  • structurecc-extract-chart    - Charts with axes, legends, data', colors.dim);
+  log('  • structurecc-extract-heatmap  - Heatmaps with color scales', colors.dim);
+  log('  • structurecc-extract-diagram  - Flowcharts, timelines, networks', colors.dim);
+  log('  • structurecc-extract-multipanel - Multi-panel figures (A,B,C,D)', colors.dim);
+  log('  • structurecc-extract-generic  - Fallback for other visuals', colors.dim);
+  log('');
 } else {
   install();
 }