vision-electronic-indexing-pi 0.1.0 → 0.1.2

package/.pi/skills/vision-inventory-workflow/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: vision-inventory-workflow
-description: Run the Vision Electronic Indexing workflow for electronics/PCB photos: process images, create parts_to_lookup.json, verify datasheets with web search, fill datasheet_cache.json, regenerate CSV, and summarize uncertainties.
+description: "Run the Vision Electronic Indexing workflow for electronics/PCB photos: process images, create parts_to_lookup.json, verify datasheets with web search, fill datasheet_cache.json, regenerate CSV, and summarize uncertainties."
 ---
 
 # Vision Inventory Workflow

package/README.md

@@ -13,8 +13,8 @@ The core vision step is a local Python MCP server that sends images to Cloudflar
 
 - Processes one electronics image or a folder of images.
 - Extracts visible IC/package markings, confidence, position hints, and review flags.
-- Runs a second IC consensus pass when multiple ICs are detected in one image.
-- Preserves individual IC marking observations, because the model may read one chip correctly and another incorrectly.
+- Supports multiple different ICs in the same image.
+- Preserves individual IC/package marking observations for audit and review.
 - Saves raw JSON for auditability.
 - Creates `parts_to_lookup.json` for datasheet enrichment.
 - Produces a final CSV, using `datasheet_cache.json` when enrichment is available.
@@ -207,21 +207,17 @@ max_side: 4000
 jpeg_quality: 96
 ```
 
-## IC consensus behavior
+## Multiple-IC behavior
 
-For this lab workflow, the program assumes that all ICs visible in one image should be the same part family/marking.
+Images may contain one IC or many different ICs. The vision step does not force all visible ICs in an image to share one marking or part family.
 
 The processing flow is:
 
-1. First pass: general visible inventory extraction.
-2. If multiple ICs are found, second pass: IC-only consensus verification.
-3. Final output includes:
-   - `items`: one consensus IC item when possible.
-   - `ic_marking_observations`: per-chip marking observations.
-   - `first_pass_items`: original first-pass IC candidates.
-   - `warnings`: notes about consensus verification.
+1. General visible inventory extraction from the image.
+2. Each visible IC/package marking is kept as its own candidate when the model returns it separately.
+3. The batch workflow builds one evidence row per image/candidate part, so a single photo can contribute multiple different BOM rows.
 
-This does not guarantee correct OCR. It helps expose uncertainty and preserves alternate readings.
+This does not guarantee correct OCR. It preserves alternate readings and marks uncertain candidates for review.
 
 ## Main batch workflow
 
@@ -339,7 +335,7 @@ output/inventory.csv
 
 ## Final CSV columns
 
-By default, `inventory.csv` is deduplicated by normalized part number. Multiple images with the same IC become one BOM row with `sighting_count` and an `images` list.
+By default, `inventory.csv` is deduplicated by normalized part number. Multiple images, or multiple candidates from the same image, with the same IC become one BOM row with `sighting_count` and an `images` list.
 
 ```text
 normalized_part
@@ -364,9 +360,9 @@ Example BOM row:
 SN74LS283N,SN74LS283N,8,2,74ls (4 bit) adder low power schottky ttl 5v DIP,https://www.ti.com/lit/ds/symlink/sn74ls283.pdf,Texas Instruments,true,high/low,true,"image_001.jpeg | image_002.jpeg","SN74LS283N | SN74S283N","output/raw/image_001.json | output/raw/image_002.json","Verified against TI datasheet"
 ```
 
-The script also writes `inventory_evidence.csv`, which keeps the non-deduplicated per-image rows used to build the BOM. It includes the same per-sighting `amount` estimate before aggregation.
+The script also writes `inventory_evidence.csv`, which keeps the non-deduplicated per-image/candidate rows used to build the BOM. It includes the same per-sighting `amount` estimate before aggregation.
 
-`amount` is estimated from the vision result's IC count. `sighting_count` is the number of image-level sightings that were merged into the BOM row.
+`amount` is estimated from the number of matching IC candidate items/evidence rows for that candidate. `sighting_count` is the number of evidence rows that were merged into the BOM row.
 
 ## MCP server usage
 
@@ -415,22 +411,22 @@ The server uses MCP `stdio` transport, so it is meant to be launched by an MCP-c
       "package_marking": "SN74LS283N",
       "marking_confidence": "medium",
       "likely_part": "SN74LS283N",
-      "description": "consensus result; 4 visible ICs",
-      "position_hint": "multiple ICs",
+      "description": "visible DIP IC marking",
+      "position_hint": "top-right",
       "needs_review": true
-    }
-  ],
-  "warnings": [
-    "Multi-pass IC consensus verification applied."
-  ],
-  "ic_marking_observations": [
+    },
     {
-      "position_hint": "top-right",
-      "package_marking": "SN74LS283N F 7936",
-      "marking_confidence": "high"
+      "item_type": "IC",
+      "count_index": 2,
+      "package_marking": "MAX232N",
+      "marking_confidence": "high",
+      "likely_part": "MAX232N",
+      "description": "visible DIP IC marking",
+      "position_hint": "bottom-left",
+      "needs_review": false
     }
   ],
-  "first_pass_items": []
+  "warnings": []
 }
 ```
 
@@ -461,7 +457,7 @@ Handled cases include:
 - Vision models can misread small or blurry IC markings.
 - A higher-resolution or closer photo usually helps more than prompt changes.
 - Full-board photos are useful for context; cropped IC close-ups are better for marking OCR.
-- The consensus pass can enforce one shared IC result, but it can still choose the wrong consensus.
+- Multiple ICs in one image can still be missed or merged by the vision model if markings are small or blurry.
 - Datasheet enrichment should be verified against official sources.
 - The script does not deduplicate the same physical part across multiple images unless you handle that in the enrichment/review step.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vision-electronic-indexing-pi",
-  "version": "0.1.0",
+  "version": "0.1.2",
   "description": "Pi package for agent-assisted electronics/PCB image inventory with Cloudflare Workers AI vision and datasheet enrichment.",
   "license": "MIT",
   "repository": {
@@ -38,8 +38,14 @@
     "typebox": "*"
   },
   "pi": {
-    "extensions": [".pi/extensions/vision-inventory-mcp/index.ts"],
-    "skills": [".pi/skills"],
-    "prompts": [".pi/prompts"]
+    "extensions": [
+      ".pi/extensions/vision-inventory-mcp/index.ts"
+    ],
+    "skills": [
+      ".pi/skills"
+    ],
+    "prompts": [
+      ".pi/prompts"
+    ]
   }
 }

package/scripts/inventory_folder_to_csv.py

@@ -92,12 +92,8 @@ def candidate_from_item(item: Dict[str, Any]) -> str:
 def extract_part_evidence(image_name: str, result: Dict[str, Any]) -> List[Dict[str, Any]]:
     evidence: List[Dict[str, Any]] = []
 
-    for source, items in (
-        ("items", result.get("items", [])),
-        ("first_pass_items", result.get("first_pass_items", [])),
-    ):
-        if not isinstance(items, list):
-            continue
+    items = result.get("items", [])
+    if isinstance(items, list):
         for item in items:
             if not isinstance(item, dict):
                 continue
@@ -108,7 +104,7 @@ def extract_part_evidence(image_name: str, result: Dict[str, Any]) -> List[Dict[
                 continue
             evidence.append({
                 "image": image_name,
-                "source": source,
+                "source": "items",
                 "position_hint": item.get("position_hint", "unknown"),
                 "observed_marking": marking,
                 "candidate_part": candidate_from_item(item),
@@ -229,18 +225,17 @@ def lookup_enrichment(part: str, cache: Dict[str, Any]) -> Dict[str, Any]:
     return {}
 
 
-def estimate_amount_for_candidate(result: Dict[str, Any], candidate: str) -> int:
-    """Estimate physical IC quantity for one image-level sighting.
+def estimate_amount_for_candidate(result: Dict[str, Any], candidate: str, evidence_count: int = 1) -> int:
+    """Estimate physical IC quantity for one candidate in one image.
 
-    The schema's historical field name is count_index, but in this lab workflow
-    the model often uses it as the count for a grouped consensus item. We sum
-    matching primary items and fall back to one when no usable count is available.
+    Count separate matching IC items. The schema field count_index is treated as
+    an ordinal/index, not a quantity. Fall back to the number of candidate
+    evidence rows when only observations are available.
     """
     items = result.get("items", [])
     if not isinstance(items, list):
-        return 1
+        return max(1, evidence_count)
 
-    amount = 0
     matched = 0
     for item in items:
         if not isinstance(item, dict):
@@ -250,17 +245,10 @@ def estimate_amount_for_candidate(result: Dict[str, Any], candidate: str) -> int
         if candidate_from_item(item).upper() != candidate.upper():
             continue
         matched += 1
-        try:
-            count = int(item.get("count_index", 1))
-        except Exception:
-            count = 1
-        amount += max(1, count)
-
-    if amount > 0:
-        return amount
+
     if matched > 0:
         return matched
-    return 1
+    return max(1, evidence_count)
 
 
 def image_part_rows(results: List[Dict[str, Any]], cache: Dict[str, Any]) -> List[Dict[str, Any]]:
@@ -288,37 +276,41 @@ def image_part_rows(results: List[Dict[str, Any]], cache: Dict[str, Any]) -> Lis
             })
             continue
 
-        # One image contributes one sighting for its most common candidate.
-        counts: Dict[str, int] = defaultdict(int)
+        # One image may contain multiple different IC candidates. Emit one
+        # evidence row per candidate instead of forcing a single image-level part.
+        evidence_by_candidate: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
         for row in evidence:
-            counts[row["candidate_part"].upper()] += 1
-        candidate = sorted(counts.items(), key=lambda kv: kv[1], reverse=True)[0][0]
-        enrichment = lookup_enrichment(candidate, cache)
-        amount = estimate_amount_for_candidate(result, candidate)
-        observed_markings = sorted({row["observed_marking"] for row in evidence})
-        observations = "; ".join(
-            f"{row['position_hint']}: {row['observed_marking']} ({row['marking_confidence']})"
-            for row in evidence
-        )
-        confidence_values = [str(row.get("marking_confidence", "unknown")) for row in evidence]
-        needs_review = any(row.get("needs_review", True) for row in evidence) or not enrichment.get("verified", False)
+            candidate = row["candidate_part"].upper()
+            if candidate and candidate.lower() not in UNKNOWN_MARKINGS:
+                evidence_by_candidate[candidate].append(row)
+
+        for candidate, candidate_evidence in sorted(evidence_by_candidate.items()):
+            enrichment = lookup_enrichment(candidate, cache)
+            amount = estimate_amount_for_candidate(result, candidate, evidence_count=len(candidate_evidence))
+            observed_markings = sorted({row["observed_marking"] for row in candidate_evidence})
+            observations = "; ".join(
+                f"{row['position_hint']}: {row['observed_marking']} ({row['marking_confidence']})"
+                for row in candidate_evidence
+            )
+            confidence_values = [str(row.get("marking_confidence", "unknown")) for row in candidate_evidence]
+            needs_review = any(row.get("needs_review", True) for row in candidate_evidence) or not enrichment.get("verified", False)
 
-        rows.append({
-            "image": image_name,
-            "candidate_part": candidate,
-            "normalized_part": enrichment.get("normalized_part", candidate),
-            "amount": amount,
-            "description": enrichment.get("description", ""),
-            "datasheet_url": enrichment.get("datasheet_url", ""),
-            "manufacturer": enrichment.get("manufacturer", ""),
-            "verified": bool(enrichment.get("verified", False)),
-            "vision_confidence": "/".join(sorted(set(confidence_values))),
-            "needs_review": needs_review,
-            "observed_markings": " | ".join(observed_markings),
-            "observations": observations,
-            "raw_json": entry["raw_json"],
-            "notes": enrichment.get("notes", "Missing datasheet enrichment"),
-        })
+            rows.append({
+                "image": image_name,
+                "candidate_part": candidate,
+                "normalized_part": enrichment.get("normalized_part", candidate),
+                "amount": amount,
+                "description": enrichment.get("description", ""),
+                "datasheet_url": enrichment.get("datasheet_url", ""),
+                "manufacturer": enrichment.get("manufacturer", ""),
+                "verified": bool(enrichment.get("verified", False)),
+                "vision_confidence": "/".join(sorted(set(confidence_values))),
+                "needs_review": needs_review,
+                "observed_markings": " | ".join(observed_markings),
+                "observations": observations,
+                "raw_json": entry["raw_json"],
+                "notes": enrichment.get("notes", "Missing datasheet enrichment"),
+            })
     return rows
 
 

package/vision_inventory_mcp.py

@@ -127,59 +127,6 @@ Rules:
 """.strip()
 
 
-def build_ic_consensus_prompt(image_name: str, first_pass: Dict[str, Any]) -> str:
-    first_pass_json = json.dumps(first_pass, indent=2, ensure_ascii=False)
-    return f"""
-Analyze this electronics image again, but focus ONLY on the IC package top markings.
-
-Image filename: {image_name}
-
-Important known constraint for this dataset:
-- All ICs visible in one image should have the same part marking.
-- The previous pass may contain OCR mistakes or hallucinated similar-looking 74LS part numbers.
-- Do not trust the previous pass. Use it only as a list of candidates to verify against the image.
-- Return ONE consensus IC inventory item for the shared marking, not separate items with different markings.
-- If any character is unclear, use [?] in that character position and set needs_review=true.
-- Prefer low confidence with [?] over a confident but guessed part number.
-- Do not use web lookup.
-
-Previous pass to verify:
-{first_pass_json}
-
-Return only valid JSON using this schema:
-
-{{
-  "image": "{image_name}",
-  "items": [
-    {{
-      "item_type": "IC",
-      "count_index": 1,
-      "package_marking": "one consensus visible marking shared by all ICs, or [?]-marked partial text",
-      "marking_confidence": "high | medium | low | unreadable",
-      "likely_part": "same as package_marking if visible, or unknown",
-      "description": "consensus result; include the visible IC count if clear",
-      "position_hint": "multiple ICs / board-wide IC group / etc.",
-      "needs_review": true
-    }}
-  ],
-  "warnings": [],
-  "ic_marking_observations": [
-    {{
-      "position_hint": "where this individual IC appears",
-      "package_marking": "best visible marking for this individual IC, or [?]-marked partial text",
-      "marking_confidence": "high | medium | low | unreadable"
-    }}
-  ]
-}}
-
-Rules:
-- Return JSON only.
-- Return exactly one consensus IC item if ICs are visible.
-- Also list each visible IC's individual marking observation in ic_marking_observations.
-- Individual observations may disagree; do not force them to match. Use [?] for unclear characters.
-- Do not return multiple different consensus IC items.
-""".strip()
-
 
 mcp = FastMCP("Vision Inventory")
 
@@ -521,50 +468,6 @@ def visible_ic_items(result: Dict[str, Any]) -> List[Dict[str, Any]]:
     ]
 
 
-def verify_ic_consensus_pass(
-    image_data_url: str,
-    image_name: str,
-    first_pass: Dict[str, Any],
-    account_id: str,
-    api_token: str,
-) -> Dict[str, Any]:
-    """Run a second vision pass that enforces the project rule that all ICs match."""
-    ic_items = visible_ic_items(first_pass)
-    if len(ic_items) < 2:
-        return first_pass
-
-    response_text, cloudflare_error = call_workers_ai(
-        image_data_url=image_data_url,
-        image_name=image_name,
-        user_prompt=build_ic_consensus_prompt(image_name, first_pass),
-        account_id=account_id,
-        api_token=api_token,
-        model=DEFAULT_MODEL,
-    )
-    if cloudflare_error:
-        first_pass.setdefault("warnings", []).append(
-            f"IC consensus verification failed: {cloudflare_error.get('message', 'unknown error')}"
-        )
-        return first_pass
-    assert response_text is not None
-
-    parsed, parse_error = extract_json_object(response_text)
-    if parse_error or parsed is None:
-        first_pass.setdefault("warnings", []).append(
-            f"IC consensus verification returned invalid JSON: {parse_error or 'unknown parse error'}"
-        )
-        first_pass["ic_consensus_raw_response"] = response_text
-        return first_pass
-
-    consensus = normalize_inventory_result(parsed, image_name)
-    previous_markings = [str(item.get("package_marking", "unknown")) for item in ic_items]
-    consensus.setdefault("warnings", []).append("Multi-pass IC consensus verification applied.")
-    consensus.setdefault("warnings", []).append(
-        "First pass IC marking candidates: " + ", ".join(previous_markings)
-    )
-    consensus["first_pass_items"] = first_pass.get("items", [])
-    return consensus
-
 
 def process_image_impl(
     image_path: str,
@@ -615,14 +518,7 @@ def process_image_impl(
             "raw_response": response_text,
         }
 
-    first_pass = normalize_inventory_result(parsed, image_name)
-    return verify_ic_consensus_pass(
-        image_data_url=image_data_url,
-        image_name=image_name,
-        first_pass=first_pass,
-        account_id=account_id,
-        api_token=api_token,
-    )
+    return normalize_inventory_result(parsed, image_name)
 
 
 @mcp.tool()