vision-electronic-indexing-pi 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.pi/extensions/vision-inventory-mcp/README.md +76 -15
- package/README.md +182 -305
- package/package.json +1 -1
|
@@ -1,14 +1,50 @@
|
|
|
1
|
-
# Vision
|
|
1
|
+
# Vision Electronic Indexing Pi Extension
|
|
2
2
|
|
|
3
|
-
Pi extension
|
|
3
|
+
Pi extension for the `vision-electronic-indexing-pi` package. It connects Pi to the bundled Python vision workflow and provides commands for electronics/PCB photo inventory.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
## Quick setup
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Install the package:
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
9
|
+
```bash
|
|
10
|
+
pi install npm:vision-electronic-indexing-pi
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Then in Pi:
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
/vision-inventory-setup
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Setup checks Python dependencies, checks whether a web-search/browser tool is available for datasheet lookup, and prompts for Cloudflare Workers AI credentials when needed.
|
|
20
|
+
|
|
21
|
+
Credentials are stored at:
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
~/.pi/agent/vision-inventory/credentials.json
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Change them later with:
|
|
28
|
+
|
|
29
|
+
```text
|
|
30
|
+
/vision-inventory-credentials
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Recommended workflow
|
|
34
|
+
|
|
35
|
+
```text
|
|
36
|
+
/vision-inventory-agent-bom ./photos ./output
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
The agent workflow:
|
|
40
|
+
|
|
41
|
+
1. Processes supported images in the folder.
|
|
42
|
+
2. Writes raw JSON evidence.
|
|
43
|
+
3. Builds `parts_to_lookup.json`.
|
|
44
|
+
4. Uses an installed web-search/browser capability to verify datasheets.
|
|
45
|
+
5. Writes `datasheet_cache.json`.
|
|
46
|
+
6. Regenerates `inventory.csv` and `inventory_evidence.csv`.
|
|
47
|
+
7. Summarizes uncertain rows.
|
|
12
48
|
|
|
13
49
|
## Commands
|
|
14
50
|
|
|
@@ -20,20 +56,45 @@ This repo can be used project-locally or installed as a Pi package. The package
|
|
|
20
56
|
/vision-inventory-agent-bom <image_folder> <output_dir> [options]
|
|
21
57
|
```
|
|
22
58
|
|
|
23
|
-
`/vision-inventory-
|
|
59
|
+
- `/vision-inventory-bom` runs only the deterministic image-to-CSV workflow.
|
|
60
|
+
- `/vision-inventory-agent-bom` runs the full agent-assisted datasheet-enrichment workflow.
|
|
61
|
+
- `/vision-inventory-restart` restarts the local Python vision bridge.
|
|
24
62
|
|
|
25
|
-
|
|
26
|
-
~/.pi/agent/vision-inventory/credentials.json
|
|
27
|
-
```
|
|
63
|
+
Options are forwarded to `scripts/inventory_folder_to_csv.py`, such as `--recursive`, `--limit`, `--max-side`, and `--jpeg-quality`.
|
|
28
64
|
|
|
29
|
-
|
|
65
|
+
## Agent tools
|
|
30
66
|
|
|
31
|
-
|
|
67
|
+
- `vision_inventory_process_image` — analyze one electronics/PCB image.
|
|
68
|
+
- `vision_inventory_process_folder` — analyze all supported images in a folder.
|
|
69
|
+
- `vision_inventory_save` — save inventory output as JSON or CSV.
|
|
32
70
|
|
|
33
71
|
## External dependencies not bundled
|
|
34
72
|
|
|
35
|
-
|
|
73
|
+
This package intentionally does **not** bundle:
|
|
74
|
+
|
|
75
|
+
- Python packages from `requirements.txt`: `mcp`, `requests`, `pillow`, `python-dotenv`; optional `pillow-heif`.
|
|
36
76
|
- A Pi web-search/browser tool or skill for datasheet lookup.
|
|
37
77
|
- Cloudflare Workers AI credentials.
|
|
38
78
|
|
|
39
|
-
|
|
79
|
+
## Output
|
|
80
|
+
|
|
81
|
+
The main output is `inventory.csv`, with columns:
|
|
82
|
+
|
|
83
|
+
```text
|
|
84
|
+
normalized_part
|
|
85
|
+
candidate_parts
|
|
86
|
+
amount
|
|
87
|
+
sighting_count
|
|
88
|
+
description
|
|
89
|
+
datasheet_url
|
|
90
|
+
manufacturer
|
|
91
|
+
verified
|
|
92
|
+
vision_confidence
|
|
93
|
+
needs_review
|
|
94
|
+
images
|
|
95
|
+
observed_markings
|
|
96
|
+
raw_json
|
|
97
|
+
notes
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
`inventory_evidence.csv` keeps the non-deduplicated per-image/per-candidate rows. One image can produce multiple rows when it contains multiple different ICs.
|
package/README.md
CHANGED
|
@@ -1,136 +1,166 @@
|
|
|
1
|
-
# Vision Electronic Indexing
|
|
1
|
+
# Vision Electronic Indexing for Pi
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Agent-assisted electronics/PCB photo indexing for Pi. The package processes images with Cloudflare Workers AI, extracts visible IC/package markings, prepares parts for datasheet lookup, and produces an enriched inventory CSV.
|
|
4
|
+
|
|
5
|
+
Typical flow:
|
|
4
6
|
|
|
5
7
|
```text
|
|
6
|
-
|
|
7
|
-
-> agent/user verifies datasheets -> enriched inventory is exported to CSV
|
|
8
|
+
photos -> vision extraction -> raw JSON + evidence -> agent datasheet verification -> inventory.csv
|
|
8
9
|
```
|
|
9
10
|
|
|
10
|
-
The
|
|
11
|
-
|
|
12
|
-
## What this project does
|
|
13
|
-
|
|
14
|
-
- Processes one electronics image or a folder of images.
|
|
15
|
-
- Extracts visible IC/package markings, confidence, position hints, and review flags.
|
|
16
|
-
- Supports multiple different ICs in the same image.
|
|
17
|
-
- Preserves individual IC/package marking observations for audit and review.
|
|
18
|
-
- Saves raw JSON for auditability.
|
|
19
|
-
- Creates `parts_to_lookup.json` for datasheet enrichment.
|
|
20
|
-
- Produces a final CSV, using `datasheet_cache.json` when enrichment is available.
|
|
11
|
+
The vision step does **not** perform datasheet lookup or invent part details. Datasheet enrichment is handled by a Pi agent with a web-search/browser tool or by manual review.
|
|
21
12
|
|
|
22
|
-
##
|
|
13
|
+
## Quick setup with Pi
|
|
23
14
|
|
|
24
|
-
|
|
15
|
+
### 1. Install the Pi package
|
|
25
16
|
|
|
26
|
-
|
|
17
|
+
```bash
|
|
18
|
+
pi install npm:vision-electronic-indexing-pi
|
|
19
|
+
```
|
|
27
20
|
|
|
28
|
-
|
|
21
|
+
For local development from this repository, open the repo in Pi and trust the project. Do not also install the npm package while working inside this repo, because the project-local extension and npm package register the same tools.
|
|
29
22
|
|
|
30
|
-
|
|
23
|
+
### 2. Install/enable an agent web-search dependency
|
|
31
24
|
|
|
32
|
-
|
|
33
|
-
vision_inventory_mcp.py # MCP server and core image-processing logic
|
|
34
|
-
inventory.py # Original GUI/helper app
|
|
35
|
-
requirements.txt # Python dependencies
|
|
36
|
-
scripts/inventory_folder_to_csv.py # Batch workflow script
|
|
37
|
-
.pi/extensions/vision-inventory-mcp/ # Project-local Pi extension
|
|
38
|
-
```
|
|
25
|
+
Datasheet enrichment requires a Pi web-search or browser tool/skill. This package intentionally does **not** bundle one.
|
|
39
26
|
|
|
40
|
-
|
|
27
|
+
Examples of acceptable capabilities:
|
|
41
28
|
|
|
42
|
-
|
|
29
|
+
- a Brave/search Pi skill
|
|
30
|
+
- a browser automation skill
|
|
31
|
+
- another trusted web-search extension/tool
|
|
43
32
|
|
|
44
|
-
|
|
33
|
+
If no search/browser capability is available, the agent workflow can still generate `parts_to_lookup.json`, but it cannot verify datasheets.
|
|
45
34
|
|
|
46
|
-
|
|
47
|
-
python3 -m pip install -r requirements.txt
|
|
48
|
-
```
|
|
35
|
+
### 3. Configure Cloudflare credentials
|
|
49
36
|
|
|
50
|
-
|
|
37
|
+
Start Pi and run:
|
|
51
38
|
|
|
52
|
-
```
|
|
53
|
-
|
|
39
|
+
```text
|
|
40
|
+
/vision-inventory-setup
|
|
54
41
|
```
|
|
55
42
|
|
|
56
|
-
|
|
43
|
+
The setup command checks Python dependencies, checks for web-search/browser capability, and prompts for Cloudflare Workers AI credentials the first time.
|
|
57
44
|
|
|
58
|
-
|
|
45
|
+
Credentials are stored at:
|
|
59
46
|
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
CLOUDFLARE_AUTH_TOKEN=your_cloudflare_workers_ai_token
|
|
47
|
+
```text
|
|
48
|
+
~/.pi/agent/vision-inventory/credentials.json
|
|
63
49
|
```
|
|
64
50
|
|
|
65
|
-
The
|
|
51
|
+
The file is written with `chmod 600` when supported.
|
|
66
52
|
|
|
67
|
-
|
|
68
|
-
|
|
53
|
+
To change credentials later:
|
|
54
|
+
|
|
55
|
+
```text
|
|
56
|
+
/vision-inventory-credentials
|
|
69
57
|
```
|
|
70
58
|
|
|
71
|
-
|
|
59
|
+
Environment variables also work and override stored credentials:
|
|
72
60
|
|
|
73
|
-
```
|
|
74
|
-
|
|
61
|
+
```bash
|
|
62
|
+
export CLOUDFLARE_ACCOUNT_ID=your_account_id
|
|
63
|
+
export CLOUDFLARE_AUTH_TOKEN=your_workers_ai_token
|
|
64
|
+
# or
|
|
65
|
+
export CLOUDFLARE_API_TOKEN=your_workers_ai_token
|
|
75
66
|
```
|
|
76
67
|
|
|
77
|
-
|
|
68
|
+
Optional model override:
|
|
78
69
|
|
|
79
70
|
```bash
|
|
80
71
|
export WORKERS_AI_MODEL=@cf/meta/llama-4-scout-17b-16e-instruct
|
|
81
72
|
```
|
|
82
73
|
|
|
83
|
-
##
|
|
74
|
+
## Recommended workflow
|
|
84
75
|
|
|
85
|
-
|
|
76
|
+
### 1. Take photos
|
|
86
77
|
|
|
87
|
-
|
|
88
|
-
pi install git:github.com/<you>/<this-repo>
|
|
89
|
-
```
|
|
78
|
+
Take clear, close photos of IC groups or PCB sections. Cropped, well-lit IC close-ups work better than full-board photos for OCR.
|
|
90
79
|
|
|
91
|
-
|
|
80
|
+
Example folder:
|
|
92
81
|
|
|
93
82
|
```text
|
|
94
|
-
|
|
95
|
-
.
|
|
96
|
-
.
|
|
83
|
+
photos/
|
|
84
|
+
image_001.jpeg
|
|
85
|
+
image_002.jpeg
|
|
86
|
+
image_003.jpeg
|
|
97
87
|
```
|
|
98
88
|
|
|
99
|
-
|
|
89
|
+
### 2. Run the full agent workflow
|
|
100
90
|
|
|
101
|
-
|
|
91
|
+
In Pi:
|
|
102
92
|
|
|
103
|
-
|
|
104
|
-
-
|
|
105
|
-
|
|
93
|
+
```text
|
|
94
|
+
/vision-inventory-agent-bom ./photos ./output
|
|
95
|
+
```
|
|
106
96
|
|
|
107
|
-
|
|
97
|
+
Useful options:
|
|
108
98
|
|
|
109
99
|
```text
|
|
110
|
-
/vision-inventory-
|
|
100
|
+
/vision-inventory-agent-bom ./photos ./output --recursive
|
|
101
|
+
/vision-inventory-agent-bom ./photos ./output --limit 3
|
|
102
|
+
/vision-inventory-agent-bom ./photos ./output --max-side 4000 --jpeg-quality 96
|
|
111
103
|
```
|
|
112
104
|
|
|
113
|
-
The
|
|
105
|
+
The agent workflow will:
|
|
106
|
+
|
|
107
|
+
1. Run the image-processing batch script.
|
|
108
|
+
2. Write raw JSON files for auditability.
|
|
109
|
+
3. Build `parts_to_lookup.json`.
|
|
110
|
+
4. Search for datasheets, preferring official manufacturer sources.
|
|
111
|
+
5. Fill `datasheet_cache.json`.
|
|
112
|
+
6. Rerun the CSV generation with `--skip-vision`.
|
|
113
|
+
7. Summarize the BOM and call out uncertain rows.
|
|
114
|
+
|
|
115
|
+
### 3. Review output
|
|
116
|
+
|
|
117
|
+
Generated files:
|
|
114
118
|
|
|
115
119
|
```text
|
|
116
|
-
|
|
120
|
+
output/
|
|
121
|
+
raw/ # one raw JSON result per image
|
|
122
|
+
parts_to_lookup.json # parts/evidence requiring datasheet lookup
|
|
123
|
+
datasheet_cache.template.json # enrichment template
|
|
124
|
+
datasheet_cache.json # agent/user-filled enrichment cache
|
|
125
|
+
inventory.csv # deduplicated final BOM
|
|
126
|
+
inventory_evidence.csv # per-image/per-candidate evidence rows
|
|
117
127
|
```
|
|
118
128
|
|
|
119
|
-
|
|
129
|
+
Always review rows where:
|
|
120
130
|
|
|
121
131
|
```text
|
|
122
|
-
|
|
132
|
+
needs_review=true
|
|
133
|
+
verified=false
|
|
123
134
|
```
|
|
124
135
|
|
|
125
|
-
|
|
136
|
+
## CSV output columns
|
|
126
137
|
|
|
127
|
-
|
|
128
|
-
|---|---|
|
|
129
|
-
| `vision_inventory_process_image` | Analyze one electronics image. |
|
|
130
|
-
| `vision_inventory_process_folder` | Analyze all supported images in a folder. |
|
|
131
|
-
| `vision_inventory_save` | Save inventory output as JSON or CSV. |
|
|
138
|
+
`inventory.csv` is deduplicated by normalized part number. Multiple images, or multiple candidates from one image, can merge into one BOM row.
|
|
132
139
|
|
|
133
|
-
|
|
140
|
+
Columns:
|
|
141
|
+
|
|
142
|
+
| Column | Description |
|
|
143
|
+
|---|---|
|
|
144
|
+
| `normalized_part` | Final normalized part number, usually from datasheet enrichment. |
|
|
145
|
+
| `candidate_parts` | Candidate part numbers extracted from visual markings. |
|
|
146
|
+
| `amount` | Estimated quantity for the merged BOM row. |
|
|
147
|
+
| `sighting_count` | Number of evidence rows merged into this BOM row. |
|
|
148
|
+
| `description` | Short datasheet-verified description. |
|
|
149
|
+
| `datasheet_url` | Datasheet/source URL used for enrichment. |
|
|
150
|
+
| `manufacturer` | Verified or likely manufacturer. |
|
|
151
|
+
| `verified` | `true` only when datasheet match was verified. |
|
|
152
|
+
| `vision_confidence` | Vision/OCR confidence values observed for the row. |
|
|
153
|
+
| `needs_review` | `true` when OCR or datasheet enrichment is uncertain. |
|
|
154
|
+
| `images` | Source images contributing to the row. |
|
|
155
|
+
| `observed_markings` | Raw visible markings seen on packages. |
|
|
156
|
+
| `raw_json` | Raw JSON files used as evidence. |
|
|
157
|
+
| `notes` | Datasheet/enrichment notes or uncertainty explanations. |
|
|
158
|
+
|
|
159
|
+
`inventory_evidence.csv` keeps the non-deduplicated evidence rows used to build the BOM. A single photo can produce multiple evidence rows when it contains multiple different ICs.
|
|
160
|
+
|
|
161
|
+
## Commands and tools
|
|
162
|
+
|
|
163
|
+
### Pi commands
|
|
134
164
|
|
|
135
165
|
```text
|
|
136
166
|
/vision-inventory-setup
|
|
@@ -140,57 +170,75 @@ It also adds these commands:
|
|
|
140
170
|
/vision-inventory-agent-bom <image_folder> <output_dir> [options]
|
|
141
171
|
```
|
|
142
172
|
|
|
143
|
-
|
|
173
|
+
Command summary:
|
|
144
174
|
|
|
145
|
-
|
|
175
|
+
| Command | Purpose |
|
|
176
|
+
|---|---|
|
|
177
|
+
| `/vision-inventory-setup` | Configure credentials and check dependencies. |
|
|
178
|
+
| `/vision-inventory-credentials` | Change stored Cloudflare credentials. |
|
|
179
|
+
| `/vision-inventory-restart` | Restart the local vision bridge process. |
|
|
180
|
+
| `/vision-inventory-bom` | Run only the deterministic image-to-CSV workflow. |
|
|
181
|
+
| `/vision-inventory-agent-bom` | Run the full agent-assisted workflow, including datasheet enrichment. |
|
|
146
182
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
183
|
+
### Pi tools exposed to agents
|
|
184
|
+
|
|
185
|
+
| Tool | Purpose |
|
|
186
|
+
|---|---|
|
|
187
|
+
| `vision_inventory_process_image` | Analyze one electronics/PCB image. |
|
|
188
|
+
| `vision_inventory_process_folder` | Analyze a folder of supported images. |
|
|
189
|
+
| `vision_inventory_save` | Save inventory output as JSON or CSV. |
|
|
190
|
+
|
|
191
|
+
## Manual shell workflow
|
|
192
|
+
|
|
193
|
+
You can run the deterministic workflow without Pi commands:
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
python3 -m pip install -r requirements.txt
|
|
197
|
+
python3 scripts/inventory_folder_to_csv.py ./photos ./output
|
|
151
198
|
```
|
|
152
199
|
|
|
153
|
-
|
|
200
|
+
Then fill `output/datasheet_cache.json` manually or with an agent, and regenerate the CSV without reprocessing images:
|
|
201
|
+
|
|
202
|
+
```bash
|
|
203
|
+
python3 scripts/inventory_folder_to_csv.py ./photos ./output --skip-vision
|
|
204
|
+
```
|
|
154
205
|
|
|
155
|
-
|
|
206
|
+
## Python requirements
|
|
207
|
+
|
|
208
|
+
Python 3.10 or newer is recommended.
|
|
209
|
+
|
|
210
|
+
Required:
|
|
156
211
|
|
|
157
212
|
```text
|
|
158
|
-
|
|
159
|
-
|
|
213
|
+
mcp
|
|
214
|
+
requests
|
|
215
|
+
pillow
|
|
216
|
+
python-dotenv
|
|
160
217
|
```
|
|
161
218
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
The extension launches:
|
|
219
|
+
Install:
|
|
165
220
|
|
|
166
221
|
```bash
|
|
167
|
-
python3
|
|
222
|
+
python3 -m pip install -r requirements.txt
|
|
168
223
|
```
|
|
169
224
|
|
|
170
|
-
|
|
225
|
+
Optional HEIC/HEIF support for iPhone photos:
|
|
171
226
|
|
|
172
227
|
```bash
|
|
173
|
-
|
|
228
|
+
python3 -m pip install pillow-heif
|
|
174
229
|
```
|
|
175
230
|
|
|
176
|
-
|
|
231
|
+
Supported image formats:
|
|
177
232
|
|
|
178
233
|
```text
|
|
179
|
-
.jpg
|
|
180
|
-
.jpeg
|
|
181
|
-
.png
|
|
182
|
-
.webp
|
|
183
|
-
.bmp
|
|
184
|
-
.gif
|
|
185
|
-
.heic
|
|
186
|
-
.heif
|
|
234
|
+
.jpg .jpeg .png .webp .bmp .gif .heic .heif
|
|
187
235
|
```
|
|
188
236
|
|
|
189
237
|
HEIC/HEIF requires `pillow-heif`.
|
|
190
238
|
|
|
191
|
-
##
|
|
239
|
+
## How image processing works
|
|
192
240
|
|
|
193
|
-
Before sending an image to Cloudflare Workers AI, the server:
|
|
241
|
+
Before sending an image to Cloudflare Workers AI, the Python server:
|
|
194
242
|
|
|
195
243
|
1. Opens the image with Pillow.
|
|
196
244
|
2. Applies EXIF orientation correction.
|
|
@@ -200,108 +248,35 @@ Before sending an image to Cloudflare Workers AI, the server:
|
|
|
200
248
|
6. Encodes it as JPEG.
|
|
201
249
|
7. Sends it as a base64 image data URL.
|
|
202
250
|
|
|
203
|
-
|
|
251
|
+
Defaults:
|
|
204
252
|
|
|
205
253
|
```text
|
|
206
254
|
max_side: 4000
|
|
207
255
|
jpeg_quality: 96
|
|
256
|
+
model: @cf/meta/llama-4-scout-17b-16e-instruct
|
|
208
257
|
```
|
|
209
258
|
|
|
210
259
|
## Multiple-IC behavior
|
|
211
260
|
|
|
212
|
-
Images may contain one IC or many different ICs. The
|
|
261
|
+
Images may contain one IC or many different ICs. The workflow does not force all visible ICs in one image to share the same marking or part family.
|
|
213
262
|
|
|
214
|
-
The
|
|
263
|
+
The batch workflow builds one evidence row per image/candidate part, so one photo can contribute several BOM rows.
|
|
215
264
|
|
|
216
|
-
|
|
217
|
-
2. Each visible IC/package marking is kept as its own candidate when the model returns it separately.
|
|
218
|
-
3. The batch workflow builds one evidence row per image/candidate part, so a single photo can contribute multiple different BOM rows.
|
|
265
|
+
This improves handling of mixed IC photos, but OCR can still miss, merge, or misread small markings. Review raw JSON and evidence rows when accuracy matters.
|
|
219
266
|
|
|
220
|
-
|
|
267
|
+
## Datasheet enrichment rules
|
|
221
268
|
|
|
222
|
-
|
|
269
|
+
The agent should:
|
|
223
270
|
|
|
224
|
-
|
|
271
|
+
- Prefer official manufacturer datasheets or product pages.
|
|
272
|
+
- Keep descriptions short.
|
|
273
|
+
- Set `verified=false` if the marking, part number, package, or source is uncertain.
|
|
274
|
+
- Do not invent part numbers, manufacturers, voltages, functions, or datasheet URLs.
|
|
225
275
|
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
```text
|
|
229
|
-
/vision-inventory-bom ./photos ./output
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
From a normal shell, run the underlying script directly as shown below.
|
|
233
|
-
|
|
234
|
-
### 1. Put images in a folder
|
|
235
|
-
|
|
236
|
-
Example:
|
|
237
|
-
|
|
238
|
-
```text
|
|
239
|
-
photos/
|
|
240
|
-
image_001.jpeg
|
|
241
|
-
image_002.jpeg
|
|
242
|
-
image_003.jpeg
|
|
243
|
-
```
|
|
244
|
-
|
|
245
|
-
### 2. Run the batch workflow
|
|
246
|
-
|
|
247
|
-
```bash
|
|
248
|
-
python3 scripts/inventory_folder_to_csv.py ./photos ./output
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
This creates:
|
|
252
|
-
|
|
253
|
-
```text
|
|
254
|
-
output/
|
|
255
|
-
raw/ # one raw JSON result per image
|
|
256
|
-
parts_to_lookup.json # parts/evidence that need datasheet lookup
|
|
257
|
-
datasheet_cache.template.json # template for enrichment
|
|
258
|
-
inventory.csv # deduplicated BOM CSV, possibly with missing enrichment fields
|
|
259
|
-
inventory_evidence.csv # per-image evidence CSV used to build the BOM
|
|
260
|
-
```
|
|
261
|
-
|
|
262
|
-
Use recursive folder scanning if needed:
|
|
263
|
-
|
|
264
|
-
```bash
|
|
265
|
-
python3 scripts/inventory_folder_to_csv.py ./photos ./output --recursive
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
Limit images during testing:
|
|
269
|
-
|
|
270
|
-
```bash
|
|
271
|
-
python3 scripts/inventory_folder_to_csv.py ./photos ./output --limit 3
|
|
272
|
-
```
|
|
273
|
-
|
|
274
|
-
### 3. Enrich datasheets with Pi/web search
|
|
275
|
-
|
|
276
|
-
Open `output/parts_to_lookup.json`. For each part, search for the datasheet and verify the description.
|
|
277
|
-
|
|
278
|
-
Prefer official manufacturer datasheets when possible:
|
|
279
|
-
|
|
280
|
-
1. Texas Instruments
|
|
281
|
-
2. Analog Devices / Maxim
|
|
282
|
-
3. STMicroelectronics
|
|
283
|
-
4. ONsemi
|
|
284
|
-
5. Nexperia
|
|
285
|
-
6. Other manufacturer PDFs
|
|
286
|
-
|
|
287
|
-
Copy the template:
|
|
288
|
-
|
|
289
|
-
```bash
|
|
290
|
-
cp output/datasheet_cache.template.json output/datasheet_cache.json
|
|
291
|
-
```
|
|
292
|
-
|
|
293
|
-
Fill each entry. Example:
|
|
276
|
+
Example `datasheet_cache.json` entry:
|
|
294
277
|
|
|
295
278
|
```json
|
|
296
279
|
{
|
|
297
|
-
"SN74LS283N": {
|
|
298
|
-
"normalized_part": "SN74LS283N",
|
|
299
|
-
"description": "74ls (4 bit) adder low power schottky ttl 5v DIP",
|
|
300
|
-
"datasheet_url": "https://www.ti.com/lit/ds/symlink/sn74ls283.pdf",
|
|
301
|
-
"manufacturer": "Texas Instruments",
|
|
302
|
-
"verified": true,
|
|
303
|
-
"notes": "Verified against TI datasheet. N package is PDIP."
|
|
304
|
-
},
|
|
305
280
|
"MAX232N": {
|
|
306
281
|
"normalized_part": "MAX232N",
|
|
307
282
|
"description": "MAX232N dual EIA-232/RS-232 driver receiver 5v DIP",
|
|
@@ -313,76 +288,25 @@ Fill each entry. Example:
|
|
|
313
288
|
}
|
|
314
289
|
```
|
|
315
290
|
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
```text
|
|
319
|
-
Read output/parts_to_lookup.json. For each part, web-search the datasheet, prefer official manufacturer PDFs, verify function/package/voltage/family, then fill output/datasheet_cache.json. Keep descriptions in this short format: "74ls (4 bit) adder low power schottky ttl 5v DIP". Set verified=false if uncertain.
|
|
320
|
-
```
|
|
321
|
-
|
|
322
|
-
### 4. Regenerate CSV without reprocessing images
|
|
323
|
-
|
|
324
|
-
After filling `datasheet_cache.json`, rerun:
|
|
325
|
-
|
|
326
|
-
```bash
|
|
327
|
-
python3 scripts/inventory_folder_to_csv.py ./photos ./output --skip-vision
|
|
328
|
-
```
|
|
329
|
-
|
|
330
|
-
This reuses `output/raw/*.json` and writes a new enriched:
|
|
331
|
-
|
|
332
|
-
```text
|
|
333
|
-
output/inventory.csv
|
|
334
|
-
```
|
|
335
|
-
|
|
336
|
-
## Final CSV columns
|
|
337
|
-
|
|
338
|
-
By default, `inventory.csv` is deduplicated by normalized part number. Multiple images, or multiple candidates from the same image, with the same IC become one BOM row with `sighting_count` and an `images` list.
|
|
339
|
-
|
|
340
|
-
```text
|
|
341
|
-
normalized_part
|
|
342
|
-
candidate_parts
|
|
343
|
-
amount
|
|
344
|
-
sighting_count
|
|
345
|
-
description
|
|
346
|
-
datasheet_url
|
|
347
|
-
manufacturer
|
|
348
|
-
verified
|
|
349
|
-
vision_confidence
|
|
350
|
-
needs_review
|
|
351
|
-
images
|
|
352
|
-
observed_markings
|
|
353
|
-
raw_json
|
|
354
|
-
notes
|
|
355
|
-
```
|
|
356
|
-
|
|
357
|
-
Example BOM row:
|
|
358
|
-
|
|
359
|
-
```csv
|
|
360
|
-
SN74LS283N,SN74LS283N,8,2,74ls (4 bit) adder low power schottky ttl 5v DIP,https://www.ti.com/lit/ds/symlink/sn74ls283.pdf,Texas Instruments,true,high/low,true,"image_001.jpeg | image_002.jpeg","SN74LS283N | SN74S283N","output/raw/image_001.json | output/raw/image_002.json","Verified against TI datasheet"
|
|
361
|
-
```
|
|
291
|
+
## Local MCP server details
|
|
362
292
|
|
|
363
|
-
|
|
293
|
+
Internally, the vision step is implemented by `vision_inventory_mcp.py`, a local Python MCP stdio server. The Pi extension starts this server lazily and exposes its functionality as Pi tools.
|
|
364
294
|
|
|
365
|
-
|
|
295
|
+
MCP tools:
|
|
366
296
|
|
|
367
|
-
|
|
297
|
+
| MCP tool | Purpose |
|
|
298
|
+
|---|---|
|
|
299
|
+
| `process_image` | Analyze one image and return structured visible inventory data. |
|
|
300
|
+
| `process_image_folder` | Analyze all supported images in a folder. |
|
|
301
|
+
| `save_inventory` | Save inventory output to JSON or CSV. |
|
|
368
302
|
|
|
369
|
-
Run
|
|
303
|
+
Run directly for MCP-compatible clients:
|
|
370
304
|
|
|
371
305
|
```bash
|
|
372
306
|
python3 vision_inventory_mcp.py
|
|
373
307
|
```
|
|
374
308
|
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
### MCP tools
|
|
378
|
-
|
|
379
|
-
| MCP tool | Purpose |
|
|
380
|
-
|---|---|
|
|
381
|
-
| `process_image` | Analyze one image and return visible inventory data. |
|
|
382
|
-
| `process_image_folder` | Analyze all supported images in a folder. |
|
|
383
|
-
| `save_inventory` | Save inventory output to JSON or CSV. |
|
|
384
|
-
|
|
385
|
-
### Example MCP client configuration
|
|
309
|
+
Example MCP client configuration:
|
|
386
310
|
|
|
387
311
|
```json
|
|
388
312
|
{
|
|
@@ -399,40 +323,9 @@ The server uses MCP `stdio` transport, so it is meant to be launched by an MCP-c
|
|
|
399
323
|
}
|
|
400
324
|
```
|
|
401
325
|
|
|
402
|
-
## Single-image output example
|
|
403
|
-
|
|
404
|
-
```json
|
|
405
|
-
{
|
|
406
|
-
"image": "image_001.jpeg",
|
|
407
|
-
"items": [
|
|
408
|
-
{
|
|
409
|
-
"item_type": "IC",
|
|
410
|
-
"count_index": 1,
|
|
411
|
-
"package_marking": "SN74LS283N",
|
|
412
|
-
"marking_confidence": "medium",
|
|
413
|
-
"likely_part": "SN74LS283N",
|
|
414
|
-
"description": "visible DIP IC marking",
|
|
415
|
-
"position_hint": "top-right",
|
|
416
|
-
"needs_review": true
|
|
417
|
-
},
|
|
418
|
-
{
|
|
419
|
-
"item_type": "IC",
|
|
420
|
-
"count_index": 2,
|
|
421
|
-
"package_marking": "MAX232N",
|
|
422
|
-
"marking_confidence": "high",
|
|
423
|
-
"likely_part": "MAX232N",
|
|
424
|
-
"description": "visible DIP IC marking",
|
|
425
|
-
"position_hint": "bottom-left",
|
|
426
|
-
"needs_review": false
|
|
427
|
-
}
|
|
428
|
-
],
|
|
429
|
-
"warnings": []
|
|
430
|
-
}
|
|
431
|
-
```
|
|
432
|
-
|
|
433
326
|
## Error handling
|
|
434
327
|
|
|
435
|
-
The server returns structured errors
|
|
328
|
+
The server returns structured errors where possible, for example:
|
|
436
329
|
|
|
437
330
|
```json
|
|
438
331
|
{
|
|
@@ -444,34 +337,18 @@ The server returns structured errors when possible:
|
|
|
444
337
|
Handled cases include:
|
|
445
338
|
|
|
446
339
|
- Missing Cloudflare credentials.
|
|
447
|
-
- Invalid image
|
|
448
|
-
- Unsupported image
|
|
340
|
+
- Invalid image paths.
|
|
341
|
+
- Unsupported image extensions.
|
|
449
342
|
- Failed image preprocessing.
|
|
450
343
|
- Cloudflare API errors.
|
|
451
344
|
- Invalid JSON from the model.
|
|
452
|
-
- Missing or invalid
|
|
345
|
+
- Missing or invalid folders.
|
|
453
346
|
- Save/write failures.
|
|
454
347
|
|
|
455
|
-
##
|
|
456
|
-
|
|
457
|
-
- Vision models can misread small or blurry IC markings.
|
|
458
|
-
- A higher-resolution or closer photo usually helps more than prompt changes.
|
|
459
|
-
- Full-board photos are useful for context; cropped IC close-ups are better for marking OCR.
|
|
460
|
-
- Multiple ICs in one image can still be missed or merged by the vision model if markings are small or blurry.
|
|
461
|
-
- Datasheet enrichment should be verified against official sources.
|
|
462
|
-
- The script does not deduplicate the same physical part across multiple images unless you handle that in the enrichment/review step.
|
|
463
|
-
|
|
464
|
-
## Recommended operating procedure
|
|
465
|
-
|
|
466
|
-
1. Install the Pi package/extension and run `/vision-inventory-setup` once.
|
|
467
|
-
2. Take a clear photo of each IC group.
|
|
468
|
-
3. Save photos into a dated folder.
|
|
469
|
-
4. Run the full agent workflow:
|
|
470
|
-
|
|
471
|
-
```text
|
|
472
|
-
/vision-inventory-agent-bom ./photos ./output
|
|
473
|
-
```
|
|
474
|
-
|
|
475
|
-
5. Review `inventory.csv`, especially rows with `needs_review=true` or `verified=false`.
|
|
348
|
+
## Limitations
|
|
476
349
|
|
|
477
|
-
|
|
350
|
+
- Vision models can misread small, blurry, or low-contrast IC markings.
|
|
351
|
+
- Cropped close-ups usually help more than prompt changes.
|
|
352
|
+
- Datasheet enrichment depends on the quality of the installed web-search/browser tool.
|
|
353
|
+
- Official datasheets should still be reviewed for important work.
|
|
354
|
+
- The workflow deduplicates by normalized part number, not by physical component identity across overlapping photos.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "vision-electronic-indexing-pi",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "Pi package for agent-assisted electronics/PCB image inventory with Cloudflare Workers AI vision and datasheet enrichment.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|