@camstack/develop 0.1.5 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/skills.d.ts.map +1 -1
- package/dist/skills.js +5 -0
- package/package.json +1 -1
- package/skills/README.md +4 -1
- package/skills/track-review.md +130 -0
package/dist/skills.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"skills.d.ts","sourceRoot":"","sources":["../src/skills.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,YAAY,CAAA;AAI3C,wBAAgB,gBAAgB,IAAI,MAAM,CAMzC;
|
|
1
|
+
{"version":3,"file":"skills.d.ts","sourceRoot":"","sources":["../src/skills.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,YAAY,CAAA;AAI3C,wBAAgB,gBAAgB,IAAI,MAAM,CAMzC;AAiCD,wBAAgB,UAAU,IAAI,SAAS,SAAS,EAAE,CAQjD;AAED,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,GAAG;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,GAAG,IAAI,CAMjF"}
|
package/dist/skills.js
CHANGED
|
@@ -24,6 +24,11 @@ const SKILL_FILES = [
|
|
|
24
24
|
file: 'architecture.md',
|
|
25
25
|
title: 'Architecture pointers for plugin authors',
|
|
26
26
|
},
|
|
27
|
+
{
|
|
28
|
+
id: 'track-review',
|
|
29
|
+
file: 'track-review.md',
|
|
30
|
+
title: 'Review wrong-class tracks and curate the re-training set',
|
|
31
|
+
},
|
|
27
32
|
];
|
|
28
33
|
/**
|
|
29
34
|
* The ids before the 2026-10-09 rename to the server repository's skill names.
|
package/package.json
CHANGED
package/skills/README.md
CHANGED
|
@@ -11,6 +11,7 @@ third-party use (no private hostnames, no hub-specific ops).
|
|
|
11
11
|
| `camstack-secrets` | [camstack-secrets.md](./camstack-secrets.md) | Password / token / key fields |
|
|
12
12
|
| `camstack-docs` | [camstack-docs.md](./camstack-docs.md) | Generated vs prose docs |
|
|
13
13
|
| `architecture` | [architecture.md](./architecture.md) | Plugin-author architecture pointers |
|
|
14
|
+
| `track-review` | [track-review.md](./track-review.md) | Find wrong-class tracks, curate the re-training set (`track_review_*` MCP tools) |
|
|
14
15
|
|
|
15
16
|
These files are the canonical public text. In the server repository the
|
|
16
17
|
matching `.claude/skills/<id>/SKILL.md` is an overlay that points here and keeps
|
|
@@ -42,7 +43,9 @@ npx @camstack/mcp-develop
|
|
|
42
43
|
```
|
|
43
44
|
|
|
44
45
|
Tools wrap the same `@camstack/develop` operations (`list_addons`,
|
|
45
|
-
`scaffold_addon`, `validate_addon`, `find_example`, `get_skill`, …).
|
|
46
|
+
`scaffold_addon`, `validate_addon`, `find_example`, `get_skill`, …). The
|
|
47
|
+
`track_review_*` tools additionally talk to a hub — see the `track-review` skill
|
|
48
|
+
for the environment variables and the confirm-before-write rule.
|
|
46
49
|
|
|
47
50
|
### Agents
|
|
48
51
|
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: track-review
|
|
3
|
+
description: Review recent tracks for a wrong class and curate the corrected frames into the re-training set — triage suspects, look at each, give a verdict, show the operator the write list, then apply it. Use for "find tracks the detector got wrong", "this animal is labelled as the wrong species", "build a retrain collection from recent mistakes", or when setting up the `track_review_*` MCP tools.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Track review
|
|
7
|
+
|
|
8
|
+
A detector's mistakes are the most valuable training data it will ever get, and
|
|
9
|
+
they hide inside thousands of correct tracks. This procedure finds the likely
|
|
10
|
+
mistakes, has a model (and then the operator) look at them, and writes only the
|
|
11
|
+
confirmed ones into the re-training set with the right boxes.
|
|
12
|
+
|
|
13
|
+
It is the MCP tools `track_review_triage`, `track_review_frame`,
|
|
14
|
+
`track_review_plan`, `track_review_apply` of `@camstack/mcp-develop`. Without the
|
|
15
|
+
MCP, the same hub methods can be called by hand; the order below still holds.
|
|
16
|
+
|
|
17
|
+
## 0. Configure the MCP
|
|
18
|
+
|
|
19
|
+
The server reads three environment variables. Nothing else, no flags.
|
|
20
|
+
|
|
21
|
+
| Variable | Meaning |
|
|
22
|
+
| --- | --- |
|
|
23
|
+
| `CAMSTACK_HUB_URL` | Hub address (`https://hub.local:4443`; a bare host is completed to the default port). |
|
|
24
|
+
| `CAMSTACK_HUB_TOKEN_FILE` | Path to a file holding ONE token, mode 0600 (preferred). `CAMSTACK_HUB_TOKEN` also works but leaks into process listings. |
|
|
25
|
+
| `CAMSTACK_HUB_CA_FILE` | PEM of the hub's CA or pinned certificate, for a self-signed hub. |
|
|
26
|
+
|
|
27
|
+
TLS verification is never turned off: a certificate error is fixed by naming the
|
|
28
|
+
CA, not by disabling the check. Triage and frame need a normal token;
|
|
29
|
+
**apply needs an admin token** and shows the hub's own refusal if the token
|
|
30
|
+
lacks it. Tokens are never printed.
|
|
31
|
+
|
|
32
|
+
```json
|
|
33
|
+
{ "mcpServers": { "camstack-develop": {
|
|
34
|
+
"command": "npx", "args": ["@camstack/mcp-develop"],
|
|
35
|
+
"env": { "CAMSTACK_HUB_URL": "https://hub.local:4443",
|
|
36
|
+
"CAMSTACK_HUB_TOKEN_FILE": "/home/me/.camstack/review.token",
|
|
37
|
+
"CAMSTACK_HUB_CA_FILE": "/home/me/.camstack/hub-ca.pem" } } } }
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## 1. Triage — rank, do not judge
|
|
41
|
+
|
|
42
|
+
`track_review_triage { hours, deviceIds?, classes? }` pages the window (default
|
|
43
|
+
72 h, at most 14 days), scores every video track, and returns the top suspects
|
|
44
|
+
with their features, plus a **random control sample** of tracks that scored low.
|
|
45
|
+
|
|
46
|
+
The score is a sum of cheap signals. None of them says "wrong"; together they say
|
|
47
|
+
"look here first":
|
|
48
|
+
|
|
49
|
+
| Signal | Why it matters | Known failure mode of the signal |
|
|
50
|
+
| --- | --- | --- |
|
|
51
|
+
| `class-label-mismatch` | the sub-label belongs to a different macro class than the track's class | rare; a real reclassification, not noise |
|
|
52
|
+
| `suspect-label` | a label small or unusual subjects are commonly mistaken for (cat, bird, teddy bear, sheep) | fires on every genuine one too; read it with the score |
|
|
53
|
+
| `low-label-score` | weak classification confidence | weak is not wrong; a good label can score low in the dark |
|
|
54
|
+
| `very-short` / `few-observations` | seen for a moment, often a mis-detection or an edge | a fast mover is also short |
|
|
55
|
+
| `tiny-subject` | the trajectory box is a sliver of the frame | a distant subject is legitimately tiny |
|
|
56
|
+
| `static-subject` | a person or vehicle that never moved: the profile of a picture of one | a person sitting still is real |
|
|
57
|
+
| `class-changed` | the track was reclassified during its life | usually the fix working |
|
|
58
|
+
|
|
59
|
+
Confusions worth looking for on purpose, because they repeat:
|
|
60
|
+
|
|
61
|
+
- a small, light-coloured dog read as a cat or a bird, worst when it is far away;
|
|
62
|
+
- vehicles and people on a **television or monitor** (they appear, never leave
|
|
63
|
+
the screen, and the same programme repeats);
|
|
64
|
+
- a person read as another class when partly hidden or lying down, and the
|
|
65
|
+
reverse: a coat, a plant or a shadow read as a person;
|
|
66
|
+
- the same subject split into two tracks with different classes.
|
|
67
|
+
|
|
68
|
+
Raise or lower `minScore`, pass `suspectLabels`, or restrict `classes` to chase
|
|
69
|
+
one confusion. If `fetched.truncated` is true the window was cut by the page
|
|
70
|
+
cap: narrow it.
|
|
71
|
+
|
|
72
|
+
## 2. Look at each suspect
|
|
73
|
+
|
|
74
|
+
`track_review_frame { deviceId, trackId }` returns a clean frame as an image
|
|
75
|
+
(bounded in size) with the features and the trajectory envelope. Judge the
|
|
76
|
+
subject in the picture, not the label. If the frame is missing or too small to
|
|
77
|
+
tell, the verdict is `unsure`.
|
|
78
|
+
|
|
79
|
+
## 3. Verdicts
|
|
80
|
+
|
|
81
|
+
| Verdict | Meaning | Written? |
|
|
82
|
+
| --- | --- | --- |
|
|
83
|
+
| `correct` | the class is right | no |
|
|
84
|
+
| `wrong-class` | the subject is something else; say what (`correctClass`) | yes |
|
|
85
|
+
| `false-positive` | nothing is there; the model drew a box on a picture, a shadow, a reflection | yes, as a `model_error` box |
|
|
86
|
+
| `pipeline-bug` | the class is fine but the track is broken (split, merged, no media) | no — report it |
|
|
87
|
+
| `unsure` | cannot tell from the evidence | no — never train on a guess |
|
|
88
|
+
|
|
89
|
+
**Annotate every subject in the frame.** A frame is evaluated as ground truth: a
|
|
90
|
+
subject left unboxed counts against the detector as a false alarm. So when
|
|
91
|
+
reviewing a wrong-class track, box the other people and animals in the same
|
|
92
|
+
frame too (`subjects`), and set `complete: true` only when every subject is
|
|
93
|
+
boxed. A frame not marked complete is kept but stays out of detector evaluation;
|
|
94
|
+
that is the safe default.
|
|
95
|
+
|
|
96
|
+
## 4. Check the misses: the control sample
|
|
97
|
+
|
|
98
|
+
The control sample is what keeps the signals honest. Review it exactly like the
|
|
99
|
+
suspects. If more than a handful of the control tracks turn out wrong, the
|
|
100
|
+
signals are missing a whole kind of mistake: say so, widen the net (lower
|
|
101
|
+
`minScore`, add `suspectLabels`) and rerun before trusting the suspect list. The
|
|
102
|
+
share of wrong tracks in the control sample is the miss rate — report it
|
|
103
|
+
alongside the findings.
|
|
104
|
+
|
|
105
|
+
## 5. Plan, show, then apply
|
|
106
|
+
|
|
107
|
+
1. `track_review_plan { verdicts }` — local, writes nothing. It returns, per
|
|
108
|
+
track, the steps (flag for training → copy the frame → save the corrected
|
|
109
|
+
boxes → mark complete → add to the collection), the subjects, and the
|
|
110
|
+
verdicts that write nothing with their reason.
|
|
111
|
+
2. **Present that write list to the operator and wait.** Name the tracks, the
|
|
112
|
+
classes you will write, the collection, and how many frames will be marked
|
|
113
|
+
complete. This is the only moment to catch a wrong call cheaply.
|
|
114
|
+
3. `track_review_apply { plan, collectionName, confirm: true }` — only after the
|
|
115
|
+
operator agrees. It refuses without `confirm: true` and an explicit
|
|
116
|
+
collection name.
|
|
117
|
+
|
|
118
|
+
Apply is safe to repeat: a frame already copied is reused, a frame that already
|
|
119
|
+
has annotations is left alone (`overwrite: true` replaces them — a save replaces
|
|
120
|
+
EVERY annotation on the frame), the collection is created once, a track that
|
|
121
|
+
fails does not stop the others, and an authorisation failure stops the run and
|
|
122
|
+
shows the hub's message. It returns a per-track log and the corpus KPI
|
|
123
|
+
(total / annotated / complete frames) before and after; quote both.
|
|
124
|
+
|
|
125
|
+
## Done when
|
|
126
|
+
|
|
127
|
+
- every suspect and every control track has a verdict;
|
|
128
|
+
- the operator saw the write list before apply;
|
|
129
|
+
- the per-track log shows each planned track `done`, or the reason it did not;
|
|
130
|
+
- the KPI moved by what was planned and the control-sample miss rate is reported.
|