@camstack/develop 0.1.5 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1 +1 @@
1
- {"version":3,"file":"skills.d.ts","sourceRoot":"","sources":["../src/skills.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,YAAY,CAAA;AAI3C,wBAAgB,gBAAgB,IAAI,MAAM,CAMzC;AA4BD,wBAAgB,UAAU,IAAI,SAAS,SAAS,EAAE,CAQjD;AAED,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,GAAG;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,GAAG,IAAI,CAMjF"}
1
+ {"version":3,"file":"skills.d.ts","sourceRoot":"","sources":["../src/skills.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,YAAY,CAAA;AAI3C,wBAAgB,gBAAgB,IAAI,MAAM,CAMzC;AAiCD,wBAAgB,UAAU,IAAI,SAAS,SAAS,EAAE,CAQjD;AAED,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,GAAG;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,GAAG,IAAI,CAMjF"}
package/dist/skills.js CHANGED
@@ -24,6 +24,11 @@ const SKILL_FILES = [
24
24
  file: 'architecture.md',
25
25
  title: 'Architecture pointers for plugin authors',
26
26
  },
27
+ {
28
+ id: 'track-review',
29
+ file: 'track-review.md',
30
+ title: 'Review wrong-class tracks and curate the re-training set',
31
+ },
27
32
  ];
28
33
  /**
29
34
  * The ids before the 2026-10-09 rename to the server repository's skill names.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@camstack/develop",
3
- "version": "0.1.5",
3
+ "version": "0.1.6",
4
4
  "description": "CamStack developer tooling — catalogue, scaffold, validate, and example lookup shared by the CLI and MCP server.",
5
5
  "keywords": [
6
6
  "camstack",
package/skills/README.md CHANGED
@@ -11,6 +11,7 @@ third-party use (no private hostnames, no hub-specific ops).
11
11
  | `camstack-secrets` | [camstack-secrets.md](./camstack-secrets.md) | Password / token / key fields |
12
12
  | `camstack-docs` | [camstack-docs.md](./camstack-docs.md) | Generated vs prose docs |
13
13
  | `architecture` | [architecture.md](./architecture.md) | Plugin-author architecture pointers |
14
+ | `track-review` | [track-review.md](./track-review.md) | Find wrong-class tracks, curate the re-training set (`track_review_*` MCP tools) |
14
15
 
15
16
  These files are the canonical public text. In the server repository the
16
17
  matching `.claude/skills/<id>/SKILL.md` is an overlay that points here and keeps
@@ -42,7 +43,9 @@ npx @camstack/mcp-develop
42
43
  ```
43
44
 
44
45
  Tools wrap the same `@camstack/develop` operations (`list_addons`,
45
- `scaffold_addon`, `validate_addon`, `find_example`, `get_skill`, …).
46
+ `scaffold_addon`, `validate_addon`, `find_example`, `get_skill`, …). The
47
+ `track_review_*` tools additionally talk to a hub — see the `track-review` skill
48
+ for the environment variables and the confirm-before-write rule.
46
49
 
47
50
  ### Agents
48
51
 
@@ -0,0 +1,130 @@
1
+ ---
2
+ name: track-review
3
+ description: Review recent tracks for a wrong class and curate the corrected frames into the re-training set — triage suspects, look at each, give a verdict, show the operator the write list, then apply it. Use for "find tracks the detector got wrong", "this animal is labelled as the wrong species", "build a retrain collection from recent mistakes", or when setting up the `track_review_*` MCP tools.
4
+ ---
5
+
6
+ # Track review
7
+
8
+ A detector's mistakes are the most valuable training data it will ever get, and
9
+ they hide inside thousands of correct tracks. This procedure finds the likely
10
+ mistakes, has a model (and then the operator) look at them, and writes only the
11
+ confirmed ones into the re-training set with the right boxes.
12
+
13
+ It is the MCP tools `track_review_triage`, `track_review_frame`,
14
+ `track_review_plan`, `track_review_apply` of `@camstack/mcp-develop`. Without the
15
+ MCP, the same hub methods can be called by hand; the order below still holds.
16
+
17
+ ## 0. Configure the MCP
18
+
19
+ The server reads three environment variables. Nothing else, no flags.
20
+
21
+ | Variable | Meaning |
22
+ | --- | --- |
23
+ | `CAMSTACK_HUB_URL` | Hub address (`https://hub.local:4443`; a bare host is completed to the default port). |
24
+ | `CAMSTACK_HUB_TOKEN_FILE` | Path to a file holding ONE token, mode 0600 (preferred). `CAMSTACK_HUB_TOKEN` also works but leaks into process listings. |
25
+ | `CAMSTACK_HUB_CA_FILE` | PEM of the hub's CA or pinned certificate, for a self-signed hub. |
26
+
27
+ TLS verification is never turned off: a certificate error is fixed by naming the
28
+ CA, not by disabling the check. Triage and frame need a normal token;
29
+ **apply needs an admin token** and shows the hub's own refusal if the token
30
+ lacks it. Tokens are never printed.
31
+
32
+ ```json
33
+ { "mcpServers": { "camstack-develop": {
34
+ "command": "npx", "args": ["@camstack/mcp-develop"],
35
+ "env": { "CAMSTACK_HUB_URL": "https://hub.local:4443",
36
+ "CAMSTACK_HUB_TOKEN_FILE": "/home/me/.camstack/review.token",
37
+ "CAMSTACK_HUB_CA_FILE": "/home/me/.camstack/hub-ca.pem" } } } }
38
+ ```
39
+
40
+ ## 1. Triage — rank, do not judge
41
+
42
+ `track_review_triage { hours, deviceIds?, classes? }` pages the window (default
43
+ 72 h, at most 14 days), scores every video track, and returns the top suspects
44
+ with their features, plus a **random control sample** of tracks that scored low.
45
+
46
+ The score is a sum of cheap signals. None of them says "wrong"; together they say
47
+ "look here first":
48
+
49
+ | Signal | Why it matters | Known failure mode of the signal |
50
+ | --- | --- | --- |
51
+ | `class-label-mismatch` | the sub-label belongs to a different macro class than the track's class | rare; a real reclassification, not noise |
52
+ | `suspect-label` | a label small or unusual subjects are commonly mistaken for (cat, bird, teddy bear, sheep) | fires on every genuine one too; read it with the score |
53
+ | `low-label-score` | weak classification confidence | weak is not wrong; a good label can score low in the dark |
54
+ | `very-short` / `few-observations` | seen for a moment, often a mis-detection or an edge | a fast mover is also short |
55
+ | `tiny-subject` | the trajectory box is a sliver of the frame | a distant subject is legitimately tiny |
56
+ | `static-subject` | a person or vehicle that never moved: the profile of a picture of one | a person sitting still is real |
57
+ | `class-changed` | the track was reclassified during its life | usually the fix working |
58
+
59
+ Confusions worth looking for on purpose, because they repeat:
60
+
61
+ - a small, light-coloured dog read as a cat or a bird, worst when it is far away;
62
+ - vehicles and people on a **television or monitor** (they appear, never leave
63
+ the screen, and the same programme repeats);
64
+ - a person read as another class when partly hidden or lying down, and the
65
+ reverse: a coat, a plant or a shadow read as a person;
66
+ - the same subject split into two tracks with different classes.
67
+
68
+ Raise or lower `minScore`, pass `suspectLabels`, or restrict `classes` to chase
69
+ one confusion. If `fetched.truncated` is true the window was cut by the page
70
+ cap: narrow it.
71
+
72
+ ## 2. Look at each suspect
73
+
74
+ `track_review_frame { deviceId, trackId }` returns a clean frame as an image
75
+ (bounded in size) with the features and the trajectory envelope. Judge the
76
+ subject in the picture, not the label. If the frame is missing or too small to
77
+ tell, the verdict is `unsure`.
78
+
79
+ ## 3. Verdicts
80
+
81
+ | Verdict | Meaning | Written? |
82
+ | --- | --- | --- |
83
+ | `correct` | the class is right | no |
84
+ | `wrong-class` | the subject is something else; say what (`correctClass`) | yes |
85
+ | `false-positive` | nothing is there; the model drew a box on a picture, a shadow, a reflection | yes, as a `model_error` box |
86
+ | `pipeline-bug` | the class is fine but the track is broken (split, merged, no media) | no — report it |
87
+ | `unsure` | cannot tell from the evidence | no — never train on a guess |
88
+
89
+ **Annotate every subject in the frame.** A frame is evaluated as ground truth: a
90
+ subject left unboxed counts against the detector as a false alarm. So when
91
+ reviewing a wrong-class track, box the other people and animals in the same
92
+ frame too (`subjects`), and set `complete: true` only when every subject is
93
+ boxed. A frame not marked complete is kept but stays out of detector evaluation;
94
+ that is the safe default.
95
+
96
+ ## 4. Check the misses: the control sample
97
+
98
+ The control sample is what keeps the signals honest. Review it exactly like the
99
+ suspects. If more than a handful of the control tracks turn out wrong, the
100
+ signals are missing a whole kind of mistake: say so, widen the net (lower
101
+ `minScore`, add `suspectLabels`) and rerun before trusting the suspect list. The
102
+ share of wrong tracks in the control sample is the miss rate — report it
103
+ alongside the findings.
104
+
105
+ ## 5. Plan, show, then apply
106
+
107
+ 1. `track_review_plan { verdicts }` — local, writes nothing. It returns, per
108
+ track, the steps (flag for training → copy the frame → save the corrected
109
+ boxes → mark complete → add to the collection), the subjects, and the
110
+ verdicts that write nothing with their reason.
111
+ 2. **Present that write list to the operator and wait.** Name the tracks, the
112
+ classes you will write, the collection, and how many frames will be marked
113
+ complete. This is the only moment to catch a wrong call cheaply.
114
+ 3. `track_review_apply { plan, collectionName, confirm: true }` — only after the
115
+ operator agrees. It refuses without `confirm: true` and an explicit
116
+ collection name.
117
+
118
+ Apply is safe to repeat: a frame already copied is reused, a frame that already
119
+ has annotations is left alone (`overwrite: true` replaces them — a save replaces
120
+ EVERY annotation on the frame), the collection is created once, a track that
121
+ fails does not stop the others, and an authorisation failure stops the run and
122
+ shows the hub's message. It returns a per-track log and the corpus KPI
123
+ (total / annotated / complete frames) before and after; quote both.
124
+
125
+ ## Done when
126
+
127
+ - every suspect and every control track has a verdict;
128
+ - the operator saw the write list before apply;
129
+ - the per-track log shows each planned track `done`, or the reason it did not;
130
+ - the KPI moved by what was planned and the control-sample miss rate is reported.