incitaciones 0.10.5 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/content/distilled/review/references/rule-of-5-universal/SKILL.md +1 -1
- package/content/distilled/review/references/rule-of-5-universal/references/typesafe-verification.md +86 -0
- package/content/manifest.json +1 -1
- package/package.json +2 -2
- package/pi-package/skills/review/references/rule-of-5-universal/SKILL.md +1 -1
- package/pi-package/skills/review/references/rule-of-5-universal/references/typesafe-verification.md +86 -0
- package/pi-package/skills/rule-of-5-universal/SKILL.md +92 -1
package/README.md
CHANGED
|
@@ -504,7 +504,7 @@ The prompts are tool-agnostic where possible, with tool-specific variations note
|
|
|
504
504
|
|
|
505
505
|
## License
|
|
506
506
|
|
|
507
|
-
[
|
|
507
|
+
The prompts and skills in this repository are licensed under [CC-BY-4.0](LICENSE) (Attribution 4.0 International). You are free to use, share, and adapt them with attribution.
|
|
508
508
|
|
|
509
509
|
## Related Projects
|
|
510
510
|
|
|
@@ -17,7 +17,7 @@ Review any intellectual artifact (code, plans, research, issues, specs, document
|
|
|
17
17
|
- **Progressive Build:** Each stage must build on previous findings.
|
|
18
18
|
- **Specificity:** Reference exact locations (file:line, section, paragraph).
|
|
19
19
|
- **Actionability:** Suggest specific solutions, don't just identify problems.
|
|
20
|
-
- **Validation:** Confirm issues exist; do not flag "potential" issues without evidence.
|
|
20
|
+
- **Validation:** Confirm issues exist; do not flag "potential" issues without evidence. If the `TYPESAFE_API_KEY` environment variable is set, apply `references/typesafe-verification.md` in place of self-reported validation. If it is not set (or the API fails), mark findings `UNVERIFIED`, print the banner defined there, and proceed with self-reported validation.
|
|
21
21
|
- **Early Stop:** Stop if convergence criteria are met before Stage 5.
|
|
22
22
|
|
|
23
23
|
## References
|
package/content/distilled/review/references/rule-of-5-universal/references/typesafe-verification.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# TypeSafe Verification Pass (conditional)
|
|
2
|
+
|
|
3
|
+
Use this procedure **only** when the `TYPESAFE_API_KEY` environment variable is set.
|
|
4
|
+
When it is not set, or the API call fails (see Failure Handling), mark every finding
|
|
5
|
+
`UNVERIFIED`, print the banner below, and fall back to self-reported validation —
|
|
6
|
+
do not block the review.
|
|
7
|
+
|
|
8
|
+
```
|
|
9
|
+
BANNER: Findings NOT verified by TypeSafe (API unavailable or key absent).
|
|
10
|
+
Self-reported validation in effect; treat false-positive estimates as unmeasured.
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Use this procedure as-is; consult the `typesafe-ai` skill or docs.typesafe.ai only
|
|
14
|
+
for SDK-language bindings beyond the HTTP call specified here. Do not re-derive
|
|
15
|
+
questions, thresholds, or request shape at review time.
|
|
16
|
+
|
|
17
|
+
## Constants (edit here, not inline)
|
|
18
|
+
|
|
19
|
+
| Constant | Value | Notes |
|
|
20
|
+
| --- | --- | --- |
|
|
21
|
+
| Endpoint | `https://api.typesafe.ai/v1/systemone` | `POST`, Bearer auth |
|
|
22
|
+
| Model | `jev-latest` | TypeSafe flagship alias |
|
|
23
|
+
| `CONFIDENCE_GATE` | `0.8` | Tunable starting point; calibrate per workflow. Below gate → human review |
|
|
24
|
+
| `VERIFY_SEVERITIES` | `CRITICAL, HIGH` | Cost cap: MEDIUM/LOW findings stay UNVERIFIED |
|
|
25
|
+
| Batching | one request | All verification questions in a single parallel call |
|
|
26
|
+
|
|
27
|
+
## Stage A — Mechanical pre-gate (no API)
|
|
28
|
+
|
|
29
|
+
For each finding at or above `VERIFY_SEVERITIES`, check mechanically:
|
|
30
|
+
|
|
31
|
+
1. Does the referenced location exist (file:line, section, paragraph)?
|
|
32
|
+
2. Does any quoted evidence appear verbatim in the artifact under review?
|
|
33
|
+
|
|
34
|
+
- Both pass → send to Stage B.
|
|
35
|
+
- Location missing or quote absent → verdict `fabricated` **without an API call**
|
|
36
|
+
(this is a CRITICAL meta-finding about the reviewer, not the artifact).
|
|
37
|
+
- No quoted evidence to check → send to Stage B anyway; the model judges context.
|
|
38
|
+
|
|
39
|
+
## Stage B — One parallel verification request
|
|
40
|
+
|
|
41
|
+
`state` = the artifact text plus the surviving findings (id, location, claim,
|
|
42
|
+
evidence). Ask **one Choice question per finding**, all in the same request:
|
|
43
|
+
|
|
44
|
+
```json
|
|
45
|
+
{
|
|
46
|
+
"model": "jev-latest",
|
|
47
|
+
"state": { "artifact": "...", "findings": [ { "id": "CORR-001", "location": "...", "claim": "...", "evidence": "..." } ] },
|
|
48
|
+
"questions": {
|
|
49
|
+
"verify_CORR-001": {
|
|
50
|
+
"type": "choice",
|
|
51
|
+
"instructions": {
|
|
52
|
+
"finding": "`findings[0]`",
|
|
53
|
+
"question": "Does the `artifact` at the location named in `finding` contain evidence that supports `finding.claim`?"
|
|
54
|
+
},
|
|
55
|
+
"criteria": {
|
|
56
|
+
"verified": "The cited location contains evidence supporting the claim",
|
|
57
|
+
"unsupported": "The location exists but says nothing about the claim",
|
|
58
|
+
"contradicted": "The location's context says the opposite of the claim",
|
|
59
|
+
"fabricated": "The claim misrepresents what the location says"
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Generate one `verify_<ID>` question per finding. Never ask about multiple findings
|
|
67
|
+
in one question.
|
|
68
|
+
|
|
69
|
+
## Stage C — Verdict mapping
|
|
70
|
+
|
|
71
|
+
| Verdict | Action |
|
|
72
|
+
| --- | --- |
|
|
73
|
+
| `verified`, confidence ≥ gate | Finding stands; counts toward convergence |
|
|
74
|
+
| `unsupported` or `contradicted` | Drop the finding; it counts as a **measured** false positive |
|
|
75
|
+
| `fabricated` (Stage A or B) | Drop; file CRITICAL meta-finding `[META-<n>]` against the review pass |
|
|
76
|
+
| Any verdict with confidence < `CONFIDENCE_GATE` | Keep finding, mark `REVIEW_REQUIRED`, route to `ESCALATE_TO_HUMAN` |
|
|
77
|
+
| Findings below `VERIFY_SEVERITIES` | Mark `UNVERIFIED`; excluded from FP-rate arithmetic |
|
|
78
|
+
|
|
79
|
+
Convergence arithmetic (in `criteria.md`) uses these measured numbers:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
False positive rate = dropped findings / (verified + dropped findings)
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
This replaces the self-estimated false-positive rate. Self-estimated rates are
|
|
86
|
+
insufficient as a sole gate but remain one input alongside other convergence criteria.
|
package/content/manifest.json
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "incitaciones",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Reusable prompts and skills for CLI LLM tools, with native pi package support.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"bugs": {
|
|
11
11
|
"url": "https://github.com/charly-vibes/incitaciones/issues"
|
|
12
12
|
},
|
|
13
|
-
"license": "
|
|
13
|
+
"license": "CC-BY-4.0",
|
|
14
14
|
"type": "module",
|
|
15
15
|
"keywords": [
|
|
16
16
|
"pi-package",
|
|
@@ -17,7 +17,7 @@ Review any intellectual artifact (code, plans, research, issues, specs, document
|
|
|
17
17
|
- **Progressive Build:** Each stage must build on previous findings.
|
|
18
18
|
- **Specificity:** Reference exact locations (file:line, section, paragraph).
|
|
19
19
|
- **Actionability:** Suggest specific solutions, don't just identify problems.
|
|
20
|
-
- **Validation:** Confirm issues exist; do not flag "potential" issues without evidence.
|
|
20
|
+
- **Validation:** Confirm issues exist; do not flag "potential" issues without evidence. If the `TYPESAFE_API_KEY` environment variable is set, apply `references/typesafe-verification.md` in place of self-reported validation. If it is not set (or the API fails), mark findings `UNVERIFIED`, print the banner defined there, and proceed with self-reported validation.
|
|
21
21
|
- **Early Stop:** Stop if convergence criteria are met before Stage 5.
|
|
22
22
|
|
|
23
23
|
## References
|
package/pi-package/skills/review/references/rule-of-5-universal/references/typesafe-verification.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# TypeSafe Verification Pass (conditional)
|
|
2
|
+
|
|
3
|
+
Use this procedure **only** when the `TYPESAFE_API_KEY` environment variable is set.
|
|
4
|
+
When it is not set, or the API call fails (see Failure Handling), mark every finding
|
|
5
|
+
`UNVERIFIED`, print the banner below, and fall back to self-reported validation —
|
|
6
|
+
do not block the review.
|
|
7
|
+
|
|
8
|
+
```
|
|
9
|
+
BANNER: Findings NOT verified by TypeSafe (API unavailable or key absent).
|
|
10
|
+
Self-reported validation in effect; treat false-positive estimates as unmeasured.
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Use this procedure as-is; consult the `typesafe-ai` skill or docs.typesafe.ai only
|
|
14
|
+
for SDK-language bindings beyond the HTTP call specified here. Do not re-derive
|
|
15
|
+
questions, thresholds, or request shape at review time.
|
|
16
|
+
|
|
17
|
+
## Constants (edit here, not inline)
|
|
18
|
+
|
|
19
|
+
| Constant | Value | Notes |
|
|
20
|
+
| --- | --- | --- |
|
|
21
|
+
| Endpoint | `https://api.typesafe.ai/v1/systemone` | `POST`, Bearer auth |
|
|
22
|
+
| Model | `jev-latest` | TypeSafe flagship alias |
|
|
23
|
+
| `CONFIDENCE_GATE` | `0.8` | Tunable starting point; calibrate per workflow. Below gate → human review |
|
|
24
|
+
| `VERIFY_SEVERITIES` | `CRITICAL, HIGH` | Cost cap: MEDIUM/LOW findings stay UNVERIFIED |
|
|
25
|
+
| Batching | one request | All verification questions in a single parallel call |
|
|
26
|
+
|
|
27
|
+
## Stage A — Mechanical pre-gate (no API)
|
|
28
|
+
|
|
29
|
+
For each finding at or above `VERIFY_SEVERITIES`, check mechanically:
|
|
30
|
+
|
|
31
|
+
1. Does the referenced location exist (file:line, section, paragraph)?
|
|
32
|
+
2. Does any quoted evidence appear verbatim in the artifact under review?
|
|
33
|
+
|
|
34
|
+
- Both pass → send to Stage B.
|
|
35
|
+
- Location missing or quote absent → verdict `fabricated` **without an API call**
|
|
36
|
+
(this is a CRITICAL meta-finding about the reviewer, not the artifact).
|
|
37
|
+
- No quoted evidence to check → send to Stage B anyway; the model judges context.
|
|
38
|
+
|
|
39
|
+
## Stage B — One parallel verification request
|
|
40
|
+
|
|
41
|
+
`state` = the artifact text plus the surviving findings (id, location, claim,
|
|
42
|
+
evidence). Ask **one Choice question per finding**, all in the same request:
|
|
43
|
+
|
|
44
|
+
```json
|
|
45
|
+
{
|
|
46
|
+
"model": "jev-latest",
|
|
47
|
+
"state": { "artifact": "...", "findings": [ { "id": "CORR-001", "location": "...", "claim": "...", "evidence": "..." } ] },
|
|
48
|
+
"questions": {
|
|
49
|
+
"verify_CORR-001": {
|
|
50
|
+
"type": "choice",
|
|
51
|
+
"instructions": {
|
|
52
|
+
"finding": "`findings[0]`",
|
|
53
|
+
"question": "Does the `artifact` at the location named in `finding` contain evidence that supports `finding.claim`?"
|
|
54
|
+
},
|
|
55
|
+
"criteria": {
|
|
56
|
+
"verified": "The cited location contains evidence supporting the claim",
|
|
57
|
+
"unsupported": "The location exists but says nothing about the claim",
|
|
58
|
+
"contradicted": "The location's context says the opposite of the claim",
|
|
59
|
+
"fabricated": "The claim misrepresents what the location says"
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Generate one `verify_<ID>` question per finding. Never ask about multiple findings
|
|
67
|
+
in one question.
|
|
68
|
+
|
|
69
|
+
## Stage C — Verdict mapping
|
|
70
|
+
|
|
71
|
+
| Verdict | Action |
|
|
72
|
+
| --- | --- |
|
|
73
|
+
| `verified`, confidence ≥ gate | Finding stands; counts toward convergence |
|
|
74
|
+
| `unsupported` or `contradicted` | Drop the finding; it counts as a **measured** false positive |
|
|
75
|
+
| `fabricated` (Stage A or B) | Drop; file CRITICAL meta-finding `[META-<n>]` against the review pass |
|
|
76
|
+
| Any verdict with confidence < `CONFIDENCE_GATE` | Keep finding, mark `REVIEW_REQUIRED`, route to `ESCALATE_TO_HUMAN` |
|
|
77
|
+
| Findings below `VERIFY_SEVERITIES` | Mark `UNVERIFIED`; excluded from FP-rate arithmetic |
|
|
78
|
+
|
|
79
|
+
Convergence arithmetic (in `criteria.md`) uses these measured numbers:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
False positive rate = dropped findings / (verified + dropped findings)
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
This replaces the self-estimated false-positive rate. Self-estimated rates are
|
|
86
|
+
insufficient as a sole gate but remain one input alongside other convergence criteria.
|
|
@@ -31,7 +31,7 @@ Review any intellectual artifact (code, plans, research, issues, specs, document
|
|
|
31
31
|
- **Progressive Build:** Each stage must build on previous findings.
|
|
32
32
|
- **Specificity:** Reference exact locations (file:line, section, paragraph).
|
|
33
33
|
- **Actionability:** Suggest specific solutions, don't just identify problems.
|
|
34
|
-
- **Validation:** Confirm issues exist; do not flag "potential" issues without evidence.
|
|
34
|
+
- **Validation:** Confirm issues exist; do not flag "potential" issues without evidence. If the `TYPESAFE_API_KEY` environment variable is set, apply `references/typesafe-verification.md` in place of self-reported validation. If it is not set (or the API fails), mark findings `UNVERIFIED`, print the banner defined there, and proceed with self-reported validation.
|
|
35
35
|
- **Early Stop:** Stop if convergence criteria are met before Stage 5.
|
|
36
36
|
|
|
37
37
|
## References
|
|
@@ -292,3 +292,94 @@ Total Issues by Severity:
|
|
|
292
292
|
|
|
293
293
|
**Rationale:** [1-2 sentences explaining the verdict]
|
|
294
294
|
```
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
#### Reference: references/typesafe-verification.md
|
|
299
|
+
|
|
300
|
+
# TypeSafe Verification Pass (conditional)
|
|
301
|
+
|
|
302
|
+
Use this procedure **only** when the `TYPESAFE_API_KEY` environment variable is set.
|
|
303
|
+
When it is not set, or the API call fails (see Failure Handling), mark every finding
|
|
304
|
+
`UNVERIFIED`, print the banner below, and fall back to self-reported validation —
|
|
305
|
+
do not block the review.
|
|
306
|
+
|
|
307
|
+
```
|
|
308
|
+
BANNER: Findings NOT verified by TypeSafe (API unavailable or key absent).
|
|
309
|
+
Self-reported validation in effect; treat false-positive estimates as unmeasured.
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
Use this procedure as-is; consult the `typesafe-ai` skill or docs.typesafe.ai only
|
|
313
|
+
for SDK-language bindings beyond the HTTP call specified here. Do not re-derive
|
|
314
|
+
questions, thresholds, or request shape at review time.
|
|
315
|
+
|
|
316
|
+
## Constants (edit here, not inline)
|
|
317
|
+
|
|
318
|
+
| Constant | Value | Notes |
|
|
319
|
+
| --- | --- | --- |
|
|
320
|
+
| Endpoint | `https://api.typesafe.ai/v1/systemone` | `POST`, Bearer auth |
|
|
321
|
+
| Model | `jev-latest` | TypeSafe flagship alias |
|
|
322
|
+
| `CONFIDENCE_GATE` | `0.8` | Tunable starting point; calibrate per workflow. Below gate → human review |
|
|
323
|
+
| `VERIFY_SEVERITIES` | `CRITICAL, HIGH` | Cost cap: MEDIUM/LOW findings stay UNVERIFIED |
|
|
324
|
+
| Batching | one request | All verification questions in a single parallel call |
|
|
325
|
+
|
|
326
|
+
## Stage A — Mechanical pre-gate (no API)
|
|
327
|
+
|
|
328
|
+
For each finding at or above `VERIFY_SEVERITIES`, check mechanically:
|
|
329
|
+
|
|
330
|
+
1. Does the referenced location exist (file:line, section, paragraph)?
|
|
331
|
+
2. Does any quoted evidence appear verbatim in the artifact under review?
|
|
332
|
+
|
|
333
|
+
- Both pass → send to Stage B.
|
|
334
|
+
- Location missing or quote absent → verdict `fabricated` **without an API call**
|
|
335
|
+
(this is a CRITICAL meta-finding about the reviewer, not the artifact).
|
|
336
|
+
- No quoted evidence to check → send to Stage B anyway; the model judges context.
|
|
337
|
+
|
|
338
|
+
## Stage B — One parallel verification request
|
|
339
|
+
|
|
340
|
+
`state` = the artifact text plus the surviving findings (id, location, claim,
|
|
341
|
+
evidence). Ask **one Choice question per finding**, all in the same request:
|
|
342
|
+
|
|
343
|
+
```json
|
|
344
|
+
{
|
|
345
|
+
"model": "jev-latest",
|
|
346
|
+
"state": { "artifact": "...", "findings": [ { "id": "CORR-001", "location": "...", "claim": "...", "evidence": "..." } ] },
|
|
347
|
+
"questions": {
|
|
348
|
+
"verify_CORR-001": {
|
|
349
|
+
"type": "choice",
|
|
350
|
+
"instructions": {
|
|
351
|
+
"finding": "`findings[0]`",
|
|
352
|
+
"question": "Does the `artifact` at the location named in `finding` contain evidence that supports `finding.claim`?"
|
|
353
|
+
},
|
|
354
|
+
"criteria": {
|
|
355
|
+
"verified": "The cited location contains evidence supporting the claim",
|
|
356
|
+
"unsupported": "The location exists but says nothing about the claim",
|
|
357
|
+
"contradicted": "The location's context says the opposite of the claim",
|
|
358
|
+
"fabricated": "The claim misrepresents what the location says"
|
|
359
|
+
}
|
|
360
|
+
}
|
|
361
|
+
}
|
|
362
|
+
}
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
Generate one `verify_<ID>` question per finding. Never ask about multiple findings
|
|
366
|
+
in one question.
|
|
367
|
+
|
|
368
|
+
## Stage C — Verdict mapping
|
|
369
|
+
|
|
370
|
+
| Verdict | Action |
|
|
371
|
+
| --- | --- |
|
|
372
|
+
| `verified`, confidence ≥ gate | Finding stands; counts toward convergence |
|
|
373
|
+
| `unsupported` or `contradicted` | Drop the finding; it counts as a **measured** false positive |
|
|
374
|
+
| `fabricated` (Stage A or B) | Drop; file CRITICAL meta-finding `[META-<n>]` against the review pass |
|
|
375
|
+
| Any verdict with confidence < `CONFIDENCE_GATE` | Keep finding, mark `REVIEW_REQUIRED`, route to `ESCALATE_TO_HUMAN` |
|
|
376
|
+
| Findings below `VERIFY_SEVERITIES` | Mark `UNVERIFIED`; excluded from FP-rate arithmetic |
|
|
377
|
+
|
|
378
|
+
Convergence arithmetic (in `criteria.md`) uses these measured numbers:
|
|
379
|
+
|
|
380
|
+
```
|
|
381
|
+
False positive rate = dropped findings / (verified + dropped findings)
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
This replaces the self-estimated false-positive rate. Self-estimated rates are
|
|
385
|
+
insufficient as a sole gate but remain one input alongside other convergence criteria.
|