sherpa-skill 2.0.6 → 2.0.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +11 -0
- package/.agents/skills/sherpa/SKILL.md +161 -0
- package/.agents/skills/sherpa/commands/README.md +43 -0
- package/.agents/skills/sherpa/commands/manifest.json +589 -0
- package/.agents/skills/sherpa/commands/sherpa-research.md +147 -0
- package/.agents/skills/sherpa/core/commands/router.py +171 -0
- package/.agents/skills/sherpa/core/configuration/manager.py +504 -0
- package/.agents/skills/sherpa/core/ideas/engine.py +564 -0
- package/.agents/skills/sherpa/docs/commands/README.md +96 -0
- package/.agents/skills/sherpa/docs/configuration/runtime-verification.md +64 -0
- package/.agents/skills/sherpa/rules/demo-preparation-rules.md +543 -0
- package/.agents/skills/sherpa/rules/engineering-rules.md +221 -0
- package/.agents/skills/sherpa/rules/final-audit-rules.md +824 -0
- package/.agents/skills/sherpa/rules/implementation-rules.md +127 -0
- package/.agents/skills/sherpa/rules/judge-rules.md +443 -0
- package/.agents/skills/sherpa/rules/mentor-rules.md +109 -0
- package/.agents/skills/sherpa/rules/postmortem-rules.md +69 -0
- package/.agents/skills/sherpa/rules/presentation-rules.md +783 -0
- package/.agents/skills/sherpa/rules/red-team-rules.md +185 -0
- package/.agents/skills/sherpa/rules/research-rules.md +184 -0
- package/.agents/skills/sherpa/rules/score-optimization-rules.md +355 -0
- package/.agents/skills/sherpa/rules/scoring-rules.md +216 -0
- package/.agents/skills/sherpa/rules/submission-rules.md +41 -0
- package/.agents/skills/sherpa/rules/testing-security-rules.md +87 -0
- package/.agents/skills/sherpa/rules/ux-rules.md +89 -0
- package/.agents/skills/sherpa/src/config_manager.py +474 -0
- package/.agents/skills/sherpa/src/verification.py +458 -0
- package/.agents/skills/sherpa/templates/audit-finding.md +20 -0
- package/.agents/skills/sherpa/templates/claim-audit.md +15 -0
- package/.agents/skills/sherpa/templates/configuration/hackathon.json +21 -0
- package/.agents/skills/sherpa/templates/configuration/integrations.env.example +3 -0
- package/.agents/skills/sherpa/templates/configuration/project.env.example +12 -0
- package/.agents/skills/sherpa/templates/configuration/providers.env.example +4 -0
- package/.agents/skills/sherpa/templates/configuration/settings.json +22 -0
- package/.agents/skills/sherpa/templates/consistency-audit.md +17 -0
- package/.agents/skills/sherpa/templates/decision-log.md +18 -0
- package/.agents/skills/sherpa/templates/demo-checkpoint.md +42 -0
- package/.agents/skills/sherpa/templates/demo-evidence.md +32 -0
- package/.agents/skills/sherpa/templates/demo-finding.md +112 -0
- package/.agents/skills/sherpa/templates/demo-flow.md +52 -0
- package/.agents/skills/sherpa/templates/demo-plan.md +265 -0
- package/.agents/skills/sherpa/templates/demo-verification.md +57 -0
- package/.agents/skills/sherpa/templates/final-audit.md +387 -0
- package/.agents/skills/sherpa/templates/final-risk.md +17 -0
- package/.agents/skills/sherpa/templates/final-verification.md +57 -0
- package/.agents/skills/sherpa/templates/gap-analysis.md +113 -0
- package/.agents/skills/sherpa/templates/idea-comparison.md +129 -0
- package/.agents/skills/sherpa/templates/idea-evaluation.md +251 -0
- package/.agents/skills/sherpa/templates/implementation-plan.md +96 -0
- package/.agents/skills/sherpa/templates/implementation-task.md +57 -0
- package/.agents/skills/sherpa/templates/improvement-candidate.md +135 -0
- package/.agents/skills/sherpa/templates/improvement.md +18 -0
- package/.agents/skills/sherpa/templates/incident.md +20 -0
- package/.agents/skills/sherpa/templates/judge-criterion.md +41 -0
- package/.agents/skills/sherpa/templates/judge-question.md +41 -0
- package/.agents/skills/sherpa/templates/judge-session.md +125 -0
- package/.agents/skills/sherpa/templates/judge-simulation.md +264 -0
- package/.agents/skills/sherpa/templates/lesson.md +14 -0
- package/.agents/skills/sherpa/templates/postmortem.md +87 -0
- package/.agents/skills/sherpa/templates/presentation-evidence.md +33 -0
- package/.agents/skills/sherpa/templates/presentation-plan.md +349 -0
- package/.agents/skills/sherpa/templates/presentation-qna.md +29 -0
- package/.agents/skills/sherpa/templates/presentation-risk.md +42 -0
- package/.agents/skills/sherpa/templates/presentation-verification.md +57 -0
- package/.agents/skills/sherpa/templates/project-plan.md +242 -0
- package/.agents/skills/sherpa/templates/project-state.md +84 -0
- package/.agents/skills/sherpa/templates/red-team-finding.md +73 -0
- package/.agents/skills/sherpa/templates/red-team-review.md +105 -0
- package/.agents/skills/sherpa/templates/red-team-scenario.md +43 -0
- package/.agents/skills/sherpa/templates/red-team-verification.md +31 -0
- package/.agents/skills/sherpa/templates/repo-analysis.md +204 -0
- package/.agents/skills/sherpa/templates/requirement-audit.md +15 -0
- package/.agents/skills/sherpa/templates/research-comparison.md +124 -0
- package/.agents/skills/sherpa/templates/research-plan.md +93 -0
- package/.agents/skills/sherpa/templates/research-report.md +92 -0
- package/.agents/skills/sherpa/templates/research-source.md +77 -0
- package/.agents/skills/sherpa/templates/score-optimization-plan.md +197 -0
- package/.agents/skills/sherpa/templates/security-finding.md +45 -0
- package/.agents/skills/sherpa/templates/security-review.md +64 -0
- package/.agents/skills/sherpa/templates/slide-claim.md +47 -0
- package/.agents/skills/sherpa/templates/slide-plan.md +127 -0
- package/.agents/skills/sherpa/templates/submission-artifact.md +15 -0
- package/.agents/skills/sherpa/templates/submission-checklist.md +24 -0
- package/.agents/skills/sherpa/templates/submission-field.md +14 -0
- package/.agents/skills/sherpa/templates/submission-link.md +14 -0
- package/.agents/skills/sherpa/templates/submission-plan.md +31 -0
- package/.agents/skills/sherpa/templates/submission-readiness.md +18 -0
- package/.agents/skills/sherpa/templates/submission-requirement.md +12 -0
- package/.agents/skills/sherpa/templates/submission-risk.md +14 -0
- package/.agents/skills/sherpa/templates/submission-snapshot.md +20 -0
- package/.agents/skills/sherpa/templates/submission-verification.md +12 -0
- package/.agents/skills/sherpa/templates/test-plan.md +59 -0
- package/.agents/skills/sherpa/templates/test-result.md +46 -0
- package/.agents/skills/sherpa/templates/timeline.md +11 -0
- package/.agents/skills/sherpa/templates/ux-finding.md +56 -0
- package/.agents/skills/sherpa/templates/ux-flow.md +46 -0
- package/.agents/skills/sherpa/templates/ux-review.md +113 -0
- package/.agents/skills/sherpa/templates/ux-verification.md +32 -0
- package/.agents/skills/sherpa/workflows/configuration.md +23 -0
- package/.agents/skills/sherpa/workflows/demo-preparation.md +399 -0
- package/.agents/skills/sherpa/workflows/final-audit.md +713 -0
- package/.agents/skills/sherpa/workflows/idea-evaluation.md +239 -0
- package/.agents/skills/sherpa/workflows/idea-intelligence.md +69 -0
- package/.agents/skills/sherpa/workflows/implementation.md +42 -0
- package/.agents/skills/sherpa/workflows/init.md +102 -0
- package/.agents/skills/sherpa/workflows/judge-simulation.md +408 -0
- package/.agents/skills/sherpa/workflows/lifecycle.md +51 -0
- package/.agents/skills/sherpa/workflows/next.md +96 -0
- package/.agents/skills/sherpa/workflows/postmortem.md +76 -0
- package/.agents/skills/sherpa/workflows/presentation.md +522 -0
- package/.agents/skills/sherpa/workflows/project-planning.md +327 -0
- package/.agents/skills/sherpa/workflows/red-team.md +373 -0
- package/.agents/skills/sherpa/workflows/research.md +142 -0
- package/.agents/skills/sherpa/workflows/score-optimization.md +276 -0
- package/.agents/skills/sherpa/workflows/status.md +107 -0
- package/.agents/skills/sherpa/workflows/submission.md +46 -0
- package/.agents/skills/sherpa/workflows/testing-security.md +44 -0
- package/.agents/skills/sherpa/workflows/ux-review.md +59 -0
- package/.claude-plugin/marketplace.json +21 -0
- package/.claude-plugin/plugin.json +13 -0
- package/.codex-plugin/plugin.json +13 -0
- package/GEMINI.md +3 -0
- package/README.md +11 -7
- package/SKILL.md +161 -0
- package/distribution/cli/sherpa +1 -1
- package/gemini-extension.json +7 -0
- package/package.json +31 -3
- package/plugin.json +20 -0
- package/qwen-extension.json +6 -0
- package/skills/sherpa/SKILL.md +161 -0
- package/skills/sherpa/commands/README.md +43 -0
- package/skills/sherpa/commands/manifest.json +589 -0
- package/skills/sherpa/commands/sherpa-research.md +147 -0
- package/skills/sherpa/core/commands/router.py +171 -0
- package/skills/sherpa/core/configuration/manager.py +504 -0
- package/skills/sherpa/core/ideas/engine.py +564 -0
- package/skills/sherpa/docs/commands/README.md +96 -0
- package/skills/sherpa/docs/configuration/runtime-verification.md +64 -0
- package/skills/sherpa/rules/demo-preparation-rules.md +543 -0
- package/skills/sherpa/rules/engineering-rules.md +221 -0
- package/skills/sherpa/rules/final-audit-rules.md +824 -0
- package/skills/sherpa/rules/implementation-rules.md +127 -0
- package/skills/sherpa/rules/judge-rules.md +443 -0
- package/skills/sherpa/rules/mentor-rules.md +109 -0
- package/skills/sherpa/rules/postmortem-rules.md +69 -0
- package/skills/sherpa/rules/presentation-rules.md +783 -0
- package/skills/sherpa/rules/red-team-rules.md +185 -0
- package/skills/sherpa/rules/research-rules.md +184 -0
- package/skills/sherpa/rules/score-optimization-rules.md +355 -0
- package/skills/sherpa/rules/scoring-rules.md +216 -0
- package/skills/sherpa/rules/submission-rules.md +41 -0
- package/skills/sherpa/rules/testing-security-rules.md +87 -0
- package/skills/sherpa/rules/ux-rules.md +89 -0
- package/skills/sherpa/src/config_manager.py +474 -0
- package/skills/sherpa/src/verification.py +458 -0
- package/skills/sherpa/templates/audit-finding.md +20 -0
- package/skills/sherpa/templates/claim-audit.md +15 -0
- package/skills/sherpa/templates/configuration/hackathon.json +21 -0
- package/skills/sherpa/templates/configuration/integrations.env.example +3 -0
- package/skills/sherpa/templates/configuration/project.env.example +12 -0
- package/skills/sherpa/templates/configuration/providers.env.example +4 -0
- package/skills/sherpa/templates/configuration/settings.json +22 -0
- package/skills/sherpa/templates/consistency-audit.md +17 -0
- package/skills/sherpa/templates/decision-log.md +18 -0
- package/skills/sherpa/templates/demo-checkpoint.md +42 -0
- package/skills/sherpa/templates/demo-evidence.md +32 -0
- package/skills/sherpa/templates/demo-finding.md +112 -0
- package/skills/sherpa/templates/demo-flow.md +52 -0
- package/skills/sherpa/templates/demo-plan.md +265 -0
- package/skills/sherpa/templates/demo-verification.md +57 -0
- package/skills/sherpa/templates/final-audit.md +387 -0
- package/skills/sherpa/templates/final-risk.md +17 -0
- package/skills/sherpa/templates/final-verification.md +57 -0
- package/skills/sherpa/templates/gap-analysis.md +113 -0
- package/skills/sherpa/templates/idea-comparison.md +129 -0
- package/skills/sherpa/templates/idea-evaluation.md +251 -0
- package/skills/sherpa/templates/implementation-plan.md +96 -0
- package/skills/sherpa/templates/implementation-task.md +57 -0
- package/skills/sherpa/templates/improvement-candidate.md +135 -0
- package/skills/sherpa/templates/improvement.md +18 -0
- package/skills/sherpa/templates/incident.md +20 -0
- package/skills/sherpa/templates/judge-criterion.md +41 -0
- package/skills/sherpa/templates/judge-question.md +41 -0
- package/skills/sherpa/templates/judge-session.md +125 -0
- package/skills/sherpa/templates/judge-simulation.md +264 -0
- package/skills/sherpa/templates/lesson.md +14 -0
- package/skills/sherpa/templates/postmortem.md +87 -0
- package/skills/sherpa/templates/presentation-evidence.md +33 -0
- package/skills/sherpa/templates/presentation-plan.md +349 -0
- package/skills/sherpa/templates/presentation-qna.md +29 -0
- package/skills/sherpa/templates/presentation-risk.md +42 -0
- package/skills/sherpa/templates/presentation-verification.md +57 -0
- package/skills/sherpa/templates/project-plan.md +242 -0
- package/skills/sherpa/templates/project-state.md +84 -0
- package/skills/sherpa/templates/red-team-finding.md +73 -0
- package/skills/sherpa/templates/red-team-review.md +105 -0
- package/skills/sherpa/templates/red-team-scenario.md +43 -0
- package/skills/sherpa/templates/red-team-verification.md +31 -0
- package/skills/sherpa/templates/repo-analysis.md +204 -0
- package/skills/sherpa/templates/requirement-audit.md +15 -0
- package/skills/sherpa/templates/research-comparison.md +124 -0
- package/skills/sherpa/templates/research-plan.md +93 -0
- package/skills/sherpa/templates/research-report.md +92 -0
- package/skills/sherpa/templates/research-source.md +77 -0
- package/skills/sherpa/templates/score-optimization-plan.md +197 -0
- package/skills/sherpa/templates/security-finding.md +45 -0
- package/skills/sherpa/templates/security-review.md +64 -0
- package/skills/sherpa/templates/slide-claim.md +47 -0
- package/skills/sherpa/templates/slide-plan.md +127 -0
- package/skills/sherpa/templates/submission-artifact.md +15 -0
- package/skills/sherpa/templates/submission-checklist.md +24 -0
- package/skills/sherpa/templates/submission-field.md +14 -0
- package/skills/sherpa/templates/submission-link.md +14 -0
- package/skills/sherpa/templates/submission-plan.md +31 -0
- package/skills/sherpa/templates/submission-readiness.md +18 -0
- package/skills/sherpa/templates/submission-requirement.md +12 -0
- package/skills/sherpa/templates/submission-risk.md +14 -0
- package/skills/sherpa/templates/submission-snapshot.md +20 -0
- package/skills/sherpa/templates/submission-verification.md +12 -0
- package/skills/sherpa/templates/test-plan.md +59 -0
- package/skills/sherpa/templates/test-result.md +46 -0
- package/skills/sherpa/templates/timeline.md +11 -0
- package/skills/sherpa/templates/ux-finding.md +56 -0
- package/skills/sherpa/templates/ux-flow.md +46 -0
- package/skills/sherpa/templates/ux-review.md +113 -0
- package/skills/sherpa/templates/ux-verification.md +32 -0
- package/skills/sherpa/workflows/configuration.md +23 -0
- package/skills/sherpa/workflows/demo-preparation.md +399 -0
- package/skills/sherpa/workflows/final-audit.md +713 -0
- package/skills/sherpa/workflows/idea-evaluation.md +239 -0
- package/skills/sherpa/workflows/idea-intelligence.md +69 -0
- package/skills/sherpa/workflows/implementation.md +42 -0
- package/skills/sherpa/workflows/init.md +102 -0
- package/skills/sherpa/workflows/judge-simulation.md +408 -0
- package/skills/sherpa/workflows/lifecycle.md +51 -0
- package/skills/sherpa/workflows/next.md +96 -0
- package/skills/sherpa/workflows/postmortem.md +76 -0
- package/skills/sherpa/workflows/presentation.md +522 -0
- package/skills/sherpa/workflows/project-planning.md +327 -0
- package/skills/sherpa/workflows/red-team.md +373 -0
- package/skills/sherpa/workflows/research.md +142 -0
- package/skills/sherpa/workflows/score-optimization.md +276 -0
- package/skills/sherpa/workflows/status.md +107 -0
- package/skills/sherpa/workflows/submission.md +46 -0
- package/skills/sherpa/workflows/testing-security.md +44 -0
- package/skills/sherpa/workflows/ux-review.md +59 -0
|
@@ -0,0 +1,355 @@
|
|
|
1
|
+
# sherpa — Score Optimization Rules
|
|
2
|
+
|
|
3
|
+
**Lifecycle Phase:** Phase 15 — Score Optimization Intelligence
|
|
4
|
+
**Purpose:** Authoritative rules for score optimization. Phase 15 reviews and recommends; it does not implement, replace prior phases, or modify locked artifacts.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## 1. Purpose and Boundaries
|
|
9
|
+
|
|
10
|
+
Phase 10 answers: "Given the official judging criteria, actual project evidence, judge simulation results, current implementation state, and remaining time, what changes provide the highest expected increase in judging score?"
|
|
11
|
+
|
|
12
|
+
Phase 10 optimizes **score**, not feature count.
|
|
13
|
+
|
|
14
|
+
### Phase 15 MAY:
|
|
15
|
+
|
|
16
|
+
- Inspect official judging criteria
|
|
17
|
+
- Inspect Phase 2 evaluation evidence
|
|
18
|
+
- Inspect Phase 3 engineering evidence
|
|
19
|
+
- Inspect Phase 4 implementation evidence
|
|
20
|
+
- Inspect Phase 5 testing/security evidence
|
|
21
|
+
- Inspect Phase 7 UX evidence
|
|
22
|
+
- Inspect Phase 8 red-team evidence
|
|
23
|
+
- Inspect Phase 9 judge simulation
|
|
24
|
+
- Inspect repository/project evidence
|
|
25
|
+
- Identify score gaps
|
|
26
|
+
- Estimate score impact
|
|
27
|
+
- Compare improvement options
|
|
28
|
+
- Prioritize improvements
|
|
29
|
+
- Identify quick wins
|
|
30
|
+
- Identify high-risk improvements
|
|
31
|
+
- Identify non-worthwhile work
|
|
32
|
+
- Create a score optimization plan
|
|
33
|
+
- Create prioritized optimization tasks
|
|
34
|
+
- Recommend what to implement next
|
|
35
|
+
- Update Phase 10 artifacts and project state
|
|
36
|
+
|
|
37
|
+
### Phase 15 MUST NOT:
|
|
38
|
+
|
|
39
|
+
- Redesign the idea
|
|
40
|
+
- Re-run idea generation
|
|
41
|
+
- Replace architecture
|
|
42
|
+
- Own implementation
|
|
43
|
+
- Replace testing/security
|
|
44
|
+
- Replace UX review
|
|
45
|
+
- Replace red-team review
|
|
46
|
+
- Replace judge simulation
|
|
47
|
+
- Create PPT
|
|
48
|
+
- Perform final submission
|
|
49
|
+
- Perform postmortem
|
|
50
|
+
- Silently modify application code
|
|
51
|
+
- Silently modify Phase 1–9 artifacts
|
|
52
|
+
- Silently redesign architecture; architecture changes require HUMAN ARCHITECTURE APPROVAL
|
|
53
|
+
- Recommend duplicate or unnecessary functionality that adds cost without score impact
|
|
54
|
+
- Claim score improvements without evidence
|
|
55
|
+
- Invent judging criteria
|
|
56
|
+
- Invent judge feedback
|
|
57
|
+
- Treat estimated scores as official scores
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## 2. Evidence Model
|
|
62
|
+
|
|
63
|
+
Every important optimization claim must identify its evidence status.
|
|
64
|
+
|
|
65
|
+
Allowed evidence labels:
|
|
66
|
+
|
|
67
|
+
| Label | Definition |
|
|
68
|
+
|-------|------------|
|
|
69
|
+
| FACT | Directly observed from repository inspection, verified execution, or external source |
|
|
70
|
+
| INFERENCE | Logical conclusion drawn from observed facts |
|
|
71
|
+
| ASSUMPTION | Not yet verified; requires confirmation |
|
|
72
|
+
| UNKNOWN | No information available; not observed |
|
|
73
|
+
|
|
74
|
+
Rules:
|
|
75
|
+
|
|
76
|
+
- Never convert UNKNOWN into a favorable assumption.
|
|
77
|
+
- No dimension score or gap classification may be justified by UNKNOWN or ASSUMPTION without explicit flagging.
|
|
78
|
+
- Every observation cites reproducible evidence: path/line, route/screen, steps, result, or evidence artifact.
|
|
79
|
+
- Generated sherpa artifacts are not independent evidence. Evidence must trace back to repository inspection, actual execution, external verification, or explicit human-provided information.
|
|
80
|
+
- Do not treat previous AI-generated recommendations as facts.
|
|
81
|
+
|
|
82
|
+
Example:
|
|
83
|
+
|
|
84
|
+
GOOD:
|
|
85
|
+
|
|
86
|
+
> FACT — Official judging criterion gives Technical Implementation 30%.
|
|
87
|
+
>
|
|
88
|
+
> INFERENCE — Improving reliability may materially improve technical and demo perception.
|
|
89
|
+
>
|
|
90
|
+
> UNKNOWN — Actual judge score impact cannot be known before judging.
|
|
91
|
+
|
|
92
|
+
BAD:
|
|
93
|
+
|
|
94
|
+
> Improving reliability will increase the score by 8 points.
|
|
95
|
+
|
|
96
|
+
Unless there is actual evidence supporting that numerical claim.
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## 3. Score Model
|
|
101
|
+
|
|
102
|
+
Build a transparent score model.
|
|
103
|
+
|
|
104
|
+
### 3.1 Criterion Weights
|
|
105
|
+
|
|
106
|
+
If official criterion weights exist (KNOWN from hackathon rules):
|
|
107
|
+
|
|
108
|
+
Use them. Label as OFFICIAL.
|
|
109
|
+
|
|
110
|
+
If official weights do not exist (UNKNOWN):
|
|
111
|
+
|
|
112
|
+
- Mark weights UNKNOWN
|
|
113
|
+
- Use a clearly labeled PROVISIONAL equal-weight model only for prioritization
|
|
114
|
+
- Never pretend provisional weights are official
|
|
115
|
+
- Explicitly state: "Judging criteria unknown; score based on equal-weight provisional assessment"
|
|
116
|
+
|
|
117
|
+
### 3.2 Score Formula
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
Criterion Score = Evidence-backed performance level (1–10)
|
|
121
|
+
Weighted Contribution = Criterion Score × Criterion Weight
|
|
122
|
+
Total Estimated Score = Σ(Weighted Contributions) / Σ(Weights)
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### 3.3 Score Labels
|
|
126
|
+
|
|
127
|
+
All score estimates must be labeled:
|
|
128
|
+
|
|
129
|
+
| Label | Meaning |
|
|
130
|
+
|-------|---------|
|
|
131
|
+
| OFFICIAL | Score comes from official judging rubric or verified judge feedback |
|
|
132
|
+
| ESTIMATED | Score is estimated from evidence with clear reasoning |
|
|
133
|
+
| PROVISIONAL | Score uses provisional/equal weights because official weights are UNKNOWN |
|
|
134
|
+
| UNKNOWN | Insufficient evidence to assign a score |
|
|
135
|
+
|
|
136
|
+
### 3.4 Numerical Precision
|
|
137
|
+
|
|
138
|
+
Do not fabricate numerical precision.
|
|
139
|
+
|
|
140
|
+
GOOD:
|
|
141
|
+
|
|
142
|
+
```
|
|
143
|
+
Technical Implementation:
|
|
144
|
+
Current assessment: 7/10
|
|
145
|
+
Confidence: MEDIUM
|
|
146
|
+
Weight: 30% OFFICIAL
|
|
147
|
+
Estimated contribution: 21/30
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
BAD:
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
Technical score = exactly X.Y/30
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Claiming decimal precision without supporting evidence. Report whole numbers or clearly labeled ranges unless evidence supports finer precision.
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## 4. Gap Analysis
|
|
161
|
+
|
|
162
|
+
For every criterion, determine:
|
|
163
|
+
|
|
164
|
+
| Field | Description |
|
|
165
|
+
|-------|-------------|
|
|
166
|
+
| Criterion | Official judging criterion name |
|
|
167
|
+
| Official Weight | Weight or UNKNOWN |
|
|
168
|
+
| Current Evidence | Summary of evidence backing current level |
|
|
169
|
+
| Current Performance | 1–10 with confidence |
|
|
170
|
+
| Confidence | HIGH / MEDIUM / LOW / UNKNOWN |
|
|
171
|
+
| Strengths | What is working well |
|
|
172
|
+
| Weaknesses | What is weak |
|
|
173
|
+
| Evidence Gap | Weak project performance vs weak evidence of performance (separate findings) |
|
|
174
|
+
| Judge Visibility | How observable is this to a judge |
|
|
175
|
+
| Improvement Potential | NONE / LOW / MEDIUM / HIGH / CRITICAL / UNKNOWN |
|
|
176
|
+
| Risk | Risk of attempting improvement |
|
|
177
|
+
|
|
178
|
+
Gap classification:
|
|
179
|
+
|
|
180
|
+
| Classification | Meaning |
|
|
181
|
+
|----------------|---------|
|
|
182
|
+
| NONE | No meaningful gap |
|
|
183
|
+
| LOW | Small gap |
|
|
184
|
+
| MEDIUM | Moderate gap |
|
|
185
|
+
| HIGH | Large gap |
|
|
186
|
+
| CRITICAL | Severe gap threatening scoring |
|
|
187
|
+
| UNKNOWN | Insufficient evidence to classify |
|
|
188
|
+
|
|
189
|
+
### 4.1 Separate Findings
|
|
190
|
+
|
|
191
|
+
Do not confuse:
|
|
192
|
+
|
|
193
|
+
**Weak project performance** — The capability itself is weak
|
|
194
|
+
|
|
195
|
+
with
|
|
196
|
+
|
|
197
|
+
**Weak evidence of project performance** — The capability may exist but is not demonstrated, documented, or observable by a judge.
|
|
198
|
+
|
|
199
|
+
These must be separate findings. A project may have strong internal implementation (weak evidence = fixable with documentation/demo) or weak implementation (weak project performance = requires actual code change).
|
|
200
|
+
|
|
201
|
+
---
|
|
202
|
+
|
|
203
|
+
## 5. Improvement Categories
|
|
204
|
+
|
|
205
|
+
Every proposed improvement must belong to one or more categories:
|
|
206
|
+
|
|
207
|
+
| ID | Category | Description |
|
|
208
|
+
|----|----------|-------------|
|
|
209
|
+
| A | Differentiation | Improves uniqueness or competitive positioning |
|
|
210
|
+
| B | Technical Credibility | Improves architecture quality, engineering quality, reliability, robustness, technical depth |
|
|
211
|
+
| C | Core Functionality | Improves the primary user-facing value |
|
|
212
|
+
| D | Demo Impact | Makes the core flow clearer, faster, more impressive, or more reliable |
|
|
213
|
+
| E | UX | Improves usability, clarity, accessibility, responsive behavior, or interaction quality |
|
|
214
|
+
| F | Security / Trust | Improves security, privacy, validation, resilience, or trust |
|
|
215
|
+
| G | Evidence | Improves the ability to prove that an existing capability works |
|
|
216
|
+
| H | Presentation | Presentation-related score opportunities within current phase scope only. Do not create PPT content; Phase 17 owns presentation. |
|
|
217
|
+
| I | Reliability | Improves stability and reduces judge-visible failure risk |
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## 6. Improvement Value Model
|
|
222
|
+
|
|
223
|
+
Each candidate improvement must be evaluated using:
|
|
224
|
+
|
|
225
|
+
| Dimension | Scale |
|
|
226
|
+
|-----------|-------|
|
|
227
|
+
| Score Impact | VERY LOW / LOW / MEDIUM / HIGH / VERY HIGH |
|
|
228
|
+
| Confidence | HIGH / MEDIUM / LOW / UNKNOWN |
|
|
229
|
+
| Judge Visibility | LOW / MEDIUM / HIGH / VERY HIGH |
|
|
230
|
+
| Implementation Cost | TRIVIAL / LOW / MEDIUM / HIGH / VERY HIGH |
|
|
231
|
+
| Risk | LOW / MEDIUM / HIGH / CRITICAL / UNKNOWN |
|
|
232
|
+
|
|
233
|
+
**OPTIMIZATION HEURISTIC** (prioritization heuristic, NOT judging mathematics):
|
|
234
|
+
|
|
235
|
+
```
|
|
236
|
+
Expected Value = Score Impact × Confidence × Judge Visibility ÷ Implementation Cost
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
Rules:
|
|
240
|
+
|
|
241
|
+
- Clearly label this as **OPTIMIZATION HEURISTIC**.
|
|
242
|
+
- Do not present it as judging mathematics.
|
|
243
|
+
- Use qualitative values where numerical precision is unsupported.
|
|
244
|
+
- The heuristic ranks candidates for prioritization; it does not produce official scores.
|
|
245
|
+
|
|
246
|
+
Each candidate must also identify:
|
|
247
|
+
|
|
248
|
+
- Time required
|
|
249
|
+
- Dependencies / blockers
|
|
250
|
+
- Whether the improvement is actually worth doing
|
|
251
|
+
- What should explicitly NOT be changed (to protect stable, high-scoring areas)
|
|
252
|
+
|
|
253
|
+
---
|
|
254
|
+
|
|
255
|
+
## 7. Priority Classification
|
|
256
|
+
|
|
257
|
+
Every optimization must receive exactly one priority:
|
|
258
|
+
|
|
259
|
+
| Priority | Definition | Action |
|
|
260
|
+
|----------|------------|--------|
|
|
261
|
+
| **P0 — CRITICAL** | Must address because it threatens judging readiness, core demo, major criterion, reliability, security, or submission viability | Implement first; human approval if architecture/scope change |
|
|
262
|
+
| **P1 — HIGH IMPACT** | Strong score improvement with realistic implementation cost | Implement after P0 items |
|
|
263
|
+
| **P2 — MEDIUM IMPACT** | Useful improvement but not essential | Implement if time permits after P0/P1 |
|
|
264
|
+
| **P3 — LOW IMPACT** | Minor polish | Implement only if all higher priorities complete and time remains |
|
|
265
|
+
| **DROP** | Do not implement | Explicitly rejected; document reason |
|
|
266
|
+
|
|
267
|
+
Every candidate must receive exactly one priority. No candidate may be unclassified.
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
## 8. Time-Aware Filtering
|
|
272
|
+
|
|
273
|
+
Apply the lifecycle time-aware strategy:
|
|
274
|
+
|
|
275
|
+
| Time Remaining | Allow | Block |
|
|
276
|
+
|----------------|-------|-------|
|
|
277
|
+
| > 24 hours | Core product, differentiation, architecture, important features, validation | Low-judging-value features |
|
|
278
|
+
| 12–24 hours | Completing MVP, high-impact features, integration, testing | New feature exploration |
|
|
279
|
+
| 6–12 hours | Stability, testing, UX, security, demo reliability, presentation | Architectural changes |
|
|
280
|
+
| 2–6 hours | Bug fixing, demo flow, critical polish, PPT, documentation, deployment reliability | Any new feature development |
|
|
281
|
+
| < 2 hours | Build working, deployment working, demo working, PPT accurate, links working, critical bugs fixed, submission requirements satisfied | All feature development |
|
|
282
|
+
|
|
283
|
+
If time remaining is UNKNOWN:
|
|
284
|
+
|
|
285
|
+
- Mark it UNKNOWN
|
|
286
|
+
- Produce a time-neutral plan
|
|
287
|
+
- Recommend establishing the deadline if necessary
|
|
288
|
+
|
|
289
|
+
---
|
|
290
|
+
|
|
291
|
+
## 9. Project State Fields
|
|
292
|
+
|
|
293
|
+
Update ONLY these compact fields in `.sherpa/project-state.md`:
|
|
294
|
+
|
|
295
|
+
| Field | Allowed Values |
|
|
296
|
+
|-------|---------------|
|
|
297
|
+
| Score Optimization Status | NOT_STARTED | IN_PROGRESS | BLOCKED | COMPLETE |
|
|
298
|
+
| Current Estimated Score | <score or UNKNOWN> |
|
|
299
|
+
| Potential Score | <score or UNKNOWN> |
|
|
300
|
+
| Largest Gaps | <concise list or UNKNOWN> |
|
|
301
|
+
| Highest-Priority Improvement | <concise or UNKNOWN> |
|
|
302
|
+
|
|
303
|
+
Rules:
|
|
304
|
+
|
|
305
|
+
- Do NOT put detailed optimization analysis inside project-state.md.
|
|
306
|
+
- Detailed material goes to `.sherpa/score-optimization.md` and `.sherpa/score-optimization/`.
|
|
307
|
+
- Do not change `selected idea`.
|
|
308
|
+
- Do not silently change architecture.
|
|
309
|
+
- Do not silently modify application code.
|
|
310
|
+
- Do not silently modify Phase 1–9 artifacts.
|
|
311
|
+
- Human approval required for major scope or architecture changes.
|
|
312
|
+
|
|
313
|
+
---
|
|
314
|
+
|
|
315
|
+
## 10. Failure Cases
|
|
316
|
+
|
|
317
|
+
| Case | Behavior |
|
|
318
|
+
|------|----------|
|
|
319
|
+
| No selected idea | Stop and request confirmation |
|
|
320
|
+
| Missing Phase 9 judge simulation | Report SCORE-OPTIMIZE = LIMITED; proceed with available evidence, mark gaps UNKNOWN |
|
|
321
|
+
| Missing judging criteria | Mark UNKNOWN; use provisional equal-weight model; state explicitly |
|
|
322
|
+
| Missing repository evidence | Lower confidence; mark unknowns explicitly; do not fabricate |
|
|
323
|
+
| Unsupported score claim | Flag as unsupported; do not treat as verified |
|
|
324
|
+
| Contradictory claims | Record contradiction; do not silently reconcile |
|
|
325
|
+
| Architecture change required | Record ARCHITECTURE CHANGE REQUIRED; route to HUMAN ARCHITECTURE APPROVAL; stop that remediation |
|
|
326
|
+
| Idea change attempt | Blocked; route back to Phase 2 / human |
|
|
327
|
+
| Unknown deadline | Do not assume one; mark TIME STRATEGY = UNKNOWN |
|
|
328
|
+
| Already-mitigated finding | If finding is VERIFIED_MITIGATED, do not optimize against the resolved issue; record as resolved |
|
|
329
|
+
|
|
330
|
+
Phase 15 MUST NOT:
|
|
331
|
+
|
|
332
|
+
- Write application code
|
|
333
|
+
- Modify application source files
|
|
334
|
+
- Modify frontend/backend implementation
|
|
335
|
+
- Install dependencies
|
|
336
|
+
- Deploy the application
|
|
337
|
+
- Change the selected idea
|
|
338
|
+
- Silently redesign architecture
|
|
339
|
+
- Replace Phase 5 testing/security
|
|
340
|
+
- Replace Phase 7 UX review
|
|
341
|
+
- Replace Phase 8 Red Team
|
|
342
|
+
- Replace Phase 9 Judge Simulation
|
|
343
|
+
- Perform final presentation generation
|
|
344
|
+
- Perform final submission
|
|
345
|
+
- Perform post-mortem
|
|
346
|
+
- Fabricate judge opinions
|
|
347
|
+
- Claim judge reactions without evidence
|
|
348
|
+
- Claim score improvements without evidence
|
|
349
|
+
- Silently override Phase 8 or Phase 9 decisions
|
|
350
|
+
|
|
351
|
+
---
|
|
352
|
+
|
|
353
|
+
## 11. Determinism
|
|
354
|
+
|
|
355
|
+
Same evidence, criteria, weights, and time remaining → same optimization plan and priority classifications every time. No randomness. No LLM-generated numeric scores without explicit numeric input.
|
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
# sherpa — Scoring Rules
|
|
2
|
+
|
|
3
|
+
Defines the Idea Intelligence Engine scoring model. All scores are deterministic:
|
|
4
|
+
- Same evidence, criteria, weights, team capability, and time → same score and recommendation.
|
|
5
|
+
- No randomness. No LLM-generated scores without explicit numeric input.
|
|
6
|
+
- No stacked multipliers. One Feasibility & Risk Gate only.
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## 1. Diagnostic Dimension Categories (21 dimensions, each in exactly one category)
|
|
11
|
+
|
|
12
|
+
Each dimension appears in exactly one category. Categories contribute to non-overlapping judging criteria.
|
|
13
|
+
Category 6 (Criteria Alignment) is a validation layer — it does not add points.
|
|
14
|
+
|
|
15
|
+
| # | Category | Dimensions | Primary Judging Criteria Mapped |
|
|
16
|
+
|---|----------|------------|--------------------------------|
|
|
17
|
+
| 1 | Problem & Solution | 1, 2, 3, 4, 5 | Problem significance, User value |
|
|
18
|
+
| 2 | Innovation & Differentiation | 6, 7 | Innovation |
|
|
19
|
+
| 3 | Technical Feasibility | 8, 9, 10 | Technical implementation |
|
|
20
|
+
| 4 | Implementation Reality | 11, 12 | Feasibility, Team capability |
|
|
21
|
+
| 5 | Demo & Presentation | 13, 14 | Demo quality, Presentation quality |
|
|
22
|
+
| 6 | Criteria Alignment | 15 | All judging criteria (meta-validation) |
|
|
23
|
+
| 7 | Impact & Market | 16, 17 | Potential impact, Business value |
|
|
24
|
+
| 8 | Competition & Risk | 18, 19 | Differentiation context, Market opportunity |
|
|
25
|
+
| 9 | Risk & Evidence | 20, 21 | Risk level, Score reliability |
|
|
26
|
+
|
|
27
|
+
**Dimension 15 (Judging-criteria alignment)** is placed in Category 6 only. It produces a boolean check: "Do all category scores map to stated criteria?" If criteria are `UNKNOWN`, this check fails and overall confidence is lowered.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## 2. Dimension Definitions (1–10 scale unless noted)
|
|
32
|
+
|
|
33
|
+
| # | Dimension | Scale | Meaning |
|
|
34
|
+
|---|-----------|-------|---------|
|
|
35
|
+
| 1 | Problem importance | 1–10 | How meaningful is the problem? |
|
|
36
|
+
| 2 | Problem clarity | 1–10 | Is the problem well-defined? |
|
|
37
|
+
| 3 | Target-user clarity | 1–10 | Who is the user, exactly? |
|
|
38
|
+
| 4 | User pain | 1–10 | How painful is the problem for users? |
|
|
39
|
+
| 5 | Solution usefulness | 1–10 | Does the solution actually help? |
|
|
40
|
+
| 6 | Innovation | 1–10 | Is it novel? |
|
|
41
|
+
| 7 | Differentiation | 1–10 | How different from competitors? |
|
|
42
|
+
| 8 | Technical feasibility | 1–10 | Can it be built? (10 = high feasibility) |
|
|
43
|
+
| 9 | AI/ML necessity | 1–10 | Is AI actually needed? (10 = essential, 1 = superficial) |
|
|
44
|
+
| 10 | Implementation complexity | 1–10 | How complex to build? (10 = high complexity) |
|
|
45
|
+
| 11 | Time required | 1–10 | How much time needed? (10 = maximum time) |
|
|
46
|
+
| 12 | Team skill fit | 1–10 | Does the team have the skills? (10 = high fit) |
|
|
47
|
+
| 13 | Demo potential | 1–10 | Is the demo compelling? |
|
|
48
|
+
| 14 | Judge appeal | 1–10 | Will judges like it? |
|
|
49
|
+
| 15 | Judging-criteria alignment | 1–10 | Do scores map to stated criteria? |
|
|
50
|
+
| 16 | Scalability/potential | 1–10 | Growth potential beyond hackathon |
|
|
51
|
+
| 17 | Real-world impact | 1–10 | Actual user value |
|
|
52
|
+
| 18 | Competitive landscape | 1–10 | Less competition = higher score |
|
|
53
|
+
| 19 | Defensibility | 1–10 | Hard to copy? |
|
|
54
|
+
| 20 | Risk | 1–10 | **10 = highest risk** (penalty increases with score) |
|
|
55
|
+
| 21 | Evidence quality | 1–10 | How well-backed are claims? |
|
|
56
|
+
|
|
57
|
+
**Risk direction (dimension 20):** 1 = lowest risk, 10 = highest risk. Higher score = higher penalty.
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## 3. Criterion-to-Category Mapping
|
|
62
|
+
|
|
63
|
+
| Judging Criterion | Contributing Categories | Dimensions |
|
|
64
|
+
|-------------------|------------------------|------------|
|
|
65
|
+
| Problem significance | Problem & Solution | 1, 2, 4 |
|
|
66
|
+
| User value | Problem & Solution | 3, 5 |
|
|
67
|
+
| Innovation | Innovation & Differentiation | 6 |
|
|
68
|
+
| Technical implementation | Technical Feasibility | 8, 9, 10 |
|
|
69
|
+
| Feasibility / Team | Implementation Reality | 11, 12 |
|
|
70
|
+
| Demo quality | Demo & Presentation | 13 |
|
|
71
|
+
| Presentation quality | Demo & Presentation | 14 |
|
|
72
|
+
| Criteria alignment | Criteria Alignment | 15 (validation only) |
|
|
73
|
+
| Impact / potential | Impact & Market | 16, 17 |
|
|
74
|
+
| Market opportunity | Competition & Risk | 18, 19 |
|
|
75
|
+
| Risk adjustment | Risk & Evidence | 20, 21 |
|
|
76
|
+
|
|
77
|
+
**No double-counting rule:** Each dimension appears in exactly one category. Each category contributes to non-overlapping judging criteria. Category 6 does not add points — it validates mapping.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## 4. Judging Criteria Weighting
|
|
82
|
+
|
|
83
|
+
### When criteria and weights are known
|
|
84
|
+
|
|
85
|
+
Use the hackathon's published judging criteria and weights as the primary scoring buckets:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
raw_score = Σ(criterion_score_i × weight_i) / Σ(weight_i)
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Where `criterion_score_i` is the weighted average of category scores mapped to that criterion.
|
|
92
|
+
|
|
93
|
+
### When criteria/weights are UNKNOWN
|
|
94
|
+
|
|
95
|
+
- Use equal weights across available categories
|
|
96
|
+
- Set overall confidence flag = `low`
|
|
97
|
+
- Explicitly state: "Judging criteria unknown; score based on equal-weight diagnostic assessment"
|
|
98
|
+
- Do NOT present false precision
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## 5. Feasibility & Risk Gate (single documented adjustment)
|
|
103
|
+
|
|
104
|
+
The five feasibility/risk dimensions (8, 10, 11, 12, 20) already feed into categories 3, 4, and 9. Instead of applying separate multipliers on top, apply **one documented gate**:
|
|
105
|
+
|
|
106
|
+
```
|
|
107
|
+
feasibility_composite = avg(
|
|
108
|
+
dimension_8, // technical feasibility (10 = high feasibility)
|
|
109
|
+
(11 - dimension_10), // implementation complexity inverted (10 = simple)
|
|
110
|
+
(11 - dimension_11), // time required inverted (10 = fits in time)
|
|
111
|
+
dimension_12, // team skill fit (10 = high fit)
|
|
112
|
+
(11 - dimension_20) // risk inverted (10 = low risk)
|
|
113
|
+
)
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
**Gate thresholds:**
|
|
117
|
+
|
|
118
|
+
| Composite Range | Gate Result | Implication |
|
|
119
|
+
|-----------------|-------------|-------------|
|
|
120
|
+
| ≥ 6.0 | PASS | Idea can be SELECTed |
|
|
121
|
+
| < 6.0 | FAIL | Idea cannot be SELECTed; must be REVISE or REJECT_ALL |
|
|
122
|
+
|
|
123
|
+
**Rationale:** This is a single gate, not stacked multipliers. The five dimensions count once in their categories; the gate is a single feasibility threshold.
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## 6. Final Score Formula
|
|
128
|
+
|
|
129
|
+
```
|
|
130
|
+
raw_score = weighted criterion average (0–100)
|
|
131
|
+
final_score = raw_score × (feasibility_composite / 10)
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
The `(feasibility_composite / 10)` factor is the **only** multiplicative adjustment. No team_fit_factor, no time_pressure_factor, no separate risk_penalty. Those dimensions are already accounted for inside `feasibility_composite`.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
## 7. Confidence Model
|
|
139
|
+
|
|
140
|
+
| Factor | High Confidence | Medium Confidence | Low Confidence |
|
|
141
|
+
|--------|----------------|-------------------|----------------|
|
|
142
|
+
| Evidence quality (dim 21) | ≥ 8 | 5–7 | ≤ 4 |
|
|
143
|
+
| Judging criteria known | Yes | Partial | UNKNOWN |
|
|
144
|
+
| Dimension scores all filled | Yes | 1–2 missing | 3+ missing |
|
|
145
|
+
|
|
146
|
+
**Overall confidence:** Lowest confidence across all factors.
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## 8. Evidence Model
|
|
151
|
+
|
|
152
|
+
Every research claim carries:
|
|
153
|
+
|
|
154
|
+
| Field | Values |
|
|
155
|
+
|-------|--------|
|
|
156
|
+
| `type` | FACT | INFERENCE | ASSUMPTION | UNKNOWN |
|
|
157
|
+
| `value` | The claim |
|
|
158
|
+
| `source` | URL, file path, or "team input" |
|
|
159
|
+
| `confidence` | low | medium | high |
|
|
160
|
+
|
|
161
|
+
**Rules:**
|
|
162
|
+
- No dimension score may be justified by `UNKNOWN` or `ASSUMPTION` without explicit flagging.
|
|
163
|
+
- Never claim research was performed unless evidence was actually retrieved.
|
|
164
|
+
- Separate FACT from INFERENCE from ASSUMPTION.
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## 9. Red-Team Outcomes (Deterministic)
|
|
169
|
+
|
|
170
|
+
| Outcome | Definition | Conditions |
|
|
171
|
+
|---------|------------|------------|
|
|
172
|
+
| **KILL** | Idea cannot be presented; fundamental flaw | (a) Feasibility gate FAIL with no credible revision path within time, OR (b) dimension_20 ≥ 9 AND dimension_8 ≤ 3, OR (c) dimension_9 ≤ 3 AND idea claims "AI-powered" (superficial AI) |
|
|
173
|
+
| **MAJOR CONCERN** | Significant weakness, fixable | Any criterion-critical dimension scores 3–5, OR feasibility gate score 6.0–7.0 |
|
|
174
|
+
| **MINOR CONCERN** | Cosmetic weakness | Any non-critical dimension scores 6–7 |
|
|
175
|
+
|
|
176
|
+
**Not unnecessarily aggressive:** KILL requires either (a) feasibility gate failure with no revision path, or (b) simultaneous high risk + low feasibility, or (c) superficial AI claim. A single medium score does not trigger KILL.
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## 10. Decision Outcomes
|
|
181
|
+
|
|
182
|
+
| Outcome | Meaning | Effect on project-state.md |
|
|
183
|
+
|---------|---------|---------------------------|
|
|
184
|
+
| **SELECT** | One idea recommended | Writes to `.sherpa/decisions.md` only. Does NOT set `selected idea` |
|
|
185
|
+
| **REVISE** | Ideas need modification | Writes revision plan to `.sherpa/decisions.md`. No change to `selected idea` |
|
|
186
|
+
| **REJECT_ALL** | No candidate ideas viable | Writes rejection rationale to `.sherpa/decisions.md`. No change to `selected idea` |
|
|
187
|
+
|
|
188
|
+
**Human authority:** Only explicit human confirmation can change `selected idea` in `.sherpa/project-state.md`.
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
## 11. Acceptance Test Definitions
|
|
193
|
+
|
|
194
|
+
| Test | Expected Result |
|
|
195
|
+
|------|----------------|
|
|
196
|
+
| Missing dimension detection | Any dimension scored `UNKNOWN` is flagged; score carries low confidence |
|
|
197
|
+
| Risk direction correctness | risk_dimension=10 produces higher penalty than risk_dimension=1 |
|
|
198
|
+
| No double-counting | Each of 21 dimensions appears in exactly one category |
|
|
199
|
+
| Known judging criteria | Score uses published weights; confidence not lowered for criteria |
|
|
200
|
+
| Unknown judging criteria | Score uses equal weights; confidence flag = low; explicit note stated |
|
|
201
|
+
| Human confirmation requirement | Engine never auto-sets `selected idea`; human must confirm explicitly |
|
|
202
|
+
| REVISE loop | REVISE → weaknesses → changes → re-evaluate → new outcome supported |
|
|
203
|
+
| Red-team kill conditions | KILL triggers only under defined conditions (gate failure, risk+feasibility, superficial AI) |
|
|
204
|
+
| Exactly one primary next action | Output always contains exactly one PRIMARY NEXT ACTION |
|
|
205
|
+
|
|
206
|
+
---
|
|
207
|
+
|
|
208
|
+
## 12. Determinism Guarantee
|
|
209
|
+
|
|
210
|
+
Two runs with identical:
|
|
211
|
+
- Evidence (same FACT/INFERENCE/ASSUMPTION/UNKNOWN tags and sources)
|
|
212
|
+
- Criteria and weights
|
|
213
|
+
- Team capability (dimension 12)
|
|
214
|
+
- Time remaining (dimension 11)
|
|
215
|
+
|
|
216
|
+
…must produce identical scores and recommendations. No randomness. No LLM-generated numeric scores without explicit input.
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# sherpa — Phase 19 Submission Rules
|
|
2
|
+
|
|
3
|
+
**Lifecycle Phase:** Phase 19 — Submission Intelligence
|
|
4
|
+
**Operation:** `SUBMISSION-PREPARE`
|
|
5
|
+
**Purpose:** Discover, verify, and freeze a submission package. Phase 19 prepares; the human submits.
|
|
6
|
+
|
|
7
|
+
## Authority and evidence
|
|
8
|
+
|
|
9
|
+
Use sources in this order: official rules; official portal instructions; organizer communications; official judging criteria; explicit project requirements; project-specific instructions; user information; internal artifacts. Preserve conflicts, identify authority, and request human verification where needed. An undocumented requirement is not mandatory absent evidence.
|
|
10
|
+
|
|
11
|
+
Classify evidence as FACT, INFERENCE, ASSUMPTION, or UNKNOWN. Each VERIFIED claim must cite its actual source and verification method. A file's existence does not prove its contents, accessibility, permissions, correctness, freshness, or format. UNKNOWN never becomes verified by inference. Do not use fabricated test, build, dependency, API, URL, access, submission, or deployment claims.
|
|
12
|
+
|
|
13
|
+
## Requirement and status models
|
|
14
|
+
|
|
15
|
+
Each requirement records ID, requirement, source, mandatory flag, evidence, artifact/field, status, owner, deadline, and notes. Status: VERIFIED, READY, MISSING, INVALID, BLOCKED, UNKNOWN, NOT_APPLICABLE.
|
|
16
|
+
|
|
17
|
+
Deadline record: deadline, timezone, source, confidence, current status, time remaining. Deadline states: UPCOMING, URGENT, DUE_SOON, DEADLINE_PASSED, UNKNOWN. Never assume a timezone or invent a deadline.
|
|
18
|
+
|
|
19
|
+
Artifact record: ID, name, type, location, version, required flag, accessibility, verification, last checked, hash/identifier, status. Field record: ID, field, required flag, value, source, evidence, character limit, format, status, notes. Unknown values stay UNKNOWN.
|
|
20
|
+
|
|
21
|
+
Link states: VERIFIED_ACCESSIBLE, ACCESSIBLE_WITH_AUTH, BROKEN, REDIRECTED, UNKNOWN, NOT_APPLICABLE. Access states: PUBLIC, RESTRICTED, AUTH_REQUIRED, EXPIRED, UNKNOWN. Verify syntax, destination, expected content, permissions, authentication, and expiry when possible. Do not expose credentials or recommend public exposure of sensitive data.
|
|
22
|
+
|
|
23
|
+
## Verification boundaries
|
|
24
|
+
|
|
25
|
+
Inspect the actual repository and available package. Check repository/branch/intended commit, README and setup documentation, required files, links, accidental private data, debug artifacts, and configuration. Verify deployment, demo/video, PPT and file limits only when actual artifacts and authoritative requirements are available. Never imply setup works without executing it. Secret checks use bounded scope language: “No secret indicators were detected within the audited submission scope.”
|
|
26
|
+
|
|
27
|
+
Keep submission form, README, PPT, demo, repository, deployment, project description, selected idea, and final audit claims consistent. Audit claims as VERIFIED, SUPPORTED, PARTIALLY_SUPPORTED, UNSUPPORTED, CONTRADICTED, UNKNOWN. Unsupported claims must be corrected or removed; contradicted claims must not be submitted; metrics require evidence.
|
|
28
|
+
|
|
29
|
+
## Snapshot, freeze, and readiness
|
|
30
|
+
|
|
31
|
+
Create a submission snapshot with snapshot ID, timestamp/timezone, repository commit, branch, frontend/backend/PPT/video/documentation versions, deployment URL, and status. Mark missing values UNKNOWN. After snapshot, classify changes REQUIRED, SAFE, RISKY, or FORBIDDEN_AFTER_FREEZE. Freeze is ACTIVE only after critical artifacts are verified; unresolved required unknowns prevent claiming complete verification. After freeze, only critical submission fixes, broken-link fixes, required metadata corrections, access fixes, and organizer-required changes are routine. Destructive changes, idea changes, major architecture changes, replacing the submission artifact, and scope changes require explicit human approval.
|
|
32
|
+
|
|
33
|
+
Readiness values: NOT_READY, CONDITIONAL, BLOCKED, READY_TO_SUBMIT. READY_TO_SUBMIT requires all mandatory requirements verified, required artifacts and formats verified, links accessible, critical claims supported, no unresolved critical blockers, a consistent package and final snapshot, deadline known or explicitly verified, and human confirmation still pending. Framework validation alone never yields live READY_TO_SUBMIT. Human submission remains external and unchecked.
|
|
34
|
+
|
|
35
|
+
Each risk records ID, description, severity, likelihood, confidence, impact, mitigation, owner, status. Keep severity and confidence separate. Critical risk blocks READY unless VERIFIED_MITIGATED or ACCEPTED_RISK with human authority.
|
|
36
|
+
|
|
37
|
+
## Human control and prohibited operations
|
|
38
|
+
|
|
39
|
+
Always surface `FINAL HUMAN CONFIRMATION REQUIRED` before the eventual human submission, including package, destination, deadline/timezone, risks, snapshot, unresolved noncritical issues, and irreversible consequences. The system must NEVER click a final submit button, submit a form or entry, publish private information, change repository visibility, upload to the portal, or accept irreversible terms. It never performs the actual submission. The final checklist item “Submission performed” remains unchecked in Phase 19.
|
|
40
|
+
|
|
41
|
+
Do not develop features, redesign architecture/demo/PPT/final audit, change the selected idea, perform post-mortem, install dependencies, deploy, or modify application code. Preserve locked Phase 1–18 behavior and exactly one `PRIMARY NEXT ACTION`.
|