quantum-reasoning-skill 1.0.0 → 1.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/SKILL.md CHANGED
@@ -1,146 +1,146 @@
1
- ---
2
- name: quantum-reasoning
3
- description: Maintain multiple genuinely different candidate hypotheses or solution paths, test them against evidence and tools, suppress weak or contradictory paths, revive useful alternatives when new evidence appears, and collapse to the best-supported answer only at the end.
4
- ---
5
-
6
- # Quantum Reasoning Skill
7
-
8
- A model-facing reasoning protocol inspired by the idea of keeping multiple possibilities alive before selection. It runs on classical models and classical hardware; it is not quantum computation.
9
-
10
- ## When to use
11
-
12
- Use this skill when a problem is difficult, ambiguous, high-stakes, multi-step, or vulnerable to early commitment: mathematics, scientific reasoning, debugging, planning, architecture, model selection, hypothesis testing, and technical diagnosis.
13
-
14
- Do not add branching overhead to trivial questions.
15
-
16
- ## Core protocol
17
-
18
- ### 1. Frame the state
19
-
20
- Identify the objective, constraints, verified facts, assumptions, unknowns, and available tools. Keep facts separate from hypotheses.
21
-
22
- ### 2. Open a diverse possibility set
23
-
24
- Create several materially different candidate paths. Diversity is mandatory: do not count paraphrases, cosmetic variants, or branches that depend on the same hidden assumption as independent possibilities.
25
-
26
- Prefer orthogonal strategy classes when applicable, such as:
27
-
28
- - direct derivation
29
- - counterexample / falsification
30
- - decomposition
31
- - alternative model or mechanism
32
- - numerical or executable test
33
- - independent reconstruction
34
- - adversarial critique
35
-
36
- ### 3. Maintain branch states
37
-
38
- Treat each candidate as one of:
39
-
40
- - **active** — worth more compute now
41
- - **dormant** — currently weak but potentially recoverable
42
- - **rejected** — contradicted or decisively invalidated
43
-
44
- Never permanently reject a branch merely because another branch initially looks more plausible.
45
-
46
- ### 4. Evaluate each branch independently
47
-
48
- Judge branches using evidence, not eloquence. Consider:
49
-
50
- - empirical or logical support
51
- - external verification
52
- - independence from other branches
53
- - information gain
54
- - contradiction count and severity
55
- - unresolved assumptions
56
- - expected value of another reasoning step
57
- - compute / tool cost
58
-
59
- Self-reported model confidence alone is weak evidence.
60
-
61
- ### 5. Use tools to falsify, not merely confirm
62
-
63
- When available, route claims through the strongest suitable verifier: executable code, numerical tests, formal solvers, source lookup, direct file inspection, unit tests, symbolic algebra, or an independent model pass.
64
-
65
- Actively search for evidence that would disprove leading branches.
66
-
67
- ### 6. Apply interference-like comparison
68
-
69
- Compare branches against each other without pretending to perform quantum interference.
70
-
71
- - Reinforce conclusions independently reached by different verified paths.
72
- - Penalize branches that conflict with verified facts.
73
- - Detect shared assumptions so correlated branches are not double-counted.
74
- - Transfer newly verified facts to every branch they constrain.
75
- - Preserve meaningful disagreement instead of averaging incompatible claims.
76
-
77
- ### 7. Allocate compute dynamically
78
-
79
- Use uncertainty to control width and depth.
80
-
81
- - **Low uncertainty:** narrow quickly and deepen the strongest path.
82
- - **High uncertainty:** widen the possibility set.
83
- - **High-value unresolved branch:** spend additional tool or reasoning budget.
84
- - **Redundant or contradicted branch:** move to dormant or rejected.
85
-
86
- Branch count is adaptive, not fixed.
87
-
88
- ### 8. Permit branch revival
89
-
90
- If new evidence invalidates a leading assumption or materially improves a dormant branch, restore that branch to active status and re-evaluate the ranking.
91
-
92
- ### 9. Stop when more search has low value
93
-
94
- Collapse the possibility set when one result is sufficiently verified, remaining branches are redundant or falsified, or additional search is unlikely to change the decision enough to justify its cost.
95
-
96
- If multiple branches remain genuinely unresolved, report the alternatives and what evidence would distinguish them.
97
-
98
- ### 10. Produce the answer
99
-
100
- Synthesize only from the strongest supported surviving branches. Run a final contradiction and requirement check before answering.
101
-
102
- Do not expose hidden chain-of-thought. Return the conclusion, concise supporting evidence, important uncertainty, and materially different surviving alternatives when necessary.
103
-
104
- ## Adaptive intensity
105
-
106
- Use a rough internal intensity level:
107
-
108
- - **light:** 2–3 distinct candidates, one comparison pass
109
- - **standard:** 4–8 candidates, verification + pruning + one revival check
110
- - **deep:** 8+ candidates only when the problem and compute budget justify it; use repeated verification and adaptive expansion
111
-
112
- Do not mechanically maximize branch count. The goal is better search of the possibility space, not more text.
113
-
114
- ## Operational reference policy
115
-
116
- When the host system can track aggregate branch metadata, use explicit normalized values in `[0, 1]` for evidence, verification, independence, information gain, contradiction severity, unresolved assumptions, compute cost, shared-assumption overlap, and semantic redundancy.
117
-
118
- A provider-agnostic reference controller is available in [`reference/branch_controller.py`](./reference/branch_controller.py). Its defaults make the qualitative protocol testable:
119
-
120
- - independence is reduced when branches share critical assumptions or are near-duplicates
121
- - strong contradiction can reject a branch
122
- - weak branches become dormant instead of disappearing
123
- - dormant branches can revive after a material evidence, verification, or contradiction change
124
- - uncertainty controls a recommended search width
125
- - collapse requires a verified leader, low contradiction, and a meaningful margin over surviving alternatives
126
-
127
- The exact scoring weights and thresholds are **reference defaults, not validated constants**. They must be calibrated against benchmark evidence rather than treated as universal truths. See [`docs/MEASUREMENT.md`](./docs/MEASUREMENT.md) for the auditable formulas and thresholds.
128
-
129
- Hosts that cannot expose these aggregate measurements should still follow the qualitative protocol above and must not invent numeric telemetry.
130
-
131
- ## Benchmark requirement
132
-
133
- Do not claim that this skill improves reasoning merely because the protocol sounds plausible. Compare the same model and task configuration with and without the skill and measure at least accuracy, compute/token cost, latency, error recovery, contradiction resolution, and branch diversity when observable.
134
-
135
- The reproducible benchmark schema and evaluator live under [`benchmark/`](./benchmark/README.md).
136
-
137
- ## Non-negotiable rules
138
-
139
- 1. Do not call this real quantum computation.
140
- 2. Do not treat repeated wording as independent evidence.
141
- 3. Do not let one branch evaluate itself without challenge when verification is possible.
142
- 4. Prefer falsification and executable checks over intuition.
143
- 5. Preserve a recoverable record of high-information dormant branches.
144
- 6. Collapse only after cross-branch comparison.
145
- 7. Never fabricate evidence, tool results, sources, certainty, or branch telemetry.
146
- 8. Do not present reference thresholds as empirically validated until benchmark evidence exists.
1
+ ---
2
+ name: quantum-reasoning
3
+ description: Maintain multiple genuinely different candidate hypotheses or solution paths, test them against evidence and tools, suppress weak or contradictory paths, revive useful alternatives when new evidence appears, and collapse to the best-supported answer only at the end.
4
+ ---
5
+
6
+ # Quantum Reasoning Skill
7
+
8
+ A model-facing reasoning protocol inspired by the idea of keeping multiple possibilities alive before selection. It runs on classical models and classical hardware; it is not quantum computation.
9
+
10
+ ## When to use
11
+
12
+ Use this skill when a problem is difficult, ambiguous, high-stakes, multi-step, or vulnerable to early commitment: mathematics, scientific reasoning, debugging, planning, architecture, model selection, hypothesis testing, and technical diagnosis.
13
+
14
+ Do not add branching overhead to trivial questions.
15
+
16
+ ## Core protocol
17
+
18
+ ### 1. Frame the state
19
+
20
+ Identify the objective, constraints, verified facts, assumptions, unknowns, and available tools. Keep facts separate from hypotheses.
21
+
22
+ ### 2. Open a diverse possibility set
23
+
24
+ Create several materially different candidate paths. Diversity is mandatory: do not count paraphrases, cosmetic variants, or branches that depend on the same hidden assumption as independent possibilities.
25
+
26
+ Prefer orthogonal strategy classes when applicable, such as:
27
+
28
+ - direct derivation
29
+ - counterexample / falsification
30
+ - decomposition
31
+ - alternative model or mechanism
32
+ - numerical or executable test
33
+ - independent reconstruction
34
+ - adversarial critique
35
+
36
+ ### 3. Maintain branch states
37
+
38
+ Treat each candidate as one of:
39
+
40
+ - **active** — worth more compute now
41
+ - **dormant** — currently weak but potentially recoverable
42
+ - **rejected** — contradicted or decisively invalidated
43
+
44
+ Never permanently reject a branch merely because another branch initially looks more plausible.
45
+
46
+ ### 4. Evaluate each branch independently
47
+
48
+ Judge branches using evidence, not eloquence. Consider:
49
+
50
+ - empirical or logical support
51
+ - external verification
52
+ - independence from other branches
53
+ - information gain
54
+ - contradiction count and severity
55
+ - unresolved assumptions
56
+ - expected value of another reasoning step
57
+ - compute / tool cost
58
+
59
+ Self-reported model confidence alone is weak evidence.
60
+
61
+ ### 5. Use tools to falsify, not merely confirm
62
+
63
+ When available, route claims through the strongest suitable verifier: executable code, numerical tests, formal solvers, source lookup, direct file inspection, unit tests, symbolic algebra, or an independent model pass.
64
+
65
+ Actively search for evidence that would disprove leading branches.
66
+
67
+ ### 6. Apply interference-like comparison
68
+
69
+ Compare branches against each other without pretending to perform quantum interference.
70
+
71
+ - Reinforce conclusions independently reached by different verified paths.
72
+ - Penalize branches that conflict with verified facts.
73
+ - Detect shared assumptions so correlated branches are not double-counted.
74
+ - Transfer newly verified facts to every branch they constrain.
75
+ - Preserve meaningful disagreement instead of averaging incompatible claims.
76
+
77
+ ### 7. Allocate compute dynamically
78
+
79
+ Use uncertainty to control width and depth.
80
+
81
+ - **Low uncertainty:** narrow quickly and deepen the strongest path.
82
+ - **High uncertainty:** widen the possibility set.
83
+ - **High-value unresolved branch:** spend additional tool or reasoning budget.
84
+ - **Redundant or contradicted branch:** move to dormant or rejected.
85
+
86
+ Branch count is adaptive, not fixed.
87
+
88
+ ### 8. Permit branch revival
89
+
90
+ If new evidence invalidates a leading assumption or materially improves a dormant branch, restore that branch to active status and re-evaluate the ranking.
91
+
92
+ ### 9. Stop when more search has low value
93
+
94
+ Collapse the possibility set when one result is sufficiently verified, remaining branches are redundant or falsified, or additional search is unlikely to change the decision enough to justify its cost.
95
+
96
+ If multiple branches remain genuinely unresolved, report the alternatives and what evidence would distinguish them.
97
+
98
+ ### 10. Produce the answer
99
+
100
+ Synthesize only from the strongest supported surviving branches. Run a final contradiction and requirement check before answering.
101
+
102
+ Do not expose hidden chain-of-thought. Return the conclusion, concise supporting evidence, important uncertainty, and materially different surviving alternatives when necessary.
103
+
104
+ ## Adaptive intensity
105
+
106
+ Use a rough internal intensity level:
107
+
108
+ - **light:** 2–3 distinct candidates, one comparison pass
109
+ - **standard:** 4–8 candidates, verification + pruning + one revival check
110
+ - **deep:** 8+ candidates only when the problem and compute budget justify it; use repeated verification and adaptive expansion
111
+
112
+ Do not mechanically maximize branch count. The goal is better search of the possibility space, not more text.
113
+
114
+ ## Operational reference policy
115
+
116
+ When the host system can track aggregate branch metadata, use explicit normalized values in `[0, 1]` for evidence, verification, independence, information gain, contradiction severity, unresolved assumptions, compute cost, shared-assumption overlap, and semantic redundancy.
117
+
118
+ A provider-agnostic reference controller is available in [`reference/branch_controller.py`](./reference/branch_controller.py). Its defaults make the qualitative protocol testable:
119
+
120
+ - independence is reduced when branches share critical assumptions or are near-duplicates
121
+ - strong contradiction can reject a branch
122
+ - weak branches become dormant instead of disappearing
123
+ - dormant branches can revive after a material evidence, verification, or contradiction change
124
+ - uncertainty controls a recommended search width
125
+ - collapse requires a verified leader, low contradiction, and a meaningful margin over surviving alternatives
126
+
127
+ The exact scoring weights and thresholds are **reference defaults, not validated constants**. They must be calibrated against benchmark evidence rather than treated as universal truths. See [`docs/MEASUREMENT.md`](./docs/MEASUREMENT.md) for the auditable formulas and thresholds.
128
+
129
+ Hosts that cannot expose these aggregate measurements should still follow the qualitative protocol above and must not invent numeric telemetry.
130
+
131
+ ## Benchmark requirement
132
+
133
+ Do not claim that this skill improves reasoning merely because the protocol sounds plausible. Compare the same model and task configuration with and without the skill and measure at least accuracy, compute/token cost, latency, error recovery, contradiction resolution, and branch diversity when observable.
134
+
135
+ The reproducible benchmark schema and evaluator live under [`benchmark/`](./benchmark/README.md).
136
+
137
+ ## Non-negotiable rules
138
+
139
+ 1. Do not call this real quantum computation.
140
+ 2. Do not treat repeated wording as independent evidence.
141
+ 3. Do not let one branch evaluate itself without challenge when verification is possible.
142
+ 4. Prefer falsification and executable checks over intuition.
143
+ 5. Preserve a recoverable record of high-information dormant branches.
144
+ 6. Collapse only after cross-branch comparison.
145
+ 7. Never fabricate evidence, tool results, sources, certainty, or branch telemetry.
146
+ 8. Do not present reference thresholds as empirically validated until benchmark evidence exists.
package/VERSION CHANGED
@@ -1 +1 @@
1
- 0.3.1
1
+ 1.1.1
package/index.js CHANGED
@@ -1 +1,207 @@
1
- module.exports = { name: 'quantum-reasoning-skill', version: '1.0.0', description: 'Quantum-inspired reasoning skill for exploring multiple solution paths', skill: require('./SKILL.md') };
1
+ #!/usr/bin/env node
2
+ 'use strict';
3
+
4
+ /**
5
+ * quantum-reasoning-skill
6
+ *
7
+ * A real, executable entry point for the Node/MCP host ecosystem. It reads the
8
+ * skill contract that ships with this package and exposes it programmatically
9
+ * and from the command line.
10
+ *
11
+ * The previous implementation did `require('./SKILL.md')`, which made Node
12
+ * parse Markdown as JavaScript and throw a SyntaxError on every import.
13
+ * SKILL.md is now read as text via fs.readFileSync.
14
+ */
15
+
16
+ const fs = require('fs');
17
+ const path = require('path');
18
+
19
+ const REQUIRED_FIELDS = ['name', 'description'];
20
+
21
+ const SKILL_FILE = path.join(__dirname, 'SKILL.md');
22
+ const VERSION_FILE = path.join(__dirname, 'VERSION');
23
+
24
+ /**
25
+ * Parse leading `---` delimited front matter from the skill document.
26
+ * Deliberately dependency-free: the contract only uses `key: value` pairs.
27
+ *
28
+ * @param {string} text
29
+ * @returns {{fields: Record<string,string>, fenceEnd: number}}
30
+ */
31
+ function parseFrontMatter(text) {
32
+ const lines = text.split(/\r?\n/);
33
+ if (lines.length === 0 || lines[0].trim() !== '---') {
34
+ throw new Error('SKILL.md must start with a `---` front matter fence');
35
+ }
36
+ let fenceEnd = -1;
37
+ for (let i = 1; i < lines.length; i += 1) {
38
+ if (lines[i].trim() === '---') {
39
+ fenceEnd = i;
40
+ break;
41
+ }
42
+ }
43
+ if (fenceEnd === -1) {
44
+ throw new Error('SKILL.md front matter is not closed');
45
+ }
46
+ const fields = Object.create(null);
47
+ for (const line of lines.slice(1, fenceEnd)) {
48
+ const trimmed = line.trim();
49
+ if (trimmed === '' || trimmed.startsWith('#')) continue;
50
+ const idx = trimmed.indexOf(':');
51
+ if (idx === -1) continue;
52
+ const key = trimmed.slice(0, idx).trim();
53
+ let value = trimmed.slice(idx + 1).trim();
54
+ if (
55
+ (value.startsWith('"') && value.endsWith('"')) ||
56
+ (value.startsWith("'") && value.endsWith("'"))
57
+ ) {
58
+ value = value.slice(1, -1);
59
+ }
60
+ fields[key] = value;
61
+ }
62
+ return { fields, fenceEnd };
63
+ }
64
+
65
+ /**
66
+ * Read the packaged skill contract from disk.
67
+ *
68
+ * @returns {string}
69
+ */
70
+ function readSkillDocument() {
71
+ return fs.readFileSync(SKILL_FILE, 'utf8');
72
+ }
73
+
74
+ /**
75
+ * Read the packaged VERSION file.
76
+ *
77
+ * @returns {string}
78
+ */
79
+ function readVersion() {
80
+ const raw = fs.readFileSync(VERSION_FILE, 'utf8').trim();
81
+ return raw.startsWith('v') ? raw.slice(1) : raw;
82
+ }
83
+
84
+ /**
85
+ * Validate the skill contract.
86
+ *
87
+ * @param {string} text
88
+ * @returns {{name: string, description: string, descriptionLength: number,
89
+ * bodyLines: number, valid: boolean, problems: string[]}}
90
+ */
91
+ function validateSkill(text) {
92
+ const { fields, fenceEnd } = parseFrontMatter(text);
93
+ const problems = [];
94
+ for (const field of REQUIRED_FIELDS) {
95
+ if (!fields[field]) {
96
+ problems.push(`front matter must define a non-empty \`${field}\``);
97
+ }
98
+ }
99
+ if (fields.name && fields.name !== 'quantum-reasoning') {
100
+ problems.push('front matter `name` must be `quantum-reasoning`');
101
+ }
102
+ const bodyLines = text.split(/\r?\n/).slice(fenceEnd + 1).length;
103
+ if (bodyLines < 20) {
104
+ problems.push(`SKILL.md body is too short (${bodyLines} lines, expected >= 20)`);
105
+ }
106
+ return {
107
+ name: fields.name || '',
108
+ description: fields.description || '',
109
+ descriptionLength: (fields.description || '').length,
110
+ bodyLines,
111
+ valid: problems.length === 0,
112
+ problems,
113
+ };
114
+ }
115
+
116
+ /**
117
+ * Load and validate the packaged skill in one call.
118
+ *
119
+ * @returns {object} the parsed skill definition
120
+ */
121
+ function loadSkill() {
122
+ const document = readSkillDocument();
123
+ const { fields } = parseFrontMatter(document);
124
+ const report = validateSkill(document);
125
+ return {
126
+ name: report.name,
127
+ description: report.description,
128
+ version: readVersion(),
129
+ document,
130
+ valid: report.valid,
131
+ problems: report.problems,
132
+ metadata: fields,
133
+ };
134
+ }
135
+
136
+ const USAGE = `quantum-reasoning — inspect the packaged skill contract
137
+
138
+ Usage:
139
+ quantum-reasoning Print the skill summary
140
+ quantum-reasoning --json Print the full report as JSON
141
+ quantum-reasoning --validate Exit non-zero if the contract is invalid
142
+ quantum-reasoning --path Print the absolute SKILL.md path
143
+ quantum-reasoning --help Show this message
144
+
145
+ Exit codes:
146
+ 0 success
147
+ 1 invalid skill contract (--validate)
148
+ 2 SKILL.md could not be read
149
+ `;
150
+
151
+ function main(argv) {
152
+ const args = argv.slice(2);
153
+ const wantsJson = args.includes('--json');
154
+ const wantsValidate = args.includes('--validate');
155
+ const wantsPath = args.includes('--path');
156
+
157
+ if (args.includes('--help') || args.includes('-h')) {
158
+ process.stdout.write(USAGE);
159
+ return 0;
160
+ }
161
+
162
+ let report;
163
+ try {
164
+ report = validateSkill(readSkillDocument());
165
+ } catch (error) {
166
+ process.stderr.write(`error: ${error.message}\n`);
167
+ return 2;
168
+ }
169
+ report.version = readVersion();
170
+ report.path = SKILL_FILE;
171
+
172
+ if (wantsPath) {
173
+ process.stdout.write(`${SKILL_FILE}\n`);
174
+ return 0;
175
+ }
176
+ if (wantsJson) {
177
+ process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
178
+ } else {
179
+ process.stdout.write(`skill: ${report.name || '<missing name>'}\n`);
180
+ process.stdout.write(`status: ${report.valid ? 'valid' : 'INVALID'}\n`);
181
+ process.stdout.write(`file: ${SKILL_FILE}\n`);
182
+ process.stdout.write(`version: ${report.version}\n`);
183
+ process.stdout.write(`body: ${report.bodyLines} lines\n`);
184
+ for (const problem of report.problems) {
185
+ process.stdout.write(` - ${problem}\n`);
186
+ }
187
+ }
188
+ if (wantsValidate && !report.valid) {
189
+ return 1;
190
+ }
191
+ return 0;
192
+ }
193
+
194
+ module.exports = {
195
+ name: 'quantum-reasoning-skill',
196
+ version: readVersion(),
197
+ skillFile: SKILL_FILE,
198
+ parseFrontMatter,
199
+ readSkillDocument,
200
+ readVersion,
201
+ validateSkill,
202
+ loadSkill,
203
+ };
204
+
205
+ if (require.main === module) {
206
+ process.exitCode = main(process.argv);
207
+ }
package/package.json CHANGED
@@ -1 +1,41 @@
1
- {"name":"quantum-reasoning-skill","version":"1.0.0","description":"Quantum-inspired reasoning skill for exploring multiple solution paths","main":"index.js","scripts":{"test":"echo \"Error: no test specified\" && exit 1"},"keywords":["ai-agents","reasoning","quantum-inspired","multi-branch"],"author":"Furox-Art","license":"MIT","repository":{"type":"git","url":"git+https://github.com/Furox-Art/quantum-reasoning-skill.git"},"bugs":{"url":"https://github.com/Furox-Art/quantum-reasoning-skill/issues"},"homepage":"https://github.com/Furox-Art/quantum-reasoning-skill#readme"}
1
+ {
2
+ "name": "quantum-reasoning-skill",
3
+ "version": "1.1.1",
4
+ "description": "Quantum-inspired reasoning skill for exploring multiple solution paths",
5
+ "main": "index.js",
6
+ "bin": {
7
+ "quantum-reasoning": "index.js"
8
+ },
9
+ "files": [
10
+ "index.js",
11
+ "SKILL.md",
12
+ "VERSION",
13
+ "LICENSE",
14
+ "README.md"
15
+ ],
16
+ "scripts": {
17
+ "test": "node index.js --validate && node -e \"const s=require('./index.js'); if(!s.loadSkill().valid){console.error('skill contract invalid');process.exit(1)}\""
18
+ },
19
+ "engines": {
20
+ "node": ">=20"
21
+ },
22
+ "keywords": [
23
+ "ai-agents",
24
+ "reasoning",
25
+ "quantum-inspired",
26
+ "multi-branch",
27
+ "agent-skills",
28
+ "mcp",
29
+ "claude-skill"
30
+ ],
31
+ "author": "Furox-Art",
32
+ "license": "MIT",
33
+ "repository": {
34
+ "type": "git",
35
+ "url": "git+https://github.com/Furox-Art/quantum-reasoning-skill.git"
36
+ },
37
+ "bugs": {
38
+ "url": "https://github.com/Furox-Art/quantum-reasoning-skill/issues"
39
+ },
40
+ "homepage": "https://github.com/Furox-Art/quantum-reasoning-skill#readme"
41
+ }
@@ -1,81 +0,0 @@
1
- name: Benchmark result
2
- description: Submit an independently run baseline-vs-skill model evaluation
3
- title: "[Benchmark] "
4
- labels: []
5
- body:
6
- - type: markdown
7
- attributes:
8
- value: |
9
- Submit reproducible measurements only. Do not include private chain-of-thought. For PR submissions, validate the full bundle with `python benchmark/validate_submission.py --root benchmark/results/community`.
10
-
11
- - type: input
12
- id: provider
13
- attributes:
14
- label: Provider
15
- placeholder: Provider or local runtime
16
- validations:
17
- required: true
18
-
19
- - type: input
20
- id: model
21
- attributes:
22
- label: Exact model / version
23
- placeholder: Exact model identifier and version if available
24
- validations:
25
- required: true
26
-
27
- - type: input
28
- id: skill_version
29
- attributes:
30
- label: Skill version / commit
31
- placeholder: v0.3.1 or commit SHA
32
- validations:
33
- required: true
34
-
35
- - type: input
36
- id: run_date
37
- attributes:
38
- label: Run date
39
- placeholder: YYYY-MM-DD
40
- validations:
41
- required: true
42
-
43
- - type: textarea
44
- id: controls
45
- attributes:
46
- label: Experimental controls
47
- description: Describe sampling settings, context/output limits, tools, repetitions, seeds, prompt wrapper and hardware/runtime when relevant.
48
- validations:
49
- required: true
50
-
51
- - type: textarea
52
- id: artifacts
53
- attributes:
54
- label: Result artifacts
55
- description: Link or attach metadata.json, cases.jsonl, baseline.jsonl, skill.jsonl, comparison.json and README.md. State whether the bundle passed validate_submission.py.
56
- validations:
57
- required: true
58
-
59
- - type: textarea
60
- id: summary
61
- attributes:
62
- label: Result summary
63
- description: Report accuracy and compute/cost trade-offs. Include failures, refusals and timeouts.
64
- validations:
65
- required: true
66
-
67
- - type: checkboxes
68
- id: integrity
69
- attributes:
70
- label: Integrity checks
71
- options:
72
- - label: Baseline and skill used the same model/version, task set, tools, sampling settings and budget policy.
73
- required: true
74
- - label: I included the exact evaluated cases and did not remove failed runs, refusals or timeouts.
75
- required: true
76
- - label: comparison.json was generated from the submitted raw files and was not edited to improve the result.
77
- required: true
78
- - label: I did not include private chain-of-thought or fabricate unavailable telemetry.
79
- required: true
80
- - label: I disclosed modifications to SKILL.md, benchmark cases, wrappers or provider adapters.
81
- required: true
@@ -1,38 +0,0 @@
1
- name: Bug report
2
- description: Report a reproducible defect in the skill, reference controller, benchmark tooling, or CI.
3
- title: "[Bug]: "
4
- labels: ["bug"]
5
- body:
6
- - type: textarea
7
- id: summary
8
- attributes:
9
- label: What happened?
10
- description: Describe the observed behavior and what you expected instead.
11
- validations:
12
- required: true
13
- - type: textarea
14
- id: reproduce
15
- attributes:
16
- label: Minimal reproduction
17
- description: Include commands, inputs, or a small example that reproduces the problem.
18
- validations:
19
- required: true
20
- - type: input
21
- id: version
22
- attributes:
23
- label: Repository version or commit
24
- placeholder: v0.3.0 or commit SHA
25
- validations:
26
- required: true
27
- - type: textarea
28
- id: environment
29
- attributes:
30
- label: Environment
31
- description: Python version, host/agent integration, OS, and relevant tool versions.
32
- - type: checkboxes
33
- id: sensitive
34
- attributes:
35
- label: Safety check
36
- options:
37
- - label: I have not included secrets, credentials, private chain-of-thought, or sensitive benchmark data.
38
- required: true