quantum-reasoning-skill 1.0.0

package/docs/MEASUREMENT.md

@@ -0,0 +1,131 @@
+# Measurement and branch-control specification
+
+This document turns the qualitative protocol in `SKILL.md` into an auditable reference policy. It does **not** claim that these constants are optimal or scientifically validated. They are initial defaults that must be calibrated against benchmark data.
+
+## 1. Observable branch metrics
+
+Each branch is represented by normalized values in `[0, 1]`:
+
+- `evidence`: support from verified facts or observations
+- `verification`: strength of independent checks
+- `independence`: independence from other branches
+- `information_gain`: expected value of further work on the branch
+- `contradiction`: severity of conflict with verified facts
+- `unresolved_assumptions`: remaining unsupported assumptions
+- `normalized_cost`: relative compute/tool cost already consumed
+- `shared_assumption_ratio`: overlap in critical assumptions with the current leader
+- `semantic_similarity_to_leader`: redundancy signal for near-duplicate branches
+
+These values are control metadata. They must not require exposing private chain-of-thought.
+
+## 2. Correlation penalty
+
+A branch should not receive full independence credit when it shares assumptions or is effectively a paraphrase of the leader.
+
+```text
+correlation = max(shared_assumption_ratio, semantic_similarity_to_leader)
+effective_independence = independence * (1 - correlation)
+```
+
+## 3. Reference support score
+
+The reference implementation uses:
+
+```text
+score = clamp01(
+    0.15
+  + 0.25 * evidence
+  + 0.25 * verification
+  + 0.15 * effective_independence
+  + 0.10 * information_gain
+  - 0.15 * contradiction
+  - 0.05 * unresolved_assumptions
+  - 0.05 * normalized_cost
+)
+```
+
+The weights are deliberately explicit so they can be benchmarked and changed. They are not hidden policy.
+
+## 4. State transitions
+
+Reference defaults:
+
+- reject when `contradiction >= 0.85`
+- move to dormant when `score < 0.35`
+- otherwise keep active
+
+A dormant branch is revived when at least one material change occurs relative to its previous measurement:
+
+- evidence increases by `>= 0.15`, or
+- verification increases by `>= 0.25`, or
+- contradiction decreases by `>= 0.20`
+
+A rejected branch is not automatically revived by the reference controller. A caller may explicitly reopen it if the supposedly verified fact that caused rejection is itself invalidated.
+
+## 5. Dynamic search width
+
+Uncertainty is estimated from the normalized Shannon entropy of surviving branch scores.
+
+Reference width policy:
+
+| uncertainty | branch range |
+|---|---:|
+| `< 0.25` | 2–3 |
+| `0.25–0.55` | 4–6 |
+| `>= 0.55` | 6–10 |
+
+This is a compute-allocation recommendation, not a requirement to generate the maximum branch count.
+
+## 6. Collapse criteria
+
+The reference controller permits collapse only when all applicable checks pass:
+
+1. leader score `>= 0.78`
+2. leader verification `>= 0.75`
+3. leader contradiction `<= 0.15`
+4. if a runner-up exists, leader margin `>= 0.12`
+5. no unresolved high-information alternative remains close enough to plausibly change the outcome
+
+If these checks fail, the system should continue verification, keep multiple surviving alternatives, or report unresolved alternatives rather than manufacture certainty.
+
+## 7. Diversity diagnostic
+
+Given pairwise semantic similarities, a pair is treated as redundant when similarity is `>= 0.85` by default.
+
+```text
+diversity_ratio = materially_distinct_pairs / all_measured_pairs
+```
+
+This metric is intentionally simple. Embedding model choice, similarity calibration and domain effects must be reported by any experiment that uses it.
+
+## 8. What must be calibrated
+
+Benchmarking should test at least:
+
+- score weights
+- contradiction rejection threshold
+- dormant threshold
+- revival deltas
+- collapse score and margin
+- semantic duplicate threshold
+- uncertainty-to-width mapping
+
+Calibration must use a development set separate from final held-out evaluation.
+
+## 9. Failure modes to measure
+
+A serious evaluation should label and count:
+
+- early lock-in
+- duplicated branches
+- shared-assumption failure
+- false contradiction detection
+- failure to revive a correct dormant branch
+- excessive search after the answer is already established
+- premature collapse
+- tool-confirmation bias
+- accuracy gain that is explained only by substantially higher compute
+
+## 10. Reproducibility rule
+
+Any reported improvement must include the baseline configuration and raw aggregate result files needed to reproduce the comparison. Do not publish only a selected percentage or a hand-picked example.

package/examples/host-integration.md

@@ -0,0 +1,57 @@
+# Host integration examples
+
+These examples describe integration shapes without depending on a specific provider.
+
+## Native skill host
+
+Install the repository or copy `SKILL.md` into the host's skill directory. Preserve the YAML front matter and load the file as a persistent skill instruction.
+
+## Persistent instruction host
+
+Conceptually, the host should build the request as:
+
+```text
+persistent instructions:
+  <contents of SKILL.md>
+
+task:
+  <user request>
+```
+
+Do not require the model to print private reasoning. The final answer may remain concise while the host records only observable aggregate telemetry.
+
+## Controller-assisted host
+
+A host-side agent loop may use the optional reference controller:
+
+```python
+from reference.branch_controller import Branch, BranchMetrics, collapse_decision
+
+branch = Branch(
+    branch_id="candidate-a",
+    metrics=BranchMetrics(
+        evidence=0.8,
+        verification=0.9,
+        independence=0.7,
+        information_gain=0.6,
+        contradiction=0.1,
+        unresolved_assumptions=0.2,
+        normalized_cost=0.3,
+        shared_assumption_ratio=0.1,
+        semantic_similarity_to_leader=0.2,
+    ),
+)
+```
+
+The host supplies only measurements it can actually observe. Missing telemetry must not be guessed.
+
+## Benchmark wrapper
+
+For an A/B benchmark, keep everything fixed except the presence of the skill:
+
+```text
+baseline = normal persistent instructions + task
+skill    = normal persistent instructions + SKILL.md + same task
+```
+
+Use the same model/version, tools, sampling settings and budget policy for both conditions. See [`../benchmark/README.md`](../benchmark/README.md).

package/examples/usage.md

@@ -0,0 +1,31 @@
+# Usage examples
+
+The skill is designed to be invoked only when branching improves reliability.
+
+## Scientific hypothesis
+
+```text
+Use the quantum-reasoning skill. Keep several materially different mechanisms alive, identify what evidence would falsify each one, use available tools to test them, and collapse only after cross-comparison.
+```
+
+## Mathematics
+
+```text
+Use the quantum-reasoning skill. Explore independent solution strategies, test edge cases and counterexamples, verify the strongest path, and return only the final derivation and important uncertainty.
+```
+
+## Debugging
+
+```text
+Use the quantum-reasoning skill. Maintain multiple root-cause hypotheses, design the cheapest discriminating test for each, eliminate contradicted causes, revive alternatives if new evidence conflicts with the leader, then give the most supported diagnosis.
+```
+
+## Planning / architecture
+
+```text
+Use the quantum-reasoning skill. Generate genuinely different architectures, compare assumptions and failure modes, allocate more analysis to unresolved high-impact choices, and synthesize the strongest design only after verification.
+```
+
+## Expected behavior
+
+The model should not print a large internal reasoning tree. It should use the branching protocol internally and return a concise result, the evidence that matters, remaining uncertainty, and genuinely unresolved alternatives when necessary.

package/index.js

@@ -0,0 +1 @@
+module.exports = { name: 'quantum-reasoning-skill', version: '1.0.0', description: 'Quantum-inspired reasoning skill for exploring multiple solution paths', skill: require('./SKILL.md') }; 

package/package.json

@@ -0,0 +1 @@
+{"name":"quantum-reasoning-skill","version":"1.0.0","description":"Quantum-inspired reasoning skill for exploring multiple solution paths","main":"index.js","scripts":{"test":"echo \"Error: no test specified\" && exit 1"},"keywords":["ai-agents","reasoning","quantum-inspired","multi-branch"],"author":"Furox-Art","license":"MIT","repository":{"type":"git","url":"git+https://github.com/Furox-Art/quantum-reasoning-skill.git"},"bugs":{"url":"https://github.com/Furox-Art/quantum-reasoning-skill/issues"},"homepage":"https://github.com/Furox-Art/quantum-reasoning-skill#readme"} 

package/reference/branch_controller.py

@@ -0,0 +1,256 @@
+"""Reference branch-control logic for the Quantum Reasoning Skill.
+
+This module is intentionally provider-agnostic. It does not generate hidden reasoning
+or call an LLM. It only manages branch telemetry and makes deterministic decisions
+from explicit, externally supplied measurements.
+
+The thresholds below are reference defaults, not scientifically validated constants.
+They are meant to be calibrated with benchmark evidence.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, replace
+from enum import Enum
+from math import log2
+from typing import Iterable, Sequence
+
+
+class BranchState(str, Enum):
+    ACTIVE = "active"
+    DORMANT = "dormant"
+    REJECTED = "rejected"
+
+
+@dataclass(frozen=True)
+class BranchMetrics:
+    evidence: float
+    verification: float
+    independence: float
+    information_gain: float
+    contradiction: float
+    unresolved_assumptions: float
+    normalized_cost: float
+    shared_assumption_ratio: float = 0.0
+    semantic_similarity_to_leader: float = 0.0
+
+    def validate(self) -> None:
+        for name, value in self.__dict__.items():
+            if not 0.0 <= value <= 1.0:
+                raise ValueError(f"{name} must be in [0, 1], got {value!r}")
+
+
+@dataclass(frozen=True)
+class Branch:
+    branch_id: str
+    metrics: BranchMetrics
+    state: BranchState = BranchState.ACTIVE
+    previous_metrics: BranchMetrics | None = None
+
+
+@dataclass(frozen=True)
+class Thresholds:
+    collapse_score: float = 0.78
+    collapse_verification: float = 0.75
+    collapse_max_contradiction: float = 0.15
+    collapse_margin: float = 0.12
+    unresolved_near_leader_margin: float = 0.10
+    reject_contradiction: float = 0.85
+    dormant_score: float = 0.35
+    revive_evidence_delta: float = 0.15
+    revive_verification_delta: float = 0.25
+    revive_contradiction_drop: float = 0.20
+
+
+DEFAULT_THRESHOLDS = Thresholds()
+
+
+WEIGHTS = {
+    "evidence": 0.25,
+    "verification": 0.25,
+    "independence": 0.15,
+    "information_gain": 0.10,
+    "contradiction": -0.15,
+    "unresolved_assumptions": -0.05,
+    "normalized_cost": -0.05,
+}
+
+
+def _clamp01(value: float) -> float:
+    return max(0.0, min(1.0, value))
+
+
+def effective_independence(metrics: BranchMetrics) -> float:
+    """Penalize correlated branches and near-duplicates.
+
+    The effective independence is the declared independence reduced by the strongest
+    observed correlation signal: shared assumptions or semantic similarity.
+    """
+    metrics.validate()
+    correlation = max(
+        metrics.shared_assumption_ratio,
+        metrics.semantic_similarity_to_leader,
+    )
+    return _clamp01(metrics.independence * (1.0 - correlation))
+
+
+def branch_score(metrics: BranchMetrics) -> float:
+    """Return a deterministic support score in [0, 1]."""
+    metrics.validate()
+    values = {
+        "evidence": metrics.evidence,
+        "verification": metrics.verification,
+        "independence": effective_independence(metrics),
+        "information_gain": metrics.information_gain,
+        "contradiction": metrics.contradiction,
+        "unresolved_assumptions": metrics.unresolved_assumptions,
+        "normalized_cost": metrics.normalized_cost,
+    }
+    raw = 0.15 + sum(WEIGHTS[name] * value for name, value in values.items())
+    return _clamp01(raw)
+
+
+def rank_branches(branches: Iterable[Branch]) -> list[Branch]:
+    return sorted(
+        (branch for branch in branches if branch.state != BranchState.REJECTED),
+        key=lambda branch: branch_score(branch.metrics),
+        reverse=True,
+    )
+
+
+def classify_branch(
+    branch: Branch,
+    thresholds: Thresholds = DEFAULT_THRESHOLDS,
+) -> Branch:
+    """Move an active branch to dormant/rejected when evidence warrants it."""
+    score = branch_score(branch.metrics)
+    if branch.metrics.contradiction >= thresholds.reject_contradiction:
+        return replace(branch, state=BranchState.REJECTED)
+    if score < thresholds.dormant_score:
+        return replace(branch, state=BranchState.DORMANT)
+    return replace(branch, state=BranchState.ACTIVE)
+
+
+def should_revive(
+    branch: Branch,
+    thresholds: Thresholds = DEFAULT_THRESHOLDS,
+) -> bool:
+    """Return True when new evidence materially improves a dormant branch."""
+    if branch.state != BranchState.DORMANT or branch.previous_metrics is None:
+        return False
+    current = branch.metrics
+    previous = branch.previous_metrics
+    current.validate()
+    previous.validate()
+    return any(
+        (
+            current.evidence - previous.evidence >= thresholds.revive_evidence_delta,
+            current.verification - previous.verification
+            >= thresholds.revive_verification_delta,
+            previous.contradiction - current.contradiction
+            >= thresholds.revive_contradiction_drop,
+        )
+    )
+
+
+def update_branch_state(
+    branch: Branch,
+    thresholds: Thresholds = DEFAULT_THRESHOLDS,
+) -> Branch:
+    if should_revive(branch, thresholds):
+        return replace(branch, state=BranchState.ACTIVE)
+    return classify_branch(branch, thresholds)
+
+
+def normalized_entropy(probabilities: Sequence[float]) -> float:
+    """Normalized Shannon entropy in [0, 1]."""
+    if not probabilities:
+        return 0.0
+    if any(value < 0 for value in probabilities):
+        raise ValueError("probabilities cannot contain negative values")
+    total = sum(probabilities)
+    if total <= 0:
+        return 0.0
+    normalized = [value / total for value in probabilities if value > 0]
+    if len(normalized) <= 1:
+        return 0.0
+    entropy = -sum(value * log2(value) for value in normalized)
+    return _clamp01(entropy / log2(len(normalized)))
+
+
+def uncertainty_from_branches(branches: Sequence[Branch]) -> float:
+    """Estimate uncertainty from the score distribution of surviving branches."""
+    surviving = [
+        branch_score(branch.metrics)
+        for branch in branches
+        if branch.state != BranchState.REJECTED
+    ]
+    return normalized_entropy(surviving)
+
+
+def recommended_width(uncertainty: float) -> tuple[int, int]:
+    """Return a reference branch-count range for the next search step."""
+    if not 0.0 <= uncertainty <= 1.0:
+        raise ValueError("uncertainty must be in [0, 1]")
+    if uncertainty < 0.25:
+        return (2, 3)
+    if uncertainty < 0.55:
+        return (4, 6)
+    return (6, 10)
+
+
+def collapse_decision(
+    branches: Sequence[Branch],
+    thresholds: Thresholds = DEFAULT_THRESHOLDS,
+) -> tuple[bool, str, Branch | None]:
+    """Determine whether the search can collapse to a single branch.
+
+    Returns ``(can_collapse, reason, leader)``. Collapse is blocked when a strong,
+    unresolved alternative remains close enough to the leader to matter.
+    """
+    ranked = rank_branches(branches)
+    if not ranked:
+        return (False, "no surviving branch", None)
+
+    leader = ranked[0]
+    leader_score = branch_score(leader.metrics)
+    if leader_score < thresholds.collapse_score:
+        return (False, "leader score below collapse threshold", leader)
+    if leader.metrics.verification < thresholds.collapse_verification:
+        return (False, "leader verification below collapse threshold", leader)
+    if leader.metrics.contradiction > thresholds.collapse_max_contradiction:
+        return (False, "leader contradiction remains too high", leader)
+
+    if len(ranked) > 1:
+        runner_up = ranked[1]
+        runner_score = branch_score(runner_up.metrics)
+        if leader_score - runner_score < thresholds.collapse_margin:
+            return (False, "leader margin is too small", leader)
+        if (
+            runner_up.metrics.information_gain >= 0.60
+            and leader_score - runner_score
+            <= thresholds.unresolved_near_leader_margin
+        ):
+            return (False, "high-information alternative remains unresolved", leader)
+
+    return (True, "collapse criteria satisfied", leader)
+
+
+def diversity_ratio(
+    pairwise_similarities: Sequence[float],
+    duplicate_threshold: float = 0.85,
+) -> float:
+    """Fraction of pairwise comparisons that are materially distinct.
+
+    Similarity values must be in [0, 1]. A pair at or above the duplicate threshold
+    is treated as correlated/redundant for this diagnostic.
+    """
+    if not pairwise_similarities:
+        return 1.0
+    if not 0.0 <= duplicate_threshold <= 1.0:
+        raise ValueError("duplicate_threshold must be in [0, 1]")
+    for value in pairwise_similarities:
+        if not 0.0 <= value <= 1.0:
+            raise ValueError("similarity values must be in [0, 1]")
+    distinct = sum(value < duplicate_threshold for value in pairwise_similarities)
+    return distinct / len(pairwise_similarities)

package/tests/test_benchmark.py

@@ -0,0 +1,120 @@
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+import unittest
+
+
+ROOT = Path(__file__).resolve().parents[1]
+SPEC = importlib.util.spec_from_file_location(
+    "benchmark_evaluate", ROOT / "benchmark" / "evaluate.py"
+)
+assert SPEC and SPEC.loader
+evaluate = importlib.util.module_from_spec(SPEC)
+SPEC.loader.exec_module(evaluate)
+
+
+CASES = [
+    {
+        "id": "c1",
+        "domain": "math",
+        "prompt": "1+1?",
+        "accepted_answers": ["2"],
+    },
+    {
+        "id": "c2",
+        "domain": "logic",
+        "prompt": "yes?",
+        "accepted_answers": ["yes"],
+    },
+]
+
+
+class BenchmarkTests(unittest.TestCase):
+    def test_summary_computes_accuracy_and_telemetry(self):
+        results = [
+            {
+                "case_id": "c1",
+                "answer": "2",
+                "tokens": 100,
+                "tool_calls": 1,
+                "latency_ms": 50,
+                "branches_total": 4,
+                "branches_distinct": 3,
+                "revived_branches": 1,
+                "recovered_errors": 1,
+                "contradictions_found": 2,
+                "contradictions_resolved": 2,
+            },
+            {
+                "case_id": "c2",
+                "answer": "no",
+                "tokens": 200,
+                "tool_calls": 3,
+                "latency_ms": 150,
+                "branches_total": 2,
+                "branches_distinct": 1,
+                "revived_branches": 0,
+                "recovered_errors": 0,
+                "contradictions_found": 1,
+                "contradictions_resolved": 0,
+            },
+        ]
+        summary = evaluate.summarize(CASES, results)
+        self.assertEqual(summary["accuracy"], 0.5)
+        self.assertEqual(summary["mean_tokens"], 150.0)
+        self.assertEqual(summary["mean_tool_calls"], 2.0)
+        self.assertEqual(summary["mean_latency_ms"], 100.0)
+        self.assertEqual(summary["branch_diversity_ratio"], 4 / 6)
+        self.assertEqual(summary["contradiction_resolution_rate"], 2 / 3)
+
+    def test_compare_reports_cost_and_accuracy_changes(self):
+        baseline = {
+            "accuracy": 0.5,
+            "mean_tokens": 100.0,
+            "mean_tool_calls": 1.0,
+            "mean_latency_ms": 100.0,
+            "branch_diversity_ratio": 0.4,
+            "contradiction_resolution_rate": 0.5,
+        }
+        skill = {
+            "accuracy": 0.75,
+            "mean_tokens": 150.0,
+            "mean_tool_calls": 2.0,
+            "mean_latency_ms": 120.0,
+            "branch_diversity_ratio": 0.7,
+            "contradiction_resolution_rate": 0.8,
+        }
+        delta = evaluate.compare(baseline, skill)
+        self.assertEqual(delta["accuracy_delta"], 0.25)
+        self.assertEqual(delta["tokens_percent_change"], 50.0)
+        self.assertEqual(delta["tool_calls_percent_change"], 100.0)
+        self.assertEqual(delta["latency_percent_change"], 20.0)
+        self.assertAlmostEqual(delta["branch_diversity_delta"], 0.3)
+        self.assertAlmostEqual(delta["contradiction_resolution_delta"], 0.3)
+
+    def test_missing_result_fields_fail(self):
+        with self.assertRaises(ValueError):
+            evaluate.summarize(
+                CASES,
+                [{"case_id": "c1", "answer": "2", "tokens": 1}],
+            )
+
+    def test_unknown_case_id_fails(self):
+        with self.assertRaises(ValueError):
+            evaluate.summarize(
+                CASES,
+                [
+                    {
+                        "case_id": "missing",
+                        "answer": "2",
+                        "tokens": 1,
+                        "tool_calls": 0,
+                        "latency_ms": 1,
+                    }
+                ],
+            )
+
+
+if __name__ == "__main__":
+    unittest.main()