questionably-ultrathink 1.0.0

package/README.md

@@ -0,0 +1,259 @@
+# Questionably UltraThink
+
+A Claude Code plugin that integrates **Chain of Verification (CoVe)** and **Atom of Thoughts (AoT)** reasoning frameworks for rigorous, verifiable analysis.
+
+## What It Does
+
+UltraThink enhances Claude's reasoning with two research-backed frameworks:
+
+- **Atom of Thoughts (AoT)** - Decomposes complex problems into atomic sub-questions organized as a DAG, solving them systematically
+- **Chain of Verification (CoVe)** - Verifies factual claims through independent questioning to reduce hallucinations
+
+## Installation
+
+```bash
+# Add the marketplace
+/plugin marketplace add snowmead/questionably-ultrathink
+
+# Install the plugin
+/plugin install questionably-ultrathink@snowmead-marketplace
+```
+
+## Development Setup
+
+For contributors working on this plugin:
+
+```bash
+./setup.sh
+```
+
+This installs dependencies (lefthook, comrak) if missing and sets up git hooks for automatic markdown formatting on commit.
+
+## Commands
+
+### `/questionably-ultrathink`
+
+Run the full reasoning pipeline on a problem:
+
+1. Clarifies intent if needed
+2. Selects analysis rigor (standard/thorough/high-stakes)
+3. Decomposes into atomic questions (AoT) with complexity flagging
+4. Verifies critical atoms in parallel by dependency level (CoVe)
+5. Propagates corrections and recomputes dependent atoms
+6. Synthesizes and verifies final response
+7. Iterates if confidence is below threshold (thorough/high-stakes only)
+
+```
+/questionably-ultrathink analyze whether this authentication approach is secure
+```
+
+### `/decompose`
+
+Break down a complex problem into atomic sub-questions:
+
+```
+/decompose how does React's reconciliation work and compare to Vue?
+```
+
+### `/verify`
+
+Verify factual claims in the most recent response:
+
+```
+/verify
+```
+
+Or verify specific content:
+
+```
+/verify the performance benchmarks mentioned above
+```
+
+## Automatic Activation
+
+The skill automatically activates when you use trigger phrases:
+
+- "be thorough", "analyze carefully", "make sure this is right"
+- "verify", "double-check", "are you sure"
+- Complex multi-part questions
+- Architecture or security decisions
+
+## Rigor Levels
+
+When running the full pipeline, you can select analysis depth:
+
+| Level           | Iterations | Verification              | Confidence Target | Use Case                           |
+| --------------- | ---------- | ------------------------- | ----------------- | ---------------------------------- |
+| **Standard**    | 1          | Flagged atoms only        | N/A               | Most questions                     |
+| **Thorough**    | Up to 2    | Atoms with factual claims | ≥70%              | Important decisions                |
+| **High-Stakes** | Up to 3    | ALL atoms                 | ≥85%              | Security, architecture, production |
+
+## Optional: Parallel.ai MCP Integration
+
+The plugin includes optional MCP servers for enhanced web search during verification:
+
+- `parallel-search` - Optimized fact-checking searches
+- `parallel-task` - Deep research capabilities
+
+**Setup:** Run `/mcp` in Claude Code and authenticate with Parallel.ai to enable them.
+
+**Fallback:** The plugin works fully without MCP authentication, using native `WebSearch` and `WebFetch` tools.
+
+## How It Works
+
+### Architecture
+
+```
+┌───────────────────────────────────────────────────────────────────┐
+│                          User Commands                            │
+│        /questionably-ultrathink  |  /decompose  |  /verify        │
+└─────────────────────────────────┬─────────────────────────────────┘
+                                  │
+                                  ▼
+┌───────────────────────────────────────────────────────────────────┐
+│                       Skill Orchestrator                          │
+│                 (skills/questionably-ultrathink)                  │
+│                                                                   │
+│  1. Clarify intent (AskUserQuestion)                              │
+│  2. Select rigor level                                            │
+│  3. Generate session ID                                           │
+│  4. Invoke agents in sequence                                     │
+│  5. Check for corrections after each verification wave            │
+│  6. Iterate if confidence below threshold                         │
+└───────────┬───────────────────────┬───────────────────┬───────────┘
+            │                       │                   │
+            ▼                       ▼                   ▼
+┌───────────────────┐   ┌───────────────────┐   ┌───────────────────┐
+│   atom-of-        │   │   chain-of-       │   │   aot-recompute   │
+│   thoughts        │   │   verification    │   │                   │
+│                   │   │                   │   │                   │
+│ Decomposes        │   │ Verifies atoms    │   │ Updates atoms     │
+│ problem into      │   │ independently     │   │ after CoV         │
+│ atomic DAG        │   │ (factored exec)   │   │ corrections       │
+└─────────┬─────────┘   └─────────┬─────────┘   └─────────┬─────────┘
+          │                       │                       │
+          └───────────────────────┼───────────────────────┘
+                                  │
+                                  ▼
+┌───────────────────────────────────────────────────────────────────┐
+│              .questionably-ultrathink/{session-id}/               │
+│                   (File-Based Communication)                      │
+│                                                                   │
+│  metadata.md            atoms/              corrections/          │
+│  ├─ session_id          ├─ A1.md            ├─ A1.md (if errors)  │
+│  ├─ rigor               ├─ A2.md            └─ ...                │
+│  ├─ atoms (levels)      ├─ A3.md                                  │
+│  └─ verification_order  └─ FINAL.md                               │
+└───────────────────────────────────────────────────────────────────┘
+```
+
+**Data Flow:**
+
+1. **User invokes command** → Skill orchestrator begins
+2. **Orchestrator → AoT**: Decomposes problem, writes `metadata.md` + atom files
+3. **Orchestrator reads** `metadata.md` to get verification order (atoms grouped by dependency level)
+4. **Orchestrator → CoV**: Verifies atoms at each level (parallel within level)
+5. **CoV writes** correction files if errors found
+6. **Orchestrator checks** for corrections after each wave
+7. **If corrections exist → aot-recompute**: Updates dependent atoms with corrected premises
+8. **Recomputed atoms re-verified** before proceeding to next level
+9. **Final synthesis** combines all verified/corrected atoms
+
+### Atom of Thoughts (AoT)
+
+Based on the paper ["Atom of Thoughts for Markov LLM Test-Time Scaling"](https://arxiv.org/abs/2502.12018) (HKUST, 2025).
+
+Key features:
+
+- Decomposes problems into atomic questions
+- Builds a DAG of dependencies with topological levels
+- Solves independent atoms in parallel
+- Contracts solved atoms into minimal context for dependent atoms
+- Follows Markov property (each step depends only on immediate dependencies)
+- Flags atoms requiring verification (`needs_cov`) based on complexity heuristics
+- Persists reasoning to files for inter-agent communication
+
+### Chain of Verification (CoVe)
+
+Based on the paper ["Chain-of-Verification Reduces Hallucination in LLMs"](https://arxiv.org/abs/2309.11495) (Meta AI, 2023).
+
+Key features:
+
+- Extracts verifiable factual claims
+- Generates targeted verification questions
+- Answers each question **independently** (factored execution)
+- Compares independent answers to original claims
+- Reports inconsistencies with corrections
+- Verifies atoms in parallel by dependency level
+- Writes corrections to disk, triggering recomputation of dependent atoms
+
+## Output Format
+
+### AoT Decomposition
+
+```
+## Atom of Thoughts Decomposition
+
+### Dependency Graph
+- [ATOM:A1] What auth standard fits a stateless API? (level 0, needs_cov: true)
+- [ATOM:A2] Where should tokens be validated? (level 0, needs_cov: false)
+- [ATOM:A3] How should tokens be stored client-side? (level 1, deps: [A1], needs_cov: true)
+- [ATOM:FINAL] Complete auth approach recommendation (level 2, deps: [A2, A3])
+
+### Solutions
+[ATOM:A1] JWT - stateless, self-contained, widely supported
+[ATOM:A2] Middleware layer before route handlers
+...
+
+### Verification Summary
+- [ATOM:A1] needs_cov: true, confidence: high
+- [ATOM:A2] needs_cov: false, confidence: high
+- [ATOM:A3] needs_cov: true, confidence: medium
+```
+
+### CoVe Report
+
+```
+## Chain of Verification Report
+
+### Verification Results
+
+**Claim 1:** "React was released in 2013"
+- Verification Q: When was React first publicly released?
+- Independent Answer: React was released in May 2013 at JSConf US
+- Status: ✓ VERIFIED
+
+**Claim 2:** "Virtual DOM was invented by React"
+- Verification Q: Who invented the virtual DOM concept?
+- Independent Answer: While React popularized it, similar concepts existed earlier
+- Status: ⚠️ INCONSISTENT
+```
+
+## Confidence Markers
+
+After using UltraThink, responses are marked:
+
+- **[VERIFIED]** - Passed CoVe verification
+- **[HIGH CONFIDENCE]** - Decomposed and analyzed systematically
+- **[NEEDS EXTERNAL VERIFICATION]** - User should confirm externally
+- **[UNCERTAIN]** - Flagged areas of doubt remain
+
+## When NOT to Use
+
+Skip UltraThink for:
+
+- Simple, direct questions
+- Opinion or recommendation requests
+- Quick lookups where speed matters
+- Questions you already have high confidence in
+
+## License
+
+MIT
+
+## References
+
+- [Chain-of-Verification Paper](https://arxiv.org/abs/2309.11495) - Meta AI, 2023
+- [Atom of Thoughts Paper](https://arxiv.org/abs/2502.12018) - HKUST, 2025
+- [CoVe Implementation](https://github.com/ritun16/chain-of-verification)
+- [AoT Implementation](https://github.com/qixucen/atom)

package/dist/claude-code/.claude-plugin/marketplace.json

@@ -0,0 +1,17 @@
+{
+  "name": "questionably-ultrathink",
+  "owner": {
+    "name": "snowmead"
+  },
+  "metadata": {
+    "description": "Plugin marketplace for UltraThink reasoning framework integrating Chain of Verification and Atom of Thoughts",
+    "version": "1.0.0"
+  },
+  "plugins": [
+    {
+      "name": "questionably-ultrathink",
+      "description": "Advanced reasoning plugin integrating Chain of Verification (CoVe) and Atom of Thoughts (AoT) frameworks for rigorous, verifiable analysis",
+      "source": "./"
+    }
+  ]
+}

package/dist/claude-code/.claude-plugin/plugin.json

@@ -0,0 +1,11 @@
+{
+  "name": "questionably-ultrathink",
+  "version": "1.0.0",
+  "description": "Advanced reasoning plugin integrating Chain of Verification (CoVe) and Atom of Thoughts (AoT) frameworks for rigorous, verifiable analysis",
+  "author": {
+    "name": "snowmead"
+  },
+  "repository": "https://github.com/snowmead/questionably-ultrathink",
+  "license": "MIT",
+  "keywords": ["reasoning", "verification", "decomposition", "cove", "aot", "analysis"]
+}

package/dist/claude-code/agents/aot-recompute.md

@@ -0,0 +1,175 @@
+---
+name: aot-recompute
+description: |
+  Use this agent to recompute atoms after Chain of Verification finds corrections.
+  This agent reads corrections from disk and updates dependent atoms with corrected premises.
+
+  ## Examples:
+  <example>
+  Context: CoV found an error in atom A1, need to recompute A3 which depends on A1
+  assistant: "I'll use the aot-recompute agent to update the dependent atoms with the correction."
+  </example>
+model: haiku
+tools: [Read, Write, Bash]
+---
+
+# Atom of Thoughts Recomputation Agent
+
+You recompute atoms after Chain of Verification has found corrections. Your job is to update dependent atoms with corrected premises.
+
+\<core\_principle\>
+
+## Correction Propagation
+
+When an upstream atom is corrected, all downstream atoms must be recomputed with the corrected information. You do NOT re-verify—you only recompute the reasoning based on new premises.
+\</core\_principle\>
+
+\<input\_format\>
+
+## Expected Input
+
+Your prompt will contain:
+
+1. **Session ID**: The session directory to work in
+2. **Corrected atoms**: List of atom IDs that were corrected
+3. **Atoms to recompute**: List of downstream atom IDs that depend on corrected atoms
+
+Example prompt:
+
+    Session ID: a1b2c3d4
+    Corrected atoms: [A1]
+    Atoms to recompute: [A3, FINAL]
+
+\</input\_format\>
+
+<process>
+## Your Process
+
+### Step 1: Read Corrections
+
+Read the correction files to understand what changed:
+
+    .questionably-ultrathink/{session-id}/corrections/{atom-id}.md
+
+Each correction file contains:
+
+- Original answer
+- Corrected answer
+- Reason for correction
+
+### Step 2: Read Session Metadata
+
+Read the DAG structure:
+
+    .questionably-ultrathink/{session-id}/metadata.md
+
+Identify the dependency chain to understand which atoms need which corrections.
+
+### Step 3: Read Original Atoms
+
+For each atom to recompute, read its current file:
+
+    .questionably-ultrathink/{session-id}/atoms/{atom-id}.md
+
+### Step 4: Recompute Each Atom
+
+For each atom in topological order (respecting dependencies):
+
+1. **Gather corrected context**: Collect the corrected answers from all dependency atoms
+2. **Re-reason**: Apply the same reasoning process but with corrected premises
+3. **Update the atom file**: Write the new reasoning and answer
+
+### Step 5: Update Metadata
+
+Update the metadata.md file:
+
+- Mark recomputed atoms with `recomputed: true`
+- Update the `verification_order` if any `needs_cov` flags changed
+
+</process>
+
+\<atom\_update\_format\>
+
+## Updated Atom File Format
+
+When recomputing an atom, write the updated file with:
+
+```markdown
+---
+atom_id: {atom-id}
+needs_cov: {true | false}
+confidence: {high | medium | low}
+dependencies: [{dependency atom IDs}]
+recomputed: true
+recomputed_due_to: [{list of corrected atom IDs that triggered this}]
+---
+
+# Atom {atom-id}: {question}
+
+## Correction Context
+- [ATOM:{corrected-id}] was corrected: {old} → {new}
+
+## Sources Consulted
+- {Tool}: {query/path} → {key finding}
+
+## Reasoning Chain
+1. {First observation, using corrected premises}
+2. {Inference or connection made}
+3. {Conclusion drawn}
+
+## Uncertainties
+- {Any gaps, assumptions, or areas of doubt}
+
+## Answer
+{The updated concise atom answer}
+```
+
+\</atom\_update\_format\>
+
+\<output\_format\>
+
+## Output Format
+
+Structure your response as:
+
+    ## Atom Recomputation Report
+
+    ### Session
+    {session-id}
+
+    ### Corrections Applied
+    - [ATOM:A1]: {old answer} → {corrected answer}
+
+    ### Atoms Recomputed
+
+    **[ATOM:A3]** (depends on: A1)
+    - Previous answer: {old}
+    - Updated answer: {new}
+    - Reasoning change: {what changed in the logic}
+
+    **[ATOM:FINAL]** (depends on: A3)
+    - Previous answer: {old}
+    - Updated answer: {new}
+    - Reasoning change: {what changed in the logic}
+
+    ### Files Updated
+    - .questionably-ultrathink/{session-id}/atoms/A3.md
+    - .questionably-ultrathink/{session-id}/atoms/FINAL.md
+    - .questionably-ultrathink/{session-id}/metadata.md
+
+    ### Verification Needs
+    {List any recomputed atoms that now need re-verification}
+    - [ATOM:A3] needs_cov: true (reasoning changed significantly)
+
+\</output\_format\>
+
+<guidelines>
+## Guidelines
+
+1. **Respect topological order** - Recompute atoms in dependency order so each atom has access to corrected upstream answers
+2. **Preserve original reasoning structure** - Only change what the correction necessitates
+3. **Be explicit about what changed** - Document the correction context clearly
+4. **Re-assess needs\_cov** - A recomputed atom may need re-verification if reasoning changed significantly
+5. **Don't expand scope** - Only recompute the atoms you were asked to recompute
+
+</guidelines>