@orangepro/orangepro-mcp 0.1.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +195 -251
- package/dist/local/analyze/analyzer.js +62 -17
- package/dist/local/analyze/treeSitter/engine.js +185 -22
- package/dist/local/autoProve.js +433 -38
- package/dist/local/cli.js +93 -10
- package/dist/local/cliArgs.js +1 -0
- package/dist/local/generate/runHints.js +9 -4
- package/dist/local/graph/factories.js +5 -2
- package/dist/local/ledger.js +1 -1
- package/dist/local/mcp.js +26 -13
- package/dist/local/operations.js +454 -52
- package/dist/local/pack/coverageReport.js +3 -3
- package/dist/local/proofDoctor.js +312 -0
- package/dist/local/rtm.js +40 -10
- package/dist/local/viz/behaviorReportData.js +79 -7
- package/dist/local/viz/behaviorReportHtml.js +526 -615
- package/dist/local/viz/html.js +1 -1
- package/docs/agent-workflow.md +10 -38
- package/docs/agents/claude-code.md +3 -9
- package/docs/agents/codex.md +3 -20
- package/docs/agents/cursor.md +2 -2
- package/docs/agents/opencode.md +2 -2
- package/docs/agents/vscode.md +2 -2
- package/docs/local-proof-kit.md +52 -19
- package/package.json +39 -6
- package/scripts/spikes/go-dynamic-proof-spike.mjs +637 -0
- package/scripts/spikes/go-mutate.go +182 -0
- package/scripts/spikes/java-dynamic-proof-spike.mjs +571 -0
- package/scripts/spikes/java-mutate.mjs +264 -0
- package/scripts/spikes/python-dynamic-proof-spike.mjs +245 -0
- package/scripts/spikes/python-mutate.py +89 -0
package/LICENSE
CHANGED
package/README.md
CHANGED
|
@@ -1,328 +1,272 @@
|
|
|
1
|
-
# OrangePro
|
|
1
|
+
# OrangePro
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**Find the behaviors your tests miss. Generate grounded tests that actually run.**
|
|
4
4
|
|
|
5
|
-
`opro` builds a
|
|
5
|
+
`opro` builds a knowledge graph from your local checkout, maps every behavior in your code, shows which ones are tested and which aren't, and generates integration-level tests grounded in real symbols — not hallucinated imports. Runs as a CLI and an MCP server.
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
> ```
|
|
7
|
+
```bash
|
|
8
|
+
npx @orangepro/mcp-server
|
|
9
|
+
cd /path/to/your/repo
|
|
10
|
+
opro
|
|
11
|
+
```
|
|
13
12
|
|
|
14
|
-
|
|
13
|
+
That's it. You get:
|
|
15
14
|
|
|
16
15
|
```
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
opro score # how test-ready is this repo? (with reasons)
|
|
22
|
-
opro gaps --limit 10 # behaviors with weak / missing tests
|
|
23
|
-
opro generate --base main # PR-scoped: tests ONLY for what the diff touches
|
|
24
|
-
opro generate --pr 1234 # check out PR #1234 (gh) and target its diff
|
|
25
|
-
opro generate --changed # target the current branch's diff vs main/master
|
|
26
|
-
opro generate --single --limit 3 # whole-repo: top-gap behavior, grounded tests
|
|
27
|
-
opro export --out evidence-pack.json --graph-html # metadata-only evidence pack + explorer
|
|
28
|
-
opro mcp # run as an MCP server (stdio)
|
|
16
|
+
.orangepro/
|
|
17
|
+
├── behavior-coverage.html ← open this: interactive gap report
|
|
18
|
+
├── rtm.md ← requirements traceability matrix
|
|
19
|
+
└── evidence-pack.json ← machine-readable metadata export
|
|
29
20
|
```
|
|
30
21
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
## Quickstart — your first Proven
|
|
34
|
-
|
|
35
|
-
**"Proven" counts only a real test that kills a targeted mutant of a behavior.** Two things follow, and they surprise people:
|
|
36
|
-
|
|
37
|
-
- **You need a BYOK model key to reach Proven** — it's what generates the test. `export OPENAI_API_KEY=…` (or `ANTHROPIC_API_KEY=…`). No key → no generated tests → `Proven 0`, and OrangePro tells you so.
|
|
38
|
-
- **`Proven 0` on a fresh `opro analyze` is normal, not broken.** It means no proof has run yet. Static tests and associations show as **Associated signal**, never Proven, until a real test kills a targeted mutant. That's the whole trust model — nothing is labeled Proven on faith.
|
|
39
|
-
|
|
40
|
-
**The intended path is one ask, not a command sequence.** In Cursor / Claude Code / Codex with the OrangePro MCP, tell your agent to cover a behavior — it runs `orangepro_generate_tests`, writes and runs the test, then `orangepro_prove`, and the behavior turns Proven. See [Use with your coding agent](#use-with-your-coding-agent-the-headline-workflow) for the exact setup.
|
|
41
|
-
|
|
42
|
-
Open `.orangepro/behavior-coverage.html` and click any behavior to see its evidence tier and exactly what to do next.
|
|
43
|
-
|
|
44
|
-
## Languages supported
|
|
22
|
+
---
|
|
45
23
|
|
|
46
|
-
|
|
47
|
-
separate behavior-status lanes — never blended into one proof number. Public
|
|
48
|
-
glossary:
|
|
49
|
-
|
|
50
|
-
- **Dynamically Proven** (shown as "Dynamically Proven" in the report, CLI, and RTM) — a dynamic targeted-proof ledger certificate closed for the behavior: baseline green, target mutated, same test failed at an assertion, and the target was not mocked. OrangePro maps the whole repo statically, then dynamically proves only the top few eligible behaviors per run (default 5, raise with `--auto-limit`); everything else stays a static signal, never Proven.
|
|
51
|
-
- **Runtime-covered** — the repo's own coverage tool executed the code. This is actual runtime coverage, but not assertion-level proof.
|
|
52
|
-
- **Static Associated signal** — a name, path, import, or structural match links a test to the code. Useful for triage, but not semantic proof and never counted as Dynamically Proven. The HTML report labels this tier **Statically Linked**.
|
|
53
|
-
- **No integration signal** — no direct static test signal was found yet. This does not mean “untested”; broad integration/e2e coverage may exist without mapping one-to-one to a symbol. In the HTML report this tier is shown as **Reachable Untested** (the symbol appears in a static flow from an entry point) or **No Signal** (no test or static-flow signal yet) — a display-only split that changes no classification.
|
|
54
|
-
|
|
55
|
-
OrangePro can ingest Go coverprofiles, JS/TS `lcov.info`, Python `coverage.xml`
|
|
56
|
-
from coverage.py/pytest-cov, and Java JaCoCo XML. `opro coverage --generate`
|
|
57
|
-
auto-runs Go coverage, repo-defined JS/TS coverage scripts, pytest-cov XML, and
|
|
58
|
-
Maven/Gradle JaCoCo where the repo has the standard local tooling installed.
|
|
59
|
-
|
|
60
|
-
| Language | Symbol extraction | Generated tests | Runtime coverage | Static assertion candidates |
|
|
61
|
-
|---|---:|---:|---:|---:|
|
|
62
|
-
| TypeScript / JavaScript | ✓ | ✓ Jest/Vitest/AVA/etc. | ✓ `lcov.info` | ✓ strongest support |
|
|
63
|
-
| Go | ✓ | ✓ same-package `*_test.go` | ✓ coverprofile | ✓ limited shapes |
|
|
64
|
-
| Python | ✓ | ✓ pytest drafts + marker-aware run hints | ✓ coverage.py / pytest-cov XML | ✓ limited pytest assert shapes |
|
|
65
|
-
| Java | ✓ | ✓ JUnit 4/5 drafts + run hints | ✓ JaCoCo XML | ✓ limited JUnit assert shapes |
|
|
66
|
-
|
|
67
|
-
> **Static assertion candidates are strongest for TypeScript/JavaScript.** “Strongest support” does not mean public Proven; it means TS/JS has the most complete deterministic static association resolver today. Public Proven is stricter and requires the dynamic targeted-proof ledger certificate. **Runtime coverage** is a first-class signal for Go, Java, Python, and TS/JS. Runtime-covered, static candidates, and associated signals never move the Proven percentage.
|
|
68
|
-
|
|
69
|
-
Extraction is language-agnostic (tree-sitter), so more languages are additive — a config entry plus a grammar.
|
|
70
|
-
|
|
71
|
-
### Open-source proof snapshot
|
|
72
|
-
|
|
73
|
-
These numbers are a reproducible snapshot, not a universal benchmark. They come from
|
|
74
|
-
the fresh-history public export of this repo at `5de25ca`, with AI disabled and no
|
|
75
|
-
runtime coverage generation:
|
|
24
|
+
## Install
|
|
76
25
|
|
|
77
26
|
```bash
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
```
|
|
27
|
+
# No install needed (npx)
|
|
28
|
+
npx @orangepro/mcp-server
|
|
81
29
|
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
| Files scanned | 344 |
|
|
85
|
-
| Eligible behaviors | 258 |
|
|
86
|
-
| Static assertion candidates | 133 (51.6%) |
|
|
87
|
-
| Runtime-covered | 0 *(coverage generation was not run for this snapshot)* |
|
|
88
|
-
| Associated signal | 91 |
|
|
89
|
-
| No integration signal | 34 |
|
|
30
|
+
# Or global install
|
|
31
|
+
npm install -g @orangepro/orangepro-mcp
|
|
90
32
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
## Test cases we generate
|
|
96
|
-
|
|
97
|
-
Generation is **evidence-gated** — a category is produced only when the graph has supporting evidence for it, never padded with generic filler. The categories:
|
|
98
|
-
|
|
99
|
-
| Category | What it targets |
|
|
100
|
-
|----------|-----------------|
|
|
101
|
-
| **Happy path** | The primary expected behavior |
|
|
102
|
-
| **Validation error** | Bad/invalid input handling |
|
|
103
|
-
| **Edge case** | Boundaries, empty/null, limits, concurrency, retries |
|
|
104
|
-
| **Integration flow** | Multi-step behavior across components |
|
|
105
|
-
| **Security / privacy** | Authz, sensitive-data, injection-style concerns |
|
|
106
|
-
| **Regression** | Pinning a previously-broken behavior |
|
|
107
|
-
|
|
108
|
-
Each generated test carries its **grounding** (the real files/symbols/tests it cites) and **run hints** (where to put it, how to run it), so an agent can write it, run it, and report pass/fail.
|
|
109
|
-
|
|
110
|
-
For runnable Python and Go output, the local validator must be able to call the normal language tools on your PATH (`python3` for pytest syntax checks, `gofmt` for Go syntax checks). If a generated body cannot be validated, OrangePro returns it as a grounded draft with no run command instead of pretending it is runnable.
|
|
111
|
-
|
|
112
|
-
## Use with your coding agent (the headline workflow)
|
|
113
|
-
|
|
114
|
-
The agent already has shell access, so `opro` prepares the graph/RTM and then generates test **code** with run hints — the agent writes the file, runs it, and reports back. `opro` itself never writes source/test files into your repo and never runs commands.
|
|
115
|
-
|
|
116
|
-
Run `opro` first in the repo. It writes `.orangepro/behavior-coverage.html`, `.orangepro/rtm.md`, and next actions. If a real model provider is configured through your environment, local `.env.provider.local` / `.env.local` / `.env`, or `opro setup`, weak AI grounding and AI-suggested candidate flows run automatically; otherwise the deterministic artifacts are still produced and AI is clearly skipped.
|
|
117
|
-
|
|
118
|
-
Report commands default to `.orangepro/`, but explicit `--out` paths can point at another local review/artifact directory, for example `opro rtm --format json --out /tmp/orangepro-run/rtm.json`.
|
|
33
|
+
# Or from source
|
|
34
|
+
git clone https://github.com/OrangeproAI/orangepro-mcp.git
|
|
35
|
+
cd orangepro-mcp && npm ci && npm run build && npm link
|
|
36
|
+
```
|
|
119
37
|
|
|
120
|
-
|
|
38
|
+
---
|
|
121
39
|
|
|
122
|
-
|
|
40
|
+
## Use with your coding agent
|
|
123
41
|
|
|
124
|
-
|
|
125
|
-
|---|---:|---|
|
|
126
|
-
| `ORANGEPRO_AI_LINK_MAX_BEHAVIORS` | `80` | Maximum behavior gaps processed in one AI-link run |
|
|
127
|
-
| `ORANGEPRO_AI_LINK_SYMBOLS_PER_BEHAVIOR` | `40` | Product CodeSymbol shortlist size per behavior |
|
|
128
|
-
| `ORANGEPRO_AI_LINK_MAX_PROMPT_TOKENS` | `30000` | Approximate prompt token ceiling per batch |
|
|
42
|
+
OrangePro runs as an MCP server. Any MCP-compatible agent (Cursor, Claude Code, Codex, Copilot, OpenCode) can drive it.
|
|
129
43
|
|
|
130
|
-
|
|
44
|
+
### Setup
|
|
131
45
|
|
|
132
|
-
|
|
46
|
+
Add to your client's MCP config:
|
|
133
47
|
|
|
134
|
-
```
|
|
135
|
-
|
|
48
|
+
```json
|
|
49
|
+
{
|
|
50
|
+
"mcpServers": {
|
|
51
|
+
"orangepro-local": {
|
|
52
|
+
"command": "npx",
|
|
53
|
+
"args": ["-y", "@orangepro/mcp-server@latest", "mcp"]
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
}
|
|
136
57
|
```
|
|
137
58
|
|
|
138
|
-
|
|
59
|
+
| Client | Config location |
|
|
60
|
+
|--------|----------------|
|
|
61
|
+
| Claude Code | `.mcp.json` or `~/.claude.json` |
|
|
62
|
+
| Cursor | `~/.cursor/mcp.json` or Settings → MCP |
|
|
63
|
+
| Codex | MCP config printed by `opro agent --client codex`; plugin install after OrangePro is listed in a configured marketplace |
|
|
64
|
+
| VS Code / Copilot | MCP settings |
|
|
65
|
+
|
|
66
|
+
### The workflow
|
|
67
|
+
|
|
68
|
+
Tell your agent:
|
|
69
|
+
|
|
70
|
+
> "Use `orangepro_start`, then `orangepro_generate_tests` with base_ref=main. Write each test to its suggested_path, run it, and report pass/fail."
|
|
71
|
+
|
|
72
|
+
The agent writes the test, runs it, calls `orangepro_prove`, and the behavior turns Dynamically Proven. One prompt, full loop.
|
|
73
|
+
|
|
74
|
+
### MCP tools (18 total)
|
|
75
|
+
|
|
76
|
+
| Tool | What it does |
|
|
77
|
+
|------|--------------|
|
|
78
|
+
| `orangepro_start` | One-command setup: analyze + report + next actions |
|
|
79
|
+
| `orangepro_analyze_sources` | Build/refresh the evidence graph |
|
|
80
|
+
| `orangepro_generate_tests` | Generate grounded tests for gaps |
|
|
81
|
+
| `orangepro_prove` | Run mutation-kill oracle on a behavior |
|
|
82
|
+
| `orangepro_prove_loop` | Setup commands + dynamic proof + report refresh for one behavior |
|
|
83
|
+
| `orangepro_find_test_gaps` | List behaviors with weak/missing tests, ranked by risk |
|
|
84
|
+
| `orangepro_graph_score` | Graph readiness score (0–100) |
|
|
85
|
+
| `orangepro_status` | Workspace state without generating anything |
|
|
86
|
+
| `orangepro_doctor` | Recommend next evidence to improve quality |
|
|
87
|
+
| `orangepro_rtm` | Requirements traceability matrix |
|
|
88
|
+
| `orangepro_stats` | Aggregate statistics |
|
|
89
|
+
| `orangepro_changed_impact` | What a diff touches (requires git + base ref) |
|
|
90
|
+
| `orangepro_record_run` | Record a test run result |
|
|
91
|
+
| `orangepro_explain_test` | Explain why a test was generated |
|
|
92
|
+
| `orangepro_export_evidence_pack` | Export metadata-only evidence pack |
|
|
93
|
+
| `orangepro_update_graph` | Incremental graph update |
|
|
94
|
+
| `orangepro_ai_links` | Weak behavior→symbol suggestions (optional AI) |
|
|
95
|
+
| `orangepro_ai_flows` | Candidate flow discovery (optional AI) |
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## CLI reference
|
|
139
100
|
|
|
140
101
|
```bash
|
|
141
|
-
|
|
142
|
-
|
|
102
|
+
opro # analyze + report + agent next actions
|
|
103
|
+
opro start --base main # same, scoped to a branch diff
|
|
104
|
+
opro analyze # build the evidence graph
|
|
105
|
+
opro score # graph readiness (0–100)
|
|
106
|
+
opro gaps --limit 10 # top 10 untested behaviors
|
|
107
|
+
opro generate --base main # tests for PR diff
|
|
108
|
+
opro generate --single # top gap, whole repo
|
|
109
|
+
opro prove # mutation-kill oracle (use the prove_run args returned by generate)
|
|
110
|
+
opro rtm # traceability matrix
|
|
111
|
+
opro export # metadata-only evidence pack
|
|
112
|
+
opro mcp # run as MCP server (stdio)
|
|
113
|
+
opro doctor # what evidence to add next
|
|
114
|
+
opro coverage # ingest runtime coverage
|
|
143
115
|
```
|
|
144
116
|
|
|
145
|
-
|
|
117
|
+
Add `--json` to any read command for machine output. Run `opro help` for the full reference.
|
|
146
118
|
|
|
147
|
-
|
|
148
|
-
codex plugin add orangepro@personal
|
|
149
|
-
```
|
|
119
|
+
---
|
|
150
120
|
|
|
151
|
-
|
|
121
|
+
## PR workflow
|
|
152
122
|
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
123
|
+
```bash
|
|
124
|
+
opro generate --base main # tests for what this branch changed
|
|
125
|
+
opro generate --pr 1234 # checks out PR #1234 — mutates your working tree; needs gh + confirmation (prefer --base)
|
|
126
|
+
opro generate --changed # current branch diff vs main
|
|
157
127
|
```
|
|
158
128
|
|
|
159
|
-
|
|
129
|
+
Each generated test includes:
|
|
130
|
+
- **Grounding** — the real files, symbols, and existing tests it cites
|
|
131
|
+
- **Run hints** — where to write it, how to run it
|
|
132
|
+
- **Scenario bucket + technique** — what failure mode it targets and how
|
|
160
133
|
|
|
161
|
-
|
|
134
|
+
---
|
|
162
135
|
|
|
163
|
-
|
|
164
|
-
- [Claude Code](docs/agents/claude-code.md)
|
|
165
|
-
- [Cursor](docs/agents/cursor.md)
|
|
166
|
-
- [OpenCode](docs/agents/opencode.md)
|
|
167
|
-
- [VS Code / Copilot](docs/agents/vscode.md)
|
|
136
|
+
## Test categories
|
|
168
137
|
|
|
169
|
-
|
|
138
|
+
Generation is evidence-gated. A category is produced only when the graph has supporting evidence — never padded with generic filler. These are the public local generation buckets. The broader concern taxonomy used by planning prompts is not a public coverage taxonomy and does not change report tiers.
|
|
170
139
|
|
|
171
|
-
|
|
140
|
+
| Category | What it targets |
|
|
141
|
+
|----------|-----------------|
|
|
142
|
+
| Happy path | Primary expected behavior |
|
|
143
|
+
| Validation error | Bad/invalid input handling |
|
|
144
|
+
| Edge case | Boundaries, empty/null, concurrency, retries |
|
|
145
|
+
| Integration flow | Multi-step behavior across services |
|
|
146
|
+
| Security / privacy | Auth, injection, data leakage |
|
|
147
|
+
| Regression | Pinning a previously-broken behavior |
|
|
172
148
|
|
|
173
|
-
|
|
174
|
-
2. Create the fresh-history public export and verify it has one commit, no private history, and no private artifacts.
|
|
175
|
-
3. Codex: keep `.agents/plugins/marketplace.json` + `plugins/orangepro`; the repo marketplace install works from this public repo. The bare `codex plugin add orangepro` path requires OrangePro to be present in a configured/default Codex marketplace.
|
|
176
|
-
4. Claude Code: keep `.claude-plugin/marketplace.json` + `plugins/orangepro`; users can add this repo as a Claude plugin marketplace. Directory-style discovery requires submitting/listing the marketplace through Claude's plugin ecosystem.
|
|
177
|
-
5. Cursor: keep `plugins/orangepro/.cursor-plugin/plugin.json`, `rules/`, and `.mcp.json`; submit the same plugin bundle to the Cursor Marketplace when ready.
|
|
178
|
-
6. OpenCode / VS Code: publish the documented MCP config snippets using `npx -y -p @orangepro/orangepro-mcp@latest opro mcp`.
|
|
179
|
-
7. Smoke each install path in a clean repo and confirm `orangepro_start` works with no pinned model and no committed key.
|
|
149
|
+
---
|
|
180
150
|
|
|
181
|
-
|
|
151
|
+
## Evidence tiers
|
|
182
152
|
|
|
183
|
-
|
|
184
|
-
You (to your agent): "Use orangepro_start, then use orangepro_generate_tests with base_ref=main.
|
|
185
|
-
Write each
|
|
186
|
-
test to its suggested_path, run it, and report pass/fail."
|
|
187
|
-
```
|
|
153
|
+
Every behavior gets exactly one tier. Nothing is labeled "tested" on faith.
|
|
188
154
|
|
|
189
|
-
|
|
155
|
+
| Tier | What it means | How you get there |
|
|
156
|
+
|------|---------------|------------------|
|
|
157
|
+
| **Dynamically Proven** | A real test kills a targeted mutant of this behavior | `opro prove` after writing/running a test |
|
|
158
|
+
| **Runtime-covered** | Coverage tool executed this code | `opro start --generate-coverage` |
|
|
159
|
+
| **Statically Linked** | Import/name/structural match links a test to this code | Automatic during analysis |
|
|
160
|
+
| **No Signal** | Nothing tests this behavior yet | — |
|
|
190
161
|
|
|
191
|
-
|
|
192
|
-
2. Run its `run_command` (or the repo's own `npm test` / `pytest`).
|
|
193
|
-
3. After a pass, call the returned `prove_run` args so OrangePro can dynamically prove the target. `record_run` is static diagnostics only.
|
|
194
|
-
4. Report pass/fail + stack traces, and propose fixes.
|
|
162
|
+
> **"Dynamically Proven 0" is normal on first run.** Static analysis always runs. Dynamic proof requires running tests against targeted mutations. That's the trust model — nothing is Dynamically Proven until a real test kills a real mutant.
|
|
195
163
|
|
|
196
|
-
|
|
164
|
+
---
|
|
197
165
|
|
|
198
|
-
##
|
|
166
|
+
## Language support
|
|
199
167
|
|
|
200
|
-
|
|
201
|
-
- **No source/test writes.** It never writes source or test files into your repo. The only writes are workspace metadata under `.orangepro/` — analyze from a throwaway directory for zero writes inside the target checkout.
|
|
202
|
-
- **Metadata-only exports.** The evidence pack and graph HTML carry file paths, names, hashes, frameworks, provenance, and scores — **not** raw source, prompts, scoring weights, or ranking traces. Generated test bodies are opt-in (`export --include-generated-bodies`).
|
|
203
|
-
- **Your keys stay yours.** Model keys are read from the environment at call time and never written into the graph, config, or pack.
|
|
168
|
+
OrangePro separates static mapping, generated tests, runtime coverage, and dynamic proof. Those are different confidence bars.
|
|
204
169
|
|
|
205
|
-
|
|
170
|
+
| Language | Static behavior extraction | Generated tests | Runtime coverage | Dynamic proof |
|
|
171
|
+
|----------|:--------------------------:|:---------------:|:----------------:|:-------------:|
|
|
172
|
+
| TypeScript / JavaScript | ✓ | ✓ Jest / Vitest / Mocha / AVA-style drafts | ✓ lcov.info | ✓ Vitest / Jest / Mocha |
|
|
173
|
+
| Python | ✓ | ✓ pytest | ✓ coverage.py / pytest-cov XML | ✓ pytest |
|
|
174
|
+
| Go | ✓ | ✓ same-package `*_test.go` | ✓ coverprofile | ✓ `go test` |
|
|
175
|
+
| Java | ✓ | ✓ JUnit 4/5 | ✓ JaCoCo XML | ✓ Maven/JUnit |
|
|
176
|
+
| Kotlin, Rust, PHP, C#, Ruby, Swift, C, C++ | ✓ static behavior extraction | planned | planned where standard coverage exists | planned proof profiles |
|
|
206
177
|
|
|
207
|
-
|
|
178
|
+
Static mapping works across many languages through tree-sitter and repo metadata. Dynamic proof is deliberately narrower: each language needs a runner, mutation locator, sandbox profile, and false-proof regressions before it can mint Dynamically Proven.
|
|
208
179
|
|
|
209
|
-
|
|
210
|
-
node scripts/smoke-gap-fill-loop.mjs
|
|
211
|
-
```
|
|
180
|
+
---
|
|
212
181
|
|
|
213
|
-
|
|
214
|
-
the row starts at `No integration signal`, writes a real assertion test, then runs
|
|
215
|
-
`opro prove` to execute the dynamic targeted-proof oracle. The row becomes
|
|
216
|
-
`Reproven (this run)` only after baseline-green → mutant assertion-fail closes a
|
|
217
|
-
metadata-only ledger certificate. This is the smallest local proof that OrangePro's
|
|
218
|
-
public Proven number moves because a real dynamic kill happened, not because an
|
|
219
|
-
LLM or static matcher claimed coverage.
|
|
182
|
+
## Model setup (BYOK)
|
|
220
183
|
|
|
221
|
-
|
|
184
|
+
Analysis, scoring, and proof need no model key. Generation does.
|
|
222
185
|
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
npm link # one-time: puts the `opro` command on your PATH
|
|
229
|
-
```
|
|
230
|
-
|
|
231
|
-
Prefer not to link? Every `opro <command>` is equivalent to `node dist/local/cli.js <command>`.
|
|
232
|
-
|
|
233
|
-
## Model selection (BYOK)
|
|
234
|
-
|
|
235
|
-
Generation uses **your** model key. Pick one provider; with none set, auto-detect order is **OpenAI-compatible → Ollama → Anthropic**.
|
|
236
|
-
|
|
237
|
-
| Provider | Env |
|
|
238
|
-
|---|---|
|
|
239
|
-
| OpenAI-compatible | `OPENAI_API_KEY`, optional `OPENAI_BASE_URL`, `OPENAI_MODEL` |
|
|
240
|
-
| Anthropic | `ANTHROPIC_API_KEY`, optional `ANTHROPIC_MODEL`, `ANTHROPIC_BASE_URL` |
|
|
241
|
-
| Ollama (local, no key) | `OLLAMA_BASE_URL`, optional `OLLAMA_MODEL` |
|
|
186
|
+
| Provider | Environment variable |
|
|
187
|
+
|----------|---------------------|
|
|
188
|
+
| OpenAI-compatible | `OPENAI_API_KEY` (optional: `OPENAI_BASE_URL`, `OPENAI_MODEL`) |
|
|
189
|
+
| Anthropic | `ANTHROPIC_API_KEY` (optional: `ANTHROPIC_MODEL`) |
|
|
190
|
+
| Ollama (local, no key) | `OLLAMA_BASE_URL` (optional: `OLLAMA_MODEL`) |
|
|
242
191
|
|
|
243
|
-
|
|
192
|
+
Auto-detect order: OpenAI → Ollama → Anthropic. Override with `--provider` and `--model`.
|
|
244
193
|
|
|
245
|
-
|
|
194
|
+
Run `opro setup` to configure interactively. Keys stay in your environment — never written to graph, config, or artifacts.
|
|
246
195
|
|
|
247
|
-
|
|
196
|
+
---
|
|
248
197
|
|
|
249
|
-
|
|
198
|
+
## AI candidate lanes
|
|
250
199
|
|
|
251
|
-
|
|
200
|
+
With a provider key, OrangePro can stage weak AI behavior→symbol links and AI-suggested candidate flows. These are ready for local use as review/generation worklists, but they are not evidence:
|
|
252
201
|
|
|
253
|
-
|
|
202
|
+
- AI links appear as `AI-linked` suggestions.
|
|
203
|
+
- AI flows are stored separately from deterministic flows.
|
|
204
|
+
- Neither lane changes Dynamically Proven, Runtime-covered, Statically Linked, denominator counts, or evidence tiers.
|
|
254
205
|
|
|
255
|
-
|
|
256
|
-
|---|---|
|
|
257
|
-
| Product/test/config/doc parsing | TypeScript/JavaScript, Python, Go, Java, Kotlin, Rust, PHP, C#, Ruby, Swift, C, C++, Markdown/config metadata |
|
|
258
|
-
| Structural associations | Broad non-TS graph signals: imports/calls/test naming/framework/package signals stay **Associated signal**, never Proven |
|
|
259
|
-
| Static assertion candidates | TypeScript/JavaScript (strongest deterministic confirmer), plus limited exact-resolved shapes for Go, Python, and Java can mint hard `COVERS`/`TESTED_BY` graph edges for diagnostics. These render as **Associated signal**, not public Proven. Public Proven requires a dynamic targeted-proof ledger certificate from `opro prove`. |
|
|
260
|
-
| Runnable generated tests | TypeScript/JavaScript (including AVA), Python (marker-aware pytest hints), Go, and Java (JUnit 4/5), with framework/path/run hints and safe draft fallback when the output cannot be validated |
|
|
261
|
-
| Runtime coverage ingestion | Go coverprofiles, JS/TS `lcov.info`, Python `coverage.xml`, Java JaCoCo XML. Go, repo-defined JS/TS coverage scripts, pytest-cov XML, and Maven/Gradle JaCoCo can be generated locally with `--generate-coverage` when the repo's normal test tooling is installed. |
|
|
262
|
-
| AI grounding | Optional weak `AI-linked` behavior→symbol suggestions and AI-suggested candidate flow worklists. These guide generation/review but **do not** change Proven coverage, deterministic flow counts, or tiers. |
|
|
206
|
+
Use them when you want the agent to find likely service-boundary flows faster; ignore them when you want a deterministic-only report.
|
|
263
207
|
|
|
264
|
-
|
|
208
|
+
---
|
|
265
209
|
|
|
266
|
-
|
|
267
|
-
- More deterministic proof recall for non-TS/Go languages, only when false-confirm tests make it safe.
|
|
210
|
+
## How it works
|
|
268
211
|
|
|
269
|
-
|
|
212
|
+
OrangePro separates **analysis** (what your code does) from **proof** (whether tests actually verify it).
|
|
270
213
|
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
214
|
+
```
|
|
215
|
+
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
|
|
216
|
+
│ Your Code │ ──► │ Knowledge │ ──► │ Evidence │
|
|
217
|
+
│ (any lang) │ │ Graph │ │ Tiers │
|
|
218
|
+
└─────────────┘ └──────────────┘ └─────────────┘
|
|
219
|
+
│
|
|
220
|
+
┌──────┴──────┐
|
|
221
|
+
▼ ▼
|
|
222
|
+
┌───────────┐ ┌──────────┐
|
|
223
|
+
│ Gap Report│ │ Generate │
|
|
224
|
+
│ + Risks │ │ Tests │
|
|
225
|
+
└───────────┘ └──────────┘
|
|
283
226
|
```
|
|
284
227
|
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
|
288
|
-
|
|
289
|
-
|
|
|
290
|
-
|
|
|
291
|
-
| Cursor | `~/.cursor/mcp.json` (or Settings → MCP) |
|
|
292
|
-
| Codex / OpenCode / Cline / Continue / Windsurf / other | that client's MCP / `mcpServers` settings |
|
|
293
|
-
|
|
294
|
-
**Tools.** *Start here*: `orangepro_start`. *Baseline* (no diff/PR needed): `orangepro_analyze_sources`, `orangepro_status`, `orangepro_graph_score`, `orangepro_doctor`, `orangepro_find_test_gaps`, `orangepro_rtm`, `orangepro_generate_tests`, `orangepro_prove`, `orangepro_record_run`, `orangepro_stats`, `orangepro_explain_test`, `orangepro_export_evidence_pack`. *Diff/PR* (require a git repo + real diff vs a base): `orangepro_changed_impact`. *Optional AI lanes*: `orangepro_ai_links` for weak behavior↔symbol suggestions, `orangepro_ai_flows` for candidate flow worklists.
|
|
228
|
+
| Phase | What happens | Needs a model key? |
|
|
229
|
+
|-------|-------------|-------------------|
|
|
230
|
+
| **Analyze** | AST walk → behaviors, flows, evidence tiers | No |
|
|
231
|
+
| **Score** | Graph readiness score (0–100) with reasons | No |
|
|
232
|
+
| **Generate** | Grounded tests for top gaps, per-behavior | Yes (BYOK) |
|
|
233
|
+
| **Prove** | Mutation-kill oracle confirms test actually breaks if behavior changes | No |
|
|
295
234
|
|
|
296
|
-
|
|
235
|
+
---
|
|
297
236
|
|
|
298
|
-
|
|
237
|
+
## Privacy
|
|
299
238
|
|
|
300
|
-
- **
|
|
301
|
-
- **
|
|
239
|
+
- **No stored source.** Reads code in-process. Never uploads to an OrangePro server.
|
|
240
|
+
- **No source mutation.** Never edits your existing files. Writes metadata to `.orangepro/`.
|
|
241
|
+
- **Metadata-only exports.** File paths, names, hashes, scores — not raw source.
|
|
242
|
+
- **Your keys stay yours.** Read from env at call time, never persisted.
|
|
302
243
|
|
|
303
|
-
|
|
304
|
-
<summary><b>Comparison mode</b></summary>
|
|
244
|
+
---
|
|
305
245
|
|
|
306
|
-
|
|
246
|
+
## What's on the hosted platform
|
|
307
247
|
|
|
308
|
-
|
|
248
|
+
This repo is the free local tool. The [OrangePro platform](https://orangepro.ai) adds:
|
|
309
249
|
|
|
310
|
-
|
|
250
|
+
- Persistent knowledge graph across PRs and repos
|
|
251
|
+
- Managed dynamic proof at scale (larger budgets, CI workers, service setup profiles)
|
|
252
|
+
- PR/CI policy gates over Dynamically Proven, Runtime-covered, and risk deltas
|
|
253
|
+
- Jira / Confluence / TestRail / OpenAPI enrichment
|
|
254
|
+
- Cross-repo intelligence and recurring-flow memory
|
|
255
|
+
- Production incident correlation and regression targeting
|
|
256
|
+
- Full test lifecycle management and team dashboards
|
|
311
257
|
|
|
312
|
-
|
|
258
|
+
---
|
|
313
259
|
|
|
314
|
-
##
|
|
260
|
+
## Contributing
|
|
315
261
|
|
|
316
262
|
```bash
|
|
317
|
-
npm run build
|
|
318
|
-
npm test
|
|
319
|
-
npm run typecheck
|
|
320
|
-
npm run local -- analyze . # run the CLI via tsx in dev
|
|
321
|
-
node scripts/smoke-local.mjs # offline end-to-end smoke test
|
|
263
|
+
npm run build # compile to dist/
|
|
264
|
+
npm test # vitest
|
|
265
|
+
npm run typecheck # type check without emitting
|
|
322
266
|
```
|
|
323
267
|
|
|
324
|
-
|
|
268
|
+
See [docs/local-proof-kit.md](docs/local-proof-kit.md) for the full development reference.
|
|
325
269
|
|
|
326
270
|
## License
|
|
327
271
|
|
|
328
|
-
[MIT](LICENSE) © OrangePro
|
|
272
|
+
[MIT](LICENSE) © OrangePro
|