@blxzer/cursor-trellis 0.1.2 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +58 -0
- package/README.md +45 -21
- package/dist/configurators/workflow.d.ts +1 -1
- package/dist/configurators/workflow.d.ts.map +1 -1
- package/dist/configurators/workflow.js +20 -9
- package/dist/configurators/workflow.js.map +1 -1
- package/dist/migrations/manifests/0.1.3.json +9 -0
- package/dist/migrations/manifests/0.1.4.json +9 -0
- package/dist/templates/cursor/rules/retrieval-routing.mdc +9 -7
- package/dist/templates/markdown/agents.md +1 -1
- package/dist/templates/markdown/index.d.ts +1 -0
- package/dist/templates/markdown/index.d.ts.map +1 -1
- package/dist/templates/markdown/index.js +1 -0
- package/dist/templates/markdown/index.js.map +1 -1
- package/dist/templates/markdown/prompts/run-semantic-slice-12.md.txt +13 -29
- package/dist/templates/markdown/spec/guides/code-reuse-thinking-guide.md.txt +0 -49
- package/dist/templates/markdown/spec/guides/cross-layer-thinking-guide.md.txt +0 -85
- package/dist/templates/markdown/spec/guides/cursor-semantic-compliance.md.txt +7 -21
- package/dist/templates/markdown/spec/guides/cursor-subagent-policy.md.txt +6 -10
- package/dist/templates/markdown/spec/guides/retrieval-daily-guide.md.txt +17 -18
- package/dist/templates/shared-hooks/inject-retrieval-plan.py +42 -23
- package/dist/templates/shared-hooks/inject-subagent-context.py +6 -2
- package/dist/templates/shared-hooks/session-start.py +17 -9
- package/dist/templates/trellis/index.d.ts +8 -4
- package/dist/templates/trellis/index.d.ts.map +1 -1
- package/dist/templates/trellis/index.js +13 -7
- package/dist/templates/trellis/index.js.map +1 -1
- package/dist/templates/trellis/scripts/common/codebase_retrieval_router.py +80 -31
- package/dist/templates/trellis/scripts/common/config.py +1 -1
- package/dist/templates/trellis/scripts/common/cursor_retrieval_env.py +92 -0
- package/dist/templates/trellis/scripts/common/parent_orchestration.py +27 -15
- package/dist/templates/trellis/scripts/common/retrieval_adapter_metadata.py +100 -9
- package/dist/templates/trellis/scripts/common/retrieval_agent_instructions.py +76 -31
- package/dist/templates/trellis/scripts/common/retrieval_tool_classification.py +18 -3
- package/dist/templates/trellis/scripts/common/semantic_plan_gate.py +19 -0
- package/dist/templates/trellis/scripts/common/smart_search_evidence.py +5 -2
- package/dist/templates/trellis/scripts/common/smart_search_resolve.py +28 -3
- package/dist/templates/trellis/scripts/common/task_gates.py +372 -73
- package/dist/templates/trellis/scripts/common/task_map.py +29 -1
- package/dist/templates/trellis/scripts/cursor_retrieval_probe.py +396 -0
- package/dist/templates/trellis/scripts/cursor_retrieval_probe_prompt.md +300 -0
- package/dist/templates/trellis/scripts/retrieval_probe_matrix_template.json +126 -0
- package/dist/templates/trellis/workflow.md +9 -53
- package/dist/utils/codebase-retrieval-router.d.ts +5 -0
- package/dist/utils/codebase-retrieval-router.d.ts.map +1 -1
- package/dist/utils/codebase-retrieval-router.js +48 -28
- package/dist/utils/codebase-retrieval-router.js.map +1 -1
- package/dist/utils/cursor-retrieval-env.d.ts +28 -0
- package/dist/utils/cursor-retrieval-env.d.ts.map +1 -0
- package/dist/utils/cursor-retrieval-env.js +89 -0
- package/dist/utils/cursor-retrieval-env.js.map +1 -0
- package/dist/utils/project-capabilities.d.ts.map +1 -1
- package/dist/utils/project-capabilities.js +22 -15
- package/dist/utils/project-capabilities.js.map +1 -1
- package/dist/utils/retrieval-agent-instructions.d.ts.map +1 -1
- package/dist/utils/retrieval-agent-instructions.js +37 -21
- package/dist/utils/retrieval-agent-instructions.js.map +1 -1
- package/dist/utils/retrieval-tool-classification.d.ts +2 -0
- package/dist/utils/retrieval-tool-classification.d.ts.map +1 -1
- package/dist/utils/retrieval-tool-classification.js +10 -2
- package/dist/utils/retrieval-tool-classification.js.map +1 -1
- package/dist/utils/semantic-plan-gate.d.ts +8 -0
- package/dist/utils/semantic-plan-gate.d.ts.map +1 -0
- package/dist/utils/semantic-plan-gate.js +42 -0
- package/dist/utils/semantic-plan-gate.js.map +1 -0
- package/package.json +5 -43
- package/bin/smart-search.js +0 -79
- package/dist/templates/trellis/scripts/aggregate_retrieval_telemetry.py +0 -342
- package/dist/templates/trellis/scripts/batch_plan_envelope.py +0 -105
- package/vendor/smart-search/LICENSE +0 -21
- package/vendor/smart-search/README.md +0 -412
- package/vendor/smart-search/README.zh-CN.md +0 -430
- package/vendor/smart-search/npm/bin/smart-search.js +0 -63
- package/vendor/smart-search/npm/scripts/postinstall.js +0 -87
- package/vendor/smart-search/npm/scripts/resolve-prerelease-version.js +0 -108
- package/vendor/smart-search/npm/scripts/set-package-version.js +0 -35
- package/vendor/smart-search/npm/scripts/sync-python-version.js +0 -22
- package/vendor/smart-search/npm/scripts/test-wrapper-repair.js +0 -99
- package/vendor/smart-search/npm/scripts/test.js +0 -85
- package/vendor/smart-search/package.json +0 -42
- package/vendor/smart-search/pyproject.toml +0 -37
- package/vendor/smart-search/skills/smart-search-cli/SKILL.md +0 -387
- package/vendor/smart-search/skills/smart-search-cli/agents/openai.yaml +0 -3
- package/vendor/smart-search/skills/smart-search-cli/examples/batch-search.md +0 -98
- package/vendor/smart-search/skills/smart-search-cli/examples/evidence-gathering.md +0 -89
- package/vendor/smart-search/skills/smart-search-cli/references/cli-contract.md +0 -337
- package/vendor/smart-search/src/smart_search/__init__.py +0 -1
- package/vendor/smart-search/src/smart_search/assets/skills/smart-search-cli/SKILL.md +0 -387
- package/vendor/smart-search/src/smart_search/assets/skills/smart-search-cli/agents/openai.yaml +0 -3
- package/vendor/smart-search/src/smart_search/assets/skills/smart-search-cli/examples/batch-search.md +0 -98
- package/vendor/smart-search/src/smart_search/assets/skills/smart-search-cli/examples/evidence-gathering.md +0 -89
- package/vendor/smart-search/src/smart_search/assets/skills/smart-search-cli/references/cli-contract.md +0 -337
- package/vendor/smart-search/src/smart_search/cli.py +0 -2003
- package/vendor/smart-search/src/smart_search/config.py +0 -580
- package/vendor/smart-search/src/smart_search/logger.py +0 -43
- package/vendor/smart-search/src/smart_search/providers/__init__.py +0 -16
- package/vendor/smart-search/src/smart_search/providers/base.py +0 -41
- package/vendor/smart-search/src/smart_search/providers/context7.py +0 -141
- package/vendor/smart-search/src/smart_search/providers/exa.py +0 -206
- package/vendor/smart-search/src/smart_search/providers/jina.py +0 -136
- package/vendor/smart-search/src/smart_search/providers/openai_compatible.py +0 -370
- package/vendor/smart-search/src/smart_search/providers/zhipu.py +0 -143
- package/vendor/smart-search/src/smart_search/service.py +0 -2865
- package/vendor/smart-search/src/smart_search/sources.py +0 -429
- package/vendor/smart-search/src/smart_search/utils.py +0 -220
|
@@ -0,0 +1,396 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Cursor retrieval capability probe (Phase 1).
|
|
3
|
+
|
|
4
|
+
Auto-detects the current Cursor environment (Native vs Cursor++ BYOK) by reading
|
|
5
|
+
~/.ccursor/routes.json (byokMode), then runs the subset of retrieval-capability
|
|
6
|
+
probes that are CLI-reachable with deterministic known answers:
|
|
7
|
+
|
|
8
|
+
P-01 Grep (rg) — local IDE literal search
|
|
9
|
+
P-02 Read — local IDE file read
|
|
10
|
+
P-05 codegraph index — on-disk .codegraph/ presence
|
|
11
|
+
P-07 smart-search CLI — doctor / status reachability
|
|
12
|
+
D-01 Experiment D — cursorEnv + BYOK fast-context install readiness
|
|
13
|
+
|
|
14
|
+
Probes P-03/04/06/08/09/10/D-01(manual inventory) require a live agent session and are documented in
|
|
15
|
+
cursor_retrieval_probe_prompt.md; their results are filled into
|
|
16
|
+
retrieval_probe_matrix_template.json manually.
|
|
17
|
+
|
|
18
|
+
This script does NOT depend on Cursor being open and does NOT modify any state.
|
|
19
|
+
"""
|
|
20
|
+
|
|
21
|
+
from __future__ import annotations
|
|
22
|
+
|
|
23
|
+
import argparse
|
|
24
|
+
import io
|
|
25
|
+
import json
|
|
26
|
+
import os
|
|
27
|
+
import re
|
|
28
|
+
import sys
|
|
29
|
+
from datetime import datetime, timezone
|
|
30
|
+
from pathlib import Path
|
|
31
|
+
|
|
32
|
+
_SCRIPTS = Path(__file__).resolve().parent
|
|
33
|
+
if str(_SCRIPTS) not in sys.path:
|
|
34
|
+
sys.path.insert(0, str(_SCRIPTS))
|
|
35
|
+
|
|
36
|
+
from common.cursor_retrieval_env import ( # noqa: E402
|
|
37
|
+
ENV_BYOK,
|
|
38
|
+
ENV_NATIVE,
|
|
39
|
+
ENV_UNKNOWN,
|
|
40
|
+
detect_cursor_retrieval_env_info,
|
|
41
|
+
)
|
|
42
|
+
|
|
43
|
+
if sys.platform.startswith("win"):
|
|
44
|
+
for _stream_name in ("stdin", "stdout", "stderr"):
|
|
45
|
+
_stream = getattr(sys, _stream_name, None)
|
|
46
|
+
if _stream is None:
|
|
47
|
+
continue
|
|
48
|
+
if hasattr(_stream, "reconfigure"):
|
|
49
|
+
try:
|
|
50
|
+
_stream.reconfigure(encoding="utf-8", errors="replace")
|
|
51
|
+
except Exception:
|
|
52
|
+
pass
|
|
53
|
+
elif hasattr(_stream, "detach"):
|
|
54
|
+
try:
|
|
55
|
+
setattr(
|
|
56
|
+
sys,
|
|
57
|
+
_stream_name,
|
|
58
|
+
io.TextIOWrapper(_stream.detach(), encoding="utf-8", errors="replace"),
|
|
59
|
+
)
|
|
60
|
+
except Exception:
|
|
61
|
+
pass
|
|
62
|
+
|
|
63
|
+
PROBE_VERSION = 1
|
|
64
|
+
|
|
65
|
+
|
|
66
|
+
def _utc_now() -> str:
|
|
67
|
+
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def detect_environment() -> dict[str, object]:
|
|
71
|
+
"""Alias for router/probe: same as detect_cursor_retrieval_env_info()."""
|
|
72
|
+
return detect_cursor_retrieval_env_info()
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
def _probe_grep(repo_root: Path) -> dict[str, object]:
|
|
76
|
+
"""P-01: Grep a known literal in a known file under the repo.
|
|
77
|
+
|
|
78
|
+
Known answer: 'TRELLIS_B2_WPeLc8' must appear in
|
|
79
|
+
.trellis/local/cursor2plus/patch_wpelc8.py (the MARKER constant).
|
|
80
|
+
"""
|
|
81
|
+
marker = "TRELLIS_B2_WPeLc8"
|
|
82
|
+
target = repo_root / ".trellis" / "local" / "cursor2plus" / "patch_wpelc8.py"
|
|
83
|
+
ok_file = target.is_file()
|
|
84
|
+
found = False
|
|
85
|
+
matches: list[str] = []
|
|
86
|
+
if ok_file:
|
|
87
|
+
try:
|
|
88
|
+
text = target.read_text(encoding="utf-8", errors="ignore")
|
|
89
|
+
for i, line in enumerate(text.splitlines(), 1):
|
|
90
|
+
if marker in line:
|
|
91
|
+
matches.append(f"{i}:{line.strip()[:120]}")
|
|
92
|
+
found = bool(matches)
|
|
93
|
+
except OSError:
|
|
94
|
+
pass
|
|
95
|
+
return {
|
|
96
|
+
"id": "P-01",
|
|
97
|
+
"label": "Grep (rg) — local IDE literal search",
|
|
98
|
+
"status": "pass" if found else "fail",
|
|
99
|
+
"expected": marker,
|
|
100
|
+
"target_file": str(target),
|
|
101
|
+
"matches": matches[:5],
|
|
102
|
+
"note": "Local IDE operation; not routed through BYOK proxy." if found else (
|
|
103
|
+
"Target file missing or marker not found." if ok_file else "Target file missing."
|
|
104
|
+
),
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+
def _probe_read(repo_root: Path) -> dict[str, object]:
|
|
109
|
+
"""P-02: Read a known file and verify a known line content.
|
|
110
|
+
|
|
111
|
+
Known answer: .cursor/mcp.json must contain a 'codegraph' server entry.
|
|
112
|
+
"""
|
|
113
|
+
target = repo_root / ".cursor" / "mcp.json"
|
|
114
|
+
ok_file = target.is_file()
|
|
115
|
+
has_codegraph = False
|
|
116
|
+
has_fast_context = False
|
|
117
|
+
servers: list[str] = []
|
|
118
|
+
if ok_file:
|
|
119
|
+
try:
|
|
120
|
+
data = json.loads(target.read_text(encoding="utf-8"))
|
|
121
|
+
mcp = data.get("mcpServers", {})
|
|
122
|
+
if isinstance(mcp, dict):
|
|
123
|
+
servers = sorted(mcp.keys())
|
|
124
|
+
has_codegraph = "codegraph" in mcp
|
|
125
|
+
has_fast_context = "fast-context" in mcp
|
|
126
|
+
except (OSError, json.JSONDecodeError):
|
|
127
|
+
pass
|
|
128
|
+
ok = has_codegraph
|
|
129
|
+
return {
|
|
130
|
+
"id": "P-02",
|
|
131
|
+
"label": "Read — local IDE file read + JSON parse",
|
|
132
|
+
"status": "pass" if ok else "fail",
|
|
133
|
+
"expected": "mcpServers.codegraph present",
|
|
134
|
+
"target_file": str(target),
|
|
135
|
+
"configured_servers": servers,
|
|
136
|
+
"has_codegraph": has_codegraph,
|
|
137
|
+
"has_fast_context": has_fast_context,
|
|
138
|
+
"note": "Local IDE operation; verifies MCP config on disk." if ok else "mcp.json missing or codegraph not configured.",
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
|
|
142
|
+
def _probe_codegraph_index(repo_root: Path) -> dict[str, object]:
|
|
143
|
+
"""P-05: codegraph index on disk (.codegraph/ directories).
|
|
144
|
+
|
|
145
|
+
Mirrors codegraph_session_smoke.py logic.
|
|
146
|
+
"""
|
|
147
|
+
found: list[Path] = []
|
|
148
|
+
direct = repo_root / ".codegraph"
|
|
149
|
+
if direct.is_dir():
|
|
150
|
+
found.append(direct.resolve())
|
|
151
|
+
for child in sorted(repo_root.iterdir()):
|
|
152
|
+
if not child.is_dir():
|
|
153
|
+
continue
|
|
154
|
+
nested = child / ".codegraph"
|
|
155
|
+
if nested.is_dir() and nested.resolve() not in found:
|
|
156
|
+
found.append(nested.resolve())
|
|
157
|
+
ok = len(found) > 0
|
|
158
|
+
return {
|
|
159
|
+
"id": "P-05",
|
|
160
|
+
"label": "codegraph index — on-disk .codegraph/ presence",
|
|
161
|
+
"status": "pass" if ok else "fail",
|
|
162
|
+
"expected": "at least one .codegraph/ dir under workspace root or top-level subproject",
|
|
163
|
+
"index_paths": [str(p) for p in found],
|
|
164
|
+
"note": "Local disk; MCP must also be enabled in Cursor for live calls." if ok else "Run codegraph init to index.",
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
|
|
168
|
+
def _probe_smart_search(repo_root: Path) -> dict[str, object]:
|
|
169
|
+
"""P-07: smart-search CLI reachability (resolve, do not execute search).
|
|
170
|
+
|
|
171
|
+
Reuses common.smart_search_resolve to find the executable; reports resolved
|
|
172
|
+
argv without running a network search. A separate `doctor` invocation is
|
|
173
|
+
attempted if the resolved entrypoint is the smart-search package CLI.
|
|
174
|
+
"""
|
|
175
|
+
scripts_dir = repo_root / ".trellis" / "scripts"
|
|
176
|
+
if str(scripts_dir) not in sys.path:
|
|
177
|
+
sys.path.insert(0, str(scripts_dir))
|
|
178
|
+
resolved: list[str] | None = None
|
|
179
|
+
resolve_error = ""
|
|
180
|
+
try:
|
|
181
|
+
from common.smart_search_resolve import resolve_smart_search_argv # type: ignore[import-not-found]
|
|
182
|
+
|
|
183
|
+
resolved = resolve_smart_search_argv(repo_root)
|
|
184
|
+
except Exception as exc:
|
|
185
|
+
resolve_error = f"{type(exc).__name__}: {exc}"
|
|
186
|
+
|
|
187
|
+
doctor_status = "skipped"
|
|
188
|
+
doctor_output = ""
|
|
189
|
+
if resolved:
|
|
190
|
+
import subprocess
|
|
191
|
+
|
|
192
|
+
entry = resolved[-1] if resolved else ""
|
|
193
|
+
try:
|
|
194
|
+
proc = subprocess.run(
|
|
195
|
+
resolved + (["doctor"] if entry.endswith((".js", ".cmd", ".bat")) or "smart-search" in str(resolved) else []),
|
|
196
|
+
capture_output=True,
|
|
197
|
+
text=True,
|
|
198
|
+
timeout=12,
|
|
199
|
+
cwd=str(repo_root),
|
|
200
|
+
encoding="utf-8",
|
|
201
|
+
errors="replace",
|
|
202
|
+
)
|
|
203
|
+
doctor_output = (proc.stdout or "") + (proc.stderr or "")
|
|
204
|
+
doctor_status = "ok" if proc.returncode == 0 else f"exit={proc.returncode}"
|
|
205
|
+
except (subprocess.SubprocessError, OSError) as exc:
|
|
206
|
+
doctor_status = f"error: {type(exc).__name__}"
|
|
207
|
+
doctor_output = str(exc)
|
|
208
|
+
|
|
209
|
+
ok = bool(resolved)
|
|
210
|
+
return {
|
|
211
|
+
"id": "P-07",
|
|
212
|
+
"label": "smart-search CLI — resolve + doctor",
|
|
213
|
+
"status": "pass" if ok else "fail",
|
|
214
|
+
"expected": "smart-search argv resolved",
|
|
215
|
+
"resolved_argv": resolved,
|
|
216
|
+
"resolve_error": resolve_error,
|
|
217
|
+
"doctor_status": doctor_status,
|
|
218
|
+
"doctor_output_excerpt": doctor_output[:400],
|
|
219
|
+
"note": "Local CLI; not routed through BYOK proxy." if ok else "smart-search not resolvable.",
|
|
220
|
+
}
|
|
221
|
+
|
|
222
|
+
|
|
223
|
+
def _probe_d01_semanticsearch_drop(
|
|
224
|
+
env_info: dict[str, object],
|
|
225
|
+
mcp_config: dict[str, object],
|
|
226
|
+
) -> dict[str, object]:
|
|
227
|
+
"""D-01 (auto): Experiment D readiness — BYOK needs fast-context in mcp.json.
|
|
228
|
+
|
|
229
|
+
Does not prove SemanticSearch absence (that is manual D-01 + P-08). Pass when:
|
|
230
|
+
- Native: env is native (built-in semantic path expected in manual probes).
|
|
231
|
+
- BYOK: fast-context configured in .cursor/mcp.json.
|
|
232
|
+
"""
|
|
233
|
+
env = str(env_info.get("env") or ENV_UNKNOWN)
|
|
234
|
+
byok_mode = env_info.get("byokMode")
|
|
235
|
+
fc = bool(mcp_config.get("fast_context_configured"))
|
|
236
|
+
cg = bool(mcp_config.get("codegraph_configured"))
|
|
237
|
+
if env == ENV_BYOK:
|
|
238
|
+
ok = fc and cg
|
|
239
|
+
status = "pass" if ok else "fail"
|
|
240
|
+
note = (
|
|
241
|
+
"BYOK: fast-context + codegraph in mcp.json — ready for P-08-SA / REC-11."
|
|
242
|
+
if ok
|
|
243
|
+
else "BYOK: missing fast-context and/or codegraph in mcp.json; "
|
|
244
|
+
"select codebase-retrieval at init or add servers manually."
|
|
245
|
+
)
|
|
246
|
+
elif env == ENV_NATIVE:
|
|
247
|
+
ok = True
|
|
248
|
+
status = "pass"
|
|
249
|
+
note = (
|
|
250
|
+
"Native: use manual D-01 / P-08 for built-in SemanticSearch inventory; "
|
|
251
|
+
f"fast-context configured={fc} (optional on Native)."
|
|
252
|
+
)
|
|
253
|
+
else:
|
|
254
|
+
ok = fc
|
|
255
|
+
status = "degraded" if fc else "fail"
|
|
256
|
+
note = (
|
|
257
|
+
"cursorEnv unknown; fast-context in mcp.json supports BYOK concept recall if env is byok."
|
|
258
|
+
if fc
|
|
259
|
+
else "cursorEnv unknown and fast-context not in mcp.json."
|
|
260
|
+
)
|
|
261
|
+
return {
|
|
262
|
+
"id": "D-01",
|
|
263
|
+
"label": "Experiment D — SemanticSearch drop readiness (env + MCP)",
|
|
264
|
+
"status": status,
|
|
265
|
+
"expected": "BYOK → fast-context+codegraph in mcp.json; Native → env native",
|
|
266
|
+
"cursor_env": env,
|
|
267
|
+
"byokMode": byok_mode,
|
|
268
|
+
"fast_context_configured": fc,
|
|
269
|
+
"codegraph_configured": cg,
|
|
270
|
+
"manual_followup": "cursor_retrieval_probe_prompt.md PROBE D-01 (tool inventory)",
|
|
271
|
+
"related_probes": ["P-08", "P-08-SA"],
|
|
272
|
+
"note": note,
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
|
|
276
|
+
def _probe_mcp_config(repo_root: Path) -> dict[str, object]:
|
|
277
|
+
"""Supplemental: read .cursor/mcp.json to report configured MCP servers.
|
|
278
|
+
|
|
279
|
+
Tells the user which MCP-based probes (P-03/04/06) should be runnable.
|
|
280
|
+
"""
|
|
281
|
+
target = repo_root / ".cursor" / "mcp.json"
|
|
282
|
+
servers: dict[str, object] = {}
|
|
283
|
+
if target.is_file():
|
|
284
|
+
try:
|
|
285
|
+
data = json.loads(target.read_text(encoding="utf-8"))
|
|
286
|
+
mcp = data.get("mcpServers", {})
|
|
287
|
+
if isinstance(mcp, dict):
|
|
288
|
+
servers = mcp
|
|
289
|
+
except (OSError, json.JSONDecodeError):
|
|
290
|
+
pass
|
|
291
|
+
return {
|
|
292
|
+
"codegraph_configured": "codegraph" in servers,
|
|
293
|
+
"fast_context_configured": "fast-context" in servers,
|
|
294
|
+
"all_servers": sorted(servers.keys()),
|
|
295
|
+
"mcp_json_path": str(target),
|
|
296
|
+
}
|
|
297
|
+
|
|
298
|
+
|
|
299
|
+
def run_auto_probes(repo_root: Path) -> dict[str, object]:
|
|
300
|
+
env_info = detect_environment()
|
|
301
|
+
mcp_config = _probe_mcp_config(repo_root)
|
|
302
|
+
return {
|
|
303
|
+
"probe_version": PROBE_VERSION,
|
|
304
|
+
"env": env_info,
|
|
305
|
+
"probed_at": _utc_now(),
|
|
306
|
+
"repo_root": str(repo_root),
|
|
307
|
+
"auto_results": [
|
|
308
|
+
_probe_grep(repo_root),
|
|
309
|
+
_probe_read(repo_root),
|
|
310
|
+
_probe_codegraph_index(repo_root),
|
|
311
|
+
_probe_smart_search(repo_root),
|
|
312
|
+
_probe_d01_semanticsearch_drop(env_info, mcp_config),
|
|
313
|
+
],
|
|
314
|
+
"mcp_config": mcp_config,
|
|
315
|
+
"manual_probe_ids": ["P-03", "P-04", "P-06", "D-01", "P-08", "P-09", "P-10"],
|
|
316
|
+
"manual_probe_doc": "cursor_retrieval_probe_prompt.md",
|
|
317
|
+
"matrix_template": "retrieval_probe_matrix_template.json",
|
|
318
|
+
}
|
|
319
|
+
|
|
320
|
+
|
|
321
|
+
def _format_human(report: dict[str, object]) -> str:
|
|
322
|
+
env_info = report.get("env", {})
|
|
323
|
+
env = env_info.get("env", "unknown")
|
|
324
|
+
lines = [
|
|
325
|
+
f"Cursor retrieval probe (Phase 1)",
|
|
326
|
+
f" environment : {env} (source: {env_info.get('source', '?')}, byokMode={env_info.get('byokMode')})",
|
|
327
|
+
f" repo_root : {report.get('repo_root')}",
|
|
328
|
+
f" probed_at : {report.get('probed_at')}",
|
|
329
|
+
"",
|
|
330
|
+
"Auto probes (CLI-reachable):",
|
|
331
|
+
]
|
|
332
|
+
for r in report.get("auto_results", []):
|
|
333
|
+
status = r.get("status", "?")
|
|
334
|
+
tag = "PASS" if status == "pass" else "FAIL"
|
|
335
|
+
lines.append(f" [{tag}] {r.get('id')} {r.get('label')}")
|
|
336
|
+
if r.get("note"):
|
|
337
|
+
lines.append(f" {r['note']}")
|
|
338
|
+
mcp = report.get("mcp_config", {})
|
|
339
|
+
lines.append("")
|
|
340
|
+
lines.append("MCP config (for manual probes P-03/04/06 and D-01 BYOK readiness):")
|
|
341
|
+
lines.append(f" codegraph : {'configured' if mcp.get('codegraph_configured') else 'MISSING'}")
|
|
342
|
+
lines.append(f" fast-context : {'configured' if mcp.get('fast_context_configured') else 'MISSING'}")
|
|
343
|
+
lines.append(f" all servers : {', '.join(mcp.get('all_servers', [])) or '(none)'}")
|
|
344
|
+
lines.append("")
|
|
345
|
+
lines.append("Manual probes (run via cursor_retrieval_probe_prompt.md):")
|
|
346
|
+
for pid in report.get("manual_probe_ids", []):
|
|
347
|
+
lines.append(f" {pid}")
|
|
348
|
+
lines.append("")
|
|
349
|
+
redirect = env_info.get("redirect_endpoints", [])
|
|
350
|
+
if env == "byok" and isinstance(redirect, list) and redirect:
|
|
351
|
+
lines.append(f"BYOK redirect endpoints ({len(redirect)}):")
|
|
352
|
+
for ep in redirect[:8]:
|
|
353
|
+
lines.append(f" - {ep}")
|
|
354
|
+
if len(redirect) > 8:
|
|
355
|
+
lines.append(f" ... +{len(redirect) - 8} more")
|
|
356
|
+
lines.append("")
|
|
357
|
+
return "\n".join(lines)
|
|
358
|
+
|
|
359
|
+
|
|
360
|
+
def main() -> int:
|
|
361
|
+
parser = argparse.ArgumentParser(description="Cursor retrieval capability probe (Phase 1)")
|
|
362
|
+
parser.add_argument("--root", type=Path, default=None, help="Workspace root (default: Trellis repo root)")
|
|
363
|
+
parser.add_argument("--json", action="store_true", help="Machine-readable JSON report")
|
|
364
|
+
parser.add_argument("--out", type=Path, default=None, help="Write report JSON to this path")
|
|
365
|
+
args = parser.parse_args()
|
|
366
|
+
|
|
367
|
+
if args.root is not None:
|
|
368
|
+
repo_root = args.root.resolve()
|
|
369
|
+
else:
|
|
370
|
+
try:
|
|
371
|
+
from common.paths import get_repo_root # type: ignore[import-not-found]
|
|
372
|
+
repo_root = get_repo_root()
|
|
373
|
+
except Exception:
|
|
374
|
+
repo_root = Path.cwd().resolve()
|
|
375
|
+
|
|
376
|
+
report = run_auto_probes(repo_root)
|
|
377
|
+
|
|
378
|
+
if args.out:
|
|
379
|
+
args.out.parent.mkdir(parents=True, exist_ok=True)
|
|
380
|
+
args.out.write_text(
|
|
381
|
+
json.dumps(report, indent=2, ensure_ascii=False) + "\n",
|
|
382
|
+
encoding="utf-8",
|
|
383
|
+
)
|
|
384
|
+
print(f"wrote {args.out}", file=sys.stderr)
|
|
385
|
+
|
|
386
|
+
if args.json:
|
|
387
|
+
print(json.dumps(report, indent=2, ensure_ascii=False))
|
|
388
|
+
else:
|
|
389
|
+
print(_format_human(report))
|
|
390
|
+
|
|
391
|
+
any_fail = any(r.get("status") != "pass" for r in report.get("auto_results", []))
|
|
392
|
+
return 1 if any_fail and args.json else 0
|
|
393
|
+
|
|
394
|
+
|
|
395
|
+
if __name__ == "__main__":
|
|
396
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,300 @@
|
|
|
1
|
+
# Cursor retrieval capability probe — manual prompts (Phase 1)
|
|
2
|
+
|
|
3
|
+
> **Purpose:** Measure retrieval capabilities that require a live agent session
|
|
4
|
+
> (MCP tools, built-in semantic search, LSP). Run each probe in **both**
|
|
5
|
+
> environments and record results in `retrieval_probe_matrix_template.json`.
|
|
6
|
+
>
|
|
7
|
+
> **Environment A (Native):** ccursor proxy OFF (or `byokMode=0`).
|
|
8
|
+
> **Environment B (BYOK):** ccursor proxy ON, `byokMode=1`.
|
|
9
|
+
|
|
10
|
+
## How to use this file
|
|
11
|
+
|
|
12
|
+
1. Switch to the target environment (Native or BYOK) in Cursor.
|
|
13
|
+
2. Copy the **probe block** (from `--- PROBE` to the next `---`) into a fresh
|
|
14
|
+
Cursor Agent chat. Use the same block in both environments.
|
|
15
|
+
3. The agent answers with a structured reply; copy its answer into the matching
|
|
16
|
+
`native`/`byok` slot of the matrix JSON.
|
|
17
|
+
4. Do **not** coach the agent. It must use whichever tools it actually has.
|
|
18
|
+
5. Record the **actual tool name** the agent called (from its tool-use output),
|
|
19
|
+
not the name we asked for — this is the BYOK detection signal.
|
|
20
|
+
|
|
21
|
+
## Deterministic known answers
|
|
22
|
+
|
|
23
|
+
Every probe has a verifiable answer derived from the `D:\MyHarness` workspace.
|
|
24
|
+
The agent either returns the right answer (pass) or does not (fail/degraded).
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
--- PROBE P-03: codegraph MCP — codegraph_search (symbol lookup)
|
|
29
|
+
|
|
30
|
+
You are a retrieval capability probe. Do NOT read any files directly. Use ONLY
|
|
31
|
+
the **codegraph_search** MCP tool to answer.
|
|
32
|
+
|
|
33
|
+
Task: Call `codegraph_search` with the query `patch_wpelc8` and report:
|
|
34
|
+
1. The file path(s) returned (expected to contain `patch_wpelc8.py` under
|
|
35
|
+
`.trellis/local/cursor2plus/`).
|
|
36
|
+
2. The symbol kind (expected: `function`).
|
|
37
|
+
3. The exact tool name you invoked (from your tool-use record).
|
|
38
|
+
|
|
39
|
+
Reply in this exact format:
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
P-03 RESULT
|
|
43
|
+
tool_invoked: <exact tool name from your tool-use record>
|
|
44
|
+
file_path: <path returned>
|
|
45
|
+
symbol_kind: <kind returned>
|
|
46
|
+
verdict: pass # pass if patch_wpelc8.py appears, else fail
|
|
47
|
+
notes: <anything notable>
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
--- PROBE P-04: codegraph MCP — codegraph_callers (caller chain)
|
|
53
|
+
|
|
54
|
+
You are a retrieval capability probe. Do NOT read any files directly. Use ONLY
|
|
55
|
+
the **codegraph_callers** MCP tool to answer.
|
|
56
|
+
|
|
57
|
+
Task: Call `codegraph_callers` with the symbol `route_codebase_retrieval` and
|
|
58
|
+
report:
|
|
59
|
+
1. The list of caller sites returned (expected: at least one caller in
|
|
60
|
+
`route_codebase_retrieval.py` under `.trellis/scripts/`).
|
|
61
|
+
2. The count of callers.
|
|
62
|
+
3. The exact tool name you invoked.
|
|
63
|
+
|
|
64
|
+
Reply in this exact format:
|
|
65
|
+
|
|
66
|
+
```
|
|
67
|
+
P-04 RESULT
|
|
68
|
+
tool_invoked: <exact tool name from your tool-use record>
|
|
69
|
+
caller_count: <number>
|
|
70
|
+
caller_sites: <semicolon-separated file:line list, or "none">
|
|
71
|
+
verdict: pass # pass if at least 1 caller found, else fail
|
|
72
|
+
notes: <anything notable>
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
--- PROBE P-06: fast-context MCP — semantic search
|
|
78
|
+
|
|
79
|
+
You are a retrieval capability probe. Do NOT read any files directly. Use ONLY
|
|
80
|
+
the **fast-context** MCP tool (fast_context_search) to answer.
|
|
81
|
+
|
|
82
|
+
Task: Call the fast-context semantic search with the query:
|
|
83
|
+
`Cursor++ BYOK subagent model routing patch`
|
|
84
|
+
|
|
85
|
+
Report:
|
|
86
|
+
1. Whether the tool returned any results.
|
|
87
|
+
2. If so, the top file path (expected to contain something under
|
|
88
|
+
`.trellis/local/cursor2plus/` or `.trellis/spec/guides/cursor-subagent-policy.md`).
|
|
89
|
+
3. The exact tool name you invoked.
|
|
90
|
+
|
|
91
|
+
Reply in this exact format:
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
P-06 RESULT
|
|
95
|
+
tool_invoked: <exact tool name from your tool-use record>
|
|
96
|
+
result_count: <number>
|
|
97
|
+
top_file: <path or "none">
|
|
98
|
+
verdict: pass # pass if tool executed and returned results, else fail
|
|
99
|
+
notes: <anything notable>
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
--- PROBE D-01: Experiment D — codebase semantic tool inventory (manual)
|
|
105
|
+
|
|
106
|
+
**Experiment D** documents why **Cursor++ BYOK** often lacks built-in **SemanticSearch**
|
|
107
|
+
(P-08 `tool_invoked: none`) and why **fast_context_search** is the designed Primary.
|
|
108
|
+
|
|
109
|
+
Before running P-08, inventory what **this Agent session** can use for **codebase-wide
|
|
110
|
+
concept recall** (not Grep on a known string). Do **not** invoke tools yet — list from
|
|
111
|
+
your **available tool table / MCP list** only.
|
|
112
|
+
|
|
113
|
+
Also run locally (or ask the user to confirm):
|
|
114
|
+
`python .\.trellis\scripts\cursor_retrieval_probe.py --json` and note `env.env`,
|
|
115
|
+
`mcp_config.fast_context_configured`, and auto `D-01` status.
|
|
116
|
+
|
|
117
|
+
Reply in this exact format:
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
D-01 RESULT
|
|
121
|
+
cursor_env: native|byok|unknown # from probe JSON env.env or your knowledge of ccursor/BYOK
|
|
122
|
+
builtin_semantic_tools: <comma-separated names, or "none"> # e.g. SemanticSearch, @codebase
|
|
123
|
+
fast_context_mcp_listed: yes|no
|
|
124
|
+
fast_context_search_listed: yes|no
|
|
125
|
+
auto_d01_status: pass|fail|degraded|not_run # from probe auto_results D-01 if you ran it
|
|
126
|
+
verdict: pass|fail|degraded
|
|
127
|
+
notes: <BYOK expects builtin none + fast-context; Native expects SemanticSearch or equivalent>
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
verdict rules:
|
|
131
|
+
- **Native:** pass if `builtin_semantic_tools` includes SemanticSearch or equivalent built-in codebase semantic.
|
|
132
|
+
- **BYOK:** pass if `builtin_semantic_tools` is `none` (or empty) **and** `fast_context_search_listed` is yes **and** auto D-01 is pass when mcp.json has fast-context+codegraph.
|
|
133
|
+
- **degraded:** BYOK with fast-context MCP missing from config/list, or Native without built-in semantic but fast-context present.
|
|
134
|
+
- **fail:** cannot determine tool table, or BYOK with neither builtin nor fast-context path.
|
|
135
|
+
|
|
136
|
+
Pair with **P-08** (actual concept question) and **P-08-SA** (fast-context only).
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
--- PROBE P-08: @codebase semantic search (Cursor built-in)
|
|
141
|
+
|
|
142
|
+
You are a retrieval capability probe. Do NOT use Grep, Read, or any MCP tool.
|
|
143
|
+
Use ONLY Cursor's **built-in codebase / semantic search** (the `@codebase`
|
|
144
|
+
capability or any built-in semantic search tool the host exposes) to answer.
|
|
145
|
+
|
|
146
|
+
The question below is deliberately written with NO literal file names, symbol
|
|
147
|
+
names, or paths — it can only be answered by semantic understanding of the
|
|
148
|
+
codebase, not by literal string matching.
|
|
149
|
+
|
|
150
|
+
Semantic question:
|
|
151
|
+
> "Where does the system decide which model a background sub-agent will run on
|
|
152
|
+
> when the user is routing their own API keys through a local proxy instead of
|
|
153
|
+
> the official cloud?"
|
|
154
|
+
|
|
155
|
+
Expected answer (for verification, do NOT reveal this to the solver path):
|
|
156
|
+
the relevant logic lives in `.trellis/local/cursor2plus/patch_wpelc8.py` (the
|
|
157
|
+
`WPeLc8` function patch) and is documented in
|
|
158
|
+
`.trellis/spec/guides/cursor-subagent-policy.md` and
|
|
159
|
+
`.trellis/spec/guides/cursor-subagent-reverse-engineering-report.md`.
|
|
160
|
+
|
|
161
|
+
Reply in this exact format:
|
|
162
|
+
|
|
163
|
+
```
|
|
164
|
+
P-08 RESULT
|
|
165
|
+
tool_invoked: <exact tool name from your tool-use record — e.g. @codebase, codebase_search, SemanticSearch, or "none">
|
|
166
|
+
answer: <your answer to the semantic question>
|
|
167
|
+
matched_expected: yes|no # yes if answer references patch_wpelc8.py or cursor-subagent-policy.md
|
|
168
|
+
verdict: pass|fail|degraded
|
|
169
|
+
notes: <did the tool execute at all? any error? did you fall back to another tool?>
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
verdict rules:
|
|
173
|
+
- pass: built-in semantic tool executed AND answer references the expected files.
|
|
174
|
+
- degraded: tool executed but answer wrong, OR you had to fall back to Grep/codegraph.
|
|
175
|
+
- fail: built-in semantic tool did not execute / not available / errored out.
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
--- PROBE P-08-SA: Concept probe — fast-context only (BYOK Primary path)
|
|
180
|
+
|
|
181
|
+
You are a retrieval capability probe. Answer the semantic question below using
|
|
182
|
+
**only** the fast-context MCP tool **`fast_context_search`** (CallMcpTool on the
|
|
183
|
+
fast-context server). Do NOT use Grep, Read, SemanticSearch, @codebase,
|
|
184
|
+
codegraph_*, Task subagent, or WebSearch.
|
|
185
|
+
|
|
186
|
+
Use a natural-language query in English (add Chinese business terms only if helpful).
|
|
187
|
+
Set `project_path` to the workspace root (e.g. D:\MyHarness).
|
|
188
|
+
|
|
189
|
+
Semantic question (same as P-08 — no literal file/symbol/path clues in your query):
|
|
190
|
+
> "Where does the system decide which model a background sub-agent will run on
|
|
191
|
+
> when the user is routing their own API keys through a local proxy instead of
|
|
192
|
+
> the official cloud?"
|
|
193
|
+
|
|
194
|
+
Expected (for verification only — do not paste paths into fast_context_search query):
|
|
195
|
+
Top hits should include `.trellis/local/cursor2plus/patch_wpelc8.py` and/or
|
|
196
|
+
`.trellis/spec/guides/cursor-subagent-policy.md`.
|
|
197
|
+
|
|
198
|
+
Reply in this exact format:
|
|
199
|
+
|
|
200
|
+
```
|
|
201
|
+
P-08-SA RESULT
|
|
202
|
+
tool_invoked: fast_context_search|none|other
|
|
203
|
+
top_paths: <comma-separated paths from fast_context_search results, or "none">
|
|
204
|
+
matched_expected: yes|no # yes if top_paths includes patch_wpelc8.py or cursor-subagent-policy.md
|
|
205
|
+
verdict: pass|fail|degraded
|
|
206
|
+
notes: <errors, empty results, or if you had to use another tool>
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
verdict rules:
|
|
210
|
+
- pass: fast_context_search executed AND matched_expected yes.
|
|
211
|
+
- degraded: executed but wrong/empty top paths.
|
|
212
|
+
- fail: did not invoke fast_context_search, or tool missing in this session.
|
|
213
|
+
|
|
214
|
+
**Native:** optional control — pass confirms MCP works; Primary for concept recall
|
|
215
|
+
remains built-in SemanticSearch (P-08). **BYOK:** required — this is the designed
|
|
216
|
+
Primary when P-08 reports `tool_invoked: none`. Pair with D-01 inventory and P-08.
|
|
217
|
+
|
|
218
|
+
---
|
|
219
|
+
|
|
220
|
+
--- PROBE P-09: DEEP_SEARCH / wide cross-cutting semantic explore
|
|
221
|
+
|
|
222
|
+
You are a retrieval capability probe. Use Cursor's **DEEP_SEARCH** capability
|
|
223
|
+
(or the equivalent wide cross-cutting semantic explore tool the host exposes)
|
|
224
|
+
to answer. Do NOT use Grep or Read until after you have attempted DEEP_SEARCH.
|
|
225
|
+
|
|
226
|
+
Semantic question (no literal clues):
|
|
227
|
+
> "How does the workspace stitch together per-query retrieval plans, agent
|
|
228
|
+
> execution policy, and per-turn hook injection across Cursor, given that one
|
|
229
|
+
> of the injection channels is known-broken?"
|
|
230
|
+
|
|
231
|
+
Expected (for verification only): the answer should weave together
|
|
232
|
+
`.cursor/hooks/inject-retrieval-plan.py` (beforeSubmitPrompt injection),
|
|
233
|
+
`.cursor/rules/retrieval-routing.mdc` (alwaysApply policy), and the
|
|
234
|
+
`sessionStart` bug #158452 noted in `.trellis/spec/guides/cursor-context-injection-guide.md`.
|
|
235
|
+
|
|
236
|
+
Reply in this exact format:
|
|
237
|
+
|
|
238
|
+
```
|
|
239
|
+
P-09 RESULT
|
|
240
|
+
tool_invoked: <exact tool name — DEEP_SEARCH, Explore subagent, or "none">
|
|
241
|
+
answer: <your answer>
|
|
242
|
+
matched_expected: yes|no # yes if answer references at least 2 of: inject-retrieval-plan.py, retrieval-routing.mdc, sessionStart bug #158452
|
|
243
|
+
verdict: pass|fail|degraded
|
|
244
|
+
notes: <did DEEP_SEARCH run? did you fall back to subagent explore / Grep?>
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
verdict rules:
|
|
248
|
+
- pass: DEEP_SEARCH (or equivalent) executed AND answer references >=2 expected anchors.
|
|
249
|
+
- degraded: fell back to Explore subagent / codegraph / Grep, or answer partial.
|
|
250
|
+
- fail: no wide explore tool executed.
|
|
251
|
+
|
|
252
|
+
---
|
|
253
|
+
|
|
254
|
+
--- PROBE P-10: Definition / reference (codegraph — Agent LSP not exposed)
|
|
255
|
+
|
|
256
|
+
Cursor Agent does **not** expose GO_TO_DEFINITION in the tool table (Native/BYOK).
|
|
257
|
+
Trellis routes definition/reference to **codegraph**. This probe measures the
|
|
258
|
+
**product path**, not raw LSP.
|
|
259
|
+
|
|
260
|
+
You are a retrieval capability probe. Use **codegraph_node** or **codegraph_search**
|
|
261
|
+
as the **primary** path. Do NOT use Grep as the first step. Use **Read** only to
|
|
262
|
+
confirm line numbers after codegraph returns a definition.
|
|
263
|
+
|
|
264
|
+
Task: Locate the **definition** of `route_codebase_retrieval` (Python function).
|
|
265
|
+
|
|
266
|
+
Expected: definition in `.trellis/scripts/common/codebase_retrieval_router.py`
|
|
267
|
+
(`def route_codebase_retrieval(`).
|
|
268
|
+
|
|
269
|
+
Reply in this exact format:
|
|
270
|
+
|
|
271
|
+
```
|
|
272
|
+
P-10 RESULT
|
|
273
|
+
tool_invoked: <exact tool name — codegraph_node, codegraph_search, or "none">
|
|
274
|
+
definition_file: <path>
|
|
275
|
+
definition_line: <line number or "unknown">
|
|
276
|
+
verdict: pass|fail|environment_limitation
|
|
277
|
+
notes: <if you tried GO_TO_DEFINITION and it was unavailable, say so; pass when codegraph + Read confirm def>
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
verdict rules:
|
|
281
|
+
- **pass**: codegraph (or equivalent MCP) returned the correct `def route_codebase_retrieval` file.
|
|
282
|
+
- **environment_limitation**: only when codegraph MCP/index is missing; not when LSP is missing.
|
|
283
|
+
- **fail**: wrong file or no structural tool executed.
|
|
284
|
+
|
|
285
|
+
---
|
|
286
|
+
|
|
287
|
+
## After running all probes
|
|
288
|
+
|
|
289
|
+
For each probe, fill the matching `native` and `byok` objects in
|
|
290
|
+
`retrieval_probe_matrix_template.json`:
|
|
291
|
+
|
|
292
|
+
```json
|
|
293
|
+
"native": { "status": "pass", "evidence": "<answer excerpt>", "actual_tool": "<tool_invoked>", "probed_at": "<ISO8601>" }
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
Then set `verdict.summary`, `verdict.degraded_capabilities` (list of probe IDs
|
|
297
|
+
where byok differs from native), and `verdict.probed_at`.
|
|
298
|
+
|
|
299
|
+
Degraded capabilities drive Phase 2 code adaptations — only those that actually
|
|
300
|
+
differ between environments get changed.
|