agentic-scorecard 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENT_PROMPT.md CHANGED
@@ -1,4 +1,4 @@
1
- # Ask an AI coding agent to run ADRB v0.2
1
+ # Ask an AI coding agent to run ADRB v0.3
2
2
 
3
3
  These prompts are vendor-neutral. The agent needs terminal access to the repository and permission
4
4
  to download the published npm package. The default benchmark runs locally, reads only tracked paths,
@@ -12,50 +12,54 @@ Copy and paste:
12
12
  Assess this repository's readiness for AI-agent pull-request work.
13
13
 
14
14
  From the repository root, run:
15
- npx --yes agentic-scorecard@0.2.0 assess . --profile pr-creation --scope tracked --format markdown --output .agentic/reports/agentic-readiness-v0.2.0.md
15
+ npx --yes agentic-scorecard@0.3.0 assess . --profile pr-creation --scope tracked --format markdown --output .agentic/reports/agentic-readiness-v0.3.0.md
16
16
 
17
17
  Do not change product source, configuration, policies, or tests. Do not invent evidence or mark an
18
18
  attestation as met. Read the report and tell me:
19
- 1. the score and highest passed readiness profile;
20
- 2. whether the pr-creation target passes;
19
+ 1. the normative score out of 40 and the repository-detected progress against its offline ceiling;
20
+ 2. the highest passed readiness profile and whether pr-creation passes;
21
21
  3. each target-profile blocker and its evidence scope;
22
22
  4. the five highest-value improvements in dependency order;
23
23
  5. which findings are repository gaps versus external or outcome evidence not yet established; and
24
24
  6. any likely false positive or false negative.
25
25
 
26
- Keep repository-detected, agent-collected, and human-attested evidence separate. Leave generated
27
- artifacts uncommitted unless I explicitly ask otherwise. If the command cannot run, explain the
28
- exact blocker without modifying the repository to work around it.
26
+ Keep repository-detected, agent-collected, and human-attested evidence separate. Treat UNKNOWN as
27
+ unresolved, not failed. Leave generated artifacts uncommitted unless I explicitly ask otherwise. If
28
+ the command cannot run, explain the exact blocker without modifying the repository to work around
29
+ it.
29
30
  ```
30
31
 
31
- ## Agent-assisted external evidence
32
+ ## Agent-assisted unresolved evidence
32
33
 
33
34
  Use this only after reviewing the repository baseline:
34
35
 
35
36
  ```text
36
- Collect the external evidence that ADRB v0.2 could not establish from tracked repository files.
37
+ Investigate the controls that the ADRB v0.3 baseline could not establish.
37
38
 
38
- First run:
39
- npx --yes agentic-scorecard@0.2.0 init-evidence .
39
+ First ensure tracked files match HEAD and the repository has a commit, then run:
40
+ npx --yes agentic-scorecard@0.3.0 init-evidence .
40
41
 
41
42
  Read `.agentic/evidence-request.md` and the empty target-bound
42
- `.agentic/agent-evidence.yaml` bundle. Before accessing any connected system, tell me which
43
- read-only tools, accounts, repositories, branches, dashboards, and time ranges you need. Wait for
44
- my authorization.
43
+ `.agentic/agent-evidence.yaml` bundle. The request includes only unresolved controls that permit
44
+ agent-collected evidence. Before accessing any connected system, tell me which read-only tools,
45
+ accounts, repositories, branches, dashboards, and time ranges you need. Wait for my authorization.
45
46
 
46
- After authorization, use only least-privileged read-only operations. For each eligible control,
47
+ After authorization, use only least-privileged read-only operations. For each attempted control,
47
48
  record the source-backed status, scope, concise derivation, collection time, 30-day-or-shorter
48
- expiry, and durable privacy-safe references. Add only claims you actually attempted. Do not paste
49
- credentials, prompts, source excerpts, raw logs, personal data, or sensitive dashboard contents. A
50
- permission error or inconclusive result must remain unknown and include the error; never infer a
51
- pass.
49
+ expiry, and durable privacy-safe references. For repository-scoped claims, cite only tracked files
50
+ at the bound commit using `repo:<path>[#Lx-Ly]`; do not use a repository claim to restate a keyword
51
+ match. Add only claims you actually investigated. A permission error or inconclusive result must
52
+ remain unknown and include the error; never infer a pass.
53
+
54
+ Do not paste credentials, prompts, source excerpts, raw logs, personal data, customer data, or
55
+ sensitive dashboard contents into the bundle.
52
56
 
53
57
  Then run:
54
- npx --yes agentic-scorecard@0.2.0 assess . --profile pr-creation --scope tracked --agent-evidence .agentic/agent-evidence.yaml --format markdown --output .agentic/reports/agentic-readiness-v0.2.0-assisted.md
58
+ npx --yes agentic-scorecard@0.3.0 assess . --profile pr-creation --scope tracked --agent-evidence .agentic/agent-evidence.yaml --format markdown --output .agentic/reports/agentic-readiness-v0.3.0-assisted.md
55
59
 
56
- Summarize every score change and keep agent-collected evidence distinct from repository evidence and
57
- human attestations. Do not describe the result as certified, compliant, safe, secure, or
58
- independently verified.
60
+ Summarize every score change and keep agent-collected evidence distinct from deterministic
61
+ repository evidence and human attestations. Do not describe the result as certified, compliant,
62
+ safe, secure, or independently verified.
59
63
  ```
60
64
 
61
65
  The agent is an evidence investigator. The scorecard remains the scoring authority.
@@ -75,7 +79,7 @@ No benchmark profile grants production deployment authority.
75
79
  ## Guided remediation
76
80
 
77
81
  ```text
78
- Read the latest ADRB v0.2 report and create a remediation proposal for the failed target-profile
82
+ Read the latest ADRB v0.3 report and create a remediation proposal for the failed target-profile
79
83
  controls. Do not implement changes yet. Group recommendations into repository changes, platform or
80
84
  organizational controls, and outcome measurement. For each include the control ID, risk reduced,
81
85
  systems affected, accountable role, verification method, dependencies, and effort estimate. Never
@@ -86,8 +90,9 @@ authorize.
86
90
  ## Reassessment
87
91
 
88
92
  ```text
89
- Re-run the pinned ADRB v0.2.0 benchmark for the same profile and tracked scope. Compare reports
90
- control by control. Explain changes using evidence and identify changed commit state, expired claims,
91
- scope differences, regressions, or newly established controls. Write a new dated report; do not
92
- overwrite the previous one.
93
+ Re-run the pinned ADRB v0.3.0 benchmark for the same profile and tracked scope. Compare reports
94
+ control by control. Explain changes using evidence and identify changed commit state, expired
95
+ claims, scope differences, regressions, or newly established controls. Compare normative and
96
+ repository-detected progress separately. Write a new dated report; do not overwrite the previous
97
+ one.
93
98
  ```
package/README.md CHANGED
@@ -16,9 +16,9 @@ The benchmark measures the harness, not the model brand. It assesses ten dimensi
16
16
  maturity levels and applies non-compensating floors to five autonomy profiles. A high total score
17
17
  cannot hide a critical security, testing, governance, or recovery gap.
18
18
 
19
- This repository is a **v0.2 reference implementation** intended for public review and piloting. It
20
- is not a certification standard. The immutable v0.1 benchmark remains available for historical
21
- reproduction; v0.1 and v0.2 scores are not directly comparable.
19
+ This repository is a **v0.3 reference implementation** intended for public review and piloting. It
20
+ is not a certification standard. The immutable v0.1 and v0.2 benchmarks remain available for
21
+ historical reproduction; scores from different benchmark versions are not directly comparable.
22
22
 
23
23
  ## Ask your coding agent
24
24
 
@@ -26,7 +26,7 @@ You do not need to install or learn the CLI yourself. Paste this prompt into Cod
26
26
  GitHub Copilot, Cursor, or another coding agent that has terminal access to your repository:
27
27
 
28
28
  > Assess this repository's readiness for AI-agent pull-request work. From the repository root, run
29
- > `npx --yes agentic-scorecard@0.2.0 assess . --profile pr-creation --scope tracked --format markdown --output .agentic/reports/agentic-readiness-v0.2.0.md`.
29
+ > `npx --yes agentic-scorecard@0.3.0 assess . --profile pr-creation --scope tracked --format markdown --output .agentic/reports/agentic-readiness-v0.3.0.md`.
30
30
  > Do not change product source code or invent attestations. Read the resulting report and summarize
31
31
  > the score, highest passed profile, whether `pr-creation` passes, target-profile blockers, and the
32
32
  > five highest-value improvements. Separate repository gaps from external and outcome evidence, and
@@ -70,7 +70,7 @@ A supported Node.js LTS release (20.19+, 22.13+, or 24+) is required. Assessment
70
70
  read-only, and offline by default.
71
71
 
72
72
  ```bash
73
- npx agentic-scorecard@0.2.0 assess /path/to/repository \
73
+ npx agentic-scorecard@0.3.0 assess /path/to/repository \
74
74
  --profile pr-creation \
75
75
  --scope tracked \
76
76
  --format markdown \
@@ -80,7 +80,7 @@ npx agentic-scorecard@0.2.0 assess /path/to/repository \
80
80
  To record controls that repository inspection cannot prove:
81
81
 
82
82
  ```bash
83
- npx agentic-scorecard@0.2.0 init /path/to/repository
83
+ npx agentic-scorecard@0.3.0 init /path/to/repository
84
84
  ```
85
85
 
86
86
  Complete `.agentic/attestations.yaml` with owners and durable evidence links, then assess again.
@@ -91,7 +91,7 @@ external systems, first generate a target-bound template. The repository must be
91
91
  with at least one commit so the bundle can bind to the exact assessed state:
92
92
 
93
93
  ```bash
94
- npx agentic-scorecard@0.2.0 init-evidence /path/to/repository
94
+ npx agentic-scorecard@0.3.0 init-evidence /path/to/repository
95
95
  ```
96
96
 
97
97
  Ask the agent to review `.agentic/evidence-request.md`, obtain approval before using
@@ -99,19 +99,27 @@ least-privileged read-only connectors, add attempted claims to `.agentic/agent-e
99
99
  rerun with `--agent-evidence`. The default bundle path is loaded automatically. See
100
100
  [AGENT_PROMPT.md](AGENT_PROMPT.md) for the complete copy-and-paste workflow.
101
101
 
102
- ### Migrating from v0.1
102
+ ### Migrating from v0.1 or v0.2
103
103
 
104
104
  Run a new tracked-scope baseline and retain the old report as historical evidence. Do not present the
105
- score change as improvement or regression because v0.2 changes evidence semantics. Re-review v0.1
106
- attestations before recreating them for v0.2; v0.2 repository-artifact controls cannot be overridden
107
- by declaration, and every human or agent-collected external claim must expire.
105
+ score change as improvement or regression because v0.3 changes evidence and control semantics.
106
+ Re-review prior attestations and agent evidence before recreating them for v0.3. All claims must
107
+ expire, and repository-scoped semantic claims require tracked files to match the commit-bound target
108
+ plus `repo:<path>[#Lx-Ly]` references whose optional line ranges exist in the cited tracked file.
109
+ Untracked generated reports do not block this workflow.
110
+
111
+ The default `.agentic/attestations.yaml` and `.agentic/agent-evidence.yaml` paths are migration-safe:
112
+ if either contains an older benchmark version, v0.3 ignores that auto-loaded file and places a
113
+ prominent warning in the report. Regenerate it with `init --force` or `init-evidence --force` before
114
+ relying on those claims. An explicitly supplied `--attestations` or `--agent-evidence` file still
115
+ fails closed on a version mismatch.
108
116
 
109
117
  For development from this checkout:
110
118
 
111
119
  ```bash
112
120
  npm ci
113
121
  npm run check
114
- npm run dev -- assess tests/fixtures/mature --profile pr-creation
122
+ npm run dev -- assess tests/fixtures/mature --profile pr-creation --attestations tests/fixtures/mature/.agentic/attestations-v0.3.yaml
115
123
  ```
116
124
 
117
125
  ## What it assesses
@@ -157,22 +165,32 @@ No profile grants production deployment authority. Organizations should evaluate
157
165
  through a separate, system-specific safety case.
158
166
 
159
167
  The exact floors live in
160
- [`benchmark/v0.2/benchmark.yaml`](benchmark/v0.2/benchmark.yaml) and their rationale in
161
- [`benchmark/v0.2/scoring-policy.md`](benchmark/v0.2/scoring-policy.md).
168
+ [`benchmark/v0.3/benchmark.yaml`](benchmark/v0.3/benchmark.yaml) and their rationale in
169
+ [`benchmark/v0.3/scoring-policy.md`](benchmark/v0.3/scoring-policy.md).
162
170
 
163
171
  ## Evidence scopes and trust labels
164
172
 
165
173
  - **Repository-detected:** the local collector found qualifying evidence in the selected path scope.
166
174
  This proves an artifact match, not consistent practice or external enforcement.
167
- - **Agent-collected:** an authorized agent supplied a target-bound, expiring, source-backed external
168
- claim. It is not independently verified.
175
+ - **Agent-collected:** an authorized agent supplied a target-bound, expiring, source-backed semantic
176
+ repository or external claim for an explicitly eligible control. It is not independently
177
+ verified and is never relabelled as repository-detected.
169
178
  - **Human-attested:** an accountable owner supplied a dated evidence link or explanation.
170
179
  - **Unknown:** evidence is unavailable, expired, unauthorized, mismatched, or inconclusive.
171
180
 
172
181
  Controls also identify whether their evidence belongs in the repository, hosting platform,
173
182
  organization, or outcome systems. This prevents expected external unknowns from masquerading as
174
- missing files. Future independently conformant adapters require a separate protocol; v0.2 does not
175
- issue certification or independent-verification claims.
183
+ missing files. Declarative detector adapters recognize common harness layouts without changing
184
+ portable control outcomes; future independently conformant external adapters require a separate
185
+ protocol. v0.3 does not issue certification or independent-verification claims.
186
+
187
+ ### Interpreting the two score views
188
+
189
+ The normative score remains `N/40` and drives readiness profiles. v0.3 also reports
190
+ `repository-detected progress: A/C`, where `C` is the maximum consecutive maturity the offline
191
+ collector can establish without platform, organization, outcome, agent-collected, or human-attested
192
+ evidence. This second view explains how complete the visible repository harness is; it does not
193
+ normalize away UNKNOWNs, change readiness floors, or certify the project.
176
194
 
177
195
  ## Reports and CI
178
196
 
@@ -200,7 +218,8 @@ The default collector:
200
218
  - reads only Git-tracked paths and records commit and dirty-worktree metadata;
201
219
  - ignores `.git`, dependencies, build output, coverage, generated reports, attestations, and imported
202
220
  evidence bundles;
203
- - caps content-scanned files at 512 KB;
221
+ - caps content scanning at 512 KB per file, 5 MB total, and 250 candidates per check, with v0.3
222
+ per-pattern balancing where configured;
204
223
  - reports file paths and match counts, never matching source snippets;
205
224
  - writes nothing unless `--output`, `init`, or `init-evidence` is explicitly requested.
206
225
 
@@ -211,7 +230,9 @@ access-controlled evidence instead. Report suspected vulnerabilities through [SE
211
230
 
212
231
  ```text
213
232
  benchmark/v0.1/ immutable historical v0.1 definition
214
- benchmark/v0.2/ current normative benchmark, schemas, and controls
233
+ benchmark/v0.2/ immutable historical v0.2 definition
234
+ benchmark/v0.3/ current normative benchmark, schemas, and controls
235
+ adapters/ declarative path/term aliases for recognized agent harness layouts
215
236
  benchmark/mappings/ informative mappings to external frameworks
216
237
  src/ reference CLI and local evidence collectors
217
238
  templates/ adoption, preflight, attestation, and remediation templates
@@ -240,8 +261,9 @@ system, and keep exceptions narrow, owned, expiring, and visible.
240
261
 
241
262
  ## Status and roadmap
242
263
 
243
- v0.2 adds integrity-safe tracked-path collection, evidence scopes, precise co-located content
244
- matching, agent-collected external evidence, and transparent report grouping. Candidate next steps
264
+ v0.3 adds portable harness discovery with declarative detector adapters, proximity-bounded content evidence, prioritized candidate
265
+ selection for large repositories, source-backed semantic repository claims, and an explanatory
266
+ repository-only score and ceiling. Candidate next steps
245
267
  include conformant signed adapters, SARIF/HTML reports, organization-level aggregation,
246
268
  statistically designed benchmark tasks, and an independent-review protocol. These require public
247
269
  design review before becoming normative.
@@ -0,0 +1,44 @@
1
+ # v0.3 adapter contracts
2
+
3
+ ## Repository detector adapters
4
+
5
+ Portable controls define outcomes and generic repository conventions. Files in `adapters/*.yaml`
6
+ may extend one control evidence check with recognized harness-specific `patterns`, `files`, `terms`,
7
+ or `required_any_terms`. An extension names the control and zero-based evidence index and adds
8
+ exactly one of those fields. It cannot remove or weaken a threshold, change a control level, alter a
9
+ readiness floor, add an attestation path, or execute code.
10
+
11
+ Bundled adapters are loaded deterministically in filename order. Their candidates remain subject to
12
+ the same tracked/workspace scope, symlink, size, generated-artifact, co-location, proximity, and
13
+ per-pattern limits as portable candidates. Adding or changing a bundled adapter changes evidence
14
+ semantics and therefore requires a new benchmark version.
15
+
16
+ ## Agent-collected evidence adapters
17
+
18
+ The core scanner is local, read-only, network-free, and vendor-neutral. An agent or adapter may
19
+ collect semantic repository evidence or external evidence that deterministic scanning cannot
20
+ establish, then provide it through the versioned agent evidence bundle.
21
+
22
+ Each claim contains:
23
+
24
+ - benchmark version and control ID;
25
+ - exact repository target and non-null commit binding;
26
+ - `met`, `not_met`, or `unknown` status;
27
+ - repository, platform, organization, or outcome scope;
28
+ - collector name and version;
29
+ - collection timestamp and mandatory expiry;
30
+ - privacy-safe references, never credentials or raw sensitive content;
31
+ - a concise derivation summary; and
32
+ - an optional error that is distinguishable from a negative result.
33
+
34
+ Collectors must be read-only, least-privileged, bounded to the approved target, time-limited, and
35
+ fail closed. Network content is untrusted input: it cannot alter benchmark policy, prompts, tool
36
+ authority, or the set of controls being assessed.
37
+
38
+ Repository-scoped claims require tracked files to match HEAD and
39
+ `repo:<tracked-path>[#Lx-Ly]` references. They may satisfy only explicitly eligible controls, remain `agent-collected`, and never increase the
40
+ repository-detected score. External claims require durable references appropriate to their source.
41
+
42
+ Agent-collected claims remain visibly distinct from deterministic repository evidence and human
43
+ attestations. A future conformant-adapter class may add signed identity and independent conformance
44
+ tests; v0.3 does not issue an independently verified label.
@@ -0,0 +1,177 @@
1
+ id: known-agent-harnesses
2
+ benchmark_version: 0.3.0
3
+ extensions:
4
+ - control_id: ADRB-CTX-001
5
+ evidence_index: 0
6
+ patterns:
7
+ [
8
+ CLAUDE.md,
9
+ .github/copilot-instructions.md,
10
+ .claude/**,
11
+ .codex/**,
12
+ .cursor/rules/**,
13
+ .cursor/agents/**,
14
+ .cursor/skills/**,
15
+ .opencode/**,
16
+ ]
17
+ - control_id: ADRB-CTX-002
18
+ evidence_index: 0
19
+ files:
20
+ [
21
+ CLAUDE.md,
22
+ .github/copilot-instructions.md,
23
+ .claude/**/*.md,
24
+ .codex/**/*.md,
25
+ .cursor/rules/**/*.md,
26
+ .cursor/agents/**/*.md,
27
+ .opencode/**/*.md,
28
+ ]
29
+ - control_id: ADRB-CTX-003
30
+ evidence_index: 0
31
+ required_any_terms: [CLAUDE.md, copilot instructions]
32
+ - control_id: ADRB-ENV-001
33
+ evidence_index: 0
34
+ files:
35
+ [
36
+ CLAUDE.md,
37
+ .claude/**/*.md,
38
+ .codex/**/*.md,
39
+ .cursor/agents/**/*.md,
40
+ .cursor/rules/**/*.md,
41
+ .cursor/skills/**/*.md,
42
+ .opencode/**/*.md,
43
+ ]
44
+ - control_id: ADRB-SPC-001
45
+ evidence_index: 0
46
+ patterns: [.kiro/specs/**, .cursor/plans/**, .codex/plans/**]
47
+ - control_id: ADRB-SPC-002
48
+ evidence_index: 0
49
+ files: [.kiro/specs/**, .cursor/plans/**, .codex/plans/**]
50
+ - control_id: ADRB-TOL-001
51
+ evidence_index: 0
52
+ patterns:
53
+ [
54
+ .claude/**,
55
+ .codex/**,
56
+ .cursor/**,
57
+ .opencode/**,
58
+ .github/copilot-instructions.md,
59
+ opencode.json,
60
+ ]
61
+ - control_id: ADRB-TOL-002
62
+ evidence_index: 0
63
+ files:
64
+ [
65
+ CLAUDE.md,
66
+ .claude/**/*.md,
67
+ .codex/**/*.md,
68
+ .cursor/agents/**/*.md,
69
+ .cursor/rules/**/*.md,
70
+ .cursor/skills/**/*.md,
71
+ .opencode/**/*.md,
72
+ ]
73
+ - control_id: ADRB-SEC-001
74
+ evidence_index: 0
75
+ files:
76
+ [
77
+ CLAUDE.md,
78
+ .claude/**/*.md,
79
+ .codex/**/*.md,
80
+ .cursor/agents/**/*.md,
81
+ .cursor/rules/**/*.md,
82
+ .cursor/skills/**/*.md,
83
+ .opencode/**/*.md,
84
+ ]
85
+ - control_id: ADRB-SEC-002
86
+ evidence_index: 0
87
+ files:
88
+ [
89
+ CLAUDE.md,
90
+ .claude/**/*.md,
91
+ .codex/**/*.md,
92
+ .cursor/agents/**/*.md,
93
+ .cursor/rules/**/*.md,
94
+ .cursor/skills/**/*.md,
95
+ .opencode/**/*.md,
96
+ ]
97
+ - control_id: ADRB-TST-002
98
+ evidence_index: 0
99
+ files:
100
+ [
101
+ CLAUDE.md,
102
+ .claude/**/*.md,
103
+ .codex/**/*.md,
104
+ .cursor/agents/**/*.md,
105
+ .cursor/rules/**/*.md,
106
+ .cursor/skills/**/*.md,
107
+ .opencode/**/*.md,
108
+ ]
109
+ - control_id: ADRB-GOV-001
110
+ evidence_index: 0
111
+ files:
112
+ [
113
+ CLAUDE.md,
114
+ .claude/**/*.md,
115
+ .codex/**/*.md,
116
+ .cursor/agents/**/*.md,
117
+ .cursor/rules/**/*.md,
118
+ .cursor/skills/**/*.md,
119
+ .opencode/**/*.md,
120
+ ]
121
+ - control_id: ADRB-GOV-002
122
+ evidence_index: 1
123
+ files:
124
+ [
125
+ CLAUDE.md,
126
+ .claude/**/*.md,
127
+ .codex/**/*.md,
128
+ .cursor/agents/**/*.md,
129
+ .cursor/rules/**/*.md,
130
+ .cursor/skills/**/*.md,
131
+ .opencode/**/*.md,
132
+ ]
133
+ - control_id: ADRB-LRN-001
134
+ evidence_index: 0
135
+ patterns: [.claude/memory/**, .codex/memory/**, .cursor/memory/**, .opencode/memory/**]
136
+ - control_id: ADRB-LRN-002
137
+ evidence_index: 0
138
+ files:
139
+ [
140
+ CLAUDE.md,
141
+ .claude/**/*.md,
142
+ .codex/**/*.md,
143
+ .cursor/agents/**/*.md,
144
+ .cursor/rules/**/*.md,
145
+ .cursor/skills/**/*.md,
146
+ .opencode/**/*.md,
147
+ ]
148
+ - control_id: ADRB-OBS-001
149
+ evidence_index: 0
150
+ files: [.cursor/runs/README*, .codex/runs/README*, .claude/runs/README*]
151
+ - control_id: ADRB-OBS-002
152
+ evidence_index: 0
153
+ files: [.cursor/runs/README*, .codex/runs/README*, .claude/runs/README*]
154
+ - control_id: ADRB-RES-001
155
+ evidence_index: 0
156
+ files:
157
+ [
158
+ CLAUDE.md,
159
+ .claude/**/*.md,
160
+ .codex/**/*.md,
161
+ .cursor/agents/**/*.md,
162
+ .cursor/rules/**/*.md,
163
+ .cursor/skills/**/*.md,
164
+ .opencode/**/*.md,
165
+ ]
166
+ - control_id: ADRB-RES-002
167
+ evidence_index: 0
168
+ files:
169
+ [
170
+ CLAUDE.md,
171
+ .claude/**/*.md,
172
+ .codex/**/*.md,
173
+ .cursor/agents/**/*.md,
174
+ .cursor/rules/**/*.md,
175
+ .cursor/skills/**/*.md,
176
+ .opencode/**/*.md,
177
+ ]
@@ -0,0 +1,18 @@
1
+ id: known-python-tooling
2
+ benchmark_version: 0.3.0
3
+ extensions:
4
+ - control_id: ADRB-ENV-001
5
+ evidence_index: 0
6
+ files: [pyproject.toml]
7
+ - control_id: ADRB-ENV-001
8
+ evidence_index: 0
9
+ terms: [requires-python, uv sync, python]
10
+ - control_id: ADRB-TST-002
11
+ evidence_index: 0
12
+ files: [run_quality_checks.sh, scripts/**/quality*.sh, tox.ini, noxfile.py]
13
+ - control_id: ADRB-TST-002
14
+ evidence_index: 0
15
+ terms: [pytest, mypy, flake8, ruff, pyright, tox, nox]
16
+ - control_id: ADRB-TST-003
17
+ evidence_index: 0
18
+ terms: [pytest, mypy, flake8, ruff, pyright, tox, nox, pytest-cov]
@@ -0,0 +1,6 @@
1
+ id: known-security-tools
2
+ benchmark_version: 0.3.0
3
+ extensions:
4
+ - control_id: ADRB-SEC-003
5
+ evidence_index: 0
6
+ terms: [gitleaks, trufflehog]
@@ -0,0 +1,67 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://agentic-readiness.dev/schema/v0.3/agent-evidence.json",
4
+ "title": "ADRB v0.3 agent-collected evidence bundle",
5
+ "type": "object",
6
+ "additionalProperties": false,
7
+ "required": ["schema_version", "benchmark_version", "target", "collector", "claims"],
8
+ "properties": {
9
+ "schema_version": { "const": "0.3.0" },
10
+ "benchmark_version": { "const": "0.3.0" },
11
+ "target": {
12
+ "type": "object",
13
+ "additionalProperties": false,
14
+ "required": ["repository", "git_head"],
15
+ "properties": {
16
+ "repository": { "type": "string", "minLength": 1 },
17
+ "git_head": { "type": "string", "minLength": 1 }
18
+ }
19
+ },
20
+ "collector": {
21
+ "type": "object",
22
+ "additionalProperties": false,
23
+ "required": ["name", "version"],
24
+ "properties": {
25
+ "name": { "type": "string", "minLength": 1 },
26
+ "version": { "type": "string", "minLength": 1 }
27
+ }
28
+ },
29
+ "claims": {
30
+ "type": "object",
31
+ "propertyNames": { "pattern": "^ADRB-[A-Z]{3}-[0-9]{3}$" },
32
+ "additionalProperties": {
33
+ "type": "object",
34
+ "additionalProperties": false,
35
+ "required": ["status", "scope", "summary", "references", "collected_at", "expires_at"],
36
+ "properties": {
37
+ "status": { "enum": ["met", "not_met", "unknown"] },
38
+ "scope": { "enum": ["repository", "platform", "organization", "outcome"] },
39
+ "summary": { "type": "string", "minLength": 1 },
40
+ "references": {
41
+ "type": "array",
42
+ "minItems": 1,
43
+ "items": { "type": "string", "minLength": 1 }
44
+ },
45
+ "collected_at": { "type": "string", "format": "date-time" },
46
+ "expires_at": { "type": "string", "format": "date-time" },
47
+ "error": { "type": ["string", "null"], "minLength": 1 }
48
+ },
49
+ "allOf": [
50
+ {
51
+ "if": { "properties": { "scope": { "const": "repository" } } },
52
+ "then": {
53
+ "properties": {
54
+ "references": {
55
+ "items": {
56
+ "type": "string",
57
+ "pattern": "^repo:(?!/)(?!\\.{1,2}(?:/|#|$))(?!.*\\/\\.{1,2}(?:/|#|$))(?!.*\\\\)[^#]+(?:#L[1-9][0-9]*(?:-L?[1-9][0-9]*)?)?$"
58
+ }
59
+ }
60
+ }
61
+ }
62
+ }
63
+ ]
64
+ }
65
+ }
66
+ }
67
+ }
@@ -0,0 +1,26 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://agentic-readiness.dev/schema/v0.3/attestations.json",
4
+ "title": "ADRB v0.3 human attestation file",
5
+ "type": "object",
6
+ "additionalProperties": false,
7
+ "required": ["benchmark_version", "attestations"],
8
+ "properties": {
9
+ "benchmark_version": { "const": "0.3.0" },
10
+ "attestations": {
11
+ "type": "object",
12
+ "additionalProperties": {
13
+ "type": "object",
14
+ "additionalProperties": false,
15
+ "required": ["status", "evidence", "owner", "reviewed_at", "expires_at"],
16
+ "properties": {
17
+ "status": { "enum": ["met", "not_met", "not_applicable", "unknown"] },
18
+ "evidence": { "type": "string", "minLength": 1 },
19
+ "owner": { "type": "string", "minLength": 1 },
20
+ "reviewed_at": { "type": "string", "format": "date" },
21
+ "expires_at": { "type": "string", "format": "date" }
22
+ }
23
+ }
24
+ }
25
+ }
26
+ }