agentic-scorecard 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENT_PROMPT.md CHANGED
@@ -1,10 +1,10 @@
1
- # Ask an AI coding agent to run the benchmark
1
+ # Ask an AI coding agent to run ADRB v0.2
2
2
 
3
- These prompts are deliberately vendor-neutral. The agent needs terminal access to the repository and
4
- permission to download the published npm package. The benchmark itself runs locally, invokes no
5
- model, uploads no source, and makes no network calls after package installation.
3
+ These prompts are vendor-neutral. The agent needs terminal access to the repository and permission
4
+ to download the published npm package. The default benchmark runs locally, reads only tracked paths,
5
+ invokes no model, uploads no source, and makes no network calls after installation.
6
6
 
7
- ## Recommended: first assessment
7
+ ## Recommended: repository baseline
8
8
 
9
9
  Copy and paste:
10
10
 
@@ -12,25 +12,57 @@ Copy and paste:
12
12
  Assess this repository's readiness for AI-agent pull-request work.
13
13
 
14
14
  From the repository root, run:
15
- npx --yes agentic-scorecard@0.1.0 assess . --profile pr-creation --format markdown --output .agentic/reports/agentic-readiness-v0.1.0.md
15
+ npx --yes agentic-scorecard@0.2.0 assess . --profile pr-creation --scope tracked --format markdown --output .agentic/reports/agentic-readiness-v0.2.0.md
16
16
 
17
- Do not change product source code, configuration, policies, or tests. Do not create or mark manual
18
- attestations as met. Read the generated report and tell me:
17
+ Do not change product source, configuration, policies, or tests. Do not invent evidence or mark an
18
+ attestation as met. Read the report and tell me:
19
19
  1. the score and highest passed readiness profile;
20
20
  2. whether the pr-creation target passes;
21
- 3. each target-profile blocker and the evidence that was or was not found;
22
- 4. the five highest-value improvements in dependency order; and
23
- 5. any likely false positive, false negative, or organizational control that repository inspection
24
- cannot prove.
25
-
26
- Keep tool-verified and self-attested evidence clearly separated. Leave the generated report
27
- uncommitted unless I explicitly ask you to commit it. If the command cannot run, explain the exact
28
- blocker without changing the repository to work around it.
21
+ 3. each target-profile blocker and its evidence scope;
22
+ 4. the five highest-value improvements in dependency order;
23
+ 5. which findings are repository gaps versus external or outcome evidence not yet established; and
24
+ 6. any likely false positive or false negative.
25
+
26
+ Keep repository-detected, agent-collected, and human-attested evidence separate. Leave generated
27
+ artifacts uncommitted unless I explicitly ask otherwise. If the command cannot run, explain the
28
+ exact blocker without modifying the repository to work around it.
29
29
  ```
30
30
 
31
+ ## Agent-assisted external evidence
32
+
33
+ Use this only after reviewing the repository baseline:
34
+
35
+ ```text
36
+ Collect the external evidence that ADRB v0.2 could not establish from tracked repository files.
37
+
38
+ First run:
39
+ npx --yes agentic-scorecard@0.2.0 init-evidence .
40
+
41
+ Read `.agentic/evidence-request.md` and the empty target-bound
42
+ `.agentic/agent-evidence.yaml` bundle. Before accessing any connected system, tell me which
43
+ read-only tools, accounts, repositories, branches, dashboards, and time ranges you need. Wait for
44
+ my authorization.
45
+
46
+ After authorization, use only least-privileged read-only operations. For each eligible control,
47
+ record the source-backed status, scope, concise derivation, collection time, 30-day-or-shorter
48
+ expiry, and durable privacy-safe references. Add only claims you actually attempted. Do not paste
49
+ credentials, prompts, source excerpts, raw logs, personal data, or sensitive dashboard contents. A
50
+ permission error or inconclusive result must remain unknown and include the error; never infer a
51
+ pass.
52
+
53
+ Then run:
54
+ npx --yes agentic-scorecard@0.2.0 assess . --profile pr-creation --scope tracked --agent-evidence .agentic/agent-evidence.yaml --format markdown --output .agentic/reports/agentic-readiness-v0.2.0-assisted.md
55
+
56
+ Summarize every score change and keep agent-collected evidence distinct from repository evidence and
57
+ human attestations. Do not describe the result as certified, compliant, safe, secure, or
58
+ independently verified.
59
+ ```
60
+
61
+ The agent is an evidence investigator. The scorecard remains the scoring authority.
62
+
31
63
  ## Choose a different authority level
32
64
 
33
- Replace `pr-creation` with exactly one of:
65
+ Replace `pr-creation` with one of:
34
66
 
35
67
  - `read-only-analysis` — inspect approved source and return advice;
36
68
  - `planning` — draft plans and specifications;
@@ -42,23 +74,20 @@ No benchmark profile grants production deployment authority.
42
74
 
43
75
  ## Guided remediation
44
76
 
45
- Use this only after reviewing the first report:
46
-
47
77
  ```text
48
- Read .agentic/reports/agentic-readiness-v0.1.0.md and create a remediation proposal for the failed
49
- target-profile controls. Do not implement changes yet. Group recommendations into quick wins,
50
- foundational engineering, and organizational evidence. For each recommendation include the ADRB
51
- control ID, risk reduced, files or systems likely affected, accountable role, verification method,
52
- dependencies, and an effort estimate. Identify claims requiring a human owner or access to external
53
- systems; never invent an attestation. End by asking which recommendations I authorize you to
54
- implement.
78
+ Read the latest ADRB v0.2 report and create a remediation proposal for the failed target-profile
79
+ controls. Do not implement changes yet. Group recommendations into repository changes, platform or
80
+ organizational controls, and outcome measurement. For each include the control ID, risk reduced,
81
+ systems affected, accountable role, verification method, dependencies, and effort estimate. Never
82
+ invent an attestation or convert unavailable evidence into a failure. End by asking which changes I
83
+ authorize.
55
84
  ```
56
85
 
57
- ## Reassessment after approved improvements
86
+ ## Reassessment
58
87
 
59
88
  ```text
60
- Re-run the pinned Agentic Development Readiness Benchmark v0.1.0 for the same readiness profile.
61
- Compare the new report to the previous report control by control. Explain score changes using evidence,
62
- not assumptions, and identify any regression or expired attestation. Do not overwrite the previous
63
- report; add the current date to the new report filename.
89
+ Re-run the pinned ADRB v0.2.0 benchmark for the same profile and tracked scope. Compare reports
90
+ control by control. Explain changes using evidence and identify changed commit state, expired claims,
91
+ scope differences, regressions, or newly established controls. Write a new dated report; do not
92
+ overwrite the previous one.
64
93
  ```
package/FEEDBACK.md ADDED
@@ -0,0 +1,53 @@
1
+ # Feedback guide
2
+
3
+ Feedback from real assessments is essential to improving the Agentic Development Readiness
4
+ Benchmark without weakening its evidence standards or vendor neutrality.
5
+
6
+ ## Choose the right channel
7
+
8
+ | Feedback | Channel |
9
+ | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
10
+ | False positive, false negative, unclear result, usability problem, or adoption experience | [Assessment feedback issue](https://github.com/Planet-B2B/agentic-readiness/issues/new?template=assessment-feedback.yml) |
11
+ | Proposed control, scoring, evidence, or readiness-profile change | [Benchmark change issue](https://github.com/Planet-B2B/agentic-readiness/issues/new?template=benchmark-change.yml) |
12
+ | Private adoption question or feedback that cannot be public | [benchmark@planetb2b.com](mailto:benchmark@planetb2b.com) |
13
+ | Suspected security vulnerability | Follow [SECURITY.md](SECURITY.md); do not open a public issue or email the benchmark contact |
14
+
15
+ Public issues are preferred when the information can be safely shared because they let other users
16
+ compare experiences and participate in the design review.
17
+
18
+ ## What useful feedback includes
19
+
20
+ - benchmark and package version;
21
+ - requested readiness profile and assessment scope;
22
+ - relevant control IDs;
23
+ - the observed result and the expected result;
24
+ - whether the concern is scanner behavior, control wording, evidence semantics, scoring policy, or
25
+ documentation;
26
+ - a minimal sanitized reproduction or description; and
27
+ - the material impact on adoption or decision-making.
28
+
29
+ Do not attach complete private reports. Never include credentials, proprietary source, private
30
+ URLs, personal data, customer data, or confidential dashboard content. A path name or summarized
31
+ evidence characteristic is usually enough to begin triage.
32
+
33
+ ## Ask a coding agent to prepare feedback
34
+
35
+ Paste this prompt into an agent that can read your assessment report:
36
+
37
+ > Review my Agentic Development Readiness report and draft feedback for the
38
+ > `Planet-B2B/agentic-readiness` repository. Include the benchmark version, target profile,
39
+ > assessment scope, relevant control IDs, observed result, expected result, and a minimal sanitized
40
+ > explanation. Classify the feedback as a likely scanner defect, documentation problem, adoption
41
+ > experience, or proposed normative benchmark change. Do not include credentials, proprietary
42
+ > source, private URLs, personal data, customer data, or full report contents. Show me the complete
43
+ > draft and ask for my approval before opening a GitHub issue. If I approve and you have GitHub
44
+ > access, use the assessment feedback issue form; otherwise give me the final text to submit.
45
+
46
+ Agents must treat opening an issue as an external write and obtain accountable human approval first.
47
+ Security concerns must follow `SECURITY.md` rather than this workflow.
48
+
49
+ ## Help others discover the benchmark
50
+
51
+ If this project helps your team, [star the repository](https://github.com/Planet-B2B/agentic-readiness).
52
+ Stars make the public benchmark easier for other engineering teams to discover; substantive feedback
53
+ and reproducible examples are even more valuable.
package/README.md CHANGED
@@ -1,3 +1,9 @@
1
+ ![Agentic Development Readiness](assets/brand/banner.png)
2
+
3
+ <p align="center">
4
+ <img src="assets/brand/avatar.png" width="96" height="96" alt="Agentic Development Readiness project mark">
5
+ </p>
6
+
1
7
  # Agentic Development Readiness Benchmark
2
8
 
3
9
  A vendor-neutral, evidence-backed benchmark for the engineering harness around AI coding agents.
@@ -10,8 +16,9 @@ The benchmark measures the harness, not the model brand. It assesses ten dimensi
10
16
  maturity levels and applies non-compensating floors to five autonomy profiles. A high total score
11
17
  cannot hide a critical security, testing, governance, or recovery gap.
12
18
 
13
- This repository is an early **v0.1 reference implementation** intended for public review and
14
- piloting. It is not a certification standard.
19
+ This repository is a **v0.2 reference implementation** intended for public review and piloting. It
20
+ is not a certification standard. The immutable v0.1 benchmark remains available for historical
21
+ reproduction; v0.1 and v0.2 scores are not directly comparable.
15
22
 
16
23
  ## Ask your coding agent
17
24
 
@@ -19,17 +26,42 @@ You do not need to install or learn the CLI yourself. Paste this prompt into Cod
19
26
  GitHub Copilot, Cursor, or another coding agent that has terminal access to your repository:
20
27
 
21
28
  > Assess this repository's readiness for AI-agent pull-request work. From the repository root, run
22
- > `npx --yes agentic-scorecard@0.1.0 assess . --profile pr-creation --format markdown --output .agentic/reports/agentic-readiness-v0.1.0.md`.
29
+ > `npx --yes agentic-scorecard@0.2.0 assess . --profile pr-creation --scope tracked --format markdown --output .agentic/reports/agentic-readiness-v0.2.0.md`.
23
30
  > Do not change product source code or invent attestations. Read the resulting report and summarize
24
31
  > the score, highest passed profile, whether `pr-creation` passes, target-profile blockers, and the
25
- > five highest-value improvements. Clearly separate tool-verified evidence from self-attested
26
- > evidence and flag likely false positives or missing organizational evidence. Leave the report
27
- > uncommitted unless I ask you to commit it.
32
+ > five highest-value improvements. Separate repository gaps from external and outcome evidence, and
33
+ > keep repository-detected, agent-collected, and human-attested evidence distinct. Flag likely false
34
+ > positives or negatives. Leave generated artifacts uncommitted unless I ask you to commit them.
28
35
 
29
36
  That is the recommended first assessment. The agent downloads the pinned benchmark package, runs the
30
37
  read-only local collector, and explains the results. For alternate authority levels or a guided
31
38
  remediation session, use the prompts in [AGENT_PROMPT.md](AGENT_PROMPT.md).
32
39
 
40
+ ## Feedback and community
41
+
42
+ Real-world assessment feedback helps make the benchmark more accurate and useful across different
43
+ engineering organizations and agent harnesses.
44
+
45
+ - Share false positives, false negatives, unclear controls, and adoption experience through the
46
+ [assessment feedback form](https://github.com/Planet-B2B/agentic-readiness/issues/new?template=assessment-feedback.yml).
47
+ - Propose scoring, control, evidence, or readiness-profile changes through the
48
+ [benchmark change form](https://github.com/Planet-B2B/agentic-readiness/issues/new?template=benchmark-change.yml).
49
+ - Send private adoption questions or feedback that cannot be shared publicly to
50
+ [benchmark@planetb2b.com](mailto:benchmark@planetb2b.com). Do not email vulnerability reports;
51
+ follow [SECURITY.md](SECURITY.md) instead.
52
+
53
+ Ask a coding agent to prepare privacy-safe feedback with this prompt:
54
+
55
+ > Review my Agentic Development Readiness report and draft feedback for the
56
+ > `Planet-B2B/agentic-readiness` repository. Include the benchmark version, target profile, relevant
57
+ > control IDs, observed result, expected result, and a minimal sanitized explanation. Distinguish a
58
+ > likely scanner defect from a proposed benchmark-policy change. Do not include credentials,
59
+ > proprietary source, private URLs, personal data, customer data, or full report contents. Show me
60
+ > the draft and ask for my approval before opening a GitHub issue.
61
+
62
+ See [FEEDBACK.md](FEEDBACK.md) for the full feedback guide. If the project is useful to you, please
63
+ [star the repository](https://github.com/Planet-B2B/agentic-readiness) so others can discover it.
64
+
33
65
  ## Quick start
34
66
 
35
67
  ### Installation and prerequisites
@@ -38,8 +70,9 @@ A supported Node.js LTS release (20.19+, 22.13+, or 24+) is required. Assessment
38
70
  read-only, and offline by default.
39
71
 
40
72
  ```bash
41
- npx agentic-scorecard@0.1.0 assess /path/to/repository \
73
+ npx agentic-scorecard@0.2.0 assess /path/to/repository \
42
74
  --profile pr-creation \
75
+ --scope tracked \
43
76
  --format markdown \
44
77
  --output agentic-readiness.md
45
78
  ```
@@ -47,11 +80,31 @@ npx agentic-scorecard@0.1.0 assess /path/to/repository \
47
80
  To record controls that repository inspection cannot prove:
48
81
 
49
82
  ```bash
50
- npx agentic-scorecard@0.1.0 init /path/to/repository
83
+ npx agentic-scorecard@0.2.0 init /path/to/repository
51
84
  ```
52
85
 
53
- Complete `.agentic/attestations.yaml` with owners and links to durable evidence, then assess again.
54
- Attested evidence remains visibly distinct from tool-verified evidence in every report.
86
+ Complete `.agentic/attestations.yaml` with owners and durable evidence links, then assess again.
87
+ Human-attested evidence remains visibly distinct in every report.
88
+
89
+ To let an authorized coding agent collect evidence from Git hosting, CI, dashboards, or other
90
+ external systems, first generate a target-bound template. The repository must be a Git worktree
91
+ with at least one commit so the bundle can bind to the exact assessed state:
92
+
93
+ ```bash
94
+ npx agentic-scorecard@0.2.0 init-evidence /path/to/repository
95
+ ```
96
+
97
+ Ask the agent to review `.agentic/evidence-request.md`, obtain approval before using
98
+ least-privileged read-only connectors, add attempted claims to `.agentic/agent-evidence.yaml`, and
99
+ rerun with `--agent-evidence`. The default bundle path is loaded automatically. See
100
+ [AGENT_PROMPT.md](AGENT_PROMPT.md) for the complete copy-and-paste workflow.
101
+
102
+ ### Migrating from v0.1
103
+
104
+ Run a new tracked-scope baseline and retain the old report as historical evidence. Do not present the
105
+ score change as improvement or regression because v0.2 changes evidence semantics. Re-review v0.1
106
+ attestations before recreating them for v0.2; v0.2 repository-artifact controls cannot be overridden
107
+ by declaration, and every human or agent-collected external claim must expire.
55
108
 
56
109
  For development from this checkout:
57
110
 
@@ -104,30 +157,32 @@ No profile grants production deployment authority. Organizations should evaluate
104
157
  through a separate, system-specific safety case.
105
158
 
106
159
  The exact floors live in
107
- [`benchmark/v0.1/benchmark.yaml`](benchmark/v0.1/benchmark.yaml) and their rationale in
108
- [`benchmark/v0.1/scoring-policy.md`](benchmark/v0.1/scoring-policy.md).
160
+ [`benchmark/v0.2/benchmark.yaml`](benchmark/v0.2/benchmark.yaml) and their rationale in
161
+ [`benchmark/v0.2/scoring-policy.md`](benchmark/v0.2/scoring-policy.md).
109
162
 
110
- ## Evidence and trust labels
163
+ ## Evidence scopes and trust labels
111
164
 
112
- - **Verified:** a deterministic local collector found the configured repository evidence.
113
- - **Attested:** an accountable owner supplied an evidence link or explanation. This may satisfy a
114
- control, but it never becomes verified evidence.
115
- - **Unmet/unknown:** evidence is absent, expired, contradictory, or still requires review.
116
- - **Independently reviewed:** reserved for a future external review protocol; the CLI does not issue
117
- this label.
165
+ - **Repository-detected:** the local collector found qualifying evidence in the selected path scope.
166
+ This proves an artifact match, not consistent practice or external enforcement.
167
+ - **Agent-collected:** an authorized agent supplied a target-bound, expiring, source-backed external
168
+ claim. It is not independently verified.
169
+ - **Human-attested:** an accountable owner supplied a dated evidence link or explanation.
170
+ - **Unknown:** evidence is unavailable, expired, unauthorized, mismatched, or inconclusive.
118
171
 
119
- Repository heuristics are intentionally explainable and conservative. They prove that an artifact
120
- or term exists—not that a team consistently follows it. Manual claims make organizational controls
121
- portable across GitHub, GitLab, Azure DevOps, internal systems, and different agent harnesses while
122
- preserving provenance.
172
+ Controls also identify whether their evidence belongs in the repository, hosting platform,
173
+ organization, or outcome systems. This prevents expected external unknowns from masquerading as
174
+ missing files. Future independently conformant adapters require a separate protocol; v0.2 does not
175
+ issue certification or independent-verification claims.
123
176
 
124
177
  ## Reports and CI
125
178
 
126
179
  The CLI emits Markdown for people and stable JSON for automation:
127
180
 
128
181
  ```bash
129
- agentic-scorecard assess . --format json --output .agentic/report.json
182
+ agentic-scorecard assess . --scope tracked --format json --output .agentic/report.json
183
+ agentic-scorecard assess . --profile pr-creation --agent-evidence .agentic/agent-evidence.yaml
130
184
  agentic-scorecard assess . --profile pr-creation --enforce
185
+ agentic-scorecard init-evidence .
131
186
  agentic-scorecard explain ADRB-SEC-003
132
187
  agentic-scorecard validate
133
188
  ```
@@ -142,11 +197,12 @@ funded the remediation plan. See
142
197
  The default collector:
143
198
 
144
199
  - makes no network calls and invokes no model;
145
- - reads only the target checkout;
146
- - ignores `.git`, dependency, build, and coverage directories;
200
+ - reads only Git-tracked paths and records commit and dirty-worktree metadata;
201
+ - ignores `.git`, dependencies, build output, coverage, generated reports, attestations, and imported
202
+ evidence bundles;
147
203
  - caps content-scanned files at 512 KB;
148
204
  - reports file paths and match counts, never matching source snippets;
149
- - writes nothing unless `--output` or `init` is explicitly requested.
205
+ - writes nothing unless `--output`, `init`, or `init-evidence` is explicitly requested.
150
206
 
151
207
  Do not place credentials, private prompts, source excerpts, or personal data in attestations. Link to
152
208
  access-controlled evidence instead. Report suspected vulnerabilities through [SECURITY.md](SECURITY.md).
@@ -154,7 +210,8 @@ access-controlled evidence instead. Report suspected vulnerabilities through [SE
154
210
  ## Repository structure
155
211
 
156
212
  ```text
157
- benchmark/v0.1/ immutable benchmark definition and controls
213
+ benchmark/v0.1/ immutable historical v0.1 definition
214
+ benchmark/v0.2/ current normative benchmark, schemas, and controls
158
215
  benchmark/mappings/ informative mappings to external frameworks
159
216
  src/ reference CLI and local evidence collectors
160
217
  templates/ adoption, preflight, attestation, and remediation templates
@@ -171,21 +228,23 @@ OIDC-provenance release process.
171
228
 
172
229
  1. Run a local baseline for the least-authoritative profile you actually need.
173
230
  2. Review every result with security, platform, and representative delivery teams.
174
- 3. Supply owned attestations for controls that cannot be mechanically verified.
175
- 4. Publish the report internally with explicit limitations and benchmark version.
176
- 5. Fund the smallest improvements that close target-profile blockers.
177
- 6. Reassess on material harness changes and at least quarterly.
178
- 7. Add a non-blocking CI report; enforce only the agreed target profile after a pilot.
231
+ 3. Use an authorized agent to collect source-backed external evidence where appropriate.
232
+ 4. Supply owned human attestations only where deterministic collection is unavailable.
233
+ 5. Publish the report internally with explicit limitations, scope, commit, and benchmark version.
234
+ 6. Fund the smallest improvements that close target-profile blockers.
235
+ 7. Reassess on material harness changes and at least quarterly.
236
+ 8. Add a non-blocking CI report; enforce only the agreed target profile after a pilot.
179
237
 
180
238
  Never optimize to the number alone. Use the control evidence and outcome metrics to improve the
181
239
  system, and keep exceptions narrow, owned, expiring, and visible.
182
240
 
183
241
  ## Status and roadmap
184
242
 
185
- v0.1 includes the normative control catalog, local collectors, attestations, JSON/Markdown reports,
186
- fixtures, and a CI example. Candidate next steps include SARIF/HTML reports, signed reports, a stable
187
- adapter SDK, organization-level aggregation, statistically designed benchmark tasks, and an
188
- independent-review protocol. These require public design review before becoming normative.
243
+ v0.2 adds integrity-safe tracked-path collection, evidence scopes, precise co-located content
244
+ matching, agent-collected external evidence, and transparent report grouping. Candidate next steps
245
+ include conformant signed adapters, SARIF/HTML reports, organization-level aggregation,
246
+ statistically designed benchmark tasks, and an independent-review protocol. These require public
247
+ design review before becoming normative.
189
248
 
190
249
  Apache-2.0 licensed. The benchmark is a community engineering tool, not legal, compliance, or
191
250
  security advice.
@@ -0,0 +1,25 @@
1
+ # v0.2 external evidence contract
2
+
3
+ The core scanner is local, read-only, network-free, and vendor-neutral. An agent or adapter may
4
+ collect evidence that the repository cannot establish, then provide it through the versioned agent
5
+ evidence bundle.
6
+
7
+ Each claim contains:
8
+
9
+ - benchmark version and control ID;
10
+ - repository target and optional commit binding;
11
+ - `met`, `not_met`, or `unknown` status;
12
+ - platform, organization, or outcome scope;
13
+ - collector name and version;
14
+ - collection timestamp and mandatory expiry;
15
+ - privacy-safe references, never credentials or raw sensitive content;
16
+ - a concise derivation summary; and
17
+ - an optional error that is distinguishable from a negative result.
18
+
19
+ Collectors must be read-only, least-privileged, bounded to the approved target, time-limited, and
20
+ fail closed. Network content is untrusted input: it cannot alter benchmark policy, prompts, tool
21
+ authority, or the set of controls being assessed.
22
+
23
+ Agent-collected claims remain visibly distinct from repository evidence and human attestations. A
24
+ future conformant-adapter class may add signed identity and independent conformance tests; v0.2 does
25
+ not issue an independently verified label.
@@ -0,0 +1,52 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://agentic-readiness.dev/schema/v0.2/agent-evidence.json",
4
+ "title": "ADRB v0.2 agent-collected evidence bundle",
5
+ "type": "object",
6
+ "additionalProperties": false,
7
+ "required": ["schema_version", "benchmark_version", "target", "collector", "claims"],
8
+ "properties": {
9
+ "schema_version": { "const": "0.2.0" },
10
+ "benchmark_version": { "const": "0.2.0" },
11
+ "target": {
12
+ "type": "object",
13
+ "additionalProperties": false,
14
+ "required": ["repository", "git_head"],
15
+ "properties": {
16
+ "repository": { "type": "string", "minLength": 1 },
17
+ "git_head": { "type": ["string", "null"], "minLength": 1 }
18
+ }
19
+ },
20
+ "collector": {
21
+ "type": "object",
22
+ "additionalProperties": false,
23
+ "required": ["name", "version"],
24
+ "properties": {
25
+ "name": { "type": "string", "minLength": 1 },
26
+ "version": { "type": "string", "minLength": 1 }
27
+ }
28
+ },
29
+ "claims": {
30
+ "type": "object",
31
+ "propertyNames": { "pattern": "^ADRB-[A-Z]{3}-[0-9]{3}$" },
32
+ "additionalProperties": {
33
+ "type": "object",
34
+ "additionalProperties": false,
35
+ "required": ["status", "scope", "summary", "references", "collected_at", "expires_at"],
36
+ "properties": {
37
+ "status": { "enum": ["met", "not_met", "unknown"] },
38
+ "scope": { "enum": ["platform", "organization", "outcome"] },
39
+ "summary": { "type": "string", "minLength": 1 },
40
+ "references": {
41
+ "type": "array",
42
+ "minItems": 1,
43
+ "items": { "type": "string", "minLength": 1 }
44
+ },
45
+ "collected_at": { "type": "string", "format": "date-time" },
46
+ "expires_at": { "type": "string", "format": "date-time" },
47
+ "error": { "type": ["string", "null"], "minLength": 1 }
48
+ }
49
+ }
50
+ }
51
+ }
52
+ }
@@ -0,0 +1,26 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://agentic-readiness.dev/schema/v0.2/attestations.json",
4
+ "title": "ADRB v0.2 human attestation file",
5
+ "type": "object",
6
+ "additionalProperties": false,
7
+ "required": ["benchmark_version", "attestations"],
8
+ "properties": {
9
+ "benchmark_version": { "const": "0.2.0" },
10
+ "attestations": {
11
+ "type": "object",
12
+ "additionalProperties": {
13
+ "type": "object",
14
+ "additionalProperties": false,
15
+ "required": ["status", "evidence", "owner", "reviewed_at", "expires_at"],
16
+ "properties": {
17
+ "status": { "enum": ["met", "not_met", "not_applicable", "unknown"] },
18
+ "evidence": { "type": "string", "minLength": 1 },
19
+ "owner": { "type": "string", "minLength": 1 },
20
+ "reviewed_at": { "type": "string", "format": "date" },
21
+ "expires_at": { "type": "string", "format": "date" }
22
+ }
23
+ }
24
+ }
25
+ }
26
+ }
@@ -0,0 +1,127 @@
1
+ id: agentic-development-readiness
2
+ version: 0.2.0
3
+ title: Agentic Development Readiness Benchmark
4
+ description: >-
5
+ A vendor-neutral, evidence-backed assessment of the engineering harness that surrounds AI agents.
6
+ It measures whether a repository and its operating controls can safely support increasing levels
7
+ of delegated software work.
8
+
9
+ maturity_levels:
10
+ 0: Absent
11
+ 1: Ad hoc
12
+ 2: Documented
13
+ 3: Enforced
14
+ 4: Measured and improving
15
+
16
+ dimensions:
17
+ - id: context
18
+ title: Context and knowledge
19
+ description: Agents can discover authoritative, scoped, current instructions without guesswork.
20
+ - id: environment
21
+ title: Reproducible environment
22
+ description: Agents can establish and verify a faithful development environment.
23
+ - id: specification
24
+ title: Specification and planning
25
+ description: Work starts from testable outcomes, constraints, and explicit acceptance criteria.
26
+ - id: tooling
27
+ title: Tooling and interfaces
28
+ description: Agent tools are discoverable, least-privileged, observable, and fail safely.
29
+ - id: security
30
+ title: Security and data governance
31
+ description: Credentials, data, autonomy, and untrusted content have explicit boundaries.
32
+ - id: testing
33
+ title: Verification and testing
34
+ description: Changes are checked at the right layers with trustworthy, reproducible signals.
35
+ - id: governance
36
+ title: Review and change governance
37
+ description: Human accountability, change scope, and merge authority remain explicit.
38
+ - id: learning
39
+ title: Learning and knowledge maintenance
40
+ description: Corrections compound into maintained instructions instead of being rediscovered.
41
+ - id: observability
42
+ title: Outcome observability
43
+ description: Runs and outcomes can be evaluated by task class, risk, cost, and quality.
44
+ - id: resilience
45
+ title: Failure containment and recovery
46
+ description: Agent failures are bounded, reversible, diagnosed, and rehearsed.
47
+
48
+ readiness_profiles:
49
+ - id: read-only-analysis
50
+ title: Read-only analysis
51
+ description: Agent may inspect approved repository content and return advice, without mutations.
52
+ floors:
53
+ context: 1
54
+ environment: 0
55
+ specification: 0
56
+ tooling: 0
57
+ security: 1
58
+ testing: 0
59
+ governance: 0
60
+ learning: 0
61
+ observability: 0
62
+ resilience: 0
63
+ - id: planning
64
+ title: Planning and specification
65
+ description: Agent may draft plans, specifications, and proposed changes for human review.
66
+ floors:
67
+ context: 2
68
+ environment: 0
69
+ specification: 1
70
+ tooling: 1
71
+ security: 1
72
+ testing: 0
73
+ governance: 1
74
+ learning: 0
75
+ observability: 0
76
+ resilience: 0
77
+ - id: local-implementation
78
+ title: Local implementation
79
+ description: Agent may edit an isolated checkout and run approved local verification.
80
+ floors:
81
+ context: 2
82
+ environment: 2
83
+ specification: 2
84
+ tooling: 2
85
+ security: 2
86
+ testing: 2
87
+ governance: 1
88
+ learning: 1
89
+ observability: 1
90
+ resilience: 1
91
+ - id: pr-creation
92
+ title: Pull request creation
93
+ description: Agent may create a branch and propose a pull request; a human retains merge authority.
94
+ floors:
95
+ context: 2
96
+ environment: 2
97
+ specification: 2
98
+ tooling: 2
99
+ security: 2
100
+ testing: 2
101
+ governance: 2
102
+ learning: 1
103
+ observability: 1
104
+ resilience: 2
105
+ - id: limited-autonomous-maintenance
106
+ title: Limited autonomous maintenance
107
+ description: >-
108
+ Agent may complete pre-approved, low-risk maintenance within strict scope, budget, and stop
109
+ conditions. This profile never grants production deployment authority.
110
+ floors:
111
+ context: 3
112
+ environment: 3
113
+ specification: 3
114
+ tooling: 3
115
+ security: 3
116
+ testing: 3
117
+ governance: 3
118
+ learning: 3
119
+ observability: 3
120
+ resilience: 3
121
+
122
+ scoring:
123
+ dimension_method: consecutive-levels
124
+ overall_method: sum
125
+ maximum_score: 40
126
+ readiness_method: non-compensating-floors
127
+ attestation_counts_as_verified: false