agentic-scorecard 0.1.0

package/benchmark/v0.1/controls/context.yaml

@@ -0,0 +1,43 @@
+dimension: context
+controls:
+  - id: ADRB-CTX-001
+    level: 1
+    title: Repository guidance is discoverable
+    outcome: A new agent can find repository-specific working instructions from the repository root.
+    risk: The agent guesses conventions or applies generic advice that conflicts with local architecture.
+    evidence:
+      - type: path_any
+        patterns: [AGENTS.md, CLAUDE.md, .github/copilot-instructions.md, .cursor/rules/**]
+    remediation: Add a concise root instruction file that points to authoritative deeper guidance.
+  - id: ADRB-CTX-002
+    level: 2
+    title: Instructions express authoritative architecture and scope
+    outcome: Guidance identifies architectural invariants, scoped instructions, and precedence rules.
+    risk: Multiple context files silently disagree or fail to protect architectural boundaries.
+    evidence:
+      - type: path_any
+        patterns: [.ai/constitution.md, docs/architecture/**, architecture/**, ADR/**, docs/adr/**]
+      - type: content_any
+        files: [AGENTS.md, CLAUDE.md, .github/copilot-instructions.md]
+        needles: [scope, architecture, precedence, invariant]
+    remediation: Document immutable constraints and how root, module, and task instructions compose.
+  - id: ADRB-CTX-003
+    level: 3
+    title: Context integrity is mechanically checked
+    outcome: CI detects broken links, oversized bootstrap context, invalid instruction structure, or drift.
+    risk: Guidance degrades unnoticed and agents operate with stale or contradictory context.
+    evidence:
+      - type: content_any
+        files:
+          [.github/workflows/**, .gitlab-ci.yml, Jenkinsfile, scripts/**, package.json, Makefile]
+        needles: [agent-doc, instruction, broken link, markdownlint, context budget, constitution]
+    remediation: Add a deterministic CI check for instruction validity, links, size budgets, and mirrors.
+  - id: ADRB-CTX-004
+    level: 4
+    title: Context quality is measured and improved
+    outcome: The organization measures agent orientation failures and improves context using those outcomes.
+    risk: More documentation accumulates without proving that agents reach the right decisions faster.
+    evidence:
+      - type: manual
+        prompt: Provide recent orientation or context-failure metrics, the review cadence, and one resulting improvement.
+    remediation: Sample failed runs, classify context causes, and track recurrence and time-to-orientation.

package/benchmark/v0.1/controls/environment.yaml

@@ -0,0 +1,74 @@
+dimension: environment
+controls:
+  - id: ADRB-ENV-001
+    level: 1
+    title: Local setup is documented
+    outcome: A contributor can locate setup, dependency, and verification instructions.
+    risk: Agents improvise an environment and produce results that cannot be reproduced.
+    evidence:
+      - type: content_any
+        files: [README.md, CONTRIBUTING.md, docs/**]
+        needles: [install, setup, prerequisites, getting started]
+    remediation: Add minimal setup and verification commands to the repository documentation.
+  - id: ADRB-ENV-002
+    level: 2
+    title: Dependencies and runtime are reproducible
+    outcome: Runtime versions and dependencies are pinned, with non-secret configuration discoverable.
+    risk: Local and CI behavior diverges because agents resolve different tools or dependency graphs.
+    evidence:
+      - type: path_any
+        patterns:
+          [
+            package-lock.json,
+            pnpm-lock.yaml,
+            yarn.lock,
+            poetry.lock,
+            uv.lock,
+            Pipfile.lock,
+            go.sum,
+            Cargo.lock,
+          ]
+      - type: path_any
+        patterns:
+          [
+            .tool-versions,
+            .node-version,
+            .nvmrc,
+            .python-version,
+            mise.toml,
+            devcontainer.json,
+            .devcontainer/**,
+            Dockerfile,
+          ]
+    remediation: Commit a lockfile and pin supported runtime or container versions.
+  - id: ADRB-ENV-003
+    level: 3
+    title: CI proves environment fidelity
+    outcome: CI creates the supported environment from clean state and runs the canonical verification entry point.
+    risk: A documented setup can rot while cached or developer-specific environments continue to pass.
+    evidence:
+      - type: content_any
+        files: [.github/workflows/**, .gitlab-ci.yml, Jenkinsfile, azure-pipelines.yml]
+        needles:
+          [
+            npm ci,
+            pnpm install --frozen-lockfile,
+            yarn --immutable,
+            uv sync,
+            poetry install,
+            go test,
+            cargo test,
+          ]
+      - type: content_any
+        files: [.github/workflows/**, .gitlab-ci.yml, Jenkinsfile, azure-pipelines.yml]
+        needles: [test, check, verify, build]
+    remediation: Rebuild dependencies from the lockfile in CI and invoke one canonical check command.
+  - id: ADRB-ENV-004
+    level: 4
+    title: Environment reliability is measured
+    outcome: Clean-checkout setup success and time are measured across supported platforms.
+    risk: Environment friction consumes agent time and causes hidden workarounds without being visible.
+    evidence:
+      - type: manual
+        prompt: Provide clean-checkout setup success rate, setup duration, supported platforms, and a recent corrective action.
+    remediation: Run scheduled clean-room setup probes and trend failures and duration.

package/benchmark/v0.1/controls/governance.yaml

@@ -0,0 +1,53 @@
+dimension: governance
+controls:
+  - id: ADRB-GOV-001
+    level: 1
+    title: Change and review process is documented
+    outcome: Contributors can find how to propose, review, and approve a change.
+    risk: Agent changes bypass normal accountability or use inconsistent review expectations.
+    evidence:
+      - type: path_any
+        patterns:
+          [
+            CONTRIBUTING.md,
+            .github/pull_request_template.md,
+            .github/PULL_REQUEST_TEMPLATE/**,
+            docs/governance/**,
+          ]
+    remediation: Document the branch, pull request, review, and approval workflow.
+  - id: ADRB-GOV-002
+    level: 2
+    title: Human accountability and merge authority are explicit
+    outcome: Agent-authored changes identify provenance, responsible human, risk, verification, and human merge authority.
+    risk: Reviewers cannot tell who owns the decision or which work was delegated.
+    evidence:
+      - type: content_any
+        files:
+          [
+            CONTRIBUTING.md,
+            .github/pull_request_template.md,
+            .github/PULL_REQUEST_TEMPLATE/**,
+            docs/governance/**,
+          ]
+        needles: [agent, ai-assisted, responsible, human review, approval, verification]
+    remediation: Add agent provenance, responsible owner, risk, evidence, and approval fields to the PR process.
+  - id: ADRB-GOV-003
+    level: 3
+    title: Review scope and quality gates are enforced
+    outcome: Protected branches, required reviewers, checks, ownership, and exception handling are enforced.
+    risk: Policy depends on voluntary compliance and can be bypassed by the same actor making the change.
+    evidence:
+      - type: path_any
+        patterns: [.github/CODEOWNERS, CODEOWNERS]
+      - type: manual
+        prompt: Show branch protection, required reviews and checks, exception ownership, and auditability.
+    remediation: Configure protected branches, independent review, required checks, ownership, and expiring exceptions.
+  - id: ADRB-GOV-004
+    level: 4
+    title: Review precision and convergence are measured
+    outcome: Review escape rate, false-positive rate, convergence rounds, and human load guide policy changes.
+    risk: More agent reviewers create duplicate noise and approval theater instead of higher assurance.
+    evidence:
+      - type: manual
+        prompt: Provide review precision, escape, convergence, and reviewer-load metrics plus a resulting improvement.
+    remediation: Measure review outcomes, deduplicate findings, bound rounds, and tune reviewers by defect class.

package/benchmark/v0.1/controls/learning.yaml

@@ -0,0 +1,40 @@
+dimension: learning
+controls:
+  - id: ADRB-LRN-001
+    level: 1
+    title: Agent corrections are recorded
+    outcome: Review corrections and post-merge lessons have an explicit durable home.
+    risk: Every agent session repeats the same repository-specific mistakes.
+    evidence:
+      - type: path_any
+        patterns: [.ai/lessons.md, docs/lessons/**, docs/decisions/**, ADR/**, docs/adr/**]
+    remediation: Create a concise correction log with context, rule, owner, and date.
+  - id: ADRB-LRN-002
+    level: 2
+    title: Knowledge promotion and ownership are documented
+    outcome: The team defines when a correction becomes an instruction, skill, decision, or reference page.
+    risk: The lesson store becomes a stale append-only archive rather than usable agent knowledge.
+    evidence:
+      - type: content_any
+        files: [AGENTS.md, .ai/lessons.md, docs/**, .agents/skills/**, .claude/skills/**]
+        needles: [curation, promote, lesson, correction, stale, owner]
+    remediation: Define a curation cadence, promotion rules, ownership, deduplication, and stale-content handling.
+  - id: ADRB-LRN-003
+    level: 3
+    title: Knowledge curation is an enforced workflow
+    outcome: A scheduled or release-gated process detects stale, duplicate, contradictory, or unpromoted knowledge.
+    risk: Knowledge quality relies on memory and quietly deteriorates as the repository evolves.
+    evidence:
+      - type: content_any
+        files: [.github/workflows/**, scripts/**, package.json, Makefile, AGENTS.md]
+        needles: [curator, curation, knowledge lint, lesson, stale, contradiction]
+    remediation: Automate mechanical checks and require periodic human judgement over knowledge quality.
+  - id: ADRB-LRN-004
+    level: 4
+    title: Knowledge effectiveness is measured
+    outcome: Recurring mistakes, stale guidance, retrieval success, and correction latency drive curation priorities.
+    risk: Knowledge grows without evidence that it changes agent behavior.
+    evidence:
+      - type: manual
+        prompt: Provide recurrence and knowledge-quality metrics plus a recent improvement linked to them.
+    remediation: Classify correction recurrence and measure whether promoted knowledge prevents repeat failures.

package/benchmark/v0.1/controls/observability.yaml

@@ -0,0 +1,54 @@
+dimension: observability
+controls:
+  - id: ADRB-OBS-001
+    level: 1
+    title: Agent activity can be reconstructed
+    outcome: Material agent runs retain enough metadata to identify task, actor or harness, result, and time.
+    risk: Teams cannot explain what happened after a poor change or security concern.
+    evidence:
+      - type: path_any
+        patterns:
+          [
+            '**/docs/observability/**',
+            '**/docs/agentic/**',
+            '**/evals/**',
+            '**/evaluations/**',
+            .agentic/**,
+            '**/observability/**',
+          ]
+    remediation: Define a privacy-safe run record with task, harness, permissions, result, timing, and evidence links.
+  - id: ADRB-OBS-002
+    level: 2
+    title: Run envelope and outcome taxonomy are documented
+    outcome: Run records cover model or harness version, tools, authority, budget, stop reason, verification, and outcome.
+    risk: Comparisons confound model, tooling, permissions, task risk, and checkout state.
+    evidence:
+      - type: content_any
+        files:
+          [
+            '**/docs/observability/**',
+            '**/docs/agentic/**',
+            '**/evals/**',
+            .agentic/**,
+            '**/observability/**',
+          ]
+        needles: [task class, tool, permission, budget, stop reason, outcome, harness version]
+    remediation: Adopt a common run envelope and outcome taxonomy with privacy and retention rules.
+  - id: ADRB-OBS-003
+    level: 3
+    title: Outcome evidence is captured consistently
+    outcome: The harness or CI emits run metadata and verification outcomes for material agent changes.
+    risk: Only memorable successes and failures shape policy, creating selection bias.
+    evidence:
+      - type: manual
+        prompt: Show automated run-envelope capture, retention controls, access controls, and completeness monitoring.
+    remediation: Emit structured run and outcome records automatically, minimizing retained source and prompt content.
+  - id: ADRB-OBS-004
+    level: 4
+    title: Agent outcomes are compared by task and risk class
+    outcome: Quality, escape rate, cycle time, cost, intervention, and rollback are used to tune autonomy.
+    risk: Token use or completion rate becomes a misleading proxy for engineering value and safety.
+    evidence:
+      - type: manual
+        prompt: Provide outcome trends segmented by task and risk class and one autonomy or harness decision they changed.
+    remediation: Evaluate outcome quality and total effort by task class; use results to narrow or expand autonomy.

package/benchmark/v0.1/controls/resilience.yaml

@@ -0,0 +1,40 @@
+dimension: resilience
+controls:
+  - id: ADRB-RES-001
+    level: 1
+    title: Changes have a recovery path
+    outcome: Contributors can locate rollback, revert, incident, or recovery guidance.
+    risk: A harmful agent change persists because recovery is improvised under pressure.
+    evidence:
+      - type: content_any
+        files: [README.md, CONTRIBUTING.md, docs/**, runbooks/**, .github/**]
+        needles: [rollback, revert, recovery, incident]
+    remediation: Document how to stop work, revert code, recover data, and escalate an incident.
+  - id: ADRB-RES-002
+    level: 2
+    title: Failure containment and stop conditions are documented
+    outcome: Tasks define isolation, mutation limits, time or cost budgets, stop triggers, and recovery owner.
+    risk: A looping or confused agent continues spending or mutating beyond the intended scope.
+    evidence:
+      - type: content_any
+        files: [AGENTS.md, SECURITY.md, docs/**, runbooks/**, .agentic/**]
+        needles: [stop condition, rollback, isolation, budget, kill switch, recovery owner]
+    remediation: Add bounded scope, budget, retry, stop, escalation, and recovery requirements to the task preflight.
+  - id: ADRB-RES-003
+    level: 3
+    title: Containment and recovery controls are tested
+    outcome: The organization tests cancellation, credential revocation, rollback, timeout, and partial-failure recovery.
+    risk: Recovery mechanisms exist on paper but fail when the harness behaves unexpectedly.
+    evidence:
+      - type: manual
+        prompt: Show recent automated tests or exercises for stop, timeout, revocation, rollback, and partial failure.
+    remediation: Add failure-injection tests and periodic exercises for containment and reversible recovery.
+  - id: ADRB-RES-004
+    level: 4
+    title: Recovery performance drives improvement
+    outcome: Time-to-detect, time-to-stop, rollback success, impact, and recurrence are measured.
+    risk: The same failure patterns repeat because recovery performance and residual impact are unknown.
+    evidence:
+      - type: manual
+        prompt: Provide recovery outcome trends and a recent containment or rollback improvement based on them.
+    remediation: Trend containment and recovery outcomes and feed them into autonomy and tool design.

package/benchmark/v0.1/controls/security.yaml

@@ -0,0 +1,64 @@
+dimension: security
+controls:
+  - id: ADRB-SEC-001
+    level: 1
+    title: Security reporting and secret handling are documented
+    outcome: Contributors can find security contacts and rules for credentials and sensitive data.
+    risk: Agents expose secrets or sensitive data and responders lack a defined reporting path.
+    evidence:
+      - type: path_any
+        patterns: [SECURITY.md, .github/SECURITY.md, '**/docs/security/**']
+      - type: content_any
+        files: [SECURITY.md, .github/SECURITY.md, '**/docs/security/**', AGENTS.md]
+        needles: [secret, credential, sensitive, vulnerability]
+    remediation: Publish a security policy covering reporting, secrets, sensitive data, and agent-specific risks.
+  - id: ADRB-SEC-002
+    level: 2
+    title: Agent data and autonomy boundaries are explicit
+    outcome: A preflight defines permitted data, tools, mutations, credentials, budgets, approvals, and stop conditions.
+    risk: Operators grant an agent more data or authority than the task requires.
+    evidence:
+      - type: content_any
+        files: [SECURITY.md, AGENTS.md, '**/docs/security/**', '**/docs/agentic/**', .agentic/**]
+        needles:
+          [
+            stop condition,
+            data classification,
+            allowed tools,
+            mutation,
+            budget,
+            human approval,
+            autonomy,
+          ]
+    remediation: Add a task preflight that binds data, tools, write scope, budget, approvals, and stop conditions.
+  - id: ADRB-SEC-003
+    level: 3
+    title: Agent security controls are enforced
+    outcome: Sandboxing, secret scanning, untrusted-content handling, and high-impact approval are enforced and tested.
+    risk: Prompt injection or credential misuse can turn repository access into broader compromise.
+    evidence:
+      - type: content_any
+        files:
+          [.github/workflows/**, .gitlab-ci.yml, scripts/**, SECURITY.md, '**/docs/security/**']
+        needles:
+          [
+            secret scan,
+            gitleaks,
+            trufflehog,
+            sandbox,
+            prompt injection,
+            untrusted content,
+            approval gate,
+          ]
+      - type: manual
+        prompt: Show that high-impact actions are technically gated and that untrusted instructions cannot expand tool authority.
+    remediation: Combine isolation, scoped credentials, secret scanning, content-boundary rules, and approval gates with tests.
+  - id: ADRB-SEC-004
+    level: 4
+    title: Agent security outcomes are measured and exercised
+    outcome: Security denials, near misses, incidents, exceptions, and adversarial exercises drive improvements.
+    risk: Controls appear complete but their effectiveness against real agent failure modes is unknown.
+    evidence:
+      - type: manual
+        prompt: Provide recent agent-security metrics or exercise results and the control changes they produced.
+    remediation: Run adversarial exercises and trend denials, exceptions, near misses, and incident recurrence.

package/benchmark/v0.1/controls/specification.yaml

@@ -0,0 +1,42 @@
+dimension: specification
+controls:
+  - id: ADRB-SPC-001
+    level: 1
+    title: Work has an explicit objective
+    outcome: Engineering work starts from a recorded issue, request, or specification.
+    risk: Agents optimize an inferred problem and completion cannot be judged consistently.
+    evidence:
+      - type: path_any
+        patterns:
+          [.github/ISSUE_TEMPLATE/**, docs/requirements/**, specs/**, .ai/specs/**, .kiro/specs/**]
+    remediation: Adopt a lightweight work-request template with objective, owner, and expected outcome.
+  - id: ADRB-SPC-002
+    level: 2
+    title: Acceptance criteria and constraints are documented
+    outcome: Change requests state testable acceptance criteria, non-goals, and material constraints.
+    risk: Implementation appears plausible but cannot be traced to a shared definition of done.
+    evidence:
+      - type: content_any
+        files:
+          [.github/ISSUE_TEMPLATE/**, specs/**, .ai/specs/**, .kiro/specs/**, docs/requirements/**]
+        needles: [acceptance criteria, non-goals, constraints, requirements]
+    remediation: Extend the work template with testable outcomes, exclusions, constraints, and verification.
+  - id: ADRB-SPC-003
+    level: 3
+    title: Specification traceability is enforced
+    outcome: CI or review policy requires changes and tests to trace to approved acceptance criteria.
+    risk: Specifications become optional prose that neither implementation nor verification follows.
+    evidence:
+      - type: content_any
+        files: [.github/workflows/**, .github/pull_request_template.md, scripts/**, package.json]
+        needles: [acceptance criteria, traceability, spec check, requirements coverage, task id]
+    remediation: Gate applicable changes on a linked spec and acceptance-criteria verification record.
+  - id: ADRB-SPC-004
+    level: 4
+    title: Specification effectiveness is measured
+    outcome: The organization measures rework and escaped ambiguity attributable to specifications.
+    risk: Specification ceremony grows without improving delivery accuracy or reducing rework.
+    evidence:
+      - type: manual
+        prompt: Provide ambiguity or rework metrics by task class and an example of a template or process improvement.
+    remediation: Review rework causes and evolve specification templates using outcome data.

package/benchmark/v0.1/controls/testing.yaml

@@ -0,0 +1,42 @@
+dimension: testing
+controls:
+  - id: ADRB-TST-001
+    level: 1
+    title: Automated tests exist
+    outcome: The repository contains executable tests for at least one important behavior.
+    risk: Agent changes rely on plausibility or manual inspection alone.
+    evidence:
+      - type: path_any
+        patterns: ['**/*.test.*', '**/*.spec.*', tests/**, test/**]
+    remediation: Add a fast automated test for a critical behavior and document how to run it.
+  - id: ADRB-TST-002
+    level: 2
+    title: Verification layers and canonical commands are documented
+    outcome: Contributors know which unit, integration, end-to-end, static, and security checks apply.
+    risk: Agents run a convenient subset and overstate confidence in the result.
+    evidence:
+      - type: content_any
+        files: [README.md, CONTRIBUTING.md, AGENTS.md, docs/**, package.json, Makefile]
+        needles: [unit test, integration test, end-to-end, e2e, typecheck, lint, test]
+    remediation: Define the test taxonomy and one canonical command that matches CI.
+  - id: ADRB-TST-003
+    level: 3
+    title: Required verification is enforced before merge
+    outcome: CI runs required checks from clean state and blocks merge when they fail.
+    risk: Documented checks become optional, flaky, or bypassed under delivery pressure.
+    evidence:
+      - type: content_any
+        files: [.github/workflows/**, .gitlab-ci.yml, Jenkinsfile, azure-pipelines.yml]
+        needles: [test, typecheck, lint, coverage, e2e]
+      - type: manual
+        prompt: Confirm required checks are branch-protection or merge-policy gates and provide the policy evidence.
+    remediation: Make appropriate tests and static checks required status gates with controlled exceptions.
+  - id: ADRB-TST-004
+    level: 4
+    title: Verification effectiveness is measured
+    outcome: Escaped defects, flakiness, mutation strength or equivalent outcome measures guide test investment.
+    risk: Test count and coverage rise while meaningful defects still escape.
+    evidence:
+      - type: manual
+        prompt: Provide verification outcome metrics by task class and a recent test-strategy improvement.
+    remediation: Trend escaped defects and signal quality, then target weak verification layers.

package/benchmark/v0.1/controls/tooling.yaml

@@ -0,0 +1,48 @@
+dimension: tooling
+controls:
+  - id: ADRB-TOL-001
+    level: 1
+    title: Agent tools are declared
+    outcome: The repository identifies supported agent interfaces or configuration.
+    risk: Agents discover tools opportunistically and may use unsupported or unsafe paths.
+    evidence:
+      - type: path_any
+        patterns:
+          [
+            .codex/**,
+            .claude/**,
+            .agents/**,
+            .cursor/**,
+            .github/copilot-instructions.md,
+            mcp.json,
+            .mcp.json,
+          ]
+    remediation: Declare supported agent surfaces and their intended use in repository configuration.
+  - id: ADRB-TOL-002
+    level: 2
+    title: Tools have documented scope and failure behavior
+    outcome: Tool guidance covers authority, expected inputs, validation, and what to do when a tool fails.
+    risk: Agents bypass failed tools, widen scope, or mistake a partial response for success.
+    evidence:
+      - type: content_any
+        files: [AGENTS.md, CLAUDE.md, .agents/**, .claude/**, docs/**]
+        needles: [least privilege, permission, read-only, failure, fallback, approval, authority]
+    remediation: Document tool authority, data boundary, success signal, failure mode, and approved fallback.
+  - id: ADRB-TOL-003
+    level: 3
+    title: Tool authority is technically bounded and audited
+    outcome: High-impact tools enforce least privilege, authorization, structured validation, and audit records.
+    risk: Prompt instructions become the only barrier between an agent and an irreversible action.
+    evidence:
+      - type: manual
+        prompt: Show technical least-privilege controls, authorization tests, audit records, and fail-closed behavior for high-impact tools.
+    remediation: Enforce scoped credentials, explicit authorization, schema validation, audit logging, and deny-by-default behavior.
+  - id: ADRB-TOL-004
+    level: 4
+    title: Tool reliability is measured and improved
+    outcome: Tool success, partial failure, denial, latency, and unsafe-fallback attempts are trended.
+    risk: Tooling quality and risky workarounds remain anecdotal until an incident occurs.
+    evidence:
+      - type: manual
+        prompt: Provide tool reliability and denial metrics plus a recent improvement linked to those metrics.
+    remediation: Instrument tool outcomes and review reliability and fallback patterns by task and risk class.

package/benchmark/v0.1/evidence-schema.json

@@ -0,0 +1,28 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://agentic-readiness.dev/schema/v0.1/attestations.json",
+  "title": "ADRB v0.1 attestation file",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["benchmark_version", "attestations"],
+  "properties": {
+    "benchmark_version": { "const": "0.1.0" },
+    "attestations": {
+      "type": "object",
+      "additionalProperties": {
+        "type": "object",
+        "additionalProperties": false,
+        "required": ["status", "evidence", "owner", "reviewed_at"],
+        "properties": {
+          "status": {
+            "enum": ["met", "not_met", "not_applicable", "unknown"]
+          },
+          "evidence": { "type": "string", "minLength": 1 },
+          "owner": { "type": "string", "minLength": 1 },
+          "reviewed_at": { "type": "string", "format": "date" },
+          "expires_at": { "type": ["string", "null"], "format": "date" }
+        }
+      }
+    }
+  }
+}

package/benchmark/v0.1/report-schema.json

@@ -0,0 +1,32 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://agentic-readiness.dev/schema/v0.1/report.json",
+  "title": "ADRB v0.1 assessment report",
+  "type": "object",
+  "required": [
+    "schema_version",
+    "benchmark",
+    "target",
+    "assessed_at",
+    "score",
+    "evidence_summary",
+    "dimensions",
+    "controls",
+    "profiles",
+    "readiness",
+    "limitations"
+  ],
+  "properties": {
+    "schema_version": { "const": "0.1.0" },
+    "benchmark": { "type": "object" },
+    "target": { "type": "object" },
+    "assessed_at": { "type": "string", "format": "date-time" },
+    "score": { "type": "object" },
+    "evidence_summary": { "type": "object" },
+    "dimensions": { "type": "array", "minItems": 10, "maxItems": 10 },
+    "controls": { "type": "array" },
+    "profiles": { "type": "array" },
+    "readiness": { "type": "object" },
+    "limitations": { "type": "array", "items": { "type": "string" } }
+  }
+}

package/benchmark/v0.1/scoring-policy.md

@@ -0,0 +1,59 @@
+# v0.1 scoring policy
+
+This document is normative for benchmark version 0.1.0.
+
+## Unit of assessment
+
+The unit is one repository plus the organizational controls explicitly attested for that repository.
+The score describes the harness and operating boundaries at the assessment time. It does not score a
+model, vendor, developer, team productivity, or code quality in general.
+
+## Dimension scores
+
+Each dimension has one or more controls at levels 1 through 4. A dimension earns level `n` only when:
+
+1. every applicable control at level `n` is met; and
+2. every lower level was earned.
+
+This consecutive rule prevents a dashboard or advanced tool from masking missing fundamentals.
+The overall score is the sum of ten dimension scores, with a maximum of 40. The percentage is
+display-only and must not determine readiness.
+
+## Readiness decisions
+
+Each autonomy profile defines a floor for every dimension. A profile passes only if every floor is
+met. Scores do not compensate across dimensions. For example, context level 4 cannot compensate for
+security level 1 when a profile requires security level 2.
+
+The highest passed profile is informative. Teams should target only the authority required for their
+use case; a higher profile is not automatically better.
+
+## Evidence classes
+
+Deterministic collectors may award `verified` confidence when every required repository check passes.
+Manual attestations may satisfy portable organizational controls or explain nonstandard evidence, but
+the report labels them `attested`. Attestation never upgrades to verified confidence.
+
+An attestation must include status, owner, review date, and a useful evidence reference. Expired
+attestations are ignored. `not_applicable` is rejected unless a control explicitly permits it.
+Core v0.1 controls do not permit not-applicable claims.
+
+Collectors are intentionally evidence-presence tests. Reviewers must not infer implementation
+quality, policy compliance, or branch settings merely from a file or word existing.
+
+## Changes and comparability
+
+Version 0.1.0 controls and floors are immutable after public release. Clarifications that do not alter
+outcomes may be published as errata. Any addition, deletion, evidence-semantic change, or floor change
+requires a new benchmark version through the RFC process.
+
+Reports are comparable only when benchmark version, target profile, repository scope, attestation
+policy, and assessment mode match. Longitudinal comparisons should also record material harness and
+organizational changes.
+
+## Claims
+
+Allowed: “We self-assessed repository X against ADRB v0.1.0 and passed the PR-creation profile on
+DATE; N controls were verified and M were attested.”
+
+Not allowed: “Certified,” “compliant,” “safe,” or “secure” based only on this self-assessment.