agent-reliability 1.0.0__tar.gz

agent_reliability-1.0.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,34 @@
+name: Bug report
+description: Report reproducible incorrect behavior
+title: "[Bug]: "
+labels: [bug]
+body:
+  - type: markdown
+    attributes:
+      value: Security vulnerabilities must not be reported here; follow SECURITY.md.
+  - type: textarea
+    id: behavior
+    attributes:
+      label: Observed behavior
+      description: What happened, and what did you expect instead?
+    validations:
+      required: true
+  - type: textarea
+    id: reproduce
+    attributes:
+      label: Minimal reproduction
+      description: Include the smallest safe code sample and traceback with secrets removed.
+    validations:
+      required: true
+  - type: input
+    id: version
+    attributes:
+      label: Agent Reliability version
+    validations:
+      required: true
+  - type: input
+    id: python
+    attributes:
+      label: Python version and operating system
+    validations:
+      required: true

agent_reliability-1.0.0/.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,24 @@
+name: Feature request
+description: Propose an OSS reliability use case or improvement
+title: "[Feature]: "
+labels: [enhancement]
+body:
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem
+      description: What developer or reliability problem cannot be solved today?
+    validations:
+      required: true
+  - type: textarea
+    id: proposal
+    attributes:
+      label: Proposed outcome
+      description: Describe the desired behavior, not only an API shape.
+    validations:
+      required: true
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives and semantic impact
+      description: What workarounds exist, and could this affect SLO, UNKNOWN, or provenance semantics?

agent_reliability-1.0.0/.github/pull_request_template.md

@@ -0,0 +1,36 @@
+## What changed?
+
+## Why?
+
+## Alternatives considered
+
+## Compatibility implications
+
+<!-- See docs/COMPATIBILITY.md. Does this change any public API,
+     telemetry contract, or documented semantics? -->
+
+## Security implications
+
+<!-- See docs/SECURITY_MODEL.md if this touches input handling,
+     serialization, metadata, or external I/O. -->
+
+## Performance implications
+
+<!-- See docs/ENGINEERING_PRINCIPLES.md #6 if this touches a hot path. -->
+
+## How was this verified?
+
+- [ ] `ruff check .`
+- [ ] `ruff format --check .`
+- [ ] `mypy src`
+- [ ] `pytest`
+- [ ] `python -m build`
+- [ ] `python scripts/verify_release_artifacts.py` (release-affecting changes)
+
+## Checklist
+
+- [ ] Tests added/updated in the correct category (see [docs/TESTING_STRATEGY.md](../docs/TESTING_STRATEGY.md))
+- [ ] Docs updated if semantics changed
+- [ ] Stable API/semantic changes include compatibility-test updates
+- [ ] Privacy and failure-isolation tests updated when a trust boundary changed
+- [ ] ADR added if this is a hard-to-reverse architectural decision (see [docs/adr/README.md](../docs/adr/README.md))

agent_reliability-1.0.0/.github/workflows/ci.yml

@@ -0,0 +1,84 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  quality:
+    name: Lint, format, and type-check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: "3.11"
+          cache: "pip"
+
+      - name: Install package with dev dependencies
+        run: python -m pip install -e ".[dev]"
+
+      - name: Lint (ruff check)
+        run: ruff check .
+
+      - name: Format check (ruff format)
+        run: ruff format --check .
+
+      - name: Type check (mypy)
+        run: mypy src
+
+  test:
+    name: Test stable contracts (py${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: "pip"
+
+      - name: Install package with test dependencies
+        run: python -m pip install -e ".[dev,otel-test]"
+
+      - name: Test with branch coverage
+        run: python -m pytest --cov --cov-report=term-missing
+
+  artifacts:
+    name: Release artifacts, extras, examples, and typing
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: "3.11"
+          cache: "pip"
+
+      - name: Build wheel and sdist
+        run: |
+          python -m pip install build
+          python -m build
+
+      - name: Verify distributed artifacts
+        run: python scripts/verify_release_artifacts.py

agent_reliability-1.0.0/.github/workflows/release.yml

@@ -0,0 +1,100 @@
+name: Release
+
+on:
+  push:
+    tags: ["v*.*.*"]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: release-${{ github.ref }}
+  cancel-in-progress: false
+
+jobs:
+  build:
+    name: Build and verify release artifacts
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: "3.11"
+          cache: "pip"
+
+      - name: Confirm tag matches package version
+        run: |
+          tag="${GITHUB_REF_NAME#v}"
+          version="$(python -c 'import re,pathlib; print(re.search(r"__version__ = \"([^\"]+)\"", pathlib.Path("src/agent_reliability/__init__.py").read_text()).group(1))')"
+          if [ "$tag" != "$version" ]; then
+            echo "Tag v$tag does not match agent_reliability.__version__ ($version)" >&2
+            exit 1
+          fi
+
+      - name: Install package and build tooling
+        run: python -m pip install -e ".[dev,otel-test]" build
+
+      - name: Full quality gates
+        run: |
+          ruff check .
+          ruff format --check .
+          mypy src
+          python -m pytest --cov --cov-report=term-missing
+
+      - name: Build sdist and wheel
+        run: python -m build
+
+      - name: Verify distributed artifacts
+        run: python scripts/verify_release_artifacts.py
+
+      - name: Upload build artifacts
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dist
+          path: dist/
+          if-no-files-found: error
+
+  publish:
+    name: Publish to PyPI
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/project/agent-reliability/
+    permissions:
+      id-token: write
+    steps:
+      - name: Download build artifacts
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI via Trusted Publishing
+        uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+
+  github-release:
+    name: Create GitHub release
+    needs: publish
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - name: Download build artifacts
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: dist
+          path: dist/
+
+      - name: Create GitHub release
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          gh release create "${GITHUB_REF_NAME}" dist/* \
+            --repo "${GITHUB_REPOSITORY}" \
+            --title "${GITHUB_REF_NAME}" \
+            --prerelease \
+            --generate-notes

agent_reliability-1.0.0/.gitignore

@@ -0,0 +1,24 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+
+# Test / coverage / type-check caches
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.hypothesis/
+.coverage
+.coverage.*
+htmlcov/
+
+# Editors / OS
+.vscode/
+.idea/
+.DS_Store
+Thumbs.db

agent_reliability-1.0.0/CHANGELOG.md

@@ -0,0 +1,185 @@
+# Changelog
+
+All notable changes to this project are documented here. Format loosely
+follows [Keep a Changelog](https://keepachangelog.com/); versioning
+follows [docs/COMPATIBILITY.md](docs/COMPATIBILITY.md).
+
+## [Unreleased]
+
+## [1.0.0]
+
+First General Availability release. The public contract frozen for `1.0.0rc1`
+is unchanged; this release differs from `1.0.0rc1` only in version, release
+metadata, and documentation status — no production code, public API, or
+documented semantic changed. See [GA_CONTRACT.md](docs/GA_CONTRACT.md) for
+the full normative 1.0 contract and [VERSIONING.md](docs/VERSIONING.md) for
+the compatibility and deprecation policy this version now follows.
+
+Stable, user-facing capabilities as of `1.0.0`:
+
+- Safe synchronous and asynchronous agent instrumentation: instrumentation
+  dependency failures are isolated from application code, and initialization
+  failure degrades to a no-telemetry handle instead of raising.
+- Optional OpenTelemetry trace interoperability, with the host owning the
+  provider, sampling, propagation, processing, and export.
+- Explicit `PASS`/`FAIL`/`UNKNOWN` evaluation outcomes, with evaluator
+  execution failure kept categorically distinct from all three.
+- A deterministic evaluator framework (`agent_reliability.evaluation`) with
+  immutable evaluator identity, decisions, results, and provenance.
+- Exact `fractions.Fraction` SLO, error-budget, and burn-rate mathematics,
+  including explicit `UnknownPolicy` handling and defined no-data/
+  zero-tolerance states.
+- Provenance-safe local reliability aggregation
+  (`agent_reliability.reliability`): incompatible measurement methodologies
+  return a typed conflict rather than a misleading combined number.
+- A typed public API (`py.typed` shipped, strict-mypy verified against the
+  installed package) with zero base runtime dependencies and fully offline,
+  local operation — no network service, database, or hosted platform is
+  required.
+- Privacy-first defaults throughout: no automatic capture of prompts,
+  responses, tool arguments/results, exception messages, tracebacks, or
+  arbitrary payloads.
+
+This is an open-source SDK release. It does not include any hosted,
+multi-tenant, or commercial control-plane functionality.
+
+## [1.0.0rc1]
+
+This first public release candidate establishes the frozen 1.0 contracts for
+privacy-first agent instrumentation, synchronous and asynchronous run
+lifecycles, instrumentation failure isolation, optional OpenTelemetry
+interoperability, deterministic evaluation with provenance, exact SLO/error
+budget/burn-rate mathematics, provenance-safe local reliability aggregation,
+and a typed zero-base-dependency API.
+
+### Changed
+
+- M7 freezes the 1.0 public and semantic contracts, defines SemVer and
+  deprecation policy, makes runtime `__version__` the package-metadata source,
+  and excludes retained exceptions from `SdkDiagnostic` representations.
+
+- M6 reframes the developer path around an executable quickstart, core
+  concepts, framework-neutral integration guidance, and a product-first
+  README. Public API review found no rename or semantic change necessary.
+
+- M5 makes evaluator provenance operational: local aggregation fails closed
+  with a typed, number-free conflict when indicator or methodology cohorts do
+  not match. M1 remains the sole source of reliability mathematics.
+
+- M4 extends `EvaluationRecorded` additively with optional immutable evaluator
+  provenance and a bounded reason code. Existing `RunHandle.record(...)`
+  remains a manual assertion and emits `provenance=None`.
+- Evaluator execution failure is now explicitly distinct from
+  `EvaluationOutcome.UNKNOWN`; safe execution never converts evaluator failure
+  into agent `FAIL` or `UNKNOWN`.
+
+- M3 OpenTelemetry interoperability adds an optional API-only trace-context
+  bridge while preserving the M2.1 default path and failure isolation. The
+  host owns provider, sampling, propagation, processing, and export.
+- Agent run spans use a privacy allowlist, the constant `invoke_agent` name,
+  and versioned Experimental `agent_reliability.*` attributes. Evaluations
+  remain vendor-neutral events while OTel's log-based Events APIs mature.
+
+- M2.1 runtime hardening: clock, run-ID, and internal run-start failures
+  now enter a no-telemetry degraded mode so sync/async application bodies
+  still execute. Degraded handles expose no fabricated IDs, valid
+  `record()` calls are no-ops, and no lifecycle/evaluation events are emitted.
+  This supersedes ADR-0004's original "clock/id-generator failure at
+  `__enter__`/`__aenter__` may raise" sub-rule (that behavior turned out
+  to prevent the application body from running at all, contradicting the
+  ADR's own safety requirement) — see the new ADR-0005, which is now the
+  authority on this boundary; ADR-0004's Status was changed to
+  "Superseded by ADR-0005" and its original text left otherwise unedited,
+  per the project's own ADR process.
+- The default diagnostic logger now emits only sanitized structural
+  metadata and the exception class name; it never renders exception
+  messages, arguments, representations, or tracebacks (also ADR-0005).
+- `AgentReliability(...)` now validates at construction that any
+  supplied `sink`/`clock`/`run_id_generator`/`diagnostic_handler`
+  structurally implements its port, raising `TypeError` immediately for
+  a wrong object type, rather than failing later and less clearly.
+- Clarified `EventSink` ordering, concurrency, exception, and lifecycle
+  contracts; confirmed `CompositeEventSink` attempts all children and
+  raises the first child `Exception` after fan-out.
+- Explicitly classified `InMemoryEventSink` as an unbounded-retention
+  test/local-inspection utility unsuitable for production.
+
+### Added
+
+- M7 adds public API/enum/signature compatibility tests, GA semantic and
+  privacy golden tests, wheel-and-sdist release verification, supported-Python
+  CI gates, and maintainer release/readiness documentation.
+
+- M6 adds four contract-tested examples, clean installed-wheel/base/OTel-extra
+  verification in CI, lightweight public issue templates, and the PEP 561
+  `py.typed` marker. Base runtime dependencies remain empty.
+
+- M5 local reliability engine:
+  - immutable observation, cohort, conflict, and report values in
+    `agent_reliability.reliability`;
+  - pure `evaluate_reliability(...)` with explicit UNKNOWN policy and optional
+    explicit burn-rate lookback;
+  - exact M1-composed ratio, SLO, error-budget, and burn-rate results;
+  - ADR-0008, `docs/LOCAL_RELIABILITY_ENGINE.md`, property/contract tests, and
+    aggregation benchmarks;
+  - no runtime dependency, I/O, persistence, logging, clock, registry,
+    framework coupling, or commercial-platform coupling.
+
+- M4 evaluator framework:
+  - public `agent_reliability.evaluation` identity, decision, provenance,
+    result, execution-failure, sync/async protocol, equality, and predicate
+    types;
+  - SDK `EvaluatorRunner` with explicit sync/async methods, injected clock, and
+    existing sanitized diagnostics;
+  - `RunHandle.record_evaluation(...)` for associating completed attributable
+    results with a run;
+  - `docs/EVALUATOR_FRAMEWORK.md`, `docs/EVALUATION_PROVENANCE.md`, and
+    ADR-0007;
+  - no new runtime dependency, provider/framework coupling, input capture,
+    registry, timeout system, or OTel evaluation mapping.
+
+- M3 `RunContextBridge`/`RunContextScope`, optional
+  `OpenTelemetryRunContextBridge`, ADR-0006, `docs/OTEL_MAPPING.md`, real OTel
+  context/parentage/failure tests, and comparison benchmarks.
+
+- Milestone M2: the Python instrumentation SDK, under
+  `agent_reliability.sdk` (not the package root), plus new
+  `agent_reliability.ports`/`agent_reliability.adapters` runtime types:
+  - `sdk`: `AgentReliability`, `RunHandle`, `current_run()`,
+    `DiagnosticHandler`/`LoggingDiagnosticHandler`/`SdkDiagnostic`
+  - `ports`: `Clock`, `RunIdGenerator`, `EventSink`,
+    `RunStarted`/`RunCompleted`/`RunFailed`/`EvaluationRecorded`/`InstrumentationEvent`
+  - `adapters`: `SystemClock`, `UuidRunIdGenerator`, `NoOpEventSink`,
+    `InMemoryEventSink`, `CompositeEventSink`
+  - ADR-0003 (SDK runtime/context architecture) and ADR-0004
+    (instrumentation failure isolation) — the latter defines the exact
+    rule for what raises and what is suppressed, applied consistently
+    across the SDK.
+  - `benchmarks/bench_sdk.py`: the first milestone with a real runtime
+    path to measure (engineering baselines only, not marketing claims).
+  - No M1 (`agent_reliability.domain`) code was modified. No new
+    runtime dependencies (standard library only).
+
+- Milestone M1: the reliability domain kernel, under
+  `agent_reliability.domain` (not the package root). Pure, typed,
+  deterministic, zero I/O:
+  - `EvaluationOutcome` (`PASS`/`FAIL`/`UNKNOWN`)
+  - `AgentIdentity`
+  - `AgentRun`, `RunStatus` (minimal four-state lifecycle)
+  - `UnknownPolicy`, `ObservationCounts`, `RatioResult`, `compute_ratio`
+  - `ObjectiveDirection`, `Slo`, `SloStatus`, `SloEvaluation`, `evaluate_slo`
+  - `BudgetStatus`, `ErrorBudget`, `BurnRate`, `compute_error_budget`,
+    `compute_burn_rate`
+  - ADR-0002 resolves the ratio-math and undefined-data semantics this
+    kernel implements, correcting an ambiguity found in the M0
+    `SLO_SEMANTICS.md` draft. All public symbols remain pre-alpha with
+    no compatibility guarantee (see `docs/COMPATIBILITY.md`).
+
+- Repository foundation (milestone M0): engineering principles,
+  architecture and domain-model specifications, telemetry contract
+  approach, SLO/error-budget/burn-rate mathematics specification,
+  security threat model, testing strategy, ADR process
+  (ADR-0001: architecture and dependency boundaries), and CI.
+- `agent_reliability` package skeleton: layered `domain` / `application`
+  / `ports` / `adapters` / `experimental` structure, all currently
+  empty placeholders. Public API is limited to `__version__`.

agent_reliability-1.0.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,38 @@
+# Code of Conduct
+
+## Our pledge
+
+We want participation in this project to be a harassment-free experience
+for everyone, regardless of age, body size, disability, ethnicity,
+gender identity and expression, level of experience, nationality,
+personal appearance, race, religion, or sexual identity and orientation.
+
+## Our standards
+
+Examples of behavior that contributes to a positive environment:
+
+- Being respectful of differing viewpoints and experiences
+- Giving and gracefully accepting constructive feedback
+- Focusing on what is best for the project and its users
+
+Examples of unacceptable behavior:
+
+- Harassment, insults, or derogatory comments, public or private
+- Publishing others' private information without explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement
+
+Instances of unacceptable behavior may be reported to the project
+maintainers through the reporting channel described in
+[SECURITY.md](SECURITY.md) (for reports where privacy matters) or via a
+public issue for less sensitive concerns. Maintainers are responsible
+for clarifying standards and are expected to take appropriate, fair
+corrective action in response to any behavior they deem inappropriate.
+
+## Scope
+
+This Code of Conduct applies within all project spaces (issues, pull
+requests, discussions) and when an individual is representing the
+project in public spaces.

agent_reliability-1.0.0/CONTRIBUTING.md

@@ -0,0 +1,62 @@
+# Contributing
+
+Thank you for your interest. Agent Reliability `1.0.0` is released; public
+APIs documented as stable in [docs/GA_CONTRACT.md](docs/GA_CONTRACT.md)
+follow Semantic Versioning. Questions and precise critique remain welcome.
+
+## Development setup
+
+Requires Python 3.11–3.13.
+
+```bash
+python -m venv .venv
+# POSIX: source .venv/bin/activate
+# Windows PowerShell: .venv\Scripts\Activate.ps1
+python -m pip install -e ".[dev]"
+```
+
+Branches use `<type>/<short-description>`, for example
+`fix/burn-rate-division-by-zero`. Keep commits reviewable and do not mix
+unrelated semantic and formatting changes.
+
+## Before opening a PR
+
+```bash
+ruff check .
+ruff format --check .
+mypy src
+pytest --cov --cov-report=term-missing
+python -m build
+python scripts/verify_release_artifacts.py
+```
+
+CI repeats these checks. New behavior needs the appropriate unit, property,
+contract, or integration tests; see [testing strategy](docs/TESTING_STRATEGY.md).
+Reliability mathematics changes require invariant/property coverage.
+
+## Architecture and documentation
+
+Preserve the dependency direction and anti-goals in
+[architecture](docs/ARCHITECTURE.md) and
+[engineering principles](docs/ENGINEERING_PRINCIPLES.md). Do not add provider,
+agent-framework, storage, or transport dependencies to the domain.
+
+Hard-to-reverse decisions require an ADR alongside the implementation; see the
+[ADR guide](docs/adr/README.md). Update documentation in the same PR whenever
+public behavior or semantics change.
+
+## Compatibility
+
+Before 1.0, public APIs may receive limited breaking refinement. See
+[compatibility](docs/COMPATIBILITY.md). Do not introduce convenience APIs that
+hide evaluator methodology, UNKNOWN policy, provenance, or SLO choices.
+
+## PR description checklist
+
+- What changed and why?
+- What alternatives were considered?
+- What are the compatibility, security, and performance implications?
+- How was it verified?
+
+Security vulnerabilities must use the private process in [SECURITY.md](SECURITY.md),
+not a public issue.