astraform-conformance 0.3.5__tar.gz

astraform_conformance-0.3.5/PKG-INFO

@@ -0,0 +1,385 @@
+Metadata-Version: 2.4
+Name: astraform-conformance
+Version: 0.3.5
+Summary: Conformance runner for Astraform domain providers using remote-domain.v1
+License-Expression: Apache-2.0
+Project-URL: Documentation, https://astraform.ai/python-quickstart#step-4
+Project-URL: Source, https://github.com/astraform/sdk-python
+Project-URL: Publishing, https://github.com/astraform/sdk-python#readme
+Keywords: astraform,domain-provider,simulation,conformance
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: httpx<1.0,>=0.28
+Requires-Dist: anyio<5.0,>=4.14.2
+Requires-Dist: jsonschema<5.0,>=4.23
+Requires-Dist: cryptography<51.0,>=50.0.0
+Requires-Dist: rfc8785==0.1.4
+Provides-Extra: test
+Requires-Dist: fastapi<1.0,>=0.115; extra == "test"
+Requires-Dist: starlette<2.0,>=1.3.1; extra == "test"
+Requires-Dist: pytest<10.0,>=9.0.3; extra == "test"
+
+# Astraform Domain Provider Conformance for Python
+
+Version **0.3.5** is prepared with the checksum-pinned, published **1.2.3**
+(`v1.2.3`) provider contract. Install this version after publication, or build
+from the checkout beforehand. Published 0.3.4 keeps its 1.2.2 contracts.
+The provider wire protocol remains **`remote-domain.v1`**.
+
+The validation tool is distributed as `astraform-conformance`, replacing
+`astraform-remote-domain-conformance`. The platform client and provider helpers
+are supplied by the separate `astraform-sdk` distribution.
+Existing Python imports and CLI commands are unchanged. See the
+[0.3.5 release notes](https://github.com/astraform/sdk-python/blob/main/docs/releases/0.3.5.md)
+for upgrade guidance.
+
+This is the reusable conformance harness for Python domain providers using
+`remote-domain.v1`.
+
+Use it to validate your provider's protocol behavior before connecting the
+service to Astraform.
+
+This package exercises the real lifecycle:
+
+- `manifest`
+- `prepare`
+- `status`
+- `execute-work`
+- `inspection`
+- `shutdown`
+
+It validates payloads against the exact embedded `remote-domain.v1` contract
+release resolved automatically during package builds. The package includes its
+release lock and provenance under `_contract/contracts/`; a
+`LOCAL_PRERELEASE` snapshot is distinguishable from the published baseline.
+
+It also includes a lightweight remote Policy Wind Tunnel provider conformance
+path for `/policy-wind-tunnel/...` routes. That path checks pack metadata,
+domain/report labels, `outcomeSchema`, CEL-compatible `policyExpressions`,
+presets, run creation, status, lifecycle control, the generic
+`policy_wind_tunnel_bundle.v1` envelope, artifact, and evidence-pack
+availability. A weak bundle that only returns `schemaVersion` and `runId` is
+not conformant; it must expose the
+domain, pack, preset, branches, metrics, segments, timeline, decision gates,
+artifacts, and domain payload that the dashboard and outcome report can render
+without custom frontend code.
+
+## Install
+
+Version **0.3.2** hardens response correlation, JSON structure, finalization
+artifact collections and native/scorecard validation. See the
+[resource and response limits](#conformance-resource-and-response-integrity-limits)
+for intentional rejection boundaries and configurable artifact/role budgets.
+Its package and harness versions follow the SDK patch release. Version
+**0.3.5** includes the **1.2.3** contract and both native profile schemas. Python
+**3.12+** is required.
+This validation tool is installed separately from the SDK; it is not a
+runtime dependency of domain services or simulation launchers. The
+`remote-domain-conformance` command is unchanged.
+
+The [PyPI package page](https://pypi.org/project/astraform-conformance/)
+lists published versions.
+Install it with:
+
+```bash
+python -m pip install astraform-conformance==0.3.5
+```
+When upgrading from 0.3.4 or older, uninstall the old distribution first. Both
+distributions provide the same Python modules and must not be installed together:
+
+```bash
+python -m pip uninstall astraform-remote-domain-conformance
+python -m pip install astraform-conformance==0.3.5
+```
+
+Update dependency declarations and regenerate application locks. Published
+0.3.4 and older releases retain the original distribution name.
+
+
+See the [0.3.2 release notes](https://github.com/astraform/sdk-python/blob/main/docs/releases/0.3.2.md)
+(repository access required).
+Repository-local development resolves published contract 1.2.3:
+
+```bash
+pip install -e './conformance-python[test]'
+```
+
+## Run Against A Live Service
+
+```bash
+remote-domain-conformance \
+  --base-url http://localhost:8092 \
+  --domain-id my-domain
+```
+
+Require Population Builder catalog conformance:
+
+```bash
+remote-domain-conformance \
+  --base-url http://localhost:8092 \
+  --domain-id my-domain \
+  --population-catalog
+```
+
+Validate a catalog file before the provider serves it:
+
+```bash
+remote-domain-conformance \
+  --domain-id my-domain \
+  --population-catalog-file ./population_catalog.json
+```
+
+## Run Against A Local ASGI App
+
+```bash
+remote-domain-conformance \
+  --app my_remote_domain.main:app \
+  --domain-id my-domain
+```
+
+Baseline conformance performs the published v1 lifecycle once. To prove the
+stronger async worker transport/effect boundary, pass an explicit out-of-band
+profile:
+
+```bash
+remote-domain-conformance \
+  --app my_remote_domain.main:app \
+  --domain-id my-domain \
+  --opportunity-worker-profile conformance/opportunity-worker-profile.json \
+  --provider-artifact-digest "sha256:<64 lowercase hex characters>" \
+  > build/conformance-run-report.json
+```
+
+Extract the nested `opportunityWorkerConformanceReport`, then sign that file in
+a separate protected job which runs no provider or repository code:
+
+```bash
+remote-domain-conformance \
+  --sign-conformance-report build/opportunity-worker-conformance-report.json \
+  --opportunity-worker-profile conformance/opportunity-worker-profile.json \
+  --attestation-private-key /secure/release-ed25519-private-key.pem \
+  --attestation-key-id partner-release \
+  --attestation-output build/opportunity-worker-attestation.dsse.json
+```
+
+The protected signer recomputes the supplied profile digest and requires
+exactly one matching passing result for every classified tool. The signed
+`conformanceReportDigest` covers the report's passing baseline
+lifecycle, cross-attempt replay, and per-tool probe outcomes, plus its exact OCI
+subject binding. The public report schema is shipped in the opportunity-worker
+contract profile. Release automation must fail when the protected signer is
+unavailable; an ephemeral-key fallback is not a release proof.
+
+The profile schema version is
+`remote_domain_opportunity_worker_conformance_profile.v1`; it contains exact
+`toolEffects`, one safe probe per tool, and optional `personaConfiguration`.
+It is not part of the `remote-domain.v1` manifest schema.
+
+Worker-profile responses are consumed from `response.content`, not decoded
+response text. Before a passing report can exist, the runner requires one exact
+JSON Content-Type, strict UTF-8, valid Unicode scalar text without U+0000,
+unique object keys, one JSON value, and lossless I-JSON/RFC 8785 numbers. Signed
+worker proof binds protocol responses to the manifest-advertised media type;
+the historical `application/json` fallback remains baseline-v1 compatibility
+only.
+
+## Native Profile Metadata
+
+The native state-transform and reference-state schemas introduced in 0.2.0
+remain included in SDK **0.3.4**, from published contract **1.2.2**.
+Checkout builds include those schemas from contract **1.2.3**.
+SDK 0.3.3 uses contract 1.2.1; SDK 0.3.2 retains the original contract 1.2.0 inputs.
+The existing manifest validation accepts both `capabilities.nativeSimulation`
+profiles and validates complete customer-scoped read-only tool descriptors.
+Legacy manifests remain supported.
+
+Validate a native metadata object with the existing validator:
+
+```python
+from astraform.remote_domain.conformance.runner import RemoteDomainSchemaValidator
+
+validator = RemoteDomainSchemaValidator()
+validator.validate("manifest", provider_manifest)
+validator.validate_native(
+    "remote-domain.native-reference-state.v1",
+    "stateMetadata",
+    provider_checkpoint["data"]["nativeSimulation"],
+)
+```
+
+The definition selects the exact canonical `$defs` entry: for example `manifest`,
+`prepareHints`, `windowContext`, `stateMetadata` or `resultMetadata`. The
+reference-state profile also supplies `customerContext`, `readContext` and
+`checkpoint`; the pure-state profile supplies `readOnlyToolDescriptor` and
+`readOnlyToolPayload`. Unknown profiles and definitions fail explicitly.
+
+These checks validate wire shape and scope metadata. They do not prove native
+transaction atomicity, restart recovery, customer authorization, or complete
+native simulation behavior. The generic lifecycle CLI does not become native
+execution certification merely because it accepts a native manifest.
+
+## Run Policy Wind Tunnel Provider Conformance
+
+```bash
+remote-domain-conformance \
+  --wind-tunnel \
+  --base-url http://localhost:8092 \
+  --pack-id my-domain-policy-pack \
+  --preset-id conformance-proof
+```
+
+When the pack advertises
+`opportunityGenerationFinalizationMode=PLATFORM_OWNED_V2`, the wind-tunnel
+runner automatically executes a clock-driven V2 probe. It pins the immutable
+provider revision, cancels a generated run, finalizes it with a stable operation
+ID, validates the strict receipt schema, downloads and hashes every declared
+artifact, proves an exact replay, and rejects contradictory reuse of the
+operation ID.
+
+V2 conformance requires a `ScenarioLabFinalizationV2ProbeDriver` backed by real
+OS processes. The driver supplies a started `subprocess.Popen` handle bound to
+the provider base URL under test, exposes
+the durable PREPARED receipt and artifact bytes, terminates the original
+provider, and launches a distinct process against the same durable state. The
+harness independently verifies that the old process exited and its endpoint is
+unavailable before accepting the replacement PID, then requires the replacement
+process to publish the byte-equivalent canonical `/pack` identity. Legacy V1
+coverage is provisioned out of band as recovery state; the harness never launches
+a fresh `PLATFORM_OWNED_V1` run. The driver also runs two distinct child processes
+against the configured receipt/artifact store so the harness can verify lock
+exclusion, cross-process visibility, and atomic publication. Both the successful
+and terminal-precedence paths cross a real process restart. Completion delivery
+is failed once deliberately and must retry independently without another
+`FINALIZE` request. The harness also injects each
+published artifact and receipt fault through the driver and verifies rejection
+through the provider HTTP boundary; an in-process ASGI app swap or a relabelled
+process is not restart proof.
+
+## Python Usage
+
+```python
+import asyncio
+
+from astraform.remote_domain.conformance.runner import ConformanceScenario
+from astraform.remote_domain.conformance.runner import OpportunityWorkerConformanceProfile
+from astraform.remote_domain.conformance.runner import PolicyWindTunnelConformanceScenario
+from astraform.remote_domain.conformance.runner import ToolProbe
+from astraform.remote_domain.conformance.runner import run_conformance
+from astraform.remote_domain.conformance.runner import run_opportunity_worker_conformance
+from astraform.remote_domain.conformance.runner import run_policy_wind_tunnel_conformance
+
+
+report = asyncio.run(
+    run_conformance(
+        base_url="http://localhost:8092",
+        scenario=ConformanceScenario(domain_id="my-domain"),
+    )
+)
+
+print(report.to_dict())
+
+worker_report = asyncio.run(
+    run_opportunity_worker_conformance(
+        base_url="http://localhost:8092",
+        scenario=ConformanceScenario(domain_id="my-domain"),
+        profile=OpportunityWorkerConformanceProfile(
+            tool_effects={"lookup_case": "READ_ONLY"},
+            tool_probes=(ToolProbe("lookup_case", {"caseId": "case-1"}),),
+        ),
+        provider_artifact_digest="sha256:<64 lowercase hex characters>",
+    )
+)
+
+wind_tunnel_report = asyncio.run(
+    run_policy_wind_tunnel_conformance(
+        base_url="http://localhost:8092",
+        scenario=PolicyWindTunnelConformanceScenario(
+            pack_id="my-domain-policy-pack",
+            preset_id="conformance-proof",
+        ),
+    )
+)
+
+print(wind_tunnel_report.to_dict())
+```
+
+## Publishing Status
+
+Release validation starts with the standalone package tests:
+
+```bash
+pytest sdk-python/tests conformance-python/tests
+```
+
+PyPI artifacts are immutable. Do not rerun a publish for an already published
+version; run smoke-only verification or bump the SDK version.
+
+Public package note: PyPI distributions expose this SDK implementation. Keep
+host runtime internals and domain-private logic out of this package.
+
+
+### HTTP safety limits
+
+All provider calls stay on the configured HTTP(S) origin. Advertised operation
+paths must be relative paths; absolute URLs, authority paths (`//host/path`),
+backslashes, fragments and control/space characters are rejected before lifecycle
+operations start. Redirects are never followed. Localhost, private-network URLs
+and base URLs mounted under a path remain supported.
+
+The Python runners accept `limits=ConformanceHttpLimits(...)` (exported from
+`astraform.remote_domain.conformance`). Defaults are 4 MiB per JSON response,
+64 MiB per artifact/evidence-pack response, and a 30-second total deadline per
+HTTP request, including headers and streamed body. The same settings apply to
+polling, tool probes and restart requests. CLI equivalents are `--max-json-bytes`,
+`--max-artifact-bytes` and `--request-timeout-seconds`. Limits must be positive;
+the deadline must also be finite. The deadline is per request, not for a full run.
+
+The runner requests `Accept-Encoding: identity` and rejects HTTP content
+compression before reading the body, preventing decompression from bypassing the
+byte limit. Providers must honor this request. ZIP artifacts are still supported
+as ordinary `application/zip` bytes. Both successful and error responses are
+bounded; exceeding a limit or hitting an HTTP transport timeout is a conformance
+failure, never successful crash or restart evidence. Connection-refused and
+disconnect errors remain available to the existing restart checks.
+
+Imported `app=` implementations, custom transports and restart probe drivers run
+trusted Python in the harness process. In particular, HTTPX's ASGI transport
+buffers local app output before returning it; these limits do not sandbox local
+Python code or bound memory allocated inside that app. Use a separate provider
+process with `base_url=` to enforce the streamed network response boundary.
+
+### Conformance resource and response integrity limits
+
+Conformance treats the selected provider's responses as untrusted protocol data.
+Every lifecycle, replay and tool envelope must echo the originating request ID
+exactly; a successful schema check cannot substitute for that correlation.
+
+JSON parsing and public canonicalization enforce a structural budget before
+recursive normalization or RFC 8785 serialization: maximum depth 64, 100,000
+values/property names, and 4,096 characters per property name. These limits
+apply to arrays as well as objects. Diagnostics display at most 256 path
+characters plus an ellipsis; payloads are rejected, never truncated.
+
+The existing ConformanceHttpLimits additionally accepts
+max_finalization_artifacts (64), max_finalization_total_bytes (128 MiB), and
+max_scorecard_roles (256). CLI flags use the corresponding hyphenated names.
+All three are positive integer limits. Existing first-three positional arguments
+retain their meaning.
+
+V2 collectors reject excessive counts and declared aggregate lengths before
+fetching or reading prepared artifacts, then enforce actual byte totals.
+PUBLISHED verification does not retain a full collection, and restart comparison
+checks one response at a time against the original bounded snapshots rather
+than retaining another collection. A comparison can hold the original bounded
+collection and one bounded response at once; HTTP buffering has a bounded
+temporary copy. Zero-byte artifacts remain valid. The trusted probe driver
+returns already allocated bytes; these limits cannot sandbox allocation inside
+that operator-provided driver.
+
+Scorecard role counts are checked before schema evaluation; duplicates remain
+forbidden by exact scorecard ID and catalog coverage. Native manifests reject
+non-string, oversized or overlong workKinds before uniqueness validation,
+including standalone native-profile validation.