focus-data-toolkit 0.12.0__tar.gz → 0.13.0__tar.gz

{focus_data_toolkit-0.12.0 → focus_data_toolkit-0.13.0}/CHANGELOG.md

@@ -9,6 +9,28 @@ policy.
 
 ## [Unreleased]
 
+## [0.13.0] — 2026-09-05
+
+Prepared in source; publishing is a separate release step.
+
+- Correct period-subscription effective costs and derive illustrative 10% taxes
+  from earlier untaxed usage with exact amount and currency lineage.
+- Stabilize offer/SKU/price identities and public prices; correct recognized storage
+  properties, AWS namespaces and owning subaccounts, including allocations.
+- Model partially used fleets of 500 machines, exact annual capacity and complete
+  budget-aware groups. Add targeted 1.3 negotiated contracts, shared contract parents
+  and a common generation registry; Contract Commitment APIs accept `include_credits`.
+- Omit inapplicable ContractApplied metric properties while retaining legacy input
+  compatibility. Preserve exact invoice aggregates and widen affected Parquet cost
+  columns to `decimal128(38,17)` without silent rounding.
+- Replace FAIL-only official gating with complete inventories, violation counts,
+  resource/source/data fingerprints, archived reports and separately reviewed
+  candidate evidence. Public `validate --official` now rejects false-green exits.
+- **New byte baseline:** regenerated provider goldens, archived before examples,
+  independent corruption tests and reproducible statistics. See
+  [generator corrections](docs/generator-corrections.md) for assumptions, residual
+  official failures and the deliberate differences from upstream PRs 6 and 7.
+
 ## [0.12.0] — 2026-08-21
 
 Back-ports the reviewed FOCUS-Sample-Data conformance fixes into the generator engine.
@@ -695,6 +717,7 @@ conformance defects.
 <!-- Reference links. 0.2.0/0.3.0 were pre-release development milestones and were never tagged
      or published, so only the first public release (0.9.0) has a tag link. -->
 [Unreleased]: https://github.com/guymano/focus-data-toolkit/compare/v0.12.0...HEAD
+[0.13.0]: https://github.com/guymano/focus-data-toolkit/compare/v0.12.0...v0.13.0
 [0.12.0]: https://github.com/guymano/focus-data-toolkit/compare/v0.11.0...v0.12.0
 [0.11.0]: https://github.com/guymano/focus-data-toolkit/compare/v0.11.0rc1...v0.11.0
 [0.11.0rc1]: https://github.com/guymano/focus-data-toolkit/compare/v0.9.0...v0.11.0rc1

{focus_data_toolkit-0.12.0 → focus_data_toolkit-0.13.0}/CONTRIBUTING.md

@@ -49,6 +49,8 @@ pytest -m packaging                        # build + install a wheel/sdist (need
 
 python scripts/check_pinned_actions.py     # every GitHub Action is SHA/digest pinned
 python scripts/verify_model_provenance.py  # the embedded FOCUS model matches its provenance
+python scripts/describe_generated_samples.py # independently audited sample statistics
+python scripts/validate_official_samples.py --check-existing # archived reports + provenance
 ```
 
 A PR is expected to be green on lint, types, and the default test suite, and to
@@ -63,7 +65,8 @@ identical toolkit version and identical parameters (`provider`, `focus_version`,
 - Keep generator draws in a fixed order; do not introduce a clock, real RNG, or
   environment-dependent behaviour into generation/conversion.
 - Golden snapshots live in `tests/fixtures/golden/`:
-  - `compatibility_golden/` — outputs verified correct against the FOCUS spec.
+  - `compatibility_golden/` — reviewed synthetic outputs with executable semantic checks
+    and documented residual official-model/validator failures.
     These are a **byte-for-byte** intra-version contract.
   - `correctness_migration/` — cases with a *known* prior defect: the old output
     is archived for comparison, and the **new** output is the spec-correct one.
@@ -74,6 +77,11 @@ identical toolkit version and identical parameters (`provider`, `focus_version`,
 - If a diff changes a snapshot you did **not** intend to touch, treat it as a
   regression and investigate before updating the fixture.
 
+Generator changes also require a fresh official-validation **candidate**, review
+of every changed rule/state/count and explicit evidence promotion. See
+[generator corrections](docs/generator-corrections.md). A successful reference
+comparison does not certify complete FOCUS conformance.
+
 ## Changing the embedded FOCUS model
 
 The FOCUS 1.4 model JSON (`src/focus_data_toolkit/model/focus_1_4_model.json`) is

{focus_data_toolkit-0.12.0 → focus_data_toolkit-0.13.0}/MANIFEST.in

@@ -18,7 +18,9 @@ include tools/extract_focus_1_4_model.py
 recursive-include scripts *.py
 recursive-include schema *.json
 recursive-include docs *.md
+include docs/generator-statistics.json
 recursive-include tests *.py *.csv *.json *.md
+recursive-include tests/fixtures/official/generator_validation *.log
 
 global-exclude *.py[cod]
 global-exclude __pycache__

{focus_data_toolkit-0.12.0 → focus_data_toolkit-0.13.0}/NOTICE

@@ -46,6 +46,17 @@ documentation page it was authored against.
 
 3. Optional dependency: the official FinOps FOCUS validator
 -----------------------------------------------------------
+The source test evidence at tests/fixtures/official/generator_validation includes
+rule expressions from the FOCUS 1.2.0.1 and 1.3.0.1 models (FinOps Foundation,
+CC-BY-4.0, source and license links in section 2 above) and captured validator
+results. Model expressions are reproduced without modification; surrounding
+explanations and synthetic example records are authored by this project.
+
+Generator corrections adapt the behavior examined in FOCUS-Sample-Data PRs 6/7,
+at commits e14921a3ba8d5f75a7b899d878e372999482ba9a and
+99917010b27f7c0ca3586f8e0daab7e1c8103dac. See docs/generator-corrections.md for
+source links and the deliberate semantic adaptations.
+
 The optional `[validator]` extra installs `focus-validator` from PyPI, which is
 published by the FinOps Foundation under its own license. It is not vendored or
 redistributed by this project; it is resolved by your package manager on

focus_data_toolkit-0.13.0/PKG-INFO

@@ -0,0 +1,157 @@
+Metadata-Version: 2.4
+Name: focus-data-toolkit
+Version: 0.13.0
+Summary: Generate provider-realistic FOCUS 1.2/1.3 sample data (AWS, Azure, GCP) and convert it to the four FOCUS 1.4 datasets — strictly from source facts, completed by client supplements including native AWS/Azure/GCP exports — with schema detection, gap analysis, a cross-dataset validation gate and atomic writes.
+Author: Guy-Hermann Adiko
+License-Expression: MIT AND CC-BY-4.0
+Project-URL: Homepage, https://github.com/guymano/focus-data-toolkit
+Project-URL: Source, https://github.com/guymano/focus-data-toolkit
+Project-URL: Issues, https://github.com/guymano/focus-data-toolkit/issues
+Project-URL: Changelog, https://github.com/guymano/focus-data-toolkit/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://github.com/guymano/focus-data-toolkit#readme
+Keywords: finops,focus,billing,cost,sample-data
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Office/Business :: Financial
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+License-File: LICENSES/CC-BY-4.0.txt
+Provides-Extra: parquet
+Requires-Dist: pyarrow<26,>=15; extra == "parquet"
+Requires-Dist: tzdata>=2024.1; sys_platform == "win32" and extra == "parquet"
+Provides-Extra: scale
+Requires-Dist: focus-data-toolkit[parquet]; extra == "scale"
+Provides-Extra: validator
+Requires-Dist: focus-validator<3,>=2.1; python_version >= "3.12" and extra == "validator"
+Provides-Extra: studio
+Requires-Dist: fastapi<1,>=0.110; extra == "studio"
+Requires-Dist: uvicorn[standard]<1,>=0.29; extra == "studio"
+Requires-Dist: python-multipart<1,>=0.0.9; extra == "studio"
+Provides-Extra: studio-all
+Requires-Dist: focus-data-toolkit[parquet,studio]; extra == "studio-all"
+Provides-Extra: all
+Requires-Dist: focus-data-toolkit[parquet,studio,validator]; extra == "all"
+Provides-Extra: release
+Requires-Dist: build<2,>=1; extra == "release"
+Requires-Dist: twine<8,>=5; extra == "release"
+Provides-Extra: dev
+Requires-Dist: focus-data-toolkit[parquet,release,studio]; extra == "dev"
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: pytest-cov<8,>=5; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: mypy<2.4,>=1.11; extra == "dev"
+Requires-Dist: jsonschema<5,>=4.21; extra == "dev"
+Requires-Dist: httpx<1,>=0.27; extra == "dev"
+Dynamic: license-file
+
+# focus-data-toolkit
+
+Generate realistic **FOCUS 1.2 / 1.3** cost and usage data (AWS, Azure, GCP), convert it to
+**FOCUS 1.4**, and validate the result. One dependency-free Python core, usable three ways.
+
+[![PyPI](https://img.shields.io/pypi/v/focus-data-toolkit)](https://pypi.org/project/focus-data-toolkit/)
+[![Python](https://img.shields.io/pypi/pyversions/focus-data-toolkit)](https://pypi.org/project/focus-data-toolkit/)
+[![License: MIT AND CC-BY-4.0](https://img.shields.io/badge/license-MIT%20AND%20CC--BY--4.0-blue)](LICENSE)
+[![CI](https://github.com/guymano/focus-data-toolkit/actions/workflows/ci.yml/badge.svg)](https://github.com/guymano/focus-data-toolkit/actions/workflows/ci.yml)
+
+[FOCUS](https://focus.finops.org) is the open standard for cloud cost and usage data. FOCUS 1.4 defines four datasets: Cost and Usage, Contract Commitment, Billing Period, and Invoice Detail. The toolkit is honest by design: any value it cannot derive from your source is left empty (**strict** mode) or filled with clearly labelled assumptions (**synthetic** mode).
+
+The source prepares **0.13.0** with corrected subscriptions, traceable taxes,
+stable offers and partially used fleets of 500 machines. See the
+[generator corrections and validation evidence](docs/generator-corrections.md)
+for the new byte baseline and sample assumptions. The Docker examples below
+continue to pin the previously released 0.12.0 image.
+
+## Three ways to use it
+
+| Interface | Best for | Get started |
+|---|---|---|
+| **Studio** | FinOps users who prefer a UI | `pip install "focus-data-toolkit[studio]"` then `focus-toolkit ui` |
+| **Runner** | Automation and large volumes | `docker run --rm ghcr.io/guymano/focus-data-toolkit:0.12.0 version` |
+| **CLI & SDK** | Engineers and scripts | `pip install focus-data-toolkit` then `focus-toolkit --help` |
+
+All three run the same core, so their outputs are byte-for-byte identical. Studio is a local web app (binds `127.0.0.1`, token-guarded, no data leaves your machine, see [docs/studio.md](docs/studio.md)). Runner is a non-root OCI image whose entrypoint is the CLI (see [docs/runner.md](docs/runner.md)).
+
+## Install
+
+```bash
+pip install focus-data-toolkit
+```
+
+The core needs only the standard library (Python 3.11+). Optional extras:
+
+| Extra | Adds |
+|---|---|
+| `parquet` | Parquet I/O: columnar, decimal128, Hive partitioning (alias: `scale`) |
+| `studio` | the local web UI (`focus-toolkit ui`) |
+| `studio-all` | `studio` + `parquet` |
+| `validator` | the official FinOps validator (`--official`, needs Python 3.12+) |
+| `all` | everything above |
+
+Also available via `pipx install`, `uv tool install`, or `docker pull ghcr.io/guymano/focus-data-toolkit:0.12.0`.
+
+## Quickstart
+
+```bash
+# 1. Generate sample data (aws|azure|gcp, FOCUS 1.2|1.3)
+focus-toolkit generate --provider aws --focus-version 1.3 --rows 1000 --out ./out
+
+# 2. Convert a 1.2/1.3 Cost and Usage file to FOCUS 1.4 (source version auto-detected)
+focus-toolkit convert --cost-and-usage out/focus_1_3_cost_and_usage_aws.csv --out ./focus-1.4
+
+# 3. Open the Studio web app (needs the studio extra)
+focus-toolkit ui
+```
+
+Or use it as a Python library, the same engine behind all three interfaces:
+
+```python
+from focus_data_toolkit import convert_files
+
+convert_files("cost_and_usage.csv", "focus-1.4", mode="strict")
+```
+
+## What it does, and what it doesn't
+
+- Converts FOCUS 1.2/1.3 Cost and Usage into the four FOCUS 1.4 datasets.
+- The three other datasets come only from [supplements](docs/supplements.md) or synthetic mode.
+  There is no FOCUS 1.4 generator and no invalid-data generator.
+- Built for real data: bounded-memory streaming (`--stream`), atomic writes, a deterministic
+  manifest with per-column lineage, structured `FDT-*` diagnostics, cross-dataset validation.
+- Single-node engine. See [docs/compatibility.md](docs/compatibility.md) for the support matrix.
+
+## Exit codes
+
+| Code | Meaning |
+|---|---|
+| `0` | success, no assumptions |
+| `1` | lint / validation / write failure |
+| `2` | invalid input or arguments |
+| `3` | strict result intentionally incomplete |
+| `4` | synthetic result contains assumptions |
+| `5` | disk space or budget exhausted (streaming / Runner) |
+| `130` | cancelled (Ctrl-C / SIGTERM), nothing partial is published |
+
+Add `--exit-policy pipeline` in `set -e` or CI scripts to treat codes `3` and `4` as `0`.
+
+## Trust
+
+- Deterministic: same input, same output bytes, all listed in `SHA256SUMS`.
+- The embedded FOCUS 1.4 model is hash-pinned to the FinOps source workbook; see [docs/model-provenance.md](docs/model-provenance.md).
+- Releases are reproducible, signed (Sigstore/cosign), with SBOM and attestations; see [docs/releasing.md](docs/releasing.md).
+- Private: no network I/O, no telemetry, no credentials; see [docs/security-model.md](docs/security-model.md).
+
+## Documentation
+
+[Studio](docs/studio.md) · [Runner](docs/runner.md) · [Supplements](docs/supplements.md) · [Compatibility](docs/compatibility.md) · [Versioning](docs/versioning.md) · [Security model](docs/security-model.md) · [Model provenance](docs/model-provenance.md) · [Releasing](docs/releasing.md) · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md) · [Security policy](SECURITY.md)
+
+## License
+
+Code is MIT (see [LICENSE](LICENSE)). The embedded FOCUS 1.4 data model derives from the FinOps FOCUS workbook, © the FinOps Foundation, CC-BY-4.0 (see [NOTICE](NOTICE)). "FOCUS" and "FinOps" are FinOps Foundation trademarks; this is an independent community project, not endorsed by the FinOps Foundation.

focus_data_toolkit-0.13.0/README.md

@@ -0,0 +1,104 @@
+# focus-data-toolkit
+
+Generate realistic **FOCUS 1.2 / 1.3** cost and usage data (AWS, Azure, GCP), convert it to
+**FOCUS 1.4**, and validate the result. One dependency-free Python core, usable three ways.
+
+[![PyPI](https://img.shields.io/pypi/v/focus-data-toolkit)](https://pypi.org/project/focus-data-toolkit/)
+[![Python](https://img.shields.io/pypi/pyversions/focus-data-toolkit)](https://pypi.org/project/focus-data-toolkit/)
+[![License: MIT AND CC-BY-4.0](https://img.shields.io/badge/license-MIT%20AND%20CC--BY--4.0-blue)](LICENSE)
+[![CI](https://github.com/guymano/focus-data-toolkit/actions/workflows/ci.yml/badge.svg)](https://github.com/guymano/focus-data-toolkit/actions/workflows/ci.yml)
+
+[FOCUS](https://focus.finops.org) is the open standard for cloud cost and usage data. FOCUS 1.4 defines four datasets: Cost and Usage, Contract Commitment, Billing Period, and Invoice Detail. The toolkit is honest by design: any value it cannot derive from your source is left empty (**strict** mode) or filled with clearly labelled assumptions (**synthetic** mode).
+
+The source prepares **0.13.0** with corrected subscriptions, traceable taxes,
+stable offers and partially used fleets of 500 machines. See the
+[generator corrections and validation evidence](docs/generator-corrections.md)
+for the new byte baseline and sample assumptions. The Docker examples below
+continue to pin the previously released 0.12.0 image.
+
+## Three ways to use it
+
+| Interface | Best for | Get started |
+|---|---|---|
+| **Studio** | FinOps users who prefer a UI | `pip install "focus-data-toolkit[studio]"` then `focus-toolkit ui` |
+| **Runner** | Automation and large volumes | `docker run --rm ghcr.io/guymano/focus-data-toolkit:0.12.0 version` |
+| **CLI & SDK** | Engineers and scripts | `pip install focus-data-toolkit` then `focus-toolkit --help` |
+
+All three run the same core, so their outputs are byte-for-byte identical. Studio is a local web app (binds `127.0.0.1`, token-guarded, no data leaves your machine, see [docs/studio.md](docs/studio.md)). Runner is a non-root OCI image whose entrypoint is the CLI (see [docs/runner.md](docs/runner.md)).
+
+## Install
+
+```bash
+pip install focus-data-toolkit
+```
+
+The core needs only the standard library (Python 3.11+). Optional extras:
+
+| Extra | Adds |
+|---|---|
+| `parquet` | Parquet I/O: columnar, decimal128, Hive partitioning (alias: `scale`) |
+| `studio` | the local web UI (`focus-toolkit ui`) |
+| `studio-all` | `studio` + `parquet` |
+| `validator` | the official FinOps validator (`--official`, needs Python 3.12+) |
+| `all` | everything above |
+
+Also available via `pipx install`, `uv tool install`, or `docker pull ghcr.io/guymano/focus-data-toolkit:0.12.0`.
+
+## Quickstart
+
+```bash
+# 1. Generate sample data (aws|azure|gcp, FOCUS 1.2|1.3)
+focus-toolkit generate --provider aws --focus-version 1.3 --rows 1000 --out ./out
+
+# 2. Convert a 1.2/1.3 Cost and Usage file to FOCUS 1.4 (source version auto-detected)
+focus-toolkit convert --cost-and-usage out/focus_1_3_cost_and_usage_aws.csv --out ./focus-1.4
+
+# 3. Open the Studio web app (needs the studio extra)
+focus-toolkit ui
+```
+
+Or use it as a Python library, the same engine behind all three interfaces:
+
+```python
+from focus_data_toolkit import convert_files
+
+convert_files("cost_and_usage.csv", "focus-1.4", mode="strict")
+```
+
+## What it does, and what it doesn't
+
+- Converts FOCUS 1.2/1.3 Cost and Usage into the four FOCUS 1.4 datasets.
+- The three other datasets come only from [supplements](docs/supplements.md) or synthetic mode.
+  There is no FOCUS 1.4 generator and no invalid-data generator.
+- Built for real data: bounded-memory streaming (`--stream`), atomic writes, a deterministic
+  manifest with per-column lineage, structured `FDT-*` diagnostics, cross-dataset validation.
+- Single-node engine. See [docs/compatibility.md](docs/compatibility.md) for the support matrix.
+
+## Exit codes
+
+| Code | Meaning |
+|---|---|
+| `0` | success, no assumptions |
+| `1` | lint / validation / write failure |
+| `2` | invalid input or arguments |
+| `3` | strict result intentionally incomplete |
+| `4` | synthetic result contains assumptions |
+| `5` | disk space or budget exhausted (streaming / Runner) |
+| `130` | cancelled (Ctrl-C / SIGTERM), nothing partial is published |
+
+Add `--exit-policy pipeline` in `set -e` or CI scripts to treat codes `3` and `4` as `0`.
+
+## Trust
+
+- Deterministic: same input, same output bytes, all listed in `SHA256SUMS`.
+- The embedded FOCUS 1.4 model is hash-pinned to the FinOps source workbook; see [docs/model-provenance.md](docs/model-provenance.md).
+- Releases are reproducible, signed (Sigstore/cosign), with SBOM and attestations; see [docs/releasing.md](docs/releasing.md).
+- Private: no network I/O, no telemetry, no credentials; see [docs/security-model.md](docs/security-model.md).
+
+## Documentation
+
+[Studio](docs/studio.md) · [Runner](docs/runner.md) · [Supplements](docs/supplements.md) · [Compatibility](docs/compatibility.md) · [Versioning](docs/versioning.md) · [Security model](docs/security-model.md) · [Model provenance](docs/model-provenance.md) · [Releasing](docs/releasing.md) · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md) · [Security policy](SECURITY.md)
+
+## License
+
+Code is MIT (see [LICENSE](LICENSE)). The embedded FOCUS 1.4 data model derives from the FinOps FOCUS workbook, © the FinOps Foundation, CC-BY-4.0 (see [NOTICE](NOTICE)). "FOCUS" and "FinOps" are FinOps Foundation trademarks; this is an independent community project, not endorsed by the FinOps Foundation.

focus_data_toolkit-0.13.0/docs/generator-corrections.md

@@ -0,0 +1,198 @@
+# Generator corrections prepared for 0.13.0
+
+This is an adapted port of [FOCUS-Sample-Data PR 6](https://github.com/FinOps-Open-Cost-and-Usage-Spec/FOCUS-Sample-Data/pull/6)
+at `e14921a3ba8d5f75a7b899d878e372999482ba9a` and
+[PR 7](https://github.com/FinOps-Open-Cost-and-Usage-Spec/FOCUS-Sample-Data/pull/7)
+at `99917010b27f7c0ca3586f8e0daab7e1c8103dac`. These are examined commits,
+not a dependency on the eventual merged PRs. The previous toolkit baseline is
+`d0c27bac6d1909fa39fe45538fce3a9d391ae0f8` (0.12.0).
+
+## Corrected behavior
+
+| Area | Previous output | New output |
+|---|---|---|
+| Period subscription | Effective cost zero, losing the fee in amortized views | Effective cost equals billed cost; pricing-currency amount follows the same exact exchange rate |
+| Tax | Independent random amount | Exactly 10% of each corresponding source amount on an earlier, previously untaxed Standard Usage record |
+| SKU | Random identifiers and list-price jitter | Stable canonical offer identity and stable public price; a separate subscription offer |
+| Storage | Generator registry omitted recognized storage properties; legacy redundancy values | One property registry; AWS Standard/Zonal and Azure Hot/Local storage values |
+| AWS resource | Payer account and inferred namespace | Owning subaccount and explicit service namespace, also for allocated resources |
+| Commitment | Two to four machine-hours per period | Fleet capacity 500 machine-hours per hourly period, partially used |
+| Group budget | Builder could overflow; final slicing could hide it | Every builder respects its budget; empty groups fall back to ordinary usage; exact final row count |
+| Ordinary 1.3 contract | Every ordinary Standard Usage row negotiated | Deterministic approximately 35% selection; 10% discount for selected rows, public price otherwise and on ordinary allocations |
+| Official result | Set of FAIL rules; an empty report could pass | Full rule inventory, states, violations, totals, process result, pinned resources and reproducible evidence |
+| Invoice aggregate | Rounded to six decimals | Exact sum, including after FOCUS 1.4 conversion and Parquet serialization |
+
+The archived pre-change examples are in
+[`pr67_before.json`](../tests/fixtures/golden/correctness_migration/pr67_before.json).
+The corresponding corrected outputs are the regenerated provider/version CSVs in
+[`compatibility_golden`](../tests/fixtures/golden/compatibility_golden/).
+The independent audit in `scripts/sample_audit.py` checks semantics rather than
+reusing generator builders. Deliberately corrupted subscriptions, taxes, SKUs,
+owners, units, storage properties, contract parents and missing purchases must fail.
+
+## Generation assumptions and identity
+
+Taxes are illustrative, not tax advice or jurisdiction-specific billing logic.
+The description contains a one-based **data record** reference (excluding the CSV
+header). The tax inherits account, service, period, currencies and tags; each
+taxable monetary column is multiplied from its own source, exactly once. If there
+is no eligible untaxed source, the scheduled tax becomes ordinary usage.
+
+Eligible Standard Usage includes a split-allocation child. In that case the tax
+applies to that child's amount, not the total host charge. The source record
+reference identifies the exact share. Tax rows do not copy `Allocated*` fields:
+they are separate tax charges, not additional members of the source allocation
+group. This intentional sample policy is tested explicitly.
+
+Prices are synthetic public and negotiated rates, not current cloud price lists.
+Offer identity includes provider, service, meter, region, unit and canonical SKU
+properties, excluding seed, account, contract and pricing category. Price identity
+also includes the public list prices and billing/pricing currencies. The fixed
+illustrative exchange rates remain part of the provider profiles. Quantity and
+price precision are bounded before multiplication; cost products are not rounded.
+
+Each commitment period contains a purchase, **two partially used fleet records**,
+and unused capacity. Used plus unused capacity is exactly 500; effective costs
+sum to the purchase cost. Fleet URNs and `SyntheticFleetSize: 500` tags explicitly
+describe synthetic aggregation rather than a single cloud machine. The fleet-size
+tag deliberately uses the JSON number 500: it is a numeric synthetic measurement,
+not an imitation of a provider's string-valued native tag export. The annual
+capacity commitment is 500 × 8,760 = **4,380,000 machine-hours**, with the existing
+unit price. Spend commitments retain their cost-based metric. All commitment
+periods lie within their annual term, including samples crossing the billing-month
+hour boundary.
+
+In 1.3, eligible compute gets an ordinary usage-minimum term, storage a negotiated
+rate term, and other services a minimum-spend term. Unselected usage and ordinary
+allocations have no negotiated application. Discount commitments share a parent
+contract in groups of at most three, with compatible provider/account/currency
+identity. Parent dates span their terms. A generation-local registry supplies both
+datasets and avoids state leaking between calls. Contract Commitment APIs now
+accept keyword-only `include_credits=False`; pass the same rows, seed and options
+when generating the two datasets separately.
+
+## ContractApplied compatibility
+
+Generated Spend elements contain applied cost; Usage elements contain applied
+quantity and unit. Inapplicable properties are omitted. The parser still reads
+older objects with explicit nulls and converts existing 1.3 objects carrying both
+metrics without rewriting their meaning.
+
+The 1.3 rule model both describes optional properties and requires five
+properties per element. This conflict is documented in the
+[examined upstream notes](https://github.com/guymano/FOCUS-Sample-Data/blob/99917010b27f7c0ca3586f8e0daab7e1c8103dac/FOCUS-1.3/validation/upstream-notes.md).
+We preserve the toolkit's applicable metric branches. This differs deliberately
+from the upstream sample object shape and remains visible in official failures.
+
+## Reproducible official evidence
+
+The reference uses **focus-validator 2.2.1**, models **1.2.0.1 / 1.3.0.1**,
+applicability **ALL**, 1,000 rows, default seeds 1202/1302, credits disabled.
+Both model hashes and the currency resource hash are checked before execution.
+Dataset roots, dependencies and nested rule references independently determine
+the expected rule inventory. Empty, duplicate, incomplete, inconsistent or crashed
+reports fail. Any state or violation-count change fails the reference comparison,
+including PASS becoming SKIPPED with no new FAIL rule.
+
+[`baseline.json`](../tests/fixtures/official/generator_validation/baseline.json)
+contains generation-source fingerprints (LF-normalized), exact CSV hashes,
+resource versions/hashes, parameters, every rule result and per-failure evidence.
+Source hashes remain a strict gate, including version and audit-script changes.
+Even a source-only edit requires a reviewed new candidate; dataset hashes cover
+the selected fixtures, while source provenance identifies the code used to
+produce and interpret the evidence. Mismatches now list the changed files.
+Each residual failure has its model expression, explanation, a minimal source
+record and independently checked population where the count represents rows.
+Composite failures count expressions instead of rows and are labeled separately.
+Raw stdout/stderr logs are archived beside it; stdout hashes preserve their exact
+original bytes. The nine archived stderr logs are required to be empty, and the
+offline check verifies this. Successful live comparisons remove their temporary
+files; failed runs retain them for investigation. Candidate directories persist.
+Live log paths/timing can differ across operating systems, but
+the complete semantic inventory must match.
+
+| Dataset | Provider | PASS | FAIL | SKIPPED |
+|---|---|---:|---:|---:|
+| Cost and Usage 1.2 | AWS | 353 | 15 | 210 |
+| Cost and Usage 1.2 | Azure / GCP, each | 357 | 11 | 210 |
+| Cost and Usage 1.3 | AWS | 428 | 39 | 297 |
+| Cost and Usage 1.3 | Azure / GCP, each | 432 | 35 | 297 |
+| Contract Commitment 1.3 | Each provider | 76 | 0 | 38 |
+
+Residual failures include AWS numeric-ID type inference, conditional/nullability
+evaluation and the ContractApplied property-count conflict. These are **reviewed
+non-regression results, not a conformance certificate**. Their counters were
+reproduced from toolkit output, not copied from upstream PRs.
+
+```bash
+# Offline: no official dependency or network required
+python scripts/validate_official_samples.py --check-existing
+python scripts/describe_generated_samples.py
+
+# Live reference comparison (Python 3.12+, isolated validator environment)
+pip install focus-validator==2.2.1
+python scripts/validate_official_samples.py
+
+# A separate candidate for review; this never updates the committed reference
+python scripts/validate_official_samples.py --record /tmp/new-focus-candidate
+```
+
+Promotion requires reviewing the candidate's raw reports, semantic audit and
+failure evidence, then explicitly copying its baseline and logs to the fixture
+directory. CI only compares; it never records new expectations. The public
+`validate --official` command shares the report parser, rejects FAIL even when the
+subprocess returns zero, and applies **none** of the sample-only exceptions.
+Its output is buffered until completion. It requires console output on stdout
+and rejects conflicting output flags with exit code 2. An installed validator
+version different from the tested 2.2.1 produces a warning; an incompatible or
+incomplete console report fails with an explanatory message.
+
+## Statistics and limits
+
+[`generator-statistics.json`](generator-statistics.json) is generated by
+`python scripts/describe_generated_samples.py --write`; the command without
+`--write` verifies it. Amounts and ratios use Decimal arithmetic. Commitment share
+is purchase cost / total billed cost; utilization and waste use used/unused
+effective costs / commitment purchases. Compute coverage is covered list cost /
+all eligible compute-usage list cost, excluding unused records. Contract application
+counts include discount-backed usage/purchases as well as ordinary negotiated usage.
+
+The six default fixtures have commitment share **18.4–30.9%**, utilization
+**69.6–70.5%**, and compute coverage **99.2–99.96%**. The high compute coverage is a
+consequence of fleets mixed with small ordinary usage, not a target or an estimate
+of real customer behavior. The only pedagogical threshold is commitment share
+at least 5% on the default 1,000-row fixtures. Small samples need not contain every
+scenario. Closed synthetic periods reconcile billed/effective totals globally and
+by account, period and currency; this is not imposed on client datasets.
+
+## Migration and verification
+
+Version 0.13.0 intentionally changes generated bytes and identities. CSV columns
+and commands are preserved. Affected Parquet monetary columns use `decimal128(38,17)`;
+unit prices retain their existing scales. The wider fractional scale reduces the
+maximum integer digits to 21 for those columns. Nonrepresentable values fail
+explicitly; no silent rounding is introduced. Pin 0.12.0 to reproduce old bytes.
+
+Verification covers three providers, both source versions, credits on/off, seeds
+0/1/42/default and sizes 1–12, 23/24/25, 100/1000/1001. FOCUS 1.4 conversion,
+invoice aggregates, CSV/Parquet, partial allocations, interleaved generation calls,
+contract references and Linux streaming/partitioned output are exercised. Strict
+conversion of external data does not rewrite costs, taxes or SKUs to imitate the
+synthetic generator.
+
+### Local verification on 2026-09-05
+
+| Check | Result |
+|---|---|
+| Complete default suite, Windows / Python 3.12 | 1,101 passed, 25 platform skips; 89.96% coverage |
+| Complete default suite, Linux / Python 3.12 | 1,126 passed; 90.21% coverage |
+| Targeted generation, golden, report and Parquet suite, Windows / Python 3.11 and 3.13 | 497 passed, 4 platform skips per version |
+| Same targeted suite, Linux / Python 3.11 and 3.13 | 501 passed per version |
+| Linux packaging and release checks | 13 passed, including wheel/sdist build and clean installation |
+| Linux bounded-memory conversion, 100,000 then 300,000 rows | Passed; Python allocation peak stays below 200 MB and grows by less than 1.5x for 3x the rows (tracemalloc excludes native allocations) |
+| Live official reference comparison | All nine complete reports match between Windows and Linux |
+| Lint, types, action pins, model provenance, offline evidence and documented statistics | Passed |
+
+These are local executions performed before PR creation. The repository's CI
+retains its full OS/Python matrix, including the existing macOS smoke run; remote
+results are reported on the PR. Publishing remains a separate release step.

focus_data_toolkit-0.13.0/docs/generator-statistics.json

@@ -0,0 +1,77 @@
+{
+  "aws_1.2": {
+    "billed_cost": "18857.124623679033750",
+    "commitment_billed_cost": "5826.912000",
+    "commitment_share": "0.3090032078741794288132899939",
+    "compute_coverage": "0.9996244701943354539595341662",
+    "contract_applied_rows": 0,
+    "effective_cost": "18857.124623679033750",
+    "rows": 1000,
+    "utilization": "0.7020407406593406593406593407",
+    "waste": "0.2979592593406593406593406593"
+  },
+  "aws_1.3": {
+    "billed_cost": "18554.856257624590813",
+    "commitment_billed_cost": "4610.304000",
+    "commitment_share": "0.2484688609810990373610109813",
+    "commitments": 75,
+    "compute_coverage": "0.9926713446624324633475395163",
+    "contract_applied_rows": 626,
+    "contracts": 28,
+    "effective_cost": "18554.856257624590813",
+    "non_discount_terms": 3,
+    "rows": 1000,
+    "utilization": "0.7018307347222222222222222222",
+    "waste": "0.2981692652777777777777777778"
+  },
+  "azure_1.2": {
+    "billed_cost": "49321.108200244240000",
+    "commitment_billed_cost": "11653.824000",
+    "commitment_share": "0.2362847151099149435004291808",
+    "compute_coverage": "0.9994860707128948614490559558",
+    "contract_applied_rows": 0,
+    "effective_cost": "49321.108200244240000",
+    "rows": 1000,
+    "utilization": "0.7052545978021978021978021978",
+    "waste": "0.2947454021978021978021978022"
+  },
+  "azure_1.3": {
+    "billed_cost": "44516.766247198048000",
+    "commitment_billed_cost": "8196.096000",
+    "commitment_share": "0.1841125645669708672631528611",
+    "commitments": 67,
+    "compute_coverage": "0.9920858412453828140790998259",
+    "contract_applied_rows": 596,
+    "contracts": 25,
+    "effective_cost": "44516.766247198048000",
+    "non_discount_terms": 3,
+    "rows": 1000,
+    "utilization": "0.699760265625",
+    "waste": "0.300239734375"
+  },
+  "gcp_1.2": {
+    "billed_cost": "31758.210633880783500",
+    "commitment_billed_cost": "7418.374000",
+    "commitment_share": "0.2335891680271751825377137440",
+    "compute_coverage": "0.9993595790441745620531488744",
+    "contract_applied_rows": 0,
+    "effective_cost": "31758.210633880783500",
+    "rows": 1000,
+    "utilization": "0.6956325204819277108433734940",
+    "waste": "0.3043674795180722891566265060"
+  },
+  "gcp_1.3": {
+    "billed_cost": "21535.602792935027500",
+    "commitment_billed_cost": "6613.972000",
+    "commitment_share": "0.3071180344285408378488339428",
+    "commitments": 77,
+    "compute_coverage": "0.9935304486425862660442853958",
+    "contract_applied_rows": 640,
+    "contracts": 30,
+    "effective_cost": "21535.602792935027500",
+    "non_discount_terms": 3,
+    "rows": 1000,
+    "utilization": "0.6959978932432432432432432432",
+    "waste": "0.3040021067567567567567567568"
+  }
+}