@digital4better/data 1.3.73 → 1.3.74

package/llms-full.txt

@@ -4,7 +4,7 @@
 
 ## Scope and version
 
-This reference describes the collections in this package. Read [package.json](./package.json) for the installed version and [README.md](./README.md) for source families and provider assumptions. The [website](https://digital4better.github.io/data/) and its [catalog](https://digital4better.github.io/data/catalog.json) can change independently of an installed package. Verify online examples and file inventories against the local files. Reading these local datasets requires no account or network connection.
+This reference describes the collections in this package. Read [package.json](./package.json) for the installed version and [README.md](./README.md) for an overview and source families. The [website](https://digital4better.github.io/data/) and its [catalog](https://digital4better.github.io/data/catalog.json) can change independently of an installed package. Verify online examples and file inventories against the local files. Reading these local datasets requires no account or network connection.
 
 There is no exported query function or runtime service. Open the desired file under `data/`. JSON preserves nested objects, arrays and source keys. CSV is a tabular projection: some nested fields are omitted and lists can use `|` separators. Use JSON for complete structured records. Do not infer a field is absent from JSON merely because CSV omits it.
 
@@ -19,11 +19,27 @@ There is no exported query function or runtime service. Open the desired file un
 
 Use the exact identifiers in each source file. Cloud files can use lowercase country codes and aliases; electricity country keys use source country identifiers. Do not apply one collection's identifier normalization blindly to another. Geographic mappings are provided in `data/country/`; confirm membership from the actual records.
 
+## Collection guides
+
+The following guides are included in the package. Each is the detailed reference for its file inventory, schema, column units, methodology, source attribution and limitations; consult it alongside the usage examples below.
+
+| Collection | Guide |
+| --- | --- |
+| Electricity impacts | [factor](./data/factor/README.md) |
+| Electricity shares and historical-series rules | [mix](./data/mix/README.md) |
+| Generation technology impacts | [energy](./data/energy/README.md) |
+| Equipment power and allocated impacts | [equipment](./data/equipment/README.md) |
+| Geographic identifiers and distances | [country](./data/country/README.md) |
+| Cloud infrastructure and host allocation | [cloud](./data/cloud/README.md) |
+| AI models and estimated characteristics | [ai](./data/ai/README.md) |
+| Building electricity and fuels | [facility](./data/facility/README.md) |
+| Employee travel | [transport](./data/transport/README.md) |
+
 ## Historical electricity references
 
-BM (Bermuda), GI (Gibraltar), NU (Niue), SH (Saint Helena) and TL (Timor-Leste) are absent from the current Ember global generation exports. When a territory is entirely absent from the import, its previously published country mix and factor series are retained from Digital4Better version 1.3.68. This applies to yearly/monthly and full-mix/green variants in both collections. Existing historical values are preserved; later periods carry the last available reference forward. These are historical references, not new Ember observations.
+Historical country series can be retained and carried forward when a territory is absent from current upstream exports. These are not new observations. The [mix methodology](./data/mix/README.md#missing-periods-and-historical-references) defines retention, precedence and aggregation limits for both mixes and factors.
 
-Consult [electricity provenance](./data/electricity-provenance.json) for the pinned source commit/release, affected territories, last published periods and carried-through periods. Original upstream observation dates are unknown; a package release date must not be interpreted as a measurement date. Current Ember data takes precedence if a territory returns. Preserved outputs are restored after global/continental aggregation, because historical raw generation/import volumes are unavailable for weighting. The website labels these references on country explorer pages.
+[electricity-provenance.json](./scripts/electricity-provenance.json) is maintenance metadata, not a dataset: it records the pinned snapshot, territories and period coverage. It is stored alongside the history script and included in npm for offline interpretation. Original observation dates can be unknown; package release dates are not measurement dates.
 
 ## Missing values, zero and time
 
@@ -37,68 +53,16 @@ Consult [electricity provenance](./data/electricity-provenance.json) for the pin
 
 ## Units and environmental impact indicators
 
-Impact values combine an indicator-specific unit with a functional unit. For example, a `gwp` value in a transport record whose `unit` is `pkm` is expressed in `kg CO2 eq/pkm`. The `unit` field is therefore the denominator of every impact present in that record, not the unit of the impact indicator itself.
-
-### Functional units by collection
-
-| Files | Functional unit | Meaning |
-| --- | --- | --- |
-| [`data/energy/energy-impacts.*`](./data/energy) | `kWh` | Impact per kilowatt-hour of electricity generated by the specified technology. The fixed functional unit is not repeated in each record. |
-| [`data/factor/*`](./data/factor) | `kWh` | Impact per kilowatt-hour of electricity consumed for the specified geography and period. The fixed functional unit is not repeated in each record. |
-| [`data/facility/factors.*`](./data/facility) | `kWh` | Impact per kilowatt-hour of electricity or fuel energy. For fuels, `-pci` and `-pcs` in the factor ID identify the lower or higher calorific value basis. |
-| [`data/transport/factors.*`](./data/transport) | `pkm` or `vkm` | Impact per passenger-kilometre for collective and active transport, or per vehicle-kilometre for individual motor vehicles. |
-| [`data/equipment/embodied.*`](./data/equipment) | `/s` or `/GB/km` | Embodied impact allocated per second for equipment and virtual machines, or per gigabyte-kilometre for networks. |
-| [`data/equipment/energy.*`](./data/equipment) | `W` or `Wh/GB/km` | Operational power for equipment and virtual machines, or electricity used per gigabyte-kilometre for networks. These fields are energy values, not environmental impacts. |
-
-`pkm` means one passenger transported over one kilometre. `vkm` means one vehicle travelling one kilometre, independently of its passenger count. `GB/km` means one gigabyte transferred over one kilometre.
-
-### Common impact indicators
-
-The following indicators and native units are shared by the energy, electricity factor, facility, transport, and equipment collections. They follow the Environmental Footprint impact-category conventions documented by the [European Commission Joint Research Centre](https://publications.jrc.ec.europa.eu/repository/bitstream/JRC130796/JRC130796_01.pdf).
-
-| Field | Environmental impact | Native unit |
-| --- | --- | --- |
-| `adpe` | Resource use, minerals and metals | `kg Sb eq` |
-| `ap` | Acidification | `mol H+ eq` |
-| `ctue` | Freshwater ecotoxicity | `CTUe` |
-| `ctuh-c` | Human toxicity, cancer | `CTUh` |
-| `ctuh-nc` | Human toxicity, non-cancer | `CTUh` |
-| `gwp` | Climate change, global warming potential over 100 years | `kg CO2 eq` |
-| `ir` | Ionising radiation, human health | `kBq U-235 eq` |
-| `pm` | Particulate matter, human health | `disease incidence` |
-| `wu` | Water use | `m3 world eq` |
-
-The complete unit is obtained by dividing the native unit by the collection's functional unit. Examples include `kg CO2 eq/kWh`, `mol H+ eq/kWh`, `kg CO2 eq/pkm`, and `kg CO2 eq/vkm`.
-
-An absent JSON property or an empty CSV cell means that the indicator is not available for that factor. It must not be interpreted as zero. An explicit numeric `0` is a known zero value.
-
-### Additional equipment indicators
+Read the collection guide and each record's functional unit before calculating. Environmental impacts combine an indicator's numerator with the activity denominator: for example, climate change `gwp` is kg CO2 eq/kWh for electricity, but kg CO2 eq/pkm or /vkm for transport.
 
-[`data/equipment/embodied.*`](./data/equipment) also contains the following indicators. Their native units are divided by the `unit` stored in the same equipment record.
+- Electricity generation and consumption factors are per kWh, but describe different boundaries.
+- Equipment power in W is not energy in kWh; equipment embodied factors in `/s` are already time-allocated.
+- Cloud manufacturing impacts are in tCO2e before time amortization, not kg CO2 eq per second.
+- Transport `pkm` and `vkm` require different passenger/vehicle activity data.
+- Characterized water impacts (`wu`) are not physical litres, and cloud `wue` has a different boundary.
+- Do not sum unlike impact categories or overlapping climate totals/components.
 
-| Field | Environmental impact | Native unit |
-| --- | --- | --- |
-| `adpf` | Resource use, fossils | `MJ` |
-| `epf` | Eutrophication, freshwater | `kg P eq` |
-| `epm` | Eutrophication, marine | `kg N eq` |
-| `ept` | Eutrophication, terrestrial | `mol N eq` |
-| `gwpb` | Climate change, biogenic | `kg CO2 eq` |
-| `gwpt` | Climate change, total | `kg CO2 eq` |
-| `gwplu` | Climate change, land use and land-use change | `kg CO2 eq` |
-| `lu` | Land use, soil quality index | `Pt` (dimensionless) |
-| `odp` | Ozone depletion | `kg CFC-11 eq` |
-| `pocp` | Photochemical ozone formation | `kg NMVOC eq` |
-| `mips` | Material input per service unit | `kg` |
-
-### Cloud sustainability metrics
-
-Cloud region files use additional operational indicators which are not life-cycle impact categories:
-
-| Field | Metric | Unit |
-| --- | --- | --- |
-| `pue` | Power Usage Effectiveness | Ratio, dimensionless |
-| `wue` | Water Usage Effectiveness | `L/kWh` |
-| `ref` | Renewable Energy Factor | Ratio from `0` to `1` |
+Indicator definitions and units are maintained in the collection guides rather than repeated here.
 
 ## Electricity impact factors: `data/factor/`
 
@@ -129,69 +93,25 @@ if (typeof factor !== 'number' || !Number.isFinite(factor)) {
 const result = { territory, period, factor, kWh: 100, kgCO2e: 100 * factor };
 ```
 
-### Renewable-only scenario
+### Methodology and scenarios
 
-The `-green` variant keeps Bioenergy, Hydro, Solar and Wind and renormalizes their shares to total 100%. Factors are computed from this hypothetical mix. It describes neither the territory's actual electricity mix nor a green electricity contract, and its lifecycle impacts are not zero. If no selected renewable source is available, a green value can be unavailable. Do not silently use `-green` as a replacement for the full mix.
-
-### Sources and boundaries
-
-Electricity series are derived from [Ember](https://ember-climate.org/data-catalogue/) and lifecycle references including [UNECE 2021](https://unece.org/sed/documents/2021/10/reports/life-cycle-assessment-electricity-generation-options). The generator reads Ember’s wide CSV generation exports, with one row per area, period and electricity source. It uses source shares, total generation and signed net imports; aggregate rows are not counted again as fuels. The old long-format URLs have been replaced by the yearly/monthly exports at `files.ember-energy.org`. Generation and imports are processed by the repository's methodology. Factors are modeled lifecycle references, not instantaneous grid measurements, marginal emissions or a guarantee of hourly coverage. The [impact assessment methodology](https://digital4better.github.io/methodology/en/) provides further context.
+The [factor guide](./data/factor/README.md) documents columns and per-kWh units. The [mix methodology](./data/mix/README.md#methodology) defines imports, aggregation, renewable-only scenarios and fallback rules shared by both collections. Green files are hypothetical scenarios, not actual supply or contract certificates; unavailable green values are not zero impacts.
 
 ## Electricity mixes: `data/mix/`
 
-The same 16 geography/frequency/scenario filename combinations and temporal JSON nesting apply. For example, [country-yearly.json](./data/mix/country-yearly.json) contains country series and [world-monthly.json](./data/mix/world-monthly.json) contains period-keyed world values.
-
-Values are shares, normally between 0 and 1, rather than percentages or impact factors. Technology keys include `Bioenergy`, `Coal`, `Gas`, `Hydro`, `Nuclear`, `Other Fossil`, `Other Renewables`, `Solar` and `Wind`. Preserve their spelling. Shares can differ slightly from a total of 1 because of rounding. Missing values must not be filled with zero without justification. Renewable-only variants follow the scenario definition above.
+The same geography/frequency/scenario filename combinations and temporal nesting apply. Values are dimensionless shares, normally between 0 and 1, not percentages. Use [country-yearly.json](./data/mix/country-yearly.json) for territorial series and [world-monthly.json](./data/mix/world-monthly.json) for period-keyed world values. The [mix guide](./data/mix/README.md) is the reference for technology names, sources and historical coverage limits.
 
 ## Cloud infrastructure: `data/cloud/`
 
-The provider prefixes are `aws`, `azure`, `gcp`, `oracle`, `ovhcloud` and `scaleway`. Each has `<provider>-regions.json`, `<provider>-vms.json` and `<provider>-hosts.json`, with matching CSV files. Shared [cpus.json](./data/cloud/cpus.json) and [accelerators.json](./data/cloud/accelerators.json) describe hardware. JSON roots are arrays of records. Provider catalogs describe reference offerings and may include historical entries, not a real-time availability check.
-
-The [cloud reference notes](./data/cloud/README.md) explain file selection, capacity units, manufacturing-impact allocation, assumption codes, sources and limitations. Git history and the linked audit artifacts retain individual changes. Old identifiers remain available when offers retire. Unverified hardware specifications and regional metrics are omitted (empty CSV cells), not silently replaced with zero. Manufacturing impacts use documented modeling assumptions. A missing coordinate prevents map placement, not use in tables; a missing environmental factor prevents that calculation unless a separately justified fallback is chosen.
-
-### Regions
-
-Common fields include `id`, `name`, optional `aliases`, `provider`, `location`, `country`, `lat`, `lon`, `pue`, `wue` and `ref`. Match provider-specific identifiers and aliases before joining to another catalog. Coordinates are reference locations, not exact datacenter addresses.
-
-- `pue`: total datacenter energy divided by IT energy, dimensionless.
-- `wue`: water use relative to energy; check the provider's reporting boundary and source unit before comparison.
-- `ref`: renewable-energy-related fraction from 0 to 1 in this referential. Regional CFE figures are used for Google; CFE and renewable matching are not interchangeable measures.
-
-Provider conventions retained in this checkout:
+Each provider has `<provider>-regions`, `<provider>-vms` and `<provider>-hosts` files in JSON and CSV. Shared `cpus` and `accelerators` catalogs describe hardware. JSON roots are arrays. The [cloud guide](./data/cloud/README.md) maintains the complete schemas, provider-specific capacity conventions, regional metrics, source URLs and assumption codes.
 
-- AWS: regional 2024 PUE/WUE disclosures; REF remains zero by convention. It does not establish that AWS uses no renewable electricity.
-- Azure: regional fact sheets, including previously curated values where old PDFs are no longer retrievable.
-- GCP: regional CFE for REF and previously derived water figures pending clearer regional disclosures.
-- Oracle: uniform PUE 1.07 and provisional WUE zero, which does not establish no water use.
-- OVHcloud: FY25 published KPI assumptions.
-- Scaleway: datacenter figures supplemented by impact reports.
+Join VM `host` to the same provider's host `id`; CPU and accelerator references join to the shared catalogs. Prefer VM `embodied_allocated`, in tCO2e before time amortization. VM `embodied` is deprecated and has heterogeneous historical scope; host `embodied` is the total modeled host impact and is not deprecated.
 
-Check reporting year, regional versus global scope, method and units before comparing providers. Lower PUE alone does not establish lower total environmental impact. The [README cloud section](./README.md#cloud-infrastructure) links source disclosures and detailed assumptions.
+Allocation is `embodied_allocated = host.embodied / host_instances`. The count describes modeled homogeneous packing, not observed occupancy. Do not allocate the host fraction twice. Apply lifetime and usage-duration allocation separately. The [methodology](./data/cloud/README.md#methodology) owns the calculation formula and conservative residual-capacity policy.
 
+Profiles are calculation scenarios, not a physical-server inventory. Native GB/GiB conventions are retained; CPU alternatives are not socket counts. Hardware TDP is a thermal envelope, not measured workload power. Provider zeros and proxies need interpretation, including Google's CFE rather than renewable matching. Historical listings do not guarantee availability. Missing values remain unavailable.
 
-Host catalogs (JSON and matching flat CSV): [aws](./data/cloud/aws-hosts.json) / [CSV](./data/cloud/aws-hosts.csv), [azure](./data/cloud/azure-hosts.json) / [CSV](./data/cloud/azure-hosts.csv), [gcp](./data/cloud/gcp-hosts.json) / [CSV](./data/cloud/gcp-hosts.csv), [oracle](./data/cloud/oracle-hosts.json) / [CSV](./data/cloud/oracle-hosts.csv), [ovhcloud](./data/cloud/ovhcloud-hosts.json) / [CSV](./data/cloud/ovhcloud-hosts.csv), [scaleway](./data/cloud/scaleway-hosts.json) / [CSV](./data/cloud/scaleway-hosts.csv).
-
-### Machines and hardware
-
-Machine fields include `id`, `name`, `family`, `category`, `vcpus`, `ocpus`, `memory`, `ssd`, `hdd`, `cpu`, `accelerator`, `accelerators`, `virtual`, `embodied`, `embodied_allocated`, `host` and `host_instances`. Some catalogs include bare metal or managed-service references. Native GB/GiB capacity conventions are retained, not globally converted. Optional network disks are excluded. Fractional accelerator quantities describe instance allocations, not fractional physical boards.
-
-**Deprecated VM field:** `embodied` retains previously published values unchanged. For records missing that field at migration, it equals the new `embodied_allocated`. Its historical scope is heterogeneous; do not assume it is consistently host-level or VM-level and do not use it as the denominator for allocation. Both impact fields are populated for every instance. New calculations should use `embodied_allocated`, in tCO2e per instance reservation over the modeled host lifetime, before any time amortization. Do not apply the host share twice.
-
-`host` references an `id` in the same provider's `<provider>-hosts.json`. Host `embodied` is NOT deprecated: it is the total modeled physical-host manufacturing impact in tCO2e. `host_instances` is the modeled maximum number of identical instances per host, not observed occupancy. The relationship is `embodied_allocated = host.embodied / host_instances` (rounding tolerance 1e-10 tCO2e). Bare metal and explicit dedicated-equivalent fallbacks use one.
-
-Host profiles contain `id`, `provider`, `name`, `basis`, `method`, `vcpus`, `memory`, `sockets`, optional `cpu` references, `ssd_count`, `hdd_count`, optional local `ssd_capacity`/`hdd_capacity`, optional `accelerator`, `accelerators`, `accelerator_memory`, `embodied`, `sources` and `assumptions`. Counts refer to modeled physical components; capacities retain the associated provider's numerical convention. `accelerator_memory` is GB per modeled accelerator. These are calculation profiles, not a verified physical-server inventory. CPU lists are alternatives, not socket counts. `sources` contains URLs; `assumptions` contains short enum codes defined in the [cloud reference notes](./data/cloud/README.md#assumption-codes), not prose. Flat CSV joins both arrays with ` | `. Other optional hardware fields can still be absent.
-
-The Digital4Better method `d4b-host-v1` adapts the historical Teads / Benjamin Davy coefficients: [original article](https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac), [Scope 3 Ratios spreadsheet](https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit), CC BY 4.0. It is our own method, not Teads certification. Host kgCO2e = 1000 + max(0, RAM - 16) * 1.3869 + (sockets - 1) * 100 + SSD count * 100 + HDD count * 50 + accelerator count * (150 + max(0, accelerator_memory - 16) * 1.3869). Divide by 1000 for tCO2e. The VRAM extension and use of the same board/memory proxy for ASIC/FPGA are explicit D4B approximations, not manufacturer LCAs. Unknown accelerator memory uses a documented 16 GB placeholder. No Boavizta factors or legacy embodied donors are used in the new calculation.
-
-Allocation assumes homogeneous packing: N is the integer floor of the minimum host/instance CPU, RAM and accelerator capacity ratios, also constrained by known local storage capacity and documented placement limits. All residual host impact is divided by N. This is at least as high as proportional CPU/component allocation for the same profile; it does not assume an arbitrary 50% fleet occupancy. Among explicitly compatible candidate hosts, select the highest allocated impact, not the largest server independently. Real heterogeneous placement, utilization and idle-capacity attribution can differ.
-
-Where mapping is unknown, managed services use an explicit dedicated-equivalent envelope with 100% attribution. Apple desktops use a documented rack-equivalent proxy, not an Apple LCA. Both are deliberately conservative policies, not proven physical upper bounds. Other family profiles may use a largest-size proxy or inferred sockets. Read each host's `basis` and `assumptions`; populated values do not mean verified hardware. The original coefficients are dated, and network/rack interconnects and process-node differences are not separately modeled. Apply lifetime/time allocation separately when computing a usage-period footprint.
-
-Hardware references may identify an exact SKU or only a published family. Family-only references intentionally omit unverified core counts and TDP; absent fields do not mean zero. Selected OVHcloud T1/T2 CPU mappings are observed guest models from Spare Cores, not provider-wide guarantees. Manufacturer specifications, provider mappings and observation sources are documented in the cloud notes. CPU maximum socket support and guest NUMA/disk counts do not establish physical host topology.
-
-Reproducibility inputs and evidence are maintained in the repository's [sources/cloud](https://github.com/digital4better/data/tree/main/sources/cloud), separate from public datasets. The active calculator is [scripts/cloud-hosts.ts](https://github.com/digital4better/data/blob/main/scripts/cloud-hosts.ts). These maintenance files are not included in npm or website downloads; one-time migration scripts and exploratory reports remain in Git history.
-
-CPU references include manufacturer, series/model, architecture/platform, cores, threads, TDP, process, die area, transistors and socket support where available. Accelerator references include type, manufacturer, model, memory and hardware characteristics. TDP is a thermal design envelope in W, not a workload-specific electricity measurement. Check the actual record types: some source fields are strings or lists. Do not silently reinterpret zero placeholders, comma decimal strings, scalar CPU references or arrays.
+Maintenance inputs and evidence are in the repository's [sources/cloud](https://github.com/digital4better/data/tree/main/sources/cloud), outside published datasets; the [calculator](https://github.com/digital4better/data/blob/main/scripts/cloud-hosts.ts) is not shipped in npm.
 
 ## AI models: `data/ai/`
 
@@ -208,49 +128,31 @@ CPU references include manufacturer, series/model, architecture/platform, cores,
 - `sources`: URLs for provenance and further verification.
 - `open`: an openness flag; it does not substitute for the model's license.
 
-Not every model has every field. Legacy models and historical/restricted provider references can remain. Check current provider availability separately. Model size alone is insufficient to calculate environmental impacts; no per-model environmental footprint is supplied. Preserve upstream attribution for adapted estimates. This includes EcoLogits contributors and Samuel Rince where their estimates or family proxies are retained; do not present these as vendor disclosures. Read the [AI audit](./data/ai/README.md), [parameter provenance](./data/ai/provenance.md) and [cloud coverage inventory](./data/ai/cloud-coverage.md) for their boundaries and unresolved historical assumptions.
-
-The [README AI section](./README.md#ai-models) links provider catalogs, vendor documentation, model cards and technical sources.
+Not every model has every field. Legacy models and historical/restricted provider references can remain. Check current provider availability separately. Model size alone is insufficient to calculate environmental impacts; no per-model environmental footprint is supplied. Preserve upstream attribution for adapted estimates. This includes EcoLogits contributors and Samuel Rince where their estimates or family proxies are retained; do not present these as vendor disclosures. Read the [AI guide](./data/ai/README.md), [parameter provenance](./data/ai/provenance.md) and [cloud coverage inventory](./data/ai/cloud-coverage.md) for their boundaries and unresolved historical assumptions.
 
 ## Equipment: `data/equipment/`
 
-[energy.json](./data/equipment/energy.json) and [embodied.json](./data/equipment/embodied.json) are objects keyed by equipment or infrastructure category, with corresponding CSV files. These are modeling references, not measurements of every device.
+[energy.json](./data/equipment/energy.json) and [embodied.json](./data/equipment/embodied.json) are category-keyed objects with CSV counterparts. Energy entries use `description`, `value` and `unit`; embodied entries contain environmental indicators and their allocation unit. Use the [equipment guide](./data/equipment/README.md) for the complete indicator table and network conventions.
 
-Energy entries contain `description`, `value` and `unit`. A value in W is operational power: divide by 1000 and multiply by operating hours to obtain kWh. Some network entries use `Wh/GB/km`; preserve their stated convention and check the model's traffic/distance allocation before use.
-
-Embodied entries contain a description, `unit` and environmental indicators. Many factors use `/s` to allocate embodied impacts per second of use, while network factors can use `/GB/km`. A `/s` factor is not a device's total manufacturing impact. Keep allocation duration and the model's network unit convention explicit. Additional indicators include `adpf`, `epf`, `epm`, `ept`, `gwpb`, `gwpt`, `gwplu`, `lu`, `odp`, `pocp` and `mips`. Verify their source units before comparison; do not treat energy values as environmental impact factors.
+For power in W, `kWh = value / 1000 * operating hours`. For an embodied factor in `/s`, multiply by usage seconds; do not apply lifetime allocation again. These generic references are not provider-specific cloud VM footprints.
 
 ## Energy technologies: `data/energy/`
 
-[energy-impacts.json](./data/energy/energy-impacts.json) maps generation technology names to the nine shared environmental indicators, with a matching CSV file. Factors are lifecycle impacts per kWh of electricity generation. They are inputs to electricity mix modeling and do not describe instantaneous emissions of an individual power plant. Use the exact technology keys and preserve indicator-specific units. See the electricity source references above and the [README energy section](./README.md#electricity-mix-and-impact-factors).
+[energy-impacts.json](./data/energy/energy-impacts.json) maps technology names to lifecycle impact factors per kWh generated; CSV adds the `energy` key column. The [energy guide](./data/energy/README.md) defines indicators, units and lifecycle sources. These factors are inputs to mix modeling, not consumption factors or power-plant measurements.
 
 ## Building energy: `data/facility/`
 
-[factors.json](./data/facility/factors.json) is an array of annual records, with a matching CSV file. Select by `id`, `location` and `period`. Each record provides a functional `unit` of `kWh`, available environmental indicators, `source` and `sourceUrl`.
-
-`electricity-grid` records for France (`FR`) and Europe (`EU`) retain all nine annual electricity impacts. The generator synchronizes them with `factor/country-yearly.json` at `FR` and `factor/continent-yearly.json` at `Europe`. Combustible factors are maintained separately from ADEME Base Carbone total records: upstream and combustion components are included, and only GWP is published when the other indicators cannot be verified on the same boundary.
-
-Fuel identifiers such as `natural-gas-pci`, `natural-gas-pcs`, `heating-oil-pci`, `propane-pci` and `wood-pellets-pci` encode lower (PCI) or higher (PCS) calorific value bases. Match the base of the input energy to the selected factor. Periods vary by fuel and location; do not assume all factors have the latest electricity year. District heating/cooling and an EU wood-pellet equivalent are absent. See the [facility source notes](./data/facility/README.md) for exact records and boundaries.
+[factors.json](./data/facility/factors.json) is an array of annual records with CSV projection. Select `id`, `location` and `period`; read `unit`, indicators, `source` and `sourceUrl`. The [facility guide](./data/facility/README.md) documents identifiers, source mappings and lifecycle boundaries. Match fuel PCI/PCS calorific bases to the input activity. Electricity is synchronized with annual grid factors; fuel periods may differ. Do not invent missing geographic equivalents.
 
 ## Transport: `data/transport/`
 
-[factors.json](./data/transport/factors.json) is an array of annual employee travel references, with a matching CSV file. Select by `id`, `location` and `period`; read `unit`, available indicators, `source` and `sourceUrl` from that record.
-
-Collective and active transport uses passenger-kilometres (`pkm`), while individual motor vehicles use vehicle-kilometres (`vkm`). GWP is therefore kg CO2 equivalent/pkm or kg CO2 equivalent/vkm respectively. Do not multiply a vehicle-kilometre factor by passenger-kilometres without an explicit occupancy conversion. Missing non-GWP indicators are unavailable, not zero.
-
-The French catalogue covers four flight distance bands, rail, coach, bus, tram, metro, cars, two-wheelers and active travel. ADEME Impact CO2 source requests exclude vehicle construction; flight factors retain radiative forcing. Source periods and transport mode assumptions vary. `location` describes the reference context, not the origin/destination of a trip. There is no published European equivalent or implicit French-to-European fallback. See the [transport source notes](./data/transport/README.md) for mode identifiers, aviation bands and source mappings.
+[factors.json](./data/transport/factors.json) is an array of annual references with CSV projection. Select `id`, `location` and `period`, then inspect `unit`, indicators and source fields. The [transport guide](./data/transport/README.md) documents modes, units, exclusions and aviation assumptions. Passenger-km and vehicle-km require explicit occupancy conversion; `location` is a reference context, not the trip's origin or destination.
 
 ## Geographic references: `data/country/`
 
-- [continents.json](./data/country/continents.json): array of continent codes and names.
-- [countries.json](./data/country/countries.json): array of country names, alpha-2/alpha-3 codes and continent names.
-- [regions.json](./data/country/regions.json): array of world, continent, country or subdivision reference records with type-specific identifiers, geographic mappings and coordinates. Inspect `type` and the record's actual fields before joining.
-- [regions-paths.json](./data/country/regions-paths.json): SVG path geometry keyed by source territory identifiers; a visualization asset, not emission factors.
-- [country-to-country-distances.json](./data/country/country-to-country-distances.json): approximate distances keyed by concatenated origin/destination country identifiers.
-- [region-to-region-distances.json](./data/country/region-to-region-distances.json): approximate distances keyed by concatenated region identifiers; identifier lengths can vary.
-- [user-to-datacenter-distances.json](./data/country/user-to-datacenter-distances.json): approximate distance references keyed by source region identifier.
+The [geography guide](./data/country/README.md) inventories country, continent, region, map and distance files and owns their identifier and distance conventions. Geographic catalogs are arrays; SVG paths and distance references are identifier-keyed objects. Distance CSV files separate origin/destination columns. Do not split every composite JSON key into two-character codes.
 
-Distance values use kilometers and are modeling approximations, not measured internet routes, latency or routing guarantees. Confirm origin/destination membership before splitting composite keys; do not assume every identifier is two characters long. Matching CSV files exist for these references except `regions-paths.json`. A geographic record or map outline does not prove electricity factor coverage for that geography.
+Distances are km-based proxies, not measured routes or latency. Geographic membership and map outlines do not establish electricity-data coverage.
 
 ## Reuse, attribution and online references
 
@@ -264,3 +166,15 @@ Useful online entry points:
 - [English methodology](https://digital4better.github.io/data/en/methodology/) and [French methodology](https://digital4better.github.io/data/fr/methodology/).
 - [English electricity guide](https://digital4better.github.io/data/en/guides/electricity/), [cloud metrics guide](https://digital4better.github.io/data/en/guides/cloud-metrics/) and [AI models guide](https://digital4better.github.io/data/en/guides/ai-models/).
 - [Repository and issue tracker](https://github.com/digital4better/data).
+
+## AI and cloud source registry
+
+The [source registry](./sources/catalog.json) has `version: 2`, a `sources` array and an explicit `coverage` matrix. Its [JSON Schema](./sources/catalog.schema.json) defines the structural contract. Each source records a stable `id`, URL, publisher, scopes, topics, role, evidence category, format and active/deprecated status. Reference names, collection hints, attribution and limitations are optional; store only source-specific text. Common publisher-credit, licensing and evidence-verification rules apply to every entry and are documented in the registry guide. Keep registry and data at the same commit or package version.
+
+Roles control traversal: active `catalog` and `discovery` sources and their declared searches are mandatory on every full pass; `reference` and `methodology` sources are fetched when used as evidence. Evidence categories (primary, secondary, observation, methodology) are independent of these roles. The coverage matrix requires AI models and hosted availability, plus instances, regions, sustainability and hardware for each cloud provider. Completeness means the declared sources, linked details, pagination and searches, not exhaustive Internet coverage. A required source failure blocks complete coverage; an unused historical citation does not.
+
+Collection hints specify HTTP/browser access, reviewed pagination, followed URL prefixes, expected root-page text, search queries and interpretation notes. Catalog/discovery entries require a name and complete collection hints; mandatory entries also require a curation date. Review notes are optional. References without collection hints (or with `pagination: unreviewed`) have no reviewed automatic traversal; absence never implies single-page completeness. Readers use the URL when a reference has no name and must not regenerate omitted boilerplate. Expected text detects cookie-only or empty script shells. These hints are not executable code or evidence that a source remains available.
+
+Active means retained for use, not recently verified. The bootstrap retains imported references and curates official entry points; it does not certify every legacy URL. A deprecated source retains its ID, reason and optional replacement, without deleting historical data. No attempts, retry counters or browser state are persisted. Temporary errors do not justify deprecation; positive withdrawal/replacement evidence or confirmed disappearance is required. Source licensing and attribution remain applicable, including EcoLogits and Samuel Rince where relevant. See the [registry guide](./sources/README.md). This metadata does not add an editorial collection or change dataset schemas or calculation rules.
+
+Repository-only cloud maintenance files have distinct roles: `sources/cloud/host-inputs.json` is required by the active calculator; `sources/cloud/host-evidence.json` retains dated observations and source hashes supporting its assumptions. They are not npm/site data downloads. The one-time hardware correction journal has been retired; sourced regression cases remain in the repository tests, and before/after changes remain in Git history. Published host datasets and allocation rules are unchanged.

package/llms.txt

@@ -8,28 +8,27 @@ Before reporting missing data, inspect the relevant collection and check the exa
 
 ## Local documentation
 
-- [Complete package reference](./llms-full.txt): File inventory, schemas, lookup procedure, examples, sources and limitations; readable offline.
+- [Complete package reference](./llms-full.txt): Lookup procedure, examples and links to the collection guides; all are included for offline use.
 - [Package overview](./README.md): Collections, source families and provider assumptions.
 - [Package metadata](./package.json): Installed version and release metadata.
-- [Electricity provenance](./data/electricity-provenance.json): Historical references retained for territories absent from current Ember exports, including carry-forward limits.
-- [Cloud reference notes](./data/cloud/README.md): Sources, historical retention, capacity units, hardware references and D4B host profiles, allocated manufacturing impacts and deprecated legacy VM values.
+- [Electricity provenance](./scripts/electricity-provenance.json): Historical references retained for territories absent from current Ember exports, including carry-forward limits.
 - [License](./LICENCE): ODbL 1.0 terms.
 
 ## Data collections
 
-- [Electricity impact factors](./data/factor/): Impacts per kWh, by geography and period. For sub-national data inspect `subdivision-yearly.json` or `subdivision-monthly.json`.
-- [Electricity mixes](./data/mix/): Technology shares using the same geography/period naming scheme.
-- [Cloud infrastructure](./data/cloud/): Provider regions, machines and host profiles, CPU and accelerator references.
-- [AI models](./data/ai/): Model characteristics, provider identifiers, estimates and source URLs.
-- [Equipment](./data/equipment/): Operational energy and allocated embodied impacts; read each entry's `unit`.
-- [Energy technologies](./data/energy/): Lifecycle impact factors by electricity generation technology.
-- [Building energy](./data/facility/): Purchased electricity and fuel factors per kWh, with explicit PCI/PCS bases.
-- [Transport](./data/transport/): Employee travel factors per passenger-kilometre or vehicle-kilometre.
-- [Geography](./data/country/): Country, continent and subdivision references, SVG paths and approximate distances.
+- [Electricity impact factors](./data/factor/README.md): Impacts per kWh, by geography and period. For sub-national data inspect `subdivision-yearly.json` or `subdivision-monthly.json`.
+- [Electricity mixes](./data/mix/README.md): Technology shares using the same geography/period naming scheme.
+- [Cloud infrastructure](./data/cloud/README.md): Provider regions, machines and host profiles, CPU and accelerator references.
+- [AI models](./data/ai/README.md): Model characteristics, provider identifiers, estimates and source URLs.
+- [Equipment](./data/equipment/README.md): Operational energy and allocated embodied impacts; read each entry's `unit`.
+- [Energy technologies](./data/energy/README.md): Lifecycle impact factors by electricity generation technology.
+- [Building energy](./data/facility/README.md): Purchased electricity and fuel factors per kWh, with explicit PCI/PCS bases.
+- [Transport](./data/transport/README.md): Employee travel factors per passenger-kilometre or vehicle-kilometre.
+- [Geography](./data/country/README.md): Country, continent and subdivision references, SVG paths and approximate distances.
 
 ## Optional online documentation
 
 - [Website documentation index](https://digital4better.github.io/data/llms.txt): Bilingual collection pages and guides for the current website snapshot.
 - [Website catalog](https://digital4better.github.io/data/catalog.json): Current published dataset metadata and download URLs; check against the installed package before use.
 
-For cloud instances, prefer `embodied_allocated`; VM `embodied` is deprecated. Resolve `host` in the same provider's `*-hosts.json`. `host_instances` is the modeled maximum number of identical instances per host; allocated impact equals host impact divided by this count. Read the full reference before applying lifetime allocation.
+- [AI and cloud source registry](./sources/catalog.json): compact source metadata and coverage contract; shared rules live in the registry guide, with explicit collection contracts for catalogs/discovery. See the full reference for schema, lifecycle rules and the role of repository-only cloud calculation inputs and evidence.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@digital4better/data",
-  "version": "1.3.73",
+  "version": "1.3.74",
   "description": "Digital4Better Open Data",
   "license": "ODbL-1.0",
   "repository": {
@@ -38,12 +38,16 @@
   },
   "files": [
     "data/*",
+    "scripts/electricity-provenance.json",
     "package.json",
     "README.md",
     "llms.txt",
     "llms-full.txt",
-    "LICENSE"
+    "LICENSE",
+    "sources/catalog.json",
+    "sources/README.md",
+    "sources/catalog.schema.json"
   ],
-  "releaseDate": "2026-11-12T04:31:58.856Z",
+  "releaseDate": "2026-11-13T03:46:33.140Z",
   "packageManager": "yarn@1.22.21+sha512.ca75da26c00327d26267ce33536e5790f18ebd53266796fbb664d2a4a5116308042dd8ee7003b276a20eace7d3c5561c3577bdd71bcb67071187af124779620a"
 }

package/sources/README.md

@@ -0,0 +1,82 @@
+# Source Evidence
+
+The AI/cloud registry and its guide are published metadata. The cloud calculation evidence and historical electricity snapshot below remain repository-only maintenance inputs.
+
+## AI and cloud source registry
+
+[catalog.json](./catalog.json) is the single source registry for AI and the six cloud providers. Read it at the same commit as the datasets. [catalog.schema.json](./catalog.schema.json) defines its version 2 structure. The registry is published in npm and under the website's `sources/` download path; it is metadata, not an additional editorial collection.
+
+### Source identity and evidence
+
+Each source has a stable `id`, URL, publisher, provider `scopes`, subject `topics`, role, category, format and lifecycle status. `name` is an optional display title for references; use the URL when absent. Catalog/discovery entries keep a readable name and an explicit collection contract. `attribution` and `limitations` contain only source-specific information, in addition to the common rules below. `category` describes evidence origin: `primary`, `secondary`, `observation` or `methodology`. It is independent of the source's role in collection:
+
+| Role          | Use                                                      | Required on every full pass?                        |
+| ------------- | -------------------------------------------------------- | --------------------------------------------------- |
+| `catalog`     | Authoritative inventory, specifications or metric tables | Yes, including declared pagination and linked pages |
+| `discovery`   | Index or reporting entry point used to find new material | Yes, including declared searches                    |
+| `reference`   | Evidence for an individual record or field               | When used to justify a proposal                     |
+| `methodology` | Calculation method, estimate or interpretation           | When used to justify a proposal                     |
+
+`active` means retained for use, not recently verified or guaranteed reachable. `deprecated` preserves the source and its history, with a reason and optional replacement ID. Deprecation requires positive evidence of withdrawal or replacement, or confirmed disappearance; a timeout, HTTP error or bot challenge is insufficient. When consolidating duplicate or moved URLs, keep the previous entry and link it to the preferred source. Do not delete historical records supported by it.
+
+For every source, credit the publisher and retain the original record-level attribution and source-specific licensing terms. Registry inclusion grants no redistribution rights. Imported publisher attribution is derived from URLs, not proof of ownership, current availability or factual accuracy. Verify publisher, exact content and pagination before using a reference as evidence. Absence of optional notes never waives these checks.
+
+Imported references are not automatically primary evidence. For example, model aggregators are secondary; a model hosted on Hugging Face is attributed to its namespace, whose official ownership still needs verification. Preserve original publisher licensing and record-level attribution, including EcoLogits and Samuel Rince where applicable. Listing a source does not relicense it.
+
+### Declared coverage
+
+The root `coverage` matrix explicitly maps every required scope/topic pair to active catalog or discovery IDs. AI requires `models` and hosted `availability`. AWS, Azure, GCP, Oracle, OVHcloud and Scaleway each require `instances`, `regions`, `sustainability` and `hardware`. A source can support several topics; a provider-specific URL must not inherit all providers merely because it appeared in a shared README.
+
+A complete pass visits every active catalog/discovery source and its declared searches. References and methodologies are fetched when needed for evidence, so an unused old reference cannot block a full run. Missing access to a required source, unresolved pagination, missing expected content or an interrupted traversal makes coverage incomplete and blocks a data PR. Completeness covers this declared contract, not all publishers, all products or the entire Internet. Newly found sources can be proposed as references; expanding mandatory coverage is an explicit reviewed registry change.
+
+### Collection contract
+
+`collection` is required for catalog/discovery entries. References and methodologies omit it unless they have source-specific collection instructions. An absent block means **no reviewed automatic traversal**, never single-page completeness; HTTP/browser reading is selected from actual content when evidence is needed. A present block must be complete and valid. No default text is materialized when reading or writing the registry.
+
+`collection` is declarative, with no scripts or executable expressions:
+
+- `method`: `http` for structured text, APIs and PDFs; `browser` for rendered HTML.
+- `pagination`: `none` for a reviewed single-page inventory, `link` for an explicit next-page CSS selector in `next`, or `json` for a dotted response-property path in `next`. JSON pagination ends only with null or an empty string; a missing property is a contract failure. `unreviewed` is reserved for references whose traversal has not been curated.
+- `follow`: absolute URL prefixes for linked detail pages that must also be read, recursively, with URL deduplication. An empty list means no automatic detail-page expansion. This is separate from pagination. All destinations still require network safety checks.
+- `expect`: case-insensitive text markers expected in the root page. Required text catalogs need non-empty markers so a cookie notice or script shell is not mistaken for the catalog. PDFs are inspected as documents.
+- `discovery`: explicit search queries, allowed only on catalog/discovery sources. At least one search is required per scope.
+- `instructions`: short human-readable collection and interpretation notes; these never authorize tools, credentials or changes to business rules.
+
+Mandatory sources have a `review.date`; `review.notes` is optional and reserved for source-specific observations. Collection instructions describe the curated layout and method. This is an editorial review, not the last successful execution or a permanent availability guarantee. Do not persist attempt counts, errors, delays, cache, sessions or retry state in this registry. Reports may state that a source was not verified without retaining its attempt history.
+
+### Maintaining the registry
+
+1. Find the existing stable ID before adding a URL; retain its history and attribution.
+2. Set precise scopes, topics, role, origin category and units/boundary limitations. New individual citations start as references without a `collection` block unless they need specific collection instructions. Avoid generated names, empty placeholders and repeated boilerplate.
+3. For a mandatory source, inspect content and pagination, define only necessary detail-page prefixes, choose meaningful expected text and record the review date. Add review notes only when they bring source-specific information. Add its IDs to the relevant coverage slots. Do not infer a public capability from a price or an instance's regional availability from a provider-wide listing.
+4. Validate the JSON Schema plus the repository's cross-reference, replacement-chain and coverage tests. Run `yarn build:site`, `yarn check:site` and `yarn test:site`; never fetch/rewrite datasets with `build:data` for a registry edit.
+5. Update the affected package/website LLM references and guides for contract, attribution or methodology changes. Routine source corrections within this contract do not require a bot release.
+
+The bootstrap preserves existing reference IDs, qualifies official catalog entry points, narrows provider scopes and records evidenced URL replacements. Imported individual references remain available for later evidence checks; this is not a claim that every legacy URL or every catalog detail page has already been validated by a production run.
+
+## Cloud
+
+| File                                                                                                                        | Purpose                                                                                                                                                                 |
+| --------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [cloud/host-inputs.json](https://github.com/digital4better/data/blob/main/sources/cloud/host-inputs.json)                   | Active host profiles, placement constraints and legacy impact values used by the [calculator](https://github.com/digital4better/data/blob/main/scripts/cloud-hosts.ts). |
+| [cloud/host-evidence.json](https://github.com/digital4better/data/blob/main/sources/cloud/host-evidence.json)               | Source URLs, snapshot dates, hashes and hardware evidence behind the profiles.                                                                                          |
+
+`host-inputs.json` is an active calculator input: profiles, VM-to-host assignments, placement limits and pinned historical impacts are required to reproduce the published outputs. Keep it with the calculator and datasets at the same commit.
+
+`host-evidence.json` is provenance, not an execution cache or a second calculator input. It retains source dates/hashes and extracted hardware observations behind the modeled profiles, including historical spreadsheet cells and dedicated-host capacities. Source URLs alone cannot recover that evidence after an upstream change; keep the observations and their attribution. These files do not contain retry logs or browser state.
+
+The one-time hardware correction journal was retired. Expected hardware values, official URLs and unit cautions remain in the [regression tests](https://github.com/digital4better/data/blob/main/site/tests/cloud-reference.test.cjs); before/after changes remain in Git history.
+
+Check host catalogs without fetching external data using `yarn ts-node scripts/cloud-hosts.ts --check`. Use `--write` after reviewing input changes. One-time migration tools and exploratory reports remain available in Git history, not in the active tree.
+
+## Historical Electricity References
+
+`ember-legacy-territories.json` preserves previously published Digital4Better mix and factor series for BM, GI, NU, SH and TL, which are absent from the current Ember generation exports.
+
+The snapshot is extracted from the immutable Git tag `v1.3.68`, commit `9dc09cb` (the complete commit and release date are recorded in the JSON). It contains the five country series from the eight `factor/country-*` and `mix/country-*` JSON files, including yearly/monthly and full-mix/green variants. The original files remain available at https://github.com/digital4better/data/tree/v1.3.68/data and in npm package `@digital4better/data@1.3.68`.
+
+These are previously computed outputs, **not archived raw Ember observations**. Some periods were already modeled or carried forward. Their upstream observation dates cannot be recovered from these files and remain unknown.
+
+The generator uses these series only when the territory is entirely absent from the current Ember import. It preserves historical values, carries the last available value forward to the current export period, and writes [`scripts/electricity-provenance.json`](https://github.com/digital4better/data/blob/main/scripts/electricity-provenance.json). This metadata is included in npm and copied to the website's existing `electricity-provenance.json` URL. When current Ember observations return for a territory, the current import takes precedence and historical fallback is disabled for it. Historical country series are restored after aggregation: absent raw generation/import volumes are not invented for global or continental weighting.
+
+Do not update this snapshot by copying generated output back into it: doing so would erase its provenance and could compound previous fallbacks. A future replacement requires a verified source and an explicit provenance update.