@jadenrazo/cloudcost-mcp 1.2.1 → 1.2.2

package/CHANGELOG.md

@@ -6,6 +6,142 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
 
 ## [Unreleased]
 
+## [1.2.2] - 2026-10-07
+
+### Security
+
+- Patch the MCP SDK OAuth issuer advisory and transitive proxy-addr,
+  fast-uri and ip-address advisories using compatible dependency updates.
+  Also patch development-only brace-expansion and source-map-js advisories.
+  Both production and full dependency audits pass at release verification.
+
+### Fixed
+
+- Select refresh liveness from the current workflow's paginated REST history
+  by branch and creation time, with diagnostics. A hosted health query had
+  selected an 85-day-old failure despite newer successful refreshes.
+- Restore completed provider verification through the established refresh
+  pipeline while preserving the 21-day freshness gate. Record source vintage,
+  response SHA-256, retrieval time and completed verification separately.
+- Refuse to certify interrupted AWS streams, invalid or expired GCP snapshots,
+  insufficient source coverage and failed generated fallback validation.
+- Recover a scheduled refresh that never acquired a hosted runner with at most
+  one narrowly scoped retry. Retain acquisition diagnostics, preserve branch
+  protections, and keep Actions write isolated from provider retrieval.
+- Scope refresh liveness to the branch being checked so another branch's preview
+  cannot certify main. Add preview and recovery probes that do not update pricing PRs
+  or issues. The GitHub cost-estimate Action now installs this release.
+
+- **Live AWS EC2 pricing selected order-dependent duplicate rows.** The bulk
+  catalog contains several Shared/Used/Linux rows for some instance types,
+  including licensed variants. The loader overwrote one cache key for every
+  match, so the last row won: `t3.micro`, `m6i.large`, and `t3.medium` were
+  overstated by 2.6x to 7.5x. It now keeps the lowest qualifying on-demand row,
+  applies identical filters to the final unterminated CSV line, and has a
+  duplicate-row regression test. See [the incident review](./docs/incidents/2026-08-pricing-drift.md).
+
+- **Azure Linux VM selection could fall through to a Windows price.** Azure's
+  current `skuName` uses `Standard_D2s_v5`, while the selector expected a
+  space-separated shape. When exact matching failed, sorted fallback order
+  selected the $0.188/hr Windows row instead of the $0.096/hr Linux row. SKU
+  comparison is now punctuation-insensitive, prefers canonical `armSkuName`,
+  and preserves the requested OS.
+
+- **Pricing drift coverage existed but never ran in Health.** The daily workflow
+  now runs the full golden-range suite, catalog misses fail instead of reporting
+  a green skip, and the reviewed `p4d.24xlarge` range reflects the live
+  $21.9576/hr rate observed on 2026-08-30.
+
+- **GCP pricing had not refreshed since 2026-04-15.** `scripts/refresh-pricing.ts`
+  read `gstatic.com/cloud-site-ux/pricing/data/gcp-compute.json`, an undocumented
+  Google asset that has returned 404 since before the first scheduled run on
+  2026-04-20. All 16 scheduled refreshes failed; the GCP fetcher never once
+  succeeded. Bundled GCP data is now rebuilt weekly from the
+  [gcosts](https://github.com/Cyclenerd/google-cloud-pricing-cost-calculator)
+  snapshot of the Cloud Billing Catalog (Apache-2.0), which also extends
+  coverage from 17 regions to 37.
+
+- **GCP compute prices were reconstructed, and the reconstruction was wrong for
+  newer families.** The old refresh derived every instance price as
+  `vcpus * coreRate + memGb * memRate`. That collapsed c3 and c4 to identical
+  prices in all 102 region/shape pairs, which they are not. The new source ships
+  assembled per-instance rates, so the derivation is gone. 944 of 1,197 existing
+  values are unchanged; the 253 that moved are corrections in the c2d, c3, c4 and
+  n4 families. `c2d-standard-4` and `c2d-standard-8` shipped in the data file but
+  were absent from the refresh allow-list, so nothing had ever refreshed them.
+
+- **Cloud Storage regional prices were multiplier-derived, not real.**
+  `asia-southeast1` was recorded as pricier than `us-central1`; both are in the
+  same $0.020 Standard tier. Two tests asserted the incorrect ordering and now
+  assert against a region that genuinely differs.
+
+- **The live GCP path could not work, and would have been harmful if it had.**
+  `CloudBillingClient` queried `cloudbilling.googleapis.com`, which answers
+  unregistered callers with `403 PERMISSION_DENIED`, so every uncached GCP lookup
+  paid a guaranteed-failing round trip before serving bundled data. Worse, its
+  compute path returned the per-vCPU "Instance Core" SKU as the whole-instance
+  price: an `e2-standard-2` would have priced at roughly $0.021/hr against a true
+  $0.067/hr, and live was preferred over bundled. The 403 was the only thing
+  preventing a silent ~3x under-estimate on every machine type. The client is
+  removed and `GcpProvider` is bundled-only.
+
+- **The GCP smoke test was built to skip on the only failure it could see.**
+  `fetchComputeSkus` returns `null` on any error, and the test read `null` as an
+  "upstream catalog miss" and called `ctx.skip()`. A hard 403 was reported as a
+  green skip, which is why the daily health check scored "live provider APIs: ok"
+  throughout the outage. It now probes the source the refresh actually depends
+  on, imported rather than retyped, and a non-2xx fails.
+
+- **The health check asked whether the refresh ran, not whether it worked.**
+  `health.yml` requested the last run's `conclusion` and used only `createdAt`,
+  so sixteen consecutive failures scored a healthy `LOOP=0`. It now evaluates the
+  conclusion, and an unparseable timestamp reports as unknown instead of being
+  mislabelled as "has not run recently".
+
+- **AWS and Azure could have rotted the same way, silently.** Their fetch helpers
+  return empty maps on failure, every SKU falls through to "no live data", and
+  the metadata was stamped with today's date regardless - no failed run, no issue.
+  Both now refuse to stamp when live coverage falls below 80% of known SKUs. GCP
+  was only ever loud because its fetcher happened to return before the stamp.
+
+- **The weekly refresh PR deadlocked every Monday.** `create-pull-request` was
+  passed no `token:`, so it force-pushed as `github-actions[bot]`; under this
+  repo's `first_time_contributors` approval policy those runs park at
+  `action_required` with zero jobs allocated. PR #38 sat that way for six days.
+  The workflow now mints a scoped GitHub App token when one is configured, and
+  falls back to the previous behaviour when it is not.
+
+### Changed
+
+- **`metadata.json` carries two dates and a policy.** `last_updated` is the
+  vintage of the numbers, `last_verified` is when the refresh loop last completed,
+  and `refresh_policy` is `automated` or `curated`. One date was doing the work of
+  two: for GCP it was real vintage, for AWS/Azure an unconditional heartbeat, and
+  one 21-day threshold was applied to both. `curated_datasets` now names the
+  tables no automation refreshes (GCP Cloud SQL, AWS EBS, Azure disk/database),
+  which the automated stamp used to silently re-certify as fresh.
+  `pricing_metadata` reports the older of the two dates, never the flattering one.
+
+- **`refresh_policy` is finally written.** `check-freshness.ts` has read the field
+  since it was created but nothing set it, so every row printed
+  `refresh=unspecified` and the gate could not tell a dead loop from overdue
+  curation.
+
+- **CI.** `fail-fast: false` on the test matrix (a formatting nit in one file
+  cancelled the Node 20 leg and destroyed its signal); Node 24.15.0 added to the
+  matrix, since that is the version that actually ships; `npm run format:check`
+  instead of a direct prettier call, so CI and the documented local gate cannot
+  drift; `build-artifact` restricted to `main`, where it is not duplicating a
+  build the matrix already ran three times.
+
+- **`cost-estimate-example.yml` moved to `examples/`.** Its first line said it was
+  a file to copy; GitHub registered and ran it on every pull request anyway. The
+  composite action's self-install pin moved from 1.0.1 to 1.2.1.
+
+- **README.** Corrected the claims that GCP pricing is fetched live from the Cloud
+  Billing Catalog API using "unauthenticated public endpoints", and that bundled
+  data is refreshed weekly - true for AWS and Azure, false for GCP for 128 days.
+
 ## [1.2.1] - 2026-08-07
 
 ### Fixed
@@ -83,11 +219,13 @@ Hardened the MCP tool surface against the attack classes catalogued in the OWASP
 First stable release. No breaking API changes from 0.5 — this version ratifies the existing surface as SemVer-locked. See [`VERSIONING.md`](./VERSIONING.md#migration-notes) for details.
 
 ### Added
+
 - **`VERSIONING.md`**: Formal stability contract defining the SemVer-locked public surface (12 MCP tools, CLI binaries, package entry points), the change-classification policy, and the 0.x → 1.0 migration notes. (At release this lived in two files, `STABILITY.md` and `MIGRATION.md`; they were later consolidated.)
 - **Smoke integration tests**: Live-API smoke coverage for AWS Bulk Pricing, Azure Retail Prices, and GCP Cloud Billing Catalog, gated behind `RUN_INTEGRATION=1`. New `integration-smoke` CI job runs on manual dispatch and weekly schedule (Mondays 12:00 UTC).
 - **Publish workflow gates**: `npm audit --audit-level=high` and `npm test` now run before `npm publish`, preventing broken or vulnerable releases.
 
 ### Security
+
 - Resolved transitive advisories via npm `overrides`:
   - `hono` → `^4.12.12` (GHSA-26pp-8wgv-hjvm, GHSA-r5rp-j6wh-rvv4, GHSA-xf4j-xp2r-rqqx, GHSA-wmmm-f939-6g9c, GHSA-xpcf-pg52-r92g)
   - `@hono/node-server` → `^1.19.13` (GHSA-92pp-h63x-v22m)
@@ -96,11 +234,13 @@ First stable release. No breaking API changes from 0.5 — this version ratifies
 - `npm audit --audit-level=high` now reports zero vulnerabilities.
 
 ### Packaging
+
 - `VERSIONING.md` and `CHANGELOG.md` are now included in the published npm tarball. (Originally shipped as `STABILITY.md` + `MIGRATION.md`, now merged.)
 
 ## [0.4.0] - 2026-03-28
 
 ### Added
+
 - **Multi-IaC support**: CloudFormation (JSON/YAML), Pulumi (stack export), and Bicep/ARM template parsing via unified `IaCParser` interface with auto-format detection
 - **`analyze_plan` tool**: Parse `terraform plan -json` output for precise before/after cost-of-change analysis
 - **`compare_actual` tool**: Parse `.tfstate` files to compare actual infrastructure costs against estimates
@@ -120,6 +260,7 @@ First stable release. No breaking API changes from 0.5 — this version ratifies
 - **`docs/architecture.md`**: Layered architecture documentation with extension guides (originally at repo root, moved to `docs/` in a later cleanup).
 
 ### Changed
+
 - Refactored `bulk-loader.ts` (929 -> 708 lines) into focused modules: csv-parser, fallback-data
 - Refactored `resource-extractor.ts` (778 -> 299 lines) into per-provider extractors
 - Refactored `retail-client.ts` (614 -> 499 lines) with extracted fallback-data
@@ -127,10 +268,12 @@ First stable release. No breaking API changes from 0.5 — this version ratifies
 - Updated CI pipeline with security audit job and format checking
 
 ### Fixed
+
 - picomatch HIGH severity vulnerability (ReDoS + method injection)
 - Unused imports and variables across codebase (ESLint cleanup)
 
 ### Security
+
 - Resolved picomatch 4.0.0-4.0.3 vulnerability via npm audit fix
 - Added `npm audit --audit-level=high` to CI pipeline
 

package/README.md

@@ -1,7 +1,7 @@
 <h1 align="center">CloudCost MCP Server</h1>
 
 <p align="center">
-  Multi-cloud cost analysis for Terraform, CloudFormation, Pulumi, and Bicep/ARM. Live pricing from AWS, Azure, and GCP.
+  Multi-cloud cost analysis for Terraform, CloudFormation, Pulumi, and Bicep/ARM. Live AWS/Azure pricing with bundled GCP pricing snapshots.
   <br />
   Built on the <a href="https://modelcontextprotocol.io">Model Context Protocol</a> for seamless AI agent integration.
 </p>
@@ -10,7 +10,7 @@
   <a href="https://github.com/jadenrazo/CloudCostMCP/actions/workflows/ci.yml"><img src="https://github.com/jadenrazo/CloudCostMCP/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
   <a href="https://www.npmjs.com/package/@jadenrazo/cloudcost-mcp"><img src="https://img.shields.io/npm/v/@jadenrazo/cloudcost-mcp.svg" alt="npm version" /></a>
   <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT" /></a>
-  <img src="https://img.shields.io/badge/node-%3E%3D24-brightgreen" alt="Node.js" />
+  <img src="https://img.shields.io/badge/node-%3E%3D20-brightgreen" alt="Node.js" />
 </p>
 
 <p align="right">
@@ -28,13 +28,13 @@
 
 ---
 
-CloudCost MCP is a [Model Context Protocol](https://modelcontextprotocol.io) server that lets AI agents parse infrastructure-as-code across multiple formats (Terraform, CloudFormation, Pulumi, Bicep/ARM), query real-time pricing data, and generate multi-cloud cost comparison reports. It connects directly to public pricing APIs from AWS, Azure, and GCP. No API keys or cloud credentials required.
+CloudCost MCP is a [Model Context Protocol](https://modelcontextprotocol.io) server that lets AI agents parse infrastructure-as-code across multiple formats (Terraform, CloudFormation, Pulumi, Bicep/ARM), query pricing data, and generate multi-cloud cost comparison reports. AWS and Azure pricing are queried from public pricing endpoints; GCP pricing is served from bundled snapshots because Google does not provide CloudCost a credential-free live pricing path. No cloud credentials are required.
 
 ### What it does
 
 - Parses Terraform HCL files, CloudFormation templates, Pulumi stack exports, and Bicep/ARM templates with automatic format detection
 - Extracts resource inventories with variable resolution, including referenced modules and OpenTofu `.tofu` files
-- Queries live on-demand pricing from AWS Bulk Pricing CSV and Azure Retail Prices REST API; GCP via live Cloud Billing Catalog API with bundled fallback
+- Queries live on-demand pricing from AWS Bulk Pricing CSV and Azure Retail Prices REST API; GCP from a weekly-refreshed bundled snapshot
 - Maps equivalent resources across AWS, Azure, and GCP (compute, database, storage, networking, Kubernetes, container registries, secrets management, DNS)
 - Generates cost estimates with per-resource breakdowns (monthly and yearly) across multiple currencies
 - Compares costs across all three providers side-by-side in markdown, JSON, CSV, or FOCUS format
@@ -44,6 +44,16 @@ CloudCost MCP is a [Model Context Protocol](https://modelcontextprotocol.io) ser
 - Tags resources for cost attribution and groups report output by team, environment, or any custom label
 - Posts cost estimate comments to pull requests via a reusable GitHub Actions composite action
 
+### Provider pricing support
+
+| Provider | Pricing source | Support level |
+|---|---|---|
+| AWS | Live public pricing APIs + bundled fallback tables | Live |
+| Azure | Live Retail Prices API + bundled fallback tables | Live |
+| GCP | Bundled pricing snapshots + fixed public rates | Supported, but **not live** |
+
+GCP cost estimation still works, but its figures are snapshot-based and should not be described as real-time Google Cloud pricing. See [How Pricing Works](#how-pricing-works) and [Limitations](#limitations) for the freshness model.
+
 ### Supported IaC Formats
 
 | Format | Extensions | Auto-detected |
@@ -61,7 +71,7 @@ CloudCostMCP targets a different surface:
 
 - **Agent-native via MCP.** Models call it as a tool *during* generation. `check_cost_budget` returns `allow` / `warn` / `block` with the specific blocking resources named, fast enough on a warm pricing cache for an agent to veto an expensive config before writing it to disk.
 - **Multi-IaC in one server.** Terraform, CloudFormation, Pulumi, Bicep/ARM — one tool, not four.
-- **Zero credentials.** All pricing comes from public endpoints. No account, no cloud IAM, no API keys.
+- **Zero cloud credentials.** AWS and Azure pricing use public endpoints; GCP pricing uses bundled snapshots. No account, cloud IAM, or cloud API keys are required.
 - **Optimization + what-if scenarios built in.** Right-sizing, reserved-pricing, cross-provider switching, and spot modeling are first-class tools.
 
 The two are complementary. Use Infracost in CI; use CloudCostMCP inside your agent or editor.
@@ -310,7 +320,7 @@ Fast cost-safety guardrail designed for AI agents. Returns `allow` / `warn` / `b
 
 ## How Pricing Works
 
-CloudCost uses a tiered approach to get the most accurate pricing available without requiring any API keys or credentials.
+CloudCost uses a tiered approach to get the most accurate pricing available without requiring any API keys or credentials. That constraint is real and it has a cost: where a provider has closed off anonymous access, CloudCost ships a snapshot instead of pretending to query live. See the GCP section below.
 
 ### AWS
 
@@ -328,11 +338,13 @@ CloudCost uses a tiered approach to get the most accurate pricing available with
 
 ### GCP
 
-1. **Live Cloud Billing Catalog API** (primary). Queries the GCP Cloud Billing Catalog API (`cloudbilling.googleapis.com`) using unauthenticated public endpoints. Results are cached for 24 hours.
+**GCP has no live path, and this is a limitation rather than a design choice.** Google has retired every key-free bulk pricing source it used to publish, and `cloudbilling.googleapis.com` answers unregistered callers with `403 PERMISSION_DENIED`. Since CloudCost ships no credentials, GCP prices come from bundled data.
+
+1. **Bundled pricing data** (only path). Served from `data/gcp-pricing/`, which ships with the package. Covers Compute Engine machine types, Cloud SQL tiers, Cloud Storage classes and Persistent Disk types across 37 regions. Every GCP price is tagged `pricing_source: "bundled"` and carries its vintage in `pricing_metadata`.
 
-2. **Bundled pricing data** (fallback). If the live API is unreachable, falls back to curated pricing data in `data/gcp-pricing/` that ships with the package. Covers Compute Engine machine types, Cloud SQL tiers, Cloud Storage classes, and Persistent Disk types across all major regions.
+2. **How it stays current.** `.github/workflows/refresh-pricing.yml` rebuilds the Compute Engine, Persistent Disk and Cloud Storage tables weekly from [gcosts](https://github.com/Cyclenerd/google-cloud-pricing-cost-calculator) (Apache-2.0), a weekly regeneration of the Cloud Billing Catalog. The recorded `last_updated` is that snapshot's own generation date, not the date our job ran. Cloud SQL is not covered by that source and remains hand-curated; `check-freshness.ts` reports it separately.
 
-3. **Infrastructure services**. Load balancer, Cloud NAT, and GKE pricing use fixed public rates.
+3. **Infrastructure services**. Load balancer, Cloud NAT, and GKE pricing use fixed public rates. GKE Autopilot is an approximation: it is billed per-pod across vCPU, memory and ephemeral storage, and the bundled figure is a per-vCPU/hour estimate.
 
 ### Pricing Source Transparency
 
@@ -488,7 +500,7 @@ Configuration priority: environment variables > config file > built-in defaults.
   CSV (public)       (public, no auth)      (bundled files)
 ```
 
-Highlights: zero API keys (all providers exposed via public endpoints), SQLite-backed price cache shared across tool calls, streaming ingest for the 267 MB AWS bulk CSV, and a graceful live → fallback → interpolated-table chain so every response carries a `pricing_source` and `confidence` field. Full layer-by-layer walkthrough and extension guides in [docs/architecture.md](./docs/architecture.md).
+Highlights: no cloud credentials required; AWS and Azure use public pricing endpoints, while GCP uses bundled pricing snapshots and fixed public rates. SQLite-backed caching is used by live/fallback pricing paths, AWS bulk CSV ingest is streamed, and every response carries `pricing_source` and `confidence` so callers can distinguish live quotes from bundled or fallback estimates. Full layer-by-layer walkthrough and extension guides in [docs/architecture.md](./docs/architecture.md).
 
 ---
 
@@ -517,8 +529,8 @@ Instance type mapping covers 70+ AWS instance types (including Graviton/ARM fami
 ## Limitations
 
 - **On-demand pricing only** by default. Prices reflect pay-as-you-go rates. The `optimize_cost` tool recommends reserved instances; AWS Savings Plans are not yet supported (tracked in [docs/roadmap.md](./docs/roadmap.md)). Pass `pricing_model: "spot"` in `what_if` scenarios to model spot/preemptible pricing.
-- **GCP live pricing** is fetched from the Cloud Billing Catalog API with automatic fallback to bundled data when the API is unreachable. Bundled prices may lag slightly behind actual rates.
-- **Fallback-data signaling.** When a live pricing API is unreachable and `estimate_cost` / `compare_providers` / `get_pricing` serve data from bundled or fallback tables, the response includes a `warnings` entry ("using fallback/bundled pricing data for …") so callers can flag stale estimates. Bundled data is refreshed weekly via CI.
+- **GCP pricing is always bundled.** There is no live GCP path: `cloudbilling.googleapis.com` rejects unauthenticated callers and CloudCost ships no credentials. The bundled tables are rebuilt weekly (see *How Pricing Works > GCP*), and every GCP figure reports its own vintage, but they are a snapshot rather than a live quote.
+- **Fallback-data signaling.** When a live pricing API is unreachable and `estimate_cost` / `compare_providers` / `get_pricing` serve data from bundled or fallback tables, the response includes a `warnings` entry ("using fallback/bundled pricing data for …") so callers can flag stale estimates. AWS and Azure bundled tables are re-verified weekly via CI; the GCP tables are rebuilt weekly from an upstream snapshot. Datasets no automation covers (GCP Cloud SQL, AWS EBS, Azure disk and database tables) are listed under `curated_datasets` in each `data/*-pricing/metadata.json` and reported by `scripts/check-freshness.ts`.
 - **Synthetic egress line item.** Unless `CLOUDCOST_INCLUDE_DATA_TRANSFER=false`, every breakdown includes an assumed 100 GB/month of internet egress per provider+region. `total_monthly` includes it (unchanged behavior); its share is reported separately as `estimated_egress_monthly` with an explanatory warning so the composition is visible.
 - **Static FX rates.** Non-USD output is converted with a bundled rate table, not a live feed. Every converted response carries `exchange_rate: { rate, rate_as_of }` so consumers know which snapshot produced the figures.
 - **First request latency**. The initial EC2 pricing lookup for a new AWS region may take 30-120 seconds as the CSV file is streamed. Subsequent lookups for the same region are instant (cached for 24 hours).

package/data/aws-pricing/metadata.json

@@ -1,8 +1,28 @@
 {
-  "last_updated": "2026-08-05",
+  "last_updated": "2026-10-07",
+  "last_verified": "2026-10-07",
+  "verified_at": "2026-10-07T06:00:57.034Z",
+  "sources": [
+    {
+      "url": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.csv",
+      "retrieved_at": "2026-10-07T06:00:56.209Z",
+      "sha256": "3b72dd89fc32fa9c9f148373663221c28124abab29d366fe4577fcbee655e736",
+      "source_vintage": "2026-10-06T17:24:54Z"
+    },
+    {
+      "url": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonRDS/current/us-east-1/index.csv",
+      "retrieved_at": "2026-10-07T06:00:56.679Z",
+      "sha256": "6c216380d45af7df6376624ac5f63e321168c29717efe938ccb26737be4fea60",
+      "source_vintage": "2026-10-06T22:40:50Z"
+    }
+  ],
+  "refresh_policy": "automated",
   "source": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws",
   "sku_count": 142,
-  "refresh_script_version": "2.0.0",
+  "refresh_script_version": "3.1.0",
   "currency": "USD",
+  "curated_datasets": [
+    "EBS_BASE_PRICES (src/pricing/aws/fallback-data.ts)"
+  ],
   "notes": "Written by scripts/refresh-pricing.ts --write"
 }