model-eol 0.6.2 → 0.8.0

package/README.md

@@ -36,7 +36,7 @@ format instead of each scraping the same pages.
   replacement, `distributions` for per-channel lifecycles, and publisher `policy`
   floors. It is small enough that a provider could serve it at
   `/.well-known/model-eol.json` in an afternoon.
-- **`feeds/`** - <!-- feeds-status -->Amazon (4 entries), Anthropic (30 entries), Google (93 entries) and OpenAI (195 entries), generated from the providers' live deprecation pages plus the AWS Bedrock and Google Vertex AI lifecycle pages, feed data generated 2026-09-07<!-- /feeds-status -->. Every
+- **`feeds/`** - <!-- feeds-status -->Amazon (4 entries), Anthropic (32 entries), Cohere (12 entries), Google (93 entries), Mistral (40 entries) and OpenAI (195 entries), generated from the providers' live deprecation pages plus the AWS Bedrock, Google Vertex AI, and Azure Foundry lifecycle pages, feed data generated 2026-09-09<!-- /feeds-status -->. Every
   dated entry carries a source URL. Anthropic's "not sooner than" dates for
   active models are included as `tentative` planning floors.
 - **`check.mjs`** - zero-dependency CLI: CI gate, PR diff gate, inventory, CycloneDX
@@ -115,7 +115,8 @@ files. The checker, PR gate, schedule, SBOM export, and badge run with no
 credentials at all. The bot needs a GitHub token, and a fine-grained PAT if you
 want its PRs to trigger checks. Provider API keys are optional in exactly two
 places: your own eval command, and the models-endpoint coverage in the feed
-refresh.
+refresh. Refresh accepts `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`
+(or `GOOGLE_API_KEY`), `MISTRAL_API_KEY`, and `COHERE_API_KEY`.
 
 Both command-line binaries ship in one zero-dependency npm package. The test
 suite packs that package, installs it into an empty directory with the network
@@ -306,10 +307,25 @@ until January 8, 2027.
 
 `distributions` in the spec carries these per-channel dates. `--via <distributor>`
 judges your repo by the channel you call. The distributor refresh keeps the
-Bedrock and Vertex dates current from their lifecycle pages. A distributor's
-later date gives you more time for the same migration. It is not a reason to
+Bedrock, Vertex, and Azure Foundry dates current from their lifecycle pages.
+Bedrock combines its [legacy table](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html)
+with [model cards](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html).
+Card floors become tentative distribution dates without announcements.
+Under `--via aws-bedrock`, these floors remain `scheduled` and display "not sooner than".
+A distributor's later date gives you more time for the same migration. It is not a reason to
 skip it.
 
+The later date is also the last day you can still call the old model, when the
+feed gives it as an exact date. A tentative or earliest floor is "at least until",
+and the old model may answer past it. When a retired or retiring model still
+answers on another channel, `check` appends
+`[still answers via aws-bedrock until 2026-10-14]` to the line, and the bot adds
+a `## Capture window` section to its PRs and issues. If your eval compares the
+replacement's outputs against the old model's, capture that baseline before the
+window closes. model-eol reports the window from the feeds and never runs the
+capture; see [docs/DESIGN_EVAL_BOUNDARY.md](docs/DESIGN_EVAL_BOUNDARY.md) for
+where that line sits and why.
+
 ## Policy floors
 
 A feed entry with no `announced` and no `shutdown` means "nothing scheduled as
@@ -440,6 +456,8 @@ receives:
 - `MODEL_EOL_OLD_ID` and `MODEL_EOL_NEW_ID` for the current migration.
 - `MODEL_EOL_PLAN`, a one-model plan containing the exact changed references.
 - `MODEL_EOL_REPORT`, the required path for a bounded, publishable Markdown receipt.
+- `MODEL_EOL_VIA`, the distributor channel the plan was built for, or empty for the publisher's direct API. Use it to map the publisher's replacement ID to your client's ID on that channel.
+- `MODEL_EOL_EVAL_MODE`, always `evaluate` today. The value `capture` is reserved for a future run on the unpatched checkout against the old ID. The bot does not run it, and the hook refuses any other value.
 
 Exit zero plus a regular report file means pass. Any other exit, a timeout, a
 missing report, a tracked workspace change, or checkout drift fails that
@@ -505,12 +523,28 @@ read-only evaluation job, and write tokens live only in the reconciliation job.
 
 ```sh
 node refresh/refresh.mjs --check                      # semantic diff vs live pages; exit 3 = PR-worthy
-node refresh/refresh.mjs --distributor aws-bedrock,vertex-ai # distributor lifecycle clocks
+node refresh/refresh.mjs --distributor aws-bedrock,vertex-ai,azure-ai-foundry # distributor lifecycle clocks
 node scripts/feed-changelog.mjs                       # local rendering of the hosted Atom feed
 ```
 
 A parse failure stops the run. The refresh never writes a guessed feed.
 
+Anthropic reads model IDs and aliases from overview pages listed in `https://platform.claude.com/docs/llms.txt`.
+It reads at most 100 unique pages with 30-second deadlines and 8 MiB response limits.
+Only the deprecations page supplies Anthropic dates.
+Offline checks use `anthropic-llms.txt` and `anthropic-model-pages/<slug>.md`; missing pages stop refresh.
+Alias attachments and moves emit notices and appear in semantic diffs.
+
+AWS changed its Bedrock lifecycle policy on 2026-09-07.
+The legacy table covers models launched before that date; newer models publish lifecycle dates on individual cards.
+The Bedrock refresh reads both sources and keeps legacy-table records when IDs overlap.
+Conflicting exact dates appear under "Source conflicts"; the legacy table wins.
+When two card floors differ, the later day-precision date applies.
+Cards without lifecycle fields or day-precision dates appear under "Model cards without lifecycle fields" and emit notices.
+The crawl reads at most 400 unique cards sequentially, with 30-second deadlines and 8 MiB response limits.
+Offline checks use `bedrock-lifecycle.html`, `bedrock-model-cards.html`, and `bedrock-model-cards/<basename>` fixtures.
+Missing card fixtures stop the refresh.
+
 This runs automatically. A weekly workflow
 (`.github/workflows/feed-refresh.yml`, Mondays 05:23 UTC) checks the live
 sources and opens a PR with the semantic diff, plus an updated README freshness
@@ -566,9 +600,9 @@ is the most useful thing any of us can do here.
 - The checker matches known IDs only. It does not discover models that are
   absent from the feeds. This is deliberate: a CI gate needs precision more than
   discovery.
-- Fetchers: OpenAI, Anthropic, Google, aws-bedrock, and vertex-ai are live.
-  Azure dates are carried where OpenAI's own page publishes them. A standalone
-  Azure lifecycle fetcher is not built yet.
+- Fetchers: OpenAI, Anthropic, Google, Mistral, Cohere, aws-bedrock, vertex-ai,
+  and azure-ai-foundry are live. xAI publishes no lifecycle index and its models
+  API carries no deprecation fields, so xAI is not ingested.
 - On npm as [`model-eol`](https://www.npmjs.com/package/model-eol). Material
   feed changes republish automatically as patch versions, with trusted
   publishing and provenance. Code releases require an explicit stable version.

package/SPEC.md

@@ -69,6 +69,11 @@ Private arrangements after EOL are not represented by this status. `retired`
 means the distributor explicitly reports the model as no longer generally
 available even when it does not publish an exact shutdown date.
 
+Bedrock model-card floors are tentative distribution dates (`date_precision: "tentative"`) without `announced`.
+
+Azure maps GA/Preview/Legacy to `active`, Deprecated to `legacy`, and Retired to `retired`; Azure Legacy permits new deployments.
+Azure OpenAI uses a resolving bare identity for single-row models only when a dated snapshot candidate is absent.
+
 A model may contain at most one distribution for each `via`. Duplicate channel
 records are ambiguous and fail runtime semantic validation. Draft-07 can enforce
 whole-object uniqueness, but cannot express uniqueness keyed by the `via` field,

package/bot/bot.mjs

@@ -25,6 +25,7 @@ import {
   stableJson,
 } from './lib/common.mjs'
 import { assertValidPlanDocument } from '../lib/validate-document.mjs'
+import { captureWindowFor, lifecycleFor } from '../lib/feeds.mjs'
 import { parseCliArgs } from '../lib/cli.mjs'
 import { loadConfig } from './lib/config.mjs'
 import { downloadFeeds } from './lib/feeds.mjs'
@@ -363,11 +364,21 @@ const buildModelGroups = (plan, config, root, records) => {
   }
   for (const group of groups.values()) {
     group.items.sort((a, b) => `${a.file}:${a.line}:${a.occurrence}`.localeCompare(`${b.file}:${b.line}:${b.occurrence}`))
-    group.feedDigest = itemDigest(group.items)
+    group.feedDigest = groupDigest(group.items, group.context?.entry)
   }
   return [...groups.values()].sort((a, b) => `${a.publisher}/${a.id}`.localeCompare(`${b.publisher}/${b.id}`))
 }
 
+// the capture section reads clocks outside the plan items, so they join the digest that gates body updates
+const clockRows = entry => entry
+  ? {
+      shutdown: entry.shutdown ?? null,
+      date_precision: entry.date_precision ?? null,
+      distributions: itemDigest((entry.distributions ?? []).map(row => ({ via: row.via, shutdown: row.shutdown ?? null, status: row.status ?? null, date_precision: row.date_precision ?? null }))),
+    }
+  : null
+const groupDigest = (items, entry) => sha256(stableJson({ items: itemDigest(items), clocks: clockRows(entry) }))
+
 const buildIssueGroups = (plan, config, root, records) => {
   const ignoredIds = ignoredModelIds(plan, config)
   const groups = new Map()
@@ -398,7 +409,7 @@ const buildIssueGroups = (plan, config, root, records) => {
   }
   for (const group of groups.values()) {
     group.issues.sort((a, b) => `${a.file}:${a.line}:${a.reason}`.localeCompare(`${b.file}:${b.line}:${b.reason}`))
-    group.feedDigest = itemDigest(group.issues)
+    group.feedDigest = groupDigest(group.issues, group.context?.entry)
   }
   return [...groups.values()].sort((a, b) => `${a.publisher}/${a.subject}/${a.shutdown ?? ''}`.localeCompare(`${b.publisher}/${b.subject}/${b.shutdown ?? ''}`))
 }
@@ -484,6 +495,43 @@ const replacementSection = (item, now) => [
     : []),
 ]
 
+const captureWindow = (group, now) => {
+  const entry = group.context?.entry
+  if (!entry) return null
+  const lifecycle = lifecycleFor(entry, {
+    days: group.items?.[0]?.threshold_days ?? group.issues?.[0]?.threshold_days ?? 0,
+    via: group.via ?? null,
+    today: now,
+  })
+  return captureWindowFor(entry, lifecycle, { today: now })
+}
+
+// the first date on which the capture section changes, so a body can be refreshed when the digest cannot see it
+const captureExpiry = capture => capture
+  ? [capture.until, ...capture.alternatives.map(alternative => alternative.until)].filter(Boolean).sort()[0] ?? null
+  : null
+
+const captureExpired = (metadata, now) => typeof metadata.capture_expires === 'string' && metadata.capture_expires <= now.toISOString().slice(0, 10)
+
+const captureSection = (group, now) => {
+  const capture = captureWindow(group, now)
+  if (!capture) return ''
+  const clock = capture.via === 'publisher' || capture.via === 'publisher-fallback'
+    ? markdownText('publisher')
+    : markdownCode(capture.via)
+  const untilText = window => `${['tentative', 'earliest'].includes(window.date_precision) ? 'at least until' : 'until'} ${markdownText(window.until)}`
+  return [
+    '## Capture window',
+    capture.until !== null
+      ? `- The old model answers on the ${clock} clock ${untilText(capture)}.`
+      : `- The old model no longer answers on the ${clock} clock. No baseline can be captured from it there.`,
+    ...capture.alternatives.map(alternative => `- It still answers via ${markdownCode(alternative.via)} ${untilText(alternative)}.`),
+    capture.until === null && capture.alternatives.length === 0
+      ? '- The feed lists no dated clock on which the old model still answers, so model-eol cannot state a capture window. model-eol does not run captures.'
+      : '- Capture a baseline from the old model before the window closes if your eval compares outputs. model-eol does not run captures.',
+  ].join('\n')
+}
+
 export const buildPullBody = ({ group, headSha, baseSha = null, now = new Date(), tokenKind = null, evalResult = null, evalConfigHash = null }) => {
   const item = group.items[0]
   const announced = markdownText(group.context?.announced ?? 'not specified')
@@ -500,6 +548,7 @@ export const buildPullBody = ({ group, headSha, baseSha = null, now = new Date()
     base_sha: baseSha,
     head_sha: headSha,
     feed_digest: group.feedDigest,
+    capture_expires: captureExpiry(captureWindow(group, now)),
     ...(evalConfigHash ? { eval_config_digest: evalConfigHash } : {}),
   }
   const sections = [
@@ -522,6 +571,8 @@ export const buildPullBody = ({ group, headSha, baseSha = null, now = new Date()
   if (group.via) {
     sections.push('', '## Distributor clock', `This migration uses the ${markdownCode(group.via)} distributor clock; the shutdown date above is the date for that channel.`)
   }
+  const capture = captureSection(group, now)
+  if (capture) sections.push('', capture)
   const evaluation = evalSection(evalResult)
   if (evaluation) sections.push('', evaluation)
   sections.push(
@@ -547,6 +598,7 @@ export const buildIssueBody = ({ group, now = new Date() }) => {
     replacement_note: issue.replacement_note,
     head_sha: null,
     feed_digest: group.feedDigest,
+    capture_expires: captureExpiry(captureWindow(group, now)),
     ...(group.channel ? { channel: group.channel } : {}),
   }
   const evidence = group.issues
@@ -580,6 +632,8 @@ export const buildIssueBody = ({ group, now = new Date() }) => {
     '## Feed notes',
     notesSection(group),
   ]
+  const capture = captureSection(group, now)
+  if (capture) sections.push('', capture)
   const evaluation = evalSection(group.evalResult)
   if (evaluation) sections.push('', evaluation)
   return sections.join('\n')
@@ -841,6 +895,8 @@ export const evaluatePlan = async ({
               cwd: clone,
               oldId: group.id,
               newId: group.items[0].replacement,
+              via: group.via ?? null,
+              mode: 'evaluate',
               planPath: selectedPlanPath,
               reportPath,
             })
@@ -1090,7 +1146,7 @@ const processModel = async ({ api, pulls, issueRecords, group, source, base, bas
     if (externalEval && externalEval.status !== 'pass') {
       return processEvalFailure({ api, issueRecords, group, evalResult: externalEval, root, now, issuesEnabled: config.issues.enabled, open })
     }
-    if (open.metadata.feed_digest === currentDigest && open.metadata.base_sha === baseHead && (!evalConfigHash || open.metadata.eval_config_digest === evalConfigHash)) {
+    if (open.metadata.feed_digest === currentDigest && open.metadata.base_sha === baseHead && (!evalConfigHash || open.metadata.eval_config_digest === evalConfigHash) && !captureExpired(open.metadata, now)) {
       return decision(group, 'skip-unchanged', { number: open.item.number })
     }
     let patch
@@ -1250,7 +1306,7 @@ const processIssue = async ({ api, issues, group, now }) => {
   const open = matches.find(record => isOpen(record.item))
   const body = buildIssueBody({ group, now })
   if (open) {
-    if (open.metadata.feed_digest === group.feedDigest) return decision(group, 'skip-unchanged', { number: open.item.number, body })
+    if (open.metadata.feed_digest === group.feedDigest && !captureExpired(open.metadata, now)) return decision(group, 'skip-unchanged', { number: open.item.number, body })
     await api.updateIssue(open.item.number, { title: issueTitle(group), body })
     return decision(group, 'update', { number: open.item.number, body })
   }

package/bot/lib/common.mjs

@@ -16,7 +16,10 @@ const stableValue = value => {
 export const stableJson = value => JSON.stringify(stableValue(value))
 
 export const sortedJsonDigest = values => {
-  const sorted = values.slice().sort((a, b) => stableJson(a).localeCompare(stableJson(b)))
+  const sorted = values.slice().sort((a, b) => {
+    const [left, right] = [stableJson(a), stableJson(b)]
+    return left < right ? -1 : left > right ? 1 : 0
+  })
   return sha256(stableJson(sorted))
 }
 
@@ -67,6 +70,7 @@ export const parseMetadata = body => {
     if (metadata.replacement !== undefined && metadata.replacement !== null && typeof metadata.replacement !== 'string') return null
     if (metadata.replacement_options !== undefined && (!Array.isArray(metadata.replacement_options) || metadata.replacement_options.some(option => typeof option !== 'string'))) return null
     if (metadata.replacement_note !== undefined && metadata.replacement_note !== null && typeof metadata.replacement_note !== 'string') return null
+    if (metadata.capture_expires !== undefined && metadata.capture_expires !== null && typeof metadata.capture_expires !== 'string') return null
     if (metadata.base_sha !== undefined && metadata.base_sha !== null && (typeof metadata.base_sha !== 'string' || !/^[0-9a-f]{40,64}$/.test(metadata.base_sha))) return null
     if (metadata.stale_closed !== undefined && metadata.stale_closed !== true) return null
     if (typeof metadata.feed_digest !== 'string' || !metadata.feed_digest) return null
@@ -111,7 +115,11 @@ export const markdownText = value => String(value ?? '')
   .map(line => line.replace(/^(\s*)([-+>]\s|\d+\.\s)/, '$1&#45; '))
   .join('\n')
 
-export const markdownCode = value => `\`${markdownText(value)}\``
+// code spans show entities literally, so drop backticks and line breaks instead of escaping
+export const markdownCode = value => {
+  const text = String(value ?? '').replace(/[\r\n]+/g, ' ').replaceAll('`', '')
+  return `\`${text === '' ? ' ' : text}\``
+}
 
 export const markdownLink = (label, value) => {
   const text = String(value ?? '')

package/bot/lib/eval.mjs

@@ -50,7 +50,8 @@ export const readReportCapped = (file, limit) => {
   }
 }
 
-export const runEvalHook = ({ command, timeoutMs, maxReportBytes, passEnv = [], cwd, oldId, newId, planPath, reportPath }) => {
+export const runEvalHook = ({ command, timeoutMs, maxReportBytes, passEnv = [], cwd, oldId, newId, planPath, reportPath, via = null, mode = 'evaluate' }) => {
+  if (mode !== 'evaluate') throw new Error(`unsupported eval mode: ${mode}`)
   if (fs.existsSync(reportPath)) fs.rmSync(reportPath, { force: true })
   const env = {}
   for (const key of ['PATH', 'HOME', 'TMPDIR', 'LANG', 'USER', 'SHELL']) {
@@ -63,6 +64,8 @@ export const runEvalHook = ({ command, timeoutMs, maxReportBytes, passEnv = [],
   env.MODEL_EOL_NEW_ID = newId
   env.MODEL_EOL_PLAN = planPath
   env.MODEL_EOL_REPORT = reportPath
+  env.MODEL_EOL_VIA = via ?? ''
+  env.MODEL_EOL_EVAL_MODE = mode
 
   const result = spawnSync(command, {
     cwd,

package/check.mjs

@@ -406,7 +406,8 @@ if (command === 'plan') {
 
 if (command === 'check') {
   if (AS_JSON) {
-    console.log(JSON.stringify({ schema: 'model-eol/check@0.1', threshold_days: DAYS, distributor: VIA, scope: SCOPE, scan_notes: scan.notes, findings: changedFindings }, null, 2))
+    // capture stays out of the 0.1 document until the contract grows; the human line and the bot carry it
+    console.log(JSON.stringify({ schema: 'model-eol/check@0.1', threshold_days: DAYS, distributor: VIA, scope: SCOPE, scan_notes: scan.notes, findings: changedFindings.map(({ capture, ...finding }) => finding) }, null, 2))
   } else {
     console.log(formatCheck({ findings: changedFindings, bad, scannedFiles: scan.files.length, days: DAYS, scope: SCOPE }))
   }

package/feeds/amazon.json

@@ -15,7 +15,7 @@
           "announced": "2026-03-30",
           "shutdown": "2026-09-30",
           "status": "legacy",
-          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html"
+          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html"
         }
       ]
     },
@@ -30,7 +30,7 @@
           "announced": "2026-03-30",
           "shutdown": "2026-09-30",
           "status": "legacy",
-          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html"
+          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html"
         }
       ]
     },
@@ -45,7 +45,7 @@
           "announced": "2026-03-13",
           "shutdown": "2026-09-14",
           "status": "legacy",
-          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html"
+          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html"
         }
       ]
     },
@@ -60,7 +60,7 @@
           "announced": "2026-03-13",
           "shutdown": "2026-09-14",
           "status": "legacy",
-          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html"
+          "source": "https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html"
         }
       ]
     }