sourcelock 0.1.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,1232 @@
1
+ """ICD-10-CM / HCPCS Level II release-train adapter.
2
+
3
+ Serves date-of-service code questions from compact derived tables vendored at
4
+ build time from the real CMS release files (see
5
+ ``hc_source/data/codes/manifest.json`` for full provenance and
6
+ ``hc_source/data/codes/regenerate.py`` for the regeneration command).
7
+
8
+ The (code, date-of-service) -> release resolution rule
9
+ ------------------------------------------------------
10
+
11
+ ICD-10-CM has no per-code effective dates; validity is a property of the
12
+ release that governs the date of service. The rule implemented by
13
+ :func:`resolve_cm_release`, verified against the live CMS release train on
14
+ 2026-08-01:
15
+
16
+ 1. ``FY(D) = D.year + 1 if D.month >= 10 else D.year``. The FY-labeled file
17
+ set takes effect on October 1 of the *prior* calendar year (FY2026 =
18
+ 2025-10-01 .. 2026-09-30).
19
+ 2. Two candidate releases exist per fiscal year: the FY base file set
20
+ (``fyYYYY``) and, in years where CMS publishes one, the April 1 update
21
+ (``fyYYYY-april``), a full republished file set effective April 1 through
22
+ September 30 of that FY.
23
+ 3. ``D`` before April 1 of FY(D) -> the FY base release governs.
24
+ ``D`` on/after April 1 -> the April release governs if it is
25
+ published ("posted" in the manifest's ``cm_april_status`` ledger); the FY
26
+ base release governs if the ledger says "absent" (a settled fact: the
27
+ manifest was generated on/after that April 1 with no April package
28
+ published); and the lookup **fails closed** if the ledger says "unknown"
29
+ (the manifest was generated before that April 1, so an update may yet be
30
+ published -- answering from the base file would be a guess).
31
+ 4. The lookup also fails closed when the governing release has no vendored
32
+ table (date of service outside the vendored release train).
33
+
34
+ The effective interval stamped on each receipt is the sub-window the answer
35
+ is certain for: an FY base answer is capped at March 31 unless the April
36
+ status for that FY is a settled "absent"; an April answer covers April 1 ..
37
+ September 30.
38
+
39
+ HCPCS Level II is quarterly and *does* carry per-record dates. Verified
40
+ against all 1,327 termed records in the July 2026 ANWEB master: the record's
41
+ validity interval is ``[code_added, termination]``; ``action_effective`` is
42
+ the date the most recent maintenance action took effect (for discontinued
43
+ codes it is always ``termination + 1 day`` -- it is **not** the start of
44
+ validity). A validity verdict is only issued when the governing quarterly for
45
+ the date of service is vendored; otherwise the lookup fails closed.
46
+
47
+ Staged releases (posted but not yet effective)
48
+ ----------------------------------------------
49
+
50
+ CMS posts FY files ~4 months before they take effect (FY2027 was posted
51
+ 2026-06-05, effective 2026-10-01), so "a newer release exists upstream" must
52
+ never alarm by itself. The canaries split the two events:
53
+
54
+ * ``codes.cm_governing_release`` / ``codes.hcpcs_governing_quarter`` compare
55
+ the release that governs *today* against the pin -- they drift exactly when
56
+ the pin stops being correct for today's date (e.g. on October 1).
57
+ * ``codes.cm_release_train`` / ``codes.hcpcs_release_train`` observe what is
58
+ *posted* upstream. A future-effective posting is pinned as part of the
59
+ observed value, so it reports ok; drift fires only when something newly
60
+ posts, and the remediation says explicitly that it is staged, not an
61
+ emergency. ``codes.latest_release`` reports the same staged state as an OK
62
+ answer with a warning.
63
+
64
+ CPT: what is excluded, and what is not
65
+ --------------------------------------
66
+
67
+ HCPCS Level I (CPT) descriptors are AMA-licensed. Only the ANWEB fixed-width
68
+ master file is ingested (verified to contain zero numeric-leading CPT and
69
+ zero D#### ADA CDT records); ``proc_notes_*.txt`` and the ``.xlsx`` members
70
+ are never read.
71
+
72
+ That excludes CPT *records* and CPT *descriptors*. It does not exclude CPT
73
+ code *numbers*. CMS writes part of its own Level II definitions as
74
+ cross-references, so the shipped text of G0317 contains "list separately in
75
+ addition to cpt codes 99306, 99310" and the shipped text of G0561 contains
76
+ "0583t". Seventeen vendored records read that way; they are named in
77
+ ``CPT_CROSS_REFERENCED_CODES`` below. The non-claim key used to be
78
+ ``DOES_NOT_INCLUDE_CPT``, which asserted the opposite of the shipped bytes;
79
+ it is now ``DOES_NOT_INCLUDE_CPT_DESCRIPTORS``, it states the
80
+ cross-references and names every record carrying one, and
81
+ ``codes.hcpcs_lookup`` lists the cited identifiers on the record itself so a
82
+ caller learns of them from the receipt rather than from a lawyer.
83
+
84
+ No pattern match can tell a government cross-reference to a CPT number from
85
+ an AMA-authored descriptor vendored by mistake -- that is a reading, not a
86
+ regex. So the reviewed set is pinned here and
87
+ ``tests/test_codes_cpt_claim.py`` re-derives it from the vendored tables on
88
+ every run: a re-vendoring that adds, drops, or moves a cross-reference fails
89
+ the build until a person decides which kind it is.
90
+
91
+ Runtime data location: ``hc_source/data/codes`` (package data, shipped in the
92
+ wheel), overridable with the ``HC_SOURCE_CODES_DATA`` env var
93
+ (also how the test suite simulates drift). ``HC_SOURCE_CODES_TODAY`` freezes
94
+ "today" for deterministic canary runs and tests.
95
+ """
96
+
97
+ from __future__ import annotations
98
+
99
+ import gzip
100
+ import hashlib
101
+ import io
102
+ import csv
103
+ import os
104
+ import re
105
+ from dataclasses import dataclass
106
+ import datetime as _dt
107
+ from datetime import date, timedelta
108
+ from pathlib import Path
109
+ from typing import Any
110
+
111
+ from pydantic import BaseModel, ConfigDict, Field
112
+
113
+ from ..http import FetchResult, SourceUnreachable, as_cache_hit, fetch
114
+ from ..interfaces import Canary, CanaryObservation, SourceAdapter, ToolResult, ToolSpec
115
+ from ..receipts import build_receipt
116
+ from ..schemas import CanaryStatus, SourceContract
117
+
118
+ SOURCE_ID = "codes"
119
+ #: Bumped to "2" when codes.hcpcs_lookup started deriving cpt_cross_references
120
+ #: from the record's description text.
121
+ TRANSFORM_VERSION = "2"
122
+
123
+ #: Environment overrides. DATA points at an alternate derived-data directory;
124
+ #: TODAY freezes the clock (ISO date) for deterministic runs.
125
+ DATA_ENV_VAR = "HC_SOURCE_CODES_DATA"
126
+ TODAY_ENV_VAR = "HC_SOURCE_CODES_TODAY"
127
+
128
+ CDC_LISTING_URL = "https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/ICD10CM/"
129
+ CMS_LANDING_URL = "https://www.cms.gov/medicare/coding-billing/icd-10-codes"
130
+ CMS_ZIP_BASE = "https://www.cms.gov/files/zip/"
131
+ HCPCS_INDEX_URL = (
132
+ "https://www.cms.gov/medicare/coding-billing/healthcare-common-procedure-system/"
133
+ "quarterly-update"
134
+ )
135
+
136
+ REGENERATE = "python hc_source/data/codes/regenerate.py"
137
+ ZIP_MAGIC = b"PK\x03\x04"
138
+
139
+ _QUARTER_MONTHS = {1: "january", 2: "april", 3: "july", 4: "october"}
140
+
141
+ CONTRACT = SourceContract(
142
+ source_id=SOURCE_ID,
143
+ authority_url=CMS_LANDING_URL,
144
+ fallback_url=CDC_LISTING_URL,
145
+ license_notes=(
146
+ "ICD-10-CM (NCHS/CDC), ICD-10-PCS and HCPCS Level II (CMS) are US government "
147
+ "works, public domain and freely redistributable. HCPCS Level I (CPT) and ADA "
148
+ "CDT descriptors are NOT: this adapter ingests only the ANWEB Level II master "
149
+ "file (verified free of CPT/CDT records) and never the proc_notes or xlsx "
150
+ "members. CMS's own Level II descriptions do cite CPT code numbers in a small, "
151
+ "named set of records (see the DOES_NOT_INCLUDE_CPT_DESCRIPTORS non-claim); "
152
+ "those citations ship as CMS published them. Redistributing a government "
153
+ "description that cites a CPT number is not the same as redistributing a CPT "
154
+ "descriptor, and neither this adapter nor these notes convey an AMA licence."
155
+ ),
156
+ cadence=(
157
+ "ICD-10-CM: annual (files posted ~June, effective October 1) plus an optional "
158
+ "April 1 mid-year update (posted ~December). HCPCS Level II: quarterly "
159
+ "(posted ~2-6 weeks before the quarter starts, reposted in place for "
160
+ "corrections)."
161
+ ),
162
+ effective_date_semantics=(
163
+ "effective_from/effective_to bound the dates of service the answering release "
164
+ "governs. ICD-10-CM: the FY base file governs October 1 - March 31, and the "
165
+ "April 1 update (when published) governs April 1 - September 30; a base-file "
166
+ "answer is capped at March 31 unless the April status for that FY is a settled "
167
+ "'absent'. HCPCS: the quarterly file governs its calendar quarter, and within "
168
+ "it a record's own [code_added, termination] interval is authoritative."
169
+ ),
170
+ invariants=[
171
+ "an ICD-10-CM release's valid-code table has an exact, release-specific row "
172
+ "count (FY2026: 74,719; FY2027: 74,879)",
173
+ "ICD-10-CM codes never change mid-FY except via a published April 1 update",
174
+ "FY files are posted months before they take effect: posted is not effective",
175
+ "the HCPCS ANWEB master contains only Level II codes and 2-char modifiers -- "
176
+ "zero CPT (numeric-leading) and zero ADA CDT (D####) records",
177
+ "CMS-authored Level II descriptions cite CPT code numbers in a fixed, reviewed "
178
+ "set of records; a change to that set is a licensing review, not a data update",
179
+ "every termed HCPCS record satisfies action_effective == termination + 1 day",
180
+ "CMS reposts artifacts in place at the same URL; URL identity never proves "
181
+ "content identity",
182
+ ],
183
+ )
184
+
185
+ # ---------------------------------------------------------------------------
186
+ # CPT identifiers cited inside CMS-authored descriptions
187
+ # ---------------------------------------------------------------------------
188
+
189
+ #: Category I CPT identifiers are five digits.
190
+ _CPT_CATEGORY_I_RE = re.compile(r"(?<![0-9A-Za-z])(\d{5})(?![0-9A-Za-z])")
191
+
192
+ #: Category II identifiers are four digits plus ``F`` and span the whole
193
+ #: leading-digit range. Category III (``T``) and Proprietary Laboratory
194
+ #: Analyses (``U``) identifiers are four digits plus a letter but are entirely
195
+ #: 0-leading, and requiring that zero is what keeps a dose abbreviation out of
196
+ #: the scan: "Inj heparin sodium per 1000u" is a quantity, "0583t" is a code.
197
+ _CPT_CATEGORY_II_III_RE = re.compile(
198
+ r"(?<![0-9A-Za-z])(\d{4}F|0\d{3}[TU])(?![0-9A-Za-z])", re.IGNORECASE
199
+ )
200
+
201
+
202
+ def scan_cpt_identifiers(*texts: str | None) -> tuple[str, ...]:
203
+ """Return the CPT-shaped identifiers cited in description text, sorted.
204
+
205
+ Deliberately over-inclusive: it reports *candidates*, because a five-digit
206
+ number in a CMS sentence could in principle be a quantity rather than a
207
+ code. Classification is a human gate, pinned in
208
+ :data:`CPT_CROSS_REFERENCED_CODES` and enforced by
209
+ ``tests/test_codes_cpt_claim.py``.
210
+ """
211
+ found: set[str] = set()
212
+ for text in texts:
213
+ if not text:
214
+ continue
215
+ found.update(_CPT_CATEGORY_I_RE.findall(text))
216
+ found.update(match.upper() for match in _CPT_CATEGORY_II_III_RE.findall(text))
217
+ return tuple(sorted(found))
218
+
219
+
220
+ def cited_cpt_identifiers(records: list[dict[str, str]]) -> tuple[str, ...]:
221
+ """CPT identifiers cited by one code's records (long and short text)."""
222
+ texts: list[str | None] = []
223
+ for record in records:
224
+ texts.append(record.get("long_description"))
225
+ texts.append(record.get("short_description"))
226
+ return scan_cpt_identifiers(*texts)
227
+
228
+
229
+ #: The vendored HCPCS Level II records whose CMS-authored description text
230
+ #: cites a CPT identifier, reviewed by hand on 2026-08-01. Pinned rather than
231
+ #: computed because the distinction that matters -- the government citing a CPT
232
+ #: *number* as part of its own definition, versus an AMA-authored *descriptor*
233
+ #: having been vendored by mistake -- is a reading, and the failure it prevents
234
+ #: is shipping an AMA descriptor under a receipt that says we ship none.
235
+ CPT_CROSS_REFERENCED_CODES = (
236
+ "AT",
237
+ "G0279",
238
+ "G0316",
239
+ "G0317",
240
+ "G0318",
241
+ "G0500",
242
+ "G0561",
243
+ "G2058",
244
+ "G2212",
245
+ "M1483",
246
+ "M1485",
247
+ "QP",
248
+ "S8055",
249
+ "S9034",
250
+ "S9123",
251
+ "V2530",
252
+ "V2531",
253
+ )
254
+
255
+
256
+ NON_CLAIMS = [
257
+ "DOES_NOT_INCLUDE_CPT_DESCRIPTORS: no HCPCS Level I (CPT) record and no ADA CDT "
258
+ "record is vendored here, and no AMA- or ADA-authored descriptor text is "
259
+ "reproduced. What is served is government-authored: NCHS ICD-10-CM codes and "
260
+ "descriptions, and CMS HCPCS Level II codes, modifiers and descriptions from the "
261
+ "ANWEB master file, verified to contain zero numeric-leading CPT and zero D#### "
262
+ f"CDT records. {len(CPT_CROSS_REFERENCED_CODES)} of those CMS descriptions cite "
263
+ "CPT code numbers inside the government's own definition text -- G0317 reads "
264
+ "\"list separately in addition to cpt codes 99306, 99310\" -- and the complete set "
265
+ f"is {', '.join(CPT_CROSS_REFERENCED_CODES)}. Those digits ship exactly as CMS "
266
+ "published them, carry no AMA descriptor with them, and are listed on the record "
267
+ "by codes.hcpcs_lookup. CPT is copyright the American Medical Association: "
268
+ "nothing here conveys a licence to it.",
269
+ "DOES_NOT_PROVE_COVERAGE: a code being valid for a date of service says nothing "
270
+ "about whether any payer covers, prices, or pays for it.",
271
+ "DOES_NOT_PROVE_CURRENCY: answers come from the release train vendored at build "
272
+ "time; CMS reposts corrections in place at the same URLs, so the vendored tables "
273
+ "may lag an unannounced correction. `hc-source doctor` checks the train.",
274
+ ]
275
+
276
+
277
+ # ---------------------------------------------------------------------------
278
+ # Clock, data directory, cached file loading
279
+ # ---------------------------------------------------------------------------
280
+
281
+
282
+ def _today() -> date:
283
+ override = os.environ.get(TODAY_ENV_VAR)
284
+ return date.fromisoformat(override) if override else date.today()
285
+
286
+
287
+ def _data_dir() -> Path:
288
+ override = os.environ.get(DATA_ENV_VAR)
289
+ if override:
290
+ return Path(override)
291
+ return Path(__file__).resolve().parents[1] / "data" / "codes"
292
+
293
+
294
+ _cache: dict[str, tuple[tuple[int, int] | None, FetchResult, Any]] = {}
295
+
296
+
297
+ def _load_file(name: str, parser) -> tuple[FetchResult, Any]:
298
+ """Read a vendored file through the shared transport, with a stat-keyed cache."""
299
+ path = _data_dir() / name
300
+ try:
301
+ stat = path.stat()
302
+ signature: tuple[int, int] | None = (stat.st_mtime_ns, stat.st_size)
303
+ except OSError:
304
+ signature = None
305
+ key = str(path)
306
+ cached = _cache.get(key)
307
+ if cached is not None and signature is not None and cached[0] == signature:
308
+ # Same object, marked as the reuse it is. `retrieved_at` stays where the
309
+ # read put it -- that was the round-1 fix -- and `cache_hit` now agrees
310
+ # with it instead of claiming every answer came off a fresh read.
311
+ return as_cache_hit(cached[1]), cached[2]
312
+ result = fetch(path.as_uri()) # raises SourceUnreachable if unreadable
313
+ parsed = parser(result.content)
314
+ if signature is not None:
315
+ _cache[key] = (signature, result, parsed)
316
+ return result, parsed
317
+
318
+
319
+ def _parse_manifest(content: bytes) -> dict[str, Any]:
320
+ import json
321
+
322
+ return json.loads(content)
323
+
324
+
325
+ def _parse_cm_table(content: bytes) -> dict[str, str]:
326
+ text = gzip.decompress(content).decode("utf-8")
327
+ reader = csv.reader(io.StringIO(text))
328
+ next(reader) # header
329
+ return {row[0]: row[1] for row in reader}
330
+
331
+
332
+ def _parse_hcpcs_table(content: bytes) -> dict[str, list[dict[str, str]]]:
333
+ text = gzip.decompress(content).decode("utf-8")
334
+ reader = csv.DictReader(io.StringIO(text))
335
+ table: dict[str, list[dict[str, str]]] = {}
336
+ for row in reader:
337
+ table.setdefault(row["code"], []).append(row)
338
+ for records in table.values():
339
+ records.sort(key=lambda r: r["seq"])
340
+ return table
341
+
342
+
343
+ def _load_manifest() -> tuple[FetchResult, dict[str, Any]]:
344
+ return _load_file("manifest.json", _parse_manifest)
345
+
346
+
347
+ # ---------------------------------------------------------------------------
348
+ # (code, date-of-service) -> release resolution
349
+ # ---------------------------------------------------------------------------
350
+
351
+
352
+ @dataclass(frozen=True)
353
+ class CmResolution:
354
+ """The ICD-10-CM release governing a date of service, and the sub-window
355
+ of dates the answer is certain for."""
356
+
357
+ release: str
358
+ effective_from: date
359
+ effective_to: date
360
+
361
+
362
+ @dataclass(frozen=True)
363
+ class CmUnresolved:
364
+ """Fail-closed outcome: no vendored release can be proven to govern."""
365
+
366
+ reason: str # stable key: AMBIGUOUS_APRIL_WINDOW | RELEASE_NOT_VENDORED
367
+ detail: str # one human sentence with the fix
368
+
369
+
370
+ def resolve_cm_release(dos: date, manifest: dict[str, Any]) -> CmResolution | CmUnresolved:
371
+ """Apply the resolution rule documented in the module docstring."""
372
+ fy = dos.year + 1 if dos.month >= 10 else dos.year
373
+ fy_start = date(fy - 1, 10, 1)
374
+ march_31 = date(fy, 3, 31)
375
+ april_1 = date(fy, 4, 1)
376
+ fy_end = date(fy, 9, 30)
377
+ april_status = manifest.get("cm_april_status", {}).get(str(fy), "unknown")
378
+
379
+ if dos < april_1:
380
+ release = f"fy{fy}"
381
+ effective_from = fy_start
382
+ effective_to = fy_end if april_status == "absent" else march_31
383
+ elif april_status == "posted":
384
+ release = f"fy{fy}-april"
385
+ effective_from = april_1
386
+ effective_to = fy_end
387
+ elif april_status == "absent":
388
+ release = f"fy{fy}"
389
+ effective_from = fy_start
390
+ effective_to = fy_end
391
+ else:
392
+ return CmUnresolved(
393
+ "AMBIGUOUS_APRIL_WINDOW",
394
+ f"{dos.isoformat()} falls in the April-update window of FY{fy} "
395
+ f"(April 1 - September 30 {fy}), and whether an April 1 {fy} update exists "
396
+ f"was unknown when the vendored manifest was generated. Failing closed: "
397
+ f"re-vendor once the April status is settled ({REGENERATE}), then retry.",
398
+ )
399
+
400
+ if release not in manifest.get("cm_releases", {}):
401
+ vendored = ", ".join(sorted(manifest.get("cm_releases", {}))) or "none"
402
+ return CmUnresolved(
403
+ "RELEASE_NOT_VENDORED",
404
+ f"{dos.isoformat()} is governed by ICD-10-CM release {release}, which is "
405
+ f"not vendored in this build (vendored: {vendored}). Add it to "
406
+ f"CM_RELEASES in regenerate.py and re-run ({REGENERATE}), then retry.",
407
+ )
408
+ return CmResolution(release=release, effective_from=effective_from, effective_to=effective_to)
409
+
410
+
411
+ def _quarter_label(d: date) -> str:
412
+ return f"{d.year}q{(d.month - 1) // 3 + 1}"
413
+
414
+
415
+ def _quarter_start(label: str) -> date:
416
+ year, q = label.split("q")
417
+ return date(int(year), (int(q) - 1) * 3 + 1, 1)
418
+
419
+
420
+ def _next_quarter(label: str) -> str:
421
+ year, q = int(label[:4]), int(label[-1])
422
+ return f"{year + 1}q1" if q == 4 else f"{year}q{q + 1}"
423
+
424
+
425
+ def _hcpcs_quarter_url(label: str, *, plural: bool = False) -> str:
426
+ month = _QUARTER_MONTHS[int(label[-1])]
427
+ suffix = "files" if plural else "file"
428
+ return f"{CMS_ZIP_BASE}{month}-{label[:4]}-alpha-numeric-hcpcs-{suffix}.zip"
429
+
430
+
431
+ # ---------------------------------------------------------------------------
432
+ # Typed public parameters
433
+ # ---------------------------------------------------------------------------
434
+
435
+
436
+ class NoParams(BaseModel):
437
+ model_config = ConfigDict(extra="forbid")
438
+
439
+
440
+ class ValidOnParams(BaseModel):
441
+ model_config = ConfigDict(extra="forbid")
442
+
443
+ code: str = Field(
444
+ pattern=r"^[A-Za-z][0-9][0-9A-Za-z]\.?[0-9A-Za-z]{0,4}$",
445
+ description="ICD-10-CM code, with or without the dot (E11.9 or E119).",
446
+ )
447
+ date: _dt.date = Field(description="Date of service (ISO, e.g. 2026-07-01).")
448
+
449
+
450
+ class LookupParams(BaseModel):
451
+ model_config = ConfigDict(extra="forbid")
452
+
453
+ code: str = Field(
454
+ pattern=r"^[A-Za-z][0-9][0-9A-Za-z]\.?[0-9A-Za-z]{0,4}$",
455
+ description="ICD-10-CM code, with or without the dot.",
456
+ )
457
+ date: _dt.date | None = Field(
458
+ default=None,
459
+ description="Date of service selecting the governing release; defaults to today.",
460
+ )
461
+
462
+
463
+ class HcpcsLookupParams(BaseModel):
464
+ model_config = ConfigDict(extra="forbid")
465
+
466
+ code: str = Field(
467
+ pattern=r"^([A-Za-z][0-9]{4}|[A-Za-z0-9]{2})$",
468
+ description="HCPCS Level II code (e.g. J0171) or 2-character modifier (e.g. A1).",
469
+ )
470
+ date: _dt.date | None = Field(
471
+ default=None,
472
+ description=(
473
+ "Date of service for a validity verdict; requires the governing quarterly "
474
+ "to be vendored. Omit for a description/date lookup from the pinned quarter."
475
+ ),
476
+ )
477
+
478
+
479
+ # ---------------------------------------------------------------------------
480
+ # Shared helpers for handlers
481
+ # ---------------------------------------------------------------------------
482
+
483
+
484
+ def _dotted(code_nodot: str) -> str:
485
+ return code_nodot if len(code_nodot) <= 3 else f"{code_nodot[:3]}.{code_nodot[3:]}"
486
+
487
+
488
+ def _fail_closed(route: str, fetch_result: FetchResult, warning: str) -> ToolResult:
489
+ return ToolResult(
490
+ data=None,
491
+ receipt=build_receipt(
492
+ contract=CONTRACT,
493
+ route=route,
494
+ fetch=fetch_result,
495
+ source_version="unresolved",
496
+ transform_version=TRANSFORM_VERSION,
497
+ warnings=[warning],
498
+ non_claims=NON_CLAIMS,
499
+ ),
500
+ )
501
+
502
+
503
+ def _cm_provenance(entry: dict[str, Any]) -> dict[str, Any]:
504
+ return {
505
+ "source_url": entry["source_url"],
506
+ "source_zip_sha256": entry["source_zip_sha256"],
507
+ "source_last_modified": entry["source_last_modified"],
508
+ "inner_file": entry["inner_file"],
509
+ "inner_sha256": entry["inner_sha256"],
510
+ "fetched_at": entry["fetched_at"],
511
+ "table": entry["table"],
512
+ "table_canonical_sha256": entry["canonical_sha256"],
513
+ "content_identical_to": entry.get("content_identical_to"),
514
+ "regenerate": REGENERATE,
515
+ }
516
+
517
+
518
+ def _iso(yyyymmdd: str) -> str | None:
519
+ return f"{yyyymmdd[:4]}-{yyyymmdd[4:6]}-{yyyymmdd[6:8]}" if yyyymmdd else None
520
+
521
+
522
+ # ---------------------------------------------------------------------------
523
+ # Handlers
524
+ # ---------------------------------------------------------------------------
525
+
526
+
527
+ def _resolve_and_load_cm(
528
+ route: str, code: str, dos: date
529
+ ) -> tuple[ToolResult | None, dict[str, Any] | None, Any, Any, CmResolution | None]:
530
+ """Common resolution path. Returns (fail_result, entry, table_fetch, table, resolution)."""
531
+ manifest_fetch, manifest = _load_manifest()
532
+ resolution = resolve_cm_release(dos, manifest)
533
+ if isinstance(resolution, CmUnresolved):
534
+ return (
535
+ _fail_closed(route, manifest_fetch, f"{resolution.reason}: {resolution.detail}"),
536
+ None,
537
+ None,
538
+ None,
539
+ None,
540
+ )
541
+ entry = manifest["cm_releases"][resolution.release]
542
+ table_fetch, table = _load_file(entry["table"], _parse_cm_table)
543
+ return None, entry, table_fetch, table, resolution
544
+
545
+
546
+ def _valid_on(params: ValidOnParams) -> ToolResult:
547
+ route = "codes.valid_on"
548
+ code_nodot = params.code.replace(".", "").upper()
549
+ failed, entry, table_fetch, table, resolution = _resolve_and_load_cm(
550
+ route, code_nodot, params.date
551
+ )
552
+ if failed is not None:
553
+ return failed
554
+ assert entry is not None and resolution is not None
555
+
556
+ description = table.get(code_nodot)
557
+ data: dict[str, Any] = {
558
+ "code": _dotted(code_nodot),
559
+ "code_nodot": code_nodot,
560
+ "code_system": "ICD-10-CM",
561
+ "date": params.date.isoformat(),
562
+ "valid": description is not None,
563
+ "description": description,
564
+ "release": resolution.release,
565
+ "release_effective_from": resolution.effective_from.isoformat(),
566
+ "release_effective_to": resolution.effective_to.isoformat(),
567
+ }
568
+ if description is None:
569
+ data["note"] = (
570
+ f"{_dotted(code_nodot)} is not in the valid/billable code set of release "
571
+ f"{resolution.release}; it may be a non-billable category header or not an "
572
+ "ICD-10-CM code at all."
573
+ )
574
+ return ToolResult(
575
+ data=data,
576
+ receipt=build_receipt(
577
+ contract=CONTRACT,
578
+ route=route,
579
+ fetch=table_fetch,
580
+ source_version=resolution.release,
581
+ transform_version=TRANSFORM_VERSION,
582
+ effective_from=resolution.effective_from,
583
+ effective_to=resolution.effective_to,
584
+ non_claims=NON_CLAIMS,
585
+ ),
586
+ )
587
+
588
+
589
+ def _lookup(params: LookupParams) -> ToolResult:
590
+ route = "codes.lookup"
591
+ dos = params.date or _today()
592
+ code_nodot = params.code.replace(".", "").upper()
593
+ failed, entry, table_fetch, table, resolution = _resolve_and_load_cm(route, code_nodot, dos)
594
+ if failed is not None:
595
+ return failed
596
+ assert entry is not None and resolution is not None
597
+
598
+ description = table.get(code_nodot)
599
+ warnings: list[str] = []
600
+ data: dict[str, Any] | None
601
+ if description is None:
602
+ warnings.append(
603
+ f"Code {_dotted(code_nodot)} not found in release {resolution.release}."
604
+ )
605
+ data = None
606
+ else:
607
+ data = {
608
+ "code": _dotted(code_nodot),
609
+ "code_nodot": code_nodot,
610
+ "code_system": "ICD-10-CM",
611
+ "description": description,
612
+ "release": resolution.release,
613
+ "release_effective_from": resolution.effective_from.isoformat(),
614
+ "release_effective_to": resolution.effective_to.isoformat(),
615
+ "provenance": _cm_provenance(entry),
616
+ }
617
+ return ToolResult(
618
+ data=data,
619
+ receipt=build_receipt(
620
+ contract=CONTRACT,
621
+ route=route,
622
+ fetch=table_fetch,
623
+ source_version=resolution.release,
624
+ transform_version=TRANSFORM_VERSION,
625
+ effective_from=resolution.effective_from,
626
+ effective_to=resolution.effective_to,
627
+ warnings=warnings,
628
+ non_claims=NON_CLAIMS,
629
+ ),
630
+ )
631
+
632
+
633
+ def _hcpcs_lookup(params: HcpcsLookupParams) -> ToolResult:
634
+ route = "codes.hcpcs_lookup"
635
+ manifest_fetch, manifest = _load_manifest()
636
+ releases: dict[str, Any] = manifest.get("hcpcs_releases", {})
637
+
638
+ if params.date is not None:
639
+ label = _quarter_label(params.date)
640
+ if label not in releases:
641
+ vendored = ", ".join(sorted(releases)) or "none"
642
+ return _fail_closed(
643
+ route,
644
+ manifest_fetch,
645
+ f"RELEASE_NOT_VENDORED: {params.date.isoformat()} is governed by the "
646
+ f"HCPCS quarterly {label}, which is not vendored in this build "
647
+ f"(vendored: {vendored}). Failing closed rather than answering from a "
648
+ f"different quarter's snapshot. Add it to HCPCS_RELEASES in "
649
+ f"regenerate.py and re-run ({REGENERATE}).",
650
+ )
651
+ else:
652
+ label = max(releases) # the pinned (latest vendored) quarterly
653
+
654
+ entry = releases[label]
655
+ table_fetch, table = _load_file(entry["table"], _parse_hcpcs_table)
656
+ code = params.code.upper()
657
+ records = table.get(code)
658
+ warnings: list[str] = []
659
+
660
+ if records is None:
661
+ return ToolResult(
662
+ data=None,
663
+ receipt=build_receipt(
664
+ contract=CONTRACT,
665
+ route=route,
666
+ fetch=table_fetch,
667
+ source_version=label,
668
+ transform_version=TRANSFORM_VERSION,
669
+ effective_from=date.fromisoformat(entry["effective_from"]),
670
+ effective_to=date.fromisoformat(entry["effective_to"]),
671
+ warnings=[f"Code {code} not found in HCPCS quarterly {label}."],
672
+ non_claims=NON_CLAIMS,
673
+ ),
674
+ )
675
+
676
+ primary = records[0]
677
+ long_description = " ".join(r["long_description"] for r in records if r["long_description"])
678
+ data: dict[str, Any] = {
679
+ "code": code,
680
+ "code_system": "HCPCS Level II",
681
+ "kind": "modifier" if len(code) == 2 else "code",
682
+ "long_description": long_description,
683
+ "short_description": primary["short_description"],
684
+ "code_added": _iso(primary["code_added"]),
685
+ "termination": _iso(primary["termination"]),
686
+ "last_action_effective": _iso(primary["action_effective"]),
687
+ "action_code": primary["action_code"],
688
+ "record_count": len(records),
689
+ "release": label,
690
+ "release_effective_from": entry["effective_from"],
691
+ "release_effective_to": entry["effective_to"],
692
+ }
693
+
694
+ # Disclose the government's own CPT cross-references on the record that
695
+ # carries them. Derived from the shipped bytes rather than from a table, so
696
+ # it cannot go stale against a re-vendoring; the reviewed pin lives in
697
+ # CPT_CROSS_REFERENCED_CODES and is enforced at test time. This prevents the
698
+ # failure the old DOES_NOT_INCLUDE_CPT non-claim caused: a caller reading
699
+ # "no CPT here" off a receipt whose description text names CPT codes.
700
+ cpt_cited = cited_cpt_identifiers(records)
701
+ if cpt_cited:
702
+ data["cpt_cross_references"] = list(cpt_cited)
703
+ warnings.append(
704
+ "CPT_CROSS_REFERENCE: this record's CMS-authored description cites the CPT "
705
+ f"identifiers {', '.join(cpt_cited)} as part of the government's own "
706
+ "definition text. The digits are reproduced as CMS published them; no AMA "
707
+ "CPT descriptor is included, and this is not a CPT licence."
708
+ )
709
+
710
+ if params.date is not None:
711
+ added = primary["code_added"]
712
+ termination = primary["termination"]
713
+ if not added:
714
+ data["valid_on"] = {"date": params.date.isoformat(), "valid": None}
715
+ warnings.append(
716
+ f"Record for {code} carries no code-added date; validity on "
717
+ f"{params.date.isoformat()} is indeterminate (failing closed)."
718
+ )
719
+ else:
720
+ added_date = date.fromisoformat(_iso(added)) # type: ignore[arg-type]
721
+ term_date = date.fromisoformat(_iso(termination)) if termination else None
722
+ valid = added_date <= params.date and (term_date is None or params.date <= term_date)
723
+ data["valid_on"] = {"date": params.date.isoformat(), "valid": valid}
724
+
725
+ return ToolResult(
726
+ data=data,
727
+ receipt=build_receipt(
728
+ contract=CONTRACT,
729
+ route=route,
730
+ fetch=table_fetch,
731
+ source_version=label,
732
+ transform_version=TRANSFORM_VERSION,
733
+ effective_from=date.fromisoformat(entry["effective_from"]),
734
+ effective_to=date.fromisoformat(entry["effective_to"]),
735
+ warnings=warnings,
736
+ non_claims=NON_CLAIMS,
737
+ ),
738
+ )
739
+
740
+
741
+ # ---------------------------------------------------------------------------
742
+ # Release-train observation (shared by codes.latest_release and canaries)
743
+ # ---------------------------------------------------------------------------
744
+
745
+ _CDC_YEAR_RE = re.compile(r"ICD10CM/(\d{4})/\"")
746
+ _CDC_APRIL_RE = re.compile(r"ICD10CM/(\d{4})-update/\"", re.IGNORECASE)
747
+ _CDC_APRIL_LONG_RE = re.compile(r"ICD10CM/April-1-(\d{4})-Update/\"", re.IGNORECASE)
748
+ _CMS_YEAR_RE = re.compile(r"/files/zip/(\d{4})-code-descriptions-tabular-order")
749
+ _CMS_APRIL_PREFIX_RE = re.compile(r"/files/zip/april-1-(\d{4})-code-descriptions")
750
+ _CMS_APRIL_SUFFIX_RE = re.compile(r"/files/zip/(\d{4})-code-descriptions-tabular-order-april")
751
+
752
+
753
+ def _parse_release_train(text: str) -> tuple[int, int | None]:
754
+ """Return (latest posted FY, latest FY with a posted April update).
755
+
756
+ Understands both the CDC IIS directory listing (authority) and the CMS
757
+ landing page (fallback). Raises ValueError when neither shape is present,
758
+ so a maintenance page can never be pinned as an observation.
759
+ """
760
+
761
+ def _years(matches: list[str]) -> set[int]:
762
+ return {int(m) for m in matches if 2000 <= int(m) <= 2100}
763
+
764
+ years = _years(_CDC_YEAR_RE.findall(text))
765
+ aprils = _years(_CDC_APRIL_RE.findall(text)) | _years(_CDC_APRIL_LONG_RE.findall(text))
766
+ if not years:
767
+ years = _years(_CMS_YEAR_RE.findall(text))
768
+ aprils = _years(_CMS_APRIL_PREFIX_RE.findall(text)) | _years(
769
+ _CMS_APRIL_SUFFIX_RE.findall(text)
770
+ )
771
+ if not years:
772
+ raise ValueError("no ICD-10-CM release-train links recognized in the listing")
773
+ return max(years), max(aprils) if aprils else None
774
+
775
+
776
+ def _fetch_release_train() -> tuple[FetchResult, int, int | None]:
777
+ result = fetch(
778
+ CDC_LISTING_URL, fallback_url=CMS_LANDING_URL, fallback_name="cms-landing-page"
779
+ )
780
+ try:
781
+ latest_fy, latest_april = _parse_release_train(
782
+ result.content.decode("utf-8", errors="replace")
783
+ )
784
+ except ValueError as exc:
785
+ raise SourceUnreachable(result.url, str(exc), status=result.status) from exc
786
+ return result, latest_fy, latest_april
787
+
788
+
789
+ def _probe_zip(label: str) -> bool:
790
+ """Range-probe a quarterly HCPCS zip: True posted, False absent (404 on
791
+ both slug variants). Anything else -- including non-ZIP bytes behind a
792
+ 2xx -- raises SourceUnreachable, because pinning it would lie."""
793
+ last_status: int | None = None
794
+ for plural in (False, True):
795
+ url = _hcpcs_quarter_url(label, plural=plural)
796
+ result = fetch(url, headers={"Range": "bytes=0-3"}, raise_for_status=False)
797
+ if result.status in (200, 206):
798
+ if result.content[:4] == ZIP_MAGIC:
799
+ return True
800
+ raise SourceUnreachable(
801
+ url,
802
+ "expected ZIP magic bytes ('PK') but got something else -- the asset "
803
+ "may have been replaced by an error page; do not re-pin automatically",
804
+ status=result.status,
805
+ )
806
+ if result.status != 404:
807
+ raise SourceUnreachable(url, "unexpected status probing quarterly", status=result.status)
808
+ last_status = result.status
809
+ assert last_status == 404
810
+ return False
811
+
812
+
813
+ def _latest_release(_: NoParams) -> ToolResult:
814
+ route = "codes.latest_release"
815
+ _, manifest = _load_manifest()
816
+ today = _today()
817
+ listing, latest_fy, latest_april = _fetch_release_train()
818
+
819
+ warnings: list[str] = []
820
+ resolution = resolve_cm_release(today, manifest)
821
+ if isinstance(resolution, CmUnresolved):
822
+ governing = None
823
+ effective_from = effective_to = None
824
+ warnings.append(f"{resolution.reason}: {resolution.detail}")
825
+ else:
826
+ governing = resolution.release
827
+ effective_from, effective_to = resolution.effective_from, resolution.effective_to
828
+
829
+ governing_fy = today.year + 1 if today.month >= 10 else today.year
830
+ staged: list[dict[str, Any]] = []
831
+ for fy in range(governing_fy + 1, latest_fy + 1):
832
+ starts = date(fy - 1, 10, 1)
833
+ release = f"fy{fy}"
834
+ staged.append(
835
+ {
836
+ "release": release,
837
+ "posted": True,
838
+ "effective_from": starts.isoformat(),
839
+ "vendored": release in manifest.get("cm_releases", {}),
840
+ "note": "posted upstream but not yet effective",
841
+ }
842
+ )
843
+ warnings.append(
844
+ f"FY{fy} ICD-10-CM files are posted upstream, effective {starts.isoformat()}; "
845
+ f"the pin will need updating before that date (run {REGENERATE}, then "
846
+ "`hc-source lock init`)."
847
+ )
848
+ if latest_april is not None and today < date(latest_april, 4, 1):
849
+ release = f"fy{latest_april}-april"
850
+ staged.append(
851
+ {
852
+ "release": release,
853
+ "posted": True,
854
+ "effective_from": date(latest_april, 4, 1).isoformat(),
855
+ "vendored": release in manifest.get("cm_releases", {}),
856
+ "note": "posted upstream but not yet effective",
857
+ }
858
+ )
859
+ warnings.append(
860
+ f"An April 1 {latest_april} ICD-10-CM update is posted upstream, effective "
861
+ f"{date(latest_april, 4, 1).isoformat()}; the pin will need updating before "
862
+ f"that date (run {REGENERATE}, then `hc-source lock init`)."
863
+ )
864
+
865
+ hcpcs_governing = _quarter_label(today)
866
+ hcpcs_next = _next_quarter(hcpcs_governing)
867
+ next_posted = _probe_zip(hcpcs_next)
868
+ if next_posted:
869
+ starts = _quarter_start(hcpcs_next)
870
+ warnings.append(
871
+ f"The {hcpcs_next} HCPCS Level II quarterly is posted upstream, effective "
872
+ f"{starts.isoformat()}; the pin will need updating before that date (run "
873
+ f"{REGENERATE}, then `hc-source lock init`)."
874
+ )
875
+
876
+ data = {
877
+ "as_of": today.isoformat(),
878
+ "icd10cm": {
879
+ "governing_release": governing,
880
+ "vendored": governing in manifest.get("cm_releases", {}) if governing else False,
881
+ "upstream_latest_posted_fy": latest_fy,
882
+ "upstream_latest_april_update_fy": latest_april,
883
+ "staged_releases": staged,
884
+ },
885
+ "hcpcs_level_ii": {
886
+ "governing_quarter": hcpcs_governing,
887
+ "vendored": hcpcs_governing in manifest.get("hcpcs_releases", {}),
888
+ "next_quarter": hcpcs_next,
889
+ "next_quarter_posted": next_posted,
890
+ },
891
+ }
892
+ return ToolResult(
893
+ data=data,
894
+ receipt=build_receipt(
895
+ contract=CONTRACT,
896
+ route=route,
897
+ fetch=listing,
898
+ source_version=governing or "unresolved",
899
+ transform_version=TRANSFORM_VERSION,
900
+ effective_from=effective_from,
901
+ effective_to=effective_to,
902
+ warnings=warnings,
903
+ non_claims=NON_CLAIMS,
904
+ ),
905
+ )
906
+
907
+
908
+ # ---------------------------------------------------------------------------
909
+ # Canaries
910
+ # ---------------------------------------------------------------------------
911
+
912
+
913
+ def _observe_cm_governing() -> CanaryObservation:
914
+ _, manifest = _load_manifest()
915
+ resolution = resolve_cm_release(_today(), manifest)
916
+ if isinstance(resolution, CmUnresolved):
917
+ return CanaryObservation(
918
+ value=f"unresolved:{resolution.reason}",
919
+ note="no vendored ICD-10-CM release provably governs today's date",
920
+ )
921
+ return CanaryObservation(
922
+ value=resolution.release,
923
+ note="ICD-10-CM release governing today's date, from the vendored manifest",
924
+ )
925
+
926
+
927
+ def _cm_governing_remediation(status: CanaryStatus, observed, expected) -> str:
928
+ if status is CanaryStatus.UNREACHABLE:
929
+ return (
930
+ "The vendored manifest could not be read. Check HC_SOURCE_CODES_DATA if set, "
931
+ f"otherwise restore hc_source/data/codes ({REGENERATE}), then re-run "
932
+ "`hc-source doctor`."
933
+ )
934
+ return (
935
+ f"The ICD-10-CM release governing today's date is {observed}, but source-lock.json "
936
+ f"pins {expected}: the pin is no longer correct for today. Update the vendored "
937
+ f"release train in hc_source/data/codes/regenerate.py (add the new FY or "
938
+ f"April release to CM_RELEASES and settle CM_APRIL_STATUS), run it "
939
+ f"({REGENERATE}), then re-pin with `hc-source lock init`."
940
+ )
941
+
942
+
943
+ def _observe_hcpcs_governing() -> CanaryObservation:
944
+ _, manifest = _load_manifest()
945
+ label = _quarter_label(_today())
946
+ vendored = label in manifest.get("hcpcs_releases", {})
947
+ return CanaryObservation(
948
+ value=label if vendored else f"{label}:unvendored",
949
+ note="HCPCS quarterly governing today's date, from the vendored manifest",
950
+ )
951
+
952
+
953
+ def _hcpcs_governing_remediation(status: CanaryStatus, observed, expected) -> str:
954
+ if status is CanaryStatus.UNREACHABLE:
955
+ return (
956
+ "The vendored manifest could not be read. Check HC_SOURCE_CODES_DATA if set, "
957
+ f"otherwise restore hc_source/data/codes ({REGENERATE}), then re-run "
958
+ "`hc-source doctor`."
959
+ )
960
+ return (
961
+ f"The HCPCS quarterly governing today's date is {observed}, but source-lock.json "
962
+ f"pins {expected}: the pin is no longer correct for today. Add the quarter to "
963
+ f"HCPCS_RELEASES in hc_source/data/codes/regenerate.py, run it "
964
+ f"({REGENERATE}), then re-pin with `hc-source lock init`."
965
+ )
966
+
967
+
968
+ def _observe_vendored_tables() -> CanaryObservation:
969
+ _, manifest = _load_manifest()
970
+ tables = sorted(
971
+ {
972
+ entry["table"]
973
+ for section in ("cm_releases", "hcpcs_releases")
974
+ for entry in manifest.get(section, {}).values()
975
+ }
976
+ )
977
+ row_lines: list[str] = []
978
+ header_lines: list[str] = []
979
+ for name in tables:
980
+ path = _data_dir() / name
981
+ try:
982
+ raw = gzip.decompress(path.read_bytes())
983
+ except (OSError, gzip.BadGzipFile, EOFError) as exc:
984
+ raise SourceUnreachable(path.as_uri(), type(exc).__name__) from exc
985
+ header, _, body = raw.decode("utf-8").partition("\n")
986
+ row_lines.append(f"{name}:{hashlib.sha256(body.encode('utf-8')).hexdigest()}")
987
+ header_lines.append(f"{name}:{header}")
988
+ return CanaryObservation(
989
+ value=hashlib.sha256("\n".join(row_lines).encode("utf-8")).hexdigest(),
990
+ schema_hash=hashlib.sha256("\n".join(header_lines).encode("utf-8")).hexdigest(),
991
+ note=f"canonical content digest of {len(tables)} vendored derived tables",
992
+ )
993
+
994
+
995
+ def _vendored_tables_remediation(status: CanaryStatus, observed, expected) -> str:
996
+ if status is CanaryStatus.SCHEMA_CHANGED:
997
+ return (
998
+ "A vendored derived table changed its column set without changing its rows. "
999
+ "If the derivation changed on purpose, bump TRANSFORM_VERSION in "
1000
+ "hc_source/adapters/codes.py, review the manifest.json diff, and re-pin with "
1001
+ "`hc-source lock init`; otherwise restore the table from git."
1002
+ )
1003
+ return (
1004
+ "The vendored derived tables changed on disk without a re-pin. If you just "
1005
+ f"regenerated them ({REGENERATE}), review the manifest.json diff (canonical "
1006
+ "hashes and row counts) and re-pin with `hc-source lock init`; otherwise "
1007
+ "restore hc_source/data/codes from git -- the answers no longer match "
1008
+ "what the lockfile attests."
1009
+ )
1010
+
1011
+
1012
+ def _cm_train_observation(
1013
+ result: FetchResult, latest_fy: int, latest_april: int | None
1014
+ ) -> CanaryObservation:
1015
+ """Build the observation, carrying WHICH source answered.
1016
+
1017
+ The CDC directory listing is the authority; the CMS landing page is a
1018
+ mirror. Round-1 finding H2: this canary reported `ok` while ftp.cdc.gov was
1019
+ dark, because the observation had nowhere to record that a mirror had
1020
+ answered. The transport tracked it correctly the whole time.
1021
+ """
1022
+ april = f"fy{latest_april}" if latest_april is not None else "none"
1023
+ return CanaryObservation(
1024
+ value=f"latest_posted=fy{latest_fy};latest_april={april}",
1025
+ upstream_status=result.status,
1026
+ note="latest ICD-10-CM FY and April update posted upstream (posted != effective)",
1027
+ fallback_used=result.fallback_used,
1028
+ fallback_name=result.fallback_name,
1029
+ )
1030
+
1031
+
1032
+ def _observe_cm_train() -> CanaryObservation:
1033
+ return _cm_train_observation(*_fetch_release_train())
1034
+
1035
+
1036
+ def _cm_train_remediation(status: CanaryStatus, observed, expected) -> str:
1037
+ if status is CanaryStatus.UNREACHABLE:
1038
+ return (
1039
+ "Neither the CDC ICD-10-CM directory listing nor the CMS landing page could "
1040
+ "be read or recognized. Retry in an hour; if it persists, check "
1041
+ f"{CDC_LISTING_URL} and {CMS_LANDING_URL} in a browser before touching the "
1042
+ "pin."
1043
+ )
1044
+ return (
1045
+ f"The ICD-10-CM release train moved upstream: observed {observed}, pinned "
1046
+ f"{expected}. A newly posted FY file set takes effect on October 1 and a newly "
1047
+ "posted April update on April 1 -- posted is not effective, so this is staged, "
1048
+ "not an emergency. Before the effective date: add the new release to "
1049
+ "CM_RELEASES (and settle CM_APRIL_STATUS) in "
1050
+ f"hc_source/data/codes/regenerate.py, run it ({REGENERATE}), then re-pin "
1051
+ "with `hc-source lock init`."
1052
+ )
1053
+
1054
+
1055
+ def _observe_hcpcs_train() -> CanaryObservation:
1056
+ today = _today()
1057
+ current = _quarter_label(today)
1058
+ upcoming = _next_quarter(current)
1059
+ current_state = "posted" if _probe_zip(current) else "absent"
1060
+ next_state = "posted" if _probe_zip(upcoming) else "absent"
1061
+ return CanaryObservation(
1062
+ value=f"current={current}:{current_state};next={upcoming}:{next_state}",
1063
+ note="HCPCS quarterly slugs posted upstream (range-probe; posted != effective)",
1064
+ )
1065
+
1066
+
1067
+ def _hcpcs_train_remediation(status: CanaryStatus, observed, expected) -> str:
1068
+ if status is CanaryStatus.UNREACHABLE:
1069
+ return (
1070
+ "A HCPCS quarterly probe failed or returned non-ZIP bytes. Do not re-pin "
1071
+ f"automatically: retry in an hour, then re-verify via {HCPCS_INDEX_URL} "
1072
+ "before re-locking."
1073
+ )
1074
+ posted = re.search(r"next=(\d{4}q\d):posted", observed or "")
1075
+ staged_line = ""
1076
+ if posted:
1077
+ starts = _quarter_start(posted.group(1))
1078
+ staged_line = (
1079
+ f" The {posted.group(1)} file takes effect {starts.isoformat()} -- staged, "
1080
+ "not an emergency; vendor it before that date."
1081
+ )
1082
+ return (
1083
+ f"The HCPCS quarterly train moved: observed {observed}, pinned {expected}."
1084
+ f"{staged_line} Add the quarter to HCPCS_RELEASES in "
1085
+ f"hc_source/data/codes/regenerate.py, run it ({REGENERATE}), then re-pin "
1086
+ "with `hc-source lock init`. If the current quarter went absent, CMS renamed "
1087
+ f"the slug: re-scrape {HCPCS_INDEX_URL}."
1088
+ )
1089
+
1090
+
1091
+ # ---------------------------------------------------------------------------
1092
+ # Adapter
1093
+ # ---------------------------------------------------------------------------
1094
+
1095
+
1096
+ class CodesAdapter(SourceAdapter):
1097
+ source_id = SOURCE_ID
1098
+ contract = CONTRACT
1099
+
1100
+ # Mixed: the vendored-table canaries never touch the network, the release
1101
+ # trains do. Retries only ever fire on a transient SourceUnreachable, so
1102
+ # declaring them at the adapter costs the offline canaries nothing.
1103
+ canary_retries = 2
1104
+
1105
+ def canaries(self) -> list[Canary]:
1106
+ return [
1107
+ Canary(
1108
+ canary_id="codes.cm_governing_release",
1109
+ source_id=SOURCE_ID,
1110
+ description=(
1111
+ "ICD-10-CM release governing today's date (clock + vendored "
1112
+ "manifest; drifts exactly when the pin stops being correct for "
1113
+ "today, e.g. on October 1)."
1114
+ ),
1115
+ observe=_observe_cm_governing,
1116
+ remediation=(
1117
+ "The ICD-10-CM release governing today is no longer the pinned one. "
1118
+ f"Re-vendor ({REGENERATE}) and re-pin with `hc-source lock init`."
1119
+ ),
1120
+ remediation_for=_cm_governing_remediation,
1121
+ ),
1122
+ Canary(
1123
+ canary_id="codes.hcpcs_governing_quarter",
1124
+ source_id=SOURCE_ID,
1125
+ description=(
1126
+ "HCPCS Level II quarterly governing today's date (clock + vendored "
1127
+ "manifest)."
1128
+ ),
1129
+ observe=_observe_hcpcs_governing,
1130
+ remediation=(
1131
+ "The HCPCS quarterly governing today is no longer the pinned one. "
1132
+ f"Re-vendor ({REGENERATE}) and re-pin with `hc-source lock init`."
1133
+ ),
1134
+ remediation_for=_hcpcs_governing_remediation,
1135
+ ),
1136
+ Canary(
1137
+ canary_id="codes.vendored_tables",
1138
+ source_id=SOURCE_ID,
1139
+ description=(
1140
+ "Canonical content digest (seed hashes) of the vendored derived "
1141
+ "tables; schema hash covers their column sets."
1142
+ ),
1143
+ observe=_observe_vendored_tables,
1144
+ remediation=(
1145
+ "The vendored derived tables changed on disk. Review the diff and "
1146
+ "re-pin with `hc-source lock init`, or restore them from git."
1147
+ ),
1148
+ remediation_for=_vendored_tables_remediation,
1149
+ ),
1150
+ Canary(
1151
+ canary_id="codes.cm_release_train",
1152
+ source_id=SOURCE_ID,
1153
+ description=(
1154
+ "Latest ICD-10-CM FY and April update posted upstream (CDC listing, "
1155
+ "CMS landing-page fallback). Future-effective postings are pinned as "
1156
+ "observed, so a staged release never alarms by itself."
1157
+ ),
1158
+ observe=_observe_cm_train,
1159
+ remediation=(
1160
+ "The ICD-10-CM release train moved upstream. Posted is not "
1161
+ f"effective: vendor the new release ({REGENERATE}) and re-pin with "
1162
+ "`hc-source lock init` before its effective date."
1163
+ ),
1164
+ remediation_for=_cm_train_remediation,
1165
+ ),
1166
+ Canary(
1167
+ canary_id="codes.hcpcs_release_train",
1168
+ source_id=SOURCE_ID,
1169
+ description=(
1170
+ "HCPCS quarterly slugs posted upstream (4-byte range probes of the "
1171
+ "current and next quarter, both slug variants)."
1172
+ ),
1173
+ observe=_observe_hcpcs_train,
1174
+ remediation=(
1175
+ "The HCPCS quarterly train moved upstream. Vendor the new quarter "
1176
+ f"({REGENERATE}) and re-pin with `hc-source lock init` before it "
1177
+ "takes effect."
1178
+ ),
1179
+ remediation_for=_hcpcs_train_remediation,
1180
+ ),
1181
+ ]
1182
+
1183
+ def tools(self) -> list[ToolSpec]:
1184
+ return [
1185
+ ToolSpec(
1186
+ name="codes.valid_on",
1187
+ description=(
1188
+ "Answer whether an ICD-10-CM code is valid/billable on a date of "
1189
+ "service, from the vendored release governing that date; fails "
1190
+ "closed when no vendored release provably governs it."
1191
+ ),
1192
+ params_model=ValidOnParams,
1193
+ handler=_valid_on,
1194
+ tags=("validity", "icd10cm"),
1195
+ ),
1196
+ ToolSpec(
1197
+ name="codes.lookup",
1198
+ description=(
1199
+ "Look up an ICD-10-CM code's description with full release "
1200
+ "provenance, from the release governing the given (or today's) "
1201
+ "date of service."
1202
+ ),
1203
+ params_model=LookupParams,
1204
+ handler=_lookup,
1205
+ tags=("lookup", "icd10cm"),
1206
+ ),
1207
+ ToolSpec(
1208
+ name="codes.hcpcs_lookup",
1209
+ description=(
1210
+ "Look up a HCPCS Level II code or modifier in the pinned quarterly "
1211
+ "file; with a date of service, adds a validity verdict from the "
1212
+ "record's code-added/termination interval."
1213
+ ),
1214
+ params_model=HcpcsLookupParams,
1215
+ handler=_hcpcs_lookup,
1216
+ tags=("lookup", "hcpcs"),
1217
+ ),
1218
+ ToolSpec(
1219
+ name="codes.latest_release",
1220
+ description=(
1221
+ "Detect the current and staged ICD-10-CM/HCPCS releases: what "
1222
+ "governs today (from the vendored manifest) and what is posted "
1223
+ "upstream (live CDC/CMS probe), with staged-release warnings."
1224
+ ),
1225
+ params_model=NoParams,
1226
+ handler=_latest_release,
1227
+ tags=("metadata", "release"),
1228
+ ),
1229
+ ]
1230
+
1231
+
1232
+ ADAPTER = CodesAdapter()