oddb2xml 3.0.31 → 3.0.32

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e8ae3ea587547caf78cba9d778e08e6474535800d111cf484f981f5eb86cc7c2
4
- data.tar.gz: cf33dfe5d3b549fa1c2fc1542cc0f7754e369a8d50c757705c869ba119b245b5
3
+ metadata.gz: 6526c05c2efd31be9fff9279dd24e817a3ae94363c9b30d8286dbef0510596d5
4
+ data.tar.gz: c8c835826c01e71719efbef0c73cb0f4805037dac8a7d89dcd82a7d58df41deb
5
5
  SHA512:
6
- metadata.gz: b49924c39835eac2d61782b95edbd7564f704ca59fdcf0404f04f8b58f4bf09390d41e2676a974a4803a10ce7c8751bbc96177f6e94ff94cda49277b2e946426
7
- data.tar.gz: 248e0058044934b2986855523747640fa8c1cc4652a039d7b0427fbcf6e29358fa5fec55a42043042d49fc7368ccc2c7cdbc20ed4224259e586cc023db5fd5fd
6
+ metadata.gz: 8358ad2fd07b3cad7d4e87c7b364fa8ea0a3f68742c547d9781dbf61614c6a2d85b3f95bd5ee8f3c7d0aa48bb08b8e2b45eb279382778e25a5333f6928440fed
7
+ data.tar.gz: e332eb5a017afd15dc78663d12f2956469ec2ed4e9de4742b28771f84dd10983593787c2a611327c130202ed82f34c56c730b75c24e9eb4ad49912cdb3cb5519
data/CLAUDE.md CHANGED
@@ -22,7 +22,7 @@ The system follows a **download → extract → build → compress** pipeline:
22
22
 
23
23
  6. **Compressor** (`lib/oddb2xml/compressor.rb`) — Optional ZIP/TAR.GZ output compression.
24
24
 
25
- 7. **FHIR support** (`lib/oddb2xml/fhir_support.rb`) — Self-contained module providing `FhirDownloader` and FHIR NDJSON parsing. Activated via `--fhir` (or `--fhir-url=<URL>`). Downloads per-language NDJSON files (`foph-sl-export-latest-{de,fr,it}.ndjson`) from `epl.bag.admin.ch` to populate French and Italian product names/descriptions. Maps legal status codes `756005022007` and `756005022008` to Swissmedic category D. Reads the BAG **Indikationscode** (`XXXXX.NN`) from the explicit `indicationCode` extension on each `RegulatedAuthorization.indication[].extension[regulatedAuthorization-limitation]` (BAG SL FHIR export >= v2.0.5; handled from 3.0.10). The BAG changelog states the limitation code (`ClinicalUseDefinition.id`) and the indication code are **independent** fields, so the older derivation — combining each indication CUD's `.NN` id-suffix with the reimbursement RA's `FOPHDossierNumber` — is kept only as a fallback for feeds lacking the extension. Exposed as `item[:indication_codes]` and per-package `:indication_codes` (each entry a `{code:, cud_id:, text:}` hash, where `cud_id` is the `limitationIndication` CUD reference used to resolve the text). From 3.0.7 onwards, `Builder#build_product` emits one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN">limitation text</INDICATION_CODE>` child per indication on every `<PRD>` in `oddb_product.xml`; live feed numbers: 539 products / 1,293 codes / 100 % with non-empty indication text. Mandatory on prescriptions/invoices for SL price-model drugs from 2026-07-01 — see issue [#113](https://github.com/zdavatz/oddb2xml/issues/113). **Limitation texts** (3.0.8 onwards): the `regulatedAuthorization-limitation` extension has no inline `limitationText` in the live BAG feed — it carries a `limitationIndication` reference to a `ClinicalUseDefinition` whose `indication.diseaseSymptomProcedure.concept.text` is the actual text. The parser stores the ref as `cud_ref` on each Limitation, `Bundle#cud_text_by_id` resolves DE, and `merge_language` propagates FR/IT from the per-language NDJSON files via the same CUD id. Coverage on the live feed jumped from 0 / 9'108 to 9'108 / 9'108 (issue [#116](https://github.com/zdavatz/oddb2xml/issues/116)). **Limitation code / LIMNAMEBAG** (3.0.12 onwards): FHIR has no native BAG limitation code (LIMCD), so `create_limitations_for_package` sets `LimitationCode = cud_ref` (the `limitationIndication` CUD id) instead of `""`. Without this, every FHIR limitation shared an empty `:code`; `Builder#build_artikelstamm` groups its `<LIMITATIONS>` section by code, so all of them collapsed into a single `<LIMITATION>` with an empty `<LIMNAMEBAG>` and only one text survived. Using the CUD id as the key makes each distinct limitation emit and be referenced from its `<PRODUCT>`. The downstream `bin/check_artikelstamm` (`semantic_check.rb`) also crashed on the lone-element output because Ox `:hash_no_attrs` collapses a one-child section into a Hash (and an empty one into nil) — `SemanticCheckXML#get_items` now normalises every section to an Array. **v6 Artikelstamm / per-article INDC (3.0.26 onwards):** `--artikelstamm` now emits the **Elexis Artikelstamm v6** format (namespace `http://elexis.ch/Elexis_Artikelstamm_v6`, file `artikelstamm_DDMMYYYY_v6.xml`/`.csv`, validated against the bundled `Elexis_Artikelstamm_v6.xsd`) — replacing v5. The new piece is a per-`<ITEM>` `<ARTSL>` block carrying the BAG Indikationscodes (issue [#113](https://github.com/zdavatz/oddb2xml/issues/113)): `<PM>true</PM>` plus one `<ARTLIM>` per limitation with `<LIMCD>` (= `cud_ref`, the BAG limitation code), `<INDCD>` (the `XXXXX.NN` indication code from the `indicationCode` extension), and `<VDAT>`/`<VTDAT>` (period start/end). To feed it, `create_limitations_for_package` now also carries `IndicationCode` (→ per-package `:indcd`) and `ValidThruDate` (→ `:vtdate`) on each limitation; `Builder#append_artsl`/`elexis_datetime` emit one `<ARTLIM>` per limitation that has a non-empty `:indcd` (so non-price-model items get no `<ARTSL>`). `PM` is always `true` here because the indication code is required only for SL price-model drugs, which is exactly the set of items that reach this block. The bundled `Elexis_Artikelstamm_v6.xsd` is the canonical MEDEVIT schema extended with oddb2xml's historical Italian elements (`DSCRI` on PRODUCT/LIMITATION/ITEM, `DOSAGE_FORMI` on ITEM) so the output still validates. The legacy `--no-fhir` path emits no `<ARTSL>` (no FHIR limitations). **Per-article INDC in the `-e`/`-b` feeds (3.0.27 onwards):** the same indication codes are also emitted per `<ART>` in `oddb_article.xml` via `Builder#append_indication_codes` — one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN" limcd="DRUG.NN" vdat=… vtdat=…>limitation text</INDICATION_CODE>` per limitation that carries an `:indcd` (sourced from the article's `pac[:limitations]`). This reuses the flat `<INDICATION_CODE>` element already emitted on `<PRD>` in `oddb_product.xml` (since 3.0.7), enriched with the limitation code and validity dates so it carries the full `<ARTSL>` payload. The bundled `oddb2xml.xsd` now defines `<INDICATION_CODE>` (simpleContent + `code`/`cud_id`/`limcd`/`vdat`/`vtdat` attributes) and references it from both `<ART>` and `<PRD>` — the `<PRD>` reference had been missing since the 3.0.7 addition, so the FHIR product feed only validated once this was added. **Legacy v5 opt-in (3.0.28 onwards):** `--artikelstamm-v5` additionally emits the older Artikelstamm v5 file alongside the v6 one (for consumers not yet migrated). It implies `--artikelstamm`; `Options.parse` sets `@opts[:artikelstamm] = true`. `build_artikelstamm` takes a `version:` keyword (default `6`) that drives the namespace (`Elexis_Artikelstamm_v#{version}`) and CSV/XML filename suffix; `build_artikelstamm_v5` calls it with `version: 5`. The only content difference is that the v5 path skips `append_artsl` (the v5 schema has no `<ARTSL>` element). The CLI `files` map adds an `:artikelstamm_v5` entry (→ `build_artikelstamm_v5` via the `to_xml` subject dispatch) and validates it against the bundled `Elexis_Artikelstamm_v5.xsd`. Both files come from one build — the `@prepared` guard means the shared download/prepare phase runs once. **Selbstbehalt / costShare (3.0.31 onwards):** the `reimbursementSL` extension's `costShare` valueInteger is the BAG Selbstbehalt percentage; it maps to the legacy `FlagSB` semantics of the old BAG `Preparations.xml` — `40` → `item[:deductible] = "Y"` (raised Selbstbehalt: `<SLOPLUS>1</SLOPLUS>` in `oddb_article.xml`, `<DEDUCTIBLE>40</DEDUCTIBLE>` in the Artikelstamm), `10` → `"N"` (ordinary Selbstbehalt: `SLOPLUS` 2 / `DEDUCTIBLE` 10), `20` → `deductible20 = "Y"` (transitional rate, issue #81). Until 3.0.31 the mapping was inverted (`10` → `"Y"`), so **every** SL article claimed a 40 % Selbstbehalt. On the live feed only 385 of 10'388 packages (222 preparations) carry `costShare` 40; the value never mixes within one preparation, so keeping the flag at sequence level is safe.
25
+ 7. **FHIR support** (`lib/oddb2xml/fhir_support.rb`) — Self-contained module providing `FhirDownloader` and FHIR NDJSON parsing. Activated via `--fhir` (or `--fhir-url=<URL>`). Downloads per-language NDJSON files (`foph-sl-publication-latest-{de,fr,it}.ndjson`) from `epl.bag.admin.ch/static/sl/publication/fhir` (BAG moved the export there on 24.08.2026; the old `/static/fhir/foph-sl-export-*` answers 404 for `-latest-` while the old dated snapshots remain, so the failure looks like an import that does nothing) to populate French and Italian product names/descriptions. Maps legal status codes `756005022007` and `756005022008` to Swissmedic category D. Reads the BAG **Indikationscode** (`XXXXX.NN`) from the explicit `indicationCode` extension on each `RegulatedAuthorization.indication[].extension[regulatedAuthorization-limitation]` (BAG SL FHIR export >= v2.0.5; handled from 3.0.10). The BAG changelog states the limitation code (`ClinicalUseDefinition.id`) and the indication code are **independent** fields, so the older derivation — combining each indication CUD's `.NN` id-suffix with the reimbursement RA's `FOPHDossierNumber` — is kept only as a fallback for feeds lacking the extension. Exposed as `item[:indication_codes]` and per-package `:indication_codes` (each entry a `{code:, cud_id:, text:}` hash, where `cud_id` is the `limitationIndication` CUD reference used to resolve the text). From 3.0.7 onwards, `Builder#build_product` emits one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN">limitation text</INDICATION_CODE>` child per indication on every `<PRD>` in `oddb_product.xml`; live feed numbers: 539 products / 1,293 codes / 100 % with non-empty indication text. Mandatory on prescriptions/invoices for SL price-model drugs from 2026-07-01 — see issue [#113](https://github.com/zdavatz/oddb2xml/issues/113). **Limitation texts** (3.0.8 onwards): the `regulatedAuthorization-limitation` extension has no inline `limitationText` in the live BAG feed — it carries a `limitationIndication` reference to a `ClinicalUseDefinition` whose `indication.diseaseSymptomProcedure.concept.text` is the actual text. The parser stores the ref as `cud_ref` on each Limitation, `Bundle#cud_text_by_id` resolves DE, and `merge_language` propagates FR/IT from the per-language NDJSON files via the same CUD id. Coverage on the live feed jumped from 0 / 9'108 to 9'108 / 9'108 (issue [#116](https://github.com/zdavatz/oddb2xml/issues/116)). **Limitation code / LIMNAMEBAG** (3.0.12 onwards): FHIR has no native BAG limitation code (LIMCD), so `create_limitations_for_package` sets `LimitationCode = cud_ref` (the `limitationIndication` CUD id) instead of `""`. Without this, every FHIR limitation shared an empty `:code`; `Builder#build_artikelstamm` groups its `<LIMITATIONS>` section by code, so all of them collapsed into a single `<LIMITATION>` with an empty `<LIMNAMEBAG>` and only one text survived. Using the CUD id as the key makes each distinct limitation emit and be referenced from its `<PRODUCT>`. The downstream `bin/check_artikelstamm` (`semantic_check.rb`) also crashed on the lone-element output because Ox `:hash_no_attrs` collapses a one-child section into a Hash (and an empty one into nil) — `SemanticCheckXML#get_items` now normalises every section to an Array. **v6 Artikelstamm / per-article INDC (3.0.26 onwards):** `--artikelstamm` now emits the **Elexis Artikelstamm v6** format (namespace `http://elexis.ch/Elexis_Artikelstamm_v6`, file `artikelstamm_DDMMYYYY_v6.xml`/`.csv`, validated against the bundled `Elexis_Artikelstamm_v6.xsd`) — replacing v5. The new piece is a per-`<ITEM>` `<ARTSL>` block carrying the BAG Indikationscodes (issue [#113](https://github.com/zdavatz/oddb2xml/issues/113)): `<PM>true</PM>` plus one `<ARTLIM>` per limitation with `<LIMCD>` (= `cud_ref`, the BAG limitation code), `<INDCD>` (the `XXXXX.NN` indication code from the `indicationCode` extension), and `<VDAT>`/`<VTDAT>` (period start/end). To feed it, `create_limitations_for_package` now also carries `IndicationCode` (→ per-package `:indcd`) and `ValidThruDate` (→ `:vtdate`) on each limitation; `Builder#append_artsl`/`elexis_datetime` emit one `<ARTLIM>` per limitation that has a non-empty `:indcd` (so non-price-model items get no `<ARTSL>`). `PM` is always `true` here because the indication code is required only for SL price-model drugs, which is exactly the set of items that reach this block. The bundled `Elexis_Artikelstamm_v6.xsd` is the canonical MEDEVIT schema extended with oddb2xml's historical Italian elements (`DSCRI` on PRODUCT/LIMITATION/ITEM, `DOSAGE_FORMI` on ITEM) so the output still validates. The legacy `--no-fhir` path emits no `<ARTSL>` (no FHIR limitations). **Per-article INDC in the `-e`/`-b` feeds (3.0.27 onwards):** the same indication codes are also emitted per `<ART>` in `oddb_article.xml` via `Builder#append_indication_codes` — one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN" limcd="DRUG.NN" vdat=… vtdat=…>limitation text</INDICATION_CODE>` per limitation that carries an `:indcd` (sourced from the article's `pac[:limitations]`). This reuses the flat `<INDICATION_CODE>` element already emitted on `<PRD>` in `oddb_product.xml` (since 3.0.7), enriched with the limitation code and validity dates so it carries the full `<ARTSL>` payload. The bundled `oddb2xml.xsd` now defines `<INDICATION_CODE>` (simpleContent + `code`/`cud_id`/`limcd`/`vdat`/`vtdat` attributes) and references it from both `<ART>` and `<PRD>` — the `<PRD>` reference had been missing since the 3.0.7 addition, so the FHIR product feed only validated once this was added. **Legacy v5 opt-in (3.0.28 onwards):** `--artikelstamm-v5` additionally emits the older Artikelstamm v5 file alongside the v6 one (for consumers not yet migrated). It implies `--artikelstamm`; `Options.parse` sets `@opts[:artikelstamm] = true`. `build_artikelstamm` takes a `version:` keyword (default `6`) that drives the namespace (`Elexis_Artikelstamm_v#{version}`) and CSV/XML filename suffix; `build_artikelstamm_v5` calls it with `version: 5`. The only content difference is that the v5 path skips `append_artsl` (the v5 schema has no `<ARTSL>` element). The CLI `files` map adds an `:artikelstamm_v5` entry (→ `build_artikelstamm_v5` via the `to_xml` subject dispatch) and validates it against the bundled `Elexis_Artikelstamm_v5.xsd`. Both files come from one build — the `@prepared` guard means the shared download/prepare phase runs once. **Selbstbehalt / costShare (3.0.31 onwards):** the `reimbursementSL` extension's `costShare` valueInteger is the BAG Selbstbehalt percentage; it maps to the legacy `FlagSB` semantics of the old BAG `Preparations.xml` — `40` → `item[:deductible] = "Y"` (raised Selbstbehalt: `<SLOPLUS>1</SLOPLUS>` in `oddb_article.xml`, `<DEDUCTIBLE>40</DEDUCTIBLE>` in the Artikelstamm), `10` → `"N"` (ordinary Selbstbehalt: `SLOPLUS` 2 / `DEDUCTIBLE` 10), `20` → `deductible20 = "Y"` (transitional rate, issue #81). Until 3.0.31 the mapping was inverted (`10` → `"Y"`), so **every** SL article claimed a 40 % Selbstbehalt. On the live feed only 385 of 10'388 packages (222 preparations) carry `costShare` 40; the value never mixes within one preparation, so keeping the flag at sequence level is safe.
26
26
 
27
27
  8. **Refdata cleanup** (`lib/oddb2xml/refdata_cleanup.rb`) — Compensates for known data-quality issues in upstream Refdata.Articles.xml before they reach the output. Each fix is guarded by a Swissmedic-side heuristic (e.g. comma in `substance_swissmedic` to distinguish mono products from real combinations). Currently fixes (a) the doubled-dose template bug (`X mg / X mg / Stk`, `fix_double_dose`, guarded by `single_substance?`); (b) the spelled-out German galenic form `Retardtabletten` → house-style abbreviation `Ret Tabl` (`normalize_galenic_form` / `GALENIC_NORMALISATIONS`, issue #112 case #13, e.g. RINVOQ — a narrow word-boundary substitution that leaves legitimate brand suffixes like `TRAMAL retard` and Mepha's `Lactab` untouched); and (c) dose info Refdata dropped from `<FullName>`, sourced from the Swissmedic composition string `pack[:composition_swissmedic]` — `fix_missing_combo_dose` (#6, appends a combination's 2nd component strength), `fix_missing_dose` (#4, inserts a mono product's missing strength before the pack count), `fix_missing_volume` (#7, appends an injectable's per-pen volume); and (d) 50-char-truncation repairs — `fix_truncated_metoject` (#1, rebuilds METOJECT Autoinjektor names from the intact `<brand> Autoinjektor <dose>/<vol>` prefix + Swissmedic `size`, localised DE/FR/IT) and `fix_truncated_volume_unit` (#3, restores the cut `ml` of the VERACTIV Vitamin D3 drops). The (c) and (d) fixes are scoped to explicit IKSNR allow-lists (`COMBO_DOSE_IKSNR`/`MISSING_DOSE_IKSNR`/`MISSING_VOLUME_IKSNR`/`METOJECT_IKSNR`/`VERACTIV_VITD3_IKSNR`): a dry run proved a blanket heuristic mis-fires on hundreds of legitimate names (sodium counter-ion doses, strength-less phyto/powder products, concentration names like `CIMZIA 200 mg/ml`), so only catalogued registrations are touched — add an IKSNR to grow coverage. Called from `Builder#apply_refdata_description_cleanups!` at the start of `prepare_articles`. See GitHub issue #112 for the catalogue.
28
28
 
@@ -45,12 +45,16 @@ YAML files in `data/` provide manual overrides and mappings: `article_overrides.
45
45
 
46
46
  These scripts run the public download server at `https://mediupdatexml.oddb.org` (Apache on this host) and are **not** part of the gem itself.
47
47
 
48
+ - **`get_transfer.sh`** (cron: `30 0 * * * zdavatz`, restored 2026-08-02) — mirrors the ZurRose IGM11 **Vollstamm** article master to `$DATA_DIR/TRANSFER.ZIP` (`/home/zdavatz/software/get_transfer`, the `$GET_TRANSFER_ZIP` that `run_oddb2xml.sh` seeds into `downloads/`). Downloads `transfer.dat`, validates it, and zips it with the entry named **lowercase `transfer.dat`** — `ZurroseDownloader` (`downloader.rb:226`) unzips into `downloads/` and opens that exact name. **Two things had gone wrong at once.** (a) The script lived *only* in `/home/zdavatz/software/get_transfer` on the host HIN deleted, so it vanished with the box and the cron line — guarded by `[ -x … ]` — went **silently** dead; it is now tracked in `scripts/` with that path a symlink, so `setup_new_server.sh` restores it. (b) Its source URL died independently around mid-July 2026: the old `zurrose.com/fileadmin/main/lib/download.php?file=…` now 301s to the marketing homepage. Current URL is `https://www.zurrose.ch/sites/default/files/media/downloads/medi/produktupdate/igm11/de/transfer.dat` (override with `TRANSFER_URL`). Consequence of both: `pillbox.oddb.org/TRANSFER.ZIP` — which is only a mirror *of this job* — froze at its 2026-07-16 copy. **Validation** is a record-shape check, not a size check: the file is fixed-width IGM11, every record exactly **98** bytes (179'856 records on the 2026-08-02 file), and fewer than `MIN_LINES` (100'000) records or any off-length record is rejected. A ZurRose error page passes a size check and fails this one. **Last-good semantics:** the download goes to a scratch dir and is published by `mv` only after validation, so a 404 or an HTML page leaves the previous `TRANSFER.ZIP` untouched — the 01:00 build reads that zip unconditionally, and half a file there aborts the whole nightly run at the unzip step. ZurRose refreshes the master roughly every 14 days, so most runs legitimately fetch an unchanged file (logged, not an error).
48
49
  - **`run_oddb2xml.sh`** — nightly build driver (cron: `0 1 * * * zdavatz`). Downloads the upstream sources **once**, then builds the `-b`/firstbase feed at price increments `45/50/55` plus `default` (no increment) into `$OUT_DIR` (`/home/zdavatz/oddb2xml`, one subdir each). The shared `downloads/` cache and transient zip live in `$BUILD_DIR` (`<OUT_DIR>-build`), **outside** `$OUT_DIR` so the transfer never uploads the multi-hundred-MB cache. Final step ("2b") regenerates the landing page. Each `oddb2xml` invocation is wrapped in `run_with_retry` (default **3 attempts, 120 s apart**, tunable via `ODDB2XML_RETRIES`/`ODDB2XML_RETRY_DELAY`): a transient upstream download failure (e.g. Swissmedic resetting the connection, `Errno::ECONNRESET`) previously aborted the whole `set -e` run 14 s in and left the feeds a day stale, so it now retries before giving up; a genuine repeated failure still stops the run. **Firstbase (GS1 NONPHARMA) last-good fallback (3.0.29 onwards):** the GS1 `GetFirstbaseHealthcare` export (`id.gs1.ch/01/07612345000961` → `apitools.gs1.ch`) has been answering `403 - Forbidden`, which blanked `firstbase.csv` and dropped **every** NONPHARMA article from the `-b` feed (landing page then showed `NONPHARMA = 0 − 1 = −1`). The script keeps the last successful `firstbase.csv` in a persistent cache `$FIRSTBASE_CACHE` (default `<OUT_DIR>-state/firstbase.csv`) **outside** `$BUILD_DIR` so it survives the nightly `rm -rf`, seeds it into `downloads/` before the build, and refreshes it after a successful download. The gem side (`FirstbaseDownloader#download`, rewritten in 3.0.29) makes the seed usable: it still attempts the live GS1 fetch on the first (downloading) build, but only overwrites `firstbase.csv` when the response is a real non-empty CSV (`firstbase_csv?` rejects HTML/`403 - Forbidden`/empty bodies and open-uri exceptions), otherwise it **keeps the existing seeded file** instead of the old `"w+"` truncate-to-zero. A recovered GS1 therefore refreshes the data automatically; while GS1 is down the feed serves yesterday's (last-good) NONPHARMA rather than nothing. `generate_index_html.sh` also guards the NONPHARMA count so an empty CSV renders `—` (not `−1`). **ZurRose/pillbox last-good seed + first-build `--skip-download` (2026-07-07):** the actual reason the nightly feed went stale after 2026-07-03 was **not** firstbase (a firstbase failure is non-fatal) but **ZurRose**: `pillbox.oddb.org:80` started refusing connections around 01:00 every night from 2026-07-04 on, so `ZurroseDownloader` wrote a 0-byte `transfer.zip`, `unzip` produced no `transfer.dat`, and `downloader.rb:221` raised `Errno::ENOENT`, aborting the whole `set -e` run (all three retries hit the same dead pillbox). Fix is **script-only, no gem release**: `get_transfer.sh` (cron 00:30) already mirrors the ZurRose zip locally at `$GET_TRANSFER_ZIP` (`/home/zdavatz/software/get_transfer/TRANSFER.ZIP`), so the **first** `build_one` now runs a `first_build_attempt` helper that (a) `seed_downloads` — resets `downloads/` to contain only that seeded `transfer.zip` — then (b) runs `oddb2xml --skip-download -b -c zip`. The `--skip-download` flag makes `Cli#run` **skip the `FileUtils.rm_f(Dir.glob(DOWNLOADS/*))` wipe (cli.rb:50)** that used to delete the seed, so `Oddb2xml.skip_download` reuses the seeded `transfer.zip` for ZurRose (no pillbox fetch) while `download_as` still fetches **every other source fresh** (Swissmedic, Refdata, BAG, FHIR NDJSON, Weleda/WALA, and firstbase — proven: `--skip-download` only reuses files already present in `downloads/`, it does not suppress downloading absent ones). Seeding happens **per attempt** inside `run_with_retry`, so a retry restarts from a clean cache (clears partials) while keeping the seed. **Consequence for firstbase:** it is now downloaded **once** by the first build and reused by the `45/50/55`/artikelstamm increments via `--skip-download` — and it is **no longer pre-seeded into `downloads/`** from `$FIRSTBASE_CACHE` (under `--skip-download` a *present* `firstbase.csv` would be reused verbatim and never refreshed, freezing NONPHARMA). GS1's `id.gs1.ch` route currently returns a real 151 MB CSV, so firstbase fetches fresh; `$FIRSTBASE_CACHE` is still refreshed from the fresh file after the build for reference/recovery. The 3.0.29 gem-side `firstbase_csv?`/keep-existing logic is retained but only engages on the non-`--skip-download` path.
49
50
  - **`generate_index_html.sh DOCROOT [FIRSTBASE_CSV]`** — single source of truth for the landing page. Writes `index.html` + a self-contained `logo.svg` **atomically** (temp + `mv`, so either owner — root from setup, `zdavatz` from cron — can refresh it). Computes live counts: PHARMA = `<SMNO>` count in `default/oddb_article.xml`, NONPHARMA = firstbase CSV rows − 1, total ART = `<ART ` count. Also runs **`visitor_stats.py`** and embeds its graph. Re-run standalone any time (it only reads already-built files); a separate cron line refreshes it **hourly** (`5 * * * * zdavatz`) so counts + graph stay current between nightly builds.
50
51
  - **`visitor_stats.py LOG_GLOB CACHE_DIR [DAYS]`** — emits the visitors/sessions/region graph as an inline-SVG HTML **fragment** (last `DAYS`, default 14): Besucher = distinct IPs/day, Sitzungen = 30-min-inactivity sessions per `(IP, User-Agent)`, plus a top-6 country breakdown by IP. Bots are filtered by User-Agent. Region lookup is **fully self-contained** — pure Python stdlib + the free **DB-IP country-lite CSV** (CC-BY, no licence key) cached in the build `downloads/` dir and refreshed monthly; **no apt package, no gem, no system GeoIP DB**. Prints nothing (page degrades to omitting the section) when the Apache log is unreadable or empty. Reading `/var/log/apache2` requires the cron user to be in the **`adm`** group (`sudo usermod -aG adm zdavatz`).
51
- - **`swissmedic_watch.sh`** — outage/block auto-recovery (cron: `*/30 * * * * zdavatz`). Since the Swissmedic platform migration (~2026-06-23, now a Swisscom-operated gateway), `www.swissmedic.ch` intermittently resets this host's automated connections **after the TLS handshake** (TCP RST), which aborts `run_oddb2xml.sh` under `set -e` and leaves the feeds stale (the block is host/IP- and client-fingerprint-sensitive: a real browser works, `curl`/`wget`/Ruby get reset, while other admin.ch hosts answer fine — so it is a WAF/bot rule, not an outage). The watcher polls Swissmedic with **oddb2xml's own client** (a Ruby `open-uri` canary on `listen_neu.html`); while blocked it is a silent no-op, and the moment it gets HTTP 200 it launches **one** build and emails. It fires **at most once per day** (stamp in `$STATE_DIR`, default `<OUT_DIR>-watch`, kept **outside** the wiped `$BUILD_DIR`), and skips when a build is already running or today's `default/oddb_article.xml` is already fresh. It exports `RBENV_VERSION=3.4.5` + the rbenv-shims PATH to match the nightly cron (the repo `.ruby-version` pins an uninstalled Ruby).
52
+ - **`swissmedic_watch.sh`** — outage/block auto-recovery (cron: `*/30 * * * * zdavatz`). Since the Swissmedic platform migration (~2026-06-23, now a Swisscom-operated gateway), `www.swissmedic.ch` intermittently resets this host's automated connections **after the TLS handshake** (TCP RST), which aborts `run_oddb2xml.sh` under `set -e` and leaves the feeds stale (the block is host/IP- and client-fingerprint-sensitive: a real browser works, `curl`/`wget`/Ruby get reset, while other admin.ch hosts answer fine — so it is a WAF/bot rule, not an outage). The watcher polls Swissmedic with **oddb2xml's own client** (a Ruby `open-uri` canary on `listen_neu.html`); while blocked it is a silent no-op, and the moment it gets HTTP 200 it launches **one** build and emails. It fires **at most once per day** (stamp in `$STATE_DIR`, default `<OUT_DIR>-watch`, kept **outside** the wiped `$BUILD_DIR`), and skips when a build is already running or today's `default/oddb_article.xml` is already fresh. **PATH assembly (2026-07-28 onwards):** cron hands the watcher a minimal PATH, so it builds one explicitly — `/usr/local/bin` (where a root `gem install` under Debian's system Ruby puts the binstub) + the user gem bin dir (`$(ruby -e 'print Gem.user_dir')/bin`, where an *unprivileged* `gem install` silently lands instead, because `/var/lib/gems` is not writable) + rbenv shims **only when they exist** (the pre-2026-07 box; `RBENV_VERSION=3.4.5` is exported only in that case). The previous hardcoded rbenv-only PATH carried no `/usr/local/bin`, so the first watcher-triggered build on the rebuilt server died `oddb2xml: Kommando nicht gefunden` (exit 127) on all three retries before aborting. `run_oddb2xml.sh` step 1 extends PATH the same way right after its `gem install`, for the same reason.
52
53
  - **`transfer.sh`** — optional hand-off (scp) of `$OUT_DIR` to the HIN host; `SCP_DEST` is required-but-unset until the HIN host is known.
53
- - **`setup_mediupdatexml_web.sh`** — one-time root setup of the Apache vhost + initial page.
54
+ - **`setup_new_server.sh`** (2026-07-28 onwards) — one-command rebuild of a bare Debian host into this download site, written after **HIN deleted the previous server**: everything that had accumulated by hand on that box, so a rebuild is not a day of archaeology. Root, idempotent. Installs the packages (`apache2` + `certbot`; **`ruby-full`** — Debian 13 ships Ruby 3.3, matching the repo `.ruby-version`, so the new host has **no rbenv**; `cron`, absent on a minimal image; `unzip`/`zip` for the ZurRose transfer and the build zip; `poppler-utils` for `pdftotext` in `tools/generate_bag_sl_group_prices.rb`; nokogiri/ffi build headers), adds `$RUN_USER` to **`adm`** (so `visitor_stats.py` may read `/var/log/apache2`), sets the home dir to **711** (so `www-data` can traverse to `$OUT_DIR` without listing it), `gem install oddb2xml`, creates the five directories (`$OUT_DIR`, `-build`, `-state`, `-watch` — the last two deliberately outside the nightly-wiped `-build` — and `$GET_TRANSFER_DIR`, symlinking the tracked `get_transfer.sh` into it), writes **`/etc/cron.d/mediupdatexml`** (the whole schedule in one re-creatable file rather than the old box's per-user crontab: `get_transfer` 00:30, nightly build 01:00, rust2xml Artikelstamm 03:00, aips2sqlite Fachinfos 04:30, landing page hourly at `:05`, Swissmedic watcher every 30 min) plus a logrotate rule for the cron logs, then hands over to `setup_aips2sqlite.sh` and `setup_mediupdatexml_web.sh`. Env: `RUN_USER`, `OUT_DIR`, `SKIP_WEB=1`, `SKIP_GEM=1`, `SKIP_AIPS=1`. **Deliberately out of scope** (gone with the old server, restored separately): an MTA for the cron `MAILTO`.
55
+ - **`setup_mediupdatexml_web.sh`** — one-time root setup of the Apache vhost + initial page. Skips the Let's Encrypt step with a hint when the domain does not yet resolve in **public** DNS (checked via Cloudflare DoH, not the local resolver, which may hold a stale or negative cache).
56
+ - **`setup_aips2sqlite.sh`** (2026-08-01 onwards) — provisions the **aips2sqlite** half of the site, restored after the server rebuild. The landing page links a whole `/aips2sqlite/` section (Fachinformationen as XML, the AmiKo `.db` files, `oddb2xml_swissmedic_sequences.csv`, `atc_codes_used_set.txt`); `setup_mediupdatexml_web.sh` writes the Apache alias `/aips2sqlite` → `<checkout>/jars/output`, but on the rebuilt host neither the JRE nor that directory existed, so **every link answered 403** — Apache falls back to the default `<Directory /> Require all denied` when the aliased path does not exist, so a *missing* target reads as *forbidden*, not *not found*. Root, idempotent: installs **`openjdk-21-jre-headless`** (the repo ships a prebuilt fat jar `jars/aips2sqlite.jar`, so a JRE suffices — no JDK/Gradle unless the jar is rebuilt; the only AWT user, `BarCode.java`/barcode4j, is off the Fachinfo path), clones the checkout if missing, creates `jars/output` + `jars/downloads` owned by `$RUN_USER`, and appends the 04:30 cron entry to `/etc/cron.d/mediupdatexml` if not already there (`SKIP_CRON=1` when called from `setup_new_server.sh`, which writes that line itself). Generating the data is a separate build-user step: `aips2sqlite/scripts/generate_aips_fi` (~1 h; `--lang=de --xml`, then `--lang=fr --xml --nodown`, then `--nodown --smsequence`, renaming `swiss_medic_sequences.csv` → `oddb2xml_swissmedic_sequences.csv`). `REFDATA_API_KEY` (developer.refdata.ch) is **optional** for this path: the Refdata Partner SOAP download it gates is consumed only by the Takeda partner export, and its failure is caught and logged, so the FI/sequences output is complete without it.
57
+ - **`setup_rust2xml.sh`** — the build-user (never root) counterpart: installs the Rust toolchain via **rustup** — not Debian's `rustc`, because `rust2xml/scripts/run_artikelstamm.sh` sources `$HOME/.cargo/env` to find cargo under cron's minimal PATH — and runs `cargo build --release --bin rust2xml` in `$RUST2XML_DIR` (default `~/software/rust2xml`), feeding the 03:00 cron entry that publishes rust2xml's own Artikelstamm to `/artikelstamm/rust2xml/`. Idempotent: re-running updates the toolchain and rebuilds only if sources changed.
54
58
 
55
59
  Only the scripts are git-tracked; the generated `index.html`/`logo.svg` and the `downloads/` cache are not.
56
60
 
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- oddb2xml (3.0.31)
4
+ oddb2xml (3.0.32)
5
5
  csv
6
6
  htmlentities
7
7
  httpi
data/History.txt CHANGED
@@ -1,3 +1,6 @@
1
+ === 3.0.32 / 26.08.2026
2
+ * Bugfix (--fhir, the default): follow the BAG SL export to its new URL. BAG moved the publication on 24.08.2026 and announced it the next evening: /static/fhir/foph-sl-export-*.ndjson became /static/sl/publication/fhir/foph-sl-publication-*.ndjson. Only the -latest- alias is gone (the old dated snapshots stay in place), so the old URL failed by downloading nothing and reporting success rather than by crashing -- ch.oddb.org ran that way for twelve mornings with three HTTP 404 lines in a log as the only trace. Without the NDJSON there are no French/Italian names, no SL prices, no limitations and no Indikationscodes. Per the announcement there is no language-less default any more; every URL has to name de, fr or it, which FhirDownloader already did. A preliminary publication now exists in parallel under /static/sl/preliminary/fhir/foph-sl-preliminary-*, which oddb2xml does not use. The file prefix is its own constant now, so the next move touches one line.
3
+
1
4
  === 3.0.31 / 14.07.2026
2
5
  * Bugfix (SLOPLUS / DEDUCTIBLE): every SL article claimed the raised 40% Selbstbehalt. The FHIR extractor mapped the BAG costShare percentage the wrong way round -- costShare 10 (the ordinary 10% Selbstbehalt, carried by ~97% of all packages) was turned into deductible "Y", which Builder#build_article emits as <SLOPLUS>1</SLOPLUS> (40%) and build_artikelstamm as <DEDUCTIBLE>40</DEDUCTIBLE>. costShare now maps 40 => "Y" (SLOPLUS 1 / DEDUCTIBLE 40), 10 => "N" (SLOPLUS 2 / DEDUCTIBLE 10), 20 => deductible20 "Y" (transitional rate, issue #81) -- the same semantics FlagSB carries in the legacy BAG Preparations.xml. The BAG publication was correct all along: the live feed has costShare 40 on 385 of 10'388 packages (222 preparations), 10 on the rest, and never mixes the two within one preparation.
3
6
 
data/README.md CHANGED
@@ -294,12 +294,12 @@ We use the following files:
294
294
  * http://bag.e-mediat.net/SL2007.Web.External/File.axd?file=XMLPublications.zip
295
295
  * https://www.medregbm.admin.ch/Publikation/CreateExcelListBetriebs
296
296
  * https://www.medregbm.admin.ch/Publikation/CreateExcelListMedizinalPersons
297
- * http://zurrose.com/fileadmin/main/lib/download.php?file=/fileadmin/user_upload/downloads/ProduktUpdate/IGM11_mit_MwSt/Vollstamm/transfer.dat
297
+ * https://www.zurrose.ch/sites/default/files/media/downloads/medi/produktupdate/igm11/de/transfer.dat (ZurRose IGM11 Vollstamm; mirrored by `scripts/get_transfer.sh`, which the gem then reads via `http://pillbox.oddb.org/TRANSFER.ZIP` or a local seed. The former `zurrose.com/fileadmin/main/lib/download.php?file=…` path now redirects to the marketing homepage.)
298
298
  * https://raw.githubusercontent.com/zdavatz/oddb2xml_files/master/NON-Pharma.xls
299
299
  * http://download.swissmedicinfo.ch/ (AipsDownload)
300
300
  * https://raw.githubusercontent.com/zdavatz/oddb2xml_files/master/LPPV.txt
301
301
  * https://raw.githubusercontent.com/zdavatz/cpp2sqlite/master/input/atc_codes_multi_lingual.txt
302
- * https://epl.bag.admin.ch/static/fhir/foph-sl-export-latest-{de,fr,it}.ndjson (FHIR NDJSON, used with `--fhir`)
302
+ * https://epl.bag.admin.ch/static/sl/publication/fhir/foph-sl-publication-latest-{de,fr,it}.ndjson (FHIR NDJSON, used with `--fhir`)
303
303
  * https://id.gs1.ch/01/07612345000961 (GS1 Switzerland firstbase CSV — full barcode registry, used with `-b`/`--firstbase`)
304
304
 
305
305
  ## Indikationscode (BAG XXXXX.NN)
@@ -335,7 +335,7 @@ The same data is also exposed in-memory on each item and package as
335
335
  `item[:indication_codes]` (an array of `{code:, cud_id:, text:}`
336
336
  hashes) for downstream consumers that build their own emitters.
337
337
 
338
- Counted on the live FOPH feed (`foph-sl-export-latest-de.ndjson`,
338
+ Counted on the live FOPH feed (`foph-sl-publication-latest-de.ndjson`,
339
339
  6'775 bundles): **539** products carry a total of **1'293** codes.
340
340
 
341
341
  Per BAG Rundschreiben vom 19. Februar 2026, the Indikationscode must
@@ -420,7 +420,7 @@ RegulatedAuthorization/65839
420
420
  ```
421
421
 
422
422
  The CUDs are identical in `id` across the three per-language NDJSON
423
- files (`foph-sl-export-latest-{de,fr,it}.ndjson`); only the
423
+ files (`foph-sl-publication-latest-{de,fr,it}.ndjson`); only the
424
424
  `concept.text` differs. `FhirExtractor` resolves DE from the primary
425
425
  file and merges FR/IT in via the same CUD id. Coverage on the live
426
426
  feed went from 0 / 9'108 to 9'108 / 9'108 (100 %). See issue
@@ -520,6 +520,51 @@ a few times on such transient failures (`ODDB2XML_RETRIES`,
520
520
  `ODDB2XML_RETRY_DELAY`); a persistent per-host block still needs one of the
521
521
  mitigations above.
522
522
 
523
+ ## Deployment (the mediupdatexml.oddb.org download site)
524
+
525
+ The `scripts/` directory automates the public download site
526
+ <https://mediupdatexml.oddb.org>. These scripts are not part of the gem, but
527
+ they are the reference setup for running oddb2xml unattended.
528
+
529
+ * `get_transfer.sh` — mirrors the ZurRose IGM11 Vollstamm (`transfer.dat`) to
530
+ `/home/zdavatz/software/get_transfer/TRANSFER.ZIP` (cron 00:30), which the
531
+ nightly build seeds into its `downloads/` cache instead of fetching
532
+ `pillbox.oddb.org` — that host is only a mirror of this same job. Validates
533
+ the fixed-width record shape (98 bytes per record) before publishing, and
534
+ keeps the previous zip when the download fails, so a ZurRose outage cannot
535
+ abort the 01:00 build. Override the source with `TRANSFER_URL`.
536
+ * `run_oddb2xml.sh` — nightly driver (cron 01:00). Downloads the sources once,
537
+ then builds the `-b` feed at price increments 45/50/55 plus `default` (no
538
+ increment) and the Artikelstamm (v6 + legacy v5), retrying transient download
539
+ failures (`ODDB2XML_RETRIES`, `ODDB2XML_RETRY_DELAY`).
540
+ * `swissmedic_watch.sh` — every 30 min. While Swissmedic blocks this host it is
541
+ a silent no-op; the moment the site answers again it launches one build (at
542
+ most once per day).
543
+ * `generate_index_html.sh` and `visitor_stats.py` — landing page with live
544
+ article counts and a visitor graph, refreshed hourly.
545
+ * `setup_new_server.sh` — provisions a bare Debian host in one command:
546
+ packages, user permissions, the build/output directories,
547
+ `/etc/cron.d/mediupdatexml`, logrotate, then the Apache vhost and Let's
548
+ Encrypt certificate via `setup_mediupdatexml_web.sh`. Run as root; idempotent.
549
+ * `setup_rust2xml.sh` — installs the Rust toolchain and builds
550
+ [rust2xml](https://github.com/zdavatz/rust2xml) for the 03:00 Artikelstamm
551
+ job. Run as the build user, not root.
552
+ * `setup_aips2sqlite.sh` — installs the Java runtime, restores the
553
+ [aips2sqlite](https://github.com/zdavatz/aips2sqlite) checkout and creates
554
+ `jars/output`, the target of the `/aips2sqlite/` Apache alias, plus the 04:30
555
+ cron entry that regenerates the Fachinformationen. Run as root; idempotent.
556
+ Without it every `/aips2sqlite/` link answers 403, because Apache denies a
557
+ path that does not exist.
558
+ * `transfer.sh` — optional scp hand-off of the output tree.
559
+
560
+ One Debian-specific pitfall worth knowing when running oddb2xml from cron: with
561
+ the system Ruby, `gem install oddb2xml` as an unprivileged user cannot write
562
+ `/var/lib/gems` and falls back to a user install under
563
+ `~/.local/share/gem/ruby/X.Y.0/bin`, which is on no default PATH — the build
564
+ then fails with `oddb2xml: command not found`. `run_oddb2xml.sh` and
565
+ `swissmedic_watch.sh` add that directory and `/usr/local/bin` to `PATH`
566
+ themselves.
567
+
523
568
  ## Testing
524
569
 
525
570
  * Calling rake spec runs spec tests.
@@ -14,7 +14,17 @@ module Oddb2xml
14
14
  include DownloadMethod
15
15
 
16
16
  BASE_URL = "https://epl.bag.admin.ch"
17
- STATIC_FHIR_PATH = "/static/fhir"
17
+ # BAG moved the export on 24.08.2026 and announced it the next evening:
18
+ # /static/fhir/foph-sl-export-* became
19
+ # /static/sl/publication/fhir/foph-sl-publication-*.
20
+ #
21
+ # The old -latest- alias answers 404 while the old dated snapshots stay in
22
+ # place, so this fails by downloading nothing rather than by crashing.
23
+ # There is no language-less default any more; per BAG each URL must name
24
+ # de, fr or it. A preliminary publication now exists in parallel under
25
+ # /static/sl/preliminary/fhir/foph-sl-preliminary-*, which this does not use.
26
+ STATIC_FHIR_PATH = "/static/sl/publication/fhir"
27
+ FILE_PREFIX = "foph-sl-publication"
18
28
  LANGUAGES = %w[de fr it].freeze
19
29
 
20
30
  def initialize(options = {})
@@ -32,7 +42,7 @@ module Oddb2xml
32
42
  else
33
43
  files = {}
34
44
  LANGUAGES.each do |lang|
35
- url = "#{BASE_URL}#{STATIC_FHIR_PATH}/foph-sl-export-latest-#{lang}.ndjson"
45
+ url = "#{BASE_URL}#{STATIC_FHIR_PATH}/#{FILE_PREFIX}-latest-#{lang}.ndjson"
36
46
  path = download_one(url)
37
47
  files[lang] = path if path
38
48
  end
@@ -1,3 +1,3 @@
1
1
  module Oddb2xml
2
- VERSION = "3.0.31"
2
+ VERSION = "3.0.32"
3
3
  end
@@ -0,0 +1,75 @@
1
+ #!/usr/bin/env bash
2
+ #
3
+ # get_transfer — mirror the ZurRose IGM11 "Vollstamm" article master locally.
4
+ #
5
+ # ZurRose publishes the full article master as a fixed-width transfer.dat. The
6
+ # nightly build (run_oddb2xml.sh, 01:00) seeds the zip produced here into its
7
+ # downloads/ cache, so oddb2xml reads ZurRose from this host instead of
8
+ # fetching http://pillbox.oddb.org/TRANSFER.ZIP — see seed_downloads() there.
9
+ #
10
+ # History worth keeping: this script used to live ONLY in
11
+ # /home/zdavatz/software/get_transfer on the old server, so when that host was
12
+ # deleted (2026-07-28) the mirror went with it, together with its download URL.
13
+ # It now lives in the repo and $DATA_DIR/get_transfer.sh is a symlink to it, so
14
+ # a rebuilt server gets it back from the checkout (setup_new_server.sh).
15
+ #
16
+ # The download URL moved too: the old
17
+ # zurrose.com/fileadmin/main/lib/download.php?file=... path 301s to the
18
+ # marketing homepage since roughly mid-July 2026. The current one is below.
19
+ #
20
+ # ZurRose refreshes the master roughly every 14 days, so most runs legitimately
21
+ # fetch a file identical to yesterday's; that is not an error.
22
+ #
23
+ # Configurable via environment:
24
+ # TRANSFER_URL source URL (default: the ZurRose IGM11 de/transfer.dat)
25
+ # DATA_DIR where TRANSFER.ZIP lands (default /home/zdavatz/software/get_transfer)
26
+ # MIN_LINES reject a file with fewer records (default 100000)
27
+ #
28
+ set -euo pipefail
29
+
30
+ TRANSFER_URL="${TRANSFER_URL:-https://www.zurrose.ch/sites/default/files/media/downloads/medi/produktupdate/igm11/de/transfer.dat}"
31
+ DATA_DIR="${DATA_DIR:-/home/zdavatz/software/get_transfer}"
32
+ MIN_LINES="${MIN_LINES:-100000}"
33
+
34
+ ZIP="$DATA_DIR/TRANSFER.ZIP"
35
+
36
+ log() { printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*"; }
37
+ die() { log "ERROR: $*"; exit 1; }
38
+
39
+ mkdir -p "$DATA_DIR"
40
+
41
+ # Work in a scratch dir so a failed or truncated download can never replace a
42
+ # good TRANSFER.ZIP. The build reads that zip unconditionally at 01:00; half a
43
+ # file there would abort the whole nightly run at oddb2xml's unzip step.
44
+ WORK="$(mktemp -d "${TMPDIR:-/tmp}/get_transfer.XXXXXX")"
45
+ trap 'rm -rf "$WORK"' EXIT
46
+
47
+ log "Downloading $TRANSFER_URL"
48
+ curl -fsSL --max-time 900 --retry 3 --retry-delay 30 \
49
+ -o "$WORK/transfer.dat" "$TRANSFER_URL" \
50
+ || die "download failed - keeping $( [[ -s $ZIP ]] && echo "the previous TRANSFER.ZIP" || echo "nothing, there is no previous TRANSFER.ZIP" )"
51
+
52
+ # Validate before publishing. transfer.dat is a fixed-width IGM11 file: every
53
+ # record is exactly 98 bytes. A login page, an error page or a truncated
54
+ # transfer would sail through a plain size check but fails this one.
55
+ lines=$(wc -l < "$WORK/transfer.dat")
56
+ (( lines >= MIN_LINES )) \
57
+ || die "only $lines records (< $MIN_LINES) - looks like an error page, not the Vollstamm"
58
+ badlen=$(awk 'length != 98 { n++ } END { print n+0 }' "$WORK/transfer.dat")
59
+ (( badlen == 0 )) \
60
+ || die "$badlen of $lines records are not 98 chars - not an IGM11 fixed-width file"
61
+
62
+ # The zip entry MUST be named transfer.dat (lowercase): oddb2xml unzips it into
63
+ # downloads/ and then opens downloads/transfer.dat by that exact name
64
+ # (ZurroseDownloader, lib/oddb2xml/downloader.rb).
65
+ ( cd "$WORK" && zip -q TRANSFER.ZIP transfer.dat )
66
+
67
+ if cmp -s "$WORK/transfer.dat" <(unzip -p "$ZIP" transfer.dat 2>/dev/null); then
68
+ log "Unchanged since the last run ($lines records) - refreshing the mirror anyway"
69
+ fi
70
+
71
+ # Atomic publish: same filesystem, so mv is a rename. A reader either sees the
72
+ # old zip or the new one, never a partial.
73
+ mv "$WORK/TRANSFER.ZIP" "$ZIP.new" && mv "$ZIP.new" "$ZIP"
74
+
75
+ log "Wrote $ZIP ($(stat -c %s "$ZIP") bytes, $lines records)"
@@ -70,9 +70,17 @@ run_with_retry() {
70
70
  }
71
71
 
72
72
  # 1. Install / update the published gem unless told otherwise.
73
+ # Under Debian's system Ruby an unprivileged `gem install` cannot write
74
+ # /var/lib/gems and silently falls back to a user install, whose bin dir is on
75
+ # nobody's PATH — so the freshly installed gem is unreachable and the build dies
76
+ # with exit 127. Put both candidate dirs on PATH, user install first, so
77
+ # $ODDB2XML_BIN resolves to whatever this step just installed.
73
78
  if [[ "${SKIP_GEM_INSTALL:-0}" != "1" ]]; then
74
79
  log "Installing oddb2xml gem"
75
80
  gem install oddb2xml
81
+ GEM_USER_BIN="$(ruby -e 'print Gem.user_dir' 2>/dev/null)/bin"
82
+ [[ -d "$GEM_USER_BIN" ]] && export PATH="$GEM_USER_BIN:$PATH"
83
+ case ":$PATH:" in */usr/local/bin:*) ;; *) export PATH="/usr/local/bin:$PATH" ;; esac
76
84
  fi
77
85
 
78
86
  # 2. Fresh working dir (keeps a shared downloads/ cache across increments).
@@ -0,0 +1,143 @@
1
+ #!/usr/bin/env bash
2
+ #
3
+ # setup_aips2sqlite.sh — provision the aips2sqlite half of mediupdatexml.oddb.org.
4
+ #
5
+ # The landing page links a whole section of aips2sqlite output (Fachinformationen
6
+ # as XML, the AmiKo databases, the Swissmedic-sequences CSV), served by Apache
7
+ # through the alias
8
+ #
9
+ # /aips2sqlite/ -> $AIPS_DIR/jars/output
10
+ #
11
+ # written by setup_mediupdatexml_web.sh. That alias is only half the story: the
12
+ # directory it points at is produced by a Java job that has to run on this host.
13
+ # On the rebuilt server (2026-07-28) neither the JRE nor the output directory
14
+ # existed, so every /aips2sqlite/ link answered 403 — Apache denies a path that
15
+ # does not exist, because the granting <Directory> block never matches.
16
+ #
17
+ # This script installs the runtime, restores the checkout, creates the output
18
+ # directory and schedules the nightly regeneration.
19
+ #
20
+ # Run with: sudo scripts/setup_aips2sqlite.sh
21
+ #
22
+ # Idempotent — safe to re-run. It does NOT generate the data (that takes ~1 h
23
+ # and belongs to the build user); run scripts/generate_aips_fi afterwards, or
24
+ # wait for the cron entry it installs.
25
+ #
26
+ # Configurable via environment:
27
+ # RUN_USER build/cron user (default zdavatz)
28
+ # AIPS_DIR checkout location (default /home/<RUN_USER>/software/aips2sqlite)
29
+ # AIPS_REPO clone URL (default https://github.com/zdavatz/aips2sqlite)
30
+ # OUT_DIR published root, only used to locate the log dir
31
+ # SKIP_CRON set to 1 to leave /etc/cron.d/mediupdatexml alone
32
+ #
33
+ set -euo pipefail
34
+
35
+ RUN_USER="${RUN_USER:-zdavatz}"
36
+ RUN_HOME="$(getent passwd "$RUN_USER" | cut -d: -f6)"
37
+ AIPS_DIR="${AIPS_DIR:-${RUN_HOME}/software/aips2sqlite}"
38
+ AIPS_REPO="${AIPS_REPO:-https://github.com/zdavatz/aips2sqlite}"
39
+ OUT_DIR="${OUT_DIR:-${RUN_HOME}/oddb2xml}"
40
+ STATE_DIR="${OUT_DIR%/}-state"
41
+ CRON_FILE=/etc/cron.d/mediupdatexml
42
+
43
+ step() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
44
+
45
+ if [[ $EUID -ne 0 ]]; then
46
+ echo "Please run as root: sudo $0" >&2
47
+ exit 1
48
+ fi
49
+ if [[ -z "$RUN_HOME" ]]; then
50
+ echo "User $RUN_USER does not exist" >&2
51
+ exit 1
52
+ fi
53
+
54
+ # --- 1. Java runtime ---------------------------------------------------------
55
+ # aips2sqlite ships a prebuilt fat jar (jars/aips2sqlite.jar), so a JRE is
56
+ # enough — no JDK and no Gradle unless the jar itself is rebuilt. README asks
57
+ # for Java 21+; Debian 13 has both 21 and 25, pin 21 to match what the jar was
58
+ # built and tested against.
59
+ #
60
+ # The font libraries are NOT optional, even though Debian lists them as mere
61
+ # Recommends of the headless JRE: every Fachinfo carries a rendered EAN13
62
+ # barcode (RealExpertInfo.updateSectionPackungen -> BarCode.encode -> barcode4j),
63
+ # and drawing its digits loads libfontmanager.so. Installing the JRE with
64
+ # --no-install-recommends therefore gets through the whole download phase and
65
+ # then dies on the first medicine with
66
+ # UnsatisfiedLinkError: libfontmanager.so: libharfbuzz.so.0: cannot open ...
67
+ # so they are listed explicitly here rather than left to apt's recommends
68
+ # handling. fonts-dejavu-core gives fontconfig an actual font to find.
69
+ step "Installing the Java runtime (openjdk-21-jre-headless + font stack)"
70
+ export DEBIAN_FRONTEND=noninteractive
71
+ apt-get update -qq
72
+ apt-get install -y --no-install-recommends \
73
+ openjdk-21-jre-headless \
74
+ libfreetype6 libharfbuzz0b libfontconfig1 fonts-dejavu-core
75
+ java -version
76
+
77
+ # --- 2. Checkout -------------------------------------------------------------
78
+ step "Ensuring the aips2sqlite checkout at $AIPS_DIR"
79
+ if [[ -d "$AIPS_DIR/.git" ]]; then
80
+ echo "already present — leaving it alone (generate_aips_fi does its own git pull)"
81
+ else
82
+ install -d -o "$RUN_USER" -g "$RUN_USER" "$(dirname "$AIPS_DIR")"
83
+ sudo -u "$RUN_USER" git clone "$AIPS_REPO" "$AIPS_DIR"
84
+ fi
85
+
86
+ # --- 3. Output + download directories ----------------------------------------
87
+ # jars/output is the Apache alias target: it must exist even while empty, or
88
+ # /aips2sqlite/ answers 403 instead of an (empty) listing.
89
+ step "Creating $AIPS_DIR/jars/{output,downloads}"
90
+ for d in "$AIPS_DIR/jars/output" "$AIPS_DIR/jars/downloads"; do
91
+ install -d -o "$RUN_USER" -g "$RUN_USER" -m 755 "$d"
92
+ done
93
+
94
+ # --- 4. Nightly regeneration -------------------------------------------------
95
+ # The schedule lives in the one cron file setup_new_server.sh writes; append
96
+ # only when that file exists and does not already carry the entry, so the two
97
+ # scripts can be run in either order without fighting over it.
98
+ if [[ "${SKIP_CRON:-0}" != "1" ]]; then
99
+ step "Scheduling the nightly Fachinfo generation (04:30)"
100
+ install -d -o "$RUN_USER" -g "$RUN_USER" "$STATE_DIR"
101
+ if [[ ! -f "$CRON_FILE" ]]; then
102
+ echo "!! $CRON_FILE does not exist yet — run setup_new_server.sh first,"
103
+ echo " it writes the full schedule including this entry."
104
+ elif grep -q 'generate_aips_fi' "$CRON_FILE"; then
105
+ echo "entry already present in $CRON_FILE"
106
+ else
107
+ cat >> "$CRON_FILE" <<EOF
108
+
109
+ # 04:30 aips2sqlite Fachinfo XMLs + AmiKo DBs + Swissmedic sequences,
110
+ # published under /aips2sqlite/.
111
+ 30 4 * * * $RUN_USER [ -x $AIPS_DIR/scripts/generate_aips_fi ] && $AIPS_DIR/scripts/generate_aips_fi >> $STATE_DIR/generate_aips_fi.log 2>&1
112
+ EOF
113
+ echo "appended to $CRON_FILE"
114
+ fi
115
+ fi
116
+
117
+ cat <<EOF
118
+
119
+ ==> aips2sqlite provisioning done.
120
+
121
+ The alias /aips2sqlite/ -> $AIPS_DIR/jars/output now resolves; it stays empty
122
+ until the generator has run once. As $RUN_USER (takes ~1 h, ~8 GB heap):
123
+
124
+ $AIPS_DIR/scripts/generate_aips_fi
125
+
126
+ Optional: export REFDATA_API_KEY (developer.refdata.ch) to also fetch the
127
+ Refdata Partner GLN data. It is only consumed by the Takeda partner export,
128
+ so the Fachinfo/sequences output is complete without it — the run just logs
129
+ one download exception.
130
+
131
+ Known upstream gap: BAG's resource index
132
+ https://epl.bag.admin.ch/api/sl/public/resources/current currently reports
133
+ "fhir": {"fileUrl": null}, so aips2sqlite's FHIR download resolves to
134
+ /static/null and the run parses 0 preparations (no SL flags, no prices). The
135
+ export itself is alive at the fixed path oddb2xml uses:
136
+
137
+ https://epl.bag.admin.ch/static/sl/publication/fhir/foph-sl-publication-latest-de.ndjson
138
+
139
+ Until the jar learns that fallback, seed it before the run:
140
+
141
+ cp <oddb2xml downloads>/foph-sl-publication-latest-de.ndjson \\
142
+ $AIPS_DIR/jars/downloads/fhir-sl.ndjson
143
+ EOF
@@ -0,0 +1,183 @@
1
+ #!/usr/bin/env bash
2
+ #
3
+ # setup_new_server.sh — bring a bare Debian host up to the point where it can
4
+ # run the mediupdatexml.oddb.org download site and its nightly builds.
5
+ #
6
+ # Written after HIN deleted the previous server (2026-07-28): everything that
7
+ # had accumulated by hand on that box is captured here, so a rebuild is one
8
+ # command instead of a day of archaeology.
9
+ #
10
+ # Run with: sudo scripts/setup_new_server.sh
11
+ #
12
+ # Idempotent — safe to re-run. It installs packages, creates the build/output
13
+ # directories, installs the oddb2xml gem, writes /etc/cron.d/mediupdatexml and
14
+ # a logrotate rule, and finally hands over to setup_mediupdatexml_web.sh for the
15
+ # Apache vhost + Let's Encrypt certificate.
16
+ #
17
+ # What it deliberately does NOT do (run these as the build user afterwards):
18
+ # scripts/setup_rust2xml.sh Rust toolchain for the rust2xml artikelstamm job
19
+ #
20
+ # Configurable via environment:
21
+ # RUN_USER build/cron user (default zdavatz)
22
+ # OUT_DIR published download root (default /home/<RUN_USER>/oddb2xml)
23
+ # SKIP_WEB set to 1 to skip the Apache/certbot step
24
+ # SKIP_GEM set to 1 to skip `gem install oddb2xml`
25
+ # SKIP_AIPS set to 1 to skip the aips2sqlite provisioning
26
+ #
27
+ set -euo pipefail
28
+
29
+ RUN_USER="${RUN_USER:-zdavatz}"
30
+ RUN_HOME="$(getent passwd "$RUN_USER" | cut -d: -f6)"
31
+ OUT_DIR="${OUT_DIR:-${RUN_HOME}/oddb2xml}"
32
+ BUILD_DIR="${OUT_DIR%/}-build"
33
+ STATE_DIR="${OUT_DIR%/}-state"
34
+ WATCH_DIR="${OUT_DIR%/}-watch"
35
+ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
36
+ REPO_DIR="$(dirname "$SCRIPT_DIR")"
37
+ RUST2XML_DIR="${RUST2XML_DIR:-${RUN_HOME}/software/rust2xml}"
38
+ GET_TRANSFER_DIR="${GET_TRANSFER_DIR:-${RUN_HOME}/software/get_transfer}"
39
+ AIPS_DIR="${AIPS_DIR:-${RUN_HOME}/software/aips2sqlite}"
40
+
41
+ step() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
42
+
43
+ if [[ $EUID -ne 0 ]]; then
44
+ echo "Please run as root: sudo $0" >&2
45
+ exit 1
46
+ fi
47
+ if [[ -z "$RUN_HOME" ]]; then
48
+ echo "User $RUN_USER does not exist" >&2
49
+ exit 1
50
+ fi
51
+
52
+ # --- 1. System packages ------------------------------------------------------
53
+ # ruby-full Debian 13 ships Ruby 3.3, matching the repo's .ruby-version --
54
+ # no rbenv needed (the old box carried an rbenv 3.4.5 install).
55
+ # unzip/zip run_oddb2xml.sh unpacks the build zip; ZurroseDownloader shells
56
+ # out to `unzip` for transfer.zip, and to `iconv` (libc-bin).
57
+ # cron not installed by default on a minimal Debian 13 image.
58
+ # poppler-utils pdftotext, for tools/generate_bag_sl_group_prices.rb.
59
+ # lib*-dev native build fallbacks for nokogiri/ffi when no precompiled gem
60
+ # matches the platform.
61
+ step "Installing system packages"
62
+ export DEBIAN_FRONTEND=noninteractive
63
+ apt-get update -qq
64
+ apt-get install -y --no-install-recommends \
65
+ apache2 certbot python3-certbot-apache \
66
+ ruby-full ruby-dev build-essential \
67
+ cron unzip zip curl wget git ca-certificates \
68
+ poppler-utils libxml2-dev libxslt1-dev zlib1g-dev libyaml-dev pkg-config
69
+ systemctl enable --now cron >/dev/null 2>&1 || true
70
+
71
+ # --- 2. User, groups, permissions -------------------------------------------
72
+ # "adm" lets visitor_stats.py read /var/log/apache2/*access.log for the graph.
73
+ step "Granting $RUN_USER read access to the Apache logs (group adm)"
74
+ usermod -aG adm "$RUN_USER"
75
+
76
+ # www-data must traverse (not list) the home dir to reach $OUT_DIR.
77
+ step "Setting $RUN_HOME to 711 so Apache can traverse it"
78
+ chmod 711 "$RUN_HOME"
79
+
80
+ # --- 3. The gem --------------------------------------------------------------
81
+ if [[ "${SKIP_GEM:-0}" != "1" ]]; then
82
+ step "Installing the oddb2xml gem (system-wide, binstub in /usr/local/bin)"
83
+ gem install oddb2xml --no-document
84
+ oddb2xml --version 2>/dev/null || true
85
+ fi
86
+
87
+ # --- 4. Directory layout -----------------------------------------------------
88
+ # OUT_DIR published, served by Apache (per-increment subdirs + landing page)
89
+ # BUILD_DIR shared downloads/ cache; wiped at the start of every build
90
+ # STATE_DIR last-good firstbase.csv + cron logs; must survive the wipe
91
+ # WATCH_DIR swissmedic_watch.sh stamps and log
92
+ # GET_TRANSFER_DIR local ZurRose mirror (TRANSFER.ZIP), seeded into the build
93
+ step "Creating $OUT_DIR, $BUILD_DIR, $STATE_DIR, $WATCH_DIR, $GET_TRANSFER_DIR"
94
+ for d in "$OUT_DIR" "$BUILD_DIR" "$STATE_DIR" "$WATCH_DIR" "$GET_TRANSFER_DIR"; do
95
+ mkdir -p "$d"
96
+ chown "$RUN_USER:$RUN_USER" "$d"
97
+ done
98
+
99
+ # The ZurRose mirror script used to exist only in $GET_TRANSFER_DIR on the old
100
+ # server, so deleting that host lost both the script and its download URL, and
101
+ # the cron line below — guarded by [ -x ] — went quietly dead. Symlink the
102
+ # tracked copy instead, so a rebuild restores the mirror from the checkout.
103
+ ln -sfn "$SCRIPT_DIR/get_transfer.sh" "$GET_TRANSFER_DIR/get_transfer.sh"
104
+ chown -h "$RUN_USER:$RUN_USER" "$GET_TRANSFER_DIR/get_transfer.sh"
105
+
106
+ # --- 5. Cron -----------------------------------------------------------------
107
+ # One /etc/cron.d file instead of the old scattered per-user crontab, so the
108
+ # whole schedule is visible (and re-creatable) in one place.
109
+ step "Writing /etc/cron.d/mediupdatexml"
110
+ cat > /etc/cron.d/mediupdatexml <<EOF
111
+ # mediupdatexml.oddb.org — nightly feed builds and download-site upkeep.
112
+ # Managed by oddb2xml/scripts/setup_new_server.sh; edit there, not here.
113
+ SHELL=/bin/bash
114
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
115
+ MAILTO=$RUN_USER
116
+
117
+ # 00:30 Mirror the ZurRose transfer.dat locally (seeds the build's downloads/).
118
+ 30 0 * * * $RUN_USER [ -x $GET_TRANSFER_DIR/get_transfer.sh ] && $GET_TRANSFER_DIR/get_transfer.sh >> $STATE_DIR/get_transfer.log 2>&1
119
+
120
+ # 01:00 Nightly oddb2xml build: default + artikelstamm (v6/v5) + 45/50/55.
121
+ 0 1 * * * $RUN_USER $SCRIPT_DIR/run_oddb2xml.sh >> $STATE_DIR/run_oddb2xml.log 2>&1
122
+
123
+ # 03:00 rust2xml's own Artikelstamm, published under /artikelstamm/rust2xml/.
124
+ 0 3 * * * $RUN_USER [ -x $RUST2XML_DIR/scripts/run_artikelstamm.sh ] && $RUST2XML_DIR/scripts/run_artikelstamm.sh >> $STATE_DIR/run_artikelstamm.log 2>&1
125
+
126
+ # 04:30 aips2sqlite Fachinfo XMLs + AmiKo DBs + Swissmedic sequences,
127
+ # published under /aips2sqlite/.
128
+ 30 4 * * * $RUN_USER [ -x $AIPS_DIR/scripts/generate_aips_fi ] && $AIPS_DIR/scripts/generate_aips_fi >> $STATE_DIR/generate_aips_fi.log 2>&1
129
+
130
+ # hourly Refresh the landing page (live counts + visitor graph).
131
+ 5 * * * * $RUN_USER $SCRIPT_DIR/generate_index_html.sh $OUT_DIR >> $STATE_DIR/index_html.log 2>&1
132
+
133
+ # every 30 min Rebuild automatically once Swissmedic answers again after a block.
134
+ */30 * * * * $RUN_USER $SCRIPT_DIR/swissmedic_watch.sh
135
+ EOF
136
+ chmod 644 /etc/cron.d/mediupdatexml
137
+
138
+ step "Writing /etc/logrotate.d/mediupdatexml"
139
+ cat > /etc/logrotate.d/mediupdatexml <<EOF
140
+ $STATE_DIR/*.log $WATCH_DIR/*.log {
141
+ weekly
142
+ rotate 8
143
+ compress
144
+ delaycompress
145
+ missingok
146
+ notifempty
147
+ su $RUN_USER $RUN_USER
148
+ create 0644 $RUN_USER $RUN_USER
149
+ }
150
+ EOF
151
+ chmod 644 /etc/logrotate.d/mediupdatexml
152
+
153
+ # --- 6. aips2sqlite (JRE, checkout, /aips2sqlite alias target) ----------------
154
+ # The landing page links a whole section of aips2sqlite output; without this the
155
+ # alias target is missing and every /aips2sqlite/ link answers 403.
156
+ if [[ "${SKIP_AIPS:-0}" != "1" ]]; then
157
+ step "Provisioning aips2sqlite (setup_aips2sqlite.sh)"
158
+ SKIP_CRON=1 "$SCRIPT_DIR/setup_aips2sqlite.sh" # cron entry already written above
159
+ fi
160
+
161
+ # --- 7. Apache vhost + HTTPS -------------------------------------------------
162
+ if [[ "${SKIP_WEB:-0}" != "1" ]]; then
163
+ step "Provisioning the Apache vhost (setup_mediupdatexml_web.sh)"
164
+ "$SCRIPT_DIR/setup_mediupdatexml_web.sh"
165
+ fi
166
+
167
+ cat <<EOF
168
+
169
+ ==> Host provisioning done.
170
+
171
+ Next steps (as $RUN_USER, not root):
172
+ 1. ZurRose mirror (needed by step 3): $GET_TRANSFER_DIR/get_transfer.sh
173
+ 2. Rust toolchain for the 03:00 job: $SCRIPT_DIR/setup_rust2xml.sh
174
+ 3. First build (takes ~1-2 h): $SCRIPT_DIR/run_oddb2xml.sh
175
+ 4. First Fachinfo run (takes ~1 h): $AIPS_DIR/scripts/generate_aips_fi
176
+ 5. Landing page: $SCRIPT_DIR/generate_index_html.sh $OUT_DIR
177
+
178
+ Step 1 before step 3: without $GET_TRANSFER_DIR/TRANSFER.ZIP the build falls
179
+ back to http://pillbox.oddb.org/TRANSFER.ZIP, which is itself only a mirror of
180
+ this job and has gone stale (and refused connections at 01:00) before.
181
+
182
+ Check the schedule with: cat /etc/cron.d/mediupdatexml
183
+ EOF
@@ -0,0 +1,50 @@
1
+ #!/usr/bin/env bash
2
+ #
3
+ # setup_rust2xml.sh — install the Rust toolchain and build rust2xml, so the
4
+ # 03:00 cron job (rust2xml/scripts/run_artikelstamm.sh) can publish its
5
+ # Artikelstamm under https://mediupdatexml.oddb.org/artikelstamm/rust2xml/.
6
+ #
7
+ # Run as the build user (NOT root): scripts/setup_rust2xml.sh
8
+ #
9
+ # Idempotent: re-running updates the toolchain and rebuilds if sources changed.
10
+ # rustup is used rather than Debian's rustc because run_artikelstamm.sh sources
11
+ # $HOME/.cargo/env to find cargo under cron's minimal PATH.
12
+ #
13
+ # Configurable via environment:
14
+ # RUST2XML_DIR rust2xml checkout (default $HOME/software/rust2xml)
15
+ #
16
+ set -euo pipefail
17
+
18
+ RUST2XML_DIR="${RUST2XML_DIR:-$HOME/software/rust2xml}"
19
+
20
+ step() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
21
+
22
+ if [[ $EUID -eq 0 ]]; then
23
+ echo "Run this as the build user, not root (rustup installs into \$HOME)." >&2
24
+ exit 1
25
+ fi
26
+ if [[ ! -d "$RUST2XML_DIR" ]]; then
27
+ echo "rust2xml checkout not found at $RUST2XML_DIR" >&2
28
+ echo " git clone https://github.com/zdavatz/rust2xml $RUST2XML_DIR" >&2
29
+ exit 1
30
+ fi
31
+
32
+ if [[ ! -x "$HOME/.cargo/bin/cargo" ]]; then
33
+ step "Installing the Rust toolchain (rustup, stable)"
34
+ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --no-modify-path
35
+ else
36
+ step "Updating the Rust toolchain"
37
+ "$HOME/.cargo/bin/rustup" update stable
38
+ fi
39
+
40
+ # shellcheck disable=SC1091
41
+ source "$HOME/.cargo/env"
42
+ cargo --version
43
+
44
+ step "Building rust2xml (release)"
45
+ cd "$RUST2XML_DIR"
46
+ cargo build --release --bin rust2xml
47
+
48
+ step "Done"
49
+ echo "Binary: $RUST2XML_DIR/target/release/rust2xml"
50
+ echo "The 03:00 cron entry in /etc/cron.d/mediupdatexml picks it up from here."
@@ -27,10 +27,20 @@ STATE_DIR="${STATE_DIR:-${OUT_DIR%/}-watch}"
27
27
  ARTICLE_XML="$OUT_DIR/default/oddb_article.xml"
28
28
  CANARY_URL="https://www.swissmedic.ch/swissmedic/de/home/services/listen_neu.html"
29
29
 
30
- # Match the nightly cron's rbenv environment (the repo's .ruby-version points at
31
- # a Ruby that isn't installed here; cron pins RBENV_VERSION instead).
32
- export RBENV_VERSION="${RBENV_VERSION:-3.4.5}"
33
- export PATH="/home/zdavatz/.rbenv/shims:/usr/bin:/bin"
30
+ # Cron hands us a minimal PATH, so spell out where the oddb2xml binstub can be,
31
+ # covering every way this host might have installed it: Debian's system Ruby
32
+ # puts a root `gem install` in /usr/local/bin, an unprivileged one in
33
+ # ~/.local/share/gem/ruby/X.Y.0/bin, and the pre-2026-07 box used rbenv shims.
34
+ # Getting this wrong is silent until build time — the watcher launched a run on
35
+ # the rebuilt server with an rbenv-only PATH and it died with exit 127.
36
+ export PATH="/usr/local/bin:/usr/bin:/bin"
37
+ HOME_DIR="${HOME:-/home/zdavatz}"
38
+ if [[ -d "$HOME_DIR/.rbenv/shims" ]]; then
39
+ export RBENV_VERSION="${RBENV_VERSION:-3.4.5}"
40
+ export PATH="$HOME_DIR/.rbenv/shims:$PATH"
41
+ fi
42
+ GEM_USER_BIN="$(ruby -e 'print Gem.user_dir' 2>/dev/null)/bin"
43
+ [[ -d "$GEM_USER_BIN" ]] && export PATH="$GEM_USER_BIN:$PATH"
34
44
 
35
45
  mkdir -p "$STATE_DIR"
36
46
  LOG="$STATE_DIR/swissmedic_watch.log"
data/spec/fhir_spec.rb CHANGED
@@ -356,3 +356,32 @@ describe "FHIR Indikationscode support" do
356
356
  end
357
357
  end
358
358
  end
359
+
360
+ # BAG moved the export on 24.08.2026 and announced it the next evening. The old
361
+ # -latest- alias answers 404 while the old dated snapshots stay in place, so the
362
+ # failure is an import that quietly downloads nothing - the same shape that let
363
+ # ch.oddb.org run for twelve mornings reporting exit 0 and importing nothing.
364
+ describe Oddb2xml::FhirDownloader do
365
+ it "builds the per-language urls under the publication path" do
366
+ base = Oddb2xml::FhirDownloader::BASE_URL
367
+ path = Oddb2xml::FhirDownloader::STATIC_FHIR_PATH
368
+ prefix = Oddb2xml::FhirDownloader::FILE_PREFIX
369
+
370
+ expect(path).to eq "/static/sl/publication/fhir"
371
+ expect(prefix).to eq "foph-sl-publication"
372
+ expect(Oddb2xml::FhirDownloader::LANGUAGES).to eq %w[de fr it]
373
+
374
+ url = "#{base}#{path}/#{prefix}-latest-de.ndjson"
375
+ expect(url).to eq "https://epl.bag.admin.ch/static/sl/publication/fhir/" \
376
+ "foph-sl-publication-latest-de.ndjson"
377
+ end
378
+
379
+ it "no longer points at the retired export path" do
380
+ combined = [
381
+ Oddb2xml::FhirDownloader::STATIC_FHIR_PATH,
382
+ Oddb2xml::FhirDownloader::FILE_PREFIX
383
+ ].join(" ")
384
+ expect(combined).not_to include "foph-sl-export"
385
+ expect(combined).not_to eq "/static/fhir"
386
+ end
387
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: oddb2xml
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.0.31
4
+ version: 3.0.32
5
5
  platform: ruby
6
6
  authors:
7
7
  - Yasuhiro Asaka, Zeno R.R. Davatz, Niklaus Giger
@@ -500,8 +500,12 @@ files:
500
500
  - oddb2xml.xsd
501
501
  - oddb_calc.xsd
502
502
  - scripts/generate_index_html.sh
503
+ - scripts/get_transfer.sh
503
504
  - scripts/run_oddb2xml.sh
505
+ - scripts/setup_aips2sqlite.sh
504
506
  - scripts/setup_mediupdatexml_web.sh
507
+ - scripts/setup_new_server.sh
508
+ - scripts/setup_rust2xml.sh
505
509
  - scripts/swissmedic_watch.sh
506
510
  - scripts/transfer.sh
507
511
  - scripts/visitor_stats.py
@@ -607,7 +611,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
607
611
  - !ruby/object:Gem::Version
608
612
  version: '0'
609
613
  requirements: []
610
- rubygems_version: 3.6.9
614
+ rubygems_version: 3.6.7
611
615
  specification_version: 4
612
616
  summary: oddb2xml creates xml files.
613
617
  test_files: