oddb2xml 3.0.32 → 3.0.34

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6526c05c2efd31be9fff9279dd24e817a3ae94363c9b30d8286dbef0510596d5
-  data.tar.gz: c8c835826c01e71719efbef0c73cb0f4805037dac8a7d89dcd82a7d58df41deb
+  metadata.gz: aba1912f4878121d6d5aa1424b2c1f33f72b1eaceefd21d7b973753209b97c6e
+  data.tar.gz: 89ba2fffeefae9dffb569cb94be5308b67a7e847813e6de9771d5fee5a626996
 SHA512:
-  metadata.gz: 8358ad2fd07b3cad7d4e87c7b364fa8ea0a3f68742c547d9781dbf61614c6a2d85b3f95bd5ee8f3c7d0aa48bb08b8e2b45eb279382778e25a5333f6928440fed
-  data.tar.gz: e332eb5a017afd15dc78663d12f2956469ec2ed4e9de4742b28771f84dd10983593787c2a611327c130202ed82f34c56c730b75c24e9eb4ad49912cdb3cb5519
+  metadata.gz: b9c54d7a0f8cf8189c5703e36aa309c40302b8880a65e552122dc8728d9827f7b15f1ae1c976abab6a342efb6547d31dc44e957ffb8b5b93ef2adb6be7cced99
+  data.tar.gz: d185bc2f3f9eadd52e618038fbae1b9050aa65ed4e36e0116cba42a68b9948d19a6794e300a131328c4db5a3075f1f1b250c7a43ef7a481129a4aaa3efd59fd4

data/CLAUDE.md

@@ -14,7 +14,7 @@ The system follows a **download → extract → build → compress** pipeline:
 
 2. **Downloaders** — 11 subclasses of `Downloader`, each fetching from a specific Swiss data source. 10 live in `lib/oddb2xml/downloader.rb`; the FHIR downloader lives in `lib/oddb2xml/fhir_support.rb`. Files cached in `./downloads/`.
 
-3. **Extractors** (`lib/oddb2xml/extractor.rb`) — Matching extractor classes that parse downloaded files into Ruby hashes. Formats include XML (nokogiri/sax-machine), XLSX (rubyXL), CSV, and fixed-width text. Refdata uses the new SwissReg XML format from a zip download (`files.refdata.ch`).
+3. **Extractors** (`lib/oddb2xml/extractor.rb`) — Matching extractor classes that parse downloaded files into Ruby hashes. Formats include XML (nokogiri/sax-machine), XLSX (rubyXL), CSV, and fixed-width text. Refdata uses the new SwissReg XML format from a zip download (`files.refdata.ch`). **Barcode-less Refdata articles (3.0.33 onwards):** Refdata publishes a few PHARMA articles with no `<DataCarrierIdentifier>` — the Swiss Red Cross blood products under the collective registration 99999, which appeared in August 2026 — and `RefdataExtractor#to_hash` called `.size` on that `nil`. `Cli#download_as` rescues extractor errors into `Oddb2xml.log`, which prints **nothing** without `--log` (issue #102 wanted the build to survive a Refdata outage), so the `NoMethodError` silently dropped all 16'093 Refdata PHARMA articles while the run still reported success: `<SMNO>` in `oddb_article.xml` fell from ~15'300 to exactly 10'210 (the rest arriving from the BAG FHIR SL feed) and stayed frozen there for twelve days, then to 50 once the 3.0.32 url move removed FHIR too. NONPHARMA was unaffected because the product-type `next` runs before the barcode is read. Such records are now skipped, and both rescued Refdata failures print a warning to stderr. See GitHub issue #122.
 
 4. **Builder** (`lib/oddb2xml/builder.rb`) — The largest file (~1900 lines). Merges extracted data and generates output XML/DAT files. Methods follow `prepare_*` (data assembly) and `build_*` (output generation) naming.
 
@@ -22,13 +22,13 @@ The system follows a **download → extract → build → compress** pipeline:
 
 6. **Compressor** (`lib/oddb2xml/compressor.rb`) — Optional ZIP/TAR.GZ output compression.
 
-7. **FHIR support** (`lib/oddb2xml/fhir_support.rb`) — Self-contained module providing `FhirDownloader` and FHIR NDJSON parsing. Activated via `--fhir` (or `--fhir-url=<URL>`). Downloads per-language NDJSON files (`foph-sl-publication-latest-{de,fr,it}.ndjson`) from `epl.bag.admin.ch/static/sl/publication/fhir` (BAG moved the export there on 24.08.2026; the old `/static/fhir/foph-sl-export-*` answers 404 for `-latest-` while the old dated snapshots remain, so the failure looks like an import that does nothing) to populate French and Italian product names/descriptions. Maps legal status codes `756005022007` and `756005022008` to Swissmedic category D. Reads the BAG **Indikationscode** (`XXXXX.NN`) from the explicit `indicationCode` extension on each `RegulatedAuthorization.indication[].extension[regulatedAuthorization-limitation]` (BAG SL FHIR export >= v2.0.5; handled from 3.0.10). The BAG changelog states the limitation code (`ClinicalUseDefinition.id`) and the indication code are **independent** fields, so the older derivation — combining each indication CUD's `.NN` id-suffix with the reimbursement RA's `FOPHDossierNumber` — is kept only as a fallback for feeds lacking the extension. Exposed as `item[:indication_codes]` and per-package `:indication_codes` (each entry a `{code:, cud_id:, text:}` hash, where `cud_id` is the `limitationIndication` CUD reference used to resolve the text). From 3.0.7 onwards, `Builder#build_product` emits one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN">limitation text</INDICATION_CODE>` child per indication on every `<PRD>` in `oddb_product.xml`; live feed numbers: 539 products / 1,293 codes / 100 % with non-empty indication text. Mandatory on prescriptions/invoices for SL price-model drugs from 2026-07-01 — see issue [#113](https://github.com/zdavatz/oddb2xml/issues/113). **Limitation texts** (3.0.8 onwards): the `regulatedAuthorization-limitation` extension has no inline `limitationText` in the live BAG feed — it carries a `limitationIndication` reference to a `ClinicalUseDefinition` whose `indication.diseaseSymptomProcedure.concept.text` is the actual text. The parser stores the ref as `cud_ref` on each Limitation, `Bundle#cud_text_by_id` resolves DE, and `merge_language` propagates FR/IT from the per-language NDJSON files via the same CUD id. Coverage on the live feed jumped from 0 / 9'108 to 9'108 / 9'108 (issue [#116](https://github.com/zdavatz/oddb2xml/issues/116)). **Limitation code / LIMNAMEBAG** (3.0.12 onwards): FHIR has no native BAG limitation code (LIMCD), so `create_limitations_for_package` sets `LimitationCode = cud_ref` (the `limitationIndication` CUD id) instead of `""`. Without this, every FHIR limitation shared an empty `:code`; `Builder#build_artikelstamm` groups its `<LIMITATIONS>` section by code, so all of them collapsed into a single `<LIMITATION>` with an empty `<LIMNAMEBAG>` and only one text survived. Using the CUD id as the key makes each distinct limitation emit and be referenced from its `<PRODUCT>`. The downstream `bin/check_artikelstamm` (`semantic_check.rb`) also crashed on the lone-element output because Ox `:hash_no_attrs` collapses a one-child section into a Hash (and an empty one into nil) — `SemanticCheckXML#get_items` now normalises every section to an Array. **v6 Artikelstamm / per-article INDC (3.0.26 onwards):** `--artikelstamm` now emits the **Elexis Artikelstamm v6** format (namespace `http://elexis.ch/Elexis_Artikelstamm_v6`, file `artikelstamm_DDMMYYYY_v6.xml`/`.csv`, validated against the bundled `Elexis_Artikelstamm_v6.xsd`) — replacing v5. The new piece is a per-`<ITEM>` `<ARTSL>` block carrying the BAG Indikationscodes (issue [#113](https://github.com/zdavatz/oddb2xml/issues/113)): `<PM>true</PM>` plus one `<ARTLIM>` per limitation with `<LIMCD>` (= `cud_ref`, the BAG limitation code), `<INDCD>` (the `XXXXX.NN` indication code from the `indicationCode` extension), and `<VDAT>`/`<VTDAT>` (period start/end). To feed it, `create_limitations_for_package` now also carries `IndicationCode` (→ per-package `:indcd`) and `ValidThruDate` (→ `:vtdate`) on each limitation; `Builder#append_artsl`/`elexis_datetime` emit one `<ARTLIM>` per limitation that has a non-empty `:indcd` (so non-price-model items get no `<ARTSL>`). `PM` is always `true` here because the indication code is required only for SL price-model drugs, which is exactly the set of items that reach this block. The bundled `Elexis_Artikelstamm_v6.xsd` is the canonical MEDEVIT schema extended with oddb2xml's historical Italian elements (`DSCRI` on PRODUCT/LIMITATION/ITEM, `DOSAGE_FORMI` on ITEM) so the output still validates. The legacy `--no-fhir` path emits no `<ARTSL>` (no FHIR limitations). **Per-article INDC in the `-e`/`-b` feeds (3.0.27 onwards):** the same indication codes are also emitted per `<ART>` in `oddb_article.xml` via `Builder#append_indication_codes` — one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN" limcd="DRUG.NN" vdat=… vtdat=…>limitation text</INDICATION_CODE>` per limitation that carries an `:indcd` (sourced from the article's `pac[:limitations]`). This reuses the flat `<INDICATION_CODE>` element already emitted on `<PRD>` in `oddb_product.xml` (since 3.0.7), enriched with the limitation code and validity dates so it carries the full `<ARTSL>` payload. The bundled `oddb2xml.xsd` now defines `<INDICATION_CODE>` (simpleContent + `code`/`cud_id`/`limcd`/`vdat`/`vtdat` attributes) and references it from both `<ART>` and `<PRD>` — the `<PRD>` reference had been missing since the 3.0.7 addition, so the FHIR product feed only validated once this was added. **Legacy v5 opt-in (3.0.28 onwards):** `--artikelstamm-v5` additionally emits the older Artikelstamm v5 file alongside the v6 one (for consumers not yet migrated). It implies `--artikelstamm`; `Options.parse` sets `@opts[:artikelstamm] = true`. `build_artikelstamm` takes a `version:` keyword (default `6`) that drives the namespace (`Elexis_Artikelstamm_v#{version}`) and CSV/XML filename suffix; `build_artikelstamm_v5` calls it with `version: 5`. The only content difference is that the v5 path skips `append_artsl` (the v5 schema has no `<ARTSL>` element). The CLI `files` map adds an `:artikelstamm_v5` entry (→ `build_artikelstamm_v5` via the `to_xml` subject dispatch) and validates it against the bundled `Elexis_Artikelstamm_v5.xsd`. Both files come from one build — the `@prepared` guard means the shared download/prepare phase runs once. **Selbstbehalt / costShare (3.0.31 onwards):** the `reimbursementSL` extension's `costShare` valueInteger is the BAG Selbstbehalt percentage; it maps to the legacy `FlagSB` semantics of the old BAG `Preparations.xml` — `40` → `item[:deductible] = "Y"` (raised Selbstbehalt: `<SLOPLUS>1</SLOPLUS>` in `oddb_article.xml`, `<DEDUCTIBLE>40</DEDUCTIBLE>` in the Artikelstamm), `10` → `"N"` (ordinary Selbstbehalt: `SLOPLUS` 2 / `DEDUCTIBLE` 10), `20` → `deductible20 = "Y"` (transitional rate, issue #81). Until 3.0.31 the mapping was inverted (`10` → `"Y"`), so **every** SL article claimed a 40 % Selbstbehalt. On the live feed only 385 of 10'388 packages (222 preparations) carry `costShare` 40; the value never mixes within one preparation, so keeping the flag at sequence level is safe.
+7. **FHIR support** (`lib/oddb2xml/fhir_support.rb`) — Self-contained module providing `FhirDownloader` and FHIR NDJSON parsing. Activated via `--fhir` (or `--fhir-url=<URL>`). Downloads per-language NDJSON files (`foph-sl-publication-latest-{de,fr,it}.ndjson`) from `epl.bag.admin.ch/static/sl/publication/fhir` (**3.0.32 onwards**; BAG moved the export there on 24.08.2026, announced it the next evening, and oddb2xml followed in 3.0.32; the old `/static/fhir/foph-sl-export-*` answers 404 for `-latest-` while the old dated snapshots remain, so the failure looks like an import that does nothing) to populate French and Italian product names/descriptions. Maps legal status codes `756005022007` and `756005022008` to Swissmedic category D. Reads the BAG **Indikationscode** (`XXXXX.NN`) from the explicit `indicationCode` extension on each `RegulatedAuthorization.indication[].extension[regulatedAuthorization-limitation]` (BAG SL FHIR export >= v2.0.5; handled from 3.0.10). The BAG changelog states the limitation code (`ClinicalUseDefinition.id`) and the indication code are **independent** fields, so the older derivation — combining each indication CUD's `.NN` id-suffix with the reimbursement RA's `FOPHDossierNumber` — is kept only as a fallback for feeds lacking the extension. Exposed as `item[:indication_codes]` and per-package `:indication_codes` (each entry a `{code:, cud_id:, text:}` hash, where `cud_id` is the `limitationIndication` CUD reference used to resolve the text). From 3.0.7 onwards, `Builder#build_product` emits one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN">limitation text</INDICATION_CODE>` child per indication on every `<PRD>` in `oddb_product.xml`; live feed numbers: 539 products / 1,293 codes / 100 % with non-empty indication text. Mandatory on prescriptions/invoices for SL price-model drugs from 2026-07-01 — see issue [#113](https://github.com/zdavatz/oddb2xml/issues/113). **Limitation texts** (3.0.8 onwards): the `regulatedAuthorization-limitation` extension has no inline `limitationText` in the live BAG feed — it carries a `limitationIndication` reference to a `ClinicalUseDefinition` whose `indication.diseaseSymptomProcedure.concept.text` is the actual text. The parser stores the ref as `cud_ref` on each Limitation, `Bundle#cud_text_by_id` resolves DE, and `merge_language` propagates FR/IT from the per-language NDJSON files via the same CUD id. Coverage on the live feed jumped from 0 / 9'108 to 9'108 / 9'108 (issue [#116](https://github.com/zdavatz/oddb2xml/issues/116)). **Limitation code / LIMNAMEBAG** (3.0.12 onwards): FHIR has no native BAG limitation code (LIMCD), so `create_limitations_for_package` sets `LimitationCode = cud_ref` (the `limitationIndication` CUD id) instead of `""`. Without this, every FHIR limitation shared an empty `:code`; `Builder#build_artikelstamm` groups its `<LIMITATIONS>` section by code, so all of them collapsed into a single `<LIMITATION>` with an empty `<LIMNAMEBAG>` and only one text survived. Using the CUD id as the key makes each distinct limitation emit and be referenced from its `<PRODUCT>`. The downstream `bin/check_artikelstamm` (`semantic_check.rb`) also crashed on the lone-element output because Ox `:hash_no_attrs` collapses a one-child section into a Hash (and an empty one into nil) — `SemanticCheckXML#get_items` now normalises every section to an Array. **v6 Artikelstamm / per-article INDC (3.0.26 onwards):** `--artikelstamm` now emits the **Elexis Artikelstamm v6** format (namespace `http://elexis.ch/Elexis_Artikelstamm_v6`, file `artikelstamm_DDMMYYYY_v6.xml`/`.csv`, validated against the bundled `Elexis_Artikelstamm_v6.xsd`) — replacing v5. The new piece is a per-`<ITEM>` `<ARTSL>` block carrying the BAG Indikationscodes (issue [#113](https://github.com/zdavatz/oddb2xml/issues/113)): `<PM>true</PM>` plus one `<ARTLIM>` per limitation with `<LIMCD>` (= `cud_ref`, the BAG limitation code), `<INDCD>` (the `XXXXX.NN` indication code from the `indicationCode` extension), and `<VDAT>`/`<VTDAT>` (period start/end). To feed it, `create_limitations_for_package` now also carries `IndicationCode` (→ per-package `:indcd`) and `ValidThruDate` (→ `:vtdate`) on each limitation; `Builder#append_artsl`/`elexis_datetime` emit one `<ARTLIM>` per limitation that has a non-empty `:indcd` (so non-price-model items get no `<ARTSL>`). `PM` is always `true` here because the indication code is required only for SL price-model drugs, which is exactly the set of items that reach this block. The bundled `Elexis_Artikelstamm_v6.xsd` is the canonical MEDEVIT schema extended with oddb2xml's historical Italian elements (`DSCRI` on PRODUCT/LIMITATION/ITEM, `DOSAGE_FORMI` on ITEM) so the output still validates. The legacy `--no-fhir` path emits no `<ARTSL>` (no FHIR limitations). **Per-article INDC in the `-e`/`-b` feeds (3.0.27 onwards):** the same indication codes are also emitted per `<ART>` in `oddb_article.xml` via `Builder#append_indication_codes` — one `<INDICATION_CODE code="XXXXX.NN" cud_id="DRUG.NN" limcd="DRUG.NN" vdat=… vtdat=…>limitation text</INDICATION_CODE>` per limitation that carries an `:indcd` (sourced from the article's `pac[:limitations]`). This reuses the flat `<INDICATION_CODE>` element already emitted on `<PRD>` in `oddb_product.xml` (since 3.0.7), enriched with the limitation code and validity dates so it carries the full `<ARTSL>` payload. The bundled `oddb2xml.xsd` now defines `<INDICATION_CODE>` (simpleContent + `code`/`cud_id`/`limcd`/`vdat`/`vtdat` attributes) and references it from both `<ART>` and `<PRD>` — the `<PRD>` reference had been missing since the 3.0.7 addition, so the FHIR product feed only validated once this was added. **Legacy v5 opt-in (3.0.28 onwards):** `--artikelstamm-v5` additionally emits the older Artikelstamm v5 file alongside the v6 one (for consumers not yet migrated). It implies `--artikelstamm`; `Options.parse` sets `@opts[:artikelstamm] = true`. `build_artikelstamm` takes a `version:` keyword (default `6`) that drives the namespace (`Elexis_Artikelstamm_v#{version}`) and CSV/XML filename suffix; `build_artikelstamm_v5` calls it with `version: 5`. The only content difference is that the v5 path skips `append_artsl` (the v5 schema has no `<ARTSL>` element). The CLI `files` map adds an `:artikelstamm_v5` entry (→ `build_artikelstamm_v5` via the `to_xml` subject dispatch) and validates it against the bundled `Elexis_Artikelstamm_v5.xsd`. Both files come from one build — the `@prepared` guard means the shared download/prepare phase runs once. **Selbstbehalt / costShare (3.0.31 onwards):** the `reimbursementSL` extension's `costShare` valueInteger is the BAG Selbstbehalt percentage; it maps to the legacy `FlagSB` semantics of the old BAG `Preparations.xml` — `40` → `item[:deductible] = "Y"` (raised Selbstbehalt: `<SLOPLUS>1</SLOPLUS>` in `oddb_article.xml`, `<DEDUCTIBLE>40</DEDUCTIBLE>` in the Artikelstamm), `10` → `"N"` (ordinary Selbstbehalt: `SLOPLUS` 2 / `DEDUCTIBLE` 10), `20` → `deductible20 = "Y"` (transitional rate, issue #81). Until 3.0.31 the mapping was inverted (`10` → `"Y"`), so **every** SL article claimed a 40 % Selbstbehalt. On the live feed only 385 of 10'388 packages (222 preparations) carry `costShare` 40; the value never mixes within one preparation, so keeping the flag at sequence level is safe.
 
 8. **Refdata cleanup** (`lib/oddb2xml/refdata_cleanup.rb`) — Compensates for known data-quality issues in upstream Refdata.Articles.xml before they reach the output. Each fix is guarded by a Swissmedic-side heuristic (e.g. comma in `substance_swissmedic` to distinguish mono products from real combinations). Currently fixes (a) the doubled-dose template bug (`X mg / X mg / Stk`, `fix_double_dose`, guarded by `single_substance?`); (b) the spelled-out German galenic form `Retardtabletten` → house-style abbreviation `Ret Tabl` (`normalize_galenic_form` / `GALENIC_NORMALISATIONS`, issue #112 case #13, e.g. RINVOQ — a narrow word-boundary substitution that leaves legitimate brand suffixes like `TRAMAL retard` and Mepha's `Lactab` untouched); and (c) dose info Refdata dropped from `<FullName>`, sourced from the Swissmedic composition string `pack[:composition_swissmedic]` — `fix_missing_combo_dose` (#6, appends a combination's 2nd component strength), `fix_missing_dose` (#4, inserts a mono product's missing strength before the pack count), `fix_missing_volume` (#7, appends an injectable's per-pen volume); and (d) 50-char-truncation repairs — `fix_truncated_metoject` (#1, rebuilds METOJECT Autoinjektor names from the intact `<brand> Autoinjektor <dose>/<vol>` prefix + Swissmedic `size`, localised DE/FR/IT) and `fix_truncated_volume_unit` (#3, restores the cut `ml` of the VERACTIV Vitamin D3 drops). The (c) and (d) fixes are scoped to explicit IKSNR allow-lists (`COMBO_DOSE_IKSNR`/`MISSING_DOSE_IKSNR`/`MISSING_VOLUME_IKSNR`/`METOJECT_IKSNR`/`VERACTIV_VITD3_IKSNR`): a dry run proved a blanket heuristic mis-fires on hundreds of legitimate names (sodium counter-ion doses, strength-less phyto/powder products, concentration names like `CIMZIA 200 mg/ml`), so only catalogued registrations are touched — add an IKSNR to grow coverage. Called from `Builder#apply_refdata_description_cleanups!` at the start of `prepare_articles`. See GitHub issue #112 for the catalogue.
 
 9. **Chapter-70 hack** (`lib/oddb2xml/chapter_70_hack.rb`) — Legacy scraper for the SL "Komplementärarzneimittel" products (homeopathic/anthroposophic/phytotherapeutic), called only from `Builder#build_artikelstamm`. **Deprecated / non-FHIR only (3.0.11 onwards):** the source page `varia_De.htm` was rebuilt as a JavaScript SPA with no static data table, so the scraper now returns nothing there. These products + limitations now come through the FHIR feed (SL classification `20. KOMPLEMENTÄRARZNEIMITTEL`, 221 products on the live DE feed with real GTINs and limitation texts), so `build_artikelstamm` **skips the scraper entirely when `@options[:fhir]`** (the default for `--artikelstamm` since 3.0.9). In `--no-fhir` mode the scraper degrades gracefully (skips non-row/`<script>` nodes and empty tables, warns, returns `[]`) instead of raising `NoMethodError`. See GitHub issue #118.
 
-10. **Weleda / Kapitel-70 SL recovery** (`lib/oddb2xml/weleda_sl.rb`, 3.0.21 onwards) — Recovers the SL flag and public price for chapter-70 complementary medicines that are **missing from the FHIR feed** (the partial-replacement gap left by the dead chapter_70_hack, issue #118/#121). Many are magistral Weleda preparations with a `7611916…` trade GTIN that arrive only via ZurRose — with no SL flag and a blanked Publikumspreis (issue #117). `WeledaSL.load` joins two CSVs (downloaded at runtime from `github.com/zdavatz/oddb2xml_files` via `WeledaDownloader` / `BagSlGroupPricesDownloader`, bundled fallback copies under `data/`): `weleda_arzneimittel.csv` (GTIN → `abgabekategorie` SL flag + `csl` = **Pharma-Gruppen-Code**) and `bag_sl_group_prices.csv` (Pharma-Gruppen-Code → public price). `weleda_arzneimittel.csv` is **regenerated** by the Rust tool `weleda_scraper/` in the `oddb2xml_files` repo (`scraper --update weleda`, prompts for the medical.weleda.ch `PHPSESSID` cookie — never stored): it walks the paginated Arzneimittel-Verzeichnis listing + per-product detail pages and rewrites the CSV with exactly the currently-listed products (delisted rows dropped), preserving the on-disk format (column order, UTF-8, CRLF, quote-when-necessary, sorted by `id`) so oddb2xml reads it unchanged. The price table is extracted **offline** from the BAG SL definition PDF *"Homoeopathica, Anthroposophica, Allergene"* via `tools/generate_bag_sl_group_prices.rb` (uses system `pdftotext`; **no runtime PDF gem** — `pdf-reader`'s `afm` dep now needs Ruby ≥ 3.2, which would break the gem's Ruby floor). The join is **GTIN → csl → price**, honouring an `N x <code>` package multiplier (price = N × group price). Produces `gtin => {sl:, price:, csl:, abgabe:}` (SL rows only; ~515 priced on the live feed). **WALA products (3.0.22 onwards):** a third runtime CSV `wala_arzneimittel.csv` (GTIN prefix `7640187…`, `WalaDownloader`, bundled fallback) is merged into the same map via `WeledaSL.build_wala_map`. Its layout differs: `;`-separated with a BOM, no `/ SL` column (a row is SL when it carries a `CSL-Code` = Kapitel-70.01 group code), and the public **package** price is given inline in the `CSL 70.01.` column — **already multiplied for the pack size** (the multiplier appears only in the galenic-form text, e.g. `Solutio ad inj. 10 x 1 ml`), so it is taken **verbatim** rather than re-joined against `bag_sl_group_prices.csv` (which holds the per-unit price and would yield 1/10 of the package price for ~120 multi-unit packs). 320 WALA SL products on the live file; Weleda wins on the (unlikely) GTIN collision. `Builder#build_artikelstamm` consumes it (CLI sets `builder.weleda_sl` only for `--artikelstamm`): for any GTIN **absent from the FHIR NDJSON** it emits `<SL_ENTRY>true</SL_ENTRY>` and `<PPUB>` from the BAG group price, mirroring the old chapter-70 behaviour (`PHARMATYPE "P"`). **The FHIR/ZurRose price always wins** — the group price only fills a gap; a zeroed ZurRose `"0.00"` pub price is treated as absent so the gap-fill can apply. Match is **by GTIN only** (no pharmacode); the Swissmedic dispensing category is untouched (still from `Swissmedic_Packungen.xlsx`). The Artikelstamm output gets `<SL_ENTRY>` + `<PPUB>`; for the `-e`/`--extended` and `-b`/`--firstbase` product feeds the BAG public price is also added to `oddb_article.xml` as an `<ARTPRI><PTYP>BAGPUB</PTYP>` entry (the raw, often-blanked `ZURROSEPUB` is preserved alongside it) — `build_article`, gated by the CLI loading `weleda_sl` when `extended || firstbase || artikelstamm`. See GitHub issue #121.
+10. **Weleda / Kapitel-70 SL recovery** (`lib/oddb2xml/weleda_sl.rb`, 3.0.21 onwards) — Recovers the SL flag and public price for chapter-70 complementary medicines that are **missing from the FHIR feed** (the partial-replacement gap left by the dead chapter_70_hack, issue #118/#121). Many are magistral Weleda preparations with a `7611916…` trade GTIN that arrive only via ZurRose — with no SL flag and a blanked Publikumspreis (issue #117). `WeledaSL.load` joins two CSVs (downloaded at runtime from `github.com/zdavatz/oddb2xml_files` via `WeledaDownloader` / `BagSlGroupPricesDownloader`, bundled fallback copies under `data/`): `weleda_arzneimittel.csv` (GTIN → `abgabekategorie` SL flag + `csl` = **Pharma-Gruppen-Code**) and `bag_sl_group_prices.csv` (Pharma-Gruppen-Code → public price). `weleda_arzneimittel.csv` is **regenerated** by the Rust tool `weleda_scraper/` in the `oddb2xml_files` repo (`scraper --update weleda`, prompts for the medical.weleda.ch `PHPSESSID` cookie — never stored): it walks the paginated Arzneimittel-Verzeichnis listing + per-product detail pages and rewrites the CSV with exactly the currently-listed products (delisted rows dropped), preserving the on-disk format (column order, UTF-8, CRLF, quote-when-necessary, sorted by `id`) so oddb2xml reads it unchanged. The price table is extracted **offline** from the BAG SL definition PDF *"Homoeopathica, Anthroposophica, Allergene"* via `tools/generate_bag_sl_group_prices.rb` (uses system `pdftotext`; **no runtime PDF gem** — `pdf-reader`'s `afm` dep now needs Ruby ≥ 3.2, which would break the gem's Ruby floor). The join is **GTIN → csl → price**, honouring an `N x <code>` package multiplier (price = N × group price). Produces `gtin => {sl:, price:, csl:, abgabe:}` (SL rows only; ~515 priced on the live feed). **WALA products (3.0.22 onwards):** a third runtime CSV `wala_arzneimittel.csv` (GTIN prefix `7640187…`, `WalaDownloader`, bundled fallback) is merged into the same map via `WeledaSL.build_wala_map`. Its layout differs: `;`-separated with a BOM, no `/ SL` column (a row is SL when it carries a `CSL-Code` = Kapitel-70.01 group code), and the public **package** price is given inline in the `CSL 70.01.` column — **already multiplied for the pack size** (the multiplier appears only in the galenic-form text, e.g. `Solutio ad inj. 10 x 1 ml`), so it is taken **verbatim** rather than re-joined against `bag_sl_group_prices.csv` (which holds the per-unit price and would yield 1/10 of the package price for ~120 multi-unit packs). 320 WALA SL products on the live file; Weleda wins on the (unlikely) GTIN collision. `Builder#build_artikelstamm` consumes it (CLI sets `builder.weleda_sl` only for `--artikelstamm`): for any GTIN **absent from the FHIR NDJSON** it emits `<SL_ENTRY>true</SL_ENTRY>` and `<PPUB>` from the BAG group price, mirroring the old chapter-70 behaviour (`PHARMATYPE "P"`). **The FHIR/ZurRose price always wins** — the group price only fills a gap; a zeroed ZurRose `"0.00"` pub price is treated as absent so the gap-fill can apply. Match is **by GTIN only** (no pharmacode); the Swissmedic dispensing category is untouched (still from `Swissmedic_Packungen.xlsx`). The Artikelstamm output gets `<SL_ENTRY>` + `<PPUB>`; for the `-e`/`--extended` and `-b`/`--firstbase` product feeds the BAG public price is also added to `oddb_article.xml` as an `<ARTPRI><PTYP>BAGPUB</PTYP>` entry (the raw, often-blanked `ZURROSEPUB` is preserved alongside it) — `build_article`, gated by the CLI loading `weleda_sl` when `extended || firstbase || artikelstamm`. **Source encoding pinned (3.0.34):** `WeledaSL.source` forces the three CSVs to UTF-8. It used to pass the downloaded body on as-is, so an ASCII-8BIT response (webmock, or any body served without a charset) made the first comparison against a UTF-8 literal raise `Encoding::CompatibilityError` — and `load` rescues **every** error into `{}`, so the entire recovery vanished with no message: no `<SL_ENTRY>`, no group price, build still "successful". Same silent-total-loss shape as issues #122 and #127; if Weleda items lose their SL flag, check the `WeledaSL:` log lines with `--log` before suspecting the data. See GitHub issue #121.
 
 11. **Rogger name preferences** (`lib/oddb2xml/rogger_names.rb`, `-r`/`--rogger`, 3.0.30 onwards) — Replaces the German article description (`desc_de`) with the preferred name from the **"Rogger Mediliste"** for every GTIN on that list. The list collects the name conflicts Frau Rogger (Vitabyte/Zur Rose, task #OX-5985-1594) reports between the Spirig/Galexis names and what Vitabyte shows from raw Refdata; most entries are the issue-#112 Refdata bugs with their corrected names (METOJECT truncation, CETIRIZIN missing strength, ATOVAQUON missing 2nd combo dose, MOUNJARO missing pen volume, RINVOQ `Retardtabletten`), plus two not (yet) catalogued in `refdata_cleanup.rb`: GABAPENTIN Spirig HC Kaps (IKSNR 66594, strength doubled as a glued brand token `Spirig HC 100mg Kaps 100 mg`) and MELATONIN Spirig HC 100 Stk (redundant `retard` before `Ret Tabl`). Source of truth is the shared Google Sheet "Rogger Mediliste" (link-shared read-only); `RoggerDownloader` fetches its CSV export (`GTIN,Mediname`) **directly** (`docs.google.com/spreadsheets/d/<id>/export?format=csv&gid=0`), so sheet edits reach the feeds without any release step, with a bundled fallback `data/rogger_liste.csv` (refresh at release time). `RoggerNames.load` returns `gtin => name` (never raises; UTF-8-forced because the list carries `µg`; `rogger_csv?` rejects non-CSV responses such as a Google sign-in page if the sheet ever loses its link-sharing, engaging the fallback). The override runs as the **last step of `Builder#apply_refdata_description_cleanups!`** (`apply_rogger_name_overrides!`), so it sees and wins over the issue-#112 cleanups; the list is German-only, FR/IT descriptions are untouched. CLI wires `builder.rogger_names` only when `--rogger` is set (default off, all feeds unchanged without it).
 
@@ -50,7 +50,7 @@ These scripts run the public download server at `https://mediupdatexml.oddb.org`
 - **`generate_index_html.sh DOCROOT [FIRSTBASE_CSV]`** — single source of truth for the landing page. Writes `index.html` + a self-contained `logo.svg` **atomically** (temp + `mv`, so either owner — root from setup, `zdavatz` from cron — can refresh it). Computes live counts: PHARMA = `<SMNO>` count in `default/oddb_article.xml`, NONPHARMA = firstbase CSV rows − 1, total ART = `<ART ` count. Also runs **`visitor_stats.py`** and embeds its graph. Re-run standalone any time (it only reads already-built files); a separate cron line refreshes it **hourly** (`5 * * * * zdavatz`) so counts + graph stay current between nightly builds.
 - **`visitor_stats.py LOG_GLOB CACHE_DIR [DAYS]`** — emits the visitors/sessions/region graph as an inline-SVG HTML **fragment** (last `DAYS`, default 14): Besucher = distinct IPs/day, Sitzungen = 30-min-inactivity sessions per `(IP, User-Agent)`, plus a top-6 country breakdown by IP. Bots are filtered by User-Agent. Region lookup is **fully self-contained** — pure Python stdlib + the free **DB-IP country-lite CSV** (CC-BY, no licence key) cached in the build `downloads/` dir and refreshed monthly; **no apt package, no gem, no system GeoIP DB**. Prints nothing (page degrades to omitting the section) when the Apache log is unreadable or empty. Reading `/var/log/apache2` requires the cron user to be in the **`adm`** group (`sudo usermod -aG adm zdavatz`).
 - **`swissmedic_watch.sh`** — outage/block auto-recovery (cron: `*/30 * * * * zdavatz`). Since the Swissmedic platform migration (~2026-06-23, now a Swisscom-operated gateway), `www.swissmedic.ch` intermittently resets this host's automated connections **after the TLS handshake** (TCP RST), which aborts `run_oddb2xml.sh` under `set -e` and leaves the feeds stale (the block is host/IP- and client-fingerprint-sensitive: a real browser works, `curl`/`wget`/Ruby get reset, while other admin.ch hosts answer fine — so it is a WAF/bot rule, not an outage). The watcher polls Swissmedic with **oddb2xml's own client** (a Ruby `open-uri` canary on `listen_neu.html`); while blocked it is a silent no-op, and the moment it gets HTTP 200 it launches **one** build and emails. It fires **at most once per day** (stamp in `$STATE_DIR`, default `<OUT_DIR>-watch`, kept **outside** the wiped `$BUILD_DIR`), and skips when a build is already running or today's `default/oddb_article.xml` is already fresh. **PATH assembly (2026-07-28 onwards):** cron hands the watcher a minimal PATH, so it builds one explicitly — `/usr/local/bin` (where a root `gem install` under Debian's system Ruby puts the binstub) + the user gem bin dir (`$(ruby -e 'print Gem.user_dir')/bin`, where an *unprivileged* `gem install` silently lands instead, because `/var/lib/gems` is not writable) + rbenv shims **only when they exist** (the pre-2026-07 box; `RBENV_VERSION=3.4.5` is exported only in that case). The previous hardcoded rbenv-only PATH carried no `/usr/local/bin`, so the first watcher-triggered build on the rebuilt server died `oddb2xml: Kommando nicht gefunden` (exit 127) on all three retries before aborting. `run_oddb2xml.sh` step 1 extends PATH the same way right after its `gem install`, for the same reason.
-- **`transfer.sh`** — optional hand-off (scp) of `$OUT_DIR` to the HIN host; `SCP_DEST` is required-but-unset until the HIN host is known.
+- **`transfer.sh`** — optional hand-off (scp) of `$OUT_DIR` to the HIN host; `SCP_DEST` is required-but-unset until the HIN host is known. **Obsolete, not merely unwired (issue [#127](https://github.com/zdavatz/oddb2xml/issues/127)):** it is not in `/etc/cron.d/mediupdatexml` and `SCP_DEST` still has no default — and it should stay that way, because **HIN no longer hosts a copy at all**. Their page `www.hin.ch/de/services/mediupdate-xml.cfm#section_2` links *this* site's URLs directly (`default/`, `45/`, `50/`, `55/`, `aips2sqlite/`) and mentions `download.hin.ch` exactly **zero** times, so there is nothing left to scp. The trap is that the unfed legacy path `https://download.hin.ch/download/oddb2xml/` **still answers HTTP 200**, serving the build of **22.06.2026** (`GENERATED_BY="oddb2xml 3.0.25"`, `Last-Modified: Sun, 21 Jun 2026 23:01:34 GMT`) — the whole directory carries that one date. It looks authoritative, it 404s nothing, and that is how issue #127 was filed against us for data that was current here all along. When someone reports stale data, **check which host they fetched from before touching the build**: `mediupdatexml.oddb.org` is the source of truth. HIN has been asked to 301 or retire the old path. `README.md` and `QA.md` now point at `mediupdatexml.oddb.org` throughout. For twenty days one link on HIN's page was genuinely broken and it was **ours**: `/aips2sqlite/oddb2xml_swissmedic_sequences.csv` answered 404 — **not** because the server rebuild lost it, but because aips2sqlite hit **the same Refdata barcode bug as issue #122**: `ean_code.length()` on the missing `<DataCarrierIdentifier>` threw a `NullPointerException` out of four copied loops, and `generate_aips_fi` deleted the published CSV *before* regenerating it, so under `set -e` each crash left nothing rather than yesterday's file. Fixed 02.09.2026 in aips2sqlite `ad2fc58` (all four loops skip barcode-less articles; the CSV is now published by `mv` only after a run that passed a row-count check). **When Refdata breaks one of the two projects, check the other** — they parse the same file with the same assumption.
 - **`setup_new_server.sh`** (2026-07-28 onwards) — one-command rebuild of a bare Debian host into this download site, written after **HIN deleted the previous server**: everything that had accumulated by hand on that box, so a rebuild is not a day of archaeology. Root, idempotent. Installs the packages (`apache2` + `certbot`; **`ruby-full`** — Debian 13 ships Ruby 3.3, matching the repo `.ruby-version`, so the new host has **no rbenv**; `cron`, absent on a minimal image; `unzip`/`zip` for the ZurRose transfer and the build zip; `poppler-utils` for `pdftotext` in `tools/generate_bag_sl_group_prices.rb`; nokogiri/ffi build headers), adds `$RUN_USER` to **`adm`** (so `visitor_stats.py` may read `/var/log/apache2`), sets the home dir to **711** (so `www-data` can traverse to `$OUT_DIR` without listing it), `gem install oddb2xml`, creates the five directories (`$OUT_DIR`, `-build`, `-state`, `-watch` — the last two deliberately outside the nightly-wiped `-build` — and `$GET_TRANSFER_DIR`, symlinking the tracked `get_transfer.sh` into it), writes **`/etc/cron.d/mediupdatexml`** (the whole schedule in one re-creatable file rather than the old box's per-user crontab: `get_transfer` 00:30, nightly build 01:00, rust2xml Artikelstamm 03:00, aips2sqlite Fachinfos 04:30, landing page hourly at `:05`, Swissmedic watcher every 30 min) plus a logrotate rule for the cron logs, then hands over to `setup_aips2sqlite.sh` and `setup_mediupdatexml_web.sh`. Env: `RUN_USER`, `OUT_DIR`, `SKIP_WEB=1`, `SKIP_GEM=1`, `SKIP_AIPS=1`. **Deliberately out of scope** (gone with the old server, restored separately): an MTA for the cron `MAILTO`.
 - **`setup_mediupdatexml_web.sh`** — one-time root setup of the Apache vhost + initial page. Skips the Let's Encrypt step with a hint when the domain does not yet resolve in **public** DNS (checked via Cloudflare DoH, not the local resolver, which may hold a stale or negative cache).
 - **`setup_aips2sqlite.sh`** (2026-08-01 onwards) — provisions the **aips2sqlite** half of the site, restored after the server rebuild. The landing page links a whole `/aips2sqlite/` section (Fachinformationen as XML, the AmiKo `.db` files, `oddb2xml_swissmedic_sequences.csv`, `atc_codes_used_set.txt`); `setup_mediupdatexml_web.sh` writes the Apache alias `/aips2sqlite` → `<checkout>/jars/output`, but on the rebuilt host neither the JRE nor that directory existed, so **every link answered 403** — Apache falls back to the default `<Directory /> Require all denied` when the aliased path does not exist, so a *missing* target reads as *forbidden*, not *not found*. Root, idempotent: installs **`openjdk-21-jre-headless`** (the repo ships a prebuilt fat jar `jars/aips2sqlite.jar`, so a JRE suffices — no JDK/Gradle unless the jar is rebuilt; the only AWT user, `BarCode.java`/barcode4j, is off the Fachinfo path), clones the checkout if missing, creates `jars/output` + `jars/downloads` owned by `$RUN_USER`, and appends the 04:30 cron entry to `/etc/cron.d/mediupdatexml` if not already there (`SKIP_CRON=1` when called from `setup_new_server.sh`, which writes that line itself). Generating the data is a separate build-user step: `aips2sqlite/scripts/generate_aips_fi` (~1 h; `--lang=de --xml`, then `--lang=fr --xml --nodown`, then `--nodown --smsequence`, renaming `swiss_medic_sequences.csv` → `oddb2xml_swissmedic_sequences.csv`). `REFDATA_API_KEY` (developer.refdata.ch) is **optional** for this path: the Refdata Partner SOAP download it gates is consumed only by the Takeda partner export, and its failure is caught and logged, so the FI/sequences output is complete without it.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    oddb2xml (3.0.32)
+    oddb2xml (3.0.34)
       csv
       htmlentities
       httpi

data/History.txt

@@ -1,3 +1,11 @@
+=== 3.0.34 / 02.09.2026
+* Bugfix (WeledaSL): pin the encoding of the three runtime CSVs so the Kapitel-70 SL recovery cannot disappear without a trace. WeledaSL.source passed the downloaded content on in whatever encoding the downloader produced; when that is ASCII-8BIT -- any response served without a charset -- the first comparison against a UTF-8 literal raises Encoding::CompatibilityError, and WeledaSL.load rescues every error into an empty Hash. The whole Weleda/WALA recovery (~835 SL products, issue #121) then vanished silently: no <SL_ENTRY>, no BAG group price, and a build that still reported success. All three lists carry non-ASCII (umlauts in the product names, the "-" in the group-price table), so the live path was one charset-less response away from this; it has not happened yet only because raw.githubusercontent.com serves UTF-8. The encoding is now forced in source, as RoggerNames#parse already did, and the bundled fallback is read with an explicit encoding.
+* This was also the last failing spec: artikelstamm_spec.rb:86, the issue #121 regression test, failed because @weleda_sl was empty under webmock and the item was emitted as PHARMATYPE="N" without SL_ENTRY/PPUB. `rspec` is green again -- 1193 examples, 0 failures, 16 pending. Closes GitHub issue #104.
+
+=== 3.0.33 / 26.08.2026
+* Bugfix (Refdata): skip PHARMA articles that carry no barcode instead of losing every single one of them. Refdata started publishing a handful of PHARMA articles without a <DataCarrierIdentifier> in August 2026 -- the Swiss Red Cross blood products under the collective registration 99999 (Erythrozytenkonzentrat & co), which have no GTIN at all. RefdataExtractor#to_hash called .size on that nil, and Cli#download_as rescues the resulting NoMethodError into an Oddb2xml.log call that prints nothing unless --log is given, so all 16'093 Refdata PHARMA articles silently vanished from the feeds while the build still reported success. On mediupdatexml.oddb.org the <SMNO> count in oddb_article.xml fell from ~15'300 to exactly 10'210 and stayed frozen there for twelve days -- the remainder came from the BAG FHIR SL feed -- and then to 50 once the BAG url move (3.0.32) took the FHIR feed away too. Refdata NONPHARMA was never affected, because the product-type check runs before the barcode is read. See GitHub issue #122.
+* The two rescued Refdata failures (download and extraction) now also print a warning to stderr. Issue #102 asked the build to continue past a Refdata error, and it still does, but a total loss of every Swissmedic-registered medicine must not look like a successful run in the nightly log.
+
 === 3.0.32 / 26.08.2026
 * Bugfix (--fhir, the default): follow the BAG SL export to its new URL. BAG moved the publication on 24.08.2026 and announced it the next evening: /static/fhir/foph-sl-export-*.ndjson became /static/sl/publication/fhir/foph-sl-publication-*.ndjson. Only the -latest- alias is gone (the old dated snapshots stay in place), so the old URL failed by downloading nothing and reporting success rather than by crashing -- ch.oddb.org ran that way for twelve mornings with three HTTP 404 lines in a log as the only trace. Without the NDJSON there are no French/Italian names, no SL prices, no limitations and no Indikationscodes. Per the announcement there is no language-less default any more; every URL has to name de, fr or it, which FhirDownloader already did. A preliminary publication now exists in parallel under /static/sl/preliminary/fhir/foph-sl-preliminary-*, which oddb2xml does not use. The file prefix is its own constant now, so the next move touches one line.
 

data/QA.md

@@ -5,7 +5,8 @@
 * SL Preise werden am Anfang des Monats publiziert, jeweils immer am 1. Es kann auch vorkommen, dass das BAG die SL-Preise/Limitationen während dem Monat anpasst.
 * Fachinfos werden täglich publiziert.
 * Swissmedic-Codes erscheinen einmal pro Monat, normalerweise in der ersten Woche.
-* Die Daten unter [MEDIupdate XML](https://www.hin.ch/services/mediupdate-xml/) werden täglich generiert.
+* Die Daten werden täglich generiert und auf <https://mediupdatexml.oddb.org> publiziert.
+* Der Spiegel unter [MEDIupdate XML](https://www.hin.ch/de/services/mediupdate-xml.cfm) / `download.hin.ch` wird seit Juni 2026 nicht mehr nachgeführt, siehe [Issue #127](https://github.com/zdavatz/oddb2xml/issues/127).
 
 #### 2. Gibt es eine Spezifikation der XML Files? 
 * Ja, siehe [oddb2xml.xsd](https://github.com/zdavatz/oddb2xml/blob/master/oddb2xml.xsd)
@@ -21,8 +22,8 @@
 * Damit man _Registrations-_ und _Sequenznummer_ besser verstehen kann, muss man einmal das File [excel-version_zugelasseneverpackungen.xlsx](https://www.swissmedic.ch/dam/swissmedic/de/dokumente/listen/excel-version_zugelasseneverpackungen.xlsx.download.xlsx/excel-version_zugelasseneverpackungen.xlsx) öffnen und die ersten paar Spalten anschauen.
 
 #### 5. Was ist der Unterschied zwischen oddb_article.xml und oddb_product.xml
-* [oddb_article.xml](http://download.hin.ch/download/oddb2xml/oddb_article.xml) enhält alle Artikel. 
-* [oddb_product.xml](http://download.hin.ch/download/oddb2xml/oddb_product.xml) enthält nur die Produkte von der Swissmedic, also die Medikamente.
+* [oddb_article.xml](https://mediupdatexml.oddb.org/default/oddb_article.xml) enhält alle Artikel.
+* [oddb_product.xml](https://mediupdatexml.oddb.org/default/oddb_product.xml) enthält nur die Produkte von der Swissmedic, also die Medikamente.
 
 #### 6. Warum hat nicht jedes Produkt im oddb_article.xml einen GTIN?
 * Nicht alle Produkte haben zur Zeit einen GTIN. Dieser wird jedoch laufend ergänzt. Ab 1.1.2019 sollte der Pharmacode komplett verschwinden. Dies wurde von der Stiftung [Refdata](http://www.refdata.ch) auch so bestätigt.
@@ -36,7 +37,7 @@
 #### 8. Ich möchte gerne ein XML-File welches alle Produkte (Pharma und Non-Pharma) und die dazugehörigen Sequenznamen enthält. Gibt es das?
 * Ja. _oddb2xml_ mit der Option _-r_ laufen lassen, siehe [usage](https://github.com/zdavatz/oddb2xml#usage) - Option "_--artikelstamm_".
 * Dieses File wird zur Zeit nicht via [MEDIupdate XML](https://www.hin.ch/services/mediupdate-xml/) zum Download zur Verfügung gestellt. Es muss selber generiert werden mittels _oddb2xml -r_ 
-* [CSV](http://pillbox.oddb.org/artikelstamm_26042018_v5.csv) oder [XML](http://pillbox.oddb.org/artikelstamm_26042018_v5.xml) Beispiel-Download vom 26.4.2018.
+* Der täglich gebaute Artikelstamm steht unter <https://mediupdatexml.oddb.org/artikelstamm/> bereit: [v6 XML](https://mediupdatexml.oddb.org/artikelstamm/artikelstamm_v6.xml), [v6 CSV](https://mediupdatexml.oddb.org/artikelstamm/artikelstamm_v6.csv) und die [Legacy-Version v5](https://mediupdatexml.oddb.org/artikelstamm/artikelstamm_v5.xml).
 
 #### 9. Wie installiere ich _oddb2xml_?
 * Neuste, stabile Version von [Ruby](http://www.ruby-lang.org/de/) installieren.

data/README.md

@@ -46,7 +46,21 @@ Generating files for Elexis Artikelstamm is discussed in the [Readme for the Art
 
 ## usage
 
-HIN (http://hin.ch) creates daily the actual file. They can be downloaded from `https://download.hin.ch/download/oddb2xml`, e.g. using  `wget https://download.hin.ch/download/oddb2xml/oddb_article.xml`
+Ready-made files are built nightly and published on
+<https://mediupdatexml.oddb.org>, e.g. using
+`wget https://mediupdatexml.oddb.org/default/oddb_article.xml`. See
+[Deployment](#deployment-the-mediupdatexmloddborg-download-site) for what the
+site publishes and how it is generated.
+
+HIN's [MEDIupdate XML](https://www.hin.ch/de/services/mediupdate-xml.cfm#section_2)
+page links straight to those URLs — it does not host a copy of its own.
+
+The old mirror at `https://download.hin.ch/download/oddb2xml` is **not linked
+from anywhere any more** and has not been updated since June 2026 (it still
+serves the build of 22.06.2026, `GENERATED_BY="oddb2xml 3.0.25"`) — the job
+feeding it stopped with the old server. It still answers `200` with
+plausible-looking data, which is exactly what makes it dangerous. Don't use it,
+see [issue #127](https://github.com/zdavatz/oddb2xml/issues/127).
 
 see `--help`.
 
@@ -555,7 +569,12 @@ they are the reference setup for running oddb2xml unattended.
   cron entry that regenerates the Fachinformationen. Run as root; idempotent.
   Without it every `/aips2sqlite/` link answers 403, because Apache denies a
   path that does not exist.
-* `transfer.sh` — optional scp hand-off of the output tree.
+* `transfer.sh` — optional scp hand-off of the output tree to the HIN
+  download server. **Obsolete, and deliberately not wired up**: `SCP_DEST` has
+  no default and the script is not in `/etc/cron.d/mediupdatexml`. HIN links
+  this site's URLs directly, so there is nothing left to copy; the unfed mirror
+  it used to push to is
+  [issue #127](https://github.com/zdavatz/oddb2xml/issues/127).
 
 One Debian-specific pitfall worth knowing when running oddb2xml from cron: with
 the system Ruby, `gem install oddb2xml` as an unprivileged user cannot write
@@ -567,9 +586,62 @@ themselves.
 
 ## Testing
 
+On a host where the system gem directory is not writable (the mediupdatexml.oddb.org
+box, for instance), point bundler somewhere it may write, otherwise `bundle install`
+aborts while installing `rake`:
+
+```bash
+BUNDLE_PATH=/tmp/oddb2xml-bundle bundle install
+BUNDLE_PATH=/tmp/oddb2xml-bundle bundle exec rspec
+```
+
 * Calling rake spec runs spec tests.
 * Calling rake test installs the gems and runs oddb2xml with the most commonly used combinations. All output is placed under under ausgabe/<timestamp>. These files should be manually compared to the ones generated by the last release to check for possible problems.
 * we use the gem VCR to record real HTTP responses.
 ** Removing the directory fixtures and running @bundle exec rspec spec/downloader_spec.rb@ gets the actual content from the different servers
 ** To minimize the downloaded size we use several @before_record@ hooks to select the desired content, eg. only the 5 items from EPha.
 
+
+## Releasing a new version
+
+1. Bump `Oddb2xml::VERSION` in `lib/oddb2xml/version.rb` and the `oddb2xml (X.Y.Z)`
+   line in `Gemfile.lock`.
+2. Prepend an entry to `History.txt` (`=== X.Y.Z / DD.MM.YYYY`).
+3. Commit, then tag and push:
+
+   ```bash
+   git tag -a vX.Y.Z -m "Version X.Y.Z"
+   git push origin master && git push origin vX.Y.Z
+   ```
+
+   The tag triggers `.github/workflows/release.yml`, which builds the gem and
+   publishes a GitHub release with the `.gem` attached. It does **not** push to
+   rubygems.org — that step is manual.
+4. Build and push the gem:
+
+   ```bash
+   gem build oddb2xml.gemspec
+   gem push oddb2xml-X.Y.Z.gem --otp <code>
+   ```
+
+The push needs an API key in the file that `ruby -e 'require "rubygems"; puts
+Gem.configuration.credentials_path'` reports — on Debian with the system Ruby
+that is `~/.local/share/gem/credentials`, not the `~/.gem/credentials` most
+documentation names. Create it with `gem signin`, or write it by hand as
+
+```yaml
+---
+:rubygems_api_key: rubygems_...
+```
+
+and `chmod 600` it. Multi-factor authentication is enabled on this gem, so
+`--otp` is required **in addition to** the API key; a missing key reports
+`Invalid credentials / code: 401` rather than anything about the OTP, which is
+easy to misread as an expired code.
+
+Finally, update the download server, which runs the released gem rather than
+this checkout:
+
+```bash
+sudo gem install oddb2xml
+```

data/lib/oddb2xml/cli.rb

@@ -85,6 +85,8 @@ module Oddb2xml
             threads << download(:refdata, type) # refdata
           rescue => error
             # Should continue even when error #102
+            $stderr.puts "WARNING: Refdata #{type} download failed (#{error.class}: #{error})"
+            $stderr.flush
             Oddb2xml.log("Error in downloading refdata #{error}")
           end
         end
@@ -377,6 +379,12 @@ module Oddb2xml
             @refdata_types[type]
           rescue => error
             # Should continue even when error https://github.com/zdavatz/oddb2xml/issues/102
+            # ...but say so loudly: without the Refdata articles the feeds lose every
+            # Swissmedic-registered medicine, and Oddb2xml.log is a no-op unless --log
+            # is given, so this used to fail completely silently. See issue #122.
+            $stderr.puts "WARNING: RefdataExtractor #{type} failed (#{error.class}: #{error}) -- " \
+              "no #{type} articles from Refdata in this run!"
+            $stderr.flush
             Oddb2xml.log("Error in RefdataExtractor #{error}")
           end
         end

data/lib/oddb2xml/extractor.rb

@@ -216,6 +216,18 @@ module Oddb2xml
           next
         end
         ean13 = @type == "PHARMA" ? article.PackagedProduct.DataCarrierIdentifier : article.MedicinalProduct.Identifier
+        if ean13.nil? || ean13.empty?
+          # Refdata publishes a handful of PHARMA articles that carry no barcode at
+          # all -- e.g. the Swiss Red Cross blood products under the collective
+          # registration 99999 (Erythrozytenkonzentrat & co), which appeared in
+          # August 2026. They have no article identifier we could key on, so skip
+          # them. Before this guard the nil blew up the whole extraction, and
+          # Cli#download_as rescues that error into a silent Oddb2xml.log call, so
+          # *every* Refdata PHARMA article vanished from the feeds while the build
+          # still reported success. See GitHub issue #122.
+          puts "Refdata #{@type} skipping #{article.MedicinalProduct.Identifier} without an EAN13" if $VERBOSE
+          next
+        end
         if ean13.size < 13
           puts "Refdata #{@type} use 13 chars not #{ean13.size} for #{ean13}" if $VERBOSE
           ean13 = ean13.rjust(13, "0")

data/lib/oddb2xml/version.rb

@@ -1,3 +1,3 @@
 module Oddb2xml
-  VERSION = "3.0.32"
+  VERSION = "3.0.34"
 end

data/lib/oddb2xml/weleda_sl.rb

@@ -78,9 +78,17 @@ module Oddb2xml
         bundled = File.join(DATA_DIR, basename)
         if File.exist?(bundled)
           Oddb2xml.log "WeledaSL: using bundled #{basename}"
-          content = File.read(bundled)
+          content = File.read(bundled, encoding: "UTF-8")
         end
       end
+      # All three lists carry non-ASCII (umlauts in the product names, the "−"
+      # in the group-price table). A downloader that hands back ASCII-8BIT --
+      # webmock does, and so does any body served without a charset -- makes
+      # the first comparison against a UTF-8 literal raise
+      # Encoding::CompatibilityError, which load's rescue turns into an empty
+      # map: the whole Weleda/WALA SL recovery then vanishes silently. Pin the
+      # encoding here, as RoggerNames#parse does.
+      content = content.dup.force_encoding(Encoding::UTF_8) if content && content.encoding != Encoding::UTF_8
       content
     end
 

data/spec/extractor_spec.rb

@@ -80,6 +80,51 @@ describe Oddb2xml::RefdataExtractor do
       expect(item_found).to eq(expected)
     end
   end
+  context "when Refdata omits the barcode (issue #122)" do
+    # The Swiss Red Cross blood products under the collective registration 99999
+    # arrived in August 2026 with no <DataCarrierIdentifier> at all. The nil used
+    # to blow up the whole extraction -- and Cli#download_as rescues that into a
+    # silent Oddb2xml.log, so every Refdata PHARMA article disappeared from the
+    # feeds while the build still reported success.
+    let(:xml) do
+      <<~XML
+        <?xml version="1.0" encoding="utf-8"?>
+        <Articles xmlns="https://simisinfo.refdata.ch/Articles/1.0/" totalArticles="2">
+          <Article>
+            <MedicinalProduct>
+              <Identifier>CH-7601001320451-62069</Identifier>
+              <ProductClassification><ProductClass>PHARMA</ProductClass><Atc>N03AX14</Atc></ProductClassification>
+            </MedicinalProduct>
+            <PackagedProduct>
+              <RegulatedAuthorisationIdentifier>62069008</RegulatedAuthorisationIdentifier>
+              <DataCarrierIdentifier>#{Oddb2xml::LEVETIRACETAM_GTIN}</DataCarrierIdentifier>
+              <Holder><Identifier>7601001320451</Identifier><Name>Desitin Pharma GmbH</Name></Holder>
+              <Name><Language>DE</Language><FullName>LEVETIRACETAM DESITIN Mini Filmtab 250 mg 30 Stk</FullName></Name>
+            </PackagedProduct>
+          </Article>
+          <Article>
+            <MedicinalProduct>
+              <Identifier>CH-7601002120890-9999900</Identifier>
+              <ProductClassification><ProductClass>PHARMA</ProductClass><Atc>Z00Z000</Atc></ProductClassification>
+            </MedicinalProduct>
+            <PackagedProduct>
+              <RegulatedAuthorisationIdentifier>99999001</RegulatedAuthorisationIdentifier>
+              <Holder><Identifier>7601002120890</Identifier><Name>Schweizerisches Rotes Kreuz - SRK</Name></Holder>
+              <Name><Language>DE</Language><FullName>Erythrozytenkonzentrat (EK)</FullName></Name>
+            </PackagedProduct>
+          </Article>
+        </Articles>
+      XML
+    end
+
+    it "skips the barcode-less article instead of losing every other one" do
+      items = Oddb2xml::RefdataExtractor.new(xml, "PHARMA").to_hash
+      expect(items.size).to eq 1
+      expect(items.keys).to eq [Oddb2xml::LEVETIRACETAM_GTIN.to_s]
+      expect(items.values.first[:no8]).to eq "62069008"
+    end
+  end
+
   context "should handle nonpharma articles" do
     subject do
       @downloader = Oddb2xml::RefdataDownloader.new({}, :nonpharma)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: oddb2xml
 version: !ruby/object:Gem::Version
-  version: 3.0.32
+  version: 3.0.34
 platform: ruby
 authors:
 - Yasuhiro Asaka, Zeno R.R. Davatz, Niklaus Giger