oddb2xml 3.0.20 → 3.0.21

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fe41f3e5cfb0e100f111a3071f32ae8dd4b4103d3f5726bf4eab061f80389323
-  data.tar.gz: 785dee3549e33031069aceac4f9be31cc844de603d3b183d94b55f976b0c482c
+  metadata.gz: 5f901623058340b0e1fb8061fcd0268c0f392dc5814707b43c34cc62ac626216
+  data.tar.gz: 62c556ccc664db477539e9fbdf237b9dfbecbc10da5d1fca96dc81731be81afd
 SHA512:
-  metadata.gz: b385af5fc0643374e878da0cd6e1b262e4d7521c5ac5dd3a3670971f8f130ca3492b4aa296410c5e9e6627021bcbb86b508dbd3e80bf1fdd044c61851df69664
-  data.tar.gz: 40a8baa557cdb31905f8711702bbe52f8d968b67488ae88f293233ea3bd1db689c9ea75d27b4b17019cf2369fb593eed3268c3867277daf5cb73c763b6a74791
+  metadata.gz: c14fe740b28315cc763f7e35d9794b4a4447525acc86d4dec14f678271e0537daa0cbc5a21f62f9798cb3a3aae2b1698256cdb2532dd555f036e53a5094fbfc3
+  data.tar.gz: 5299458e63aa3e152ca66ef98babf572fc3e1785a693e38ff3e392a5b6d7af3c578beeecfedbad023d1f19d7e23e4505b3b68da74664798826c192da933cd87a

data/CLAUDE.md

@@ -53,6 +53,8 @@ The system follows a **download → extract → build → compress** pipeline:
 
 9. **Chapter-70 hack** (`lib/oddb2xml/chapter_70_hack.rb`) — Legacy scraper for the SL "Komplementärarzneimittel" products (homeopathic/anthroposophic/phytotherapeutic), called only from `Builder#build_artikelstamm`. **Deprecated / non-FHIR only (3.0.11 onwards):** the source page `varia_De.htm` was rebuilt as a JavaScript SPA with no static data table, so the scraper now returns nothing there. These products + limitations now come through the FHIR feed (SL classification `20. KOMPLEMENTÄRARZNEIMITTEL`, 221 products on the live DE feed with real GTINs and limitation texts), so `build_artikelstamm` **skips the scraper entirely when `@options[:fhir]`** (the default for `--artikelstamm` since 3.0.9). In `--no-fhir` mode the scraper degrades gracefully (skips non-row/`<script>` nodes and empty tables, warns, returns `[]`) instead of raising `NoMethodError`. See GitHub issue #118.
 
+10. **Weleda / Kapitel-70 SL recovery** (`lib/oddb2xml/weleda_sl.rb`, 3.0.21 onwards) — Recovers the SL flag and public price for chapter-70 complementary medicines that are **missing from the FHIR feed** (the partial-replacement gap left by the dead chapter_70_hack, issue #118/#121). Many are magistral Weleda preparations with a `7611916…` trade GTIN that arrive only via ZurRose — with no SL flag and a blanked Publikumspreis (issue #117). `WeledaSL.load` joins two CSVs (downloaded at runtime from `github.com/zdavatz/oddb2xml_files` via `WeledaDownloader` / `BagSlGroupPricesDownloader`, bundled fallback copies under `data/`): `weleda_arzneimittel.csv` (GTIN → `abgabekategorie` SL flag + `csl` = **Pharma-Gruppen-Code**) and `bag_sl_group_prices.csv` (Pharma-Gruppen-Code → public price). The price table is extracted **offline** from the BAG SL definition PDF *"Homoeopathica, Anthroposophica, Allergene"* via `tools/generate_bag_sl_group_prices.rb` (uses system `pdftotext`; **no runtime PDF gem** — `pdf-reader`'s `afm` dep now needs Ruby ≥ 3.2, which would break the gem's Ruby floor). The join is **GTIN → csl → price**, honouring an `N x <code>` package multiplier (price = N × group price). Produces `gtin => {sl:, price:, csl:, abgabe:}` (SL rows only; ~515 priced on the live feed). `Builder#build_artikelstamm` consumes it (CLI sets `builder.weleda_sl` only for `--artikelstamm`): for any GTIN **absent from the FHIR NDJSON** it emits `<SL_ENTRY>true</SL_ENTRY>` and `<PPUB>` from the BAG group price, mirroring the old chapter-70 behaviour (`PHARMATYPE "P"`). **The FHIR/ZurRose price always wins** — the group price only fills a gap; a zeroed ZurRose `"0.00"` pub price is treated as absent so the gap-fill can apply. Match is **by GTIN only** (no pharmacode); the Swissmedic dispensing category is untouched (still from `Swissmedic_Packungen.xlsx`). The Artikelstamm output gets `<SL_ENTRY>` + `<PPUB>`; for the `-e`/`--extended` and `-b`/`--firstbase` product feeds the BAG public price is also added to `oddb_article.xml` as an `<ARTPRI><PTYP>BAGPUB</PTYP>` entry (the raw, often-blanked `ZURROSEPUB` is preserved alongside it) — `build_article`, gated by the CLI loading `weleda_sl` when `extended || firstbase || artikelstamm`. See GitHub issue #121.
+
 ### Key data identifiers
 - **GTIN/EAN13**: Primary article identifier (13-digit barcode)
 - **Pharmacode**: Swiss pharmacy code

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    oddb2xml (3.0.19)
+    oddb2xml (3.0.21)
       csv
       htmlentities
       httpi

data/History.txt

@@ -1,3 +1,7 @@
+=== 3.0.21 / 10.06.2026
+* New (Artikelstamm): recover the SL flag and the public price for "Kapitel 70" complementary medicines (Homöopathika / Anthroposophika / Phytotherapeutika) that are missing from the BAG FHIR feed (issue #121). These products -- many of them magistral Weleda preparations with a 7611916… trade GTIN -- arrive only via ZurRose, where they currently have no SL flag (the legacy chapter-70 varia-page scraper is dead, issue #118) and a blanked public price (ZurRose zeroed the Publikumspreis column, issue #117). Two small CSVs in github.com/zdavatz/oddb2xml_files (downloaded at runtime, so they can be refreshed without a gem release, with a bundled copy under data/ as offline fallback) close the gap: weleda_arzneimittel.csv maps GTIN -> Abgabekategorie (the "… / SL" flag) and csl (= Pharma-Gruppen-Code), and bag_sl_group_prices.csv maps the Pharma-Gruppen-Code -> public price (CHF incl. MWST), extracted from the authoritative BAG SL definition PDF "Homoeopathica, Anthroposophica, Allergene" (regenerate with tools/generate_bag_sl_group_prices.rb). build_artikelstamm now joins GTIN -> csl -> price (honouring an "N x <code>" package multiplier) and, for any GTIN absent from the FHIR NDJSON, emits <SL_ENTRY>true</SL_ENTRY> and <PPUB> from the BAG group price, mirroring the old chapter-70 behaviour (PHARMATYPE "P"). The FHIR/ZurRose price always wins; the BAG group price only fills a gap. The Swissmedic dispensing category is unchanged (still from Swissmedic_Packungen.xlsx). No new runtime gem dependency -- the PDF is converted to CSV offline via pdftotext (pdf-reader was rejected because its afm dependency now needs Ruby >= 3.2). The Artikelstamm output gets <SL_ENTRY> + <PPUB>; for the -e/--extended and -b/--firstbase product feeds the BAG public price is also added to oddb_article.xml as an <ARTPRI><PTYP>BAGPUB</PTYP> entry (the raw, often blanked, ZURROSEPUB is preserved alongside it). Live feed: ~515 SL products gain a public price.
+* Bugfix: in the Artikelstamm non-pharma branch a zeroed ZurRose public price ("0.00") is no longer emitted as <PPUB>0.00</PPUB> (it is now treated as absent, consistent with the sibling ZurRose-detail path), which also lets the BAG group price fill the gap.
+
 === 3.0.20 / 09.06.2026
 * Bugfix/robustness: handle truncated or failed Swissmedic .xlsx downloads instead of crashing later in the parser (issue #121). A download through a scanning/allow-list proxy can return an incomplete .xlsx -- a valid ZIP/Excel header (so `file` still reports "Microsoft Excel 2007+") but a missing end-of-central-directory record -- which made RubyXL die deep in rubyzip with the cryptic "Zip end of central directory signature not found". New Oddb2xml.valid_zip? verifies an archive is complete (PK header + EOCD signature in the tail). SwissmedicDownloader now validates the downloaded file and automatically re-fetches it (up to the normal retry count) when it is empty or truncated, and no longer reuses a cached broken file under --skip-download. As a last resort SwissmedicExtractor raises a clear, actionable error (pointing at connectivity / "oddb2xml --proxy-check") instead of the rubyzip backtrace.
 

data/data/bag_sl_group_prices.csv

@@ -0,0 +1,143 @@
+pharma_group_code,price_chf_incl_vat,description,limitation
+2069562,13.40,Urtinktur 1−10 g/ml,""
+2069579,16.05,Urtinktur 11−20 g/ml,""
+2069585,20.80,Urtinktur 21−40 g/ml,""
+2069591,26.95,Urtinktur 41−60 g/ml,""
+2069616,39.80,Urtinktur 61−125 g/ml,""
+2069622,25.20,17117 Ceres Urtinktur 20 ml,L1
+2069639,31.35,Ceres Urtinktur gemäss L2 20 ml,"L1, L2"
+2069645,7.65,D/C 1−9,""
+2069651,14.20,D/C 1−9 11−20 g/ml,""
+2069668,18.00,D/C 1−9 21−40 g/ml,""
+2069674,24.00,D/C 1−9 41−60 g/ml,""
+2069680,28.65,D/C 1−9 61−125 g/ml,""
+2069705,8.50,D/C 10−29,""
+2069711,15.75,D/C 10−29 11−20 g/ml,""
+2069728,19.95,D/C 10−29 21−40 g/ml,""
+2069734,26.55,D/C 10−29 41−60 g/ml,""
+2069740,31.65,D/C 10−29 61−125 g/ml,""
+2069763,11.30,D/C 30−60,""
+2069786,21.05,D/C 30−60 11−20 g/ml,""
+2069792,27.20,D/C 30−60 21−40 g/ml,""
+2069800,35.10,D/C 30−60 41−60 g/ml,""
+2069817,41.85,D/C 30−60 61−125 g/ml,""
+2069852,19.95,"D/C 100, 200",""
+2069869,29.80,"D/C 300, 400, 500",L3
+2069875,40.20,D/C 1000 1−10 g/ml,L3
+2069881,20.05,LM I−XXX (Q 1-30) 1−10 g/ml,""
+3793722,37.10,LM I−XXX (Q 1-30) 11−20 g/ml,""
+3793768,29.70,LM ab XXXI (Q ab 31) 1−10 g/ml,""
+3793780,54.90,LM ab XXXI (Q ab 31) 11−20 g/ml,""
+3487669,5.45,CF 30 1 Dose,""
+2069898,5.45,K 30 1 Dose / 1 g,""
+2069912,6.15,K 30 2−5 g,""
+2069935,16.30,K 30 5 Dosen,""
+3487729,7.60,CF 200 1 Dose,""
+2069941,7.60,K 200 1 Dose / 1 g,""
+2069958,10.15,K 200 2−5 g,""
+2069964,22.85,K 200 5 Dosen,""
+3487735,8.65,CF 1000 (MCF) 1 Dose,""
+2069970,8.65,K 1000 (MK) 1 Dose /1 g,""
+2069987,12.85,K 1000 (MK) 2−5 g,L3
+2069993,26.05,K 1000 (MK) 5 Dosen,L3
+3487741,9.80,CF 10 000 (XMCF) 1 Dose,""
+2070016,9.80,K 10 000 (XMK) 1 Dose / 1 g,""
+2070022,15.25,K 10 000 (XMK) 2−5 g,L3
+2070039,29.15,K 10 000 (XMK) 5 Dosen,L3
+3487758,16.05,CF 50 000 (LMCF) 1 Dose,L3
+2070045,16.05,K 50 000 (LMK) 1 Dose / 1 g,L3
+2070051,17.80,K 50 000 (LMK) 2−5 g,L3
+2070068,47.10,K 50 000 (LMK) 5 Dosen,L3
+3487764,18.10,CF 100 000 (CMCF) 1 Dose,L3
+2070074,18.10,K 100 000 (CMK) 1 Dose / 1 g,L3
+2070080,18.85,K 100 000 (CMK) 2−5 g,L3
+2070097,53.00,K 100 000 (CMK) 5 Dosen,L3
+3793857,23.40,Kapseln 10−50 Stk,""
+3793834,37.20,Kapseln 51−100 Stk,""
+3793811,69.35,Kapseln 101−200 Stk,""
+3793892,23.65,Mundspray 21−40 ml,""
+3793917,31.45,Mundspray 41−60 ml,""
+2070111,11.70,"Äusserliche Tinkturen, Essentia 10−50 g/ml",""
+2070128,16.50,"Äusserliche Tinkturen, Essentia 51−100 g/ml",""
+2070140,16.25,"Cremes, Salben, Pasten, Gele 10−50 g/ml",""
+2070157,26.45,"Cremes, Salben, Pasten, Gele 51−100 g/ml",""
+2070186,10.85,"Emulsionen, Öle 10−50 g/ml",""
+2070192,19.90,"Emulsionen, Öle 51−100 g/ml",""
+2070200,8.25,"Nasen-, Ohrentropfen 1−10 g/ml",""
+3793969,14.05,Nasenspray 20 ml,""
+3793946,13.90,Augensalbe 5−10 g,""
+2070217,10.85,Augentropfen 1−10 g/ml,""
+2070223,1.70,"Suppositorien, Ovula (1-12 Stk) 1 Stk",""
+2070631,2.50,D/C 1−9 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+2070648,1.95,D/C 1−9 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+2070654,2.95,D/C 10−29 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+2070660,2.35,D/C 10−29 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+2070677,4.45,D/C 30−499 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+2070683,3.55,D/C 30−499 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+2070708,4.45,D/C 500 Ampulle 1−9 ml (1−10 Stk) 1 Stk,L3
+2070714,3.55,D/C 500 Ampulle 1−9 ml (11−50 Stk) 1 Stk,L3
+4961009,5.45,D 1000 Ampulle 1−9 ml (1−10 Stk) 1 Stk,L3
+4961038,4.55,D 1000 Ampulle 1−9 ml (11−50 Stk) 1 Stk,L3
+2070720,7.55,D/C 1−60 Ampulle 10 ml 1 Stk,""
+2070246,16.60,Urtinktur 1−10 g/ml,""
+2070252,18.35,Urtinktur 11−20 g/ml,""
+2070269,22.40,Urtinktur 21−40 g/ml,""
+2070275,28.45,Urtinktur 41−60 g/ml,""
+2070281,45.95,Urtinktur 61−125 g/ml,""
+2070298,13.35,D/C 1−9 1−10 g/ml,""
+2070306,16.55,D/C 1−9 11−20 g/ml,""
+2070312,18.45,D/C 1−9 21−40 g/ml,""
+2070329,25.70,D/C 1−9 41−60 g/ml,""
+2070335,41.50,D/C 1−9 61−125 g/ml,""
+2070341,14.80,D/C 10−29 1−10 g/ml,""
+2070358,18.35,D/C 10−29 11−20 g/ml,""
+2070364,20.50,D/C 10−29 21−40 g/ml,""
+2070370,28.45,D/C 10−29 41−60 g/ml,""
+2070387,45.95,D/C 10−29 61−125 g/ml,""
+2070393,23.20,D/C 30−60 1−10 g/ml,""
+2070401,27.30,D/C 30−60 11−20 g/ml,""
+2070418,32.35,D/C 30−60 21−40 g/ml,""
+2070424,41.10,D/C 30−60 41−60 g/ml,""
+2070430,66.25,D/C 30−60 61−125 g/ml,""
+2070447,29.45,"D/C 100, 200 1−10 g/ml",""
+4960990,36.65,D 1000 30 g/ml,""
+3794130,29.30,Kapseln 10−50 Stk,""
+3794118,46.50,Kapseln 51−100 Stk,""
+3794101,86.75,Kapseln 101−200 Stk,""
+3794207,24.45,Mundspray 21−40 ml,""
+3794199,32.60,Mundspray 41−60 ml,""
+2070453,14.70,"Äusserliche Tinkturen, Essentia 10−50 g/ml",""
+2070476,20.60,"Äusserliche Tinkturen, Essentia 51−100 g/ml",""
+2070482,20.25,"Cremes, Salben, Pasten, Gele 10−50 g/ml",""
+2070499,32.80,"Cremes, Salben, Pasten, Gele 50−100 g/ml",""
+2070507,13.90,"Emulsionen, Öle 10−50 g/ml",""
+2070513,24.80,"Emulsionen, Öle 51−100 g/ml",""
+2070536,10.40,"Nasen-, Ohrentropfen 1−10 g/ml",""
+3794182,17.55,Nasenspray 20 ml,""
+3794176,17.40,Augensalbe 5−10 g,""
+2070542,13.90,Augentropfen 1−10 g/ml,""
+2070559,2.30,"Suppositorien, Ovula (1−12 Stk) 1 Stk",""
+2070565,3.05,D/C 1−9 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+2070571,2.40,D/C 1−9 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+2070588,3.90,D/C 10−29 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+2070594,3.10,D/C 10−29 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+2070602,5.90,D/C 30−60 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+2070619,4.80,D/C 30−60 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+4961021,7.90,D 60−200 Ampulle 1−9 ml (1−10 Stk) 1 Stk,""
+4961015,6.50,D 60−200 Ampulle 1−9 ml (11−50 Stk) 1 Stk,""
+2070625,11.15,D/C 1−60 Ampulle 10 ml 1 Stk,""
+5001116,433.95,433.95 L,L
+5001122,405.50,405.50 L,L
+5001151,281.45,281.45 L,L
+5001168,301.05,301.05 L,L
+5001174,193.05,193.05 L,L
+5001464,223.50,3 Fl 10 ml,L
+5001470,301.75,orale Lösung 3 Fl 10 ml,L
+5001487,143.50,4 Fl 10 ml,L
+5001493,115.25,orale Lösung 1 Fl 10 ml,L
+5120223,289.20,5 Stk,L
+5120246,235.10,1 Stk,L
+5120252,359.75,4 x 5 ml,L
+5120269,291.40,5 ml,L
+6516710,273.85,273.85 L,L
+6516727,312.15,312.15 L,L