oddb2xml 3.0.24 → 3.0.26

data/History.txt

@@ -1,3 +1,9 @@
+=== 3.0.26 / 29.06.2026
+* New (--artikelstamm): emit the Elexis Artikelstamm v6 format with per-article BAG Indikationscodes (INDC). The output bumps from v5 to v6 -- namespace http://elexis.ch/Elexis_Artikelstamm_v6, file artikelstamm_DDMMYYYY_v6.xml/.csv, validated against the new bundled Elexis_Artikelstamm_v6.xsd -- and every SL price-model <ITEM> now carries an <ARTSL> block: <PM>true</PM> plus one <ARTLIM> per limitation with <LIMCD> (BAG limitation code), <INDCD> (the Indikationscode XXXXX.NN), and the validity dates <VDAT>/<VTDAT>. The indication codes were already read from the BAG SL FHIR feed's indicationCode extension (used since 3.0.7 for oddb_product.xml's <INDICATION_CODE>); they are now also carried per package limitation and grouped onto each article. Mandatory on prescriptions/invoices for SL price-model drugs from 2026-07-01 (issue #113). The v6 XSD is the canonical MEDEVIT schema extended with oddb2xml's historical Italian elements (DSCRI / DOSAGE_FORMI) so the output keeps validating. Consumers must switch from the v5 to the v6 file.
+
+=== 3.0.25 / 17.06.2026
+* Bugfix (-b/--firstbase): the GS1 Switzerland firstbase CSV (id.gs1.ch) is served with a UTF-8 BOM. FirstbaseExtractor read it with encoding "UTF-8", so the BOM glued onto the first header ("Gtin") and row["Gtin"] returned nil for every row — every line was skipped as having an empty GTIN. The result: @firstbase came out empty and all NONPHARMA articles from the firstbase feed were silently missing, so oddb_article.xml shipped only the ~17k Refdata/Swissmedic base instead of the ~190k GS1 set. Now read with encoding "bom|utf-8", which strips the BOM (live file: all 192'807 rows parse).
+
 === 3.0.24 / 16.06.2026
 * Bugfix (--skip-download): cached files fetched with a write mode were silently emptied on every --skip-download run. DownloadMethod#download_as restored the file from the ./downloads cache and then re-opened it with the caller's mode (e.g. "w+"), which truncated it to zero bytes before the read — so the returned data was empty. This blanked epha_interactions.csv (oddb_interaction.xml came out with NBR_RECORD=0) and any other source pulled with a "w+"-style mode (LPPV, Weleda/WALA SL, BAG SL group prices) whenever its file was present in the cache. It surfaced in deploy scripts that download once and rebuild several price increments from the shared cache (e.g. -b -I 45/50/55): the first build was correct, every --skip-download rebuild lost the interactions. The skip branch now opens the restored file read-only, preserving any encoding suffix ("w+:iso-8859-1:utf-8" -> "r:iso-8859-1:utf-8").
 

data/README.md

@@ -56,7 +56,7 @@ see `--help`.
     oddb2xml [option]
       produced files are found under data
     -a, --append              Additional target nonpharma
-    -r, --artikelstamm        Create Artikelstamm Version 3 and 5 for Elexis >= 3.1
+    -r, --artikelstamm        Create Artikelstamm Version 6 for Elexis >= 3.1
     -c, --compress-ext=<s>    format F. {tar.gz|zip}
     -e, --extended            pharma, non-pharma plus prices and non-pharma from zurrose.
                                                           Products without EAN-Code will also be listed.
@@ -337,6 +337,40 @@ drugs from **2026-07-01**; from **2027-01-01** insurers may reject
 invoices without it. See issue
 [#113](https://github.com/zdavatz/oddb2xml/issues/113).
 
+Since 3.0.26 the same codes are also carried per article in the
+**Elexis Artikelstamm v6** output (`--artikelstamm`). Each SL
+price-model `<ITEM>` gains an `<ARTSL>` block with one `<ARTLIM>` per
+limitation:
+
+```xml
+<ITEM PHARMATYPE="P">
+  <GTIN>7680543780251</GTIN>
+  ...
+  <PRODNO>58398</PRODNO>
+  <ARTSL>
+    <PM>true</PM>
+    <ARTLIMS>
+      <ARTLIM>
+        <LIMCD>MABTHERA.01</LIMCD>   <!-- BAG limitation code -->
+        <INDCD>17079.01</INDCD>       <!-- Indikationscode XXXXX.NN -->
+        <VDAT>2023-05-01T00:00:00</VDAT>
+      </ARTLIM>
+      <ARTLIM>
+        <LIMCD>MABTHERA.04</LIMCD>
+        <INDCD>17079.03</INDCD>
+        <VDAT>2026-05-01T00:00:00</VDAT>
+        <VTDAT>2026-06-30T00:00:00</VTDAT>
+      </ARTLIM>
+    </ARTLIMS>
+  </ARTSL>
+</ITEM>
+```
+
+The output bumps from Artikelstamm v5 to **v6** (namespace
+`http://elexis.ch/Elexis_Artikelstamm_v6`, file
+`artikelstamm_DDMMYYYY_v6.xml`, validated against the bundled
+`Elexis_Artikelstamm_v6.xsd`); consumers must switch to the v6 file.
+
 ## Limitation texts in `--fhir` mode
 
 In 3.0.8 we fixed empty `<DescriptionDe/Fr/It>` on every `<Limitation>`
@@ -432,6 +466,29 @@ You can also run
 for your currently open Terminal to download and set the Certificate.
 
 
+## Troubleshooting downloads
+
+oddb2xml runs a connectivity precheck and prints an `oddb2xml CONNECTIVITY
+WARNING` listing any source host it cannot reach; set
+`ODDB2XML_SKIP_PROXY_CHECK=1` to silence it.
+
+A source host may finish the TLS handshake and then **reset the connection
+before sending any HTTP response** (`Errno::ECONNRESET` / "Connection reset by
+peer"; `wget` reports "Read error (Error in the pull function.) in headers").
+When this happens for only one host while everything else is reachable, and a
+normal browser still opens the site, it is usually an **anti-bot gateway (WAF)
+that blocks by TLS fingerprint or source-IP reputation** rather than an outage
+— a User-Agent change does not help. Mitigations: retry later from the same
+host, run the download from a different egress IP, fetch the affected file with
+a browser-grade client (a TLS-impersonating downloader or a headless browser)
+into `downloads/` and continue with `--skip-download`, or ask the source to
+allow-list the host.
+
+The deployment driver `scripts/run_oddb2xml.sh` already retries the whole build
+a few times on such transient failures (`ODDB2XML_RETRIES`,
+`ODDB2XML_RETRY_DELAY`); a persistent per-host block still needs one of the
+mitigations above.
+
 ## Testing
 
 * Calling rake spec runs spec tests.

data/lib/oddb2xml/builder.rb

@@ -1562,9 +1562,47 @@ module Oddb2xml
       end
     end
 
+    # Normalise a BAG validity date to an xs:dateTime for <VDAT>/<VTDAT>.
+    # A bare date (YYYY-MM-DD) gets midnight appended (unzoned, still a valid
+    # xs:dateTime); a full timestamp is passed through; blank -> nil.
+    def elexis_datetime(str)
+      s = str.to_s.strip
+      return nil if s.empty?
+      s = "#{s}T00:00:00" if /\A\d{4}-\d{2}-\d{2}\z/.match?(s)
+      s
+    end
+
+    # Emit the v6 <ARTSL> block for one ITEM: one <ARTLIM> per limitation that
+    # carries a BAG Indikationscode (INDCD). Mandatory on prescriptions/invoices
+    # for SL price-model drugs from 2026-07-01 (issue #113). <PM> is "true"
+    # because the BAG indication code is required only for price-model SL drugs,
+    # which is exactly the set of items that reach this block.
+    def append_artsl(xml, limitations)
+      artlims = (limitations || []).select { |lim| lim[:indcd] && !lim[:indcd].to_s.empty? }
+      artlims = artlims.uniq { |lim| [lim[:code], lim[:indcd]] }
+      return if artlims.empty?
+      xml.ARTSL do
+        xml.PM "true"
+        xml.ARTLIMS do
+          artlims.each do |lim|
+            xml.ARTLIM do
+              xml.LIMCD lim[:code].to_s
+              xml.INDCD lim[:indcd].to_s
+              if (vdat = elexis_datetime(lim[:vdate]))
+                xml.VDAT vdat
+              end
+              if (vtdat = elexis_datetime(lim[:vtdate]))
+                xml.VTDAT vtdat
+              end
+            end
+          end
+        end
+      end
+    end
+
     def build_artikelstamm
-      @@emitted_v5_gtins = []
-      @csv_file = CSV.open(File.join(WORK_DIR, "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v5.csv"), "w+")
+      @@emitted_v6_gtins = []
+      @csv_file = CSV.open(File.join(WORK_DIR, "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v6.csv"), "w+")
       @csv_file << ["gtin", "name", "pkg_size", "galenic_form", "price_ex_factory", "price_public", "prodno", "atc_code", "active_substance", "original", "it-code", "sl-liste"]
       @csv_file.sync = true
       variant = "build_artikelstamm"
@@ -1649,10 +1687,10 @@ module Oddb2xml
                   end
                 end
                 info = @calc_items[pkg_gtin]
-                if @@emitted_v5_gtins.index(pkg_gtin)
+                if @@emitted_v6_gtins.index(pkg_gtin)
                   next
                 else
-                  @@emitted_v5_gtins << pkg_gtin.clone
+                  @@emitted_v6_gtins << pkg_gtin.clone
                 end
                 options = {"PHARMATYPE" => "P"}
                 xml.ITEM(options) do
@@ -1771,6 +1809,8 @@ module Oddb2xml
                     end
                   end
                   xml.PRODNO prodno if prodno
+                  # v6 <ARTSL>: per-article BAG Indikationscodes (issue #113).
+                  append_artsl(xml, package[:limitations])
                   @csv_file << [pkg_gtin, name, package[:unit], measure,
                     pexf || "",
                     ppub || "",
@@ -1780,10 +1820,10 @@ module Oddb2xml
                 end
               end
             else # non pharma
-              if @@emitted_v5_gtins.index(ean13)
+              if @@emitted_v6_gtins.index(ean13)
                 next
               else
-                @@emitted_v5_gtins << ean13.clone
+                @@emitted_v6_gtins << ean13.clone
               end
               # Set the pharmatype to 'Y' for outdated products, which are no longer found
               # in refdata/packungen
@@ -1896,7 +1936,7 @@ module Oddb2xml
         elexis_strftime_format = "%FT%T\.%L%:z"
         @@cumul_ver = (Date.today.year - 2013) * 12 + Date.today.month
         options_xml = {
-          "xmlns" => "http://elexis.ch/Elexis_Artikelstamm_v5",
+          "xmlns" => "http://elexis.ch/Elexis_Artikelstamm_v6",
           "CREATION_DATETIME" => Time.new.strftime(elexis_strftime_format),
           "BUILD_DATETIME" => Time.new.strftime(elexis_strftime_format),
           "DATA_SOURCE" => "oddb2xml"
@@ -2008,7 +2048,7 @@ module Oddb2xml
           end
         end
       end
-      Oddb2xml.log "#{variant}. Done #{nr_products} of #{@products.size} products, #{@limitations.size} limitations and #{nr_articles}/#{@nr_articles} articles. @@emitted_v5_gtins #{@@emitted_v5_gtins.size}"
+      Oddb2xml.log "#{variant}. Done #{nr_products} of #{@products.size} products, #{@limitations.size} limitations and #{nr_articles}/#{@nr_articles} articles. @@emitted_v6_gtins #{@@emitted_v6_gtins.size}"
       # we don't add a SHA256 hash for each element in the article
       # Oddb2xml.add_hash(a_builder.to_xml)
       # doc = REXML::Document.new( source, { :raw => :all })
@@ -2018,13 +2058,13 @@ module Oddb2xml
       lines << "  - #{sprintf("%5d", @limitations.size)} limitations"
       lines << "  - #{sprintf("%5d", @nr_articles)} articles"
       lines << "  - #{sprintf("%5d", @@gtin2ignore.size)} ignored GTINS"
-      @@articlestamm_v5_info_lines = lines
+      @@articlestamm_v6_info_lines = lines
       a_builder.to_xml({indent: 4, encoding: "UTF-8"})
     end
 
     private_class_method
-    def self.articlestamm_v5_info_lines
-      @@articlestamm_v5_info_lines
+    def self.articlestamm_v6_info_lines
+      @@articlestamm_v6_info_lines
     end
   end
 end

data/lib/oddb2xml/cli.rb

@@ -103,8 +103,8 @@ module Oddb2xml
       end
       build
       if @options[:artikelstamm] && system("which xmllint")
-        elexis_v5_xsd = File.expand_path(File.join(__FILE__, "..", "..", "..", "Elexis_Artikelstamm_v5.xsd"))
-        cmd = "xmllint --noout --schema #{elexis_v5_xsd} #{@the_files[:artikelstamm]}"
+        elexis_v6_xsd = File.expand_path(File.join(__FILE__, "..", "..", "..", "Elexis_Artikelstamm_v6.xsd"))
+        cmd = "xmllint --noout --schema #{elexis_v6_xsd} #{@the_files[:artikelstamm]}"
         if system(cmd)
           puts "Validatied #{@the_files[:artikelstamm]}"
         else
@@ -399,7 +399,7 @@ module Oddb2xml
           @the_files[:calc] = "oddb_calc.xml"
         end
         if @options[:artikelstamm]
-          @the_files[:artikelstamm] = "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v5.xml"
+          @the_files[:artikelstamm] = "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v6.xml"
         elsif @options[:address]
           @the_files[:company] = "#{prefix}_betrieb.xml"
           @the_files[:person] = "#{prefix}_medizinalperson.xml"
@@ -438,7 +438,7 @@ module Oddb2xml
       end
       if @options[:artikelstamm]
         lines << "Generated artikelstamm.xml for Elexis"
-        lines += Builder.articlestamm_v5_info_lines
+        lines += Builder.articlestamm_v6_info_lines
       elsif @options[:address]
         {
           "Betrieb" => :@companies,

data/lib/oddb2xml/extractor.rb

@@ -622,7 +622,10 @@ module Oddb2xml
     def to_hash
       data = {}
       return data unless @file && File.exist?(@file)
-      CSV.foreach(@file, headers: true, encoding: "UTF-8") do |row|
+      # The GS1 firstbase CSV is served with a UTF-8 BOM. Without "bom|" the BOM
+      # glues onto the first header ("Gtin"), so row["Gtin"] is nil and every
+      # row is skipped — dropping all -b/firstbase NONPHARMA articles.
+      CSV.foreach(@file, headers: true, encoding: "bom|utf-8") do |row|
         gtin = row["Gtin"].to_s.gsub(/^0+/, "")
         next if gtin.empty?
         data[gtin] = {

data/lib/oddb2xml/fhir_support.rb

@@ -586,6 +586,10 @@ module Oddb2xml
           limitation.LimitationNiveau = ""  # Not in FHIR
           limitation.LimitationValue = ""  # Not in FHIR
           limitation.LimitationCudRef = cud_ref  # carried through for FR/IT resolution
+          # BAG Indikationscode (XXXXX.NN): the v6 Artikelstamm <ARTSL>/<ARTLIM>
+          # block carries it per article (issue #113). Independent of the CUD id
+          # (= LimitationCode), so read the explicit indicationCode extension.
+          limitation.IndicationCode = lim[:indication_code] || ""
           limitation.DescriptionDe = text_de
           limitation.DescriptionFr = ""  # filled by merge_language from FR bundle
           limitation.DescriptionIt = ""  # filled by merge_language from IT bundle
@@ -819,7 +823,9 @@ module Oddb2xml
                   desc_fr: lim.DescriptionFr || "",
                   desc_it: lim.DescriptionIt || "",
                   cud_ref: lim.LimitationCudRef,
+                  indcd: lim.IndicationCode || "",
                   vdate: lim.ValidFromDate || "",
+                  vtdate: lim.ValidThruDate || "",
                   del: is_deleted
                 }
               end

data/lib/oddb2xml/options.rb

@@ -17,7 +17,7 @@ module Oddb2xml
           produced files are found under data
         EOS
         opt :append, "Additional target nonpharma", default: false
-        opt :artikelstamm, "Create Artikelstamm Version 3 and 5 for Elexis >= 3.1"
+        opt :artikelstamm, "Create Artikelstamm Version 6 for Elexis >= 3.1"
         opt :compress_ext, "format F. {tar.gz|zip}", type: :string, default: nil, short: "c"
         opt :extended, "pharma, non-pharma plus prices and non-pharma from zurrose.
                             Products without EAN-Code will also be listed.

data/lib/oddb2xml/version.rb

@@ -1,3 +1,3 @@
 module Oddb2xml
-  VERSION = "3.0.24"
+  VERSION = "3.0.26"
 end

data/scripts/generate_index_html.sh

@@ -0,0 +1,202 @@
+#!/usr/bin/env bash
+#
+# Generate the mediupdatexml.oddb.org landing page ($DOCROOT/index.html) with
+# live product counts:
+#   PHARMA    = Swissmedic-registered medicines (<SMNO> in default/oddb_article.xml)
+#   NONPHARMA = GTINs in the GS1 firstbase download (firstbase.csv, minus header)
+#
+# Usage: generate_index_html.sh DOCROOT [FIRSTBASE_CSV]
+#   DOCROOT        where to write index.html (default /home/zdavatz/oddb2xml)
+#   FIRSTBASE_CSV  the GS1 firstbase CSV (default <DOCROOT>-build/downloads/firstbase.csv)
+#
+# Called from run_oddb2xml.sh (after each build) and from
+# setup_mediupdatexml_web.sh (initial creation). Counts that can't be computed
+# are shown as "—".
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+DOCROOT="${1:-/home/zdavatz/oddb2xml}"
+FIRSTBASE_CSV="${2:-${DOCROOT%/}-build/downloads/firstbase.csv}"
+ARTICLE_XML="${DOCROOT%/}/default/oddb_article.xml"
+
+# Apache combined access log(s) for the visitors/region graph (last STATS_DAYS).
+# Readable by the user only if it is in the "adm" group (sudo usermod -aG adm <user>).
+ACCESS_LOG_GLOB="${ACCESS_LOG_GLOB:-/var/log/apache2/mediupdatexml.oddb.org_access.log*}"
+STATS_DAYS="${STATS_DAYS:-14}"
+
+# Swiss-style thousands separator (192807 -> 192'807); "—" passes through.
+group() { [[ "$1" =~ ^[0-9]+$ ]] && printf "%s" "$1" | sed -re ":a;s/([0-9])([0-9]{3})($|[^0-9])/\1'\2\3/;ta" || printf "%s" "$1"; }
+
+total="—"
+[[ -f "$ARTICLE_XML" ]] && total=$(grep -c '<ART ' "$ARTICLE_XML" || true)
+
+pharma="—"
+[[ -f "$ARTICLE_XML" ]] && pharma=$(grep -c '<SMNO>' "$ARTICLE_XML" || true)
+
+nonpharma="—"
+[[ -f "$FIRSTBASE_CSV" ]] && nonpharma=$(( $(wc -l < "$FIRSTBASE_CSV") - 1 ))
+
+stand=$(date '+%d.%m.%Y %H:%M')
+
+# Visitors/sessions/region graph as a ready-to-embed inline-SVG HTML fragment.
+# Self-contained (pure Python stdlib + cached DB-IP country CSV). Stays empty
+# when the access log is unreadable or has no data, so the page degrades to
+# simply omitting the section.
+stats_fragment=""
+if [[ -f "$SCRIPT_DIR/visitor_stats.py" ]]; then
+  stats_fragment=$(python3 "$SCRIPT_DIR/visitor_stats.py" \
+    "$ACCESS_LOG_GLOB" "$(dirname "$FIRSTBASE_CSV")" "$STATS_DAYS" 2>/dev/null || true)
+fi
+
+mkdir -p "$DOCROOT"
+
+# Logo (self-contained SVG): brand-blue rounded badge, white pharma/Swiss cross
+# flanked by XML angle brackets "< >". Written atomically next to index.html and
+# used both as the top-right header image and as the favicon.
+logo_tmp="${DOCROOT%/}/.logo.svg.$$"
+cat > "$logo_tmp" <<'SVG'
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="oddb2xml">
+  <defs>
+    <linearGradient id="g" x1="0" y1="0" x2="1" y2="1">
+      <stop offset="0" stop-color="#1a6dff"/>
+      <stop offset="1" stop-color="#0a3d8f"/>
+    </linearGradient>
+  </defs>
+  <rect width="64" height="64" rx="14" fill="url(#g)"/>
+  <g stroke="#e2231a" stroke-width="3.4" stroke-linecap="round" stroke-linejoin="round" fill="none">
+    <polyline points="15,24 9,32 15,40"/>
+    <polyline points="49,24 55,32 49,40"/>
+  </g>
+  <g fill="#ffffff">
+    <rect x="26.5" y="18" width="11" height="28" rx="2.5"/>
+    <rect x="18" y="26.5" width="28" height="11" rx="2.5"/>
+  </g>
+</svg>
+SVG
+chmod 644 "$logo_tmp"
+mv -f "$logo_tmp" "${DOCROOT%/}/logo.svg"
+
+# Write atomically via temp + mv so the page can be refreshed regardless of who
+# owns the existing index.html (setup runs as root, run_oddb2xml.sh as the user);
+# mv only needs write on the directory, which both have.
+tmp="${DOCROOT%/}/.index.html.$$"
+cat > "$tmp" <<HTML
+<!DOCTYPE html>
+<html lang="de">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>mediupdatexml.oddb.org — Downloads</title>
+  <!-- open every link in a new tab -->
+  <base target="_blank">
+  <meta name="referrer" content="strict-origin-when-cross-origin">
+  <link rel="icon" type="image/svg+xml" href="logo.svg">
+  <style>
+    body { font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
+           max-width: 820px; margin: 2.5rem auto; padding: 0 1.2rem; color: #1a1a1a; line-height: 1.5; }
+    h1 { font-size: 1.6rem; margin-bottom: .2rem; }
+    h2 { font-size: 1.15rem; margin-top: 2rem; border-bottom: 1px solid #ddd; padding-bottom: .3rem; }
+    .sub { color: #666; margin-top: 0; }
+    ul { list-style: none; padding-left: 0; }
+    li { margin: .4rem 0; }
+    a { color: #0a58ca; text-decoration: none; }
+    a:hover { text-decoration: underline; }
+    .desc { color: #666; font-size: .9rem; }
+    code { background: #f4f4f4; padding: .1rem .3rem; border-radius: 3px; }
+    .stats { display: flex; gap: 1.5rem; margin: 1rem 0; flex-wrap: wrap; }
+    .stat { background: #f4f7fb; border: 1px solid #dce4ef; border-radius: 8px; padding: .8rem 1.2rem; }
+    .stat .n { font-size: 1.5rem; font-weight: 600; color: #0a58ca; }
+    .stat .l { color: #555; font-size: .85rem; }
+    footer { margin-top: 3rem; color: #888; font-size: .85rem; }
+    ul.firms { columns: 2; column-gap: 2rem; }
+    ul.firms li { margin: .25rem 0; break-inside: avoid; }
+    .topbar { display: flex; justify-content: space-between; align-items: flex-start; gap: 1rem; }
+    .topbar .title h1 { margin: 0 0 .2rem; }
+    .topbar .logo { width: 64px; height: 64px; flex: 0 0 auto; }
+  </style>
+</head>
+<body>
+  <header class="topbar">
+    <div class="title">
+      <h1>oddb2xml &amp; aips2sqlite Downloads</h1>
+      <p class="sub">Schweizer Arzneimitteldaten — täglich aktualisiert (01:00 Uhr). Stand: ${stand}</p>
+    </div>
+    <a href="mailto:zdavatz@ywesee.com" title="Fragen? zdavatz at ywesee dot com"><img class="logo" src="logo.svg" alt="oddb2xml Logo" width="64" height="64"></a>
+  </header>
+
+  <div class="stats">
+    <div class="stat"><div class="n">$(group "$pharma")</div><div class="l">Medikamente (PHARMA)</div></div>
+    <div class="stat"><div class="n"><a href="https://id.gs1.ch/01/07612345000961">$(group "$nonpharma")</a></div><div class="l">Firstbase-Produkte (NONPHARMA)</div></div>
+    <div class="stat"><div class="n"><a href="default/oddb_article.xml">$(group "$total")</a></div><div class="l">Artikel total (<a href="default/oddb_article.xml"><code>oddb_article.xml</code></a>)</div></div>
+  </div>
+
+  <h2>oddb2xml — Artikel-/Produkt-Feeds (<code>-b</code> firstbase)</h2>
+  <ul>
+    <li><a href="default/">default/</a> <span class="desc">— ohne Preisaufschlag</span></li>
+    <li><a href="45/">45/</a> <span class="desc">— Wiederverkaufspreis +45&nbsp;%</span></li>
+    <li><a href="50/">50/</a> <span class="desc">— Wiederverkaufspreis +50&nbsp;%</span></li>
+    <li><a href="55/">55/</a> <span class="desc">— Wiederverkaufspreis +55&nbsp;%</span></li>
+  </ul>
+  <p class="desc">Jedes Verzeichnis enthält die gleichen Dateien (Direktlinks zum <code>default/</code>-Feed):
+  <a href="default/oddb_article.xml">oddb_article.xml</a>,
+  <a href="default/oddb_product.xml">oddb_product.xml</a>,
+  <a href="default/oddb_calc.xml">oddb_calc.xml</a>,
+  <a href="default/oddb_interaction.xml">oddb_interaction.xml</a>,
+  <a href="default/oddb_limitation.xml">oddb_limitation.xml</a>,
+  <a href="default/oddb_substance.xml">oddb_substance.xml</a>,
+  <a href="default/oddb_code.xml">oddb_code.xml</a> sowie
+  <a href="default/oddb2xml.zip">oddb2xml.zip</a> (alle Dateien gepackt).</p>
+
+  <h2>aips2sqlite — Fachinformationen &amp; AmiKo-Datenbanken</h2>
+  <ul>
+    <li><a href="/aips2sqlite/fis/">fis/</a> <span class="desc">— Fachinformationen als XML/HTML (DE/FR/IT)</span></li>
+    <li><a href="/aips2sqlite/amiko_db_full_idx_de.db">amiko_db_full_idx_de.db</a> <span class="desc">— AmiKo-Datenbank Deutsch</span></li>
+    <li><a href="/aips2sqlite/amiko_db_full_idx_fr.db">amiko_db_full_idx_fr.db</a> <span class="desc">— AmiKo-Datenbank Französisch</span></li>
+    <li><a href="/aips2sqlite/oddb2xml_swissmedic_sequences.csv">oddb2xml_swissmedic_sequences.csv</a> <span class="desc">— Swissmedic-Sequenzen</span></li>
+    <li><a href="/aips2sqlite/atc_codes_used_set.txt">atc_codes_used_set.txt</a> <span class="desc">— verwendete ATC-Codes</span></li>
+    <li><a href="/aips2sqlite/">/aips2sqlite/</a> <span class="desc">— gesamtes Verzeichnis durchsuchen</span></li>
+  </ul>
+
+  <h2>MediUpdate XML bei HIN</h2>
+  <ul>
+    <li><a href="https://www.hin.ch/de/services/mediupdate-xml.cfm">www.hin.ch/de/services/mediupdate-xml.cfm</a></li>
+  </ul>
+
+  <h2>Softwarehäuser, die oddb2xml einsetzen</h2>
+  <ul class="firms">
+    <li><a href="https://www.advancedconcepts.ch/">Advanced Concepts AG</a></li>
+    <li><a href="https://www.bluecare.ch/">Bluecare AG</a></li>
+    <li><a href="https://corona.ch/">Corona Informatik AG</a></li>
+    <li><a href="https://www.derma2go.com/de/">derma2go AG</a></li>
+    <li><a href="https://www.diagnosia.com/">Diagnosia Internetservices GmbH</a></li>
+    <li><a href="https://elexis.ch/glp/index.html">Elexis</a></li>
+    <li><a href="https://www.emedswiss.ch/">emedSwiss SA</a></li>
+    <li><a href="https://gartenmann.ch/">Gartenmann Software AG</a></li>
+    <li><a href="https://hexabit.ch/">Hexabit GmbH</a></li>
+    <li><a href="https://www.hausarztmedizin.uzh.ch/de.html">Institut für Hausarztmedizin</a></li>
+    <li><a href="https://www.itw-informatik.ch/de/">ITW INFORMATIK AG</a></li>
+    <li><a href="https://www.lama-media.com/">Lama Media</a></li>
+    <li><a href="https://medab.org/country/ch">MEDAB</a></li>
+    <li><a href="https://www.pharmedsolutions.ch/">Pharmed Solutions GmbH</a></li>
+    <li><a href="https://praxinova.ch/">Praxinova AG</a></li>
+    <li><a href="https://seantis.ch/">seantis gmbh</a></li>
+    <li><a href="https://www.geteyesoft.ch/">Siplus SA – Eyesoft</a></li>
+    <li><a href="https://swiss-mr.ch/">SMR – Swiss Medical Record GmbH</a></li>
+    <li><a href="https://triboni.com/site/">Triboni AG</a></li>
+    <li><a href="https://www.vitabyte.ch/">Vitabyte AG</a></li>
+    <li><a href="https://zollsoft.de/">zollsoft GmbH</a></li>
+  </ul>
+
+  ${stats_fragment}
+
+  <footer>
+    Fragen: <a href="mailto:zdavatz@ywesee.com">zdavatz at ywesee dot com</a> &middot; Tel: 043 540 05 50
+  </footer>
+</body>
+</html>
+HTML
+chmod 644 "$tmp"
+mv -f "$tmp" "${DOCROOT%/}/index.html"
+
+echo "Wrote ${DOCROOT%/}/index.html  (PHARMA=$pharma NONPHARMA=$nonpharma)"

data/scripts/run_oddb2xml.sh

@@ -38,9 +38,35 @@ BUILD_DIR="${BUILD_DIR:-${OUT_DIR%/}-build}"
 INCREMENTS="${INCREMENTS:-45 50 55}"
 ODDB2XML_BIN="${ODDB2XML_BIN:-oddb2xml}"
 TRANSFER_CMD="${TRANSFER_CMD:-$SCRIPT_DIR/transfer.sh}"
+# Transient upstream download failures (e.g. Swissmedic resetting the
+# connection, Errno::ECONNRESET) used to abort the whole nightly run under
+# `set -e`. Retry the oddb2xml build a few times before giving up.
+ODDB2XML_RETRIES="${ODDB2XML_RETRIES:-3}"
+ODDB2XML_RETRY_DELAY="${ODDB2XML_RETRY_DELAY:-120}"
 
 log() { printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*"; }
 
+# run_with_retry <description> -- <command...>
+# Retry a flaky command up to ODDB2XML_RETRIES times, sleeping
+# ODDB2XML_RETRY_DELAY seconds between attempts. Running the command as the
+# `until` condition keeps it exempt from `set -e`, so a failed attempt retries
+# instead of killing the script; the final failure is propagated via return.
+run_with_retry() {
+  local desc="$1"; shift
+  [[ "${1:-}" == "--" ]] && shift
+  local attempt=1 rc=0
+  until "$@"; do
+    rc=$?
+    if [[ $attempt -ge $ODDB2XML_RETRIES ]]; then
+      log "ERROR: $desc failed after $attempt attempts (last exit $rc)"
+      return $rc
+    fi
+    log "WARNING: $desc failed (exit $rc), attempt $attempt/$ODDB2XML_RETRIES; retrying in ${ODDB2XML_RETRY_DELAY}s"
+    sleep "$ODDB2XML_RETRY_DELAY"
+    attempt=$((attempt + 1))
+  done
+}
+
 # 1. Install / update the published gem unless told otherwise.
 if [[ "${SKIP_GEM_INSTALL:-0}" != "1" ]]; then
   log "Installing oddb2xml gem"
@@ -69,7 +95,10 @@ build_one() {
 
   log "Building increment '${inc:-none}' -> $dest"
   rm -f oddb*.zip
-  "$ODDB2XML_BIN" "${dl_opt[@]}" -b "${inc_opt[@]}" -c zip
+  # On a retry the first build re-downloads from scratch (dl_opt empty), which
+  # also clears any partial download left by the failed attempt.
+  run_with_retry "oddb2xml build '${inc:-none}'" -- \
+    "$ODDB2XML_BIN" "${dl_opt[@]}" -b "${inc_opt[@]}" -c zip
 
   shopt -s nullglob
   local zips=(oddb*.zip)
@@ -89,6 +118,14 @@ for inc in $INCREMENTS; do
 done
 build_one "" "default"           # final run with no increment
 
+# 2b. Refresh the download landing page with the live PHARMA/NONPHARMA counts
+#     (PHARMA from default/oddb_article.xml, NONPHARMA from the GS1 firstbase CSV).
+if [[ -x "$SCRIPT_DIR/generate_index_html.sh" ]]; then
+  log "Refreshing landing page index.html"
+  "$SCRIPT_DIR/generate_index_html.sh" "$OUT_DIR" "$BUILD_DIR/downloads/firstbase.csv" || \
+    log "WARNING: could not regenerate index.html"
+fi
+
 # 3. Optional hand-off to the transfer step (scripts/transfer.sh).
 if [[ "${RUN_TRANSFER:-0}" == "1" ]]; then
   log "Running transfer: $TRANSFER_CMD"