oddb2xml 3.0.25 → 3.0.26

data/History.txt

@@ -1,3 +1,6 @@
+=== 3.0.26 / 29.06.2026
+* New (--artikelstamm): emit the Elexis Artikelstamm v6 format with per-article BAG Indikationscodes (INDC). The output bumps from v5 to v6 -- namespace http://elexis.ch/Elexis_Artikelstamm_v6, file artikelstamm_DDMMYYYY_v6.xml/.csv, validated against the new bundled Elexis_Artikelstamm_v6.xsd -- and every SL price-model <ITEM> now carries an <ARTSL> block: <PM>true</PM> plus one <ARTLIM> per limitation with <LIMCD> (BAG limitation code), <INDCD> (the Indikationscode XXXXX.NN), and the validity dates <VDAT>/<VTDAT>. The indication codes were already read from the BAG SL FHIR feed's indicationCode extension (used since 3.0.7 for oddb_product.xml's <INDICATION_CODE>); they are now also carried per package limitation and grouped onto each article. Mandatory on prescriptions/invoices for SL price-model drugs from 2026-07-01 (issue #113). The v6 XSD is the canonical MEDEVIT schema extended with oddb2xml's historical Italian elements (DSCRI / DOSAGE_FORMI) so the output keeps validating. Consumers must switch from the v5 to the v6 file.
+
 === 3.0.25 / 17.06.2026
 * Bugfix (-b/--firstbase): the GS1 Switzerland firstbase CSV (id.gs1.ch) is served with a UTF-8 BOM. FirstbaseExtractor read it with encoding "UTF-8", so the BOM glued onto the first header ("Gtin") and row["Gtin"] returned nil for every row — every line was skipped as having an empty GTIN. The result: @firstbase came out empty and all NONPHARMA articles from the firstbase feed were silently missing, so oddb_article.xml shipped only the ~17k Refdata/Swissmedic base instead of the ~190k GS1 set. Now read with encoding "bom|utf-8", which strips the BOM (live file: all 192'807 rows parse).
 

data/README.md

@@ -56,7 +56,7 @@ see `--help`.
     oddb2xml [option]
       produced files are found under data
     -a, --append              Additional target nonpharma
-    -r, --artikelstamm        Create Artikelstamm Version 3 and 5 for Elexis >= 3.1
+    -r, --artikelstamm        Create Artikelstamm Version 6 for Elexis >= 3.1
     -c, --compress-ext=<s>    format F. {tar.gz|zip}
     -e, --extended            pharma, non-pharma plus prices and non-pharma from zurrose.
                                                           Products without EAN-Code will also be listed.
@@ -337,6 +337,40 @@ drugs from **2026-07-01**; from **2027-01-01** insurers may reject
 invoices without it. See issue
 [#113](https://github.com/zdavatz/oddb2xml/issues/113).
 
+Since 3.0.26 the same codes are also carried per article in the
+**Elexis Artikelstamm v6** output (`--artikelstamm`). Each SL
+price-model `<ITEM>` gains an `<ARTSL>` block with one `<ARTLIM>` per
+limitation:
+
+```xml
+<ITEM PHARMATYPE="P">
+  <GTIN>7680543780251</GTIN>
+  ...
+  <PRODNO>58398</PRODNO>
+  <ARTSL>
+    <PM>true</PM>
+    <ARTLIMS>
+      <ARTLIM>
+        <LIMCD>MABTHERA.01</LIMCD>   <!-- BAG limitation code -->
+        <INDCD>17079.01</INDCD>       <!-- Indikationscode XXXXX.NN -->
+        <VDAT>2023-05-01T00:00:00</VDAT>
+      </ARTLIM>
+      <ARTLIM>
+        <LIMCD>MABTHERA.04</LIMCD>
+        <INDCD>17079.03</INDCD>
+        <VDAT>2026-05-01T00:00:00</VDAT>
+        <VTDAT>2026-06-30T00:00:00</VTDAT>
+      </ARTLIM>
+    </ARTLIMS>
+  </ARTSL>
+</ITEM>
+```
+
+The output bumps from Artikelstamm v5 to **v6** (namespace
+`http://elexis.ch/Elexis_Artikelstamm_v6`, file
+`artikelstamm_DDMMYYYY_v6.xml`, validated against the bundled
+`Elexis_Artikelstamm_v6.xsd`); consumers must switch to the v6 file.
+
 ## Limitation texts in `--fhir` mode
 
 In 3.0.8 we fixed empty `<DescriptionDe/Fr/It>` on every `<Limitation>`
@@ -432,6 +466,29 @@ You can also run
 for your currently open Terminal to download and set the Certificate.
 
 
+## Troubleshooting downloads
+
+oddb2xml runs a connectivity precheck and prints an `oddb2xml CONNECTIVITY
+WARNING` listing any source host it cannot reach; set
+`ODDB2XML_SKIP_PROXY_CHECK=1` to silence it.
+
+A source host may finish the TLS handshake and then **reset the connection
+before sending any HTTP response** (`Errno::ECONNRESET` / "Connection reset by
+peer"; `wget` reports "Read error (Error in the pull function.) in headers").
+When this happens for only one host while everything else is reachable, and a
+normal browser still opens the site, it is usually an **anti-bot gateway (WAF)
+that blocks by TLS fingerprint or source-IP reputation** rather than an outage
+— a User-Agent change does not help. Mitigations: retry later from the same
+host, run the download from a different egress IP, fetch the affected file with
+a browser-grade client (a TLS-impersonating downloader or a headless browser)
+into `downloads/` and continue with `--skip-download`, or ask the source to
+allow-list the host.
+
+The deployment driver `scripts/run_oddb2xml.sh` already retries the whole build
+a few times on such transient failures (`ODDB2XML_RETRIES`,
+`ODDB2XML_RETRY_DELAY`); a persistent per-host block still needs one of the
+mitigations above.
+
 ## Testing
 
 * Calling rake spec runs spec tests.

data/lib/oddb2xml/builder.rb

@@ -1562,9 +1562,47 @@ module Oddb2xml
       end
     end
 
+    # Normalise a BAG validity date to an xs:dateTime for <VDAT>/<VTDAT>.
+    # A bare date (YYYY-MM-DD) gets midnight appended (unzoned, still a valid
+    # xs:dateTime); a full timestamp is passed through; blank -> nil.
+    def elexis_datetime(str)
+      s = str.to_s.strip
+      return nil if s.empty?
+      s = "#{s}T00:00:00" if /\A\d{4}-\d{2}-\d{2}\z/.match?(s)
+      s
+    end
+
+    # Emit the v6 <ARTSL> block for one ITEM: one <ARTLIM> per limitation that
+    # carries a BAG Indikationscode (INDCD). Mandatory on prescriptions/invoices
+    # for SL price-model drugs from 2026-07-01 (issue #113). <PM> is "true"
+    # because the BAG indication code is required only for price-model SL drugs,
+    # which is exactly the set of items that reach this block.
+    def append_artsl(xml, limitations)
+      artlims = (limitations || []).select { |lim| lim[:indcd] && !lim[:indcd].to_s.empty? }
+      artlims = artlims.uniq { |lim| [lim[:code], lim[:indcd]] }
+      return if artlims.empty?
+      xml.ARTSL do
+        xml.PM "true"
+        xml.ARTLIMS do
+          artlims.each do |lim|
+            xml.ARTLIM do
+              xml.LIMCD lim[:code].to_s
+              xml.INDCD lim[:indcd].to_s
+              if (vdat = elexis_datetime(lim[:vdate]))
+                xml.VDAT vdat
+              end
+              if (vtdat = elexis_datetime(lim[:vtdate]))
+                xml.VTDAT vtdat
+              end
+            end
+          end
+        end
+      end
+    end
+
     def build_artikelstamm
-      @@emitted_v5_gtins = []
-      @csv_file = CSV.open(File.join(WORK_DIR, "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v5.csv"), "w+")
+      @@emitted_v6_gtins = []
+      @csv_file = CSV.open(File.join(WORK_DIR, "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v6.csv"), "w+")
       @csv_file << ["gtin", "name", "pkg_size", "galenic_form", "price_ex_factory", "price_public", "prodno", "atc_code", "active_substance", "original", "it-code", "sl-liste"]
       @csv_file.sync = true
       variant = "build_artikelstamm"
@@ -1649,10 +1687,10 @@ module Oddb2xml
                   end
                 end
                 info = @calc_items[pkg_gtin]
-                if @@emitted_v5_gtins.index(pkg_gtin)
+                if @@emitted_v6_gtins.index(pkg_gtin)
                   next
                 else
-                  @@emitted_v5_gtins << pkg_gtin.clone
+                  @@emitted_v6_gtins << pkg_gtin.clone
                 end
                 options = {"PHARMATYPE" => "P"}
                 xml.ITEM(options) do
@@ -1771,6 +1809,8 @@ module Oddb2xml
                     end
                   end
                   xml.PRODNO prodno if prodno
+                  # v6 <ARTSL>: per-article BAG Indikationscodes (issue #113).
+                  append_artsl(xml, package[:limitations])
                   @csv_file << [pkg_gtin, name, package[:unit], measure,
                     pexf || "",
                     ppub || "",
@@ -1780,10 +1820,10 @@ module Oddb2xml
                 end
               end
             else # non pharma
-              if @@emitted_v5_gtins.index(ean13)
+              if @@emitted_v6_gtins.index(ean13)
                 next
               else
-                @@emitted_v5_gtins << ean13.clone
+                @@emitted_v6_gtins << ean13.clone
               end
               # Set the pharmatype to 'Y' for outdated products, which are no longer found
               # in refdata/packungen
@@ -1896,7 +1936,7 @@ module Oddb2xml
         elexis_strftime_format = "%FT%T\.%L%:z"
         @@cumul_ver = (Date.today.year - 2013) * 12 + Date.today.month
         options_xml = {
-          "xmlns" => "http://elexis.ch/Elexis_Artikelstamm_v5",
+          "xmlns" => "http://elexis.ch/Elexis_Artikelstamm_v6",
           "CREATION_DATETIME" => Time.new.strftime(elexis_strftime_format),
           "BUILD_DATETIME" => Time.new.strftime(elexis_strftime_format),
           "DATA_SOURCE" => "oddb2xml"
@@ -2008,7 +2048,7 @@ module Oddb2xml
           end
         end
       end
-      Oddb2xml.log "#{variant}. Done #{nr_products} of #{@products.size} products, #{@limitations.size} limitations and #{nr_articles}/#{@nr_articles} articles. @@emitted_v5_gtins #{@@emitted_v5_gtins.size}"
+      Oddb2xml.log "#{variant}. Done #{nr_products} of #{@products.size} products, #{@limitations.size} limitations and #{nr_articles}/#{@nr_articles} articles. @@emitted_v6_gtins #{@@emitted_v6_gtins.size}"
       # we don't add a SHA256 hash for each element in the article
       # Oddb2xml.add_hash(a_builder.to_xml)
       # doc = REXML::Document.new( source, { :raw => :all })
@@ -2018,13 +2058,13 @@ module Oddb2xml
       lines << "  - #{sprintf("%5d", @limitations.size)} limitations"
       lines << "  - #{sprintf("%5d", @nr_articles)} articles"
       lines << "  - #{sprintf("%5d", @@gtin2ignore.size)} ignored GTINS"
-      @@articlestamm_v5_info_lines = lines
+      @@articlestamm_v6_info_lines = lines
       a_builder.to_xml({indent: 4, encoding: "UTF-8"})
     end
 
     private_class_method
-    def self.articlestamm_v5_info_lines
-      @@articlestamm_v5_info_lines
+    def self.articlestamm_v6_info_lines
+      @@articlestamm_v6_info_lines
     end
   end
 end

data/lib/oddb2xml/cli.rb

@@ -103,8 +103,8 @@ module Oddb2xml
       end
       build
       if @options[:artikelstamm] && system("which xmllint")
-        elexis_v5_xsd = File.expand_path(File.join(__FILE__, "..", "..", "..", "Elexis_Artikelstamm_v5.xsd"))
-        cmd = "xmllint --noout --schema #{elexis_v5_xsd} #{@the_files[:artikelstamm]}"
+        elexis_v6_xsd = File.expand_path(File.join(__FILE__, "..", "..", "..", "Elexis_Artikelstamm_v6.xsd"))
+        cmd = "xmllint --noout --schema #{elexis_v6_xsd} #{@the_files[:artikelstamm]}"
         if system(cmd)
           puts "Validatied #{@the_files[:artikelstamm]}"
         else
@@ -399,7 +399,7 @@ module Oddb2xml
           @the_files[:calc] = "oddb_calc.xml"
         end
         if @options[:artikelstamm]
-          @the_files[:artikelstamm] = "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v5.xml"
+          @the_files[:artikelstamm] = "artikelstamm_#{Date.today.strftime("%d%m%Y")}_v6.xml"
         elsif @options[:address]
           @the_files[:company] = "#{prefix}_betrieb.xml"
           @the_files[:person] = "#{prefix}_medizinalperson.xml"
@@ -438,7 +438,7 @@ module Oddb2xml
       end
       if @options[:artikelstamm]
         lines << "Generated artikelstamm.xml for Elexis"
-        lines += Builder.articlestamm_v5_info_lines
+        lines += Builder.articlestamm_v6_info_lines
       elsif @options[:address]
         {
           "Betrieb" => :@companies,

data/lib/oddb2xml/fhir_support.rb

@@ -586,6 +586,10 @@ module Oddb2xml
           limitation.LimitationNiveau = ""  # Not in FHIR
           limitation.LimitationValue = ""  # Not in FHIR
           limitation.LimitationCudRef = cud_ref  # carried through for FR/IT resolution
+          # BAG Indikationscode (XXXXX.NN): the v6 Artikelstamm <ARTSL>/<ARTLIM>
+          # block carries it per article (issue #113). Independent of the CUD id
+          # (= LimitationCode), so read the explicit indicationCode extension.
+          limitation.IndicationCode = lim[:indication_code] || ""
           limitation.DescriptionDe = text_de
           limitation.DescriptionFr = ""  # filled by merge_language from FR bundle
           limitation.DescriptionIt = ""  # filled by merge_language from IT bundle
@@ -819,7 +823,9 @@ module Oddb2xml
                   desc_fr: lim.DescriptionFr || "",
                   desc_it: lim.DescriptionIt || "",
                   cud_ref: lim.LimitationCudRef,
+                  indcd: lim.IndicationCode || "",
                   vdate: lim.ValidFromDate || "",
+                  vtdate: lim.ValidThruDate || "",
                   del: is_deleted
                 }
               end

data/lib/oddb2xml/options.rb

@@ -17,7 +17,7 @@ module Oddb2xml
           produced files are found under data
         EOS
         opt :append, "Additional target nonpharma", default: false
-        opt :artikelstamm, "Create Artikelstamm Version 3 and 5 for Elexis >= 3.1"
+        opt :artikelstamm, "Create Artikelstamm Version 6 for Elexis >= 3.1"
         opt :compress_ext, "format F. {tar.gz|zip}", type: :string, default: nil, short: "c"
         opt :extended, "pharma, non-pharma plus prices and non-pharma from zurrose.
                             Products without EAN-Code will also be listed.

data/lib/oddb2xml/version.rb

@@ -1,3 +1,3 @@
 module Oddb2xml
-  VERSION = "3.0.25"
+  VERSION = "3.0.26"
 end

data/scripts/generate_index_html.sh

@@ -0,0 +1,202 @@
+#!/usr/bin/env bash
+#
+# Generate the mediupdatexml.oddb.org landing page ($DOCROOT/index.html) with
+# live product counts:
+#   PHARMA    = Swissmedic-registered medicines (<SMNO> in default/oddb_article.xml)
+#   NONPHARMA = GTINs in the GS1 firstbase download (firstbase.csv, minus header)
+#
+# Usage: generate_index_html.sh DOCROOT [FIRSTBASE_CSV]
+#   DOCROOT        where to write index.html (default /home/zdavatz/oddb2xml)
+#   FIRSTBASE_CSV  the GS1 firstbase CSV (default <DOCROOT>-build/downloads/firstbase.csv)
+#
+# Called from run_oddb2xml.sh (after each build) and from
+# setup_mediupdatexml_web.sh (initial creation). Counts that can't be computed
+# are shown as "—".
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+DOCROOT="${1:-/home/zdavatz/oddb2xml}"
+FIRSTBASE_CSV="${2:-${DOCROOT%/}-build/downloads/firstbase.csv}"
+ARTICLE_XML="${DOCROOT%/}/default/oddb_article.xml"
+
+# Apache combined access log(s) for the visitors/region graph (last STATS_DAYS).
+# Readable by the user only if it is in the "adm" group (sudo usermod -aG adm <user>).
+ACCESS_LOG_GLOB="${ACCESS_LOG_GLOB:-/var/log/apache2/mediupdatexml.oddb.org_access.log*}"
+STATS_DAYS="${STATS_DAYS:-14}"
+
+# Swiss-style thousands separator (192807 -> 192'807); "—" passes through.
+group() { [[ "$1" =~ ^[0-9]+$ ]] && printf "%s" "$1" | sed -re ":a;s/([0-9])([0-9]{3})($|[^0-9])/\1'\2\3/;ta" || printf "%s" "$1"; }
+
+total="—"
+[[ -f "$ARTICLE_XML" ]] && total=$(grep -c '<ART ' "$ARTICLE_XML" || true)
+
+pharma="—"
+[[ -f "$ARTICLE_XML" ]] && pharma=$(grep -c '<SMNO>' "$ARTICLE_XML" || true)
+
+nonpharma="—"
+[[ -f "$FIRSTBASE_CSV" ]] && nonpharma=$(( $(wc -l < "$FIRSTBASE_CSV") - 1 ))
+
+stand=$(date '+%d.%m.%Y %H:%M')
+
+# Visitors/sessions/region graph as a ready-to-embed inline-SVG HTML fragment.
+# Self-contained (pure Python stdlib + cached DB-IP country CSV). Stays empty
+# when the access log is unreadable or has no data, so the page degrades to
+# simply omitting the section.
+stats_fragment=""
+if [[ -f "$SCRIPT_DIR/visitor_stats.py" ]]; then
+  stats_fragment=$(python3 "$SCRIPT_DIR/visitor_stats.py" \
+    "$ACCESS_LOG_GLOB" "$(dirname "$FIRSTBASE_CSV")" "$STATS_DAYS" 2>/dev/null || true)
+fi
+
+mkdir -p "$DOCROOT"
+
+# Logo (self-contained SVG): brand-blue rounded badge, white pharma/Swiss cross
+# flanked by XML angle brackets "< >". Written atomically next to index.html and
+# used both as the top-right header image and as the favicon.
+logo_tmp="${DOCROOT%/}/.logo.svg.$$"
+cat > "$logo_tmp" <<'SVG'
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="oddb2xml">
+  <defs>
+    <linearGradient id="g" x1="0" y1="0" x2="1" y2="1">
+      <stop offset="0" stop-color="#1a6dff"/>
+      <stop offset="1" stop-color="#0a3d8f"/>
+    </linearGradient>
+  </defs>
+  <rect width="64" height="64" rx="14" fill="url(#g)"/>
+  <g stroke="#e2231a" stroke-width="3.4" stroke-linecap="round" stroke-linejoin="round" fill="none">
+    <polyline points="15,24 9,32 15,40"/>
+    <polyline points="49,24 55,32 49,40"/>
+  </g>
+  <g fill="#ffffff">
+    <rect x="26.5" y="18" width="11" height="28" rx="2.5"/>
+    <rect x="18" y="26.5" width="28" height="11" rx="2.5"/>
+  </g>
+</svg>
+SVG
+chmod 644 "$logo_tmp"
+mv -f "$logo_tmp" "${DOCROOT%/}/logo.svg"
+
+# Write atomically via temp + mv so the page can be refreshed regardless of who
+# owns the existing index.html (setup runs as root, run_oddb2xml.sh as the user);
+# mv only needs write on the directory, which both have.
+tmp="${DOCROOT%/}/.index.html.$$"
+cat > "$tmp" <<HTML
+<!DOCTYPE html>
+<html lang="de">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>mediupdatexml.oddb.org — Downloads</title>
+  <!-- open every link in a new tab -->
+  <base target="_blank">
+  <meta name="referrer" content="strict-origin-when-cross-origin">
+  <link rel="icon" type="image/svg+xml" href="logo.svg">
+  <style>
+    body { font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
+           max-width: 820px; margin: 2.5rem auto; padding: 0 1.2rem; color: #1a1a1a; line-height: 1.5; }
+    h1 { font-size: 1.6rem; margin-bottom: .2rem; }
+    h2 { font-size: 1.15rem; margin-top: 2rem; border-bottom: 1px solid #ddd; padding-bottom: .3rem; }
+    .sub { color: #666; margin-top: 0; }
+    ul { list-style: none; padding-left: 0; }
+    li { margin: .4rem 0; }
+    a { color: #0a58ca; text-decoration: none; }
+    a:hover { text-decoration: underline; }
+    .desc { color: #666; font-size: .9rem; }
+    code { background: #f4f4f4; padding: .1rem .3rem; border-radius: 3px; }
+    .stats { display: flex; gap: 1.5rem; margin: 1rem 0; flex-wrap: wrap; }
+    .stat { background: #f4f7fb; border: 1px solid #dce4ef; border-radius: 8px; padding: .8rem 1.2rem; }
+    .stat .n { font-size: 1.5rem; font-weight: 600; color: #0a58ca; }
+    .stat .l { color: #555; font-size: .85rem; }
+    footer { margin-top: 3rem; color: #888; font-size: .85rem; }
+    ul.firms { columns: 2; column-gap: 2rem; }
+    ul.firms li { margin: .25rem 0; break-inside: avoid; }
+    .topbar { display: flex; justify-content: space-between; align-items: flex-start; gap: 1rem; }
+    .topbar .title h1 { margin: 0 0 .2rem; }
+    .topbar .logo { width: 64px; height: 64px; flex: 0 0 auto; }
+  </style>
+</head>
+<body>
+  <header class="topbar">
+    <div class="title">
+      <h1>oddb2xml &amp; aips2sqlite Downloads</h1>
+      <p class="sub">Schweizer Arzneimitteldaten — täglich aktualisiert (01:00 Uhr). Stand: ${stand}</p>
+    </div>
+    <a href="mailto:zdavatz@ywesee.com" title="Fragen? zdavatz at ywesee dot com"><img class="logo" src="logo.svg" alt="oddb2xml Logo" width="64" height="64"></a>
+  </header>
+
+  <div class="stats">
+    <div class="stat"><div class="n">$(group "$pharma")</div><div class="l">Medikamente (PHARMA)</div></div>
+    <div class="stat"><div class="n"><a href="https://id.gs1.ch/01/07612345000961">$(group "$nonpharma")</a></div><div class="l">Firstbase-Produkte (NONPHARMA)</div></div>
+    <div class="stat"><div class="n"><a href="default/oddb_article.xml">$(group "$total")</a></div><div class="l">Artikel total (<a href="default/oddb_article.xml"><code>oddb_article.xml</code></a>)</div></div>
+  </div>
+
+  <h2>oddb2xml — Artikel-/Produkt-Feeds (<code>-b</code> firstbase)</h2>
+  <ul>
+    <li><a href="default/">default/</a> <span class="desc">— ohne Preisaufschlag</span></li>
+    <li><a href="45/">45/</a> <span class="desc">— Wiederverkaufspreis +45&nbsp;%</span></li>
+    <li><a href="50/">50/</a> <span class="desc">— Wiederverkaufspreis +50&nbsp;%</span></li>
+    <li><a href="55/">55/</a> <span class="desc">— Wiederverkaufspreis +55&nbsp;%</span></li>
+  </ul>
+  <p class="desc">Jedes Verzeichnis enthält die gleichen Dateien (Direktlinks zum <code>default/</code>-Feed):
+  <a href="default/oddb_article.xml">oddb_article.xml</a>,
+  <a href="default/oddb_product.xml">oddb_product.xml</a>,
+  <a href="default/oddb_calc.xml">oddb_calc.xml</a>,
+  <a href="default/oddb_interaction.xml">oddb_interaction.xml</a>,
+  <a href="default/oddb_limitation.xml">oddb_limitation.xml</a>,
+  <a href="default/oddb_substance.xml">oddb_substance.xml</a>,
+  <a href="default/oddb_code.xml">oddb_code.xml</a> sowie
+  <a href="default/oddb2xml.zip">oddb2xml.zip</a> (alle Dateien gepackt).</p>
+
+  <h2>aips2sqlite — Fachinformationen &amp; AmiKo-Datenbanken</h2>
+  <ul>
+    <li><a href="/aips2sqlite/fis/">fis/</a> <span class="desc">— Fachinformationen als XML/HTML (DE/FR/IT)</span></li>
+    <li><a href="/aips2sqlite/amiko_db_full_idx_de.db">amiko_db_full_idx_de.db</a> <span class="desc">— AmiKo-Datenbank Deutsch</span></li>
+    <li><a href="/aips2sqlite/amiko_db_full_idx_fr.db">amiko_db_full_idx_fr.db</a> <span class="desc">— AmiKo-Datenbank Französisch</span></li>
+    <li><a href="/aips2sqlite/oddb2xml_swissmedic_sequences.csv">oddb2xml_swissmedic_sequences.csv</a> <span class="desc">— Swissmedic-Sequenzen</span></li>
+    <li><a href="/aips2sqlite/atc_codes_used_set.txt">atc_codes_used_set.txt</a> <span class="desc">— verwendete ATC-Codes</span></li>
+    <li><a href="/aips2sqlite/">/aips2sqlite/</a> <span class="desc">— gesamtes Verzeichnis durchsuchen</span></li>
+  </ul>
+
+  <h2>MediUpdate XML bei HIN</h2>
+  <ul>
+    <li><a href="https://www.hin.ch/de/services/mediupdate-xml.cfm">www.hin.ch/de/services/mediupdate-xml.cfm</a></li>
+  </ul>
+
+  <h2>Softwarehäuser, die oddb2xml einsetzen</h2>
+  <ul class="firms">
+    <li><a href="https://www.advancedconcepts.ch/">Advanced Concepts AG</a></li>
+    <li><a href="https://www.bluecare.ch/">Bluecare AG</a></li>
+    <li><a href="https://corona.ch/">Corona Informatik AG</a></li>
+    <li><a href="https://www.derma2go.com/de/">derma2go AG</a></li>
+    <li><a href="https://www.diagnosia.com/">Diagnosia Internetservices GmbH</a></li>
+    <li><a href="https://elexis.ch/glp/index.html">Elexis</a></li>
+    <li><a href="https://www.emedswiss.ch/">emedSwiss SA</a></li>
+    <li><a href="https://gartenmann.ch/">Gartenmann Software AG</a></li>
+    <li><a href="https://hexabit.ch/">Hexabit GmbH</a></li>
+    <li><a href="https://www.hausarztmedizin.uzh.ch/de.html">Institut für Hausarztmedizin</a></li>
+    <li><a href="https://www.itw-informatik.ch/de/">ITW INFORMATIK AG</a></li>
+    <li><a href="https://www.lama-media.com/">Lama Media</a></li>
+    <li><a href="https://medab.org/country/ch">MEDAB</a></li>
+    <li><a href="https://www.pharmedsolutions.ch/">Pharmed Solutions GmbH</a></li>
+    <li><a href="https://praxinova.ch/">Praxinova AG</a></li>
+    <li><a href="https://seantis.ch/">seantis gmbh</a></li>
+    <li><a href="https://www.geteyesoft.ch/">Siplus SA – Eyesoft</a></li>
+    <li><a href="https://swiss-mr.ch/">SMR – Swiss Medical Record GmbH</a></li>
+    <li><a href="https://triboni.com/site/">Triboni AG</a></li>
+    <li><a href="https://www.vitabyte.ch/">Vitabyte AG</a></li>
+    <li><a href="https://zollsoft.de/">zollsoft GmbH</a></li>
+  </ul>
+
+  ${stats_fragment}
+
+  <footer>
+    Fragen: <a href="mailto:zdavatz@ywesee.com">zdavatz at ywesee dot com</a> &middot; Tel: 043 540 05 50
+  </footer>
+</body>
+</html>
+HTML
+chmod 644 "$tmp"
+mv -f "$tmp" "${DOCROOT%/}/index.html"
+
+echo "Wrote ${DOCROOT%/}/index.html  (PHARMA=$pharma NONPHARMA=$nonpharma)"

data/scripts/run_oddb2xml.sh

@@ -38,9 +38,35 @@ BUILD_DIR="${BUILD_DIR:-${OUT_DIR%/}-build}"
 INCREMENTS="${INCREMENTS:-45 50 55}"
 ODDB2XML_BIN="${ODDB2XML_BIN:-oddb2xml}"
 TRANSFER_CMD="${TRANSFER_CMD:-$SCRIPT_DIR/transfer.sh}"
+# Transient upstream download failures (e.g. Swissmedic resetting the
+# connection, Errno::ECONNRESET) used to abort the whole nightly run under
+# `set -e`. Retry the oddb2xml build a few times before giving up.
+ODDB2XML_RETRIES="${ODDB2XML_RETRIES:-3}"
+ODDB2XML_RETRY_DELAY="${ODDB2XML_RETRY_DELAY:-120}"
 
 log() { printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*"; }
 
+# run_with_retry <description> -- <command...>
+# Retry a flaky command up to ODDB2XML_RETRIES times, sleeping
+# ODDB2XML_RETRY_DELAY seconds between attempts. Running the command as the
+# `until` condition keeps it exempt from `set -e`, so a failed attempt retries
+# instead of killing the script; the final failure is propagated via return.
+run_with_retry() {
+  local desc="$1"; shift
+  [[ "${1:-}" == "--" ]] && shift
+  local attempt=1 rc=0
+  until "$@"; do
+    rc=$?
+    if [[ $attempt -ge $ODDB2XML_RETRIES ]]; then
+      log "ERROR: $desc failed after $attempt attempts (last exit $rc)"
+      return $rc
+    fi
+    log "WARNING: $desc failed (exit $rc), attempt $attempt/$ODDB2XML_RETRIES; retrying in ${ODDB2XML_RETRY_DELAY}s"
+    sleep "$ODDB2XML_RETRY_DELAY"
+    attempt=$((attempt + 1))
+  done
+}
+
 # 1. Install / update the published gem unless told otherwise.
 if [[ "${SKIP_GEM_INSTALL:-0}" != "1" ]]; then
   log "Installing oddb2xml gem"
@@ -69,7 +95,10 @@ build_one() {
 
   log "Building increment '${inc:-none}' -> $dest"
   rm -f oddb*.zip
-  "$ODDB2XML_BIN" "${dl_opt[@]}" -b "${inc_opt[@]}" -c zip
+  # On a retry the first build re-downloads from scratch (dl_opt empty), which
+  # also clears any partial download left by the failed attempt.
+  run_with_retry "oddb2xml build '${inc:-none}'" -- \
+    "$ODDB2XML_BIN" "${dl_opt[@]}" -b "${inc_opt[@]}" -c zip
 
   shopt -s nullglob
   local zips=(oddb*.zip)
@@ -89,6 +118,14 @@ for inc in $INCREMENTS; do
 done
 build_one "" "default"           # final run with no increment
 
+# 2b. Refresh the download landing page with the live PHARMA/NONPHARMA counts
+#     (PHARMA from default/oddb_article.xml, NONPHARMA from the GS1 firstbase CSV).
+if [[ -x "$SCRIPT_DIR/generate_index_html.sh" ]]; then
+  log "Refreshing landing page index.html"
+  "$SCRIPT_DIR/generate_index_html.sh" "$OUT_DIR" "$BUILD_DIR/downloads/firstbase.csv" || \
+    log "WARNING: could not regenerate index.html"
+fi
+
 # 3. Optional hand-off to the transfer step (scripts/transfer.sh).
 if [[ "${RUN_TRANSFER:-0}" == "1" ]]; then
   log "Running transfer: $TRANSFER_CMD"

data/scripts/setup_mediupdatexml_web.sh

@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+#
+# Provision Apache to serve the oddb2xml output at https://mediupdatexml.oddb.org
+# (Let's Encrypt HTTPS + browsable downloads, like pillbox.oddb.org).
+#
+# Serves two areas on one domain:
+#   /              -> $DOCROOT          (oddb2xml feeds: default/ 45/ 50/ 55/)
+#   /aips2sqlite/  -> $AIPS_OUT         (FI XMLs, AmiKo .db, sequences CSV)
+# plus a curated landing page ($DOCROOT/index.html) linking both.
+#
+# Run with:  sudo scripts/setup_mediupdatexml_web.sh
+#
+# Idempotent: safe to re-run. HTTPS is issued/renewed automatically once the
+# domain's DNS points at this host.
+
+set -euo pipefail
+
+DOMAIN="mediupdatexml.oddb.org"
+DOCROOT="/home/zdavatz/oddb2xml"
+AIPS_OUT="/home/zdavatz/software/aips2sqlite/jars/output"
+EMAIL="zdavatz@gmail.com"
+SITE="/etc/apache2/sites-available/${DOMAIN}.conf"
+
+if [[ $EUID -ne 0 ]]; then
+  echo "Please run as root:  sudo $0" >&2
+  exit 1
+fi
+
+echo "==> Installing apache2 + certbot"
+export DEBIAN_FRONTEND=noninteractive
+apt-get update -qq
+apt-get install -y apache2 certbot python3-certbot-apache
+
+echo "==> Enabling required Apache modules"
+a2enmod autoindex headers >/dev/null
+
+echo "==> Allowing www-data to traverse /home/zdavatz (711 = no listing, path access only)"
+chmod 711 /home/zdavatz
+
+echo "==> Writing vhost $SITE"
+cat > "$SITE" <<EOF
+<VirtualHost *:80>
+    ServerName ${DOMAIN}
+    DocumentRoot ${DOCROOT}
+
+    <Directory ${DOCROOT}>
+        Options +Indexes +FollowSymLinks
+        IndexOptions FancyIndexing HTMLTable NameWidth=* SuppressDescription FoldersFirst
+        Require all granted
+        AllowOverride None
+    </Directory>
+
+    ErrorLog  \${APACHE_LOG_DIR}/${DOMAIN}_error.log
+    CustomLog \${APACHE_LOG_DIR}/${DOMAIN}_access.log combined
+</VirtualHost>
+EOF
+
+echo "==> Writing aips2sqlite download alias (/aips2sqlite -> jars/output)"
+cat > /etc/apache2/conf-available/mediupdatexml-aips.conf <<EOF
+# aips2sqlite output (FI XMLs, AmiKo .db, swissmedic sequences CSV)
+# Served on both the HTTP and HTTPS vhosts of ${DOMAIN}.
+Alias /aips2sqlite ${AIPS_OUT}
+<Directory ${AIPS_OUT}>
+    Options +Indexes +FollowSymLinks
+    IndexOptions FancyIndexing HTMLTable NameWidth=* SuppressDescription FoldersFirst
+    Require all granted
+    AllowOverride None
+</Directory>
+EOF
+a2enconf mediupdatexml-aips >/dev/null
+
+# Curated landing page (with live PHARMA/NONPHARMA counts). Lives in the OUT_DIR
+# root, which run_oddb2xml.sh never touches (it only rebuilds the per-increment
+# subdirs), so it survives rebuilds — and run_oddb2xml.sh refreshes the counts
+# after every build via the same generator.
+echo "==> Writing landing page ${DOCROOT}/index.html"
+"$(dirname "$0")/generate_index_html.sh" "${DOCROOT}"
+
+echo "==> Enabling site, disabling default"
+a2ensite "${DOMAIN}.conf" >/dev/null
+a2dissite 000-default.conf >/dev/null 2>&1 || true
+
+echo "==> Testing config and reloading Apache"
+apache2ctl configtest
+systemctl reload apache2
+systemctl enable apache2 >/dev/null 2>&1 || true
+
+echo
+echo "==> HTTP is live: http://${DOMAIN}/  (serving ${DOCROOT})"
+echo
+
+# --- HTTPS via Let's Encrypt -------------------------------------------------
+# Check a public resolver (Cloudflare DoH), not the local one, which may still
+# hold a negative cache. Let's Encrypt validates from its own resolvers anyway.
+resolved=$(curl -s --max-time 8 \
+  "https://1.1.1.1/dns-query?name=${DOMAIN}&type=A" \
+  -H 'accept: application/dns-json' \
+  | python3 -c "import sys,json; a=json.load(sys.stdin).get('Answer',[]); print(a[0]['data'] if a else '')" 2>/dev/null || true)
+if [[ -z "$resolved" ]]; then
+  echo "!! ${DOMAIN} does not resolve in public DNS yet."
+  echo "   Once the A/AAAA records propagate, run:"
+  echo "     sudo certbot --apache -d ${DOMAIN} --redirect -m ${EMAIL} --agree-tos -n"
+  exit 0
+fi
+
+echo "==> ${DOMAIN} resolves to ${resolved} — requesting Let's Encrypt cert"
+certbot --apache -d "${DOMAIN}" --redirect -m "${EMAIL}" --agree-tos -n
+echo
+echo "==> Done. HTTPS is live: https://${DOMAIN}/"
+echo "    Certbot installed a systemd timer for auto-renewal."