smarter_csv 1.18.0 → 1.18.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3335e39a1c0792f01df9e95401c7f3885c49a0d64eeb9c76e5c20e25d01a62f5
-  data.tar.gz: e43f00228777b56fc1ee0814a74acaa6a23c51fe8da6f64e42ad92fe1b54002f
+  metadata.gz: 8b820fa45fcde042d9a96230a53d4065b104db91e50d48629b9d43cb099a6dca
+  data.tar.gz: 8bff7d0845fe3b6c3eecc71c58065364809fafb87bc25f238b53c35cf5b49519
 SHA512:
-  metadata.gz: 2abcd136f30d284c3c27cbd2b6c9782aec4235ec62cc27ea7620380ae9efc889f9f1c05c10ad957d6e7c84d65be75d35d96d4ac59ba5780dc5e65e0151c661e6
-  data.tar.gz: 6c61062c08d0a89dea2c91a7faafd88f6d09c88015ad8c7f4facb2eb474b44b18e0c5eaffc36fa5e2885f0b2c02487497f2460d66ce2433f62c09bf42d92a4ce
+  metadata.gz: c8b4a0d5cb0620b542617c8426eb3004dbb7a89602b38174280636194c634d1ee2e9951eac727a1972761d5434b66bb1f1bf9c90ec2659790405cc8c34159062
+  data.tar.gz: f6e042464354f8714aa5504eb04c16f137806766057bc22a6db0677d65f49eb4d57f5d54d1378f106fa0899c61408c9cb890d5e20d20f1802a1225338a7b9ba8

data/CHANGELOG.md

@@ -4,6 +4,29 @@
 > [!TIP]
 > **Upgrading?** The [SmarterCSV Upgrade Wizard](https://tilo.github.io/smarter_csv/upgrade_wizard.html) walks you through what (if anything) you need to change for your specific version. Most steps do not require any changes.
 
+## 1.18.1 (2026-06-30)
+
+### Bug Fixes
+
+  - **Portable builds by default — fixes the "Illegal instruction" crash on heterogeneous CPUs ([#343](https://github.com/tilo/smarter_csv/issues/343)).**
+
+  Since 1.14.3 the C extension was compiled with `-march=native` on every platform except Apple Silicon, baking-in the build host's CPU instructions (e.g. AVX-512).
+  A binary built on one machine then could encounter `Illegal instruction` when run on a CPU lacking those instructions — common when the build host differs from the run host (CI/build servers, Docker images, mixed-hardware fleets).
+
+  The C extension is now built **portable** by default (no host-specific instructions). Thanks to [@paholg](https://github.com/paholg) for the report.
+
+### New Features
+
+  - **`SMARTER_CSV_PERFORMANCE` build option** (`portable` default, `tuned`, or `max`)
+
+  | Level                | Flags added                               | Portable?                        | Use when                              |
+  |----------------------|-------------------------------------------|----------------------------------|---------------------------------------|
+  | `portable` (default) | none                                      | Yes, any CPU of the arch         | Build host may differ from run host   |
+  | `tuned`              | `-mtune=native`                           | Yes, instruction scheduling only | Build and run hosts share a microarch |
+  | `max`                | `-march=native`, or `-mcpu=native` on ARM | No, host instruction optimization| Build host and run host are the same  |
+
+  See the [Introduction](docs/_introduction.md#build-time-performance-tuning-smarter_csv_performance) for details.
+
 ## 1.18.0 (2026-06-17)
 
 This release is focused on both performance and the introduction of automatic conversion of decimals to big_decimal or float, preserving the precision, and also supporting scientific notation.

data/README.md

@@ -297,6 +297,27 @@ Or install it yourself as:
     $ gem install smarter_csv
 ```
 
+### CPU Optimization (`SMARTER_CSV_PERFORMANCE`)
+
+The C extension is compiled when the gem is installed. By default it is built **portable**: it uses no CPU-specific instructions, so a binary built on one machine runs on any other CPU of the same architecture. Set `SMARTER_CSV_PERFORMANCE` at install time to trade portability for speed:
+
+
+| Level                | Flags added                               | Portable?                        | Use when                              |
+|----------------------|-------------------------------------------|----------------------------------|---------------------------------------|
+| `portable` (default) | none                                      | Yes, any CPU of the arch         | Build host may differ from run host   |
+| `tuned`              | `-mtune=native`                           | Yes, instruction scheduling only | Build and run hosts share a microarch |
+| `max`                | `-march=native`, or `-mcpu=native` on ARM | No, host instruction optimization| Build host and run host are the same  |
+
+`max` enables host-specific instructions, so a binary built with it can crash with `Illegal instruction` if it later runs on a CPU that lacks them (for example, built on an AVX-512 machine and run on one without). `tuned` only changes instruction scheduling, never the instruction set, so it stays portable. Every flag is probed against your compiler at build time and skipped if unsupported, so an unavailable flag never breaks the build.
+
+```bash
+SMARTER_CSV_PERFORMANCE=tuned gem install smarter_csv   # portable, tuned for this machine's microarchitecture
+SMARTER_CSV_PERFORMANCE=max   gem install smarter_csv   # fastest, NOT portable — only when you build on the machine you run on
+SMARTER_CSV_PERFORMANCE=tuned bundle install            # same, under Bundler
+```
+
+For a fixed baseline instead of `native` (e.g. a portable-but-newer instruction set), pass flags directly via `CFLAGS`, which the build also honors: `CFLAGS="-march=x86-64-v2" gem install smarter_csv`.
+
 ## Documentation
 
   * [Introduction](docs/_introduction.md)

data/UPGRADING.md

@@ -2,7 +2,7 @@
 
 > [!TIP]
 > Prefer the interactive [Upgrade Wizard](https://tilo.github.io/smarter_csv/upgrade_wizard.html) for a guided walk-through with Yes/No questions.  
-> This document is auto-generated from `CHANGELOG.md` and `docs/upgrade_path.json` by `bin/gen-upgrading-md`.
+> This document is auto-generated from `CHANGELOG.md` and `docs/upgrade_path.json` by `bin/generate-upgrading-md`.
 
 ## How to use this guide
 
@@ -12,25 +12,40 @@
 
 Prefer an interactive walk-through? The [Upgrade Wizard](https://tilo.github.io/smarter_csv/upgrade_wizard.html) asks one question at a time and only shows the migration steps that apply to your code.
 
-**Latest release:** `1.17.3` (in the `1.17.x` series).
+**Latest release:** `1.18.1` (in the `1.18.x` series).
 
 ---
 
-## 1.17.x — latest series
+## 1.18.x — latest series
 
 **Versions in this series:**  
-[1.17.0, 1.17.1, 1.17.2, 1.17.3]
+[1.18.0, 1.18.1]
 
-**Latest release:** `1.17.3`
+> ⚠️ **In-series notes** worth checking:
+> - **1.18.0:** This version is particularly interesting if you have geolocation, scientific, or high-precision data.
+
+**Latest release:** `1.18.1`
 
 Update your Gemfile to:
 
 ```ruby
-gem 'smarter_csv', '~> 1.17.0'
+gem 'smarter_csv', '~> 1.18.0'
 ```
 
 Then run `bundle update smarter_csv`.
 
+## Series 1.17 → 1.18
+
+**Coming from any 1.17 version:**
+[1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4]
+
+**Upgrading to 1.18.x** (latest: `1.18.1`):
+
+- **If** you build and run the gem on the same machine (or a fleet of identical CPUs) and want the previous native-optimized build for maximum speed:
+  → set `SMARTER_CSV_PERFORMANCE=max` (or `tuned`) at install time — 1.18.1 builds <strong>portable</strong> by default (no host-specific CPU instructions) to fix an `Illegal instruction` crash when a binary built on one CPU is run on another (<a href="https://github.com/tilo/smarter_csv/issues/343">#343</a>). The default is safe everywhere; the env var opts back into host optimization.
+
+---
+
 ## Series 1.16 → 1.17
 
 **Coming from any 1.16 version:**  
@@ -40,7 +55,7 @@ Then run `bundle update smarter_csv`.
 > - **1.16.1:** **Fibers:** `SmarterCSV.errors` uses `Thread.current` for storage, which is **shared across all fibers running in the same thread**. If you process CSV files concurrently in fibers (e.g. with `Async`, `Falcon`, or manual `Fiber` scheduling), `SmarterCSV.errors` may return stale or wrong results. **Use `SmarterCSV::Reader` directly** — errors are scoped to the reader instance and are always correct regardless of fiber context.
 > - **1.16.2:** If your code references auto-generated keys for blank headers, update those to use the absolute column position.
 
-**Upgrading to 1.17.x** (latest: `1.17.3`): you can upgrade all the way — no code changes needed.
+**Upgrading to 1.17.x** (latest: `1.17.4`): you can upgrade all the way — no code changes needed.
 
 ---
 

data/docs/_introduction.md

@@ -75,6 +75,28 @@ SmarterCSV was created to solve exactly these problems: nightly imports of large
 * **CSV writing:**
   `SmarterCSV.generate` writes arrays of hashes to CSV, with support for header renaming and value converters on output. See [The Basic Write API](./basic_write_api.md).
 
+## Build-Time Performance Tuning (`SMARTER_CSV_PERFORMANCE`)
+
+The C extension is compiled when the gem is installed. By default it is built **portable**: it uses no CPU-specific instructions, so a binary compiled on one machine runs on any other CPU of the same architecture. This matters whenever the machine that builds the gem differs from the machine that runs it — a CI or build server, a Docker image moved between hosts, or a mixed-hardware fleet. A build that bakes in instructions the run host lacks (such as AVX-512) would otherwise crash with `Illegal instruction`.
+
+Set `SMARTER_CSV_PERFORMANCE` at install time to trade portability for speed:
+
+| Level                | Flags added                               | Portable?                        | Use when                              |
+|----------------------|-------------------------------------------|----------------------------------|---------------------------------------|
+| `portable` (default) | none                                      | Yes, any CPU of the arch         | Build host may differ from run host   |
+| `tuned`              | `-mtune=native`                           | Yes, instruction scheduling only | Build and run hosts share a microarch |
+| `max`                | `-march=native`, or `-mcpu=native` on ARM | No, host instruction optimization| Build host and run host are the same  |
+
+`tuned` only changes instruction scheduling, never the instruction set, so it stays portable — and it pays off when the build and run hosts share a microarchitecture (the same chip, or a fleet of identical instances). `max` enables host-specific instructions and is the fastest, but a binary built with it can crash on a different CPU. Every flag is probed against your compiler at build time and skipped if unsupported, so an unavailable flag never breaks the build.
+
+```bash
+SMARTER_CSV_PERFORMANCE=tuned gem install smarter_csv   # portable, tuned for this machine's microarchitecture
+SMARTER_CSV_PERFORMANCE=max   gem install smarter_csv   # fastest, NOT portable — only when you build on the machine you run on
+SMARTER_CSV_PERFORMANCE=tuned bundle install            # same, under Bundler
+```
+
+For a fixed baseline instead of `native` (e.g. a portable-but-newer instruction set), pass flags directly via `CFLAGS`, which the build also honors: `CFLAGS="-march=x86-64-v2" gem install smarter_csv`.
+
 ---------------
 
 NEXT: [Migrating from Ruby CSV](./migrating_from_csv.md) | UP: [README](../README.md)

data/docs/upgrade_path.json

@@ -1,6 +1,6 @@
 {
-  "latest": "1.17",
-  "latest_release": "1.17.2",
+  "latest": "1.18",
+  "latest_release": "1.18.0",
   "path": {
     "1.0": {
       "to": "1.1",
@@ -170,6 +170,17 @@
       "to": "1.17",
       "latest_release": "1.16.6",
       "actions": []
+    },
+    "1.17": {
+      "to": "1.18",
+      "latest_release": "1.17.4",
+      "actions": [
+        {
+          "if": "you build and run the gem on the same machine (or a fleet of identical CPUs) and want the previous native-optimized build for maximum speed",
+          "then": "set <code>SMARTER_CSV_PERFORMANCE=max</code> (or <code>tuned</code>) at install time &mdash; 1.18.1 builds <strong>portable</strong> by default (no host-specific CPU instructions) to fix an <code>Illegal instruction</code> crash when a binary built on one CPU is run on another (<a href=\"https://github.com/tilo/smarter_csv/issues/343\">#343</a>). The default is safe everywhere; the env var opts back into host optimization."
+        }
+      ],
+      "note": "<strong>Bonus:</strong> 1.18 automatically converts scientific notation (e.g. <code>1e3</code>, <code>6.022e23</code>) to numbers, and preserves full precision on long decimals by returning a <code>BigDecimal</code> when a value carries more than 16 significant digits (<code>decimal_precision: :auto</code>, the default). Nothing to change &mdash; this just works."
     }
   }
 }

data/docs/upgrade_wizard.html

@@ -94,6 +94,15 @@ input[type="text"] {
   color: var(--green);
   font-weight: 600;
 }
+.benefit {
+  background: var(--green-soft);
+  border: 1px solid #b6dab6;
+  border-radius: 8px;
+  padding: 1em 1.25em;
+  margin-bottom: 1em;
+  color: var(--fg);
+}
+.benefit strong { color: var(--green); }
 .check {
   padding: 0.85em 0;
   border-top: 1px solid var(--border);
@@ -229,7 +238,7 @@ function renderHop(series, originalVersion) {
   if (hop.actions.length === 0) {
     body = `<div class="hop dropin">
       You can upgrade directly to version ${targetRelease}. No changes needed.
-    </div>`;
+    </div>${hop.note ? `<div class="benefit">${hop.note}</div>` : ""}`;
   } else {
     body = `<div class="hop">
       ${hop.actions.map((a, i) => `
@@ -324,6 +333,7 @@ function renderHop(series, originalVersion) {
       from:   series,
       to:     hop.to,
       dropIn: hop.actions.length === 0,
+      note:   hop.note || null,
       matched
     });
 
@@ -377,7 +387,8 @@ function renderSummary() {
     const targetRelease = latestReleaseFor(d.to);
     const heading = `<p class="summary-hop-heading"><strong>${d.from}.x &rarr; ${targetRelease}</strong></p>`;
     if (d.dropIn) {
-      return `<div class="summary-hop">${heading}<p class="muted">No code changes needed for this step.</p></div>`;
+      const noteHTML = d.note ? `<p>${d.note}</p>` : "";
+      return `<div class="summary-hop">${heading}<p class="muted">No code changes needed for this step.</p>${noteHTML}</div>`;
     }
     if (d.matched.length === 0) {
       return `<div class="summary-hop">${heading}<p class="muted">None of the conditions in this step applied to your code.</p></div>`;

data/ext/smarter_csv/cpu_flags.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module SmarterCSV
+  # Pure (mkmf-free) selection of CPU-optimization flags from the
+  # SMARTER_CSV_PERFORMANCE environment variable. Kept separate from extconf.rb
+  # so the logic can be unit-tested without invoking a compiler.
+  #
+  # Levels:
+  #   portable - no host-specific flags. The binary runs on any CPU of the same
+  #              architecture. The safe default: a binary built here will not
+  #              crash with "Illegal instruction" on an older/different CPU.
+  #   tuned    - -mtune=native: tunes instruction scheduling for the build host's
+  #              microarchitecture WITHOUT changing the instruction set, so the
+  #              binary stays portable. A real win when build and run hosts share
+  #              a microarchitecture (same chip or a homogeneous fleet).
+  #   max      - host-specific instructions: -march=native, or -mcpu=native on
+  #              ARM/Clang where -march=native is rejected. Fastest, but NOT
+  #              portable -- may crash on a CPU lacking the build host's
+  #              instructions. Use only when build host and run host match.
+  #
+  # `accepts` is a predicate (in the real build, a wrapper over mkmf's
+  # try_compile) returning true when the compiler accepts a given flag; each
+  # candidate is probed so an unsupported flag is skipped rather than breaking
+  # the build.
+  module CpuFlags
+    LEVELS = %w[portable tuned max].freeze
+
+    # Candidate flags per level, in preference order. The first one the compiler
+    # accepts wins. `max` degrades march -> mcpu -> mtune; tuned only ever
+    # considers -mtune=native (never an instruction-set flag).
+    CANDIDATES = {
+      'portable' => [].freeze,
+      'tuned' => ['-mtune=native'].freeze,
+      'max' => ['-march=native', '-mcpu=native', '-mtune=native'].freeze,
+    }.freeze
+
+    # Returns a Hash: { level: String, flags: Array<String>, warning: String|nil }.
+    def self.select(raw_level, accepts:)
+      level, warning = normalize(raw_level)
+      chosen = CANDIDATES[level].find { |flag| accepts.call(flag) }
+      { level: level, flags: chosen ? [chosen] : [], warning: warning }
+    end
+
+    # Normalizes the env value to a known level. Unknown values fall back to
+    # 'portable' (a typo can then only ever be slower, never non-portable) and
+    # return a warning naming the bad value and the fallback.
+    def self.normalize(raw_level)
+      value = raw_level.to_s.strip.downcase
+      return ['portable', nil] if value.empty?
+      return [value, nil] if LEVELS.include?(value)
+
+      ['portable', "SMARTER_CSV_PERFORMANCE=#{raw_level.inspect} is not one of #{LEVELS.join('|')}; using 'portable'."]
+    end
+  end
+end

data/ext/smarter_csv/extconf.rb

@@ -2,6 +2,7 @@
 
 require 'mkmf'
 require "rbconfig"
+require_relative 'cpu_flags'
 
 if RbConfig::MAKEFILE_CONFIG["CFLAGS"].include?("-g -O3")
   fixed_CFLAGS = RbConfig::MAKEFILE_CONFIG["CFLAGS"].sub("-g -O3", "-O3 $(cflags)")
@@ -9,11 +10,31 @@ if RbConfig::MAKEFILE_CONFIG["CFLAGS"].include?("-g -O3")
   RbConfig::MAKEFILE_CONFIG["CFLAGS"] = fixed_CFLAGS
 end
 
+# Probe whether the compiler accepts a flag by compiling a trivial program with
+# it. Lets us skip flags the toolchain rejects (e.g. -march=native on Clang/ARM,
+# or GCC-only flags on MSVC) instead of breaking the build. Replaces the old
+# RUBY_PLATFORM string guesses: ask the actual compiler, don't infer from the OS.
+def compiler_accepts?(flag)
+  try_compile("int main(void){return 0;}", flag)
+end
+
 optflags = "-O3 -flto -fomit-frame-pointer -DNDEBUG".dup
-optflags << " -march=native" unless RUBY_PLATFORM.start_with?("arm64-darwin")
+
+# CPU optimization level, set via SMARTER_CSV_PERFORMANCE (default: portable).
+# See cpu_flags.rb for the full description of each level.
+#
+#   portable (default) - no host-specific flags; runs on any CPU of the same arch.
+#   tuned              - -mtune=native; host scheduling tuning, still portable.
+#   max                - host instruction set (-march/-mcpu native); fastest, but
+#                        NOT portable -- may crash on a CPU lacking those instructions.
+cpu = SmarterCSV::CpuFlags.select(ENV["SMARTER_CSV_PERFORMANCE"], accepts: method(:compiler_accepts?))
+warn(cpu[:warning]) if cpu[:warning]
+cpu[:flags].each { |flag| optflags << " #{flag}" }
+puts("SmarterCSV performance level: #{cpu[:level]} -- optflags: #{optflags}")
+
 # -fno-semantic-interposition: GCC/Clang only (not MSVC). Allows intra-library
 # calls to bypass the PLT on Linux and enables more aggressive LTO inlining.
-optflags << " -fno-semantic-interposition" unless RUBY_PLATFORM.include?("mswin")
+optflags << " -fno-semantic-interposition" if compiler_accepts?("-fno-semantic-interposition")
 
 append_cflags('-Wno-compound-token-split-by-macro')
 

data/lib/smarter_csv/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module SmarterCSV
-  VERSION = "1.18.0"
+  VERSION = "1.18.1"
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: smarter_csv
 version: !ruby/object:Gem::Version
-  version: 1.18.0
+  version: 1.18.1
 platform: ruby
 authors:
 - Tilo Sloboda
 bindir: bin
 cert_chain: []
-date: 2026-06-19 00:00:00.000000000 Z
+date: 2026-07-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bigdecimal
@@ -83,6 +83,7 @@ files:
 - docs/upgrade_wizard.html
 - docs/value_converters.md
 - docs/warnings.md
+- ext/smarter_csv/cpu_flags.rb
 - ext/smarter_csv/extconf.rb
 - ext/smarter_csv/smarter_csv.c
 - ext/smarter_csv/vendor/LICENSE-fast_float-MIT
@@ -140,7 +141,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.9
+rubygems_version: 4.0.15
 specification_version: 4
 summary: Fastest end-to-end CSV ingestion for Ruby with smart defaults and Rails-ready
   hash output