smarter_csv 1.17.4 → 1.18.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 140f9359c26f8903b9075faeb59e9c1fc4b5c4b9dd5fcef664e12bb53fe13073
-  data.tar.gz: 0e7be3195610bcddb77870744a24f0eee9431643c4c28fcbea12d4b8663bb2db
+  metadata.gz: 8b820fa45fcde042d9a96230a53d4065b104db91e50d48629b9d43cb099a6dca
+  data.tar.gz: 8bff7d0845fe3b6c3eecc71c58065364809fafb87bc25f238b53c35cf5b49519
 SHA512:
-  metadata.gz: 176daa024372ade1d6431e5e4fd5355175cd1ebf03e7a216d70b8ff4554eb259a023969783b53118eba6c3fbf747a08e286ed7daad455ebc5cf8d7b61475d3d1
-  data.tar.gz: 8b1ca7263e5a54fc642c8f76bfd56577f6056784910acdcdcd0f18c6222c8793280016e749f2e08a61485bf8255cf341250ac1250a8d3ff424f8b07b1edbd51b
+  metadata.gz: c8b4a0d5cb0620b542617c8426eb3004dbb7a89602b38174280636194c634d1ee2e9951eac727a1972761d5434b66bb1f1bf9c90ec2659790405cc8c34159062
+  data.tar.gz: f6e042464354f8714aa5504eb04c16f137806766057bc22a6db0677d65f49eb4d57f5d54d1378f106fa0899c61408c9cb890d5e20d20f1802a1225338a7b9ba8

data/CHANGELOG.md

@@ -4,6 +4,70 @@
 > [!TIP]
 > **Upgrading?** The [SmarterCSV Upgrade Wizard](https://tilo.github.io/smarter_csv/upgrade_wizard.html) walks you through what (if anything) you need to change for your specific version. Most steps do not require any changes.
 
+## 1.18.1 (2026-06-30)
+
+### Bug Fixes
+
+  - **Portable builds by default — fixes the "Illegal instruction" crash on heterogeneous CPUs ([#343](https://github.com/tilo/smarter_csv/issues/343)).**
+
+  Since 1.14.3 the C extension was compiled with `-march=native` on every platform except Apple Silicon, baking-in the build host's CPU instructions (e.g. AVX-512).
+  A binary built on one machine then could encounter `Illegal instruction` when run on a CPU lacking those instructions — common when the build host differs from the run host (CI/build servers, Docker images, mixed-hardware fleets).
+
+  The C extension is now built **portable** by default (no host-specific instructions). Thanks to [@paholg](https://github.com/paholg) for the report.
+
+### New Features
+
+  - **`SMARTER_CSV_PERFORMANCE` build option** (`portable` default, `tuned`, or `max`)
+
+  | Level                | Flags added                               | Portable?                        | Use when                              |
+  |----------------------|-------------------------------------------|----------------------------------|---------------------------------------|
+  | `portable` (default) | none                                      | Yes, any CPU of the arch         | Build host may differ from run host   |
+  | `tuned`              | `-mtune=native`                           | Yes, instruction scheduling only | Build and run hosts share a microarch |
+  | `max`                | `-march=native`, or `-mcpu=native` on ARM | No, host instruction optimization| Build host and run host are the same  |
+
+  See the [Introduction](docs/_introduction.md#build-time-performance-tuning-smarter_csv_performance) for details.
+
+## 1.18.0 (2026-06-17)
+
+This release is focused on both performance and the introduction of automatic conversion of decimals to big_decimal or float, preserving the precision, and also supporting scientific notation.
+
+⚠️ This version is particularly interesting if you have geolocation, scientific, or high-precision data.
+
+### New Features
+
+  - **`decimal_precision` option** (`:auto` default, or `:float` / `:bigdecimal`) — controls how decimal values are converted. `:auto` returns a `Float` unless the value carries more than 16 significant digits, in which case it returns a `BigDecimal` so no precision is lost; `:float` always returns `Float`; `:bigdecimal` always returns `BigDecimal`. Integers are unaffected (always `Integer`). Works identically on the C and Ruby paths. (Ruby's standard-library CSV has no high-precision option — its `:numeric`/`:float` converters use `Float()` and lose precision.)
+  - **Float** conversion on the C path now uses the fast **Eisel-Lemire** algorithm (fast_float, vendored) for mantissas up to 19 significant digits — correctly rounded, bit-for-bit identical to `String#to_f` — with a `strtod` fallback beyond that (more than 19 digits / extreme exponents). High-precision values that become `BigDecimal` under `:auto`/`:bigdecimal` are parsed by Ruby's `BigDecimal`.
+
+### Behavior Changes
+
+  - **Scientific notation now converts to a number** (e.g. `"1e3"`, `"1.5e-5"`, `"6.022e23"`). Previously the Ruby path left these as Strings and the C path was inconsistent.
+  - **The C and Ruby numeric-conversion paths are now aligned.** Bare-dot forms like `".5"` and `"3."` stay Strings on **both** paths (the shared grammar requires an integer part and, when a dot is present, a fraction digit). Previously the C path converted these and the Ruby path did not.
+  - With the default `decimal_precision: :auto`, decimal values carrying more than 16 significant digits are now returned as `BigDecimal` instead of `Float`. Pass `decimal_precision: :float` to keep the previous always-`Float` behavior.
+  - `bigdecimal` is now a runtime dependency (it is no longer a default gem on Ruby 3.4+).
+
+### Performance
+
+  The C-accelerated path is faster across the board, **up to ~1.5× on the right shapes** — numeric-heavy data and backslash-escaped quoted fields — and ~1.04–1.08× on typical files.
+
+  - Eisel-Lemire (Mushtak-Lemire) algorithm on the C path to convert decimals to `Float` or `BigDecimal`. Numeric-heavy data (many float/decimal columns) parses significantly faster.
+  - SIMD scanner for backslash-escaped quoted fields (C-path), using NEON (arm64) and SSE2 (x86-64) with a scalar fallback. Speeds up `quote_escaping: :backslash` parsing of long quoted fields.
+
+  | File                            | C-path                           | driver                |
+  |---------------------------------|----------------------------------|-----------------------|
+  | backslash_long_fields_60k       | 1.48× faster (0.1880s → 0.1273s) | SIMD quote/backslash scanner |
+  | sensor_data_50krows_50cols      | 1.40× faster (0.2763s → 0.1975s) | Eisel-Lemire numeric conversion |
+
+### Improvements
+
+  - Improved robustness of symbol-valued enum option processing.
+
+### Tests
+
+  - added parity tests for long quoted-field scanning across 16-byte boundaries, running on both the C and Ruby paths.
+  - added tests for string-to-symbol coercion of the enum options.
+
+
+
 ## 1.17.4 (2026-06-03)
 
 ### Bug Fix

data/README.md

@@ -15,6 +15,9 @@
 
   > See [**Ruby CSV Pitfalls**](docs/ruby_csv_pitfalls.md) for 10 ways `CSV.read` silently corrupts or loses data, and how SmarterCSV handles them.
 
+  > [!TIP]
+  > **No silent precision loss (new in 1.18.0).** For scientific data, GPS/geo coordinates, and financial figures — which routinely carry 16+ significant digits — Ruby's standard CSV converts with `Float()`, so a value like `1234567890.123456789` is silently rounded to `1234567890.1234567`. SmarterCSV's default `decimal_precision: :auto` returns a `BigDecimal` for values beyond 16 significant digits (and `Float` otherwise) — full precision, no data loss. Floats are decoded with the Eisel-Lemire algorithm: correctly rounded, bit-for-bit identical to `String#to_f`.
+
   Beyond raw speed, SmarterCSV is designed to provide a significantly more convenient and developer-friendly interface than traditional CSV libraries. Instead of returning raw arrays that require substantial post-processing, SmarterCSV produces Rails-ready hashes for each row, making the data immediately usable with ActiveRecord, Sidekiq pipelines, parallel processing, and JSON-based workflows such as S3.
 
   In a Rails app, warnings auto-route through `Rails.logger` and instrumentation hooks compose with `ActiveSupport::Notifications` — no setup required. Outside Rails, warnings fall back to `$stderr` and the same APIs work without any framework dependency.
@@ -89,6 +92,8 @@ rows = SmarterCSV.process('data.csv')
 data = SmarterCSV.parse(csv_string)
 ```
 
+Numeric conversion is also more accurate: where Ruby's `:numeric`/`:float` converters round high-precision decimals through `Float()`, SmarterCSV's default `decimal_precision: :auto` returns a `BigDecimal` past 16 significant digits, so no precision is lost (pass `decimal_precision: :float` for like-for-like `Float` output).
+
 * See [**Migrating from Ruby CSV**](docs/migrating_from_csv.md) for a full comparison of options, behavior differences, and a quick-reference table.
 
 ## Examples
@@ -292,6 +297,27 @@ Or install it yourself as:
     $ gem install smarter_csv
 ```
 
+### CPU Optimization (`SMARTER_CSV_PERFORMANCE`)
+
+The C extension is compiled when the gem is installed. By default it is built **portable**: it uses no CPU-specific instructions, so a binary built on one machine runs on any other CPU of the same architecture. Set `SMARTER_CSV_PERFORMANCE` at install time to trade portability for speed:
+
+
+| Level                | Flags added                               | Portable?                        | Use when                              |
+|----------------------|-------------------------------------------|----------------------------------|---------------------------------------|
+| `portable` (default) | none                                      | Yes, any CPU of the arch         | Build host may differ from run host   |
+| `tuned`              | `-mtune=native`                           | Yes, instruction scheduling only | Build and run hosts share a microarch |
+| `max`                | `-march=native`, or `-mcpu=native` on ARM | No, host instruction optimization| Build host and run host are the same  |
+
+`max` enables host-specific instructions, so a binary built with it can crash with `Illegal instruction` if it later runs on a CPU that lacks them (for example, built on an AVX-512 machine and run on one without). `tuned` only changes instruction scheduling, never the instruction set, so it stays portable. Every flag is probed against your compiler at build time and skipped if unsupported, so an unavailable flag never breaks the build.
+
+```bash
+SMARTER_CSV_PERFORMANCE=tuned gem install smarter_csv   # portable, tuned for this machine's microarchitecture
+SMARTER_CSV_PERFORMANCE=max   gem install smarter_csv   # fastest, NOT portable — only when you build on the machine you run on
+SMARTER_CSV_PERFORMANCE=tuned bundle install            # same, under Bundler
+```
+
+For a fixed baseline instead of `native` (e.g. a portable-but-newer instruction set), pass flags directly via `CFLAGS`, which the build also honors: `CFLAGS="-march=x86-64-v2" gem install smarter_csv`.
+
 ## Documentation
 
   * [Introduction](docs/_introduction.md)

data/UPGRADING.md

@@ -2,7 +2,7 @@
 
 > [!TIP]
 > Prefer the interactive [Upgrade Wizard](https://tilo.github.io/smarter_csv/upgrade_wizard.html) for a guided walk-through with Yes/No questions.  
-> This document is auto-generated from `CHANGELOG.md` and `docs/upgrade_path.json` by `bin/gen-upgrading-md`.
+> This document is auto-generated from `CHANGELOG.md` and `docs/upgrade_path.json` by `bin/generate-upgrading-md`.
 
 ## How to use this guide
 
@@ -12,25 +12,40 @@
 
 Prefer an interactive walk-through? The [Upgrade Wizard](https://tilo.github.io/smarter_csv/upgrade_wizard.html) asks one question at a time and only shows the migration steps that apply to your code.
 
-**Latest release:** `1.17.3` (in the `1.17.x` series).
+**Latest release:** `1.18.1` (in the `1.18.x` series).
 
 ---
 
-## 1.17.x — latest series
+## 1.18.x — latest series
 
 **Versions in this series:**  
-[1.17.0, 1.17.1, 1.17.2, 1.17.3]
+[1.18.0, 1.18.1]
 
-**Latest release:** `1.17.3`
+> ⚠️ **In-series notes** worth checking:
+> - **1.18.0:** This version is particularly interesting if you have geolocation, scientific, or high-precision data.
+
+**Latest release:** `1.18.1`
 
 Update your Gemfile to:
 
 ```ruby
-gem 'smarter_csv', '~> 1.17.0'
+gem 'smarter_csv', '~> 1.18.0'
 ```
 
 Then run `bundle update smarter_csv`.
 
+## Series 1.17 → 1.18
+
+**Coming from any 1.17 version:**
+[1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4]
+
+**Upgrading to 1.18.x** (latest: `1.18.1`):
+
+- **If** you build and run the gem on the same machine (or a fleet of identical CPUs) and want the previous native-optimized build for maximum speed:
+  → set `SMARTER_CSV_PERFORMANCE=max` (or `tuned`) at install time — 1.18.1 builds <strong>portable</strong> by default (no host-specific CPU instructions) to fix an `Illegal instruction` crash when a binary built on one CPU is run on another (<a href="https://github.com/tilo/smarter_csv/issues/343">#343</a>). The default is safe everywhere; the env var opts back into host optimization.
+
+---
+
 ## Series 1.16 → 1.17
 
 **Coming from any 1.16 version:**  
@@ -40,7 +55,7 @@ Then run `bundle update smarter_csv`.
 > - **1.16.1:** **Fibers:** `SmarterCSV.errors` uses `Thread.current` for storage, which is **shared across all fibers running in the same thread**. If you process CSV files concurrently in fibers (e.g. with `Async`, `Falcon`, or manual `Fiber` scheduling), `SmarterCSV.errors` may return stale or wrong results. **Use `SmarterCSV::Reader` directly** — errors are scoped to the reader instance and are always correct regardless of fiber context.
 > - **1.16.2:** If your code references auto-generated keys for blank headers, update those to use the absolute column position.
 
-**Upgrading to 1.17.x** (latest: `1.17.3`): you can upgrade all the way — no code changes needed.
+**Upgrading to 1.17.x** (latest: `1.17.4`): you can upgrade all the way — no code changes needed.
 
 ---
 

data/docs/_introduction.md

@@ -75,6 +75,28 @@ SmarterCSV was created to solve exactly these problems: nightly imports of large
 * **CSV writing:**
   `SmarterCSV.generate` writes arrays of hashes to CSV, with support for header renaming and value converters on output. See [The Basic Write API](./basic_write_api.md).
 
+## Build-Time Performance Tuning (`SMARTER_CSV_PERFORMANCE`)
+
+The C extension is compiled when the gem is installed. By default it is built **portable**: it uses no CPU-specific instructions, so a binary compiled on one machine runs on any other CPU of the same architecture. This matters whenever the machine that builds the gem differs from the machine that runs it — a CI or build server, a Docker image moved between hosts, or a mixed-hardware fleet. A build that bakes in instructions the run host lacks (such as AVX-512) would otherwise crash with `Illegal instruction`.
+
+Set `SMARTER_CSV_PERFORMANCE` at install time to trade portability for speed:
+
+| Level                | Flags added                               | Portable?                        | Use when                              |
+|----------------------|-------------------------------------------|----------------------------------|---------------------------------------|
+| `portable` (default) | none                                      | Yes, any CPU of the arch         | Build host may differ from run host   |
+| `tuned`              | `-mtune=native`                           | Yes, instruction scheduling only | Build and run hosts share a microarch |
+| `max`                | `-march=native`, or `-mcpu=native` on ARM | No, host instruction optimization| Build host and run host are the same  |
+
+`tuned` only changes instruction scheduling, never the instruction set, so it stays portable — and it pays off when the build and run hosts share a microarchitecture (the same chip, or a fleet of identical instances). `max` enables host-specific instructions and is the fastest, but a binary built with it can crash on a different CPU. Every flag is probed against your compiler at build time and skipped if unsupported, so an unavailable flag never breaks the build.
+
+```bash
+SMARTER_CSV_PERFORMANCE=tuned gem install smarter_csv   # portable, tuned for this machine's microarchitecture
+SMARTER_CSV_PERFORMANCE=max   gem install smarter_csv   # fastest, NOT portable — only when you build on the machine you run on
+SMARTER_CSV_PERFORMANCE=tuned bundle install            # same, under Bundler
+```
+
+For a fixed baseline instead of `native` (e.g. a portable-but-newer instruction set), pass flags directly via `CFLAGS`, which the build also honors: `CFLAGS="-march=x86-64-v2" gem install smarter_csv`.
+
 ---------------
 
 NEXT: [Migrating from Ruby CSV](./migrating_from_csv.md) | UP: [README](../README.md)

data/docs/data_transformations.md

@@ -156,6 +156,39 @@ data = SmarterCSV.process(file,
   convert_values_to_numeric: { only: [:quantity, :price] })
 ```
 
+Scientific notation (e.g. `"1.5e3"`, `"6.022e23"`) is recognized and converted too. Bare-dot forms like `".5"` and `"3."` are left as Strings (they are not valid numbers here). Integers and floats convert identically on the C-accelerated and pure-Ruby paths.
+
+---
+
+## `decimal_precision`
+
+**Default: `:auto`**
+
+Controls how decimal values (those with a `.` or an exponent) are converted. Integers are unaffected — they are always returned as `Integer`.
+
+| Value         | Result                                                                                  |
+|---------------|-----------------------------------------------------------------------------------------|
+| `:auto`       | `Float`, unless the value carries more than 16 significant digits — then `BigDecimal`.   |
+| `:float`      | Always `Float` (correctly rounded; matches `String#to_f`).                               |
+| `:bigdecimal` | Always `BigDecimal` (full precision).                                                    |
+
+```ruby
+# :auto (default) — keeps full precision only when needed
+SmarterCSV.process(file)
+# "3.14"                 => 3.14                              (Float)
+# "1234567890.123456789" => 0.1234567890123456789e10          (BigDecimal — >16 sig digits)
+
+# :float — always Float (faster, may lose precision on long decimals)
+SmarterCSV.process(file, decimal_precision: :float)
+# "1234567890.123456789" => 1234567890.1234567               (Float)
+
+# :bigdecimal — always BigDecimal
+SmarterCSV.process(file, decimal_precision: :bigdecimal)
+# "3.14" => 0.314e1 (BigDecimal)
+```
+
+Unlike Ruby's standard-library CSV — whose `:numeric`/`:float` converters use `Float()` and silently lose precision — `:auto` preserves high-precision decimals as `BigDecimal`. Decimal values are decoded on the C path with the Eisel-Lemire algorithm (correctly rounded, identical to `String#to_f`).
+
 ---
 
 ## `remove_empty_hashes`

data/docs/migrating_from_csv.md

@@ -223,6 +223,24 @@ rows = SmarterCSV.process('sample.csv',
   convert_values_to_numeric: { except: [:zip_code, :phone, :account_number] })
 ```
 
+**High-precision decimals — scientific data and geo coordinates.** GPS/geo coordinates, scientific measurements, and financial figures routinely carry 16+ significant digits, where Ruby's `Float()`-based conversion (`converters: :numeric` / `:float`) silently rounds the value. SmarterCSV's default `decimal_precision: :auto` returns a `BigDecimal` once a value exceeds 16 significant digits (and a `Float` otherwise), so the full value is preserved; scientific notation (`6.022e23`, `1.6e-19`) is recognized as numeric too.
+
+**With Ruby CSV (precision lost):**
+```ruby
+CSV.read('locations.csv', headers: true, converters: :float).first['lat']
+# => -122.42200352825247   ← Float() dropped the last digits of -122.422003528252475
+```
+
+**With SmarterCSV (full precision kept):**
+```ruby
+SmarterCSV.process('locations.csv').first[:lat]
+# => -0.122422003528252475e3   (BigDecimal — all 18 significant digits preserved)
+
+# Force Float everywhere, like-for-like with Ruby CSV:
+SmarterCSV.process('locations.csv', decimal_precision: :float).first[:lat]
+# => -122.42200352825247   (Float)
+```
+
 ### 3. Empty values are removed by default
 
 SmarterCSV drops key/value pairs where the value is `nil` or blank

data/docs/options.md

@@ -121,7 +121,8 @@ See [Parsing Strategy](./parsing_strategy.md) for full details on quote handling
 | Option | Default | Explanation |
 |--------|---------|-------------|
 | `:strip_whitespace` | `true` | Remove whitespace before/after values and headers. |
-| `:convert_values_to_numeric` | `true` | Convert strings containing integers or floats to the appropriate numeric type. Accepts `{except: [:key1, :key2]}` or `{only: :key3}` to limit which columns. |
+| `:convert_values_to_numeric` | `true` | Convert strings containing integers or floats (including scientific notation like `1.5e3`) to the appropriate numeric type. Accepts `{except: [:key1, :key2]}` or `{only: :key3}` to limit which columns. |
+| `:decimal_precision` | `:auto` | How decimals are converted: `:auto` returns `Float` but `BigDecimal` above 16 significant digits (no precision loss); `:float` always returns `Float`; `:bigdecimal` always returns `BigDecimal`. Integers are unaffected. |
 | `:value_converters` | `nil` | Hash of `:header => converter`; converter can be a lambda/Proc or a class implementing `self.convert(value)`. See [Value Converters](./value_converters.md). |
 | `:remove_empty_values` | `true` | Remove key/value pairs where the value is `nil`, empty, or whitespace-only — any Unicode whitespace, same as Ruby's `String#blank?`. |
 | `:remove_zero_values` | `false` | Remove key/value pairs whose value is zero — numeric `0` / `0.0`, or any textual form of zero (`"0"`, `"0.0"`, `"00.00"`, `"+0"`, `"-0.0"`, …). |

data/docs/upgrade_path.json

@@ -1,6 +1,6 @@
 {
-  "latest": "1.17",
-  "latest_release": "1.17.2",
+  "latest": "1.18",
+  "latest_release": "1.18.0",
   "path": {
     "1.0": {
       "to": "1.1",
@@ -170,6 +170,17 @@
       "to": "1.17",
       "latest_release": "1.16.6",
       "actions": []
+    },
+    "1.17": {
+      "to": "1.18",
+      "latest_release": "1.17.4",
+      "actions": [
+        {
+          "if": "you build and run the gem on the same machine (or a fleet of identical CPUs) and want the previous native-optimized build for maximum speed",
+          "then": "set <code>SMARTER_CSV_PERFORMANCE=max</code> (or <code>tuned</code>) at install time &mdash; 1.18.1 builds <strong>portable</strong> by default (no host-specific CPU instructions) to fix an <code>Illegal instruction</code> crash when a binary built on one CPU is run on another (<a href=\"https://github.com/tilo/smarter_csv/issues/343\">#343</a>). The default is safe everywhere; the env var opts back into host optimization."
+        }
+      ],
+      "note": "<strong>Bonus:</strong> 1.18 automatically converts scientific notation (e.g. <code>1e3</code>, <code>6.022e23</code>) to numbers, and preserves full precision on long decimals by returning a <code>BigDecimal</code> when a value carries more than 16 significant digits (<code>decimal_precision: :auto</code>, the default). Nothing to change &mdash; this just works."
     }
   }
 }

data/docs/upgrade_wizard.html

@@ -94,6 +94,15 @@ input[type="text"] {
   color: var(--green);
   font-weight: 600;
 }
+.benefit {
+  background: var(--green-soft);
+  border: 1px solid #b6dab6;
+  border-radius: 8px;
+  padding: 1em 1.25em;
+  margin-bottom: 1em;
+  color: var(--fg);
+}
+.benefit strong { color: var(--green); }
 .check {
   padding: 0.85em 0;
   border-top: 1px solid var(--border);
@@ -229,7 +238,7 @@ function renderHop(series, originalVersion) {
   if (hop.actions.length === 0) {
     body = `<div class="hop dropin">
       You can upgrade directly to version ${targetRelease}. No changes needed.
-    </div>`;
+    </div>${hop.note ? `<div class="benefit">${hop.note}</div>` : ""}`;
   } else {
     body = `<div class="hop">
       ${hop.actions.map((a, i) => `
@@ -324,6 +333,7 @@ function renderHop(series, originalVersion) {
       from:   series,
       to:     hop.to,
       dropIn: hop.actions.length === 0,
+      note:   hop.note || null,
       matched
     });
 
@@ -377,7 +387,8 @@ function renderSummary() {
     const targetRelease = latestReleaseFor(d.to);
     const heading = `<p class="summary-hop-heading"><strong>${d.from}.x &rarr; ${targetRelease}</strong></p>`;
     if (d.dropIn) {
-      return `<div class="summary-hop">${heading}<p class="muted">No code changes needed for this step.</p></div>`;
+      const noteHTML = d.note ? `<p>${d.note}</p>` : "";
+      return `<div class="summary-hop">${heading}<p class="muted">No code changes needed for this step.</p>${noteHTML}</div>`;
     }
     if (d.matched.length === 0) {
       return `<div class="summary-hop">${heading}<p class="muted">None of the conditions in this step applied to your code.</p></div>`;

data/ext/smarter_csv/cpu_flags.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module SmarterCSV
+  # Pure (mkmf-free) selection of CPU-optimization flags from the
+  # SMARTER_CSV_PERFORMANCE environment variable. Kept separate from extconf.rb
+  # so the logic can be unit-tested without invoking a compiler.
+  #
+  # Levels:
+  #   portable - no host-specific flags. The binary runs on any CPU of the same
+  #              architecture. The safe default: a binary built here will not
+  #              crash with "Illegal instruction" on an older/different CPU.
+  #   tuned    - -mtune=native: tunes instruction scheduling for the build host's
+  #              microarchitecture WITHOUT changing the instruction set, so the
+  #              binary stays portable. A real win when build and run hosts share
+  #              a microarchitecture (same chip or a homogeneous fleet).
+  #   max      - host-specific instructions: -march=native, or -mcpu=native on
+  #              ARM/Clang where -march=native is rejected. Fastest, but NOT
+  #              portable -- may crash on a CPU lacking the build host's
+  #              instructions. Use only when build host and run host match.
+  #
+  # `accepts` is a predicate (in the real build, a wrapper over mkmf's
+  # try_compile) returning true when the compiler accepts a given flag; each
+  # candidate is probed so an unsupported flag is skipped rather than breaking
+  # the build.
+  module CpuFlags
+    LEVELS = %w[portable tuned max].freeze
+
+    # Candidate flags per level, in preference order. The first one the compiler
+    # accepts wins. `max` degrades march -> mcpu -> mtune; tuned only ever
+    # considers -mtune=native (never an instruction-set flag).
+    CANDIDATES = {
+      'portable' => [].freeze,
+      'tuned' => ['-mtune=native'].freeze,
+      'max' => ['-march=native', '-mcpu=native', '-mtune=native'].freeze,
+    }.freeze
+
+    # Returns a Hash: { level: String, flags: Array<String>, warning: String|nil }.
+    def self.select(raw_level, accepts:)
+      level, warning = normalize(raw_level)
+      chosen = CANDIDATES[level].find { |flag| accepts.call(flag) }
+      { level: level, flags: chosen ? [chosen] : [], warning: warning }
+    end
+
+    # Normalizes the env value to a known level. Unknown values fall back to
+    # 'portable' (a typo can then only ever be slower, never non-portable) and
+    # return a warning naming the bad value and the fallback.
+    def self.normalize(raw_level)
+      value = raw_level.to_s.strip.downcase
+      return ['portable', nil] if value.empty?
+      return [value, nil] if LEVELS.include?(value)
+
+      ['portable', "SMARTER_CSV_PERFORMANCE=#{raw_level.inspect} is not one of #{LEVELS.join('|')}; using 'portable'."]
+    end
+  end
+end

data/ext/smarter_csv/extconf.rb

@@ -2,6 +2,7 @@
 
 require 'mkmf'
 require "rbconfig"
+require_relative 'cpu_flags'
 
 if RbConfig::MAKEFILE_CONFIG["CFLAGS"].include?("-g -O3")
   fixed_CFLAGS = RbConfig::MAKEFILE_CONFIG["CFLAGS"].sub("-g -O3", "-O3 $(cflags)")
@@ -9,11 +10,31 @@ if RbConfig::MAKEFILE_CONFIG["CFLAGS"].include?("-g -O3")
   RbConfig::MAKEFILE_CONFIG["CFLAGS"] = fixed_CFLAGS
 end
 
+# Probe whether the compiler accepts a flag by compiling a trivial program with
+# it. Lets us skip flags the toolchain rejects (e.g. -march=native on Clang/ARM,
+# or GCC-only flags on MSVC) instead of breaking the build. Replaces the old
+# RUBY_PLATFORM string guesses: ask the actual compiler, don't infer from the OS.
+def compiler_accepts?(flag)
+  try_compile("int main(void){return 0;}", flag)
+end
+
 optflags = "-O3 -flto -fomit-frame-pointer -DNDEBUG".dup
-optflags << " -march=native" unless RUBY_PLATFORM.start_with?("arm64-darwin")
+
+# CPU optimization level, set via SMARTER_CSV_PERFORMANCE (default: portable).
+# See cpu_flags.rb for the full description of each level.
+#
+#   portable (default) - no host-specific flags; runs on any CPU of the same arch.
+#   tuned              - -mtune=native; host scheduling tuning, still portable.
+#   max                - host instruction set (-march/-mcpu native); fastest, but
+#                        NOT portable -- may crash on a CPU lacking those instructions.
+cpu = SmarterCSV::CpuFlags.select(ENV["SMARTER_CSV_PERFORMANCE"], accepts: method(:compiler_accepts?))
+warn(cpu[:warning]) if cpu[:warning]
+cpu[:flags].each { |flag| optflags << " #{flag}" }
+puts("SmarterCSV performance level: #{cpu[:level]} -- optflags: #{optflags}")
+
 # -fno-semantic-interposition: GCC/Clang only (not MSVC). Allows intra-library
 # calls to bypass the PLT on Linux and enables more aggressive LTO inlining.
-optflags << " -fno-semantic-interposition" unless RUBY_PLATFORM.include?("mswin")
+optflags << " -fno-semantic-interposition" if compiler_accepts?("-fno-semantic-interposition")
 
 append_cflags('-Wno-compound-token-split-by-macro')