smart_csv_import 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: f663c2506a13b2fd5f495969fe22337c6a5a6408ecf8dc5c9f12805e73c15c65
+  data.tar.gz: fe62bc86016e471136c30362659fc28aecdaa8d690a5a772cfcf0c880021efe0
+SHA512:
+  metadata.gz: 013d1838ac0a41c353c4f63423a2e891e99509eb71197f956250dafff745ff0e91d6a00402b161acb547c1ed13c87e2e35e236d97fae0fd4b8a3cc30001ff722
+  data.tar.gz: 62f20e4e4fe40ebff5f43af35e528fda2e6d8197183521072c95aa3de491248f1d0fbbd5c44a3beb716776d8ec8bf2f95391c9ddba4e49e4d29a5a091713b181

data/LICENSE.adoc

@@ -0,0 +1,134 @@
+= Hippocratic License
+
+Version: 2.1.0.
+
+Purpose. The purpose of this License is for the Licensor named above to
+permit the Licensee (as defined below) broad permission, if consistent
+with Human Rights Laws and Human Rights Principles (as each is defined
+below), to use and work with the Software (as defined below) within the
+full scope of Licensor’s copyright and patent rights, if any, in the
+Software, while ensuring attribution and protecting the Licensor from
+liability.
+
+Permission and Conditions. The Licensor grants permission by this
+license ("License"), free of charge, to the extent of Licensor’s
+rights under applicable copyright and patent law, to any person or
+entity (the "Licensee") obtaining a copy of this software and
+associated documentation files (the "Software"), to do everything with
+the Software that would otherwise infringe (i) the Licensor’s copyright
+in the Software or (ii) any patent claims to the Software that the
+Licensor can license or becomes able to license, subject to all of the
+following terms and conditions:
+
+* Acceptance. This License is automatically offered to every person and
+entity subject to its terms and conditions. Licensee accepts this
+License and agrees to its terms and conditions by taking any action with
+the Software that, absent this License, would infringe any intellectual
+property right held by Licensor.
+* Notice. Licensee must ensure that everyone who gets a copy of any part
+of this Software from Licensee, with or without changes, also receives
+the License and the above copyright notice (and if included by the
+Licensor, patent, trademark and attribution notice). Licensee must cause
+any modified versions of the Software to carry prominent notices stating
+that Licensee changed the Software. For clarity, although Licensee is
+free to create modifications of the Software and distribute only the
+modified portion created by Licensee with additional or different terms,
+the portion of the Software not modified must be distributed pursuant to
+this License. If anyone notifies Licensee in writing that Licensee has
+not complied with this Notice section, Licensee can keep this License by
+taking all practical steps to comply within 30 days after the notice. If
+Licensee does not do so, Licensee’s License (and all rights licensed
+hereunder) shall end immediately.
+* Compliance with Human Rights Principles and Human Rights Laws.
+[arabic]
+. Human Rights Principles.
+[loweralpha]
+.. Licensee is advised to consult the articles of the United Nations
+Universal Declaration of Human Rights and the United Nations Global
+Compact that define recognized principles of international human rights
+(the "Human Rights Principles"). Licensee shall use the Software in a
+manner consistent with Human Rights Principles.
+.. Unless the Licensor and Licensee agree otherwise, any dispute,
+controversy, or claim arising out of or relating to (i) Section 1(a)
+regarding Human Rights Principles, including the breach of Section 1(a),
+termination of this License for breach of the Human Rights Principles,
+or invalidity of Section 1(a) or (ii) a determination of whether any Law
+is consistent or in conflict with Human Rights Principles pursuant to
+Section 2, below, shall be settled by arbitration in accordance with the
+Hague Rules on Business and Human Rights Arbitration (the "Rules");
+provided, however, that Licensee may elect not to participate in such
+arbitration, in which event this License (and all rights licensed
+hereunder) shall end immediately. The number of arbitrators shall be one
+unless the Rules require otherwise.
++
+Unless both the Licensor and Licensee agree to the contrary: (1) All
+documents and information concerning the arbitration shall be public and
+may be disclosed by any party; (2) The repository referred to under
+Article 43 of the Rules shall make available to the public in a timely
+manner all documents concerning the arbitration which are communicated
+to it, including all submissions of the parties, all evidence admitted
+into the record of the proceedings, all transcripts or other recordings
+of hearings and all orders, decisions and awards of the arbitral
+tribunal, subject only to the arbitral tribunal’s powers to take such
+measures as may be necessary to safeguard the integrity of the arbitral
+process pursuant to Articles 18, 33, 41 and 42 of the Rules; and (3)
+Article 26(6) of the Rules shall not apply.
+. Human Rights Laws. The Software shall not be used by any person or
+entity for any systems, activities, or other uses that violate any Human
+Rights Laws. "Human Rights Laws" means any applicable laws,
+regulations, or rules (collectively, "Laws") that protect human,
+civil, labor, privacy, political, environmental, security, economic, due
+process, or similar rights; provided, however, that such Laws are
+consistent and not in conflict with Human Rights Principles (a dispute
+over the consistency or a conflict between Laws and Human Rights
+Principles shall be determined by arbitration as stated above). Where
+the Human Rights Laws of more than one jurisdiction are applicable or in
+conflict with respect to the use of the Software, the Human Rights Laws
+that are most protective of the individuals or groups harmed shall
+apply.
+. Indemnity. Licensee shall hold harmless and indemnify Licensor (and
+any other contributor) against all losses, damages, liabilities,
+deficiencies, claims, actions, judgments, settlements, interest, awards,
+penalties, fines, costs, or expenses of whatever kind, including
+Licensor’s reasonable attorneys’ fees, arising out of or relating to
+Licensee’s use of the Software in violation of Human Rights Laws or
+Human Rights Principles.
+* Failure to Comply. Any failure of Licensee to act according to the
+terms and conditions of this License is both a breach of the License and
+an infringement of the intellectual property rights of the Licensor
+(subject to exceptions under Laws, e.g., fair use). In the event of a
+breach or infringement, the terms and conditions of this License may be
+enforced by Licensor under the Laws of any jurisdiction to which
+Licensee is subject. Licensee also agrees that the Licensor may enforce
+the terms and conditions of this License against Licensee through
+specific performance (or similar remedy under Laws) to the extent
+permitted by Laws. For clarity, except in the event of a breach of this
+License, infringement, or as otherwise stated in this License, Licensor
+may not terminate this License with Licensee.
+* Enforceability and Interpretation. If any term or provision of this
+License is determined to be invalid, illegal, or unenforceable by a
+court of competent jurisdiction, then such invalidity, illegality, or
+unenforceability shall not affect any other term or provision of this
+License or invalidate or render unenforceable such term or provision in
+any other jurisdiction; provided, however, subject to a court
+modification pursuant to the immediately following sentence, if any term
+or provision of this License pertaining to Human Rights Laws or Human
+Rights Principles is deemed invalid, illegal, or unenforceable against
+Licensee by a court of competent jurisdiction, all rights in the
+Software granted to Licensee shall be deemed null and void as between
+Licensor and Licensee. Upon a determination that any term or provision
+is invalid, illegal, or unenforceable, to the extent permitted by Laws,
+the court may modify this License to affect the original purpose that
+the Software be used in compliance with Human Rights Principles and
+Human Rights Laws as closely as possible. The language in this License
+shall be interpreted as to its fair meaning and not strictly for or
+against any party.
+* Disclaimer. TO THE FULL EXTENT ALLOWED BY LAW, THIS SOFTWARE COMES
+"AS IS," WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, AND LICENSOR AND
+ANY OTHER CONTRIBUTOR SHALL NOT BE LIABLE TO ANYONE FOR ANY DAMAGES OR
+OTHER LIABILITY ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE
+OR THIS LICENSE, UNDER ANY KIND OF LEGAL CLAIM.
+
+This Hippocratic License is an link:https://ethicalsource.dev[Ethical Source license] and is offered
+for use by licensors and licensees at their own risk, on an "AS IS" basis, and with no warranties
+express or implied, to the maximum extent permitted by Laws.

data/README.md

@@ -0,0 +1,534 @@
+# SmartCsvImport
+
+[![CI](https://github.com/Nroulston/smart_csv_import/actions/workflows/ci.yml/badge.svg)](https://github.com/Nroulston/smart_csv_import/actions/workflows/ci.yml)
+[![Gem Version](https://badge.fury.io/rb/smart_csv_import.svg)](https://badge.fury.io/rb/smart_csv_import)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE.adoc)
+
+A Rails Engine for CSV importing with AI-powered header matching. Drop in a form object, describe your fields in plain English, and SmartCsvImport automatically maps whatever column names show up in the CSV — messy, abbreviated, or domain-specific — to your fields. No brittle column-name checks to maintain.
+
+Matching uses a two-tier strategy: **vector similarity** (fast, cached embeddings) handles the common case, and **LLM fallback** handles ambiguous headers that need cross-field reasoning. Row data is never sent to an AI service — only headers and field descriptions.
+
+---
+
+## Table of Contents
+
+- [Quickstart](#quickstart)
+- [Reviewing Failed Rows](#reviewing-failed-rows)
+- [Form Object DSL](#form-object-dsl)
+- [Configuration](#configuration)
+- [Matching Strategies](#matching-strategies)
+- [Processing Modes](#processing-modes)
+- [Advanced](#advanced)
+- [Contributing](#contributing)
+
+---
+
+## Quickstart
+
+### 1. Configure an AI provider
+
+SmartCsvImport delegates all AI calls to [ruby_llm](https://github.com/crmne/ruby_llm). Configure it with your provider credentials before using vector or LLM matching strategies.
+
+```ruby
+# config/initializers/ruby_llm.rb
+RubyLLM.configure do |config|
+  # OpenAI
+  config.openai_api_key = ENV["OPENAI_API_KEY"]
+
+  # — or Anthropic —
+  config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"]
+
+  # — or Google (Gemini embedding is free tier, good for getting started) —
+  config.gemini_api_key = ENV["GEMINI_API_KEY"]
+end
+```
+
+> **Free tier option:** The default models (`gemini-embedding-001` for embeddings, `claude-haiku-4-5-20251001` for LLM fallback) work well for development and small workloads. A Google AI Studio key gives you free Gemini embeddings. See [Model Selection](#model-selection) for upgrading.
+
+### 2. Install the gem
+
+```ruby
+# Gemfile
+gem "smart_csv_import"
+```
+
+```bash
+bundle install
+```
+
+### 3. Run the install generator
+
+```bash
+rails generate smart_csv_import:install
+```
+
+This creates:
+
+- `config/initializers/smart_csv_import.rb` — configuration with all options commented out at their defaults
+- `db/migrate/YYYYMMDDHHMMSS_create_smart_csv_import_imports.rb` — import tracking table
+- `tmp/smart_csv_import/` and `tmp/smart_csv_import/embeddings_cache/` — storage directories
+
+```bash
+rails db:migrate
+```
+
+### 4. Generate an import form
+
+```bash
+rails generate smart_csv_import:import Employee first_name last_name email
+```
+
+This creates `app/forms/employee_import_row.rb`:
+
+```ruby
+class EmployeeImportRow
+  include ActiveModel::Validations
+  include SmartCsvImport::Matchable
+
+  attr_accessor :first_name, :last_name, :email
+
+  # The description is what the AI matches against — be specific.
+  # "Employee first/given name" beats "first name" for ambiguous CSVs.
+  csv_field :first_name, description: "Employee first/given name", required: true
+  csv_field :last_name,  description: "Employee last/family name",  required: true
+  csv_field :email,      description: "Employee email address"
+
+  validates :first_name, :last_name, presence: true
+
+  def save
+    Employee.create!(first_name: first_name, last_name: last_name, email: email)
+    true
+  rescue ActiveRecord::RecordInvalid
+    false
+  end
+end
+```
+
+Edit the `save` method to persist however your app needs. SmartCsvImport calls `save` once per row.
+
+### 5. Process a CSV
+
+```ruby
+result = SmartCsvImport.process("path/to/employees.csv", form_class: EmployeeImportRow)
+
+result.completed?       # => true
+result.imported         # => 98
+result.failed           # => 2
+result.total            # => 100
+result.errors           # => [#<RowError row=42 column=:email messages=["is invalid"]>]
+result.warnings         # => [#<RowWarning message="Column 'Nickname' was not mapped">]
+result.header_mappings  # => {"First Name" => "first_name", "Surname" => "last_name", ...}
+```
+
+That's it. SmartCsvImport reads the CSV headers, maps them to your fields, and calls `save` on a form object for each row.
+
+---
+
+## Reviewing Failed Rows
+
+Every row that fails — whether due to CSV parsing (malformed quotes, encoding issues) or form validation — is captured as structured data so you can build whatever review experience you want: a paginated admin UI, a corrective re-export, a retry workflow.
+
+### In-memory: the `Result` object (sync callers)
+
+```ruby
+result = SmartCsvImport.process("employees.csv", form_class: EmployeeImportRow)
+
+result.errors        # => [#<RowError row=42 column=:email messages=["is invalid"]>, ...]
+result.parse_errors  # => [#<ParseError line_number=87 raw_line='"Bob,Jones...' error_message="Unclosed quoted field">, ...]
+```
+
+- `RowError(row, column, messages)` — one per field that failed validation. A single CSV row that fails on multiple columns produces multiple `RowError` records.
+- `ParseError(line_number, raw_line, error_message)` — one per CSV row that couldn't be parsed at all.
+
+### Persistent: the `Import#row_errors` association (async, later, or both)
+
+Every failure is persisted to the `smart_csv_import_import_row_errors` table, giving you an ActiveRecord association to query at any time — including after an async `ImportJob` has completed and the in-memory `Result` is gone.
+
+```ruby
+import = SmartCsvImport::Import.find(import_id)
+
+import.row_errors.count                        # => 7
+import.row_errors.validation_errors.count      # => 5
+import.row_errors.parse_errors.count           # => 2
+
+# Pagination, filtering, grouping — all plain ActiveRecord:
+import.row_errors.validation_errors.where(column_name: "email").limit(50).offset(100)
+import.row_errors.group(:error_type, :column_name).count
+```
+
+Each `SmartCsvImport::ImportRowError` record:
+
+| Field | Type | Populated for | Description |
+|---|---|---|---|
+| `import_id` | integer | both | FK to `smart_csv_import_imports` (cascade delete) |
+| `row_number` | integer | both | Physical CSV line number (1-indexed, headers on line 1) |
+| `error_type` | string | both | `"validation"` or `"parse"` |
+| `column_name` | string | validation | Form field that failed (e.g. `"email"`) |
+| `messages` | json (array) | validation | Error messages from the form object (e.g. `["is invalid"]`) |
+| `raw_line` | text | parse | Literal CSV row text that failed to parse |
+| `error_message` | text | parse | Parser's description of the failure |
+
+### Downloadable "fix and re-upload" CSV
+
+Re-export failed rows to a CSV with the original headers plus an `_error` column — drop it in front of a user, let them fix the rows, and re-upload.
+
+```ruby
+# From a sync Result:
+output_path = SmartCsvImport::FailedRowExporter.new(
+  result: result,
+  csv_path: "path/to/original.csv"
+).call
+
+# Or from a persisted Import (e.g. after an async job):
+output_path = SmartCsvImport::FailedRowExporter.new(
+  import: SmartCsvImport::Import.find(import_id),
+  csv_path: "path/to/original.csv"
+).call
+# => "tmp/smart_csv_import/failed_rows/20260423142530_failed.csv"
+```
+
+The exporter writes only validation failures — parse errors keep their `raw_line` intact on the `row_errors` record, which is usually more useful for inspecting malformed CSV than re-exporting.
+
+---
+
+## Form Object DSL
+
+Include `SmartCsvImport::Matchable` in any class with `attr_accessor` declarations and use `csv_field` to register fields for matching.
+
+### `csv_field`
+
+```ruby
+csv_field :field_name, description: "...", required: false
+```
+
+| Parameter | Required | Description |
+|---|---|---|
+| `name` | yes | Must match an `attr_accessor` on the class |
+| `description:` | yes | Plain-English description — this is what the AI matches CSV headers against |
+| `required:` | no | When `true`, a failed match transitions the import to `mapping_review` instead of processing rows (see [Import Tracking](#import-tracking)) |
+
+**Write good descriptions.** The description is the only signal the AI has. Vague descriptions produce weaker matches.
+
+```ruby
+# Weaker — too generic
+csv_field :amount, description: "amount"
+
+# Stronger — specific and unambiguous
+csv_field :amount, description: "Invoice total amount in dollars"
+```
+
+### `csv_source` and `csv_context`
+
+Optional class-level hints that give the LLM domain knowledge for disambiguating headers it can't resolve from descriptions alone.
+
+```ruby
+class EmployeeImportRow
+  include SmartCsvImport::Matchable
+
+  # Where the CSV comes from
+  csv_source "ADP Workforce payroll export"
+
+  # The business domain of your app
+  csv_context "HR platform for staffing agencies"
+
+  csv_field :mobile_phone, description: "Employee mobile phone number"
+  # ...
+end
+```
+
+Without context, the LLM sees "Cell" and has no way to know if it means a mobile number, a prison cell, or a biological cell. With `csv_source` and `csv_context`, it can reason correctly.
+
+---
+
+## Configuration
+
+```ruby
+# config/initializers/smart_csv_import.rb
+SmartCsvImport.configure do |config|
+  config.confidence_threshold = 0.80
+  config.batch_size            = 500
+  config.storage_path          = "tmp/smart_csv_import"
+  config.default_strategy      = :vector
+  config.llm_model             = "claude-haiku-4-5-20251001"
+  config.embedding_model       = "gemini-embedding-001"
+  config.value_hint_rows       = 5
+end
+```
+
+| Option | Default | Description |
+|---|---|---|
+| `confidence_threshold` | `0.80` | Minimum cosine similarity score to accept a vector or LLM match. Headers below this threshold fall through to the next strategy tier or become unmatched. |
+| `batch_size` | `500` | How often (in rows) the import record is updated with progress counts during processing. |
+| `storage_path` | `"tmp/smart_csv_import"` | Root directory for stored CSV files and the embedding cache. |
+| `default_strategy` | `:vector` | Which strategy tier to start with when no custom strategy is set on the form class. |
+| `llm_model` | `"claude-haiku-4-5-20251001"` | The LLM used for fallback matching. Any model supported by ruby_llm can be used. |
+| `embedding_model` | `"gemini-embedding-001"` | The embedding model used by the vector strategy. |
+| `value_hint_rows` | `5` | Number of sample rows inspected to apply value-based confidence adjustments (e.g. boosting confidence when cell values look like dates or emails). |
+
+### Model selection
+
+Better models produce better matching accuracy on ambiguous or domain-specific headers. The defaults are suitable for development and light workloads. To upgrade:
+
+```ruby
+config.embedding_model = "text-embedding-3-large"  # OpenAI — higher dimensionality
+config.llm_model       = "claude-sonnet-4-6"        # Anthropic — stronger reasoning
+```
+
+Any model listed in the [ruby_llm documentation](https://github.com/crmne/ruby_llm) works — no other changes needed.
+
+---
+
+## Matching Strategies
+
+### How the fallback chain works
+
+For each unmatched header, SmartCsvImport tries strategies in order and accepts the first result that meets the confidence threshold:
+
+```
+CSV headers
+    │
+    ▼
+Custom strategy (if set on form class)
+    │ unmatched or below threshold
+    ▼
+Vector strategy (embedding cosine similarity)
+    │ unmatched or below threshold
+    ▼
+LLM strategy (structured prompt)
+    │ unmatched
+    ▼
+UnmatchedResult → warning on the result object
+```
+
+Once a header is matched, it does not pass to the next tier. A header that clears all three tiers unmatched becomes an `UnmatchedResult` and generates a warning — it does not cause the import to fail.
+
+### Vector strategy
+
+Computes embeddings for your field descriptions and the incoming CSV headers, then accepts the highest-scoring mutual match above the confidence threshold.
+
+Field embeddings are cached to disk (keyed by your field definitions) so the API is only called once per unique set of fields — subsequent imports of the same type are fast.
+
+Only headers and field descriptions are sent to the embedding API. Row data is never transmitted.
+
+### LLM strategy
+
+Fires for headers that the vector strategy couldn't match confidently. Sends all remaining headers and all field definitions together in a single prompt, letting the LLM reason across the full set:
+
+> "Cell" next to `first_name`, `last_name`, `email` → clearly a phone number.  
+> "Cell" in isolation → ambiguous.
+
+Cross-field context is what makes this effective. Only headers and descriptions are sent — never row data.
+
+### Lookup strategy
+
+Zero AI. For systems with fixed, known column names.
+
+```ruby
+class HrSystemMapping < SmartCsvImport::Strategies::Lookup
+  mappings(
+    "EMP_ID"  => :employee_id,
+    "FNAME"   => :first_name,
+    "LNAME"   => :last_name,
+    "DOB"     => :date_of_birth
+  )
+end
+
+class EmployeeImportRow
+  include SmartCsvImport::Matchable
+
+  self.matching_strategy = HrSystemMapping.new
+  # ...
+end
+```
+
+Because a custom strategy runs first in the chain, matches from the Lookup table skip vector and LLM entirely.
+
+### Custom strategy
+
+Subclass `SmartCsvImport::Strategy` and implement `match`:
+
+```ruby
+class MyStrategy < SmartCsvImport::Strategy
+  def match(csv_headers:, form_class:, sample_rows: [])
+    csv_headers.each_with_object({}) do |header, results|
+      next unless header.downcase == "emp_id"
+
+      results[header] = SmartCsvImport::MatchResult.matched(
+        target_field: :employee_id,
+        confidence:   1.0,
+        strategy_name: "my_strategy"
+      )
+    end
+  end
+end
+```
+
+Return only the headers your strategy can confidently match. Headers you omit fall through to the next tier.
+
+---
+
+## Processing Modes
+
+### Synchronous (default)
+
+Blocks until all rows are processed. Returns a result object.
+
+```ruby
+result = SmartCsvImport.process("file.csv", form_class: MyImportRow)
+result.completed?  # => true
+```
+
+Use for small files, scripts, or rake tasks.
+
+### Asynchronous
+
+Enqueues a background job and returns immediately.
+
+```ruby
+result = SmartCsvImport.process("file.csv", form_class: MyImportRow, mode: :async)
+result.queued?    # => true
+result.import_id  # => 42
+```
+
+The job runs on the `:smart_csv_import` queue. Requires a queue backend — [Sidekiq](https://github.com/sidekiq/sidekiq), [GoodJob](https://github.com/bensheldon/good_job), or any Active Job adapter. Use for user-facing uploads where you don't want to block a web request.
+
+### Dry run
+
+Validates every row without persisting anything.
+
+```ruby
+result = SmartCsvImport.process("file.csv", form_class: MyImportRow, dry_run: true)
+result.dry_run?  # => true
+result.imported  # => 95  (would succeed)
+result.failed    # => 5   (would fail, with errors)
+```
+
+Use to preview results before committing an import.
+
+---
+
+## Advanced
+
+### Header matching only
+
+Inspect the raw mapping decisions without processing any rows:
+
+```ruby
+mappings = SmartCsvImport.match_headers("file.csv", form_class: MyImportRow)
+# => {
+#      "First Name" => #<MatchResult target_field=:first_name confidence=0.97 strategy="vector">,
+#      "Cell"       => #<MatchResult target_field=:mobile_phone confidence=0.91 strategy="llm">,
+#      "Nickname"   => #<UnmatchedResult csv_header="Nickname" attempted_strategies=["vector", "llm"]>
+#    }
+```
+
+Useful for building a review UI before committing large imports.
+
+### Import tracking
+
+Every `SmartCsvImport.process` call creates a `SmartCsvImport::Import` record:
+
+| Status | Meaning |
+|---|---|
+| `pending` | Created, not yet started |
+| `processing` | Actively running |
+| `completed` | All rows processed successfully |
+| `partial_failure` | Some rows failed validation |
+| `failed` | Processing stopped due to a database error |
+| `mapping_review` | A `required:` field could not be matched — no rows were processed |
+
+The record also stores the header mappings used, row counts, and a SHA-256 hash of the file. Duplicate file detection compares this hash before processing — a warning is added to the result if a match is found.
+
+### Stability analysis
+
+After running several imports of the same type, check which header mappings have solidified:
+
+```ruby
+report   = SmartCsvImport::StabilityReport.new(import_type: "EmployeeImportRow")
+analysis = report.analyze
+
+analysis.imports_analyzed  # => 20
+analysis.stable_fields     # => fields consistent >= 90% of the time
+analysis.unstable_fields   # => fields with varying resolutions
+
+puts report.summary
+# Stability report for EmployeeImportRow (20 imports analyzed):
+#   Stable fields (3):
+#     - First Name -> first_name (100.0% consistent)
+#     - Last Name  -> last_name  (100.0% consistent)
+#     - Email      -> email      (95.0% consistent)
+```
+
+Fields stable at >= 90% are good candidates for promotion to a [Lookup strategy](#lookup-strategy). Doing so eliminates AI calls for those fields entirely on future imports.
+
+### Normalizers
+
+Built-in converters for common CSV data types. Use them in your `save` method:
+
+```ruby
+SmartCsvImport::Normalizers::DateConverter.call("03/15/2024")  # => #<Date: 2024-03-15>
+SmartCsvImport::Normalizers::DateConverter.call("2024-03-15")  # => #<Date: 2024-03-15>
+SmartCsvImport::Normalizers::BooleanConverter.call("yes")      # => true
+SmartCsvImport::Normalizers::BooleanConverter.call("0")        # => false
+
+# In your form object:
+def save
+  Employee.create!(
+    name:       name,
+    hired_on:   SmartCsvImport::Normalizers::DateConverter.call(hired_on),
+    active:     SmartCsvImport::Normalizers::BooleanConverter.call(active)
+  )
+  true
+rescue ActiveRecord::RecordInvalid
+  false
+end
+```
+
+---
+
+## Contributing
+
+### Architecture overview
+
+```
+SmartCsvImport.process(file, form_class:)
+    └── Processor
+            ├── FileStorage        stores file, computes hash, checks duplicates
+            ├── Matcher            runs strategy chain, returns header → MatchResult map
+            │       ├── Strategies::Vector   cosine similarity on embeddings (cached)
+            │       └── Strategies::Llm      structured LLM prompt
+            └── (per row) form_class.new(attrs).save
+```
+
+Key files:
+
+| File | Purpose |
+|---|---|
+| `lib/smart_csv_import.rb` | Public API: `.process`, `.match_headers`, `.configure` |
+| `lib/smart_csv_import/processor.rb` | Orchestrates matching + row processing + result building |
+| `lib/smart_csv_import/matcher.rb` | Runs the strategy chain, applies value hints |
+| `lib/smart_csv_import/matchable.rb` | `csv_field` DSL mixed into form objects |
+| `lib/smart_csv_import/strategies/` | Vector, LLM, Lookup, and base Strategy class |
+| `lib/smart_csv_import/result.rb` | Result value objects returned from `.process` |
+
+The `Configuration` object is a global singleton accessed via `SmartCsvImport.configuration`. The `Engine` class hooks it into Rails' initializer and migration loading.
+
+### Getting started
+
+```bash
+git clone https://github.com/Nroulston/smart_csv_import
+cd smart_csv_import
+bin/setup
+bin/rake        # runs the full test suite
+bin/console     # IRB with all gem code loaded
+```
+
+### Design decisions
+
+Before sending a PR that touches the matching strategies, read [`ROADMAP.md`](ROADMAP.md). It documents two approaches that were evaluated and explicitly rejected (HyDE, asymmetric embedding augmentation) with the reasoning — understanding why they were ruled out will save you from rediscovering the same dead ends.
+
+---
+
+## License
+
+[MIT](LICENSE.adoc)