ixport 0.2.0 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 48f9af7983bb1c63e4df9cf1c02681f6a0aba4f40a7bf86fe7cfbfe072cf82d2
-  data.tar.gz: 816d85ef4796bc08057d087d8ef67c96366a2d3f3726680be4b645479fbdc703
+  metadata.gz: 7010d4338968ec32338a7f02a446a59a091487b0860f955c090f5e180b61d65a
+  data.tar.gz: 82f2bf80f348f9a308a0c3674efaf62441c0b4865cfe867e599243861b5227fe
 SHA512:
-  metadata.gz: 9592b85531796f33f18e3610de6574ff1530d64cc8b692161e54621589a2ef52ed1e80e40ace9622522bd402e96548e710e16e18db35ab1b018a30d56d972171
-  data.tar.gz: 744e6ff530c9a5b1b79bdcdc8f6fb3388c4c8f329948b90f5cddea19bf0c40a85db7a3c10caa8653719326878b67ee32711f92b35b7d43c15300dd4e6aca755a
+  metadata.gz: 363acc8c4d91440e255240436d96196607831fd98ebcab6d90fc3c817b94609aa5a2978ac16aab41eb3b6fd4371f9d565367eec0bb08c32596e351c647fdb683
+  data.tar.gz: cda9b8842119709c211008e76820185f0d36544b831d718b8bfa743f0b681b4a9e79919b341b3a73acf0c600b98d15472a72632861b5d6a51635089bea0d935b

data/README.md

@@ -41,9 +41,12 @@ Ixport.configure do |config|
   config.layout = "application"                      # the layout ixport's pages are drawn in
   config.inline_row_limit = 1_000                    # the most rows an import may have when it runs inside the request
   config.background_runner = nil                     # :wallflower, or the name of the app's own runner class
+  config.base_controller = "::ApplicationController" # the controller ixport's pages inherit, with its checks and helpers
 end
 ```
 
+`config.base_controller` is read when ixport's controllers load, so set it in an initializer. Name a controller such as an admin area's base controller to give ixport's pages that area's access checks.
+
 ## Mapped imports and importers
 
 A mapped import is a kind of CSV a person can import into one of the app's records. Register each one in an initializer with its key, its title and the name of its importer class:
@@ -56,7 +59,7 @@ An app whose registered importer class does not exist fails to boot, with a mess
 
 The engine's root page lists every registered import by its title, each linking to its upload page.
 
-An importer lists the fields a person maps the file's columns to, each with the label the person sees. It names the one field rows are matched on, and the records a row may match, which it reaches through the `person` and `account` the import is for:
+An importer lists the fields a person maps the file's columns to, each with the label the person sees. It names the field or fields rows are matched on, and the records a row may match, which it reaches through the `person` and `account` the import is for:
 
 ```ruby
 class ProductImporter < Ixport::Importer
@@ -70,7 +73,9 @@ class ProductImporter < Ixport::Importer
 end
 ```
 
-Each field's key is the record's attribute name. ixport looks up every match value in the file with one query against `records`. A row whose match value matches one of those records updates it, and a row that matches nothing adds a new record built from `records`. A row whose match value is blank, or whose file has no column mapped to the match field, always adds a record.
+Each field's key is the record's attribute name. ixport looks up every match value in the file with one query against `records`. A row whose match values equal one of those records on every match field updates it, and a row that matches nothing adds a new record built from `records`. Values are compared as the record's own types, so `12.50` in a file matches `12.5` stored on a record. A row with a blank value in any match field, or whose file has no column mapped to one, always adds a record.
+
+To match on several fields, name them all, for example `match_on :happened_on, :amount, :name`.
 
 ## Uploading and mapping a file
 

data/app/controllers/ixport/application_controller.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Ixport
-  class ApplicationController < ::ApplicationController
+  class ApplicationController < Ixport.configuration.base_controller.constantize
     before_action { send(Ixport.configuration.authentication_method) }
     layout -> { Ixport.configuration.layout }
 

data/app/models/ixport/row_walk.rb

@@ -19,7 +19,7 @@ module Ixport
 
     def row_for(cells, number)
       values = values_of(cells)
-      existing = existing_records[values[match_field]]
+      existing = own_copy(existing_records[match_key(values)])
       record = existing ? existing.tap { |found| found.assign_attributes(values) } : @importer.records.new(values)
       return Row.new(number: number, record: record, outcome: :refused, reason: record.errors.full_messages.to_sentence) unless record.valid?
 
@@ -34,14 +34,29 @@ module Ixport
       @import.mapping.to_h { |column, field| [ field.to_sym, cells[column.to_i] ] }
     end
 
-    def match_field
-      @importer.class.match_field
+    def own_copy(record)
+      record&.class&.instantiate(record.attributes_before_type_cast)
+    end
+
+    def match_fields
+      @importer.class.match_fields
+    end
+
+    def match_key(values)
+      return if match_fields.empty?
+
+      key = match_fields.map { |field| cast(field, values[field]) }
+      key unless key.any?(&:blank?)
+    end
+
+    def cast(field, value)
+      @importer.records.klass.type_for_attribute(field.to_s).cast(value)
     end
 
     def existing_records
       @existing_records ||= begin
-        match_values = data_rows.map { |cells| values_of(cells)[match_field] }.compact_blank
-        @importer.records.where(match_field => match_values).index_by { |record| record.public_send(match_field).to_s }
+        candidates = match_fields.to_h { |field| [ field, data_rows.map { |cells| cast(field, values_of(cells)[field]) }.compact_blank ] }
+        @importer.records.where(candidates).index_by { |record| match_fields.map { |field| record.public_send(field) } }
       end
     end
   end

data/lib/ixport/configuration.rb

@@ -2,7 +2,7 @@
 
 module Ixport
   class Configuration
-    attr_accessor :authentication_method, :current_person_method, :current_account_method, :layout, :inline_row_limit, :background_runner
+    attr_accessor :authentication_method, :current_person_method, :current_account_method, :layout, :inline_row_limit, :background_runner, :base_controller
 
     def initialize
       @authentication_method = :authenticate_user!
@@ -11,6 +11,7 @@ module Ixport
       @layout = "application"
       @inline_row_limit = 1_000
       @background_runner = nil
+      @base_controller = "::ApplicationController"
     end
   end
 

data/lib/ixport/importer.rb

@@ -19,12 +19,16 @@ module Ixport
       @fields ||= []
     end
 
-    def self.match_on(key)
-      @match_field = key.to_sym
+    def self.match_on(*keys)
+      @match_fields = keys.map(&:to_sym)
+    end
+
+    def self.match_fields
+      @match_fields || []
     end
 
     def self.match_field
-      @match_field
+      match_fields.first
     end
   end
 end

data/lib/ixport/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Ixport
-  VERSION = "0.2.0"
+  VERSION = "0.3.1"
 end

data/the_local/agents/ixport-develop.md

@@ -1,6 +1,6 @@
 ---
 name: ixport-develop
-description: Use PROACTIVELY for setting up CSV imports and exports in a Rails app that has ixport — registering a mapped import, writing its importer class, declaring the fields a person maps a CSV's columns to, choosing the field rows are matched on, scoping the records an import may add or update to the signed-in person or account, limiting which people may use an import or export with an access rule such as the app's own Pundit policy, linking to an import's upload page, explaining the upload, column-matching, preview and result pages and their messages, explaining the list of a person's past imports on the imports page, explaining what running an import saves and refuses, explaining what a person sees when an import fails partway and which rows it keeps, running imports in a background job through the app's own runner class, explaining what a person sees while an import runs in the background, registering a fixed export or a chosen export, writing its exporter class, declaring its columns including columns taken from a related record, computing a column's value, scoping the records an export may hold, letting an account save named templates of picked columns for an export, explaining the new, edit and delete template pages and what a template's download holds, linking to the exports page, one export's CSV download or one template's download, making the app's own import job fail with Ixport::ImportFailed, and fixing a boot that fails with Ixport::MissingImporter or Ixport::MissingExporter — MUST BE USED instead of hand-writing a CSV upload controller, a column-mapping form, a row-matching loop, an import preview, a loop that saves imported rows, a background job that saves imported rows, a rescue that records why an import stopped, a permission check in front of an import or export page, a CSV download action, a loop that writes records to a CSV, or a model and form for saved column picks.
+description: Use PROACTIVELY for setting up CSV imports and exports in a Rails app that has ixport — registering a mapped import, writing its importer class, declaring the fields a person maps a CSV's columns to, choosing the field or fields rows are matched on, scoping the records an import may add or update to the signed-in person or account, limiting which people may use an import or export with an access rule such as the app's own Pundit policy, linking to an import's upload page, explaining the upload, column-matching, preview and result pages and their messages, explaining the list of a person's past imports on the imports page, explaining what running an import saves and refuses, explaining what a person sees when an import fails partway and which rows it keeps, running imports in a background job through the app's own runner class, explaining what a person sees while an import runs in the background, registering a fixed export or a chosen export, writing its exporter class, declaring its columns including columns taken from a related record, computing a column's value, scoping the records an export may hold, letting an account save named templates of picked columns for an export, explaining the new, edit and delete template pages and what a template's download holds, linking to the exports page, one export's CSV download or one template's download, making the app's own import job fail with Ixport::ImportFailed, and fixing a boot that fails with Ixport::MissingImporter or Ixport::MissingExporter — MUST BE USED instead of hand-writing a CSV upload controller, a column-mapping form, a row-matching loop, an import preview, a loop that saves imported rows, a background job that saves imported rows, a rescue that records why an import stopped, a permission check in front of an import or export page, a CSV download action, a loop that writes records to a CSV, or a model and form for saved column picks.
 tools: Bash, Read, Write, Edit, Grep
 scope: CSV import and export — a mounted engine where a signed-in person sees the mapped imports and the exports the host app registers, each import backed by an importer class and each export by an exporter class the host writes, where an export is either fixed or chosen, and a chosen export lets the account save named templates of picked columns
 ---
@@ -9,14 +9,14 @@ This local adds mapped imports, fixed exports and chosen exports to a Rails app
 
 ## What ixport is
 
-ixport is a Rails engine that lets a signed-in person import a host app's records from CSV files and download them as CSV files. The app registers each kind of CSV a person can import, called a mapped import, and writes an importer class that names the fields the CSV's columns map to, the one field rows are matched on, and the records a row may add to or update. A person picks an import, uploads a CSV file for it, matches each of the file's columns to one of the importer's fields, sees a preview of how many rows will add a record, update one or be refused, runs the import to save those rows, and then sees how many rows were added and updated and each refused row. When the run stops on an error, the import is marked failed, keeps the rows it saved before stopping, and its page shows the error. An import runs within the request by default, or, when the app is set up for it, in the background through wallflower or through a runner class the app writes. The imports page also lists the person's own past imports in the current account, each opening at the step it has reached. The app also registers each CSV a person can download, and writes an exporter class that names the columns the file can hold, in order, and the records it holds one row each for. An export is either fixed or chosen. A fixed export always holds every declared column, and a person downloads it from the exports page. A chosen export lets people in the current account save named templates, each a picked list of the exporter's columns in a picked order, and a person downloads a template from the exports page to get a file holding only those columns. Fire this local when a developer wants people to load one of the app's records from a spreadsheet or download them as one, wants people to pick which columns an export holds, wants to link to an import's upload page, the exports page, an export's download or a template's download, asks what the upload, column-matching, preview, result, exports or template pages do, asks what the list of past imports shows, asks what running an import saves, what happens when a run fails, or what an export's or template's file holds, wants imports run in a background job of the app's own, or when the app fails to boot with `Ixport::MissingImporter` or `Ixport::MissingExporter`.
+ixport is a Rails engine that lets a signed-in person import a host app's records from CSV files and download them as CSV files. The app registers each kind of CSV a person can import, called a mapped import, and writes an importer class that names the fields the CSV's columns map to, the field or fields rows are matched on, and the records a row may add to or update. A person picks an import, uploads a CSV file for it, matches each of the file's columns to one of the importer's fields, sees a preview of how many rows will add a record, update one or be refused, runs the import to save those rows, and then sees how many rows were added and updated and each refused row. When the run stops on an error, the import is marked failed, keeps the rows it saved before stopping, and its page shows the error. An import runs within the request by default, or, when the app is set up for it, in the background through wallflower or through a runner class the app writes. The imports page also lists the person's own past imports in the current account, each opening at the step it has reached. The app also registers each CSV a person can download, and writes an exporter class that names the columns the file can hold, in order, and the records it holds one row each for. An export is either fixed or chosen. A fixed export always holds every declared column, and a person downloads it from the exports page. A chosen export lets people in the current account save named templates, each a picked list of the exporter's columns in a picked order, and a person downloads a template from the exports page to get a file holding only those columns. Fire this local when a developer wants people to load one of the app's records from a spreadsheet or download them as one, wants people to pick which columns an export holds, wants to link to an import's upload page, the exports page, an export's download or a template's download, asks what the upload, column-matching, preview, result, exports or template pages do, asks what the list of past imports shows, asks what running an import saves, what happens when a run fails, or what an export's or template's file holds, wants imports run in a background job of the app's own, or when the app fails to boot with `Ixport::MissingImporter` or `Ixport::MissingExporter`.
 
 ## Interface
 
 - `Ixport.register_import` — `Ixport.register_import(key, title:, importer:, allow: nil)` registers one mapped import under a key, with the title a person sees, the name of its importer class, and optionally an access rule that decides which people may use it.
 - `Ixport::Importer` — the base class every importer inherits from.
 - `field` — `field key, label:` is called in an importer's class body and declares one field a CSV column can be mapped to, with the label a person sees.
-- `match_on` — `match_on key` is called in an importer's class body and names the one declared field a row is matched to an existing record on.
+- `match_on` — `match_on key` or `match_on key, key, ...` is called in an importer's class body and names the declared field or fields a row is matched to an existing record on, where a row matches a record only when it equals the record on every one of them.
 - `records` — an instance method every importer and every exporter defines: in an importer it returns the Active Record relation a row may update a record from and adds a new record to, and in an exporter it returns the records the CSV holds, one row each, in the order returned.
 - `person` — returns the signed-in person, for use inside `records` and `value`: in an importer the person who uploaded the file, in an exporter the person downloading it.
 - `account` — returns the current account, for use inside `records` and `value`: in an importer the account current when the file was uploaded, in an exporter the account current when the file is downloaded, and `nil` when the app has no current account.
@@ -56,7 +56,7 @@ Steps 1 to 24 add a mapped import, and steps 19 to 23 of those cover running imp
 
 2. Ask the developer which of the app's records the import loads, such as products or customers. Do not pick one.
 3. Ask the developer which attributes of that record a person may import, and the label a person should see for each, such as "SKU" for `sku` or "Price" for `price_cents`. Do not decide which attributes are importable.
-4. Ask the developer which of those fields identifies an existing record, such as the SKU or the email. Do not pick the match field.
+4. Ask the developer which of those fields, alone or together, identify an existing record, such as the SKU, or a transaction's date, amount and name. Do not pick the match fields.
 5. Ask the developer which records a person may add to or update, such as the current account's products or the signed-in person's contacts. Do not pick the scope. An import that reaches every row of the table lets one account's file update another account's records.
 6. Write the importer class in `app/importers/`, named after the record, inheriting from `Ixport::Importer`:
    ```ruby
@@ -73,7 +73,8 @@ Steps 1 to 24 add a mapped import, and steps 19 to 23 of those cover running imp
    - Call `field` once per attribute from step 3, in the order a person should see them. The key is the record's attribute name as a symbol, and `label:` is required.
    - Each field key must be an attribute the record can be assigned, because a row's mapped values are assigned to the record by those keys.
    - The labels are the choices on the column-matching page, in the order the fields are declared.
-   - Call `match_on` once, with the key of a field declared with `field`, from step 4.
+   - Call `match_on` once, with the key of every match field from step 4, each one a field declared with `field`, such as `match_on :sku` or `match_on :happened_on, :amount, :name`.
+   - Each match field must be a column of the record's table, because the match is looked up with a database query.
    - Define `records` to return a relation of the record from step 5, built from `person` or `account`. A new record is built from this relation, so a record added through `account.products` already belongs to that account.
    - `account` is `nil` when the app has no current account, so a `records` that calls a method on `account` fails in that case. Ask the developer whether that can happen in this app.
    - Never define `initialize` in an importer. ixport builds the importer with the import's person and account.
@@ -113,10 +114,12 @@ Steps 1 to 24 add a mapped import, and steps 19 to 23 of those cover running imp
     - the file has more data rows than the inline row limit, not counting the header row: "This file has N rows, and the most an import can have is L." The limit is set by `ixport-install`, and it applies only when imports run within the request. With a background runner, a file of any length is accepted.
 13. After an upload is accepted, the person is on the "Match columns" page. Each column header from the file's first row is listed with a select of the importer's field labels. "Skip" leaves that column unmapped. Pressing "Save mapping" saves the choices and takes the person to the "Preview" page.
 14. The preview decides each data row's outcome this way:
-    - The row's value for the match field is looked up in `records`, with one query for the whole file.
-    - A row matches a record only when its value is exactly the same text as the record's match attribute written as text.
-    - A row that matches a record will update it with the row's mapped values.
-    - A row that matches nothing, a row whose match value is blank, and every row of a file with no column mapped to the match field will add a new record built from `records`.
+    - The rows' values for the match fields are looked up in `records`, with one query for the whole file.
+    - Each match value from the file is first converted to the type of the record's attribute, so "12.50" matches a decimal 12.5, "100.0" matches an integer 100, and "2026-10-10" matches that date. A text attribute still matches only the same text, with its case and spaces as written.
+    - A row matches a record only when its converted values equal the record's on every match field.
+    - A row that matches a record will update it with the row's mapped values. When several records equal the row on every match field, it updates only one of them, so tell the developer the match fields must identify one record.
+    - A row that matches nothing, a row with a blank value in any match field, and every row of a file with no column mapped to one of the match fields will add a new record built from `records`.
+    - A value of false for a true-or-false match field counts as blank, so that row adds a record.
     - A row whose added or updated record is not valid will be refused, with the record's own validation messages as the reason.
 15. The preview page shows three counts, "Will add", "Will update" and "Will be refused". Under them it lists each row that will update a record under "Will update", and each refused row with its reason under "Will be refused", each by its row number in the file, where the header row is row 1. The refusal reasons are the record's validation messages, so tell the developer that the record's validations decide which rows are refused and what a person reads.
 16. Viewing the preview adds and changes no record. Its back link returns the person to the "Match columns" page to change the mapping.
@@ -277,7 +280,7 @@ Whether imports run within the request, through wallflower, or through the app's
 
 - Every registration names an importer or exporter class that exists. The app refuses to boot otherwise, so never register an import or export before its class is written.
 - Each import key is registered once, and each export key is registered once. Registering the same key a second time replaces the first registration.
-- Every importer declares `match_on` and defines `records`.
+- Every importer declares `match_on` with at least one field and defines `records`.
 - Every exporter declares at least one `column` and defines `records`.
 - `records` is always scoped to the `person` or `account` the developer chose, never left to the whole table without the developer saying so.
 - An importer inherits from `Ixport::Importer` directly and declares all of its own fields and its own match field. An exporter inherits from `Ixport::Exporter` directly and declares all of its own columns. None of these is inherited by a class that inherits from another importer or exporter.
@@ -286,7 +289,7 @@ Whether imports run within the request, through wallflower, or through the app's
 - When an importer or exporter class is renamed, update its `importer:` or `exporter:` string in `config/initializers/ixport.rb` in the same change.
 - When an import's or export's key is changed, update every link to its upload page or download in the same change.
 - Past imports record their import's key. Before changing an import's key or removing its registration, tell the developer that past imports under the old key drop off "Your imports" on the engine's root page, and that opening one of their pages raises an error, and ask what should happen to those imports.
-- Who may use an import or export is decided only by its `allow:` rule. Never write a before action, a policy check or a redirect in front of the engine's pages.
+- Every engine page first runs the before actions of the app controller ixport's pages inherit from, which is set by `ixport-install`, so a person that controller turns away never reaches an import or export. Among the people it lets through, who may use an import or export is decided only by its `allow:` rule. Never write a before action, a policy check or a redirect in front of the engine's pages.
 - When an import or export has an `allow:` rule, every link to it from the app's own pages is shown only to people the same rule allows.
 - Saved templates record a chosen export's key and its picked column keys. Before changing a chosen export's key, removing its registration, or changing it to fixed, ask the developer what happens to the templates already saved for it:
   - A template whose export key is no longer registered no longer shows on the exports page, and opening its download or edit page raises an error.
@@ -296,4 +299,4 @@ Whether imports run within the request, through wallflower, or through the app's
 - The app's own runner only starts a job, and the job only calls `Ixport.run_import`. Never write a job that reads the file, matches rows or saves records itself.
 - When the app's own runner class is renamed, tell the developer that the class name `ixport-install` set must be changed in the same change.
 - Never wrap `Ixport.run_import` in a rescue, a transaction or a retry of the app's own. It records its own failure, keeps the rows saved before it, and reports the error.
-- Installing ixport, running the update generator for the export templates table, the background runner's column or the failed import's error column, subscribing an error service to Rails' error reporter, mounting the engine, the tables an import, its refused rows and export templates are stored in, file storage, sign-in, how the current person and account are found, the layout, the inline row limit, and choosing whether imports run within the request, through wallflower or through the app's own runner class are out of scope for this local and belong to `ixport-install`.
+- Installing ixport, running the update generator for the export templates table, the background runner's column or the failed import's error column, subscribing an error service to Rails' error reporter, mounting the engine, the tables an import, its refused rows and export templates are stored in, file storage, sign-in, how the current person and account are found, the layout, the inline row limit, which of the app's controllers the engine's pages inherit from, and choosing whether imports run within the request, through wallflower or through the app's own runner class are out of scope for this local and belong to `ixport-install`.

data/the_local/agents/ixport-info.md

@@ -34,20 +34,20 @@ Every export file's first row holds the column labels, and each later row holds
 
 By default every signed-in person can use every import and export. The host can give any import or export an access rule, which is handed the signed-in person and the current account and allows or refuses them, and which can ask the app's own permission checks, such as a Pundit policy. A person the rule refuses does not see that import or export listed, does not see their past imports of it, and is answered not found on every one of its pages and downloads, including its upload page, an existing import's mapping, preview, run and result pages, and a chosen export's template pages and downloads. The rule is checked again on every request, so a person whose permissions change is let in or kept out from their next page.
 
-The engine owns the screens, the access check, and its own tables of import records, refused rows and export templates, with the uploaded file stored through Active Storage. The host owns what each import and export means: which columns it reads or writes, which column identifies an existing record, and which of the app's records a row is checked against, saved into, or read from. Reach for ixport when a Rails app needs people to load data from spreadsheets or take it out into one, rather than writing a one-off upload or download page per record type.
+The engine owns the screens, the access check, and its own tables of import records, refused rows and export templates, with the uploaded file stored through Active Storage. The host owns what each import and export means: which columns it reads or writes, which column or columns identify an existing record, and which of the app's records a row is checked against, saved into, or read from. Reach for ixport when a Rails app needs people to load data from spreadsheets or take it out into one, rather than writing a one-off upload or download page per record type.
 
-The engine renders its pages with Keystone UI inside a layout the host chooses, and it runs its access check through a method the host already has.
+The engine renders its pages with Keystone UI inside a layout the host chooses, and it runs its access check through a method the host already has. Its pages inherit from a controller the host names, which is the host's own application controller unless the host names another, so the filters, helpers and error handling the host puts on that controller apply to ixport's pages as well.
 
 ## Interface
 
 ixport declares nothing for this local to document. Its surface is split between the other two locals:
 
-- **ixport-install** owns hooking ixport into a host app: adding the gem, running its install generator, running its update generator to bring an existing install's tables up to date, mounting the engine, and the configuration that names the host's sign-in check, current person, current account, layout, row limit and background runner.
+- **ixport-install** owns hooking ixport into a host app: adding the gem, running its install generator, running its update generator to bring an existing install's tables up to date, mounting the engine, and the configuration that names the host's sign-in check, current person, current account, layout, row limit, background runner and the controller ixport's pages inherit from.
 - **ixport-develop** owns building against ixport: registering a mapped import and the importer class behind it, registering a fixed or chosen export and the exporter class behind it, giving an import or export an access rule, writing the app's own background runner and running an import from it, the error a wallflower task raises when its import fails, the errors raised when a registration names a class that does not exist, and the engine's pages for imports, exports and export templates.
 
 ## How to use it
 
-- The app does not have ixport yet, the app installed ixport before export templates or background runs existed and needs their tables, you are choosing whether imports run inside the request, through wallflower or through the app's own runner, or you are changing how ixport reaches the app's sign-in, people, accounts, layout or row limit: use **ixport-install**.
+- The app does not have ixport yet, the app installed ixport before export templates or background runs existed and needs their tables, you are choosing whether imports run inside the request, through wallflower or through the app's own runner, or you are changing how ixport reaches the app's sign-in, people, accounts, layout or row limit, or which of the app's controllers its pages inherit from: use **ixport-install**.
 - ixport is already installed and you are adding or changing an import or an export, making an export chosen or fixed, changing how an import's rows are matched, checked or saved, changing which records or columns an export holds, limiting an import or export to some people, writing the app's own runner that runs imports in the background, linking to the engine's pages, or the app fails to boot naming a missing importer or exporter: use **ixport-develop**.
 
 ## Conventions
@@ -59,11 +59,12 @@ ixport declares nothing for this local to document. Its surface is split between
 - **Picked columns** — the columns a template holds, in the order they were chosen. A slot left on None is dropped, a column picked twice is kept once, and a column the exporter no longer declares is left out of the download.
 - **Key** — the short name an import or export is registered and looked up by, held as a symbol. A fixed export's file is named after its key.
 - **Title** — the words a person sees for the import or export on the engine's pages.
-- **Importer** — a class the host app writes that declares the columns the import reads, each with a key and a label, names the column that identifies an existing record, and supplies the set of the app's records the rows are checked against and saved into.
+- **Importer** — a class the host app writes that declares the columns the import reads, each with a key and a label, names the column or columns that together identify an existing record, and supplies the set of the app's records the rows are checked against and saved into.
 - **Exporter** — a class the host app writes that declares the columns the export writes, each with a key and a label, and supplies the set of the app's records the file holds.
 - **Export column** — one column of an export's file, headed by its label and filled from the attribute its key names on each record.
 - **Through** — an export column can be read from a record related to each row's record rather than from the record itself, and it is left blank when that related record is missing.
-- **Match column** — the one declared import column whose value decides whether a row is an existing record or a new one.
+- **Match column** — a declared import column whose value decides whether a row is an existing record or a new one. An import can have one or several, and a row is an existing record only when it equals that record on every one of them.
+- **Matched value** — a row's value in a match column, compared as the record's own type rather than as text, so 100.0 in a file matches 100 stored on a record.
 - **Import record** — one upload by one person, stored by the engine with the key of its mapped import, the uploaded file, the mapping once it is saved, and the added and updated counts once it has run.
 - **Headers** — the first row of the uploaded file, read as the names of its columns.
 - **Mapping** — which of the importer's columns each header in the file fills, stored by the header's position in the row. A header left on "Skip" is not stored.
@@ -71,8 +72,8 @@ ixport declares nothing for this local to document. Its surface is split between
 - **Background runner** — what runs an import outside the request: none, wallflower, or a class the host app writes. Having wallflower in the app does not choose it, the host has to.
 - **Wallflower** — a separate gem that runs long tasks in the background and gives each task a page showing its progress. ixport registers its imports with it as a kind of task titled Import.
 - **Row number** — a row's line in the file, counting the header as row 1, so the first data row is row 2.
-- **Add** — a row whose match column value is found in no existing record, so it creates one.
-- **Update** — a row whose match column value is found in an existing record, so it changes that record.
+- **Add** — a row that no existing record equals on every match column, or that is blank in any match column, or comes from a file with no column mapped to one, so it creates a record.
+- **Update** — a row that equals an existing record on every match column, so it changes that record.
 - **Refused** — a row whose record, new or changed, fails the host model's own validations. Its reason is that model's error messages.
 - **Preview** — the page that shows the add, update and refused counts and lists the update and refused rows by row number. It builds each row's record in memory and saves none of them.
 - **Run** — the step that saves the added and updated records one row at a time and stores each refused row's number and reason. A run that raises an error stops there and keeps the rows it already saved, and an import that has finished or failed is never run a second time.
@@ -85,6 +86,7 @@ ixport declares nothing for this local to document. Its surface is split between
 - **Step** — how far an import record has got: mapping when no mapping is saved, preview once one is, and result once it is running, finished or failed.
 - **Result page** — the import's own page, showing the added and updated counts and each refused row as they were stored by the run, and the error's message when the run failed, so it shows the same thing each time it is opened.
 - **Person** — whoever is signed in and running the import or downloading the export. ixport reaches them through a method the host names, so the host's user model needs no particular name.
+- **Parent controller** — the host app's controller that every ixport page inherits from, the app's application controller by default. Whatever it adds to every request, such as a filter, a helper or a rescue, runs on ixport's pages too, and the sign-in check, current person and current account methods ixport calls are looked up on it.
 - **Account** — the optional owner the person is working inside, such as a team or company, reached the same way. An app with no accounts runs without one. Both the importer and the exporter are given the person and the account.
 - **Ownership** — an import record belongs to the person and account that uploaded it, and anyone else who opens it, including the same person from another account, is answered not found. An export template belongs to its account, and anyone opening it from another account is answered not found.
 - **Access rule** — an optional check the host gives one import or export when registering it, handed the person and the account and answering whether they may use it. It is checked on top of ownership, so even the person who uploaded an import is answered not found for it once the rule refuses them. An import or export with no rule is open to every signed-in person.

data/the_local/agents/ixport-install.md

@@ -1,6 +1,6 @@
 ---
 name: ixport-install
-description: Use to hook ixport into a project — adding the gem, checking Active Storage is set up, running the install generator and its migrations, running the update generator on an app that installed an earlier ixport, mounting the engine, and configuring the sign-in check, the current person method, the current account method, the layout, the inline row limit and the background runner.
+description: Use to hook ixport into a project — adding the gem, checking Active Storage is set up, running the install generator and its migrations, running the update generator on an app that installed an earlier ixport, mounting the engine, and configuring the sign-in check, the current person method, the current account method, the layout, the inline row limit, the background runner and the base controller.
 tools: Bash, Read, Edit
 scope: CSV import and export — a mounted engine where a signed-in person sees the mapped imports and the exports the host app registers, each import backed by an importer class and each export by an exporter class the host writes, where an export is either fixed or chosen, and a chosen export lets the account save named templates of picked columns
 ---
@@ -17,13 +17,14 @@ ixport is a Rails engine with two pages of work for a signed-in person. On the i
 - `bin/rails generate ixport:install` — copies five migrations into `db/migrate/`: one that creates the `ixport_imports` table, one that creates the `ixport_import_refusals` table, one that creates the `ixport_export_templates` table, one that adds a `wallflower_task_id` column to `ixport_imports`, and one that adds an `error_message` column to `ixport_imports`.
 - `bin/rails generate ixport:update` — copies the three migrations added since ixport first shipped, the one that creates the `ixport_export_templates` table, the one that adds the `wallflower_task_id` column and the one that adds the `error_message` column, into an app that installed an earlier ixport.
 - `mount Ixport::Engine` — the line in `config/routes.rb` that serves ixport's pages under a path the app chooses. The imports page is at that path and the exports page is at that path followed by `/exports`.
-- `Ixport.configure` — takes a block that receives the configuration object, where the app sets any of the six settings below.
+- `Ixport.configure` — takes a block that receives the configuration object, where the app sets any of the seven settings below.
 - `config.authentication_method` — the name of the controller method run before every ixport page, imports, exports and export templates alike, as a symbol. It defaults to `:authenticate_user!`.
 - `config.current_person_method` — the name of the controller method that returns the signed-in person running imports and exports and saving export templates, as a symbol. It defaults to `:current_user`.
 - `config.current_account_method` — the name of the controller method that returns the account the person is working in, as a symbol. It defaults to `:current_account`.
 - `config.layout` — the name of the host layout ixport's pages are drawn inside, as a string. It defaults to `"application"`.
 - `config.inline_row_limit` — the most data rows an uploaded import file may have, not counting its header row, as an integer. A file with more is refused on the upload page with a message giving the limit and the file's row count. It applies only while `config.background_runner` is `nil`, and it does not limit exports. It defaults to `1_000`.
 - `config.background_runner` — what runs an import once the person starts it. It defaults to `nil`, which runs the import inside the request. `:wallflower` runs every import as a wallflower task. The name of a class in the app, as a string, hands every import to that class. With any value other than `nil`, every import runs through the runner whatever its size, and the inline row limit does not apply.
+- `config.base_controller` — the name of the host controller ixport's controllers inherit from, as a string. Every ixport page runs that controller's own before actions first, then the sign-in check, and can call that controller's methods. It defaults to `"::ApplicationController"`.
 
 ## How to use it
 
@@ -59,11 +60,11 @@ ixport is a Rails engine with two pages of work for a signed-in person. On the i
    mount Ixport::Engine => "/data_imports"
    ```
    If the app's routes put signed-in pages inside a block such as `authenticate :user do`, ask the developer whether the mount goes inside it. Do not give the mount a name with `as:`. A wallflower task links to the import's result page through the route name `ixport`, so the mount keeps that name. `bin/rails routes` lists the mount with `ixport` in its name column.
-8. Read the app's `ApplicationController` and the controller methods it gets from its sign-in setup. ixport's controllers inherit from the app's `ApplicationController`, so every method named in the next step must be callable from it.
+8. Ask the developer which controller ixport's pages should inherit from, and do not pick for them. The default is the app's `ApplicationController`. Where ixport's pages belong inside an area of the app with its own base controller, such as an admin area whose controller limits who may enter, naming that controller gives ixport's pages the same checks. Then read the chosen controller and the controller methods it gets from its sign-in setup, since every method named in the next step must be callable from it.
 9. Compare the app's method and layout names with the defaults in the Interface section. Every default matches a Jumpstart app, so an app that uses all of the defaults needs no configuration. Where any name differs, or a method does not exist, ask the developer which method or layout to use rather than picking one:
    - The sign-in check: the method that sends a person who is not signed in to sign in, such as `:authenticate_user!` or `:require_login`.
    - The current person: the method returning the signed-in person, such as `:current_user` or `:current_member`. An import belongs to this person, and another person who opens it gets a not found response. The imports page lists only the imports this person uploaded. Each export is built for this person when it is downloaded. An export template records this person as the one who saved it. Any access rule the app gives an import or export is handed this person.
-   - The current account: the method returning the team or company the person works inside, such as `:current_account` or `:current_team`. An import belongs to this account too, and the same person who opens it from another account gets a not found response. The imports page lists only the person's imports uploaded in this account. Each export is built for this account when it is downloaded. An export template belongs to this account, and every person in the account can see, download from, edit and delete it, while a person in another account gets a not found response. Any access rule the app gives an import or export is handed this account. Ask whether the app has accounts at all. ixport calls the named method on every ixport page, so an app with no accounts still needs a method of that name callable from `ApplicationController`, and it returns `nil`. In an app whose method returns `nil`, every person shares every export template, and every access rule is handed `nil` as the account. Tell the developer this and ask before adding the method.
+   - The current account: the method returning the team or company the person works inside, such as `:current_account` or `:current_team`. An import belongs to this account too, and the same person who opens it from another account gets a not found response. The imports page lists only the person's imports uploaded in this account. Each export is built for this account when it is downloaded. An export template belongs to this account, and every person in the account can see, download from, edit and delete it, while a person in another account gets a not found response. Any access rule the app gives an import or export is handed this account. Ask whether the app has accounts at all. ixport calls the named method on every ixport page, so an app with no accounts still needs a method of that name callable from the base controller chosen in step 8, and it returns `nil`. In an app whose method returns `nil`, every person shares every export template, and every access rule is handed `nil` as the account. Tell the developer this and ask before adding the method.
    - The layout: the file in `app/views/layouts/` that ixport's pages should be drawn inside.
 10. Ask the developer how imports should run, and do not pick for them. Having wallflower in the `Gemfile` does not choose it. The three answers:
     - Inside the request: leave `config.background_runner` out. Each import file is limited to the inline row limit, and the person waits on the run page until the import finishes.
@@ -84,6 +85,7 @@ ixport is a Rails engine with two pages of work for a signed-in person. On the i
       config.layout = "back_office"
       config.inline_row_limit = 5_000
       config.background_runner = :wallflower
+      config.base_controller = "BackOfficeController"
     end
     ```
     The values above are examples, and each line is written only when the developer's answer differs from the default. Ask about `config.inline_row_limit` only when imports run inside the request and the developer wants a limit other than 1,000 rows per import file, since it has no effect with a background runner.
@@ -93,14 +95,15 @@ ixport is a Rails engine with two pages of work for a signed-in person. On the i
     bin/rails generate ixport:update
     bin/rails db:migrate
     ```
-    Until the `error_message` migration has run, an import that fails cannot record its error and the run raises an error naming the missing column. The generator copies whichever of the three migrations step 5 describes are missing, reports each one already present as identical, and touches no other file. Then read the app's `config/initializers/ixport.rb`, or the defaults if it has none. Where the export templates migration was just added, put the current account part of step 9 to the developer, since export templates are shared by that account. Where `config.background_runner` is not set, put step 10 to the developer. Visit the mounted path followed by `/exports` while signed in and check that the page still loads. Then visit the mounted path itself and check that the imports page loads, with a "Your imports" list of the person's earlier imports when they have any. An earlier import of a kind whose access rule refuses the signed-in person is left off that list. Access rules need no migration and no setting.
+    Until the `error_message` migration has run, an import that fails cannot record its error and the run raises an error naming the missing column. The generator copies whichever of the three migrations step 5 describes are missing, reports each one already present as identical, and touches no other file. Then read the app's `config/initializers/ixport.rb`, or the defaults if it has none. Where the export templates migration was just added, put the current account part of step 9 to the developer, since export templates are shared by that account. Where `config.background_runner` is not set, put step 10 to the developer. Where `config.base_controller` is not set, put step 8 to the developer. Visit the mounted path followed by `/exports` while signed in and check that the page still loads. Then visit the mounted path itself and check that the imports page loads, with a "Your imports" list of the person's earlier imports when they have any. An earlier import of a kind whose access rule refuses the signed-in person is left off that list. Access rules need no migration and no setting.
 
 ## Conventions
 
 - After installing or updating, check that `bin/rails db:migrate:status` lists the create-ixport-imports, create-ixport-import-refusals, create-ixport-export-templates, add-wallflower-task-to-ixport-imports and add-error-message-to-ixport-imports migrations as up, that `bin/rails routes` lists the mount path, and that both pages load as described in step 12. With `config.background_runner = :wallflower`, also check that `bin/rails routes` lists ixport's mount under the name `ixport` and wallflower's mount under the name `wallflower`.
 - Run the install generator once, and the update generator only on an app installed with an earlier ixport. A second run of either against unchanged copies reports each migration as identical and adds nothing. If a copied migration differs from the one the generator would write, the run stops with an error naming that migration, and copies nothing after it. Do not delete or edit a copied migration once it has been run.
 - An import that stops on an error is marked failed, keeps the rows it saved before stopping, and reports the error through Rails' error reporter, so it reaches whatever error service the app has subscribed there. ixport needs no setting for this, and an app with no subscriber records the error only on the import's page.
-- When the app renames its sign-in method, its current person or account method, the layout ixport uses, or its own runner class, update the matching line in `config/initializers/ixport.rb` in the same change.
+- Set `config.base_controller` only in `config/initializers/ixport.rb`. ixport reads it once, when its controllers load, so a value set anywhere later has no effect. A name that is not a controller in the app raises a `NameError` naming it when ixport's controllers load.
+- When the app renames its sign-in method, its current person or account method, the layout ixport uses, its own runner class, or the base controller ixport inherits from, update the matching line in `config/initializers/ixport.rb` in the same change.
 - A setting given the same value as its default is left out of the initializer. Remove a line when its value goes back to the default.
 - If the app fails to boot after installing, with an error saying an ixport import or export names a class that does not exist, the app registered an import or export whose class is missing. Fixing it is out of scope for this local and belongs to `ixport-develop`.
 - Every signed-in person who passes the sign-in check can use every import and export until the app gives one an access rule. Ask the developer whether some imports or exports should be limited to some people, and hand that work to `ixport-develop`.

data/the_local/interface.yml

@@ -12,6 +12,7 @@ install:
   - config.layout
   - config.inline_row_limit
   - config.background_runner
+  - config.base_controller
 
 develop:
   - Ixport.register_import

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ixport
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.1
 platform: ruby
 authors:
 - Tyler Schneider