data_porter 1.0.2 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7ca6bfabfc9f831d71c60a1942516a5dccf95c85e3787f16a1217188c9feb3a0
-  data.tar.gz: f703da9261612953fcacad2674e38bef3037b804191e7fb3087577846b096461
+  metadata.gz: '0439ba2634bbf47987015524141c3f961b54c362d234dcaeecc7811895c4ec34'
+  data.tar.gz: b4b2daf3519743518b2119b75863ca83114db258599fa2026e1b7a2ae844668e
 SHA512:
-  metadata.gz: a7d8ad32cb5d80e027d9adfe2a089a84703c6e8e5b00901c0d057a4b2bb24cb2ffbe0f6edb53f61021219fd03384830517192657fbe4c693dd74b6977279b22a
-  data.tar.gz: 6d96ecefa39d191cea801ff8e4075f5c9bb4e13979f7754041db33a6685701d98f4521230500a1d0a47a417ce141f1b08973b5cbfd65868b0c3920c99afb61f6
+  metadata.gz: b2ddc41b18ab043cbabb97518594ec27a0d1ec4ea0f4f1e542a6dde664c6914d6d1b61c609e6436d9a9bd741298c31766c623886eb3964a417b9a5ebec532b44
+  data.tar.gz: 2fe7cc8d5910005f86b9198db007afc406bb186d933d0ff1a5e4c147fab191ade9386550fdbb5b85f4756c6795da979f81da5b4681782c027f7901763f1dd972

data/CHANGELOG.md

@@ -5,6 +5,36 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.0.0] - 2026-02-19
+
+### Breaking
+
+- **Default parent controller changed to `ActionController::Base`** -- Engine controllers no longer inherit from `ApplicationController` by default, avoiding conflicts with authorization gems (Pundit, CanCanCan, etc.). Set `config.parent_controller = "ApplicationController"` to restore the previous behavior
+- **Engine routes mounted at root** -- `resources :imports` now uses `path: "/"` so the mount point controls the full URL (e.g. `mount DataPorter::Engine, at: "/imports"` gives `/imports`, not `/imports/imports`)
+
+### Added
+
+- **Back to mapping** -- `back_to_mapping` action resets a previewing import to the mapping step, preserving file headers and column mapping for re-mapping
+- **Saved mapping persistence** -- Mapping form restores previously saved column mapping instead of resetting to defaults
+
+### Changed
+
+- 423 RSpec examples (up from 413), 0 failures
+
+## [1.1.0] - 2026-02-08
+
+### Added
+
+- **Scoped imports** -- `config.scope` lambda returns the owner object (any ActiveRecord model) for multi-tenant isolation; used for both storage and filtering; IDOR-safe
+- **Scoped mapping templates** -- Templates are filtered and assigned by owner, consistent with import scope
+- **`ScopeManagement` concern** -- Shared `resolve_owner` logic extracted for both controllers
+- **Polymorphic user on `mapping_templates`** -- Migration template adds `user` reference for template ownership
+
+### Changed
+
+- `build_import` uses `resolve_owner` instead of raw `current_user`
+- 413 RSpec examples (up from 402), 0 failures
+
 ## [1.0.2] - 2026-02-07
 
 ### Changed

data/README.md

@@ -1,23 +1,10 @@
 # DataPorter
 
-> [!CAUTION]
-> **This gem is under active development and not yet production-ready.**
-> APIs and features may change without notice. Use at your own risk.
-
 A mountable Rails engine for data import workflows: **Upload**, **Map**, **Preview**, **Import**.
 
 Supports CSV, JSON, XLSX, and API sources with a declarative DSL for defining import targets. Business-agnostic by design -- all domain logic lives in your host app.
 
-<table>
-  <tr>
-    <td><img src="docs/screenshots/index-with-previewing.jpg" width="400" alt="Import list with status badges" /></td>
-    <td><img src="docs/screenshots/modal-new-import.jpg" width="400" alt="New import modal with dropzone" /></td>
-  </tr>
-  <tr>
-    <td><img src="docs/screenshots/mapping.jpg" width="400" alt="Interactive column mapping with templates" /></td>
-    <td><img src="docs/screenshots/preview.jpg" width="400" alt="Preview with summary cards and data table" /></td>
-  </tr>
-</table>
+![DataPorter demo](docs/screenshots/demo_fast.gif)
 
 ## Features
 
@@ -31,7 +18,8 @@ Supports CSV, JSON, XLSX, and API sources with a declarative DSL for defining im
 - **Per-target source filtering** -- Each target declares its allowed sources, the UI filters accordingly
 - **Import deletion & auto-purge** -- Delete imports from the UI, or schedule `rake data_porter:purge` for automatic cleanup
 - **Reject rows export** -- Download a CSV of failed/errored records with error messages after import
-- **Security validations** -- File size limit, MIME type check, strong parameter whitelisting
+- **Scoped imports** -- `config.scope` for multi-tenant isolation; each user only sees their own imports
+- **Security validations** -- File size limit, MIME type check, strong parameter whitelisting, IDOR protection via scope
 - **Safety guards** -- Max records limit (`config.max_records`), configurable transaction mode (`:per_record` or `:all`)
 - **Declarative Target DSL** -- One class per import type, zero boilerplate ([docs](docs/TARGETS.md))
 
@@ -141,7 +129,7 @@ pending -> parsing -> previewing -> importing -> completed
 git clone https://github.com/SerylLns/data_porter.git
 cd data_porter
 bin/setup
-bundle exec rspec     # 391 specs
+bundle exec rspec     # 413 specs
 bundle exec rubocop   # 0 offenses
 ```
 

data/app/controllers/data_porter/concerns/mapping_management.rb

@@ -12,14 +12,22 @@ module DataPorter
         columns = target._columns || []
         @file_headers = @import.config["file_headers"] || []
         @target_columns = columns.map { |c| [c.label, c.name.to_s, c.required] }
-        @default_mapping = (target._csv_mappings || {}).transform_values(&:to_s)
+        @default_mapping = saved_or_default_mapping(target)
         @templates = load_templates
       end
 
       def load_templates
         return [] unless defined?(DataPorter::MappingTemplate)
 
-        DataPorter::MappingTemplate.for_target(@import.target_key)
+        scope = scoped_template_base
+        scope.for_target(@import.target_key)
+      end
+
+      def saved_or_default_mapping(target)
+        saved = @import.config&.dig("column_mapping")
+        return saved if saved.present?
+
+        (target._csv_mappings || {}).transform_values(&:to_s)
       end
 
       def save_column_mapping
@@ -31,10 +39,19 @@ module DataPorter
         return unless params[:save_template] == "1"
         return unless defined?(DataPorter::MappingTemplate)
 
-        DataPorter::MappingTemplate.find_or_initialize_by(
+        template = scoped_template_base.find_or_initialize_by(
           target_key: @import.target_key,
           name: params[:template_name].presence || "Default"
-        ).update!(mapping: permitted_column_mapping)
+        )
+        template.user ||= resolve_owner
+        template.update!(mapping: permitted_column_mapping)
+      end
+
+      def scoped_template_base
+        owner = resolve_owner
+        return DataPorter::MappingTemplate unless owner
+
+        DataPorter::MappingTemplate.where(user: owner)
       end
 
       def permitted_column_mapping

data/app/controllers/data_porter/concerns/scope_management.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module DataPorter
+  module Concerns
+    module ScopeManagement
+      private
+
+      def resolve_owner
+        return unless respond_to?(:current_user, true)
+        return unless current_user
+
+        scope = DataPorter.configuration.scope
+        scope ? scope.call(current_user) : current_user
+      end
+    end
+  end
+end

data/app/controllers/data_porter/imports_controller.rb

@@ -5,14 +5,16 @@ module DataPorter
     include Concerns::ImportValidation
     include Concerns::MappingManagement
     include Concerns::RecordPagination
+    include Concerns::ScopeManagement
 
     layout "data_porter/application"
 
-    before_action :set_import, only: %i[show parse confirm cancel dry_run update_mapping status export_rejects destroy]
+    before_action :set_import, only: %i[show parse confirm cancel dry_run update_mapping
+                                        status export_rejects destroy back_to_mapping]
     before_action :load_targets, only: %i[index new create]
 
     def index
-      @imports = DataPorter::DataImport.order(created_at: :desc)
+      @imports = scoped_imports.order(created_at: :desc)
     end
 
     def new
@@ -62,6 +64,11 @@ module DataPorter
       redirect_to imports_path
     end
 
+    def back_to_mapping
+      @import.reset_to_mapping!
+      redirect_to import_path(@import)
+    end
+
     def dry_run
       @import.update!(status: :pending)
       DataPorter::DryRunJob.perform_later(@import.id)
@@ -88,7 +95,14 @@ module DataPorter
     private
 
     def set_import
-      @import = DataPorter::DataImport.find(params[:id])
+      @import = scoped_imports.find(params[:id])
+    end
+
+    def scoped_imports
+      owner = resolve_owner
+      return DataPorter::DataImport.all unless owner
+
+      DataPorter::DataImport.where(user: owner)
     end
 
     def load_targets
@@ -97,7 +111,7 @@ module DataPorter
 
     def build_import
       @import = DataPorter::DataImport.new(import_params)
-      @import.user = current_user if respond_to?(:current_user, true)
+      @import.user = resolve_owner
       @import.status = :pending
     end
 

data/app/controllers/data_porter/mapping_templates_controller.rb

@@ -2,12 +2,14 @@
 
 module DataPorter
   class MappingTemplatesController < DataPorter.configuration.parent_controller.constantize
+    include Concerns::ScopeManagement
+
     layout "data_porter/application"
 
     before_action :set_template, only: %i[edit update destroy]
 
     def index
-      @templates = MappingTemplate.order(:target_key, :name)
+      @templates = scoped_templates.order(:target_key, :name)
       @grouped = @templates.group_by(&:target_key)
     end
 
@@ -18,6 +20,7 @@ module DataPorter
 
     def create
       @template = MappingTemplate.new(template_params)
+      @template.user = resolve_owner
 
       if @template.save
         redirect_to mapping_templates_path
@@ -48,7 +51,14 @@ module DataPorter
     private
 
     def set_template
-      @template = MappingTemplate.find(params[:id])
+      @template = scoped_templates.find(params[:id])
+    end
+
+    def scoped_templates
+      owner = resolve_owner
+      return MappingTemplate.all unless owner
+
+      MappingTemplate.where(user: owner)
     end
 
     def template_params

data/app/models/data_porter/data_import.rb

@@ -53,6 +53,15 @@ module DataPorter
       records.group_by(&:status).transform_values(&:count)
     end
 
+    def reset_to_mapping!
+      update!(
+        status: :mapping,
+        records: [],
+        report: StoreModels::Report.new,
+        config: (config || {}).except("progress")
+      )
+    end
+
     def file_based?
       %w[csv xlsx].include?(source_type)
     end

data/app/models/data_porter/mapping_template.rb

@@ -4,6 +4,8 @@ module DataPorter
   class MappingTemplate < ActiveRecord::Base
     self.table_name = "data_porter_mapping_templates"
 
+    belongs_to :user, polymorphic: true, optional: true
+
     attribute :mapping, :json, default: -> { {} }
 
     validates :target_key, presence: true

data/app/views/data_porter/imports/show.html.erb

@@ -65,6 +65,10 @@
           Dry Run
         <% end %>
       <% end %>
+      <% if @import.file_based? %>
+        <%= button_to "Back to Mapping", back_to_mapping_import_path(@import),
+              method: :post, class: "dp-btn dp-btn--secondary" %>
+      <% end %>
       <%= button_to "Cancel", cancel_import_path(@import),
             method: :post, class: "dp-btn dp-btn--danger" %>
     </div>

data/bookmarklet.md

@@ -0,0 +1,217 @@
+---
+title: "Building a Product Clipper Bookmarklet with Shadow DOM and Structured Data"
+published: false
+tags: javascript, webdev, architecture, bookmarklet
+---
+
+# Building a Product Clipper Bookmarklet with Shadow DOM and Structured Data
+
+We’re in 2008.
+
+The iPhone 3G just dropped. Facebook crosses 100 million users. Bitcoin quietly appears on a cryptography mailing list. The web is shifting.
+
+And while the world is obsessing over apps and platforms… we’re going back to something beautifully simple.
+
+**A bookmarklet.**
+
+No extension store review.  
+No packaging.  
+No deployment delays.
+
+Just a small piece of JavaScript living inside a browser bookmark — capable of injecting a clean, isolated UI into any e-commerce page, detecting product data automatically, and sending it to your backend.
+
+One click. Any product page. Instant extraction.
+
+Here’s how the architecture works.
+
+---
+
+## High-Level Architecture
+
+The clipper follows a simple execution model:
+
+1. The bookmarklet injects a remote script into the current page.
+2. The script scans the DOM using multiple detection strategies.
+3. A sidebar panel renders inside a Shadow DOM (fully isolated).
+4. Detected products are visually highlighted.
+5. The user selects items to import.
+6. Selected data is sent to the backend for processing.
+
+No browser extension. No build complexity. Just runtime execution.
+
+---
+
+## Why a Bookmarklet?
+
+For a user-triggered action ("Import this page"), a bookmarklet offers strong trade-offs:
+
+|             | Bookmarklet           | Browser Extension                |
+| ----------- | --------------------- | -------------------------------- |
+| Install     | Drag a link           | Store review required            |
+| Updates     | Instant (server-side) | Requires store re-approval       |
+| Permissions | None                  | Explicit permission prompts      |
+| Maintenance | Single hosted file    | Multi-file manifest architecture |
+
+If your tool runs only when explicitly triggered, a bookmarklet is often the leanest solution.
+
+---
+
+## Product Detection Strategy
+
+No single detection method works across all e-commerce sites.
+
+A robust clipper layers multiple strategies, ordered by confidence:
+
+- **Structured Data (JSON-LD)**  
+  Many sites expose `schema.org/Product` data for SEO.
+- **Microdata attributes**
+- **OpenGraph metadata**
+- **Heuristic DOM scanning**
+- **URL pattern matching (fallback)**
+
+The key principle is:
+
+> Prefer high-confidence structured data, then gracefully degrade.
+
+### Example: Extracting JSON-LD Products
+
+```javascript
+function extractFromJsonLd() {
+  const scripts = document.querySelectorAll(
+    'script[type="application/ld+json"]',
+  );
+  const products = [];
+
+  scripts.forEach((script) => {
+    try {
+      const data = JSON.parse(script.textContent);
+      // Traverse recursively and collect Product objects
+      collectProducts(data, products);
+    } catch (e) {}
+  });
+
+  return products;
+}
+```
+
+In practice, you normalize URLs, merge duplicates, and score sources by confidence before presenting results.
+
+The goal is reliability, not perfection.
+
+---
+
+## Shadow DOM: Isolation Is Non-Negotiable
+
+Injecting UI into arbitrary websites is dangerous.
+
+CSS resets, `!important` rules, framework styles — they will break your interface.
+
+Shadow DOM solves this by creating an isolated rendering tree:
+
+```javascript
+const host = document.createElement("div");
+document.body.appendChild(host);
+
+const shadow = host.attachShadow({ mode: "open" });
+shadow.innerHTML = `
+  <style>
+    :host { all: initial; font-family: system-ui; }
+    .panel { position: fixed; right: 0; top: 0; }
+  </style>
+  <div class="panel">Clipper UI</div>
+`;
+```
+
+Key principle:
+
+> Your UI must behave identically on Shopify, Magento, custom React apps, or legacy PHP pages.
+
+Isolation is mandatory.
+
+---
+
+## Visual Feedback & Interaction Control
+
+When products are detected, highlighting them directly on the page improves user confidence.
+
+Because many e-commerce sites attach their own click handlers (analytics, routing, SPA navigation), event handling must be carefully managed.
+
+Best practice:
+
+- Use capture phase listeners
+- Prevent unintended navigation
+- Clean up all listeners and styles on teardown
+
+A clipper should leave **zero traces** after closing.
+
+---
+
+## Backend Communication
+
+Once items are selected, the clipper sends a structured payload to your backend.
+
+The backend typically:
+
+- Normalizes URLs
+- Deduplicates products
+- Associates them with a source domain
+- Triggers downstream processing (price tracking, enrichment, etc.)
+
+Security considerations:
+
+- Use scoped, short-lived API tokens
+- Never expose sensitive credentials
+- Sanitize all extracted DOM content before rendering
+
+---
+
+## Limitations
+
+A bookmarklet runs inside the page’s context. That comes with constraints.
+
+### Content Security Policy (CSP)
+
+Strict `script-src` headers can block injected scripts entirely.
+There is no client-side workaround. A browser extension is required in those cases.
+
+### Single Page Applications (SPAs)
+
+React/Next.js apps often load content asynchronously.
+Mutation observers or delayed scans improve detection reliability.
+
+### Bot Protection (Backend)
+
+While the bookmarklet runs in the user’s real browser session, backend scraping of submitted URLs may face anti-bot systems.
+That is a server-side concern.
+
+---
+
+## Legal & Ethical Considerations
+
+If you build a commercial tool around product extraction:
+
+- Only collect publicly visible data
+- Do not bypass authentication or CAPTCHAs
+- Respect rate limits
+- Be transparent about data usage
+- Review relevant laws (CFAA, GDPR, local regulations)
+
+User-initiated clipping is typically lower risk than automated crawling, but not risk-free.
+
+---
+
+## Lessons Learned
+
+1. **Isolation first.** Shadow DOM prevents 90% of UI conflicts.
+2. **Layered detection beats single heuristics.**
+3. **Keep it framework-free.** Dependencies increase fragility.
+4. **Design for hostile environments.** You do not control the host page.
+5. **Simplicity wins.** A single hosted file can outperform complex extension architectures.
+
+---
+
+A bookmarklet is not flashy.
+
+But when designed correctly, it becomes a powerful bridge between arbitrary web pages and your product.
+
+Sometimes the most effective architecture is the one that avoids complexity entirely.

data/config/routes.rb

@@ -1,11 +1,12 @@
 # frozen_string_literal: true
 
 DataPorter::Engine.routes.draw do
-  resources :imports, only: %i[index new create show destroy] do
+  resources :imports, path: "/", only: %i[index new create show destroy] do
     member do
       post :parse
       post :confirm
       post :cancel
+      post :back_to_mapping
       post :dry_run
       patch :update_mapping
       get :status
@@ -13,5 +14,5 @@ DataPorter::Engine.routes.draw do
     end
   end
 
-  resources :mapping_templates, except: :show
+  resources :mapping_templates
 end

data/lib/data_porter/configuration.rb

@@ -16,8 +16,8 @@ module DataPorter
                   :transaction_mode
 
     def initialize
-      @parent_controller = "ApplicationController"
-      @queue_name = :imports
+      @parent_controller = "ActionController::Base"
+      @queue_name = :default
       @storage_service = :local
       @cable_channel_prefix = "data_porter"
       @context_builder = nil

data/lib/data_porter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DataPorter
-  VERSION = "1.0.2"
+  VERSION = "2.0.0"
 end

data/lib/generators/data_porter/install/templates/create_data_porter_mapping_templates.rb.erb

@@ -7,6 +7,8 @@ class CreateDataPorterMappingTemplates < ActiveRecord::Migration[<%= ActiveRecor
       t.string :name,       null: false
       t.jsonb  :mapping,    null: false, default: {}
 
+      t.references :user, polymorphic: true
+
       t.timestamps
     end
 

data/lib/generators/data_porter/install/templates/initializer.rb

@@ -2,11 +2,12 @@
 
 DataPorter.configure do |config|
   # Parent controller for the engine's controllers to inherit from.
-  # This controls authentication, layouts, and helpers.
+  # Defaults to ActionController::Base. Set to "ApplicationController" to inherit
+  # authentication, layouts, and helpers from your app.
   # config.parent_controller = "ApplicationController"
 
   # ActiveJob queue name for import jobs.
-  # config.queue_name = :imports
+  # config.queue_name = :default
 
   # ActiveStorage service for uploaded files.
   # config.storage_service = :local
@@ -26,6 +27,12 @@ DataPorter.configure do |config|
   # Enabled source types.
   # config.enabled_sources = %i[csv json xlsx api]
 
+  # Scope imports per owner (multi-tenant isolation).
+  # The lambda receives current_user and returns the owner object.
+  # Works with any model: User, Member, Hotel, Organization...
+  # config.scope = ->(user) { user }
+  # config.scope = ->(user) { user.hotel }
+
   # Auto-purge completed/failed imports older than this duration.
   # Set to nil to disable auto-purge. Run `rake data_porter:purge` manually or via cron.
   # config.purge_after = 60.days

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: data_porter
 version: !ruby/object:Gem::Version
-  version: 1.0.2
+  version: 2.0.0
 platform: ruby
 authors:
 - Seryl Lounis
@@ -122,6 +122,7 @@ files:
 - app/controllers/data_porter/concerns/import_validation.rb
 - app/controllers/data_porter/concerns/mapping_management.rb
 - app/controllers/data_porter/concerns/record_pagination.rb
+- app/controllers/data_porter/concerns/scope_management.rb
 - app/controllers/data_porter/imports_controller.rb
 - app/controllers/data_porter/mapping_templates_controller.rb
 - app/jobs/data_porter/dry_run_job.rb
@@ -138,6 +139,7 @@ files:
 - app/views/data_porter/mapping_templates/index.html.erb
 - app/views/data_porter/mapping_templates/new.html.erb
 - app/views/layouts/data_porter/application.html.erb
+- bookmarklet.md
 - config/routes.rb
 - lib/data_porter.rb
 - lib/data_porter/broadcaster.rb