data_porter 2.3.0 → 2.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +36 -0
- data/ROADMAP.md +0 -16
- data/app/assets/javascripts/data_porter/theme_controller.js +33 -0
- data/app/assets/stylesheets/data_porter/alerts.css +3 -3
- data/app/assets/stylesheets/data_porter/application.css +1 -0
- data/app/assets/stylesheets/data_porter/base.css +14 -6
- data/app/assets/stylesheets/data_porter/cards.css +1 -1
- data/app/assets/stylesheets/data_porter/dark.css +73 -0
- data/app/assets/stylesheets/data_porter/layout.css +33 -6
- data/app/assets/stylesheets/data_porter/mapping.css +9 -9
- data/app/assets/stylesheets/data_porter/modal.css +1 -1
- data/app/assets/stylesheets/data_porter/preview.css +1 -1
- data/app/assets/stylesheets/data_porter/progress.css +1 -1
- data/app/assets/stylesheets/data_porter/table.css +3 -3
- data/app/controllers/data_porter/concerns/mapping_management.rb +15 -1
- data/app/views/data_porter/imports/index.html.erb +1 -1
- data/app/views/data_porter/imports/show.html.erb +2 -2
- data/app/views/layouts/data_porter/application.html.erb +7 -2
- data/lib/data_porter/auto_mapper.rb +87 -0
- data/lib/data_porter/dsl/column.rb +3 -2
- data/lib/data_porter/engine.rb +1 -0
- data/lib/data_porter/orchestrator/bulk_importer.rb +54 -0
- data/lib/data_porter/orchestrator/importer.rb +3 -1
- data/lib/data_porter/orchestrator.rb +2 -0
- data/lib/data_porter/target.rb +16 -1
- data/lib/data_porter/version.rb +1 -1
- data/lib/data_porter.rb +1 -0
- data/lib/generators/data_porter/install/templates/initializer.rb +4 -0
- data/lib/generators/data_porter/target/templates/target.rb.tt +1 -0
- data/lib/generators/data_porter/views/views_generator.rb +43 -0
- data/mkdocs.yml +2 -0
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 99b0792c8088d9d2e55826f898fcfa99d872bea179f41d352b1e4aaff9ab14d8
|
|
4
|
+
data.tar.gz: 77c539c666176f992747f2d326a9ce8284cb19426b5f1ae470a317f7dfd740f1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b863d64f885f55ba6f530ce99173fef55ca473f74c7b0fc6736890bd770583949cf2101389c0635707ea38ceede63aebc529fc1a1e5990c76bf5c0750de318d6
|
|
7
|
+
data.tar.gz: 392ebe242c8ae947b37961bc8d59373a3d9f6a84730f0c946cd35af0a621ecc1a1cfb8b09b361f3a74fcc826c098fad317a4350d9b213206197406d88f763a01
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,42 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- **Auto-map heuristics** -- Smart column suggestions that pre-fill mapping selects when CSV/XLSX headers match target fields by exact name or built-in synonym (e.g. "E-mail Address" → email, "fname" → first_name). Supports per-column custom synonyms via `synonyms:` keyword in column DSL. Fallback chain: saved mapping > code-defined > auto-map > empty
|
|
13
|
+
|
|
14
|
+
## [2.5.1] - 2026-02-21
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- Display target icon (from `icon` DSL) in the imports index table and show page title/details. Previously the icon was stored in the Registry but never rendered in the UI
|
|
19
|
+
|
|
20
|
+
## [2.5.0] - 2026-02-21
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
|
|
24
|
+
- **Bulk import mode** -- Opt-in per-target batch persistence via `bulk_mode batch_size: 500, on_conflict: :retry_per_record`. Uses `insert_all` by default (with auto-injected timestamps) for 10-100x throughput on simple create scenarios. Custom batch logic via `persist_batch` override. Configurable conflict strategy: `:retry_per_record` (default) retries failed batches record-by-record, `:fail_batch` marks entire batch as errored. Progress broadcasts per batch instead of per record
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- 551 RSpec examples (up from 540), 0 failures
|
|
29
|
+
|
|
30
|
+
## [2.4.0] - 2026-02-21
|
|
31
|
+
|
|
32
|
+
### Added
|
|
33
|
+
|
|
34
|
+
- **View generator** -- `rails g data_porter:views` copies ERB templates into the host app for structural customization. Supports scoped copy: `imports`, `mapping_templates`, `layout`
|
|
35
|
+
- **Dark mode** -- Built-in dark theme via `.dp-dark` CSS class with automatic OS preference detection via `@media (prefers-color-scheme: dark)`. Theme toggle button with localStorage persistence
|
|
36
|
+
- **Semantic CSS variables** -- 8 high-level `--dp-*` variables (`--dp-bg`, `--dp-bg-elevated`, `--dp-text`, etc.) for easy theming without modifying templates
|
|
37
|
+
|
|
38
|
+
### Changed
|
|
39
|
+
|
|
40
|
+
- All hardcoded `white` backgrounds replaced with semantic CSS variables across 8 stylesheets
|
|
41
|
+
- Light mode color palette strengthened for better contrast (success, warning, danger backgrounds)
|
|
42
|
+
- 519 RSpec examples (up from 508), 0 failures
|
|
43
|
+
|
|
8
44
|
## [2.3.0] - 2026-02-20
|
|
9
45
|
|
|
10
46
|
### Added
|
data/ROADMAP.md
CHANGED
|
@@ -2,10 +2,6 @@
|
|
|
2
2
|
|
|
3
3
|
## Next
|
|
4
4
|
|
|
5
|
-
### Bulk import
|
|
6
|
-
|
|
7
|
-
High-volume import support using `insert_all` / `upsert_all` for batch persistence. Opt-in per target to bypass per-record `persist` calls, enabling 10-100x throughput for simple create/upsert scenarios. Configurable batch size, with fallback to per-record mode on conflict.
|
|
8
|
-
|
|
9
5
|
### Update & diff mode
|
|
10
6
|
|
|
11
7
|
Support update (upsert) imports alongside create-only. Given a `deduplicate_by` key, detect existing records and show a diff preview: new records, changed fields (highlighted), unchanged rows. User confirms which changes to apply. Enables recurring data sync workflows.
|
|
@@ -34,18 +30,6 @@ Headless REST API for programmatic imports:
|
|
|
34
30
|
- Auth via `config.api_authenticate` lambda (API key or Bearer token)
|
|
35
31
|
- Reuses existing job pipeline (parse, import, dry run)
|
|
36
32
|
|
|
37
|
-
### View generator & theming
|
|
38
|
-
|
|
39
|
-
Customizable UI in two layers:
|
|
40
|
-
|
|
41
|
-
- **View generator** — `rails g data_porter:views` copies the 7 ERB templates into the host app for structural customization (layout, buttons, sections). Similar to `devise:views`.
|
|
42
|
-
- **CSS theming** — All styles use `--dp-*` custom properties. Host apps override variables to match their design system, no ERB changes needed.
|
|
43
|
-
- **Light / dark mode** — Two built-in presets toggled via `prefers-color-scheme` or a `.dp-dark` class.
|
|
44
|
-
|
|
45
|
-
### Auto-map heuristics
|
|
46
|
-
|
|
47
|
-
Smart column mapping suggestions using tokenized header matching and synonym dictionaries. When a CSV has "E-mail Address", auto-suggest mapping to `:email`. Built-in synonyms for common patterns (phone → phone_number, first name → first_name). Configurable synonym lists per target.
|
|
48
|
-
|
|
49
33
|
---
|
|
50
34
|
|
|
51
35
|
## Ideas
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import { Controller } from "@hotwired/stimulus"
|
|
2
|
+
|
|
3
|
+
export default class extends Controller {
|
|
4
|
+
static targets = ["icon"]
|
|
5
|
+
|
|
6
|
+
connect() {
|
|
7
|
+
this.applyTheme(this.currentTheme())
|
|
8
|
+
}
|
|
9
|
+
|
|
10
|
+
toggle() {
|
|
11
|
+
const next = this.currentTheme() === "dark" ? "light" : "dark"
|
|
12
|
+
localStorage.setItem("dp-theme", next)
|
|
13
|
+
this.applyTheme(next)
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
currentTheme() {
|
|
17
|
+
const stored = localStorage.getItem("dp-theme")
|
|
18
|
+
if (stored) return stored
|
|
19
|
+
return window.matchMedia("(prefers-color-scheme: dark)").matches ? "dark" : "light"
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
applyTheme(theme) {
|
|
23
|
+
const root = this.element
|
|
24
|
+
if (theme === "dark") {
|
|
25
|
+
root.classList.add("dp-dark")
|
|
26
|
+
} else {
|
|
27
|
+
root.classList.remove("dp-dark")
|
|
28
|
+
}
|
|
29
|
+
if (this.hasIconTarget) {
|
|
30
|
+
this.iconTarget.textContent = theme === "dark" ? "\u2600" : "\u263D"
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
}
|
|
@@ -12,8 +12,8 @@
|
|
|
12
12
|
}
|
|
13
13
|
|
|
14
14
|
.dp-results--partial {
|
|
15
|
-
background: var(--dp-
|
|
16
|
-
border: 1px solid var(--dp-
|
|
15
|
+
background: var(--dp-danger-light);
|
|
16
|
+
border: 1px solid var(--dp-danger-border);
|
|
17
17
|
}
|
|
18
18
|
|
|
19
19
|
.dp-results__icon {
|
|
@@ -39,7 +39,7 @@
|
|
|
39
39
|
.dp-results__stat {
|
|
40
40
|
padding: 1rem;
|
|
41
41
|
border-radius: var(--dp-radius-md);
|
|
42
|
-
background:
|
|
42
|
+
background: var(--dp-bg-elevated);
|
|
43
43
|
border: 1px solid var(--dp-gray-200);
|
|
44
44
|
}
|
|
45
45
|
|
|
@@ -3,15 +3,15 @@
|
|
|
3
3
|
--dp-primary-hover: #4338ca;
|
|
4
4
|
--dp-primary-light: #eef2ff;
|
|
5
5
|
--dp-success: #059669;
|
|
6
|
-
--dp-success-light: #
|
|
7
|
-
--dp-success-border: #
|
|
6
|
+
--dp-success-light: #d1fae5;
|
|
7
|
+
--dp-success-border: #6ee7b7;
|
|
8
8
|
--dp-warning: #d97706;
|
|
9
|
-
--dp-warning-light: #
|
|
10
|
-
--dp-warning-border: #
|
|
9
|
+
--dp-warning-light: #fef3c7;
|
|
10
|
+
--dp-warning-border: #fcd34d;
|
|
11
11
|
--dp-danger: #dc2626;
|
|
12
12
|
--dp-danger-hover: #b91c1c;
|
|
13
|
-
--dp-danger-light: #
|
|
14
|
-
--dp-danger-border: #
|
|
13
|
+
--dp-danger-light: #fee2e2;
|
|
14
|
+
--dp-danger-border: #fca5a5;
|
|
15
15
|
--dp-info: #2563eb;
|
|
16
16
|
--dp-info-light: #eff6ff;
|
|
17
17
|
--dp-info-border: #bfdbfe;
|
|
@@ -35,6 +35,14 @@
|
|
|
35
35
|
--dp-shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.05);
|
|
36
36
|
--dp-shadow-md: 0 4px 6px -1px rgba(0, 0, 0, 0.07), 0 2px 4px -2px rgba(0, 0, 0, 0.05);
|
|
37
37
|
--dp-shadow-lg: 0 10px 15px -3px rgba(0, 0, 0, 0.08), 0 4px 6px -4px rgba(0, 0, 0, 0.04);
|
|
38
|
+
--dp-bg: white;
|
|
39
|
+
--dp-bg-elevated: white;
|
|
40
|
+
--dp-bg-subtle: var(--dp-gray-50);
|
|
41
|
+
--dp-text: var(--dp-gray-800);
|
|
42
|
+
--dp-text-heading: var(--dp-gray-900);
|
|
43
|
+
--dp-text-muted: var(--dp-gray-500);
|
|
44
|
+
--dp-border: var(--dp-gray-200);
|
|
45
|
+
--dp-border-strong: var(--dp-gray-300);
|
|
38
46
|
}
|
|
39
47
|
|
|
40
48
|
.data-porter {
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
border-radius: var(--dp-radius-lg);
|
|
11
11
|
border: 1px solid var(--dp-gray-200);
|
|
12
12
|
text-align: center;
|
|
13
|
-
background:
|
|
13
|
+
background: var(--dp-bg-elevated);
|
|
14
14
|
box-shadow: var(--dp-shadow-sm);
|
|
15
15
|
transition: transform 0.15s ease, box-shadow 0.15s ease;
|
|
16
16
|
}
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
.dp-dark {
|
|
2
|
+
background: #111827;
|
|
3
|
+
color: #e5e7eb;
|
|
4
|
+
--dp-bg: #111827;
|
|
5
|
+
--dp-bg-elevated: #1f2937;
|
|
6
|
+
--dp-bg-subtle: #1f2937;
|
|
7
|
+
--dp-text: #e5e7eb;
|
|
8
|
+
--dp-text-heading: #f9fafb;
|
|
9
|
+
--dp-text-muted: #9ca3af;
|
|
10
|
+
--dp-border: #374151;
|
|
11
|
+
--dp-border-strong: #4b5563;
|
|
12
|
+
--dp-gray-50: #1f2937;
|
|
13
|
+
--dp-gray-100: #374151;
|
|
14
|
+
--dp-gray-200: #374151;
|
|
15
|
+
--dp-gray-300: #4b5563;
|
|
16
|
+
--dp-gray-400: #6b7280;
|
|
17
|
+
--dp-gray-500: #9ca3af;
|
|
18
|
+
--dp-gray-600: #d1d5db;
|
|
19
|
+
--dp-gray-700: #e5e7eb;
|
|
20
|
+
--dp-gray-800: #e5e7eb;
|
|
21
|
+
--dp-gray-900: #f9fafb;
|
|
22
|
+
--dp-primary-light: rgba(79, 70, 229, 0.25);
|
|
23
|
+
--dp-success-light: rgba(5, 150, 105, 0.25);
|
|
24
|
+
--dp-success-border: rgba(5, 150, 105, 0.4);
|
|
25
|
+
--dp-warning-light: rgba(217, 119, 6, 0.25);
|
|
26
|
+
--dp-warning-border: rgba(217, 119, 6, 0.4);
|
|
27
|
+
--dp-danger-light: rgba(220, 38, 38, 0.25);
|
|
28
|
+
--dp-danger-border: rgba(220, 38, 38, 0.4);
|
|
29
|
+
--dp-info-light: rgba(37, 99, 235, 0.25);
|
|
30
|
+
--dp-info-border: rgba(37, 99, 235, 0.4);
|
|
31
|
+
--dp-purple-light: rgba(124, 58, 237, 0.25);
|
|
32
|
+
--dp-purple-border: rgba(124, 58, 237, 0.4);
|
|
33
|
+
--dp-shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.3);
|
|
34
|
+
--dp-shadow-md: 0 4px 6px -1px rgba(0, 0, 0, 0.4), 0 2px 4px -2px rgba(0, 0, 0, 0.3);
|
|
35
|
+
--dp-shadow-lg: 0 10px 15px -3px rgba(0, 0, 0, 0.5), 0 4px 6px -4px rgba(0, 0, 0, 0.4);
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
@media (prefers-color-scheme: dark) {
|
|
39
|
+
.dp-auto {
|
|
40
|
+
--dp-bg: #111827;
|
|
41
|
+
--dp-bg-elevated: #1f2937;
|
|
42
|
+
--dp-bg-subtle: #1f2937;
|
|
43
|
+
--dp-text: #e5e7eb;
|
|
44
|
+
--dp-text-heading: #f9fafb;
|
|
45
|
+
--dp-text-muted: #9ca3af;
|
|
46
|
+
--dp-border: #374151;
|
|
47
|
+
--dp-border-strong: #4b5563;
|
|
48
|
+
--dp-gray-50: #1f2937;
|
|
49
|
+
--dp-gray-100: #374151;
|
|
50
|
+
--dp-gray-200: #374151;
|
|
51
|
+
--dp-gray-300: #4b5563;
|
|
52
|
+
--dp-gray-400: #6b7280;
|
|
53
|
+
--dp-gray-500: #9ca3af;
|
|
54
|
+
--dp-gray-600: #d1d5db;
|
|
55
|
+
--dp-gray-700: #e5e7eb;
|
|
56
|
+
--dp-gray-800: #e5e7eb;
|
|
57
|
+
--dp-gray-900: #f9fafb;
|
|
58
|
+
--dp-primary-light: rgba(79, 70, 229, 0.25);
|
|
59
|
+
--dp-success-light: rgba(5, 150, 105, 0.25);
|
|
60
|
+
--dp-success-border: rgba(5, 150, 105, 0.4);
|
|
61
|
+
--dp-warning-light: rgba(217, 119, 6, 0.25);
|
|
62
|
+
--dp-warning-border: rgba(217, 119, 6, 0.4);
|
|
63
|
+
--dp-danger-light: rgba(220, 38, 38, 0.25);
|
|
64
|
+
--dp-danger-border: rgba(220, 38, 38, 0.4);
|
|
65
|
+
--dp-info-light: rgba(37, 99, 235, 0.25);
|
|
66
|
+
--dp-info-border: rgba(37, 99, 235, 0.4);
|
|
67
|
+
--dp-purple-light: rgba(124, 58, 237, 0.25);
|
|
68
|
+
--dp-purple-border: rgba(124, 58, 237, 0.4);
|
|
69
|
+
--dp-shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.3);
|
|
70
|
+
--dp-shadow-md: 0 4px 6px -1px rgba(0, 0, 0, 0.4), 0 2px 4px -2px rgba(0, 0, 0, 0.3);
|
|
71
|
+
--dp-shadow-lg: 0 10px 15px -3px rgba(0, 0, 0, 0.5), 0 4px 6px -4px rgba(0, 0, 0, 0.4);
|
|
72
|
+
}
|
|
73
|
+
}
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
|
|
13
13
|
.dp-form {
|
|
14
14
|
max-width: 520px;
|
|
15
|
-
background:
|
|
15
|
+
background: var(--dp-bg-elevated);
|
|
16
16
|
padding: 2rem;
|
|
17
17
|
border: 1px solid var(--dp-gray-200);
|
|
18
18
|
border-radius: var(--dp-radius-lg);
|
|
@@ -36,8 +36,8 @@
|
|
|
36
36
|
border: 1px solid var(--dp-gray-300);
|
|
37
37
|
border-radius: var(--dp-radius-md);
|
|
38
38
|
font-size: 0.9375rem;
|
|
39
|
-
background:
|
|
40
|
-
color: var(--dp-
|
|
39
|
+
background: var(--dp-bg-elevated);
|
|
40
|
+
color: var(--dp-text);
|
|
41
41
|
transition: border-color 0.15s ease, box-shadow 0.15s ease;
|
|
42
42
|
appearance: none;
|
|
43
43
|
}
|
|
@@ -95,7 +95,7 @@
|
|
|
95
95
|
.dp-btn--primary { background: var(--dp-primary); color: white; box-shadow: var(--dp-shadow-sm); }
|
|
96
96
|
.dp-btn--primary:hover { background: var(--dp-primary-hover); box-shadow: var(--dp-shadow-md); }
|
|
97
97
|
|
|
98
|
-
.dp-btn--secondary { background:
|
|
98
|
+
.dp-btn--secondary { background: var(--dp-bg-elevated); color: var(--dp-gray-700); border-color: var(--dp-gray-300); box-shadow: var(--dp-shadow-sm); }
|
|
99
99
|
.dp-btn--secondary:hover { background: var(--dp-gray-50); border-color: var(--dp-gray-400); }
|
|
100
100
|
|
|
101
101
|
.dp-btn--danger { background: var(--dp-danger); color: white; box-shadow: var(--dp-shadow-sm); }
|
|
@@ -112,7 +112,7 @@
|
|
|
112
112
|
.dp-empty-state {
|
|
113
113
|
text-align: center;
|
|
114
114
|
padding: 4rem 2rem;
|
|
115
|
-
background:
|
|
115
|
+
background: var(--dp-bg-elevated);
|
|
116
116
|
border: 2px dashed var(--dp-gray-300);
|
|
117
117
|
border-radius: var(--dp-radius-xl);
|
|
118
118
|
}
|
|
@@ -121,7 +121,7 @@
|
|
|
121
121
|
.dp-empty-state__text { font-size: 1.125rem; color: var(--dp-gray-500); margin: 0 0 1.5rem; }
|
|
122
122
|
|
|
123
123
|
.dp-import-details {
|
|
124
|
-
background:
|
|
124
|
+
background: var(--dp-bg-elevated);
|
|
125
125
|
padding: 1.5rem;
|
|
126
126
|
border: 1px solid var(--dp-gray-200);
|
|
127
127
|
border-radius: var(--dp-radius-lg);
|
|
@@ -145,3 +145,30 @@
|
|
|
145
145
|
margin: 0;
|
|
146
146
|
color: var(--dp-gray-800);
|
|
147
147
|
}
|
|
148
|
+
|
|
149
|
+
.dp-theme-toggle {
|
|
150
|
+
position: fixed;
|
|
151
|
+
bottom: 1.5rem;
|
|
152
|
+
right: 1.5rem;
|
|
153
|
+
width: 2.75rem;
|
|
154
|
+
height: 2.75rem;
|
|
155
|
+
border-radius: 50%;
|
|
156
|
+
border: 1px solid var(--dp-border-strong);
|
|
157
|
+
background: var(--dp-bg-elevated);
|
|
158
|
+
color: var(--dp-text-muted);
|
|
159
|
+
font-size: 1.25rem;
|
|
160
|
+
cursor: pointer;
|
|
161
|
+
box-shadow: var(--dp-shadow-md);
|
|
162
|
+
transition: all 0.15s ease;
|
|
163
|
+
display: flex;
|
|
164
|
+
align-items: center;
|
|
165
|
+
justify-content: center;
|
|
166
|
+
z-index: 9998;
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
.dp-theme-toggle:hover {
|
|
170
|
+
background: var(--dp-bg-subtle);
|
|
171
|
+
color: var(--dp-text);
|
|
172
|
+
box-shadow: var(--dp-shadow-lg);
|
|
173
|
+
transform: scale(1.05);
|
|
174
|
+
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
.dp-mapping-form {
|
|
2
2
|
max-width: 700px;
|
|
3
|
-
background:
|
|
3
|
+
background: var(--dp-bg-elevated);
|
|
4
4
|
padding: 2rem;
|
|
5
5
|
border: 1px solid var(--dp-gray-200);
|
|
6
6
|
border-radius: var(--dp-radius-lg);
|
|
@@ -53,8 +53,8 @@
|
|
|
53
53
|
}
|
|
54
54
|
|
|
55
55
|
.dp-mapping-row--duplicate {
|
|
56
|
-
border-color:
|
|
57
|
-
background:
|
|
56
|
+
border-color: var(--dp-warning-border);
|
|
57
|
+
background: var(--dp-warning-light);
|
|
58
58
|
}
|
|
59
59
|
|
|
60
60
|
.dp-mapping-required-warning,
|
|
@@ -67,13 +67,13 @@
|
|
|
67
67
|
}
|
|
68
68
|
|
|
69
69
|
.dp-mapping-required-warning {
|
|
70
|
-
background:
|
|
71
|
-
border: 1px solid
|
|
72
|
-
color:
|
|
70
|
+
background: var(--dp-danger-light);
|
|
71
|
+
border: 1px solid var(--dp-danger-border);
|
|
72
|
+
color: var(--dp-danger);
|
|
73
73
|
}
|
|
74
74
|
|
|
75
75
|
.dp-mapping-duplicate-warning {
|
|
76
|
-
background:
|
|
77
|
-
border: 1px solid
|
|
78
|
-
color:
|
|
76
|
+
background: var(--dp-warning-light);
|
|
77
|
+
border: 1px solid var(--dp-warning-border);
|
|
78
|
+
color: var(--dp-warning);
|
|
79
79
|
}
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
.dp-modal__backdrop { position: fixed; inset: 0; background: rgba(0,0,0,0.4); backdrop-filter: blur(4px); }
|
|
4
4
|
.dp-modal__content {
|
|
5
5
|
position: relative;
|
|
6
|
-
background:
|
|
6
|
+
background: var(--dp-bg-elevated);
|
|
7
7
|
border-radius: var(--dp-radius-xl);
|
|
8
8
|
box-shadow: var(--dp-shadow-lg), 0 25px 50px -12px rgba(0,0,0,0.15);
|
|
9
9
|
width: 100%;
|
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
}
|
|
15
15
|
|
|
16
16
|
.dp-row--complete { background: var(--dp-success-light); }
|
|
17
|
-
.dp-row--partial { background: var(--dp-
|
|
17
|
+
.dp-row--partial { background: var(--dp-danger-light); }
|
|
18
18
|
.dp-row--missing { background: var(--dp-danger-light); }
|
|
19
19
|
|
|
20
20
|
.dp-errors {
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
border-collapse: separate;
|
|
4
4
|
border-spacing: 0;
|
|
5
5
|
margin-bottom: 1.5rem;
|
|
6
|
-
background:
|
|
6
|
+
background: var(--dp-bg-elevated);
|
|
7
7
|
border: 1px solid var(--dp-gray-200);
|
|
8
8
|
border-radius: var(--dp-radius-lg);
|
|
9
9
|
overflow: hidden;
|
|
@@ -65,7 +65,7 @@
|
|
|
65
65
|
font-size: 0.875rem;
|
|
66
66
|
font-weight: 500;
|
|
67
67
|
color: var(--dp-primary);
|
|
68
|
-
background:
|
|
68
|
+
background: var(--dp-bg-elevated);
|
|
69
69
|
border: 1px solid var(--dp-gray-300);
|
|
70
70
|
border-radius: var(--dp-radius-md);
|
|
71
71
|
text-decoration: none;
|
|
@@ -85,7 +85,7 @@
|
|
|
85
85
|
}
|
|
86
86
|
|
|
87
87
|
.dp-pagination__btn--disabled:hover {
|
|
88
|
-
background:
|
|
88
|
+
background: var(--dp-bg-elevated);
|
|
89
89
|
border-color: var(--dp-gray-300);
|
|
90
90
|
}
|
|
91
91
|
|
|
@@ -27,7 +27,21 @@ module DataPorter
|
|
|
27
27
|
saved = @import.config&.dig("column_mapping")
|
|
28
28
|
return saved if saved.present?
|
|
29
29
|
|
|
30
|
-
(target._csv_mappings || {}).transform_values(&:to_s)
|
|
30
|
+
code_mapping = (target._csv_mappings || {}).transform_values(&:to_s)
|
|
31
|
+
return code_mapping if code_mapping.present?
|
|
32
|
+
|
|
33
|
+
auto_map_suggestions(target)
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def auto_map_suggestions(target)
|
|
37
|
+
columns = target._columns || []
|
|
38
|
+
return {} if columns.empty? || @file_headers.empty?
|
|
39
|
+
|
|
40
|
+
custom = columns.each_with_object({}) do |col, hash|
|
|
41
|
+
hash[col.name] = col.synonyms if col.synonyms.any?
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
AutoMapper.new(@file_headers, columns.map(&:name), custom_synonyms: custom).call
|
|
31
45
|
end
|
|
32
46
|
|
|
33
47
|
def save_column_mapping
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
<% @imports.each do |import| %>
|
|
29
29
|
<tr>
|
|
30
30
|
<td><%= import.id %></td>
|
|
31
|
-
<td
|
|
31
|
+
<td><% target_cls = import.target_class rescue nil %><% if target_cls&._icon.present? %><i class="<%= target_cls._icon %>"></i> <% end %><%= target_cls&._label || import.target_key %></td>
|
|
32
32
|
<td><%= import.source_type %></td>
|
|
33
33
|
<td><%= raw DataPorter::Components::Shared::StatusBadge.new(status: import.status).call %></td>
|
|
34
34
|
<td><%= import.created_at&.strftime("%Y-%m-%d %H:%M") %></td>
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
<%= link_to t("data_porter.imports.back_to_imports"), imports_path, class: "dp-btn dp-btn--secondary" %>
|
|
5
5
|
</div>
|
|
6
6
|
<h1 class="dp-title">
|
|
7
|
-
<%= t("data_porter.imports.show_title", target: @target._label, id: @import.id) %>
|
|
7
|
+
<% if @target._icon.present? %><i class="<%= @target._icon %>"></i> <% end %><%= t("data_porter.imports.show_title", target: @target._label, id: @import.id) %>
|
|
8
8
|
</h1>
|
|
9
9
|
<%= raw DataPorter::Components::Shared::StatusBadge.new(status: @import.status).call %>
|
|
10
10
|
</div>
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
<div class="dp-import-details">
|
|
13
13
|
<dl class="dp-details-grid">
|
|
14
14
|
<dt><%= t("data_porter.imports.details.target") %></dt>
|
|
15
|
-
<dd
|
|
15
|
+
<dd><% if @target._icon.present? %><i class="<%= @target._icon %>"></i> <% end %><%= @target._label %></dd>
|
|
16
16
|
<dt><%= t("data_porter.imports.details.source") %></dt>
|
|
17
17
|
<dd><%= @import.source_type.upcase %></dd>
|
|
18
18
|
<% if @import.file.attached? %>
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
<title>DataPorter</title>
|
|
7
7
|
<%= csrf_meta_tags %>
|
|
8
8
|
<%= stylesheet_link_tag "data_porter/application" %>
|
|
9
|
+
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/css/all.min.css" integrity="sha512-DTOQO9RWCH3ppGqcWaEA1BIZOC6xxalwEsw9c2QQeAIftl+Vegovlnee1c9QX4TctnWMn13TZye+giMm8e2LwA==" crossorigin="anonymous" referrerpolicy="no-referrer" />
|
|
9
10
|
<script type="importmap">
|
|
10
11
|
{
|
|
11
12
|
"imports": {
|
|
@@ -14,7 +15,8 @@
|
|
|
14
15
|
"data_porter/mapping_controller": "<%= asset_path('data_porter/mapping_controller.js') %>",
|
|
15
16
|
"data_porter/template_form_controller": "<%= asset_path('data_porter/template_form_controller.js') %>",
|
|
16
17
|
"data_porter/progress_controller": "<%= asset_path('data_porter/progress_controller.js') %>",
|
|
17
|
-
"data_porter/import_form_controller": "<%= asset_path('data_porter/import_form_controller.js') %>"
|
|
18
|
+
"data_porter/import_form_controller": "<%= asset_path('data_porter/import_form_controller.js') %>",
|
|
19
|
+
"data_porter/theme_controller": "<%= asset_path('data_porter/theme_controller.js') %>"
|
|
18
20
|
}
|
|
19
21
|
}
|
|
20
22
|
</script>
|
|
@@ -25,15 +27,18 @@
|
|
|
25
27
|
import TemplateFormController from "data_porter/template_form_controller"
|
|
26
28
|
import ProgressController from "data_porter/progress_controller"
|
|
27
29
|
import ImportFormController from "data_porter/import_form_controller"
|
|
30
|
+
import ThemeController from "data_porter/theme_controller"
|
|
28
31
|
|
|
29
32
|
const application = Application.start()
|
|
30
33
|
application.register("data-porter--mapping", MappingController)
|
|
31
34
|
application.register("data-porter--template-form", TemplateFormController)
|
|
32
35
|
application.register("data-porter--progress", ProgressController)
|
|
33
36
|
application.register("data-porter--import-form", ImportFormController)
|
|
37
|
+
application.register("data-porter--theme", ThemeController)
|
|
34
38
|
</script>
|
|
35
39
|
</head>
|
|
36
|
-
<body>
|
|
40
|
+
<body data-controller="data-porter--theme">
|
|
37
41
|
<%= yield %>
|
|
42
|
+
<button class="dp-theme-toggle" data-action="data-porter--theme#toggle" data-data-porter--theme-target="icon" aria-label="Toggle theme">☽</button>
|
|
38
43
|
</body>
|
|
39
44
|
</html>
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module DataPorter
|
|
4
|
+
class AutoMapper
|
|
5
|
+
SYNONYMS = {
|
|
6
|
+
email: %w[email e_mail email_address e_mail_address courriel mail],
|
|
7
|
+
first_name: %w[first_name firstname fname first prenom],
|
|
8
|
+
last_name: %w[last_name lastname lname last nom],
|
|
9
|
+
name: %w[name full_name fullname nom_complet],
|
|
10
|
+
phone_number: %w[phone_number phone tel telephone mobile cell],
|
|
11
|
+
address: %w[address addr street adresse],
|
|
12
|
+
city: %w[city ville town],
|
|
13
|
+
zip_code: %w[zip_code zip postal_code postcode code_postal],
|
|
14
|
+
country: %w[country pays nation],
|
|
15
|
+
company: %w[company company_name organization organisation entreprise societe],
|
|
16
|
+
title: %w[title job_title position titre poste],
|
|
17
|
+
description: %w[description desc notes],
|
|
18
|
+
quantity: %w[quantity qty amount],
|
|
19
|
+
price: %w[price unit_price prix montant],
|
|
20
|
+
date: %w[date created_at updated_at],
|
|
21
|
+
status: %w[status state statut etat],
|
|
22
|
+
id: %w[id identifier external_id ref reference]
|
|
23
|
+
}.freeze
|
|
24
|
+
|
|
25
|
+
def initialize(headers, target_columns, custom_synonyms: {})
|
|
26
|
+
@headers = headers
|
|
27
|
+
@target_columns = target_columns.map(&:to_s)
|
|
28
|
+
@custom_synonyms = custom_synonyms
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def call
|
|
32
|
+
used = Set.new
|
|
33
|
+
@headers.each_with_object({}) do |header, mapping|
|
|
34
|
+
match = find_match(header, used)
|
|
35
|
+
used.add(match) if match
|
|
36
|
+
mapping[header] = match || ""
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
def find_match(header, used)
|
|
43
|
+
normalized = normalize(header)
|
|
44
|
+
return nil if normalized.empty?
|
|
45
|
+
|
|
46
|
+
exact_match(normalized, used) || synonym_match(normalized, used)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def exact_match(normalized, used)
|
|
50
|
+
@target_columns.find { |col| col == normalized && !used.include?(col) }
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def synonym_match(normalized, used)
|
|
54
|
+
lookup_table[normalized]&.find { |col| !used.include?(col) }
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def lookup_table
|
|
58
|
+
@lookup_table ||= build_lookup_table
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def build_lookup_table
|
|
62
|
+
table = Hash.new { |h, k| h[k] = [] }
|
|
63
|
+
merged_synonyms.each do |column, synonyms|
|
|
64
|
+
col_name = column.to_s
|
|
65
|
+
next unless @target_columns.include?(col_name)
|
|
66
|
+
|
|
67
|
+
synonyms.each { |syn| table[syn] << col_name }
|
|
68
|
+
end
|
|
69
|
+
table
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def merged_synonyms
|
|
73
|
+
result = SYNONYMS.transform_values(&:dup)
|
|
74
|
+
@custom_synonyms.each do |column, syns|
|
|
75
|
+
key = column.to_sym
|
|
76
|
+
result[key] = (result.fetch(key, []) + syns.map { |s| normalize(s) }).uniq
|
|
77
|
+
end
|
|
78
|
+
result
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def normalize(header)
|
|
82
|
+
return "" if header.nil?
|
|
83
|
+
|
|
84
|
+
header.to_s.strip.downcase.gsub(/[\s-]+/, "_").gsub(/[^a-z0-9_]/, "")
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
end
|
|
@@ -2,14 +2,15 @@
|
|
|
2
2
|
|
|
3
3
|
module DataPorter
|
|
4
4
|
module DSL
|
|
5
|
-
Column = Struct.new(:name, :type, :required, :label, :transform, :options, keyword_init: true) do
|
|
6
|
-
def initialize(name:, type: :string, required: false, label: nil, transform: [], **options)
|
|
5
|
+
Column = Struct.new(:name, :type, :required, :label, :transform, :synonyms, :options, keyword_init: true) do
|
|
6
|
+
def initialize(name:, type: :string, required: false, label: nil, transform: [], synonyms: [], **options)
|
|
7
7
|
super(
|
|
8
8
|
name: name.to_sym,
|
|
9
9
|
type: type.to_sym,
|
|
10
10
|
required: required,
|
|
11
11
|
label: label || name.to_s.humanize,
|
|
12
12
|
transform: Array(transform),
|
|
13
|
+
synonyms: Array(synonyms),
|
|
13
14
|
options: options
|
|
14
15
|
)
|
|
15
16
|
end
|
data/lib/data_porter/engine.rb
CHANGED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module DataPorter
|
|
4
|
+
class Orchestrator
|
|
5
|
+
module BulkImporter
|
|
6
|
+
private
|
|
7
|
+
|
|
8
|
+
def import_bulk
|
|
9
|
+
importable = @data_import.importable_records
|
|
10
|
+
context = build_context
|
|
11
|
+
config = @target.class._bulk_config
|
|
12
|
+
results = { created: 0, errored: 0 }
|
|
13
|
+
total = importable.size
|
|
14
|
+
processed = 0
|
|
15
|
+
|
|
16
|
+
importable.each_slice(config[:batch_size]) do |batch|
|
|
17
|
+
persist_batch_with_fallback(batch, context, config, results)
|
|
18
|
+
processed += batch.size
|
|
19
|
+
broadcast_progress(processed, total)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
finalize_import(results)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def persist_batch_with_fallback(batch, context, config, results)
|
|
26
|
+
@target.persist_batch(batch, context: context)
|
|
27
|
+
results[:created] += batch.size
|
|
28
|
+
rescue StandardError => e
|
|
29
|
+
handle_batch_failure(batch, context, config, results, e)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def handle_batch_failure(batch, context, config, results, error)
|
|
33
|
+
if config[:on_conflict] == :fail_batch
|
|
34
|
+
fail_batch(batch, results, error)
|
|
35
|
+
else
|
|
36
|
+
retry_per_record(batch, context, results)
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def fail_batch(batch, results, error)
|
|
41
|
+
batch.each do |record|
|
|
42
|
+
record.add_error(error.message)
|
|
43
|
+
end
|
|
44
|
+
results[:errored] += batch.size
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def retry_per_record(batch, context, results)
|
|
48
|
+
batch.each do |record|
|
|
49
|
+
persist_record(record, context, results)
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
end
|
|
@@ -6,7 +6,9 @@ module DataPorter
|
|
|
6
6
|
private
|
|
7
7
|
|
|
8
8
|
def import_records
|
|
9
|
-
if
|
|
9
|
+
if @target.class._bulk_config
|
|
10
|
+
import_bulk
|
|
11
|
+
elsif DataPorter.configuration.transaction_mode == :all
|
|
10
12
|
import_all_or_nothing
|
|
11
13
|
else
|
|
12
14
|
import_per_record
|
|
@@ -2,12 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
require_relative "orchestrator/record_builder"
|
|
4
4
|
require_relative "orchestrator/importer"
|
|
5
|
+
require_relative "orchestrator/bulk_importer"
|
|
5
6
|
require_relative "orchestrator/dry_runner"
|
|
6
7
|
|
|
7
8
|
module DataPorter
|
|
8
9
|
class Orchestrator
|
|
9
10
|
include RecordBuilder
|
|
10
11
|
include Importer
|
|
12
|
+
include BulkImporter
|
|
11
13
|
include DryRunner
|
|
12
14
|
|
|
13
15
|
def initialize(data_import, content: nil)
|
data/lib/data_porter/target.rb
CHANGED
|
@@ -10,7 +10,8 @@ module DataPorter
|
|
|
10
10
|
class << self
|
|
11
11
|
attr_reader :_label, :_model_name, :_icon, :_sources,
|
|
12
12
|
:_columns, :_csv_mappings, :_dedup_keys, :_json_root,
|
|
13
|
-
:_api_config, :_dry_run_enabled, :_params, :_webhooks
|
|
13
|
+
:_api_config, :_dry_run_enabled, :_params, :_webhooks,
|
|
14
|
+
:_bulk_config
|
|
14
15
|
|
|
15
16
|
def label(value)
|
|
16
17
|
@_label = value
|
|
@@ -82,6 +83,10 @@ module DataPorter
|
|
|
82
83
|
@_webhooks << DSL::Webhook.new(url: url, **)
|
|
83
84
|
end
|
|
84
85
|
|
|
86
|
+
def bulk_mode(batch_size: 500, on_conflict: :retry_per_record)
|
|
87
|
+
@_bulk_config = { batch_size: batch_size, on_conflict: on_conflict }
|
|
88
|
+
end
|
|
89
|
+
|
|
85
90
|
private
|
|
86
91
|
|
|
87
92
|
def auto_register
|
|
@@ -108,6 +113,16 @@ module DataPorter
|
|
|
108
113
|
raise NotImplementedError
|
|
109
114
|
end
|
|
110
115
|
|
|
116
|
+
def persist_batch(records, context: nil) # rubocop:disable Lint/UnusedMethodArgument
|
|
117
|
+
raise Error, "model_name is required for default persist_batch" unless self.class._model_name
|
|
118
|
+
|
|
119
|
+
now = Time.current
|
|
120
|
+
model_class = self.class._model_name.constantize
|
|
121
|
+
model_class.insert_all(
|
|
122
|
+
records.map { |r| r.data.merge("created_at" => now, "updated_at" => now) }
|
|
123
|
+
)
|
|
124
|
+
end
|
|
125
|
+
|
|
111
126
|
def after_import(_results, context:); end
|
|
112
127
|
|
|
113
128
|
def on_error(_record, _error, context:); end
|
data/lib/data_porter/version.rb
CHANGED
data/lib/data_porter.rb
CHANGED
|
@@ -20,6 +20,7 @@ require_relative "data_porter/record_validator"
|
|
|
20
20
|
require_relative "data_porter/broadcaster"
|
|
21
21
|
require_relative "data_porter/webhook_notifier"
|
|
22
22
|
require_relative "data_porter/orchestrator"
|
|
23
|
+
require_relative "data_porter/auto_mapper"
|
|
23
24
|
require_relative "data_porter/rejects_csv_builder"
|
|
24
25
|
require_relative "data_porter/components"
|
|
25
26
|
require_relative "data_porter/engine"
|
|
@@ -37,6 +37,10 @@ DataPorter.configure do |config|
|
|
|
37
37
|
# Set to nil to disable auto-purge. Run `rake data_porter:purge` manually or via cron.
|
|
38
38
|
# config.purge_after = 60.days
|
|
39
39
|
|
|
40
|
+
# Bulk import: enable per-target via `bulk_mode` in your Target class.
|
|
41
|
+
# Uses insert_all for 10-100x throughput on large imports.
|
|
42
|
+
# See docs/ADVANCED.md for configuration options.
|
|
43
|
+
|
|
40
44
|
# HMAC-SHA256 secret for signing webhook payloads.
|
|
41
45
|
# When set, every webhook request includes an X-DataPorter-Signature header.
|
|
42
46
|
# Set to nil to disable signing (default).
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
|
|
5
|
+
module DataPorter
|
|
6
|
+
module Generators
|
|
7
|
+
class ViewsGenerator < Rails::Generators::Base
|
|
8
|
+
SCOPES = %w[imports mapping_templates layout].freeze
|
|
9
|
+
|
|
10
|
+
source_root DataPorter::Engine.root.join("app/views")
|
|
11
|
+
|
|
12
|
+
argument :scope, type: :string, required: false, default: nil
|
|
13
|
+
|
|
14
|
+
def copy_views
|
|
15
|
+
validate_scope! if scope
|
|
16
|
+
scopes = scope ? [scope] : SCOPES
|
|
17
|
+
|
|
18
|
+
scopes.each { |s| send(:"copy_#{s}") }
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
private
|
|
22
|
+
|
|
23
|
+
def copy_imports
|
|
24
|
+
directory "data_porter/imports", "app/views/data_porter/imports"
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def copy_mapping_templates
|
|
28
|
+
directory "data_porter/mapping_templates", "app/views/data_porter/mapping_templates"
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def copy_layout
|
|
32
|
+
layout_src = DataPorter::Engine.root.join("app/views/layouts/data_porter/application.html.erb")
|
|
33
|
+
copy_file layout_src, "app/views/layouts/data_porter/application.html.erb"
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def validate_scope!
|
|
37
|
+
return if SCOPES.include?(scope)
|
|
38
|
+
|
|
39
|
+
raise Thor::Error, "Unknown scope '#{scope}'. Valid scopes: #{SCOPES.join(", ")}"
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
data/mkdocs.yml
CHANGED
|
@@ -92,7 +92,9 @@ nav:
|
|
|
92
92
|
- Targets: TARGETS.md
|
|
93
93
|
- Sources: SOURCES.md
|
|
94
94
|
- Column Mapping: MAPPING.md
|
|
95
|
+
- Views & Theming: VIEWS.md
|
|
95
96
|
- Routes: routes.md
|
|
97
|
+
- Advanced: ADVANCED.md
|
|
96
98
|
- Roadmap: ROADMAP.md
|
|
97
99
|
- Changelog: changelog.md
|
|
98
100
|
- Contributing: contributing.md
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: data_porter
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 2.
|
|
4
|
+
version: 2.5.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Seryl Lounis
|
|
@@ -106,12 +106,14 @@ files:
|
|
|
106
106
|
- app/assets/javascripts/data_porter/progress_controller.js
|
|
107
107
|
- app/assets/javascripts/data_porter/stimulus.min.js
|
|
108
108
|
- app/assets/javascripts/data_porter/template_form_controller.js
|
|
109
|
+
- app/assets/javascripts/data_porter/theme_controller.js
|
|
109
110
|
- app/assets/javascripts/data_porter/turbo.min.js
|
|
110
111
|
- app/assets/stylesheets/data_porter/alerts.css
|
|
111
112
|
- app/assets/stylesheets/data_porter/application.css
|
|
112
113
|
- app/assets/stylesheets/data_porter/badges.css
|
|
113
114
|
- app/assets/stylesheets/data_porter/base.css
|
|
114
115
|
- app/assets/stylesheets/data_porter/cards.css
|
|
116
|
+
- app/assets/stylesheets/data_porter/dark.css
|
|
115
117
|
- app/assets/stylesheets/data_porter/layout.css
|
|
116
118
|
- app/assets/stylesheets/data_porter/mapping.css
|
|
117
119
|
- app/assets/stylesheets/data_porter/modal.css
|
|
@@ -144,6 +146,7 @@ files:
|
|
|
144
146
|
- config/locales/fr.yml
|
|
145
147
|
- config/routes.rb
|
|
146
148
|
- lib/data_porter.rb
|
|
149
|
+
- lib/data_porter/auto_mapper.rb
|
|
147
150
|
- lib/data_porter/broadcaster.rb
|
|
148
151
|
- lib/data_porter/column_transformer.rb
|
|
149
152
|
- lib/data_porter/components.rb
|
|
@@ -165,6 +168,7 @@ files:
|
|
|
165
168
|
- lib/data_porter/dsl/webhook.rb
|
|
166
169
|
- lib/data_porter/engine.rb
|
|
167
170
|
- lib/data_porter/orchestrator.rb
|
|
171
|
+
- lib/data_porter/orchestrator/bulk_importer.rb
|
|
168
172
|
- lib/data_porter/orchestrator/dry_runner.rb
|
|
169
173
|
- lib/data_porter/orchestrator/importer.rb
|
|
170
174
|
- lib/data_porter/orchestrator/record_builder.rb
|
|
@@ -191,6 +195,7 @@ files:
|
|
|
191
195
|
- lib/generators/data_porter/locale/locale_generator.rb
|
|
192
196
|
- lib/generators/data_porter/target/target_generator.rb
|
|
193
197
|
- lib/generators/data_porter/target/templates/target.rb.tt
|
|
198
|
+
- lib/generators/data_porter/views/views_generator.rb
|
|
194
199
|
- lib/tasks/data_porter.rake
|
|
195
200
|
- mkdocs.yml
|
|
196
201
|
- sig/data_porter.rbs
|