data_porter 1.1.0 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +37 -0
- data/README.md +7 -14
- data/ROADMAP.md +84 -74
- data/app/assets/javascripts/data_porter/progress_controller.js +3 -3
- data/app/controllers/data_porter/concerns/import_validation.rb +5 -5
- data/app/controllers/data_porter/concerns/mapping_management.rb +8 -1
- data/app/controllers/data_porter/imports_controller.rb +7 -1
- data/app/models/data_porter/data_import.rb +9 -0
- data/app/views/data_porter/imports/index.html.erb +23 -23
- data/app/views/data_porter/imports/new.html.erb +11 -11
- data/app/views/data_porter/imports/show.html.erb +22 -18
- data/app/views/data_porter/mapping_templates/_form.html.erb +10 -10
- data/app/views/data_porter/mapping_templates/edit.html.erb +2 -2
- data/app/views/data_porter/mapping_templates/index.html.erb +10 -10
- data/app/views/data_porter/mapping_templates/new.html.erb +2 -2
- data/config/locales/en.yml +123 -0
- data/config/locales/fr.yml +123 -0
- data/config/routes.rb +4 -3
- data/lib/data_porter/components/mapping/column_row.rb +1 -1
- data/lib/data_porter/components/mapping/form.rb +4 -4
- data/lib/data_porter/components/mapping/template_select.rb +1 -1
- data/lib/data_porter/components/preview/results_summary.rb +13 -5
- data/lib/data_porter/components/preview/summary_cards.rb +5 -4
- data/lib/data_porter/components/preview/table.rb +3 -3
- data/lib/data_porter/components/progress/bar.rb +9 -2
- data/lib/data_porter/components/shared/pagination.rb +9 -5
- data/lib/data_porter/components/shared/status_badge.rb +3 -1
- data/lib/data_porter/configuration.rb +2 -2
- data/lib/data_porter/engine.rb +4 -0
- data/lib/data_porter/orchestrator/record_builder.rb +1 -1
- data/lib/data_porter/record_validator.rb +2 -2
- data/lib/data_porter/version.rb +1 -1
- data/lib/generators/data_porter/install/templates/initializer.rb +3 -2
- data/lib/generators/data_porter/locale/locale_generator.rb +42 -0
- data/mkdocs.yml +98 -0
- metadata +6 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: '04931dd3e74ae9a12b2225ddb1c5467022458013d49fb35a07282121296ec22b'
|
|
4
|
+
data.tar.gz: 4d0d04002509ae87358375f0df64bec0d8c134201131ead1a1a07460c9e19297
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 11ba16dcc818425722fc20e1a919833e4c4f77def52c07b46068707a1e511bb045bf7801edfb9b3186750fbf242a633d526e81e6b1bf3cd4b54b806c47e701aa
|
|
7
|
+
data.tar.gz: 4c99d6a1c4d04e4cee197199db9dd3517bc7d44b283e511a7289e46703ddf3d69f91a8cd7e4fb4f43ce5e8fdaaccbe805b5a649846d1c5e096a7803eafb07442
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,43 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [2.1.1] - 2026-02-20
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
|
|
12
|
+
- **Route ordering** -- `mapping_templates` routes declared before the catch-all `imports` resource to prevent `/mapping_templates` from matching `imports#show`
|
|
13
|
+
- **Locale generator** -- Fixed `source_root` resolution so `copy_file` finds locale templates correctly
|
|
14
|
+
- **Locale generator** -- Added post-generation instructions (next steps message)
|
|
15
|
+
|
|
16
|
+
## [2.1.0] - 2026-02-20
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **i18n** -- All UI strings, error messages, and status labels are now translatable via Rails I18n. Ships with English and French locales (~100 keys each)
|
|
21
|
+
- **Locale generator** -- `rails g data_porter:locale fr` copies a locale file with all keys pre-filled for translation
|
|
22
|
+
- **Documentation site** -- MkDocs Material site with GitHub Pages deployment, search, dark mode, and full API reference
|
|
23
|
+
- **Progress labels via data attributes** -- Stimulus progress controller reads translated labels from the server instead of hardcoded JS strings
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
|
|
27
|
+
- 438 RSpec examples (up from 423), 0 failures
|
|
28
|
+
|
|
29
|
+
## [2.0.0] - 2026-02-19
|
|
30
|
+
|
|
31
|
+
### Breaking
|
|
32
|
+
|
|
33
|
+
- **Default parent controller changed to `ActionController::Base`** -- Engine controllers no longer inherit from `ApplicationController` by default, avoiding conflicts with authorization gems (Pundit, CanCanCan, etc.). Set `config.parent_controller = "ApplicationController"` to restore the previous behavior
|
|
34
|
+
- **Engine routes mounted at root** -- `resources :imports` now uses `path: "/"` so the mount point controls the full URL (e.g. `mount DataPorter::Engine, at: "/imports"` gives `/imports`, not `/imports/imports`)
|
|
35
|
+
|
|
36
|
+
### Added
|
|
37
|
+
|
|
38
|
+
- **Back to mapping** -- `back_to_mapping` action resets a previewing import to the mapping step, preserving file headers and column mapping for re-mapping
|
|
39
|
+
- **Saved mapping persistence** -- Mapping form restores previously saved column mapping instead of resetting to defaults
|
|
40
|
+
|
|
41
|
+
### Changed
|
|
42
|
+
|
|
43
|
+
- 423 RSpec examples (up from 413), 0 failures
|
|
44
|
+
|
|
8
45
|
## [1.1.0] - 2026-02-08
|
|
9
46
|
|
|
10
47
|
### Added
|
data/README.md
CHANGED
|
@@ -1,23 +1,13 @@
|
|
|
1
1
|
# DataPorter
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
> APIs and features may change without notice. Use at your own risk.
|
|
3
|
+
[](https://badge.fury.io/rb/data_porter)
|
|
4
|
+
[](https://seryllns.github.io/data_porter/)
|
|
6
5
|
|
|
7
6
|
A mountable Rails engine for data import workflows: **Upload**, **Map**, **Preview**, **Import**.
|
|
8
7
|
|
|
9
8
|
Supports CSV, JSON, XLSX, and API sources with a declarative DSL for defining import targets. Business-agnostic by design -- all domain logic lives in your host app.
|
|
10
9
|
|
|
11
|
-
|
|
12
|
-
<tr>
|
|
13
|
-
<td><img src="docs/screenshots/index-with-previewing.jpg" width="400" alt="Import list with status badges" /></td>
|
|
14
|
-
<td><img src="docs/screenshots/modal-new-import.jpg" width="400" alt="New import modal with dropzone" /></td>
|
|
15
|
-
</tr>
|
|
16
|
-
<tr>
|
|
17
|
-
<td><img src="docs/screenshots/mapping.jpg" width="400" alt="Interactive column mapping with templates" /></td>
|
|
18
|
-
<td><img src="docs/screenshots/preview.jpg" width="400" alt="Preview with summary cards and data table" /></td>
|
|
19
|
-
</tr>
|
|
20
|
-
</table>
|
|
10
|
+

|
|
21
11
|
|
|
22
12
|
## Features
|
|
23
13
|
|
|
@@ -111,6 +101,8 @@ pending -> parsing -> previewing -> importing -> completed
|
|
|
111
101
|
|
|
112
102
|
## Documentation
|
|
113
103
|
|
|
104
|
+
**[Full documentation on GitHub Pages](https://seryllns.github.io/data_porter/)**
|
|
105
|
+
|
|
114
106
|
| Topic | Description |
|
|
115
107
|
|---|---|
|
|
116
108
|
| [Configuration](docs/CONFIGURATION.md) | All options, authentication, context builder, real-time updates |
|
|
@@ -132,6 +124,7 @@ pending -> parsing -> previewing -> importing -> completed
|
|
|
132
124
|
| POST | `/imports/:id/parse` | Parse source |
|
|
133
125
|
| POST | `/imports/:id/confirm` | Run import |
|
|
134
126
|
| POST | `/imports/:id/cancel` | Cancel import |
|
|
127
|
+
| POST | `/imports/:id/back_to_mapping` | Reset to mapping step |
|
|
135
128
|
| POST | `/imports/:id/dry_run` | Dry run validation |
|
|
136
129
|
| GET | `/imports/:id/export_rejects` | Download rejects CSV |
|
|
137
130
|
| | `/mapping_templates` | Full CRUD for templates |
|
|
@@ -142,7 +135,7 @@ pending -> parsing -> previewing -> importing -> completed
|
|
|
142
135
|
git clone https://github.com/SerylLns/data_porter.git
|
|
143
136
|
cd data_porter
|
|
144
137
|
bin/setup
|
|
145
|
-
bundle exec rspec #
|
|
138
|
+
bundle exec rspec # 423 specs
|
|
146
139
|
bundle exec rubocop # 0 offenses
|
|
147
140
|
```
|
|
148
141
|
|
data/ROADMAP.md
CHANGED
|
@@ -1,89 +1,99 @@
|
|
|
1
1
|
# Roadmap
|
|
2
2
|
|
|
3
|
-
##
|
|
4
|
-
|
|
5
|
-
###
|
|
6
|
-
|
|
7
|
-
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
- ~~Turbo Drive for instant page navigation~~
|
|
23
|
-
- ~~Import details card on show page~~
|
|
24
|
-
- ~~Improved template management UI~~
|
|
3
|
+
## Next
|
|
4
|
+
|
|
5
|
+
### Column transformers
|
|
6
|
+
|
|
7
|
+
Built-in transformation pipeline applied per-column before the target's `transform` method. Declarative DSL in the target:
|
|
8
|
+
|
|
9
|
+
```ruby
|
|
10
|
+
columns do
|
|
11
|
+
column :email, type: :string, transform: [:strip, :downcase]
|
|
12
|
+
column :phone, type: :string, transform: [:strip, :normalize_phone]
|
|
13
|
+
column :born_on, type: :date, transform: [:parse_date]
|
|
14
|
+
end
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Ships with common transformers (`strip`, `downcase`, `titleize`, `normalize_phone`, `parse_date`). Custom transformers via a registry.
|
|
18
|
+
|
|
19
|
+
### Webhooks
|
|
20
|
+
|
|
21
|
+
HTTP callbacks on import lifecycle events (started, completed, failed). Configurable per-target with URL, headers, and payload template. Enables integration with Slack notifications, CI pipelines, or external dashboards.
|
|
25
22
|
|
|
26
23
|
---
|
|
27
24
|
|
|
28
25
|
## Planned
|
|
29
26
|
|
|
30
|
-
###
|
|
27
|
+
### Bulk import
|
|
28
|
+
|
|
29
|
+
High-volume import support using `insert_all` / `upsert_all` for batch persistence. Opt-in per target to bypass per-record `persist` calls, enabling 10-100x throughput for simple create/upsert scenarios. Configurable batch size, with fallback to per-record mode on conflict.
|
|
31
30
|
|
|
32
|
-
|
|
33
|
-
- `ExportTarget` DSL mirroring the import Target
|
|
34
|
-
- Define query scope, columns, and output format (CSV, JSON, XLSX)
|
|
35
|
-
- Background job with progress bar (reuse existing ActionCable infrastructure)
|
|
36
|
-
- Download link on completion
|
|
31
|
+
### Update & diff mode
|
|
37
32
|
|
|
38
|
-
|
|
39
|
-
- Process large files in configurable batches (default: 1,000 records)
|
|
40
|
-
- Use `insert_all` / `upsert_all` for bulk persistence
|
|
41
|
-
- Granular progress: "12,000 / 150,000 records"
|
|
42
|
-
- Memory-efficient streaming parser for CSV and XLSX
|
|
33
|
+
Support update (upsert) imports alongside create-only. Given a `deduplicate_by` key, detect existing records and show a diff preview: new records, changed fields (highlighted), unchanged rows. User confirms which changes to apply. Enables recurring data sync workflows.
|
|
43
34
|
|
|
44
|
-
|
|
45
|
-
- Cron-like configuration in the Target DSL: `schedule "0 3 * * *"`
|
|
46
|
-
- Recurring API source imports (fetch external data on a timer)
|
|
47
|
-
- Dashboard for scheduled imports with last run status and next run time
|
|
48
|
-
- Built on ActiveJob + `solid_queue` or host app's queue adapter
|
|
35
|
+
### Resume / retry on failure
|
|
49
36
|
|
|
50
|
-
|
|
37
|
+
If an import fails mid-way (timeout, crash, transient error), resume from the last successful record instead of restarting from scratch. Track a checkpoint index in the report. Critical for large imports (5k+ records) where re-processing everything is not acceptable.
|
|
38
|
+
|
|
39
|
+
### API pagination
|
|
40
|
+
|
|
41
|
+
Support paginated API sources. The current API source does a single GET, which works for small datasets but not for APIs returning thousands of records across multiple pages. Support offset, cursor, and link-header pagination strategies via `api_config`:
|
|
51
42
|
|
|
52
|
-
#### Column Transformers
|
|
53
|
-
- Inline transform lambdas in the column DSL
|
|
54
|
-
- Built-in transformers: `downcase`, `strip`, `normalize_phone`, `parse_date`
|
|
55
43
|
```ruby
|
|
56
|
-
|
|
44
|
+
api_config do
|
|
45
|
+
endpoint "https://api.example.com/contacts"
|
|
46
|
+
pagination :cursor, param: "after", root: "data", next_key: "meta.next_cursor"
|
|
47
|
+
end
|
|
57
48
|
```
|
|
58
49
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
-
|
|
73
|
-
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
-
|
|
77
|
-
|
|
78
|
-
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
50
|
+
### Import API (REST)
|
|
51
|
+
|
|
52
|
+
Headless REST API for programmatic imports:
|
|
53
|
+
|
|
54
|
+
- `POST /api/imports` — create import (multipart file upload or JSON payload)
|
|
55
|
+
- `GET /api/imports/:id` — status + results
|
|
56
|
+
- Auth via `config.api_authenticate` lambda (API key or Bearer token)
|
|
57
|
+
- Reuses existing job pipeline (parse, import, dry run)
|
|
58
|
+
|
|
59
|
+
### View generator & theming
|
|
60
|
+
|
|
61
|
+
Customizable UI in two layers:
|
|
62
|
+
|
|
63
|
+
- **View generator** — `rails g data_porter:views` copies the 7 ERB templates into the host app for structural customization (layout, buttons, sections). Similar to `devise:views`.
|
|
64
|
+
- **CSS theming** — All styles use `--dp-*` custom properties. Host apps override variables to match their design system, no ERB changes needed.
|
|
65
|
+
- **Light / dark mode** — Two built-in presets toggled via `prefers-color-scheme` or a `.dp-dark` class.
|
|
66
|
+
|
|
67
|
+
### Auto-map heuristics
|
|
68
|
+
|
|
69
|
+
Smart column mapping suggestions using tokenized header matching and synonym dictionaries. When a CSV has "E-mail Address", auto-suggest mapping to `:email`. Built-in synonyms for common patterns (phone → phone_number, first name → first_name). Configurable synonym lists per target.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Ideas
|
|
74
|
+
|
|
75
|
+
### Export (reverse workflow)
|
|
76
|
+
|
|
77
|
+
`ExportTarget` DSL mirroring the import Target. Define query scope, columns, and output format (CSV, JSON, XLSX). Background job with progress bar and download link on completion.
|
|
78
|
+
|
|
79
|
+
### External connectors
|
|
80
|
+
|
|
81
|
+
Source plugins beyond local files and HTTP APIs:
|
|
82
|
+
|
|
83
|
+
- **Google Sheets** — OAuth2 + Sheets API, treat a spreadsheet as a source
|
|
84
|
+
- **SFTP** — Poll a remote directory for new files
|
|
85
|
+
- **AWS S3** — Watch a bucket/prefix for uploads
|
|
86
|
+
|
|
87
|
+
Each connector implements the `Sources::Base` interface. Installed as optional companion gems (`data_porter-google_sheets`, `data_porter-s3`).
|
|
88
|
+
|
|
89
|
+
### Scheduled imports
|
|
90
|
+
|
|
91
|
+
Recurring imports from API or remote sources on a cron schedule. A target declares a schedule, and DataPorter automatically fetches and imports at the configured interval. Built on ActiveJob with configurable queue.
|
|
92
|
+
|
|
93
|
+
### Rollback
|
|
94
|
+
|
|
95
|
+
Undo a completed import by soft-deleting the created records. Confirmation step with summary of records to be reverted.
|
|
96
|
+
|
|
97
|
+
### Dashboard & analytics
|
|
98
|
+
|
|
99
|
+
Import statistics dashboard: success rates, average duration, records per import, most-used targets, failure trends. Mountable as an admin-only route.
|
|
@@ -2,7 +2,7 @@ import { Controller } from "@hotwired/stimulus"
|
|
|
2
2
|
|
|
3
3
|
export default class extends Controller {
|
|
4
4
|
static targets = ["bar", "text", "label"]
|
|
5
|
-
static values = { id: Number, url: String }
|
|
5
|
+
static values = { id: Number, url: String, labels: Object }
|
|
6
6
|
|
|
7
7
|
connect() {
|
|
8
8
|
this.poll()
|
|
@@ -32,8 +32,8 @@ export default class extends Controller {
|
|
|
32
32
|
|
|
33
33
|
updateLabel(status) {
|
|
34
34
|
if (!this.hasLabelTarget) return
|
|
35
|
-
var labels =
|
|
36
|
-
this.labelTarget.textContent = labels[status] || "
|
|
35
|
+
var labels = this.hasLabelsValue ? this.labelsValue : {}
|
|
36
|
+
this.labelTarget.textContent = labels[status] || labels["processing"] || status
|
|
37
37
|
}
|
|
38
38
|
|
|
39
39
|
disconnect() {
|
|
@@ -18,7 +18,7 @@ module DataPorter
|
|
|
18
18
|
allowed = target._sources || DataPorter.configuration.enabled_sources
|
|
19
19
|
return true if allowed.map(&:to_s).include?(@import.source_type.to_s)
|
|
20
20
|
|
|
21
|
-
@import.errors.add(:source_type, "
|
|
21
|
+
@import.errors.add(:source_type, I18n.t("data_porter.errors.source_unavailable", source: @import.source_type))
|
|
22
22
|
false
|
|
23
23
|
end
|
|
24
24
|
|
|
@@ -26,7 +26,7 @@ module DataPorter
|
|
|
26
26
|
return true unless %w[csv json xlsx].include?(@import.source_type)
|
|
27
27
|
return true if @import.file.attached?
|
|
28
28
|
|
|
29
|
-
@import.errors.add(:file, "
|
|
29
|
+
@import.errors.add(:file, I18n.t("data_porter.errors.file_required", source: @import.source_type.upcase))
|
|
30
30
|
false
|
|
31
31
|
end
|
|
32
32
|
|
|
@@ -34,7 +34,7 @@ module DataPorter
|
|
|
34
34
|
missing = missing_required_params
|
|
35
35
|
return true if missing.empty?
|
|
36
36
|
|
|
37
|
-
missing.each { |p| @import.errors.add(:base, "
|
|
37
|
+
missing.each { |p| @import.errors.add(:base, I18n.t("data_porter.errors.required", label: p.label)) }
|
|
38
38
|
false
|
|
39
39
|
end
|
|
40
40
|
|
|
@@ -55,7 +55,7 @@ module DataPorter
|
|
|
55
55
|
max = DataPorter.configuration.max_file_size
|
|
56
56
|
return true if @import.file.blob.byte_size <= max
|
|
57
57
|
|
|
58
|
-
@import.errors.add(:file, "
|
|
58
|
+
@import.errors.add(:file, I18n.t("data_porter.errors.file_too_large", max: max / 1.megabyte))
|
|
59
59
|
false
|
|
60
60
|
end
|
|
61
61
|
|
|
@@ -68,7 +68,7 @@ module DataPorter
|
|
|
68
68
|
content_type = @import.file.blob.content_type
|
|
69
69
|
return true if allowed.include?(content_type)
|
|
70
70
|
|
|
71
|
-
@import.errors.add(:file, "
|
|
71
|
+
@import.errors.add(:file, I18n.t("data_porter.errors.invalid_content_type", type: content_type))
|
|
72
72
|
false
|
|
73
73
|
end
|
|
74
74
|
end
|
|
@@ -12,7 +12,7 @@ module DataPorter
|
|
|
12
12
|
columns = target._columns || []
|
|
13
13
|
@file_headers = @import.config["file_headers"] || []
|
|
14
14
|
@target_columns = columns.map { |c| [c.label, c.name.to_s, c.required] }
|
|
15
|
-
@default_mapping = (target
|
|
15
|
+
@default_mapping = saved_or_default_mapping(target)
|
|
16
16
|
@templates = load_templates
|
|
17
17
|
end
|
|
18
18
|
|
|
@@ -23,6 +23,13 @@ module DataPorter
|
|
|
23
23
|
scope.for_target(@import.target_key)
|
|
24
24
|
end
|
|
25
25
|
|
|
26
|
+
def saved_or_default_mapping(target)
|
|
27
|
+
saved = @import.config&.dig("column_mapping")
|
|
28
|
+
return saved if saved.present?
|
|
29
|
+
|
|
30
|
+
(target._csv_mappings || {}).transform_values(&:to_s)
|
|
31
|
+
end
|
|
32
|
+
|
|
26
33
|
def save_column_mapping
|
|
27
34
|
merged = (@import.config || {}).merge("column_mapping" => permitted_column_mapping)
|
|
28
35
|
@import.update!(config: merged, status: :pending)
|
|
@@ -9,7 +9,8 @@ module DataPorter
|
|
|
9
9
|
|
|
10
10
|
layout "data_porter/application"
|
|
11
11
|
|
|
12
|
-
before_action :set_import, only: %i[show parse confirm cancel dry_run update_mapping
|
|
12
|
+
before_action :set_import, only: %i[show parse confirm cancel dry_run update_mapping
|
|
13
|
+
status export_rejects destroy back_to_mapping]
|
|
13
14
|
before_action :load_targets, only: %i[index new create]
|
|
14
15
|
|
|
15
16
|
def index
|
|
@@ -63,6 +64,11 @@ module DataPorter
|
|
|
63
64
|
redirect_to imports_path
|
|
64
65
|
end
|
|
65
66
|
|
|
67
|
+
def back_to_mapping
|
|
68
|
+
@import.reset_to_mapping!
|
|
69
|
+
redirect_to import_path(@import)
|
|
70
|
+
end
|
|
71
|
+
|
|
66
72
|
def dry_run
|
|
67
73
|
@import.update!(status: :pending)
|
|
68
74
|
DataPorter::DryRunJob.perform_later(@import.id)
|
|
@@ -53,6 +53,15 @@ module DataPorter
|
|
|
53
53
|
records.group_by(&:status).transform_values(&:count)
|
|
54
54
|
end
|
|
55
55
|
|
|
56
|
+
def reset_to_mapping!
|
|
57
|
+
update!(
|
|
58
|
+
status: :mapping,
|
|
59
|
+
records: [],
|
|
60
|
+
report: StoreModels::Report.new,
|
|
61
|
+
config: (config || {}).except("progress")
|
|
62
|
+
)
|
|
63
|
+
end
|
|
64
|
+
|
|
56
65
|
def file_based?
|
|
57
66
|
%w[csv xlsx].include?(source_type)
|
|
58
67
|
end
|
|
@@ -3,11 +3,11 @@
|
|
|
3
3
|
data-data-porter--import-form-params-value="<%= @targets.map { |t| [t[:key], t[:params]] }.to_h.to_json %>"
|
|
4
4
|
data-action="keydown@document->data-porter--import-form#closeModal">
|
|
5
5
|
<div class="dp-header">
|
|
6
|
-
<h1 class="dp-title"
|
|
6
|
+
<h1 class="dp-title"><%= t("data_porter.imports.title") %></h1>
|
|
7
7
|
<div class="dp-header__actions">
|
|
8
|
-
<%= link_to "
|
|
8
|
+
<%= link_to t("data_porter.mapping_templates.title"), mapping_templates_path, class: "dp-btn dp-btn--secondary" %>
|
|
9
9
|
<button type="button" class="dp-btn dp-btn--primary" data-action="data-porter--import-form#openModal">
|
|
10
|
-
|
|
10
|
+
<%= t("data_porter.imports.new_import") %>
|
|
11
11
|
</button>
|
|
12
12
|
</div>
|
|
13
13
|
</div>
|
|
@@ -16,11 +16,11 @@
|
|
|
16
16
|
<table class="dp-table">
|
|
17
17
|
<thead>
|
|
18
18
|
<tr>
|
|
19
|
-
<th
|
|
20
|
-
<th
|
|
21
|
-
<th
|
|
22
|
-
<th
|
|
23
|
-
<th
|
|
19
|
+
<th><%= t("data_porter.imports.table.id") %></th>
|
|
20
|
+
<th><%= t("data_porter.imports.table.target") %></th>
|
|
21
|
+
<th><%= t("data_porter.imports.table.source") %></th>
|
|
22
|
+
<th><%= t("data_porter.imports.table.status") %></th>
|
|
23
|
+
<th><%= t("data_porter.imports.table.created") %></th>
|
|
24
24
|
<th></th>
|
|
25
25
|
</tr>
|
|
26
26
|
</thead>
|
|
@@ -33,11 +33,11 @@
|
|
|
33
33
|
<td><%= raw DataPorter::Components::Shared::StatusBadge.new(status: import.status).call %></td>
|
|
34
34
|
<td><%= import.created_at&.strftime("%Y-%m-%d %H:%M") %></td>
|
|
35
35
|
<td class="dp-table__actions">
|
|
36
|
-
<%= link_to "
|
|
36
|
+
<%= link_to t("data_porter.imports.view"), import_path(import), class: "dp-btn dp-btn--sm dp-btn--secondary" %>
|
|
37
37
|
<% if import.completed? || import.failed? %>
|
|
38
|
-
<%= button_to "
|
|
38
|
+
<%= button_to t("data_porter.imports.delete"), import_path(import),
|
|
39
39
|
method: :delete, class: "dp-btn dp-btn--sm dp-btn--danger",
|
|
40
|
-
data: { turbo_confirm: "
|
|
40
|
+
data: { turbo_confirm: t("data_porter.imports.delete_confirm") } %>
|
|
41
41
|
<% end %>
|
|
42
42
|
</td>
|
|
43
43
|
</tr>
|
|
@@ -47,9 +47,9 @@
|
|
|
47
47
|
<% else %>
|
|
48
48
|
<div class="dp-empty-state">
|
|
49
49
|
<div class="dp-empty-state__icon">📦</div>
|
|
50
|
-
<p class="dp-empty-state__text"
|
|
50
|
+
<p class="dp-empty-state__text"><%= t("data_porter.imports.no_imports") %></p>
|
|
51
51
|
<button type="button" class="dp-btn dp-btn--primary" data-action="data-porter--import-form#openModal">
|
|
52
|
-
|
|
52
|
+
<%= t("data_porter.imports.create_first") %>
|
|
53
53
|
</button>
|
|
54
54
|
</div>
|
|
55
55
|
<% end %>
|
|
@@ -58,16 +58,16 @@
|
|
|
58
58
|
<div class="dp-modal__backdrop" data-action="click->data-porter--import-form#closeModalClick"></div>
|
|
59
59
|
<div class="dp-modal__content">
|
|
60
60
|
<div class="dp-modal__header">
|
|
61
|
-
<h2 class="dp-modal__title"
|
|
61
|
+
<h2 class="dp-modal__title"><%= t("data_porter.imports.new_import") %></h2>
|
|
62
62
|
<button type="button" class="dp-modal__close" data-action="data-porter--import-form#closeModalClick">×</button>
|
|
63
63
|
</div>
|
|
64
64
|
|
|
65
65
|
<%= form_with model: DataPorter::DataImport.new, url: imports_path, class: "dp-modal__body", multipart: true, data: { turbo: false } do |f| %>
|
|
66
66
|
<div class="dp-field">
|
|
67
|
-
<%= f.label :target_key, "
|
|
67
|
+
<%= f.label :target_key, t("data_porter.imports.form.target_label"), class: "dp-label" %>
|
|
68
68
|
<%= f.select :target_key,
|
|
69
69
|
@targets.map { |t| [t[:label], t[:key]] },
|
|
70
|
-
{ prompt: "
|
|
70
|
+
{ prompt: t("data_porter.imports.form.target_prompt") },
|
|
71
71
|
class: "dp-select",
|
|
72
72
|
data: {
|
|
73
73
|
data_porter__import_form_target: "targetSelect",
|
|
@@ -78,10 +78,10 @@
|
|
|
78
78
|
<div data-data-porter--import-form-target="paramsContainer"></div>
|
|
79
79
|
|
|
80
80
|
<div class="dp-field">
|
|
81
|
-
<%= f.label :source_type, "
|
|
81
|
+
<%= f.label :source_type, t("data_porter.imports.form.source_label"), class: "dp-label" %>
|
|
82
82
|
<%= f.select :source_type,
|
|
83
83
|
DataPorter.configuration.enabled_sources.map { |s| [s.to_s.upcase, s] },
|
|
84
|
-
{ prompt: "
|
|
84
|
+
{ prompt: t("data_porter.imports.form.source_prompt") },
|
|
85
85
|
class: "dp-select",
|
|
86
86
|
data: {
|
|
87
87
|
data_porter__import_form_target: "sourceSelect",
|
|
@@ -90,7 +90,7 @@
|
|
|
90
90
|
</div>
|
|
91
91
|
|
|
92
92
|
<div class="dp-field" data-data-porter--import-form-target="fileField">
|
|
93
|
-
<%= f.label :file, "
|
|
93
|
+
<%= f.label :file, t("data_porter.imports.form.file_label"), class: "dp-label" %>
|
|
94
94
|
<label class="dp-dropzone" data-data-porter--import-form-target="dropzone"
|
|
95
95
|
data-action="dragover->data-porter--import-form#dragover dragleave->data-porter--import-form#dragleave drop->data-porter--import-form#drop">
|
|
96
96
|
<input type="file" name="data_import[file]" class="dp-dropzone__input"
|
|
@@ -98,16 +98,16 @@
|
|
|
98
98
|
data-action="data-porter--import-form#handleFile" />
|
|
99
99
|
<div class="dp-dropzone__content">
|
|
100
100
|
<div class="dp-dropzone__icon">📄</div>
|
|
101
|
-
<span class="dp-dropzone__text"
|
|
102
|
-
<span class="dp-dropzone__hint"
|
|
101
|
+
<span class="dp-dropzone__text"><%= raw t("data_porter.imports.form.dropzone_text") %></span>
|
|
102
|
+
<span class="dp-dropzone__hint"><%= t("data_porter.imports.form.dropzone_hint") %></span>
|
|
103
103
|
</div>
|
|
104
104
|
<div class="dp-dropzone__selected" data-data-porter--import-form-target="fileName" style="display: none;"></div>
|
|
105
105
|
</label>
|
|
106
106
|
</div>
|
|
107
107
|
|
|
108
108
|
<div class="dp-modal__footer">
|
|
109
|
-
<%= f.submit "
|
|
110
|
-
<button type="button" class="dp-btn dp-btn--secondary" data-action="data-porter--import-form#closeModalClick"
|
|
109
|
+
<%= f.submit t("data_porter.imports.start_import"), class: "dp-btn dp-btn--primary" %>
|
|
110
|
+
<button type="button" class="dp-btn dp-btn--secondary" data-action="data-porter--import-form#closeModalClick"><%= t("data_porter.imports.cancel") %></button>
|
|
111
111
|
</div>
|
|
112
112
|
<% end %>
|
|
113
113
|
</div>
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
<div class="data-porter">
|
|
2
2
|
<div class="dp-header">
|
|
3
|
-
<h1 class="dp-title"
|
|
3
|
+
<h1 class="dp-title"><%= t("data_porter.imports.new_title") %></h1>
|
|
4
4
|
</div>
|
|
5
5
|
|
|
6
6
|
<% if @import.errors.any? %>
|
|
@@ -13,10 +13,10 @@
|
|
|
13
13
|
|
|
14
14
|
<%= form_with model: @import, url: imports_path, class: "dp-form", multipart: true, data: { turbo: false } do |f| %>
|
|
15
15
|
<div class="dp-field">
|
|
16
|
-
<%= f.label :target_key, "
|
|
16
|
+
<%= f.label :target_key, t("data_porter.imports.form.target_label"), class: "dp-label" %>
|
|
17
17
|
<%= f.select :target_key,
|
|
18
18
|
@targets.map { |t| [t[:label], t[:key]] },
|
|
19
|
-
{ prompt: "
|
|
19
|
+
{ prompt: t("data_porter.imports.form.target_prompt") },
|
|
20
20
|
id: "dp-target-select-new",
|
|
21
21
|
class: "dp-select",
|
|
22
22
|
data: {
|
|
@@ -28,30 +28,30 @@
|
|
|
28
28
|
<div id="dp-params-container" class="dp-params-container"></div>
|
|
29
29
|
|
|
30
30
|
<div class="dp-field">
|
|
31
|
-
<%= f.label :source_type, "
|
|
31
|
+
<%= f.label :source_type, t("data_porter.imports.form.source_label"), class: "dp-label" %>
|
|
32
32
|
<%= f.select :source_type,
|
|
33
33
|
DataPorter.configuration.enabled_sources.map { |s| [s.to_s.upcase, s] },
|
|
34
|
-
{ prompt: "
|
|
34
|
+
{ prompt: t("data_porter.imports.form.source_prompt") },
|
|
35
35
|
id: "dp-source-select-new",
|
|
36
36
|
class: "dp-select" %>
|
|
37
37
|
</div>
|
|
38
38
|
|
|
39
39
|
<div id="dp-file-field-new" class="dp-field">
|
|
40
|
-
<%= f.label :file, "
|
|
40
|
+
<%= f.label :file, t("data_porter.imports.form.file_label"), class: "dp-label" %>
|
|
41
41
|
<label class="dp-dropzone" id="dp-dropzone-new">
|
|
42
42
|
<input type="file" name="data_import[file]" id="dp-file-input-new" class="dp-dropzone__input" />
|
|
43
43
|
<div class="dp-dropzone__content">
|
|
44
44
|
<div class="dp-dropzone__icon">📄</div>
|
|
45
|
-
<span class="dp-dropzone__text"
|
|
46
|
-
<span class="dp-dropzone__hint"
|
|
45
|
+
<span class="dp-dropzone__text"><%= raw t("data_porter.imports.form.dropzone_text") %></span>
|
|
46
|
+
<span class="dp-dropzone__hint"><%= t("data_porter.imports.form.dropzone_hint") %></span>
|
|
47
47
|
</div>
|
|
48
48
|
<div class="dp-dropzone__selected" id="dp-file-name-new" style="display: none;"></div>
|
|
49
49
|
</label>
|
|
50
50
|
</div>
|
|
51
51
|
|
|
52
52
|
<div class="dp-actions">
|
|
53
|
-
<%= f.submit "
|
|
54
|
-
<%= link_to "
|
|
53
|
+
<%= f.submit t("data_porter.imports.start_import"), class: "dp-btn dp-btn--primary" %>
|
|
54
|
+
<%= link_to t("data_porter.imports.cancel"), imports_path, class: "dp-btn dp-btn--secondary" %>
|
|
55
55
|
</div>
|
|
56
56
|
<% end %>
|
|
57
57
|
</div>
|
|
@@ -84,7 +84,7 @@
|
|
|
84
84
|
input.className = "dp-select";
|
|
85
85
|
var blank = document.createElement("option");
|
|
86
86
|
blank.value = "";
|
|
87
|
-
blank.textContent = "
|
|
87
|
+
blank.textContent = "<%= j t("data_porter.imports.form.select_prompt") %>";
|
|
88
88
|
input.appendChild(blank);
|
|
89
89
|
p.collection.forEach(function(opt) {
|
|
90
90
|
var o = document.createElement("option");
|