data_porter 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (113) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +13 -1
  3. data/README.md +15 -5
  4. data/ROADMAP.md +71 -0
  5. data/app/models/data_porter/data_import.rb +1 -1
  6. data/app/views/data_porter/imports/index.html.erb +1 -1
  7. data/app/views/data_porter/imports/new.html.erb +1 -1
  8. data/lib/data_porter/configuration.rb +1 -1
  9. data/lib/data_porter/sources/xlsx.rb +68 -0
  10. data/lib/data_porter/sources.rb +3 -1
  11. data/lib/data_porter/version.rb +1 -1
  12. metadata +19 -104
  13. data/.claude/commands/blog-status.md +0 -10
  14. data/.claude/commands/blog.md +0 -109
  15. data/.claude/commands/task-done.md +0 -27
  16. data/.claude/commands/tm/add-dependency.md +0 -58
  17. data/.claude/commands/tm/add-subtask.md +0 -79
  18. data/.claude/commands/tm/add-task.md +0 -81
  19. data/.claude/commands/tm/analyze-complexity.md +0 -124
  20. data/.claude/commands/tm/analyze-project.md +0 -100
  21. data/.claude/commands/tm/auto-implement-tasks.md +0 -100
  22. data/.claude/commands/tm/command-pipeline.md +0 -80
  23. data/.claude/commands/tm/complexity-report.md +0 -120
  24. data/.claude/commands/tm/convert-task-to-subtask.md +0 -74
  25. data/.claude/commands/tm/expand-all-tasks.md +0 -52
  26. data/.claude/commands/tm/expand-task.md +0 -52
  27. data/.claude/commands/tm/fix-dependencies.md +0 -82
  28. data/.claude/commands/tm/help.md +0 -101
  29. data/.claude/commands/tm/init-project-quick.md +0 -49
  30. data/.claude/commands/tm/init-project.md +0 -53
  31. data/.claude/commands/tm/install-taskmaster.md +0 -118
  32. data/.claude/commands/tm/learn.md +0 -106
  33. data/.claude/commands/tm/list-tasks-by-status.md +0 -42
  34. data/.claude/commands/tm/list-tasks-with-subtasks.md +0 -30
  35. data/.claude/commands/tm/list-tasks.md +0 -46
  36. data/.claude/commands/tm/next-task.md +0 -69
  37. data/.claude/commands/tm/parse-prd-with-research.md +0 -51
  38. data/.claude/commands/tm/parse-prd.md +0 -52
  39. data/.claude/commands/tm/project-status.md +0 -67
  40. data/.claude/commands/tm/quick-install-taskmaster.md +0 -23
  41. data/.claude/commands/tm/remove-all-subtasks.md +0 -94
  42. data/.claude/commands/tm/remove-dependency.md +0 -65
  43. data/.claude/commands/tm/remove-subtask.md +0 -87
  44. data/.claude/commands/tm/remove-subtasks.md +0 -89
  45. data/.claude/commands/tm/remove-task.md +0 -110
  46. data/.claude/commands/tm/setup-models.md +0 -52
  47. data/.claude/commands/tm/show-task.md +0 -85
  48. data/.claude/commands/tm/smart-workflow.md +0 -58
  49. data/.claude/commands/tm/sync-readme.md +0 -120
  50. data/.claude/commands/tm/tm-main.md +0 -147
  51. data/.claude/commands/tm/to-cancelled.md +0 -58
  52. data/.claude/commands/tm/to-deferred.md +0 -50
  53. data/.claude/commands/tm/to-done.md +0 -47
  54. data/.claude/commands/tm/to-in-progress.md +0 -39
  55. data/.claude/commands/tm/to-pending.md +0 -35
  56. data/.claude/commands/tm/to-review.md +0 -43
  57. data/.claude/commands/tm/update-single-task.md +0 -122
  58. data/.claude/commands/tm/update-task.md +0 -75
  59. data/.claude/commands/tm/update-tasks-from-id.md +0 -111
  60. data/.claude/commands/tm/validate-dependencies.md +0 -72
  61. data/.claude/commands/tm/view-models.md +0 -52
  62. data/.env.example +0 -12
  63. data/.mcp.json +0 -24
  64. data/.taskmaster/CLAUDE.md +0 -435
  65. data/.taskmaster/config.json +0 -44
  66. data/.taskmaster/docs/prd.txt +0 -2044
  67. data/.taskmaster/state.json +0 -6
  68. data/.taskmaster/tasks/task_001.md +0 -19
  69. data/.taskmaster/tasks/task_002.md +0 -19
  70. data/.taskmaster/tasks/task_003.md +0 -19
  71. data/.taskmaster/tasks/task_004.md +0 -19
  72. data/.taskmaster/tasks/task_005.md +0 -19
  73. data/.taskmaster/tasks/task_006.md +0 -19
  74. data/.taskmaster/tasks/task_007.md +0 -19
  75. data/.taskmaster/tasks/task_008.md +0 -19
  76. data/.taskmaster/tasks/task_009.md +0 -19
  77. data/.taskmaster/tasks/task_010.md +0 -19
  78. data/.taskmaster/tasks/task_011.md +0 -19
  79. data/.taskmaster/tasks/task_012.md +0 -19
  80. data/.taskmaster/tasks/task_013.md +0 -19
  81. data/.taskmaster/tasks/task_014.md +0 -19
  82. data/.taskmaster/tasks/task_015.md +0 -19
  83. data/.taskmaster/tasks/task_016.md +0 -19
  84. data/.taskmaster/tasks/task_017.md +0 -19
  85. data/.taskmaster/tasks/task_018.md +0 -19
  86. data/.taskmaster/tasks/task_019.md +0 -19
  87. data/.taskmaster/tasks/task_020.md +0 -19
  88. data/.taskmaster/tasks/tasks.json +0 -299
  89. data/.taskmaster/templates/example_prd.txt +0 -47
  90. data/.taskmaster/templates/example_prd_rpg.txt +0 -511
  91. data/CLAUDE.md +0 -65
  92. data/config/database.yml +0 -3
  93. data/docs/SPEC.md +0 -2012
  94. data/docs/UI.md +0 -32
  95. data/docs/blog/001-why-build-a-data-import-engine.md +0 -166
  96. data/docs/blog/002-scaffolding-a-rails-engine.md +0 -188
  97. data/docs/blog/003-configuration-dsl.md +0 -222
  98. data/docs/blog/004-store-model-jsonb.md +0 -237
  99. data/docs/blog/005-target-dsl.md +0 -284
  100. data/docs/blog/006-parsing-csv-sources.md +0 -300
  101. data/docs/blog/007-orchestrator.md +0 -247
  102. data/docs/blog/008-actioncable-stimulus.md +0 -376
  103. data/docs/blog/009-phlex-ui-components.md +0 -446
  104. data/docs/blog/010-controllers-routing.md +0 -374
  105. data/docs/blog/011-generators.md +0 -364
  106. data/docs/blog/012-json-api-sources.md +0 -323
  107. data/docs/blog/013-testing-rails-engine.md +0 -618
  108. data/docs/blog/014-dry-run.md +0 -307
  109. data/docs/blog/015-publishing-retro.md +0 -264
  110. data/docs/blog/016-erb-view-templates.md +0 -431
  111. data/docs/blog/017-showcase-final-retro.md +0 -220
  112. data/docs/blog/BACKLOG.md +0 -8
  113. data/docs/blog/SERIES.md +0 -154
data/docs/blog/SERIES.md DELETED
@@ -1,154 +0,0 @@
1
- ---
2
- series: "Building DataPorter - A Data Import Engine for Rails"
3
- author: ""
4
- repo: ""
5
- status: complete
6
- ---
7
-
8
- # Series Plan
9
-
10
- ## Target audience
11
- Ruby/Rails developers (intermediate+) interested in gem architecture, Rails engines, and clean DSL design.
12
-
13
- ## Series structure
14
- Each article: ~5-8 min read, one clear concept, real code, decisions explained.
15
-
16
- ---
17
-
18
- ## Articles
19
-
20
- ### Part 1 — Why build a data import engine?
21
- - **Tasks:** None (intro)
22
- - **Hook:** The repetitive import pattern in Rails apps
23
- - **Covers:** Problem statement, existing solutions (maintenance_tasks, activeadmin_import), why a new gem, what we'll build (3-step workflow diagram)
24
- - **Key decisions:** Mountable engine vs. lib, isolate_namespace, target audience
25
- - **File:** `docs/blog/001-why-build-a-data-import-engine.md`
26
- - **Status:** draft
27
-
28
- ### Part 2 — Scaffolding a Rails Engine gem
29
- - **Tasks:** #1
30
- - **Hook:** `bundle gem` is just the start
31
- - **Covers:** Gem structure, Engine setup, isolate_namespace, directory tree, gemspec dependencies
32
- - **Key decisions:** Why isolate_namespace, directory conventions, dependency choices (store_model, phlex, turbo)
33
- - **File:** `docs/blog/002-scaffolding-a-rails-engine.md`
34
- - **Status:** draft
35
-
36
- ### Part 3 — Configuration DSL: making the gem flexible
37
- - **Tasks:** #2
38
- - **Hook:** A good gem adapts to any host app
39
- - **Covers:** Configuration singleton pattern, DSL with `configure` block, sensible defaults, context_builder lambda
40
- - **Key decisions:** Why not Rails config, lambda vs block for context, what to make configurable vs. convention
41
- - **File:** `docs/blog/003-configuration-dsl.md`
42
- - **Status:** draft
43
-
44
- ### Part 4 — Modeling import data with StoreModel & JSONB
45
- - **Tasks:** #3, #4
46
- - **Hook:** Storing structured data without extra tables
47
- - **Covers:** StoreModel gem, ImportRecord/Error/Report models, JSONB attributes, TypeValidator
48
- - **Key decisions:** JSONB vs. separate tables, store_model vs. hand-rolled, validation strategy (column-level vs DB-level)
49
- - **File:** `docs/blog/004-store-model-jsonb.md`
50
- - **Status:** draft
51
-
52
- ### Part 5 — Designing a Target DSL
53
- - **Tasks:** #5, #6
54
- - **Hook:** One file = one import type, zero boilerplate
55
- - **Covers:** Target base class, class-level DSL (label, model, columns, csv_mapping), Registry pattern, auto-discovery
56
- - **Key decisions:** DSL design (class methods vs. instance), class_attribute vs. class instance vars, hook pattern for extensibility
57
- - **File:** `docs/blog/005-target-dsl.md`
58
- - **Status:** draft
59
-
60
- ### Part 6 — Parsing CSV data with Sources
61
- - **Tasks:** #7, #8
62
- - **Hook:** The first end-to-end flow
63
- - **Covers:** DataImport model, migration, Source base class, CSV source, ActiveStorage integration, column mapping
64
- - **Key decisions:** Polymorphic user association, enum state machine, auto-mapping vs explicit mapping
65
- - **File:** `docs/blog/006-parsing-csv-sources.md`
66
- - **Status:** draft
67
-
68
- ### Part 7 — The Orchestrator: coordinating the import workflow
69
- - **Tasks:** #9, #10
70
- - **Hook:** The brain of the engine
71
- - **Covers:** Orchestrator class (parse! and import!), state transitions, error handling per-record, ActiveJob integration
72
- - **Key decisions:** Why an orchestrator (not controller logic), transaction boundaries, error recovery strategy
73
- - **File:** `docs/blog/007-orchestrator.md`
74
- - **Status:** draft
75
-
76
- ### Part 8 — Real-time progress with ActionCable & Stimulus
77
- - **Tasks:** #11, #14
78
- - **Hook:** Users hate staring at a spinner with no feedback
79
- - **Covers:** Broadcaster service, ImportChannel, Stimulus controller, progress bar updates, auto-reload on completion
80
- - **Key decisions:** ActionCable vs. polling vs. SSE, channel naming strategy, Stimulus vs. Turbo Streams
81
- - **File:** `docs/blog/008-actioncable-stimulus.md`
82
- - **Status:** draft
83
-
84
- ### Part 9 — Building the UI with Phlex & Tailwind
85
- - **Tasks:** #13
86
- - **Hook:** Auto-generated preview tables from a DSL
87
- - **Covers:** Phlex components, scoped Tailwind (dp- prefix), preview table with dynamic columns, status badges, CSS custom properties for theming
88
- - **Key decisions:** Phlex vs. ERB/ViewComponent, Tailwind prefix strategy, how to not pollute host app styles
89
- - **File:** `docs/blog/009-phlex-ui-components.md`
90
- - **Status:** draft
91
-
92
- ### Part 10 — Controllers & routing in a Rails Engine
93
- - **Tasks:** #12
94
- - **Hook:** Engine controllers are tricky — here's the clean way
95
- - **Covers:** ImportsController, inheriting from host's parent controller, engine routes, strong params, Turbo integration
96
- - **Key decisions:** Dynamic parent controller inheritance, route namespacing, how auth flows from host to engine
97
- - **File:** `docs/blog/010-controllers-routing.md`
98
- - **Status:** draft
99
-
100
- ### Part 11 — Generators: install & target scaffolding
101
- - **Tasks:** #15, #16
102
- - **Hook:** Great gems install in one command
103
- - **Covers:** Install generator (migration, initializer, routes), Target generator (column parsing, template), Rails::Generators API
104
- - **Key decisions:** What to generate vs. what to configure, template format (ERB .tt), route injection strategy
105
- - **File:** `docs/blog/011-generators.md`
106
- - **Status:** draft
107
-
108
- ### Part 12 — Adding JSON & API sources
109
- - **Tasks:** #18, #19
110
- - **Hook:** CSV is just the beginning
111
- - **Covers:** JSON source (file + raw text), API source (HTTP client, params injection), source plugin architecture
112
- - **Key decisions:** Source abstraction design, how to handle auth headers, response_root extraction
113
- - **File:** `docs/blog/012-json-api-sources.md`
114
- - **Status:** draft
115
-
116
- ### Part 13 — Testing a Rails Engine with RSpec
117
- - **Tasks:** #17
118
- - **Hook:** Testing an engine is different from testing an app
119
- - **Covers:** Dummy app setup, spec organization, testing StoreModels, mocking ActiveStorage, testing jobs, controller specs
120
- - **Key decisions:** Dummy app vs. shared examples, factory vs. fixture, what to unit test vs. integration test
121
- - **File:** `docs/blog/013-testing-rails-engine.md`
122
- - **Status:** draft
123
-
124
- ### Part 14 — Dry Run: validate before you persist
125
- - **Tasks:** #20
126
- - **Hook:** Preview catches column errors, dry-run catches DB errors
127
- - **Covers:** Transaction + rollback pattern, enriching records with DB errors, DryRunJob, UI integration
128
- - **Key decisions:** Why two validation layers, transaction rollback approach, when to offer dry-run
129
- - **File:** `docs/blog/014-dry-run.md`
130
- - **Status:** draft
131
-
132
- ### Part 15 — Publishing the gem & retrospective
133
- - **Tasks:** None (wrap-up)
134
- - **Hook:** From idea to rubygems.org
135
- - **Covers:** Gemspec final polish, CHANGELOG, versioning, publishing, series recap, what worked, what we'd do differently
136
- - **Key decisions:** Semantic versioning strategy, open source considerations
137
- - **File:** `docs/blog/015-publishing-retro.md`
138
- - **Status:** draft
139
-
140
- ### Part 16 — ERB View Templates: Composing Phlex Components
141
- - **Tasks:** None (view layer completion)
142
- - **Hook:** Phlex components are pure Ruby. ERB is what the browser sees. The glue is `.call`.
143
- - **Covers:** ERB templates composing Phlex, has_one_attached bugfix, ActiveStorage test setup, view testing in Rails 8, plain CSS stylesheet
144
- - **Key decisions:** raw component.call vs phlex-rails, plain CSS vs build step, view test infrastructure
145
- - **File:** `docs/blog/016-erb-view-templates.md`
146
- - **Status:** draft
147
-
148
- ### Part 17 — Showcase & Final Retrospective
149
- - **Tasks:** None (series finale)
150
- - **Hook:** 17 articles, 22 components, one complete gem. DataPorter in action.
151
- - **Covers:** Full workflow walkthrough with screenshots, architecture overview, series recap, lessons learned, what's next
152
- - **Key decisions:** N/A (retrospective)
153
- - **File:** `docs/blog/017-showcase-final-retro.md`
154
- - **Status:** draft