exwiw 0.2.5 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +16 -0
- data/README.md +38 -1
- data/lib/exwiw/schema_generator.rb +84 -45
- data/lib/exwiw/table_config.rb +8 -1
- data/lib/exwiw/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9ad123bcfe1cadde34a031d1db34fb4592892b88765d3b979c2fb827b53578f7
|
|
4
|
+
data.tar.gz: 59b2263b51e51f95e9f8d79cbc214dc0f385202e7ca120d108f3f64e18832a52
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f79b7338d57d93eebbfba42d81ff6a3f3c232dbb9163118e42d4f67b73844c50234910d3add9eafe72ff4f854d5497191ac7cb37a4ec1da1647dfb1987c118be
|
|
7
|
+
data.tar.gz: 22c97b02f4cb68badb0bc3e404996af4a48f46f0c32f0366cf0845a6e683b5d4a6a1af3ed499dae425d9adafe131f5c86eca933d5f51eac3d3d4458e508f36a8
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,22 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.2.7] - 2026-05-30
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- `schema:generate` no longer crashes on models with a composite primary key (`primary_key` returns an Array). Such tables are emitted with `skip: true`, tagged `type: "unsupported_composite_primary_key"`, and annotated with a `comment` listing the key columns. `columns` / `belongs_tos` are retained so the entry can be wired up once composite-key support lands; for now `skip: true` excludes it from extraction. `skip: true` tables no longer require `primary_key` at load time.
|
|
10
|
+
|
|
11
|
+
### Notes
|
|
12
|
+
|
|
13
|
+
- `schema:generate` multi-database support: each database keeps its own Rails migration history, so a per-database `schema_migrations` / `ar_internal_metadata` entry is emitted for every database that has one. Known limitation: the rails-managed table *name* is resolved from the global `ActiveRecord::Base.schema_migrations_table_name` / `internal_metadata_table_name` accessors, so a per-database override of those names is not detected and the table will be missing from that database's generated configs.
|
|
14
|
+
|
|
15
|
+
## [0.2.6] - 2026-05-29
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
|
|
19
|
+
- `schema:generate` now supports Rails multiple-database setups (`connects_to`). Models are bucketed by their database (`connection_db_config.name`, e.g. `primary` / `analytics`) and each database's config files are written into its own subdirectory under `OUTPUT_DIR_PATH` (`exwiw/primary/`, `exwiw/analytics/`, ...). Rails-managed tables (`schema_migrations` / `ar_internal_metadata`) are emitted under whichever database actually owns them. Single-database apps are unaffected and still write flat into the output directory. This replaces the previous behavior of raising `MultipleDatabasesNotSupportedError`.
|
|
20
|
+
|
|
5
21
|
## [0.2.5] - 2026-05-29
|
|
6
22
|
|
|
7
23
|
### Added
|
data/README.md
CHANGED
|
@@ -128,6 +128,26 @@ By default, the schema files will be saved in the `exwiw` directory. You can spe
|
|
|
128
128
|
OUTPUT_DIR_PATH=custom_directory bundle exec rake exwiw:schema:generate
|
|
129
129
|
```
|
|
130
130
|
|
|
131
|
+
#### Multiple databases
|
|
132
|
+
|
|
133
|
+
If the application uses Rails' multiple-database support (`connects_to`), `schema:generate` buckets models by the database they connect to and writes each database's config files into its own subdirectory of the output directory, named after the database config name (`primary`, `analytics`, ...):
|
|
134
|
+
|
|
135
|
+
```
|
|
136
|
+
exwiw/
|
|
137
|
+
primary/
|
|
138
|
+
shops.json
|
|
139
|
+
users.json
|
|
140
|
+
schema_migrations.json
|
|
141
|
+
analytics/
|
|
142
|
+
analytics_events.json
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Each database keeps its own Rails migration history, so a `schema_migrations` (and `ar_internal_metadata`) entry is emitted under every database that contains one — the example above shows `primary/schema_migrations.json` and would also produce `analytics/schema_migrations.json` when the analytics database has its own migration table. Single-database applications are unaffected and continue to write files flat into the output directory.
|
|
146
|
+
|
|
147
|
+
**Limitations**
|
|
148
|
+
|
|
149
|
+
- The rails-managed table *names* are resolved from the global `ActiveRecord::Base.schema_migrations_table_name` / `internal_metadata_table_name` accessors, which are shared across all connections. A per-database override of these names is not detected, so such a table will be missing from that database's generated configs.
|
|
150
|
+
|
|
131
151
|
### Configuration
|
|
132
152
|
|
|
133
153
|
This is an example of the one table schema:
|
|
@@ -250,7 +270,24 @@ Constraints:
|
|
|
250
270
|
|
|
251
271
|
- Defining `primary_key`, `columns`, or `belongs_tos` on a rails-managed entry is rejected with `ArgumentError` on load.
|
|
252
272
|
- A rails-managed table cannot be used as `--target-table`.
|
|
253
|
-
-
|
|
273
|
+
- In multi-database setups, the rails-managed entry is emitted under whichever database's connection actually contains the table (see [Multiple databases](#multiple-databases)). The table name itself is still derived from the global `ActiveRecord::Base.schema_migrations_table_name` / `internal_metadata_table_name` (prefix/suffix) accessors.
|
|
274
|
+
|
|
275
|
+
### Composite primary keys (unsupported)
|
|
276
|
+
|
|
277
|
+
exwiw does not yet support tables with a composite primary key. When `exwiw:schema:generate` encounters a model whose `primary_key` is an array, it still emits a config entry so the table is not silently dropped, but marks it `skip: true`, tags it `type: "unsupported_composite_primary_key"`, and records the key columns in a `comment`:
|
|
278
|
+
|
|
279
|
+
```json
|
|
280
|
+
{
|
|
281
|
+
"name": "composite_pk_records",
|
|
282
|
+
"type": "unsupported_composite_primary_key",
|
|
283
|
+
"skip": true,
|
|
284
|
+
"comment": "exwiw does not support composite primary keys (organization_id, location_id); data extraction is skipped.",
|
|
285
|
+
"belongs_tos": [],
|
|
286
|
+
"columns": [{ "name": "organization_id" }, { "name": "location_id" }, { "name": "name" }]
|
|
287
|
+
}
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
Unlike rails-managed entries, `columns` and `belongs_tos` are retained so the entry is ready to wire up once composite-key support lands. The `type` is purely a marker — `skip: true` is what actually excludes the table from extraction, so removing `skip` (and supplying a workable `primary_key`) lets you opt the table back in manually.
|
|
254
291
|
|
|
255
292
|
### Bulk insert chunk size
|
|
256
293
|
|
|
@@ -5,8 +5,6 @@ require "json"
|
|
|
5
5
|
|
|
6
6
|
module Exwiw
|
|
7
7
|
class SchemaGenerator
|
|
8
|
-
class MultipleDatabasesNotSupportedError < StandardError; end
|
|
9
|
-
|
|
10
8
|
def self.from_rails_application(output_dir:)
|
|
11
9
|
Rails.application.eager_load!
|
|
12
10
|
new(models: ActiveRecord::Base.descendants, output_dir: output_dir)
|
|
@@ -18,33 +16,53 @@ module Exwiw
|
|
|
18
16
|
end
|
|
19
17
|
|
|
20
18
|
def generate!
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
19
|
+
groups = build_table_groups
|
|
20
|
+
write_groups(groups)
|
|
21
|
+
groups
|
|
24
22
|
end
|
|
25
23
|
|
|
26
|
-
|
|
24
|
+
# Returns a Hash keyed by the database name.
|
|
25
|
+
#
|
|
26
|
+
# - Single-database setup: the only key is `nil`, signalling that the table
|
|
27
|
+
# configs should be written flat into `output_dir` (backwards compatible).
|
|
28
|
+
# - Multi-database setup (Rails `connects_to`): one key per database
|
|
29
|
+
# (`connection_db_config.name`, e.g. "primary" / "analytics"), each
|
|
30
|
+
# mapping to that database's table configs. They are written into
|
|
31
|
+
# `output_dir/<db_name>/`.
|
|
32
|
+
def build_table_groups
|
|
27
33
|
models = concrete_models
|
|
28
|
-
|
|
34
|
+
grouped = models.group_by { |model| database_name_for(model) }
|
|
29
35
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
name: table_name,
|
|
34
|
-
primary_key: representative.primary_key,
|
|
35
|
-
belongs_tos: aggregate_belongs_tos(model_group),
|
|
36
|
-
columns: representative.column_names.map { |name| { name: name } },
|
|
37
|
-
)
|
|
36
|
+
if grouped.size <= 1
|
|
37
|
+
conn = models.empty? ? ActiveRecord::Base.connection : models.first.connection
|
|
38
|
+
return { nil => build_tables_for(models, conn) }
|
|
38
39
|
end
|
|
39
40
|
|
|
40
|
-
|
|
41
|
+
grouped.each_with_object({}) do |(db_name, group_models), result|
|
|
42
|
+
conn = group_models.first.connection
|
|
43
|
+
result[db_name] = build_tables_for(group_models, conn)
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# Backwards-compatible flat list of all table configs. Only meaningful for
|
|
48
|
+
# a single-database setup; for multi-database setups prefer
|
|
49
|
+
# `#build_table_groups` so the database association is preserved.
|
|
50
|
+
def build_tables
|
|
51
|
+
build_table_groups.values.flatten
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def write_groups(groups)
|
|
55
|
+
groups.each do |db_name, tables|
|
|
56
|
+
dir = db_name.nil? ? @output_dir : File.join(@output_dir, db_name)
|
|
57
|
+
write_files(dir, tables)
|
|
58
|
+
end
|
|
41
59
|
end
|
|
42
60
|
|
|
43
|
-
def write_files(tables)
|
|
44
|
-
FileUtils.mkdir_p(
|
|
61
|
+
def write_files(dir, tables)
|
|
62
|
+
FileUtils.mkdir_p(dir)
|
|
45
63
|
|
|
46
64
|
tables.each do |table|
|
|
47
|
-
path = File.join(
|
|
65
|
+
path = File.join(dir, "#{table.name}.json")
|
|
48
66
|
config_to_write =
|
|
49
67
|
if File.exist?(path)
|
|
50
68
|
TableConfig.from(JSON.parse(File.read(path))).merge(table)
|
|
@@ -55,20 +73,50 @@ module Exwiw
|
|
|
55
73
|
end
|
|
56
74
|
end
|
|
57
75
|
|
|
76
|
+
private def build_tables_for(models, conn)
|
|
77
|
+
tables_from_models = models.group_by(&:table_name).map do |table_name, model_group|
|
|
78
|
+
representative = model_group.first
|
|
79
|
+
primary_key = representative.primary_key
|
|
80
|
+
|
|
81
|
+
# 複合主キー (`representative.primary_key` が Array) のテーブルは現状未対応。
|
|
82
|
+
# primary_key を省略し、type で非対応である旨を明示したうえで skip:true を
|
|
83
|
+
# 付与して出力する。type を付けておくことで将来対応する際の目印になる。
|
|
84
|
+
# 利用者が必要に応じて手動で skip を外して設定し直せるよう、設定ファイル
|
|
85
|
+
# 自体は生成しておく。
|
|
86
|
+
if primary_key.is_a?(Array)
|
|
87
|
+
TableConfig.from_symbol_keys(
|
|
88
|
+
name: table_name,
|
|
89
|
+
type: TableConfig::UNSUPPORTED_COMPOSITE_PRIMARY_KEY,
|
|
90
|
+
skip: true,
|
|
91
|
+
comment: "exwiw does not support composite primary keys " \
|
|
92
|
+
"(#{primary_key.join(', ')}); data extraction is skipped.",
|
|
93
|
+
belongs_tos: aggregate_belongs_tos(model_group),
|
|
94
|
+
columns: representative.column_names.map { |name| { name: name } },
|
|
95
|
+
)
|
|
96
|
+
else
|
|
97
|
+
TableConfig.from_symbol_keys(
|
|
98
|
+
name: table_name,
|
|
99
|
+
primary_key: primary_key,
|
|
100
|
+
belongs_tos: aggregate_belongs_tos(model_group),
|
|
101
|
+
columns: representative.column_names.map { |name| { name: name } },
|
|
102
|
+
)
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
tables_from_models + build_rails_managed_tables(conn)
|
|
107
|
+
end
|
|
108
|
+
|
|
58
109
|
private def concrete_models
|
|
59
110
|
@models.reject(&:abstract_class?).select(&:table_exists?)
|
|
60
111
|
end
|
|
61
112
|
|
|
62
|
-
#
|
|
63
|
-
#
|
|
64
|
-
#
|
|
65
|
-
#
|
|
66
|
-
#
|
|
67
|
-
#
|
|
68
|
-
|
|
69
|
-
# テーブル名を取り、connection 毎にエントリを生成する必要がある。
|
|
70
|
-
private def build_rails_managed_tables
|
|
71
|
-
conn = ActiveRecord::Base.connection
|
|
113
|
+
# rails-managed テーブル (`schema_migrations` / `ar_internal_metadata`) は
|
|
114
|
+
# モデルクラスを持たないため `ActiveRecord::Base.descendants` からは拾えない。
|
|
115
|
+
# multi-DB 構成では各 connection が独立した migration 履歴テーブルを持つので、
|
|
116
|
+
# 対象 connection を受け取り、その connection 上に該当テーブルが存在する場合のみ
|
|
117
|
+
# エントリを生成する。テーブル名そのものは prefix/suffix を含むグローバル設定
|
|
118
|
+
# (`ActiveRecord::Base.schema_migrations_table_name` 等) から得る。
|
|
119
|
+
private def build_rails_managed_tables(conn)
|
|
72
120
|
result = []
|
|
73
121
|
|
|
74
122
|
schema_migrations_name = ActiveRecord::Base.schema_migrations_table_name
|
|
@@ -108,22 +156,13 @@ module Exwiw
|
|
|
108
156
|
end
|
|
109
157
|
end
|
|
110
158
|
|
|
111
|
-
#
|
|
112
|
-
#
|
|
113
|
-
#
|
|
114
|
-
#
|
|
115
|
-
#
|
|
116
|
-
private def
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
specs = models.map(&:connection_specification_name).uniq
|
|
120
|
-
return if specs.size <= 1
|
|
121
|
-
|
|
122
|
-
raise MultipleDatabasesNotSupportedError, <<~MSG
|
|
123
|
-
exwiw does not yet support Rails multiple-database setup.
|
|
124
|
-
Detected connection specifications: #{specs.inspect}
|
|
125
|
-
Track progress at https://github.com/riseshia/exwiw/issues
|
|
126
|
-
MSG
|
|
159
|
+
# Identifies which database a model belongs to. With Rails multi-DB
|
|
160
|
+
# (`connects_to` backed by `database.yml`), `connection_db_config.name`
|
|
161
|
+
# returns the configuration name ("primary", "analytics", ...) which is
|
|
162
|
+
# stable across roles/shards and makes a natural per-database directory
|
|
163
|
+
# name. Single-database apps all share one name, collapsing into one group.
|
|
164
|
+
private def database_name_for(model)
|
|
165
|
+
model.connection_db_config.name
|
|
127
166
|
end
|
|
128
167
|
end
|
|
129
168
|
end
|
data/lib/exwiw/table_config.rb
CHANGED
|
@@ -11,6 +11,11 @@ module Exwiw
|
|
|
11
11
|
RAILS_MANAGED_INTERNAL_METADATA,
|
|
12
12
|
].freeze
|
|
13
13
|
|
|
14
|
+
# exwiw が現状サポートしていない複合主キーのテーブルを表す type。
|
|
15
|
+
# schema:generate が skip:true と併せて付与する。将来サポートする際の
|
|
16
|
+
# 目印になるよう、rails-managed とは異なり columns/belongs_tos は保持する。
|
|
17
|
+
UNSUPPORTED_COMPOSITE_PRIMARY_KEY = "unsupported_composite_primary_key"
|
|
18
|
+
|
|
14
19
|
attribute :name, String
|
|
15
20
|
attribute :primary_key, optional(String), skip_serializing_if_nil: true
|
|
16
21
|
attribute :type, optional(String), skip_serializing_if_nil: true
|
|
@@ -138,7 +143,9 @@ module Exwiw
|
|
|
138
143
|
"Table '#{name}' has type=#{type}; columns must not be defined."
|
|
139
144
|
end
|
|
140
145
|
else
|
|
141
|
-
|
|
146
|
+
# skip:true のテーブルはデータ抽出を行わないため primary_key を要求しない。
|
|
147
|
+
# (例: exwiw 非対応の複合主キーテーブル)
|
|
148
|
+
if primary_key.nil? && !skip
|
|
142
149
|
raise ArgumentError, "Table '#{name}' requires primary_key."
|
|
143
150
|
end
|
|
144
151
|
end
|
data/lib/exwiw/version.rb
CHANGED