better_seeder 0.2.2 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +53 -72
- data/lib/better_seeder/configuration.rb +3 -1
- data/lib/better_seeder/farms/farmer.rb +108 -91
- data/lib/better_seeder/utils.rb +28 -13
- data/lib/better_seeder/version.rb +1 -1
- data/lib/better_seeder.rb +94 -61
- data/lib/generators/better_seeder/structure_generator.rb +15 -0
- metadata +3 -3
- data/lib/generators/better_seeder/structure/structure_generator.rb +0 -17
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 20203d64e75c796bd405cb5caaabeb697ee4cd64b2a21770cc2703efaaedf815
|
4
|
+
data.tar.gz: 8b218cd82cc8950284fa4fe0468443481760e092161c9a96f9ea2b0ae9974c28
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f193936be5c4bb4416bc27b2cdbb869c10c8dbe4a8d3046d63cc0ffd1af4e31edd7e0e613ab1828913df5a64fd225523211a158662b783491022a2109233ccb4
|
7
|
+
data.tar.gz: 806324afb167aa6376f8c5dbf5fbe13083f4bcc9ec1e89c8bcbb46b021f5c29b27318580b63e2dacc42d4bfb3bf54d22afde05142acecd6a49aee82a23c561ae
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# BetterSeeder
|
2
2
|
|
3
|
-
|
3
|
+
BetterSeeder is a Rails gem designed to simplify and centralize your application's seeding process. It offers a flexible system to generate dynamic data, validate it using Dry-schema, enforce uniqueness constraints, load data into the database, and export it in various formats (SQL, CSV, JSON). The configuration is managed through a Rails initializer, while model-specific logic is defined in dedicated structure files.
|
4
4
|
|
5
5
|
---
|
6
6
|
|
@@ -10,17 +10,19 @@
|
|
10
10
|
Define custom data generators for each model in dedicated structure files.
|
11
11
|
|
12
12
|
- **Validation and Uniqueness**
|
13
|
-
Validate generated records using Dry-schema and enforce uniqueness constraints
|
13
|
+
Validate generated records using Dry-schema and enforce uniqueness constraints across one or multiple columns.
|
14
14
|
|
15
15
|
- **Loading & Exporting**
|
16
|
-
|
17
|
-
- Export data as a single SQL INSERT statement, or in CSV or JSON formats.
|
16
|
+
Load generated data directly into your database—supporting parent/child relationships—and export the data as a single SQL INSERT statement, CSV, or JSON file.
|
18
17
|
|
19
18
|
- **Centralized Configuration**
|
20
|
-
|
19
|
+
Customize settings such as `log_language`, `structure_path`, and `preload_path` via a Rails initializer. If no initializer is provided, default settings apply.
|
21
20
|
|
22
|
-
- **
|
23
|
-
|
21
|
+
- **Initializer Installation**
|
22
|
+
An install method creates the necessary initializer file in your Rails application to streamline setup.
|
23
|
+
|
24
|
+
- **Rails Generator**
|
25
|
+
A custom Rails generator scaffolds a structure file template for your models, ensuring a consistent configuration format.
|
24
26
|
|
25
27
|
---
|
26
28
|
|
@@ -42,7 +44,7 @@ bundle install
|
|
42
44
|
|
43
45
|
## Configuration
|
44
46
|
|
45
|
-
BetterSeeder uses a centralized configuration defined in `BetterSeeder.configuration`. You can override the default settings
|
47
|
+
BetterSeeder uses a centralized configuration defined in `BetterSeeder.configuration`. You can override the default settings by creating an initializer. For example, create a file:
|
46
48
|
|
47
49
|
```ruby
|
48
50
|
# config/initializers/better_seeder.rb
|
@@ -55,53 +57,43 @@ BetterSeeder.configure do |config|
|
|
55
57
|
end
|
56
58
|
```
|
57
59
|
|
58
|
-
If these values are set in the initializer, they will be used; otherwise, the gem will
|
60
|
+
If these values are set in the initializer, they will be used; otherwise, the gem will use its default values.
|
59
61
|
|
60
62
|
---
|
61
63
|
|
62
|
-
##
|
64
|
+
## Initializer Installation
|
63
65
|
|
64
|
-
|
66
|
+
To streamline the setup, execute the following command in your Rails console:
|
65
67
|
|
66
68
|
```ruby
|
67
69
|
BetterSeeder.install
|
68
70
|
```
|
69
71
|
|
70
|
-
This command creates (if not already present) the file `config/initializers/better_seeder.rb` with
|
71
|
-
|
72
|
-
```ruby
|
73
|
-
# BetterSeeder initializer
|
74
|
-
BetterSeeder.configure do |config|
|
75
|
-
config.log_language = :en
|
76
|
-
config.structure_path = Rails.root.join('db', 'seed', 'structure')
|
77
|
-
config.preload_path = Rails.root.join('db', 'seed', 'preload')
|
78
|
-
end
|
79
|
-
```
|
72
|
+
This command creates (if not already present) the file `config/initializers/better_seeder.rb` with the necessary configuration.
|
80
73
|
|
81
74
|
---
|
82
75
|
|
83
76
|
## Structure Files
|
84
77
|
|
85
|
-
For each model, create a structure file that centralizes the logic for generating, validating, and configuring seed data. Each structure file should
|
78
|
+
For each model, create a structure file that centralizes the logic for generating, validating, and configuring seed data. Each structure file should include:
|
86
79
|
|
87
80
|
- **`structure`**
|
88
|
-
Returns a hash where each key
|
81
|
+
Returns a hash where each key is an attribute and its value is an array in the format `[type, lambda_generator]`.
|
89
82
|
|
90
83
|
- **`seed_schema` (Optional)**
|
91
|
-
Defines a Dry-schema for validating
|
84
|
+
Defines a Dry-schema for validating generated records.
|
92
85
|
|
93
86
|
- **`seed_config`**
|
94
87
|
Returns a hash with model-specific seeding settings:
|
95
|
-
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
|
88
|
+
- `file_name`: The output file name (without extension)
|
89
|
+
- `columns: { excluded: [...] }`: Columns to exclude from the generated data
|
90
|
+
- `generate_data`: Boolean flag indicating whether to generate data dynamically (if false, existing records are used)
|
91
|
+
- `count`: The number of records to generate (default: 10)
|
92
|
+
- `load_data`: Boolean flag indicating whether the generated records should be inserted into the database
|
93
|
+
- `parent`: For child models, specifies the parent model(s) used for injecting foreign keys
|
101
94
|
|
102
95
|
- **`unique_keys` (Optional)**
|
103
|
-
Returns an array of column groups (each group is an array of symbols) that must be unique.
|
104
|
-
For example:
|
96
|
+
Returns an array of column groups (each group is an array of symbols) that must be unique. For example:
|
105
97
|
|
106
98
|
```ruby
|
107
99
|
def self.unique_keys
|
@@ -111,13 +103,10 @@ For each model, create a structure file that centralizes the logic for generatin
|
|
111
103
|
|
112
104
|
### Example Structure File
|
113
105
|
|
114
|
-
For a generic model `MyModel` in the namespace `MyNamespace`, create a file at `db/seed/structure/my_namespace/my_model_structure.rb`:
|
115
|
-
|
116
106
|
```ruby
|
117
107
|
# db/seed/structure/my_namespace/my_model_structure.rb
|
118
108
|
module MyNamespace
|
119
109
|
class MyModelStructure < BetterSeeder::StructureBase
|
120
|
-
# Defines generators for each attribute.
|
121
110
|
def self.structure
|
122
111
|
{
|
123
112
|
name: [:string, -> { FFaker::Name.name }],
|
@@ -126,7 +115,6 @@ module MyNamespace
|
|
126
115
|
}
|
127
116
|
end
|
128
117
|
|
129
|
-
# Optional: Validate generated records using Dry-schema.
|
130
118
|
def self.seed_schema
|
131
119
|
Dry::Schema.Params do
|
132
120
|
required(:name).filled(:string)
|
@@ -135,7 +123,6 @@ module MyNamespace
|
|
135
123
|
end
|
136
124
|
end
|
137
125
|
|
138
|
-
# Specific seeding configuration for MyModel.
|
139
126
|
def self.seed_config
|
140
127
|
{
|
141
128
|
file_name: 'my_model_seed',
|
@@ -147,7 +134,6 @@ module MyNamespace
|
|
147
134
|
}
|
148
135
|
end
|
149
136
|
|
150
|
-
# Optional: Uniqueness constraints; for example, email must be unique.
|
151
137
|
def self.unique_keys
|
152
138
|
[[:email]]
|
153
139
|
end
|
@@ -162,19 +148,19 @@ end
|
|
162
148
|
When you call `BetterSeeder.magic` with a configuration that contains an array of model names (as strings), the gem will:
|
163
149
|
|
164
150
|
1. **Load Structure Files**
|
165
|
-
|
151
|
+
Retrieve the corresponding structure file from the directory defined by `BetterSeeder.configuration.structure_path` for each model.
|
166
152
|
|
167
153
|
2. **Retrieve Seeding Configurations**
|
168
|
-
|
154
|
+
Invoke the model's `seed_config` method to obtain its specific settings.
|
169
155
|
|
170
156
|
3. **Generate or Retrieve Records**
|
171
|
-
|
157
|
+
Use the `structure` method to generate data dynamically (or fetch existing records) and validate them using `seed_schema` if defined. Uniqueness is enforced via `unique_keys`.
|
172
158
|
|
173
159
|
4. **Handle Parent/Child Relationships**
|
174
|
-
|
160
|
+
Automatically inject foreign keys into child models using records from parent models.
|
175
161
|
|
176
|
-
5. **Load and Export**
|
177
|
-
If enabled (`load_data: true`), the generated records are inserted into the database and
|
162
|
+
5. **Load and Export Data**
|
163
|
+
If enabled (`load_data: true`), the generated records are inserted into the database and exported in the specified format (SQL, CSV, or JSON). Export files are saved in the directory specified by `BetterSeeder.configuration.preload_path`.
|
178
164
|
|
179
165
|
### Example Usage
|
180
166
|
|
@@ -190,24 +176,17 @@ BetterSeeder.magic(
|
|
190
176
|
)
|
191
177
|
```
|
192
178
|
|
193
|
-
This command processes each model by:
|
194
|
-
|
195
|
-
- Reading its structure file and retrieving its configuration via `seed_config`.
|
196
|
-
- Generating or fetching data according to the specified rules.
|
197
|
-
- Inserting the data into the database (if `load_data` is enabled).
|
198
|
-
- Exporting the data as an SQL file (or CSV/JSON, depending on `export_type`).
|
199
|
-
|
200
179
|
---
|
201
180
|
|
202
|
-
##
|
181
|
+
## Rails Generator
|
203
182
|
|
204
|
-
|
183
|
+
A custom Rails generator is available to scaffold a structure file template for your models. Execute the following command:
|
205
184
|
|
206
185
|
```bash
|
207
186
|
rails generate better_seeder:structure MyNamespace::MyModel
|
208
187
|
```
|
209
188
|
|
210
|
-
This command creates a structure file template in the appropriate subdirectory under `db/seed/structure`. The generated
|
189
|
+
This command creates a structure file template in the appropriate subdirectory under `db/seed/structure`. The generated template includes placeholders for:
|
211
190
|
|
212
191
|
- Attribute generators (via the `structure` method)
|
213
192
|
- Validation schema (via the `seed_schema` method)
|
@@ -219,21 +198,18 @@ This command creates a structure file template in the appropriate subdirectory u
|
|
219
198
|
```ruby
|
220
199
|
module MyNamespace
|
221
200
|
class MyModelStructure < BetterSeeder::StructureBase
|
222
|
-
# Defines generators for each attribute.
|
223
201
|
def self.structure
|
224
202
|
{
|
225
203
|
attribute_name: [:string, -> { "your value" }]
|
226
204
|
}
|
227
205
|
end
|
228
206
|
|
229
|
-
# Optional: Validate generated records using Dry-schema.
|
230
207
|
def self.seed_schema
|
231
208
|
Dry::Schema.Params do
|
232
209
|
required(:attribute_name).filled(:string)
|
233
210
|
end
|
234
211
|
end
|
235
212
|
|
236
|
-
# Specific seeding configuration for MyModel.
|
237
213
|
def self.seed_config
|
238
214
|
{
|
239
215
|
file_name: 'my_model_seed',
|
@@ -245,7 +221,6 @@ module MyNamespace
|
|
245
221
|
}
|
246
222
|
end
|
247
223
|
|
248
|
-
# Optional: Uniqueness constraints.
|
249
224
|
def self.unique_keys
|
250
225
|
[]
|
251
226
|
end
|
@@ -253,27 +228,33 @@ module MyNamespace
|
|
253
228
|
end
|
254
229
|
```
|
255
230
|
|
256
|
-
This generator
|
231
|
+
This generator ensures a consistent structure for your seeding configuration across models.
|
257
232
|
|
258
|
-
|
233
|
+
---
|
259
234
|
|
260
|
-
|
261
|
-
|
262
|
-
|
235
|
+
## Contact & Feature Requests
|
236
|
+
|
237
|
+
For suggestions, bug reports, or to report new feature requests, please reach out via email at: **alessio.bussolari@pandev.it**.
|
263
238
|
|
264
239
|
---
|
265
240
|
|
266
|
-
##
|
241
|
+
## Upcoming Features
|
267
242
|
|
268
|
-
|
243
|
+
The development team is continuously enhancing BetterSeeder. Current efforts include:
|
269
244
|
|
270
|
-
- **
|
271
|
-
|
245
|
+
- **Additional Export Formats:** Expanding beyond SQL, CSV, and JSON to offer more options.
|
246
|
+
- **Enhanced Rails Associations Integration:** Improving support for complex parent/child relationships.
|
247
|
+
- **Expanded Configuration Options:** Providing more granular control over the seeding process.
|
248
|
+
- **Performance Optimizations:** Refining data generation algorithms for faster processing.
|
249
|
+
|
250
|
+
---
|
251
|
+
|
252
|
+
## Conclusion
|
272
253
|
|
273
|
-
|
274
|
-
Define generation, validation, and configuration logic for each model in dedicated structure files.
|
254
|
+
BetterSeeder provides a modular, configurable, and extensible system for seeding your Rails application's data. Its key benefits include:
|
275
255
|
|
276
|
-
- **
|
277
|
-
|
256
|
+
- **Centralized Configuration:** Manage settings through a simple Rails initializer.
|
257
|
+
- **Modular Structure Files:** Define data generation, validation, and configuration logic on a per-model basis.
|
258
|
+
- **Seamless Data Handling:** Efficiently generate, validate, load, and export seed data while supporting complex relationships.
|
278
259
|
|
279
|
-
For further details or
|
260
|
+
For further details or to contribute, please refer to the official repository or documentation.
|
@@ -2,15 +2,17 @@
|
|
2
2
|
|
3
3
|
module BetterSeeder
|
4
4
|
class Configuration
|
5
|
-
attr_accessor :log_language, :structure_path, :preload_path
|
5
|
+
attr_accessor :log_language, :log_level, :structure_path, :preload_path
|
6
6
|
|
7
7
|
def initialize
|
8
8
|
if defined?(Rails) && Rails.respond_to?(:root)
|
9
9
|
@log_language = :en
|
10
|
+
@log_level = :info
|
10
11
|
@structure_path = Rails.root.join('db', 'seed', 'structure')
|
11
12
|
@preload_path = Rails.root.join('db', 'seed', 'preload')
|
12
13
|
else
|
13
14
|
@log_language = :en
|
15
|
+
@log_level = :info
|
14
16
|
@structure_path = File.join(Dir.pwd, 'db', 'seed', 'structure')
|
15
17
|
@preload_path = File.join(Dir.pwd, 'db', 'seed', 'preload')
|
16
18
|
end
|
@@ -1,116 +1,133 @@
|
|
1
1
|
module BetterSeeder
|
2
2
|
module Farms
|
3
3
|
class Farmer
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
4
|
+
|
5
|
+
class << self
|
6
|
+
def generate(options = {})
|
7
|
+
model_name = options[:model] or raise ArgumentError, "Missing :model option"
|
8
|
+
count = options[:count] || 10
|
9
|
+
|
10
|
+
# Costruisce il percorso del file di structure.
|
11
|
+
structure_file = File.expand_path(
|
12
|
+
File.join(BetterSeeder.configuration.structure_path, "#{model_name.underscore}_structure.rb"),
|
13
|
+
Dir.pwd
|
14
|
+
)
|
15
|
+
raise "Structure file not found: #{structure_file}" unless File.exist?(structure_file)
|
16
|
+
|
17
|
+
# Carica il file di structure.
|
18
|
+
load structure_file
|
19
|
+
|
20
|
+
# Costruisce il nome della classe di structure: es. "Media::Participant" => "Media::ParticipantStructure"
|
21
|
+
structure_class_name = "#{model_name}Structure"
|
22
|
+
begin
|
23
|
+
structure_class = Object.const_get(structure_class_name)
|
24
|
+
rescue error
|
25
|
+
message = "Structure class not found: #{structure_class_name}"
|
26
|
+
BetterSeeder::Utils.logger(message: message)
|
27
|
+
raise error
|
28
|
+
end
|
29
|
+
|
30
|
+
generated_records = []
|
31
|
+
|
32
|
+
while generated_records.size < count
|
33
|
+
new_record = nil
|
34
|
+
loop do
|
35
|
+
new_record = build_record(model_name, structure_class)
|
36
|
+
new_record = inject_parent_keys(model_name, new_record, structure_class)
|
37
|
+
break if validate_record(new_record, structure_class) &&
|
38
|
+
!record_exists?(model_name, new_record, structure_class, generated_records)
|
39
|
+
end
|
40
|
+
generated_records.push(new_record)
|
41
|
+
end
|
42
|
+
|
43
|
+
generated_records
|
43
44
|
end
|
44
45
|
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
#
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
46
|
+
private
|
47
|
+
|
48
|
+
def record_exists?(model_name, record, structure_class, generated_records)
|
49
|
+
# Se non è definito il metodo unique_keys, non eseguiamo il controllo
|
50
|
+
return false unless structure_class.respond_to?(:unique_keys)
|
51
|
+
unique_key_sets = structure_class.unique_keys
|
52
|
+
return false if unique_key_sets.empty?
|
53
|
+
|
54
|
+
# Determina il modello associato: si assume che il nome del modello sia
|
55
|
+
# dato dalla rimozione della stringa "Structure" dal nome della classe di structure.
|
56
|
+
model_class_name = structure_class.to_s.sub(/Structure$/, '')
|
57
|
+
model_class = Object.const_get(model_class_name)
|
58
|
+
|
59
|
+
# Per ogni set di chiavi uniche, costruiamo le condizioni della query
|
60
|
+
unique_key_sets.each do |key_set|
|
61
|
+
conditions = {}
|
62
|
+
key_set.each do |col|
|
63
|
+
conditions[col] = record[col]
|
64
|
+
end
|
65
|
+
# Se esiste un record nel database che soddisfa le condizioni, restituisce true.
|
66
|
+
return true if generated_records.find do |record|
|
67
|
+
conditions.all? { |key, value| record[key] == value }
|
68
|
+
end.present?
|
69
|
+
return true if model_class.where(conditions).exists?
|
64
70
|
end
|
65
|
-
|
71
|
+
|
72
|
+
false
|
66
73
|
end
|
67
74
|
|
68
|
-
|
69
|
-
|
70
|
-
|
75
|
+
def build_record(model_name, structure_class)
|
76
|
+
generation_rules = structure_class.structure
|
77
|
+
raise "Structure must be a Hash" unless generation_rules.is_a?(Hash)
|
71
78
|
|
72
|
-
# Continua a generare record finché non si raggiunge il numero richiesto.
|
73
|
-
while generated_records.size < count
|
74
|
-
attempts += 1
|
75
79
|
record = {}
|
76
80
|
generation_rules.each do |attribute, rule|
|
77
|
-
# Ogni rule è un array
|
81
|
+
# Ogni rule è un array nel formato [tipo, generatore]
|
78
82
|
generator = rule[1]
|
79
83
|
value = generator.respond_to?(:call) ? generator.call : generator
|
80
84
|
record[attribute] = value
|
81
85
|
end
|
82
86
|
|
83
|
-
|
84
|
-
|
85
|
-
result = schema.call(record)
|
86
|
-
unless result.success?
|
87
|
-
message = "[ERROR] Record validation failed for #{model_name}: #{result.errors.to_h}"
|
88
|
-
BetterSeeder::Utils.logger(message: message)
|
89
|
-
progressbar.increment
|
90
|
-
next # Rigenera il record se la validazione fallisce.
|
91
|
-
end
|
92
|
-
end
|
87
|
+
record
|
88
|
+
end
|
93
89
|
|
94
|
-
|
95
|
-
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
90
|
+
def inject_parent_keys(model_name, record, structure_class)
|
91
|
+
config = structure_class.respond_to?(:seed_config) ? structure_class.seed_config : {}
|
92
|
+
parents_spec = config[:parents]
|
93
|
+
return record unless parents_spec && !parents_spec.empty?
|
94
|
+
|
95
|
+
parents_spec.each do |parent_config|
|
96
|
+
parent_model = parent_config[:model]
|
97
|
+
column = parent_config[:column]
|
100
98
|
|
101
|
-
|
102
|
-
|
103
|
-
|
104
|
-
|
99
|
+
# Tenta di ottenere un record del parent dal pool BetterSeeder.generated_records se disponibile.
|
100
|
+
# Usiamo il nome del modello come chiave nel pool.
|
101
|
+
pool_key = parent_model.to_s
|
102
|
+
parent_record = if defined?(BetterSeeder.generated_records) &&
|
103
|
+
BetterSeeder.generated_records[pool_key] &&
|
104
|
+
!BetterSeeder.generated_records[pool_key].empty?
|
105
|
+
BetterSeeder.generated_records[pool_key].sample
|
106
|
+
else
|
107
|
+
BetterSeeder.generated_records[pool_key] = parent_model.all
|
108
|
+
BetterSeeder.generated_records[pool_key].sample
|
109
|
+
end
|
110
|
+
|
111
|
+
raise "Parent record not found for #{parent_model}" unless parent_record
|
112
|
+
|
113
|
+
# Inietta nel record la chiave esterna indicata nella configurazione.
|
114
|
+
# binding.pry if model_name == "Media::Participant"
|
115
|
+
record[column] = parent_record[:id]
|
105
116
|
end
|
106
117
|
|
107
|
-
|
108
|
-
|
118
|
+
record
|
119
|
+
end
|
120
|
+
|
121
|
+
def validate_record(record, structure_class)
|
122
|
+
return true unless structure_class.respond_to?(:seed_schema_validation)
|
123
|
+
|
124
|
+
schema = structure_class.seed_schema_validation
|
125
|
+
result = schema.call(record)
|
126
|
+
return true if result.success?
|
127
|
+
|
128
|
+
raise result.errors.to_h
|
109
129
|
end
|
110
130
|
|
111
|
-
message = "[INFO] Generated #{generated_records.size} unique records for #{model_name} after #{attempts} attempts."
|
112
|
-
BetterSeeder::Utils.logger(message: message)
|
113
|
-
generated_records
|
114
131
|
end
|
115
132
|
end
|
116
133
|
end
|
data/lib/better_seeder/utils.rb
CHANGED
@@ -2,21 +2,36 @@
|
|
2
2
|
|
3
3
|
module BetterSeeder
|
4
4
|
module Utils
|
5
|
-
# Trasforma un nome di classe in snake_case.
|
6
|
-
# Esempio: "Campaigns::Campaign" => "campaigns_campaign"
|
7
|
-
def self.transform_class_name(class_name)
|
8
|
-
elements = class_name.split("::").map(&:underscore)
|
9
|
-
# Aggiunge "_structure.rb" all'ultimo elemento
|
10
|
-
elements[-1] = "#{elements[-1]}_structure.rb"
|
11
|
-
elements.join("/")
|
12
|
-
end
|
13
5
|
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
6
|
+
class << self
|
7
|
+
# Trasforma un nome di classe in snake_case.
|
8
|
+
# Esempio: "Campaigns::Campaign" => "campaigns_campaign"
|
9
|
+
def transform_class_name(class_name)
|
10
|
+
elements = class_name.split("::").map(&:underscore)
|
11
|
+
# Aggiunge "_structure.rb" all'ultimo elemento
|
12
|
+
elements[-1] = "#{elements[-1]}_structure.rb"
|
13
|
+
elements.join("/")
|
14
|
+
end
|
15
|
+
|
16
|
+
def logger(message: nil)
|
17
|
+
if defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
|
18
|
+
Rails.logger.info message
|
19
|
+
else
|
20
|
+
puts message
|
21
|
+
end
|
19
22
|
end
|
23
|
+
|
24
|
+
def log_level_setup
|
25
|
+
level = case BetterSeeder.configuration.log_level
|
26
|
+
when :debug then Logger::DEBUG
|
27
|
+
when :info then Logger::INFO
|
28
|
+
when :error then Logger::ERROR
|
29
|
+
else Logger::DEBUG
|
30
|
+
end
|
31
|
+
|
32
|
+
ActiveRecord::Base.logger.level = level
|
33
|
+
end
|
34
|
+
|
20
35
|
end
|
21
36
|
end
|
22
37
|
end
|
data/lib/better_seeder.rb
CHANGED
@@ -14,17 +14,31 @@ module BetterSeeder
|
|
14
14
|
|
15
15
|
def initialize
|
16
16
|
if defined?(Rails) && Rails.respond_to?(:root)
|
17
|
-
@log_language
|
17
|
+
@log_language = :en
|
18
|
+
@log_level = :info
|
18
19
|
@structure_path = Rails.root.join('db', 'seed', 'structure')
|
19
|
-
@preload_path
|
20
|
+
@preload_path = Rails.root.join('db', 'seed', 'preload')
|
20
21
|
else
|
21
|
-
@log_language
|
22
|
+
@log_language = :en
|
23
|
+
@log_level = :info
|
22
24
|
@structure_path = File.join(Dir.pwd, 'db', 'seed', 'structure')
|
23
|
-
@preload_path
|
25
|
+
@preload_path = File.join(Dir.pwd, 'db', 'seed', 'preload')
|
24
26
|
end
|
25
27
|
end
|
26
28
|
end
|
27
29
|
|
30
|
+
# Definisce un hash che fungerà da pool per memorizzare i record generati per ciascun modello.
|
31
|
+
@generated_records = {}
|
32
|
+
|
33
|
+
def self.generated_records
|
34
|
+
@generated_records
|
35
|
+
end
|
36
|
+
|
37
|
+
def self.store_generated_record(model_name, record)
|
38
|
+
@generated_records[model_name.to_s] ||= []
|
39
|
+
@generated_records[model_name.to_s] << record
|
40
|
+
end
|
41
|
+
|
28
42
|
# Restituisce l'istanza globale di configurazione. Se non esiste, viene creata con i valori di default.
|
29
43
|
def self.configuration
|
30
44
|
@configuration ||= Configuration.new
|
@@ -47,6 +61,7 @@ module BetterSeeder
|
|
47
61
|
# Metodo install che crea l'initializer di BetterSeeder in config/initializers
|
48
62
|
# con le seguenti impostazioni:
|
49
63
|
# - log_language: lingua da usare per i log (es. :en, :it)
|
64
|
+
# - log_level: livello di logging nell'output dei dati del seeder (:debug, :info, :error)
|
50
65
|
# - structure_path: percorso dove sono memorizzati i file di structure (default: Rails.root/db/seed/structure)
|
51
66
|
# - preload_path: percorso dove verranno salvati i file esportati (default: Rails.root/db/seed/preload)
|
52
67
|
def self.install
|
@@ -60,6 +75,7 @@ module BetterSeeder
|
|
60
75
|
# This file was generated by BetterSeeder.install
|
61
76
|
BetterSeeder.configure do |config|
|
62
77
|
config.log_language = :en
|
78
|
+
config.log_level = :error
|
63
79
|
config.structure_path = Rails.root.join('db', 'seed', 'structure')
|
64
80
|
config.preload_path = Rails.root.join('db', 'seed', 'preload')
|
65
81
|
end
|
@@ -94,9 +110,15 @@ module BetterSeeder
|
|
94
110
|
# - Se abilitato, i record vengono caricati nel database e successivamente esportati nel formato richiesto
|
95
111
|
# - Vengono raccolte statistiche e loggato il tempo totale di esecuzione
|
96
112
|
def self.magic(config)
|
113
|
+
previous_log_level = ActiveRecord::Base.logger.level
|
114
|
+
BetterSeeder::Utils.log_level_setup
|
115
|
+
|
116
|
+
message = "[LOGGER] previous log level: #{previous_log_level}, actual log level: #{ActiveRecord::Base.logger.level}"
|
117
|
+
BetterSeeder::Utils.logger(message: message)
|
118
|
+
|
97
119
|
start_time = Time.now
|
98
|
-
stats = {}
|
99
|
-
parent_loaded_records = {}
|
120
|
+
stats = {} # Statistiche: modello => numero di record caricati
|
121
|
+
parent_loaded_records = {} # Per memorizzare i record creati per i modelli parent
|
100
122
|
|
101
123
|
ActiveRecord::Base.transaction do
|
102
124
|
export_type = config[:configurations][:export_type]
|
@@ -106,11 +128,12 @@ module BetterSeeder
|
|
106
128
|
end
|
107
129
|
end
|
108
130
|
|
131
|
+
ActiveRecord::Base.logger.level = previous_log_level
|
109
132
|
total_time = Time.now - start_time
|
110
133
|
log_statistics(stats, total_time)
|
111
134
|
end
|
112
135
|
|
113
|
-
def self.generate_structure(model_name:
|
136
|
+
def self.generate_structure(model_name:)
|
114
137
|
BetterSeeder::Builders::Structure.generate(model_name)
|
115
138
|
end
|
116
139
|
|
@@ -146,12 +169,17 @@ module BetterSeeder
|
|
146
169
|
# Recupera la configurazione specifica dal file di structure tramite il metodo seed_config.
|
147
170
|
# Se non definito, vengono usati dei valori di default.
|
148
171
|
seed_config = structure_class.respond_to?(:seed_config) ? structure_class.seed_config : {}
|
149
|
-
file_name
|
150
|
-
excluded_columns =
|
151
|
-
|
152
|
-
|
153
|
-
|
154
|
-
|
172
|
+
file_name = seed_config[:file_name] || "#{model_name.underscore}_seed"
|
173
|
+
excluded_columns = if export_type.to_s.downcase != 'sql'
|
174
|
+
seed_config.dig(:columns, :excluded) || []
|
175
|
+
else
|
176
|
+
[]
|
177
|
+
end
|
178
|
+
generate_data = seed_config.fetch(:generate_data, true)
|
179
|
+
count = seed_config[:count] || 10
|
180
|
+
load_data = seed_config.fetch(:load_data, true)
|
181
|
+
parent = seed_config[:parent] # nil oppure valore (o array) per modelli child
|
182
|
+
superclass = seed_config[:superclass] # nil oppure valore (o array) per modelli child
|
155
183
|
|
156
184
|
# Log per indicare se il modello è parent o child.
|
157
185
|
message = if parent.nil?
|
@@ -170,74 +198,54 @@ module BetterSeeder
|
|
170
198
|
raise Object.const_get(model_name)
|
171
199
|
end
|
172
200
|
|
173
|
-
#
|
174
|
-
|
175
|
-
|
176
|
-
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
|
181
|
-
|
182
|
-
|
201
|
+
# Se abilitato, carica i record nel database.
|
202
|
+
if load_data && File.exist?("#{BetterSeeder.configuration.preload_path.to_s}/#{seed_config[:file_name]}.sql")
|
203
|
+
load_data_from_file(seed_config)
|
204
|
+
stats[model_name] = model_class.all.count
|
205
|
+
else
|
206
|
+
if generate_data
|
207
|
+
records = Farms::Farmer.generate(model: model_name, count: count)
|
208
|
+
total_records = records.size
|
209
|
+
stats[model_name] = total_records
|
210
|
+
created_records = load_records_into_db(model_class, records, total_records, model_name, superclass)
|
211
|
+
# Se il modello è parent, salva i record creati per poterli utilizzare in seguito per i modelli child.
|
212
|
+
parent_loaded_records[model_name] = created_records if parent.nil?
|
213
|
+
else
|
214
|
+
model_class.all.map(&:attributes)
|
215
|
+
end
|
183
216
|
end
|
184
217
|
|
185
|
-
#
|
186
|
-
|
187
|
-
|
188
|
-
|
189
|
-
|
190
|
-
message = "[ERROR] No loaded records found for parent model #{parent_model_name}. Cannot assign foreign key for #{model_name}."
|
191
|
-
BetterSeeder::Utils.logger(message: message)
|
192
|
-
else
|
193
|
-
# Il nome della foreign key è ottenuto prendendo l'ultima parte del nome del modello padre,
|
194
|
-
# trasformandola in minuscolo e in forma singolare, e aggiungendo "_id".
|
195
|
-
foreign_key = parent_model_name.split("::").last.underscore.singularize + "_id"
|
196
|
-
processed_records.each do |record|
|
197
|
-
record[foreign_key] = parent_records.sample.id
|
198
|
-
end
|
199
|
-
end
|
218
|
+
# Rimuove le colonne escluse.
|
219
|
+
unless BetterSeeder.generated_records[(superclass || model_name).to_s].nil?
|
220
|
+
processed_records = BetterSeeder.generated_records[(superclass || model_name).to_s]
|
221
|
+
processed_records = processed_records.map do |campaign|
|
222
|
+
campaign.attributes.except(*excluded_columns.map(&:to_s))
|
200
223
|
end
|
201
|
-
end
|
202
224
|
|
203
|
-
|
204
|
-
|
205
|
-
total_records = processed_records.size
|
206
|
-
stats[model_name] = total_records
|
207
|
-
created_records = load_records_into_db(model_class, processed_records, total_records, model_name)
|
208
|
-
# Se il modello è parent, salva i record creati per poterli utilizzare in seguito per i modelli child.
|
209
|
-
parent_loaded_records[model_name] = created_records if parent.nil?
|
210
|
-
else
|
211
|
-
stats[model_name] = 0
|
225
|
+
# Esporta i record nel formato richiesto.
|
226
|
+
export_records(model_class, processed_records, export_type, file_name)
|
212
227
|
end
|
213
|
-
|
214
|
-
# Esporta i record nel formato richiesto.
|
215
|
-
export_records(model_class, processed_records, export_type, file_name)
|
216
228
|
end
|
217
229
|
|
218
230
|
# Carica i record nel database, utilizzando una progress bar per monitorare il progresso.
|
219
231
|
# I log delle query SQL vengono temporaneamente disabilitati.
|
220
232
|
#
|
221
233
|
# @return [Array<Object>] Array dei record creati (istanze ActiveRecord)
|
222
|
-
def self.load_records_into_db(model_class, processed_records, total_records, model_name)
|
223
|
-
created_records = []
|
234
|
+
def self.load_records_into_db(model_class, processed_records, total_records, model_name, superclass)
|
224
235
|
progressbar = ProgressBar.create(total: total_records, format: '%a %B %p%% %t')
|
225
236
|
message = "[INFO] Starting to load #{total_records} records for model #{model_name}..."
|
226
237
|
BetterSeeder::Utils.logger(message: message)
|
227
238
|
|
228
|
-
previous_level = ActiveRecord::Base.logger.level
|
229
|
-
ActiveRecord::Base.logger.level = Logger::ERROR
|
230
|
-
|
231
239
|
processed_records.each do |record|
|
232
240
|
created = model_class.create!(record)
|
233
|
-
|
241
|
+
BetterSeeder.store_generated_record(superclass || model_name, created)
|
234
242
|
progressbar.increment
|
235
243
|
end
|
236
244
|
|
237
|
-
ActiveRecord::Base.logger.level = previous_level
|
238
245
|
message = "[INFO] Finished loading #{total_records} records into model #{model_name}."
|
239
246
|
BetterSeeder::Utils.logger(message: message)
|
240
|
-
|
247
|
+
|
248
|
+
BetterSeeder.generated_records[model_name]
|
241
249
|
end
|
242
250
|
|
243
251
|
# Esporta i record nel formato specificato (json, csv, sql).
|
@@ -261,8 +269,8 @@ module BetterSeeder
|
|
261
269
|
|
262
270
|
# Log finale con le statistiche raccolte e il tempo totale di esecuzione.
|
263
271
|
def self.log_statistics(stats, total_time)
|
264
|
-
stats_message = stats.map { |model, count| "#{model}: #{count} records" }.join("
|
265
|
-
message = "[INFO] Finished processing all models in #{total_time.round(2)} seconds. Statistics: #{stats_message}"
|
272
|
+
stats_message = stats.map { |model, count| "#{model}: #{count} records" }.join("\n")
|
273
|
+
message = "[INFO] Finished processing all models in #{total_time.round(2)} seconds. Statistics: \n#{stats_message}"
|
266
274
|
BetterSeeder::Utils.logger(message: message)
|
267
275
|
end
|
268
276
|
|
@@ -271,4 +279,29 @@ module BetterSeeder
|
|
271
279
|
def self.transform_class_name(class_name)
|
272
280
|
class_name.split("::").map(&:underscore).join("_")
|
273
281
|
end
|
282
|
+
|
283
|
+
def self.load_data_from_file(seed_config)
|
284
|
+
config = seed_config
|
285
|
+
return unless config[:load_data] # Se load_data non è abilitato, esce senza fare nulla.
|
286
|
+
|
287
|
+
# Costruisce il nome del file di seed: ad esempio, se config[:file_name] è "my_model_seed",
|
288
|
+
# il file atteso sarà "my_model_seed_seed.sql".
|
289
|
+
seed_file_name = "#{config[:file_name]}.sql"
|
290
|
+
seed_file_path = File.join(BetterSeeder.configuration.preload_path, seed_file_name)
|
291
|
+
|
292
|
+
unless File.exist?(seed_file_path)
|
293
|
+
BetterSeeder::Utils.logger(message: "[WARN] Seed file not found: #{seed_file_path}")
|
294
|
+
return false
|
295
|
+
end
|
296
|
+
|
297
|
+
sql = File.read(seed_file_path)
|
298
|
+
begin
|
299
|
+
ActiveRecord::Base.connection.execute(sql)
|
300
|
+
BetterSeeder::Utils.logger(message: "[INFO] Loaded seed file: #{seed_file_path}")
|
301
|
+
true
|
302
|
+
rescue => e
|
303
|
+
BetterSeeder::Utils.logger(message: "[ERROR] Failed to load seed file: #{seed_file_path} - Error: #{e.message}")
|
304
|
+
false
|
305
|
+
end
|
306
|
+
end
|
274
307
|
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
# lib/generators/better_seeder/structure_generator.rb
|
2
|
+
require "rails/generators"
|
3
|
+
|
4
|
+
module BetterSeeder
|
5
|
+
class StructureGenerator < Rails::Generators::NamedBase
|
6
|
+
# Optionally, if you have a template directory, you can set it here:
|
7
|
+
# source_root File.expand_path("templates", __dir__)
|
8
|
+
|
9
|
+
def create_structure_file
|
10
|
+
say_status("info", "Generating structure file for #{name}", :green)
|
11
|
+
file_path = BetterSeeder.generate_structure(model_name: name)
|
12
|
+
say_status("info", "Structure file created at #{file_path}", :green)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: better_seeder
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- alessio_bussolari
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2025-02-
|
11
|
+
date: 2025-02-11 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: ffaker
|
@@ -143,7 +143,7 @@ files:
|
|
143
143
|
- lib/better_seeder/structure/utils.rb
|
144
144
|
- lib/better_seeder/utils.rb
|
145
145
|
- lib/better_seeder/version.rb
|
146
|
-
- lib/generators/better_seeder/
|
146
|
+
- lib/generators/better_seeder/structure_generator.rb
|
147
147
|
homepage: https://github.com/alessiobussolari/better_seeder
|
148
148
|
licenses:
|
149
149
|
- MIT
|
@@ -1,17 +0,0 @@
|
|
1
|
-
# lib/generators/better_seeder/structure/structure_generator.rb
|
2
|
-
require "rails/generators"
|
3
|
-
|
4
|
-
module BetterSeeder
|
5
|
-
module Structure
|
6
|
-
class StructureGenerator < Rails::Generators::NamedBase
|
7
|
-
# Optionally, if you have a template directory, you can set it here:
|
8
|
-
# source_root File.expand_path("templates", __dir__)
|
9
|
-
|
10
|
-
def create_structure_file
|
11
|
-
say_status("info", "Generating structure file for #{name}", :green)
|
12
|
-
file_path = BetterSeeder.generate_structure(model_name: name)
|
13
|
-
say_status("info", "Structure file created at #{file_path}", :green)
|
14
|
-
end
|
15
|
-
end
|
16
|
-
end
|
17
|
-
end
|