pokotarou 1.1.2 → 1.1.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +38 -483
- data/lib/pokotarou.rb +36 -2
- data/lib/pokotarou/data_register.rb +124 -46
- data/lib/pokotarou/data_structure.rb +11 -9
- data/lib/pokotarou/expression_parser.rb +46 -12
- data/lib/pokotarou/pokotarou_handler.rb +4 -0
- data/lib/pokotarou/seeder.rb +0 -2
- data/lib/pokotarou/version.rb +1 -1
- metadata +3 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e5c1ca5e0b1537a768623eb035c99be512b4fa31a57714ee3f0b1b5a0d4e29db
|
4
|
+
data.tar.gz: 55fa713f56db0dc7d928ef3f4edc70727fc24a1c2c936d273fb58964d0cabc59
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 74de5054c2bab3a5c8908239a6973442c431486ab6018bb779f598cdde3de7f6d6b596237843bd336cd963c327635fbf05d7c7c96ca3abd039bf179558336ad8
|
7
|
+
data.tar.gz: 9e33defb98f9ada2032d6de9b4d300272a3dda290ad5a09590960e8412ea868616bbd110903ac53c0dd39a177ad57c2dc7b730160c2b676ff22a07221f613805
|
data/README.md
CHANGED
@@ -4,17 +4,21 @@
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/pokotarou.svg)](https://badge.fury.io/rb/pokotarou)
|
5
5
|
[![Build Status](https://travis-ci.org/Kashiwara0205/Pokotarou.svg?branch=master)](https://travis-ci.org/Kashiwara0205/Pokotarou)
|
6
6
|
|
7
|
-
Pokotarou is convenient seeder of 'Ruby on Rails'
|
8
|
-
In MySql, operation has been confirmed
|
9
|
-
|
10
7
|
## Features
|
11
8
|
|
12
|
-
|
13
|
-
|
14
|
-
|
9
|
+
__Pokotarou__ is convinient Mysql seeder of Ruby on Rails.
|
10
|
+
|
11
|
+
__Easy to use!!__
|
12
|
+
Just use yml file.
|
13
|
+
very very very simple!
|
14
|
+
You dont' have to write ruby program about seed data!
|
15
15
|
|
16
|
-
|
17
|
-
|
16
|
+
__Fast speed!!__
|
17
|
+
__Pokotarou__ is fast seeder.
|
18
|
+
Because contains [Activerecord-import](https://github.com/zdennis/activerecord-import).
|
19
|
+
so, always run bulk insert when execute rails db:seed command!
|
20
|
+
|
21
|
+
If you use __Pokotarou__ about following table.
|
18
22
|
|
19
23
|
|Field|Type|NULL|
|
20
24
|
|:---|:---|:---|
|
@@ -23,9 +27,7 @@ If it is the following table, 10,000 records can regist in 0.41s on average
|
|
23
27
|
|created_at|datetime|NO|
|
24
28
|
|updated_at|datetime|NO|
|
25
29
|
|
26
|
-
|
27
|
-
|
28
|
-
|
30
|
+
__Pokotarou__ can register in 0.41s on average.
|
29
31
|
|
30
32
|
## Getting started
|
31
33
|
Add this line to your application's Gemfile:
|
@@ -44,40 +46,12 @@ Or install it yourself as:
|
|
44
46
|
$ gem install pokotarou
|
45
47
|
```
|
46
48
|
|
47
|
-
##
|
48
|
-
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|
49
|
-
|
50
|
-
## Usage
|
51
|
-
|
52
|
-
Following yml file become seed data.
|
53
|
-
|
54
|
-
Please make following yml file in your favorite dir.
|
55
|
-
|
56
|
-
```yml
|
57
|
-
Default:
|
58
|
-
Pref:
|
59
|
-
loop: 3
|
60
|
-
```
|
61
|
-
|
62
|
-
and write following ruby code in seeds.rb.
|
63
|
-
|
64
|
-
```ruby
|
65
|
-
Pokotarou.execute("./config_filepath")
|
66
|
-
```
|
67
|
-
|
68
|
-
when you finished writing, then run rails db:seed
|
69
|
-
|
70
|
-
```bash
|
71
|
-
$ rails db:seed
|
72
|
-
```
|
73
|
-
|
74
|
-
As a result, seed data is registerd your db.
|
49
|
+
## Basic Usage
|
75
50
|
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
Table name below is 'prefs' and model name is 'Pref'
|
51
|
+
Just make yml file.
|
52
|
+
If you register data to following prefecture data.
|
53
|
+
|
54
|
+
__prefecture table__
|
81
55
|
|
82
56
|
|Field|Type|NULL|
|
83
57
|
|:---|:---|:---|
|
@@ -87,463 +61,44 @@ Table name below is 'prefs' and model name is 'Pref'
|
|
87
61
|
|updated_at|datetime|NO|
|
88
62
|
|
89
63
|
|
90
|
-
|
91
|
-
|
92
|
-
|Field|Type|NULL|
|
93
|
-
|:---|:---|:---|
|
94
|
-
|id|bigint(20)| NO|
|
95
|
-
|name|varchar(255)|YES|
|
96
|
-
|remarks|text|YES|
|
97
|
-
|birthday|date|YES|
|
98
|
-
|pref_id|bigint(20)|YES|
|
99
|
-
|created_at|datetime|NO|
|
100
|
-
|updated_at|datetime|NO|
|
101
|
-
|
102
|
-
|
103
|
-
### Standerd Setting
|
104
|
-
The basic setting method is written below
|
105
|
-
|
106
|
-
#### Automatic data entry
|
107
|
-
|
108
|
-
If there is no definition about col, then automatically prepared data is registrd.
|
109
|
-
|
110
|
-
For example, in the case of below, it is registered automatically prepared data three times.
|
111
|
-
|
112
|
-
```yml
|
113
|
-
Default:
|
114
|
-
Pref:
|
115
|
-
loop: 3
|
116
|
-
```
|
117
|
-
|
118
|
-
also you can set seed_data by yourself.
|
119
|
-
```yml
|
120
|
-
Default:
|
121
|
-
Pref:
|
122
|
-
loop: 3
|
123
|
-
col:
|
124
|
-
name: "Hokkaido"
|
125
|
-
```
|
126
|
-
|
127
|
-
#### Omitted loop
|
128
|
-
If you want to register the test data at once, I suggest ommited loop
|
129
|
-
|
130
|
-
```yml
|
131
|
-
Default:
|
132
|
-
Pref:
|
133
|
-
col:
|
134
|
-
name: ["Hokkaido"]
|
135
|
-
```
|
136
|
-
|
137
|
-
```ruby
|
138
|
-
["Hokkaido"]
|
139
|
-
```
|
140
|
-
|
141
|
-
#### Array
|
142
|
-
You can set array_data.
|
143
|
-
|
144
|
-
Array data is registerd one by one.
|
145
|
-
|
146
|
-
```yml
|
147
|
-
Default:
|
148
|
-
Pref:
|
149
|
-
loop: 3
|
150
|
-
col:
|
151
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
152
|
-
```
|
153
|
-
|
154
|
-
#### Maked function
|
155
|
-
'maked' is very useful function.
|
156
|
-
it is hash and accumulate data created in the past.
|
157
|
-
|
158
|
-
For example, in the case of below, reffer name of Pref in Default block by maked
|
159
|
-
|
160
|
-
```yml
|
161
|
-
Default:
|
162
|
-
Pref:
|
163
|
-
loop: 2
|
164
|
-
col:
|
165
|
-
name: ["Hokkaido", "Aomori"]
|
166
|
-
Member:
|
167
|
-
loop: 2
|
168
|
-
col:
|
169
|
-
name: <maked[:Default][:Pref][:name]>
|
170
|
-
pref_id: F|Pref
|
171
|
-
```
|
172
|
-
|
173
|
-
#### Foreign key
|
174
|
-
|
175
|
-
**※ If you set association(belongs_to, has_many...), Pokotarou automatically register foreign keys**
|
176
|
-
|
177
|
-
' F| ' means foreign key. 'F|' is Model.all.pluck(:id)
|
178
|
-
|
179
|
-
For example, in the case of below, Member model record is registerd with pref_id(foregin key).
|
180
|
-
|
181
|
-
```yml
|
182
|
-
Default:
|
183
|
-
Pref:
|
184
|
-
loop: 3
|
185
|
-
col:
|
186
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
187
|
-
Member:
|
188
|
-
loop: 3
|
189
|
-
col:
|
190
|
-
pref_id: F|Pref
|
191
|
-
```
|
192
|
-
|
193
|
-
#### Expression expansion
|
194
|
-
'< >' means expression expansion.
|
195
|
-
You can run ruby code in '< >'.
|
196
|
-
|
197
|
-
```yml
|
198
|
-
Default:
|
199
|
-
Pref:
|
200
|
-
loop: 3
|
201
|
-
col:
|
202
|
-
name: <["Hokkaido", "Aomori", "Iwate"]>
|
203
|
-
created_at: <Date.parse('1997/02/05')>
|
204
|
-
```
|
205
|
-
|
206
|
-
#### Additional method
|
207
|
-
You can add method and use it in pokotarou
|
208
|
-
|
209
|
-
```yml
|
210
|
-
Default:
|
211
|
-
Pref:
|
212
|
-
loop: 3
|
213
|
-
col:
|
214
|
-
name: <pref_name>
|
215
|
-
```
|
216
|
-
|
217
|
-
Prepare the following ruby file
|
218
|
-
|
219
|
-
```ruby
|
220
|
-
def pref_name
|
221
|
-
["Hokkaido", "Aomori", "Iwate"]
|
222
|
-
end
|
223
|
-
```
|
224
|
-
|
225
|
-
and run the following code in seeds.rb.
|
226
|
-
|
227
|
-
```ruby
|
228
|
-
Pokotarou.import("./method_filepath")
|
229
|
-
Pokotarou.execute("./config_filepath")
|
230
|
-
```
|
231
|
-
|
232
|
-
As as result, pokotarou can call pref_name method, and seed data is registrd by pref_name method.
|
233
|
-
|
234
|
-
#### Multiple blocks
|
235
|
-
|
236
|
-
You can use multiple blocks.
|
237
|
-
|
238
|
-
```yml
|
239
|
-
Default:
|
240
|
-
Pref:
|
241
|
-
loop: 3
|
242
|
-
Default2:
|
243
|
-
Pref:
|
244
|
-
loop: 3
|
245
|
-
```
|
246
|
-
|
247
|
-
and, You can change the name of the block
|
248
|
-
|
249
|
-
```yml
|
250
|
-
Hoge:
|
251
|
-
Pref:
|
252
|
-
loop: 3
|
253
|
-
Fuga:
|
254
|
-
Pref:
|
255
|
-
loop: 3
|
256
|
-
```
|
257
|
-
|
258
|
-
### option
|
259
|
-
Option is useful function.
|
260
|
-
If you can master it, it may be easier to create test data.
|
261
|
-
|
262
|
-
#### Random
|
263
|
-
Shuffle seed data when regist
|
264
|
-
|
265
|
-
```yml
|
266
|
-
Default:
|
267
|
-
Pref:
|
268
|
-
loop: 3
|
269
|
-
col:
|
270
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
271
|
-
option:
|
272
|
-
name: ["random"]
|
273
|
-
```
|
274
|
-
|
275
|
-
The following results change from run to run
|
276
|
-
|
277
|
-
```ruby
|
278
|
-
["Aomori", "Iwate", "Iwate"]
|
279
|
-
```
|
280
|
-
|
281
|
-
#### Add_id
|
282
|
-
Add individual number to seed data of String type
|
283
|
-
|
284
|
-
```yml
|
285
|
-
Default:
|
286
|
-
Pref:
|
287
|
-
loop: 3
|
288
|
-
col:
|
289
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
290
|
-
option:
|
291
|
-
name: ["add_id"]
|
292
|
-
```
|
293
|
-
|
294
|
-
```ruby
|
295
|
-
["Hokkaido_0", "Aomori"_1, "Iwate_2"]
|
296
|
-
```
|
297
|
-
|
298
|
-
#### Combine serveral options
|
299
|
-
Combination of options is possible
|
300
|
-
|
301
|
-
```yml
|
302
|
-
Default:
|
303
|
-
Pref:
|
304
|
-
loop: 3
|
305
|
-
col:
|
306
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
307
|
-
option:
|
308
|
-
name: ["add_id", "random"]
|
309
|
-
```
|
310
|
-
|
311
|
-
The following results change from run to run
|
312
|
-
|
313
|
-
```ruby
|
314
|
-
["Hokkaido_0", "Iwate_1", "Hokkaido_2"]
|
315
|
-
```
|
316
|
-
|
317
|
-
### Advanced Setting
|
318
|
-
The advanced setting method is written below
|
319
|
-
|
320
|
-
#### Validation
|
321
|
-
|
322
|
-
Run validation when regist
|
323
|
-
|
324
|
-
```yml
|
325
|
-
Default:
|
326
|
-
Pref:
|
327
|
-
loop: 3
|
328
|
-
validate: true
|
329
|
-
```
|
330
|
-
|
331
|
-
#### Disable Autoincrement
|
332
|
-
|
333
|
-
You can disable the autoincrement setting
|
334
|
-
|
335
|
-
If you disable the setting, you can register id data prepared by yourself
|
336
|
-
|
337
|
-
```yml
|
338
|
-
Default:
|
339
|
-
Pref:
|
340
|
-
loop: 3
|
341
|
-
autoincrement: false
|
342
|
-
col:
|
343
|
-
id: [100, 101, 102]
|
344
|
-
```
|
345
|
-
|
346
|
-
#### Pokotarou Handler
|
347
|
-
|
348
|
-
If you want to use configlation yml data in ruby code then you can use "PokotarouHandler"
|
349
|
-
|
350
|
-
When you use "PokotarouHandler", can update pokotarou's parameter
|
351
|
-
in ruby code.
|
352
|
-
|
353
|
-
|
354
|
-
<b>Change Operation</b>
|
355
|
-
|
356
|
-
In the following example, the number of loops is changed
|
357
|
-
|
358
|
-
```ruby
|
359
|
-
handler = Pokotarou.gen_handler("./config_filepath")
|
360
|
-
# change loop config
|
361
|
-
handler.change_loop(:Default, :Pref, 6)
|
362
|
-
Pokotarou.execute(handler.get_data)
|
363
|
-
```
|
364
|
-
|
365
|
-
|
366
|
-
In the following example, seed data is changed
|
367
|
-
|
368
|
-
```ruby
|
369
|
-
handler = Pokotarou.gen_handler("./config_filepath")
|
370
|
-
# change seed data config number
|
371
|
-
handler.change_seed(:Default, :Pref, :name, ["a", "b", "c"])
|
372
|
-
Pokotarou.execute(handler.get_data)
|
373
|
-
```
|
374
|
-
|
375
|
-
<b>Delete Operation</b>
|
376
|
-
|
377
|
-
In the following example, delete block config
|
378
|
-
|
379
|
-
```ruby
|
380
|
-
handler = Pokotarou.gen_handler("./config_filepath")
|
381
|
-
# delete model config in parameter
|
382
|
-
handler.delete_block(:Default)
|
383
|
-
Pokotarou.execute(handler.get_data)
|
384
|
-
```
|
385
|
-
|
386
|
-
In the following example, delete model config
|
64
|
+
__prefecture model__
|
387
65
|
|
388
|
-
|
389
|
-
|
390
|
-
|
391
|
-
handler.delete_model(:Default, :Pref)
|
392
|
-
Pokotarou.execute(handler.get_data)
|
393
|
-
```
|
394
|
-
|
395
|
-
In the following example, delete col config
|
396
|
-
|
397
|
-
```ruby
|
398
|
-
handler = Pokotarou.gen_handler("./config_filepath")
|
399
|
-
# delete col config in parameter
|
400
|
-
handler.delete_col(:Default, :Pref, :name)
|
401
|
-
Pokotarou.execute(handler.get_data)
|
402
|
-
```
|
66
|
+
|Model|
|
67
|
+
|:---|
|
68
|
+
|Pref|
|
403
69
|
|
404
|
-
|
405
|
-
|
70
|
+
First, Please make following yml file your favorite directory of rails project.
|
71
|
+
The file name can be anything.
|
72
|
+
In my case, made yml file in db directory and named file __pref_data__.
|
406
73
|
|
407
74
|
```yml
|
408
|
-
const':
|
409
|
-
name: "hoge"
|
410
75
|
Default:
|
411
76
|
Pref:
|
412
77
|
loop: 3
|
413
|
-
col:
|
414
|
-
name: <const[:name]>
|
415
|
-
```
|
416
|
-
|
417
|
-
#### Grouping
|
418
|
-
Grouping is very useful function.
|
419
|
-
Especially useful when setting multiple options.
|
420
|
-
|
421
|
-
|
422
|
-
```yml
|
423
|
-
Default:
|
424
|
-
Member:
|
425
|
-
grouping:
|
426
|
-
# set columns you want to group
|
427
|
-
hoge_g: ["name", "remark"]
|
428
|
-
col:
|
429
|
-
# you can use "hoge_g" at col
|
430
|
-
hoge_g: <['fugafuga!']>
|
431
|
-
option:
|
432
|
-
# also you can use "hoge_g" at option
|
433
|
-
hoge_g: ["add_id"]
|
434
|
-
|
435
|
-
```
|
436
|
-
|
437
|
-
#### Template
|
438
|
-
You can set template config by template' key.
|
439
|
-
|
440
|
-
The template can be overwritten with the one set later.
|
441
|
-
|
442
|
-
```yml
|
443
|
-
template':
|
444
|
-
pref_template:
|
445
|
-
loop: 3
|
446
|
-
col:
|
447
|
-
pref_id: F|Pref
|
448
|
-
name: ["hogeta", "fuga", "pokota"]
|
449
|
-
Pref:
|
450
|
-
Pref:
|
451
|
-
loop: 3
|
452
|
-
col:
|
453
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
454
|
-
|
455
|
-
Member1:
|
456
|
-
Member:
|
457
|
-
template: pref_template
|
458
|
-
|
459
|
-
Member2:
|
460
|
-
Member:
|
461
|
-
template: pref_template
|
462
|
-
col:
|
463
|
-
name: ["hogeta2", "fuga2", "pokota2"]
|
464
78
|
```
|
465
79
|
|
466
|
-
|
467
|
-
You can set return val by return' key.
|
468
|
-
|
469
|
-
```yml
|
470
|
-
Default:
|
471
|
-
Pref:
|
472
|
-
loop: 3
|
473
|
-
col:
|
474
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
475
|
-
|
476
|
-
return': <maked[:Default][:Pref][:name]>
|
477
|
-
|
478
|
-
```
|
80
|
+
and write following ruby code in seeds.rb.
|
479
81
|
|
480
82
|
```ruby
|
481
|
-
|
482
|
-
puts return_val
|
483
|
-
```
|
484
|
-
|
485
|
-
result
|
486
|
-
```
|
487
|
-
Hokkaido
|
488
|
-
Aomori
|
489
|
-
Iwate
|
83
|
+
Pokotarou.execute("./db/pref_data.yml")
|
490
84
|
```
|
491
|
-
#### Args
|
492
85
|
|
493
|
-
|
86
|
+
and execute following command!!!!!
|
494
87
|
|
495
|
-
```
|
496
|
-
|
497
|
-
Pref:
|
498
|
-
loop: 3
|
499
|
-
col:
|
500
|
-
name: <args[:name]>
|
501
|
-
```
|
502
|
-
```ruby
|
503
|
-
Pokotarou.set_args({ name: ["Hokkaido", "Aomori", "Iwate"] })
|
504
|
-
Pokotarou.execute("filepath")
|
88
|
+
```bash
|
89
|
+
$ rails db:seed
|
505
90
|
```
|
506
91
|
|
507
|
-
|
508
|
-
|
509
|
-
|
510
|
-
#### convert option
|
511
|
-
|
512
|
-
|convert |description |
|
513
|
-
|:---------|------------------------------------------|
|
514
|
-
| empty | convert val to empty |
|
515
|
-
| nil | convert val to nil |
|
516
|
-
| big_text | convert val to big_text("text" * 50) |
|
517
|
-
| br_text | convert val to br_text("text\n" * 5) |
|
518
|
-
|
519
|
-
For example, following configfile register seed data while replacing with nil
|
520
|
-
|
521
|
-
```yml
|
522
|
-
Default:
|
523
|
-
Pref:
|
524
|
-
loop: 3
|
525
|
-
col:
|
526
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
527
|
-
convert:
|
528
|
-
name: ["nil(0..2)"]
|
529
|
-
```
|
92
|
+
prefecture data is registerd your db.
|
93
|
+
Let's check with the following code.
|
530
94
|
|
531
95
|
```ruby
|
532
|
-
|
96
|
+
# You have to get 3
|
97
|
+
Pref.all.count => 3
|
533
98
|
```
|
534
99
|
|
535
|
-
|
100
|
+
## Documentaion
|
101
|
+
You can read a lot more about Pokotarou in its [official docs](https://kashiwara0205.github.io/PokotarouDocs/)
|
536
102
|
|
537
|
-
|
538
|
-
|
539
|
-
Pref:
|
540
|
-
loop: 3
|
541
|
-
col:
|
542
|
-
name: ["Hokkaido", "Aomori", "Iwate"]
|
543
|
-
convert:
|
544
|
-
name: ["empty(0..0)", "nil(1..2)"]
|
545
|
-
```
|
546
|
-
|
547
|
-
```ruby
|
548
|
-
["", nil, nil]
|
549
|
-
```
|
103
|
+
## License
|
104
|
+
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|
data/lib/pokotarou.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
Dir[File.expand_path('../pokotarou', __FILE__) << '/*.rb'].each do |file|
|
2
2
|
require file
|
3
3
|
end
|
4
|
+
|
4
5
|
require "activerecord-import"
|
5
6
|
|
6
7
|
module Pokotarou
|
@@ -10,10 +11,35 @@ module Pokotarou
|
|
10
11
|
def execute input
|
11
12
|
# if input is filepath, generate config_data
|
12
13
|
if input.kind_of?(String)
|
13
|
-
DataRegister.
|
14
|
+
DataRegister.register(gen_config(input))
|
14
15
|
else
|
15
|
-
DataRegister.
|
16
|
+
DataRegister.register(input)
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
def pipleine_execute input_arr
|
21
|
+
return_vals = []
|
22
|
+
input_arr.each do |e|
|
23
|
+
handler = gen_handler_with_cache(e[:filepath])
|
24
|
+
|
25
|
+
if e[:change_data].present?
|
26
|
+
e[:change_data].each do |block, config|
|
27
|
+
config.each do |model, seed|
|
28
|
+
seed.each do |col_name, val|
|
29
|
+
handler.change_seed(block, model, col_name, val)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
e[:args] ||= {}
|
36
|
+
e[:args][:passed_return_val] = return_vals.last
|
37
|
+
set_args(e[:args])
|
38
|
+
|
39
|
+
return_vals << Pokotarou.execute(handler.get_data())
|
16
40
|
end
|
41
|
+
|
42
|
+
return_vals
|
17
43
|
end
|
18
44
|
|
19
45
|
def import filepath
|
@@ -27,12 +53,20 @@ module Pokotarou
|
|
27
53
|
def reset
|
28
54
|
AdditionalMethods.remove()
|
29
55
|
Arguments.remove()
|
56
|
+
@handler_chache = {}
|
30
57
|
end
|
31
58
|
|
32
59
|
def gen_handler filepath
|
33
60
|
PokotarouHandler.new(gen_config(filepath))
|
34
61
|
end
|
35
62
|
|
63
|
+
def gen_handler_with_cache filepath
|
64
|
+
@handler_cache ||= {}
|
65
|
+
@handler_cache[filepath] ||= PokotarouHandler.new(gen_config(filepath))
|
66
|
+
|
67
|
+
@handler_cache[filepath].deep_dup
|
68
|
+
end
|
69
|
+
|
36
70
|
private
|
37
71
|
|
38
72
|
def gen_config filepath
|
@@ -1,58 +1,112 @@
|
|
1
1
|
class RegistError < StandardError; end
|
2
|
+
class SettingError < StandardError; end
|
2
3
|
class SeedError < StandardError; end
|
3
4
|
|
4
5
|
class DataRegister
|
5
6
|
class << self
|
6
|
-
def
|
7
|
+
def register data
|
7
8
|
# init maked to accumulate maded data
|
8
|
-
maked =
|
9
|
+
maked = {}
|
10
|
+
maked_col = {}
|
9
11
|
# init model_data to cache data of model
|
10
|
-
model_cache =
|
12
|
+
model_cache = {}
|
11
13
|
ActiveRecord::Base.transaction do
|
12
14
|
begin
|
13
15
|
data.each do |sym_block, model_data|
|
14
16
|
next if is_dush?(sym_block.to_s)
|
15
|
-
|
17
|
+
setting_register_val_for_bulk(sym_block, model_data, maked, model_cache, maked_col)
|
16
18
|
end
|
17
19
|
rescue => e
|
18
20
|
raise StandardError.new("#{e.message}")
|
19
21
|
end
|
20
22
|
end
|
21
23
|
|
22
|
-
ReturnExpressionParser.parse(data[:"return'"], maked)
|
24
|
+
ReturnExpressionParser.parse(data[:"return'"], maked, maked_col)
|
23
25
|
end
|
24
26
|
|
25
27
|
private
|
26
28
|
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
29
|
+
=begin
|
30
|
+
Commented because the influence range was large
|
31
|
+
Used in later versions
|
32
|
+
|
33
|
+
def merge_block data
|
34
|
+
bulk_hash = {}
|
35
|
+
data.each do |block, val|
|
36
|
+
next if is_dush?(block.to_s)
|
37
|
+
val.each do |model, config|
|
38
|
+
if bulk_hash[model].nil?
|
39
|
+
bulk_hash[model] = config
|
40
|
+
else
|
41
|
+
merge_loop(model, config, bulk_hash) if config[:loop].present?
|
42
|
+
merge_seed_data(model, config, bulk_hash) if config[:col].present?
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
bulk_hash
|
48
|
+
end
|
49
|
+
=end
|
50
|
+
|
51
|
+
def merge_loop model, config, bulk_hash
|
52
|
+
bulk_hash[model][:loop] += config[:loop]
|
53
|
+
end
|
54
|
+
|
55
|
+
def merge_seed_data model, config, bulk_hash
|
56
|
+
config[:col].each do |col_name, seed|
|
57
|
+
bulk_hash[model][:col][col_name].concat(seed)
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
def setting_register_val_for_bulk sym_block, model_data, maked, model_cache, maked_col
|
62
|
+
begin
|
63
|
+
model_data.each do |e|
|
64
|
+
str_model = e.first.to_s
|
65
|
+
save_model_cache(model_cache, str_model)
|
66
|
+
|
67
|
+
model = model_cache[str_model][:model]
|
68
|
+
sym_model = model_cache[str_model][:sym_model]
|
69
|
+
|
70
|
+
# model_data.values is config_data
|
71
|
+
config_data = e.second
|
72
|
+
# set expand expression for loop '<>' and ':' and so on...
|
73
|
+
set_loop_expand_expression(config_data, maked, maked_col)
|
74
|
+
# if there is no setting data, set default seed data
|
75
|
+
set_default_seed(config_data)
|
76
|
+
# seed_arr: [[col1_element, col1_element], [col2_element, col2_element]...]
|
77
|
+
set_seed_arr(model, sym_block, sym_model, config_data, maked, maked_col)
|
78
|
+
|
79
|
+
output_log(config_data[:log])
|
80
|
+
register_by_bulk(sym_block, str_model, config_data ,model_cache)
|
81
|
+
end
|
82
|
+
rescue => e
|
83
|
+
raise SettingError.new("
|
84
|
+
Failed setting for bulk insert...
|
85
|
+
|
86
|
+
block: #{sym_block}
|
87
|
+
model: #{str_model}
|
88
|
+
message: #{e.message}
|
89
|
+
")
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
def register_by_bulk sym_block, str_model, config_data, model_cache
|
94
|
+
# col_arr: [:col1, :col2, :col3]
|
95
|
+
col_arr = config_data[:col].keys
|
96
|
+
# seed_arr: [[elem1, elem2, elem3...]]
|
97
|
+
seed_arr = config_data[:col].map{|_, val| val }.transpose
|
98
|
+
|
46
99
|
begin
|
47
|
-
model_cache[str_model][:model].import(col_arr, seed_arr
|
100
|
+
model_cache[str_model][:model].import(col_arr, seed_arr, validate: config_data[:validate], timestamps: false)
|
48
101
|
rescue => e
|
49
102
|
raise RegistError.new("
|
103
|
+
Failed bulk insert...
|
104
|
+
|
50
105
|
block: #{sym_block}
|
51
106
|
model: #{str_model}
|
52
107
|
message: #{e.message}
|
53
108
|
")
|
54
109
|
end
|
55
|
-
end
|
56
110
|
end
|
57
111
|
|
58
112
|
def save_model_cache model_cache, str_model
|
@@ -74,23 +128,30 @@ class DataRegister
|
|
74
128
|
# if there is data already, skip
|
75
129
|
next if config_data[:col][key].present?
|
76
130
|
|
77
|
-
config_data[:col][key] = Seeder.gen(config_data, key)
|
131
|
+
config_data[:col][key] = Seeder.gen(config_data, key)
|
78
132
|
end
|
79
133
|
end
|
80
|
-
|
81
|
-
def
|
134
|
+
|
135
|
+
def set_seed_arr model, sym_block, sym_model, config_data, maked, maked_col
|
82
136
|
options = config_data[:option]
|
83
137
|
convert_conf = config_data[:convert]
|
84
138
|
loop_size = config_data[:loop]
|
85
139
|
|
86
140
|
if apply_autoincrement?(config_data[:autoincrement])
|
87
|
-
|
141
|
+
prev_id_arr =
|
142
|
+
if maked_col[sym_model].present? && maked_col[sym_model].has_key?(:id)
|
143
|
+
maked_col[sym_model][:id]
|
144
|
+
else
|
145
|
+
[]
|
146
|
+
end
|
147
|
+
|
148
|
+
set_autoincrement(config_data, model, loop_size, prev_id_arr)
|
88
149
|
end
|
89
150
|
|
90
|
-
config_data[:col].
|
91
|
-
begin
|
151
|
+
config_data[:col].each do |key, val|
|
152
|
+
begin
|
92
153
|
# set expand expression '<>' and ':' and so on...
|
93
|
-
set_expand_expression(config_data, key, val, maked)
|
154
|
+
set_expand_expression(config_data, key, val, maked, maked_col)
|
94
155
|
expanded_val = config_data[:col][key]
|
95
156
|
expanded_val_size = expanded_val.size
|
96
157
|
|
@@ -104,7 +165,7 @@ class DataRegister
|
|
104
165
|
|
105
166
|
# Take count yourself, because .with_index is slow
|
106
167
|
cnt = 0
|
107
|
-
seeds =
|
168
|
+
seeds =
|
108
169
|
loop_size.times.map do
|
109
170
|
seed =
|
110
171
|
option_conf.nil? ? get_seed(expanded_val, expanded_val_size, cnt) : get_seed_with_option(expanded_val, expanded_val_size, option_conf, cnt)
|
@@ -112,11 +173,13 @@ class DataRegister
|
|
112
173
|
|
113
174
|
seed
|
114
175
|
end
|
115
|
-
update_maked_data(maked, sym_block, sym_model, key, seeds)
|
116
176
|
|
117
|
-
seeds
|
177
|
+
update_maked_data(maked, sym_block, sym_model, key, seeds )
|
178
|
+
update_maked_col(maked_col, sym_model, key, config_data[:col][key])
|
179
|
+
config_data[:col][key] = seeds
|
118
180
|
rescue => e
|
119
181
|
raise SeedError.new("
|
182
|
+
Failed generate seed data...
|
120
183
|
block: #{sym_block}
|
121
184
|
model: #{sym_model}
|
122
185
|
column: #{key}
|
@@ -132,25 +195,32 @@ class DataRegister
|
|
132
195
|
autoincrement_flg
|
133
196
|
end
|
134
197
|
|
135
|
-
def set_autoincrement config_data, model, loop_size
|
198
|
+
def set_autoincrement config_data, model, loop_size, prev_id_arr
|
136
199
|
last_record = model.last
|
137
|
-
|
138
|
-
|
200
|
+
current_id =
|
201
|
+
if prev_id_arr.present?
|
202
|
+
prev_id_arr.last
|
203
|
+
elsif last_record.nil?
|
204
|
+
0
|
205
|
+
else
|
206
|
+
last_record.id
|
207
|
+
end
|
208
|
+
|
139
209
|
additions = current_id + loop_size
|
140
210
|
next_id = current_id + 1
|
141
211
|
|
142
212
|
config_data[:col][:id] = [*next_id..additions]
|
143
213
|
end
|
144
214
|
|
145
|
-
def set_expand_expression config_data, key, val, maked
|
215
|
+
def set_expand_expression config_data, key, val, maked, maked_col
|
146
216
|
# if it exists type, there is no need for doing 'expand expression'
|
147
217
|
return if config_data[:type][key].present?
|
148
|
-
config_data[:col][key] = SeedExpressionParser.parse(val, maked)
|
218
|
+
config_data[:col][key] = SeedExpressionParser.parse(val, maked, maked_col)
|
149
219
|
end
|
150
220
|
|
151
|
-
def set_loop_expand_expression config_data, maked
|
152
|
-
config_data[:loop] =
|
153
|
-
LoopExpressionParser.parse(config_data[:loop], maked)
|
221
|
+
def set_loop_expand_expression config_data, maked, maked_col
|
222
|
+
config_data[:loop] =
|
223
|
+
LoopExpressionParser.parse(config_data[:loop], maked, maked_col)
|
154
224
|
end
|
155
225
|
|
156
226
|
def get_seed arr, size, cnt
|
@@ -163,11 +233,19 @@ class DataRegister
|
|
163
233
|
|
164
234
|
def update_maked_data maked, sym_block, sym_model, col, seed
|
165
235
|
# maked: { key: Model, value: {key: col1, val: [col1_element, col1_element]} }
|
166
|
-
maked[sym_block] ||=
|
167
|
-
maked[sym_block][sym_model] ||=
|
236
|
+
maked[sym_block] ||= {}
|
237
|
+
maked[sym_block][sym_model] ||= {}
|
168
238
|
maked[sym_block][sym_model][col] = seed
|
169
239
|
end
|
170
240
|
|
241
|
+
def update_maked_col maked_col, sym_model, column, vals
|
242
|
+
maked_col[sym_model] ||= {}
|
243
|
+
maked_col[sym_model][column] ||= []
|
244
|
+
maked_col[sym_model][column].concat(vals)
|
245
|
+
|
246
|
+
maked_col
|
247
|
+
end
|
248
|
+
|
171
249
|
def output_log log
|
172
250
|
return if log.nil?
|
173
251
|
puts log
|
@@ -4,7 +4,7 @@ class DataStructure
|
|
4
4
|
execute_template_option_setting(data)
|
5
5
|
# return data structure bellow
|
6
6
|
# [{ block_name => { model_name => { column_configration }}}, ...]
|
7
|
-
data.reduce(
|
7
|
+
data.reduce({}) do |acc, r|
|
8
8
|
if is_dush?(r.first.to_s)
|
9
9
|
acc[r.first] = r.second
|
10
10
|
else
|
@@ -46,7 +46,7 @@ class DataStructure
|
|
46
46
|
def deep_overwrite from_hash, to_hash
|
47
47
|
from_hash.each do |key, val|
|
48
48
|
if val.kind_of?(Hash)
|
49
|
-
to_hash[key] ||=
|
49
|
+
to_hash[key] ||= {}
|
50
50
|
deep_overwrite(val, to_hash[key])
|
51
51
|
else
|
52
52
|
to_hash[key] = val
|
@@ -88,7 +88,7 @@ class DataStructure
|
|
88
88
|
end
|
89
89
|
|
90
90
|
def gen_structure model_data
|
91
|
-
model_data.reduce(
|
91
|
+
model_data.reduce({}) do |acc, r|
|
92
92
|
# r.second is config_data, like {loop: 3, ...}
|
93
93
|
set_col_type(r.second, r[0].to_s)
|
94
94
|
acc[r[0]] = r.second
|
@@ -101,7 +101,7 @@ class DataStructure
|
|
101
101
|
model = eval(str_model)
|
102
102
|
foreign_key_data = get_foreign_key_data(model)
|
103
103
|
|
104
|
-
config_data[:col] ||=
|
104
|
+
config_data[:col] ||= {}
|
105
105
|
model.columns.each do |e|
|
106
106
|
symbol_col_name = e.name.to_sym
|
107
107
|
|
@@ -111,20 +111,22 @@ class DataStructure
|
|
111
111
|
config_data[:col][symbol_col_name] = nil
|
112
112
|
|
113
113
|
# set type info
|
114
|
-
config_data[:type] ||=
|
114
|
+
config_data[:type] ||= {}
|
115
115
|
config_data[:type][symbol_col_name] = e.type.to_s
|
116
116
|
|
117
117
|
# set enum info
|
118
|
-
config_data[:enum] ||=
|
118
|
+
config_data[:enum] ||= {}
|
119
119
|
if is_enum?(e.sql_type.to_s)
|
120
120
|
config_data[:enum][symbol_col_name] =
|
121
121
|
e.sql_type.to_s[5..-2].tr("'", "").split(",")
|
122
122
|
end
|
123
123
|
|
124
124
|
# set foreign_key info
|
125
|
-
config_data[:foreign_key] ||= Hash.new
|
126
125
|
if is_foreign_key?(symbol_col_name, foreign_key_data)
|
127
|
-
|
126
|
+
# delete type val for don't run default seeder
|
127
|
+
config_data[:type].delete(symbol_col_name)
|
128
|
+
# use F function for set foreign key
|
129
|
+
config_data[:col][symbol_col_name] = "F|#{foreign_key_data[symbol_col_name].to_s}"
|
128
130
|
end
|
129
131
|
end
|
130
132
|
|
@@ -134,7 +136,7 @@ class DataStructure
|
|
134
136
|
def get_foreign_key_data model
|
135
137
|
associations = model.reflect_on_all_associations(:belongs_to)
|
136
138
|
return { } if associations.empty?
|
137
|
-
associations.reduce(
|
139
|
+
associations.reduce({})do |acc, r|
|
138
140
|
model = r.name.to_s.camelize
|
139
141
|
if Object.const_defined?(model.to_sym)
|
140
142
|
acc[r.foreign_key.to_sym] = eval(model)
|
@@ -3,23 +3,29 @@ require "pokotarou/additional_variables/additional_variables.rb"
|
|
3
3
|
require "pokotarou/arguments/arguments.rb"
|
4
4
|
class ParseError < StandardError; end
|
5
5
|
FOREIGN_KEY_SYMBOL = "F|"
|
6
|
+
COLUMN_SYMBOL = "C|"
|
6
7
|
|
7
8
|
class ExpressionParser
|
8
9
|
class << self
|
9
|
-
def parse config_val, maked = nil
|
10
|
+
def parse config_val, maked = nil, maked_col = nil
|
10
11
|
begin
|
11
12
|
case
|
13
|
+
|
12
14
|
# Array
|
13
15
|
when is_array?(config_val)
|
14
16
|
array_procees(config_val)
|
15
17
|
|
16
18
|
# ForeignKey
|
17
19
|
when is_foreign_key?(config_val)
|
18
|
-
foreign_key_process(config_val)
|
20
|
+
foreign_key_process(config_val, maked_col)
|
21
|
+
|
22
|
+
# Column
|
23
|
+
when is_column_symbol?(config_val)
|
24
|
+
column_symbol_process(config_val, maked_col)
|
19
25
|
|
20
26
|
# Expression
|
21
27
|
when is_expression?(config_val)
|
22
|
-
expression_process(config_val, maked)
|
28
|
+
expression_process(config_val, maked, maked_col)
|
23
29
|
|
24
30
|
# Integer
|
25
31
|
when is_integer?(config_val)
|
@@ -43,20 +49,28 @@ class ExpressionParser
|
|
43
49
|
return val
|
44
50
|
end
|
45
51
|
|
46
|
-
def foreign_key_process val
|
52
|
+
def foreign_key_process val, maked_col
|
47
53
|
# remove 'F|'
|
48
54
|
str_model = val.sub(FOREIGN_KEY_SYMBOL, "")
|
49
55
|
model = eval(str_model)
|
50
56
|
return model.pluck(:id)
|
51
57
|
end
|
52
58
|
|
53
|
-
def
|
59
|
+
def column_symbol_process val, maked_col
|
60
|
+
# remove 'C|'
|
61
|
+
str_model, column = val.sub(COLUMN_SYMBOL, "").split("|")
|
62
|
+
model = eval(str_model)
|
63
|
+
elemnts = model.pluck(column.to_sym)
|
64
|
+
return elemnts.concat(maked_col[str_model.to_sym][column.to_sym])
|
65
|
+
end
|
66
|
+
|
67
|
+
def expression_process val, maked, maked_col
|
54
68
|
# remove '<>'
|
55
69
|
expression = val.strip[1..-2]
|
56
70
|
require AdditionalVariables.filepath if AdditionalVariables.const.present?
|
57
71
|
require AdditionalMethods.filepath if AdditionalMethods.filepath.present?
|
58
72
|
require Arguments.filepath if Arguments.filepath.present?
|
59
|
-
return self.parse(eval(expression), maked)
|
73
|
+
return self.parse(eval(expression), maked, maked_col)
|
60
74
|
end
|
61
75
|
|
62
76
|
def integer_process val
|
@@ -100,6 +114,20 @@ class ReturnExpressionParser < ExpressionParser
|
|
100
114
|
def output_error e
|
101
115
|
ParseError.new("Failed Const Expression parse:#{e.message}")
|
102
116
|
end
|
117
|
+
|
118
|
+
def foreign_key_process val, _
|
119
|
+
# remove 'F|'
|
120
|
+
str_model = val.sub(FOREIGN_KEY_SYMBOL, "")
|
121
|
+
model = eval(str_model)
|
122
|
+
return model.pluck(:id)
|
123
|
+
end
|
124
|
+
|
125
|
+
def column_symbol_process val, _
|
126
|
+
# remove 'C|'
|
127
|
+
str_model, column = val.sub(COLUMN_SYMBOL, "").split("|")
|
128
|
+
model = eval(str_model)
|
129
|
+
return model.pluck(column.to_sym)
|
130
|
+
end
|
103
131
|
end
|
104
132
|
end
|
105
133
|
|
@@ -111,7 +139,7 @@ class LoopExpressionParser < ExpressionParser
|
|
111
139
|
val.size
|
112
140
|
end
|
113
141
|
|
114
|
-
def foreign_key_process val
|
142
|
+
def foreign_key_process val, maked_col
|
115
143
|
# remove 'F|'
|
116
144
|
str_model = val.sub(FOREIGN_KEY_SYMBOL, "")
|
117
145
|
model = eval(str_model)
|
@@ -136,7 +164,7 @@ end
|
|
136
164
|
class ConstExpressionParser < ExpressionParser
|
137
165
|
class << self
|
138
166
|
private
|
139
|
-
def expression_process val, _
|
167
|
+
def expression_process val, _, _
|
140
168
|
# remove '<>'
|
141
169
|
expression = val.strip[1..-2]
|
142
170
|
require AdditionalMethods.filepath if AdditionalMethods.filepath.present?
|
@@ -155,16 +183,22 @@ class ConstExpressionParser < ExpressionParser
|
|
155
183
|
end
|
156
184
|
end
|
157
185
|
|
158
|
-
|
186
|
+
FOREIGN_KEY_REGEXP = /^F\|[A-Z][:A-Za-z0-9]*$/
|
159
187
|
def is_foreign_key? val
|
160
188
|
return false unless val.kind_of?(String)
|
161
|
-
|
189
|
+
FOREIGN_KEY_REGEXP =~ val
|
190
|
+
end
|
191
|
+
|
192
|
+
COLUMN_REGEXP = /^C\|[A-Z][:A-Za-z0-9]*\|[a-z][:a-z0-9]*$/
|
193
|
+
def is_column_symbol? val
|
194
|
+
return false unless val.kind_of?(String)
|
195
|
+
COLUMN_REGEXP =~ val
|
162
196
|
end
|
163
197
|
|
164
|
-
|
198
|
+
EXPRESSION_REGEXP = /^\s*<.*>\s*$/
|
165
199
|
def is_expression? val
|
166
200
|
return false unless val.kind_of?(String)
|
167
|
-
|
201
|
+
EXPRESSION_REGEXP =~ val
|
168
202
|
end
|
169
203
|
|
170
204
|
def is_array? val
|
data/lib/pokotarou/seeder.rb
CHANGED
@@ -4,9 +4,7 @@ class Seeder
|
|
4
4
|
n = config_data[:loop]
|
5
5
|
type = config_data[:type][key]
|
6
6
|
enum = config_data[:enum][key]
|
7
|
-
foreign_key = config_data[:foreign_key][key]
|
8
7
|
|
9
|
-
return foreign_key.pluck(:id) if foreign_key.present?
|
10
8
|
return enum if enum.present?
|
11
9
|
return make_array(n, ->(){ rand(100) }) if type == "integer"
|
12
10
|
return make_array(n, ->(){ rand(0.0..100.0) }) if type == "float"
|
data/lib/pokotarou/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pokotarou
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.1.
|
4
|
+
version: 1.1.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Kashiwara
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2020-02-08 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rails
|
@@ -84,8 +84,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
84
84
|
- !ruby/object:Gem::Version
|
85
85
|
version: '0'
|
86
86
|
requirements: []
|
87
|
-
|
88
|
-
rubygems_version: 2.7.3
|
87
|
+
rubygems_version: 3.0.1
|
89
88
|
signing_key:
|
90
89
|
specification_version: 4
|
91
90
|
summary: This gem is seeder which is very usefull
|