schema-model 0.7.3 → 0.7.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/dependabot.yml +18 -0
- data/.release-please-manifest.json +1 -1
- data/.rubocop.yml +0 -1
- data/ARCHITECTURE.md +2 -2
- data/CHANGELOG.md +9 -0
- data/README.md +10 -0
- data/bin/schema-json2csv +19 -33
- data/lib/schema/array_headers.rb +23 -6
- data/lib/schema/associations/base.rb +4 -1
- data/lib/schema/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f273128aa733d5737e486345a39b1c59f5af1bf3babea0917bc8d6e6a44707c8
|
|
4
|
+
data.tar.gz: 393e5e391cde21e5288da99d4321bb619f370d1a937052f427e32f570facd68e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: df5f93d0615667c6bb455bf920a08f44507f59eec3031cde8fb5e73eb99403738d0d3afe447d53a4a39d1dd67df7e8d3ff2f375810580f26a5804851289c8016
|
|
7
|
+
data.tar.gz: 01eb576246b0efa42c3935c0cb1c73038b350062053a767a815c8eaced1b4e1aec3b8c79fca3c32d2810fb26618b8eda397c4eb6b1ecc08e6baf79ff5467f138
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
version: 2
|
|
2
|
+
updates:
|
|
3
|
+
# keeps the SHA-pinned actions in .github/workflows current
|
|
4
|
+
- package-ecosystem: github-actions
|
|
5
|
+
directory: /
|
|
6
|
+
schedule:
|
|
7
|
+
interval: weekly
|
|
8
|
+
commit-message:
|
|
9
|
+
prefix: ci
|
|
10
|
+
include: scope
|
|
11
|
+
|
|
12
|
+
- package-ecosystem: bundler
|
|
13
|
+
directory: /
|
|
14
|
+
schedule:
|
|
15
|
+
interval: weekly
|
|
16
|
+
commit-message:
|
|
17
|
+
prefix: build
|
|
18
|
+
include: scope
|
data/.rubocop.yml
CHANGED
data/ARCHITECTURE.md
CHANGED
|
@@ -99,7 +99,7 @@ end
|
|
|
99
99
|
|
|
100
100
|
The `type_field` option tells SchemaCreator which key in the nested data determines the subclass (the type field attribute's aliases are checked too); `external_type_field` reads the type from an attribute on the parent instead. `type_ignorecase: true` compares type names case-insensitively. Each `add_type` creates a subclass of the association class, defined on the parent class as `<Name>AssociationType<Type>`.
|
|
101
101
|
|
|
102
|
-
`Associations::Base` stores the parent class by name and resolves it with `Object.const_get
|
|
102
|
+
`Associations::Base` stores the parent class by name and resolves it with `Object.const_get` (so reloaded constants are picked up); a parent that was anonymous when the association was declared is kept by reference instead.
|
|
103
103
|
|
|
104
104
|
### Schema::Utils (`lib/schema/utils.rb`)
|
|
105
105
|
|
|
@@ -127,7 +127,7 @@ Parsing errors use these codes from `Schema::ParsingErrors`: `invalid`, `incompa
|
|
|
127
127
|
|
|
128
128
|
### CSV and Arrays
|
|
129
129
|
|
|
130
|
-
- **`Schema::ArrayHeaders`** (`lib/schema/array_headers.rb`): `map_headers_to_attributes(headers)` returns a nested hash of `{ field: { index: n } }` entries. Has-many fields are matched as `<prefix><n><key>` (prefix is the association name or an alias, `n` counts from 1) and map to `{ indexes: [...] }`.
|
|
130
|
+
- **`Schema::ArrayHeaders`** (`lib/schema/array_headers.rb`): `map_headers_to_attributes(headers)` returns a nested hash of `{ field: { index: n } }` entries. Has-many fields are matched as `<prefix><n><key>` (prefix is the association name or an alias, `n` counts from 1) and map to `{ indexes: [...] }`. Has-one fields are matched by their own key or alias; a column already claimed by the parent or an earlier has-one isn't reused. Associations never map to a column of their own.
|
|
131
131
|
- **`Schema::Arrays`** (`lib/schema/arrays.rb`): `to_headers`, `to_empty_array`, `to_a` and `from_array(array, mapped_headers)`. Has-many associations need a `size:` option for the fixed-width methods.
|
|
132
132
|
- **`Schema::CSVParser`** (`lib/schema/csv_parser.rb`): wraps a `CSV` object, maps its header row once, and yields a model per row (`each`, `shift`, `missing_fields`).
|
|
133
133
|
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.7.4](https://github.com/dougyouch/schema/compare/v0.7.3...v0.7.4) (2026-10-04)
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
### Bug Fixes
|
|
7
|
+
|
|
8
|
+
* **array-headers:** fix nested field names, association columns and column collisions ([f0373e0](https://github.com/dougyouch/schema/commit/f0373e0450e9d474dcde5e02b1db76b9c7bbc7d4))
|
|
9
|
+
* **associations:** support dynamic types in anonymous parent classes ([45d22da](https://github.com/dougyouch/schema/commit/45d22da233ec97703ba3c3bd90a91e577bac5390))
|
|
10
|
+
* **cli:** write csv headers to stdout and lint schema-json2csv ([58ebb13](https://github.com/dougyouch/schema/commit/58ebb138d58cbddca60c2e7214ab980d3c0aae12))
|
|
11
|
+
|
|
3
12
|
## [0.7.3](https://github.com/dougyouch/schema/compare/v0.7.2...v0.7.3) (2026-10-04)
|
|
4
13
|
|
|
5
14
|
|
data/README.md
CHANGED
|
@@ -12,6 +12,12 @@ Add this line to your application's Gemfile:
|
|
|
12
12
|
|
|
13
13
|
```ruby
|
|
14
14
|
gem 'schema-model'
|
|
15
|
+
|
|
16
|
+
# needed for Schema::All and Schema::ActiveModelValidations (already present in Rails apps)
|
|
17
|
+
gem 'activemodel'
|
|
18
|
+
|
|
19
|
+
# needed for Schema::CSVParser and bin/schema-json2csv on Ruby 3.4+, where csv is no longer a default gem
|
|
20
|
+
gem 'csv'
|
|
15
21
|
```
|
|
16
22
|
|
|
17
23
|
And then execute:
|
|
@@ -20,6 +26,8 @@ And then execute:
|
|
|
20
26
|
$ bundle install
|
|
21
27
|
```
|
|
22
28
|
|
|
29
|
+
`Schema::Model` on its own has no dependencies beyond `inheritance-helper`.
|
|
30
|
+
|
|
23
31
|
## Quick Start
|
|
24
32
|
|
|
25
33
|
```ruby
|
|
@@ -521,6 +529,8 @@ schema-json2csv --require ./my_schema.rb --schema MySchema --json data.json --cs
|
|
|
521
529
|
cat data.json | schema-json2csv --require ./my_schema.rb --schema MySchema - --csv output.csv
|
|
522
530
|
```
|
|
523
531
|
|
|
532
|
+
The JSON can be one object or an array of objects. Output goes to stdout when `--csv` is omitted. A header row (from `to_headers`) is written to stdout or to a new file; an existing `--csv` file is appended to without repeating it. Has-many associations need a `size:` so every row has the same columns.
|
|
533
|
+
|
|
524
534
|
## Contributing
|
|
525
535
|
|
|
526
536
|
1. Fork the repository
|
data/bin/schema-json2csv
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
2
3
|
|
|
3
4
|
require 'schema-model'
|
|
4
5
|
require 'csv'
|
|
@@ -13,7 +14,7 @@ options = {
|
|
|
13
14
|
}
|
|
14
15
|
|
|
15
16
|
OptionParser.new do |opts|
|
|
16
|
-
opts.banner = "Usage:
|
|
17
|
+
opts.banner = "Usage: #{File.basename(__FILE__)} --require FILE --schema CLASS (--json FILE | -) [--csv FILE]"
|
|
17
18
|
|
|
18
19
|
opts.on('--json JSON_FILE', 'JSON data') do |v|
|
|
19
20
|
options[:json_file] = v
|
|
@@ -23,7 +24,7 @@ OptionParser.new do |opts|
|
|
|
23
24
|
options[:require_file] = v
|
|
24
25
|
end
|
|
25
26
|
|
|
26
|
-
opts.on('--csv CSV_FILE', 'File to output csv data') do |v|
|
|
27
|
+
opts.on('--csv CSV_FILE', 'File to output csv data (appended to if it exists)') do |v|
|
|
27
28
|
options[:csv_file] = v
|
|
28
29
|
end
|
|
29
30
|
|
|
@@ -32,54 +33,39 @@ OptionParser.new do |opts|
|
|
|
32
33
|
end
|
|
33
34
|
end.parse!
|
|
34
35
|
|
|
35
|
-
|
|
36
|
-
|
|
36
|
+
abort('no file required') unless options[:require_file]
|
|
37
|
+
abort('no schema specified') unless options[:schema_class_name]
|
|
38
|
+
abort("file #{options[:require_file]} not found") unless File.exist?(options[:require_file])
|
|
37
39
|
|
|
38
|
-
|
|
39
|
-
raise("file #{options[:require_file]} not found") unless require(options[:require_file])
|
|
40
|
+
require File.expand_path(options[:require_file])
|
|
40
41
|
|
|
41
|
-
|
|
42
|
+
abort("schema #{options[:schema_class_name]} not found") unless Object.const_defined?(options[:schema_class_name])
|
|
42
43
|
|
|
43
44
|
schema_class = Object.const_get(options[:schema_class_name])
|
|
44
45
|
schema_class.schema_include Schema::Arrays
|
|
45
46
|
|
|
46
47
|
json_data =
|
|
47
48
|
if options[:json_file]
|
|
48
|
-
|
|
49
|
+
abort("json file #{options[:json_file]} not found") unless File.exist?(options[:json_file])
|
|
49
50
|
File.read(options[:json_file])
|
|
50
51
|
elsif ARGV.last == '-'
|
|
51
|
-
|
|
52
|
+
$stdin.read
|
|
52
53
|
else
|
|
53
|
-
|
|
54
|
+
abort('no json data specified')
|
|
54
55
|
end
|
|
55
56
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
io.print str
|
|
62
|
-
end
|
|
57
|
+
# headers are written to stdout and to new files; an existing file is appended to as is
|
|
58
|
+
csv_file = options[:csv_file]
|
|
59
|
+
write_headers = csv_file.nil? || !File.exist?(csv_file)
|
|
60
|
+
io = csv_file ? File.open(csv_file, 'ab') : $stdout
|
|
63
61
|
|
|
64
|
-
io
|
|
65
|
-
if file = options[:csv_file]
|
|
66
|
-
if File.exist?(file)
|
|
67
|
-
File.open(options[:csv_file], 'ab')
|
|
68
|
-
else
|
|
69
|
-
headers = CSV.generate do |csv|
|
|
70
|
-
csv << schema_class.to_headers
|
|
71
|
-
end
|
|
72
|
-
f = File.open(options[:csv_file], 'wb')
|
|
73
|
-
f.print headers
|
|
74
|
-
f
|
|
75
|
-
end
|
|
76
|
-
else
|
|
77
|
-
$stdout
|
|
78
|
-
end
|
|
62
|
+
io.print CSV.generate_line(schema_class.to_headers) if write_headers
|
|
79
63
|
|
|
80
64
|
json_data = JSON.parse(json_data)
|
|
81
65
|
json_data = [json_data] unless json_data.is_a?(Array)
|
|
82
66
|
|
|
83
67
|
json_data.each do |data|
|
|
84
|
-
|
|
68
|
+
io.print CSV.generate_line(schema_class.from_hash(data).to_a.flatten)
|
|
85
69
|
end
|
|
70
|
+
|
|
71
|
+
io.close if csv_file
|
data/lib/schema/array_headers.rb
CHANGED
|
@@ -45,13 +45,11 @@ module Schema
|
|
|
45
45
|
|
|
46
46
|
private
|
|
47
47
|
|
|
48
|
+
# has_many columns are reported as <prefix>X<field>; has_one fields share the parent's prefix
|
|
48
49
|
def get_model_field_names(field_name, field_options, mapped_headers, header_prefix, mapped)
|
|
49
50
|
mapped_model = mapped_headers[field_name] || {}
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
header_prefix || field_options[:aliases]&.first,
|
|
53
|
-
mapped
|
|
54
|
-
)
|
|
51
|
+
header_prefix ||= (field_options[:aliases]&.first || field_name).to_s if field_options[:type] == :has_many
|
|
52
|
+
const_get(field_options[:class_name]).get_field_names(mapped_model, header_prefix, mapped)
|
|
55
53
|
end
|
|
56
54
|
|
|
57
55
|
def skip_field?(field_name, mapped_headers, mapped)
|
|
@@ -72,20 +70,36 @@ module Schema
|
|
|
72
70
|
const_get(field_options[:class_name]).map_headers_to_attributes(headers, header_prefix)
|
|
73
71
|
end
|
|
74
72
|
|
|
73
|
+
# a column already claimed by the parent (or an earlier has_one) isn't reused for a nested field
|
|
75
74
|
def map_headers_to_has_one_associations(headers, mapped_headers, header_prefix)
|
|
75
|
+
claimed_indexes = mapped_header_indexes(mapped_headers)
|
|
76
76
|
schema.each do |field_name, field_options|
|
|
77
77
|
next unless field_options[:type] == :has_one
|
|
78
78
|
# aliases are matched through the association's own entry
|
|
79
79
|
next if field_options[:alias_of]
|
|
80
80
|
|
|
81
|
-
|
|
81
|
+
available_headers = headers.each_with_index.map { |header, idx| claimed_indexes.include?(idx) ? nil : header }
|
|
82
|
+
mapped_model = get_mapped_model(field_options, available_headers, header_prefix)
|
|
82
83
|
next if mapped_model.empty?
|
|
83
84
|
|
|
84
85
|
mapped_headers[field_name] = mapped_model
|
|
86
|
+
claimed_indexes.concat(mapped_header_indexes(mapped_model))
|
|
85
87
|
end
|
|
86
88
|
mapped_headers
|
|
87
89
|
end
|
|
88
90
|
|
|
91
|
+
def mapped_header_indexes(mapped_headers)
|
|
92
|
+
mapped_headers.each_value.flat_map do |info|
|
|
93
|
+
if info.key?(:index)
|
|
94
|
+
[info[:index]]
|
|
95
|
+
elsif info.key?(:indexes)
|
|
96
|
+
info[:indexes]
|
|
97
|
+
else
|
|
98
|
+
mapped_header_indexes(info)
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
|
|
89
103
|
def map_headers_to_has_many_associations(headers, mapped_headers)
|
|
90
104
|
schema.each do |field_name, field_options|
|
|
91
105
|
next unless field_options[:type] == :has_many
|
|
@@ -104,6 +118,9 @@ module Schema
|
|
|
104
118
|
|
|
105
119
|
def map_headers_to_fields(headers, mapped_headers, header_prefix)
|
|
106
120
|
schema.each do |field_name, field_options|
|
|
121
|
+
# associations are mapped through their nested fields, not a column of their own
|
|
122
|
+
next if field_options[:association]
|
|
123
|
+
|
|
107
124
|
if header_prefix
|
|
108
125
|
unless (indexes = find_indexes_for_field(headers, field_options, header_prefix)).empty?
|
|
109
126
|
mapped_headers[field_options[:alias_of] || field_name] = { indexes: indexes }
|
|
@@ -12,12 +12,15 @@ module Schema
|
|
|
12
12
|
module ClassMethods
|
|
13
13
|
attr_accessor :schema_name
|
|
14
14
|
|
|
15
|
+
# named classes are looked up by name so reloaded constants are picked up;
|
|
16
|
+
# anonymous classes have no name, so they are kept by reference
|
|
15
17
|
def base_schema_class=(kls)
|
|
18
|
+
@base_schema_class = kls
|
|
16
19
|
@base_schema_class_name = kls.name
|
|
17
20
|
end
|
|
18
21
|
|
|
19
22
|
def base_schema_class
|
|
20
|
-
Object.const_get(@base_schema_class_name)
|
|
23
|
+
@base_schema_class_name ? Object.const_get(@base_schema_class_name) : @base_schema_class
|
|
21
24
|
end
|
|
22
25
|
|
|
23
26
|
def schema_options
|
data/lib/schema/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: schema-model
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.7.
|
|
4
|
+
version: 0.7.4
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Doug Youch
|
|
@@ -34,6 +34,7 @@ executables:
|
|
|
34
34
|
extensions: []
|
|
35
35
|
extra_rdoc_files: []
|
|
36
36
|
files:
|
|
37
|
+
- ".github/dependabot.yml"
|
|
37
38
|
- ".github/workflows/ci.yml"
|
|
38
39
|
- ".github/workflows/release.yml"
|
|
39
40
|
- ".gitignore"
|