tarantool-schema 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/README.md +66 -6
- data/lib/tarantool/schema/alter.rb +20 -0
- data/lib/tarantool/schema/comparison.rb +7 -3
- data/lib/tarantool/schema/creation.rb +14 -0
- data/lib/tarantool/schema/definition.rb +20 -5
- data/lib/tarantool/schema/format_comparison.rb +26 -5
- data/lib/tarantool/schema/inspector.rb +1 -1
- data/lib/tarantool/schema/live_definition.rb +46 -0
- data/lib/tarantool/schema/live_space.rb +2 -2
- data/lib/tarantool/schema/plan.rb +16 -6
- data/lib/tarantool/schema/source.rb +108 -0
- data/lib/tarantool/schema/space_lister.rb +24 -0
- data/lib/tarantool/schema/version.rb +1 -1
- data/lib/tarantool/schema.rb +4 -0
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b3260cbb85da71cf020abb9e22fabfb4feb286ecd95a8b6c4b6547208f7f6f06
|
|
4
|
+
data.tar.gz: 40d311e9036ffcacd2c65d5968d27f07ea33b3d49ebc643366d72f07e1d00d00
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e4eb8745bb8495e4a66a51c07fe8b734de1054b5dae1691fe27a465173d3be8511e113c5c56a9ac037f42fe1de1926caac599a200148517ab371cefe96e032f3
|
|
7
|
+
data.tar.gz: b59ea715167985b0f98f989a026f56941a5633e8a8de283e311acbf2652e51f8a0338b903465bdcea58cca0d917eb8dc822c46fc0eb205c32278b5daa00cdeae
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.2.0 (2026-10-17)
|
|
4
|
+
|
|
5
|
+
- `Definition.from_live(client, only:)`: the live user spaces as a definition, for bootstrapping an existing
|
|
6
|
+
database; `Definition#to_ruby` renders a definition as DSL source.
|
|
7
|
+
- `plan`/`apply` take `alter: [:add_field]` to append nullable fields to the end of a live format. Without it a
|
|
8
|
+
longer declared format stays a mismatch, as before.
|
|
9
|
+
|
|
3
10
|
## 0.1.0 (2026-10-17)
|
|
4
11
|
|
|
5
12
|
First release.
|
data/README.md
CHANGED
|
@@ -2,8 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
Idempotent schema declaration for Tarantool 2.11+ and 3.x, meant for deploys. You declare spaces, indexes and
|
|
4
4
|
primary key sequences in Ruby; `apply` creates whatever is missing and checks everything that already exists.
|
|
5
|
-
If the live schema differs from the declaration, it raises an error with a diff and creates nothing. It
|
|
6
|
-
|
|
5
|
+
If the live schema differs from the declaration, it raises an error with a diff and creates nothing. It alters
|
|
6
|
+
only what you allow (appending nullable fields) and never drops anything: changing a field type or rebuilding an
|
|
7
|
+
index is a migration and is out of scope. An existing database can be read back into the DSL with `from_live`.
|
|
7
8
|
|
|
8
9
|
## Installation
|
|
9
10
|
|
|
@@ -95,7 +96,7 @@ Connect to the read-write instance. `apply`:
|
|
|
95
96
|
2. compares it with the declaration and collects the differences across all spaces;
|
|
96
97
|
3. if there are any, raises `Tarantool::Schema::MismatchError` and creates nothing;
|
|
97
98
|
4. creates the missing spaces and indexes in declaration order, with `if_not_exists` so two deploys racing each
|
|
98
|
-
other do not fail;
|
|
99
|
+
other do not fail (and sets the format of spaces that get new fields, see Altering);
|
|
99
100
|
5. reads everything again and raises `MismatchError` if something is still missing or differs, for example
|
|
100
101
|
when another deploy created the space differently in the meantime. Racing another deploy can also surface
|
|
101
102
|
as `Tarantool::ServerError` raised directly from `create_index`, before this re-verification runs.
|
|
@@ -119,6 +120,64 @@ schema mismatch:
|
|
|
119
120
|
|
|
120
121
|
`MismatchError#mismatches` returns the same lines as an Array, each prefixed with `space <name>: `.
|
|
121
122
|
|
|
123
|
+
## Altering
|
|
124
|
+
|
|
125
|
+
Appending nullable fields to the end of a live format is the one change `apply` makes to an existing space, and
|
|
126
|
+
only when asked:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
MyApp::SCHEMA.plan(client, alter: [:add_field]) # => ["format codes: add phone, archived_at"]
|
|
130
|
+
MyApp::SCHEMA.apply(client, alter: [:add_field]) # => ["format codes: add phone, archived_at"]
|
|
131
|
+
MyApp::SCHEMA.apply(client, alter: [:add_field]) # => []
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
`alter:` takes `:add_field` only; anything else raises `ArgumentError`. It applies when the live format is a
|
|
135
|
+
prefix of the declared one (same names, types and nullability) and every field past it is `is_nullable: true`,
|
|
136
|
+
which is what Tarantool requires while the space holds tuples. The whole declared format is sent with
|
|
137
|
+
`box.space[name]:format(...)`, before any new index of that space, so an index on a new field works. Existing
|
|
138
|
+
tuples read the new fields as nil.
|
|
139
|
+
|
|
140
|
+
Everything else stays a mismatch: a field that is not nullable (`field 4 flag: must be is_nullable: true to be
|
|
141
|
+
added`), a live format longer than the declared one, a field differing inside the prefix. Nothing is ever
|
|
142
|
+
dropped.
|
|
143
|
+
|
|
144
|
+
## Bootstrapping from a live schema
|
|
145
|
+
|
|
146
|
+
```ruby
|
|
147
|
+
definition = Tarantool::Schema::Definition.from_live(client) # every user space
|
|
148
|
+
definition = Tarantool::Schema::Definition.from_live(client, only: %i[codes users])
|
|
149
|
+
definition.plan(client) # => []
|
|
150
|
+
File.write('config/tarantool_schema.rb', definition.to_ruby)
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
`from_live` reads the spaces with ids past the system range whose names do not start with `_` (vshard's
|
|
154
|
+
`_bucket` and crud's `_ddl_*` stay out); `only:` names the spaces to read instead, with or without a leading
|
|
155
|
+
underscore, and raises `ArgumentError` for a name that does not exist. Each space is read as `ensure_space`
|
|
156
|
+
would declare it: engine, format, indexes in id order with their parts. A part that inherits its type and
|
|
157
|
+
nullability from the field is a bare field name; a JSON path part keeps its type. Index options the DSL does not
|
|
158
|
+
know (`hint`, collations, `exclude_null`, vinyl tuning) are not read, as `plan` ignores them anyway. A
|
|
159
|
+
functional index, or a part pointing past the format, raises `ArgumentError` naming the space and index: the
|
|
160
|
+
DSL cannot express them.
|
|
161
|
+
|
|
162
|
+
`to_ruby` renders a definition as source for a `Tarantool::Schema::Definition.new do ... end` block, defaults
|
|
163
|
+
left out, one space per `ensure_space`, `ensure_index` entries after them; it evaluates back to the same `to_h`:
|
|
164
|
+
|
|
165
|
+
```ruby
|
|
166
|
+
Tarantool::Schema::Definition.new do
|
|
167
|
+
ensure_space :codes,
|
|
168
|
+
format: [
|
|
169
|
+
{ name: :id, type: :unsigned },
|
|
170
|
+
{ name: :code, type: :string },
|
|
171
|
+
{ name: :expires_at, type: :unsigned, is_nullable: true }
|
|
172
|
+
],
|
|
173
|
+
indexes: [
|
|
174
|
+
{ name: :primary, parts: [:id], sequence: true },
|
|
175
|
+
{ name: :by_code, parts: [:code] },
|
|
176
|
+
{ name: :by_expiry, parts: [:expires_at], unique: false }
|
|
177
|
+
]
|
|
178
|
+
end
|
|
179
|
+
```
|
|
180
|
+
|
|
122
181
|
## Rake
|
|
123
182
|
|
|
124
183
|
```ruby
|
|
@@ -138,9 +197,9 @@ namespace :tarantool do
|
|
|
138
197
|
with_schema_client { |client| puts MyApp::SCHEMA.plan(client) }
|
|
139
198
|
end
|
|
140
199
|
|
|
141
|
-
desc 'Create missing Tarantool spaces and indexes; fail on drift'
|
|
200
|
+
desc 'Create missing Tarantool spaces and indexes, append nullable fields; fail on drift'
|
|
142
201
|
task :apply do
|
|
143
|
-
with_schema_client { |client| puts MyApp::SCHEMA.apply(client) }
|
|
202
|
+
with_schema_client { |client| puts MyApp::SCHEMA.apply(client, alter: [:add_field]) }
|
|
144
203
|
end
|
|
145
204
|
end
|
|
146
205
|
end
|
|
@@ -218,7 +277,8 @@ INTEGRATION=1 TARANTOOL_URI=tcp://app:secret@localhost:3302 bundle exec rspec
|
|
|
218
277
|
|
|
219
278
|
## Public API
|
|
220
279
|
|
|
221
|
-
`Tarantool::Schema::Definition`
|
|
280
|
+
`Tarantool::Schema::Definition` (`new`, `from_live`, `ensure_space`, `ensure_index`, `plan`, `apply`, `to_h`,
|
|
281
|
+
`to_ruby`) and the error classes above.
|
|
222
282
|
|
|
223
283
|
Everything else, including anything tagged `@api private` in its source comment, is an implementation detail
|
|
224
284
|
and may change in any release.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Tarantool
|
|
4
|
+
module Schema
|
|
5
|
+
# What plan and apply may change on a live space besides creating what is missing. Each entry is opted into
|
|
6
|
+
# by name; nothing here ever drops or rewrites a field or an index.
|
|
7
|
+
# @api private
|
|
8
|
+
module Alter
|
|
9
|
+
ALLOWED = %i[add_field].freeze
|
|
10
|
+
|
|
11
|
+
def self.check(list)
|
|
12
|
+
list = Array(list).map(&:to_sym)
|
|
13
|
+
unknown = list - ALLOWED
|
|
14
|
+
return list.freeze if unknown.empty?
|
|
15
|
+
|
|
16
|
+
raise ArgumentError, "alter: unknown #{unknown.map(&:inspect).join(', ')}; allowed: #{ALLOWED.join(', ')}"
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
end
|
|
@@ -7,14 +7,16 @@ module Tarantool
|
|
|
7
7
|
# from the live format.
|
|
8
8
|
# @api private
|
|
9
9
|
class Comparison
|
|
10
|
-
attr_reader :name, :space, :mismatches, :missing_indexes
|
|
10
|
+
attr_reader :name, :space, :mismatches, :missing_indexes, :format_additions
|
|
11
11
|
|
|
12
|
-
def initialize(name:, indexes:, live:, space: nil)
|
|
12
|
+
def initialize(name:, indexes:, live:, space: nil, alter: [])
|
|
13
13
|
@name = name
|
|
14
14
|
@space = space
|
|
15
15
|
@live = live
|
|
16
|
+
@alter = alter
|
|
16
17
|
@mismatches = []
|
|
17
18
|
@missing_indexes = []
|
|
19
|
+
@format_additions = []
|
|
18
20
|
live ? compare(indexes) : absent(indexes)
|
|
19
21
|
end
|
|
20
22
|
|
|
@@ -37,7 +39,9 @@ module Tarantool
|
|
|
37
39
|
|
|
38
40
|
def compare_space
|
|
39
41
|
@mismatches << "engine: declared #{@space.engine}, actual #{@live.engine}" if @space.engine != @live.engine
|
|
40
|
-
|
|
42
|
+
format = FormatComparison.new(@space.format, @live.format, add_field: @alter.include?(:add_field))
|
|
43
|
+
@mismatches.concat(format.mismatches)
|
|
44
|
+
@format_additions = format.additions
|
|
41
45
|
compare_primary
|
|
42
46
|
end
|
|
43
47
|
|
|
@@ -16,6 +16,11 @@ module Tarantool
|
|
|
16
16
|
box.space[space]:create_index(name, options)
|
|
17
17
|
LUA
|
|
18
18
|
|
|
19
|
+
FORMAT_LUA = <<~LUA
|
|
20
|
+
local name, format = ...
|
|
21
|
+
box.space[name]:format(format)
|
|
22
|
+
LUA
|
|
23
|
+
|
|
19
24
|
Space = Data.define(:spec) do
|
|
20
25
|
def apply(client)
|
|
21
26
|
client.eval(SPACE_LUA, [spec.name.to_s, spec.to_lua.merge('if_not_exists' => true)])
|
|
@@ -24,6 +29,15 @@ module Tarantool
|
|
|
24
29
|
def to_s = "space #{spec.name}"
|
|
25
30
|
end
|
|
26
31
|
|
|
32
|
+
# The whole declared format goes out: the live one is a prefix of it, verified before apply.
|
|
33
|
+
Format = Data.define(:space, :fields) do
|
|
34
|
+
def apply(client)
|
|
35
|
+
client.eval(FORMAT_LUA, [space.name.to_s, space.format.map(&:to_lua)])
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def to_s = "format #{space.name}: add #{fields.map(&:name).join(', ')}"
|
|
39
|
+
end
|
|
40
|
+
|
|
27
41
|
Index = Data.define(:space, :spec) do
|
|
28
42
|
def apply(client)
|
|
29
43
|
client.eval(INDEX_LUA, [space.to_s, spec.name.to_s, spec.to_lua.merge('if_not_exists' => true)])
|
|
@@ -14,6 +14,17 @@ module Tarantool
|
|
|
14
14
|
freeze
|
|
15
15
|
end
|
|
16
16
|
|
|
17
|
+
# Every user space as the server has it, or only the named ones. A space the DSL cannot express raises.
|
|
18
|
+
def self.from_live(client, only: nil)
|
|
19
|
+
arguments = SpaceLister.call(client, only:).to_h do |name|
|
|
20
|
+
live = Inspector.call(client, name) || raise(ArgumentError, "space #{name}: not found")
|
|
21
|
+
|
|
22
|
+
[name, LiveDefinition.arguments(name, live)]
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
new { arguments.each { |name, space| ensure_space(name, **space) } }
|
|
26
|
+
end
|
|
27
|
+
|
|
17
28
|
def ensure_space(name, format:, indexes:, engine: :memtx)
|
|
18
29
|
raise ArgumentError, 'space without name' if name.nil?
|
|
19
30
|
|
|
@@ -40,21 +51,25 @@ module Tarantool
|
|
|
40
51
|
|
|
41
52
|
def to_h = @spaces.transform_values(&:to_h)
|
|
42
53
|
|
|
43
|
-
def
|
|
54
|
+
def to_ruby = Source.render(to_h, @external.transform_values { |indexes| indexes.map(&:to_h) })
|
|
55
|
+
|
|
56
|
+
# alter: lists the changes to live spaces that are allowed besides creating what is missing, see Alter.
|
|
57
|
+
def plan(client, alter: []) = build_plan(client, Alter.check(alter)).verify.creations.map(&:to_s)
|
|
44
58
|
|
|
45
59
|
# The second plan re-reads everything, so a space another deploy created differently in the meantime fails
|
|
46
60
|
# here instead of being reported as created.
|
|
47
|
-
def apply(client)
|
|
48
|
-
|
|
61
|
+
def apply(client, alter: [])
|
|
62
|
+
alter = Alter.check(alter)
|
|
63
|
+
creations = build_plan(client, alter).verify.creations
|
|
49
64
|
creations.each { |creation| creation.apply(client) }
|
|
50
|
-
build_plan(client).verify(complete: true)
|
|
65
|
+
build_plan(client, alter).verify(complete: true)
|
|
51
66
|
|
|
52
67
|
creations.map(&:to_s)
|
|
53
68
|
end
|
|
54
69
|
|
|
55
70
|
private
|
|
56
71
|
|
|
57
|
-
def build_plan(client) = Plan.new(spaces: @spaces, external: @external, client:)
|
|
72
|
+
def build_plan(client, alter) = Plan.new(spaces: @spaces, external: @external, client:, alter:)
|
|
58
73
|
|
|
59
74
|
# A space declared elsewhere already has its primary key, so an index added here never gets a sequence.
|
|
60
75
|
def add_external(space, index)
|
|
@@ -2,14 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
module Tarantool
|
|
4
4
|
module Schema
|
|
5
|
-
# Differences between a declared format and the live one, field by field in order.
|
|
5
|
+
# Differences between a declared format and the live one, field by field in order. With add_field, declared
|
|
6
|
+
# fields past the end of a matching live format are additions instead of a mismatch, as long as they are
|
|
7
|
+
# nullable: Tarantool rejects a longer format otherwise while the space holds tuples.
|
|
6
8
|
# @api private
|
|
7
9
|
class FormatComparison
|
|
8
10
|
ATTRIBUTES = %i[name type is_nullable].freeze
|
|
9
11
|
|
|
10
|
-
def initialize(declared, live)
|
|
12
|
+
def initialize(declared, live, add_field: false)
|
|
11
13
|
@declared = declared
|
|
12
14
|
@live = live
|
|
15
|
+
@add_field = add_field
|
|
13
16
|
end
|
|
14
17
|
|
|
15
18
|
def mismatches
|
|
@@ -17,15 +20,33 @@ module Tarantool
|
|
|
17
20
|
live ? field_mismatches(index + 1, declared, live) : []
|
|
18
21
|
end
|
|
19
22
|
|
|
20
|
-
count_mismatch + fields
|
|
23
|
+
count_mismatch(fields.empty?) + fields
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# The declared fields to append, when that is allowed and the live format is a matching prefix.
|
|
27
|
+
def additions
|
|
28
|
+
return [] unless @add_field && mismatches.empty? && @live.size < @declared.size
|
|
29
|
+
|
|
30
|
+
extra
|
|
21
31
|
end
|
|
22
32
|
|
|
23
33
|
private
|
|
24
34
|
|
|
25
|
-
def
|
|
35
|
+
def extra = @declared.drop(@live.size)
|
|
36
|
+
|
|
37
|
+
def count_mismatch(prefix_matches)
|
|
26
38
|
return [] if @declared.size == @live.size
|
|
39
|
+
return [] if prefix_matches && @add_field && @live.size < @declared.size && extra.all?(&:is_nullable)
|
|
27
40
|
|
|
28
|
-
["format: declared #{@declared.size} fields, actual #{@live.size}"]
|
|
41
|
+
["format: declared #{@declared.size} fields, actual #{@live.size}"] + addition_hints
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def addition_hints
|
|
45
|
+
return [] unless @add_field && @live.size < @declared.size
|
|
46
|
+
|
|
47
|
+
extra.reject(&:is_nullable).map do |field|
|
|
48
|
+
"field #{@declared.index(field) + 1} #{field.name}: must be is_nullable: true to be added"
|
|
49
|
+
end
|
|
29
50
|
end
|
|
30
51
|
|
|
31
52
|
def field_mismatches(number, declared, live)
|
|
@@ -17,7 +17,7 @@ module Tarantool
|
|
|
17
17
|
path = part.path})
|
|
18
18
|
end
|
|
19
19
|
table.insert(indexes, {id = id, name = index.name, type = index.type, unique = index.unique,
|
|
20
|
-
parts = parts, sequence = index.sequence_id ~= nil})
|
|
20
|
+
parts = parts, sequence = index.sequence_id ~= nil, func = index.func ~= nil})
|
|
21
21
|
end
|
|
22
22
|
end
|
|
23
23
|
return {engine = space.engine, format = space:format(), indexes = indexes}
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Tarantool
|
|
4
|
+
module Schema
|
|
5
|
+
# The ensure_space arguments that declare a live space as it is. A part is written as a bare field name when
|
|
6
|
+
# it inherits everything from the field, so the result reads like a hand-written declaration.
|
|
7
|
+
# @api private
|
|
8
|
+
module LiveDefinition
|
|
9
|
+
module_function
|
|
10
|
+
|
|
11
|
+
def arguments(name, live)
|
|
12
|
+
fields = live.fields
|
|
13
|
+
|
|
14
|
+
{ engine: live.engine, format: live.format.map(&:to_h),
|
|
15
|
+
indexes: live.indexes.map { |index| index_arguments(name, index, fields) } }
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def index_arguments(space, index, fields)
|
|
19
|
+
raise ArgumentError, "space #{space}: index #{index.name} is functional, not supported" if index.func
|
|
20
|
+
|
|
21
|
+
{ name: index.name, type: index.type, unique: index.unique, sequence: index.sequence,
|
|
22
|
+
parts: index.parts.map { |part| part_argument(space, index, part, fields) } }
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def part_argument(space, index, part, fields)
|
|
26
|
+
field = fields[part.field]
|
|
27
|
+
raise ArgumentError, "space #{space}: index #{index.name}: part #{part.field} is beyond the format" unless field
|
|
28
|
+
|
|
29
|
+
part.path ? path_part(part, field) : plain_part(part, field)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def plain_part(part, field)
|
|
33
|
+
differing = { type: part.type, is_nullable: part.is_nullable }.reject { |key, value| field.send(key) == value }
|
|
34
|
+
|
|
35
|
+
differing.empty? ? part.field : { field: part.field, **differing }
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# A path part always declares its type; nullability still inherits from the field.
|
|
39
|
+
def path_part(part, field)
|
|
40
|
+
nullable = part.is_nullable == field.is_nullable ? {} : { is_nullable: part.is_nullable }
|
|
41
|
+
|
|
42
|
+
{ field: part.field, type: part.type, **nullable, path: part.path }
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
end
|
|
@@ -23,10 +23,10 @@ module Tarantool
|
|
|
23
23
|
def fields = format.to_h { |field| [field.name, field] }
|
|
24
24
|
end
|
|
25
25
|
|
|
26
|
-
LiveSpace::Index = Data.define(:id, :name, :type, :unique, :parts, :sequence) do
|
|
26
|
+
LiveSpace::Index = Data.define(:id, :name, :type, :unique, :parts, :sequence, :func) do
|
|
27
27
|
def self.from_eval(raw, format)
|
|
28
28
|
new(id: raw.fetch('id'), name: raw.fetch('name').to_sym, type: raw.fetch('type').downcase,
|
|
29
|
-
unique: raw.fetch('unique'), sequence: raw.fetch('sequence'),
|
|
29
|
+
unique: raw.fetch('unique'), sequence: raw.fetch('sequence'), func: raw.fetch('func', false),
|
|
30
30
|
parts: Array(raw['parts']).map { |part| part_from_eval(part, format) })
|
|
31
31
|
end
|
|
32
32
|
|
|
@@ -5,7 +5,8 @@ module Tarantool
|
|
|
5
5
|
# The live state of everything a definition declares, compared with it, and what is left to create.
|
|
6
6
|
# @api private
|
|
7
7
|
class Plan
|
|
8
|
-
def initialize(spaces:, external:, client:)
|
|
8
|
+
def initialize(spaces:, external:, client:, alter: [])
|
|
9
|
+
@alter = alter
|
|
9
10
|
@comparisons = spaces.values.map { |space| compare(client, space.name, space.indexes, space) } +
|
|
10
11
|
external.map { |name, indexes| compare(client, name, indexes) }
|
|
11
12
|
end
|
|
@@ -21,24 +22,33 @@ module Tarantool
|
|
|
21
22
|
self
|
|
22
23
|
end
|
|
23
24
|
|
|
25
|
+
# Per space: the space itself or its format change, then its indexes, so an index on a new field works.
|
|
24
26
|
def creations
|
|
25
27
|
@comparisons.flat_map do |comparison|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
space + comparison.missing_indexes.map { |index| Creation::Index.new(space: comparison.name, spec: index) }
|
|
28
|
+
space_creation(comparison) +
|
|
29
|
+
comparison.missing_indexes.map { |index| Creation::Index.new(space: comparison.name, spec: index) }
|
|
29
30
|
end
|
|
30
31
|
end
|
|
31
32
|
|
|
32
33
|
private
|
|
33
34
|
|
|
35
|
+
def space_creation(comparison)
|
|
36
|
+
return [Creation::Space.new(spec: comparison.space)] if comparison.missing_space?
|
|
37
|
+
return [] if comparison.format_additions.empty?
|
|
38
|
+
|
|
39
|
+
[Creation::Format.new(space: comparison.space, fields: comparison.format_additions)]
|
|
40
|
+
end
|
|
41
|
+
|
|
34
42
|
def compare(client, name, indexes, space = nil)
|
|
35
|
-
Comparison.new(name:, indexes:, live: Inspector.call(client, name), space:)
|
|
43
|
+
Comparison.new(name:, indexes:, live: Inspector.call(client, name), space:, alter: @alter)
|
|
36
44
|
end
|
|
37
45
|
|
|
38
46
|
def still_missing(comparison)
|
|
39
47
|
return ['missing after apply'] if comparison.missing_space?
|
|
40
48
|
|
|
41
|
-
comparison.
|
|
49
|
+
format = comparison.format_additions.empty? ? [] : ['format: fields missing after apply']
|
|
50
|
+
|
|
51
|
+
format + comparison.missing_indexes.map { |index| "index #{index.name}: missing after apply" }
|
|
42
52
|
end
|
|
43
53
|
end
|
|
44
54
|
end
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Tarantool
|
|
4
|
+
module Schema
|
|
5
|
+
# Ruby source of a definition, from its normalized Hash: a Definition.new block with one ensure_space per
|
|
6
|
+
# space and one ensure_index per index on a space declared elsewhere. Defaults are left out and a part that
|
|
7
|
+
# inherits everything from its field is a bare name, so the result reads like a hand-written declaration.
|
|
8
|
+
# @api private
|
|
9
|
+
module Source
|
|
10
|
+
WIDTH = 120
|
|
11
|
+
ARGUMENT_INDENT = ' ' * 15
|
|
12
|
+
ENTRY_INDENT = ' ' * 17
|
|
13
|
+
|
|
14
|
+
module_function
|
|
15
|
+
|
|
16
|
+
def render(spaces, external)
|
|
17
|
+
entries = spaces.map { |name, space| space(name, space) }
|
|
18
|
+
entries += external.map { |name, indexes| indexes(name, indexes) }
|
|
19
|
+
|
|
20
|
+
"Tarantool::Schema::Definition.new do\n#{entries.join("\n\n")}#{"\n" if entries.any?}end\n"
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def space(name, space)
|
|
24
|
+
fields = space[:format].to_h { |field| [field[:name], field] }
|
|
25
|
+
format = list('format', space[:format].map { |field| field(field) })
|
|
26
|
+
indexes = list('indexes', space[:indexes].map { |index| index(index, fields) })
|
|
27
|
+
|
|
28
|
+
" ensure_space #{symbol(name)},\n#{engine(space)}#{format},\n#{indexes}"
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def engine(space) = space[:engine] == 'memtx' ? '' : "#{ARGUMENT_INDENT}engine: #{symbol(space[:engine])},\n"
|
|
32
|
+
|
|
33
|
+
def indexes(space, indexes)
|
|
34
|
+
indexes.map { |index| " ensure_index #{symbol(space)}, #{symbol(index[:name])}, #{options(index, {})}" }
|
|
35
|
+
.join("\n")
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def list(key, entries)
|
|
39
|
+
"#{ARGUMENT_INDENT}#{key}: [\n#{entries.map { |entry| ENTRY_INDENT + entry }.join(",\n")}\n#{ARGUMENT_INDENT}]"
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def field(field)
|
|
43
|
+
nullable = field[:is_nullable] ? ', is_nullable: true' : ''
|
|
44
|
+
|
|
45
|
+
"{ name: #{symbol(field[:name])}, type: #{symbol(field[:type])}#{nullable} }"
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def index(index, fields)
|
|
49
|
+
line = "{ name: #{symbol(index[:name])}, #{options(index, fields)} }"
|
|
50
|
+
return line if ENTRY_INDENT.size + line.size <= WIDTH
|
|
51
|
+
|
|
52
|
+
wrapped_index(index, fields)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def wrapped_index(index, fields)
|
|
56
|
+
indent = "#{ENTRY_INDENT} "
|
|
57
|
+
parts = index[:parts].map { |part| "#{indent} #{part(part, fields)}" }.join(",\n")
|
|
58
|
+
head = "{ name: #{symbol(index[:name])},#{type_option(index).rstrip}"
|
|
59
|
+
tail = [unique_option(index), sequence_option(index)].join
|
|
60
|
+
|
|
61
|
+
"#{head}\n#{indent}parts: [\n#{parts}\n#{indent}]#{tail} }"
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def options(index, fields)
|
|
65
|
+
parts = index[:parts].map { |part| part(part, fields) }.join(', ')
|
|
66
|
+
|
|
67
|
+
"#{type_option(index).lstrip}parts: [#{parts}]#{unique_option(index)}#{sequence_option(index)}"
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def type_option(index) = index[:type] == 'tree' ? ' ' : " type: #{symbol(index[:type])}, "
|
|
71
|
+
|
|
72
|
+
def unique_option(index) = index[:unique] ? '' : ', unique: false'
|
|
73
|
+
|
|
74
|
+
def sequence_option(index) = index[:sequence] ? ', sequence: true' : ''
|
|
75
|
+
|
|
76
|
+
# Unresolved parts (external indexes) carry nil for what they inherit; resolved ones repeat the field.
|
|
77
|
+
def part(part, fields)
|
|
78
|
+
options = part_options(part, fields[part[:field]])
|
|
79
|
+
return symbol(part[:field]) if options.empty? && part[:path].nil?
|
|
80
|
+
|
|
81
|
+
"{ field: #{symbol(part[:field])}, #{options.map { |key, value| "#{key}: #{value}" }.join(', ')} }"
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# A path part always declares its type; everything else is written only when it differs from the field.
|
|
85
|
+
def part_options(part, field)
|
|
86
|
+
options = differing(part, field)
|
|
87
|
+
options[:type] = part[:type] if part[:path]
|
|
88
|
+
options = options.sort_by { |key, _| key == :type ? 0 : 1 }.to_h { |key, value| [key, literal(value)] }
|
|
89
|
+
options[:path] = string(part[:path]) if part[:path]
|
|
90
|
+
|
|
91
|
+
options
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def differing(part, field)
|
|
95
|
+
options = { type: part[:type], is_nullable: part[:is_nullable] }.compact
|
|
96
|
+
return options unless field
|
|
97
|
+
|
|
98
|
+
options.reject { |key, value| field[key] == value }
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
def literal(value) = value.is_a?(String) ? symbol(value) : value.to_s
|
|
102
|
+
|
|
103
|
+
def symbol(name) = name.to_sym.inspect
|
|
104
|
+
|
|
105
|
+
def string(value) = value.match?(/['\\]/) ? value.inspect : "'#{value}'"
|
|
106
|
+
end
|
|
107
|
+
end
|
|
108
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Tarantool
|
|
4
|
+
module Schema
|
|
5
|
+
# Names of the spaces an application owns: ids past the system range, names without a leading underscore
|
|
6
|
+
# (vshard's _bucket, crud's _ddl_* stay out). An explicit list skips the server.
|
|
7
|
+
# @api private
|
|
8
|
+
module SpaceLister
|
|
9
|
+
LUA = <<~LUA
|
|
10
|
+
local names = {}
|
|
11
|
+
for _, space in box.space._space:pairs(box.schema.SYSTEM_ID_MAX + 1, {iterator = 'GE'}) do
|
|
12
|
+
if not space.name:match('^_') then table.insert(names, space.name) end
|
|
13
|
+
end
|
|
14
|
+
return names
|
|
15
|
+
LUA
|
|
16
|
+
|
|
17
|
+
def self.call(client, only: nil)
|
|
18
|
+
return only.map(&:to_sym) if only
|
|
19
|
+
|
|
20
|
+
Array(client.eval(LUA).first).map(&:to_sym)
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
data/lib/tarantool/schema.rb
CHANGED
|
@@ -5,6 +5,7 @@ require 'tarantool/client'
|
|
|
5
5
|
require_relative 'schema/version'
|
|
6
6
|
require_relative 'schema/errors'
|
|
7
7
|
require_relative 'schema/options'
|
|
8
|
+
require_relative 'schema/alter'
|
|
8
9
|
require_relative 'schema/types'
|
|
9
10
|
require_relative 'schema/key_rules'
|
|
10
11
|
require_relative 'schema/field_spec'
|
|
@@ -16,6 +17,9 @@ require_relative 'schema/plan'
|
|
|
16
17
|
require_relative 'schema/definition'
|
|
17
18
|
require_relative 'schema/live_space'
|
|
18
19
|
require_relative 'schema/inspector'
|
|
20
|
+
require_relative 'schema/space_lister'
|
|
21
|
+
require_relative 'schema/live_definition'
|
|
22
|
+
require_relative 'schema/source'
|
|
19
23
|
require_relative 'schema/format_comparison'
|
|
20
24
|
require_relative 'schema/index_comparison'
|
|
21
25
|
require_relative 'schema/comparison'
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: tarantool-schema
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Starlexxx
|
|
@@ -37,6 +37,7 @@ files:
|
|
|
37
37
|
- LICENSE
|
|
38
38
|
- README.md
|
|
39
39
|
- lib/tarantool/schema.rb
|
|
40
|
+
- lib/tarantool/schema/alter.rb
|
|
40
41
|
- lib/tarantool/schema/comparison.rb
|
|
41
42
|
- lib/tarantool/schema/creation.rb
|
|
42
43
|
- lib/tarantool/schema/definition.rb
|
|
@@ -47,10 +48,13 @@ files:
|
|
|
47
48
|
- lib/tarantool/schema/index_spec.rb
|
|
48
49
|
- lib/tarantool/schema/inspector.rb
|
|
49
50
|
- lib/tarantool/schema/key_rules.rb
|
|
51
|
+
- lib/tarantool/schema/live_definition.rb
|
|
50
52
|
- lib/tarantool/schema/live_space.rb
|
|
51
53
|
- lib/tarantool/schema/options.rb
|
|
52
54
|
- lib/tarantool/schema/part_spec.rb
|
|
53
55
|
- lib/tarantool/schema/plan.rb
|
|
56
|
+
- lib/tarantool/schema/source.rb
|
|
57
|
+
- lib/tarantool/schema/space_lister.rb
|
|
54
58
|
- lib/tarantool/schema/space_rules.rb
|
|
55
59
|
- lib/tarantool/schema/space_spec.rb
|
|
56
60
|
- lib/tarantool/schema/types.rb
|