neo4j_bolt 0.4.5 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cc5ccd396277de9c75f73bd338f89a2af476475940d53e0399cfbbeaa9589dc6
4
- data.tar.gz: 90a393bb3f4819245613ca34206d24932487de5af4959b27e26232137766e15f
3
+ metadata.gz: 865849878d27a8936cb83b5d6567b2ef5d4256f422dc716cc0d496fcdfa31657
4
+ data.tar.gz: e3d4d1b1a57214d0ef7f99c75cd6a76c9a0c2fbb187fb02bb815b821f5dbc4b2
5
5
  SHA512:
6
- metadata.gz: e4f8c0aa87b72f4081d19476dbb684ced5fed6851b729f7db4bb7d8b06f7e41f1607581498b1195043edc63433fe91b8ca7aadf0c58c3d8c7f14bf79dcae1395
7
- data.tar.gz: '028fc3fc677e0a8d83f6f5554d851ff29b1ab7bf6b0c059ea646f18062f646e4c43554f612bb6c69055658af56e4f08bda30c5756b57899475b98178b91d6e87'
6
+ metadata.gz: 36b289414eaed0af1eb8b628863e7d2f8cd88155377c0104bff8dc4edfda3e12bdf6157ed61e832be04856bb7cffe37d8be96a5d7c6ae5dac01d50a4fb6a75c5
7
+ data.tar.gz: 2fe7a4858a2c03f1b96e9260d93d85e77bdfef60f05fe0c1a2793b7c125e3e0d3db35902ec07ba6fc9cec4ce15cc02d2b8bb7cfad75c7d73503bf612a890ed38
data/README.md CHANGED
@@ -4,14 +4,14 @@ Neo4jBolt 0.4 is a small compatibility and convenience layer for Ruby applicatio
4
4
 
5
5
  Neo4jBolt no longer implements the Bolt wire protocol itself.
6
6
 
7
- `0.4.5` requires Ruby 3.4 or newer and pins `neo4j-ruby-driver` to `6.2.1.beta.4`. Applications on older Rubies can remain on the Neo4jBolt 0.3.x line; this prerelease is intentionally not an automatic upgrade for them.
7
+ `0.5.1` requires Ruby 3.4 or newer and pins `neo4j-ruby-driver` to `6.2.1.beta.4`. Applications on older Rubies can remain on the Neo4jBolt 0.3.x line; this prerelease is intentionally not an automatic upgrade for them.
8
8
 
9
9
  ## Installation
10
10
 
11
11
  For this prerelease, specify the version explicitly:
12
12
 
13
13
  ```ruby
14
- gem "neo4j_bolt", "0.4.5"
14
+ gem "neo4j_bolt", "0.5.1"
15
15
  ```
16
16
 
17
17
  Then run `bundle install`. A running Neo4j database is required.
@@ -163,14 +163,31 @@ Managed entries retain the `neo4j_bolt_` prefix. Setup removes obsolete entries
163
163
 
164
164
  ## Dump and load
165
165
 
166
- The textual format is unchanged:
166
+ The textual format remains line-oriented and backwards compatible. Dumps now include supported schema entries by default:
167
167
 
168
168
  ```text
169
- n {"id":0,"labels":["Person"],"properties":{"name":"Ada"}}
170
- n {"id":1,"labels":["Person"],"properties":{"name":"Grace"}}
169
+ u {"name":"person_email","entity_type":"NODE","label_or_type":"Person","properties":["email"]}
170
+ k {"name":"person_id","entity_type":"NODE","label_or_type":"Person","properties":["id"]}
171
+ i {"name":"person_name","entity_type":"NODE","label_or_type":"Person","properties":["name"]}
172
+ n {"id":0,"labels":["Person"],"properties":{"id":1,"name":"Ada","email":"ada@example.test"}}
173
+ n {"id":1,"labels":["Person"],"properties":{"id":2,"name":"Grace","email":"grace@example.test"}}
171
174
  r {"from":0,"to":1,"type":"KNOWS","properties":{"since":2024}}
172
175
  ```
173
176
 
177
+ The record prefixes are:
178
+
179
+ - `u` — property uniqueness constraint
180
+ - `k` — node or relationship key constraint
181
+ - `i` — ordinary property index
182
+ - `n` — node
183
+ - `r` — relationship
184
+
185
+ Schema records preserve the schema entry name, whether it belongs to nodes or relationships, the label or relationship type, and the indexed/constrained properties. Constraint-backing indexes are not written as separate `i` records. Ordinary Neo4j 4.4 `BTREE` indexes and modern `RANGE` indexes are treated as the same portable property-index concept, so a logical dump can move between supported Neo4j generations. Neo4j's token lookup indexes are intentionally omitted because they are database infrastructure rather than application property indexes.
186
+
187
+ Other constraint or index types are not silently dropped. A schema dump fails with a clear error instead; use data-only dumping if the application intentionally needs only nodes and relationships.
188
+
189
+ Key constraints are dumped when they exist. Loading a dump containing a `k` record requires a Neo4j edition/version that can create that key; otherwise the load fails instead of silently weakening the schema.
190
+
174
191
  ```ruby
175
192
  File.open("database.dump", "w") { |io| dump_database(io, progress_io: $stderr) }
176
193
  File.open("database.dump", "r") do |io|
@@ -178,6 +195,13 @@ File.open("database.dump", "r") do |io|
178
195
  end
179
196
  ```
180
197
 
198
+ Schema dumping and restoration can be disabled explicitly:
199
+
200
+ ```ruby
201
+ File.open("data.dump", "w") { |io| dump_database(io, schema: false) }
202
+ File.open("database.dump", "r") { |io| load_database_dump(io, schema: false) }
203
+ ```
204
+
181
205
  Passing `progress_io:` is optional for the Ruby API. The CLI always reports dump/load progress on stderr, so dump data written to stdout remains safe to redirect or pipe. In `auto` mode a terminal gets a colored, in-place progress bar while redirected stderr receives periodic plain-text progress lines. Set `NEO4J_BOLT_PROGRESS=pretty` to force the terminal display through a container or other wrapper that hides the TTY, or `NEO4J_BOLT_PROGRESS=plain` to force log-friendly output. `NO_COLOR` disables colors without disabling the in-place display.
182
206
 
183
207
  Both `dump` and `load` accept `--color COLOR` to choose the spinner and filled progress-bar accent. The default is `cyan`. The Ruby API exposes the same setting as `progress_color:`. Available colors are `red`, `green`, `yellow`, `blue`, `magenta`, `cyan`, `white`, `bright-red`, `bright-green`, `bright-yellow`, `bright-blue`, `bright-magenta`, `bright-cyan`, and `bright-white`.
@@ -187,13 +211,26 @@ neo4j_bolt dump --color magenta -o database.dump
187
211
  neo4j_bolt load --color bright-blue database.dump
188
212
  ```
189
213
 
190
- The `0, 1, 2, ...` IDs are synthetic dump-local IDs. Database-internal IDs and element IDs are never written to the persistent format. Dumping keeps the count, node, and relationship reads in one transaction, so the driver-provided entity identity used to connect the two streamed result sets stays within Neo4j's transaction-scoped identity guarantee. To keep dump numbering deterministic, ordering uses `elementId()` on modern Neo4j and an internal `id()` fallback only on Neo4j 4.4, where `elementId()` does not exist. Old Neo4jBolt dumps remain loadable.
214
+ Both commands also accept `--data-only`. On `dump`, it omits `u`, `k`, and `i` records. On `load`, it accepts a full dump but ignores its schema records:
215
+
216
+ ```bash
217
+ neo4j_bolt dump --data-only -o data.dump
218
+ neo4j_bolt load --data-only database.dump
219
+ ```
220
+
221
+ Property values that JSON cannot express keep their type: dates, datetimes, local datetimes, local times, times, durations and points are written as maps tagged with `"$neo4j"`, for example `{"$neo4j":"date","value":"1815-12-10"}`, carrying the driver's own fields so that loading restores the exact value, and a readable `value` where there is one. Neo4j properties are never maps, so a tagged map cannot clash with an ordinary value; older dumps, which wrote such values as strings, still load as before. A datetime with a named zone (`[Europe/Berlin]`) comes back with its offset only, as the driver reads it that way.
222
+
223
+ The `0, 1, 2, ...` IDs are synthetic dump-local IDs. Database-internal IDs and element IDs are never written to the persistent format. Dumping keeps the count, node, and relationship reads in one transaction, so the driver-provided entity identity used to connect the two streamed result sets stays within Neo4j's transaction-scoped identity guarantee. To keep dump numbering deterministic, ordering uses `elementId()` on modern Neo4j and an internal `id()` fallback only on Neo4j 4.4, where `elementId()` does not exist. Old Neo4jBolt dumps containing only `n` and `r` records remain loadable.
191
224
 
192
225
  For relational loads, the adapter assigns a random temporary label and the fixed reserved property `__neo4j_bolt_load_id` to imported nodes. After the nodes are committed it builds a temporary `neo4j_bolt_` index on that property, uses indexed lookups while creating relationships, drops the index, and removes the temporary metadata in batches. The fixed property key is reused by every load instead of registering a new random property-key token each time. Neo4j can keep the property key registered after all values have been removed, so it may remain visible in Browser's property-key list; no imported node retains the property after successful cleanup. Relational loads refuse to run if the dump itself, or existing data during `force_append`, currently uses the reserved property. No database-internal identity is carried from one load transaction to another. Loading a relational dump therefore requires permission to create and drop an index.
193
226
 
194
227
  Loads start with batches of 5,000 records by default. This is an initial ceiling rather than a claimed optimum: if Neo4j returns a transaction-memory/resource error whose server semantics guarantee rollback, the loader halves the failed batch and retries it, then keeps the smaller size for the rest of that phase. Node and relationship phases adapt independently. Other errors are not retried, because a generic connection failure cannot safely prove that a `CREATE` transaction did not commit. Callers can change the initial ceiling with `initial_batch_size:`; the CLI exposes the same setting as `--batch-size`.
195
228
 
196
- Loading requires an empty database unless `force_append: true` is passed. `load_database_dump` owns its batch transactions and therefore rejects being called from inside `transaction`.
229
+ Loading data requires an empty database unless `force_append: true` is passed. For a dump that contains `u`, `k`, or `i` records and is being loaded with schema enabled, the target must also have no application constraints or property indexes. Neo4j's built-in token lookup indexes do not count as application schema. Old `n`/`r`-only dumps, and loads using `schema: false` / `--data-only`, keep the old data-only emptiness rule and may load into a data-empty database that already has application schema.
230
+
231
+ With a normal full restore, the schema preflight happens before any data is written and the saved schema is created strictly after the data load. With `force_append: true`, existing schema is allowed and restored schema entries use `IF NOT EXISTS`. Existing constraints and indexes are never removed by dump loading.
232
+
233
+ Data is loaded before schema records are restored, so bulk loading does not pay constraint/index maintenance costs for every inserted record. `load_database_dump` owns its batch transactions and therefore rejects being called from inside `transaction`.
197
234
 
198
235
  ## CLI
199
236
 
@@ -203,8 +240,8 @@ The `neo4j_bolt` executable retains these commands:
203
240
  | --- | --- |
204
241
  | `neo4j_bolt console` | Open an IRB console with Neo4jBolt loaded |
205
242
  | `neo4j_bolt clear --srsly` | Delete all nodes and relationships |
206
- | `neo4j_bolt dump [--color COLOR]` | Write the textual database dump |
207
- | `neo4j_bolt load [--force] [--batch-size N] [--color COLOR] PATH` | Load a textual dump |
243
+ | `neo4j_bolt dump [--data-only] [--color COLOR]` | Write the textual database dump |
244
+ | `neo4j_bolt load [--force] [--data-only] [--batch-size N] [--color COLOR] PATH` | Load a textual dump |
208
245
  | `neo4j_bolt index ls` | List constraints and indexes |
209
246
  | `neo4j_bolt index rm --force` | Remove all constraints and indexes |
210
247
  | `neo4j_bolt visualize` | Generate a GraphViz document |
@@ -213,13 +250,18 @@ Use `--host HOST:PORT`, `--username USER`, `--password PASSWORD`, and `--databas
213
250
 
214
251
  ## Tested Neo4j versions
215
252
 
216
- The same complete integration suite is run against these exact Community images:
253
+ The complete integration suite is run against both Community and Enterprise for each pinned Neo4j generation:
217
254
 
218
255
  - `neo4j:4.4.48-community`
256
+ - `neo4j:4.4.48-enterprise`
219
257
  - `neo4j:5.26.28-community`
258
+ - `neo4j:5.26.28-enterprise`
220
259
  - `neo4j:2026.06.0-community`
260
+ - `neo4j:2026.06.0-enterprise`
261
+
262
+ No compatibility beyond this matrix is claimed for `0.5.1`.
221
263
 
222
- No compatibility beyond this matrix is claimed for `0.4.5`.
264
+ Enterprise test containers are started with `NEO4J_ACCEPT_LICENSE_AGREEMENT=yes`. The Community targets verify that Community-compatible features remain portable; the Enterprise targets additionally exercise successful node-key dump/load instead of merely observing that Community rejects node keys.
223
265
 
224
266
  Run one modern LTS target:
225
267
 
@@ -257,7 +299,11 @@ To point RSpec itself at an already disposable database, set `NEO4J_BOLT_TEST_HO
257
299
  | CLI commands | Preserved |
258
300
  | `BoltSocket`, `BoltBuffer`, protocol markers/state/parser/packer | Intentionally removed private implementation details |
259
301
 
260
- ## 0.4.5 migration notes
302
+ ## 0.5.1
303
+
304
+ - Dumps keep dates, datetimes, local datetimes, local times, times, durations and points; before, they were written as text and came back as strings. Older dumps load as before.
305
+
306
+ ## 0.5.0 migration notes
261
307
 
262
308
  - Ruby 3.4 or newer is required; Ruby 2.x/3.0–3.3 applications should stay on 0.3.x until upgraded.
263
309
  - The exact prerelease upstream dependency is pinned while no stable `neo4j-ruby-driver` 6.2.x exists.
data/bin/neo4j_bolt CHANGED
@@ -53,13 +53,20 @@ class App
53
53
  # --------------------------------------------
54
54
 
55
55
  desc 'Dump database'
56
- long_desc 'Dump all nodes and relationships.'
56
+ long_desc 'Dump nodes, relationships, constraints, and property indexes.'
57
57
  command :dump do |c|
58
58
  c.flag [:o, 'out-file'.to_sym], :default_value => '/dev/stdout'
59
59
  c.flag [:color], :default_value => 'cyan', :desc => 'progress accent color'
60
+ c.switch ['data-only'.to_sym], :default_value => false, :negatable => false,
61
+ :desc => 'dump nodes and relationships only'
60
62
  c.action do |global_options, options|
61
63
  File.open(options['out-file'.to_sym], 'w') do |f|
62
- dump_database(f, progress_io: $stderr, progress_color: options[:color])
64
+ dump_database(
65
+ f,
66
+ progress_io: $stderr,
67
+ progress_color: options[:color],
68
+ schema: !options['data-only'.to_sym]
69
+ )
63
70
  end
64
71
  end
65
72
  end
@@ -67,13 +74,16 @@ class App
67
74
  # --------------------------------------------
68
75
 
69
76
  desc 'Load database dump'
70
- long_desc 'Load nodes and relationships from a database dump.'
77
+ long_desc 'Load nodes, relationships, constraints, and property indexes from a database dump.'
71
78
  command :load do |c|
72
79
  # c.flag [:i, :in_file], :desc => 'input path', :required => true
73
- c.switch [:f, :force], :default_value => false, :desc => 'force appending nodes even if the database is not empty'
80
+ c.switch [:f, :force], :default_value => false,
81
+ :desc => 'force loading into a database that already contains data or application schema'
74
82
  c.flag [:b, 'batch-size'.to_sym], :default_value => Neo4jBolt::LOAD_INITIAL_BATCH_SIZE,
75
83
  :desc => 'initial batch size; automatically reduced on transaction-memory errors'
76
84
  c.flag [:color], :default_value => 'cyan', :desc => 'progress accent color'
85
+ c.switch ['data-only'.to_sym], :default_value => false, :negatable => false,
86
+ :desc => 'ignore constraints and indexes stored in the dump'
77
87
  c.action do |global_options, options, args|
78
88
  help_now!('input path is required') if args.empty?
79
89
  path = args.shift
@@ -83,7 +93,8 @@ class App
83
93
  force_append: options[:force],
84
94
  progress_io: $stderr,
85
95
  progress_color: options[:color],
86
- initial_batch_size: options['batch-size'.to_sym]
96
+ initial_batch_size: options['batch-size'.to_sym],
97
+ schema: !options['data-only'.to_sym]
87
98
  )
88
99
  end
89
100
  end
@@ -1,3 +1,3 @@
1
1
  module Neo4jBolt
2
- VERSION = "0.4.5"
2
+ VERSION = "0.5.1"
3
3
  end
data/lib/neo4j_bolt.rb CHANGED
@@ -1,9 +1,11 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "date"
3
4
  require "json"
4
5
  require "securerandom"
5
6
  require "set"
6
7
  require "thread"
8
+ require "time"
7
9
  require "uri"
8
10
  require "neo4j/driver"
9
11
  require "neo4j_bolt/version"
@@ -20,6 +22,10 @@ module Neo4jBolt
20
22
  LOAD_PROGRESS_STEP = 5_000
21
23
  DUMP_PROGRESS_STEP = 1_000
22
24
  LOAD_ID_PROPERTY = "__neo4j_bolt_load_id"
25
+ # Dumps write temporal and spatial property values as maps tagged with
26
+ # this key. Neo4j properties are never maps, so a tagged map cannot be an
27
+ # ordinary value, and dumps without such values read as before.
28
+ DUMP_TYPE_KEY = "$neo4j"
23
29
  private_constant :LOAD_MEMORY_ERROR_CODES, :LOAD_MEMORY_GQL_STATUSES,
24
30
  :LOAD_PROGRESS_STEP, :DUMP_PROGRESS_STEP, :LOAD_ID_PROPERTY
25
31
 
@@ -594,11 +600,13 @@ module Neo4jBolt
594
600
  nil
595
601
  end
596
602
 
597
- def dump_database(io, progress_io: nil, progress_color: "cyan")
603
+ def dump_database(io, progress_io: nil, progress_color: "cyan", schema: true)
598
604
  progress = ProgressReporter.new(progress_io, color: progress_color)
599
605
  dumped_nodes = 0
600
606
  dumped_relationships = 0
601
607
 
608
+ dump_schema(io) if schema
609
+
602
610
  transaction do
603
611
  identity_function = dump_identity_function
604
612
  total_nodes = neo4j_query_expect_one("MATCH (n) RETURN count(n) AS count")["count"]
@@ -616,7 +624,7 @@ module Neo4jBolt
616
624
  "ORDER BY #{identity_function}(n)"
617
625
  ) do |row|
618
626
  dump_ids[row["identity"].to_s] = dumped_nodes if dump_ids
619
- io.puts "n #{JSON.generate(id: dumped_nodes, labels: row["labels"], properties: row["properties"])}"
627
+ io.puts "n #{JSON.generate(id: dumped_nodes, labels: row["labels"], properties: encode_dump_properties(row["properties"]))}"
620
628
  dumped_nodes += 1
621
629
  if (dumped_nodes % DUMP_PROGRESS_STEP).zero?
622
630
  progress.update(progress_message(
@@ -636,7 +644,7 @@ module Neo4jBolt
636
644
  from: dump_ids.fetch(row["from_identity"].to_s),
637
645
  to: dump_ids.fetch(row["to_identity"].to_s),
638
646
  type: row["type"],
639
- properties: row["properties"]
647
+ properties: encode_dump_properties(row["properties"])
640
648
  )}"
641
649
  dumped_relationships += 1
642
650
  if (dumped_relationships % DUMP_PROGRESS_STEP).zero?
@@ -655,7 +663,7 @@ module Neo4jBolt
655
663
  end
656
664
 
657
665
  def load_database_dump(io, force_append: false, progress_io: nil, progress_color: "cyan",
658
- initial_batch_size: LOAD_INITIAL_BATCH_SIZE)
666
+ initial_batch_size: LOAD_INITIAL_BATCH_SIZE, schema: true)
659
667
  raise Error, "load_database_dump cannot run inside a transaction" if transaction_context
660
668
 
661
669
  initial_batch_size = Integer(initial_batch_size)
@@ -669,7 +677,12 @@ module Neo4jBolt
669
677
  progress = ProgressReporter.new(progress_io, color: progress_color)
670
678
  node_batches = Hash.new { |hash, key| hash[key] = [] }
671
679
  relationship_batches = Hash.new { |hash, key| hash[key] = [] }
672
- total_nodes, total_relationships = parse_dump(io, node_batches, relationship_batches, progress)
680
+ total_nodes, total_relationships, schema_entries =
681
+ parse_dump(io, node_batches, relationship_batches, progress)
682
+
683
+ if !force_append && schema && !schema_entries.empty?
684
+ ensure_database_has_no_application_schema!
685
+ end
673
686
 
674
687
  temporary_token = SecureRandom.hex(12)
675
688
  temporary_label = "__neo4j_bolt_load_#{temporary_token}"
@@ -802,6 +815,10 @@ module Neo4jBolt
802
815
  raise cleanup_error if original_error.nil? && cleanup_error
803
816
  end
804
817
 
818
+ if schema && !schema_entries.empty?
819
+ restore_dump_schema(schema_entries, progress, if_not_exists: force_append)
820
+ end
821
+
805
822
  progress.finish("Loaded: #{loaded_nodes} nodes, #{loaded_relationships} relationships")
806
823
  nil
807
824
  ensure
@@ -927,6 +944,157 @@ module Neo4jBolt
927
944
  "`#{identifier.to_s.gsub("`", "``")}`"
928
945
  end
929
946
 
947
+ def dump_schema(io)
948
+ entries = []
949
+ constraint_names = Set.new
950
+
951
+ neo4j_query("SHOW CONSTRAINTS").each do |row|
952
+ kind = dump_constraint_kind(row["type"])
953
+ value = schema_dump_value(row, kind)
954
+ constraint_names << value.fetch(:name)
955
+ entries << [kind, value]
956
+ end
957
+
958
+ neo4j_query("SHOW INDEXES").each do |row|
959
+ next unless dumpable_property_index?(row, constraint_names)
960
+
961
+ entries << ["i", schema_dump_value(row, "i")]
962
+ end
963
+
964
+ kind_order = { "u" => 0, "k" => 1, "i" => 2 }
965
+ entries.sort_by { |kind, value| [kind_order.fetch(kind), value.fetch(:name)] }.each do |kind, value|
966
+ io.puts "#{kind} #{JSON.generate(value)}"
967
+ end
968
+ end
969
+
970
+ def dump_constraint_kind(type)
971
+ case type
972
+ when "UNIQUENESS", "NODE_PROPERTY_UNIQUENESS", "RELATIONSHIP_PROPERTY_UNIQUENESS"
973
+ "u"
974
+ when "NODE_KEY", "RELATIONSHIP_KEY"
975
+ "k"
976
+ else
977
+ raise Error,
978
+ "Cannot dump #{type.inspect} constraint; use schema: false (or --data-only in the CLI)"
979
+ end
980
+ end
981
+
982
+ def dumpable_property_index?(row, constraint_names)
983
+ return false if row["owningConstraint"]
984
+ return false if row["uniqueness"] == "UNIQUE"
985
+ return false if constraint_names.include?(row["name"])
986
+ return false if row["type"] == "LOOKUP"
987
+ return true if %w[BTREE RANGE].include?(row["type"])
988
+
989
+ raise Error,
990
+ "Cannot dump #{row['type'].inspect} index #{row['name'].inspect}; " \
991
+ "use schema: false (or --data-only in the CLI)"
992
+ end
993
+
994
+ def schema_dump_value(row, kind)
995
+ name = row["name"]
996
+ entity_type = row["entityType"]
997
+ labels_or_types = row["labelsOrTypes"]
998
+ properties = row["properties"]
999
+
1000
+ unless name.is_a?(String) && !name.empty? &&
1001
+ %w[NODE RELATIONSHIP].include?(entity_type) &&
1002
+ labels_or_types.is_a?(Array) && labels_or_types.size == 1 &&
1003
+ properties.is_a?(Array) && !properties.empty? &&
1004
+ properties.all? { |property| property.is_a?(String) && !property.empty? }
1005
+ raise Error, "Cannot dump #{kind} schema entry #{name.inspect}: unsupported schema shape"
1006
+ end
1007
+
1008
+ {
1009
+ name: name,
1010
+ entity_type: entity_type,
1011
+ label_or_type: labels_or_types.first,
1012
+ properties: properties
1013
+ }
1014
+ end
1015
+
1016
+ def restore_dump_schema(entries, progress, if_not_exists: false)
1017
+ progress&.note("Restoring schema: #{entries.size} entries...")
1018
+ entries.each do |kind, value|
1019
+ neo4j_query(schema_create_statement(kind, value, if_not_exists: if_not_exists))
1020
+ end
1021
+ end
1022
+
1023
+ def schema_create_statement(kind, value, if_not_exists: false)
1024
+ validate_schema_dump_value!(kind, value)
1025
+
1026
+ name = quote_identifier(value.fetch("name"))
1027
+ if_not_exists_clause = if_not_exists ? " IF NOT EXISTS" : ""
1028
+ entity_type = value.fetch("entity_type")
1029
+ label_or_type = quote_identifier(value.fetch("label_or_type"))
1030
+ properties = value.fetch("properties")
1031
+ variable = entity_type == "NODE" ? "n" : "r"
1032
+ pattern = if entity_type == "NODE"
1033
+ "(#{variable}:#{label_or_type})"
1034
+ else
1035
+ "()-[#{variable}:#{label_or_type}]-()"
1036
+ end
1037
+ property_expressions = properties.map { |property| "#{variable}.#{quote_identifier(property)}" }
1038
+
1039
+ case kind
1040
+ when "u", "k"
1041
+ property_expression = if property_expressions.one?
1042
+ property_expressions.first
1043
+ else
1044
+ "(#{property_expressions.join(', ')})"
1045
+ end
1046
+ constraint_type = if kind == "u"
1047
+ "UNIQUE"
1048
+ elsif entity_type == "NODE"
1049
+ "NODE KEY"
1050
+ else
1051
+ "RELATIONSHIP KEY"
1052
+ end
1053
+ "CREATE CONSTRAINT #{name}#{if_not_exists_clause} FOR #{pattern} " \
1054
+ "REQUIRE #{property_expression} IS #{constraint_type}"
1055
+ when "i"
1056
+ "CREATE INDEX #{name}#{if_not_exists_clause} FOR #{pattern} ON (#{property_expressions.join(', ')})"
1057
+ else
1058
+ raise Error, "Unexpected schema dump kind: #{kind}"
1059
+ end
1060
+ end
1061
+
1062
+ def validate_schema_dump_value!(kind, value)
1063
+ unless %w[u k i].include?(kind) && value.is_a?(Hash)
1064
+ raise Error, "Invalid #{kind} schema dump entry"
1065
+ end
1066
+
1067
+ name = value.fetch("name")
1068
+ entity_type = value.fetch("entity_type")
1069
+ label_or_type = value.fetch("label_or_type")
1070
+ properties = value.fetch("properties")
1071
+
1072
+ return if name.is_a?(String) && !name.empty? &&
1073
+ %w[NODE RELATIONSHIP].include?(entity_type) &&
1074
+ label_or_type.is_a?(String) && !label_or_type.empty? &&
1075
+ properties.is_a?(Array) && !properties.empty? &&
1076
+ properties.all? { |property| property.is_a?(String) && !property.empty? }
1077
+
1078
+ raise Error, "Invalid #{kind} schema dump entry"
1079
+ end
1080
+
1081
+ def ensure_database_has_no_application_schema!
1082
+ constraints = neo4j_query("SHOW CONSTRAINTS").filter_map { |row| row["name"] }
1083
+ indexes = neo4j_query("SHOW INDEXES").filter_map do |row|
1084
+ next if row["type"] == "LOOKUP"
1085
+ next if row["owningConstraint"]
1086
+ next if row["uniqueness"] == "UNIQUE"
1087
+
1088
+ row["name"]
1089
+ end
1090
+ schema_names = constraints + indexes
1091
+ return if schema_names.empty?
1092
+
1093
+ raise Error,
1094
+ "There are constraints or indexes in this database, exiting now: " \
1095
+ "#{schema_names.sort.join(', ')}"
1096
+ end
1097
+
930
1098
  # Neo4j 4.4 has no elementId() function. Its id() fallback is used only for
931
1099
  # deterministic in-process ordering; those database IDs never enter a dump.
932
1100
  def dump_identity_function
@@ -936,9 +1104,77 @@ module Neo4jBolt
936
1104
  version.to_s.split(".").first.to_i >= 5 ? "elementId" : "id"
937
1105
  end
938
1106
 
1107
+ # Property values for a dump: JSON values as they are; dates, times,
1108
+ # durations and points as maps tagged with DUMP_TYPE_KEY that carry the
1109
+ # driver's own fields (so that loading restores the exact value) and, for
1110
+ # people reading the dump, the value as text.
1111
+ def encode_dump_properties(properties)
1112
+ return properties unless properties.is_a?(Hash)
1113
+
1114
+ properties.to_h { |key, value| [key, encode_dump_value(value)] }
1115
+ end
1116
+
1117
+ def encode_dump_value(value)
1118
+ tagged = ->(type, fields) { { DUMP_TYPE_KEY => type }.merge(fields) }
1119
+ case value
1120
+ when Array
1121
+ value.map { |item| encode_dump_value(item) }
1122
+ when Time
1123
+ tagged.call("datetime", "value" => value.iso8601(9))
1124
+ when DateTime
1125
+ tagged.call("datetime", "value" => value.to_time.iso8601(9))
1126
+ when Date
1127
+ tagged.call("date", "value" => value.iso8601)
1128
+ when Neo4j::Driver::Types::LocalDateTime
1129
+ tagged.call("local_datetime", "epoch_seconds" => value.epoch_seconds, "nanoseconds" => value.nanoseconds,
1130
+ "value" => value.to_s)
1131
+ when Neo4j::Driver::Types::LocalTime
1132
+ tagged.call("local_time", "nanoseconds" => value.nanoseconds, "value" => value.to_s)
1133
+ when Neo4j::Driver::Types::OffsetTime
1134
+ tagged.call("time", "nanoseconds" => value.nanoseconds, "tz_offset_seconds" => value.tz_offset_seconds,
1135
+ "value" => value.to_s)
1136
+ when Neo4j::Driver::Types::Duration
1137
+ tagged.call("duration", "months" => value.months, "days" => value.days, "seconds" => value.seconds,
1138
+ "nanoseconds" => value.nanoseconds, "value" => value.to_s)
1139
+ when Neo4j::Driver::Types::Point
1140
+ tagged.call("point", { "srid" => value.srid, "x" => value.x, "y" => value.y, "z" => value.z }.compact)
1141
+ else
1142
+ value
1143
+ end
1144
+ end
1145
+
1146
+ def decode_dump_properties(properties)
1147
+ return properties unless properties.is_a?(Hash)
1148
+
1149
+ properties.to_h { |key, value| [key, decode_dump_value(value)] }
1150
+ end
1151
+
1152
+ def decode_dump_value(value)
1153
+ return value.map { |item| decode_dump_value(item) } if value.is_a?(Array)
1154
+ return value unless value.is_a?(Hash) && value.key?(DUMP_TYPE_KEY)
1155
+
1156
+ case value[DUMP_TYPE_KEY]
1157
+ when "datetime" then Time.iso8601(value.fetch("value"))
1158
+ when "date" then Date.iso8601(value.fetch("value"))
1159
+ when "local_datetime"
1160
+ Neo4j::Driver::Types::LocalDateTime.new(value.fetch("epoch_seconds"), value.fetch("nanoseconds"))
1161
+ when "local_time" then Neo4j::Driver::Types::LocalTime.new(value.fetch("nanoseconds"))
1162
+ when "time" then Neo4j::Driver::Types::OffsetTime.new(value.fetch("nanoseconds"), value.fetch("tz_offset_seconds"))
1163
+ when "duration"
1164
+ Neo4j::Driver::Types::Duration.new(value.fetch("months"), value.fetch("days"), value.fetch("seconds"),
1165
+ value.fetch("nanoseconds"))
1166
+ when "point"
1167
+ Neo4j::Driver::Types::Point.new(srid: value.fetch("srid"), x: value.fetch("x"), y: value.fetch("y"),
1168
+ z: value["z"])
1169
+ else
1170
+ raise Error, "Unknown value type in dump: #{value[DUMP_TYPE_KEY].inspect}"
1171
+ end
1172
+ end
1173
+
939
1174
  def parse_dump(io, node_batches, relationship_batches, progress = nil)
940
1175
  node_count = 0
941
1176
  relationship_count = 0
1177
+ schema_entries = []
942
1178
  progress&.update("Reading dump: 0 nodes, 0 relationships", force: true)
943
1179
 
944
1180
  io.each_line.with_index(1) do |line, line_number|
@@ -946,16 +1182,24 @@ module Neo4jBolt
946
1182
  next if line.empty?
947
1183
 
948
1184
  kind, json = line.split(" ", 2)
949
- raise Error, "Invalid dump entry on line #{line_number}" unless json && %w[n r].include?(kind)
1185
+ unless json && %w[n r u k i].include?(kind)
1186
+ raise Error, "Invalid dump entry on line #{line_number}"
1187
+ end
950
1188
 
951
1189
  value = JSON.parse(json)
952
- if kind == "n"
1190
+ case kind
1191
+ when "n"
1192
+ value["properties"] = decode_dump_properties(value["properties"])
953
1193
  labels = value.fetch("labels")
954
1194
  node_batches[labels.sort] << value
955
1195
  node_count += 1
956
- else
1196
+ when "r"
1197
+ value["properties"] = decode_dump_properties(value["properties"])
957
1198
  relationship_batches[value.fetch("type")] << value
958
1199
  relationship_count += 1
1200
+ else
1201
+ validate_schema_dump_value!(kind, value)
1202
+ schema_entries << [kind, value]
959
1203
  end
960
1204
  if ((node_count + relationship_count) % LOAD_PROGRESS_STEP).zero?
961
1205
  progress&.update("Reading dump: #{node_count} nodes, #{relationship_count} relationships")
@@ -965,7 +1209,7 @@ module Neo4jBolt
965
1209
  end
966
1210
 
967
1211
  progress&.finish("Read dump: #{node_count} nodes, #{relationship_count} relationships")
968
- [node_count, relationship_count]
1212
+ [node_count, relationship_count, schema_entries]
969
1213
  end
970
1214
 
971
1215
  def adaptive_each_slice(items, batch_size, progress, kind)
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: neo4j_bolt
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.5
4
+ version: 0.5.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Specht