neo4j_bolt 0.4.4 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e6322e3f763f052647f81029aaa96e45d6cb6f41a8e8d625c198d025b1408a81
4
- data.tar.gz: ddf55ab34e93f5b65b36273a2c148037f4455e18252aba692296e52598668902
3
+ metadata.gz: f9f38fc0e16014ae13f2f31d2ba77f119185249821da7d4f7858fa0ef3f9010f
4
+ data.tar.gz: 5ab718f1dd9829aa1f118a6614f7d29d184d63eb20b4ce7d3e36fc92fea5f204
5
5
  SHA512:
6
- metadata.gz: bff514f49ee709624ac1b178e2216edd001ad168845c397aff2f9d28a3d3752a4c3fcbf0d7f9ea4bc6c0a1f68ffd6feef8fae560127774c0aa0277a4d6cf6ffd
7
- data.tar.gz: d5ef2dbebc19f6f1538ee461993ec20a9184b734ba4b490ca6a941f4de7b2cb526f22ad59daa9f8578afee00f0be25728c16eed29f6ddfab1a581901b8eabb42
6
+ metadata.gz: df61ebc16c46698ccab04c94e9a8abd6120ef07d899b7e52e8e9a80d13bcc4b2f8d17bddff330f6d10ff794faabd705ebfba3d3531fdad311cb8ea5de02dbc8c
7
+ data.tar.gz: 6522b70e44f279f182c289ae3397dafd11d5dff4e72a3437247a5f25bd66d4185e6696b2aa2c96bce976f518d63a30c98c48ea86b2823d28383c3989df24b7eb
data/README.md CHANGED
@@ -4,14 +4,14 @@ Neo4jBolt 0.4 is a small compatibility and convenience layer for Ruby applicatio
4
4
 
5
5
  Neo4jBolt no longer implements the Bolt wire protocol itself.
6
6
 
7
- `0.4.4` requires Ruby 3.4 or newer and pins `neo4j-ruby-driver` to `6.2.1.beta.4`. Applications on older Rubies can remain on the Neo4jBolt 0.3.x line; this prerelease is intentionally not an automatic upgrade for them.
7
+ `0.5.0` requires Ruby 3.4 or newer and pins `neo4j-ruby-driver` to `6.2.1.beta.4`. Applications on older Rubies can remain on the Neo4jBolt 0.3.x line; this prerelease is intentionally not an automatic upgrade for them.
8
8
 
9
9
  ## Installation
10
10
 
11
11
  For this prerelease, specify the version explicitly:
12
12
 
13
13
  ```ruby
14
- gem "neo4j_bolt", "0.4.4"
14
+ gem "neo4j_bolt", "0.5.0"
15
15
  ```
16
16
 
17
17
  Then run `bundle install`. A running Neo4j database is required.
@@ -26,17 +26,20 @@ Neo4jBolt.bolt_port = 7687
26
26
  Neo4jBolt.bolt_verbosity = 0
27
27
  ```
28
28
 
29
- Authentication and database selection can use the standard Neo4j client environment variables:
29
+ The connection endpoint, authentication, and database selection can use the standard Neo4j client environment variables:
30
30
 
31
31
  ```bash
32
+ export NEO4J_URI=neo4j://localhost:7687
32
33
  export NEO4J_USERNAME=neo4j
33
34
  export NEO4J_PASSWORD=secret
34
35
  export NEO4J_DATABASE=neo4j
35
36
  ```
36
37
 
37
- They are read when `neo4j_bolt` is loaded. Explicit Ruby configuration overrides those defaults:
38
+ They are read when `neo4j_bolt` is loaded. `NEO4J_URI` accepts `neo4j://HOST[:PORT]` and `bolt://HOST[:PORT]` endpoint notation; Neo4jBolt uses the host and port while retaining its existing direct Bolt connection behavior. The default port is 7687. Explicit Ruby configuration overrides those defaults:
38
39
 
39
40
  ```ruby
41
+ Neo4jBolt.bolt_host = "localhost"
42
+ Neo4jBolt.bolt_port = 7687
40
43
  Neo4jBolt.bolt_username = "neo4j"
41
44
  Neo4jBolt.bolt_password = "secret"
42
45
  Neo4jBolt.bolt_database = "movies"
@@ -160,14 +163,31 @@ Managed entries retain the `neo4j_bolt_` prefix. Setup removes obsolete entries
160
163
 
161
164
  ## Dump and load
162
165
 
163
- The textual format is unchanged:
166
+ The textual format remains line-oriented and backwards compatible. Dumps now include supported schema entries by default:
164
167
 
165
168
  ```text
166
- n {"id":0,"labels":["Person"],"properties":{"name":"Ada"}}
167
- n {"id":1,"labels":["Person"],"properties":{"name":"Grace"}}
169
+ u {"name":"person_email","entity_type":"NODE","label_or_type":"Person","properties":["email"]}
170
+ k {"name":"person_id","entity_type":"NODE","label_or_type":"Person","properties":["id"]}
171
+ i {"name":"person_name","entity_type":"NODE","label_or_type":"Person","properties":["name"]}
172
+ n {"id":0,"labels":["Person"],"properties":{"id":1,"name":"Ada","email":"ada@example.test"}}
173
+ n {"id":1,"labels":["Person"],"properties":{"id":2,"name":"Grace","email":"grace@example.test"}}
168
174
  r {"from":0,"to":1,"type":"KNOWS","properties":{"since":2024}}
169
175
  ```
170
176
 
177
+ The record prefixes are:
178
+
179
+ - `u` — property uniqueness constraint
180
+ - `k` — node or relationship key constraint
181
+ - `i` — ordinary property index
182
+ - `n` — node
183
+ - `r` — relationship
184
+
185
+ Schema records preserve the schema entry name, whether it belongs to nodes or relationships, the label or relationship type, and the indexed/constrained properties. Constraint-backing indexes are not written as separate `i` records. Ordinary Neo4j 4.4 `BTREE` indexes and modern `RANGE` indexes are treated as the same portable property-index concept, so a logical dump can move between supported Neo4j generations. Neo4j's token lookup indexes are intentionally omitted because they are database infrastructure rather than application property indexes.
186
+
187
+ Other constraint or index types are not silently dropped. A schema dump fails with a clear error instead; use data-only dumping if the application intentionally needs only nodes and relationships.
188
+
189
+ Key constraints are dumped when they exist. Loading a dump containing a `k` record requires a Neo4j edition/version that can create that key; otherwise the load fails instead of silently weakening the schema.
190
+
171
191
  ```ruby
172
192
  File.open("database.dump", "w") { |io| dump_database(io, progress_io: $stderr) }
173
193
  File.open("database.dump", "r") do |io|
@@ -175,6 +195,13 @@ File.open("database.dump", "r") do |io|
175
195
  end
176
196
  ```
177
197
 
198
+ Schema dumping and restoration can be disabled explicitly:
199
+
200
+ ```ruby
201
+ File.open("data.dump", "w") { |io| dump_database(io, schema: false) }
202
+ File.open("database.dump", "r") { |io| load_database_dump(io, schema: false) }
203
+ ```
204
+
178
205
  Passing `progress_io:` is optional for the Ruby API. The CLI always reports dump/load progress on stderr, so dump data written to stdout remains safe to redirect or pipe. In `auto` mode a terminal gets a colored, in-place progress bar while redirected stderr receives periodic plain-text progress lines. Set `NEO4J_BOLT_PROGRESS=pretty` to force the terminal display through a container or other wrapper that hides the TTY, or `NEO4J_BOLT_PROGRESS=plain` to force log-friendly output. `NO_COLOR` disables colors without disabling the in-place display.
179
206
 
180
207
  Both `dump` and `load` accept `--color COLOR` to choose the spinner and filled progress-bar accent. The default is `cyan`. The Ruby API exposes the same setting as `progress_color:`. Available colors are `red`, `green`, `yellow`, `blue`, `magenta`, `cyan`, `white`, `bright-red`, `bright-green`, `bright-yellow`, `bright-blue`, `bright-magenta`, `bright-cyan`, and `bright-white`.
@@ -184,13 +211,24 @@ neo4j_bolt dump --color magenta -o database.dump
184
211
  neo4j_bolt load --color bright-blue database.dump
185
212
  ```
186
213
 
187
- The `0, 1, 2, ...` IDs are synthetic dump-local IDs. Database-internal IDs and element IDs are never written to the persistent format. Dumping keeps the count, node, and relationship reads in one transaction, so the driver-provided entity identity used to connect the two streamed result sets stays within Neo4j's transaction-scoped identity guarantee. To keep dump numbering deterministic, ordering uses `elementId()` on modern Neo4j and an internal `id()` fallback only on Neo4j 4.4, where `elementId()` does not exist. Old Neo4jBolt dumps remain loadable.
214
+ Both commands also accept `--data-only`. On `dump`, it omits `u`, `k`, and `i` records. On `load`, it accepts a full dump but ignores its schema records:
215
+
216
+ ```bash
217
+ neo4j_bolt dump --data-only -o data.dump
218
+ neo4j_bolt load --data-only database.dump
219
+ ```
188
220
 
189
- For relational loads, the adapter assigns a random temporary label and dump-ID property to imported nodes. After the nodes are committed it builds a temporary `neo4j_bolt_` index on that property, uses indexed lookups while creating relationships, drops the index, and removes the temporary metadata in batches. No database-internal identity is carried from one load transaction to another. Loading a relational dump therefore requires permission to create and drop an index.
221
+ The `0, 1, 2, ...` IDs are synthetic dump-local IDs. Database-internal IDs and element IDs are never written to the persistent format. Dumping keeps the count, node, and relationship reads in one transaction, so the driver-provided entity identity used to connect the two streamed result sets stays within Neo4j's transaction-scoped identity guarantee. To keep dump numbering deterministic, ordering uses `elementId()` on modern Neo4j and an internal `id()` fallback only on Neo4j 4.4, where `elementId()` does not exist. Old Neo4jBolt dumps containing only `n` and `r` records remain loadable.
222
+
223
+ For relational loads, the adapter assigns a random temporary label and the fixed reserved property `__neo4j_bolt_load_id` to imported nodes. After the nodes are committed it builds a temporary `neo4j_bolt_` index on that property, uses indexed lookups while creating relationships, drops the index, and removes the temporary metadata in batches. The fixed property key is reused by every load instead of registering a new random property-key token each time. Neo4j can keep the property key registered after all values have been removed, so it may remain visible in Browser's property-key list; no imported node retains the property after successful cleanup. Relational loads refuse to run if the dump itself, or existing data during `force_append`, currently uses the reserved property. No database-internal identity is carried from one load transaction to another. Loading a relational dump therefore requires permission to create and drop an index.
190
224
 
191
225
  Loads start with batches of 5,000 records by default. This is an initial ceiling rather than a claimed optimum: if Neo4j returns a transaction-memory/resource error whose server semantics guarantee rollback, the loader halves the failed batch and retries it, then keeps the smaller size for the rest of that phase. Node and relationship phases adapt independently. Other errors are not retried, because a generic connection failure cannot safely prove that a `CREATE` transaction did not commit. Callers can change the initial ceiling with `initial_batch_size:`; the CLI exposes the same setting as `--batch-size`.
192
226
 
193
- Loading requires an empty database unless `force_append: true` is passed. `load_database_dump` owns its batch transactions and therefore rejects being called from inside `transaction`.
227
+ Loading data requires an empty database unless `force_append: true` is passed. For a dump that contains `u`, `k`, or `i` records and is being loaded with schema enabled, the target must also have no application constraints or property indexes. Neo4j's built-in token lookup indexes do not count as application schema. Old `n`/`r`-only dumps, and loads using `schema: false` / `--data-only`, keep the old data-only emptiness rule and may load into a data-empty database that already has application schema.
228
+
229
+ With a normal full restore, the schema preflight happens before any data is written and the saved schema is created strictly after the data load. With `force_append: true`, existing schema is allowed and restored schema entries use `IF NOT EXISTS`. Existing constraints and indexes are never removed by dump loading.
230
+
231
+ Data is loaded before schema records are restored, so bulk loading does not pay constraint/index maintenance costs for every inserted record. `load_database_dump` owns its batch transactions and therefore rejects being called from inside `transaction`.
194
232
 
195
233
  ## CLI
196
234
 
@@ -200,23 +238,28 @@ The `neo4j_bolt` executable retains these commands:
200
238
  | --- | --- |
201
239
  | `neo4j_bolt console` | Open an IRB console with Neo4jBolt loaded |
202
240
  | `neo4j_bolt clear --srsly` | Delete all nodes and relationships |
203
- | `neo4j_bolt dump [--color COLOR]` | Write the textual database dump |
204
- | `neo4j_bolt load [--force] [--batch-size N] [--color COLOR] PATH` | Load a textual dump |
241
+ | `neo4j_bolt dump [--data-only] [--color COLOR]` | Write the textual database dump |
242
+ | `neo4j_bolt load [--force] [--data-only] [--batch-size N] [--color COLOR] PATH` | Load a textual dump |
205
243
  | `neo4j_bolt index ls` | List constraints and indexes |
206
244
  | `neo4j_bolt index rm --force` | Remove all constraints and indexes |
207
245
  | `neo4j_bolt visualize` | Generate a GraphViz document |
208
246
 
209
- Use `--host HOST:PORT`, `--username USER`, `--password PASSWORD`, and `--database DATABASE` to override connection settings. Username, password, and database otherwise inherit `NEO4J_USERNAME`, `NEO4J_PASSWORD`, and `NEO4J_DATABASE`. Prefer the password environment variable for routine use because command-line arguments may be visible in shell history or process listings. `gli` remains a runtime dependency for the CLI, and `pry` remains for the separate `bin/console` executable.
247
+ Use `--host HOST:PORT`, `--username USER`, `--password PASSWORD`, and `--database DATABASE` to override connection settings. Host and port otherwise inherit `NEO4J_URI` when present, falling back to `localhost:7687`; username, password, and database inherit `NEO4J_USERNAME`, `NEO4J_PASSWORD`, and `NEO4J_DATABASE`. Prefer the password environment variable for routine use because command-line arguments may be visible in shell history or process listings. `gli` remains a runtime dependency for the CLI, and `pry` remains for the separate `bin/console` executable.
210
248
 
211
249
  ## Tested Neo4j versions
212
250
 
213
- The same complete integration suite is run against these exact Community images:
251
+ The complete integration suite is run against both Community and Enterprise for each pinned Neo4j generation:
214
252
 
215
253
  - `neo4j:4.4.48-community`
254
+ - `neo4j:4.4.48-enterprise`
216
255
  - `neo4j:5.26.28-community`
256
+ - `neo4j:5.26.28-enterprise`
217
257
  - `neo4j:2026.06.0-community`
258
+ - `neo4j:2026.06.0-enterprise`
259
+
260
+ No compatibility beyond this matrix is claimed for `0.5.0`.
218
261
 
219
- No compatibility beyond this matrix is claimed for `0.4.4`.
262
+ Enterprise test containers are started with `NEO4J_ACCEPT_LICENSE_AGREEMENT=yes`. The Community targets verify that Community-compatible features remain portable; the Enterprise targets additionally exercise successful node-key dump/load instead of merely observing that Community rejects node keys.
220
263
 
221
264
  Run one modern LTS target:
222
265
 
@@ -254,13 +297,15 @@ To point RSpec itself at an already disposable database, set `NEO4J_BOLT_TEST_HO
254
297
  | CLI commands | Preserved |
255
298
  | `BoltSocket`, `BoltBuffer`, protocol markers/state/parser/packer | Intentionally removed private implementation details |
256
299
 
257
- ## 0.4.4 migration notes
300
+ ## 0.5.0 migration notes
258
301
 
259
302
  - Ruby 3.4 or newer is required; Ruby 2.x/3.0–3.3 applications should stay on 0.3.x until upgraded.
260
303
  - The exact prerelease upstream dependency is pinned while no stable `neo4j-ruby-driver` 6.2.x exists.
261
304
  - Connections are pooled and safe for concurrent use instead of one mutable socket per including object.
262
- - Basic authentication and database selection are now exposed through `bolt_username`, `bolt_password`, and `bolt_database`, with `NEO4J_USERNAME`, `NEO4J_PASSWORD`, and `NEO4J_DATABASE` as environment defaults.
263
- - TLS, routing, URI schemes, and other upstream-driver configuration are still not exposed through the compatibility API.
305
+ - `NEO4J_URI` now supplies the default host and port for `bolt://` and `neo4j://` endpoint notation; an explicit CLI `--host` or Ruby `bolt_host`/`bolt_port` setting overrides it.
306
+ - Basic authentication and database selection are exposed through `bolt_username`, `bolt_password`, and `bolt_database`, with `NEO4J_USERNAME`, `NEO4J_PASSWORD`, and `NEO4J_DATABASE` as environment defaults.
307
+ - TLS, routing behavior, URI query parameters, and other upstream-driver configuration are still not exposed through the compatibility API.
308
+ - Relational loads reuse one reserved `__neo4j_bolt_load_id` property key while retaining the existing indexed lookup algorithm, preventing a new property-key token from being registered on every load.
264
309
  - `BoltSocket`, `BoltBuffer`, `ServerState`, `BoltMarker`, `UnexpectedServerResponse`, `State`, and `CypherError` were undocumented wire internals and are removed.
265
310
 
266
311
  ## Development
data/bin/neo4j_bolt CHANGED
@@ -21,15 +21,18 @@ class App
21
21
  version Neo4jBolt::VERSION
22
22
 
23
23
  flag [:v, :verbosity], :default_value => 0
24
- flag [:h, :host], :default_value => 'localhost:7687'
24
+ flag [:h, :host], :default_value => nil,
25
+ :desc => 'Neo4j host:port (default: NEO4J_URI or localhost:7687)'
25
26
  flag [:u, :username], :default_value => nil, :desc => 'Neo4j username (default: NEO4J_USERNAME)'
26
27
  flag [:p, :password], :default_value => nil, :desc => 'Neo4j password (default: NEO4J_PASSWORD)'
27
28
  flag [:d, :database], :default_value => nil, :desc => 'Neo4j database (default: NEO4J_DATABASE)'
28
29
 
29
30
  pre do |global_options, command, options, args|
30
31
  host = global_options[:host]
31
- Neo4jBolt.bolt_host = host.split(':').first
32
- Neo4jBolt.bolt_port = host.split(':').last.to_i
32
+ unless host.nil?
33
+ Neo4jBolt.bolt_host = host.split(':').first
34
+ Neo4jBolt.bolt_port = host.split(':').last.to_i
35
+ end
33
36
  Neo4jBolt.bolt_verbosity = global_options[:verbosity].to_i
34
37
  Neo4jBolt.bolt_username = global_options[:username] unless global_options[:username].nil?
35
38
  Neo4jBolt.bolt_password = global_options[:password] unless global_options[:password].nil?
@@ -50,13 +53,20 @@ class App
50
53
  # --------------------------------------------
51
54
 
52
55
  desc 'Dump database'
53
- long_desc 'Dump all nodes and relationships.'
56
+ long_desc 'Dump nodes, relationships, constraints, and property indexes.'
54
57
  command :dump do |c|
55
58
  c.flag [:o, 'out-file'.to_sym], :default_value => '/dev/stdout'
56
59
  c.flag [:color], :default_value => 'cyan', :desc => 'progress accent color'
60
+ c.switch ['data-only'.to_sym], :default_value => false, :negatable => false,
61
+ :desc => 'dump nodes and relationships only'
57
62
  c.action do |global_options, options|
58
63
  File.open(options['out-file'.to_sym], 'w') do |f|
59
- dump_database(f, progress_io: $stderr, progress_color: options[:color])
64
+ dump_database(
65
+ f,
66
+ progress_io: $stderr,
67
+ progress_color: options[:color],
68
+ schema: !options['data-only'.to_sym]
69
+ )
60
70
  end
61
71
  end
62
72
  end
@@ -64,13 +74,16 @@ class App
64
74
  # --------------------------------------------
65
75
 
66
76
  desc 'Load database dump'
67
- long_desc 'Load nodes and relationships from a database dump.'
77
+ long_desc 'Load nodes, relationships, constraints, and property indexes from a database dump.'
68
78
  command :load do |c|
69
79
  # c.flag [:i, :in_file], :desc => 'input path', :required => true
70
- c.switch [:f, :force], :default_value => false, :desc => 'force appending nodes even if the database is not empty'
80
+ c.switch [:f, :force], :default_value => false,
81
+ :desc => 'force loading into a database that already contains data or application schema'
71
82
  c.flag [:b, 'batch-size'.to_sym], :default_value => Neo4jBolt::LOAD_INITIAL_BATCH_SIZE,
72
83
  :desc => 'initial batch size; automatically reduced on transaction-memory errors'
73
84
  c.flag [:color], :default_value => 'cyan', :desc => 'progress accent color'
85
+ c.switch ['data-only'.to_sym], :default_value => false, :negatable => false,
86
+ :desc => 'ignore constraints and indexes stored in the dump'
74
87
  c.action do |global_options, options, args|
75
88
  help_now!('input path is required') if args.empty?
76
89
  path = args.shift
@@ -80,7 +93,8 @@ class App
80
93
  force_append: options[:force],
81
94
  progress_io: $stderr,
82
95
  progress_color: options[:color],
83
- initial_batch_size: options['batch-size'.to_sym]
96
+ initial_batch_size: options['batch-size'.to_sym],
97
+ schema: !options['data-only'.to_sym]
84
98
  )
85
99
  end
86
100
  end
@@ -1,3 +1,3 @@
1
1
  module Neo4jBolt
2
- VERSION = "0.4.4"
2
+ VERSION = "0.5.0"
3
3
  end
data/lib/neo4j_bolt.rb CHANGED
@@ -4,6 +4,7 @@ require "json"
4
4
  require "securerandom"
5
5
  require "set"
6
6
  require "thread"
7
+ require "uri"
7
8
  require "neo4j/driver"
8
9
  require "neo4j_bolt/version"
9
10
 
@@ -18,8 +19,9 @@ module Neo4jBolt
18
19
  LOAD_MEMORY_GQL_STATUSES = %w[51N72 51N73].freeze
19
20
  LOAD_PROGRESS_STEP = 5_000
20
21
  DUMP_PROGRESS_STEP = 1_000
22
+ LOAD_ID_PROPERTY = "__neo4j_bolt_load_id"
21
23
  private_constant :LOAD_MEMORY_ERROR_CODES, :LOAD_MEMORY_GQL_STATUSES,
22
- :LOAD_PROGRESS_STEP, :DUMP_PROGRESS_STEP
24
+ :LOAD_PROGRESS_STEP, :DUMP_PROGRESS_STEP, :LOAD_ID_PROPERTY
23
25
 
24
26
  class ProgressReporter
25
27
  UPDATE_INTERVAL = 0.5
@@ -396,8 +398,29 @@ module Neo4jBolt
396
398
  end
397
399
  end
398
400
 
399
- self.bolt_host = "localhost"
400
- self.bolt_port = 7687
401
+ neo4j_uri = ENV["NEO4J_URI"]
402
+ if neo4j_uri.nil? || neo4j_uri.empty?
403
+ self.bolt_host = "localhost"
404
+ self.bolt_port = 7687
405
+ else
406
+ begin
407
+ parsed_neo4j_uri = URI.parse(neo4j_uri)
408
+ rescue URI::InvalidURIError => error
409
+ raise Error, "Invalid NEO4J_URI: #{error.message}"
410
+ end
411
+
412
+ unless %w[bolt neo4j].include?(parsed_neo4j_uri.scheme) && parsed_neo4j_uri.host
413
+ raise Error, "NEO4J_URI must be a bolt:// or neo4j:// URI with a host"
414
+ end
415
+ unless parsed_neo4j_uri.userinfo.nil? &&
416
+ ["", "/"].include?(parsed_neo4j_uri.path.to_s) &&
417
+ parsed_neo4j_uri.query.nil? && parsed_neo4j_uri.fragment.nil?
418
+ raise Error, "NEO4J_URI may only contain scheme, host, and port"
419
+ end
420
+
421
+ self.bolt_host = parsed_neo4j_uri.host
422
+ self.bolt_port = parsed_neo4j_uri.port || 7687
423
+ end
401
424
  self.bolt_verbosity = 0
402
425
  self.bolt_username = ENV["NEO4J_USERNAME"]
403
426
  self.bolt_password = ENV["NEO4J_PASSWORD"]
@@ -571,11 +594,13 @@ module Neo4jBolt
571
594
  nil
572
595
  end
573
596
 
574
- def dump_database(io, progress_io: nil, progress_color: "cyan")
597
+ def dump_database(io, progress_io: nil, progress_color: "cyan", schema: true)
575
598
  progress = ProgressReporter.new(progress_io, color: progress_color)
576
599
  dumped_nodes = 0
577
600
  dumped_relationships = 0
578
601
 
602
+ dump_schema(io) if schema
603
+
579
604
  transaction do
580
605
  identity_function = dump_identity_function
581
606
  total_nodes = neo4j_query_expect_one("MATCH (n) RETURN count(n) AS count")["count"]
@@ -632,7 +657,7 @@ module Neo4jBolt
632
657
  end
633
658
 
634
659
  def load_database_dump(io, force_append: false, progress_io: nil, progress_color: "cyan",
635
- initial_batch_size: LOAD_INITIAL_BATCH_SIZE)
660
+ initial_batch_size: LOAD_INITIAL_BATCH_SIZE, schema: true)
636
661
  raise Error, "load_database_dump cannot run inside a transaction" if transaction_context
637
662
 
638
663
  initial_batch_size = Integer(initial_batch_size)
@@ -646,16 +671,41 @@ module Neo4jBolt
646
671
  progress = ProgressReporter.new(progress_io, color: progress_color)
647
672
  node_batches = Hash.new { |hash, key| hash[key] = [] }
648
673
  relationship_batches = Hash.new { |hash, key| hash[key] = [] }
649
- total_nodes, total_relationships = parse_dump(io, node_batches, relationship_batches, progress)
674
+ total_nodes, total_relationships, schema_entries =
675
+ parse_dump(io, node_batches, relationship_batches, progress)
676
+
677
+ if !force_append && schema && !schema_entries.empty?
678
+ ensure_database_has_no_application_schema!
679
+ end
650
680
 
651
681
  temporary_token = SecureRandom.hex(12)
652
682
  temporary_label = "__neo4j_bolt_load_#{temporary_token}"
653
- temporary_id_property = "__neo4j_bolt_load_id_#{temporary_token}"
683
+ temporary_id_property = LOAD_ID_PROPERTY
654
684
  temporary_index = "neo4j_bolt_load_#{temporary_token}"
655
685
  quoted_temporary_label = quote_identifier(temporary_label)
656
686
  quoted_temporary_id = quote_identifier(temporary_id_property)
657
687
  quoted_temporary_index = quote_identifier(temporary_index)
658
688
  needs_relationship_lookup = total_relationships.positive?
689
+
690
+ if needs_relationship_lookup
691
+ dump_uses_reserved_property = node_batches.each_value.any? do |nodes|
692
+ nodes.any? { |node| node.fetch("properties", {}).key?(LOAD_ID_PROPERTY) }
693
+ end
694
+ if dump_uses_reserved_property
695
+ raise Error, "Dump contains reserved Neo4jBolt load property #{LOAD_ID_PROPERTY.inspect}."
696
+ end
697
+
698
+ if force_append
699
+ reserved_property_in_use = neo4j_query(
700
+ "MATCH (n) WHERE n.#{quoted_temporary_id} IS NOT NULL RETURN 1 AS present LIMIT 1"
701
+ ).any?
702
+ if reserved_property_in_use
703
+ raise Error,
704
+ "Database contains reserved Neo4jBolt load property #{LOAD_ID_PROPERTY.inspect}."
705
+ end
706
+ end
707
+ end
708
+
659
709
  loaded_nodes = 0
660
710
  loaded_relationships = 0
661
711
  index_created = false
@@ -759,6 +809,10 @@ module Neo4jBolt
759
809
  raise cleanup_error if original_error.nil? && cleanup_error
760
810
  end
761
811
 
812
+ if schema && !schema_entries.empty?
813
+ restore_dump_schema(schema_entries, progress, if_not_exists: force_append)
814
+ end
815
+
762
816
  progress.finish("Loaded: #{loaded_nodes} nodes, #{loaded_relationships} relationships")
763
817
  nil
764
818
  ensure
@@ -884,6 +938,157 @@ module Neo4jBolt
884
938
  "`#{identifier.to_s.gsub("`", "``")}`"
885
939
  end
886
940
 
941
+ def dump_schema(io)
942
+ entries = []
943
+ constraint_names = Set.new
944
+
945
+ neo4j_query("SHOW CONSTRAINTS").each do |row|
946
+ kind = dump_constraint_kind(row["type"])
947
+ value = schema_dump_value(row, kind)
948
+ constraint_names << value.fetch(:name)
949
+ entries << [kind, value]
950
+ end
951
+
952
+ neo4j_query("SHOW INDEXES").each do |row|
953
+ next unless dumpable_property_index?(row, constraint_names)
954
+
955
+ entries << ["i", schema_dump_value(row, "i")]
956
+ end
957
+
958
+ kind_order = { "u" => 0, "k" => 1, "i" => 2 }
959
+ entries.sort_by { |kind, value| [kind_order.fetch(kind), value.fetch(:name)] }.each do |kind, value|
960
+ io.puts "#{kind} #{JSON.generate(value)}"
961
+ end
962
+ end
963
+
964
+ def dump_constraint_kind(type)
965
+ case type
966
+ when "UNIQUENESS", "NODE_PROPERTY_UNIQUENESS", "RELATIONSHIP_PROPERTY_UNIQUENESS"
967
+ "u"
968
+ when "NODE_KEY", "RELATIONSHIP_KEY"
969
+ "k"
970
+ else
971
+ raise Error,
972
+ "Cannot dump #{type.inspect} constraint; use schema: false (or --data-only in the CLI)"
973
+ end
974
+ end
975
+
976
+ def dumpable_property_index?(row, constraint_names)
977
+ return false if row["owningConstraint"]
978
+ return false if row["uniqueness"] == "UNIQUE"
979
+ return false if constraint_names.include?(row["name"])
980
+ return false if row["type"] == "LOOKUP"
981
+ return true if %w[BTREE RANGE].include?(row["type"])
982
+
983
+ raise Error,
984
+ "Cannot dump #{row['type'].inspect} index #{row['name'].inspect}; " \
985
+ "use schema: false (or --data-only in the CLI)"
986
+ end
987
+
988
+ def schema_dump_value(row, kind)
989
+ name = row["name"]
990
+ entity_type = row["entityType"]
991
+ labels_or_types = row["labelsOrTypes"]
992
+ properties = row["properties"]
993
+
994
+ unless name.is_a?(String) && !name.empty? &&
995
+ %w[NODE RELATIONSHIP].include?(entity_type) &&
996
+ labels_or_types.is_a?(Array) && labels_or_types.size == 1 &&
997
+ properties.is_a?(Array) && !properties.empty? &&
998
+ properties.all? { |property| property.is_a?(String) && !property.empty? }
999
+ raise Error, "Cannot dump #{kind} schema entry #{name.inspect}: unsupported schema shape"
1000
+ end
1001
+
1002
+ {
1003
+ name: name,
1004
+ entity_type: entity_type,
1005
+ label_or_type: labels_or_types.first,
1006
+ properties: properties
1007
+ }
1008
+ end
1009
+
1010
+ def restore_dump_schema(entries, progress, if_not_exists: false)
1011
+ progress&.note("Restoring schema: #{entries.size} entries...")
1012
+ entries.each do |kind, value|
1013
+ neo4j_query(schema_create_statement(kind, value, if_not_exists: if_not_exists))
1014
+ end
1015
+ end
1016
+
1017
+ def schema_create_statement(kind, value, if_not_exists: false)
1018
+ validate_schema_dump_value!(kind, value)
1019
+
1020
+ name = quote_identifier(value.fetch("name"))
1021
+ if_not_exists_clause = if_not_exists ? " IF NOT EXISTS" : ""
1022
+ entity_type = value.fetch("entity_type")
1023
+ label_or_type = quote_identifier(value.fetch("label_or_type"))
1024
+ properties = value.fetch("properties")
1025
+ variable = entity_type == "NODE" ? "n" : "r"
1026
+ pattern = if entity_type == "NODE"
1027
+ "(#{variable}:#{label_or_type})"
1028
+ else
1029
+ "()-[#{variable}:#{label_or_type}]-()"
1030
+ end
1031
+ property_expressions = properties.map { |property| "#{variable}.#{quote_identifier(property)}" }
1032
+
1033
+ case kind
1034
+ when "u", "k"
1035
+ property_expression = if property_expressions.one?
1036
+ property_expressions.first
1037
+ else
1038
+ "(#{property_expressions.join(', ')})"
1039
+ end
1040
+ constraint_type = if kind == "u"
1041
+ "UNIQUE"
1042
+ elsif entity_type == "NODE"
1043
+ "NODE KEY"
1044
+ else
1045
+ "RELATIONSHIP KEY"
1046
+ end
1047
+ "CREATE CONSTRAINT #{name}#{if_not_exists_clause} FOR #{pattern} " \
1048
+ "REQUIRE #{property_expression} IS #{constraint_type}"
1049
+ when "i"
1050
+ "CREATE INDEX #{name}#{if_not_exists_clause} FOR #{pattern} ON (#{property_expressions.join(', ')})"
1051
+ else
1052
+ raise Error, "Unexpected schema dump kind: #{kind}"
1053
+ end
1054
+ end
1055
+
1056
+ def validate_schema_dump_value!(kind, value)
1057
+ unless %w[u k i].include?(kind) && value.is_a?(Hash)
1058
+ raise Error, "Invalid #{kind} schema dump entry"
1059
+ end
1060
+
1061
+ name = value.fetch("name")
1062
+ entity_type = value.fetch("entity_type")
1063
+ label_or_type = value.fetch("label_or_type")
1064
+ properties = value.fetch("properties")
1065
+
1066
+ return if name.is_a?(String) && !name.empty? &&
1067
+ %w[NODE RELATIONSHIP].include?(entity_type) &&
1068
+ label_or_type.is_a?(String) && !label_or_type.empty? &&
1069
+ properties.is_a?(Array) && !properties.empty? &&
1070
+ properties.all? { |property| property.is_a?(String) && !property.empty? }
1071
+
1072
+ raise Error, "Invalid #{kind} schema dump entry"
1073
+ end
1074
+
1075
+ def ensure_database_has_no_application_schema!
1076
+ constraints = neo4j_query("SHOW CONSTRAINTS").filter_map { |row| row["name"] }
1077
+ indexes = neo4j_query("SHOW INDEXES").filter_map do |row|
1078
+ next if row["type"] == "LOOKUP"
1079
+ next if row["owningConstraint"]
1080
+ next if row["uniqueness"] == "UNIQUE"
1081
+
1082
+ row["name"]
1083
+ end
1084
+ schema_names = constraints + indexes
1085
+ return if schema_names.empty?
1086
+
1087
+ raise Error,
1088
+ "There are constraints or indexes in this database, exiting now: " \
1089
+ "#{schema_names.sort.join(', ')}"
1090
+ end
1091
+
887
1092
  # Neo4j 4.4 has no elementId() function. Its id() fallback is used only for
888
1093
  # deterministic in-process ordering; those database IDs never enter a dump.
889
1094
  def dump_identity_function
@@ -896,6 +1101,7 @@ module Neo4jBolt
896
1101
  def parse_dump(io, node_batches, relationship_batches, progress = nil)
897
1102
  node_count = 0
898
1103
  relationship_count = 0
1104
+ schema_entries = []
899
1105
  progress&.update("Reading dump: 0 nodes, 0 relationships", force: true)
900
1106
 
901
1107
  io.each_line.with_index(1) do |line, line_number|
@@ -903,16 +1109,22 @@ module Neo4jBolt
903
1109
  next if line.empty?
904
1110
 
905
1111
  kind, json = line.split(" ", 2)
906
- raise Error, "Invalid dump entry on line #{line_number}" unless json && %w[n r].include?(kind)
1112
+ unless json && %w[n r u k i].include?(kind)
1113
+ raise Error, "Invalid dump entry on line #{line_number}"
1114
+ end
907
1115
 
908
1116
  value = JSON.parse(json)
909
- if kind == "n"
1117
+ case kind
1118
+ when "n"
910
1119
  labels = value.fetch("labels")
911
1120
  node_batches[labels.sort] << value
912
1121
  node_count += 1
913
- else
1122
+ when "r"
914
1123
  relationship_batches[value.fetch("type")] << value
915
1124
  relationship_count += 1
1125
+ else
1126
+ validate_schema_dump_value!(kind, value)
1127
+ schema_entries << [kind, value]
916
1128
  end
917
1129
  if ((node_count + relationship_count) % LOAD_PROGRESS_STEP).zero?
918
1130
  progress&.update("Reading dump: #{node_count} nodes, #{relationship_count} relationships")
@@ -922,7 +1134,7 @@ module Neo4jBolt
922
1134
  end
923
1135
 
924
1136
  progress&.finish("Read dump: #{node_count} nodes, #{relationship_count} relationships")
925
- [node_count, relationship_count]
1137
+ [node_count, relationship_count, schema_entries]
926
1138
  end
927
1139
 
928
1140
  def adaptive_each_slice(items, batch_size, progress, kind)
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: neo4j_bolt
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.4
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Specht