grumlin 0.21.1 → 0.22.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +40 -10
- data/lib/grumlin/action.rb +1 -1
- data/lib/grumlin/client.rb +8 -42
- data/lib/grumlin/dummy_transaction.rb +25 -0
- data/lib/grumlin/repository/instance_methods.rb +18 -17
- data/lib/grumlin/steps.rb +3 -1
- data/lib/grumlin/steps_serializers/bytecode.rb +8 -6
- data/lib/grumlin/transaction.rb +9 -21
- data/lib/grumlin/traversal_start.rb +21 -3
- data/lib/grumlin/version.rb +1 -1
- data/lib/grumlin.rb +3 -0
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 519d93b97b2cd8f07a1e11e75e45b0b7ca780947e82b3624d5e6c7c6d719ec27
|
4
|
+
data.tar.gz: 5eed9eab9e99fe02246995adbc8c01988bd21d150df234946a575d01cda518f2
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 76cd4990acd94d119b2a716e25b10d35bdc970e794a77de4d25dcc7e3f67f5129236ff1c037a4f0f0c3208643afca9695b6dc602bb84ae01d61a84c1b89364fe
|
7
|
+
data.tar.gz: afda51dc7010f1fc508e96b203727ebee971597b6425aa2ec29932d7957e1b2b12d8959526a5fc10d92e46926e94dab1ca3d62eb583bbb36498f99bbd3e11689
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -64,8 +64,7 @@ explicitly specify the provider you use.
|
|
64
64
|
#### Provider features
|
65
65
|
|
66
66
|
Every provider is described by a set of features. In the future `Grumlin` may decide to disable or enable
|
67
|
-
some parts of it's functionality to comply provider's supported features.
|
68
|
-
in behaviour when working with different providers.
|
67
|
+
some parts of it's functionality to comply provider's supported features.
|
69
68
|
|
70
69
|
To check current providers supported features use
|
71
70
|
|
@@ -73,6 +72,12 @@ To check current providers supported features use
|
|
73
72
|
Grumlin.features
|
74
73
|
```
|
75
74
|
|
75
|
+
Current differences between providers:
|
76
|
+
|
77
|
+
| Feature | TinkerGraph |AWS Neptune|
|
78
|
+
|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
|
79
|
+
| Transactions | Transaction semantic is ignoroed, data is always writen, `tx.rollback` does nothing, an info is printed every time transactions are used with TinkerGraph |Full support
|
80
|
+
|
76
81
|
### Traversing graphs
|
77
82
|
|
78
83
|
**Warning**: Not all steps and expressions defined in the reference documentation are supported.
|
@@ -240,21 +245,28 @@ Each `return_mode` is mapped to a particular termination step:
|
|
240
245
|
- `:traversal` - do not execute the query and return the traversal as is
|
241
246
|
|
242
247
|
`Grumlin::Repository` also provides a set of generic CRUD operations:
|
243
|
-
- `add_vertex(label, id = nil, **properties)`
|
244
|
-
- `add_edge(label, id = nil, from:, to:, **properties)`
|
245
|
-
- `drop_vertex(id)`
|
246
|
-
- `drop_edge(id = nil, from: nil, to: nil, label: nil)`
|
248
|
+
- `add_vertex(label, id = nil, start: g, **properties)`
|
249
|
+
- `add_edge(label, id = nil, from:, to:, start: g, **properties)`
|
250
|
+
- `drop_vertex(id, start: g)`
|
251
|
+
- `drop_edge(id = nil, from: nil, to: nil, label: nil, start: g)`
|
247
252
|
|
248
253
|
and a few methods that emulate upserts:
|
249
|
-
- `upsert_vertex(label, id, create_properties: {}, update_properties: {}, on_failure: :retry, **params)`
|
250
|
-
- `upsert_edge(label, from:, to:, create_properties: {}, update_properties: {}, on_failure: :retry, **params)`
|
251
|
-
- `upsert_edges(edges, batch_size: 100, on_failure: :retry, **params)`
|
252
|
-
- `upsert_vertices(edges, batch_size: 100, on_failure: :retry, **params)`
|
254
|
+
- `upsert_vertex(label, id, create_properties: {}, update_properties: {}, on_failure: :retry, start: g, **params)`
|
255
|
+
- `upsert_edge(label, from:, to:, create_properties: {}, update_properties: {}, on_failure: :retry, start: g, **params)`
|
256
|
+
- `upsert_edges(edges, batch_size: 100, on_failure: :retry, start: g, **params)`
|
257
|
+
- `upsert_vertices(edges, batch_size: 100, on_failure: :retry, start: g, **params)`
|
253
258
|
|
254
259
|
All of them support 3 different modes for error handling: `:retry`, `:ignore` and `:raise`. Retry mode is implemented
|
255
260
|
with [retryable](https://github.com/nfedyashev/retryable). **params will be merged to the default config for upserts
|
256
261
|
and passed to `Retryable.retryable`. In case if you want to modify retryable behaviour you are to do so.
|
257
262
|
|
263
|
+
If you want to use these methods inside a transaction simply pass your `gtx` as `start` parameter:
|
264
|
+
```ruby
|
265
|
+
g.tx do |gtx|
|
266
|
+
add_vertex(:vertex, start: gtx)
|
267
|
+
end
|
268
|
+
```
|
269
|
+
|
258
270
|
If you don't want to define you own repository, simply use
|
259
271
|
|
260
272
|
`Grumlin::Repository.new` returns an instance of an anonymous class extending `Grumlin::Repository`.
|
@@ -282,6 +294,24 @@ it may be useful for debugging. Note that one needs to call a termination step m
|
|
282
294
|
|
283
295
|
method will return profiling data of the results.
|
284
296
|
|
297
|
+
#### Transactions
|
298
|
+
|
299
|
+
Since 0.22.0 `Grumlin` supports transactions when working with providers that supports them:
|
300
|
+
```ruby
|
301
|
+
# Using Transaction directly
|
302
|
+
tx = g.tx
|
303
|
+
gtx = tx.begin
|
304
|
+
gtx.addV(:vertex).iterate
|
305
|
+
tx.commit # or tx.rollback
|
306
|
+
|
307
|
+
# Using with a block
|
308
|
+
g.tx do |gtx|
|
309
|
+
gtx.addV(:vertex).iterate
|
310
|
+
# raise Grumlin::Rollback to manually rollback
|
311
|
+
# any other exception will also rollback the transaction and will be reraised
|
312
|
+
end # commits automatically
|
313
|
+
```
|
314
|
+
|
285
315
|
#### IRB
|
286
316
|
|
287
317
|
An example of how to start an IRB session with support for executing gremlin queries:
|
data/lib/grumlin/action.rb
CHANGED
data/lib/grumlin/client.rb
CHANGED
@@ -30,12 +30,6 @@ module Grumlin
|
|
30
30
|
@count += 1
|
31
31
|
end
|
32
32
|
|
33
|
-
def finalize_tx(action, session_id)
|
34
|
-
@client.finalize_tx(action, session_id)
|
35
|
-
ensure
|
36
|
-
@count += 1
|
37
|
-
end
|
38
|
-
|
39
33
|
def viable?
|
40
34
|
!closed?
|
41
35
|
end
|
@@ -104,15 +98,15 @@ module Grumlin
|
|
104
98
|
raise NotConnectedError unless connected?
|
105
99
|
|
106
100
|
request = to_query(bytecode, session_id: session_id)
|
107
|
-
submit_request(request)
|
108
|
-
end
|
109
101
|
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
|
114
|
-
|
115
|
-
|
102
|
+
channel = @request_dispatcher.add_request(request)
|
103
|
+
begin
|
104
|
+
@transport.write(request)
|
105
|
+
channel.dequeue.flat_map { |item| Typing.cast(item) }
|
106
|
+
rescue Async::Stop, Async::TimeoutError
|
107
|
+
close(check_requests: false)
|
108
|
+
raise
|
109
|
+
end
|
116
110
|
end
|
117
111
|
|
118
112
|
def inspect
|
@@ -125,18 +119,6 @@ module Grumlin
|
|
125
119
|
|
126
120
|
private
|
127
121
|
|
128
|
-
def submit_request(request)
|
129
|
-
channel = @request_dispatcher.add_request(request)
|
130
|
-
@transport.write(request)
|
131
|
-
|
132
|
-
begin
|
133
|
-
channel.dequeue.flat_map { |item| Typing.cast(item) }
|
134
|
-
rescue Async::Stop, Async::TimeoutError
|
135
|
-
close(check_requests: false)
|
136
|
-
raise
|
137
|
-
end
|
138
|
-
end
|
139
|
-
|
140
122
|
# This might be overridden in successors
|
141
123
|
def build_transport
|
142
124
|
Transport.new(@url, parent: @parent, **@client_options)
|
@@ -157,21 +139,5 @@ module Grumlin
|
|
157
139
|
}.compact
|
158
140
|
}
|
159
141
|
end
|
160
|
-
|
161
|
-
def finalize_tx_query(action, session_id)
|
162
|
-
{
|
163
|
-
requestId: SecureRandom.uuid,
|
164
|
-
op: :bytecode,
|
165
|
-
processor: session_id ? :session : :traversal,
|
166
|
-
args: {
|
167
|
-
gremlin: {
|
168
|
-
:@type => "g:Bytecode",
|
169
|
-
:@value => { source: [[:tx, action]] }
|
170
|
-
},
|
171
|
-
aliases: { g: :g },
|
172
|
-
session: session_id
|
173
|
-
}.compact
|
174
|
-
}
|
175
|
-
end
|
176
142
|
end
|
177
143
|
end
|
@@ -0,0 +1,25 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Grumlin
|
4
|
+
class DummyTransaction < Transaction
|
5
|
+
attr_reader :uuid
|
6
|
+
|
7
|
+
include Console
|
8
|
+
|
9
|
+
def initialize(traversal_start_class, pool: Grumlin.default_pool) # rubocop:disable Lint/MissingSuper, Lint/UnusedMethodArgument
|
10
|
+
@traversal_start_class = traversal_start_class
|
11
|
+
|
12
|
+
logger.info(self) do
|
13
|
+
"#{Grumlin.config.provider} does not support transactions. commit and rollback are ignored, data will be saved"
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
def commit
|
18
|
+
nil
|
19
|
+
end
|
20
|
+
|
21
|
+
def rollback
|
22
|
+
nil
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
@@ -22,51 +22,51 @@ module Grumlin
|
|
22
22
|
self.class.shortcuts
|
23
23
|
end
|
24
24
|
|
25
|
-
def drop_vertex(id)
|
26
|
-
|
25
|
+
def drop_vertex(id, start: g)
|
26
|
+
start.V(id).drop.iterate
|
27
27
|
end
|
28
28
|
|
29
|
-
def drop_edge(id = nil, from: nil, to: nil, label: nil) # rubocop:disable Metrics/AbcSize
|
29
|
+
def drop_edge(id = nil, from: nil, to: nil, label: nil, start: g) # rubocop:disable Metrics/AbcSize
|
30
30
|
raise ArgumentError, "either id or from:, to: and label: must be passed" if [id, from, to, label].all?(&:nil?)
|
31
|
-
return
|
31
|
+
return start.E(id).drop.iterate unless id.nil?
|
32
32
|
|
33
33
|
raise ArgumentError, "from:, to: and label: must be passed" if [from, to, label].any?(&:nil?)
|
34
34
|
|
35
|
-
|
35
|
+
start.V(from).outE(label).where(__.inV.hasId(to)).limit(1).drop.iterate
|
36
36
|
end
|
37
37
|
|
38
|
-
def add_vertex(label, id = nil, **properties)
|
38
|
+
def add_vertex(label, id = nil, start: g, **properties)
|
39
39
|
id ||= properties[T.id]
|
40
40
|
properties = except(properties, T.id)
|
41
41
|
|
42
|
-
t =
|
42
|
+
t = start.addV(label)
|
43
43
|
t = t.props(T.id => id) unless id.nil?
|
44
44
|
t.props(**properties).next
|
45
45
|
end
|
46
46
|
|
47
|
-
def add_edge(label, id = nil, from:, to:, **properties)
|
47
|
+
def add_edge(label, id = nil, from:, to:, start: g, **properties)
|
48
48
|
id ||= properties[T.id]
|
49
49
|
properties = except(properties, T.label)
|
50
50
|
properties[T.id] = id
|
51
51
|
|
52
|
-
|
52
|
+
start.addE(label).from(__.V(from)).to(__.V(to)).props(**properties).next
|
53
53
|
end
|
54
54
|
|
55
|
-
def upsert_vertex(label, id, create_properties: {}, update_properties: {}, on_failure: :retry, **params)
|
55
|
+
def upsert_vertex(label, id, create_properties: {}, update_properties: {}, on_failure: :retry, start: g, **params) # rubocop:disable Metrics/ParameterLists
|
56
56
|
with_upsert_error_handling(on_failure, params) do
|
57
57
|
create_properties, update_properties = cleanup_properties(create_properties, update_properties)
|
58
58
|
|
59
|
-
|
59
|
+
start.upsertV(label, id, create_properties, update_properties).id.next
|
60
60
|
end
|
61
61
|
end
|
62
62
|
|
63
63
|
# vertices:
|
64
64
|
# [["label", "id", {create: :properties}, {update: properties}]]
|
65
65
|
# params can override Retryable config from UPSERT_RETRY_PARAMS
|
66
|
-
def upsert_vertices(vertices, batch_size: 100, on_failure: :retry, **params)
|
66
|
+
def upsert_vertices(vertices, batch_size: 100, on_failure: :retry, start: g, **params)
|
67
67
|
vertices.each_slice(batch_size) do |slice|
|
68
68
|
with_upsert_error_handling(on_failure, params) do
|
69
|
-
slice.reduce(
|
69
|
+
slice.reduce(start) do |t, (label, id, create_properties, update_properties)|
|
70
70
|
create_properties, update_properties = cleanup_properties(create_properties, update_properties)
|
71
71
|
|
72
72
|
t.upsertV(label, id, create_properties, update_properties)
|
@@ -77,20 +77,21 @@ module Grumlin
|
|
77
77
|
|
78
78
|
# Only from and to are used to find the existing edge, if one wants to assign an id to a created edge,
|
79
79
|
# it must be passed as T.id in create_properties.
|
80
|
-
def upsert_edge(label, from:, to:, create_properties: {}, update_properties: {},
|
80
|
+
def upsert_edge(label, from:, to:, create_properties: {}, update_properties: {}, # rubocop:disable Metrics/ParameterLists
|
81
|
+
on_failure: :retry, start: g, **params)
|
81
82
|
with_upsert_error_handling(on_failure, params) do
|
82
83
|
create_properties, update_properties = cleanup_properties(create_properties, update_properties, T.label)
|
83
|
-
|
84
|
+
start.upsertE(label, from, to, create_properties, update_properties).id.next
|
84
85
|
end
|
85
86
|
end
|
86
87
|
|
87
88
|
# edges:
|
88
89
|
# [["label", "from", "to", {create: :properties}, {update: properties}]]
|
89
90
|
# params can override Retryable config from UPSERT_RETRY_PARAMS
|
90
|
-
def upsert_edges(edges, batch_size: 100, on_failure: :retry, **params)
|
91
|
+
def upsert_edges(edges, batch_size: 100, on_failure: :retry, start: g, **params)
|
91
92
|
edges.each_slice(batch_size) do |slice|
|
92
93
|
with_upsert_error_handling(on_failure, params) do
|
93
|
-
slice.reduce(
|
94
|
+
slice.reduce(start) do |t, (label, from, to, create_properties, update_properties)|
|
94
95
|
create_properties, update_properties = cleanup_properties(create_properties, update_properties, T.label)
|
95
96
|
|
96
97
|
t.upsertE(label, from, to, create_properties, update_properties)
|
data/lib/grumlin/steps.rb
CHANGED
@@ -32,7 +32,9 @@ module Grumlin
|
|
32
32
|
end
|
33
33
|
|
34
34
|
def add(name, args: [], params: {})
|
35
|
-
|
35
|
+
if CONFIGURATION_STEPS.include?(name) || name.to_sym == :tx
|
36
|
+
return add_configuration_step(name, args: args, params: params)
|
37
|
+
end
|
36
38
|
|
37
39
|
StepData.new(name, args: cast_arguments(args), params: params).tap do |step|
|
38
40
|
@steps << step
|
@@ -10,17 +10,19 @@ module Grumlin
|
|
10
10
|
|
11
11
|
def serialize
|
12
12
|
steps = ShortcutsApplyer.call(@steps)
|
13
|
-
no_return = @params
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
}.tap do |v|
|
18
|
-
v.merge!(source: steps.configuration_steps.map { |s| serialize_step(s) }) if steps.configuration_steps.any?
|
13
|
+
no_return = @params.fetch(:no_return, false)
|
14
|
+
{}.tap do |result|
|
15
|
+
result[:step] = serialize_steps(steps.steps + (no_return ? [NONE_STEP] : [])) if steps.steps.any?
|
16
|
+
result[:source] = serialize_steps(steps.configuration_steps) if steps.configuration_steps.any?
|
19
17
|
end
|
20
18
|
end
|
21
19
|
|
22
20
|
private
|
23
21
|
|
22
|
+
def serialize_steps(steps)
|
23
|
+
steps.map { |s| serialize_step(s) }
|
24
|
+
end
|
25
|
+
|
24
26
|
def serialize_step(step)
|
25
27
|
[step.name].tap do |result|
|
26
28
|
step.args.each do |arg|
|
data/lib/grumlin/transaction.rb
CHANGED
@@ -2,49 +2,37 @@
|
|
2
2
|
|
3
3
|
module Grumlin
|
4
4
|
class Transaction
|
5
|
-
attr_reader :
|
5
|
+
attr_reader :session_id
|
6
6
|
|
7
7
|
include Console
|
8
8
|
|
9
|
+
COMMIT = Grumlin::Repository.new.g.step(:tx, :commit).bytecode
|
10
|
+
ROLLBACK = Grumlin::Repository.new.g.step(:tx, :rollback).bytecode
|
11
|
+
|
9
12
|
def initialize(traversal_start_class, pool: Grumlin.default_pool)
|
10
13
|
@traversal_start_class = traversal_start_class
|
11
14
|
@pool = pool
|
12
15
|
|
13
|
-
|
14
|
-
@uuid = SecureRandom.uuid
|
15
|
-
return
|
16
|
-
end
|
17
|
-
|
18
|
-
logger.info(self) do
|
19
|
-
"#{Grumlin.config.provider} does not support transactions. commit and rollback are ignored, data will be saved"
|
20
|
-
end
|
21
|
-
end
|
22
|
-
|
23
|
-
def supported?
|
24
|
-
Grumlin.features.supports_transactions?
|
16
|
+
@session_id = SecureRandom.uuid
|
25
17
|
end
|
26
18
|
|
27
19
|
def begin
|
28
|
-
@traversal_start_class.new(session_id: @
|
20
|
+
@traversal_start_class.new(session_id: @session_id)
|
29
21
|
end
|
30
22
|
|
31
23
|
def commit
|
32
|
-
|
33
|
-
|
34
|
-
finalize(:commit)
|
24
|
+
finalize(COMMIT)
|
35
25
|
end
|
36
26
|
|
37
27
|
def rollback
|
38
|
-
|
39
|
-
|
40
|
-
finalize(:rollback)
|
28
|
+
finalize(ROLLBACK)
|
41
29
|
end
|
42
30
|
|
43
31
|
private
|
44
32
|
|
45
33
|
def finalize(action)
|
46
34
|
@pool.acquire do |client|
|
47
|
-
client.
|
35
|
+
client.write(action, session_id: @session_id)
|
48
36
|
end
|
49
37
|
end
|
50
38
|
end
|
@@ -5,12 +5,24 @@ module Grumlin
|
|
5
5
|
include WithExtension
|
6
6
|
|
7
7
|
class TraversalError < Grumlin::Error; end
|
8
|
-
class
|
8
|
+
class AlreadyBoundToTransactionError < TraversalError; end
|
9
9
|
|
10
10
|
def tx
|
11
|
-
raise
|
11
|
+
raise AlreadyBoundToTransactionError if @session_id
|
12
12
|
|
13
|
-
|
13
|
+
transaction = tx_class.new(self.class, pool: @pool)
|
14
|
+
return transaction unless block_given?
|
15
|
+
|
16
|
+
begin
|
17
|
+
yield transaction.begin
|
18
|
+
rescue Grumlin::Rollback
|
19
|
+
transaction.rollback
|
20
|
+
rescue StandardError
|
21
|
+
transaction.rollback
|
22
|
+
raise
|
23
|
+
else
|
24
|
+
transaction.commit
|
25
|
+
end
|
14
26
|
end
|
15
27
|
|
16
28
|
def to_s(*)
|
@@ -20,5 +32,11 @@ module Grumlin
|
|
20
32
|
def inspect
|
21
33
|
self.class.inspect
|
22
34
|
end
|
35
|
+
|
36
|
+
private
|
37
|
+
|
38
|
+
def tx_class
|
39
|
+
Grumlin.features.supports_transactions? ? Transaction : DummyTransaction
|
40
|
+
end
|
23
41
|
end
|
24
42
|
end
|
data/lib/grumlin/version.rb
CHANGED
data/lib/grumlin.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: grumlin
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.22.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Gleb Sinyavskiy
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-08-
|
11
|
+
date: 2022-08-12 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: async-pool
|
@@ -117,6 +117,7 @@ files:
|
|
117
117
|
- lib/grumlin/benchmark/repository.rb
|
118
118
|
- lib/grumlin/client.rb
|
119
119
|
- lib/grumlin/config.rb
|
120
|
+
- lib/grumlin/dummy_transaction.rb
|
120
121
|
- lib/grumlin/edge.rb
|
121
122
|
- lib/grumlin/expressions/cardinality.rb
|
122
123
|
- lib/grumlin/expressions/column.rb
|