solis 0.114.0 → 0.115.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +83 -1
- data/lib/solis/model.rb +210 -100
- data/lib/solis/store/sparql/client.rb +9 -3
- data/lib/solis/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7c9039f8bba386333b11237b9914c7fdeee57ad8094ea57058815f2fc008d4a7
|
|
4
|
+
data.tar.gz: ea7269d0380b39c1a88a7d064c503c972194a548441c6c32459ab4ca7187cab8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 17d2aa3f39b2bee0cb267996467ba8a0fe08d433e3e63c9dbab3df8624cac6ebf45703bc90c606f3c7d90d4c0ab12b72c3794056c11481f008141c456fe27675
|
|
7
|
+
data.tar.gz: 0e9faa327c21a51aa3413ed0d90c20f781186c360ae4e33c3d2b6432c1e9779e19c100008197b4fbed8969eefecdf2d9d5776cf18ab72872f0dff95ac334160b
|
data/README.md
CHANGED
|
@@ -4,7 +4,7 @@ Solis means 'the sun' in Latin or just Silos spelled backwards.
|
|
|
4
4
|
A simple idea to use SHACL to describe models and API on top of a data store.
|
|
5
5
|
|
|
6
6
|
# Status
|
|
7
|
-
Although I'm using it in production I must shamefully acknowledge that it actually needs refactoring. Solis is currently still grown organically driven by my needs and understanding of problem domain(RDF, triples, ...).
|
|
7
|
+
Although I'm using it in production I must shamefully acknowledge that it actually needs refactoring. Solis is currently still grown organically driven by my needs and understanding of the problem domain(RDF, triples, ...).
|
|
8
8
|
When my current project is live I'll start to refactor the code and release it as 1.0.
|
|
9
9
|
|
|
10
10
|
# From Google Sheets to RDF, Shacl, ... to API
|
|
@@ -422,6 +422,88 @@ Solis::ConfigFile.path = './'
|
|
|
422
422
|
solis = Solis::Graph.new(Solis::Shape::Reader::File.read(Solis::ConfigFile[:solis][:shacl]), Solis::ConfigFile[:solis][:env])
|
|
423
423
|
```
|
|
424
424
|
|
|
425
|
+
## Performance Optimizations
|
|
426
|
+
|
|
427
|
+
Entity create/update operations have been optimized to reduce SPARQL round-trips from ~25+ down to ~5-7 for a typical update with embedded entities.
|
|
428
|
+
|
|
429
|
+
### What changed
|
|
430
|
+
|
|
431
|
+
| Optimization | Impact |
|
|
432
|
+
|---|---|
|
|
433
|
+
| **Cached `up?` health check** (30s TTL) | Eliminates ~10 redundant ASK queries per operation |
|
|
434
|
+
| **Removed UUID collision check** | Saves 2 round-trips per new entity (UUID v4 collision is ~1 in 2^122) |
|
|
435
|
+
| **Eliminated redundant re-fetches in `update()`** | Saves 1-3 SELECT queries by returning in-memory data |
|
|
436
|
+
| **`known_entities` cache in graph building** | Avoids re-fetching entities from store during `build_ttl_objekt` recursion |
|
|
437
|
+
| **Batched existence checks** (`batch_exists?`) | Replaces N individual ASK queries with 1 SELECT for embedded entities |
|
|
438
|
+
| **Batched orphan reference checks** (`batch_referenced?`) | Replaces N individual ASK queries with 1 SELECT for orphan detection |
|
|
439
|
+
| **SPARQL client reuse** | `save()` passes its client to `update()` when delegating |
|
|
440
|
+
|
|
441
|
+
## PATCH vs PUT Updates
|
|
442
|
+
|
|
443
|
+
The `update()` method supports both PUT (full replace) and PATCH (partial merge) semantics via the `patch:` keyword argument.
|
|
444
|
+
|
|
445
|
+
```ruby
|
|
446
|
+
# PUT (default) — replaces embedded arrays entirely, orphans removed entities
|
|
447
|
+
entity.update({ 'id' => '1', 'students' => [{ 'id' => 's1' }] })
|
|
448
|
+
|
|
449
|
+
# PATCH — merges into existing arrays, no orphan deletion
|
|
450
|
+
entity.update({ 'id' => '1', 'students' => [{ 'id' => 's3' }] }, true, true, nil, patch: true)
|
|
451
|
+
```
|
|
452
|
+
|
|
453
|
+
### PUT mode (default, `patch: false`)
|
|
454
|
+
- Embedded entity arrays are **fully replaced** by the provided data
|
|
455
|
+
- Entities removed from the array are detected as orphans and may be deleted
|
|
456
|
+
- Omitted attributes keep their original values
|
|
457
|
+
|
|
458
|
+
### PATCH mode (`patch: true`)
|
|
459
|
+
- Embedded entity arrays are **merged**: new IDs are added, existing IDs are updated, unmentioned IDs are kept
|
|
460
|
+
- **No orphan detection or deletion** — removing an entity requires an explicit PUT or destroy
|
|
461
|
+
- Omitted embedded attributes are left completely untouched
|
|
462
|
+
|
|
463
|
+
## Orphan Protection (Embedded Entity Lifecycle)
|
|
464
|
+
|
|
465
|
+
When updating an entity with PUT semantics and removing a reference to an embedded entity, Solis decides whether to delete the orphaned entity using this decision chain:
|
|
466
|
+
|
|
467
|
+
| # | Check | Result |
|
|
468
|
+
|---|-------|--------|
|
|
469
|
+
| 1 | Entity in `embedded_readonly`? | **Never delete** (code tables, lookup values) |
|
|
470
|
+
| 2 | Top-level entity + NOT in `embedded_delete`? | **Unlink only** (safe default) |
|
|
471
|
+
| 3 | Top-level entity + in `embedded_delete`? | **Delete** (opt-in cascade) |
|
|
472
|
+
| 4 | Still referenced by other entities? | **Never delete** (safety net) |
|
|
473
|
+
|
|
474
|
+
A **top-level entity** is one that has its own `sh:NodeShape` + `sh:targetClass` in the SHACL schema — it is independently addressable with its own CRUD endpoint.
|
|
475
|
+
|
|
476
|
+
### Configuration
|
|
477
|
+
|
|
478
|
+
```yaml
|
|
479
|
+
:solis:
|
|
480
|
+
# Code tables / lookup values: never modified, never deleted
|
|
481
|
+
:embedded_readonly:
|
|
482
|
+
- :Codetable
|
|
483
|
+
|
|
484
|
+
# Child/join entities that should cascade-delete when orphaned
|
|
485
|
+
:embedded_delete:
|
|
486
|
+
- :About
|
|
487
|
+
- :Contributor
|
|
488
|
+
- :Comment
|
|
489
|
+
- :Link
|
|
490
|
+
- :ImageObject
|
|
491
|
+
- :Identifier
|
|
492
|
+
- :Provider
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
### Example
|
|
496
|
+
|
|
497
|
+
Given an Item with 5 About entities (join between Item and Tag+Role):
|
|
498
|
+
|
|
499
|
+
```ruby
|
|
500
|
+
# PUT: reduce from 5 Abouts to 1 — the other 4 are deleted (About is in embedded_delete)
|
|
501
|
+
item.update({ 'id' => '1', 'about' => [{ 'id' => 'a1' }] })
|
|
502
|
+
|
|
503
|
+
# But if Item references a Tag directly, removing it just unlinks — Tag is top-level, not in embedded_delete
|
|
504
|
+
```
|
|
505
|
+
|
|
506
|
+
Entities like `Tag`, `Agens`, `Person`, `Organization`, `Owner`, `Book` are protected automatically because they are top-level entities not listed in `embedded_delete`.
|
|
425
507
|
|
|
426
508
|
TODO:
|
|
427
509
|
- extract sparql layer into its own gem
|
data/lib/solis/model.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
require 'securerandom'
|
|
2
2
|
require 'iso8601'
|
|
3
3
|
require 'hashdiff'
|
|
4
|
+
require 'set'
|
|
4
5
|
require_relative 'query'
|
|
5
6
|
|
|
6
7
|
module Solis
|
|
@@ -150,33 +151,40 @@ values ?s {<#{self.graph_id}>}
|
|
|
150
151
|
|
|
151
152
|
if self.exists?(sparql)
|
|
152
153
|
data = properties_to_hash(self)
|
|
153
|
-
result = update(data)
|
|
154
|
+
result = update(data, validate_dependencies, top_level, sparql)
|
|
154
155
|
else
|
|
155
156
|
data = properties_to_hash(self)
|
|
156
157
|
attributes = data.include?('attributes') ? data['attributes'] : data
|
|
157
158
|
readonly_list = (Solis::Options.instance.get[:embedded_readonly] || []).map(&:to_s)
|
|
158
159
|
|
|
160
|
+
# Collect all embedded entities first for batched existence check
|
|
161
|
+
all_embedded = []
|
|
159
162
|
attributes.each_pair do |key, value|
|
|
160
163
|
unless self.class.metadata[:attributes][key][:node].nil?
|
|
161
164
|
value = [value] unless value.is_a?(Array)
|
|
162
165
|
value.each do |sub_value|
|
|
163
166
|
embedded = self.class.graph.shape_as_model(self.class.metadata[:attributes][key][:datatype].to_s).new(sub_value)
|
|
167
|
+
all_embedded << embedded
|
|
168
|
+
end
|
|
169
|
+
end
|
|
170
|
+
end
|
|
164
171
|
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
172
|
+
# Batch check existence of all embedded entities in one query
|
|
173
|
+
existing_ids = self.class.batch_exists?(sparql, all_embedded)
|
|
174
|
+
|
|
175
|
+
all_embedded.each do |embedded|
|
|
176
|
+
entity_exists = existing_ids.include?(embedded.graph_id)
|
|
177
|
+
|
|
178
|
+
if readonly_entity?(embedded, readonly_list)
|
|
179
|
+
unless entity_exists
|
|
180
|
+
Solis::LOGGER.warn("#{embedded.class.name} (id: #{embedded.id}) is readonly but does not exist in database. Skipping.")
|
|
181
|
+
end
|
|
182
|
+
else
|
|
183
|
+
if entity_exists
|
|
184
|
+
embedded_data = properties_to_hash(embedded)
|
|
185
|
+
embedded.update(embedded_data, validate_dependencies, false)
|
|
186
|
+
else
|
|
187
|
+
embedded.save(validate_dependencies, false)
|
|
180
188
|
end
|
|
181
189
|
end
|
|
182
190
|
end
|
|
@@ -195,14 +203,28 @@ values ?s {<#{self.graph_id}>}
|
|
|
195
203
|
raise e
|
|
196
204
|
end
|
|
197
205
|
|
|
198
|
-
|
|
206
|
+
# Update an entity.
|
|
207
|
+
#
|
|
208
|
+
# @param data [Hash] the attributes to update. Must include 'id'.
|
|
209
|
+
# @param validate_dependencies [Boolean] whether to validate dependencies (default: true)
|
|
210
|
+
# @param top_level [Boolean] whether this is a top-level call (default: true)
|
|
211
|
+
# @param sparql_client [SPARQL::Client, nil] optional reusable SPARQL client
|
|
212
|
+
# @param patch [Boolean] when true, uses PATCH semantics:
|
|
213
|
+
# - Only provided attributes are changed; omitted attributes are untouched
|
|
214
|
+
# - Embedded entity arrays are merged (new IDs added, existing IDs updated,
|
|
215
|
+
# missing IDs are kept as-is — not orphaned)
|
|
216
|
+
# - No orphan detection or deletion
|
|
217
|
+
# When false (default), uses PUT semantics:
|
|
218
|
+
# - Embedded entity arrays are fully replaced
|
|
219
|
+
# - Entities removed from the array are orphaned and deleted if unreferenced
|
|
220
|
+
def update(data, validate_dependencies = true, top_level = true, sparql_client = nil, patch: false)
|
|
199
221
|
raise Solis::Error::GeneralError, "I need a SPARQL endpoint" if self.class.sparql_endpoint.nil?
|
|
200
222
|
|
|
201
223
|
attributes = data.include?('attributes') ? data['attributes'] : data
|
|
202
224
|
raise "id is mandatory when updating" unless attributes.keys.include?('id')
|
|
203
225
|
|
|
204
226
|
id = attributes.delete('id')
|
|
205
|
-
sparql = SPARQL::Client.new(self.class.sparql_endpoint)
|
|
227
|
+
sparql = sparql_client || SPARQL::Client.new(self.class.sparql_endpoint)
|
|
206
228
|
|
|
207
229
|
original_klass = self.query.filter({ language: self.class.language, filters: { id: [id] } }).find_all.map { |m| m }&.first
|
|
208
230
|
raise Solis::Error::NotFoundError if original_klass.nil?
|
|
@@ -211,63 +233,89 @@ values ?s {<#{self.graph_id}>}
|
|
|
211
233
|
# Cache readonly entities list once
|
|
212
234
|
readonly_list = (Solis::Options.instance.get[:embedded_readonly] || []).map(&:to_s)
|
|
213
235
|
|
|
214
|
-
# Track entities to potentially delete
|
|
236
|
+
# Track entities to potentially delete (only used in PUT mode)
|
|
215
237
|
entities_to_check_for_deletion = {}
|
|
216
238
|
|
|
239
|
+
# First pass: collect all embedded entities for batched existence check
|
|
240
|
+
embedded_by_key = {}
|
|
217
241
|
attributes.each_pair do |key, value|
|
|
218
242
|
unless original_klass.class.metadata[:attributes][key][:node].nil?
|
|
219
243
|
value = [value] unless value.is_a?(Array)
|
|
244
|
+
embedded_by_key[key] = value.map do |sub_value|
|
|
245
|
+
self.class.graph.shape_as_model(original_klass.class.metadata[:attributes][key][:datatype].to_s).new(sub_value)
|
|
246
|
+
end
|
|
247
|
+
end
|
|
248
|
+
end
|
|
220
249
|
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
original_embedded = [original_embedded] unless original_embedded.nil? || original_embedded.is_a?(Array)
|
|
224
|
-
original_embedded ||= []
|
|
250
|
+
all_embedded = embedded_by_key.values.flatten
|
|
251
|
+
existing_ids = self.class.batch_exists?(sparql, all_embedded)
|
|
225
252
|
|
|
226
|
-
|
|
227
|
-
|
|
253
|
+
# Second pass: process embedded entities using batched results
|
|
254
|
+
embedded_by_key.each do |key, embedded_list|
|
|
255
|
+
value = attributes[key]
|
|
256
|
+
value = [value] unless value.is_a?(Array)
|
|
228
257
|
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
258
|
+
# Get original embedded entities for this attribute
|
|
259
|
+
original_embedded = original_klass.instance_variable_get("@#{key}")
|
|
260
|
+
original_embedded = [original_embedded] unless original_embedded.nil? || original_embedded.is_a?(Array)
|
|
261
|
+
original_embedded ||= []
|
|
232
262
|
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
new_ids << embedded.id if embedded.id
|
|
263
|
+
# Track original IDs
|
|
264
|
+
original_ids = original_embedded.map { |e| solis_model?(e) ? e.id : nil }.compact
|
|
236
265
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
266
|
+
# Build new array of embedded entities
|
|
267
|
+
new_embedded_values = []
|
|
268
|
+
new_ids = []
|
|
269
|
+
|
|
270
|
+
embedded_list.each do |embedded|
|
|
271
|
+
new_ids << embedded.id if embedded.id
|
|
272
|
+
entity_exists = existing_ids.include?(embedded.graph_id)
|
|
273
|
+
|
|
274
|
+
if readonly_entity?(embedded, readonly_list)
|
|
275
|
+
if entity_exists
|
|
276
|
+
new_embedded_values << embedded
|
|
245
277
|
else
|
|
246
|
-
#
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
278
|
+
Solis::LOGGER.warn("#{embedded.class.name} (id: #{embedded.id}) is readonly but does not exist in database. Skipping.")
|
|
279
|
+
end
|
|
280
|
+
else
|
|
281
|
+
if entity_exists
|
|
282
|
+
embedded_data = properties_to_hash(embedded)
|
|
283
|
+
embedded.update(embedded_data, validate_dependencies, false)
|
|
284
|
+
new_embedded_values << embedded
|
|
285
|
+
else
|
|
286
|
+
embedded_value = embedded.save(validate_dependencies, false)
|
|
287
|
+
new_embedded_values << embedded_value
|
|
255
288
|
end
|
|
256
289
|
end
|
|
290
|
+
end
|
|
291
|
+
|
|
292
|
+
if patch
|
|
293
|
+
# PATCH mode: merge new embedded entities into the original array.
|
|
294
|
+
# Keep original entities that were not mentioned in the update data.
|
|
295
|
+
unmentioned = original_embedded.select do |e|
|
|
296
|
+
solis_model?(e) && !new_ids.include?(e.id)
|
|
297
|
+
end
|
|
298
|
+
merged_values = unmentioned + new_embedded_values
|
|
299
|
+
else
|
|
300
|
+
# PUT mode: replace the entire array; detect orphans for deletion
|
|
301
|
+
merged_values = new_embedded_values
|
|
257
302
|
|
|
258
|
-
# Identify orphaned entities (in original but not in new)
|
|
259
|
-
# Note: Readonly entities will be filtered out in delete_orphaned_entities
|
|
260
303
|
orphaned_ids = original_ids - new_ids
|
|
261
304
|
unless orphaned_ids.empty?
|
|
262
305
|
orphaned_entities = original_embedded.select { |e| solis_model?(e) && orphaned_ids.include?(e.id) }
|
|
263
306
|
entities_to_check_for_deletion[key] = orphaned_entities
|
|
264
307
|
end
|
|
265
|
-
|
|
266
|
-
# Replace entire array with new values
|
|
267
|
-
maxcount = original_klass.class.metadata[:attributes][key][:maxcount]
|
|
268
|
-
value = maxcount && maxcount == 1 ? new_embedded_values.first : new_embedded_values
|
|
269
308
|
end
|
|
270
309
|
|
|
310
|
+
maxcount = original_klass.class.metadata[:attributes][key][:maxcount]
|
|
311
|
+
embedded_value = maxcount && maxcount == 1 ? merged_values.first : merged_values
|
|
312
|
+
updated_klass.instance_variable_set("@#{key}", embedded_value)
|
|
313
|
+
end
|
|
314
|
+
|
|
315
|
+
# Process non-embedded attributes
|
|
316
|
+
attributes.each_pair do |key, value|
|
|
317
|
+
next unless original_klass.class.metadata[:attributes][key][:node].nil?
|
|
318
|
+
|
|
271
319
|
updated_klass.instance_variable_set("@#{key}", value)
|
|
272
320
|
end
|
|
273
321
|
|
|
@@ -278,9 +326,11 @@ values ?s {<#{self.graph_id}>}
|
|
|
278
326
|
|
|
279
327
|
if Hashdiff.best_diff(properties_original_klass, properties_updated_klass).empty?
|
|
280
328
|
Solis::LOGGER.info("#{original_klass.class.name} unchanged, skipping")
|
|
281
|
-
data =
|
|
329
|
+
data = original_klass
|
|
282
330
|
else
|
|
283
|
-
|
|
331
|
+
# Pre-populate known entities to avoid re-fetching during graph building
|
|
332
|
+
known_entities = { id => original_klass }
|
|
333
|
+
delete_graph = as_graph(original_klass, false, known_entities)
|
|
284
334
|
where_graph = RDF::Graph.new(graph_name: RDF::URI("#{self.class.graph_name}#{tableized_class_name(self)}/#{id}"), data: RDF::Repository.new)
|
|
285
335
|
|
|
286
336
|
if id.is_a?(Array)
|
|
@@ -291,21 +341,22 @@ values ?s {<#{self.graph_id}>}
|
|
|
291
341
|
where_graph << [RDF::URI("#{self.class.graph_name}#{tableized_class_name(self)}/#{id}"), :p, :o]
|
|
292
342
|
end
|
|
293
343
|
|
|
294
|
-
insert_graph = as_graph(updated_klass, true)
|
|
344
|
+
insert_graph = as_graph(updated_klass, true, known_entities)
|
|
295
345
|
|
|
296
346
|
delete_insert_query = SPARQL::Client::Update::DeleteInsert.new(delete_graph, insert_graph, where_graph, graph: insert_graph.name).to_s
|
|
297
347
|
delete_insert_query.gsub!('_:p', '?p')
|
|
298
348
|
|
|
299
|
-
|
|
349
|
+
sparql.query(delete_insert_query)
|
|
300
350
|
|
|
351
|
+
# Verify the update succeeded by re-fetching; fallback to insert if needed
|
|
301
352
|
data = self.query.filter({ filters: { id: [id] } }).find_all.map { |m| m }&.first
|
|
302
353
|
if data.nil?
|
|
303
354
|
sparql.insert_data(insert_graph, graph: insert_graph.name)
|
|
304
|
-
data =
|
|
355
|
+
data = updated_klass
|
|
305
356
|
end
|
|
306
357
|
|
|
307
|
-
# Delete orphaned entities after successful update
|
|
308
|
-
delete_orphaned_entities(entities_to_check_for_deletion, sparql)
|
|
358
|
+
# Delete orphaned entities after successful update (PUT mode only)
|
|
359
|
+
delete_orphaned_entities(entities_to_check_for_deletion, sparql) unless patch
|
|
309
360
|
end
|
|
310
361
|
|
|
311
362
|
after_update_proc&.call(updated_klass, data)
|
|
@@ -333,18 +384,24 @@ values ?s {<#{self.graph_id}>}
|
|
|
333
384
|
sparql.query("ASK WHERE { <#{self.graph_id}> ?p ?o }")
|
|
334
385
|
end
|
|
335
386
|
|
|
387
|
+
# Check existence of multiple entities in a single SPARQL query
|
|
388
|
+
# Returns a Set of graph_ids that exist
|
|
389
|
+
def self.batch_exists?(sparql, entities)
|
|
390
|
+
return Set.new if entities.empty?
|
|
391
|
+
return Set.new([entities.first.graph_id]) if entities.size == 1 && entities.first.exists?(sparql)
|
|
392
|
+
return Set.new if entities.size == 1
|
|
393
|
+
|
|
394
|
+
values = entities.map { |e| "<#{e.graph_id}>" }.join(' ')
|
|
395
|
+
query = "SELECT DISTINCT ?s WHERE { VALUES ?s { #{values} } . ?s ?p ?o }"
|
|
396
|
+
results = sparql.query(query)
|
|
397
|
+
Set.new(results.map { |r| r[:s].to_s })
|
|
398
|
+
end
|
|
399
|
+
|
|
336
400
|
def self.make_id_for(model)
|
|
337
|
-
raise "I need a SPARQL endpoint" if self.sparql_endpoint.nil?
|
|
338
|
-
sparql = Solis::Store::Sparql::Client.new(self.sparql_endpoint)
|
|
339
401
|
id = model.instance_variable_get("@id")
|
|
340
402
|
if id.nil? || (id.is_a?(String) && id&.empty?)
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
while id.nil? || sparql.query("ASK WHERE { ?s <#{self.graph_name}id> \"#{id}\" }")
|
|
344
|
-
id = SecureRandom.uuid
|
|
345
|
-
id_retries += 1
|
|
346
|
-
end
|
|
347
|
-
LOGGER.info("ID(#{id}) generated for #{self.name} in #{id_retries} retries") if ConfigFile[:debug]
|
|
403
|
+
id = SecureRandom.uuid
|
|
404
|
+
LOGGER.info("ID(#{id}) generated for #{self.name}") if ConfigFile[:debug]
|
|
348
405
|
model.instance_variable_set("@id", id)
|
|
349
406
|
elsif id.to_s =~ /^https?:\/\//
|
|
350
407
|
id = id.to_s.split('/').last
|
|
@@ -497,6 +554,13 @@ values ?s {<#{self.graph_id}>}
|
|
|
497
554
|
(entity.class.ancestors.map(&:to_s) & readonly_list).any?
|
|
498
555
|
end
|
|
499
556
|
|
|
557
|
+
# Helper method to check if an entity is a top-level entity (has its own shape definition).
|
|
558
|
+
# Top-level entities are independently addressable and should not be cascade-deleted
|
|
559
|
+
# when unlinked from a parent, unless explicitly opted in via embedded_delete config.
|
|
560
|
+
def top_level_entity?(entity)
|
|
561
|
+
self.class.graph.shape?(entity.class.name)
|
|
562
|
+
end
|
|
563
|
+
|
|
500
564
|
# Helper method to get tableized class name
|
|
501
565
|
def tableized_class_name(obj)
|
|
502
566
|
obj.class.name.tableize
|
|
@@ -514,50 +578,85 @@ values ?s {<#{self.graph_id}>}
|
|
|
514
578
|
RDF::URI("#{self.class.graph_name}#{class_name.tableize}/#{id}")
|
|
515
579
|
end
|
|
516
580
|
|
|
517
|
-
# Delete orphaned entities that are no longer referenced
|
|
581
|
+
# Delete orphaned entities that are no longer referenced.
|
|
582
|
+
#
|
|
583
|
+
# Decision logic (in order of precedence):
|
|
584
|
+
# 1. embedded_readonly → never delete (code tables)
|
|
585
|
+
# 2. Top-level entity (has own shape) + NOT in embedded_delete → unlink only, don't delete
|
|
586
|
+
# 3. Top-level entity + listed in embedded_delete → delete (opt-in override)
|
|
587
|
+
# 4. Still referenced by other entities (batch_referenced?) → never delete (safety net)
|
|
518
588
|
def delete_orphaned_entities(entities_to_check, sparql)
|
|
519
589
|
return if entities_to_check.nil? || entities_to_check.empty?
|
|
520
590
|
|
|
521
591
|
readonly_list = (Solis::Options.instance.get[:embedded_readonly] || []).map(&:to_s)
|
|
592
|
+
delete_list = (Solis::Options.instance.get[:embedded_delete] || []).map(&:to_s)
|
|
522
593
|
|
|
523
|
-
|
|
594
|
+
# Collect all deletable orphans
|
|
595
|
+
deletable_orphans = []
|
|
596
|
+
entities_to_check.each do |_key, orphaned_entities|
|
|
524
597
|
next if orphaned_entities.nil?
|
|
525
|
-
|
|
526
598
|
orphaned_entities.each do |orphaned_entity|
|
|
527
599
|
next unless solis_model?(orphaned_entity)
|
|
528
600
|
|
|
529
|
-
#
|
|
601
|
+
# 1. Never delete readonly entities (code tables)
|
|
530
602
|
if readonly_entity?(orphaned_entity, readonly_list)
|
|
531
603
|
Solis::LOGGER.info("#{orphaned_entity.class.name} (id: #{orphaned_entity.id}) is in embedded_readonly list. Skipping deletion.")
|
|
532
604
|
next
|
|
533
605
|
end
|
|
534
606
|
|
|
535
|
-
#
|
|
536
|
-
if
|
|
537
|
-
|
|
538
|
-
|
|
607
|
+
# 2. Top-level entities are never auto-deleted unless opted in via embedded_delete
|
|
608
|
+
if top_level_entity?(orphaned_entity)
|
|
609
|
+
explicitly_deletable = (orphaned_entity.class.ancestors.map(&:to_s) & delete_list).any?
|
|
610
|
+
unless explicitly_deletable
|
|
611
|
+
Solis::LOGGER.info("#{orphaned_entity.class.name} (id: #{orphaned_entity.id}) is a top-level entity. Skipping deletion (unlink only).")
|
|
612
|
+
next
|
|
613
|
+
end
|
|
539
614
|
end
|
|
540
615
|
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
616
|
+
deletable_orphans << orphaned_entity
|
|
617
|
+
end
|
|
618
|
+
end
|
|
619
|
+
|
|
620
|
+
return if deletable_orphans.empty?
|
|
621
|
+
|
|
622
|
+
# 4. Safety net: batch check which orphans are still referenced elsewhere
|
|
623
|
+
referenced_ids = batch_referenced?(sparql, deletable_orphans)
|
|
624
|
+
|
|
625
|
+
deletable_orphans.each do |orphaned_entity|
|
|
626
|
+
if referenced_ids.include?(orphaned_entity.graph_id)
|
|
627
|
+
Solis::LOGGER.info("#{orphaned_entity.class.name} (id: #{orphaned_entity.id}) is still referenced elsewhere. Skipping deletion.")
|
|
628
|
+
next
|
|
629
|
+
end
|
|
630
|
+
|
|
631
|
+
begin
|
|
632
|
+
Solis::LOGGER.info("Deleting orphaned entity: #{orphaned_entity.class.name} (id: #{orphaned_entity.id})")
|
|
633
|
+
orphaned_entity.destroy
|
|
634
|
+
rescue StandardError => e
|
|
635
|
+
Solis::LOGGER.error("Failed to delete orphaned entity #{orphaned_entity.class.name} (id: #{orphaned_entity.id}): #{e.message}")
|
|
548
636
|
end
|
|
549
637
|
end
|
|
550
638
|
end
|
|
551
639
|
|
|
552
|
-
|
|
640
|
+
# Batch check which entities are still referenced by other entities
|
|
641
|
+
# Returns a Set of graph_ids that are referenced
|
|
642
|
+
def batch_referenced?(sparql, entities)
|
|
643
|
+
return Set.new if entities.empty?
|
|
644
|
+
|
|
645
|
+
values = entities.map { |e| "<#{e.graph_id}>" }.join(' ')
|
|
646
|
+
query = "SELECT DISTINCT ?o WHERE { VALUES ?o { #{values} } . ?s ?p ?o . FILTER(!CONTAINS(STR(?s), 'audit') && !CONTAINS(STR(?p), 'audit')) }"
|
|
647
|
+
results = sparql.query(query)
|
|
648
|
+
Set.new(results.map { |r| r[:o].to_s })
|
|
649
|
+
end
|
|
650
|
+
|
|
651
|
+
def as_graph(klass = self, resolve_all = true, known_entities = {})
|
|
553
652
|
graph = RDF::Graph.new
|
|
554
653
|
graph.name = RDF::URI(self.class.graph_name)
|
|
555
|
-
id = build_ttl_objekt(graph, klass, [], resolve_all)
|
|
654
|
+
id = build_ttl_objekt(graph, klass, [], resolve_all, known_entities)
|
|
556
655
|
|
|
557
656
|
graph
|
|
558
657
|
end
|
|
559
658
|
|
|
560
|
-
def build_ttl_objekt(graph, klass, hierarchy = [], resolve_all = true)
|
|
659
|
+
def build_ttl_objekt(graph, klass, hierarchy = [], resolve_all = true, known_entities = {})
|
|
561
660
|
hierarchy.push("#{klass.class.name}(#{klass.instance_variables.include?(:@id) ? klass.instance_variable_get("@id") : ''})")
|
|
562
661
|
|
|
563
662
|
graph_name = self.class.graph_name
|
|
@@ -568,8 +667,11 @@ values ?s {<#{self.graph_id}>}
|
|
|
568
667
|
|
|
569
668
|
graph << [id, RDF::RDFV.type, klass_metadata[:target_class]]
|
|
570
669
|
|
|
571
|
-
#
|
|
572
|
-
original_klass =
|
|
670
|
+
# Use cached entity if available, otherwise query the store
|
|
671
|
+
original_klass = known_entities[uuid]
|
|
672
|
+
if original_klass.nil?
|
|
673
|
+
original_klass = klass.query.filter({ filters: { id: [uuid] } }).find_all { |f| f.id == uuid }.first || nil
|
|
674
|
+
end
|
|
573
675
|
|
|
574
676
|
if original_klass.nil?
|
|
575
677
|
original_klass = klass
|
|
@@ -583,8 +685,11 @@ values ?s {<#{self.graph_id}>}
|
|
|
583
685
|
end
|
|
584
686
|
end
|
|
585
687
|
|
|
688
|
+
# Cache entity for potential reuse in recursive calls
|
|
689
|
+
known_entities[uuid] = original_klass
|
|
690
|
+
|
|
586
691
|
begin
|
|
587
|
-
make_graph(graph, hierarchy, id, original_klass, klass_metadata, resolve_all)
|
|
692
|
+
make_graph(graph, hierarchy, id, original_klass, klass_metadata, resolve_all, known_entities)
|
|
588
693
|
rescue => e
|
|
589
694
|
Solis::LOGGER.error(e.message)
|
|
590
695
|
raise e
|
|
@@ -594,14 +699,19 @@ values ?s {<#{self.graph_id}>}
|
|
|
594
699
|
id
|
|
595
700
|
end
|
|
596
701
|
|
|
597
|
-
def make_graph(graph, hierarchy, id, klass, klass_metadata, resolve_all)
|
|
702
|
+
def make_graph(graph, hierarchy, id, klass, klass_metadata, resolve_all, known_entities = {})
|
|
598
703
|
klass_metadata[:attributes].each do |attribute, metadata|
|
|
599
704
|
data = klass.instance_variable_get("@#{attribute}")
|
|
600
705
|
|
|
601
706
|
if data.nil? && metadata.key?(:mincount) && (metadata[:mincount].nil? || metadata[:mincount] > 0) && graph.query(RDF::Query.new({ attribute.to_sym => { RDF.type => metadata[:node] } })).size == 0
|
|
602
707
|
if data.nil?
|
|
603
708
|
uuid = id.value.split('/').last
|
|
604
|
-
|
|
709
|
+
# Use cached entity if available
|
|
710
|
+
original_klass = known_entities[uuid]
|
|
711
|
+
if original_klass.nil?
|
|
712
|
+
original_klass = klass.query.filter({ filters: { id: [uuid] } }).find_all { |f| f.id == uuid }.first || nil
|
|
713
|
+
known_entities[uuid] = original_klass if original_klass
|
|
714
|
+
end
|
|
605
715
|
unless original_klass.nil?
|
|
606
716
|
klass = original_klass
|
|
607
717
|
data = klass.instance_variable_get("@#{attribute}")
|
|
@@ -642,10 +752,10 @@ values ?s {<#{self.graph_id}>}
|
|
|
642
752
|
if self.class.graph.shape_as_model(d.class.name).metadata[:attributes].select { |_, v| v[:node_kind].is_a?(RDF::URI) }.size > 0 &&
|
|
643
753
|
hierarchy.select { |s| s =~ /^#{d.class.name}/ }.size == 0
|
|
644
754
|
internal_resolve = false
|
|
645
|
-
d = build_ttl_objekt(graph, d, hierarchy, internal_resolve)
|
|
755
|
+
d = build_ttl_objekt(graph, d, hierarchy, internal_resolve, known_entities)
|
|
646
756
|
elsif self.class.graph.shape_as_model(d.class.name) && hierarchy.select { |s| s =~ /^#{d.class.name}/ }.size == 0
|
|
647
757
|
internal_resolve = false
|
|
648
|
-
d = build_ttl_objekt(graph, d, hierarchy, internal_resolve)
|
|
758
|
+
d = build_ttl_objekt(graph, d, hierarchy, internal_resolve, known_entities)
|
|
649
759
|
else
|
|
650
760
|
d = "#{klass.class.graph_name}#{d.class.name.tableize}/#{d.id}"
|
|
651
761
|
end
|
|
@@ -764,21 +874,21 @@ values ?s {<#{self.graph_id}>}
|
|
|
764
874
|
model_instance = model.new(d) if model_instance.nil?
|
|
765
875
|
|
|
766
876
|
if resolve_all
|
|
767
|
-
d = build_ttl_objekt(graph, model_instance, hierarchy, false)
|
|
877
|
+
d = build_ttl_objekt(graph, model_instance, hierarchy, false, known_entities)
|
|
768
878
|
else
|
|
769
|
-
real_model = model_instance.query.filter({ filters: { id: model_instance.id } }).find_all { |f| f.id == model_instance.id }&.first
|
|
879
|
+
real_model = known_entities[model_instance.id] || model_instance.query.filter({ filters: { id: model_instance.id } }).find_all { |f| f.id == model_instance.id }&.first
|
|
770
880
|
d = RDF::URI("#{self.class.graph_name}#{real_model ? real_model.class.name.tableize : model_instance.class.name.tableize}/#{model_instance.id}")
|
|
771
881
|
end
|
|
772
882
|
elsif model && d.is_a?(model)
|
|
773
883
|
if resolve_all
|
|
774
884
|
if parent_model
|
|
775
885
|
model_instance = parent_model.new({ id: d.id })
|
|
776
|
-
d = build_ttl_objekt(graph, model_instance, hierarchy, false)
|
|
886
|
+
d = build_ttl_objekt(graph, model_instance, hierarchy, false, known_entities)
|
|
777
887
|
else
|
|
778
|
-
d = build_ttl_objekt(graph, d, hierarchy, false)
|
|
888
|
+
d = build_ttl_objekt(graph, d, hierarchy, false, known_entities)
|
|
779
889
|
end
|
|
780
890
|
else
|
|
781
|
-
real_model = model.new.query.filter({ filters: { id: d.id } }).find_all { |f| f.id == d.id }&.first
|
|
891
|
+
real_model = known_entities[d.id] || model.new.query.filter({ filters: { id: d.id } }).find_all { |f| f.id == d.id }&.first
|
|
782
892
|
d = RDF::URI("#{self.class.graph_name}#{real_model ? real_model.class.name.tableize : model.name.tableize}/#{d.id}")
|
|
783
893
|
end
|
|
784
894
|
else
|
|
@@ -27,12 +27,18 @@ module Solis
|
|
|
27
27
|
end
|
|
28
28
|
|
|
29
29
|
def up?
|
|
30
|
-
|
|
30
|
+
now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
31
|
+
if @up_checked_at && (now - @up_checked_at) < 30
|
|
32
|
+
return @up_result
|
|
33
|
+
end
|
|
34
|
+
@up_result = nil
|
|
31
35
|
@pool.with do |c|
|
|
32
|
-
|
|
36
|
+
@up_result = c.query("ASK WHERE { ?s ?p ?o }")
|
|
33
37
|
end
|
|
34
|
-
|
|
38
|
+
@up_checked_at = now
|
|
39
|
+
@up_result
|
|
35
40
|
rescue HTTP::Error => e
|
|
41
|
+
@up_checked_at = nil
|
|
36
42
|
return false
|
|
37
43
|
end
|
|
38
44
|
|
data/lib/solis/version.rb
CHANGED