iris-vector-graph 4.4.1__tar.gz → 4.5.0__tar.gz

{iris_vector_graph-4.4.1 → iris_vector_graph-4.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: iris-vector-graph
-Version: 4.4.1
+Version: 4.5.0
 Summary: Native graph database layer for InterSystems IRIS — openCypher, hybrid vector search, temporal property graph, RDF export, SHACL validation
 Project-URL: Homepage, https://github.com/intersystems-community/iris-vector-graph
 Project-URL: Documentation, https://github.com/intersystems-community/iris-vector-graph/tree/main/docs
@@ -108,7 +108,7 @@ Description-Content-Type: text/markdown
 
 [![PyPI](https://img.shields.io/pypi/v/iris-vector-graph)](https://pypi.org/project/iris-vector-graph/)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
-[![IRIS 2024.1+](https://img.shields.io/badge/IRIS-2024.1+-purple.svg)](https://www.intersystems.com/products/intersystems-iris/)
+[![IRIS 2025.1+](https://img.shields.io/badge/IRIS-2025.1+-purple.svg)](https://www.intersystems.com/products/intersystems-iris/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
 
 ---
@@ -204,7 +204,7 @@ Hardware: M3 Ultra, Community IRIS 2026.1, ARM64 Docker.
 Against Neo4j and Memgraph on the LDBC Social Network Benchmark at scale factor 10 (AWS `r7g.8xlarge`,
 2026-10-23, native Cypher with compiled plans), IVG's median time over 39 queries is 0.90x Neo4j's and 0.57x
 Memgraph's (Memgraph from the previous SF10 run on the same instance type)
-([report](docs/performance/alt-bench/REPORT_TEMPLATE.md#sf10-after-spec-257-aws-2026-10-23)).
+([report](docs/performance/ALT_VS_BASELINE.md#sf10-after-spec-257-aws-2026-10-23)).
 
 ### Algorithm comparison (vs Neo4j GDS and networkx)
 
@@ -264,7 +264,7 @@ and [docs/performance/GRAPH_ALGORITHMS.md](docs/performance/GRAPH_ALGORITHMS.md)
 └─────────────────────────────────────────────────────────────────────┘
 ```
 
-Full schema and ObjectScript class reference: [docs/architecture/ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md).
+The Rust tier is used for an algorithm only when the loaded library passes IVG's reader gate for it; as of arno-callout 1.2.0, pagerank, ppr, wcc, cdlp and subgraph do not, and answer from ObjectScript ([KNOWN_ISSUES](docs/KNOWN_ISSUES.md), "Arno callout defects"). Full schema and ObjectScript class reference: [docs/architecture/ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md).
 
 ---
 

{iris_vector_graph-4.4.1 → iris_vector_graph-4.5.0}/README.md

@@ -6,7 +6,7 @@
 
 [![PyPI](https://img.shields.io/pypi/v/iris-vector-graph)](https://pypi.org/project/iris-vector-graph/)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
-[![IRIS 2024.1+](https://img.shields.io/badge/IRIS-2024.1+-purple.svg)](https://www.intersystems.com/products/intersystems-iris/)
+[![IRIS 2025.1+](https://img.shields.io/badge/IRIS-2025.1+-purple.svg)](https://www.intersystems.com/products/intersystems-iris/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
 
 ---
@@ -102,7 +102,7 @@ Hardware: M3 Ultra, Community IRIS 2026.1, ARM64 Docker.
 Against Neo4j and Memgraph on the LDBC Social Network Benchmark at scale factor 10 (AWS `r7g.8xlarge`,
 2026-10-23, native Cypher with compiled plans), IVG's median time over 39 queries is 0.90x Neo4j's and 0.57x
 Memgraph's (Memgraph from the previous SF10 run on the same instance type)
-([report](docs/performance/alt-bench/REPORT_TEMPLATE.md#sf10-after-spec-257-aws-2026-10-23)).
+([report](docs/performance/ALT_VS_BASELINE.md#sf10-after-spec-257-aws-2026-10-23)).
 
 ### Algorithm comparison (vs Neo4j GDS and networkx)
 
@@ -162,7 +162,7 @@ and [docs/performance/GRAPH_ALGORITHMS.md](docs/performance/GRAPH_ALGORITHMS.md)
 └─────────────────────────────────────────────────────────────────────┘
 ```
 
-Full schema and ObjectScript class reference: [docs/architecture/ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md).
+The Rust tier is used for an algorithm only when the loaded library passes IVG's reader gate for it; as of arno-callout 1.2.0, pagerank, ppr, wcc, cdlp and subgraph do not, and answer from ObjectScript ([KNOWN_ISSUES](docs/KNOWN_ISSUES.md), "Arno callout defects"). Full schema and ObjectScript class reference: [docs/architecture/ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md).
 
 ---
 

{iris_vector_graph-4.4.1 → iris_vector_graph-4.5.0}/iris_src/src/Graph/KG/Adjacency.cls

@@ -528,7 +528,18 @@ ClassMethod Indexed() As %Boolean
 /// key (0 for the default graph). With pTrustIndex the SQL is skipped when ^KG already has the triple,
 /// which the index keeps in step with the rows: further timestamps of one triple cost a $Data. The
 /// upgrade backfill passes 0, because there ^KG may hold a pre-ADR-0009 shadow with no row behind it.
-ClassMethod AssertTemporalRow(pGKey, pSrc As %String, pPred As %String, pDst As %String, pWeight As %String = "1.0", pTrustIndex As %Boolean = 1) As %Status
+///
+/// pNodes (by reference) is the caller's memory of the endpoint nodes it has already seen in this graph:
+/// pNodes(id) set means "the row exists, do not insert it again". A batch that writes many edges from a few
+/// sources passes one array and pays for each source's INSERT once. The memory can be stale, because a
+/// purge that retires bare nodes (Graph.KG.Eraser.RetireBareNodes) or any other delete can remove a node in
+/// between; the edge INSERT then fails its foreign key (SQLCODE -121), the entries are dropped and the
+/// nodes and the edge are written again, up to three times. Rows and ^KG are the same as without pNodes.
+///
+/// Embedded SQL: measured at 55 us for the three statements against 129 us as three %ExecDirect (a dynamic
+/// statement is prepared on every call), with the same rows. The remaining ~50 us is the foreign keys and the
+/// two indexes the edge row maintains.
+ClassMethod AssertTemporalRow(pGKey, pSrc As %String, pPred As %String, pDst As %String, pWeight As %String = "1.0", pTrustIndex As %Boolean = 1, ByRef pNodes) As %Status
 {
 	If (pTrustIndex) && ($Data(^KG("out", pGKey, pSrc, pPred, pDst))) {
 		Quit $$$OK
@@ -537,24 +548,35 @@ ClassMethod AssertTemporalRow(pGKey, pSrc As %String, pPred As %String, pDst As
 	Try {
 		// $Char(0) is SQL's '': a bound "" arrives as NULL, which graph_id refuses (the default graph is '')
 		Set tGraphId = $Select(pGKey = 0: $Char(0), 1: pGKey)
-		Set tNodeSQL = "INSERT INTO Graph_KG.nodes (node_id, graph_id) SELECT ?, ? WHERE NOT EXISTS "
-		Set tNodeSQL = tNodeSQL_"(SELECT 1 FROM Graph_KG.nodes WHERE node_id = ? AND COALESCE(graph_id, '') = COALESCE(?, ''))"
-		For tNode = pSrc, pDst {
-			Set tRS = ##class(%SQL.Statement).%ExecDirect(, tNodeSQL, tNode, tGraphId, tNode, tGraphId)
-			If tRS.%SQLCODE < 0 {
-				$$$ThrowStatus($$$ERROR($$$GeneralError, "AssertTemporalRow node "_tNode_": SQLCODE "_tRS.%SQLCODE_" "_tRS.%Message))
-			}
-		}
 		Set tQuals = ##class(%DynamicObject).%New()
 		Do tQuals.%Set("temporal", 1, "boolean")
 		If (pWeight '= "") && (+pWeight '= 1) {
 			Do tQuals.%Set("weight", +pWeight, "number")
 		}
-		Set tEdgeSQL = "INSERT INTO Graph_KG.rdf_edges (s, p, o_id, graph_id, qualifiers) SELECT ?, ?, ?, ?, ? WHERE NOT EXISTS "
-		Set tEdgeSQL = tEdgeSQL_"(SELECT 1 FROM Graph_KG.rdf_edges WHERE s = ? AND p = ? AND o_id = ? AND COALESCE(graph_id, '') = COALESCE(?, ''))"
-		Set tRS = ##class(%SQL.Statement).%ExecDirect(, tEdgeSQL, pSrc, pPred, pDst, tGraphId, tQuals.%ToJSON(), pSrc, pPred, pDst, tGraphId)
-		If tRS.%SQLCODE < 0 {
-			$$$ThrowStatus($$$ERROR($$$GeneralError, "AssertTemporalRow edge: SQLCODE "_tRS.%SQLCODE_" "_tRS.%Message))
+		Set tQ = tQuals.%ToJSON()
+		Set tS = pSrc, tP = pPred, tO = pDst, tG = tGraphId
+		For tTry = 1:1:3 {
+			For tNode = pSrc, pDst {
+				If $Data(pNodes(tNode)) {
+					Continue
+				}
+				Set tN = tNode
+				&sql(INSERT INTO Graph_KG.nodes (node_id, graph_id) SELECT :tN, :tG WHERE NOT EXISTS (SELECT 1 FROM Graph_KG.nodes WHERE node_id = :tN AND COALESCE(graph_id, '') = COALESCE(:tG, '')))
+				If SQLCODE < 0 {
+					$$$ThrowStatus($$$ERROR($$$GeneralError, "AssertTemporalRow node "_tNode_": SQLCODE "_SQLCODE_" "_$Get(%msg)))
+				}
+				Set pNodes(tNode) = ""
+			}
+			&sql(INSERT INTO Graph_KG.rdf_edges (s, p, o_id, graph_id, qualifiers) SELECT :tS, :tP, :tO, :tG, :tQ WHERE NOT EXISTS (SELECT 1 FROM Graph_KG.rdf_edges WHERE s = :tS AND p = :tP AND o_id = :tO AND COALESCE(graph_id, '') = COALESCE(:tG, '')))
+			If (SQLCODE = -121) && (tTry < 3) {
+				// a node row this call relied on is gone: it was remembered, or deleted since it was written
+				Kill pNodes(pSrc), pNodes(pDst)
+				Continue
+			}
+			If SQLCODE < 0 {
+				$$$ThrowStatus($$$ERROR($$$GeneralError, "AssertTemporalRow edge: SQLCODE "_SQLCODE_" "_$Get(%msg)))
+			}
+			Quit
 		}
 	} Catch tEx {
 		Set tSC = tEx.AsStatus()

{iris_vector_graph-4.4.1 → iris_vector_graph-4.5.0}/iris_src/src/Graph/KG/Eraser.cls

@@ -195,6 +195,18 @@ ClassMethod EraseGraph(pGraph As %String = "") As %Integer
 			WHERE COALESCE(graph_id, '') = COALESCE(:tName, ''))
 		If (SQLCODE '= 0) && (SQLCODE '= 100) { $$$ThrowStatus(..SqlErr("kg_EdgeEmbeddings", SQLCODE)) }
 
+		// Whether ^NKG can hold any edge of this graph, decided HERE and not after the
+		// delete below. Since spec 252 the functional index on rdf_edges clears
+		// ^KG("out", graph) row by row as the rows go, so a read of ^KG afterwards finds
+		// nothing for a graph whose edges are rows. That read (it was
+		// $Data(^KG("out", key))) was always 0, InvalidateDerived never dropped ^NKG, and
+		// the sketches and edgeCount of the erased edges stayed behind: a sketch cannot
+		// have an element taken out, so CountDistinctKHop went on counting the far ends
+		// of edges that no longer exist. The rows and ^KG("out") are both still whole at
+		// this point, and this is the last statement before the rows go: the window left
+		// is the delete itself.
+		Set tNkgAtRisk = ..GraphEdgesInNKG(tKey, tName)
+
 		&sql(DELETE FROM Graph_KG.rdf_edges
 			WHERE COALESCE(graph_id, '') = COALESCE(:tName, ''))
 		If (SQLCODE '= 0) && (SQLCODE '= 100) { $$$ThrowStatus(..SqlErr("rdf_edges", SQLCODE)) }
@@ -210,7 +222,6 @@ ClassMethod EraseGraph(pGraph As %String = "") As %Integer
 		// One Kill per store, because the graph key is the FIRST subscript in every
 		// one of them. That is the whole return on ADR-0001: erasing a graph from
 		// ^KG is thirteen Kills rather than thirteen walks.
-		Set tHadEdges = $Data(^KG("out", tKey))
 		// out, in and the degree cache (gen, degc) belong to Graph.KG.Adjacency (spec 248),
 		// which kills them under the writer lock inside this transaction.
 		Set tSC = ##class(Graph.KG.Adjacency).RemoveGraph(tKey)
@@ -224,6 +235,9 @@ ClassMethod EraseGraph(pGraph As %String = "") As %Integer
 		Kill ^KG("deg2p_exact", tKey)
 		// Spec 234: an erased graph is no longer a multigraph.
 		Kill ^IVG.GraphMode(tKey)
+		// ADR-0015: ^IVG.TemporalStructural(tKey, predicate), which predicates are temporal-only, is
+		// configuration and not content, so it is NOT killed here (as the registration in fhir_graphs
+		// outlives an erase): a re-ingest after the erase must stay temporal-only. EraseAll empties it.
 		// Spec 236: cached algorithm answers derived from the erased content.
 		Kill ^IVG.AlgoCache(tKey)
 		// ^KG("labelset") is content-addressed interning of temporal label sets and
@@ -231,7 +245,7 @@ ClassMethod EraseGraph(pGraph As %String = "") As %Integer
 		// purge touches it. A surviving entry is a hash with no referent, not
 		// content, so it is left alone here and emptied only by EraseAll.
 
-		Do ..InvalidateDerived(tHadEdges)
+		Do ..InvalidateDerived(tNkgAtRisk)
 		Do ..EraseLedgerFor(tName)
 
 		TCOMMIT
@@ -339,6 +353,11 @@ ClassMethod EraseNodes(pGraph As %String = "", pPrefix As %String = "") As %Inte
 /// every id named, present or not, so adjacency left behind by an earlier
 /// SQL-only delete is removed too.
 ///
+/// ^NKG is dropped when edges went AND an erased id (or both ends of a removed edge) is
+/// interned in it. A purge of ids that were never interned leaves ^NKG standing: none
+/// of their edges can be in it (NKGMayHoldAny). ^KG("__version") and the cached
+/// algorithm answers are invalidated either way.
+///
 /// Returns the number of Graph_KG.nodes rows removed. One transaction for the
 /// whole list: the caller bounds the list, not this method.
 ClassMethod EraseNodeIds(pGraph As %String = "", pIdsJson As %String = "[]") As %Integer
@@ -377,6 +396,10 @@ ClassMethod EraseEdges(pGraph As %String = "", pEdgesJson As %String = "[]") As
 	Do ..CollectOrphanRoutes(.tRoute)
 
 	Set tRemoved = 0
+	// 1 once an edge this call names was in ^KG("out") before it was touched. The row count
+	// alone cannot say: an adjacency entry with no row behind it removes nothing from the
+	// table, yet its ^NKG sketch and edgeCount are exactly as stale as a row's.
+	Set tHadAdj = 0
 	Try {
 		TSTART
 		Set tIter = tArr.%GetIterator()
@@ -400,6 +423,10 @@ ClassMethod EraseEdges(pGraph As %String = "", pEdgesJson As %String = "[]") As
 				Set tEdge(tEid) = ""
 			}
 			&sql(CLOSE ivgEraseEdgeRows)
+			// Read before EraseEdgeRow: the per-row index clears the entry as the row goes.
+			If $Data(^KG("out", tKey, tS, tP, tO)) {
+				Set tHadAdj = 1
+			}
 			Set tEid = ""
 			For {
 				Set tEid = $Order(tEdge(tEid))
@@ -410,7 +437,7 @@ ClassMethod EraseEdges(pGraph As %String = "", pEdgesJson As %String = "[]") As
 			// when the entry is already gone.
 			Do ##class(Graph.KG.Adjacency).Remove(tKey, tS, tP, tO)
 		}
-		If tRemoved > 0 {
+		If (tRemoved > 0) || tHadAdj {
 			Kill ^KG("deg2p", tKey)
 			Kill ^KG("deg2p_exact", tKey)
 			Do ..InvalidateDerived(1)
@@ -437,6 +464,9 @@ ClassMethod EraseIdSet(pKey, pName As %String) As %Integer [ Private ]
 
 	Set tRemoved = 0
 	Set tHadEdges = 0
+	// 1 once anything this erase removed could be in ^NKG (see NKGMayHoldAny). Starts
+	// at 0: the answer is "no" until a node or an edge proves otherwise.
+	Set tNkgAtRisk = 0
 	Try {
 		TSTART
 		Set tId = ""
@@ -478,6 +508,13 @@ ClassMethod EraseIdSet(pKey, pName As %String) As %Integer [ Private ]
 			For {
 				Set tEid = $Order(tEdge(tEid))
 				Quit:tEid=""
+				// An edge is in ^NKG only if both its ends are interned (every ^NKG writer
+				// interns both). Looked at on the row's own spelling of the ends, not on tId:
+				// a legacy column with a case-folding collation can match this id to a row
+				// that spells it differently.
+				If 'tNkgAtRisk && ..NodeInterned($ListGet(tEdge(tEid), 1)) && ..NodeInterned($ListGet(tEdge(tEid), 3)) {
+					Set tNkgAtRisk = 1
+				}
 				Do ..EraseEdgeRow(tEid, tEdge(tEid), pKey, pName, .tRoute)
 				Set tHadEdges = 1
 			}
@@ -559,7 +596,14 @@ ClassMethod EraseIdSet(pKey, pName As %String) As %Integer [ Private ]
 			Kill ^KG("deg2p_exact", pKey)
 		}
 		If (tRemoved > 0) || tHadEdges {
-			Do ..InvalidateDerived(tHadEdges)
+			// ^NKG is dropped only for an erase that touched edges AND could have changed
+			// it. The ids are looked at here, after every removal above and not before: the
+			// interning is permanent, so a writer that interned an id while this loop ran
+			// is seen, and the window left is the few instructions to the TCOMMIT.
+			If tHadEdges && 'tNkgAtRisk {
+				Set tNkgAtRisk = ..NKGMayHoldAny()
+			}
+			Do ..InvalidateDerived(tHadEdges && tNkgAtRisk)
 		}
 		TCOMMIT
 	} Catch tEx {
@@ -737,6 +781,7 @@ ClassMethod EraseAll() As %Integer
 		Do ..InvalidateDerived(1)
 		Kill ^ArnoKG
 		Kill ^IVG.GraphMode
+		Kill ^IVG.TemporalStructural
 		Kill ^IVG.AlgoCache
 		Kill ^IVG.Ledger("stmt")
 		Kill ^IVG.Ledger("tuple")
@@ -775,8 +820,137 @@ ClassMethod EraseAll() As %Integer
 	Quit tRemoved
 }
 
+/// 1 when ^NKG may hold an edge of graph pName (index key pKey), 0 when it cannot.
+///
+/// The same argument as NKGMayHoldAny, put to a whole graph: every writer of an ^NKG edge
+/// interns BOTH its ends, so an edge is in ^NKG only if both ends have an
+/// ^NKG("$NI", id) entry. A graph none of whose edges has two interned ends has none in
+/// ^NKG, nor any count or sketch built from one, and erasing it leaves ^NKG as correct as
+/// it was. ^NKG has no graph dimension, so an edge the graph shares with a survivor is
+/// found here too (same ends, same spelling): the answer is then 1 and ^NKG is dropped,
+/// which is the safe direction.
+///
+/// Both homes of an edge are read, because neither implies the other: the adjacency
+/// (^KG("out"), which also holds edges with no row behind them) and the rows (which can
+/// be ahead of an adjacency that was never built). Either one finding an edge with two
+/// interned ends answers 1 and stops.
+///
+/// Called before the rows go: the per-row index (spec 252) clears ^KG("out", graph) as
+/// they do, and a read of either afterwards sees an empty graph.
+///
+/// The failure mode is a drop that was not needed (NodeInterned answers 1 when it cannot
+/// tell), never a stale index kept. The cost is one walk of the graph's edges, against an
+/// erase that already removes each of them from six stores.
+ClassMethod GraphEdgesInNKG(pKey, pName As %String) As %Boolean [ Private ]
+{
+	// The adjacency. A source that is not interned has no edge in ^NKG, so its subtree is
+	// not walked at all.
+	Set tS = ""
+	For {
+		Set tS = $Order(^KG("out", pKey, tS))
+		Quit:tS=""
+		If '..NodeInterned(tS) {
+			Continue
+		}
+		Set tP = ""
+		For {
+			Set tP = $Order(^KG("out", pKey, tS, tP))
+			Quit:tP=""
+			Set tO = ""
+			For {
+				Set tO = $Order(^KG("out", pKey, tS, tP, tO))
+				Quit:tO=""
+				If ..NodeInterned(tO) {
+					Return 1
+				}
+			}
+		}
+	}
+
+	// The rows. COALESCE on both sides, as everywhere in this class: the default graph has
+	// two spellings in graph_id, and an empty host variable binds as NULL.
+	Set tRowS = ""
+	Set tRowO = ""
+	Set tHit = 0
+	&sql(DECLARE ivgEraseGraphEnds CURSOR FOR
+		SELECT s, o_id INTO :tRowS, :tRowO
+		FROM Graph_KG.rdf_edges
+		WHERE COALESCE(graph_id, '') = COALESCE(:pName, ''))
+	&sql(OPEN ivgEraseGraphEnds)
+	If SQLCODE < 0 {
+		// Cannot read the rows, cannot say: the drop is the safe answer.
+		Return 1
+	}
+	For {
+		&sql(FETCH ivgEraseGraphEnds)
+		Quit:SQLCODE'=0
+		If ..NodeInterned(tRowS) && ..NodeInterned(tRowO) {
+			Set tHit = 1
+			Quit
+		}
+	}
+	&sql(CLOSE ivgEraseGraphEnds)
+	Quit tHit
+}
+
+/// 1 when any id in ^||IVG.EraseIds is interned in ^NKG.
+///
+/// WHY THIS IS THE TEST. Every writer of an ^NKG edge interns BOTH of its ends first:
+/// Graph.KG.Adjacency:Put, Graph.KG.Traversal:BuildNKG and the Rust kg_build_nkg all
+/// do, and nothing else writes ^NKG(-1), ^NKG(-2), ^NKG(-3) or ^NKG("$agg") (the
+/// per-source sketch is keyed by interned indexes too). So an edge with an end that
+/// has no ^NKG("$NI", id) entry is not in ^NKG, and neither is any count or sketch
+/// built from it. An erase whose ids are all un-interned removes only such edges and
+/// leaves ^NKG exactly as correct as it was.
+///
+/// The converse is why the drop stays for an interned id: its edges are in ^NKG, and
+/// Graph.KG.GraphIndex:DeleteIndex (reached from Adjacency:Remove) takes only the
+/// adjacency entries, not $meta edgeCount or the sketches.
+///
+/// ^NKG("$NI") is the only interning table. Interning is append-only (an id keeps its
+/// index when its last edge goes), so a "yes" can be an id whose edges are gone, and
+/// the drop is then merely not needed. Never the other way round.
+ClassMethod NKGMayHoldAny() As %Boolean [ Private ]
+{
+	Set tId = ""
+	For {
+		Set tId = $Order(^||IVG.EraseIds(tId))
+		Quit:tId=""
+		If ..NodeInterned(tId) {
+			Quit
+		}
+	}
+	Quit (tId '= "")
+}
+
+/// 1 when pId has an entry in ^NKG("$NI"), or when that cannot be told.
+///
+/// A subscript ^NKG could not hold raises <SUBSCRIPT> on the read. Interning such an id
+/// would have raised the same, so it is not interned; but "cannot tell" answers 1, so
+/// that the failure mode of this check is a drop that was not needed and never a
+/// stale index kept. An empty id is no node and is never an edge end.
+ClassMethod NodeInterned(pId As %String) As %Boolean [ Private ]
+{
+	If pId = "" {
+		Quit 0
+	}
+	Set tInterned = 1
+	Try {
+		Set tInterned = ($Data(^NKG("$NI", pId)) > 0)
+	} Catch {
+		Set tInterned = 1
+	}
+	Quit tInterned
+}
+
 /// Drop the derived indexes that cannot be attributed to one graph.
 ///
+/// pTouchedStructural is "this erase may have changed what ^NKG holds", not "this erase
+/// removed an edge": EraseIdSet passes 0 for an erase whose removed edges cannot be in
+/// ^NKG (NKGMayHoldAny), and EraseGraph for a graph none of whose edges can be
+/// (GraphEdgesInNKG). EraseEdges and EraseAll pass 1. The version bump
+/// and the AlgoCache purge below do not depend on it.
+///
 /// ^NKG interns node identifiers with no graph dimension and its counters are
 /// append-only, so there is no subtree to kill and no way to decrement it. The
 /// choice is between an absent ^NKG and a stale one. Absent is a state every
@@ -919,6 +1093,177 @@ ClassMethod EraseFHIRGraphRows(pName As %String, pAll As %Boolean) [ Private ]
 	}
 }
 
+/// Remove the nodes of one graph that are BARE: no edge names them (as source or target), and they hold no
+/// label, property, document or embedding (the two legacy tables and every routed one). The maintenance sweep for
+/// the nodes a temporal workload leaves behind (opsreview, 4.4.1: one per span, for ever).
+///
+/// pGraph       the graph; "" is the default graph. Another graph's row of the same id is not touched.
+/// pBatch       nodes per transaction (default 10000); the list of what to delete is built first, so a
+///              node that stays (see below) is not looked at again in the same call.
+/// pDryRun      1 counts and changes nothing at all: no row, no global, not ^KG("__version").
+/// pMinAgeSec   leave nodes created less than this long ago. A writer that has just inserted its endpoint
+///              nodes and not yet its edge is the only one that can lose a race here, and it loses loudly
+///              (its edge INSERT fails the foreign key); this keeps a sweep off such nodes altogether.
+///
+/// RACE SAFETY. The statement that removes a node carries every bareness test again, and the table's own
+/// constraints decide the rest: fk_edges_source, fk_edges_dest, fk_labels_node and the embedding tables'
+/// foreign keys refuse a DELETE of a node with a dependant (SQLCODE -124), and they see a dependant another
+/// connection has written and not committed. The two child tables with no foreign key (properties,
+/// documents) are covered by the statement's NOT EXISTS only: a writer that finds the node present, and
+/// writes a property after this statement has evaluated it, leaves a property row without a node. Writers
+/// insert the node first, so that needs a node this sweep is already deleting; the age floor keeps the sweep
+/// off every node created since.
+///
+/// A node whose ^KG("out"/"in") entry exists is left (it is not bare, whatever the rows say).
+///
+/// Returns JSON {"graph", "bare", "erased", "kept", "batches", "dry_run"}. bare: nodes found bare;
+/// erased: removed; kept: found bare and left (adjacency present, a dependant appeared, a foreign key).
+/// Throws, after rolling back the batch it was in, when a statement fails; earlier batches stay committed.
+ClassMethod EraseBareNodes(pGraph As %String = "", pBatch As %Integer = 10000, pDryRun As %Boolean = 0, pMinAgeSec As %Integer = 0) As %String
+{
+	Set tKey = ##class(Graph.KG.GraphKey).ForIndex(pGraph)
+	Set tName = ##class(Graph.KG.GraphKey).ForName(pGraph)
+	Set pBatch = $Select(+pBatch < 1: 10000, 1: +pBatch \ 1)
+	Kill tRoute
+	Do ..CollectOrphanRoutes(.tRoute)
+	Set tAge = ""
+	If +pMinAgeSec > 0 {
+		Set tAge = " AND (n.created_at IS NULL OR n.created_at <= DATEADD(ss, -" _ (+pMinAgeSec \ 1) _ ", CURRENT_TIMESTAMP))"
+	}
+	Set tWhere = " FROM Graph_KG.nodes n WHERE COALESCE(n.graph_id, '') = COALESCE(?, '')" _ ..BareClauses(.tRoute) _ tAge
+	Set tBare = 0, tErased = 0, tKept = 0, tBatches = 0
+	Try {
+		If pDryRun {
+			Set tRS = ##class(%SQL.Statement).%ExecDirect(, "SELECT COUNT(*)" _ tWhere, tName)
+			If tRS.%SQLCODE < 0 {
+				$$$ThrowStatus($$$ERROR($$$GeneralError, "EraseBareNodes count: SQLCODE " _ tRS.%SQLCODE _ " " _ tRS.%Message))
+			}
+			If tRS.%Next() {
+				Set tBare = +tRS.%GetData(1)
+			}
+		} Else {
+			Kill ^||IVG.BareNodes
+			Set tRS = ##class(%SQL.Statement).%ExecDirect(, "SELECT n.node_id" _ tWhere, tName)
+			If tRS.%SQLCODE < 0 {
+				$$$ThrowStatus($$$ERROR($$$GeneralError, "EraseBareNodes scan: SQLCODE " _ tRS.%SQLCODE _ " " _ tRS.%Message))
+			}
+			While tRS.%Next() {
+				Set ^||IVG.BareNodes(tRS.%GetData(1)) = ""
+				Set tBare = tBare + 1
+			}
+			Set tErased = ..DeleteBareSet(tKey, tName, $Name(^||IVG.BareNodes), pBatch, .tKept, .tBatches)
+			Kill ^||IVG.BareNodes
+		}
+	} Catch tEx {
+		Kill ^||IVG.BareNodes
+		Throw tEx
+	}
+	Quit {"graph": (tName), "bare": (tBare), "erased": (tErased), "kept": (tKept), "batches": (tBatches), "dry_run": (+pDryRun)}.%ToJSON()
+}
+
+/// Retire the nodes named in @pRoot@(id) from graph pName (index key pKey) that are bare, as EraseBareNodes
+/// defines it; the others stay. For a purge that has just retracted the triples those ids were ends of
+/// (Graph.KG.TemporalIndex.RetireExpired). One transaction per 10,000 ids. Returns the nodes removed.
+ClassMethod RetireBareNodes(pKey, pName As %String, pRoot As %String) As %Integer
+{
+	Quit ..DeleteBareSet(pKey, pName, pRoot, 10000, .tKept, .tBatches)
+}
+
+/// The shared body: delete each id of @pRoot@(id) from graph pName while it is bare, pBatch ids to a
+/// transaction. pKept: ids left. pBatches: transactions run. Returns the number of nodes removed.
+ClassMethod DeleteBareSet(pKey, pName As %String, pRoot As %String, pBatch As %Integer, Output pKept As %Integer, Output pBatches As %Integer) As %Integer [ Private ]
+{
+	Set pKept = 0, pBatches = 0
+	Set tId = $Order(@pRoot@(""))
+	If tId = "" {
+		Quit 0
+	}
+	Kill tRoute
+	Do ..CollectOrphanRoutes(.tRoute)
+	Set tDel = ##class(%SQL.Statement).%New()
+	$$$ThrowOnError(tDel.%Prepare("DELETE FROM Graph_KG.nodes AS n WHERE n.node_id = ? AND COALESCE(n.graph_id, '') = COALESCE(?, '')" _ ..BareClauses(.tRoute)))
+	Set tErased = 0, tN = 0
+	Kill tDone
+	Try {
+		TSTART
+		While tId '= "" {
+			// an adjacency entry says the node has an edge, whatever the rows say
+			If $Data(^KG("out", pKey, tId)) || $Data(^KG("in", pKey, tId)) {
+				Set pKept = pKept + 1
+			} Else {
+				Set tRS = tDel.%Execute(tId, pName)
+				If tRS.%SQLCODE = 0 {
+					If tRS.%ROWCOUNT > 0 {
+						Set tErased = tErased + 1, tN = tN + 1, tDone(tId) = ""
+					}
+				} ElseIf tRS.%SQLCODE = 100 {
+					// no row: gone already, or another statement's dependant appeared
+				} ElseIf tRS.%SQLCODE = -124 {
+					// a foreign key found a dependant a writer added since the scan: the node stays
+					Set pKept = pKept + 1
+				} Else {
+					$$$ThrowStatus($$$ERROR($$$GeneralError, "bare node delete of " _ tId _ ": SQLCODE " _ tRS.%SQLCODE _ " " _ tRS.%Message))
+				}
+			}
+			Set tId = $Order(@pRoot@(tId))
+			If (tN >= pBatch) || (tId = "") {
+				TCOMMIT
+				Set pBatches = pBatches + 1
+				Do ..ForgetNodes(pKey, .tDone)
+				Kill tDone
+				Set tN = 0
+				If tId '= "" {
+					TSTART
+				}
+			}
+		}
+	} Catch tEx {
+		If $TLevel > 0 { TROLLBACK }
+		Throw tEx
+	}
+	If tErased > 0 {
+		Do ..InvalidateDerived(0)
+	}
+	Quit tErased
+}
+
+/// The derived entries of nodes that no longer exist: Adjacency.RemoveSource kills the generation, the cached
+/// degrees and the stored counters an older version left (ADR-0011: Adjacency is their only writer), and the
+/// property mirror goes here. A bare node has no edge, so RemoveSource finds no adjacency to remove.
+ClassMethod ForgetNodes(pKey, ByRef pIds) [ Private ]
+{
+	Set tId = ""
+	For {
+		Set tId = $Order(pIds(tId))
+		Quit:tId=""
+		$$$ThrowOnError(##class(Graph.KG.Adjacency).RemoveSource(pKey, tId))
+		Kill ^KG("prop", pKey, tId)
+	}
+}
+
+/// The "has none of these" tests for a node n (alias n, Graph_KG.nodes) as SQL text: no edge either way, no label,
+/// no property, no document, no embedding in the legacy tables or a routed node table.
+ClassMethod BareClauses(ByRef pRoute) As %String [ Private ]
+{
+	Set tEq = "COALESCE(x.graph_id, '') = COALESCE(n.graph_id, '')"
+	Set tC = " AND NOT EXISTS (SELECT 1 FROM Graph_KG.rdf_edges x WHERE x.s = n.node_id AND " _ tEq _ ")"
+	Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG.rdf_edges x WHERE x.o_id = n.node_id AND " _ tEq _ ")"
+	Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG.rdf_labels x WHERE x.s = n.node_id AND " _ tEq _ ")"
+	Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG.rdf_props x WHERE x.s = n.node_id AND " _ tEq _ ")"
+	Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG.docs x WHERE x.id = n.node_id AND " _ tEq _ ")"
+	Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG.kg_NodeEmbeddings x WHERE x.node_id = n.node_id AND " _ tEq _ ")"
+	Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG.kg_NodeEmbeddings_optimized x WHERE x.node_id = n.node_id AND " _ tEq _ ")"
+	Set tT = ""
+	For {
+		Set tT = $Order(pRoute(tT))
+		Quit:tT=""
+		Continue:$Extract(tT, 1, 8)="kg_eemb_"
+		Continue:'##class(%SYSTEM.SQL.Schema).TableExists("Graph_KG." _ tT)
+		Set tC = tC _ " AND NOT EXISTS (SELECT 1 FROM Graph_KG." _ tT _ " x WHERE x.node_id = n.node_id AND " _ tEq _ ")"
+	}
+	Quit tC
+}
+
 /// One error shape for every failed step, so the rollback path has one reason.
 ClassMethod SqlErr(pStore As %String, pCode As %Integer) As %Status [ Private ]
 {

{iris_vector_graph-4.4.1 → iris_vector_graph-4.5.0}/iris_src/src/Graph/KG/GraphStores.cls

@@ -104,6 +104,11 @@ ClassMethod Inventory() As %DynamicArray
 	// ^IVG.GraphMode(gKey)="multi" for each graph that allows parallel edges.
 	Do tStores.%Push(..Entry("^IVG.GraphMode", "global", "subscript 1", 1))
 
+	// --- graph-scoped globals: temporal-only predicates (ADR-0015) -------------
+	// ^IVG.TemporalStructural(gKey, predicate)=0 for each predicate of a graph whose temporal edges imply no
+	// structural row. Configuration: EraseGraph keeps it, EraseAll empties it, a snapshot carries it.
+	Do tStores.%Push(..Entry("^IVG.TemporalStructural", "global", "subscript 1", 1))
+
 	// --- graph-scoped globals: algorithm result cache (spec 236) -----------
 	// ^IVG.AlgoCache(gKey, algorithm, paramHash) = $LB(token, payload). Derived from
 	// the graph and recomputable, so erasing a graph drops its entries (ADR-0004: no

{iris_vector_graph-4.4.1 → iris_vector_graph-4.5.0}/iris_src/src/Graph/KG/NKGAccelAdjacency.cls

@@ -57,8 +57,11 @@ ClassMethod ExportAdjacencyNKG() As %String [ Private ]
     If chunkNum > 0 {
         Set n = $Get(^NKG("$meta", "nodeCount"), 0)
         Set nodemap = "NODEMAP:" _ n _ $Char(10)
+        // n bounds the indices; an index with no $ND entry (a gap a rebuild left after
+        // removing a node, spec 260 D-NKG-SWAP) is no node and is not listed
         For i = 0:1:n-1 {
-            Set nodemap = nodemap _ i _ "=" _ $Get(^NKG("$ND", i), "") _ $Char(10)
+            Continue:'$Data(^NKG("$ND", i))
+            Set nodemap = nodemap _ i _ "=" _ ^NKG("$ND", i) _ $Char(10)
         }
         Set nodemap = nodemap _ "---" _ $Char(10)
         Set ^||adj = chunkNum + 1