iris-vector-graph 1.99.2__tar.gz → 1.99.3__tar.gz

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: iris-vector-graph
-Version: 1.99.2
+Version: 1.99.3
 Summary: Transactional Graph + Vector retrieval system for InterSystems IRIS with hybrid search, openCypher, and GraphQL APIs
 Project-URL: Homepage, https://github.com/intersystems-community/iris-vector-graph
 Project-URL: Documentation, https://github.com/intersystems-community/iris-vector-graph/tree/main/docs
@@ -816,12 +816,7 @@ conditions = tool("patient-123")  # → {"conditions": [...], "error": None}
 
  ## Changelog
 
-### v1.99.2 (2026-05-29)
-- **perf(spec-165)**: `BuildNKGComplete` — collapses 7 Python→IRIS round-trips in `engine.rebuild_nkg()` to 1. New `Graph.KG.Traversal.BuildNKGComplete()` ObjectScript method runs `BuildNKG + Build2HopStats` in one call; returns discriminated `"OS:<edges>:<nodes>"` / `"RUST:<edges>:<nodes>"` prefix for zero-probe path discrimination. `engine.rebuild_nkg()` now returns `{"path": "objectscript"|"rust"|"fallback", "edge_count": N, "node_count": M}` (was `bool`; backward-compatible — still truthy on success).
-- **fix(spec-165)**: `Graph.KG.GraphIndex.EmptyHLL` — applied to `main` branch; `..EmptyHLL()` → `##class(Graph.KG.GraphIndex).EmptyHLL()` prevents `<No such method>` compile error on `Graph.KG.Edge.cls` in fresh containers. (Previously only on the `164-seed-local-khop-fast-path` branch.)
-- **fix(spec-165)**: `Graph.KG.Traversal.BuildNKG` and `Build2HopExactStats` — added `##class(Graph.KG.NKGAccel).IsLoaded()` guards before `InvalidateAdjCache()` and `Build2HopExact()` calls to prevent non-trappable `<DYNAMIC LIBRARY LOAD>` errors when `libarno_callout.so` is absent.
-
-
+### v1.99.0 (2026-05-28)
 - **feat**: Spec 163 — Community Detection & Cluster Analysis Suite. Four new graph algorithms via the GraphStore protocol + Cypher procedures + dual-path architecture (arno Rust accelerator primary + LazyKG pure-Python fallback):
   - `engine.leiden_communities(max_levels, gamma, tol, top_k, mem_budget_mb, random_seed, progress_callback)` — Leiden community detection (Traag et al. 2019). At `gamma=1.0` uses `ModularityVertexPartition` (canonical Leiden); at `gamma != 1.0` uses `CPMVertexPartition` for resolution control. ARI = 1.0 with `leidenalg` reference (4-way benchmark on karate, ER(500), ER(2000)).
   - `engine.triangle_count(top_k, progress_callback)` — symmetrized triangle count + LCC. Pearson > 0.95 with `networkx.triangles(networkx.Graph(G_directed))` on Erdős-Rényi 100-node fixture.

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/README.md

@@ -733,12 +733,7 @@ conditions = tool("patient-123")  # → {"conditions": [...], "error": None}
 
  ## Changelog
 
-### v1.99.2 (2026-05-29)
-- **perf(spec-165)**: `BuildNKGComplete` — collapses 7 Python→IRIS round-trips in `engine.rebuild_nkg()` to 1. New `Graph.KG.Traversal.BuildNKGComplete()` ObjectScript method runs `BuildNKG + Build2HopStats` in one call; returns discriminated `"OS:<edges>:<nodes>"` / `"RUST:<edges>:<nodes>"` prefix for zero-probe path discrimination. `engine.rebuild_nkg()` now returns `{"path": "objectscript"|"rust"|"fallback", "edge_count": N, "node_count": M}` (was `bool`; backward-compatible — still truthy on success).
-- **fix(spec-165)**: `Graph.KG.GraphIndex.EmptyHLL` — applied to `main` branch; `..EmptyHLL()` → `##class(Graph.KG.GraphIndex).EmptyHLL()` prevents `<No such method>` compile error on `Graph.KG.Edge.cls` in fresh containers. (Previously only on the `164-seed-local-khop-fast-path` branch.)
-- **fix(spec-165)**: `Graph.KG.Traversal.BuildNKG` and `Build2HopExactStats` — added `##class(Graph.KG.NKGAccel).IsLoaded()` guards before `InvalidateAdjCache()` and `Build2HopExact()` calls to prevent non-trappable `<DYNAMIC LIBRARY LOAD>` errors when `libarno_callout.so` is absent.
-
-
+### v1.99.0 (2026-05-28)
 - **feat**: Spec 163 — Community Detection & Cluster Analysis Suite. Four new graph algorithms via the GraphStore protocol + Cypher procedures + dual-path architecture (arno Rust accelerator primary + LazyKG pure-Python fallback):
   - `engine.leiden_communities(max_levels, gamma, tol, top_k, mem_budget_mb, random_seed, progress_callback)` — Leiden community detection (Traag et al. 2019). At `gamma=1.0` uses `ModularityVertexPartition` (canonical Leiden); at `gamma != 1.0` uses `CPMVertexPartition` for resolution control. ARI = 1.0 with `leidenalg` reference (4-way benchmark on karate, ER(500), ER(2000)).
   - `engine.triangle_count(top_k, progress_callback)` — symmetrized triangle count + LCC. Pearson > 0.95 with `networkx.triangles(networkx.Graph(G_directed))` on Erdős-Rényi 100-node fixture.

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/iris_src/src/Graph/KG/GraphIndex.cls

@@ -91,7 +91,7 @@ ClassMethod UpdateStructuralHLL(sIdx As %Integer, pIdx As %Integer, oIdx As %Int
 	} Else {
 		While (b1 # 2) = 0 { Set lz = lz + 1, b1 = b1 \ 2 }
 	}
-	Set hll = $Get(^NKG("$agg", sIdx, pIdx, "hll"), ##class(Graph.KG.GraphIndex).EmptyHLL())
+	Set hll = $Get(^NKG("$agg", sIdx, pIdx, "hll"), ..EmptyHLL())
 	If lz > $List(hll, regIdx) {
 		Set $List(hll, regIdx) = lz
 		Set ^NKG("$agg", sIdx, pIdx, "hll") = hll

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/iris_src/src/Graph/KG/Traversal.cls

@@ -83,7 +83,7 @@ ClassMethod BuildNKG() As %Status
 	// Kill ^NKG resets the version to 0; $Increment makes it 1.
 	// If the stale cache also has version 1, CacheNKGAdj sees cachedVer==currentVer
 	// and skips the rebuild, leaving Arno BFS using data from the previous graph.
-    Try { If ##class(Graph.KG.NKGAccel).IsLoaded() { Do ##class(Graph.KG.NKGAccel).InvalidateAdjCache() } } Catch {}
+	Try { Do ##class(Graph.KG.NKGAccel).InvalidateAdjCache() } Catch {}
 	Kill ^NKG
 	Do ##class(Graph.KG.GraphIndex).InitStructuralLabels()
 	
@@ -116,34 +116,12 @@ ClassMethod BuildNKG() As %Status
 	Set ^NKG("$meta", "edgeCount") = edgeCount
 	Set tmp = $Increment(^NKG("$meta", "version"))
 
-    Do ..Build2HopStats()
-    Do ..Build2HopExactStats()
+	Do ..Build2HopStats()
+	Do ..Build2HopExactStats()
 
-    Return $$$OK
+	Return $$$OK
 }
 
-/// Spec 165 (FR-165-001..004): Batch all 5 NKG-rebuild steps into 1 ObjectScript call,
-/// eliminating 7 Python→IRIS round-trips that engine.rebuild_nkg() previously made.
-///
-/// Algorithm:
-///   1. Try arno Rust kg_nkg_build kernel via $ZF(-5) — fastest path
-///   2. Fallback: ..BuildNKG() (ObjectScript $Order walk over ^KG)
-///   3. Always: Build2HopStats + Build2HopExactStats + InvalidateAdjCache
-///
-/// Returns: "RUST:<edgeCount>:<nodeCount>" | "OS:<edgeCount>:<nodeCount>" | "ERROR:<msg>"
-/// Python caller parses prefix to set _arno_available without extra round-trip.
-ClassMethod BuildNKGComplete(libPath As %String = "/usr/irissys/mgr/libarno_callout.so") As %String
-{
-    Do ..BuildNKG()
-    Do ..Build2HopStats()
-    Set ec = $Get(^NKG("$meta", "edgeCount"), 0)
-    Set nc = 0, nid = $Order(^NKG("$NI", ""))
-    While nid '= "" { Set nc = nc + 1, nid = $Order(^NKG("$NI", nid)) }
-    Set ^NKG("$meta", "nodeCount") = nc
-    Quit "OS:"_ec_":"_nc
-}
-
-
 ClassMethod BackfillDegp() As %Integer
 {
 	Kill ^KG("deg")
@@ -1247,7 +1225,7 @@ ClassMethod KHop2CountExact(srcId As %String, pred As %String = "") As %Integer
 ClassMethod Build2HopExactStats() As %Integer
 {
     Kill ^KG("deg2p_exact")
-    If ##class(Graph.KG.NKGAccel).IsLoaded() && ##class(Graph.KG.NKGAccel).Build2HopExact() {
+    If ##class(Graph.KG.NKGAccel).Build2HopExact() {
         Set n = ""
         Set cnt = 0
         For {

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/iris_vector_graph/engine.py

@@ -8,7 +8,6 @@ that can be used across any domain.
 """
 
 import json
-import os
 from pathlib import Path
 from typing import Callable, List, Tuple, Optional, Dict, Any
 import logging
@@ -134,6 +133,7 @@ class IRISGraphEngine:
         self._rel_mapping_cache: Optional[Dict[tuple, dict]] = None
         self._connection_params: Optional[Dict[str, Any]] = None
         self._nkg_dirty: bool = False
+        self._shared_lkg = None
         self._index_registry: Dict[str, str] = self._build_index_registry()
         if vector_dtype == "DOUBLE":
             self.vector_dtype = self._detect_stored_vector_dtype()
@@ -146,6 +146,16 @@ class IRISGraphEngine:
         logger.debug("IRISGraphEngine initialized (dim=%s dtype=%s)",
                      embedding_dimension or "auto", self.vector_dtype)
 
+    def _get_shared_lkg(self, include_sinks: bool = True):
+        """Return the shared LazyKG instance, creating it on first call (spec 166 FR-166-002)."""
+        if self._shared_lkg is None:
+            from iris_vector_graph.stores.lazy_kg import LazyKG
+            self._shared_lkg = LazyKG(self.conn, include_sinks=include_sinks)
+        return self._shared_lkg
+
+    def _invalidate_shared_lkg(self) -> None:
+        self._shared_lkg = None
+
     @classmethod
     def from_connect(
         cls,
@@ -3290,80 +3300,14 @@ class IRISGraphEngine:
             _call_classmethod(self.conn, "Graph.KG.Traversal", "BuildKG")
             self.capabilities.kg_built = True
             self._nkg_dirty = True
+            self._invalidate_shared_lkg()
             logger.info("^KG adjacency index rebuilt successfully")
             return True
         except Exception as e:
             logger.warning("rebuild_kg failed: %s", e)
             return False
 
-    def rebuild_nkg(self) -> dict:
-        """Rebuild `^NKG` integer index from `^KG` adjacency.
-
-        Spec 165: dispatches to `BuildNKGComplete` (1 IRIS round-trip),
-        replacing the legacy 5-call sequence.
-        Returns {"path": "rust"|"objectscript"|"fallback", "edge_count": N, "node_count": M}.
-        Backward-compatible: return value is truthy on success (was `True`/`False`).
-        """
-        try:
-            iris_obj = self._iris_obj()
-            raw = str(iris_obj.classMethodValue(
-                "Graph.KG.Traversal", "BuildNKGComplete",
-                os.environ.get("IVG_ARNO_LIB", "/usr/irissys/mgr/libarno_callout.so"),
-            ))
-            if raw.startswith("RUST:"):
-                parts = raw.split(":")
-                self._arno_available = True
-                self._nkg_dirty = False
-                return {"path": "rust", "edge_count": int(parts[1]), "node_count": int(parts[2])}
-            elif raw.startswith("OS:"):
-                parts = raw.split(":")
-                self._arno_available = False
-                self._nkg_dirty = False
-                try:
-                    iris_obj.classMethodValue("Graph.KG.Traversal", "Build2HopExactStats")
-                except Exception:
-                    pass
-                try:
-                    iris_obj.classMethodVoid("Graph.KG.NKGAccel", "InvalidateAdjCache")
-                except Exception:
-                    pass
-                return {"path": "objectscript", "edge_count": int(parts[1]), "node_count": int(parts[2])}
-        except Exception as e:
-            logger.warning("BuildNKGComplete failed, falling back to legacy sequence: %s", e)
-
-        try:
-            iris_obj = self._iris_obj()
-            rust_succeeded = False
-            if self._detect_arno() and self._arno_capabilities.get("rust_callout"):
-                try:
-                    import json as _json
-                    raw = str(iris_obj.classMethodValue("Graph.KG.NKGAccel", "BuildNKGRust"))
-                    result = _json.loads(raw)
-                    if "error" not in result:
-                        logger.info("^NKG rebuilt via Rust: %s", result)
-                        rust_succeeded = True
-                    else:
-                        logger.warning("BuildNKGRust returned error (%s), falling back to ObjectScript", result["error"])
-                except Exception as rust_exc:
-                    logger.warning("BuildNKGRust raised (%s), falling back to ObjectScript", rust_exc)
-            if not rust_succeeded:
-                iris_obj.classMethodVoid("Graph.KG.Traversal", "BuildNKG")
-            iris_obj.classMethodValue("Graph.KG.Traversal", "Build2HopStats")
-            try:
-                iris_obj.classMethodValue("Graph.KG.Traversal", "Build2HopExactStats")
-            except Exception:
-                pass
-            try:
-                iris_obj.classMethodVoid("Graph.KG.NKGAccel", "InvalidateAdjCache")
-            except Exception:
-                pass
-            self._nkg_dirty = False
-            return {"path": "fallback", "edge_count": 0, "node_count": 0}
-        except Exception as e:
-            logger.warning("rebuild_nkg failed: %s", e)
-            self._nkg_dirty = False
-            return {"path": "fallback", "edge_count": 0, "node_count": 0}
-
+    def rebuild_nkg(self) -> bool:
         try:
             iris_obj = self._iris_obj()
             rust_succeeded = False
@@ -3388,11 +3332,10 @@ class IRISGraphEngine:
             except Exception:
                 pass
             self._nkg_dirty = False
-            return {"path": "fallback", "edge_count": 0, "node_count": 0}
+            return True
         except Exception as e:
             logger.warning("rebuild_nkg failed: %s", e)
-            self._nkg_dirty = False
-            return {"path": "fallback", "edge_count": 0, "node_count": 0}
+            return False
 
     def backfill_degp(self) -> int:
         try:
@@ -3480,6 +3423,7 @@ class IRISGraphEngine:
             n += 1
         self.conn.commit()
         self._nkg_dirty = True
+        self._invalidate_shared_lkg()
         return n
 
     def load_networkx(
@@ -7189,6 +7133,7 @@ class IRISGraphEngine:
             validated.direction,
             validated.predicate or "",
             validated.top_k,
+            lkg=self._get_shared_lkg(),
         )
         if result.error:
             return []
@@ -7242,6 +7187,7 @@ class IRISGraphEngine:
             validated.top_k,
             validated.mem_budget_mb,
             progress_callback,
+            lkg=self._get_shared_lkg(),
         )
         if result.error:
             return []
@@ -7296,6 +7242,7 @@ class IRISGraphEngine:
             validated.max_hops,
             validated.top_k,
             progress_callback,
+            lkg=self._get_shared_lkg(),
         )
         if result.error:
             return []
@@ -7340,6 +7287,7 @@ class IRISGraphEngine:
             validated.tol,
             validated.top_k,
             progress_callback,
+            lkg=self._get_shared_lkg(),
         )
         if result.error:
             return []
@@ -7412,6 +7360,7 @@ class IRISGraphEngine:
             validated.max_levels, validated.gamma, validated.tol,
             validated.top_k, validated.mem_budget_mb,
             validated.random_seed, progress_callback,
+            lkg=self._get_shared_lkg(),
         )
         if result.error:
             return []
@@ -7440,7 +7389,7 @@ class IRISGraphEngine:
                 f"{type(self._store).__name__ if getattr(self, '_store', None) else 'None'}"
             )
 
-        result = self._store.execute_triangle_count(validated.top_k, progress_callback)
+        result = self._store.execute_triangle_count(validated.top_k, progress_callback, lkg=self._get_shared_lkg())
         if result.error:
             return []
         return [{"id": row[0], "triangles": row[1], "lcc": row[2]} for row in result.rows]
@@ -7459,7 +7408,7 @@ class IRISGraphEngine:
                 f"{type(self._store).__name__ if getattr(self, '_store', None) else 'None'}"
             )
 
-        result = self._store.execute_scc(validated.top_k, progress_callback)
+        result = self._store.execute_scc(validated.top_k, progress_callback, lkg=self._get_shared_lkg())
         if result.error:
             return []
         return [{"id": row[0], "component": row[1], "size": row[2]} for row in result.rows]
@@ -7478,7 +7427,7 @@ class IRISGraphEngine:
                 f"{type(self._store).__name__ if getattr(self, '_store', None) else 'None'}"
             )
 
-        result = self._store.execute_k_core(validated.top_k, progress_callback)
+        result = self._store.execute_k_core(validated.top_k, progress_callback, lkg=self._get_shared_lkg())
         if result.error:
             return []
         return [{"id": row[0], "coreness": row[1]} for row in result.rows]

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/iris_vector_graph/schema.py

@@ -801,9 +801,77 @@ LANGUAGE OBJECTSCRIPT
 """,
         ]
 
-    # ------------------------------------------------------------------
-    # ObjectScript (.cls) deployment helpers
-    # ------------------------------------------------------------------
+    @staticmethod
+    def get_graph_json_sql_list(table_schema: str = "Graph_KG") -> list:
+        """SQL OBJECTSCRIPT functions for server-side ^KG→JSON export (spec 167 FR-167-001/002)."""
+        TS = table_schema
+        build_ddl = "\n".join([
+            "CREATE OR REPLACE FUNCTION " + TS + ".ivg_graph_json_build(inclsinks INT DEFAULT 1)",
+            "RETURNS VARCHAR(64)",
+            "LANGUAGE OBJECTSCRIPT",
+            "{",
+            "    kill ^||gji, ^||graphjsonchunks",
+            "    set n = 0, e = 0",
+            '    set nodeList = "{""nodes"":["',
+            '    set src = $Order(^KG("out",0,""))',
+            '    while src \'= "" {',
+            "        set ^||gji(src) = n",
+            '        if n > 0 set nodeList = nodeList_","',
+            '        set nodeList = nodeList_""""_src_""""',
+            "        set n = n + 1",
+            '        set src = $Order(^KG("out",0,src))',
+            "    }",
+            "    if inclsinks {",
+            '        set dst = $Order(^KG("in",0,""))',
+            '        while dst \'= "" {',
+            "            if '$Data(^||gji(dst)) {",
+            '                if n > 0 set nodeList = nodeList_","',
+            '                set nodeList = nodeList_""""_dst_""""',
+            "                set ^||gji(dst) = n",
+            "                set n = n + 1",
+            "            }",
+            '            set dst = $Order(^KG("in",0,dst))',
+            "        }",
+            "    }",
+            '    set edgeList = "],""edges"":["',
+            '    set src = $Order(^KG("out",0,""))',
+            '    while src \'= "" {',
+            '        set pred = $Order(^KG("out",0,src,""))',
+            '        while pred \'= "" {',
+            '            set dst = $Order(^KG("out",0,src,pred,""))',
+            '            while dst \'= "" {',
+            '                if e > 0 set edgeList = edgeList_","',
+            '                set edgeList = edgeList_"{""s"":""""_src_"""",""d"":""""_dst_""""}"',
+            "                set e = e + 1",
+            '                set dst = $Order(^KG("out",0,src,pred,dst))',
+            "            }",
+            '            set pred = $Order(^KG("out",0,src,pred))',
+            "        }",
+            '        set src = $Order(^KG("out",0,src))',
+            "    }",
+            '    set full = nodeList_edgeList_"]}"',
+            "    set chunkNum = 0, pos = 1, csize = 12000",
+            "    while pos <= $Length(full) {",
+            "        set chunkNum = chunkNum + 1",
+            "        set ^||graphjsonchunks(chunkNum) = $Extract(full,pos,pos+csize-1)",
+            "        set pos = pos + csize",
+            "    }",
+            "    set ^||graphjsonchunks = chunkNum",
+            '    quit "OK:"_n_":"_e_":"_chunkNum',
+            "}",
+        ])
+        chunk_ddl = "\n".join([
+            "CREATE OR REPLACE FUNCTION " + TS + ".ivg_graph_json_chunk(idx INT)",
+            "RETURNS VARCHAR(32000)",
+            "LANGUAGE OBJECTSCRIPT",
+            "{",
+            "    quit $Get(^||graphjsonchunks(idx))",
+            "}",
+        ])
+        return [build_ddl, chunk_ddl]
+
+
+
 
     @staticmethod
     def check_objectscript_classes(cursor, conn=None) -> "IRISCapabilities":

{iris_vector_graph-1.99.2 → iris_vector_graph-1.99.3}/iris_vector_graph/stores/arno_bridge.py

@@ -782,3 +782,59 @@ LANGUAGE OBJECTSCRIPT
     full = "".join(chunks)
     idx_to_node = full.rstrip("\x1f").split("\x1f") if full else []
     return idx_to_node, edge_count
+
+
+def build_graph_json_serverside(conn, include_sinks: bool = True) -> dict:
+    """Spec 167 FR-167-003: Build ^KG graph as JSON in one server-side ObjectScript call.
+
+    Calls ivg_graph_json_build() (SQL OBJECTSCRIPT), fetches JSON chunks
+    via ivg_graph_json_chunk(), returns {"nodes": [...], "edges": [{"s":..,"d":..},..]}.
+    Falls back to build_kg_adjacency_json() on any exception (FR-167-007).
+
+    This collapses O(V+E) Python→IRIS nextSubscript round-trips to 2-5 SQL calls.
+    """
+    import json as _json
+    from iris_vector_graph.schema import GraphSchema
+
+    key = _conn_key(conn)
+    cache = _probe_cache.get(key, {})
+    lib_path = cache.get("lib_path", DEFAULT_LIB_PATH)
+
+    cur = conn.cursor()
+    if not cache.get("graph_json_ddl_installed"):
+        try:
+            ddl_list = GraphSchema.get_graph_json_sql_list("Graph_KG")
+            for ddl in ddl_list:
+                cur.execute(ddl.strip())
+            conn.commit()
+            if key not in _probe_cache:
+                _probe_cache[key] = {}
+            _probe_cache[key]["graph_json_ddl_installed"] = True
+        except Exception as e:
+            raise ArnoError(f"ivg_graph_json DDL install failed: {e}") from e
+
+    try:
+        sinks_arg = 1 if include_sinks else 0
+        cur.execute(f"SELECT Graph_KG.ivg_graph_json_build({sinks_arg})")
+        row = cur.fetchone()
+        if not row or not row[0]:
+            raise ArnoError("ivg_graph_json_build returned NULL")
+        status = str(row[0])
+        if not status.startswith("OK:"):
+            raise ArnoError(f"ivg_graph_json_build: {status}")
+        parts = status[3:].split(":")
+        n, e, chunk_count = int(parts[0]), int(parts[1]), int(parts[2])
+
+        chunks: list = []
+        for i in range(1, chunk_count + 1):
+            cur.execute(f"SELECT Graph_KG.ivg_graph_json_chunk({i})")
+            chunk_row = cur.fetchone()
+            if chunk_row and chunk_row[0]:
+                chunks.append(str(chunk_row[0]))
+        full_json = "".join(chunks)
+        return _json.loads(full_json)
+
+    except ArnoError:
+        raise
+    except Exception as e:
+        raise ArnoError(f"build_graph_json_serverside failed: {e}") from e