@andespindola/brainlink 0.1.0-beta.118 → 0.1.0-beta.119

package/README.md

@@ -84,11 +84,11 @@ Legacy `.jsonl.gz` packs are upgraded to `.blpk` automatically on first search/c
 - Graph renderer optimized for large datasets with viewport-driven node culling and edge lookup by visible nodes.
 - Canvas graph rendering uses the same batched node and edge pipeline for every graph size, reducing per-frame draw calls while keeping selected and hovered items highlighted.
 - WebGL acceleration is used when available for dense node and edge drawing, with Canvas 2D preserved as the interaction and fallback layer.
-- Graph rendering uses a rebuilt Bloom-like flat scene: no macro galaxy, no recursive subgraphs, no cluster replacement layer and no synthetic 3D projection.
+- Graph rendering keeps the flat node scene and adds stable hierarchical groups for vaults above 1000 notes, with recursive parent groups when a level itself exceeds 1000 groups.
 - Large graph layout API automatically uses compact payload encoding with link-coverage-aware edge selection to reduce initial client load without hiding major relationships.
 - Large-segment layout spacing now grows logarithmically to keep initial visual density consistent between medium and very large vaults (for example, ~1k vs ~50k notes).
 - Graph coordinates are visually compacted across graph sizes so reset starts from a stable fitted scene and zoom-in progressively reveals local detail.
-- Zoomed-out graph LOD samples visible nodes and priority links without switching to nested subgraphs or cluster markers.
+- Zoomed-out graph LOD samples visible nodes, priority links and stable hierarchy groups without replacing the whole scene during zoom.
 - Graph reset fits the full graph scene instead of starting in a separate macro overview mode.
 - Graph filtering runs in a dedicated browser worker to keep the UI thread responsive during heavy datasets.
 - Edge rendering budgets adapt to zoom level to prevent frame spikes on large graph panoramas.
@@ -596,7 +596,7 @@ The graph UI shows:
 - graph controls for zoom in, zoom out, fit visible nodes and reset-to-fit-all
 - wheel zoom (including `cmd+scroll` and `ctrl+scroll`) anchored to cursor position for faster navigation in large graphs
 - continuous target-scale interpolation for wheel/button zoom to avoid abrupt jumps while keeping cursor-anchored focus
-- Bloom-like scene navigation: reset fits the current graph scene, wheel zoom stays anchored to the cursor, and WebGL acceleration is used only to draw the same flat graph faster
+- Bloom-like scene navigation: reset fits the current graph scene, wheel zoom stays anchored to the cursor, and WebGL acceleration draws the dense node and edge layer faster
 - zoom-out floor for large and massive graphs to keep the scene reachable without switching into a separate macro graph mode
 - keyboard shortcuts: `+` zoom in, `-` zoom out, `0` reset fit
 - double-click on canvas zooms in at cursor position
@@ -604,7 +604,7 @@ The graph UI shows:
 - graph rendering safeguards (batched canvas drawing across graph sizes, edge draw caps, lower redraw rate, zoom-aware interaction)
 - adaptive CPU safeguards for large graphs: idle frame pacing, throttled background physics updates and cached viewport dimensions to reduce redraw/layout overhead while preserving interaction responsiveness
 - WebGL node and edge acceleration when supported, falling back to Canvas 2D without changing graph behavior
-- large graph LOD keeps the same scene model as smaller graphs: visible nodes are sampled near the viewport/focus, real links are prioritized, and no macro galaxy, recursive graph-of-graphs, cluster replacement layer or synthetic 3D projection is applied
+- large graph LOD keeps the same real-node scene model as smaller graphs: visible nodes are sampled near the viewport/focus, real links are prioritized, and stable hierarchy groups provide zoom-out context without a synthetic 3D projection
 
 The server indexes before starting by default. Use `--no-index` to skip that step:
 

package/dist/application/frontend/client-js.js

@@ -11,6 +11,7 @@ const massiveOverviewScaleThreshold = 0.065
 const massiveSegmentedScaleThreshold = 0.45
 const massiveSegmentRepresentativeBudget = 760
 const massiveAutoFitMacroScale = 0.018
+const hierarchyGroupScaleThreshold = 0.62
 const minNodePixelRadius = 2.3
 const viewportPaddingPx = 280
 const worldCoordinateLimit = 5_000_000
@@ -38,6 +39,9 @@ const physicsStepDeltaCapMs = 96
 const state = {
   graph: { nodes: [], edges: [] },
   nodes: [],
+  groups: [],
+  groupById: new Map(),
+  leafGroups: [],
   nodeById: new Map(),
   edges: [],
   visibleNodes: [],
@@ -1151,6 +1155,7 @@ const drawAcceleratedGraph = (width, height, drawEdges) => {
   ctx.save()
   ctx.translate(state.transform.x, state.transform.y)
   ctx.scale(state.transform.scale, state.transform.scale)
+  drawHierarchyGroups()
   if (drawEdges) {
     graphParts.selectedEdges.forEach(edge => drawGraphEdge(edge))
   }
@@ -1161,6 +1166,101 @@ const drawAcceleratedGraph = (width, height, drawEdges) => {
   return true
 }
 
+const focusedGroup = () => {
+  if (state.groups.length === 0) {
+    return null
+  }
+
+  const selectedId = state.selected?.id ?? state.hovered?.id
+  if (selectedId) {
+    const selectedGroup = state.leafGroups.find(group => group.nodeIds.includes(selectedId))
+    if (selectedGroup) {
+      return selectedGroup
+    }
+  }
+
+  const focusPoint = performance.now() - state.lastZoomFocus.at <= 1800
+    ? state.lastZoomFocus
+    : viewportCenterWorldPoint()
+  return state.leafGroups
+    .map(group => ({
+      group,
+      distance: Math.hypot(group.x - focusPoint.x, group.y - focusPoint.y)
+    }))
+    .sort((left, right) => left.distance - right.distance)[0]?.group ?? null
+}
+
+const hierarchyGroupOpacity = (group, focus) => {
+  if (state.transform.scale >= hierarchyGroupScaleThreshold || state.groups.length === 0) {
+    return 0
+  }
+
+  const base = state.transform.scale < 0.08 ? 0.28 : state.transform.scale < 0.22 ? 0.2 : 0.12
+  if (!focus) {
+    return base
+  }
+  if (group.id === focus.id || group.childGroupIds.includes(focus.id) || group.id === focus.parentId) {
+    return Math.min(0.36, base + 0.12)
+  }
+
+  return base * 0.58
+}
+
+const visibleHierarchyGroups = () => {
+  if (state.groups.length === 0 || state.transform.scale >= hierarchyGroupScaleThreshold) {
+    return []
+  }
+
+  const viewport = expandViewportBounds(worldViewportBounds(), 900 / Math.max(state.transform.scale, 0.0001))
+  const groups = state.groups.filter(group =>
+    group.x + group.radius >= viewport.minX &&
+    group.x - group.radius <= viewport.maxX &&
+    group.y + group.radius >= viewport.minY &&
+    group.y - group.radius <= viewport.maxY
+  )
+  const level = groups.some(group => group.parentId === null && group.childGroupIds.length > 0 && state.transform.scale < 0.18)
+    ? 1
+    : 0
+
+  return groups
+    .filter(group => group.level === level || group.parentId === null)
+    .slice(0, 1400)
+}
+
+const drawHierarchyGroups = () => {
+  const groups = visibleHierarchyGroups()
+  if (groups.length === 0) {
+    return
+  }
+
+  const focus = focusedGroup()
+  for (let index = 0; index < groups.length; index += 1) {
+    const group = groups[index]
+    const opacity = hierarchyGroupOpacity(group, focus)
+    if (opacity <= 0) {
+      continue
+    }
+
+    const isFocus = focus && (group.id === focus.id || group.childGroupIds.includes(focus.id) || group.id === focus.parentId)
+    ctx.beginPath()
+    ctx.arc(group.x, group.y, group.radius * (isFocus ? 1.04 : 1), 0, Math.PI * 2)
+    ctx.fillStyle = 'rgba(148, 163, 184, ' + (opacity * 0.12) + ')'
+    ctx.fill()
+    ctx.lineWidth = isFocus ? 2.2 : 1.2
+    ctx.strokeStyle = 'rgba(148, 163, 184, ' + opacity + ')'
+    ctx.stroke()
+
+    if (state.transform.scale >= 0.12 && groups.length <= 220) {
+      ctx.fillStyle = 'rgba(226, 232, 240, ' + Math.min(0.86, opacity + 0.34) + ')'
+      ctx.font = '12px Inter, system-ui, sans-serif'
+      ctx.textAlign = 'center'
+      ctx.textBaseline = 'middle'
+      ctx.fillText(group.title.slice(0, 36), group.x, group.y)
+    }
+  }
+  ctx.globalAlpha = 1
+}
+
 const edgePairKey = (source, target) =>
   source < target ? source + '|' + target : target + '|' + source
 
@@ -1767,6 +1867,44 @@ const layoutDensityScaleForNodeCount = (nodeCount) => {
   return 0.88
 }
 
+const createHierarchyGroups = (graph, densityScale) => {
+  const groupRows = Array.isArray(graph.groups) ? graph.groups : []
+
+  return groupRows.map(group => {
+    if (Array.isArray(group)) {
+      const [id, level, parentId, title, x, y, radius, segment, folderGroup, nodeIds, childGroupIds] = group
+      return {
+        id: typeof id === 'string' ? id : '',
+        level: Number.isFinite(level) ? level : 0,
+        parentId: typeof parentId === 'string' ? parentId : null,
+        title: typeof title === 'string' ? title : 'Group',
+        x: Number.isFinite(x) ? x * densityScale : 0,
+        y: Number.isFinite(y) ? y * densityScale : 0,
+        radius: Number.isFinite(radius) ? radius * densityScale : 120,
+        segment: typeof segment === 'string' ? segment : 'root',
+        group: typeof folderGroup === 'string' ? folderGroup : 'root',
+        nodeIds: Array.isArray(nodeIds) ? nodeIds.filter(nodeId => typeof nodeId === 'string') : [],
+        childGroupIds: Array.isArray(childGroupIds) ? childGroupIds.filter(groupId => typeof groupId === 'string') : []
+      }
+    }
+
+    return {
+      ...group,
+      id: typeof group.id === 'string' ? group.id : '',
+      level: Number.isFinite(group.level) ? group.level : 0,
+      parentId: typeof group.parentId === 'string' ? group.parentId : null,
+      title: typeof group.title === 'string' ? group.title : 'Group',
+      x: Number.isFinite(group.x) ? group.x * densityScale : 0,
+      y: Number.isFinite(group.y) ? group.y * densityScale : 0,
+      radius: Number.isFinite(group.radius) ? group.radius * densityScale : 120,
+      segment: typeof group.segment === 'string' ? group.segment : 'root',
+      group: typeof group.group === 'string' ? group.group : 'root',
+      nodeIds: Array.isArray(group.nodeIds) ? group.nodeIds.filter(nodeId => typeof nodeId === 'string') : [],
+      childGroupIds: Array.isArray(group.childGroupIds) ? group.childGroupIds.filter(groupId => typeof groupId === 'string') : []
+    }
+  }).filter(group => group.id)
+}
+
 const createLayout = graph => {
   const nodeRows = Array.isArray(graph.nodes) ? graph.nodes : []
   const edgeRows = Array.isArray(graph.edges) ? graph.edges : []
@@ -1799,6 +1937,7 @@ const createLayout = graph => {
     }
   })
   const nodeMap = new Map(nodes.map(node => [node.id, node]))
+  const groups = createHierarchyGroups(graph, densityScale)
   const edges = edgeRows
     .map(edge => {
       if (Array.isArray(edge)) {
@@ -1815,7 +1954,7 @@ const createLayout = graph => {
     })
     .filter(edge => edge.target && nodeMap.has(edge.source) && nodeMap.has(edge.target))
     .map(edge => ({ ...edge, sourceNode: nodeMap.get(edge.source), targetNode: nodeMap.get(edge.target) }))
-  return { nodes, edges }
+  return { nodes, edges, groups }
 }
 
 const encodeEntityTag = (value) => {
@@ -2438,6 +2577,7 @@ const render = now => {
     ctx.save()
     ctx.translate(state.transform.x, state.transform.y)
     ctx.scale(state.transform.scale, state.transform.scale)
+    drawHierarchyGroups()
     if (drawEdges) {
       drawGraphEdges()
     }
@@ -2843,6 +2983,9 @@ const loadGraph = async (options = { reset: false }) => {
   state.graphSignature = signature
   state.graph = graph
   state.nodes = layout.nodes
+  state.groups = layout.groups
+  state.groupById = new Map(state.groups.map(group => [group.id, group]))
+  state.leafGroups = state.groups.filter(group => group.nodeIds.length > 0)
   state.nodeById = new Map(state.nodes.map((node) => [node.id, node]))
   state.edges = layout.edges
   state.nodeDegrees = state.edges.reduce((degrees, edge) => {

package/dist/application/server/routes.js

@@ -114,11 +114,25 @@ const compactLayoutPayload = (layout) => {
     const compactNodes = layout.nodes.map((node) => [node.id, node.title, node.x, node.y, node.group, node.segment]);
     const compactEdgeRows = compactEdges
         .map((edge) => [edge.source, edge.target, edge.weight, edge.priority]);
+    const compactGroups = layout.groups?.map((group) => [
+        group.id,
+        group.level,
+        group.parentId,
+        group.title,
+        group.x,
+        group.y,
+        group.radius,
+        group.segment,
+        group.group,
+        group.nodeIds,
+        group.childGroupIds
+    ]);
     return {
         compact: true,
         layout: {
             nodes: compactNodes,
-            edges: compactEdgeRows
+            edges: compactEdgeRows,
+            ...(compactGroups && compactGroups.length > 0 ? { groups: compactGroups } : {})
         },
         totals: {
             nodes: layout.nodes.length,

package/dist/domain/graph-layout.js

@@ -1,3 +1,4 @@
+const hierarchyGroupNodeLimit = 1000;
 const groupLabels = {
     '00-maps': 'maps',
     '10-agent-memory': 'agent-memory',
@@ -192,6 +193,132 @@ const createSegmentNodes = (segments, degrees, segmentCount) => ([segment, nodes
     });
 };
 const distanceBetween = (left, right) => Math.hypot(right.x - left.x, right.y - left.y);
+const layoutBounds = (nodes) => {
+    if (nodes.length === 0) {
+        return { x: 0, y: 0, radius: 1 };
+    }
+    const bounds = nodes.reduce((current, node) => ({
+        minX: Math.min(current.minX, node.x),
+        maxX: Math.max(current.maxX, node.x),
+        minY: Math.min(current.minY, node.y),
+        maxY: Math.max(current.maxY, node.y)
+    }), {
+        minX: Number.POSITIVE_INFINITY,
+        maxX: Number.NEGATIVE_INFINITY,
+        minY: Number.POSITIVE_INFINITY,
+        maxY: Number.NEGATIVE_INFINITY
+    });
+    const x = (bounds.minX + bounds.maxX) / 2;
+    const y = (bounds.minY + bounds.maxY) / 2;
+    const radius = nodes.reduce((largest, node) => Math.max(largest, Math.hypot(node.x - x, node.y - y)), 1);
+    return { x, y, radius: Math.max(radius + 72, 120) };
+};
+const edgeTouchesGroup = (edge, nodeIds) => nodeIds.has(edge.source) || Boolean(edge.target && nodeIds.has(edge.target));
+const edgeInsideGroup = (edge, nodeIds) => nodeIds.has(edge.source) && Boolean(edge.target && nodeIds.has(edge.target));
+const groupTitle = (segment, level, index, nodes) => nodes.length === 1
+    ? nodes[0]?.title ?? segment
+    : `${segment} ${level + 1}.${index + 1}`;
+const chunkNodes = (nodes, degrees, groupNodeLimit = hierarchyGroupNodeLimit) => {
+    const sortedNodes = [...nodes].sort((left, right) => {
+        const segmentDelta = left.segment.localeCompare(right.segment);
+        if (segmentDelta !== 0)
+            return segmentDelta;
+        const groupDelta = left.group.localeCompare(right.group);
+        if (groupDelta !== 0)
+            return groupDelta;
+        const degreeDelta = (degrees.get(right.id) ?? 0) - (degrees.get(left.id) ?? 0);
+        if (degreeDelta !== 0)
+            return degreeDelta;
+        return left.title.localeCompare(right.title);
+    });
+    const chunks = [];
+    for (let index = 0; index < sortedNodes.length; index += groupNodeLimit) {
+        chunks.push(sortedNodes.slice(index, index + groupNodeLimit));
+    }
+    return chunks;
+};
+const groupEdges = (edges, nodeIds) => ({
+    internalEdges: edges.filter((edge) => edgeInsideGroup(edge, nodeIds)),
+    externalEdges: edges.filter((edge) => edgeTouchesGroup(edge, nodeIds) && !edgeInsideGroup(edge, nodeIds))
+});
+const groupBounds = (groups) => {
+    if (groups.length === 0) {
+        return { x: 0, y: 0, radius: 1 };
+    }
+    const nodes = groups.map((group) => ({
+        x: group.x,
+        y: group.y,
+        radius: group.radius
+    }));
+    const x = nodes.reduce((sum, node) => sum + node.x, 0) / nodes.length;
+    const y = nodes.reduce((sum, node) => sum + node.y, 0) / nodes.length;
+    const radius = nodes.reduce((largest, node) => Math.max(largest, Math.hypot(node.x - x, node.y - y) + node.radius), 1);
+    return { x, y, radius: Math.max(radius + 120, 180) };
+};
+const descendantNodeIds = (groups) => groups.flatMap((group) => group.nodeIds);
+const createParentGroups = (groups, edges, level, groupNodeLimit) => {
+    if (groups.length <= groupNodeLimit) {
+        return groups;
+    }
+    const parentGroups = [];
+    for (let index = 0; index < groups.length; index += groupNodeLimit) {
+        const chunk = groups.slice(index, index + groupNodeLimit);
+        const nodeIds = new Set(descendantNodeIds(chunk));
+        const bounds = groupBounds(chunk);
+        const segment = chunk[0]?.segment ?? 'root';
+        const group = chunk[0]?.group ?? 'root';
+        const groupIndex = index / groupNodeLimit;
+        const id = ['root', level, groupIndex, chunk[0]?.id ?? 'empty', chunk.length].join(':');
+        const edgeGroups = groupEdges(edges, nodeIds);
+        parentGroups.push({
+            id,
+            level,
+            parentId: null,
+            title: `${segment} ${level + 1}.${Math.floor(groupIndex) + 1}`,
+            segment,
+            group,
+            x: bounds.x,
+            y: bounds.y,
+            radius: bounds.radius,
+            nodeIds: [],
+            childGroupIds: chunk.map((child) => child.id),
+            internalEdges: edgeGroups.internalEdges,
+            externalEdges: edgeGroups.externalEdges
+        });
+    }
+    const relinkedChildren = groups.map((group) => {
+        const parent = parentGroups.find((candidate) => candidate.childGroupIds.includes(group.id));
+        return parent ? { ...group, parentId: parent.id } : group;
+    });
+    return [...createParentGroups(parentGroups, edges, level + 1, groupNodeLimit), ...relinkedChildren];
+};
+export const createGraphLayoutHierarchy = (nodes, edges, degrees, groupNodeLimit = hierarchyGroupNodeLimit) => {
+    if (nodes.length <= groupNodeLimit) {
+        return [];
+    }
+    const leafGroups = chunkNodes(nodes, degrees, groupNodeLimit).map((chunk, index) => {
+        const nodeIds = new Set(chunk.map((node) => node.id));
+        const bounds = layoutBounds(chunk);
+        const segment = chunk[0]?.segment ?? 'root';
+        const group = chunk[0]?.group ?? 'root';
+        const id = ['leaf', 0, index, chunk[0]?.id ?? 'empty', chunk.length].join(':');
+        return {
+            id,
+            level: 0,
+            parentId: null,
+            title: groupTitle(segment, 0, index, chunk),
+            segment,
+            group,
+            x: bounds.x,
+            y: bounds.y,
+            radius: bounds.radius,
+            nodeIds: chunk.map((node) => node.id),
+            childGroupIds: [],
+            ...groupEdges(edges, nodeIds)
+        };
+    });
+    return createParentGroups(leafGroups, edges, 1, groupNodeLimit);
+};
 const resolveCollisionPair = (left, right, minDistance) => {
     const dx = right.x - left.x;
     const dy = right.y - left.y;
@@ -291,9 +418,11 @@ export const createCauliflowerGraphLayout = (graph) => {
     const nodes = relaxCollisions(segmentGroups.flatMap(createSegmentNodes(segments, degrees, segmentGroups.length)));
     const primaryHubId = selectPrimaryHubId(graph.nodes, degrees);
     const centeredNodes = centerLayoutByNode(nodes, primaryHubId);
+    const groups = createGraphLayoutHierarchy(centeredNodes, graph.edges, degrees);
     return {
         nodes: centeredNodes,
-        edges: graph.edges
+        edges: graph.edges,
+        ...(groups.length > 0 ? { groups } : {})
     };
 };
 export const getMinimumLayoutDistance = (nodes) => nodes.reduce((minimumDistance, leftNode, leftIndex) => nodes.slice(leftIndex + 1).reduce((innerMinimum, rightNode) => Math.min(innerMinimum, distanceBetween(leftNode, rightNode)), minimumDistance), Number.POSITIVE_INFINITY);

package/docs/AGENT_USAGE.md

@@ -607,7 +607,7 @@ Without `--vault`, the graph UI serves `$HOME/.brainlink/vault`.
 
 The frontend includes an agent selector that shows only the agent id. Selecting an agent calls the same read APIs with `agent=<agent-id>` and renders that namespace instead of merging every agent into one graph.
 
-Graph navigation controls include zoom in, zoom out, fit visible nodes and reset-to-fit-all nodes. Mouse wheel zoom (including `cmd+scroll` and `ctrl+scroll`) is anchored to the cursor. Keyboard shortcuts are `+` (zoom in), `-` (zoom out) and `0` (reset fit). Double-click on canvas zooms in at cursor position. Totals for notes, links and tags stay visible as floating metrics under the Brainlink title, and node details open on click in a modal (tags, outgoing links, backlinks and Markdown content).
+Graph navigation controls include zoom in, zoom out, fit visible nodes and reset-to-fit-all nodes. Mouse wheel zoom (including `cmd+scroll` and `ctrl+scroll`) is anchored to the cursor. Keyboard shortcuts are `+` (zoom in), `-` (zoom out) and `0` (reset fit). Double-click on canvas zooms in at cursor position. Totals for notes, links and tags stay visible as floating metrics under the Brainlink title, and node details open on click in a modal (tags, outgoing links, backlinks and Markdown content). Vaults above 1000 notes also expose stable hierarchy groups of up to 1000 direct nodes, with recursive parent groups when a level exceeds 1000 groups; zoom-out draws those groups as context while preserving the real-node scene.
 During graph filtering, Brainlink keeps hub context nodes visible (`Memory Hub`/`MOC`/high-degree fallback) so filtered views still show relationship anchors.
 
 The command reindexes by default, then serves:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@andespindola/brainlink",
-  "version": "0.1.0-beta.118",
+  "version": "0.1.0-beta.119",
   "description": "Local-first knowledge memory for agents with Markdown, backlinks, indexing and context retrieval.",
   "type": "module",
   "license": "MIT",