@utaba/deep-memory-storage-neo4j 0.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,190 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to the Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by the Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding any notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ Copyright 2024 Tim Wheeler
179
+
180
+ Licensed under the Apache License, Version 2.0 (the "License");
181
+ you may not use this file except in compliance with the License.
182
+ You may obtain a copy of the License at
183
+
184
+ http://www.apache.org/licenses/LICENSE-2.0
185
+
186
+ Unless required by applicable law or agreed to in writing, software
187
+ distributed under the License is distributed on an "AS IS" BASIS,
188
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
189
+ See the License for the specific language governing permissions and
190
+ limitations under the License.
package/README.md ADDED
@@ -0,0 +1,337 @@
1
+ # @utaba/deep-memory-storage-neo4j
2
+
3
+ Neo4j storage provider for [`@utaba/deep-memory`](https://www.npmjs.com/package/@utaba/deep-memory). Implements both `StorageProvider` and `GraphTraversalProvider` against Neo4j Community Edition over Bolt — a single instance gives deep-memory persistent storage **and** native Cypher graph queries.
4
+
5
+ ## Installation
6
+
7
+ ```bash
8
+ pnpm add @utaba/deep-memory @utaba/deep-memory-storage-neo4j
9
+ ```
10
+
11
+ **Runtime dependency:** [`neo4j-driver`](https://www.npmjs.com/package/neo4j-driver) (the official Neo4j JavaScript driver, Apache-2.0, types bundled).
12
+
13
+ ## Quick Start
14
+
15
+ ```typescript
16
+ import { DeepMemory } from '@utaba/deep-memory';
17
+ import { Neo4jStorageProvider } from '@utaba/deep-memory-storage-neo4j';
18
+
19
+ const provider = new Neo4jStorageProvider({
20
+ uri: 'bolt://localhost:7687',
21
+ username: 'neo4j',
22
+ password: 'DeepMem-Dev-1234',
23
+ database: 'neo4j',
24
+ });
25
+
26
+ await provider.initialize(); // verifies connectivity
27
+ await provider.ensureSchema(); // creates constraints + indexes (idempotent)
28
+
29
+ const dm = new DeepMemory({
30
+ storage: provider,
31
+ graphTraversal: provider, // same instance — implements both interfaces
32
+ });
33
+ ```
34
+
35
+ For local development with Docker, see [Local development setup](#local-development-setup) below.
36
+
37
+ ## Configuration
38
+
39
+ ### `Neo4jStorageProviderConfig`
40
+
41
+ | Option | Type | Default | Description |
42
+ |--------|------|---------|-------------|
43
+ | `uri` | `string` | *required* | Bolt URI. `bolt://` for plain TCP, `bolt+s://` for TLS, `neo4j://` for routed clusters, `neo4j+s://` for AuraDB. |
44
+ | `username` | `string` | *required* | Basic-auth username. |
45
+ | `password` | `string` | *required* | Basic-auth password. |
46
+ | `database` | `string` | `'neo4j'` | Database name. The driver manual recommends specifying this explicitly even on Community Edition single-database instances. |
47
+ | `userAgent` | `string` | `'@utaba/deep-memory-storage-neo4j'` | User-agent string sent on the Bolt handshake. |
48
+ | `maxTransactionRetryTime` | `number` | driver default | Maximum time (ms) the driver will retry a managed transaction on transient errors. |
49
+ | `reportUsage` | `UsageSink` | `undefined` | Optional sink invoked once per public method call with the server-side time (ms) consumed. See [Usage tracking](#usage-tracking). |
50
+ | `profileTraversals` | `boolean` | `false` | When `true`, prepends `PROFILE` to compiled traversal queries and surfaces the plan summary on the sink record. `PROFILE` more than doubles wall-clock on short traversals — turn it on only when actively investigating planner behaviour. |
51
+
52
+ The provider holds a single Neo4j `Driver` per instance, per the driver's documented "create once, share, close on shutdown" lifecycle.
53
+
54
+ ## Lifecycle
55
+
56
+ ```typescript
57
+ const provider = new Neo4jStorageProvider({ ... });
58
+
59
+ await provider.initialize(); // verifyConnectivity over Bolt
60
+ await provider.ensureSchema(); // CREATE CONSTRAINT/INDEX … IF NOT EXISTS
61
+
62
+ const dm = new DeepMemory({ storage: provider, graphTraversal: provider });
63
+ // ... use ...
64
+
65
+ await provider.dispose(); // closes the Bolt driver
66
+ ```
67
+
68
+ `ensureSchema()` runs constraint and index DDL idempotently against the configured database and writes a `_Meta` schema-version handshake. Subsequent calls detect the existing schema and return early. It does **not** create the database itself — Neo4j Community Edition has a single user database; the operator is responsible for the target database existing before the provider connects.
69
+
70
+ ## Data Model
71
+
72
+ ### Multi-tenancy via `repositoryId`
73
+
74
+ Neo4j Community Edition has a single user database, so multiple repositories share one Neo4j database and are isolated by a `repositoryId` property on every node and edge. **Every Cypher statement** issued by this provider — apart from a small allowlist of system queries (`ensureSchema`, `listRepositories`, `_Meta` reads) — carries a required `$rid` parameter and references it in a predicate. The `Neo4jConnection` chokepoint enforces this at runtime: a Cypher string that omits `$rid` raises `ProviderError`, and no other file in the package is allowed to touch the driver directly.
75
+
76
+ Operators who need physical isolation between tenants can run one Neo4j instance per tenant and create one `Neo4jStorageProvider` per URI — that is an operations choice, not a provider feature.
77
+
78
+ ### Label scheme
79
+
80
+ | Node kind | Labels | Notes |
81
+ |-----------|--------|-------|
82
+ | Entity | `:_Entity` | Single umbrella label. The entity type lives in `n.entityType` (indexed). Per-type labels are deliberately **not** written — the steady-state per-call cost of interpolating a parameter into the label slot is not worth the query-convenience benefit. |
83
+ | Repository | `:_Repository` | One node per repository. |
84
+ | Vocabulary | `:_Vocabulary` | One node per repository; stores the vocabulary as a JSON string. |
85
+ | Vocabulary change log | `:_VocabularyChangeLog` | Append-only audit trail. |
86
+ | Schema meta | `:_Meta` | Singleton; carries `schemaVersion`. |
87
+
88
+ Relationship types in Cypher are the vocabulary relationship type slug, uppercased per Cypher convention (e.g. `:KNOWS`, `:REPORTS_TO`). Stored on `StoredRelationship.type` verbatim — the provider applies a deterministic case transform at the boundary.
89
+
90
+ ### Property storage
91
+
92
+ | Data | Storage | Notes |
93
+ |------|---------|-------|
94
+ | Schema-managed scalars (`entityType`, `slug`, provenance fields, timestamps) | Native Neo4j properties on the node | Indexed where appropriate. Timestamps are ISO-8601 strings — the driver does not auto-convert `Date` ↔ string, so keeping strings avoids a conversion dance on every read/write. |
95
+ | User-supplied entity properties | **Both** native Neo4j scalars (one property per key) **and** a `properties` JSON string | Native scalars exist so `findEntities` predicates resolve to exact server-side equality checks. The JSON blob remains authoritative for round-trip fidelity — values Neo4j cannot represent natively (nested objects, `null`, heterogeneous arrays) preserve their shape via the blob but are **not** predicate-queryable. User keys are validated against the bare-Cypher-identifier shape and the reserved schema-field list on every write. |
96
+ | Embeddings | Native `list<float>` on the node (`embedding`) | Pass-through, no JSON encoding step. Excluded from read projections unless `loadEmbeddings: true`. |
97
+ | Vocabulary | Single JSON string on the `_Vocabulary` node | Cached in-process for 60 s (see [Vocabulary cache](#vocabulary-cache)). |
98
+
99
+ ### Schema DDL
100
+
101
+ `ensureSchema()` runs the following statements idempotently. Composite indexes lead with `repositoryId` so the planner picks it as the cheap discriminator.
102
+
103
+ ```cypher
104
+ CREATE CONSTRAINT dm_entity_unique IF NOT EXISTS
105
+ FOR (n:_Entity) REQUIRE (n.repositoryId, n.id) IS UNIQUE;
106
+
107
+ CREATE CONSTRAINT dm_entity_slug_unique IF NOT EXISTS
108
+ FOR (n:_Entity) REQUIRE (n.repositoryId, n.slug) IS UNIQUE;
109
+
110
+ CREATE CONSTRAINT dm_repository_unique IF NOT EXISTS
111
+ FOR (n:_Repository) REQUIRE n.repositoryId IS UNIQUE;
112
+
113
+ CREATE INDEX dm_entity_type_lookup IF NOT EXISTS
114
+ FOR (n:_Entity) ON (n.repositoryId, n.entityType);
115
+
116
+ CREATE INDEX dm_entity_modified IF NOT EXISTS
117
+ FOR (n:_Entity) ON (n.repositoryId, n.modifiedAt);
118
+
119
+ CREATE FULLTEXT INDEX dm_entity_text IF NOT EXISTS
120
+ FOR (n:_Entity) ON EACH [n.label, n.summary];
121
+ ```
122
+
123
+ All constraints and indexes are supported on Neo4j Community Edition. No Enterprise-only features (property-existence, property-type, node-key, or relationship-key constraints, multi-database) are used.
124
+
125
+ To inspect the statements without connecting:
126
+
127
+ ```typescript
128
+ import { getSchemaCypher, SCHEMA_VERSION } from '@utaba/deep-memory-storage-neo4j';
129
+
130
+ const statements = getSchemaCypher(); // string[]
131
+ ```
132
+
133
+ ### Vocabulary cache
134
+
135
+ `getVocabulary` reads through a 60-second in-process cache (per `repositoryId`). Vocabulary is compile-time context for graph traversal and changes rarely; the cache turns the hot path into zero round-trips. Cross-process staleness is bounded by the 60 s TTL; writes inside this process invalidate immediately.
136
+
137
+ ## Search behaviour (`findEntities`)
138
+
139
+ Every filter shape resolves to an exact server-side predicate; `total` is always exact (no `total: undefined` escape hatch). The data and count queries share the same `WHERE` fragment by construction, so they count the same set.
140
+
141
+ | Filter shape | How it resolves |
142
+ |-------------|------------------|
143
+ | `entityTypes` | Predicate on `n.entityType`, backed by `dm_entity_type_lookup`. |
144
+ | `searchTerm` | Routes through `CALL db.index.fulltext.queryNodes('dm_entity_text', $term) YIELD node, score WHERE node.repositoryId = $rid …`. Lucene query syntax flows through `$term`. |
145
+ | `query.properties` | Server-side exact `n.<key> = $val` against native-scalar copies of user properties. Non-storable filter values (nested objects, `null`, heterogeneous arrays) raise `ProviderError` at predicate-build time rather than silently missing matches. |
146
+ | `provenance.actors` | `(n.createdBy IN $actors OR n.modifiedBy IN $actors)`. |
147
+ | `provenance.conversationIds` | `(n.createdInConversation IN $convIds OR n.modifiedInConversation IN $convIds)`. |
148
+ | `provenance.dateRange` | ISO-8601 string comparison on `createdAt` / `modifiedAt` — chronologically correct because the canonical Z-suffixed format compares lexicographically. |
149
+
150
+ ### Fulltext vs `CONTAINS`
151
+
152
+ The search branch ships the fulltext-index path only — no `WHERE … CONTAINS` fallback. Measured behaviour on `neo4j:5-community`:
153
+
154
+ - At ~1k entities, `CONTAINS` keeps up to within ~0.7 ms.
155
+ - At 10k entities the fulltext path is uniformly 3–6× faster (the gap widens with cohort size because `CONTAINS` is O(N) in entities while fulltext is O(matches)).
156
+
157
+ A dual-path branch was rejected — the small win at 1k disappears as cohorts grow and the extra code surface is not worth carrying.
158
+
159
+ Note that fulltext is token-based (Lucene). Sub-token matches like `alph` matching `alpha` would work under `CONTAINS` but **not** under tokenised fulltext. This is by design — the schema's intent is term-based search.
160
+
161
+ ## Graph traversal capabilities
162
+
163
+ `Neo4jStorageProvider` implements `GraphTraversalProvider` and reports:
164
+
165
+ | Capability | Value |
166
+ |-----------|-------|
167
+ | `supportsNativeQuery` | `true` |
168
+ | `nativeQueryLanguage` | `'cypher'` |
169
+ | `maxTraversalDepth` | `10` |
170
+ | `supportsRelationshipPropertyFilters` | `true` |
171
+ | `supportsEntityPropertyFilters` | `true` |
172
+ | `supportsAggregation` | `true` |
173
+ | `supportsRepeat` | `true` |
174
+ | `supportsDedup` | `true` |
175
+ | `supportsRelationshipSummary` | `false` |
176
+
177
+ `traverse`, `exploreNeighborhood`, and `findPaths` are all compiled via the shared `CypherCompiler` and submitted as native Cypher. `findPaths` resolves in a single Bolt round-trip via `MATCH p = (s)-[*1..N]-(t)`; edge-uniqueness inside each returned path is automatic in Cypher 25 (default `DIFFERENT RELATIONSHIPS` match mode), so no application-side dedup filter is needed.
178
+
179
+ `QueryMetadata.resourceCost` is populated on every traversal result as `{ units: 'server_ms', value }` — the server-side time the database spent producing the result. With `profileTraversals: true` the result's `details.profile` also carries the `PROFILE` plan summary.
180
+
181
+ ## Bulk operations
182
+
183
+ `exportAll()` returns an async iterable of chunks (batches of 100), entities first then relationships. Pagination is cursor-based (`WHERE n.id > $cursor ORDER BY n.id LIMIT $batchSize`) rather than `SKIP`/`LIMIT`, so reads stay O(n) instead of O(n²) on large repositories. Embeddings are included in export projections for round-trip fidelity.
184
+
185
+ ```typescript
186
+ for await (const chunk of provider.exportAll(repositoryId)) {
187
+ // chunk.type: 'entities' | 'relationships'
188
+ // chunk.data: StoredEntity[] | StoredRelationship[]
189
+ // chunk.isLast: boolean
190
+ }
191
+ ```
192
+
193
+ `importBulk()` uses fixed-shape `UNWIND` templates — one Cypher string per chunk regardless of contents — so the plan cache stays at a single entry per import. Default chunk size is 100; concurrency is a simple bounded pool (default 8) — Neo4j Community has no per-query cost limit, so there is no adaptive controller. Use `skipExistenceCheck: true` when the caller knows the data is fresh (faster `CREATE` path); leave it `false` for idempotent `MERGE`-based upsert.
194
+
195
+ ## Native query escape hatch
196
+
197
+ `executeNativeQuery(repositoryId, cypher, params)` runs a raw Cypher statement through the provider's connection. This bypasses the repository-scoping discipline that the rest of the provider enforces — the caller is fully responsible for scoping the query themselves.
198
+
199
+ **Do not expose this method to AI-agent-facing surfaces.** It exists for admin tooling and migration scripts only; the MCP server intentionally does not surface it.
200
+
201
+ ## Error handling
202
+
203
+ All errors use the `@utaba/deep-memory` error hierarchy. Mapping is by `error.code`:
204
+
205
+ | Driver code | Maps to |
206
+ |-------------|---------|
207
+ | `Neo.ClientError.Schema.ConstraintValidationFailed` (entity scope) | `DuplicateEntityError` |
208
+ | `Neo.ClientError.Schema.ConstraintValidationFailed` (relationship scope) | `DuplicateRelationshipError` |
209
+ | `Neo.ClientError.Schema.ConstraintValidationFailed` (repository scope) | `DuplicateRepositoryError` |
210
+ | `Neo.ClientError.Statement.SyntaxError` | `ProviderError` |
211
+ | `Neo.ClientError.Security.*` | `ProviderError` (original code attached) |
212
+ | Anything else | `ProviderError` with `cause: error` |
213
+
214
+ "Not found" outcomes (`EntityNotFoundError`, `RelationshipNotFoundError`, `RepositoryNotFoundError`) come from inspecting the result summary's counters — they are not driver errors.
215
+
216
+ Transient errors are retried automatically by `driver.executeQuery` and `session.executeWrite/Read`; the provider does not check `error.isRetryable()` itself on those code paths.
217
+
218
+ When `summary.gqlStatusObjects` carries a non-`INFORMATION` notification (missing index, cartesian product, deprecation), the connection emits a single `console.warn` with the truncated query text and the notification list. The sink record's `details` does not carry the full notification array — keeps the sink shape bounded.
219
+
220
+ ## Usage tracking
221
+
222
+ When `reportUsage` is supplied, the provider emits one `OperationUsage` record per public method call:
223
+
224
+ ```typescript
225
+ {
226
+ provider: 'neo4j',
227
+ operation: 'findEntities',
228
+ unit: 'server_ms',
229
+ value: 12, // sum of summary.resultConsumedAfter across all round-trips
230
+ repositoryId: 'my-repo',
231
+ timestamp: new Date(),
232
+ details: {
233
+ calls: 2, // round-trips inside the operation
234
+ retries: 0,
235
+ recordCount: 47,
236
+ counters: { … }, // aggregated nodesCreated, relationshipsCreated, etc.
237
+ availableAfterMs: 8,
238
+ profile: { … }, // present only when profileTraversals: true
239
+ },
240
+ }
241
+ ```
242
+
243
+ `server_ms` is the Neo4j-native equivalent of CosmosDB's RU — it is the time the server spent producing the result. See [docs/usage-tracking.md](../../docs/usage-tracking.md) for how to wire a sink for billing, rate limiting, or observability.
244
+
245
+ ## Local development setup
246
+
247
+ The repo ships a `docker-compose.neo4j.yml` at its root:
248
+
249
+ ```bash
250
+ docker compose -f docker-compose.neo4j.yml up -d
251
+ ```
252
+
253
+ This starts `neo4j:5.26-community` with:
254
+
255
+ - Bolt on `7687`
256
+ - Browser UI on `http://localhost:7474`
257
+ - Credentials: `neo4j` / `DeepMem-Dev-1234`
258
+ - APOC plugin installed (not used by the provider, but useful for ad-hoc admin work)
259
+
260
+ The default password is for local development only — change it before exposing the instance to anything other than localhost.
261
+
262
+ ## AuraDB / production deployment
263
+
264
+ The `neo4j+s://` URI scheme works against AuraDB out of the box:
265
+
266
+ ```typescript
267
+ const provider = new Neo4jStorageProvider({
268
+ uri: 'neo4j+s://<dbid>.databases.neo4j.io',
269
+ username: 'neo4j',
270
+ password: process.env.NEO4J_PASSWORD!,
271
+ database: 'neo4j',
272
+ });
273
+ ```
274
+
275
+ AuraDB-specific test coverage (cert pinning, IAM-style auth) is deferred — file an issue if you need it.
276
+
277
+ ## Differences from the CosmosDB provider
278
+
279
+ Operators familiar with `@utaba/deep-memory-storage-cosmosdb` should know the following are intentional:
280
+
281
+ | Topic | CosmosDB provider | Neo4j provider | Why the difference |
282
+ |-------|-------------------|----------------|---------------------|
283
+ | Multi-tenant isolation | One CosmosDB partition per repository | `repositoryId` property on every node/edge with a connection-layer chokepoint | Neo4j has no partition model. Property-scoping with composite indexes is the idiomatic Cypher approach; a root-`(:_Repository)-[:CONTAINS]->(:_Entity)` pattern would create supernodes (an anti-pattern). |
284
+ | Repository listing | Sentinel `_repository_index` vertex in `_index` partition | Direct `MATCH (r:_Repository) RETURN r` | No partition fan-out cost to amortise — the sentinel exists in Cosmos *because of* its cost model. |
285
+ | `findEntities` totals | `total: number \| undefined` depending on filter shape | Always exact `total: number` | Cypher's `count(n)` runs as a parallel server-side aggregation against the same `WHERE` fragment. |
286
+ | Search backend | Slug-based `TextP.containing()` | Fulltext index via `db.index.fulltext.queryNodes` | Neo4j has a first-class fulltext index; Cosmos's Gremlin subset does not. |
287
+ | `getRepositoryStats` | Gremlin `.group().by(label).by(count())` per metric | Native Cypher `count()` aggregation | Cypher's aggregation is direct; Gremlin's path is gymnastic. |
288
+ | Bulk import concurrency | Adaptive controller that dials down on 429s | Fixed bounded pool (default 8) | Neo4j has no per-query cost limit and no equivalent throttle signal — adaptation has nothing to react to. |
289
+ | Usage unit | `RU` | `server_ms` | The Neo4j-native cost-adjacent signal is `summary.resultConsumedAfter`. |
290
+ | `findPaths` | Application-level BFS / Gremlin `repeat().emit()` | Single `MATCH p = (s)-[*1..N]-(t)` | Cypher's variable-length pattern resolves in one round-trip; edge-uniqueness within a path is automatic in Cypher 25. |
291
+ | Greedy-expand on traverse pages | Required (Gremlin streams nodes and edges into a single deduped stream that `.range()` slices by element) | Not needed | Cypher's `MATCH` binds endpoints to relationships at MATCH time and `LIMIT` slices whole rows, so an endpoint can never fall outside a page without its row going with it. |
292
+
293
+ The two providers share the same `StorageProvider` / `GraphTraversalProvider` contract — application code is portable between them.
294
+
295
+ ## Testing
296
+
297
+ The conformance suite is gated on `NEO4J_URI`:
298
+
299
+ ```bash
300
+ NEO4J_URI=bolt://localhost:7687 \
301
+ NEO4J_USERNAME=neo4j \
302
+ NEO4J_PASSWORD=DeepMem-Dev-1234 \
303
+ pnpm --filter @utaba/deep-memory-storage-neo4j test
304
+ ```
305
+
306
+ Without `NEO4J_URI`, the live tests are skipped. The pure-unit tests (mapping, schema snapshot, isolation chokepoint guards) always run.
307
+
308
+ ## Licensing
309
+
310
+ | Component | License | Notes |
311
+ |-----------|---------|-------|
312
+ | `@utaba/deep-memory-storage-neo4j` (this package) | **Apache-2.0** | Same as the rest of the monorepo. |
313
+ | `neo4j-driver` (npm runtime dependency) | **Apache-2.0** | TypeScript types bundled in the package — no separate `@types/neo4j-driver` needed. |
314
+ | `neo4j:5-community` (Docker image referenced for local dev) | **GPLv3** (binary) | The Dockerfile scripts are Apache-2.0; the binary itself is GPLv3. |
315
+
316
+ This package speaks to Neo4j over the Bolt protocol — that is mere aggregation, the same model that has allowed GPLv2/v3 database clients to ship inside non-GPL applications for the last twenty years. Your application linking this package does **not** bring GPL obligations.
317
+
318
+ If you bundle or redistribute the Neo4j binary inside your own distribution, GPLv3 obligations on the binary attach to *your* distribution, not to this package. We **do not** depend on or reference Neo4j Enterprise (no `-enterprise` tags, no Enterprise-only features such as property-existence / property-type / node-key / relationship-key constraints, multi-database, or fine-grained role auth).
319
+
320
+ ## Exports
321
+
322
+ ```typescript
323
+ import {
324
+ Neo4jStorageProvider,
325
+ getSchemaCypher,
326
+ SCHEMA_VERSION,
327
+ } from '@utaba/deep-memory-storage-neo4j';
328
+
329
+ import type { Neo4jStorageProviderConfig } from '@utaba/deep-memory-storage-neo4j';
330
+ ```
331
+
332
+ ## See also
333
+
334
+ - [Architecture](../../docs/architecture.md) — Component architecture and provider interface contracts.
335
+ - [Usage tracking](../../docs/usage-tracking.md) — Wiring a `UsageSink` for billing, rate limiting, or observability.
336
+ - [CosmosDB Storage Provider](../storage-cosmosdb/README.md) — The dual-interface graph-native provider this one most closely parallels.
337
+ - [SQL Server Storage Provider](../storage-sqlserver/README.md) — The relational alternative when a graph database is not available.