@pentatonic-ai/ai-agent-sdk 0.9.5 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/bin/cli.js +1 -1
- package/bin/commands/config.js +1 -1
- package/dist/index.cjs +39 -72
- package/dist/index.js +36 -69
- package/package.json +10 -3
- package/packages/doctor/src/checks/local-memory.js +2 -2
- package/packages/memory/README.md +2 -2
- package/packages/memory/openclaw-plugin/README.md +2 -2
- package/packages/memory/openclaw-plugin/openclaw.plugin.json +1 -1
- package/packages/memory/package-lock.json +49 -33
- package/packages/memory/package.json +4 -1
- package/packages/memory/src/__tests__/engine.test.js +40 -5
- package/packages/memory/src/engine.js +38 -3
- package/packages/memory/src/server.js +2 -2
- package/packages/memory-engine-v2/.env.example +30 -0
- package/packages/memory-engine-v2/README.md +125 -0
- package/packages/memory-engine-v2/compat/Dockerfile +11 -0
- package/packages/memory-engine-v2/compat/requirements.txt +6 -0
- package/packages/memory-engine-v2/compat/server.py +1047 -0
- package/packages/memory-engine-v2/docker-compose.aws.yml +78 -0
- package/packages/memory-engine-v2/docker-compose.yml +206 -0
- package/packages/memory-engine-v2/extractor-async/Dockerfile +14 -0
- package/packages/memory-engine-v2/extractor-async/confidence.py +62 -0
- package/packages/memory-engine-v2/extractor-async/noise_filter.py +144 -0
- package/packages/memory-engine-v2/extractor-async/requirements.txt +2 -0
- package/packages/memory-engine-v2/extractor-async/test_confidence.py +76 -0
- package/packages/memory-engine-v2/extractor-async/test_noise_filter.py +177 -0
- package/packages/memory-engine-v2/extractor-async/worker.py +797 -0
- package/packages/memory-engine-v2/extractor-sync/Dockerfile +11 -0
- package/packages/memory-engine-v2/extractor-sync/requirements.txt +4 -0
- package/packages/memory-engine-v2/extractor-sync/server.py +424 -0
- package/packages/memory-engine-v2/org-model/migrations/001_init.sql +390 -0
- package/packages/memory-engine-v2/tests/e2e_smoke.py +356 -0
- package/packages/memory-engine-v2/tests/fixtures/generate_synthetic_corpus.py +758 -0
- package/packages/memory-engine/.env.example +0 -13
- package/packages/memory-engine/MIGRATION.md +0 -219
- package/packages/memory-engine/README.md +0 -145
- package/packages/memory-engine/bench/README.md +0 -99
- package/packages/memory-engine/bench/scorecards-engine/agent-coding__pentatonic-baseline__20260427-142523.json +0 -1115
- package/packages/memory-engine/bench/scorecards-engine/chat-recall__pentatonic-baseline__20260427-142648.json +0 -819
- package/packages/memory-engine/bench/scorecards-engine/circular-economy__pentatonic-baseline__20260427-142757.json +0 -1278
- package/packages/memory-engine/bench/scorecards-engine/customer-support__pentatonic-baseline__20260427-142900.json +0 -1018
- package/packages/memory-engine/bench/scorecards-engine/marketplace-ops__pentatonic-baseline__20260427-142957.json +0 -1038
- package/packages/memory-engine/bench/scorecards-engine/product-catalogue__pentatonic-baseline__20260427-143122.json +0 -961
- package/packages/memory-engine/bench/scorecards-engine-via-docker/agent-coding__pentatonic-memory__20260427-161812.json +0 -1115
- package/packages/memory-engine/bench/scorecards-engine-via-docker/chat-recall__pentatonic-memory__20260427-161701.json +0 -819
- package/packages/memory-engine/bench/scorecards-engine-via-docker/circular-economy__pentatonic-memory__20260427-161713.json +0 -1278
- package/packages/memory-engine/bench/scorecards-engine-via-docker/customer-support__pentatonic-memory__20260427-161723.json +0 -1018
- package/packages/memory-engine/bench/scorecards-engine-via-docker/marketplace-ops__pentatonic-memory__20260427-161732.json +0 -1038
- package/packages/memory-engine/bench/scorecards-engine-via-docker/product-catalogue__pentatonic-memory__20260427-161741.json +0 -937
- package/packages/memory-engine/bench/scorecards-engine-via-l2-7-layer-populated/agent-coding__pentatonic-memory__20260427-184718.json +0 -1115
- package/packages/memory-engine/bench/scorecards-engine-via-l2-7-layer-populated/chat-recall__pentatonic-memory__20260427-184614.json +0 -819
- package/packages/memory-engine/bench/scorecards-engine-via-l2-7-layer-populated/circular-economy__pentatonic-memory__20260427-184809.json +0 -1278
- package/packages/memory-engine/bench/scorecards-engine-via-l2-7-layer-populated/customer-support__pentatonic-memory__20260427-184854.json +0 -1018
- package/packages/memory-engine/bench/scorecards-engine-via-l2-7-layer-populated/marketplace-ops__pentatonic-memory__20260427-184929.json +0 -1038
- package/packages/memory-engine/bench/scorecards-engine-via-l2-7-layer-populated/product-catalogue__pentatonic-memory__20260427-185015.json +0 -961
- package/packages/memory-engine/bench/scorecards-engine-via-l2-empty-layers/agent-coding__pentatonic-memory__20260427-175252.json +0 -1115
- package/packages/memory-engine/bench/scorecards-engine-via-l2-empty-layers/chat-recall__pentatonic-memory__20260427-175312.json +0 -819
- package/packages/memory-engine/bench/scorecards-engine-via-l2-empty-layers/circular-economy__pentatonic-memory__20260427-175335.json +0 -1278
- package/packages/memory-engine/bench/scorecards-engine-via-l2-empty-layers/customer-support__pentatonic-memory__20260427-175355.json +0 -1018
- package/packages/memory-engine/bench/scorecards-engine-via-l2-empty-layers/marketplace-ops__pentatonic-memory__20260427-175413.json +0 -1038
- package/packages/memory-engine/bench/scorecards-engine-via-l2-empty-layers/product-catalogue__pentatonic-memory__20260427-175430.json +0 -883
- package/packages/memory-engine/bench/scorecards-engine-via-shim/agent-coding__pentatonic-memory__20260427-155409.json +0 -1115
- package/packages/memory-engine/bench/scorecards-engine-via-shim/chat-recall__pentatonic-memory__20260427-155421.json +0 -819
- package/packages/memory-engine/bench/scorecards-engine-via-shim/circular-economy__pentatonic-memory__20260427-155433.json +0 -1278
- package/packages/memory-engine/bench/scorecards-engine-via-shim/customer-support__pentatonic-memory__20260427-155443.json +0 -1018
- package/packages/memory-engine/bench/scorecards-engine-via-shim/marketplace-ops__pentatonic-memory__20260427-155453.json +0 -1038
- package/packages/memory-engine/bench/scorecards-engine-via-shim/product-catalogue__pentatonic-memory__20260427-155503.json +0 -937
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/agent-coding__pentatonic-memory-latest__20260427-145103.json +0 -1115
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/agent-coding__pentatonic-memory__20260427-144909.json +0 -1115
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/chat-recall__pentatonic-memory-latest__20260427-145153.json +0 -819
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/chat-recall__pentatonic-memory__20260427-145120.json +0 -542
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/circular-economy__pentatonic-memory-latest__20260427-145313.json +0 -1278
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/circular-economy__pentatonic-memory__20260427-145207.json +0 -894
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/customer-support__pentatonic-memory-latest__20260427-145412.json +0 -1018
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/customer-support__pentatonic-memory__20260427-145327.json +0 -680
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/marketplace-ops__pentatonic-memory-latest__20260427-145517.json +0 -1038
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/marketplace-ops__pentatonic-memory__20260427-145422.json +0 -693
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/product-catalogue__pentatonic-memory-latest__20260427-145616.json +0 -961
- package/packages/memory-engine/bench/scorecards-pentatonic-baseline/product-catalogue__pentatonic-memory__20260427-145528.json +0 -727
- package/packages/memory-engine/compat/Dockerfile +0 -22
- package/packages/memory-engine/compat/server.py +0 -1255
- package/packages/memory-engine/docker-compose.test.yml +0 -59
- package/packages/memory-engine/docker-compose.yml +0 -240
- package/packages/memory-engine/engine/README.md +0 -52
- package/packages/memory-engine/engine/l2-hybridrag-proxy.py +0 -1543
- package/packages/memory-engine/engine/l5-comms-layer.py +0 -663
- package/packages/memory-engine/engine/l6-document-store.py +0 -1018
- package/packages/memory-engine/engine/services/_shared/__init__.py +0 -1
- package/packages/memory-engine/engine/services/_shared/embed_provider.py +0 -468
- package/packages/memory-engine/engine/services/l2/Dockerfile +0 -50
- package/packages/memory-engine/engine/services/l2/init_databases.py +0 -81
- package/packages/memory-engine/engine/services/l2/l2-hybridrag-proxy.py +0 -2721
- package/packages/memory-engine/engine/services/l5/Dockerfile +0 -11
- package/packages/memory-engine/engine/services/l5/l5-comms-layer.py +0 -808
- package/packages/memory-engine/engine/services/l6/Dockerfile +0 -30
- package/packages/memory-engine/engine/services/l6/l6-document-store.py +0 -1221
- package/packages/memory-engine/engine/services/nv-embed/Dockerfile +0 -28
- package/packages/memory-engine/engine/services/nv-embed/server.py +0 -152
- package/packages/memory-engine/pme_memory/__init__.py +0 -0
- package/packages/memory-engine/pme_memory/__main__.py +0 -129
- package/packages/memory-engine/pme_memory/artifacts.py +0 -95
- package/packages/memory-engine/pme_memory/embed.py +0 -74
- package/packages/memory-engine/pme_memory/health.py +0 -36
- package/packages/memory-engine/pme_memory/hygiene.py +0 -159
- package/packages/memory-engine/pme_memory/indexer.py +0 -200
- package/packages/memory-engine/pme_memory/needs.py +0 -55
- package/packages/memory-engine/pme_memory/provenance.py +0 -80
- package/packages/memory-engine/pme_memory/scoring.py +0 -168
- package/packages/memory-engine/pme_memory/search.py +0 -52
- package/packages/memory-engine/pme_memory/store.py +0 -86
- package/packages/memory-engine/pme_memory/synthesis.py +0 -114
- package/packages/memory-engine/pyproject.toml +0 -65
- package/packages/memory-engine/scripts/kg-extractor.py +0 -557
- package/packages/memory-engine/scripts/kg-preflexor-v2.py +0 -738
- package/packages/memory-engine/scripts/wipe-legacy-l3-entities.py +0 -128
- package/packages/memory-engine/tests/e2e_arena.sh +0 -259
- package/packages/memory-engine/tests/embed_stub/Dockerfile +0 -13
- package/packages/memory-engine/tests/embed_stub/server.py +0 -80
- package/packages/memory-engine/tests/test_aggregate.py +0 -333
- package/packages/memory-engine/tests/test_api_contract.sh +0 -57
- package/packages/memory-engine/tests/test_arena_safety.py +0 -232
- package/packages/memory-engine/tests/test_channel_stat_reader.py +0 -437
- package/packages/memory-engine/tests/test_channel_stat_rollups.py +0 -308
- package/packages/memory-engine/tests/test_compat_nv_embed_probe.py +0 -48
- package/packages/memory-engine/tests/test_embed_provider.py +0 -492
- package/packages/memory-engine/tests/test_l2_qmd_vec_search.py +0 -280
- package/packages/memory-engine/tests/test_l3_arena_isolation.py +0 -412
- package/packages/memory-engine/tests/test_l6_module_load.py +0 -84
- package/packages/memory-engine/tests/test_people_list_reader.py +0 -432
|
@@ -1,20 +1,19 @@
|
|
|
1
1
|
{
|
|
2
|
-
"name": "
|
|
3
|
-
"version": "0.1.0",
|
|
2
|
+
"name": "memory",
|
|
4
3
|
"lockfileVersion": 3,
|
|
5
4
|
"requires": true,
|
|
6
5
|
"packages": {
|
|
7
6
|
"": {
|
|
8
|
-
"name": "
|
|
7
|
+
"name": "memory",
|
|
9
8
|
"dependencies": {
|
|
10
9
|
"@modelcontextprotocol/sdk": "^1.0.0",
|
|
11
10
|
"pg": "^8.13.0"
|
|
12
11
|
}
|
|
13
12
|
},
|
|
14
13
|
"node_modules/@hono/node-server": {
|
|
15
|
-
"version": "1.19.
|
|
16
|
-
"resolved": "https://registry.npmjs.org/@hono/node-server/-/node-server-1.19.
|
|
17
|
-
"integrity": "sha512-
|
|
14
|
+
"version": "1.19.14",
|
|
15
|
+
"resolved": "https://registry.npmjs.org/@hono/node-server/-/node-server-1.19.14.tgz",
|
|
16
|
+
"integrity": "sha512-GwtvgtXxnWsucXvbQXkRgqksiH2Qed37H9xHZocE5sA3N8O8O8/8FA3uclQXxXVzc9XBZuEOMK7+r02FmSpHtw==",
|
|
18
17
|
"license": "MIT",
|
|
19
18
|
"engines": {
|
|
20
19
|
"node": ">=18.14.1"
|
|
@@ -77,9 +76,9 @@
|
|
|
77
76
|
}
|
|
78
77
|
},
|
|
79
78
|
"node_modules/ajv": {
|
|
80
|
-
"version": "8.
|
|
81
|
-
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.
|
|
82
|
-
"integrity": "sha512-
|
|
79
|
+
"version": "8.20.0",
|
|
80
|
+
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz",
|
|
81
|
+
"integrity": "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==",
|
|
83
82
|
"license": "MIT",
|
|
84
83
|
"dependencies": {
|
|
85
84
|
"fast-deep-equal": "^3.1.3",
|
|
@@ -355,9 +354,9 @@
|
|
|
355
354
|
}
|
|
356
355
|
},
|
|
357
356
|
"node_modules/eventsource-parser": {
|
|
358
|
-
"version": "3.0.
|
|
359
|
-
"resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.0.
|
|
360
|
-
"integrity": "sha512-
|
|
357
|
+
"version": "3.0.8",
|
|
358
|
+
"resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.0.8.tgz",
|
|
359
|
+
"integrity": "sha512-70QWGkr4snxr0OXLRWsFLeRBIRPuQOvt4s8QYjmUlmlkyTZkRqS7EDVRZtzU3TiyDbXSzaOeF0XUKy8PchzukQ==",
|
|
361
360
|
"license": "MIT",
|
|
362
361
|
"engines": {
|
|
363
362
|
"node": ">=18.0.0"
|
|
@@ -407,12 +406,12 @@
|
|
|
407
406
|
}
|
|
408
407
|
},
|
|
409
408
|
"node_modules/express-rate-limit": {
|
|
410
|
-
"version": "8.
|
|
411
|
-
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-8.
|
|
412
|
-
"integrity": "sha512-
|
|
409
|
+
"version": "8.5.1",
|
|
410
|
+
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-8.5.1.tgz",
|
|
411
|
+
"integrity": "sha512-5O6KYmyJEpuPJV5hNTXKbAHWRqrzyu+OI3vUnSd2kXFubIVpG7ezpgxQy76Zo5GQZtrQBg86hF+CM/NX+cioiQ==",
|
|
413
412
|
"license": "MIT",
|
|
414
413
|
"dependencies": {
|
|
415
|
-
"ip-address": "10.
|
|
414
|
+
"ip-address": "^10.2.0"
|
|
416
415
|
},
|
|
417
416
|
"engines": {
|
|
418
417
|
"node": ">= 16"
|
|
@@ -556,9 +555,9 @@
|
|
|
556
555
|
}
|
|
557
556
|
},
|
|
558
557
|
"node_modules/hasown": {
|
|
559
|
-
"version": "2.0.
|
|
560
|
-
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.
|
|
561
|
-
"integrity": "sha512-
|
|
558
|
+
"version": "2.0.3",
|
|
559
|
+
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.3.tgz",
|
|
560
|
+
"integrity": "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg==",
|
|
562
561
|
"license": "MIT",
|
|
563
562
|
"dependencies": {
|
|
564
563
|
"function-bind": "^1.1.2"
|
|
@@ -619,9 +618,9 @@
|
|
|
619
618
|
"license": "ISC"
|
|
620
619
|
},
|
|
621
620
|
"node_modules/ip-address": {
|
|
622
|
-
"version": "10.
|
|
623
|
-
"resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.
|
|
624
|
-
"integrity": "sha512
|
|
621
|
+
"version": "10.2.0",
|
|
622
|
+
"resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
|
|
623
|
+
"integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
|
|
625
624
|
"license": "MIT",
|
|
626
625
|
"engines": {
|
|
627
626
|
"node": ">= 12"
|
|
@@ -649,9 +648,9 @@
|
|
|
649
648
|
"license": "ISC"
|
|
650
649
|
},
|
|
651
650
|
"node_modules/jose": {
|
|
652
|
-
"version": "6.2.
|
|
653
|
-
"resolved": "https://registry.npmjs.org/jose/-/jose-6.2.
|
|
654
|
-
"integrity": "sha512-
|
|
651
|
+
"version": "6.2.3",
|
|
652
|
+
"resolved": "https://registry.npmjs.org/jose/-/jose-6.2.3.tgz",
|
|
653
|
+
"integrity": "sha512-YYVDInQKFJfR/xa3ojUTl8c2KoTwiL1R5Wg9YCydwH0x0B9grbzlg5HC7mMjCtUJjbQ/YnGEZIhI5tCgfTb4Hw==",
|
|
655
654
|
"license": "MIT",
|
|
656
655
|
"funding": {
|
|
657
656
|
"url": "https://github.com/sponsors/panva"
|
|
@@ -1201,17 +1200,34 @@
|
|
|
1201
1200
|
}
|
|
1202
1201
|
},
|
|
1203
1202
|
"node_modules/type-is": {
|
|
1204
|
-
"version": "2.0
|
|
1205
|
-
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.0.
|
|
1206
|
-
"integrity": "sha512-
|
|
1203
|
+
"version": "2.1.0",
|
|
1204
|
+
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz",
|
|
1205
|
+
"integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==",
|
|
1207
1206
|
"license": "MIT",
|
|
1208
1207
|
"dependencies": {
|
|
1209
|
-
"content-type": "^
|
|
1208
|
+
"content-type": "^2.0.0",
|
|
1210
1209
|
"media-typer": "^1.1.0",
|
|
1211
1210
|
"mime-types": "^3.0.0"
|
|
1212
1211
|
},
|
|
1213
1212
|
"engines": {
|
|
1214
|
-
"node": ">=
|
|
1213
|
+
"node": ">= 18"
|
|
1214
|
+
},
|
|
1215
|
+
"funding": {
|
|
1216
|
+
"type": "opencollective",
|
|
1217
|
+
"url": "https://opencollective.com/express"
|
|
1218
|
+
}
|
|
1219
|
+
},
|
|
1220
|
+
"node_modules/type-is/node_modules/content-type": {
|
|
1221
|
+
"version": "2.0.0",
|
|
1222
|
+
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
|
|
1223
|
+
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
|
|
1224
|
+
"license": "MIT",
|
|
1225
|
+
"engines": {
|
|
1226
|
+
"node": ">=18"
|
|
1227
|
+
},
|
|
1228
|
+
"funding": {
|
|
1229
|
+
"type": "opencollective",
|
|
1230
|
+
"url": "https://opencollective.com/express"
|
|
1215
1231
|
}
|
|
1216
1232
|
},
|
|
1217
1233
|
"node_modules/unpipe": {
|
|
@@ -1263,9 +1279,9 @@
|
|
|
1263
1279
|
}
|
|
1264
1280
|
},
|
|
1265
1281
|
"node_modules/zod": {
|
|
1266
|
-
"version": "4.3
|
|
1267
|
-
"resolved": "https://registry.npmjs.org/zod/-/zod-4.3.
|
|
1268
|
-
"integrity": "sha512-
|
|
1282
|
+
"version": "4.4.3",
|
|
1283
|
+
"resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz",
|
|
1284
|
+
"integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==",
|
|
1269
1285
|
"license": "MIT",
|
|
1270
1286
|
"funding": {
|
|
1271
1287
|
"url": "https://github.com/sponsors/colinhacks"
|
|
@@ -1,10 +1,13 @@
|
|
|
1
1
|
{
|
|
2
2
|
"private": true,
|
|
3
3
|
"name": "memory",
|
|
4
|
-
"description": "Memory subsystem
|
|
4
|
+
"description": "Memory subsystem \u2014 imported via @pentatonic-ai/ai-agent-sdk/memory",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"dependencies": {
|
|
7
7
|
"@modelcontextprotocol/sdk": "^1.0.0",
|
|
8
8
|
"pg": "^8.13.0"
|
|
9
|
+
},
|
|
10
|
+
"overrides": {
|
|
11
|
+
"ip-address": "^10.1.1"
|
|
9
12
|
}
|
|
10
13
|
}
|
|
@@ -730,15 +730,22 @@ describe("engine HTTP client", () => {
|
|
|
730
730
|
});
|
|
731
731
|
|
|
732
732
|
describe("engineForget", () => {
|
|
733
|
-
it("forwards id when provided", async () => {
|
|
733
|
+
it("forwards id when provided (no arena composition for id-based deletes)", async () => {
|
|
734
734
|
mockOk({ deleted: 1 });
|
|
735
735
|
await engineForget("https://e", { clientId: "acme", id: "abc" });
|
|
736
736
|
const body = JSON.parse(calls[0].init.body);
|
|
737
737
|
expect(calls[0].url).toBe("https://e/forget");
|
|
738
|
-
|
|
738
|
+
// id-only deletes target the global record id; the engine's
|
|
739
|
+
// id path doesn't read arena scope, so we don't inject it.
|
|
740
|
+
expect(body).toEqual({ id: "abc" });
|
|
739
741
|
});
|
|
740
742
|
|
|
741
|
-
it("forwards metadata_contains
|
|
743
|
+
it("forwards metadata_contains and injects arena INSIDE it (tenant default)", async () => {
|
|
744
|
+
// The engine reads `metadata_contains.arena` (not top-level
|
|
745
|
+
// arena) to scope a forget at L2. Pre-2026-05-14 this helper
|
|
746
|
+
// put arena at the top level, which the engine silently
|
|
747
|
+
// ignored — only L6 ever got wiped. Pinning the post-fix
|
|
748
|
+
// contract here so a regression can't sneak back in.
|
|
742
749
|
mockOk({ deleted: 5 });
|
|
743
750
|
await engineForget("https://e", {
|
|
744
751
|
clientId: "acme",
|
|
@@ -746,11 +753,39 @@ describe("engine HTTP client", () => {
|
|
|
746
753
|
});
|
|
747
754
|
const body = JSON.parse(calls[0].init.body);
|
|
748
755
|
expect(body).toEqual({
|
|
749
|
-
arena: "acme",
|
|
750
|
-
|
|
756
|
+
metadata_contains: { arena: "acme", source_repo: "monorepo" },
|
|
757
|
+
});
|
|
758
|
+
// Top-level arena must NOT be sent — the engine ignores it and
|
|
759
|
+
// its presence would mislead anyone reading wire dumps.
|
|
760
|
+
expect(body.arena).toBeUndefined();
|
|
761
|
+
});
|
|
762
|
+
|
|
763
|
+
it("composes user-scoped arena when userId is supplied", async () => {
|
|
764
|
+
mockOk({ deleted: 12 });
|
|
765
|
+
await engineForget("https://e", {
|
|
766
|
+
clientId: "acme",
|
|
767
|
+
userId: "u-1",
|
|
768
|
+
metadataContains: { actor_user_id: "u-1" },
|
|
769
|
+
});
|
|
770
|
+
const body = JSON.parse(calls[0].init.body);
|
|
771
|
+
expect(body).toEqual({
|
|
772
|
+
metadata_contains: { arena: "acme:u-1", actor_user_id: "u-1" },
|
|
751
773
|
});
|
|
752
774
|
});
|
|
753
775
|
|
|
776
|
+
it("respects caller-supplied arena inside metadataContains (super-admin override)", async () => {
|
|
777
|
+
// Super-admin tooling that wipes "some other tenant's user arena"
|
|
778
|
+
// — pass the explicit arena and the SDK leaves it alone instead
|
|
779
|
+
// of recomposing from (clientId, userId).
|
|
780
|
+
mockOk({ deleted: 99 });
|
|
781
|
+
await engineForget("https://e", {
|
|
782
|
+
clientId: "tes-admin",
|
|
783
|
+
metadataContains: { arena: "victim-tenant:u-7", source: "x" },
|
|
784
|
+
});
|
|
785
|
+
const body = JSON.parse(calls[0].init.body);
|
|
786
|
+
expect(body.metadata_contains.arena).toBe("victim-tenant:u-7");
|
|
787
|
+
});
|
|
788
|
+
|
|
754
789
|
it("requires id or metadataContains", async () => {
|
|
755
790
|
await expect(
|
|
756
791
|
engineForget("https://e", { clientId: "acme" })
|
|
@@ -328,9 +328,31 @@ export async function engineSearch(engineUrl, opts) {
|
|
|
328
328
|
*
|
|
329
329
|
* Caller must supply exactly one of `id` or `metadataContains`.
|
|
330
330
|
*
|
|
331
|
+
* Arena scope: the engine extracts the arena from `metadata_contains.arena`
|
|
332
|
+
* (see memory-engine `compat/server.py:1048-1052`). Top-level `arena` is
|
|
333
|
+
* NOT read by the engine — previous versions of this helper put it there
|
|
334
|
+
* and the resulting calls only ever wiped L6, leaving L0/L2/L3/L4 records
|
|
335
|
+
* untouched. The 2026-05-14 Pip dedup cutover surfaced the bug: an
|
|
336
|
+
* actor_user_id wipe returned 0 against an arena that personFacets
|
|
337
|
+
* confirmed held thousands of records. This helper now injects `arena`
|
|
338
|
+
* into `metadata_contains` so the engine forwards to L2 /forget-internal
|
|
339
|
+
* and actually wipes the cross-layer arena.
|
|
340
|
+
*
|
|
341
|
+
* By default the row is **user-scoped** (`arena = clientId:userId`) when
|
|
342
|
+
* `userId` is supplied, otherwise **tenant-wide** (`arena = clientId`).
|
|
343
|
+
* Pass `scope: "tenant"` explicitly to bypass the user-arena scope from a
|
|
344
|
+
* user-context. Matches `engineStore`'s arena semantics for symmetry.
|
|
345
|
+
*
|
|
346
|
+
* If the caller passes `arena` inside `metadataContains` themselves, the
|
|
347
|
+
* SDK respects it as-is and skips composition — useful for super-admin
|
|
348
|
+
* tools that need to wipe an arena other than the one derived from
|
|
349
|
+
* (clientId, userId).
|
|
350
|
+
*
|
|
331
351
|
* @param {string} engineUrl
|
|
332
352
|
* @param {object} opts
|
|
333
353
|
* @param {string} opts.clientId
|
|
354
|
+
* @param {string} [opts.userId] user id within the tenant; controls default scope
|
|
355
|
+
* @param {"tenant"|"user"} [opts.scope] override the default scope. "user" requires userId.
|
|
334
356
|
* @param {string} [opts.id] forget a single record by engine id
|
|
335
357
|
* @param {object} [opts.metadataContains] forget all records matching every key=value pair
|
|
336
358
|
* @param {Record<string,string>} [opts.headers] forwarded HTTP headers
|
|
@@ -338,15 +360,28 @@ export async function engineSearch(engineUrl, opts) {
|
|
|
338
360
|
* @returns {Promise<{deleted: number}>}
|
|
339
361
|
*/
|
|
340
362
|
export async function engineForget(engineUrl, opts) {
|
|
341
|
-
const { clientId, id, metadataContains, headers } = opts || {};
|
|
363
|
+
const { clientId, userId, scope, id, metadataContains, headers } = opts || {};
|
|
342
364
|
if (!clientId) throw new Error("engineForget: clientId required");
|
|
343
365
|
if (!id && !metadataContains) {
|
|
344
366
|
throw new Error("engineForget: provide id or metadataContains");
|
|
345
367
|
}
|
|
368
|
+
|
|
369
|
+
// Compose arena from (clientId, userId, scope) using the same shape
|
|
370
|
+
// engineStore uses. Caller-supplied `metadataContains.arena` wins —
|
|
371
|
+
// the SDK shouldn't second-guess a super-admin explicitly targeting
|
|
372
|
+
// a specific arena.
|
|
373
|
+
let mergedMetadata;
|
|
374
|
+
if (metadataContains) {
|
|
375
|
+
const hasExplicitArena =
|
|
376
|
+
typeof metadataContains.arena === "string" && metadataContains.arena;
|
|
377
|
+
mergedMetadata = hasExplicitArena
|
|
378
|
+
? metadataContains
|
|
379
|
+
: { ...metadataContains, arena: composeArena(clientId, userId, scope) };
|
|
380
|
+
}
|
|
381
|
+
|
|
346
382
|
const body = {
|
|
347
|
-
arena: clientId,
|
|
348
383
|
...(id ? { id } : {}),
|
|
349
|
-
...(
|
|
384
|
+
...(mergedMetadata ? { metadata_contains: mergedMetadata } : {}),
|
|
350
385
|
};
|
|
351
386
|
return fetchEngine(engineUrl, "/forget", body, { headers });
|
|
352
387
|
}
|
|
@@ -42,14 +42,14 @@ process.on("unhandledRejection", (err) => {
|
|
|
42
42
|
|
|
43
43
|
// Deprecation notice — see packages/memory/README.md for context.
|
|
44
44
|
// This MCP server (Postgres+pgvector+Ollama, single-process) is being
|
|
45
|
-
// retired in favour of the 7-layer engine at packages/memory-engine/.
|
|
45
|
+
// retired in favour of the 7-layer engine at packages/memory-engine-v2/.
|
|
46
46
|
// Targeted for removal in v1.0; in the meantime everything keeps
|
|
47
47
|
// working. Print once on startup so operators see the signal in logs
|
|
48
48
|
// without flooding the conversation surface.
|
|
49
49
|
if (process.env.PENTATONIC_DEPRECATION_QUIET !== "1") {
|
|
50
50
|
process.stderr.write(
|
|
51
51
|
"[memory-server] DEPRECATED: this server (Postgres+pgvector+Ollama MCP) " +
|
|
52
|
-
"is superseded by the 7-layer memory engine at packages/memory-engine/. " +
|
|
52
|
+
"is superseded by the 7-layer memory engine at packages/memory-engine-v2/. " +
|
|
53
53
|
"Existing deployments keep working; removal targeted for v1.0. " +
|
|
54
54
|
"See README → Memory → Local for the migration path. " +
|
|
55
55
|
"Suppress this warning with PENTATONIC_DEPRECATION_QUIET=1.\n"
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# pentatonic-memory-engine v2 environment.
|
|
2
|
+
# Copy to .env on the engine box; never commit a populated copy.
|
|
3
|
+
|
|
4
|
+
# --- Embed gateway (same as v1) -----------------------------------
|
|
5
|
+
NV_EMBED_URL=https://lambda-gateway.pentatonic.com/v1/embed
|
|
6
|
+
PENTATONIC_AI_GATEWAY_KEY=<rotate-on-deploy>
|
|
7
|
+
NV_EMBED_PROVIDER=pentatonic-gateway
|
|
8
|
+
|
|
9
|
+
# --- Postgres (org-model) -----------------------------------------
|
|
10
|
+
PME_V2_PG_USER=pme
|
|
11
|
+
PME_V2_PG_PASSWORD=<random-strong-password>
|
|
12
|
+
PME_V2_PG_DB=org_model
|
|
13
|
+
PME_V2_PG_DSN=postgresql://pme:<password>@org-model:5432/org_model
|
|
14
|
+
|
|
15
|
+
# --- Ports (host-side) --------------------------------------------
|
|
16
|
+
# v1 runs on 8099; if you want v2 alongside, give it a different port
|
|
17
|
+
# until the cutover. Compose maps these to internal 8099 / 5432 / 6333.
|
|
18
|
+
PME_V2_COMPAT_PORT=8199 # v2 alongside v1
|
|
19
|
+
PME_V2_ORG_MODEL_PORT=15433
|
|
20
|
+
PME_V2_QDRANT_HTTP_PORT=16335
|
|
21
|
+
PME_V2_QDRANT_GRPC_PORT=16336
|
|
22
|
+
|
|
23
|
+
# --- Embedding dimension ------------------------------------------
|
|
24
|
+
PME_V2_EMBED_DIM=4096
|
|
25
|
+
|
|
26
|
+
# --- Cloudflared (optional, only set if exposing v2 publicly) -----
|
|
27
|
+
# TUNNEL_TOKEN_V2=<cut-a-new-tunnel-or-reuse-v1>
|
|
28
|
+
|
|
29
|
+
# --- LLM endpoint (extractor-async; stub mode if empty) -----------
|
|
30
|
+
# PME_V2_LLM_ENDPOINT=https://lambda-gateway.pentatonic.com/v1/chat/completions
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
# pentatonic-memory-engine v2
|
|
2
|
+
|
|
3
|
+
Keystone-first rebuild. Three stores, not seven. Wire-compatible with
|
|
4
|
+
v1's compat shim so TES can flip via a single env var.
|
|
5
|
+
|
|
6
|
+
## Status
|
|
7
|
+
|
|
8
|
+
- Infrastructure scaffolded (this directory)
|
|
9
|
+
- Not yet released to npm; not yet CI'd
|
|
10
|
+
- Currently deployed to AWS engine box at `/opt/engine-v2` via rsync
|
|
11
|
+
- Real LLM extraction (extractor-async) stubbed pending keystone spec
|
|
12
|
+
(#285 extraction-objectives.md)
|
|
13
|
+
|
|
14
|
+
## Layout
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
packages/memory-engine-v2/
|
|
18
|
+
docker-compose.yml # 5 services: org-model, vector-index,
|
|
19
|
+
# extractor-sync, extractor-async, compat
|
|
20
|
+
docker-compose.aws.yml # AWS overlay (bind-mount volumes, prod env)
|
|
21
|
+
compat/ # FastAPI v1-wire-compatible shim
|
|
22
|
+
Dockerfile + server.py + requirements.txt
|
|
23
|
+
extractor-sync/ # Deterministic per-source extraction
|
|
24
|
+
Dockerfile + server.py + requirements.txt
|
|
25
|
+
extractor-async/ # LLM distillation worker (stub for now)
|
|
26
|
+
Dockerfile + worker.py + requirements.txt
|
|
27
|
+
org-model/migrations/ # Postgres schema
|
|
28
|
+
001_init.sql # Initial schema with provenance cols
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## What's in here vs v1
|
|
32
|
+
|
|
33
|
+
| v1 has | v2 has | Notes |
|
|
34
|
+
|---|---|---|
|
|
35
|
+
| L0 (sqlite FTS5) inside L2 | not yet — falls back to vector-only on /search | Keyword fallback comes once the typed router lands |
|
|
36
|
+
| L2 HybridRAG proxy | compat shim does the routing | Single shim, no separate proxy service |
|
|
37
|
+
| L3 Neo4j knowledge graph | recursive-CTE on org-model (pending spike #278) | If quality acceptable, Neo4j retires entirely |
|
|
38
|
+
| L4 sqlite-vec sidecar | gone | Already removed in SDK PR #51 |
|
|
39
|
+
| L4 QMD (vectors as JSON text in SQLite) | gone | Replaced by Qdrant + content-hash IDs |
|
|
40
|
+
| L5 Milvus chats | folded into Qdrant via source_kind=chat | One vector store, payload-filtered |
|
|
41
|
+
| L6 Milvus doc-store | folded into Qdrant via source_kind=doc | Same |
|
|
42
|
+
| Compat shim | compat (same wire contract) | Same external surface, totally new internals |
|
|
43
|
+
| nv-embed service | external (NV_EMBED_URL points at GH200) | Same as v1's AWS deploy |
|
|
44
|
+
| HyDE per query | gone | Confidence-gated query rewrite if needed later |
|
|
45
|
+
|
|
46
|
+
## Deploy (AWS engine box, via rsync — no CI)
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
# From SDK repo on your laptop:
|
|
50
|
+
rsync -av --delete \
|
|
51
|
+
packages/memory-engine-v2/ \
|
|
52
|
+
ubuntu@<engine-host>:/opt/engine-v2/
|
|
53
|
+
|
|
54
|
+
# On the box:
|
|
55
|
+
ssh ubuntu@<engine-host>
|
|
56
|
+
cd /opt/engine-v2
|
|
57
|
+
# CRITICAL: production deploys MUST include `-f docker-compose.aws.yml`.
|
|
58
|
+
# The base `docker-compose.yml` uses named docker volumes (anonymous,
|
|
59
|
+
# project-scoped). The AWS overlay re-points those to bind mounts under
|
|
60
|
+
# `/var/lib/pme-v2/<service>` — that's where the actual production data
|
|
61
|
+
# lives. Omitting the overlay creates fresh empty volumes and effectively
|
|
62
|
+
# "hides" the real data until the overlay is re-applied. The data is not
|
|
63
|
+
# lost; the container just mounts the wrong path.
|
|
64
|
+
sudo docker compose --env-file .env \
|
|
65
|
+
-f docker-compose.yml -f docker-compose.aws.yml \
|
|
66
|
+
up -d --build
|
|
67
|
+
|
|
68
|
+
# Verify:
|
|
69
|
+
curl http://localhost:8099/health # cheap liveness
|
|
70
|
+
curl http://localhost:8099/health/deep # round-trips all stores
|
|
71
|
+
|
|
72
|
+
# Single-service redeploy (e.g. just rebuild extractor-async after a
|
|
73
|
+
# code change). --no-deps avoids touching other healthy services.
|
|
74
|
+
# Same overlay rule applies here.
|
|
75
|
+
sudo docker compose --env-file .env \
|
|
76
|
+
-f docker-compose.yml -f docker-compose.aws.yml \
|
|
77
|
+
up -d --no-deps --build extractor-async
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Required env vars (set in `.env` next to the compose file):
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
NV_EMBED_URL=https://lambda-gateway.pentatonic.com/v1/embed
|
|
84
|
+
PENTATONIC_AI_GATEWAY_KEY=<the key>
|
|
85
|
+
NV_EMBED_PROVIDER=pentatonic-gateway
|
|
86
|
+
PME_V2_PG_PASSWORD=<random>
|
|
87
|
+
PME_V2_PG_DSN=postgresql://pme:<password>@org-model:5432/org_model
|
|
88
|
+
|
|
89
|
+
# Optional:
|
|
90
|
+
PME_V2_LLM_ENDPOINT=https://lambda-gateway.pentatonic.com/v1/chat/completions
|
|
91
|
+
PME_V2_COMPAT_PORT=8099 # match v1 so the env-var flip is one line
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Cutover from v1 → v2
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
# On TES wrangler secrets:
|
|
98
|
+
wrangler pages secret put MEMORY_ENGINE_URL --project-name <project>
|
|
99
|
+
# Enter: https://memory-engine-v2.thingeventsystem.ai (or whichever
|
|
100
|
+
# cloudflared tunnel hostname routes to v2's compat:8099)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Then v1's containers can be stopped:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
cd /opt/engine
|
|
107
|
+
sudo docker compose stop l2 l3 l5 l6 compat
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Roll back by reverting the wrangler secret.
|
|
111
|
+
|
|
112
|
+
## Open work (not in this scaffold)
|
|
113
|
+
|
|
114
|
+
1. **Keystone (PR #285)** — extraction-objectives.md. Per-source typed
|
|
115
|
+
schemas, κ / false-omission bars. Drives the real extractor-async
|
|
116
|
+
prompts. Until this lands, extractor-async is a stub that drains
|
|
117
|
+
the queue at no-op speed.
|
|
118
|
+
2. **Typed router** in /search — intent classify → org-model and/or
|
|
119
|
+
vector-index → fuse. Currently /search is vector-only.
|
|
120
|
+
3. **BM25 fallback** — sqlite FTS5 service or inline in compat.
|
|
121
|
+
4. **Graph spike (#278)** — recursive-CTE personFacets/peopleList over
|
|
122
|
+
org-model. Outcome decides whether Neo4j retires.
|
|
123
|
+
5. **Memory-proxy Worker** (separate from v2) — removes the public
|
|
124
|
+
tunnel surface. Design doc at
|
|
125
|
+
`thing-event-system/modules/pentatonic-memory/proxy/DESIGN.md`.
|