@mnexium/core 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +37 -0
- package/README.md +37 -0
- package/docs/API.md +299 -0
- package/docs/BEHAVIOR.md +224 -0
- package/docs/OPERATIONS.md +116 -0
- package/docs/SETUP.md +239 -0
- package/package.json +22 -0
- package/scripts/e2e.lib.mjs +604 -0
- package/scripts/e2e.routes.mjs +32 -0
- package/scripts/e2e.sh +76 -0
- package/scripts/e2e.webapp.client.js +408 -0
- package/scripts/e2e.webapp.mjs +1065 -0
- package/sql/postgres/schema.sql +275 -0
- package/src/adapters/postgres/PostgresCoreStore.ts +1017 -0
- package/src/ai/memoryExtractionService.ts +265 -0
- package/src/ai/recallService.ts +442 -0
- package/src/ai/types.ts +11 -0
- package/src/contracts/storage.ts +137 -0
- package/src/contracts/types.ts +138 -0
- package/src/dev.ts +144 -0
- package/src/index.ts +15 -0
- package/src/providers/cerebras.ts +101 -0
- package/src/providers/openaiChat.ts +116 -0
- package/src/providers/openaiEmbedding.ts +52 -0
- package/src/server/createCoreServer.ts +1154 -0
- package/src/server/memoryEventBus.ts +57 -0
- package/tsconfig.json +14 -0
package/docs/SETUP.md
ADDED
|
@@ -0,0 +1,239 @@
|
|
|
1
|
+
# ๐ Setup and Initialization
|
|
2
|
+
|
|
3
|
+
This guide helps you get CORE running quickly, then validates that memory + claim routes are working end-to-end.
|
|
4
|
+
|
|
5
|
+
## ๐งญ Choose Your Path
|
|
6
|
+
|
|
7
|
+
- **Path A (Recommended):** local Docker Postgres + live web dashboard + interactive route testing.
|
|
8
|
+
- **Path B:** one-shot Docker-backed E2E script.
|
|
9
|
+
|
|
10
|
+
> โ
If you are starting from scratch, use **Path A** first.
|
|
11
|
+
|
|
12
|
+
## โ
Prerequisites
|
|
13
|
+
|
|
14
|
+
- Node.js 18+ (Node 20+ recommended)
|
|
15
|
+
- Postgres 14+ (15+ recommended)
|
|
16
|
+
- `psql` CLI
|
|
17
|
+
- Docker (optional, required for Path A and Path B)
|
|
18
|
+
|
|
19
|
+
Required Postgres extension:
|
|
20
|
+
|
|
21
|
+
- `pgvector`
|
|
22
|
+
|
|
23
|
+
## โ๏ธ Quick Local Bootstrap
|
|
24
|
+
|
|
25
|
+
1. Install dependencies:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
npm install
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
2. Create env file:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
cp .env.example .env
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
3. Apply schema (required, no auto-init at startup):
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
psql "postgresql://USER:PASSWORD@HOST:5432/DB" \
|
|
41
|
+
-f sql/postgres/schema.sql
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
4. Start CORE:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npm run dev
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
5. Check health:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
curl -s http://localhost:8080/health
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Expected shape:
|
|
57
|
+
|
|
58
|
+
```json
|
|
59
|
+
{"ok":true,"service":"mnexium-core","timestamp":"..."}
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## ๐ณ Path A (Recommended): Dashboard + Persistent Docker DB
|
|
63
|
+
|
|
64
|
+
1. Start Postgres:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
docker rm -f mnx-core-db >/dev/null 2>&1 || true
|
|
68
|
+
docker run -d \
|
|
69
|
+
--name mnx-core-db \
|
|
70
|
+
-e POSTGRES_DB=mnexium_core \
|
|
71
|
+
-e POSTGRES_USER=mnexium \
|
|
72
|
+
-e POSTGRES_PASSWORD=mnexium_dev_password \
|
|
73
|
+
-p 5432:5432 \
|
|
74
|
+
pgvector/pgvector:pg16
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
2. Apply schema:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
cat sql/postgres/schema.sql | docker exec -i mnx-core-db psql -U mnexium -d mnexium_core
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
3. Configure `.env` (minimum fields):
|
|
84
|
+
|
|
85
|
+
```dotenv
|
|
86
|
+
POSTGRES_HOST=127.0.0.1
|
|
87
|
+
POSTGRES_PORT=5432
|
|
88
|
+
POSTGRES_DB=mnexium_core
|
|
89
|
+
POSTGRES_USER=mnexium
|
|
90
|
+
POSTGRES_PASSWORD=mnexium_dev_password
|
|
91
|
+
CORE_DEFAULT_PROJECT_ID=default-project
|
|
92
|
+
PORT=8080
|
|
93
|
+
CORE_AI_MODE=simple
|
|
94
|
+
USE_RETRIEVAL_EXPAND=false
|
|
95
|
+
CORE_DEBUG=false
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
4. Start CORE:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
npm run dev
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
5. Start dashboard in a second terminal:
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
npm run e2e:web
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
6. Open dashboard:
|
|
111
|
+
|
|
112
|
+
- `http://127.0.0.1:8091`
|
|
113
|
+
|
|
114
|
+
Suggested dashboard values:
|
|
115
|
+
|
|
116
|
+
- Core Base URL: `http://127.0.0.1:8080`
|
|
117
|
+
- Project ID: `default-project`
|
|
118
|
+
- Subject ID: `user_web_e2e`
|
|
119
|
+
- Postgres Host: `127.0.0.1`
|
|
120
|
+
- Postgres Port: `5432`
|
|
121
|
+
- Postgres DB: `mnexium_core`
|
|
122
|
+
- Postgres User: `mnexium`
|
|
123
|
+
- Postgres Password: `mnexium_dev_password`
|
|
124
|
+
|
|
125
|
+
## ๐งช Path B: One-shot Docker E2E
|
|
126
|
+
|
|
127
|
+
Run:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
npm run e2e
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Keep DB container after run:
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
CORE_E2E_KEEP_DB=true npm run e2e
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
What this script does:
|
|
140
|
+
|
|
141
|
+
- boots Docker Postgres
|
|
142
|
+
- applies `sql/postgres/schema.sql`
|
|
143
|
+
- starts CORE on a temporary port (default `18080`)
|
|
144
|
+
- runs route suite
|
|
145
|
+
- tears down server and DB (unless keep flag is true)
|
|
146
|
+
|
|
147
|
+
### E2E env overrides
|
|
148
|
+
|
|
149
|
+
- `CORE_E2E_DB_CONTAINER` (default `mnx-core-e2e-db`)
|
|
150
|
+
- `CORE_E2E_DB_IMAGE` (default `pgvector/pgvector:pg16`)
|
|
151
|
+
- `CORE_E2E_DB_PORT` (default `5432`)
|
|
152
|
+
- `CORE_E2E_DB_NAME` (default `mnexium_core`)
|
|
153
|
+
- `CORE_E2E_DB_USER` (default `mnexium`)
|
|
154
|
+
- `CORE_E2E_DB_PASSWORD` (default `mnexium_dev_password`)
|
|
155
|
+
- `CORE_E2E_SERVER_PORT` (default `18080`)
|
|
156
|
+
- `CORE_E2E_PROJECT_ID` (default `default-project`)
|
|
157
|
+
- `CORE_E2E_KEEP_DB` (default `false`)
|
|
158
|
+
|
|
159
|
+
## ๐ Dashboard env overrides (`npm run e2e:web`)
|
|
160
|
+
|
|
161
|
+
- `CORE_E2E_WEB_PORT` (default `8091`)
|
|
162
|
+
- `CORE_E2E_WEB_HOST` (default `127.0.0.1`)
|
|
163
|
+
- `CORE_E2E_BASE_URL` (default `http://127.0.0.1:${PORT||8080}`)
|
|
164
|
+
- `CORE_E2E_PROJECT_ID` (default `CORE_DEFAULT_PROJECT_ID` or `default-project`)
|
|
165
|
+
- `CORE_E2E_SUBJECT_ID` (default `user_web_e2e`)
|
|
166
|
+
- `CORE_E2E_WEB_DB_HOST` (default `127.0.0.1`)
|
|
167
|
+
- `CORE_E2E_WEB_DB_PORT` (default `5432`)
|
|
168
|
+
- `CORE_E2E_WEB_DB_NAME` (default `mnexium_core`)
|
|
169
|
+
- `CORE_E2E_WEB_DB_USER` (default `mnexium`)
|
|
170
|
+
- `CORE_E2E_WEB_DB_PASSWORD` (default `mnexium_dev_password`)
|
|
171
|
+
|
|
172
|
+
## ๐ Environment Reference
|
|
173
|
+
|
|
174
|
+
### Database
|
|
175
|
+
|
|
176
|
+
- `POSTGRES_HOST` (required)
|
|
177
|
+
- `POSTGRES_PORT` (optional, default `5432`)
|
|
178
|
+
- `POSTGRES_DB` (required)
|
|
179
|
+
- `POSTGRES_USER` (required)
|
|
180
|
+
- `POSTGRES_PASSWORD` (required)
|
|
181
|
+
|
|
182
|
+
### Server
|
|
183
|
+
|
|
184
|
+
- `PORT` (optional, default `8080`)
|
|
185
|
+
- `CORE_DEFAULT_PROJECT_ID` (optional)
|
|
186
|
+
|
|
187
|
+
Note:
|
|
188
|
+
|
|
189
|
+
- In `src/dev.ts`, when `CORE_DEFAULT_PROJECT_ID` is unset, startup uses `default-project`.
|
|
190
|
+
|
|
191
|
+
### AI routing and retrieval
|
|
192
|
+
|
|
193
|
+
- `CORE_AI_MODE` (optional, default `auto`)
|
|
194
|
+
- `auto`: Cerebras if `CEREBRAS_API` exists, else OpenAI if `OPENAI_API_KEY` exists, else `simple`
|
|
195
|
+
- `cerebras`: requires `CEREBRAS_API`; if missing, falls back to `simple`
|
|
196
|
+
- `openai`: requires `OPENAI_API_KEY`; if missing, falls back to `simple`
|
|
197
|
+
- `simple`: no LLM client
|
|
198
|
+
- `USE_RETRIEVAL_EXPAND` (optional, default `true`)
|
|
199
|
+
- controls `/api/v1/memories/search` expansion/rerank path only
|
|
200
|
+
- `RETRIEVAL_MODEL` (optional)
|
|
201
|
+
- model id passed to selected LLM provider
|
|
202
|
+
|
|
203
|
+
### Provider keys
|
|
204
|
+
|
|
205
|
+
- `CEREBRAS_API` (required to use Cerebras mode)
|
|
206
|
+
- `OPENAI_API_KEY` (required for OpenAI chat mode and embeddings)
|
|
207
|
+
- `OPENAI_EMBED_MODEL` (optional, default `text-embedding-3-small`)
|
|
208
|
+
|
|
209
|
+
## ๐งฏ Troubleshooting
|
|
210
|
+
|
|
211
|
+
### `project_id_required`
|
|
212
|
+
|
|
213
|
+
Cause:
|
|
214
|
+
|
|
215
|
+
- neither `x-project-id` header nor server default project is available.
|
|
216
|
+
|
|
217
|
+
Fix:
|
|
218
|
+
|
|
219
|
+
- set `CORE_DEFAULT_PROJECT_ID`, or send `x-project-id` per request.
|
|
220
|
+
|
|
221
|
+
### `vector` / `pgvector` errors during schema apply
|
|
222
|
+
|
|
223
|
+
Cause:
|
|
224
|
+
|
|
225
|
+
- `pgvector` extension is not available in your Postgres instance.
|
|
226
|
+
|
|
227
|
+
Fix:
|
|
228
|
+
|
|
229
|
+
- install/enable `pgvector`, then re-run schema apply.
|
|
230
|
+
|
|
231
|
+
### Search works but feels lexical-only
|
|
232
|
+
|
|
233
|
+
Cause:
|
|
234
|
+
|
|
235
|
+
- embeddings are unavailable (often missing `OPENAI_API_KEY`).
|
|
236
|
+
|
|
237
|
+
Fix:
|
|
238
|
+
|
|
239
|
+
- add `OPENAI_API_KEY`, or keep lexical fallback intentionally for local/simple mode.
|
package/package.json
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@mnexium/core",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"type": "module",
|
|
5
|
+
"publishConfig": {
|
|
6
|
+
"access": "public"
|
|
7
|
+
},
|
|
8
|
+
"scripts": {
|
|
9
|
+
"dev": "tsx src/dev.ts",
|
|
10
|
+
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
11
|
+
"e2e": "bash scripts/e2e.sh",
|
|
12
|
+
"e2e:web": "node scripts/e2e.webapp.mjs"
|
|
13
|
+
},
|
|
14
|
+
"dependencies": {
|
|
15
|
+
"dotenv": "^17.2.3",
|
|
16
|
+
"pg": "^8.18.0"
|
|
17
|
+
},
|
|
18
|
+
"devDependencies": {
|
|
19
|
+
"tsx": "^4.20.5",
|
|
20
|
+
"typescript": "^5.9.3"
|
|
21
|
+
}
|
|
22
|
+
}
|