@cerefox/memory 1.15.1 → 1.15.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/bin/cerefox.js +27474 -34488
- package/dist/frontend/assets/index-B0H_N4if.css +1 -0
- package/dist/frontend/assets/index-DWeD_pcf.js +121 -0
- package/dist/frontend/assets/index-DWeD_pcf.js.map +1 -0
- package/dist/frontend/index.html +2 -2
- package/dist/server-assets/_shared/ef-meta/index.ts +3 -3
- package/dist/server-assets/_shared/embeddings/onnx-embedder.ts +40 -0
- package/dist/server-assets/_shared/mcp-tools/ingest.ts +7 -6
- package/dist/server-assets/supabase/functions/cerefox-ingest/index.ts +2 -2
- package/docs/api/openapi.json +5790 -0
- package/docs/guides/api.md +297 -0
- package/docs/guides/linking.md +118 -0
- package/docs/guides/ops-scripts.md +47 -0
- package/docs/guides/securing-local-access.md +253 -0
- package/docs/guides/staging-env.md +334 -0
- package/package.json +9 -9
- package/dist/frontend/assets/index-Dm_zCch4.css +0 -1
- package/dist/frontend/assets/index-xHSz7_7E.js +0 -121
- package/dist/frontend/assets/index-xHSz7_7E.js.map +0 -1
|
@@ -0,0 +1,253 @@
|
|
|
1
|
+
# Securing local access (the API key)
|
|
2
|
+
|
|
3
|
+
`cerefox web` and Cerefox Local serve an HTTP API on a port. This guide is
|
|
4
|
+
about who is allowed to use it.
|
|
5
|
+
|
|
6
|
+
**Most people need to do nothing.** Read the first section, confirm you are in
|
|
7
|
+
the common case, and stop.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## The rule
|
|
12
|
+
|
|
13
|
+
> **A request arriving on the loopback interface (`127.0.0.1`) is allowed
|
|
14
|
+
> without a credential. A request arriving on any other interface must present
|
|
15
|
+
> the server's API key.**
|
|
16
|
+
|
|
17
|
+
Loopback means "a connection that started on this machine". Your browser, an
|
|
18
|
+
agent running on your laptop, a script you typed into your own terminal: all
|
|
19
|
+
loopback, all exempt, all work with no configuration.
|
|
20
|
+
|
|
21
|
+
Introduced in **v1.12.0** ([#229](https://github.com/fstamatelopoulos/cerefox/issues/229)).
|
|
22
|
+
|
|
23
|
+
## Do you need a key?
|
|
24
|
+
|
|
25
|
+
**It depends on how Cerefox itself is running**, so start there.
|
|
26
|
+
|
|
27
|
+
### If you run Cerefox Local (Docker)
|
|
28
|
+
|
|
29
|
+
| Your publish address | Cerefox checks credentials? | What protects the port | You do |
|
|
30
|
+
|---|---|---|---|
|
|
31
|
+
| `127.0.0.1` (the default) | **No** | **Docker.** The port is bound to this machine's loopback only, so nothing outside it can open a connection at all. | Nothing |
|
|
32
|
+
| Anything wider (`CEREFOX_LOCAL_BIND=0.0.0.0`) | **Yes, for every caller** | The key | Give the key to every client, including your own browser |
|
|
33
|
+
|
|
34
|
+
**Read the middle column carefully.** On a default install Cerefox is *not*
|
|
35
|
+
refusing remote callers — it never sees them. Docker declines the connection at
|
|
36
|
+
the socket, before Cerefox is involved. "No key required" and "protected" are
|
|
37
|
+
both true at the same time, for different reasons, and it is worth knowing
|
|
38
|
+
which is doing the work: widen the publish address and the first protection
|
|
39
|
+
disappears instantly, which is why the key turns on in the same move.
|
|
40
|
+
|
|
41
|
+
There is no middle setting here, and that is deliberate rather than a
|
|
42
|
+
limitation we forgot to lift. **Docker rewrites the source address of every
|
|
43
|
+
request that comes through a published port**, so a server inside a container
|
|
44
|
+
cannot tell a request from your own machine apart from one that crossed the
|
|
45
|
+
network — both arrive from the same bridge address. Since the distinction
|
|
46
|
+
cannot be made, the gate does not pretend to make it: it is either off, with
|
|
47
|
+
the publish address as the boundary, or on for everybody.
|
|
48
|
+
|
|
49
|
+
`cerefox-local` sets this for you from `CEREFOX_LOCAL_BIND`. To force the gate
|
|
50
|
+
on while still publishing to loopback:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
echo 'CEREFOX_API_REQUIRE_KEY=1' >> ~/.cerefox/local/.env
|
|
54
|
+
cerefox-local restart
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
> **v1.12.0 got this wrong** and injected a key into every container without
|
|
58
|
+
> require-mode, so the loopback exemption never matched and every caller — the
|
|
59
|
+
> web UI included — got a `401`. Fixed in **v1.12.1**. If you are on v1.12.0
|
|
60
|
+
> and seeing 401s from a container, upgrade.
|
|
61
|
+
|
|
62
|
+
### If you run `cerefox web` directly (npm install)
|
|
63
|
+
|
|
64
|
+
Here the server sees real client addresses, so the loopback rule applies as
|
|
65
|
+
written.
|
|
66
|
+
|
|
67
|
+
| Your situation | Need a key? |
|
|
68
|
+
|---|---|
|
|
69
|
+
| You use the web UI in your browser | **No** |
|
|
70
|
+
| An agent or script runs on the same machine | **No** |
|
|
71
|
+
| A container reaches your host server via `host.docker.internal` | **No** — it arrives on the host's loopback |
|
|
72
|
+
| You bind `--host 0.0.0.0` to reach Cerefox from another machine | **Yes** |
|
|
73
|
+
| A reverse proxy sits in front on the same machine | **Yes**, plus `CEREFOX_API_REQUIRE_KEY=1` (see below) |
|
|
74
|
+
|
|
75
|
+
If every row that applies to you says No, you are done.
|
|
76
|
+
|
|
77
|
+
**Not sure?** Ask for the version endpoint the way your client will. `200`
|
|
78
|
+
means you are on the exempt path; `401` means you need a key.
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
docker exec <your-client-container> \
|
|
82
|
+
curl -s -o /dev/null -w '%{http_code}\n' http://host.docker.internal:8010/api/v1/version
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## Why it works this way
|
|
86
|
+
|
|
87
|
+
The attacker worth defending against can reach the port but **cannot read your
|
|
88
|
+
filesystem**: a machine on the café wifi after you widened the bind, a
|
|
89
|
+
container on a shared Docker network, a malicious page open in your browser.
|
|
90
|
+
|
|
91
|
+
Someone who *can* read your filesystem already has the key file, your `.env`,
|
|
92
|
+
and your database credentials. Demanding a key from them would protect nothing
|
|
93
|
+
while making the common case worse for everyone. On a single-user machine,
|
|
94
|
+
loopback access and key-file access are the same trust boundary, so the gate is
|
|
95
|
+
drawn at the place where the trust actually changes: the network.
|
|
96
|
+
|
|
97
|
+
This is also why **the browser never holds a key**. The web UI is a static
|
|
98
|
+
file, so a key embedded in the page could be read by anything that can load the
|
|
99
|
+
page — precisely the attacker the key exists to stop. A key in an
|
|
100
|
+
unauthenticated page is not a secret. The UI works because it is local, not
|
|
101
|
+
because it holds a credential.
|
|
102
|
+
|
|
103
|
+
## Getting a key
|
|
104
|
+
|
|
105
|
+
### `cerefox web` (npm install)
|
|
106
|
+
|
|
107
|
+
No key exists until you ask for one. Until then the gate is off and the server
|
|
108
|
+
behaves exactly as it did before v1.12.0, which is what makes upgrading safe.
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
cerefox api-key generate # mint one, write it to your .env, print it once
|
|
112
|
+
cerefox api-key show # print it again, in full
|
|
113
|
+
cerefox api-key rotate # replace it
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
`generate` refuses if a key already exists. Silently replacing one would give
|
|
117
|
+
every client you had already configured a `401` with no explanation.
|
|
118
|
+
|
|
119
|
+
### Cerefox Local (Docker)
|
|
120
|
+
|
|
121
|
+
The container mints a key **automatically at first boot** and persists it on
|
|
122
|
+
the data volume, so it survives `cerefox-local upgrade`. You never create it;
|
|
123
|
+
you only read it.
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
cerefox-local api-key # print it (and say whether it is enforced)
|
|
127
|
+
cerefox-local api-key --rotate # mint a new one and restart
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
The key exists whether or not the gate is on, so turning the gate on later
|
|
131
|
+
never changes the value your clients were given. `cerefox-local api-key` tells
|
|
132
|
+
you which state you are in — printing a key while implying it is being enforced
|
|
133
|
+
when it is not would be worse than printing nothing.
|
|
134
|
+
|
|
135
|
+
## Using it
|
|
136
|
+
|
|
137
|
+
A remote caller sends it as a bearer token:
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
curl http://<host>:8010/api/v1/documents/<uuid> \
|
|
141
|
+
-H 'Authorization: Bearer cfx_lak_…' \
|
|
142
|
+
-H 'X-Cerefox-Requestor: my-bot'
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Without a valid key the server answers `401` with a `WWW-Authenticate: Bearer`
|
|
146
|
+
challenge and a `detail` field explaining what to do.
|
|
147
|
+
|
|
148
|
+
The key authenticates; it does not identify. `X-Cerefox-Requestor` and
|
|
149
|
+
`X-Cerefox-Author` are still declared labels for attribution, exactly as
|
|
150
|
+
described in [`api.md`](api.md#attribution-is-not-authentication). Holding the
|
|
151
|
+
key does not make the name you send true.
|
|
152
|
+
|
|
153
|
+
## Recipe: a harness in its own container
|
|
154
|
+
|
|
155
|
+
The case where a key is genuinely needed, and the one worth automating.
|
|
156
|
+
|
|
157
|
+
**The ordering that avoids a chicken-and-egg problem.** You could boot Cerefox,
|
|
158
|
+
read its key, then configure the client — but that is two steps and a manual
|
|
159
|
+
copy every time either side is rebuilt. Instead, mint one key on the host and
|
|
160
|
+
give the same value to both containers. Cerefox prefers an injected key over
|
|
161
|
+
the one it would generate, so this pins it across recreates.
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
# 1. Mint once, on the host.
|
|
165
|
+
KEY="cfx_lak_$(openssl rand -base64 32 | tr -d '=+/' | cut -c1-43)"
|
|
166
|
+
|
|
167
|
+
# 2. Give it to Cerefox Local. Both variables are on the passthrough allowlist,
|
|
168
|
+
# so they survive `cerefox-local upgrade` and any recreate. REQUIRE_KEY is
|
|
169
|
+
# what turns the gate on: inside a container there is no loopback exemption
|
|
170
|
+
# to fall back on, so the gate is all-or-nothing.
|
|
171
|
+
{ echo "CEREFOX_API_KEY=$KEY"; echo "CEREFOX_API_REQUIRE_KEY=1"; } >> ~/.cerefox/local/.env
|
|
172
|
+
cerefox-local restart
|
|
173
|
+
|
|
174
|
+
# Note: with the gate on, YOUR BROWSER needs the key too. If you use the web UI
|
|
175
|
+
# daily, prefer leaving Cerefox published on loopback with the gate off and
|
|
176
|
+
# giving your harness host networking instead.
|
|
177
|
+
|
|
178
|
+
# 3. Give the same value to your client container, however it takes config.
|
|
179
|
+
docker run -e CEREFOX_API_KEY="$KEY" … your-harness
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
**Rotating** is the same two writes plus two restarts, in this order: update
|
|
183
|
+
Cerefox first, then the client. Between them the client gets `401`s, so keep
|
|
184
|
+
the window short, or accept both keys briefly by setting a comma-separated list
|
|
185
|
+
(`CEREFOX_API_KEY=new,old`), migrating the client, then dropping the old value.
|
|
186
|
+
|
|
187
|
+
## Recipe: a reverse proxy on the same machine
|
|
188
|
+
|
|
189
|
+
**This is the case the loopback rule cannot handle on its own, so read it if it
|
|
190
|
+
applies to you.**
|
|
191
|
+
|
|
192
|
+
A proxy on the same host terminates the client's connection and opens its own
|
|
193
|
+
to Cerefox, from `127.0.0.1`. So *every* proxied request looks local and is
|
|
194
|
+
exempt, including requests that came from the internet.
|
|
195
|
+
|
|
196
|
+
Cerefox deliberately does **not** read `X-Forwarded-For` to recover the
|
|
197
|
+
original address. That header is set by the caller, so trusting it would let
|
|
198
|
+
anyone claim to be local — strictly worse than having no gate at all.
|
|
199
|
+
|
|
200
|
+
The answer is to stop exempting loopback:
|
|
201
|
+
|
|
202
|
+
```bash
|
|
203
|
+
CEREFOX_API_REQUIRE_KEY=1
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Now every caller presents the key, proxy included. Your browser will need it
|
|
207
|
+
too, so this mode suits headless deployments rather than daily UI use.
|
|
208
|
+
|
|
209
|
+
## What this does not protect
|
|
210
|
+
|
|
211
|
+
**Other programs on your own machine.** Any local process can reach a loopback
|
|
212
|
+
port. This is a network boundary, not process isolation.
|
|
213
|
+
|
|
214
|
+
**Traffic on the wire.** There is no TLS. The key authenticates; it does not
|
|
215
|
+
encrypt. Anyone who can observe the connection can read the key and everything
|
|
216
|
+
else.
|
|
217
|
+
|
|
218
|
+
**Fine-grained access.** One key, all-or-nothing. No per-client scoping, no
|
|
219
|
+
read-only keys, no expiry, no revocation beyond rotating the single value.
|
|
220
|
+
|
|
221
|
+
**Being on the internet.** A key makes a widened bind survivable. It does not
|
|
222
|
+
turn this into an internet-facing API: there is no rate limiting, no TLS, no
|
|
223
|
+
audit of failed attempts. If you need Cerefox reachable over a network you do
|
|
224
|
+
not control, use the Edge Functions — that is the surface built for it, and it
|
|
225
|
+
authenticates every request against a rotatable token. See
|
|
226
|
+
[`access-paths.md`](access-paths.md).
|
|
227
|
+
|
|
228
|
+
## Troubleshooting
|
|
229
|
+
|
|
230
|
+
**Everything returns 401 after upgrading.** You have `CEREFOX_API_REQUIRE_KEY=1`
|
|
231
|
+
set, which removes the loopback exemption. Unset it unless you are running a
|
|
232
|
+
reverse proxy.
|
|
233
|
+
|
|
234
|
+
**My container gets 401 but curl on the host works.** With `cerefox web` on the
|
|
235
|
+
host, that is the gate behaving correctly: your container reaches Cerefox over
|
|
236
|
+
a Docker network, not loopback. Give it the key (see the recipe above).
|
|
237
|
+
|
|
238
|
+
**Everything gets 401 from a Cerefox Local container, including the web UI.**
|
|
239
|
+
That is the v1.12.0 bug, not a configuration problem. Run
|
|
240
|
+
`cerefox-local upgrade` to get v1.12.1 or newer.
|
|
241
|
+
|
|
242
|
+
**`cerefox-local api-key` says no key was found.** The image predates v1.12.0.
|
|
243
|
+
Run `cerefox-local upgrade`.
|
|
244
|
+
|
|
245
|
+
**I rotated and now everything is broken.** Rotation invalidates the old key at
|
|
246
|
+
the next restart. Update every remote client. Local callers are unaffected,
|
|
247
|
+
which is a useful way to confirm the server itself is healthy: if
|
|
248
|
+
`curl http://127.0.0.1:<port>/api/v1/version` works from the host, the server
|
|
249
|
+
is fine and the problem is client configuration.
|
|
250
|
+
|
|
251
|
+
**I lost the key.** You cannot recover it from the server, but you can read it
|
|
252
|
+
from where it is stored: `cerefox api-key show`, or `cerefox-local api-key`.
|
|
253
|
+
Failing that, rotate and reconfigure your clients.
|
|
@@ -0,0 +1,334 @@
|
|
|
1
|
+
# Running a staging environment alongside production
|
|
2
|
+
|
|
3
|
+
Cerefox can talk to more than one backend from a single machine. This guide sets
|
|
4
|
+
up a **staging** environment — a second Supabase project you can deploy
|
|
5
|
+
pre-release builds to, import production data into, and break freely — while
|
|
6
|
+
production stays untouched.
|
|
7
|
+
|
|
8
|
+
> **Nothing here affects a normal single-environment install.** Every mechanism
|
|
9
|
+
> below is opt-in and inert unless you set `CEREFOX_CONFIG_DIR`. If you never
|
|
10
|
+
> want staging, you can stop reading; `cerefox …` behaves exactly as documented
|
|
11
|
+
> in [`setup-supabase.md`](setup-supabase.md).
|
|
12
|
+
|
|
13
|
+
**Status**: validated end to end — config, database, backups, CLI version, and
|
|
14
|
+
the web server all separate cleanly. One gap remains (agent/MCP wiring); see
|
|
15
|
+
[Known limits](#known-limits).
|
|
16
|
+
|
|
17
|
+
This is a **convention, not a feature**. There is no `--env` flag and no profile
|
|
18
|
+
system to learn: two environment variables and a shell alias do the whole job,
|
|
19
|
+
which is why none of it can destabilise a normal single-environment install.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Why bother
|
|
24
|
+
|
|
25
|
+
Three things, and the third turned out to matter most:
|
|
26
|
+
|
|
27
|
+
- **Upgrades.** A release that changes the schema will run its migrations
|
|
28
|
+
against your real data exactly once. Staging lets you rehearse that on a copy.
|
|
29
|
+
- **Destructive or expensive operations.** `server migrate-format` re-embeds
|
|
30
|
+
every legacy document (real spend, rewrites chunks). Proving it on a copy
|
|
31
|
+
first turns an act of faith into a measurement.
|
|
32
|
+
- **Development.** A store you are willing to break is the cheapest way to turn
|
|
33
|
+
a claim about runtime behaviour into a measurement. During the 1.1.0 cycle this
|
|
34
|
+
found a retry loop executing 68,825 times per request, a `migrate-format` that
|
|
35
|
+
reported success while converting nothing, and a `backup create` that failed
|
|
36
|
+
against older servers — none visible from the code or the test suite. Keep it
|
|
37
|
+
available during development stretches, not only around releases.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## 1. Create the staging Supabase project
|
|
42
|
+
|
|
43
|
+
Follow [`setup-supabase.md` Step 1](setup-supabase.md), with two differences:
|
|
44
|
+
|
|
45
|
+
- Name it distinctly (`cerefox-staging`) — the dashboard project switcher is
|
|
46
|
+
your first defense against acting on the wrong database.
|
|
47
|
+
- Pick the **same region as production** if you care about comparable timings.
|
|
48
|
+
A different region is fine functionally; just never compare *speed* across
|
|
49
|
+
the two.
|
|
50
|
+
|
|
51
|
+
The creation-time security options are the same as production, and the
|
|
52
|
+
"automatically expose new tables" setting is worth disabling on both — Cerefox
|
|
53
|
+
grants its tables to `service_role` explicitly.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## 2. Give staging its own config directory
|
|
58
|
+
|
|
59
|
+
`CEREFOX_CONFIG_DIR` overrides where Cerefox reads `.env`. Production lives at
|
|
60
|
+
`~/.cerefox/.env` and is never consulted when the override is set.
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
mkdir -p ~/.cerefox/staging && chmod 700 ~/.cerefox/staging
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Then either run the interactive setup against it:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
CEREFOX_CONFIG_DIR=~/.cerefox/staging cerefox init
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
…or write `~/.cerefox/staging/.env` (mode 0600) directly:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
CEREFOX_ENV_LABEL=staging
|
|
76
|
+
CEREFOX_SUPABASE_URL=https://<staging-ref>.supabase.co
|
|
77
|
+
CEREFOX_SUPABASE_KEY=sb_secret_… # staging project's secret key
|
|
78
|
+
CEREFOX_DATABASE_URL=postgresql://postgres.<staging-ref>:…@aws-0-<region>.pooler.supabase.com:5432/postgres?sslmode=require
|
|
79
|
+
|
|
80
|
+
# Isolate snapshots. CEREFOX_BACKUP_DIR does NOT follow the config dir, so
|
|
81
|
+
# without this line staging backups land beside production's.
|
|
82
|
+
# Use an ABSOLUTE path: a relative value resolves against the working
|
|
83
|
+
# directory, so snapshots scatter wherever you happen to run the command.
|
|
84
|
+
CEREFOX_BACKUP_DIR=~/.cerefox/staging/backups
|
|
85
|
+
|
|
86
|
+
OPENAI_API_KEY=sk-…
|
|
87
|
+
SUPABASE_ACCESS_TOKEN=sbp_… # account-level; same as prod
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
> **Snapshot isolation needs v1.1.0-beta.3 or later.** On **beta.2 and
|
|
91
|
+
> earlier**, `backup create` read `CEREFOX_BACKUP_DIR` *before* loading the
|
|
92
|
+
> `.env` file, so the setting above was silently ignored and staging snapshots
|
|
93
|
+
> landed in production's directory. Until staging is on a build with the fix,
|
|
94
|
+
> pass the destination explicitly: `cfx-stg backup create --output-dir
|
|
95
|
+
> ~/.cerefox/staging/backups`.
|
|
96
|
+
|
|
97
|
+
**Mirror production's tuning.** Chunking and embedder settings still live in
|
|
98
|
+
`.env` (`CEREFOX_MAX_CHUNK_CHARS`, `CEREFOX_MIN_CHUNK_CHARS`, `CEREFOX_EMBEDDER`);
|
|
99
|
+
retrieval and retention moved into the database in v1.1.0, so mirror those with
|
|
100
|
+
`cerefox config set` against staging (or the Settings page).
|
|
101
|
+
A staging result only means something if the knobs match; different chunk sizes
|
|
102
|
+
make every comparison noise. **Do not** copy credentials — staging has its own.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 3. Pin a Cerefox version for staging
|
|
107
|
+
|
|
108
|
+
`CEREFOX_CONFIG_DIR` switches the *database*, not the *code*. The `cerefox` on
|
|
109
|
+
your PATH is a single global npm install, so on its own it would point both
|
|
110
|
+
environments at the same version — which defeats the main purpose of staging:
|
|
111
|
+
rehearsing a **pre-release** build before production ever sees it.
|
|
112
|
+
|
|
113
|
+
Give staging its own install tree. npm's `--prefix` keeps it entirely separate
|
|
114
|
+
from the global one, so production's binary is never touched:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
# Rehearsing a pre-release (the usual case):
|
|
118
|
+
npm install -g --prefix ~/.cerefox/staging/cli @cerefox/memory@beta
|
|
119
|
+
|
|
120
|
+
# Rehearsing a NORMAL release, where no beta was cut: pin the version
|
|
121
|
+
# explicitly. A bare `@cerefox/memory` resolves `latest` at install time, so
|
|
122
|
+
# staging silently records nothing about what it is running — and running it
|
|
123
|
+
# before the publish completes would install the PREVIOUS version, which is the
|
|
124
|
+
# one thing staging exists to avoid.
|
|
125
|
+
npm install -g --prefix ~/.cerefox/staging/cli @cerefox/memory@1.4.0
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Then bind **both axes — version and environment — in a single alias**, so they
|
|
129
|
+
can never drift apart:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
alias cfx-stg='CEREFOX_CONFIG_DIR=~/.cerefox/staging ~/.cerefox/staging/cli/bin/cerefox'
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Put it in `~/.zshrc` (or `~/.bashrc`) and reload. From here on, `cerefox` means
|
|
136
|
+
production-version-on-production-data and `cfx-stg` means
|
|
137
|
+
staging-version-on-staging-data. There is no combination of the two that a typo
|
|
138
|
+
can produce.
|
|
139
|
+
|
|
140
|
+
> **The alias is the protection — scripts and agents that bypass it lose it.**
|
|
141
|
+
> `CEREFOX_CONFIG_DIR=~/.cerefox/staging cerefox …` (bare binary, staging
|
|
142
|
+
> config) recombines the axes: production *code* on staging *data*. Aliases
|
|
143
|
+
> don't exist in non-interactive shells, which is exactly where automation
|
|
144
|
+
> runs, so spell out the pinned binary path there. And remember the global
|
|
145
|
+
> binary is itself a production component: `cerefox self-update` replaces it
|
|
146
|
+
> no matter which `CEREFOX_CONFIG_DIR` is set — updating "for staging" that
|
|
147
|
+
> way is a production change.
|
|
148
|
+
|
|
149
|
+
Verify the split before trusting it:
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
cerefox --version # production, e.g. 1.0.8
|
|
153
|
+
cfx-stg --version # staging, e.g. 1.1.0-beta.2
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
To move staging to a newer pre-release later, re-run the `npm install` above;
|
|
157
|
+
production is unaffected. If you'd rather not keep an install tree at all,
|
|
158
|
+
`npx --package=@cerefox/memory@beta cerefox …` works too, but it re-resolves the
|
|
159
|
+
package on every call and makes the version harder to pin down when something
|
|
160
|
+
misbehaves.
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## 4. Deploy and verify
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
cfx-stg server deploy # fresh database → schema + RPCs + 9 Edge Functions
|
|
168
|
+
cfx-stg doctor
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
`doctor` should report the staging config path and URL. Two lines are expected
|
|
172
|
+
on a fresh project:
|
|
173
|
+
|
|
174
|
+
- `ℹ edge functions … No CEREFOX_ACCESS_TOKEN set` — staging has no Cerefox
|
|
175
|
+
access token yet. Run `cfx-stg token generate` if you plan to test the Edge
|
|
176
|
+
Function / remote-MCP path there.
|
|
177
|
+
- `✓ content format no documents yet` — nothing imported.
|
|
178
|
+
|
|
179
|
+
**Checking database/schema state.** There is no `cerefox db status` command.
|
|
180
|
+
Use one of:
|
|
181
|
+
|
|
182
|
+
| Want | Command |
|
|
183
|
+
|---|---|
|
|
184
|
+
| Deployed schema version, reachability, drift banner | `cfx-stg doctor` |
|
|
185
|
+
| What a deploy *would* do, without doing it | `cfx-stg server deploy --dry-run` |
|
|
186
|
+
| Per-migration applied/pending list | `bun scripts/db_migrate.ts --status` (repo clone; contributor-only) |
|
|
187
|
+
|
|
188
|
+
`doctor` is the one to reach for; the migration list is only interesting when a
|
|
189
|
+
deploy behaved unexpectedly.
|
|
190
|
+
|
|
191
|
+
> **Contributor scripts and a repo `.env`.** The `bun scripts/*.ts` tools honour
|
|
192
|
+
> `CEREFOX_CONFIG_DIR` — but only from the version that fixed this. Bun
|
|
193
|
+
> auto-loads `.env` from the working directory, so on **v1.1.0-beta.1 and
|
|
194
|
+
> earlier** a repo clone with its own `.env` silently outranked the config dir:
|
|
195
|
+
> `CEREFOX_CONFIG_DIR=…/staging bun scripts/db_migrate.ts --status` reported
|
|
196
|
+
> *production*. If you're on an older build, run those scripts from a directory
|
|
197
|
+
> with no `.env`, and confirm the target before trusting the output.
|
|
198
|
+
|
|
199
|
+
---
|
|
200
|
+
|
|
201
|
+
## 5. Populate from a production snapshot
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
cerefox backup create # production
|
|
205
|
+
cfx-stg backup restore ~/.cerefox/backups/<snapshot>.json
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Backups carry documents, chunks, projects, memberships and (from v1.1.0)
|
|
209
|
+
relations and lifecycle status. Restore is idempotent — re-running it is safe
|
|
210
|
+
and converges on the same state rather than duplicating anything.
|
|
211
|
+
|
|
212
|
+
Restoring an **older** snapshot into a **newer** schema works — new columns take
|
|
213
|
+
their defaults. The reverse is not supported.
|
|
214
|
+
|
|
215
|
+
Two count details that look like bugs and are not:
|
|
216
|
+
|
|
217
|
+
- **Trashed documents are not carried over.** `backup create` captures live
|
|
218
|
+
documents only (`deleted_at IS NULL`), so anything in production's trash is
|
|
219
|
+
absent from staging. Nothing in the trash is lost *in production* — it just
|
|
220
|
+
isn't part of the snapshot.
|
|
221
|
+
- **A membership shortfall on pre-v1.1.0 snapshots.** Older `backup create`
|
|
222
|
+
captured the document↔project junction unfiltered, including rows belonging
|
|
223
|
+
to trashed documents. Restore drops those (their document isn't in the file),
|
|
224
|
+
so the summary reports fewer memberships than the header claims — 362 of 369
|
|
225
|
+
on the maintainer's store. From v1.1.0 the capture filters them out, and the
|
|
226
|
+
two numbers agree.
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
## 6. Run both web servers at once
|
|
231
|
+
|
|
232
|
+
The daemon's pidfile and log follow `CEREFOX_CONFIG_DIR`, so staging writes to
|
|
233
|
+
`~/.cerefox/staging/web.{pid,log}` and production to `~/.cerefox/web.{pid,log}`.
|
|
234
|
+
Each `web stop` / `web status` acts only on its own environment.
|
|
235
|
+
|
|
236
|
+
> **Requires v1.1.0-beta.2 or later.** On **beta.1 and earlier** both
|
|
237
|
+
> environments shared `~/.cerefox/web.pid`, so `cfx-stg web start` refuses with
|
|
238
|
+
> *"A Cerefox web daemon is already running on :8000"* — it is reading
|
|
239
|
+
> production's pidfile. Until staging is on a build with the fix, start it in
|
|
240
|
+
> the **foreground** instead, which never touches the pidfile:
|
|
241
|
+
>
|
|
242
|
+
> ```bash
|
|
243
|
+
> cfx-stg web --port 8030 # no `start` subcommand; Ctrl-C to stop
|
|
244
|
+
> ```
|
|
245
|
+
|
|
246
|
+
Pick a different port for staging so the two can run simultaneously:
|
|
247
|
+
|
|
248
|
+
```bash
|
|
249
|
+
cfx-stg web start --port 8010
|
|
250
|
+
cfx-stg web status # staging only
|
|
251
|
+
cerefox web status # production only — unaffected
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
Set `CEREFOX_ENV_LABEL=staging` in the staging `.env` (Step 2) and the
|
|
255
|
+
environment names itself everywhere it matters — two tabs that look identical
|
|
256
|
+
otherwise:
|
|
257
|
+
|
|
258
|
+
| Surface | With the label set |
|
|
259
|
+
|---|---|
|
|
260
|
+
| Web UI | A banner on every page: *"STAGING environment — not production."* |
|
|
261
|
+
| `doctor` | Title line reads `Cerefox doctor [STAGING]` |
|
|
262
|
+
| `backup create` | Filename becomes `cerefox-staging-<stamp>.json`, and the label is stored in the payload |
|
|
263
|
+
| `backup restore` | Warns when the snapshot's environment differs from the target's |
|
|
264
|
+
|
|
265
|
+
The filename and restore warning matter because `CEREFOX_BACKUP_DIR` does not
|
|
266
|
+
follow `CEREFOX_CONFIG_DIR`: staging and production snapshots can end up in one
|
|
267
|
+
directory, and a restore that picks "the most recent file" there would
|
|
268
|
+
otherwise seed production from staging without saying so.
|
|
269
|
+
|
|
270
|
+
**Requires v1.1.0-beta.3 or later** — on earlier builds `CEREFOX_ENV_LABEL` is
|
|
271
|
+
read but nothing acts on it.
|
|
272
|
+
|
|
273
|
+
---
|
|
274
|
+
|
|
275
|
+
## Known limits
|
|
276
|
+
|
|
277
|
+
| Area | Problem |
|
|
278
|
+
|---|---|
|
|
279
|
+
| **Agent (MCP) config** | Safe since v1.4.0 ([#168](https://github.com/fstamatelopoulos/cerefox/issues/168)). `configure-agent` names the server after `CEREFOX_ENV_LABEL`, so a staging environment registers as **`cerefox-staging`** alongside your production `cerefox` entry rather than replacing it, and an agent can hold both. The labelled entry also carries its own `CEREFOX_CONFIG_DIR` and `CEREFOX_ENV_LABEL` in the config it writes — without that the client would spawn the server with *its own* environment (a GUI client launched from the dock has none), `CEREFOX_CONFIG_DIR` would be absent, and an entry named `cerefox-staging` would quietly serve production. Before v1.4.0 it registered every environment as `cerefox` and running it from staging silently repointed all your agents. |
|
|
280
|
+
| **`doctor`'s `mcp clients` line** | It inspects your global agent configs, which point at production, and reports them even in staging mode. Informative, but easy to misread as "staging is wired to my agents". |
|
|
281
|
+
| **`doctor` remediation commands** | Environment-aware since v1.8.0: in a staging environment the remediations tell you to prefix `CEREFOX_CONFIG_DIR` (or use the alias). Before that, a staging `doctor` printed bare `cerefox …` remediations that act on production. |
|
|
282
|
+
|
|
283
|
+
Everything else — config, database, backups, CLI version, web daemon — is
|
|
284
|
+
separated and validated.
|
|
285
|
+
|
|
286
|
+
### What is *not* isolated, by design
|
|
287
|
+
|
|
288
|
+
Two things are shared on purpose, and both are safe:
|
|
289
|
+
|
|
290
|
+
- **`SUPABASE_ACCESS_TOKEN`** is an account-level credential, not a
|
|
291
|
+
project-level one. The same value belongs in both `.env` files; the project
|
|
292
|
+
it acts on comes from the URL and database URL beside it.
|
|
293
|
+
- **Your OpenAI key.** Staging embeds against the same account, so staging
|
|
294
|
+
ingestion and `migrate-format` rehearsals cost real money. That is the point —
|
|
295
|
+
it is how you measure the spend before committing production to it.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## Housekeeping
|
|
300
|
+
|
|
301
|
+
Pause the staging project in the Supabase dashboard between test rounds — a
|
|
302
|
+
paused project keeps its data without consuming free-tier compute. Check your
|
|
303
|
+
plan's limit on simultaneously active projects before creating the second one.
|
|
304
|
+
|
|
305
|
+
## Pause it when you are not using it
|
|
306
|
+
|
|
307
|
+
A Supabase project on the free tier costs nothing while paused, and leaving a
|
|
308
|
+
staging environment running between bursts of work is pure waste. Pause it from
|
|
309
|
+
the Supabase dashboard when you finish, and unpause it when you start.
|
|
310
|
+
|
|
311
|
+
Two things follow from that:
|
|
312
|
+
|
|
313
|
+
- **Give it a minute after unpausing.** The first queries against a
|
|
314
|
+
just-resumed project are slow while the instance wakes.
|
|
315
|
+
- **A paused project does not announce itself.** Commands hang or fail with
|
|
316
|
+
connection and timeout errors, and `cerefox doctor` reports the Data API as
|
|
317
|
+
unreachable. That looks identical to a broken deployment or a bad credential.
|
|
318
|
+
Check whether the project is paused before diagnosing anything else.
|
|
319
|
+
|
|
320
|
+
## Running the test suites against staging
|
|
321
|
+
|
|
322
|
+
The package suite's live tests write real documents. They resolve credentials
|
|
323
|
+
exactly as the CLI does, so a bare `bun test` on a maintainer's machine targets
|
|
324
|
+
**production** — which is how a run once left ~79 audit-log entries there. Since
|
|
325
|
+
v1.4.0 those suites **skip unless the target carries `CEREFOX_ENV_LABEL`**, so
|
|
326
|
+
point them at staging explicitly:
|
|
327
|
+
|
|
328
|
+
```bash
|
|
329
|
+
CEREFOX_CONFIG_DIR=~/.cerefox/staging bun test
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
`CEREFOX_ALLOW_PROD_WRITE_TESTS=1` overrides the guard, for the rare case where
|
|
333
|
+
production really is the intended target. Pointing every *command* at staging is
|
|
334
|
+
not enough: the test runner is a separate process with its own resolution.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cerefox/memory",
|
|
3
|
-
"version": "1.15.
|
|
3
|
+
"version": "1.15.3",
|
|
4
4
|
"description": "Cerefox \u2014 user-owned shared memory for AI agents. CLI + stdio MCP server + web UI + ingestion for a knowledge base on your own Supabase project (or fully self-hosted with Cerefox Local).",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"homepage": "https://github.com/fstamatelopoulos/cerefox",
|
|
@@ -40,27 +40,27 @@
|
|
|
40
40
|
],
|
|
41
41
|
"dependencies": {
|
|
42
42
|
"@hono/node-server": "^2.1.1",
|
|
43
|
-
"@modelcontextprotocol/sdk": "^1.30.
|
|
44
|
-
"@supabase/supabase-js": "^2.
|
|
43
|
+
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
44
|
+
"@supabase/supabase-js": "^2.117.2",
|
|
45
45
|
"cli-progress": "^3.12.0",
|
|
46
46
|
"commander": "^15.0.0",
|
|
47
|
-
"hono": "^4.13.
|
|
48
|
-
"mammoth": "^1.
|
|
47
|
+
"hono": "^4.13.9",
|
|
48
|
+
"mammoth": "^1.13.0",
|
|
49
49
|
"ora": "^9.4.0",
|
|
50
50
|
"picocolors": "^1.0.0",
|
|
51
51
|
"postgres": "^3.4.9",
|
|
52
52
|
"prompts": "^2.4.2",
|
|
53
|
-
"smol-toml": "^1.
|
|
53
|
+
"smol-toml": "^1.9.0",
|
|
54
54
|
"zod": "^4.6.5"
|
|
55
55
|
},
|
|
56
56
|
"optionalDependencies": {
|
|
57
|
-
"@huggingface/transformers": "^4.
|
|
57
|
+
"@huggingface/transformers": "^4.3.0",
|
|
58
58
|
"onnxruntime-node": "^1.21.0"
|
|
59
59
|
},
|
|
60
60
|
"devDependencies": {
|
|
61
|
-
"@types/bun": "^1.
|
|
61
|
+
"@types/bun": "^1.4.2",
|
|
62
62
|
"@types/cli-progress": "^3.11.5",
|
|
63
|
-
"@types/node": "^24.
|
|
63
|
+
"@types/node": "^24.19.0",
|
|
64
64
|
"@types/prompts": "^2.4.9",
|
|
65
65
|
"typescript": "^6.0.3"
|
|
66
66
|
},
|