create-ragen-app 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +47 -31
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +347 -30
- package/dist/cli.js.map +1 -1
- package/dist/config-file.d.ts +14 -0
- package/dist/config-file.d.ts.map +1 -0
- package/dist/config-file.js +21 -0
- package/dist/config-file.js.map +1 -0
- package/dist/config-template.d.ts +30 -0
- package/dist/config-template.d.ts.map +1 -0
- package/dist/config-template.js +126 -0
- package/dist/config-template.js.map +1 -0
- package/dist/encryption-provider.d.ts +35 -0
- package/dist/encryption-provider.d.ts.map +1 -0
- package/dist/encryption-provider.js +63 -0
- package/dist/encryption-provider.js.map +1 -0
- package/dist/llm-provider.d.ts +49 -10
- package/dist/llm-provider.d.ts.map +1 -1
- package/dist/llm-provider.js +69 -16
- package/dist/llm-provider.js.map +1 -1
- package/dist/manifest.d.ts.map +1 -1
- package/dist/manifest.js +6 -22
- package/dist/manifest.js.map +1 -1
- package/dist/manual-setup.d.ts.map +1 -1
- package/dist/manual-setup.js +84 -23
- package/dist/manual-setup.js.map +1 -1
- package/dist/route-table.d.ts +39 -0
- package/dist/route-table.d.ts.map +1 -0
- package/dist/route-table.js +53 -0
- package/dist/route-table.js.map +1 -0
- package/dist/storage-provider.d.ts +49 -0
- package/dist/storage-provider.d.ts.map +1 -0
- package/dist/storage-provider.js +84 -0
- package/dist/storage-provider.js.map +1 -0
- package/dist/worker-runtime.d.ts +101 -0
- package/dist/worker-runtime.d.ts.map +1 -0
- package/dist/worker-runtime.js +118 -0
- package/dist/worker-runtime.js.map +1 -0
- package/package.json +2 -2
- package/dist/litellm-config.d.ts +0 -17
- package/dist/litellm-config.d.ts.map +0 -1
- package/dist/litellm-config.js +0 -28
- package/dist/litellm-config.js.map +0 -1
package/README.md
CHANGED
|
@@ -9,25 +9,32 @@ npx create-ragen-app my-ragen-app
|
|
|
9
9
|
|
|
10
10
|
It clones the repo, generates every secret it safely can, lets you paste a
|
|
11
11
|
plain OpenAI or Anthropic API key instead of configuring an enterprise model
|
|
12
|
-
provider, starts the backing services (Postgres, Qdrant,
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
terminals:
|
|
12
|
+
provider, starts the backing services (Postgres, Qdrant, Redis, …) in Docker,
|
|
13
|
+
and runs the app's own first-run setup (Prisma client, migrations, seed
|
|
14
|
+
data). What is left is starting the three processes, in separate terminals:
|
|
16
15
|
|
|
17
16
|
```bash
|
|
18
17
|
cd my-ragen-app
|
|
19
|
-
npm run api:dev
|
|
20
|
-
npm run web:dev
|
|
18
|
+
npm run api:dev # apps/api — the web app creates threads through it
|
|
19
|
+
npm run web:dev # apps/web — http://localhost:3000
|
|
20
|
+
npm run worker:dev # apps/worker — document ingest runs here
|
|
21
21
|
```
|
|
22
22
|
|
|
23
23
|
`apps/api` is not optional: `apps/web` delegates thread creation, the thread
|
|
24
24
|
sidebar and notifications to it (ADR-21), so running only the web app gets you
|
|
25
|
-
a panel that loads and a chat that cannot open a thread.
|
|
25
|
+
a panel that loads and a chat that cannot open a thread. Neither is the
|
|
26
|
+
worker: without it an upload is accepted and never parsed — the queue fills
|
|
27
|
+
and nothing drains it.
|
|
28
|
+
|
|
29
|
+
The scaffold answers BullMQ, which runs on the Redis the compose file already
|
|
30
|
+
starts (ADR-44). Temporal is still selectable and is no longer a service this
|
|
31
|
+
install runs, so choosing it asks for the address of the server you run
|
|
32
|
+
yourself.
|
|
26
33
|
|
|
27
34
|
## Node version
|
|
28
35
|
|
|
29
36
|
A Ragen installation needs Node 24, and the wizard refuses to create one on
|
|
30
|
-
anything older
|
|
37
|
+
anything older _before_ it clones. Not because the CLI needs it — it does not,
|
|
31
38
|
and 0.2.0 was published and verified on Node 22 by accident — but because the
|
|
32
39
|
setup it runs for you (`npm install` across the monorepo, `prisma generate`,
|
|
33
40
|
`migrate deploy`, the seed) runs under the caller's Node and leaves a tree
|
|
@@ -44,21 +51,23 @@ requirement".
|
|
|
44
51
|
This package is the only thing that exercises the first-run path.
|
|
45
52
|
`.github/workflows/installer.yml` runs it on pushes to `main` and on pull
|
|
46
53
|
requests from this repository: it packs the package, installs the tarball,
|
|
47
|
-
scaffolds from the
|
|
54
|
+
scaffolds from the commit under review and asserts the result is configured.
|
|
48
55
|
That catches a broken installer, but only for the cases the assertions cover —
|
|
49
56
|
`AGENTS.md`'s Post-Task Workflow still asks for an update here in the same PR,
|
|
50
57
|
and for the PR description to say so.
|
|
51
58
|
|
|
52
59
|
What counts as install-affecting:
|
|
53
60
|
|
|
54
|
-
| Change
|
|
55
|
-
|
|
|
56
|
-
| A new or renamed env var that a fresh install must set
|
|
57
|
-
| A default model, embedding model or `VECTOR_SIZE` change
|
|
58
|
-
|
|
|
59
|
-
|
|
|
60
|
-
| A
|
|
61
|
-
| A
|
|
61
|
+
| Change | What to update here |
|
|
62
|
+
| --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
63
|
+
| A new or renamed env var that a fresh install must set | `src/manifest.ts` (generated secret or corrected default) |
|
|
64
|
+
| A default model, embedding model or `VECTOR_SIZE` change | `src/llm-provider.ts` — and the two must stay consistent, or Qdrant rejects every upsert |
|
|
65
|
+
| A new `@ragenai/llm-gateway` provider, or a change to route-table shape | `src/llm-provider.ts` (`gatewayProvider`, `upstreamModel`) and `src/route-table.ts` — the scaffold writes `infra/llm-gateway/routes.yaml`, which is what a scaffolded install actually reads |
|
|
66
|
+
| An app the web app can no longer run without | the outro in `src/cli.ts`, which tells people what to start |
|
|
67
|
+
| A variable a provider seam makes required | `src/worker-runtime.ts`, `src/storage-provider.ts` or `src/encryption-provider.ts` — the selection writes it, and `tests/architecture/create-ragen-app-knows-the-provider-seams.test.ts` fails when it does not |
|
|
68
|
+
| A new first-run step (migration, seed, generate) | `src/tasks.ts` and `maybeRunFirstTimeSetup` |
|
|
69
|
+
| A `docker-compose.yml` service, port or name change | `src/tasks.ts`, and the collision check in `ragenStackVolumeExists` |
|
|
70
|
+
| A gate that a brand-new install cannot pass (registration, licensing, onboarding) | usually the app, not this package — a fresh install must be able to reach a working chat without an administrator who does not exist yet |
|
|
62
71
|
|
|
63
72
|
`tests/architecture/create-ragen-app-manifest-is-current.test.ts` catches one
|
|
64
73
|
direction only: a key this package writes that the `.env.example` no longer
|
|
@@ -67,12 +76,12 @@ about — has no textual signal, which is why the rule above is a rule.
|
|
|
67
76
|
|
|
68
77
|
## Flags
|
|
69
78
|
|
|
70
|
-
| Flag
|
|
71
|
-
|
|
|
72
|
-
| `--ref=<
|
|
73
|
-
| `--skip-docker`
|
|
74
|
-
| `--skip-install`
|
|
75
|
-
| `--yes`
|
|
79
|
+
| Flag | Effect |
|
|
80
|
+
| ------------------------------ | --------------------------------------------------------------------------------- |
|
|
81
|
+
| `--ref=<ref>` | Clone a branch, tag or commit SHA other than `main` |
|
|
82
|
+
| `--skip-docker` | Write the `.env.local` files but don't start Docker |
|
|
83
|
+
| `--skip-install` | Skip `npm install` / Prisma / seed |
|
|
84
|
+
| `--yes` | Accept every default without prompting |
|
|
76
85
|
| `--provider=openai\|anthropic` | Take the API key from `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` instead of prompting |
|
|
77
86
|
|
|
78
87
|
Anything the wizard doesn't ask about (S3 storage, encryption at rest,
|
|
@@ -83,11 +92,11 @@ Stripe, email, MCP connectors) ships exactly as documented in the repo's own
|
|
|
83
92
|
## Declining to paste a key
|
|
84
93
|
|
|
85
94
|
Typing a provider API key into someone else's CLI is a reasonable thing to
|
|
86
|
-
refuse, so the provider step is skippable — pick
|
|
87
|
-
myself"
|
|
95
|
+
refuse, so the provider step is skippable — pick _"I will configure the routes
|
|
96
|
+
myself"_, or press enter on an empty key. The install still completes, and
|
|
88
97
|
`SETUP-LLM.md` is written into the new directory with the exact `.env.local`
|
|
89
|
-
values and
|
|
90
|
-
ordering constraint that matters (`VECTOR_SIZE` has to be right
|
|
98
|
+
values and route-table entries for OpenAI and Anthropic, plus the one
|
|
99
|
+
ordering constraint that matters (`VECTOR_SIZE` has to be right _before_ the
|
|
91
100
|
first document is indexed). `src/manual-setup.ts` generates that file from
|
|
92
101
|
`LLM_PROVIDERS`, so it cannot drift from what the wizard would have done.
|
|
93
102
|
|
|
@@ -95,7 +104,7 @@ first document is indexed). `src/manual-setup.ts` generates that file from
|
|
|
95
104
|
|
|
96
105
|
`src/manifest.ts` is the single list of which env vars get a generated
|
|
97
106
|
secret or a corrected local-dev default, and which files receive them. It is
|
|
98
|
-
deliberately an
|
|
107
|
+
deliberately an _overrides_ list — anything not in it is left exactly as the
|
|
99
108
|
cloned repo's own `.env.example` already has it. See the file's own comments
|
|
100
109
|
for the sharing rules (some secrets are the same value in two files, most are
|
|
101
110
|
independent).
|
|
@@ -117,7 +126,7 @@ npm install --prefix "$pack" "$pack"/create-ragen-app-*.tgz
|
|
|
117
126
|
Three details, each of which this file previously got wrong:
|
|
118
127
|
|
|
119
128
|
- **Install the tarball, do not hand it to `npx`.** `npx ./thing.tgz` does not
|
|
120
|
-
install and run a local package — it tries to
|
|
129
|
+
install and run a local package — it tries to _execute_ the file, and fails
|
|
121
130
|
with `Permission denied`. `.github/workflows/installer.yml` installs it,
|
|
122
131
|
which is why CI passed while the command written here never ran.
|
|
123
132
|
- **Glob the filename.** `npm pack` names the tarball after the version in
|
|
@@ -127,9 +136,16 @@ Three details, each of which this file previously got wrong:
|
|
|
127
136
|
expands to several paths, and `npm install` would be handed all of them.
|
|
128
137
|
|
|
129
138
|
`--ref` is the part people forget: the CLI clones the repository from GitHub,
|
|
130
|
-
so testing a change to the
|
|
139
|
+
so testing a change to the _app_ needs that branch pushed. Without it you are
|
|
131
140
|
testing new installer code against old repository content.
|
|
132
141
|
|
|
142
|
+
It takes anything GitHub's tarball endpoint resolves — a branch, a tag, or a
|
|
143
|
+
full commit SHA. The `Installer` workflow passes the **commit**, because a
|
|
144
|
+
branch can be deleted by the merge while the job is still running and the
|
|
145
|
+
endpoint then 404s on a name that resolved seconds earlier. Prefer a SHA
|
|
146
|
+
anywhere the ref is chosen by a machine rather than typed by hand. See
|
|
147
|
+
[the lesson](../../docs/lessons/a-ci-job-that-clones-a-branch-races-the-merge.md).
|
|
148
|
+
|
|
133
149
|
`prepare` runs `clean` before `build` deliberately. `files` publishes `dist`
|
|
134
150
|
wholesale and `tsc` does not remove the output of a source file that no
|
|
135
151
|
longer exists, so without the clean a renamed or deleted module ships as a
|
|
@@ -164,7 +180,7 @@ everyone else can see, rather than from whatever happens to be in the working
|
|
|
164
180
|
directory.
|
|
165
181
|
|
|
166
182
|
The two middle lines of the first block are there because of what `npm
|
|
167
|
-
version` does
|
|
183
|
+
version` does _not_ do for a workspace. Both have already cost this package a
|
|
168
184
|
release:
|
|
169
185
|
|
|
170
186
|
- **It does not commit, and does not tag.** In a single-package repository it
|
package/dist/cli.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":"AA+EA;;;;;;GAMG;AACH,wBAAsB,GAAG,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC,OAAO,CAAC,CAwO1D"}
|
package/dist/cli.js
CHANGED
|
@@ -40,13 +40,17 @@ const clack = __importStar(require("@clack/prompts"));
|
|
|
40
40
|
const dotenv_1 = require("dotenv");
|
|
41
41
|
const args_1 = require("./args");
|
|
42
42
|
const clone_1 = require("./clone");
|
|
43
|
+
const config_file_1 = require("./config-file");
|
|
44
|
+
const encryption_provider_1 = require("./encryption-provider");
|
|
43
45
|
const env_file_1 = require("./env-file");
|
|
44
|
-
const litellm_config_1 = require("./litellm-config");
|
|
45
46
|
const manual_setup_1 = require("./manual-setup");
|
|
47
|
+
const route_table_1 = require("./route-table");
|
|
46
48
|
const node_version_1 = require("./node-version");
|
|
47
49
|
const llm_provider_1 = require("./llm-provider");
|
|
48
50
|
const manifest_1 = require("./manifest");
|
|
49
51
|
const secrets_1 = require("./secrets");
|
|
52
|
+
const storage_provider_1 = require("./storage-provider");
|
|
53
|
+
const worker_runtime_1 = require("./worker-runtime");
|
|
50
54
|
const tasks_1 = require("./tasks");
|
|
51
55
|
/** Written into the new install whenever a model still has to be wired by hand. */
|
|
52
56
|
const MANUAL_SETUP_FILENAME = 'SETUP-LLM.md';
|
|
@@ -104,6 +108,34 @@ async function run(argv) {
|
|
|
104
108
|
if (llmChoice) {
|
|
105
109
|
Object.assign(rootOverrides, llmChoice.envUpdates);
|
|
106
110
|
}
|
|
111
|
+
// Storage and encryption were skipped by every earlier version of this
|
|
112
|
+
// wizard, and both are worse to answer later than now: turning encryption on
|
|
113
|
+
// afterwards leaves everything already written in plaintext, because nothing
|
|
114
|
+
// re-encrypts history, and moving storage strands the files already uploaded.
|
|
115
|
+
const storagePrompt = await resolveStorage(args);
|
|
116
|
+
if (storagePrompt.cancelled) {
|
|
117
|
+
clack.cancel('Cancelled.');
|
|
118
|
+
return false;
|
|
119
|
+
}
|
|
120
|
+
const encryptionPrompt = await resolveEncryption(args);
|
|
121
|
+
if (encryptionPrompt.cancelled) {
|
|
122
|
+
clack.cancel('Cancelled.');
|
|
123
|
+
return false;
|
|
124
|
+
}
|
|
125
|
+
// Asked here rather than left to an env edit, because choosing BullMQ is
|
|
126
|
+
// also what configures the queue dashboard — and an install that picked the
|
|
127
|
+
// runtime but not the dashboard would have no view of its own queues, which
|
|
128
|
+
// is what the Temporal UI gave away for free.
|
|
129
|
+
const runtimePrompt = await resolveWorkerRuntime(args);
|
|
130
|
+
if (runtimePrompt.cancelled) {
|
|
131
|
+
clack.cancel('Cancelled.');
|
|
132
|
+
return false;
|
|
133
|
+
}
|
|
134
|
+
const workerRuntime = runtimePrompt.selection;
|
|
135
|
+
Object.assign(rootOverrides, workerRuntime.envUpdates);
|
|
136
|
+
const storage = storagePrompt.selection;
|
|
137
|
+
const encryption = encryptionPrompt.selection;
|
|
138
|
+
Object.assign(rootOverrides, storage.envUpdates, encryption.envUpdates);
|
|
107
139
|
const rootWrite = writeEnvFile(targetDir, 'root', rootOverrides);
|
|
108
140
|
const adminWrite = writeEnvFile(targetDir, 'admin', adminOverrides);
|
|
109
141
|
const missingKeys = [...rootWrite.missingKeys, ...adminWrite.missingKeys];
|
|
@@ -114,8 +146,34 @@ async function run(argv) {
|
|
|
114
146
|
].join('\n'));
|
|
115
147
|
return false;
|
|
116
148
|
}
|
|
149
|
+
// After the env write, so a run that stops on drifted keys has not already
|
|
150
|
+
// rewritten a file in the clone.
|
|
151
|
+
try {
|
|
152
|
+
(0, config_file_1.writeRagenConfig)(targetDir, storage, encryption);
|
|
153
|
+
}
|
|
154
|
+
catch (error) {
|
|
155
|
+
clack.cancel(`Could not update ragen.config.ts: ${String(error)}\n` +
|
|
156
|
+
'The environment is written and correct; only the typed config is out of date. Fix its storage/encryption block by hand.');
|
|
157
|
+
return false;
|
|
158
|
+
}
|
|
159
|
+
if (encryption.generatedKey) {
|
|
160
|
+
clack.log.info([
|
|
161
|
+
'Generated an encryption key into .env.local as ENCRYPTION_MASTER_KEY.',
|
|
162
|
+
'Back it up: messages and documents encrypted with it cannot be read',
|
|
163
|
+
'without it, and nothing can re-derive it.',
|
|
164
|
+
].join(' '));
|
|
165
|
+
}
|
|
166
|
+
if (encryption.provider === 'none') {
|
|
167
|
+
clack.log.warn([
|
|
168
|
+
'Encryption is off, so messages and documents are stored in plaintext.',
|
|
169
|
+
'Turning it on later leaves everything written before then unencrypted —',
|
|
170
|
+
'nothing re-encrypts history.',
|
|
171
|
+
].join(' '));
|
|
172
|
+
}
|
|
117
173
|
if (llmChoice) {
|
|
118
|
-
|
|
174
|
+
// The route table is the whole of it now: B6 removed the proxy, so there
|
|
175
|
+
// is no second file to keep in step and no rollback that needs one.
|
|
176
|
+
(0, route_table_1.writeRouteTable)(targetDir, llmChoice.routes);
|
|
119
177
|
if (!llmChoice.embeddingsConfigured) {
|
|
120
178
|
clack.log.warn([
|
|
121
179
|
'Chat is configured, but this provider has no embeddings API — the',
|
|
@@ -133,10 +191,11 @@ async function run(argv) {
|
|
|
133
191
|
clack.note([
|
|
134
192
|
'No LLM provider configured, so chat and the knowledge base are off.',
|
|
135
193
|
'',
|
|
136
|
-
'Two files to edit
|
|
137
|
-
' .env.local
|
|
138
|
-
'
|
|
139
|
-
'
|
|
194
|
+
'Two files to edit:',
|
|
195
|
+
' .env.local — your key, DEFAULT_MODEL,',
|
|
196
|
+
' REPHRASE_MODEL, EMBEDDINGS_MODEL,',
|
|
197
|
+
' VECTOR_SIZE',
|
|
198
|
+
` ${route_table_1.ROUTE_TABLE_PATH} — a route per model`,
|
|
140
199
|
'',
|
|
141
200
|
`Exact values for OpenAI and Anthropic are written to ${guidePath}.`,
|
|
142
201
|
].join('\n'), 'Configure a model later');
|
|
@@ -157,18 +216,34 @@ async function run(argv) {
|
|
|
157
216
|
return false;
|
|
158
217
|
}
|
|
159
218
|
}
|
|
219
|
+
// Printed before the outro, and printed at all because it is the only time
|
|
220
|
+
// this value is readable: it is written to .env.local and never shown again.
|
|
221
|
+
if (workerRuntime.dashboard) {
|
|
222
|
+
const { user, password, port } = workerRuntime.dashboard;
|
|
223
|
+
clack.note([
|
|
224
|
+
`http://localhost:${port}`,
|
|
225
|
+
`user: ${user}`,
|
|
226
|
+
`password: ${password}`,
|
|
227
|
+
'',
|
|
228
|
+
'Generated for this install and written to .env.local. It is not a',
|
|
229
|
+
'shared default — every install gets its own — so save it now if you',
|
|
230
|
+
'want it somewhere other than that file.',
|
|
231
|
+
].join('\n'), 'Queue dashboard');
|
|
232
|
+
}
|
|
160
233
|
clack.outro([
|
|
161
234
|
'Done. Next steps:',
|
|
162
235
|
` cd ${targetDir}`,
|
|
163
236
|
' npm run api:dev # in one terminal',
|
|
164
237
|
' npm run web:dev # in another',
|
|
238
|
+
' npm run worker:dev # in a third — document ingest runs here',
|
|
165
239
|
'',
|
|
166
240
|
// Both, not just web: apps/web delegates thread creation, the thread
|
|
167
241
|
// sidebar and notifications to apps/api (ADR-21), so starting only the
|
|
168
242
|
// web app gets you a panel that loads and a chat that cannot open a
|
|
169
243
|
// thread.
|
|
170
244
|
'apps/api is not optional — the web app creates threads through it,',
|
|
171
|
-
'so chat fails without it.',
|
|
245
|
+
'so chat fails without it. Without the worker an upload is accepted and',
|
|
246
|
+
'never parsed: the queue fills and nothing drains it.',
|
|
172
247
|
'',
|
|
173
248
|
'App: http://localhost:3000',
|
|
174
249
|
'API: http://localhost:3001',
|
|
@@ -241,6 +316,218 @@ function writeEnvFile(targetDir, target, overrides) {
|
|
|
241
316
|
(0, node_fs_1.writeFileSync)(localPath, content, { mode: 0o600 });
|
|
242
317
|
return { content, missingKeys };
|
|
243
318
|
}
|
|
319
|
+
/**
|
|
320
|
+
* Whether the wizard may ask anything at all.
|
|
321
|
+
*
|
|
322
|
+
* `--yes` is the obvious one. `--provider=` is the other, and it is not
|
|
323
|
+
* obvious: that flag exists so CI can run this wizard, *because CI cannot
|
|
324
|
+
* answer a prompt*. Adding a question that `--provider` does not silence would
|
|
325
|
+
* hang every automated install — the flag's whole purpose, undone by a
|
|
326
|
+
* question about buckets.
|
|
327
|
+
*
|
|
328
|
+
* A person who would rather not type an API key into a CLI, and does want to
|
|
329
|
+
* be asked about storage, omits the flag and exports the key instead.
|
|
330
|
+
*/
|
|
331
|
+
function isUnattended(args) {
|
|
332
|
+
return args.yes || args.provider !== undefined;
|
|
333
|
+
}
|
|
334
|
+
/**
|
|
335
|
+
* Unattended installs take the shipped defaults: files on disk (ADR-27) and a
|
|
336
|
+
* generated encryption key.
|
|
337
|
+
*
|
|
338
|
+
* Encryption is *on* in that path on purpose. The alternative defaults an
|
|
339
|
+
* install to plaintext, and the only cost of the key is a line in `.env.local`
|
|
340
|
+
* the wizard writes itself — where the cost of not having it is that every
|
|
341
|
+
* message written before someone thinks to turn it on stays readable forever.
|
|
342
|
+
*/
|
|
343
|
+
async function resolveStorage(args) {
|
|
344
|
+
if (isUnattended(args)) {
|
|
345
|
+
return { cancelled: false, selection: (0, storage_provider_1.resolveStorageSelection)('local') };
|
|
346
|
+
}
|
|
347
|
+
return promptStorage();
|
|
348
|
+
}
|
|
349
|
+
async function resolveEncryption(args) {
|
|
350
|
+
if (isUnattended(args)) {
|
|
351
|
+
return { cancelled: false, selection: (0, encryption_provider_1.resolveEncryptionSelection)('local') };
|
|
352
|
+
}
|
|
353
|
+
return promptEncryption();
|
|
354
|
+
}
|
|
355
|
+
/**
|
|
356
|
+
* Unattended installs take BullMQ, the shipped default.
|
|
357
|
+
*
|
|
358
|
+
* The reasoning did not change when the answer did: `--yes` means "accept the
|
|
359
|
+
* defaults", not "pick an engine on the operator's behalf". It used to answer
|
|
360
|
+
* Temporal because that is what the compose file ran; ADR-44 took Temporal out
|
|
361
|
+
* of it, so answering Temporal now would scaffold an install whose worker
|
|
362
|
+
* connects to a server nobody started. An install that wants durable execution
|
|
363
|
+
* answers the prompt, and brings its own address with it.
|
|
364
|
+
*/
|
|
365
|
+
async function resolveWorkerRuntime(args) {
|
|
366
|
+
if (isUnattended(args)) {
|
|
367
|
+
return {
|
|
368
|
+
cancelled: false,
|
|
369
|
+
selection: (0, worker_runtime_1.resolveWorkerRuntimeSelection)('bullmq'),
|
|
370
|
+
};
|
|
371
|
+
}
|
|
372
|
+
return promptWorkerRuntime();
|
|
373
|
+
}
|
|
374
|
+
async function promptWorkerRuntime() {
|
|
375
|
+
const choice = await clack.select({
|
|
376
|
+
message: 'Which engine should run background jobs?',
|
|
377
|
+
// BullMQ first, because clack's first option is the one a bare Enter
|
|
378
|
+
// takes — and it is the one this install's compose file can actually run.
|
|
379
|
+
options: ['bullmq', 'temporal'].map((value) => ({
|
|
380
|
+
value,
|
|
381
|
+
label: worker_runtime_1.WORKER_RUNTIME_LABELS[value],
|
|
382
|
+
})),
|
|
383
|
+
});
|
|
384
|
+
if (clack.isCancel(choice)) {
|
|
385
|
+
return { cancelled: true };
|
|
386
|
+
}
|
|
387
|
+
if (choice !== 'temporal') {
|
|
388
|
+
return {
|
|
389
|
+
cancelled: false,
|
|
390
|
+
selection: (0, worker_runtime_1.resolveWorkerRuntimeSelection)('bullmq'),
|
|
391
|
+
};
|
|
392
|
+
}
|
|
393
|
+
// Asked rather than defaulted, because the answer stopped being local.
|
|
394
|
+
// Temporal left the compose file with ADR-44, so this address names a server
|
|
395
|
+
// the operator runs; `localhost:7233` is right on a laptop and silent
|
|
396
|
+
// everywhere else — a deployed worker pointing at its own container connects
|
|
397
|
+
// to nothing and processes nothing, with no error to read.
|
|
398
|
+
clack.log.warn([
|
|
399
|
+
'Temporal is not in this install’s docker-compose.yml (ADR-44) — it is an',
|
|
400
|
+
'adapter now, and you run the server yourself. Nothing below starts one.',
|
|
401
|
+
].join(' '));
|
|
402
|
+
const address = await clack.text({
|
|
403
|
+
message: 'Temporal server address?',
|
|
404
|
+
initialValue: worker_runtime_1.DEFAULT_TEMPORAL_SERVER_ADDRESS,
|
|
405
|
+
placeholder: worker_runtime_1.DEFAULT_TEMPORAL_SERVER_ADDRESS,
|
|
406
|
+
});
|
|
407
|
+
if (clack.isCancel(address)) {
|
|
408
|
+
return { cancelled: true };
|
|
409
|
+
}
|
|
410
|
+
return {
|
|
411
|
+
cancelled: false,
|
|
412
|
+
selection: (0, worker_runtime_1.resolveWorkerRuntimeSelection)('temporal', {
|
|
413
|
+
temporalServerAddress: String(address),
|
|
414
|
+
}),
|
|
415
|
+
};
|
|
416
|
+
}
|
|
417
|
+
async function promptStorage() {
|
|
418
|
+
const choice = await clack.select({
|
|
419
|
+
message: 'Where should uploaded documents be stored?',
|
|
420
|
+
options: ['local', 's3'].map((value) => ({
|
|
421
|
+
value,
|
|
422
|
+
label: storage_provider_1.STORAGE_LABELS[value],
|
|
423
|
+
})),
|
|
424
|
+
});
|
|
425
|
+
if (clack.isCancel(choice)) {
|
|
426
|
+
return { cancelled: true };
|
|
427
|
+
}
|
|
428
|
+
if (choice === 'local') {
|
|
429
|
+
return { cancelled: false, selection: (0, storage_provider_1.resolveStorageSelection)('local') };
|
|
430
|
+
}
|
|
431
|
+
const bucket = await clack.text({
|
|
432
|
+
message: 'Bucket name',
|
|
433
|
+
validate: (value) => (value.trim() ? undefined : 'Required for S3.'),
|
|
434
|
+
});
|
|
435
|
+
if (clack.isCancel(bucket)) {
|
|
436
|
+
return { cancelled: true };
|
|
437
|
+
}
|
|
438
|
+
const region = await clack.text({
|
|
439
|
+
message: 'Region',
|
|
440
|
+
placeholder: 'fr-par, us-east-1, auto',
|
|
441
|
+
validate: (value) => (value.trim() ? undefined : 'Required for S3.'),
|
|
442
|
+
});
|
|
443
|
+
if (clack.isCancel(region)) {
|
|
444
|
+
return { cancelled: true };
|
|
445
|
+
}
|
|
446
|
+
const endpoint = await clack.text({
|
|
447
|
+
message: 'Endpoint URL (blank for AWS)',
|
|
448
|
+
placeholder: 'https://s3.fr-par.scw.cloud',
|
|
449
|
+
defaultValue: '',
|
|
450
|
+
});
|
|
451
|
+
if (clack.isCancel(endpoint)) {
|
|
452
|
+
return { cancelled: true };
|
|
453
|
+
}
|
|
454
|
+
const accessKeyId = await clack.text({
|
|
455
|
+
message: 'Access key ID',
|
|
456
|
+
validate: (value) => (value.trim() ? undefined : 'Required for S3.'),
|
|
457
|
+
});
|
|
458
|
+
if (clack.isCancel(accessKeyId)) {
|
|
459
|
+
return { cancelled: true };
|
|
460
|
+
}
|
|
461
|
+
// password, not text: this one is a secret and should not be echoed into a
|
|
462
|
+
// terminal someone may be sharing or recording.
|
|
463
|
+
const secretAccessKey = await clack.password({
|
|
464
|
+
message: 'Secret access key',
|
|
465
|
+
});
|
|
466
|
+
if (clack.isCancel(secretAccessKey)) {
|
|
467
|
+
return { cancelled: true };
|
|
468
|
+
}
|
|
469
|
+
if (!secretAccessKey.trim()) {
|
|
470
|
+
clack.log.warn('No secret key given — falling back to local storage.');
|
|
471
|
+
return { cancelled: false, selection: (0, storage_provider_1.resolveStorageSelection)('local') };
|
|
472
|
+
}
|
|
473
|
+
return {
|
|
474
|
+
cancelled: false,
|
|
475
|
+
selection: (0, storage_provider_1.resolveStorageSelection)('s3', {
|
|
476
|
+
bucket,
|
|
477
|
+
region,
|
|
478
|
+
endpoint,
|
|
479
|
+
accessKeyId,
|
|
480
|
+
secretAccessKey,
|
|
481
|
+
}),
|
|
482
|
+
};
|
|
483
|
+
}
|
|
484
|
+
async function promptEncryption() {
|
|
485
|
+
const choice = await clack.select({
|
|
486
|
+
message: 'Encrypt messages and documents at rest?',
|
|
487
|
+
options: ['local', 'scaleway', 'kms', 'none'].map((value) => ({
|
|
488
|
+
value,
|
|
489
|
+
label: encryption_provider_1.ENCRYPTION_LABELS[value],
|
|
490
|
+
})),
|
|
491
|
+
});
|
|
492
|
+
if (clack.isCancel(choice)) {
|
|
493
|
+
return { cancelled: true };
|
|
494
|
+
}
|
|
495
|
+
const provider = choice;
|
|
496
|
+
if (provider === 'none' || provider === 'local') {
|
|
497
|
+
return {
|
|
498
|
+
cancelled: false,
|
|
499
|
+
selection: (0, encryption_provider_1.resolveEncryptionSelection)(provider),
|
|
500
|
+
};
|
|
501
|
+
}
|
|
502
|
+
const keyId = await clack.text({
|
|
503
|
+
message: provider === 'kms' ? 'KMS key id or ARN' : 'Key Manager key id',
|
|
504
|
+
validate: (value) => (value.trim() ? undefined : 'Required.'),
|
|
505
|
+
});
|
|
506
|
+
if (clack.isCancel(keyId)) {
|
|
507
|
+
return { cancelled: true };
|
|
508
|
+
}
|
|
509
|
+
if (provider === 'kms') {
|
|
510
|
+
return {
|
|
511
|
+
cancelled: false,
|
|
512
|
+
selection: (0, encryption_provider_1.resolveEncryptionSelection)('kms', { keyId, apiKey: '' }),
|
|
513
|
+
};
|
|
514
|
+
}
|
|
515
|
+
// Validated rather than given a fallback, unlike the S3 secret: someone who
|
|
516
|
+
// chose Scaleway has no second choice to fall back to, and a blank key would
|
|
517
|
+
// write `SCW_API_KEY=` — a provider named with a credential that reads as
|
|
518
|
+
// unset, which the boot check then refuses.
|
|
519
|
+
const apiKey = await clack.password({
|
|
520
|
+
message: 'Scaleway API secret key',
|
|
521
|
+
validate: (value) => (value.trim() ? undefined : 'Required for Scaleway.'),
|
|
522
|
+
});
|
|
523
|
+
if (clack.isCancel(apiKey)) {
|
|
524
|
+
return { cancelled: true };
|
|
525
|
+
}
|
|
526
|
+
return {
|
|
527
|
+
cancelled: false,
|
|
528
|
+
selection: (0, encryption_provider_1.resolveEncryptionSelection)('scaleway', { keyId, apiKey }),
|
|
529
|
+
};
|
|
530
|
+
}
|
|
244
531
|
/**
|
|
245
532
|
* Three ways to arrive at a provider, in precedence order: named on the
|
|
246
533
|
* command line with the key in the environment, declined wholesale by
|
|
@@ -270,21 +557,41 @@ async function resolveLlmProvider(args) {
|
|
|
270
557
|
* a typo'd variable name in a CI job.
|
|
271
558
|
*/
|
|
272
559
|
function providerFromEnvironment(choice) {
|
|
273
|
-
const
|
|
560
|
+
const config = llm_provider_1.LLM_PROVIDERS[choice];
|
|
561
|
+
const { apiKeyEnvVar, label } = config;
|
|
274
562
|
const apiKey = process.env[apiKeyEnvVar]?.trim();
|
|
275
563
|
if (!apiKey) {
|
|
276
564
|
clack.cancel(`--provider=${choice} needs the ${label} key in ${apiKeyEnvVar}, and that variable is empty. Export it and run again, or drop the flag to be asked for it.`);
|
|
277
565
|
return { cancelled: true };
|
|
278
566
|
}
|
|
279
|
-
|
|
567
|
+
// Same rule as the key: an explicit `--provider` that cannot be configured
|
|
568
|
+
// stops the install rather than scaffolding a half-wired one.
|
|
569
|
+
// `.trim()` for the same reason the prompt does it: an exported variable
|
|
570
|
+
// carries whatever the shell had, and whitespace is not a base URL.
|
|
571
|
+
const baseUrl = config.baseUrl
|
|
572
|
+
? process.env[config.baseUrl.envVar]?.trim()
|
|
573
|
+
: undefined;
|
|
574
|
+
if (config.baseUrl && !baseUrl) {
|
|
575
|
+
clack.cancel(`--provider=${choice} also needs ${config.baseUrl.envVar} — its endpoint is per-project, so a key alone cannot reach it. Export it and run again.`);
|
|
576
|
+
return { cancelled: true };
|
|
577
|
+
}
|
|
578
|
+
return {
|
|
579
|
+
cancelled: false,
|
|
580
|
+
choice: (0, llm_provider_1.resolveLlmProviderChoice)(choice, apiKey, baseUrl),
|
|
581
|
+
};
|
|
280
582
|
}
|
|
281
583
|
async function promptLlmProvider() {
|
|
282
584
|
const choice = await clack.select({
|
|
283
585
|
message: 'Which LLM provider should power chat?',
|
|
586
|
+
// Ordered by how far one key gets a new install, not by vendor fame. The
|
|
587
|
+
// first two configure chat *and* embeddings, so the knowledge base works
|
|
588
|
+
// rather than only the chat box.
|
|
284
589
|
options: [
|
|
590
|
+
{ value: 'openrouter', label: llm_provider_1.LLM_PROVIDERS.openrouter.label },
|
|
591
|
+
{ value: 'scaleway', label: llm_provider_1.LLM_PROVIDERS.scaleway.label },
|
|
285
592
|
{ value: 'openai', label: llm_provider_1.LLM_PROVIDERS.openai.label },
|
|
286
593
|
{ value: 'anthropic', label: llm_provider_1.LLM_PROVIDERS.anthropic.label },
|
|
287
|
-
{ value: 'skip', label:
|
|
594
|
+
{ value: 'skip', label: 'I will configure the routes myself' },
|
|
288
595
|
],
|
|
289
596
|
});
|
|
290
597
|
if (clack.isCancel(choice)) {
|
|
@@ -293,8 +600,9 @@ async function promptLlmProvider() {
|
|
|
293
600
|
if (choice === 'skip') {
|
|
294
601
|
return { cancelled: false, choice: undefined };
|
|
295
602
|
}
|
|
603
|
+
const config = llm_provider_1.LLM_PROVIDERS[choice];
|
|
296
604
|
const apiKey = await clack.password({
|
|
297
|
-
message: `Paste your ${
|
|
605
|
+
message: `Paste your ${config.label} API key`,
|
|
298
606
|
});
|
|
299
607
|
if (clack.isCancel(apiKey)) {
|
|
300
608
|
return { cancelled: true };
|
|
@@ -302,22 +610,29 @@ async function promptLlmProvider() {
|
|
|
302
610
|
if (!apiKey) {
|
|
303
611
|
return { cancelled: false, choice: undefined };
|
|
304
612
|
}
|
|
613
|
+
// A provider whose endpoint is not a constant needs a second value, and
|
|
614
|
+
// needs it before the install is written — Scaleway's base URL carries the
|
|
615
|
+
// project id, so a key on its own scaffolds something that 404s later.
|
|
616
|
+
let baseUrl;
|
|
617
|
+
if (config.baseUrl) {
|
|
618
|
+
const answer = await clack.text({ message: config.baseUrl.prompt });
|
|
619
|
+
if (clack.isCancel(answer)) {
|
|
620
|
+
return { cancelled: true };
|
|
621
|
+
}
|
|
622
|
+
// Trimmed before it is judged, not after. A pasted URL routinely carries a
|
|
623
|
+
// trailing newline or space, and a string of only whitespace is truthy —
|
|
624
|
+
// so `!answer` alone would accept " " and write it as `SCW_API_BASE`,
|
|
625
|
+
// which is the half-wired install this prompt exists to prevent.
|
|
626
|
+
baseUrl = answer.trim();
|
|
627
|
+
if (!baseUrl) {
|
|
628
|
+
return { cancelled: false, choice: undefined };
|
|
629
|
+
}
|
|
630
|
+
}
|
|
305
631
|
return {
|
|
306
632
|
cancelled: false,
|
|
307
|
-
choice: (0, llm_provider_1.resolveLlmProviderChoice)(choice, apiKey),
|
|
633
|
+
choice: (0, llm_provider_1.resolveLlmProviderChoice)(choice, apiKey, baseUrl),
|
|
308
634
|
};
|
|
309
635
|
}
|
|
310
|
-
function patchLiteLLMConfig(targetDir, entries) {
|
|
311
|
-
const configPath = (0, node_path_1.join)(targetDir, 'infra/litellm/config.yaml');
|
|
312
|
-
// Reversed because each insert goes directly under `model_list:`, so the
|
|
313
|
-
// last one applied ends up on top. Reading the file afterwards should show
|
|
314
|
-
// the chat model first and the embedding model under it, in the order the
|
|
315
|
-
// wizard asked about them.
|
|
316
|
-
const patched = [...entries]
|
|
317
|
-
.reverse()
|
|
318
|
-
.reduce((config, entry) => (0, litellm_config_1.addLiteLLMModel)(config, entry), (0, node_fs_1.readFileSync)(configPath, 'utf8'));
|
|
319
|
-
(0, node_fs_1.writeFileSync)(configPath, patched);
|
|
320
|
-
}
|
|
321
636
|
/**
|
|
322
637
|
* Leaves the manual instructions on disk as well as on screen. A terminal
|
|
323
638
|
* that has just scrolled a `docker compose` build past them is exactly the
|
|
@@ -373,18 +688,20 @@ async function maybeStartDocker(targetDir, yes) {
|
|
|
373
688
|
` cd ${targetDir} \\`,
|
|
374
689
|
' && RAGEN_STACK_NAME=my-ragen \\',
|
|
375
690
|
' POSTGRES_PORT=55532 QDRANT_PORT=6343 QDRANT_GRPC_PORT=6344 \\',
|
|
376
|
-
' TEMPORAL_PORT=7243 TEMPORAL_UI_PORT=8090 \\',
|
|
377
|
-
' LITELLM_PORT=4010 LITELLM_DB_PORT=55533 \\',
|
|
378
691
|
' DOCLING_PORT=5011 REDIS_PORT=56479 \\',
|
|
379
692
|
' PRESIDIO_ANALYZER_PORT=5012 PRESIDIO_ANONYMIZER_PORT=5013 \\',
|
|
380
693
|
' docker compose up -d',
|
|
381
694
|
'',
|
|
382
|
-
'Then update .env.local to match: DATABASE_URL, QDRANT_URL,',
|
|
383
|
-
'
|
|
384
|
-
'
|
|
695
|
+
'Then update .env.local to match: DATABASE_URL, QDRANT_URL, REDIS_URL,',
|
|
696
|
+
'DOCLING_URL and the two PRESIDIO_* URLs all name a host port.',
|
|
697
|
+
'',
|
|
698
|
+
// Not a compose port: the worker serves it itself, so two workers on
|
|
699
|
+
// one machine collide on it without docker-compose having an opinion.
|
|
700
|
+
'WORKER_ADMIN_PORT too, if both installs run a worker — the queue',
|
|
701
|
+
'dashboard is served by the worker process, not by a container.',
|
|
385
702
|
].join('\n'));
|
|
386
703
|
}
|
|
387
|
-
const proceed = await confirmOrSkip('Start the backing services now? (Postgres, Qdrant,
|
|
704
|
+
const proceed = await confirmOrSkip('Start the backing services now? (Postgres, Qdrant, Redis, …)', yes);
|
|
388
705
|
if (!proceed) {
|
|
389
706
|
return true;
|
|
390
707
|
}
|
|
@@ -392,7 +709,7 @@ async function maybeStartDocker(targetDir, yes) {
|
|
|
392
709
|
clack.log.warn('Docker does not seem to be available — skipping. Install Docker and run `docker compose up -d` yourself.');
|
|
393
710
|
return true;
|
|
394
711
|
}
|
|
395
|
-
return runStep(clack.spinner(), 'Starting docker compose (Postgres, Qdrant,
|
|
712
|
+
return runStep(clack.spinner(), 'Starting docker compose (Postgres, Qdrant, Redis, …)', 'Backing services started.', () => (0, tasks_1.startDockerServices)({ cwd: targetDir }));
|
|
396
713
|
}
|
|
397
714
|
async function maybeRunFirstTimeSetup(targetDir, yes, env) {
|
|
398
715
|
const proceed = await confirmOrSkip('Install dependencies and run first-time setup (prisma generate, migrate, seed)?', yes);
|