@cloudgrid-io/mcp 0.20.5 → 0.20.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +2 -1
- package/src/corpus/capability-map.md +21 -13
- package/src/corpus/cloudgrid-yaml.md +5 -3
- package/src/corpus/examples/ai-app/index.md +4 -3
- package/src/corpus/templates/ai-app/README.md +4 -4
- package/src/corpus/templates/ai-app/index.md +4 -4
- package/src/corpus/templates/ai-knowledge-base/AGENTS.md +17 -17
- package/src/corpus/templates/ai-knowledge-base/README.md +5 -4
- package/src/corpus/templates/ai-knowledge-base/cloudgrid.yaml +5 -4
- package/src/corpus/templates/ai-knowledge-base/index.md +15 -11
- package/src/corpus/templates/semantic-search/AGENTS.md +9 -9
- package/src/corpus/templates/semantic-search/README.md +4 -3
- package/src/corpus/templates/semantic-search/cloudgrid.yaml +6 -5
- package/src/corpus/templates/semantic-search/index.md +7 -4
- package/src/corpus/templates/simple-semantic-search/README.md +20 -0
- package/src/corpus/templates/simple-semantic-search/cloudgrid.yaml +235 -0
- package/src/corpus/templates/simple-semantic-search/index.md +353 -0
- package/src/corpus/templates/simple-semantic-search/services/web/package.json +15 -0
- package/src/corpus/templates/simple-semantic-search/services/web/src/document.txt +40 -0
- package/src/corpus/templates/simple-semantic-search/services/web/src/index.js +225 -0
- package/src/corpus/workflows/ai-app.md +5 -4
- package/src/corpus/workflows/ai-knowledge-base.md +17 -14
- package/src/corpus/workflows/semantic-search.md +9 -7
- package/src/corpus/workflows/simple-semantic-search.md +64 -0
- package/src/session-logger.js +9 -2
- package/src/tools.js +25 -2
- package/src/web.js +4 -2
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cloudgrid-io/mcp",
|
|
3
|
-
"version": "0.20.
|
|
3
|
+
"version": "0.20.7",
|
|
4
4
|
"description": "Use CloudGrid via MCP to brainstorm, build, deploy, and improve apps, games, websites, dashboards, and tools and get a live URL - templates, recipes, managed databases and AI, instant single-page publish, and full-app deploys, for developers and non-technical users. Three editions: local stdio (full toolset), hosted web (light, CLI-free), and a public docs-search server, over MCP.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -29,6 +29,7 @@
|
|
|
29
29
|
"test:grid-picker": "node test/grid-picker.test.mjs",
|
|
30
30
|
"test:plug-timeout": "node test/plug-timeout.test.mjs",
|
|
31
31
|
"test:manifest-confirm": "node test/manifest-confirm.test.mjs",
|
|
32
|
+
"test:qa-courier": "node test/qa-courier.test.mjs",
|
|
32
33
|
"test:drift-guard": "node test/drift-guard.mjs",
|
|
33
34
|
"smoke:drop": "node test/smoke-drop.mjs",
|
|
34
35
|
"smoke:login": "node test/smoke-login.mjs",
|
|
@@ -18,7 +18,7 @@ cloudgrid.yaml schema (every field, the `needs:` injection table, the
|
|
|
18
18
|
requires-vs-needs caveat, validation rules), fetch the companion reference:
|
|
19
19
|
`grid_get_template("doc", "cloudgrid-yaml")`.
|
|
20
20
|
|
|
21
|
-
## The
|
|
21
|
+
## The 61 templates
|
|
22
22
|
|
|
23
23
|
| Intent (match on `when:`) | Template | `needs:` | Deploy | Edition |
|
|
24
24
|
|---|---|---|---|---|
|
|
@@ -63,7 +63,8 @@ requires-vs-needs caveat, validation rules), fetch the companion reference:
|
|
|
63
63
|
| revenue dashboard, sales dashboard, MRR/revenue view | `revenue-dashboard` | `database: true` | runtime (async, poll) | local |
|
|
64
64
|
| property listings, real estate site, rentals/homes listings | `property-listings` | `database: true` | runtime (async, poll) | local |
|
|
65
65
|
| project management, projects and tasks tracker, team project board | `project-management` | `database: true` | runtime (async, poll) | local |
|
|
66
|
-
| search over
|
|
66
|
+
| semantic search over a document / article / text, search-by-meaning box, question-answering search over content, embeddings / vector / similarity search — ONE document or small text set, one service | `simple-semantic-search` | `vector: pgvector, ai: true` (native pgvector, verified live 2026-07-16) | runtime (async, poll) | local |
|
|
67
|
+
| search over my documents / PDFs / notes / knowledge base with upload UI and scheduled refresh, document search portfolio, searchable archive (multi-service) | `semantic-search` | `database: true` (template uses Mongo embeddings; `vector: pgvector` now available, #1545 shipped; active daily refresh cron) | runtime (async, poll) | local |
|
|
67
68
|
|
|
68
69
|
**Rule of thumb:** if the app must SAVE/remember data, share state across
|
|
69
70
|
users/sessions, log in, or store submissions → it is persistent → runtime, local
|
|
@@ -86,8 +87,10 @@ real runnable code in a different shape from the Next.js family: a React (Vite)
|
|
|
86
87
|
**static** frontend at `/` plus a **Python FastAPI** backend at `/backend`, on the
|
|
87
88
|
same grid-shared Mongo (`needs: { database: true }`). Hybrid search = Mongo
|
|
88
89
|
`$text` + in-app NumPy cosine over embedding arrays + metadata filters. It stores
|
|
89
|
-
embeddings in the Mongo `chunks` collection (
|
|
90
|
-
|
|
90
|
+
embeddings in the Mongo `chunks` collection (the template does not declare
|
|
91
|
+
`needs: vector`; `vector: pgvector` is now available — **#1545** shipped, verified
|
|
92
|
+
live 2026-07-16 — but the template code still ranks in-app) and refreshes via a
|
|
93
|
+
manager endpoint (NO active cron — a
|
|
91
94
|
Python `type: cron` is blocked on **#1585**). Read its `AGENTS.md` for the source
|
|
92
95
|
adapter (dropbox/local/url) + embeddings wiring and the health-without-secrets /
|
|
93
96
|
startup-index patterns.
|
|
@@ -97,7 +100,7 @@ startup-index patterns.
|
|
|
97
100
|
| Intent | Would need | Status |
|
|
98
101
|
|---|---|---|
|
|
99
102
|
| scheduled task, cron job, "run every day/hour", periodic worker | a `type: cron` service (`schedule`, `timezone`) | **SUPPORTED** on CLI 0.14.0 — Python and Node `type: cron` services validate, deploy, and fire (platform #1585 fixed). See the semantic-search `refresh` cron for a working Python-job example. |
|
|
100
|
-
| RAG chatbot, "answer over my own docs", semantic search, retrieval-augmented Q&A | `ai-app` + `needs: { vector: pgvector }` |
|
|
103
|
+
| RAG chatbot, "answer over my own docs", semantic search, retrieval-augmented Q&A | `ai-app` + `needs: { vector: pgvector }` | AVAILABLE — #1545 shipped (verified live 2026-07-16). `vector: pgvector` deploys and injects `VECTOR_PGVECTOR_URL`; the shipped template still stores embeddings in Mongo until updated. |
|
|
101
104
|
|
|
102
105
|
## Blueprints (structure + cloudgrid.yaml; adapt the app — some pending platform support)
|
|
103
106
|
|
|
@@ -129,14 +132,15 @@ runtime, async build, **local edition only**. Auth/payment secrets map through a
|
|
|
129
132
|
| appointment booking, clinic/salon booking, appointment scheduler | `appointment-booking` | `database: true` (+ auth/Stripe via `vault:`); reminder cron supported (Python/Node `type: cron`, 0.14.0) | runtime (async, poll) | local |
|
|
130
133
|
| restaurant website with reservations, table booking, restaurant site | `restaurant-reservations` | `database: true` (+ Stripe/email via `vault:`); reminder cron supported (Python/Node `type: cron`, 0.14.0) | runtime (async, poll) | local |
|
|
131
134
|
| travel booking portal, trip booking, flights/hotels booking, travel reservations | `travel-booking` | `database: true` (+ Stripe/auth via `vault:`); reminder cron supported (Python/Node `type: cron`, 0.14.0) | runtime (async, poll) | local |
|
|
132
|
-
| RAG, ask my docs, knowledge base chatbot, retrieval-augmented search over documents | `ai-knowledge-base` | `ai: true, database: true`; ideal `vector: pgvector`
|
|
135
|
+
| RAG, ask my docs, knowledge base chatbot, retrieval-augmented search over documents | `ai-knowledge-base` | `ai: true, database: true`; ideal `vector: pgvector` now AVAILABLE (#1545 shipped; the shipped blueprint still uses Mongo embeddings until updated) | runtime (async, poll) | local |
|
|
133
136
|
|
|
134
137
|
The booking family (`booking-system`, `calendar-scheduler`, `appointment-booking`,
|
|
135
138
|
`restaurant-reservations`, `travel-booking`) is fully buildable today, **including**
|
|
136
139
|
the reminder cron — Python and Node `type: cron` services work on CLI 0.14.0
|
|
137
140
|
(platform #1585 fixed). `ai-knowledge-base` builds today on the AI Gateway + Mongo; its
|
|
138
|
-
ideal `needs: { vector: pgvector }` embedding store is
|
|
139
|
-
|
|
141
|
+
ideal `needs: { vector: pgvector }` embedding store is now AVAILABLE (**#1545**
|
|
142
|
+
shipped, verified live 2026-07-16) — the shipped blueprint still stores chunks in
|
|
143
|
+
Mongo and cosine-ranks in-app until it is updated to pgvector.
|
|
140
144
|
|
|
141
145
|
## The full `needs:` vocabulary (the whole menu)
|
|
142
146
|
|
|
@@ -152,8 +156,8 @@ engine hint. Cron is NOT a need — it is a **service type** (`type: cron` with
|
|
|
152
156
|
| `kv: true` | Redis, no eviction | `KV_REDIS_URL` | Injects via `needs:` |
|
|
153
157
|
| `queue: true` | Redis durable job queue | `QUEUE_REDIS_URL` | Injects via `needs:` |
|
|
154
158
|
| `pubsub: true` | Redis pub/sub broadcast | `PUBSUB_REDIS_URL` | Injects via `needs:` |
|
|
155
|
-
| `vector: pgvector` | pgvector embeddings DB | `VECTOR_PGVECTOR_URL` (+legacy `PGVECTOR_URL`) | **Injects via `needs:`** |
|
|
156
|
-
| `object_storage: true` | GCS bucket | `OBJECT_STORAGE_GCS_BUCKET`, `OBJECT_STORAGE_GCS_REGION` |
|
|
159
|
+
| `vector: pgvector` | pgvector embeddings DB | `VECTOR_PGVECTOR_URL` (+legacy `PGVECTOR_URL`) | **Injects via `needs:`** (#1545 shipped, verified live 2026-07-16) |
|
|
160
|
+
| `object_storage: true` | GCS bucket | `OBJECT_STORAGE_GCS_BUCKET`, `OBJECT_STORAGE_GCS_REGION` | **GATED (#1678)** - rejected at plug-time; use `disk` or a BYO bucket via secret |
|
|
157
161
|
| `disk: true` | Persistent filesystem at `/data` | `DISK_PATH` | Injects via `needs:` |
|
|
158
162
|
| `ai: true` | AI Gateway access | `AI_GATEWAY_URL` | Injects via `needs:` |
|
|
159
163
|
| `type: cron` (service) | Scheduled job (`schedule`, `timezone`) | — | Service type, not a need |
|
|
@@ -162,8 +166,11 @@ engine hint. Cron is NOT a need — it is a **service type** (`type: cron` with
|
|
|
162
166
|
|
|
163
167
|
- **`needs:` injects the connection env vars today.** `needs: { database: true }`
|
|
164
168
|
→ `DATABASE_MONGODB_URL` (+legacy `MONGODB_URL`); `needs: { cache: true }` →
|
|
165
|
-
`CACHE_REDIS_URL` (+legacy `REDIS_URL`); `needs: {
|
|
166
|
-
`
|
|
169
|
+
`CACHE_REDIS_URL` (+legacy `REDIS_URL`); `needs: { queue: true }` →
|
|
170
|
+
`QUEUE_REDIS_URL`; and so on across the durable needs. Author `needs:`.
|
|
171
|
+
`vector: pgvector` → `VECTOR_PGVECTOR_URL` is now AVAILABLE (#1545 shipped,
|
|
172
|
+
verified live 2026-07-16). (**GATED, do NOT author yet:** `object_storage`
|
|
173
|
+
#1678 - rejected at plug-time, use `disk` or a BYO bucket via secret.)
|
|
167
174
|
(*Historical note: the deployer once did not inject from `needs:` — platform
|
|
168
175
|
bug #1527, now fixed and verified live.*)
|
|
169
176
|
- **`requires:` is the deprecated v1 alias.** Don't author new yaml with it.
|
|
@@ -178,7 +185,8 @@ engine hint. Cron is NOT a need — it is a **service type** (`type: cron` with
|
|
|
178
185
|
2. Adopt that template's `needs:`. Persistence → `database` (`app-with-data` or
|
|
179
186
|
`api-service`); talks to an LLM → `ai: true` + `database` (`ai-app`); scheduled
|
|
180
187
|
work → a `type: cron` service (Python/Node, supported on 0.14.0). (Held for now:
|
|
181
|
-
RAG's ideal `vector: pgvector
|
|
188
|
+
RAG's ideal `vector: pgvector` now works - #1545 shipped; the shipped
|
|
189
|
+
templates still store embeddings in Mongo until they are updated.)
|
|
182
190
|
3. One self-contained HTML page (`needs: none`, assets inlined) → publish as an
|
|
183
191
|
inspiration with `grid_deploy` and the inline `html` param (instant, any
|
|
184
192
|
edition). Separate files/folders/assets, or anything with a `needs:` →
|
|
@@ -152,8 +152,8 @@ needs:
|
|
|
152
152
|
kv: true # Redis, no eviction. Injects KV_REDIS_URL.
|
|
153
153
|
queue: true # Redis for BullMQ etc. Injects QUEUE_REDIS_URL.
|
|
154
154
|
pubsub: true # Redis pub/sub. Injects PUBSUB_REDIS_URL.
|
|
155
|
-
vector: pgvector # pgvector. Injects VECTOR_PGVECTOR_URL.
|
|
156
|
-
object_storage: true #
|
|
155
|
+
vector: pgvector # pgvector. Injects VECTOR_PGVECTOR_URL. (#1545 shipped, verified live 2026-07-16.)
|
|
156
|
+
object_storage: true # GATED (platform #1678) - rejected at plug-time; use disk: or a BYO bucket via secret.
|
|
157
157
|
disk: true # Persistent filesystem at /data. Injects DISK_PATH.
|
|
158
158
|
ai: true # AI Gateway access. Injects AI_GATEWAY_URL.
|
|
159
159
|
|
|
@@ -404,7 +404,7 @@ service type.**
|
|
|
404
404
|
| `queue` | Durable job queue substrate | `QUEUE_REDIS_URL` | Redis |
|
|
405
405
|
| `pubsub` | Broadcast pub/sub | `PUBSUB_REDIS_URL` | Redis |
|
|
406
406
|
| `vector` | Vector embeddings DB | `VECTOR_PGVECTOR_URL` (+legacy `PGVECTOR_URL`) | pgvector (Postgres) |
|
|
407
|
-
| `object_storage` | Object/blob bucket | `OBJECT_STORAGE_GCS_BUCKET`, `OBJECT_STORAGE_GCS_REGION` | GCS |
|
|
407
|
+
| `object_storage` | Object/blob bucket - **GATED (#1678)**: rejected at plug-time, use `disk` or a BYO bucket via secret | `OBJECT_STORAGE_GCS_BUCKET`, `OBJECT_STORAGE_GCS_REGION` | GCS |
|
|
408
408
|
| `disk` | Persistent filesystem mount | `DISK_PATH` | PVC |
|
|
409
409
|
| `ai` | AI Gateway access | `AI_GATEWAY_URL` | CloudGrid AI Gateway |
|
|
410
410
|
|
|
@@ -424,10 +424,12 @@ needs:
|
|
|
424
424
|
cache: true # Platform Redis
|
|
425
425
|
cache: { engine: external, secret: MY_REDIS } # Bring your own (same forms for kv/queue/pubsub)
|
|
426
426
|
|
|
427
|
+
# vector - AVAILABLE (#1545 shipped, verified live 2026-07-16):
|
|
427
428
|
vector: true # Platform default engine
|
|
428
429
|
vector: pgvector # pgvector explicitly
|
|
429
430
|
vector: { engine: pgvector, dim: 1536 } # With dimension hint
|
|
430
431
|
|
|
432
|
+
# object_storage - GATED (#1678), rejected at plug-time; use disk: or a BYO bucket:
|
|
431
433
|
object_storage: true # Default size (100MB)
|
|
432
434
|
object_storage: { size: 10Gi } # Custom size
|
|
433
435
|
|
|
@@ -32,8 +32,8 @@ services:
|
|
|
32
32
|
needs:
|
|
33
33
|
ai: true
|
|
34
34
|
database: true
|
|
35
|
-
# RAG over your own docs needs `needs: { vector: pgvector }` —
|
|
36
|
-
# (
|
|
35
|
+
# RAG over your own docs needs `needs: { vector: pgvector }` — available now,
|
|
36
|
+
# #1545 shipped (verified live 2026-07-16). This example does not use it.
|
|
37
37
|
```
|
|
38
38
|
|
|
39
39
|
## services/web/package.json
|
|
@@ -234,4 +234,5 @@ export default function Chat({ initialHistory }) {
|
|
|
234
234
|
- All app code lives under `services/web/`; the connection is read lazily inside
|
|
235
235
|
`getDb`, never at module top level.
|
|
236
236
|
- To answer over your own documents (RAG) you'd add `needs: { vector: pgvector }`
|
|
237
|
-
—
|
|
237
|
+
— available now (#1545 shipped); this example ships without it, so you'd also
|
|
238
|
+
write the retrieval code.
|
|
@@ -87,9 +87,9 @@ services/web/app/api/chat/route.js # POST: createClient().chat(...) -> persis
|
|
|
87
87
|
- Change the `messages` collection / fields; add users, sessions, titles.
|
|
88
88
|
- Restyle the chat UI in `layout.js` / `chat.js`.
|
|
89
89
|
|
|
90
|
-
## RAG / retrieval (not
|
|
90
|
+
## RAG / retrieval (not included)
|
|
91
91
|
|
|
92
92
|
To answer over **your own documents** (retrieval-augmented generation) you need a
|
|
93
|
-
vector store — `needs: { vector: pgvector }`. That
|
|
94
|
-
(
|
|
95
|
-
need
|
|
93
|
+
vector store — `needs: { vector: pgvector }`. That need is **now available** —
|
|
94
|
+
#1545 shipped (verified live 2026-07-16) — but this template is a plain chatbot:
|
|
95
|
+
declare the need and build the retrieval code on top if you want RAG.
|
|
@@ -253,9 +253,9 @@ export default function Chat({ initialHistory }) {
|
|
|
253
253
|
- Change the `messages` collection / fields; add users, sessions, titles.
|
|
254
254
|
- Run `grid dev` to test locally, `grid plug` to deploy (async — poll to live).
|
|
255
255
|
|
|
256
|
-
## RAG / retrieval (not
|
|
256
|
+
## RAG / retrieval (not included)
|
|
257
257
|
|
|
258
258
|
To answer over **your own documents** (retrieval-augmented generation) you need a
|
|
259
|
-
vector store — `needs: { vector: pgvector }`. That
|
|
260
|
-
(
|
|
261
|
-
need
|
|
259
|
+
vector store — `needs: { vector: pgvector }`. That need is **now available** —
|
|
260
|
+
#1545 shipped (verified live 2026-07-16) — but this template is a plain chatbot:
|
|
261
|
+
declare the need and build the retrieval code on top if you want RAG.
|
|
@@ -13,13 +13,13 @@ them (with citations). Both the embedding and the answer step run through
|
|
|
13
13
|
CloudGrid's **AI Gateway** (`@cloudgrid-io/ai`, injected via `needs: { ai: true }`).
|
|
14
14
|
|
|
15
15
|
> **Vector-store status (read this first):** the ideal store for embeddings is
|
|
16
|
-
> `needs: vector` (pgvector on Postgres, injecting `VECTOR_PGVECTOR_URL`),
|
|
17
|
-
> is **
|
|
18
|
-
>
|
|
19
|
-
>
|
|
20
|
-
>
|
|
21
|
-
> `
|
|
22
|
-
> Section 7 describes the drop-in swap
|
|
16
|
+
> `needs: vector` (pgvector on Postgres, injecting `VECTOR_PGVECTOR_URL`), and it
|
|
17
|
+
> is **now available** — #1545 shipped (verified live 2026-07-16). This blueprint
|
|
18
|
+
> still stores embeddings in **Mongo** (a `chunks` collection with an
|
|
19
|
+
> `embedding: number[]` field) and computes cosine similarity in the app; its
|
|
20
|
+
> `cloudgrid.yaml` declares `needs: { ai: true, database: true }` and keeps
|
|
21
|
+
> `vector: pgvector` commented. Uncomment it only if you also build the retrieval
|
|
22
|
+
> on pgvector — Section 7 describes the drop-in swap.
|
|
23
23
|
|
|
24
24
|
---
|
|
25
25
|
|
|
@@ -72,9 +72,9 @@ collections (`_id` is the Mongo ObjectId on every doc):
|
|
|
72
72
|
- `embedding` (number[] — the embedding vector; e.g. length 1536)
|
|
73
73
|
- `createdAt` (Date)
|
|
74
74
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
keeps only the text + a foreign key (Section 7).
|
|
75
|
+
As shipped, `chunks.embedding` lives in Mongo and you rank in the app
|
|
76
|
+
(Section 3). If you swap to pgvector (available — #1545 shipped), the vectors
|
|
77
|
+
move to Postgres and `chunks` keeps only the text + a foreign key (Section 7).
|
|
78
78
|
|
|
79
79
|
---
|
|
80
80
|
|
|
@@ -87,7 +87,7 @@ injected as env vars at `grid dev` (local) and at runtime (after `grid plug`):
|
|
|
87
87
|
|------|----------------------------|---------------------|--------------|
|
|
88
88
|
| MongoDB | `needs: { database: true }` | `DATABASE_MONGODB_URL` (+ legacy `MONGODB_URL`) | `lib/db.js` |
|
|
89
89
|
| AI Gateway | `needs: { ai: true }` | `AI_GATEWAY_URL` | `lib/ai.js` (via `@cloudgrid-io/ai`) |
|
|
90
|
-
| Vector store (
|
|
90
|
+
| Vector store (available — #1545 shipped) | `needs: { vector: pgvector }` | `VECTOR_PGVECTOR_URL` (+ legacy `PGVECTOR_URL`) | `lib/retrieve.js` — not used by this blueprint (Section 7 for the swap) |
|
|
91
91
|
| Auth key (optional) | `vault: { AUTH_PROVIDER_KEY: auth-provider-key }` | `AUTH_PROVIDER_KEY` | `lib/auth.js` |
|
|
92
92
|
| Stripe key (optional) | `vault: { STRIPE_KEY: stripe-live-key }` | `STRIPE_KEY` | `lib/stripe.js` |
|
|
93
93
|
|
|
@@ -149,12 +149,12 @@ export async function chat(messages) {
|
|
|
149
149
|
|
|
150
150
|
### Retrieve (`lib/retrieve.js`)
|
|
151
151
|
1. `embed(question)` -> query vector.
|
|
152
|
-
2.
|
|
153
|
-
the app, keep the top-k (e.g. k=5). Keep it a pure, testable
|
|
154
|
-
so it swaps cleanly to a pgvector query later.
|
|
152
|
+
2. As shipped (Mongo embeddings): load candidate `chunks`, compute cosine
|
|
153
|
+
similarity in the app, keep the top-k (e.g. k=5). Keep it a pure, testable
|
|
154
|
+
ranking function so it swaps cleanly to a pgvector query later.
|
|
155
155
|
|
|
156
156
|
```js
|
|
157
|
-
// lib/retrieve.js — cosine top-k over Mongo-stored embeddings (
|
|
157
|
+
// lib/retrieve.js — cosine top-k over Mongo-stored embeddings (as shipped)
|
|
158
158
|
import { getDb } from "./db.js";
|
|
159
159
|
import { embed } from "./ai.js";
|
|
160
160
|
const dot = (a, b) => a.reduce((s, x, i) => s + x * b[i], 0);
|
|
@@ -224,9 +224,9 @@ linked directory, so `init` first.
|
|
|
224
224
|
|
|
225
225
|
---
|
|
226
226
|
|
|
227
|
-
## 7.
|
|
227
|
+
## 7. The swap to pgvector (now available — #1545 shipped)
|
|
228
228
|
|
|
229
|
-
|
|
229
|
+
`needs: vector` is available; to migrate off the in-app cosine ranking:
|
|
230
230
|
|
|
231
231
|
1. Uncomment `vector: pgvector` in `cloudgrid.yaml`'s `needs:` block (dim 1536 to
|
|
232
232
|
match your embedding model). The grid then injects `VECTOR_PGVECTOR_URL` (+
|
|
@@ -6,14 +6,14 @@ and answers questions grounded in the retrieved chunks (with citations) — the
|
|
|
6
6
|
`app-with-data` runtime shape with a RAG loop layered on. Embeddings and answers
|
|
7
7
|
run through CloudGrid's AI Gateway (`@cloudgrid-io/ai`, injected via
|
|
8
8
|
`needs: { ai: true }` — works today); the ideal vector store `needs: vector`
|
|
9
|
-
(pgvector) is **
|
|
10
|
-
|
|
9
|
+
(pgvector) is **now available** — #1545 shipped (verified live 2026-07-16) — but
|
|
10
|
+
this blueprint stores embeddings in Mongo and computes similarity in-app. Because RAG involves choices
|
|
11
11
|
(chunking, embedding model, retrieval), this template ships the **structure and
|
|
12
12
|
`cloudgrid.yaml`, not finished app code** — adapt it and build.
|
|
13
13
|
|
|
14
14
|
> **This is a blueprint: structure + cloudgrid.yaml, adapt and build.** Read
|
|
15
15
|
> `AGENTS.md` for the file tree, Mongo collections, CloudGrid injection table
|
|
16
|
-
> (needs + AI Gateway + the pgvector
|
|
16
|
+
> (needs + AI Gateway + the pgvector note), the RAG loop, optional
|
|
17
17
|
> auth/payments wiring, and the deploy flow — then generate the app under
|
|
18
18
|
> `services/web/` and `grid plug` it (runtime, local edition, async — poll to a
|
|
19
19
|
> live URL).
|
|
@@ -22,7 +22,8 @@ similarity is computed in-app until it ships. Because RAG involves choices
|
|
|
22
22
|
|
|
23
23
|
- `cloudgrid.yaml` — active fields: `name`, `services.web` (nextjs, `/`), and
|
|
24
24
|
`needs: { ai: true, database: true }`. `vector: pgvector` is present but
|
|
25
|
-
commented (
|
|
25
|
+
commented (available now, #1545 shipped — this blueprint uses Mongo
|
|
26
|
+
embeddings); the optional `vault:` block (auth/payments) is
|
|
26
27
|
commented too.
|
|
27
28
|
- `AGENTS.md` — the structure guide (read this first).
|
|
28
29
|
- `index.md` — the fetch-bundle summary + cloudgrid.yaml.
|
|
@@ -89,10 +89,11 @@ needs:
|
|
|
89
89
|
# pubsub: true
|
|
90
90
|
# # vector — pgvector on Postgres. Injects VECTOR_PGVECTOR_URL (+ PGVECTOR_URL).
|
|
91
91
|
# # true | pgvector | { engine: pgvector, dim: 1536 }
|
|
92
|
-
# # NOTE: `needs: vector` (pgvector) is the intended store for RAG embeddings
|
|
93
|
-
# # is
|
|
94
|
-
# #
|
|
95
|
-
# #
|
|
92
|
+
# # NOTE: `needs: vector` (pgvector) is the intended store for RAG embeddings and
|
|
93
|
+
# # is available now (#1545 shipped). This template still stores embeddings in
|
|
94
|
+
# # Mongo (a `chunks` collection with an `embedding: number[]` field) and does
|
|
95
|
+
# # cosine similarity in the app — uncomment `vector: pgvector` only if you also
|
|
96
|
+
# # switch the retrieval code to pgvector.
|
|
96
97
|
# vector: pgvector
|
|
97
98
|
# # object_storage — GCS bucket. Injects OBJECT_STORAGE_GCS_BUCKET, _GCS_REGION.
|
|
98
99
|
# # true | { size: 10Gi }
|
|
@@ -13,7 +13,7 @@ the app following it.
|
|
|
13
13
|
|
|
14
14
|
- **Structure guide:** `grid_get_template("template", "ai-knowledge-base")` → read
|
|
15
15
|
**`AGENTS.md`** (file tree, Mongo collections, CloudGrid injection table, the
|
|
16
|
-
RAG loop, the pgvector
|
|
16
|
+
RAG loop, the pgvector note, optional auth/payments, deploy steps).
|
|
17
17
|
|
|
18
18
|
**Key rules (same runtime discipline as `app-with-data`):**
|
|
19
19
|
|
|
@@ -29,10 +29,11 @@ the app following it.
|
|
|
29
29
|
model's parametric memory alone. Keep chunking/retrieval as pure functions.
|
|
30
30
|
|
|
31
31
|
**Vector store:** `needs: vector` (pgvector, injecting `VECTOR_PGVECTOR_URL`) is
|
|
32
|
-
the intended embedding store
|
|
33
|
-
|
|
34
|
-
collection and
|
|
35
|
-
ORDER-BY-distance query
|
|
32
|
+
the intended embedding store and is **now available** — #1545 shipped (verified
|
|
33
|
+
live 2026-07-16). This blueprint still stores embeddings in a Mongo `chunks`
|
|
34
|
+
collection and ranks by cosine similarity in the app; to use pgvector instead,
|
|
35
|
+
uncomment the need and swap to a pgvector ORDER-BY-distance query (see
|
|
36
|
+
`AGENTS.md` §7).
|
|
36
37
|
|
|
37
38
|
Runtime app → **local edition** only (async build, poll to a live URL).
|
|
38
39
|
|
|
@@ -42,7 +43,8 @@ Runtime app → **local edition** only (async build, poll to a live URL).
|
|
|
42
43
|
# On disk this file is the full-annotated reference (templates/_cloudgrid.yaml.reference) with EVERY
|
|
43
44
|
# field present as a comment; only the fields below are uncommented, so it
|
|
44
45
|
# deploys to exactly these active fields. `vector: pgvector` is present but kept
|
|
45
|
-
# COMMENTED (
|
|
46
|
+
# COMMENTED (available now — #1545 shipped — but this blueprint uses Mongo
|
|
47
|
+
# embeddings; uncomment only if you also build the retrieval on pgvector).
|
|
46
48
|
name: my-knowledge-base
|
|
47
49
|
services:
|
|
48
50
|
web:
|
|
@@ -56,15 +58,16 @@ needs:
|
|
|
56
58
|
> **Capability:** `needs: { database: true }` → the deployer provisions Mongo and
|
|
57
59
|
> injects `DATABASE_MONGODB_URL` (plus legacy `MONGODB_URL`). `needs: { ai: true }`
|
|
58
60
|
> → injects `AI_GATEWAY_URL`; `@cloudgrid-io/ai` routes embeddings + chat through
|
|
59
|
-
> it with no API key to manage. `needs: { vector: pgvector }` (
|
|
60
|
-
> `VECTOR_PGVECTOR_URL`) is
|
|
61
|
+
> it with no API key to manage. `needs: { vector: pgvector }` (injects
|
|
62
|
+
> `VECTOR_PGVECTOR_URL`) is available (#1545 shipped) but not declared here —
|
|
63
|
+
> this blueprint uses Mongo embeddings. See the capability-map for the full
|
|
61
64
|
> injection table.
|
|
62
65
|
|
|
63
66
|
## Structure guide
|
|
64
67
|
|
|
65
68
|
The real content of this blueprint is **`AGENTS.md`** — fetch the template bundle
|
|
66
69
|
and read it. It covers the `services/web/` file tree, the `documents` / `chunks`
|
|
67
|
-
collections, the CloudGrid injection table (needs + AI Gateway + the
|
|
70
|
+
collections, the CloudGrid injection table (needs + AI Gateway + the pgvector note),
|
|
68
71
|
the ingest → retrieve → answer RAG loop with lazy `lib/db.js` / `lib/ai.js` /
|
|
69
72
|
`lib/retrieve.js`, optional auth/payments via `vault:`, and the
|
|
70
73
|
`grid init → fill → grid plug → poll` deploy flow.
|
|
@@ -75,7 +78,8 @@ the ingest → retrieve → answer RAG loop with lazy `lib/db.js` / `lib/ai.js`
|
|
|
75
78
|
`lib/ai.js`; tune retrieval `k`.
|
|
76
79
|
- Add auth (uncomment `vault: { AUTH_PROVIDER_KEY }`) to gate the KB, or payments
|
|
77
80
|
(`vault: { STRIPE_KEY }`) to charge for access.
|
|
78
|
-
-
|
|
79
|
-
`lib/retrieve.js` to an
|
|
81
|
+
- To move to pgvector (now available — #1545 shipped), uncomment
|
|
82
|
+
`needs: { vector: pgvector }` and swap `lib/retrieve.js` to an
|
|
83
|
+
ORDER-BY-distance query (`AGENTS.md` §7).
|
|
80
84
|
- Build under `services/web/`, then `grid dev` (local) / `grid plug` (async — poll
|
|
81
85
|
to a live URL).
|
|
@@ -77,12 +77,12 @@ os.environ.get("MONGODB_URL")` inside `get_client()`, never at module top level.
|
|
|
77
77
|
`requires:` together (the validator rejects the combination).
|
|
78
78
|
|
|
79
79
|
**No `needs: vector`.** The ideal embedding store is pgvector
|
|
80
|
-
(`needs: { vector: pgvector }` → `VECTOR_PGVECTOR_URL`),
|
|
81
|
-
|
|
82
|
-
in the Mongo `chunks.embedding` float array and
|
|
83
|
-
(`search.py::_cosine`).
|
|
84
|
-
|
|
85
|
-
|
|
80
|
+
(`needs: { vector: pgvector }` → `VECTOR_PGVECTOR_URL`), and it is **now
|
|
81
|
+
available** — #1545 shipped (verified live 2026-07-16). This template still keeps
|
|
82
|
+
embeddings in the Mongo `chunks.embedding` float array and cosine-ranks in-app
|
|
83
|
+
(`search.py::_cosine`). To swap, move the vectors to pgvector and replace the
|
|
84
|
+
in-app cosine block with an `ORDER BY embedding <=> $q LIMIT k` query — the rest
|
|
85
|
+
of the pipeline is unchanged.
|
|
86
86
|
|
|
87
87
|
---
|
|
88
88
|
|
|
@@ -155,9 +155,9 @@ to store the same way. Non-secret public config can also go via `grid env`.
|
|
|
155
155
|
source or embeddings are unconfigured. `GET /backend/status` reports capability
|
|
156
156
|
**booleans** (`source_configured`, `embeddings_configured`, …), never secret
|
|
157
157
|
values.
|
|
158
|
-
4. **No `needs: vector`.** `needs:` is `database` only — pgvector
|
|
159
|
-
|
|
160
|
-
§8.)
|
|
158
|
+
4. **No `needs: vector`.** `needs:` is `database` only — the pgvector need is
|
|
159
|
+
available (#1545 shipped), but this template's code uses Mongo embeddings (§3
|
|
160
|
+
for the swap). (The scheduled cron refresh IS active — see §8.)
|
|
161
161
|
|
|
162
162
|
---
|
|
163
163
|
|
|
@@ -17,9 +17,10 @@ aggregated to the document level with a best-passage highlight.
|
|
|
17
17
|
| `backend` | `python` (FastAPI) | `/backend` | `services/backend/src/main.py` |
|
|
18
18
|
|
|
19
19
|
`needs: { database: true }` → CloudGrid injects `DATABASE_MONGODB_URL` (fallback
|
|
20
|
-
`MONGODB_URL`). **No `needs: vector`** —
|
|
21
|
-
|
|
22
|
-
cosine-
|
|
20
|
+
`MONGODB_URL`). **No `needs: vector`** — the need is available now (#1545
|
|
21
|
+
shipped, verified live 2026-07-16), but this template stores embeddings in the
|
|
22
|
+
Mongo `chunks` collection and cosine-ranks in-app; declare `vector: pgvector`
|
|
23
|
+
only if you also switch the code to pgvector. Refresh runs two ways: the manager "Refresh now" endpoint
|
|
23
24
|
(on demand) and an active Python `type: cron` job (`services/refresh/`, daily 03:00
|
|
24
25
|
UTC) — both call the same `indexing.run_sync` (see AGENTS.md §8).
|
|
25
26
|
|
|
@@ -89,9 +89,9 @@ name: semantic-search
|
|
|
89
89
|
# pubsub: true
|
|
90
90
|
# # vector — pgvector on Postgres. Injects VECTOR_PGVECTOR_URL (+ PGVECTOR_URL).
|
|
91
91
|
# # true | pgvector | { engine: pgvector, dim: 1536 }
|
|
92
|
-
# # NOT USED HERE —
|
|
93
|
-
# #
|
|
94
|
-
# #
|
|
92
|
+
# # NOT USED HERE — available now (#1545 shipped). This template stores
|
|
93
|
+
# # embeddings in the Mongo `chunks` collection and cosine-ranks in-app.
|
|
94
|
+
# # Uncomment `vector: pgvector` only if you also switch the code to pgvector.
|
|
95
95
|
# vector: pgvector
|
|
96
96
|
# # object_storage — GCS bucket. Injects OBJECT_STORAGE_GCS_BUCKET, _GCS_REGION.
|
|
97
97
|
# # true | { size: 10Gi }
|
|
@@ -204,8 +204,9 @@ services:
|
|
|
204
204
|
timezone: UTC
|
|
205
205
|
run: job # batch job — runs services/refresh/src/main.py once
|
|
206
206
|
# CloudGrid managed Mongo → injects DATABASE_MONGODB_URL (+ legacy MONGODB_URL).
|
|
207
|
-
# NO `needs: vector` —
|
|
208
|
-
# embeddings
|
|
207
|
+
# NO `needs: vector` — available now (#1545 shipped), but this template stores
|
|
208
|
+
# embeddings in the Mongo `chunks` collection and cosine-ranks in-app;
|
|
209
|
+
# uncomment `vector: pgvector` only if you also switch the code to pgvector.
|
|
209
210
|
needs:
|
|
210
211
|
database: true
|
|
211
212
|
|
|
@@ -24,9 +24,11 @@ level with a best-passage highlight.
|
|
|
24
24
|
`$text` index exists at boot — `$text` returns `[]` on an empty catalog rather
|
|
25
25
|
than raising `IndexNotFound` (a 500). The `$text` query is also guarded in
|
|
26
26
|
`except OperationFailure` as belt-and-suspenders.
|
|
27
|
-
5. **No `needs: vector`** —
|
|
28
|
-
|
|
29
|
-
|
|
27
|
+
5. **No `needs: vector`** — the need is available now (#1545 shipped, verified
|
|
28
|
+
live 2026-07-16), but this
|
|
29
|
+
template stores embeddings in the Mongo `chunks` collection (a float array)
|
|
30
|
+
and cosine-ranks in-app; declare `vector: pgvector` only if you also switch
|
|
31
|
+
the code to pgvector. **The refresh cron IS active** — a Python `type: cron`,
|
|
30
32
|
`run: job` service (`services/refresh/`) reindexes daily at 03:00 UTC, and the
|
|
31
33
|
manager "Refresh now" endpoint reindexes on demand; both call the same
|
|
32
34
|
`indexing.run_sync` (the cron vendors the backend `app/` modules — see AGENTS.md §8).
|
|
@@ -86,7 +88,8 @@ needs:
|
|
|
86
88
|
|
|
87
89
|
> **Capability:** the need is `database: true` — the deployer provisions Mongo and
|
|
88
90
|
> injects `DATABASE_MONGODB_URL` (+ legacy `MONGODB_URL`). NO `needs: vector`
|
|
89
|
-
> (#1545
|
|
91
|
+
> (available, #1545 shipped — this template uses Mongo embeddings). Three
|
|
92
|
+
> services: `web` (static), `backend` (python), `refresh` (cron).
|
|
90
93
|
> `requires:` is the deprecated v1 alias — don't mix it with `needs:`.
|
|
91
94
|
|
|
92
95
|
## Configure + deploy
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# simple-semantic-search
|
|
2
|
+
|
|
3
|
+
Semantic search over a document in one node service, on the CloudGrid managed
|
|
4
|
+
vector substrate. `needs: { vector: pgvector }` provisions pgvector and injects
|
|
5
|
+
`VECTOR_PGVECTOR_URL`; `needs: { ai: true }` gives embeddings through
|
|
6
|
+
`@cloudgrid-io/runtime` with no API key. On startup the service embeds
|
|
7
|
+
`document.txt` (one vector per paragraph) into a `chunks` table sized from a
|
|
8
|
+
probe embedding, then `POST /search` embeds the query and returns the nearest
|
|
9
|
+
passages by cosine distance. `/health` returns 503 until the seed finishes, so
|
|
10
|
+
the platform holds the pod out of traffic until search works.
|
|
11
|
+
|
|
12
|
+
Proven by a real end-to-end deploy (live in under 3 minutes, 2026-07-16).
|
|
13
|
+
|
|
14
|
+
- Deploy: `grid init app <name> --here`, then `grid plug`.
|
|
15
|
+
- The CLI writes `.cloudgrid/link.json` (folder ↔ entity binding) — never
|
|
16
|
+
hand-author it; `rm -rf .cloudgrid` + re-init to take a copied folder as your
|
|
17
|
+
own app.
|
|
18
|
+
- Replace `services/web/src/document.txt` with the content to index.
|
|
19
|
+
- For a multi-service search portfolio (upload UI + refresh cron), use the
|
|
20
|
+
`semantic-search` blueprint instead.
|