@cloudgrid-io/mcp 0.20.6 → 0.20.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/src/corpus/capability-map.md +21 -13
- package/src/corpus/cloudgrid-yaml.md +5 -3
- package/src/corpus/examples/ai-app/index.md +4 -3
- package/src/corpus/templates/ai-app/README.md +4 -4
- package/src/corpus/templates/ai-app/index.md +4 -4
- package/src/corpus/templates/ai-knowledge-base/AGENTS.md +17 -17
- package/src/corpus/templates/ai-knowledge-base/README.md +5 -4
- package/src/corpus/templates/ai-knowledge-base/cloudgrid.yaml +5 -4
- package/src/corpus/templates/ai-knowledge-base/index.md +15 -11
- package/src/corpus/templates/semantic-search/AGENTS.md +9 -9
- package/src/corpus/templates/semantic-search/README.md +4 -3
- package/src/corpus/templates/semantic-search/cloudgrid.yaml +6 -5
- package/src/corpus/templates/semantic-search/index.md +7 -4
- package/src/corpus/templates/simple-semantic-search/README.md +20 -0
- package/src/corpus/templates/simple-semantic-search/cloudgrid.yaml +235 -0
- package/src/corpus/templates/simple-semantic-search/index.md +353 -0
- package/src/corpus/templates/simple-semantic-search/services/web/package.json +15 -0
- package/src/corpus/templates/simple-semantic-search/services/web/src/document.txt +40 -0
- package/src/corpus/templates/simple-semantic-search/services/web/src/index.js +225 -0
- package/src/corpus/workflows/ai-app.md +5 -4
- package/src/corpus/workflows/ai-knowledge-base.md +17 -14
- package/src/corpus/workflows/semantic-search.md +9 -7
- package/src/corpus/workflows/simple-semantic-search.md +64 -0
|
@@ -0,0 +1,235 @@
|
|
|
1
|
+
# ============================================================================
|
|
2
|
+
# cloudgrid.yaml â Complete Example
|
|
3
|
+
# ============================================================================
|
|
4
|
+
#
|
|
5
|
+
# This file shows EVERY field the platform supports, with comments explaining
|
|
6
|
+
# each one. It is a reference â no real project would use all of these at once.
|
|
7
|
+
#
|
|
8
|
+
# STANDARD: every CloudGrid template ships its cloudgrid.yaml in THIS full form.
|
|
9
|
+
# Uncomment ONLY the fields the archetype needs (always: name + services; plus
|
|
10
|
+
# needs: for runtime apps). Leave everything else commented. Comments are
|
|
11
|
+
# ignored by the parser, so the file deploys to exactly the active fields.
|
|
12
|
+
#
|
|
13
|
+
# Only two fields are required: name and services. Everything else is optional.
|
|
14
|
+
# Full docs: docs/cloudgrid-yaml-reference.md
|
|
15
|
+
# ============================================================================
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
# âââ IDENTITY âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
19
|
+
|
|
20
|
+
# Entity slug (REQUIRED). 2-42 chars, lowercase alphanumeric + hyphens.
|
|
21
|
+
# Must start and end with a letter or digit.
|
|
22
|
+
# Becomes part of your public URL: <name>-<4hex>.<org>.cloudgrid.io
|
|
23
|
+
name: simple-semantic-search
|
|
24
|
+
description: Semantic search over a bundled document, backed by pgvector.
|
|
25
|
+
icon: "🔎"
|
|
26
|
+
|
|
27
|
+
# One-line summary. Max 280 chars. If omitted, Grid Brain auto-generates one.
|
|
28
|
+
# description: Order processing API for the e-commerce platform
|
|
29
|
+
|
|
30
|
+
# Single emoji or short text shown as the entity icon. Max 64 chars.
|
|
31
|
+
# icon: "ð¦"
|
|
32
|
+
|
|
33
|
+
# Entity type: 'app' (default) or 'agent'. An agent is an app with an agent: block.
|
|
34
|
+
# kind: app
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
# âââ NETWORKING âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
38
|
+
|
|
39
|
+
# Whether this entity serves public HTTP traffic.
|
|
40
|
+
# true = public URL + Ingress (default). false = internal only.
|
|
41
|
+
# expose: true
|
|
42
|
+
|
|
43
|
+
# Additional domain names this entity serves on (the platform URL is always active).
|
|
44
|
+
# custom_domains:
|
|
45
|
+
# - orders.example.com
|
|
46
|
+
# - shop.example.com
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
# âââ HEALTH PROBE (entity-level override) ââââââââââââââââââââââââââââââââââ
|
|
50
|
+
|
|
51
|
+
# Override the default health check for the entire entity.
|
|
52
|
+
# probe:
|
|
53
|
+
# path: /healthz # HTTP path to probe (default: /).
|
|
54
|
+
# expected: "<500" # '<500' (default), '200', '2xx'.
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
# âââ DEPLOY NOTIFICATIONS ââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
58
|
+
|
|
59
|
+
# Slack notification to the org's default channel on deploy failure.
|
|
60
|
+
# true = notify (default), false = silent.
|
|
61
|
+
# notify: true
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
# âââ SCALING ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
65
|
+
|
|
66
|
+
# Replica scaling intent.
|
|
67
|
+
# 'auto' â platform decides from the grid tier (default).
|
|
68
|
+
# 'alert' â single-replica, emit a SPOF warning at plug time.
|
|
69
|
+
# 1-10 â explicit replica count, clamped to the grid max.
|
|
70
|
+
# scale: auto
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
# âââ INFRASTRUCTURE NEEDS ââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
74
|
+
#
|
|
75
|
+
# Declare infrastructure; the platform provisions it and injects connection
|
|
76
|
+
# details as env vars. Nine needs: database, cache, kv, queue, pubsub, vector,
|
|
77
|
+
# object_storage, disk, ai. Simplest form is `true`.
|
|
78
|
+
#
|
|
79
|
+
# needs:
|
|
80
|
+
# # database â MongoDB. Injects DATABASE_MONGODB_URL (+ legacy MONGODB_URL).
|
|
81
|
+
# # true | pool | dedicated | { tier: external, secret: MY_DB }
|
|
82
|
+
# database: true
|
|
83
|
+
# # cache â Redis, LRU eviction (derived data). Injects CACHE_REDIS_URL (+ REDIS_URL).
|
|
84
|
+
# # true | redis | { engine: external, secret: MY_REDIS_URL }
|
|
85
|
+
# cache: true
|
|
86
|
+
# # kv â Redis, no eviction (durable: flags, idempotency). Injects KV_REDIS_URL.
|
|
87
|
+
# kv: true
|
|
88
|
+
# # queue â Redis job-queue substrate (BullMQ). Injects QUEUE_REDIS_URL.
|
|
89
|
+
# queue: true
|
|
90
|
+
# # pubsub â Redis broadcast pub/sub. Injects PUBSUB_REDIS_URL.
|
|
91
|
+
# pubsub: true
|
|
92
|
+
# # vector â pgvector on Postgres. Injects VECTOR_PGVECTOR_URL (+ PGVECTOR_URL).
|
|
93
|
+
# # true | pgvector | { engine: pgvector, dim: 1536 }
|
|
94
|
+
# vector: pgvector
|
|
95
|
+
# # object_storage â GCS bucket. Injects OBJECT_STORAGE_GCS_BUCKET, _GCS_REGION.
|
|
96
|
+
# # true | { size: 10Gi }
|
|
97
|
+
# object_storage: true
|
|
98
|
+
# # disk â persistent FS mount (PVC). Injects DISK_PATH. Pins to 1 replica.
|
|
99
|
+
# # true | { size: 1Gi, mount: /var/data } (forbidden: /proc /sys /dev)
|
|
100
|
+
# disk: true
|
|
101
|
+
# # ai â AI Gateway via @cloudgrid-io/ai. Injects AI_GATEWAY_URL. true | false.
|
|
102
|
+
# ai: true
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
# âââ INTER-ENTITY COMMUNICATION ââââââââââââââââââââââââââââââââââââââââââââ
|
|
106
|
+
|
|
107
|
+
# Entities this app calls (outbound). Service-account token injected per pair.
|
|
108
|
+
# calls:
|
|
109
|
+
# - org:atomic/billing-api
|
|
110
|
+
|
|
111
|
+
# Inbound caller policy â who can call YOUR entity.
|
|
112
|
+
# callers:
|
|
113
|
+
# policy: auto # 'auto' | 'manual' (default).
|
|
114
|
+
# scope: org # 'org' (default) | 'space' | 'explicit'.
|
|
115
|
+
# fallback: deny # 'deny' (default) | 'warn'.
|
|
116
|
+
# # allowed_apps: # required only when scope: explicit
|
|
117
|
+
# # - org:atomic/frontend
|
|
118
|
+
|
|
119
|
+
|
|
120
|
+
# âââ VAULT ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
121
|
+
|
|
122
|
+
# Map org vault items to env vars. Keys UPPER_SNAKE_CASE; values = vault item keys.
|
|
123
|
+
# vault:
|
|
124
|
+
# STRIPE_KEY: stripe-live-key
|
|
125
|
+
# SENDGRID_API_KEY: sendgrid-key
|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
# âââ CONNECTORS AND HOOKS ââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
129
|
+
|
|
130
|
+
# connectors:
|
|
131
|
+
# - id: team-slack
|
|
132
|
+
# target: org # 'system' | 'org'.
|
|
133
|
+
# channel: "#deploys"
|
|
134
|
+
# hooks:
|
|
135
|
+
# - on: deploy.failed # release.published | deploy.failed | feedback.received | entity.created
|
|
136
|
+
# via: team-slack
|
|
137
|
+
# template: "Deploy failed for {entity_name}: {error}"
|
|
138
|
+
# channel: "#alerts"
|
|
139
|
+
# triggers:
|
|
140
|
+
# - from: team-slack
|
|
141
|
+
# event: message
|
|
142
|
+
# channel: "#support"
|
|
143
|
+
# route: /webhooks/slack
|
|
144
|
+
|
|
145
|
+
|
|
146
|
+
# âââ LOCAL DEVELOPMENT ââââââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
147
|
+
|
|
148
|
+
# Config for `grid dev`. Redact fields in dev data snapshots.
|
|
149
|
+
# dev:
|
|
150
|
+
# snapshot:
|
|
151
|
+
# mask:
|
|
152
|
+
# - users.email
|
|
153
|
+
# - users.phone
|
|
154
|
+
|
|
155
|
+
|
|
156
|
+
# âââ AGENT METADATA ââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
157
|
+
|
|
158
|
+
# An agent is an app with kind: agent (above) and an agent: block.
|
|
159
|
+
# agent:
|
|
160
|
+
# purpose: Sync customer data from external CRM every night
|
|
161
|
+
# schedule: "0 2 * * *"
|
|
162
|
+
# trigger: webhook
|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
# âââ PRIVATE RESOURCE AUTO-DEPENDENCIES ââââââââââââââââââââââââââââââââââââ
|
|
166
|
+
|
|
167
|
+
# When true (default), private resources are auto-injected as depends_on.
|
|
168
|
+
# auto_depends_on_private_requires: true
|
|
169
|
+
|
|
170
|
+
|
|
171
|
+
# âââ SERVICES (REQUIRED) âââââââââââââââââââââââââââââââââââââââââââââââââââ
|
|
172
|
+
#
|
|
173
|
+
# At least one service is required. Key = service name (slug rules).
|
|
174
|
+
# Types: node, nextjs, python, static, cron.
|
|
175
|
+
#
|
|
176
|
+
services:
|
|
177
|
+
web:
|
|
178
|
+
# type (REQUIRED): node | nextjs | python | static | cron
|
|
179
|
+
type: node
|
|
180
|
+
# HTTP route: '/' | '/api' | false (internal) | omitted (=false + warning).
|
|
181
|
+
# Only one service per entity can claim '/'.
|
|
182
|
+
path: /
|
|
183
|
+
# Container port (default 8080; read process.env.PORT). Not for cron.
|
|
184
|
+
# port: 3000
|
|
185
|
+
# nextjs only: base_image: distroless (default) | alpine
|
|
186
|
+
# base_image: distroless
|
|
187
|
+
# node only: lang: javascript (default) | typescript
|
|
188
|
+
# lang: typescript
|
|
189
|
+
# Custom source location (default services/<name>/). '.' = project root.
|
|
190
|
+
# source:
|
|
191
|
+
# path: src/worker
|
|
192
|
+
# Health probe: false/omitted (TCP) | true (/health) | "/path" | {path,expected}
|
|
193
|
+
# Returns 503 until the startup seed finishes, so traffic waits for a ready index.
|
|
194
|
+
health: /health
|
|
195
|
+
# Static runtime env vars (strings). Reserved: PORT, APP_NAME, SERVICE_NAME,
|
|
196
|
+
# NODE_ENV, MONGODB_URL, REDIS_URL, AI_GATEWAY_URL, N8N_WEBHOOK_URL, CLOUDGRID_*.
|
|
197
|
+
# env:
|
|
198
|
+
# LOG_LEVEL: info
|
|
199
|
+
# Sibling services/private resources that must be Ready first (no cron, no cycles).
|
|
200
|
+
# depends_on:
|
|
201
|
+
# - worker
|
|
202
|
+
# Per-service needs override (merges with app-level needs).
|
|
203
|
+
# needs:
|
|
204
|
+
# queue: true
|
|
205
|
+
# Static-only build step:
|
|
206
|
+
# build:
|
|
207
|
+
# command: npm run build
|
|
208
|
+
# output: dist
|
|
209
|
+
# env:
|
|
210
|
+
# VITE_API_URL: https://api.example.com
|
|
211
|
+
# node_version: "22" # static build step: '18'|'20'|'22'|'24' (quoted)
|
|
212
|
+
# Cron-only: schedule (REQUIRED), timezone (UTC|EST|PST), run (job | https URL)
|
|
213
|
+
# schedule: "0 9 * * 1"
|
|
214
|
+
# timezone: UTC
|
|
215
|
+
# run: job
|
|
216
|
+
# Verbose per-service persistence (prefer needs.disk):
|
|
217
|
+
# persist:
|
|
218
|
+
# - name: scratch
|
|
219
|
+
# path: /tmp/scratch
|
|
220
|
+
# size: 500Mi
|
|
221
|
+
|
|
222
|
+
|
|
223
|
+
|
|
224
|
+
needs:
|
|
225
|
+
vector: pgvector # pgvector on Postgres. Injects VECTOR_PGVECTOR_URL (+legacy PGVECTOR_URL).
|
|
226
|
+
ai: true # AI Gateway for embeddings via @cloudgrid-io/runtime. Injects AI_GATEWAY_URL.
|
|
227
|
+
|
|
228
|
+
|
|
229
|
+
# ============================================================================
|
|
230
|
+
# NOTES
|
|
231
|
+
# - NOT here: org:/entity_id: (â .cloudgrid/link.json), secrets (â grid secrets set),
|
|
232
|
+
# Dockerfile (platform generates from services.*.type).
|
|
233
|
+
# - Deploy: grid plug | Local with grid resources: grid dev
|
|
234
|
+
# - Full reference: docs/cloudgrid-yaml-reference.md | CLI: docs/cli-reference.md
|
|
235
|
+
# ============================================================================
|
|
@@ -0,0 +1,353 @@
|
|
|
1
|
+
# Template: simple-semantic-search (single-service pgvector search)
|
|
2
|
+
|
|
3
|
+
Semantic search over a document in ONE node service — the smallest real vector
|
|
4
|
+
app on the grid. The platform provisions pgvector from `needs: { vector:
|
|
5
|
+
pgvector }` and injects `VECTOR_PGVECTOR_URL`; embeddings come from the AI
|
|
6
|
+
Gateway via `@cloudgrid-io/runtime` (`needs: { ai: true }`, no API key). Proven
|
|
7
|
+
by a real end-to-end deploy (live in under 3 minutes, 2026-07-16).
|
|
8
|
+
|
|
9
|
+
For a multi-service search portfolio (upload UI, separate refresh cron), use the
|
|
10
|
+
heavier `semantic-search` blueprint instead. This template is the default for
|
|
11
|
+
"search my document by meaning".
|
|
12
|
+
|
|
13
|
+
**Key rules (all proven by the live deploy):**
|
|
14
|
+
|
|
15
|
+
1. **Size the vector column from a probe, never hardcode the dimension.** Embed
|
|
16
|
+
one chunk first (`runtime.ai.embeddings`), read `probe.dim`, and create the
|
|
17
|
+
table with `vector(${dim})`. Hardcoding 1536 breaks when the platform model
|
|
18
|
+
differs.
|
|
19
|
+
2. **Seed in the background; report readiness via `/health`.** Bind the HTTP
|
|
20
|
+
port immediately (the platform startup probe wants it open), run the
|
|
21
|
+
embed+insert seed async, and return 503 from `/health` until it finishes —
|
|
22
|
+
the readiness probe then holds the pod out of traffic until search works.
|
|
23
|
+
The seed is idempotent (`CREATE ... IF NOT EXISTS`, count check, upsert).
|
|
24
|
+
3. **Read `VECTOR_PGVECTOR_URL` lazily inside the connect function** (with the
|
|
25
|
+
legacy `PGVECTOR_URL` fallback). Never at module top level, never hardcoded.
|
|
26
|
+
4. **Query with the pgvector cosine operator.** `ORDER BY embedding <=> $1::vector`
|
|
27
|
+
with score `1 - (embedding <=> $1::vector)`. Pass vectors as literals:
|
|
28
|
+
`'[0.1,0.2,...]'` + `::vector` cast.
|
|
29
|
+
5. **Service code lives under `services/web/`**, not the project root.
|
|
30
|
+
|
|
31
|
+
## File tree
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
cloudgrid.yaml # name + needs: { vector: pgvector, ai: true } + services.web (node, health: /health)
|
|
35
|
+
services/web/package.json # @cloudgrid-io/runtime, express, pg — nothing else
|
|
36
|
+
services/web/src/index.js # seed + /search + minimal UI (single file)
|
|
37
|
+
services/web/src/document.txt # the content to index — replace with the user's document
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## What the CLI writes for you: `.cloudgrid/link.json`
|
|
41
|
+
|
|
42
|
+
After `grid init app <name> --here` (or the first `grid plug`), the CLI binds
|
|
43
|
+
the folder to its grid entity by writing `.cloudgrid/link.json`. It looks like:
|
|
44
|
+
|
|
45
|
+
```json
|
|
46
|
+
{
|
|
47
|
+
"entity_id": "923b2d46-ce6e-42e5-9976-311f8dab5755",
|
|
48
|
+
"org_id": "602fbff7-920b-4221-afb4-7440915b6bab",
|
|
49
|
+
"org_slug": "atomic",
|
|
50
|
+
"kind": "app",
|
|
51
|
+
"entity_slug": "simple-semantic-search-86f2",
|
|
52
|
+
"created_at": "2026-07-16T13:28:32.744Z",
|
|
53
|
+
"cli_version": "0.15.4"
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
**Never hand-author or copy this file** — the CLI writes it, and every value is
|
|
58
|
+
minted at init time (the slug gets a 4-hex suffix). Two practical consequences:
|
|
59
|
+
|
|
60
|
+
- A folder **with** `.cloudgrid/link.json` is bound: `grid plug` UPDATES that
|
|
61
|
+
entity. To fork someone else's project into your own app, `rm -rf .cloudgrid`
|
|
62
|
+
(and remove any `name:` you don't own from `cloudgrid.yaml`), then
|
|
63
|
+
`grid init app <your-name> --here` for a fresh identity.
|
|
64
|
+
- `link.json` is why identity does NOT belong in `cloudgrid.yaml` — the
|
|
65
|
+
manifest describes the app; the link binds the folder.
|
|
66
|
+
|
|
67
|
+
## Deploy
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
grid init app <name> --here # once, registers the entity + writes .cloudgrid/link.json
|
|
71
|
+
grid plug # build + deploy, async — live URL in ~2-3 min
|
|
72
|
+
grid visibility set <slug> link # default is grid-only (403 outside the org)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Then open the URL: a search box over the document, ranked by embedding
|
|
76
|
+
similarity. Replace `services/web/src/document.txt` with the user's content and
|
|
77
|
+
re-run `grid plug` to re-seed (delete rows or bump the table if the document
|
|
78
|
+
shrinks — the seed only tops up).
|
|
79
|
+
|
|
80
|
+
## cloudgrid.yaml (active fields)
|
|
81
|
+
|
|
82
|
+
```yaml
|
|
83
|
+
name: simple-semantic-search
|
|
84
|
+
description: Semantic search over a bundled document, backed by pgvector.
|
|
85
|
+
icon: "🔎"
|
|
86
|
+
|
|
87
|
+
needs:
|
|
88
|
+
vector: pgvector # injects VECTOR_PGVECTOR_URL (+legacy PGVECTOR_URL)
|
|
89
|
+
ai: true # embeddings via @cloudgrid-io/runtime, no API key
|
|
90
|
+
|
|
91
|
+
services:
|
|
92
|
+
web:
|
|
93
|
+
type: node
|
|
94
|
+
path: /
|
|
95
|
+
health: /health # 503 until the seed finishes
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## services/web/package.json
|
|
99
|
+
|
|
100
|
+
```json
|
|
101
|
+
{
|
|
102
|
+
"name": "simple-semantic-search-web",
|
|
103
|
+
"version": "1.0.0",
|
|
104
|
+
"private": true,
|
|
105
|
+
"description": "Semantic search over a bundled document on the CloudGrid vector substrate (pgvector).",
|
|
106
|
+
"main": "src/index.js",
|
|
107
|
+
"scripts": {
|
|
108
|
+
"start": "node src/index.js"
|
|
109
|
+
},
|
|
110
|
+
"dependencies": {
|
|
111
|
+
"@cloudgrid-io/runtime": "^1.0.3",
|
|
112
|
+
"express": "^4.21.2",
|
|
113
|
+
"pg": "^8.13.0"
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
## services/web/src/index.js
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
'use strict';
|
|
122
|
+
|
|
123
|
+
// Simple semantic search — single-service reference for the CloudGrid managed vector
|
|
124
|
+
// substrate (pgvector) + the platform embeddings endpoint.
|
|
125
|
+
//
|
|
126
|
+
// On startup it embeds the bundled document (one vector per passage)
|
|
127
|
+
// via runtime.ai.embeddings and stores the vectors in the entity's own pgvector
|
|
128
|
+
// schema. POST /search embeds the query and returns the nearest passages by
|
|
129
|
+
// cosine distance. No keyword matching — relevance comes entirely from the
|
|
130
|
+
// embeddings.
|
|
131
|
+
|
|
132
|
+
const fs = require('fs');
|
|
133
|
+
const path = require('path');
|
|
134
|
+
const express = require('express');
|
|
135
|
+
const { Client } = require('pg');
|
|
136
|
+
const { runtime } = require('@cloudgrid-io/runtime');
|
|
137
|
+
|
|
138
|
+
const PORT = process.env.PORT || 8080;
|
|
139
|
+
const PGVECTOR_URL = process.env.VECTOR_PGVECTOR_URL || process.env.PGVECTOR_URL;
|
|
140
|
+
|
|
141
|
+
// One passage per non-empty paragraph in the article.
|
|
142
|
+
function loadChunks() {
|
|
143
|
+
const raw = fs.readFileSync(path.join(__dirname, 'document.txt'), 'utf8');
|
|
144
|
+
return raw
|
|
145
|
+
.split(/\n\s*\n/)
|
|
146
|
+
.map((p) => p.replace(/\s+/g, ' ').trim())
|
|
147
|
+
.filter((p) => p.length > 0);
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
// Format a JS number[] as a pgvector literal: '[0.1,0.2,...]'.
|
|
151
|
+
function toVectorLiteral(values) {
|
|
152
|
+
return '[' + values.join(',') + ']';
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
let seeded = false;
|
|
156
|
+
let seedError = null;
|
|
157
|
+
let chunkCount = 0;
|
|
158
|
+
let embeddingDim = 0;
|
|
159
|
+
|
|
160
|
+
async function connect() {
|
|
161
|
+
if (!PGVECTOR_URL) {
|
|
162
|
+
throw new Error('VECTOR_PGVECTOR_URL is not set — the vector need did not provision');
|
|
163
|
+
}
|
|
164
|
+
const client = new Client({ connectionString: PGVECTOR_URL });
|
|
165
|
+
await client.connect();
|
|
166
|
+
return client;
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
// Idempotent seed: ensure the extension + table exist, sized to the embedding
|
|
170
|
+
// model's actual output length, and populate the table once.
|
|
171
|
+
async function seed() {
|
|
172
|
+
const chunks = loadChunks();
|
|
173
|
+
const client = await connect();
|
|
174
|
+
try {
|
|
175
|
+
await client.query('CREATE EXTENSION IF NOT EXISTS vector');
|
|
176
|
+
|
|
177
|
+
// Size the column to the model's real output length — embed a probe first.
|
|
178
|
+
const probe = await runtime.ai.embeddings({ text: chunks[0] });
|
|
179
|
+
embeddingDim = probe.dim;
|
|
180
|
+
console.log(`[seed] embedding model=${probe.model} dim=${embeddingDim}`);
|
|
181
|
+
|
|
182
|
+
await client.query(
|
|
183
|
+
`CREATE TABLE IF NOT EXISTS chunks (
|
|
184
|
+
id integer PRIMARY KEY,
|
|
185
|
+
chunk text NOT NULL,
|
|
186
|
+
embedding vector(${embeddingDim}) NOT NULL
|
|
187
|
+
)`,
|
|
188
|
+
);
|
|
189
|
+
|
|
190
|
+
const { rows } = await client.query('SELECT count(*)::int AS n FROM chunks');
|
|
191
|
+
const existing = rows[0].n;
|
|
192
|
+
if (existing >= chunks.length) {
|
|
193
|
+
chunkCount = existing;
|
|
194
|
+
console.log(`[seed] already populated with ${existing} chunks — skipping embed`);
|
|
195
|
+
seeded = true;
|
|
196
|
+
return;
|
|
197
|
+
}
|
|
198
|
+
|
|
199
|
+
console.log(`[seed] embedding ${chunks.length} chunks`);
|
|
200
|
+
// Batch-embed every passage in input order.
|
|
201
|
+
const vectors = await runtime.ai.embeddings({ text: chunks });
|
|
202
|
+
for (let i = 0; i < chunks.length; i++) {
|
|
203
|
+
const literal = toVectorLiteral(vectors[i].values);
|
|
204
|
+
await client.query(
|
|
205
|
+
`INSERT INTO chunks (id, chunk, embedding)
|
|
206
|
+
VALUES ($1, $2, $3::vector)
|
|
207
|
+
ON CONFLICT (id) DO UPDATE SET chunk = EXCLUDED.chunk, embedding = EXCLUDED.embedding`,
|
|
208
|
+
[i, chunks[i], literal],
|
|
209
|
+
);
|
|
210
|
+
console.log(`[seed] inserted chunk ${i + 1}/${chunks.length}`);
|
|
211
|
+
}
|
|
212
|
+
chunkCount = chunks.length;
|
|
213
|
+
seeded = true;
|
|
214
|
+
console.log(`[seed] done — ${chunkCount} chunks stored`);
|
|
215
|
+
} finally {
|
|
216
|
+
await client.end().catch(() => {});
|
|
217
|
+
}
|
|
218
|
+
}
|
|
219
|
+
|
|
220
|
+
async function search(query, limit = 5) {
|
|
221
|
+
const q = await runtime.ai.embeddings({ text: query });
|
|
222
|
+
const literal = toVectorLiteral(q.values);
|
|
223
|
+
const client = await connect();
|
|
224
|
+
try {
|
|
225
|
+
const { rows } = await client.query(
|
|
226
|
+
`SELECT chunk, 1 - (embedding <=> $1::vector) AS score
|
|
227
|
+
FROM chunks
|
|
228
|
+
ORDER BY embedding <=> $1::vector
|
|
229
|
+
LIMIT $2`,
|
|
230
|
+
[literal, limit],
|
|
231
|
+
);
|
|
232
|
+
return rows.map((r) => ({ chunk: r.chunk, score: Number(r.score) }));
|
|
233
|
+
} finally {
|
|
234
|
+
await client.end().catch(() => {});
|
|
235
|
+
}
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
const app = express();
|
|
239
|
+
app.use(express.json());
|
|
240
|
+
|
|
241
|
+
app.get('/health', (_req, res) => {
|
|
242
|
+
// Return not-ready until the seed completes so the platform's readiness probe
|
|
243
|
+
// holds the pod out of traffic until search actually works.
|
|
244
|
+
if (seedError) return res.status(500).json({ status: 'seed_failed', error: seedError });
|
|
245
|
+
if (!seeded) return res.status(503).json({ status: 'seeding' });
|
|
246
|
+
res.json({ status: 'ok', chunks: chunkCount, dim: embeddingDim });
|
|
247
|
+
});
|
|
248
|
+
|
|
249
|
+
app.post('/search', async (req, res) => {
|
|
250
|
+
const query = req.body && typeof req.body.query === 'string' ? req.body.query.trim() : '';
|
|
251
|
+
if (!query) return res.status(400).json({ error: 'Provide a query string.' });
|
|
252
|
+
if (!seeded) return res.status(503).json({ error: 'The index is still loading. Try again shortly.' });
|
|
253
|
+
try {
|
|
254
|
+
const results = await search(query);
|
|
255
|
+
res.json({ query, results });
|
|
256
|
+
} catch (err) {
|
|
257
|
+
console.error('[search] failed', err);
|
|
258
|
+
res.status(500).json({ error: 'Search failed.' });
|
|
259
|
+
}
|
|
260
|
+
});
|
|
261
|
+
|
|
262
|
+
app.get('/', (_req, res) => {
|
|
263
|
+
res.type('html').send(PAGE);
|
|
264
|
+
});
|
|
265
|
+
|
|
266
|
+
const PAGE = `<!doctype html>
|
|
267
|
+
<html lang="en">
|
|
268
|
+
<head>
|
|
269
|
+
<meta charset="utf-8" />
|
|
270
|
+
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
271
|
+
<title>Semantic search</title>
|
|
272
|
+
<style>
|
|
273
|
+
:root { color-scheme: light; }
|
|
274
|
+
* { box-sizing: border-box; }
|
|
275
|
+
body { margin: 0; font: 16px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; color: #1a1a1a; background: #fafafa; }
|
|
276
|
+
main { max-width: 680px; margin: 0 auto; padding: 48px 20px 80px; }
|
|
277
|
+
h1 { font-size: 24px; margin: 0 0 4px; }
|
|
278
|
+
p.lede { color: #555; margin: 0 0 28px; }
|
|
279
|
+
form { display: flex; gap: 8px; margin-bottom: 24px; }
|
|
280
|
+
input { flex: 1; padding: 12px 14px; font-size: 16px; border: 1px solid #ccc; border-radius: 8px; background: #fff; }
|
|
281
|
+
input:focus { outline: none; border-color: #888; }
|
|
282
|
+
button { padding: 12px 20px; font-size: 16px; border: none; border-radius: 8px; background: #1a1a1a; color: #fff; cursor: pointer; }
|
|
283
|
+
button:disabled { opacity: .5; cursor: default; }
|
|
284
|
+
.result { padding: 16px 0; border-top: 1px solid #eee; }
|
|
285
|
+
.result .score { font-size: 13px; color: #888; margin-bottom: 4px; font-variant-numeric: tabular-nums; }
|
|
286
|
+
.result .text { margin: 0; }
|
|
287
|
+
.status { color: #888; font-size: 14px; }
|
|
288
|
+
</style>
|
|
289
|
+
</head>
|
|
290
|
+
<body>
|
|
291
|
+
<main>
|
|
292
|
+
<h1>Semantic search</h1>
|
|
293
|
+
<p class="lede">Search the document by meaning. Results are ranked by embedding similarity, not keyword match.</p>
|
|
294
|
+
<form id="f">
|
|
295
|
+
<input id="q" type="text" placeholder="Ask a question about the document" autocomplete="off" />
|
|
296
|
+
<button id="b" type="submit">Search</button>
|
|
297
|
+
</form>
|
|
298
|
+
<div id="out"></div>
|
|
299
|
+
</main>
|
|
300
|
+
<script>
|
|
301
|
+
var f = document.getElementById('f');
|
|
302
|
+
var q = document.getElementById('q');
|
|
303
|
+
var b = document.getElementById('b');
|
|
304
|
+
var out = document.getElementById('out');
|
|
305
|
+
f.addEventListener('submit', function (e) {
|
|
306
|
+
e.preventDefault();
|
|
307
|
+
var query = q.value.trim();
|
|
308
|
+
if (!query) return;
|
|
309
|
+
b.disabled = true;
|
|
310
|
+
out.innerHTML = '<p class="status">Searching.</p>';
|
|
311
|
+
fetch('/search', {
|
|
312
|
+
method: 'POST',
|
|
313
|
+
headers: { 'Content-Type': 'application/json' },
|
|
314
|
+
body: JSON.stringify({ query: query }),
|
|
315
|
+
})
|
|
316
|
+
.then(function (r) { return r.json(); })
|
|
317
|
+
.then(function (data) {
|
|
318
|
+
b.disabled = false;
|
|
319
|
+
if (!data.results || !data.results.length) {
|
|
320
|
+
out.innerHTML = '<p class="status">No results.</p>';
|
|
321
|
+
return;
|
|
322
|
+
}
|
|
323
|
+
out.innerHTML = data.results.map(function (r) {
|
|
324
|
+
return '<div class="result"><div class="score">score ' + r.score.toFixed(3) +
|
|
325
|
+
'</div><p class="text">' + r.chunk.replace(/</g, '<') + '</p></div>';
|
|
326
|
+
}).join('');
|
|
327
|
+
})
|
|
328
|
+
.catch(function () {
|
|
329
|
+
b.disabled = false;
|
|
330
|
+
out.innerHTML = '<p class="status">Search failed. Try again.</p>';
|
|
331
|
+
});
|
|
332
|
+
});
|
|
333
|
+
</script>
|
|
334
|
+
</body>
|
|
335
|
+
</html>`;
|
|
336
|
+
|
|
337
|
+
app.listen(PORT, () => {
|
|
338
|
+
console.log(`simple-semantic-search listening on ${PORT}`);
|
|
339
|
+
// Seed in the background so the HTTP server binds immediately (the platform's
|
|
340
|
+
// startup probe wants the port open). /health reports seeding progress.
|
|
341
|
+
seed().catch((err) => {
|
|
342
|
+
seedError = String((err && err.message) || err);
|
|
343
|
+
console.error('[seed] FAILED', err);
|
|
344
|
+
});
|
|
345
|
+
});
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
## services/web/src/document.txt
|
|
349
|
+
|
|
350
|
+
Any plain-text document, one passage per blank-line-separated paragraph. The
|
|
351
|
+
template ships a short factual solar-system article so the demo answers
|
|
352
|
+
questions like "which planet is the hottest?" — replace it with the user's
|
|
353
|
+
content.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "simple-semantic-search-web",
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"private": true,
|
|
5
|
+
"description": "Semantic search over a bundled document on the CloudGrid vector substrate (pgvector).",
|
|
6
|
+
"main": "src/index.js",
|
|
7
|
+
"scripts": {
|
|
8
|
+
"start": "node src/index.js"
|
|
9
|
+
},
|
|
10
|
+
"dependencies": {
|
|
11
|
+
"@cloudgrid-io/runtime": "^1.0.3",
|
|
12
|
+
"express": "^4.21.2",
|
|
13
|
+
"pg": "^8.13.0"
|
|
14
|
+
}
|
|
15
|
+
}
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
The solar system formed about 4.6 billion years ago from the gravitational
|
|
2
|
+
collapse of a giant molecular cloud. Most of the collapsing mass gathered at
|
|
3
|
+
the center to form the Sun, while the rest flattened into a disk from which
|
|
4
|
+
the planets, moons, and asteroids emerged.
|
|
5
|
+
|
|
6
|
+
Mercury is the smallest planet and the closest to the Sun. Despite that, it is
|
|
7
|
+
not the hottest: it has almost no atmosphere to trap heat, so its night side
|
|
8
|
+
plunges to minus 180 degrees Celsius.
|
|
9
|
+
|
|
10
|
+
Venus is the hottest planet. Its thick carbon-dioxide atmosphere creates a
|
|
11
|
+
runaway greenhouse effect that keeps the surface near 465 degrees Celsius, hot
|
|
12
|
+
enough to melt lead, day and night alike.
|
|
13
|
+
|
|
14
|
+
Earth is the only planet known to support life. Liquid water covers about 71
|
|
15
|
+
percent of its surface, and its magnetic field shields the atmosphere from the
|
|
16
|
+
solar wind.
|
|
17
|
+
|
|
18
|
+
Mars is a cold desert world with the largest volcano in the solar system,
|
|
19
|
+
Olympus Mons, which stands about three times the height of Mount Everest.
|
|
20
|
+
Evidence suggests liquid water flowed on Mars billions of years ago.
|
|
21
|
+
|
|
22
|
+
Jupiter is the largest planet, with more than twice the mass of all the other
|
|
23
|
+
planets combined. Its Great Red Spot is a storm larger than Earth that has
|
|
24
|
+
raged for at least 300 years.
|
|
25
|
+
|
|
26
|
+
Saturn is famous for its rings, which are made mostly of water ice. The rings
|
|
27
|
+
span up to 282,000 kilometers across but are on average only about 10 meters
|
|
28
|
+
thick.
|
|
29
|
+
|
|
30
|
+
Uranus rotates on its side, with an axial tilt of about 98 degrees, probably
|
|
31
|
+
the result of a giant collision early in its history. Its seasons each last
|
|
32
|
+
about 21 Earth years.
|
|
33
|
+
|
|
34
|
+
Neptune is the windiest planet, with gusts reaching 2,100 kilometers per hour.
|
|
35
|
+
It was the first planet located through mathematical prediction rather than
|
|
36
|
+
direct observation.
|
|
37
|
+
|
|
38
|
+
Pluto, reclassified as a dwarf planet in 2006, orbits in the Kuiper Belt. It
|
|
39
|
+
has five known moons, and its largest, Charon, is so big that the two bodies
|
|
40
|
+
orbit a point in space between them.
|