@cloudgrid-io/mcp 0.14.0 → 0.15.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cloudgrid-io/mcp",
-  "version": "0.14.0",
+  "version": "0.15.0",
   "description": "MCP server for CloudGrid. Three editions: a local stdio server (full toolset), a hosted web server (light, CLI-free toolset), and a public docs server (search_cloudgrid_documentation) over MCP Streamable HTTP.",
   "type": "module",
   "bin": {

package/src/corpus/capability-map.md

@@ -13,7 +13,7 @@ cloudgrid.yaml schema (every field, the `needs:` injection table, the
 requires-vs-needs caveat, validation rules), fetch the companion reference:
 `gridctl_fetch("doc", "cloudgrid-yaml")`.
 
-## The 58 templates
+## The 59 templates
 
 | Intent (match on `when:`) | Template | `needs:` | Deploy | Edition |
 |---|---|---|---|---|
@@ -57,6 +57,7 @@ requires-vs-needs caveat, validation rules), fetch the companion reference:
 | revenue dashboard, sales dashboard, MRR/revenue view | `revenue-dashboard` | `database: true` | runtime (async, poll) | local |
 | property listings, real estate site, rentals/homes listings | `property-listings` | `database: true` | runtime (async, poll) | local |
 | project management, projects and tasks tracker, team project board | `project-management` | `database: true` | runtime (async, poll) | local |
+| search over my documents / PDFs / notes / knowledge base, semantic search, document search, find across my files, searchable archive | `semantic-search` | `database: true` (NO `vector` #1545; NO cron #1585) | runtime (async, poll) | local |
 
 **Rule of thumb:** if the app must SAVE/remember data, share state across
 users/sessions, log in, or store submissions → it is persistent → runtime, local
@@ -73,6 +74,18 @@ differ only by domain schema + UI. Match the request to the closest one by its
 domain, use `app-with-data` (the minimal reference). Deploy path is identical for
 all of them.
 
+**Document search (`semantic-search`).** For "search over my own documents / PDFs
+/ notes / knowledge base" with keyword **and** meaning, `semantic-search` ships
+real runnable code in a different shape from the Next.js family: a React (Vite)
+**static** frontend at `/` plus a **Python FastAPI** backend at `/backend`, on the
+same grid-shared Mongo (`needs: { database: true }`). Hybrid search = Mongo
+`$text` + in-app NumPy cosine over embedding arrays + metadata filters. It stores
+embeddings in the Mongo `chunks` collection (NO `needs: vector` — pgvector is
+blocked on **#1545**) and refreshes via a manager endpoint (NO active cron — a
+Python `type: cron` is blocked on **#1585**). Read its `AGENTS.md` for the source
+adapter (dropbox/local/url) + embeddings wiring and the health-without-secrets /
+startup-index patterns.
+
 ## Held / pending platform (not yet buildable)
 
 Two more archetypes are designed but blocked on platform work — the LLM should

package/src/corpus/templates/semantic-search/AGENTS.md

@@ -0,0 +1,192 @@
+# AGENTS.md — semantic-search template (CloudGrid wiring guide)
+
+This template is a **runnable app**, not a blueprint — it ships real FastAPI +
+React code. Read this guide, adapt the naming/metadata to the user's domain, wire
+the secrets, and deploy. The four "baked-in learnings" (health-without-secrets,
+startup index, `$text` guard, no vector/no cron) are already in the code — do not
+regress them.
+
+---
+
+## 1. File tree
+
+Every service's code lives under `services/<name>/` — `path:` in `cloudgrid.yaml`
+is the URL mount, NOT the filesystem path. Files at the repo root fail with
+`Error: Service directory not found: …/services/<name>`.
+
+```
+cloudgrid.yaml                          # name + services(web,backend) + needs:{database:true}
+services/web/                           # React + Vite static frontend, served at /
+  package.json  vite.config.js  index.html
+  src/main.jsx  App.jsx  api.js  styles.css
+  src/components/SearchView.jsx ManagerLogin.jsx Settings.jsx Dashboard.jsx
+services/backend/                       # FastAPI, mounted at /backend
+  requirements.txt
+  src/main.py                           # uvicorn app.main:app (reads PORT)
+  src/app/
+    main.py       config.py   db.py      search.py   indexing.py
+    source.py     embeddings.py  answer.py  auth.py
+```
+
+---
+
+## 2. Mongo collections + fields
+
+The grid provisions one MongoDB database (`needs: { database: true }`). Two
+primary collections (`_id` is the Mongo ObjectId on every doc):
+
+**`documents`** — one row per source document
+- `title` (string), `collection` (string, free-form grouping tag, nullable),
+  `date` (ISO date string, nullable) — the metadata filters.
+- `source_filename` (string, unique), `source_rev` (string — content hash / rev
+  for incremental sync), `file_path` (string, source-internal, never exposed).
+- `has_media` (bool), `word_count` (int), `indexed_at` (Date).
+
+**`chunks`** — one embeddable passage per document (the retrieval unit)
+- `document_id` (ObjectId → documents), `chunk_index` (int),
+- `content` (string — indexed by the `$text` index, returned as the passage),
+- `embedding` (float[] — the semantic vector; `null` if embeddings unconfigured).
+
+Support collections: `settings` (feature flags, synonyms, `catalog.last_refresh`),
+`usage_events`, `index_reports`, `sessions`.
+
+**Adapt the metadata to the domain:** rename/extend `collection` and `date` in
+`indexing.py::_parse_metadata`, and mirror the field names in `search.py`,
+`main.py`, and the web `SearchView`.
+
+---
+
+## 3. How CloudGrid injects the datastore
+
+You never provision infra or set a connection string by hand. `needs:{database:
+true}` provisions Mongo and injects it as an env var at `grid dev` (local) and at
+runtime (after `grid plug`):
+
+| What | Declared in cloudgrid.yaml | Injected env var(s) | Read in code |
+|------|----------------------------|---------------------|--------------|
+| MongoDB | `needs: { database: true }` | `DATABASE_MONGODB_URL` (+ legacy `MONGODB_URL`) | `app/db.py::get_client` (lazy) |
+
+Read it LAZILY — `os.environ.get("DATABASE_MONGODB_URL") or
+os.environ.get("MONGODB_URL")` inside `get_client()`, never at module top level.
+`requires:` is the deprecated v1 alias; never author it and never set `needs:` and
+`requires:` together (the validator rejects the combination).
+
+**No `needs: vector`.** The ideal embedding store is pgvector
+(`needs: { vector: pgvector }` → `VECTOR_PGVECTOR_URL`), but the runtime
+role-grant is **blocked on platform issue #1545**. Until it lands, embeddings live
+in the Mongo `chunks.embedding` float array and are cosine-ranked in-app
+(`search.py::_cosine`). When #1545 ships, move the vectors to pgvector and replace
+the in-app cosine block with an `ORDER BY embedding <=> $q LIMIT k` query — the
+rest of the pipeline is unchanged.
+
+---
+
+## 4. Pluggable document source (SOURCE_TYPE)
+
+The source is where raw documents live, not the query DB. `SOURCE_TYPE` selects
+the adapter in `app/source.py`; all creds are gated in `app/config.py` and read
+lazily:
+
+- **`dropbox`** (default) — read-only, offline refresh-token flow. Secrets:
+  `DROPBOX_APP_KEY`, `DROPBOX_APP_SECRET`, `DROPBOX_REFRESH_TOKEN`,
+  `DROPBOX_FOLDER_PATH` (optional).
+- **`local`** — a directory on the container filesystem. Set `LOCAL_SOURCE_PATH`
+  (or add `needs: { disk: true }` and it defaults to the injected `DISK_PATH`).
+- **`url`** — a manifest of document URLs. Set `URL_SOURCE_MANIFEST` to a
+  newline- or comma-separated list.
+
+**Add your own adapter:** subclass `Source` (implement `list_files()` returning
+`SourceFile[]` and `download(path) -> bytes`), gate its config in
+`config.py::source_configured`, and return it from `get_source()`. Text
+extraction for txt/md/pdf/docx is shared in `source.extract_text`.
+
+---
+
+## 5. Embeddings (one config point)
+
+`app/embeddings.py` is the single provider seam — OpenAI-compatible by default.
+Secrets/config (all gated, read lazily):
+
+- `EMBEDDINGS_API_KEY` — the key (semantic search + generated answers degrade
+  gracefully to lexical/extractive when unset).
+- `EMBEDDINGS_MODEL` (default `text-embedding-3-small`) — changing it changes
+  vector dimensions and requires a full reindex.
+- `EMBEDDINGS_BASE_URL` (default `https://api.openai.com/v1`) — point at any
+  OpenAI-compatible endpoint.
+- `ANSWER_MODEL` (default `gpt-4o-mini`) — the grounded-answer chat model.
+
+---
+
+## 6. Secrets via `grid secrets set`
+
+Store secrets with the CLI; never commit them, never read them at import time:
+
+```bash
+grid secrets set EMBEDDINGS_API_KEY=sk-...
+grid secrets set SOURCE_TYPE=dropbox
+grid secrets set DROPBOX_APP_KEY=... DROPBOX_APP_SECRET=... DROPBOX_REFRESH_TOKEN=...
+grid secrets set MANAGER_PASSWORD_HASH=$(printf '%s' 'your-password' | shasum -a 256 | cut -d' ' -f1)
+```
+
+`MANAGER_PASSWORD_HASH` is the sha256 hex of the manager password, compared
+server-side in `auth.py`. The in-app "Change password" screen returns a new hash
+to store the same way. Non-secret public config can also go via `grid env`.
+
+---
+
+## 7. The baked-in learnings (do NOT regress)
+
+1. **Startup index.** `main.py` has `@app.on_event("startup")` →
+   `db.ensure_indexes()`, which creates the chunks `$text` index (plus the
+   documents metadata indexes) at boot. Without it, `$text` raises
+   `IndexNotFound` (a 500) on an empty catalog. `ensure_indexes()` is safe to call
+   repeatedly and never blocks startup if Mongo is briefly unavailable.
+2. **`$text` guard.** The lexical query in `search.py` is wrapped in
+   `except OperationFailure` so a missing index degrades to "lexical contributes
+   nothing", never a 500 — belt-and-suspenders on top of the startup index.
+3. **Health without secrets.** No secret is read at import time; every secret is
+   gated inside a function (`config.py`). `GET /backend/health` needs only Mongo.
+   Search/indexing degrade gracefully (empty results + a clear note) when the
+   source or embeddings are unconfigured. `GET /backend/status` reports capability
+   **booleans** (`source_configured`, `embeddings_configured`, …), never secret
+   values.
+4. **No `needs: vector` / no active cron.** `needs:` is `database` only (#1545).
+   The cron refresh service is commented out (#1585) — see §8.
+
+---
+
+## 8. Refresh: manager endpoint now, cron later
+
+The supported refresh path today is the manager-only endpoint
+`POST /backend/admin/refresh` (wired to the "Refresh now" button in Settings). It
+runs the same incremental sync pipeline (`indexing.run_sync`) a cron would.
+
+A scheduled `type: cron` refresh service is a **follow-up**: a Python cron entity
+is currently blocked on **platform issue #1585**. The `cloudgrid.yaml` keeps a
+`refresh:` cron service commented out with a `services/refresh` source pointing at
+the same `indexing.run_sync`. When #1585 lands, uncomment it (`schedule`,
+`timezone`, `run: job`) and add a thin `services/refresh/src/main.py` that calls
+`app.indexing.run_sync(triggered_by="cron")`.
+
+---
+
+## 9. Deploy + edition note
+
+This is a built + deployed multi-service app → the **local edition** (Claude
+Desktop / Claude Code) or the CLI; the hosted edition cannot build a runtime
+container.
+
+1. `grid init semantic-search` — creates the entity + `.cloudgrid/link.json`.
+2. Put the code under `services/web/` and `services/backend/`; set `cloudgrid.yaml`
+   to the active fields (web static+build, backend python, `needs:{database:true}`).
+3. `grid secrets set …` for embeddings + source + `MANAGER_PASSWORD_HASH` (§6).
+4. (Optional) `grid dev` to run locally against injected dev Mongo.
+5. **Build the frontend first:** `(cd services/web && npm install && npm run build)`.
+   CloudGrid validates that a `type: static` service carrying a `build:` block
+   already has `services/web/dist/index.html` on disk at plug time — plug fails
+   validation otherwise.
+6. `grid plug` — a runtime deploy is **ASYNC**: the first response is
+   `status: building`, not a live URL. Poll `grid status` (surface a liveness
+   signal, never a bare wait) until Ready.
+7. Sign in as manager, hit "Refresh now" to index the source, then search.
+   Re-plug the same entity to update the same URL.

package/src/corpus/templates/semantic-search/README.md

@@ -0,0 +1,74 @@
+# semantic-search template — CloudGrid-native document search
+
+A deployable, domain-agnostic **document search** app: index a folder of
+documents (PDF / DOCX / TXT / MD), then search them by keyword, by meaning, and
+by metadata — with a grounded answer mode and a manager admin. Data lives in the
+grid-shared MongoDB, so the catalog survives refresh and is shared across users.
+
+**Hybrid search** = Mongo `$text` (lexical) + in-app NumPy cosine over chunk
+embeddings (semantic) + metadata filters (date / collection), blended and
+aggregated to the document level with a best-passage highlight.
+
+## Services
+
+| Service | Type | Path | Entry |
+|---|---|---|---|
+| `web` | `static` (React, Vite) | `/` | build → `dist` |
+| `backend` | `python` (FastAPI) | `/backend` | `services/backend/src/main.py` |
+
+`needs: { database: true }` → CloudGrid injects `DATABASE_MONGODB_URL` (fallback
+`MONGODB_URL`). **No `needs: vector`** — pgvector runtime role-grant is blocked
+(platform #1545), so embeddings live in the Mongo `chunks` collection and are
+cosine-ranked in-app. A scheduled (cron) refresh is a follow-up, blocked on
+platform #1585 — ship the manager "Refresh now" endpoint for now (see AGENTS.md).
+
+## Pluggable document source
+
+`SOURCE_TYPE` selects the adapter, all credentials from env secrets:
+
+- `dropbox` (default) — read-only Dropbox via the offline refresh-token flow.
+- `local` — a directory on the container filesystem (`LOCAL_SOURCE_PATH`, or a
+  `needs: disk` mount at `DISK_PATH`).
+- `url` — a manifest of document URLs (`URL_SOURCE_MANIFEST`).
+
+Add your own by subclassing `Source` in
+`services/backend/src/app/source.py` and returning it from `get_source()`.
+
+## Embeddings
+
+Behind one config point (`services/backend/src/app/embeddings.py`), OpenAI by
+default. `EMBEDDINGS_API_KEY` from env; `EMBEDDINGS_MODEL` / `EMBEDDINGS_BASE_URL`
+to point at any OpenAI-compatible provider.
+
+## Secrets (set with `grid secrets set KEY=VALUE`)
+
+`EMBEDDINGS_API_KEY`, `SOURCE_TYPE`, and the source's own secrets
+(`DROPBOX_APP_KEY` / `DROPBOX_APP_SECRET` / `DROPBOX_REFRESH_TOKEN` /
+`DROPBOX_FOLDER_PATH`, or `LOCAL_SOURCE_PATH`, or `URL_SOURCE_MANIFEST`), and
+`MANAGER_PASSWORD_HASH` (sha256 hex of the manager password).
+
+The app starts and passes `GET /backend/health` with **none** of these set —
+search / indexing degrade to a clear "not indexed / missing secret" state.
+
+## Health without secrets (baked-in)
+
+- `@app.on_event("startup")` calls `db.ensure_indexes()` so the chunks `$text`
+  index exists at boot — `$text` returns `[]` on an empty catalog, never a 500.
+- The `$text` query is additionally guarded in `except OperationFailure`.
+- No secret is read at import time; every secret is gated inside a function.
+- `/backend/status` reports capability booleans, never secret values.
+
+## Deploy (manager does this)
+
+```bash
+grid init semantic-search   # creates the entity + .cloudgrid/link.json
+grid secrets set EMBEDDINGS_API_KEY=sk-...   # + source secrets + MANAGER_PASSWORD_HASH
+
+# Build the static frontend BEFORE plug — CloudGrid validates that a
+# type:static service with a build: block already has services/web/dist/index.html.
+(cd services/web && npm install && npm run build)
+
+grid plug                    # async build + deploy; poll to a live URL
+```
+
+See `AGENTS.md` for the full wiring guide.

package/src/corpus/templates/semantic-search/cloudgrid.yaml

@@ -0,0 +1,217 @@
+# ============================================================================
+# cloudgrid.yaml — Complete Example
+# ============================================================================
+#
+# This file shows EVERY field the platform supports, with comments explaining
+# each one. It is a reference — no real project would use all of these at once.
+#
+# STANDARD: every CloudGrid template ships its cloudgrid.yaml in THIS full form.
+# Uncomment ONLY the fields the archetype needs (always: name + services; plus
+# needs: for runtime apps). Leave everything else commented. Comments are
+# ignored by the parser, so the file deploys to exactly the active fields.
+#
+# Only two fields are required: name and services. Everything else is optional.
+# Full docs: docs/cloudgrid-yaml-reference.md
+# ============================================================================
+
+
+# ─── IDENTITY ───────────────────────────────────────────────────────────────
+
+# Entity slug (REQUIRED). 2-42 chars, lowercase alphanumeric + hyphens.
+# Must start and end with a letter or digit.
+# Becomes part of your public URL: <name>-<4hex>.<org>.cloudgrid.io
+name: semantic-search
+
+# One-line summary. Max 280 chars. If omitted, Grid Brain auto-generates one.
+# description: Hybrid semantic + keyword search over your documents
+
+# Single emoji or short text shown as the entity icon. Max 64 chars.
+# icon: "🔎"
+
+# Entity type: 'app' (default) or 'agent'. An agent is an app with an agent: block.
+# kind: app
+
+
+# ─── NETWORKING ─────────────────────────────────────────────────────────────
+
+# Whether this entity serves public HTTP traffic.
+# true = public URL + Ingress (default). false = internal only.
+# expose: true
+
+# Additional domain names this entity serves on (the platform URL is always active).
+# custom_domains:
+#   - orders.example.com
+#   - shop.example.com
+
+
+# ─── HEALTH PROBE (entity-level override) ──────────────────────────────────
+
+# Override the default health check for the entire entity.
+# probe:
+#   path: /healthz               # HTTP path to probe (default: /).
+#   expected: "<500"             # '<500' (default), '200', '2xx'.
+
+
+# ─── DEPLOY NOTIFICATIONS ──────────────────────────────────────────────────
+
+# Slack notification to the org's default channel on deploy failure.
+# true = notify (default), false = silent.
+# notify: true
+
+
+# ─── SCALING ────────────────────────────────────────────────────────────────
+
+# Replica scaling intent.
+#   'auto'  — platform decides from the grid tier (default).
+#   'alert' — single-replica, emit a SPOF warning at plug time.
+#   1-10    — explicit replica count, clamped to the grid max.
+# scale: auto
+
+
+# ─── INFRASTRUCTURE NEEDS ──────────────────────────────────────────────────
+#
+# Declare infrastructure; the platform provisions it and injects connection
+# details as env vars. Nine needs: database, cache, kv, queue, pubsub, vector,
+# object_storage, disk, ai. Simplest form is `true`.
+#
+# needs:
+#   # database — MongoDB. Injects DATABASE_MONGODB_URL (+ legacy MONGODB_URL).
+#   #   true | pool | dedicated | { tier: external, secret: MY_DB }
+#   database: true
+#   # cache — Redis, LRU eviction (derived data). Injects CACHE_REDIS_URL (+ REDIS_URL).
+#   #   true | redis | { engine: external, secret: MY_REDIS_URL }
+#   cache: true
+#   # kv — Redis, no eviction (durable: flags, idempotency). Injects KV_REDIS_URL.
+#   kv: true
+#   # queue — Redis job-queue substrate (BullMQ). Injects QUEUE_REDIS_URL.
+#   queue: true
+#   # pubsub — Redis broadcast pub/sub. Injects PUBSUB_REDIS_URL.
+#   pubsub: true
+#   # vector — pgvector on Postgres. Injects VECTOR_PGVECTOR_URL (+ PGVECTOR_URL).
+#   #   true | pgvector | { engine: pgvector, dim: 1536 }
+#   #   NOT USED HERE — pgvector runtime role-grant is blocked (platform #1545).
+#   #   This template stores embeddings in the Mongo `chunks` collection and
+#   #   cosine-ranks in-app instead. Swap to `vector: pgvector` once #1545 lands.
+#   vector: pgvector
+#   # object_storage — GCS bucket. Injects OBJECT_STORAGE_GCS_BUCKET, _GCS_REGION.
+#   #   true | { size: 10Gi }
+#   object_storage: true
+#   # disk — persistent FS mount (PVC). Injects DISK_PATH. Pins to 1 replica.
+#   #   true | { size: 1Gi, mount: /var/data }  (forbidden: /proc /sys /dev)
+#   #   Use with SOURCE_TYPE=local to read documents from the mount.
+#   disk: true
+#   # ai — AI Gateway via @cloudgrid-io/ai. Injects AI_GATEWAY_URL. true | false.
+#   ai: true
+
+
+# ─── INTER-ENTITY COMMUNICATION ────────────────────────────────────────────
+
+# Entities this app calls (outbound). Service-account token injected per pair.
+# calls:
+#   - org:atomic/billing-api
+
+# Inbound caller policy — who can call YOUR entity.
+# callers:
+#   policy: auto                 # 'auto' | 'manual' (default).
+#   scope: org                   # 'org' (default) | 'space' | 'explicit'.
+#   fallback: deny               # 'deny' (default) | 'warn'.
+#   # allowed_apps:              # required only when scope: explicit
+#   #   - org:atomic/frontend
+
+
+# ─── VAULT ──────────────────────────────────────────────────────────────────
+
+# Map org vault items to env vars. Keys UPPER_SNAKE_CASE; values = vault item keys.
+# The source + embeddings secrets can also be stored directly with
+# `grid secrets set KEY=VALUE` — see AGENTS.md.
+# vault:
+#   EMBEDDINGS_API_KEY: embeddings-api-key
+#   DROPBOX_REFRESH_TOKEN: dropbox-refresh-token
+
+
+# ─── CONNECTORS AND HOOKS ──────────────────────────────────────────────────
+
+# connectors:
+#   - id: team-slack
+#     target: org                # 'system' | 'org'.
+#     channel: "#deploys"
+# hooks:
+#   - on: deploy.failed          # release.published | deploy.failed | feedback.received | entity.created
+#     via: team-slack
+#     template: "Deploy failed for {entity_name}: {error}"
+#     channel: "#alerts"
+# triggers:
+#   - from: team-slack
+#     event: message
+#     channel: "#support"
+#     route: /webhooks/slack
+
+
+# ─── LOCAL DEVELOPMENT ──────────────────────────────────────────────────────
+
+# Config for `grid dev`. Redact fields in dev data snapshots.
+# dev:
+#   snapshot:
+#     mask:
+#       - users.email
+#       - users.phone
+
+
+# ─── AGENT METADATA ────────────────────────────────────────────────────────
+
+# An agent is an app with kind: agent (above) and an agent: block.
+# agent:
+#   purpose: Sync customer data from external CRM every night
+#   schedule: "0 2 * * *"
+#   trigger: webhook
+
+
+# ─── PRIVATE RESOURCE AUTO-DEPENDENCIES ────────────────────────────────────
+
+# When true (default), private resources are auto-injected as depends_on.
+# auto_depends_on_private_requires: true
+
+
+# ─── SERVICES (REQUIRED) ───────────────────────────────────────────────────
+#
+# At least one service is required. Key = service name (slug rules).
+# Types: node, nextjs, python, static, cron.
+#
+services:
+  # React (Vite) built to static assets, served at the root.
+  web:
+    type: static
+    path: /
+    build:
+      command: npm run build
+      output: dist
+    node_version: "22"
+  # Python FastAPI backend — hybrid search, answer mode, manager admin.
+  # Entry: services/backend/src/main.py (uvicorn app.main:app).
+  backend:
+    type: python
+    path: /backend
+  # A scheduled `type: cron` refresh service is a follow-up — a Python cron
+  # entity is currently blocked on platform issue #1585. Until it lands, use the
+  # manager "Refresh now" endpoint (POST /backend/admin/refresh). See AGENTS.md.
+  # refresh:
+  #   type: cron
+  #   schedule: "0 3 * * *"
+  #   timezone: UTC
+  #   run: job
+  #   source:
+  #     path: services/refresh
+# CloudGrid managed Mongo → injects DATABASE_MONGODB_URL (+ legacy MONGODB_URL).
+# NO `needs: vector` — pgvector runtime role-grant is still blocked (#1545);
+# embeddings live in the Mongo `chunks` collection and are cosine-ranked in-app.
+needs:
+  database: true
+
+
+# ============================================================================
+# NOTES
+# - NOT here: org:/entity_id: (→ .cloudgrid/link.json), secrets (→ grid secrets set),
+#   Dockerfile (platform generates from services.*.type).
+# - Deploy: grid plug   |   Local with grid resources: grid dev
+# - Full reference: docs/cloudgrid-yaml-reference.md  |  CLI: docs/cli-reference.md
+# ============================================================================

package/src/corpus/templates/semantic-search/index.md

@@ -0,0 +1,97 @@
+# Template: semantic-search (CloudGrid-native document search)
+
+A deployable, domain-agnostic **document search** app — index a folder of
+documents (PDF/DOCX/TXT/MD), then search by keyword, by meaning, and by metadata,
+with a grounded answer mode and a manager admin. Two services: a React (Vite)
+static frontend at `/` and a FastAPI backend at `/backend`, backed by
+grid-shared Mongo. This template ships **real, runnable code** — read `AGENTS.md`,
+adapt the naming/metadata to your domain, then deploy.
+
+**Hybrid search** = Mongo `$text` (lexical) + in-app NumPy cosine over chunk
+embeddings (semantic) + metadata filters, blended and aggregated to the document
+level with a best-passage highlight.
+
+**Key rules (proven by a live build — do not regress these):**
+
+1. **Every service's code lives under `services/<name>/`.** `path:` in
+   `cloudgrid.yaml` is the URL mount, NOT the filesystem path. The `web` service
+   → `services/web/`; the `backend` service → `services/backend/`.
+2. **Read the Mongo URL LAZILY** — `os.environ.get("DATABASE_MONGODB_URL") or
+   os.environ.get("MONGODB_URL")` inside `get_client()`, never at import time.
+3. **Health is green with NO secrets set.** No secret is read at import time;
+   every secret is gated inside a function. `GET /backend/health` needs only Mongo.
+4. **`@app.on_event("startup")` calls `db.ensure_indexes()`** so the chunks
+   `$text` index exists at boot — `$text` returns `[]` on an empty catalog rather
+   than raising `IndexNotFound` (a 500). The `$text` query is also guarded in
+   `except OperationFailure` as belt-and-suspenders.
+5. **No `needs: vector`** — pgvector runtime role-grant is blocked (#1545), so
+   embeddings live in the Mongo `chunks` collection (a float array) and are
+   cosine-ranked in-app. **No active cron** — a Python `type: cron` refresh is
+   blocked (#1585); ship the manager "Refresh now" endpoint and add cron later.
+
+## File tree
+
+```
+cloudgrid.yaml                                 # name + services(web static+build, backend python) + needs:{database:true}
+services/web/                                  # React + Vite static frontend (LTR, neutral)
+  package.json  vite.config.js  index.html
+  src/main.jsx  App.jsx  api.js  styles.css
+  src/components/SearchView.jsx ManagerLogin.jsx Settings.jsx Dashboard.jsx
+services/backend/                              # FastAPI backend, mounted at /backend
+  requirements.txt
+  src/main.py                                  # uvicorn entry (reads PORT; no secret at import)
+  src/app/
+    main.py                                    # routes + startup ensure_indexes()
+    config.py                                  # lazy secret gating (SOURCE_TYPE, EMBEDDINGS_*, ...)
+    db.py                                       # lazy Mongo; ensure_indexes(); collections documents+chunks
+    search.py                                   # hybrid: $text + cosine + metadata, blended to doc level
+    indexing.py                                 # discover -> extract -> chunk -> embed -> store (incremental)
+    source.py                                   # pluggable adapter: dropbox | local | url
+    embeddings.py                               # one config point (OpenAI-compatible)
+    answer.py                                   # grounded answer + citations (extractive fallback)
+    auth.py                                     # manager session auth (MANAGER_PASSWORD_HASH)
+```
+
+## cloudgrid.yaml (active fields)
+
+```yaml
+# On disk this file is the full-annotated reference (templates/_cloudgrid.yaml.reference)
+# with EVERY field present as a comment; only the fields below are uncommented.
+name: semantic-search
+services:
+  web:
+    type: static
+    path: /
+    build:
+      command: npm run build
+      output: dist
+    node_version: "22"
+  backend:
+    type: python
+    path: /backend
+needs:
+  database: true
+```
+
+> **Capability:** the need is `database: true` — the deployer provisions Mongo and
+> injects `DATABASE_MONGODB_URL` (+ legacy `MONGODB_URL`). NO `needs: vector`
+> (#1545) and NO active cron (#1585). `requires:` is the deprecated v1 alias —
+> don't mix it with `needs:`.
+
+## Configure + deploy
+
+1. `grid init semantic-search` — creates the entity + `.cloudgrid/link.json`.
+2. `grid secrets set EMBEDDINGS_API_KEY=sk-…` plus the source secrets
+   (`SOURCE_TYPE` + Dropbox/local/url config) and `MANAGER_PASSWORD_HASH`
+   (sha256 hex of the manager password). See `AGENTS.md`.
+3. (Optional) `grid dev` to run locally against the injected dev Mongo.
+4. `grid plug` to deploy — a runtime deploy is **async**: poll status until live,
+   then sign in as manager and hit "Refresh now" to index the source.
+
+## Adapt it
+
+- Rename the `collection` / `date` metadata to your domain in
+  `services/backend/src/app/indexing.py` (`_parse_metadata`).
+- Add a source adapter: subclass `Source` in `source.py`, return it from
+  `get_source()`, gate its secrets in `config.py`.
+- Point embeddings at any OpenAI-compatible provider via `EMBEDDINGS_BASE_URL`.

package/src/corpus/templates/semantic-search/services/backend/requirements.txt

@@ -0,0 +1,7 @@
+fastapi==0.111.0
+uvicorn[standard]==0.30.1
+pymongo==4.8.0
+numpy==1.26.4
+httpx==0.27.0
+pypdf==4.2.0
+python-docx==1.1.2