@yugabytedb/meko-mcp 1.0.8 → 1.2.0

package/assets/skills/meko-mcp-tools-desktop/references/tools-memory-vs-conversation.md

@@ -0,0 +1,217 @@
+# When to Use Memory Tools vs Conversation Tools
+
+**WRONG:** `memory_add(text="User asked about pricing. I explained tiers. User chose Pro.")` — mem0 extracts facts, discarding conversational structure, turn ordering, and reasoning traces.
+
+## Use `memory_add` when storing:
+
+- A **fact**: "User prefers dark mode"
+- A **preference**: "Customer budget is $50k"
+- An **entity relationship**: "Alice works at Acme Corp"
+
+Memory is backed by pgvector (semantic search) and Apache AGE (entity-relationship graph). It retrieves relevant facts by meaning, not by conversation order.
+
+```
+memory_add(scope="write", agent_id="support_bot",
+    text="Customer Alice (alice@acme.com) prefers email, has Pro plan.", user_id="alice_123")
+
+memory_search(scope="read", query="What plan does Alice have?", agent_id="support_bot")
+```
+
+## Use `conversation_create` + `conversation_add_message` when storing:
+
+- A **full dialog exchange** with input/output pairs
+- A **chain-of-thought** reasoning trace alongside the response
+- A **multi-turn chat** where turn order matters
+
+Conversations are backed by Langfuse sessions and traces. They preserve full structure: who said what, in what order, with what reasoning.
+
+```
+conversation_create(scope="write", agent_id="support_bot", user_id="alice_123", title="Pricing discussion")
+-- Returns: {"id": "conv-uuid-here"}
+
+conversation_add_message(scope="write", conversation_id="conv-uuid-here", agent_id="support_bot",
+    input="What are your pricing tiers?",
+    output="We offer Starter ($10/mo), Pro ($50/mo), and Enterprise (custom).",
+    reasoning="Retrieved pricing page data. No special discounts apply.")
+```
+
+## Decision Matrix
+
+| Signal | Tool |
+|--------|------|
+| "remember that...", "note that...", "keep in mind..." | `memory_add` |
+| "store this conversation", "save this chat" | `conversation_create` + `conversation_add_message` |
+| "what do you know about X?", "recall..." | `memory_search` |
+| "show me our past conversation about..." | `conversation_list` + `conversation_get` |
+| "the user prefers...", "their budget is..." | `memory_add` |
+| "save my query and your response" | `conversation_add_message` |
+
+## Verbatim Content Rules for conversation_add_message
+
+**WRONG:** Summarizing or rephrasing content before storing:
+```
+conversation_add_message(...,
+    input="User inquired about database performance",        -- rephrased
+    output="I provided optimization suggestions",            -- summarized
+    reasoning="Analyzed the query and suggested improvements" -- condensed
+)
+```
+
+All three fields — `input`, `output`, and `reasoning` — must contain the **exact, verbatim text**. Never summarize, rephrase, editorialize, or condense any field:
+
+- **`input`**: The user's exact prompt, word for word
+- **`output`**: The assistant's complete response, word for word — including code blocks, formatting, and all detail
+- **`reasoning`**: The full chain-of-thought or internal reasoning trace, unedited
+
+The purpose of conversation storage is to create a faithful, replayable record. A rephrased summary loses the original wording, tone, and detail — making the stored conversation useless for review, debugging, or audit.
+
+## Personal memory vs. team-shared knowledge — the three read paths
+
+Agents on Cloud Meko have three distinct read surfaces. Pick the right one for the question.
+
+| Read path | What it returns | How to call |
+|---|---|---|
+| Your own desktop memories | Memories you wrote under this `agent_id`, filtered by `(datapack_id, user_id, agent_id)` | `memory_search(agent_id="claude_desktop", query="...", ...)` |
+| Cross-project common bucket | Memories any agent (including this one) wrote with `agent_id="meko_agent"` — the empty-default bucket for genuinely cross-project facts | `memory_search(agent_id="meko_agent", query="...", ...)` |
+| Team's shared knowledge | Promoted memories + uploaded documents, visible to every member of the datapack | `knowledgebase_search(agent_id="<anything>", datapack_id="<datapack UUID>", query="...")` — `agent_id` is ignored |
+
+For a true fan-out across other clients' buckets (e.g. also see what was written from `claude_code:some-repo` or `cursor:foo`), call `memory_search` once per agent_id — there's no single-call shortcut. Empty string is rewritten to `meko_agent` server-side; it does not drop the filter.
+
+### How content gets into each surface
+
+- **Personal memories** — written by `memory_add`. Scoped per-user and per-agent from the moment of write. Only the writer sees them via MCP reads.
+- **Team-shared Shared Knowledge** — arrives two ways:
+  1. The **user** (never the agent) promotes a personal memory to Knowledge via the Cloud UI's Learnings tab. That copies the row into the shared `knowledge_base` table with the `user_id` stripped, making it team-wide. Originating `agent_id` is preserved in metadata for provenance.
+  2. The user uploads a file via Datapack → Actions → **Add Knowledge** in the Cloud UI. PDF/TXT/MD/JSON/MP4 up to 5MB each.
+
+  Both show up in `knowledgebase_search`, tagged `metadata_filters.source: "memory"` vs other values so the agent can distinguish provenance in responses.
+
+**No MCP tool promotes to Shared Knowledge.** The agent can flag memories it thinks are worth promoting and suggest the user do it from the UI — but the action is user-initiated.
+
+### When the user asks "what do you know about X?"
+
+A full sweep is three calls (budget for it — each is 2-6 seconds):
+
+```
+memory_search(agent_id="claude_desktop", query="X", conversation_id=..., ...)
+memory_search(agent_id="meko_agent", query="X", conversation_id=..., ...)  # cross-project common bucket
+knowledgebase_search(agent_id="<anything>", datapack_id="<uuid>", query="X", conversation_id=..., ...)
+```
+
+When you answer, be explicit about scope so the user knows why something is or isn't there:
+
+- "I found this in your own desktop memories…"
+- "I found this in your common cross-project bucket (something you or another agent wrote under `meko_agent`)…"
+- "I found this in your team's Shared Knowledge — someone (you or a teammate) promoted it earlier…"
+- "I couldn't find anything in your personal memories. You might want to check the Cloud UI's Memory Summary tab for a cross-agent view."
+
+For a true fan-out across other clients' buckets (e.g. also see what was written from `claude_code:some-repo` or `cursor:foo`), call `memory_search` once per known agent_id — empty string just resolves to `meko_agent`, not a fan-out.
+
+## Memory limitations for structured data
+
+**WRONG:** Using `memory_add` to ingest CSV rows, data dictionaries, or tabular schemas — Mem0 extracts individual facts and may drop columns, rows, or structural relationships.
+
+On Cloud Meko today the `db_run_write_query` path isn't available (returns `{"error": "not_available"}`), so there's no MCP-side "just put it in a database table" option. Workarounds:
+
+1. **Write a single narrative summary** as a memory: `"The Customer Complaints dataset has fields: Complaint ID (int), Date Submitted (date), Product (text), Sub-product (text), Issue (text)..."` — one call, not row-by-row.
+2. **Suggest the user upload the file** via the Cloud UI's Add Knowledge flow if it's a document (CSV files aren't in the supported list — PDF/TXT/MD/JSON/MP4 are; for CSV, convert to MD or JSON first).
+3. **Never ingest CSV row-by-row** into memory — each row becomes a fragmented fact with lost context.
+
+## Proactive Memory Storage
+
+Don't wait for the user to say "remember this." Store context proactively when you detect:
+
+- **Personal info** → `memory_add`: "User is VP of Product, prefers concise responses"
+- **Team conventions** → `memory_add`: "Team uses Python for backend, Go for infrastructure"
+- **Domain knowledge** → `memory_add`: "Meko traces are decision traces that capture how and why decisions were made"
+- **Corrections** → `memory_add`: "User corrected: always use 'resilience' instead of 'high availability'"
+
+Use `memory_search` at the start of sessions to recall what you already know before asking the user to repeat themselves.
+
+## Using Both Together
+
+A common pattern: store the full conversation (preserves structure) AND extract key facts with `memory_add` (enables semantic search later).
+
+## Key Retrieval Difference
+
+- `memory_search`: **semantic similarity** across all memories via pgvector. Returns relevant facts regardless of when stored.
+- `conversation_get`: **ordered list of message turns** for a specific conversation. No semantic search. Requires `conversation_id`.
+
+## Automatic Conversation Capture
+
+The Meko plugin captures conversations automatically via three mechanisms:
+
+1. **Periodic checkpoint (~10 min)** — Agent-driven via CronCreate, uses `conversation_add_message` MCP tool directly
+2. **PreCompact hook** — Shell script that fires before Claude Code's auto-compaction, captures via MCP JSON-RPC
+3. **SessionEnd hook** — Shell script that fires at session termination, captures final exchanges
+
+### What gets captured
+
+The full transcript including:
+- **User prompts** — verbatim text (the `input` field)
+- **Assistant responses** — all text blocks (the `output` field)
+- **Tool calls** — tool name + input parameters (the `reasoning` field)
+- **Tool results** — output from tool executions (appended to `reasoning`)
+
+This is more than just text — tool calls and results are critical context for session replay, cross-agent sharing, and compaction.
+
+### Seed-based deduplication
+
+Each message's seed is `<conversation_id>:<user_message_uuid>`. All three capture mechanisms generate the same seed for the same exchange, producing the same deterministic trace ID via `sha256(seed)[:16].hex()`. This means duplicate writes from overlapping captures are idempotent.
+
+### When to manually store conversations
+
+Automatic capture handles the raw exchange. You should still manually use `conversation_create` + `conversation_add_message` when:
+- The user explicitly asks to "save this conversation"
+- You want to add curated `reasoning` traces beyond raw tool calls
+- You want to store a selected subset of the conversation with a descriptive title
+
+### Watermark coordination
+
+A shared watermark file at `~/.claude/meko-capture/<session-id>.watermark.json` tracks what has been saved. Both hooks and the agent's periodic checkpoint use this to avoid reprocessing already-captured exchanges.
+
+## Observability: conversation_id IS the Langfuse trace ID
+
+This is the feature agents tend to miss. Verified live 2026-05-07 in the Meko UI's Observe hub.
+
+- `conversation_create` returns a `conversation_id` (e.g. `d71fc8b61a7f45d585eea9a6436e0c3f`).
+- That ID is also the **Langfuse trace_id** for the conversation. Not a separate trace — literally the same identifier.
+- Every subsequent MCP tool call made while passing that `conversation_id` becomes a **span under that trace**, named after the tool: `Memory Add (Meko MCP)`, `Memory Search (Meko MCP)`, `Knowledgebase Search (Meko MCP)`, `Conversation Add Message (Meko MCP)`, etc.
+- Child spans reveal internals. Observed structure:
+  - `memory_add`: `Memory Add (Meko MCP)` → `Create a new memory in Mem0` → `Bulk Payload Update (yugabytedb / vector)` → `Execute Query (yugabytedb / vector)`
+  - `memory_search`: `Memory Search (Meko MCP)` → `Search for memories based on a query in Mem0` (Retriever)
+  - `conversation_add_message`: `Conversation Add Message (Meko MCP)` → `Conversation Message Added` span → `Reasoning` generation (the `reasoning` field is stored as a Langfuse *generation*-typed span, not a plain text field)
+  - `knowledgebase_search`: `Knowledgebase Search (Meko MCP)` → multiple `Execute Query (yugabytedb)` child spans
+- Trace tags include `agent:<agent_id>` and the Meko org UUID. Both are searchable/filterable in Langfuse.
+- Each datapack has its own Langfuse project — traces for a tool call land in the project for whichever `datapack_id` was on the call (or the default datapack if omitted).
+
+### Finding a trace in the UI
+
+Two navigation paths, both verified in the Meko UI:
+
+1. **Datapack → Conversations tab** — every row has an **Open in Observe** button that deep-links to the trace with `session=<conversation_id>` pre-filled. Fastest when you know which conversation to inspect.
+2. **Observe hub** — browse all traces across a datapack. URL shape: `/observe-hub?project=<langfuse_project_id>&session=<conversation_id>`.
+
+The Conversations-tab row preview shows the first user turn verbatim, plus the agent_id tag, message count, and started-at timestamp. If you want a conversation to be recognizable in the UI, call `conversation_add_message` with a clean first-turn `input` early in the session.
+
+### Observed latencies on prod (2026-05-07)
+
+From real traces, not the docstring happy path:
+
+| Tool | Latency per call |
+|---|---|
+| `memory_add` | 12-21 seconds (calls OpenAI for extraction, then writes to vector + graph) |
+| `memory_search` | 1.8-5.9 seconds |
+| `knowledgebase_search` | 0.6 seconds (empty index; will grow with index size) |
+| `conversation_add_message` | 0.11 seconds (fire-and-forget; queued to Langfuse) |
+
+`memory_add` is the expensive one. If you're capturing a long conversation, batch through `conversation_add_message` (cheap, queued) and let the server-side Mem0 extractor handle the heavy lifting on its own schedule, rather than firing `memory_add(messages=...)` per turn.
+
+### Proactive diagnostic pattern
+
+If a user reports "my memory_add didn't seem to save", open the Observe hub for their datapack, filter on their `conversation_id`, and look at the span tree. You will see either:
+
+- A `Memory Add (Meko MCP)` span with `Create a new memory in Mem0` child → the call reached Mem0. The fact extractor may have returned `"results": []` (common on short or non-fact-shaped text; not a bug, but lossy).
+- No span at all → the call never reached the server. Likely a transport / auth / `conversation_id` mismatch issue.
+
+The mem0 extractor is lossy — it extracts atomic facts and graph triples, and may drop prose that doesn't look fact-shaped to it. Observed 2026-05-07: 2 of 5 `memory_add` calls returned `"results": []` and only the graph side landed. When preserving authoritative text matters, store it via `conversation_add_message` (verbatim in `input`/`output`/`reasoning`) — the conversation path preserves text without LLM re-extraction.

package/assets/skills/meko-mcp-tools-desktop/references/tools-overview.md

@@ -0,0 +1,112 @@
+# Complete Tool Catalog and Decision Tree
+
+**26 tools on cloud Meko** (verified against `https://mcp.mekodata.ai/mcp` on 2026-05-07). Earlier counts of 25 or 31 in other docs are stale. Grouped: Memory (8), Conversation (6), Knowledge Base (4), Datapack (5), Database (3).
+
+## Cloud Free-Tier gating at a glance
+
+Some tools return `{"error": "not_available", "detail": "..."}` on cloud Free Tier. This is a server-side tier block — don't retry.
+
+- **Gated on Free Tier**: `db_summarize_database`, `db_run_read_only_query`, `db_run_write_query`, `knowledgebase_trigger_index_creation`, `knowledgebase_add_source_to_index`, `knowledgebase_check_index_status`
+- **Works on Free Tier**: all memory tools, all conversation tools, `knowledgebase_search`, all datapack tools, `flush_pending_memory_candidates`
+
+## Quick health check before using tools
+
+Before starting a session, verify that the Meko MCP tools are working:
+
+1. `memory_search(scope="read", query="test", agent_id="<your-session-agent-id>", conversation_id="<session-conversation-id>")` — Confirms memory subsystem is up. Use the `agent_id` and `conversation_id` from the SessionStart `additionalContext`. May fail with "connection already closed" — see `tools-troubleshooting.md`.
+2. `datapack_list(scope="read")` — Confirms Meko API connectivity
+3. `db_summarize_database(scope="read")` — *Optional, paid-tier only.* On free-tier datapacks this returns `{"error": "not_available"}`. The old "use this for `agent_id` discovery" protocol is retired — use your client's bucket (`claude_desktop` for Claude Desktop) or `meko_agent` for the common bucket.
+
+## Decision tree: which tool do I need?
+
+```
+User wants to...
+├── Query or explore the database?
+│   ├── See what tables exist? ---------> db_summarize_database (read)
+│   ├── Run a SELECT / read query? -----> db_run_read_only_query (read)
+│   └── INSERT / UPDATE / DELETE? -------> db_run_write_query (write)
+├── Query or build a knowledge base (RAG)?
+│   ├── Search chunks in an existing KB? -> knowledgebase_search (read) — WORKS on cloud today
+│   ├── Create a new index from sources? -> knowledgebase_trigger_index_creation (write) — Free-Tier gated, self-hosted only
+│   ├── Add source to existing index? ---> knowledgebase_add_source_to_index (write) — Free-Tier gated
+│   └── Check build progress? -----------> knowledgebase_check_index_status (read) — Free-Tier gated
+├── Store or recall information?
+│   ├── Store a fact/preference/entity? -----------> memory_add (write)
+│   ├── Store a full conversation (multi-turn)? ---> conversation_create + conversation_add_message (write)
+│   ├── Search past knowledge? --------------------> memory_search (read)
+│   └── Retrieve a past conversation? -------------> conversation_get (read)
+└── Manage datapacks?
+    └── CRUD datapack? -> datapack_create/list/describe/update/delete
+```
+
+Agent / knowledge-base lifecycle is managed **outside** the MCP surface — typically in the Meko control-plane UI — and is not exposed as tools here.
+
+## Database Tools (3)
+
+| Tool | Scope | Purpose |
+|------|-------|---------|
+| `db_summarize_database(scope, schema="public", datapack_id=None)` | read | List tables with column schemas and row counts |
+| `db_run_read_only_query(scope, query, datapack_id=None)` | read | Run SELECT, returns JSON. Uses `BEGIN READ ONLY` + `ROLLBACK` |
+| `db_run_write_query(scope, query, datapack_id=None)` | write | Execute INSERT/UPDATE/DELETE/CREATE TABLE. Guardrail-validated. Auto-committed |
+
+## RAG / Knowledge Base Tools (4)
+
+| Tool | Scope | Purpose |
+|------|-------|---------|
+| `knowledgebase_search(scope, query, agent_id, conversation_id, datapack_id, limit=10)` | read | Semantic search across KB chunks. `datapack_id` is REQUIRED (no default). Works on cloud today. |
+| `knowledgebase_trigger_index_creation(scope, conversation_id, source_uris, index_name, ai_provider="OPENAI", embedding_model="text-embedding-3-large", embedding_dimensions=1536, chunk_size=1024, chunk_overlap=256, datapack_id=None)` | write | Register sources + init vector index + trigger build. **S3-only worker + no user creds; Free-Tier gated on cloud.** See `tools-rag-workflow.md`. |
+| `knowledgebase_add_source_to_index(scope, conversation_id, index_name, source_uri, chunk_size=1024, chunk_overlap=256, datapack_id=None)` | write | Add a new source (URI, not id) to an existing index. Free-Tier gated on cloud. |
+| `knowledgebase_check_index_status(scope, conversation_id, index_name, datapack_id=None)` | read | Check build progress via `dist_rag.vector_index_pipeline_details`. Free-Tier gated on cloud. |
+
+## Memory Tools (8)
+
+| Tool | Scope | Purpose |
+|------|-------|---------|
+| `memory_add(scope, text, agent_id, conversation_id, user_id=None, app_id=None, run_id=None, metadata=None, messages=None, datapack_id=None)` | write | Store fact/preference/entity as long-term memory |
+| `memory_search(scope, query, agent_id, conversation_id="", user_id=None, limit=10, datapack_id=None)` | read | Semantic search across memories + graph relations. `conversation_id` optional (omit or pass `""` for cross-conversation discovery). |
+| `memory_get_by_id(scope, memory_id, agent_id, conversation_id, datapack_id=None)` | read | Direct pgvector row lookup by UUID. Preferred over `memory_search` for exact-id verification. |
+| `memory_get_all(scope, agent_id, conversation_id, user_id=None, app_id=None, run_id=None, datapack_id=None)` | read | List all memories for agent |
+| `memory_update(scope, memory_id, text, agent_id, conversation_id, datapack_id=None)` | write | Overwrite memory text |
+| `memory_delete_by_id(scope, memory_id, agent_id, conversation_id, datapack_id=None)` | write | Delete a single memory |
+| `memory_delete_all(scope, agent_id, conversation_id, user_id=None, app_id=None, run_id=None, datapack_id=None)` | admin | Delete all memories for agent (destructive) |
+| `flush_pending_memory_candidates(scope, agent_id)` | read | Desktop-skill helper: returns an instruction payload telling the agent to scan recent user turns and `memory_add` unsaved facts. No server-side DB write. |
+
+## Conversation Tools (6)
+
+| Tool | Scope | Purpose |
+|------|-------|---------|
+| `conversation_create(scope, agent_id, user_id=None, app_id=None, run_id=None, title=None, metadata=None, session_id="")` | write | Create conversation container (Langfuse session) |
+| `conversation_add_message(scope, conversation_id, agent_id, input, output=None, reasoning=None, metadata=None, seed=None, trace_id="")` | write | Add a message turn (Langfuse trace) |
+| `conversation_get(scope, conversation_id, agent_id, include_messages=False, limit=100, offset=0)` | read | Retrieve conversation, optionally with messages |
+| `conversation_list(scope, agent_id, conversation_id="", user_id=None, limit=20, offset=0)` | read | List conversations for agent. `conversation_id` optional (omit for browse). |
+| `conversation_update(scope, conversation_id, agent_id, title=None, metadata=None)` | write | Update title or metadata |
+| `conversation_delete(scope, conversation_id, agent_id)` | admin | Delete entire conversation (destructive) |
+
+## Datapack Management Tools (5)
+
+| Tool | Scope | Purpose |
+|------|-------|---------|
+| `datapack_create(scope, name)` | write | Create new datapack |
+| `datapack_list(scope)` | read | List all datapacks |
+| `datapack_describe(scope, name, include_status=False)` | read | Describe datapack |
+| `datapack_update(scope, name, connection_string)` | write | Update connection string |
+| `datapack_delete(scope, name)` | admin | Delete datapack (irreversible) |
+
+## Platform capabilities NOT exposed via MCP
+
+Some things the Meko platform can do are not wired into the MCP tool surface today. If a user asks about them, do not invent MCP tool calls — point them at the appropriate control plane instead. Agents attempting these as MCP tools will hit "tool not found".
+
+| Capability | Where it lives | How to reach it |
+|---|---|---|
+| Create / list / delete **agents** within a datapack | `yugabyte/meko` API server | REST: `POST/GET/DELETE /datapacks/:name/agents` (`POST/GET/DELETE /agents/:agent` for targeted deletes), or the Meko control-plane UI |
+| Add / list / delete **knowledge-base sources** on a datapack | `yugabyte/meko` API server | REST: `POST/GET/DELETE /datapacks/:name/knowledge-bases`, plus `upload-url`, `upload-complete`, `create`, `status` subroutes. Also available in the Meko UI. |
+| Langfuse project-key generation | `yugabyte/meko` API server | REST: `POST /datapacks/:name/langfuse/project-keys` |
+| Account / billing / tier management | Meko UI only | Not available via MCP or public REST |
+
+The MCP-exposed `datapack_create` / `datapack_list` / `datapack_describe` / `datapack_update` / `datapack_delete` tools above are a deliberate subset — the common CRUD that an agent reasonably needs mid-conversation. Anything involving agent or KB lifecycle is control-plane territory.
+
+For knowledge-base content on cloud today, the canonical path is **UI upload**: Datapack → Actions → **Add Knowledge** (PDF/TXT/MD/JSON/MP4, 5MB each, 10/batch). `knowledgebase_search` queries the resulting index from MCP. The MCP ingestion tools (`knowledgebase_trigger_index_creation`, `knowledgebase_add_source_to_index`, `knowledgebase_check_index_status`) exist but are Free-Tier gated and rely on an S3-only worker with no user-supplied credentials — see `tools-rag-workflow.md` for the full picture. Tracked upstream as [#89](https://github.com/yugabyte/meko-mcp-server/issues/89) / MEKO-96.
+
+## Common Parameter Patterns
+
+Every tool accepts `scope` as the first parameter. All DB, RAG, and memory tools accept optional `datapack_id` to target a specific datapack's database. Memory and conversation tools require `agent_id` for namespace isolation (see `tools-agent-id-conventions.md`).

package/assets/skills/meko-mcp-tools-desktop/references/tools-rag-workflow.md

@@ -0,0 +1,134 @@
+# Knowledge Base / RAG workflow
+
+As of 2026-05-07, **knowledge-base ingestion on cloud Meko is a UI-only activity**. The MCP ingestion tools exist but are gated off on cloud and rely on an S3-only worker that has no mechanism to read user-supplied buckets. `knowledgebase_search` is the only KB tool agents can rely on today.
+
+This page tells you which tool to reach for, what's usable where, and where the current gaps are tracked.
+
+## Decision table
+
+| Situation | Use this | Why |
+|---|---|---|
+| Search an existing KB | `knowledgebase_search` | The one KB tool that works on cloud Free Tier and prod today |
+| Add documents to a KB on cloud | **Meko UI** — Datapack → Actions menu → **Add Knowledge** | File-upload dialog (PDF/TXT/MD/JSON/MP4, 5MB each, 10/batch). The MCP ingestion path is not usable on cloud |
+| Create a new vector index (self-hosted / paid) | `knowledgebase_trigger_index_creation` | Only works when the ingestion worker has IAM access to the S3 bucket you pass |
+| Add a source to an existing index (self-hosted / paid) | `knowledgebase_add_source_to_index` | Same caveat as above |
+| Poll a build (self-hosted / paid) | `knowledgebase_check_index_status` | Gated on cloud Free Tier |
+| Anything that needs KB lifecycle management (list, delete, rename) | **UI** | No MCP tools for KB lifecycle |
+
+## knowledgebase_search — the one you'll actually call
+
+```
+knowledgebase_search(
+    scope="read",
+    query="natural-language question",
+    agent_id="claude_desktop",
+    conversation_id="<uuid from conversation_create>",
+    datapack_id="<datapack UUID — REQUIRED on this tool>",
+    limit=10)
+```
+
+Required: `scope`, `query`, `agent_id`, `conversation_id`, `datapack_id`. Unlike the memory tools, `datapack_id` has no default here — you must pass it explicitly. `agent_id` is **ignored for filtering** on this tool (KB results are team-shared on the datapack), but you still pass it — your session's value is fine.
+
+Response shape (verified 2026-05-07 against prod):
+```
+{"results": [], "count": 0}
+```
+
+On a populated KB each result carries chunk content and similarity scores. On an empty or nonexistent KB you get an empty array and `count: 0` — not an error.
+
+## MCP ingestion tools — what actually happens when you call them
+
+This applies to `knowledgebase_trigger_index_creation` and `knowledgebase_add_source_to_index`. Understanding the two-layer architecture is the only way to predict whether a call will succeed.
+
+**MCP layer (`src/tools.py`):** takes `source_uris`, splits on `,`, passes each string to `dist_rag.create_source(<uri>)` with **zero validation**. No scheme check, no reachability probe. Returns `"build_triggered"` immediately.
+
+**Worker layer (`yugabyte-db/python/ai/rag_agent/rag_pipeline/aws_source_processor.py`):** only handles `s3://` URIs. Any other scheme raises `ValueError("Invalid source URI")` inside the worker — **async, never surfaced to the MCP caller**. The S3 crawler is instantiated with no credentials; it relies on boto3's default chain (env vars, IAM role, `~/.aws/credentials` on the worker host). So the worker can only read buckets its own AWS identity has access to.
+
+**Implication:** pointing the MCP tool at a user-owned S3 bucket from cloud will return `"build_triggered"` and silently stall at INIT forever. There is no way for the user to supply AWS credentials through the MCP tool.
+
+## Free-Tier cloud gating (verified 2026-05-07 against `https://mcp.mekodata.ai/mcp`)
+
+| Tool | Verbatim response |
+|---|---|
+| `knowledgebase_trigger_index_creation` | `{"error": "not_available", "detail": "Vector index creation is not available on the free tier."}` |
+| `knowledgebase_add_source_to_index` | Same as above |
+| `knowledgebase_check_index_status` | `{"error": "not_available", "detail": "Index status check is not available on the free tier."}` |
+| `knowledgebase_search` | Works: `{"results": [], "count": 0}` |
+
+`{"error": "not_available"}` is a server-side tier block. Don't retry.
+
+## UI ingestion (cloud Meko) — what the user sees
+
+From `cloud.mekodata.ai`:
+
+1. Open a datapack (`/datapacks/<name>`).
+2. Click **Actions** → **Add Knowledge**.
+3. Drag-and-drop or browse for files. Supported: PDF, TXT, MD, JSON, MP4. Per-file limit 5 MB. Up to 10 files per batch.
+4. The UI handles upload, storage, and index build. The user does not pick a bucket or provide credentials.
+5. Search results from `knowledgebase_search` become available once indexing completes.
+
+If a user asks you "how do I add a document to my knowledge base?", point them at this flow. Do not suggest `knowledgebase_trigger_index_creation` with an S3 URI — on cloud that path does not work.
+
+## Tracked gaps
+
+- **Upstream GitHub**: [yugabyte/meko-mcp-server#89](https://github.com/yugabyte/meko-mcp-server/issues/89) — disable or gate KB ingestion tools on cloud until the S3 path works end-to-end; return a clearer error than "not_available"; validate schemes at MCP-call time.
+- **Upstream Jira**: MEKO-96 (same issue, cross-linked).
+
+When either lands, the free-tier-gate responses above will change and this page should be revisited.
+
+## Response-shape reference (for agents that need to parse)
+
+### knowledgebase_search
+
+Populated KB:
+```
+{
+  "results": [
+    {
+      "content": "<chunk text>",
+      "similarity": 0.87,
+      "source_uri": "<original source URI>",
+      "document_name": "<file key>",
+      ...
+    },
+    ...
+  ],
+  "count": <int>
+}
+```
+
+Empty or nonexistent KB: `{"results": [], "count": 0}` (no error).
+
+### knowledgebase_trigger_index_creation (when not gated)
+
+Success shape:
+```
+{
+  "source_ids": ["<uuid>", ...],
+  "index_id": "<uuid>",
+  "index_name": "<name>",
+  "status": "build_triggered",
+  "message": "N source(s) registered, index '...' initialised, and build triggered."
+}
+```
+
+Do NOT retry this on failure. Each call creates a new index.
+
+### knowledgebase_add_source_to_index (when not gated)
+
+The historical `r_index_name` vs `r_index_id` parameter-mismatch bug is fixed on main (`src/tools.py` resolves `index_name → index_id` via `SELECT id FROM dist_rag.vector_indexes WHERE index_name = %s` before calling the dist_rag function). If `index_name` doesn't exist you get `{"error": "index_not_found", "detail": "No vector index found with name '...'"}`.
+
+Param is `source_uri` (URI string like `s3://bucket/path/`), **not** `source_id` as older docs may claim.
+
+### knowledgebase_check_index_status (when not gated)
+
+Returns `pipeline_details` (per-document status) and `pipeline_stats` (aggregated). Check `completion_rate_percent` in `pipeline_stats` and `document_status` in `pipeline_details`.
+
+Stuck at INIT for >5 minutes under self-hosted usually means the worker couldn't reach the bucket — check AWS credentials on the worker host, not the MCP server.
+
+## Prerequisites (self-hosted)
+
+- `dist_rag` PostgreSQL extension enabled on the target YugabyteDB database
+- Ingestion workers running (`yugabyte-db/python/ai/rag_agent/`)
+- AWS credentials on the worker host for any S3 bucket you intend to ingest from
+- `datapack_id` passed on every call when targeting a non-default datapack

package/assets/skills/meko-mcp-tools-desktop/references/tools-scope-permissions.md

@@ -0,0 +1,75 @@
+# Scope Permission Hierarchy and Per-Tool Requirements
+
+**WRONG:** `db_run_write_query(scope="read", ...)` → PermissionError: requires 'write' or higher.
+**WRONG:** `memory_delete_all(scope="write", ...)` → PermissionError: requires 'admin'.
+
+## Scope Hierarchy
+
+```
+admin  (level 2)  -- can do everything
+  ↓
+write  (level 1)  -- can read + write, not admin-only
+  ↓
+read   (level 0)  -- can only read
+```
+
+Higher scopes include lower scopes. An unrecognized scope string always fails.
+
+## Per-Tool Requirements
+
+### read scope (11 tools)
+
+| Tool | Group |
+|------|-------|
+| `db_summarize_database` | Database |
+| `db_run_read_only_query` | Database |
+| `knowledgebase_check_index_status` | RAG |
+| `memory_search` | Memory |
+| `memory_get_by_id` | Memory |
+| `memory_get_all` | Memory |
+| `flush_pending_memory_candidates` | Memory |
+| `conversation_get` | Conversation |
+| `conversation_list` | Conversation |
+| `datapack_list` | Datapack |
+| `datapack_describe` | Datapack |
+
+### write scope (11 tools)
+
+| Tool | Group |
+|------|-------|
+| `db_run_write_query` | Database |
+| `knowledgebase_trigger_index_creation` | RAG |
+| `knowledgebase_add_source_to_index` | RAG |
+| `memory_add` | Memory |
+| `memory_update` | Memory |
+| `memory_delete_by_id` | Memory |
+| `conversation_create` | Conversation |
+| `conversation_add_message` | Conversation |
+| `conversation_update` | Conversation |
+| `datapack_create` | Datapack |
+| `datapack_update` | Datapack |
+
+### admin scope (3 tools — destructive, irreversible)
+
+| Tool | Group |
+|------|-------|
+| `memory_delete_all` | Memory |
+| `conversation_delete` | Conversation |
+| `datapack_delete` | Datapack |
+
+Agent and knowledge-base lifecycle tools are not MCP-exposed — see "Platform capabilities NOT exposed via MCP" in `tools-overview.md`.
+
+## Best Practices
+
+1. **Use the minimum scope needed.** `scope="read"` for reads documents intent and prevents accidental writes.
+2. **The scope is a parameter, not an auth token.** When OAuth is enabled, the actual permission check happens against the OAuth token's claims. Both `scope` parameter and OAuth permissions must be satisfied.
+
+## Common Mistakes
+
+**WRONG:** `scope="all"` — there is no "all" scope. Valid values are exactly: `"read"`, `"write"`, `"admin"`. Nothing else.
+
+Other invalid values that will fail: `"readwrite"`, `"rw"`, `"full"`, `"any"`.
+
+The error `Insufficient scope: 'all'. This tool requires 'read' or higher.` means the scope string was not recognized as a valid value. Always use one of the three valid strings.
+
+**Default choice:** Start with `scope="read"` unless you specifically need to write or delete.