coalesce-transform-mcp 0.4.8 → 0.5.0-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +480 -211
- package/agents/lineage-analyst.agent.md +21 -0
- package/agents/pipeline-builder.agent.md +33 -0
- package/agents/run-operator.agent.md +28 -0
- package/agents/workspace-auditor.agent.md +23 -0
- package/dist/cache-dir.d.ts +1 -0
- package/dist/cache-dir.d.ts.map +1 -1
- package/dist/cache-dir.js +13 -1
- package/dist/cache-dir.js.map +1 -1
- package/dist/client.d.ts.map +1 -1
- package/dist/client.js +6 -13
- package/dist/client.js.map +1 -1
- package/dist/coalesce/api/jobs.d.ts +4 -0
- package/dist/coalesce/api/jobs.d.ts.map +1 -1
- package/dist/coalesce/api/jobs.js +4 -0
- package/dist/coalesce/api/jobs.js.map +1 -1
- package/dist/coalesce/api/runs.d.ts.map +1 -1
- package/dist/coalesce/api/runs.js +5 -2
- package/dist/coalesce/api/runs.js.map +1 -1
- package/dist/coalesce/run-schemas.d.ts +28 -31
- package/dist/coalesce/run-schemas.d.ts.map +1 -1
- package/dist/coalesce/run-schemas.js +5 -77
- package/dist/coalesce/run-schemas.js.map +1 -1
- package/dist/coalesce/tool-response.d.ts.map +1 -1
- package/dist/coalesce/tool-response.js +2 -2
- package/dist/coalesce/tool-response.js.map +1 -1
- package/dist/mcp/coa.d.ts +323 -0
- package/dist/mcp/coa.d.ts.map +1 -0
- package/dist/mcp/coa.js +529 -0
- package/dist/mcp/coa.js.map +1 -0
- package/dist/mcp/environments.d.ts.map +1 -1
- package/dist/mcp/environments.js +17 -2
- package/dist/mcp/environments.js.map +1 -1
- package/dist/mcp/git-accounts.d.ts.map +1 -1
- package/dist/mcp/git-accounts.js +24 -2
- package/dist/mcp/git-accounts.js.map +1 -1
- package/dist/mcp/jobs.d.ts.map +1 -1
- package/dist/mcp/jobs.js +22 -3
- package/dist/mcp/jobs.js.map +1 -1
- package/dist/mcp/lineage.d.ts.map +1 -1
- package/dist/mcp/lineage.js +53 -3
- package/dist/mcp/lineage.js.map +1 -1
- package/dist/mcp/nodes.d.ts.map +1 -1
- package/dist/mcp/nodes.js +22 -3
- package/dist/mcp/nodes.js.map +1 -1
- package/dist/mcp/pipelines.js +6 -6
- package/dist/mcp/pipelines.js.map +1 -1
- package/dist/mcp/projects.d.ts.map +1 -1
- package/dist/mcp/projects.js +34 -2
- package/dist/mcp/projects.js.map +1 -1
- package/dist/mcp/repo-node-types.js +4 -4
- package/dist/mcp/repo-node-types.js.map +1 -1
- package/dist/mcp/runs.d.ts.map +1 -1
- package/dist/mcp/runs.js +27 -4
- package/dist/mcp/runs.js.map +1 -1
- package/dist/mcp/setup.d.ts +4 -0
- package/dist/mcp/setup.d.ts.map +1 -0
- package/dist/mcp/setup.js +15 -0
- package/dist/mcp/setup.js.map +1 -0
- package/dist/mcp/subgraphs.d.ts.map +1 -1
- package/dist/mcp/subgraphs.js +28 -2
- package/dist/mcp/subgraphs.js.map +1 -1
- package/dist/mcp/tool-helpers.d.ts +43 -1
- package/dist/mcp/tool-helpers.d.ts.map +1 -1
- package/dist/mcp/tool-helpers.js +84 -2
- package/dist/mcp/tool-helpers.js.map +1 -1
- package/dist/mcp/users.d.ts.map +1 -1
- package/dist/mcp/users.js +54 -2
- package/dist/mcp/users.js.map +1 -1
- package/dist/prompts/index.d.ts.map +1 -1
- package/dist/prompts/index.js +82 -0
- package/dist/prompts/index.js.map +1 -1
- package/dist/resources/coa-describe.d.ts +12 -0
- package/dist/resources/coa-describe.d.ts.map +1 -0
- package/dist/resources/coa-describe.js +106 -0
- package/dist/resources/coa-describe.js.map +1 -0
- package/dist/resources/context/aggregation-patterns.md +2 -2
- package/dist/resources/context/ecosystem-boundaries.md +130 -0
- package/dist/resources/context/tool-usage.md +2 -2
- package/dist/resources/index.d.ts.map +1 -1
- package/dist/resources/index.js +9 -0
- package/dist/resources/index.js.map +1 -1
- package/dist/server.d.ts +1 -1
- package/dist/server.d.ts.map +1 -1
- package/dist/server.js +17 -0
- package/dist/server.js.map +1 -1
- package/dist/services/cache/snapshots.d.ts.map +1 -1
- package/dist/services/cache/snapshots.js +11 -10
- package/dist/services/cache/snapshots.js.map +1 -1
- package/dist/services/coa/describe.d.ts +49 -0
- package/dist/services/coa/describe.d.ts.map +1 -0
- package/dist/services/coa/describe.js +182 -0
- package/dist/services/coa/describe.js.map +1 -0
- package/dist/services/coa/preflight.d.ts +40 -0
- package/dist/services/coa/preflight.d.ts.map +1 -0
- package/dist/services/coa/preflight.js +461 -0
- package/dist/services/coa/preflight.js.map +1 -0
- package/dist/services/coa/project.d.ts +11 -0
- package/dist/services/coa/project.d.ts.map +1 -0
- package/dist/services/coa/project.js +35 -0
- package/dist/services/coa/project.js.map +1 -0
- package/dist/services/coa/redact.d.ts +22 -0
- package/dist/services/coa/redact.d.ts.map +1 -0
- package/dist/services/coa/redact.js +84 -0
- package/dist/services/coa/redact.js.map +1 -0
- package/dist/services/coa/resolver.d.ts +21 -0
- package/dist/services/coa/resolver.d.ts.map +1 -0
- package/dist/services/coa/resolver.js +90 -0
- package/dist/services/coa/resolver.js.map +1 -0
- package/dist/services/coa/runner.d.ts +30 -0
- package/dist/services/coa/runner.d.ts.map +1 -0
- package/dist/services/coa/runner.js +117 -0
- package/dist/services/coa/runner.js.map +1 -0
- package/dist/services/config/coa-config.d.ts +52 -0
- package/dist/services/config/coa-config.d.ts.map +1 -0
- package/dist/services/config/coa-config.js +159 -0
- package/dist/services/config/coa-config.js.map +1 -0
- package/dist/services/config/credentials.d.ts +37 -0
- package/dist/services/config/credentials.d.ts.map +1 -0
- package/dist/services/config/credentials.js +124 -0
- package/dist/services/config/credentials.js.map +1 -0
- package/dist/services/lineage/lineage-cache.d.ts.map +1 -1
- package/dist/services/lineage/lineage-cache.js +26 -7
- package/dist/services/lineage/lineage-cache.js.map +1 -1
- package/dist/services/lineage/lineage-propagation.d.ts +1 -0
- package/dist/services/lineage/lineage-propagation.d.ts.map +1 -1
- package/dist/services/lineage/lineage-propagation.js +139 -56
- package/dist/services/lineage/lineage-propagation.js.map +1 -1
- package/dist/services/pipelines/intent-resolution.d.ts.map +1 -1
- package/dist/services/pipelines/intent-resolution.js +7 -1
- package/dist/services/pipelines/intent-resolution.js.map +1 -1
- package/dist/services/pipelines/planning-types.d.ts +4 -4
- package/dist/services/repo/path.d.ts.map +1 -1
- package/dist/services/repo/path.js +6 -1
- package/dist/services/repo/path.js.map +1 -1
- package/dist/services/setup/diagnose.d.ts +131 -0
- package/dist/services/setup/diagnose.d.ts.map +1 -0
- package/dist/services/setup/diagnose.js +435 -0
- package/dist/services/setup/diagnose.js.map +1 -0
- package/dist/services/setup/hint.d.ts +15 -0
- package/dist/services/setup/hint.d.ts.map +1 -0
- package/dist/services/setup/hint.js +21 -0
- package/dist/services/setup/hint.js.map +1 -0
- package/dist/services/workspace/join-helpers.d.ts +9 -0
- package/dist/services/workspace/join-helpers.d.ts.map +1 -1
- package/dist/services/workspace/join-helpers.js +20 -2
- package/dist/services/workspace/join-helpers.js.map +1 -1
- package/dist/services/workspace/join-operations.d.ts.map +1 -1
- package/dist/services/workspace/join-operations.js +14 -2
- package/dist/services/workspace/join-operations.js.map +1 -1
- package/package.json +5 -3
package/README.md
CHANGED
|
@@ -1,41 +1,153 @@
|
|
|
1
1
|
# coalesce-transform-mcp
|
|
2
2
|
|
|
3
|
-
MCP server for
|
|
3
|
+
MCP server for [Coalesce](https://coalesce.io/). Connect AI assistants like Claude, Cursor, or Windsurf to Coalesce to manage nodes, pipelines, environments, jobs, and runs, and drive the local-first [`coa`](https://www.npmjs.com/package/@coalescesoftware/coa) CLI from the same server: validate a project, preview DDL/DML, plan a deployment, and apply it to a cloud environment. One install, two execution surfaces.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
- **Cloud REST tools** — build pipelines declaratively, edit node YAML, review lineage, run deployed jobs, audit documentation.
|
|
6
|
+
- **Local COA CLI tools** — validate projects before check-in, preview generated DDL/DML (`--dry-run`), iterate on V2 `.sql` node files, run `plan → deploy → refresh` cycles. COA is bundled — no separate install.
|
|
6
7
|
|
|
7
|
-
|
|
8
|
+
The two surfaces are orthogonal. Use both, one, or neither. Every destructive tool — on either surface — requires explicit confirmation before running. New? Run the `/coalesce-setup` prompt after install — it walks you through anything missing.
|
|
9
|
+
|
|
10
|
+
## I want to…
|
|
11
|
+
|
|
12
|
+
| Task | Jump to |
|
|
13
|
+
| ---- | ------- |
|
|
14
|
+
| Get running in 2 minutes | [Quick start](#quick-start) |
|
|
15
|
+
| Authenticate (env var or `~/.coa/config`) | [Credentials](#credentials) |
|
|
16
|
+
| Run against multiple Coalesce environments | [Multiple environments](#multiple-environments) |
|
|
17
|
+
| Lock prod down to read-only | [Safety model](#safety-model) |
|
|
18
|
+
| Use the `coa` CLI tools | [Using the COA CLI tools](#using-the-coa-cli-tools) |
|
|
19
|
+
| Try a prerelease build | [Prerelease channel](#prerelease-channel) |
|
|
20
|
+
| Debug "why isn't auth working?" | [Diagnosing setup](#diagnosing-setup) |
|
|
21
|
+
| Customize agent behavior | [Context skills](#context-skills) |
|
|
22
|
+
| Find a specific tool | [Tool reference](#tool-reference) |
|
|
23
|
+
| Query warehouse data (add companion MCP) | [Companion MCPs](#companion-mcps) |
|
|
24
|
+
|
|
25
|
+
## Quick start
|
|
26
|
+
|
|
27
|
+
**Requirements:**
|
|
28
|
+
|
|
29
|
+
- [Node.js](https://nodejs.org/) 22+
|
|
30
|
+
- A [Coalesce](https://coalesce.io/) account with a workspace
|
|
31
|
+
- An MCP-compatible AI client (Claude Code, Claude Desktop, Cursor, Windsurf)
|
|
32
|
+
- Snowflake credentials — only if you plan to use run tools or `coa_create`/`coa_run` (see [Credentials](#credentials))
|
|
33
|
+
- Install footprint is ~76 MB unpacked (the bundled `@coalescesoftware/coa` CLI ships its own runtime; the MCP tarball itself is under 1 MB)
|
|
34
|
+
|
|
35
|
+
**1. Clone your project**
|
|
36
|
+
|
|
37
|
+
If your team already has a Coalesce project in Git, clone it locally — the bundled `coa` CLI operates on a project directory, so most local create/run tools require one on disk:
|
|
8
38
|
|
|
9
39
|
```bash
|
|
10
|
-
|
|
40
|
+
git clone <your-coalesce-project-repo-url>
|
|
41
|
+
cd my-project
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**Don't have a Git-linked project yet?** In the Coalesce UI, open your workspace → **Settings → Git** and connect a repo (or create one via your Git provider and paste the URL). Coalesce will commit the project skeleton on first push; clone that repo locally once it's populated.
|
|
45
|
+
|
|
46
|
+
A Coalesce project has this shape:
|
|
47
|
+
|
|
48
|
+
```text
|
|
49
|
+
my-project/
|
|
50
|
+
├── data.yml # Root metadata (fileVersion, platformKind)
|
|
51
|
+
├── locations.yml # Storage location manifest
|
|
52
|
+
├── nodes/ # Pipeline nodes (.yml for V1, .sql for V2)
|
|
53
|
+
├── nodeTypes/ # Node type definitions with templates
|
|
54
|
+
├── environments/ # Environment configs with storage mappings
|
|
55
|
+
├── macros/ # Reusable SQL macros
|
|
56
|
+
├── jobs/ # Job definitions
|
|
57
|
+
└── subgraphs/ # Subgraph definitions
|
|
11
58
|
```
|
|
12
59
|
|
|
13
|
-
|
|
60
|
+
> **V1 vs V2** — the format is pinned by `fileVersion` in `data.yml`. **V1** (`fileVersion: 1` or `2`) stores each node as a single YAML file with columns, transforms, and config inline. **V2** (`fileVersion: 3`) is SQL-first: the node body lives in a `.sql` file using `@id` / `@nodeType` annotations and `{{ ref() }}` references, with YAML retained for config. New projects default to V2; existing V1 projects keep working unchanged.
|
|
61
|
+
|
|
62
|
+
Point the MCP at this directory by setting `repoPath` in `~/.coa/config` or `COALESCE_REPO_PATH` in your env block.
|
|
63
|
+
|
|
64
|
+
### Create `workspaces.yml`
|
|
65
|
+
|
|
66
|
+
This file is **required** for `coa_create` / `coa_run` and their dry-run variants. It maps each storage location declared in `locations.yml` to a physical database + schema for local development. It's typically gitignored (per-developer), so cloning the project does not give it to you — you have to create it.
|
|
67
|
+
|
|
68
|
+
The `/coalesce-setup` prompt detects a missing `workspaces.yml` and walks you through it. If you'd rather do it directly, pick one of:
|
|
69
|
+
|
|
70
|
+
- **Let COA bootstrap it** (easiest): from the project root, run
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
npx @coalescesoftware/coa doctor --fix
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Or from your MCP client, call the `coa_bootstrap_workspaces` tool (requires `confirmed: true`) which runs the same command.
|
|
14
77
|
|
|
15
|
-
|
|
78
|
+
> **⚠️ The generated file contains placeholder values.** `coa doctor --fix` seeds `database`/`schema` with defaults that won't match your real warehouse. Open the file and replace every placeholder before running `coa_create` / `coa_run` — otherwise the generated DDL/DML will target the wrong (or non-existent) database.
|
|
79
|
+
- **Hand-write it.** Authoritative schema (from `coa describe schema workspaces` — no top-level wrapper, no `fileVersion`):
|
|
16
80
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
81
|
+
```yaml
|
|
82
|
+
# workspaces.yml — keys are workspace names; `dev` is the default if --workspace is omitted
|
|
83
|
+
dev:
|
|
84
|
+
connection: snowflake # required — name of the connection block COA should use
|
|
85
|
+
locations: # optional — one entry per storage location name from locations.yml
|
|
86
|
+
SRC_INGEST_TASTY_BITES:
|
|
87
|
+
database: JESSE_DEV # required
|
|
88
|
+
schema: INGEST_TASTY_BITES # required
|
|
89
|
+
ETL_STAGE:
|
|
90
|
+
database: JESSE_DEV
|
|
91
|
+
schema: ETL_STAGE
|
|
92
|
+
ANALYTICS:
|
|
93
|
+
database: JESSE_DEV
|
|
94
|
+
schema: ANALYTICS
|
|
95
|
+
```
|
|
23
96
|
|
|
24
|
-
|
|
97
|
+
Verify with `coa_doctor` (or `npx @coalescesoftware/coa doctor`) — it checks `data.yml`, `workspaces.yml`, credentials, and warehouse connectivity end to end.
|
|
98
|
+
|
|
99
|
+
**2. Pick an auth path:**
|
|
100
|
+
|
|
101
|
+
- **Option A — env var** (simplest for first-time MCP users). Generate a `COALESCE_ACCESS_TOKEN` from Coalesce → Deploy → User Settings.
|
|
102
|
+
- **Option B — reuse `~/.coa/config`** (best if you already use the `coa` CLI). The server reads the same file — nothing to duplicate. Skip to step 3 and drop the `env` block below. See [Credentials](#credentials) for the schema.
|
|
103
|
+
|
|
104
|
+
When both sources set a field, the env var wins.
|
|
105
|
+
|
|
106
|
+
**3. Add the server to your MCP client config.** Pick your client below and paste the block into the indicated file. Replace `<YOUR_TOKEN>` with a real token only if your client does not support env var substitution (noted per client).
|
|
107
|
+
|
|
108
|
+
#### Claude Code
|
|
109
|
+
|
|
110
|
+
File: `.mcp.json` in project root (or `~/.claude.json` for global).
|
|
25
111
|
|
|
26
112
|
```json
|
|
27
113
|
{
|
|
28
|
-
"
|
|
29
|
-
"
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
"
|
|
114
|
+
"mcpServers": {
|
|
115
|
+
"coalesce-transform": {
|
|
116
|
+
"command": "npx",
|
|
117
|
+
"args": ["coalesce-transform-mcp"],
|
|
118
|
+
"env": {
|
|
119
|
+
"COALESCE_ACCESS_TOKEN": "${COALESCE_ACCESS_TOKEN}"
|
|
120
|
+
}
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Claude Code expands `${VAR}` from your shell env at load time. Omit the `env` block entirely if you're using `~/.coa/config` (Option B).
|
|
127
|
+
|
|
128
|
+
#### Claude Desktop
|
|
129
|
+
|
|
130
|
+
File: `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows).
|
|
131
|
+
|
|
132
|
+
```json
|
|
133
|
+
{
|
|
134
|
+
"mcpServers": {
|
|
135
|
+
"coalesce-transform": {
|
|
136
|
+
"command": "npx",
|
|
137
|
+
"args": ["coalesce-transform-mcp"],
|
|
138
|
+
"env": {
|
|
139
|
+
"COALESCE_ACCESS_TOKEN": "<YOUR_TOKEN>"
|
|
140
|
+
}
|
|
33
141
|
}
|
|
34
142
|
}
|
|
35
143
|
}
|
|
36
144
|
```
|
|
37
145
|
|
|
38
|
-
|
|
146
|
+
Claude Desktop does **not** expand `${VAR}` — paste the literal token, or drop the `env` block and use `~/.coa/config` (Option B) so nothing sensitive lives in this file.
|
|
147
|
+
|
|
148
|
+
#### Cursor
|
|
149
|
+
|
|
150
|
+
File: `.cursor/mcp.json` in project root (or `~/.cursor/mcp.json` for global).
|
|
39
151
|
|
|
40
152
|
```json
|
|
41
153
|
{
|
|
@@ -44,36 +156,86 @@ Generate a token from the Deploy tab in your Coalesce workspace ([docs](https://
|
|
|
44
156
|
"command": "npx",
|
|
45
157
|
"args": ["coalesce-transform-mcp"],
|
|
46
158
|
"env": {
|
|
47
|
-
"COALESCE_ACCESS_TOKEN": "
|
|
159
|
+
"COALESCE_ACCESS_TOKEN": "<YOUR_TOKEN>"
|
|
48
160
|
}
|
|
49
161
|
}
|
|
50
162
|
}
|
|
51
163
|
}
|
|
52
164
|
```
|
|
53
165
|
|
|
54
|
-
|
|
166
|
+
Cursor does **not** expand `${VAR}` — paste the literal token, or drop the `env` block and use `~/.coa/config` (Option B).
|
|
167
|
+
|
|
168
|
+
#### Windsurf
|
|
169
|
+
|
|
170
|
+
File: `~/.codeium/windsurf/mcp_config.json`.
|
|
171
|
+
|
|
172
|
+
```json
|
|
173
|
+
{
|
|
174
|
+
"mcpServers": {
|
|
175
|
+
"coalesce-transform": {
|
|
176
|
+
"command": "npx",
|
|
177
|
+
"args": ["coalesce-transform-mcp"],
|
|
178
|
+
"env": {
|
|
179
|
+
"COALESCE_ACCESS_TOKEN": "<YOUR_TOKEN>"
|
|
180
|
+
}
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
Windsurf does **not** expand `${VAR}` — paste the literal token, or drop the `env` block and use `~/.coa/config` (Option B).
|
|
187
|
+
|
|
188
|
+
**4. Restart your client**, then run the `/coalesce-setup` prompt to verify everything is wired up.
|
|
189
|
+
|
|
190
|
+
> **Never hardcode credentials in git-tracked config files.** Only Claude Code's `.mcp.json` expands `${VAR}` from your shell env. For any other client, keep secrets in `~/.coa/config` (Option B) or a secrets manager your client integrates with — don't commit literals into these JSON files.
|
|
55
191
|
|
|
56
|
-
|
|
192
|
+
If you have more than one Coalesce environment to manage, see [Multiple environments](#multiple-environments).
|
|
57
193
|
|
|
58
|
-
##
|
|
194
|
+
## Configuration
|
|
59
195
|
|
|
60
|
-
|
|
61
|
-
- A [Coalesce](https://coalesce.io/) account with a workspace and access token
|
|
62
|
-
- An MCP-compatible AI client
|
|
63
|
-
- **For run tools only:** Snowflake key pair authentication (see below)
|
|
196
|
+
### Credentials
|
|
64
197
|
|
|
65
|
-
|
|
198
|
+
The server reads credentials from two sources and merges them with **env-wins precedence** — a matching env var always overrides the profile value, so you can pin a single field per session without editing the config file. Call `diagnose_setup` to see which source supplied each value.
|
|
66
199
|
|
|
67
|
-
|
|
200
|
+
#### Source 1: `~/.coa/config` (shared with the `coa` CLI)
|
|
201
|
+
|
|
202
|
+
COA stores credentials in a standard INI file. You create it by hand, or let `coa` write it as you use the CLI. The MCP reads the profile selected by `COALESCE_PROFILE` (default `[default]`) and maps the keys below onto their matching env vars.
|
|
203
|
+
|
|
204
|
+
```ini
|
|
205
|
+
[default]
|
|
206
|
+
token=<your-coalesce-refresh-token>
|
|
207
|
+
domain=https://your-org.app.coalescesoftware.io
|
|
208
|
+
snowflakeAccount=<your-snowflake-account> # e.g., abc12345.us-east-1 — required by coa CLI
|
|
209
|
+
snowflakeUsername=YOUR_USER
|
|
210
|
+
snowflakeRole=YOUR_ROLE
|
|
211
|
+
snowflakeWarehouse=YOUR_WAREHOUSE
|
|
212
|
+
snowflakeKeyPairKey=/Users/you/.coa/rsa_key.p8 # see deprecation note below
|
|
213
|
+
snowflakeAuthType=KeyPair
|
|
214
|
+
orgID=<your-org-id> # optional; fallback for cancel-run
|
|
215
|
+
repoPath=/Users/you/path/to/repo # optional; for repo-backed tools
|
|
216
|
+
cacheDir=/Users/you/.coa/cache # optional; per-profile cache isolation
|
|
217
|
+
|
|
218
|
+
[staging]
|
|
219
|
+
# …additional profiles; select with COALESCE_PROFILE
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
> **`snowflakeKeyPairKey` deprecation loop (known quirk).** The `coa` CLI currently emits a deprecation warning on `snowflakeKeyPairKey` and points you at `snowflakeKeyPairPath`, but `snowflakeKeyPairPath` does not yet accept a file path value. Until the upstream fix ships, keep using `snowflakeKeyPairKey=` (the name shown in `coa describe config`) — the deprecation warning is harmless.
|
|
223
|
+
|
|
224
|
+
Key mapping: `token` ↔ `COALESCE_ACCESS_TOKEN`, `domain` ↔ `COALESCE_BASE_URL`, each `snowflake*` key ↔ its corresponding `SNOWFLAKE_*` env var, `orgID` ↔ `COALESCE_ORG_ID`, `repoPath` ↔ `COALESCE_REPO_PATH`, `cacheDir` ↔ `COALESCE_CACHE_DIR`. `snowflakeAuthType` is read by COA itself (not mapped to an env var) — include it when you're using key-pair auth. `orgID`, `repoPath`, and `cacheDir` are MCP-specific (the COA CLI ignores them). Only the fields the MCP needs are shown above — COA's config supports many more (run `npx @coalescesoftware/coa describe config` for the authoritative reference). Unknown keys are ignored.
|
|
225
|
+
|
|
226
|
+
If `~/.coa/config` doesn't exist the server runs env-only — startup never fails on a missing or malformed profile file; it just logs a stderr warning.
|
|
227
|
+
|
|
228
|
+
#### Source 2: env vars in your MCP config
|
|
68
229
|
|
|
69
230
|
<!-- ENV_METADATA_CORE_TABLE_START -->
|
|
70
231
|
| Variable | Description | Default |
|
|
71
232
|
| -------- | -------- | -------- |
|
|
72
|
-
| `COALESCE_ACCESS_TOKEN` |
|
|
233
|
+
| `COALESCE_ACCESS_TOKEN` | Bearer token from the Coalesce Deploy tab. Optional when `~/.coa/config` provides a `token`. | — |
|
|
234
|
+
| `COALESCE_PROFILE` | Selects which `~/.coa/config` profile to load. | `default` |
|
|
73
235
|
| `COALESCE_BASE_URL` | Region-specific base URL. | `https://app.coalescesoftware.io (US)` |
|
|
74
|
-
| `COALESCE_ORG_ID` | Fallback org ID for cancel-run. | — |
|
|
75
|
-
| `COALESCE_REPO_PATH` | Local repo root for repo-backed tools and pipeline planning. | — |
|
|
76
|
-
| `COALESCE_CACHE_DIR` | Base directory for the local data cache. When set, cache files are written here instead of the working directory. | — |
|
|
236
|
+
| `COALESCE_ORG_ID` | Fallback org ID for cancel-run. Also readable from `orgID` in the active ~/.coa/config profile. | — |
|
|
237
|
+
| `COALESCE_REPO_PATH` | Local repo root for repo-backed tools and pipeline planning. Also readable from `repoPath` in the active ~/.coa/config profile. | — |
|
|
238
|
+
| `COALESCE_CACHE_DIR` | Base directory for the local data cache. When set, cache files are written here instead of the working directory. Also readable from `cacheDir` in the active ~/.coa/config profile. | — |
|
|
77
239
|
| `COALESCE_MCP_AUTO_CACHE_MAX_BYTES` | JSON size threshold before auto-caching to disk. | `32768` |
|
|
78
240
|
| `COALESCE_MCP_LINEAGE_TTL_MS` | In-memory lineage cache TTL in milliseconds. | `1800000` |
|
|
79
241
|
| `COALESCE_MCP_MAX_REQUEST_BODY_BYTES` | Max outbound API request body size. | `524288` |
|
|
@@ -81,13 +243,14 @@ Only `COALESCE_ACCESS_TOKEN` is required. Everything else is optional.
|
|
|
81
243
|
| `COALESCE_MCP_SKILLS_DIR` | Directory for customizable AI skill resources. When set, reads context resources from this directory and seeds defaults on first run. Users can augment or override any skill. | — |
|
|
82
244
|
<!-- ENV_METADATA_CORE_TABLE_END -->
|
|
83
245
|
|
|
84
|
-
|
|
246
|
+
#### Snowflake credentials (run tools only)
|
|
85
247
|
|
|
86
|
-
|
|
248
|
+
`start_run`, `retry_run`, `run_and_wait`, `retry_and_wait`, and the warehouse-touching COA tools (`coa_create`, `coa_run`) need Snowflake credentials. These normally come from `~/.coa/config`. Override any field via env var:
|
|
87
249
|
|
|
88
250
|
<!-- ENV_METADATA_SNOWFLAKE_TABLE_START -->
|
|
89
251
|
| Variable | Required | Description |
|
|
90
252
|
| -------- | -------- | -------- |
|
|
253
|
+
| `SNOWFLAKE_ACCOUNT` | Yes | Snowflake account identifier (e.g., `abc12345.us-east-1`). Required by the local `coa` CLI and `coa doctor`; not used by the MCP's REST run path. |
|
|
91
254
|
| `SNOWFLAKE_USERNAME` | Yes | Snowflake account username |
|
|
92
255
|
| `SNOWFLAKE_KEY_PAIR_KEY` | No | Path to PEM-encoded private key (required if SNOWFLAKE_PAT not set) |
|
|
93
256
|
| `SNOWFLAKE_PAT` | No | Snowflake Programmatic Access Token (alternative to key pair) |
|
|
@@ -96,23 +259,11 @@ Required for `start_run`, `retry_run`, `run_and_wait`, and `retry_and_wait`. The
|
|
|
96
259
|
| `SNOWFLAKE_ROLE` | Yes | Snowflake user role |
|
|
97
260
|
<!-- ENV_METADATA_SNOWFLAKE_TABLE_END -->
|
|
98
261
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
**`~/.zshrc`:**
|
|
262
|
+
"Required" means one of env OR the matching `~/.coa/config` field must supply the value. **`SNOWFLAKE_PAT` is env-only** — COA's config uses `snowflakePassword` for Basic auth (a different concept), which this server deliberately doesn't read.
|
|
102
263
|
|
|
103
|
-
|
|
104
|
-
export COALESCE_ACCESS_TOKEN="your-token-here"
|
|
105
|
-
export COALESCE_BASE_URL="https://app.eu.coalescesoftware.io"
|
|
106
|
-
export COALESCE_REPO_PATH="/path/to/local/coalesce-repo"
|
|
107
|
-
export SNOWFLAKE_USERNAME="your-username"
|
|
108
|
-
export SNOWFLAKE_KEY_PAIR_KEY="/path/to/snowflake_key.pem" # Option A: Key Pair
|
|
109
|
-
export SNOWFLAKE_KEY_PAIR_PASS="your-passphrase" # (only if key is encrypted)
|
|
110
|
-
export SNOWFLAKE_PAT="your-programmatic-access-token" # Option B: PAT (if both set, Key Pair wins)
|
|
111
|
-
export SNOWFLAKE_WAREHOUSE="your-warehouse"
|
|
112
|
-
export SNOWFLAKE_ROLE="your-role"
|
|
113
|
-
```
|
|
264
|
+
#### Field-level overrides
|
|
114
265
|
|
|
115
|
-
|
|
266
|
+
To pin a profile but override one field without editing the config file:
|
|
116
267
|
|
|
117
268
|
```json
|
|
118
269
|
{
|
|
@@ -120,49 +271,144 @@ export SNOWFLAKE_ROLE="your-role"
|
|
|
120
271
|
"command": "npx",
|
|
121
272
|
"args": ["coalesce-transform-mcp"],
|
|
122
273
|
"env": {
|
|
123
|
-
"
|
|
124
|
-
"
|
|
125
|
-
"COALESCE_REPO_PATH": "${COALESCE_REPO_PATH}",
|
|
126
|
-
"SNOWFLAKE_USERNAME": "${SNOWFLAKE_USERNAME}",
|
|
127
|
-
"SNOWFLAKE_KEY_PAIR_KEY": "${SNOWFLAKE_KEY_PAIR_KEY}",
|
|
128
|
-
"SNOWFLAKE_KEY_PAIR_PASS": "${SNOWFLAKE_KEY_PAIR_PASS}",
|
|
129
|
-
"SNOWFLAKE_PAT": "${SNOWFLAKE_PAT}",
|
|
130
|
-
"SNOWFLAKE_WAREHOUSE": "${SNOWFLAKE_WAREHOUSE}",
|
|
131
|
-
"SNOWFLAKE_ROLE": "${SNOWFLAKE_ROLE}"
|
|
274
|
+
"COALESCE_PROFILE": "staging",
|
|
275
|
+
"SNOWFLAKE_ROLE": "TRANSFORMER_ADMIN"
|
|
132
276
|
}
|
|
133
277
|
}
|
|
134
278
|
}
|
|
135
279
|
```
|
|
136
280
|
|
|
137
|
-
|
|
281
|
+
Reads: "use the `[staging]` profile, but override its `snowflakeRole`."
|
|
138
282
|
|
|
139
|
-
|
|
283
|
+
### Multiple environments
|
|
140
284
|
|
|
141
|
-
|
|
285
|
+
If you work across several Coalesce environments (dev/staging/prod, or multiple orgs), register the package once per profile under distinct server names:
|
|
142
286
|
|
|
143
|
-
|
|
287
|
+
```json
|
|
288
|
+
{
|
|
289
|
+
"mcpServers": {
|
|
290
|
+
"coalesce-prod": {
|
|
291
|
+
"command": "npx",
|
|
292
|
+
"args": ["coalesce-transform-mcp"],
|
|
293
|
+
"env": {
|
|
294
|
+
"COALESCE_PROFILE": "prod",
|
|
295
|
+
"COALESCE_MCP_READ_ONLY": "true"
|
|
296
|
+
}
|
|
297
|
+
},
|
|
298
|
+
"coalesce-dev": {
|
|
299
|
+
"command": "npx",
|
|
300
|
+
"args": ["coalesce-transform-mcp"],
|
|
301
|
+
"env": { "COALESCE_PROFILE": "dev" }
|
|
302
|
+
}
|
|
303
|
+
}
|
|
304
|
+
}
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
Why this pattern:
|
|
308
|
+
|
|
309
|
+
- **Namespaced tools.** The client surfaces `coalesce-prod__*` vs `coalesce-dev__*`, so an agent can't accidentally mutate the wrong environment.
|
|
310
|
+
- **Per-environment safety.** Pair prod with `COALESCE_MCP_READ_ONLY=true` to hide every write tool on that server while leaving dev fully writable.
|
|
311
|
+
- **No per-call profile juggling.** Each server is pinned at startup.
|
|
312
|
+
|
|
313
|
+
Skip this pattern if you only use one environment — a single registration is simpler. For 2–3 environments it's worth the extra config; beyond that, each server is a separate Node process, so consider whether you actually need them all loaded at once.
|
|
314
|
+
|
|
315
|
+
### Safety model
|
|
316
|
+
|
|
317
|
+
Three layers prevent destructive surprises:
|
|
318
|
+
|
|
319
|
+
1. **Tool annotations.** Every tool carries MCP annotations (`readOnlyHint`, `destructiveHint`, `idempotentHint`). Clients that respect them can filter proactively. The ⚠️ marker in [Tool reference](#tool-reference) marks `destructiveHint: true` tools.
|
|
320
|
+
2. **`COALESCE_MCP_READ_ONLY=true`** hides all write/mutation tools at server startup. Only read, list, search, cache, analyze, review, diagnose, and plan tools are registered. Use it for audits, agent sandboxes, or pairing with a prod profile (see [Multiple environments](#multiple-environments)).
|
|
321
|
+
3. **Explicit confirmation for destructive ops.** Tools marked destructive require `confirmed: true`. When the MCP client supports elicitation, the server prompts interactively; otherwise it returns a `STOP_AND_CONFIRM` response the agent must surface before retrying with `confirmed: true`. Applies to: `delete_*`, `propagate_column_change`, `cancel_run`, `clear_data_cache`, `coa_create`, `coa_run`, `coa_deploy`, `coa_refresh`.
|
|
322
|
+
|
|
323
|
+
**COA preflight.** Local COA write tools run preflight validation before shelling out. Errors block execution; warnings pass through in the tool response as `preflightWarnings` so agents can surface them.
|
|
324
|
+
|
|
325
|
+
| Code | Level | What it catches |
|
|
326
|
+
| ---- | ----- | --------------- |
|
|
327
|
+
| `SQL_DOUBLE_QUOTED_REF` | error | `.sql` nodes using `ref("…")` — silently returns `UNKNOWN` columns; must be single-quoted |
|
|
328
|
+
| `WORKSPACES_YML_MISSING` | error | `workspaces.yml` not in project root — required for local create/run |
|
|
329
|
+
| `SELECTOR_COMBINED_OR` | error | `{ A \|\| B }` selector form — matches zero nodes; must be `{ A } \|\| { B }` |
|
|
330
|
+
| `SQL_LITERAL_UNION_ALL` | warning | Literal `UNION ALL` in a V2 `.sql` node — silently dropped by the V2 parser; use `insertStrategy: UNION ALL` instead |
|
|
331
|
+
| `DATA_YML_UNEXPECTED_FILEVERSION` | warning | `data.yml` missing or not `fileVersion: 3` |
|
|
332
|
+
| `DATA_YML_NO_FILEVERSION` | warning | `data.yml` has no `fileVersion` field |
|
|
333
|
+
|
|
334
|
+
### Using the COA CLI tools
|
|
335
|
+
|
|
336
|
+
COA is bundled — no extra install. Usage notes:
|
|
337
|
+
|
|
338
|
+
- **Local commands** (`coa_doctor`, `coa_validate`, `coa_dry_run_create`, `coa_dry_run_run`, `coa_create`, `coa_run`, `coa_plan`) need a COA project directory (one that contains `data.yml`). Pass the path via the `projectPath` tool argument.
|
|
339
|
+
- **Cloud commands** (`coa_list_environments`, `coa_list_environment_nodes`, `coa_list_runs`, `coa_deploy`, `coa_refresh`) read credentials from `~/.coa/config` — the same file the MCP uses. Populate it once and both surfaces agree.
|
|
340
|
+
- **Profile resolution.** Cloud tools accept an optional `profile` arg. When omitted, they fall back to `COALESCE_PROFILE`, then to COA's own `[default]` — so you don't have to pass it on every call.
|
|
341
|
+
- **Warehouse-touching commands** (`coa_create`, `coa_run`) need a valid `workspaces.yml` in the project root with storage-location mappings. Preflight catches a missing file before execution.
|
|
342
|
+
|
|
343
|
+
### Prerelease channel
|
|
344
|
+
|
|
345
|
+
Prerelease builds publish to `@alpha` while `@latest` stays on stable. Point `npx` at the alpha channel:
|
|
346
|
+
|
|
347
|
+
```json
|
|
348
|
+
{
|
|
349
|
+
"coalesce-transform": {
|
|
350
|
+
"command": "npx",
|
|
351
|
+
"args": ["coalesce-transform-mcp@alpha"]
|
|
352
|
+
}
|
|
353
|
+
}
|
|
354
|
+
```
|
|
355
|
+
|
|
356
|
+
Restart your MCP client after changing the config so `npx` re-resolves. To pin an exact prerelease rather than whatever `@alpha` resolves to today, replace `@alpha` with the full version, e.g. `coalesce-transform-mcp@0.5.0-alpha.2`. If `npx` serves a stale cached copy when `@alpha` advances, force a fresh fetch with `npx -y coalesce-transform-mcp@alpha`.
|
|
357
|
+
|
|
358
|
+
To run alpha and stable side-by-side, register both under different server names (e.g. `coalesce-transform` for stable and `coalesce-transform-alpha` for the prerelease).
|
|
359
|
+
|
|
360
|
+
### Diagnosing setup
|
|
361
|
+
|
|
362
|
+
`diagnose_setup` is a stateless probe that reports which first-time-setup pieces are configured: access token, Snowflake credentials, `~/.coa/config` profile, local repo path, and a best-effort `coa doctor` check. It returns a structured report plus ordered `nextSteps` and per-field `source` markers (`env`, `profile:<name>`, or `missing`).
|
|
363
|
+
|
|
364
|
+
It pairs with the `/coalesce-setup` MCP prompt, which walks a user through any remaining gaps. Run it any time something isn't working the way you expect.
|
|
365
|
+
|
|
366
|
+
## Companion MCPs
|
|
367
|
+
|
|
368
|
+
This server manages Coalesce node definitions — **not** live warehouse data. For Snowflake data questions (tables, schemas, row counts, sample data, permissions), add [Cortex Code](https://ai.snowflake.com) as a companion MCP server. The agent will route Snowflake questions to cortex and node/pipeline questions to Coalesce.
|
|
369
|
+
|
|
370
|
+
```bash
|
|
371
|
+
curl -LsS https://ai.snowflake.com/static/cc-scripts/install.sh | sh
|
|
372
|
+
cortex connections # interactive connection setup
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
```json
|
|
376
|
+
{
|
|
377
|
+
"cortex": {
|
|
378
|
+
"command": "cortex",
|
|
379
|
+
"args": ["--mcp-server"]
|
|
380
|
+
}
|
|
381
|
+
}
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
## Resources
|
|
385
|
+
|
|
386
|
+
Resources are read-only context documents exposed via MCP that clients can pull into their prompts on demand. Two families.
|
|
387
|
+
|
|
388
|
+
### Context skills
|
|
389
|
+
|
|
390
|
+
24 curated markdown resources under `coalesce://context/*` guide how agents interact with the server — SQL conventions per warehouse, node-type selection, pipeline workflows, lineage/impact guidance. Set `COALESCE_MCP_SKILLS_DIR` to make them editable on disk:
|
|
144
391
|
|
|
145
392
|
```bash
|
|
146
393
|
export COALESCE_MCP_SKILLS_DIR="/path/to/my-skills"
|
|
147
394
|
```
|
|
148
395
|
|
|
149
|
-
On first run
|
|
396
|
+
On first run the server seeds the directory with two files per skill:
|
|
150
397
|
|
|
151
398
|
- `coalesce_skills.<name>.md` — the default skill content (editable)
|
|
152
399
|
- `user_skills.<name>.md` — your customization file (starts as an inactive stub with instructions)
|
|
153
400
|
|
|
154
|
-
### How customization works
|
|
155
|
-
|
|
156
401
|
Each resource resolves using this priority:
|
|
157
402
|
|
|
158
403
|
1. **Override** — `user_skills.<name>.md` starts with `<!-- OVERRIDE -->` → only the user file is served
|
|
159
404
|
2. **Augment** — `user_skills.<name>.md` has custom content (remove the `<!-- STUB -->` line first) → default + user content are concatenated
|
|
160
405
|
3. **Default** — `user_skills.<name>.md` is missing, empty, or still has the seeded stub → default skill content is served
|
|
161
|
-
4. **Disabled** — both files deleted → empty content is served
|
|
406
|
+
4. **Disabled** — both files deleted → empty content is served
|
|
162
407
|
|
|
163
408
|
Seeding is idempotent — it never overwrites files you've already modified.
|
|
164
409
|
|
|
165
|
-
|
|
410
|
+
<details>
|
|
411
|
+
<summary><strong>All context skills (24)</strong></summary>
|
|
166
412
|
|
|
167
413
|
| Skill | File | Description |
|
|
168
414
|
| ----- | ---- | ----------- |
|
|
@@ -189,204 +435,227 @@ Seeding is idempotent — it never overwrites files you've already modified.
|
|
|
189
435
|
| Run Diagnostics Guide | `run-diagnostics-guide` | Using `diagnose_run_failure` to analyze failed runs and determine fixes |
|
|
190
436
|
| Pipeline Review Guide | `pipeline-review-guide` | Using `review_pipeline` for pipeline analysis and optimization |
|
|
191
437
|
| Pipeline Workshop Guide | `pipeline-workshop-guide` | Using pipeline workshop tools for iterative, conversational pipeline building |
|
|
438
|
+
| Ecosystem Boundaries | `ecosystem-boundaries` | Scope of this MCP vs adjacent data engineering MCPs (Snowflake, Fivetran, dbt, Catalog) |
|
|
192
439
|
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
⚠️ = Destructive operation
|
|
440
|
+
</details>
|
|
196
441
|
|
|
197
|
-
###
|
|
442
|
+
### COA describe topics
|
|
198
443
|
|
|
199
|
-
|
|
444
|
+
10 resources under `coalesce://coa/describe/*` surface the bundled COA CLI's self-describing documentation. Content is fetched from `coa describe <topic>` on first access and cached to disk, keyed by the pinned COA version — agents always see docs that match the CLI they're driving. Topics: `overview`, `commands`, `selectors`, `schemas`, `workflow`, `structure`, `concepts`, `sql-format`, `node-types`, `config`.
|
|
200
445
|
|
|
201
|
-
|
|
446
|
+
For parameterized topics (`command <name>`, `schema <type>`), use the `coa_describe` tool with a `subtopic` argument.
|
|
202
447
|
|
|
203
|
-
|
|
204
|
-
- `get_environment` - Get details of a specific environment
|
|
205
|
-
- `create_environment` - Create a new environment within a project
|
|
206
|
-
- `delete_environment` - Delete an environment ⚠️
|
|
448
|
+
## Tool reference
|
|
207
449
|
|
|
208
|
-
|
|
450
|
+
⚠️ = Destructive (requires `confirmed: true`). 🧰 = Runs bundled `coa` CLI.
|
|
209
451
|
|
|
210
|
-
|
|
211
|
-
|
|
452
|
+
<details>
|
|
453
|
+
<summary><strong>Cloud REST tools (49)</strong> — Coalesce platform resources via the Deploy API</summary>
|
|
212
454
|
|
|
213
|
-
|
|
455
|
+
### Environments
|
|
214
456
|
|
|
215
|
-
- `
|
|
216
|
-
- `
|
|
217
|
-
- `
|
|
218
|
-
- `
|
|
219
|
-
- `set_workspace_node` - Replace a workspace node with a full body
|
|
220
|
-
- `update_workspace_node` - Safely update selected fields of a workspace node
|
|
221
|
-
- `delete_workspace_node` - Delete a node from a workspace ⚠️
|
|
457
|
+
- `list_environments` — List all available environments
|
|
458
|
+
- `get_environment` — Get details of a specific environment
|
|
459
|
+
- `create_environment` — Create a new environment within a project
|
|
460
|
+
- `delete_environment` — Delete an environment ⚠️
|
|
222
461
|
|
|
223
|
-
|
|
462
|
+
### Workspaces
|
|
224
463
|
|
|
225
|
-
- `
|
|
226
|
-
- `
|
|
227
|
-
- `get_environment_job` - Get details of a specific job (via environment)
|
|
228
|
-
- `update_workspace_job` - Update a job's name and node selectors
|
|
229
|
-
- `delete_workspace_job` - Delete a job ⚠️
|
|
464
|
+
- `list_workspaces` — List all workspaces
|
|
465
|
+
- `get_workspace` — Get details of a specific workspace
|
|
230
466
|
|
|
231
|
-
|
|
467
|
+
### Nodes
|
|
232
468
|
|
|
233
|
-
- `
|
|
234
|
-
- `
|
|
235
|
-
- `
|
|
236
|
-
- `
|
|
237
|
-
- `
|
|
469
|
+
- `list_environment_nodes` — List nodes in an environment
|
|
470
|
+
- `list_workspace_nodes` — List nodes in a workspace
|
|
471
|
+
- `get_environment_node` — Get a specific environment node
|
|
472
|
+
- `get_workspace_node` — Get a specific workspace node
|
|
473
|
+
- `set_workspace_node` — Replace a workspace node with a full body
|
|
474
|
+
- `update_workspace_node` — Safely update selected fields of a workspace node
|
|
475
|
+
- `delete_workspace_node` — Delete a node from a workspace ⚠️
|
|
238
476
|
|
|
239
|
-
|
|
477
|
+
### Jobs
|
|
240
478
|
|
|
241
|
-
- `
|
|
242
|
-
- `
|
|
243
|
-
- `
|
|
244
|
-
- `
|
|
245
|
-
- `
|
|
246
|
-
- `run_status` - Check status of a running job
|
|
247
|
-
- `retry_run` - Retry a failed run; requires Snowflake auth (Key Pair or PAT, credentials from env vars)
|
|
248
|
-
- `cancel_run` - Cancel a running job (requires `runID` and `environmentID`; `orgID` may come from `COALESCE_ORG_ID`) ⚠️
|
|
479
|
+
- `list_environment_jobs` — List all jobs for an environment
|
|
480
|
+
- `create_workspace_job` — Create a job in a workspace with node include/exclude selectors
|
|
481
|
+
- `get_environment_job` — Get details of a specific job (via environment)
|
|
482
|
+
- `update_workspace_job` — Update a job's name and node selectors
|
|
483
|
+
- `delete_workspace_job` — Delete a job ⚠️
|
|
249
484
|
|
|
250
|
-
|
|
485
|
+
### Subgraphs
|
|
251
486
|
|
|
252
|
-
- `
|
|
253
|
-
- `
|
|
254
|
-
- `
|
|
255
|
-
- `
|
|
256
|
-
- `
|
|
487
|
+
- `list_workspace_subgraphs` — List subgraphs in a workspace
|
|
488
|
+
- `get_workspace_subgraph` — Get details of a specific subgraph
|
|
489
|
+
- `create_workspace_subgraph` — Create a subgraph to group nodes visually
|
|
490
|
+
- `update_workspace_subgraph` — Update a subgraph's name and node membership
|
|
491
|
+
- `delete_workspace_subgraph` — Delete a subgraph (nodes are NOT deleted) ⚠️
|
|
257
492
|
|
|
258
|
-
|
|
493
|
+
### Runs
|
|
259
494
|
|
|
260
|
-
- `
|
|
261
|
-
- `
|
|
262
|
-
- `
|
|
263
|
-
- `
|
|
264
|
-
- `
|
|
495
|
+
- `diagnose_run_failure` — Diagnose a failed run with error classification, root-cause analysis, and actionable fix suggestions
|
|
496
|
+
- `list_runs` — List runs with optional filters
|
|
497
|
+
- `get_run` — Get details of a specific run
|
|
498
|
+
- `get_run_results` — Get results of a completed run
|
|
499
|
+
- `start_run` — Start a new run; requires Snowflake auth (Key Pair or PAT, credentials from env vars)
|
|
500
|
+
- `run_status` — Check status of a running job
|
|
501
|
+
- `retry_run` — Retry a failed run; requires Snowflake auth (Key Pair or PAT, credentials from env vars)
|
|
502
|
+
- `cancel_run` — Cancel a running job (requires `runID` and `environmentID`; `orgID` may come from `COALESCE_ORG_ID` or the `orgID` field in your ~/.coa/config profile) ⚠️
|
|
265
503
|
|
|
266
|
-
|
|
504
|
+
### Projects
|
|
267
505
|
|
|
268
|
-
- `
|
|
269
|
-
- `
|
|
270
|
-
- `
|
|
271
|
-
- `
|
|
272
|
-
- `
|
|
273
|
-
- `delete_project_role` - Remove project role from a user ⚠️
|
|
274
|
-
- `set_env_role` - Set environment role for a user
|
|
275
|
-
- `delete_env_role` - Remove environment role from a user ⚠️
|
|
506
|
+
- `list_projects` — List all projects
|
|
507
|
+
- `get_project` — Get project details
|
|
508
|
+
- `create_project` — Create a new project
|
|
509
|
+
- `update_project` — Update a project
|
|
510
|
+
- `delete_project` — Delete a project ⚠️
|
|
276
511
|
|
|
277
|
-
###
|
|
512
|
+
### Git Accounts
|
|
278
513
|
|
|
279
|
-
|
|
514
|
+
- `list_git_accounts` — List all git accounts
|
|
515
|
+
- `get_git_account` — Get git account details
|
|
516
|
+
- `create_git_account` — Create a new git account
|
|
517
|
+
- `update_git_account` — Update a git account
|
|
518
|
+
- `delete_git_account` — Delete a git account ⚠️
|
|
280
519
|
|
|
281
|
-
|
|
520
|
+
### Users and roles
|
|
282
521
|
|
|
283
|
-
- `
|
|
284
|
-
- `
|
|
285
|
-
- `
|
|
286
|
-
- `
|
|
287
|
-
- `
|
|
288
|
-
- `
|
|
289
|
-
- `
|
|
290
|
-
- `
|
|
291
|
-
- `analyze_workspace_patterns` - Analyze workspace nodes to detect package adoption, pipeline layers, methodology, and generate recommendations
|
|
522
|
+
- `list_org_users` — List all organization users
|
|
523
|
+
- `get_user_roles` — Get roles for a specific user
|
|
524
|
+
- `list_user_roles` — List all user roles
|
|
525
|
+
- `set_org_role` — Set organization role for a user
|
|
526
|
+
- `set_project_role` — Set project role for a user
|
|
527
|
+
- `delete_project_role` — Remove project role from a user ⚠️
|
|
528
|
+
- `set_env_role` — Set environment role for a user
|
|
529
|
+
- `delete_env_role` — Remove environment role from a user ⚠️
|
|
292
530
|
|
|
293
|
-
|
|
531
|
+
</details>
|
|
294
532
|
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
- `create_pipeline_from_sql` - Plan and create a pipeline directly from SQL
|
|
298
|
-
- `build_pipeline_from_intent` - Build a pipeline from a natural language goal with automatic entity resolution and node type selection
|
|
299
|
-
- `review_pipeline` - Analyze an existing pipeline for redundant nodes, missing joins, layer violations, naming issues, and optimization opportunities
|
|
300
|
-
- `parse_sql_structure` - Parse a SQL statement into structural components (CTEs, source tables, projected columns) without touching the workspace
|
|
301
|
-
- `select_pipeline_node_type` - Rank and select the best Coalesce node type for a pipeline step using the deliberative selection loop against repo or workspace-observed types
|
|
533
|
+
<details>
|
|
534
|
+
<summary><strong>Intelligent tools (46)</strong> — pipeline planning, config completion, join analysis, lineage</summary>
|
|
302
535
|
|
|
303
|
-
|
|
536
|
+
### Node creation and configuration
|
|
304
537
|
|
|
305
|
-
- `
|
|
306
|
-
- `
|
|
307
|
-
- `
|
|
308
|
-
- `
|
|
538
|
+
- `create_workspace_node_from_scratch` — Create a workspace node with no predecessors, apply fields to the requested completion level, and run automatic config completion
|
|
539
|
+
- `create_workspace_node_from_predecessor` — Create a node from predecessor nodes, verify column coverage, suggest join columns, and run automatic config completion
|
|
540
|
+
- `replace_workspace_node_columns` — Replace `metadata.columns` wholesale and optionally apply additional changes for complex column rewrites
|
|
541
|
+
- `convert_join_to_aggregation` — Convert a join-style node into an aggregated fact-style node with generated JOIN/GROUP BY analysis
|
|
542
|
+
- `apply_join_condition` — Auto-generate and write a FROM/JOIN/ON clause for a multi-predecessor node
|
|
543
|
+
- `create_node_from_external_schema` — Create a workspace node whose columns match an existing warehouse table or external schema
|
|
544
|
+
- `complete_node_configuration` — Intelligently complete a node's configuration by analyzing context and applying best-practice rules
|
|
545
|
+
- `list_workspace_node_types` — List distinct node types observed in current workspace nodes
|
|
546
|
+
- `analyze_workspace_patterns` — Analyze workspace nodes to detect package adoption, pipeline layers, methodology, and generate recommendations
|
|
309
547
|
|
|
310
|
-
|
|
548
|
+
### Pipeline planning and execution
|
|
311
549
|
|
|
312
|
-
- `
|
|
313
|
-
- `
|
|
314
|
-
- `
|
|
315
|
-
- `
|
|
550
|
+
- `plan_pipeline` — Plan a pipeline from SQL or a natural-language goal without mutating the workspace; ranks best-fit node types from the local repo
|
|
551
|
+
- `create_pipeline_from_plan` — Execute an approved pipeline plan using predecessor-based creation
|
|
552
|
+
- `create_pipeline_from_sql` — Plan and create a pipeline directly from SQL
|
|
553
|
+
- `build_pipeline_from_intent` — Build a pipeline from a natural language goal with automatic entity resolution and node type selection
|
|
554
|
+
- `review_pipeline` — Analyze an existing pipeline for redundant nodes, missing joins, layer violations, naming issues, and optimization opportunities
|
|
555
|
+
- `parse_sql_structure` — Parse a SQL statement into structural components (CTEs, source tables, projected columns) without touching the workspace
|
|
556
|
+
- `select_pipeline_node_type` — Rank and select the best Coalesce node type for a pipeline step using the deliberative selection loop against repo or workspace-observed types
|
|
557
|
+
|
|
558
|
+
### Pipeline workshop
|
|
559
|
+
|
|
560
|
+
- `pipeline_workshop_open` — Open an iterative pipeline builder session with workspace context pre-loaded
|
|
561
|
+
- `pipeline_workshop_instruct` — Send a natural language instruction to modify the current workshop plan
|
|
562
|
+
- `get_pipeline_workshop_status` — Get the current state of a workshop session
|
|
563
|
+
- `pipeline_workshop_close` — Close a workshop session and release resources
|
|
564
|
+
|
|
565
|
+
### Repo-backed node types and templates
|
|
566
|
+
|
|
567
|
+
- `list_repo_packages` — Inspect a committed local Coalesce repo and list package aliases plus enabled node-type coverage
|
|
568
|
+
- `list_repo_node_types` — List exact resolvable committed node-type identifiers from `nodeTypes/`
|
|
569
|
+
- `get_repo_node_type_definition` — Resolve one exact committed node type and return its outer definition plus parsed `nodeMetadataSpec`
|
|
570
|
+
- `generate_set_workspace_node_template` — Generate a YAML-friendly `set_workspace_node` body template from a definition object or committed repo definition
|
|
571
|
+
- `search_node_type_variants` — Search the committed node-type corpus by normalized family, package, primitive, or support status
|
|
572
|
+
- `get_node_type_variant` — Load one exact node-type corpus variant by variant key
|
|
573
|
+
- `generate_set_workspace_node_template_from_variant` — Generate a `set_workspace_node` body template from a committed corpus variant
|
|
574
|
+
|
|
575
|
+
### Lineage and impact
|
|
576
|
+
|
|
577
|
+
- `get_upstream_nodes` — Walk the full upstream dependency graph for a node
|
|
578
|
+
- `get_downstream_nodes` — Walk the full downstream dependency graph for a node
|
|
579
|
+
- `get_column_lineage` — Trace a column through the pipeline upstream and downstream via column-level references
|
|
580
|
+
- `analyze_impact` — Analyze downstream impact of changing a node or specific column — returns impacted counts, grouped by depth, and critical path
|
|
581
|
+
- `propagate_column_change` — Update all downstream columns after a column rename or data type change ⚠️
|
|
582
|
+
- `search_workspace_content` — Search across node SQL, column names, descriptions, and config values using the lineage cache as a searchable index
|
|
583
|
+
- `audit_documentation_coverage` — Scan all workspace nodes and columns for missing descriptions and report coverage statistics
|
|
584
|
+
|
|
585
|
+
### Cache and snapshots
|
|
586
|
+
|
|
587
|
+
- `cache_workspace_nodes` — Fetch every page of workspace nodes, write a full snapshot, and return cache metadata
|
|
588
|
+
- `cache_environment_nodes` — Fetch every page of environment nodes, write a full snapshot, and return cache metadata
|
|
589
|
+
- `cache_runs` — Fetch every page of run results, write a full snapshot, and return cache metadata
|
|
590
|
+
- `cache_org_users` — Fetch every page of organization users, write a full snapshot, and return cache metadata
|
|
591
|
+
- `clear_data_cache` — Delete all cached snapshots, auto-cached responses, and plan summaries ⚠️
|
|
592
|
+
|
|
593
|
+
### Run workflows
|
|
594
|
+
|
|
595
|
+
- `run_and_wait` — Start a run and poll until completion; requires Snowflake auth (Key Pair or PAT)
|
|
596
|
+
- `retry_and_wait` — Retry a failed run and poll until completion; requires Snowflake auth (Key Pair or PAT)
|
|
597
|
+
- `get_run_details` — Get run metadata and results in one call
|
|
598
|
+
- `get_environment_overview` — Get environment details with full node list
|
|
599
|
+
- `get_environment_health` — Comprehensive health dashboard: node counts by type, run statuses, failed runs in last 24h, stale nodes, dependency health, and overall health score (walks all paginated environment runs before scoring — slower on busy environments)
|
|
600
|
+
|
|
601
|
+
### Skills
|
|
316
602
|
|
|
317
|
-
|
|
603
|
+
- `personalize_skills` — Export bundled skill files to a local directory for customization; creates editable `coalesce_skills.{name}.md` and `user_skills.{name}.md` pairs (idempotent — never overwrites existing files)
|
|
318
604
|
|
|
319
|
-
|
|
320
|
-
- `get_node_type_variant` - Load one exact node-type corpus variant by variant key
|
|
321
|
-
- `generate_set_workspace_node_template_from_variant` - Generate a `set_workspace_node` body template from a committed corpus variant without needing the original external source repo at runtime; partial variants are rejected unless `allowPartial=true`
|
|
322
|
-
|
|
323
|
-
#### Cache and Snapshots
|
|
324
|
-
|
|
325
|
-
- `cache_workspace_nodes` - Fetch every page of workspace nodes, write the full snapshot to `coalesce_transform_mcp_data_cache/nodes/`, and return only cache metadata
|
|
326
|
-
- `cache_environment_nodes` - Fetch every page of environment nodes, write the full snapshot to `coalesce_transform_mcp_data_cache/nodes/`, and return only cache metadata
|
|
327
|
-
- `cache_runs` - Fetch every page of run results, write the full snapshot to `coalesce_transform_mcp_data_cache/runs/`, and return only cache metadata
|
|
328
|
-
- `cache_org_users` - Fetch every page of organization users, write the full snapshot to `coalesce_transform_mcp_data_cache/users/`, and return only cache metadata
|
|
329
|
-
- `clear_data_cache` - Delete all cached snapshots, auto-cached responses, and plan summaries under `coalesce_transform_mcp_data_cache/` ⚠️
|
|
330
|
-
|
|
331
|
-
#### Workflows
|
|
605
|
+
### Setup
|
|
332
606
|
|
|
333
|
-
- `
|
|
334
|
-
- `retry_and_wait` - Retry a failed run and poll until completion; requires Snowflake auth (Key Pair or PAT)
|
|
335
|
-
- `get_run_details` - Get run metadata and results in one call
|
|
336
|
-
- `get_environment_overview` - Get environment details with full node list
|
|
337
|
-
- `get_environment_health` - Get a comprehensive health dashboard: node counts by type, run statuses, failed runs in last 24h, stale nodes, dependency health, and overall health score. This walks all paginated environment runs before scoring, so it can take longer on busy environments.
|
|
607
|
+
- `diagnose_setup` — Stateless probe reporting which first-time-setup pieces are configured: access token, Snowflake credentials, `~/.coa/config` profile, local repo path, and a best-effort `coa doctor` check. Returns a structured report plus ordered `nextSteps` and per-field `source` markers (`env`, `profile:<name>`, or `missing`). Pairs with the `/coalesce-setup` MCP prompt.
|
|
338
608
|
|
|
339
|
-
|
|
609
|
+
</details>
|
|
340
610
|
|
|
341
|
-
|
|
611
|
+
<details>
|
|
612
|
+
<summary><strong>COA CLI tools (14)</strong> — bundled <code>@coalescesoftware/coa</code> CLI</summary>
|
|
342
613
|
|
|
343
|
-
|
|
614
|
+
All local tools accept a `projectPath` argument and validate that it contains `data.yml` before shelling out. Destructive tools run preflight validation; see [Safety model](#safety-model).
|
|
344
615
|
|
|
345
|
-
|
|
346
|
-
- `get_downstream_nodes` - Walk the full downstream dependency graph for a node and return every dependent with depth level (no depth limit)
|
|
347
|
-
- `get_column_lineage` - Trace a column through the entire pipeline upstream and downstream via column-level references
|
|
348
|
-
- `analyze_impact` - Analyze downstream impact of changing a node or specific column — returns impacted counts, grouped by depth, and critical path
|
|
349
|
-
- `propagate_column_change` - Update all downstream columns after a column rename or data type change ⚠️
|
|
350
|
-
- `search_workspace_content` - Search across node SQL, column names, descriptions, and config values using the lineage cache as a searchable index
|
|
351
|
-
- `audit_documentation_coverage` - Scan all workspace nodes and columns for missing descriptions and report coverage statistics
|
|
616
|
+
### Read-only, local
|
|
352
617
|
|
|
353
|
-
|
|
618
|
+
- 🧰 `coa_doctor` — Check config, credentials, and warehouse connectivity for a project. Wraps `coa doctor --json`
|
|
619
|
+
- 🧰 `coa_validate` — Validate YAML schemas and scan a project for configuration problems. Wraps `coa validate --json`
|
|
620
|
+
- 🧰 `coa_list_project_nodes` — List all nodes defined in a local project (pre-deploy). Wraps `coa create --list-nodes`
|
|
621
|
+
- 🧰 `coa_dry_run_create` — Preview DDL without executing against the warehouse. Forces `--dry-run --verbose`. Does **not** validate that referenced columns/types exist in the warehouse — catches SQL generation bugs, not schema-drift bugs
|
|
622
|
+
- 🧰 `coa_dry_run_run` — Preview DML without executing against the warehouse. Forces `--dry-run --verbose`. Same caveat as `coa_dry_run_create`: SQL that looks valid here can still fail at run-time on missing columns
|
|
354
623
|
|
|
355
|
-
|
|
624
|
+
### Read-only, cloud (require `~/.coa/config`)
|
|
356
625
|
|
|
357
|
-
|
|
626
|
+
- 🧰 `coa_list_environments` — List deployment environments. Wraps `coa environments list --format json`
|
|
627
|
+
- 🧰 `coa_list_environment_nodes` — List deployed nodes in an environment. Wraps `coa nodes list --environmentID ...`
|
|
628
|
+
- 🧰 `coa_list_runs` — List pipeline runs in a cloud environment (or across all environments). Wraps `coa runs list`
|
|
358
629
|
|
|
359
|
-
|
|
630
|
+
### Describe
|
|
360
631
|
|
|
361
|
-
|
|
362
|
-
curl -LsS https://ai.snowflake.com/static/cc-scripts/install.sh | sh
|
|
363
|
-
cortex connections # interactive connection setup
|
|
364
|
-
```
|
|
632
|
+
- 🧰 `coa_describe` — Fetch a section of COA's self-describing documentation by topic + optional subtopic. Also exposed as `coalesce://coa/describe/*` [resources](#coa-describe-topics)
|
|
365
633
|
|
|
366
|
-
|
|
634
|
+
### Write and deploy
|
|
367
635
|
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
}
|
|
374
|
-
}
|
|
375
|
-
```
|
|
636
|
+
- 🧰 `coa_plan` — Generate a deployment plan JSON by diffing the local project against a cloud environment. Writes `coa-plan.json` (configurable via `out`). Non-destructive
|
|
637
|
+
- 🧰 `coa_create` — Run DDL (CREATE/REPLACE) against the warehouse for selected nodes. Preflight-gated. ⚠️
|
|
638
|
+
- 🧰 `coa_run` — Run DML (INSERT/MERGE) to populate selected nodes. Preflight-gated. ⚠️
|
|
639
|
+
- 🧰 `coa_deploy` — Apply a plan JSON to a cloud environment. Verifies the plan file exists before running. ⚠️
|
|
640
|
+
- 🧰 `coa_refresh` — Run DML for selected nodes in an already-deployed environment (no local project required). ⚠️
|
|
376
641
|
|
|
377
|
-
|
|
642
|
+
</details>
|
|
378
643
|
|
|
379
|
-
##
|
|
644
|
+
## Design notes
|
|
380
645
|
|
|
381
|
-
- **Caching:** Large responses are auto-cached to disk. Use `cache_workspace_nodes` and similar tools when you want a reusable snapshot. Configure the threshold with `COALESCE_MCP_AUTO_CACHE_MAX_BYTES`.
|
|
382
|
-
- **Repo-backed tools:** Set `COALESCE_REPO_PATH` to your local Coalesce repo root (containing `nodeTypes/`, `nodes/`, `packages/`) or pass `repoPath` on individual tool calls. The server does not clone repos or install packages.
|
|
383
646
|
- **SQL override is disallowed.** Nodes are built via YAML/config (columns, transforms, join conditions), not raw SQL. Template generation strips `overrideSQLToggle`, and write helpers reject `overrideSQL` fields.
|
|
647
|
+
- **Caching.** Large responses are auto-cached to disk. Use `cache_workspace_nodes` and siblings when you want a reusable snapshot. Configure the threshold with `COALESCE_MCP_AUTO_CACHE_MAX_BYTES`.
|
|
648
|
+
- **Repo-backed tools.** Set `COALESCE_REPO_PATH` (or add `repoPath=` to your ~/.coa/config profile) to your local Coalesce repo root (containing `nodeTypes/`, `nodes/`, `packages/`), or pass `repoPath` on individual tool calls. The server does not clone repos or install packages.
|
|
649
|
+
- **COA CLI versioning.** The bundled COA CLI is pinned to an exact alpha version — *not* a floating `@next` tag. Every release of this MCP ships with a known-good COA build. Changelog and bump policy: [docs/RELEASES.md](docs/RELEASES.md).
|
|
650
|
+
- **COA describe cache.** COA describe output is cached under `~/.cache/coalesce-transform-mcp/coa-describe/<coa-version>/` after first access. Cache is version-keyed — upgrading the MCP automatically invalidates stale content.
|
|
384
651
|
|
|
385
652
|
## Links
|
|
386
653
|
|
|
387
654
|
- [Coalesce Docs](https://docs.coalesce.io/docs)
|
|
388
655
|
- [Coalesce API Docs](https://docs.coalesce.io/docs/api/authentication)
|
|
656
|
+
- [Coalesce CLI (`coa`)](https://docs.coalesce.io/docs/cli)
|
|
389
657
|
- [Coalesce Marketplace Docs](https://docs.coalesce.io/docs/marketplace)
|
|
658
|
+
- [Model Context Protocol](https://modelcontextprotocol.io/)
|
|
390
659
|
|
|
391
660
|
## License
|
|
392
661
|
|