@pingvinen/donna-assistant 0.5.0 → 0.7.0

package/workflows/relearn-tools.md

@@ -54,6 +54,26 @@ git -C <storage_repo> diff --cached --quiet || git -C <storage_repo> commit -m "
 
 If `auto_push` is true in config, also push.
 
+**`backfill-tool-type`:** Backfill `type` on existing tool sections in tools.md using heuristic detection.
+
+Read `<storage_repo>/donna/tools.md` with the Read tool. If the file does not exist or has no tool sections, skip this handler.
+
+For each tool section (starting with `## <tool_name>`), check if a `- type:` line already exists. If the `- type:` line is missing, detect the correct type:
+
+1. If the tool section contains a `- command:` line where the value starts with `mcp:` (e.g., `- command: mcp:linear`), insert `- type: mcp` immediately after the `- command:` line.
+2. Else, if the tool section contains a `- base_url:` line:
+   - If the capabilities section contains entries that look like GraphQL queries (contain `query {` or `mutation {`), insert `- type: graphql` immediately after `## <tool_name>` (REST/GraphQL tools have no `- command:` line).
+   - Otherwise, insert `- type: rest` immediately after `## <tool_name>`.
+3. Else (no `mcp:` prefix, no `base_url` field), insert `- type: cli` immediately after the `- command:` line.
+
+Write the updated file back with the Write tool. If any changes were made, commit:
+```bash
+git -C <storage_repo> add -A
+git -C <storage_repo> diff --cached --quiet || git -C <storage_repo> commit -m "donna(migrate): backfill tool types on existing tools"
+```
+
+If `auto_push` is true in config, also push.
+
 After processing all pending migrations, update `~/.donna/state.md` with the Write tool: remove the completed entries from `pending_migrations`. If no entries remain, write:
 ```markdown
 ---
@@ -70,31 +90,92 @@ Read `<storage_repo>/donna/tools.md` with the Read tool. If the file does not ex
 Stop.
 
 Parse each tool section (starting with `## <tool_name>`). For each tool, extract:
-- `command` — the CLI command to run
-- `version` — the stored version string
+- `command` — the CLI command to run (CLI/MCP tools only)
+- `type` — the tool type field (if absent, treat as "cli")
+- `version` — the stored version string (CLI tools only)
 - `learned` — the date capabilities were last learned
+- `base_url` — the API endpoint (REST/GraphQL tools only)
+- `auth_header` — the auth header name, if present (REST/GraphQL tools only)
+- `auth_secret` — the secret key name, if present (REST/GraphQL tools only)
 - capabilities list under `### Capabilities`
 
 Store the parsed tools as `<registered_tools>`.
 </step>
 
 <step name="check-versions">
-For each tool in `<registered_tools>`, run via Bash:
-```bash
-<command> --version 2>/dev/null | head -1
-```
+For each tool in `<registered_tools>`:
+
+If `<type>` is `rest` or `mcp`:
+  Add to `<unchanged_tools>` — version checking is not applicable for REST/MCP tools.
+  Continue to next tool.
+
+If `<type>` is `graphql`:
+  Run a GraphQL introspection query to detect schema changes.
+
+  **IMPORTANT: Auth is OPTIONAL for GraphQL tools. Public APIs work without any secret. You MUST always attempt introspection regardless of whether auth is configured. NEVER skip a GraphQL tool just because it has no auth_secret.**
+
+  1. Attempt to read `<storage_repo>/donna/secrets.md` with the Read tool. If the file exists and contains the `auth_secret` key with a value that is not a placeholder (does not contain "REPLACE_WITH"), set `<resolved_secret>` to that value. Otherwise, set `<resolved_secret>` to empty (no auth). **An empty resolved_secret is perfectly valid — proceed to step 2 regardless.**
+
+  2. Run via Bash with a 15-second timeout. Include the auth header only when a real secret was resolved:
+     - If `<resolved_secret>` is non-empty:
+       ```bash
+       timeout 15 curl -s -X POST \
+         -H "<auth_header>: <resolved_secret>" \
+         -H "Content-Type: application/json" \
+         -d '{"query":"{ __schema { types { name fields { name type { name } } } } }"}' \
+         "<base_url>" 2>&1
+       ```
+     - If `<resolved_secret>` is empty (public API or no secret configured):
+       ```bash
+       timeout 15 curl -s -X POST \
+         -H "Content-Type: application/json" \
+         -d '{"query":"{ __schema { types { name fields { name type { name } } } } }"}' \
+         "<base_url>" 2>&1
+       ```
+
+  3. If the request fails (non-zero exit, timeout, or error response), add to `<unchanged_tools>` with note "introspection failed — skipped" and continue.
+
+  4. If successful, compare the returned schema against stored capabilities:
+     - Extract field names and types from the introspection response for the types/queries relevant to stored capabilities.
+     - Check if any stored capability references fields that no longer exist in the schema (removed fields).
+     - Check if the schema has new fields on types used by stored capabilities that might be useful.
+
+  5. If no meaningful changes detected, add to `<unchanged_tools>`.
+
+  6. If changes detected, add to `<changed_tools>` with a `<schema_changes>` annotation listing:
+     - Removed fields (fields in stored capabilities no longer in schema)
+     - New fields (fields in schema not referenced by any stored capability)
+
+If `<type>` is `cli` (or absent; treat as `cli`):
+  Run via Bash:
+  ```bash
+  <command> --version 2>/dev/null | head -1
+  ```
+
+  Compare the output against the stored `version` field using simple string equality — "is it different?" is sufficient; no semver parsing required.
+
+  If `<command>` is not found (command fails), treat it as changed with a warning note.
+
+  Add to:
+  - `<changed_tools>` — installed version differs from stored version (or command not found)
+  - `<unchanged_tools>` — installed version matches stored version exactly
+</step>
 
-Compare the output against the stored `version` field using simple string equality — "is it different?" is sufficient; no semver parsing required.
+<step name="report-unchanged">
+For each tool in `<unchanged_tools>`:
 
-If `<command>` is not found (command fails), treat it as changed with a warning note.
+If `<type>` is `rest` or `mcp`, print:
+```
+⊘ <tool_name>: <type> tool — re-learning not applicable (capabilities are user-defined)
+```
 
-Collect tools into two lists:
-- `<changed_tools>` — installed version differs from stored version (or command not found)
-- `<unchanged_tools>` — installed version matches stored version exactly
-</step>
+If `<type>` is `graphql`, print using the skip reason from check-versions:
+```
+⊘ <tool_name>: graphql tool — <skip_reason>
+```
+Where `<skip_reason>` is "no schema changes detected" or "introspection failed — skipped".
 
-<step name="report-unchanged">
-For each tool in `<unchanged_tools>`, print:
+Otherwise (CLI tools), print:
 ```
 ⊘ <tool_name>: unchanged at <version> — skipped
 ```
@@ -106,7 +187,34 @@ If ALL tools are in `<unchanged_tools>` (no changes found), print:
 Stop.
 </step>
 
+<step name="relearn-graphql">
+For each graphql tool in `<changed_tools>`:
+
+Print:
+```
+⚠ <tool_name>: schema changes detected
+  Removed fields: <list or "none">
+  New fields: <list or "none">
+```
+
+Use AskUserQuestion:
+```
+Update capabilities for <tool_name>?
+```
+Suggest "yes" and "no" as options.
+
+If yes:
+  Show the current capabilities and the detected changes side by side. Use AskUserQuestion to let the user update capabilities interactively (same editing loop as adjust-tool: "remove <number>", "add <name>: <query>", "edit <number> <new_query>", "done").
+
+  Store the updated capabilities. Update `learned` date to today.
+
+If no:
+  Skip — keep existing capabilities unchanged. Update `learned` date to today (to avoid re-checking next run).
+</step>
+
 <step name="relearn-changed">
+Note: Re-learning is supported for CLI tools (version-based) and GraphQL tools (schema introspection). REST and MCP tool capabilities are user-defined and not auto-learned.
+
 For each tool in `<changed_tools>`, apply the same learn-capabilities logic as add-tool.md's learn-capabilities step.
 
 Determine if the tool is well-known (gh, jira, kubectl) or unknown. For well-known tools, synthesize capabilities from training data. Do NOT parse --help for well-known tools.

package/workflows/run-tools.md

@@ -54,6 +54,26 @@ git -C <storage_repo> diff --cached --quiet || git -C <storage_repo> commit -m "
 
 If `auto_push` is true in config, also push.
 
+**`backfill-tool-type`:** Backfill `type` on existing tool sections in tools.md using heuristic detection.
+
+Read `<storage_repo>/donna/tools.md` with the Read tool. If the file does not exist or has no tool sections, skip this handler.
+
+For each tool section (starting with `## <tool_name>`), check if a `- type:` line already exists. If the `- type:` line is missing, detect the correct type:
+
+1. If the tool section contains a `- command:` line where the value starts with `mcp:` (e.g., `- command: mcp:linear`), insert `- type: mcp` immediately after the `- command:` line.
+2. Else, if the tool section contains a `- base_url:` line:
+   - If the capabilities section contains entries that look like GraphQL queries (contain `query {` or `mutation {`), insert `- type: graphql` immediately after `## <tool_name>` (REST/GraphQL tools have no `- command:` line).
+   - Otherwise, insert `- type: rest` immediately after `## <tool_name>`.
+3. Else (no `mcp:` prefix, no `base_url` field), insert `- type: cli` immediately after the `- command:` line.
+
+Write the updated file back with the Write tool. If any changes were made, commit:
+```bash
+git -C <storage_repo> add -A
+git -C <storage_repo> diff --cached --quiet || git -C <storage_repo> commit -m "donna(migrate): backfill tool types on existing tools"
+```
+
+If `auto_push` is true in config, also push.
+
 After processing all pending migrations, update `~/.donna/state.md` with the Write tool: remove the completed entries from `pending_migrations`. If no entries remain, write:
 ```markdown
 ---
@@ -69,7 +89,7 @@ Read `<storage_repo>/donna/tools.md` with the Read tool. If the file does not ex
 ```
 Stop.
 
-Parse each tool section (starting with `## <tool_name>`). For each tool, extract the `command` field and the capabilities list under `### Capabilities`.
+Parse each tool section (starting with `## <tool_name>`). For each tool, extract the `type` field (if absent, treat as "cli"), the `command` field, and the capabilities list under `### Capabilities`.
 
 Store the parsed tools and their capabilities as `<registered_tools>`.
 </step>
@@ -101,7 +121,27 @@ If the `## From Tools` section does not exist in today's file, set `<existing_to
 </step>
 
 <step name="pull-fresh-data">
-For each tool in `<registered_tools>`, for each capability:
+**If only one tool is registered**, run it directly (no Task spawning needed) using the type-aware execution logic below.
+
+**If multiple tools are registered**, spawn one Task agent per tool. Each agent receives:
+
+- The tool name, type, and its capabilities list
+- For REST/GraphQL tools: the base_url, auth_header, and auth_secret fields
+- For MCP tools: the capability names with mcp: prefix
+- Instructions to execute all capabilities and return results as a structured list
+- Instructions to NEVER write to any file or run git commands
+
+**CRITICAL constraints for Task agents:**
+- Agents return raw task lists and warnings ONLY
+- Agents must NOT write to any file (no Write tool calls to daily file)
+- Agents must NOT run git commands (SSH signing constraint from CLAUDE.md)
+- All file writing happens in the main workflow after agents return
+
+**Global timeout:** Wait for all Task agents to complete, up to 2 minutes total. After 2 minutes, collect whatever results have been returned and treat non-responding tools as failed with warning: `! <tool_name>: timed out (2-minute batch limit)`.
+
+**Type-aware execution within each agent (or direct execution for single tool):**
+
+For `type: cli` capabilities (format: `<name>: <cli_invocation>`):
 
 Run the capability command via Bash with a 10-second timeout:
 ```bash
@@ -133,6 +173,41 @@ For other tools: use Claude's understanding to extract task-like items from the
 ```
 Continue to the next capability. Never retry.
 
+For `type: rest` capabilities (format: `<name>: <METHOD> /path`):
+1. Read `<storage_repo>/donna/secrets.md` with the Read tool. Parse key-value pairs from under the frontmatter.
+2. Resolve the `auth_secret` key to get the actual secret value. If the key is not found in secrets.md or the value is `REPLACE_WITH_YOUR_SECRET`, add warning `! <tool_name>: missing secret <auth_secret> — edit donna/secrets.md` and skip this tool.
+3. For each capability, run via Bash with a 10-second timeout:
+```bash
+timeout 10 curl -s -H "<auth_header>: <resolved_secret>" "<base_url><path>" 2>&1
+```
+4. Parse the JSON response using Claude's understanding to extract task-like items. Format:
+`- [ ] (<tool_name>) <description> [<identifier>](<url>)`
+
+**On any failure** (non-zero exit, timeout, missing secret):
+Add a warning: `! <tool_name>: <error_description>`
+Continue. Never retry.
+
+For `type: graphql` capabilities (format: `<name>: <graphql_query>`):
+1. Same secrets resolution as REST.
+2. For each capability, run via Bash with a 10-second timeout:
+```bash
+timeout 10 curl -s -X POST -H "<auth_header>: <resolved_secret>" -H "Content-Type: application/json" -d '{"query":"<graphql_query>"}' "<base_url>" 2>&1
+```
+3. Parse the JSON response to extract task-like items. Same format as REST.
+
+**On any failure** (non-zero exit, timeout, missing secret):
+Add a warning: `! <tool_name>: <error_description>`
+Continue. Never retry.
+
+For `type: mcp` capabilities (format: `<name>: mcp:<server>/<tool>`):
+1. Invoke the MCP tool directly using Claude's native MCP tool invocation (NOT via Bash).
+2. Parse the MCP tool's response using Claude's understanding of the tool to extract task-like items. Format:
+`- [ ] (<tool_name>) <description> [<identifier>](<url>)`
+
+**On any failure:**
+Add a warning: `! <tool_name>: <error_description>`
+Continue. Never retry.
+
 Store all fresh tasks as `<fresh_tool_tasks>`: a map of URL to task line.
 Collect all warning messages as `<tool_warnings>`.
 </step>