pulsemcp-cms-admin-mcp-server 0.6.13 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (16) hide show
  1. package/README.md +61 -58
  2. package/build/shared/src/exam-result-store.js +59 -0
  3. package/build/shared/src/tools/get-exam-result.js +98 -0
  4. package/build/shared/src/tools/run-exam-for-mirror.js +51 -2
  5. package/build/shared/src/tools/save-results-for-mirror.js +89 -13
  6. package/build/shared/src/tools.js +11 -3
  7. package/package.json +1 -1
  8. package/shared/exam-result-store.d.ts +37 -0
  9. package/shared/exam-result-store.js +59 -0
  10. package/shared/tools/get-exam-result.d.ts +40 -0
  11. package/shared/tools/get-exam-result.js +98 -0
  12. package/shared/tools/run-exam-for-mirror.js +51 -2
  13. package/shared/tools/save-results-for-mirror.d.ts +8 -4
  14. package/shared/tools/save-results-for-mirror.js +89 -13
  15. package/shared/tools.d.ts +6 -3
  16. package/shared/tools.js +11 -3
package/README.md CHANGED
@@ -40,60 +40,61 @@ This is an MCP ([Model Context Protocol](https://modelcontextprotocol.io/)) Serv
40
40
 
41
41
  This server is built and tested on macOS with Claude Desktop. It should work with other MCP clients as well.
42
42
 
43
- | Tool Name | Tool Group | Read/Write | Description |
44
- | -------------------------------------- | ------------------ | ---------- | ----------------------------------------------------------------------------- |
45
- | `get_newsletter_posts` | newsletter | read | List newsletter posts with search, sorting, and pagination options. |
46
- | `get_newsletter_post` | newsletter | read | Retrieve a specific newsletter post by its unique slug. |
47
- | `draft_newsletter_post` | newsletter | write | Create a new draft newsletter post with title, body, and metadata. |
48
- | `update_newsletter_post` | newsletter | write | Update an existing newsletter post's content and metadata (except status). |
49
- | `upload_image` | newsletter | write | Upload an image and attach it to a specific newsletter post. |
50
- | `get_authors` | newsletter | read | Get a list of authors with optional search and pagination. |
51
- | `search_mcp_implementations` | server_directory | read | Search for MCP servers and clients in the PulseMCP registry. |
52
- | `get_draft_mcp_implementations` | server_directory | read | Retrieve paginated list of draft MCP implementations needing review. |
53
- | `find_providers` | server_directory | read | Search for providers by ID, name, URL, or slug. |
54
- | `save_mcp_implementation` | server_directory | write | Update an MCP implementation (replicates Admin panel "Save Changes" button). |
55
- | `send_impl_posted_notif` | server_directory | write | Send email notification when MCP implementation goes live. |
56
- | `get_official_mirror_queue_items` | official_queue | read | List and filter official mirror queue entries with pagination and search. |
57
- | `get_official_mirror_queue_item` | official_queue | read | Get detailed information about a single official mirror queue entry. |
58
- | `approve_official_mirror_queue_item` | official_queue | write | Approve a queue entry and link it to an existing MCP server (async). |
59
- | `approve_mirror_no_modify` | official_queue | write | Approve without updating the linked server. |
60
- | `reject_official_mirror_queue_item` | official_queue | write | Reject a queue entry (async operation). |
61
- | `add_official_mirror_to_regular_queue` | official_queue | write | Convert a queue entry to a draft MCP implementation (async). |
62
- | `unlink_official_mirror_queue_item` | official_queue | write | Unlink a queue entry from its linked MCP server. |
63
- | `get_unofficial_mirrors` | unofficial_mirrors | read | List unofficial mirrors with search, pagination, and MCP server filtering. |
64
- | `get_unofficial_mirror` | unofficial_mirrors | read | Get detailed unofficial mirror info by ID or name. |
65
- | `create_unofficial_mirror` | unofficial_mirrors | write | Create a new unofficial mirror entry with JSON data. |
66
- | `update_unofficial_mirror` | unofficial_mirrors | write | Update an existing unofficial mirror by ID. |
67
- | `delete_unofficial_mirror` | unofficial_mirrors | write | Delete an unofficial mirror by ID (irreversible). |
68
- | `get_official_mirrors` | official_mirrors | read | List official mirrors with search, status, and processing filters. |
69
- | `get_official_mirror` | official_mirrors | read | Get detailed official mirror info by ID or name. |
70
- | `get_tenants` | tenants | read | List tenants with search and admin status filtering. |
71
- | `get_tenant` | tenants | read | Get detailed tenant info by ID or slug. |
72
- | `get_mcp_jsons` | mcp_jsons | read | List MCP JSON configs with mirror and server filtering. |
73
- | `get_mcp_json` | mcp_jsons | read | Get a single MCP JSON configuration by ID. |
74
- | `create_mcp_json` | mcp_jsons | write | Create a new MCP JSON configuration for an unofficial mirror. |
75
- | `update_mcp_json` | mcp_jsons | write | Update an existing MCP JSON configuration by ID. |
76
- | `delete_mcp_json` | mcp_jsons | write | Delete an MCP JSON configuration by ID (irreversible). |
77
- | `list_mcp_servers` | mcp_servers | read | List/search MCP servers with filtering by status, classification, pagination. |
78
- | `get_mcp_server` | mcp_servers | read | Get detailed MCP server info by slug (unified view of all admin UI fields). |
79
- | `update_mcp_server` | mcp_servers | write | Update an MCP server's fields (all admin UI fields supported). |
80
- | `get_redirects` | redirects | read | List URL redirects with search, status filtering, and pagination. |
81
- | `get_redirect` | redirects | read | Get detailed redirect info by ID. |
82
- | `create_redirect` | redirects | write | Create a new URL redirect entry. |
83
- | `update_redirect` | redirects | write | Update an existing URL redirect by ID. |
84
- | `delete_redirect` | redirects | write | Delete a URL redirect by ID (irreversible). |
85
- | `list_good_jobs` | good_jobs | read | List and filter background jobs by queue, status, job class, and date range. |
86
- | `get_good_job` | good_jobs | read | Get detailed information about a specific background job. |
87
- | `list_good_job_cron_schedules` | good_jobs | read | List all configured cron schedules. |
88
- | `list_good_job_processes` | good_jobs | read | List active worker processes. |
89
- | `get_good_job_queue_statistics` | good_jobs | read | Get aggregate job statistics by status. |
90
- | `retry_good_job` | good_jobs | write | Retry a failed or discarded background job. |
91
- | `discard_good_job` | good_jobs | write | Discard a background job to prevent retries. |
92
- | `reschedule_good_job` | good_jobs | write | Reschedule a background job to a new time. |
93
- | `force_trigger_good_job_cron` | good_jobs | write | Force trigger a cron schedule immediately. |
94
- | `cleanup_good_jobs` | good_jobs | write | Clean up old background jobs by status and age. |
95
- | `run_exam_for_mirror` | proctor | write | Run proctor exams against unofficial mirrors via Fly Machines. |
96
- | `save_results_for_mirror` | proctor | write | Save proctor exam results (must use output from `run_exam_for_mirror`). |
43
+ | Tool Name | Tool Group | Read/Write | Description |
44
+ | -------------------------------------- | ------------------ | ---------- | ---------------------------------------------------------------------------------------------------------- |
45
+ | `get_newsletter_posts` | newsletter | read | List newsletter posts with search, sorting, and pagination options. |
46
+ | `get_newsletter_post` | newsletter | read | Retrieve a specific newsletter post by its unique slug. |
47
+ | `draft_newsletter_post` | newsletter | write | Create a new draft newsletter post with title, body, and metadata. |
48
+ | `update_newsletter_post` | newsletter | write | Update an existing newsletter post's content and metadata (except status). |
49
+ | `upload_image` | newsletter | write | Upload an image and attach it to a specific newsletter post. |
50
+ | `get_authors` | newsletter | read | Get a list of authors with optional search and pagination. |
51
+ | `search_mcp_implementations` | server_directory | read | Search for MCP servers and clients in the PulseMCP registry. |
52
+ | `get_draft_mcp_implementations` | server_directory | read | Retrieve paginated list of draft MCP implementations needing review. |
53
+ | `find_providers` | server_directory | read | Search for providers by ID, name, URL, or slug. |
54
+ | `save_mcp_implementation` | server_directory | write | Update an MCP implementation (replicates Admin panel "Save Changes" button). |
55
+ | `send_impl_posted_notif` | server_directory | write | Send email notification when MCP implementation goes live. |
56
+ | `get_official_mirror_queue_items` | official_queue | read | List and filter official mirror queue entries with pagination and search. |
57
+ | `get_official_mirror_queue_item` | official_queue | read | Get detailed information about a single official mirror queue entry. |
58
+ | `approve_official_mirror_queue_item` | official_queue | write | Approve a queue entry and link it to an existing MCP server (async). |
59
+ | `approve_mirror_no_modify` | official_queue | write | Approve without updating the linked server. |
60
+ | `reject_official_mirror_queue_item` | official_queue | write | Reject a queue entry (async operation). |
61
+ | `add_official_mirror_to_regular_queue` | official_queue | write | Convert a queue entry to a draft MCP implementation (async). |
62
+ | `unlink_official_mirror_queue_item` | official_queue | write | Unlink a queue entry from its linked MCP server. |
63
+ | `get_unofficial_mirrors` | unofficial_mirrors | read | List unofficial mirrors with search, pagination, and MCP server filtering. |
64
+ | `get_unofficial_mirror` | unofficial_mirrors | read | Get detailed unofficial mirror info by ID or name. |
65
+ | `create_unofficial_mirror` | unofficial_mirrors | write | Create a new unofficial mirror entry with JSON data. |
66
+ | `update_unofficial_mirror` | unofficial_mirrors | write | Update an existing unofficial mirror by ID. |
67
+ | `delete_unofficial_mirror` | unofficial_mirrors | write | Delete an unofficial mirror by ID (irreversible). |
68
+ | `get_official_mirrors` | official_mirrors | read | List official mirrors with search, status, and processing filters. |
69
+ | `get_official_mirror` | official_mirrors | read | Get detailed official mirror info by ID or name. |
70
+ | `get_tenants` | tenants | read | List tenants with search and admin status filtering. |
71
+ | `get_tenant` | tenants | read | Get detailed tenant info by ID or slug. |
72
+ | `get_mcp_jsons` | mcp_jsons | read | List MCP JSON configs with mirror and server filtering. |
73
+ | `get_mcp_json` | mcp_jsons | read | Get a single MCP JSON configuration by ID. |
74
+ | `create_mcp_json` | mcp_jsons | write | Create a new MCP JSON configuration for an unofficial mirror. |
75
+ | `update_mcp_json` | mcp_jsons | write | Update an existing MCP JSON configuration by ID. |
76
+ | `delete_mcp_json` | mcp_jsons | write | Delete an MCP JSON configuration by ID (irreversible). |
77
+ | `list_mcp_servers` | mcp_servers | read | List/search MCP servers with filtering by status, classification, pagination. |
78
+ | `get_mcp_server` | mcp_servers | read | Get detailed MCP server info by slug (unified view of all admin UI fields). |
79
+ | `update_mcp_server` | mcp_servers | write | Update an MCP server's fields (all admin UI fields supported). |
80
+ | `get_redirects` | redirects | read | List URL redirects with search, status filtering, and pagination. |
81
+ | `get_redirect` | redirects | read | Get detailed redirect info by ID. |
82
+ | `create_redirect` | redirects | write | Create a new URL redirect entry. |
83
+ | `update_redirect` | redirects | write | Update an existing URL redirect by ID. |
84
+ | `delete_redirect` | redirects | write | Delete a URL redirect by ID (irreversible). |
85
+ | `list_good_jobs` | good_jobs | read | List and filter background jobs by queue, status, job class, and date range. |
86
+ | `get_good_job` | good_jobs | read | Get detailed information about a specific background job. |
87
+ | `list_good_job_cron_schedules` | good_jobs | read | List all configured cron schedules. |
88
+ | `list_good_job_processes` | good_jobs | read | List active worker processes. |
89
+ | `get_good_job_queue_statistics` | good_jobs | read | Get aggregate job statistics by status. |
90
+ | `retry_good_job` | good_jobs | write | Retry a failed or discarded background job. |
91
+ | `discard_good_job` | good_jobs | write | Discard a background job to prevent retries. |
92
+ | `reschedule_good_job` | good_jobs | write | Reschedule a background job to a new time. |
93
+ | `force_trigger_good_job_cron` | good_jobs | write | Force trigger a cron schedule immediately. |
94
+ | `cleanup_good_jobs` | good_jobs | write | Clean up old background jobs by status and age. |
95
+ | `run_exam_for_mirror` | proctor | write | Run proctor exams against unofficial mirrors via Fly Machines. Returns truncated summary with `result_id`. |
96
+ | `get_exam_result` | proctor | read | Retrieve full untruncated exam results by `result_id`, with optional section/mirror filtering. |
97
+ | `save_results_for_mirror` | proctor | write | Save proctor exam results. Accepts `result_id` or explicit results array. |
97
98
 
98
99
  # Tool Groups
99
100
 
@@ -126,7 +127,8 @@ This server organizes tools into groups that can be selectively enabled or disab
126
127
  | `redirects_readonly` | 2 | URL redirects read-only (list, get) |
127
128
  | `good_jobs` | 10 | Full GoodJob background job management (read + write) |
128
129
  | `good_jobs_readonly` | 5 | GoodJob read-only (list, get, stats, processes, cron) |
129
- | `proctor` | 2 | Proctor exam execution and result storage (write-only, no readonly variant) |
130
+ | `proctor` | 3 | Proctor exam execution, result retrieval, and result storage (read + write) |
131
+ | `proctor_readonly` | 1 | Proctor results read-only (`get_exam_result`) |
130
132
 
131
133
  ### Tools by Group
132
134
 
@@ -158,7 +160,8 @@ This server organizes tools into groups that can be selectively enabled or disab
158
160
  - **good_jobs** / **good_jobs_readonly**:
159
161
  - Read-only: `list_good_jobs`, `get_good_job`, `list_good_job_cron_schedules`, `list_good_job_processes`, `get_good_job_queue_statistics`
160
162
  - Write: `retry_good_job`, `discard_good_job`, `reschedule_good_job`, `force_trigger_good_job_cron`, `cleanup_good_jobs`
161
- - **proctor** (no readonly variant — both tools trigger side effects):
163
+ - **proctor** / **proctor_readonly**:
164
+ - Read-only: `get_exam_result`
162
165
  - Write: `run_exam_for_mirror`, `save_results_for_mirror`
163
166
 
164
167
  ## Environment Variables
@@ -190,10 +193,10 @@ TOOL_GROUPS=server_directory_readonly
190
193
  Enable all groups with read-only access:
191
194
 
192
195
  ```bash
193
- TOOL_GROUPS=newsletter_readonly,server_directory_readonly,official_queue_readonly,unofficial_mirrors_readonly,official_mirrors_readonly,tenants_readonly,mcp_jsons_readonly,mcp_servers_readonly,redirects_readonly,good_jobs_readonly
196
+ TOOL_GROUPS=newsletter_readonly,server_directory_readonly,official_queue_readonly,unofficial_mirrors_readonly,official_mirrors_readonly,tenants_readonly,mcp_jsons_readonly,mcp_servers_readonly,redirects_readonly,good_jobs_readonly,proctor_readonly
194
197
  ```
195
198
 
196
- Note: `proctor` has no readonly variant since both tools trigger side effects.
199
+ Note: `proctor_readonly` includes only `get_exam_result` for retrieving stored results. `notifications` has no readonly variant since sending emails is always a write operation.
197
200
 
198
201
  Mix full and read-only access per group:
199
202
 
package/build/shared/src/exam-result-store.js ADDED
@@ -0,0 +1,59 @@
1
+ import { randomUUID } from 'crypto';
2
+ /**
3
+ * Maximum number of results to keep in memory. Oldest results are evicted
4
+ * when this limit is reached (FIFO). Each result can be 60KB+ for servers
5
+ * with many tools, so 100 entries ≈ 6MB worst case.
6
+ */
7
+ const MAX_RESULTS = 100;
8
+ /**
9
+ * In-memory store for proctor exam results.
10
+ *
11
+ * When `run_exam_for_mirror` completes, the full result is stored here
12
+ * and a UUID `result_id` is returned. This avoids dumping large payloads
13
+ * (~60KB+ for servers with many tools) into the LLM context.
14
+ *
15
+ * Eviction: When the store exceeds MAX_RESULTS entries, the oldest result
16
+ * is evicted (FIFO). Results are also deleted after successful save via
17
+ * `save_results_for_mirror`.
18
+ *
19
+ * Consumers can:
20
+ * - Use `get_exam_result` to drill into the full result on demand
21
+ * - Pass `result_id` to `save_results_for_mirror` instead of the full payload
22
+ */
23
+ class ExamResultStore {
24
+ results = new Map();
25
+ store(mirrorIds, runtimeId, examType, lines) {
26
+ const resultId = randomUUID();
27
+ // Evict oldest entries if at capacity (Map preserves insertion order)
28
+ while (this.results.size >= MAX_RESULTS) {
29
+ const oldestKey = this.results.keys().next().value;
30
+ if (oldestKey !== undefined) {
31
+ this.results.delete(oldestKey);
32
+ }
33
+ }
34
+ this.results.set(resultId, {
35
+ result_id: resultId,
36
+ mirror_ids: mirrorIds,
37
+ runtime_id: runtimeId,
38
+ exam_type: examType,
39
+ lines,
40
+ stored_at: new Date().toISOString(),
41
+ });
42
+ return resultId;
43
+ }
44
+ get(resultId) {
45
+ return this.results.get(resultId);
46
+ }
47
+ delete(resultId) {
48
+ return this.results.delete(resultId);
49
+ }
50
+ get size() {
51
+ return this.results.size;
52
+ }
53
+ /** For testing only */
54
+ clear() {
55
+ this.results.clear();
56
+ }
57
+ }
58
+ /** Singleton instance shared across all tool factories */
59
+ export const examResultStore = new ExamResultStore();
package/build/shared/src/tools/get-exam-result.js ADDED
@@ -0,0 +1,98 @@
1
+ import { z } from 'zod';
2
+ import { examResultStore } from '../exam-result-store.js';
3
+ const PARAM_DESCRIPTIONS = {
4
+ result_id: 'The UUID returned by run_exam_for_mirror identifying the stored result to retrieve.',
5
+ section: 'Optional filter to retrieve only a specific section of the result. "exam_results" returns only exam_result lines, "logs" returns only log lines, "summary" returns only the summary line, "errors" returns only error lines. If omitted, returns all lines.',
6
+ mirror_id: 'Optional mirror ID filter. When provided, only returns exam_result lines for this specific mirror. Useful when the exam was run against multiple mirrors.',
7
+ };
8
+ const GetExamResultSchema = z.object({
9
+ result_id: z.string().uuid().describe(PARAM_DESCRIPTIONS.result_id),
10
+ section: z
11
+ .enum(['exam_results', 'logs', 'summary', 'errors'])
12
+ .optional()
13
+ .describe(PARAM_DESCRIPTIONS.section),
14
+ mirror_id: z.number().optional().describe(PARAM_DESCRIPTIONS.mirror_id),
15
+ });
16
+ export function getExamResult(_server, _clientFactory) {
17
+ return {
18
+ name: 'get_exam_result',
19
+ description: `Retrieve the full, untruncated proctor exam result stored by a prior run_exam_for_mirror call.
20
+
21
+ run_exam_for_mirror returns a truncated summary to keep the response within MCP size limits. This tool provides on-demand access to the complete result data, including full tool input schemas and detailed exam output.
22
+
23
+ Supports filtering by section (exam_results, logs, summary, errors) and by mirror_id to drill into specific parts of the result without loading the entire payload.
24
+
25
+ **Tip**: For servers with many tools, the full result can be very large. Use the section and/or mirror_id filters to retrieve only the data you need.
26
+
27
+ Typical usage:
28
+ 1. Call run_exam_for_mirror — note the returned result_id
29
+ 2. Call get_exam_result with that result_id and a section filter (e.g., section="exam_results")
30
+ 3. Optionally add mirror_id to narrow further for multi-mirror exams`,
31
+ inputSchema: {
32
+ type: 'object',
33
+ properties: {
34
+ result_id: { type: 'string', format: 'uuid', description: PARAM_DESCRIPTIONS.result_id },
35
+ section: {
36
+ type: 'string',
37
+ enum: ['exam_results', 'logs', 'summary', 'errors'],
38
+ description: PARAM_DESCRIPTIONS.section,
39
+ },
40
+ mirror_id: { type: 'number', description: PARAM_DESCRIPTIONS.mirror_id },
41
+ },
42
+ required: ['result_id'],
43
+ },
44
+ handler: async (args) => {
45
+ const validatedArgs = GetExamResultSchema.parse(args);
46
+ const stored = examResultStore.get(validatedArgs.result_id);
47
+ if (!stored) {
48
+ return {
49
+ content: [
50
+ {
51
+ type: 'text',
52
+ text: `No stored result found for result_id "${validatedArgs.result_id}". Results are stored in-memory and may have been lost if the server restarted.`,
53
+ },
54
+ ],
55
+ isError: true,
56
+ };
57
+ }
58
+ let lines = stored.lines;
59
+ // Filter by section
60
+ if (validatedArgs.section) {
61
+ const typeMap = {
62
+ exam_results: 'exam_result',
63
+ logs: 'log',
64
+ summary: 'summary',
65
+ errors: 'error',
66
+ };
67
+ const targetType = typeMap[validatedArgs.section];
68
+ lines = lines.filter((line) => line.type === targetType);
69
+ }
70
+ // Filter by mirror_id
71
+ if (validatedArgs.mirror_id !== undefined) {
72
+ lines = lines.filter((line) => line.type !== 'exam_result' || line.mirror_id === validatedArgs.mirror_id);
73
+ }
74
+ let content = `**Exam Result Details**\n\n`;
75
+ content += `Result ID: ${stored.result_id}\n`;
76
+ content += `Mirrors: ${stored.mirror_ids.join(', ')}\n`;
77
+ content += `Exam Type: ${stored.exam_type}\n`;
78
+ content += `Runtime: ${stored.runtime_id}\n`;
79
+ content += `Stored At: ${stored.stored_at}\n`;
80
+ if (validatedArgs.section) {
81
+ content += `Section Filter: ${validatedArgs.section}\n`;
82
+ }
83
+ if (validatedArgs.mirror_id !== undefined) {
84
+ content += `Mirror Filter: ${validatedArgs.mirror_id}\n`;
85
+ }
86
+ content += `\n---\n\n`;
87
+ if (lines.length === 0) {
88
+ content += 'No matching lines found for the given filters.\n';
89
+ }
90
+ else {
91
+ for (const line of lines) {
92
+ content += JSON.stringify(line, null, 2) + '\n\n';
93
+ }
94
+ }
95
+ return { content: [{ type: 'text', text: content.trim() }] };
96
+ },
97
+ };
98
+ }
package/build/shared/src/tools/run-exam-for-mirror.js CHANGED
@@ -1,4 +1,5 @@
1
1
  import { z } from 'zod';
2
+ import { examResultStore } from '../exam-result-store.js';
2
3
  const PARAM_DESCRIPTIONS = {
3
4
  mirror_ids: 'Array of unofficial mirror IDs to run exams against. Mirrors without saved mcp_json configs will be skipped.',
4
5
  runtime_id: 'The Fly Machines runtime ID to use for running the exam containers (e.g., "fly-machines-v1")',
@@ -13,6 +14,46 @@ const RunExamForMirrorSchema = z.object({
13
14
  .describe(PARAM_DESCRIPTIONS.exam_type),
14
15
  max_retries: z.number().min(0).max(10).optional().describe(PARAM_DESCRIPTIONS.max_retries),
15
16
  });
17
+ /**
18
+ * Build a truncated summary of exam results for the LLM response.
19
+ * Omits large keys like full input schemas from tool listings to keep
20
+ * the payload within MCP size limits. The full result is accessible
21
+ * via the `get_exam_result` tool using the returned `result_id`.
22
+ */
23
+ function truncateExamResultData(data) {
24
+ const truncated = {};
25
+ for (const [key, value] of Object.entries(data)) {
26
+ if (key === 'tools' && Array.isArray(value)) {
27
+ // For tool listings, include only name and description, omit inputSchema
28
+ truncated[key] = value.map((tool) => ({
29
+ name: tool.name,
30
+ ...(tool.description ? { description: String(tool.description).slice(0, 100) } : {}),
31
+ }));
32
+ truncated['tools_count'] = value.length;
33
+ truncated['tools_truncated'] = true;
34
+ }
35
+ else if (key === 'inputSchema' || key === 'input_schema') {
36
+ // Omit full input schemas
37
+ truncated[key] = '(truncated — use get_exam_result to see full data)';
38
+ }
39
+ else if (typeof value === 'string' && value.length > 500) {
40
+ truncated[key] = value.slice(0, 500) + '... (truncated)';
41
+ }
42
+ else if (typeof value === 'object' && value !== null) {
43
+ const serialized = JSON.stringify(value);
44
+ if (serialized.length > 1000) {
45
+ truncated[key] = '(truncated — use get_exam_result to see full data)';
46
+ }
47
+ else {
48
+ truncated[key] = value;
49
+ }
50
+ }
51
+ else {
52
+ truncated[key] = value;
53
+ }
54
+ }
55
+ return truncated;
56
+ }
16
57
  export function runExamForMirror(_server, clientFactory) {
17
58
  return {
18
59
  name: 'run_exam_for_mirror',
@@ -23,7 +64,9 @@ Available exam types:
23
64
  - **init-tools-list**: Connects to the mirror and retrieves its list of MCP tools, verifying the server initializes properly
24
65
  - **both**: Runs both exams sequentially
25
66
 
26
- Mirrors without saved mcp_json configurations are automatically skipped. Results are returned as a stream of events including logs, exam results, and a final summary.
67
+ Mirrors without saved mcp_json configurations are automatically skipped.
68
+
69
+ Results are stored server-side and a \`result_id\` UUID is returned. The response includes a truncated summary (status, tool names/counts, errors) that fits within MCP size limits. Use \`get_exam_result\` to drill into full details, or pass the \`result_id\` directly to \`save_results_for_mirror\`.
27
70
 
28
71
  Use cases:
29
72
  - Test if an unofficial mirror's MCP server is working correctly before linking it
@@ -64,7 +107,10 @@ Use cases:
64
107
  exam_type: validatedArgs.exam_type,
65
108
  max_retries: validatedArgs.max_retries,
66
109
  });
110
+ // Store the full result server-side
111
+ const resultId = examResultStore.store(validatedArgs.mirror_ids, validatedArgs.runtime_id, validatedArgs.exam_type, response.lines);
67
112
  let content = `**Proctor Exam Results**\n\n`;
113
+ content += `Result ID: ${resultId}\n`;
68
114
  content += `Mirrors: ${validatedArgs.mirror_ids.join(', ')}\n`;
69
115
  content += `Exam Type: ${validatedArgs.exam_type}\n`;
70
116
  content += `Runtime: ${validatedArgs.runtime_id}\n\n`;
@@ -78,7 +124,8 @@ Use cases:
78
124
  content += ` Exam: ${line.exam_id || line.exam_type || 'unknown'}\n`;
79
125
  content += ` Status: ${line.status || 'unknown'}\n`;
80
126
  if (line.data) {
81
- content += ` Data: ${JSON.stringify(line.data, null, 2)}\n`;
127
+ const truncatedData = truncateExamResultData(line.data);
128
+ content += ` Data: ${JSON.stringify(truncatedData, null, 2)}\n`;
82
129
  }
83
130
  break;
84
131
  case 'summary':
@@ -95,6 +142,8 @@ Use cases:
95
142
  content += `${JSON.stringify(line)}\n`;
96
143
  }
97
144
  }
145
+ content += `\n\nUse \`get_exam_result\` with result_id "${resultId}" to see full untruncated data.`;
146
+ content += `\nUse \`save_results_for_mirror\` with result_id "${resultId}" to save these results.`;
98
147
  return { content: [{ type: 'text', text: content.trim() }] };
99
148
  }
100
149
  catch (error) {
package/build/shared/src/tools/save-results-for-mirror.js CHANGED
@@ -1,8 +1,10 @@
1
1
  import { z } from 'zod';
2
+ import { examResultStore } from '../exam-result-store.js';
2
3
  const PARAM_DESCRIPTIONS = {
3
4
  mirror_id: 'The ID of the unofficial mirror to save results for',
4
5
  runtime_id: 'The runtime ID that was used to run the exams',
5
- results: 'Array of exam results to save. Each result must include exam_id, status, and optional data.',
6
+ result_id: 'The UUID returned by run_exam_for_mirror. When provided, the server retrieves the full result from the in-memory store — no need to pass the results array. This is the preferred approach.',
7
+ results: 'Array of exam results to save. Each result must include exam_id, status, and optional data. Only needed if result_id is not provided.',
6
8
  exam_id: 'The exam identifier (e.g., "auth-check", "init-tools-list")',
7
9
  status: 'The result status (e.g., "pass", "fail", "error", "skip")',
8
10
  data: 'Optional detailed result data. Sensitive fields (tokens, secrets, passwords) will be automatically redacted before storage.',
@@ -12,29 +14,46 @@ const ResultSchema = z.object({
12
14
  status: z.string().describe(PARAM_DESCRIPTIONS.status),
13
15
  data: z.record(z.unknown()).optional().describe(PARAM_DESCRIPTIONS.data),
14
16
  });
15
- const SaveResultsForMirrorSchema = z.object({
17
+ const SaveResultsForMirrorSchema = z
18
+ .object({
16
19
  mirror_id: z.number().describe(PARAM_DESCRIPTIONS.mirror_id),
17
- runtime_id: z.string().describe(PARAM_DESCRIPTIONS.runtime_id),
18
- results: z.array(ResultSchema).min(1).describe(PARAM_DESCRIPTIONS.results),
20
+ runtime_id: z.string().optional().describe(PARAM_DESCRIPTIONS.runtime_id),
21
+ result_id: z.string().uuid().optional().describe(PARAM_DESCRIPTIONS.result_id),
22
+ results: z.array(ResultSchema).optional().describe(PARAM_DESCRIPTIONS.results),
23
+ })
24
+ .refine((data) => data.result_id || (data.results && data.results.length > 0), {
25
+ message: 'Either result_id or a non-empty results array must be provided',
26
+ })
27
+ .refine((data) => !(data.result_id && data.results && data.results.length > 0), {
28
+ message: 'Provide either result_id or results, not both. Use result_id (preferred) to retrieve from the store, or results for direct submission.',
19
29
  });
20
30
  export function saveResultsForMirror(_server, clientFactory) {
21
31
  return {
22
32
  name: 'save_results_for_mirror',
23
- description: `Save proctor exam results for an unofficial mirror. The results passed here must come exactly from a prior run_exam_for_mirror call — do not construct or modify results manually.
33
+ description: `Save proctor exam results for an unofficial mirror.
34
+
35
+ **Preferred**: Pass the \`result_id\` returned by \`run_exam_for_mirror\`. The full result is retrieved from the in-memory store server-side — no need to pass the large results payload through the LLM context.
36
+
37
+ **Fallback**: Pass results directly (as before) if result_id is not available.
24
38
 
25
39
  Results are sanitized server-side to redact sensitive data (OAuth tokens, client secrets, passwords, etc.) before being persisted.
26
40
 
27
41
  Supports partial success - if some results fail to save, successfully saved results are still returned along with error details for failures.
28
42
 
29
43
  Typical workflow:
30
- 1. Call run_exam_for_mirror to execute exams against a mirror
31
- 2. Extract the exam_result entries from the NDJSON response
32
- 3. Pass those results directly to this tool without modification`,
44
+ 1. Call run_exam_for_mirror — note the returned result_id
45
+ 2. Call save_results_for_mirror with result_id and mirror_id
46
+ 3. Results are saved without the LLM needing to parse or relay the full payload`,
33
47
  inputSchema: {
34
48
  type: 'object',
35
49
  properties: {
36
50
  mirror_id: { type: 'number', description: PARAM_DESCRIPTIONS.mirror_id },
37
51
  runtime_id: { type: 'string', description: PARAM_DESCRIPTIONS.runtime_id },
52
+ result_id: {
53
+ type: 'string',
54
+ format: 'uuid',
55
+ description: PARAM_DESCRIPTIONS.result_id,
56
+ },
38
57
  results: {
39
58
  type: 'array',
40
59
  items: {
@@ -50,24 +69,77 @@ Typical workflow:
50
69
  },
51
70
  required: ['exam_id', 'status'],
52
71
  },
53
- minItems: 1,
54
72
  description: PARAM_DESCRIPTIONS.results,
55
73
  },
56
74
  },
57
- required: ['mirror_id', 'runtime_id', 'results'],
75
+ required: ['mirror_id'],
58
76
  },
59
77
  handler: async (args) => {
60
78
  const validatedArgs = SaveResultsForMirrorSchema.parse(args);
61
79
  const client = clientFactory();
62
80
  try {
81
+ let results = validatedArgs.results;
82
+ let runtimeId = validatedArgs.runtime_id;
83
+ // If result_id is provided, retrieve from the store
84
+ if (validatedArgs.result_id) {
85
+ const stored = examResultStore.get(validatedArgs.result_id);
86
+ if (!stored) {
87
+ return {
88
+ content: [
89
+ {
90
+ type: 'text',
91
+ text: `No stored result found for result_id "${validatedArgs.result_id}". Results are stored in-memory and may have been lost if the server restarted. Pass the results array directly instead.`,
92
+ },
93
+ ],
94
+ isError: true,
95
+ };
96
+ }
97
+ // Extract exam_result lines from stored data
98
+ results = stored.lines
99
+ .filter((line) => line.type === 'exam_result')
100
+ .map((line) => ({
101
+ exam_id: (line.exam_id || line.exam_type || 'unknown'),
102
+ status: (line.status || 'unknown'),
103
+ ...(line.data ? { data: line.data } : {}),
104
+ }));
105
+ if (!runtimeId) {
106
+ runtimeId = stored.runtime_id;
107
+ }
108
+ }
109
+ if (!results || results.length === 0) {
110
+ return {
111
+ content: [
112
+ {
113
+ type: 'text',
114
+ text: 'No exam results to save. Either provide a result_id from run_exam_for_mirror or pass results directly.',
115
+ },
116
+ ],
117
+ isError: true,
118
+ };
119
+ }
120
+ if (!runtimeId) {
121
+ return {
122
+ content: [
123
+ {
124
+ type: 'text',
125
+ text: 'runtime_id is required. Provide it directly or use a result_id which includes the runtime_id.',
126
+ },
127
+ ],
128
+ isError: true,
129
+ };
130
+ }
63
131
  const response = await client.saveResultsForMirror({
64
132
  mirror_id: validatedArgs.mirror_id,
65
- runtime_id: validatedArgs.runtime_id,
66
- results: validatedArgs.results,
133
+ runtime_id: runtimeId,
134
+ results,
67
135
  });
68
136
  let content = `**Proctor Results Saved**\n\n`;
69
137
  content += `Mirror ID: ${validatedArgs.mirror_id}\n`;
70
- content += `Runtime: ${validatedArgs.runtime_id}\n\n`;
138
+ content += `Runtime: ${runtimeId}\n`;
139
+ if (validatedArgs.result_id) {
140
+ content += `Result ID: ${validatedArgs.result_id}\n`;
141
+ }
142
+ content += '\n';
71
143
  if (response.saved.length > 0) {
72
144
  content += `**Successfully Saved (${response.saved.length}):**\n`;
73
145
  for (const saved of response.saved) {
@@ -86,6 +158,10 @@ Typical workflow:
86
158
  }
87
159
  }
88
160
  }
161
+ // Clean up stored result after successful save (all results persisted)
162
+ if (validatedArgs.result_id && response.errors.length === 0) {
163
+ examResultStore.delete(validatedArgs.result_id);
164
+ }
89
165
  return { content: [{ type: 'text', text: content.trim() }] };
90
166
  }
91
167
  catch (error) {
package/build/shared/src/tools.js CHANGED
@@ -58,6 +58,7 @@ import { forceTriggerGoodJobCron } from './tools/force-trigger-good-job-cron.js'
58
58
  import { cleanupGoodJobs } from './tools/cleanup-good-jobs.js';
59
59
  // Proctor tools
60
60
  import { runExamForMirror } from './tools/run-exam-for-mirror.js';
61
+ import { getExamResult } from './tools/get-exam-result.js';
61
62
  import { saveResultsForMirror } from './tools/save-results-for-mirror.js';
62
63
  // Discovered URLs tools
63
64
  import { listDiscoveredUrls } from './tools/list-discovered-urls.js';
@@ -83,12 +84,13 @@ const ALL_TOOLS = [
83
84
  isWriteOperation: false,
84
85
  },
85
86
  { factory: saveMCPImplementation, groups: ['server_directory'], isWriteOperation: true },
87
+ { factory: findProviders, groups: ['server_directory'], isWriteOperation: false },
88
+ // Notification tools (separated from server_directory for isolation)
86
89
  {
87
90
  factory: sendMCPImplementationPostingNotification,
88
- groups: ['server_directory'],
91
+ groups: ['notifications'],
89
92
  isWriteOperation: true,
90
93
  },
91
- { factory: findProviders, groups: ['server_directory'], isWriteOperation: false },
92
94
  // Official mirror queue tools (also in server_directory)
93
95
  {
94
96
  factory: getOfficialMirrorQueueItems,
@@ -226,6 +228,7 @@ const ALL_TOOLS = [
226
228
  { factory: cleanupGoodJobs, groups: ['good_jobs'], isWriteOperation: true },
227
229
  // Proctor tools
228
230
  { factory: runExamForMirror, groups: ['proctor'], isWriteOperation: true },
231
+ { factory: getExamResult, groups: ['proctor'], isWriteOperation: false },
229
232
  { factory: saveResultsForMirror, groups: ['proctor'], isWriteOperation: true },
230
233
  // Discovered URLs tools
231
234
  { factory: listDiscoveredUrls, groups: ['discovered_urls'], isWriteOperation: false },
@@ -261,8 +264,10 @@ const VALID_TOOL_GROUPS = [
261
264
  'good_jobs',
262
265
  'good_jobs_readonly',
263
266
  'proctor',
267
+ 'proctor_readonly',
264
268
  'discovered_urls',
265
269
  'discovered_urls_readonly',
270
+ 'notifications',
266
271
  ];
267
272
  /**
268
273
  * Base groups (without _readonly suffix) - used for default "all groups" behavior
@@ -280,6 +285,7 @@ const BASE_TOOL_GROUPS = [
280
285
  'good_jobs',
281
286
  'proctor',
282
287
  'discovered_urls',
288
+ 'notifications',
283
289
  ];
284
290
  /**
285
291
  * Parse enabled tool groups from environment variable or parameter
@@ -360,9 +366,11 @@ function shouldIncludeTool(toolDef, enabledGroups) {
360
366
  * - redirects_readonly: URL redirect tools (read only)
361
367
  * - good_jobs: GoodJob background job management tools (read + write)
362
368
  * - good_jobs_readonly: GoodJob tools (read only)
363
- * - proctor: Proctor exam execution and result storage tools (write-only, no readonly variant)
369
+ * - proctor: Proctor exam execution and result storage tools (read + write)
370
+ * - proctor_readonly: Proctor tools (read only - get_exam_result for retrieving stored results)
364
371
  * - discovered_urls: Discovered URL management tools for processing URLs into MCP implementations (read + write)
365
372
  * - discovered_urls_readonly: Discovered URL tools (read only - list and stats)
373
+ * - notifications: Notification email tools - send_impl_posted_notif (write-only, no readonly variant)
366
374
  *
367
375
  * @param clientFactory - Factory function that creates client instances
368
376
  * @param enabledGroups - Optional comma-separated list of enabled tool groups (overrides env var)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pulsemcp-cms-admin-mcp-server",
3
- "version": "0.6.13",
3
+ "version": "0.7.0",
4
4
  "description": "Local implementation of PulseMCP CMS Admin MCP server",
5
5
  "mcpName": "com.pulsemcp.servers/pulsemcp-cms-admin",
6
6
  "main": "build/index.js",
package/shared/exam-result-store.d.ts ADDED
@@ -0,0 +1,37 @@
1
+ import type { ProctorExamStreamLine } from './types.js';
2
+ export interface StoredExamResult {
3
+ result_id: string;
4
+ mirror_ids: number[];
5
+ runtime_id: string;
6
+ exam_type: string;
7
+ lines: ProctorExamStreamLine[];
8
+ stored_at: string;
9
+ }
10
+ /**
11
+ * In-memory store for proctor exam results.
12
+ *
13
+ * When `run_exam_for_mirror` completes, the full result is stored here
14
+ * and a UUID `result_id` is returned. This avoids dumping large payloads
15
+ * (~60KB+ for servers with many tools) into the LLM context.
16
+ *
17
+ * Eviction: When the store exceeds MAX_RESULTS entries, the oldest result
18
+ * is evicted (FIFO). Results are also deleted after successful save via
19
+ * `save_results_for_mirror`.
20
+ *
21
+ * Consumers can:
22
+ * - Use `get_exam_result` to drill into the full result on demand
23
+ * - Pass `result_id` to `save_results_for_mirror` instead of the full payload
24
+ */
25
+ declare class ExamResultStore {
26
+ private results;
27
+ store(mirrorIds: number[], runtimeId: string, examType: string, lines: ProctorExamStreamLine[]): string;
28
+ get(resultId: string): StoredExamResult | undefined;
29
+ delete(resultId: string): boolean;
30
+ get size(): number;
31
+ /** For testing only */
32
+ clear(): void;
33
+ }
34
+ /** Singleton instance shared across all tool factories */
35
+ export declare const examResultStore: ExamResultStore;
36
+ export {};
37
+ //# sourceMappingURL=exam-result-store.d.ts.map
package/shared/exam-result-store.js ADDED
@@ -0,0 +1,59 @@
1
+ import { randomUUID } from 'crypto';
2
+ /**
3
+ * Maximum number of results to keep in memory. Oldest results are evicted
4
+ * when this limit is reached (FIFO). Each result can be 60KB+ for servers
5
+ * with many tools, so 100 entries ≈ 6MB worst case.
6
+ */
7
+ const MAX_RESULTS = 100;
8
+ /**
9
+ * In-memory store for proctor exam results.
10
+ *
11
+ * When `run_exam_for_mirror` completes, the full result is stored here
12
+ * and a UUID `result_id` is returned. This avoids dumping large payloads
13
+ * (~60KB+ for servers with many tools) into the LLM context.
14
+ *
15
+ * Eviction: When the store exceeds MAX_RESULTS entries, the oldest result
16
+ * is evicted (FIFO). Results are also deleted after successful save via
17
+ * `save_results_for_mirror`.
18
+ *
19
+ * Consumers can:
20
+ * - Use `get_exam_result` to drill into the full result on demand
21
+ * - Pass `result_id` to `save_results_for_mirror` instead of the full payload
22
+ */
23
+ class ExamResultStore {
24
+ results = new Map();
25
+ store(mirrorIds, runtimeId, examType, lines) {
26
+ const resultId = randomUUID();
27
+ // Evict oldest entries if at capacity (Map preserves insertion order)
28
+ while (this.results.size >= MAX_RESULTS) {
29
+ const oldestKey = this.results.keys().next().value;
30
+ if (oldestKey !== undefined) {
31
+ this.results.delete(oldestKey);
32
+ }
33
+ }
34
+ this.results.set(resultId, {
35
+ result_id: resultId,
36
+ mirror_ids: mirrorIds,
37
+ runtime_id: runtimeId,
38
+ exam_type: examType,
39
+ lines,
40
+ stored_at: new Date().toISOString(),
41
+ });
42
+ return resultId;
43
+ }
44
+ get(resultId) {
45
+ return this.results.get(resultId);
46
+ }
47
+ delete(resultId) {
48
+ return this.results.delete(resultId);
49
+ }
50
+ get size() {
51
+ return this.results.size;
52
+ }
53
+ /** For testing only */
54
+ clear() {
55
+ this.results.clear();
56
+ }
57
+ }
58
+ /** Singleton instance shared across all tool factories */
59
+ export const examResultStore = new ExamResultStore();
package/shared/tools/get-exam-result.d.ts ADDED
@@ -0,0 +1,40 @@
1
+ import { Server } from '@modelcontextprotocol/sdk/server/index.js';
2
+ import type { ClientFactory } from '../server.js';
3
+ export declare function getExamResult(_server: Server, _clientFactory: ClientFactory): {
4
+ name: string;
5
+ description: string;
6
+ inputSchema: {
7
+ type: string;
8
+ properties: {
9
+ result_id: {
10
+ type: string;
11
+ format: string;
12
+ description: "The UUID returned by run_exam_for_mirror identifying the stored result to retrieve.";
13
+ };
14
+ section: {
15
+ type: string;
16
+ enum: string[];
17
+ description: "Optional filter to retrieve only a specific section of the result. \"exam_results\" returns only exam_result lines, \"logs\" returns only log lines, \"summary\" returns only the summary line, \"errors\" returns only error lines. If omitted, returns all lines.";
18
+ };
19
+ mirror_id: {
20
+ type: string;
21
+ description: "Optional mirror ID filter. When provided, only returns exam_result lines for this specific mirror. Useful when the exam was run against multiple mirrors.";
22
+ };
23
+ };
24
+ required: string[];
25
+ };
26
+ handler: (args: unknown) => Promise<{
27
+ content: {
28
+ type: string;
29
+ text: string;
30
+ }[];
31
+ isError: boolean;
32
+ } | {
33
+ content: {
34
+ type: string;
35
+ text: string;
36
+ }[];
37
+ isError?: undefined;
38
+ }>;
39
+ };
40
+ //# sourceMappingURL=get-exam-result.d.ts.map
package/shared/tools/get-exam-result.js ADDED
@@ -0,0 +1,98 @@
1
+ import { z } from 'zod';
2
+ import { examResultStore } from '../exam-result-store.js';
3
+ const PARAM_DESCRIPTIONS = {
4
+ result_id: 'The UUID returned by run_exam_for_mirror identifying the stored result to retrieve.',
5
+ section: 'Optional filter to retrieve only a specific section of the result. "exam_results" returns only exam_result lines, "logs" returns only log lines, "summary" returns only the summary line, "errors" returns only error lines. If omitted, returns all lines.',
6
+ mirror_id: 'Optional mirror ID filter. When provided, only returns exam_result lines for this specific mirror. Useful when the exam was run against multiple mirrors.',
7
+ };
8
+ const GetExamResultSchema = z.object({
9
+ result_id: z.string().uuid().describe(PARAM_DESCRIPTIONS.result_id),
10
+ section: z
11
+ .enum(['exam_results', 'logs', 'summary', 'errors'])
12
+ .optional()
13
+ .describe(PARAM_DESCRIPTIONS.section),
14
+ mirror_id: z.number().optional().describe(PARAM_DESCRIPTIONS.mirror_id),
15
+ });
16
+ export function getExamResult(_server, _clientFactory) {
17
+ return {
18
+ name: 'get_exam_result',
19
+ description: `Retrieve the full, untruncated proctor exam result stored by a prior run_exam_for_mirror call.
20
+
21
+ run_exam_for_mirror returns a truncated summary to keep the response within MCP size limits. This tool provides on-demand access to the complete result data, including full tool input schemas and detailed exam output.
22
+
23
+ Supports filtering by section (exam_results, logs, summary, errors) and by mirror_id to drill into specific parts of the result without loading the entire payload.
24
+
25
+ **Tip**: For servers with many tools, the full result can be very large. Use the section and/or mirror_id filters to retrieve only the data you need.
26
+
27
+ Typical usage:
28
+ 1. Call run_exam_for_mirror — note the returned result_id
29
+ 2. Call get_exam_result with that result_id and a section filter (e.g., section="exam_results")
30
+ 3. Optionally add mirror_id to narrow further for multi-mirror exams`,
31
+ inputSchema: {
32
+ type: 'object',
33
+ properties: {
34
+ result_id: { type: 'string', format: 'uuid', description: PARAM_DESCRIPTIONS.result_id },
35
+ section: {
36
+ type: 'string',
37
+ enum: ['exam_results', 'logs', 'summary', 'errors'],
38
+ description: PARAM_DESCRIPTIONS.section,
39
+ },
40
+ mirror_id: { type: 'number', description: PARAM_DESCRIPTIONS.mirror_id },
41
+ },
42
+ required: ['result_id'],
43
+ },
44
+ handler: async (args) => {
45
+ const validatedArgs = GetExamResultSchema.parse(args);
46
+ const stored = examResultStore.get(validatedArgs.result_id);
47
+ if (!stored) {
48
+ return {
49
+ content: [
50
+ {
51
+ type: 'text',
52
+ text: `No stored result found for result_id "${validatedArgs.result_id}". Results are stored in-memory and may have been lost if the server restarted.`,
53
+ },
54
+ ],
55
+ isError: true,
56
+ };
57
+ }
58
+ let lines = stored.lines;
59
+ // Filter by section
60
+ if (validatedArgs.section) {
61
+ const typeMap = {
62
+ exam_results: 'exam_result',
63
+ logs: 'log',
64
+ summary: 'summary',
65
+ errors: 'error',
66
+ };
67
+ const targetType = typeMap[validatedArgs.section];
68
+ lines = lines.filter((line) => line.type === targetType);
69
+ }
70
+ // Filter by mirror_id
71
+ if (validatedArgs.mirror_id !== undefined) {
72
+ lines = lines.filter((line) => line.type !== 'exam_result' || line.mirror_id === validatedArgs.mirror_id);
73
+ }
74
+ let content = `**Exam Result Details**\n\n`;
75
+ content += `Result ID: ${stored.result_id}\n`;
76
+ content += `Mirrors: ${stored.mirror_ids.join(', ')}\n`;
77
+ content += `Exam Type: ${stored.exam_type}\n`;
78
+ content += `Runtime: ${stored.runtime_id}\n`;
79
+ content += `Stored At: ${stored.stored_at}\n`;
80
+ if (validatedArgs.section) {
81
+ content += `Section Filter: ${validatedArgs.section}\n`;
82
+ }
83
+ if (validatedArgs.mirror_id !== undefined) {
84
+ content += `Mirror Filter: ${validatedArgs.mirror_id}\n`;
85
+ }
86
+ content += `\n---\n\n`;
87
+ if (lines.length === 0) {
88
+ content += 'No matching lines found for the given filters.\n';
89
+ }
90
+ else {
91
+ for (const line of lines) {
92
+ content += JSON.stringify(line, null, 2) + '\n\n';
93
+ }
94
+ }
95
+ return { content: [{ type: 'text', text: content.trim() }] };
96
+ },
97
+ };
98
+ }
package/shared/tools/run-exam-for-mirror.js CHANGED
@@ -1,4 +1,5 @@
1
1
  import { z } from 'zod';
2
+ import { examResultStore } from '../exam-result-store.js';
2
3
  const PARAM_DESCRIPTIONS = {
3
4
  mirror_ids: 'Array of unofficial mirror IDs to run exams against. Mirrors without saved mcp_json configs will be skipped.',
4
5
  runtime_id: 'The Fly Machines runtime ID to use for running the exam containers (e.g., "fly-machines-v1")',
@@ -13,6 +14,46 @@ const RunExamForMirrorSchema = z.object({
13
14
  .describe(PARAM_DESCRIPTIONS.exam_type),
14
15
  max_retries: z.number().min(0).max(10).optional().describe(PARAM_DESCRIPTIONS.max_retries),
15
16
  });
17
+ /**
18
+ * Build a truncated summary of exam results for the LLM response.
19
+ * Omits large keys like full input schemas from tool listings to keep
20
+ * the payload within MCP size limits. The full result is accessible
21
+ * via the `get_exam_result` tool using the returned `result_id`.
22
+ */
23
+ function truncateExamResultData(data) {
24
+ const truncated = {};
25
+ for (const [key, value] of Object.entries(data)) {
26
+ if (key === 'tools' && Array.isArray(value)) {
27
+ // For tool listings, include only name and description, omit inputSchema
28
+ truncated[key] = value.map((tool) => ({
29
+ name: tool.name,
30
+ ...(tool.description ? { description: String(tool.description).slice(0, 100) } : {}),
31
+ }));
32
+ truncated['tools_count'] = value.length;
33
+ truncated['tools_truncated'] = true;
34
+ }
35
+ else if (key === 'inputSchema' || key === 'input_schema') {
36
+ // Omit full input schemas
37
+ truncated[key] = '(truncated — use get_exam_result to see full data)';
38
+ }
39
+ else if (typeof value === 'string' && value.length > 500) {
40
+ truncated[key] = value.slice(0, 500) + '... (truncated)';
41
+ }
42
+ else if (typeof value === 'object' && value !== null) {
43
+ const serialized = JSON.stringify(value);
44
+ if (serialized.length > 1000) {
45
+ truncated[key] = '(truncated — use get_exam_result to see full data)';
46
+ }
47
+ else {
48
+ truncated[key] = value;
49
+ }
50
+ }
51
+ else {
52
+ truncated[key] = value;
53
+ }
54
+ }
55
+ return truncated;
56
+ }
16
57
  export function runExamForMirror(_server, clientFactory) {
17
58
  return {
18
59
  name: 'run_exam_for_mirror',
@@ -23,7 +64,9 @@ Available exam types:
23
64
  - **init-tools-list**: Connects to the mirror and retrieves its list of MCP tools, verifying the server initializes properly
24
65
  - **both**: Runs both exams sequentially
25
66
 
26
- Mirrors without saved mcp_json configurations are automatically skipped. Results are returned as a stream of events including logs, exam results, and a final summary.
67
+ Mirrors without saved mcp_json configurations are automatically skipped.
68
+
69
+ Results are stored server-side and a \`result_id\` UUID is returned. The response includes a truncated summary (status, tool names/counts, errors) that fits within MCP size limits. Use \`get_exam_result\` to drill into full details, or pass the \`result_id\` directly to \`save_results_for_mirror\`.
27
70
 
28
71
  Use cases:
29
72
  - Test if an unofficial mirror's MCP server is working correctly before linking it
@@ -64,7 +107,10 @@ Use cases:
64
107
  exam_type: validatedArgs.exam_type,
65
108
  max_retries: validatedArgs.max_retries,
66
109
  });
110
+ // Store the full result server-side
111
+ const resultId = examResultStore.store(validatedArgs.mirror_ids, validatedArgs.runtime_id, validatedArgs.exam_type, response.lines);
67
112
  let content = `**Proctor Exam Results**\n\n`;
113
+ content += `Result ID: ${resultId}\n`;
68
114
  content += `Mirrors: ${validatedArgs.mirror_ids.join(', ')}\n`;
69
115
  content += `Exam Type: ${validatedArgs.exam_type}\n`;
70
116
  content += `Runtime: ${validatedArgs.runtime_id}\n\n`;
@@ -78,7 +124,8 @@ Use cases:
78
124
  content += ` Exam: ${line.exam_id || line.exam_type || 'unknown'}\n`;
79
125
  content += ` Status: ${line.status || 'unknown'}\n`;
80
126
  if (line.data) {
81
- content += ` Data: ${JSON.stringify(line.data, null, 2)}\n`;
127
+ const truncatedData = truncateExamResultData(line.data);
128
+ content += ` Data: ${JSON.stringify(truncatedData, null, 2)}\n`;
82
129
  }
83
130
  break;
84
131
  case 'summary':
@@ -95,6 +142,8 @@ Use cases:
95
142
  content += `${JSON.stringify(line)}\n`;
96
143
  }
97
144
  }
145
+ content += `\n\nUse \`get_exam_result\` with result_id "${resultId}" to see full untruncated data.`;
146
+ content += `\nUse \`save_results_for_mirror\` with result_id "${resultId}" to save these results.`;
98
147
  return { content: [{ type: 'text', text: content.trim() }] };
99
148
  }
100
149
  catch (error) {
package/shared/tools/save-results-for-mirror.d.ts CHANGED
@@ -14,6 +14,11 @@ export declare function saveResultsForMirror(_server: Server, clientFactory: Cli
14
14
  type: string;
15
15
  description: "The runtime ID that was used to run the exams";
16
16
  };
17
+ result_id: {
18
+ type: string;
19
+ format: string;
20
+ description: "The UUID returned by run_exam_for_mirror. When provided, the server retrieves the full result from the in-memory store — no need to pass the results array. This is the preferred approach.";
21
+ };
17
22
  results: {
18
23
  type: string;
19
24
  items: {
@@ -35,8 +40,7 @@ export declare function saveResultsForMirror(_server: Server, clientFactory: Cli
35
40
  };
36
41
  required: string[];
37
42
  };
38
- minItems: number;
39
- description: "Array of exam results to save. Each result must include exam_id, status, and optional data.";
43
+ description: "Array of exam results to save. Each result must include exam_id, status, and optional data. Only needed if result_id is not provided.";
40
44
  };
41
45
  };
42
46
  required: string[];
@@ -46,13 +50,13 @@ export declare function saveResultsForMirror(_server: Server, clientFactory: Cli
46
50
  type: string;
47
51
  text: string;
48
52
  }[];
49
- isError?: undefined;
53
+ isError: boolean;
50
54
  } | {
51
55
  content: {
52
56
  type: string;
53
57
  text: string;
54
58
  }[];
55
- isError: boolean;
59
+ isError?: undefined;
56
60
  }>;
57
61
  };
58
62
  //# sourceMappingURL=save-results-for-mirror.d.ts.map
package/shared/tools/save-results-for-mirror.js CHANGED
@@ -1,8 +1,10 @@
1
1
  import { z } from 'zod';
2
+ import { examResultStore } from '../exam-result-store.js';
2
3
  const PARAM_DESCRIPTIONS = {
3
4
  mirror_id: 'The ID of the unofficial mirror to save results for',
4
5
  runtime_id: 'The runtime ID that was used to run the exams',
5
- results: 'Array of exam results to save. Each result must include exam_id, status, and optional data.',
6
+ result_id: 'The UUID returned by run_exam_for_mirror. When provided, the server retrieves the full result from the in-memory store — no need to pass the results array. This is the preferred approach.',
7
+ results: 'Array of exam results to save. Each result must include exam_id, status, and optional data. Only needed if result_id is not provided.',
6
8
  exam_id: 'The exam identifier (e.g., "auth-check", "init-tools-list")',
7
9
  status: 'The result status (e.g., "pass", "fail", "error", "skip")',
8
10
  data: 'Optional detailed result data. Sensitive fields (tokens, secrets, passwords) will be automatically redacted before storage.',
@@ -12,29 +14,46 @@ const ResultSchema = z.object({
12
14
  status: z.string().describe(PARAM_DESCRIPTIONS.status),
13
15
  data: z.record(z.unknown()).optional().describe(PARAM_DESCRIPTIONS.data),
14
16
  });
15
- const SaveResultsForMirrorSchema = z.object({
17
+ const SaveResultsForMirrorSchema = z
18
+ .object({
16
19
  mirror_id: z.number().describe(PARAM_DESCRIPTIONS.mirror_id),
17
- runtime_id: z.string().describe(PARAM_DESCRIPTIONS.runtime_id),
18
- results: z.array(ResultSchema).min(1).describe(PARAM_DESCRIPTIONS.results),
20
+ runtime_id: z.string().optional().describe(PARAM_DESCRIPTIONS.runtime_id),
21
+ result_id: z.string().uuid().optional().describe(PARAM_DESCRIPTIONS.result_id),
22
+ results: z.array(ResultSchema).optional().describe(PARAM_DESCRIPTIONS.results),
23
+ })
24
+ .refine((data) => data.result_id || (data.results && data.results.length > 0), {
25
+ message: 'Either result_id or a non-empty results array must be provided',
26
+ })
27
+ .refine((data) => !(data.result_id && data.results && data.results.length > 0), {
28
+ message: 'Provide either result_id or results, not both. Use result_id (preferred) to retrieve from the store, or results for direct submission.',
19
29
  });
20
30
  export function saveResultsForMirror(_server, clientFactory) {
21
31
  return {
22
32
  name: 'save_results_for_mirror',
23
- description: `Save proctor exam results for an unofficial mirror. The results passed here must come exactly from a prior run_exam_for_mirror call — do not construct or modify results manually.
33
+ description: `Save proctor exam results for an unofficial mirror.
34
+
35
+ **Preferred**: Pass the \`result_id\` returned by \`run_exam_for_mirror\`. The full result is retrieved from the in-memory store server-side — no need to pass the large results payload through the LLM context.
36
+
37
+ **Fallback**: Pass results directly (as before) if result_id is not available.
24
38
 
25
39
  Results are sanitized server-side to redact sensitive data (OAuth tokens, client secrets, passwords, etc.) before being persisted.
26
40
 
27
41
  Supports partial success - if some results fail to save, successfully saved results are still returned along with error details for failures.
28
42
 
29
43
  Typical workflow:
30
- 1. Call run_exam_for_mirror to execute exams against a mirror
31
- 2. Extract the exam_result entries from the NDJSON response
32
- 3. Pass those results directly to this tool without modification`,
44
+ 1. Call run_exam_for_mirror — note the returned result_id
45
+ 2. Call save_results_for_mirror with result_id and mirror_id
46
+ 3. Results are saved without the LLM needing to parse or relay the full payload`,
33
47
  inputSchema: {
34
48
  type: 'object',
35
49
  properties: {
36
50
  mirror_id: { type: 'number', description: PARAM_DESCRIPTIONS.mirror_id },
37
51
  runtime_id: { type: 'string', description: PARAM_DESCRIPTIONS.runtime_id },
52
+ result_id: {
53
+ type: 'string',
54
+ format: 'uuid',
55
+ description: PARAM_DESCRIPTIONS.result_id,
56
+ },
38
57
  results: {
39
58
  type: 'array',
40
59
  items: {
@@ -50,24 +69,77 @@ Typical workflow:
50
69
  },
51
70
  required: ['exam_id', 'status'],
52
71
  },
53
- minItems: 1,
54
72
  description: PARAM_DESCRIPTIONS.results,
55
73
  },
56
74
  },
57
- required: ['mirror_id', 'runtime_id', 'results'],
75
+ required: ['mirror_id'],
58
76
  },
59
77
  handler: async (args) => {
60
78
  const validatedArgs = SaveResultsForMirrorSchema.parse(args);
61
79
  const client = clientFactory();
62
80
  try {
81
+ let results = validatedArgs.results;
82
+ let runtimeId = validatedArgs.runtime_id;
83
+ // If result_id is provided, retrieve from the store
84
+ if (validatedArgs.result_id) {
85
+ const stored = examResultStore.get(validatedArgs.result_id);
86
+ if (!stored) {
87
+ return {
88
+ content: [
89
+ {
90
+ type: 'text',
91
+ text: `No stored result found for result_id "${validatedArgs.result_id}". Results are stored in-memory and may have been lost if the server restarted. Pass the results array directly instead.`,
92
+ },
93
+ ],
94
+ isError: true,
95
+ };
96
+ }
97
+ // Extract exam_result lines from stored data
98
+ results = stored.lines
99
+ .filter((line) => line.type === 'exam_result')
100
+ .map((line) => ({
101
+ exam_id: (line.exam_id || line.exam_type || 'unknown'),
102
+ status: (line.status || 'unknown'),
103
+ ...(line.data ? { data: line.data } : {}),
104
+ }));
105
+ if (!runtimeId) {
106
+ runtimeId = stored.runtime_id;
107
+ }
108
+ }
109
+ if (!results || results.length === 0) {
110
+ return {
111
+ content: [
112
+ {
113
+ type: 'text',
114
+ text: 'No exam results to save. Either provide a result_id from run_exam_for_mirror or pass results directly.',
115
+ },
116
+ ],
117
+ isError: true,
118
+ };
119
+ }
120
+ if (!runtimeId) {
121
+ return {
122
+ content: [
123
+ {
124
+ type: 'text',
125
+ text: 'runtime_id is required. Provide it directly or use a result_id which includes the runtime_id.',
126
+ },
127
+ ],
128
+ isError: true,
129
+ };
130
+ }
63
131
  const response = await client.saveResultsForMirror({
64
132
  mirror_id: validatedArgs.mirror_id,
65
- runtime_id: validatedArgs.runtime_id,
66
- results: validatedArgs.results,
133
+ runtime_id: runtimeId,
134
+ results,
67
135
  });
68
136
  let content = `**Proctor Results Saved**\n\n`;
69
137
  content += `Mirror ID: ${validatedArgs.mirror_id}\n`;
70
- content += `Runtime: ${validatedArgs.runtime_id}\n\n`;
138
+ content += `Runtime: ${runtimeId}\n`;
139
+ if (validatedArgs.result_id) {
140
+ content += `Result ID: ${validatedArgs.result_id}\n`;
141
+ }
142
+ content += '\n';
71
143
  if (response.saved.length > 0) {
72
144
  content += `**Successfully Saved (${response.saved.length}):**\n`;
73
145
  for (const saved of response.saved) {
@@ -86,6 +158,10 @@ Typical workflow:
86
158
  }
87
159
  }
88
160
  }
161
+ // Clean up stored result after successful save (all results persisted)
162
+ if (validatedArgs.result_id && response.errors.length === 0) {
163
+ examResultStore.delete(validatedArgs.result_id);
164
+ }
89
165
  return { content: [{ type: 'text', text: content.trim() }] };
90
166
  }
91
167
  catch (error) {
package/shared/tools.d.ts CHANGED
@@ -23,10 +23,11 @@ import { ClientFactory } from './server.js';
23
23
  * - mcp_servers / mcp_servers_readonly: Unified MCP server tools (abstracted interface)
24
24
  * - redirects / redirects_readonly: URL redirect management tools
25
25
  * - good_jobs / good_jobs_readonly: GoodJob background job management tools
26
- * - proctor: Proctor exam execution and result storage tools (write-only, no readonly variant since both tools trigger side effects)
26
+ * - proctor / proctor_readonly: Proctor exam execution and result storage tools. The readonly variant includes get_exam_result for retrieving stored results without running exams or saving
27
27
  * - discovered_urls / discovered_urls_readonly: Discovered URL management tools for processing URLs into MCP implementations
28
+ * - notifications: Notification email tools (send_impl_posted_notif). Separated from server_directory so notification capability can be granted independently.
28
29
  */
29
- export type ToolGroup = 'newsletter' | 'newsletter_readonly' | 'server_directory' | 'server_directory_readonly' | 'official_queue' | 'official_queue_readonly' | 'unofficial_mirrors' | 'unofficial_mirrors_readonly' | 'official_mirrors' | 'official_mirrors_readonly' | 'tenants' | 'tenants_readonly' | 'mcp_jsons' | 'mcp_jsons_readonly' | 'mcp_servers' | 'mcp_servers_readonly' | 'redirects' | 'redirects_readonly' | 'good_jobs' | 'good_jobs_readonly' | 'proctor' | 'discovered_urls' | 'discovered_urls_readonly';
30
+ export type ToolGroup = 'newsletter' | 'newsletter_readonly' | 'server_directory' | 'server_directory_readonly' | 'official_queue' | 'official_queue_readonly' | 'unofficial_mirrors' | 'unofficial_mirrors_readonly' | 'official_mirrors' | 'official_mirrors_readonly' | 'tenants' | 'tenants_readonly' | 'mcp_jsons' | 'mcp_jsons_readonly' | 'mcp_servers' | 'mcp_servers_readonly' | 'redirects' | 'redirects_readonly' | 'good_jobs' | 'good_jobs_readonly' | 'proctor' | 'proctor_readonly' | 'discovered_urls' | 'discovered_urls_readonly' | 'notifications';
30
31
  /**
31
32
  * Parse enabled tool groups from environment variable or parameter
32
33
  * @param enabledGroupsParam - Comma-separated list of tool groups (e.g., "newsletter,server_directory_readonly")
@@ -67,9 +68,11 @@ export declare function parseEnabledToolGroups(enabledGroupsParam?: string): Too
67
68
  * - redirects_readonly: URL redirect tools (read only)
68
69
  * - good_jobs: GoodJob background job management tools (read + write)
69
70
  * - good_jobs_readonly: GoodJob tools (read only)
70
- * - proctor: Proctor exam execution and result storage tools (write-only, no readonly variant)
71
+ * - proctor: Proctor exam execution and result storage tools (read + write)
72
+ * - proctor_readonly: Proctor tools (read only - get_exam_result for retrieving stored results)
71
73
  * - discovered_urls: Discovered URL management tools for processing URLs into MCP implementations (read + write)
72
74
  * - discovered_urls_readonly: Discovered URL tools (read only - list and stats)
75
+ * - notifications: Notification email tools - send_impl_posted_notif (write-only, no readonly variant)
73
76
  *
74
77
  * @param clientFactory - Factory function that creates client instances
75
78
  * @param enabledGroups - Optional comma-separated list of enabled tool groups (overrides env var)
package/shared/tools.js CHANGED
@@ -58,6 +58,7 @@ import { forceTriggerGoodJobCron } from './tools/force-trigger-good-job-cron.js'
58
58
  import { cleanupGoodJobs } from './tools/cleanup-good-jobs.js';
59
59
  // Proctor tools
60
60
  import { runExamForMirror } from './tools/run-exam-for-mirror.js';
61
+ import { getExamResult } from './tools/get-exam-result.js';
61
62
  import { saveResultsForMirror } from './tools/save-results-for-mirror.js';
62
63
  // Discovered URLs tools
63
64
  import { listDiscoveredUrls } from './tools/list-discovered-urls.js';
@@ -83,12 +84,13 @@ const ALL_TOOLS = [
83
84
  isWriteOperation: false,
84
85
  },
85
86
  { factory: saveMCPImplementation, groups: ['server_directory'], isWriteOperation: true },
87
+ { factory: findProviders, groups: ['server_directory'], isWriteOperation: false },
88
+ // Notification tools (separated from server_directory for isolation)
86
89
  {
87
90
  factory: sendMCPImplementationPostingNotification,
88
- groups: ['server_directory'],
91
+ groups: ['notifications'],
89
92
  isWriteOperation: true,
90
93
  },
91
- { factory: findProviders, groups: ['server_directory'], isWriteOperation: false },
92
94
  // Official mirror queue tools (also in server_directory)
93
95
  {
94
96
  factory: getOfficialMirrorQueueItems,
@@ -226,6 +228,7 @@ const ALL_TOOLS = [
226
228
  { factory: cleanupGoodJobs, groups: ['good_jobs'], isWriteOperation: true },
227
229
  // Proctor tools
228
230
  { factory: runExamForMirror, groups: ['proctor'], isWriteOperation: true },
231
+ { factory: getExamResult, groups: ['proctor'], isWriteOperation: false },
229
232
  { factory: saveResultsForMirror, groups: ['proctor'], isWriteOperation: true },
230
233
  // Discovered URLs tools
231
234
  { factory: listDiscoveredUrls, groups: ['discovered_urls'], isWriteOperation: false },
@@ -261,8 +264,10 @@ const VALID_TOOL_GROUPS = [
261
264
  'good_jobs',
262
265
  'good_jobs_readonly',
263
266
  'proctor',
267
+ 'proctor_readonly',
264
268
  'discovered_urls',
265
269
  'discovered_urls_readonly',
270
+ 'notifications',
266
271
  ];
267
272
  /**
268
273
  * Base groups (without _readonly suffix) - used for default "all groups" behavior
@@ -280,6 +285,7 @@ const BASE_TOOL_GROUPS = [
280
285
  'good_jobs',
281
286
  'proctor',
282
287
  'discovered_urls',
288
+ 'notifications',
283
289
  ];
284
290
  /**
285
291
  * Parse enabled tool groups from environment variable or parameter
@@ -360,9 +366,11 @@ function shouldIncludeTool(toolDef, enabledGroups) {
360
366
  * - redirects_readonly: URL redirect tools (read only)
361
367
  * - good_jobs: GoodJob background job management tools (read + write)
362
368
  * - good_jobs_readonly: GoodJob tools (read only)
363
- * - proctor: Proctor exam execution and result storage tools (write-only, no readonly variant)
369
+ * - proctor: Proctor exam execution and result storage tools (read + write)
370
+ * - proctor_readonly: Proctor tools (read only - get_exam_result for retrieving stored results)
364
371
  * - discovered_urls: Discovered URL management tools for processing URLs into MCP implementations (read + write)
365
372
  * - discovered_urls_readonly: Discovered URL tools (read only - list and stats)
373
+ * - notifications: Notification email tools - send_impl_posted_notif (write-only, no readonly variant)
366
374
  *
367
375
  * @param clientFactory - Factory function that creates client instances
368
376
  * @param enabledGroups - Optional comma-separated list of enabled tool groups (overrides env var)