@jaypie/mcp 0.2.10 → 0.3.0

package/prompts/Jaypie_MCP_Package.md

@@ -0,0 +1,249 @@
+---
+description: MCP server package with documentation, AWS, Datadog, and LLM tools
+include: "packages/mcp/**"
+---
+
+# @jaypie/mcp Package
+
+The `@jaypie/mcp` package provides a Model Context Protocol (MCP) server for AI agents working with Jaypie projects. It includes tools for accessing documentation, AWS CLI operations, Datadog observability, and LLM debugging.
+
+## Package Structure
+
+```
+packages/mcp/
+├── src/
+│   ├── index.ts              # CLI entrypoint, exports createMcpServer and mcpExpressHandler
+│   ├── createMcpServer.ts    # Core MCP server factory with tool definitions
+│   ├── mcpExpressHandler.ts  # Express middleware for HTTP transport
+│   ├── aws.ts                # AWS CLI integration (spawn-based)
+│   ├── datadog.ts            # Datadog API integration (https-based)
+│   └── llm.ts                # LLM debug utilities
+├── prompts/                  # Markdown guides served via list_prompts/read_prompt tools
+└── dist/                     # Built output
+```
+
+## Usage
+
+### CLI (stdio transport)
+
+```bash
+npx jaypie-mcp          # Run MCP server via stdio
+npx jaypie-mcp --verbose # Run with debug logging
+```
+
+### Express Integration (HTTP transport)
+
+```typescript
+import express from "express";
+import { mcpExpressHandler } from "@jaypie/mcp";
+
+const app = express();
+app.use(express.json());
+app.use("/mcp", await mcpExpressHandler({ version: "1.0.0" }));
+```
+
+### Direct Server Creation
+
+```typescript
+import { createMcpServer } from "@jaypie/mcp";
+
+const server = createMcpServer({ version: "1.0.0", verbose: true });
+```
+
+## MCP Tools (27 total)
+
+### Documentation Tools (3)
+
+| Tool | Description |
+|------|-------------|
+| `list_prompts` | Lists all `.md` files in `prompts/` with descriptions and required file patterns |
+| `read_prompt` | Returns content of a specific prompt file |
+| `version` | Returns package version string |
+
+### AWS CLI Tools (16)
+
+Requires AWS CLI installed and configured. All tools accept optional `profile` and `region` parameters.
+
+| Tool | Description |
+|------|-------------|
+| `aws_list_profiles` | List available AWS profiles from ~/.aws/config and credentials |
+| `aws_stepfunctions_list_executions` | List Step Function executions for a state machine |
+| `aws_stepfunctions_stop_execution` | Stop a running Step Function execution |
+| `aws_lambda_list_functions` | List Lambda functions with optional prefix filtering |
+| `aws_lambda_get_function` | Get configuration and details for a specific Lambda function |
+| `aws_logs_filter_log_events` | Search CloudWatch Logs with pattern and time range filtering |
+| `aws_s3_list_objects` | List objects in an S3 bucket with optional prefix filtering |
+| `aws_cloudformation_describe_stack` | Get details and status of a CloudFormation stack |
+| `aws_dynamodb_describe_table` | Get metadata about a DynamoDB table |
+| `aws_dynamodb_scan` | Scan a DynamoDB table (use sparingly on large tables) |
+| `aws_dynamodb_query` | Query a DynamoDB table by partition key |
+| `aws_dynamodb_get_item` | Get a single item from a DynamoDB table by primary key |
+| `aws_sqs_list_queues` | List SQS queues with optional prefix filtering |
+| `aws_sqs_get_queue_attributes` | Get queue attributes including message counts |
+| `aws_sqs_receive_message` | Peek at messages in an SQS queue (does not delete) |
+| `aws_sqs_purge_queue` | Delete all messages from an SQS queue (irreversible) |
+
+### Datadog Tools (6)
+
+Requires `DATADOG_API_KEY` and `DATADOG_APP_KEY` environment variables.
+
+| Tool | Description |
+|------|-------------|
+| `datadog_logs` | Search individual log entries |
+| `datadog_log_analytics` | Aggregate logs with groupBy operations |
+| `datadog_monitors` | List and filter monitors by status/tags |
+| `datadog_synthetics` | List synthetic tests or get results for a specific test |
+| `datadog_metrics` | Query timeseries metrics |
+| `datadog_rum` | Search Real User Monitoring events |
+
+### LLM Tools (2)
+
+| Tool | Description |
+|------|-------------|
+| `llm_debug_call` | Debug LLM API calls and inspect raw responses |
+| `llm_list_providers` | List available LLM providers with their models |
+
+## Environment Variables
+
+### AWS CLI Integration
+
+| Variable | Description |
+|----------|-------------|
+| `AWS_PROFILE` | Default profile if not specified per-call |
+| `AWS_REGION` or `AWS_DEFAULT_REGION` | Default region if not specified |
+
+AWS tools use the host's existing credential chain:
+- `~/.aws/credentials` and `~/.aws/config` files
+- Environment variables (`AWS_ACCESS_KEY_ID`, etc.)
+- SSO sessions established via `aws sso login`
+
+### Datadog Integration
+
+| Variable | Description |
+|----------|-------------|
+| `DATADOG_API_KEY` or `DD_API_KEY` | Datadog API key |
+| `DATADOG_APP_KEY` or `DD_APP_KEY` | Datadog Application key |
+| `DD_ENV` | Default environment filter |
+| `DD_SERVICE` | Default service filter |
+| `DD_SOURCE` | Default log source (defaults to "lambda") |
+| `DD_QUERY` | Default query terms appended to searches |
+
+## Adding New Tools
+
+Tools are registered in `createMcpServer.ts` using the MCP SDK pattern:
+
+```typescript
+server.tool(
+  "tool_name",
+  "Description shown to AI agents",
+  {
+    // Zod schema for parameters
+    param1: z.string().describe("Parameter description"),
+    param2: z.number().optional().describe("Optional parameter"),
+  },
+  async ({ param1, param2 }) => {
+    // Implementation
+    return {
+      content: [{ type: "text" as const, text: "Result text" }],
+    };
+  },
+);
+```
+
+### AWS Tool Pattern
+
+AWS tools use `child_process.spawn` to call the AWS CLI:
+
+```typescript
+// In aws.ts
+export async function myAwsOperation(
+  options: MyOperationOptions,
+  logger: Logger = nullLogger,
+): Promise<AwsCommandResult<MyResultType>> {
+  const args = ["--required-arg", options.requiredArg];
+  if (options.optionalArg) {
+    args.push("--optional-arg", options.optionalArg);
+  }
+
+  return executeAwsCommand(
+    "service-name",    // e.g., "stepfunctions", "lambda", "s3api"
+    "command-name",    // e.g., "list-executions", "get-function"
+    args,
+    { profile: options.profile, region: options.region },
+    logger,
+  );
+}
+```
+
+### Datadog Tool Pattern
+
+Datadog tools use Node.js `https` module directly:
+
+```typescript
+// In datadog.ts
+export async function myDatadogOperation(
+  credentials: DatadogCredentials,
+  options: MyOptions,
+  logger: Logger = nullLogger,
+): Promise<MyResult> {
+  const requestOptions = {
+    hostname: "api.datadoghq.com",
+    port: 443,
+    path: "/api/v2/endpoint",
+    method: "POST",
+    headers: {
+      "DD-API-KEY": credentials.apiKey,
+      "DD-APPLICATION-KEY": credentials.appKey,
+      "Content-Type": "application/json",
+    },
+  };
+
+  return new Promise((resolve) => {
+    const req = https.request(requestOptions, (res) => {
+      // Handle response
+    });
+    req.write(JSON.stringify(body));
+    req.end();
+  });
+}
+```
+
+## Adding New Prompts
+
+Prompts are markdown files in `prompts/` with optional YAML frontmatter:
+
+```yaml
+---
+description: Brief description shown in list_prompts
+include: "packages/express/**"  # File patterns this guide applies to
+---
+
+# Prompt Title
+
+Markdown content here...
+```
+
+Prompts are automatically available via `list_prompts` and `read_prompt` tools.
+
+## Exports
+
+```typescript
+import { createMcpServer, mcpExpressHandler } from "@jaypie/mcp";
+import type { CreateMcpServerOptions, McpExpressHandlerOptions } from "@jaypie/mcp";
+```
+
+## Commands
+
+```bash
+npm run build -w packages/mcp      # Build with rollup
+npm run test -w packages/mcp       # Run tests (vitest run)
+npm run typecheck -w packages/mcp  # Type check (tsc --noEmit)
+npm run format packages/mcp        # Format with eslint --fix
+```
+
+## Security Considerations
+
+1. **Allowlisted operations only** - No arbitrary command execution
+2. **Read-heavy design** - Most tools are read-only; mutating operations have explicit warnings
+3. **No credential exposure** - Credentials never passed through MCP; uses host's credential chain
+4. **Profile isolation** - Each call can specify a different profile for multi-account work

package/prompts/Jaypie_Streaming.md

@@ -0,0 +1,408 @@
+---
+description: Complete guide to streaming in Jaypie - Lambda, Express, and SSE patterns
+globs: packages/express/**, packages/lambda/**, packages/aws/**
+---
+
+# Jaypie Streaming Guide
+
+Jaypie provides three distinct streaming patterns for different deployment scenarios. This guide explains when to use each approach and how to implement streaming correctly.
+
+## Quick Reference
+
+| Handler | Package | Express Required | Use Case |
+|---------|---------|------------------|----------|
+| `lambdaStreamHandler` | `@jaypie/lambda` | No | Pure Lambda Function URL streaming |
+| `expressStreamHandler` | `@jaypie/express` | Yes | Express routes with SSE |
+| `createLambdaStreamHandler` | `@jaypie/express` | Yes | Express app deployed to Lambda with streaming |
+
+### Decision Guide
+
+- **Building a pure Lambda function?** Use `lambdaStreamHandler`
+- **Building an Express app for non-Lambda deployment?** Use `expressStreamHandler`
+- **Building an Express app that runs on Lambda?** Use `createLambdaStreamHandler`
+
+## Streaming Utilities
+
+All streaming utilities are exported from `@jaypie/aws` and re-exported through `jaypie`:
+
+| Function | Purpose |
+|----------|---------|
+| `createLambdaStream(stream, writer)` | Pipe async iterable to Lambda response writer |
+| `createExpressStream(stream, res)` | Pipe async iterable to Express response with SSE headers |
+| `JaypieStream` | Wrapper class with `.toLambda()` and `.toExpress()` methods |
+| `createJaypieStream(source)` | Factory function for JaypieStream |
+| `formatSSE(chunk)` | Format a chunk as SSE event string |
+| `streamToSSE(stream)` | Convert async iterable to SSE-formatted strings |
+
+### Types
+
+```typescript
+import type {
+  ExpressStreamResponse,
+  LambdaStreamWriter,
+  SSEEvent,
+  StreamChunk,
+} from "jaypie";
+```
+
+## Pattern 1: lambdaStreamHandler
+
+Use for pure Lambda functions without Express. Requires AWS Lambda Response Streaming via Function URL.
+
+### Basic Usage
+
+```typescript
+import { lambdaStreamHandler } from "jaypie";
+import type { StreamHandlerContext } from "@jaypie/lambda";
+
+const streamWorker = lambdaStreamHandler(
+  async (event: unknown, context: StreamHandlerContext) => {
+    const { responseStream } = context;
+
+    responseStream.write("event: start\ndata: {}\n\n");
+
+    for (let i = 0; i < 5; i++) {
+      responseStream.write(`event: data\ndata: {"count": ${i}}\n\n`);
+    }
+
+    responseStream.write("event: done\ndata: {}\n\n");
+    // Handler automatically calls responseStream.end()
+  },
+  {
+    name: "streamWorker",
+    contentType: "text/event-stream",
+  }
+);
+
+// Wrap with AWS streamifyResponse for Lambda
+declare const awslambda: { streamifyResponse: <T>(handler: T) => T };
+export const handler = awslambda.streamifyResponse(streamWorker);
+```
+
+### With LLM Streaming
+
+```typescript
+import { lambdaStreamHandler, createLambdaStream, Llm } from "jaypie";
+import type { StreamHandlerContext } from "@jaypie/lambda";
+
+interface PromptEvent {
+  prompt?: string;
+}
+
+const llmStreamHandler = lambdaStreamHandler(
+  async (event: PromptEvent, context: StreamHandlerContext) => {
+    const llm = new Llm("anthropic");
+    const stream = llm.stream(event.prompt || "Hello");
+
+    // createLambdaStream pipes chunks as SSE events
+    await createLambdaStream(stream, context.responseStream);
+  },
+  {
+    name: "llmStream",
+    secrets: ["ANTHROPIC_API_KEY"],
+  }
+);
+
+declare const awslambda: { streamifyResponse: <T>(handler: T) => T };
+export const handler = awslambda.streamifyResponse(llmStreamHandler);
+```
+
+### Handler Options
+
+```typescript
+import type { LambdaStreamHandlerOptions } from "@jaypie/lambda";
+
+const options: LambdaStreamHandlerOptions = {
+  name: "myStreamHandler",          // Handler name for logging
+  contentType: "text/event-stream", // Response content type (default)
+  chaos: "low",                     // Chaos testing level
+  secrets: ["API_KEY"],             // AWS secrets to load into process.env
+  setup: [],                        // Setup function(s)
+  teardown: [],                     // Teardown function(s)
+  validate: [],                     // Validation function(s)
+  throw: false,                     // Re-throw errors instead of SSE error
+  unavailable: false,               // Return 503 if true
+};
+```
+
+### CDK Configuration
+
+```typescript
+import { JaypieLambda } from "@jaypie/constructs";
+import { FunctionUrlAuthType, InvokeMode } from "aws-cdk-lib/aws-lambda";
+
+const streamingLambda = new JaypieLambda(this, "StreamingFunction", {
+  code: "dist",
+  handler: "streamWorker.handler",
+  timeout: Duration.minutes(5),
+});
+
+// Enable Lambda Response Streaming
+streamingLambda.addFunctionUrl({
+  authType: FunctionUrlAuthType.NONE,
+  invokeMode: InvokeMode.RESPONSE_STREAM,
+});
+```
+
+## Pattern 2: expressStreamHandler
+
+Use for Express applications not running on Lambda. Sets SSE headers automatically.
+
+### Basic Usage
+
+```typescript
+import { expressStreamHandler } from "jaypie";
+import type { Request, Response } from "express";
+
+const streamRoute = expressStreamHandler(async (req: Request, res: Response) => {
+  // Write SSE events directly to response
+  res.write("event: message\ndata: {\"text\": \"Hello\"}\n\n");
+  res.write("event: message\ndata: {\"text\": \"World\"}\n\n");
+  // Handler automatically ends the stream
+});
+
+app.get("/stream", streamRoute);
+```
+
+### With LLM Streaming
+
+```typescript
+import { expressStreamHandler, createExpressStream, Llm } from "jaypie";
+
+const llmStreamRoute = expressStreamHandler(async (req: Request, res: Response) => {
+  const llm = new Llm("anthropic");
+  const stream = llm.stream(req.body.prompt);
+
+  // createExpressStream pipes LLM chunks as SSE events
+  await createExpressStream(stream, res);
+});
+
+app.post("/chat", llmStreamRoute);
+```
+
+### Handler Options
+
+```typescript
+import type { ExpressStreamHandlerOptions } from "jaypie";
+
+const options: ExpressStreamHandlerOptions = {
+  name: "myStreamHandler",          // Handler name for logging
+  contentType: "text/event-stream", // Default SSE content type
+  chaos: "low",                     // Chaos testing level
+  secrets: ["API_KEY"],             // Secrets to load
+  setup: [],                        // Setup function(s)
+  teardown: [],                     // Teardown function(s)
+  validate: [],                     // Validation function(s)
+  locals: {},                       // Values to set on req.locals
+  unavailable: false,               // Return 503 if true
+};
+```
+
+### SSE Headers
+
+`expressStreamHandler` automatically sets these headers:
+
+- `Content-Type: text/event-stream`
+- `Cache-Control: no-cache`
+- `Connection: keep-alive`
+- `X-Accel-Buffering: no` (disables nginx buffering)
+
+## Pattern 3: createLambdaStreamHandler
+
+Use for Express applications deployed to AWS Lambda with streaming support.
+
+### Basic Usage
+
+```typescript
+import express from "express";
+import { createLambdaStreamHandler, expressStreamHandler } from "jaypie";
+
+const app = express();
+
+app.get("/stream", expressStreamHandler(async (req, res) => {
+  res.write("event: message\ndata: {\"text\": \"Hello\"}\n\n");
+  res.write("event: message\ndata: {\"text\": \"World\"}\n\n");
+}));
+
+// Export streaming Lambda handler for Express app
+export const handler = createLambdaStreamHandler(app);
+```
+
+### Combined Buffered and Streaming
+
+When you need both regular endpoints and streaming endpoints:
+
+```typescript
+import express from "express";
+import {
+  createLambdaHandler,
+  createLambdaStreamHandler,
+  expressHandler,
+  expressStreamHandler,
+  cors,
+} from "jaypie";
+
+const app = express();
+app.use(express.json());
+app.use(cors());
+
+// Standard buffered route
+app.get("/api/data", expressHandler(async (req, res) => {
+  return { data: "buffered response" };
+}));
+
+// SSE streaming route
+app.get("/api/stream", expressStreamHandler(async (req, res) => {
+  for (let i = 0; i < 5; i++) {
+    res.write(`event: update\ndata: {"count": ${i}}\n\n`);
+  }
+}));
+
+// Choose based on needs:
+// - createLambdaHandler for buffered-only
+// - createLambdaStreamHandler for streaming support
+export const handler = createLambdaStreamHandler(app);
+```
+
+## Streaming Utilities Deep Dive
+
+### JaypieStream Class
+
+Wraps an async iterable for convenient streaming to different targets:
+
+```typescript
+import { JaypieStream, createJaypieStream, Llm } from "jaypie";
+
+const llm = new Llm("anthropic");
+const stream = createJaypieStream(llm.stream("Hello"));
+
+// Pipe to Lambda
+await stream.toLambda(context.responseStream);
+
+// Or pipe to Express
+await stream.toExpress(res);
+
+// Or iterate directly
+for await (const chunk of stream) {
+  console.log(chunk);
+}
+
+// Or convert to SSE strings
+for await (const sseEvent of stream.toSSE()) {
+  console.log(sseEvent);
+}
+```
+
+### Manual SSE Formatting
+
+```typescript
+import { formatSSE } from "jaypie";
+
+const chunk = { type: "message", content: "Hello" };
+const sseString = formatSSE(chunk);
+// "event: message\ndata: {\"type\":\"message\",\"content\":\"Hello\"}\n\n"
+```
+
+### Converting Async Iterables
+
+```typescript
+import { streamToSSE } from "jaypie";
+
+async function* myGenerator() {
+  yield { type: "start", data: {} };
+  yield { type: "data", value: 42 };
+  yield { type: "end", data: {} };
+}
+
+for await (const sseEvent of streamToSSE(myGenerator())) {
+  responseStream.write(sseEvent);
+}
+```
+
+## Error Handling
+
+Errors in streaming handlers are written as SSE error events:
+
+```typescript
+// Format: event: error\ndata: {"errors":[{"status":500,"title":"Internal Error"}]}\n\n
+```
+
+For `lambdaStreamHandler`, set `throw: true` to re-throw errors instead of writing to stream:
+
+```typescript
+const handler = lambdaStreamHandler(
+  async (event, context) => {
+    throw new InternalError("Something went wrong");
+  },
+  {
+    throw: true, // Re-throw instead of SSE error
+  }
+);
+```
+
+## Testing Streaming Handlers
+
+### Mocking for Unit Tests
+
+```typescript
+import { describe, expect, it, vi } from "vitest";
+
+vi.mock("jaypie", async () => {
+  const testkit = await import("@jaypie/testkit/mock");
+  return testkit;
+});
+
+import { handler } from "./streamWorker.js";
+
+describe("Stream Handler", () => {
+  it("writes expected events", async () => {
+    const writes: string[] = [];
+    const mockWriter = {
+      write: vi.fn((chunk) => writes.push(chunk)),
+      end: vi.fn(),
+    };
+
+    const event = { prompt: "test" };
+    const context = { responseStream: mockWriter };
+
+    await handler(event, context);
+
+    expect(mockWriter.write).toHaveBeenCalled();
+    expect(writes.some(w => w.includes("event:"))).toBe(true);
+    expect(mockWriter.end).toHaveBeenCalled();
+  });
+});
+```
+
+### Integration Testing
+
+Use the Docker setup in `packages/express/docker/` for local Lambda streaming tests.
+
+## TypeScript Types
+
+```typescript
+// From @jaypie/lambda
+import type {
+  LambdaStreamHandlerOptions,
+  StreamHandlerContext,
+  ResponseStream,
+  AwsStreamingHandler,
+} from "@jaypie/lambda";
+
+// From @jaypie/express (or jaypie)
+import type {
+  ExpressStreamHandlerOptions,
+  ExpressStreamHandlerLocals,
+  JaypieStreamHandlerSetup,
+  JaypieStreamHandlerTeardown,
+  JaypieStreamHandlerValidate,
+} from "jaypie";
+
+// From @jaypie/aws (or jaypie)
+import type {
+  ExpressStreamResponse,
+  LambdaStreamWriter,
+  SSEEvent,
+  StreamChunk,
+} from "jaypie";
+```