@hailer/mcp 1.1.13 → 1.1.15

package/.claude/skills/insight-join-patterns/SKILL.md

@@ -0,0 +1,174 @@
+---
+name: insight-join-patterns
+description: Correct JOIN syntax for Hailer insights with ActivityLink fields
+version: 1.0.0
+triggers: JOIN query errors, missing columns, NULL results in insight queries
+---
+
+**Prerequisite:** Before using JOINs, review `SDK-insight-queries` skill for basic insight syntax and single-workflow queries.
+
+<problem>
+When joining workflows with ActivityLink fields in Hailer insights, you must:
+1. Include `_id` meta field in BOTH source definitions
+2. Join ON the activitylink field value equals target _id
+3. Use the activitylink fieldId (NOT the key) for the JOIN condition
+</problem>
+
+<rules>
+- Both workflows need `{ name: 'id', meta: '_id' }` in their fields array
+- JOIN condition: `source1.activityLinkFieldName = source2.id`
+- Use LEFT JOIN for optional relationships (activitylink can be null)
+- Use INNER JOIN only when relationship must exist
+</rules>
+
+<correct>
+```javascript
+// Players workflow has "club" field (activitylink to Clubs workflow)
+{
+  sources: [
+    {
+      name: 'p',
+      workflowId: '68446dc05b30685f67c6fcd4',
+      fields: [
+        { name: 'player_name', meta: 'name' },
+        { name: 'id', meta: '_id' },              // Required!
+        { name: 'club', fieldId: '684d5e45...' }  // ActivityLink field
+      ]
+    },
+    {
+      name: 'c',
+      workflowId: '691ea936ccb6bdeebc0cbf77',
+      fields: [
+        { name: 'club_name', meta: 'name' },
+        { name: 'id', meta: '_id' }               // Required!
+      ]
+    }
+  ],
+  query: 'SELECT p.player_name, c.club_name FROM p LEFT JOIN c ON p.club = c.id'
+}
+```
+</correct>
+
+<wrong>
+```javascript
+// ❌ WRONG - Missing _id in clubs source
+{
+  sources: [
+    {
+      name: 'p',
+      workflowId: 'players-id',
+      fields: [
+        { name: 'player_name', meta: 'name' },
+        { name: 'id', meta: '_id' },
+        { name: 'club', fieldId: 'club-field-id' }
+      ]
+    },
+    {
+      name: 'c',
+      workflowId: 'clubs-id',
+      fields: [
+        { name: 'club_name', meta: 'name' }
+        // Missing: { name: 'id', meta: '_id' }
+      ]
+    }
+  ],
+  query: 'SELECT p.player_name, c.club_name FROM p LEFT JOIN c ON p.club = c.id'
+}
+// Error: "no such column: c.id"
+```
+</wrong>
+
+<examples>
+### Three-Way JOIN (Tasks -> Topics -> Projects)
+```javascript
+{
+  sources: [
+    {
+      name: 't',
+      workflowId: 'tasks-workflow-id',
+      fields: [
+        { name: 'task_name', meta: 'name' },
+        { name: 'id', meta: '_id' },
+        { name: 'topic', fieldId: 'topic-field-id' }
+      ]
+    },
+    {
+      name: 'top',
+      workflowId: 'topics-workflow-id',
+      fields: [
+        { name: 'topic_name', meta: 'name' },
+        { name: 'id', meta: '_id' },
+        { name: 'project', fieldId: 'project-field-id' }
+      ]
+    },
+    {
+      name: 'p',
+      workflowId: 'projects-workflow-id',
+      fields: [
+        { name: 'project_name', meta: 'name' },
+        { name: 'id', meta: '_id' }
+      ]
+    }
+  ],
+  query: `
+    SELECT t.task_name, top.topic_name, p.project_name
+    FROM t
+    LEFT JOIN top ON t.topic = top.id
+    LEFT JOIN p ON top.project = p.id
+  `
+}
+```
+
+### Aggregation with JOIN
+```javascript
+{
+  sources: [
+    {
+      name: 'matches',
+      workflowId: 'matches-workflow-id',
+      fields: [
+        { name: 'match_date', fieldId: 'date-field-id' },
+        { name: 'id', meta: '_id' },
+        { name: 'home_team', fieldId: 'home-team-field-id' }
+      ]
+    },
+    {
+      name: 'teams',
+      workflowId: 'teams-workflow-id',
+      fields: [
+        { name: 'team_name', meta: 'name' },
+        { name: 'id', meta: '_id' }
+      ]
+    }
+  ],
+  query: `
+    SELECT teams.team_name, COUNT(*) as match_count
+    FROM matches
+    LEFT JOIN teams ON matches.home_team = teams.id
+    GROUP BY teams.team_name
+  `
+}
+```
+</examples>
+
+<troubleshooting>
+**Error: "no such column: c.id"**
+- Missing `{ name: 'id', meta: '_id' }` in target workflow source
+
+**Error: "no such column: p.club"**
+- ActivityLink field not included in source fields array
+- Check you used correct fieldId from `get_workflow_schema`
+
+**NULL results for joined data**
+- ActivityLink field is empty/null for some activities (expected with LEFT JOIN)
+- Use INNER JOIN if you only want activities with relationships
+</troubleshooting>
+
+<checklist>
+Before creating an insight with JOINs:
+- [ ] Both workflow sources include `{ name: 'id', meta: '_id' }`
+- [ ] ActivityLink field uses `fieldId` (NOT `key`)
+- [ ] JOIN condition uses column names from sources (e.g., `p.club = c.id`)
+- [ ] Using LEFT JOIN (unless relationship required)
+- [ ] Tested with `preview_insight` first
+</checklist>

package/.claude/skills/integration-patterns/SKILL.md

@@ -0,0 +1,421 @@
+---
+name: integration-patterns
+description: Hailer integration microservice patterns - activity movers, webhooks, SCIM, Kafka consumers
+version: 1.0.0
+triggers: Build integration, activity mover, webhook handler, SCIM endpoint, Kafka consumer
+---
+
+# Integration Patterns
+
+## Architecture Overview
+
+```
+┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
+│  External       │     │  Integration    │     │  Hailer API     │
+│  System         │────▶│  Service        │────▶│                 │
+│  (Kafka/HTTP)   │     │  (Node.js)      │     │  (WebSocket)    │
+└─────────────────┘     └─────────────────┘     └─────────────────┘
+                               │
+                               ▼
+                        ┌─────────────────┐
+                        │  Logging &      │
+                        │  Monitoring     │
+                        └─────────────────┘
+```
+
+---
+
+## 1. Activity Mover Pattern
+
+Monitors activities and moves linked activities when triggers fire.
+
+### Configuration Structure
+
+```typescript
+interface ActivityMoverConfig {
+  logDiscussionId: string;      // Discussion for audit logging
+  email: string;                // Integration user email
+  password: string;             // Or "ENV:HAILER_PASSWORD"
+  project: string;              // Project identifier
+  triggers: Trigger[];
+}
+
+interface Trigger {
+  processId: string;            // Source workflow ID
+  phaseId: string;              // Phase that triggers action
+  metaDataId: string;           // Hidden field for tracking ("Seen"/"Not seen")
+  linkedProcesses: LinkedProcess[];
+}
+
+interface LinkedProcess {
+  processId: string;            // Target workflow ID
+  sourcePhases: string[];       // Move FROM these phases
+  targetPhase: string;          // Move TO this phase
+}
+```
+
+### Example Config
+
+```json
+{
+  "logDiscussionId": "694c9536acfa30f6df13201b",
+  "email": "integration@workspace.com",
+  "password": "ENV:HAILER_PASSWORD",
+  "project": "orders-sync",
+  "triggers": [
+    {
+      "processId": "67dc1b7d3d2c9f6cf9a5468d",
+      "phaseId": "67dc1b7d3d2c9f6cf9a546c4",
+      "metaDataId": "67e697da6ada809b961c35b5",
+      "linkedProcesses": [
+        {
+          "processId": "67dc1b7d3d2c9f6cf9a54690",
+          "sourcePhases": ["67dc1b7d3d2c9f6cf9a54691", "67dc1b7d3d2c9f6cf9a54692"],
+          "targetPhase": "67dc1b7d3d2c9f6cf9a54695"
+        }
+      ]
+    }
+  ]
+}
+```
+
+### Hailer Setup Requirements
+
+1. **Metadata field** on trigger workflow:
+   - Type: TEXT (hidden)
+   - Default value: "Not seen"
+   - Used to track processed activities
+
+2. **Link field** connecting workflows:
+   - Activity link from trigger workflow to target workflow
+
+3. **Integration user**:
+   - Dedicated user account
+   - Edit permissions on both workflows
+
+4. **Discussion activity**:
+   - For audit logging
+   - Integration user must have access
+
+### Core Logic
+
+```typescript
+// Listen for activity updates
+client.on('activities.updated', async (data) => {
+  for (const trigger of config.triggers) {
+    if (data.processId === trigger.processId && data.phaseId === trigger.phaseId) {
+      await processTrigger(data.activityId, trigger);
+    }
+  }
+});
+
+async function processTrigger(activityId: string, trigger: Trigger) {
+  // Check if already processed
+  const activity = await client.request('v3.activity.get', [activityId]);
+  if (activity.fields[trigger.metaDataId]?.value === 'Seen') return;
+
+  // Get linked activities
+  const linked = await client.request('v3.activity.linkedFrom.overview', [activityId]);
+
+  // Move matching activities
+  for (const process of trigger.linkedProcesses) {
+    const toMove = linked.filter(a =>
+      a.processId === process.processId &&
+      process.sourcePhases.includes(a.phaseId)
+    );
+
+    if (toMove.length > 0) {
+      await client.request('v3.activity.updateMany', [
+        toMove.map(a => a._id),
+        { phaseId: process.targetPhase }
+      ]);
+    }
+  }
+
+  // Mark as processed
+  await client.request('v3.activity.update', [activityId, {
+    fields: { [trigger.metaDataId]: 'Seen' }
+  }]);
+}
+```
+
+---
+
+## 2. Webhook Handler Pattern
+
+HTTP endpoints that trigger Hailer operations.
+
+### Express Setup
+
+```typescript
+import express from 'express';
+import crypto from 'crypto';
+
+const app = express();
+app.use(express.json());
+
+// Webhook signature validation
+function validateSignature(req: express.Request, secret: string): boolean {
+  const signature = req.headers['x-webhook-signature'];
+  const payload = JSON.stringify(req.body);
+  const expected = crypto.createHmac('sha256', secret).update(payload).digest('hex');
+  return signature === expected;
+}
+
+// Webhook endpoint
+app.post('/api/v1/webhook/:type', async (req, res) => {
+  const { type } = req.params;
+
+  if (!validateSignature(req, config.webhookSecret)) {
+    return res.status(401).json({ error: 'Invalid signature' });
+  }
+
+  try {
+    switch (type) {
+      case 'order.created':
+        await handleOrderCreated(req.body);
+        break;
+      case 'order.updated':
+        await handleOrderUpdated(req.body);
+        break;
+      default:
+        logger.warn({ type }, 'Unknown webhook type');
+    }
+    res.json({ status: 'ok' });
+  } catch (error) {
+    logger.error({ error, type }, 'Webhook processing failed');
+    res.status(500).json({ error: 'Processing failed' });
+  }
+});
+```
+
+### Webhook Handler Example
+
+```typescript
+async function handleOrderCreated(payload: OrderPayload) {
+  // Map external data to Hailer fields
+  const fields = {
+    [FieldIds.external_id]: payload.orderId,
+    [FieldIds.customer_name]: payload.customer.name,
+    [FieldIds.total_amount]: payload.total,
+    [FieldIds.order_date]: new Date(payload.createdAt).getTime()
+  };
+
+  // Create activity in Hailer
+  const result = await hailerClient.createActivity(
+    WorkflowIds.orders,
+    PhaseIds.new,
+    fields
+  );
+
+  logger.info({ orderId: payload.orderId, activityId: result._id }, 'Order created in Hailer');
+}
+```
+
+---
+
+## 3. SCIM 2.0 Pattern
+
+User provisioning from identity providers (Microsoft Entra, Okta).
+
+### Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | /scim/v2/Users | Create user |
+| GET | /scim/v2/Users/:id | Get user |
+| PATCH | /scim/v2/Users/:id | Update user |
+| DELETE | /scim/v2/Users/:id | Deactivate user |
+| GET | /scim/v2/Users | List/filter users |
+
+### User Mapping
+
+```typescript
+interface ScimUser {
+  schemas: string[];
+  userName: string;
+  name: { givenName: string; familyName: string };
+  emails: { value: string; primary: boolean }[];
+  active: boolean;
+  roles?: { value: string }[];
+}
+
+function mapScimToHailer(scim: ScimUser): HailerUserPayload {
+  return {
+    email: scim.emails.find(e => e.primary)?.value || scim.userName,
+    firstName: scim.name.givenName,
+    lastName: scim.name.familyName,
+    active: scim.active,
+    teams: mapRolesToTeams(scim.roles)
+  };
+}
+
+function mapRolesToTeams(roles?: { value: string }[]): string[] {
+  const teamMapping: Record<string, string> = {
+    'admin': TeamIds.admins,
+    'manager': TeamIds.managers,
+    'user': TeamIds.users
+  };
+  return (roles || [])
+    .map(r => teamMapping[r.value])
+    .filter(Boolean);
+}
+```
+
+### SCIM Response Format
+
+```typescript
+function toScimUser(hailerUser: HailerUser, baseUrl: string): ScimUser {
+  return {
+    schemas: ['urn:ietf:params:scim:schemas:core:2.0:User'],
+    id: hailerUser._id,
+    userName: hailerUser.email,
+    name: {
+      givenName: hailerUser.firstName,
+      familyName: hailerUser.lastName
+    },
+    emails: [{ value: hailerUser.email, primary: true }],
+    active: hailerUser.active,
+    meta: {
+      resourceType: 'User',
+      location: `${baseUrl}/scim/v2/Users/${hailerUser._id}`
+    }
+  };
+}
+```
+
+---
+
+## 4. Kafka Consumer Pattern
+
+Event-driven processing from message queues.
+
+### Consumer Setup
+
+```typescript
+import { Kafka, Consumer, EachMessagePayload } from 'kafkajs';
+
+const kafka = new Kafka({
+  clientId: 'hailer-integration',
+  brokers: config.kafkaBrokers,
+  ssl: {
+    ca: [fs.readFileSync(config.caCertPath)],
+    key: fs.readFileSync(config.keyPath),
+    cert: fs.readFileSync(config.certPath)
+  }
+});
+
+const consumer = kafka.consumer({ groupId: 'hailer-consumer-group' });
+
+async function startConsumer() {
+  await consumer.connect();
+  await consumer.subscribe({ topic: 'events', fromBeginning: false });
+
+  await consumer.run({
+    eachMessage: async ({ topic, partition, message }: EachMessagePayload) => {
+      const event = JSON.parse(message.value.toString());
+
+      try {
+        await processEvent(event);
+        logger.info({ eventId: event.id, offset: message.offset }, 'Event processed');
+      } catch (error) {
+        logger.error({ error, event }, 'Event processing failed');
+        await sendToDeadLetterQueue(message);
+      }
+    }
+  });
+}
+```
+
+### Event Processing
+
+```typescript
+async function processEvent(event: ExternalEvent) {
+  switch (event.type) {
+    case 'student.created':
+      await createStudent(event.data);
+      break;
+    case 'student.updated':
+      await updateStudent(event.data);
+      break;
+    case 'student.deleted':
+      await deactivateStudent(event.data);
+      break;
+    default:
+      logger.warn({ eventType: event.type }, 'Unknown event type');
+  }
+}
+```
+
+---
+
+## Common Patterns
+
+### Retry with Exponential Backoff
+
+```typescript
+async function withRetry<T>(
+  fn: () => Promise<T>,
+  maxRetries: number = 3,
+  baseDelay: number = 1000
+): Promise<T> {
+  let lastError: Error;
+
+  for (let i = 0; i < maxRetries; i++) {
+    try {
+      return await fn();
+    } catch (error) {
+      lastError = error;
+      const delay = baseDelay * Math.pow(2, i);
+      logger.warn({ attempt: i + 1, delay, error: error.message }, 'Retrying...');
+      await sleep(delay);
+    }
+  }
+
+  throw lastError;
+}
+```
+
+### Graceful Shutdown
+
+```typescript
+async function gracefulShutdown(signal: string) {
+  logger.info({ signal }, 'Shutdown signal received');
+
+  // Stop accepting new requests
+  server.close();
+
+  // Disconnect from Hailer
+  await hailerClient.disconnect();
+
+  // Disconnect from Kafka
+  await consumer.disconnect();
+
+  logger.info('Graceful shutdown complete');
+  process.exit(0);
+}
+
+process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
+process.on('SIGINT', () => gracefulShutdown('SIGINT'));
+```
+
+### Health Check
+
+```typescript
+app.get('/api/v1/health', async (req, res) => {
+  const checks = {
+    hailer: await checkHailerConnection(),
+    kafka: await checkKafkaConnection(),
+    uptime: process.uptime(),
+    memory: process.memoryUsage(),
+    lastActivity: lastActivityTimestamp
+  };
+
+  const healthy = checks.hailer.connected && checks.kafka.connected;
+  res.status(healthy ? 200 : 503).json({
+    status: healthy ? 'healthy' : 'unhealthy',
+    timestamp: new Date().toISOString(),
+    checks
+  });
+});
+```

package/.claude/skills/json-only-output/SKILL.md

@@ -0,0 +1,72 @@
+---
+name: json-only-output
+description: Fix agents adding prose after JSON responses
+version: 1.0.1
+triggers: Agent outputs explanation text after valid JSON
+---
+
+# JSON Only Output
+
+<purpose>
+Ensure agents output ONLY valid JSON with no prose, explanations, or commentary after the closing brace.
+</purpose>
+
+<why-this-happens>
+## Why Agents Add Prose
+
+LLMs have a natural tendency to be helpful and explanatory. After completing a task, they want to:
+- Explain what they did
+- Suggest next steps
+- Provide context
+- Confirm their understanding
+
+This is normally good behavior, but for **subagents** returning structured data to an orchestrator, it breaks JSON parsing.
+
+**The `<protocol>` section isn't enough** because:
+- LLMs treat protocol as "guidelines" not hard rules
+- The helpful instinct overrides weak instructions
+- Protocol is at the end of the prompt, less salient
+
+**Why identity/rules work:**
+- `<identity>` shapes the agent's core persona ("I output JSON. Full stop.")
+- `<rules>` are processed as constraints, not suggestions
+- Positioned early in prompt, higher salience
+- Explicit prohibition is stronger than implicit expectation
+</why-this-happens>
+
+<patterns>
+## Pattern 1: Stop at the Closing Brace
+
+Output JSON. Full stop. Nothing after the closing brace.
+
+## Pattern 2: Agent Configuration Fix
+
+1. Add to `<identity>`: "Output JSON. Full stop."
+2. Add to `<rules>`: **JSON ONLY** - Output closing brace, then STOP. Zero prose after JSON.
+3. Protocol section is NOT enough - agents ignore it without identity/rules reinforcement.
+
+## Pattern 3: Include Summary Inside JSON
+
+If context is helpful, include it IN the JSON `summary` field, not after the JSON.
+</patterns>
+
+<examples>
+## Example 1: Correct - Pure JSON Output
+
+```json
+{"status":"success","result":{"fields":["taskName","priority"]},"summary":"Read 2 fields"}
+```
+**STOP HERE. Nothing after closing brace.**
+
+## Example 2: Wrong - Prose After JSON
+
+```json
+{"status":"success","result":{"fields":["taskName","priority"]},"summary":"Read 2 fields"}
+```
+
+The workflow has 2 fields defined in workspace/Tasks_123/fields.ts. You can now use these field IDs with dmitri for activity creation.
+
+**Action Required**: Run `npm run pull` to refresh.
+
+This violates JSON-only protocol by adding explanation text AFTER the valid JSON response.
+</examples>