@zibby/skills 0.1.8 → 0.1.9

package/docs/running-tests.md

@@ -0,0 +1,134 @@
+---
+sidebar_position: 5
+title: Running Tests
+---
+
+# Running Tests
+
+Zibby runs tests locally using an AI agent that drives a real browser. It records video, captures actions, and generates reusable Playwright scripts.
+
+## Basic Usage
+
+```bash
+zibby test test-specs/login.txt --agent cursor
+```
+
+## From a Test Spec File
+
+Create a plain-text file with test steps:
+
+```text title="test-specs/login.txt"
+1. Navigate to the login page
+2. Enter email: test@example.com
+3. Enter password: TestPass123
+4. Click the Sign In button
+5. Verify the dashboard page loads
+6. Verify the user's name appears in the header
+```
+
+Run it:
+
+```bash
+zibby test test-specs/login.txt
+```
+
+## From Cloud Test Cases
+
+After running analysis on a Jira ticket in the dashboard:
+
+```bash
+zibby test --sources TC001,TC002 --execution abc-123-def --sync
+```
+
+## What Happens During a Run
+
+The default workflow executes three nodes:
+
+1. **Preflight** — AI extracts a title and assertions from the test spec
+2. **Execute Live** — AI drives a Playwright browser, following the test steps
+3. **Generate Script** — AI produces a reusable `.spec.js` from recorded actions
+
+```
+Test Spec → Preflight → Execute Live → Generate Script → Done
+```
+
+## Choosing an AI Agent
+
+```bash
+# Cursor Agent (uses Cursor IDE or CURSOR_API_KEY)
+zibby test test-specs/login.txt --agent cursor
+
+# Claude (uses ANTHROPIC_API_KEY)
+zibby test test-specs/login.txt --agent claude
+
+# Codex (uses OPENAI_API_KEY + codex CLI)
+zibby test test-specs/login.txt --agent codex
+```
+
+## Headless Mode
+
+Run without a visible browser (for CI/CD):
+
+```bash
+zibby test test-specs/login.txt --headless
+```
+
+## Cloud Sync
+
+Upload results to the [Zibby dashboard](https://zibby.app):
+
+```bash
+# Upload to dashboard
+zibby test test-specs/login.txt --sync
+
+# Organize into a collection
+zibby test test-specs/login.txt --collection "Auth Tests" --sync
+
+# With subfolder
+zibby test test-specs/login.txt --collection "Auth Tests" --folder "Login" --sync
+
+# Local only (no upload)
+zibby test test-specs/login.txt --no-sync
+```
+
+The `cloudSync` setting in `.zibby.config.js` sets the default. CLI flags override it.
+
+## Running a Single Node
+
+Re-run just one step of the workflow (useful for debugging):
+
+```bash
+# Re-execute the browser test (reusing previous session)
+zibby test test-specs/login.txt --node execute_live --session last
+
+# Only regenerate the Playwright script
+zibby test test-specs/login.txt --node generate_script --session last
+```
+
+## Custom Workflows
+
+If you've created a custom workflow in `.zibby/graph.js`:
+
+```bash
+zibby test test-specs/login.txt --workflow QuickSmokeWorkflow
+```
+
+Supported formats: `QuickSmokeWorkflow`, `QuickSmoke`, `quick-smoke`.
+
+## Opening Results in Browser
+
+```bash
+zibby test test-specs/login.txt --sync --open
+```
+
+The `--open` flag opens the dashboard results page after upload.
+
+## Debugging
+
+```bash
+# Info-level logs
+zibby test test-specs/login.txt --verbose
+
+# Full debug logs
+zibby test test-specs/login.txt --debug
+```

package/docs/triggering-workflows.md

@@ -0,0 +1,552 @@
+---
+sidebar_position: 6
+title: Triggering Workflows
+---
+
+# Triggering Workflows
+
+Learn how to trigger Zibby workflows using the API, CLI, webhooks, cron jobs, and third-party integrations.
+
+## Overview
+
+Once deployed, workflows can be triggered from anywhere via HTTP API. Every trigger spawns an isolated execution in Zibby Cloud (ECS Fargate), with full logging and quota tracking.
+
+### Execution Flow
+
+1. Client triggers workflow via API
+2. Backend validates quota & permissions
+3. ECS Fargate task spawned with `zibby run-workflow`
+4. Workflow sources fetched from S3
+5. Graph executes in isolated container
+6. Results uploaded to cloud
+7. Logs streamed to CloudWatch
+
+## Trigger Methods
+
+### 1. CLI (Local Development)
+
+Trigger workflows directly from your terminal using workflow UUID (project auto-discovered):
+
+```bash
+# Trigger by workflow UUID (simplest - no project needed)
+zibby trigger 562e48d7-ef67-4900-96b7-0d7b51a33405
+
+# With input data
+zibby trigger 562e48d7-ef67-4900-96b7-0d7b51a33405 \
+  --input '{"ticket":"JIRA-123"}'
+
+# By workflow name (requires project ID)
+zibby trigger custom-flow --project <id> \
+  --input '{"key":"value"}'
+
+# With idempotency key (prevents duplicate runs)
+zibby trigger 562e48d7-ef67-4900-96b7-0d7b51a33405 \
+  --idempotency-key "daily-scan-2026-04-27"
+
+# Stream logs immediately after triggering
+zibby trigger 562e48d7-ef67-4900-96b7-0d7b51a33405 && \
+  zibby logs 562e48d7-ef67-4900-96b7-0d7b51a33405 -t
+```
+
+### 2. HTTP API
+
+#### Standard Endpoint
+
+Always available for all workflows:
+
+```bash
+curl -X POST https://api.zibby.app/projects/{projectId}/workflows/{type}/trigger \
+  -H "Authorization: Bearer zby_xxx" \
+  -H "Content-Type: application/json" \
+  -d '{"input": {"ticket": "JIRA-123"}}'
+```
+
+**Request Body:**
+```json
+{
+  "input": {
+    "ticket": "JIRA-123",
+    "branch": "main"
+  },
+  "idempotencyKey": "optional-unique-key-for-deduplication"
+}
+```
+
+**Response (202 Accepted):**
+```json
+{
+  "jobId": "job-...",
+  "status": "accepted",
+  "workflow": "custom-flow",
+  "version": 2,
+  "projectId": "...",
+  "triggeredAt": "2026-04-24T..."
+}
+```
+
+#### Custom Subdomain (Optional)
+
+Each deployed workflow gets a unique subdomain URL for cleaner integration:
+
+```bash
+curl -X POST https://my-security-scan.workflows.zibby.app/trigger \
+  -H "Authorization: Bearer zby_xxx" \
+  -H "Content-Type: application/json" \
+  -d '{"input": {"branch": "main"}}'
+```
+
+The subdomain maps to a specific `{projectId, workflowType}` pair, ideal for:
+- Webhooks (cleaner URLs)
+- Public APIs
+- Third-party integrations
+- Branded endpoints
+
+### 3. Scheduled Execution
+
+#### AWS EventBridge (Serverless)
+
+```yaml
+# CloudFormation
+EventBridgeRule:
+  Type: AWS::Events::Rule
+  Properties:
+    ScheduleExpression: 'cron(0 9 * * ? *)'  # Daily 9am
+    Targets:
+      - Arn: !GetAtt TriggerFunction.Arn
+        Input: |
+          {
+            "projectId": "6b60049d-...",
+            "workflowType": "custom-flow",
+            "input": {"source": "cron"}
+          }
+```
+
+#### GitHub Actions
+
+```yaml
+name: Daily Security Scan
+on:
+  schedule:
+    - cron: '0 0 * * *'  # Daily midnight
+
+jobs:
+  trigger:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Trigger Zibby Workflow
+        run: |
+          curl -X POST https://api.zibby.app/projects/$PROJECT_ID/workflows/security-scan/trigger \
+            -H "Authorization: Bearer $ZIBBY_API_KEY" \
+            -d '{"input": {"branch": "${{ github.ref }}"}}'
+        env:
+          ZIBBY_API_KEY: ${{ secrets.ZIBBY_API_KEY }}
+          PROJECT_ID: ${{ vars.ZIBBY_PROJECT_ID }}
+```
+
+#### Traditional Cron (Linux)
+
+```bash
+# /etc/crontab or crontab -e
+0 9 * * * zibby trigger custom-flow --project <id> >> /var/log/zibby.log 2>&1
+```
+
+### 4. Webhooks
+
+#### Jira Webhook Integration
+
+```javascript
+// AWS Lambda or Express endpoint
+app.post('/webhooks/jira', async (req, res) => {
+  const { issue } = req.body;
+  
+  if (issue.fields.status === 'Ready for Testing') {
+    await fetch('https://api.zibby.app/projects/{projectId}/workflows/test-gen/trigger', {
+      method: 'POST',
+      headers: {
+        'Authorization': `Bearer ${process.env.ZIBBY_API_KEY}`,
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify({
+        input: {
+          ticketKey: issue.key,
+          title: issue.fields.summary,
+          description: issue.fields.description,
+        },
+        idempotencyKey: `jira-${issue.key}-${issue.updated}`,
+      }),
+    });
+  }
+  
+  res.sendStatus(200);
+});
+```
+
+#### GitHub Pull Request Trigger
+
+```yaml
+# .github/workflows/zibby-analyze.yml
+name: Zibby Code Analysis
+on:
+  pull_request:
+    types: [opened, synchronize]
+
+jobs:
+  analyze:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Trigger Zibby Analysis
+        run: |
+          curl -X POST https://api.zibby.app/projects/$PROJECT_ID/workflows/code-review/trigger \
+            -H "Authorization: Bearer $ZIBBY_API_KEY" \
+            -d '{
+              "input": {
+                "prNumber": "${{ github.event.pull_request.number }}",
+                "repo": "${{ github.repository }}",
+                "branch": "${{ github.head_ref }}"
+              },
+              "idempotencyKey": "pr-${{ github.event.pull_request.number }}-${{ github.sha }}"
+            }'
+        env:
+          ZIBBY_API_KEY: ${{ secrets.ZIBBY_API_KEY }}
+          PROJECT_ID: ${{ vars.ZIBBY_PROJECT_ID }}
+```
+
+### 5. Third-Party Integrations
+
+#### No-Code Platforms (Zapier/Make.com)
+
+Configure an action to POST to the Zibby API:
+- **Trigger**: Google Sheets row added, Slack message, etc.
+- **Action**: HTTP POST to `https://api.zibby.app/projects/{id}/workflows/{type}/trigger`
+- **Headers**: Add `Authorization: Bearer {your-token}`
+- **Body**: JSON with `input` field
+
+#### AWS Step Functions
+
+```json
+{
+  "StartAt": "CheckCode",
+  "States": {
+    "CheckCode": {
+      "Type": "Task",
+      "Resource": "arn:aws:lambda:...:TriggerZibbyWorkflow",
+      "Parameters": {
+        "workflow": "security-scan"
+      },
+      "Next": "WaitForCompletion"
+    },
+    "WaitForCompletion": {
+      "Type": "Wait",
+      "Seconds": 300,
+      "Next": "CheckResults"
+    }
+  }
+}
+```
+
+## Authentication
+
+Zibby uses two types of authentication tokens:
+
+### Personal Access Tokens (PAT)
+
+For programmatic access (API, CLI, webhooks, CI/CD):
+
+```
+Format: zby_pat_xxxxx
+Scope: All projects user has access to
+```
+
+**Create a PAT:**
+1. Go to Settings > API Tokens
+2. Click "Create Token"
+3. Name it (e.g., "Production Webhook")
+4. Optionally set expiration
+5. Copy token (shown only once!)
+
+**Usage:**
+
+```bash
+# CLI
+export ZIBBY_API_KEY=zby_pat_xxxxx
+zibby trigger custom-flow --project <id>
+
+# API Request
+curl -X POST https://api.zibby.app/projects/<id>/workflows/custom-flow/trigger \
+  -H "Authorization: Bearer zby_pat_xxxxx" \
+  -d '{"input": {"ticket": "JIRA-123"}}'
+
+# GitHub Actions
+env:
+  ZIBBY_API_KEY: ${{ secrets.ZIBBY_PAT }}
+```
+
+**Characteristics:**
+- User creates in UI (Settings > API Tokens)
+- Long-lived (or custom expiration)
+- Can be revoked individually
+- Named tokens for tracking
+- User's access controls apply
+
+### JWT Tokens
+
+For web UI sessions only (automatic):
+
+```
+Format: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
+Scope: Full user access
+Usage: Web UI only (automatic)
+```
+
+**Characteristics:**
+- Short-lived (24 hours)
+- Created automatically via OAuth login
+- Auto-refreshed by frontend
+- Not for API/CLI use
+
+## Security Best Practices
+
+### 1. Use PAT for All Programmatic Access
+
+```bash
+# ✅ Correct - Use PAT
+export ZIBBY_API_KEY=zby_pat_xxxxx
+
+# ❌ Wrong - Don't try to extract JWT from browser
+```
+
+### 2. Create Separate PATs per Use Case
+
+In the UI, create multiple tokens:
+- "Production Webhooks" (never expires, heavily monitored)
+- "CI/CD Pipeline" (expires yearly)
+- "Development Testing" (expires monthly)
+- "Partner Integration - Acme Corp" (expires quarterly)
+
+### 3. Token Rotation
+
+Best practice: Rotate annually or when compromised
+
+1. Create new PAT: "CI/CD Pipeline v2"
+2. Update CI/CD secrets
+3. Test with new token
+4. Revoke old PAT after grace period
+
+### 4. Monitor Token Usage
+
+In UI: Settings > API Tokens
+- View last used timestamp
+- Track execution count
+- Monitor for unusual activity
+
+### 5. Webhook Signatures
+
+Verify webhook authenticity:
+
+```javascript
+const crypto = require('crypto');
+const signature = crypto
+  .createHmac('sha256', WEBHOOK_SECRET)
+  .update(JSON.stringify(req.body))
+  .digest('hex');
+
+if (signature !== req.headers['x-zibby-signature']) {
+  return res.status(401).send('Invalid signature');
+}
+```
+
+### 6. Network Restrictions
+
+```yaml
+# AWS Security Group
+Ingress:
+  - IpProtocol: tcp
+    FromPort: 443
+    ToPort: 443
+    CidrIp: 52.89.214.238/32
+    Description: GitHub Actions webhook
+```
+
+## Quota & Billing
+
+### Tiers
+
+- **Free Tier**: 100 workflow executions/month
+- **Pro Tier**: 1,000 executions/month
+- **Enterprise**: Unlimited
+
+### Metering
+
+- Each trigger = 1 execution
+- Idempotency: Same key within 24h = no charge
+- CloudWatch logs for audit trail
+
+### Quota Exceeded Response
+
+```json
+{
+  "error": "Quota exceeded",
+  "statusCode": 429,
+  "limit": 1000,
+  "used": 1000,
+  "resetDate": "2026-05-01T00:00:00Z"
+}
+```
+
+## Best Practices
+
+### 1. Use Idempotency Keys
+
+Prevent duplicate executions:
+
+```bash
+zibby trigger custom-flow \
+  --idempotency-key "daily-scan-2026-04-24"
+```
+
+Same key within 24 hours = no additional charge, returns existing job ID.
+
+### 2. Input Validation
+
+Validate input in your workflow:
+
+```javascript
+// In workflow graph.mjs
+buildGraph() {
+  const graph = new WorkflowGraph();
+  
+  graph.addNode('validate_input', async (state) => {
+    const { ticketKey } = state.input;
+    if (!ticketKey) {
+      throw new Error('ticketKey required');
+    }
+    return { valid: true };
+  });
+  
+  // ...
+}
+```
+
+### 3. Error Handling & Retries
+
+Client-side retry logic:
+
+```javascript
+async function triggerWithRetry(workflow, input, maxRetries = 3) {
+  for (let i = 0; i < maxRetries; i++) {
+    try {
+      const response = await fetch(`${API_URL}/workflows/${workflow}/trigger`, {
+        method: 'POST',
+        headers: { 'Authorization': `Bearer ${API_KEY}` },
+        body: JSON.stringify({ input }),
+      });
+      
+      if (response.status === 429) {
+        await new Promise(r => setTimeout(r, 60000 * (i + 1)));
+        continue;
+      }
+      
+      return await response.json();
+    } catch (err) {
+      if (i === maxRetries - 1) throw err;
+    }
+  }
+}
+```
+
+## Monitoring
+
+### View Logs
+
+```bash
+# Fetch complete logs from latest execution (one-time)
+zibby logs 562e48d7-ef67-4900-96b7-0d7b51a33405
+
+# Stream logs in real-time (like "heroku logs -t")
+zibby logs 562e48d7-ef67-4900-96b7-0d7b51a33405 -t
+
+# By workflow name
+zibby logs --workflow custom-flow --project <id> -t
+```
+
+**Log Architecture:**
+
+- **Without `-t`**: Fetches from CloudWatch (permanent storage). Shows complete historical logs from any execution, even old ones.
+- **With `-t`**: Streams in real-time via SSE from DynamoDB hot cache (24h TTL). Only works for active/recent executions.
+
+```
+CloudWatch Logs (permanent) ──────────> CLI (without -t)
+     │
+     └──> Kinesis ──> Lambda ──> DynamoDB (24h TTL) ──> SSE ──> CLI (with -t)
+```
+
+### CloudWatch Dashboards
+
+Monitor:
+- Execution count by workflow
+- Success/failure rates
+- P50/P95/P99 latencies
+- Quota utilization
+
+### Alerts
+
+```yaml
+# EventBridge Rule → SNS → Email/Slack
+Pattern:
+  source: [zibby.workflows]
+  detail-type: [WorkflowFailed]
+  detail:
+    workflow: [security-scan]
+```
+
+## Common Patterns
+
+### Pattern 1: Daily Report Generation
+
+```bash
+# Cron job that triggers workflow by UUID and streams logs
+WORKFLOW_UUID="562e48d7-ef67-4900-96b7-0d7b51a33405"
+
+0 9 * * * zibby trigger $WORKFLOW_UUID \
+  --input '{"date":"'$(date +%Y-%m-%d)'"}' \
+  --idempotency-key "daily-report-$(date +%Y-%m-%d)" \
+  && sleep 5 \
+  && zibby logs $WORKFLOW_UUID | mail -s "Daily Report" team@company.com
+```
+
+### Pattern 2: Event-Driven Testing
+
+```javascript
+// GitHub webhook → test generation → PR comment
+app.post('/webhook/github', async (req, res) => {
+  const { pull_request } = req.body;
+  
+  const { jobId } = await triggerWorkflow('test-gen', {
+    prNumber: pull_request.number,
+    repo: pull_request.base.repo.full_name,
+  });
+  
+  await postComment(pull_request.number, `Generating tests... [Job ${jobId}]`);
+  res.sendStatus(200);
+});
+```
+
+### Pattern 3: Conditional Execution
+
+```javascript
+// Only trigger if conditions are met
+const shouldTrigger = await checkConditions();
+if (shouldTrigger) {
+  await fetch('https://api.zibby.app/projects/.../workflows/deploy/trigger', {
+    method: 'POST',
+    headers: { 'Authorization': `Bearer ${PAT}` },
+    body: JSON.stringify({ input: { environment: 'production' } }),
+  });
+}
+```
+
+## Next Steps
+
+- [Custom Workflows](custom-workflows.md) - Build your own workflows
+- [CLI Reference](cli-reference.md) - Full command documentation
+- [Integrations](integrations/github.md) - Connect with GitHub, Jira, etc.