crewly 1.5.11 → 1.5.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/config/constants.ts +3 -3
- package/config/orchestrator_tasks/prompts/orchestrator-prompt.md +73 -0
- package/config/roles/architect/prompt.md +9 -0
- package/config/roles/backend-developer/prompt.md +9 -0
- package/config/roles/content-strategist/prompt.md +10 -0
- package/config/roles/designer/prompt.md +9 -0
- package/config/roles/developer/prompt.md +9 -0
- package/config/roles/frontend-developer/prompt.md +9 -0
- package/config/roles/fullstack-dev/prompt.md +9 -0
- package/config/roles/generalist/prompt.md +9 -0
- package/config/roles/ops/prompt.md +9 -0
- package/config/roles/product-manager/prompt.md +9 -0
- package/config/roles/qa/prompt.md +9 -0
- package/config/roles/qa-engineer/prompt.md +9 -0
- package/config/roles/researcher/prompt.md +9 -0
- package/config/roles/sales/prompt.md +9 -0
- package/config/roles/support/prompt.md +9 -0
- package/config/roles/team-leader/prompt.md +11 -0
- package/config/roles/tpm/prompt.md +9 -0
- package/config/roles/ux-designer/prompt.md +9 -0
- package/config/skills/_common/lib.sh +31 -0
- package/config/skills/_common/lib.test.sh +164 -0
- package/config/skills/agent/core/block-task/execute.sh +3 -1
- package/config/skills/agent/core/pipe-to-sink/execute.sh +41 -0
- package/config/skills/agent/core/read-task/execute.sh +3 -1
- package/config/skills/agent/core/report-progress/execute.sh +3 -1
- package/config/skills/agent/screenshot-compare/SKILL.md +75 -0
- package/config/skills/agent/screenshot-compare/execute.sh +182 -0
- package/config/skills/agent/screenshot-compare/skill.json +10 -0
- package/config/skills/agent/xiaoyuzhoufm-transcript/SKILL.md +85 -0
- package/config/skills/agent/xiaoyuzhoufm-transcript/execute.sh +306 -0
- package/config/skills/agent/xiaoyuzhoufm-transcript/skill.json +10 -0
- package/config/skills/orchestrator/cancel-cron/SKILL.md +44 -0
- package/config/skills/orchestrator/create-cron/SKILL.md +58 -0
- package/config/skills/orchestrator/list-cron/SKILL.md +51 -0
- package/config/skills/orchestrator/update-cron/SKILL.md +52 -0
- package/dist/backend/backend/src/constants.d.ts +7 -4
- package/dist/backend/backend/src/constants.d.ts.map +1 -1
- package/dist/backend/backend/src/constants.js +6 -3
- package/dist/backend/backend/src/constants.js.map +1 -1
- package/dist/backend/backend/src/controllers/browser/browser.controller.d.ts +21 -2
- package/dist/backend/backend/src/controllers/browser/browser.controller.d.ts.map +1 -1
- package/dist/backend/backend/src/controllers/browser/browser.controller.js +167 -29
- package/dist/backend/backend/src/controllers/browser/browser.controller.js.map +1 -1
- package/dist/backend/backend/src/controllers/browser/browser.routes.d.ts +1 -1
- package/dist/backend/backend/src/controllers/browser/browser.routes.d.ts.map +1 -1
- package/dist/backend/backend/src/controllers/browser/browser.routes.js +7 -3
- package/dist/backend/backend/src/controllers/browser/browser.routes.js.map +1 -1
- package/dist/backend/backend/src/controllers/data/data.controller.d.ts +47 -0
- package/dist/backend/backend/src/controllers/data/data.controller.d.ts.map +1 -0
- package/dist/backend/backend/src/controllers/data/data.controller.js +201 -0
- package/dist/backend/backend/src/controllers/data/data.controller.js.map +1 -0
- package/dist/backend/backend/src/controllers/data/data.routes.d.ts +18 -0
- package/dist/backend/backend/src/controllers/data/data.routes.d.ts.map +1 -0
- package/dist/backend/backend/src/controllers/data/data.routes.js +44 -0
- package/dist/backend/backend/src/controllers/data/data.routes.js.map +1 -0
- package/dist/backend/backend/src/controllers/monitoring/token-usage.controller.d.ts +3 -2
- package/dist/backend/backend/src/controllers/monitoring/token-usage.controller.d.ts.map +1 -1
- package/dist/backend/backend/src/controllers/monitoring/token-usage.controller.js +5 -3
- package/dist/backend/backend/src/controllers/monitoring/token-usage.controller.js.map +1 -1
- package/dist/backend/backend/src/controllers/system/cron-task.controller.d.ts +4 -0
- package/dist/backend/backend/src/controllers/system/cron-task.controller.d.ts.map +1 -1
- package/dist/backend/backend/src/controllers/system/cron-task.controller.js +20 -0
- package/dist/backend/backend/src/controllers/system/cron-task.controller.js.map +1 -1
- package/dist/backend/backend/src/controllers/task-management/task-management.controller.d.ts.map +1 -1
- package/dist/backend/backend/src/controllers/task-management/task-management.controller.js +18 -0
- package/dist/backend/backend/src/controllers/task-management/task-management.controller.js.map +1 -1
- package/dist/backend/backend/src/controllers/team/team-export.controller.d.ts +32 -0
- package/dist/backend/backend/src/controllers/team/team-export.controller.d.ts.map +1 -0
- package/dist/backend/backend/src/controllers/team/team-export.controller.js +61 -0
- package/dist/backend/backend/src/controllers/team/team-export.controller.js.map +1 -0
- package/dist/backend/backend/src/controllers/team/team.routes.d.ts.map +1 -1
- package/dist/backend/backend/src/controllers/team/team.routes.js +7 -0
- package/dist/backend/backend/src/controllers/team/team.routes.js.map +1 -1
- package/dist/backend/backend/src/index.d.ts.map +1 -1
- package/dist/backend/backend/src/index.js +37 -7
- package/dist/backend/backend/src/index.js.map +1 -1
- package/dist/backend/backend/src/routes/api.routes.d.ts.map +1 -1
- package/dist/backend/backend/src/routes/api.routes.js +4 -1
- package/dist/backend/backend/src/routes/api.routes.js.map +1 -1
- package/dist/backend/backend/src/services/agent/agent-registration.service.d.ts.map +1 -1
- package/dist/backend/backend/src/services/agent/agent-registration.service.js +6 -2
- package/dist/backend/backend/src/services/agent/agent-registration.service.js.map +1 -1
- package/dist/backend/backend/src/services/agent/idle-detection.service.d.ts.map +1 -1
- package/dist/backend/backend/src/services/agent/idle-detection.service.js +17 -2
- package/dist/backend/backend/src/services/agent/idle-detection.service.js.map +1 -1
- package/dist/backend/backend/src/services/agent/runtime-agent.service.abstract.d.ts +1 -1
- package/dist/backend/backend/src/services/agent/runtime-agent.service.abstract.js +2 -2
- package/dist/backend/backend/src/services/agent/runtime-agent.service.abstract.js.map +1 -1
- package/dist/backend/backend/src/services/agent/task-planning.service.d.ts +134 -0
- package/dist/backend/backend/src/services/agent/task-planning.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/agent/task-planning.service.js +291 -0
- package/dist/backend/backend/src/services/agent/task-planning.service.js.map +1 -0
- package/dist/backend/backend/src/services/ai/prompt-modules/communication.module.d.ts +11 -0
- package/dist/backend/backend/src/services/ai/prompt-modules/communication.module.d.ts.map +1 -1
- package/dist/backend/backend/src/services/ai/prompt-modules/communication.module.js +47 -18
- package/dist/backend/backend/src/services/ai/prompt-modules/communication.module.js.map +1 -1
- package/dist/backend/backend/src/services/ai/prompt-modules/skills-reference.module.d.ts +14 -0
- package/dist/backend/backend/src/services/ai/prompt-modules/skills-reference.module.d.ts.map +1 -1
- package/dist/backend/backend/src/services/ai/prompt-modules/skills-reference.module.js +47 -4
- package/dist/backend/backend/src/services/ai/prompt-modules/skills-reference.module.js.map +1 -1
- package/dist/backend/backend/src/services/browser/browser-bridge.service.d.ts +13 -9
- package/dist/backend/backend/src/services/browser/browser-bridge.service.d.ts.map +1 -1
- package/dist/backend/backend/src/services/browser/browser-bridge.service.js +44 -12
- package/dist/backend/backend/src/services/browser/browser-bridge.service.js.map +1 -1
- package/dist/backend/backend/src/services/browser/browser-proxy.service.d.ts +176 -0
- package/dist/backend/backend/src/services/browser/browser-proxy.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/browser/browser-proxy.service.js +441 -0
- package/dist/backend/backend/src/services/browser/browser-proxy.service.js.map +1 -0
- package/dist/backend/backend/src/services/browser/browser-relay-adapter.service.d.ts +162 -0
- package/dist/backend/backend/src/services/browser/browser-relay-adapter.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/browser/browser-relay-adapter.service.js +350 -0
- package/dist/backend/backend/src/services/browser/browser-relay-adapter.service.js.map +1 -0
- package/dist/backend/backend/src/services/cloud/cloud-initializer.d.ts +8 -0
- package/dist/backend/backend/src/services/cloud/cloud-initializer.d.ts.map +1 -1
- package/dist/backend/backend/src/services/cloud/cloud-initializer.js +27 -0
- package/dist/backend/backend/src/services/cloud/cloud-initializer.js.map +1 -1
- package/dist/backend/backend/src/services/cloud/cloud-sync.types.d.ts +1 -1
- package/dist/backend/backend/src/services/cloud/cloud-sync.types.d.ts.map +1 -1
- package/dist/backend/backend/src/services/cloud/cloud-sync.types.js +2 -0
- package/dist/backend/backend/src/services/cloud/cloud-sync.types.js.map +1 -1
- package/dist/backend/backend/src/services/core/team-export.service.d.ts +103 -0
- package/dist/backend/backend/src/services/core/team-export.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/core/team-export.service.js +182 -0
- package/dist/backend/backend/src/services/core/team-export.service.js.map +1 -0
- package/dist/backend/backend/src/services/data/data-object-store.service.d.ts +160 -0
- package/dist/backend/backend/src/services/data/data-object-store.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/data/data-object-store.service.js +434 -0
- package/dist/backend/backend/src/services/data/data-object-store.service.js.map +1 -0
- package/dist/backend/backend/src/services/data/data-object.types.d.ts +190 -0
- package/dist/backend/backend/src/services/data/data-object.types.d.ts.map +1 -0
- package/dist/backend/backend/src/services/data/data-object.types.js +143 -0
- package/dist/backend/backend/src/services/data/data-object.types.js.map +1 -0
- package/dist/backend/backend/src/services/data/schema-registry.service.d.ts +108 -0
- package/dist/backend/backend/src/services/data/schema-registry.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/data/schema-registry.service.js +290 -0
- package/dist/backend/backend/src/services/data/schema-registry.service.js.map +1 -0
- package/dist/backend/backend/src/services/data/sink-registry.service.d.ts +87 -0
- package/dist/backend/backend/src/services/data/sink-registry.service.d.ts.map +1 -0
- package/dist/backend/backend/src/services/data/sink-registry.service.js +188 -0
- package/dist/backend/backend/src/services/data/sink-registry.service.js.map +1 -0
- package/dist/backend/backend/src/services/messaging/message-router.service.d.ts.map +1 -1
- package/dist/backend/backend/src/services/messaging/message-router.service.js +7 -0
- package/dist/backend/backend/src/services/messaging/message-router.service.js.map +1 -1
- package/dist/backend/backend/src/services/monitoring/token-usage.service.d.ts +55 -2
- package/dist/backend/backend/src/services/monitoring/token-usage.service.d.ts.map +1 -1
- package/dist/backend/backend/src/services/monitoring/token-usage.service.js +89 -5
- package/dist/backend/backend/src/services/monitoring/token-usage.service.js.map +1 -1
- package/dist/backend/backend/src/services/session/pty/pty-session-backend.js +1 -1
- package/dist/backend/backend/src/services/session/pty/pty-session-backend.js.map +1 -1
- package/dist/backend/backend/src/services/workflow/cron-task.service.d.ts +105 -14
- package/dist/backend/backend/src/services/workflow/cron-task.service.d.ts.map +1 -1
- package/dist/backend/backend/src/services/workflow/cron-task.service.js +400 -123
- package/dist/backend/backend/src/services/workflow/cron-task.service.js.map +1 -1
- package/dist/backend/backend/src/types/cron-task.types.d.ts +1 -1
- package/dist/backend/backend/src/types/data-object.types.d.ts +117 -0
- package/dist/backend/backend/src/types/data-object.types.d.ts.map +1 -0
- package/dist/backend/backend/src/types/data-object.types.js +23 -0
- package/dist/backend/backend/src/types/data-object.types.js.map +1 -0
- package/dist/backend/backend/src/types/settings.types.js +1 -1
- package/dist/backend/config/constants.d.ts +3 -3
- package/dist/backend/config/constants.js +3 -3
- package/dist/backend/config/constants.js.map +1 -1
- package/dist/cli/backend/src/constants.d.ts +7 -4
- package/dist/cli/backend/src/constants.d.ts.map +1 -1
- package/dist/cli/backend/src/constants.js +6 -3
- package/dist/cli/backend/src/constants.js.map +1 -1
- package/dist/cli/backend/src/types/settings.types.js +1 -1
- package/dist/cli/config/constants.d.ts +3 -3
- package/dist/cli/config/constants.js +3 -3
- package/dist/cli/config/constants.js.map +1 -1
- package/frontend/dist/assets/index-371b68d4.css +33 -0
- package/frontend/dist/assets/{index-9af2ea40.js → index-506f70da.js} +321 -321
- package/frontend/dist/index.html +2 -2
- package/package.json +1 -1
- package/frontend/dist/assets/index-b19b2478.css +0 -33
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# pipe-to-sink: Route structured data to a registered sink via the Data API.
|
|
3
|
+
#
|
|
4
|
+
# Input (JSON):
|
|
5
|
+
# {
|
|
6
|
+
# "sinkId": "steve-inputs",
|
|
7
|
+
# "data": { "title": "...", "source": "slack", ... },
|
|
8
|
+
# "projectPath": "/path/to/project" (optional — falls back to CREWLY_PROJECT_PATH)
|
|
9
|
+
# }
|
|
10
|
+
#
|
|
11
|
+
# Calls: POST /api/v2/data/sinks/:sinkId
|
|
12
|
+
# Returns: JSON with the created DataObject ID on success, or error details.
|
|
13
|
+
set -euo pipefail
|
|
14
|
+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
15
|
+
source "${SCRIPT_DIR}/../../_common/lib.sh"
|
|
16
|
+
|
|
17
|
+
INPUT=$(read_json_input "${1:-}")
|
|
18
|
+
[ -z "$INPUT" ] && error_exit "Usage: execute.sh '{\"sinkId\":\"...\",\"data\":{...}}' or echo '{...}' | execute.sh"
|
|
19
|
+
|
|
20
|
+
SINK_ID=$(printf '%s' "$INPUT" | jq -r '.sinkId // empty')
|
|
21
|
+
require_param "sinkId" "$SINK_ID"
|
|
22
|
+
|
|
23
|
+
DATA=$(printf '%s' "$INPUT" | jq '.data // empty')
|
|
24
|
+
if [ "$DATA" = "null" ] || [ "$DATA" = "" ] || [ "$DATA" = "empty" ]; then
|
|
25
|
+
error_exit "Missing required parameter: data"
|
|
26
|
+
fi
|
|
27
|
+
|
|
28
|
+
# Resolve projectPath: explicit > env var > omit
|
|
29
|
+
PROJECT_PATH=$(printf '%s' "$INPUT" | jq -r '.projectPath // empty')
|
|
30
|
+
if [ -z "$PROJECT_PATH" ] && [ -n "${CREWLY_PROJECT_PATH:-}" ]; then
|
|
31
|
+
PROJECT_PATH="$CREWLY_PROJECT_PATH"
|
|
32
|
+
fi
|
|
33
|
+
|
|
34
|
+
# Build request body
|
|
35
|
+
BODY=$(jq -n \
|
|
36
|
+
--argjson data "$DATA" \
|
|
37
|
+
--arg projectPath "$PROJECT_PATH" \
|
|
38
|
+
'{data: $data} + (if $projectPath != "" then {projectPath: $projectPath} else {} end)')
|
|
39
|
+
|
|
40
|
+
# POST to the data API
|
|
41
|
+
api_call POST "/v2/data/sinks/${SINK_ID}" "$BODY"
|
|
@@ -10,6 +10,8 @@ INPUT=$(read_json_input "${1:-}")
|
|
|
10
10
|
ABSOLUTE_TASK_PATH=$(printf '%s' "$INPUT" | jq -r '.absoluteTaskPath // empty')
|
|
11
11
|
require_param "absoluteTaskPath" "$ABSOLUTE_TASK_PATH"
|
|
12
12
|
|
|
13
|
-
BODY=$(jq -n --arg absoluteTaskPath "$ABSOLUTE_TASK_PATH"
|
|
13
|
+
BODY=$(jq -n --arg absoluteTaskPath "$ABSOLUTE_TASK_PATH" \
|
|
14
|
+
--arg taskPath "$ABSOLUTE_TASK_PATH" \
|
|
15
|
+
'{absoluteTaskPath: $absoluteTaskPath, taskPath: $taskPath}')
|
|
14
16
|
|
|
15
17
|
api_call POST "/task-management/read-task" "$BODY"
|
|
@@ -5,7 +5,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
|
5
5
|
source "${SCRIPT_DIR}/../../_common/lib.sh"
|
|
6
6
|
|
|
7
7
|
INPUT=$(read_json_input "${1:-}")
|
|
8
|
-
[ -z "$INPUT" ] && error_exit "Usage: execute.sh '{\"sessionName\":\"dev-1\",\"progress\":50,\"current\":\"Implementing tests\"}'"
|
|
8
|
+
[ -z "$INPUT" ] && error_exit "Usage: execute.sh '{\"sessionName\":\"dev-1\",\"progress\":50,\"current\":\"Implementing tests\",\"taskGroup\":\"delegated\"}'"
|
|
9
9
|
|
|
10
10
|
SESSION_NAME=$(printf '%s' "$INPUT" | jq -r '.sessionName // empty')
|
|
11
11
|
PROGRESS=$(printf '%s' "$INPUT" | jq -r '.progress // empty')
|
|
@@ -20,6 +20,8 @@ BODY=$(printf '%s' "$INPUT" | jq '{
|
|
|
20
20
|
progress: (.progress | tonumber),
|
|
21
21
|
current: .current
|
|
22
22
|
} +
|
|
23
|
+
(if .projectPath then {projectPath: .projectPath} else {} end) +
|
|
24
|
+
(if .taskGroup then {taskGroup: .taskGroup} else {} end) +
|
|
23
25
|
(if .completed then {completed: .completed} else {} end) +
|
|
24
26
|
(if .nextSteps then {nextSteps: .nextSteps} else {} end) +
|
|
25
27
|
(if .blockers then {blockers: .blockers} else {} end) +
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: screenshot-compare
|
|
3
|
+
description: Compare two screenshots (e.g., iOS reference vs Web implementation) using Gemini Vision API. Returns a structured diff report with categorized issues, severity levels, and CSS fix suggestions. Use when verifying UI parity, checking design implementation accuracy, or auditing visual consistency across platforms.
|
|
4
|
+
category: qa
|
|
5
|
+
assignableRoles:
|
|
6
|
+
- "*"
|
|
7
|
+
version: "1.0.0"
|
|
8
|
+
tags:
|
|
9
|
+
- screenshot
|
|
10
|
+
- visual-qa
|
|
11
|
+
- comparison
|
|
12
|
+
- gemini
|
|
13
|
+
- parity
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Screenshot Compare
|
|
17
|
+
|
|
18
|
+
Compare two UI screenshots using Gemini Vision API and get a structured diff report.
|
|
19
|
+
|
|
20
|
+
## Actions
|
|
21
|
+
|
|
22
|
+
| Parameter | Required | Description |
|
|
23
|
+
|-----------|----------|-------------|
|
|
24
|
+
| `reference` | Yes | Path to the reference screenshot (source of truth, e.g., iOS app) |
|
|
25
|
+
| `target` | Yes | Path to the target screenshot (implementation to verify, e.g., web app) |
|
|
26
|
+
| `focus` | No | Comma-separated focus areas: `icons,layout,colors,text,images,spacing` (default: all) |
|
|
27
|
+
| `context` | No | Additional context (e.g., "This is the settings page", "Dark mode") |
|
|
28
|
+
|
|
29
|
+
## Usage
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
# Basic comparison
|
|
33
|
+
bash execute.sh '{"reference":"/path/to/ios-screenshot.png","target":"/path/to/web-screenshot.png"}'
|
|
34
|
+
|
|
35
|
+
# Focused comparison (only icons and colors)
|
|
36
|
+
bash execute.sh '{"reference":"ref.png","target":"web.png","focus":"icons,colors"}'
|
|
37
|
+
|
|
38
|
+
# With context
|
|
39
|
+
bash execute.sh '{"reference":"ios-home.png","target":"web-home.png","context":"Home screen, light mode"}'
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Output Format
|
|
43
|
+
|
|
44
|
+
Returns JSON with:
|
|
45
|
+
- `matchScore` — 0-100 overall similarity score
|
|
46
|
+
- `totalIssues` — count of differences found
|
|
47
|
+
- `issues[]` — array of categorized differences:
|
|
48
|
+
- `type` — icon_missing, layout_shift, color_mismatch, text_mismatch, font_mismatch, spacing, etc.
|
|
49
|
+
- `severity` — critical, major, minor
|
|
50
|
+
- `element` — which UI element is affected
|
|
51
|
+
- `suggestion` — specific CSS/code fix
|
|
52
|
+
|
|
53
|
+
## Issue Types
|
|
54
|
+
|
|
55
|
+
| Type | Description |
|
|
56
|
+
|------|-------------|
|
|
57
|
+
| `icon_missing` | Icon present in reference but missing in target |
|
|
58
|
+
| `icon_wrong` | Icon exists but is the wrong icon |
|
|
59
|
+
| `layout_shift` | Elements positioned differently |
|
|
60
|
+
| `spacing` | Padding/margin differences |
|
|
61
|
+
| `color_mismatch` | Colors don't match between reference and target |
|
|
62
|
+
| `text_mismatch` | Text content differs |
|
|
63
|
+
| `font_mismatch` | Font family, size, or weight differs |
|
|
64
|
+
| `image_missing` | Image present in reference but missing/broken in target |
|
|
65
|
+
| `image_wrong` | Image exists but is different |
|
|
66
|
+
| `border_radius` | Rounded corners differ |
|
|
67
|
+
| `shadow` | Box shadow differences |
|
|
68
|
+
| `alignment` | Vertical/horizontal alignment issues |
|
|
69
|
+
| `responsive` | Layout doesn't adapt correctly |
|
|
70
|
+
|
|
71
|
+
## Prerequisites
|
|
72
|
+
|
|
73
|
+
- `GEMINI_API_KEY` environment variable or configured in Settings > API Keys
|
|
74
|
+
- Both image files must exist and be PNG, JPG, GIF, or WebP
|
|
75
|
+
- Images should be under 4MB each
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# Screenshot Compare — Gemini Vision API dual-image comparison
|
|
3
|
+
#
|
|
4
|
+
# Sends two screenshots (e.g., iOS reference + Web implementation) to
|
|
5
|
+
# Gemini Vision and returns a structured diff report.
|
|
6
|
+
#
|
|
7
|
+
# Usage:
|
|
8
|
+
# bash execute.sh '{"reference":"/path/to/ios-screenshot.png","target":"/path/to/web-screenshot.png"}'
|
|
9
|
+
# bash execute.sh '{"reference":"ref.png","target":"web.png","focus":"icons,layout,colors"}'
|
|
10
|
+
#
|
|
11
|
+
# Requires: GEMINI_API_KEY environment variable
|
|
12
|
+
#
|
|
13
|
+
# Output: JSON with differences categorized by type (icon, layout, color, text, image)
|
|
14
|
+
|
|
15
|
+
set -euo pipefail
|
|
16
|
+
|
|
17
|
+
# ── Configuration ──────────────────────────────────────────────────────────────
|
|
18
|
+
MODEL="${GEMINI_MODEL:-gemini-2.5-flash-preview-05-20}"
|
|
19
|
+
GEMINI_API_BASE="https://generativelanguage.googleapis.com"
|
|
20
|
+
GEMINI_CONTENT_URL="${GEMINI_API_BASE}/v1beta/models/${MODEL}:generateContent"
|
|
21
|
+
MAX_IMAGE_SIZE_MB=4
|
|
22
|
+
|
|
23
|
+
# ── Input parsing ──────────────────────────────────────────────────────────────
|
|
24
|
+
INPUT="${1:-}"
|
|
25
|
+
if [ -z "$INPUT" ]; then
|
|
26
|
+
echo '{"success":false,"error":"Usage: execute.sh \u0027{\"reference\":\"/path/to/ref.png\",\"target\":\"/path/to/target.png\"}\u0027"}'
|
|
27
|
+
exit 1
|
|
28
|
+
fi
|
|
29
|
+
|
|
30
|
+
REFERENCE=$(echo "$INPUT" | jq -r '.reference // empty')
|
|
31
|
+
TARGET=$(echo "$INPUT" | jq -r '.target // empty')
|
|
32
|
+
FOCUS=$(echo "$INPUT" | jq -r '.focus // "all"')
|
|
33
|
+
CONTEXT=$(echo "$INPUT" | jq -r '.context // ""')
|
|
34
|
+
|
|
35
|
+
if [ -z "$REFERENCE" ] || [ -z "$TARGET" ]; then
|
|
36
|
+
echo '{"success":false,"error":"Both reference and target image paths are required"}'
|
|
37
|
+
exit 1
|
|
38
|
+
fi
|
|
39
|
+
|
|
40
|
+
if [ ! -f "$REFERENCE" ]; then
|
|
41
|
+
echo "{\"success\":false,\"error\":\"Reference image not found: $REFERENCE\"}"
|
|
42
|
+
exit 1
|
|
43
|
+
fi
|
|
44
|
+
|
|
45
|
+
if [ ! -f "$TARGET" ]; then
|
|
46
|
+
echo "{\"success\":false,\"error\":\"Target image not found: $TARGET\"}"
|
|
47
|
+
exit 1
|
|
48
|
+
fi
|
|
49
|
+
|
|
50
|
+
# ── API key resolution ─────────────────────────────────────────────────────────
|
|
51
|
+
if [ -z "${GEMINI_API_KEY:-}" ]; then
|
|
52
|
+
# Try loading from settings via Crewly API
|
|
53
|
+
GEMINI_API_KEY=$(curl -sf "http://localhost:8787/api/settings" 2>/dev/null | jq -r '.data.apiKeys.global.gemini // empty' 2>/dev/null || true)
|
|
54
|
+
fi
|
|
55
|
+
|
|
56
|
+
if [ -z "${GEMINI_API_KEY:-}" ]; then
|
|
57
|
+
echo '{"success":false,"error":"GEMINI_API_KEY not set. Set via environment or Settings > API Keys."}'
|
|
58
|
+
exit 1
|
|
59
|
+
fi
|
|
60
|
+
|
|
61
|
+
# ── Image encoding ─────────────────────────────────────────────────────────────
|
|
62
|
+
get_mime_type() {
|
|
63
|
+
local file="$1"
|
|
64
|
+
local ext="${file##*.}"
|
|
65
|
+
case "$ext" in
|
|
66
|
+
png) echo "image/png" ;;
|
|
67
|
+
jpg|jpeg) echo "image/jpeg" ;;
|
|
68
|
+
gif) echo "image/gif" ;;
|
|
69
|
+
webp) echo "image/webp" ;;
|
|
70
|
+
*) echo "image/png" ;; # default
|
|
71
|
+
esac
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
REF_MIME=$(get_mime_type "$REFERENCE")
|
|
75
|
+
TGT_MIME=$(get_mime_type "$TARGET")
|
|
76
|
+
REF_B64=$(base64 < "$REFERENCE" | tr -d '\n')
|
|
77
|
+
TGT_B64=$(base64 < "$TARGET" | tr -d '\n')
|
|
78
|
+
|
|
79
|
+
# ── Build prompt ───────────────────────────────────────────────────────────────
|
|
80
|
+
FOCUS_INSTRUCTION=""
|
|
81
|
+
if [ "$FOCUS" != "all" ]; then
|
|
82
|
+
FOCUS_INSTRUCTION="Focus specifically on these areas: ${FOCUS}."
|
|
83
|
+
fi
|
|
84
|
+
|
|
85
|
+
CONTEXT_INSTRUCTION=""
|
|
86
|
+
if [ -n "$CONTEXT" ]; then
|
|
87
|
+
CONTEXT_INSTRUCTION="Additional context: ${CONTEXT}"
|
|
88
|
+
fi
|
|
89
|
+
|
|
90
|
+
PROMPT="You are a pixel-perfect UI comparison expert. Compare these two screenshots:
|
|
91
|
+
|
|
92
|
+
IMAGE 1 (Reference — the source of truth, e.g., iOS app or design mockup)
|
|
93
|
+
IMAGE 2 (Target — the implementation to verify, e.g., web app)
|
|
94
|
+
|
|
95
|
+
${FOCUS_INSTRUCTION}
|
|
96
|
+
${CONTEXT_INSTRUCTION}
|
|
97
|
+
|
|
98
|
+
Analyze every visible difference and return a JSON array of issues found. Each issue must have:
|
|
99
|
+
- \"type\": one of \"icon_missing\", \"icon_wrong\", \"layout_shift\", \"spacing\", \"color_mismatch\", \"text_mismatch\", \"font_mismatch\", \"image_missing\", \"image_wrong\", \"border_radius\", \"shadow\", \"alignment\", \"responsive\", \"other\"
|
|
100
|
+
- \"severity\": \"critical\" (functionality broken), \"major\" (clearly visible), \"minor\" (subtle)
|
|
101
|
+
- \"element\": which UI element is affected (e.g., \"header logo\", \"submit button\", \"nav bar\")
|
|
102
|
+
- \"reference\": what it looks like in Image 1
|
|
103
|
+
- \"target\": what it looks like in Image 2
|
|
104
|
+
- \"suggestion\": specific CSS/code fix suggestion
|
|
105
|
+
|
|
106
|
+
Return ONLY valid JSON in this format:
|
|
107
|
+
{
|
|
108
|
+
\"summary\": \"Brief overall assessment\",
|
|
109
|
+
\"matchScore\": 85,
|
|
110
|
+
\"totalIssues\": 5,
|
|
111
|
+
\"issues\": [
|
|
112
|
+
{
|
|
113
|
+
\"type\": \"...\",
|
|
114
|
+
\"severity\": \"...\",
|
|
115
|
+
\"element\": \"...\",
|
|
116
|
+
\"reference\": \"...\",
|
|
117
|
+
\"target\": \"...\",
|
|
118
|
+
\"suggestion\": \"...\"
|
|
119
|
+
}
|
|
120
|
+
]
|
|
121
|
+
}"
|
|
122
|
+
|
|
123
|
+
# ── API call ───────────────────────────────────────────────────────────────────
|
|
124
|
+
REQUEST_BODY=$(cat <<ENDJSON
|
|
125
|
+
{
|
|
126
|
+
"contents": [{
|
|
127
|
+
"parts": [
|
|
128
|
+
{"text": $(echo "$PROMPT" | jq -Rs .)},
|
|
129
|
+
{
|
|
130
|
+
"inline_data": {
|
|
131
|
+
"mime_type": "${REF_MIME}",
|
|
132
|
+
"data": "${REF_B64}"
|
|
133
|
+
}
|
|
134
|
+
},
|
|
135
|
+
{
|
|
136
|
+
"inline_data": {
|
|
137
|
+
"mime_type": "${TGT_MIME}",
|
|
138
|
+
"data": "${TGT_B64}"
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
]
|
|
142
|
+
}],
|
|
143
|
+
"generationConfig": {
|
|
144
|
+
"temperature": 0.1,
|
|
145
|
+
"maxOutputTokens": 4096,
|
|
146
|
+
"responseMimeType": "application/json"
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
ENDJSON
|
|
150
|
+
)
|
|
151
|
+
|
|
152
|
+
TMPFILE=$(mktemp /tmp/gemini-compare-XXXXXX.json)
|
|
153
|
+
echo "$REQUEST_BODY" > "$TMPFILE"
|
|
154
|
+
RESPONSE=$(curl -sf -X POST \
|
|
155
|
+
"${GEMINI_CONTENT_URL}?key=${GEMINI_API_KEY}" \
|
|
156
|
+
-H "Content-Type: application/json" \
|
|
157
|
+
-d "@${TMPFILE}" 2>&1)
|
|
158
|
+
rm -f "$TMPFILE"
|
|
159
|
+
|
|
160
|
+
HTTP_CODE=$?
|
|
161
|
+
if [ $HTTP_CODE -ne 0 ]; then
|
|
162
|
+
echo "{\"success\":false,\"error\":\"Gemini API request failed (curl exit $HTTP_CODE)\"}"
|
|
163
|
+
exit 1
|
|
164
|
+
fi
|
|
165
|
+
|
|
166
|
+
# ── Parse response ─────────────────────────────────────────────────────────────
|
|
167
|
+
ERROR_MSG=$(echo "$RESPONSE" | jq -r '.error.message // empty' 2>/dev/null)
|
|
168
|
+
if [ -n "$ERROR_MSG" ]; then
|
|
169
|
+
echo "{\"success\":false,\"error\":\"Gemini API error: $ERROR_MSG\"}"
|
|
170
|
+
exit 1
|
|
171
|
+
fi
|
|
172
|
+
|
|
173
|
+
# Extract the text content from Gemini response
|
|
174
|
+
RESULT_TEXT=$(echo "$RESPONSE" | jq -r '.candidates[0].content.parts[0].text // empty' 2>/dev/null)
|
|
175
|
+
|
|
176
|
+
if [ -z "$RESULT_TEXT" ]; then
|
|
177
|
+
echo '{"success":false,"error":"Empty response from Gemini API"}'
|
|
178
|
+
exit 1
|
|
179
|
+
fi
|
|
180
|
+
|
|
181
|
+
# Output: wrap the Gemini JSON result in a success envelope
|
|
182
|
+
echo "{\"success\":true,\"model\":\"${MODEL}\",\"reference\":\"${REFERENCE}\",\"target\":\"${TARGET}\",\"comparison\":${RESULT_TEXT}}"
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "screenshot-compare",
|
|
3
|
+
"displayName": "Screenshot Compare (Gemini Vision)",
|
|
4
|
+
"description": "Compare two UI screenshots using Gemini Vision API. Returns categorized diffs with severity and fix suggestions.",
|
|
5
|
+
"category": "qa",
|
|
6
|
+
"assignableRoles": ["*"],
|
|
7
|
+
"version": "1.0.0",
|
|
8
|
+
"author": "Crewly Team",
|
|
9
|
+
"requires": ["GEMINI_API_KEY"]
|
|
10
|
+
}
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: xiaoyuzhoufm-transcript
|
|
3
|
+
description: Extract and transcribe 小宇宙 (Xiaoyuzhou FM) podcast episodes. Downloads audio from episode URL and uses Gemini Audio API for transcription with speaker identification and timestamps.
|
|
4
|
+
category: content
|
|
5
|
+
assignableRoles:
|
|
6
|
+
- "*"
|
|
7
|
+
version: "1.0.0"
|
|
8
|
+
tags:
|
|
9
|
+
- podcast
|
|
10
|
+
- transcript
|
|
11
|
+
- audio
|
|
12
|
+
- gemini
|
|
13
|
+
- xiaoyuzhou
|
|
14
|
+
- chinese
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
# 小宇宙 Podcast Transcript
|
|
18
|
+
|
|
19
|
+
Extract and transcribe podcast episodes from 小宇宙 (Xiaoyuzhou FM) using Gemini Audio API.
|
|
20
|
+
|
|
21
|
+
## Pipeline
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
Episode URL → Fetch HTML → Extract audio URL → Download m4a → Upload to Gemini → Transcribe → Output
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Usage
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
# Transcribe from URL
|
|
31
|
+
bash execute.sh '{"url":"https://www.xiaoyuzhoufm.com/episode/69bbc8ea3c625cc5ae21b461"}'
|
|
32
|
+
|
|
33
|
+
# Save transcript to file
|
|
34
|
+
bash execute.sh '{"url":"https://www.xiaoyuzhoufm.com/episode/...","outputFile":"./transcripts/episode.md"}'
|
|
35
|
+
|
|
36
|
+
# Transcribe a local audio file (skip download)
|
|
37
|
+
bash execute.sh '{"audioFile":"/path/to/podcast.m4a"}'
|
|
38
|
+
|
|
39
|
+
# Specify language hint
|
|
40
|
+
bash execute.sh '{"url":"...","language":"Chinese"}'
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Parameters
|
|
44
|
+
|
|
45
|
+
| Parameter | Required | Description |
|
|
46
|
+
|-----------|----------|-------------|
|
|
47
|
+
| `url` | Yes* | 小宇宙 episode URL |
|
|
48
|
+
| `audioFile` | Yes* | Path to local audio file (alternative to url) |
|
|
49
|
+
| `outputFile` | No | Save transcript as markdown to this path |
|
|
50
|
+
| `language` | No | Language hint for Gemini (default: auto-detect) |
|
|
51
|
+
|
|
52
|
+
*Either `url` or `audioFile` is required.
|
|
53
|
+
|
|
54
|
+
## Output Format
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"success": true,
|
|
59
|
+
"title": "Episode Title",
|
|
60
|
+
"url": "https://www.xiaoyuzhoufm.com/episode/...",
|
|
61
|
+
"audioUrl": "https://media.xyzcdn.net/.../audio.m4a",
|
|
62
|
+
"model": "gemini-2.5-flash-preview-05-20",
|
|
63
|
+
"transcriptLength": 12345,
|
|
64
|
+
"transcript": "**Speaker 1:** [00:00:00] Welcome to the show..."
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Transcript Format
|
|
69
|
+
|
|
70
|
+
- Speaker turns identified with `**Speaker Name:** text`
|
|
71
|
+
- Timestamps at segment/topic changes: `[HH:MM:SS]`
|
|
72
|
+
- Preserves original language (Chinese/English/mixed)
|
|
73
|
+
|
|
74
|
+
## Prerequisites
|
|
75
|
+
|
|
76
|
+
- `GEMINI_API_KEY` environment variable or configured in Settings > API Keys
|
|
77
|
+
- `curl` and `jq` available in PATH
|
|
78
|
+
- Internet access to xiaoyuzhoufm.com and Gemini API
|
|
79
|
+
|
|
80
|
+
## Notes
|
|
81
|
+
|
|
82
|
+
- Audio files are typically 30-90 minutes (20-80 MB)
|
|
83
|
+
- Gemini processing takes 30-120 seconds depending on audio length
|
|
84
|
+
- Maximum wait time: 5 minutes for Gemini processing
|
|
85
|
+
- Temp files are auto-cleaned on exit
|