opencastle 0.33.9 → 0.34.0

package/src/orchestrator/plugins/convex/references/subscription-cost.md

@@ -0,0 +1,252 @@
+# Subscription Cost
+
+Use these rules when the problem is too many reactive subscriptions, queries invalidating too frequently, or React components re-rendering excessively due to Convex state changes.
+
+## Core Principle
+
+Every `useQuery` and `usePaginatedQuery` call creates a live subscription. The server tracks the query's read set and re-executes the query whenever any document in that read set changes. Subscription cost scales with:
+
+`subscriptions x invalidation_frequency x query_cost`
+
+Subscriptions are not inherently bad. Convex reactivity is often the right default. The goal is to reduce unnecessary invalidation work, not to eliminate subscriptions on principle.
+
+## Symptoms
+
+- Dashboard shows high active subscription count
+- UI feels sluggish or laggy despite fast individual queries
+- React profiling shows frequent re-renders from Convex state
+- Pages with many components each running their own `useQuery`
+- Paginated lists where every loaded page stays subscribed
+
+## Common Causes
+
+### Reactive queries on low-freshness flows
+
+Some user flows are read-heavy and do not need live updates every time the underlying data changes. In those cases, ongoing subscriptions may cost more than they are worth.
+
+### Overly broad queries
+
+A query that returns a large result set invalidates whenever any document in that set changes. The broader the query, the more frequent the invalidation.
+
+### Too many subscriptions per page
+
+A page with 20 list items, each running its own `useQuery` to fetch related data, creates 20+ subscriptions per visitor.
+
+### Paginated queries keeping all pages live
+
+`usePaginatedQuery` with `loadMore` keeps every loaded page subscribed. On a page where a user has scrolled through 10 pages, all 10 stay reactive.
+
+### Frequently-updated fields on widely-read documents
+
+A document that many queries touch gets a frequently-updated field (like `lastSeen`, `lastActiveAt`, or a counter). Every write to that field invalidates every subscription that reads the document, even if those subscriptions never use the field. This is different from OCC conflicts (see `occ-conflicts.md`), which are write-vs-write contention. This is write-vs-subscription: the write succeeds fine, but it forces hundreds of queries to re-run for no reason.
+
+## Fix Order
+
+### 1. Use point-in-time reads when live updates are not valuable
+
+Keep `useQuery` and `usePaginatedQuery` by default when the product benefits from fresh live data.
+
+Consider a point-in-time read instead when all of these are true:
+
+- the flow is high-read
+- the underlying data changes less often than users need to see
+- explicit refresh, periodic refresh, or a fresh read on navigation is acceptable
+
+Possible implementations depend on environment:
+
+- a server-rendered fetch
+- a framework helper like `fetchQuery`
+- a point-in-time client read such as `ConvexHttpClient.query()`
+
+```ts
+// Reactive by default when fresh live data matters
+function TeamPresence() {
+  const presence = useQuery(api.teams.livePresence, { teamId });
+  return <PresenceList users={presence} />;
+}
+```
+
+```ts
+// Point-in-time read when explicit refresh is acceptable
+import { ConvexHttpClient } from "convex/browser";
+
+const client = new ConvexHttpClient(import.meta.env.VITE_CONVEX_URL);
+
+function SnapshotView() {
+  const [items, setItems] = useState<Item[]>([]);
+
+  useEffect(() => {
+    client.query(api.items.snapshot).then(setItems);
+  }, []);
+
+  return <ItemGrid items={items} />;
+}
+```
+
+Good candidates for point-in-time reads:
+
+- aggregate snapshots
+- reports
+- low-churn listings
+- flows where explicit refresh is already acceptable
+
+Keep reactive for:
+
+- collaborative editing
+- live dashboards
+- presence-heavy views
+- any surface where users expect fresh changes to appear automatically
+
+### 2. Batch related data into fewer queries
+
+Instead of N components each fetching their own related data, fetch it in a single query.
+
+```ts
+// Bad: each card fetches its own author
+function ProjectCard({ project }: { project: Project }) {
+  const author = useQuery(api.users.get, { id: project.authorId });
+  return <Card title={project.name} author={author?.name} />;
+}
+```
+
+```ts
+// Good: parent query returns projects with author names included
+function ProjectList() {
+  const projects = useQuery(api.projects.listWithAuthors);
+  return projects?.map((p) => (
+    <Card key={p._id} title={p.name} author={p.authorName} />
+  ));
+}
+```
+
+This can use denormalized fields or server-side joins in the query handler. Either way, it is one subscription instead of N.
+
+This is not automatically better. If the combined query becomes much broader and invalidates much more often, several narrower subscriptions may be the better tradeoff. Optimize for total invalidation cost, not raw subscription count.
+
+### 3. Use skip to avoid unnecessary subscriptions
+
+The `"skip"` value prevents a subscription from being created when the arguments are not ready.
+
+```ts
+// Bad: subscribes with undefined args, wastes a subscription slot
+const profile = useQuery(api.users.getProfile, { userId: selectedId! });
+```
+
+```ts
+// Good: skip when there is nothing to fetch
+const profile = useQuery(
+  api.users.getProfile,
+  selectedId ? { userId: selectedId } : "skip",
+);
+```
+
+### 4. Isolate frequently-updated fields into separate documents
+
+If a document is widely read but has a field that changes often, move that field to a separate document. Queries that do not need the field will no longer be invalidated by its writes.
+
+```ts
+// Bad: lastSeen lives on the user doc, every heartbeat invalidates
+// every query that reads this user
+const users = defineTable({
+  name: v.string(),
+  email: v.string(),
+  lastSeen: v.number(),
+});
+```
+
+```ts
+// Good: lastSeen lives in a separate heartbeat doc
+const users = defineTable({
+  name: v.string(),
+  email: v.string(),
+  heartbeatId: v.id("heartbeats"),
+});
+
+const heartbeats = defineTable({
+  lastSeen: v.number(),
+});
+```
+
+Queries that only need `name` and `email` no longer re-run on every heartbeat. Queries that actually need online status fetch the heartbeat document explicitly.
+
+For an even further optimization, if you only need a coarse online/offline boolean rather than the exact `lastSeen` timestamp, add a separate presence document with an `isOnline` flag. Update it immediately when a user comes online, and use a cron to batch-mark users offline when their heartbeat goes stale. This way the presence query only invalidates when online status actually changes, not on every heartbeat.
+
+### 5. Use the aggregate component for counts and sums
+
+Reactive global counts (`SELECT COUNT(*)` equivalent) invalidate on every insert or delete to the table. The [`@convex-dev/aggregate`](https://www.npmjs.com/package/@convex-dev/aggregate) component maintains denormalized COUNT, SUM, and MAX values efficiently so you do not need a reactive query scanning the full table.
+
+Use it for leaderboards, totals, "X items" badges, or any stat that would otherwise require scanning many rows reactively.
+
+If the aggregate component is not appropriate, prefer point-in-time reads for global stats, or precomputed summary rows updated by a cron or trigger, over reactive queries that scan large tables.
+
+### 6. Narrow query read sets
+
+Queries that return less data and touch fewer documents invalidate less often.
+
+```ts
+// Bad: returns all fields, invalidates on any field change
+export const list = query({
+  handler: async (ctx) => {
+    return await ctx.db.query("projects").collect();
+  },
+});
+```
+
+```ts
+// Good: use a digest table with only the fields the list needs
+export const listDigests = query({
+  handler: async (ctx) => {
+    return await ctx.db.query("projectDigests").collect();
+  },
+});
+```
+
+Writes to fields not in the digest table do not invalidate the digest query.
+
+### 7. Remove `Date.now()` from queries
+
+Using `Date.now()` inside a query defeats Convex's query cache. The cache is invalidated frequently to avoid showing stale time-dependent results, which increases database work even when the underlying data has not changed.
+
+```ts
+// Bad: Date.now() defeats query caching and causes frequent re-evaluation
+const releasedPosts = await ctx.db
+  .query("posts")
+  .withIndex("by_released_at", (q) => q.lte("releasedAt", Date.now()))
+  .take(100);
+```
+
+```ts
+// Good: use a boolean field updated by a scheduled function
+const releasedPosts = await ctx.db
+  .query("posts")
+  .withIndex("by_is_released", (q) => q.eq("isReleased", true))
+  .take(100);
+```
+
+If the query must compare against a time value, pass it as an explicit argument from the client and round it to a coarse interval (e.g. the most recent minute) so requests within that window share the same cache entry.
+
+### 8. Consider pagination strategy
+
+For long lists where users scroll through many pages:
+
+- If the data does not need live updates, use point-in-time fetching with manual "load more"
+- If it does need live updates, accept the subscription cost but limit the number of loaded pages
+- Consider whether older pages can be unloaded as the user scrolls forward
+
+### 9. Separate backend cost from UI churn
+
+If the main problem is loading flash or UI churn when query arguments change, stabilizing the reactive UI behavior may be better than replacing reactivity altogether.
+
+Treat this as a UX problem first when:
+
+- the underlying query is already reasonably cheap
+- the complaint is flicker, loading flashes, or re-render churn
+- live updates are still desirable once fresh data arrives
+
+## Verification
+
+1. Subscription count in dashboard is lower for the affected pages
+2. UI responsiveness has improved
+3. React profiling shows fewer unnecessary re-renders
+4. Surfaces that do not need live updates are not paying for persistent subscriptions unnecessarily
+5. Sibling pages with similar patterns were updated consistently

package/src/orchestrator/plugins/coolify/SKILL.md

@@ -0,0 +1,134 @@
+---
+name: coolify-deployment
+description: "Deploys applications, databases, and services on self-hosted Coolify instances, manages environments and env vars, runs infrastructure diagnostics, and performs batch operations. Use when deploying apps to Coolify, managing Coolify servers, debugging deployment issues, setting up databases, or managing Coolify infrastructure."
+---
+
+# Coolify Deployment
+
+## Topic Routing
+
+Read the matching reference before working on any of these topics:
+
+| Topic | Reference |
+|-------|-----------|
+| Application deployment | `references/applications.md` |
+| Database & service management | `references/databases-services.md` |
+| Infrastructure & diagnostics | `references/infrastructure.md` |
+| Docker Compose templates | `references/docker-compose.md` |
+| CI/CD & Docker image deploys | `references/ci-cd-webhooks.md` |
+
+## Critical Rules
+
+**Project Structure**
+- Coolify organizes resources as: Server → Project → Environment → Resources (apps, databases, services)
+- Always start with `get_infrastructure_overview` to understand the current state before making changes
+- Use `projects` tool with action params (`list`, `get`, `create`) — one tool, multiple operations
+
+**Application Deployment**
+- Deploy from public repos, private GitHub repos (via GitHub App), SSH keys, or Docker images
+- Configure health checks (path, interval, retries) during `application` create to avoid unhealthy deployments
+- Use `deploy` with `force_rebuild: true` only when cache issues are suspected — normal deploys are faster
+- Check deployment status with `deployment` action `list_for_app` after triggering a deploy
+
+**Diagnostics**
+- `diagnose_app` accepts name or domain (e.g., `"my-app"` or `"example.com"`) — not just UUIDs
+- `diagnose_server` accepts name or IP address — use for connectivity and resource issues
+- `find_issues` scans all infrastructure — use as a health check before and after changes
+
+**Environment Variables**
+- Use `env_vars` tool with `resource: "application"` or `resource: "service"` — supports `list`, `create`, `update`, `delete`
+- `bulk_env_update` updates a variable across multiple apps at once — use for shared config like API URLs
+- Env vars require app restart to take effect — use `control` with `action: "restart"` after updates
+
+**Database Management**
+- Supports 8 database types: PostgreSQL, MySQL, MariaDB, MongoDB, Redis, KeyDB, ClickHouse, Dragonfly
+- Configure backup schedules via `database_backups` with `action: "create"` — set frequency and retention
+- Backups can target S3-compatible storage — configure via backup schedule creation
+
+**Security**
+- Store `COOLIFY_ACCESS_TOKEN` in `.env` — never commit to source control
+- Private keys for SSH deployments: manage via `private_keys` tool, never hardcode
+- Use `validate_server` to verify SSH connectivity before deploying
+
+**Docker Compose**
+- Coolify supports two compose modes: Raw (paste YAML, no `build:`) and Repository (git URL, full features)
+- Use magic variables (`SERVICE_PASSWORD_*`, `SERVICE_URL_*`) for auto-generated credentials and proxy URLs — never hardcode
+- Remove `ports:` for proxied services — Traefik handles routing via `SERVICE_URL_NAME_PORT`
+
+**CI/CD Integration**
+- Trigger deploys via webhook with tag or UUID — use for automated pipelines
+- Docker image deployments (`create_dockerimage`) skip the build step — use for pre-built images from registries
+
+## Workflow: Deploy Application
+
+1. **Survey** — `get_infrastructure_overview` → identify target server and project
+2. **Create project** (if needed) — `projects` action `create` with name and description
+3. **Create environment** — `environments` action `create` in the target project
+4. **Deploy** — `application` action `create_public` (or `create_github` / `create_dockerimage`) with health check config
+5. **Verify** — `deployment` action `list_for_app` → wait for `finished` status; if `failed` → `application_logs` → fix → redeploy
+6. **Set env vars** — `env_vars` resource `application` action `create` → `control` action `restart`
+7. **Validate** — `diagnose_app` by name → confirm status `running` and health check passing
+
+## Workflow: Diagnose & Fix
+
+1. **Scan** — `find_issues` → list all unhealthy resources
+2. **Investigate** — `diagnose_app` or `diagnose_server` with name/domain/IP
+3. **Check logs** — `application_logs` for the affected app
+4. **Fix** — update config, env vars, or redeploy as needed
+5. **Restart** — `control` resource `application` action `restart`
+6. **Verify** — `diagnose_app` again → confirm status `running`
+
+## Deployment Failure Recovery
+
+```
+Deploy failed → check status
+├── Health check failing → verify health check path returns 200 → fix app → redeploy
+├── Build error → application_logs → fix Dockerfile/build config → deploy with force_rebuild
+├── Port conflict → check server_resources → update port mapping → redeploy
+├── SSH key invalid → validate_server → private_keys action update → retry
+└── Out of resources → server diagnostics → scale server or stop unused apps
+```
+
+## Deploy Public Repo — Full Example
+
+```
+# 1. Create app from public repo
+application(
+  action="create_public",
+  project_uuid="proj-abc",
+  environment_name="production",
+  server_uuid="srv-xyz",
+  repository_url="https://github.com/org/my-app",
+  branch="main",
+  build_pack="dockerfile",
+  health_check_path="/health",
+  health_check_interval=30,
+  health_check_retries=3
+)
+# → returns { uuid: "app-123", status: "deploying" }
+
+# 2. Set env vars
+env_vars(resource="application", action="create", uuid="app-123", key="DATABASE_URL", value="postgres://...", is_preview=false)
+
+# 3. Restart to pick up env vars
+control(resource="application", action="restart", uuid="app-123")
+
+# 4. Verify
+diagnose_app(identifier="my-app")
+# → status: "running", health_check: "healthy"
+```
+
+## MCP Config (VS Code)
+
+```json
+{
+  "servers": {
+    "Coolify": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@masonator/coolify-mcp"],
+      "envFile": "${workspaceFolder}/.env"
+    }
+  }
+}
+```

package/src/orchestrator/plugins/coolify/config.ts

@@ -0,0 +1,29 @@
+import type { PluginConfig } from '../types.js';
+
+export const config: PluginConfig = {
+  id: 'coolify',
+  name: 'Coolify',
+  category: 'tech',
+  subCategory: 'deployment',
+  label: 'Coolify',
+  hint: 'Self-hosted PaaS — deploy apps, databases, and services',
+  skillName: 'coolify-deployment',
+  mcpServerKey: 'Coolify',
+  mcpConfig: {
+    type: 'stdio',
+    command: 'npx',
+    args: ['-y', '@masonator/coolify-mcp'],
+  },
+  authType: 'env-token',
+  envVars: [
+    { name: 'COOLIFY_ACCESS_TOKEN', hint: 'Generate in Coolify Settings → API' },
+    { name: 'COOLIFY_BASE_URL', hint: 'Your Coolify instance URL (e.g. https://coolify.example.com)' },
+  ],
+  agentToolMap: {
+    'developer': ['list_applications', 'get_application', 'application_logs', 'deploy'],
+    'devops-expert': ['get_infrastructure_overview', 'diagnose_app', 'diagnose_server', 'find_issues', 'control', 'deploy', 'restart_project_apps'],
+  },
+  docsUrl: null,
+  officialDocs: 'https://coolify.io/docs',
+  mcpPackage: '@masonator/coolify-mcp',
+};

package/src/orchestrator/plugins/coolify/references/applications.md

@@ -0,0 +1,65 @@
+# Application Deployment
+
+## Deployment Sources
+
+| Source | Action | Key Args |
+|--------|--------|----------|
+| Public Git repo | `create_public` | `repository_url`, `branch` |
+| GitHub App | `create_github` | `github_app_uuid`, `repository_url`, `branch` |
+| SSH key | `create_key` | `private_key_uuid`, `repository_url`, `branch` |
+| Docker image | `create_dockerimage` | `docker_image`, `docker_registry` |
+
+## Create Application
+
+```
+application action="create_public"
+  project_uuid="<uuid>"
+  environment_name="production"
+  server_uuid="<uuid>"
+  repository_url="https://github.com/org/repo"
+  branch="main"
+  build_pack="dockerfile"
+  health_check_path="/health"
+  health_check_interval=30
+  health_check_retries=3
+```
+
+## Application Lifecycle
+
+```bash
+# List all apps (summaries)
+list_applications
+
+# Full details for one app
+get_application uuid="<uuid>"
+
+# View logs
+application_logs uuid="<uuid>" lines=100
+
+# Deploy (with optional force rebuild)
+deploy uuid="<uuid>" force_rebuild=true
+
+# Restart/stop/start
+control resource="application" action="restart" uuid="<uuid>"
+```
+
+## Environment Variables
+
+```bash
+# List env vars
+env_vars resource="application" action="list" uuid="<uuid>"
+
+# Create env var
+env_vars resource="application" action="create" uuid="<uuid>" key="DATABASE_URL" value="postgres://..." is_preview=false
+
+# Bulk update across apps
+bulk_env_update key="API_URL" value="https://api.example.com" uuids=["<uuid1>","<uuid2>"]
+```
+
+## Health Check Configuration
+
+Always configure health checks to prevent deploying broken apps:
+
+- `health_check_path` — endpoint that returns 200 (e.g., `/health`, `/api/health`)
+- `health_check_interval` — seconds between checks (default: 30)
+- `health_check_retries` — failures before marking unhealthy (default: 3)

package/src/orchestrator/plugins/coolify/references/ci-cd-webhooks.md

@@ -0,0 +1,73 @@
+# CI/CD & Docker Image Deployments
+
+## Webhook-Triggered Deployments
+
+```bash
+# Trigger deploy by image tag
+deploy_webhook tag="v1.2.3"
+
+# Trigger deploy by application UUID
+deploy_webhook uuid="<app-uuid>"
+```
+
+REST API equivalent (outside MCP):
+```bash
+# By tag
+curl -H "Authorization: Bearer $COOLIFY_TOKEN" \
+     "$COOLIFY_URL/api/v1/deployments/deploy?tag=v1.2.3"
+
+# By UUID
+curl -H "Authorization: Bearer $COOLIFY_TOKEN" \
+     "$COOLIFY_URL/api/v1/deployments/deploy?uuid=<app-uuid>"
+```
+
+## Docker Image Deployment (no git repo)
+
+```
+application action="create_dockerimage"
+  project_uuid="<uuid>"
+  environment_name="production"
+  server_uuid="<uuid>"
+  docker_image="nginx:latest"
+  docker_registry="docker.io"
+  health_check_path="/"
+  health_check_interval=30
+  health_check_retries=3
+```
+
+**Private registry** (credentials configured in Coolify dashboard):
+```
+application action="create_dockerimage"
+  project_uuid="<uuid>"
+  environment_name="production"
+  server_uuid="<uuid>"
+  docker_image="ghcr.io/org/app:latest"
+  docker_registry="ghcr.io"
+  health_check_path="/health"
+```
+
+## GitHub Actions Integration
+
+```yaml
+# .github/workflows/deploy.yml
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Trigger Coolify Deploy
+        run: |
+          curl -sf -H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}" \
+               "${{ secrets.COOLIFY_URL }}/api/v1/deployments/deploy?uuid=${{ secrets.APP_UUID }}"
+```
+
+Required GitHub secrets: `COOLIFY_TOKEN`, `COOLIFY_URL`, `APP_UUID`
+
+## Deployment Strategy Reference
+
+| Strategy | When to Use |
+|----------|-------------|
+| Git push + auto-deploy | Default for most apps |
+| Webhook by tag | Version-pinned releases, staging promotion |
+| Webhook by UUID | CI/CD pipeline integration (specific app) |
+| Docker image (`create_dockerimage`) | Pre-built images, registry-first workflow |
+| `force_rebuild: true` | Cache corruption or dependency issues |

package/src/orchestrator/plugins/coolify/references/databases-services.md

@@ -0,0 +1,57 @@
+# Database & Service Management
+
+## Supported Database Types
+
+PostgreSQL, MySQL, MariaDB, MongoDB, Redis, KeyDB, ClickHouse, Dragonfly.
+
+## Create Database
+
+```
+database action="create"
+  type="postgresql"
+  project_uuid="<uuid>"
+  environment_name="production"
+  server_uuid="<uuid>"
+```
+
+## Backup Management
+
+```bash
+# List backup schedules
+database_backups action="list_schedules" database_uuid="<uuid>"
+
+# Create schedule
+database_backups action="create" database_uuid="<uuid>"
+  frequency="0 2 * * *"  # Daily at 2 AM
+  retention=7
+
+# View executions
+database_backups action="list_executions" database_uuid="<uuid>"
+```
+
+## Services
+
+One-click service deployments (e.g., Plausible, Gitea, Uptime Kuma):
+
+```bash
+# List available services
+list_services
+
+# Create service
+service action="create"
+  project_uuid="<uuid>"
+  environment_name="production"
+  server_uuid="<uuid>"
+  type="plausible"
+
+# Manage env vars
+env_vars resource="service" action="list" uuid="<uuid>"
+```
+
+## Database Lifecycle
+
+```bash
+control resource="database" action="start" uuid="<uuid>"
+control resource="database" action="stop" uuid="<uuid>"
+control resource="database" action="restart" uuid="<uuid>"
+```