opencastle 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +215 -0
- package/bin/cli.mjs +69 -0
- package/dist/cli/adapters/claude-code.d.ts +22 -0
- package/dist/cli/adapters/claude-code.d.ts.map +1 -0
- package/dist/cli/adapters/claude-code.js +237 -0
- package/dist/cli/adapters/claude-code.js.map +1 -0
- package/dist/cli/adapters/cursor.d.ts +20 -0
- package/dist/cli/adapters/cursor.d.ts.map +1 -0
- package/dist/cli/adapters/cursor.js +231 -0
- package/dist/cli/adapters/cursor.js.map +1 -0
- package/dist/cli/adapters/vscode.d.ts +20 -0
- package/dist/cli/adapters/vscode.d.ts.map +1 -0
- package/dist/cli/adapters/vscode.js +132 -0
- package/dist/cli/adapters/vscode.js.map +1 -0
- package/dist/cli/copy.d.ts +14 -0
- package/dist/cli/copy.d.ts.map +1 -0
- package/dist/cli/copy.js +62 -0
- package/dist/cli/copy.js.map +1 -0
- package/dist/cli/dashboard.d.ts +3 -0
- package/dist/cli/dashboard.d.ts.map +1 -0
- package/dist/cli/dashboard.js +183 -0
- package/dist/cli/dashboard.js.map +1 -0
- package/dist/cli/diff.d.ts +3 -0
- package/dist/cli/diff.d.ts.map +1 -0
- package/dist/cli/diff.js +27 -0
- package/dist/cli/diff.js.map +1 -0
- package/dist/cli/eject.d.ts +3 -0
- package/dist/cli/eject.d.ts.map +1 -0
- package/dist/cli/eject.js +27 -0
- package/dist/cli/eject.js.map +1 -0
- package/dist/cli/init.d.ts +3 -0
- package/dist/cli/init.d.ts.map +1 -0
- package/dist/cli/init.js +92 -0
- package/dist/cli/init.js.map +1 -0
- package/dist/cli/manifest.d.ts +14 -0
- package/dist/cli/manifest.d.ts.map +1 -0
- package/dist/cli/manifest.js +34 -0
- package/dist/cli/manifest.js.map +1 -0
- package/dist/cli/mcp.d.ts +14 -0
- package/dist/cli/mcp.d.ts.map +1 -0
- package/dist/cli/mcp.js +35 -0
- package/dist/cli/mcp.js.map +1 -0
- package/dist/cli/prompt.d.ts +12 -0
- package/dist/cli/prompt.d.ts.map +1 -0
- package/dist/cli/prompt.js +104 -0
- package/dist/cli/prompt.js.map +1 -0
- package/dist/cli/run/adapters/claude-code.d.ts +16 -0
- package/dist/cli/run/adapters/claude-code.d.ts.map +1 -0
- package/dist/cli/run/adapters/claude-code.js +82 -0
- package/dist/cli/run/adapters/claude-code.js.map +1 -0
- package/dist/cli/run/adapters/copilot.d.ts +16 -0
- package/dist/cli/run/adapters/copilot.d.ts.map +1 -0
- package/dist/cli/run/adapters/copilot.js +84 -0
- package/dist/cli/run/adapters/copilot.js.map +1 -0
- package/dist/cli/run/adapters/cursor.d.ts +16 -0
- package/dist/cli/run/adapters/cursor.d.ts.map +1 -0
- package/dist/cli/run/adapters/cursor.js +81 -0
- package/dist/cli/run/adapters/cursor.js.map +1 -0
- package/dist/cli/run/adapters/index.d.ts +14 -0
- package/dist/cli/run/adapters/index.d.ts.map +1 -0
- package/dist/cli/run/adapters/index.js +35 -0
- package/dist/cli/run/adapters/index.js.map +1 -0
- package/dist/cli/run/executor.d.ts +15 -0
- package/dist/cli/run/executor.d.ts.map +1 -0
- package/dist/cli/run/executor.js +249 -0
- package/dist/cli/run/executor.js.map +1 -0
- package/dist/cli/run/reporter.d.ts +10 -0
- package/dist/cli/run/reporter.d.ts.map +1 -0
- package/dist/cli/run/reporter.js +112 -0
- package/dist/cli/run/reporter.js.map +1 -0
- package/dist/cli/run/schema.d.ts +28 -0
- package/dist/cli/run/schema.d.ts.map +1 -0
- package/dist/cli/run/schema.js +511 -0
- package/dist/cli/run/schema.js.map +1 -0
- package/dist/cli/run.d.ts +6 -0
- package/dist/cli/run.d.ts.map +1 -0
- package/dist/cli/run.js +123 -0
- package/dist/cli/run.js.map +1 -0
- package/dist/cli/stack-config.d.ts +12 -0
- package/dist/cli/stack-config.d.ts.map +1 -0
- package/dist/cli/stack-config.js +146 -0
- package/dist/cli/stack-config.js.map +1 -0
- package/dist/cli/types.d.ts +169 -0
- package/dist/cli/types.d.ts.map +1 -0
- package/dist/cli/types.js +2 -0
- package/dist/cli/types.js.map +1 -0
- package/dist/cli/update.d.ts +3 -0
- package/dist/cli/update.d.ts.map +1 -0
- package/dist/cli/update.js +50 -0
- package/dist/cli/update.js.map +1 -0
- package/package.json +48 -0
- package/src/cli/adapters/claude-code.ts +287 -0
- package/src/cli/adapters/cursor.ts +377 -0
- package/src/cli/adapters/vscode.ts +168 -0
- package/src/cli/copy.ts +79 -0
- package/src/cli/dashboard.ts +225 -0
- package/src/cli/diff.ts +44 -0
- package/src/cli/eject.ts +39 -0
- package/src/cli/init.ts +120 -0
- package/src/cli/manifest.ts +45 -0
- package/src/cli/mcp.ts +49 -0
- package/src/cli/prompt.ts +115 -0
- package/src/cli/run/adapters/claude-code.ts +95 -0
- package/src/cli/run/adapters/copilot.ts +97 -0
- package/src/cli/run/adapters/cursor.ts +94 -0
- package/src/cli/run/adapters/index.ts +40 -0
- package/src/cli/run/executor.ts +292 -0
- package/src/cli/run/reporter.ts +129 -0
- package/src/cli/run/schema.ts +595 -0
- package/src/cli/run.ts +137 -0
- package/src/cli/stack-config.ts +180 -0
- package/src/cli/types.ts +207 -0
- package/src/cli/update.ts +75 -0
- package/src/dashboard/astro.config.mjs +6 -0
- package/src/dashboard/package-lock.json +5455 -0
- package/src/dashboard/package.json +14 -0
- package/src/dashboard/public/data/delegations.ndjson +35 -0
- package/src/dashboard/public/data/panels.ndjson +13 -0
- package/src/dashboard/public/data/sessions.ndjson +50 -0
- package/src/dashboard/public/icon-192.png +0 -0
- package/src/dashboard/scripts/generate-seed-data.ts +355 -0
- package/src/dashboard/src/layouts/Layout.astro +25 -0
- package/src/dashboard/src/pages/index.astro +1070 -0
- package/src/dashboard/src/styles/dashboard.css +1078 -0
- package/src/dashboard/tsconfig.json +6 -0
- package/src/orchestrator/agent-workflows/README.md +22 -0
- package/src/orchestrator/agent-workflows/bug-fix.md +128 -0
- package/src/orchestrator/agent-workflows/data-pipeline.md +145 -0
- package/src/orchestrator/agent-workflows/database-migration.md +159 -0
- package/src/orchestrator/agent-workflows/feature-implementation.md +223 -0
- package/src/orchestrator/agent-workflows/performance-optimization.md +125 -0
- package/src/orchestrator/agent-workflows/refactoring.md +142 -0
- package/src/orchestrator/agent-workflows/schema-changes.md +164 -0
- package/src/orchestrator/agent-workflows/security-audit.md +148 -0
- package/src/orchestrator/agent-workflows/shared-delivery-phase.md +33 -0
- package/src/orchestrator/agents/api-designer.agent.md +68 -0
- package/src/orchestrator/agents/architect.agent.md +129 -0
- package/src/orchestrator/agents/content-engineer.agent.md +57 -0
- package/src/orchestrator/agents/copywriter.agent.md +95 -0
- package/src/orchestrator/agents/data-expert.agent.md +63 -0
- package/src/orchestrator/agents/database-engineer.agent.md +62 -0
- package/src/orchestrator/agents/developer.agent.md +66 -0
- package/src/orchestrator/agents/devops-expert.agent.md +57 -0
- package/src/orchestrator/agents/documentation-writer.agent.md +60 -0
- package/src/orchestrator/agents/performance-expert.agent.md +58 -0
- package/src/orchestrator/agents/release-manager.agent.md +72 -0
- package/src/orchestrator/agents/researcher.agent.md +145 -0
- package/src/orchestrator/agents/reviewer.agent.md +62 -0
- package/src/orchestrator/agents/security-expert.agent.md +64 -0
- package/src/orchestrator/agents/seo-specialist.agent.md +67 -0
- package/src/orchestrator/agents/team-lead.agent.md +644 -0
- package/src/orchestrator/agents/testing-expert.agent.md +85 -0
- package/src/orchestrator/agents/ui-ux-expert.agent.md +63 -0
- package/src/orchestrator/copilot-instructions.md +3 -0
- package/src/orchestrator/customizations/AGENT-EXPERTISE.md +325 -0
- package/src/orchestrator/customizations/AGENT-FAILURES.md +69 -0
- package/src/orchestrator/customizations/AGENT-PERFORMANCE.md +58 -0
- package/src/orchestrator/customizations/DISPUTES.md +162 -0
- package/src/orchestrator/customizations/KNOWLEDGE-GRAPH.md +10 -0
- package/src/orchestrator/customizations/LESSONS-LEARNED.md +70 -0
- package/src/orchestrator/customizations/README.md +59 -0
- package/src/orchestrator/customizations/agents/agent-registry.md +46 -0
- package/src/orchestrator/customizations/agents/skill-matrix.md +142 -0
- package/src/orchestrator/customizations/logs/README.md +181 -0
- package/src/orchestrator/customizations/logs/delegations.ndjson +1 -0
- package/src/orchestrator/customizations/logs/panels.ndjson +1 -0
- package/src/orchestrator/customizations/logs/sessions.ndjson +1 -0
- package/src/orchestrator/customizations/project/docs-structure.md +23 -0
- package/src/orchestrator/customizations/project/tracker-config.md +45 -0
- package/src/orchestrator/customizations/project.instructions.md +64 -0
- package/src/orchestrator/customizations/stack/api-config.md +37 -0
- package/src/orchestrator/customizations/stack/cms-config.md +26 -0
- package/src/orchestrator/customizations/stack/data-pipeline-config.md +41 -0
- package/src/orchestrator/customizations/stack/database-config.md +44 -0
- package/src/orchestrator/customizations/stack/deployment-config.md +45 -0
- package/src/orchestrator/customizations/stack/testing-config.md +56 -0
- package/src/orchestrator/instructions/ai-optimization.instructions.md +143 -0
- package/src/orchestrator/instructions/general.instructions.md +194 -0
- package/src/orchestrator/mcp.json +55 -0
- package/src/orchestrator/prompts/bootstrap-customizations.prompt.md +235 -0
- package/src/orchestrator/prompts/brainstorm.prompt.md +115 -0
- package/src/orchestrator/prompts/bug-fix.prompt.md +141 -0
- package/src/orchestrator/prompts/create-skill.prompt.md +103 -0
- package/src/orchestrator/prompts/generate-task-spec.prompt.md +154 -0
- package/src/orchestrator/prompts/implement-feature.prompt.md +124 -0
- package/src/orchestrator/prompts/metrics-report.prompt.md +142 -0
- package/src/orchestrator/prompts/quick-refinement.prompt.md +137 -0
- package/src/orchestrator/prompts/resolve-pr-comments.prompt.md +100 -0
- package/src/orchestrator/skills/accessibility-standards/SKILL.md +164 -0
- package/src/orchestrator/skills/agent-hooks/SKILL.md +147 -0
- package/src/orchestrator/skills/agent-memory/SKILL.md +144 -0
- package/src/orchestrator/skills/api-patterns/SKILL.md +106 -0
- package/src/orchestrator/skills/browser-testing/SKILL.md +203 -0
- package/src/orchestrator/skills/code-commenting/SKILL.md +133 -0
- package/src/orchestrator/skills/contentful-cms/SKILL.md +43 -0
- package/src/orchestrator/skills/context-map/SKILL.md +135 -0
- package/src/orchestrator/skills/convex-database/SKILL.md +80 -0
- package/src/orchestrator/skills/data-engineering/SKILL.md +99 -0
- package/src/orchestrator/skills/deployment-infrastructure/SKILL.md +49 -0
- package/src/orchestrator/skills/documentation-standards/SKILL.md +85 -0
- package/src/orchestrator/skills/fast-review/SKILL.md +327 -0
- package/src/orchestrator/skills/frontend-design/SKILL.md +42 -0
- package/src/orchestrator/skills/jira-management/SKILL.md +168 -0
- package/src/orchestrator/skills/memory-merger/SKILL.md +123 -0
- package/src/orchestrator/skills/nextjs-patterns/SKILL.md +75 -0
- package/src/orchestrator/skills/nx-workspace/SKILL.md +192 -0
- package/src/orchestrator/skills/panel-majority-vote/SKILL.md +184 -0
- package/src/orchestrator/skills/panel-majority-vote/panel-report.template.md +38 -0
- package/src/orchestrator/skills/performance-optimization/SKILL.md +101 -0
- package/src/orchestrator/skills/react-development/SKILL.md +117 -0
- package/src/orchestrator/skills/sanity-cms/SKILL.md +18 -0
- package/src/orchestrator/skills/security-hardening/SKILL.md +118 -0
- package/src/orchestrator/skills/self-improvement/SKILL.md +137 -0
- package/src/orchestrator/skills/seo-patterns/SKILL.md +40 -0
- package/src/orchestrator/skills/session-checkpoints/SKILL.md +205 -0
- package/src/orchestrator/skills/slack-notifications/SKILL.md +211 -0
- package/src/orchestrator/skills/strapi-cms/SKILL.md +43 -0
- package/src/orchestrator/skills/supabase-database/SKILL.md +24 -0
- package/src/orchestrator/skills/task-management/SKILL.md +143 -0
- package/src/orchestrator/skills/team-lead-reference/SKILL.md +317 -0
- package/src/orchestrator/skills/teams-notifications/SKILL.md +249 -0
- package/src/orchestrator/skills/testing-workflow/SKILL.md +134 -0
- package/src/orchestrator/skills/validation-gates/SKILL.md +100 -0
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: context-map
|
|
3
|
+
description: "Generate a structured file impact map before making changes. Identifies all files that will be affected, their relationships, and cascade effects — improving file partitioning for parallel work and reducing unexpected side effects."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Skill: Context Map
|
|
7
|
+
|
|
8
|
+
Generate a structured **file impact map** before any code changes begin. This map identifies all files that will be touched, their relationships, and cascade effects — directly improving the Team Lead's file partitioning for parallel agents.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- Before **every feature implementation** (Phase 1: Research)
|
|
13
|
+
- Before **refactoring** (Phase 1: Scope & Baseline)
|
|
14
|
+
- Before **schema changes** that cascade through queries and components
|
|
15
|
+
- Before **any task touching shared libraries** (`libs/`)
|
|
16
|
+
- Optional for isolated bug fixes affecting 1-2 files
|
|
17
|
+
|
|
18
|
+
## How to Generate a Context Map
|
|
19
|
+
|
|
20
|
+
### Step 1: Identify the Entry Points
|
|
21
|
+
|
|
22
|
+
Start from the task description and identify the primary files that MUST change:
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
Entry Points:
|
|
26
|
+
- [file path] — [why it must change]
|
|
27
|
+
- [file path] — [why it must change]
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
### Step 2: Trace Dependencies (Outward)
|
|
31
|
+
|
|
32
|
+
For each entry point, trace what depends on it:
|
|
33
|
+
|
|
34
|
+
1. **Imports** — what files import this module? (`grep_search` or `list_code_usages`)
|
|
35
|
+
2. **Type consumers** — what files use types/interfaces defined here?
|
|
36
|
+
3. **Route references** — what pages render this component?
|
|
37
|
+
4. **Query consumers** — what components or pages call this query?
|
|
38
|
+
5. **Test files** — what test files cover this code?
|
|
39
|
+
|
|
40
|
+
### Step 3: Trace Sources (Inward)
|
|
41
|
+
|
|
42
|
+
For each entry point, trace what it depends on:
|
|
43
|
+
|
|
44
|
+
1. **Data sources** — which Sanity schemas, GROQ queries, or Supabase tables feed this code?
|
|
45
|
+
2. **Shared utilities** — which `libs/` modules does it use?
|
|
46
|
+
3. **Configuration** — which config files affect its behavior?
|
|
47
|
+
|
|
48
|
+
### Step 4: Build the Map
|
|
49
|
+
|
|
50
|
+
Produce a structured map in this format:
|
|
51
|
+
|
|
52
|
+
```markdown
|
|
53
|
+
## Context Map: [Task Name]
|
|
54
|
+
|
|
55
|
+
### Entry Points (MUST change)
|
|
56
|
+
| File | Reason | Owner |
|
|
57
|
+
|------|--------|-------|
|
|
58
|
+
| `libs/queries/src/lib/places.ts` | Add new query field | Content Engineer |
|
|
59
|
+
| `libs/ui-kit/src/lib/components/PlaceCard/` | Display new field | UI/UX Expert |
|
|
60
|
+
|
|
61
|
+
### Cascade Effects (WILL change)
|
|
62
|
+
| File | Triggered By | Reason | Owner |
|
|
63
|
+
|------|-------------|--------|-------|
|
|
64
|
+
| `apps/tastebeer.eu/app/places/page.tsx` | PlaceCard change | Update props | Frontend Dev |
|
|
65
|
+
| `apps/tastecoffee.eu/app/places/page.tsx` | PlaceCard change | Update props | Frontend Dev |
|
|
66
|
+
| `libs/queries/src/lib/__tests__/places.test.ts` | Query change | Update test | Testing Expert |
|
|
67
|
+
|
|
68
|
+
### Shared Boundaries (WATCH for conflicts)
|
|
69
|
+
| File | Risk | Mitigation |
|
|
70
|
+
|------|------|------------|
|
|
71
|
+
| `libs/ui-kit/src/lib/index.ts` | Barrel export — may conflict | Merge sequentially |
|
|
72
|
+
|
|
73
|
+
### Unaffected (explicitly safe)
|
|
74
|
+
| Area | Why |
|
|
75
|
+
|------|-----|
|
|
76
|
+
| `supabase/migrations/` | No DB changes |
|
|
77
|
+
| `libs/supabase-auth/` | No auth changes |
|
|
78
|
+
| `apps/cms-studio/` | No schema changes |
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Step 5: Derive File Partitions
|
|
82
|
+
|
|
83
|
+
From the context map, assign file ownership to agents:
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
Agent A (Content Engineer): libs/queries/src/lib/places.ts
|
|
87
|
+
Agent B (UI/UX Expert): libs/ui-kit/src/lib/components/PlaceCard/
|
|
88
|
+
Agent C (Frontend Dev): apps/tastebeer.eu/app/places/, apps/tastecoffee.eu/app/places/
|
|
89
|
+
Agent D (Testing Expert): **/*test*, **/*spec*
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
**Rules:**
|
|
93
|
+
- No file appears in two partitions
|
|
94
|
+
- Shared boundaries are assigned to ONE agent and merged first
|
|
95
|
+
- Test files belong to the Testing Expert unless tightly coupled to a specific change
|
|
96
|
+
|
|
97
|
+
## Context Map Depth Levels
|
|
98
|
+
|
|
99
|
+
Scale the depth to the task complexity:
|
|
100
|
+
|
|
101
|
+
| Task Complexity | Depth | What to Trace |
|
|
102
|
+
|----------------|-------|---------------|
|
|
103
|
+
| **Small** (1-3 files) | Entry points only | Direct imports/exports |
|
|
104
|
+
| **Medium** (4-8 files) | Entry + cascade | 1 hop of dependencies |
|
|
105
|
+
| **Large** (9+ files) | Full map | Complete dependency graph |
|
|
106
|
+
|
|
107
|
+
## Integration with Team Lead Workflow
|
|
108
|
+
|
|
109
|
+
The context map is produced in **Phase 1 (Research)** and consumed by:
|
|
110
|
+
|
|
111
|
+
1. **Decomposition (Step 2)** — the map directly informs file partitions
|
|
112
|
+
2. **Delegation prompts** — include the relevant section of the map so agents know their boundaries
|
|
113
|
+
3. **Verification (QA Gate)** — compare actual changed files against the map to detect scope creep
|
|
114
|
+
|
|
115
|
+
### Including in Delegation Prompts
|
|
116
|
+
|
|
117
|
+
```markdown
|
|
118
|
+
## Your File Partition (from Context Map)
|
|
119
|
+
|
|
120
|
+
You own these files — modify only these:
|
|
121
|
+
- `libs/queries/src/lib/places.ts`
|
|
122
|
+
- `libs/queries/src/lib/__tests__/places.test.ts`
|
|
123
|
+
|
|
124
|
+
Do NOT modify:
|
|
125
|
+
- `libs/ui-kit/` (owned by UI/UX Expert)
|
|
126
|
+
- `apps/` (owned by Developer)
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
## Anti-Patterns
|
|
130
|
+
|
|
131
|
+
- **Skipping the map for "obvious" tasks** — even small tasks can have unexpected cascades in shared libraries
|
|
132
|
+
- **Mapping without searching** — don't guess dependencies; use `grep_search`, `list_code_usages`, and import tracing
|
|
133
|
+
- **Over-mapping** — for a 2-file bug fix, don't trace the entire dependency graph. Match depth to complexity
|
|
134
|
+
- **Stale maps** — if the plan changes during execution, update the map. A stale map is worse than no map
|
|
135
|
+
- **Mapping files you won't change** — the "Unaffected" section is for explicitly noting what's safe, not for cataloging the entire codebase
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: convex-database
|
|
3
|
+
description: "Convex reactive database patterns, schema design, real-time queries, mutations, actions, and deployment best practices. Use when designing Convex schemas, writing queries/mutations, or managing the Convex backend."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Convex Database
|
|
7
|
+
|
|
8
|
+
Generic Convex development methodology. For project-specific schema, functions, and deployment details, see [database-config.md](../../customizations/stack/database-config.md).
|
|
9
|
+
|
|
10
|
+
## Critical Development Rules
|
|
11
|
+
|
|
12
|
+
1. **Schema-first design** — define schema in `convex/schema.ts` using `defineSchema` and `defineTable`
|
|
13
|
+
2. **Queries are reactive** — Convex queries automatically re-run when underlying data changes
|
|
14
|
+
3. **Mutations are transactional** — mutations run as ACID transactions; leverage this for consistency
|
|
15
|
+
4. **Actions for side effects** — use actions (not mutations) for external API calls, file uploads, etc.
|
|
16
|
+
5. **Never await queries in mutations** — queries and mutations run in separate contexts
|
|
17
|
+
6. **Use validators** — validate all function arguments with `v` (Convex's validator library)
|
|
18
|
+
7. **Index design** — create indexes for frequently filtered/sorted fields in the schema
|
|
19
|
+
8. **Paginated queries** — use `.paginate()` for large result sets
|
|
20
|
+
9. **File storage** — use Convex's built-in file storage API, not external services
|
|
21
|
+
10. **Environment variables** — set via Convex dashboard, access with `process.env` in actions only
|
|
22
|
+
|
|
23
|
+
## Schema Patterns
|
|
24
|
+
|
|
25
|
+
### Defining Tables
|
|
26
|
+
```typescript
|
|
27
|
+
import { defineSchema, defineTable } from "convex/server";
|
|
28
|
+
import { v } from "convex/values";
|
|
29
|
+
|
|
30
|
+
export default defineSchema({
|
|
31
|
+
users: defineTable({
|
|
32
|
+
name: v.string(),
|
|
33
|
+
email: v.string(),
|
|
34
|
+
role: v.union(v.literal("admin"), v.literal("user")),
|
|
35
|
+
}).index("by_email", ["email"]),
|
|
36
|
+
});
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### Query Functions
|
|
40
|
+
```typescript
|
|
41
|
+
import { query } from "./_generated/server";
|
|
42
|
+
import { v } from "convex/values";
|
|
43
|
+
|
|
44
|
+
export const list = query({
|
|
45
|
+
args: { role: v.optional(v.string()) },
|
|
46
|
+
handler: async (ctx, args) => {
|
|
47
|
+
if (args.role) {
|
|
48
|
+
return await ctx.db.query("users").filter(q => q.eq(q.field("role"), args.role)).collect();
|
|
49
|
+
}
|
|
50
|
+
return await ctx.db.query("users").collect();
|
|
51
|
+
},
|
|
52
|
+
});
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
### Mutations
|
|
56
|
+
```typescript
|
|
57
|
+
import { mutation } from "./_generated/server";
|
|
58
|
+
import { v } from "convex/values";
|
|
59
|
+
|
|
60
|
+
export const create = mutation({
|
|
61
|
+
args: { name: v.string(), email: v.string() },
|
|
62
|
+
handler: async (ctx, args) => {
|
|
63
|
+
return await ctx.db.insert("users", { ...args, role: "user" });
|
|
64
|
+
},
|
|
65
|
+
});
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Real-Time Patterns
|
|
69
|
+
|
|
70
|
+
- Use `useQuery` hook for automatic real-time subscriptions
|
|
71
|
+
- Queries re-run when any referenced table data changes
|
|
72
|
+
- Use `useMutation` for optimistic updates
|
|
73
|
+
- Paginated queries support real-time updates with `.paginate()`
|
|
74
|
+
|
|
75
|
+
## Deployment
|
|
76
|
+
|
|
77
|
+
- Deploy with `npx convex deploy`
|
|
78
|
+
- Use `npx convex dev` for local development with hot reload
|
|
79
|
+
- Schema changes are automatically migrated
|
|
80
|
+
- Use `npx convex import` / `npx convex export` for data management
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: data-engineering
|
|
3
|
+
description: "Data pipeline ETL workflows, web scraping with Puppeteer, NDJSON processing, and CMS data import. Use when building scrapers, processing data, running CLI tools, or importing to a CMS."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Data Engineering
|
|
7
|
+
|
|
8
|
+
Generic data pipeline patterns and scraping methodology. For project-specific pipeline architecture, sources, CLI commands, and data status, see [data-pipeline-config.md](../../customizations/stack/data-pipeline-config.md).
|
|
9
|
+
|
|
10
|
+
## Scraper Architecture
|
|
11
|
+
|
|
12
|
+
### Base Scraper Pattern
|
|
13
|
+
|
|
14
|
+
```typescript
|
|
15
|
+
interface ScraperConfig {
|
|
16
|
+
source: string;
|
|
17
|
+
query: string;
|
|
18
|
+
maxPages: number;
|
|
19
|
+
concurrency: number;
|
|
20
|
+
delay: { min: number; max: number };
|
|
21
|
+
outputPath: string;
|
|
22
|
+
headless: boolean;
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
abstract class BaseScraper {
|
|
26
|
+
abstract scrape(config: ScraperConfig): Promise<void>;
|
|
27
|
+
abstract extractVenue(page: Page): Promise<RawVenue>;
|
|
28
|
+
abstract getNextPage(page: Page): Promise<string | null>;
|
|
29
|
+
}
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
### Puppeteer Cluster Setup
|
|
33
|
+
|
|
34
|
+
```typescript
|
|
35
|
+
const cluster = await Cluster.launch({
|
|
36
|
+
concurrency: Cluster.CONCURRENCY_CONTEXT,
|
|
37
|
+
maxConcurrency: config.concurrency,
|
|
38
|
+
puppeteerOptions: {
|
|
39
|
+
headless: config.headless,
|
|
40
|
+
args: ['--no-sandbox', '--disable-setuid-sandbox'],
|
|
41
|
+
},
|
|
42
|
+
retryLimit: 3,
|
|
43
|
+
retryDelay: 5000,
|
|
44
|
+
timeout: 30000,
|
|
45
|
+
});
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### Anti-Detection Measures
|
|
49
|
+
|
|
50
|
+
- Rotate user agents from a curated list
|
|
51
|
+
- Random delays between requests (2-5 seconds default)
|
|
52
|
+
- Randomize viewport sizes
|
|
53
|
+
- Block unnecessary resources (images, fonts, CSS) for speed
|
|
54
|
+
- Use stealth plugin for Puppeteer
|
|
55
|
+
- Request interception for resource optimization
|
|
56
|
+
|
|
57
|
+
### Error Recovery
|
|
58
|
+
|
|
59
|
+
- Retry failed pages with exponential backoff (3 retries default)
|
|
60
|
+
- Log failed URLs for manual review
|
|
61
|
+
- Save partial results on crash/interruption
|
|
62
|
+
- Checkpoint/resume for long-running scrapes
|
|
63
|
+
|
|
64
|
+
## NDJSON Output Format
|
|
65
|
+
|
|
66
|
+
Each scraper produces one record per line:
|
|
67
|
+
|
|
68
|
+
```json
|
|
69
|
+
{"name":"Example Venue","lat":50.0755,"lng":14.4378,"source":"google-maps","sourceId":"ChIJ...","category":"bar","address":"Street 30, City","rating":4.5,"reviewCount":120}
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
### Required Fields
|
|
73
|
+
|
|
74
|
+
| Field | Priority | Notes |
|
|
75
|
+
|-------|----------|-------|
|
|
76
|
+
| `name` | Required | Preserve original encoding |
|
|
77
|
+
| `lat`/`lng` | Required | GPS coordinates |
|
|
78
|
+
| `address` | Required | Full text address |
|
|
79
|
+
| `source` | Required | Source identifier (e.g., `google-maps`) |
|
|
80
|
+
| `sourceId` | Required | Source-specific unique ID |
|
|
81
|
+
| `category` | Required | Domain-specific category |
|
|
82
|
+
|
|
83
|
+
### Optional Fields
|
|
84
|
+
|
|
85
|
+
`rating`, `reviewCount`, `phone`, `website`, `openingHours`, `photos`, `priceLevel`
|
|
86
|
+
|
|
87
|
+
## Design Principles
|
|
88
|
+
|
|
89
|
+
- Pipelines as composable, single-responsibility stages
|
|
90
|
+
- Use streams for large file processing to minimize memory
|
|
91
|
+
- Idempotent imports with `createOrReplace` and deterministic `_id` generation
|
|
92
|
+
- Dry-run mode for all destructive operations
|
|
93
|
+
- Generate normalized names by stripping diacritics for search
|
|
94
|
+
- Structured addresses: `{ street, city, postalCode, country, countryCode }`
|
|
95
|
+
- Track data lineage — record source and transformation history
|
|
96
|
+
- Handle errors gracefully — skip bad records, don't halt pipeline
|
|
97
|
+
- Backup before bulk operations
|
|
98
|
+
- Respect `robots.txt` and rate limit all scraping requests
|
|
99
|
+
- Only scrape publicly available data with source attribution
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: deployment-infrastructure
|
|
3
|
+
description: "Deployment architecture, environment variables, cron jobs, security headers, and caching patterns. Use when configuring deployments, managing environment variables, setting up cron jobs, or troubleshooting build/deployment issues."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Deployment Infrastructure
|
|
7
|
+
|
|
8
|
+
All deployment configuration is project-specific. See [deployment-config.md](../../customizations/stack/deployment-config.md) for the full architecture, environment variables, cron jobs, caching headers, and key files.
|
|
9
|
+
|
|
10
|
+
## Generic Deployment Principles
|
|
11
|
+
|
|
12
|
+
- Use platform-native Git integration for CI/CD (push to main = production, push to branch = preview)
|
|
13
|
+
- Store all secrets as environment variables — never in code, commits, or logs
|
|
14
|
+
- Use `Bearer` token auth for cron job endpoints
|
|
15
|
+
- Apply security headers via framework config (HSTS, CSP, X-Frame-Options, Permissions-Policy)
|
|
16
|
+
- Set immutable cache headers for static assets (`max-age=31536000, immutable`)
|
|
17
|
+
- Use short cache durations for frequently changing assets (e.g., favicon: `max-age=86400`)
|
|
18
|
+
- Load the **security-hardening** skill for full header inventory and CSP configuration
|
|
19
|
+
|
|
20
|
+
## Release Process
|
|
21
|
+
|
|
22
|
+
### 1. Pre-Release Audit
|
|
23
|
+
- Run `yarn nx affected -t lint,test,build` to verify all affected projects
|
|
24
|
+
- Review all changed files since last release (`git diff` against last tag/release)
|
|
25
|
+
- Check for uncommitted work or unmerged branches
|
|
26
|
+
- Verify no draft PRs are accidentally included
|
|
27
|
+
|
|
28
|
+
### 2. Regression Check
|
|
29
|
+
- Identify features adjacent to changes and spot-check them
|
|
30
|
+
- Run full test suites for all affected projects (not just changed files)
|
|
31
|
+
- Check deployment preview builds for visual regressions
|
|
32
|
+
- Verify critical user flows still work (homepage, search, venue detail)
|
|
33
|
+
|
|
34
|
+
### 3. Changelog & Release Notes
|
|
35
|
+
- Generate changelog from commit messages and PR titles since last release
|
|
36
|
+
- Categorize changes: Features, Bug Fixes, Performance, Breaking Changes, Internal
|
|
37
|
+
- Write human-readable release notes summarizing impact
|
|
38
|
+
- Include migration notes for any breaking changes
|
|
39
|
+
|
|
40
|
+
### 4. Version Management
|
|
41
|
+
- Follow semver: MAJOR (breaking), MINOR (features), PATCH (fixes)
|
|
42
|
+
- Tag releases in git with the version number
|
|
43
|
+
- Update version references in relevant files
|
|
44
|
+
|
|
45
|
+
### 5. Release Verification
|
|
46
|
+
- Confirm deployment succeeded on production
|
|
47
|
+
- Smoke-test production URLs for critical pages
|
|
48
|
+
- Monitor error rates and performance metrics post-release
|
|
49
|
+
- Document rollback steps if issues arise
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: documentation-standards
|
|
3
|
+
description: "Documentation templates, structure, and standards for project docs, roadmaps, ADRs, and known issues. Use when writing or updating documentation files."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Documentation Standards
|
|
7
|
+
|
|
8
|
+
Generic documentation templates and writing standards. For project-specific directory structure and practices, see [docs-structure.md](../../customizations/project/docs-structure.md).
|
|
9
|
+
|
|
10
|
+
## Issue Documentation Template
|
|
11
|
+
|
|
12
|
+
```markdown
|
|
13
|
+
### ISSUE-ID: Brief Description
|
|
14
|
+
|
|
15
|
+
**Issue ID:** ISSUE-ID
|
|
16
|
+
**Status:** Known Limitation | Fixed | Workaround Available
|
|
17
|
+
**Severity:** Critical | High | Medium | Low
|
|
18
|
+
**Impact:** [What user/developer experience is affected]
|
|
19
|
+
|
|
20
|
+
#### Problem
|
|
21
|
+
[Clear description of the issue]
|
|
22
|
+
|
|
23
|
+
#### Root Cause
|
|
24
|
+
[Technical explanation]
|
|
25
|
+
|
|
26
|
+
#### Solution Options
|
|
27
|
+
1. **Option A** — [Description]
|
|
28
|
+
- Pros: ...
|
|
29
|
+
- Cons: ...
|
|
30
|
+
2. **Option B** — [Description]
|
|
31
|
+
|
|
32
|
+
#### Related Files
|
|
33
|
+
- `path/to/file.ts` — [What it does]
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Roadmap Update Template
|
|
37
|
+
|
|
38
|
+
When a feature is completed:
|
|
39
|
+
1. Change status to `COMPLETE`
|
|
40
|
+
2. Add completion date
|
|
41
|
+
3. List modified files
|
|
42
|
+
4. Update the summary table at the top
|
|
43
|
+
5. Move to completed section if applicable
|
|
44
|
+
|
|
45
|
+
## Architecture Decision Record Template
|
|
46
|
+
|
|
47
|
+
```markdown
|
|
48
|
+
## ADR-NNN: Decision Title
|
|
49
|
+
|
|
50
|
+
**Date:** YYYY-MM-DD
|
|
51
|
+
**Status:** Accepted | Superseded | Deprecated
|
|
52
|
+
**Context:** [Why this decision was needed]
|
|
53
|
+
**Decision:** [What was decided]
|
|
54
|
+
**Consequences:** [Impact of the decision]
|
|
55
|
+
**Alternatives Considered:** [What else was evaluated]
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## Writing Guidelines
|
|
59
|
+
|
|
60
|
+
- Write clear, concise prose — avoid jargon unless necessary
|
|
61
|
+
- Include diagrams (Mermaid or ASCII) for architecture
|
|
62
|
+
- Link to related files and docs using relative paths
|
|
63
|
+
- Keep line length under 400 characters; break at 80 for readability
|
|
64
|
+
- Use tables for structured data
|
|
65
|
+
- Include "Last Updated" dates on all documents
|
|
66
|
+
- Archive outdated docs rather than deleting
|
|
67
|
+
- Cross-reference between documents when relevant
|
|
68
|
+
|
|
69
|
+
### Formatting Rules
|
|
70
|
+
|
|
71
|
+
- **Headings**: Use H2 for sections, H3 for subsections. Do not use H1 — generated from title. Avoid H4+
|
|
72
|
+
- **Lists**: Use `-` for bullet points and `1.` for numbered lists; indent nested lists with two spaces
|
|
73
|
+
- **Code Blocks**: Use fenced code blocks with language specified for syntax highlighting
|
|
74
|
+
- **Links**: Use `[link text](URL)` with descriptive text and valid URLs
|
|
75
|
+
- **Images**: Use `` with brief descriptive alt text
|
|
76
|
+
- **Tables**: Use `|` tables with properly aligned columns and headers
|
|
77
|
+
- **Whitespace**: Use blank lines to separate sections; avoid excessive whitespace
|
|
78
|
+
|
|
79
|
+
### Front Matter
|
|
80
|
+
|
|
81
|
+
Include YAML front matter at the beginning of instruction/skill files:
|
|
82
|
+
|
|
83
|
+
- `title` / `name`: The title of the document
|
|
84
|
+
- `description`: A brief description of the document content
|
|
85
|
+
- `applyTo`: (for instruction files) Glob pattern for which files the instructions apply to
|