opencastle 0.32.5 → 0.32.6

package/src/orchestrator/skills/api-patterns/SKILL.md

@@ -3,19 +3,17 @@ name: api-patterns
 description: "API design patterns for route handlers, Server Actions, Zod validation, and external API integration. Use when creating API routes, Server Actions, or integrating external services."
 ---
 
-<!-- ⚠️ This file is managed by OpenCastle. Edits will be overwritten on update. Customize in the .opencastle/ directory instead. -->
-
 # API Patterns
 
-Generic API design patterns for server-rendered framework projects. For project-specific endpoints, actions, and external API inventory, see [api-config.md](../../.opencastle/stack/api-config.md).
+Project-specific config: [api-config.md](../../.opencastle/stack/api-config.md).
 
 ## Architecture
 
-This project uses **App Router** API patterns (resolve the specific framework via the **framework** capability slot in the skill matrix):
-
-- **Server Actions** (preferred for mutations) — form submissions, data writes, auth operations
-- **Route Handlers** (`route.ts`) — analytics endpoints, autocomplete, external integrations
-- **Proxy layer** — IP rate limiting, fingerprinting, bot detection
+| Layer | Use for |
+|-------|---------|
+| **Server Actions** (preferred) | mutations, form submissions, data writes, auth |
+| **Route Handlers** (`route.ts`) | analytics, autocomplete, external integrations |
+| **Proxy layer** | IP rate limiting, fingerprinting, bot detection |
 
 ## Code Patterns
 
@@ -25,16 +23,11 @@ This project uses **App Router** API patterns (resolve the specific framework vi
 // app/api/example/route.ts
 import { NextRequest, NextResponse } from 'next/server';
 import { z } from 'zod';
-
 const schema = z.object({ query: z.string().min(1).max(200) });
 
 export async function GET(request: NextRequest) {
-  const params = Object.fromEntries(request.nextUrl.searchParams);
-  const result = schema.safeParse(params);
-  if (!result.success) {
-    return NextResponse.json({ error: 'Invalid input' }, { status: 400 });
-  }
-  // ... process
+  const result = schema.safeParse(Object.fromEntries(request.nextUrl.searchParams));
+  if (!result.success) return NextResponse.json({ error: 'Invalid input' }, { status: 400 });
   return NextResponse.json(data);
 }
 ```
@@ -47,62 +40,21 @@ import { createServerClient } from '@libs/auth';
 import { revalidatePath } from 'next/cache';
 
 export async function submitAction(formData: FormData) {
-  const client = await createServerClient();
-  const { data: { user } } = await client.auth.getUser();
+  const { data: { user } } = await (await createServerClient()).auth.getUser();
   if (!user) return { error: 'Unauthorized' };
-  // ... validate and process
   revalidatePath('/places');
   return { success: true };
 }
 ```
 
-## Design Principles
-
-- Prefer Server Actions for mutations over API routes
-- Always validate input with Zod schemas on the server side
-- Return appropriate HTTP status codes and error messages
-- Protect sensitive routes with middleware or role checks
-- Rate limit public endpoints to prevent abuse
-- Use Web `Request`/`Response` APIs with `NextRequest`/`NextResponse`
-- Use CDN caching headers for public, cacheable responses
-- Document new API endpoints in project documentation
-
-## API Design Principles
-
-### Route Architecture
-- RESTful resource naming: `/api/v1/places`, `/api/v1/places/:slug`
-- Use HTTP methods correctly: `GET` (read), `POST` (create), `PATCH` (partial update), `DELETE` (remove)
-- Group related endpoints under a common prefix
-- Keep URLs noun-based, not verb-based (`/api/places` not `/api/getPlaces`)
-
-### Request/Response Schemas
-- Define Zod schemas for all request bodies, query params, and responses
-- Use consistent envelope format for responses:
-  ```json
-  { "data": ..., "meta": { "total": 42, "page": 1 } }
-  ```
-- Error responses follow a standard shape:
-  ```json
-  { "error": { "code": "VALIDATION_ERROR", "message": "...", "details": [...] } }
-  ```
-
-### Error Handling
-- Use appropriate HTTP status codes (400, 401, 403, 404, 422, 429, 500)
-- Return machine-readable error codes alongside human-readable messages
-- Never leak internal errors — sanitize stack traces in production
-- Provide actionable error messages when possible
-
-### Pagination & Filtering
-- Cursor-based pagination for large datasets (offset-based as fallback)
-- Consistent query parameter names: `limit`, `cursor`, `sort`, `order`
-- Filter parameters match field names: `?type=brewery&city=prague`
-
-### Versioning
-- URL-based versioning: `/api/v1/...`
-- Never break existing contracts — add fields, never remove or rename
-- Deprecation notices in response headers before removal
-
-### Rate Limiting & Caching
-- Define rate limits per endpoint sensitivity
-- Set `Cache-Control` headers appropriate to content freshness
-- Use `ETag` / `If-None-Match` for conditional requests where applicable
+## Design Rules
+
+- Server Actions for mutations; Route Handlers for external/public endpoints
+- Validate all input with Zod on the server
+- RESTful nouns: `/api/v1/places/:slug`; HTTP methods: `GET` read, `POST` create, `PATCH` update, `DELETE` remove
+- Response envelope: `{ "data": ..., "meta": { "total": 42, "page": 1 } }`
+- Error shape: `{ "error": { "code": "VALIDATION_ERROR", "message": "...", "details": [...] } }`
+- Status codes: 400, 401, 403, 404, 422, 429, 500 — never leak stack traces
+- Pagination: cursor-based preferred; params: `limit`, `cursor`, `sort`, `order`
+- Versioning: `/api/v1/...`; add fields only, never remove/rename; deprecation headers before removal
+- Rate-limit public endpoints; set `Cache-Control` and `ETag`/`If-None-Match` headers

package/src/orchestrator/skills/code-commenting/SKILL.md

@@ -3,133 +3,81 @@ name: code-commenting
 description: "Guidelines for writing self-explanatory code with minimal comments. Covers when to comment (WHY not WHAT), anti-patterns to avoid, annotation tags, and public API documentation. Use when writing or reviewing code comments."
 ---
 
-<!-- ⚠️ This file is managed by OpenCastle. Edits will be overwritten on update. Customize in the .opencastle/ directory instead. -->
+# Code Commenting
 
-# Self-explanatory Code Commenting
+**Comment WHY, not WHAT.** Prefer renaming over commenting.
 
-## Core Principle
+## When to Comment
 
-**Write code that speaks for itself. Comment only when necessary to explain WHY, not WHAT.**
-We do not need comments most of the time.
+| Situation | Action |
+|-----------|--------|
+| Self-explanatory code | No comment |
+| Bad name is the real problem | Rename instead |
+| Complex business logic / non-obvious algorithm | Comment WHY |
+| Regex, API constraints, gotchas | Comment WHY |
+| Public API function/method | JSDoc |
+| Magic number / config constant | Inline rationale |
 
-## Decision Framework
-
-Before writing a comment, ask:
-
-1. **Is the code self-explanatory?** → No comment needed
-2. **Would a better variable/function name eliminate the need?** → Refactor instead
-3. **Does this explain WHY, not WHAT?** → Good comment
-4. **Will this help future maintainers?** → Good comment
-
-## Comments to AVOID
-
-**Obvious Comments**
+## Examples
 
 ```javascript
+// ✗ Obvious
 let counter = 0; // Initialize counter to zero
-counter++; // Increment counter by one
-```
-
-**Redundant Comments**
-
-```javascript
-function getUserName() {
-  return user.name; // Return the user's name
-}
-```
 
-**Outdated Comments**
-
-```javascript
-// Calculate tax at 5% rate
-const tax = price * 0.08; // Actually 8%
-```
-
-## Comments to WRITE
-
-**Complex Business Logic**
-
-```javascript
+// ✓ WHY
 // Apply progressive tax brackets: 10% up to 10k, 20% above
 const tax = calculateProgressiveTax(income, [0.1, 0.2], [10000]);
-```
-
-**Non-obvious Algorithms**
 
-```javascript
-// Using Floyd-Warshall for all-pairs shortest paths
-// because we need distances between all nodes
+// ✓ Algorithm rationale
+// Floyd-Warshall: need all-pairs distances, not just single-source
 for (let k = 0; k < vertices; k++) { /* ... */ }
-```
-
-**Regex Patterns**
 
-```javascript
-// Match email format: username@domain.extension
-const emailPattern = /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/;
-```
-
-**API Constraints or Gotchas**
-
-```javascript
-// GitHub API rate limit: 5000 requests/hour for authenticated users
+// ✓ API constraint
+// GitHub API: 5000 req/hr for authenticated users
 await rateLimiter.wait();
+
+// ✓ Config rationale
+const MAX_RETRIES = 3;     // network reliability baseline
+const API_TIMEOUT = 5000;  // Lambda max is 15 s — leave headroom
 ```
 
-## Public APIs
+## Public APIs — JSDoc
 
 ```javascript
 /**
- * Calculate compound interest using the standard formula.
- *
- * @param {number} principal - Initial amount invested
- * @param {number} rate - Annual interest rate (as decimal, e.g., 0.05 for 5%)
- * @param {number} time - Time period in years
- * @param {number} compoundFrequency - How many times per year interest compounds (default: 1)
- * @returns {number} Final amount after compound interest
+ * @param principal - Initial amount
+ * @param rate - Annual rate as decimal (0.05 = 5%)
+ * @param time - Years
+ * @param n - Compounds per year (default 1)
+ * @returns Final amount
  */
-function calculateCompoundInterest(principal, rate, time, compoundFrequency = 1) {
-  // ... implementation
-}
-```
-
-## Configuration and Constants
-
-```javascript
-const MAX_RETRIES = 3; // Based on network reliability studies
-const API_TIMEOUT = 5000; // AWS Lambda timeout is 15s, leaving buffer
+function calculateCompoundInterest(principal, rate, time, n = 1) { ... }
 ```
 
 ## Annotation Tags
 
-```javascript
-// TODO: Replace with proper user authentication after security review
-// FIXME: Memory leak in production - investigate connection pooling
-// HACK: Workaround for bug in library v2.1.0 - remove after upgrade
-// NOTE: This implementation assumes UTC timezone for all calculations
-// WARNING: This function modifies the original array instead of creating a copy
-// PERF: Consider caching this result if called frequently in hot path
-// SECURITY: Validate input to prevent SQL injection before using in query
-// BUG: Edge case failure when array is empty - needs investigation
-// REFACTOR: Extract this logic into separate utility function for reusability
-// DEPRECATED: Use newApiFunction() instead - this will be removed in v3.0
-```
+| Tag | Use |
+|-----|-----|
+| `TODO` | Planned work |
+| `FIXME` | Known bug needing fix |
+| `HACK` | Workaround — note why and when to remove |
+| `NOTE` | Important non-obvious constraint |
+| `WARNING` | Side effect / mutation risk |
+| `PERF` | Hot path — optimization opportunity |
+| `SECURITY` | Security-sensitive code |
+| `DEPRECATED` | Note replacement and removal version |
 
 ## Anti-Patterns
 
-- **Dead code comments** — Don't comment out code; delete it (git has history)
-- **Changelog comments** — Don't maintain change history in comments; use git
-- **Divider comments** — Don't use decorative separators; use proper file structure
-
-## Quality Checklist
-
-Before committing, ensure your comments:
+| Anti-pattern | Rule |
+|--------------|------|
+| Commented-out code | Delete it — git has history |
+| Changelog in comments | Use git log |
+| Decorative dividers | Use proper file/section structure |
 
-- [ ] Explain WHY, not WHAT
-- [ ] Are grammatically correct and clear
-- [ ] Will remain accurate as code evolves
-- [ ] Add genuine value to code understanding
-- [ ] Are placed appropriately (above the code they describe)
-- [ ] Use proper spelling and professional language
+## Checklist
 
-**The best comment is the one you don't need to write because the code is self-documenting.**
+- [ ] Explains WHY, not WHAT
+- [ ] Still accurate after the change
+- [ ] Adds genuine value
+- [ ] Placed above the code it describes

package/src/orchestrator/skills/context-map/SKILL.md

@@ -3,53 +3,30 @@ name: context-map
 description: "Generate a structured file impact map before making changes. Identifies all files that will be affected, their relationships, and cascade effects — improving file partitioning for parallel work and reducing unexpected side effects."
 ---
 
-<!-- ⚠️ This file is managed by OpenCastle. Edits will be overwritten on update. Customize in the .opencastle/ directory instead. -->
-
 # Skill: Context Map
 
-Generate a structured **file impact map** before any code changes begin. This map identifies all files that will be touched, their relationships, and cascade effects — directly improving the Team Lead's file partitioning for parallel agents.
+Generate a **file impact map** before code changes to identify affected files, relationships, and cascades — improving agent file partitions for parallel work.
 
 ## When to Use
 
-- Before **every feature implementation** (Phase 1: Research)
-- Before **refactoring** (Phase 1: Scope & Baseline)
-- Before **schema changes** that cascade through queries and components
-- Before **any task touching shared libraries** (`libs/`)
-- Optional for isolated bug fixes affecting 1-2 files
-
-## How to Generate a Context Map
-
-### Step 1: Identify the Entry Points
-
-Start from the task description and identify the primary files that MUST change:
-
-```
-Entry Points:
-- [file path] — [why it must change]
-- [file path] — [why it must change]
-```
-
-### Step 2: Trace Dependencies (Outward)
-
-For each entry point, trace what depends on it:
+| Use | Skip |
+|-----|------|
+| Feature implementation (Phase 1) | Isolated bug fixes ≤2 files |
+| Refactoring (Phase 1 Scope) | |
+| Schema changes cascading through queries/components | |
+| Any task touching `libs/` | |
 
-1. **Imports** — what files import this module? (`grep_search` or `list_code_usages`)
-2. **Type consumers** — what files use types/interfaces defined here?
-3. **Route references** — what pages render this component?
-4. **Query consumers** — what components or pages call this query?
-5. **Test files** — what test files cover this code?
+## Steps
 
-### Step 3: Trace Sources (Inward)
+### 1 — Entry Points
+Identify files that MUST change from the task description.
 
-For each entry point, trace what it depends on:
+### 2 — Trace Outward (dependents)
+Use `grep_search` / `list_code_usages`: imports, type consumers, route references, query consumers, test files.
 
-1. **Data sources** — which CMS schemas, content queries, or database tables feed this code?
-2. **Shared utilities** — which `libs/` modules does it use?
-3. **Configuration** — which config files affect its behavior?
-
-### Step 4: Build the Map
-
-Produce a structured map in this format:
+### 3 — Trace Inward (sources)
+CMS schemas, `libs/` utilities, config files.
+### 4 — Build the Map
 
 ```markdown
 ## Context Map: [Task Name]
@@ -57,81 +34,63 @@ Produce a structured map in this format:
 ### Entry Points (MUST change)
 | File | Reason | Owner |
 |------|--------|-------|
-| `libs/queries/src/lib/places.ts` | Add new query field | Content Engineer |
-| `libs/ui-kit/src/lib/components/PlaceCard/` | Display new field | UI/UX Expert |
+| `libs/queries/src/lib/places.ts` | Add query field | Content Engineer |
+| `libs/ui-kit/.../PlaceCard/` | Display new field | UI/UX Expert |
 
 ### Cascade Effects (WILL change)
 | File | Triggered By | Reason | Owner |
 |------|-------------|--------|-------|
-| `apps/web-app/places/page.tsx` | PlaceCard change | Update props | Frontend Dev |
-| `apps/admin-panel/places/page.tsx` | PlaceCard change | Update props | Frontend Dev |
-| `libs/queries/src/lib/__tests__/places.test.ts` | Query change | Update test | Testing Expert |
+| `apps/web-app/places/page.tsx` | PlaceCard | Update props | Frontend Dev |
+| `libs/queries/src/lib/__tests__/places.test.ts` | Query | Update test | Testing Expert |
 
-### Shared Boundaries (WATCH for conflicts)
+### Shared Boundaries (WATCH)
 | File | Risk | Mitigation |
 |------|------|------------|
-| `libs/ui-kit/src/lib/index.ts` | Barrel export — may conflict | Merge sequentially |
+| `libs/ui-kit/src/lib/index.ts` | Barrel export conflict | Merge sequentially |
 
-### Unaffected (explicitly safe)
+### Unaffected
 | Area | Why |
 |------|-----|
 | `db/migrations/` | No DB changes |
 | `libs/auth/` | No auth changes |
-| `apps/cms-studio/` | No schema changes |
 ```
 
-### Step 5: Derive File Partitions
+### 5 — Derive File Partitions
 
-From the context map, assign file ownership to agents:
+Assign ownership — no file in two partitions; shared boundaries to one agent (merged first); test files to Testing Expert unless tightly coupled.
 
 ```
-Agent A (Content Engineer):    libs/queries/src/lib/places.ts
-Agent B (UI/UX Expert):     libs/ui-kit/src/lib/components/PlaceCard/
-Agent C (Frontend Dev):      apps/web-app/places/, apps/admin-panel/places/
-Agent D (Testing Expert):   **/*test*, **/*spec*
+Agent A: libs/queries/src/lib/places.ts
+Agent B: libs/ui-kit/.../PlaceCard/
+Agent C: apps/web-app/places/, apps/admin-panel/places/
+Agent D: **/*test*, **/*spec*
 ```
 
-**Rules:**
-- No file appears in two partitions
-- Shared boundaries are assigned to ONE agent and merged first
-- Test files belong to the Testing Expert unless tightly coupled to a specific change
-
-## Context Map Depth Levels
-
-Scale the depth to the task complexity:
+## Depth Levels
 
-| Task Complexity | Depth | What to Trace |
-|----------------|-------|---------------|
-| **Small** (1-3 files) | Entry points only | Direct imports/exports |
-| **Medium** (4-8 files) | Entry + cascade | 1 hop of dependencies |
-| **Large** (9+ files) | Full map | Complete dependency graph |
+| Complexity | Files | Depth |
+|------------|-------|-------|
+| Small | 1–3 | Entry points + direct imports |
+| Medium | 4–8 | Entry + 1-hop cascade |
+| Large | 9+ | Full dependency graph |
 
-## Integration with Team Lead Workflow
+## Team Lead Integration
 
-The context map is produced in **Phase 1 (Research)** and consumed by:
-
-1. **Decomposition (Step 2)** — the map directly informs file partitions
-2. **Delegation prompts** — include the relevant section of the map so agents know their boundaries
-3. **Verification (QA Gate)** — compare actual changed files against the map to detect scope creep
-
-### Including in Delegation Prompts
+Produced in **Phase 1**; consumed by:
+- **Decomposition** — informs file partitions
+- **Delegation prompts** — agents receive their map section
+- **QA Gate** — compare actual changes against map to detect scope creep
 
+Delegation prompt snippet:
 ```markdown
-## Your File Partition (from Context Map)
-
-You own these files — modify only these:
-- `libs/queries/src/lib/places.ts`
-- `libs/queries/src/lib/__tests__/places.test.ts`
-
-Do NOT modify:
-- `libs/ui-kit/` (owned by UI/UX Expert)
-- `apps/` (owned by Developer)
+## Your File Partition
+Modify only: `libs/queries/src/lib/places.ts`, `libs/queries/src/lib/__tests__/places.test.ts`
+Do NOT modify: `libs/ui-kit/` (UI/UX Expert), `apps/` (Developer)
 ```
 
 ## Anti-Patterns
 
-- **Skipping the map for "obvious" tasks** — even small tasks can have unexpected cascades in shared libraries
-- **Mapping without searching** — don't guess dependencies; use `grep_search`, `list_code_usages`, and import tracing
-- **Over-mapping** — for a 2-file bug fix, don't trace the entire dependency graph. Match depth to complexity
-- **Stale maps** — if the plan changes during execution, update the map. A stale map is worse than no map
-- **Mapping files you won't change** — the "Unaffected" section is for explicitly noting what's safe, not for cataloging the entire codebase
+- Skipping for "obvious" tasks — shared libs cascade unexpectedly
+- Guessing dependencies instead of using `grep_search` / `list_code_usages`
+- Over-mapping a 2-file fix
+- Using a stale map after plan changes

package/src/orchestrator/skills/data-engineering/SKILL.md

@@ -3,27 +3,17 @@ name: data-engineering
 description: "Data pipeline ETL workflows, web scraping, NDJSON processing, and CMS data import. Use when building scrapers, processing data, running CLI tools, or importing to a CMS."
 ---
 
-<!-- ⚠️ This file is managed by OpenCastle. Edits will be overwritten on update. Customize in the .opencastle/ directory instead. -->
-
 # Data Engineering
 
-Generic data pipeline patterns and scraping methodology. For project-specific pipeline architecture, sources, CLI commands, and data status, see [data-pipeline-config.md](../../.opencastle/stack/data-pipeline-config.md).
+Generic pipeline patterns. For project-specific sources, CLI commands, and data status see [data-pipeline-config.md](../../.opencastle/stack/data-pipeline-config.md).
 
 ## Scraper Architecture
 
-### Base Scraper Pattern
-
 ```typescript
 interface ScraperConfig {
-  source: string;
-  query: string;
-  maxPages: number;
-  concurrency: number;
-  delay: { min: number; max: number };
-  outputPath: string;
-  headless: boolean;
+  source: string; query: string; maxPages: number; concurrency: number;
+  delay: { min: number; max: number }; outputPath: string; headless: boolean;
 }
-
 abstract class BaseScraper {
   abstract scrape(config: ScraperConfig): Promise<void>;
   abstract extractVenue(page: Page): Promise<RawVenue>;
@@ -31,74 +21,37 @@ abstract class BaseScraper {
 }
 ```
 
-### Browser-Based Scraper Setup
-
-Use a headless browser cluster for concurrent scraping (e.g., Puppeteer Cluster, Playwright):
-
-```typescript
-// Example using Puppeteer Cluster — adapt to your project's scraping library
-const cluster = await Cluster.launch({
-  concurrency: Cluster.CONCURRENCY_CONTEXT,
-  maxConcurrency: config.concurrency,
-  puppeteerOptions: {
-    headless: config.headless,
-    args: ['--no-sandbox', '--disable-setuid-sandbox'],
-  },
-  retryLimit: 3,
-  retryDelay: 5000,
-  timeout: 30000,
-});
-```
-
-### Anti-Detection Measures
-
-- Rotate user agents from a curated list
-- Random delays between requests (2-5 seconds default)
-- Randomize viewport sizes
-- Block unnecessary resources (images, fonts, CSS) for speed
-- Use stealth plugin for the scraping library
-- Request interception for resource optimization
+Launch a headless browser cluster (Puppeteer Cluster / Playwright) with `retryLimit: 3`, `retryDelay: 5000`, `timeout: 30000`, `args: ['--no-sandbox', '--disable-setuid-sandbox']`.
 
-### Error Recovery
+**Anti-detection:** rotate user-agents; random 2–5 s delays; randomize viewport; block images/fonts/CSS; use stealth plugin.
 
-- Retry failed pages with exponential backoff (3 retries default)
-- Log failed URLs for manual review
-- Save partial results on crash/interruption
-- Checkpoint/resume for long-running scrapes
+**Error recovery:** exponential backoff (3 retries); log failed URLs; save partial results; checkpoint/resume for long runs.
 
-## NDJSON Output Format
+## NDJSON Output
 
-Each scraper produces one record per line:
-
-```json
-{"name":"Example Venue","lat":50.0755,"lng":14.4378,"source":"google-maps","sourceId":"ChIJ...","category":"bar","address":"Street 30, City","rating":4.5,"reviewCount":120}
-```
+One record per line: `{"name":"…","lat":50.0755,"lng":14.4378,"source":"google-maps","sourceId":"ChIJ…","category":"bar","address":"…","rating":4.5,"reviewCount":120}`
 
-### Required Fields
-
-| Field | Priority | Notes |
-|-------|----------|-------|
+| Field | Type | Notes |
+|-------|------|-------|
 | `name` | Required | Preserve original encoding |
 | `lat`/`lng` | Required | GPS coordinates |
-| `address` | Required | Full text address |
-| `source` | Required | Source identifier (e.g., `google-maps`) |
-| `sourceId` | Required | Source-specific unique ID |
-| `category` | Required | Domain-specific category |
-
-### Optional Fields
-
-`rating`, `reviewCount`, `phone`, `website`, `openingHours`, `photos`, `priceLevel`
+| `address` | Required | Full text |
+| `source` | Required | e.g. `google-maps` |
+| `sourceId` | Required | Source-unique ID |
+| `category` | Required | Domain category |
+| `rating`, `reviewCount`, `phone`, `website`, `openingHours`, `photos`, `priceLevel` | Optional | — |
 
 ## Design Principles
 
-- Pipelines as composable, single-responsibility stages
-- Use streams for large file processing to minimize memory
-- Idempotent imports with `createOrReplace` and deterministic `_id` generation
-- Dry-run mode for all destructive operations
-- Generate normalized names by stripping diacritics for search
-- Structured addresses: `{ street, city, postalCode, country, countryCode }`
-- Track data lineage — record source and transformation history
-- Handle errors gracefully — skip bad records, don't halt pipeline
-- Backup before bulk operations
-- Respect `robots.txt` and rate limit all scraping requests
-- Only scrape publicly available data with source attribution
+| Principle | Detail |
+|-----------|--------|
+| Composable stages | Single-responsibility pipeline steps |
+| Streams | Use for large files to minimize memory |
+| Idempotent imports | `createOrReplace` + deterministic `_id` |
+| Dry-run mode | Required for all destructive operations |
+| Normalized names | Strip diacritics for search |
+| Structured addresses | `{ street, city, postalCode, country, countryCode }` |
+| Data lineage | Record source and transformation history |
+| Error handling | Skip bad records; don't halt pipeline |
+| Backup | Before all bulk operations |
+| Rate limiting | Respect `robots.txt`; attribute sources |