@jetrabbits/agentic 0.0.1

package/areas/software/backend/skills/api-design/SKILL.md

@@ -0,0 +1,170 @@
+---
+name: api-design
+type: skill
+description: Design and implement REST APIs with consistent conventions, versioning, error contracts, and security.
+related-rules:
+  - architecture.md
+  - security.md
+allowed-tools: Read, Write, Edit, Bash
+---
+
+# API Design Skill
+
+> **Expertise:** REST API design, HTTP semantics, versioning strategies, error contracts, OpenAPI, pagination, idempotency.
+
+## HTTP Method & Status Code Reference
+
+| Operation | Method | Success | Error cases |
+|---|---|---|---|
+| Create resource | POST | 201 Created | 400 Validation, 409 Conflict |
+| Read resource | GET | 200 OK | 404 Not Found, 403 Forbidden |
+| Full update | PUT | 200 OK | 400, 404, 409 |
+| Partial update | PATCH | 200 OK | 400, 404 |
+| Delete | DELETE | 204 No Content | 404, 409 (has dependents) |
+| Async action | POST | 202 Accepted | 400 |
+
+## URL Design Rules
+
+```
+✅ Nouns, plural, lowercase, kebab-case
+   GET  /users/{id}
+   POST /orders
+   GET  /product-categories
+
+❌ Verbs in path
+   POST /createOrder
+   GET  /getUser?id=123
+
+✅ Nesting only for true ownership (max 2 levels)
+   GET  /orders/{order_id}/items          ✅ items belong to order
+   GET  /orders/{order_id}/items/{item_id}/tags   ❌ too deep
+
+✅ Actions as sub-resources
+   POST /orders/{id}/cancel
+   POST /invoices/{id}/resend
+```
+
+## Standard Error Contract
+
+All error responses must follow the same shape:
+```json
+{
+  "error": {
+    "code": "ORDER_NOT_FOUND",        // machine-readable, stable
+    "message": "Order ord_123 not found",  // human-readable
+    "details": [                       // optional: per-field errors
+      { "field": "items[0].quantity", "issue": "must be > 0" }
+    ],
+    "request_id": "req_abc123"        // traceable
+  }
+}
+```
+
+```python
+# FastAPI implementation
+from fastapi import HTTPException
+from pydantic import BaseModel
+from typing import Optional, List
+
+class ErrorDetail(BaseModel):
+    field: str
+    issue: str
+
+class ErrorResponse(BaseModel):
+    code: str
+    message: str
+    details: Optional[List[ErrorDetail]] = None
+    request_id: Optional[str] = None
+
+# Usage
+raise HTTPException(
+    status_code=404,
+    detail=ErrorResponse(
+        code="ORDER_NOT_FOUND",
+        message=f"Order {order_id} not found",
+        request_id=request.state.request_id
+    ).model_dump()
+)
+```
+
+## Pagination Patterns
+
+### Cursor-based (preferred for large/live datasets)
+```python
+class PaginatedResponse(BaseModel, Generic[T]):
+    items: List[T]
+    next_cursor: Optional[str] = None   # opaque, base64-encoded
+    total_count: Optional[int] = None   # only if cheap to compute
+
+# Encode cursor: hide implementation detail from client
+import base64, json
+
+def encode_cursor(last_id: int, last_created_at: datetime) -> str:
+    payload = {"id": last_id, "ts": last_created_at.isoformat()}
+    return base64.b64encode(json.dumps(payload).encode()).decode()
+
+def decode_cursor(cursor: str) -> dict:
+    return json.loads(base64.b64decode(cursor).decode())
+```
+
+### Offset-based (only for small, static datasets)
+```python
+# Only when total count is cheap and dataset doesn't change under pagination
+@router.get("/products")
+async def list_products(page: int = 1, page_size: int = Query(20, le=100)):
+    offset = (page - 1) * page_size
+    # ...
+```
+
+## Versioning Strategy
+
+```
+URL versioning (recommended for major breaking changes):
+  /api/v1/orders
+  /api/v2/orders  ← new schema, old still supported
+
+Header versioning (for minor variations):
+  Accept: application/vnd.myapi.v2+json
+
+Rules:
+- v1 stays alive for minimum 6 months after v2 launch
+- Deprecated endpoints return: Deprecation: true, Sunset: <date> headers
+- Never remove a field from a response without a major version bump
+```
+
+## Idempotency
+
+```python
+# Idempotency key pattern for POST mutations
+@router.post("/orders", status_code=201)
+async def create_order(
+    body: CreateOrderRequest,
+    idempotency_key: Optional[str] = Header(None, alias="X-Idempotency-Key"),
+    db: AsyncSession = Depends(get_db),
+    redis: Redis = Depends(get_redis),
+):
+    if idempotency_key:
+        cached = await redis.get(f"idempotency:{idempotency_key}")
+        if cached:
+            return JSONResponse(json.loads(cached), status_code=200)  # 200 = already done
+
+    order = await order_service.create(db, body)
+    response = OrderResponse.model_validate(order)
+
+    if idempotency_key:
+        await redis.setex(
+            f"idempotency:{idempotency_key}",
+            86400,  # 24h TTL
+            response.model_dump_json()
+        )
+    return response
+```
+
+## Security Checklist
+
+- [ ] Auth middleware applied before any handler logic — never inside handler
+- [ ] Ownership check: `if order.user_id != current_user.id: raise 403`
+- [ ] Input validation via Pydantic (FastAPI) or Joi/Zod (Node) on every endpoint
+- [ ] Rate limiting on public endpoints (especially auth, password reset)
+- [ ] No sensitive data in URLs (tokens, passwords) — use request body or headers
+- [ ] CORS configured explicitly: no `allow_origins=["*"]` in production

package/areas/software/backend/skills/async-processing/SKILL.md

@@ -0,0 +1,152 @@
+---
+name: async-processing
+type: skill
+description: Design and implement async task queues, message consumers, and background job patterns.
+related-rules:
+  - architecture.md
+  - data_access.md
+allowed-tools: Read, Write, Edit, Bash
+---
+
+# Async Processing Skill
+
+> **Expertise:** Task queues (Celery, ARQ, Dramatiq), Kafka/NATS consumers, background jobs, retry strategies, idempotency, dead-letter queues.
+
+## When to Use Async Processing
+
+```
+Use async when:
+✅ Operation takes > 200ms (send email, resize image, call slow 3rd party)
+✅ Work can be retried independently (payment webhook, notification)
+✅ Decoupling producers from consumers is required
+✅ Fan-out to multiple consumers needed
+
+Keep synchronous:
+❌ Response depends on the result (user sees outcome immediately)
+❌ Must be transactional with the triggering DB write
+```
+
+## Task Queue (Celery + Redis)
+
+```python
+# tasks.py
+from celery import Celery
+from celery.utils.log import get_task_logger
+
+app = Celery("myapp", broker="redis://localhost:6379/1", backend="redis://localhost:6379/2")
+app.conf.update(
+    task_serializer="json",
+    result_expires=3600,
+    task_acks_late=True,          # Ack after completion, not on receive
+    task_reject_on_worker_lost=True,
+    task_default_retry_delay=60,  # 1 min base delay
+    task_max_retries=5,
+)
+
+logger = get_task_logger(__name__)
+
+@app.task(bind=True, max_retries=5, default_retry_delay=30)
+def send_order_confirmation(self, order_id: int) -> None:
+    try:
+        order = Order.objects.get(id=order_id)
+        email_service.send_confirmation(order)
+        logger.info("email.sent", extra={"order_id": order_id})
+    except EmailServiceError as exc:
+        # Exponential backoff: 30s, 60s, 120s, 240s, 480s
+        delay = 30 * (2 ** self.request.retries)
+        raise self.retry(exc=exc, countdown=delay)
+    except Order.DoesNotExist:
+        logger.error("order.not_found", extra={"order_id": order_id})
+        # Don't retry — data issue, not transient
+```
+
+## Message Consumer (Kafka / aiokafka)
+
+```python
+from aiokafka import AIOKafkaConsumer
+import asyncio, json
+
+async def consume_order_events():
+    consumer = AIOKafkaConsumer(
+        "orders.events",
+        bootstrap_servers="kafka:9092",
+        group_id="notification-service",
+        auto_offset_reset="earliest",
+        enable_auto_commit=False,    # Manual commit — control exactly-once
+        value_deserializer=lambda v: json.loads(v.decode()),
+    )
+    await consumer.start()
+    try:
+        async for msg in consumer:
+            event = msg.value
+            try:
+                await handle_event(event)
+                await consumer.commit()           # Only commit on success
+            except TransientError as e:
+                logger.warning("event.retry", event_type=event["type"], error=str(e))
+                await asyncio.sleep(5)            # Back off, do NOT commit
+            except PermanentError as e:
+                logger.error("event.dead_letter", event=event, error=str(e))
+                await dead_letter_queue.publish(event)
+                await consumer.commit()           # Commit to move past poison message
+    finally:
+        await consumer.stop()
+
+# Idempotency — always check before processing
+async def handle_event(event: dict) -> None:
+    event_id = event["event_id"]
+    if await redis.exists(f"processed:{event_id}"):
+        return  # Already handled — skip
+
+    await process(event)
+    await redis.setex(f"processed:{event_id}", 86400, "1")
+```
+
+## Background Jobs (ARQ — lightweight async)
+
+```python
+# worker.py
+from arq import cron
+from arq.connections import RedisSettings
+
+async def cleanup_expired_sessions(ctx):
+    deleted = await db.sessions.delete_expired()
+    ctx["log"].info(f"Cleaned up {deleted} expired sessions")
+
+async def startup(ctx):
+    ctx["db"] = await create_db_pool()
+    ctx["log"] = structlog.get_logger()
+
+class WorkerSettings:
+    functions = [cleanup_expired_sessions]
+    cron_jobs = [cron(cleanup_expired_sessions, hour=3, minute=0)]  # 3 AM daily
+    redis_settings = RedisSettings(host="redis")
+    on_startup = startup
+    max_jobs = 10
+    job_timeout = 300   # 5 min max per job
+```
+
+## Retry Strategy Reference
+
+| Error type | Strategy |
+|---|---|
+| Transient (network timeout, DB lock) | Exponential backoff, up to 5 retries |
+| Rate limit (429) | Respect Retry-After header |
+| Data validation (bad payload) | Dead-letter immediately — no retry |
+| Downstream service down | Circuit breaker + retry queue |
+| DB connection pool full | Short delay (5s) + 3 retries |
+
+## Dead-Letter Queue Pattern
+
+```python
+async def send_to_dlq(original_message: dict, error: Exception, queue_name: str):
+    dlq_message = {
+        "original": original_message,
+        "error": str(error),
+        "error_type": type(error).__name__,
+        "failed_at": datetime.utcnow().isoformat(),
+        "source_queue": queue_name,
+    }
+    await redis.lpush("dlq", json.dumps(dlq_message))
+    metrics.increment("tasks.dead_lettered", tags={"queue": queue_name})
+```

package/areas/software/backend/skills/database-modeling/SKILL.md

@@ -0,0 +1,173 @@
+---
+name: database-modeling
+type: skill
+description: Design relational schemas, write efficient queries, plan indexes, and implement safe migrations.
+related-rules:
+  - data_access.md
+  - architecture.md
+allowed-tools: Read, Write, Edit, Bash
+---
+
+# Database Modeling Skill
+
+> **Expertise:** PostgreSQL schema design, SQLAlchemy (async), query optimization, indexing, migrations (Alembic), safe schema changes.
+
+## Schema Design Patterns
+
+### Standard column set (all tables)
+```python
+from sqlalchemy import Column, Integer, DateTime, func
+from sqlalchemy.orm import DeclarativeBase
+
+class Base(DeclarativeBase):
+    pass
+
+class TimestampMixin:
+    created_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True), server_default=func.now(), nullable=False
+    )
+    updated_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True), server_default=func.now(),
+        onupdate=func.now(), nullable=False
+    )
+
+class Order(TimestampMixin, Base):
+    __tablename__ = "orders"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id"), nullable=False, index=True)
+    status: Mapped[str] = mapped_column(String(20), nullable=False, default="pending")
+    total_amount: Mapped[Decimal] = mapped_column(Numeric(12, 2), nullable=False)
+```
+
+### Soft delete pattern
+```python
+class SoftDeleteMixin:
+    deleted_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
+
+    @property
+    def is_deleted(self) -> bool:
+        return self.deleted_at is not None
+
+# Always filter in repository, never expose deleted records by default
+class OrderRepository:
+    async def list_active(self, session: AsyncSession):
+        return await session.execute(
+            select(Order).where(Order.deleted_at.is_(None))
+        )
+```
+
+## Indexing Strategy
+
+```sql
+-- Single column: high-cardinality columns used in WHERE/JOIN/ORDER BY
+CREATE INDEX idx_orders_user_id ON orders(user_id);
+CREATE INDEX idx_orders_status ON orders(status) WHERE deleted_at IS NULL;  -- partial index
+
+-- Composite: query uses both columns together (order matters: equality first, then range)
+CREATE INDEX idx_orders_user_created ON orders(user_id, created_at DESC);
+
+-- Full-text search
+CREATE INDEX idx_products_search ON products USING gin(to_tsvector('english', name || ' ' || description));
+
+-- Never index: low-cardinality boolean columns, small tables (<1000 rows)
+```
+
+### EXPLAIN ANALYZE checklist
+```sql
+-- Run before every new query on a table with >10k rows
+EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
+SELECT * FROM orders WHERE user_id = 123 ORDER BY created_at DESC LIMIT 20;
+
+-- Watch for: Seq Scan on large table → add index
+--            Index Scan with high actual rows >> estimated rows → ANALYZE the table
+--            Nested Loop with large inner side → consider Hash Join
+```
+
+## Repository Pattern
+
+```python
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy import select, update
+
+class OrderRepository:
+    def __init__(self, session: AsyncSession):
+        self.session = session
+
+    async def get_by_id(self, order_id: int) -> Optional[Order]:
+        result = await self.session.execute(
+            select(Order).where(Order.id == order_id, Order.deleted_at.is_(None))
+        )
+        return result.scalar_one_or_none()
+
+    async def list_by_user(
+        self, user_id: int, *, limit: int = 20, cursor_id: Optional[int] = None
+    ) -> list[Order]:
+        q = select(Order).where(Order.user_id == user_id, Order.deleted_at.is_(None))
+        if cursor_id:
+            q = q.where(Order.id < cursor_id)  # cursor-based pagination
+        q = q.order_by(Order.id.desc()).limit(limit)
+        result = await self.session.execute(q)
+        return list(result.scalars())
+
+    async def update_status(self, order_id: int, status: str) -> None:
+        await self.session.execute(
+            update(Order).where(Order.id == order_id).values(status=status)
+        )
+        # No commit here — caller (service layer) owns the transaction
+```
+
+## Migration Safety (Alembic)
+
+```bash
+# Generate migration
+alembic revision --autogenerate -m "add_index_orders_user_id"
+
+# ALWAYS review generated file before applying
+alembic show head
+
+# Apply
+alembic upgrade head
+
+# Rollback one step
+alembic downgrade -1
+```
+
+### Safe vs. unsafe schema operations
+
+| Operation | Safe to deploy | Strategy |
+|---|---|---|
+| Add nullable column | ✅ Non-breaking | Apply directly |
+| Add column with default | ✅ (PostgreSQL 11+) | Apply directly |
+| Add NOT NULL column | ⚠️ Breaking | Add nullable → backfill → add constraint |
+| Add index | ✅ with CONCURRENTLY | `CREATE INDEX CONCURRENTLY` |
+| Rename column | ❌ Breaking | Expand/contract (add new → migrate code → drop old) |
+| Drop column | ❌ Breaking | Deprecate in code → drop in next release |
+| Change type | ❌ Breaking | Add new column with new type → migrate → drop old |
+
+```python
+# Alembic: create index without locking table
+def upgrade():
+    op.execute("CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_orders_user_id ON orders(user_id)")
+
+def downgrade():
+    op.execute("DROP INDEX CONCURRENTLY IF EXISTS idx_orders_user_id")
+```
+
+## N+1 Query Prevention
+
+```python
+# ❌ N+1: loads orders, then 1 query per order to get user
+orders = await session.execute(select(Order)).scalars()
+for order in orders:
+    print(order.user.name)  # each access fires a query
+
+# ✅ Eager load with joinedload
+from sqlalchemy.orm import joinedload
+
+orders = await session.execute(
+    select(Order)
+    .options(joinedload(Order.user))  # single JOIN
+    .where(Order.status == "pending")
+)
+```

package/areas/software/backend/skills/observability/SKILL.md

@@ -0,0 +1,162 @@
+---
+name: observability
+type: skill
+description: Implement structured logging, distributed tracing, and metrics for production-ready backend services.
+related-rules:
+  - architecture.md
+allowed-tools: Read, Write, Edit, Bash
+---
+
+# Observability Skill
+
+> **Expertise:** Structured JSON logging, OpenTelemetry distributed tracing, Prometheus/RED metrics, alert design.
+
+## Structured Logging
+
+```python
+import structlog
+import logging
+
+# Configure once at app startup
+structlog.configure(
+    processors=[
+        structlog.contextvars.merge_contextvars,
+        structlog.processors.add_log_level,
+        structlog.processors.TimeStamper(fmt="iso"),
+        structlog.processors.JSONRenderer(),   # machine-parseable
+    ],
+    logger_factory=structlog.PrintLoggerFactory(),
+)
+
+log = structlog.get_logger()
+
+# Bind context per request (FastAPI middleware)
+@app.middleware("http")
+async def logging_middleware(request: Request, call_next):
+    request_id = request.headers.get("X-Request-ID") or str(uuid4())
+    structlog.contextvars.bind_contextvars(
+        request_id=request_id,
+        method=request.method,
+        path=request.url.path,
+    )
+    response = await call_next(request)
+    structlog.contextvars.unbind_contextvars("request_id", "method", "path")
+    return response
+
+# Usage in service/repository layer
+log.info("order.created", order_id=order.id, user_id=user.id, amount=str(order.total))
+log.warning("payment.retry", order_id=order_id, attempt=attempt, reason=str(error))
+log.error("db.query_failed", table="orders", query_type="insert", exc_info=True)
+```
+
+### What NOT to log
+```python
+# ❌ Never log PII or secrets
+log.info("user.login", email=user.email)      # PII — omit or hash
+log.debug("auth.token", token=access_token)   # secret — never
+
+# ✅ Log identifiers, not values
+log.info("user.login", user_id=user.id)
+log.info("auth.issued", token_jti=token_payload["jti"])
+```
+
+## Distributed Tracing (OpenTelemetry)
+
+```python
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+
+# Setup (once at startup)
+provider = TracerProvider()
+provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
+trace.set_tracer_provider(provider)
+
+tracer = trace.get_tracer("order-service")
+
+# Instrument a service method
+async def create_order(self, user_id: int, items: list) -> Order:
+    with tracer.start_as_current_span("order.create") as span:
+        span.set_attribute("user.id", user_id)
+        span.set_attribute("items.count", len(items))
+
+        try:
+            order = await self.repo.create(user_id, items)
+            span.set_attribute("order.id", order.id)
+            return order
+        except Exception as e:
+            span.record_exception(e)
+            span.set_status(trace.StatusCode.ERROR)
+            raise
+```
+
+## Metrics (Prometheus / RED)
+
+RED method: **R**ate · **E**rror rate · **D**uration for every service boundary.
+
+```python
+from prometheus_client import Counter, Histogram, start_http_server
+
+# Define metrics at module level (not inside functions)
+REQUEST_COUNT = Counter(
+    "http_requests_total",
+    "Total HTTP requests",
+    ["method", "endpoint", "status_code"]
+)
+
+REQUEST_DURATION = Histogram(
+    "http_request_duration_seconds",
+    "HTTP request duration",
+    ["method", "endpoint"],
+    buckets=[0.01, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0]
+)
+
+DB_QUERY_DURATION = Histogram(
+    "db_query_duration_seconds",
+    "DB query duration",
+    ["operation", "table"],
+    buckets=[0.001, 0.005, 0.01, 0.05, 0.1, 0.5]
+)
+
+# FastAPI middleware
+@app.middleware("http")
+async def metrics_middleware(request: Request, call_next):
+    start = time.monotonic()
+    response = await call_next(request)
+    duration = time.monotonic() - start
+
+    endpoint = request.url.path
+    REQUEST_COUNT.labels(request.method, endpoint, response.status_code).inc()
+    REQUEST_DURATION.labels(request.method, endpoint).observe(duration)
+    return response
+```
+
+## Health Check Endpoint
+
+```python
+@router.get("/health", include_in_schema=False)
+async def health(db: AsyncSession = Depends(get_db)):
+    checks = {}
+    try:
+        await db.execute(text("SELECT 1"))
+        checks["database"] = "ok"
+    except Exception as e:
+        checks["database"] = f"error: {e}"
+
+    is_healthy = all(v == "ok" for v in checks.values())
+    return JSONResponse(
+        {"status": "ok" if is_healthy else "degraded", "checks": checks},
+        status_code=200 if is_healthy else 503
+    )
+```
+
+## Alerting Rules (Reference)
+
+| Signal | Threshold | Severity |
+|---|---|---|
+| Error rate | > 1% over 5 min | P1 |
+| p99 latency | > 2s over 5 min | P1 |
+| p95 latency | > 500ms over 15 min | P2 |
+| DB connection pool saturation | > 80% for 5 min | P2 |
+| Health check failures | 2 consecutive | P1 |