start-vibing-stacks 2.6.0 → 2.7.0

package/stacks/_shared/skills/observability/SKILL.md

@@ -0,0 +1,351 @@
+---
+name: observability
+version: 1.0.0
+description: Structured logging, correlation IDs, OpenTelemetry tracing, error tracking (Sentry), metrics, and PII redaction. Invoke when adding logging, instrumentation, or debugging production issues.
+---
+
+# Observability — Logs, Traces, Metrics
+
+**ALWAYS invoke when adding logging, instrumentation, error tracking, or analyzing production issues.**
+
+> Three pillars: **Logs** (what happened) + **Traces** (where time was spent) + **Metrics** (how much / how often).
+> One signal: **correlation/trace IDs** that thread through all three.
+
+---
+
+## 1. Structured Logging — Mandatory
+
+Logs are JSON, not text. Free-form strings can't be queried, aggregated, or alerted on.
+
+### Required fields per log line
+
+| Field | Source |
+|---|---|
+| `timestamp` | ISO-8601, UTC |
+| `level` | `trace` / `debug` / `info` / `warn` / `error` / `fatal` |
+| `msg` | Short human description |
+| `service` | Service name |
+| `env` | `production` / `staging` / `development` |
+| `trace_id` | OpenTelemetry trace id (W3C `traceparent`) |
+| `span_id` | Current span |
+| `request_id` | Inbound HTTP request id (mirror to `x-request-id` header) |
+| `user_id` | Authenticated user (hash if PII concerns) — **never** email/full name |
+
+### Node.js — pino
+```ts
+// lib/logger.ts
+import pino from 'pino';
+import { randomUUID } from 'crypto';
+
+export const logger = pino({
+  level: process.env['LOG_LEVEL'] ?? 'info',
+  base: {
+    service: 'api',
+    env: process.env['NODE_ENV'],
+    pid: process.pid,
+  },
+  timestamp: pino.stdTimeFunctions.isoTime,
+  redact: {
+    paths: [
+      'req.headers.authorization',
+      'req.headers.cookie',
+      'req.body.password',
+      'req.body.token',
+      '*.password',
+      '*.creditCard',
+      '*.ssn',
+    ],
+    censor: '[REDACTED]',
+  },
+});
+
+// HTTP middleware: attach request_id + child logger to req
+export function requestLogger(req, res, next) {
+  const requestId = req.headers['x-request-id'] ?? randomUUID();
+  res.setHeader('x-request-id', requestId);
+  req.log = logger.child({ request_id: requestId, method: req.method, path: req.path });
+  req.log.info({ event: 'request.start' });
+  res.on('finish', () => {
+    req.log.info({ event: 'request.end', status: res.statusCode, duration_ms: Date.now() - req.startTime });
+  });
+  next();
+}
+```
+
+### Python — structlog
+```python
+import structlog, logging, sys, uuid
+
+structlog.configure(
+    processors=[
+        structlog.contextvars.merge_contextvars,
+        structlog.processors.add_log_level,
+        structlog.processors.TimeStamper(fmt="iso", utc=True),
+        structlog.processors.StackInfoRenderer(),
+        structlog.processors.format_exc_info,
+        structlog.processors.JSONRenderer(),
+    ],
+    wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
+    logger_factory=structlog.PrintLoggerFactory(file=sys.stdout),
+)
+
+log = structlog.get_logger()
+
+# FastAPI middleware
+@app.middleware("http")
+async def request_logger(request, call_next):
+    request_id = request.headers.get("x-request-id", str(uuid.uuid4()))
+    structlog.contextvars.bind_contextvars(
+        request_id=request_id, method=request.method, path=request.url.path
+    )
+    log.info("request.start")
+    response = await call_next(request)
+    response.headers["x-request-id"] = request_id
+    log.info("request.end", status=response.status_code)
+    structlog.contextvars.clear_contextvars()
+    return response
+```
+
+### PHP — Monolog
+```php
+// config/logging.php
+'channels' => [
+    'json' => [
+        'driver' => 'monolog',
+        'handler' => Monolog\Handler\StreamHandler::class,
+        'with' => ['stream' => 'php://stdout'],
+        'formatter' => Monolog\Formatter\JsonFormatter::class,
+        'processors' => [
+            Monolog\Processor\WebProcessor::class,
+            Monolog\Processor\UidProcessor::class,
+        ],
+    ],
+],
+```
+
+---
+
+## 2. Log Levels — Use Them Right
+
+| Level | When |
+|---|---|
+| `trace` | Verbose diagnostics, off in prod |
+| `debug` | Development helper, off in prod by default |
+| `info` | Business events: signup, payment, login, job completed |
+| `warn` | Recoverable anomaly: retry, deprecated path, fallback used |
+| `error` | Operation failed for one user/request, system continues |
+| `fatal` | Cannot continue, process exits |
+
+Default prod level: `info`. Errors should always page or alert. `debug` flooding logs is a cost issue.
+
+---
+
+## 3. PII Redaction — Mandatory
+
+Never log:
+- Passwords, hashes, tokens, cookies, `Authorization` headers
+- Full credit card / IBAN / SSN / passport
+- Plaintext email if your jurisdiction (GDPR/LGPD) treats it as PII without legitimate basis
+- Full request body or full response body without filtering
+
+Redact at the logger level (so it can't be bypassed by a forgetful caller):
+
+```ts
+// pino redact paths (above)
+// or wrap manually:
+function safeBody(body: any) {
+  const out = { ...body };
+  for (const k of ['password', 'token', 'cookie', 'authorization']) {
+    if (k in out) out[k] = '[REDACTED]';
+  }
+  return out;
+}
+```
+
+For email: log a hash of the email or the first 2 chars + domain (`jo***@example.com`). Document the policy.
+
+---
+
+## 4. Distributed Tracing — OpenTelemetry
+
+Tracing answers "where did the request spend time" across services.
+
+### Node.js (auto-instrumentation)
+```ts
+// instrumentation.ts — load BEFORE any other import
+import { NodeSDK } from '@opentelemetry/sdk-node';
+import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
+import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
+import { Resource } from '@opentelemetry/resources';
+import { ATTR_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
+
+new NodeSDK({
+  resource: new Resource({ [ATTR_SERVICE_NAME]: 'api' }),
+  traceExporter: new OTLPTraceExporter({ url: process.env['OTEL_EXPORTER_OTLP_ENDPOINT'] }),
+  instrumentations: [getNodeAutoInstrumentations()],
+}).start();
+```
+
+Run: `node --import ./instrumentation.js dist/index.js`
+
+### Python (FastAPI)
+```python
+from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
+from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor
+from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
+
+FastAPIInstrumentor.instrument_app(app)
+SQLAlchemyInstrumentor().instrument(engine=engine)
+HTTPXClientInstrumentor().instrument()
+```
+
+### Manual span — when you want to time business logic
+```ts
+import { trace } from '@opentelemetry/api';
+const tracer = trace.getTracer('billing');
+
+await tracer.startActiveSpan('charge_customer', async (span) => {
+  try {
+    span.setAttribute('customer.id', customerId);
+    const charge = await stripe.charges.create({ amount, customer: customerId });
+    span.setAttribute('charge.id', charge.id);
+    return charge;
+  } catch (err) {
+    span.recordException(err);
+    span.setStatus({ code: 2 /* ERROR */ });
+    throw err;
+  } finally {
+    span.end();
+  }
+});
+```
+
+---
+
+## 5. Error Tracking — Sentry
+
+Pair with structured logging. Sentry is for **alertable** errors with full stack + breadcrumbs; logs are for **everything**.
+
+### Node.js
+```ts
+import * as Sentry from '@sentry/node';
+
+Sentry.init({
+  dsn: process.env['SENTRY_DSN'],
+  environment: process.env['NODE_ENV'],
+  tracesSampleRate: 0.1,
+  profilesSampleRate: 0.1,
+  beforeSend(event) {
+    // Defense in depth — strip if logger redaction missed it
+    if (event.request?.cookies) delete event.request.cookies;
+    if (event.request?.headers?.['authorization']) {
+      event.request.headers['authorization'] = '[REDACTED]';
+    }
+    return event;
+  },
+});
+```
+
+### Python
+```python
+import sentry_sdk
+from sentry_sdk.integrations.fastapi import FastApiIntegration
+from sentry_sdk.scrubber import EventScrubber, DEFAULT_DENYLIST
+
+sentry_sdk.init(
+    dsn=os.environ["SENTRY_DSN"],
+    environment=os.environ["APP_ENV"],
+    traces_sample_rate=0.1,
+    integrations=[FastApiIntegration()],
+    event_scrubber=EventScrubber(denylist=DEFAULT_DENYLIST + ["jwt", "session"]),
+    send_default_pii=False,
+)
+```
+
+---
+
+## 6. Metrics
+
+Track **RED** (Rate / Errors / Duration) on every endpoint and **USE** (Utilization / Saturation / Errors) on every resource.
+
+OpenTelemetry metrics + Prometheus exporter, or vendor SDK (Datadog, New Relic).
+
+```ts
+import { metrics } from '@opentelemetry/api';
+const meter = metrics.getMeter('api');
+
+const httpDuration = meter.createHistogram('http.server.duration', { unit: 'ms' });
+const ordersCreated = meter.createCounter('orders.created');
+
+httpDuration.record(durationMs, { method, route, status: String(statusCode) });
+ordersCreated.add(1, { plan: order.plan });
+```
+
+Cardinality rule: never tag with user_id, email, or unbounded values — explodes time-series storage. Use `route`, `status`, `plan`, etc.
+
+---
+
+## 7. Health Checks
+
+Two endpoints, distinct semantics:
+
+```ts
+app.get('/healthz', (_, res) => res.json({ ok: true }));   // liveness — am I running?
+
+app.get('/readyz', async (_, res) => {                      // readiness — can I serve?
+  const checks = await Promise.allSettled([
+    db.raw('SELECT 1'),
+    redis.ping(),
+  ]);
+  const allOk = checks.every(c => c.status === 'fulfilled');
+  res.status(allOk ? 200 : 503).json({ ok: allOk, checks });
+});
+```
+
+Kubernetes uses `livenessProbe` → `/healthz` (restart on fail) and `readinessProbe` → `/readyz` (remove from LB on fail).
+
+---
+
+## 8. Audit Log — Separate Stream
+
+Security-relevant events go to a dedicated, append-only stream:
+
+- Logins (success + failure)
+- Authorization denials
+- Permission/role changes
+- Admin actions (impersonation, data exports, deletes)
+- Payment events
+- Data access (when required by compliance: HIPAA, SOC 2)
+
+Different retention (often longer), different access (security team), different integrity guarantees (immutable, hash-chained).
+
+---
+
+## Pre-Commit Checklist
+
+- [ ] Logger configured with redact paths for password/token/cookie/authorization
+- [ ] All endpoints emit `request.start` / `request.end` with `request_id`
+- [ ] Errors include stack trace and `trace_id`
+- [ ] No `console.log` / `print` / `dd()` left in code (use the logger)
+- [ ] No PII in info-level logs
+- [ ] Sentry initialized with `beforeSend` scrubber
+- [ ] Tracing initialized at process start (before app code)
+- [ ] `/healthz` + `/readyz` endpoints exist
+
+## FORBIDDEN
+
+| Pattern | Reason |
+|---|---|
+| `console.log(req.body)` | Leaks passwords, tokens |
+| Logger without redaction | Future caller will leak |
+| Unbounded label cardinality (user_id as metric tag) | Cost explosion |
+| Same severity for everything (all `info`) | No alerting signal |
+| Tracing in dev only | Prod is where you need it |
+| Sentry without scrubber | PII in error reports |
+| Audit log in same stream as app log | Tamper risk, retention conflict |
+
+## See Also
+
+- `secrets-management` — what NEVER to log
+- `error-handling` — Result types and error taxonomy
+- `security-baseline` — A09: Security Logging Failures

package/stacks/_shared/skills/performance-patterns/SKILL.md

@@ -1,3 +1,8 @@
+---
+name: performance-patterns
+version: 1.0.0
+---
+
 # Performance Patterns
 
 **Invoke when app is slow or needs optimization.**

package/stacks/_shared/skills/playwright-automation/SKILL.md

@@ -1,3 +1,8 @@
+---
+name: playwright-automation
+version: 1.0.0
+---
+
 # Playwright Automation — E2E Testing
 
 **ALWAYS invoke when writing E2E tests or browser automation.**

package/stacks/_shared/skills/quality-gate/SKILL.md

@@ -1,3 +1,8 @@
+---
+name: quality-gate
+version: 1.0.0
+---
+
 # Quality Gate — Verification System
 
 **ALWAYS invoke BEFORE any commit.**

package/stacks/_shared/skills/research-cache/SKILL.md

@@ -1,3 +1,8 @@
+---
+name: research-cache
+version: 1.0.0
+---
+
 # Research Cache — Best Practices Storage
 
 **ALWAYS invoke BEFORE any web research.**

package/stacks/_shared/skills/secrets-management/SKILL.md

@@ -0,0 +1,245 @@
+---
+name: secrets-management
+version: 1.0.0
+description: Environment variable hygiene, secret detection, rotation, and secret-store patterns. Invoke whenever .env, secrets, API keys, or env-var-reading code are touched.
+---
+
+# Secrets Management
+
+**ALWAYS invoke when touching `.env`, `process.env`, `os.environ`, secrets, API keys, or auth credentials.**
+
+## Core Rules
+
+1. **No secrets in code, ever.** Not in tests, not in fixtures, not in comments.
+2. **No secrets in logs.** Redact before logging.
+3. **No secrets in URLs / query strings.** Headers or body only — URLs land in proxy logs.
+4. **No secrets in error messages** returned to clients.
+5. **`.env` is in `.gitignore`. `.env.example` is committed.** No exceptions.
+6. **Rotate after any leak.** Even suspected. The risk window is the time it takes to rotate, not when you think the leak started.
+
+---
+
+## File Layout
+
+```
+.env                  # gitignored, real values, local dev
+.env.example          # committed, placeholders only
+.env.production       # NEVER committed; use platform secret store
+.env.test             # gitignored if real keys; committed if all-fake
+```
+
+`.gitignore`:
+```
+.env
+.env.*
+!.env.example
+!.env.test.example
+```
+
+`.env.example` template:
+```bash
+# Application
+NODE_ENV=development
+APP_URL=http://localhost:3000
+
+# Database
+DATABASE_URL=postgres://user:pass@localhost:5432/dbname
+
+# Auth
+JWT_SECRET=<generate: openssl rand -base64 32>
+SESSION_SECRET=<generate: openssl rand -base64 32>
+
+# Third-party (use service prefix to scan with allowlists)
+STRIPE_SECRET_KEY=sk_test_...
+OPENAI_API_KEY=sk-...
+```
+
+---
+
+## Reading Env Vars Safely
+
+### Node.js / TypeScript
+```ts
+// Validate at boot — fail fast if missing
+import { z } from 'zod';
+
+const Env = z.object({
+  NODE_ENV: z.enum(['development', 'test', 'production']),
+  DATABASE_URL: z.string().url(),
+  JWT_SECRET: z.string().min(32),
+  STRIPE_SECRET_KEY: z.string().startsWith('sk_'),
+});
+
+export const env = Env.parse(process.env);
+// Now `env.JWT_SECRET` is typed and guaranteed present
+```
+
+Bracket notation only (`tsconfig` strict):
+```ts
+process.env['JWT_SECRET']   // CORRECT
+```
+
+### Python
+```python
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+class Settings(BaseSettings):
+    model_config = SettingsConfigDict(env_file=".env", extra="forbid")
+    database_url: str
+    jwt_secret: str
+    stripe_secret_key: str
+
+settings = Settings()  # raises if missing/invalid
+```
+
+### PHP / Laravel
+```php
+// config/services.php — read once, cached via `php artisan config:cache`
+return [
+    'stripe' => ['secret' => env('STRIPE_SECRET_KEY')],
+];
+// Use config('services.stripe.secret') in app code — NEVER env() at runtime
+```
+
+---
+
+## Public vs Server-Only Vars
+
+| Stack | Public prefix | Rule |
+|---|---|---|
+| Next.js | `NEXT_PUBLIC_` | Anything with this prefix is shipped to the browser |
+| Vite | `VITE_` | Same — bundled into JS |
+| CRA | `REACT_APP_` | Same |
+
+`security-rules.json` enforces: nothing matching `*SECRET|*TOKEN|*PRIVATE|*PASSWORD|*CREDENTIAL` may have a public prefix.
+
+---
+
+## Secret Stores (Production)
+
+| Tool | When |
+|---|---|
+| **Vercel/Netlify env vars** | Simple deployments |
+| **AWS Secrets Manager** / **GCP Secret Manager** / **Azure Key Vault** | Cloud-native, IAM-scoped |
+| **Doppler** | Multi-environment sync |
+| **HashiCorp Vault** | On-prem / self-hosted, dynamic creds |
+| **SOPS + age** | Git-encrypted secrets (GitOps) |
+| **1Password Connect / op-cli** | Team-shared dev secrets |
+
+**Rule:** `.env.production` is **never** committed. Production secrets live in the platform store and are injected at deploy/runtime.
+
+---
+
+## Detection — Block Before Commit
+
+### gitleaks (recommended)
+
+Install:
+```bash
+brew install gitleaks       # macOS
+# or: docker run --rm -v $(pwd):/repo zricethezav/gitleaks:latest detect -s /repo
+```
+
+Pre-commit hook:
+```bash
+gitleaks protect --staged --redact --verbose
+```
+
+CI step:
+```yaml
+- name: Gitleaks scan
+  uses: gitleaks/gitleaks-action@v2
+```
+
+### Quick grep (as a fallback)
+
+```bash
+# Block obvious patterns in staged diff
+git diff --cached -U0 | grep -nEi \
+  '(api[_-]?key|secret|token|bearer|password|aws_(access|secret)|private_key)\s*[:=]\s*["'\''][a-zA-Z0-9/+=_-]{16,}'
+```
+
+---
+
+## Rotation Playbook
+
+When a secret leaks (or is suspected):
+
+1. **Revoke** at the provider immediately. Do not wait for git history rewrites.
+2. **Issue new secret**, deploy with new value.
+3. **Update audit log** — what leaked, when, who, scope of access.
+4. **Scan history**: `gitleaks detect --log-opts="--all"` — find every commit/branch that contained it.
+5. **History rewrite is optional** — git filter-repo / BFG only if you also rotate. The leaked secret in history is already gone from the world's POV once rotated.
+6. **Notify** — if customer data was reachable with the leaked secret, follow incident-disclosure policy.
+
+Token lifetimes (default targets):
+- Access token: ≤ 15 min
+- Refresh token: ≤ 7 days, rotated on use
+- Service-to-service token: ≤ 90 days, rotated automatically
+- Long-lived (e.g. cron API keys): document expiry, calendar reminder
+
+---
+
+## Logging Without Leaks
+
+```ts
+// Redact known fields before logging
+const REDACT = ['password', 'token', 'authorization', 'cookie', 'secret', 'apiKey', 'creditCard'];
+
+function safeLog(obj: unknown) {
+  return JSON.parse(JSON.stringify(obj, (key, val) =>
+    REDACT.some(r => key.toLowerCase().includes(r.toLowerCase())) ? '[REDACTED]' : val
+  ));
+}
+
+logger.info({ event: 'login_attempt', body: safeLog(req.body) });
+```
+
+Use `pino-noir`, `winston`'s `format.printf` filter, or your APM's redaction. Same for Python: `structlog` processors; PHP: Monolog `RedactProcessor`.
+
+See `observability` skill for full structured-logging setup.
+
+---
+
+## Supply-Chain Hygiene
+
+| Stack | Audit |
+|---|---|
+| Node.js | `npm audit --audit-level=high`, `bun audit`, `pnpm audit` |
+| Python | `pip-audit`, `safety check` |
+| PHP | `composer audit` |
+
+Lockfile rules:
+- **Always commit** `package-lock.json` / `bun.lock` / `pnpm-lock.yaml` / `poetry.lock` / `uv.lock` / `composer.lock`.
+- **Renovate** or **Dependabot** for automated PRs.
+- Pin major versions; allow minor/patch auto-merge after CI passes.
+
+---
+
+## FORBIDDEN
+
+| Pattern | Reason |
+|---|---|
+| `.env` committed | Trivial leak |
+| Secret in `README.md` / docstring | Leaks via search engines |
+| Secret in test fixtures | Leaks via PR diffs / forks |
+| Secret in `console.log` / `print` / `Log::info` | Ends up in CloudWatch / Datadog forever |
+| Secret in URL query string | Logged by every proxy in the chain |
+| Secret in commit message | Cannot delete, history rewrites costly |
+| `process.env.SECRET ?? "fallback-value"` with real fallback | Hardcoded backup secret |
+| Sharing via Slack/email | Use 1Password / Vault sharing |
+
+## Pre-Commit Checklist
+
+- [ ] `.env` in `.gitignore`
+- [ ] `.env.example` updated when new var added
+- [ ] gitleaks (or fallback grep) clean
+- [ ] No secret in code, tests, fixtures, comments, logs
+- [ ] No `NEXT_PUBLIC_*SECRET|*TOKEN|*PRIVATE` patterns
+- [ ] Env vars validated at boot (Zod / Pydantic / config())
+
+## See Also
+
+- `security-baseline` — broader OWASP scope
+- `observability` — log redaction details
+- Stack `api-security-*` — usage patterns