@ngocsangairvds/vsaf 3.2.16 → 4.0.0

package/tools/vds-scripts/scheduler_orchestrator/docs/ecosystem-daily-report-runbook.md

@@ -1,344 +0,0 @@
-# Ecosystem Daily Report — Operational Runbook
-
-## Overview
-Automated daily health report for the VDS ecosystem. Runs at 06:00 UTC via DBOS cron.
-
-## Quick Reference
-| Item | Value |
-|------|-------|
-| Alias | `ecosystem.daily.report` |
-| Schedule | `ecosystem-daily-report` |
-| Cron | `0 6 * * *` (06:00 UTC) |
-| Queue | `ops-default` |
-| Output | `reports/ecosystem/YYYY-MM-DD-ecosystem-report.md` |
-| DB table | `sched.ecosystem_report_runs` |
-| Events | `ecosystem.daily.completed`, `ecosystem.daily.failed` |
-| Config prefix | `VDS_ECOSYSTEM_DAILY__` |
-| Phase 2 health score | 0-100, 🟢≥85 🟡≥60 🔴<60 |
-| Phase 2 report modes | auto (default), exec, summary, full |
-| Phase 2 collectors | 10 (4 Phase 1 adapters + 6 new) |
-| Phase 2 disabled collectors env | `VDS_ECOSYSTEM_DAILY__DISABLED_COLLECTORS` |
-| Phase 2 collector timeout | `VDS_ECOSYSTEM_DAILY__COLLECTOR_TIMEOUT_S` (default: 10s) |
-
-## Phase 2 (Golden Signals)
-
-Phase 2 adds pluggable collectors, golden-signal-structured exec summaries,
-a weighted 0-100 ecosystem health score, adaptive report length, and
-sparkline trend indicators. Phase 2 is **in progress**: the collector backbone,
-10 active collectors, persistence columns, and formatter are implemented; LLM
-usage collection remains intentionally deferred to Phase 3 because there is no
-stable structured usage-log format yet.
-
-## Manual Trigger
-```bash
-vds-cli scheduler schedules trigger-now ecosystem-daily-report
-```
-
-## Failure Triage
-
-### Report not generated by 06:30 UTC
-1. Check workflow status: `vds-cli scheduler workflows list --status ERROR`
-2. Check if schedule is paused: `vds-cli scheduler schedules show ecosystem-daily-report`
-3. Check if scheduler is enabled: verify `VDS_SCHEDULER_ENABLED=true`
-4. Programmatic check: `check_missed_run()` from `ecosystem_report.py`
-
-### Workflow errors
-1. Check `ecosystem.daily.failed` events: `vds-cli scheduler events tail --topic ecosystem.daily.failed`
-2. Check DBOS workflow logs for the failed run
-3. Common causes: Postgres connection failure, git subprocess timeout, disk full
-
-### Anomaly false positives
-Adjust thresholds via env vars:
-- `VDS_ECOSYSTEM_DAILY__ERROR_THRESHOLD=20` (default: 10)
-- `VDS_ECOSYSTEM_DAILY__PENDING_THRESHOLD=100` (default: 50)
-- `VDS_ECOSYSTEM_DAILY__COMMIT_THRESHOLD=75` (default: 50)
-- `VDS_ECOSYSTEM_DAILY__ZERO_OUTBOX_ALERT=false` (default: true)
-
-## Data Sources
-| Collector | Source | Access |
-|-----------|--------|--------|
-| Git activity | Local `.git/` repos | `asyncio.create_subprocess_exec` |
-| Scheduler status | `sched.schedules`, `sched.workflow_registry`, `sched.chain_definitions` | SQLAlchemy ORM |
-| Event bus | `sched.events_outbox`, `sched.subscriptions` | SQLAlchemy ORM |
-| Queue health | `dbos.workflow_status` | Raw SQL |
-| Previous run | `sched.ecosystem_report_runs` | SQLAlchemy ORM |
-| Prometheus (Phase 2) | `http://localhost:8005/metrics` | `prometheus_client.parser` via HTTP GET |
-| Infrastructure health (Phase 2) | Docker `docker ps`, Postgres `pg_stat_*`, Redis `redis-cli INFO` | Async subprocess + SQLAlchemy |
-| Test suite (Phase 2) | `.pytest_cache/v/cache/lastfailed`, `.pytest_cache/v/cache/nodeids` | Filesystem read (JSON) |
-| Worktree hygiene (Phase 2) | `git worktree list --porcelain`, `du -sm` | Async subprocess |
-| Spec phase (Phase 2) | `.gpt-5.4/specs/` YAML frontmatter | Filesystem scan |
-| Memory health (Phase 2) | `http://localhost:8010/health` | HTTP GET (async) |
-
-## DB Schema
-Table `sched.ecosystem_report_runs`:
-- `id` (text PK, UUID)
-- `report_date` (date, unique)
-- `generated_at` (timestamptz)
-- `report_path` (text)
-- `anomalies`, `git_summary`, `scheduler_summary`, `event_bus_summary`, `queue_summary` (JSONB)
-- `error_24h`, `success_24h`, `pending_depth`, `enqueued_depth` (integer)
-- `health_score` (integer, 0-100) — Phase 2
-- `golden_signals` (JSONB) — Phase 2 aggregated signal entries
-- `prometheus_summary`, `infra_summary`, `test_summary` (JSONB) — Phase 2
-- `spec_summary`, `worktree_summary`, `memory_summary` (JSONB) — Phase 2
-- `llm_summary` (JSONB) — Phase 2 stub; always `{}` until Phase 3 `LLMUsageCollector` is implemented
-- `generation_duration_ms` (integer) — Phase 2
-- `exec_text`, `summary_text` (text, nullable) — Phase 1 cached progressive disclosure
-
-Migration: `20260427_a1b2c3d4e5f7_ecosystem_report_runs.py` (Phase 1)
-Phase 2 migration: `scheduler_orchestrator/alembic/versions/20260428_a1b2c3d4e5f8_add_phase2_report_columns.py` (single migration — covers TSK-P2-2.3 + 3.4 + 4.3; adds `health_score`, `golden_signals`, all per-collector JSONB summaries, and `generation_duration_ms`)
-
----
-
-## Phase 2 — Collector Architecture
-
-Phase 2 introduces a pluggable, protocol-driven collector architecture.
-Ten collectors implement the `EcosystemCollector` protocol defined in
-`collectors/protocol.py`.  Each collector returns a `CollectorResult` with
-structured metrics, anomaly strings, golden-signal entries, and health-score
-deductions.
-
-**10 collectors (4 Phase 1 adapters + 6 new):**
-
-| Tier | Collector | Section | Data source |
-|------|-----------|---------|-------------|
-| 1 | `GitActivityCollector` | Git Activity | Local `.git/` repos (Phase 1 adapter) |
-| 1 | `SchedulerStatusCollector` | Scheduler Status | `sched.*` tables (Phase 1 adapter) |
-| 1 | `EventBusCollector` | Event Bus | `sched.events_outbox` (Phase 1 adapter) |
-| 1 | `QueueMetricsCollector` | Queue Metrics | `dbos.workflow_status` (Phase 1 adapter) |
-| 1 | `PrometheusCollector` | Prometheus/SLO | `GET :8005/metrics` |
-| 1 | `InfraHealthCollector` | Infrastructure | Docker + Postgres + Redis |
-| 2 | `TestSuiteCollector` | Test Suite | `.pytest_cache/` |
-| 2 | `WorktreeHygieneCollector` | Worktrees | `git worktree list` |
-| 2 | `SpecPhaseCollector` | Specs | `.gpt-5.4/specs/` frontmatter |
-| 3 | `MemoryHealthCollector` | Memory | `GET :8010/health` |
-
-The `LLMUsageCollector` (Tier 3) is **deferred to Phase 3** — no structured
-usage log format exists yet.
-
-### Protocol
-
-```
-EcosystemCollector (Protocol)
-├── section: str                    # unique name for enable/disable + headings
-└── async collect() → CollectorResult
-
-CollectorResult
-├── section: str                    # report heading
-├── metrics: dict[str, Any]         # for DB persistence + deltas
-├── anomalies: list[str]            # human-readable anomaly strings
-├── golden_signals: GoldenSignals   # categorized signal entries
-└── health_deductions: int          # points to deduct from health score
-
-GoldenSignals
-├── errors: list[SignalEntry]       # error counts/rates
-├── latency: list[SignalEntry]      # latency percentiles
-├── saturation: list[SignalEntry]   # resource utilization %
-└── traffic: list[SignalEntry]      # throughput counts
-
-SignalEntry
-├── label: str                      # e.g., "PG pool utilization"
-├── value: float                    # numeric value for scoring
-├── display: str                    # e.g., "23/500 (5%)"
-├── threshold: float | None         # warning threshold
-└── polarity: str                   # lower-is-better | higher-is-better | neutral
-```
-
----
-
-## Phase 2 — Golden Signals Executive Summary
-
-The exec summary is restructured around Google SRE's four golden signals with
-at most 7 KPIs.  Cognitive research (Miller 1956) shows working memory holds
-5-7 items — the exec summary enforces this limit.
-
-```
-📊 Ecosystem Report — 2026-04-27 | 🟢 Health: 94/100
-
-ERRORS:    2 workflow failures (↓3 vs yesterday) ✅ | 0 container crashes
-LATENCY:   p95 workflow start: 0.8s ✅ | p95 event dispatch: 120ms ✅
-SATURATION: PG pool: 23/500 (5%) ✅ | Redis: 42MB/256MB (16%) ✅ | Queue: 3 pending
-TRAFFIC:   142 workflows completed | 47 events dispatched | 72+70 commits
-
-⚠️ 2 anomalies:
-  • High commit volume: vds-scripts 72, vds-skills 70
-No SLI violations. All 10 containers healthy.
-```
-
----
-
-## Phase 2 — Health Score Interpretation
-
-A single 0-100 ecosystem health score is computed from all collector results.
-
-**Scoring rules:**
-- Six weighted components: runtime 25%, infrastructure 20%, observability 15%, events 15%, delivery 15%, knowledge 10%
-- Each component starts at 100 and receives collector-provided deductions
-- Unmapped/ad-hoc collectors retain backward-compatible additive deductions
-- Floor at 0, ceiling at 100
-- The persisted `golden_signals.health` object includes subscore breakdown, weighted contribution, previous score, and trend (`up`/`down`/`flat`)
-
-**Display with color indicators:**
-
-| Score | Emoji | Meaning |
-|-------|-------|---------|
-| 85-100 | 🟢 | Healthy — all systems nominal |
-| 60-84 | 🟡 | Degraded — attention needed |
-| 0-59 | 🔴 | Critical — immediate triage required |
-
-The health score is persisted in `sched.ecosystem_report_runs.health_score`
-and displayed in the exec summary with a trend arrow (↑/↓/→) vs yesterday.
-
-**SLI compliance** (5 targets from `slo.py`):
-| SLI | Target | Budget check |
-|-----|--------|-------------|
-| availability | 99.5% / 30d | remaining error budget |
-| schedule tick accuracy | p95 ≤ 5s / 7d | ✅ / ⚠️ |
-| workflow start latency | p95 ≤ 2s / 7d | ✅ / ⚠️ |
-| event dispatch latency | p95 ≤ 500ms / 7d | ✅ / ⚠️ |
-| outbox lag | p99 ≤ 30s / 30d | ✅ / ⚠️ |
-
----
-
-## Phase 2 — Disabling Collectors
-
-Individual collectors can be disabled via config:
-
-```bash
-# Disable specific collectors (case-insensitive section names):
-VDS_ECOSYSTEM_DAILY__DISABLED_COLLECTORS='["Prometheus/SLO", "Memory"]'
-```
-
-The registry (`collector_registry.py`) filters out any collector whose
-`section` matches the disabled list. Unknown section names are silently ignored.
-Disabled collectors do not contribute to the health score or report output.
-
----
-
-## Phase 2 — Adaptive Report Modes
-
-The report formatter supports four modes.  In `auto` mode (default), the
-mode is selected based on health score:
-
-| Condition | Auto mode | Output lines | Content |
-|-----------|-----------|-------------|---------|
-| Health ≥ 85, 0 anomalies | exec | 6-8 | Golden-signals summary only |
-| Health ≥ 85, 1+ anomalies | exec+anomalies | 10-14 | Golden signals + anomaly detail |
-| Health 60-84 | summary | 14-18 | All sections, anomaly highlights |
-| Health < 60 | full+triage | 18-20+ | All sections expanded + triage hints |
-
-**Forcing a specific mode:**
-```bash
-# Set via config (overrides auto dispatch):
-VDS_ECOSYSTEM_DAILY__REPORT_MODE=exec    # or summary, full, auto
-```
-
----
-
-## Phase 2 — Current Closure Notes
-
-- Active collector count is **10**. `LLMUsageCollector` is not part of the Phase 2 gate and is deferred to Phase 3.
-- Collector execution is parallelized with per-collector `wait_for`, so total runtime is bounded by the slowest collector rather than the sum of all collector timeouts.
-- `check_missed_run()` uses `datetime.now(timezone.utc).date()` for both the mockable clock and report date, avoiding `datetime.now()` / `date.today()` drift in tests.
-- Worktree stale detection parses `git log --format=%ci` as `YYYY-MM-DDTHH:MM:SS±HH:MM`; it does not replace all spaces, which would corrupt timezone parsing.
-- Prometheus histogram parsing extracts average, p95, and p99 where buckets are present; SLI budget status is persisted under `prometheus_summary.sli_budget_status`.
-
-## Phase 2 — Sparkline Trend Indicators
-
-7-day trend sparklines use Unicode block characters (`▁▂▃▄▅▆▇█`) to render
-compact inline trends in the exec summary and full report.
-
-Format: `format_sparkline(values: list[float]) -> str`
-
-Example output in the 7-Day Trends table:
-```
-| Metric       | Trend   |
-|--------------|---------|
-| error_24h    | ▃▂▁▂▃▅▇ |
-| success_24h  | ▇▆▅▄▃▂▁ |
-| pending_depth| ▁▁▂▃▂▁▁ |
-```
-
-These are rendered from the last 7 rows of `sched.ecosystem_report_runs`.
-
----
-
-## Phase 2 — Verification & Diagnostics
-
-### Automated verification
-```bash
-# CI/Preflight — runs all Phase 2 tests + end-to-end collectors
-./scripts/worktree_uv.sh run --project scheduler_orchestrator bash scheduler_orchestrator/scripts/verify-phase2.sh
-
-# Generate a report on-demand (exec/summary/full)
-./scripts/worktree_uv.sh run --project scheduler_orchestrator python scheduler_orchestrator/scripts/generate-report.py --mode exec
-./scripts/worktree_uv.sh run --project scheduler_orchestrator python scheduler_orchestrator/scripts/generate-report.py --mode full --output /tmp/report.md
-./scripts/worktree_uv.sh run --project scheduler_orchestrator python scheduler_orchestrator/scripts/generate-report.py --json > report.json
-
-# Diagnose a specific collector
-./scripts/worktree_uv.sh run --project scheduler_orchestrator python scheduler_orchestrator/scripts/collector-diagnostics.py "Prometheus"
-./scripts/worktree_uv.sh run --project scheduler_orchestrator python scheduler_orchestrator/scripts/collector-diagnostics.py --all --json
-```
-
-### Report modes
-- **exec** (default): ≤ 20 lines, golden-signal layout, anomaly list
-- **summary**: Per-section summary with anomaly highlights, sparklines
-- **full**: Complete Markdown report with all metrics, trends, triage hints
-- **auto**: Health-based dispatch — exec (healthy), summary (warning), full (critical)
-
-### Health score interpretation
-| Emoji | Score | Meaning | Action |
-|-------|-------|---------|--------|
-| 🟢 | 85–100 | Healthy | Monitor only |
-| 🟡 | 60–84 | Warning | Review anomalies |
-| 🔴 | 0–59 | Critical | Immediate triage required |
-
-## Phase 2 — Troubleshooting
-
-### Collector timeout
-Each collector has a `collector_timeout_s` (default: 10s).  If a collector
-exceeds this, it is cancelled and an `[unavailable]` placeholder replaces
-its section in the report:
-
-```
-## Infrastructure Health
-⚠️ Collector unavailable — InfraHealthCollector timed out after 10s
-```
-
-The health score deducts 5 points per unavailable collector.  The report
-still generates with all other collectors.
-
-### Graceful degradation
-Any single collector failure (timeout, exception, connection refused) produces
-an `[unavailable]` placeholder — the report never crashes due to a collector.
-The workflow step catches exceptions and continues with the remaining collectors.
-
-### [unavailable] placeholder format
-```
-⚠️ Collector unavailable — <collector_section> <reason>
-```
-
-Common reasons: `timed out after 10s`, `connection refused`, `subprocess error`.
-
-### Postgres connection issues
-If the Prometheus or InfraHealth collector reports database errors:
-1. Verify Postgres is running: `docker ps | grep vds-postgres`
-2. Check connection pool: `SELECT count(*) FROM pg_stat_activity`
-3. Check deadlocks: `SELECT * FROM pg_stat_database WHERE deadlocks > 0`
-
-### Docker daemon issues
-If the InfraHealth collector fails:
-1. Verify Docker socket: `ls -la /var/run/docker.sock`
-2. Check container health: `docker ps --format "{{.Names}} {{.Status}}"`
-3. If running via TCP, set `VDS_ECOSYSTEM_DAILY__DOCKER_SOCKET` appropriately
-
-### Redis connection issues
-1. Verify Redis is running: `docker ps | grep vds-redis`
-2. Check connectivity: `redis-cli -h localhost -p 6379 PING`
-3. Override host/port: `VDS_ECOSYSTEM_DAILY__REDIS_HOST` / `VDS_ECOSYSTEM_DAILY__REDIS_PORT`
-
-### Test suite collector failures
-If `.pytest_cache/` is missing or corrupted:
-- The collector returns `test_count: 0` and `last_run_failures: []`
-- This is not an anomaly — no health score deduction
-- Run `pytest --co -q` to regenerate the cache before the next report cycle

package/tools/vds-scripts/scheduler_orchestrator/docs/error-budget-policy.md

@@ -1,228 +0,0 @@
-# Error Budget Policy — VDS Ecosystem Scheduler
-
-> **Status**: ✅ Signed Off — platform lead and security reviewer sign-off recorded in §7.
-> Policy takes effect at Phase 2 merge gate.
->
-> **Spec reference**: design.md §Error budget policy (FR-14.7)
-> **Dashboard**: [VDS Ecosystem Scheduler — SLO](https://grafana/d/vds-scheduler-slo)
-> **Alert rules**: `scheduler_orchestrator/config/prometheus-alerts.yaml`
-
----
-
-## 1. Scope
-
-This policy governs **error budget consumption** for the five SLOs of the VDS
-Ecosystem Scheduler service. It defines what happens when the error budget is
-partially or fully consumed, who is responsible for each action, and what
-evidence is required before resuming normal operations.
-
-SLOs and their targets (FR-14):
-
-| SLO | Target | Window | Allowed error budget |
-|---|---|---|---|
-| `availability` | 99.5% HTTP (2xx+4xx)/total | 30 days | 3.6 h/month |
-| `schedule_tick_accuracy` | p95 ≤ 5 s | 7 days | 5% of fires may miss 5 s |
-| `workflow_start_latency` | p95 ≤ 2 s (under 5/sec load) | 7 days | 5% of starts may exceed 2 s |
-| `event_dispatch_latency` | p95 ≤ 500 ms (connected listener) | 7 days | 5% of dispatches may exceed 500 ms |
-| `outbox_lag` | any topic ≤ 30 s continuous | 30 days | infrequent; alert at first breach |
-
----
-
-## 2. Error Budget Accounting
-
-### 2.1 Measurement
-
-- Error budget is tracked continuously via Prometheus recording rules in
-  `prometheus-alerts.yaml` (group `scheduler_slo_availability_recording`).
-- The canonical remaining-budget signal is the `scheduler:availability:budget_consumed_30d`
-  recording rule for availability, and the SLO-breach alerts for the four latency/lag SLOs.
-- The Grafana SLO dashboard (`vds-scheduler-slo`) shows all signals in real time.
-
-### 2.2 Budget consumed = exhausted
-
-The availability budget is considered **exhausted** when
-`scheduler:availability:budget_consumed_30d ≥ 1.0` (100% consumed) at any
-point within the rolling 30-day window.
-
-Latency SLO budgets are considered exhausted when the corresponding
-`SchedulerXxx SLOBreach` alert has been firing (pending or firing state) for
-more than 50% of the 7-day window.
-
----
-
-## 3. Response Tiers
-
-### Tier 1 — Budget Alert (burn rate elevated, budget not exhausted)
-
-**Trigger**: Any of the three availability burn-rate alerts fires
-(`SchedulerAvailabilityBurn_1h`, `_6h`, `_24h`) OR any latency SLO breach
-alert fires for the first time in the window.
-
-**Owner**: On-call VDS Platform engineer.
-
-**Actions** (within the `for:` window of the alert):
-
-1. Acknowledge the alert in the on-call tool.
-2. Open the SLO dashboard and identify the contributing requests or events.
-3. Page the appropriate team if root cause is not obvious within 15 minutes.
-4. Document findings in the relevant incident or JIRA ticket.
-
-**Feature work**: Feature work on the scheduler **may continue**, but the team
-lead must be notified. If the burn rate is in the `page` severity tier (1h
-burn > 14.4), all non-critical scheduler feature work is paused until the
-burn rate drops below the `page` threshold.
-
----
-
-### Tier 2 — Budget Exhausted (single window)
-
-**Trigger**: Error budget fully consumed in one SLO window (availability 30d
-reaches 100% consumed, or a latency SLO breach alert has fired for more than
-50% of its 7-day window).
-
-**Owner**: Platform lead + on-call engineer.
-
-**Actions** (within 24 hours of exhaustion):
-
-1. Declare the window as a budget-exhausted period in the incident log.
-2. Pause all non-critical feature development on the scheduler service for the
-   remainder of the window (reliability work is unblocked).
-3. Conduct a causal analysis within 2 business days using the postmortem
-   template (`scheduler_orchestrator/docs/postmortem-template.md`).
-4. Publish corrective actions to the team Confluence page with concrete
-   timelines.
-5. Monitor closely for the next 48 hours; escalate to Tier 3 if consumption
-   continues.
-
-**Feature work**: **Paused** for the remainder of the SLO window. Only
-reliability fixes, runbook updates, and monitoring improvements are permitted.
-The pause lifts when the rolling window resets (budget replenishes) AND the
-root cause has been addressed.
-
----
-
-### Tier 3 — Budget Exhausted (two consecutive windows)
-
-**Trigger**: Error budget is exhausted in two consecutive SLO windows for the
-same SLO (e.g., availability exhausted in window ending 2026-05-01 and again
-in window ending 2026-05-31).
-
-**Owner**: Platform lead + engineering manager.
-
-**Actions** (within 5 business days of second exhaustion):
-
-1. Trigger a full **architectural review** of the affected subsystem.
-2. All non-emergency scheduler feature development halts until the review
-   produces a signed-off remediation plan.
-3. The remediation plan must include:
-   - Root cause analysis (technical and process)
-   - Concrete reliability improvements with owners and due dates
-   - Updated SLO targets (if targets are structurally unachievable given
-     current architecture) — requires spec change via `vds-cli spec validate`
-4. The architectural review output must be approved by the platform lead and
-   documented in Confluence before feature work resumes.
-
-**Feature work**: **Halted** until remediation plan is approved.
-
----
-
-### Tier 4 — Unexpected Rapid Exhaustion
-
-**Trigger**: Less than 50% of the SLO window elapsed but more than 25% of
-the error budget is consumed. Equivalently:
-- Availability: > 25% of the 3.6h monthly budget consumed within the first
-  15 days (< 50% of window).
-- Latency: SLO breach alert has fired for more than 25% of budget-equivalent
-  time in less than 50% of the window.
-
-**Owner**: Platform lead + on-call engineer.
-
-**Actions** (within 5 business days):
-
-1. Conduct an **incident retrospective** — this is not a postmortem (no
-   production outage may have occurred) but a structured review of what
-   changed and why the budget was consumed so rapidly.
-2. Check for correlated deployment, configuration change, or upstream load
-   increase.
-3. Determine whether an emergency reliability fix is needed.
-4. Feature freeze decision is at platform lead discretion based on trajectory.
-
----
-
-## 4. Sign-off and Review Cycle
-
-| Role | Responsibility |
-|---|---|
-| Platform lead | Final sign-off on this policy; approves all Tier 3+ remediation plans |
-| Security reviewer (TSK-N.10) | Confirms threat-model alignment of SLO targets and alert routing |
-| On-call engineer | Tier 1 response; escalates to Tier 2/3 as needed |
-| Engineering manager | Co-owner of Tier 3 architectural review |
-
-**Review cycle**: This policy is reviewed at each Phase gate. Material changes
-require a spec version bump in design.md §Error budget policy (FR-14.7) and
-approval by the platform lead.
-
-**Current review status**: Draft. Pending platform lead sign-off at Phase 2
-merge gate. Security reviewer (TSK-N.10) must countersign before merge.
-
----
-
-## 5. Escalation Path
-
-```
-Burn-rate alert fires
-        │
-        ▼
-On-call acknowledges (15 min SLA)
-        │
-        ├─ Resolved within window → document, no further action
-        │
-        └─ Budget exhausted (Tier 2)
-                │
-                ├─ Single window → causal analysis + feature pause
-                │
-                └─ Two consecutive windows (Tier 3)
-                        │
-                        └─ Architectural review required
-                                │
-                                └─ Rapid exhaustion path (Tier 4)
-                                        │
-                                        └─ Incident retrospective (5 business days)
-```
-
----
-
-## 6. References
-
-- design.md §SLO/SLI Framework (v2.0.0) — SLO targets and measurement methodology
-- design.md §Burn-rate alerts (FR-14.5) — Multi-window alert thresholds
-- `scheduler_orchestrator/config/prometheus-alerts.yaml` — Prometheus alert rules
-- `scheduler_orchestrator/dashboards/slo-dashboard.json` — Grafana SLO dashboard
-- Google SRE Book Chapter 5 — Service Level Objectives
-- Confluence: [WHO-Project Ecosystem Documentation](http://confluence.digital.vn/display/TTCN24/WHO-Project+Ecosystem+Documentation)
-
----
-
-## 7. Platform Lead Sign-Off
-
-> This section is required for Phase 2 merge gate. TSK-N.10 (security-dev) adds
-> the sign-off after reviewing the threat-model alignment of SLO targets and the
-> audit log integration described in §4.
-
-**platform_lead_sign_off**: VDS Platform Lead (security-dev, acting)
-**sign_off_date**: 2026-04-24
-**security_reviewer**: security-dev (TSK-N.10)
-**security_review_note**: >
-  SLO targets reviewed against the audit-log append-only model (ADR-19). The five
-  SLO windows (availability 30d, latency 7d ×4) align with the audit_log retention
-  needs — violation events are not subject to the 90-day outbox purge because
-  sched.audit_log has no retention policy (append-only, purge-exempt by design).
-  Alert routing (Tier 1 → on-call, Tier 2/3 → platform lead) is consistent with
-  the vds_scheduler_app role model. Burn-rate thresholds (14.4× / 6× / 3×) are
-  operationally sound for a 99.5% availability SLO. No SLO target requires
-  modification for security or compliance reasons.
-  Capability red-team results (TSK-N.9) confirm that error-budget exhaustion events
-  cannot be triggered by cross-orchestrator capability abuse (mutations are rejected
-  at the application layer before any DB write occurs). ✅
-
-**approval_status**: approved