@gravito/zenith 0.1.0-beta.1 → 1.0.0-beta.1

package/ALERTING_GUIDE.md

@@ -0,0 +1,71 @@
+# 🔔 Zenith Alerting Guide
+
+This guide explains how to configure and manage the alerting system in Zenith to ensure your infrastructure and queues remain healthy.
+
+---
+
+## 🚀 Overview
+
+Zenith's alerting engine is **Redis-Native** and **Stateless**.
+*   **Persistence**: Rules are stored in Redis (`gravito:zenith:alerts:rules`).
+*   **Evaluation**: The server evaluates all rules every 2 seconds against real-time metrics.
+*   **Delivery**: Alerts are dispatched via Slack Webhooks.
+
+---
+
+## 🛠️ Configuration Fields
+
+When adding a new rule in **Settings > Alerting**, you will encounter these fields:
+
+### 1. Rule Name
+A descriptive label for the alert (e.g., `Critical Backlog`, `Agent Offline`). This name will appear in the Slack notification.
+
+### 2. Type (Metric Category)
+*   **Queue Backlog**: Monitors the number of jobs in the `waiting` state.
+*   **High Failure Count**: Monitors the number of jobs in the `failed` state.
+*   **Worker Loss**: Monitors the total number of active worker nodes.
+*   **Node CPU (%)**: Monitors process-level CPU usage reported by Quasar Agents.
+*   **Node RAM (%)**: Monitors process-level RAM usage (RSS) relative to system total.
+
+### 3. Threshold
+The numeric value that triggers the alert.
+*   For **Backlog/Failure**: The number of jobs (e.g., `1000`).
+*   For **CPU/RAM**: The percentage (e.g., `90`).
+*   For **Worker Loss**: The *minimum* number of workers expected (e.g., alert triggers if count is `< 2`).
+
+### 4. Cooldown (Minutes)
+**Crucial Concept**: The period the system "stays silent" after an alert is fired.
+*   **Logic**: Once a rule triggers and sends a notification, it enters a "lock" state for the duration of the cooldown.
+*   **Purpose**: Prevents "Alert Fatigue" and notification storms.
+*   **Example**: If set to `30`, and a backlog spike occurs, you get **one** notification. You won't get another one for the same rule for 30 minutes, even if the backlog remains high.
+
+### 5. Queue (Optional)
+Specify a specific queue name (e.g., `orders`, `emails`) to monitor. If left empty, the rule applies to the **total sum** of all queues.
+
+---
+
+## 🌊 Best Practices
+
+### The "Instant Fire" Design
+Zenith alerts are designed for **instant awareness**.
+*   If a threshold is met during a 2-second check, the alert fires **immediately**.
+*   It does **not** wait for the condition to persist for multiple minutes (Debouncing).
+*   **Pro Tip**: If you have frequent "tiny spikes" that resolve themselves in seconds, set your **Threshold** slightly higher than the spikes to avoid noise.
+
+### Recommended Settings
+
+| Scenario | Type | Threshold | Cooldown |
+| :--- | :--- | :--- | :--- |
+| **Critical Failure** | High Failure Count | 50 | 15m |
+| **System Overload** | Node CPU | 90 | 30m |
+| **Quiet Hours** | Queue Backlog | 5000 | 120m |
+| **Fatal Shutdown** | Worker Loss | 1 | 10m |
+
+---
+
+## 🔗 Slack Integration
+To receive notifications, ensure the `SLACK_WEBHOOK_URL` environment variable is set before starting the Zenith server.
+
+```bash
+export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/Txxx/Bxxx/Xxxx
+```

package/QUASAR_MASTER_PLAN.md

@@ -0,0 +1,137 @@
+# 🌌 Project Quasar: Master Implementation Plan
+
+**Version**: 1.0.0 (Unified)
+**Target**: Zenith v1.0
+**Context**: This document supersedes all previous "Pulse" plans. It is the single source of truth for the Quasar monitoring ecosystem.
+
+---
+
+## 1. Vision & Identity
+
+**Quasar** is the comprehensive observability layer for the Gravito ecosystem. It unifies infrastructure monitoring (CPU/RAM), application insights (Queues/Slow Logs), and availability checks into a single stream.
+
+> **Slogan**: *"The brightest signal in your infrastructure."*
+
+---
+
+## 2. Architecture & Deployment Matrix
+
+We employ a "Right Tool for the Job" strategy for deployment:
+
+| Ecosystem | Tool | Package | Strategy |
+| :--- | :--- | :--- | :--- |
+| **Node.js / Bun** | **SDK** | `@gravito/quasar` | **In-App Integration**. Directly imports into the app. Captures Event Loop, Heap, and Queues. |
+| **Legacy / Polyglot** | **Agent** | `gravito/quasar-agent` | **Sidecar / Daemon**. Standalone Go binary. Captures OS-level metrics and external Queue states via Redis/API. |
+
+### 🚀 Deployment Methods (Zero Friction)
+1.  **NPM**: `npm install @gravito/quasar` (For Node developers)
+2.  **Docker**: `image: gravito/quasar-agent:latest` (For Container/K8s/Laravel Sail)
+3.  **Shell**: `curl -sL get.gravito.dev/quasar | bash` (For Bare Metal/VM)
+
+---
+
+## 3. Data Protocol (The Quasar Schema)
+
+All agents/SDKs report to Redis using this unified schema.
+
+**Namespace**: `gravito:quasar:`
+
+### A. Heartbeat (Infrastructure)
+*   **Key**: `gravito:quasar:node:{service_name}:{node_id}`
+*   **TTL**: 30 seconds
+*   **Metrics Philosophy**: Report **BOTH** Process and System metrics to isolate resource usage.
+    *   `process`: metrics for the specific service (RAM usage, CPU time).
+    *   `system`: metrics for the host OS (Load avg, Total RAM).
+
+### B. Queues (Workload)
+*   **Key**: `gravito:quasar:queues:{service_name}`
+*   **TTL**: 30 seconds
+*   **Purpose**: Snapshots of queue depths from various drivers.
+    *   Gravito Stream (Native)
+    *   Laravel Horizon (Redis)
+    *   BullMQ (Redis)
+    *   AWS SQS (API)
+
+### C. Insights (Performance)
+*   **Key**: `gravito:quasar:slow:{service_name}` (Stream)
+*   **Purpose**: Log requests or jobs that exceed performance thresholds.
+
+---
+
+## 4. Execution Roadmap
+
+### Phase 1: Foundation & Application Monitoring (Pulse Node)
+**Goal**: Establish the basic dashboard and Node.js SDK for monitoring application health (CPU/RAM).
+
+- [x] **Define Schema**: Update `PULSE_SPEC.md` with new Redis key patterns (`gravito:quasar:node:*`) and payload structure.
+- [x] **SDK Update**: Refactor `@gravito/quasar` (formerly pulse-node) to support:
+    - [x] Automatic runtime detection (Node, Bun, Deno).
+    - [x] System/Process split metrics.
+    - [x] Correct Redis namespacing.
+- [x] **Server Update**: Update Zenith's `PulseService` to scan new key patterns.
+- [x] **UI Overhaul**: Redesign `PulsePage` in Zenith:
+    - [x] Implement "Card" layout for nodes.
+    - [x] Rich metrics visualization (CPU/RAM split bars).
+    - [x] Add brand icons for runtimes (Node, Bun, Deno, PHP, Go, Python).
+    - [x] **Layout Optimization**: Compact Grid for Service Groups.
+
+---
+
+### Phase 2: Architecture Evolution - "The Brain-Hand Model" 🧠 🖐️
+To support advanced features like **Queue Insights** (Phase 2) and **Remote Control** (Phase 3), we are adopting a bidirectional architecture.
+
+*   **Metric Transport (The Mouth)**: Agent sends metrics to Zenith (via shared Redis).
+*   **Local Insight (The Eyes)**: Agent inspects *its own* environment (Local Redis, Local Queue) to gather data. Zenith doesn't need to connect to the App DB directly.
+*   **Command execution (The Hand)**: Zenith publishes commands (Retry/Delete), and Agent listens and executes them locally.
+
+#### Revised Phase 2: Application Insights (Queues) - **In Progress** 🟡
+**Goal**: Enable Quasar Agent to "see" local queues and report their status.
+
+- [x] **SDK Architecture**: Update `QuasarAgent` to handle **Dual Connections**:
+    - `transport`: Connection to Zenith (for sending heartbeat).
+    - `app`: Connection to Local App (for inspecting queues/bull/laravel).
+- [x] **Probe Implementation**: Create `QueueProbe` interface and implementations:
+    - `RedisListProbe`: Simple `LLEN` checks.
+    - [x] `BullProbe` (Future): Check `bull:*:waiting`, etc.
+    - [x] `LaravelProbe`: Check `queues:default`, `queues:reserved`, `queues:delayed`.
+- [x] **SDK API**: Expose `.monitorQueue(name, type)` method.
+- [x] **UI Update**: Update `NodeCard` to render a "Queues" section if queue data is present in payload.
+
+### Phase 3: Remote Control (Command & Control) - **Completed** ✅
+**Goal**: Allow Zenith to instruct Quasar to perform actions (Retry Job, Delete Job).
+
+- [x] **Protocol**: Define Command Protocol (Redis Pub/Sub: `gravito:quasar:cmd:{service}:{node_id}`).
+- [x] **Agent**: Implement `CommandListener` in SDK.
+- [x] **Command Executors**: Implement `RetryJobExecutor` and `DeleteJobExecutor`.
+- [x] **Security (Allowlist Strategy)**:
+    - [x] Implement **Command Allowlist** inside Agent code (only `RETRY_JOB`, `DELETE_JOB` allowed).
+    - [ ] (Future) Use **Redis ACL** (v6+) to restrict Agent's `transport` connection.
+- [x] **Server**: Add `CommandService` and `/api/pulse/command` endpoint.
+- [x] **UI**: Add "Retry/Delete" buttons in Zenith `PulsePage` for failed queue jobs.
+- [x] **Documentation**: Created `ALERTING_GUIDE.md` for configuration best practices.
+
+### Phase 4: Polyglot Agent - **In Progress** 🟡
+*   [x] Create `gravito-framework/quasar` repo (`quasar-go`).
+*   [x] Develop Go Agent core (utilizing `gopsutil`).
+    *   [x] System Probe (CPU/RAM)
+    *   [x] Agent heartbeat loop
+    *   [x] Config management (env vars)
+*   [x] Implement Queue Monitoring in Go Agent:
+    *   [x] Redis List Probe
+    *   [x] Laravel Queue Probe
+*   [x] Implement Remote Control in Go Agent:
+    *   [x] Command Listener (Pub/Sub)
+    *   [x] RETRY_JOB / DELETE_JOB Executors
+    *   [x] **Laravel Deep Integration**:
+        *   [x] `LARAVEL_ACTION` Executor (runs `artisan` safely).
+        *   [x] Auto-discovery of Laravel project root via process inspection.
+        *   [x] Support for `retry-all`, `retry {id}`, and `restart` (graceful worker reload).
+*   [x] Docker & Makefile setup.
+*   [x] Binary Release pipeline (GitHub Actions).
+*   [x] Publish to Docker Hub (`carllee/quasar-go-agent`).
+
+---
+
+## 5. Security & Access
+*   **Auth**: Agents authenticate via a shared secret (`QUASAR_TOKEN`) if writing to a remote Redis.
+*   **Isolation**: Process metrics only report what they have access to. System metrics require readable `/proc` (in Docker).