@gravito/zenith 1.1.2 → 1.1.6

package/docs/QUASAR_MASTER_PLAN.md

@@ -1,140 +0,0 @@
-# 🌌 Project Quasar: Master Implementation Plan
-
-**Version**: 1.0.0 (Unified)
-**Target**: Zenith v1.0
-**Context**: This document supersedes all previous "Pulse" plans. It is the single source of truth for the Quasar monitoring ecosystem.
-
----
-
-## 1. Vision & Identity
-
-**Quasar** is the comprehensive observability layer for the Gravito ecosystem. It unifies infrastructure monitoring (CPU/RAM), application insights (Queues/Slow Logs), and availability checks into a single stream.
-
-> **Slogan**: *"The brightest signal in your infrastructure."*
-
----
-
-## 2. Architecture & Deployment Matrix
-
-We employ a "Right Tool for the Job" strategy for deployment:
-
-| Ecosystem | Tool | Package | Strategy |
-| :--- | :--- | :--- | :--- |
-| **Node.js / Bun** | **SDK** | `@gravito/quasar` | **In-App Integration**. Directly imports into the app. Captures Event Loop, Heap, and Queues. |
-| **Legacy / Polyglot** | **Agent** | `gravito/quasar-agent` | **Sidecar / Daemon**. Standalone Go binary. Captures OS-level metrics and external Queue states via Redis/API. |
-| **PHP / Laravel** | **Package** | `gravito/laravel-zenith` | **Native Integration**. Laravel Service Provider. Captures Jobs, Logs, and Exceptions. |
-
-### 🚀 Deployment Methods (Zero Friction)
-1.  **NPM**: `npm install @gravito/quasar` (For Node developers)
-2.  **Docker**: `image: gravito/quasar-agent:latest` (For Container/K8s/Laravel Sail)
-3.  **Shell**: `curl -sL get.gravito.dev/quasar | bash` (For Bare Metal/VM)
-
----
-
-## 3. Data Protocol (The Quasar Schema)
-
-All agents/SDKs report to Redis using this unified schema.
-
-**Namespace**: `gravito:quasar:`
-
-### A. Heartbeat (Infrastructure)
-*   **Key**: `gravito:quasar:node:{service_name}:{node_id}`
-*   **TTL**: 30 seconds
-*   **Metrics Philosophy**: Report **BOTH** Process and System metrics to isolate resource usage.
-    *   `process`: metrics for the specific service (RAM usage, CPU time).
-    *   `system`: metrics for the host OS (Load avg, Total RAM).
-
-### B. Queues (Workload)
-*   **Key**: `gravito:quasar:queues:{service_name}`
-*   **TTL**: 30 seconds
-*   **Purpose**: Snapshots of queue depths from various drivers.
-    *   Gravito Stream (Native)
-    *   Laravel Horizon (Redis)
-    *   BullMQ (Redis)
-    *   AWS SQS (API)
-
-### C. Insights (Performance)
-*   **Key**: `gravito:quasar:slow:{service_name}` (Stream)
-*   **Purpose**: Log requests or jobs that exceed performance thresholds.
-
----
-
-## 4. Execution Roadmap
-
-### Phase 1: Foundation & Application Monitoring (Pulse Node)
-**Goal**: Establish the basic dashboard and Node.js SDK for monitoring application health (CPU/RAM).
-
-- [x] **Define Schema**: Update `PULSE_SPEC.md` with new Redis key patterns (`gravito:quasar:node:*`) and payload structure.
-- [x] **SDK Update**: Refactor `@gravito/quasar` (formerly pulse-node) to support:
-    - [x] Automatic runtime detection (Node, Bun, Deno).
-    - [x] System/Process split metrics.
-    - [x] Correct Redis namespacing.
-- [x] **Server Update**: Update Zenith's `PulseService` to scan new key patterns.
-- [x] **UI Overhaul**: Redesign `PulsePage` in Zenith:
-    - [x] Implement "Card" layout for nodes.
-    - [x] Rich metrics visualization (CPU/RAM split bars).
-    - [x] Add brand icons for runtimes (Node, Bun, Deno, PHP, Go, Python).
-    - [x] **Layout Optimization**: Compact Grid for Service Groups.
-
----
-
-### Phase 2: Architecture Evolution - "The Brain-Hand Model" 🧠 🖐️ - **Completed** ✅
-To support advanced features like **Queue Insights** (Phase 2) and **Remote Control** (Phase 3), we are adopting a bidirectional architecture.
-
-*   **Metric Transport (The Mouth)**: Agent sends metrics to Zenith (via shared Redis).
-*   **Local Insight (The Eyes)**: Agent inspects *its own* environment (Local Redis, Local Queue) to gather data. Zenith doesn't need to connect to the App DB directly.
-*   **Command execution (The Hand)**: Zenith publishes commands (Retry/Delete), and Agent listens and executes them locally.
-
-#### Revised Phase 2: Application Insights (Queues) - **Completed** ✅
-**Goal**: Enable Quasar Agent to "see" local queues and report their status.
-
-- [x] **SDK Architecture**: Update `QuasarAgent` to handle **Dual Connections**:
-    - `transport`: Connection to Zenith (for sending heartbeat).
-    - `app`: Connection to Local App (for inspecting queues/bull/laravel).
-- [x] **Probe Implementation**: Create `QueueProbe` interface and implementations:
-    - `RedisListProbe`: Simple `LLEN` checks.
-    - [x] `BullProbe` (Future): Check `bull:*:waiting`, etc.
-    - [x] `LaravelProbe`: Check `queues:default`, `queues:reserved`, `queues:delayed`.
-- [x] **SDK API**: Expose `.monitorQueue(name, type)` method.
-- [x] **UI Update**: Update `NodeCard` to render a "Queues" section if queue data is present in payload.
-
-### Phase 3: Remote Control (Command & Control) - **Completed** ✅
-**Goal**: Allow Zenith to instruct Quasar to perform actions (Retry Job, Delete Job).
-
-- [x] **Protocol**: Define Command Protocol (Redis Pub/Sub: `gravito:quasar:cmd:{service}:{node_id}`).
-- [x] **Agent**: Implement `CommandListener` in SDK.
-- [x] **Command Executors**: Implement `RetryJobExecutor` and `DeleteJobExecutor`.
-- [x] **Security (Allowlist Strategy)**:
-    - [x] Implement **Command Allowlist** inside Agent code (only `RETRY_JOB`, `DELETE_JOB` allowed).
-    - [ ] (Future) Use **Redis ACL** (v6+) to restrict Agent's `transport` connection.
-- [x] **Server**: Add `CommandService` and `/api/pulse/command` endpoint.
-- [x] **UI**: Add "Retry/Delete" buttons in Zenith `PulsePage` for failed queue jobs.
-- [x] **Documentation**: Created `ALERTING_GUIDE.md` for configuration best practices.
-
-### Phase 4: Polyglot Agent - **Completed** ✅
-*   [x] Create `gravito-framework/quasar` repo (`quasar-go`).
-*   [x] Develop Go Agent core (utilizing `gopsutil`).
-    *   [x] System Probe (CPU/RAM)
-    *   [x] Agent heartbeat loop
-    *   [x] Config management (env vars)
-*   [x] Implement Queue Monitoring in Go Agent:
-    *   [x] Redis List Probe
-    *   [x] Laravel Queue Probe
-*   [x] Implement Remote Control in Go Agent:
-    *   [x] Command Listener (Pub/Sub)
-    *   [x] RETRY_JOB / DELETE_JOB Executors
-    *   [x] **Laravel Deep Integration**:
-        *   [x] `LARAVEL_ACTION` Executor (runs `artisan` safely).
-        *   [x] Auto-discovery of Laravel project root via process inspection.
-        *   [x] **Advanced Process Introspection**: Captures real-time CPU/RAM usage per Laravel Worker process.
-        *   [x] **Virtual Node Mapping**: Visualizes individual Laravel Workers as distinct nodes in Zenith UI.
-        *   [x] Support for `retry-all`, `retry {id}`, and `restart` (graceful worker reload).
-*   [x] Docker & Makefile setup.
-*   [x] Binary Release pipeline (GitHub Actions).
-*   [x] Publish to Docker Hub (`carllee/quasar-go-agent`).
-
----
-
-## 5. Security & Access
-*   **Auth**: Agents authenticate via a shared secret (`QUASAR_TOKEN`) if writing to a remote Redis.
-*   **Isolation**: Process metrics only report what they have access to. System metrics require readable `/proc` (in Docker).

package/docs/QUICK_TEST_GUIDE.md

@@ -1,72 +0,0 @@
-# 🎯 Flux Console 統一測試指南
-
-## 🚀 快速開始
-
-我們已經將所有測試腳本整合為 `seed.ts` 與 `worker.ts`，以下是常用的測試流程。
-
-### 1. 初始化資料 (Seeding)
-
-根據測試需求選擇模式：
-
-```bash
-# 基本資料（Waiting, Delayed, Failed）
-bun scripts/seed.ts standard
-
-# 壓力測試（建立 15 個佇列，每個佇列有大量資料）
-bun scripts/seed.ts stress
-
-# 批次操作測試（專用大量資料）
-bun scripts/seed.ts batch
-
-# 註冊排程管理 (Cron Jobs)
-bun scripts/seed.ts cron
-
-# 懶人包：一次執行以上所有模式
-bun scripts/seed.ts all
-
-# 清理所有資料
-bun scripts/seed.ts cleanup
-```
-
-### 2. 啟動背景處理 (Workers)
-
-模擬工作正在被處理、成功或失敗的過程：
-
-```bash
-# 啟動預設佇列的處理
-bun scripts/worker.ts
-
-# 模擬高失敗率與處理延遲
-bun scripts/worker.ts orders,reports --fail=0.3 --delay=500
-```
-
----
-
-## 🧪 核心測試情境
-
-### 測試 1：排程管理 (Schedules)
-1. 執行 `bun scripts/seed.ts cron`。
-2. 在 UI 點擊側邊欄的 **"Schedules"**。
-3. 檢查是否顯示 `cleanup-tmp`, `daily-report` 等排程。
-4. 點擊 "Run Now" 檢查工作是否立即進入 `system` 或 `reports` 佇列。
-
-### 測試 2：批次操作 (Batch Operations)
-1. 執行 `bun scripts/seed.ts batch`。
-2. 進入 `test-batch` 佇列。
-3. 使用 **Cmd+A** 全選，然後點擊 "Delete Selected"。
-4. 測試頁面底部的「刪除所有 X 個工作」警告橫幅。
-
-### 測試 3：排版與搜尋
-1. 執行 `bun scripts/seed.ts stress`。
-2. 檢查側邊欄的佇列列表是否正確收合/捲動。
-3. 使用 **Cmd+K** 開啟命令列，輸入 `billing` 跳轉至該佇列。
-
----
-
-## 🧹 清理空間
-
-測試完成後，建議執行：
-```bash
-bun scripts/seed.ts cleanup
-```
-這會清除 Redis 中所有帶有 `queue:` 前綴的 Key，以及 Console 的日誌快取。

package/docs/ROADMAP.md

@@ -1,85 +0,0 @@
-# Gravito Zenith Roadmap (Control Plane)
-
-This document outlines the future development plan for **Gravito Zenith**, moving from a basic monitoring tool to a comprehensive polyglot control plane for any job processing system.
-
-## 🚀 High Priority (Immediate Next Steps)
-
-### 1. Polyglot & Framework Integration (P0)
-**Goal**: Official support for non-Node.js workers, making Zenith a universal dashboard.
-- **Tasks**:
-    - [x] **Protocol Specification**: Defined standard Redis structures for Heartbeat and Logs.
-    - [x] **Laravel Integration**: Blueprint for `ZenithServiceProvider` and `ZenithConnector` defined.
-    - [ ] **Go/Python SDK**: Create lightweight client libraries for other languages.
-
-### 2. Docker & Cloud-Native Deployment (P1)
-**Goal**: Enable "One-Click Deployment" for any environment (local, EC2, K8s).
-- **Current Blocker**: Local workspace dependencies (`workspace:*`) cause build failures in standard Docker contexts.
-- **Tasks**:
-- [x] Fix `Dockerfile` dependency resolution (via multi-stage builds).
-- [x] Create `docker-compose.yml` for a full stack setup (Console + Redis + Demo Worker).
-- [x] Implementation of Scheduled Jobs (Cron) Management.
-
-### 3. System Pulse Monitoring (Lightweight APM) (P0 - NEW)
-**Goal**: Application-Aware lightweight resource monitoring and alerting.
-**Philosophy**: "If you can connect to Redis, you are monitored." No extra agents, no Prometheus config.
-- **Features**:
-    - **Live Resource Cards**: CPU / RAM / Disk Usage per service (Node.js, PHP, Python).
-    - **Health Heartbeats**: Auto-discovery of active services via `pulse:{service}:{id}` keys.
-    - **Proactive Alerting**: Slack notifications for Disk Space < 10% or High CPU Load.
-    - **Protocol-Based**: Language-agnostic design (Gravito Pulse Protocol).
-
-
-### 3. History Persistence (SQL Archive) (Completed ✅)
-**Goal**: Store job history permanently for auditing and long-term analysis.
-- [x] Implement a `PersistenceAdapter` in `@gravito/stream`.
-- [x] Automatically archive completed/failed jobs to a SQL database.
-- [x] **Zero-Config (SQLite)**: Integrated support for local testing.
-- [x] **Time Travel Audit**: comprehensive UI for tracing job history.
-
-## ✨ Feature Enhancements (Mid-Term)
-
-### 4. Alerting & Notifications (Completed ✅)
-**Goal**: Proactive issue notification system.
-- [x] **AlertService**: Lightweight rules for failure spikes, backlog, and worker loss.
-- [x] **Cooldown Mechanism**: Prevents alerting storms.
-- [x] **Slack Integration**: Webhook support with test notification UI.
-- [x] **Real-time Monitoring**: Integrated directly into server metrics loop.
-
-### 5. Scheduled Jobs (Cron) Management (Completed ✅)
-**Goal**: UI-based management for recurring tasks.
-- [x] Dashboard to view all registered Cron jobs.
-- [x] Ability to "Trigger Now" manually.
-- [x] Ability to Pause/Resume (Delete/Register) specific Cron schedules.
-- [x] Real-time ticking via the Console server.
-
-### 6. Batch Operations (Completed ✅)
-**Goal**: Bulk management actions.
-- **Problem**: Can only retry/delete one job or "all" jobs. Hard to handle "the 50 jobs that failed due to the bug yesterday".
-- **Tasks**:
-    - [x] Multi-select checkboxes in job lists.
-    - [x] Bulk Retry / Bulk Delete.
-    - [x] Select All Matching Query (Delete/Retry ALL jobs of a type).
-    - [x] Confirmation dialogs with loading states.
-    - [x] Keyboard shortcuts (Ctrl+A, Escape).
-    - [x] Visual feedback and total count display.
-
-## 🔮 Enterprise Features (Long-Term)
-
-### 7. Role-Based Access Control (RBAC)
-**Goal**: Granular permission management for teams.
-- **Problem**: Single password for everyone. Risky for junior devs to have "Delete Queue" power.
-- **Features**:
-    - Roles: `Viewer` (Read-only), `Operator` (Retry/Pause), `Admin` (Delete/Purge).
-    - User Management system (potentially integrated with OAuth/SSO).
-
-### 8. Multi-Cluster Management
-**Goal**: Centralized control pane for multiple environments.
-- **Problem**: Need to open 3 different tabs for Dev, Staging, and Prop.
-- **Features**:
-    - Connection Switcher in the UI header.
-    - Unified view of multiple Redis instances.
-
-### 9. Enhanced Search (Indexer)
-**Goal**: Full-text search on Job Payloads.
-- **Problem**: Can only search by Job ID. Cannot search by "email: user@example.com".
-- **Solution**: Implement a lightweight search index (RedisSearch or external engine) to index payloads.

package/docs/integrations/LARAVEL.md

@@ -1,207 +0,0 @@
-# Laravel Integration Guide for Gravito Zenith
-
-This guide outlines the architecture and implementation details for integrating Laravel applications with **Gravito Zenith**, enabling centralized monitoring, logging, and auditing for Laravel Queues.
-
-## Architecture Overview
-
-+----------------+       +--------------+       +----------------+
-| Laravel App    |       | Redis Broker |       | Gravito Zenith |
-| (Horizon/Queue)| ----> | (Shared)     | <---- | Control Plane  |
-+----------------+       +--------------+       +----------------+
-        |                       ^
-        |                       | Redis Pub/Sub & Lists
-        +--- [Zenith Connector] +
-
-The **Zenith Connector** acts as a bridge, translating Laravel's internal queue events into the Zenith Protocol.
-
----
-
-## 1. Protocol Specification
-
-To be visible in Zenith, the Laravel connector must implement the following Redis interactions:
-
-### Namespace
-Default Prefix: `flux:` (configurable). Ensure this matches your Zenith configuration.
-
-### A. Worker Heartbeat (Process Discovery)
-The connector must run a background process (or scheduled command) to announce the worker's presence.
-
-- **Key**: `flux_console:worker:<worker-id>`
-- **TTL**: 60 seconds (refresh every 30s)
-- **Format**:
-```json
-{
-  "id": "laravel-worker-supervisor-1",
-  "hostname": "app-server-01",
-  "pid": 1234,
-  "uptime": 3600,
-  "queues": ["default", "emails"],
-  "concurrency": 10,
-  "memory": {
-    "rss": "100MB",
-    "heapTotal": "N/A",
-    "heapUsed": "N/A"
-  },
-  "timestamp": "ISO-8601 String"
-}
-```
-
-### B. Real-time Logs (Event Stream)
-The connector listens to Laravel Queue events and publishes them to Zenith.
-
-- **Channel**: `flux_console:logs`
-- **Format**:
-```json
-{
-  "level": "info",    // info | warn | error | success
-  "message": "Processing Job: App\\Jobs\\SendWelcomeEmail",
-  "workerId": "laravel-worker-supervisor-1",
-  "queue": "emails",
-  "timestamp": "ISO-8601 String",
-  "jobId": "uuid-..."  // Optional, enables specific tracing
-}
-```
-
-### C. Job Auditing (Time Travel)
-(Optional) For "Time Travel Audit" features, the connector should write to the persistent store if configured, or rely on Zenith's Redis scanning if utilizing standard Flux queue structures. Since Laravel uses its own queue structure, **Real-time Logs** are the primary integration point for v1.
-
----
-
-## 2. Implementation Blueprint (PHP)
-
-This section provides the reference implementation for the `gravito/zenith-laravel` composer package.
-
-### Service Provider: `ZenithServiceProvider.php`
-
-This provider hooks into Laravel's Queue events.
-
-```php
-<?php
-
-namespace Gravito\Zenith\Laravel;
-
-use Illuminate\Support\ServiceProvider;
-use Illuminate\Support\Facades\Queue;
-use Illuminate\Support\Facades\Redis;
-use Illuminate\Queue\Events\JobProcessing;
-use Illuminate\Queue\Events\JobProcessed;
-use Illuminate\Queue\Events\JobFailed;
-
-class ZenithServiceProvider extends ServiceProvider
-{
-    public function boot()
-    {
-        // 1. Job Started
-        Queue::before(function (JobProcessing $event) {
-            $this->publishLog('info', $event);
-        });
-
-        // 2. Job Success
-        Queue::after(function (JobProcessed $event) {
-             $this->publishLog('success', $event);
-        });
-
-        // 3. Job Failed
-        Queue::failing(function (JobFailed $event) {
-             $this->publishLog('error', $event, $event->exception->getMessage());
-        });
-    }
-
-    protected function publishLog($level, $event, $extraMessage = '')
-    {
-        $payload = $event->job->payload();
-        $jobName = $payload['displayName'] ?? 'Unknown Job';
-        
-        // Simplify Job Name (remove namespace for display)
-        $shortName = class_basename($jobName);
-
-        $message = match($level) {
-            'info' => "Processing {$shortName}",
-            'success' => "Completed {$shortName}",
-            'error' => "Failed {$shortName}: {$extraMessage}",
-        };
-
-        $log = [
-            'level' => $level,
-            'message' => $message,
-            'workerId' => gethostname() . '-' . getmypid(), // Simple ID generation
-            'queue' => $event->job->getQueue(),
-            'timestamp' => now()->toIso8601String(),
-            'jobId' => $event->job->getJobId()
-        ];
-
-        // Fire and forget to Redis
-        try {
-            Redis::connection('zenith')->publish('flux_console:logs', json_encode($log));
-        } catch (\Exception $e) {
-            // Silently fail to not disrupt main application
-        }
-    }
-}
-```
-
-### Heartbeat Command: `zenith:heartbeat`
-
-This command should be run as a daemon (Supervisor) or scheduled every minute (less precise). For best results, run as a sidecar process.
-
-```php
-<?php
-
-namespace Gravito\Zenith\Laravel\Console;
-
-use Illuminate\Console\Command;
-use Illuminate\Support\Facades\Redis;
-
-class ZenithHeartbeat extends Command
-{
-    protected $signature = 'zenith:heartbeat';
-    protected $description = 'Send heartbeat to Gravito Zenith';
-
-    public function handle()
-    {
-        $this->info('Starting Zenith Heartbeat...');
-
-        while (true) {
-            $workerId = gethostname() . '-' . getmypid();
-            
-            $payload = [
-                'id' => $workerId,
-                'hostname' => gethostname(),
-                'pid' => getmypid(),
-                'uptime' => 0, // Calculate real uptime if needed
-                'queues' => config('queue.connections.redis.queue', ['default']),
-                'memory' => [
-                    'rss' => round(memory_get_usage() / 1024 / 1024, 2) . ' MB',
-                    'heapUsed' => 'N/A',
-                    'heapTotal' => 'N/A'
-                ],
-                'timestamp' => now()->toIso8601String()
-            ];
-
-            Redis::connection('zenith')->setex(
-                "flux_console:worker:{$workerId}",
-                30, // 30s TTL
-                json_encode($payload)
-            );
-
-            sleep(5);
-        }
-    }
-}
-```
-
-## 3. Configuration
-
-Users will need to configure a dedicated Redis connection for Zenith in `config/database.php` to avoid prefix collisions if they modify their default Redis prefix.
-
-```php
-'redis' => [
-    'zenith' => [
-        'host' => env('ZENITH_REDIS_HOST', '127.0.0.1'),
-        'password' => env('ZENITH_REDIS_PASSWORD', null),
-        'port' => env('ZENITH_REDIS_PORT', '6379'),
-        'database' => env('ZENITH_REDIS_DB', '0'),
-        'prefix' => '', // Ensure no prefix or match Zenith's expectation
-    ],
-],
-```

package/postcss.config.js

@@ -1,6 +0,0 @@
-export default {
-    plugins: {
-        tailwindcss: {},
-        autoprefixer: {},
-    },
-}

package/scripts/debug_redis_keys.ts

@@ -1,24 +0,0 @@
-import { Redis } from 'ioredis'
-
-const redis = new Redis('redis://localhost:6379')
-
-async function check() {
-  console.log('Connecting to Redis...')
-  try {
-    const keys = await redis.keys('gravito:quasar:node:*')
-    console.log('Keys found count:', keys.length)
-    console.log('Keys:', keys)
-
-    if (keys.length > 0) {
-      const val = await redis.get(keys[0])
-      console.log('--- Value of first key ---')
-      console.log(val)
-      console.log('--- End Value ---')
-    }
-  } catch (err) {
-    console.error('Redis Error:', err)
-  }
-  process.exit(0)
-}
-
-check()

package/scripts/flood-logs.ts

@@ -1,21 +0,0 @@
-import { QueueService } from '../src/server/services/QueueService'
-
-const svc = new QueueService(process.env.REDIS_URL || 'redis://localhost:6379')
-await svc.connect()
-
-console.log('🌊 Flooding 500 logs in burst...')
-const start = Date.now()
-const promises = []
-for (let i = 0; i < 500; i++) {
-  promises.push(
-    svc.publishLog({
-      level: 'info',
-      message: `Flood log ${i} - ${Date.now()}`,
-      workerId: 'flood-bot',
-      queue: 'test-flood',
-    })
-  )
-}
-await Promise.all(promises)
-console.log(`✅ Sent 500 logs in ${Date.now() - start}ms`)
-process.exit(0)