@sriinnu/runic 1.2.0 → 2.11.1

package/docs/research/feature_uiux_roadmap_2026-02-22.md

@@ -1,202 +0,0 @@
-# Runic Feature And UI/UX Roadmap (2026-02-22)
-
-Branch: `feat/research-uiux-core-next`
-
-## Objective
-
-Identify high-impact features and UI/UX upgrades for Runic's macOS-first product and cross-platform roadmap, using:
-
-- Current codebase direction (`RunicCore` + menu insights + budgets/forecast/anomaly work)
-- External product signals (GitHub OSS LLM observability/gateway stacks)
-- Research signals (forecasting and anomaly detection papers)
-
-## External Signals (Primary Sources)
-
-1. Langfuse: LLM observability, prompt management, evaluations, datasets, and OTel integrations.
-   - https://github.com/langfuse/langfuse
-2. Helicone: AI gateway + observability, metrics (including cost), prompt/version workflows, and 100+ model access.
-   - https://github.com/Helicone/helicone
-   - https://github.com/Helicone/ai-gateway
-3. LiteLLM features: spend tracking by model/key/user/team and budget/rate-limit controls.
-   - https://www.litellm.ai/features
-4. Portkey gateway: reliability primitives (fallbacks/retries/load balancing), cost management (smart caching, usage analytics, provider optimization).
-   - https://github.com/Portkey-AI/gateway
-5. OpenLIT: OTel-native observability with cost tracking for custom/fine-tuned models.
-   - https://github.com/openlit/openlit
-6. OpenTelemetry GenAI/OpenAI semantic conventions (`Status: Development`; migration opt-in flags).
-   - https://opentelemetry.io/docs/specs/semconv/gen-ai/openai/
-7. Chronos (2024): pretrained probabilistic forecasting on 42 datasets; strong zero-shot behavior.
-   - https://arxiv.org/abs/2403.07815
-   - https://github.com/amazon-science/chronos-forecasting
-8. TimesFM (ICML 2024): decoder-only time-series foundation model with zero-shot forecasting focus.
-   - https://arxiv.org/abs/2310.10688
-   - https://github.com/google-research/timesfm
-9. TimeEval: anomaly detection benchmarking toolkit with 700+ datasets.
-   - https://github.com/TimeEval/TimeEval
-10. SigLLM: two anomaly pipelines (prompt-based and forecast-residual detector).
-   - https://github.com/sintel-dev/sigllm
-11. Apple VoiceOver evaluation guidance for complete task coverage and accurate labels, including chart accessibility guidance.
-   - https://developer-mdn.apple.com/help/app-store-connect/manage-app-accessibility/voiceover-evaluation-criteria
-
-## What This Means For Runic (Inference)
-
-Inference from the above sources:
-
-- The market baseline is moving from "usage display" to "control plane": budgets, policy, routing, reliability, and explainability.
-- Forecasting is shifting from single deterministic projections to probabilistic forecasts with confidence bands.
-- Anomaly systems are becoming multi-signal (tokens, spend, request patterns) and benchmarked/validated.
-- UI quality expectation is now "operator-ready": fast triage, actionable next-step copy, and reliable accessibility.
-
-## Priority Feature Tracks
-
-### P0: Cost Control Center (Operator-Grade)
-
-Add a policy engine that can do more than alert:
-
-- Budget scopes: provider, model, project, team, user
-- Enforcement modes: warn, soft-limit, hard-stop, cooldown
-- Automatic actions:
-  - downgrade model recommendation
-  - route to cheaper provider/model
-  - temporary key throttling
-- Audit log for every enforcement decision
-
-Why:
-
-- Aligns with LiteLLM/Portkey/OpenMeter direction.
-- Turns Runic from observer to controller.
-
-### P0: Probabilistic Forecasts (Not Single-Point)
-
-Upgrade forecast insight lines to include confidence:
-
-- Show `p50`, `p80`, `p95` month-end forecast
-- Visualize confidence band in menu detail popover/card
-- Add "risk of budget breach by date" as probability
-
-Why:
-
-- Chronos/TimesFM emphasize robust zero-shot forecasting; practical value is uncertainty-aware decisions.
-
-### P0: Explainable Anomaly Engine v2
-
-Current anomaly support exists; extend it:
-
-- Detect across tokens, spend, request count, cost-per-1k, burstiness
-- Add root-cause hints:
-  - top model delta
-  - top project delta
-  - top provider shift
-- Severity with rationale:
-  - "High due to +182% tokens and +74% cost vs 7d baseline"
-
-Why:
-
-- Improves trust and operator response speed.
-- Matches SigLLM/TimeEval-style evaluation mindset.
-
-### P1: Unified Telemetry Import (OTel GenAI)
-
-Add ingest path for OTel GenAI events/spans:
-
-- Parse selected semantic fields (model, operation, token usage, errors)
-- Normalize into Runic ledger schema
-- Keep compatibility flag around OTel semconv instability (`gen_ai_latest_experimental` migration concerns)
-
-Why:
-
-- Vendor-neutral ingest path.
-- Enables cross-tool interoperability (Langfuse/OpenLIT ecosystems).
-
-### P1: Team FinOps Layer
-
-Add internal chargeback/showback views:
-
-- Team/org/project rollups
-- Cost allocation percentages
-- Shared budget pools with split ownership
-- Monthly export (CSV/JSON)
-
-Why:
-
-- Natural extension of existing provider/project logic and budget store.
-- Strong fit for organizations with multi-provider AI spend.
-
-### P1: Cross-Platform Feature Parity Tracker
-
-For Windows/Linux/web direction, define explicit parity matrix:
-
-- Feature rows: menu insights, alerting, budgets, anomaly, forecast, keychain/secret handling
-- Platform columns: macOS, Windows, Linux, web
-- Status: done / partial / blocked (with owner)
-
-Why:
-
-- Reduces roadmap drift.
-- Makes release quality measurable.
-
-## UI/UX Upgrades (Focused, Non-Boilerplate)
-
-### 1. Three-Layer Insight Density
-
-Use progressive disclosure consistently:
-
-- Glance: 1-line health + spend trend arrow + risk chip
-- Analyst: model/project deltas + anomaly and forecast confidence
-- Operator: policy state, throttles, budget enforcement history
-
-### 2. Action-First Error Copy
-
-Every failure line should provide:
-
-- what failed
-- likely cause
-- immediate recovery action
-- copyable diagnostic key
-
-### 3. Contextual Sparkline + Delta Chips
-
-For each provider card:
-
-- 7-day sparkline for tokens and spend
-- chips: `+/- % vs 7d avg`, `burn/hr`, `budget risk`
-
-### 4. Accessibility Hardening (Ship Quality)
-
-Explicitly verify:
-
-- all actionable controls have concise accessibility labels
-- common user tasks are possible using only VoiceOver
-- data visualizations include text alternatives
-
-### 5. Notification Hygiene
-
-- Suppress redundant alerts during known cooldown windows
-- Group related anomalies into one actionable notification
-- Include one-click route to relevant menu section
-
-## Suggested Implementation Sequence
-
-1. Forecast confidence bands (low-medium complexity, high user value)
-2. Anomaly explainability v2
-3. Cost Control Center policy engine (warn + soft-limit first)
-4. OTel import beta (feature flag)
-5. Team FinOps rollups and exports
-6. Cross-platform parity gating in CI/docs
-
-## Concrete Next Coding Tasks
-
-1. Extend `UsageLedgerSpendForecast` to carry quantiles and confidence metadata.
-2. Add `UsageLedgerAnomalyExplanation` model and formatter surface for menu cards.
-3. Introduce `PolicyDecision` and `PolicyEngine` in `RunicCore` (platform-agnostic).
-4. Add UI chips and operator detail row in `MenuCardView` and menu helpers.
-5. Add tests:
-   - forecast quantile formatting
-   - anomaly explanation precedence
-   - policy decision routing logic
-6. Add `docs/research/parity_matrix.md` template for platform rollout governance.
-
-## Notes
-
-- This document intentionally mixes external evidence + implementation inference.
-- External evidence is linked above; internal recommendations are adapted to the current Runic architecture and recent branch history.

package/docs/research/parity_matrix.md

@@ -1,49 +0,0 @@
-# Cross-Platform Parity Matrix
-
-Last updated: 2026-02-23  
-Owner: Runic core team
-
-## Status Legend
-
-- `done`: implemented and validated
-- `partial`: available with notable gaps
-- `planned`: roadmap item, not implemented yet
-- `blocked`: cannot proceed until dependency/platform issue is resolved
-
-## Product Surface Parity
-
-| Capability | macOS | Windows | Linux | Web |
-|---|---|---|---|---|
-| Provider usage fetch (Codex/Claude/Gemini/Copilot/etc.) | done | planned | planned | planned |
-| Usage ledger aggregation (daily/session/models/projects) | done | planned | planned | planned |
-| Forecast confidence bands (`p50/p80/p95`) | done | planned | planned | planned |
-| Anomaly explainability (severity + rationale) | done | planned | planned | planned |
-| Policy engine (warn/soft-limit/hard-stop core model) | done | planned | planned | planned |
-| Team showback/chargeback export model (JSON/CSV) | done | planned | planned | planned |
-| OTel GenAI ingestion adapter (feature-flagged) | done | planned | planned | planned |
-| Menu bar status icon + animated refresh states | done | blocked | blocked | blocked |
-| Provider settings pane + diagnostics | done | planned | planned | planned |
-| Calendar history view (provider activity) | done | planned | planned | planned |
-| Secure secret storage (API tokens) | partial | planned | planned | planned |
-| CLI usage/cost/insights | done | planned | planned | planned |
-| Widget extension | done | blocked | blocked | blocked |
-
-## Provider-Detail Granularity Parity
-
-| Provider | Model detail | Token detail | Project attribution | Notes |
-|---|---|---|---|---|
-| Claude | done | done | done | Local ledger-backed |
-| Codex | done | done | done | Local ledger-backed |
-| Copilot | partial | partial | planned | Quota-category windows, no full project-level attribution |
-| Gemini | partial | partial | planned | Model quota windows, no full project-level attribution |
-| Antigravity | partial | partial | planned | Model quota windows, no full project-level attribution |
-| Cursor | partial | partial | planned | Depends on provider data exposure |
-| API providers (OpenRouter/Azure/Bedrock/etc.) | partial | partial | planned | Varies by provider API payload |
-
-## Gate To Mark A Platform As `done`
-
-1. Build and test pipeline is green on that platform for core + CLI.
-2. Usage ledger summaries and provider identity isolation are validated.
-3. Secrets are stored using platform-appropriate secure storage.
-4. Forecast/anomaly/policy insights pass deterministic snapshot tests.
-5. Release gate checklist is fully passed for that platform.

package/docs/research/provider_expansion_candidates_2026-02-22.md

@@ -1,80 +0,0 @@
-# Provider Expansion Candidates (2026-02-22)
-
-## Scope
-
-Runic currently tracks: codex, claude, cursor, factory, gemini, antigravity, copilot, zai, minimax, openrouter, groq.
-
-This note identifies additional providers or higher-fidelity data paths with primary-source evidence and implementation fit.
-
-## High-Priority Additions
-
-### 1) Perplexity API
-- Why: Official OpenAI-compatible chat/completions endpoint with explicit `usage` object in response.
-- Integration path:
-  - Auth: API key (Keychain + env fallback)
-  - Data: response `usage.prompt_tokens`, `usage.completion_tokens`, `usage.total_tokens`
-  - UX: token usage now; spend estimate via local pricing table until billing endpoint is available
-- Source:
-  - https://docs.perplexity.ai/api-reference/chat-completions-post
-
-### 2) Fireworks AI
-- Why: Chat completions API documents a detailed `usage` object, including cached token fields.
-- Integration path:
-  - Auth: API key
-  - Data: `usage.prompt_tokens`, `completion_tokens`, `total_tokens`, cached token details
-  - UX: supports richer efficiency metrics (cache hit behavior)
-- Source:
-  - https://docs.fireworks.ai/api-reference/post-chatcompletions
-
-### 3) DeepSeek API
-- Why: API reference includes standard `usage` token fields on chat completion responses.
-- Integration path:
-  - Auth: API key
-  - Data: prompt/completion/total token counters
-  - UX: direct compatibility with existing token-based insight pipeline
-- Source:
-  - https://api-docs.deepseek.com/api/create-chat-completion
-
-### 4) Anthropic Org Cost Path (for Claude)
-- Why: Official Admin Usage & Cost API can provide authoritative org-level usage/cost.
-- Integration path:
-  - Keep existing Claude CLI/Web probes
-  - Add optional enterprise path for org admins via Anthropic Admin API
-- Source:
-  - https://docs.anthropic.com/en/api/usage-cost-api
-
-## Existing Provider Upgrade Opportunity
-
-### OpenRouter (already in Runic): credits + limits enhancement
-- Why: Official endpoint provides credits/limits model that can improve remaining/budget accuracy.
-- Integration path:
-  - Enhance current OpenRouter provider with direct credits fetch and tighter budget telemetry
-- Source:
-  - https://openrouter.ai/docs/api-reference/limits-and-credits/get-credits
-
-## Medium-Priority (Needs Validation)
-
-### Mistral API
-- Known signal: official docs expose enterprise-focused limits/rate-limits pages.
-- Gap: we still need a stable public usage/cost endpoint for robust quota accounting.
-- Source:
-  - https://docs.mistral.ai/deployment/laplateforme/tier/
-
-## Security and Performance Guardrails
-
-- Security:
-  - API keys only in Keychain/env; never persisted in plaintext logs
-  - Calendar/history views must use aggregated local data only (no raw prompt payloads)
-  - Diagnostics copy paths must redact secrets
-- Performance:
-  - On-demand month fetches only (no always-on scans)
-  - TTL cache for month snapshots
-  - bounded scan windows (e.g., 90–180 days max)
-
-## Suggested Execution Order
-
-1. Perplexity
-2. Fireworks
-3. DeepSeek
-4. OpenRouter credits enhancement
-5. Anthropic Admin usage-cost optional path

package/docs/research/provider_model_token_project_matrix_2026-02-23.md

@@ -1,46 +0,0 @@
-# provider model/token/project coverage matrix (2026-02-23)
-
-## question addressed
-
-Can Runic show per-model usage, token counts, and project-level attribution for providers like Copilot, OpenRouter, Azure OpenAI, and AWS Bedrock?
-
-## capability summary
-
-| provider | model usage | token metrics | project attribution | notes |
-| --- | --- | --- | --- | --- |
-| copilot | yes | no (not exposed in official usage metrics) | no (no explicit repo/project field in metrics schema) | org/admin usage metrics can still show model mix |
-| openrouter | yes | yes | partial (needs caller-side project tagging) | usage/cost endpoints expose model and token/cost data |
-| azure openai | yes | yes | yes (resource/project scope + custom telemetry) | best source is Azure Monitor metrics + project resources |
-| aws bedrock | yes | yes | partial (via request metadata + logs) | CloudWatch tokens by model; project mapping requires metadata/log strategy |
-
-## current Runic status
-
-- Added UI coverage badges for providers through `ProviderUsageCoverage`.
-- Marked current coverage in metadata:
-  - `codex`: models + tokens + projects
-  - `claude`: models + tokens + projects
-  - `openrouter`: models + tokens
-  - `copilot`: models
-
-## implementation plan (cross-platform)
-
-1. OpenRouter deep usage:
-   - Add per-model aggregation from generation/usage API into `UsageLedger`-compatible summaries.
-   - Expose project attribution through optional request metadata key (for example `X-Runic-Project`).
-
-2. Azure OpenAI provider:
-   - Add provider with `endpoint + api key` settings and Azure Monitor pull path.
-   - Ingest `InputTokens`, `OutputTokens`, `TotalTokens` by model/deployment.
-   - Add optional Entra auth mode later for enterprise setups.
-
-3. Bedrock provider:
-   - Add provider with `region + profile` settings.
-   - Pull model/token metrics from CloudWatch (`ModelId` dimension).
-   - Add optional project grouping from invocation log metadata.
-
-4. Unified data contract:
-   - Normalize provider outputs into one cross-platform schema:
-     - `model_usage[]`
-     - `token_totals`
-     - `project_usage[]`
-   - Keep all logic in `RunicCore` so macOS/Linux/Windows reuse the same pipeline.

package/docs/research/release_gate_checklist.md

@@ -1,57 +0,0 @@
-# Release Gate Checklist
-
-Last updated: 2026-02-23
-
-Use this checklist before promoting a branch to `main` or cutting a tagged release.
-
-## 1) Branch And Scope Hygiene
-
-- [ ] Feature branch is rebased/merged cleanly against target.
-- [ ] No unrelated local changes in working tree.
-- [ ] All migration notes are included for settings/schema changes.
-
-## 2) Build Gates
-
-- [ ] `swift build` succeeds.
-- [ ] `swift test` succeeds.
-- [ ] `./Scripts/compile_and_run.sh` succeeds and relaunches app.
-- [ ] Release packaging script (if release cut): `./Scripts/package_app.sh`.
-
-## 3) Runtime Validation Gates
-
-- [ ] Core providers fetch successfully in real environment.
-- [ ] Menu bar remains responsive under active refresh load.
-- [ ] No repeated credential prompts/regressions after rebuild/relaunch.
-- [ ] Error states include actionable next steps for user recovery.
-
-## 4) Data Correctness Gates
-
-- [ ] Provider identity isolation verified (no cross-provider leakage).
-- [ ] Ledger summaries (daily/models/projects) validated with test fixtures.
-- [ ] Budget/forecast/anomaly outputs are deterministic in tests.
-- [ ] New exports/ingestion paths have schema tests and sample fixtures.
-
-## 5) Security And Privacy Gates
-
-- [ ] Secrets are stored in secure platform storage (no plaintext persistence).
-- [ ] Logs do not include raw API tokens or auth headers.
-- [ ] Export payloads avoid sensitive fields by default.
-- [ ] Any new network path has explicit timeout/error handling.
-
-## 6) UX And Accessibility Gates
-
-- [ ] Provider settings spacing and layout render correctly in light/dark modes.
-- [ ] Primary workflows are reachable via keyboard and VoiceOver labels.
-- [ ] Charts/visual summaries expose textual alternatives.
-
-## 7) Docs And Operations Gates
-
-- [ ] Provider docs and research docs updated for behavior changes.
-- [ ] Parity matrix updated for feature/platform status shifts.
-- [ ] Open TODO checkboxes updated to reflect shipped items.
-
-## 8) Post-Merge Verification
-
-- [ ] Merge commit or fast-forward push confirmed on remote.
-- [ ] Fresh pull + build on clean environment succeeds.
-- [ ] Known follow-up tasks filed with owners and priority.