@exellix/jobs-ui 2.0.1 → 2.0.3

package/README.md

@@ -1,6 +1,6 @@
 # @exellix/jobs-ui
 
-**Jobs dashboard** — React operator SPA for the Exellix job queue. Queue overview, run detail, record-centric views, Source→Records workflow, and work-factory wizards (continuous work, batches, on-demand).
+**Jobs dashboard** — React operator SPA for the Exellix job factory. Monitor queue health, browse Memorix information and graph outputs, inspect the run journal, configure ongoing work and on-demand batches, and analyze pipeline performance.
 
 | Package | Role |
 |---------|------|
@@ -9,7 +9,38 @@
 | **`@exellix/jobs-ui`** (this) | Jobs **dashboard** — React SPA only |
 | [`@exellix/jobs-db`](../jobs-db/README.md) | Jobs **data tier** — Mongo persistence |
 
-> A **JobRun** is one graph run on one input. See [`temp/jobs/`](../temp/jobs/README.md) for the design.
+> A **factory run** is one graph execution on one information item. Continuous work and on-demand flows both enqueue runs into the same journal. See [`temp/jobs/`](../temp/jobs/README.md) for the design.
+
+> **Migration from 1.x:** `@exellix/jobs-ui@1.x` bundled the HTTP API and dashboard in one package. **2.x is dashboard-only** — use [`@exellix/jobs-api`](../jobs-api/README.md) for the REST server and programmatic queries.
+
+## Core concepts
+
+### Information items (not just “database records”)
+
+The UI speaks in operator terms — **Information** and **item** — but the API and Memorix layer still use **record** in paths and types (`core-records`, `recordId`, etc.). Treat those as the same idea: **one processable unit of information** that a pipeline can run on.
+
+That unit is **not** limited to a classic entity row. What it *is* depends entirely on how the object type is modeled in Schema UI and stored in Memorix. Examples:
+
+| What you might process | Typical Memorix shape |
+|------------------------|------------------------|
+| A business object (asset, ticket, person) | Core record on an **entity** |
+| A document, page, or media asset | **Content** in a scoped content type |
+| One chat turn | A row in a messages / content collection |
+| An entire conversation | The **core** record for a chat entity (with related message content assembled at run time) |
+| A time-stamped occurrence | An **event** record |
+| A knowledge article or embedding source | A **knowledge** record |
+| Graph output from a prior run | A row in a **processing-output** collection (shown under Outputs) |
+
+So “record” here means **whatever your schema defines as the input identity for a run** — not necessarily a single Mongo document in `core`, and not necessarily one physical row if you merge related content types when queuing.
+
+For the job factory:
+
+- **One factory run = one item** — identified by `entity` + `collection` + `id` (plus optional related content pulled in as context).
+- **Narratives** tag items for matching (ongoing work and batch selection) — e.g. “needs summary”, “high priority”.
+- **Processing stamps** track which pipelines have touched an item and whether they succeeded.
+- **Assembly** (on-demand single) lets you choose which content types and fields are merged into graph input without changing which item the run is *for*.
+
+Browse and inspect items under **Information** (inputs) and **Outputs** (saved pipeline results). The journal always refers back to the item that was processed, whatever shape it had.
 
 ## Install
 
@@ -21,15 +52,35 @@ The dashboard requires a running `@exellix/jobs-api` instance (or any host servi
 
 ## Quick start
 
+**Recommended** — one command starts API, queue worker, and UI (needs `MONGO_URI` in `graph-engine/.env`):
+
 ```bash
-# terminal 1 — API backend (needs MONGO_URI)
+npm run dev            # from jobs-ui — full stack via scripts/dev-jobs.mjs
+# or from monorepo root:
+npm run dev:jobs
+```
+
+| URL | Service |
+|-----|---------|
+| http://localhost:5190 | Dashboard (Vite, proxies `/api`) |
+| http://localhost:3099 | jobs-api REST server |
+
+**Split terminals** (UI-only with HMR against an API you start separately):
+
+```bash
+# terminal 1 — API backend
 cd ../jobs-api && npm run serve
 
-# terminal 2 — UI with HMR
-npm run dev            # Vite on :5190, proxies /api → http://localhost:3099
+# terminal 2 — UI only
+npm run dev:ui         # Vite on :5190, proxies /api → http://localhost:3099
 ```
 
-Set `VITE_API_PROXY` to point at a remote API during dev. Set `VITE_DEV_PORT` to change the dev server port.
+### Dev configuration
+
+| Variable | Default | Effect |
+|----------|---------|--------|
+| `VITE_API_PROXY` | `http://localhost:<PORT>` | API target for the Vite `/api` proxy (`PORT` from `jobs-api/.env`, default 3099) |
+| `VITE_DEV_PORT` | `5190` | Vite dev server port |
 
 ### Combined deploy (single port)
 
@@ -40,32 +91,212 @@ npm run build
 cd ../jobs-api && JOBS_UI_STATIC_DIR=../jobs-ui/dist npm run serve
 ```
 
-## Browser routes
+When the API sets `JOBS_UI_TOKEN`, the SPA must send `Authorization: Bearer <token>` on `/api/*` requests (configure at the reverse proxy or API layer).
+
+## Architecture
+
+- **React 19** SPA with **React Router** and **TanStack Query** for data fetching and cache invalidation.
+- All data flows through `web/src/api/real.ts` → `/api/*` on the jobs-api host.
+- **Agent scope** — the top-bar `AgentScopeSelect` (`@x12i/memorix-agents`) filters entities, ongoing work, information lists, and journal views to a single Memorix agent when selected. Persisted in `localStorage` under `exellix-memorix-ui.selected-agent`.
+- **Queue manager** — compact pause/resume control in the top bar; full card on Settings. Polls queue settings every 10s.
+- **Infinite scroll** — Information browse and Journal list load pages on demand via `useCoreRecordsInfinite` / `useFactoryRunsInfinite`.
+
+## Navigation
+
+| Nav item | Route(s) | Purpose |
+|----------|----------|---------|
+| Overview | `/` | Factory health, status counts, throughput chart, recent failed/running/pending runs |
+| Work | `/work`, `/work/new`, `/work/:id` | Ongoing (continuous) work definitions |
+| On-Demand | `/on-demand`, `/on-demand/single`, `/on-demand/batch`, `/on-demand/batch/:id` | Ad-hoc single-item and batch runs |
+| Information | `/information`, `/information/:recordId` | Browse Memorix information items (any schema-defined processable unit) |
+| Outputs | `/outputs`, `/outputs/:recordId` | Browse graph output collections and saved results |
+| Journal | `/journal`, `/journal/:id` | Factory run list and per-run inspection |
+| Pipeline analytics | `/pipelines`, `/pipelines/:graphId`, `/pipelines/entity/:entityName` | Aggregated performance by pipeline or entity |
+| Settings | `/settings` | Queue manager, concurrency, admission interval |
+| Item timeline (legacy) | `/items/:itemId` | Multi-run DAG timeline for a single item ID |
+
+### Legacy redirects
+
+Older bookmarks still resolve:
+
+| Old path | Redirects to |
+|----------|--------------|
+| `/records`, `/records/:id`, `/sources` | `/information` |
+| `/runs`, `/runs/:id` | `/journal` |
+| `/job-defs`, `/job-defs/:id` | `/work` |
+| `/jobs/new` | `/on-demand` |
+
+## Features by area
+
+### Overview (`/`)
+
+- Live status cards for all run states: blocked, pending, running, done, failed, cancelled — click a card to open Journal filtered by that status.
+- Filters: source type (continuous / on-demand single / on-demand batch), ongoing work, entity.
+- 24-hour throughput chart.
+- Preview columns: recent failed, currently running, oldest pending (with inline cancel for pending/blocked).
+- Subtitle shows job manager state (running/paused), active slot usage, and selected agent scope.
+
+### Ongoing work (`/work`)
+
+Continuous work definitions poll Memorix on a schedule, match items by narratives and processing rules, and enqueue factory runs automatically.
+
+**List** — name, status (active/inactive), entity, narratives, reprocessing mode, graph, re-scan interval, last scan, jobs created, last evaluation result. Actions: evaluate now, activate/deactivate.
+
+**Create wizard** (`/work/new`) — six steps:
+
+1. Basic info — name, description, entity
+2. Narratives — select information tags on the entity
+3. Prerequisites — reprocessing TTL/skip, exclude in-flight, required graphs, cross-entity prerequisites
+4. Graph — pipeline chooser with optional persistency override
+5. Execution policy — re-scan interval (5 min – 24 h), priority
+6. Review — create inactive or create and activate
+
+**Detail** (`/work/:id`) — tabs:
+
+- **Overview** — configuration, schedule, jobs created, estimated matching
+- **Evaluations** — history of each scan cycle (matched, created, excluded)
+- **Matching Information** — live preview of items that would match current rules
+- **Runs** — infinite-scroll journal filtered to this work definition
+- **Settings** — edit name, narratives, processing, graph, policy, persistency; exclusion panel for last evaluation
+
+### On-Demand (`/on-demand`)
 
-| Route | Page |
-|-------|------|
-| `/` | Queue Overview |
-| `/runs` | Runs list |
-| `/runs/:id` | Run detail |
-| `/sources` | Source picker |
-| `/records` | Records list |
-| `/records/:recordId` | Record detail |
-| `/items/:itemId` | Item timeline + DAG |
-| `/jobs/new` | Create / enqueue job |
-| `/job-defs` | Job definition list |
-| `/job-defs/:id` | Job definition detail |
-| `/settings` | App-wide queue execution settings |
+**Hub** — choose single item or batch mode.
 
-Work-factory routes: `/work`, `/work/:id`, `/batches/:id`, on-demand flows — see [`jobs-api` HTTP docs](../jobs-api/README.md).
+**Single item** (`/on-demand/single`) — seven-step wizard:
+
+1. Choose item — entity, collection, narratives, pick a specific item or a random sample
+2. Context — optional related records (ContentTypeAssembler)
+3. Choose graph — entity-compatible pipelines from Catalox
+4. Result writeback — simulation, graph default, same-entity inferences, or custom persistency
+5. Properties — assembly selection (which content types/fields feed the graph)
+6. Review & run — priority, summary
+7. Done — action result with link to journal entry
+
+Supports deep-link `?entity=` from Information detail.
+
+**Batch** (`/on-demand/batch`) — seven-step wizard:
+
+1. Entity
+2. Narratives
+3. Processing filters (reprocessing, exclude in-flight, prerequisites)
+4. Graph
+5. Review — matching count preview
+6. Queue policy — batch name, priority
+7. Queued — batch created; link to batch detail
+
+**Batch detail** (`/on-demand/batch/:id`) — status counts, selection summary, excluded items panel, infinite-scroll run list.
+
+### Information (`/information`)
+
+Browse Memorix items for an entity — see [Information items](#information-items-not-just-database-records) above. The default view is the entity’s **core** collection, but an “item” may represent a full object, a content slice, a message, a chat, or any other shape your schema defines.
+
+- Filters: entity, collection (core and related), narratives (chip multi-select), processed-by graph, processing status (succeeded / failed / running / queued / not-processed / unknown).
+- Infinite-scroll table with dynamic scalar columns from the first row.
+- Actions: run on-demand for entity, create batch from current filter.
+- Row click → item detail.
+
+**Item detail** (`/information/:recordId?entity=&collection=`) — tabs:
+
+- **Core** — id, entity, label, scalar fields, timestamps
+- **Narratives** — assigned tags with provenance
+- **Processing** — per-graph processing stamps and status
+- **Snapshots** — Memorix snapshot history
+- **Journal** — factory runs for this item (infinite scroll)
+
+Shortcut: **Run on-demand** pre-fills the single-item wizard.
+
+### Outputs (`/outputs`)
+
+Same browse UI as Information, scoped to `processing-output` collections.
+
+- Filter by entity and output collection.
+- **Clear collection** — typed confirmation (`{entity}-{collection}`) deletes all items in that output collection and clears linked stamps.
+- **Item detail** — Output tab (full JSON document), Journal tab; delete a single saved result.
+
+### Journal (`/journal`)
+
+Factory run list with infinite scroll.
+
+- Scope: total / today / this week / this month.
+- Filters: status, source type, ongoing work, batch, graph, entity, record ID.
+- Bulk cancel for pending/blocked runs (multi-select).
+- Status count chips from API.
+
+**Run detail** (`/journal/:id`) — tabs:
+
+- **Overview** — summary, source link (work or batch), processing result card, queue admission state, stale-running warning
+- **Console** — live event stream while running (auto-refresh 1.5s)
+- **Pipeline** — execution diagram and performance panel (duration, tokens, cost per node)
+- **Input & Output** — assembled input, graph result, Memorix writeback (`RunIoPanel`)
+- **Details** — raw metadata, assembly selection, related records, error trace
+
+Actions: **Re-run** (new journal entry), **Execute now** (inline worker), **Run next when slot available**, **Retry same entry**, **Cancel**.
+
+### Pipeline analytics (`/pipelines`)
+
+Two modes (tab switcher):
+
+- **By pipeline** — pick a graph; see aggregate runs, success rate, duration/token/cost stats per node and sub-step; execution diagram; performance guide.
+- **By entity** — roll up all graphs for an entity.
+
+Drill-down routes: `/pipelines/:graphId`, `/pipelines/entity/:entityName`.
+
+Links from run detail and journal rows open the matching pipeline view.
+
+### Settings (`/settings`)
+
+- **Job manager** card — pause/resume with running/pending/held counts.
+- **Queue execution** — max concurrent jobs, admission check interval (15s – 2 min). Saved via `PUT /api/settings`.
+
+### Global controls (top bar)
+
+- **Queue manager toggle** — pause/resume without leaving the current page.
+- **Agent scope** — filter multi-tenant Memorix agents; affects entity lists, work, information, journal, and pipeline entity mode.
+- **Runtime badge** — Mongo database and host from `GET /api/config`.
+
+## Shared UI building blocks
+
+| Component | Used for |
+|-----------|----------|
+| `Wizard` / `WizardSteps` | Multi-step create flows (work, on-demand) |
+| `GraphChooser` | Entity-compatible pipeline picker |
+| `NarrativeChecklist` | Narrative multi-select |
+| `CollectionSourcePicker` | Core vs auxiliary collections |
+| `MatchingPreview` | Count/sample of items matching selection rules |
+| `PrerequisitesEditor` | Reprocessing and cross-graph prerequisites |
+| `PersistencyOverrideEditor` / `ResultWritebackEditor` | Where graph results are saved |
+| `ContentTypeAssembler` | Related-record context for single runs |
+| `PipelineExecutionDiagram` / `PipelinePerformancePanel` | Run and aggregate analytics |
+| `ExclusionPanel` | Why items were skipped in evaluations/batches |
+| `InfiniteScrollFooter` | Paginated list footers |
+| `LiveConsole` | Streaming run events |
+| `DagView` | Item-level multi-run dependency graph (`/items/:itemId`) |
 
 ## Scripts
 
 ```bash
 npm run build       # vite build → dist/
-npm run dev         # Vite dev server + API proxy
-npm test            # build smoke test + graph-picker unit tests
+npm run dev         # API + queue worker + UI (full local stack)
+npm run dev:ui      # Vite only — requires jobs-api on :3099
+npm run dev:all     # same as npm run dev
+npm test            # build smoke test, graph-picker + pipeline-performance unit tests
+npm run test:watch  # vitest watch mode
 ```
 
+## API contract
+
+The UI expects the full jobs-api surface documented in [`jobs-api`](../jobs-api/README.md). Key groups used by the dashboard:
+
+- **Health / config** — `/api/health`, `/api/config`
+- **Queue settings** — `GET|PUT /api/settings`, `POST /api/queue/push`
+- **Factory runs** — list, detail, execute, retry, cancel, queue-next, activities
+- **Work factory** — work CRUD, evaluate, batches, on-demand single/batch
+- **Memorix records** — entities, narratives, core-records, clear/delete, processing stamps
+- **Selection** — matching preview, assembler properties/preview
+- **Stats** — throughput, stuck (overview/journal helpers)
+- **Agents proxy** — `/api/agents/*` for `AgentScopeSelect`
+
 ## License
 
 exellix-license