ultravisor 1.0.0 → 1.0.3

package/docs/features/cli.md

@@ -0,0 +1,182 @@
+# CLI Commands
+
+Ultravisor provides a command-line interface for managing tasks, operations,
+schedules, and the API server. The CLI is invoked via the `ultravisor`
+command (or `node source/cli/Ultravisor-Run.cjs`).
+
+## Command Reference
+
+### start
+
+Start the API server.
+
+```bash
+ultravisor start
+ultravisor start --verbose
+```
+
+| Option | Description |
+|--------|-------------|
+| `-v, --verbose` | Enable verbose console output |
+
+The server starts on the port configured in `.ultravisor.json`
+(`UltravisorAPIServerPort`, default `54321`).
+
+### stop
+
+Stop the Hypervisor scheduler. Deactivates all running cron jobs.
+
+```bash
+ultravisor stop
+```
+
+### updatetask
+
+Add or update a task definition. The task is persisted to
+`.ultravisor.json`.
+
+```bash
+ultravisor updatetask -g disk-check -n "Check Disk" -t Command -p "df -h"
+```
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-g, --guid` | Task GUID | (none) |
+| `-c, --code` | Task code | (none) |
+| `-n, --name` | Task name | (none) |
+| `-t, --type` | Task type | `CRON` |
+| `-p, --parameters` | Task parameters | `0 0 * * * *` |
+| `-f, --file` | Path to JSON task definition file | (none) |
+
+When `-f` is provided, the JSON file is loaded and then any CLI parameters
+override the file values:
+
+```bash
+# Load from file, but override the GUID
+ultravisor updatetask -f ./my-task.json -g custom-guid
+```
+
+Example JSON file:
+
+```json
+{
+    "GUIDTask": "api-health",
+    "Name": "API Health Check",
+    "Type": "Request",
+    "URL": "https://api.example.com/health"
+}
+```
+
+### singletask (alias: task)
+
+Execute a single task immediately.
+
+```bash
+ultravisor singletask disk-check
+ultravisor task disk-check
+ultravisor singletask disk-check --dry_run
+```
+
+| Argument | Description |
+|----------|-------------|
+| `<task>` | The task GUID to execute |
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-o, --operation` | Scope task to an operation | `Default` |
+| `-d, --dry_run` | Print what would happen without executing | `false` |
+
+### singleoperation (alias: operation)
+
+Execute a single operation immediately.
+
+```bash
+ultravisor singleoperation etl-pipeline
+ultravisor operation etl-pipeline
+ultravisor singleoperation etl-pipeline --dry_run
+```
+
+| Argument | Description |
+|----------|-------------|
+| `<operation>` | The operation GUID to execute |
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-d, --dry_run` | Print what would happen without executing | `false` |
+
+### schedule (alias: cal)
+
+View the current schedule.
+
+```bash
+ultravisor schedule
+ultravisor cal
+```
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-f, --format` | Visualization format (day, week, month) | `day` |
+
+Example output:
+
+```
+=== Ultravisor Schedule (2 entries) ===
+
+  [INACTIVE] Task: disk-check
+           Schedule: cron (*/5 * * * *)
+           GUID: sched-task-disk-check-1707566400000
+
+  [INACTIVE] Operation: etl-pipeline
+           Schedule: daily (0 2 * * *)
+           GUID: sched-op-etl-pipeline-1707566400001
+```
+
+### schedule_task (alias: st)
+
+Add a task to the schedule.
+
+```bash
+ultravisor schedule_task disk-check -t cron -p "*/5 * * * *"
+ultravisor st disk-check -t hourly
+```
+
+| Argument | Description |
+|----------|-------------|
+| `<task_guid>` | The task GUID to schedule |
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-t, --type` | Schedule type (cron, daily, hourly, solver) | `cron` |
+| `-p, --parameters` | Cron expression or schedule parameters | (empty) |
+
+### schedule_operation (alias: so)
+
+Add an operation to the schedule.
+
+```bash
+ultravisor schedule_operation etl-pipeline -t daily -p "0 2 * * *"
+ultravisor so etl-pipeline -t cron -p "0 6 * * 1-5"
+```
+
+| Argument | Description |
+|----------|-------------|
+| `<operation_guid>` | The operation GUID to schedule |
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-t, --type` | Schedule type (cron, daily, hourly, solver) | `cron` |
+| `-p, --parameters` | Cron expression or schedule parameters | (empty) |
+
+### configuration
+
+Auto-generated command that explains how Ultravisor resolved its
+configuration (which files were loaded, what values came from where).
+
+```bash
+ultravisor configuration
+```
+
+## Exit Codes
+
+- `0` -- command completed successfully
+- Non-zero -- an error occurred during execution

package/docs/features/configuration.md

@@ -0,0 +1,245 @@
+# Configuration
+
+Ultravisor uses a layered JSON configuration system. Configuration is
+gathered automatically at startup and merged into a single object.
+
+## Configuration File
+
+The primary configuration file is `.ultravisor.json`, located in the
+working directory. This file stores both runtime settings and persisted
+task/operation definitions.
+
+### Minimal Configuration
+
+```json
+{
+    "UltravisorAPIServerPort": 54321
+}
+```
+
+### Full Configuration
+
+```json
+{
+    "UltravisorAPIServerPort": 54321,
+    "UltravisorFileStorePath": "/var/data/ultravisor_datastore",
+    "UltravisorStagingRoot": "/var/data/ultravisor_staging",
+    "UltravisorTickIntervalMilliseconds": 60000,
+    "Tasks": {
+        "my-task": {
+            "GUIDTask": "my-task",
+            "Name": "My Task",
+            "Type": "Command",
+            "Command": "echo hello"
+        }
+    },
+    "Operations": {
+        "my-op": {
+            "GUIDOperation": "my-op",
+            "Name": "My Operation",
+            "Tasks": ["my-task"]
+        }
+    }
+}
+```
+
+## Configuration Options
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| `UltravisorAPIServerPort` | Number | `54321` | Port for the REST API server |
+| `UltravisorFileStorePath` | String | `${cwd}/dist/ultravisor_datastore` | Path for the output file store |
+| `UltravisorStagingRoot` | String | `${cwd}/dist/ultravisor_staging` | Root folder for per-operation staging directories |
+| `UltravisorTickIntervalMilliseconds` | Number | `60000` | Base tick interval in milliseconds |
+| `UltravisorCommandTimeoutMilliseconds` | Number | `300000` | Timeout for command task execution (5 minutes) |
+| `UltravisorCommandMaxBufferBytes` | Number | `10485760` | Max stdout/stderr buffer size for commands (10 MB) |
+| `Tasks` | Object | `{}` | Map of task GUIDs to task definitions |
+| `Operations` | Object | `{}` | Map of operation GUIDs to operation definitions |
+
+## Configuration Layering
+
+Configuration is resolved in order (later layers override earlier):
+
+1. **Default Program Configuration** -- hardcoded defaults in
+   `source/config/Ultravisor-Default-Command-Configuration.cjs`:
+
+   ```json
+   {
+       "UltravisorAPIServerPort": 54321,
+       "UltravisorFileStorePath": "${cwd}/dist/ultravisor_datastore",
+       "UltravisorStagingRoot": "${cwd}/dist/ultravisor_staging",
+       "UltravisorTickIntervalMilliseconds": 60000,
+       "UltravisorCommandTimeoutMilliseconds": 300000,
+       "UltravisorCommandMaxBufferBytes": 10485760
+   }
+   ```
+
+2. **`.ultravisor.json`** -- project-level file searched from the current
+   working directory upward through parent directories
+
+The merged result is available as `fable.ProgramConfiguration` within
+services.
+
+## Viewing Configuration Resolution
+
+Use the built-in `configuration` command to see how Ultravisor resolved
+its configuration:
+
+```bash
+ultravisor configuration
+```
+
+This shows each gather phase, what file was loaded, and what values came
+from where.
+
+## State Persistence
+
+When tasks or operations are created/updated via the CLI (`updatetask`)
+or the API (`POST /Task`, `POST /Operation`), the state service merges
+the changes back into the `.ultravisor.json` file.
+
+The merge logic:
+- The existing file is read
+- New or updated tasks/operations are merged (existing fields preserved,
+  new fields added)
+- The entire configuration is written back to the file
+
+This means the `.ultravisor.json` file serves as both configuration and
+persistent storage.
+
+## Example Configurations
+
+### Simple API monitoring
+
+```json
+{
+    "UltravisorAPIServerPort": 8080,
+    "Tasks": {
+        "health-check": {
+            "GUIDTask": "health-check",
+            "Name": "API Health Check",
+            "Type": "Request",
+            "URL": "https://api.example.com/health"
+        }
+    },
+    "Operations": {}
+}
+```
+
+### Multi-task data pipeline
+
+```json
+{
+    "UltravisorAPIServerPort": 54321,
+    "UltravisorFileStorePath": "/data/ultravisor",
+    "Tasks": {
+        "extract": {
+            "GUIDTask": "extract",
+            "Name": "Extract Data",
+            "Type": "Command",
+            "Command": "python3 /scripts/extract.py --output /tmp/extract.json"
+        },
+        "transform": {
+            "GUIDTask": "transform",
+            "Name": "Transform Data",
+            "Type": "Command",
+            "Command": "python3 /scripts/transform.py --input /tmp/extract.json --output /tmp/transformed.json"
+        },
+        "load": {
+            "GUIDTask": "load",
+            "Name": "Load Data",
+            "Type": "Command",
+            "Command": "python3 /scripts/load.py --input /tmp/transformed.json"
+        }
+    },
+    "Operations": {
+        "etl": {
+            "GUIDOperation": "etl",
+            "Name": "Full ETL Pipeline",
+            "Tasks": ["extract", "transform", "load"]
+        }
+    }
+}
+```
+
+### IoT sensor monitoring
+
+```json
+{
+    "UltravisorAPIServerPort": 9000,
+    "Tasks": {
+        "read-temp-living": {
+            "GUIDTask": "read-temp-living",
+            "Name": "Read Living Room Temperature",
+            "Type": "Request",
+            "URL": "http://192.168.1.100/api/temperature"
+        },
+        "read-temp-bedroom": {
+            "GUIDTask": "read-temp-bedroom",
+            "Name": "Read Bedroom Temperature",
+            "Type": "Request",
+            "URL": "http://192.168.1.101/api/temperature"
+        },
+        "read-temp-garage": {
+            "GUIDTask": "read-temp-garage",
+            "Name": "Read Garage Temperature",
+            "Type": "Request",
+            "URL": "http://192.168.1.102/api/temperature"
+        },
+        "log-readings": {
+            "GUIDTask": "log-readings",
+            "Name": "Log Temperature Readings",
+            "Type": "Command",
+            "Command": "python3 /scripts/log_temps.py"
+        }
+    },
+    "Operations": {
+        "temp-sweep": {
+            "GUIDOperation": "temp-sweep",
+            "Name": "Temperature Sweep",
+            "Tasks": [
+                "read-temp-living",
+                "read-temp-bedroom",
+                "read-temp-garage",
+                "log-readings"
+            ]
+        }
+    }
+}
+```
+
+### Media processing
+
+```json
+{
+    "UltravisorAPIServerPort": 54321,
+    "UltravisorFileStorePath": "/media/output",
+    "Tasks": {
+        "scan-input": {
+            "GUIDTask": "scan-input",
+            "Name": "Scan Input Directory",
+            "Type": "Command",
+            "Command": "ls -1 /media/input/*.avi > /tmp/pending_files.txt"
+        },
+        "transcode-batch": {
+            "GUIDTask": "transcode-batch",
+            "Name": "Transcode Video Batch",
+            "Type": "Command",
+            "Command": "bash /scripts/transcode_batch.sh /tmp/pending_files.txt /media/output"
+        },
+        "generate-thumbnails": {
+            "GUIDTask": "generate-thumbnails",
+            "Name": "Generate Thumbnails",
+            "Type": "Command",
+            "Command": "bash /scripts/thumbnails.sh /media/output"
+        }
+    },
+    "Operations": {
+        "media-ingest": {
+            "GUIDOperation": "media-ingest",
+            "Name": "Media Ingest Pipeline",
+            "Tasks": ["scan-input", "transcode-batch", "generate-thumbnails"]
+        }
+    }
+}
+```

package/docs/features/manifests.md

@@ -0,0 +1,177 @@
+# Manifests (Metaoutput)
+
+Manifests track the output from executing operations. Every operation
+execution produces a manifest that records timing, success/failure, task
+results and logs.
+
+## Manifest Model
+
+```json
+{
+    "GUIDOperation": "etl-pipeline",
+    "GUIDRun": "etl-pipeline-1707566400000",
+    "Name": "ETL Pipeline",
+    "StagingPath": "/var/data/ultravisor_staging/etl-pipeline",
+    "ManifestFilePath": "/var/data/ultravisor_staging/etl-pipeline/Manifest_etl-pipeline.json",
+    "StartTime": "2026-02-10T02:00:00.000Z",
+    "StopTime": "2026-02-10T02:00:12.500Z",
+    "Status": "Complete",
+    "Success": true,
+    "Summary": "Operation etl-pipeline Complete: 3 task(s) executed.",
+    "TaskResults": [
+        {
+            "GUIDTask": "pull-data",
+            "Name": "Pull Data from API",
+            "Type": "Request",
+            "StartTime": "2026-02-10T02:00:00.100Z",
+            "StopTime": "2026-02-10T02:00:03.200Z",
+            "Status": "Complete",
+            "Success": true,
+            "Output": "{\"records\": 150}",
+            "Log": ["..."]
+        },
+        {
+            "GUIDTask": "process-data",
+            "Name": "Process Downloaded Data",
+            "Type": "Command",
+            "StartTime": "2026-02-10T02:00:03.300Z",
+            "StopTime": "2026-02-10T02:00:10.100Z",
+            "Status": "Complete",
+            "Success": true,
+            "Output": "Processed 150 records\n",
+            "Log": ["..."]
+        },
+        {
+            "GUIDTask": "notify",
+            "Name": "Send Notification",
+            "Type": "Command",
+            "StartTime": "2026-02-10T02:00:10.200Z",
+            "StopTime": "2026-02-10T02:00:12.400Z",
+            "Status": "Complete",
+            "Success": true,
+            "Output": "",
+            "Log": ["..."]
+        }
+    ],
+    "Log": [
+        "Operation etl-pipeline started at 2026-02-10T02:00:00.000Z",
+        "Task pull-data completed with status: Complete",
+        "Task process-data completed with status: Complete",
+        "Task notify completed with status: Complete",
+        "Operation etl-pipeline Complete: 3 task(s) executed."
+    ]
+}
+```
+
+### Fields
+
+| Field | Description |
+|-------|-------------|
+| `GUIDOperation` | The operation that was executed |
+| `GUIDRun` | Unique ID for this specific execution run |
+| `Name` | Operation name |
+| `StartTime` | ISO timestamp when the operation started |
+| `StopTime` | ISO timestamp when the operation finished |
+| `Status` | `Running`, `Complete`, or `Error` |
+| `Success` | `true` if all tasks succeeded, `false` otherwise |
+| `Summary` | Human-readable one-line summary |
+| `StagingPath` | Absolute path to the operation's staging folder |
+| `ManifestFilePath` | Absolute path to the manifest JSON file on disk |
+| `TaskResults` | Array of task manifest entries (see below) |
+| `Log` | Array of log messages from the operation lifecycle |
+
+### Task Result Fields
+
+| Field | Description |
+|-------|-------------|
+| `GUIDTask` | The task that was executed |
+| `Name` | Task name |
+| `Type` | Task type (Command, Request, etc.) |
+| `StartTime` | ISO timestamp when the task started |
+| `StopTime` | ISO timestamp when the task finished |
+| `Status` | `Running`, `Complete`, `Error`, or `Unsupported` |
+| `Success` | `true` if the task succeeded |
+| `Output` | stdout or response body from the execution |
+| `Log` | Array of log messages from the task execution |
+
+## Manifest Lifecycle
+
+1. **Create** -- `createManifest()` initializes a new manifest with status
+   `Running` and assigns a `GUIDRun`
+2. **Add Results** -- as each task completes, `addTaskResult()` appends the
+   task result to `TaskResults`
+3. **Finalize** -- `finalizeManifest()` sets `StopTime`, calculates overall
+   `Success` (all tasks must succeed), sets `Status` and `Summary`, and
+   writes the manifest JSON to the operation's staging folder
+
+## Success Determination
+
+An operation manifest is marked as `Success: true` only if **every** task
+in the `TaskResults` array has `Success: true`. If any task has
+`Success: false`, the entire operation is marked as failed.
+
+## Storage
+
+Manifests are stored in two places:
+
+1. **In memory** -- for the duration of the server session, accessible via
+   the API. Restarting the server clears the in-memory manifest store.
+
+2. **On disk** -- when an operation completes, a
+   `Manifest_{GUIDOperation}.json` file is written to the operation's
+   staging folder. This provides a persistent record that survives
+   restarts. The file path is recorded in the manifest's
+   `ManifestFilePath` field.
+
+The on-disk manifest contains the complete manifest object including all
+task results, timing data and logs. It is written as pretty-printed JSON
+(4-space indentation).
+
+The staging folder location defaults to
+`./dist/ultravisor_staging/{GUIDOperation}/` and can be configured via
+`UltravisorStagingRoot` in `.ultravisor.json` (see
+[Configuration](configuration.md)).
+
+## Accessing Manifests
+
+### Via API
+
+```bash
+# List all manifests
+curl http://localhost:54321/Manifest
+
+# Get a specific manifest
+curl http://localhost:54321/Manifest/etl-pipeline-1707566400000
+```
+
+### Example: Checking if the last run succeeded
+
+```bash
+# Get the manifest list and check the last entry
+curl -s http://localhost:54321/Manifest | jq '.[-1].Success'
+```
+
+## Example: Partial Failure
+
+When one task fails in an operation, the manifest captures it:
+
+```json
+{
+    "GUIDOperation": "data-sync",
+    "Status": "Error",
+    "Success": false,
+    "Summary": "Operation data-sync Error: 2 task(s) executed.",
+    "TaskResults": [
+        {
+            "GUIDTask": "fetch-data",
+            "Status": "Complete",
+            "Success": true
+        },
+        {
+            "GUIDTask": "upload-data",
+            "Status": "Error",
+            "Success": false
+        }
+    ]
+}
+```