ultravisor 1.0.21 → 1.0.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/_version.json +7 -0
- package/docs/css/docuserve.css +277 -23
- package/docs/features/beacon-authentication.md +24 -31
- package/docs/features/beacon-providers.md +31 -37
- package/docs/features/beacons.md +20 -19
- package/docs/features/case-study-retold-remote.md +28 -28
- package/docs/features/llm-model-setup.md +15 -15
- package/docs/features/llm.md +29 -27
- package/docs/features/platform-cards.md +10 -10
- package/docs/features/reachability-matrix.md +12 -12
- package/docs/features/tasks-content-system.md +32 -32
- package/docs/features/tasks-data-transform.md +64 -64
- package/docs/features/tasks-extension.md +14 -14
- package/docs/features/tasks-file-system.md +94 -94
- package/docs/features/tasks-flow-control.md +38 -38
- package/docs/features/tasks-http-client.md +40 -40
- package/docs/features/tasks-llm.md +58 -58
- package/docs/features/tasks-meadow-api.md +50 -50
- package/docs/features/tasks-user-interaction.md +12 -12
- package/docs/features/tasks.md +20 -20
- package/docs/features/universal-addressing.md +12 -12
- package/docs/index.html +2 -2
- package/docs/retold-catalog.json +30 -1
- package/docs/retold-keyword-index.json +15389 -12741
- package/package.json +4 -3
- package/source/services/Ultravisor-Beacon-Coordinator.cjs +39 -0
- package/source/services/Ultravisor-OperationAuditor.cjs +471 -0
- package/source/web_server/Ultravisor-API-Server.cjs +54 -0
package/docs/features/beacons.md
CHANGED
|
@@ -14,24 +14,25 @@ bridge between Ultravisor's operation engine and the outside world.
|
|
|
14
14
|
|
|
15
15
|
The Beacon system uses the same **WaitingForInput** pause/resume mechanism
|
|
16
16
|
that powers interactive tasks like `value-input`. When a `beacon-dispatch`
|
|
17
|
-
node executes, it does not block the engine
|
|
17
|
+
node executes, it does not block the engine -- instead it enqueues a work
|
|
18
18
|
item and returns `WaitingForInput`. The operation pauses at that node until
|
|
19
19
|
a Beacon picks up the work, executes it remotely, and reports results.
|
|
20
20
|
|
|
21
21
|
The full cycle looks like this:
|
|
22
22
|
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
23
|
+
```mermaid
|
|
24
|
+
sequenceDiagram
|
|
25
|
+
participant OG as Operation Graph
|
|
26
|
+
participant BC as BeaconCoordinator<br/>(server-side)
|
|
27
|
+
participant B as Beacon<br/>(remote)
|
|
28
|
+
OG->>BC: beacon-dispatch (enqueue)
|
|
29
|
+
Note left of OG: Returns WaitingForInput<br/>(graph paused)
|
|
30
|
+
B->>BC: poll loop
|
|
31
|
+
BC->>B: assign work item
|
|
32
|
+
B->>B: execute locally
|
|
33
|
+
B->>BC: report result
|
|
34
|
+
BC->>OG: resumeOperation()
|
|
35
|
+
Note left of OG: Graph continues
|
|
35
36
|
```
|
|
36
37
|
|
|
37
38
|
### Transport Agnostic
|
|
@@ -46,7 +47,7 @@ channel without changing the core dispatch logic.
|
|
|
46
47
|
|
|
47
48
|
Many real-world pipelines require that related tasks execute on the same
|
|
48
49
|
worker. For example, a video processing pipeline might split a file across
|
|
49
|
-
multiple encoding passes
|
|
50
|
+
multiple encoding passes -- each pass needs access to the same local copy of
|
|
50
51
|
the source footage.
|
|
51
52
|
|
|
52
53
|
The **AffinityKey** setting on `beacon-dispatch` nodes solves this.
|
|
@@ -55,7 +56,7 @@ The **AffinityKey** setting on `beacon-dispatch` nodes solves this.
|
|
|
55
56
|
|
|
56
57
|
1. The first `beacon-dispatch` task with a given AffinityKey is claimed by
|
|
57
58
|
whichever Beacon picks it up. The coordinator records a binding:
|
|
58
|
-
`AffinityKey
|
|
59
|
+
`AffinityKey -> BeaconID`.
|
|
59
60
|
2. Subsequent tasks with the **same** AffinityKey are pre-assigned to that
|
|
60
61
|
same Beacon. When the Beacon polls, it receives affinity-assigned items
|
|
61
62
|
first.
|
|
@@ -102,7 +103,7 @@ Common capability names:
|
|
|
102
103
|
| `MLInference` | Run machine-learning model inference |
|
|
103
104
|
| `MediaProcessing` | Transcode video/audio, generate thumbnails |
|
|
104
105
|
|
|
105
|
-
You can define any capability string you like
|
|
106
|
+
You can define any capability string you like -- the coordinator treats them
|
|
106
107
|
as opaque labels for matching purposes.
|
|
107
108
|
|
|
108
109
|
## API Reference
|
|
@@ -437,7 +438,7 @@ these optional fields:
|
|
|
437
438
|
|
|
438
439
|
| Field | Type | Description |
|
|
439
440
|
|-------|------|-------------|
|
|
440
|
-
| `Percent` | number | Completion percentage (0
|
|
441
|
+
| `Percent` | number | Completion percentage (0-100) |
|
|
441
442
|
| `Message` | string | Human-readable status message |
|
|
442
443
|
| `Step` | number | Current step number |
|
|
443
444
|
| `TotalSteps` | number | Total number of steps |
|
|
@@ -478,7 +479,7 @@ Report progress on an in-flight work item.
|
|
|
478
479
|
|
|
479
480
|
| Field | Type | Required | Description |
|
|
480
481
|
|-------|------|----------|-------------|
|
|
481
|
-
| `Percent` | number | no | Completion percentage (0
|
|
482
|
+
| `Percent` | number | no | Completion percentage (0-100) |
|
|
482
483
|
| `Message` | string | no | Human-readable status message |
|
|
483
484
|
| `Step` | number | no | Current step number |
|
|
484
485
|
| `TotalSteps` | number | no | Total number of steps |
|
|
@@ -486,7 +487,7 @@ Report progress on an in-flight work item.
|
|
|
486
487
|
|
|
487
488
|
**Response:** `{ Success: true }`
|
|
488
489
|
|
|
489
|
-
Progress is fire-and-forget from the Beacon's perspective
|
|
490
|
+
Progress is fire-and-forget from the Beacon's perspective -- failures to
|
|
490
491
|
report progress do not affect work item execution.
|
|
491
492
|
|
|
492
493
|
## Running a Beacon
|
|
@@ -64,26 +64,26 @@ sequenceDiagram
|
|
|
64
64
|
|
|
65
65
|
RR->>UV: WebSocket: BeaconRegister<br/>{Name: "retold-remote", Contexts: {File: {BaseURL: "http://localhost:7827/content/"}},<br/>BindAddresses: [{IP: "127.0.0.1", Port: 7827}],<br/>Operations: [rr-image-thumbnail, rr-video-thumbnail, ...]}
|
|
66
66
|
UV->>UV: Register beacon, store 9 operation definitions
|
|
67
|
-
UV->>UV: Probe: retold-remote
|
|
67
|
+
UV->>UV: Probe: retold-remote <-> orator-conversion = REACHABLE
|
|
68
68
|
|
|
69
69
|
Note over UV: Both beacons registered.<br/>14 beacon task types available.<br/>9 operations from retold-remote.<br/>Reachability: all pairs reachable.
|
|
70
70
|
```
|
|
71
71
|
|
|
72
72
|
### 3. Auto-Generated Operations
|
|
73
73
|
|
|
74
|
-
retold-remote registers 9 operation definitions during beacon connection. These are complete operation graphs with nodes, connections, and state wiring
|
|
74
|
+
retold-remote registers 9 operation definitions during beacon connection. These are complete operation graphs with nodes, connections, and state wiring -- built programmatically by `_buildPipelineOperation()`:
|
|
75
75
|
|
|
76
76
|
| Operation | Trigger Parameters | Pipeline |
|
|
77
77
|
|-----------|-------------------|----------|
|
|
78
|
-
| `rr-image-thumbnail` | ImageAddress, Width, Height, Format, Quality | resolve
|
|
79
|
-
| `rr-video-thumbnail` | VideoAddress, Timestamp, Width | resolve
|
|
80
|
-
| `rr-video-frame-extraction` | VideoAddress, Timestamp, Width | resolve
|
|
81
|
-
| `rr-audio-waveform` | AudioAddress, SampleRate, Samples | resolve
|
|
82
|
-
| `rr-audio-segment` | AudioAddress, Start, Duration, Codec | resolve
|
|
83
|
-
| `rr-pdf-page-render` | PdfAddress, Page, LongSidePixels | resolve
|
|
84
|
-
| `rr-image-convert` | ImageAddress, Format, Quality | resolve
|
|
85
|
-
| `rr-ebook-convert` | EbookAddress | resolve
|
|
86
|
-
| `rr-media-probe` | MediaAddress | resolve
|
|
78
|
+
| `rr-image-thumbnail` | ImageAddress, Width, Height, Format, Quality | resolve -> transfer -> resize -> send-result |
|
|
79
|
+
| `rr-video-thumbnail` | VideoAddress, Timestamp, Width | resolve -> transfer -> extract frame -> send-result |
|
|
80
|
+
| `rr-video-frame-extraction` | VideoAddress, Timestamp, Width | resolve -> transfer -> probe -> extract -> send-result |
|
|
81
|
+
| `rr-audio-waveform` | AudioAddress, SampleRate, Samples | resolve -> transfer -> waveform -> send-result |
|
|
82
|
+
| `rr-audio-segment` | AudioAddress, Start, Duration, Codec | resolve -> transfer -> extract -> send-result |
|
|
83
|
+
| `rr-pdf-page-render` | PdfAddress, Page, LongSidePixels | resolve -> transfer -> render -> send-result |
|
|
84
|
+
| `rr-image-convert` | ImageAddress, Format, Quality | resolve -> transfer -> convert -> send-result |
|
|
85
|
+
| `rr-ebook-convert` | EbookAddress | resolve -> transfer -> ebook-convert -> send-result |
|
|
86
|
+
| `rr-media-probe` | MediaAddress | resolve -> transfer -> ffprobe -> send-result |
|
|
87
87
|
|
|
88
88
|
Each pipeline follows the same pattern:
|
|
89
89
|
|
|
@@ -102,9 +102,9 @@ graph LR
|
|
|
102
102
|
```
|
|
103
103
|
|
|
104
104
|
State connections wire outputs from earlier nodes into inputs of later nodes:
|
|
105
|
-
- `resolve.URL`
|
|
106
|
-
- `resolve.Filename`
|
|
107
|
-
- `transfer.LocalPath`
|
|
105
|
+
- `resolve.URL` -> `transfer.SourceURL`
|
|
106
|
+
- `resolve.Filename` -> `transfer.Filename`
|
|
107
|
+
- `transfer.LocalPath` -> `process.InputFile`
|
|
108
108
|
|
|
109
109
|
## End-to-End: Image Thumbnail Generation
|
|
110
110
|
|
|
@@ -119,13 +119,13 @@ sequenceDiagram
|
|
|
119
119
|
|
|
120
120
|
Browser->>RR: GET /content/preview/photo.jpg?w=400&h=300
|
|
121
121
|
|
|
122
|
-
Note over RR: Check cache
|
|
122
|
+
Note over RR: Check cache -> miss
|
|
123
123
|
|
|
124
124
|
RR->>UV: POST /Operation/rr-image-thumbnail/Trigger<br/>{Parameters: {ImageAddress: ">retold-remote/File/photo.jpg",<br/>Width: 400, Height: 300, Format: "webp", Quality: 80}}
|
|
125
125
|
|
|
126
126
|
Note over UV: Start operation (sync mode)
|
|
127
127
|
|
|
128
|
-
UV->>UV: resolve-address<br/>>retold-remote/File/photo.jpg<br
|
|
128
|
+
UV->>UV: resolve-address<br/>>retold-remote/File/photo.jpg<br/>-> http://localhost:7827/content/photo.jpg
|
|
129
129
|
|
|
130
130
|
UV->>RR: HTTP GET http://localhost:7827/content/photo.jpg
|
|
131
131
|
RR-->>UV: 200 OK (image bytes)
|
|
@@ -136,12 +136,12 @@ sequenceDiagram
|
|
|
136
136
|
OC->>UV: POST /Beacon/Work/Poll
|
|
137
137
|
UV-->>OC: WorkItem: {Action: "ImageResize",<br/>Settings: {InputFile: "/.../photo.jpg",<br/>OutputFile: "/.../thumbnail.jpg", Width: 400, ...}}
|
|
138
138
|
|
|
139
|
-
OC->>OC: Sharp: resize photo.jpg
|
|
139
|
+
OC->>OC: Sharp: resize photo.jpg -> thumbnail.jpg
|
|
140
140
|
OC->>UV: POST /Beacon/Work/{hash}/Upload<br/>(raw binary: thumbnail.jpg)
|
|
141
141
|
UV->>UV: Write to operation staging
|
|
142
142
|
|
|
143
143
|
OC->>UV: POST /Beacon/Work/{hash}/Complete
|
|
144
|
-
UV->>UV: resumeOperation
|
|
144
|
+
UV->>UV: resumeOperation -> send-result<br/>finds thumbnail.jpg in staging
|
|
145
145
|
|
|
146
146
|
UV-->>RR: 200 OK<br/>Content-Type: application/octet-stream<br/>Body: (thumbnail bytes)
|
|
147
147
|
|
|
@@ -184,9 +184,9 @@ this._dispatcher.triggerOperation('rr-image-thumbnail',
|
|
|
184
184
|
orator-conversion handles arbitrarily large images (including 256MB+ scans):
|
|
185
185
|
|
|
186
186
|
- **File-path mode**: Sharp receives the file path directly instead of loading the entire file into a buffer
|
|
187
|
-
- **No pixel limit**: `sharp(filePath, { limitInputPixels: false })`
|
|
187
|
+
- **No pixel limit**: `sharp(filePath, { limitInputPixels: false })` -- disables Sharp's default pixel limit
|
|
188
188
|
- **Streaming**: File transfer uses Node.js streams, avoiding full-file buffering
|
|
189
|
-
- **Binary upload**: Result files transfer as raw bytes over HTTP or WebSocket
|
|
189
|
+
- **Binary upload**: Result files transfer as raw bytes over HTTP or WebSocket -- no base64 encoding
|
|
190
190
|
|
|
191
191
|
## Error Handling
|
|
192
192
|
|
|
@@ -197,9 +197,9 @@ flowchart TD
|
|
|
197
197
|
A[Beacon processes work item] --> B{ExitCode == 0?}
|
|
198
198
|
B -->|Yes| C[Upload output file]
|
|
199
199
|
C --> D[Report completion]
|
|
200
|
-
D --> E[Operation resumes
|
|
200
|
+
D --> E[Operation resumes -> send-result -> binary stream]
|
|
201
201
|
B -->|No| F[Report error]
|
|
202
|
-
F --> G[Operation resumes
|
|
202
|
+
F --> G[Operation resumes -> Error event -> End node]
|
|
203
203
|
G --> H[Trigger returns JSON with Success: false]
|
|
204
204
|
H --> I[retold-remote falls back to local processing]
|
|
205
205
|
```
|
|
@@ -215,9 +215,9 @@ npm start -- -u -l # orator-conversion-2026-03-21T...log
|
|
|
215
215
|
```
|
|
216
216
|
|
|
217
217
|
Key log prefixes for tracing the pipeline:
|
|
218
|
-
- `[TriggerOp]`
|
|
219
|
-
- `[Trigger]`
|
|
220
|
-
- `[Engine]`
|
|
221
|
-
- `[Coordinator]`
|
|
222
|
-
- `[OratorConversion]`
|
|
223
|
-
- `[Beacon]`
|
|
218
|
+
- `[TriggerOp]` -- retold-remote dispatcher
|
|
219
|
+
- `[Trigger]` -- Ultravisor trigger endpoint
|
|
220
|
+
- `[Engine]` -- Ultravisor execution engine
|
|
221
|
+
- `[Coordinator]` -- Beacon coordinator (work queue, uploads)
|
|
222
|
+
- `[OratorConversion]` -- orator-conversion provider
|
|
223
|
+
- `[Beacon]` -- Beacon client (execution, upload, completion)
|
|
@@ -5,7 +5,7 @@ Each section covers installation, configuration, and verification for a
|
|
|
5
5
|
specific backend.
|
|
6
6
|
|
|
7
7
|
You do not need machine learning expertise to set this up. Each LLM
|
|
8
|
-
provider has its own API that the Beacon wraps
|
|
8
|
+
provider has its own API that the Beacon wraps -- you just need the
|
|
9
9
|
provider running and a configuration file pointing the Beacon at it.
|
|
10
10
|
|
|
11
11
|
## Prerequisites
|
|
@@ -64,7 +64,7 @@ Download the installer from https://ollama.com/download
|
|
|
64
64
|
# Start the Ollama server (runs in background)
|
|
65
65
|
ollama serve
|
|
66
66
|
|
|
67
|
-
# Pull a model
|
|
67
|
+
# Pull a model -- this downloads the model weights (may take a few minutes)
|
|
68
68
|
ollama pull llama3.2
|
|
69
69
|
|
|
70
70
|
# Verify it works
|
|
@@ -135,7 +135,7 @@ You should see:
|
|
|
135
135
|
[Beacon CLI] Loaded config from /path/to/.ultravisor-beacon.json
|
|
136
136
|
[LLM] Provider initialized: backend=ollama, model=llama3.2
|
|
137
137
|
[LLM] Ollama server reachable at localhost:11434
|
|
138
|
-
[ProviderRegistry] Registered "LLM"
|
|
138
|
+
[ProviderRegistry] Registered "LLM" -> LLM [ChatCompletion, Embedding, ToolUse]
|
|
139
139
|
[Beacon] Loaded 1 capability provider(s).
|
|
140
140
|
[Beacon] Capabilities: LLM
|
|
141
141
|
[Beacon] Registered as bcn-llm-ollama-...
|
|
@@ -144,12 +144,12 @@ You should see:
|
|
|
144
144
|
|
|
145
145
|
### Troubleshooting
|
|
146
146
|
|
|
147
|
-
- **"Ollama server not reachable"**
|
|
147
|
+
- **"Ollama server not reachable"** -- Make sure `ollama serve` is running.
|
|
148
148
|
Check with `curl http://localhost:11434/api/tags`.
|
|
149
|
-
- **Slow responses**
|
|
149
|
+
- **Slow responses** -- First request after pulling a model is slower
|
|
150
150
|
(loading into memory). Subsequent requests are faster. On CPU-only
|
|
151
151
|
machines, expect 5-30 seconds per response depending on model size.
|
|
152
|
-
- **Out of memory**
|
|
152
|
+
- **Out of memory** -- Use a smaller model. `llama3.2:3b` or `phi3` work
|
|
153
153
|
well on machines with 8GB RAM. The 70B models need 48GB+ RAM or a GPU
|
|
154
154
|
with equivalent VRAM.
|
|
155
155
|
|
|
@@ -166,7 +166,7 @@ an API key with billing set up.
|
|
|
166
166
|
2. Create an account or sign in
|
|
167
167
|
3. Navigate to API Keys
|
|
168
168
|
4. Create a new key
|
|
169
|
-
5. Copy the key
|
|
169
|
+
5. Copy the key -- it starts with `sk-ant-`
|
|
170
170
|
|
|
171
171
|
### Step 2: Set the API key as an environment variable
|
|
172
172
|
|
|
@@ -232,12 +232,12 @@ node source/beacon/Ultravisor-Beacon-CLI.cjs
|
|
|
232
232
|
|
|
233
233
|
### Troubleshooting
|
|
234
234
|
|
|
235
|
-
- **"401 Unauthorized"**
|
|
235
|
+
- **"401 Unauthorized"** -- Your API key is missing or invalid. Check
|
|
236
236
|
`echo $ANTHROPIC_API_KEY` to verify it's set.
|
|
237
|
-
- **"429 Too Many Requests"**
|
|
237
|
+
- **"429 Too Many Requests"** -- You've hit rate limits. Anthropic has
|
|
238
238
|
per-minute and per-day limits that vary by plan. Reduce `MaxConcurrent`
|
|
239
239
|
or add delays between requests.
|
|
240
|
-
- **"400 max_tokens"**
|
|
240
|
+
- **"400 max_tokens"** -- Anthropic requires `max_tokens` on every
|
|
241
241
|
request. The provider defaults to 4096 if not set, but some models
|
|
242
242
|
support higher values.
|
|
243
243
|
|
|
@@ -250,7 +250,7 @@ node source/beacon/Ultravisor-Beacon-CLI.cjs
|
|
|
250
250
|
1. Go to https://platform.openai.com/
|
|
251
251
|
2. Sign in and navigate to API Keys
|
|
252
252
|
3. Create a new secret key
|
|
253
|
-
4. Copy the key
|
|
253
|
+
4. Copy the key -- it starts with `sk-`
|
|
254
254
|
|
|
255
255
|
### Step 2: Set the API key
|
|
256
256
|
|
|
@@ -311,9 +311,9 @@ node source/beacon/Ultravisor-Beacon-CLI.cjs
|
|
|
311
311
|
|
|
312
312
|
### Troubleshooting
|
|
313
313
|
|
|
314
|
-
- **"401" or "invalid_api_key"**
|
|
314
|
+
- **"401" or "invalid_api_key"** -- Check that `OPENAI_API_KEY` is set
|
|
315
315
|
and has billing enabled. Free-tier keys have very limited access.
|
|
316
|
-
- **"model_not_found"**
|
|
316
|
+
- **"model_not_found"** -- Make sure you have access to the model. Some
|
|
317
317
|
models require specific account tiers.
|
|
318
318
|
|
|
319
319
|
---
|
|
@@ -444,7 +444,7 @@ source/beacon/
|
|
|
444
444
|
└── Ultravisor-Beacon-Provider-LLM.cjs
|
|
445
445
|
```
|
|
446
446
|
|
|
447
|
-
These files have zero npm dependencies
|
|
447
|
+
These files have zero npm dependencies -- they use only Node.js built-in
|
|
448
448
|
modules. Copy the `source/beacon/` directory to your remote machine,
|
|
449
449
|
create a `.ultravisor-beacon.json` config file, and run:
|
|
450
450
|
|
|
@@ -482,7 +482,7 @@ Or use the CLI:
|
|
|
482
482
|
node source/cli/Ultravisor-Run.cjs execute --operation llm-summarize
|
|
483
483
|
```
|
|
484
484
|
|
|
485
|
-
Watch the Beacon terminal
|
|
485
|
+
Watch the Beacon terminal -- you should see it pick up the work item,
|
|
486
486
|
make the LLM API call, and report back.
|
|
487
487
|
|
|
488
488
|
---
|
package/docs/features/llm.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# LLM Integration
|
|
2
2
|
|
|
3
3
|
Ultravisor can send requests to large language models (LLMs) through the
|
|
4
|
-
Beacon system. An LLM Beacon wraps an API
|
|
5
|
-
Ollama instance, or any OpenAI-compatible endpoint
|
|
4
|
+
Beacon system. An LLM Beacon wraps an API -- OpenAI, Anthropic, a local
|
|
5
|
+
Ollama instance, or any OpenAI-compatible endpoint -- and exposes it as a
|
|
6
6
|
capability that operation graphs can use like any other task.
|
|
7
7
|
|
|
8
8
|
This means you can build workflows that read data, send it to an LLM for
|
|
@@ -13,29 +13,31 @@ Ultravisor operation graph.
|
|
|
13
13
|
|
|
14
14
|
The LLM functionality has two layers:
|
|
15
15
|
|
|
16
|
-
1. **Beacon Provider** (`LLM`)
|
|
16
|
+
1. **Beacon Provider** (`LLM`) -- runs on the Beacon worker, wraps the
|
|
17
17
|
actual HTTP calls to the LLM API. This is the "thin wrapper" that
|
|
18
18
|
handles authentication, request formatting, and response normalization
|
|
19
19
|
across different backends.
|
|
20
20
|
|
|
21
21
|
2. **Graph Task Types** (`llm-chat-completion`, `llm-embedding`,
|
|
22
|
-
`llm-tool-use`)
|
|
22
|
+
`llm-tool-use`) -- run on the Ultravisor server, dispatch work to an
|
|
23
23
|
LLM Beacon, manage conversation history, and route results back into
|
|
24
24
|
the operation state.
|
|
25
25
|
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
26
|
+
```mermaid
|
|
27
|
+
sequenceDiagram
|
|
28
|
+
participant OG as Operation Graph
|
|
29
|
+
participant BC as BeaconCoordinator
|
|
30
|
+
participant LB as LLM Beacon
|
|
31
|
+
participant API as LLM API<br/>(OpenAI / Anthropic / Ollama)
|
|
32
|
+
OG->>BC: llm-chat-completion (enqueue)<br/>builds messages, manages history
|
|
33
|
+
Note left of OG: WaitingForInput...
|
|
34
|
+
LB->>BC: poll
|
|
35
|
+
BC->>LB: assign work
|
|
36
|
+
LB->>API: HTTP call
|
|
37
|
+
API-->>LB: response
|
|
38
|
+
LB->>BC: result
|
|
39
|
+
BC->>OG: resumeOperation()
|
|
40
|
+
Note left of OG: Updates history,<br/>writes to state
|
|
39
41
|
```
|
|
40
42
|
|
|
41
43
|
## Supported Backends
|
|
@@ -50,7 +52,7 @@ The LLM functionality has two layers:
|
|
|
50
52
|
The provider normalizes differences between backends automatically. For
|
|
51
53
|
example, Anthropic uses `x-api-key` headers and a different message
|
|
52
54
|
format; Ollama puts parameters in an `options` object. You don't need to
|
|
53
|
-
worry about these details
|
|
55
|
+
worry about these details -- just set the `Backend` config value and
|
|
54
56
|
provide your messages.
|
|
55
57
|
|
|
56
58
|
## Actions
|
|
@@ -157,7 +159,7 @@ The primary task for using LLMs in operation graphs.
|
|
|
157
159
|
| Name | Description |
|
|
158
160
|
|--------------------|----------------------------------------------------------|
|
|
159
161
|
| UserPrompt | User message text (builds Messages for you) |
|
|
160
|
-
| InputAddress | State address to read context data from
|
|
162
|
+
| InputAddress | State address to read context data from -- appended to UserPrompt |
|
|
161
163
|
| Destination | State address to write the completion content to |
|
|
162
164
|
| AffinityKey | Route to a specific Beacon |
|
|
163
165
|
| TimeoutMs | Override timeout (default: 120000) |
|
|
@@ -184,8 +186,8 @@ stored in operation or global state.
|
|
|
184
186
|
|----------------------------|--------------------------------------------------------|
|
|
185
187
|
| ConversationAddress | State address for the message history array |
|
|
186
188
|
| AppendToConversation | Append this exchange to history (default: true) |
|
|
187
|
-
| ConversationMaxMessages | Sliding window
|
|
188
|
-
| ConversationMaxTokens | Token budget
|
|
189
|
+
| ConversationMaxMessages | Sliding window -- keep only the N most recent messages |
|
|
190
|
+
| ConversationMaxTokens | Token budget -- trim oldest messages when exceeded |
|
|
189
191
|
| PersistConversation | Copy history to GlobalState on completion |
|
|
190
192
|
| ConversationPersistAddress | GlobalState address for cross-operation persistence |
|
|
191
193
|
|
|
@@ -201,17 +203,17 @@ When `ConversationAddress` is set, the task:
|
|
|
201
203
|
6. Appends the assistant response to the history
|
|
202
204
|
7. Writes updated history back to the state address
|
|
203
205
|
|
|
204
|
-
**Within a single operation**
|
|
206
|
+
**Within a single operation** -- use `Operation.` addresses:
|
|
205
207
|
|
|
206
208
|
```
|
|
207
209
|
llm-node-1 (ConversationAddress: "Operation.Chat")
|
|
208
|
-
|
|
209
|
-
|
|
210
|
+
-> llm-node-2 (ConversationAddress: "Operation.Chat")
|
|
211
|
+
-> llm-node-3 (ConversationAddress: "Operation.Chat")
|
|
210
212
|
```
|
|
211
213
|
|
|
212
214
|
Each node sees the full conversation so far and adds to it.
|
|
213
215
|
|
|
214
|
-
**Across operations**
|
|
216
|
+
**Across operations** -- use `Global.` addresses:
|
|
215
217
|
|
|
216
218
|
Set `ConversationAddress: "Global.Sessions.MyAgent"` to store history in
|
|
217
219
|
GlobalState, which persists across operation runs.
|
|
@@ -252,11 +254,11 @@ it up. Subsequent requests with the same key go to the same Beacon.
|
|
|
252
254
|
|
|
253
255
|
Two example operations are included in `operation-library/`:
|
|
254
256
|
|
|
255
|
-
- **llm-summarize.json**
|
|
257
|
+
- **llm-summarize.json** -- Reads a file, sends it to an LLM for
|
|
256
258
|
summarization, writes the result. Demonstrates single-turn usage with
|
|
257
259
|
`InputAddress` and `Destination`.
|
|
258
260
|
|
|
259
|
-
- **llm-multi-turn-analysis.json**
|
|
261
|
+
- **llm-multi-turn-analysis.json** -- Three chained LLM calls sharing
|
|
260
262
|
a `ConversationAddress`: initial analysis, follow-up question, and
|
|
261
263
|
final synthesis. Demonstrates multi-turn conversation with persistence
|
|
262
264
|
to GlobalState.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Platform Cards & Task Types
|
|
2
2
|
|
|
3
|
-
Ultravisor's operation graphs are built from cards (task types). Platform cards handle the infrastructure layer
|
|
3
|
+
Ultravisor's operation graphs are built from cards (task types). Platform cards handle the infrastructure layer -- address resolution, file transfer, and result delivery. Extension cards handle beacon dispatch. Beacon cards are auto-generated from registered beacon capabilities.
|
|
4
4
|
|
|
5
5
|
## Platform Cards
|
|
6
6
|
|
|
@@ -52,7 +52,7 @@ Encode a staging file to a base64 string or decode a base64 string to a staging
|
|
|
52
52
|
|
|
53
53
|
### beacon-dispatch
|
|
54
54
|
|
|
55
|
-
The generic card for dispatching work to beacon workers. Most users won't use this directly
|
|
55
|
+
The generic card for dispatching work to beacon workers. Most users won't use this directly -- catalog-generated beacon cards provide typed wrappers.
|
|
56
56
|
|
|
57
57
|
| Setting | Type | Required | Description |
|
|
58
58
|
|---------|------|----------|-------------|
|
|
@@ -79,7 +79,7 @@ When a beacon registers with action schemas, Ultravisor auto-generates typed tas
|
|
|
79
79
|
```mermaid
|
|
80
80
|
flowchart LR
|
|
81
81
|
A[Action Schema<br/>from beacon registration] --> B[_registerCatalogTaskTypes]
|
|
82
|
-
B --> C["Task type: beacon-mediaconversion-imageresize<br/>Settings: InputFile, OutputFile, Width, Height, Format, Quality<br/>Execute: beaconDispatch
|
|
82
|
+
B --> C["Task type: beacon-mediaconversion-imageresize<br/>Settings: InputFile, OutputFile, Width, Height, Format, Quality<br/>Execute: beaconDispatch -> enqueueWorkItem -> WaitingForInput"]
|
|
83
83
|
```
|
|
84
84
|
|
|
85
85
|
The naming convention is: `beacon-{capability}-{action}` (lowercased, non-alphanumeric replaced with hyphens).
|
|
@@ -90,10 +90,10 @@ The naming convention is: `beacon-{capability}-{action}` (lowercased, non-alphan
|
|
|
90
90
|
2. Coordinator's `_updateActionCatalog()` stores them persistently
|
|
91
91
|
3. `_registerCatalogTaskTypes()` iterates the catalog and creates a task type config for each entry
|
|
92
92
|
4. Each config gets an `Execute` function via `_createBeaconDispatchExecutor()` that:
|
|
93
|
-
- Coerces setting types (template-resolved strings
|
|
93
|
+
- Coerces setting types (template-resolved strings -> numbers/booleans per schema)
|
|
94
94
|
- Calls `beaconDispatch()` helper to enqueue the work item
|
|
95
95
|
- Returns `WaitingForInput` with `ResumeEventName: 'Complete'`
|
|
96
|
-
5. Built-in task types take precedence
|
|
96
|
+
5. Built-in task types take precedence -- catalog types are only registered if no type with the same hash exists
|
|
97
97
|
|
|
98
98
|
### Current orator-conversion Actions
|
|
99
99
|
|
|
@@ -116,7 +116,7 @@ The naming convention is: `beacon-{capability}-{action}` (lowercased, non-alphan
|
|
|
116
116
|
|
|
117
117
|
## Binary Output Return
|
|
118
118
|
|
|
119
|
-
When a beacon processes a work item that produces a file output, the result is transferred back to Ultravisor's staging directory as raw binary
|
|
119
|
+
When a beacon processes a work item that produces a file output, the result is transferred back to Ultravisor's staging directory as raw binary -- no base64 encoding.
|
|
120
120
|
|
|
121
121
|
```mermaid
|
|
122
122
|
sequenceDiagram
|
|
@@ -129,17 +129,17 @@ sequenceDiagram
|
|
|
129
129
|
Exec->>Exec: Download source file
|
|
130
130
|
Exec->>Beacon: provider.execute(action, workItem, context)
|
|
131
131
|
Beacon-->>Exec: { Outputs: { Result: outputPath }, Log }
|
|
132
|
-
Exec->>Exec: _collectOutputFiles
|
|
132
|
+
Exec->>Exec: _collectOutputFiles -> set OutputFilePath
|
|
133
133
|
|
|
134
134
|
alt HTTP Transport
|
|
135
135
|
Client->>UV: POST /Beacon/Work/{hash}/Upload<br/>Content-Type: application/octet-stream<br/>X-Output-Filename: thumbnail.jpg<br/>Body: raw bytes
|
|
136
|
-
UV->>UV: recordResultUpload
|
|
136
|
+
UV->>UV: recordResultUpload -> write to operation staging
|
|
137
137
|
else WebSocket Transport
|
|
138
138
|
Client->>UV: JSON: { Action: "WorkResultUpload", WorkItemHash, OutputFilename, OutputSize }
|
|
139
139
|
Client->>UV: Binary frame: raw file bytes
|
|
140
|
-
UV->>UV: recordResultUpload
|
|
140
|
+
UV->>UV: recordResultUpload -> write to operation staging
|
|
141
141
|
end
|
|
142
142
|
|
|
143
143
|
Client->>UV: JSON completion: { Outputs, Log }
|
|
144
|
-
UV->>UV: resumeOperation
|
|
144
|
+
UV->>UV: resumeOperation -> send-result finds file in staging
|
|
145
145
|
```
|
|
@@ -50,23 +50,23 @@ sequenceDiagram
|
|
|
50
50
|
UV->>UV: registerBeacon() stores BindAddresses
|
|
51
51
|
UV->>UV: onBeaconRegistered("bcn-orator-conversion-...")
|
|
52
52
|
|
|
53
|
-
Note over UV: Probe orator-conversion
|
|
53
|
+
Note over UV: Probe orator-conversion -> retold-remote
|
|
54
54
|
UV->>RR: HTTP GET http://127.0.0.1:7827/
|
|
55
55
|
RR-->>UV: 200 OK (any response = reachable)
|
|
56
|
-
UV->>UV: Matrix: orator-conversion
|
|
56
|
+
UV->>UV: Matrix: orator-conversion->retold-remote = REACHABLE (12ms)
|
|
57
57
|
|
|
58
|
-
Note over UV: Probe retold-remote
|
|
58
|
+
Note over UV: Probe retold-remote -> orator-conversion
|
|
59
59
|
UV->>OC: HTTP GET http://127.0.0.1:8765/
|
|
60
60
|
OC-->>UV: 200 OK
|
|
61
|
-
UV->>UV: Matrix: retold-remote
|
|
61
|
+
UV->>UV: Matrix: retold-remote->orator-conversion = REACHABLE (8ms)
|
|
62
62
|
```
|
|
63
63
|
|
|
64
64
|
### Probe Details
|
|
65
65
|
|
|
66
66
|
- **Method**: HTTP GET to the beacon's first BindAddress (`{Protocol}://{IP}:{Port}/`)
|
|
67
|
-
- **Success criteria**: Any HTTP response (even 404) counts as reachable
|
|
67
|
+
- **Success criteria**: Any HTTP response (even 404) counts as reachable -- we're testing network connectivity, not service correctness
|
|
68
68
|
- **Timeout**: 5 seconds per probe
|
|
69
|
-
- **Cache TTL**: 15 minutes
|
|
69
|
+
- **Cache TTL**: 15 minutes -- entries older than this are re-probed on next use
|
|
70
70
|
- **Trigger**: Probes fire on beacon registration and can be triggered manually via the API
|
|
71
71
|
|
|
72
72
|
### BindAddresses
|
|
@@ -79,7 +79,7 @@ BindAddresses: [
|
|
|
79
79
|
]
|
|
80
80
|
```
|
|
81
81
|
|
|
82
|
-
These are IP addresses, not DNS hostnames
|
|
82
|
+
These are IP addresses, not DNS hostnames -- this avoids NAT/split-horizon DNS complications. The reachability service uses the first BindAddress to construct probe URLs.
|
|
83
83
|
|
|
84
84
|
## Matrix Storage
|
|
85
85
|
|
|
@@ -97,7 +97,7 @@ Value: {
|
|
|
97
97
|
}
|
|
98
98
|
```
|
|
99
99
|
|
|
100
|
-
Probing is **directional**
|
|
100
|
+
Probing is **directional** -- A reaching B doesn't guarantee B can reach A (asymmetric firewalls, NAT).
|
|
101
101
|
|
|
102
102
|
## How resolve-address Uses the Matrix
|
|
103
103
|
|
|
@@ -107,7 +107,7 @@ When the resolve-address card receives a `RequestingBeaconID`, it consults the r
|
|
|
107
107
|
flowchart TD
|
|
108
108
|
A[resolve-address receives Address + RequestingBeaconID] --> B{Same beacon?}
|
|
109
109
|
B -->|Yes| C["Strategy: local<br/>URL: context BaseURL"]
|
|
110
|
-
B -->|No| D{Check matrix:<br/>requesting
|
|
110
|
+
B -->|No| D{Check matrix:<br/>requesting -> source}
|
|
111
111
|
D -->|reachable| E["Strategy: direct<br/>URL: source BindAddress + context path"]
|
|
112
112
|
D -->|unreachable / untested| F["Strategy: proxy<br/>URL: Ultravisor proxy endpoint"]
|
|
113
113
|
```
|
|
@@ -138,8 +138,8 @@ The Beacons screen includes a visual reachability map (SVG) showing:
|
|
|
138
138
|
- Beacons as labeled circles arranged in a ring
|
|
139
139
|
- Ultravisor as a center hub
|
|
140
140
|
- Lines between beacon pairs, color-coded:
|
|
141
|
-
- **Green solid**
|
|
142
|
-
- **Red dashed**
|
|
143
|
-
- **Grey dotted**
|
|
141
|
+
- **Green solid** -- Direct connectivity confirmed
|
|
142
|
+
- **Red dashed** -- Unreachable
|
|
143
|
+
- **Grey dotted** -- Untested
|
|
144
144
|
- Hover tooltip with latency and last probe time
|
|
145
145
|
- "Probe All" button to trigger a fresh probe sweep
|