ultravisor 1.0.3 → 1.0.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (248) hide show
  1. package/.claude/launch.json +11 -0
  2. package/.claude/ultravisor-dev-config.json +3 -0
  3. package/card-audit.json +1545 -0
  4. package/docs/_sidebar.md +17 -0
  5. package/docs/css/docuserve.css +73 -0
  6. package/docs/features/beacon-authentication.md +209 -0
  7. package/docs/features/beacon-providers.md +1299 -0
  8. package/docs/features/beacons.md +530 -0
  9. package/docs/features/capabilities.md +193 -0
  10. package/docs/features/llm-model-setup.md +505 -0
  11. package/docs/features/llm.md +262 -0
  12. package/docs/features/tasks-content-system.md +110 -0
  13. package/docs/features/tasks-data-transform.md +233 -0
  14. package/docs/features/tasks-extension.md +36 -0
  15. package/docs/features/tasks-file-system.md +203 -0
  16. package/docs/features/tasks-flow-control.md +125 -0
  17. package/docs/features/tasks-http-client.md +119 -0
  18. package/docs/features/tasks-llm.md +118 -0
  19. package/docs/features/tasks-meadow-api.md +163 -0
  20. package/docs/features/tasks-user-interaction.md +49 -0
  21. package/docs/features/tasks.md +14 -0
  22. package/docs/index.html +1 -1
  23. package/docs/retold-catalog.json +1 -1
  24. package/docs/retold-keyword-index.json +17374 -5
  25. package/operation-library/api-data-pipeline.json +143 -0
  26. package/operation-library/beacon-remote-command.json +72 -0
  27. package/operation-library/conditional-file-backup.json +143 -0
  28. package/operation-library/config-processor.json +136 -0
  29. package/operation-library/csv-data-analysis.json +228 -0
  30. package/operation-library/csv-to-json.json +160 -0
  31. package/operation-library/directory-inventory.json +128 -0
  32. package/operation-library/expression-calculator.json +114 -0
  33. package/operation-library/file-search-replace.json +175 -0
  34. package/operation-library/git-status-report.json +368 -0
  35. package/operation-library/health-check-ping.json +329 -0
  36. package/operation-library/json-config-merge.json +313 -0
  37. package/operation-library/llm-embedding-tool-use.json +314 -0
  38. package/operation-library/llm-multi-turn-analysis.json +322 -0
  39. package/operation-library/llm-summarize.json +264 -0
  40. package/operation-library/log-line-counter.json +185 -0
  41. package/operation-library/meadow-crud-lifecycle.json +264 -0
  42. package/operation-library/npm-project-validator.json +193 -0
  43. package/operation-library/rest-api-orchestrator.json +218 -0
  44. package/operation-library/shell-system-info.json +137 -0
  45. package/operation-library/simple-echo.json +95 -0
  46. package/operation-library/template-processor.json +299 -0
  47. package/operation-library/text-sanitizer.json +176 -0
  48. package/package.json +12 -8
  49. package/source/Ultravisor.cjs +30 -3
  50. package/source/beacon/Ultravisor-Beacon-CLI.cjs +143 -0
  51. package/source/beacon/Ultravisor-Beacon-CapabilityProvider.cjs +129 -0
  52. package/source/beacon/Ultravisor-Beacon-Client.cjs +568 -0
  53. package/source/beacon/Ultravisor-Beacon-Executor.cjs +500 -0
  54. package/source/beacon/Ultravisor-Beacon-ProviderRegistry.cjs +330 -0
  55. package/source/beacon/providers/Ultravisor-Beacon-Provider-FileSystem.cjs +331 -0
  56. package/source/beacon/providers/Ultravisor-Beacon-Provider-LLM.cjs +966 -0
  57. package/source/beacon/providers/Ultravisor-Beacon-Provider-Shell.cjs +95 -0
  58. package/source/cli/Ultravisor-CLIProgram.cjs +44 -22
  59. package/source/cli/commands/Ultravisor-Command-SingleOperation.cjs +29 -18
  60. package/source/cli/commands/Ultravisor-Command-SingleTask.cjs +62 -19
  61. package/source/cli/commands/Ultravisor-Command-UpdateTask.cjs +27 -15
  62. package/source/config/Ultravisor-Default-Command-Configuration.cjs +12 -3
  63. package/source/services/Ultravisor-Beacon-Coordinator.cjs +966 -0
  64. package/source/services/Ultravisor-ExecutionEngine.cjs +1093 -0
  65. package/source/services/Ultravisor-ExecutionManifest.cjs +629 -0
  66. package/source/services/Ultravisor-Hypervisor-State.cjs +270 -97
  67. package/source/services/Ultravisor-Hypervisor.cjs +185 -75
  68. package/source/services/Ultravisor-Schedule-Persistence-Base.cjs +45 -0
  69. package/source/services/Ultravisor-StateManager.cjs +253 -0
  70. package/source/services/Ultravisor-TaskTypeRegistry.cjs +213 -0
  71. package/source/services/persistence/Ultravisor-Schedule-Persistence-JSONFile.cjs +140 -0
  72. package/source/services/tasks/Ultravisor-BuiltIn-TaskConfigs.cjs +19 -0
  73. package/source/services/tasks/Ultravisor-TaskHelper-BeaconDispatch.cjs +107 -0
  74. package/source/services/tasks/Ultravisor-TaskType-Base.cjs +125 -0
  75. package/source/services/tasks/content-system/Ultravisor-TaskConfigs-ContentSystem.cjs +90 -0
  76. package/source/services/tasks/content-system/definitions/content-create-folder.json +24 -0
  77. package/source/services/tasks/content-system/definitions/content-list-files.json +27 -0
  78. package/source/services/tasks/content-system/definitions/content-read-file.json +26 -0
  79. package/source/services/tasks/content-system/definitions/content-save-file.json +26 -0
  80. package/source/services/tasks/data-transform/Ultravisor-TaskConfigs-DataTransform.cjs +577 -0
  81. package/source/services/tasks/data-transform/Ultravisor-TaskType-ReplaceString.cjs +80 -0
  82. package/source/services/tasks/data-transform/Ultravisor-TaskType-SetValues.cjs +74 -0
  83. package/source/services/tasks/data-transform/Ultravisor-TaskType-StringAppender.cjs +90 -0
  84. package/source/services/tasks/data-transform/definitions/comprehension-intersect.json +24 -0
  85. package/source/services/tasks/data-transform/definitions/csv-transform.json +24 -0
  86. package/source/services/tasks/data-transform/definitions/expression-solver.json +20 -0
  87. package/source/services/tasks/data-transform/definitions/histogram.json +23 -0
  88. package/source/services/tasks/data-transform/definitions/parse-csv.json +27 -0
  89. package/source/services/tasks/data-transform/definitions/replace-string.json +27 -0
  90. package/source/services/tasks/data-transform/definitions/set-values.json +17 -0
  91. package/source/services/tasks/data-transform/definitions/string-appender.json +22 -0
  92. package/source/services/tasks/data-transform/definitions/template-string.json +20 -0
  93. package/source/services/tasks/extension/Ultravisor-TaskConfigs-Extension.cjs +87 -0
  94. package/source/services/tasks/extension/definitions/beacon-dispatch.json +32 -0
  95. package/source/services/tasks/file-system/Ultravisor-TaskConfigs-FileSystem.cjs +509 -0
  96. package/source/services/tasks/file-system/Ultravisor-TaskType-ReadFile.cjs +96 -0
  97. package/source/services/tasks/file-system/Ultravisor-TaskType-ReadFileBuffered.cjs +123 -0
  98. package/source/services/tasks/file-system/Ultravisor-TaskType-WriteFile.cjs +106 -0
  99. package/source/services/tasks/file-system/definitions/copy-file.json +27 -0
  100. package/source/services/tasks/file-system/definitions/list-files.json +27 -0
  101. package/source/services/tasks/file-system/definitions/read-file-buffered.json +31 -0
  102. package/source/services/tasks/file-system/definitions/read-file.json +26 -0
  103. package/source/services/tasks/file-system/definitions/read-json.json +23 -0
  104. package/source/services/tasks/file-system/definitions/write-file.json +29 -0
  105. package/source/services/tasks/file-system/definitions/write-json.json +30 -0
  106. package/source/services/tasks/flow-control/Ultravisor-TaskConfigs-FlowControl.cjs +353 -0
  107. package/source/services/tasks/flow-control/Ultravisor-TaskType-IfConditional.cjs +125 -0
  108. package/source/services/tasks/flow-control/Ultravisor-TaskType-LaunchOperation.cjs +186 -0
  109. package/source/services/tasks/flow-control/Ultravisor-TaskType-SplitExecute.cjs +164 -0
  110. package/source/services/tasks/flow-control/definitions/if-conditional.json +25 -0
  111. package/source/services/tasks/flow-control/definitions/launch-operation.json +27 -0
  112. package/source/services/tasks/flow-control/definitions/split-execute.json +32 -0
  113. package/source/services/tasks/http-client/Ultravisor-TaskConfigs-HttpClient.cjs +281 -0
  114. package/source/services/tasks/http-client/definitions/get-json.json +26 -0
  115. package/source/services/tasks/http-client/definitions/get-text.json +26 -0
  116. package/source/services/tasks/http-client/definitions/rest-request.json +32 -0
  117. package/source/services/tasks/http-client/definitions/send-json.json +28 -0
  118. package/source/services/tasks/llm/Ultravisor-TaskConfigs-LLM.cjs +605 -0
  119. package/source/services/tasks/llm/definitions/llm-chat-completion.json +46 -0
  120. package/source/services/tasks/llm/definitions/llm-embedding.json +31 -0
  121. package/source/services/tasks/llm/definitions/llm-tool-use.json +42 -0
  122. package/source/services/tasks/meadow-api/Ultravisor-TaskConfigs-MeadowApi.cjs +341 -0
  123. package/source/services/tasks/meadow-api/definitions/meadow-count.json +26 -0
  124. package/source/services/tasks/meadow-api/definitions/meadow-create.json +26 -0
  125. package/source/services/tasks/meadow-api/definitions/meadow-delete.json +23 -0
  126. package/source/services/tasks/meadow-api/definitions/meadow-read.json +26 -0
  127. package/source/services/tasks/meadow-api/definitions/meadow-reads.json +29 -0
  128. package/source/services/tasks/meadow-api/definitions/meadow-update.json +26 -0
  129. package/source/services/tasks/shell/Ultravisor-TaskConfigs-Shell.cjs +63 -0
  130. package/source/services/tasks/shell/definitions/command.json +29 -0
  131. package/source/services/tasks/user-interaction/Ultravisor-TaskConfigs-UserInteraction.cjs +64 -0
  132. package/source/services/tasks/user-interaction/Ultravisor-TaskType-ErrorMessage.cjs +55 -0
  133. package/source/services/tasks/user-interaction/Ultravisor-TaskType-ValueInput.cjs +47 -0
  134. package/source/services/tasks/user-interaction/definitions/error-message.json +18 -0
  135. package/source/services/tasks/user-interaction/definitions/value-input.json +23 -0
  136. package/source/web_server/Ultravisor-API-Server.cjs +849 -124
  137. package/test/Ultravisor_browser_tests.js +2226 -0
  138. package/test/Ultravisor_operation_library_tests.js +894 -0
  139. package/test/Ultravisor_tests.js +4946 -5044
  140. package/test/workflows/test-content-concatenate-all.json +339 -0
  141. package/test/workflows/test-content-full-pipeline.json +159 -0
  142. package/test/workflows/test-content-list-files.json +76 -0
  143. package/test/workflows/test-content-save-and-read.json +106 -0
  144. package/webinterface/css/ultravisor-themes.css +668 -0
  145. package/webinterface/css/ultravisor.css +37 -14
  146. package/webinterface/docs/card-help/beacon-dispatch.md +30 -0
  147. package/webinterface/docs/card-help/command.md +27 -0
  148. package/webinterface/docs/card-help/comprehension-intersect.md +24 -0
  149. package/webinterface/docs/card-help/content-create-folder.md +22 -0
  150. package/webinterface/docs/card-help/content-list-files.md +25 -0
  151. package/webinterface/docs/card-help/content-read-file.md +24 -0
  152. package/webinterface/docs/card-help/content-save-file.md +24 -0
  153. package/webinterface/docs/card-help/copy-file.md +25 -0
  154. package/webinterface/docs/card-help/csv-transform.md +24 -0
  155. package/webinterface/docs/card-help/end.md +11 -0
  156. package/webinterface/docs/card-help/error-message.md +16 -0
  157. package/webinterface/docs/card-help/expression-solver.md +21 -0
  158. package/webinterface/docs/card-help/get-json.md +24 -0
  159. package/webinterface/docs/card-help/get-text.md +24 -0
  160. package/webinterface/docs/card-help/histogram.md +23 -0
  161. package/webinterface/docs/card-help/if-conditional.md +23 -0
  162. package/webinterface/docs/card-help/launch-operation.md +25 -0
  163. package/webinterface/docs/card-help/list-files.md +25 -0
  164. package/webinterface/docs/card-help/llm-chat-completion.md +43 -0
  165. package/webinterface/docs/card-help/llm-embedding.md +24 -0
  166. package/webinterface/docs/card-help/llm-tool-use.md +39 -0
  167. package/webinterface/docs/card-help/meadow-count.md +24 -0
  168. package/webinterface/docs/card-help/meadow-create.md +24 -0
  169. package/webinterface/docs/card-help/meadow-delete.md +19 -0
  170. package/webinterface/docs/card-help/meadow-read.md +24 -0
  171. package/webinterface/docs/card-help/meadow-reads.md +27 -0
  172. package/webinterface/docs/card-help/meadow-update.md +24 -0
  173. package/webinterface/docs/card-help/parse-csv.md +27 -0
  174. package/webinterface/docs/card-help/read-file-buffered.md +29 -0
  175. package/webinterface/docs/card-help/read-file.md +24 -0
  176. package/webinterface/docs/card-help/read-json.md +21 -0
  177. package/webinterface/docs/card-help/replace-string.md +25 -0
  178. package/webinterface/docs/card-help/rest-request.md +30 -0
  179. package/webinterface/docs/card-help/send-json.md +26 -0
  180. package/webinterface/docs/card-help/set-values.md +16 -0
  181. package/webinterface/docs/card-help/split-execute.md +35 -0
  182. package/webinterface/docs/card-help/start.md +11 -0
  183. package/webinterface/docs/card-help/string-appender.md +22 -0
  184. package/webinterface/docs/card-help/template-string.md +21 -0
  185. package/webinterface/docs/card-help/value-input.md +24 -0
  186. package/webinterface/docs/card-help/write-file.md +27 -0
  187. package/webinterface/docs/card-help/write-json.md +28 -0
  188. package/webinterface/html/index.html +5 -0
  189. package/webinterface/package.json +13 -4
  190. package/webinterface/source/Pict-Application-Ultravisor.js +564 -70
  191. package/webinterface/source/Ultravisor-TimingUtils.js +99 -0
  192. package/webinterface/source/card-help-content.js +46 -0
  193. package/webinterface/source/cards/Ultravisor-BuiltIn-CardConfigs.js +174 -0
  194. package/webinterface/source/cards/Ultravisor-CardConfigGenerator.js +431 -0
  195. package/webinterface/source/data/ExampleFlow-CSVPipeline.js +231 -0
  196. package/webinterface/source/data/ExampleFlow-FileProcessor.js +315 -0
  197. package/webinterface/source/data/ExampleFlow-MeadowPipeline.js +328 -0
  198. package/webinterface/source/panels/Ultravisor-CardSettingsPanel.js +902 -0
  199. package/webinterface/source/providers/PictRouter-Ultravisor-Configuration.json +20 -8
  200. package/webinterface/source/views/PictView-Ultravisor-BeaconList.js +817 -0
  201. package/webinterface/source/views/PictView-Ultravisor-BottomBar.js +5 -5
  202. package/webinterface/source/views/PictView-Ultravisor-Dashboard.js +25 -25
  203. package/webinterface/source/views/PictView-Ultravisor-Documentation.js +856 -0
  204. package/webinterface/source/views/PictView-Ultravisor-FlowEditor.js +777 -0
  205. package/webinterface/source/views/PictView-Ultravisor-ManifestList.js +321 -58
  206. package/webinterface/source/views/PictView-Ultravisor-OperationEdit.js +36 -91
  207. package/webinterface/source/views/PictView-Ultravisor-OperationList.js +388 -16
  208. package/webinterface/source/views/PictView-Ultravisor-PendingInput.js +314 -0
  209. package/webinterface/source/views/PictView-Ultravisor-Schedule.js +521 -65
  210. package/webinterface/source/views/PictView-Ultravisor-TimingView.js +333 -71
  211. package/webinterface/source/views/PictView-Ultravisor-TopBar.js +257 -21
  212. package/webinterface/theme-sampler.html +645 -0
  213. package/debug/Harness.js +0 -6
  214. package/source/services/Ultravisor-Operation-Manifest.cjs +0 -160
  215. package/source/services/Ultravisor-Operation.cjs +0 -200
  216. package/source/services/Ultravisor-Task.cjs +0 -349
  217. package/source/services/events/Ultravisor-Hypervisor-Event-Solver.cjs +0 -11
  218. package/source/services/tasks/Ultravisor-Task-Base.cjs +0 -264
  219. package/source/services/tasks/Ultravisor-Task-CollectValues.cjs +0 -188
  220. package/source/services/tasks/Ultravisor-Task-Command.cjs +0 -65
  221. package/source/services/tasks/Ultravisor-Task-CommandEach.cjs +0 -190
  222. package/source/services/tasks/Ultravisor-Task-Conditional.cjs +0 -104
  223. package/source/services/tasks/Ultravisor-Task-DateWindow.cjs +0 -72
  224. package/source/services/tasks/Ultravisor-Task-GeneratePagedOperation.cjs +0 -336
  225. package/source/services/tasks/Ultravisor-Task-LaunchOperation.cjs +0 -143
  226. package/source/services/tasks/Ultravisor-Task-LaunchTask.cjs +0 -146
  227. package/source/services/tasks/Ultravisor-Task-LineMatch.cjs +0 -158
  228. package/source/services/tasks/Ultravisor-Task-Request.cjs +0 -56
  229. package/source/services/tasks/Ultravisor-Task-Solver.cjs +0 -89
  230. package/source/services/tasks/Ultravisor-Task-TemplateString.cjs +0 -93
  231. package/source/services/tasks/rest/Ultravisor-Task-GetBinary.cjs +0 -127
  232. package/source/services/tasks/rest/Ultravisor-Task-GetJSON.cjs +0 -119
  233. package/source/services/tasks/rest/Ultravisor-Task-GetText.cjs +0 -109
  234. package/source/services/tasks/rest/Ultravisor-Task-GetXML.cjs +0 -112
  235. package/source/services/tasks/rest/Ultravisor-Task-RestRequest.cjs +0 -499
  236. package/source/services/tasks/rest/Ultravisor-Task-SendJSON.cjs +0 -150
  237. package/source/services/tasks/stagingfiles/Ultravisor-Task-CopyFile.cjs +0 -110
  238. package/source/services/tasks/stagingfiles/Ultravisor-Task-ListFiles.cjs +0 -89
  239. package/source/services/tasks/stagingfiles/Ultravisor-Task-ReadBinary.cjs +0 -87
  240. package/source/services/tasks/stagingfiles/Ultravisor-Task-ReadJSON.cjs +0 -67
  241. package/source/services/tasks/stagingfiles/Ultravisor-Task-ReadText.cjs +0 -66
  242. package/source/services/tasks/stagingfiles/Ultravisor-Task-ReadXML.cjs +0 -69
  243. package/source/services/tasks/stagingfiles/Ultravisor-Task-WriteBinary.cjs +0 -95
  244. package/source/services/tasks/stagingfiles/Ultravisor-Task-WriteJSON.cjs +0 -96
  245. package/source/services/tasks/stagingfiles/Ultravisor-Task-WriteText.cjs +0 -99
  246. package/source/services/tasks/stagingfiles/Ultravisor-Task-WriteXML.cjs +0 -102
  247. package/webinterface/source/views/PictView-Ultravisor-TaskEdit.js +0 -220
  248. package/webinterface/source/views/PictView-Ultravisor-TaskList.js +0 -248
@@ -0,0 +1,1299 @@
1
+ # Building Beacon Providers
2
+
3
+ This guide walks through building custom Beacon providers from scratch,
4
+ with two full examples: a headless Chrome browser automation provider and
5
+ an FFmpeg video transcoding provider. By the end, you will know how to
6
+ create a provider, configure it, wire it into a Beacon, and dispatch work
7
+ to it from an operation manifest.
8
+
9
+ For reference-level documentation on the provider interface, built-in
10
+ providers, and progress reporting, see [Beacons](beacons.md).
11
+
12
+ ## Quick-Start: Running a Beacon
13
+
14
+ Before writing custom providers, let's make sure you can run a basic Beacon
15
+ against an Ultravisor server.
16
+
17
+ ### 1. Start the Ultravisor Server
18
+
19
+ ```bash
20
+ node source/Ultravisor-Server.cjs --config ./my-config.json
21
+ ```
22
+
23
+ ### 2. Start a Shell-Only Beacon
24
+
25
+ The simplest Beacon uses only the built-in Shell provider:
26
+
27
+ ```bash
28
+ node source/beacon/Ultravisor-Beacon-CLI.cjs \
29
+ --server http://localhost:54321 \
30
+ --name "shell-worker" \
31
+ --capabilities Shell
32
+ ```
33
+
34
+ ### 3. Dispatch Work to It
35
+
36
+ Create a manifest with a `beacon-dispatch` node:
37
+
38
+ ```json
39
+ {
40
+ "Scope": "Operation",
41
+ "Nodes":
42
+ {
43
+ "start": { "Type": "start" },
44
+ "run-command":
45
+ {
46
+ "Type": "beacon-dispatch",
47
+ "Settings":
48
+ {
49
+ "Capability": "Shell",
50
+ "Action": "Execute",
51
+ "Command": "echo",
52
+ "Parameters": "hello from the Beacon"
53
+ }
54
+ },
55
+ "end": { "Type": "end" }
56
+ },
57
+ "Edges":
58
+ {
59
+ "start": [{ "Target": "run-command", "Event": "Complete" }],
60
+ "run-command": [{ "Target": "end", "Event": "Complete" }]
61
+ }
62
+ }
63
+ ```
64
+
65
+ Start an operation from this manifest, and the Beacon picks it up, runs
66
+ `echo hello from the Beacon`, and reports back the output. The graph
67
+ resumes and completes.
68
+
69
+ ### 4. Use a Config File
70
+
71
+ For anything beyond quick tests, use a `.ultravisor-beacon.json` file:
72
+
73
+ ```json
74
+ {
75
+ "ServerURL": "http://localhost:54321",
76
+ "Name": "dev-worker",
77
+ "Capabilities": ["Shell", "FileSystem"],
78
+ "MaxConcurrent": 2,
79
+ "PollIntervalMs": 3000,
80
+ "StagingPath": "/tmp/beacon-staging"
81
+ }
82
+ ```
83
+
84
+ Then start the Beacon with no arguments (it reads from the config file):
85
+
86
+ ```bash
87
+ cd /path/to/config && node /path/to/Ultravisor-Beacon-CLI.cjs
88
+ ```
89
+
90
+ ---
91
+
92
+ ## Anatomy of a Provider
93
+
94
+ Every provider is a plain JavaScript class that extends
95
+ `UltravisorBeaconCapabilityProvider`. There is no Fable dependency — keep
96
+ providers lightweight and self-contained.
97
+
98
+ ```
99
+ ┌──────────────────────────────────────────┐
100
+ │ CapabilityProvider │
101
+ │ │
102
+ │ Name "MyProvider" │
103
+ │ Capability "SomeCapability" │
104
+ │ │
105
+ │ get actions() → { ActionA, ActionB } │
106
+ │ │
107
+ │ initialize(fCallback) ← optional │
108
+ │ shutdown(fCallback) ← optional │
109
+ │ │
110
+ │ execute(pAction, pWorkItem, pContext, │
111
+ │ fCallback, fReportProgress) │
112
+ │ │
113
+ │ _ProviderConfig ← per-provider cfg │
114
+ └──────────────────────────────────────────┘
115
+ ```
116
+
117
+ ### The `execute` Contract
118
+
119
+ ```js
120
+ execute(pAction, pWorkItem, pContext, fCallback, fReportProgress)
121
+ ```
122
+
123
+ | Parameter | Description |
124
+ |-----------|-------------|
125
+ | `pAction` | The action string (e.g. `'Transcode'`, `'Screenshot'`) |
126
+ | `pWorkItem` | Work item record. Use `pWorkItem.Settings` for task-specific parameters |
127
+ | `pContext` | Execution context: `{ StagingPath, NodeHash, ... }` |
128
+ | `fCallback(pError, pResult)` | Completion callback. `pResult` must include `{ Outputs: { StdOut, ExitCode, Result }, Log: [] }` |
129
+ | `fReportProgress(pData)` | Optional progress callback: `{ Percent, Message, Step, TotalSteps, Log }` |
130
+
131
+ The `fCallback` result shape:
132
+
133
+ ```js
134
+ fCallback(null, {
135
+ Outputs: {
136
+ StdOut: 'Human-readable summary of what happened',
137
+ ExitCode: 0, // 0 = success, non-zero = failure
138
+ Result: '...' // Structured output data (string or JSON string)
139
+ },
140
+ Log: [
141
+ 'Step 1: did X',
142
+ 'Step 2: did Y'
143
+ ]
144
+ });
145
+ ```
146
+
147
+ ---
148
+
149
+ ## Example 1: Headless Chrome Provider
150
+
151
+ This provider launches a headless Chrome browser to take screenshots,
152
+ generate PDFs, or scrape content from web pages. It uses Puppeteer under
153
+ the hood.
154
+
155
+ ### Provider Implementation
156
+
157
+ ```js
158
+ // headless-chrome-provider.cjs
159
+
160
+ const libBeaconCapabilityProvider = require('ultravisor').BeaconCapabilityProvider;
161
+
162
+ let libPuppeteer;
163
+ try { libPuppeteer = require('puppeteer'); }
164
+ catch (pErr) { libPuppeteer = null; }
165
+
166
+ class HeadlessChromeProvider extends libBeaconCapabilityProvider
167
+ {
168
+ constructor(pProviderConfig)
169
+ {
170
+ super(pProviderConfig);
171
+
172
+ this.Name = 'HeadlessChrome';
173
+ this.Capability = 'Browser';
174
+
175
+ // Provider config with sensible defaults
176
+ this._NavigationTimeoutMs = this._ProviderConfig.NavigationTimeoutMs || 30000;
177
+ this._DefaultViewport = this._ProviderConfig.Viewport || { width: 1280, height: 800 };
178
+ this._AllowedDomains = this._ProviderConfig.AllowedDomains || [];
179
+
180
+ this._Browser = null;
181
+ }
182
+
183
+ get actions()
184
+ {
185
+ return {
186
+ 'Screenshot':
187
+ {
188
+ Description: 'Navigate to a URL and capture a screenshot as a PNG file.',
189
+ SettingsSchema:
190
+ [
191
+ { Name: 'URL', DataType: 'String', Required: true, Description: 'The page URL to screenshot' },
192
+ { Name: 'OutputFile', DataType: 'String', Required: false, Description: 'Output filename (default: screenshot.png)' },
193
+ { Name: 'FullPage', DataType: 'Boolean', Required: false, Description: 'Capture the full scrollable page (default: false)' },
194
+ { Name: 'WaitForSelector', DataType: 'String', Required: false, Description: 'CSS selector to wait for before capture' }
195
+ ]
196
+ },
197
+ 'PDF':
198
+ {
199
+ Description: 'Navigate to a URL and render the page as a PDF document.',
200
+ SettingsSchema:
201
+ [
202
+ { Name: 'URL', DataType: 'String', Required: true, Description: 'The page URL to render' },
203
+ { Name: 'OutputFile', DataType: 'String', Required: false, Description: 'Output filename (default: page.pdf)' },
204
+ { Name: 'Format', DataType: 'String', Required: false, Description: 'Page format: Letter, A4, etc. (default: Letter)' }
205
+ ]
206
+ },
207
+ 'Scrape':
208
+ {
209
+ Description: 'Navigate to a URL and extract text content or element data.',
210
+ SettingsSchema:
211
+ [
212
+ { Name: 'URL', DataType: 'String', Required: true, Description: 'The page URL to scrape' },
213
+ { Name: 'Selector', DataType: 'String', Required: false, Description: 'CSS selector to extract (default: body)' },
214
+ { Name: 'ExtractAttribute', DataType: 'String', Required: false, Description: 'Element attribute to extract instead of text (e.g. href, src)' }
215
+ ]
216
+ }
217
+ };
218
+ }
219
+
220
+ // -------------------------------------------------------------------
221
+ // Lifecycle
222
+ // -------------------------------------------------------------------
223
+
224
+ initialize(fCallback)
225
+ {
226
+ if (!libPuppeteer)
227
+ {
228
+ return fCallback(new Error('Puppeteer is not installed. Run: npm install puppeteer'));
229
+ }
230
+
231
+ let tmpThis = this;
232
+
233
+ libPuppeteer.launch({
234
+ headless: 'new',
235
+ args: ['--no-sandbox', '--disable-setuid-sandbox']
236
+ })
237
+ .then(function (pBrowser)
238
+ {
239
+ tmpThis._Browser = pBrowser;
240
+ return fCallback(null);
241
+ })
242
+ .catch(function (pError)
243
+ {
244
+ return fCallback(new Error('Failed to launch Chrome: ' + pError.message));
245
+ });
246
+ }
247
+
248
+ shutdown(fCallback)
249
+ {
250
+ if (this._Browser)
251
+ {
252
+ this._Browser.close()
253
+ .then(function () { return fCallback(null); })
254
+ .catch(function () { return fCallback(null); });
255
+ }
256
+ else
257
+ {
258
+ return fCallback(null);
259
+ }
260
+ }
261
+
262
+ // -------------------------------------------------------------------
263
+ // Action dispatcher
264
+ // -------------------------------------------------------------------
265
+
266
+ execute(pAction, pWorkItem, pContext, fCallback, fReportProgress)
267
+ {
268
+ switch (pAction)
269
+ {
270
+ case 'Screenshot':
271
+ return this._executeScreenshot(pWorkItem, pContext, fCallback, fReportProgress);
272
+ case 'PDF':
273
+ return this._executePDF(pWorkItem, pContext, fCallback, fReportProgress);
274
+ case 'Scrape':
275
+ return this._executeScrape(pWorkItem, pContext, fCallback, fReportProgress);
276
+ default:
277
+ return fCallback(null, {
278
+ Outputs: { StdOut: 'Unknown action: ' + pAction, ExitCode: -1, Result: '' },
279
+ Log: ['HeadlessChrome: unsupported action [' + pAction + '].']
280
+ });
281
+ }
282
+ }
283
+
284
+ // -------------------------------------------------------------------
285
+ // Domain guard
286
+ // -------------------------------------------------------------------
287
+
288
+ _isDomainAllowed(pURL)
289
+ {
290
+ if (this._AllowedDomains.length === 0) return true;
291
+ try
292
+ {
293
+ let tmpHostname = new URL(pURL).hostname;
294
+ for (let i = 0; i < this._AllowedDomains.length; i++)
295
+ {
296
+ if (tmpHostname === this._AllowedDomains[i]) return true;
297
+ if (tmpHostname.endsWith('.' + this._AllowedDomains[i])) return true;
298
+ }
299
+ return false;
300
+ }
301
+ catch (pErr)
302
+ {
303
+ return false;
304
+ }
305
+ }
306
+
307
+ // -------------------------------------------------------------------
308
+ // Actions
309
+ // -------------------------------------------------------------------
310
+
311
+ _executeScreenshot(pWorkItem, pContext, fCallback, fReportProgress)
312
+ {
313
+ let tmpSettings = pWorkItem.Settings || {};
314
+ let tmpURL = tmpSettings.URL;
315
+ let tmpOutputFile = tmpSettings.OutputFile || 'screenshot.png';
316
+ let tmpFullPage = tmpSettings.FullPage || false;
317
+ let tmpWaitFor = tmpSettings.WaitForSelector || null;
318
+ let tmpThis = this;
319
+
320
+ if (!tmpURL)
321
+ {
322
+ return fCallback(null, {
323
+ Outputs: { StdOut: 'No URL specified.', ExitCode: -1, Result: '' },
324
+ Log: ['HeadlessChrome Screenshot: no URL specified.']
325
+ });
326
+ }
327
+
328
+ if (!this._isDomainAllowed(tmpURL))
329
+ {
330
+ return fCallback(null, {
331
+ Outputs: { StdOut: 'Domain not in AllowedDomains: ' + tmpURL, ExitCode: -1, Result: '' },
332
+ Log: ['HeadlessChrome Screenshot: domain blocked by AllowedDomains.']
333
+ });
334
+ }
335
+
336
+ // Resolve output path relative to staging
337
+ let libPath = require('path');
338
+ let tmpOutputPath = libPath.isAbsolute(tmpOutputFile)
339
+ ? tmpOutputFile
340
+ : libPath.resolve(pContext.StagingPath || process.cwd(), tmpOutputFile);
341
+
342
+ let tmpPage;
343
+ let tmpLog = [];
344
+
345
+ this._Browser.newPage()
346
+ .then(function (pPage)
347
+ {
348
+ tmpPage = pPage;
349
+ tmpLog.push('Opened new browser tab.');
350
+ fReportProgress({ Percent: 10, Message: 'Navigating to ' + tmpURL });
351
+
352
+ return tmpPage.setViewport(tmpThis._DefaultViewport);
353
+ })
354
+ .then(function ()
355
+ {
356
+ return tmpPage.goto(tmpURL, {
357
+ waitUntil: 'networkidle2',
358
+ timeout: tmpThis._NavigationTimeoutMs
359
+ });
360
+ })
361
+ .then(function ()
362
+ {
363
+ tmpLog.push('Navigation complete: ' + tmpURL);
364
+ fReportProgress({ Percent: 50, Message: 'Page loaded' });
365
+
366
+ if (tmpWaitFor)
367
+ {
368
+ tmpLog.push('Waiting for selector: ' + tmpWaitFor);
369
+ return tmpPage.waitForSelector(tmpWaitFor, { timeout: 10000 });
370
+ }
371
+ })
372
+ .then(function ()
373
+ {
374
+ fReportProgress({ Percent: 70, Message: 'Capturing screenshot' });
375
+
376
+ return tmpPage.screenshot({
377
+ path: tmpOutputPath,
378
+ fullPage: tmpFullPage
379
+ });
380
+ })
381
+ .then(function ()
382
+ {
383
+ tmpLog.push('Screenshot saved: ' + tmpOutputPath);
384
+ fReportProgress({ Percent: 100, Message: 'Done' });
385
+
386
+ return tmpPage.close();
387
+ })
388
+ .then(function ()
389
+ {
390
+ return fCallback(null, {
391
+ Outputs: {
392
+ StdOut: 'Screenshot saved to ' + tmpOutputPath,
393
+ ExitCode: 0,
394
+ Result: tmpOutputPath
395
+ },
396
+ Log: tmpLog
397
+ });
398
+ })
399
+ .catch(function (pError)
400
+ {
401
+ tmpLog.push('Error: ' + pError.message);
402
+ if (tmpPage) { tmpPage.close().catch(function () {}); }
403
+ return fCallback(null, {
404
+ Outputs: { StdOut: 'Screenshot failed: ' + pError.message, ExitCode: 1, Result: '' },
405
+ Log: tmpLog
406
+ });
407
+ });
408
+ }
409
+
410
+ _executePDF(pWorkItem, pContext, fCallback, fReportProgress)
411
+ {
412
+ let tmpSettings = pWorkItem.Settings || {};
413
+ let tmpURL = tmpSettings.URL;
414
+ let tmpOutputFile = tmpSettings.OutputFile || 'page.pdf';
415
+ let tmpFormat = tmpSettings.Format || 'Letter';
416
+ let tmpThis = this;
417
+
418
+ if (!tmpURL)
419
+ {
420
+ return fCallback(null, {
421
+ Outputs: { StdOut: 'No URL specified.', ExitCode: -1, Result: '' },
422
+ Log: ['HeadlessChrome PDF: no URL specified.']
423
+ });
424
+ }
425
+
426
+ if (!this._isDomainAllowed(tmpURL))
427
+ {
428
+ return fCallback(null, {
429
+ Outputs: { StdOut: 'Domain not in AllowedDomains: ' + tmpURL, ExitCode: -1, Result: '' },
430
+ Log: ['HeadlessChrome PDF: domain blocked by AllowedDomains.']
431
+ });
432
+ }
433
+
434
+ let libPath = require('path');
435
+ let tmpOutputPath = libPath.isAbsolute(tmpOutputFile)
436
+ ? tmpOutputFile
437
+ : libPath.resolve(pContext.StagingPath || process.cwd(), tmpOutputFile);
438
+
439
+ let tmpPage;
440
+ let tmpLog = [];
441
+
442
+ this._Browser.newPage()
443
+ .then(function (pPage)
444
+ {
445
+ tmpPage = pPage;
446
+ fReportProgress({ Percent: 10, Message: 'Navigating to ' + tmpURL });
447
+ return tmpPage.goto(tmpURL, {
448
+ waitUntil: 'networkidle2',
449
+ timeout: tmpThis._NavigationTimeoutMs
450
+ });
451
+ })
452
+ .then(function ()
453
+ {
454
+ tmpLog.push('Navigation complete: ' + tmpURL);
455
+ fReportProgress({ Percent: 60, Message: 'Rendering PDF' });
456
+ return tmpPage.pdf({ path: tmpOutputPath, format: tmpFormat, printBackground: true });
457
+ })
458
+ .then(function ()
459
+ {
460
+ tmpLog.push('PDF saved: ' + tmpOutputPath);
461
+ fReportProgress({ Percent: 100, Message: 'Done' });
462
+ return tmpPage.close();
463
+ })
464
+ .then(function ()
465
+ {
466
+ return fCallback(null, {
467
+ Outputs: { StdOut: 'PDF saved to ' + tmpOutputPath, ExitCode: 0, Result: tmpOutputPath },
468
+ Log: tmpLog
469
+ });
470
+ })
471
+ .catch(function (pError)
472
+ {
473
+ tmpLog.push('Error: ' + pError.message);
474
+ if (tmpPage) { tmpPage.close().catch(function () {}); }
475
+ return fCallback(null, {
476
+ Outputs: { StdOut: 'PDF failed: ' + pError.message, ExitCode: 1, Result: '' },
477
+ Log: tmpLog
478
+ });
479
+ });
480
+ }
481
+
482
+ _executeScrape(pWorkItem, pContext, fCallback, fReportProgress)
483
+ {
484
+ let tmpSettings = pWorkItem.Settings || {};
485
+ let tmpURL = tmpSettings.URL;
486
+ let tmpSelector = tmpSettings.Selector || 'body';
487
+ let tmpAttribute = tmpSettings.ExtractAttribute || null;
488
+ let tmpThis = this;
489
+
490
+ if (!tmpURL)
491
+ {
492
+ return fCallback(null, {
493
+ Outputs: { StdOut: 'No URL specified.', ExitCode: -1, Result: '' },
494
+ Log: ['HeadlessChrome Scrape: no URL specified.']
495
+ });
496
+ }
497
+
498
+ if (!this._isDomainAllowed(tmpURL))
499
+ {
500
+ return fCallback(null, {
501
+ Outputs: { StdOut: 'Domain not in AllowedDomains: ' + tmpURL, ExitCode: -1, Result: '' },
502
+ Log: ['HeadlessChrome Scrape: domain blocked by AllowedDomains.']
503
+ });
504
+ }
505
+
506
+ let tmpPage;
507
+ let tmpLog = [];
508
+
509
+ this._Browser.newPage()
510
+ .then(function (pPage)
511
+ {
512
+ tmpPage = pPage;
513
+ fReportProgress({ Percent: 20, Message: 'Loading page' });
514
+ return tmpPage.goto(tmpURL, {
515
+ waitUntil: 'networkidle2',
516
+ timeout: tmpThis._NavigationTimeoutMs
517
+ });
518
+ })
519
+ .then(function ()
520
+ {
521
+ tmpLog.push('Loaded: ' + tmpURL);
522
+ fReportProgress({ Percent: 60, Message: 'Extracting content' });
523
+
524
+ if (tmpAttribute)
525
+ {
526
+ return tmpPage.$$eval(tmpSelector, function (pElements, pAttr)
527
+ {
528
+ return pElements.map(function (pEl) { return pEl.getAttribute(pAttr) || ''; });
529
+ }, tmpAttribute);
530
+ }
531
+ else
532
+ {
533
+ return tmpPage.$$eval(tmpSelector, function (pElements)
534
+ {
535
+ return pElements.map(function (pEl) { return pEl.textContent.trim(); });
536
+ });
537
+ }
538
+ })
539
+ .then(function (pResults)
540
+ {
541
+ tmpLog.push('Extracted ' + pResults.length + ' elements matching [' + tmpSelector + '].');
542
+ fReportProgress({ Percent: 100, Message: 'Done' });
543
+ return tmpPage.close().then(function () { return pResults; });
544
+ })
545
+ .then(function (pResults)
546
+ {
547
+ let tmpResultStr = JSON.stringify(pResults);
548
+
549
+ return fCallback(null, {
550
+ Outputs: {
551
+ StdOut: 'Scraped ' + pResults.length + ' elements.',
552
+ ExitCode: 0,
553
+ Result: tmpResultStr
554
+ },
555
+ Log: tmpLog
556
+ });
557
+ })
558
+ .catch(function (pError)
559
+ {
560
+ tmpLog.push('Error: ' + pError.message);
561
+ if (tmpPage) { tmpPage.close().catch(function () {}); }
562
+ return fCallback(null, {
563
+ Outputs: { StdOut: 'Scrape failed: ' + pError.message, ExitCode: 1, Result: '' },
564
+ Log: tmpLog
565
+ });
566
+ });
567
+ }
568
+ }
569
+
570
+ module.exports = HeadlessChromeProvider;
571
+ ```
572
+
573
+ ### Configuration
574
+
575
+ ```json
576
+ {
577
+ "ServerURL": "http://orchestrator:54321",
578
+ "Name": "browser-worker-1",
579
+ "MaxConcurrent": 2,
580
+ "StagingPath": "/data/browser-staging",
581
+ "Providers": [
582
+ { "Source": "Shell" },
583
+ { "Source": "FileSystem" },
584
+ {
585
+ "Source": "./headless-chrome-provider.cjs",
586
+ "Config": {
587
+ "NavigationTimeoutMs": 60000,
588
+ "Viewport": { "width": 1920, "height": 1080 },
589
+ "AllowedDomains": ["example.com", "internal.corp.net"]
590
+ }
591
+ }
592
+ ]
593
+ }
594
+ ```
595
+
596
+ ### Dispatching Browser Work
597
+
598
+ Screenshot a page:
599
+
600
+ ```json
601
+ {
602
+ "Type": "beacon-dispatch",
603
+ "Settings":
604
+ {
605
+ "Capability": "Browser",
606
+ "Action": "Screenshot",
607
+ "URL": "https://example.com/dashboard",
608
+ "OutputFile": "dashboard.png",
609
+ "FullPage": true,
610
+ "WaitForSelector": ".dashboard-loaded"
611
+ }
612
+ }
613
+ ```
614
+
615
+ Generate a PDF report:
616
+
617
+ ```json
618
+ {
619
+ "Type": "beacon-dispatch",
620
+ "Settings":
621
+ {
622
+ "Capability": "Browser",
623
+ "Action": "PDF",
624
+ "URL": "https://internal.corp.net/reports/monthly",
625
+ "OutputFile": "monthly-report.pdf",
626
+ "Format": "A4"
627
+ }
628
+ }
629
+ ```
630
+
631
+ Scrape product prices:
632
+
633
+ ```json
634
+ {
635
+ "Type": "beacon-dispatch",
636
+ "Settings":
637
+ {
638
+ "Capability": "Browser",
639
+ "Action": "Scrape",
640
+ "URL": "https://example.com/products",
641
+ "Selector": ".product-card .price",
642
+ "ExtractAttribute": null
643
+ }
644
+ }
645
+ ```
646
+
647
+ ### Key Design Decisions
648
+
649
+ 1. **Browser pool via `initialize`/`shutdown`.** Chrome launches once when the
650
+ Beacon starts and stays alive across work items. Each action opens a new
651
+ tab, does its work, and closes the tab. This avoids the cost of launching
652
+ a fresh browser for every request.
653
+
654
+ 2. **Domain allow-list.** The `AllowedDomains` config prevents the Beacon
655
+ from being used to scrape arbitrary websites. When the list is empty, all
656
+ domains are allowed (useful for development).
657
+
658
+ 3. **Progress reporting.** Each action reports progress at key milestones
659
+ (navigation, rendering, completion) so the orchestrator can show real-time
660
+ status for long-loading pages.
661
+
662
+ 4. **Output paths resolve against StagingPath.** Relative `OutputFile` values
663
+ resolve against the Beacon's staging directory, keeping outputs organized
664
+ and accessible to subsequent tasks via affinity.
665
+
666
+ ---
667
+
668
+ ## Example 2: FFmpeg Transcode Provider
669
+
670
+ This provider transcodes video files to browser-streamable formats using
671
+ FFmpeg. It supports configurable presets and reports real-time encoding
672
+ progress by parsing FFmpeg's output.
673
+
674
+ ### Provider Implementation
675
+
676
+ ```js
677
+ // ffmpeg-transcode-provider.cjs
678
+
679
+ const libBeaconCapabilityProvider = require('ultravisor').BeaconCapabilityProvider;
680
+ const libChildProcess = require('child_process');
681
+ const libPath = require('path');
682
+ const libFS = require('fs');
683
+
684
+ class FFmpegTranscodeProvider extends libBeaconCapabilityProvider
685
+ {
686
+ constructor(pProviderConfig)
687
+ {
688
+ super(pProviderConfig);
689
+
690
+ this.Name = 'FFmpegTranscode';
691
+ this.Capability = 'MediaProcessing';
692
+
693
+ this._FFmpegPath = this._ProviderConfig.FFmpegPath || 'ffmpeg';
694
+ this._FFprobePath = this._ProviderConfig.FFprobePath || 'ffprobe';
695
+
696
+ // Preset library — each preset is an array of FFmpeg arguments
697
+ this._Presets =
698
+ {
699
+ // H.264 + AAC in MP4 — plays in all modern browsers
700
+ 'browser-mp4':
701
+ [
702
+ '-c:v', 'libx264',
703
+ '-preset', 'medium',
704
+ '-crf', '23',
705
+ '-c:a', 'aac',
706
+ '-b:a', '128k',
707
+ '-movflags', '+faststart', // moves moov atom for streaming
708
+ '-pix_fmt', 'yuv420p' // broad compatibility
709
+ ],
710
+
711
+ // VP9 + Opus in WebM — smaller files, good for web
712
+ 'browser-webm':
713
+ [
714
+ '-c:v', 'libvpx-vp9',
715
+ '-crf', '30',
716
+ '-b:v', '0',
717
+ '-c:a', 'libopus',
718
+ '-b:a', '96k'
719
+ ],
720
+
721
+ // H.264 Low — mobile-friendly, small file
722
+ 'browser-mobile':
723
+ [
724
+ '-c:v', 'libx264',
725
+ '-preset', 'fast',
726
+ '-crf', '28',
727
+ '-vf', 'scale=-2:720', // cap at 720p
728
+ '-c:a', 'aac',
729
+ '-b:a', '96k',
730
+ '-movflags', '+faststart',
731
+ '-pix_fmt', 'yuv420p'
732
+ ],
733
+
734
+ // Thumbnail extraction — single frame to JPEG
735
+ 'thumbnail':
736
+ [
737
+ '-vframes', '1',
738
+ '-q:v', '2'
739
+ ],
740
+
741
+ // Audio extraction — AAC in M4A container
742
+ 'audio-only':
743
+ [
744
+ '-vn',
745
+ '-c:a', 'aac',
746
+ '-b:a', '192k'
747
+ ]
748
+ };
749
+
750
+ // Merge any custom presets from config
751
+ if (this._ProviderConfig.Presets)
752
+ {
753
+ let tmpCustom = this._ProviderConfig.Presets;
754
+ for (let tmpName in tmpCustom)
755
+ {
756
+ if (tmpCustom.hasOwnProperty(tmpName))
757
+ {
758
+ this._Presets[tmpName] = tmpCustom[tmpName];
759
+ }
760
+ }
761
+ }
762
+ }
763
+
764
+ get actions()
765
+ {
766
+ return {
767
+ 'Transcode':
768
+ {
769
+ Description: 'Transcode a video file using a named preset or custom FFmpeg arguments.',
770
+ SettingsSchema:
771
+ [
772
+ { Name: 'InputFile', DataType: 'String', Required: true, Description: 'Path to the source video file' },
773
+ { Name: 'OutputFile', DataType: 'String', Required: true, Description: 'Path for the transcoded output' },
774
+ { Name: 'Preset', DataType: 'String', Required: false, Description: 'Named preset (browser-mp4, browser-webm, browser-mobile, thumbnail, audio-only)' },
775
+ { Name: 'CustomArgs', DataType: 'String', Required: false, Description: 'Custom FFmpeg arguments (used instead of preset)' }
776
+ ]
777
+ },
778
+ 'Probe':
779
+ {
780
+ Description: 'Inspect a media file and return format/codec/duration metadata.',
781
+ SettingsSchema:
782
+ [
783
+ { Name: 'InputFile', DataType: 'String', Required: true, Description: 'Path to the media file' }
784
+ ]
785
+ }
786
+ };
787
+ }
788
+
789
+ // -------------------------------------------------------------------
790
+ // Lifecycle — validate FFmpeg is installed
791
+ // -------------------------------------------------------------------
792
+
793
+ initialize(fCallback)
794
+ {
795
+ let tmpThis = this;
796
+
797
+ libChildProcess.exec(this._FFmpegPath + ' -version', function (pError, pStdOut)
798
+ {
799
+ if (pError)
800
+ {
801
+ return fCallback(new Error(
802
+ 'FFmpeg not found at [' + tmpThis._FFmpegPath + ']. ' +
803
+ 'Install it or set FFmpegPath in provider config.'
804
+ ));
805
+ }
806
+
807
+ // Extract version for logging
808
+ let tmpMatch = pStdOut.match(/ffmpeg version (\S+)/);
809
+ let tmpVersion = tmpMatch ? tmpMatch[1] : 'unknown';
810
+ console.log('[FFmpegTranscode] Initialized with FFmpeg ' + tmpVersion);
811
+ console.log('[FFmpegTranscode] Available presets: ' + Object.keys(tmpThis._Presets).join(', '));
812
+ return fCallback(null);
813
+ });
814
+ }
815
+
816
+ // -------------------------------------------------------------------
817
+ // Action dispatcher
818
+ // -------------------------------------------------------------------
819
+
820
+ execute(pAction, pWorkItem, pContext, fCallback, fReportProgress)
821
+ {
822
+ switch (pAction)
823
+ {
824
+ case 'Transcode':
825
+ return this._executeTranscode(pWorkItem, pContext, fCallback, fReportProgress);
826
+ case 'Probe':
827
+ return this._executeProbe(pWorkItem, pContext, fCallback, fReportProgress);
828
+ default:
829
+ return fCallback(null, {
830
+ Outputs: { StdOut: 'Unknown action: ' + pAction, ExitCode: -1, Result: '' },
831
+ Log: ['FFmpegTranscode: unsupported action [' + pAction + '].']
832
+ });
833
+ }
834
+ }
835
+
836
+ // -------------------------------------------------------------------
837
+ // Transcode
838
+ // -------------------------------------------------------------------
839
+
840
+ _executeTranscode(pWorkItem, pContext, fCallback, fReportProgress)
841
+ {
842
+ let tmpSettings = pWorkItem.Settings || {};
843
+ let tmpInputFile = tmpSettings.InputFile || '';
844
+ let tmpOutputFile = tmpSettings.OutputFile || '';
845
+ let tmpPresetName = tmpSettings.Preset || 'browser-mp4';
846
+ let tmpCustomArgs = tmpSettings.CustomArgs || '';
847
+ let tmpThis = this;
848
+
849
+ if (!tmpInputFile || !tmpOutputFile)
850
+ {
851
+ return fCallback(null, {
852
+ Outputs: { StdOut: 'InputFile and OutputFile are required.', ExitCode: -1, Result: '' },
853
+ Log: ['FFmpegTranscode: missing InputFile or OutputFile.']
854
+ });
855
+ }
856
+
857
+ // Resolve relative paths
858
+ let tmpStagingPath = pContext.StagingPath || process.cwd();
859
+ if (!libPath.isAbsolute(tmpInputFile)) tmpInputFile = libPath.resolve(tmpStagingPath, tmpInputFile);
860
+ if (!libPath.isAbsolute(tmpOutputFile)) tmpOutputFile = libPath.resolve(tmpStagingPath, tmpOutputFile);
861
+
862
+ if (!libFS.existsSync(tmpInputFile))
863
+ {
864
+ return fCallback(null, {
865
+ Outputs: { StdOut: 'Input file not found: ' + tmpInputFile, ExitCode: 1, Result: '' },
866
+ Log: ['FFmpegTranscode: input not found: ' + tmpInputFile]
867
+ });
868
+ }
869
+
870
+ // Ensure output directory exists
871
+ let tmpOutputDir = libPath.dirname(tmpOutputFile);
872
+ if (!libFS.existsSync(tmpOutputDir))
873
+ {
874
+ libFS.mkdirSync(tmpOutputDir, { recursive: true });
875
+ }
876
+
877
+ let tmpLog = [];
878
+
879
+ // Step 1: Probe input to get duration (for progress %)
880
+ tmpThis._getDuration(tmpInputFile, function (pDurationSec)
881
+ {
882
+ tmpLog.push('Input file: ' + tmpInputFile);
883
+ tmpLog.push('Duration: ' + (pDurationSec > 0 ? pDurationSec + 's' : 'unknown'));
884
+ fReportProgress({ Percent: 5, Message: 'Starting transcode' });
885
+
886
+ // Build FFmpeg command
887
+ let tmpArgs = ['-i', tmpInputFile, '-y']; // -y = overwrite output
888
+
889
+ if (tmpCustomArgs)
890
+ {
891
+ // Use custom args (split on spaces, respecting quotes)
892
+ tmpArgs = tmpArgs.concat(tmpCustomArgs.split(/\s+/));
893
+ }
894
+ else
895
+ {
896
+ let tmpPreset = tmpThis._Presets[tmpPresetName];
897
+ if (!tmpPreset)
898
+ {
899
+ return fCallback(null, {
900
+ Outputs: {
901
+ StdOut: 'Unknown preset: ' + tmpPresetName + '. Available: ' +
902
+ Object.keys(tmpThis._Presets).join(', '),
903
+ ExitCode: -1,
904
+ Result: ''
905
+ },
906
+ Log: tmpLog.concat(['Unknown preset: ' + tmpPresetName])
907
+ });
908
+ }
909
+ tmpArgs = tmpArgs.concat(tmpPreset);
910
+ tmpLog.push('Using preset: ' + tmpPresetName);
911
+ }
912
+
913
+ tmpArgs.push(tmpOutputFile);
914
+
915
+ // Step 2: Run FFmpeg with progress parsing
916
+ let tmpProcess = libChildProcess.spawn(tmpThis._FFmpegPath, tmpArgs, {
917
+ stdio: ['pipe', 'pipe', 'pipe']
918
+ });
919
+
920
+ let tmpStdErr = '';
921
+
922
+ tmpProcess.stderr.on('data', function (pData)
923
+ {
924
+ tmpStdErr += pData.toString();
925
+
926
+ // Parse FFmpeg progress output: "time=00:01:23.45"
927
+ if (pDurationSec > 0)
928
+ {
929
+ let tmpTimeMatch = pData.toString().match(/time=(\d+):(\d+):(\d+\.\d+)/);
930
+ if (tmpTimeMatch)
931
+ {
932
+ let tmpCurrentSec = (parseInt(tmpTimeMatch[1]) * 3600) +
933
+ (parseInt(tmpTimeMatch[2]) * 60) +
934
+ parseFloat(tmpTimeMatch[3]);
935
+ let tmpPercent = Math.min(95, Math.round((tmpCurrentSec / pDurationSec) * 100));
936
+ fReportProgress({
937
+ Percent: tmpPercent,
938
+ Message: 'Encoding: ' + Math.round(tmpCurrentSec) + 's / ' + Math.round(pDurationSec) + 's'
939
+ });
940
+ }
941
+ }
942
+ });
943
+
944
+ tmpProcess.on('close', function (pCode)
945
+ {
946
+ if (pCode !== 0)
947
+ {
948
+ // Extract last few lines of stderr for the error message
949
+ let tmpErrLines = tmpStdErr.trim().split('\n');
950
+ let tmpErrMsg = tmpErrLines.slice(-3).join(' | ');
951
+ tmpLog.push('FFmpeg exited with code ' + pCode + ': ' + tmpErrMsg);
952
+ return fCallback(null, {
953
+ Outputs: { StdOut: 'Transcode failed (exit code ' + pCode + ')', ExitCode: pCode, Result: '' },
954
+ Log: tmpLog
955
+ });
956
+ }
957
+
958
+ // Get output file size
959
+ let tmpStats = libFS.statSync(tmpOutputFile);
960
+ let tmpSizeMB = (tmpStats.size / (1024 * 1024)).toFixed(2);
961
+ tmpLog.push('Output: ' + tmpOutputFile + ' (' + tmpSizeMB + ' MB)');
962
+ fReportProgress({ Percent: 100, Message: 'Transcode complete' });
963
+
964
+ return fCallback(null, {
965
+ Outputs: {
966
+ StdOut: 'Transcoded to ' + tmpOutputFile + ' (' + tmpSizeMB + ' MB)',
967
+ ExitCode: 0,
968
+ Result: JSON.stringify({
969
+ OutputFile: tmpOutputFile,
970
+ SizeBytes: tmpStats.size,
971
+ Preset: tmpPresetName
972
+ })
973
+ },
974
+ Log: tmpLog
975
+ });
976
+ });
977
+
978
+ tmpProcess.on('error', function (pError)
979
+ {
980
+ tmpLog.push('Spawn error: ' + pError.message);
981
+ return fCallback(null, {
982
+ Outputs: { StdOut: 'Failed to start FFmpeg: ' + pError.message, ExitCode: 1, Result: '' },
983
+ Log: tmpLog
984
+ });
985
+ });
986
+ });
987
+ }
988
+
989
+ // -------------------------------------------------------------------
990
+ // Probe — return media metadata as JSON
991
+ // -------------------------------------------------------------------
992
+
993
+ _executeProbe(pWorkItem, pContext, fCallback, fReportProgress)
994
+ {
995
+ let tmpSettings = pWorkItem.Settings || {};
996
+ let tmpInputFile = tmpSettings.InputFile || '';
997
+
998
+ if (!tmpInputFile)
999
+ {
1000
+ return fCallback(null, {
1001
+ Outputs: { StdOut: 'No InputFile specified.', ExitCode: -1, Result: '' },
1002
+ Log: ['FFmpegTranscode Probe: no InputFile specified.']
1003
+ });
1004
+ }
1005
+
1006
+ let tmpStagingPath = pContext.StagingPath || process.cwd();
1007
+ if (!libPath.isAbsolute(tmpInputFile)) tmpInputFile = libPath.resolve(tmpStagingPath, tmpInputFile);
1008
+
1009
+ fReportProgress({ Percent: 50, Message: 'Probing ' + libPath.basename(tmpInputFile) });
1010
+
1011
+ let tmpCmd = this._FFprobePath +
1012
+ ' -v quiet -print_format json -show_format -show_streams "' +
1013
+ tmpInputFile + '"';
1014
+
1015
+ libChildProcess.exec(tmpCmd, { maxBuffer: 5 * 1024 * 1024 }, function (pError, pStdOut)
1016
+ {
1017
+ if (pError)
1018
+ {
1019
+ return fCallback(null, {
1020
+ Outputs: { StdOut: 'Probe failed: ' + pError.message, ExitCode: 1, Result: '' },
1021
+ Log: ['FFprobe error: ' + pError.message]
1022
+ });
1023
+ }
1024
+
1025
+ fReportProgress({ Percent: 100, Message: 'Probe complete' });
1026
+
1027
+ return fCallback(null, {
1028
+ Outputs: {
1029
+ StdOut: 'Probe complete for ' + tmpInputFile,
1030
+ ExitCode: 0,
1031
+ Result: pStdOut.trim()
1032
+ },
1033
+ Log: ['FFprobe succeeded for ' + tmpInputFile]
1034
+ });
1035
+ });
1036
+ }
1037
+
1038
+ // -------------------------------------------------------------------
1039
+ // Helper: get duration in seconds via FFprobe
1040
+ // -------------------------------------------------------------------
1041
+
1042
+ _getDuration(pFilePath, fCallback)
1043
+ {
1044
+ let tmpCmd = this._FFprobePath +
1045
+ ' -v quiet -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "' +
1046
+ pFilePath + '"';
1047
+
1048
+ libChildProcess.exec(tmpCmd, function (pError, pStdOut)
1049
+ {
1050
+ if (pError || !pStdOut.trim())
1051
+ {
1052
+ return fCallback(0);
1053
+ }
1054
+ return fCallback(parseFloat(pStdOut.trim()) || 0);
1055
+ });
1056
+ }
1057
+ }
1058
+
1059
+ module.exports = FFmpegTranscodeProvider;
1060
+ ```
1061
+
1062
+ ### Configuration
1063
+
1064
+ ```json
1065
+ {
1066
+ "ServerURL": "http://orchestrator:54321",
1067
+ "Name": "media-worker-1",
1068
+ "MaxConcurrent": 2,
1069
+ "StagingPath": "/data/media-staging",
1070
+ "Providers": [
1071
+ { "Source": "Shell" },
1072
+ {
1073
+ "Source": "FileSystem",
1074
+ "Config": {
1075
+ "AllowedPaths": ["/data/media-staging", "/data/media-output"]
1076
+ }
1077
+ },
1078
+ {
1079
+ "Source": "./ffmpeg-transcode-provider.cjs",
1080
+ "Config": {
1081
+ "FFmpegPath": "/usr/bin/ffmpeg",
1082
+ "FFprobePath": "/usr/bin/ffprobe",
1083
+ "Presets": {
1084
+ "hls-720p": [
1085
+ "-c:v", "libx264", "-preset", "fast",
1086
+ "-crf", "23", "-vf", "scale=-2:720",
1087
+ "-c:a", "aac", "-b:a", "128k",
1088
+ "-f", "hls", "-hls_time", "6",
1089
+ "-hls_playlist_type", "vod"
1090
+ ]
1091
+ }
1092
+ }
1093
+ }
1094
+ ]
1095
+ }
1096
+ ```
1097
+
1098
+ ### Dispatching Transcode Work
1099
+
1100
+ Convert a raw upload to browser-streamable MP4:
1101
+
1102
+ ```json
1103
+ {
1104
+ "Type": "beacon-dispatch",
1105
+ "Settings":
1106
+ {
1107
+ "Capability": "MediaProcessing",
1108
+ "Action": "Transcode",
1109
+ "InputFile": "uploads/raw-footage.mov",
1110
+ "OutputFile": "output/footage-web.mp4",
1111
+ "Preset": "browser-mp4"
1112
+ }
1113
+ }
1114
+ ```
1115
+
1116
+ Generate a mobile-optimized version:
1117
+
1118
+ ```json
1119
+ {
1120
+ "Type": "beacon-dispatch",
1121
+ "Settings":
1122
+ {
1123
+ "Capability": "MediaProcessing",
1124
+ "Action": "Transcode",
1125
+ "InputFile": "uploads/raw-footage.mov",
1126
+ "OutputFile": "output/footage-mobile.mp4",
1127
+ "Preset": "browser-mobile"
1128
+ }
1129
+ }
1130
+ ```
1131
+
1132
+ Extract a thumbnail at the 10-second mark:
1133
+
1134
+ ```json
1135
+ {
1136
+ "Type": "beacon-dispatch",
1137
+ "Settings":
1138
+ {
1139
+ "Capability": "MediaProcessing",
1140
+ "Action": "Transcode",
1141
+ "InputFile": "uploads/raw-footage.mov",
1142
+ "OutputFile": "output/thumb.jpg",
1143
+ "Preset": "thumbnail",
1144
+ "CustomArgs": "-ss 10"
1145
+ }
1146
+ }
1147
+ ```
1148
+
1149
+ Probe file metadata before deciding how to transcode:
1150
+
1151
+ ```json
1152
+ {
1153
+ "Type": "beacon-dispatch",
1154
+ "Settings":
1155
+ {
1156
+ "Capability": "MediaProcessing",
1157
+ "Action": "Probe",
1158
+ "InputFile": "uploads/raw-footage.mov"
1159
+ }
1160
+ }
1161
+ ```
1162
+
1163
+ ### Full Pipeline Manifest
1164
+
1165
+ A complete manifest that probes a file, then transcodes it into browser-MP4,
1166
+ mobile-MP4, and thumbnail — distributed across media workers:
1167
+
1168
+ ```json
1169
+ {
1170
+ "Scope": "Operation",
1171
+ "Nodes":
1172
+ {
1173
+ "start": { "Type": "start" },
1174
+
1175
+ "probe-input":
1176
+ {
1177
+ "Type": "beacon-dispatch",
1178
+ "Settings":
1179
+ {
1180
+ "Capability": "MediaProcessing",
1181
+ "Action": "Probe",
1182
+ "InputFile": "{~D:Record.State.InputFile~}",
1183
+ "AffinityKey": "{~D:Record.State.InputFile~}"
1184
+ }
1185
+ },
1186
+
1187
+ "transcode-web":
1188
+ {
1189
+ "Type": "beacon-dispatch",
1190
+ "Settings":
1191
+ {
1192
+ "Capability": "MediaProcessing",
1193
+ "Action": "Transcode",
1194
+ "InputFile": "{~D:Record.State.InputFile~}",
1195
+ "OutputFile": "{~D:Record.State.OutputDir~}/web.mp4",
1196
+ "Preset": "browser-mp4",
1197
+ "AffinityKey": "{~D:Record.State.InputFile~}"
1198
+ }
1199
+ },
1200
+
1201
+ "transcode-mobile":
1202
+ {
1203
+ "Type": "beacon-dispatch",
1204
+ "Settings":
1205
+ {
1206
+ "Capability": "MediaProcessing",
1207
+ "Action": "Transcode",
1208
+ "InputFile": "{~D:Record.State.InputFile~}",
1209
+ "OutputFile": "{~D:Record.State.OutputDir~}/mobile.mp4",
1210
+ "Preset": "browser-mobile",
1211
+ "AffinityKey": "{~D:Record.State.InputFile~}"
1212
+ }
1213
+ },
1214
+
1215
+ "extract-thumbnail":
1216
+ {
1217
+ "Type": "beacon-dispatch",
1218
+ "Settings":
1219
+ {
1220
+ "Capability": "MediaProcessing",
1221
+ "Action": "Transcode",
1222
+ "InputFile": "{~D:Record.State.InputFile~}",
1223
+ "OutputFile": "{~D:Record.State.OutputDir~}/thumb.jpg",
1224
+ "Preset": "thumbnail",
1225
+ "AffinityKey": "{~D:Record.State.InputFile~}"
1226
+ }
1227
+ },
1228
+
1229
+ "end": { "Type": "end" }
1230
+ },
1231
+ "Edges":
1232
+ {
1233
+ "start": [{ "Target": "probe-input", "Event": "Complete" }],
1234
+ "probe-input": [{ "Target": "transcode-web", "Event": "Complete" }],
1235
+ "transcode-web": [{ "Target": "transcode-mobile", "Event": "Complete" }],
1236
+ "transcode-mobile": [{ "Target": "extract-thumbnail","Event": "Complete" }],
1237
+ "extract-thumbnail": [{ "Target": "end", "Event": "Complete" }]
1238
+ }
1239
+ }
1240
+ ```
1241
+
1242
+ All four `beacon-dispatch` nodes share the same `AffinityKey` (the input
1243
+ filename), so the coordinator pins them all to whichever media worker picks
1244
+ up the first task. This avoids copying the source file across workers.
1245
+
1246
+ ### Key Design Decisions
1247
+
1248
+ 1. **Preset system.** Named presets encapsulate best-practice FFmpeg args for
1249
+ common targets. The `browser-mp4` preset uses `-movflags +faststart` so
1250
+ the browser can begin playback before the full file downloads.
1251
+
1252
+ 2. **Real-time progress from FFmpeg output.** The provider parses
1253
+ `time=HH:MM:SS.ss` from FFmpeg's stderr to compute a percentage against
1254
+ the probed duration. This feeds into `fReportProgress` so the
1255
+ orchestrator shows encoding progress.
1256
+
1257
+ 3. **`initialize` validates FFmpeg.** The Beacon will not start if FFmpeg is
1258
+ missing. This prevents the Beacon from registering and then failing every
1259
+ work item.
1260
+
1261
+ 4. **Affinity for multi-output pipelines.** The example manifest uses the
1262
+ same AffinityKey across all transcode steps so they land on the same
1263
+ worker, where the source file is already on disk.
1264
+
1265
+ 5. **`spawn` vs `exec`.** Transcode uses `spawn` (not `exec`) so we can
1266
+ stream stderr for progress parsing without buffering the entire output.
1267
+ Probe uses `exec` because it produces small, bounded JSON output.
1268
+
1269
+ ---
1270
+
1271
+ ## Provider Checklist
1272
+
1273
+ Use this as a reference when building your own provider:
1274
+
1275
+ - [ ] **Extend `CapabilityProvider`** — set `Name` and `Capability` in the
1276
+ constructor.
1277
+ - [ ] **Declare actions** — implement `get actions()` returning the action
1278
+ map. Each action should have a `Description` and optionally a
1279
+ `SettingsSchema` array.
1280
+ - [ ] **Implement `execute`** — dispatch on `pAction`, read from
1281
+ `pWorkItem.Settings`, resolve paths against `pContext.StagingPath`, call
1282
+ `fCallback` when done.
1283
+ - [ ] **Return structured results** — always include `Outputs.ExitCode`
1284
+ (0 = success), `Outputs.StdOut` (summary), and `Outputs.Result` (data).
1285
+ - [ ] **Use `fReportProgress`** — for any operation that takes more than a
1286
+ few seconds, report progress so the orchestrator can track it.
1287
+ - [ ] **Implement `initialize`** — validate prerequisites (is the binary
1288
+ installed? can we connect to the service?). Return an error to prevent
1289
+ the Beacon from starting if the provider cannot function.
1290
+ - [ ] **Implement `shutdown`** — release resources (close browser, disconnect
1291
+ from service, clean up temp files).
1292
+ - [ ] **Handle errors gracefully** — catch exceptions and return them as
1293
+ `ExitCode: 1` results rather than throwing. The Beacon must stay alive
1294
+ even if individual work items fail.
1295
+ - [ ] **Resolve paths** — use `pContext.StagingPath` to resolve relative file
1296
+ paths. This keeps file operations predictable across machines.
1297
+ - [ ] **Guard external access** — if the provider makes network requests or
1298
+ accesses the filesystem, add configuration to restrict what it can reach
1299
+ (e.g. `AllowedDomains`, `AllowedPaths`).