cyclo-manager 1.1.0.dev3__tar.gz → 1.1.0.dev5__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (24) hide show
  1. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/PKG-INFO +54 -9
  2. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/README.md +53 -7
  3. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/routers/repos.py +41 -20
  4. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager.egg-info/PKG-INFO +54 -9
  5. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager.egg-info/SOURCES.txt +1 -2
  6. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager.egg-info/requires.txt +0 -1
  7. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager_cli/docker/docker-compose.yml +2 -0
  8. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/pyproject.toml +1 -2
  9. cyclo_manager-1.1.0.dev3/tests/test_cpu_usage.py +0 -169
  10. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/__init__.py +0 -0
  11. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/cpu_usage.py +0 -0
  12. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/main.py +0 -0
  13. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/models.py +0 -0
  14. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/routers/__init__.py +0 -0
  15. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/routers/files.py +0 -0
  16. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/routers/system_stats.py +0 -0
  17. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_host_agent/routers/update.py +0 -0
  18. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager.egg-info/dependency_links.txt +0 -0
  19. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager.egg-info/entry_points.txt +0 -0
  20. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager.egg-info/top_level.txt +0 -0
  21. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager_cli/__init__.py +0 -0
  22. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager_cli/cli.py +0 -0
  23. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/cyclo_manager_cli/config/config.yml +0 -0
  24. {cyclo_manager-1.1.0.dev3 → cyclo_manager-1.1.0.dev5}/setup.cfg +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cyclo-manager
3
- Version: 1.1.0.dev3
3
+ Version: 1.1.0.dev5
4
4
  Summary: cyclo_manager CLI: pip-installable launcher for cyclo_manager server and UI containers. Run 'cyclo_manager up' to start Docker stack.
5
5
  License-Expression: Apache-2.0
6
6
  Requires-Python: >=3.10
@@ -9,7 +9,6 @@ Requires-Dist: fastapi>=0.100.0
9
9
  Requires-Dist: uvicorn[standard]>=0.23.0
10
10
  Requires-Dist: psutil>=5.9.0
11
11
  Provides-Extra: dev
12
- Requires-Dist: pytest; extra == "dev"
13
12
  Requires-Dist: ruff; extra == "dev"
14
13
 
15
14
  # cyclo_manager CLI
@@ -38,6 +37,7 @@ For the full monorepo (API source, UI source, dev compose), see the [repository
38
37
  - [What `cyclo_manager up` does](#what-cyclo_manager-up-does)
39
38
  - [Host agent (`cyclo_host_agent`)](#host-agent-cyclo_host_agent)
40
39
  - [Compose services](#compose-services)
40
+ - [Recording storage](#recording-storage)
41
41
  - [Configuration](#configuration)
42
42
  - [Agent sockets on the host](#agent-sockets-on-the-host)
43
43
  - [Environment variables](#environment-variables)
@@ -133,7 +133,12 @@ The API container reaches it at `/agents/host/host_agent.sock` (see bundled `con
133
133
  - Check update availability from remote tags and local `package.xml` versions.
134
134
  - Update repos on allowed branches (`main`, `jazzy`) using stash or reset workflows.
135
135
  - Stop/start a repo's `docker/container.sh` helper during an update when requested by the UI.
136
+ - Run container starts as background jobs with a 10-minute timeout covering the helper and its output streams; timeout or cancellation terminates the helper's process group and ends the job.
136
137
  - Run Cyclo Manager package updates from the UI by delegating to `cyclo_manager update`.
138
+ - Provide host CPU/memory/disk statistics and process information. CPU summaries
139
+ share a background sampler's average of the latest three one-second samples.
140
+ - Serve the Files UI's browsing, filename search, upload, text editing, Git diff
141
+ and file/directory management operations under the configured workspace.
137
142
 
138
143
  `cyclo_manager up` installs the `refresh-host-agent` CLI subcommand into sudoers. It is registered as `sys.executable -m cyclo_manager_cli.cli refresh-host-agent` with `SETENV`, so UI-triggered updates can pass the current `PYTHONPATH` to sudo and do not depend on `PATH`. The command is idempotent and refreshes the socket directory, sudoers file, systemd unit, and service state after package upgrades.
139
144
 
@@ -145,10 +150,10 @@ Repository scanning and the Files UI use `CYCLO_HOST_AGENT_WORKSPACE` when set.
145
150
 
146
151
  Defined in [`cyclo_manager_cli/docker/docker-compose.yml`](cyclo_manager_cli/docker/docker-compose.yml). All services use **`network_mode: host`**.
147
152
 
148
- | Compose service | Container name | `cyclo_manager up` | Image (example) |
153
+ | Compose service | Container name | `cyclo_manager up` | Image in this checkout |
149
154
  |-----------------|----------------|--------------------|-----------------|
150
- | `cyclo_manager` | `cyclo_manager` | **Started** | `robotis/cyclo-manager:1.0.1` |
151
- | `ui` | `cyclo_manager_ui` | **Started** | `robotis/cyclo-manager-ui:1.0.1` |
155
+ | `cyclo_manager` | `cyclo_manager` | **Started** | `robotis/cyclo-manager:1.1.0` |
156
+ | `ui` | `cyclo_manager_ui` | **Started** | `robotis/cyclo-manager-ui:1.1.0` |
152
157
  | `rmw_zenoh` | `zenoh_daemon` | **Created only** | `robotis/zenoh-daemon:latest` |
153
158
  | `novnc-server` | `novnc-server` | **Created only** | `robotis/novnc-server:latest` |
154
159
 
@@ -156,6 +161,27 @@ Start optional containers from the UI or manually, e.g. `docker start zenoh_daem
156
161
 
157
162
  ---
158
163
 
164
+ ## Recording storage
165
+
166
+ The packaged stack and dev compose both mount:
167
+
168
+ ```yaml
169
+ - ${HOME}/cyclo_manager_ros_bags:/cyclo_manager_ros_bags
170
+ ```
171
+
172
+ Compose resolves `${HOME}` on the host at invocation time. The API container uses
173
+ `RECORDINGS_DIR=/cyclo_manager_ros_bags`, which is also the server default when the
174
+ variable is absent. The host directory is independent of
175
+ `CYCLO_HOST_AGENT_WORKSPACE`; each recording stores its metadata and MCAP bag under
176
+ its generated ID. Container recreation, `down` and package updates preserve this
177
+ bind-mounted data. Applying a new mount requires container recreation.
178
+
179
+ Recording and playback run in the API container using `rosbag2_py` and the MCAP
180
+ storage plugin included in the manager image. They are not host-agent jobs.
181
+ See [Record & Play](../docs/record-play.md) for capture, repeat playback and stops.
182
+
183
+ ---
184
+
159
185
  ## Configuration
160
186
 
161
187
  The CLI always mounts the **bundled** config into the API container. There is **no** `-c` / `--config` flag on the pip CLI.
@@ -168,10 +194,13 @@ The API reads it as `CONFIG_FILE=/app/config.yml` inside the container.
168
194
 
169
195
  | Key | Description |
170
196
  |-----|-------------|
171
- | **`supported_robot_containers`** | Robot Docker container names that can open the System page (e.g. `ai_worker`, `open_manipulator`). Each must be a key in `sockets` and cannot be `host_agent`. |
197
+ | **`supported_robot_containers`** | Containers eligible for System navigation and server-side motion profile detection (e.g. `ai_worker`, `open_manipulator`). Each must be a key in `sockets` and cannot be `host_agent`. |
172
198
  | **`sockets`** | Map of logical name → agent **Unix socket path as seen inside the API container** (under `/agents/...`). Include robot/service containers and `host_agent`. |
173
199
 
174
- s6 **service names** are not listed in config; each in-container agent reports them at runtime.
200
+ s6 **service names** are defined in the manager's System/motion profiles, not in
201
+ this config. Motion detection checks the existing individual status endpoints for
202
+ `ai_worker_bringup` and `open_manipulator_bringup`, then reads `/run/robot_type`
203
+ inside the selected container through Docker. This needs no new s6-agent endpoint.
175
204
 
176
205
  ### Example (bundled default)
177
206
 
@@ -198,6 +227,7 @@ On the host, sockets typically live under:
198
227
  ```text
199
228
  /var/run/robotis/agent_sockets/
200
229
  ├── ai_worker/s6_agent.sock
230
+ ├── open_manipulator/s6_agent.sock
201
231
  ├── cyclo_intelligence/s6_agent.sock
202
232
  └── host/host_agent.sock ← created by cyclo_manager up
203
233
  ```
@@ -220,7 +250,11 @@ So `/agents/ai_worker/s6_agent.sock` in config corresponds to the host path abov
220
250
  | **`CYCLO_HOST_AGENT_WORKSPACE`** | User or CLI-generated systemd unit | Workspace scanned by `cyclo_host_agent` for managed git repositories and used as the Files UI/API root |
221
251
  | **`HOSTNAME`** | CLI (default: machine hostname) | Passed to API as `HOST_HOSTNAME` |
222
252
  | **`CONFIG_FILE`** | Compose (`/app/config.yml`) | Path inside the API container |
223
- | **`ROS_DOMAIN_ID`** | **Not** set by CLI | Set inside robot containers (e.g. `~/.bashrc`) so DDS matches your fleet |
253
+ | **`HOME`** | Host environment used by Compose | Selects the host `${HOME}/cyclo_manager_ros_bags` bind-mount directory |
254
+ | **`RECORDINGS_DIR`** | Compose (`/cyclo_manager_ros_bags`) | Server-side recording directory inside the API container; same default without the variable |
255
+ | **`ROS_DOMAIN_ID`** | Manager image shell startup (`30`); not set by CLI | Must match the robot; the manager image configures it in `/root/.bashrc`, which the API startup sources |
256
+ | **`RMW_IMPLEMENTATION`** | Manager image shell startup (`rmw_zenoh_cpp`) | ROS middleware used by the manager; requires a reachable Zenoh router |
257
+ | **`NEXT_PUBLIC_API_URL`** | UI dev/build environment | Browser API address; production bundles embed it at build time, so a compose runtime value does not reconfigure a prebuilt UI |
224
258
 
225
259
  ---
226
260
 
@@ -234,6 +268,12 @@ With `network_mode: host` and default ports:
234
268
  | cyclo_manager API | http://127.0.0.1:8081 |
235
269
  | OpenAPI (Swagger) | http://127.0.0.1:8081/docs |
236
270
 
271
+ These loopback URLs apply on the robot host. From another device, use the robot's
272
+ hostname/IP and a browser-reachable API URL. Both compose files contain a UI
273
+ runtime value of `http://127.0.0.1:8081`; it takes effect in development, while a
274
+ production bundle uses its build-time value or hostname fallback. See the
275
+ [UI API address guide](../cyclo_manager_ui/README_UI.md#api-address).
276
+
237
277
  ---
238
278
 
239
279
  ## Custom config and development
@@ -247,6 +287,11 @@ For development (local API/UI source, custom `config.yml`, hot reload):
247
287
  docker compose -f docker-compose.dev.yml up
248
288
  ```
249
289
 
290
+ The dev compose command starts all defined services and does not install the host
291
+ agent. API/UI source mounts do not update the Python code used by the separately
292
+ installed `cyclo_host_agent.service`; install the updated CLI package and refresh
293
+ that service when testing host-agent changes.
294
+
250
295
  See the [root README](../README.md#development).
251
296
 
252
297
  ---
@@ -259,7 +304,7 @@ Python dependencies declared in [`pyproject.toml`](pyproject.toml):
259
304
  |---------|---------|
260
305
  | **fastapi** | `cyclo_host_agent` HTTP API |
261
306
  | **uvicorn** | `cyclo_host_agent` server |
262
- | **psutil** | Declared package dependency for host/system utility support |
307
+ | **psutil** | Host CPU sampler, memory/disk statistics and process inspection |
263
308
 
264
309
  The **`cyclo_manager`** CLI itself uses only the Python standard library plus **`subprocess`** calls to **`docker compose`**, **`pip`**, **`systemctl`**, and **`sudo`**.
265
310
 
@@ -24,6 +24,7 @@ For the full monorepo (API source, UI source, dev compose), see the [repository
24
24
  - [What `cyclo_manager up` does](#what-cyclo_manager-up-does)
25
25
  - [Host agent (`cyclo_host_agent`)](#host-agent-cyclo_host_agent)
26
26
  - [Compose services](#compose-services)
27
+ - [Recording storage](#recording-storage)
27
28
  - [Configuration](#configuration)
28
29
  - [Agent sockets on the host](#agent-sockets-on-the-host)
29
30
  - [Environment variables](#environment-variables)
@@ -119,7 +120,12 @@ The API container reaches it at `/agents/host/host_agent.sock` (see bundled `con
119
120
  - Check update availability from remote tags and local `package.xml` versions.
120
121
  - Update repos on allowed branches (`main`, `jazzy`) using stash or reset workflows.
121
122
  - Stop/start a repo's `docker/container.sh` helper during an update when requested by the UI.
123
+ - Run container starts as background jobs with a 10-minute timeout covering the helper and its output streams; timeout or cancellation terminates the helper's process group and ends the job.
122
124
  - Run Cyclo Manager package updates from the UI by delegating to `cyclo_manager update`.
125
+ - Provide host CPU/memory/disk statistics and process information. CPU summaries
126
+ share a background sampler's average of the latest three one-second samples.
127
+ - Serve the Files UI's browsing, filename search, upload, text editing, Git diff
128
+ and file/directory management operations under the configured workspace.
123
129
 
124
130
  `cyclo_manager up` installs the `refresh-host-agent` CLI subcommand into sudoers. It is registered as `sys.executable -m cyclo_manager_cli.cli refresh-host-agent` with `SETENV`, so UI-triggered updates can pass the current `PYTHONPATH` to sudo and do not depend on `PATH`. The command is idempotent and refreshes the socket directory, sudoers file, systemd unit, and service state after package upgrades.
125
131
 
@@ -131,10 +137,10 @@ Repository scanning and the Files UI use `CYCLO_HOST_AGENT_WORKSPACE` when set.
131
137
 
132
138
  Defined in [`cyclo_manager_cli/docker/docker-compose.yml`](cyclo_manager_cli/docker/docker-compose.yml). All services use **`network_mode: host`**.
133
139
 
134
- | Compose service | Container name | `cyclo_manager up` | Image (example) |
140
+ | Compose service | Container name | `cyclo_manager up` | Image in this checkout |
135
141
  |-----------------|----------------|--------------------|-----------------|
136
- | `cyclo_manager` | `cyclo_manager` | **Started** | `robotis/cyclo-manager:1.0.1` |
137
- | `ui` | `cyclo_manager_ui` | **Started** | `robotis/cyclo-manager-ui:1.0.1` |
142
+ | `cyclo_manager` | `cyclo_manager` | **Started** | `robotis/cyclo-manager:1.1.0` |
143
+ | `ui` | `cyclo_manager_ui` | **Started** | `robotis/cyclo-manager-ui:1.1.0` |
138
144
  | `rmw_zenoh` | `zenoh_daemon` | **Created only** | `robotis/zenoh-daemon:latest` |
139
145
  | `novnc-server` | `novnc-server` | **Created only** | `robotis/novnc-server:latest` |
140
146
 
@@ -142,6 +148,27 @@ Start optional containers from the UI or manually, e.g. `docker start zenoh_daem
142
148
 
143
149
  ---
144
150
 
151
+ ## Recording storage
152
+
153
+ The packaged stack and dev compose both mount:
154
+
155
+ ```yaml
156
+ - ${HOME}/cyclo_manager_ros_bags:/cyclo_manager_ros_bags
157
+ ```
158
+
159
+ Compose resolves `${HOME}` on the host at invocation time. The API container uses
160
+ `RECORDINGS_DIR=/cyclo_manager_ros_bags`, which is also the server default when the
161
+ variable is absent. The host directory is independent of
162
+ `CYCLO_HOST_AGENT_WORKSPACE`; each recording stores its metadata and MCAP bag under
163
+ its generated ID. Container recreation, `down` and package updates preserve this
164
+ bind-mounted data. Applying a new mount requires container recreation.
165
+
166
+ Recording and playback run in the API container using `rosbag2_py` and the MCAP
167
+ storage plugin included in the manager image. They are not host-agent jobs.
168
+ See [Record & Play](../docs/record-play.md) for capture, repeat playback and stops.
169
+
170
+ ---
171
+
145
172
  ## Configuration
146
173
 
147
174
  The CLI always mounts the **bundled** config into the API container. There is **no** `-c` / `--config` flag on the pip CLI.
@@ -154,10 +181,13 @@ The API reads it as `CONFIG_FILE=/app/config.yml` inside the container.
154
181
 
155
182
  | Key | Description |
156
183
  |-----|-------------|
157
- | **`supported_robot_containers`** | Robot Docker container names that can open the System page (e.g. `ai_worker`, `open_manipulator`). Each must be a key in `sockets` and cannot be `host_agent`. |
184
+ | **`supported_robot_containers`** | Containers eligible for System navigation and server-side motion profile detection (e.g. `ai_worker`, `open_manipulator`). Each must be a key in `sockets` and cannot be `host_agent`. |
158
185
  | **`sockets`** | Map of logical name → agent **Unix socket path as seen inside the API container** (under `/agents/...`). Include robot/service containers and `host_agent`. |
159
186
 
160
- s6 **service names** are not listed in config; each in-container agent reports them at runtime.
187
+ s6 **service names** are defined in the manager's System/motion profiles, not in
188
+ this config. Motion detection checks the existing individual status endpoints for
189
+ `ai_worker_bringup` and `open_manipulator_bringup`, then reads `/run/robot_type`
190
+ inside the selected container through Docker. This needs no new s6-agent endpoint.
161
191
 
162
192
  ### Example (bundled default)
163
193
 
@@ -184,6 +214,7 @@ On the host, sockets typically live under:
184
214
  ```text
185
215
  /var/run/robotis/agent_sockets/
186
216
  ├── ai_worker/s6_agent.sock
217
+ ├── open_manipulator/s6_agent.sock
187
218
  ├── cyclo_intelligence/s6_agent.sock
188
219
  └── host/host_agent.sock ← created by cyclo_manager up
189
220
  ```
@@ -206,7 +237,11 @@ So `/agents/ai_worker/s6_agent.sock` in config corresponds to the host path abov
206
237
  | **`CYCLO_HOST_AGENT_WORKSPACE`** | User or CLI-generated systemd unit | Workspace scanned by `cyclo_host_agent` for managed git repositories and used as the Files UI/API root |
207
238
  | **`HOSTNAME`** | CLI (default: machine hostname) | Passed to API as `HOST_HOSTNAME` |
208
239
  | **`CONFIG_FILE`** | Compose (`/app/config.yml`) | Path inside the API container |
209
- | **`ROS_DOMAIN_ID`** | **Not** set by CLI | Set inside robot containers (e.g. `~/.bashrc`) so DDS matches your fleet |
240
+ | **`HOME`** | Host environment used by Compose | Selects the host `${HOME}/cyclo_manager_ros_bags` bind-mount directory |
241
+ | **`RECORDINGS_DIR`** | Compose (`/cyclo_manager_ros_bags`) | Server-side recording directory inside the API container; same default without the variable |
242
+ | **`ROS_DOMAIN_ID`** | Manager image shell startup (`30`); not set by CLI | Must match the robot; the manager image configures it in `/root/.bashrc`, which the API startup sources |
243
+ | **`RMW_IMPLEMENTATION`** | Manager image shell startup (`rmw_zenoh_cpp`) | ROS middleware used by the manager; requires a reachable Zenoh router |
244
+ | **`NEXT_PUBLIC_API_URL`** | UI dev/build environment | Browser API address; production bundles embed it at build time, so a compose runtime value does not reconfigure a prebuilt UI |
210
245
 
211
246
  ---
212
247
 
@@ -220,6 +255,12 @@ With `network_mode: host` and default ports:
220
255
  | cyclo_manager API | http://127.0.0.1:8081 |
221
256
  | OpenAPI (Swagger) | http://127.0.0.1:8081/docs |
222
257
 
258
+ These loopback URLs apply on the robot host. From another device, use the robot's
259
+ hostname/IP and a browser-reachable API URL. Both compose files contain a UI
260
+ runtime value of `http://127.0.0.1:8081`; it takes effect in development, while a
261
+ production bundle uses its build-time value or hostname fallback. See the
262
+ [UI API address guide](../cyclo_manager_ui/README_UI.md#api-address).
263
+
223
264
  ---
224
265
 
225
266
  ## Custom config and development
@@ -233,6 +274,11 @@ For development (local API/UI source, custom `config.yml`, hot reload):
233
274
  docker compose -f docker-compose.dev.yml up
234
275
  ```
235
276
 
277
+ The dev compose command starts all defined services and does not install the host
278
+ agent. API/UI source mounts do not update the Python code used by the separately
279
+ installed `cyclo_host_agent.service`; install the updated CLI package and refresh
280
+ that service when testing host-agent changes.
281
+
236
282
  See the [root README](../README.md#development).
237
283
 
238
284
  ---
@@ -245,7 +291,7 @@ Python dependencies declared in [`pyproject.toml`](pyproject.toml):
245
291
  |---------|---------|
246
292
  | **fastapi** | `cyclo_host_agent` HTTP API |
247
293
  | **uvicorn** | `cyclo_host_agent` server |
248
- | **psutil** | Declared package dependency for host/system utility support |
294
+ | **psutil** | Host CPU sampler, memory/disk statistics and process inspection |
249
295
 
250
296
  The **`cyclo_manager`** CLI itself uses only the Python standard library plus **`subprocess`** calls to **`docker compose`**, **`pip`**, **`systemctl`**, and **`sudo`**.
251
297
 
@@ -22,6 +22,7 @@ import asyncio
22
22
  import os
23
23
  from pathlib import Path
24
24
  import re
25
+ import signal
25
26
  import time
26
27
  from typing import Optional
27
28
 
@@ -45,6 +46,7 @@ router = APIRouter(prefix='/repos', tags=['repos'])
45
46
  GIT_TIMEOUT = 120.0
46
47
  GIT_REMOTE_TIMEOUT = 5.0
47
48
  CONTAINER_SCRIPT_TIMEOUT = 600.0
49
+ CONTAINER_TERMINATE_TIMEOUT = 2.0
48
50
  CONTAINER_JOB_RETENTION_SECONDS = 3600.0
49
51
  CONTAINER_JOB_OUTPUT_LIMIT = 40000
50
52
 
@@ -429,13 +431,28 @@ async def _stop_container_start_process(
429
431
  proc: asyncio.subprocess.Process | None,
430
432
  readers: list[asyncio.Task],
431
433
  ) -> None:
432
- """Stop a container-start subprocess and consume its reader tasks."""
433
- if proc is not None and proc.returncode is None:
434
+ """Stop the isolated process group, with bounded pipe and process cleanup."""
435
+ if proc is not None:
436
+ def signal_group(sig):
437
+ try:
438
+ os.killpg(proc.pid, sig)
439
+ except ProcessLookupError:
440
+ pass
441
+
442
+ signal_group(signal.SIGTERM)
434
443
  try:
435
- proc.kill()
436
- except ProcessLookupError:
444
+ await asyncio.wait_for(proc.wait(), CONTAINER_TERMINATE_TIMEOUT)
445
+ except asyncio.TimeoutError:
437
446
  pass
438
- await proc.wait()
447
+ # Children may still exist even after their shell has exited.
448
+ signal_group(signal.SIGKILL)
449
+ try:
450
+ await asyncio.wait_for(proc.wait(), CONTAINER_TERMINATE_TIMEOUT)
451
+ except asyncio.TimeoutError:
452
+ pass
453
+ for reader in readers:
454
+ if not reader.done():
455
+ reader.cancel()
439
456
  await asyncio.gather(*readers, return_exceptions=True)
440
457
 
441
458
 
@@ -477,6 +494,7 @@ async def _run_container_start(repo_path: Path, name: str) -> None:
477
494
  'bash', str(script), 'start',
478
495
  cwd=str(repo_path / 'docker'),
479
496
  env=env,
497
+ start_new_session=True,
480
498
  stdin=asyncio.subprocess.PIPE,
481
499
  stdout=asyncio.subprocess.PIPE,
482
500
  stderr=asyncio.subprocess.PIPE,
@@ -490,28 +508,31 @@ async def _run_container_start(repo_path: Path, name: str) -> None:
490
508
  asyncio.create_task(_read_container_job_stream(proc.stdout, job)),
491
509
  asyncio.create_task(_read_container_job_stream(proc.stderr, job)),
492
510
  ]
493
- await asyncio.wait_for(proc.wait(), timeout=CONTAINER_SCRIPT_TIMEOUT)
494
- await asyncio.gather(*readers)
511
+ await asyncio.wait_for(
512
+ asyncio.gather(proc.wait(), *readers), timeout=CONTAINER_SCRIPT_TIMEOUT)
495
513
  except asyncio.TimeoutError:
496
514
  error = 'Timeout waiting for container.sh'
497
- await _stop_container_start_process(proc, readers)
498
- _append_container_job_output(job, error)
499
- _set_container_job_status(job, running=False, success=False, error=error)
515
+ try:
516
+ await _stop_container_start_process(proc, readers)
517
+ finally:
518
+ _append_container_job_output(job, error)
519
+ _set_container_job_status(job, running=False, success=False, error=error)
500
520
  return
501
521
  except asyncio.CancelledError:
502
- await _stop_container_start_process(proc, readers)
503
- _set_container_job_status(
504
- job,
505
- running=False,
506
- success=False,
507
- error='Container start operation was cancelled',
508
- )
522
+ try:
523
+ await _stop_container_start_process(proc, readers)
524
+ finally:
525
+ _set_container_job_status(
526
+ job, running=False, success=False,
527
+ error='Container start operation was cancelled')
509
528
  raise
510
529
  except Exception as exc:
511
- await _stop_container_start_process(proc, readers)
512
530
  error = str(exc) or type(exc).__name__
513
- _append_container_job_output(job, error)
514
- _set_container_job_status(job, running=False, success=False, error=error)
531
+ try:
532
+ await _stop_container_start_process(proc, readers)
533
+ finally:
534
+ _append_container_job_output(job, error)
535
+ _set_container_job_status(job, running=False, success=False, error=error)
515
536
  return
516
537
 
517
538
  assert proc is not None
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cyclo-manager
3
- Version: 1.1.0.dev3
3
+ Version: 1.1.0.dev5
4
4
  Summary: cyclo_manager CLI: pip-installable launcher for cyclo_manager server and UI containers. Run 'cyclo_manager up' to start Docker stack.
5
5
  License-Expression: Apache-2.0
6
6
  Requires-Python: >=3.10
@@ -9,7 +9,6 @@ Requires-Dist: fastapi>=0.100.0
9
9
  Requires-Dist: uvicorn[standard]>=0.23.0
10
10
  Requires-Dist: psutil>=5.9.0
11
11
  Provides-Extra: dev
12
- Requires-Dist: pytest; extra == "dev"
13
12
  Requires-Dist: ruff; extra == "dev"
14
13
 
15
14
  # cyclo_manager CLI
@@ -38,6 +37,7 @@ For the full monorepo (API source, UI source, dev compose), see the [repository
38
37
  - [What `cyclo_manager up` does](#what-cyclo_manager-up-does)
39
38
  - [Host agent (`cyclo_host_agent`)](#host-agent-cyclo_host_agent)
40
39
  - [Compose services](#compose-services)
40
+ - [Recording storage](#recording-storage)
41
41
  - [Configuration](#configuration)
42
42
  - [Agent sockets on the host](#agent-sockets-on-the-host)
43
43
  - [Environment variables](#environment-variables)
@@ -133,7 +133,12 @@ The API container reaches it at `/agents/host/host_agent.sock` (see bundled `con
133
133
  - Check update availability from remote tags and local `package.xml` versions.
134
134
  - Update repos on allowed branches (`main`, `jazzy`) using stash or reset workflows.
135
135
  - Stop/start a repo's `docker/container.sh` helper during an update when requested by the UI.
136
+ - Run container starts as background jobs with a 10-minute timeout covering the helper and its output streams; timeout or cancellation terminates the helper's process group and ends the job.
136
137
  - Run Cyclo Manager package updates from the UI by delegating to `cyclo_manager update`.
138
+ - Provide host CPU/memory/disk statistics and process information. CPU summaries
139
+ share a background sampler's average of the latest three one-second samples.
140
+ - Serve the Files UI's browsing, filename search, upload, text editing, Git diff
141
+ and file/directory management operations under the configured workspace.
137
142
 
138
143
  `cyclo_manager up` installs the `refresh-host-agent` CLI subcommand into sudoers. It is registered as `sys.executable -m cyclo_manager_cli.cli refresh-host-agent` with `SETENV`, so UI-triggered updates can pass the current `PYTHONPATH` to sudo and do not depend on `PATH`. The command is idempotent and refreshes the socket directory, sudoers file, systemd unit, and service state after package upgrades.
139
144
 
@@ -145,10 +150,10 @@ Repository scanning and the Files UI use `CYCLO_HOST_AGENT_WORKSPACE` when set.
145
150
 
146
151
  Defined in [`cyclo_manager_cli/docker/docker-compose.yml`](cyclo_manager_cli/docker/docker-compose.yml). All services use **`network_mode: host`**.
147
152
 
148
- | Compose service | Container name | `cyclo_manager up` | Image (example) |
153
+ | Compose service | Container name | `cyclo_manager up` | Image in this checkout |
149
154
  |-----------------|----------------|--------------------|-----------------|
150
- | `cyclo_manager` | `cyclo_manager` | **Started** | `robotis/cyclo-manager:1.0.1` |
151
- | `ui` | `cyclo_manager_ui` | **Started** | `robotis/cyclo-manager-ui:1.0.1` |
155
+ | `cyclo_manager` | `cyclo_manager` | **Started** | `robotis/cyclo-manager:1.1.0` |
156
+ | `ui` | `cyclo_manager_ui` | **Started** | `robotis/cyclo-manager-ui:1.1.0` |
152
157
  | `rmw_zenoh` | `zenoh_daemon` | **Created only** | `robotis/zenoh-daemon:latest` |
153
158
  | `novnc-server` | `novnc-server` | **Created only** | `robotis/novnc-server:latest` |
154
159
 
@@ -156,6 +161,27 @@ Start optional containers from the UI or manually, e.g. `docker start zenoh_daem
156
161
 
157
162
  ---
158
163
 
164
+ ## Recording storage
165
+
166
+ The packaged stack and dev compose both mount:
167
+
168
+ ```yaml
169
+ - ${HOME}/cyclo_manager_ros_bags:/cyclo_manager_ros_bags
170
+ ```
171
+
172
+ Compose resolves `${HOME}` on the host at invocation time. The API container uses
173
+ `RECORDINGS_DIR=/cyclo_manager_ros_bags`, which is also the server default when the
174
+ variable is absent. The host directory is independent of
175
+ `CYCLO_HOST_AGENT_WORKSPACE`; each recording stores its metadata and MCAP bag under
176
+ its generated ID. Container recreation, `down` and package updates preserve this
177
+ bind-mounted data. Applying a new mount requires container recreation.
178
+
179
+ Recording and playback run in the API container using `rosbag2_py` and the MCAP
180
+ storage plugin included in the manager image. They are not host-agent jobs.
181
+ See [Record & Play](../docs/record-play.md) for capture, repeat playback and stops.
182
+
183
+ ---
184
+
159
185
  ## Configuration
160
186
 
161
187
  The CLI always mounts the **bundled** config into the API container. There is **no** `-c` / `--config` flag on the pip CLI.
@@ -168,10 +194,13 @@ The API reads it as `CONFIG_FILE=/app/config.yml` inside the container.
168
194
 
169
195
  | Key | Description |
170
196
  |-----|-------------|
171
- | **`supported_robot_containers`** | Robot Docker container names that can open the System page (e.g. `ai_worker`, `open_manipulator`). Each must be a key in `sockets` and cannot be `host_agent`. |
197
+ | **`supported_robot_containers`** | Containers eligible for System navigation and server-side motion profile detection (e.g. `ai_worker`, `open_manipulator`). Each must be a key in `sockets` and cannot be `host_agent`. |
172
198
  | **`sockets`** | Map of logical name → agent **Unix socket path as seen inside the API container** (under `/agents/...`). Include robot/service containers and `host_agent`. |
173
199
 
174
- s6 **service names** are not listed in config; each in-container agent reports them at runtime.
200
+ s6 **service names** are defined in the manager's System/motion profiles, not in
201
+ this config. Motion detection checks the existing individual status endpoints for
202
+ `ai_worker_bringup` and `open_manipulator_bringup`, then reads `/run/robot_type`
203
+ inside the selected container through Docker. This needs no new s6-agent endpoint.
175
204
 
176
205
  ### Example (bundled default)
177
206
 
@@ -198,6 +227,7 @@ On the host, sockets typically live under:
198
227
  ```text
199
228
  /var/run/robotis/agent_sockets/
200
229
  ├── ai_worker/s6_agent.sock
230
+ ├── open_manipulator/s6_agent.sock
201
231
  ├── cyclo_intelligence/s6_agent.sock
202
232
  └── host/host_agent.sock ← created by cyclo_manager up
203
233
  ```
@@ -220,7 +250,11 @@ So `/agents/ai_worker/s6_agent.sock` in config corresponds to the host path abov
220
250
  | **`CYCLO_HOST_AGENT_WORKSPACE`** | User or CLI-generated systemd unit | Workspace scanned by `cyclo_host_agent` for managed git repositories and used as the Files UI/API root |
221
251
  | **`HOSTNAME`** | CLI (default: machine hostname) | Passed to API as `HOST_HOSTNAME` |
222
252
  | **`CONFIG_FILE`** | Compose (`/app/config.yml`) | Path inside the API container |
223
- | **`ROS_DOMAIN_ID`** | **Not** set by CLI | Set inside robot containers (e.g. `~/.bashrc`) so DDS matches your fleet |
253
+ | **`HOME`** | Host environment used by Compose | Selects the host `${HOME}/cyclo_manager_ros_bags` bind-mount directory |
254
+ | **`RECORDINGS_DIR`** | Compose (`/cyclo_manager_ros_bags`) | Server-side recording directory inside the API container; same default without the variable |
255
+ | **`ROS_DOMAIN_ID`** | Manager image shell startup (`30`); not set by CLI | Must match the robot; the manager image configures it in `/root/.bashrc`, which the API startup sources |
256
+ | **`RMW_IMPLEMENTATION`** | Manager image shell startup (`rmw_zenoh_cpp`) | ROS middleware used by the manager; requires a reachable Zenoh router |
257
+ | **`NEXT_PUBLIC_API_URL`** | UI dev/build environment | Browser API address; production bundles embed it at build time, so a compose runtime value does not reconfigure a prebuilt UI |
224
258
 
225
259
  ---
226
260
 
@@ -234,6 +268,12 @@ With `network_mode: host` and default ports:
234
268
  | cyclo_manager API | http://127.0.0.1:8081 |
235
269
  | OpenAPI (Swagger) | http://127.0.0.1:8081/docs |
236
270
 
271
+ These loopback URLs apply on the robot host. From another device, use the robot's
272
+ hostname/IP and a browser-reachable API URL. Both compose files contain a UI
273
+ runtime value of `http://127.0.0.1:8081`; it takes effect in development, while a
274
+ production bundle uses its build-time value or hostname fallback. See the
275
+ [UI API address guide](../cyclo_manager_ui/README_UI.md#api-address).
276
+
237
277
  ---
238
278
 
239
279
  ## Custom config and development
@@ -247,6 +287,11 @@ For development (local API/UI source, custom `config.yml`, hot reload):
247
287
  docker compose -f docker-compose.dev.yml up
248
288
  ```
249
289
 
290
+ The dev compose command starts all defined services and does not install the host
291
+ agent. API/UI source mounts do not update the Python code used by the separately
292
+ installed `cyclo_host_agent.service`; install the updated CLI package and refresh
293
+ that service when testing host-agent changes.
294
+
250
295
  See the [root README](../README.md#development).
251
296
 
252
297
  ---
@@ -259,7 +304,7 @@ Python dependencies declared in [`pyproject.toml`](pyproject.toml):
259
304
  |---------|---------|
260
305
  | **fastapi** | `cyclo_host_agent` HTTP API |
261
306
  | **uvicorn** | `cyclo_host_agent` server |
262
- | **psutil** | Declared package dependency for host/system utility support |
307
+ | **psutil** | Host CPU sampler, memory/disk statistics and process inspection |
263
308
 
264
309
  The **`cyclo_manager`** CLI itself uses only the Python standard library plus **`subprocess`** calls to **`docker compose`**, **`pip`**, **`systemctl`**, and **`sudo`**.
265
310
 
@@ -19,5 +19,4 @@ cyclo_manager.egg-info/top_level.txt
19
19
  cyclo_manager_cli/__init__.py
20
20
  cyclo_manager_cli/cli.py
21
21
  cyclo_manager_cli/config/config.yml
22
- cyclo_manager_cli/docker/docker-compose.yml
23
- tests/test_cpu_usage.py
22
+ cyclo_manager_cli/docker/docker-compose.yml
@@ -3,5 +3,4 @@ uvicorn[standard]>=0.23.0
3
3
  psutil>=5.9.0
4
4
 
5
5
  [dev]
6
- pytest
7
6
  ruff
@@ -21,8 +21,10 @@ services:
21
21
  - ${CYCLO_MANAGER_CONFIG_FILE}:/app/config.yml # For hot-reload on config changes
22
22
  - /var/run/docker.sock:/var/run/docker.sock # For Docker API access
23
23
  - /:/host_root:ro # For host disk usage and OS info
24
+ - ${HOME}/cyclo_manager_ros_bags:/cyclo_manager_ros_bags
24
25
  environment:
25
26
  - CONFIG_FILE=/app/config.yml
27
+ - RECORDINGS_DIR=/cyclo_manager_ros_bags
26
28
  - HOST_HOSTNAME=${HOSTNAME}
27
29
 
28
30
  ui:
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "cyclo-manager"
7
- version = "1.1.0.dev3"
7
+ version = "1.1.0.dev5"
8
8
  description = "cyclo_manager CLI: pip-installable launcher for cyclo_manager server and UI containers. Run 'cyclo_manager up' to start Docker stack."
9
9
  readme = "README.md"
10
10
  license = "Apache-2.0"
@@ -17,7 +17,6 @@ dependencies = [
17
17
 
18
18
  [project.optional-dependencies]
19
19
  dev = [
20
- "pytest",
21
20
  "ruff",
22
21
  ]
23
22
 
@@ -1,169 +0,0 @@
1
- #!/usr/bin/env python3
2
- #
3
- # Copyright 2026 ROBOTIS CO., LTD.
4
- #
5
- # Licensed under the Apache License, Version 2.0 (the "License");
6
- # you may not use this file except in compliance with the License.
7
- # You may obtain a copy of the License at
8
- #
9
- # http://www.apache.org/licenses/LICENSE-2.0
10
- #
11
- # Unless required by applicable law or agreed to in writing, software
12
- # distributed under the License is distributed on an "AS IS" BASIS,
13
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
- # See the License for the specific language governing permissions and
15
- # limitations under the License.
16
- #
17
- # Author: Hyungyu Kim
18
-
19
- """Regression tests for continuous CPU sampling and shared API summaries."""
20
-
21
- from types import SimpleNamespace
22
- import threading
23
- import time
24
- import unittest
25
- from unittest.mock import Mock, call, patch
26
-
27
- from cyclo_host_agent.cpu_usage import CpuUsageSampler
28
- from cyclo_host_agent.main import app, lifespan
29
- from cyclo_host_agent.routers import system_stats
30
- from fastapi import HTTPException
31
-
32
-
33
- class CpuUsageTests(unittest.TestCase):
34
- """Check measurement windows, startup behavior, and sampling ownership."""
35
-
36
- def test_rolling_window_discards_prime_and_oldest_sample(self):
37
- """Average available samples at startup and only the last three afterward."""
38
- sampler = CpuUsageSampler()
39
- observed = []
40
-
41
- def next_interval(interval):
42
- self.assertEqual(interval, 1.0)
43
- observed.append(sampler.average())
44
- if len(observed) == 5:
45
- sampler._stop.set()
46
- return True
47
- return False
48
-
49
- with patch('cyclo_host_agent.cpu_usage.psutil.cpu_percent',
50
- side_effect=[99.0, 10.0, 20.0, 60.0, 90.0]) as cpu:
51
- with patch.object(sampler._stop, 'wait', side_effect=next_interval):
52
- sampler._run()
53
- self.assertEqual(observed, [None, 10.0, 15.0, 30.0, 56.7])
54
- self.assertEqual(cpu.call_args_list, [call(interval=None)] * 5)
55
-
56
- def test_background_sampling_is_independent_of_readers(self):
57
- """Prime and sample on one thread, even without API requests."""
58
- sampler = CpuUsageSampler(interval=0.01)
59
- thread_ids = []
60
-
61
- def cpu_percent(interval):
62
- thread_ids.append(threading.get_ident())
63
- return 30.0
64
-
65
- with patch('cyclo_host_agent.cpu_usage.psutil.cpu_percent',
66
- side_effect=cpu_percent):
67
- sampler.start()
68
- worker = sampler._thread
69
- try:
70
- sampler.start()
71
- self.assertIs(sampler._thread, worker)
72
- deadline = time.monotonic() + 2
73
- while sampler.average() is None and time.monotonic() < deadline:
74
- time.sleep(0.005)
75
- self.assertEqual(sampler.average(), 30.0)
76
- finally:
77
- sampler.stop()
78
- self.assertFalse(worker.is_alive())
79
- count = len(thread_ids)
80
- for _ in range(50):
81
- self.assertEqual(sampler.average(), 30.0)
82
- self.assertEqual(len(thread_ids), count)
83
- self.assertGreaterEqual(count, 2)
84
- self.assertEqual(len(set(thread_ids)), 1)
85
- self.assertNotEqual(thread_ids[0], threading.get_ident())
86
- sampler._interval = 1.0
87
- sampler.start()
88
- try:
89
- self.assertIsNone(sampler.average())
90
- finally:
91
- sampler.stop()
92
-
93
- def test_warmup_does_not_report_a_fake_zero(self):
94
- """Return a retryable response before a complete sample exists."""
95
- with patch.object(system_stats, 'cpu_sampler', CpuUsageSampler()):
96
- with self.assertRaises(HTTPException) as result:
97
- system_stats.get_system_stats()
98
- self.assertEqual(result.exception.status_code, 503)
99
-
100
- def test_sampling_failure_clears_history_and_primes_again(self):
101
- """Do not keep old CPU values when a measurement fails."""
102
- sampler = CpuUsageSampler()
103
- observed = []
104
-
105
- def next_interval(interval):
106
- observed.append(sampler.average())
107
- if len(observed) == 5:
108
- sampler._stop.set()
109
- return True
110
- return False
111
-
112
- with patch('cyclo_host_agent.cpu_usage.psutil.cpu_percent',
113
- side_effect=[99.0, 10.0, OSError('CPU read failed'), 99.0, 60.0]), \
114
- patch('cyclo_host_agent.cpu_usage.logger.exception') as log, \
115
- patch.object(sampler._stop, 'wait', side_effect=next_interval):
116
- sampler._run()
117
- self.assertEqual(observed, [None, 10.0, None, None, 60.0])
118
- log.assert_called_once()
119
-
120
- def test_status_and_process_summary_read_the_same_average(self):
121
- """Total CPU is shared while process rows retain their own measurements."""
122
- process = Mock()
123
- process.as_dict.return_value = {
124
- 'pid': 42, 'username': 'robot', 'cpu_percent': 80.0,
125
- 'memory_percent': 1.0, 'memory_info': SimpleNamespace(rss=1024),
126
- 'cmdline': ['robot-node'], 'name': 'robot-node',
127
- }
128
- with patch.object(system_stats.cpu_sampler, 'average', return_value=12.3), \
129
- patch.object(system_stats.psutil, 'cpu_percent',
130
- side_effect=AssertionError('API must not resample total CPU')), \
131
- patch.object(system_stats.psutil, 'virtual_memory',
132
- return_value=SimpleNamespace(used=1024**3, total=8*1024**3)), \
133
- patch.object(system_stats.psutil, 'disk_usage',
134
- return_value=SimpleNamespace(used=10*1024**3, total=100*1024**3)), \
135
- patch.object(system_stats, '_extra_storage_disk_usage', return_value=(None, None)), \
136
- patch.object(system_stats, '_temperature', return_value=None), \
137
- patch.object(system_stats.psutil, 'process_iter', return_value=[process]), \
138
- patch.object(system_stats.psutil, 'cpu_count', return_value=4), \
139
- patch.object(system_stats.time, 'sleep') as sleep:
140
- status = system_stats.get_system_stats()
141
- processes = system_stats._sample_processes()
142
- self.assertEqual(status.cpu_percent, 12.3)
143
- self.assertEqual(processes.cpu_percent, status.cpu_percent)
144
- self.assertEqual(processes.processes[0].cpu_percent, 20.0)
145
- process.cpu_percent.assert_called_once_with(None)
146
- sleep.assert_called_once_with(0.2)
147
-
148
-
149
- class CpuLifespanTests(unittest.IsolatedAsyncioTestCase):
150
- """Check sampler cleanup when the API exits."""
151
-
152
- async def test_lifespan_stops_sampler_even_on_failure(self):
153
- """Stop the background worker on both normal and exceptional shutdown."""
154
- sampler = Mock()
155
- with patch.object(system_stats, 'cpu_sampler', sampler):
156
- async with lifespan(app):
157
- sampler.start.assert_called_once_with()
158
- sampler.stop.assert_not_called()
159
- sampler.stop.assert_called_once_with()
160
- sampler.reset_mock()
161
- with self.assertRaises(RuntimeError):
162
- async with lifespan(app):
163
- raise RuntimeError('shutdown')
164
- sampler.start.assert_called_once_with()
165
- sampler.stop.assert_called_once_with()
166
-
167
-
168
- if __name__ == '__main__':
169
- unittest.main()