loft-cli-agent 0.6.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- loft_cli_agent-0.6.3/PKG-INFO +8 -0
- loft_cli_agent-0.6.3/README.md +153 -0
- loft_cli_agent-0.6.3/loft_cli_agent/__init__.py +3 -0
- loft_cli_agent-0.6.3/loft_cli_agent/cli.py +272 -0
- loft_cli_agent-0.6.3/loft_cli_agent/executor.py +539 -0
- loft_cli_agent-0.6.3/loft_cli_agent/lock.py +77 -0
- loft_cli_agent-0.6.3/loft_cli_agent/paths.py +34 -0
- loft_cli_agent-0.6.3/loft_cli_agent/state.py +74 -0
- loft_cli_agent-0.6.3/loft_cli_agent.egg-info/PKG-INFO +8 -0
- loft_cli_agent-0.6.3/loft_cli_agent.egg-info/SOURCES.txt +14 -0
- loft_cli_agent-0.6.3/loft_cli_agent.egg-info/dependency_links.txt +1 -0
- loft_cli_agent-0.6.3/loft_cli_agent.egg-info/entry_points.txt +2 -0
- loft_cli_agent-0.6.3/loft_cli_agent.egg-info/requires.txt +3 -0
- loft_cli_agent-0.6.3/loft_cli_agent.egg-info/top_level.txt +1 -0
- loft_cli_agent-0.6.3/pyproject.toml +21 -0
- loft_cli_agent-0.6.3/setup.cfg +4 -0
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: loft-cli-agent
|
|
3
|
+
Version: 0.6.3
|
|
4
|
+
Summary: Server-side agent for loft-cli — executes plans locally on managed servers
|
|
5
|
+
Requires-Python: >=3.11
|
|
6
|
+
Requires-Dist: loft-cli-core>=0.5.0
|
|
7
|
+
Requires-Dist: typer[all]>=0.9.0
|
|
8
|
+
Requires-Dist: rich>=13.0
|
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
# loft-cli-agent
|
|
2
|
+
|
|
3
|
+
> Server-side agent for loft-cli -- executes plans locally on managed Linux servers.
|
|
4
|
+
|
|
5
|
+
`loft-cli-agent` is the agent package. It runs on every managed server and executes plans locally via subprocess. The agent is intentionally minimal — it depends only on `loft-cli-core`, `typer`, and `rich`. No Fabric, no paramiko, no sqlcipher.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Installation
|
|
10
|
+
|
|
11
|
+
The agent is typically installed automatically during `loft-cli apply` (bootstrap). For manual installation:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
pip install loft-cli-agent
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Or download the standalone binary from the [Releases](../../releases) page:
|
|
18
|
+
|
|
19
|
+
| Platform | File |
|
|
20
|
+
|---|---|
|
|
21
|
+
| Linux (x86-64) | `loft-cli-agent-linux-amd64` |
|
|
22
|
+
| Linux (ARM64) | `loft-cli-agent-linux-arm64` |
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Dependencies
|
|
27
|
+
|
|
28
|
+
- `loft-cli-core>=0.5.0` -- shared models, specs, policy engine
|
|
29
|
+
- `typer[all]>=0.9.0` -- CLI framework
|
|
30
|
+
- `rich>=13.0` -- terminal formatting
|
|
31
|
+
|
|
32
|
+
No Fabric, no paramiko, no sqlcipher3. This keeps the binary small and avoids system library dependencies on managed servers.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Module Structure
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
loft_cli_agent/
|
|
40
|
+
__init__.py Re-exports __version__ from loft-cli-core
|
|
41
|
+
cli.py Typer CLI entry point — apply, status, version, doctor
|
|
42
|
+
executor.py AgentExecutor — local plan execution via subprocess + policy enforcement
|
|
43
|
+
lock.py Mutation locking via fcntl.flock (one apply at a time)
|
|
44
|
+
paths.py ensure_agent_dirs() + re-exports path constants from core
|
|
45
|
+
state.py State management: load/save RuntimeState, resource change detection
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## CLI Commands
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
loft-cli-agent apply <plan.json> Execute a plan locally
|
|
54
|
+
loft-cli-agent status Show current agent state
|
|
55
|
+
loft-cli-agent version Print agent version
|
|
56
|
+
loft-cli-agent doctor <plan.json> Compare desired state against runtime state
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Server-Side Paths
|
|
62
|
+
|
|
63
|
+
The agent uses these directories on the managed server:
|
|
64
|
+
|
|
65
|
+
| Path | Purpose |
|
|
66
|
+
|---|---|
|
|
67
|
+
| `/etc/loft-cli/` | Configuration (policy.yaml, platform.conf) |
|
|
68
|
+
| `/var/lib/loft-cli/` | Runtime state, locks, desired state |
|
|
69
|
+
| `/var/lib/loft-cli/desired/desired-state.json` | Last-applied desired state |
|
|
70
|
+
| `/var/lib/loft-cli/runtime-state.json` | Current runtime state (hashes, timestamps) |
|
|
71
|
+
| `/var/lib/loft-cli/locks/` | Mutation lock files |
|
|
72
|
+
| `/var/log/loft-cli/` | Agent execution logs |
|
|
73
|
+
|
|
74
|
+
`paths.py` provides `ensure_agent_dirs()` which creates all required directories on first run.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## Execution Model
|
|
79
|
+
|
|
80
|
+
The `AgentExecutor` processes a plan as follows:
|
|
81
|
+
|
|
82
|
+
1. **Acquire mutation lock** — `fcntl.flock` ensures one apply at a time
|
|
83
|
+
2. **Load policy** — reads `/etc/loft-cli/policy.yaml` (if present)
|
|
84
|
+
3. **For each step:**
|
|
85
|
+
a. Check if step is idempotent-skippable (content hash matches runtime state)
|
|
86
|
+
b. Evaluate policy — `auto_apply`, `require_approval`, or `deny`
|
|
87
|
+
c. Execute via subprocess (shell commands), direct file write, or built-in handler (e.g. `http_check:` gate)
|
|
88
|
+
d. Update runtime state with new hash and timestamp
|
|
89
|
+
4. **Save runtime state** — atomic write to `runtime-state.json`
|
|
90
|
+
5. **Save desired state** — persist plan to `desired-state.json`
|
|
91
|
+
6. **Release lock**
|
|
92
|
+
|
|
93
|
+
### Idempotent Re-Apply
|
|
94
|
+
|
|
95
|
+
The agent skips unchanged resources on re-apply. Each step's content is hashed (SHA-256). If the hash matches what's recorded in `runtime-state.json`, the step is skipped. This makes `loft-cli apply` safe to run repeatedly.
|
|
96
|
+
|
|
97
|
+
### Policy Enforcement
|
|
98
|
+
|
|
99
|
+
If a `policy.yaml` exists on the server, every step is evaluated against its rules before execution:
|
|
100
|
+
|
|
101
|
+
- **`auto_apply`** — step executes normally
|
|
102
|
+
- **`require_approval`** — step needs a valid HMAC-SHA256 approval token (time-limited)
|
|
103
|
+
- **`deny`** — step is rejected and logged
|
|
104
|
+
|
|
105
|
+
No policy file = no checks = agent executes everything it's told.
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## Mutation Locking
|
|
110
|
+
|
|
111
|
+
`lock.py` uses `fcntl.flock` to ensure only one `loft-cli-agent apply` runs at a time. If a lock is held, a second apply attempt fails immediately with a clear error message. Locks are automatically released when the process exits.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## State Management
|
|
116
|
+
|
|
117
|
+
`state.py` provides:
|
|
118
|
+
|
|
119
|
+
| Function | Purpose |
|
|
120
|
+
|---|---|
|
|
121
|
+
| `load_state(path)` | Load `RuntimeState` from JSON (returns empty state if file missing) |
|
|
122
|
+
| `save_state(state, path)` | Atomic write (write to temp + rename) |
|
|
123
|
+
| `resource_changed(state, resource_id, content_hash)` | Check if a resource needs re-apply |
|
|
124
|
+
| `update_resource(state, resource_id, content_hash)` | Record a resource as applied |
|
|
125
|
+
|
|
126
|
+
`RuntimeState` is a dict mapping resource IDs to `ResourceState` objects (hash + timestamp). This enables both idempotent skip and drift detection.
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## Binary Build
|
|
131
|
+
|
|
132
|
+
The agent binary is built with PyInstaller:
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
make build-agent-binary
|
|
136
|
+
# or: python scripts/build_agent_binary.py
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
The build script installs `packages/core` + `packages/agent`, then produces a single-file executable. No system library dependencies are needed.
|
|
140
|
+
|
|
141
|
+
Output: `dist/loft-cli-agent`
|
|
142
|
+
|
|
143
|
+
The agent binary targets **Linux only** (Debian/Ubuntu). There is no macOS or Windows build — the agent runs on managed servers, not developer machines.
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Import Boundary
|
|
148
|
+
|
|
149
|
+
`loft-cli-agent` may import from:
|
|
150
|
+
- `loft_cli_core` (shared models, specs, policy engine)
|
|
151
|
+
- Standard library
|
|
152
|
+
|
|
153
|
+
`loft-cli-agent` must **never** import from `loft-cli` (client). The agent and client are independent consumers of core.
|
|
@@ -0,0 +1,272 @@
|
|
|
1
|
+
"""loft-cli-agent CLI — server-side execution commands.
|
|
2
|
+
|
|
3
|
+
Commands:
|
|
4
|
+
apply <plan.json> — Execute a plan from a JSON file
|
|
5
|
+
doctor <plan.json> — Check drift between desired plan and actual state
|
|
6
|
+
status — Print current runtime state
|
|
7
|
+
version — Print agent version
|
|
8
|
+
"""
|
|
9
|
+
|
|
10
|
+
from __future__ import annotations
|
|
11
|
+
|
|
12
|
+
import contextlib
|
|
13
|
+
import json
|
|
14
|
+
from pathlib import Path
|
|
15
|
+
|
|
16
|
+
import typer
|
|
17
|
+
from rich.console import Console
|
|
18
|
+
from rich.panel import Panel
|
|
19
|
+
|
|
20
|
+
app = typer.Typer(
|
|
21
|
+
name="loft-cli-agent",
|
|
22
|
+
help="Server-side agent for loft-cli — executes plans locally.",
|
|
23
|
+
no_args_is_help=True,
|
|
24
|
+
)
|
|
25
|
+
|
|
26
|
+
console = Console()
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
@app.command()
|
|
30
|
+
def apply(
|
|
31
|
+
plan_file: Path = typer.Argument(..., help="Path to plan JSON file", exists=True),
|
|
32
|
+
) -> None:
|
|
33
|
+
"""Execute a plan from a JSON file."""
|
|
34
|
+
from loft_cli_agent.executor import AgentExecutor
|
|
35
|
+
from loft_cli_agent.lock import LockError, MutationLock
|
|
36
|
+
from loft_cli_agent.paths import ensure_agent_dirs
|
|
37
|
+
from loft_cli_core.plan.models import Plan
|
|
38
|
+
|
|
39
|
+
ensure_agent_dirs()
|
|
40
|
+
|
|
41
|
+
# Load the plan
|
|
42
|
+
try:
|
|
43
|
+
plan_data = json.loads(plan_file.read_text(encoding="utf-8"))
|
|
44
|
+
plan = Plan.model_validate(plan_data)
|
|
45
|
+
except Exception as e:
|
|
46
|
+
console.print(f"[bold red]Failed to load plan:[/bold red] {e}")
|
|
47
|
+
raise typer.Exit(1) from None
|
|
48
|
+
|
|
49
|
+
console.print(
|
|
50
|
+
Panel(
|
|
51
|
+
f"[bold]Spec:[/bold] {plan.spec_name} ({plan.spec_kind})\n"
|
|
52
|
+
f"[bold]Steps:[/bold] {len(plan.steps)}",
|
|
53
|
+
title="loft-cli-agent apply",
|
|
54
|
+
border_style="bright_blue",
|
|
55
|
+
)
|
|
56
|
+
)
|
|
57
|
+
|
|
58
|
+
# Acquire mutation lock and execute
|
|
59
|
+
try:
|
|
60
|
+
with MutationLock():
|
|
61
|
+
executor = AgentExecutor(plan=plan)
|
|
62
|
+
result = executor.apply()
|
|
63
|
+
except LockError as e:
|
|
64
|
+
console.print(f"[bold red]Lock error:[/bold red] {e}")
|
|
65
|
+
raise typer.Exit(1) from None
|
|
66
|
+
|
|
67
|
+
# Print results
|
|
68
|
+
icons = {"success": "✓", "failed": "✗", "skipped": "○", "unchanged": "≡"}
|
|
69
|
+
colors = {"success": "green", "failed": "red", "skipped": "dim", "unchanged": "blue"}
|
|
70
|
+
|
|
71
|
+
for sr in result.step_results:
|
|
72
|
+
icon = icons.get(sr.status, "?")
|
|
73
|
+
color = colors.get(sr.status, "white")
|
|
74
|
+
duration = f"{sr.duration_seconds:.1f}s" if sr.duration_seconds else ""
|
|
75
|
+
console.print(
|
|
76
|
+
f" [{color}]{icon}[/{color}] [{sr.step_index:>2}] {sr.step_id[:50]}"
|
|
77
|
+
+ (f" [{duration}]" if duration else ""),
|
|
78
|
+
)
|
|
79
|
+
if sr.status == "failed" and sr.error:
|
|
80
|
+
console.print(f" [red]{sr.error[:120]}[/red]")
|
|
81
|
+
|
|
82
|
+
# Summary
|
|
83
|
+
console.print(
|
|
84
|
+
f"\n[bold]Applied:[/bold] {result.applied_count} "
|
|
85
|
+
f"[bold]Unchanged:[/bold] {result.unchanged_count} "
|
|
86
|
+
f"[bold]Status:[/bold] {result.status}"
|
|
87
|
+
)
|
|
88
|
+
|
|
89
|
+
# Write result JSON to stdout for the client to parse
|
|
90
|
+
result_json = result.model_dump_json(indent=2)
|
|
91
|
+
# Write to a result file that the client can download
|
|
92
|
+
result_path = Path("/var/lib/loft-cli/last-result.json")
|
|
93
|
+
with contextlib.suppress(Exception):
|
|
94
|
+
result_path.write_text(result_json, encoding="utf-8")
|
|
95
|
+
|
|
96
|
+
# Store desired state for drift detection (doctor command)
|
|
97
|
+
desired_path = Path("/var/lib/loft-cli/desired/desired-state.json")
|
|
98
|
+
with contextlib.suppress(Exception):
|
|
99
|
+
desired_path.parent.mkdir(parents=True, exist_ok=True)
|
|
100
|
+
desired_path.write_text(plan_file.read_text(encoding="utf-8"), encoding="utf-8")
|
|
101
|
+
|
|
102
|
+
if result.status == "failed":
|
|
103
|
+
raise typer.Exit(1)
|
|
104
|
+
|
|
105
|
+
|
|
106
|
+
@app.command()
|
|
107
|
+
def status() -> None:
|
|
108
|
+
"""Print current runtime state."""
|
|
109
|
+
from loft_cli_agent.state import load_state
|
|
110
|
+
|
|
111
|
+
state = load_state()
|
|
112
|
+
|
|
113
|
+
if not state.last_applied:
|
|
114
|
+
console.print("[dim]No apply has been run yet.[/dim]")
|
|
115
|
+
return
|
|
116
|
+
|
|
117
|
+
console.print(
|
|
118
|
+
Panel(
|
|
119
|
+
f"[bold]Agent version:[/bold] {state.version}\n"
|
|
120
|
+
f"[bold]Last applied:[/bold] {state.last_applied}\n"
|
|
121
|
+
f"[bold]Spec hash:[/bold] {state.spec_hash[:16]}…\n"
|
|
122
|
+
f"[bold]Plan hash:[/bold] {state.plan_hash[:16]}…\n"
|
|
123
|
+
f"[bold]Resources:[/bold] {len(state.resources)}",
|
|
124
|
+
title="loft-cli-agent status",
|
|
125
|
+
border_style="bright_blue",
|
|
126
|
+
)
|
|
127
|
+
)
|
|
128
|
+
|
|
129
|
+
for rid, rs in state.resources.items():
|
|
130
|
+
status_color = "green" if rs.status == "applied" else "red"
|
|
131
|
+
console.print(f" [{status_color}]●[/{status_color}] {rid} ({rs.applied_at})")
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
@app.command()
|
|
135
|
+
def doctor(
|
|
136
|
+
plan_file: Path = typer.Argument(
|
|
137
|
+
None, help="Path to desired-state plan JSON (optional — uses stored state if omitted)"
|
|
138
|
+
),
|
|
139
|
+
) -> None:
|
|
140
|
+
"""Check drift between desired state and actual runtime state.
|
|
141
|
+
|
|
142
|
+
Compares the desired plan against the current runtime-state.json to
|
|
143
|
+
identify resources that have drifted (content hash mismatch), are
|
|
144
|
+
missing (new), or are orphaned (present in state but not in plan).
|
|
145
|
+
|
|
146
|
+
Outputs a JSON report to /var/lib/loft-cli/doctor-result.json.
|
|
147
|
+
"""
|
|
148
|
+
from loft_cli_agent.state import load_state
|
|
149
|
+
from loft_cli_core.plan.models import Plan, StepScope
|
|
150
|
+
from loft_cli_core.utils.hashing import sha256_string
|
|
151
|
+
|
|
152
|
+
# Load desired plan
|
|
153
|
+
desired_path = plan_file
|
|
154
|
+
if desired_path is None:
|
|
155
|
+
desired_path = Path("/var/lib/loft-cli/desired/desired-state.json")
|
|
156
|
+
if not desired_path.exists():
|
|
157
|
+
console.print(
|
|
158
|
+
"[bold red]No desired state found.[/bold red] "
|
|
159
|
+
"Run 'loft-cli apply' first, or provide a plan file."
|
|
160
|
+
)
|
|
161
|
+
raise typer.Exit(1)
|
|
162
|
+
|
|
163
|
+
try:
|
|
164
|
+
plan_data = json.loads(desired_path.read_text(encoding="utf-8"))
|
|
165
|
+
plan = Plan.model_validate(plan_data)
|
|
166
|
+
except Exception as e:
|
|
167
|
+
console.print(f"[bold red]Failed to load desired state:[/bold red] {e}")
|
|
168
|
+
raise typer.Exit(1) from None
|
|
169
|
+
|
|
170
|
+
state = load_state()
|
|
171
|
+
|
|
172
|
+
console.print(
|
|
173
|
+
Panel(
|
|
174
|
+
f"[bold]Spec:[/bold] {plan.spec_name} ({plan.spec_kind})\n"
|
|
175
|
+
f"[bold]Last applied:[/bold] {state.last_applied or 'never'}",
|
|
176
|
+
title="loft-cli-agent doctor",
|
|
177
|
+
border_style="bright_blue",
|
|
178
|
+
)
|
|
179
|
+
)
|
|
180
|
+
|
|
181
|
+
# Compare desired steps against runtime state
|
|
182
|
+
remote_steps = [s for s in plan.steps if s.scope in (StepScope.REMOTE, StepScope.VERIFY)]
|
|
183
|
+
|
|
184
|
+
drifted = []
|
|
185
|
+
missing = []
|
|
186
|
+
unchanged = []
|
|
187
|
+
always_run_steps = []
|
|
188
|
+
|
|
189
|
+
desired_ids = set()
|
|
190
|
+
for step in remote_steps:
|
|
191
|
+
desired_ids.add(step.id)
|
|
192
|
+
parts = [step.id, step.command or "", step.file_content or "", step.target_path or ""]
|
|
193
|
+
content_hash = sha256_string("".join(parts))
|
|
194
|
+
is_always = "always" in step.tags or step.gate or step.scope == StepScope.VERIFY
|
|
195
|
+
|
|
196
|
+
if is_always:
|
|
197
|
+
always_run_steps.append(step.id)
|
|
198
|
+
continue
|
|
199
|
+
|
|
200
|
+
existing = state.resources.get(step.id)
|
|
201
|
+
if existing is None:
|
|
202
|
+
missing.append(step.id)
|
|
203
|
+
elif existing.content_hash != content_hash:
|
|
204
|
+
drifted.append(step.id)
|
|
205
|
+
else:
|
|
206
|
+
unchanged.append(step.id)
|
|
207
|
+
|
|
208
|
+
# Orphaned resources: in state but not in desired plan
|
|
209
|
+
orphaned = [rid for rid in state.resources if rid not in desired_ids]
|
|
210
|
+
|
|
211
|
+
# Print results
|
|
212
|
+
icons = {"drifted": "~", "missing": "+", "unchanged": "=", "orphaned": "-"}
|
|
213
|
+
colors = {"drifted": "yellow", "missing": "green", "unchanged": "dim", "orphaned": "red"}
|
|
214
|
+
|
|
215
|
+
for rid in missing:
|
|
216
|
+
console.print(
|
|
217
|
+
f" [{colors['missing']}]{icons['missing']}[/{colors['missing']}] {rid} (new — not yet applied)"
|
|
218
|
+
)
|
|
219
|
+
for rid in drifted:
|
|
220
|
+
console.print(
|
|
221
|
+
f" [{colors['drifted']}]{icons['drifted']}[/{colors['drifted']}] {rid} (drifted — content changed)"
|
|
222
|
+
)
|
|
223
|
+
for rid in orphaned:
|
|
224
|
+
console.print(
|
|
225
|
+
f" [{colors['orphaned']}]{icons['orphaned']}[/{colors['orphaned']}] {rid} (orphaned — no longer in spec)"
|
|
226
|
+
)
|
|
227
|
+
for rid in unchanged:
|
|
228
|
+
console.print(
|
|
229
|
+
f" [{colors['unchanged']}]{icons['unchanged']}[/{colors['unchanged']}] {rid}"
|
|
230
|
+
)
|
|
231
|
+
|
|
232
|
+
# Summary
|
|
233
|
+
console.print(
|
|
234
|
+
f"\n[green]+{len(missing)} new[/green] "
|
|
235
|
+
f"[yellow]~{len(drifted)} drifted[/yellow] "
|
|
236
|
+
f"[red]-{len(orphaned)} orphaned[/red] "
|
|
237
|
+
f"[dim]={len(unchanged)} unchanged[/dim] "
|
|
238
|
+
f"[cyan]{len(always_run_steps)} always-run[/cyan]"
|
|
239
|
+
)
|
|
240
|
+
|
|
241
|
+
# Write doctor result as JSON for client to download
|
|
242
|
+
doctor_result = {
|
|
243
|
+
"spec_name": plan.spec_name,
|
|
244
|
+
"spec_kind": plan.spec_kind,
|
|
245
|
+
"drifted": drifted,
|
|
246
|
+
"missing": missing,
|
|
247
|
+
"orphaned": orphaned,
|
|
248
|
+
"unchanged": unchanged,
|
|
249
|
+
"always_run": always_run_steps,
|
|
250
|
+
"healthy": len(drifted) == 0 and len(missing) == 0 and len(orphaned) == 0,
|
|
251
|
+
}
|
|
252
|
+
result_path = Path("/var/lib/loft-cli/doctor-result.json")
|
|
253
|
+
with contextlib.suppress(Exception):
|
|
254
|
+
result_path.write_text(json.dumps(doctor_result, indent=2), encoding="utf-8")
|
|
255
|
+
|
|
256
|
+
if drifted or missing or orphaned:
|
|
257
|
+
console.print("\n[bold yellow]Server has drifted from desired state.[/bold yellow]")
|
|
258
|
+
raise typer.Exit(1)
|
|
259
|
+
else:
|
|
260
|
+
console.print("\n[bold green]Server matches desired state.[/bold green]")
|
|
261
|
+
|
|
262
|
+
|
|
263
|
+
@app.command()
|
|
264
|
+
def version() -> None:
|
|
265
|
+
"""Print agent version."""
|
|
266
|
+
from loft_cli_core import __version__
|
|
267
|
+
|
|
268
|
+
console.print(f"loft-cli-agent {__version__}")
|
|
269
|
+
|
|
270
|
+
|
|
271
|
+
if __name__ == "__main__":
|
|
272
|
+
app()
|
|
@@ -0,0 +1,539 @@
|
|
|
1
|
+
"""Agent-side plan executor.
|
|
2
|
+
|
|
3
|
+
Executes plan steps locally on the managed server via subprocess,
|
|
4
|
+
rather than over SSH. Supports idempotent re-apply by tracking
|
|
5
|
+
resource state via content hashes.
|
|
6
|
+
|
|
7
|
+
Policy integration: when a policy.yaml is present at
|
|
8
|
+
``/etc/loft-cli/policy.yaml``, each step is evaluated against
|
|
9
|
+
the policy before execution.
|
|
10
|
+
"""
|
|
11
|
+
|
|
12
|
+
from __future__ import annotations
|
|
13
|
+
|
|
14
|
+
import json
|
|
15
|
+
import subprocess
|
|
16
|
+
import time
|
|
17
|
+
from datetime import UTC, datetime
|
|
18
|
+
from pathlib import Path
|
|
19
|
+
|
|
20
|
+
from loft_cli_agent.state import (
|
|
21
|
+
load_state,
|
|
22
|
+
resource_changed,
|
|
23
|
+
save_state,
|
|
24
|
+
update_resource,
|
|
25
|
+
)
|
|
26
|
+
from loft_cli_core.agent_models import AgentApplyResult, AgentStepResult
|
|
27
|
+
from loft_cli_core.agent_paths import AGENT_CONFIG_DIR
|
|
28
|
+
from loft_cli_core.plan.models import Plan, Step, StepScope
|
|
29
|
+
from loft_cli_core.policy import PolicyAction, PolicyConfig, evaluate_step, load_policy
|
|
30
|
+
from loft_cli_core.state import RuntimeState
|
|
31
|
+
from loft_cli_core.utils.hashing import sha256_string
|
|
32
|
+
|
|
33
|
+
# Default policy.yaml location on the managed server.
|
|
34
|
+
_POLICY_PATH = AGENT_CONFIG_DIR / "policy.yaml"
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
class AgentExecutor:
|
|
38
|
+
"""Execute plan steps locally on the managed server."""
|
|
39
|
+
|
|
40
|
+
def __init__(
|
|
41
|
+
self,
|
|
42
|
+
plan: Plan,
|
|
43
|
+
state_path: Path | None = None,
|
|
44
|
+
policy_path: Path | None = None,
|
|
45
|
+
approval_tokens: list[str] | None = None,
|
|
46
|
+
) -> None:
|
|
47
|
+
self._plan = plan
|
|
48
|
+
self._state_path = state_path
|
|
49
|
+
self._state: RuntimeState = load_state(state_path)
|
|
50
|
+
self._policy: PolicyConfig | None = load_policy(
|
|
51
|
+
policy_path if policy_path is not None else _POLICY_PATH
|
|
52
|
+
)
|
|
53
|
+
self._approval_tokens = {t.split(":")[0]: t for t in (approval_tokens or [])}
|
|
54
|
+
|
|
55
|
+
def apply(self) -> AgentApplyResult:
|
|
56
|
+
"""Execute all remote/verify steps in order, with idempotent skip logic."""
|
|
57
|
+
started_at = datetime.now(UTC).isoformat()
|
|
58
|
+
step_results: list[AgentStepResult] = []
|
|
59
|
+
aborted_at: int | None = None
|
|
60
|
+
unchanged_count = 0
|
|
61
|
+
applied_count = 0
|
|
62
|
+
|
|
63
|
+
# Filter to remote and verify steps only — local steps run on the client
|
|
64
|
+
steps = [s for s in self._plan.steps if s.scope in (StepScope.REMOTE, StepScope.VERIFY)]
|
|
65
|
+
|
|
66
|
+
for step in steps:
|
|
67
|
+
# Check dependencies
|
|
68
|
+
if self._has_failed_dependency(step, step_results):
|
|
69
|
+
result = AgentStepResult(
|
|
70
|
+
step_index=step.index,
|
|
71
|
+
step_id=step.id,
|
|
72
|
+
scope=step.scope.value,
|
|
73
|
+
status="skipped",
|
|
74
|
+
error="Dependency failed",
|
|
75
|
+
)
|
|
76
|
+
step_results.append(result)
|
|
77
|
+
if step.gate:
|
|
78
|
+
aborted_at = step.index
|
|
79
|
+
break
|
|
80
|
+
continue
|
|
81
|
+
|
|
82
|
+
if aborted_at is not None:
|
|
83
|
+
step_results.append(
|
|
84
|
+
AgentStepResult(
|
|
85
|
+
step_index=step.index,
|
|
86
|
+
step_id=step.id,
|
|
87
|
+
scope=step.scope.value,
|
|
88
|
+
status="skipped",
|
|
89
|
+
error="Plan aborted",
|
|
90
|
+
)
|
|
91
|
+
)
|
|
92
|
+
continue
|
|
93
|
+
|
|
94
|
+
# Idempotency check: skip unchanged resources
|
|
95
|
+
content_hash = self._step_content_hash(step)
|
|
96
|
+
is_always = "always" in step.tags or step.gate or step.scope == StepScope.VERIFY
|
|
97
|
+
if not is_always and not resource_changed(self._state, step.id, content_hash):
|
|
98
|
+
result = AgentStepResult(
|
|
99
|
+
step_index=step.index,
|
|
100
|
+
step_id=step.id,
|
|
101
|
+
scope=step.scope.value,
|
|
102
|
+
status="unchanged",
|
|
103
|
+
output="Resource unchanged — skipped",
|
|
104
|
+
)
|
|
105
|
+
step_results.append(result)
|
|
106
|
+
unchanged_count += 1
|
|
107
|
+
continue
|
|
108
|
+
|
|
109
|
+
# Policy check: evaluate step against policy rules
|
|
110
|
+
decision = evaluate_step(self._policy, step.id, step.kind, step.tags)
|
|
111
|
+
if decision.action == PolicyAction.DENY:
|
|
112
|
+
result = AgentStepResult(
|
|
113
|
+
step_index=step.index,
|
|
114
|
+
step_id=step.id,
|
|
115
|
+
scope=step.scope.value,
|
|
116
|
+
status="failed",
|
|
117
|
+
error=f"Denied by policy: {decision.reason}",
|
|
118
|
+
)
|
|
119
|
+
step_results.append(result)
|
|
120
|
+
if step.gate:
|
|
121
|
+
aborted_at = step.index
|
|
122
|
+
break
|
|
123
|
+
continue
|
|
124
|
+
if decision.action == PolicyAction.REQUIRE_APPROVAL:
|
|
125
|
+
token = self._approval_tokens.get(step.id)
|
|
126
|
+
if token is None:
|
|
127
|
+
result = AgentStepResult(
|
|
128
|
+
step_index=step.index,
|
|
129
|
+
step_id=step.id,
|
|
130
|
+
scope=step.scope.value,
|
|
131
|
+
status="failed",
|
|
132
|
+
error=f"Requires approval: {decision.reason}. Pass --approval-token.",
|
|
133
|
+
)
|
|
134
|
+
step_results.append(result)
|
|
135
|
+
if step.gate:
|
|
136
|
+
aborted_at = step.index
|
|
137
|
+
break
|
|
138
|
+
continue
|
|
139
|
+
|
|
140
|
+
# Execute the step
|
|
141
|
+
result = self._execute_step(step)
|
|
142
|
+
step_results.append(result)
|
|
143
|
+
applied_count += 1
|
|
144
|
+
|
|
145
|
+
# Update state tracking
|
|
146
|
+
update_resource(
|
|
147
|
+
self._state,
|
|
148
|
+
step.id,
|
|
149
|
+
content_hash,
|
|
150
|
+
status="applied" if result.status == "success" else "failed",
|
|
151
|
+
)
|
|
152
|
+
|
|
153
|
+
if result.status == "failed" and (step.index == 0 or step.gate):
|
|
154
|
+
aborted_at = step.index
|
|
155
|
+
break
|
|
156
|
+
|
|
157
|
+
# Update plan-level state
|
|
158
|
+
from loft_cli_core import __version__
|
|
159
|
+
|
|
160
|
+
self._state.version = __version__
|
|
161
|
+
self._state.spec_hash = self._plan.spec_hash
|
|
162
|
+
self._state.plan_hash = self._plan.plan_hash
|
|
163
|
+
|
|
164
|
+
# Save state atomically
|
|
165
|
+
save_state(self._state, self._state_path)
|
|
166
|
+
|
|
167
|
+
finished_at = datetime.now(UTC).isoformat()
|
|
168
|
+
any_failed = aborted_at is not None or any(r.status == "failed" for r in step_results)
|
|
169
|
+
|
|
170
|
+
return AgentApplyResult(
|
|
171
|
+
plan_hash=self._plan.plan_hash,
|
|
172
|
+
spec_hash=self._plan.spec_hash,
|
|
173
|
+
step_results=step_results,
|
|
174
|
+
status="failed" if any_failed else "success",
|
|
175
|
+
aborted_at=aborted_at,
|
|
176
|
+
started_at=started_at,
|
|
177
|
+
finished_at=finished_at,
|
|
178
|
+
unchanged_count=unchanged_count,
|
|
179
|
+
applied_count=applied_count,
|
|
180
|
+
)
|
|
181
|
+
|
|
182
|
+
def _has_failed_dependency(self, step: Step, results: list[AgentStepResult]) -> bool:
|
|
183
|
+
for dep_idx in step.depends_on:
|
|
184
|
+
dep_result = next((r for r in results if r.step_index == dep_idx), None)
|
|
185
|
+
if dep_result and dep_result.status == "failed":
|
|
186
|
+
return True
|
|
187
|
+
return False
|
|
188
|
+
|
|
189
|
+
def _step_content_hash(self, step: Step) -> str:
|
|
190
|
+
"""Compute a content hash for idempotency tracking."""
|
|
191
|
+
parts = [step.id, step.command or "", step.file_content or "", step.target_path or ""]
|
|
192
|
+
return sha256_string("".join(parts))
|
|
193
|
+
|
|
194
|
+
def _execute_step(self, step: Step) -> AgentStepResult:
|
|
195
|
+
"""Execute a single step locally."""
|
|
196
|
+
start = time.monotonic()
|
|
197
|
+
try:
|
|
198
|
+
if step.kind in ("ssh_command", "agent_command"):
|
|
199
|
+
result = self._execute_command(step)
|
|
200
|
+
elif step.kind in ("ssh_upload", "agent_file_write"):
|
|
201
|
+
result = self._execute_file_write(step)
|
|
202
|
+
elif step.kind == "gate":
|
|
203
|
+
result = self._execute_gate(step)
|
|
204
|
+
elif step.kind == "verify":
|
|
205
|
+
result = self._execute_verify(step)
|
|
206
|
+
elif step.kind in ("compose_health_check", "agent_compose_health"):
|
|
207
|
+
result = self._execute_compose_health(step)
|
|
208
|
+
else:
|
|
209
|
+
result = AgentStepResult(
|
|
210
|
+
step_index=step.index,
|
|
211
|
+
step_id=step.id,
|
|
212
|
+
scope=step.scope.value,
|
|
213
|
+
status="failed",
|
|
214
|
+
error=f"Unknown step kind: '{step.kind}'",
|
|
215
|
+
)
|
|
216
|
+
except Exception as e:
|
|
217
|
+
result = AgentStepResult(
|
|
218
|
+
step_index=step.index,
|
|
219
|
+
step_id=step.id,
|
|
220
|
+
scope=step.scope.value,
|
|
221
|
+
status="failed",
|
|
222
|
+
error=str(e),
|
|
223
|
+
)
|
|
224
|
+
result.duration_seconds = time.monotonic() - start
|
|
225
|
+
return result
|
|
226
|
+
|
|
227
|
+
def _execute_command(self, step: Step) -> AgentStepResult:
|
|
228
|
+
"""Execute a shell command locally."""
|
|
229
|
+
cmd = step.command or ""
|
|
230
|
+
try:
|
|
231
|
+
if step.sudo:
|
|
232
|
+
proc = subprocess.run(
|
|
233
|
+
["sudo", "bash", "-c", cmd],
|
|
234
|
+
capture_output=True,
|
|
235
|
+
text=True,
|
|
236
|
+
timeout=300,
|
|
237
|
+
)
|
|
238
|
+
else:
|
|
239
|
+
proc = subprocess.run(
|
|
240
|
+
["bash", "-c", cmd],
|
|
241
|
+
capture_output=True,
|
|
242
|
+
text=True,
|
|
243
|
+
timeout=300,
|
|
244
|
+
)
|
|
245
|
+
return AgentStepResult(
|
|
246
|
+
step_index=step.index,
|
|
247
|
+
step_id=step.id,
|
|
248
|
+
scope=step.scope.value,
|
|
249
|
+
status="success" if proc.returncode == 0 else "failed",
|
|
250
|
+
output=proc.stdout,
|
|
251
|
+
error=proc.stderr if proc.returncode != 0 else "",
|
|
252
|
+
)
|
|
253
|
+
except subprocess.TimeoutExpired:
|
|
254
|
+
return AgentStepResult(
|
|
255
|
+
step_index=step.index,
|
|
256
|
+
step_id=step.id,
|
|
257
|
+
scope=step.scope.value,
|
|
258
|
+
status="failed",
|
|
259
|
+
error=f"Command timed out after 300s: {cmd[:80]}",
|
|
260
|
+
)
|
|
261
|
+
|
|
262
|
+
def _execute_file_write(self, step: Step) -> AgentStepResult:
|
|
263
|
+
"""Write file content directly to the filesystem."""
|
|
264
|
+
if not step.file_content or not step.target_path:
|
|
265
|
+
return AgentStepResult(
|
|
266
|
+
step_index=step.index,
|
|
267
|
+
step_id=step.id,
|
|
268
|
+
scope=step.scope.value,
|
|
269
|
+
status="failed",
|
|
270
|
+
error="Missing file_content or target_path",
|
|
271
|
+
)
|
|
272
|
+
|
|
273
|
+
target = Path(step.target_path)
|
|
274
|
+
try:
|
|
275
|
+
# Ensure parent directory exists
|
|
276
|
+
target.parent.mkdir(parents=True, exist_ok=True)
|
|
277
|
+
target.write_text(step.file_content, encoding="utf-8")
|
|
278
|
+
|
|
279
|
+
if step.sudo:
|
|
280
|
+
# Fix ownership to root when running as agent
|
|
281
|
+
subprocess.run(
|
|
282
|
+
["chown", "root:root", str(target)],
|
|
283
|
+
capture_output=True,
|
|
284
|
+
check=False,
|
|
285
|
+
)
|
|
286
|
+
subprocess.run(
|
|
287
|
+
["chmod", "600", str(target)],
|
|
288
|
+
capture_output=True,
|
|
289
|
+
check=False,
|
|
290
|
+
)
|
|
291
|
+
|
|
292
|
+
return AgentStepResult(
|
|
293
|
+
step_index=step.index,
|
|
294
|
+
step_id=step.id,
|
|
295
|
+
scope=step.scope.value,
|
|
296
|
+
status="success",
|
|
297
|
+
output=f"Written: {target}",
|
|
298
|
+
)
|
|
299
|
+
except Exception as e:
|
|
300
|
+
return AgentStepResult(
|
|
301
|
+
step_index=step.index,
|
|
302
|
+
step_id=step.id,
|
|
303
|
+
scope=step.scope.value,
|
|
304
|
+
status="failed",
|
|
305
|
+
error=f"File write failed: {e}",
|
|
306
|
+
)
|
|
307
|
+
|
|
308
|
+
def _execute_gate(self, step: Step) -> AgentStepResult:
|
|
309
|
+
"""Execute a gate step — locally verify SSH or service accessibility."""
|
|
310
|
+
# On the agent, gate checks can verify services are reachable locally
|
|
311
|
+
if step.command and step.command.startswith("ssh_check:"):
|
|
312
|
+
_, host, port_str, user = step.command.split(":", 3)
|
|
313
|
+
import socket
|
|
314
|
+
|
|
315
|
+
try:
|
|
316
|
+
with socket.create_connection((host, int(port_str)), timeout=5):
|
|
317
|
+
return AgentStepResult(
|
|
318
|
+
step_index=step.index,
|
|
319
|
+
step_id=step.id,
|
|
320
|
+
scope=step.scope.value,
|
|
321
|
+
status="success",
|
|
322
|
+
output=f"Port {port_str} reachable on {host}",
|
|
323
|
+
)
|
|
324
|
+
except OSError as e:
|
|
325
|
+
return AgentStepResult(
|
|
326
|
+
step_index=step.index,
|
|
327
|
+
step_id=step.id,
|
|
328
|
+
scope=step.scope.value,
|
|
329
|
+
status="failed",
|
|
330
|
+
error=f"Port {port_str} unreachable on {host}: {e}",
|
|
331
|
+
)
|
|
332
|
+
|
|
333
|
+
if step.command and step.command.startswith("http_check:"):
|
|
334
|
+
return self._execute_http_check(step)
|
|
335
|
+
|
|
336
|
+
# Fallback: run as command
|
|
337
|
+
return self._execute_command(step)
|
|
338
|
+
|
|
339
|
+
def _execute_verify(self, step: Step) -> AgentStepResult:
|
|
340
|
+
"""Execute a verification step locally."""
|
|
341
|
+
if step.command and step.command.startswith("check:"):
|
|
342
|
+
return self._execute_local_check(step)
|
|
343
|
+
if step.command:
|
|
344
|
+
return self._execute_command(step)
|
|
345
|
+
return AgentStepResult(
|
|
346
|
+
step_index=step.index,
|
|
347
|
+
step_id=step.id,
|
|
348
|
+
scope=step.scope.value,
|
|
349
|
+
status="success",
|
|
350
|
+
output="ok",
|
|
351
|
+
)
|
|
352
|
+
|
|
353
|
+
def _execute_local_check(self, step: Step) -> AgentStepResult:
|
|
354
|
+
"""Dispatch a postflight check locally."""
|
|
355
|
+
parts = step.command.split(":", maxsplit=2) if step.command else []
|
|
356
|
+
check_type = parts[1] if len(parts) > 1 else ""
|
|
357
|
+
params = parts[2] if len(parts) > 2 else ""
|
|
358
|
+
|
|
359
|
+
try:
|
|
360
|
+
if check_type == "port_open":
|
|
361
|
+
import socket
|
|
362
|
+
|
|
363
|
+
host, port_str = params.split(":", 1)
|
|
364
|
+
try:
|
|
365
|
+
with socket.create_connection((host, int(port_str)), timeout=5):
|
|
366
|
+
return self._check_result(step, True, f"Port {port_str} open on {host}")
|
|
367
|
+
except OSError:
|
|
368
|
+
return self._check_result(step, False, f"Port {port_str} closed on {host}")
|
|
369
|
+
|
|
370
|
+
elif check_type in ("postgres_ready", "nginx_ready", "wireguard_up"):
|
|
371
|
+
# Run a simple local service check
|
|
372
|
+
check_cmds = {
|
|
373
|
+
"postgres_ready": "pg_isready",
|
|
374
|
+
"nginx_ready": "nginx -t",
|
|
375
|
+
"wireguard_up": f"wg show {params}",
|
|
376
|
+
}
|
|
377
|
+
cmd = check_cmds[check_type]
|
|
378
|
+
proc = subprocess.run(
|
|
379
|
+
["bash", "-c", cmd], capture_output=True, text=True, timeout=10
|
|
380
|
+
)
|
|
381
|
+
return self._check_result(step, proc.returncode == 0, proc.stdout or proc.stderr)
|
|
382
|
+
|
|
383
|
+
elif check_type == "container_running":
|
|
384
|
+
proc = subprocess.run(
|
|
385
|
+
["docker", "inspect", "-f", "{{.State.Running}}", params],
|
|
386
|
+
capture_output=True,
|
|
387
|
+
text=True,
|
|
388
|
+
timeout=10,
|
|
389
|
+
)
|
|
390
|
+
running = proc.stdout.strip() == "true"
|
|
391
|
+
return self._check_result(step, running, f"Container {params}: running={running}")
|
|
392
|
+
|
|
393
|
+
elif check_type == "http":
|
|
394
|
+
import urllib.request
|
|
395
|
+
|
|
396
|
+
url, status_str = params.rsplit(":", 1)
|
|
397
|
+
try:
|
|
398
|
+
resp = urllib.request.urlopen(url, timeout=10) # noqa: S310
|
|
399
|
+
actual = resp.status
|
|
400
|
+
passed = actual == int(status_str)
|
|
401
|
+
return self._check_result(
|
|
402
|
+
step, passed, f"HTTP {url}: {actual} (expected {status_str})"
|
|
403
|
+
)
|
|
404
|
+
except Exception as e:
|
|
405
|
+
return self._check_result(step, False, f"HTTP {url}: {e}")
|
|
406
|
+
|
|
407
|
+
else:
|
|
408
|
+
return self._check_result(step, False, f"Unknown check type: {check_type}")
|
|
409
|
+
|
|
410
|
+
except Exception as exc:
|
|
411
|
+
return self._check_result(step, False, f"Check error: {exc}")
|
|
412
|
+
|
|
413
|
+
def _check_result(self, step: Step, passed: bool, message: str) -> AgentStepResult:
|
|
414
|
+
return AgentStepResult(
|
|
415
|
+
step_index=step.index,
|
|
416
|
+
step_id=step.id,
|
|
417
|
+
scope=step.scope.value,
|
|
418
|
+
status="success" if passed else "failed",
|
|
419
|
+
output=message if passed else "",
|
|
420
|
+
error="" if passed else message,
|
|
421
|
+
)
|
|
422
|
+
|
|
423
|
+
def _execute_http_check(self, step: Step) -> AgentStepResult:
|
|
424
|
+
"""Execute an HTTP readiness check with retry loop."""
|
|
425
|
+
import urllib.request
|
|
426
|
+
|
|
427
|
+
# The URL may contain colons (e.g. http://host:port/), so parse
|
|
428
|
+
# from the right: the trailing 4 fields are always integers.
|
|
429
|
+
rest = step.command[len("http_check:") :]
|
|
430
|
+
parts = rest.rsplit(":", 4)
|
|
431
|
+
if len(parts) != 5:
|
|
432
|
+
return AgentStepResult(
|
|
433
|
+
step_index=step.index,
|
|
434
|
+
step_id=step.id,
|
|
435
|
+
scope=step.scope.value,
|
|
436
|
+
status="failed",
|
|
437
|
+
error=f"Malformed http_check command: {step.command}",
|
|
438
|
+
)
|
|
439
|
+
|
|
440
|
+
url, status_str, retries_str, interval_str, timeout_str = parts
|
|
441
|
+
expected = int(status_str)
|
|
442
|
+
retries = int(retries_str)
|
|
443
|
+
interval = int(interval_str)
|
|
444
|
+
req_timeout = int(timeout_str)
|
|
445
|
+
|
|
446
|
+
last_error = ""
|
|
447
|
+
for attempt in range(retries):
|
|
448
|
+
try:
|
|
449
|
+
resp = urllib.request.urlopen(url, timeout=req_timeout) # noqa: S310
|
|
450
|
+
if resp.status == expected:
|
|
451
|
+
return AgentStepResult(
|
|
452
|
+
step_index=step.index,
|
|
453
|
+
step_id=step.id,
|
|
454
|
+
scope=step.scope.value,
|
|
455
|
+
status="success",
|
|
456
|
+
output=f"HTTP {url}: {resp.status} (expected {expected}) "
|
|
457
|
+
f"on attempt {attempt + 1}/{retries}",
|
|
458
|
+
)
|
|
459
|
+
last_error = f"HTTP {url}: got {resp.status}, expected {expected}"
|
|
460
|
+
except Exception as e:
|
|
461
|
+
last_error = f"HTTP {url}: {e}"
|
|
462
|
+
if attempt < retries - 1:
|
|
463
|
+
time.sleep(interval)
|
|
464
|
+
|
|
465
|
+
return AgentStepResult(
|
|
466
|
+
step_index=step.index,
|
|
467
|
+
step_id=step.id,
|
|
468
|
+
scope=step.scope.value,
|
|
469
|
+
status="failed",
|
|
470
|
+
error=f"HTTP check failed after {retries} attempts: {last_error}",
|
|
471
|
+
)
|
|
472
|
+
|
|
473
|
+
def _execute_compose_health(self, step: Step) -> AgentStepResult:
|
|
474
|
+
"""Poll docker compose ps for container health."""
|
|
475
|
+
if not step.command or not step.command.startswith("compose_health:"):
|
|
476
|
+
return AgentStepResult(
|
|
477
|
+
step_index=step.index,
|
|
478
|
+
step_id=step.id,
|
|
479
|
+
scope=step.scope.value,
|
|
480
|
+
status="failed",
|
|
481
|
+
error=f"Invalid compose_health command: {step.command}",
|
|
482
|
+
)
|
|
483
|
+
|
|
484
|
+
parts = step.command.split(":", 5)
|
|
485
|
+
if len(parts) != 6:
|
|
486
|
+
return AgentStepResult(
|
|
487
|
+
step_index=step.index,
|
|
488
|
+
step_id=step.id,
|
|
489
|
+
scope=step.scope.value,
|
|
490
|
+
status="failed",
|
|
491
|
+
error="Malformed compose_health command",
|
|
492
|
+
)
|
|
493
|
+
|
|
494
|
+
_, directory, compose_file, project_name, timeout_s, interval_s = parts
|
|
495
|
+
timeout = int(timeout_s)
|
|
496
|
+
interval = int(interval_s)
|
|
497
|
+
deadline = time.monotonic() + timeout
|
|
498
|
+
|
|
499
|
+
while time.monotonic() < deadline:
|
|
500
|
+
proc = subprocess.run(
|
|
501
|
+
[
|
|
502
|
+
"bash",
|
|
503
|
+
"-c",
|
|
504
|
+
f"cd {directory} && docker compose -f {compose_file} "
|
|
505
|
+
f"-p {project_name} ps --format json",
|
|
506
|
+
],
|
|
507
|
+
capture_output=True,
|
|
508
|
+
text=True,
|
|
509
|
+
timeout=30,
|
|
510
|
+
)
|
|
511
|
+
if proc.returncode == 0 and proc.stdout.strip():
|
|
512
|
+
try:
|
|
513
|
+
# Parse container statuses
|
|
514
|
+
all_healthy = True
|
|
515
|
+
for line in proc.stdout.strip().splitlines():
|
|
516
|
+
container = json.loads(line)
|
|
517
|
+
health = container.get("Health", container.get("Status", ""))
|
|
518
|
+
if "healthy" not in health.lower() and "running" not in health.lower():
|
|
519
|
+
all_healthy = False
|
|
520
|
+
break
|
|
521
|
+
if all_healthy:
|
|
522
|
+
return AgentStepResult(
|
|
523
|
+
step_index=step.index,
|
|
524
|
+
step_id=step.id,
|
|
525
|
+
scope=step.scope.value,
|
|
526
|
+
status="success",
|
|
527
|
+
output=f"All containers healthy in {project_name}",
|
|
528
|
+
)
|
|
529
|
+
except (json.JSONDecodeError, KeyError):
|
|
530
|
+
pass
|
|
531
|
+
time.sleep(interval)
|
|
532
|
+
|
|
533
|
+
return AgentStepResult(
|
|
534
|
+
step_index=step.index,
|
|
535
|
+
step_id=step.id,
|
|
536
|
+
scope=step.scope.value,
|
|
537
|
+
status="failed",
|
|
538
|
+
error=f"Health check timed out after {timeout}s for {project_name}",
|
|
539
|
+
)
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
"""Mutation locking for the loft-cli agent.
|
|
2
|
+
|
|
3
|
+
Only one mutation (apply) can run at a time on a managed server.
|
|
4
|
+
Uses file-based locking with fcntl for atomic, non-blocking acquisition.
|
|
5
|
+
"""
|
|
6
|
+
|
|
7
|
+
from __future__ import annotations
|
|
8
|
+
|
|
9
|
+
import contextlib
|
|
10
|
+
import fcntl
|
|
11
|
+
import json
|
|
12
|
+
import os
|
|
13
|
+
import time
|
|
14
|
+
|
|
15
|
+
from loft_cli_agent import paths as agent_paths
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
class LockError(RuntimeError):
|
|
19
|
+
"""Raised when the mutation lock cannot be acquired."""
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
class MutationLock:
|
|
23
|
+
"""Context manager for exclusive mutation locking.
|
|
24
|
+
|
|
25
|
+
Usage::
|
|
26
|
+
|
|
27
|
+
with MutationLock():
|
|
28
|
+
# Only one apply runs at a time
|
|
29
|
+
agent_executor.apply(plan)
|
|
30
|
+
"""
|
|
31
|
+
|
|
32
|
+
def __init__(self, lock_name: str = "apply") -> None:
|
|
33
|
+
self._lock_name = lock_name
|
|
34
|
+
self._fd: int | None = None
|
|
35
|
+
|
|
36
|
+
@property
|
|
37
|
+
def _lock_path(self):
|
|
38
|
+
return agent_paths.AGENT_LOCK_DIR / f"{self._lock_name}.lock"
|
|
39
|
+
|
|
40
|
+
def __enter__(self) -> MutationLock:
|
|
41
|
+
self._lock_path.parent.mkdir(parents=True, exist_ok=True)
|
|
42
|
+
|
|
43
|
+
self._fd = os.open(str(self._lock_path), os.O_CREAT | os.O_RDWR, 0o644)
|
|
44
|
+
try:
|
|
45
|
+
fcntl.flock(self._fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
|
46
|
+
except OSError:
|
|
47
|
+
os.close(self._fd)
|
|
48
|
+
self._fd = None
|
|
49
|
+
# Try to read who holds the lock
|
|
50
|
+
holder = self._read_holder()
|
|
51
|
+
raise LockError(
|
|
52
|
+
f"Another mutation is in progress. Lock: {self._lock_path}"
|
|
53
|
+
+ (f" (held by PID {holder.get('pid')})" if holder else "")
|
|
54
|
+
) from None
|
|
55
|
+
|
|
56
|
+
# Write PID and timestamp to lock file
|
|
57
|
+
os.ftruncate(self._fd, 0)
|
|
58
|
+
os.lseek(self._fd, 0, os.SEEK_SET)
|
|
59
|
+
info = json.dumps({"pid": os.getpid(), "started_at": time.time()})
|
|
60
|
+
os.write(self._fd, info.encode())
|
|
61
|
+
return self
|
|
62
|
+
|
|
63
|
+
def __exit__(self, *exc) -> None:
|
|
64
|
+
if self._fd is not None:
|
|
65
|
+
fcntl.flock(self._fd, fcntl.LOCK_UN)
|
|
66
|
+
os.close(self._fd)
|
|
67
|
+
self._fd = None
|
|
68
|
+
# Remove lock file on clean exit
|
|
69
|
+
with contextlib.suppress(OSError):
|
|
70
|
+
self._lock_path.unlink(missing_ok=True)
|
|
71
|
+
|
|
72
|
+
def _read_holder(self) -> dict | None:
|
|
73
|
+
try:
|
|
74
|
+
content = self._lock_path.read_text(encoding="utf-8")
|
|
75
|
+
return json.loads(content)
|
|
76
|
+
except Exception:
|
|
77
|
+
return None
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
"""Server-side path management for the loft-cli agent.
|
|
2
|
+
|
|
3
|
+
Path constants are defined in loft_cli_core.agent_paths (shared with client).
|
|
4
|
+
This module re-exports them for convenience and adds agent-only functions.
|
|
5
|
+
"""
|
|
6
|
+
|
|
7
|
+
from __future__ import annotations
|
|
8
|
+
|
|
9
|
+
from loft_cli_core.agent_paths import (
|
|
10
|
+
AGENT_BINARY_PATH,
|
|
11
|
+
AGENT_CONFIG_DIR,
|
|
12
|
+
AGENT_DESIRED_DIR,
|
|
13
|
+
AGENT_LOCK_DIR,
|
|
14
|
+
AGENT_LOG_DIR,
|
|
15
|
+
AGENT_STATE_DIR,
|
|
16
|
+
AGENT_STATE_FILE,
|
|
17
|
+
)
|
|
18
|
+
|
|
19
|
+
__all__ = [
|
|
20
|
+
"AGENT_BINARY_PATH",
|
|
21
|
+
"AGENT_CONFIG_DIR",
|
|
22
|
+
"AGENT_DESIRED_DIR",
|
|
23
|
+
"AGENT_LOCK_DIR",
|
|
24
|
+
"AGENT_LOG_DIR",
|
|
25
|
+
"AGENT_STATE_DIR",
|
|
26
|
+
"AGENT_STATE_FILE",
|
|
27
|
+
"ensure_agent_dirs",
|
|
28
|
+
]
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
def ensure_agent_dirs() -> None:
|
|
32
|
+
"""Create all agent directories if they don't exist."""
|
|
33
|
+
for d in (AGENT_CONFIG_DIR, AGENT_STATE_DIR, AGENT_LOCK_DIR, AGENT_DESIRED_DIR, AGENT_LOG_DIR):
|
|
34
|
+
d.mkdir(parents=True, exist_ok=True)
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
"""Runtime state management for the loft-cli agent.
|
|
2
|
+
|
|
3
|
+
Tracks which resources have been applied and their content hashes,
|
|
4
|
+
enabling idempotent re-apply (skip unchanged resources).
|
|
5
|
+
|
|
6
|
+
The state models (RuntimeState, ResourceState) live in loft_cli_core.state
|
|
7
|
+
so that both the client (for diff rendering) and the agent can use them.
|
|
8
|
+
This module provides the I/O functions for loading and saving state on disk.
|
|
9
|
+
"""
|
|
10
|
+
|
|
11
|
+
from __future__ import annotations
|
|
12
|
+
|
|
13
|
+
import contextlib
|
|
14
|
+
import os
|
|
15
|
+
import tempfile
|
|
16
|
+
from datetime import UTC, datetime
|
|
17
|
+
from pathlib import Path
|
|
18
|
+
|
|
19
|
+
from loft_cli_core.agent_paths import AGENT_STATE_FILE
|
|
20
|
+
from loft_cli_core.state import ResourceState, RuntimeState
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def load_state(state_path: Path | None = None) -> RuntimeState:
|
|
24
|
+
"""Load runtime state from disk. Returns empty state if file doesn't exist."""
|
|
25
|
+
path = state_path or AGENT_STATE_FILE
|
|
26
|
+
if not path.exists():
|
|
27
|
+
return RuntimeState()
|
|
28
|
+
try:
|
|
29
|
+
content = path.read_text(encoding="utf-8")
|
|
30
|
+
return RuntimeState.model_validate_json(content)
|
|
31
|
+
except Exception:
|
|
32
|
+
return RuntimeState()
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def save_state(state: RuntimeState, state_path: Path | None = None) -> None:
|
|
36
|
+
"""Atomically write runtime state to disk (write to tmp, rename)."""
|
|
37
|
+
path = state_path or AGENT_STATE_FILE
|
|
38
|
+
path.parent.mkdir(parents=True, exist_ok=True)
|
|
39
|
+
|
|
40
|
+
# Write to temp file in same directory, then rename for atomicity
|
|
41
|
+
fd, tmp_path = tempfile.mkstemp(dir=str(path.parent), suffix=".tmp")
|
|
42
|
+
try:
|
|
43
|
+
os.write(fd, state.model_dump_json(indent=2).encode("utf-8"))
|
|
44
|
+
os.close(fd)
|
|
45
|
+
os.rename(tmp_path, str(path))
|
|
46
|
+
except Exception:
|
|
47
|
+
os.close(fd) if not os.get_inheritable(fd) else None
|
|
48
|
+
with contextlib.suppress(OSError):
|
|
49
|
+
os.unlink(tmp_path)
|
|
50
|
+
raise
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def resource_changed(state: RuntimeState, resource_id: str, content_hash: str) -> bool:
|
|
54
|
+
"""Return True if the resource has changed or was never applied."""
|
|
55
|
+
existing = state.resources.get(resource_id)
|
|
56
|
+
if existing is None:
|
|
57
|
+
return True
|
|
58
|
+
return existing.content_hash != content_hash
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
def update_resource(
|
|
62
|
+
state: RuntimeState,
|
|
63
|
+
resource_id: str,
|
|
64
|
+
content_hash: str,
|
|
65
|
+
status: str = "applied",
|
|
66
|
+
) -> None:
|
|
67
|
+
"""Update a resource's state after execution."""
|
|
68
|
+
state.resources[resource_id] = ResourceState(
|
|
69
|
+
resource_id=resource_id,
|
|
70
|
+
content_hash=content_hash,
|
|
71
|
+
applied_at=datetime.now(UTC).isoformat(),
|
|
72
|
+
status=status,
|
|
73
|
+
)
|
|
74
|
+
state.last_applied = datetime.now(UTC).isoformat()
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: loft-cli-agent
|
|
3
|
+
Version: 0.6.3
|
|
4
|
+
Summary: Server-side agent for loft-cli — executes plans locally on managed servers
|
|
5
|
+
Requires-Python: >=3.11
|
|
6
|
+
Requires-Dist: loft-cli-core>=0.5.0
|
|
7
|
+
Requires-Dist: typer[all]>=0.9.0
|
|
8
|
+
Requires-Dist: rich>=13.0
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
README.md
|
|
2
|
+
pyproject.toml
|
|
3
|
+
loft_cli_agent/__init__.py
|
|
4
|
+
loft_cli_agent/cli.py
|
|
5
|
+
loft_cli_agent/executor.py
|
|
6
|
+
loft_cli_agent/lock.py
|
|
7
|
+
loft_cli_agent/paths.py
|
|
8
|
+
loft_cli_agent/state.py
|
|
9
|
+
loft_cli_agent.egg-info/PKG-INFO
|
|
10
|
+
loft_cli_agent.egg-info/SOURCES.txt
|
|
11
|
+
loft_cli_agent.egg-info/dependency_links.txt
|
|
12
|
+
loft_cli_agent.egg-info/entry_points.txt
|
|
13
|
+
loft_cli_agent.egg-info/requires.txt
|
|
14
|
+
loft_cli_agent.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
loft_cli_agent
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=68.0"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "loft-cli-agent"
|
|
7
|
+
version = "0.6.3"
|
|
8
|
+
description = "Server-side agent for loft-cli — executes plans locally on managed servers"
|
|
9
|
+
requires-python = ">=3.11"
|
|
10
|
+
dependencies = [
|
|
11
|
+
"loft-cli-core>=0.5.0",
|
|
12
|
+
"typer[all]>=0.9.0",
|
|
13
|
+
"rich>=13.0",
|
|
14
|
+
]
|
|
15
|
+
|
|
16
|
+
[project.scripts]
|
|
17
|
+
loft-cli-agent = "loft_cli_agent.cli:app"
|
|
18
|
+
|
|
19
|
+
[tool.setuptools.packages.find]
|
|
20
|
+
where = ["."]
|
|
21
|
+
include = ["loft_cli_agent*"]
|