hostctl 0.1.41 → 0.1.44

package/README.md

@@ -1,30 +1,63 @@
 # hostctl
 
-`hostctl` is a modern task runner for managing fleets of hosts. It makes it easy to execute TypeScript or JavaScript automations locally or over SSH while keeping secrets, inventories, and host metadata in one place.
+`hostctl` is a modern task runner for managing fleets of hosts. It executes TypeScript or JavaScript automations locally or over SSH while keeping inventories, tags, and secrets in one place.
 
 ## Why hostctl
-- Treat remote automation like regular code: tasks are modules with strong typing and rich logging.
+
+- Treat remote automation like regular code: tasks are modules with strong typing and structured logging.
 - First-class inventory and secrets management built around [AGE](https://age-encryption.org/).
-- Layered runtime isolates CLI parsing, orchestration, and local/remote execution for easy extension.
+- Layered runtime cleanly separates CLI parsing, orchestration, and execution for easy extension.
 
-## Quick Start
-1. **Install dependencies**  
-   - Node.js 24+ (the CLI runs through `tsx` to execute TypeScript directly)  
-   - `age`: `brew install age` (macOS) or follow <https://age-encryption.org>.
-2. **Clone and bootstrap**
-   ```bash
-   git clone https://github.com/monopod/hostctl.git
-   cd hostctl
-   npm install
-   ```
-3. **Generate AGE identities**
+## System Requirements
+
+- Node.js **24+** (CLI runs through `tsx`).
+- [`age`](https://age-encryption.org) for encrypting secrets.
+- Git (recommended) for cloning packages and publishing tasks.
+
+Check your toolchain:
+
+```bash
+node -v
+npm -v
+age --version
+```
+
+## Install hostctl
+
+### Clone & Develop
+
+```bash
+git clone https://github.com/monopod/hostctl.git
+cd hostctl
+npm install
+```
+
+Use `./hostctl` during development or `npm run build && ./dist/bin/hostctl.js ...` to test the bundled CLI.
+
+### Global CLI
+
+```bash
+npm install -g hostctl
+hostctl --help
+```
+
+### One-off Execution
+
+Run straight from npm without installing globally:
+
+```bash
+npx hostctl run core.echo message:hello
+```
+
+## Bootstrap identities, inventory, and secrets
+
+1. **Generate AGE identities**
    ```bash
    mkdir -p ~/.hostctl/age
-   age-keygen -o ~/.hostctl/age/jane.priv
-   age-keygen -y ~/.hostctl/age/jane.priv > ~/.hostctl/age/jane.pub
+   age-keygen -o ~/.hostctl/age/you.priv
+   age-keygen -y ~/.hostctl/age/you.priv > ~/.hostctl/age/you.pub
    ```
-4. **Author an inventory**  
-   Create `~/.hostctl/hostctl.yaml`:
+2. **Author an inventory** at `~/.hostctl/hostctl.yaml`:
    ```yaml
    hosts:
      ubuntu-vm:
@@ -34,59 +67,162 @@
        tags: [ubuntu, testvm]
    secrets:
      vagrant-password:
-       ids: jane
+       ids: you
        value: vagrant
    ids:
-     jane: age1...
+     you: age1...
+   ```
+3. **Manage the file**
+   ```bash
+   hostctl inventory encrypt   # wrap with AGE recipients
+   hostctl inventory decrypt   # view/edit locally
+   hostctl inventory list      # inspect hosts & tags
    ```
-   See `docs/system_management.md` for more host examples and encryption workflows.
-5. **Run an example task**
+   Set `AGE_IDS="~/.hostctl/age/*.priv"` before running commands if the file is encrypted.
+
+## Running tasks
+
+- **Local script**
   ```bash
   npm run build
   ./dist/bin/hostctl.js run example/echo.ts args:hello
   ```
-  Use `AGE_IDS="~/.hostctl/age/*.priv"` to point at custom identities.
-
-6. **Target remote hosts by tag**
-   ```bash
-   ./dist/bin/hostctl.js run -r -t xcpng-lab --json core.net.interfaces
-   ```
-   Flags:
-   - `-r` executes the task against matching remote hosts from your inventory.
-   - `-t <tag>` narrows the target set; multiple tags may be comma-separated.
-   - `--json` emits machine-readable results per host.
+- **Local package**
+  ```bash
+  hostctl run ./my-package hello name:Sam
+  ```
+  Local directories are executed directly and are not installable via `hostctl pkg install`.
+- **Remote orchestration**
+  ```bash
+  hostctl run -r -t ubuntu core.net.interfaces --json
+  ```
 
-## Install via npm
-- Global CLI:
+  - `-r/--remote` targets hosts selected by tags via SSH.
+  - `-t/--tag` is greedy; use `--` before positional args when needed.
+- **From npm or git**
+  ```bash
+  hostctl run hostctl-hello greet name:Phil
+  hostctl run https://github.com/monopod/hostctl-example echo args:hello,world
+  ```
+- **Install Docker anywhere**
   ```bash
-  npm install -g hostctl
-  hostctl run example/echo.ts args:hello
+  hostctl run -r -t ubuntu core.docker.install users:hostctl
   ```
-- One-off execution:
+  The task follows Docker’s official installation guides for Ubuntu/Debian and Fedora/RHEL/Rocky (matching the current `xcpng-e2e` templates), so the same invocation works across your lab images.
+- **Run containers**
   ```bash
-  npx hostctl run core.echo message:hello
+  hostctl run core.docker.run-container image:alpine command:'["/bin/sh","-c","echo from container"]'
+  hostctl run core.docker.run-container-detached image:redis name:ci-cache
   ```
-The published binary keeps its `tsx`-based runtime so TypeScript scripts can execute without pre-compilation; ensure your runtime Node version is 24 or newer.
-
-## Documentation Map
-- **Architecture & Design**: [ARCHITECTURE.md](ARCHITECTURE.md) captures module boundaries, runtime flows, and design principles.  
-- **Operational Guides**: User-facing how‑tos live in `docs/` (e.g., `docs/process_management.md`, `docs/package_management.md`, `docs/system_management.md`).  
-- **Detailed Onboarding**: [docs/user-onboarding.md](docs/user-onboarding.md) preserves the full setup, CLI, and task catalog reference.  
-- **XCP-ng Provisioning**: [docs/xcp-ng-operations.md](docs/xcp-ng-operations.md) covers the emerging hypervisor-native testing workflow.  
-- **LLM Contributor Guide**: See [AGENTS.md](AGENTS.md) for automated contributors.  
-- **Community Standards**: [CONTRIBUTING.md](CONTRIBUTING.md) outlines coding standards, issue triage, and release process.
-
-## Developer Workflow
-- Format & typing: `npm run format` (Prettier) and `npm run lint` (TypeScript no-emit).
-- Build artifacts: `npm run build` (tsup) produces `dist/` bundles consumed by the CLI and published package.
+  The first task streams container output back to the CLI; the detached variant returns the created container ID/name so you can manage it later.
+
+**Passing parameters**
+
+- `key:value` pairs after the task name.
+- `--params '{"key":"value"}'` for JSON blobs.
+- `--file params.json` to load structured arguments.
+
+## Managing task packages
+
+`hostctl pkg` commands wrap the npm-only package manager:
+
+```bash
+hostctl pkg create my-task --lang typescript
+hostctl pkg install hostctl-hello
+hostctl pkg list
+hostctl pkg remove hostctl-hello
+```
+
+Installed packages live under `~/.hostctl/packages` and can be run offline once cached.
+Use `pkg install` for npm registry names (scoped + versioned) or git URLs. Local directories should be run directly without installing (`hostctl run ./path/to/pkg task args`); this avoids polluting the manifest with workstation paths and keeps the install story aligned with reproducible npm/git sources.
+
+## Designing tasks
+
+Define tasks with the `task` helper and a typed `TaskContext`:
+
+```ts
+import { task, type TaskContext } from 'hostctl';
+
+interface EchoParams {
+  message: string;
+}
+interface EchoResult {
+  repeated: string;
+}
+
+async function run(context: TaskContext<EchoParams>): Promise<EchoResult> {
+  const { params, info } = context;
+  info(`Echo: ${params.message}`);
+  return { repeated: params.message };
+}
+
+export default task(run, { name: 'echo', description: 'Prints a message' });
+```
+
+Export one default task per file. Qualified task names come from a registry when present; otherwise they are derived from the task file path relative to the package root.
+
+Key `TaskContext` capabilities (see [`docs/task-api.md`](docs/task-api.md) for details):
+
+- Structured logging via `info`, `warn`, `error`, `debug`, or `log(level, ...)`.
+- Command execution with `exec(command, { sudo, env, cwd, stdin, pty })`.
+- Remote fan-out using `ssh(tags, remoteFn)`.
+- Subtask orchestration with `run(otherTask(params))`.
+- Secrets & credentials via `getSecret(name)` and `getPassword()`.
+- Inventory helpers: `inventory(tags)` and `selectedInventory(tags?)`.
+- File helpers: `file.read`, `file.write`, `file.exists`, `file.mkdir`, `file.rm` that respect local vs. remote runtime.
+
+## Building & testing task packages
+
+1. Scaffold: `hostctl pkg create awesome-firewall --lang typescript`.
+2. Install deps: `cd awesome-firewall && npm install`.
+3. Build: `npm run build` (defaults to `tsc`).
+4. Test locally without publishing: `npx hostctl run . args:foo`.
+5. Add unit tests with Vitest if needed, or invoke tasks directly in scripts.
+
+## Publishing task packages
+
+1. Update `package.json` metadata (`name`, `version`, `description`, `files`).
+2. Build artifacts (`npm run build` or rely on `prepublishOnly`).
+3. Authenticate with npm (`npm login` or `NPM_TOKEN`).
+4. Publish:
+   ```bash
+   npm version patch   # or minor/major
+   npm publish --access public
+   ```
+5. Tag releases in git or automate with tools like `release-it` (this repo ships `release.sh` as an example flow).
+6. Verify: `npm info your-package version`.
+
+## Consuming published packages
+
+- One-off: `npx hostctl run your-package taskName param:value`.
+- Cache locally: `hostctl pkg install your-package@1.2.3` then run offline.
+- Compose in other codebases by adding the package to `dependencies` and importing its exported tasks.
+
+## Troubleshooting
+
+- **Task not found**: confirm the package exports the task name and that `exports` exposes it.
+- **Version mismatch**: keep `package.json`, `src/version.ts`, and `jsr.json` in sync before releasing.
+- **SSH failures**: verify inventory entries (hostname, user, auth) and only pass `-r` when you intend remote execution.
+- **Secrets**: when inventories are encrypted, ensure `AGE_IDS` includes the private keys so `hostctl` can decrypt secrets.
+- **Environment drift**: run `hostctl runtime` to inspect prerequisites or `hostctl runtime install` to bootstrap them.
+
+## Developer workflow
+
+- Format & typing: `npm run format` (Prettier) and `npm run lint` (`tsc --noEmit`).
+- Build artifacts: `npm run build` (tsup) produces `dist/` bundles for the CLI and published package.
 - Tests:
   - `npm run test` → unit + functional suites.
   - `npm run test:unit`, `npm run test:functional`, `npm run test:e2e` for focused runs.
-- VM-backed runs currently lean on legacy Vagrant boxes. Those helpers (`npm run vm:*`) remain for continuity but are **deprecated** while we shift to provisioning test VMs directly on XCP-ng hypervisors. See [docs/xcp-ng-operations.md](docs/xcp-ng-operations.md) for the in-progress workflow and contribute new test setups there.
-- Release scripts rely on `npm run release` and accompanying tooling described in `NPM_MIGRATION_PLAN.md`.
-
-## Next Steps
-- Explore bundled examples under `example/` to learn the task API.
-- Review [ARCHITECTURE.md](ARCHITECTURE.md) for runtime internals and extension guidelines.
-- Familiarise yourself with the new XCP-ng task suite to help migrate integration tests off the Vagrant stack.
-- Join discussions or open issues at <https://github.com/monopod/hostctl/issues>.
+- VM helpers (`npm run vm:*`) are legacy while XCP-ng provisioning matures; see `docs/xcp-ng-operations.md`.
+- Releases rely on `npm run release` + `release-it`; see `NPM_MIGRATION_PLAN.md` for npm-only context.
+
+## Documentation map
+
+- **Task API reference**: [`docs/task-api.md`](docs/task-api.md).
+- **Task package authoring**: [`docs/task-package-authoring.md`](docs/task-package-authoring.md).
+- **Operational guides**: `docs/*.md` (process, package, system management, etc.).
+- **Architecture & design**: [`ARCHITECTURE.md`](ARCHITECTURE.md) and supporting design docs.
+- **Contributor guide**: [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`AGENTS.md`](AGENTS.md).
+- **Issue tracking**: <https://github.com/monopod/hostctl/issues>.
+
+Explore the examples under `example/`, get familiar with the task API, and share automations with the community via npm-compatible packages.