hostctl 0.1.37 → 0.1.40

package/README.md

@@ -202,7 +202,7 @@ johndoe: - age1jx3ec0y5hctkfwfut4p8cj08ulcezqex8r8slwlwgwy8k2plx5ksdzy8ll
 janedoe: - age1qdqv054hmfmnxk56dkjm3cq8qz4lxfl7tvqfp0rqdl0nyygjlccqw0ly40
 
 hostctl on  main [!] is 📦 v0.1.31 via 🥟 v1.2.10 via  v23.11.0
-❯ AGE_IDS="example/sample-age-ids/*.priv" hostctl inventory -c example/hostctl.yaml decrypt -v
+❯ AGE_IDS="example/sample-age-ids/\*.priv" hostctl inventory -c example/hostctl.yaml decrypt -v
 Decrypting inventory file: example/hostctl.yaml
 Decrypted with one or more of the following private keys:
 example/sample-age-ids/johndoe.priv
@@ -227,7 +227,7 @@ ids:
 johndoe: - age1jx3ec0y5hctkfwfut4p8cj08ulcezqex8r8slwlwgwy8k2plx5ksdzy8ll
 janedoe: - age1qdqv054hmfmnxk56dkjm3cq8qz4lxfl7tvqfp0rqdl0nyygjlccqw0ly40
 
-```
+````
 
 ## Vision
 
@@ -294,6 +294,24 @@ The vision for `hostctl` is to be an Ansible replacement that is significantly e
 - **`core.pkg.remove`**: Removes a package.
 - **`core.pkg.update`**: Updates a package.
 
+#### Abstract Package Management (Cross-Platform, Non-Interactive)
+- **`core.pkg.abstract-install`**: Install packages using auto-detected or specified package manager (non-interactive, no progress bars).
+- **`core.pkg.abstract-uninstall`**: Uninstall packages using auto-detected or specified package manager (non-interactive, no progress bars).
+- **`core.pkg.abstract-update`**: Update package sources using auto-detected or specified package manager (non-interactive, no progress bars).
+- **`core.pkg.abstract-upgrade`**: Upgrade packages using auto-detected or specified package manager (non-interactive, no progress bars).
+- **`core.pkg.abstract-search`**: Search for packages using auto-detected or specified package manager (non-interactive, no progress bars).
+- **`core.pkg.abstract-list`**: List installed packages using auto-detected or specified package manager (non-interactive, no progress bars).
+- **`core.pkg.abstract-clean`**: Clean package cache using auto-detected or specified package manager (non-interactive, no progress bars).
+
+**Supported Package Managers**: Apt, Apk, Yum, Dnf, Pacman, Zypper, Eopkg, Winget, Choco, Brew
+
+**Non-Interactive Features**: All abstract package management tasks run completely non-interactively with:
+- No user prompts or confirmations
+- No progress bars or interactive indicators
+- No color output or terminal formatting
+- Automatic confirmations via environment variables and command flags
+- Clean, machine-readable output suitable for CI/CD pipelines
+
 ### Service and Process Management
 - **`core.systemd.disable`**: Disables a systemd service.
 - **`core.systemd.enable`**: Enables a systemd service.
@@ -437,6 +455,186 @@ hostctl bundle
 hostctl bundle ./my-hostctl-project
 ```
 
+#### `pkg`
+
+Manages `hostctl` packages with a comprehensive package management system that maintains a manifest of installed packages and their tasks.
+
+**Usage:**
+
+```bash
+hostctl pkg [subcommand]
+```
+
+**Subcommands:**
+
+- **`create <package-name>`**
+  - Scaffolds a new `hostctl` package in the current directory.
+  - **Usage:** `hostctl pkg create my-new-task`
+  - **Options:**
+    - `--lang <language>`: Specify the language for the package (`typescript` or `javascript`). Defaults to `typescript`.
+
+- **`install <package-source>`**
+  - Installs a `hostctl` package and maintains it in the package manifest.
+  - The `<package-source>` can be a local path, a Git URL, or an npm package name.
+  - **Usage:**
+
+    ```bash
+    # Install from a local directory
+    hostctl pkg install ./my-local-task
+
+    # Install from a Git repository
+    hostctl pkg install https://github.com/user/my-hostctl-task.git
+
+    # Install from npm
+    hostctl pkg install my-npm-hostctl-task
+    ```
+
+- **`list`** (alias: `ls`)
+  - Lists all installed `hostctl` packages with their discovered tasks.
+  - **Usage:** `hostctl pkg list` or `hostctl pkg ls`
+  - **Output Example:**
+
+    ```
+    Found 1 package(s):
+
+    • hostsfile (v1.0.0) - hostsfile package [https_github.com_davidkellis_hostsfile]
+      └─ add_entry
+    ```
+
+- **`remove <package-identifier>`** (alias: `rm`)
+  - Removes an installed `hostctl` package and updates the manifest.
+  - The `<package-identifier>` can be either the package name or the directory name.
+  - **Usage:** `hostctl pkg remove my-package` or `hostctl pkg rm my-package`
+
+## Package Management System
+
+`hostctl` includes a comprehensive package management system that allows you to install, manage, and run tasks from external packages. This system maintains a manifest of installed packages and provides intelligent task discovery and resolution.
+
+### Package Manifest
+
+The package management system maintains a `manifest.json` file in `~/.hostctl/packages/` that tracks all installed packages. This manifest contains:
+
+- **Package Source Mapping**: Each package source (URL, Git repo, local path) is mapped to package information
+- **Package Metadata**: Name, version, description, and installation directory
+- **Task Discovery**: Automatically discovered tasks within each package
+- **Persistent Storage**: Package information persists across sessions
+
+### Task Discovery
+
+When a package is installed, `hostctl` automatically discovers tasks within the package by scanning for:
+
+- **TypeScript/JavaScript Files**: `.ts` and `.js` files in the package root
+- **Subdirectories**: Tasks in `tasks/` and `src/` subdirectories
+- **Default Tasks**: Common default task files like `index.ts`, `main.ts`, etc.
+
+### Task Resolution
+
+The `hostctl run` command now supports multiple resolution strategies:
+
+1. **Built-in Core Tasks**: `hostctl run core.echo hello`
+2. **Installed Package Tasks**:
+   - Default task: `hostctl run https://github.com/user/repo`
+   - Specific task: `hostctl run https://github.com/user/repo:taskname`
+3. **Relative Paths**: `hostctl run ./script.ts`
+
+### Package Resolution Examples
+
+```bash
+# Install a package
+hostctl pkg install https://github.com/davidkellis/hostsfile
+
+# List installed packages and their tasks
+hostctl pkg list
+
+# Run the default task from an installed package
+hostctl run https://github.com/davidkellis/hostsfile names:foo,bar
+
+# Run a specific task from an installed package
+hostctl run https://github.com/davidkellis/hostsfile:add_entry names:foo,bar
+
+# Run core tasks
+hostctl run core.echo hello
+
+# Run relative paths (still works)
+hostctl run example/echo.ts hello
+```
+
+### Abstract Package Management Examples
+
+The abstract package management system automatically detects your platform's package manager and provides a unified interface:
+
+```bash
+# Auto-detect and update package sources
+hostctl run core.pkg.abstractUpdate
+
+# Search for packages (auto-detects package manager)
+hostctl run core.pkg.abstractSearch query:nginx
+
+# Install packages (auto-detects package manager)
+hostctl run core.pkg.abstractInstall package:curl
+
+# Install multiple packages
+hostctl run core.pkg.abstractInstall package:curl,wget,git
+
+# Force use of specific package manager
+hostctl run core.pkg.abstractInstall package:nginx packageManager:apt
+
+# List installed packages
+hostctl run core.pkg.abstractList
+
+# Upgrade all packages
+hostctl run core.pkg.abstractUpgrade
+
+# Clean package cache
+hostctl run core.pkg.abstractClean
+```
+
+**Supported Platforms**: Ubuntu/Debian (apt), Alpine (apk), CentOS/RHEL (yum/dnf), Arch Linux (pacman), openSUSE (zypper), Solus (eopkg), Windows (winget/choco), macOS (brew)
+
+### Package Structure
+
+A `hostctl` package should follow this structure:
+
+```
+my-hostctl-package/
+├── package.json          # Package metadata
+├── index.ts             # Default task (optional)
+├── task1.ts             # Individual task
+├── task2.ts             # Individual task
+├── tasks/               # Task subdirectory (optional)
+│   ├── task3.ts
+│   └── task4.ts
+└── src/                 # Source subdirectory (optional)
+    ├── task5.ts
+    └── task6.ts
+```
+
+### Package Dependencies
+
+Each installed package maintains its own `node_modules` directory, ensuring that:
+
+- Dependencies are isolated per package
+- The `hostctl` runtime is available to each package
+- Packages can have their own version requirements
+
+### Package Sources
+
+The package management system supports multiple source types:
+
+- **Local Paths**: `./my-package` or `/path/to/package`
+- **Git Repositories**: `https://github.com/user/repo.git` or `git@github.com:user/repo.git`
+- **GitHub Shorthand**: `github.com/user/repo`
+- **NPM Packages**: `@scope/package-name` or `package-name`
+
+### Task Execution Context
+
+When running tasks from installed packages:
+
+- Tasks have access to the full `hostctl` runtime API
+- Package dependencies are available in the task's `node_modules`
+- Tasks can use all core tasks and utilities
+- Remote execution works seamlessly with installed package tasks
+
 #### `inventory` (alias: `i`)
 
 Manages and displays information about the host inventory.
@@ -565,7 +763,6 @@ When working with the inventory decryption functionality, it's important to unde
 1. **Hostname Storage**: In the YAML configuration, hostnames are stored as the keys in the `hosts` map, not as properties inside each host object. When a host is serialized to YAML via `Host.toYAML()`, the `hostname` field is not included because it's represented by the map key.
 
 2. **Verbosity and Logging**: The inventory decryption process follows the general verbosity rules of hostctl:
-
    - By default, only ERROR level messages are shown
    - INFO level messages (like "No encrypted secrets found to decrypt. Inventory is already decrypted.") require the `-v` flag to be visible
    - For idempotent operations like decrypting an already-decrypted inventory, you need to use the `-v` flag to see confirmation messages
@@ -705,7 +902,7 @@ The `TaskContext` is the heart of the scripting API. It's passed to your `run` f
 - `ssh(hostSelector, taskDefinition, params?): Promise<TRemoteReturn[] | Error>`: Executes another `hostctl` task on one or more remote hosts.
   - **`hostSelector`**: A string, an array of strings, a `Host` object, or an array of `Host` objects to specify the target hosts from your inventory.
   - **`taskDefinition`**: The task to execute on the remote host(s).
-  - **`params`**: The parameters to pass to the remote task.
+  - `params`: The parameters to pass to the remote task.
 - `file: FileOperations`: An object for performing file operations on the target host.
   - `read(path)`: Reads the content of a file.
   - `write(path, content)`: Writes content to a file.
@@ -835,191 +1032,6 @@ async function run(context: TaskContext<InstallNginxParams>): Promise<void> {
 export default task(run, 'Install Nginx on a Debian-based system');
 ```
 
-## Roadmap: Expanding Capabilities
-
-To further enhance `hostctl` and position it as a comprehensive Ansible alternative, the following task primitives are planned for future development. These aim to provide built-in support for common system administration and automation workflows:
-
-1. **File Management:**
-   - `file.replace` (regex substitution)
-   - `file.fetch` (remote → local)
-
-2. **Package Management:**
-   - `pkg.purge`
-   - `pkg.autoremove`
-
-3. **User & Group Management:**
-   - `user.password` (change password)
-
-4. **System Operations:**
-   - `system.timezone`
-   - `system.mount`
-
-5. **Archive Management:**
-   - `archive.compress`
-   - `archive.extract`
-   - `archive.unarchive` (auto-detect format)
-
-6. **Networking & Firewall:**
-   - `firewall.status`
-   - `firewall.add_rule`
-   - `firewall.remove_rule`
-   - `network.interfaces`
-   - `network.get_facts`
-
-7. **Templating & Control Flow:**
-   - Enhanced templating (diff mode, variables, idempotent)
-   - `retry`
-   - `until` (wait for condition)
-   - `include_tasks` (run another script)
-   - `wait_for`
-   - `assert`
-   - Improved looping & retry helpers
-
-This roadmap will evolve, and contributions are welcome!
-
-3.  **Define ResultTypes** (optional but recommended): Specify the structure of the result your task will return (e.g., `MyTaskResult`).
-4.  **Implement the `run` function**: This is the core logic of your task.
-5.  **Export the task**: Use `export default task(runFunction, {description: "Optional description"});`.
-
-Here's a template:
-
-```typescript
-// my-task.ts
-import { task, type TaskContext, type IInvocation, type LogLevel } from 'hostctl'; // Adjust path if developing hostctl itself
-// For users of the hostctl package, it would typically be:
-// import { task, type TaskContext, type IInvocation, type LogLevel, Verbosity } from "hostctl";
-
-// (If developing hostctl, Verbosity might be imported from a relative path like "../app")
-import { Verbosity } from '../app'; // Example for internal development path
-
-// 1. Define the structure of parameters your task expects
-export type MyTaskParams = {
-  targetSystem: string;
-  enableFeature?: boolean;
-};
-
-// 2. Define the structure of the result your task will return
-export interface MyTaskResult {
-  success: boolean;
-  details?: string;
-}
-
-// 3. Implement the core task logic in a 'run' function
-async function run(
-  context: TaskContext<MyTaskParams>, // First argument is the TaskContext
-): Promise<MyTaskResult> {
-  // Access parameters passed to the script via context.params
-  const { targetSystem, enableFeature = false } = context.params;
-
-  // Use 'this' for invocation-specific operations
-  this.log(Verbosity.INFO as LogLevel, `Configuring ${targetSystem}...`);
-
-  try {
-    // Use context for runtime utilities like 'exec' or 'file' operations
-    const { stdout } = await context.exec(
-      `configure-system --target ${targetSystem} ${enableFeature ? '--enable' : ''}`,
-    );
-    this.log(Verbosity.DEBUG as LogLevel, `Configuration output: ${stdout}`);
-
-    // Example of using this.sh (often a shorthand for context.exec for simple commands)
-    // const { stdout: diskSpace } = await this.sh("df -h /");
-    // this.log(Verbosity.INFO as LogLevel, `Disk space: ${diskSpace}`);
-
-    return { success: true, details: 'System configured successfully.' };
-  } catch (error: any) {
-    this.log(Verbosity.ERROR as LogLevel, `Error during configuration: ${error.message}`);
-    return { success: false, details: error.message };
-  }
-}
-
-// 4. Export the task, optionally with a dynamic description
-// The '{{ args?.join(" ") }}' uses Handlebars templating.
-// 'args' needs to be available in the template context for this to work.
-export default task(run, 'Configures {{ targetSystem }} with feature: {{ enableFeature }}');
-```
-
-### The `run` Function Signature
-
-The `run` function is where your task's main logic resides. It must conform to the following signature:
-
-`async function run(context: TaskContext<TParams>): Promise<TResult>`
-
-- **`context: TaskContext<TParams>`**: This is the **first and only argument** passed to your `run` function. It's a crucial object providing access to:
-
-  - `context.params: TParams`: An object containing the parameters passed to your script from the command line (e.g., `args:key,value` or via `--params <json_string>`). `TParams` is a type interface you define to structure these expected parameters.
-  - `context.log`, `context.exec`: The same logging and command execution functions available via `this`.
-  - `context.file: FileSystemOperations`: An object with methods for file system operations (`read`, `write`, `exists`, `mkdir`, `rm`) that work on the target host (local or remote).
-  - `context.host?: Host`: If the task is executing on a remote host, this object provides details about that host.
-  - `context.cwd: string`: The current working directory of the task.
-
-- **`Promise<TResult>`**: Your `run` function must be `async` and should return a `Promise`. The value this promise resolves to is considered the result of your task. `TResult` is a type interface you define for structuring this result.
-
-### Defining Parameters (`TParams`) and Results (`TResult`)
-
-It's highly recommended to define TypeScript interfaces or types for your task's parameters (`TParams`) and its return value (`TResult`). This greatly improves code clarity, maintainability, and enables type checking.
-
-```typescript
-// For parameters
-export type MyScriptParams = {
-  inputFile: string;
-  outputDir?: string;
-  forceOverwrite: boolean;
-};
-
-// For results
-export interface MyScriptResult {
-  filesProcessed: number;
-  errorsEncountered: number;
-  outputLocation?: string;
-}
-```
-
-### Example: The `echo` Task (from `src/core/echo.ts`)
-
-This task demonstrates the core concepts:
-
-```typescript
-import { task, type TaskContext, type IInvocation, type LogLevel } from 'hostctl';
-import { Verbosity } from 'hostctl/app'; // Path for user-facing package
-
-// 1. Define Parameters
-export type EchoParams = {
-  args: string[]; // Expects an array of strings named 'args'
-};
-
-// 2. Define Result
-export interface EchoResult {
-  stdout: string; // The echoed output
-}
-
-// 3. Implement the 'run' function
-async function run(context: TaskContext<EchoParams>): Promise<EchoResult> {
-  const { params, exec, log } = context;
-  // Provide a default for args in case it's not passed from the CLI
-  const { args = [] } = params;
-
-  log(Verbosity.DEBUG as LogLevel, `Echoing arguments: ${args.join(' ')}`);
-
-  // Use context.exec with an array of command and arguments
-  // This is generally safer than manually joining strings for shell execution.
-  const commandArray = ['echo', ...args];
-  const { stdout } = await exec(commandArray);
-
-  return {
-    stdout, // Return the standard output of the echo command
-  };
-}
-
-// 4. Export the task with a dynamic description
-// The '{{ args?.join(" ") }}' uses Handlebars templating.
-// 'args' needs to be available in the template context for this to work.
-export default task(run, "Echoes the provided arguments: {{ args?.join(' ') }}");
-```
-
-To run this echo task:
-`hostctl run path/to/echo.ts args:hello,world`
-Inside the `run` function, `context.params` would be `{ args: ["hello", "world"] }`.
-
 ## TODO
 
 - [ ] Further refine the script execution interface.