@fredlackey/devutils 0.0.19 → 0.1.0

package/README.md

@@ -1,64 +1,255 @@
 # DevUtils CLI
 
-> **Rebuild in Progress (v0.0.19)** — This project is being restructured from the ground up. The previous version (v0.0.18) tried to do too much too fast. The new approach is config-driven, user-driven, and machine-aware. If you're looking for the old code, it's been preserved in the `_rebuild/` directory.
+> **Work in Progress** -- This project is under active development and has not been fully tested in real-world environments. Commands may change, break, or behave unexpectedly. Use at your own risk. The stable, production-ready release will ship as **v1.0**. Until then, versions in the `0.x` range should be considered pre-release.
 
-## What Happened
+A config-driven CLI toolkit for bootstrapping and managing development environments across any machine.
 
-The original DevUtils CLI was built as a rigid, opinionated toolkit. It assumed a specific workflow, shipped dozens of global scripts, and tried to be everything to everyone out of the gate. After real-world usage and feedback, it became clear that this approach wouldn't hold up. It was too prescriptive, too fragile across environments, and too hard for users to adapt to their own needs.
+DevUtils replaces scattered dotfiles, setup scripts, and manual configuration with a single `dev` command. You tell it what you need, and it handles the rest -- whether you're setting up a new laptop, syncing git identities across machines, or managing tool installations.
 
-## What's Changing
+## Installation
 
-The new version takes a fundamentally different approach:
+```bash
+npm install -g @fredlackey/devutils
+```
 
-### Config-Based, Not Opinion-Based
+Requires Node.js 18 or later.
 
-Instead of shipping a fixed set of behaviors, DevUtils will be driven by user configuration. You tell it what matters to you, and it adapts.
+## Quick Start
 
-### User Onboarding
+Once installed, the `dev` command is available globally:
 
-The first run walks you through a lightweight onboarding process. It learns who you are, what tools you care about, and how your machine is set up. No assumptions.
+```bash
+dev help               # Show all available services and commands
+dev version            # Print the current version
+dev config init        # Walk through first-time setup
+dev status             # Check the health of your environment
+dev ignore node        # Add Node.js patterns to .gitignore
+dev tools list         # See what tools are installed
+dev identity list      # List your configured git identities
+dev util list          # Browse available utility functions
+```
 
-### Machine-Aware Profiles
+## Command Reference
 
-Configuration is scoped to the machine you're on. A laptop, a server, and a VM can each have their own rules without conflicting.
+DevUtils organizes commands by service. The general pattern is:
 
-### Rule-Based Behavior
+```
+dev <service> <method> [arguments] [flags]
+```
 
-Users define rules for folders, tools, and workflows. DevUtils enforces those rules rather than imposing its own. For example:
-- "This folder always uses this git identity"
-- "This machine should have these tools installed"
-- "Run these checks when I open a new terminal"
+### Services
 
-### Temporary Workspace
+#### config -- User configuration and onboarding
 
-DevUtils sets up a managed workspace inside the user's home directory for staging, caching, and intermediate state. Nothing touches system-level paths unless you ask for it.
+| Method   | Description                                  |
+|----------|----------------------------------------------|
+| `init`   | Run first-time setup and create config files |
+| `show`   | Display the current configuration            |
+| `get`    | Read a specific config value                 |
+| `set`    | Update a specific config value               |
+| `reset`  | Reset configuration to defaults              |
+| `export` | Export config to a file                       |
+| `import` | Import config from a file                    |
 
-## Installation
+#### machine -- Machine profiles and detection
+
+| Method   | Description                         |
+|----------|-------------------------------------|
+| `detect` | Auto-detect the current machine     |
+| `show`   | Show the active machine profile     |
+| `set`    | Set a machine profile value         |
+| `list`   | List all known machine profiles     |
+
+#### identity -- Git identities, SSH keys, GPG signing
+
+| Method   | Description                                 |
+|----------|---------------------------------------------|
+| `add`    | Register a new git identity                 |
+| `remove` | Remove a registered identity                |
+| `list`   | List all identities                         |
+| `show`   | Show details for a specific identity        |
+| `link`   | Link an identity to a folder or repo        |
+| `unlink` | Remove a folder/repo identity link          |
+| `sync`   | Sync identity configs to git and SSH        |
+
+#### tools -- Tool installation and management
+
+| Method    | Description                              |
+|-----------|------------------------------------------|
+| `install` | Install a tool using the platform's package manager |
+| `check`   | Check if a tool is installed             |
+| `list`    | List available or installed tools        |
+| `search`  | Search for tools by name                 |
+
+#### ignore -- .gitignore pattern management
+
+| Method   | Description                                  |
+|----------|----------------------------------------------|
+| `add`    | Add technology patterns to .gitignore        |
+| `remove` | Remove a technology's patterns               |
+| `list`   | List available technologies                  |
+| `show`   | Show patterns for a specific technology      |
+
+#### util -- Utility functions
+
+| Method   | Description                           |
+|----------|---------------------------------------|
+| `add`    | Register a custom utility             |
+| `remove` | Unregister a utility                  |
+| `list`   | List all available utilities          |
+| `show`   | Show details for a specific utility   |
+| `run`    | Execute a utility                     |
+
+#### alias -- Shorthand bin entries
+
+| Method   | Description                              |
+|----------|------------------------------------------|
+| `add`    | Create a new alias                       |
+| `remove` | Delete an alias                          |
+| `list`   | List all registered aliases              |
+| `sync`   | Regenerate alias wrapper scripts         |
+
+#### auth -- OAuth and credential management
+
+| Method    | Description                          |
+|-----------|--------------------------------------|
+| `login`   | Authenticate with a service          |
+| `logout`  | Revoke credentials for a service     |
+| `list`    | List authenticated services          |
+| `status`  | Show current auth status             |
+| `refresh` | Refresh an expired token             |
+
+#### api -- API plugin system
+
+| Method    | Description                         |
+|-----------|-------------------------------------|
+| `list`    | List installed API plugins          |
+| `enable`  | Enable a plugin                     |
+| `disable` | Disable a plugin                    |
+| `update`  | Update a plugin to latest version   |
+
+#### ai -- AI coding assistant launcher
+
+| Method     | Description                             |
+|------------|-----------------------------------------|
+| `launch`   | Start an AI coding session              |
+| `resume`   | Resume a previous session               |
+| `list`     | List configured AI tools                |
+| `sessions` | List past sessions                      |
+| `show`     | Show details for a session or AI tool   |
+| `set`      | Update AI tool settings                 |
+
+#### search -- Markdown search
+
+| Method        | Description                              |
+|---------------|------------------------------------------|
+| `query`       | Run a general search                     |
+| `keyword`     | Search by keyword                        |
+| `semantic`    | Search by meaning (requires AI plugin)   |
+| `get`         | Retrieve a specific search result        |
+| `collections` | List indexed collections                 |
+| `index`       | Build or rebuild the search index        |
+| `status`      | Show indexing status                     |
+
+### Top-Level Commands
+
+These commands don't belong to a service and are called directly:
+
+| Command   | Description                        |
+|-----------|------------------------------------|
+| `status`  | Overall health check               |
+| `version` | Show current version               |
+| `help`    | Show the help message              |
+| `schema`  | Introspect available commands      |
+| `update`  | Update DevUtils to latest version  |
+
+### Global Flags
+
+These flags work with any command:
+
+| Flag                              | Description                        |
+|-----------------------------------|------------------------------------|
+| `--format <json\|table\|yaml\|csv>` | Set the output format             |
+| `--dry-run`                       | Show what would happen without doing it |
+| `--verbose`                       | Increase output detail             |
+| `--quiet`                         | Suppress non-essential output      |
+| `--json <data>`                   | Pass structured input as JSON      |
+| `--help`, `-h`                    | Show help                          |
+| `--version`, `-v`                 | Show version                       |
+
+## API Plugin System
+
+DevUtils doesn't bundle API integrations directly. Instead, API wrappers are installed as separate plugin packages and managed through the `dev api` service.
+
+Plugins live in `~/.devutils/plugins/` and are registered in `~/.devutils/plugins.json`. Each plugin is a standard npm package that exports a defined interface. This keeps the core CLI small and lets you add only the integrations you actually use.
 
 ```bash
-npm install -g @fredlackey/devutils
+dev api list               # See what's installed
+dev api enable <plugin>    # Enable a plugin
+dev api disable <plugin>   # Disable a plugin
+dev api update <plugin>    # Update to latest version
 ```
 
-> The CLI is published but actively being rebuilt. Expect breaking changes until v0.1.0.
+## Configuration
+
+All user data lives in `~/.devutils/`, created during `dev config init`:
+
+- `config.json` -- User preferences, profile name, backup location
+- `aliases.json` -- Registered alias mappings
+- `ai.json` -- AI tool configurations
+- `plugins.json` -- Installed API plugin registry
+- `machines/` -- Machine profiles
+- `auth/` -- OAuth tokens and API credentials
+- `plugins/` -- Installed API plugin packages
+- `utils/` -- User-added custom utilities
+- `bin/` -- Generated alias wrapper scripts (added to PATH)
+- `cache/` -- Temporary data
+
+## Supported Platforms
+
+| Platform          | Package Manager     |
+|-------------------|---------------------|
+| macOS             | Homebrew            |
+| Ubuntu            | APT, Snap           |
+| Raspberry Pi OS   | APT, Snap           |
+| Amazon Linux      | DNF, YUM            |
+| Windows           | Chocolatey, winget  |
+| Git Bash          | Manual / Portable   |
+
+## Current Status
 
-## Previous Version
+DevUtils is in **pre-release** (`0.1.x`). The core framework, command routing, and service structure are in place. Basic smoke tests pass on Ubuntu 24.04 in Docker, but deeper integration testing -- real Git operations, SSH key workflows, GitHub auth, tool installation, and interactive prompts -- has not been completed yet.
 
-The v0.0.18 codebase (commands, scripts, installers, utilities) has been moved to `_rebuild/` and is preserved for reference. Useful patterns and utilities will be pulled forward into the new architecture as needed.
+What's working:
+- Command routing and service discovery across all 11 services
+- Config init, show, get, set, reset, file-based export/import
+- Machine detection and profile management
+- Gitignore pattern management (add, remove, list, show)
+- Tool check, list, search, and dry-run install
+- Identity CRUD (add, list, show, remove)
+- Alias management and wrapper generation
+- AI tool configuration
+- Schema introspection
+- Platform detection (macOS, Ubuntu, Raspberry Pi OS, Amazon Linux, Windows, Git Bash)
 
-## Roadmap
+What still needs real-world testing:
+- SSH key generation and GitHub integration
+- Git identity sync to actual repositories
+- Config backup/restore via remote Git repo
+- OAuth login flows
+- Tool installation on each supported platform
+- API plugin installation and lifecycle
+- AI session launch and resume
+- QMD search indexing and queries
 
-- [ ] User onboarding flow
-- [ ] Config file schema (`~/.devutils/config.json`)
-- [ ] Machine profile detection and storage
-- [ ] Rule engine for folder and tool behaviors
-- [ ] Migrate useful scripts and installers from `_rebuild/`
+Patch versions (`0.1.1`, `0.1.2`, etc.) will ship as issues are found and fixed during hands-on use. Minor version bumps (`0.2.0`) are reserved for breaking changes. The first stable release will be **v1.0.0**.
 
 ## Contact
 
 **Fred Lackey**
-- Email: [fred.lackey@gmail.com](mailto:fred.lackey@gmail.com)  
-- Website: [fredlackey.com](https://fredlackey.com)  
-- GitHub: [@FredLackey](https://github.com/FredLackey)  
+- Email: [fred.lackey@gmail.com](mailto:fred.lackey@gmail.com)
+- Website: [fredlackey.com](https://fredlackey.com)
+- GitHub: [@FredLackey](https://github.com/FredLackey)
 
 ## License
 

package/package.json

@@ -1,17 +1,19 @@
 {
   "name": "@fredlackey/devutils",
-  "version": "0.0.19",
+  "version": "0.1.0",
   "description": "A config-driven CLI toolkit for bootstrapping and managing development environments across any machine.",
   "main": "src/index.js",
   "bin": {
-    "dev": "./bin/dev.js"
+    "dev": "./src/cli.js"
   },
   "files": [
-    "bin/",
-    "src/"
+    "src/",
+    "package.json",
+    "README.md",
+    "LICENSE"
   ],
   "scripts": {
-    "test": "echo \"Error: no test specified\" && exit 1"
+    "test": "node test/index.js"
   },
   "dependencies": {
     "commander": "^12.0.0"

package/src/api/loader.js

@@ -0,0 +1,229 @@
+'use strict';
+
+const fs = require('fs');
+const path = require('path');
+const os = require('os');
+
+const DEVUTILS_DIR = path.join(os.homedir(), '.devutils');
+const PLUGINS_FILE = path.join(DEVUTILS_DIR, 'plugins.json');
+const PLUGINS_DIR = path.join(DEVUTILS_DIR, 'plugins');
+
+/**
+ * Reads and parses ~/.devutils/plugins.json.
+ * Returns an empty object if the file does not exist or is unreadable.
+ * Logs a warning to stderr if the file exists but contains invalid JSON.
+ *
+ * @returns {object} A map of plugin names to their entries, or {}.
+ */
+function readPluginsJson() {
+  try {
+    const raw = fs.readFileSync(PLUGINS_FILE, 'utf8');
+    return JSON.parse(raw);
+  } catch (err) {
+    // If the file exists but JSON is invalid, warn the user
+    if (err instanceof SyntaxError && fs.existsSync(PLUGINS_FILE)) {
+      process.stderr.write(`Warning: ${PLUGINS_FILE} contains invalid JSON. Treating as empty.\n`);
+    }
+    return {};
+  }
+}
+
+/**
+ * Validates that a plugin module exports the required contract fields.
+ * Returns null if valid, or a structured error object if something is missing.
+ *
+ * @param {object} pluginModule - The required plugin module.
+ * @param {string} pluginName - The plugin name (for error messages).
+ * @returns {object|null} Null if valid, or { error, code, message } if invalid.
+ */
+function validateContract(pluginModule, pluginName) {
+  const required = ['name', 'description', 'version', 'auth', 'resources'];
+  const missing = required.filter(field => !pluginModule[field]);
+
+  if (missing.length > 0) {
+    return {
+      error: true,
+      code: 'INVALID_CONTRACT',
+      message: `Plugin "${pluginName}" is missing required fields: ${missing.join(', ')}.\nThe plugin may be outdated or incorrectly built.`
+    };
+  }
+
+  if (typeof pluginModule.resources !== 'object') {
+    return {
+      error: true,
+      code: 'INVALID_CONTRACT',
+      message: `Plugin "${pluginName}" has an invalid resources export. Expected an object.`
+    };
+  }
+
+  return null;
+}
+
+/**
+ * Loads a plugin by name from the installed plugins directory.
+ * Reads plugins.json, requires the package, and validates the contract.
+ *
+ * @param {string} pluginName - The short plugin name (e.g., 'gmail').
+ * @returns {object} { error: false, plugin } on success, or { error: true, code, message } on failure.
+ */
+function loadPlugin(pluginName) {
+  const plugins = readPluginsJson();
+  const entry = plugins[pluginName];
+
+  if (!entry) {
+    return {
+      error: true,
+      code: 'NOT_INSTALLED',
+      message: `API plugin "${pluginName}" is not installed.\nRun "dev api enable ${pluginName}" to install it.`
+    };
+  }
+
+  const packagePath = path.join(PLUGINS_DIR, 'node_modules', entry.package);
+
+  let pluginModule;
+  try {
+    pluginModule = require(packagePath);
+  } catch (err) {
+    return {
+      error: true,
+      code: 'LOAD_FAILED',
+      message: `Failed to load plugin "${pluginName}" from ${packagePath}.\nThe package may be corrupted. Try "dev api disable ${pluginName}" then "dev api enable ${pluginName}".`
+    };
+  }
+
+  // Validate the plugin contract
+  const validation = validateContract(pluginModule, pluginName);
+  if (validation) {
+    return validation;
+  }
+
+  return { error: false, plugin: pluginModule };
+}
+
+/**
+ * Resolves a specific command from a plugin's resource tree.
+ * Given a plugin name, resource name, and command name, loads the plugin
+ * and walks the resource/command tree to find the command module.
+ *
+ * @param {string} pluginName - The plugin name (e.g., 'gmail').
+ * @param {string} resourceName - The resource name (e.g., 'messages').
+ * @param {string} commandName - The command name (e.g., 'list').
+ * @returns {object} { error: false, command, plugin } on success, or { error: true, code, message } on failure.
+ */
+function resolveCommand(pluginName, resourceName, commandName) {
+  const result = loadPlugin(pluginName);
+  if (result.error) return result;
+
+  const plugin = result.plugin;
+  const resource = plugin.resources[resourceName];
+
+  if (!resource) {
+    const available = Object.keys(plugin.resources).join(', ');
+    return {
+      error: true,
+      code: 'UNKNOWN_RESOURCE',
+      message: `Plugin "${pluginName}" has no resource "${resourceName}".\nAvailable resources: ${available}`
+    };
+  }
+
+  const commandLoader = resource.commands[commandName];
+  if (!commandLoader) {
+    const available = Object.keys(resource.commands).join(', ');
+    return {
+      error: true,
+      code: 'UNKNOWN_COMMAND',
+      message: `Resource "${resourceName}" in plugin "${pluginName}" has no command "${commandName}".\nAvailable commands: ${available}`
+    };
+  }
+
+  let commandModule;
+  try {
+    commandModule = typeof commandLoader === 'function' ? commandLoader() : commandLoader;
+  } catch (err) {
+    return {
+      error: true,
+      code: 'COMMAND_LOAD_FAILED',
+      message: `Failed to load command "${commandName}" from plugin "${pluginName}": ${err.message}`
+    };
+  }
+
+  return {
+    error: false,
+    command: commandModule,
+    plugin: plugin
+  };
+}
+
+/**
+ * Builds the context object that plugin commands receive.
+ * Reads the auth credential for the plugin's declared auth service
+ * and unwraps the credential envelope so plugins get flat access.
+ *
+ * @param {object} plugin - The plugin's contract object (must have an auth field).
+ * @param {object} coreContext - The core CLI context (output, errors, config, shell, platform).
+ * @returns {object} The enriched plugin context with auth, output, errors, config, shell, platform.
+ */
+function buildPluginContext(plugin, coreContext) {
+  const authService = plugin.auth;
+
+  // Read the auth credential for this plugin's declared service
+  const authFilePath = path.join(DEVUTILS_DIR, 'auth', `${authService}.json`);
+  let authCredential = null;
+  try {
+    const raw = fs.readFileSync(authFilePath, 'utf8');
+    const credentialFile = JSON.parse(raw);
+
+    // Unwrap the credential envelope. The auth system stores files as:
+    //   { service, type, credentials: { ... } }
+    // Plugins expect flat access (e.g., context.auth.accessKeyId), so we
+    // pass credentialFile.credentials directly instead of the full envelope.
+    authCredential = credentialFile.credentials || credentialFile;
+  } catch (err) {
+    // Auth not available - plugin commands will get null
+  }
+
+  return {
+    auth: authCredential,
+    output: coreContext.output,
+    errors: coreContext.errors,
+    config: coreContext.config,
+    shell: coreContext.shell,
+    platform: coreContext.platform
+  };
+}
+
+/**
+ * Returns all installed plugins from plugins.json.
+ * Returns an empty object if no plugins are installed.
+ *
+ * @returns {object} A map of plugin names to their entries.
+ */
+function getInstalledPlugins() {
+  return readPluginsJson();
+}
+
+/**
+ * Returns the list of available plugins from the bundled registry.
+ * Returns an empty array if the registry file is missing or unreadable.
+ *
+ * @returns {Array<object>} An array of registry plugin entries.
+ */
+function getRegistryPlugins() {
+  try {
+    return require('./registry.json');
+  } catch (err) {
+    return [];
+  }
+}
+
+module.exports = {
+  resolveCommand,
+  loadPlugin,
+  buildPluginContext,
+  validateContract,
+  getInstalledPlugins,
+  getRegistryPlugins,
+  readPluginsJson,
+  PLUGINS_FILE,
+  PLUGINS_DIR
+};

package/src/api/registry.json

@@ -0,0 +1,62 @@
+[
+  {
+    "name": "gmail",
+    "package": "@fredlackey/devutils-api-gmail",
+    "description": "Google Gmail (messages, labels, drafts, threads)",
+    "auth": "google"
+  },
+  {
+    "name": "drive",
+    "package": "@fredlackey/devutils-api-drive",
+    "description": "Google Drive (files, folders, permissions)",
+    "auth": "google"
+  },
+  {
+    "name": "sheets",
+    "package": "@fredlackey/devutils-api-sheets",
+    "description": "Google Sheets (spreadsheets, values, sheets)",
+    "auth": "google"
+  },
+  {
+    "name": "docs",
+    "package": "@fredlackey/devutils-api-docs",
+    "description": "Google Docs (documents)",
+    "auth": "google"
+  },
+  {
+    "name": "aws",
+    "package": "@fredlackey/devutils-api-aws",
+    "description": "Amazon Web Services (compute, storage, functions, groups)",
+    "auth": "aws"
+  },
+  {
+    "name": "cloudflare",
+    "package": "@fredlackey/devutils-api-cloudflare",
+    "description": "Cloudflare (zones, DNS records, API tokens)",
+    "auth": "cloudflare"
+  },
+  {
+    "name": "dokploy",
+    "package": "@fredlackey/devutils-api-dokploy",
+    "description": "Dokploy (applications, projects, domains, servers)",
+    "auth": "dokploy"
+  },
+  {
+    "name": "namecheap",
+    "package": "@fredlackey/devutils-api-namecheap",
+    "description": "Namecheap (domains, DNS, SSL certificates)",
+    "auth": "namecheap"
+  },
+  {
+    "name": "flowroute",
+    "package": "@fredlackey/devutils-api-flowroute",
+    "description": "Flowroute (SMS, MMS, phone numbers)",
+    "auth": "flowroute"
+  },
+  {
+    "name": "mailu",
+    "package": "@fredlackey/devutils-api-mailu",
+    "description": "Mailu (email users, aliases, domains)",
+    "auth": "mailu"
+  }
+]