nduka 1.0.0

package/docs/tool-configuration-guide.md

@@ -0,0 +1,257 @@
+# Tool Configuration Guide
+
+Status: reference guide for configured tools. Keep this document aligned with the live JSON schemas and tests.
+
+## Core rule
+
+Tools must be controlled by configuration, not hardcoded route checks. TypeScript may validate, load, filter, and execute configured tools, but it must not contain business-specific routing such as matching one service name and calling one fixed command.
+
+## Top-level shape
+
+A tool config may contain providers, tools, routing metadata, templates, and verification settings. Different tool types need different nested fields.
+
+```json
+{
+  "providers": {},
+  "tools": {}
+}
+```
+
+## Common tool fields
+
+Every tool should document these fields when applicable:
+
+- `name`: stable internal tool name.
+- `type`: executor type, such as mock, http, mcp, or workflow.
+- `description`: compact purpose statement.
+- `enabled`: whether the tool may be used.
+- `riskLevel`: low, medium, or high.
+- `permission`: customer, owner, or admin permission level.
+- `timeoutMs`: per-call timeout.
+- `maxRetries`: retry limit.
+- `tenantScoped`: whether tenant isolation applies.
+- `requiresApproval`: whether execution needs approval.
+- `ownerOnly`: whether customers are blocked.
+- `access`: chat and actor access rules.
+- `naturalLanguage`: customer-facing routing metadata.
+- `inputSchema`: accepted arguments.
+- `outputSchema`: required response shape.
+- `mockResponse`: deterministic local test response.
+- `mcp`: MCP executor settings.
+- `http`: HTTP executor settings.
+- `verification`: identity or approval verification rules.
+- `routing`: deterministic or catalog routing hints.
+- `templates`: user-facing response templates.
+- `asyncEligible`: whether a durable job may be created.
+- `async`: async-specific timeout, retry, and wording settings.
+
+## Natural-language routing
+
+Natural-language routing must be opt-in.
+
+```json
+{
+  "naturalLanguage": {
+    "enabled": true,
+    "customerFacingName": "account balance lookup",
+    "useWhen": "Use this when a linked customer asks for a verified account balance.",
+    "triggerPhrases": ["account balance", "amount due"],
+    "examples": ["Can you check my balance?"],
+    "tags": ["billing"],
+    "approvalPrompt": "I can check that after you approve this lookup.",
+    "clarificationPrompt": "Which account should I check?"
+  }
+}
+```
+
+The runtime should use this metadata to select candidates. It should not contain tool-specific phrases in TypeScript.
+
+## Input schema fields
+
+Supported field options:
+
+- `type`
+- `required`
+- `presence`
+- `source`
+- `label`
+- `prompt`
+- `secret`
+- `askIfMissing`
+- `value`
+- `envName`
+- `systemKey`
+- `dynamicVariableName`
+- `validationRegex`
+- `validationMessage`
+- `minLength`
+- `maxLength`
+- `enum`
+- `maxAttempts`
+- `requiredWhen`
+
+Example:
+
+```json
+{
+  "inputSchema": {
+    "customer_id": {
+      "type": "string",
+      "required": true,
+      "source": "profile_or_customer",
+      "askIfMissing": true,
+      "prompt": "Please send your customer ID.",
+      "validationRegex": "^[A-Za-z0-9_-]{3,64}$",
+      "validationMessage": "That customer ID format does not look right.",
+      "maxAttempts": 3
+    }
+  }
+}
+```
+
+## Source types
+
+- `constant`: static configured value.
+- `env_secret`: environment variable secret.
+- `system`: runtime-provided system value.
+- `profile`: trusted identity profile only.
+- `owner_provided`: owner-supplied value.
+- `customer`: customer-provided value.
+- `customer_secret`: customer-provided secret value.
+- `profile_or_customer`: profile first, customer only when allowed.
+- `dynamic_variable`: runtime variable resolved by name.
+- `tool_response`: value from a prior tool step.
+
+Important: `profile` values must not be replaced by model-proposed or customer-provided values.
+
+## Verification modes
+
+- `none`: no extra verification.
+- `chat_link`: require linked chat profile.
+- `owner_approval`: owner must approve.
+- `collect_and_verify`: collect fields and verify with configured verifier.
+- `otp`: one-time code verification.
+
+## Access modes
+
+Use configuration to decide:
+
+- customer-facing or owner-only
+- group allowed or group blocked
+- linked profile required or optional
+- approval required or not
+- tenant-scoped or global
+
+## Tool type examples
+
+### Mock tool
+
+Use mock tools for tests and local demos. Mock outputs must still satisfy output schema.
+
+### Secure private lookup
+
+Use identity, tenant scope, approval, and strict input sources. Do not allow the model to invent private values.
+
+### Verification tool
+
+Use when a collected value must be checked before the main tool runs.
+
+### Public search tool
+
+Use only when enabled in config. Customers must not invoke raw slash search commands.
+
+### News tool
+
+Use only through configured tool metadata. If disabled or missing, normal chat must not claim current news.
+
+### URL fetch tool
+
+Require allowlists and URL validation. Do not allow arbitrary customer-supplied URLs unless explicitly designed and tested.
+
+### MCP tool
+
+Treat unstructured text or one-letter outputs as unsafe unless the output schema explicitly permits them.
+
+### Chained workflow tool
+
+Each step must have validated inputs and structured success or error output.
+
+### Async-eligible tool
+
+Set `asyncEligible: true` only when the tool can safely run through a durable job store and worker.
+
+## Error behavior
+
+The runtime must distinguish these failures:
+
+- schema validation error
+- missing required field
+- output validation error
+- unstructured MCP output
+- timeout
+- upstream HTTP failure
+- approval denied
+- verification failed
+
+WhatsApp replies should be safe and brief. Logs and audit should preserve the technical reason.
+
+## Valid minimal example
+
+```json
+{
+  "tools": {
+    "balance_lookup": {
+      "name": "balance_lookup",
+      "type": "mock",
+      "enabled": true,
+      "permission": "customer",
+      "riskLevel": "medium",
+      "tenantScoped": true,
+      "requiresApproval": true,
+      "naturalLanguage": {
+        "enabled": true,
+        "customerFacingName": "balance lookup",
+        "useWhen": "Use this when a linked customer asks for a verified balance.",
+        "triggerPhrases": ["balance", "amount due"],
+        "examples": ["How much do I owe?"]
+      },
+      "inputSchema": {
+        "customer_id": {
+          "type": "string",
+          "required": true,
+          "source": "profile",
+          "secret": false
+        }
+      },
+      "outputSchema": {
+        "amount": { "type": "number", "required": true },
+        "currency": { "type": "string", "required": true }
+      },
+      "mockResponse": {
+        "amount": 100,
+        "currency": "AED"
+      }
+    }
+  }
+}
+```
+
+## Invalid examples
+
+Invalid: enabled customer-facing tool with no access rules.
+
+Invalid: private lookup using `source: customer` when the value must come from a trusted profile.
+
+Invalid: current-news answer through normal chat when no configured news tool exists.
+
+Invalid: async acknowledgement when no durable job was persisted.
+
+## Migration checklist for adding a tool
+
+1. Add tool config.
+2. Add natural-language metadata only when the tool should route from customer text.
+3. Add input schema and output schema.
+4. Add access, tenant, approval, and verification rules.
+5. Add tests for routing, access denial, validation, and failure.
+6. Add Docker/local config notes only if setup changes.
+7. Run build, focused tests, full tests, CLI checks, Docker checks, and live WhatsApp route tests before release.

package/docs/tool-plugin-and-packaging-roadmap.md

@@ -0,0 +1,278 @@
+# Tool Plugin and Packaging Roadmap
+
+Date: 2026-06-28
+Branch: feature/support-inbox-profiles-labels
+
+## Current conclusion
+
+The bot is not yet fully OpenClaw-style for tools. It is OpenClaw-inspired for chatbot config, group mention behavior, provider settings, web/news/fetch diagnostics, and model routing. However, a user cannot currently add a brand-new arbitrary tool only by editing `config/chatbot.config.json`.
+
+Current config can safely configure known behavior:
+
+- WhatsApp DM policy.
+- WhatsApp group policy.
+- Group activation, defaulting to mention-only.
+- Group mention patterns.
+- Known web/news/fetch provider settings.
+- Known media model settings.
+- Existing model routing through environment variables and task routes.
+
+Current code still hardcodes the concrete tool implementations in `src/tools/registry.ts` and `src/tools/search-orchestrator.ts`. The built-in command tools are web search, news search, and URL fetch. This means config can allow, disable, tune, and route existing tools, but it cannot create an entirely new runtime capability without code.
+
+## OpenClaw comparison
+
+OpenClaw separates these layers:
+
+1. Tool policy: profile, allow, deny, sender restrictions, provider restrictions.
+2. Tool configuration: web search, fetch, media, execution, loop detection.
+3. New runtime capability: plugins or MCP servers.
+4. Skills: instruction packs that teach workflows but do not create new runtime actions by themselves.
+
+The same split should be used here, scaled down for this chatbot.
+
+## Target design for this bot
+
+This is a WhatsApp chatbot, not a full workstation agent. The plugin system should be intentionally smaller and safer than OpenClaw.
+
+The correct target is:
+
+1. JSON can add safe HTTP tools without code.
+2. JSON can select models and providers for known tasks.
+3. JavaScript plugins can add advanced tools when installed deliberately.
+4. The owner CLI can add, list, test, enable, disable, and diagnose tools.
+5. Ordinary customers cannot invoke admin-only tools.
+6. The LLM should never receive raw unrestricted tools in ordinary chat by default.
+7. Tool results should be synthesized into a helpful answer, not dumped raw.
+
+## Proposed config shape
+
+Add this section to `config/chatbot.config.json`:
+
+```json
+{
+  "tools": {
+    "enabled": true,
+    "autoToolsInChat": false,
+    "profile": "chatbot",
+    "allow": ["web_search", "news_search", "url_fetch"],
+    "deny": [],
+    "http": {
+      "weather_lookup": {
+        "enabled": true,
+        "description": "Look up weather from a configured HTTP endpoint",
+        "method": "GET",
+        "url": "https://example-weather.invalid/current",
+        "query": {
+          "q": "{{location}}"
+        },
+        "headersFromEnv": {
+          "Authorization": "WEATHER_API_TOKEN"
+        },
+        "inputSchema": {
+          "location": { "type": "string", "required": true }
+        },
+        "resultMode": "json",
+        "timeoutSeconds": 30,
+        "maxResponseBytes": 200000,
+        "public": false
+      }
+    },
+    "plugins": {
+      "enabled": false,
+      "directories": ["./plugins"],
+      "allow": []
+    }
+  }
+}
+```
+
+Important: secrets stay in `.env`. Config stores only env variable names, never secret values.
+
+## Implementation milestones
+
+### Milestone 1: Tool policy layer
+
+Add:
+
+- `src/tools/tool-policy.ts`
+- `src/tools/__tests__/tool-policy.test.ts`
+
+Responsibilities:
+
+- Apply `tools.enabled`.
+- Apply `tools.profile`.
+- Apply `tools.allow` and `tools.deny`.
+- Deny dangerous tools by default.
+- Keep customer-visible tools separate from owner-only tools.
+- Keep ordinary chat `autoToolsInChat=false` by default.
+
+### Milestone 2: Config-defined HTTP tools
+
+Add:
+
+- `src/tools/http-tool-loader.ts`
+- `src/tools/http-tool-runner.ts`
+- `src/tools/__tests__/http-tool-loader.test.ts`
+- `src/tools/__tests__/http-tool-runner.test.ts`
+
+Capabilities:
+
+- Load HTTP tool definitions from `config/chatbot.config.json`.
+- Validate method, URL, headers, query/body mapping, timeout, response size, and input schema.
+- Reject private, local, metadata, unsupported, or credential-embedded URLs.
+- Read secrets only through env variable names.
+- Return structured results and diagnostics.
+
+### Milestone 3: Unified tool catalog
+
+Refactor `ToolRegistry` so it no longer only registers hardcoded tools.
+
+It should combine:
+
+- built-in tools: web_search, news_search, url_fetch.
+- config-defined HTTP tools.
+- later plugin tools.
+
+Expose:
+
+- `getToolDefinitions()`
+- `getTools()`
+- `handleCommand()`
+- `formatToolStatus()`
+- `doctor()`
+
+### Milestone 4: Synthesis-first command tools
+
+Currently natural-language direct tools synthesize results through `AIResponseEngine`, but command tools such as `/news` and `/fetch` still return formatted result blocks.
+
+Change `/search`, `/news`, and `/fetch` so they can use synthesis mode:
+
+- collect structured result.
+- pass result to tool synthesis model.
+- answer intelligently.
+- include short source note if useful.
+- fall back to concise operational error if the API fails.
+
+Keep a debug mode for raw results:
+
+- `/searchraw <query>`
+- `/newsraw <query>`
+- `/fetchraw <url>`
+
+### Milestone 5: CLI
+
+Add:
+
+- `src/cli/index.ts`
+- package.json `bin` entry.
+
+Target command name:
+
+- `wa-chatbot`
+
+Commands:
+
+- `wa-chatbot init`
+- `wa-chatbot doctor`
+- `wa-chatbot config get <path>`
+- `wa-chatbot config set <path> <json-value>`
+- `wa-chatbot config unset <path>`
+- `wa-chatbot tools list`
+- `wa-chatbot tools test <name> <json-args>`
+- `wa-chatbot tools add-http <name>`
+- `wa-chatbot models list`
+- `wa-chatbot models set <task> <provider/model>`
+- `wa-chatbot start`
+- `wa-chatbot worker`
+
+### Milestone 6: Docker
+
+Add:
+
+- `Dockerfile`
+- `.dockerignore`
+- `docker-compose.yml`
+- `docs/docker.md`
+
+Requirements:
+
+- multi-stage build.
+- non-root runtime user.
+- persistent mounted volumes for auth, data, config, and media.
+- env file support.
+- healthcheck command.
+- Linux amd64 and arm64 support where native dependencies allow.
+- clear instructions for QR/pairing and session persistence.
+
+### Milestone 7: npm/global installation
+
+Add package metadata and CLI entry.
+
+Package should support:
+
+```bash
+npm install -g wa-chatbot
+wa-chatbot init
+wa-chatbot doctor
+wa-chatbot start
+```
+
+The package must include compiled `dist`, config templates, schema files, docs, and migration scripts.
+
+### Milestone 8: installers
+
+Add optional convenience installers:
+
+- `install.sh` for Linux/macOS.
+- `install.ps1` for Windows PowerShell.
+
+They should:
+
+- verify Node version.
+- verify npm.
+- install dependencies or global package.
+- create config directory.
+- create env template.
+- run doctor.
+
+### Milestone 9: CI and release
+
+Add GitHub Actions:
+
+- test on Linux, macOS, Windows.
+- Node 20 and 22.
+- build Docker image.
+- run CLI smoke test.
+- run package dry-run.
+
+## Tests required before calling this production-ready
+
+- JSON-defined HTTP tool loads and validates.
+- Bad HTTP tool config refuses startup.
+- Missing env secret shows doctor warning, not crash.
+- Private URL is blocked before provider call.
+- Oversized response is truncated safely.
+- HTTP tool can be disabled by config.
+- HTTP tool respects allow/deny policy.
+- Owner can run tool test command.
+- Customer cannot run owner-only tool.
+- Natural-language direct tool answers synthesized response, not raw dump.
+- `/newsraw` and `/searchraw` intentionally dump raw results only when requested.
+- Docker container starts and persists Baileys auth.
+- Docker restart does not trigger repeated replies.
+- CLI config set/get/unset works cross-platform.
+- Global npm install exposes the CLI binary.
+
+## Final product target
+
+The goal is not to clone OpenClaw. The goal is to make this chatbot installable and configurable with the best OpenClaw-style pieces:
+
+- config-driven setup.
+- strict validation.
+- doctor command.
+- safe tool policy.
+- model routing.
+- plugin-friendly extension points.
+- Docker and npm install paths.
+
+The bot should remain simpler than OpenClaw because customers only chat and the owner/admin controls privileged capabilities.

package/package.json

@@ -0,0 +1,55 @@
+{
+  "name": "nduka",
+  "version": "1.0.0",
+  "description": "WhatsApp AI Gateway",
+  "type": "module",
+  "main": "./dist/gateway/server.js",
+  "bin": {
+    "nduka": "./dist/cli/index.js",
+    "wa-chatbot": "./dist/cli/index.js"
+  },
+  "files": [
+    "dist",
+    "config",
+    "docs",
+    "package.json"
+  ],
+  "scripts": {
+    "dev": "tsx src/gateway/server.ts",
+    "dev:worker": "MEDIA_QUEUE_WORKER_ENABLED=true tsx src/gateway/bootstrap.ts",
+    "build": "tsc",
+    "start": "node dist/gateway/server.js",
+    "start:worker": "MEDIA_QUEUE_WORKER_ENABLED=true node dist/gateway/bootstrap.js",
+    "cli": "tsx src/cli/index.ts",
+    "test": "vitest run --no-cache",
+    "test:watch": "vitest --no-cache",
+    "test:owner": "vitest run --no-cache src/owner/__tests__/owner-router.test.ts"
+  },
+  "dependencies": {
+    "@whiskeysockets/baileys": "WhiskeySockets/Baileys#097254c49faf4ed757605d5a278097be1bfd4409",
+    "async-mutex": "^0.5.0",
+    "better-sqlite3": "^12.11.1",
+    "link-preview-js": "^3.2.0",
+    "qrcode-terminal": "^0.12.0"
+  },
+  "devDependencies": {
+    "@types/better-sqlite3": "^7.6.13",
+    "@types/node": "^22.0.0",
+    "@types/qrcode-terminal": "^0.12.2",
+    "tsx": "^4.0.0",
+    "typescript": "^5.5.0",
+    "vitest": "^2.0.0"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/Ndukka/baileys-whatsapp.git"
+  },
+  "bugs": {
+    "url": "https://github.com/Ndukka/baileys-whatsapp/issues"
+  },
+  "homepage": "https://github.com/Ndukka/baileys-whatsapp#readme",
+  "license": "MIT",
+  "engines": {
+    "node": ">=22"
+  }
+}