nduka 1.0.0

package/docs/installation.md

@@ -0,0 +1,384 @@
+# WhatsApp AI Gateway Installation Guide
+
+Current release: `chatbot-v2-tool-system`
+
+This guide explains how to install, configure, test, and run the WhatsApp AI Gateway locally, globally through npm, and through Docker.
+
+## 1. Requirements
+
+Install these first:
+
+- Node.js 22 or newer
+- npm
+- Git
+- Docker Desktop, only for the Docker run path
+- A WhatsApp account that can link a device
+- API keys for at least one text model provider
+
+Recommended provider keys:
+
+```bash
+OPENROUTER_API_KEY=...
+KILOCODE_API_KEY=...
+GROQ_API_KEY=...
+CRUNCHTROVE_API=...
+BOT_TIMEZONE=Asia/Dubai
+```
+
+Optional keys:
+
+```bash
+GEMINI_API_KEY=...
+ANTHROPIC_API_KEY=...
+OPENAI_API_KEY=...
+```
+
+Do not commit `.env`, auth state, databases, media files, or local data.
+
+## 2. Clone and install locally
+
+```bash
+git clone <repo-url>
+cd baileys-whatsapp
+git checkout main
+npm install
+```
+
+Create your local `.env` file:
+
+```bash
+cp .env.example .env 2>/dev/null || touch .env
+```
+
+Add your provider keys to `.env`.
+
+## 3. Initialize local config files
+
+The repo includes default config files under `config/`. To create or refresh local defaults, run:
+
+```bash
+npm run cli -- init
+```
+
+To overwrite existing local config files with defaults:
+
+```bash
+npm run cli -- init --force
+```
+
+The active config paths are:
+
+```text
+CHATBOT_CONFIG_PATH=./config/chatbot.config.json
+CHATBOT_TOOLS_PATH=./config/tools.chatbot.json
+CHATBOT_PROMPTS_PATH=./config/prompts.chatbot.json
+```
+
+You can override them in `.env` if needed.
+
+## 4. Build and test
+
+Run these before starting the bot:
+
+```bash
+npm run build
+npm test
+npm run cli -- doctor
+npm run cli -- config validate
+```
+
+Expected current green status for this release:
+
+```text
+75 test files passed
+659 tests passed
+```
+
+The exact timing can vary by machine.
+
+## 5. Useful CLI commands
+
+Diagnostics:
+
+```bash
+npm run cli -- doctor
+npm run cli -- smoke
+npm run cli -- tools list
+npm run cli -- models status
+npm run cli -- models list
+npm run cli -- prompts list
+npm run cli -- prompts get system
+npm run cli -- config paths
+npm run cli -- config validate
+npm run cli -- config show
+npm run cli -- config get tools.enabled
+```
+
+Known-tool tests:
+
+```bash
+npm run cli -- tools test news_search "US Iran today"
+npm run cli -- tools test web_search "current president of Nigeria"
+npm run cli -- tools test url_fetch "https://gulfnews.com"
+```
+
+Runtime command previews:
+
+```bash
+npm run cli -- start --print
+npm run cli -- worker --print
+```
+
+Expected output:
+
+```text
+node dist/gateway/server.js
+MEDIA_QUEUE_WORKER_ENABLED=true node dist/gateway/bootstrap.js
+```
+
+## 6. Start locally
+
+Use the worker entrypoint for normal operation:
+
+```bash
+npm run start:worker
+```
+
+First run may show a QR code or pairing flow. Link it from WhatsApp as a linked device. Auth is stored locally so later restarts should reuse the session.
+
+Stop with `Ctrl+C`. The gateway handles shutdown and stops the media worker.
+
+## 7. Manual smoke test
+
+After the gateway shows READY, test these paths.
+
+### Customer DM
+
+Send:
+
+```text
+/news US Iran today
+/search president of Nigeria
+/fetch https://gulfnews.com
+```
+
+Expected:
+
+```text
+No commands are needed...
+```
+
+Customers must not be able to invoke slash search/news/fetch tools.
+
+Then send natural-language questions:
+
+```text
+What is the news about US and Iran today?
+Who is the current president of Nigeria and when did he enter office?
+What is obesity?
+```
+
+Expected:
+
+- Current/news questions should answer naturally using deterministic direct tool routing where available.
+- General knowledge questions should answer normally.
+- No slash command access should be required or exposed to customers.
+
+### Owner self-chat
+
+Send:
+
+```text
+/help
+/news US Iran today
+/search president of Nigeria
+/fetch https://gulfnews.com
+/newsraw US Iran today
+/searchraw president of Nigeria
+/fetchraw https://gulfnews.com
+/toolstatus
+/model
+/models
+```
+
+Expected:
+
+- `/help` shows the full owner/admin help, including inbox commands.
+- `/news`, `/search`, and `/fetch` return synthesized answers.
+- `/newsraw`, `/searchraw`, and `/fetchraw` return raw debug output.
+- `/toolstatus`, `/model`, and `/models` work only for the owner.
+
+### Group chat
+
+Unmentioned group message:
+
+```text
+We are doing our best.
+```
+
+Expected: no reply.
+
+Mentioned group question:
+
+```text
+@bot what is today's update?
+```
+
+Expected: concise group reply or safe fallback.
+
+## 8. Docker run path
+
+Docker should be validated after local build, tests, CLI, and WhatsApp smoke pass.
+
+Start Docker Desktop first, then run:
+
+```bash
+docker compose build
+docker compose run --rm wa-chatbot node dist/cli/index.js doctor
+docker compose run --rm wa-chatbot node dist/cli/index.js config validate
+docker compose run --rm wa-chatbot node dist/cli/index.js smoke
+```
+
+If those pass, start the service:
+
+```bash
+docker compose up
+```
+
+The container mounts:
+
+```text
+./data   -> /app/data
+./config -> /app/config:ro
+```
+
+Important runtime paths inside Docker:
+
+```text
+AUTH_FILE_PATH=/app/data/auth_state.json
+DATABASE_PATH=/app/data/bot.db
+MEDIA_DIR=/app/data/media
+CHATBOT_CONFIG_PATH=/app/config/chatbot.config.json
+CHATBOT_TOOLS_PATH=/app/config/tools.chatbot.json
+CHATBOT_PROMPTS_PATH=/app/config/prompts.chatbot.json
+```
+
+Stop Docker:
+
+```bash
+docker compose down
+```
+
+Rebuild after code changes:
+
+```bash
+docker compose build --no-cache
+docker compose up
+```
+
+## 9. Global npm package smoke test
+
+From the repo root:
+
+```bash
+npm run build
+npm pack
+npm install -g ./whatsapp-automation-baileys-1.0.0.tgz
+```
+
+Then test:
+
+```bash
+wa-chatbot doctor
+wa-chatbot tools list
+wa-chatbot models status
+wa-chatbot prompts list
+wa-chatbot config validate
+wa-chatbot smoke
+wa-chatbot worker --print
+```
+
+Uninstall:
+
+```bash
+npm uninstall -g whatsapp-automation-baileys
+```
+
+## 10. Behavior guarantees in this branch
+
+- Customer slash tool commands are blocked.
+- Owner slash tool commands work in owner self-chat.
+- Natural-language customer current/news questions can still route through deterministic tools.
+- Known tools come from the catalog.
+- Model-visible tool definitions are generated from the catalog.
+- Tool command output is synthesis-first by default.
+- Raw commands exist only for owner debugging.
+- Configurable prompts load from `config/prompts.chatbot.json`.
+- Docker does not bake secrets or auth into the image.
+
+## 11. Troubleshooting
+
+### Build passes but web/news results are poor
+
+The web/search provider can return weak snippets, missing content, or `No content`. This is an upstream provider issue. Use raw commands to inspect:
+
+```text
+/newsraw US Iran today
+/searchraw current president of Nigeria
+/fetchraw https://example.com
+```
+
+### Customer can run `/news` or `/search`
+
+This is a regression. Re-run:
+
+```bash
+npm test -- src/conversation/__tests__/intent-routing.test.ts
+npm test -- src/conversation/__tests__/runtime-group.test.ts
+```
+
+Expected behavior: customer slash tool commands receive normal customer command guidance, not search results.
+
+### Owner `/help` shows only tool help
+
+This is a regression. Owner `/help` must fall through to the full owner/admin router, not the tool registry.
+
+### Docker starts but QR/auth does not persist
+
+Check that `./data` exists and is writable:
+
+```bash
+mkdir -p data
+ls -la data
+```
+
+The auth file should persist at:
+
+```text
+./data/auth_state.json
+```
+
+### Docker cannot read config
+
+Check:
+
+```bash
+ls -la config
+cat config/chatbot.config.json
+cat config/tools.chatbot.json
+cat config/prompts.chatbot.json
+```
+
+The compose file mounts config read-only. Edit config files on the host, then restart the container.
+
+### Frequent disconnects or reconnect loops
+
+This branch treats logout, forbidden, connection replaced, device removed, and multi-device mismatch as terminal conditions. It reconnects immediately for restart-required lifecycle events and uses backoff for transient disconnects. If terminal disconnect happens, re-link WhatsApp instead of forcing reconnect loops.
+
+## 12. Intentionally not included yet
+
+- Arbitrary custom HTTP tool execution.
+- `config set` / `config unset` CLI writing commands.
+- Publishing to npm.
+
+These should be considered future v2.x work, not part of `chatbot-v2-tool-system`.

package/docs/media-worker.md

@@ -0,0 +1,114 @@
+# Media Queue Worker Guide
+
+The media queue worker is the background processor for media jobs that the WhatsApp gateway has already accepted and stored.
+
+The gateway receives messages, downloads or routes media, and queues safe work quickly. The worker then picks up queued media jobs in small batches and runs the configured media processor, such as OCR, document summary, spreadsheet summary, presentation summary, or image description.
+
+## Why the worker exists
+
+Media processing can be slow or retryable. Running it directly inside the live WhatsApp message handler would make the bot less responsive. The worker keeps the gateway responsive by processing queued media separately.
+
+The worker:
+
+- claims due media queue jobs
+- processes a bounded batch at a time
+- prevents overlapping ticks
+- marks successful jobs as completed
+- retries retryable failures with backoff
+- moves unsupported or risky jobs to owner review
+- marks final failures as failed
+- logs sanitized activity summaries only when there is actual work
+
+## Current startup paths
+
+Standard gateway without the worker:
+
+```bash
+npm run dev
+```
+
+Worker-enabled gateway path for local development:
+
+```bash
+npm run dev:worker
+```
+
+After building:
+
+```bash
+npm run build
+npm run start:worker
+```
+
+Equivalent direct commands:
+
+```bash
+MEDIA_QUEUE_WORKER_ENABLED=true npx tsx src/gateway/bootstrap.ts
+MEDIA_QUEUE_WORKER_ENABLED=true node dist/gateway/bootstrap.js
+```
+
+The bootstrap path starts the media queue worker first, registers synchronous worker cleanup for process exit, and then imports the normal gateway server. The gateway server remains the owner of SIGINT/SIGTERM shutdown so the worker does not compete with the main shutdown handler.
+
+## Environment variables
+
+| Variable | Default | Purpose |
+| --- | --- | --- |
+| `MEDIA_QUEUE_WORKER_ENABLED` | `false` | Set to `true` to start the worker. The worker npm scripts set this for you. |
+| `MEDIA_QUEUE_WORKER_INTERVAL_MS` | `15000` | Worker tick interval. The worker enforces a minimum of 1000ms. |
+| `MEDIA_QUEUE_WORKER_LIMIT` | `5` | Max jobs claimed per tick. The worker caps this between 1 and 25. |
+
+Example local development command with custom limits:
+
+```bash
+MEDIA_QUEUE_WORKER_INTERVAL_MS=10000 MEDIA_QUEUE_WORKER_LIMIT=3 npm run dev:worker
+```
+
+## Owner command flow
+
+Owner media commands are still handled from the owner self-chat. Useful commands include:
+
+```text
+/media
+/media review
+/media queued
+/media failed
+/media summary <id>
+/media approve <id>
+/media retry <id>
+/media cancel <id> [reason]
+```
+
+A typical owner-review flow is:
+
+1. Customer sends media.
+2. Policy decides whether the item is safe to queue or needs owner review.
+3. Owner checks `/media review`.
+4. Owner approves with `/media approve <id>` if it should be processed.
+5. Worker picks it up on a future tick and processes it.
+
+## Safety notes
+
+- The worker does not give customers access to admin commands.
+- The worker does not process group media automatically.
+- The worker is disabled unless explicitly enabled.
+- The worker prevents overlapping ticks so a slow media job does not start a second concurrent batch.
+- The worker cleanup runs on process exit without adding another SIGINT/SIGTERM listener.
+- Logs use sanitized fields to reduce log injection and accidental identifier exposure.
+
+## Test commands
+
+Run worker-related tests:
+
+```bash
+npx vitest run --no-cache src/media/__tests__/queue-worker.test.ts
+npx vitest run --no-cache src/media/__tests__/queue-worker-edge.test.ts
+npx vitest run --no-cache src/gateway/__tests__/media-queue-worker-runtime.test.ts
+npx vitest run --no-cache src/gateway/__tests__/process-lifecycle.test.ts
+npx vitest run --no-cache src/gateway/__tests__/exit-shutdown.test.ts
+```
+
+Run the full suite:
+
+```bash
+npm test
+```

package/docs/npm-install.md

@@ -0,0 +1,75 @@
+# npm / Global CLI Installation
+
+The package exposes a CLI binary named `wa-chatbot` through `package.json`.
+
+## Local development
+
+```bash
+npm install
+npm run build
+npm run cli -- doctor
+npm run cli -- tools list
+npm run cli -- prompts list
+```
+
+## Local global install smoke test
+
+From the repository root:
+
+```bash
+npm run build
+npm pack
+npm install -g ./whatsapp-automation-baileys-1.0.0.tgz
+wa-chatbot doctor
+wa-chatbot tools list
+wa-chatbot prompts list
+```
+
+Uninstall after testing:
+
+```bash
+npm uninstall -g whatsapp-automation-baileys
+```
+
+## Runtime commands
+
+The gateway still runs through npm scripts or the built Node entrypoint:
+
+```bash
+npm run start:worker
+# or
+MEDIA_QUEUE_WORKER_ENABLED=true node dist/gateway/bootstrap.js
+```
+
+The CLI is currently for diagnostics and configuration visibility:
+
+```bash
+wa-chatbot doctor
+wa-chatbot tools list
+wa-chatbot prompts list
+wa-chatbot config paths
+```
+
+## Package contents
+
+The package includes:
+
+- `dist`
+- `config`
+- `docs`
+- `package.json`
+
+Secrets are not packaged. `.env`, auth state, database files, media files, and local data are excluded by convention and should remain local.
+
+## Not done yet
+
+Future CLI milestones:
+
+- `wa-chatbot init`
+- `wa-chatbot config get/set/unset`
+- `wa-chatbot models list/set`
+- `wa-chatbot start`
+- `wa-chatbot worker`
+- `wa-chatbot tools test`
+
+These should be added after the catalog-backed registry and synthesis-first command flow are fully stable.