nduka 1.0.0

package/docs/production-handover.md

@@ -0,0 +1,255 @@
+# WhatsApp AI Gateway v2 Production Handover
+
+Release: `chatbot-v2-tool-system`
+
+Main commit at release: `18be9aecf9174c67e326d2a40a3f87218b37124f`
+
+Active follow-up branch: `feature/v2.2-config-write-commands`
+
+Status at release:
+
+```text
+75 test files passed
+659 tests passed
+Docker runtime reached READY
+Manual local WhatsApp smoke passed
+Manual Docker WhatsApp smoke passed
+```
+
+## 1. What v2 adds
+
+This release adds the v2 tool system and production hardening layer:
+
+- Docker image and Compose runtime.
+- CLI diagnostics and smoke commands.
+- Configurable prompt registry.
+- Configurable tool catalog.
+- Owner-only slash tool commands.
+- Customer slash-command blocking.
+- Natural-language current-information routing.
+- Tool output synthesis.
+- Raw debug commands for owner inspection.
+- Model-visible tool definitions from catalog.
+- Baileys reconnect hardening.
+- Response sanitizer to prevent model preambles or internal text from reaching WhatsApp replies.
+- Installation guide under `docs/installation.md`.
+
+## 2. v2.2 follow-up branch
+
+The branch `feature/v2.2-config-write-commands` adds safe CLI mutation for `config/chatbot.config.json` only:
+
+```bash
+npm run cli -- config set <path> <json-value>
+npm run cli -- config unset <path>
+npm run cli -- config backup
+npm run cli -- config restore <backup-file>
+npm run cli -- config diff
+npm run cli -- config validate
+```
+
+Safety rules for this branch:
+
+- Backup before every successful mutation.
+- Validate candidate config before writing.
+- Write through a temporary file, then replace the target file.
+- Use stable pretty JSON.
+- Reject unknown or unsafe paths.
+- Reject prototype-pollution style path segments.
+- Reject secret-like values.
+- Keep `config/tools.chatbot.json` read-only until custom tool schema safety exists.
+
+See `docs/config-write-commands.md` for details.
+
+## 3. Verified commands
+
+Before deployment or after any change, run:
+
+```bash
+git checkout main
+git pull origin main
+npm install
+npm run build
+npm test
+npm run cli -- doctor
+npm run cli -- smoke
+npm run cli -- config validate
+npm run cli -- start --print
+npm run cli -- worker --print
+```
+
+Expected release baseline:
+
+```text
+75 test files passed
+659 tests passed
+```
+
+For v2.2 config write changes, also run:
+
+```bash
+npm run cli -- config backup
+npm run cli -- config diff
+npm run cli -- config set debug.traceTools false
+npm run cli -- config get debug.traceTools
+npm run cli -- config unset debug.traceTools
+npm run cli -- config get debug.traceTools
+npm run cli -- config validate
+```
+
+## 4. Local runtime smoke
+
+Start locally:
+
+```bash
+npm run start:worker
+```
+
+Wait for READY:
+
+```text
+READY — processing messages now
+```
+
+Smoke test three surfaces:
+
+1. Customer direct message.
+2. Owner self-chat.
+3. Group chat.
+
+Expected behavior:
+
+- Customer slash tool commands should receive normal customer guidance, not tool results.
+- Customer natural-language questions should answer normally.
+- Owner slash tools should work only in owner self-chat.
+- Owner raw commands should return raw debug output.
+- Unmentioned group messages should not trigger a reply.
+- Mentioned group questions should receive a concise reply or safe fallback.
+- WhatsApp replies should not include model preambles, hidden planning, or tool-call artifacts.
+
+## 5. Docker runtime smoke
+
+Start Docker Desktop, then run:
+
+```bash
+docker compose down
+docker compose build --no-cache
+docker compose run --rm wa-chatbot node dist/cli/index.js doctor
+docker compose run --rm wa-chatbot node dist/cli/index.js config validate
+docker compose run --rm wa-chatbot node dist/cli/index.js smoke
+docker compose run --rm wa-chatbot node dist/cli/index.js worker --print
+docker compose up
+```
+
+Expected READY path:
+
+```text
+Connected to WhatsApp
+READY — processing messages now
+```
+
+A 515 restart-required event immediately after pairing is acceptable if it is followed by automatic reconnect and READY.
+
+Stop Docker:
+
+```bash
+docker compose down
+```
+
+## 6. Persistent runtime data
+
+Docker mounts:
+
+```text
+./data   -> /app/data
+./config -> /app/config:ro
+```
+
+Important container paths:
+
+```text
+AUTH_FILE_PATH=/app/data/auth_state.json
+DATABASE_PATH=/app/data/bot.db
+MEDIA_DIR=/app/data/media
+CHATBOT_CONFIG_PATH=/app/config/chatbot.config.json
+CHATBOT_TOOLS_PATH=/app/config/tools.chatbot.json
+CHATBOT_PROMPTS_PATH=/app/config/prompts.chatbot.json
+```
+
+Do not delete `data/` unless you intentionally want to reset auth/session state.
+
+## 7. Known external limitation
+
+The web/news provider can return weak snippets, empty content, or incomplete results. That is an upstream provider-quality issue, not a routing or command-policy issue. Use owner raw debug commands to inspect provider output before assuming an app bug.
+
+## 8. Regression test map
+
+Use these targeted test files when debugging:
+
+```bash
+npm test -- src/conversation/__tests__/intent-routing.test.ts
+npm test -- src/conversation/__tests__/runtime-group.test.ts
+npm test -- src/gateway/reconnect/__tests__/manager.test.ts
+npm test -- src/llm/__tests__/response-sanitizer.test.ts
+npm test -- src/cli/__tests__/cli.test.ts
+npm test -- src/cli/__tests__/cli-config-write.test.ts
+npm test -- src/config/__tests__/config-editor.test.ts
+npm test -- src/tools/__tests__/tools.test.ts
+```
+
+What they protect:
+
+- Customer slash command blocking.
+- Owner command behavior.
+- Group activation behavior.
+- Baileys reconnect decisions.
+- Response sanitization.
+- CLI documented command behavior.
+- Config write safety.
+- Tool registry and provider formatting.
+
+## 9. Do not change these without tests
+
+Treat these as release-critical behavior:
+
+- Customers must not invoke slash search/news/fetch tools.
+- Owner self-chat slash tools must continue to work.
+- Owner `/help` must show full owner/admin help, not just tool help.
+- Raw debug commands must stay owner-only.
+- Natural-language customer questions must still be allowed to route through tools when the gateway decides they need current information.
+- Docker must not bake secrets, auth, database files, or media into the image.
+- Terminal disconnects such as logout, forbidden, connection replaced, device removed, and multi-device mismatch should not reconnect in a loop.
+- 515 restart-required should reconnect immediately.
+- Model preambles or hidden planning text must not be sent to WhatsApp users.
+- Config write commands must not edit tool catalog definitions, prompts, secrets, or owner/customer command policy.
+
+## 10. Future work intentionally left out of the frozen v2 tag
+
+These are not part of `chatbot-v2-tool-system`:
+
+- Arbitrary custom HTTP tool execution.
+- CLI config writing commands such as config set/unset, except on the v2.2 follow-up branch.
+- npm publishing.
+- More provider fallback tuning for weak web/news results.
+
+Handle those as future v2.x work with separate tests.
+
+## 11. Branch cleanup after release
+
+After confirming `main` and `chatbot-v2-tool-system` are correct, the merged feature branch can be deleted.
+
+Local cleanup:
+
+```bash
+git checkout main
+git pull origin main
+git branch -d feature/tool-system-v2
+```
+
+Remote cleanup:
+
+```bash
+git push origin --delete feature/tool-system-v2
+git fetch --prune
+```
+
+If you want to keep the branch for archive/reference, do not delete it. The current release is already preserved by the `chatbot-v2-tool-system` tag.

package/docs/production-monitoring.md

@@ -0,0 +1,117 @@
+# Production Monitoring and Runtime Safety
+
+Branch: `feature/v2.6-production-monitoring`
+
+This milestone adds local and Docker runtime checks that do not start WhatsApp and do not send messages.
+
+## Runtime health command
+
+```bash
+npm run cli -- health
+npm run cli -- health --quiet
+```
+
+The command checks:
+
+- Chatbot config can load.
+- Runtime data directory exists.
+- Auth state file exists or warns when first pairing may be needed.
+- Database file exists or warns when it may be created on first run.
+- Media directory exists.
+- Basic disk space near the runtime data directory.
+
+Exit code behavior:
+
+- `ok`: exits `0`.
+- `warn`: exits `0`, because first-run auth/database warnings should not make Docker restart forever.
+- `fail`: exits `1`, for example when config cannot load or the data directory is missing.
+
+## Docker healthcheck
+
+The Docker image and Compose service use:
+
+```bash
+node dist/cli/index.js health --quiet
+```
+
+This checks runtime health without opening a WhatsApp socket and without sending messages.
+
+Compose keeps:
+
+```yaml
+restart: unless-stopped
+```
+
+This restarts the service after ordinary failures, but it will not restart again after an operator intentionally stops it.
+
+## Log rotation
+
+Compose uses bounded container logs:
+
+```yaml
+logging:
+  driver: json-file
+  options:
+    max-size: "10m"
+    max-file: "3"
+```
+
+This prevents long-running container logs from growing without limit.
+
+## Runtime data backup
+
+Create a backup of runtime data:
+
+```bash
+bash scripts/backup-data.sh
+```
+
+Optional environment overrides:
+
+```bash
+DATA_DIR=./data BACKUP_DIR=./backups bash scripts/backup-data.sh
+```
+
+The backup script archives runtime data only and skips local secret files.
+
+## Runtime data restore
+
+Restore a backup intentionally:
+
+```bash
+bash scripts/restore-data.sh ./backups/data-YYYYMMDDTHHMMSSZ.tar.gz --yes
+```
+
+The restore script refuses to run without `--yes`. If runtime data already exists, it creates a pre-restore backup before replacing it.
+
+## Manual validation
+
+Run:
+
+```bash
+npm run build
+npm test
+npm run cli -- health
+npm run cli -- health --quiet
+bash scripts/check-runtime.sh
+```
+
+Docker validation:
+
+```bash
+docker compose config
+docker compose build --no-cache
+docker compose run --rm wa-chatbot node dist/cli/index.js health
+docker compose run --rm wa-chatbot node dist/cli/index.js config validate
+docker compose up
+```
+
+Expected result: Docker reports a health status based on config/data checks, not a fake always-success healthcheck.
+
+## Guardrails
+
+- Health checks must not send WhatsApp messages.
+- Health checks must not start WhatsApp pairing or gateway sockets.
+- Backup archives must not include local secret files.
+- Restore must require explicit confirmation.
+- Missing auth or database files are warnings, not failures, because first-run deployments may create them later.

package/docs/routing-state-machine.md

@@ -0,0 +1,47 @@
+# WhatsApp Routing State Machine
+
+## Order
+
+1. Hard gates
+   - Slash commands
+   - Owner commands
+   - Group policy
+   - Blocked/resolved chats
+   - Rate limits
+   - Secure/private tool approval flows
+
+2. Structured context resolver
+   - Builds candidate context from quoted WhatsApp text and the latest successful async tool result for the same chat.
+   - Returns one mode only: `standalone`, `quoted_context`, `previous_tool_context`, `combined_context`, or `clarify`.
+   - Rewrites the current user message into a self-contained `resolvedText` only when context is selected.
+   - Defaults safely to `standalone` if there is no candidate context, invalid JSON, low confidence, or invalid resolver output.
+   - If the resolver returns `clarify`, routing stops before triage/tool matching and asks for clarification.
+
+3. Message triage planner
+   - answer_direct
+   - needs_tool
+   - clarify
+   - human_review
+   - blocked
+
+4. Tool match planner
+   - Sees only configured tools from `config/tools.chatbot.json`
+   - Returns toolName and args only from the catalog
+   - Runtime validates enabled/public/ownerOnly/asyncEligible/schema
+   - If matching fails while selected context is attached, retry once without conversation context.
+
+5. Durable async job
+   - Enqueue before first acknowledgement
+   - No job IDs in customer replies
+
+6. Tool execution and synthesis
+   - Answer only from tool result
+   - No invented URLs or fake sources
+   - Weak or empty result triggers clarification or manual review
+
+## Failure policy
+
+- First uncertainty: ask one clear clarification question.
+- Repeated uncertainty: set HUMAN_PENDING and label manual_review.
+- Private account data without verified lookup: never answer directly.
+- Response wording for routing acknowledgements, clarification, manual review, and blocked messages belongs in `config/tools.chatbot.json` or the default catalog response policy, not in runtime routing branches.

package/docs/support-inbox-milestones.md

@@ -0,0 +1,159 @@
+# Support Inbox Milestones
+
+Branch: `feature/support-inbox-profiles-labels`
+
+This file records what has been implemented and what should happen next so the branch can be resumed safely after any interruption.
+
+## Core rule
+
+Owner control must stay separate from customer behavior.
+
+- Owner commands work only in the owner self-chat.
+- Customers cannot run owner commands.
+- Owner messages inside a customer chat are treated as human takeover messages, not admin commands.
+- Group chats must not receive owner command access.
+- Future media processing must be DM-only unless explicitly changed later.
+
+## Research basis
+
+- Chatwoot AgentBot uses conversation status to separate bot triage and human handoff.
+- Dify Question Classifier routes user input to different workflow paths before answering.
+- LangChain treats routing, tools, model selection, memory, and fault tolerance as harness concerns around the model.
+
+## Milestone 1: Support inbox foundation
+
+Status: implemented and live smoke-tested.
+
+Files:
+
+- `src/support/store.ts`
+- `src/support/__tests__/support-state.test.ts`
+- `src/conversation/__tests__/support-commands.test.ts`
+- `src/conversation/runtime.ts`
+
+Features implemented:
+
+- Conversation support statuses:
+  - `AI_ACTIVE`
+  - `HUMAN_PENDING`
+  - `HUMAN_ACTIVE`
+  - `WAITING_ON_CUSTOMER`
+  - `WAITING_ON_OWNER`
+  - `RESOLVED`
+  - `BLOCKED`
+- Labels and tags.
+- Priority: `low`, `normal`, `high`, `urgent`.
+- Customer profiles.
+- Owner notes.
+- `/inbox` with filters.
+- `/profile <alias>`.
+- `/resolve` and `/reopen`.
+- `/pause` and `/continue` now mirror support status.
+
+Owner commands added:
+
+```text
+/inbox [filter]
+/profile <target>
+/status <target> [status]
+/resolve <target>
+/reopen <target>
+/pending <target>
+/waiting <target> customer|owner
+/block <target>
+/unblock <target>
+/label <target> <label>
+/labels <target>
+/unlabel <target> <label>
+/priority <target> low|normal|high|urgent
+/note <target> <note>
+/notes <target>
+/language <target> <language>
+```
+
+Target means alias, numbered shortcut, or raw chat JID.
+
+Verified by user:
+
+- Focused handoff tests pass.
+- Full suite passes: 20 files, 397 tests.
+- Live WhatsApp smoke test passed for profile, labels, priority, notes, inbox filters, pause, resolve, reopen, and continue.
+
+## Milestone 2: Detailed owner help
+
+Status: implemented, awaiting local verification.
+
+Files:
+
+- `src/owner/help.ts`
+- `src/conversation/runtime.ts`
+
+Behavior:
+
+- Owner `/help` returns a detailed command guide with examples.
+- Customer `/help` remains simple and does not expose owner controls.
+
+## Milestone 3: Intent classifier before AI reply
+
+Status: implemented, awaiting local verification.
+
+Files:
+
+- `src/intent/classifier.ts`
+- `src/intent/__tests__/classifier.test.ts`
+- `src/conversation/__tests__/intent-routing.test.ts`
+- `src/conversation/runtime.ts`
+
+Classes:
+
+- `smalltalk`
+- `support_question`
+- `current_info`
+- `handoff_request`
+- `complaint`
+- `sales_lead`
+- `risk_flag`
+- `media_required`
+- `owner_command_attempt`
+
+Implementation notes:
+
+- The first version is deterministic and model-independent.
+- Handoff requests set `HUMAN_PENDING`, add a follow-up label, and do not trigger normal AI generation.
+- Complaints add a complaint label and raise priority.
+- Sales leads add lead labels.
+- Media-needed messages add a media label.
+- Current-info messages continue through the existing tool path.
+- Owner-command attempts remain blocked.
+
+## Next milestone: Task-specific model router
+
+Status: next.
+
+Goal:
+
+- Add a task router so different models can handle different jobs.
+- Keep API keys in environment variables, not raw SQLite values.
+- Add owner commands to inspect and set task model routing later.
+
+Planned tasks:
+
+- `chat_reply`
+- `tool_synthesis`
+- `summary`
+- `classification`
+- `safety`
+- `long_reasoning`
+- `vision_ocr`
+- `pdf_summary`
+- `document_summary`
+- `audio_transcription`
+- `embedding`
+
+Future milestones after model router:
+
+- Knowledge base and RAG commands.
+- DM-only media understanding.
+- Optional group mode.
+- Better tool debug commands.
+- Auto-summary memory.