dialog-survey 1.7.8 → 1.11.0

package/README.md

@@ -1,307 +1,300 @@
 # Dialog Survey
 
-AI-powered interview robot that conducts async multi-turn interviews via DingTalk with intelligent follow-up questions, multi-turn context memory, and voice input support.
+> **Turn DingTalk conversations into structured intelligence.**
+>
+> An AI-powered survey dialog bot that conducts async multi-turn conversations via DingTalk — with LLM-driven follow-ups, context memory, and automated report generation.
 
-## Features
+[![Version](https://img.shields.io/badge/version-1.10.0-blue)](https://github.com/boyingliu01/dialog-survey)
+[![Node](https://img.shields.io/badge/node-%3E%3D20.19-brightgreen)]()
+[![License](https://img.shields.io/badge/license-MIT-green)]()
 
-- **AI 智能追问**: 基于 LLM 的完全智能型追问能力
-- **多轮上下文记忆**: 支持跨多天、多条消息的对话连贯性
-- **语音输入**: 支持语音消息回答
-- **异步消息式访谈**: 被访谈者可以在碎片时间回答
-- **自动生成报告**: 访谈结束后自动生成 Markdown 报告
-- **私有部署**: 数据全留存，确保安全性
+---
 
-## Tech Stack
-
-- **对话引擎**: Custom LangGraph-inspired workflow (not StateGraph API)
-- **LLM**: OpenAI-compatible API（支持本地部署：如 ollama / vLLM / LocalAI；也可对接云服务）
-- **语音识别**: 阿里云 Fun-ASR
-- **消息平台**: DingTalk Stream Mode (WebSocket)
-- **数据存储**: PostgreSQL (Prisma ORM)
-- **Web 框架**: Fastify 5.x
-- **模板引擎**: Nunjucks (server-side views)
-- **语言**: TypeScript (strict mode, ESM)
-
-## Installation
+## Quickstart
 
 ```bash
-# Interactive installation — prompts for each value
+# One-line install — interactive prompts for each value
 npx dialog-survey install
 
-# Non-interactive installation (all required flags)
-npx dialog-survey install \
-  --db-url "postgresql://user:pass@localhost:5432/dialog_survey" \
-  --dingtalk-client-id "xxx" \
-  --dingtalk-client-secret "xxx" \
-  --dingtalk-agent-id "xxx"
-
-# Lifecycle management
+# Start the service
 npx dialog-survey start
-npx dialog-survey stop
-npx dialog-survey status
-npx dialog-survey uninstall
 
-# View help
-npx dialog-survey help
+# That's it. Your dialog bot is live on DingTalk.
 ```
 
-**Prerequisites**: Node.js >=20, PostgreSQL 14+.
-- **Linux**: [PM2](https://pm2.keymetrics.io/) (`npm install -g pm2`) recommended for production process management
-- **Windows**: PM2 在 Windows 上的稳定性存在问题，CLI 会自动使用直接 `node` 启动（TypeScript 编译必须可用）
+**Prerequisites:** Node.js >= 20.19, PostgreSQL 14+, a DingTalk application (Client ID/Secret/Agent ID).
 
-> For local LLM deployment, leave `--llm-api-key` empty during install;
-> the service will connect to an OpenAI-compatible endpoint at `http://localhost:11434/v1`
-> by default. Set `LLM_BASE_URL` and `LLM_MODEL` in the generated `.env` file if needed.
+---
 
-## Usage
+## Why — 为什么做这个项目？
 
-After installation, use the CLI commands for lifecycle management:
+**One-on-one conversations are great.** People enjoy real dialogue — a skilled facilitator builds rapport, reads between the lines, and asks exactly the right follow-up. But sitting down with hundreds or thousands of people one by one is expensive and doesn't scale.
 
-```bash
-# Start the service
-npx dialog-survey start
+**Surveys scale beautifully**, but they're shallow. A fixed questionnaire can't pivot when a respondent says something unexpected.
 
-# Stop the service
-npx dialog-survey stop
+Dialog Survey fills the gap between these two. It's **deeper than a form, broader than a human-led session.** An AI-powered dialog bot probes interesting responses like a skilled facilitator would, while reaching hundreds of participants in parallel via DingTalk. It won't replace a great human conversation — but when you need structured, first-hand insights from a large group, it's the closest thing without hiring an army of researchers.
 
-# Check service status
-npx dialog-survey status
+---
 
-# Uninstall completely
-npx dialog-survey uninstall
+## What — 这个项目是什么？
 
-# View help
-npx dialog-survey help
-```
+Dialog Survey is an **async survey dialog bot** that lives inside DingTalk. You design a survey template, invite participants, and the AI conducts a multi-turn conversation with each person — asking initial questions, probing deeper on interesting answers, and remembering context across days and messages.
 
-## Quick Start (Development)
+### Key Capabilities
 
-### 1. Install Dependencies
+| Capability | Description |
+|-----------|-------------|
+| **AI-driven follow-ups** | LLM-powered intelligent probing — when a participant says something interesting, the bot digs deeper automatically |
+| **Multi-turn context memory** | Coherent conversations that span days and multiple messages, not isolated Q&A rounds |
+| **Async messaging** | Participants answer in their spare time on DingTalk — no scheduled calls, no pressure |
+| **Auto report generation** | Markdown reports generated automatically after dialogs complete — PDF/Excel export via Playwright |
+| **Self-hosted** | All data stays on your infrastructure. PostgreSQL, your LLM (ollama / vLLM / cloud), your DingTalk app |
 
-```bash
-cd dialog-survey
-npm install
-```
+### What's New (v1.10.x)
 
-### 2. Configure Environment
+- **v1.10.0** — Prisma 6 → 7 with driver adapters. Tests no longer need PostgreSQL: the suite runs on PGlite (isolated WASM database per test file), so `npm test` works with zero manual setup. Node.js requirement raised to >= 20.19.0.
+- **v1.8.1** — Full project doc refresh: compressed AGENTS.md to 88 lines, added subdirectory AGENTS.md for repositories/integrations/utils
+- **v1.8.0** — Removed unused ASR/voice recognition. Fixed COMPLETED dialogs stuck in CONTINUE loop. Fixed maxFollowups not persisted to DB. Fixed add-member modal swallowing errors silently.
+- **v1.7.9** — CI stability: fixed flaky integration tests, cleaned up secrets from sample configs
+- **v1.7.8** — Unified tsconfig.json, fixed 81 type errors across 35 files
 
-```bash
-cp .env.example .env
-# Edit .env with your configuration
-```
+> Full changelog in [CHANGELOG.md](./CHANGELOG.md).
 
-### 3. Initialize Database
+---
 
-```bash
-npx prisma generate
-npx prisma db push
-```
+## How — 怎么使用这个项目？
 
-### 4. Start Service
+### Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| **Conversation Engine** | Custom LangGraph-inspired workflow (not StateGraph API) |
+| **LLM** | OpenAI-compatible API — ollama / vLLM / LocalAI / cloud |
+| **Messaging** | DingTalk Stream Mode (WebSocket) |
+| **Database** | PostgreSQL + Prisma 7 (driver adapters) |
+| **Web Framework** | Fastify 5.x |
+| **Templates** | Nunjucks (admin UI) |
+| **Language** | TypeScript (strict mode, ESM) |
+
+---
+
+### Installation
+
+#### Option A: Quick CLI (recommended for production)
 
 ```bash
-npm run dev
+# Interactive
+npx dialog-survey install
+
+# Non-interactive (CI / automation)
+npx dialog-survey install \
+  --db-url "postgresql://user:pass@localhost:5432/dialog_survey" \
+  --dingtalk-client-id "xxx" \
+  --dingtalk-client-secret "xxx" \
+  --dingtalk-agent-id "xxx"
 ```
 
-## 项目结构
+This generates `.env`, applies database migrations with `prisma migrate deploy` (Prisma 7), and sets up PM2 (Linux/macOS) or direct node launch (Windows).
 
-```
-dialog-survey/
-├── src/
-│   ├── api/          # API 层 (Fastify routes)
-│   ├── core/         # 核心逻辑 (LangGraph StateGraph)
-│   ├── services/     # 服务层 (LLM, ASR, 钉钉)
-│   ├── repositories/ # 数据访问层 (Prisma)
-│   ├── integrations/ # 外部服务集成
-│   ├── domains/      # 领域实体
-│   ├── middleware/   # Fastify 中间件
-│   ├── views/        # Nunjucks 模板
-│   └── utils/        # 工具函数
-├── tests/            # 测试 (Vitest)
-├── scripts/          # CLI 脚本和部署脚本
-├── prisma/           # Prisma schema 和迁移
-├── templates/        # 访谈模板
-├── reports/          # 生成的报告
-├── package.json
-├── tsconfig.json
-└── vitest.config.ts
+#### Option B: Docker Compose (recommended for evaluation)
+
+```bash
+cp .env.example .env
+# Fill in your DingTalk + LLM config
+docker compose up -d
+curl http://localhost:3001/health
 ```
 
-## 开发命令
+#### Option C: Manual setup (development)
 
 ```bash
-# 类型检查
-npm run type-check
-
-# 运行测试
-npm test
+git clone https://github.com/boyingliu01/dialog-survey
+cd dialog-survey
+npm install
+cp .env.example .env
+# Edit .env with your config
+npx prisma generate && npx prisma migrate dev
+npm run dev
+```
 
-# 测试覆盖率
-npm run test:coverage
+---
 
-# 代码检查
-npm run lint
+### Lifecycle Management
 
-# 构建
-npm run build
+```bash
+npx dialog-survey start     # Start service
+npx dialog-survey stop      # Stop service
+npx dialog-survey status    # Check status
+npx dialog-survey uninstall # Remove completely (data preserved)
+npx dialog-survey help      # View all commands
 ```
 
-## 部署
+---
 
-### 环境要求
+### Configuration
 
-| 组件 | 版本/说明 |
-|------|-----------|
-| Node.js | >= 20.0.0 (推荐 v20 LTS) |
-| PostgreSQL | 14+ |
-| npm | 随 Node.js 自带 |
-| 操作系统 | Linux / macOS / Windows |
+Key environment variables (see `.env.example` for the full list):
 
-### 部署步骤
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `DATABASE_URL` | PostgreSQL connection string | `postgresql://user:pass@localhost:5432/dialog_survey` |
+| `LLM_BASE_URL` | LLM endpoint (OpenAI-compatible) | `http://localhost:11434/v1` |
+| `LLM_MODEL` | Model name | `qwen2.5` |
+| `DINGTALK_CLIENT_ID` | DingTalk Client ID | From DingTalk Open Platform |
+| `DINGTALK_CLIENT_SECRET` | DingTalk Client Secret | From DingTalk Open Platform |
+| `DINGTALK_AGENT_ID` | DingTalk Agent ID | From DingTalk Open Platform |
+| `DISABLE_STARTUP_RESEND` | Skip the startup resend of unsent messages (`1`/`true`/`yes`) | `1` |
+| `ADMIN_API_KEY` | Admin panel API key | Your custom key |
 
-#### 1. 克隆或复制项目
+> Leave `LLM_API_KEY` empty to use a local LLM at `http://localhost:11434/v1`.
+> Without `DINGTALK_CLIENT_ID`/`DINGTALK_CLIENT_SECRET` the server boots read-only (no Stream connection, no outbound messages). Startup resend additionally requires `DINGTALK_AGENT_ID`.
 
-```bash
-# 推荐：从 Git 仓库克隆
-git clone <repo-url> dialog-survey
-cd dialog-survey
+---
 
-# 或：直接拷贝项目目录到目标机器
-```
+### Admin Authentication
 
-#### 2. 安装依赖
+The admin UI requires authentication. Configure the following environment variables:
 
 ```bash
-npm install
+# Admin credentials
+ADMIN_USERNAME=admin
+ADMIN_PASSWORD_HASH=<bcrypt-hash>  # Generate with bcrypt
+
+# Session configuration
+SESSION_SECRET=<64-char-hex>     # 32 random bytes — hex string of at least 32 characters
+SESSION_SALT=<32-char-hex>       # exactly 16 random bytes — must be 32 hex chars or startup fails
+SESSION_MAX_AGE=28800  # 8 hours in seconds
 ```
 
-#### 3. 配置环境变量
+#### Generate Password Hash
 
 ```bash
-cp .env.example .env
-# 编辑 .env，填入实际配置值（见下方环境变量说明）
+node -e "const bcrypt = require('bcryptjs'); bcrypt.hash('your-password', 12).then(console.log)"
 ```
 
-#### 4. 初始化数据库
+Copy the output hash and set it as `ADMIN_PASSWORD_HASH`.
 
-```bash
-npx prisma generate   # 生成 Prisma Client
-npx prisma db push     # 同步 schema 到数据库
-```
+#### Rotate the Admin Password
 
-#### 5. 启动服务
+Password changes are deployment-managed. There is no online password-change route. To rotate the password:
 
-**开发模式** (热重载):
-```bash
-npm run dev
-# 默认监听 http://0.0.0.0:3001
-```
+1. Generate a new bcrypt hash with the command above.
+2. Update `ADMIN_PASSWORD_HASH` in the deployment secret or configuration. Do not commit the password or hash to the repository.
+3. Restart or roll the application so it loads the updated configuration.
+4. Verify that the new password logs in at `/admin` and that the old password no longer works.
 
-**生产模式** (推荐用于长期运行):
+The application reads its configuration at runtime and never writes `.env` files.
 
-**Linux / macOS** — 使用 PM2 管理进程：
-```bash
-npm install -g pm2
-npm run build   # 编译到 dist/
-pm2 start dist/src/server.ts --name dialog-survey
-pm2 save
-pm2 startup  # 设置开机自启
-```
+#### Generate Session Secret and Salt
 
-**Windows** — 直接使用 Node 启动（PM2 在 Windows 上不稳定）：
 ```bash
-npm run build   # 编译到 dist/
-node dist/src/server.ts
-```
+# Generate SESSION_SECRET (32 bytes)
+node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
 
-### 环境变量说明
-
-| 变量名 | 说明 | 示例 |
-|--------|------|------|
-| `NODE_ENV` | 运行环境 | `production` |
-| `PORT` | 服务端口 | `3001` |
-| `HOST` | 监听地址 | `0.0.0.0` |
-| `DATABASE_URL` | PostgreSQL 连接串 | `postgresql://user:pass@localhost:5432/dialog_survey?schema=public` |
-| `LLM_BASE_URL` | LLM 服务地址（OpenAI-compatible），指向本地或云服务 | `http://localhost:11434/v1` |
-| `LLM_API_KEY` | LLM API Key | `sk-xxx` |
-| `LLM_MODEL` | LLM 模型名称 | `qwen2.5` |
-| `DINGTALK_CLIENT_ID` | 钉钉 Client ID | 开放平台获取 |
-| `DINGTALK_CLIENT_SECRET` | 钉钉 Client Secret | 开放平台获取 |
-| `DINGTALK_AGENT_ID` | 钉钉 Agent ID | 开放平台获取 |
-| `DINGTALK_APP_KEY` | 钉钉 AppKey | 开放平台获取 |
-| `DINGTALK_APP_SECRET` | 钉钉 AppSecret | 开放平台获取 |
-| `PUBLIC_URL` | 公网回调地址 | `https://your-domain.com` |
-| `FUN_ASR_API_KEY` | 阿里云 Fun-ASR API Key | 语音识别用 |
-| `ENCRYPTION_KEY` | AES 加密密钥 | 32 字节 hex 字符串 |
-| `ADMIN_API_KEY` | 管理后台 API Key | 自定义密钥 |
-| `REPORTS_DIR` | 报告存储目录 | `./reports` |
-| `LOG_LEVEL` | 日志级别 | `info` |
-
-安全起见，`ENCRYPTION_KEY` 可通过 Node.js 生成:
-```bash
+# Generate SESSION_SALT (16 bytes)
 node -e "console.log(require('crypto').randomBytes(16).toString('hex'))"
 ```
 
-### 健康检查
+#### Access the Admin UI
+
+1. Navigate to `http://localhost:3001/admin`
+2. Enter your username and password
+3. You will be redirected to the admin dashboard
+
+#### Session Management
+
+- Sessions expire after 8 hours of inactivity (configurable via `SESSION_MAX_AGE`)
+- Session data is stored in encrypted cookies (no server-side storage)
+- To log out, submit the logout form, which sends a `POST` request to `/admin/logout`; clearing the browser cookie also removes the session from that browser
+- Because sessions are stateless encrypted cookies, logout removes the browser's cookie but cannot revoke a copied old cookie without a server-side session store. A copied cookie still expires after the inactivity limit
+
+#### API Key Backward Compatibility
+
+The `ADMIN_API_KEY` environment variable is optional. Configure it only when automation or an existing integration needs header-based admin access. API access via the `X-Admin-Key` header is supported for those scripts and integrations:
 
-服务启动后可访问:
 ```bash
-curl http://localhost:3001/health
+curl -H "X-Admin-Key: <admin-api-key>" http://localhost:3001/admin
 ```
 
-### Docker 部署 (推荐)
+#### Security Features
 
-使用 Docker Compose 一键部署应用和 PostgreSQL 数据库。
+- **CSRF Protection**: All form submissions include CSRF tokens automatically
+- **Session Timeout**: Inactive sessions expire after 8 hours
+- **Encrypted Cookies**: Session data is encrypted and signed
+- **Password Hashing**: Passwords are hashed using bcrypt (cost factor 12)
 
-#### 1. 配置环境变量
+---
 
+### Production Deployment
+
+**Linux / macOS** (PM2):
 ```bash
-cp .env.example .env
-# 编辑 .env，填入 DingTalk、LLM、ASR 等实际配置值
+npm run build
+pm2 start dist/src/server.js --name dialog-survey
+pm2 save && pm2 startup
 ```
 
-#### 2. 启动服务
+**Windows** (direct node):
+```bash
+npm run build
+node dist/src/server.js
+```
 
+**Docker**:
 ```bash
-# 构建并启动所有服务
-docker-compose up -d
+docker compose up -d
+docker compose logs -f app
+```
 
-# 查看日志
-docker-compose logs -f app
+Health check: `curl http://localhost:3001/health`
 
-# 查看所有服务状态
-docker-compose ps
-```
+---
 
-#### 3. 验证部署
+### Development Commands
 
-服务启动后，访问健康检查端点：
 ```bash
-curl http://localhost:3001/health
+npx prisma generate  # Generate Prisma client (required after checkout)
+npm run dev          # Hot-reload dev server on :3001
+npm run type-check   # TypeScript check (tsc --noEmit)
+npm test             # Vitest watch; npx vitest run for one-shot — no PostgreSQL needed (PGlite)
+npm run lint         # Biome lint
+npm run build        # Compile to dist/
+npm run smoke        # Quick sanity check (type-check + lint + key tests)
 ```
 
-#### 常用 Docker 命令
+---
 
-```bash
-# 停止服务
-docker-compose down
+### Project Structure
+
+```
+dialog-survey/
+├── src/
+│   ├── api/          # Fastify routes
+│   ├── core/         # Graph workflow engine
+│   ├── services/     # Business logic (18 services)
+│   ├── repositories/ # Data access (Prisma)
+│   ├── integrations/ # DingTalk + LLM clients
+│   ├── middleware/   # Fastify middleware
+│   ├── views/        # Nunjucks admin UI (HTMX + Alpine.js)
+│   ├── generated/    # Prisma client output (gitignored; npx prisma generate)
+│   └── utils/        # Helpers (logger, security, PII, etc.) + Prisma facade/factory
+├── tests/            # 117 files, ~1266 tests (Vitest + PGlite, no PostgreSQL needed)
+├── scripts/          # CLI binary + deploy scripts
+├── prisma/           # Schema + migrations + seeds
+└── docs/             # Architecture & design docs
+```
 
-# 重启服务
-docker-compose restart
+---
 
-# 查看数据库日志
-docker-compose logs -f postgres
+### Further Reading
 
-# 进入数据库 shell
-docker-compose exec postgres psql -U dialog_survey -d dialog_survey
+- [DEPLOY.md](./DEPLOY.md) — Detailed deployment guide
+- [AGENTS.md](./AGENTS.md) — Codebase map for AI agents
+- [CHANGELOG.md](./CHANGELOG.md) — Full release history
+- [Architecture](./architecture.yaml) — Architecture validation rules
 
-# 重新构建应用镜像
-docker-compose build --no-cache app
-docker-compose up -d app
-```
+---
 
-### 预提交检查 (开发时)
+## License
 
-```bash
-npm run type-check && npm run lint && npm run test:coverage
-```
+MIT