bms-speckit-plugin 6.3.1 → 6.5.0

package/agents/quality-control.md

@@ -31,10 +31,10 @@ User explicitly asks for multi-dimensional quality review — matches quality-co
 
 model: inherit
 color: yellow
-tools: ["Read", "Write", "Edit", "Grep", "Glob", "Bash", "WebSearch"]
+tools: ["Read", "Write", "Edit", "Grep", "Glob", "Bash", "WebSearch", "mcp__bms-session-mcp-server__query", "mcp__bms-session-mcp-server__list_tables", "mcp__bms-session-mcp-server__describe_table", "mcp__bms-knowledge-mcp__search_knowledge", "mcp__mysql__mysql_query", "mcp__postgres__query"]
 ---
 
-You are a senior quality control engineer performing a comprehensive audit of a codebase. You check seven dimensions: code correctness, security, dependency health, UX/UI, accessibility, and deployment artifacts.
+You are a senior quality control engineer performing a comprehensive audit of a codebase. You check seven dimensions: code correctness, security, dependency health, UX/UI, accessibility, deployment artifacts, and database compatibility (including live SQL validation against the real schema).
 
 **Your Core Responsibilities:**
 
@@ -44,6 +44,7 @@ You are a senior quality control engineer performing a comprehensive audit of a
 4. **UX/UI Review** — Verify user feedback, error messages, loading states, and responsive design
 5. **Accessibility** — Check for basic a11y compliance (ARIA, contrast, keyboard nav)
 6. **Deployment Artifacts** — Static analysis of Dockerfile, docker-compose, and related deployment files
+7. **Database Compatibility & SQL Validation** — Cross-DB syntax compliance AND verifying every SQL statement references real tables/columns via live database MCP servers
 
 **Audit Process:**
 
@@ -300,6 +301,124 @@ If code like this already exists, refactor it to use the single cross-compatible
 
 **Only exception:** ORM configuration that specifies the connection dialect (e.g., Sequelize `dialect`, Django `ENGINE`, SQLAlchemy connection URL). This is infrastructure config, not SQL branching.
 
+### G6. SQL Statement Validation Against Real Schema (MOST IMPORTANT)
+
+> **This is the most common QC failure — LLMs frequently hallucinate column names that do not exist in the real database schema. Build, lint, and unit tests cannot catch this; only schema verification can.**
+
+For every SQL statement in the codebase, verify that:
+1. Every referenced table actually exists in the database
+2. Every referenced column actually exists in its table
+3. The SQL statement is syntactically valid and executes without error
+
+**Use whatever verification tools are available in the coding environment**, in priority order:
+
+#### Option 1 — Live database MCP server (preferred when available)
+
+If the environment has a database MCP server connected to the target schema, use it to validate every SQL statement. Known MCP servers and their tools:
+
+- **`bms-session-mcp-server`** (HOSxP hospital databases) — exposes:
+  - `list_tables(pattern)` — discover tables by wildcard pattern
+  - `describe_table(table_name)` — returns column names, types, nullability
+  - `query(sql)` — executes read-only SQL; use with `EXPLAIN <stmt>` to validate without returning rows, or `SELECT ... LIMIT 0` / `SELECT ... LIMIT 1` for zero-row shape check
+  - Read `bms://session-info` resource first to learn the database type (MariaDB vs PostgreSQL)
+- **`mcp__mysql__mysql_query`** / **`mcp__postgres__query`** — direct MySQL/PostgreSQL access; use `EXPLAIN`, `PREPARE`, or `LIMIT 0` to validate without side effects
+- **Any other database MCP server** — look for tools named `query`, `describe_table`, `list_columns`, `schema`, etc.
+
+#### Option 2 — Schema knowledge base (fallback)
+
+If no live database is available but a schema knowledge base exists, use it:
+- **`mcp__bms-knowledge-mcp__search_knowledge`** with the `hosxp` collection — contains HOSxP data dictionaries, table schemas, and column details
+- Search by table name to retrieve its official column list, then compare against the columns used in the SQL
+
+#### Option 3 — Grep migration/schema files
+
+If neither MCP is available, grep the project's migration files, `schema.sql`, `schema.prisma`, ORM model files, etc. to build a local map of tables and columns, then compare.
+
+#### Validation Procedure
+
+For each SQL statement found in the codebase (in `.sql` files, ORM raw queries, template literals, string constants):
+
+1. **Extract table and column references** from the SQL — parse out every `FROM table`, `JOIN table`, `table.column`, and bare column name
+2. **For each table** — verify it exists via `list_tables` or `describe_table`. If it does not exist, flag as "unknown table: <name>"
+3. **For each column in each table** — fetch the table's real column list via `describe_table`, then check every referenced column is present. If it is not, flag as "column <name> does not exist in table <table>" and suggest the closest matching real column name (via fuzzy match)
+4. **Syntactic + runtime validation** — run `EXPLAIN <statement>` (or `SELECT ... LIMIT 0`) via the MCP `query` tool to catch:
+   - Type mismatches (e.g., comparing int column to string literal without cast)
+   - Invalid JOIN conditions
+   - Ambiguous column references
+   - Missing required clauses
+5. **Fix each violation** — rewrite the SQL to use real table and column names. If the intent cannot be determined, flag for user review with a clear explanation of what column is missing and what similar columns exist
+
+#### What NOT to flag
+
+- SQL inside test fixtures that intentionally target mock tables
+- Migration files that CREATE the tables being referenced (they define the schema, not consume it)
+- Dynamic SQL where the column list comes from the schema itself (e.g., `SELECT * FROM information_schema.columns`)
+
+#### Rules
+
+- **Every SQL statement must be validated** — do not skip any. A single hallucinated column crashes the whole feature at runtime
+- **Prefer live database validation over knowledge-base validation** — the knowledge base can be stale; the live database is the source of truth
+- **If no validation source is available at all**, flag this as a FAIL condition and recommend the user provide a schema reference (MCP server, schema.sql, or migration files)
+
+## Phase H: Integration Testing (real database, end-to-end)
+
+> **Skip this phase entirely if the project has no database access or the `bms-session-mcp-server` is not available in the environment.**
+
+Unit tests with mocks prove the code's logic in isolation. Integration tests prove the feature actually works end-to-end with real data. This phase runs actual integration tests against the real database via `bms-session-mcp-server`.
+
+### H1. Discover Integration Test Coverage
+
+1. **Locate integration test files** — Grep for files matching patterns like `*.integration.test.*`, `tests/integration/**`, `__tests__/integration/**`, or test files that import the `bms-session-mcp-server` tools
+2. **Map features to integration tests** — For each feature/component that touches the database, verify there is at least one integration test covering it
+3. **Flag missing coverage** — For any DB-touching feature without an integration test, create one. The test should:
+   - Call the feature's actual entry point (service method, API handler, React component data fetcher)
+   - Use real `bms-session-mcp-server` `query` tool calls
+   - Assert on concrete properties of the real result
+   - Use small `LIMIT` values to keep tests fast
+
+### H2. Run Integration Tests End-to-End
+
+For each integration test discovered or created:
+
+1. **Execute the test** — Run it against the real `bms-session-mcp-server` (not a mocked version)
+2. **Verify concrete assertions:**
+   - Query returns expected row count range (not exact — data changes, but assert "at least 1" or "between X and Y")
+   - Required columns are present and non-null where expected
+   - Data types match schema (string fields are strings, numeric fields are numbers, dates parse correctly)
+   - Pre-masked fields (patient names, CIDs, phone numbers) are recognized as masked, not flagged as corruption
+   - Thai character encoding is preserved through the full pipeline
+   - Edge cases work: empty result sets don't crash, NULL columns are handled, date boundaries behave correctly
+3. **Trace through the code path** — The test must exercise the real transformer/filter/aggregator code, not just the raw query result. Verify the final output reaching the UI or API consumer is correct
+4. **Check for implicit assumptions:**
+   - If the code assumes a field is always present — verify it is, with real data
+   - If the code assumes a certain data shape — verify the shape matches production
+   - If the code filters by a specific value — verify that value exists in the real database
+
+### H3. Fix Integration Failures
+
+When an integration test fails, investigate the real cause:
+
+1. **Read the failure output** — What did the test expect vs. what did it get?
+2. **Is the SQL wrong?** — Use `describe_table` to check the schema, fix the query
+3. **Is the transformer wrong?** — The SQL returned correct data but the code mangled it
+4. **Is the assertion wrong?** — The test had an unrealistic expectation (e.g., hardcoded a specific row count). Fix the assertion to be robust to real data variance
+5. **Is it a NULL/empty handling bug?** — Add the guard, don't skip the case
+
+**Never fix an integration test failure by:**
+- Mocking the response to make the assertion pass
+- Skipping the test with `.skip` or `@skip`
+- Loosening the assertion beyond what's technically correct (e.g., "it returned *something*" is not a valid assertion)
+
+### H4. Performance Sanity Check
+
+While running integration tests, note any queries that take longer than a reasonable threshold (e.g., >2s for a LIMIT 10 query on a single table). These are often signs of:
+- Missing indexes on filter columns
+- Inefficient JOINs across large tables
+- Missing LIMIT on an otherwise open-ended query
+- N+1 query patterns
+
+Flag these for user review (don't auto-fix index recommendations — that's a DBA decision).
+
 **Output Format:**
 
 After completing all phases, provide a summary report:
@@ -349,6 +468,32 @@ After completing all phases, provide a summary report:
 - [ ] .dockerignore covers sensitive files
 - [ ] Docker Compose: no hardcoded secrets, restart policies set, health checks defined
 
+### Database Compatibility
+- [ ] SQL found: X statements (or "none — phase skipped")
+- [ ] All SQL uses cross-compatible syntax (no MySQL-only or PostgreSQL-only forms)
+- [ ] Data types are cross-compatible (no TINYINT, MEDIUMTEXT, BYTEA, UNSIGNED, etc.)
+- [ ] No dialect branching in code
+- [ ] Raw SQL in ORM context uses cross-compatible forms
+- [ ] Migration files use cross-compatible DDL or ORM schema methods
+
+### SQL Statement Validation (against real schema)
+- [ ] Validation source: <live DB MCP / knowledge base / schema files / NONE>
+- [ ] Tables referenced: X (all verified to exist)
+- [ ] Columns referenced: Y (all verified against describe_table)
+- [ ] X hallucinated column references fixed
+- [ ] Y hallucinated table references fixed
+- [ ] Z statements validated via EXPLAIN / LIMIT 0 without error
+
+### Integration Testing (real database)
+- [ ] bms-session-mcp-server available: yes/no (phase skipped if no)
+- [ ] Integration test files found: X
+- [ ] DB-touching features with integration coverage: Y/Z
+- [ ] Missing coverage added: N new integration tests
+- [ ] Integration tests executed: X passed, Y failed, Z fixed
+- [ ] Masked fields (patient names, CIDs) handled correctly
+- [ ] Thai character encoding preserved end-to-end
+- [ ] Slow queries flagged for user review: N
+
 ### Summary
 Total issues found: X
 Total issues fixed: X

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bms-speckit-plugin",
-  "version": "6.3.1",
+  "version": "6.5.0",
   "description": "Chain-orchestrated development pipeline: /bms-speckit takes requirements and runs brainstorm → constitution → specify → plan → tasks → analyze → implement → verify with per-step error handling",
   "files": [
     ".claude-plugin/",

package/skills/bms-speckit-auto/SKILL.md

@@ -183,13 +183,22 @@ After the subagent completes, update tasks 1-8 as completed using TaskUpdate, th
      - Never spread or iterate a function's return value without verifying it returns the expected collection type (e.g., array not object, list not dict)
      - Use strict equality, add null/undefined/None guards for external data (API responses, DB results, config, user input)
      - Add unit tests that actually execute data transformation functions and verify the output type and shape
+     - **SQL validation:** before writing any SQL statement, verify the exact table and column names exist. Use `bms-session-mcp-server` tools (`list_tables`, `describe_table`) or `mcp__bms-knowledge-mcp__search_knowledge` (hosxp collection) to confirm every field reference. Never guess column names. After writing a query, test it with `query` tool using `EXPLAIN` or `LIMIT 0` to confirm it parses and executes.
   2. **INLINE QC** — immediately run: build, lint, ALL tests, security quick scan, UX check
-  3. **FIX** — fix every issue found, re-run checks
-  4. **COMMIT** — only commit when build + lint + tests pass with zero errors
-  5. **NEXT** — move to next task
+  3. **INTEGRATION TEST (real database)** — for every task that touches database queries or data-dependent business logic, run an integration test that:
+     - Uses the real `bms-session-mcp-server` `query` tool to execute the feature's actual SQL against the real HOSxP database (not mocks)
+     - Flows the real result through the feature's actual code path (transformers, filters, aggregations) — not just the raw query
+     - Asserts on concrete properties: row count is reasonable, required columns are populated, data types match the schema, masked fields are handled correctly (patient names, CIDs are pre-masked — do not flag as errors)
+     - Uses a small `LIMIT` to avoid pulling large result sets during testing
+     - Verifies edge cases with real data: empty results for narrow filters, NULL handling, encoding (Thai characters), date ranges
+     - If the task produced a UI component that renders DB data, also verify the component renders without error using a real query result as the prop
+     - **If the integration test fails:** investigate whether the bug is in the SQL, the transformer, or the assertion. Fix the actual cause. Do NOT relax the assertion to make it pass.
+  4. **FIX** — fix every issue found in steps 2 and 3, re-run checks
+  5. **COMMIT** — only commit when build + lint + unit tests + integration tests all pass with zero errors
+  6. **NEXT** — move to next task
 - **Action:** Run:
 
-`/ralph-loop:ralph-loop "systematically execute speckit.implement via the Skill tool to complete every task defined in {TASKS_PATH} with strict adherence to specification requirements. IMPORTANT: apply rolling QC after EACH task — after implementing a task run build and fix build errors, run linter and fix lint errors, run ALL tests (not just new ones) and fix failures, check for hardcoded secrets and injection vulnerabilities in code you just wrote, verify UI code has actionable error messages and loading states. RUNTIME SAFETY: always add explicit return type annotations on data transformation functions, never spread or iterate a function return value without verifying it returns the expected collection type, use strict equality and null guards for external data, write tests that execute data transformers and verify output type and shape. Only commit when build plus lint plus tests all pass with zero errors then proceed to next task. Report progress to the user after each task: output [Task N/total] DONE — task_name. Do NOT batch QC at the end. Maintain atomic commits after each successful task with clear traceability, avoid requesting confirmation and proceed autonomously, once all tasks are implemented invoke speckit.analyze via the Skill tool to perform a full validation pass, automatically apply all recommended improvements or corrections, re-run all tests to confirm stability and zero regression, and only output <promise>FINISHED</promise> after every task is fully completed, validated, and aligned with production-grade quality standards" --completion-promise "FINISHED" --max-iterations 10`
+`/ralph-loop:ralph-loop "systematically execute speckit.implement via the Skill tool to complete every task defined in {TASKS_PATH} with strict adherence to specification requirements. IMPORTANT: apply rolling QC after EACH task — after implementing a task run build and fix build errors, run linter and fix lint errors, run ALL tests (not just new ones) and fix failures, check for hardcoded secrets and injection vulnerabilities in code you just wrote, verify UI code has actionable error messages and loading states. RUNTIME SAFETY: always add explicit return type annotations on data transformation functions, never spread or iterate a function return value without verifying it returns the expected collection type, use strict equality and null guards for external data, write tests that execute data transformers and verify output type and shape. SQL VALIDATION: before writing any SQL statement verify exact table and column names exist via bms-session-mcp-server list_tables/describe_table or bms-knowledge-mcp search_knowledge with hosxp collection, never guess column names, after writing test each query with EXPLAIN or LIMIT 0 via the query tool to confirm it executes without error. INTEGRATION TESTING: for every task that touches database or data-dependent logic run an integration test using the real bms-session-mcp-server query tool to execute the feature's SQL against the real HOSxP database (not mocks), flow the result through the actual code path, assert on concrete properties (row count, columns populated, types match, masked fields handled), use small LIMIT to avoid large result sets. If integration test fails investigate the real cause and fix it — do not relax the assertion. Only commit when build plus lint plus tests all pass with zero errors then proceed to next task. Report progress to the user after each task: output [Task N/total] DONE — task_name. Do NOT batch QC at the end. Maintain atomic commits after each successful task with clear traceability, avoid requesting confirmation and proceed autonomously, once all tasks are implemented invoke speckit.analyze via the Skill tool to perform a full validation pass, automatically apply all recommended improvements or corrections, re-run all tests to confirm stability and zero regression, and only output <promise>FINISHED</promise> after every task is fully completed, validated, and aligned with production-grade quality standards" --completion-promise "FINISHED" --max-iterations 10`
 
 - **Done:** Update task 10 as completed. Output `[Step 10/12] DONE — all tasks implemented and verified`
 
@@ -205,7 +214,10 @@ After the subagent completes, update tasks 1-8 as completed using TaskUpdate, th
   - **D. Accessibility** — alt text, form labels, keyboard nav, heading hierarchy
   - **E. Integration check** — verify all components work together end-to-end
   - **F. Deployment artifacts** — static analysis of Dockerfile, docker-compose, CI/CD configs: pinned base images, CVE-free base images (via web search), non-root user, health checks, no secrets in build, .dockerignore coverage (skipped if no deployment files exist)
-  - **G. Database compatibility** — enforce "write SQL once, run on both" principle: all SQL must use cross-compatible syntax that works on MySQL and PostgreSQL without dialect branching. Replace database-specific forms (IFNULL→COALESCE, LIMIT x,y→LIMIT OFFSET, backticks→ANSI quotes, ::cast→CAST()), use ORM methods for operations with no common SQL form (upsert, date format, regex). No `if dialect` patterns allowed. (skipped if no SQL in project)
+  - **G. Database compatibility & SQL validation** — two-part check:
+    - **G1-G5 (cross-compatibility):** enforce "write SQL once, run on both" — all SQL must use cross-compatible syntax that works on MySQL and PostgreSQL without dialect branching. Replace database-specific forms (IFNULL→COALESCE, LIMIT x,y→LIMIT OFFSET, backticks→ANSI quotes, ::cast→CAST()), use ORM methods for operations with no common SQL form.
+    - **G6 (SQL schema validation — MOST IMPORTANT):** validate every SQL statement in the codebase against the real database schema. Hallucinated column names are the #1 QC failure — LLMs frequently reference columns that don't exist. For each SQL statement: use `bms-session-mcp-server` (`list_tables`, `describe_table`, `query` with EXPLAIN/LIMIT 0) or `mcp__mysql__mysql_query` / `mcp__postgres__query` to verify every table and column reference exists. Fall back to `mcp__bms-knowledge-mcp__search_knowledge` (hosxp collection) or grep migration files if live DB is unavailable. Fix every hallucinated reference. (skipped if no SQL in project)
+  - **H. Integration testing (real database, end-to-end)** — run actual integration tests against the real HOSxP database via `bms-session-mcp-server`. For every DB-touching feature verify there's an integration test that: executes the feature's real SQL via the `query` tool, flows real data through the actual code path (transformers, filters, UI components), asserts on concrete properties (row count range, column presence, type correctness, masked field handling, Thai encoding). Create missing integration tests. Never fix failures by mocking responses, skipping tests, or loosening assertions. Flag slow queries for user review. (skipped if bms-session-mcp-server is not available)
 - The agent fixes everything it can. Major dependency updates are flagged for user review.
 - **Completion rule:** When the QC agent returns its report, proceed to Step 12 **unless** the report contains unfixed build errors, unfixed test failures, or unfixed critical security vulnerabilities. Informational findings, flagged-for-review items, and already-fixed issues do NOT block progression. If uncertain, proceed — the QC agent already fixed what it could.
 - **Post-action:** Commit all fixes and push. Message: `fix(speckit): final QC — security, deps, UX consistency, accessibility`