bms-speckit-plugin 6.4.0 → 6.5.0

package/agents/quality-control.md

@@ -360,6 +360,65 @@ For each SQL statement found in the codebase (in `.sql` files, ORM raw queries,
 - **Prefer live database validation over knowledge-base validation** — the knowledge base can be stale; the live database is the source of truth
 - **If no validation source is available at all**, flag this as a FAIL condition and recommend the user provide a schema reference (MCP server, schema.sql, or migration files)
 
+## Phase H: Integration Testing (real database, end-to-end)
+
+> **Skip this phase entirely if the project has no database access or the `bms-session-mcp-server` is not available in the environment.**
+
+Unit tests with mocks prove the code's logic in isolation. Integration tests prove the feature actually works end-to-end with real data. This phase runs actual integration tests against the real database via `bms-session-mcp-server`.
+
+### H1. Discover Integration Test Coverage
+
+1. **Locate integration test files** — Grep for files matching patterns like `*.integration.test.*`, `tests/integration/**`, `__tests__/integration/**`, or test files that import the `bms-session-mcp-server` tools
+2. **Map features to integration tests** — For each feature/component that touches the database, verify there is at least one integration test covering it
+3. **Flag missing coverage** — For any DB-touching feature without an integration test, create one. The test should:
+   - Call the feature's actual entry point (service method, API handler, React component data fetcher)
+   - Use real `bms-session-mcp-server` `query` tool calls
+   - Assert on concrete properties of the real result
+   - Use small `LIMIT` values to keep tests fast
+
+### H2. Run Integration Tests End-to-End
+
+For each integration test discovered or created:
+
+1. **Execute the test** — Run it against the real `bms-session-mcp-server` (not a mocked version)
+2. **Verify concrete assertions:**
+   - Query returns expected row count range (not exact — data changes, but assert "at least 1" or "between X and Y")
+   - Required columns are present and non-null where expected
+   - Data types match schema (string fields are strings, numeric fields are numbers, dates parse correctly)
+   - Pre-masked fields (patient names, CIDs, phone numbers) are recognized as masked, not flagged as corruption
+   - Thai character encoding is preserved through the full pipeline
+   - Edge cases work: empty result sets don't crash, NULL columns are handled, date boundaries behave correctly
+3. **Trace through the code path** — The test must exercise the real transformer/filter/aggregator code, not just the raw query result. Verify the final output reaching the UI or API consumer is correct
+4. **Check for implicit assumptions:**
+   - If the code assumes a field is always present — verify it is, with real data
+   - If the code assumes a certain data shape — verify the shape matches production
+   - If the code filters by a specific value — verify that value exists in the real database
+
+### H3. Fix Integration Failures
+
+When an integration test fails, investigate the real cause:
+
+1. **Read the failure output** — What did the test expect vs. what did it get?
+2. **Is the SQL wrong?** — Use `describe_table` to check the schema, fix the query
+3. **Is the transformer wrong?** — The SQL returned correct data but the code mangled it
+4. **Is the assertion wrong?** — The test had an unrealistic expectation (e.g., hardcoded a specific row count). Fix the assertion to be robust to real data variance
+5. **Is it a NULL/empty handling bug?** — Add the guard, don't skip the case
+
+**Never fix an integration test failure by:**
+- Mocking the response to make the assertion pass
+- Skipping the test with `.skip` or `@skip`
+- Loosening the assertion beyond what's technically correct (e.g., "it returned *something*" is not a valid assertion)
+
+### H4. Performance Sanity Check
+
+While running integration tests, note any queries that take longer than a reasonable threshold (e.g., >2s for a LIMIT 10 query on a single table). These are often signs of:
+- Missing indexes on filter columns
+- Inefficient JOINs across large tables
+- Missing LIMIT on an otherwise open-ended query
+- N+1 query patterns
+
+Flag these for user review (don't auto-fix index recommendations — that's a DBA decision).
+
 **Output Format:**
 
 After completing all phases, provide a summary report:
@@ -409,6 +468,32 @@ After completing all phases, provide a summary report:
 - [ ] .dockerignore covers sensitive files
 - [ ] Docker Compose: no hardcoded secrets, restart policies set, health checks defined
 
+### Database Compatibility
+- [ ] SQL found: X statements (or "none — phase skipped")
+- [ ] All SQL uses cross-compatible syntax (no MySQL-only or PostgreSQL-only forms)
+- [ ] Data types are cross-compatible (no TINYINT, MEDIUMTEXT, BYTEA, UNSIGNED, etc.)
+- [ ] No dialect branching in code
+- [ ] Raw SQL in ORM context uses cross-compatible forms
+- [ ] Migration files use cross-compatible DDL or ORM schema methods
+
+### SQL Statement Validation (against real schema)
+- [ ] Validation source: <live DB MCP / knowledge base / schema files / NONE>
+- [ ] Tables referenced: X (all verified to exist)
+- [ ] Columns referenced: Y (all verified against describe_table)
+- [ ] X hallucinated column references fixed
+- [ ] Y hallucinated table references fixed
+- [ ] Z statements validated via EXPLAIN / LIMIT 0 without error
+
+### Integration Testing (real database)
+- [ ] bms-session-mcp-server available: yes/no (phase skipped if no)
+- [ ] Integration test files found: X
+- [ ] DB-touching features with integration coverage: Y/Z
+- [ ] Missing coverage added: N new integration tests
+- [ ] Integration tests executed: X passed, Y failed, Z fixed
+- [ ] Masked fields (patient names, CIDs) handled correctly
+- [ ] Thai character encoding preserved end-to-end
+- [ ] Slow queries flagged for user review: N
+
 ### Summary
 Total issues found: X
 Total issues fixed: X

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bms-speckit-plugin",
-  "version": "6.4.0",
+  "version": "6.5.0",
   "description": "Chain-orchestrated development pipeline: /bms-speckit takes requirements and runs brainstorm → constitution → specify → plan → tasks → analyze → implement → verify with per-step error handling",
   "files": [
     ".claude-plugin/",

package/skills/bms-speckit-auto/SKILL.md

@@ -185,12 +185,20 @@ After the subagent completes, update tasks 1-8 as completed using TaskUpdate, th
      - Add unit tests that actually execute data transformation functions and verify the output type and shape
      - **SQL validation:** before writing any SQL statement, verify the exact table and column names exist. Use `bms-session-mcp-server` tools (`list_tables`, `describe_table`) or `mcp__bms-knowledge-mcp__search_knowledge` (hosxp collection) to confirm every field reference. Never guess column names. After writing a query, test it with `query` tool using `EXPLAIN` or `LIMIT 0` to confirm it parses and executes.
   2. **INLINE QC** — immediately run: build, lint, ALL tests, security quick scan, UX check
-  3. **FIX** — fix every issue found, re-run checks
-  4. **COMMIT** — only commit when build + lint + tests pass with zero errors
-  5. **NEXT** — move to next task
+  3. **INTEGRATION TEST (real database)** — for every task that touches database queries or data-dependent business logic, run an integration test that:
+     - Uses the real `bms-session-mcp-server` `query` tool to execute the feature's actual SQL against the real HOSxP database (not mocks)
+     - Flows the real result through the feature's actual code path (transformers, filters, aggregations) — not just the raw query
+     - Asserts on concrete properties: row count is reasonable, required columns are populated, data types match the schema, masked fields are handled correctly (patient names, CIDs are pre-masked — do not flag as errors)
+     - Uses a small `LIMIT` to avoid pulling large result sets during testing
+     - Verifies edge cases with real data: empty results for narrow filters, NULL handling, encoding (Thai characters), date ranges
+     - If the task produced a UI component that renders DB data, also verify the component renders without error using a real query result as the prop
+     - **If the integration test fails:** investigate whether the bug is in the SQL, the transformer, or the assertion. Fix the actual cause. Do NOT relax the assertion to make it pass.
+  4. **FIX** — fix every issue found in steps 2 and 3, re-run checks
+  5. **COMMIT** — only commit when build + lint + unit tests + integration tests all pass with zero errors
+  6. **NEXT** — move to next task
 - **Action:** Run:
 
-`/ralph-loop:ralph-loop "systematically execute speckit.implement via the Skill tool to complete every task defined in {TASKS_PATH} with strict adherence to specification requirements. IMPORTANT: apply rolling QC after EACH task — after implementing a task run build and fix build errors, run linter and fix lint errors, run ALL tests (not just new ones) and fix failures, check for hardcoded secrets and injection vulnerabilities in code you just wrote, verify UI code has actionable error messages and loading states. RUNTIME SAFETY: always add explicit return type annotations on data transformation functions, never spread or iterate a function return value without verifying it returns the expected collection type, use strict equality and null guards for external data, write tests that execute data transformers and verify output type and shape. SQL VALIDATION: before writing any SQL statement verify exact table and column names exist via bms-session-mcp-server list_tables/describe_table or bms-knowledge-mcp search_knowledge with hosxp collection, never guess column names, after writing test each query with EXPLAIN or LIMIT 0 via the query tool to confirm it executes without error. Only commit when build plus lint plus tests all pass with zero errors then proceed to next task. Report progress to the user after each task: output [Task N/total] DONE — task_name. Do NOT batch QC at the end. Maintain atomic commits after each successful task with clear traceability, avoid requesting confirmation and proceed autonomously, once all tasks are implemented invoke speckit.analyze via the Skill tool to perform a full validation pass, automatically apply all recommended improvements or corrections, re-run all tests to confirm stability and zero regression, and only output <promise>FINISHED</promise> after every task is fully completed, validated, and aligned with production-grade quality standards" --completion-promise "FINISHED" --max-iterations 10`
+`/ralph-loop:ralph-loop "systematically execute speckit.implement via the Skill tool to complete every task defined in {TASKS_PATH} with strict adherence to specification requirements. IMPORTANT: apply rolling QC after EACH task — after implementing a task run build and fix build errors, run linter and fix lint errors, run ALL tests (not just new ones) and fix failures, check for hardcoded secrets and injection vulnerabilities in code you just wrote, verify UI code has actionable error messages and loading states. RUNTIME SAFETY: always add explicit return type annotations on data transformation functions, never spread or iterate a function return value without verifying it returns the expected collection type, use strict equality and null guards for external data, write tests that execute data transformers and verify output type and shape. SQL VALIDATION: before writing any SQL statement verify exact table and column names exist via bms-session-mcp-server list_tables/describe_table or bms-knowledge-mcp search_knowledge with hosxp collection, never guess column names, after writing test each query with EXPLAIN or LIMIT 0 via the query tool to confirm it executes without error. INTEGRATION TESTING: for every task that touches database or data-dependent logic run an integration test using the real bms-session-mcp-server query tool to execute the feature's SQL against the real HOSxP database (not mocks), flow the result through the actual code path, assert on concrete properties (row count, columns populated, types match, masked fields handled), use small LIMIT to avoid large result sets. If integration test fails investigate the real cause and fix it — do not relax the assertion. Only commit when build plus lint plus tests all pass with zero errors then proceed to next task. Report progress to the user after each task: output [Task N/total] DONE — task_name. Do NOT batch QC at the end. Maintain atomic commits after each successful task with clear traceability, avoid requesting confirmation and proceed autonomously, once all tasks are implemented invoke speckit.analyze via the Skill tool to perform a full validation pass, automatically apply all recommended improvements or corrections, re-run all tests to confirm stability and zero regression, and only output <promise>FINISHED</promise> after every task is fully completed, validated, and aligned with production-grade quality standards" --completion-promise "FINISHED" --max-iterations 10`
 
 - **Done:** Update task 10 as completed. Output `[Step 10/12] DONE — all tasks implemented and verified`
 
@@ -209,6 +217,7 @@ After the subagent completes, update tasks 1-8 as completed using TaskUpdate, th
   - **G. Database compatibility & SQL validation** — two-part check:
     - **G1-G5 (cross-compatibility):** enforce "write SQL once, run on both" — all SQL must use cross-compatible syntax that works on MySQL and PostgreSQL without dialect branching. Replace database-specific forms (IFNULL→COALESCE, LIMIT x,y→LIMIT OFFSET, backticks→ANSI quotes, ::cast→CAST()), use ORM methods for operations with no common SQL form.
     - **G6 (SQL schema validation — MOST IMPORTANT):** validate every SQL statement in the codebase against the real database schema. Hallucinated column names are the #1 QC failure — LLMs frequently reference columns that don't exist. For each SQL statement: use `bms-session-mcp-server` (`list_tables`, `describe_table`, `query` with EXPLAIN/LIMIT 0) or `mcp__mysql__mysql_query` / `mcp__postgres__query` to verify every table and column reference exists. Fall back to `mcp__bms-knowledge-mcp__search_knowledge` (hosxp collection) or grep migration files if live DB is unavailable. Fix every hallucinated reference. (skipped if no SQL in project)
+  - **H. Integration testing (real database, end-to-end)** — run actual integration tests against the real HOSxP database via `bms-session-mcp-server`. For every DB-touching feature verify there's an integration test that: executes the feature's real SQL via the `query` tool, flows real data through the actual code path (transformers, filters, UI components), asserts on concrete properties (row count range, column presence, type correctness, masked field handling, Thai encoding). Create missing integration tests. Never fix failures by mocking responses, skipping tests, or loosening assertions. Flag slow queries for user review. (skipped if bms-session-mcp-server is not available)
 - The agent fixes everything it can. Major dependency updates are flagged for user review.
 - **Completion rule:** When the QC agent returns its report, proceed to Step 12 **unless** the report contains unfixed build errors, unfixed test failures, or unfixed critical security vulnerabilities. Informational findings, flagged-for-review items, and already-fixed issues do NOT block progression. If uncertain, proceed — the QC agent already fixed what it could.
 - **Post-action:** Commit all fixes and push. Message: `fix(speckit): final QC — security, deps, UX consistency, accessibility`