mcp-maestro-mobile-ai 1.3.1 → 1.6.0

package/CHANGELOG.md

@@ -1,152 +1,344 @@
-# Changelog
-
-All notable changes to this project will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-
-## [Unreleased]
-
-### Planned
-- Security boundaries (Safe Mode)
-- JUnit XML report generation
-- CI headless mode
-- iOS support
-
----
-
-## [1.3.1] - 2025-01-06
-
-### Fixed
-- **MCP Schema Fix**: Fixed `generate_report` tool array parameter missing `items` definition
-  - This was causing "tool parameters array type must have items" validation error
-  - Now properly defines the structure of test result objects
-
----
-
-## [1.2.0] - 2025-01-06
-
-### Added
-- **YAML Generation Instructions System**: Ensures consistent YAML generation across different environments
-  - `get_yaml_instructions` - AI MUST call this before generating YAML (provides exact rules)
-  - `validate_yaml_structure` - Validates YAML for common issues (like missing tapOn before inputText)
-  - `get_test_pattern` - Get standard patterns for login, search, navigation, form tests
-- **Critical Fix**: Input text pattern now enforced - prevents password going to username field issue
-- Standard test patterns for common scenarios (login, search, navigation, form)
-
-### Fixed
-- YAML generation inconsistency between different environments
-- Text input going to wrong fields due to missing tapOn commands
-
----
-
-## [1.1.1] - 2025-01-06
-
-### Fixed
-- Version bump for npm publish
-
----
-
-## [1.1.0] - 2025-01-06
-
-### Changed
-- **Package Renamed**: Changed from `@krunal.mahera/maestro-mcp` to `mcp-maestro-mobile-ai` for easier configuration
-- **YAML Storage**: Temp YAML files now stored in hidden system directory (`~/.maestro-mcp/`) instead of project folder
-- Test results and screenshots now stored in `~/.maestro-mcp/output/`
-
-### Added
-- **Automatic Prerequisites Check**: 
-  - Runs automatically after `npm install`
-  - Checks for Node.js 18+, Java 17+, Maestro CLI, Android SDK
-  - Shows clear error messages with installation hints
-  - Manual check available via `npm run check`
-- **Runtime Validation**: Server validates prerequisites on startup and exits gracefully if critical deps missing
-- **App Context Training System**: New tools to teach the AI about your app's UI
-  - `register_elements` - Register testIDs, accessibilityLabels for app elements
-  - `register_screen` - Define screen structures and available actions
-  - `save_successful_flow` - Save working test patterns for AI reference
-  - `get_saved_flows` - Retrieve saved flow patterns
-  - `delete_flow` - Remove saved patterns
-  - `get_ai_context` - Get formatted context for AI (call before generating tests!)
-  - `get_full_context` - Get complete raw context data
-  - `clear_app_context` - Clear all context for an app
-  - `list_app_contexts` - List all apps with saved context
-
-### Improved
-- AI test generation accuracy when context is provided
-- Cleaner project directory (no temp files visible)
-
----
-
-## [1.0.0] - 2025-01-05
-
-### Added
-- Initial public release on npm as `mcp-maestro-mobile-ai`
-- MCP server implementation with stdio transport
-- 14 MCP tools for mobile test automation:
-  - `read_prompt_file` - Read test prompts from files
-  - `list_prompt_files` - List available prompt files
-  - `list_devices` - List connected Android devices
-  - `select_device` - Select specific device for testing
-  - `clear_device` - Clear device selection
-  - `check_device` - Verify device connection
-  - `check_app` - Verify app installation
-  - `get_app_config` - Get server configuration
-  - `validate_maestro_yaml` - Validate YAML syntax
-  - `run_test` - Execute single test
-  - `run_test_suite` - Execute multiple tests
-  - `get_test_results` - Retrieve test results
-  - `take_screenshot` - Capture device screen
-  - `cleanup_results` - Clean up old results
-- Automatic retry mechanism for failed tests
-- Pre-flight checks (device, app) before test execution
-- Screenshot capture on test failure
-- Auto-cleanup of old results based on `MAX_RESULTS`
-- Improved error messages with hints
-- Support for physical devices via USB
-- Device selection for multi-device environments
-- Winston-based logging
-- Environment variable configuration
-- Example prompt files
-
-### Documentation
-- Comprehensive README with setup guides
-- MCP client configuration examples (Cursor, VS Code, Claude Desktop)
-- Template configuration files
-- React Native automation guidelines
-
----
-
-## [0.1.0] - 2025-01-01
-
-### Added
-- Initial proof of concept
-- Basic Maestro CLI integration
-- Simple test execution
-
----
-
-## Release Notes Format
-
-### Version Numbering
-
-- **MAJOR** (X.0.0): Breaking API changes
-- **MINOR** (0.X.0): New features, backward compatible
-- **PATCH** (0.0.X): Bug fixes, backward compatible
-
-### Change Categories
-
-- **Added**: New features
-- **Changed**: Changes in existing functionality
-- **Deprecated**: Features to be removed in future
-- **Removed**: Removed features
-- **Fixed**: Bug fixes
-- **Security**: Security-related changes
-
----
-
-[Unreleased]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/compare/v1.1.0...HEAD
-[1.1.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.1.0
-[1.0.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.0.0
-[0.1.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v0.1.0
-
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Planned
+
+- JUnit XML report generation
+- CI headless mode (`--ci` flag)
+- iOS support
+- TypeScript migration
+
+---
+
+## [1.6.0] - 2025-01-07
+
+### Added
+
+- **YAML Caching System**: Intelligent caching for faster recurring test execution
+
+  - `save_to_cache` - Save successful test YAML for future reuse
+  - `lookup_cache` - Check if a cached YAML exists for a prompt
+  - `list_cache` - List all cached tests with usage statistics
+  - `clear_cache` - Clear all cached YAMLs
+  - `delete_from_cache` - Delete specific cached test
+  - `get_cache_stats` - Get cache statistics (total cached, execution counts)
+  - `run_test_with_cache` - Run test with automatic cache lookup
+
+- **Prompt-Based Hashing**: Unique identification of test prompts
+  - SHA-256 based hashing with normalization
+  - Handles minor formatting differences (whitespace, quotes)
+  - AppId included in hash for app-specific caching
+
+- **Hidden Cache Storage**: Internal cache location (`~/.maestro-mcp/cache/`)
+  - Index file for fast lookup
+  - YAML files stored with hash-based names
+  - Not visible to end users
+
+- **Usage Statistics**: Track cache effectiveness
+  - Execution count per cached test
+  - Last used timestamp
+  - Most used tests identification
+
+- **Auto-Cache on Success**: After successful test execution
+  - Automatically saves YAML to cache (no user prompt needed)
+  - Failed tests are NOT cached (fix and re-run to cache)
+  - Automatic reuse when same prompt is run again
+
+### Changed
+
+- **Server Version**: Updated to v1.6.0
+- **runTest function**: Now includes cache prompt for successful tests
+- **runTestSuiteWithReport**: Offers bulk caching for successful tests
+
+### Benefits
+
+| Feature | Benefit |
+|---------|---------|
+| Faster Execution | Skip YAML generation for recurring tests |
+| Reduced AI Calls | Cached YAML reused automatically |
+| Clean Separation | Cache hidden from user-facing directories |
+| Smart Invalidation | New YAML generated if prompt changes |
+
+---
+
+## [1.5.0] - 2025-01-07
+
+### Added
+
+- **Prompt Analysis System (Action-Element-Verification Model)**: Intelligent prompt parsing for reliable YAML generation
+
+  - `validate_prompt` - PRIMARY tool for converting natural language to Maestro YAML
+  - `analyze_prompt` - Analyze prompts without generating YAML (debugging/inspection)
+  - Extracts: Actions, Elements, Values, Verifications, Sequence
+
+- **Tiered Clarification System**: Smart handling of incomplete prompts
+
+  - **COMPLETE**: All required info present → generates clean YAML
+  - **GENERATABLE**: Missing some info → asks clarification OR generates with warnings
+  - **NEEDS_CLARIFICATION**: Missing important info → asks specific questions
+  - **INSUFFICIENT**: Missing critical info (app ID) → requests minimum required info
+
+- **Force Generate with Warnings**: Option to generate YAML with assumptions
+  - Automatically adds warning comments for assumed values
+  - Lists all assumptions made during generation
+  - Placeholder values clearly marked for user to replace
+
+- **New Utilities**:
+  - `src/mcp-server/utils/promptAnalyzer.js` - Action-Element-Verification parser
+  - `src/mcp-server/utils/yamlGenerator.js` - YAML generation with warnings support
+  - `validateAndGenerate()` - Orchestrator function in contextTools.js
+
+- **Intelligent Clarification Questions**: Context-aware questions based on what's missing
+  - Critical questions (blocking): App ID
+  - Important questions: Field labels, button text
+  - Recommended questions: Verification elements, dropdown options
+
+### Changed
+
+- **Server Version**: Updated to v1.5.0
+- **Description**: Updated to highlight prompt analysis and tiered clarification
+- **Keywords**: Added `prompt-analysis`, `natural-language`, `yaml-generator`
+
+### Developer Experience
+
+- Generic, framework-agnostic approach works for ANY mobile app
+- No hard-coded app-specific values
+- Interaction patterns automatically applied
+- Comprehensive test script for verification
+
+---
+
+## [1.4.0] - 2025-01-07
+
+### Security
+
+- **Safe Mode (v1.1.0 Roadmap)**: Enterprise-grade security boundaries now implemented
+
+  - Safe Mode enabled by default (`SAFE_MODE=true`)
+  - Blocks potentially destructive operations (install/uninstall apps, clear data)
+  - Set `SAFE_MODE=false` to enable full mode (use with caution)
+
+- **Command Allowlists**: Strict validation of all CLI commands
+
+  - Maestro commands restricted to: `test`, `validate`, `screenshot`, `--version`, `hierarchy`
+  - ADB commands categorized into Safe Mode and Full Mode allowlists
+  - 40+ dangerous ADB commands permanently blocked (rm, root, reboot, settings, etc.)
+
+- **Blocked Pattern Detection**: Prevents injection attacks
+
+  - Shell injection: `; & | && ||`
+  - Command substitution: `` `cmd` ``, `$(cmd)`
+  - Path traversal: `../`
+  - Environment variable expansion: `${VAR}`, `$HOME`
+  - Null byte injection: `\x00`, `%00`
+  - Script injection: `<script>`, `javascript:`
+
+- **Input Validation with Zod**: All 30 MCP tools now have strict input validation
+
+  - App ID format validation (`com.example.app`)
+  - Device ID format validation
+  - File path security (no traversal)
+  - YAML content security checks
+  - String length limits and type checking
+
+- **Security Audit Logging**: Comprehensive event logging for compliance
+  - `TOOL_EXECUTION_START/SUCCESS/ERROR` events
+  - `TOOL_VALIDATION_FAILED` for rejected inputs
+  - `TOOL_SECURITY_ERROR` for security violations
+  - `SERVER_STARTED` with security config summary
+
+### Added
+
+- **New Security Module**: `src/mcp-server/utils/security.js`
+
+  - `SecurityError` class with error codes
+  - `isSafeModeEnabled()`, `getSecurityMode()`, `getSecurityConfig()`
+  - `validateAppId()`, `validateDeviceId()`, `sanitizeInput()`
+  - `isMaestroCommandAllowed()`, `isAdbCommandAllowed()`
+  - `containsBlockedPattern()`, `assertNoBlockedPatterns()`
+  - `checkYamlSecurity()` for YAML content validation
+  - `logSecurityEvent()` for audit trail
+
+- **Zod Schema Validation**: `src/mcp-server/schemas/toolSchemas.js`
+
+  - Individual schemas for all 30 MCP tools
+  - Reusable schema components (`safeFilePath`, `appIdSchema`, etc.)
+  - `validateToolInput()` utility function
+  - `toolSchemas` registry for easy lookup
+
+- **Validation Middleware**: Integrated into main request handler
+  - All tool inputs validated before execution
+  - Clear error messages with field-level details
+  - Security error handling with proper response format
+
+### Changed
+
+- **Server Version**: Updated to v1.4.0
+- **Startup Logging**: Now displays security configuration summary
+- **Error Responses**: Enhanced with validation details and security codes
+- **Command Execution**: All Maestro/ADB commands validated against allowlists
+
+### Environment Variables
+
+- `SAFE_MODE` - Enable/disable Safe Mode (default: `true`)
+- `LOG_SECURITY_EVENTS` - Enable security event logging (default: `true`)
+- `MAESTRO_DEVICE` - Pre-select a specific device for testing
+
+---
+
+## [1.3.1] - 2025-01-06
+
+### Fixed
+
+- **MCP Schema Fix**: Fixed `generate_report` tool array parameter missing `items` definition
+  - This was causing "tool parameters array type must have items" validation error
+  - Now properly defines the structure of test result objects
+
+---
+
+## [1.2.0] - 2025-01-06
+
+### Added
+
+- **YAML Generation Instructions System**: Ensures consistent YAML generation across different environments
+  - `get_yaml_instructions` - AI MUST call this before generating YAML (provides exact rules)
+  - `validate_yaml_structure` - Validates YAML for common issues (like missing tapOn before inputText)
+  - `get_test_pattern` - Get standard patterns for login, search, navigation, form tests
+- **Critical Fix**: Input text pattern now enforced - prevents password going to username field issue
+- Standard test patterns for common scenarios (login, search, navigation, form)
+
+### Fixed
+
+- YAML generation inconsistency between different environments
+- Text input going to wrong fields due to missing tapOn commands
+
+---
+
+## [1.1.1] - 2025-01-06
+
+### Fixed
+
+- Version bump for npm publish
+
+---
+
+## [1.1.0] - 2025-01-06
+
+### Changed
+
+- **Package Renamed**: Changed from `@krunal.mahera/maestro-mcp` to `mcp-maestro-mobile-ai` for easier configuration
+- **YAML Storage**: Temp YAML files now stored in hidden system directory (`~/.maestro-mcp/`) instead of project folder
+- Test results and screenshots now stored in `~/.maestro-mcp/output/`
+
+### Added
+
+- **Automatic Prerequisites Check**:
+  - Runs automatically after `npm install`
+  - Checks for Node.js 18+, Java 17+, Maestro CLI, Android SDK
+  - Shows clear error messages with installation hints
+  - Manual check available via `npm run check`
+- **Runtime Validation**: Server validates prerequisites on startup and exits gracefully if critical deps missing
+- **App Context Training System**: New tools to teach the AI about your app's UI
+  - `register_elements` - Register testIDs, accessibilityLabels for app elements
+  - `register_screen` - Define screen structures and available actions
+  - `save_successful_flow` - Save working test patterns for AI reference
+  - `get_saved_flows` - Retrieve saved flow patterns
+  - `delete_flow` - Remove saved patterns
+  - `get_ai_context` - Get formatted context for AI (call before generating tests!)
+  - `get_full_context` - Get complete raw context data
+  - `clear_app_context` - Clear all context for an app
+  - `list_app_contexts` - List all apps with saved context
+
+### Improved
+
+- AI test generation accuracy when context is provided
+- Cleaner project directory (no temp files visible)
+
+---
+
+## [1.0.0] - 2025-01-05
+
+### Added
+
+- Initial public release on npm as `mcp-maestro-mobile-ai`
+- MCP server implementation with stdio transport
+- 14 MCP tools for mobile test automation:
+  - `read_prompt_file` - Read test prompts from files
+  - `list_prompt_files` - List available prompt files
+  - `list_devices` - List connected Android devices
+  - `select_device` - Select specific device for testing
+  - `clear_device` - Clear device selection
+  - `check_device` - Verify device connection
+  - `check_app` - Verify app installation
+  - `get_app_config` - Get server configuration
+  - `validate_maestro_yaml` - Validate YAML syntax
+  - `run_test` - Execute single test
+  - `run_test_suite` - Execute multiple tests
+  - `get_test_results` - Retrieve test results
+  - `take_screenshot` - Capture device screen
+  - `cleanup_results` - Clean up old results
+- Automatic retry mechanism for failed tests
+- Pre-flight checks (device, app) before test execution
+- Screenshot capture on test failure
+- Auto-cleanup of old results based on `MAX_RESULTS`
+- Improved error messages with hints
+- Support for physical devices via USB
+- Device selection for multi-device environments
+- Winston-based logging
+- Environment variable configuration
+- Example prompt files
+
+### Documentation
+
+- Comprehensive README with setup guides
+- MCP client configuration examples (Cursor, VS Code, Claude Desktop)
+- Template configuration files
+- React Native automation guidelines
+
+---
+
+## [0.1.0] - 2025-01-01
+
+### Added
+
+- Initial proof of concept
+- Basic Maestro CLI integration
+- Simple test execution
+
+---
+
+## Release Notes Format
+
+### Version Numbering
+
+- **MAJOR** (X.0.0): Breaking API changes
+- **MINOR** (0.X.0): New features, backward compatible
+- **PATCH** (0.0.X): Bug fixes, backward compatible
+
+### Change Categories
+
+- **Added**: New features
+- **Changed**: Changes in existing functionality
+- **Deprecated**: Features to be removed in future
+- **Removed**: Removed features
+- **Fixed**: Bug fixes
+- **Security**: Security-related changes
+
+---
+
+[Unreleased]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/compare/v1.6.0...HEAD
+[1.6.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.6.0
+[1.5.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.5.0
+[1.4.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.4.0
+[1.3.1]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.3.1
+[1.2.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.2.0
+[1.1.1]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.1.1
+[1.1.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.1.0
+[1.0.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v1.0.0
+[0.1.0]: https://github.com/krunal-mahera/mcp-maestro-mobile-ai/releases/tag/v0.1.0

package/ROADMAP.md

@@ -37,16 +37,18 @@ Transform how teams approach mobile testing by enabling:
 
 **Theme**: Enterprise Trust & Automation
 
-### v1.1.0 - Security Boundaries
+### v1.4.0 - Security Boundaries ✅ RELEASED
 
 | Feature | Priority | Status |
 |---------|----------|--------|
-| Safe Mode (default ON) | 🔴 Critical | 🔲 Planned |
-| Command allowlist | 🔴 Critical | 🔲 Planned |
-| Blocked operations list | 🔴 Critical | 🔲 Planned |
-| Input validation (Zod schemas) | 🟠 High | 🔲 Planned |
+| Safe Mode (default ON) | 🔴 Critical | ✅ Done |
+| Command allowlist | 🔴 Critical | ✅ Done |
+| Blocked operations list | 🔴 Critical | ✅ Done |
+| Input validation (Zod schemas) | 🟠 High | ✅ Done |
+| Security audit logging | 🟠 High | ✅ Done |
+| Pattern detection (injection prevention) | 🟠 High | ✅ Done |
 
-### v1.2.0 - CI/CD Mode
+### v1.5.0 - CI/CD Mode
 
 | Feature | Priority | Status |
 |---------|----------|--------|
@@ -55,11 +57,11 @@ Transform how teams approach mobile testing by enabling:
 | `--prompt-file` direct execution | 🟠 High | 🔲 Planned |
 | JUnit XML output | 🟠 High | 🔲 Planned |
 
-### v1.3.0 - Audit & Observability
+### v1.6.0 - Audit & Observability
 
 | Feature | Priority | Status |
 |---------|----------|--------|
-| Enhanced audit trail | 🟠 High | 🔲 Planned |
+| Enhanced audit trail | 🟠 High | ⚠️ Partial (security events) |
 | YAML preservation | 🟠 High | 🔲 Planned |
 | Structured logging (JSON) | 🟡 Medium | 🔲 Planned |
 | Health check tool | 🟡 Medium | 🔲 Planned |
@@ -212,6 +214,17 @@ Transform how teams approach mobile testing by enabling:
 
 ## Completed Milestones
 
+### ✅ v1.4.0 - Security Boundaries (January 2025)
+
+- [x] Safe Mode (default ON) - blocks destructive operations
+- [x] Command allowlists for Maestro and ADB
+- [x] 40+ blocked dangerous commands
+- [x] Pattern detection (shell injection, path traversal, etc.)
+- [x] Input validation with Zod schemas for all 30 tools
+- [x] Security audit logging
+- [x] SecurityError class with error codes
+- [x] Validation middleware in request handler
+
 ### ✅ v1.0.0 - Foundation (January 2025)
 
 - [x] MCP server implementation

package/package.json

@@ -1,8 +1,8 @@
 {
     "name": "mcp-maestro-mobile-ai",
-    "version": "1.3.1",
+    "version": "1.6.0",
     "private": false,
-    "description": "MCP Server for AI-Assisted Mobile Automation using Maestro - Run mobile tests with natural language prompts",
+    "description": "MCP Server for AI-Assisted Mobile Automation using Maestro - Run mobile tests with natural language prompts. Features prompt analysis, tiered clarification, and enterprise-grade security.",
     "main": "src/mcp-server/index.js",
     "type": "module",
     "bin": {
@@ -51,7 +51,13 @@
         "cursor",
         "testing",
         "automation",
-        "ai"
+        "ai",
+        "security",
+        "validation",
+        "zod",
+        "prompt-analysis",
+        "natural-language",
+        "yaml-generator"
     ],
     "author": "Krunal Mahera <krunal.mahera@gmail.com>",
     "license": "MIT",