cli-test-framework 0.3.0__tar.gz → 0.3.2__tar.gz

{cli_test_framework-0.3.0 → cli_test_framework-0.3.2}/CHANGELOG.md

@@ -1,5 +1,30 @@
 # 变更日志
 
+## [v0.3.2] - 2024-12-29
+
+### 🔧 重大改进
+- **PyPI文档更新**：PyPI项目页面现在显示完整的用户手册内容（包含Setup模块文档）
+- **README重构**：更新README.md添加Setup模块使用示例和配置说明
+- **文档源切换**：setup.py现在使用`docs/user_manual.md`作为PyPI项目描述
+
+### 📚 文档增强
+- 在README中添加了Setup模块的快速使用示例
+- 添加了带环境变量配置的JSON/YAML示例
+- 增加了指向完整用户手册的链接和说明
+- 改善了文档结构和可读性
+
+## [v0.3.1] - 2024-12-29
+
+### 🔧 改进
+- **文档更新**：完善了用户手册中的Setup模块文档
+- **示例增强**：增加了更多详细的配置示例和最佳实践说明
+- **PyPI发布**：确保最新的用户手册包含在PyPI包中
+
+### 📚 文档修复
+- 修复了用户手册格式问题
+- 改进了Setup模块的使用示例
+- 补充了更多自定义插件开发指南
+
 ## [v0.3.0] - 2025-06-07
 
 ### 🎉 重大新功能 - Setup模块系统

cli_test_framework-0.3.2/PKG-INFO

@@ -0,0 +1,546 @@
+Metadata-Version: 2.4
+Name: cli-test-framework
+Version: 0.3.2
+Summary: A powerful command line testing framework in Python with setup modules, parallel execution, and file comparison capabilities.
+Home-page: https://github.com/yourusername/cli-test-framework
+Author: Xiaotong Wang
+Author-email: xiaotongwang98@gmail.com
+Project-URL: Documentation, https://github.com/yourusername/cli-test-framework/docs/user_manual.md
+Project-URL: Source, https://github.com/yourusername/cli-test-framework
+Project-URL: Tracker, https://github.com/yourusername/cli-test-framework/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: dukpy==0.5.0
+Requires-Dist: h5py>=3.8.0
+Requires-Dist: numpy>=2.0.1
+Requires-Dist: setuptools>=75.8.0
+Requires-Dist: wheel>=0.45.1
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: project-url
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# CLI Testing Framework User Manual
+
+## Table of Contents
+1. [Introduction](#introduction)
+2. [Installation](#installation)
+3. [Basic Usage](#basic-usage)
+4. [Test Case Definition](#test-case-definition)
+5. [Setup Module](#setup-module)
+6. [Parallel Testing](#parallel-testing)
+7. [File Comparison](#file-comparison)
+8. [Advanced Features](#advanced-features)
+9. [Troubleshooting](#troubleshooting)
+10. [API Reference](#api-reference)
+11. [Examples](#examples)
+
+## Introduction
+
+The CLI Testing Framework is a powerful tool designed for testing command-line applications and scripts. It provides a structured way to define, execute, and verify test cases, with support for parallel execution and advanced file comparison capabilities.
+
+### Key Features
+- Parallel test execution with thread and process support
+- JSON/YAML test case definition
+- Advanced file comparison capabilities
+- Comprehensive reporting
+- Extensible architecture
+
+## Installation
+
+### Prerequisites
+- Python 3.6 or higher
+- pip package manager
+
+### Basic Installation
+```bash
+pip install cli-test-framework
+```
+
+### Development Installation
+```bash
+git clone https://github.com/yourusername/cli-test-framework.git
+cd cli-test-framework
+pip install -e .
+```
+
+## Basic Usage
+
+### Creating a Test Case
+
+1. Create a JSON test case file (e.g., `test_cases.json`):
+```json
+{
+    "test_cases": [
+        {
+            "name": "Basic Command Test",
+            "command": "echo",
+            "args": ["Hello, World!"],
+            "expected": {
+                "return_code": 0,
+                "output_contains": ["Hello, World!"]
+            }
+        }
+    ]
+}
+```
+
+2. Run the test:
+```python
+from cli_test_framework.runners import JSONRunner
+
+runner = JSONRunner(
+    config_file="test_cases.json",
+    workspace="/path/to/workspace"
+)
+success = runner.run_tests()
+```
+
+### Using the Command Line
+
+```bash
+# Run tests from a JSON file
+cli-test run test_cases.json
+
+# Run tests in parallel
+cli-test run test_cases.json --parallel --workers 4
+```
+
+## Test Case Definition
+
+### JSON Format
+
+```json
+{
+    "test_cases": [
+        {
+            "name": "Test Case Name",
+            "command": "command_to_execute",
+            "args": ["arg1", "arg2"],
+            "expected": {
+                "return_code": 0,
+                "output_contains": ["expected text"],
+                "output_matches": [".*regex pattern.*"]
+            }
+        }
+    ]
+}
+```
+
+### YAML Format
+
+```yaml
+test_cases:
+  - name: Test Case Name
+    command: command_to_execute
+    args:
+      - arg1
+      - arg2
+    expected:
+      return_code: 0
+      output_contains:
+        - expected text
+      output_matches:
+        - ".*regex pattern.*"
+```
+
+## Setup Module
+
+The Setup Module provides a plugin-based system for executing pre-test setup tasks and post-test cleanup. It's designed to be extensible and supports multiple setup plugins running in sequence.
+
+### Key Features
+- **Plugin Architecture**: Extensible design allowing custom setup plugins
+- **Built-in Environment Plugin**: Set environment variables for tests
+- **Full Runner Support**: Works with JSONRunner, YAMLRunner, and ParallelJSONRunner
+- **Lifecycle Management**: Automatic setup and teardown with proper cleanup
+
+### Environment Variable Setup
+
+#### JSON Configuration
+```json
+{
+  "setup": {
+    "environment_variables": {
+      "TEST_ENV": "development",
+      "DEBUG_MODE": "true",
+      "API_URL": "http://localhost:8080",
+      "DATABASE_URL": "sqlite:///test.db"
+    }
+  },
+  "test_cases": [
+    {
+      "name": "Test with environment variables",
+      "command": "python",
+      "args": ["-c", "import os; print(f'Env: {os.environ.get(\"TEST_ENV\")}')"],
+      "expected": {
+        "return_code": 0,
+        "output_contains": ["Env: development"]
+      }
+    }
+  ]
+}
+```
+
+#### YAML Configuration
+```yaml
+setup:
+  environment_variables:
+    TEST_ENV: "production"
+    DATABASE_URL: "postgresql://localhost:5432/test"
+    MAX_CONNECTIONS: "10"
+    TIMEOUT_SECONDS: "30"
+
+test_cases:
+  - name: "Test database environment"
+    command: "python"
+    args: 
+      - "-c"
+      - "import os; print(f'DB: {os.environ.get(\"DATABASE_URL\")}')"
+    expected:
+      return_code: 0
+      output_contains:
+        - "DB: postgresql://localhost:5432/test"
+```
+
+### Custom Setup Plugins
+
+#### Creating Custom Plugins
+```python
+from cli_test_framework import BaseSetup
+
+class DatabaseSetup(BaseSetup):
+    def setup(self):
+        """Initialize test database"""
+        print("Setting up test database...")
+        # Your database initialization code here
+        
+    def teardown(self):
+        """Clean up test database"""
+        print("Cleaning up test database...")
+        # Your database cleanup code here
+
+class ServiceSetup(BaseSetup):
+    def setup(self):
+        """Start test services"""
+        self.service_port = self.config.get('port', 8080)
+        print(f"Starting test service on port {self.service_port}")
+        # Your service startup code here
+        
+    def teardown(self):
+        """Stop test services"""
+        print("Stopping test services...")
+        # Your service shutdown code here
+```
+
+#### Using Custom Plugins
+```python
+from cli_test_framework import JSONRunner
+
+# Create runner
+runner = JSONRunner("test_cases.json")
+
+# Add custom setup plugins
+db_setup = DatabaseSetup({"connection": "test_db"})
+service_setup = ServiceSetup({"port": 9090})
+
+runner.setup_manager.add_setup(db_setup)
+runner.setup_manager.add_setup(service_setup)
+
+# Run tests (setup will be executed automatically)
+success = runner.run_tests()
+```
+
+### Execution Flow
+
+1. **Load Configuration**: Read setup configuration from test file
+2. **Setup Phase**: Execute all setup plugins in order
+   - Environment variables are set
+   - Custom setups are executed
+   - Setup status is reported
+3. **Test Execution**: Run all test cases with setup environment
+4. **Teardown Phase**: Clean up all setups in reverse order
+   - Environment variables are restored
+   - Custom cleanups are executed
+   - Cleanup is guaranteed even if tests fail
+
+### Best Practices
+
+1. **Idempotent Operations**: Make setup operations safe to run multiple times
+2. **Proper Cleanup**: Always implement teardown to avoid side effects
+3. **Error Handling**: Setup failures stop test execution, teardown failures don't
+4. **Resource Management**: Use try-finally patterns in custom plugins
+5. **Configuration Validation**: Check required configuration parameters in setup
+
+### Integration Examples
+
+#### With JSON Runner
+```bash
+cli-test test_with_setup.json
+```
+
+#### With YAML Runner
+```bash
+cli-test test_with_setup.yaml --runner yaml
+```
+
+#### With Parallel Runner
+```bash
+cli-test test_with_setup.json --runner parallel --max-workers 4
+```
+
+Note: In parallel mode, setup and teardown run in the main thread to ensure environment consistency.
+
+## Parallel Testing
+
+### Thread Mode
+```python
+from cli_test_framework.runners import ParallelJSONRunner
+
+runner = ParallelJSONRunner(
+    config_file="test_cases.json",
+    max_workers=4,
+    execution_mode="thread"
+)
+success = runner.run_tests()
+```
+
+### Process Mode
+```python
+runner = ParallelJSONRunner(
+    config_file="test_cases.json",
+    max_workers=2,
+    execution_mode="process"
+)
+success = runner.run_tests()
+```
+
+## File Comparison
+
+### Basic File Comparison
+```bash
+# Compare two text files
+compare-files file1.txt file2.txt
+
+# Compare with specific options
+compare-files file1.txt file2.txt --start-line 10 --end-line 20
+```
+
+### JSON File Comparison
+```bash
+# Exact comparison
+compare-files data1.json data2.json
+
+# Key-based comparison
+compare-files data1.json data2.json --json-compare-mode key-based --json-key-field id
+```
+
+### HDF5 File Comparison
+```bash
+# Compare specific tables
+compare-files data1.h5 data2.h5 --h5-table table1,table2
+
+# Compare with numerical tolerance
+compare-files data1.h5 data2.h5 --h5-rtol 1e-5 --h5-atol 1e-8
+```
+
+### Binary File Comparison
+```bash
+# Compare with similarity check
+compare-files binary1.bin binary2.bin --similarity
+
+# Compare with custom chunk size
+compare-files binary1.bin binary2.bin --chunk-size 16384
+```
+
+## Advanced Features
+
+### Custom Assertions
+```python
+from cli_test_framework.assertions import BaseAssertion
+
+class CustomAssertion(BaseAssertion):
+    def assert_custom_condition(self, actual, expected):
+        if not self._check_custom_condition(actual, expected):
+            raise AssertionError("Custom condition not met")
+```
+
+### Custom Runners
+```python
+from cli_test_framework.runners import BaseRunner
+
+class CustomRunner(BaseRunner):
+    def load_test_cases(self):
+        # Custom test case loading logic
+        pass
+
+    def run_test(self, test_case):
+        # Custom test execution logic
+        pass
+```
+
+### Output Formats
+```python
+# JSON output
+runner = JSONRunner(config_file="test_cases.json", output_format="json")
+
+# HTML output
+runner = JSONRunner(config_file="test_cases.json", output_format="html")
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Command Not Found**
+   - Ensure the command is in the system PATH
+   - Use absolute paths for scripts
+   - Check command permissions
+
+2. **Parallel Execution Issues**
+   - Reduce number of workers
+   - Check for resource conflicts
+   - Use process mode for CPU-intensive tests
+
+3. **File Comparison Issues**
+   - Verify file permissions
+   - Check file encoding
+   - Ensure sufficient memory for large files
+
+### Debug Mode
+```python
+runner = JSONRunner(
+    config_file="test_cases.json",
+    debug=True
+)
+```
+
+## API Reference
+
+### Core Classes
+
+#### JSONRunner
+```python
+class JSONRunner:
+    def __init__(self, config_file, workspace=None, debug=False):
+        """
+        Initialize JSONRunner
+        :param config_file: Path to JSON test case file
+        :param workspace: Working directory for test execution
+        :param debug: Enable debug mode
+        """
+```
+
+#### ParallelJSONRunner
+```python
+class ParallelJSONRunner:
+    def __init__(self, config_file, max_workers=None, execution_mode="thread"):
+        """
+        Initialize ParallelJSONRunner
+        :param config_file: Path to JSON test case file
+        :param max_workers: Maximum number of parallel workers
+        :param execution_mode: "thread" or "process"
+        """
+```
+
+### File Comparison
+
+#### ComparatorFactory
+```python
+class ComparatorFactory:
+    @staticmethod
+    def create_comparator(file_type, **kwargs):
+        """
+        Create a comparator instance
+        :param file_type: Type of file to compare
+        :param kwargs: Additional comparator options
+        :return: Comparator instance
+        """
+```
+
+## Examples
+
+### Complete Test Suite
+```python
+from cli_test_framework.runners import JSONRunner
+from cli_test_framework.assertions import Assertions
+
+# Create test runner
+runner = JSONRunner(
+    config_file="test_suite.json",
+    workspace="/project/root",
+    debug=True
+)
+
+# Run tests
+success = runner.run_tests()
+
+# Process results
+if success:
+    print("All tests passed!")
+else:
+    print("Some tests failed:")
+    for result in runner.results["details"]:
+        if result["status"] == "failed":
+            print(f"- {result['name']}: {result['message']}")
+```
+
+### Parallel Test Suite
+```python
+from cli_test_framework.runners import ParallelJSONRunner
+import os
+
+# Create parallel runner
+runner = ParallelJSONRunner(
+    config_file="test_suite.json",
+    max_workers=os.cpu_count() * 2,
+    execution_mode="thread"
+)
+
+# Run tests in parallel
+success = runner.run_tests()
+
+# Generate report
+runner.generate_report("test_report.html")
+```
+
+### File Comparison Suite
+```python
+from cli_test_framework.file_comparator import ComparatorFactory
+
+# Compare text files
+text_comparator = ComparatorFactory.create_comparator(
+    "text",
+    encoding="utf-8",
+    verbose=True
+)
+text_result = text_comparator.compare_files("file1.txt", "file2.txt")
+
+# Compare JSON files
+json_comparator = ComparatorFactory.create_comparator(
+    "json",
+    compare_mode="key-based",
+    key_field="id"
+)
+json_result = json_comparator.compare_files("data1.json", "data2.json")
+```
+
+## Contributing
+
+We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. 

cli_test_framework-0.3.0/src/cli_test_framework.egg-info/PKG-INFO → cli_test_framework-0.3.2/README.md

@@ -1,38 +1,3 @@
-Metadata-Version: 2.4
-Name: cli-test-framework
-Version: 0.3.0
-Summary: A powerful command line testing framework in Python with setup modules, parallel execution, and file comparison capabilities.
-Home-page: https://github.com/yourusername/cli-test-framework
-Author: Xiaotong Wang
-Author-email: xiaotongwang98@gmail.com
-Project-URL: Documentation, https://github.com/yourusername/cli-test-framework/docs/user_manual.md
-Project-URL: Source, https://github.com/yourusername/cli-test-framework
-Project-URL: Tracker, https://github.com/yourusername/cli-test-framework/issues
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: OS Independent
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Topic :: Software Development :: Testing
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-Requires-Dist: dukpy==0.5.0
-Requires-Dist: h5py>=3.8.0
-Requires-Dist: numpy>=2.0.1
-Requires-Dist: setuptools>=75.8.0
-Requires-Dist: wheel>=0.45.1
-Dynamic: author
-Dynamic: author-email
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: project-url
-Dynamic: requires-dist
-Dynamic: requires-python
-Dynamic: summary
-
 # CLI Testing Framework
 
 ## 1. Overview
@@ -42,7 +7,8 @@ This is a lightweight and extensible automated testing framework that supports d
 ## 2. Features
 
 - **🚀 Parallel Test Execution**: Support for multi-threading and multi-processing parallel testing with significant performance improvements
-- **🏗️ Modular Architecture**: Decoupled design of core components (runner/assertion/report)
+- **🔧 Setup Module System**: Plugin-based architecture for pre-test setup tasks (environment variables, database initialization, service startup)
+- **🏗️ Modular Architecture**: Decoupled design of core components (runner/assertion/report/setup)
 - **📄 Multi-Format Support**: Native support for JSON/YAML test case formats
 - **🧠 Intelligent Command Parsing**: Smart handling of complex commands like `"python ./script.py"`
 - **📁 Smart Path Resolution**: Automatic handling of relative and absolute path conversions
@@ -89,6 +55,21 @@ runner = ParallelJSONRunner(
 success = runner.run_tests()
 ```
 
+### Setup Module Usage
+
+```python
+from cli_test_framework import JSONRunner, EnvironmentSetup
+
+# Using built-in environment variable setup
+runner = JSONRunner("test_cases.json")
+env_setup = EnvironmentSetup({
+    "TEST_ENV": "development",
+    "API_URL": "http://localhost:8080"
+})
+runner.setup_manager.add_setup(env_setup)
+success = runner.run_tests()
+```
+
 ### File Comparison
 
 ```bash
@@ -111,9 +92,25 @@ compare-files binary1.bin binary2.bin --similarity
 
 ```json
 {
+    "setup": {
+        "environment_variables": {
+            "TEST_ENV": "development",
+            "API_URL": "http://localhost:8080",
+            "DEBUG_MODE": "true"
+        }
+    },
     "test_cases": [
         {
-            "name": "File Comparison Test",
+            "name": "Environment Variable Test",
+            "command": "python",
+            "args": ["-c", "import os; print(f'Environment: {os.environ.get(\"TEST_ENV\")}')"],
+            "expected": {
+                "return_code": 0,
+                "output_contains": ["Environment: development"]
+            }
+        },
+        {
+            "name": "File Comparison Test", 
             "command": "compare-files",
             "args": ["file1.txt", "file2.txt", "--verbose"],
             "expected": {
@@ -129,7 +126,22 @@ compare-files binary1.bin binary2.bin --similarity
 ### YAML Format
 
 ```yaml
+setup:
+  environment_variables:
+    TEST_ENV: "production"
+    DATABASE_URL: "sqlite:///test.db"
+
 test_cases:
+  - name: Environment Test
+    command: python
+    args:
+      - "-c"
+      - "import os; print(f'DB: {os.environ.get(\"DATABASE_URL\")}')"
+    expected:
+      return_code: 0
+      output_contains:
+        - "DB: sqlite:///test.db"
+  
   - name: Directory Scan Test
     command: ls
     args:
@@ -440,6 +452,22 @@ This project is licensed under the MIT License - see the LICENSE file for detail
 
 ---
 
-**🚀 Ready to supercharge your testing workflow with parallel execution and advanced file comparison!**
+## 📚 Complete Documentation
+
+For comprehensive documentation including detailed Setup Module guide, API reference, and advanced usage examples, see:
+
+**[📖 Complete User Manual](docs/user_manual.md)**
+
+The user manual includes:
+- 🔧 **Setup Module**: Complete guide for environment variables and custom plugins
+- 🚀 **Parallel Testing**: Advanced parallel execution strategies  
+- 📁 **File Comparison**: Detailed comparison capabilities for all file types
+- 🔌 **API Reference**: Full API documentation and examples
+- 🛠️ **Troubleshooting**: Common issues and solutions
+- 📝 **Best Practices**: Recommended patterns and configurations
+
+---
+
+**🚀 Ready to supercharge your testing workflow with setup modules, parallel execution and advanced file comparison!**
 
 For detailed parallel testing guide, see: [PARALLEL_TESTING_GUIDE.md](PARALLEL_TESTING_GUIDE.md)