codemie-test-harness 0.1.201__tar.gz → 0.1.202__tar.gz

{codemie_test_harness-0.1.201 → codemie_test_harness-0.1.202}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: codemie-test-harness
-Version: 0.1.201
+Version: 0.1.202
 Summary: Autotest for CodeMie backend and UI
 Author: Anton Yeromin
 Author-email: anton_yeromin@epam.com
@@ -13,7 +13,7 @@ Requires-Dist: aws-assume-role-lib (>=2.10.0,<3.0.0)
 Requires-Dist: boto3 (>=1.39.8,<2.0.0)
 Requires-Dist: click (>=8.1.7,<9.0.0)
 Requires-Dist: codemie-plugins (>=0.1.123,<0.2.0)
-Requires-Dist: codemie-sdk-python (==0.1.201)
+Requires-Dist: codemie-sdk-python (==0.1.202)
 Requires-Dist: pytest (>=8.4.1,<9.0.0)
 Requires-Dist: pytest-playwright (>=0.7.0,<0.8.0)
 Requires-Dist: pytest-repeat (>=0.9.3,<0.10.0)
@@ -59,6 +59,7 @@ The CLI now provides four main command groups:
 2. **`run`** - Enhanced test execution with flexible parameters
 3. **`assistant`** - Direct assistant interaction and chat capabilities
 4. **`workflow`** - Workflow execution
+5. **`marks`** - List all available pytest marks in the test suite.
 
 Each command group includes extensive help and validation features.
 
@@ -101,6 +102,49 @@ codemie-test-harness config set PYTEST_RERUNS 2
 codemie-test-harness config set PYTEST_COUNT 10  # For performance testing (optional)
 ```
 
+#### Running Tests Locally (Minimal Configuration)
+
+If you're running tests against a **local CodeMie instance** and want to use integration settings already stored in AWS Parameter Store (instead of configuring each integration manually), you only need to set these **4 minimal keys**:
+
+```shell
+# Minimal local setup - uses AWS Parameter Store for all integrations
+codemie-test-harness config set AWS_ACCESS_KEY <your_aws_access_key>
+codemie-test-harness config set AWS_SECRET_KEY <your_aws_secret_key>
+codemie-test-harness config set CODEMIE_API_DOMAIN http://localhost:8080
+codemie-test-harness config set TEST_USER_FULL_NAME "dev-codemie-user"
+```
+
+Then run your tests:
+
+```shell
+# Run all tests locally
+codemie-test-harness run
+
+# Run specific test categories
+codemie-test-harness run --marks smoke
+codemie-test-harness run --marks "api and not not_for_parallel_run" -n 8
+```
+
+**How it works:**
+- The AWS credentials allow the test harness to automatically fetch integration credentials (GitLab, JIRA, Confluence, etc.) from AWS Parameter Store
+- You don't need to manually configure individual integrations unless you want to override specific values
+- All 86+ integration variables are automatically pulled from Parameter Store as needed
+- This is ideal for local development and testing against your local CodeMie backend
+
+**To override specific integrations locally:**
+
+If you need to use your own tokens instead of shared Parameter Store values:
+
+```shell
+# Override with personal GitLab token
+codemie-test-harness config set GITLAB_TOKEN <your_personal_token>
+
+# Override with personal JIRA credentials
+codemie-test-harness config set JIRA_TOKEN <your_personal_token>
+```
+
+Values set explicitly in the config take priority over AWS Parameter Store values.
+
 #### Integration Categories & Management
 
 The CLI supports **10 major integration categories** with comprehensive credential management:
@@ -296,6 +340,35 @@ codemie-test-harness --git-env github run --marks git
 codemie-test-harness run --marks ui --headless
 ```
 
+#### Advanced Marks Usage (Logical Operators)
+
+Combine multiple markers using `and`, `or`, and `not` keywords for fine-grained test selection:
+
+```shell
+# OR operator - run tests with either marker
+codemie-test-harness run --marks "smoke or gitlab" -n 8
+codemie-test-harness run --marks "jira_kb or confluence_kb" -n 6
+
+# AND operator - run tests with both markers
+codemie-test-harness run --marks "api and smoke" -n 10
+codemie-test-harness run --marks "gitlab and code_kb" -n 4
+
+# NOT operator - exclude specific markers
+codemie-test-harness run --marks "api and not ui" -n 10
+codemie-test-harness run --marks "not not_for_parallel_run" -n 12
+
+# Complex combinations with parentheses
+codemie-test-harness run --marks "(smoke or api) and not ui" -n 8
+codemie-test-harness run --marks "(gitlab or github) and not not_for_parallel_run" -n 10
+codemie-test-harness run --marks "smoke and (jira_kb or confluence_kb)" -n 6
+
+# Exclude multiple markers
+codemie-test-harness run --marks "api and not (ui or not_for_parallel_run)" -n 10
+
+# Run all knowledge base tests except code
+codemie-test-harness run --marks "(jira_kb or confluence_kb) and not code_kb" -n 8
+```
+
 #### Performance and Load Testing
 
 Run tests multiple times in parallel to simulate load and measure performance:
@@ -542,7 +615,7 @@ The credentials manager supports **86+ environment variables** across **10 categ
 2) Create a .env file in the project root. If you provide AWS credentials, the suite will fetch additional values from AWS Systems Manager Parameter Store and recreate .env accordingly.
 
 ```properties
-ENV=local
+CODEMIE_API_DOMAIN=http://localhost:8080
 
 AWS_ACCESS_KEY=<aws_access_token>
 AWS_SECRET_KEY=<aws_secret_key>
@@ -594,9 +667,42 @@ pytest -n 10 --count 50 -m excel_generation  # Run 50 times with 10 workers
 pytest -n 20 --count 100 -m smoke --reruns 2  # Heavy load with retries
 ```
 
+**Advanced Marks Usage with Logical Operators:**
+
+Combine markers using `and`, `or`, and `not` for precise test selection:
+
+```shell
+# OR operator - run tests with either marker
+pytest -n 8 -m "smoke or gitlab" --reruns 2
+pytest -n 6 -m "jira_kb or confluence_kb" --reruns 2
+
+# AND operator - run tests with both markers
+pytest -n 10 -m "api and smoke" --reruns 2
+pytest -n 4 -m "gitlab and code_kb" --reruns 2
+
+# NOT operator - exclude specific markers
+pytest -n 10 -m "api and not ui" --reruns 2
+pytest -n 12 -m "not not_for_parallel_run" --reruns 2
+
+# Complex combinations with parentheses
+pytest -n 8 -m "(smoke or api) and not ui" --reruns 2
+pytest -n 10 -m "(gitlab or github) and not not_for_parallel_run" --reruns 2
+pytest -n 6 -m "smoke and (jira_kb or confluence_kb)" --reruns 2
+
+# Exclude multiple markers
+pytest -n 10 -m "api and not (ui or not_for_parallel_run)" --reruns 2
+
+# Run all knowledge base tests except code KB
+pytest -n 8 -m "(jira_kb or confluence_kb) and not code_kb" --reruns 2
+
+# Run all Git-related tests (GitLab or GitHub)
+pytest -n 8 -m "gitlab or github or git" --reruns 2
+```
+
 **Notes:**
 - `--reruns 2` uses pytest-rerunfailures to improve resiliency in flaky environments
 - `--count N` uses pytest-repeat to run each test N times (useful for performance/load testing)
+- Use quotes around marker expressions containing spaces or special characters
 
 #### Test Timeout Configuration
 

{codemie_test_harness-0.1.201 → codemie_test_harness-0.1.202}/README.md

@@ -30,6 +30,7 @@ The CLI now provides four main command groups:
 2. **`run`** - Enhanced test execution with flexible parameters
 3. **`assistant`** - Direct assistant interaction and chat capabilities
 4. **`workflow`** - Workflow execution
+5. **`marks`** - List all available pytest marks in the test suite.
 
 Each command group includes extensive help and validation features.
 
@@ -72,6 +73,49 @@ codemie-test-harness config set PYTEST_RERUNS 2
 codemie-test-harness config set PYTEST_COUNT 10  # For performance testing (optional)
 ```
 
+#### Running Tests Locally (Minimal Configuration)
+
+If you're running tests against a **local CodeMie instance** and want to use integration settings already stored in AWS Parameter Store (instead of configuring each integration manually), you only need to set these **4 minimal keys**:
+
+```shell
+# Minimal local setup - uses AWS Parameter Store for all integrations
+codemie-test-harness config set AWS_ACCESS_KEY <your_aws_access_key>
+codemie-test-harness config set AWS_SECRET_KEY <your_aws_secret_key>
+codemie-test-harness config set CODEMIE_API_DOMAIN http://localhost:8080
+codemie-test-harness config set TEST_USER_FULL_NAME "dev-codemie-user"
+```
+
+Then run your tests:
+
+```shell
+# Run all tests locally
+codemie-test-harness run
+
+# Run specific test categories
+codemie-test-harness run --marks smoke
+codemie-test-harness run --marks "api and not not_for_parallel_run" -n 8
+```
+
+**How it works:**
+- The AWS credentials allow the test harness to automatically fetch integration credentials (GitLab, JIRA, Confluence, etc.) from AWS Parameter Store
+- You don't need to manually configure individual integrations unless you want to override specific values
+- All 86+ integration variables are automatically pulled from Parameter Store as needed
+- This is ideal for local development and testing against your local CodeMie backend
+
+**To override specific integrations locally:**
+
+If you need to use your own tokens instead of shared Parameter Store values:
+
+```shell
+# Override with personal GitLab token
+codemie-test-harness config set GITLAB_TOKEN <your_personal_token>
+
+# Override with personal JIRA credentials
+codemie-test-harness config set JIRA_TOKEN <your_personal_token>
+```
+
+Values set explicitly in the config take priority over AWS Parameter Store values.
+
 #### Integration Categories & Management
 
 The CLI supports **10 major integration categories** with comprehensive credential management:
@@ -267,6 +311,35 @@ codemie-test-harness --git-env github run --marks git
 codemie-test-harness run --marks ui --headless
 ```
 
+#### Advanced Marks Usage (Logical Operators)
+
+Combine multiple markers using `and`, `or`, and `not` keywords for fine-grained test selection:
+
+```shell
+# OR operator - run tests with either marker
+codemie-test-harness run --marks "smoke or gitlab" -n 8
+codemie-test-harness run --marks "jira_kb or confluence_kb" -n 6
+
+# AND operator - run tests with both markers
+codemie-test-harness run --marks "api and smoke" -n 10
+codemie-test-harness run --marks "gitlab and code_kb" -n 4
+
+# NOT operator - exclude specific markers
+codemie-test-harness run --marks "api and not ui" -n 10
+codemie-test-harness run --marks "not not_for_parallel_run" -n 12
+
+# Complex combinations with parentheses
+codemie-test-harness run --marks "(smoke or api) and not ui" -n 8
+codemie-test-harness run --marks "(gitlab or github) and not not_for_parallel_run" -n 10
+codemie-test-harness run --marks "smoke and (jira_kb or confluence_kb)" -n 6
+
+# Exclude multiple markers
+codemie-test-harness run --marks "api and not (ui or not_for_parallel_run)" -n 10
+
+# Run all knowledge base tests except code
+codemie-test-harness run --marks "(jira_kb or confluence_kb) and not code_kb" -n 8
+```
+
 #### Performance and Load Testing
 
 Run tests multiple times in parallel to simulate load and measure performance:
@@ -513,7 +586,7 @@ The credentials manager supports **86+ environment variables** across **10 categ
 2) Create a .env file in the project root. If you provide AWS credentials, the suite will fetch additional values from AWS Systems Manager Parameter Store and recreate .env accordingly.
 
 ```properties
-ENV=local
+CODEMIE_API_DOMAIN=http://localhost:8080
 
 AWS_ACCESS_KEY=<aws_access_token>
 AWS_SECRET_KEY=<aws_secret_key>
@@ -565,9 +638,42 @@ pytest -n 10 --count 50 -m excel_generation  # Run 50 times with 10 workers
 pytest -n 20 --count 100 -m smoke --reruns 2  # Heavy load with retries
 ```
 
+**Advanced Marks Usage with Logical Operators:**
+
+Combine markers using `and`, `or`, and `not` for precise test selection:
+
+```shell
+# OR operator - run tests with either marker
+pytest -n 8 -m "smoke or gitlab" --reruns 2
+pytest -n 6 -m "jira_kb or confluence_kb" --reruns 2
+
+# AND operator - run tests with both markers
+pytest -n 10 -m "api and smoke" --reruns 2
+pytest -n 4 -m "gitlab and code_kb" --reruns 2
+
+# NOT operator - exclude specific markers
+pytest -n 10 -m "api and not ui" --reruns 2
+pytest -n 12 -m "not not_for_parallel_run" --reruns 2
+
+# Complex combinations with parentheses
+pytest -n 8 -m "(smoke or api) and not ui" --reruns 2
+pytest -n 10 -m "(gitlab or github) and not not_for_parallel_run" --reruns 2
+pytest -n 6 -m "smoke and (jira_kb or confluence_kb)" --reruns 2
+
+# Exclude multiple markers
+pytest -n 10 -m "api and not (ui or not_for_parallel_run)" --reruns 2
+
+# Run all knowledge base tests except code KB
+pytest -n 8 -m "(jira_kb or confluence_kb) and not code_kb" --reruns 2
+
+# Run all Git-related tests (GitLab or GitHub)
+pytest -n 8 -m "gitlab or github or git" --reruns 2
+```
+
 **Notes:**
 - `--reruns 2` uses pytest-rerunfailures to improve resiliency in flaky environments
 - `--count N` uses pytest-repeat to run each test N times (useful for performance/load testing)
+- Use quotes around marker expressions containing spaces or special characters
 
 #### Test Timeout Configuration
 

{codemie_test_harness-0.1.201 → codemie_test_harness-0.1.202}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "codemie-test-harness"
-version = "0.1.201"
+version = "0.1.202"
 description = "Autotest for CodeMie backend and UI"
 authors = [
     "Anton Yeromin <anton_yeromin@epam.com>",
@@ -12,7 +12,7 @@ packages = [
 
 [tool.poetry.dependencies]
 python = "^3.12"
-codemie-sdk-python = "0.1.201"
+codemie-sdk-python = "0.1.202"
 pytest = "^8.4.1"
 python-dotenv = "^1.1.0"
 PyHamcrest = "^2.1.0"