quantikmind-cli 0.1.0__tar.gz

quantikmind_cli-0.1.0/PKG-INFO

@@ -0,0 +1,695 @@
+Metadata-Version: 2.4
+Name: quantikmind-cli
+Version: 0.1.0
+Summary: Command-line tools for Quantik Mind test selection and runtime signal collection.
+Author-email: Quantik Mind <info@quantikmind.com>
+Project-URL: Homepage, https://www.quantikmind.com
+Project-URL: Repository, https://github.com/Quantik-Mind/quantikmind-platform
+Project-URL: Issues, https://github.com/Quantik-Mind/quantikmind-platform/issues
+Keywords: quantik-mind,quantikmind,testing,qa,quality-assurance,ci,ci-cd,cli,risk-based-testing,test-selection,observability,devops,devsecops
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Information Technology
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Build Tools
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Utilities
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: click>=8.1
+Requires-Dist: requests>=2.31
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: rich>=13.0
+Provides-Extra: test
+Requires-Dist: pytest>=8.0; extra == "test"
+
+# Quantik Mind CLI
+
+Current release: `0.1.0b1` Free Trial Beta (`0.1.0 Beta`) for early adopters
+and design partners. This is not a stable 1.0 GA contract.
+
+`qmind` connects a repository, CI pipeline, and local Prometheus metrics to
+Quantik Mind. The current CLI supports the core customer integration flow:
+
+```bash
+qmind init
+qmind sync library
+qmind record build
+pytest $(qmind subset) --junitxml=test-results/junit.xml
+qmind record tests --results-dir test-results
+qmind status
+qmind observability configure prometheus --file prometheus-signals.yaml
+qmind observability status
+```
+
+## Installation
+
+Package publishing is not part of this Free Trial Beta readiness PR. For local
+development, free-trial, or early-adopter installs, install the CLI from this
+repository:
+
+```bash
+cd quantikmind-cli
+pip install -e .
+```
+
+This installs the `qmind` command in the active Python environment. A packaged
+install flow can be added after package publishing is available.
+
+## Quickstart
+
+Run these commands from the repository root of the project you want Quantik
+Mind to select tests for.
+
+```bash
+qmind init
+qmind sync library
+qmind status
+```
+
+`qmind init` creates `qmind.yaml` in the repository and stores your API key in
+`~/.qmind/credentials.yaml`. `qmind sync library` scans implemented framework
+types and uploads the functional test library. `qmind status` checks
+connectivity, project setup, signal readiness, and subscription usage.
+
+## Project-Scoped API Keys
+
+The full CLI workflow is designed to run with one project-scoped API key. In
+the Settings API Access UI, create a key with the `CLI full project workflow`
+preset for the target project.
+
+Required scopes for that preset:
+
+```text
+library:read
+library:write
+build:read
+build:write
+selection:read
+selection:run
+results:read
+results:write
+history:read
+history:write
+observability:read
+observability:write
+```
+
+Project-scoped keys cannot create projects, cannot access other projects, and
+do not grant organization-admin permissions. Revoke or rotate keys from the
+same API Access screen. After rotating, replace `QMIND_API_KEY` in CI secrets
+or rerun `qmind init --api-key <new-key> --project-id <project-id>`.
+
+For CI or non-interactive setup, pass values explicitly or use environment
+variables:
+
+```bash
+qmind init \
+  --api-key "$QMIND_API_KEY" \
+  --api-url "$QMIND_API_URL" \
+  --project-id "$QMIND_PROJECT_ID" \
+  --framework pytest \
+  --test-dir tests \
+  --results-dir test-results \
+  --non-interactive
+```
+
+For an existing project, `--project-id` binds the local repository to that
+project and validates the key against it. It does not create a project:
+
+```bash
+qmind init \
+  --api-key "$QMIND_API_KEY" \
+  --api-url "http://localhost:8000" \
+  --project-id "$QMIND_PROJECT_ID" \
+  --framework pytest \
+  --non-interactive
+```
+
+You can also use an explicit YAML or JSON library file instead of scanner
+discovery:
+
+```bash
+qmind sync library --file ./test-library.yaml --dry-run --output ./qmind-library.preview.json
+qmind sync library --file ./test-library.yaml
+```
+
+Manifest duration fields are preserved for savings metrics. Use any one of
+`avg_duration_ms`, `duration_ms`, `estimated_duration_ms`, or
+`constraints.estimated_duration_ms`; the CLI uploads them as `avg_duration_ms`.
+
+## Recommended CI Flow
+
+Record the build, request the selected subset, run your existing test runner,
+then import JUnit XML results:
+
+```bash
+qmind record build
+pytest $(qmind subset) --junitxml=test-results/junit.xml
+qmind record tests --results-dir test-results
+```
+
+During early rollout, use a fallback policy that matches your risk tolerance.
+Many teams start by falling back to the full suite when selection is unavailable
+or no tests are selected.
+
+`qmind record build` reads Git metadata by default. Outside a git repository,
+pass explicit metadata:
+
+```bash
+qmind record build --commit "$BUILD_SHA" --branch "$BRANCH_NAME" --message "$BUILD_NAME"
+```
+
+`qmind record tests --results-dir` imports JUnit XML. CSV history import is not
+supported yet.
+
+## Project Observability
+
+Prometheus is configured per project. The default flow is URL-only. Quantik
+Mind connects to Prometheus, tries to auto-detect a supported metric preset,
+and creates the project observability profile when detection succeeds.
+
+```bash
+qmind observability configure prometheus --url "http://prometheus:9090"
+qmind observability status
+```
+
+If detection is partial, the CLI shows which preset signals were detected and
+which were missing. Custom PromQL mapping is the advanced fallback:
+
+```bash
+qmind observability configure prometheus --file prometheus-signals.yaml
+```
+
+Supported runtime signal names are `error_rate`, `latency_p95`,
+`request_rate`, and `cpu`/`cpu_load`.
+
+Example:
+
+```yaml
+observability:
+  provider: prometheus
+  url: http://prometheus:9090
+  service_label: service
+  signals:
+    error_rate:
+      query: |
+        sum(rate(http_requests_total{status=~"5.."}[5m])) by (service)
+        /
+        sum(rate(http_requests_total[5m])) by (service)
+    latency_p95:
+      query: |
+        histogram_quantile(
+          0.95,
+          sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service)
+        )
+    request_rate:
+      query: |
+        sum(rate(http_requests_total[5m])) by (service)
+```
+
+`qmind status` reports runtime/observability as `not configured` until a
+project-level Prometheus config or fresh project agent metrics are present.
+
+## Test Runner Examples
+
+### pytest
+
+```bash
+qmind record build
+mkdir -p test-results
+pytest $(qmind subset --framework pytest) --junitxml=test-results/junit.xml
+qmind record tests --results-dir test-results
+```
+
+### jest
+
+```bash
+qmind record build
+mkdir -p test-results
+npx jest $(qmind subset --framework jest) --ci --reporters=default --reporters=jest-junit
+qmind record tests --results-dir test-results
+```
+
+Configure `jest-junit` so it writes JUnit XML into `test-results`, for example
+with `JEST_JUNIT_OUTPUT_DIR=test-results`.
+
+### Maven
+
+```bash
+qmind record build
+mvn test $(qmind subset --framework maven)
+qmind record tests --results-dir target/surefire-reports
+```
+
+### Gradle
+
+```bash
+qmind record build
+./gradlew test $(qmind subset --framework gradle)
+qmind record tests --results-dir build/test-results
+```
+
+## CI Examples
+
+### GitHub Actions
+
+Store `QMIND_API_KEY`, `QMIND_API_URL`, and `QMIND_PROJECT_ID` as repository or
+environment secrets.
+
+```yaml
+name: qmind-tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    env:
+      QMIND_API_KEY: ${{ secrets.QMIND_API_KEY }}
+      QMIND_API_URL: ${{ secrets.QMIND_API_URL }}
+      QMIND_PROJECT_ID: ${{ secrets.QMIND_PROJECT_ID }}
+      QMIND_FRAMEWORK: pytest
+      QMIND_TEST_DIR: tests
+      QMIND_RESULTS_DIR: test-results
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: |
+          pip install -e quantikmind-cli
+          pip install -r requirements.txt
+
+      - name: Configure Quantik Mind
+        run: qmind init --non-interactive
+
+      - name: Sync test library
+        run: qmind sync library
+
+      - name: Run selected tests
+        run: |
+          qmind record build
+          mkdir -p test-results
+          pytest $(qmind subset) --junitxml=test-results/junit.xml
+          qmind record tests --results-dir test-results
+```
+
+### GitLab CI
+
+Set `QMIND_API_KEY`, `QMIND_API_URL`, and `QMIND_PROJECT_ID` as protected CI/CD
+variables.
+
+```yaml
+qmind_tests:
+  image: python:3.11
+  stage: test
+  variables:
+    QMIND_FRAMEWORK: pytest
+    QMIND_TEST_DIR: tests
+    QMIND_RESULTS_DIR: test-results
+  script:
+    - pip install -e quantikmind-cli
+    - pip install -r requirements.txt
+    - qmind init --non-interactive
+    - qmind sync library
+    - qmind record build
+    - mkdir -p test-results
+    - pytest $(qmind subset) --junitxml=test-results/junit.xml
+    - qmind record tests --results-dir test-results
+```
+
+### Jenkins
+
+Store the API key in Jenkins credentials and provide the API URL and project ID
+as environment variables or folder-level configuration.
+
+```groovy
+pipeline {
+  agent any
+
+  environment {
+    QMIND_API_URL = credentials('qmind-api-url')
+    QMIND_PROJECT_ID = credentials('qmind-project-id')
+    QMIND_FRAMEWORK = 'pytest'
+    QMIND_TEST_DIR = 'tests'
+    QMIND_RESULTS_DIR = 'test-results'
+  }
+
+  stages {
+    stage('Test') {
+      steps {
+        withCredentials([string(credentialsId: 'qmind-api-key', variable: 'QMIND_API_KEY')]) {
+          sh '''
+            pip install -e quantikmind-cli
+            pip install -r requirements.txt
+            qmind init --non-interactive
+            qmind sync library
+            qmind record build
+            mkdir -p test-results
+            pytest $(qmind subset) --junitxml=test-results/junit.xml
+            qmind record tests --results-dir test-results
+          '''
+        }
+      }
+    }
+  }
+}
+```
+
+## Runtime Signals Via Agent
+
+The Prometheus agent runs inside the customer network. It reads local
+Prometheus and pushes selected runtime metrics outbound to Quantik Mind over
+HTTPS:
+
+```bash
+qmind agent init --prometheus-url http://localhost:9090 --push-interval 30
+qmind agent start
+```
+
+No inbound firewall rule is required for this agent flow. Quantik Mind reads
+the agent-pushed metrics cache first. If the cache is empty, stale, or
+unavailable, Quantik Mind falls back to direct Prometheus access when that
+direct path has been configured separately.
+
+`qmind agent start` is a foreground process. Run it under your existing process
+supervisor if you need long-running behavior today. The CLI does not currently
+install a systemd unit, launchd service, Docker container, or other daemon
+wrapper.
+
+## Command Reference
+
+### `qmind init`
+
+Creates repository configuration and stores local credentials.
+
+Common options:
+
+- `--api-key` or `QMIND_API_KEY`
+- `--api-url` or `QMIND_API_URL`
+- `--project-id` or `QMIND_PROJECT_ID`
+- `--project-name` or `QMIND_PROJECT_NAME`
+- `--framework` or `QMIND_FRAMEWORK`
+- `--test-dir` or `QMIND_TEST_DIR`
+- `--results-dir` or `QMIND_RESULTS_DIR`
+- `--non-interactive`
+
+### `qmind sync library`
+
+Scans the configured test directory and uploads the functional test library.
+Current scanner support includes `pytest`, `jest`, `maven`, `gradle`,
+`cypress`, `playwright`, and `robot`.
+
+Optional overrides:
+
+```bash
+qmind sync library --framework pytest --test-dir tests
+```
+
+For proprietary or non-discoverable enterprise test suites, manifest import is
+the recommended path. This includes TestComplete projects, custom enterprise
+runners, Selenium/Appium wrappers, legacy suites, and internal tools where
+scanner discovery cannot reliably infer the complete inventory.
+
+```bash
+qmind sync library --manifest qmind-tests.yaml
+```
+
+The manifest may be YAML or JSON and uses the same upload path as scanner-based
+sync. `qmind sync library --manifest ...` reads `project_id` and `api_url` from
+`qmind.yaml`, loads credentials from `~/.qmind/credentials.yaml`, skips scanner
+discovery, and uploads the declared tests.
+
+Example TestComplete/custom runner manifest:
+
+```yaml
+tests:
+  - test_id: Login.ValidUser
+    name: Valid user login
+    framework: testcomplete
+    service: auth-service
+    suite: LoginSuite
+    business_criticality: critical
+    repo_ref:
+      selector: LoginSuite.ValidUser
+      path: TestComplete/LoginSuite
+    code_mapping:
+      file_globs:
+        - src/auth/**
+    tags:
+      - smoke
+      - login
+```
+
+Required fields are `test_id`, `name`, `framework`, and `service`. Optional
+fields are `suite`, `business_criticality`, `repo_ref`, `code_mapping`, `tags`,
+and `metadata`. `suite` defaults to `default`; `business_criticality` defaults
+to `medium` and must be one of `low`, `medium`, `high`, or `critical`.
+`repo_ref`, `code_mapping`, and `tags` are preserved in the upload payload when
+present. `metadata` is accepted in the manifest for forward compatibility but is
+not uploaded because the current library upload contract does not document a
+metadata field.
+
+### `qmind record build`
+
+Records the current git commit, message, author, and changed files for the
+configured project.
+
+```bash
+qmind record build
+```
+
+### `qmind subset`
+
+Requests the selected tests and prints canonical `test_id` values to stdout.
+Selection stats are printed to stderr so command substitution remains usable.
+Use `--internal-ids` only when you explicitly need backend numeric IDs for
+diagnostics.
+
+```bash
+pytest $(qmind subset --framework pytest)
+mvn test $(qmind subset --framework maven)
+./gradlew test $(qmind subset --framework gradle)
+```
+
+Use `--json` when another tool needs the complete machine-readable selection
+payload, including Dynamic Risk Intelligence metrics returned by the backend:
+
+```bash
+qmind subset --json
+```
+
+Example:
+
+```json
+{
+  "selected_tests": [
+    "tests/test_checkout.py::test_checkout_pays",
+    "tests/test_checkout.py::TestInvoices::test_invoice_renders"
+  ],
+  "business_metrics": {
+    "risk_coverage": 84.2,
+    "top_risk_coverage": 91.5,
+    "residual_risk": 15.8,
+    "risk_efficiency": 1.68
+  },
+  "risk_coverage": 84.2,
+  "risk_efficiency": 1.68
+}
+```
+
+### `qmind record tests`
+
+Imports JUnit XML test results.
+
+```bash
+qmind record tests --results-dir test-results
+```
+
+If `--results-dir` is omitted, the CLI uses `results_dir` from `qmind.yaml`.
+History import requires each JUnit testcase to resolve to a `test_id` that was
+already uploaded by `qmind sync library` or manifest import. The CLI normalizes
+supported runner output to the scanner ID shape when the report preserves enough
+identity:
+
+- pytest dotted `classname` values such as `tests.test_login` plus
+  `name="test_valid_login"` become `tests/test_login.py::test_valid_login`.
+- pytest class cases such as `tests.test_login.TestLogin` become
+  `tests/test_login.py::TestLogin::test_valid_login`.
+- Maven/Gradle Surefire-style `classname="com.acme.InvoiceTest"` plus
+  `name="createsInvoice"` becomes `InvoiceTest.createsInvoice`.
+- Cypress, Playwright, Jest, and Robot reports match scanner IDs when
+  `classname` or `file` contains the spec/test path, for example
+  `tests/checkout/payment.spec.ts::card payment succeeds`.
+
+Generic JUnit XML that only contains a suite name and a testcase name remains
+deterministic, but may not match scanner-generated IDs. Configure the runner's
+JUnit reporter to include the file path or use a manifest with explicit
+`test_id` values for custom runners.
+
+### `qmind status`
+
+Shows project configuration, API connectivity, signal readiness, selection
+readiness, and subscription usage.
+
+```bash
+qmind status
+```
+
+### `qmind agent init`
+
+Validates local Prometheus connectivity, checks expected metric names, and
+writes agent configuration into `qmind.yaml`.
+
+```bash
+qmind agent init --prometheus-url http://localhost:9090 --push-interval 30
+```
+
+### `qmind agent start`
+
+Starts the foreground metrics push loop.
+
+```bash
+qmind agent start
+```
+
+For a single collect-and-push cycle:
+
+```bash
+qmind agent start --once
+```
+
+### `qmind agent status`
+
+Shows local agent configuration and the last successful push recorded by the
+CLI.
+
+```bash
+qmind agent status
+```
+
+### `qmind agent stop`
+
+`qmind agent stop` is present but does not manage a real daemon yet. Stop a
+foreground `qmind agent start` process with `Ctrl+C` or through the external
+process supervisor you used to launch it.
+
+## Framework Support Matrix
+
+| Capability | Current support |
+| --- | --- |
+| Test library scanner | `pytest`, `jest`, `maven`, `gradle`, `cypress`, `playwright`, `robot` |
+| Subset argument formatting | `pytest`, `jest`, `maven`, `gradle`, `cypress`, `playwright`, `rspec`, `cucumber`, `robot` |
+| Result import | JUnit XML with scanner-compatible IDs for pytest, Maven/Gradle Surefire, and path-preserving JS/Robot reporters; deterministic fallback for generic JUnit XML |
+
+Selenium and Appium suites are typically covered through pytest, Maven, Gradle,
+JUnit, or TestNG-style runners rather than native Selenium/Appium scanning.
+TestComplete, proprietary runners, and non-discoverable enterprise tools should
+use manifest import or JUnit-compatible reports; native scanners are not
+implemented for those tools today.
+
+## Security Notes
+
+- API keys are stored in `~/.qmind/credentials.yaml`.
+- API keys are not stored in `qmind.yaml`.
+- `qmind.yaml` is repository-safe project configuration.
+- The agent uses outbound HTTPS to push metrics to Quantik Mind.
+- The agent flow does not require inbound access to the customer network.
+- Tenant and project isolation is enforced server-side.
+
+## Current Limitations
+
+- Package publishing is not available yet; use `pip install -e .`.
+- `qmind sync library` scans pytest, jest, Maven, Gradle, Cypress, Playwright,
+  and Robot Framework projects.
+- JUnit XML is the current result import format.
+- `qmind agent start` runs in the foreground and is not a service installer.
+- `qmind agent stop` does not stop a managed daemon.
+- Native TestComplete, Appium, Selenium, and proprietary runner scanners are not
+  implemented by this CLI release; use manifest import for those suites.
+- Direct Prometheus fallback requires a separately configured direct path; the
+  local agent itself uses outbound HTTPS only.
+
+## Troubleshooting
+
+### Missing `qmind.yaml`
+
+Run commands from the repository root after initialization:
+
+```bash
+qmind init
+```
+
+### Missing Credentials
+
+If credentials are missing or invalid, rerun initialization with a valid API
+key:
+
+```bash
+qmind init --api-key "$QMIND_API_KEY"
+```
+
+Credentials are stored in `~/.qmind/credentials.yaml`.
+
+### No Tests Found
+
+Check that `framework` and `test_dir` in `qmind.yaml` match the repository:
+
+```bash
+qmind sync library --framework pytest --test-dir tests
+```
+
+Only pytest, jest, Maven, and Gradle scanning are implemented today.
+
+### No Selected Tests
+
+`qmind subset` exits with an error when Quantik Mind returns no selected tests.
+Check `qmind status` for library and signal readiness. In CI, use an explicit
+fallback to the full suite if that is your rollout policy.
+
+### No JUnit Results Found
+
+Confirm the runner writes JUnit XML into the directory passed to
+`qmind record tests`:
+
+```bash
+pytest $(qmind subset) --junitxml=test-results/junit.xml
+qmind record tests --results-dir test-results
+```
+
+### Prometheus Unavailable
+
+Verify Prometheus is reachable from the machine running the agent:
+
+```bash
+qmind agent init --prometheus-url http://localhost:9090 --push-interval 30
+```
+
+The command validates connectivity before saving agent settings.
+
+### Agent Metrics Endpoint Unavailable
+
+`qmind agent start` logs endpoint errors and backs off between retries. Confirm
+the API URL, credentials, and backend deployment expose
+`POST /api/v1/agent/metrics`.
+
+### Weak Signal Readiness
+
+`qmind status` may report partial or not-ready selection readiness when the
+project has little historical or runtime signal. Sync the test library, record
+builds, import JUnit XML results, and start the Prometheus agent to improve
+signal coverage.