PyPI - schemathesis - Versions diffs - 4.0.0a12__tar.gz → 4.0.0b1__tar.gz - Mend

schemathesis 4.0.0a12tar.gz → 4.0.0b1tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (840) hide show

{schemathesis-4.0.0a12 → schemathesis-4.0.0b1}/.github/workflows/build.yml RENAMED Viewed

@@ -40,7 +40,7 @@ jobs:
   tests:
     strategy:
       matrix:
-        os: [ubuntu-22.04, windows-2019]
+        os: [ubuntu-22.04, windows-2022]
         python: ["3.9", "3.10", "3.11", "3.12", "3.13"]
       fail-fast: false
@@ -72,7 +72,7 @@ jobs:
   tests-pipe-with-encoding:
     strategy:
       matrix:
-        os: [ubuntu-22.04, windows-2019]
+        os: [ubuntu-22.04, windows-2022]
       fail-fast: false
     name: Test piping output to a file with CP1252 encoding on ${{ matrix.os }}
     runs-on: ${{ matrix.os }}

{schemathesis-4.0.0a12 → schemathesis-4.0.0b1}/.github/workflows/example-build.yml RENAMED Viewed

@@ -4,7 +4,7 @@ on:
   pull_request:
     paths:
       - ".github/workflows/example-build.yml"
-      - "example/**"
+      - "examples/**"
   push:
     branches:
       - master
@@ -20,19 +20,19 @@ jobs:
       - name: Start containers
         run: docker compose up -d --build
-        working-directory: "./example"
+        working-directory: "./examples/booking"
       # Runs positive Schemathesis tests
       - uses: schemathesis/action@v1
         continue-on-error: true
         with:
           # A local API schema location
-          schema: "http://127.0.0.1:5123/openapi.json"
+          schema: "http://127.0.0.1:8080/openapi.json"
           max-examples: "10"
           # To avoid failing the build
-          args: "-E does-not-exist"
+          args: "--include-path does-not-exist --experimental=openapi-3.1"
       - name: Stop containers
         if: always()
         run: docker compose down
-        working-directory: "./example"
+        working-directory: "./examples/booking"

schemathesis-4.0.0b1/ARCHITECTURE.md ADDED Viewed

@@ -0,0 +1,71 @@
+# Architecture
+This document outlines the internal structure of Schemathesis for developers working on the codebase.
+## Overview
+Schemathesis follows a layered architecture that separates logic into multiple groups:
+- **Core Layer**: Framework-agnostic utilities and data structures
+- **Generation Layer**: Hypothesis-powered test case generation and execution
+- **Interface Layer**: CLI, pytest integration, and user-facing APIs
+- **Engine**: Test orchestration, reporting, and execution management
+## Core Layer
+Framework-agnostic foundation with no external dependencies on testing frameworks.
+### `core/loaders`
+Load and parse API schemas from files, URLs, and applications into internal representations.
+### `core/transports`
+Transport abstractions for HTTP communication, providing unified interfaces for different client libraries.
+### `core/output`
+Output formatting, sanitization, and rendering logic for CLI and reporting.
+### `core/marks`
+Metadata attachment system for integrating with external testing frameworks (pytest, unittest).
+### `core/failures`
+Failure classification and structured error representations for different types of API validation issues.
+### `core/config`
+Configuration management and validation for project settings and runtime options.
+## Generation Layer
+Test case generation and execution built on Hypothesis.
+### `generation/hypothesis`
+Integration with Hypothesis framework:
+- Strategy creation from API schemas
+- Test case generation
+- Example collection and management
+### `generation/case`
+The `Case` data structure containing all test data (headers, body, parameters) for API requests.
+### `generation/stateful`
+State machine implementations for testing API operation sequences using OpenAPI links.
+### `checks`
+Built-in validation checks for API responses (schema conformance, status codes, headers).
+### `hooks`
+Extension system for customizing test generation and execution at various lifecycle points.
+## Internal Subsystems
+### Engine
+- Test execution orchestration
+- Event system for tracking test progress
+- Test phases management
+### API Specifications
+- OpenAPI implementation
+- GraphQL implementation

{schemathesis-4.0.0a12 → schemathesis-4.0.0b1}/CHANGELOG.md RENAMED Viewed

@@ -1,6 +1,56 @@
 # Changelog
-## [Unreleased](https://github.com/schemathesis/schemathesis/compare/v4.0.0-alpha.12...HEAD) - TBD
+## [Unreleased](https://github.com/schemathesis/schemathesis/compare/v4.0.0-beta.1...HEAD) - TBD
+## [4.0.0-beta.1](https://github.com/schemathesis/schemathesis/compare/v4.0.0-alpha.12...v4.0.0-beta.1) - 2025-06-07
+### :rocket: Added
+- Support for validating `requests` / `httpx` / `werkzeug` responses in `Case.validate_response`. [#1718](https://github.com/schemathesis/schemathesis/issues/1718)
+### :wrench: Changed
+- Generate at least one non-NULL character in path parameters. [#2790](https://github.com/schemathesis/schemathesis/issues/2790)
+- Improved failure messages for `unsupported_method` and `missing_required_header` checks.
+- Restore `Case.formatted_path`.
+- Update serialization error messages.
+- Update documentation links in error messages.
+- Improved failure messages for `ignored_auth`. [#2779](https://github.com/schemathesis/schemathesis/issues/2779)
+- Do not show the "Missing test data" warning if stateful testing has non-404 responses. [#2795](https://github.com/schemathesis/schemathesis/issues/2795)
+- Restore support for validating `requests` / `httpx` / `werkzeug` responses in `validate_response` and `is_response_valid`.
+- Rename `is_response_valid` to `is_valid_response` for consistency with other functions in the codebase.
+- Change default values for `query` / `path_parameters` / `headers` / `cookies` to an empty dict instead of `None`.
+- **INTERNAL**: Add `Case.__slots__`.
+### :bug: Fixed
+- Internal error during negative testing in some cases on Open API 3.1.
+- **Pytest**: not using all built-in checks by default.
+- **Pytest**: Ignored `headers` from config file.
+- **Pytest**: Ignored `basic_auth` from config file.
+- **Pytest**: Ignored Hypothesis settings from config file.
+- Make header overrides case-insensitive.
+- Incorrect auth detection in `ignored_auth` when it is passed to `call` or `call_and_validate`. [#2846](https://github.com/schemathesis/schemathesis/issues/2846)
+- Incorrect check for cookie and query param based authorization in `ignored_auth`.
+- Do not include `parameters` in error message about missing HTTP method.
+### :fire: Removed
+- `Schema.add_link`. Adjust your API schema manually instead.
+- `Schema.configure`. Use the config file instead. If you used to pass `app` to `config`, pass it to `Case.call` or `Case.call_and_validate` instead.
+- `@schema.override`. Use `parameters` configuration option instead.
+Change this:
+```python
+@schema.override(path_parameters={"user_id": 42})
+```
+To:
+```toml
+parameters = { user_id = 42 }
+```
 ## [4.0.0-alpha.12](https://github.com/schemathesis/schemathesis/compare/v4.0.0-alpha.11...v4.0.0-alpha.12) - 2025-06-02

schemathesis-4.0.0b1/MIGRATION.md ADDED Viewed

@@ -0,0 +1,139 @@
+# Upgrading from v3 to v4
+Schemathesis v4 is a major reimplementation of the core engine, Python API, and pytest integration. This rebuild enables new features that weren't possible in previous versions while simplifying the overall architecture.
+Some functionality has been removed, either replaced with better alternatives or discontinued entirely.
+The following sections cover migration for both major Schemathesis use cases: CLI and pytest integration.
+The major behavior changes include:
+- Running all data generation modes by default
+- Running all available checks by default
+## Command-Line Interface
+### Removed
+There are a few commands that were removed:
+- `schemathesis replay`. This command had limited usefulness. A better version could be re-introduced in the future.
+- `schemathesis auth` & `schemathesis upload`. Removed as Schemathesis.io has been discontinued.
+The following CLI options were removed without replacement:
+- `--app`. Removed to simplify the core engine architecture.
+- `-A` / `--auth-type`. HTTP Digest authentication was removed. Use `--auth` for HTTP Basic authentication.
+- `--contrib-openapi-formats-uuid`. The `uuid` format is now enabled by default.
+- `--code-sample-style`. Only cURL code samples are now supported.
+- `--debug-output-file`.
+- `--dry-run`. This functionality will be available via a separate CLI command for generating test data.
+- `--experimental`. All experimental features have been stabilized.
+- `--fixups`.
+- `--force-schema-version`.
+- Not useful Hypothesis options - `--hypothesis-{deadline,report-multiple-bugs,verbosity,no-phases}`. Tuning them was not that useful and v4 configures them in the way they don't interfere with the CLI output or testing process.
+- `--show-trace`. Stack traces are now always included in error reports.
+- `--validate-schema`. The output was based on JSON Schema validation errors and was confusing and not helpful most of the time. There are other tools that do this job way better that Schemathesis used to do.
+- `--schemathesis-io-{token,url,telemetry}`, `--hosts-file` and optional `API_NAME` argument - The Schemathesis.io service has been discontinued. The `--report` option now provides different reporting options.
+- `--verbosity`. It didn't really do anything.
+The following options have alternatives:
+| Removed Option | Alternative |
+|----------------|-----------------|
+| `-b` / `--base-url` | `-u` / `--url` |
+| `--cassette-format` | See `--report` below |
+| `--cassette-preserve-exact-body-bytes` | `--report-preserve-bytes` |
+| `--contrib-openapi-fill-missing-examples` | Use `[phases.examples]` configuration table instead |
+| `--contrib-unique-data` | `--generation-unique-inputs` |
+| `--data-generation-methods` | `--mode` |
+| `--endpoint` | `--include-path` / `--exclude-path` |
+| `--exitfirst` | Use `--max-failures=1` instead |
+| `--experimental-no-failfast` | `--continue-on-failure` |
+| `--hypothesis-database` | `--generation-database` |
+| `--hypothesis-derandomize` | `--generation-deterministic` |
+| `--hypothesis-max-examples` | `--max-examples` |
+| `--hypothesis-phases` / `--hypothesis-no-phases` | See `--phases` below |
+| `--hypothesis-seed` | `--generation-seed` |
+| `--hypothesis-suppress-health-check` | `--suppress-health-check` |
+| `--junit-xml` | `--report=junit` |
+| `--method` | `--include-method` / `--exclude-method` |
+| `--operation-id` | `--include-operation-id` / `--exclude-operation-id` |
+| `--pre-run` | Use the `SCHEMATHESIS_HOOKS` environment variable instead |
+| `--request-proxy` | `--proxy` |
+| `--request-tls-verify` | `--tls-verify` |
+| `--sanitize-output` | `--output-sanitize` |
+| `--set-{header,cookie,path,query}` | Use parameter overrides in the configuration file instead |
+| `--skip-deprecated-operations` | `--exclude-deprecated` |
+| `--stateful` | `--phases=stateful` |
+| `--store-network-log` / `--cassette-path` | `--report-vcr-path` |
+| `--tag` | `--include-tag` / `--exclude-tag` |
+| `--targets` | `--generation-maximize` |
+### Reports
+The `--report` CLI option now controls the report formats generated by Schemathesis:
+```bash
+schemathesis run --report=vcr,har <SCHEMA_URL>
+```
+### Test Phases
+The new `--phases` option now controls what test phases are executed by Schemathesis:
+- `examples` (formerly `explicit`): Runs examples specified in the API schema
+- `fuzzing` (formerly `generate`): Testing with randomly generated test cases
+- `coverage`: Deterministic testing of schema constraints and boundary values
+- `stateful`: Chains real API calls, passing data from response to request
+Control over other phases have been replaced by different CLI options:
+- `reuse` and `shrink` remain enabled by default. Disable via `--generation-database=none` and `--no-shrink`.
+- `target` phase available via `--generation-maximize=<METRIC>`
+```bash
+schemathesis run --phases=examples,coverage <SCHEMA_URL>
+```
+## Python API
+### Removed
+- `Schema.add_link`. Adjust your API schema manually instead.
+- `SCHEMATHESIS_VERSION`. Use `__version__` instead.
+- `schemathesis.register_check`. Use `schemathesis.check` instead.
+- `schemathesis.register_target`. Use `schemathesis.metric` instead.
+- `schemathesis.register_string_format`. Use `schemathesis.openapi.format` instead.
+- `schemathesis.GenericResponse`. Use `schemathesis.Response` instead.
+- `schemathesis.from_aiohttp`. Support for `aiohttp` has been removed.
+- `schemathesis.GenerationConfig`. Use config file instead.
+- `schemathesis.HeaderConfig`. Use config file instead.
+- `schemathesis.runner`. The Schemathesis engine is not a public API anymore.
+- `add_case` hook.
+- `SCHEMA_ANALYSIS` experimental feature. Parts of this feature will be open sourced in the future.
+Schema loaders are moved to namespaces:
+| v3 | v4 |
+| -- | -- |
+| `schemathesis.from_asgi` | `schemathesis.openapi.from_asgi` |
+| `schemathesis.from_dict` | `schemathesis.openapi.from_dict` |
+| `schemathesis.from_file` | `schemathesis.openapi.from_file` |
+| `schemathesis.from_path` | `schemathesis.openapi.from_path` |
+| `schemathesis.from_wsgi` | `schemathesis.openapi.from_wsgi` |
+| `schemathesis.from_uri` | `schemathesis.openapi.from_url` |
+| `schemathesis.from_pytest_fixture` | `schemathesis.pytest.from_fixture` |
+### Renamed
+| v3 | v4 |
+| -- | -- |
+| `schemathesis.DataGenerationMethod` | `schemathesis.GenerationMode` |
+| `schemathesis.target` | `schemathesis.metric` |
+| `schemathesis.TargetContext` | `schemathesis.MetricContext` |
+### Changed
+- `AuthProvider.get` now always accepts two arguments (`case`, `ctx`).
+- `@schemathesis.serializer` now expects a function that accepts `ctx` & `value` and returns bytes.

{schemathesis-4.0.0a12 → schemathesis-4.0.0b1}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: schemathesis
-Version: 4.0.0a12
+Version: 4.0.0b1
 Summary: Property-based testing framework for Open API and GraphQL based apps
 Project-URL: Documentation, https://schemathesis.readthedocs.io/en/stable/
 Project-URL: Changelog, https://schemathesis.readthedocs.io/en/stable/changelog.html
@@ -146,6 +146,9 @@ uvx schemathesis run https://your-api.com/openapi.json
 - 🚪 **Validation bypasses** where invalid data gets accepted
 - 🔗 **Integration failures** when responses don't match client expectations
+> ⚠️ **Upgrading from older versions?** Check our [Migration Guide](https://github.com/schemathesis/schemathesis/blob/master/MIGRATION.md) for key changes.
 # Installation & Usage
 **Command Line:**
@@ -181,13 +184,14 @@ Used by teams at **[Spotify](https://github.com/backstage/backstage)**, **[WordP
 ## Documentation
-📚 **[Complete documentation](https://schemathesis.readthedocs.io/)** with guides, examples, and API reference.
+📚 **[Complete documentation](https://schemathesis.readthedocs.io/en/latest/)** with guides, examples, and API reference.
+> **Note:** See also [docs](https://schemathesis.readthedocs.io/en/stable/) for the stable V3 release
 ## Get Help
 - 💬 [Discord community](https://discord.gg/R9ASRAmHnA)
 - 🐛 [GitHub issues](https://github.com/schemathesis/schemathesis/issues)
-- ✉️ [Email support](mailto:support@schemathesis.io)
 ## Contributing

{schemathesis-4.0.0a12 → schemathesis-4.0.0b1}/README.md RENAMED Viewed

@@ -49,6 +49,9 @@ uvx schemathesis run https://your-api.com/openapi.json
 - 🚪 **Validation bypasses** where invalid data gets accepted
 - 🔗 **Integration failures** when responses don't match client expectations
+> ⚠️ **Upgrading from older versions?** Check our [Migration Guide](https://github.com/schemathesis/schemathesis/blob/master/MIGRATION.md) for key changes.
 # Installation & Usage
 **Command Line:**
@@ -84,13 +87,14 @@ Used by teams at **[Spotify](https://github.com/backstage/backstage)**, **[WordP
 ## Documentation
-📚 **[Complete documentation](https://schemathesis.readthedocs.io/)** with guides, examples, and API reference.
+📚 **[Complete documentation](https://schemathesis.readthedocs.io/en/latest/)** with guides, examples, and API reference.
+> **Note:** See also [docs](https://schemathesis.readthedocs.io/en/stable/) for the stable V3 release
 ## Get Help
 - 💬 [Discord community](https://discord.gg/R9ASRAmHnA)
 - 🐛 [GitHub issues](https://github.com/schemathesis/schemathesis/issues)
-- ✉️ [Email support](mailto:support@schemathesis.io)
 ## Contributing

schemathesis-4.0.0b1/docs/configuration.md ADDED Viewed

@@ -0,0 +1,220 @@
+# Configuration
+Most Schemathesis usage works without configuration. Use a config file for authentication, adjusting test generation, or customizing behavior for specific API operations.
+## Quick Start
+### Creating Your First Config File
+Create `schemathesis.toml` in your project directory:
+```toml
+# Run more tests for better coverage
+generation.max-examples = 500
+# Increase timeout for slow APIs
+request-timeout = 10.0
+# Basic authentication
+[auth]
+basic = { username = "${USERNAME}", password = "${PASSWORD}" }
+```
+Set environment variables:
+```bash
+export USERNAME="your_username"
+export PASSWORD="your_password"
+```
+Run Schemathesis - it automatically finds and uses the config:
+```bash
+schemathesis run https://api.example.com/openapi.json
+```
+### Config File Location
+Schemathesis looks for `schemathesis.toml` in:
+1. Current directory
+2. Parent directories (up to project root)
+3. Path specified with `--config-file`
+## Common Configuration Scenarios
+### Authentication
+**API Key in Headers:**
+```toml
+headers = { "X-API-Key" = "${API_KEY}" }
+```
+**Bearer Token:**
+```toml
+headers = { Authorization = "Bearer ${TOKEN}" }
+```
+**Basic Authentication:**
+```toml
+[auth]
+basic = { username = "${USERNAME}", password = "${PASSWORD}" }
+```
+**Different Auth for Specific Operations:**
+```toml
+# Default auth for most operations
+headers = { Authorization = "Bearer ${TOKEN}" }
+# Special auth for admin operations
+[[operations]]
+include-tag = "admin"
+headers = { Authorization = "Bearer ${ADMIN_TOKEN}" }
+```
+### Adjusting Test Generation
+**Run More Tests (Better Coverage, Slower):**
+```toml
+generation.max-examples = 1000
+```
+**Run Fewer Tests (Faster Feedback):**
+```toml
+generation.max-examples = 50
+```
+**Thorough Testing for Important Endpoints:**
+```toml
+# Default: fast testing
+generation.max-examples = 100
+# Critical endpoints: more testing
+[[operations]]
+include-path-regex = "/(payments|users)/"
+generation.max-examples = 500
+```
+### Handling Slow APIs
+**Increase Timeouts:**
+```toml
+request-timeout = 30.0  # 30 seconds
+```
+**Different Timeouts by Operation:**
+```toml
+# Default timeout
+request-timeout = 5.0
+# Slow operations need more time
+[[operations]]
+include-tag = "slow"
+request-timeout = 30.0
+```
+### Environment-Specific Configuration
+**Different Base URLs per Environment:**
+```toml
+base-url = "https://${ENVIRONMENT}.api.example.com"
+```
+```bash
+# Development
+export ENVIRONMENT="dev"
+# Production
+export ENVIRONMENT="prod"
+```
+**Skip Problematic Endpoints in Development:**
+```toml
+[[operations]]
+exclude-path = "/billing/charge"
+enabled = false
+```
+## Advanced Configuration
+**Advanced Filtering with Expressions:**
+```toml
+# Operations without operationId
+[[operations]]
+include-by = "operationId == null"
+enabled = false
+# Operations with specific response descriptions
+[[operations]]
+exclude-by = "responses/200/description != 'Success'"
+checks.response_schema_conformance.enabled = false
+```
+### Test Phase Control
+**Disable Specific Testing Phases:**
+```toml
+[phases]
+# Skip stateful testing for faster runs
+stateful.enabled = false
+# OR: Reduce examples for stateful tests
+stateful.generation.max-examples = 10
+```
+**Phase-Specific Settings per Operation:**
+```toml
+[[operations]]
+include-name = "POST /orders"
+# More examples for fuzzing
+phases.fuzzing.generation.max-examples = 500
+# Fewer for stateful testing
+phases.stateful.generation.max-examples = 20
+```
+### Multi-Project Configuration
+**Testing Multiple APIs:**
+```toml
+# Global defaults
+generation.max-examples = 100
+[[project]]
+title = "Payment API"
+base-url = "https://payments.example.com"
+generation.max-examples = 200
+[[project.operations]]
+include-tag = "critical"
+generation.max-examples = 500
+[[project]]
+title = "User API"
+base-url = "https://users.example.com"
+request-timeout = 2.0
+```
+### Custom Check Configuration
+**Disable Problematic Checks:**
+```toml
+[checks]
+# Disable globally
+content_type_conformance.enabled = false
+# Different expectations for specific operations
+[[operations]]
+include-name = "POST /uploads"
+checks.positive_data_acceptance.expected-statuses = [200, 201, 202]
+```
+**Enable Only Specific Checks:**
+```toml
+[checks]
+enabled = false
+# Only check for server errors and schema compliance
+not_a_server_error.enabled = true
+response_schema_conformance.enabled = true
+```
+## What's next?
+For a complete list of settings, see the [Configuration Reference](reference/configuration.md).

{schemathesis-4.0.0a12 → schemathesis-4.0.0b1}/docs/explanations/data-generation.md RENAMED Viewed

@@ -119,8 +119,8 @@ Schemathesis supports many common media types out of the box, including JSON, XM
 # Generated Python object
 {"user_id": 123, "name": "test"}
-# For application/json → {"user_id": 123, "name": "test"}
-# For application/xml → <data><user_id>123</user_id><name>test</name></data>
+# For application/json -> {"user_id": 123, "name": "test"}
+# For application/xml -> <data><user_id>123</user_id><name>test</name></data>
 ```
 If Schemathesis can't serialize data for a media type, those test cases are skipped. This keeps your test runs focused on actually testable scenarios.

schemathesis 4.0.0a12__tar.gz → 4.0.0b1__tar.gz

schemathesis 4.0.0a12tar.gz → 4.0.0b1tar.gz