schemathesis 4.0.7__tar.gz → 4.0.8__tar.gz

{schemathesis-4.0.7 → schemathesis-4.0.8}/.github/workflows/release.yml

@@ -32,6 +32,22 @@ jobs:
           fi
         shell: bash
 
+
+      - name: Generate stable tag
+        id: stable-tag
+        run: |
+          if [ "${{ env.is_prerelease }}" == "false" ]; then
+            if [ "${{ matrix.tag-suffix }}" == "" ]; then
+              echo "stable_tag=type=raw,value=stable" >> $GITHUB_OUTPUT
+            elif [ "${{ matrix.tag-suffix }}" == "-bookworm" ]; then
+              echo "stable_tag=type=raw,value=stable-bookworm" >> $GITHUB_OUTPUT
+            else
+              echo "stable_tag=" >> $GITHUB_OUTPUT
+            fi
+          else
+            echo "stable_tag=" >> $GITHUB_OUTPUT
+          fi
+
       - name: Docker meta
         id: meta
         uses: docker/metadata-action@v5
@@ -44,7 +60,13 @@ jobs:
           tags: |
             type=semver,pattern={{raw}}${{ matrix.tag-suffix }}
             type=semver,pattern={{version}}${{ matrix.tag-suffix }}
-            ${{ env.is_prerelease == 'false' && 'type=raw,value=stable${{ matrix.tag-suffix }}' || '' }}
+            ${{ steps.stable-tag.outputs.stable_tag }}
+
+      - name: Build metadata
+        run: |
+          echo "Tags: ${{ steps.meta.outputs.tags }}"
+          echo "Is prerelease: ${{ env.is_prerelease }}"
+          echo "Tag suffix: '${{ matrix.tag-suffix }}'"
 
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v3

{schemathesis-4.0.7 → schemathesis-4.0.8}/CHANGELOG.md

@@ -1,6 +1,15 @@
 # Changelog
 
-## [Unreleased](https://github.com/schemathesis/schemathesis/compare/v4.0.7...HEAD) - TBD
+## [Unreleased](https://github.com/schemathesis/schemathesis/compare/v4.0.8...HEAD) - TBD
+
+## [4.0.8](https://github.com/schemathesis/schemathesis/compare/v4.0.7...v4.0.8) - 2025-07-12
+
+### :bug: Fixed
+
+- Use `utf-8` encoding when generating JUnit reports.
+- Properly display Schemathesis version in CLI.
+- Test case metadata containing incorrect info about generated components causing false positive failures in `negative_data_rejection` and `positive_data_acceptance` checks. [#2900](https://github.com/schemathesis/schemathesis/issues/2900)
+- Negative test cases with invalid query parameters mistakenly becoming positive test cases.
 
 ## [4.0.7](https://github.com/schemathesis/schemathesis/compare/v4.0.6...v4.0.7) - 2025-07-08
 

{schemathesis-4.0.7 → schemathesis-4.0.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: schemathesis
-Version: 4.0.7
+Version: 4.0.8
 Summary: Property-based testing framework for Open API and GraphQL based apps
 Project-URL: Documentation, https://schemathesis.readthedocs.io/en/stable/
 Project-URL: Changelog, https://github.com/schemathesis/schemathesis/blob/master/CHANGELOG.md

{schemathesis-4.0.7 → schemathesis-4.0.8}/docs/faq.md

@@ -5,10 +5,10 @@
 Schemathesis generates three types of data:
 
 - **Schema examples** from your API documentation
-- **Valid test data** that follows schema constraints  
+- **Valid test data** that follows schema constraints
 - **Invalid test data** that deliberately breaks constraints
 
-The data covers all JSON Schema types for OpenAPI and valid queries for GraphQL. 
+The data covers all JSON Schema types for OpenAPI and valid queries for GraphQL.
 
 Note, that some generated data may be rejected by your API if the validation rules are not expressed in your schema.
 
@@ -16,22 +16,27 @@ Note, that some generated data may be rejected by your API if the validation rul
 
 Schemathesis identifies problems in three main categories:
 
-**API Contract Violations**
+**Schema Compliance Issues**
 
-- Responses not matching documented schemas
+- Response bodies not matching schemas
 - Undocumented status codes
-- Missing or incorrect content types
+- Missing required headers
+- Wrong content types
 
 **Implementation Flaws**
 
-- Server errors (5xx responses)
-- Data validation issues (accepting invalid data or rejecting valid data)
+- Server crashes (5xx responses)
+- Accepting invalid data
+- Rejecting valid data
+- Allowing missing required headers
 - Authentication bypasses
 
 **Stateful Behavior Issues**
 
-- Resources accessible after deletion
-- Resources unavailable after creation
+- Deleted resources still accessible
+- Created resources not available
+
+See more details in the [Checks reference](reference/checks.md).
 
 ## How should I run Schemathesis?
 

schemathesis-4.0.8/docs/reference/checks.md

@@ -0,0 +1,229 @@
+# Checks
+
+Schemathesis automatically validates API responses using various checks. Each check targets specific aspects of API behavior and its compliance to the specification.
+
+## Core Response Validation
+
+These checks validate that API responses conform to your API schema.
+
+### `not_a_server_error`
+
+Catches server-side errors and implementation issues
+
+**Examples** 
+
+- `500 Internal Server Error`
+- `502 Bad Gateway` 
+- GraphQL response with `"errors": [{...}]`
+
+---
+
+### `status_code_conformance`
+
+Verifies response status codes match schema documentation
+
+**Examples**
+
+- Schema documents `200, 404` but API returns `403`
+- Schema documents `201` but API returns `200`
+
+---
+
+### `content_type_conformance`
+
+Validates `Content-Type` header matches schema
+
+**Examples**
+
+- Schema specifies `application/json` but API returns `text/html`
+- Schema specifies `application/xml` but API returns `application/json`
+
+---
+
+### `response_headers_conformance`
+
+Ensures required response headers are present and valid
+
+**Example**
+
+Schema specifies response must include `X-RateLimit-Remaining` header:
+
+```http
+GET /api/data
+
+Response: 200 OK
+Content-Type: application/json
+# Missing required X-RateLimit-Remaining header ❌
+
+{"data": [...]}
+```
+
+---
+
+### `response_schema_conformance`
+
+Validates response body against JSON Schema
+
+**Example**
+
+Schema defines user object as:
+
+```yaml
+type: object
+required: [id, name, email]
+properties:
+  id: {type: integer}
+  name: {type: string}
+  email: {type: string, format: email}
+```
+
+But API returns:
+
+```http
+GET /users/789
+
+Response: 200 OK
+{
+  "id": "789",          ❌ Should be integer, not string
+  "name": "Eve"         ❌ Missing required 'email' field
+}
+```
+
+## Input Handling Validation
+
+These checks verify how your API processes different types of request data.
+
+### `negative_data_rejection`
+
+Verifies API rejects invalid data with appropriate error responses
+
+**Example**
+
+Schema specifies `age` must be an integer:
+
+```http
+POST /users
+Content-Type: application/json
+
+{"name": "Charlie", "age": "twenty-five"}
+
+Response: 200 OK  ❌ Should return 400 Bad Request
+{"id": 456, "name": "Charlie", "age": null}
+```
+
+---
+
+### `positive_data_acceptance`
+
+Verifies API accepts valid request data
+
+**Example**
+
+Request perfectly matches schema:
+
+```http
+POST /users
+Content-Type: application/json
+
+{"name": "Diana", "age": 25, "email": "diana@example.com"}
+
+Response: 400 Bad Request  ❌ Should accept valid data
+{"error": "Invalid request"}
+```
+
+---
+
+### `missing_required_header`
+
+Verifies APIs return 4xx for missing required headers
+
+**Example**
+
+Schema requires `X-API-Key` header, but:
+
+```http
+GET /protected-resource
+# Missing required X-API-Key header
+
+Response: 200 OK  ❌ Should return 400 Bad Request or 401 Unauthorized
+```
+
+---
+
+### `unsupported_method`
+
+Verifies APIs return 405 for undocumented HTTP methods
+
+**Example**
+
+Schema only documents `GET /users/{id}` and `PUT /users/{id}`, but:
+
+```http
+PATCH /users/123
+Content-Type: application/json
+
+{"name": "Bob"}
+
+Response: 200 OK  ❌ Should return 405 Method Not Allowed
+```
+
+## Stateful Behavior
+
+These checks test API behavior across sequences of operations.
+
+### `use_after_free`
+
+Detects when deleted resources remain accessible
+
+**Example**
+
+```http
+DELETE /users/123
+
+Response: 204 No Content
+```
+
+Then:
+
+```http
+GET /users/123
+
+Response: 200 OK  ❌ Should return 404 Not Found
+```
+
+---
+
+### `ensure_resource_availability`
+
+Verifies created resources are immediately accessible
+
+**Example**
+
+```http
+POST /users
+Content-Type: application/json
+
+{"name": "Alice", "email": "alice@example.com"}
+
+Response: 201 Created
+Location: /users/123
+```
+
+Then immediately:
+
+```http
+GET /users/123
+
+Response: 404 Not Found  ❌ Should return the created user
+```
+
+## Security
+
+### `ignored_auth`
+
+Verifies authentication is properly enforced
+
+**Example**
+
+- Endpoint requires authentication but accepts requests without auth headers
+- Returns `200 OK` instead of `401 Unauthorized`

{schemathesis-4.0.7 → schemathesis-4.0.8}/docs/reference/hooks.md

@@ -10,8 +10,6 @@
 | **before\_init\_operation** | `(context, operation: APIOperation) -> None`  | Adjust an `APIOperation` before building strategies (e.g., restrict enum choices, set defaults). |
 | **before\_add\_examples**   | `(context, examples: list[Case]) -> None`     | Append explicit `Case` instances before the `examples` phase.                              |
 
-## Data-Generation Hooks
-
 ## Request-Lifecycle Hooks
 
 Mutate or inspect the test case and HTTP request flow.

{schemathesis-4.0.7 → schemathesis-4.0.8}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "schemathesis"
-version = "4.0.7"
+version = "4.0.8"
 description = "Property-based testing framework for Open API and GraphQL based apps"
 keywords = ["pytest", "hypothesis", "openapi", "graphql", "testing"]
 authors = [{ name = "Dmitry Dygalo", email = "dmitry@dygalo.dev" }]

{schemathesis-4.0.7 → schemathesis-4.0.8}/src/schemathesis/cli/commands/run/handlers/junitxml.py

@@ -40,7 +40,7 @@ class JunitXMLHandler(EventHandler):
             test_suites = [
                 TestSuite("schemathesis", test_cases=list(self.test_cases.values()), hostname=platform.node())
             ]
-            with open(self.path, "w") as fd:
+            with open(self.path, "w", encoding="utf-8") as fd:
                 to_xml_report_file(file_descriptor=fd, test_suites=test_suites, prettyprint=True, encoding="utf-8")
 
     def get_or_create_test_case(self, label: str) -> TestCase:

{schemathesis-4.0.7 → schemathesis-4.0.8}/src/schemathesis/core/version.py

@@ -1,7 +1,7 @@
 from importlib import metadata
 
 try:
-    SCHEMATHESIS_VERSION = metadata.version(__package__)
+    SCHEMATHESIS_VERSION = metadata.version("schemathesis")
 except metadata.PackageNotFoundError:
     # Local run without installation
     SCHEMATHESIS_VERSION = "dev"

{schemathesis-4.0.7 → schemathesis-4.0.8}/src/schemathesis/specs/openapi/_hypothesis.py

@@ -14,7 +14,7 @@ from hypothesis_jsonschema import from_schema
 from requests.structures import CaseInsensitiveDict
 
 from schemathesis.config import GenerationConfig
-from schemathesis.core import NOT_SET, NotSet, media_types
+from schemathesis.core import NOT_SET, media_types
 from schemathesis.core.control import SkipTest
 from schemathesis.core.errors import SERIALIZERS_SUGGESTION_MESSAGE, SerializationNotPossible
 from schemathesis.core.transforms import deepclone
@@ -57,10 +57,10 @@ def openapi_cases(
     hooks: HookDispatcher | None = None,
     auth_storage: auths.AuthStorage | None = None,
     generation_mode: GenerationMode = GenerationMode.POSITIVE,
-    path_parameters: NotSet | dict[str, Any] = NOT_SET,
-    headers: NotSet | dict[str, Any] = NOT_SET,
-    cookies: NotSet | dict[str, Any] = NOT_SET,
-    query: NotSet | dict[str, Any] = NOT_SET,
+    path_parameters: dict[str, Any] | None = None,
+    headers: dict[str, Any] | None = None,
+    cookies: dict[str, Any] | None = None,
+    query: dict[str, Any] | None = None,
     body: Any = NOT_SET,
     media_type: str | None = None,
     phase: TestPhase = TestPhase.FUZZING,
@@ -220,7 +220,7 @@ def _get_body_strategy(
 
 
 def get_parameters_value(
-    value: NotSet | dict[str, Any],
+    value: dict[str, Any] | None,
     location: str,
     draw: Callable,
     operation: APIOperation,
@@ -234,7 +234,7 @@ def get_parameters_value(
     If the value is not set, then generate it from the relevant strategy. Otherwise, check what is missing in it and
     generate those parts.
     """
-    if isinstance(value, NotSet) or not value:
+    if value is None:
         strategy = get_parameters_strategy(operation, strategy_factory, location, generation_config)
         strategy = apply_hooks(operation, ctx, hooks, strategy, location)
         return draw(strategy)
@@ -274,7 +274,7 @@ def any_negated_values(values: list[ValueContainer]) -> bool:
 
 def generate_parameter(
     location: str,
-    explicit: NotSet | dict[str, Any],
+    explicit: dict[str, Any] | None,
     operation: APIOperation,
     draw: Callable,
     ctx: HookContext,

{schemathesis-4.0.7 → schemathesis-4.0.8}/src/schemathesis/specs/openapi/negative/mutations.py

@@ -223,6 +223,7 @@ def change_type(context: MutationContext, draw: Draw, schema: Schema) -> Mutatio
     if len(candidates) == 1:
         new_type = candidates.pop()
         schema["type"] = new_type
+        _ensure_query_serializes_to_non_empty(context, schema)
         prevent_unsatisfiable_schema(schema, new_type)
         return MutationResult.SUCCESS
     # Choose one type that will be present in the final candidates list
@@ -235,10 +236,18 @@ def change_type(context: MutationContext, draw: Draw, schema: Schema) -> Mutatio
     ]
     new_type = draw(st.sampled_from(remaining_candidates))
     schema["type"] = new_type
+    _ensure_query_serializes_to_non_empty(context, schema)
     prevent_unsatisfiable_schema(schema, new_type)
     return MutationResult.SUCCESS
 
 
+def _ensure_query_serializes_to_non_empty(context: MutationContext, schema: Schema) -> None:
+    if context.is_query_location and schema.get("type") == "array":
+        # Query parameters with empty arrays or arrays of `None` or empty arrays / objects will not appear in the final URL
+        schema["minItems"] = schema.get("minItems") or 1
+        schema.setdefault("items", {}).update({"not": {"enum": [None, [], {}]}})
+
+
 def _get_type_candidates(context: MutationContext, schema: Schema) -> set[str]:
     types = set(get_type(schema))
     if context.is_path_location:

{schemathesis-4.0.7 → schemathesis-4.0.8}/test/conftest.py

@@ -32,8 +32,10 @@ from werkzeug.test import TestResponse
 import schemathesis.cli
 from schemathesis import auths, hooks
 from schemathesis.cli.commands.run.executor import CUSTOM_HANDLERS
+from schemathesis.cli.commands.run.handlers import output
 from schemathesis.core.hooks import HOOKS_MODULE_ENV_VAR
 from schemathesis.core.transport import Response
+from schemathesis.core.version import SCHEMATHESIS_VERSION
 from schemathesis.specs.openapi import media_types
 
 from .apps import _graphql as graphql
@@ -59,6 +61,8 @@ logging.getLogger("pyrate_limiter").setLevel(logging.CRITICAL)
 # `pytest test -m hypothesis --hypothesis-profile <profile-name>`
 settings.register_profile("CI", max_examples=2000)
 
+output.SCHEMATHESIS_VERSION = "dev"
+
 
 @pytest.fixture(autouse=True)
 def reset_hooks():
@@ -357,6 +361,9 @@ class CliSnapshotConfig:
             data,
             flags=re.MULTILINE,
         )
+        version_line = "Schemathesis dev"
+        data = data.replace(f"Schemathesis v{SCHEMATHESIS_VERSION}", version_line)
+        data = re.sub("━+", "━" * len(version_line), data)
         data = data.replace(str(SITE_PACKAGES), site_packages)
         data = re.sub(", line [0-9]+,", ", line XXX,", data)
         data = re.sub(r"Scenarios:.*\d+", r"Scenarios:    N", data)

{schemathesis-4.0.7 → schemathesis-4.0.8}/test/python/test_models.py

@@ -14,6 +14,7 @@ from schemathesis.core.failures import Failure, FailureGroup
 from schemathesis.core.transforms import merge_at
 from schemathesis.core.transport import USER_AGENT, Response
 from schemathesis.generation import GenerationMode
+from schemathesis.generation.meta import ComponentKind
 from schemathesis.schemas import APIOperation
 from schemathesis.specs.openapi.checks import content_type_conformance, response_schema_conformance
 from schemathesis.transport.prepare import get_default_headers
@@ -255,6 +256,25 @@ def test_call_and_validate(openapi3_schema_url, kwargs):
     test()
 
 
+@pytest.mark.operations("custom_format")
+def test_metadata_has_only_relevant_components(openapi3_schema_url):
+    api_schema = schemathesis.openapi.from_url(openapi3_schema_url)
+
+    operation = api_schema["/custom_format"]["GET"]
+
+    @given(
+        case=operation.as_strategy(generation_mode=GenerationMode.POSITIVE)
+        | operation.as_strategy(generation_mode=GenerationMode.NEGATIVE)
+    )
+    @settings(max_examples=10, deadline=None)
+    def test(case):
+        # Metadata should only contain components relevant to the API operation
+        assert len(case.meta.components) == 1
+        assert ComponentKind.QUERY in case.meta.components
+
+    test()
+
+
 @pytest.mark.operations("success")
 def test_call_and_validate_for_asgi(fastapi_app):
     api_schema = schemathesis.openapi.from_dict(fastapi_app.openapi())

{schemathesis-4.0.7 → schemathesis-4.0.8}/test/specs/openapi/test_negative.py

@@ -333,6 +333,40 @@ def test_optional_query_param_negation(ctx):
     test()
 
 
+@pytest.mark.hypothesis_nested
+def test_negating_multiple_query_params(ctx):
+    # When all query parameters are optional
+    schema = ctx.openapi.build_schema(
+        {
+            "/bug": {
+                "get": {
+                    "parameters": [
+                        {"name": "key1", "in": "query", "required": False, "schema": {"type": "integer"}},
+                        {"name": "key2", "in": "query", "required": False, "schema": {"type": "integer"}},
+                    ],
+                    "responses": {"200": {"description": "OK"}},
+                }
+            }
+        }
+    )
+
+    schema = schemathesis.openapi.from_dict(schema)
+
+    @given(case=schema["/bug"]["get"].as_strategy(generation_mode=GenerationMode.NEGATIVE))
+    @settings(deadline=None, suppress_health_check=SUPPRESSED_HEALTH_CHECKS)
+    def test(case):
+        request = requests.PreparedRequest()
+        request.prepare(**case.as_transport_kwargs(base_url="http://127.0.0.1"))
+        # Then negated parameter should always be serialized
+        query = urlparse(request.url).query
+        if "key1" in case.query:
+            assert "key1" in query, case.query
+        if "key2" in case.query:
+            assert "key2" in query, case.query
+
+    test()
+
+
 @pytest.mark.parametrize(
     ("schema", "new_type"),
     [

{schemathesis-4.0.7 → schemathesis-4.0.8}/test-corpus/test_corpus.py

@@ -412,7 +412,13 @@ def test_coverage_phase(corpus, filename):
     modes = list(GenerationMode)
     for operation in schema.get_all_operations():
         if isinstance(operation, Ok):
-            for _ in _iter_coverage_cases(operation.ok(), modes, generate_duplicate_query_parameters=False):
+            for _ in _iter_coverage_cases(
+                operation=operation.ok(),
+                generation_modes=modes,
+                generate_duplicate_query_parameters=False,
+                unexpected_methods=set(),
+                generation_config=schema.config.generation,
+            ):
                 pass
 
 

schemathesis-4.0.7/docs/reference/checks.md

@@ -1,80 +0,0 @@
-# Checks
-
-Schemathesis automatically validates API responses using various checks. Each check targets specific aspects of API behavior and its compliance to the specification.
-
-## Core Response Validation
-
-These checks validate that API responses conform to your API schema.
-
-### `not_a_server_error`
-
-**Detects server-side errors and implementation issues**
-
-**Triggers on:** 5xx HTTP status codes or GraphQL `errors` field
-
-### `status_code_conformance`
-**Verifies response status codes match schema documentation**
-
-**Triggers on:** Status codes not documented in schema
-
-### `content_type_conformance`
-**Validates `Content-Type` header matches schema**
-
-**Triggers on:** Content types not matching schema-documented media types  
-
-### `response_headers_conformance`
-**Ensures required response headers are present and valid**
-
-**Triggers on:** Missing or invalid required headers
-
-### `response_schema_conformance`
-**Validates response body against JSON Schema**
-
-**Triggers on:** Response body not matching schema structure
-
-## Input Handling Validation
-
-These checks verify how your API processes different types of request data.
-
-### `negative_data_rejection`
-**Verifies API rejects invalid request data with appropriate errors**
-
-**Triggers on:** API accepting schema-violating requests
-
-### `positive_data_acceptance`
-**Ensures API accepts valid request data**
-
-**Triggers on:** API rejecting schema-compliant requests  
-
-### `missing_required_header`
-**Checks APIs return 4xx for missing required headers**
-
-**Triggers on:** API not rejecting requests missing required headers
-
-### `unsupported_method`
-**Verifies APIs return 405 for undocumented HTTP methods**
-
-**Triggers on:** Non-405 responses for undocumented methods on valid paths
-
----
-
-## Stateful Behavior
-
-These checks test API behavior across sequences of operations.
-
-### `use_after_free`
-**Detects when deleted resources remain accessible**
-
-**Triggers on:** Accessing deleted resources doesn't return 404
-
-### `ensure_resource_availability`
-**Verifies created resources are immediately accessible**
-
-**Triggers on:** Newly created resources can't be retrieved/modified
-
-## Security
-
-### `ignored_auth` <small>*authentication*</small>
-**Tests whether authentication requirements are enforced**
-
-**Triggers on:** Protected endpoints accepting requests without proper auth