pytest-httpchain 0.4.0__tar.gz → 0.6.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: pytest-httpchain
3
- Version: 0.4.0
3
+ Version: 0.6.0
4
4
  Summary: pytest plugin for HTTP testing using JSON files
5
5
  Keywords: testing,pytest,requests
6
6
  Author: Alexander Eresov
@@ -198,17 +198,7 @@ pip install 'git+https://github.com/aeresov/pytest-httpchain@main'
198
198
 
199
199
  ## AI agent support
200
200
 
201
- `pytest-httpchain` ships tooling to help AI coding agents (and humans) author and check test scenarios.
202
-
203
- ### Claude Code skill
204
-
205
- Install the authoring skill into your project (or `--global` for personal scope):
206
-
207
- ```bash
208
- uvx pytest-httpchain install
209
- ```
210
-
211
- This writes `.claude/skills/pytest-httpchain/SKILL.md` with guidance for writing scenarios.
201
+ `pytest-httpchain` ships a scenario validator to help AI coding agents (and humans) author and check test scenarios.
212
202
 
213
203
  ### Scenario validation
214
204
 
@@ -242,6 +232,27 @@ A JSON Schema is published for as-you-type validation and autocomplete. Referenc
242
232
  }
243
233
  ```
244
234
 
235
+ The hosted schema tracks the latest release; to pin the schema matching your installed version (e.g. for CI), emit it locally:
236
+
237
+ ```bash
238
+ uvx pytest-httpchain schema --output scenario.schema.json
239
+ ```
240
+
241
+ ### Inspecting scenarios
242
+
243
+ More read-only commands help author and debug scenarios offline — no network, no test run:
244
+
245
+ ```bash
246
+ # Print a scenario with all $ref/$include/$merge inlined and deep-merged
247
+ uvx pytest-httpchain resolve tests/test_login.http.json
248
+
249
+ # Summarize stages and the variable data-flow (which stage saves what, who consumes it)
250
+ uvx pytest-httpchain show tests/test_login.http.json
251
+
252
+ # Render the stage data-flow as a Mermaid flowchart
253
+ uvx pytest-httpchain graph tests/test_login.http.json
254
+ ```
255
+
245
256
  ## Documentation
246
257
 
247
258
  - [Full Documentation](https://aeresov.github.io/pytest-httpchain) - Complete usage guide
@@ -171,17 +171,7 @@ pip install 'git+https://github.com/aeresov/pytest-httpchain@main'
171
171
 
172
172
  ## AI agent support
173
173
 
174
- `pytest-httpchain` ships tooling to help AI coding agents (and humans) author and check test scenarios.
175
-
176
- ### Claude Code skill
177
-
178
- Install the authoring skill into your project (or `--global` for personal scope):
179
-
180
- ```bash
181
- uvx pytest-httpchain install
182
- ```
183
-
184
- This writes `.claude/skills/pytest-httpchain/SKILL.md` with guidance for writing scenarios.
174
+ `pytest-httpchain` ships a scenario validator to help AI coding agents (and humans) author and check test scenarios.
185
175
 
186
176
  ### Scenario validation
187
177
 
@@ -215,6 +205,27 @@ A JSON Schema is published for as-you-type validation and autocomplete. Referenc
215
205
  }
216
206
  ```
217
207
 
208
+ The hosted schema tracks the latest release; to pin the schema matching your installed version (e.g. for CI), emit it locally:
209
+
210
+ ```bash
211
+ uvx pytest-httpchain schema --output scenario.schema.json
212
+ ```
213
+
214
+ ### Inspecting scenarios
215
+
216
+ More read-only commands help author and debug scenarios offline — no network, no test run:
217
+
218
+ ```bash
219
+ # Print a scenario with all $ref/$include/$merge inlined and deep-merged
220
+ uvx pytest-httpchain resolve tests/test_login.http.json
221
+
222
+ # Summarize stages and the variable data-flow (which stage saves what, who consumes it)
223
+ uvx pytest-httpchain show tests/test_login.http.json
224
+
225
+ # Render the stage data-flow as a Mermaid flowchart
226
+ uvx pytest-httpchain graph tests/test_login.http.json
227
+ ```
228
+
218
229
  ## Documentation
219
230
 
220
231
  - [Full Documentation](https://aeresov.github.io/pytest-httpchain) - Complete usage guide
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "pytest-httpchain"
3
- version = "0.4.0"
3
+ version = "0.6.0"
4
4
  description = "pytest plugin for HTTP testing using JSON files"
5
5
  readme = "README.md"
6
6
  requires-python = ">=3.13,<4.0"
@@ -79,12 +79,24 @@ class Carrier:
79
79
  max_parallel_iterations: ClassVar[int] = 10_000
80
80
 
81
81
  @classmethod
82
- def execute_stage(cls, stage: Stage, fixture_kwargs: dict[str, Any]) -> None:
82
+ def _resolve_always_run(cls, stage: Stage, stage_fixtures: dict[str, Any]) -> bool:
83
+ """Resolve ``always_run``, evaluating a template form against the context
84
+ available at stage start: fixtures and parametrize parameters plus the
85
+ global context (scenario substitutions and earlier saves). Stage
86
+ substitutions are not yet processed at this point. The result is coerced
87
+ with Python truthiness."""
88
+ if isinstance(stage.always_run, bool):
89
+ return stage.always_run
83
90
  try:
84
- if cls.aborted and not stage.always_run:
85
- pytest.skip(reason="Flow aborted")
91
+ return bool(walk(stage.always_run, ChainMap(stage_fixtures, cls.global_context)))
92
+ except TemplatesError as e:
93
+ raise StageExecutionError(f"Failed to evaluate always_run template: {e}") from e
86
94
 
87
- # prepare stage context
95
+ @classmethod
96
+ def execute_stage(cls, stage: Stage, fixture_kwargs: dict[str, Any]) -> None:
97
+ try:
98
+ # prepare stage context (before the abort check, so an always_run
99
+ # template sees the same fixture layer as stage templates)
88
100
  stage_fixtures: dict[str, Any] = {}
89
101
  for name, value in fixture_kwargs.items():
90
102
  if callable(value) and not inspect.isclass(value):
@@ -92,6 +104,9 @@ class Carrier:
92
104
  else:
93
105
  stage_fixtures[name] = value
94
106
 
107
+ if cls.aborted and not cls._resolve_always_run(stage, stage_fixtures):
108
+ pytest.skip(reason="Flow aborted")
109
+
95
110
  # Build base context for iterations (substitutions + fixtures + global)
96
111
  local_context = ChainMap(stage_fixtures, cls.global_context)
97
112
  stage_substitutions = process_substitutions(stage.substitutions, local_context)
@@ -519,7 +534,7 @@ def create_test_class(scenario: Scenario, class_name: str, max_parallel_iteratio
519
534
  parametrize_marker = pytest.mark.parametrize(",".join(param_names), param_values, ids=param_ids)
520
535
  stage_method = parametrize_marker(stage_method)
521
536
 
522
- all_fixtures = ["self"] + all_param_names + stage.fixtures
537
+ all_fixtures = ["self"] + list(dict.fromkeys(all_param_names + stage.fixtures + scenario.fixtures))
523
538
  stage_method.__signature__ = inspect.Signature([inspect.Parameter(name, inspect.Parameter.POSITIONAL_OR_KEYWORD) for name in all_fixtures]) # type: ignore[assignment]
524
539
 
525
540
  all_marks = [f"order({i})"] + stage.marks
@@ -0,0 +1,222 @@
1
+ import enum
2
+ import json
3
+ from pathlib import Path
4
+ from typing import Annotated
5
+
6
+ import typer
7
+
8
+ app = typer.Typer()
9
+
10
+
11
+ class OutputFormat(enum.StrEnum):
12
+ text = "text"
13
+ json = "json"
14
+
15
+
16
+ class GraphDirection(enum.StrEnum):
17
+ TD = "TD"
18
+ LR = "LR"
19
+
20
+
21
+ @app.callback()
22
+ def main() -> None:
23
+ """pytest-httpchain command-line tools."""
24
+
25
+
26
+ @app.command()
27
+ def validate(
28
+ paths: Annotated[list[Path], typer.Argument(help="Scenario JSON file(s) to validate.")],
29
+ ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
30
+ output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
31
+ deep: Annotated[bool, typer.Option("--deep", help="Run deep checks: resolve user-function imports/signatures and referenced files. Imports user modules.")] = False,
32
+ syspath: Annotated[list[Path] | None, typer.Option("--syspath", help="Extra directories to add to sys.path for --deep import resolution (repeatable).")] = None,
33
+ strict: Annotated[bool, typer.Option("--strict", help="Treat warnings as failures for the exit code.")] = False,
34
+ ) -> None:
35
+ """Validate pytest-httpchain scenario file(s).
36
+
37
+ Reports errors and warnings (each with a stable HTTPCHAINxxx diagnostic code)
38
+ per file and exits non-zero if any file is invalid (or, with --strict, has any
39
+ warnings).
40
+ """
41
+ from pytest_httpchain.validation import validate_scenario
42
+
43
+ results = [(path, validate_scenario(path, ref_parent_traversal_depth=ref_parent_traversal_depth, deep=deep, syspaths=list(syspath or []))) for path in paths]
44
+
45
+ def passed(result) -> bool:
46
+ return result.valid and not (strict and result.warnings)
47
+
48
+ all_passed = all(passed(result) for _, result in results)
49
+
50
+ if output_format is OutputFormat.json:
51
+ payload = {
52
+ "valid": all_passed,
53
+ "files": [{"path": str(path), "result": result.model_dump()} for path, result in results],
54
+ }
55
+ typer.echo(json.dumps(payload, indent=2, default=str))
56
+ else:
57
+ for path, result in results:
58
+ if not result.valid:
59
+ status = "INVALID"
60
+ elif result.warnings:
61
+ status = "FAILED (warnings)" if strict else "OK with warnings"
62
+ else:
63
+ status = "OK"
64
+ typer.echo(f"{path}: {status}")
65
+ for diagnostic in result.diagnostics:
66
+ typer.echo(f" {diagnostic.severity} [{diagnostic.code}]: {diagnostic.message}")
67
+
68
+ raise typer.Exit(0 if all_passed else 1)
69
+
70
+
71
+ @app.command()
72
+ def schema(
73
+ output: Annotated[Path | None, typer.Option("--output", "-o", help="Write schema to PATH instead of stdout.")] = None,
74
+ ) -> None:
75
+ """Emit the JSON Schema for scenario files (editor autocomplete/validation)."""
76
+ from pytest_httpchain.schema import build_schema
77
+
78
+ text = json.dumps(build_schema(), indent=2, default=str)
79
+ if output is not None:
80
+ output.write_text(text + "\n")
81
+ typer.echo(f"Wrote schema to {output}")
82
+ else:
83
+ typer.echo(text)
84
+
85
+
86
+ @app.command()
87
+ def resolve(
88
+ scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to resolve.")],
89
+ output: Annotated[Path | None, typer.Option("--output", "-o", help="Write resolved JSON to PATH instead of stdout.")] = None,
90
+ ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
91
+ ) -> None:
92
+ """Resolve $ref/$include/$merge and print the merged scenario JSON."""
93
+ import pytest_httpchain_jsonref.loader
94
+ from pytest_httpchain_jsonref.exceptions import ReferenceResolverError
95
+
96
+ from pytest_httpchain.validation import resolve_root_path
97
+
98
+ try:
99
+ data = pytest_httpchain_jsonref.loader.load_json(
100
+ scenario,
101
+ max_parent_traversal_depth=ref_parent_traversal_depth,
102
+ root_path=resolve_root_path(scenario),
103
+ )
104
+ except (ReferenceResolverError, json.JSONDecodeError, OSError) as e:
105
+ typer.echo(f"error: {e}", err=True)
106
+ raise typer.Exit(1) from e
107
+
108
+ text = json.dumps(data, indent=2, default=str)
109
+ if output is not None:
110
+ output.write_text(text + "\n")
111
+ typer.echo(f"Wrote resolved scenario to {output}")
112
+ else:
113
+ typer.echo(text)
114
+
115
+
116
+ def _load_for_inspection(path: Path, depth: int):
117
+ """Load + validate a scenario for show/graph, mapping failures to Exit(1)."""
118
+ from pydantic import ValidationError
119
+ from pytest_httpchain_jsonref.exceptions import ReferenceResolverError
120
+
121
+ from pytest_httpchain.dataflow import load_scenario
122
+
123
+ try:
124
+ return load_scenario(path, ref_parent_traversal_depth=depth)
125
+ except (ReferenceResolverError, json.JSONDecodeError, OSError) as e:
126
+ typer.echo(f"error: cannot load {path}: {e}", err=True)
127
+ raise typer.Exit(1) from e
128
+ except ValidationError:
129
+ typer.echo(f"error: {path} is not a valid scenario — run `pytest-httpchain validate {path}` for details", err=True)
130
+ raise typer.Exit(1) from None
131
+
132
+
133
+ def _render_show_text(path: Path, scenario, flow) -> list[str]:
134
+ producer_of: dict[tuple[int, str], int] = {}
135
+ for edge in flow.edges:
136
+ for name in edge.vars:
137
+ producer_of[(edge.consumer, name)] = edge.producer
138
+
139
+ all_fixtures = sorted({*flow.scenario_fixtures, *(f for s in flow.stages for f in s.fixtures)})
140
+ lines: list[str] = [scenario.description or path.name]
141
+ summary = f"{len(flow.stages)} stage(s)"
142
+ if all_fixtures:
143
+ summary += f" · fixtures: {all_fixtures}"
144
+ if flow.scenario_vars:
145
+ summary += f" · vars: {flow.scenario_vars}"
146
+ lines.append(summary)
147
+ lines.append("")
148
+
149
+ for s in flow.stages:
150
+ name = s.name or f"(stage {s.index + 1})"
151
+ lines.append(f"{s.index + 1} · {name} {s.method} {s.url}")
152
+ if s.saves:
153
+ lines.append(f" saves: {', '.join(s.saves)}")
154
+ if s.consumes:
155
+ parts: list[str] = []
156
+ for name in s.consumes:
157
+ producer = producer_of.get((s.index, name))
158
+ if producer is None:
159
+ parts.append(name)
160
+ else:
161
+ producer_name = flow.stages[producer].name or f"stage {producer + 1}"
162
+ parts.append(f"{name} (from #{producer + 1} {producer_name})")
163
+ lines.append(f" consumes: {', '.join(parts)}")
164
+ if s.marks:
165
+ lines.append(f" marks: {', '.join(s.marks)}")
166
+ return lines
167
+
168
+
169
+ @app.command()
170
+ def show(
171
+ scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to summarize.")],
172
+ output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
173
+ ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
174
+ ) -> None:
175
+ """Summarize a scenario's stages and variable data-flow."""
176
+ from pytest_httpchain.dataflow import analyze_dataflow
177
+
178
+ sc, test_data = _load_for_inspection(scenario, ref_parent_traversal_depth)
179
+ flow = analyze_dataflow(sc, test_data)
180
+
181
+ if output_format is OutputFormat.json:
182
+ payload = flow.model_dump()
183
+ payload["description"] = sc.description or None
184
+ typer.echo(json.dumps(payload, indent=2, default=str))
185
+ else:
186
+ for line in _render_show_text(scenario, sc, flow):
187
+ typer.echo(line)
188
+
189
+
190
+ def _mermaid_label(text: str) -> str:
191
+ return text.replace('"', "'").replace("\n", " ")
192
+
193
+
194
+ def _to_mermaid(flow, direction: str = "TD") -> str:
195
+ lines = [f"flowchart {direction}"]
196
+ if not flow.stages:
197
+ lines.append(" %% (no stages)")
198
+ return "\n".join(lines)
199
+ for s in flow.stages:
200
+ label = f"{s.index + 1} · {s.name}" if s.name else f"{s.index + 1}"
201
+ lines.append(f' S{s.index}["{_mermaid_label(label)}"]')
202
+ for edge in flow.edges:
203
+ lines.append(f" S{edge.producer} -->|{', '.join(edge.vars)}| S{edge.consumer}")
204
+ return "\n".join(lines)
205
+
206
+
207
+ @app.command()
208
+ def graph(
209
+ scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to graph.")],
210
+ direction: Annotated[GraphDirection, typer.Option("--direction", help="Flowchart orientation.")] = GraphDirection.TD,
211
+ ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
212
+ ) -> None:
213
+ """Emit a Mermaid flowchart of the stage data-flow."""
214
+ from pytest_httpchain.dataflow import analyze_dataflow
215
+
216
+ sc, test_data = _load_for_inspection(scenario, ref_parent_traversal_depth)
217
+ flow = analyze_dataflow(sc, test_data)
218
+ typer.echo(_to_mermaid(flow, direction.value))
219
+
220
+
221
+ if __name__ == "__main__":
222
+ app()
@@ -0,0 +1,137 @@
1
+ """Structured stage data-flow analysis for the ``show`` and ``graph`` CLI commands.
2
+
3
+ Reuses the same extraction helpers as the order-aware validator
4
+ (``validation._dataflow_diagnostics``) but produces a graph model instead of
5
+ diagnostics: which variables each stage saves, which it consumes from earlier
6
+ stages, and the producer -> consumer edges between stages.
7
+ """
8
+
9
+ from pathlib import Path
10
+ from typing import Any
11
+
12
+ import pytest_httpchain_jsonref.loader
13
+ from pydantic import BaseModel
14
+ from pytest_httpchain_models import Scenario
15
+
16
+ from pytest_httpchain.validation import (
17
+ _parameter_names,
18
+ extract_template_variables,
19
+ raw_stages,
20
+ resolve_root_path,
21
+ saved_in_stage,
22
+ stage_defined_names,
23
+ substitution_names,
24
+ )
25
+
26
+
27
+ class DataFlowEdge(BaseModel):
28
+ """A data dependency: ``vars`` saved by stage ``producer`` are referenced by stage ``consumer``."""
29
+
30
+ producer: int
31
+ consumer: int
32
+ vars: list[str]
33
+
34
+
35
+ class StageFlow(BaseModel):
36
+ """Per-stage data-flow summary."""
37
+
38
+ index: int
39
+ name: str
40
+ method: str
41
+ url: str
42
+ fixtures: list[str]
43
+ marks: list[str]
44
+ saves: list[str]
45
+ consumes: list[str]
46
+
47
+
48
+ class DataFlow(BaseModel):
49
+ """Whole-scenario data-flow graph."""
50
+
51
+ stages: list[StageFlow]
52
+ edges: list[DataFlowEdge]
53
+ scenario_fixtures: list[str] = []
54
+ scenario_vars: list[str] = []
55
+
56
+
57
+ def analyze_dataflow(scenario: Scenario, test_data: dict[str, Any]) -> DataFlow:
58
+ """Build the stage data-flow graph for a validated scenario.
59
+
60
+ A stage *consumes* a variable when its request/response/substitutions/parallel
61
+ templates reference a name saved by an earlier stage and not redefined locally
62
+ (own substitutions, parametrize/foreach parameters, or stage/scenario fixtures).
63
+ ``always_run`` references count too, but only fixtures and parametrize
64
+ parameters shadow them — always_run resolves before stage substitutions exist.
65
+ ``parametrize`` values are excluded — they resolve against scenario scope,
66
+ never saved values.
67
+ """
68
+ raws = raw_stages(test_data)
69
+ saves_by_stage = [saved_in_stage(stage) for stage in scenario.stages]
70
+ scenario_fixture_names = set(scenario.fixtures)
71
+
72
+ first_save_stage: dict[str, int] = {}
73
+ for i, saved in enumerate(saves_by_stage):
74
+ for name in saved:
75
+ first_save_stage.setdefault(name, i)
76
+
77
+ stages: list[StageFlow] = []
78
+ edges: list[DataFlowEdge] = []
79
+ cumulative_saves: set[str] = set()
80
+
81
+ for i, stage in enumerate(scenario.stages):
82
+ raw = raws[i] if i < len(raws) and isinstance(raws[i], dict) else {}
83
+
84
+ refs: set[str] = set()
85
+ for key in ("request", "response", "substitutions", "parallel"):
86
+ extract_template_variables(raw.get(key), refs)
87
+
88
+ # Scenario fixtures count as local everywhere: at runtime they sit above
89
+ # the global context in the ChainMap, shadowing any same-named save.
90
+ local = stage_defined_names(stage) | scenario_fixture_names
91
+ consumes = {name for name in refs if name in cumulative_saves and name not in local}
92
+
93
+ # always_run resolves before stage substitutions exist, so only fixtures
94
+ # and parametrize parameters shadow an earlier save there.
95
+ always_run_refs = extract_template_variables(raw.get("always_run"))
96
+ always_run_local = set(stage.fixtures) | _parameter_names(stage.parametrize) | scenario_fixture_names
97
+ consumes |= {name for name in always_run_refs if name in cumulative_saves and name not in always_run_local}
98
+
99
+ by_producer: dict[int, list[str]] = {}
100
+ for name in consumes:
101
+ by_producer.setdefault(first_save_stage[name], []).append(name)
102
+ for producer in sorted(by_producer):
103
+ edges.append(DataFlowEdge(producer=producer, consumer=i, vars=sorted(by_producer[producer])))
104
+
105
+ stages.append(
106
+ StageFlow(
107
+ index=i,
108
+ name=stage.name,
109
+ method=str(stage.request.method),
110
+ url=str(stage.request.url),
111
+ fixtures=sorted(stage.fixtures),
112
+ marks=list(stage.marks),
113
+ saves=sorted(saves_by_stage[i]),
114
+ consumes=sorted(consumes),
115
+ )
116
+ )
117
+
118
+ cumulative_saves |= saves_by_stage[i]
119
+
120
+ scenario_var_names = set(substitution_names(scenario.substitutions))
121
+
122
+ return DataFlow(stages=stages, edges=edges, scenario_fixtures=sorted(scenario.fixtures), scenario_vars=sorted(scenario_var_names))
123
+
124
+
125
+ def load_scenario(path: Path, ref_parent_traversal_depth: int = 3) -> tuple[Scenario, dict[str, Any]]:
126
+ """Load + ``$ref``-resolve + validate a scenario file.
127
+
128
+ Returns ``(scenario, raw_test_data)``. Raises ``ReferenceResolverError``,
129
+ ``json.JSONDecodeError`` or ``pydantic.ValidationError`` on failure — callers
130
+ map these to user-facing errors.
131
+ """
132
+ test_data = pytest_httpchain_jsonref.loader.load_json(
133
+ path,
134
+ max_parent_traversal_depth=ref_parent_traversal_depth,
135
+ root_path=resolve_root_path(path),
136
+ )
137
+ return Scenario.model_validate(test_data), test_data
@@ -0,0 +1,116 @@
1
+ """Build the JSON Schema for pytest-httpchain scenario files.
2
+
3
+ Shared by the ``pytest-httpchain schema`` CLI command and
4
+ ``scripts/generate_schema.py``. The schema is derived from the Pydantic
5
+ ``Scenario`` model and augmented so editors accept pytest-httpchain's
6
+ ``$ref``/``$include``/``$merge`` reference directives at any object level.
7
+ """
8
+
9
+ from typing import Any
10
+
11
+ from pytest_httpchain_models import Scenario
12
+
13
+ SCHEMA_DIALECT = "https://json-schema.org/draft/2020-12/schema"
14
+ SCHEMA_ID = "https://aeresov.github.io/pytest-httpchain/schema/scenario.schema.json"
15
+
16
+
17
+ def _one_of_to_any_of(node: Any) -> None:
18
+ """Recursively rename ``oneOf`` to ``anyOf`` in place."""
19
+ if isinstance(node, dict):
20
+ if "oneOf" in node:
21
+ node["anyOf"] = node.pop("oneOf")
22
+ for value in node.values():
23
+ _one_of_to_any_of(value)
24
+ elif isinstance(node, list):
25
+ for item in node:
26
+ _one_of_to_any_of(item)
27
+
28
+
29
+ def _add_jsonref_support(schema: dict[str, Any]) -> dict[str, Any]:
30
+ """Allow ``$include``/``$merge``/``$ref`` objects as alternatives anywhere.
31
+
32
+ pytest-httpchain-jsonref can substitute any element at runtime, so each
33
+ definition and root property is wrapped in an ``anyOf`` that also accepts a
34
+ reference object — otherwise editors flag missing required properties when a
35
+ reference is used.
36
+ """
37
+ if "$defs" not in schema:
38
+ schema["$defs"] = {}
39
+
40
+ schema["$defs"]["JsonRef"] = {
41
+ "type": "object",
42
+ "description": "Reference to external JSON file or JSON pointer. Use $include or $merge (preferred) or $ref. Resolved at runtime by pytest-httpchain-jsonref.",
43
+ "properties": {
44
+ "$include": {
45
+ "type": "string",
46
+ "description": "Path to external JSON file, JSON pointer (#/path), or combined (file.json#/path). Preferred over $ref to avoid VS Code conflicts.",
47
+ },
48
+ "$merge": {
49
+ "type": "string",
50
+ "description": "Alias for $include. Path to external JSON file, JSON pointer (#/path), or combined (file.json#/path).",
51
+ },
52
+ "$ref": {
53
+ "type": "string",
54
+ "description": "Legacy alias for $include. May conflict with VS Code's own $ref handling.",
55
+ },
56
+ },
57
+ # Without at least one directive key this branch would match EVERY
58
+ # object, silencing the strict alternative in the surrounding anyOf.
59
+ "anyOf": [
60
+ {"required": ["$include"]},
61
+ {"required": ["$merge"]},
62
+ {"required": ["$ref"]},
63
+ ],
64
+ "additionalProperties": True,
65
+ }
66
+
67
+ # Pydantic emits oneOf for tagged unions. A reference object matches the
68
+ # JsonRef branch of EVERY union member (each $defs entry is wrapped below),
69
+ # which oneOf counts as "valid under more than one" and rejects. anyOf
70
+ # keeps the same accept set otherwise: members forbid each other's tag
71
+ # fields, so a non-reference object can never match two branches.
72
+ _one_of_to_any_of(schema)
73
+
74
+ for type_name, original_def in list(schema["$defs"].items()):
75
+ if type_name == "JsonRef":
76
+ continue
77
+ schema["$defs"][type_name] = {"anyOf": [{"$ref": "#/$defs/JsonRef"}, original_def]}
78
+ if "title" in original_def:
79
+ schema["$defs"][type_name]["title"] = original_def.pop("title")
80
+ if "description" in original_def:
81
+ schema["$defs"][type_name]["description"] = original_def.pop("description")
82
+
83
+ for prop_name, prop_def in list(schema.get("properties", {}).items()):
84
+ schema["properties"][prop_name] = {
85
+ "anyOf": [{"$ref": "#/$defs/JsonRef"}, prop_def],
86
+ "title": prop_def.get("title", prop_name),
87
+ }
88
+ if "description" in prop_def:
89
+ schema["properties"][prop_name]["description"] = prop_def.get("description")
90
+
91
+ # The Scenario model forbids extra keys, so the root carries
92
+ # additionalProperties: false. Keys that are legitimate in a scenario
93
+ # *file* but handled before model validation must be declared explicitly:
94
+ # "$schema" (editor metadata, stripped by the loader) and the reference
95
+ # directives (resolved by the loader, supported at the document root).
96
+ schema.setdefault("properties", {})
97
+ schema["properties"]["$schema"] = {
98
+ "type": "string",
99
+ "description": "URL of this schema, for editor as-you-type validation. Stripped before the file is parsed.",
100
+ }
101
+ for directive, directive_def in schema["$defs"]["JsonRef"]["properties"].items():
102
+ schema["properties"][directive] = directive_def
103
+ schema["properties"]["$ref"] = {
104
+ "type": "string",
105
+ "description": "Legacy alias for $include. May conflict with VS Code's own $ref handling.",
106
+ }
107
+
108
+ return schema
109
+
110
+
111
+ def build_schema() -> dict[str, Any]:
112
+ """Return the augmented JSON Schema dict for the ``Scenario`` model."""
113
+ schema = Scenario.model_json_schema()
114
+ schema["$schema"] = SCHEMA_DIALECT
115
+ schema["$id"] = SCHEMA_ID
116
+ return _add_jsonref_support(schema)
@@ -24,12 +24,14 @@ Code Severity Meaning
24
24
  006 warning Verify step asserts nothing (no-op)
25
25
  007 error Body ``contains``/``not_contains`` list the same substring
26
26
  008 error Body ``matches``/``not_matches`` list the same pattern
27
+ 009 warning Saved variable is shadowed by a scenario-level fixture
27
28
  010 error File not found
28
29
  011 error Path is not a file
29
30
  012 error ``$ref`` resolution failed
30
31
  013 warning File extension is not ``.json``
31
32
  014 error Invalid JSON syntax
32
33
  015 error Failed to parse JSON file
34
+ 016 error Fixture referenced in a scenario-level template
33
35
  020 warning Referenced file does not exist (deep, opt-in)
34
36
  021 warning Schema file is not valid JSON / not a valid schema (deep)
35
37
  022 warning User function cannot be imported (deep)
@@ -70,12 +72,14 @@ class DiagnosticCode:
70
72
  NOOP_VERIFY = "HTTPCHAIN006"
71
73
  CONTAINS_CONTRADICTION = "HTTPCHAIN007"
72
74
  MATCHES_CONTRADICTION = "HTTPCHAIN008"
75
+ FIXTURE_SHADOWS_SAVE = "HTTPCHAIN009"
73
76
  FILE_NOT_FOUND = "HTTPCHAIN010"
74
77
  NOT_A_FILE = "HTTPCHAIN011"
75
78
  REF_ERROR = "HTTPCHAIN012"
76
79
  WRONG_EXTENSION = "HTTPCHAIN013"
77
80
  INVALID_JSON = "HTTPCHAIN014"
78
81
  PARSE_ERROR = "HTTPCHAIN015"
82
+ FIXTURE_IN_SCENARIO_TEMPLATE = "HTTPCHAIN016"
79
83
  # Deep (opt-in) checks: imports, signatures, referenced files.
80
84
  REFERENCED_FILE_NOT_FOUND = "HTTPCHAIN020"
81
85
  SCHEMA_FILE_INVALID = "HTTPCHAIN021"
@@ -152,7 +156,7 @@ def extract_template_variables(obj: Any, variables: set[str] | None = None) -> s
152
156
  return variables
153
157
 
154
158
 
155
- def _substitution_names(substitutions: Any) -> set[str]:
159
+ def substitution_names(substitutions: Any) -> set[str]:
156
160
  """Names introduced by a list of ``vars``/``functions`` substitution entries."""
157
161
  names: set[str] = set()
158
162
  for sub in substitutions or []:
@@ -165,7 +169,7 @@ def _substitution_names(substitutions: Any) -> set[str]:
165
169
  return names
166
170
 
167
171
 
168
- def _saved_in_stage(stage: Stage) -> set[str]:
172
+ def saved_in_stage(stage: Stage) -> set[str]:
169
173
  """Variable names a single stage's response steps save into the context."""
170
174
  saved: set[str] = set()
171
175
  for response_step in stage.response:
@@ -177,7 +181,7 @@ def _saved_in_stage(stage: Stage) -> set[str]:
177
181
  saved.update(jmespath.keys())
178
182
  substitutions = getattr(save, "substitutions", None)
179
183
  if substitutions is not None:
180
- saved |= _substitution_names(substitutions)
184
+ saved |= substitution_names(substitutions)
181
185
  # user_functions saves return arbitrary dict keys -> not statically known.
182
186
  return saved
183
187
 
@@ -186,7 +190,7 @@ def extract_saved_variables(scenario: Scenario) -> set[str]:
186
190
  """Extract variable names saved across all response steps in the scenario."""
187
191
  saved_vars: set[str] = set()
188
192
  for stage in scenario.stages:
189
- saved_vars |= _saved_in_stage(stage)
193
+ saved_vars |= saved_in_stage(stage)
190
194
  return saved_vars
191
195
 
192
196
 
@@ -210,10 +214,10 @@ def _parameter_names(params: Any) -> set[str]:
210
214
  return names
211
215
 
212
216
 
213
- def _stage_defined_names(stage: Stage) -> set[str]:
217
+ def stage_defined_names(stage: Stage) -> set[str]:
214
218
  """Names available *within a single stage*: its substitutions, parametrize /
215
219
  foreach parameters, and its declared fixtures."""
216
- names = _substitution_names(stage.substitutions)
220
+ names = substitution_names(stage.substitutions)
217
221
  names |= _parameter_names(stage.parametrize)
218
222
  if stage.parallel is not None:
219
223
  names |= _parameter_names(getattr(stage.parallel, "foreach", None))
@@ -221,7 +225,7 @@ def _stage_defined_names(stage: Stage) -> set[str]:
221
225
  return names
222
226
 
223
227
 
224
- def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) -> set[str]:
228
+ def extract_defined_variables(scenario: Scenario) -> set[str]:
225
229
  """Extract variable names made available before/within templates (scenario-wide).
226
230
 
227
231
  Sources: ``vars`` and ``functions`` substitutions (scenario- and stage-level),
@@ -232,14 +236,10 @@ def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) ->
232
236
  """
233
237
  defined_vars: set[str] = set()
234
238
 
235
- # Defensive: a top-level "vars" key is not a model field but is tolerated.
236
- if isinstance(test_data.get("vars"), dict):
237
- defined_vars.update(k for k in test_data["vars"] if isinstance(k, str))
238
-
239
- defined_vars |= _substitution_names(scenario.substitutions)
239
+ defined_vars |= substitution_names(scenario.substitutions)
240
240
 
241
241
  for stage in scenario.stages:
242
- defined_vars |= _substitution_names(stage.substitutions)
242
+ defined_vars |= substitution_names(stage.substitutions)
243
243
  defined_vars |= _parameter_names(stage.parametrize)
244
244
  if stage.parallel is not None:
245
245
  defined_vars |= _parameter_names(getattr(stage.parallel, "foreach", None))
@@ -247,7 +247,7 @@ def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) ->
247
247
  return defined_vars
248
248
 
249
249
 
250
- def _raw_stages(test_data: dict[str, Any]) -> list[Any]:
250
+ def raw_stages(test_data: dict[str, Any]) -> list[Any]:
251
251
  """Raw (pre-validation) stage bodies in declaration order.
252
252
 
253
253
  Stages may be authored as a list or as a ``{name: stage}`` mapping; both
@@ -283,13 +283,16 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
283
283
  Walks stages in execution order tracking which variables are available at
284
284
  each reference site, mirroring the runtime scoping in ``carrier.py``:
285
285
 
286
- * scenario-level substitutions (and a tolerated top-level ``vars``) are
287
- available everywhere;
286
+ * scenario-level substitutions and scenario-level fixtures are available
287
+ everywhere;
288
288
  * a stage's own substitutions, parametrize/foreach parameters and fixtures
289
289
  are available to that stage's request and response;
290
290
  * a value saved in a stage's response is available to that stage's response
291
291
  and to *later* stages — but never to the same stage's request, and never
292
- to earlier stages.
292
+ to earlier stages;
293
+ * ``always_run`` is evaluated at stage start, before stage substitutions are
294
+ processed and before any iteration runs: only fixtures, parametrize
295
+ parameters, scenario substitutions and earlier saves are in scope.
293
296
 
294
297
  A reference that is unavailable is reported as :data:`DiagnosticCode.FORWARD_REF`
295
298
  if the name is saved somewhere later (an ordering bug) or
@@ -298,7 +301,7 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
298
301
  diagnostics: list[Diagnostic] = []
299
302
 
300
303
  all_saved = extract_saved_variables(scenario)
301
- saves_by_stage: list[set[str]] = [_saved_in_stage(stage) for stage in scenario.stages]
304
+ saves_by_stage: list[set[str]] = [saved_in_stage(stage) for stage in scenario.stages]
302
305
  first_save_stage: dict[str, int] = {}
303
306
  for i, saved in enumerate(saves_by_stage):
304
307
  for name in saved:
@@ -309,22 +312,17 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
309
312
  # fixtures, no stage substitutions, no parameter names, no saved values exist
310
313
  # yet. ``scenario_scope`` is that narrow set; ``scenario_available`` is the
311
314
  # everywhere-available set used for ordinary request/response references.
312
- scenario_scope: set[str] = set()
313
- if isinstance(test_data.get("vars"), dict):
314
- scenario_scope |= {k for k in test_data["vars"] if isinstance(k, str)}
315
- scenario_scope |= _substitution_names(scenario.substitutions)
315
+ scenario_scope: set[str] = set(substitution_names(scenario.substitutions))
316
316
 
317
- scenario_available = set(scenario_scope)
318
- if isinstance(test_data.get("fixtures"), list):
319
- scenario_available |= {f for f in test_data["fixtures"] if isinstance(f, str)}
317
+ scenario_available = scenario_scope | set(scenario.fixtures)
320
318
 
321
- raw_stages = _raw_stages(test_data)
319
+ raws = raw_stages(test_data)
322
320
  cumulative_saves: set[str] = set()
323
321
 
324
322
  for i, stage in enumerate(scenario.stages):
325
- raw = raw_stages[i] if i < len(raw_stages) and isinstance(raw_stages[i], dict) else {}
323
+ raw = raws[i] if i < len(raws) and isinstance(raws[i], dict) else {}
326
324
 
327
- request_available = scenario_available | _stage_defined_names(stage) | cumulative_saves
325
+ request_available = scenario_available | stage_defined_names(stage) | cumulative_saves
328
326
  response_available = request_available | saves_by_stage[i]
329
327
 
330
328
  # ``parallel.foreach`` values are resolved at stage execution against the
@@ -347,6 +345,31 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
347
345
  )
348
346
  )
349
347
 
348
+ # ``always_run`` resolves at stage start (carrier.execute_stage) against
349
+ # fixtures + parametrize parameters + global context — stage substitutions
350
+ # and foreach parameters don't exist yet, and neither do this stage's saves.
351
+ always_run_available = scenario_available | set(stage.fixtures) | _parameter_names(stage.parametrize) | cumulative_saves
352
+ for name in sorted(extract_template_variables(raw.get("always_run"))):
353
+ if name in always_run_available:
354
+ continue
355
+ if name in all_saved:
356
+ j = first_save_stage[name]
357
+ if j == i:
358
+ msg = f"Stage '{stage.name}': always_run references '{name}', which is only saved in this stage's response — always_run is evaluated before the stage runs"
359
+ else:
360
+ msg = f"Stage '{stage.name}': always_run references '{name}' before it is saved (saved in stage '{scenario.stages[j].name}')"
361
+ diagnostics.append(_diag(DiagnosticCode.FORWARD_REF, "warning", msg, location=stage.name))
362
+ else:
363
+ diagnostics.append(
364
+ _diag(
365
+ DiagnosticCode.UNDEFINED_VAR,
366
+ "warning",
367
+ f"Stage '{stage.name}': always_run references '{name}' — potentially not in scope; only fixtures, "
368
+ f"parametrize parameters, scenario substitutions, and variables saved by earlier stages are available",
369
+ location=stage.name,
370
+ )
371
+ )
372
+
350
373
  undefined_here: set[str] = set()
351
374
 
352
375
  for refs, available, in_request in (
@@ -642,14 +665,12 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
642
665
  )
643
666
  )
644
667
 
645
- fixtures: list[str] = []
646
- if isinstance(test_data.get("fixtures"), list):
647
- fixtures.extend(test_data["fixtures"])
668
+ fixtures: list[str] = list(scenario.fixtures)
648
669
  for stage in scenario.stages:
649
670
  fixtures.extend(stage.fixtures)
650
671
  fixtures = sorted(set(fixtures))
651
672
 
652
- vars_defined = extract_defined_variables(scenario, test_data)
673
+ vars_defined = extract_defined_variables(scenario)
653
674
  vars_saved = extract_saved_variables(scenario)
654
675
  vars_referenced = extract_template_variables(test_data)
655
676
 
@@ -663,6 +684,35 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
663
684
  )
664
685
  )
665
686
 
687
+ # Scenario fixtures are injected into every stage and sit above the global
688
+ # context in the runtime ChainMap, so a save under the same name can never
689
+ # be read back — the fixture value always wins.
690
+ shadowed_saves = set(scenario.fixtures) & vars_saved
691
+ if shadowed_saves:
692
+ diagnostics.append(
693
+ _diag(
694
+ DiagnosticCode.FIXTURE_SHADOWS_SAVE,
695
+ "warning",
696
+ f"Saved variables shadowed by scenario-level fixtures: {sorted(shadowed_saves)} (fixture values win in every stage; these saves can never be read)",
697
+ )
698
+ )
699
+
700
+ # Scenario-level templates (substitutions/auth/ssl) resolve once at class
701
+ # creation (carrier.create_test_class), before any fixture exists — a fixture
702
+ # reference there is a guaranteed collection-time crash.
703
+ for key in ("substitutions", "auth", "ssl"):
704
+ scenario_level_refs = extract_template_variables(test_data.get(key))
705
+ fixture_refs = scenario_level_refs & set(fixtures)
706
+ if fixture_refs:
707
+ diagnostics.append(
708
+ _diag(
709
+ DiagnosticCode.FIXTURE_IN_SCENARIO_TEMPLATE,
710
+ "error",
711
+ f"Fixtures referenced in scenario-level '{key}' templates: {sorted(fixture_refs)} (scenario-level templates resolve at collection time, before fixtures exist)",
712
+ location=key,
713
+ )
714
+ )
715
+
666
716
  # NOTE: response data (response/status_code/body/json/text/headers/cookies) is
667
717
  # NOT ambient in {{ }} templates — it reaches save/verify handlers directly and
668
718
  # only enters the template context via an earlier `save` step. So there are no
@@ -736,6 +786,17 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
736
786
  return diagnostics, scenario_info
737
787
 
738
788
 
789
+ def resolve_root_path(path: Path) -> Path:
790
+ """Directory that constrains ``$ref`` resolution: the nearest ``tests/``
791
+ ancestor of ``path``, else the file's own parent."""
792
+ potential_root = path.parent
793
+ while potential_root.parent != potential_root:
794
+ if potential_root.name == "tests":
795
+ return potential_root
796
+ potential_root = potential_root.parent
797
+ return path.parent
798
+
799
+
739
800
  def validate_scenario(
740
801
  path: Path,
741
802
  ref_parent_traversal_depth: int = 3,
@@ -772,14 +833,7 @@ def validate_scenario(
772
833
  )
773
834
 
774
835
  if root_path is None:
775
- potential_root = path.parent
776
- while potential_root.parent != potential_root:
777
- if potential_root.name == "tests":
778
- root_path = potential_root
779
- break
780
- potential_root = potential_root.parent
781
- else:
782
- root_path = path.parent
836
+ root_path = resolve_root_path(path)
783
837
 
784
838
  try:
785
839
  test_data = pytest_httpchain_jsonref.loader.load_json(
@@ -1,83 +0,0 @@
1
- import enum
2
- import json
3
- from pathlib import Path
4
- from typing import Annotated
5
-
6
- import typer
7
-
8
- app = typer.Typer()
9
-
10
- SKILL_FILE = Path(__file__).parent / "skill.md"
11
-
12
-
13
- class OutputFormat(enum.StrEnum):
14
- text = "text"
15
- json = "json"
16
-
17
-
18
- @app.command()
19
- def validate(
20
- paths: Annotated[list[Path], typer.Argument(help="Scenario JSON file(s) to validate.")],
21
- ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
22
- output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
23
- deep: Annotated[bool, typer.Option("--deep", help="Run deep checks: resolve user-function imports/signatures and referenced files. Imports user modules.")] = False,
24
- syspath: Annotated[list[Path] | None, typer.Option("--syspath", help="Extra directories to add to sys.path for --deep import resolution (repeatable).")] = None,
25
- strict: Annotated[bool, typer.Option("--strict", help="Treat warnings as failures for the exit code.")] = False,
26
- ) -> None:
27
- """Validate pytest-httpchain scenario file(s).
28
-
29
- Reports errors and warnings (each with a stable HTTPCHAINxxx diagnostic code)
30
- per file and exits non-zero if any file is invalid (or, with --strict, has any
31
- warnings).
32
- """
33
- from pytest_httpchain.validation import validate_scenario
34
-
35
- results = [(path, validate_scenario(path, ref_parent_traversal_depth=ref_parent_traversal_depth, deep=deep, syspaths=list(syspath or []))) for path in paths]
36
-
37
- def passed(result) -> bool:
38
- return result.valid and not (strict and result.warnings)
39
-
40
- all_passed = all(passed(result) for _, result in results)
41
-
42
- if output_format is OutputFormat.json:
43
- payload = {
44
- "valid": all_passed,
45
- "files": [{"path": str(path), "result": result.model_dump()} for path, result in results],
46
- }
47
- typer.echo(json.dumps(payload, indent=2, default=str))
48
- else:
49
- for path, result in results:
50
- if not result.valid:
51
- status = "INVALID"
52
- elif result.warnings:
53
- status = "FAILED (warnings)" if strict else "OK with warnings"
54
- else:
55
- status = "OK"
56
- typer.echo(f"{path}: {status}")
57
- for diagnostic in result.diagnostics:
58
- typer.echo(f" {diagnostic.severity} [{diagnostic.code}]: {diagnostic.message}")
59
-
60
- raise typer.Exit(0 if all_passed else 1)
61
-
62
-
63
- @app.command()
64
- def install(
65
- global_: Annotated[bool, typer.Option("--global", "-g", help="Install to ~/.claude (personal scope) instead of project")] = False,
66
- project_dir: Annotated[Path, typer.Option(help="Project directory (ignored with --global)")] = Path("."),
67
- ) -> None:
68
- """Install the Claude Code skill for authoring test scenarios."""
69
- if global_:
70
- _install_skill(Path.home() / ".claude" / "skills" / "pytest-httpchain")
71
- else:
72
- _install_skill(project_dir.resolve() / ".claude" / "skills" / "pytest-httpchain")
73
-
74
-
75
- def _install_skill(skill_dir: Path) -> None:
76
- skill_dir.mkdir(parents=True, exist_ok=True)
77
- dest = skill_dir / "SKILL.md"
78
- dest.write_text(SKILL_FILE.read_text())
79
- typer.echo(f"Installed skill to {dest}")
80
-
81
-
82
- if __name__ == "__main__":
83
- app()
@@ -1,282 +0,0 @@
1
- ---
2
- name: pytest-httpchain
3
- description: Write and edit pytest-httpchain HTTP API test scenarios in JSON format
4
- ---
5
-
6
- # pytest-httpchain test authoring
7
-
8
- pytest-httpchain is a pytest plugin for declarative HTTP API integration testing. Test scenarios are JSON files discovered by pattern `test_<name>.http.json`.
9
-
10
- ## Scenario structure
11
-
12
- ```json
13
- {
14
- "description": "optional scenario description",
15
- "marks": ["optional_pytest_markers"],
16
- "substitutions": [],
17
- "stages": []
18
- }
19
- ```
20
-
21
- ## Stage structure
22
-
23
- ```json
24
- {
25
- "name": "stage name",
26
- "description": "optional",
27
- "fixtures": ["fixture_name"],
28
- "marks": ["skip", "xfail(reason='not ready')"],
29
- "always_run": false,
30
- "substitutions": [],
31
- "parametrize": [],
32
- "parallel": null,
33
- "request": { ... },
34
- "response": [ ... ]
35
- }
36
- ```
37
-
38
- Stages run sequentially and share a global context. Values saved in one stage are available in subsequent stages.
39
-
40
- Stages can also be written as a dict (keys become stage names):
41
-
42
- ```json
43
- {
44
- "stages": {
45
- "create user": { "request": { ... }, "response": [ ... ] },
46
- "get user": { "request": { ... }, "response": [ ... ] }
47
- }
48
- }
49
- ```
50
-
51
- ## Request
52
-
53
- ```json
54
- {
55
- "url": "{{ server }}/api/users",
56
- "method": "POST",
57
- "headers": { "Authorization": "Bearer {{ token }}" },
58
- "params": { "page": 1 },
59
- "body": { "json": { "name": "Alice" } },
60
- "timeout": 30.0,
61
- "allow_redirects": true
62
- }
63
- ```
64
-
65
- **Body types** (use exactly one key):
66
- - `{"json": { ... }}` - JSON body
67
- - `{"form": { ... }}` - URL-encoded form
68
- - `{"text": "..."}` - raw text
69
- - `{"xml": "<root/>"}` - XML
70
- - `{"base64": "..."}` - base64-encoded binary
71
- - `{"binary": "/path/to/file"}` - file upload
72
- - `{"files": {"field": "/path/to/file"}}` - multipart file upload
73
- - `{"graphql": {"query": "...", "variables": {}}}` - GraphQL
74
-
75
- ## Response steps
76
-
77
- Response is a list of verify and save steps, executed in order:
78
-
79
- ```json
80
- "response": [
81
- {
82
- "verify": {
83
- "status": 200,
84
- "headers": { "content-type": "application/json" },
85
- "body": {
86
- "schema": { "type": "object", "required": ["id"] },
87
- "contains": ["expected text"],
88
- "not_contains": ["error"],
89
- "matches": ["\\d{4}-\\d{2}-\\d{2}"],
90
- "not_matches": ["forbidden"]
91
- }
92
- }
93
- },
94
- {
95
- "save": {
96
- "jmespath": {
97
- "user_id": "data.id",
98
- "user_name": "data.name",
99
- "total": "length(items)"
100
- }
101
- }
102
- },
103
- {
104
- "verify": {
105
- "expressions": [
106
- "{{ total > 0 }}",
107
- "{{ user_name != '' }}"
108
- ]
109
- }
110
- }
111
- ]
112
- ```
113
-
114
- **Important:** `verify.expressions` are `{{ }}` templates evaluated against the **context** (saved variables, fixtures, substitutions). The HTTP response is **not** ambient in templates — there is no `response`/`status_code`/`body`/`json` variable. To assert on response data, either:
115
- - use `verify.status`, `verify.headers`, `verify.body` (these check the response directly), or
116
- - `save` the value first (e.g. via `jmespath`) and reference the saved variable in a later `expressions` step (as shown above).
117
-
118
- **Save types:**
119
- - `{"jmespath": {...}}` - extract values from JSON response via JMESPath
120
- - `{"substitutions": [...]}` - compute values using template expressions
121
- - `{"user_functions": [...]}` - call Python functions to process response
122
-
123
- ## Template expressions
124
-
125
- Use `{{ expr }}` syntax. Expressions are evaluated with Python semantics.
126
-
127
- **Available context:** all saved variables, fixture values, and substitution results.
128
-
129
- **Built-in functions:** `len`, `min`, `max`, `sum`, `abs`, `round`, `sorted`, `range`, `zip`, `enumerate`, `bool`, `int`, `float`, `str`, `dict`, `list`, `tuple`, `set`, `uuid4()`, `env(var, default)`, `get(var, default)`, `exists(var)`, `rand()`, `randint(a, b)`
130
-
131
- **JSON literals:** `true`, `false`, `null` map to Python `True`, `False`, `None`.
132
-
133
- ## Substitutions
134
-
135
- Define variables before stages run:
136
-
137
- ```json
138
- "substitutions": [
139
- { "vars": { "base_url": "https://api.example.com", "count": "{{ 2 + 3 }}" } },
140
- { "functions": { "generate_token": "mymodule:create_jwt" } }
141
- ]
142
- ```
143
-
144
- Substitutions can appear at scenario level (global) or stage level (local).
145
-
146
- ## References ($include / $ref)
147
-
148
- Split scenarios across files using `$include` (preferred) or `$ref`:
149
-
150
- ```json
151
- {
152
- "request": {
153
- "$include": "common.json#/requests/get_user"
154
- }
155
- }
156
- ```
157
-
158
- Sibling properties are deep-merged with the referenced content:
159
-
160
- ```json
161
- {
162
- "$include": "base_request.json",
163
- "headers": { "X-Custom": "override" }
164
- }
165
- ```
166
-
167
- ## Parametrize
168
-
169
- Run a stage with different inputs:
170
-
171
- ```json
172
- "parametrize": [
173
- {
174
- "individual": { "user_id": [1, 2, 3] },
175
- "ids": ["user-one", "user-two", "user-three"]
176
- }
177
- ]
178
- ```
179
-
180
- Or use combinations:
181
-
182
- ```json
183
- "parametrize": [
184
- {
185
- "combinations": [
186
- { "method": "GET", "expected": 200 },
187
- { "method": "DELETE", "expected": 403 }
188
- ]
189
- }
190
- ]
191
- ```
192
-
193
- ## Parallel execution
194
-
195
- Execute requests concurrently for load testing:
196
-
197
- ```json
198
- "parallel": {
199
- "repeat": 100,
200
- "max_concurrency": 10,
201
- "calls_per_sec": 50
202
- }
203
- ```
204
-
205
- Or iterate over parameter sets in parallel:
206
-
207
- ```json
208
- "parallel": {
209
- "foreach": [{ "individual": { "id": [1, 2, 3, 4, 5] } }],
210
- "max_concurrency": 5
211
- }
212
- ```
213
-
214
- ## Complete example: multi-stage API test
215
-
216
- ```json
217
- {
218
- "substitutions": [
219
- { "vars": { "base": "{{ env('API_URL', 'http://localhost:8000') }}" } }
220
- ],
221
- "stages": [
222
- {
223
- "name": "create user",
224
- "request": {
225
- "url": "{{ base }}/users",
226
- "method": "POST",
227
- "body": { "json": { "name": "Alice", "email": "alice@example.com" } }
228
- },
229
- "response": [
230
- { "verify": { "status": 201 } },
231
- { "save": { "jmespath": { "user_id": "id" } } }
232
- ]
233
- },
234
- {
235
- "name": "get user",
236
- "request": {
237
- "url": "{{ base }}/users/{{ user_id }}"
238
- },
239
- "response": [
240
- { "verify": { "status": 200 } },
241
- { "save": { "jmespath": { "name": "name" } } },
242
- { "verify": { "expressions": ["{{ name == 'Alice' }}"] } }
243
- ]
244
- },
245
- {
246
- "name": "delete user",
247
- "request": {
248
- "url": "{{ base }}/users/{{ user_id }}",
249
- "method": "DELETE"
250
- },
251
- "response": [
252
- { "verify": { "status": 204 } }
253
- ]
254
- }
255
- ]
256
- }
257
- ```
258
-
259
- ## Validate your scenario
260
-
261
- After writing a scenario, validate it (no server or network needed):
262
-
263
- ```bash
264
- pytest-httpchain validate test_<name>.http.json
265
- ```
266
-
267
- It checks structure plus semantics a JSON Schema cannot, each with a stable `HTTPCHAINxxx` code:
268
-
269
- - `HTTPCHAIN003` — a `{{ var }}` that is never defined, saved, or provided as a fixture (likely a typo).
270
- - `HTTPCHAIN004` — a variable used **before** the stage that saves it, or used in a stage's request when it is only saved in that same stage's response. Remember: a value `save`d in a stage's response is available to *later* response steps and *later* stages, never to the request that produced it.
271
- - `HTTPCHAIN006` — a `verify` step that asserts nothing.
272
- - `HTTPCHAIN007` / `HTTPCHAIN008` — body `contains`/`not_contains` (or `matches`/`not_matches`) that list the same value, which can never pass.
273
-
274
- Add `--format json` for machine-readable output. The same checks run automatically during `pytest --collect-only`.
275
-
276
- For a deeper check that imports your `module:func` references (confirming they resolve and their signatures match — including the injected `response` for save/verify functions) and verifies referenced files/schemas exist, add `--deep` (optionally `--syspath <dir>` for import roots, `--strict` to fail on warnings):
277
-
278
- ```bash
279
- pytest-httpchain validate --deep test_<name>.http.json
280
- ```
281
-
282
- Note: the HTTP response is **not** ambient in `{{ }}` templates — `save` what you need from a response first, then reference the saved variable.