pytest-httpchain 0.4.0__tar.gz → 0.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/PKG-INFO +23 -12
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/README.md +22 -11
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/pyproject.toml +1 -1
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/carrier.py +20 -5
- pytest_httpchain-0.6.0/src/pytest_httpchain/cli.py +222 -0
- pytest_httpchain-0.6.0/src/pytest_httpchain/dataflow.py +137 -0
- pytest_httpchain-0.6.0/src/pytest_httpchain/schema.py +116 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/validation.py +94 -40
- pytest_httpchain-0.4.0/src/pytest_httpchain/cli.py +0 -83
- pytest_httpchain-0.4.0/src/pytest_httpchain/skill.md +0 -282
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/LICENSE +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/__init__.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/constants.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/exceptions.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/har_writer.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/plugin.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/report_formatter.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.6.0}/src/pytest_httpchain/utils.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pytest-httpchain
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.6.0
|
|
4
4
|
Summary: pytest plugin for HTTP testing using JSON files
|
|
5
5
|
Keywords: testing,pytest,requests
|
|
6
6
|
Author: Alexander Eresov
|
|
@@ -198,17 +198,7 @@ pip install 'git+https://github.com/aeresov/pytest-httpchain@main'
|
|
|
198
198
|
|
|
199
199
|
## AI agent support
|
|
200
200
|
|
|
201
|
-
`pytest-httpchain` ships
|
|
202
|
-
|
|
203
|
-
### Claude Code skill
|
|
204
|
-
|
|
205
|
-
Install the authoring skill into your project (or `--global` for personal scope):
|
|
206
|
-
|
|
207
|
-
```bash
|
|
208
|
-
uvx pytest-httpchain install
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
This writes `.claude/skills/pytest-httpchain/SKILL.md` with guidance for writing scenarios.
|
|
201
|
+
`pytest-httpchain` ships a scenario validator to help AI coding agents (and humans) author and check test scenarios.
|
|
212
202
|
|
|
213
203
|
### Scenario validation
|
|
214
204
|
|
|
@@ -242,6 +232,27 @@ A JSON Schema is published for as-you-type validation and autocomplete. Referenc
|
|
|
242
232
|
}
|
|
243
233
|
```
|
|
244
234
|
|
|
235
|
+
The hosted schema tracks the latest release; to pin the schema matching your installed version (e.g. for CI), emit it locally:
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
uvx pytest-httpchain schema --output scenario.schema.json
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
### Inspecting scenarios
|
|
242
|
+
|
|
243
|
+
More read-only commands help author and debug scenarios offline — no network, no test run:
|
|
244
|
+
|
|
245
|
+
```bash
|
|
246
|
+
# Print a scenario with all $ref/$include/$merge inlined and deep-merged
|
|
247
|
+
uvx pytest-httpchain resolve tests/test_login.http.json
|
|
248
|
+
|
|
249
|
+
# Summarize stages and the variable data-flow (which stage saves what, who consumes it)
|
|
250
|
+
uvx pytest-httpchain show tests/test_login.http.json
|
|
251
|
+
|
|
252
|
+
# Render the stage data-flow as a Mermaid flowchart
|
|
253
|
+
uvx pytest-httpchain graph tests/test_login.http.json
|
|
254
|
+
```
|
|
255
|
+
|
|
245
256
|
## Documentation
|
|
246
257
|
|
|
247
258
|
- [Full Documentation](https://aeresov.github.io/pytest-httpchain) - Complete usage guide
|
|
@@ -171,17 +171,7 @@ pip install 'git+https://github.com/aeresov/pytest-httpchain@main'
|
|
|
171
171
|
|
|
172
172
|
## AI agent support
|
|
173
173
|
|
|
174
|
-
`pytest-httpchain` ships
|
|
175
|
-
|
|
176
|
-
### Claude Code skill
|
|
177
|
-
|
|
178
|
-
Install the authoring skill into your project (or `--global` for personal scope):
|
|
179
|
-
|
|
180
|
-
```bash
|
|
181
|
-
uvx pytest-httpchain install
|
|
182
|
-
```
|
|
183
|
-
|
|
184
|
-
This writes `.claude/skills/pytest-httpchain/SKILL.md` with guidance for writing scenarios.
|
|
174
|
+
`pytest-httpchain` ships a scenario validator to help AI coding agents (and humans) author and check test scenarios.
|
|
185
175
|
|
|
186
176
|
### Scenario validation
|
|
187
177
|
|
|
@@ -215,6 +205,27 @@ A JSON Schema is published for as-you-type validation and autocomplete. Referenc
|
|
|
215
205
|
}
|
|
216
206
|
```
|
|
217
207
|
|
|
208
|
+
The hosted schema tracks the latest release; to pin the schema matching your installed version (e.g. for CI), emit it locally:
|
|
209
|
+
|
|
210
|
+
```bash
|
|
211
|
+
uvx pytest-httpchain schema --output scenario.schema.json
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
### Inspecting scenarios
|
|
215
|
+
|
|
216
|
+
More read-only commands help author and debug scenarios offline — no network, no test run:
|
|
217
|
+
|
|
218
|
+
```bash
|
|
219
|
+
# Print a scenario with all $ref/$include/$merge inlined and deep-merged
|
|
220
|
+
uvx pytest-httpchain resolve tests/test_login.http.json
|
|
221
|
+
|
|
222
|
+
# Summarize stages and the variable data-flow (which stage saves what, who consumes it)
|
|
223
|
+
uvx pytest-httpchain show tests/test_login.http.json
|
|
224
|
+
|
|
225
|
+
# Render the stage data-flow as a Mermaid flowchart
|
|
226
|
+
uvx pytest-httpchain graph tests/test_login.http.json
|
|
227
|
+
```
|
|
228
|
+
|
|
218
229
|
## Documentation
|
|
219
230
|
|
|
220
231
|
- [Full Documentation](https://aeresov.github.io/pytest-httpchain) - Complete usage guide
|
|
@@ -79,12 +79,24 @@ class Carrier:
|
|
|
79
79
|
max_parallel_iterations: ClassVar[int] = 10_000
|
|
80
80
|
|
|
81
81
|
@classmethod
|
|
82
|
-
def
|
|
82
|
+
def _resolve_always_run(cls, stage: Stage, stage_fixtures: dict[str, Any]) -> bool:
|
|
83
|
+
"""Resolve ``always_run``, evaluating a template form against the context
|
|
84
|
+
available at stage start: fixtures and parametrize parameters plus the
|
|
85
|
+
global context (scenario substitutions and earlier saves). Stage
|
|
86
|
+
substitutions are not yet processed at this point. The result is coerced
|
|
87
|
+
with Python truthiness."""
|
|
88
|
+
if isinstance(stage.always_run, bool):
|
|
89
|
+
return stage.always_run
|
|
83
90
|
try:
|
|
84
|
-
|
|
85
|
-
|
|
91
|
+
return bool(walk(stage.always_run, ChainMap(stage_fixtures, cls.global_context)))
|
|
92
|
+
except TemplatesError as e:
|
|
93
|
+
raise StageExecutionError(f"Failed to evaluate always_run template: {e}") from e
|
|
86
94
|
|
|
87
|
-
|
|
95
|
+
@classmethod
|
|
96
|
+
def execute_stage(cls, stage: Stage, fixture_kwargs: dict[str, Any]) -> None:
|
|
97
|
+
try:
|
|
98
|
+
# prepare stage context (before the abort check, so an always_run
|
|
99
|
+
# template sees the same fixture layer as stage templates)
|
|
88
100
|
stage_fixtures: dict[str, Any] = {}
|
|
89
101
|
for name, value in fixture_kwargs.items():
|
|
90
102
|
if callable(value) and not inspect.isclass(value):
|
|
@@ -92,6 +104,9 @@ class Carrier:
|
|
|
92
104
|
else:
|
|
93
105
|
stage_fixtures[name] = value
|
|
94
106
|
|
|
107
|
+
if cls.aborted and not cls._resolve_always_run(stage, stage_fixtures):
|
|
108
|
+
pytest.skip(reason="Flow aborted")
|
|
109
|
+
|
|
95
110
|
# Build base context for iterations (substitutions + fixtures + global)
|
|
96
111
|
local_context = ChainMap(stage_fixtures, cls.global_context)
|
|
97
112
|
stage_substitutions = process_substitutions(stage.substitutions, local_context)
|
|
@@ -519,7 +534,7 @@ def create_test_class(scenario: Scenario, class_name: str, max_parallel_iteratio
|
|
|
519
534
|
parametrize_marker = pytest.mark.parametrize(",".join(param_names), param_values, ids=param_ids)
|
|
520
535
|
stage_method = parametrize_marker(stage_method)
|
|
521
536
|
|
|
522
|
-
all_fixtures = ["self"] + all_param_names + stage.fixtures
|
|
537
|
+
all_fixtures = ["self"] + list(dict.fromkeys(all_param_names + stage.fixtures + scenario.fixtures))
|
|
523
538
|
stage_method.__signature__ = inspect.Signature([inspect.Parameter(name, inspect.Parameter.POSITIONAL_OR_KEYWORD) for name in all_fixtures]) # type: ignore[assignment]
|
|
524
539
|
|
|
525
540
|
all_marks = [f"order({i})"] + stage.marks
|
|
@@ -0,0 +1,222 @@
|
|
|
1
|
+
import enum
|
|
2
|
+
import json
|
|
3
|
+
from pathlib import Path
|
|
4
|
+
from typing import Annotated
|
|
5
|
+
|
|
6
|
+
import typer
|
|
7
|
+
|
|
8
|
+
app = typer.Typer()
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
class OutputFormat(enum.StrEnum):
|
|
12
|
+
text = "text"
|
|
13
|
+
json = "json"
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
class GraphDirection(enum.StrEnum):
|
|
17
|
+
TD = "TD"
|
|
18
|
+
LR = "LR"
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
@app.callback()
|
|
22
|
+
def main() -> None:
|
|
23
|
+
"""pytest-httpchain command-line tools."""
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
@app.command()
|
|
27
|
+
def validate(
|
|
28
|
+
paths: Annotated[list[Path], typer.Argument(help="Scenario JSON file(s) to validate.")],
|
|
29
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
30
|
+
output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
|
|
31
|
+
deep: Annotated[bool, typer.Option("--deep", help="Run deep checks: resolve user-function imports/signatures and referenced files. Imports user modules.")] = False,
|
|
32
|
+
syspath: Annotated[list[Path] | None, typer.Option("--syspath", help="Extra directories to add to sys.path for --deep import resolution (repeatable).")] = None,
|
|
33
|
+
strict: Annotated[bool, typer.Option("--strict", help="Treat warnings as failures for the exit code.")] = False,
|
|
34
|
+
) -> None:
|
|
35
|
+
"""Validate pytest-httpchain scenario file(s).
|
|
36
|
+
|
|
37
|
+
Reports errors and warnings (each with a stable HTTPCHAINxxx diagnostic code)
|
|
38
|
+
per file and exits non-zero if any file is invalid (or, with --strict, has any
|
|
39
|
+
warnings).
|
|
40
|
+
"""
|
|
41
|
+
from pytest_httpchain.validation import validate_scenario
|
|
42
|
+
|
|
43
|
+
results = [(path, validate_scenario(path, ref_parent_traversal_depth=ref_parent_traversal_depth, deep=deep, syspaths=list(syspath or []))) for path in paths]
|
|
44
|
+
|
|
45
|
+
def passed(result) -> bool:
|
|
46
|
+
return result.valid and not (strict and result.warnings)
|
|
47
|
+
|
|
48
|
+
all_passed = all(passed(result) for _, result in results)
|
|
49
|
+
|
|
50
|
+
if output_format is OutputFormat.json:
|
|
51
|
+
payload = {
|
|
52
|
+
"valid": all_passed,
|
|
53
|
+
"files": [{"path": str(path), "result": result.model_dump()} for path, result in results],
|
|
54
|
+
}
|
|
55
|
+
typer.echo(json.dumps(payload, indent=2, default=str))
|
|
56
|
+
else:
|
|
57
|
+
for path, result in results:
|
|
58
|
+
if not result.valid:
|
|
59
|
+
status = "INVALID"
|
|
60
|
+
elif result.warnings:
|
|
61
|
+
status = "FAILED (warnings)" if strict else "OK with warnings"
|
|
62
|
+
else:
|
|
63
|
+
status = "OK"
|
|
64
|
+
typer.echo(f"{path}: {status}")
|
|
65
|
+
for diagnostic in result.diagnostics:
|
|
66
|
+
typer.echo(f" {diagnostic.severity} [{diagnostic.code}]: {diagnostic.message}")
|
|
67
|
+
|
|
68
|
+
raise typer.Exit(0 if all_passed else 1)
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
@app.command()
|
|
72
|
+
def schema(
|
|
73
|
+
output: Annotated[Path | None, typer.Option("--output", "-o", help="Write schema to PATH instead of stdout.")] = None,
|
|
74
|
+
) -> None:
|
|
75
|
+
"""Emit the JSON Schema for scenario files (editor autocomplete/validation)."""
|
|
76
|
+
from pytest_httpchain.schema import build_schema
|
|
77
|
+
|
|
78
|
+
text = json.dumps(build_schema(), indent=2, default=str)
|
|
79
|
+
if output is not None:
|
|
80
|
+
output.write_text(text + "\n")
|
|
81
|
+
typer.echo(f"Wrote schema to {output}")
|
|
82
|
+
else:
|
|
83
|
+
typer.echo(text)
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
@app.command()
|
|
87
|
+
def resolve(
|
|
88
|
+
scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to resolve.")],
|
|
89
|
+
output: Annotated[Path | None, typer.Option("--output", "-o", help="Write resolved JSON to PATH instead of stdout.")] = None,
|
|
90
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
91
|
+
) -> None:
|
|
92
|
+
"""Resolve $ref/$include/$merge and print the merged scenario JSON."""
|
|
93
|
+
import pytest_httpchain_jsonref.loader
|
|
94
|
+
from pytest_httpchain_jsonref.exceptions import ReferenceResolverError
|
|
95
|
+
|
|
96
|
+
from pytest_httpchain.validation import resolve_root_path
|
|
97
|
+
|
|
98
|
+
try:
|
|
99
|
+
data = pytest_httpchain_jsonref.loader.load_json(
|
|
100
|
+
scenario,
|
|
101
|
+
max_parent_traversal_depth=ref_parent_traversal_depth,
|
|
102
|
+
root_path=resolve_root_path(scenario),
|
|
103
|
+
)
|
|
104
|
+
except (ReferenceResolverError, json.JSONDecodeError, OSError) as e:
|
|
105
|
+
typer.echo(f"error: {e}", err=True)
|
|
106
|
+
raise typer.Exit(1) from e
|
|
107
|
+
|
|
108
|
+
text = json.dumps(data, indent=2, default=str)
|
|
109
|
+
if output is not None:
|
|
110
|
+
output.write_text(text + "\n")
|
|
111
|
+
typer.echo(f"Wrote resolved scenario to {output}")
|
|
112
|
+
else:
|
|
113
|
+
typer.echo(text)
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
def _load_for_inspection(path: Path, depth: int):
|
|
117
|
+
"""Load + validate a scenario for show/graph, mapping failures to Exit(1)."""
|
|
118
|
+
from pydantic import ValidationError
|
|
119
|
+
from pytest_httpchain_jsonref.exceptions import ReferenceResolverError
|
|
120
|
+
|
|
121
|
+
from pytest_httpchain.dataflow import load_scenario
|
|
122
|
+
|
|
123
|
+
try:
|
|
124
|
+
return load_scenario(path, ref_parent_traversal_depth=depth)
|
|
125
|
+
except (ReferenceResolverError, json.JSONDecodeError, OSError) as e:
|
|
126
|
+
typer.echo(f"error: cannot load {path}: {e}", err=True)
|
|
127
|
+
raise typer.Exit(1) from e
|
|
128
|
+
except ValidationError:
|
|
129
|
+
typer.echo(f"error: {path} is not a valid scenario — run `pytest-httpchain validate {path}` for details", err=True)
|
|
130
|
+
raise typer.Exit(1) from None
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
def _render_show_text(path: Path, scenario, flow) -> list[str]:
|
|
134
|
+
producer_of: dict[tuple[int, str], int] = {}
|
|
135
|
+
for edge in flow.edges:
|
|
136
|
+
for name in edge.vars:
|
|
137
|
+
producer_of[(edge.consumer, name)] = edge.producer
|
|
138
|
+
|
|
139
|
+
all_fixtures = sorted({*flow.scenario_fixtures, *(f for s in flow.stages for f in s.fixtures)})
|
|
140
|
+
lines: list[str] = [scenario.description or path.name]
|
|
141
|
+
summary = f"{len(flow.stages)} stage(s)"
|
|
142
|
+
if all_fixtures:
|
|
143
|
+
summary += f" · fixtures: {all_fixtures}"
|
|
144
|
+
if flow.scenario_vars:
|
|
145
|
+
summary += f" · vars: {flow.scenario_vars}"
|
|
146
|
+
lines.append(summary)
|
|
147
|
+
lines.append("")
|
|
148
|
+
|
|
149
|
+
for s in flow.stages:
|
|
150
|
+
name = s.name or f"(stage {s.index + 1})"
|
|
151
|
+
lines.append(f"{s.index + 1} · {name} {s.method} {s.url}")
|
|
152
|
+
if s.saves:
|
|
153
|
+
lines.append(f" saves: {', '.join(s.saves)}")
|
|
154
|
+
if s.consumes:
|
|
155
|
+
parts: list[str] = []
|
|
156
|
+
for name in s.consumes:
|
|
157
|
+
producer = producer_of.get((s.index, name))
|
|
158
|
+
if producer is None:
|
|
159
|
+
parts.append(name)
|
|
160
|
+
else:
|
|
161
|
+
producer_name = flow.stages[producer].name or f"stage {producer + 1}"
|
|
162
|
+
parts.append(f"{name} (from #{producer + 1} {producer_name})")
|
|
163
|
+
lines.append(f" consumes: {', '.join(parts)}")
|
|
164
|
+
if s.marks:
|
|
165
|
+
lines.append(f" marks: {', '.join(s.marks)}")
|
|
166
|
+
return lines
|
|
167
|
+
|
|
168
|
+
|
|
169
|
+
@app.command()
|
|
170
|
+
def show(
|
|
171
|
+
scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to summarize.")],
|
|
172
|
+
output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
|
|
173
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
174
|
+
) -> None:
|
|
175
|
+
"""Summarize a scenario's stages and variable data-flow."""
|
|
176
|
+
from pytest_httpchain.dataflow import analyze_dataflow
|
|
177
|
+
|
|
178
|
+
sc, test_data = _load_for_inspection(scenario, ref_parent_traversal_depth)
|
|
179
|
+
flow = analyze_dataflow(sc, test_data)
|
|
180
|
+
|
|
181
|
+
if output_format is OutputFormat.json:
|
|
182
|
+
payload = flow.model_dump()
|
|
183
|
+
payload["description"] = sc.description or None
|
|
184
|
+
typer.echo(json.dumps(payload, indent=2, default=str))
|
|
185
|
+
else:
|
|
186
|
+
for line in _render_show_text(scenario, sc, flow):
|
|
187
|
+
typer.echo(line)
|
|
188
|
+
|
|
189
|
+
|
|
190
|
+
def _mermaid_label(text: str) -> str:
|
|
191
|
+
return text.replace('"', "'").replace("\n", " ")
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
def _to_mermaid(flow, direction: str = "TD") -> str:
|
|
195
|
+
lines = [f"flowchart {direction}"]
|
|
196
|
+
if not flow.stages:
|
|
197
|
+
lines.append(" %% (no stages)")
|
|
198
|
+
return "\n".join(lines)
|
|
199
|
+
for s in flow.stages:
|
|
200
|
+
label = f"{s.index + 1} · {s.name}" if s.name else f"{s.index + 1}"
|
|
201
|
+
lines.append(f' S{s.index}["{_mermaid_label(label)}"]')
|
|
202
|
+
for edge in flow.edges:
|
|
203
|
+
lines.append(f" S{edge.producer} -->|{', '.join(edge.vars)}| S{edge.consumer}")
|
|
204
|
+
return "\n".join(lines)
|
|
205
|
+
|
|
206
|
+
|
|
207
|
+
@app.command()
|
|
208
|
+
def graph(
|
|
209
|
+
scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to graph.")],
|
|
210
|
+
direction: Annotated[GraphDirection, typer.Option("--direction", help="Flowchart orientation.")] = GraphDirection.TD,
|
|
211
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
212
|
+
) -> None:
|
|
213
|
+
"""Emit a Mermaid flowchart of the stage data-flow."""
|
|
214
|
+
from pytest_httpchain.dataflow import analyze_dataflow
|
|
215
|
+
|
|
216
|
+
sc, test_data = _load_for_inspection(scenario, ref_parent_traversal_depth)
|
|
217
|
+
flow = analyze_dataflow(sc, test_data)
|
|
218
|
+
typer.echo(_to_mermaid(flow, direction.value))
|
|
219
|
+
|
|
220
|
+
|
|
221
|
+
if __name__ == "__main__":
|
|
222
|
+
app()
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
"""Structured stage data-flow analysis for the ``show`` and ``graph`` CLI commands.
|
|
2
|
+
|
|
3
|
+
Reuses the same extraction helpers as the order-aware validator
|
|
4
|
+
(``validation._dataflow_diagnostics``) but produces a graph model instead of
|
|
5
|
+
diagnostics: which variables each stage saves, which it consumes from earlier
|
|
6
|
+
stages, and the producer -> consumer edges between stages.
|
|
7
|
+
"""
|
|
8
|
+
|
|
9
|
+
from pathlib import Path
|
|
10
|
+
from typing import Any
|
|
11
|
+
|
|
12
|
+
import pytest_httpchain_jsonref.loader
|
|
13
|
+
from pydantic import BaseModel
|
|
14
|
+
from pytest_httpchain_models import Scenario
|
|
15
|
+
|
|
16
|
+
from pytest_httpchain.validation import (
|
|
17
|
+
_parameter_names,
|
|
18
|
+
extract_template_variables,
|
|
19
|
+
raw_stages,
|
|
20
|
+
resolve_root_path,
|
|
21
|
+
saved_in_stage,
|
|
22
|
+
stage_defined_names,
|
|
23
|
+
substitution_names,
|
|
24
|
+
)
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
class DataFlowEdge(BaseModel):
|
|
28
|
+
"""A data dependency: ``vars`` saved by stage ``producer`` are referenced by stage ``consumer``."""
|
|
29
|
+
|
|
30
|
+
producer: int
|
|
31
|
+
consumer: int
|
|
32
|
+
vars: list[str]
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
class StageFlow(BaseModel):
|
|
36
|
+
"""Per-stage data-flow summary."""
|
|
37
|
+
|
|
38
|
+
index: int
|
|
39
|
+
name: str
|
|
40
|
+
method: str
|
|
41
|
+
url: str
|
|
42
|
+
fixtures: list[str]
|
|
43
|
+
marks: list[str]
|
|
44
|
+
saves: list[str]
|
|
45
|
+
consumes: list[str]
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
class DataFlow(BaseModel):
|
|
49
|
+
"""Whole-scenario data-flow graph."""
|
|
50
|
+
|
|
51
|
+
stages: list[StageFlow]
|
|
52
|
+
edges: list[DataFlowEdge]
|
|
53
|
+
scenario_fixtures: list[str] = []
|
|
54
|
+
scenario_vars: list[str] = []
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def analyze_dataflow(scenario: Scenario, test_data: dict[str, Any]) -> DataFlow:
|
|
58
|
+
"""Build the stage data-flow graph for a validated scenario.
|
|
59
|
+
|
|
60
|
+
A stage *consumes* a variable when its request/response/substitutions/parallel
|
|
61
|
+
templates reference a name saved by an earlier stage and not redefined locally
|
|
62
|
+
(own substitutions, parametrize/foreach parameters, or stage/scenario fixtures).
|
|
63
|
+
``always_run`` references count too, but only fixtures and parametrize
|
|
64
|
+
parameters shadow them — always_run resolves before stage substitutions exist.
|
|
65
|
+
``parametrize`` values are excluded — they resolve against scenario scope,
|
|
66
|
+
never saved values.
|
|
67
|
+
"""
|
|
68
|
+
raws = raw_stages(test_data)
|
|
69
|
+
saves_by_stage = [saved_in_stage(stage) for stage in scenario.stages]
|
|
70
|
+
scenario_fixture_names = set(scenario.fixtures)
|
|
71
|
+
|
|
72
|
+
first_save_stage: dict[str, int] = {}
|
|
73
|
+
for i, saved in enumerate(saves_by_stage):
|
|
74
|
+
for name in saved:
|
|
75
|
+
first_save_stage.setdefault(name, i)
|
|
76
|
+
|
|
77
|
+
stages: list[StageFlow] = []
|
|
78
|
+
edges: list[DataFlowEdge] = []
|
|
79
|
+
cumulative_saves: set[str] = set()
|
|
80
|
+
|
|
81
|
+
for i, stage in enumerate(scenario.stages):
|
|
82
|
+
raw = raws[i] if i < len(raws) and isinstance(raws[i], dict) else {}
|
|
83
|
+
|
|
84
|
+
refs: set[str] = set()
|
|
85
|
+
for key in ("request", "response", "substitutions", "parallel"):
|
|
86
|
+
extract_template_variables(raw.get(key), refs)
|
|
87
|
+
|
|
88
|
+
# Scenario fixtures count as local everywhere: at runtime they sit above
|
|
89
|
+
# the global context in the ChainMap, shadowing any same-named save.
|
|
90
|
+
local = stage_defined_names(stage) | scenario_fixture_names
|
|
91
|
+
consumes = {name for name in refs if name in cumulative_saves and name not in local}
|
|
92
|
+
|
|
93
|
+
# always_run resolves before stage substitutions exist, so only fixtures
|
|
94
|
+
# and parametrize parameters shadow an earlier save there.
|
|
95
|
+
always_run_refs = extract_template_variables(raw.get("always_run"))
|
|
96
|
+
always_run_local = set(stage.fixtures) | _parameter_names(stage.parametrize) | scenario_fixture_names
|
|
97
|
+
consumes |= {name for name in always_run_refs if name in cumulative_saves and name not in always_run_local}
|
|
98
|
+
|
|
99
|
+
by_producer: dict[int, list[str]] = {}
|
|
100
|
+
for name in consumes:
|
|
101
|
+
by_producer.setdefault(first_save_stage[name], []).append(name)
|
|
102
|
+
for producer in sorted(by_producer):
|
|
103
|
+
edges.append(DataFlowEdge(producer=producer, consumer=i, vars=sorted(by_producer[producer])))
|
|
104
|
+
|
|
105
|
+
stages.append(
|
|
106
|
+
StageFlow(
|
|
107
|
+
index=i,
|
|
108
|
+
name=stage.name,
|
|
109
|
+
method=str(stage.request.method),
|
|
110
|
+
url=str(stage.request.url),
|
|
111
|
+
fixtures=sorted(stage.fixtures),
|
|
112
|
+
marks=list(stage.marks),
|
|
113
|
+
saves=sorted(saves_by_stage[i]),
|
|
114
|
+
consumes=sorted(consumes),
|
|
115
|
+
)
|
|
116
|
+
)
|
|
117
|
+
|
|
118
|
+
cumulative_saves |= saves_by_stage[i]
|
|
119
|
+
|
|
120
|
+
scenario_var_names = set(substitution_names(scenario.substitutions))
|
|
121
|
+
|
|
122
|
+
return DataFlow(stages=stages, edges=edges, scenario_fixtures=sorted(scenario.fixtures), scenario_vars=sorted(scenario_var_names))
|
|
123
|
+
|
|
124
|
+
|
|
125
|
+
def load_scenario(path: Path, ref_parent_traversal_depth: int = 3) -> tuple[Scenario, dict[str, Any]]:
|
|
126
|
+
"""Load + ``$ref``-resolve + validate a scenario file.
|
|
127
|
+
|
|
128
|
+
Returns ``(scenario, raw_test_data)``. Raises ``ReferenceResolverError``,
|
|
129
|
+
``json.JSONDecodeError`` or ``pydantic.ValidationError`` on failure — callers
|
|
130
|
+
map these to user-facing errors.
|
|
131
|
+
"""
|
|
132
|
+
test_data = pytest_httpchain_jsonref.loader.load_json(
|
|
133
|
+
path,
|
|
134
|
+
max_parent_traversal_depth=ref_parent_traversal_depth,
|
|
135
|
+
root_path=resolve_root_path(path),
|
|
136
|
+
)
|
|
137
|
+
return Scenario.model_validate(test_data), test_data
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
"""Build the JSON Schema for pytest-httpchain scenario files.
|
|
2
|
+
|
|
3
|
+
Shared by the ``pytest-httpchain schema`` CLI command and
|
|
4
|
+
``scripts/generate_schema.py``. The schema is derived from the Pydantic
|
|
5
|
+
``Scenario`` model and augmented so editors accept pytest-httpchain's
|
|
6
|
+
``$ref``/``$include``/``$merge`` reference directives at any object level.
|
|
7
|
+
"""
|
|
8
|
+
|
|
9
|
+
from typing import Any
|
|
10
|
+
|
|
11
|
+
from pytest_httpchain_models import Scenario
|
|
12
|
+
|
|
13
|
+
SCHEMA_DIALECT = "https://json-schema.org/draft/2020-12/schema"
|
|
14
|
+
SCHEMA_ID = "https://aeresov.github.io/pytest-httpchain/schema/scenario.schema.json"
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def _one_of_to_any_of(node: Any) -> None:
|
|
18
|
+
"""Recursively rename ``oneOf`` to ``anyOf`` in place."""
|
|
19
|
+
if isinstance(node, dict):
|
|
20
|
+
if "oneOf" in node:
|
|
21
|
+
node["anyOf"] = node.pop("oneOf")
|
|
22
|
+
for value in node.values():
|
|
23
|
+
_one_of_to_any_of(value)
|
|
24
|
+
elif isinstance(node, list):
|
|
25
|
+
for item in node:
|
|
26
|
+
_one_of_to_any_of(item)
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
def _add_jsonref_support(schema: dict[str, Any]) -> dict[str, Any]:
|
|
30
|
+
"""Allow ``$include``/``$merge``/``$ref`` objects as alternatives anywhere.
|
|
31
|
+
|
|
32
|
+
pytest-httpchain-jsonref can substitute any element at runtime, so each
|
|
33
|
+
definition and root property is wrapped in an ``anyOf`` that also accepts a
|
|
34
|
+
reference object — otherwise editors flag missing required properties when a
|
|
35
|
+
reference is used.
|
|
36
|
+
"""
|
|
37
|
+
if "$defs" not in schema:
|
|
38
|
+
schema["$defs"] = {}
|
|
39
|
+
|
|
40
|
+
schema["$defs"]["JsonRef"] = {
|
|
41
|
+
"type": "object",
|
|
42
|
+
"description": "Reference to external JSON file or JSON pointer. Use $include or $merge (preferred) or $ref. Resolved at runtime by pytest-httpchain-jsonref.",
|
|
43
|
+
"properties": {
|
|
44
|
+
"$include": {
|
|
45
|
+
"type": "string",
|
|
46
|
+
"description": "Path to external JSON file, JSON pointer (#/path), or combined (file.json#/path). Preferred over $ref to avoid VS Code conflicts.",
|
|
47
|
+
},
|
|
48
|
+
"$merge": {
|
|
49
|
+
"type": "string",
|
|
50
|
+
"description": "Alias for $include. Path to external JSON file, JSON pointer (#/path), or combined (file.json#/path).",
|
|
51
|
+
},
|
|
52
|
+
"$ref": {
|
|
53
|
+
"type": "string",
|
|
54
|
+
"description": "Legacy alias for $include. May conflict with VS Code's own $ref handling.",
|
|
55
|
+
},
|
|
56
|
+
},
|
|
57
|
+
# Without at least one directive key this branch would match EVERY
|
|
58
|
+
# object, silencing the strict alternative in the surrounding anyOf.
|
|
59
|
+
"anyOf": [
|
|
60
|
+
{"required": ["$include"]},
|
|
61
|
+
{"required": ["$merge"]},
|
|
62
|
+
{"required": ["$ref"]},
|
|
63
|
+
],
|
|
64
|
+
"additionalProperties": True,
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
# Pydantic emits oneOf for tagged unions. A reference object matches the
|
|
68
|
+
# JsonRef branch of EVERY union member (each $defs entry is wrapped below),
|
|
69
|
+
# which oneOf counts as "valid under more than one" and rejects. anyOf
|
|
70
|
+
# keeps the same accept set otherwise: members forbid each other's tag
|
|
71
|
+
# fields, so a non-reference object can never match two branches.
|
|
72
|
+
_one_of_to_any_of(schema)
|
|
73
|
+
|
|
74
|
+
for type_name, original_def in list(schema["$defs"].items()):
|
|
75
|
+
if type_name == "JsonRef":
|
|
76
|
+
continue
|
|
77
|
+
schema["$defs"][type_name] = {"anyOf": [{"$ref": "#/$defs/JsonRef"}, original_def]}
|
|
78
|
+
if "title" in original_def:
|
|
79
|
+
schema["$defs"][type_name]["title"] = original_def.pop("title")
|
|
80
|
+
if "description" in original_def:
|
|
81
|
+
schema["$defs"][type_name]["description"] = original_def.pop("description")
|
|
82
|
+
|
|
83
|
+
for prop_name, prop_def in list(schema.get("properties", {}).items()):
|
|
84
|
+
schema["properties"][prop_name] = {
|
|
85
|
+
"anyOf": [{"$ref": "#/$defs/JsonRef"}, prop_def],
|
|
86
|
+
"title": prop_def.get("title", prop_name),
|
|
87
|
+
}
|
|
88
|
+
if "description" in prop_def:
|
|
89
|
+
schema["properties"][prop_name]["description"] = prop_def.get("description")
|
|
90
|
+
|
|
91
|
+
# The Scenario model forbids extra keys, so the root carries
|
|
92
|
+
# additionalProperties: false. Keys that are legitimate in a scenario
|
|
93
|
+
# *file* but handled before model validation must be declared explicitly:
|
|
94
|
+
# "$schema" (editor metadata, stripped by the loader) and the reference
|
|
95
|
+
# directives (resolved by the loader, supported at the document root).
|
|
96
|
+
schema.setdefault("properties", {})
|
|
97
|
+
schema["properties"]["$schema"] = {
|
|
98
|
+
"type": "string",
|
|
99
|
+
"description": "URL of this schema, for editor as-you-type validation. Stripped before the file is parsed.",
|
|
100
|
+
}
|
|
101
|
+
for directive, directive_def in schema["$defs"]["JsonRef"]["properties"].items():
|
|
102
|
+
schema["properties"][directive] = directive_def
|
|
103
|
+
schema["properties"]["$ref"] = {
|
|
104
|
+
"type": "string",
|
|
105
|
+
"description": "Legacy alias for $include. May conflict with VS Code's own $ref handling.",
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
return schema
|
|
109
|
+
|
|
110
|
+
|
|
111
|
+
def build_schema() -> dict[str, Any]:
|
|
112
|
+
"""Return the augmented JSON Schema dict for the ``Scenario`` model."""
|
|
113
|
+
schema = Scenario.model_json_schema()
|
|
114
|
+
schema["$schema"] = SCHEMA_DIALECT
|
|
115
|
+
schema["$id"] = SCHEMA_ID
|
|
116
|
+
return _add_jsonref_support(schema)
|
|
@@ -24,12 +24,14 @@ Code Severity Meaning
|
|
|
24
24
|
006 warning Verify step asserts nothing (no-op)
|
|
25
25
|
007 error Body ``contains``/``not_contains`` list the same substring
|
|
26
26
|
008 error Body ``matches``/``not_matches`` list the same pattern
|
|
27
|
+
009 warning Saved variable is shadowed by a scenario-level fixture
|
|
27
28
|
010 error File not found
|
|
28
29
|
011 error Path is not a file
|
|
29
30
|
012 error ``$ref`` resolution failed
|
|
30
31
|
013 warning File extension is not ``.json``
|
|
31
32
|
014 error Invalid JSON syntax
|
|
32
33
|
015 error Failed to parse JSON file
|
|
34
|
+
016 error Fixture referenced in a scenario-level template
|
|
33
35
|
020 warning Referenced file does not exist (deep, opt-in)
|
|
34
36
|
021 warning Schema file is not valid JSON / not a valid schema (deep)
|
|
35
37
|
022 warning User function cannot be imported (deep)
|
|
@@ -70,12 +72,14 @@ class DiagnosticCode:
|
|
|
70
72
|
NOOP_VERIFY = "HTTPCHAIN006"
|
|
71
73
|
CONTAINS_CONTRADICTION = "HTTPCHAIN007"
|
|
72
74
|
MATCHES_CONTRADICTION = "HTTPCHAIN008"
|
|
75
|
+
FIXTURE_SHADOWS_SAVE = "HTTPCHAIN009"
|
|
73
76
|
FILE_NOT_FOUND = "HTTPCHAIN010"
|
|
74
77
|
NOT_A_FILE = "HTTPCHAIN011"
|
|
75
78
|
REF_ERROR = "HTTPCHAIN012"
|
|
76
79
|
WRONG_EXTENSION = "HTTPCHAIN013"
|
|
77
80
|
INVALID_JSON = "HTTPCHAIN014"
|
|
78
81
|
PARSE_ERROR = "HTTPCHAIN015"
|
|
82
|
+
FIXTURE_IN_SCENARIO_TEMPLATE = "HTTPCHAIN016"
|
|
79
83
|
# Deep (opt-in) checks: imports, signatures, referenced files.
|
|
80
84
|
REFERENCED_FILE_NOT_FOUND = "HTTPCHAIN020"
|
|
81
85
|
SCHEMA_FILE_INVALID = "HTTPCHAIN021"
|
|
@@ -152,7 +156,7 @@ def extract_template_variables(obj: Any, variables: set[str] | None = None) -> s
|
|
|
152
156
|
return variables
|
|
153
157
|
|
|
154
158
|
|
|
155
|
-
def
|
|
159
|
+
def substitution_names(substitutions: Any) -> set[str]:
|
|
156
160
|
"""Names introduced by a list of ``vars``/``functions`` substitution entries."""
|
|
157
161
|
names: set[str] = set()
|
|
158
162
|
for sub in substitutions or []:
|
|
@@ -165,7 +169,7 @@ def _substitution_names(substitutions: Any) -> set[str]:
|
|
|
165
169
|
return names
|
|
166
170
|
|
|
167
171
|
|
|
168
|
-
def
|
|
172
|
+
def saved_in_stage(stage: Stage) -> set[str]:
|
|
169
173
|
"""Variable names a single stage's response steps save into the context."""
|
|
170
174
|
saved: set[str] = set()
|
|
171
175
|
for response_step in stage.response:
|
|
@@ -177,7 +181,7 @@ def _saved_in_stage(stage: Stage) -> set[str]:
|
|
|
177
181
|
saved.update(jmespath.keys())
|
|
178
182
|
substitutions = getattr(save, "substitutions", None)
|
|
179
183
|
if substitutions is not None:
|
|
180
|
-
saved |=
|
|
184
|
+
saved |= substitution_names(substitutions)
|
|
181
185
|
# user_functions saves return arbitrary dict keys -> not statically known.
|
|
182
186
|
return saved
|
|
183
187
|
|
|
@@ -186,7 +190,7 @@ def extract_saved_variables(scenario: Scenario) -> set[str]:
|
|
|
186
190
|
"""Extract variable names saved across all response steps in the scenario."""
|
|
187
191
|
saved_vars: set[str] = set()
|
|
188
192
|
for stage in scenario.stages:
|
|
189
|
-
saved_vars |=
|
|
193
|
+
saved_vars |= saved_in_stage(stage)
|
|
190
194
|
return saved_vars
|
|
191
195
|
|
|
192
196
|
|
|
@@ -210,10 +214,10 @@ def _parameter_names(params: Any) -> set[str]:
|
|
|
210
214
|
return names
|
|
211
215
|
|
|
212
216
|
|
|
213
|
-
def
|
|
217
|
+
def stage_defined_names(stage: Stage) -> set[str]:
|
|
214
218
|
"""Names available *within a single stage*: its substitutions, parametrize /
|
|
215
219
|
foreach parameters, and its declared fixtures."""
|
|
216
|
-
names =
|
|
220
|
+
names = substitution_names(stage.substitutions)
|
|
217
221
|
names |= _parameter_names(stage.parametrize)
|
|
218
222
|
if stage.parallel is not None:
|
|
219
223
|
names |= _parameter_names(getattr(stage.parallel, "foreach", None))
|
|
@@ -221,7 +225,7 @@ def _stage_defined_names(stage: Stage) -> set[str]:
|
|
|
221
225
|
return names
|
|
222
226
|
|
|
223
227
|
|
|
224
|
-
def extract_defined_variables(scenario: Scenario
|
|
228
|
+
def extract_defined_variables(scenario: Scenario) -> set[str]:
|
|
225
229
|
"""Extract variable names made available before/within templates (scenario-wide).
|
|
226
230
|
|
|
227
231
|
Sources: ``vars`` and ``functions`` substitutions (scenario- and stage-level),
|
|
@@ -232,14 +236,10 @@ def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) ->
|
|
|
232
236
|
"""
|
|
233
237
|
defined_vars: set[str] = set()
|
|
234
238
|
|
|
235
|
-
|
|
236
|
-
if isinstance(test_data.get("vars"), dict):
|
|
237
|
-
defined_vars.update(k for k in test_data["vars"] if isinstance(k, str))
|
|
238
|
-
|
|
239
|
-
defined_vars |= _substitution_names(scenario.substitutions)
|
|
239
|
+
defined_vars |= substitution_names(scenario.substitutions)
|
|
240
240
|
|
|
241
241
|
for stage in scenario.stages:
|
|
242
|
-
defined_vars |=
|
|
242
|
+
defined_vars |= substitution_names(stage.substitutions)
|
|
243
243
|
defined_vars |= _parameter_names(stage.parametrize)
|
|
244
244
|
if stage.parallel is not None:
|
|
245
245
|
defined_vars |= _parameter_names(getattr(stage.parallel, "foreach", None))
|
|
@@ -247,7 +247,7 @@ def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) ->
|
|
|
247
247
|
return defined_vars
|
|
248
248
|
|
|
249
249
|
|
|
250
|
-
def
|
|
250
|
+
def raw_stages(test_data: dict[str, Any]) -> list[Any]:
|
|
251
251
|
"""Raw (pre-validation) stage bodies in declaration order.
|
|
252
252
|
|
|
253
253
|
Stages may be authored as a list or as a ``{name: stage}`` mapping; both
|
|
@@ -283,13 +283,16 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
|
|
|
283
283
|
Walks stages in execution order tracking which variables are available at
|
|
284
284
|
each reference site, mirroring the runtime scoping in ``carrier.py``:
|
|
285
285
|
|
|
286
|
-
* scenario-level substitutions
|
|
287
|
-
|
|
286
|
+
* scenario-level substitutions and scenario-level fixtures are available
|
|
287
|
+
everywhere;
|
|
288
288
|
* a stage's own substitutions, parametrize/foreach parameters and fixtures
|
|
289
289
|
are available to that stage's request and response;
|
|
290
290
|
* a value saved in a stage's response is available to that stage's response
|
|
291
291
|
and to *later* stages — but never to the same stage's request, and never
|
|
292
|
-
to earlier stages
|
|
292
|
+
to earlier stages;
|
|
293
|
+
* ``always_run`` is evaluated at stage start, before stage substitutions are
|
|
294
|
+
processed and before any iteration runs: only fixtures, parametrize
|
|
295
|
+
parameters, scenario substitutions and earlier saves are in scope.
|
|
293
296
|
|
|
294
297
|
A reference that is unavailable is reported as :data:`DiagnosticCode.FORWARD_REF`
|
|
295
298
|
if the name is saved somewhere later (an ordering bug) or
|
|
@@ -298,7 +301,7 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
|
|
|
298
301
|
diagnostics: list[Diagnostic] = []
|
|
299
302
|
|
|
300
303
|
all_saved = extract_saved_variables(scenario)
|
|
301
|
-
saves_by_stage: list[set[str]] = [
|
|
304
|
+
saves_by_stage: list[set[str]] = [saved_in_stage(stage) for stage in scenario.stages]
|
|
302
305
|
first_save_stage: dict[str, int] = {}
|
|
303
306
|
for i, saved in enumerate(saves_by_stage):
|
|
304
307
|
for name in saved:
|
|
@@ -309,22 +312,17 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
|
|
|
309
312
|
# fixtures, no stage substitutions, no parameter names, no saved values exist
|
|
310
313
|
# yet. ``scenario_scope`` is that narrow set; ``scenario_available`` is the
|
|
311
314
|
# everywhere-available set used for ordinary request/response references.
|
|
312
|
-
scenario_scope: set[str] = set()
|
|
313
|
-
if isinstance(test_data.get("vars"), dict):
|
|
314
|
-
scenario_scope |= {k for k in test_data["vars"] if isinstance(k, str)}
|
|
315
|
-
scenario_scope |= _substitution_names(scenario.substitutions)
|
|
315
|
+
scenario_scope: set[str] = set(substitution_names(scenario.substitutions))
|
|
316
316
|
|
|
317
|
-
scenario_available = set(
|
|
318
|
-
if isinstance(test_data.get("fixtures"), list):
|
|
319
|
-
scenario_available |= {f for f in test_data["fixtures"] if isinstance(f, str)}
|
|
317
|
+
scenario_available = scenario_scope | set(scenario.fixtures)
|
|
320
318
|
|
|
321
|
-
|
|
319
|
+
raws = raw_stages(test_data)
|
|
322
320
|
cumulative_saves: set[str] = set()
|
|
323
321
|
|
|
324
322
|
for i, stage in enumerate(scenario.stages):
|
|
325
|
-
raw =
|
|
323
|
+
raw = raws[i] if i < len(raws) and isinstance(raws[i], dict) else {}
|
|
326
324
|
|
|
327
|
-
request_available = scenario_available |
|
|
325
|
+
request_available = scenario_available | stage_defined_names(stage) | cumulative_saves
|
|
328
326
|
response_available = request_available | saves_by_stage[i]
|
|
329
327
|
|
|
330
328
|
# ``parallel.foreach`` values are resolved at stage execution against the
|
|
@@ -347,6 +345,31 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
|
|
|
347
345
|
)
|
|
348
346
|
)
|
|
349
347
|
|
|
348
|
+
# ``always_run`` resolves at stage start (carrier.execute_stage) against
|
|
349
|
+
# fixtures + parametrize parameters + global context — stage substitutions
|
|
350
|
+
# and foreach parameters don't exist yet, and neither do this stage's saves.
|
|
351
|
+
always_run_available = scenario_available | set(stage.fixtures) | _parameter_names(stage.parametrize) | cumulative_saves
|
|
352
|
+
for name in sorted(extract_template_variables(raw.get("always_run"))):
|
|
353
|
+
if name in always_run_available:
|
|
354
|
+
continue
|
|
355
|
+
if name in all_saved:
|
|
356
|
+
j = first_save_stage[name]
|
|
357
|
+
if j == i:
|
|
358
|
+
msg = f"Stage '{stage.name}': always_run references '{name}', which is only saved in this stage's response — always_run is evaluated before the stage runs"
|
|
359
|
+
else:
|
|
360
|
+
msg = f"Stage '{stage.name}': always_run references '{name}' before it is saved (saved in stage '{scenario.stages[j].name}')"
|
|
361
|
+
diagnostics.append(_diag(DiagnosticCode.FORWARD_REF, "warning", msg, location=stage.name))
|
|
362
|
+
else:
|
|
363
|
+
diagnostics.append(
|
|
364
|
+
_diag(
|
|
365
|
+
DiagnosticCode.UNDEFINED_VAR,
|
|
366
|
+
"warning",
|
|
367
|
+
f"Stage '{stage.name}': always_run references '{name}' — potentially not in scope; only fixtures, "
|
|
368
|
+
f"parametrize parameters, scenario substitutions, and variables saved by earlier stages are available",
|
|
369
|
+
location=stage.name,
|
|
370
|
+
)
|
|
371
|
+
)
|
|
372
|
+
|
|
350
373
|
undefined_here: set[str] = set()
|
|
351
374
|
|
|
352
375
|
for refs, available, in_request in (
|
|
@@ -642,14 +665,12 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
|
|
|
642
665
|
)
|
|
643
666
|
)
|
|
644
667
|
|
|
645
|
-
fixtures: list[str] =
|
|
646
|
-
if isinstance(test_data.get("fixtures"), list):
|
|
647
|
-
fixtures.extend(test_data["fixtures"])
|
|
668
|
+
fixtures: list[str] = list(scenario.fixtures)
|
|
648
669
|
for stage in scenario.stages:
|
|
649
670
|
fixtures.extend(stage.fixtures)
|
|
650
671
|
fixtures = sorted(set(fixtures))
|
|
651
672
|
|
|
652
|
-
vars_defined = extract_defined_variables(scenario
|
|
673
|
+
vars_defined = extract_defined_variables(scenario)
|
|
653
674
|
vars_saved = extract_saved_variables(scenario)
|
|
654
675
|
vars_referenced = extract_template_variables(test_data)
|
|
655
676
|
|
|
@@ -663,6 +684,35 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
|
|
|
663
684
|
)
|
|
664
685
|
)
|
|
665
686
|
|
|
687
|
+
# Scenario fixtures are injected into every stage and sit above the global
|
|
688
|
+
# context in the runtime ChainMap, so a save under the same name can never
|
|
689
|
+
# be read back — the fixture value always wins.
|
|
690
|
+
shadowed_saves = set(scenario.fixtures) & vars_saved
|
|
691
|
+
if shadowed_saves:
|
|
692
|
+
diagnostics.append(
|
|
693
|
+
_diag(
|
|
694
|
+
DiagnosticCode.FIXTURE_SHADOWS_SAVE,
|
|
695
|
+
"warning",
|
|
696
|
+
f"Saved variables shadowed by scenario-level fixtures: {sorted(shadowed_saves)} (fixture values win in every stage; these saves can never be read)",
|
|
697
|
+
)
|
|
698
|
+
)
|
|
699
|
+
|
|
700
|
+
# Scenario-level templates (substitutions/auth/ssl) resolve once at class
|
|
701
|
+
# creation (carrier.create_test_class), before any fixture exists — a fixture
|
|
702
|
+
# reference there is a guaranteed collection-time crash.
|
|
703
|
+
for key in ("substitutions", "auth", "ssl"):
|
|
704
|
+
scenario_level_refs = extract_template_variables(test_data.get(key))
|
|
705
|
+
fixture_refs = scenario_level_refs & set(fixtures)
|
|
706
|
+
if fixture_refs:
|
|
707
|
+
diagnostics.append(
|
|
708
|
+
_diag(
|
|
709
|
+
DiagnosticCode.FIXTURE_IN_SCENARIO_TEMPLATE,
|
|
710
|
+
"error",
|
|
711
|
+
f"Fixtures referenced in scenario-level '{key}' templates: {sorted(fixture_refs)} (scenario-level templates resolve at collection time, before fixtures exist)",
|
|
712
|
+
location=key,
|
|
713
|
+
)
|
|
714
|
+
)
|
|
715
|
+
|
|
666
716
|
# NOTE: response data (response/status_code/body/json/text/headers/cookies) is
|
|
667
717
|
# NOT ambient in {{ }} templates — it reaches save/verify handlers directly and
|
|
668
718
|
# only enters the template context via an earlier `save` step. So there are no
|
|
@@ -736,6 +786,17 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
|
|
|
736
786
|
return diagnostics, scenario_info
|
|
737
787
|
|
|
738
788
|
|
|
789
|
+
def resolve_root_path(path: Path) -> Path:
|
|
790
|
+
"""Directory that constrains ``$ref`` resolution: the nearest ``tests/``
|
|
791
|
+
ancestor of ``path``, else the file's own parent."""
|
|
792
|
+
potential_root = path.parent
|
|
793
|
+
while potential_root.parent != potential_root:
|
|
794
|
+
if potential_root.name == "tests":
|
|
795
|
+
return potential_root
|
|
796
|
+
potential_root = potential_root.parent
|
|
797
|
+
return path.parent
|
|
798
|
+
|
|
799
|
+
|
|
739
800
|
def validate_scenario(
|
|
740
801
|
path: Path,
|
|
741
802
|
ref_parent_traversal_depth: int = 3,
|
|
@@ -772,14 +833,7 @@ def validate_scenario(
|
|
|
772
833
|
)
|
|
773
834
|
|
|
774
835
|
if root_path is None:
|
|
775
|
-
|
|
776
|
-
while potential_root.parent != potential_root:
|
|
777
|
-
if potential_root.name == "tests":
|
|
778
|
-
root_path = potential_root
|
|
779
|
-
break
|
|
780
|
-
potential_root = potential_root.parent
|
|
781
|
-
else:
|
|
782
|
-
root_path = path.parent
|
|
836
|
+
root_path = resolve_root_path(path)
|
|
783
837
|
|
|
784
838
|
try:
|
|
785
839
|
test_data = pytest_httpchain_jsonref.loader.load_json(
|
|
@@ -1,83 +0,0 @@
|
|
|
1
|
-
import enum
|
|
2
|
-
import json
|
|
3
|
-
from pathlib import Path
|
|
4
|
-
from typing import Annotated
|
|
5
|
-
|
|
6
|
-
import typer
|
|
7
|
-
|
|
8
|
-
app = typer.Typer()
|
|
9
|
-
|
|
10
|
-
SKILL_FILE = Path(__file__).parent / "skill.md"
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
class OutputFormat(enum.StrEnum):
|
|
14
|
-
text = "text"
|
|
15
|
-
json = "json"
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
@app.command()
|
|
19
|
-
def validate(
|
|
20
|
-
paths: Annotated[list[Path], typer.Argument(help="Scenario JSON file(s) to validate.")],
|
|
21
|
-
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
22
|
-
output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
|
|
23
|
-
deep: Annotated[bool, typer.Option("--deep", help="Run deep checks: resolve user-function imports/signatures and referenced files. Imports user modules.")] = False,
|
|
24
|
-
syspath: Annotated[list[Path] | None, typer.Option("--syspath", help="Extra directories to add to sys.path for --deep import resolution (repeatable).")] = None,
|
|
25
|
-
strict: Annotated[bool, typer.Option("--strict", help="Treat warnings as failures for the exit code.")] = False,
|
|
26
|
-
) -> None:
|
|
27
|
-
"""Validate pytest-httpchain scenario file(s).
|
|
28
|
-
|
|
29
|
-
Reports errors and warnings (each with a stable HTTPCHAINxxx diagnostic code)
|
|
30
|
-
per file and exits non-zero if any file is invalid (or, with --strict, has any
|
|
31
|
-
warnings).
|
|
32
|
-
"""
|
|
33
|
-
from pytest_httpchain.validation import validate_scenario
|
|
34
|
-
|
|
35
|
-
results = [(path, validate_scenario(path, ref_parent_traversal_depth=ref_parent_traversal_depth, deep=deep, syspaths=list(syspath or []))) for path in paths]
|
|
36
|
-
|
|
37
|
-
def passed(result) -> bool:
|
|
38
|
-
return result.valid and not (strict and result.warnings)
|
|
39
|
-
|
|
40
|
-
all_passed = all(passed(result) for _, result in results)
|
|
41
|
-
|
|
42
|
-
if output_format is OutputFormat.json:
|
|
43
|
-
payload = {
|
|
44
|
-
"valid": all_passed,
|
|
45
|
-
"files": [{"path": str(path), "result": result.model_dump()} for path, result in results],
|
|
46
|
-
}
|
|
47
|
-
typer.echo(json.dumps(payload, indent=2, default=str))
|
|
48
|
-
else:
|
|
49
|
-
for path, result in results:
|
|
50
|
-
if not result.valid:
|
|
51
|
-
status = "INVALID"
|
|
52
|
-
elif result.warnings:
|
|
53
|
-
status = "FAILED (warnings)" if strict else "OK with warnings"
|
|
54
|
-
else:
|
|
55
|
-
status = "OK"
|
|
56
|
-
typer.echo(f"{path}: {status}")
|
|
57
|
-
for diagnostic in result.diagnostics:
|
|
58
|
-
typer.echo(f" {diagnostic.severity} [{diagnostic.code}]: {diagnostic.message}")
|
|
59
|
-
|
|
60
|
-
raise typer.Exit(0 if all_passed else 1)
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
@app.command()
|
|
64
|
-
def install(
|
|
65
|
-
global_: Annotated[bool, typer.Option("--global", "-g", help="Install to ~/.claude (personal scope) instead of project")] = False,
|
|
66
|
-
project_dir: Annotated[Path, typer.Option(help="Project directory (ignored with --global)")] = Path("."),
|
|
67
|
-
) -> None:
|
|
68
|
-
"""Install the Claude Code skill for authoring test scenarios."""
|
|
69
|
-
if global_:
|
|
70
|
-
_install_skill(Path.home() / ".claude" / "skills" / "pytest-httpchain")
|
|
71
|
-
else:
|
|
72
|
-
_install_skill(project_dir.resolve() / ".claude" / "skills" / "pytest-httpchain")
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
def _install_skill(skill_dir: Path) -> None:
|
|
76
|
-
skill_dir.mkdir(parents=True, exist_ok=True)
|
|
77
|
-
dest = skill_dir / "SKILL.md"
|
|
78
|
-
dest.write_text(SKILL_FILE.read_text())
|
|
79
|
-
typer.echo(f"Installed skill to {dest}")
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
if __name__ == "__main__":
|
|
83
|
-
app()
|
|
@@ -1,282 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: pytest-httpchain
|
|
3
|
-
description: Write and edit pytest-httpchain HTTP API test scenarios in JSON format
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# pytest-httpchain test authoring
|
|
7
|
-
|
|
8
|
-
pytest-httpchain is a pytest plugin for declarative HTTP API integration testing. Test scenarios are JSON files discovered by pattern `test_<name>.http.json`.
|
|
9
|
-
|
|
10
|
-
## Scenario structure
|
|
11
|
-
|
|
12
|
-
```json
|
|
13
|
-
{
|
|
14
|
-
"description": "optional scenario description",
|
|
15
|
-
"marks": ["optional_pytest_markers"],
|
|
16
|
-
"substitutions": [],
|
|
17
|
-
"stages": []
|
|
18
|
-
}
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
## Stage structure
|
|
22
|
-
|
|
23
|
-
```json
|
|
24
|
-
{
|
|
25
|
-
"name": "stage name",
|
|
26
|
-
"description": "optional",
|
|
27
|
-
"fixtures": ["fixture_name"],
|
|
28
|
-
"marks": ["skip", "xfail(reason='not ready')"],
|
|
29
|
-
"always_run": false,
|
|
30
|
-
"substitutions": [],
|
|
31
|
-
"parametrize": [],
|
|
32
|
-
"parallel": null,
|
|
33
|
-
"request": { ... },
|
|
34
|
-
"response": [ ... ]
|
|
35
|
-
}
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
Stages run sequentially and share a global context. Values saved in one stage are available in subsequent stages.
|
|
39
|
-
|
|
40
|
-
Stages can also be written as a dict (keys become stage names):
|
|
41
|
-
|
|
42
|
-
```json
|
|
43
|
-
{
|
|
44
|
-
"stages": {
|
|
45
|
-
"create user": { "request": { ... }, "response": [ ... ] },
|
|
46
|
-
"get user": { "request": { ... }, "response": [ ... ] }
|
|
47
|
-
}
|
|
48
|
-
}
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
## Request
|
|
52
|
-
|
|
53
|
-
```json
|
|
54
|
-
{
|
|
55
|
-
"url": "{{ server }}/api/users",
|
|
56
|
-
"method": "POST",
|
|
57
|
-
"headers": { "Authorization": "Bearer {{ token }}" },
|
|
58
|
-
"params": { "page": 1 },
|
|
59
|
-
"body": { "json": { "name": "Alice" } },
|
|
60
|
-
"timeout": 30.0,
|
|
61
|
-
"allow_redirects": true
|
|
62
|
-
}
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
**Body types** (use exactly one key):
|
|
66
|
-
- `{"json": { ... }}` - JSON body
|
|
67
|
-
- `{"form": { ... }}` - URL-encoded form
|
|
68
|
-
- `{"text": "..."}` - raw text
|
|
69
|
-
- `{"xml": "<root/>"}` - XML
|
|
70
|
-
- `{"base64": "..."}` - base64-encoded binary
|
|
71
|
-
- `{"binary": "/path/to/file"}` - file upload
|
|
72
|
-
- `{"files": {"field": "/path/to/file"}}` - multipart file upload
|
|
73
|
-
- `{"graphql": {"query": "...", "variables": {}}}` - GraphQL
|
|
74
|
-
|
|
75
|
-
## Response steps
|
|
76
|
-
|
|
77
|
-
Response is a list of verify and save steps, executed in order:
|
|
78
|
-
|
|
79
|
-
```json
|
|
80
|
-
"response": [
|
|
81
|
-
{
|
|
82
|
-
"verify": {
|
|
83
|
-
"status": 200,
|
|
84
|
-
"headers": { "content-type": "application/json" },
|
|
85
|
-
"body": {
|
|
86
|
-
"schema": { "type": "object", "required": ["id"] },
|
|
87
|
-
"contains": ["expected text"],
|
|
88
|
-
"not_contains": ["error"],
|
|
89
|
-
"matches": ["\\d{4}-\\d{2}-\\d{2}"],
|
|
90
|
-
"not_matches": ["forbidden"]
|
|
91
|
-
}
|
|
92
|
-
}
|
|
93
|
-
},
|
|
94
|
-
{
|
|
95
|
-
"save": {
|
|
96
|
-
"jmespath": {
|
|
97
|
-
"user_id": "data.id",
|
|
98
|
-
"user_name": "data.name",
|
|
99
|
-
"total": "length(items)"
|
|
100
|
-
}
|
|
101
|
-
}
|
|
102
|
-
},
|
|
103
|
-
{
|
|
104
|
-
"verify": {
|
|
105
|
-
"expressions": [
|
|
106
|
-
"{{ total > 0 }}",
|
|
107
|
-
"{{ user_name != '' }}"
|
|
108
|
-
]
|
|
109
|
-
}
|
|
110
|
-
}
|
|
111
|
-
]
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
**Important:** `verify.expressions` are `{{ }}` templates evaluated against the **context** (saved variables, fixtures, substitutions). The HTTP response is **not** ambient in templates — there is no `response`/`status_code`/`body`/`json` variable. To assert on response data, either:
|
|
115
|
-
- use `verify.status`, `verify.headers`, `verify.body` (these check the response directly), or
|
|
116
|
-
- `save` the value first (e.g. via `jmespath`) and reference the saved variable in a later `expressions` step (as shown above).
|
|
117
|
-
|
|
118
|
-
**Save types:**
|
|
119
|
-
- `{"jmespath": {...}}` - extract values from JSON response via JMESPath
|
|
120
|
-
- `{"substitutions": [...]}` - compute values using template expressions
|
|
121
|
-
- `{"user_functions": [...]}` - call Python functions to process response
|
|
122
|
-
|
|
123
|
-
## Template expressions
|
|
124
|
-
|
|
125
|
-
Use `{{ expr }}` syntax. Expressions are evaluated with Python semantics.
|
|
126
|
-
|
|
127
|
-
**Available context:** all saved variables, fixture values, and substitution results.
|
|
128
|
-
|
|
129
|
-
**Built-in functions:** `len`, `min`, `max`, `sum`, `abs`, `round`, `sorted`, `range`, `zip`, `enumerate`, `bool`, `int`, `float`, `str`, `dict`, `list`, `tuple`, `set`, `uuid4()`, `env(var, default)`, `get(var, default)`, `exists(var)`, `rand()`, `randint(a, b)`
|
|
130
|
-
|
|
131
|
-
**JSON literals:** `true`, `false`, `null` map to Python `True`, `False`, `None`.
|
|
132
|
-
|
|
133
|
-
## Substitutions
|
|
134
|
-
|
|
135
|
-
Define variables before stages run:
|
|
136
|
-
|
|
137
|
-
```json
|
|
138
|
-
"substitutions": [
|
|
139
|
-
{ "vars": { "base_url": "https://api.example.com", "count": "{{ 2 + 3 }}" } },
|
|
140
|
-
{ "functions": { "generate_token": "mymodule:create_jwt" } }
|
|
141
|
-
]
|
|
142
|
-
```
|
|
143
|
-
|
|
144
|
-
Substitutions can appear at scenario level (global) or stage level (local).
|
|
145
|
-
|
|
146
|
-
## References ($include / $ref)
|
|
147
|
-
|
|
148
|
-
Split scenarios across files using `$include` (preferred) or `$ref`:
|
|
149
|
-
|
|
150
|
-
```json
|
|
151
|
-
{
|
|
152
|
-
"request": {
|
|
153
|
-
"$include": "common.json#/requests/get_user"
|
|
154
|
-
}
|
|
155
|
-
}
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
Sibling properties are deep-merged with the referenced content:
|
|
159
|
-
|
|
160
|
-
```json
|
|
161
|
-
{
|
|
162
|
-
"$include": "base_request.json",
|
|
163
|
-
"headers": { "X-Custom": "override" }
|
|
164
|
-
}
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
## Parametrize
|
|
168
|
-
|
|
169
|
-
Run a stage with different inputs:
|
|
170
|
-
|
|
171
|
-
```json
|
|
172
|
-
"parametrize": [
|
|
173
|
-
{
|
|
174
|
-
"individual": { "user_id": [1, 2, 3] },
|
|
175
|
-
"ids": ["user-one", "user-two", "user-three"]
|
|
176
|
-
}
|
|
177
|
-
]
|
|
178
|
-
```
|
|
179
|
-
|
|
180
|
-
Or use combinations:
|
|
181
|
-
|
|
182
|
-
```json
|
|
183
|
-
"parametrize": [
|
|
184
|
-
{
|
|
185
|
-
"combinations": [
|
|
186
|
-
{ "method": "GET", "expected": 200 },
|
|
187
|
-
{ "method": "DELETE", "expected": 403 }
|
|
188
|
-
]
|
|
189
|
-
}
|
|
190
|
-
]
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
## Parallel execution
|
|
194
|
-
|
|
195
|
-
Execute requests concurrently for load testing:
|
|
196
|
-
|
|
197
|
-
```json
|
|
198
|
-
"parallel": {
|
|
199
|
-
"repeat": 100,
|
|
200
|
-
"max_concurrency": 10,
|
|
201
|
-
"calls_per_sec": 50
|
|
202
|
-
}
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
Or iterate over parameter sets in parallel:
|
|
206
|
-
|
|
207
|
-
```json
|
|
208
|
-
"parallel": {
|
|
209
|
-
"foreach": [{ "individual": { "id": [1, 2, 3, 4, 5] } }],
|
|
210
|
-
"max_concurrency": 5
|
|
211
|
-
}
|
|
212
|
-
```
|
|
213
|
-
|
|
214
|
-
## Complete example: multi-stage API test
|
|
215
|
-
|
|
216
|
-
```json
|
|
217
|
-
{
|
|
218
|
-
"substitutions": [
|
|
219
|
-
{ "vars": { "base": "{{ env('API_URL', 'http://localhost:8000') }}" } }
|
|
220
|
-
],
|
|
221
|
-
"stages": [
|
|
222
|
-
{
|
|
223
|
-
"name": "create user",
|
|
224
|
-
"request": {
|
|
225
|
-
"url": "{{ base }}/users",
|
|
226
|
-
"method": "POST",
|
|
227
|
-
"body": { "json": { "name": "Alice", "email": "alice@example.com" } }
|
|
228
|
-
},
|
|
229
|
-
"response": [
|
|
230
|
-
{ "verify": { "status": 201 } },
|
|
231
|
-
{ "save": { "jmespath": { "user_id": "id" } } }
|
|
232
|
-
]
|
|
233
|
-
},
|
|
234
|
-
{
|
|
235
|
-
"name": "get user",
|
|
236
|
-
"request": {
|
|
237
|
-
"url": "{{ base }}/users/{{ user_id }}"
|
|
238
|
-
},
|
|
239
|
-
"response": [
|
|
240
|
-
{ "verify": { "status": 200 } },
|
|
241
|
-
{ "save": { "jmespath": { "name": "name" } } },
|
|
242
|
-
{ "verify": { "expressions": ["{{ name == 'Alice' }}"] } }
|
|
243
|
-
]
|
|
244
|
-
},
|
|
245
|
-
{
|
|
246
|
-
"name": "delete user",
|
|
247
|
-
"request": {
|
|
248
|
-
"url": "{{ base }}/users/{{ user_id }}",
|
|
249
|
-
"method": "DELETE"
|
|
250
|
-
},
|
|
251
|
-
"response": [
|
|
252
|
-
{ "verify": { "status": 204 } }
|
|
253
|
-
]
|
|
254
|
-
}
|
|
255
|
-
]
|
|
256
|
-
}
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
## Validate your scenario
|
|
260
|
-
|
|
261
|
-
After writing a scenario, validate it (no server or network needed):
|
|
262
|
-
|
|
263
|
-
```bash
|
|
264
|
-
pytest-httpchain validate test_<name>.http.json
|
|
265
|
-
```
|
|
266
|
-
|
|
267
|
-
It checks structure plus semantics a JSON Schema cannot, each with a stable `HTTPCHAINxxx` code:
|
|
268
|
-
|
|
269
|
-
- `HTTPCHAIN003` — a `{{ var }}` that is never defined, saved, or provided as a fixture (likely a typo).
|
|
270
|
-
- `HTTPCHAIN004` — a variable used **before** the stage that saves it, or used in a stage's request when it is only saved in that same stage's response. Remember: a value `save`d in a stage's response is available to *later* response steps and *later* stages, never to the request that produced it.
|
|
271
|
-
- `HTTPCHAIN006` — a `verify` step that asserts nothing.
|
|
272
|
-
- `HTTPCHAIN007` / `HTTPCHAIN008` — body `contains`/`not_contains` (or `matches`/`not_matches`) that list the same value, which can never pass.
|
|
273
|
-
|
|
274
|
-
Add `--format json` for machine-readable output. The same checks run automatically during `pytest --collect-only`.
|
|
275
|
-
|
|
276
|
-
For a deeper check that imports your `module:func` references (confirming they resolve and their signatures match — including the injected `response` for save/verify functions) and verifies referenced files/schemas exist, add `--deep` (optionally `--syspath <dir>` for import roots, `--strict` to fail on warnings):
|
|
277
|
-
|
|
278
|
-
```bash
|
|
279
|
-
pytest-httpchain validate --deep test_<name>.http.json
|
|
280
|
-
```
|
|
281
|
-
|
|
282
|
-
Note: the HTTP response is **not** ambient in `{{ }}` templates — `save` what you need from a response first, then reference the saved variable.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|