pytest-httpchain 0.4.0__tar.gz → 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/PKG-INFO +23 -12
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/README.md +22 -11
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/pyproject.toml +1 -1
- pytest_httpchain-0.5.0/src/pytest_httpchain/cli.py +222 -0
- pytest_httpchain-0.5.0/src/pytest_httpchain/dataflow.py +127 -0
- pytest_httpchain-0.5.0/src/pytest_httpchain/schema.py +69 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/validation.py +26 -22
- pytest_httpchain-0.4.0/src/pytest_httpchain/cli.py +0 -83
- pytest_httpchain-0.4.0/src/pytest_httpchain/skill.md +0 -282
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/LICENSE +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/__init__.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/carrier.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/constants.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/exceptions.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/har_writer.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/plugin.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/report_formatter.py +0 -0
- {pytest_httpchain-0.4.0 → pytest_httpchain-0.5.0}/src/pytest_httpchain/utils.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pytest-httpchain
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.5.0
|
|
4
4
|
Summary: pytest plugin for HTTP testing using JSON files
|
|
5
5
|
Keywords: testing,pytest,requests
|
|
6
6
|
Author: Alexander Eresov
|
|
@@ -198,17 +198,7 @@ pip install 'git+https://github.com/aeresov/pytest-httpchain@main'
|
|
|
198
198
|
|
|
199
199
|
## AI agent support
|
|
200
200
|
|
|
201
|
-
`pytest-httpchain` ships
|
|
202
|
-
|
|
203
|
-
### Claude Code skill
|
|
204
|
-
|
|
205
|
-
Install the authoring skill into your project (or `--global` for personal scope):
|
|
206
|
-
|
|
207
|
-
```bash
|
|
208
|
-
uvx pytest-httpchain install
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
This writes `.claude/skills/pytest-httpchain/SKILL.md` with guidance for writing scenarios.
|
|
201
|
+
`pytest-httpchain` ships a scenario validator to help AI coding agents (and humans) author and check test scenarios.
|
|
212
202
|
|
|
213
203
|
### Scenario validation
|
|
214
204
|
|
|
@@ -242,6 +232,27 @@ A JSON Schema is published for as-you-type validation and autocomplete. Referenc
|
|
|
242
232
|
}
|
|
243
233
|
```
|
|
244
234
|
|
|
235
|
+
The hosted schema tracks the latest release; to pin the schema matching your installed version (e.g. for CI), emit it locally:
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
uvx pytest-httpchain schema --output scenario.schema.json
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
### Inspecting scenarios
|
|
242
|
+
|
|
243
|
+
More read-only commands help author and debug scenarios offline — no network, no test run:
|
|
244
|
+
|
|
245
|
+
```bash
|
|
246
|
+
# Print a scenario with all $ref/$include/$merge inlined and deep-merged
|
|
247
|
+
uvx pytest-httpchain resolve tests/test_login.http.json
|
|
248
|
+
|
|
249
|
+
# Summarize stages and the variable data-flow (which stage saves what, who consumes it)
|
|
250
|
+
uvx pytest-httpchain show tests/test_login.http.json
|
|
251
|
+
|
|
252
|
+
# Render the stage data-flow as a Mermaid flowchart
|
|
253
|
+
uvx pytest-httpchain graph tests/test_login.http.json
|
|
254
|
+
```
|
|
255
|
+
|
|
245
256
|
## Documentation
|
|
246
257
|
|
|
247
258
|
- [Full Documentation](https://aeresov.github.io/pytest-httpchain) - Complete usage guide
|
|
@@ -171,17 +171,7 @@ pip install 'git+https://github.com/aeresov/pytest-httpchain@main'
|
|
|
171
171
|
|
|
172
172
|
## AI agent support
|
|
173
173
|
|
|
174
|
-
`pytest-httpchain` ships
|
|
175
|
-
|
|
176
|
-
### Claude Code skill
|
|
177
|
-
|
|
178
|
-
Install the authoring skill into your project (or `--global` for personal scope):
|
|
179
|
-
|
|
180
|
-
```bash
|
|
181
|
-
uvx pytest-httpchain install
|
|
182
|
-
```
|
|
183
|
-
|
|
184
|
-
This writes `.claude/skills/pytest-httpchain/SKILL.md` with guidance for writing scenarios.
|
|
174
|
+
`pytest-httpchain` ships a scenario validator to help AI coding agents (and humans) author and check test scenarios.
|
|
185
175
|
|
|
186
176
|
### Scenario validation
|
|
187
177
|
|
|
@@ -215,6 +205,27 @@ A JSON Schema is published for as-you-type validation and autocomplete. Referenc
|
|
|
215
205
|
}
|
|
216
206
|
```
|
|
217
207
|
|
|
208
|
+
The hosted schema tracks the latest release; to pin the schema matching your installed version (e.g. for CI), emit it locally:
|
|
209
|
+
|
|
210
|
+
```bash
|
|
211
|
+
uvx pytest-httpchain schema --output scenario.schema.json
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
### Inspecting scenarios
|
|
215
|
+
|
|
216
|
+
More read-only commands help author and debug scenarios offline — no network, no test run:
|
|
217
|
+
|
|
218
|
+
```bash
|
|
219
|
+
# Print a scenario with all $ref/$include/$merge inlined and deep-merged
|
|
220
|
+
uvx pytest-httpchain resolve tests/test_login.http.json
|
|
221
|
+
|
|
222
|
+
# Summarize stages and the variable data-flow (which stage saves what, who consumes it)
|
|
223
|
+
uvx pytest-httpchain show tests/test_login.http.json
|
|
224
|
+
|
|
225
|
+
# Render the stage data-flow as a Mermaid flowchart
|
|
226
|
+
uvx pytest-httpchain graph tests/test_login.http.json
|
|
227
|
+
```
|
|
228
|
+
|
|
218
229
|
## Documentation
|
|
219
230
|
|
|
220
231
|
- [Full Documentation](https://aeresov.github.io/pytest-httpchain) - Complete usage guide
|
|
@@ -0,0 +1,222 @@
|
|
|
1
|
+
import enum
|
|
2
|
+
import json
|
|
3
|
+
from pathlib import Path
|
|
4
|
+
from typing import Annotated
|
|
5
|
+
|
|
6
|
+
import typer
|
|
7
|
+
|
|
8
|
+
app = typer.Typer()
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
class OutputFormat(enum.StrEnum):
|
|
12
|
+
text = "text"
|
|
13
|
+
json = "json"
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
class GraphDirection(enum.StrEnum):
|
|
17
|
+
TD = "TD"
|
|
18
|
+
LR = "LR"
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
@app.callback()
|
|
22
|
+
def main() -> None:
|
|
23
|
+
"""pytest-httpchain command-line tools."""
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
@app.command()
|
|
27
|
+
def validate(
|
|
28
|
+
paths: Annotated[list[Path], typer.Argument(help="Scenario JSON file(s) to validate.")],
|
|
29
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
30
|
+
output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
|
|
31
|
+
deep: Annotated[bool, typer.Option("--deep", help="Run deep checks: resolve user-function imports/signatures and referenced files. Imports user modules.")] = False,
|
|
32
|
+
syspath: Annotated[list[Path] | None, typer.Option("--syspath", help="Extra directories to add to sys.path for --deep import resolution (repeatable).")] = None,
|
|
33
|
+
strict: Annotated[bool, typer.Option("--strict", help="Treat warnings as failures for the exit code.")] = False,
|
|
34
|
+
) -> None:
|
|
35
|
+
"""Validate pytest-httpchain scenario file(s).
|
|
36
|
+
|
|
37
|
+
Reports errors and warnings (each with a stable HTTPCHAINxxx diagnostic code)
|
|
38
|
+
per file and exits non-zero if any file is invalid (or, with --strict, has any
|
|
39
|
+
warnings).
|
|
40
|
+
"""
|
|
41
|
+
from pytest_httpchain.validation import validate_scenario
|
|
42
|
+
|
|
43
|
+
results = [(path, validate_scenario(path, ref_parent_traversal_depth=ref_parent_traversal_depth, deep=deep, syspaths=list(syspath or []))) for path in paths]
|
|
44
|
+
|
|
45
|
+
def passed(result) -> bool:
|
|
46
|
+
return result.valid and not (strict and result.warnings)
|
|
47
|
+
|
|
48
|
+
all_passed = all(passed(result) for _, result in results)
|
|
49
|
+
|
|
50
|
+
if output_format is OutputFormat.json:
|
|
51
|
+
payload = {
|
|
52
|
+
"valid": all_passed,
|
|
53
|
+
"files": [{"path": str(path), "result": result.model_dump()} for path, result in results],
|
|
54
|
+
}
|
|
55
|
+
typer.echo(json.dumps(payload, indent=2, default=str))
|
|
56
|
+
else:
|
|
57
|
+
for path, result in results:
|
|
58
|
+
if not result.valid:
|
|
59
|
+
status = "INVALID"
|
|
60
|
+
elif result.warnings:
|
|
61
|
+
status = "FAILED (warnings)" if strict else "OK with warnings"
|
|
62
|
+
else:
|
|
63
|
+
status = "OK"
|
|
64
|
+
typer.echo(f"{path}: {status}")
|
|
65
|
+
for diagnostic in result.diagnostics:
|
|
66
|
+
typer.echo(f" {diagnostic.severity} [{diagnostic.code}]: {diagnostic.message}")
|
|
67
|
+
|
|
68
|
+
raise typer.Exit(0 if all_passed else 1)
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
@app.command()
|
|
72
|
+
def schema(
|
|
73
|
+
output: Annotated[Path | None, typer.Option("--output", "-o", help="Write schema to PATH instead of stdout.")] = None,
|
|
74
|
+
) -> None:
|
|
75
|
+
"""Emit the JSON Schema for scenario files (editor autocomplete/validation)."""
|
|
76
|
+
from pytest_httpchain.schema import build_schema
|
|
77
|
+
|
|
78
|
+
text = json.dumps(build_schema(), indent=2, default=str)
|
|
79
|
+
if output is not None:
|
|
80
|
+
output.write_text(text + "\n")
|
|
81
|
+
typer.echo(f"Wrote schema to {output}")
|
|
82
|
+
else:
|
|
83
|
+
typer.echo(text)
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
@app.command()
|
|
87
|
+
def resolve(
|
|
88
|
+
scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to resolve.")],
|
|
89
|
+
output: Annotated[Path | None, typer.Option("--output", "-o", help="Write resolved JSON to PATH instead of stdout.")] = None,
|
|
90
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
91
|
+
) -> None:
|
|
92
|
+
"""Resolve $ref/$include/$merge and print the merged scenario JSON."""
|
|
93
|
+
import pytest_httpchain_jsonref.loader
|
|
94
|
+
from pytest_httpchain_jsonref.exceptions import ReferenceResolverError
|
|
95
|
+
|
|
96
|
+
from pytest_httpchain.validation import resolve_root_path
|
|
97
|
+
|
|
98
|
+
try:
|
|
99
|
+
data = pytest_httpchain_jsonref.loader.load_json(
|
|
100
|
+
scenario,
|
|
101
|
+
max_parent_traversal_depth=ref_parent_traversal_depth,
|
|
102
|
+
root_path=resolve_root_path(scenario),
|
|
103
|
+
)
|
|
104
|
+
except (ReferenceResolverError, json.JSONDecodeError, OSError) as e:
|
|
105
|
+
typer.echo(f"error: {e}", err=True)
|
|
106
|
+
raise typer.Exit(1) from e
|
|
107
|
+
|
|
108
|
+
text = json.dumps(data, indent=2, default=str)
|
|
109
|
+
if output is not None:
|
|
110
|
+
output.write_text(text + "\n")
|
|
111
|
+
typer.echo(f"Wrote resolved scenario to {output}")
|
|
112
|
+
else:
|
|
113
|
+
typer.echo(text)
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
def _load_for_inspection(path: Path, depth: int):
|
|
117
|
+
"""Load + validate a scenario for show/graph, mapping failures to Exit(1)."""
|
|
118
|
+
from pydantic import ValidationError
|
|
119
|
+
from pytest_httpchain_jsonref.exceptions import ReferenceResolverError
|
|
120
|
+
|
|
121
|
+
from pytest_httpchain.dataflow import load_scenario
|
|
122
|
+
|
|
123
|
+
try:
|
|
124
|
+
return load_scenario(path, ref_parent_traversal_depth=depth)
|
|
125
|
+
except (ReferenceResolverError, json.JSONDecodeError, OSError) as e:
|
|
126
|
+
typer.echo(f"error: cannot load {path}: {e}", err=True)
|
|
127
|
+
raise typer.Exit(1) from e
|
|
128
|
+
except ValidationError:
|
|
129
|
+
typer.echo(f"error: {path} is not a valid scenario — run `pytest-httpchain validate {path}` for details", err=True)
|
|
130
|
+
raise typer.Exit(1) from None
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
def _render_show_text(path: Path, scenario, flow) -> list[str]:
|
|
134
|
+
producer_of: dict[tuple[int, str], int] = {}
|
|
135
|
+
for edge in flow.edges:
|
|
136
|
+
for name in edge.vars:
|
|
137
|
+
producer_of[(edge.consumer, name)] = edge.producer
|
|
138
|
+
|
|
139
|
+
all_fixtures = sorted({*flow.scenario_fixtures, *(f for s in flow.stages for f in s.fixtures)})
|
|
140
|
+
lines: list[str] = [scenario.description or path.name]
|
|
141
|
+
summary = f"{len(flow.stages)} stage(s)"
|
|
142
|
+
if all_fixtures:
|
|
143
|
+
summary += f" · fixtures: {all_fixtures}"
|
|
144
|
+
if flow.scenario_vars:
|
|
145
|
+
summary += f" · vars: {flow.scenario_vars}"
|
|
146
|
+
lines.append(summary)
|
|
147
|
+
lines.append("")
|
|
148
|
+
|
|
149
|
+
for s in flow.stages:
|
|
150
|
+
name = s.name or f"(stage {s.index + 1})"
|
|
151
|
+
lines.append(f"{s.index + 1} · {name} {s.method} {s.url}")
|
|
152
|
+
if s.saves:
|
|
153
|
+
lines.append(f" saves: {', '.join(s.saves)}")
|
|
154
|
+
if s.consumes:
|
|
155
|
+
parts: list[str] = []
|
|
156
|
+
for name in s.consumes:
|
|
157
|
+
producer = producer_of.get((s.index, name))
|
|
158
|
+
if producer is None:
|
|
159
|
+
parts.append(name)
|
|
160
|
+
else:
|
|
161
|
+
producer_name = flow.stages[producer].name or f"stage {producer + 1}"
|
|
162
|
+
parts.append(f"{name} (from #{producer + 1} {producer_name})")
|
|
163
|
+
lines.append(f" consumes: {', '.join(parts)}")
|
|
164
|
+
if s.marks:
|
|
165
|
+
lines.append(f" marks: {', '.join(s.marks)}")
|
|
166
|
+
return lines
|
|
167
|
+
|
|
168
|
+
|
|
169
|
+
@app.command()
|
|
170
|
+
def show(
|
|
171
|
+
scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to summarize.")],
|
|
172
|
+
output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
|
|
173
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
174
|
+
) -> None:
|
|
175
|
+
"""Summarize a scenario's stages and variable data-flow."""
|
|
176
|
+
from pytest_httpchain.dataflow import analyze_dataflow
|
|
177
|
+
|
|
178
|
+
sc, test_data = _load_for_inspection(scenario, ref_parent_traversal_depth)
|
|
179
|
+
flow = analyze_dataflow(sc, test_data)
|
|
180
|
+
|
|
181
|
+
if output_format is OutputFormat.json:
|
|
182
|
+
payload = flow.model_dump()
|
|
183
|
+
payload["description"] = sc.description or None
|
|
184
|
+
typer.echo(json.dumps(payload, indent=2, default=str))
|
|
185
|
+
else:
|
|
186
|
+
for line in _render_show_text(scenario, sc, flow):
|
|
187
|
+
typer.echo(line)
|
|
188
|
+
|
|
189
|
+
|
|
190
|
+
def _mermaid_label(text: str) -> str:
|
|
191
|
+
return text.replace('"', "'").replace("\n", " ")
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
def _to_mermaid(flow, direction: str = "TD") -> str:
|
|
195
|
+
lines = [f"flowchart {direction}"]
|
|
196
|
+
if not flow.stages:
|
|
197
|
+
lines.append(" %% (no stages)")
|
|
198
|
+
return "\n".join(lines)
|
|
199
|
+
for s in flow.stages:
|
|
200
|
+
label = f"{s.index + 1} · {s.name}" if s.name else f"{s.index + 1}"
|
|
201
|
+
lines.append(f' S{s.index}["{_mermaid_label(label)}"]')
|
|
202
|
+
for edge in flow.edges:
|
|
203
|
+
lines.append(f" S{edge.producer} -->|{', '.join(edge.vars)}| S{edge.consumer}")
|
|
204
|
+
return "\n".join(lines)
|
|
205
|
+
|
|
206
|
+
|
|
207
|
+
@app.command()
|
|
208
|
+
def graph(
|
|
209
|
+
scenario: Annotated[Path, typer.Argument(help="Scenario JSON file to graph.")],
|
|
210
|
+
direction: Annotated[GraphDirection, typer.Option("--direction", help="Flowchart orientation.")] = GraphDirection.TD,
|
|
211
|
+
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
212
|
+
) -> None:
|
|
213
|
+
"""Emit a Mermaid flowchart of the stage data-flow."""
|
|
214
|
+
from pytest_httpchain.dataflow import analyze_dataflow
|
|
215
|
+
|
|
216
|
+
sc, test_data = _load_for_inspection(scenario, ref_parent_traversal_depth)
|
|
217
|
+
flow = analyze_dataflow(sc, test_data)
|
|
218
|
+
typer.echo(_to_mermaid(flow, direction.value))
|
|
219
|
+
|
|
220
|
+
|
|
221
|
+
if __name__ == "__main__":
|
|
222
|
+
app()
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
"""Structured stage data-flow analysis for the ``show`` and ``graph`` CLI commands.
|
|
2
|
+
|
|
3
|
+
Reuses the same extraction helpers as the order-aware validator
|
|
4
|
+
(``validation._dataflow_diagnostics``) but produces a graph model instead of
|
|
5
|
+
diagnostics: which variables each stage saves, which it consumes from earlier
|
|
6
|
+
stages, and the producer -> consumer edges between stages.
|
|
7
|
+
"""
|
|
8
|
+
|
|
9
|
+
from pathlib import Path
|
|
10
|
+
from typing import Any
|
|
11
|
+
|
|
12
|
+
import pytest_httpchain_jsonref.loader
|
|
13
|
+
from pydantic import BaseModel
|
|
14
|
+
from pytest_httpchain_models import Scenario
|
|
15
|
+
|
|
16
|
+
from pytest_httpchain.validation import (
|
|
17
|
+
extract_template_variables,
|
|
18
|
+
raw_stages,
|
|
19
|
+
resolve_root_path,
|
|
20
|
+
saved_in_stage,
|
|
21
|
+
stage_defined_names,
|
|
22
|
+
substitution_names,
|
|
23
|
+
)
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
class DataFlowEdge(BaseModel):
|
|
27
|
+
"""A data dependency: ``vars`` saved by stage ``producer`` are referenced by stage ``consumer``."""
|
|
28
|
+
|
|
29
|
+
producer: int
|
|
30
|
+
consumer: int
|
|
31
|
+
vars: list[str]
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
class StageFlow(BaseModel):
|
|
35
|
+
"""Per-stage data-flow summary."""
|
|
36
|
+
|
|
37
|
+
index: int
|
|
38
|
+
name: str
|
|
39
|
+
method: str
|
|
40
|
+
url: str
|
|
41
|
+
fixtures: list[str]
|
|
42
|
+
marks: list[str]
|
|
43
|
+
saves: list[str]
|
|
44
|
+
consumes: list[str]
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
class DataFlow(BaseModel):
|
|
48
|
+
"""Whole-scenario data-flow graph."""
|
|
49
|
+
|
|
50
|
+
stages: list[StageFlow]
|
|
51
|
+
edges: list[DataFlowEdge]
|
|
52
|
+
scenario_fixtures: list[str] = []
|
|
53
|
+
scenario_vars: list[str] = []
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def analyze_dataflow(scenario: Scenario, test_data: dict[str, Any]) -> DataFlow:
|
|
57
|
+
"""Build the stage data-flow graph for a validated scenario.
|
|
58
|
+
|
|
59
|
+
A stage *consumes* a variable when its request/response/substitutions/parallel
|
|
60
|
+
templates reference a name saved by an earlier stage and not redefined locally
|
|
61
|
+
(own substitutions, parametrize/foreach parameters, or fixtures). ``parametrize``
|
|
62
|
+
values are excluded — they resolve against scenario scope, never saved values.
|
|
63
|
+
"""
|
|
64
|
+
raws = raw_stages(test_data)
|
|
65
|
+
saves_by_stage = [saved_in_stage(stage) for stage in scenario.stages]
|
|
66
|
+
|
|
67
|
+
first_save_stage: dict[str, int] = {}
|
|
68
|
+
for i, saved in enumerate(saves_by_stage):
|
|
69
|
+
for name in saved:
|
|
70
|
+
first_save_stage.setdefault(name, i)
|
|
71
|
+
|
|
72
|
+
stages: list[StageFlow] = []
|
|
73
|
+
edges: list[DataFlowEdge] = []
|
|
74
|
+
cumulative_saves: set[str] = set()
|
|
75
|
+
|
|
76
|
+
for i, stage in enumerate(scenario.stages):
|
|
77
|
+
raw = raws[i] if i < len(raws) and isinstance(raws[i], dict) else {}
|
|
78
|
+
|
|
79
|
+
refs: set[str] = set()
|
|
80
|
+
for key in ("request", "response", "substitutions", "parallel"):
|
|
81
|
+
extract_template_variables(raw.get(key), refs)
|
|
82
|
+
|
|
83
|
+
local = stage_defined_names(stage)
|
|
84
|
+
consumes = {name for name in refs if name in cumulative_saves and name not in local}
|
|
85
|
+
|
|
86
|
+
by_producer: dict[int, list[str]] = {}
|
|
87
|
+
for name in consumes:
|
|
88
|
+
by_producer.setdefault(first_save_stage[name], []).append(name)
|
|
89
|
+
for producer in sorted(by_producer):
|
|
90
|
+
edges.append(DataFlowEdge(producer=producer, consumer=i, vars=sorted(by_producer[producer])))
|
|
91
|
+
|
|
92
|
+
stages.append(
|
|
93
|
+
StageFlow(
|
|
94
|
+
index=i,
|
|
95
|
+
name=stage.name,
|
|
96
|
+
method=str(stage.request.method),
|
|
97
|
+
url=str(stage.request.url),
|
|
98
|
+
fixtures=sorted(stage.fixtures),
|
|
99
|
+
marks=list(stage.marks),
|
|
100
|
+
saves=sorted(saves_by_stage[i]),
|
|
101
|
+
consumes=sorted(consumes),
|
|
102
|
+
)
|
|
103
|
+
)
|
|
104
|
+
|
|
105
|
+
cumulative_saves |= saves_by_stage[i]
|
|
106
|
+
|
|
107
|
+
scenario_fixtures = sorted(f for f in test_data["fixtures"] if isinstance(f, str)) if isinstance(test_data.get("fixtures"), list) else []
|
|
108
|
+
scenario_var_names = set(substitution_names(scenario.substitutions))
|
|
109
|
+
if isinstance(test_data.get("vars"), dict):
|
|
110
|
+
scenario_var_names |= {k for k in test_data["vars"] if isinstance(k, str)}
|
|
111
|
+
|
|
112
|
+
return DataFlow(stages=stages, edges=edges, scenario_fixtures=scenario_fixtures, scenario_vars=sorted(scenario_var_names))
|
|
113
|
+
|
|
114
|
+
|
|
115
|
+
def load_scenario(path: Path, ref_parent_traversal_depth: int = 3) -> tuple[Scenario, dict[str, Any]]:
|
|
116
|
+
"""Load + ``$ref``-resolve + validate a scenario file.
|
|
117
|
+
|
|
118
|
+
Returns ``(scenario, raw_test_data)``. Raises ``ReferenceResolverError``,
|
|
119
|
+
``json.JSONDecodeError`` or ``pydantic.ValidationError`` on failure — callers
|
|
120
|
+
map these to user-facing errors.
|
|
121
|
+
"""
|
|
122
|
+
test_data = pytest_httpchain_jsonref.loader.load_json(
|
|
123
|
+
path,
|
|
124
|
+
max_parent_traversal_depth=ref_parent_traversal_depth,
|
|
125
|
+
root_path=resolve_root_path(path),
|
|
126
|
+
)
|
|
127
|
+
return Scenario.model_validate(test_data), test_data
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
"""Build the JSON Schema for pytest-httpchain scenario files.
|
|
2
|
+
|
|
3
|
+
Shared by the ``pytest-httpchain schema`` CLI command and
|
|
4
|
+
``scripts/generate_schema.py``. The schema is derived from the Pydantic
|
|
5
|
+
``Scenario`` model and augmented so editors accept pytest-httpchain's
|
|
6
|
+
``$ref``/``$include``/``$merge`` reference directives at any object level.
|
|
7
|
+
"""
|
|
8
|
+
|
|
9
|
+
from typing import Any
|
|
10
|
+
|
|
11
|
+
from pytest_httpchain_models import Scenario
|
|
12
|
+
|
|
13
|
+
SCHEMA_DIALECT = "https://json-schema.org/draft/2020-12/schema"
|
|
14
|
+
SCHEMA_ID = "https://aeresov.github.io/pytest-httpchain/schema/scenario.schema.json"
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def _add_jsonref_support(schema: dict[str, Any]) -> dict[str, Any]:
|
|
18
|
+
"""Allow ``$include``/``$merge``/``$ref`` objects as alternatives anywhere.
|
|
19
|
+
|
|
20
|
+
pytest-httpchain-jsonref can substitute any element at runtime, so each
|
|
21
|
+
definition and root property is wrapped in an ``anyOf`` that also accepts a
|
|
22
|
+
reference object — otherwise editors flag missing required properties when a
|
|
23
|
+
reference is used.
|
|
24
|
+
"""
|
|
25
|
+
if "$defs" not in schema:
|
|
26
|
+
schema["$defs"] = {}
|
|
27
|
+
|
|
28
|
+
schema["$defs"]["JsonRef"] = {
|
|
29
|
+
"type": "object",
|
|
30
|
+
"description": "Reference to external JSON file or JSON pointer. Use $include or $merge (preferred) or $ref. Resolved at runtime by pytest-httpchain-jsonref.",
|
|
31
|
+
"properties": {
|
|
32
|
+
"$include": {
|
|
33
|
+
"type": "string",
|
|
34
|
+
"description": "Path to external JSON file, JSON pointer (#/path), or combined (file.json#/path). Preferred over $ref to avoid VS Code conflicts.",
|
|
35
|
+
},
|
|
36
|
+
"$merge": {
|
|
37
|
+
"type": "string",
|
|
38
|
+
"description": "Alias for $include. Path to external JSON file, JSON pointer (#/path), or combined (file.json#/path).",
|
|
39
|
+
},
|
|
40
|
+
},
|
|
41
|
+
"additionalProperties": True,
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
for type_name, original_def in list(schema["$defs"].items()):
|
|
45
|
+
if type_name == "JsonRef":
|
|
46
|
+
continue
|
|
47
|
+
schema["$defs"][type_name] = {"anyOf": [{"$ref": "#/$defs/JsonRef"}, original_def]}
|
|
48
|
+
if "title" in original_def:
|
|
49
|
+
schema["$defs"][type_name]["title"] = original_def.pop("title")
|
|
50
|
+
if "description" in original_def:
|
|
51
|
+
schema["$defs"][type_name]["description"] = original_def.pop("description")
|
|
52
|
+
|
|
53
|
+
for prop_name, prop_def in list(schema.get("properties", {}).items()):
|
|
54
|
+
schema["properties"][prop_name] = {
|
|
55
|
+
"anyOf": [{"$ref": "#/$defs/JsonRef"}, prop_def],
|
|
56
|
+
"title": prop_def.get("title", prop_name),
|
|
57
|
+
}
|
|
58
|
+
if "description" in prop_def:
|
|
59
|
+
schema["properties"][prop_name]["description"] = prop_def.get("description")
|
|
60
|
+
|
|
61
|
+
return schema
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def build_schema() -> dict[str, Any]:
|
|
65
|
+
"""Return the augmented JSON Schema dict for the ``Scenario`` model."""
|
|
66
|
+
schema = Scenario.model_json_schema()
|
|
67
|
+
schema["$schema"] = SCHEMA_DIALECT
|
|
68
|
+
schema["$id"] = SCHEMA_ID
|
|
69
|
+
return _add_jsonref_support(schema)
|
|
@@ -152,7 +152,7 @@ def extract_template_variables(obj: Any, variables: set[str] | None = None) -> s
|
|
|
152
152
|
return variables
|
|
153
153
|
|
|
154
154
|
|
|
155
|
-
def
|
|
155
|
+
def substitution_names(substitutions: Any) -> set[str]:
|
|
156
156
|
"""Names introduced by a list of ``vars``/``functions`` substitution entries."""
|
|
157
157
|
names: set[str] = set()
|
|
158
158
|
for sub in substitutions or []:
|
|
@@ -165,7 +165,7 @@ def _substitution_names(substitutions: Any) -> set[str]:
|
|
|
165
165
|
return names
|
|
166
166
|
|
|
167
167
|
|
|
168
|
-
def
|
|
168
|
+
def saved_in_stage(stage: Stage) -> set[str]:
|
|
169
169
|
"""Variable names a single stage's response steps save into the context."""
|
|
170
170
|
saved: set[str] = set()
|
|
171
171
|
for response_step in stage.response:
|
|
@@ -177,7 +177,7 @@ def _saved_in_stage(stage: Stage) -> set[str]:
|
|
|
177
177
|
saved.update(jmespath.keys())
|
|
178
178
|
substitutions = getattr(save, "substitutions", None)
|
|
179
179
|
if substitutions is not None:
|
|
180
|
-
saved |=
|
|
180
|
+
saved |= substitution_names(substitutions)
|
|
181
181
|
# user_functions saves return arbitrary dict keys -> not statically known.
|
|
182
182
|
return saved
|
|
183
183
|
|
|
@@ -186,7 +186,7 @@ def extract_saved_variables(scenario: Scenario) -> set[str]:
|
|
|
186
186
|
"""Extract variable names saved across all response steps in the scenario."""
|
|
187
187
|
saved_vars: set[str] = set()
|
|
188
188
|
for stage in scenario.stages:
|
|
189
|
-
saved_vars |=
|
|
189
|
+
saved_vars |= saved_in_stage(stage)
|
|
190
190
|
return saved_vars
|
|
191
191
|
|
|
192
192
|
|
|
@@ -210,10 +210,10 @@ def _parameter_names(params: Any) -> set[str]:
|
|
|
210
210
|
return names
|
|
211
211
|
|
|
212
212
|
|
|
213
|
-
def
|
|
213
|
+
def stage_defined_names(stage: Stage) -> set[str]:
|
|
214
214
|
"""Names available *within a single stage*: its substitutions, parametrize /
|
|
215
215
|
foreach parameters, and its declared fixtures."""
|
|
216
|
-
names =
|
|
216
|
+
names = substitution_names(stage.substitutions)
|
|
217
217
|
names |= _parameter_names(stage.parametrize)
|
|
218
218
|
if stage.parallel is not None:
|
|
219
219
|
names |= _parameter_names(getattr(stage.parallel, "foreach", None))
|
|
@@ -236,10 +236,10 @@ def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) ->
|
|
|
236
236
|
if isinstance(test_data.get("vars"), dict):
|
|
237
237
|
defined_vars.update(k for k in test_data["vars"] if isinstance(k, str))
|
|
238
238
|
|
|
239
|
-
defined_vars |=
|
|
239
|
+
defined_vars |= substitution_names(scenario.substitutions)
|
|
240
240
|
|
|
241
241
|
for stage in scenario.stages:
|
|
242
|
-
defined_vars |=
|
|
242
|
+
defined_vars |= substitution_names(stage.substitutions)
|
|
243
243
|
defined_vars |= _parameter_names(stage.parametrize)
|
|
244
244
|
if stage.parallel is not None:
|
|
245
245
|
defined_vars |= _parameter_names(getattr(stage.parallel, "foreach", None))
|
|
@@ -247,7 +247,7 @@ def extract_defined_variables(scenario: Scenario, test_data: dict[str, Any]) ->
|
|
|
247
247
|
return defined_vars
|
|
248
248
|
|
|
249
249
|
|
|
250
|
-
def
|
|
250
|
+
def raw_stages(test_data: dict[str, Any]) -> list[Any]:
|
|
251
251
|
"""Raw (pre-validation) stage bodies in declaration order.
|
|
252
252
|
|
|
253
253
|
Stages may be authored as a list or as a ``{name: stage}`` mapping; both
|
|
@@ -298,7 +298,7 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
|
|
|
298
298
|
diagnostics: list[Diagnostic] = []
|
|
299
299
|
|
|
300
300
|
all_saved = extract_saved_variables(scenario)
|
|
301
|
-
saves_by_stage: list[set[str]] = [
|
|
301
|
+
saves_by_stage: list[set[str]] = [saved_in_stage(stage) for stage in scenario.stages]
|
|
302
302
|
first_save_stage: dict[str, int] = {}
|
|
303
303
|
for i, saved in enumerate(saves_by_stage):
|
|
304
304
|
for name in saved:
|
|
@@ -312,19 +312,19 @@ def _dataflow_diagnostics(scenario: Scenario, test_data: dict[str, Any]) -> list
|
|
|
312
312
|
scenario_scope: set[str] = set()
|
|
313
313
|
if isinstance(test_data.get("vars"), dict):
|
|
314
314
|
scenario_scope |= {k for k in test_data["vars"] if isinstance(k, str)}
|
|
315
|
-
scenario_scope |=
|
|
315
|
+
scenario_scope |= substitution_names(scenario.substitutions)
|
|
316
316
|
|
|
317
317
|
scenario_available = set(scenario_scope)
|
|
318
318
|
if isinstance(test_data.get("fixtures"), list):
|
|
319
319
|
scenario_available |= {f for f in test_data["fixtures"] if isinstance(f, str)}
|
|
320
320
|
|
|
321
|
-
|
|
321
|
+
raws = raw_stages(test_data)
|
|
322
322
|
cumulative_saves: set[str] = set()
|
|
323
323
|
|
|
324
324
|
for i, stage in enumerate(scenario.stages):
|
|
325
|
-
raw =
|
|
325
|
+
raw = raws[i] if i < len(raws) and isinstance(raws[i], dict) else {}
|
|
326
326
|
|
|
327
|
-
request_available = scenario_available |
|
|
327
|
+
request_available = scenario_available | stage_defined_names(stage) | cumulative_saves
|
|
328
328
|
response_available = request_available | saves_by_stage[i]
|
|
329
329
|
|
|
330
330
|
# ``parallel.foreach`` values are resolved at stage execution against the
|
|
@@ -736,6 +736,17 @@ def check_scenario(scenario: Scenario, test_data: dict[str, Any]) -> tuple[list[
|
|
|
736
736
|
return diagnostics, scenario_info
|
|
737
737
|
|
|
738
738
|
|
|
739
|
+
def resolve_root_path(path: Path) -> Path:
|
|
740
|
+
"""Directory that constrains ``$ref`` resolution: the nearest ``tests/``
|
|
741
|
+
ancestor of ``path``, else the file's own parent."""
|
|
742
|
+
potential_root = path.parent
|
|
743
|
+
while potential_root.parent != potential_root:
|
|
744
|
+
if potential_root.name == "tests":
|
|
745
|
+
return potential_root
|
|
746
|
+
potential_root = potential_root.parent
|
|
747
|
+
return path.parent
|
|
748
|
+
|
|
749
|
+
|
|
739
750
|
def validate_scenario(
|
|
740
751
|
path: Path,
|
|
741
752
|
ref_parent_traversal_depth: int = 3,
|
|
@@ -772,14 +783,7 @@ def validate_scenario(
|
|
|
772
783
|
)
|
|
773
784
|
|
|
774
785
|
if root_path is None:
|
|
775
|
-
|
|
776
|
-
while potential_root.parent != potential_root:
|
|
777
|
-
if potential_root.name == "tests":
|
|
778
|
-
root_path = potential_root
|
|
779
|
-
break
|
|
780
|
-
potential_root = potential_root.parent
|
|
781
|
-
else:
|
|
782
|
-
root_path = path.parent
|
|
786
|
+
root_path = resolve_root_path(path)
|
|
783
787
|
|
|
784
788
|
try:
|
|
785
789
|
test_data = pytest_httpchain_jsonref.loader.load_json(
|
|
@@ -1,83 +0,0 @@
|
|
|
1
|
-
import enum
|
|
2
|
-
import json
|
|
3
|
-
from pathlib import Path
|
|
4
|
-
from typing import Annotated
|
|
5
|
-
|
|
6
|
-
import typer
|
|
7
|
-
|
|
8
|
-
app = typer.Typer()
|
|
9
|
-
|
|
10
|
-
SKILL_FILE = Path(__file__).parent / "skill.md"
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
class OutputFormat(enum.StrEnum):
|
|
14
|
-
text = "text"
|
|
15
|
-
json = "json"
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
@app.command()
|
|
19
|
-
def validate(
|
|
20
|
-
paths: Annotated[list[Path], typer.Argument(help="Scenario JSON file(s) to validate.")],
|
|
21
|
-
ref_parent_traversal_depth: Annotated[int, typer.Option(help="Maximum $ref parent directory traversal depth.")] = 3,
|
|
22
|
-
output_format: Annotated[OutputFormat, typer.Option("--format", help="Output format: human-readable text or machine-readable JSON.")] = OutputFormat.text,
|
|
23
|
-
deep: Annotated[bool, typer.Option("--deep", help="Run deep checks: resolve user-function imports/signatures and referenced files. Imports user modules.")] = False,
|
|
24
|
-
syspath: Annotated[list[Path] | None, typer.Option("--syspath", help="Extra directories to add to sys.path for --deep import resolution (repeatable).")] = None,
|
|
25
|
-
strict: Annotated[bool, typer.Option("--strict", help="Treat warnings as failures for the exit code.")] = False,
|
|
26
|
-
) -> None:
|
|
27
|
-
"""Validate pytest-httpchain scenario file(s).
|
|
28
|
-
|
|
29
|
-
Reports errors and warnings (each with a stable HTTPCHAINxxx diagnostic code)
|
|
30
|
-
per file and exits non-zero if any file is invalid (or, with --strict, has any
|
|
31
|
-
warnings).
|
|
32
|
-
"""
|
|
33
|
-
from pytest_httpchain.validation import validate_scenario
|
|
34
|
-
|
|
35
|
-
results = [(path, validate_scenario(path, ref_parent_traversal_depth=ref_parent_traversal_depth, deep=deep, syspaths=list(syspath or []))) for path in paths]
|
|
36
|
-
|
|
37
|
-
def passed(result) -> bool:
|
|
38
|
-
return result.valid and not (strict and result.warnings)
|
|
39
|
-
|
|
40
|
-
all_passed = all(passed(result) for _, result in results)
|
|
41
|
-
|
|
42
|
-
if output_format is OutputFormat.json:
|
|
43
|
-
payload = {
|
|
44
|
-
"valid": all_passed,
|
|
45
|
-
"files": [{"path": str(path), "result": result.model_dump()} for path, result in results],
|
|
46
|
-
}
|
|
47
|
-
typer.echo(json.dumps(payload, indent=2, default=str))
|
|
48
|
-
else:
|
|
49
|
-
for path, result in results:
|
|
50
|
-
if not result.valid:
|
|
51
|
-
status = "INVALID"
|
|
52
|
-
elif result.warnings:
|
|
53
|
-
status = "FAILED (warnings)" if strict else "OK with warnings"
|
|
54
|
-
else:
|
|
55
|
-
status = "OK"
|
|
56
|
-
typer.echo(f"{path}: {status}")
|
|
57
|
-
for diagnostic in result.diagnostics:
|
|
58
|
-
typer.echo(f" {diagnostic.severity} [{diagnostic.code}]: {diagnostic.message}")
|
|
59
|
-
|
|
60
|
-
raise typer.Exit(0 if all_passed else 1)
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
@app.command()
|
|
64
|
-
def install(
|
|
65
|
-
global_: Annotated[bool, typer.Option("--global", "-g", help="Install to ~/.claude (personal scope) instead of project")] = False,
|
|
66
|
-
project_dir: Annotated[Path, typer.Option(help="Project directory (ignored with --global)")] = Path("."),
|
|
67
|
-
) -> None:
|
|
68
|
-
"""Install the Claude Code skill for authoring test scenarios."""
|
|
69
|
-
if global_:
|
|
70
|
-
_install_skill(Path.home() / ".claude" / "skills" / "pytest-httpchain")
|
|
71
|
-
else:
|
|
72
|
-
_install_skill(project_dir.resolve() / ".claude" / "skills" / "pytest-httpchain")
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
def _install_skill(skill_dir: Path) -> None:
|
|
76
|
-
skill_dir.mkdir(parents=True, exist_ok=True)
|
|
77
|
-
dest = skill_dir / "SKILL.md"
|
|
78
|
-
dest.write_text(SKILL_FILE.read_text())
|
|
79
|
-
typer.echo(f"Installed skill to {dest}")
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
if __name__ == "__main__":
|
|
83
|
-
app()
|
|
@@ -1,282 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: pytest-httpchain
|
|
3
|
-
description: Write and edit pytest-httpchain HTTP API test scenarios in JSON format
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# pytest-httpchain test authoring
|
|
7
|
-
|
|
8
|
-
pytest-httpchain is a pytest plugin for declarative HTTP API integration testing. Test scenarios are JSON files discovered by pattern `test_<name>.http.json`.
|
|
9
|
-
|
|
10
|
-
## Scenario structure
|
|
11
|
-
|
|
12
|
-
```json
|
|
13
|
-
{
|
|
14
|
-
"description": "optional scenario description",
|
|
15
|
-
"marks": ["optional_pytest_markers"],
|
|
16
|
-
"substitutions": [],
|
|
17
|
-
"stages": []
|
|
18
|
-
}
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
## Stage structure
|
|
22
|
-
|
|
23
|
-
```json
|
|
24
|
-
{
|
|
25
|
-
"name": "stage name",
|
|
26
|
-
"description": "optional",
|
|
27
|
-
"fixtures": ["fixture_name"],
|
|
28
|
-
"marks": ["skip", "xfail(reason='not ready')"],
|
|
29
|
-
"always_run": false,
|
|
30
|
-
"substitutions": [],
|
|
31
|
-
"parametrize": [],
|
|
32
|
-
"parallel": null,
|
|
33
|
-
"request": { ... },
|
|
34
|
-
"response": [ ... ]
|
|
35
|
-
}
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
Stages run sequentially and share a global context. Values saved in one stage are available in subsequent stages.
|
|
39
|
-
|
|
40
|
-
Stages can also be written as a dict (keys become stage names):
|
|
41
|
-
|
|
42
|
-
```json
|
|
43
|
-
{
|
|
44
|
-
"stages": {
|
|
45
|
-
"create user": { "request": { ... }, "response": [ ... ] },
|
|
46
|
-
"get user": { "request": { ... }, "response": [ ... ] }
|
|
47
|
-
}
|
|
48
|
-
}
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
## Request
|
|
52
|
-
|
|
53
|
-
```json
|
|
54
|
-
{
|
|
55
|
-
"url": "{{ server }}/api/users",
|
|
56
|
-
"method": "POST",
|
|
57
|
-
"headers": { "Authorization": "Bearer {{ token }}" },
|
|
58
|
-
"params": { "page": 1 },
|
|
59
|
-
"body": { "json": { "name": "Alice" } },
|
|
60
|
-
"timeout": 30.0,
|
|
61
|
-
"allow_redirects": true
|
|
62
|
-
}
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
**Body types** (use exactly one key):
|
|
66
|
-
- `{"json": { ... }}` - JSON body
|
|
67
|
-
- `{"form": { ... }}` - URL-encoded form
|
|
68
|
-
- `{"text": "..."}` - raw text
|
|
69
|
-
- `{"xml": "<root/>"}` - XML
|
|
70
|
-
- `{"base64": "..."}` - base64-encoded binary
|
|
71
|
-
- `{"binary": "/path/to/file"}` - file upload
|
|
72
|
-
- `{"files": {"field": "/path/to/file"}}` - multipart file upload
|
|
73
|
-
- `{"graphql": {"query": "...", "variables": {}}}` - GraphQL
|
|
74
|
-
|
|
75
|
-
## Response steps
|
|
76
|
-
|
|
77
|
-
Response is a list of verify and save steps, executed in order:
|
|
78
|
-
|
|
79
|
-
```json
|
|
80
|
-
"response": [
|
|
81
|
-
{
|
|
82
|
-
"verify": {
|
|
83
|
-
"status": 200,
|
|
84
|
-
"headers": { "content-type": "application/json" },
|
|
85
|
-
"body": {
|
|
86
|
-
"schema": { "type": "object", "required": ["id"] },
|
|
87
|
-
"contains": ["expected text"],
|
|
88
|
-
"not_contains": ["error"],
|
|
89
|
-
"matches": ["\\d{4}-\\d{2}-\\d{2}"],
|
|
90
|
-
"not_matches": ["forbidden"]
|
|
91
|
-
}
|
|
92
|
-
}
|
|
93
|
-
},
|
|
94
|
-
{
|
|
95
|
-
"save": {
|
|
96
|
-
"jmespath": {
|
|
97
|
-
"user_id": "data.id",
|
|
98
|
-
"user_name": "data.name",
|
|
99
|
-
"total": "length(items)"
|
|
100
|
-
}
|
|
101
|
-
}
|
|
102
|
-
},
|
|
103
|
-
{
|
|
104
|
-
"verify": {
|
|
105
|
-
"expressions": [
|
|
106
|
-
"{{ total > 0 }}",
|
|
107
|
-
"{{ user_name != '' }}"
|
|
108
|
-
]
|
|
109
|
-
}
|
|
110
|
-
}
|
|
111
|
-
]
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
**Important:** `verify.expressions` are `{{ }}` templates evaluated against the **context** (saved variables, fixtures, substitutions). The HTTP response is **not** ambient in templates — there is no `response`/`status_code`/`body`/`json` variable. To assert on response data, either:
|
|
115
|
-
- use `verify.status`, `verify.headers`, `verify.body` (these check the response directly), or
|
|
116
|
-
- `save` the value first (e.g. via `jmespath`) and reference the saved variable in a later `expressions` step (as shown above).
|
|
117
|
-
|
|
118
|
-
**Save types:**
|
|
119
|
-
- `{"jmespath": {...}}` - extract values from JSON response via JMESPath
|
|
120
|
-
- `{"substitutions": [...]}` - compute values using template expressions
|
|
121
|
-
- `{"user_functions": [...]}` - call Python functions to process response
|
|
122
|
-
|
|
123
|
-
## Template expressions
|
|
124
|
-
|
|
125
|
-
Use `{{ expr }}` syntax. Expressions are evaluated with Python semantics.
|
|
126
|
-
|
|
127
|
-
**Available context:** all saved variables, fixture values, and substitution results.
|
|
128
|
-
|
|
129
|
-
**Built-in functions:** `len`, `min`, `max`, `sum`, `abs`, `round`, `sorted`, `range`, `zip`, `enumerate`, `bool`, `int`, `float`, `str`, `dict`, `list`, `tuple`, `set`, `uuid4()`, `env(var, default)`, `get(var, default)`, `exists(var)`, `rand()`, `randint(a, b)`
|
|
130
|
-
|
|
131
|
-
**JSON literals:** `true`, `false`, `null` map to Python `True`, `False`, `None`.
|
|
132
|
-
|
|
133
|
-
## Substitutions
|
|
134
|
-
|
|
135
|
-
Define variables before stages run:
|
|
136
|
-
|
|
137
|
-
```json
|
|
138
|
-
"substitutions": [
|
|
139
|
-
{ "vars": { "base_url": "https://api.example.com", "count": "{{ 2 + 3 }}" } },
|
|
140
|
-
{ "functions": { "generate_token": "mymodule:create_jwt" } }
|
|
141
|
-
]
|
|
142
|
-
```
|
|
143
|
-
|
|
144
|
-
Substitutions can appear at scenario level (global) or stage level (local).
|
|
145
|
-
|
|
146
|
-
## References ($include / $ref)
|
|
147
|
-
|
|
148
|
-
Split scenarios across files using `$include` (preferred) or `$ref`:
|
|
149
|
-
|
|
150
|
-
```json
|
|
151
|
-
{
|
|
152
|
-
"request": {
|
|
153
|
-
"$include": "common.json#/requests/get_user"
|
|
154
|
-
}
|
|
155
|
-
}
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
Sibling properties are deep-merged with the referenced content:
|
|
159
|
-
|
|
160
|
-
```json
|
|
161
|
-
{
|
|
162
|
-
"$include": "base_request.json",
|
|
163
|
-
"headers": { "X-Custom": "override" }
|
|
164
|
-
}
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
## Parametrize
|
|
168
|
-
|
|
169
|
-
Run a stage with different inputs:
|
|
170
|
-
|
|
171
|
-
```json
|
|
172
|
-
"parametrize": [
|
|
173
|
-
{
|
|
174
|
-
"individual": { "user_id": [1, 2, 3] },
|
|
175
|
-
"ids": ["user-one", "user-two", "user-three"]
|
|
176
|
-
}
|
|
177
|
-
]
|
|
178
|
-
```
|
|
179
|
-
|
|
180
|
-
Or use combinations:
|
|
181
|
-
|
|
182
|
-
```json
|
|
183
|
-
"parametrize": [
|
|
184
|
-
{
|
|
185
|
-
"combinations": [
|
|
186
|
-
{ "method": "GET", "expected": 200 },
|
|
187
|
-
{ "method": "DELETE", "expected": 403 }
|
|
188
|
-
]
|
|
189
|
-
}
|
|
190
|
-
]
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
## Parallel execution
|
|
194
|
-
|
|
195
|
-
Execute requests concurrently for load testing:
|
|
196
|
-
|
|
197
|
-
```json
|
|
198
|
-
"parallel": {
|
|
199
|
-
"repeat": 100,
|
|
200
|
-
"max_concurrency": 10,
|
|
201
|
-
"calls_per_sec": 50
|
|
202
|
-
}
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
Or iterate over parameter sets in parallel:
|
|
206
|
-
|
|
207
|
-
```json
|
|
208
|
-
"parallel": {
|
|
209
|
-
"foreach": [{ "individual": { "id": [1, 2, 3, 4, 5] } }],
|
|
210
|
-
"max_concurrency": 5
|
|
211
|
-
}
|
|
212
|
-
```
|
|
213
|
-
|
|
214
|
-
## Complete example: multi-stage API test
|
|
215
|
-
|
|
216
|
-
```json
|
|
217
|
-
{
|
|
218
|
-
"substitutions": [
|
|
219
|
-
{ "vars": { "base": "{{ env('API_URL', 'http://localhost:8000') }}" } }
|
|
220
|
-
],
|
|
221
|
-
"stages": [
|
|
222
|
-
{
|
|
223
|
-
"name": "create user",
|
|
224
|
-
"request": {
|
|
225
|
-
"url": "{{ base }}/users",
|
|
226
|
-
"method": "POST",
|
|
227
|
-
"body": { "json": { "name": "Alice", "email": "alice@example.com" } }
|
|
228
|
-
},
|
|
229
|
-
"response": [
|
|
230
|
-
{ "verify": { "status": 201 } },
|
|
231
|
-
{ "save": { "jmespath": { "user_id": "id" } } }
|
|
232
|
-
]
|
|
233
|
-
},
|
|
234
|
-
{
|
|
235
|
-
"name": "get user",
|
|
236
|
-
"request": {
|
|
237
|
-
"url": "{{ base }}/users/{{ user_id }}"
|
|
238
|
-
},
|
|
239
|
-
"response": [
|
|
240
|
-
{ "verify": { "status": 200 } },
|
|
241
|
-
{ "save": { "jmespath": { "name": "name" } } },
|
|
242
|
-
{ "verify": { "expressions": ["{{ name == 'Alice' }}"] } }
|
|
243
|
-
]
|
|
244
|
-
},
|
|
245
|
-
{
|
|
246
|
-
"name": "delete user",
|
|
247
|
-
"request": {
|
|
248
|
-
"url": "{{ base }}/users/{{ user_id }}",
|
|
249
|
-
"method": "DELETE"
|
|
250
|
-
},
|
|
251
|
-
"response": [
|
|
252
|
-
{ "verify": { "status": 204 } }
|
|
253
|
-
]
|
|
254
|
-
}
|
|
255
|
-
]
|
|
256
|
-
}
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
## Validate your scenario
|
|
260
|
-
|
|
261
|
-
After writing a scenario, validate it (no server or network needed):
|
|
262
|
-
|
|
263
|
-
```bash
|
|
264
|
-
pytest-httpchain validate test_<name>.http.json
|
|
265
|
-
```
|
|
266
|
-
|
|
267
|
-
It checks structure plus semantics a JSON Schema cannot, each with a stable `HTTPCHAINxxx` code:
|
|
268
|
-
|
|
269
|
-
- `HTTPCHAIN003` — a `{{ var }}` that is never defined, saved, or provided as a fixture (likely a typo).
|
|
270
|
-
- `HTTPCHAIN004` — a variable used **before** the stage that saves it, or used in a stage's request when it is only saved in that same stage's response. Remember: a value `save`d in a stage's response is available to *later* response steps and *later* stages, never to the request that produced it.
|
|
271
|
-
- `HTTPCHAIN006` — a `verify` step that asserts nothing.
|
|
272
|
-
- `HTTPCHAIN007` / `HTTPCHAIN008` — body `contains`/`not_contains` (or `matches`/`not_matches`) that list the same value, which can never pass.
|
|
273
|
-
|
|
274
|
-
Add `--format json` for machine-readable output. The same checks run automatically during `pytest --collect-only`.
|
|
275
|
-
|
|
276
|
-
For a deeper check that imports your `module:func` references (confirming they resolve and their signatures match — including the injected `response` for save/verify functions) and verifies referenced files/schemas exist, add `--deep` (optionally `--syspath <dir>` for import roots, `--strict` to fail on warnings):
|
|
277
|
-
|
|
278
|
-
```bash
|
|
279
|
-
pytest-httpchain validate --deep test_<name>.http.json
|
|
280
|
-
```
|
|
281
|
-
|
|
282
|
-
Note: the HTTP response is **not** ambient in `{{ }}` templates — `save` what you need from a response first, then reference the saved variable.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|