strobengine 0.4.1__tar.gz → 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {strobengine-0.4.1 → strobengine-0.5.0}/.gitignore +3 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/CHANGELOG.md +64 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/Cargo.lock +19 -1
- {strobengine-0.4.1 → strobengine-0.5.0}/Cargo.toml +3 -2
- strobengine-0.5.0/PKG-INFO +86 -0
- strobengine-0.5.0/README.md +68 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/cli.md +6 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/quickstart.md +16 -0
- strobengine-0.5.0/docs/reports.md +149 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/roadmap.md +43 -11
- strobengine-0.5.0/docs/sse.md +197 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/table_of_contents.md +2 -0
- strobengine-0.5.0/examples/sse.py +113 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/pyproject.toml +4 -3
- {strobengine-0.4.1 → strobengine-0.5.0}/scripts/release.py +4 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/config.rs +32 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/lib.rs +57 -38
- {strobengine-0.4.1 → strobengine-0.5.0}/src/metrics.rs +6 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/grpc.rs +3 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/http.rs +3 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/http3.rs +6 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/mod.rs +16 -0
- strobengine-0.5.0/src/protocols/sse.rs +577 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/websocket.rs +15 -0
- strobengine-0.5.0/src/report/mod.rs +2 -0
- strobengine-0.5.0/src/report/schema.rs +392 -0
- strobengine-0.5.0/src/report/writer.rs +245 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/_strobengine.pyi +8 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/cli.py +237 -4
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/engine.py +16 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/reporter.py +136 -0
- strobengine-0.5.0/src/strobengine/reporting/assets/chart.min.js +7 -0
- strobengine-0.5.0/src/strobengine/reporting/baseline.py +144 -0
- strobengine-0.5.0/src/strobengine/reporting/csv_report.py +56 -0
- strobengine-0.5.0/src/strobengine/reporting/html_report.py +238 -0
- strobengine-0.5.0/src/strobengine/reporting/junit_report.py +130 -0
- strobengine-0.5.0/src/strobengine/reporting/markdown_report.py +144 -0
- strobengine-0.5.0/tests/e2e/__init__.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/mock_server.py +23 -0
- strobengine-0.5.0/tests/e2e/test_persistence_e2e.py +83 -0
- strobengine-0.5.0/tests/e2e/test_sse_e2e.py +81 -0
- strobengine-0.5.0/tests/test_reporting.py +252 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/uv.lock +89 -1
- strobengine-0.4.1/PKG-INFO +0 -83
- strobengine-0.4.1/README.md +0 -66
- {strobengine-0.4.1 → strobengine-0.5.0}/.dockerignore +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/.github/workflows/ci.yml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/.github/workflows/docker-release.yml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/.github/workflows/publish-test.yml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/.github/workflows/publish.yml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/.pre-commit-config.yaml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/.python-version +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/Dockerfile +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/LICENSE +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/Makefile +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/cliff.toml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/benchmarks.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/dependencies.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/docker.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/grpc.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/http3.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/http_methods.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/infrastructure.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/install.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/release.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/testing.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/docs/websockets.md +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/grpc.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/http3.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/http_methods.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/load_test.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/spike_test.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/stress_test.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/examples/websocket.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/rust-toolchain.toml +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/chaos.rs +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/logging.rs +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/progress.rs +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/grpc_parser.rs +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/protocols/grpc_reflection.rs +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/__init__.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/constants.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/src/strobengine/py.typed +0 -0
- {strobengine-0.4.1/tests → strobengine-0.5.0/src/strobengine/reporting}/__init__.py +0 -0
- {strobengine-0.4.1/tests/e2e → strobengine-0.5.0/tests}/__init__.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/conftest.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/conftest.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/test_cli_interface.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/test_grpc_e2e.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/test_http3_e2e.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/test_http_scenarios.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/test_performance_sanity.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/e2e/test_websocket_e2e.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/fixtures/service.proto +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/test_cli.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/test_engine.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/test_logging.py +0 -0
- {strobengine-0.4.1 → strobengine-0.5.0}/tests/test_reporter.py +0 -0
|
@@ -4,6 +4,66 @@ All notable changes to `strobengine` will be documented in this file.
|
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
5
5
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.5.0] - 2026-08-30
|
|
8
|
+
|
|
9
|
+
### Documentation
|
|
10
|
+
|
|
11
|
+
- Convert README links to absolute GitHub URLs
|
|
12
|
+
- *(cli)* Add --sse/--no-sse and --sse-max-events to option tables
|
|
13
|
+
- *(examples)* Add SSE streaming load test examples
|
|
14
|
+
- Add SSE load testing reference guide
|
|
15
|
+
- Add SSE to README, quickstart, and table of contents
|
|
16
|
+
- Mark HTML report generator as complete in roadmap
|
|
17
|
+
- Mark baseline comparison as complete in roadmap
|
|
18
|
+
- Add reporting and visualizations reference guide
|
|
19
|
+
- Add Reporting & Visualizations to documentation indexes
|
|
20
|
+
- Add Advanced Metrics & Observability epic to roadmap
|
|
21
|
+
- Mark Export Formats as complete in roadmap
|
|
22
|
+
|
|
23
|
+
### Features
|
|
24
|
+
|
|
25
|
+
- *(config)* Add sse_enabled, sse_max_events to TestConfig
|
|
26
|
+
- *(metrics)* Add sse_events_received, sse_first_event_us, sse_event_interval_us to RequestMetric
|
|
27
|
+
- *(sse)* Implement SSE frame parser and SseEngine
|
|
28
|
+
- *(sse)* Wire SSE engine into protocol detection and config routing
|
|
29
|
+
- *(sse)* Centralize protocol routing and add URL normalization
|
|
30
|
+
- *(deps)* Add serde with derive feature for report serialization
|
|
31
|
+
- *(report)* Add ReportArtifact schema with serde serialization
|
|
32
|
+
- *(config)* Add output_dir and no_save to TestConfig
|
|
33
|
+
- *(python)* Wire report persistence through CLI and engine
|
|
34
|
+
- *(report)* Add ReportArtifact::from_summary_and_config converter
|
|
35
|
+
- *(report)* Add save_report_json with atomic writes and latest.json
|
|
36
|
+
- *(report)* Invoke save_report_json on benchmark completion
|
|
37
|
+
- *(reporting)* Add standalone HTML report generator with Chart.js
|
|
38
|
+
- *(cli)* Add --html flag for standalone HTML report generation
|
|
39
|
+
- *(reporting)* Add baseline artifact loading and delta comparison
|
|
40
|
+
- *(reporting)* Add Historical Comparison section to HTML report
|
|
41
|
+
- *(cli)* Add --compare-to flag for baseline comparison
|
|
42
|
+
- *(reporting)* Inline Chart.js from local asset for 100% offline reports
|
|
43
|
+
- *(reporting)* Add CLI terminal comparison and extend compute_comparison
|
|
44
|
+
- *(engine)* Add get_config() convenience method to StrobEngine
|
|
45
|
+
- *(reporting)* Add Markdown, JUnit XML, and CSV export generators
|
|
46
|
+
- *(cli)* Add --markdown, --junit, --csv export flags
|
|
47
|
+
- *(reporting)* Add generate_markdown_summary for GitHub Actions / PR comments
|
|
48
|
+
- *(reporting)* Add generate_junit_report and generate_csv_report
|
|
49
|
+
|
|
50
|
+
### Miscellaneous Tasks
|
|
51
|
+
|
|
52
|
+
- *(pyproject)* Simplify readme config and update documentation URL
|
|
53
|
+
- *(scripts)* Run uv lock during release version bump
|
|
54
|
+
|
|
55
|
+
### Styling
|
|
56
|
+
|
|
57
|
+
- *(tests)* Reorder imports in test_reporting.py for ruff isort compliance
|
|
58
|
+
|
|
59
|
+
### Testing
|
|
60
|
+
|
|
61
|
+
- *(config)* Update TestConfig::new calls with SSE parameters
|
|
62
|
+
- *(e2e)* Add SSE streaming endpoint to mock server
|
|
63
|
+
- *(e2e)* Add SSE integration tests for error paths and streaming
|
|
64
|
+
- *(e2e)* Add persistence E2E tests for report artifact disk writing
|
|
65
|
+
- *(reporting)* Add unit tests for Markdown summary generator
|
|
66
|
+
- *(reporting)* Add JUnit XML and CSV export unit tests
|
|
7
67
|
## [0.4.1] - 2026-08-26
|
|
8
68
|
|
|
9
69
|
### Bug Fixes
|
|
@@ -22,6 +82,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
22
82
|
- Slim README and link to documentation pages
|
|
23
83
|
- Add multi-target, reporting, SSE, messaging, and caching epics to roadmap
|
|
24
84
|
|
|
85
|
+
### Miscellaneous Tasks
|
|
86
|
+
|
|
87
|
+
- *(release)* Bump version to 0.4.1
|
|
88
|
+
|
|
25
89
|
### Testing
|
|
26
90
|
|
|
27
91
|
- *(e2e)* Remove bytes_received assertion from PingPong mode test
|
|
@@ -1461,6 +1461,7 @@ dependencies = [
|
|
|
1461
1461
|
"base64",
|
|
1462
1462
|
"bytes",
|
|
1463
1463
|
"futures-core",
|
|
1464
|
+
"futures-util",
|
|
1464
1465
|
"h2",
|
|
1465
1466
|
"http",
|
|
1466
1467
|
"http-body",
|
|
@@ -1479,12 +1480,14 @@ dependencies = [
|
|
|
1479
1480
|
"sync_wrapper",
|
|
1480
1481
|
"tokio",
|
|
1481
1482
|
"tokio-rustls",
|
|
1483
|
+
"tokio-util",
|
|
1482
1484
|
"tower",
|
|
1483
1485
|
"tower-http",
|
|
1484
1486
|
"tower-service",
|
|
1485
1487
|
"url",
|
|
1486
1488
|
"wasm-bindgen",
|
|
1487
1489
|
"wasm-bindgen-futures",
|
|
1490
|
+
"wasm-streams",
|
|
1488
1491
|
"web-sys",
|
|
1489
1492
|
]
|
|
1490
1493
|
|
|
@@ -1654,6 +1657,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
|
1654
1657
|
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
|
|
1655
1658
|
dependencies = [
|
|
1656
1659
|
"serde_core",
|
|
1660
|
+
"serde_derive",
|
|
1657
1661
|
]
|
|
1658
1662
|
|
|
1659
1663
|
[[package]]
|
|
@@ -1781,7 +1785,7 @@ checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
|
|
|
1781
1785
|
|
|
1782
1786
|
[[package]]
|
|
1783
1787
|
name = "strobengine"
|
|
1784
|
-
version = "0.
|
|
1788
|
+
version = "0.5.0"
|
|
1785
1789
|
dependencies = [
|
|
1786
1790
|
"async-trait",
|
|
1787
1791
|
"base64",
|
|
@@ -1801,6 +1805,7 @@ dependencies = [
|
|
|
1801
1805
|
"pyo3",
|
|
1802
1806
|
"quinn",
|
|
1803
1807
|
"reqwest",
|
|
1808
|
+
"serde",
|
|
1804
1809
|
"serde_json",
|
|
1805
1810
|
"tokio",
|
|
1806
1811
|
"tokio-stream",
|
|
@@ -2355,6 +2360,19 @@ dependencies = [
|
|
|
2355
2360
|
"unicode-ident",
|
|
2356
2361
|
]
|
|
2357
2362
|
|
|
2363
|
+
[[package]]
|
|
2364
|
+
name = "wasm-streams"
|
|
2365
|
+
version = "0.5.0"
|
|
2366
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
2367
|
+
checksum = "9d1ec4f6517c9e11ae630e200b2b65d193279042e28edd4a2cda233e46670bbb"
|
|
2368
|
+
dependencies = [
|
|
2369
|
+
"futures-util",
|
|
2370
|
+
"js-sys",
|
|
2371
|
+
"wasm-bindgen",
|
|
2372
|
+
"wasm-bindgen-futures",
|
|
2373
|
+
"web-sys",
|
|
2374
|
+
]
|
|
2375
|
+
|
|
2358
2376
|
[[package]]
|
|
2359
2377
|
name = "web-sys"
|
|
2360
2378
|
version = "0.3.103"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[package]
|
|
2
2
|
name = "strobengine"
|
|
3
|
-
version = "0.
|
|
3
|
+
version = "0.5.0"
|
|
4
4
|
edition = "2024"
|
|
5
5
|
readme = "README.md"
|
|
6
6
|
|
|
@@ -15,7 +15,7 @@ crate-type = ["cdylib", "rlib"]
|
|
|
15
15
|
# "abi3-py311" tells pyo3 (and maturin) to build using the stable ABI with
|
|
16
16
|
# minimum Python version 3.11
|
|
17
17
|
pyo3 = { version = "0.29", features = ["extension-module", "abi3-py311"] }
|
|
18
|
-
reqwest = { version = "0.13.4", default-features = false, features = ["rustls", "http2"] }
|
|
18
|
+
reqwest = { version = "0.13.4", default-features = false, features = ["rustls", "http2", "stream"] }
|
|
19
19
|
http = "1"
|
|
20
20
|
bytes = "1"
|
|
21
21
|
tokio = { version = "1.53.0", features = ["sync", "time", "rt", "rt-multi-thread", "macros", "signal"] }
|
|
@@ -37,6 +37,7 @@ base64 = "0.22"
|
|
|
37
37
|
hex = "0.4"
|
|
38
38
|
prost-reflect = { version = "0.16.5", features = ["serde"] }
|
|
39
39
|
protox = "0.9.1"
|
|
40
|
+
serde = { version = "1", features = ["derive"] }
|
|
40
41
|
serde_json = "1.0.151"
|
|
41
42
|
tonic-reflection = { version = "0.14.6", default-features = false }
|
|
42
43
|
quinn = { version = "0.11", default-features = false, features = ["runtime-tokio", "rustls-ring"] }
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: strobengine
|
|
3
|
+
Version: 0.5.0
|
|
4
|
+
Requires-Dist: rich>=13.0.0
|
|
5
|
+
Requires-Dist: typer>=0.27.0
|
|
6
|
+
Requires-Dist: jinja2>=3.1
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
Summary: A load testing framework with a Python UI and a bare-metal Rust engine core.
|
|
9
|
+
Author-email: riccione <28346232+riccione@users.noreply.github.com>
|
|
10
|
+
License-Expression: Apache-2.0
|
|
11
|
+
Requires-Python: >=3.11
|
|
12
|
+
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
13
|
+
Project-URL: Changelog, https://github.com/strobeops/strobengine/blob/main/CHANGELOG.md
|
|
14
|
+
Project-URL: Documentation, https://github.com/strobeops/strobengine/blob/main/docs/table_of_contents.md
|
|
15
|
+
Project-URL: Homepage, https://github.com/strobeops/strobengine
|
|
16
|
+
Project-URL: Source, https://github.com/strobeops/strobengine
|
|
17
|
+
|
|
18
|
+
# strobengine
|
|
19
|
+
|
|
20
|
+
A high-performance HTTP, WebSocket, gRPC, and SSE load testing engine with a Python API and a bare-metal Rust core.
|
|
21
|
+
|
|
22
|
+
## Install
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
pip install strobengine
|
|
26
|
+
# or from source
|
|
27
|
+
git clone https://github.com/strobeops/strobengine.git && cd strobengine && uv sync
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
> See [Installation Guide](https://github.com/strobeops/strobengine/blob/main/docs/install.md) for Docker, PyPI, and development setup.
|
|
31
|
+
|
|
32
|
+
## Quick Start
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
from strobengine import StrobEngine
|
|
36
|
+
from strobengine.reporter import print_summary
|
|
37
|
+
|
|
38
|
+
engine = StrobEngine(url="http://localhost:8080/api/health", concurrency=50, duration=30)
|
|
39
|
+
summary = engine.run()
|
|
40
|
+
print_summary(summary)
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
strobengine load http://localhost:8080/api/health -c 50 -d 30
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
> See [Quick Start Guide](https://github.com/strobeops/strobengine/blob/main/docs/quickstart.md) for Python API and CLI examples.
|
|
48
|
+
|
|
49
|
+
## Documentation
|
|
50
|
+
|
|
51
|
+
| Section | Description |
|
|
52
|
+
|---------|-------------|
|
|
53
|
+
| [Installation](https://github.com/strobeops/strobengine/blob/main/docs/install.md) | Build from source, Docker, PyPI |
|
|
54
|
+
| [Dependencies](https://github.com/strobeops/strobengine/blob/main/docs/dependencies.md) | System requirements, Rust crates, Python packages |
|
|
55
|
+
| [Quick Start](https://github.com/strobeops/strobengine/blob/main/docs/quickstart.md) | Python API and CLI usage examples |
|
|
56
|
+
| [CLI Reference](https://github.com/strobeops/strobengine/blob/main/docs/cli.md) | All subcommands, flags, and options |
|
|
57
|
+
| [HTTP Methods](https://github.com/strobeops/strobengine/blob/main/docs/http_methods.md) | Supported methods, request bodies, headers |
|
|
58
|
+
| [WebSocket](https://github.com/strobeops/strobengine/blob/main/docs/websockets.md) | WS/WSS modes, pub/sub broadcasting, metrics |
|
|
59
|
+
| [gRPC](https://github.com/strobeops/strobengine/blob/main/docs/grpc.md) | Unary calls, protobuf, deadline, chaos |
|
|
60
|
+
| [HTTP/3](https://github.com/strobeops/strobengine/blob/main/docs/http3.md) | QUIC transport, 0-RTT resumption, loss recovery |
|
|
61
|
+
| [SSE](https://github.com/strobeops/strobengine/blob/main/docs/sse.md) | Server-Sent Events streaming, TTFB, event intervals |
|
|
62
|
+
| [Reporting](https://github.com/strobeops/strobengine/blob/main/docs/reports.md) | JSON persistence, HTML reports, baseline comparison |
|
|
63
|
+
| [Docker](https://github.com/strobeops/strobengine/blob/main/docs/docker.md) | Pulling images, running containers |
|
|
64
|
+
| [Testing](https://github.com/strobeops/strobengine/blob/main/docs/testing.md) | Test structure, e2e, CI/CD |
|
|
65
|
+
| [Roadmap](https://github.com/strobeops/strobengine/blob/main/docs/roadmap.md) | Planned features |
|
|
66
|
+
|
|
67
|
+
## How to Contribute
|
|
68
|
+
|
|
69
|
+
1. Fork the repository
|
|
70
|
+
2. Create a feature branch (`git checkout -b feat/my-feature`)
|
|
71
|
+
3. Install dependencies and set up pre-push Git hooks:
|
|
72
|
+
```bash
|
|
73
|
+
uv sync
|
|
74
|
+
uv run pre-commit install --hook-type pre-push
|
|
75
|
+
```
|
|
76
|
+
4. Run formatting, linting, and tests manually (or let git push run them automatically):
|
|
77
|
+
```bash
|
|
78
|
+
make check
|
|
79
|
+
```
|
|
80
|
+
5. Commit your changes following [Conventional Commits](https://www.conventionalcommits.org/)
|
|
81
|
+
6. Push to your branch and open a Pull Request
|
|
82
|
+
|
|
83
|
+
## License
|
|
84
|
+
|
|
85
|
+
This project is licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/strobeops/strobengine/blob/main/LICENSE) for the full text.
|
|
86
|
+
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# strobengine
|
|
2
|
+
|
|
3
|
+
A high-performance HTTP, WebSocket, gRPC, and SSE load testing engine with a Python API and a bare-metal Rust core.
|
|
4
|
+
|
|
5
|
+
## Install
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
pip install strobengine
|
|
9
|
+
# or from source
|
|
10
|
+
git clone https://github.com/strobeops/strobengine.git && cd strobengine && uv sync
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
> See [Installation Guide](https://github.com/strobeops/strobengine/blob/main/docs/install.md) for Docker, PyPI, and development setup.
|
|
14
|
+
|
|
15
|
+
## Quick Start
|
|
16
|
+
|
|
17
|
+
```python
|
|
18
|
+
from strobengine import StrobEngine
|
|
19
|
+
from strobengine.reporter import print_summary
|
|
20
|
+
|
|
21
|
+
engine = StrobEngine(url="http://localhost:8080/api/health", concurrency=50, duration=30)
|
|
22
|
+
summary = engine.run()
|
|
23
|
+
print_summary(summary)
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
strobengine load http://localhost:8080/api/health -c 50 -d 30
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
> See [Quick Start Guide](https://github.com/strobeops/strobengine/blob/main/docs/quickstart.md) for Python API and CLI examples.
|
|
31
|
+
|
|
32
|
+
## Documentation
|
|
33
|
+
|
|
34
|
+
| Section | Description |
|
|
35
|
+
|---------|-------------|
|
|
36
|
+
| [Installation](https://github.com/strobeops/strobengine/blob/main/docs/install.md) | Build from source, Docker, PyPI |
|
|
37
|
+
| [Dependencies](https://github.com/strobeops/strobengine/blob/main/docs/dependencies.md) | System requirements, Rust crates, Python packages |
|
|
38
|
+
| [Quick Start](https://github.com/strobeops/strobengine/blob/main/docs/quickstart.md) | Python API and CLI usage examples |
|
|
39
|
+
| [CLI Reference](https://github.com/strobeops/strobengine/blob/main/docs/cli.md) | All subcommands, flags, and options |
|
|
40
|
+
| [HTTP Methods](https://github.com/strobeops/strobengine/blob/main/docs/http_methods.md) | Supported methods, request bodies, headers |
|
|
41
|
+
| [WebSocket](https://github.com/strobeops/strobengine/blob/main/docs/websockets.md) | WS/WSS modes, pub/sub broadcasting, metrics |
|
|
42
|
+
| [gRPC](https://github.com/strobeops/strobengine/blob/main/docs/grpc.md) | Unary calls, protobuf, deadline, chaos |
|
|
43
|
+
| [HTTP/3](https://github.com/strobeops/strobengine/blob/main/docs/http3.md) | QUIC transport, 0-RTT resumption, loss recovery |
|
|
44
|
+
| [SSE](https://github.com/strobeops/strobengine/blob/main/docs/sse.md) | Server-Sent Events streaming, TTFB, event intervals |
|
|
45
|
+
| [Reporting](https://github.com/strobeops/strobengine/blob/main/docs/reports.md) | JSON persistence, HTML reports, baseline comparison |
|
|
46
|
+
| [Docker](https://github.com/strobeops/strobengine/blob/main/docs/docker.md) | Pulling images, running containers |
|
|
47
|
+
| [Testing](https://github.com/strobeops/strobengine/blob/main/docs/testing.md) | Test structure, e2e, CI/CD |
|
|
48
|
+
| [Roadmap](https://github.com/strobeops/strobengine/blob/main/docs/roadmap.md) | Planned features |
|
|
49
|
+
|
|
50
|
+
## How to Contribute
|
|
51
|
+
|
|
52
|
+
1. Fork the repository
|
|
53
|
+
2. Create a feature branch (`git checkout -b feat/my-feature`)
|
|
54
|
+
3. Install dependencies and set up pre-push Git hooks:
|
|
55
|
+
```bash
|
|
56
|
+
uv sync
|
|
57
|
+
uv run pre-commit install --hook-type pre-push
|
|
58
|
+
```
|
|
59
|
+
4. Run formatting, linting, and tests manually (or let git push run them automatically):
|
|
60
|
+
```bash
|
|
61
|
+
make check
|
|
62
|
+
```
|
|
63
|
+
5. Commit your changes following [Conventional Commits](https://www.conventionalcommits.org/)
|
|
64
|
+
6. Push to your branch and open a Pull Request
|
|
65
|
+
|
|
66
|
+
## License
|
|
67
|
+
|
|
68
|
+
This project is licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/strobeops/strobengine/blob/main/LICENSE) for the full text.
|
|
@@ -43,6 +43,8 @@ By default, this spawns **10 concurrent workers** for **10 seconds** with a **10
|
|
|
43
43
|
| `--http3/--no-http3` | off | Enable HTTP/3 over QUIC |
|
|
44
44
|
| `--quic-zero-rtt` | off | Enable QUIC 0-RTT connection testing |
|
|
45
45
|
| `--quic-max-idle-timeout` | none | QUIC max idle timeout in ms |
|
|
46
|
+
| `--sse/--no-sse` | off | Enable SSE streaming mode |
|
|
47
|
+
| `--sse-max-events` | none | Maximum events to receive per connection |
|
|
46
48
|
|
|
47
49
|
## Stress Subcommand Options
|
|
48
50
|
|
|
@@ -75,6 +77,8 @@ By default, this spawns **10 concurrent workers** for **10 seconds** with a **10
|
|
|
75
77
|
| `--http3/--no-http3` | off | Enable HTTP/3 over QUIC |
|
|
76
78
|
| `--quic-zero-rtt` | off | Enable QUIC 0-RTT connection testing |
|
|
77
79
|
| `--quic-max-idle-timeout` | none | QUIC max idle timeout in ms |
|
|
80
|
+
| `--sse/--no-sse` | off | Enable SSE streaming mode |
|
|
81
|
+
| `--sse-max-events` | none | Maximum events to receive per connection |
|
|
78
82
|
|
|
79
83
|
## Spike Subcommand Options
|
|
80
84
|
|
|
@@ -108,6 +112,8 @@ By default, this spawns **10 concurrent workers** for **10 seconds** with a **10
|
|
|
108
112
|
| `--http3/--no-http3` | off | Enable HTTP/3 over QUIC |
|
|
109
113
|
| `--quic-zero-rtt` | off | Enable QUIC 0-RTT connection testing |
|
|
110
114
|
| `--quic-max-idle-timeout` | none | QUIC max idle timeout in ms |
|
|
115
|
+
| `--sse/--no-sse` | off | Enable SSE streaming mode |
|
|
116
|
+
| `--sse-max-events` | none | Maximum events to receive per connection |
|
|
111
117
|
|
|
112
118
|
## Global Options
|
|
113
119
|
|
|
@@ -76,6 +76,19 @@ summary = engine.run()
|
|
|
76
76
|
print_summary(summary)
|
|
77
77
|
```
|
|
78
78
|
|
|
79
|
+
### SSE Streaming
|
|
80
|
+
|
|
81
|
+
```python
|
|
82
|
+
engine = StrobEngine(
|
|
83
|
+
url="http://localhost:8080/sse",
|
|
84
|
+
concurrency=10,
|
|
85
|
+
duration=30,
|
|
86
|
+
options=RequestOptions(sse_enabled=True),
|
|
87
|
+
)
|
|
88
|
+
summary = engine.run()
|
|
89
|
+
print_summary(summary)
|
|
90
|
+
```
|
|
91
|
+
|
|
79
92
|
### Async Execution
|
|
80
93
|
|
|
81
94
|
For async contexts (FastAPI, Typer, etc.):
|
|
@@ -111,6 +124,9 @@ strobengine load grpc://localhost:50051 \
|
|
|
111
124
|
--grpc-method SayHello \
|
|
112
125
|
--grpc-payload CgR0ZXN0 \
|
|
113
126
|
-c 10 -d 30
|
|
127
|
+
|
|
128
|
+
# SSE streaming
|
|
129
|
+
strobengine load http://localhost:8080/sse --sse -c 10 -d 30
|
|
114
130
|
```
|
|
115
131
|
|
|
116
132
|
> See [CLI Reference](cli.md) for all available flags and options.
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# Reporting & Visualizations
|
|
2
|
+
|
|
3
|
+
strobengine automatically persists execution reports as JSON artifacts
|
|
4
|
+
and optionally generates standalone HTML reports with interactive
|
|
5
|
+
Chart.js visualizations and historical baseline comparisons.
|
|
6
|
+
|
|
7
|
+
## Implicit Report Persistence
|
|
8
|
+
|
|
9
|
+
After every load test, strobengine writes a JSON report artifact
|
|
10
|
+
to disk by default.
|
|
11
|
+
|
|
12
|
+
| Property | Value |
|
|
13
|
+
|----------|-------|
|
|
14
|
+
| Default directory | `./.strobengine/reports/` |
|
|
15
|
+
| Filename pattern | `report_YYYYMMDD_HHMMSS_<host>.json` |
|
|
16
|
+
| Atomic writes | Yes — tmp file + rename prevents corruption |
|
|
17
|
+
| `latest.json` pointer | Yes — tracks most recent report |
|
|
18
|
+
|
|
19
|
+
Reports are written by both the Rust engine (`run_load_test`) and
|
|
20
|
+
the Python reporter (`save_report`).
|
|
21
|
+
|
|
22
|
+
## Configuration Flags
|
|
23
|
+
|
|
24
|
+
These flags appear on all three subcommands (`load`, `stress`, `spike`):
|
|
25
|
+
|
|
26
|
+
| Flag | Type | Default | Description |
|
|
27
|
+
|------|------|---------|-------------|
|
|
28
|
+
| `--output-dir <DIR>` | string | `.strobengine/reports/` | Override report storage directory |
|
|
29
|
+
| `--no-save` | flag | false | Disable disk persistence entirely |
|
|
30
|
+
| `--html <PATH>` | string | none | Generate standalone HTML report at filepath |
|
|
31
|
+
| `--compare-to <PATH>` | string | none | Baseline JSON report for comparison |
|
|
32
|
+
|
|
33
|
+
## JSON Artifact Schema
|
|
34
|
+
|
|
35
|
+
### Structure
|
|
36
|
+
|
|
37
|
+
| Section | Key Fields |
|
|
38
|
+
|---------|------------|
|
|
39
|
+
| `metadata` | `timestamp`, `duration_secs`, `target_url`, `cli_options`, `system_info` |
|
|
40
|
+
| `cli_options` | `method`, `concurrency`, `timeout_secs`, `chaos`, `chaos_rate`, `body`, `headers` |
|
|
41
|
+
| `system_info` | `hostname`, `platform`, `version` |
|
|
42
|
+
| `summary` | `total_requests`, `successful_requests`, `failed_requests`, `rps`, `bytes_transferred` |
|
|
43
|
+
| `latency_percentiles` | `p50_us`, `p90_us`, `p95_us`, `p99_us`, `min_us`, `max_us`, `mean_us` |
|
|
44
|
+
| `error_breakdown` | Status code string → count (e.g., `"200": 950, "500": 50`) |
|
|
45
|
+
|
|
46
|
+
All latency values are stored in **microseconds**.
|
|
47
|
+
|
|
48
|
+
### Example
|
|
49
|
+
|
|
50
|
+
```json
|
|
51
|
+
{
|
|
52
|
+
"metadata": {
|
|
53
|
+
"timestamp": "2026-08-28T09:48:19Z",
|
|
54
|
+
"duration_secs": 30.0,
|
|
55
|
+
"target_url": "http://localhost:8080/api",
|
|
56
|
+
"cli_options": { "method": "GET", "concurrency": 50, "chaos": false },
|
|
57
|
+
"system_info": { "hostname": "ci-runner-01", "platform": "linux", "version": "0.5.0" }
|
|
58
|
+
},
|
|
59
|
+
"summary": { "total_requests": 1500, "successful_requests": 1485, "failed_requests": 15, "rps": 50.0, "bytes_transferred": 524288 },
|
|
60
|
+
"latency_percentiles": { "p50_us": 1200.0, "p90_us": 3500.0, "p95_us": 5200.0, "p99_us": 9800.0, "min_us": 200.0, "max_us": 15000.0, "mean_us": 2800.0 },
|
|
61
|
+
"error_breakdown": { "200": 1485, "500": 15 }
|
|
62
|
+
}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## HTML Report Generation
|
|
66
|
+
|
|
67
|
+
### Dependencies
|
|
68
|
+
|
|
69
|
+
- **Chart.js** — bundled locally at `src/strobengine/reporting/assets/chart.min.js`
|
|
70
|
+
(186KB, embedded inline in generated HTML — 100% offline, no CDN required)
|
|
71
|
+
- **Jinja2** — used for template rendering (`jinja2>=3.1`)
|
|
72
|
+
|
|
73
|
+
### CLI Usage
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
strobengine load http://localhost:8080/api -c 10 -d 30 --html report.html
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### Features
|
|
80
|
+
|
|
81
|
+
- **Self-contained single-file HTML** — no external dependencies
|
|
82
|
+
- **Embedded Chart.js** — 100% offline, no internet required
|
|
83
|
+
- **Dark theme** — slate/navy color palette
|
|
84
|
+
- **Bar chart**: Latency percentiles (P50, P90, P95, P99) in milliseconds
|
|
85
|
+
- **Doughnut chart**: Status code distribution (2xx, 4xx, 5xx, Other)
|
|
86
|
+
- **Metadata grid**: Target URL, method, concurrency, duration, RPS, error rate
|
|
87
|
+
|
|
88
|
+
### Python API
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
from strobengine import StrobEngine
|
|
92
|
+
from strobengine.reporting.html_report import save_html_report
|
|
93
|
+
|
|
94
|
+
engine = StrobEngine(url="http://localhost:8080", concurrency=10, duration=30)
|
|
95
|
+
summary = engine.run()
|
|
96
|
+
filepath = save_html_report(summary, engine.get_config(), "report.html")
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Historical Baseline Comparison
|
|
100
|
+
|
|
101
|
+
### CLI Usage
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
# First run — generates baseline report
|
|
105
|
+
strobengine load http://localhost:8080/api -c 10 -d 30
|
|
106
|
+
|
|
107
|
+
# Second run — compare against baseline (terminal output)
|
|
108
|
+
strobengine load http://localhost:8080/api -c 10 -d 30 \
|
|
109
|
+
--compare-to .strobengine/reports/report_baseline.json
|
|
110
|
+
|
|
111
|
+
# Second run — compare with HTML report
|
|
112
|
+
strobengine load http://localhost:8080/api -c 10 -d 30 \
|
|
113
|
+
--html report.html --compare-to .strobengine/reports/report_baseline.json
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
### Terminal Output
|
|
117
|
+
|
|
118
|
+
`--compare-to` displays a Rich table comparing baseline vs current metrics,
|
|
119
|
+
even without `--html`. The table includes:
|
|
120
|
+
|
|
121
|
+
| Metric | Baseline | Current | Delta |
|
|
122
|
+
|--------|----------|---------|-------|
|
|
123
|
+
| RPS | 33.00 | 40.00 | +21.21% (green) |
|
|
124
|
+
| P95 Latency | 35.00 ms | 25.00 ms | -28.57% (green) |
|
|
125
|
+
| Error Rate | 10.00% | 5.00% | -5.00pp (green) |
|
|
126
|
+
|
|
127
|
+
Color coding:
|
|
128
|
+
- **Green**: Improvement (higher RPS, lower latency, fewer errors)
|
|
129
|
+
- **Red**: Regression (lower RPS, higher latency, more errors)
|
|
130
|
+
- **Gray**: No change (0% delta)
|
|
131
|
+
|
|
132
|
+
### HTML Output
|
|
133
|
+
|
|
134
|
+
When used with `--html`, the comparison section is embedded in the report
|
|
135
|
+
with the same color-coded deltas.
|
|
136
|
+
|
|
137
|
+
### Delta Calculation
|
|
138
|
+
|
|
139
|
+
| Metric | Formula | Unit |
|
|
140
|
+
|--------|---------|------|
|
|
141
|
+
| RPS | `((current - baseline) / baseline) * 100` | % (positive = improvement) |
|
|
142
|
+
| P95 Latency | `((current - baseline) / baseline) * 100` | % (negative = improvement) |
|
|
143
|
+
| Error Rate | `current_rate - baseline_rate` | pp (negative = improvement) |
|
|
144
|
+
|
|
145
|
+
## Limitations
|
|
146
|
+
|
|
147
|
+
- Baseline comparison assumes same target URL (cross-URL comparison not supported)
|
|
148
|
+
- Report schema version is not tracked (no migration support)
|
|
149
|
+
- `--compare-to` requires the baseline file to be a valid strobengine JSON artifact
|
|
@@ -72,15 +72,47 @@ This document outlines the planned trajectory and upcoming feature epics for **s
|
|
|
72
72
|
### Epic: Test Persistence & Visual Reporting
|
|
73
73
|
*Target Focus: Long-Term Benchmark Tracking & Visualization*
|
|
74
74
|
|
|
75
|
-
- [
|
|
76
|
-
- [
|
|
77
|
-
- [
|
|
78
|
-
- [
|
|
79
|
-
- [
|
|
80
|
-
- [
|
|
81
|
-
- [
|
|
82
|
-
- [
|
|
83
|
-
- [
|
|
75
|
+
- [x] **Artifact Persistence Engine** `[v0.5.0]`
|
|
76
|
+
- [x] Automatic disk storage for raw JSON execution metrics (`--output-dir`, `--no-save`). `[v0.5.0]`
|
|
77
|
+
- [x] Standardized report schema featuring execution metadata (timestamps, CLI flags, system specs, latency percentiles). `[v0.5.0]`
|
|
78
|
+
- [x] **HTML / Visual Report Generator** `[v0.5.0]`
|
|
79
|
+
- [x] Standalone HTML report output with embedded latency distribution charts (p50, p90, p95, p99). `[v0.5.0]`
|
|
80
|
+
- [x] Summary dashboards comparing current runs against historical baseline artifacts. `[v0.5.0]`
|
|
81
|
+
- [x] **Export Formats** `[v0.5.0]`
|
|
82
|
+
- [x] Markdown summary generation for CI/CD Pull Request integration (GitHub Actions step comments). `[v0.5.0]`
|
|
83
|
+
- [x] CSV/JUnit XML exports for standard test runner ingestion. `[v0.5.0]`
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
### Epic: Advanced Metrics & Observability
|
|
88
|
+
*Target Focus: Deep Performance Insights & System Monitoring*
|
|
89
|
+
|
|
90
|
+
- [ ] **Aggregate Existing Protocol Metrics** `[v0.6.0]`
|
|
91
|
+
- [ ] Forward `connection_latency_us`, `quic_*`, `sse_*` fields through aggregator to `TestSummary`
|
|
92
|
+
- [ ] Add protocol-specific sections to JSON report artifact
|
|
93
|
+
|
|
94
|
+
- [ ] **Chaos Fault Tracking** `[v0.6.0]`
|
|
95
|
+
- [ ] Count injected faults by type (ConnectionDrop, LatencySpike, etc.)
|
|
96
|
+
- [ ] Include fault breakdown in JSON report artifact
|
|
97
|
+
|
|
98
|
+
- [ ] **Extended Latency Statistics** `[v0.6.0]`
|
|
99
|
+
- [ ] Compute latency jitter (standard deviation)
|
|
100
|
+
- [ ] Add p99.99 percentile
|
|
101
|
+
- [ ] Add latency histogram buckets for distribution visualization
|
|
102
|
+
|
|
103
|
+
- [ ] **Client System Resource Monitoring** `[v0.7.0]`
|
|
104
|
+
- [ ] Track engine CPU %, memory (RSS), thread count during test
|
|
105
|
+
- [ ] Sample at configurable interval (e.g., every 1s)
|
|
106
|
+
- [ ] Include in JSON report artifact and HTML visualization
|
|
107
|
+
|
|
108
|
+
- [ ] **Connection Pool Metrics** `[v0.7.0]`
|
|
109
|
+
- [ ] Track socket creation rate vs reuse rate
|
|
110
|
+
- [ ] Report DNS resolution duration (if instrumentable)
|
|
111
|
+
|
|
112
|
+
- [ ] **Protocol-Specific Deep Metrics** `[v0.8.0]`
|
|
113
|
+
- [ ] WebSocket: unsolicited pings/pongs, backpressure depth
|
|
114
|
+
- [ ] gRPC: stream concurrency utilization, flow control window exhaustion
|
|
115
|
+
- [ ] HTTP/3: congestion window size, connection migration success rate
|
|
84
116
|
|
|
85
117
|
---
|
|
86
118
|
|
|
@@ -102,8 +134,8 @@ This document outlines the planned trajectory and upcoming feature epics for **s
|
|
|
102
134
|
- [x] **HTTP/3 (QUIC)** `[v0.4.0] - 2026-08-23`
|
|
103
135
|
- [x] UDP-based QUIC transport layer support. `[v0.4.0] - 2026-08-23`
|
|
104
136
|
- [x] Zero-RTT connection testing and loss recovery benchmarking. `[v0.4.0] - 2026-08-23`
|
|
105
|
-
- [
|
|
106
|
-
- [
|
|
137
|
+
- [x] **Server-Sent Events (SSE / HTTP Streaming)** `[v0.5.0]`
|
|
138
|
+
- [x] Persistent HTTP streaming response parsing (Time-to-First-Byte, chunk latency, LLM stream benchmarking). `[v0.5.0]`
|
|
107
139
|
|
|
108
140
|
---
|
|
109
141
|
|