secure-log2test 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,39 @@
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+ *.so
6
+ .Python
7
+ build/
8
+ dist/
9
+ *.egg-info/
10
+ *.egg
11
+
12
+ # Virtual environments
13
+ .venv/
14
+ venv/
15
+ env/
16
+ ENV/
17
+
18
+ # Testing
19
+ .pytest_cache/
20
+ .coverage
21
+ htmlcov/
22
+ .tox/
23
+
24
+ # IDE
25
+ .idea/
26
+ .vscode/
27
+ *.swp
28
+ *.swo
29
+
30
+ # OS
31
+ .DS_Store
32
+ Thumbs.db
33
+
34
+ # Generated output
35
+ generated_tests/
36
+ tests_generated.py
37
+
38
+ # Local release runbooks (not for public repo)
39
+ PUBLISH_*.md
@@ -0,0 +1,42 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented here. Format is loosely based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project follows [Semantic Versioning](https://semver.org/).
4
+
5
+ ## [Unreleased]
6
+
7
+ ## [1.0.0] - 2026-05-10
8
+
9
+ First stable release. Public API surface (CLI flags, JSON input shape, generated test layout) is now considered stable. Future minor versions will add features without breaking existing usage.
10
+
11
+ ### Added
12
+ - `pyproject.toml` with PEP 621 metadata, hatchling build backend, console-script entry point.
13
+ - `secure-log2test` console command available after `pip install -e .` or `pip install secure-log2test` (post-PyPI publish).
14
+ - `python -m secure_log2test` invocation via `__main__.py`.
15
+ - Python 3.12 added to CI matrix.
16
+ - CONTRIBUTING guide for issue reports and pull requests.
17
+ - MIT license file.
18
+ - This changelog.
19
+ - README badges for CI status, supported Python versions, and licence.
20
+ - Self-roadmap GitHub issues for v1.1 (response body assertions, schema match) and v1.2 (custom redaction rules via config file).
21
+
22
+ ### Changed
23
+ - Repackaged into `secure_log2test/` Python package. `core/` moved to `secure_log2test/core/`. `templates/` moved to `secure_log2test/templates/`.
24
+ - `main.py` removed at top level; CLI entry now lives in `secure_log2test/cli.py`.
25
+ - CI workflow installs the project as a package (`pip install -e ".[dev]"`) and exercises the installed CLI.
26
+ - Test imports updated to reference `secure_log2test.core.parser`.
27
+ - `secure_log2test.__version__` now reads from installed package metadata via `importlib.metadata`, so it always matches the wheel version. Previously hardcoded.
28
+
29
+ ## [0.2.0] - 2026-05-05
30
+
31
+ ### Added
32
+ - GitHub Actions CI workflow with a Python 3.10 and 3.11 matrix.
33
+ - CLI smoke test inside CI: generate a suite from `data/sample_kibana_export.json` and run `ast.parse` on the output.
34
+ - Edge-case tests covering auth headers, empty request bodies, and 5xx responses.
35
+
36
+ ## [0.1.0] - 2026-05-03
37
+
38
+ ### Added
39
+ - Pytest test generator and CLI entry point (`python main.py <kibana_export.json>`).
40
+ - Parser unit tests and a sample Kibana fixture under `data/`.
41
+ - Core parser that turns Kibana request logs into pytest test cases.
42
+ - Initial project scaffold and README.
@@ -0,0 +1,38 @@
1
+ # Contributing
2
+
3
+ Thanks for your interest in secure-log2test. This is a small project, so the contribution flow is simple.
4
+
5
+ ## Reporting a bug
6
+
7
+ Open an issue with:
8
+
9
+ - What you ran (command + Python version)
10
+ - What you expected
11
+ - What happened instead
12
+ - A minimal Kibana export sample if the bug is parser-related (strip secrets first)
13
+
14
+ ## Suggesting a feature
15
+
16
+ Open an issue first so we can talk through the use case before you write code. Keeps both of us from wasting time.
17
+
18
+ ## Submitting a pull request
19
+
20
+ 1. Fork the repo and create a branch from `main`.
21
+ 2. Make your changes. Keep the diff focused on one thing.
22
+ 3. Add or update tests in `tests/`. The CI runs `pytest -v` on Python 3.10 and 3.11.
23
+ 4. Run the tests locally before pushing:
24
+ ```bash
25
+ pip install -r requirements.txt
26
+ pytest -v
27
+ ```
28
+ 5. Open the PR with a short description of what changed and why.
29
+
30
+ ## Code style
31
+
32
+ - Plain Python, no extra dependencies unless there's a real reason.
33
+ - Function and variable names in English, lowercase with underscores.
34
+ - One responsibility per function. If a function grows past 30-40 lines, split it.
35
+
36
+ ## Security
37
+
38
+ If you find something that could leak secrets from a real log file, please email me directly instead of opening a public issue. Address is on my GitHub profile.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Mikhail Golikov
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,177 @@
1
+ Metadata-Version: 2.4
2
+ Name: secure-log2test
3
+ Version: 1.0.0
4
+ Summary: Convert Kibana API log exports into runnable pytest suites with auth header redaction.
5
+ Project-URL: Homepage, https://github.com/golikovichev/secure-log2test
6
+ Project-URL: Repository, https://github.com/golikovichev/secure-log2test
7
+ Project-URL: Issues, https://github.com/golikovichev/secure-log2test/issues
8
+ Project-URL: Changelog, https://github.com/golikovichev/secure-log2test/blob/main/CHANGELOG.md
9
+ Author-email: Mikhail Golikov <golikovichev@gmail.com>
10
+ Maintainer-email: Mikhail Golikov <golikovichev@gmail.com>
11
+ License: MIT License
12
+
13
+ Copyright (c) 2026 Mikhail Golikov
14
+
15
+ Permission is hereby granted, free of charge, to any person obtaining a copy
16
+ of this software and associated documentation files (the "Software"), to deal
17
+ in the Software without restriction, including without limitation the rights
18
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
19
+ copies of the Software, and to permit persons to whom the Software is
20
+ furnished to do so, subject to the following conditions:
21
+
22
+ The above copyright notice and this permission notice shall be included in all
23
+ copies or substantial portions of the Software.
24
+
25
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
26
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
27
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
28
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
29
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
30
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
31
+ SOFTWARE.
32
+ License-File: LICENSE
33
+ Keywords: api-testing,kibana,log-parsing,pytest,qa,test-generation
34
+ Classifier: Development Status :: 4 - Beta
35
+ Classifier: Intended Audience :: Developers
36
+ Classifier: License :: OSI Approved :: MIT License
37
+ Classifier: Operating System :: OS Independent
38
+ Classifier: Programming Language :: Python :: 3
39
+ Classifier: Programming Language :: Python :: 3.10
40
+ Classifier: Programming Language :: Python :: 3.11
41
+ Classifier: Programming Language :: Python :: 3.12
42
+ Classifier: Topic :: Software Development :: Quality Assurance
43
+ Classifier: Topic :: Software Development :: Testing
44
+ Requires-Python: >=3.10
45
+ Requires-Dist: jinja2>=3.1
46
+ Requires-Dist: pydantic>=2.0
47
+ Requires-Dist: requests>=2.31
48
+ Provides-Extra: dev
49
+ Requires-Dist: build>=1.0; extra == 'dev'
50
+ Requires-Dist: pytest>=7.0; extra == 'dev'
51
+ Requires-Dist: twine>=4.0; extra == 'dev'
52
+ Description-Content-Type: text/markdown
53
+
54
+ # secure-log2test
55
+
56
+ [![CI](https://github.com/golikovichev/secure-log2test/actions/workflows/ci.yml/badge.svg)](https://github.com/golikovichev/secure-log2test/actions/workflows/ci.yml)
57
+ [![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://pypi.org/project/secure-log2test/)
58
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
59
+
60
+ Turn a Kibana API log export into an executable pytest suite. Auth headers redacted by default.
61
+
62
+ Status: alpha. v1.0.0 lands May 2026.
63
+
64
+ ## Why
65
+
66
+ You have Kibana logs from staging or production. Each entry is a real request: method, URL, status, duration, headers, body. That's a regression suite waiting to happen. Most teams either ignore it, screenshot interesting failures into Jira, or hand-write pytest cases from log entries one at a time.
67
+
68
+ I needed a faster path. `secure-log2test` reads a Kibana JSON export and writes a pytest module you can run and commit. Auth values get replaced with `***REDACTED***` before they ever touch the output, so a generated suite is safe to push to a public repo.
69
+
70
+ The tool exists because at Лента I kept doing the same five steps by hand for every production incident: open Kibana, scroll, copy the failing request, paste into a new test, repeat. Five minutes per request times ten requests means an hour gone before any actual debugging starts.
71
+
72
+ ## Quickstart
73
+
74
+ ```bash
75
+ git clone https://github.com/golikovichev/secure-log2test
76
+ cd secure-log2test
77
+ python -m venv .venv && source .venv/bin/activate # or .venv\Scripts\activate on Windows
78
+ pip install -r requirements.txt
79
+
80
+ python main.py data/sample_kibana_export.json --output tests_generated.py
81
+ pytest tests_generated.py -v
82
+ ```
83
+
84
+ The sample export ships with the repo (`data/sample_kibana_export.json`), so you can see real output without setting up a Kibana instance first.
85
+
86
+ PyPI install (`pip install secure-log2test`) lands with v1.0.0.
87
+
88
+ ## How it works
89
+
90
+ Two stages, kept separate.
91
+
92
+ **Parse** (`core/parser.py`). Reads the Kibana JSON and validates each entry through Pydantic v2. Sensitive headers are redacted before any further processing. The redaction list lives in `SENSITIVE_HEADERS`: `authorization`, `proxy-authorization`, `cookie`, `set-cookie`, `x-api-key`, `x-auth-token`, `authentication`. Match is case-insensitive. Header values get replaced with `***REDACTED***`. The original input dict is not mutated.
93
+
94
+ **Generate** (`core/generator.py`). Takes the cleaned entries and renders a Jinja2 template (`templates/test_module.py.j2`) into a pytest module. Each log entry becomes one `test_*` function. The slug filter turns `/api/v1/users/42` into a stable function name. A `--base-url` flag lets you target staging vs production at runtime.
95
+
96
+ The split lets you reuse the parser for other formats. If you want to generate Locust scripts, k6 scenarios, or an OpenAPI spec from the same logs, the parser stays. Only the template changes.
97
+
98
+ ## Sample output
99
+
100
+ Given this Kibana log entry:
101
+
102
+ ```json
103
+ {
104
+ "method": "POST",
105
+ "url": "/api/v1/users",
106
+ "status": 201,
107
+ "headers": {"Authorization": "Bearer abc.xyz", "Content-Type": "application/json"},
108
+ "body": {"name": "Test", "email": "test@example.com"}
109
+ }
110
+ ```
111
+
112
+ The generator emits something like:
113
+
114
+ ```python
115
+ def test_post_api_v1_users():
116
+ response = requests.post(
117
+ f"{BASE_URL}/api/v1/users",
118
+ headers={"Authorization": "***REDACTED***", "Content-Type": "application/json"},
119
+ json={"name": "Test", "email": "test@example.com"},
120
+ )
121
+ assert response.status_code == 201, (
122
+ f"Expected 201, got {response.status_code}: {response.text[:200]}"
123
+ )
124
+ ```
125
+
126
+ The `Authorization` value never leaves the parser intact. You set the real token in your environment at run time.
127
+
128
+ ## Limitations
129
+
130
+ This is what v0.2.0 does **not** handle. Calling them out so the tool stays trustworthy.
131
+
132
+ - OAuth flows. Only static `Authorization` headers, redacted to a placeholder.
133
+ - Multipart bodies and file uploads.
134
+ - Streaming responses or chunked transfer.
135
+ - Kibana exports older than the schema in `data/sample_kibana_export.json`. Fixture-driven; if your export looks different, open an issue with a redacted sample.
136
+ - Response body assertions. Status code only for now.
137
+ - Pre-request scripts or test scripts (the project takes only what is in the log entry).
138
+
139
+ If something on this list blocks you, open an issue. v1.0 stays small on purpose; the roadmap below is where new scope goes.
140
+
141
+ ## Roadmap
142
+
143
+ | Version | Adds |
144
+ | --- | --- |
145
+ | 0.3 | Response body assertions, optional schema match. |
146
+ | 0.4 | Custom redaction rules via config file. |
147
+ | 0.5 | Multiple output formats (k6, Locust). |
148
+ | 1.0 | First stable API, PyPI release, full docs site. |
149
+
150
+ Target dates live in `CHANGELOG.md` once the work is in flight.
151
+
152
+ ## Tests
153
+
154
+ ```bash
155
+ pytest tests/ -v
156
+ ```
157
+
158
+ Coverage as of 0.2.0:
159
+ - Parser unit tests for valid input, malformed input, header redaction, empty bodies.
160
+ - Edge cases for 5xx responses and missing fields.
161
+ - CI smoke test that runs the CLI end-to-end on the sample export and parses the generated Python with `ast.parse`.
162
+
163
+ CI runs on Python 3.10 and 3.11 via GitHub Actions.
164
+
165
+ ## Security note
166
+
167
+ The default redaction list is conservative. If your team uses a custom auth header (`X-Company-Token`, anything not in the standard list), add it to `SENSITIVE_HEADERS` in `core/parser.py` before generating output. PRs welcome.
168
+
169
+ Never commit a generated suite that includes real production tokens. The redaction is a safety net, not a substitute for review.
170
+
171
+ ## Contributing
172
+
173
+ Issue templates and PR guidance live in [CONTRIBUTING.md](CONTRIBUTING.md). Bug reports with a redacted sample log are the most useful kind.
174
+
175
+ ## Licence
176
+
177
+ MIT. See [LICENSE](LICENSE).
@@ -0,0 +1,124 @@
1
+ # secure-log2test
2
+
3
+ [![CI](https://github.com/golikovichev/secure-log2test/actions/workflows/ci.yml/badge.svg)](https://github.com/golikovichev/secure-log2test/actions/workflows/ci.yml)
4
+ [![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://pypi.org/project/secure-log2test/)
5
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
6
+
7
+ Turn a Kibana API log export into an executable pytest suite. Auth headers redacted by default.
8
+
9
+ Status: alpha. v1.0.0 lands May 2026.
10
+
11
+ ## Why
12
+
13
+ You have Kibana logs from staging or production. Each entry is a real request: method, URL, status, duration, headers, body. That's a regression suite waiting to happen. Most teams either ignore it, screenshot interesting failures into Jira, or hand-write pytest cases from log entries one at a time.
14
+
15
+ I needed a faster path. `secure-log2test` reads a Kibana JSON export and writes a pytest module you can run and commit. Auth values get replaced with `***REDACTED***` before they ever touch the output, so a generated suite is safe to push to a public repo.
16
+
17
+ The tool exists because at Лента I kept doing the same five steps by hand for every production incident: open Kibana, scroll, copy the failing request, paste into a new test, repeat. Five minutes per request times ten requests means an hour gone before any actual debugging starts.
18
+
19
+ ## Quickstart
20
+
21
+ ```bash
22
+ git clone https://github.com/golikovichev/secure-log2test
23
+ cd secure-log2test
24
+ python -m venv .venv && source .venv/bin/activate # or .venv\Scripts\activate on Windows
25
+ pip install -r requirements.txt
26
+
27
+ python main.py data/sample_kibana_export.json --output tests_generated.py
28
+ pytest tests_generated.py -v
29
+ ```
30
+
31
+ The sample export ships with the repo (`data/sample_kibana_export.json`), so you can see real output without setting up a Kibana instance first.
32
+
33
+ PyPI install (`pip install secure-log2test`) lands with v1.0.0.
34
+
35
+ ## How it works
36
+
37
+ Two stages, kept separate.
38
+
39
+ **Parse** (`core/parser.py`). Reads the Kibana JSON and validates each entry through Pydantic v2. Sensitive headers are redacted before any further processing. The redaction list lives in `SENSITIVE_HEADERS`: `authorization`, `proxy-authorization`, `cookie`, `set-cookie`, `x-api-key`, `x-auth-token`, `authentication`. Match is case-insensitive. Header values get replaced with `***REDACTED***`. The original input dict is not mutated.
40
+
41
+ **Generate** (`core/generator.py`). Takes the cleaned entries and renders a Jinja2 template (`templates/test_module.py.j2`) into a pytest module. Each log entry becomes one `test_*` function. The slug filter turns `/api/v1/users/42` into a stable function name. A `--base-url` flag lets you target staging vs production at runtime.
42
+
43
+ The split lets you reuse the parser for other formats. If you want to generate Locust scripts, k6 scenarios, or an OpenAPI spec from the same logs, the parser stays. Only the template changes.
44
+
45
+ ## Sample output
46
+
47
+ Given this Kibana log entry:
48
+
49
+ ```json
50
+ {
51
+ "method": "POST",
52
+ "url": "/api/v1/users",
53
+ "status": 201,
54
+ "headers": {"Authorization": "Bearer abc.xyz", "Content-Type": "application/json"},
55
+ "body": {"name": "Test", "email": "test@example.com"}
56
+ }
57
+ ```
58
+
59
+ The generator emits something like:
60
+
61
+ ```python
62
+ def test_post_api_v1_users():
63
+ response = requests.post(
64
+ f"{BASE_URL}/api/v1/users",
65
+ headers={"Authorization": "***REDACTED***", "Content-Type": "application/json"},
66
+ json={"name": "Test", "email": "test@example.com"},
67
+ )
68
+ assert response.status_code == 201, (
69
+ f"Expected 201, got {response.status_code}: {response.text[:200]}"
70
+ )
71
+ ```
72
+
73
+ The `Authorization` value never leaves the parser intact. You set the real token in your environment at run time.
74
+
75
+ ## Limitations
76
+
77
+ This is what v0.2.0 does **not** handle. Calling them out so the tool stays trustworthy.
78
+
79
+ - OAuth flows. Only static `Authorization` headers, redacted to a placeholder.
80
+ - Multipart bodies and file uploads.
81
+ - Streaming responses or chunked transfer.
82
+ - Kibana exports older than the schema in `data/sample_kibana_export.json`. Fixture-driven; if your export looks different, open an issue with a redacted sample.
83
+ - Response body assertions. Status code only for now.
84
+ - Pre-request scripts or test scripts (the project takes only what is in the log entry).
85
+
86
+ If something on this list blocks you, open an issue. v1.0 stays small on purpose; the roadmap below is where new scope goes.
87
+
88
+ ## Roadmap
89
+
90
+ | Version | Adds |
91
+ | --- | --- |
92
+ | 0.3 | Response body assertions, optional schema match. |
93
+ | 0.4 | Custom redaction rules via config file. |
94
+ | 0.5 | Multiple output formats (k6, Locust). |
95
+ | 1.0 | First stable API, PyPI release, full docs site. |
96
+
97
+ Target dates live in `CHANGELOG.md` once the work is in flight.
98
+
99
+ ## Tests
100
+
101
+ ```bash
102
+ pytest tests/ -v
103
+ ```
104
+
105
+ Coverage as of 0.2.0:
106
+ - Parser unit tests for valid input, malformed input, header redaction, empty bodies.
107
+ - Edge cases for 5xx responses and missing fields.
108
+ - CI smoke test that runs the CLI end-to-end on the sample export and parses the generated Python with `ast.parse`.
109
+
110
+ CI runs on Python 3.10 and 3.11 via GitHub Actions.
111
+
112
+ ## Security note
113
+
114
+ The default redaction list is conservative. If your team uses a custom auth header (`X-Company-Token`, anything not in the standard list), add it to `SENSITIVE_HEADERS` in `core/parser.py` before generating output. PRs welcome.
115
+
116
+ Never commit a generated suite that includes real production tokens. The redaction is a safety net, not a substitute for review.
117
+
118
+ ## Contributing
119
+
120
+ Issue templates and PR guidance live in [CONTRIBUTING.md](CONTRIBUTING.md). Bug reports with a redacted sample log are the most useful kind.
121
+
122
+ ## Licence
123
+
124
+ MIT. See [LICENSE](LICENSE).
@@ -0,0 +1,39 @@
1
+ {
2
+ "hits": {
3
+ "total": {"value": 4, "relation": "eq"},
4
+ "hits": [
5
+ {
6
+ "_source": {
7
+ "method": "GET",
8
+ "url": "/api/v1/users",
9
+ "status": 200,
10
+ "duration": 42
11
+ }
12
+ },
13
+ {
14
+ "_source": {
15
+ "method": "POST",
16
+ "url": "/api/v1/users",
17
+ "status": 201,
18
+ "duration": 110
19
+ }
20
+ },
21
+ {
22
+ "_source": {
23
+ "method": "PUT",
24
+ "url": "/api/v1/users/42",
25
+ "status": 204,
26
+ "duration": 87
27
+ }
28
+ },
29
+ {
30
+ "_source": {
31
+ "method": "DELETE",
32
+ "url": "/api/v1/users/99",
33
+ "status": 404,
34
+ "duration": 21
35
+ }
36
+ }
37
+ ]
38
+ }
39
+ }
@@ -0,0 +1,77 @@
1
+ [build-system]
2
+ requires = ["hatchling>=1.21"]
3
+ build-backend = "hatchling.build"
4
+
5
+ [project]
6
+ name = "secure-log2test"
7
+ version = "1.0.0"
8
+ description = "Convert Kibana API log exports into runnable pytest suites with auth header redaction."
9
+ readme = "README.md"
10
+ license = { file = "LICENSE" }
11
+ requires-python = ">=3.10"
12
+ authors = [
13
+ { name = "Mikhail Golikov", email = "golikovichev@gmail.com" },
14
+ ]
15
+ maintainers = [
16
+ { name = "Mikhail Golikov", email = "golikovichev@gmail.com" },
17
+ ]
18
+ keywords = [
19
+ "kibana",
20
+ "pytest",
21
+ "test-generation",
22
+ "api-testing",
23
+ "log-parsing",
24
+ "qa",
25
+ ]
26
+ classifiers = [
27
+ "Development Status :: 4 - Beta",
28
+ "Intended Audience :: Developers",
29
+ "License :: OSI Approved :: MIT License",
30
+ "Operating System :: OS Independent",
31
+ "Programming Language :: Python :: 3",
32
+ "Programming Language :: Python :: 3.10",
33
+ "Programming Language :: Python :: 3.11",
34
+ "Programming Language :: Python :: 3.12",
35
+ "Topic :: Software Development :: Testing",
36
+ "Topic :: Software Development :: Quality Assurance",
37
+ ]
38
+ dependencies = [
39
+ "pydantic>=2.0",
40
+ "jinja2>=3.1",
41
+ "requests>=2.31",
42
+ ]
43
+
44
+ [project.optional-dependencies]
45
+ dev = [
46
+ "pytest>=7.0",
47
+ "build>=1.0",
48
+ "twine>=4.0",
49
+ ]
50
+
51
+ [project.scripts]
52
+ secure-log2test = "secure_log2test.cli:main"
53
+
54
+ [project.urls]
55
+ Homepage = "https://github.com/golikovichev/secure-log2test"
56
+ Repository = "https://github.com/golikovichev/secure-log2test"
57
+ Issues = "https://github.com/golikovichev/secure-log2test/issues"
58
+ Changelog = "https://github.com/golikovichev/secure-log2test/blob/main/CHANGELOG.md"
59
+
60
+ [tool.hatch.build.targets.wheel]
61
+ packages = ["secure_log2test"]
62
+
63
+ [tool.hatch.build.targets.sdist]
64
+ include = [
65
+ "secure_log2test/",
66
+ "tests/",
67
+ "data/sample_kibana_export.json",
68
+ "README.md",
69
+ "LICENSE",
70
+ "CHANGELOG.md",
71
+ "CONTRIBUTING.md",
72
+ "pyproject.toml",
73
+ ]
74
+
75
+ [tool.pytest.ini_options]
76
+ testpaths = ["tests"]
77
+ addopts = "-ra"
@@ -0,0 +1,9 @@
1
+ """secure-log2test: convert Kibana API log exports into runnable pytest suites."""
2
+
3
+ from importlib.metadata import PackageNotFoundError, version as _pkg_version
4
+
5
+ try:
6
+ __version__ = _pkg_version("secure-log2test")
7
+ except PackageNotFoundError:
8
+ # Running from a source checkout without an installed distribution.
9
+ __version__ = "0.0.0+unknown"
@@ -0,0 +1,6 @@
1
+ """Allow `python -m secure_log2test` invocation."""
2
+
3
+ from .cli import main
4
+
5
+ if __name__ == "__main__":
6
+ raise SystemExit(main())
@@ -0,0 +1,66 @@
1
+ """CLI entry for secure-log2test.
2
+
3
+ Wired through `console_scripts` entry в pyproject.toml so installs expose
4
+ a `secure-log2test` command. Also runnable via `python -m secure_log2test`.
5
+ """
6
+
7
+ from __future__ import annotations
8
+
9
+ import argparse
10
+ import logging
11
+ import sys
12
+ from pathlib import Path
13
+
14
+ from .core.generator import KibanaTestGenerator
15
+ from .core.parser import KibanaLogParser
16
+
17
+
18
+ def main(argv: list[str] | None = None) -> int:
19
+ parser = argparse.ArgumentParser(
20
+ prog="secure-log2test",
21
+ description="Convert Kibana API log export to executable pytest suite",
22
+ )
23
+ parser.add_argument("input", type=Path, help="Path to Kibana JSON export")
24
+ parser.add_argument(
25
+ "--output",
26
+ type=Path,
27
+ default=Path("tests_generated.py"),
28
+ help="Output pytest module (default: tests_generated.py)",
29
+ )
30
+ parser.add_argument(
31
+ "--base-url",
32
+ default="",
33
+ help="Base URL prefix for generated requests (default: empty)",
34
+ )
35
+ parser.add_argument(
36
+ "--templates",
37
+ type=Path,
38
+ default=Path(__file__).parent / "templates",
39
+ help="Templates directory (default: bundled package templates)",
40
+ )
41
+ parser.add_argument("--verbose", action="store_true")
42
+ args = parser.parse_args(argv)
43
+
44
+ logging.basicConfig(
45
+ level=logging.DEBUG if args.verbose else logging.INFO,
46
+ format="%(levelname)s %(message)s",
47
+ )
48
+
49
+ if not args.input.exists():
50
+ print(f"Input file not found: {args.input}", file=sys.stderr)
51
+ return 1
52
+
53
+ log_parser = KibanaLogParser(args.input)
54
+ entries = log_parser.parse()
55
+ if not entries:
56
+ print("No entries parsed from input log.", file=sys.stderr)
57
+ return 1
58
+
59
+ generator = KibanaTestGenerator(args.templates)
60
+ generator.write(entries, args.output, base_url=args.base_url)
61
+ print(f"Generated {len(entries)} tests -> {args.output}")
62
+ return 0
63
+
64
+
65
+ if __name__ == "__main__":
66
+ sys.exit(main())
File without changes
@@ -0,0 +1,46 @@
1
+ import logging
2
+ import re
3
+ from pathlib import Path
4
+
5
+ from jinja2 import Environment, FileSystemLoader, select_autoescape
6
+
7
+ from .parser import KibanaLogEntry
8
+
9
+
10
+ logger = logging.getLogger(__name__)
11
+
12
+
13
+ def _slugify(value):
14
+ value = value.lower()
15
+ value = re.sub(r"[^a-z0-9]+", "_", value)
16
+ value = value.strip("_")
17
+ return value or "endpoint"
18
+
19
+
20
+ class KibanaTestGenerator:
21
+ def __init__(self, templates_dir):
22
+ self.env = Environment(
23
+ loader=FileSystemLoader(str(templates_dir)),
24
+ autoescape=select_autoescape([]),
25
+ trim_blocks=True,
26
+ lstrip_blocks=True,
27
+ )
28
+ self.env.filters["slug"] = _slugify
29
+
30
+ def render(self, entries, base_url=""):
31
+ template = self.env.get_template("test_module.py.j2")
32
+ cleaned = []
33
+ for e in entries:
34
+ if isinstance(e, KibanaLogEntry):
35
+ cleaned.append(e)
36
+ else:
37
+ cleaned.append(KibanaLogEntry(**e))
38
+ return template.render(entries=cleaned, base_url=base_url)
39
+
40
+ def write(self, entries, output_path, base_url=""):
41
+ output_path = Path(output_path)
42
+ output_path.parent.mkdir(parents=True, exist_ok=True)
43
+ rendered = self.render(entries, base_url=base_url)
44
+ output_path.write_text(rendered, encoding="utf-8")
45
+ logger.info("Wrote %d entries to %s", len(entries), output_path)
46
+ return output_path
@@ -0,0 +1,73 @@
1
+ import json
2
+ import logging
3
+ from pathlib import Path
4
+ from typing import Any
5
+
6
+ from pydantic import BaseModel, Field, field_validator
7
+
8
+
9
+ logger = logging.getLogger(__name__)
10
+
11
+
12
+ SENSITIVE_HEADERS = frozenset({
13
+ "authorization",
14
+ "proxy-authorization",
15
+ "cookie",
16
+ "set-cookie",
17
+ "x-api-key",
18
+ "x-auth-token",
19
+ "authentication",
20
+ })
21
+
22
+ REDACTED = "***REDACTED***"
23
+
24
+
25
+ def redact_headers(headers):
26
+ """Replace sensitive header values with a fixed redaction marker.
27
+
28
+ Header name match is case-insensitive. Returns a new dict; the input
29
+ is not mutated.
30
+ """
31
+ if not headers:
32
+ return {}
33
+ return {
34
+ name: (REDACTED if name.lower() in SENSITIVE_HEADERS else value)
35
+ for name, value in headers.items()
36
+ }
37
+
38
+
39
+ class KibanaLogEntry(BaseModel):
40
+ method: str
41
+ url: str
42
+ status: int
43
+ duration: int = 0
44
+ headers: dict[str, str] = Field(default_factory=dict)
45
+ body: Any = None
46
+
47
+ @field_validator("method")
48
+ @classmethod
49
+ def normalize_method(cls, v):
50
+ return v.upper()
51
+
52
+ @field_validator("headers")
53
+ @classmethod
54
+ def redact_sensitive_headers(cls, v):
55
+ return redact_headers(v)
56
+
57
+
58
+ class KibanaLogParser:
59
+ def __init__(self, path):
60
+ self.path = Path(path)
61
+
62
+ def parse(self):
63
+ with open(self.path) as f:
64
+ data = json.load(f)
65
+
66
+ entries = []
67
+ for hit in data.get("hits", {}).get("hits", []):
68
+ try:
69
+ entries.append(KibanaLogEntry(**hit["_source"]))
70
+ except Exception as e:
71
+ logger.warning(f"Skipping bad entry: {e}")
72
+
73
+ return entries
@@ -0,0 +1,20 @@
1
+ """Auto-generated regression suite. Do not edit by hand."""
2
+
3
+ import pytest
4
+ import requests
5
+
6
+
7
+ BASE_URL = "{{ base_url }}"
8
+
9
+
10
+ {% for entry in entries %}
11
+ def test_{{ entry.method | lower }}_{{ entry.url | slug }}_{{ loop.index }}():
12
+ response = requests.request(
13
+ method="{{ entry.method }}",
14
+ url=BASE_URL + "{{ entry.url }}",
15
+ timeout=10,
16
+ )
17
+ assert response.status_code == {{ entry.status }}
18
+
19
+
20
+ {% endfor %}
File without changes
@@ -0,0 +1,152 @@
1
+ """Edge cases: header redaction, empty bodies, server error codes."""
2
+
3
+ import json
4
+
5
+ import pytest
6
+
7
+ from secure_log2test.core.parser import (
8
+ KibanaLogEntry,
9
+ KibanaLogParser,
10
+ REDACTED,
11
+ redact_headers,
12
+ )
13
+
14
+
15
+ # ---------------------------------------------------------------------------
16
+ # Header redaction
17
+ # ---------------------------------------------------------------------------
18
+
19
+ def test_authorization_header_redacted():
20
+ cleaned = redact_headers({"Authorization": "Bearer secret-token-xyz"})
21
+ assert cleaned == {"Authorization": REDACTED}
22
+
23
+
24
+ def test_cookie_header_redacted():
25
+ cleaned = redact_headers({"Cookie": "session=abc123"})
26
+ assert cleaned["Cookie"] == REDACTED
27
+
28
+
29
+ def test_redaction_is_case_insensitive():
30
+ cleaned = redact_headers({
31
+ "AUTHORIZATION": "Bearer x",
32
+ "x-api-key": "k1",
33
+ "X-Auth-Token": "t1",
34
+ })
35
+ assert cleaned["AUTHORIZATION"] == REDACTED
36
+ assert cleaned["x-api-key"] == REDACTED
37
+ assert cleaned["X-Auth-Token"] == REDACTED
38
+
39
+
40
+ def test_safe_headers_preserved():
41
+ headers = {
42
+ "Content-Type": "application/json",
43
+ "Accept": "*/*",
44
+ "User-Agent": "pytest/7",
45
+ }
46
+ assert redact_headers(headers) == headers
47
+
48
+
49
+ def test_redaction_does_not_mutate_input():
50
+ original = {"Authorization": "Bearer x", "Accept": "*/*"}
51
+ snapshot = dict(original)
52
+ redact_headers(original)
53
+ assert original == snapshot
54
+
55
+
56
+ def test_empty_headers_return_empty_dict():
57
+ assert redact_headers({}) == {}
58
+ assert redact_headers(None) == {}
59
+
60
+
61
+ def test_entry_redacts_headers_at_construction():
62
+ entry = KibanaLogEntry(
63
+ method="GET",
64
+ url="/api/v1/users",
65
+ status=200,
66
+ headers={"Authorization": "Bearer leaky", "Accept": "json"},
67
+ )
68
+ assert entry.headers["Authorization"] == REDACTED
69
+ assert entry.headers["Accept"] == "json"
70
+
71
+
72
+ def test_entry_redaction_on_parser_output(tmp_path):
73
+ payload = {
74
+ "hits": {
75
+ "hits": [
76
+ {"_source": {
77
+ "method": "POST",
78
+ "url": "/api/v1/login",
79
+ "status": 200,
80
+ "headers": {"Authorization": "Bearer leaked"},
81
+ }},
82
+ ]
83
+ }
84
+ }
85
+ src = tmp_path / "with_auth.json"
86
+ src.write_text(json.dumps(payload))
87
+ entries = KibanaLogParser(src).parse()
88
+ assert len(entries) == 1
89
+ assert entries[0].headers["Authorization"] == REDACTED
90
+
91
+
92
+ # ---------------------------------------------------------------------------
93
+ # Empty / missing body
94
+ # ---------------------------------------------------------------------------
95
+
96
+ def test_body_defaults_to_none():
97
+ entry = KibanaLogEntry(method="GET", url="/", status=200)
98
+ assert entry.body is None
99
+
100
+
101
+ def test_empty_string_body_preserved():
102
+ entry = KibanaLogEntry(method="POST", url="/api/echo", status=200, body="")
103
+ assert entry.body == ""
104
+
105
+
106
+ def test_empty_dict_body_preserved():
107
+ entry = KibanaLogEntry(method="POST", url="/api/echo", status=200, body={})
108
+ assert entry.body == {}
109
+
110
+
111
+ def test_parser_handles_entries_without_body(tmp_path):
112
+ payload = {
113
+ "hits": {
114
+ "hits": [
115
+ {"_source": {"method": "GET", "url": "/a", "status": 200}},
116
+ {"_source": {"method": "GET", "url": "/b", "status": 200, "body": ""}},
117
+ {"_source": {"method": "GET", "url": "/c", "status": 200, "body": None}},
118
+ ]
119
+ }
120
+ }
121
+ src = tmp_path / "no_body.json"
122
+ src.write_text(json.dumps(payload))
123
+ entries = KibanaLogParser(src).parse()
124
+ assert len(entries) == 3
125
+ assert [e.body for e in entries] == [None, "", None]
126
+
127
+
128
+ # ---------------------------------------------------------------------------
129
+ # Server-error status codes (5xx)
130
+ # ---------------------------------------------------------------------------
131
+
132
+ @pytest.mark.parametrize("code", [500, 502, 503, 504, 599])
133
+ def test_5xx_codes_accepted(code):
134
+ entry = KibanaLogEntry(method="GET", url="/api/health", status=code)
135
+ assert entry.status == code
136
+
137
+
138
+ def test_parser_preserves_5xx_codes(tmp_path):
139
+ payload = {
140
+ "hits": {
141
+ "hits": [
142
+ {"_source": {"method": "GET", "url": "/x", "status": 500}},
143
+ {"_source": {"method": "GET", "url": "/y", "status": 503}},
144
+ {"_source": {"method": "GET", "url": "/z", "status": 504}},
145
+ ]
146
+ }
147
+ }
148
+ src = tmp_path / "errors.json"
149
+ src.write_text(json.dumps(payload))
150
+ entries = KibanaLogParser(src).parse()
151
+ statuses = sorted(e.status for e in entries)
152
+ assert statuses == [500, 503, 504]
@@ -0,0 +1,62 @@
1
+ import json
2
+ from pathlib import Path
3
+
4
+ import pytest
5
+
6
+ from secure_log2test.core.parser import KibanaLogEntry, KibanaLogParser
7
+
8
+
9
+ SAMPLE = Path(__file__).parent.parent / "data" / "sample_kibana_export.json"
10
+
11
+
12
+ def test_parser_reads_sample():
13
+ parser = KibanaLogParser(SAMPLE)
14
+ entries = parser.parse()
15
+ assert len(entries) == 4
16
+
17
+
18
+ def test_method_normalised_to_upper():
19
+ parser = KibanaLogParser(SAMPLE)
20
+ entries = parser.parse()
21
+ methods = {e.method for e in entries}
22
+ assert methods == {"GET", "POST", "PUT", "DELETE"}
23
+
24
+
25
+ def test_status_codes_preserved():
26
+ parser = KibanaLogParser(SAMPLE)
27
+ entries = parser.parse()
28
+ statuses = sorted(e.status for e in entries)
29
+ assert statuses == [200, 201, 204, 404]
30
+
31
+
32
+ def test_skips_malformed_entry(tmp_path):
33
+ bad = tmp_path / "bad.json"
34
+ bad.write_text(json.dumps({
35
+ "hits": {
36
+ "hits": [
37
+ {"_source": {"method": "GET", "url": "/ok", "status": 200}},
38
+ {"_source": {"url": "/missing-method", "status": 200}},
39
+ ]
40
+ }
41
+ }))
42
+ parser = KibanaLogParser(bad)
43
+ entries = parser.parse()
44
+ assert len(entries) == 1
45
+ assert entries[0].url == "/ok"
46
+
47
+
48
+ def test_empty_hits(tmp_path):
49
+ empty = tmp_path / "empty.json"
50
+ empty.write_text(json.dumps({"hits": {"hits": []}}))
51
+ parser = KibanaLogParser(empty)
52
+ assert parser.parse() == []
53
+
54
+
55
+ def test_invalid_method_still_normalised():
56
+ entry = KibanaLogEntry(method="get", url="/", status=200)
57
+ assert entry.method == "GET"
58
+
59
+
60
+ def test_default_duration_zero():
61
+ entry = KibanaLogEntry(method="GET", url="/", status=200)
62
+ assert entry.duration == 0