polyglot-code-guard 0.3.0__tar.gz → 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- polyglot_code_guard-0.5.0/PKG-INFO +185 -0
- polyglot_code_guard-0.5.0/README.md +158 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/pyproject.toml +14 -3
- polyglot_code_guard-0.5.0/src/polyglot_code_guard.egg-info/PKG-INFO +185 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyglot_code_guard.egg-info/SOURCES.txt +2 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyglot_code_guard.egg-info/requires.txt +1 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyguard/SKILL.md +5 -5
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyguard/__init__.py +1 -1
- polyglot_code_guard-0.5.0/src/polyguard/py.typed +0 -0
- polyglot_code_guard-0.5.0/src/polyguard/runner.py +1755 -0
- polyglot_code_guard-0.5.0/tests/test_coverage_paths.py +880 -0
- polyglot_code_guard-0.5.0/tests/test_runner.py +1898 -0
- polyglot_code_guard-0.3.0/PKG-INFO +0 -170
- polyglot_code_guard-0.3.0/README.md +0 -147
- polyglot_code_guard-0.3.0/src/polyglot_code_guard.egg-info/PKG-INFO +0 -170
- polyglot_code_guard-0.3.0/src/polyguard/runner.py +0 -551
- polyglot_code_guard-0.3.0/tests/test_runner.py +0 -277
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/LICENSE +0 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/setup.cfg +0 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyglot_code_guard.egg-info/dependency_links.txt +0 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyglot_code_guard.egg-info/entry_points.txt +0 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyglot_code_guard.egg-info/top_level.txt +0 -0
- {polyglot_code_guard-0.3.0 → polyglot_code_guard-0.5.0}/src/polyguard/cli.py +0 -0
|
@@ -0,0 +1,185 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: polyglot-code-guard
|
|
3
|
+
Version: 0.5.0
|
|
4
|
+
Summary: Fail-closed, multi-language test verification for AI-generated code.
|
|
5
|
+
Author: Furox-Art
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/Furox-Art/polyglot-code-guard
|
|
8
|
+
Project-URL: Repository, https://github.com/Furox-Art/polyglot-code-guard
|
|
9
|
+
Project-URL: Issues, https://github.com/Furox-Art/polyglot-code-guard/issues
|
|
10
|
+
Keywords: testing,code-verification,polyglot,ai-generated-code,quality-gate
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Operating System :: OS Independent
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
18
|
+
Classifier: Topic :: Software Development :: Testing
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE
|
|
22
|
+
Requires-Dist: tomli>=2; python_version < "3.11"
|
|
23
|
+
Provides-Extra: test
|
|
24
|
+
Requires-Dist: pytest>=8.0; extra == "test"
|
|
25
|
+
Requires-Dist: pytest-cov>=6.0; extra == "test"
|
|
26
|
+
Dynamic: license-file
|
|
27
|
+
|
|
28
|
+
# Polyguard
|
|
29
|
+
|
|
30
|
+
**Fail-closed verification for AI-generated code across multiple programming languages.**
|
|
31
|
+
|
|
32
|
+
Polyguard discovers supported project types, checks for a test suite, runs the relevant build/test commands, and returns a one-word verdict by default. A passing command exit code is not enough: Polyguard also requires a positive test-success summary from a recognized runner output format.
|
|
33
|
+
|
|
34
|
+
## Install
|
|
35
|
+
|
|
36
|
+
\`\`\`bash
|
|
37
|
+
python -m pip install --upgrade polyglot-code-guard
|
|
38
|
+
\`\`\`
|
|
39
|
+
|
|
40
|
+
## Run
|
|
41
|
+
|
|
42
|
+
From the target repository:
|
|
43
|
+
|
|
44
|
+
\`\`\`bash
|
|
45
|
+
polyguard verify
|
|
46
|
+
\`\`\`
|
|
47
|
+
|
|
48
|
+
Default output:
|
|
49
|
+
|
|
50
|
+
\`\`\`text
|
|
51
|
+
PASS
|
|
52
|
+
\`\`\`
|
|
53
|
+
|
|
54
|
+
or:
|
|
55
|
+
|
|
56
|
+
\`\`\`text
|
|
57
|
+
FAIL
|
|
58
|
+
\`\`\`
|
|
59
|
+
|
|
60
|
+
Diagnostics and automation:
|
|
61
|
+
|
|
62
|
+
\`\`\`bash
|
|
63
|
+
polyguard verify --details
|
|
64
|
+
polyguard verify --json
|
|
65
|
+
polyguard verify --path ./my-project --timeout 180
|
|
66
|
+
polyguard verify --config ./ci/polyguard.toml
|
|
67
|
+
polyguard --version
|
|
68
|
+
\`\`\`
|
|
69
|
+
|
|
70
|
+
Exit code is **0** for PASS and **1** for FAIL. Missing tests, missing toolchains, invalid configuration, a build/test failure, timeout, or missing positive test-execution evidence returns FAIL. On timeout, Polyguard terminates the command's process group/tree to avoid leaving test-runner descendants behind. Captured output is streamed and limited to the final 256 KiB per command.
|
|
71
|
+
|
|
72
|
+
## Built-in language support
|
|
73
|
+
|
|
74
|
+
| Language | Default build/test plan | Positive execution evidence |
|
|
75
|
+
| --- | --- | --- |
|
|
76
|
+
| Python | \`compileall\`, then \`pytest -q\` | Pytest passed-test summary |
|
|
77
|
+
| Rust | \`cargo test --all-targets\` | Cargo test-result summary |
|
|
78
|
+
| Go | \`go test -json ./...\` | Passing test events in JSON stream |
|
|
79
|
+
| JavaScript / TypeScript | Direct Vitest/Jest/Mocha/TAP CLI when declared, otherwise `node --test --test-reporter=junit <absolute discovered test-file paths>`; TypeScript also runs `tsc --noEmit` when configured | Node fallback: JUnit testcase records cross-checked against a pre-execution snapshot; only that explicit file list runs; missing-name records and file-path collisions fail closed; declared runners: recognized runner summary |
|
|
80
|
+
| Java | Maven `clean test` or Gradle `clean test` | Parsed Surefire/Gradle JUnit XML; at least one pass, zero failures |
|
|
81
|
+
| C# | dotnet test with a Polyguard TRX prefix | Parsed TRX counters; at least one pass, zero failures |
|
|
82
|
+
| PHP | PHPUnit with --log-junit | Parsed JUnit XML; at least one pass, zero failures |
|
|
83
|
+
| Ruby | RSpec, or Ruby's Minitest runner | RSpec/Minitest success summary |
|
|
84
|
+
| Swift | `swift test --parallel` | Recognized Swift Testing/XCTest runner summary with a positive passed-test count |
|
|
85
|
+
| C | CMake build and CTest | Counts per-test `Passed` records; aggregate pass percentages do not prove execution |
|
|
86
|
+
| C++ | CMake build and CTest | Counts per-test `Passed` records; aggregate pass percentages do not prove execution |
|
|
87
|
+
|
|
88
|
+
C and C++ are separate language identifiers in configuration. JavaScript and TypeScript share the same project plan; TypeScript type checking does not replace running tests. The Node built-in test-runner fallback uses its JUnit reporter and requires Node.js 18.15 or later. Polyguard passes only the test-file paths found in its pre-execution snapshot, rather than allowing Node to recursively discover files in generated or pruned directories. TypeScript test files (`.ts`, `.cts`, and `.mts`) additionally require a Node release with built-in type stripping enabled. When a supported runner dependency is declared, Polyguard invokes that runner's CLI and parses its recognized success summary. Project-defined npm `test` scripts are never treated as test-execution evidence.
|
|
89
|
+
|
|
90
|
+
### Monorepos and workspaces
|
|
91
|
+
|
|
92
|
+
Polyguard recursively discovers independent package roots and runs each package in its own working directory. It also recognizes common aggregator manifests and delegates execution to the native workspace command once, avoiding duplicate child-package runs:
|
|
93
|
+
|
|
94
|
+
- Maven `<modules>` and Gradle `settings.gradle(.kts)`
|
|
95
|
+
- Cargo `[workspace].members`
|
|
96
|
+
- npm `workspaces`
|
|
97
|
+
- .NET solution (`.sln`) files
|
|
98
|
+
|
|
99
|
+
Only projects explicitly declared as Maven modules, Gradle includes, Cargo members, npm workspace packages, or .NET solution projects are coalesced under the aggregator root. Unlisted nested projects remain independent units. Each independently discovered project is reported separately. A failure or incomplete test report in any required project makes the overall result FAIL. For workspace aggregators, native package managers remain responsible for workspace dependency resolution and execution.
|
|
100
|
+
|
|
101
|
+
## Configure custom test commands
|
|
102
|
+
|
|
103
|
+
For non-standard project layouts or test runners, create \`.polyguard.toml\` in the project root or pass one with \`--config\`.
|
|
104
|
+
|
|
105
|
+
\`\`\`toml
|
|
106
|
+
[project]
|
|
107
|
+
languages = ["python"]
|
|
108
|
+
|
|
109
|
+
[checks.python]
|
|
110
|
+
commands = [
|
|
111
|
+
["python3", "-m", "compileall", "-q", "src"],
|
|
112
|
+
["python3", "-m", "pytest", "-q"],
|
|
113
|
+
]
|
|
114
|
+
test_patterns = ["tests/test_*.py", "tests/**/*.py"]
|
|
115
|
+
\`\`\`
|
|
116
|
+
|
|
117
|
+
Configuration fields:
|
|
118
|
+
- \`commands\`: non-empty argument arrays. Commands are launched without a shell.
|
|
119
|
+
- \`test_patterns\`: at least one matching file must exist inside the project root.
|
|
120
|
+
|
|
121
|
+
**Evidence boundary:** custom \`success_patterns\` are rejected. An arbitrary output regex can be satisfied by a dummy command and cannot establish that tests executed. Custom commands may execute for diagnostics, but they cannot establish PASS: their stdout and generated reports are not trusted execution evidence. PASS requires the exact built-in command plan for the detected language project. A custom command adapter is not currently supported.
|
|
122
|
+
|
|
123
|
+
## Test evidence and verdict rules
|
|
124
|
+
|
|
125
|
+
A language check returns PASS only when:
|
|
126
|
+
1. The project and required test suite are detected.
|
|
127
|
+
2. Every required command completes successfully.
|
|
128
|
+
3. The exact built-in command plan for the detected project ran; custom commands cannot establish PASS.
|
|
129
|
+
4. A built-in runner summary or a freshly generated native structured test report proves that at least one test passed.
|
|
130
|
+
5. No command failure or timeout occurs.
|
|
131
|
+
|
|
132
|
+
A zero-test run, an all-skipped suite, an output with no recognized success summary, inconsistent report counters, a malformed or stale native report, or a non-zero exit code returns FAIL. Java and PHP require freshly generated JUnit XML; C# requires a fresh TRX report. Swift uses a positive test count from the SwiftPM runner summary because Swift Testing does not produce a usable xUnit report in this workflow. A successful command exit alone is insufficient. C# currently targets the VSTest TRX logger contract; Microsoft.Testing.Platform projects that do not emit this TRX report fail closed until an MTP-specific adapter is configured. JSON and detailed output include passed, failed, skipped, and total counts when the runner report exposes them.
|
|
133
|
+
|
|
134
|
+
## Integration tests
|
|
135
|
+
|
|
136
|
+
The repository includes disposable real-project fixtures for all eleven supported language identifiers, including real Java/JUnit, C#/.NET, PHP/PHPUnit, and SwiftPM/Swift Testing projects. The suite also covers mixed-language and native-workspace monorepos. Run:
|
|
137
|
+
|
|
138
|
+
\`\`\`bash
|
|
139
|
+
python -m pip install -e ".[test]"
|
|
140
|
+
python -m pytest -q
|
|
141
|
+
\`\`\`
|
|
142
|
+
|
|
143
|
+
Integration fixtures are created in temporary directories and run through Polyguard's actual project discovery and command execution path. All required local toolchains must be installed; missing toolchains make the corresponding integration tests fail rather than skip. The repository-wide pytest hook also fails the session if any test is skipped. The native-integration CI job verifies toolchain availability and runs the complete fixture suite. The CI workflow also runs runner tests on Linux, Windows, and macOS, plus a dedicated SwiftPM fixture on macOS.
|
|
144
|
+
|
|
145
|
+
## AI coding-agent skill
|
|
146
|
+
|
|
147
|
+
The repository includes \`SKILL.md\`, also shipped in the installed package. Agents should run:
|
|
148
|
+
|
|
149
|
+
\`\`\`bash
|
|
150
|
+
polyguard verify --path . --details
|
|
151
|
+
\`\`\`
|
|
152
|
+
|
|
153
|
+
Agents must not report successful verification unless the command exits 0 and returns PASS. They must not delete, skip, or weaken tests to manufacture a passing result.
|
|
154
|
+
|
|
155
|
+
## Important limitations and security
|
|
156
|
+
|
|
157
|
+
Polyguard is a test-execution gate, **not a formal proof of program correctness**. PASS means that the built-in command plan succeeded and positive test-pass evidence was recognized. It does not guarantee adequate test coverage, correct requirements, or absence of defects.
|
|
158
|
+
|
|
159
|
+
The verifier executes project build scripts, package managers, and tests. Treat repositories and dependencies as untrusted. Run them in a disposable container or VM without secrets, privileged access, or sensitive host mounts. Polyguard does not itself provide operating-system isolation.
|
|
160
|
+
|
|
161
|
+
Automatic language discovery is conservative. Custom frameworks can be configured for diagnostic execution, but custom commands cannot establish PASS. Only the built-in command plan currently produces trusted execution evidence. Polyguard discovers supported package manifests recursively and executes each discovered language package from its own project root. Multiple packages using the same language are reported separately; the overall result is FAIL if any package fails. A root-level .polyguard.toml applies its language configuration to each matching package, with test patterns evaluated relative to that package root. Workspace-specific dependency orchestration remains the responsibility of the native package manager.
|
|
162
|
+
|
|
163
|
+
## Development
|
|
164
|
+
|
|
165
|
+
The CI unit-test gate requires **100% line and branch coverage** of Polyguard's own Python implementation. This measures Polyguard's code, not the test coverage of projects it verifies.
|
|
166
|
+
|
|
167
|
+
\`\`\`bash
|
|
168
|
+
python -m pip install -e ".[test]"
|
|
169
|
+
python -m pytest -q
|
|
170
|
+
python -m pytest -q --cov=polyguard --cov-branch --cov-report=term-missing --cov-fail-under=100
|
|
171
|
+
python -m build
|
|
172
|
+
python -m twine check dist/*
|
|
173
|
+
\`\`\`
|
|
174
|
+
|
|
175
|
+
## Publishing
|
|
176
|
+
|
|
177
|
+
GitHub Actions builds the distributions and publishes with PyPI Trusted Publishing. Configure the PyPI publisher for:
|
|
178
|
+
- Owner: \`Furox-Art\`
|
|
179
|
+
- Repository: \`polyglot-code-guard\`
|
|
180
|
+
- Workflow: \`publish.yml\`
|
|
181
|
+
- Environment: \`pypi\`
|
|
182
|
+
|
|
183
|
+
## License
|
|
184
|
+
|
|
185
|
+
MIT
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
# Polyguard
|
|
2
|
+
|
|
3
|
+
**Fail-closed verification for AI-generated code across multiple programming languages.**
|
|
4
|
+
|
|
5
|
+
Polyguard discovers supported project types, checks for a test suite, runs the relevant build/test commands, and returns a one-word verdict by default. A passing command exit code is not enough: Polyguard also requires a positive test-success summary from a recognized runner output format.
|
|
6
|
+
|
|
7
|
+
## Install
|
|
8
|
+
|
|
9
|
+
\`\`\`bash
|
|
10
|
+
python -m pip install --upgrade polyglot-code-guard
|
|
11
|
+
\`\`\`
|
|
12
|
+
|
|
13
|
+
## Run
|
|
14
|
+
|
|
15
|
+
From the target repository:
|
|
16
|
+
|
|
17
|
+
\`\`\`bash
|
|
18
|
+
polyguard verify
|
|
19
|
+
\`\`\`
|
|
20
|
+
|
|
21
|
+
Default output:
|
|
22
|
+
|
|
23
|
+
\`\`\`text
|
|
24
|
+
PASS
|
|
25
|
+
\`\`\`
|
|
26
|
+
|
|
27
|
+
or:
|
|
28
|
+
|
|
29
|
+
\`\`\`text
|
|
30
|
+
FAIL
|
|
31
|
+
\`\`\`
|
|
32
|
+
|
|
33
|
+
Diagnostics and automation:
|
|
34
|
+
|
|
35
|
+
\`\`\`bash
|
|
36
|
+
polyguard verify --details
|
|
37
|
+
polyguard verify --json
|
|
38
|
+
polyguard verify --path ./my-project --timeout 180
|
|
39
|
+
polyguard verify --config ./ci/polyguard.toml
|
|
40
|
+
polyguard --version
|
|
41
|
+
\`\`\`
|
|
42
|
+
|
|
43
|
+
Exit code is **0** for PASS and **1** for FAIL. Missing tests, missing toolchains, invalid configuration, a build/test failure, timeout, or missing positive test-execution evidence returns FAIL. On timeout, Polyguard terminates the command's process group/tree to avoid leaving test-runner descendants behind. Captured output is streamed and limited to the final 256 KiB per command.
|
|
44
|
+
|
|
45
|
+
## Built-in language support
|
|
46
|
+
|
|
47
|
+
| Language | Default build/test plan | Positive execution evidence |
|
|
48
|
+
| --- | --- | --- |
|
|
49
|
+
| Python | \`compileall\`, then \`pytest -q\` | Pytest passed-test summary |
|
|
50
|
+
| Rust | \`cargo test --all-targets\` | Cargo test-result summary |
|
|
51
|
+
| Go | \`go test -json ./...\` | Passing test events in JSON stream |
|
|
52
|
+
| JavaScript / TypeScript | Direct Vitest/Jest/Mocha/TAP CLI when declared, otherwise `node --test --test-reporter=junit <absolute discovered test-file paths>`; TypeScript also runs `tsc --noEmit` when configured | Node fallback: JUnit testcase records cross-checked against a pre-execution snapshot; only that explicit file list runs; missing-name records and file-path collisions fail closed; declared runners: recognized runner summary |
|
|
53
|
+
| Java | Maven `clean test` or Gradle `clean test` | Parsed Surefire/Gradle JUnit XML; at least one pass, zero failures |
|
|
54
|
+
| C# | dotnet test with a Polyguard TRX prefix | Parsed TRX counters; at least one pass, zero failures |
|
|
55
|
+
| PHP | PHPUnit with --log-junit | Parsed JUnit XML; at least one pass, zero failures |
|
|
56
|
+
| Ruby | RSpec, or Ruby's Minitest runner | RSpec/Minitest success summary |
|
|
57
|
+
| Swift | `swift test --parallel` | Recognized Swift Testing/XCTest runner summary with a positive passed-test count |
|
|
58
|
+
| C | CMake build and CTest | Counts per-test `Passed` records; aggregate pass percentages do not prove execution |
|
|
59
|
+
| C++ | CMake build and CTest | Counts per-test `Passed` records; aggregate pass percentages do not prove execution |
|
|
60
|
+
|
|
61
|
+
C and C++ are separate language identifiers in configuration. JavaScript and TypeScript share the same project plan; TypeScript type checking does not replace running tests. The Node built-in test-runner fallback uses its JUnit reporter and requires Node.js 18.15 or later. Polyguard passes only the test-file paths found in its pre-execution snapshot, rather than allowing Node to recursively discover files in generated or pruned directories. TypeScript test files (`.ts`, `.cts`, and `.mts`) additionally require a Node release with built-in type stripping enabled. When a supported runner dependency is declared, Polyguard invokes that runner's CLI and parses its recognized success summary. Project-defined npm `test` scripts are never treated as test-execution evidence.
|
|
62
|
+
|
|
63
|
+
### Monorepos and workspaces
|
|
64
|
+
|
|
65
|
+
Polyguard recursively discovers independent package roots and runs each package in its own working directory. It also recognizes common aggregator manifests and delegates execution to the native workspace command once, avoiding duplicate child-package runs:
|
|
66
|
+
|
|
67
|
+
- Maven `<modules>` and Gradle `settings.gradle(.kts)`
|
|
68
|
+
- Cargo `[workspace].members`
|
|
69
|
+
- npm `workspaces`
|
|
70
|
+
- .NET solution (`.sln`) files
|
|
71
|
+
|
|
72
|
+
Only projects explicitly declared as Maven modules, Gradle includes, Cargo members, npm workspace packages, or .NET solution projects are coalesced under the aggregator root. Unlisted nested projects remain independent units. Each independently discovered project is reported separately. A failure or incomplete test report in any required project makes the overall result FAIL. For workspace aggregators, native package managers remain responsible for workspace dependency resolution and execution.
|
|
73
|
+
|
|
74
|
+
## Configure custom test commands
|
|
75
|
+
|
|
76
|
+
For non-standard project layouts or test runners, create \`.polyguard.toml\` in the project root or pass one with \`--config\`.
|
|
77
|
+
|
|
78
|
+
\`\`\`toml
|
|
79
|
+
[project]
|
|
80
|
+
languages = ["python"]
|
|
81
|
+
|
|
82
|
+
[checks.python]
|
|
83
|
+
commands = [
|
|
84
|
+
["python3", "-m", "compileall", "-q", "src"],
|
|
85
|
+
["python3", "-m", "pytest", "-q"],
|
|
86
|
+
]
|
|
87
|
+
test_patterns = ["tests/test_*.py", "tests/**/*.py"]
|
|
88
|
+
\`\`\`
|
|
89
|
+
|
|
90
|
+
Configuration fields:
|
|
91
|
+
- \`commands\`: non-empty argument arrays. Commands are launched without a shell.
|
|
92
|
+
- \`test_patterns\`: at least one matching file must exist inside the project root.
|
|
93
|
+
|
|
94
|
+
**Evidence boundary:** custom \`success_patterns\` are rejected. An arbitrary output regex can be satisfied by a dummy command and cannot establish that tests executed. Custom commands may execute for diagnostics, but they cannot establish PASS: their stdout and generated reports are not trusted execution evidence. PASS requires the exact built-in command plan for the detected language project. A custom command adapter is not currently supported.
|
|
95
|
+
|
|
96
|
+
## Test evidence and verdict rules
|
|
97
|
+
|
|
98
|
+
A language check returns PASS only when:
|
|
99
|
+
1. The project and required test suite are detected.
|
|
100
|
+
2. Every required command completes successfully.
|
|
101
|
+
3. The exact built-in command plan for the detected project ran; custom commands cannot establish PASS.
|
|
102
|
+
4. A built-in runner summary or a freshly generated native structured test report proves that at least one test passed.
|
|
103
|
+
5. No command failure or timeout occurs.
|
|
104
|
+
|
|
105
|
+
A zero-test run, an all-skipped suite, an output with no recognized success summary, inconsistent report counters, a malformed or stale native report, or a non-zero exit code returns FAIL. Java and PHP require freshly generated JUnit XML; C# requires a fresh TRX report. Swift uses a positive test count from the SwiftPM runner summary because Swift Testing does not produce a usable xUnit report in this workflow. A successful command exit alone is insufficient. C# currently targets the VSTest TRX logger contract; Microsoft.Testing.Platform projects that do not emit this TRX report fail closed until an MTP-specific adapter is configured. JSON and detailed output include passed, failed, skipped, and total counts when the runner report exposes them.
|
|
106
|
+
|
|
107
|
+
## Integration tests
|
|
108
|
+
|
|
109
|
+
The repository includes disposable real-project fixtures for all eleven supported language identifiers, including real Java/JUnit, C#/.NET, PHP/PHPUnit, and SwiftPM/Swift Testing projects. The suite also covers mixed-language and native-workspace monorepos. Run:
|
|
110
|
+
|
|
111
|
+
\`\`\`bash
|
|
112
|
+
python -m pip install -e ".[test]"
|
|
113
|
+
python -m pytest -q
|
|
114
|
+
\`\`\`
|
|
115
|
+
|
|
116
|
+
Integration fixtures are created in temporary directories and run through Polyguard's actual project discovery and command execution path. All required local toolchains must be installed; missing toolchains make the corresponding integration tests fail rather than skip. The repository-wide pytest hook also fails the session if any test is skipped. The native-integration CI job verifies toolchain availability and runs the complete fixture suite. The CI workflow also runs runner tests on Linux, Windows, and macOS, plus a dedicated SwiftPM fixture on macOS.
|
|
117
|
+
|
|
118
|
+
## AI coding-agent skill
|
|
119
|
+
|
|
120
|
+
The repository includes \`SKILL.md\`, also shipped in the installed package. Agents should run:
|
|
121
|
+
|
|
122
|
+
\`\`\`bash
|
|
123
|
+
polyguard verify --path . --details
|
|
124
|
+
\`\`\`
|
|
125
|
+
|
|
126
|
+
Agents must not report successful verification unless the command exits 0 and returns PASS. They must not delete, skip, or weaken tests to manufacture a passing result.
|
|
127
|
+
|
|
128
|
+
## Important limitations and security
|
|
129
|
+
|
|
130
|
+
Polyguard is a test-execution gate, **not a formal proof of program correctness**. PASS means that the built-in command plan succeeded and positive test-pass evidence was recognized. It does not guarantee adequate test coverage, correct requirements, or absence of defects.
|
|
131
|
+
|
|
132
|
+
The verifier executes project build scripts, package managers, and tests. Treat repositories and dependencies as untrusted. Run them in a disposable container or VM without secrets, privileged access, or sensitive host mounts. Polyguard does not itself provide operating-system isolation.
|
|
133
|
+
|
|
134
|
+
Automatic language discovery is conservative. Custom frameworks can be configured for diagnostic execution, but custom commands cannot establish PASS. Only the built-in command plan currently produces trusted execution evidence. Polyguard discovers supported package manifests recursively and executes each discovered language package from its own project root. Multiple packages using the same language are reported separately; the overall result is FAIL if any package fails. A root-level .polyguard.toml applies its language configuration to each matching package, with test patterns evaluated relative to that package root. Workspace-specific dependency orchestration remains the responsibility of the native package manager.
|
|
135
|
+
|
|
136
|
+
## Development
|
|
137
|
+
|
|
138
|
+
The CI unit-test gate requires **100% line and branch coverage** of Polyguard's own Python implementation. This measures Polyguard's code, not the test coverage of projects it verifies.
|
|
139
|
+
|
|
140
|
+
\`\`\`bash
|
|
141
|
+
python -m pip install -e ".[test]"
|
|
142
|
+
python -m pytest -q
|
|
143
|
+
python -m pytest -q --cov=polyguard --cov-branch --cov-report=term-missing --cov-fail-under=100
|
|
144
|
+
python -m build
|
|
145
|
+
python -m twine check dist/*
|
|
146
|
+
\`\`\`
|
|
147
|
+
|
|
148
|
+
## Publishing
|
|
149
|
+
|
|
150
|
+
GitHub Actions builds the distributions and publishes with PyPI Trusted Publishing. Configure the PyPI publisher for:
|
|
151
|
+
- Owner: \`Furox-Art\`
|
|
152
|
+
- Repository: \`polyglot-code-guard\`
|
|
153
|
+
- Workflow: \`publish.yml\`
|
|
154
|
+
- Environment: \`pypi\`
|
|
155
|
+
|
|
156
|
+
## License
|
|
157
|
+
|
|
158
|
+
MIT
|
|
@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "polyglot-code-guard"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.5.0"
|
|
8
8
|
description = "Fail-closed, multi-language test verification for AI-generated code."
|
|
9
9
|
readme = "README.md"
|
|
10
|
-
requires-python = ">=3.
|
|
10
|
+
requires-python = ">=3.11"
|
|
11
11
|
license = "MIT"
|
|
12
12
|
authors = [{ name = "Furox-Art" }]
|
|
13
13
|
keywords = ["testing", "code-verification", "polyglot", "ai-generated-code", "quality-gate"]
|
|
@@ -16,12 +16,15 @@ classifiers = [
|
|
|
16
16
|
"Intended Audience :: Developers",
|
|
17
17
|
"Operating System :: OS Independent",
|
|
18
18
|
"Programming Language :: Python :: 3",
|
|
19
|
+
"Programming Language :: Python :: 3.11",
|
|
20
|
+
"Programming Language :: Python :: 3.12",
|
|
21
|
+
"Programming Language :: Python :: 3.13",
|
|
19
22
|
"Topic :: Software Development :: Testing",
|
|
20
23
|
]
|
|
21
24
|
dependencies = ["tomli>=2; python_version < '3.11'"]
|
|
22
25
|
|
|
23
26
|
[project.optional-dependencies]
|
|
24
|
-
test = ["pytest>=8.0"]
|
|
27
|
+
test = ["pytest>=8.0", "pytest-cov>=6.0"]
|
|
25
28
|
|
|
26
29
|
[project.scripts]
|
|
27
30
|
polyguard = "polyguard.cli:main"
|
|
@@ -44,3 +47,11 @@ polyguard = ["SKILL.md"]
|
|
|
44
47
|
[tool.pytest.ini_options]
|
|
45
48
|
testpaths = ["tests"]
|
|
46
49
|
addopts = "-ra"
|
|
50
|
+
|
|
51
|
+
[tool.coverage.run]
|
|
52
|
+
branch = true
|
|
53
|
+
source = ["polyguard"]
|
|
54
|
+
|
|
55
|
+
[tool.coverage.report]
|
|
56
|
+
fail_under = 100
|
|
57
|
+
show_missing = true
|
|
@@ -0,0 +1,185 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: polyglot-code-guard
|
|
3
|
+
Version: 0.5.0
|
|
4
|
+
Summary: Fail-closed, multi-language test verification for AI-generated code.
|
|
5
|
+
Author: Furox-Art
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/Furox-Art/polyglot-code-guard
|
|
8
|
+
Project-URL: Repository, https://github.com/Furox-Art/polyglot-code-guard
|
|
9
|
+
Project-URL: Issues, https://github.com/Furox-Art/polyglot-code-guard/issues
|
|
10
|
+
Keywords: testing,code-verification,polyglot,ai-generated-code,quality-gate
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Operating System :: OS Independent
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
18
|
+
Classifier: Topic :: Software Development :: Testing
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE
|
|
22
|
+
Requires-Dist: tomli>=2; python_version < "3.11"
|
|
23
|
+
Provides-Extra: test
|
|
24
|
+
Requires-Dist: pytest>=8.0; extra == "test"
|
|
25
|
+
Requires-Dist: pytest-cov>=6.0; extra == "test"
|
|
26
|
+
Dynamic: license-file
|
|
27
|
+
|
|
28
|
+
# Polyguard
|
|
29
|
+
|
|
30
|
+
**Fail-closed verification for AI-generated code across multiple programming languages.**
|
|
31
|
+
|
|
32
|
+
Polyguard discovers supported project types, checks for a test suite, runs the relevant build/test commands, and returns a one-word verdict by default. A passing command exit code is not enough: Polyguard also requires a positive test-success summary from a recognized runner output format.
|
|
33
|
+
|
|
34
|
+
## Install
|
|
35
|
+
|
|
36
|
+
\`\`\`bash
|
|
37
|
+
python -m pip install --upgrade polyglot-code-guard
|
|
38
|
+
\`\`\`
|
|
39
|
+
|
|
40
|
+
## Run
|
|
41
|
+
|
|
42
|
+
From the target repository:
|
|
43
|
+
|
|
44
|
+
\`\`\`bash
|
|
45
|
+
polyguard verify
|
|
46
|
+
\`\`\`
|
|
47
|
+
|
|
48
|
+
Default output:
|
|
49
|
+
|
|
50
|
+
\`\`\`text
|
|
51
|
+
PASS
|
|
52
|
+
\`\`\`
|
|
53
|
+
|
|
54
|
+
or:
|
|
55
|
+
|
|
56
|
+
\`\`\`text
|
|
57
|
+
FAIL
|
|
58
|
+
\`\`\`
|
|
59
|
+
|
|
60
|
+
Diagnostics and automation:
|
|
61
|
+
|
|
62
|
+
\`\`\`bash
|
|
63
|
+
polyguard verify --details
|
|
64
|
+
polyguard verify --json
|
|
65
|
+
polyguard verify --path ./my-project --timeout 180
|
|
66
|
+
polyguard verify --config ./ci/polyguard.toml
|
|
67
|
+
polyguard --version
|
|
68
|
+
\`\`\`
|
|
69
|
+
|
|
70
|
+
Exit code is **0** for PASS and **1** for FAIL. Missing tests, missing toolchains, invalid configuration, a build/test failure, timeout, or missing positive test-execution evidence returns FAIL. On timeout, Polyguard terminates the command's process group/tree to avoid leaving test-runner descendants behind. Captured output is streamed and limited to the final 256 KiB per command.
|
|
71
|
+
|
|
72
|
+
## Built-in language support
|
|
73
|
+
|
|
74
|
+
| Language | Default build/test plan | Positive execution evidence |
|
|
75
|
+
| --- | --- | --- |
|
|
76
|
+
| Python | \`compileall\`, then \`pytest -q\` | Pytest passed-test summary |
|
|
77
|
+
| Rust | \`cargo test --all-targets\` | Cargo test-result summary |
|
|
78
|
+
| Go | \`go test -json ./...\` | Passing test events in JSON stream |
|
|
79
|
+
| JavaScript / TypeScript | Direct Vitest/Jest/Mocha/TAP CLI when declared, otherwise `node --test --test-reporter=junit <absolute discovered test-file paths>`; TypeScript also runs `tsc --noEmit` when configured | Node fallback: JUnit testcase records cross-checked against a pre-execution snapshot; only that explicit file list runs; missing-name records and file-path collisions fail closed; declared runners: recognized runner summary |
|
|
80
|
+
| Java | Maven `clean test` or Gradle `clean test` | Parsed Surefire/Gradle JUnit XML; at least one pass, zero failures |
|
|
81
|
+
| C# | dotnet test with a Polyguard TRX prefix | Parsed TRX counters; at least one pass, zero failures |
|
|
82
|
+
| PHP | PHPUnit with --log-junit | Parsed JUnit XML; at least one pass, zero failures |
|
|
83
|
+
| Ruby | RSpec, or Ruby's Minitest runner | RSpec/Minitest success summary |
|
|
84
|
+
| Swift | `swift test --parallel` | Recognized Swift Testing/XCTest runner summary with a positive passed-test count |
|
|
85
|
+
| C | CMake build and CTest | Counts per-test `Passed` records; aggregate pass percentages do not prove execution |
|
|
86
|
+
| C++ | CMake build and CTest | Counts per-test `Passed` records; aggregate pass percentages do not prove execution |
|
|
87
|
+
|
|
88
|
+
C and C++ are separate language identifiers in configuration. JavaScript and TypeScript share the same project plan; TypeScript type checking does not replace running tests. The Node built-in test-runner fallback uses its JUnit reporter and requires Node.js 18.15 or later. Polyguard passes only the test-file paths found in its pre-execution snapshot, rather than allowing Node to recursively discover files in generated or pruned directories. TypeScript test files (`.ts`, `.cts`, and `.mts`) additionally require a Node release with built-in type stripping enabled. When a supported runner dependency is declared, Polyguard invokes that runner's CLI and parses its recognized success summary. Project-defined npm `test` scripts are never treated as test-execution evidence.
|
|
89
|
+
|
|
90
|
+
### Monorepos and workspaces
|
|
91
|
+
|
|
92
|
+
Polyguard recursively discovers independent package roots and runs each package in its own working directory. It also recognizes common aggregator manifests and delegates execution to the native workspace command once, avoiding duplicate child-package runs:
|
|
93
|
+
|
|
94
|
+
- Maven `<modules>` and Gradle `settings.gradle(.kts)`
|
|
95
|
+
- Cargo `[workspace].members`
|
|
96
|
+
- npm `workspaces`
|
|
97
|
+
- .NET solution (`.sln`) files
|
|
98
|
+
|
|
99
|
+
Only projects explicitly declared as Maven modules, Gradle includes, Cargo members, npm workspace packages, or .NET solution projects are coalesced under the aggregator root. Unlisted nested projects remain independent units. Each independently discovered project is reported separately. A failure or incomplete test report in any required project makes the overall result FAIL. For workspace aggregators, native package managers remain responsible for workspace dependency resolution and execution.
|
|
100
|
+
|
|
101
|
+
## Configure custom test commands
|
|
102
|
+
|
|
103
|
+
For non-standard project layouts or test runners, create \`.polyguard.toml\` in the project root or pass one with \`--config\`.
|
|
104
|
+
|
|
105
|
+
\`\`\`toml
|
|
106
|
+
[project]
|
|
107
|
+
languages = ["python"]
|
|
108
|
+
|
|
109
|
+
[checks.python]
|
|
110
|
+
commands = [
|
|
111
|
+
["python3", "-m", "compileall", "-q", "src"],
|
|
112
|
+
["python3", "-m", "pytest", "-q"],
|
|
113
|
+
]
|
|
114
|
+
test_patterns = ["tests/test_*.py", "tests/**/*.py"]
|
|
115
|
+
\`\`\`
|
|
116
|
+
|
|
117
|
+
Configuration fields:
|
|
118
|
+
- \`commands\`: non-empty argument arrays. Commands are launched without a shell.
|
|
119
|
+
- \`test_patterns\`: at least one matching file must exist inside the project root.
|
|
120
|
+
|
|
121
|
+
**Evidence boundary:** custom \`success_patterns\` are rejected. An arbitrary output regex can be satisfied by a dummy command and cannot establish that tests executed. Custom commands may execute for diagnostics, but they cannot establish PASS: their stdout and generated reports are not trusted execution evidence. PASS requires the exact built-in command plan for the detected language project. A custom command adapter is not currently supported.
|
|
122
|
+
|
|
123
|
+
## Test evidence and verdict rules
|
|
124
|
+
|
|
125
|
+
A language check returns PASS only when:
|
|
126
|
+
1. The project and required test suite are detected.
|
|
127
|
+
2. Every required command completes successfully.
|
|
128
|
+
3. The exact built-in command plan for the detected project ran; custom commands cannot establish PASS.
|
|
129
|
+
4. A built-in runner summary or a freshly generated native structured test report proves that at least one test passed.
|
|
130
|
+
5. No command failure or timeout occurs.
|
|
131
|
+
|
|
132
|
+
A zero-test run, an all-skipped suite, an output with no recognized success summary, inconsistent report counters, a malformed or stale native report, or a non-zero exit code returns FAIL. Java and PHP require freshly generated JUnit XML; C# requires a fresh TRX report. Swift uses a positive test count from the SwiftPM runner summary because Swift Testing does not produce a usable xUnit report in this workflow. A successful command exit alone is insufficient. C# currently targets the VSTest TRX logger contract; Microsoft.Testing.Platform projects that do not emit this TRX report fail closed until an MTP-specific adapter is configured. JSON and detailed output include passed, failed, skipped, and total counts when the runner report exposes them.
|
|
133
|
+
|
|
134
|
+
## Integration tests
|
|
135
|
+
|
|
136
|
+
The repository includes disposable real-project fixtures for all eleven supported language identifiers, including real Java/JUnit, C#/.NET, PHP/PHPUnit, and SwiftPM/Swift Testing projects. The suite also covers mixed-language and native-workspace monorepos. Run:
|
|
137
|
+
|
|
138
|
+
\`\`\`bash
|
|
139
|
+
python -m pip install -e ".[test]"
|
|
140
|
+
python -m pytest -q
|
|
141
|
+
\`\`\`
|
|
142
|
+
|
|
143
|
+
Integration fixtures are created in temporary directories and run through Polyguard's actual project discovery and command execution path. All required local toolchains must be installed; missing toolchains make the corresponding integration tests fail rather than skip. The repository-wide pytest hook also fails the session if any test is skipped. The native-integration CI job verifies toolchain availability and runs the complete fixture suite. The CI workflow also runs runner tests on Linux, Windows, and macOS, plus a dedicated SwiftPM fixture on macOS.
|
|
144
|
+
|
|
145
|
+
## AI coding-agent skill
|
|
146
|
+
|
|
147
|
+
The repository includes \`SKILL.md\`, also shipped in the installed package. Agents should run:
|
|
148
|
+
|
|
149
|
+
\`\`\`bash
|
|
150
|
+
polyguard verify --path . --details
|
|
151
|
+
\`\`\`
|
|
152
|
+
|
|
153
|
+
Agents must not report successful verification unless the command exits 0 and returns PASS. They must not delete, skip, or weaken tests to manufacture a passing result.
|
|
154
|
+
|
|
155
|
+
## Important limitations and security
|
|
156
|
+
|
|
157
|
+
Polyguard is a test-execution gate, **not a formal proof of program correctness**. PASS means that the built-in command plan succeeded and positive test-pass evidence was recognized. It does not guarantee adequate test coverage, correct requirements, or absence of defects.
|
|
158
|
+
|
|
159
|
+
The verifier executes project build scripts, package managers, and tests. Treat repositories and dependencies as untrusted. Run them in a disposable container or VM without secrets, privileged access, or sensitive host mounts. Polyguard does not itself provide operating-system isolation.
|
|
160
|
+
|
|
161
|
+
Automatic language discovery is conservative. Custom frameworks can be configured for diagnostic execution, but custom commands cannot establish PASS. Only the built-in command plan currently produces trusted execution evidence. Polyguard discovers supported package manifests recursively and executes each discovered language package from its own project root. Multiple packages using the same language are reported separately; the overall result is FAIL if any package fails. A root-level .polyguard.toml applies its language configuration to each matching package, with test patterns evaluated relative to that package root. Workspace-specific dependency orchestration remains the responsibility of the native package manager.
|
|
162
|
+
|
|
163
|
+
## Development
|
|
164
|
+
|
|
165
|
+
The CI unit-test gate requires **100% line and branch coverage** of Polyguard's own Python implementation. This measures Polyguard's code, not the test coverage of projects it verifies.
|
|
166
|
+
|
|
167
|
+
\`\`\`bash
|
|
168
|
+
python -m pip install -e ".[test]"
|
|
169
|
+
python -m pytest -q
|
|
170
|
+
python -m pytest -q --cov=polyguard --cov-branch --cov-report=term-missing --cov-fail-under=100
|
|
171
|
+
python -m build
|
|
172
|
+
python -m twine check dist/*
|
|
173
|
+
\`\`\`
|
|
174
|
+
|
|
175
|
+
## Publishing
|
|
176
|
+
|
|
177
|
+
GitHub Actions builds the distributions and publishes with PyPI Trusted Publishing. Configure the PyPI publisher for:
|
|
178
|
+
- Owner: \`Furox-Art\`
|
|
179
|
+
- Repository: \`polyglot-code-guard\`
|
|
180
|
+
- Workflow: \`publish.yml\`
|
|
181
|
+
- Environment: \`pypi\`
|
|
182
|
+
|
|
183
|
+
## License
|
|
184
|
+
|
|
185
|
+
MIT
|
|
@@ -12,10 +12,10 @@ Use this skill whenever code is generated, changed, refactored, or described as
|
|
|
12
12
|
1. Identify every language, build system, package root, and test framework touched by the change.
|
|
13
13
|
2. Add requirement-based tests for normal behavior, edge cases, invalid inputs, error paths, and regressions.
|
|
14
14
|
3. Run \`polyguard verify --path . --details\`.
|
|
15
|
-
4. For a non-standard test runner,
|
|
15
|
+
4. For a non-standard test runner, explicit commands and file patterns can be configured for diagnostics; custom commands cannot establish PASS until a trusted runner adapter is implemented.
|
|
16
16
|
5. Run the complete configured test suite. Do not omit a detected or required language.
|
|
17
|
-
6. Require a positive passed-test count in
|
|
18
|
-
7. If a command fails, a toolchain is missing, tests cannot be found, the runner reports no passing tests, execution times out, or output evidence is missing, the verdict is FAIL.
|
|
17
|
+
6. Require a positive passed-test count from the exact built-in command plan. Java and PHP default checks must produce fresh JUnit XML; C# must produce a fresh TRX report. Swift must emit a positive native test-runner summary. A successful command exit code alone is not sufficient.
|
|
18
|
+
7. If a command fails, a toolchain is missing, tests cannot be found, the runner reports no passing tests, execution times out (the command process tree is terminated), or output evidence is missing, the verdict is FAIL.
|
|
19
19
|
8. Fix the implementation or tests and rerun all checks. Never delete, skip, weaken, or rewrite a valid test merely to obtain PASS.
|
|
20
20
|
9. Retain command output and the JSON report when inspectable evidence is requested.
|
|
21
21
|
|
|
@@ -28,11 +28,11 @@ The CLI exits 0 only for PASS and 1 for FAIL. Default output is exactly one word
|
|
|
28
28
|
|
|
29
29
|
## Custom command trust
|
|
30
30
|
|
|
31
|
-
|
|
31
|
+
Custom \`success_patterns\` are rejected. Custom commands may execute for diagnostics, but their output and generated reports cannot establish PASS because a command can forge them. Only the exact built-in command plan for the detected project is accepted as execution evidence. Java, C#, and PHP additionally require fresh native reports; Swift uses its runner summary.
|
|
32
32
|
|
|
33
33
|
## Scope
|
|
34
34
|
|
|
35
|
-
Built-in support: Python, Rust, Go, JavaScript/TypeScript, Java, C#, PHP, Ruby, Swift, C, and C++ with CMake/CTest. Use explicit configuration for unconventional project layouts and custom runners.
|
|
35
|
+
Built-in support: Python, Rust, Go, JavaScript/TypeScript, Java, C#, PHP, Ruby, Swift, C, and C++ with CMake/CTest. Use explicit configuration for unconventional project layouts and custom runners. Polyguard discovers supported package manifests recursively, executes each language package from its own root, and reports multiple packages of the same language separately. A root-level configuration is applied to each matching package with test patterns evaluated relative to that package root. Maven modules, Gradle settings builds, Cargo workspaces, npm workspaces, and .NET solutions are delegated to their native root command once to avoid duplicate child-package execution.
|
|
36
36
|
|
|
37
37
|
## Security
|
|
38
38
|
|
|
File without changes
|