api-tests-coverage 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +703 -0
- package/config.yaml.example +227 -0
- package/dist/action/src/index.d.ts +2 -0
- package/dist/action/src/index.d.ts.map +1 -0
- package/dist/action/src/index.js +349 -0
- package/dist/action/src/prComment.d.ts +34 -0
- package/dist/action/src/prComment.d.ts.map +1 -0
- package/dist/action/src/prComment.js +146 -0
- package/dist/src/ast/astAnalysisOrchestrator.d.ts +36 -0
- package/dist/src/ast/astAnalysisOrchestrator.d.ts.map +1 -0
- package/dist/src/ast/astAnalysisOrchestrator.js +123 -0
- package/dist/src/ast/astTypes.d.ts +105 -0
- package/dist/src/ast/astTypes.d.ts.map +1 -0
- package/dist/src/ast/astTypes.js +9 -0
- package/dist/src/ast/languageAnalyzer.d.ts +46 -0
- package/dist/src/ast/languageAnalyzer.d.ts.map +1 -0
- package/dist/src/ast/languageAnalyzer.js +9 -0
- package/dist/src/ast/languageCapabilities.d.ts +24 -0
- package/dist/src/ast/languageCapabilities.d.ts.map +1 -0
- package/dist/src/ast/languageCapabilities.js +92 -0
- package/dist/src/ast/parseFile.d.ts +16 -0
- package/dist/src/ast/parseFile.d.ts.map +1 -0
- package/dist/src/ast/parseFile.js +65 -0
- package/dist/src/ast/parserRegistry.d.ts +39 -0
- package/dist/src/ast/parserRegistry.d.ts.map +1 -0
- package/dist/src/ast/parserRegistry.js +66 -0
- package/dist/src/buildSummary.d.ts +26 -0
- package/dist/src/buildSummary.d.ts.map +1 -0
- package/dist/src/buildSummary.js +193 -0
- package/dist/src/businessCoverage.d.ts +68 -0
- package/dist/src/businessCoverage.d.ts.map +1 -0
- package/dist/src/businessCoverage.js +290 -0
- package/dist/src/compatibilityCoverage.d.ts +83 -0
- package/dist/src/compatibilityCoverage.d.ts.map +1 -0
- package/dist/src/compatibilityCoverage.js +501 -0
- package/dist/src/config/defaultConfig.d.ts +9 -0
- package/dist/src/config/defaultConfig.d.ts.map +1 -0
- package/dist/src/config/defaultConfig.js +97 -0
- package/dist/src/config/index.d.ts +12 -0
- package/dist/src/config/index.d.ts.map +1 -0
- package/dist/src/config/index.js +37 -0
- package/dist/src/config/loadConfig.d.ts +29 -0
- package/dist/src/config/loadConfig.d.ts.map +1 -0
- package/dist/src/config/loadConfig.js +135 -0
- package/dist/src/config/mergeConfig.d.ts +15 -0
- package/dist/src/config/mergeConfig.d.ts.map +1 -0
- package/dist/src/config/mergeConfig.js +57 -0
- package/dist/src/config/schema.d.ts +15 -0
- package/dist/src/config/schema.d.ts.map +1 -0
- package/dist/src/config/schema.js +30 -0
- package/dist/src/config/types.d.ts +175 -0
- package/dist/src/config/types.d.ts.map +1 -0
- package/dist/src/config/types.js +9 -0
- package/dist/src/config/validateConfig.d.ts +22 -0
- package/dist/src/config/validateConfig.d.ts.map +1 -0
- package/dist/src/config/validateConfig.js +171 -0
- package/dist/src/config.d.ts +168 -0
- package/dist/src/config.d.ts.map +1 -0
- package/dist/src/config.js +204 -0
- package/dist/src/coverage/deep-analysis/callGraph.d.ts +67 -0
- package/dist/src/coverage/deep-analysis/callGraph.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/callGraph.js +275 -0
- package/dist/src/coverage/deep-analysis/deepEndpointResolver.d.ts +23 -0
- package/dist/src/coverage/deep-analysis/deepEndpointResolver.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/deepEndpointResolver.js +394 -0
- package/dist/src/coverage/deep-analysis/index.d.ts +17 -0
- package/dist/src/coverage/deep-analysis/index.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/index.js +63 -0
- package/dist/src/coverage/deep-analysis/resolveAssertions.d.ts +60 -0
- package/dist/src/coverage/deep-analysis/resolveAssertions.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/resolveAssertions.js +121 -0
- package/dist/src/coverage/deep-analysis/resolveConstants.d.ts +36 -0
- package/dist/src/coverage/deep-analysis/resolveConstants.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/resolveConstants.js +92 -0
- package/dist/src/coverage/deep-analysis/resolveEnums.d.ts +55 -0
- package/dist/src/coverage/deep-analysis/resolveEnums.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/resolveEnums.js +152 -0
- package/dist/src/coverage/deep-analysis/resolveMethodChains.d.ts +70 -0
- package/dist/src/coverage/deep-analysis/resolveMethodChains.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/resolveMethodChains.js +152 -0
- package/dist/src/coverage/deep-analysis/resolvePaths.d.ts +80 -0
- package/dist/src/coverage/deep-analysis/resolvePaths.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/resolvePaths.js +216 -0
- package/dist/src/coverage/deep-analysis/resolveRequestWrappers.d.ts +71 -0
- package/dist/src/coverage/deep-analysis/resolveRequestWrappers.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/resolveRequestWrappers.js +226 -0
- package/dist/src/coverage/deep-analysis/symbolTable.d.ts +58 -0
- package/dist/src/coverage/deep-analysis/symbolTable.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/symbolTable.js +230 -0
- package/dist/src/coverage/deep-analysis/types.d.ts +122 -0
- package/dist/src/coverage/deep-analysis/types.d.ts.map +1 -0
- package/dist/src/coverage/deep-analysis/types.js +21 -0
- package/dist/src/discovery/fileClassifier.d.ts +50 -0
- package/dist/src/discovery/fileClassifier.d.ts.map +1 -0
- package/dist/src/discovery/fileClassifier.js +238 -0
- package/dist/src/discovery/projectDiscovery.d.ts +66 -0
- package/dist/src/discovery/projectDiscovery.d.ts.map +1 -0
- package/dist/src/discovery/projectDiscovery.js +287 -0
- package/dist/src/endpointCoverage.d.ts +70 -0
- package/dist/src/endpointCoverage.d.ts.map +1 -0
- package/dist/src/endpointCoverage.js +381 -0
- package/dist/src/errorCoverage.d.ts +93 -0
- package/dist/src/errorCoverage.d.ts.map +1 -0
- package/dist/src/errorCoverage.js +698 -0
- package/dist/src/index.d.ts +3 -0
- package/dist/src/index.d.ts.map +1 -0
- package/dist/src/index.js +1441 -0
- package/dist/src/inference/businessRuleInference.d.ts +63 -0
- package/dist/src/inference/businessRuleInference.d.ts.map +1 -0
- package/dist/src/inference/businessRuleInference.js +268 -0
- package/dist/src/inference/integrationFlowInference.d.ts +56 -0
- package/dist/src/inference/integrationFlowInference.d.ts.map +1 -0
- package/dist/src/inference/integrationFlowInference.js +266 -0
- package/dist/src/integrationCoverage.d.ts +72 -0
- package/dist/src/integrationCoverage.d.ts.map +1 -0
- package/dist/src/integrationCoverage.js +317 -0
- package/dist/src/intelligence/index.d.ts +20 -0
- package/dist/src/intelligence/index.d.ts.map +1 -0
- package/dist/src/intelligence/index.js +105 -0
- package/dist/src/intelligence/linkageEngine.d.ts +20 -0
- package/dist/src/intelligence/linkageEngine.d.ts.map +1 -0
- package/dist/src/intelligence/linkageEngine.js +522 -0
- package/dist/src/intelligence/markdownReporter.d.ts +12 -0
- package/dist/src/intelligence/markdownReporter.d.ts.map +1 -0
- package/dist/src/intelligence/markdownReporter.js +265 -0
- package/dist/src/intelligence/riskScoring.d.ts +53 -0
- package/dist/src/intelligence/riskScoring.d.ts.map +1 -0
- package/dist/src/intelligence/riskScoring.js +181 -0
- package/dist/src/intelligence/types.d.ts +121 -0
- package/dist/src/intelligence/types.d.ts.map +1 -0
- package/dist/src/intelligence/types.js +8 -0
- package/dist/src/languageDetection.d.ts +100 -0
- package/dist/src/languageDetection.d.ts.map +1 -0
- package/dist/src/languageDetection.js +349 -0
- package/dist/src/languages/java/index.d.ts +16 -0
- package/dist/src/languages/java/index.d.ts.map +1 -0
- package/dist/src/languages/java/index.js +103 -0
- package/dist/src/languages/java/parser.d.ts +7 -0
- package/dist/src/languages/java/parser.d.ts.map +1 -0
- package/dist/src/languages/java/parser.js +50 -0
- package/dist/src/languages/java/semanticBuilder.d.ts +21 -0
- package/dist/src/languages/java/semanticBuilder.d.ts.map +1 -0
- package/dist/src/languages/java/semanticBuilder.js +358 -0
- package/dist/src/languages/javascript/annotationExtractor.d.ts +20 -0
- package/dist/src/languages/javascript/annotationExtractor.d.ts.map +1 -0
- package/dist/src/languages/javascript/annotationExtractor.js +94 -0
- package/dist/src/languages/javascript/assertionResolver.d.ts +18 -0
- package/dist/src/languages/javascript/assertionResolver.d.ts.map +1 -0
- package/dist/src/languages/javascript/assertionResolver.js +150 -0
- package/dist/src/languages/javascript/callResolver.d.ts +23 -0
- package/dist/src/languages/javascript/callResolver.d.ts.map +1 -0
- package/dist/src/languages/javascript/callResolver.js +236 -0
- package/dist/src/languages/javascript/httpInteractionExtractor.d.ts +23 -0
- package/dist/src/languages/javascript/httpInteractionExtractor.d.ts.map +1 -0
- package/dist/src/languages/javascript/httpInteractionExtractor.js +205 -0
- package/dist/src/languages/javascript/index.d.ts +20 -0
- package/dist/src/languages/javascript/index.d.ts.map +1 -0
- package/dist/src/languages/javascript/index.js +136 -0
- package/dist/src/languages/javascript/parser.d.ts +14 -0
- package/dist/src/languages/javascript/parser.d.ts.map +1 -0
- package/dist/src/languages/javascript/parser.js +38 -0
- package/dist/src/languages/javascript/symbolResolver.d.ts +31 -0
- package/dist/src/languages/javascript/symbolResolver.d.ts.map +1 -0
- package/dist/src/languages/javascript/symbolResolver.js +183 -0
- package/dist/src/languages/kotlin/index.d.ts +16 -0
- package/dist/src/languages/kotlin/index.d.ts.map +1 -0
- package/dist/src/languages/kotlin/index.js +151 -0
- package/dist/src/languages/kotlin/parser.d.ts +11 -0
- package/dist/src/languages/kotlin/parser.d.ts.map +1 -0
- package/dist/src/languages/kotlin/parser.js +74 -0
- package/dist/src/languages/python/index.d.ts +15 -0
- package/dist/src/languages/python/index.d.ts.map +1 -0
- package/dist/src/languages/python/index.js +293 -0
- package/dist/src/languages/ruby/index.d.ts +15 -0
- package/dist/src/languages/ruby/index.d.ts.map +1 -0
- package/dist/src/languages/ruby/index.js +274 -0
- package/dist/src/languages/shared/treeSitterUtils.d.ts +43 -0
- package/dist/src/languages/shared/treeSitterUtils.d.ts.map +1 -0
- package/dist/src/languages/shared/treeSitterUtils.js +100 -0
- package/dist/src/languages/typescript/index.d.ts +14 -0
- package/dist/src/languages/typescript/index.d.ts.map +1 -0
- package/dist/src/languages/typescript/index.js +25 -0
- package/dist/src/lib/index.d.ts +228 -0
- package/dist/src/lib/index.d.ts.map +1 -0
- package/dist/src/lib/index.js +486 -0
- package/dist/src/mcp/client/index.d.ts +37 -0
- package/dist/src/mcp/client/index.d.ts.map +1 -0
- package/dist/src/mcp/client/index.js +235 -0
- package/dist/src/mcp/config.d.ts +50 -0
- package/dist/src/mcp/config.d.ts.map +1 -0
- package/dist/src/mcp/config.js +125 -0
- package/dist/src/mcp/events.d.ts +24 -0
- package/dist/src/mcp/events.d.ts.map +1 -0
- package/dist/src/mcp/events.js +48 -0
- package/dist/src/mcp/fallback/index.d.ts +50 -0
- package/dist/src/mcp/fallback/index.d.ts.map +1 -0
- package/dist/src/mcp/fallback/index.js +216 -0
- package/dist/src/mcp/index.d.ts +67 -0
- package/dist/src/mcp/index.d.ts.map +1 -0
- package/dist/src/mcp/index.js +212 -0
- package/dist/src/mcp/normalizer.d.ts +21 -0
- package/dist/src/mcp/normalizer.d.ts.map +1 -0
- package/dist/src/mcp/normalizer.js +99 -0
- package/dist/src/mcp/prompts/index.d.ts +86 -0
- package/dist/src/mcp/prompts/index.d.ts.map +1 -0
- package/dist/src/mcp/prompts/index.js +304 -0
- package/dist/src/mcp/templates/index.d.ts +35 -0
- package/dist/src/mcp/templates/index.d.ts.map +1 -0
- package/dist/src/mcp/templates/index.js +143 -0
- package/dist/src/mcp/testing/mock-server/index.d.ts +47 -0
- package/dist/src/mcp/testing/mock-server/index.d.ts.map +1 -0
- package/dist/src/mcp/testing/mock-server/index.js +157 -0
- package/dist/src/mcp/types.d.ts +127 -0
- package/dist/src/mcp/types.d.ts.map +1 -0
- package/dist/src/mcp/types.js +8 -0
- package/dist/src/observability.d.ts +138 -0
- package/dist/src/observability.d.ts.map +1 -0
- package/dist/src/observability.js +519 -0
- package/dist/src/parameterCoverage.d.ts +75 -0
- package/dist/src/parameterCoverage.d.ts.map +1 -0
- package/dist/src/parameterCoverage.js +629 -0
- package/dist/src/perfResilienceCoverage.d.ts +155 -0
- package/dist/src/perfResilienceCoverage.d.ts.map +1 -0
- package/dist/src/perfResilienceCoverage.js +670 -0
- package/dist/src/pluginLoader.d.ts +51 -0
- package/dist/src/pluginLoader.d.ts.map +1 -0
- package/dist/src/pluginLoader.js +72 -0
- package/dist/src/publishing.d.ts +63 -0
- package/dist/src/publishing.d.ts.map +1 -0
- package/dist/src/publishing.js +379 -0
- package/dist/src/qualityGate.d.ts +58 -0
- package/dist/src/qualityGate.d.ts.map +1 -0
- package/dist/src/qualityGate.js +118 -0
- package/dist/src/reporting.d.ts +41 -0
- package/dist/src/reporting.d.ts.map +1 -0
- package/dist/src/reporting.js +278 -0
- package/dist/src/screenshots.d.ts +71 -0
- package/dist/src/screenshots.d.ts.map +1 -0
- package/dist/src/screenshots.js +141 -0
- package/dist/src/security/gate/index.d.ts +11 -0
- package/dist/src/security/gate/index.d.ts.map +1 -0
- package/dist/src/security/gate/index.js +65 -0
- package/dist/src/security/index.d.ts +30 -0
- package/dist/src/security/index.d.ts.map +1 -0
- package/dist/src/security/index.js +342 -0
- package/dist/src/security/normalizers/semgrep.d.ts +10 -0
- package/dist/src/security/normalizers/semgrep.d.ts.map +1 -0
- package/dist/src/security/normalizers/semgrep.js +104 -0
- package/dist/src/security/normalizers/trivy.d.ts +10 -0
- package/dist/src/security/normalizers/trivy.d.ts.map +1 -0
- package/dist/src/security/normalizers/trivy.js +78 -0
- package/dist/src/security/normalizers/zap.d.ts +10 -0
- package/dist/src/security/normalizers/zap.d.ts.map +1 -0
- package/dist/src/security/normalizers/zap.js +104 -0
- package/dist/src/security/scanners/semgrep.d.ts +6 -0
- package/dist/src/security/scanners/semgrep.d.ts.map +1 -0
- package/dist/src/security/scanners/semgrep.js +125 -0
- package/dist/src/security/scanners/trivy.d.ts +6 -0
- package/dist/src/security/scanners/trivy.d.ts.map +1 -0
- package/dist/src/security/scanners/trivy.js +115 -0
- package/dist/src/security/scanners/zap.d.ts +6 -0
- package/dist/src/security/scanners/zap.d.ts.map +1 -0
- package/dist/src/security/scanners/zap.js +135 -0
- package/dist/src/security/types.d.ts +146 -0
- package/dist/src/security/types.d.ts.map +1 -0
- package/dist/src/security/types.js +6 -0
- package/dist/src/securityCoverage.d.ts +116 -0
- package/dist/src/securityCoverage.d.ts.map +1 -0
- package/dist/src/securityCoverage.js +725 -0
- package/dist/src/summary/buildSummary.d.ts +28 -0
- package/dist/src/summary/buildSummary.d.ts.map +1 -0
- package/dist/src/summary/buildSummary.js +257 -0
- package/dist/src/summary/evaluateMetrics.d.ts +31 -0
- package/dist/src/summary/evaluateMetrics.d.ts.map +1 -0
- package/dist/src/summary/evaluateMetrics.js +118 -0
- package/dist/src/summary/index.d.ts +10 -0
- package/dist/src/summary/index.d.ts.map +1 -0
- package/dist/src/summary/index.js +22 -0
- package/dist/src/summary/markdownRenderer.d.ts +139 -0
- package/dist/src/summary/markdownRenderer.d.ts.map +1 -0
- package/dist/src/summary/markdownRenderer.js +459 -0
- package/dist/src/summary/prSummary.d.ts +24 -0
- package/dist/src/summary/prSummary.d.ts.map +1 -0
- package/dist/src/summary/prSummary.js +233 -0
- package/dist/src/summary/summaryTypes.d.ts +35 -0
- package/dist/src/summary/summaryTypes.d.ts.map +1 -0
- package/dist/src/summary/summaryTypes.js +27 -0
- package/package.json +84 -0
package/README.md
ADDED
|
@@ -0,0 +1,703 @@
|
|
|
1
|
+
# API Test Coverage Analyzer
|
|
2
|
+
|
|
3
|
+
[](https://github.com/q-intel/apiTestsCoverageAnalyzer/actions)
|
|
4
|
+
[](LICENSE)
|
|
5
|
+
[](https://nodejs.org)
|
|
6
|
+
|
|
7
|
+
A CLI tool that measures how thoroughly your test suite exercises your API surface area. Rather than simply counting passing tests, it asks:
|
|
8
|
+
|
|
9
|
+
- Are all **endpoints** reachable via at least one test?
|
|
10
|
+
- Are **parameters** tested with valid, boundary, missing, and invalid values?
|
|
11
|
+
- Are **business rules** (discount logic, rate limiting, etc.) explicitly validated?
|
|
12
|
+
- Do **integration flows** (multi-step user journeys) run end-to-end?
|
|
13
|
+
- Are **security scenarios** (auth bypass, injection, IDOR) covered?
|
|
14
|
+
- Are **error paths** (4xx/5xx) handled correctly?
|
|
15
|
+
- Is there **performance and resilience** evidence (JMeter/k6 data)?
|
|
16
|
+
- Does the API **remain compatible** between versions?
|
|
17
|
+
|
|
18
|
+
The answers appear in rich **HTML**, **JSON**, **CSV**, and **JUnit** reports that can be enforced as pass/fail gates in any CI pipeline.
|
|
19
|
+
|
|
20
|
+
### Coverage Intelligence
|
|
21
|
+
|
|
22
|
+
Beyond raw percentages, the **Coverage Intelligence** engine answers:
|
|
23
|
+
|
|
24
|
+
> **What is missing? What matters most? What should be tested next?**
|
|
25
|
+
|
|
26
|
+
It identifies **functional findings**, links them to **missing test recommendations**, assigns **risk scores** (0–100), and prioritises work as **P0/P1/P2/P3**. Outputs are AI-friendly markdown — ready for LLM consumption or CI gating.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Table of Contents
|
|
31
|
+
|
|
32
|
+
- [Prerequisites](#prerequisites)
|
|
33
|
+
- [Installation](#installation)
|
|
34
|
+
- [Quickstart](#quickstart)
|
|
35
|
+
- [Using as a Library](#using-as-a-library)
|
|
36
|
+
- [GitHub Action](#github-action)
|
|
37
|
+
- [Commands overview](#commands-overview)
|
|
38
|
+
- [Configuration](#configuration)
|
|
39
|
+
- [AST Analysis](#ast-analysis)
|
|
40
|
+
- [UI Dashboard](#ui-dashboard)
|
|
41
|
+
- [Documentation](#documentation)
|
|
42
|
+
- [Contributing](#contributing)
|
|
43
|
+
- [License](#license)
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## Prerequisites
|
|
48
|
+
|
|
49
|
+
- **Node.js** ≥ 18 LTS
|
|
50
|
+
- **npm** ≥ 9
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
node --version # v20.x
|
|
54
|
+
npm --version # 10.x
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Installation
|
|
58
|
+
|
|
59
|
+
### Option 1 – Install globally from npm (recommended)
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
npm install -g api-test-coverage-analyzer
|
|
63
|
+
api-coverage --help
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### Option 2 – Run without installing (npx)
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
npx api-test-coverage-analyzer endpoint-coverage \
|
|
70
|
+
--spec openapi.yaml \
|
|
71
|
+
--tests 'tests/**/*.ts'
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### Option 3 – Clone the repository (development)
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
git clone https://github.com/q-intel/apiTestsCoverageAnalyzer.git
|
|
78
|
+
cd apiTestsCoverageAnalyzer
|
|
79
|
+
|
|
80
|
+
# Install dependencies
|
|
81
|
+
npm install
|
|
82
|
+
|
|
83
|
+
# Compile TypeScript
|
|
84
|
+
npm run build
|
|
85
|
+
|
|
86
|
+
# Verify
|
|
87
|
+
node dist/src/index.js --help
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Alternatively, use `ts-node` to skip the build step:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
node -r ts-node/register src/index.ts --help
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Quickstart
|
|
97
|
+
|
|
98
|
+
Run all coverage types against the included sample project:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
# Endpoint coverage (using globally installed CLI)
|
|
102
|
+
api-coverage endpoint-coverage \
|
|
103
|
+
--spec sample/openapi.yaml \
|
|
104
|
+
--tests "sample/tests/**/*.ts" \
|
|
105
|
+
--format json,html \
|
|
106
|
+
--threshold-endpoint 80
|
|
107
|
+
|
|
108
|
+
# Business rule coverage
|
|
109
|
+
api-coverage business-coverage \
|
|
110
|
+
--rules sample/business-rules.yaml \
|
|
111
|
+
--tests "sample/tests/**/*.ts" \
|
|
112
|
+
--format json,html
|
|
113
|
+
|
|
114
|
+
# Integration flow coverage
|
|
115
|
+
api-coverage integration-coverage \
|
|
116
|
+
--flows sample/integration-flows.yaml \
|
|
117
|
+
--tests "sample/tests/**/*.ts" \
|
|
118
|
+
--format json,html
|
|
119
|
+
|
|
120
|
+
# Security coverage
|
|
121
|
+
api-coverage security-coverage \
|
|
122
|
+
--spec sample/openapi-security.yaml \
|
|
123
|
+
--tests "sample/tests/**/*.ts" \
|
|
124
|
+
--format json,html
|
|
125
|
+
|
|
126
|
+
# Compatibility check (breaking changes between v1 and v2)
|
|
127
|
+
api-coverage compatibility-check \
|
|
128
|
+
--old-spec sample/v1.yaml \
|
|
129
|
+
--new-spec sample/v2.yaml \
|
|
130
|
+
--contracts "sample/contracts/**/*.json" \
|
|
131
|
+
--format json,html
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Reports are written to the `reports/` directory.
|
|
135
|
+
|
|
136
|
+
## Using as a Library
|
|
137
|
+
|
|
138
|
+
Install as a project dependency:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
npm install api-test-coverage-analyzer
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Then import the analysis functions in your own scripts:
|
|
145
|
+
|
|
146
|
+
```js
|
|
147
|
+
const {
|
|
148
|
+
analyzeEndpoints,
|
|
149
|
+
analyzeParameters,
|
|
150
|
+
analyzeBusinessRules,
|
|
151
|
+
analyzeIntegrationFlows,
|
|
152
|
+
analyzeErrorHandling,
|
|
153
|
+
analyzeSecurityControls,
|
|
154
|
+
analyzePerfResilience,
|
|
155
|
+
analyzeCompatibility,
|
|
156
|
+
checkThresholds,
|
|
157
|
+
} = require('api-test-coverage-analyzer');
|
|
158
|
+
|
|
159
|
+
async function runCoverage() {
|
|
160
|
+
// Endpoint coverage
|
|
161
|
+
const endpointResult = await analyzeEndpoints({
|
|
162
|
+
spec: 'openapi.yaml',
|
|
163
|
+
tests: 'tests/**/*.ts',
|
|
164
|
+
format: 'json,html',
|
|
165
|
+
thresholdEndpoint: 80,
|
|
166
|
+
});
|
|
167
|
+
console.log(`Endpoint coverage: ${endpointResult.coveragePercent}%`);
|
|
168
|
+
|
|
169
|
+
// Business rule coverage
|
|
170
|
+
const businessResult = await analyzeBusinessRules({
|
|
171
|
+
rules: 'business-rules.yaml',
|
|
172
|
+
tests: 'tests/**/*.ts',
|
|
173
|
+
});
|
|
174
|
+
console.log(`Business coverage: ${businessResult.coveragePercent}%`);
|
|
175
|
+
|
|
176
|
+
// Check thresholds
|
|
177
|
+
const failures = checkThresholds(
|
|
178
|
+
[endpointResult, businessResult],
|
|
179
|
+
{ endpoint: 80, business: 60 }
|
|
180
|
+
);
|
|
181
|
+
if (failures.length > 0) {
|
|
182
|
+
console.error('Threshold failures:', failures);
|
|
183
|
+
process.exitCode = 1;
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
runCoverage();
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
TypeScript users get full type definitions out of the box.
|
|
191
|
+
|
|
192
|
+
## GitHub Action
|
|
193
|
+
|
|
194
|
+
Add API coverage analysis to your CI/CD pipeline with zero setup:
|
|
195
|
+
|
|
196
|
+
```yaml
|
|
197
|
+
steps:
|
|
198
|
+
- name: Checkout
|
|
199
|
+
uses: actions/checkout@v4
|
|
200
|
+
|
|
201
|
+
- name: Run API coverage analysis
|
|
202
|
+
id: coverage
|
|
203
|
+
uses: q-intel/apiTestsCoverageAnalyzer/action@v1
|
|
204
|
+
with:
|
|
205
|
+
spec: 'sample/openapi.yaml'
|
|
206
|
+
tests: 'tests/**/*.ts'
|
|
207
|
+
format: 'json,html'
|
|
208
|
+
coverage-types: 'endpoint,error,security'
|
|
209
|
+
threshold-endpoint: '80'
|
|
210
|
+
|
|
211
|
+
- name: Print endpoint coverage
|
|
212
|
+
run: echo "Endpoint coverage ${{ steps.coverage.outputs.endpoint-coverage }}%"
|
|
213
|
+
|
|
214
|
+
- name: Upload reports
|
|
215
|
+
uses: actions/upload-artifact@v4
|
|
216
|
+
with:
|
|
217
|
+
name: coverage-reports
|
|
218
|
+
path: reports/
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
### Action inputs
|
|
222
|
+
|
|
223
|
+
| Input | Description | Default |
|
|
224
|
+
|-------|-------------|---------|
|
|
225
|
+
| `spec` | Path to OpenAPI/Swagger spec file | `sample/openapi.yaml` |
|
|
226
|
+
| `tests` | Glob pattern for test files | `tests/**/*.ts` |
|
|
227
|
+
| `format` | Comma-separated report formats (`json,html,csv,junit`) | `json,html` |
|
|
228
|
+
| `coverage-types` | Coverage types to run (`endpoint,parameter,business,integration,error,security`) | `endpoint` |
|
|
229
|
+
| `rules` | Path to business rules YAML (required for `business` type) | — |
|
|
230
|
+
| `flows` | Path to integration flows YAML (required for `integration` type) | — |
|
|
231
|
+
| `language` | Test language(s) (`auto,typescript,javascript,java,python,ruby,cucumber`) | `auto` |
|
|
232
|
+
| `threshold-endpoint` | Minimum required endpoint coverage % | `0` |
|
|
233
|
+
| `threshold-parameter` | Minimum required parameter coverage % | `0` |
|
|
234
|
+
| `threshold-business` | Minimum required business rule coverage % | `0` |
|
|
235
|
+
| `threshold-integration` | Minimum required integration flow coverage % | `0` |
|
|
236
|
+
| `threshold-error` | Minimum required error handling coverage % | `0` |
|
|
237
|
+
| `threshold-security` | Minimum required security coverage % | `0` |
|
|
238
|
+
| `reports-dir` | Directory to write reports into | `reports` |
|
|
239
|
+
|
|
240
|
+
### Action outputs
|
|
241
|
+
|
|
242
|
+
| Output | Description |
|
|
243
|
+
|--------|-------------|
|
|
244
|
+
| `endpoint-coverage` | Endpoint coverage percentage |
|
|
245
|
+
| `parameter-coverage` | Parameter coverage percentage |
|
|
246
|
+
| `business-coverage` | Business rule coverage percentage |
|
|
247
|
+
| `integration-coverage` | Integration flow coverage percentage |
|
|
248
|
+
| `error-coverage` | Error handling coverage percentage |
|
|
249
|
+
| `security-coverage` | Security coverage percentage |
|
|
250
|
+
| `reports-dir` | Absolute path to the generated reports directory |
|
|
251
|
+
| `overallStatus` | `"passed"` or `"failed"` |
|
|
252
|
+
| `overallCoverage` | Average coverage % across all analyzed categories |
|
|
253
|
+
| `failedGates` | Comma-separated list of failed category names (empty when all pass) |
|
|
254
|
+
| `summaryPath` | Path to the generated `build-summary.md` |
|
|
255
|
+
|
|
256
|
+
The action fails (non-zero exit code) when any coverage threshold is not met.
|
|
257
|
+
Summary files are always generated – even when the gate fails.
|
|
258
|
+
|
|
259
|
+
## Commands overview
|
|
260
|
+
|
|
261
|
+
| Command | What it measures | Key flag |
|
|
262
|
+
|---------|-----------------|---------|
|
|
263
|
+
| `endpoint-coverage` | % of spec endpoints hit by tests | `--spec`, `--tests` |
|
|
264
|
+
| `parameter-coverage` | valid/boundary/missing/invalid param testing | `--spec`, `--tests` |
|
|
265
|
+
| `business-coverage` | % of business rules covered (`@businessRule`) | `--rules`, `--tests` |
|
|
266
|
+
| `integration-coverage` | % of integration flows covered (`@flow`) | `--flows`, `--tests` |
|
|
267
|
+
| `error-coverage` | 4xx/5xx/validation/timeout scenarios | `--spec`, `--tests` |
|
|
268
|
+
| `security-coverage` | OWASP Top-10 security scenarios | `--spec`, `--tests` |
|
|
269
|
+
| `perf-resilience-coverage` | Load-test SLA + resilience patterns | `--spec`, `--load-results` |
|
|
270
|
+
| `compatibility-check` | Breaking changes + Pact contract violations | `--old-spec`, `--new-spec` |
|
|
271
|
+
| `generate-md-report` | Markdown summary from JSON reports | `--reports`, `--output` |
|
|
272
|
+
| `coverage-intelligence` | Identify findings, missing tests, risk scores, P0–P3 priorities | `--reports-dir`, `--out-dir` |
|
|
273
|
+
|
|
274
|
+
All commands accept `--format json,html,csv,junit` and `--threshold-*` flags.
|
|
275
|
+
|
|
276
|
+
### Coverage Intelligence (`coverage-intelligence`)
|
|
277
|
+
|
|
278
|
+
The intelligence command ingests all coverage reports and produces prioritised, AI-friendly outputs:
|
|
279
|
+
|
|
280
|
+
```bash
|
|
281
|
+
# Generate intelligence reports after running other coverage commands
|
|
282
|
+
api-coverage coverage-intelligence \
|
|
283
|
+
--reports-dir reports \
|
|
284
|
+
--out-dir reports \
|
|
285
|
+
--project-name my-api \
|
|
286
|
+
--languages typescript \
|
|
287
|
+
--frameworks jest
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
**Generated files:**
|
|
291
|
+
|
|
292
|
+
| File | Description |
|
|
293
|
+
|------|-------------|
|
|
294
|
+
| `reports/coverage-intelligence.json` | Full intelligence report (findings + recommendations) |
|
|
295
|
+
| `reports/coverage-intelligence.md` | AI-friendly summary with top 10 findings and recommendations |
|
|
296
|
+
| `reports/missing-tests-recommendations.json` | Prioritised missing test recommendations |
|
|
297
|
+
| `reports/missing-tests-recommendations.md` | Markdown recommendations per recommendation |
|
|
298
|
+
| `reports/risk-prioritization.json` | Risk breakdown by score, category, and endpoint |
|
|
299
|
+
| `reports/risk-prioritization.md` | Risk prioritisation narrative |
|
|
300
|
+
|
|
301
|
+
**Functional Findings** map gaps in coverage to specific root causes (e.g. "no auth test on DELETE /users/{id}").
|
|
302
|
+
**Missing Test Recommendations** are prioritised P0–P3 by a risk formula:
|
|
303
|
+
|
|
304
|
+
```
|
|
305
|
+
Risk Score =
|
|
306
|
+
0.30 × SeverityWeight +
|
|
307
|
+
0.20 × ExposureWeight +
|
|
308
|
+
0.15 × CriticalityWeight +
|
|
309
|
+
0.15 × MissingCoverageWeight +
|
|
310
|
+
0.10 × SecuritySignalWeight +
|
|
311
|
+
0.05 × FlowImpactWeight +
|
|
312
|
+
0.05 × ChangeVolatilityWeight
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
| Score | Risk Band | Priority |
|
|
316
|
+
|-------|-----------|---------|
|
|
317
|
+
| 85–100 | Critical | P0 — immediate action |
|
|
318
|
+
| 70–84 | Critical | P1 — high urgency |
|
|
319
|
+
| 50–69 | High | P2 — address soon |
|
|
320
|
+
| 0–49 | Moderate/Low | P3 — backlog |
|
|
321
|
+
|
|
322
|
+
Security findings, money-movement endpoints, and auth/authz gaps are never rated below P1 regardless of formula score.
|
|
323
|
+
|
|
324
|
+
## Configuration
|
|
325
|
+
|
|
326
|
+
Create a `config.yaml` at your project root. Running `analyze` with no arguments discovers this
|
|
327
|
+
file automatically. If it is absent the analyzer runs the full default profile and emits a warning.
|
|
328
|
+
|
|
329
|
+
```yaml
|
|
330
|
+
version: 1
|
|
331
|
+
|
|
332
|
+
project:
|
|
333
|
+
name: my-api
|
|
334
|
+
|
|
335
|
+
analysis:
|
|
336
|
+
defaultMode: full
|
|
337
|
+
ast:
|
|
338
|
+
enabled: true # master switch for AST analysis
|
|
339
|
+
fallbackHeuristics: true # run regex when AST returns 0 results
|
|
340
|
+
maxCallDepth: 4 # wrapper/helper tracing depth
|
|
341
|
+
assertionAware: true # link HTTP calls to response assertions
|
|
342
|
+
languages: # per-language enable/disable toggles
|
|
343
|
+
javascript: { enabled: true }
|
|
344
|
+
typescript: { enabled: true }
|
|
345
|
+
java: { enabled: true }
|
|
346
|
+
kotlin: { enabled: true }
|
|
347
|
+
python: { enabled: true }
|
|
348
|
+
ruby: { enabled: true }
|
|
349
|
+
cucumber: { enabled: true }
|
|
350
|
+
|
|
351
|
+
scans:
|
|
352
|
+
coverage:
|
|
353
|
+
enabled: true
|
|
354
|
+
types:
|
|
355
|
+
- endpoint
|
|
356
|
+
- parameter
|
|
357
|
+
- business
|
|
358
|
+
- integration
|
|
359
|
+
- error
|
|
360
|
+
- security
|
|
361
|
+
- performance
|
|
362
|
+
- compatibility
|
|
363
|
+
security:
|
|
364
|
+
enabled: true
|
|
365
|
+
scanners:
|
|
366
|
+
- semgrep
|
|
367
|
+
- trivy
|
|
368
|
+
- zap
|
|
369
|
+
intelligence:
|
|
370
|
+
enabled: true
|
|
371
|
+
types:
|
|
372
|
+
- ai-summary
|
|
373
|
+
- risk-prioritization
|
|
374
|
+
- recommendations
|
|
375
|
+
- scanner-interpretation
|
|
376
|
+
|
|
377
|
+
thresholds:
|
|
378
|
+
global: 80
|
|
379
|
+
endpoint: 90
|
|
380
|
+
|
|
381
|
+
qualityGate:
|
|
382
|
+
enabled: true
|
|
383
|
+
mode: warn
|
|
384
|
+
|
|
385
|
+
reports:
|
|
386
|
+
outputDir: reports
|
|
387
|
+
formats:
|
|
388
|
+
- json
|
|
389
|
+
- html
|
|
390
|
+
|
|
391
|
+
mcp:
|
|
392
|
+
enabled: false
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
Use `--config <path>` to load an arbitrary YAML file:
|
|
396
|
+
|
|
397
|
+
```bash
|
|
398
|
+
analyze --config ./configs/staging.yaml
|
|
399
|
+
```
|
|
400
|
+
|
|
401
|
+
See [`docs/guides/configuration.md`](docs/guides/configuration.md) for the full field reference.
|
|
402
|
+
If you are migrating from `coverage.config.json`, see
|
|
403
|
+
[`docs/guides/migration-to-config-yaml.md`](docs/guides/migration-to-config-yaml.md).
|
|
404
|
+
|
|
405
|
+
CLI threshold flags (`--threshold-endpoint`, etc.) still work but are deprecated. Migrate
|
|
406
|
+
values to the `thresholds` block in `config.yaml`.
|
|
407
|
+
|
|
408
|
+
## AST Analysis
|
|
409
|
+
|
|
410
|
+
The analyzer uses a **true multi-language AST-based engine** to detect HTTP calls with far
|
|
411
|
+
higher accuracy than regex scanning. Analysis happens through a three-tier fallback cascade:
|
|
412
|
+
|
|
413
|
+
| Tier | Condition | Result |
|
|
414
|
+
|------|-----------|--------|
|
|
415
|
+
| 1 | AST parse succeeds | `confidence: high` or `medium` |
|
|
416
|
+
| 2 | AST yields 0 results + `fallbackHeuristics: true` | regex run, `confidence: low` |
|
|
417
|
+
| 3 | AST disabled or parse error | existing regex pipeline |
|
|
418
|
+
|
|
419
|
+
### Supported languages
|
|
420
|
+
|
|
421
|
+
| Language | Parser | Notes |
|
|
422
|
+
|----------|--------|-------|
|
|
423
|
+
| JavaScript | `@typescript-eslint/typescript-estree` | axios, fetch, supertest, got |
|
|
424
|
+
| TypeScript | `@typescript-eslint/typescript-estree` | full type annotation support |
|
|
425
|
+
| Java | `tree-sitter-java` | RestAssured, MockMvc, WebTestClient |
|
|
426
|
+
| Kotlin | `tree-sitter-kotlin` → `tree-sitter-java` → regex | Ktor DSL, Spring Boot |
|
|
427
|
+
| Python | `tree-sitter-python` | requests, httpx, Django/Flask test clients |
|
|
428
|
+
| Ruby | `tree-sitter-ruby` | Rails request specs, HTTParty, Faraday |
|
|
429
|
+
| Cucumber | (step dispatch) | `@Given/@When/@Then` annotations |
|
|
430
|
+
|
|
431
|
+
### Resolution types
|
|
432
|
+
|
|
433
|
+
Each covered endpoint carries a `resolutionType` indicating how the URL was found:
|
|
434
|
+
|
|
435
|
+
| Type | Description | Confidence |
|
|
436
|
+
|------|-------------|------------|
|
|
437
|
+
| `direct` | String literal URL in source | high |
|
|
438
|
+
| `constant` | Named constant resolved to URL | high/medium |
|
|
439
|
+
| `enum` | Enum member resolved to URL | high/medium |
|
|
440
|
+
| `string-template` | Template literal / f-string / interpolation | medium |
|
|
441
|
+
| `wrapper-method` | HTTP call traced through a helper function | medium |
|
|
442
|
+
| `request-builder` | Builder object (RequestEntity, etc.) | medium |
|
|
443
|
+
| `client-mapping` | Explicit client ↔ HTTP mapping | high |
|
|
444
|
+
| `interpolated-path` | URL with run-time segment interpolation | medium |
|
|
445
|
+
| `cucumber-step` | HTTP call inside a Cucumber step definition | medium |
|
|
446
|
+
| `heuristic` | Regex fallback | low |
|
|
447
|
+
|
|
448
|
+
### Disable AST for a specific language
|
|
449
|
+
|
|
450
|
+
```yaml
|
|
451
|
+
analysis:
|
|
452
|
+
ast:
|
|
453
|
+
enabled: true
|
|
454
|
+
languages:
|
|
455
|
+
kotlin:
|
|
456
|
+
enabled: false # use regex fallback for Kotlin only
|
|
457
|
+
```
|
|
458
|
+
|
|
459
|
+
### Debugging resolution
|
|
460
|
+
|
|
461
|
+
Add `--format json` to any command and inspect the `matches[].resolutionType` field
|
|
462
|
+
in the endpoint coverage JSON report.
|
|
463
|
+
|
|
464
|
+
## Built-in Summary Engine
|
|
465
|
+
|
|
466
|
+
The library owns summary generation. No custom scripting is needed.
|
|
467
|
+
|
|
468
|
+
### Generated files
|
|
469
|
+
|
|
470
|
+
When reports are generated (via CLI or GitHub Action), the following summary files
|
|
471
|
+
are automatically written to the configured `--reports-dir`:
|
|
472
|
+
|
|
473
|
+
| File | Description |
|
|
474
|
+
|------|-------------|
|
|
475
|
+
| `reports/build-summary.md` | Full CI/build summary (Markdown) |
|
|
476
|
+
| `reports/pr-summary.md` | Concise PR comment summary (Markdown) |
|
|
477
|
+
| `reports/summary.json` | Machine-readable summary data |
|
|
478
|
+
| `reports/ai-summary.md` | AI-optimized Markdown for agents |
|
|
479
|
+
| `reports/ai-summary.json` | AI-optimized JSON for agents |
|
|
480
|
+
|
|
481
|
+
### Gate-aware inclusion
|
|
482
|
+
|
|
483
|
+
Sections are included only when the analyzer ran or a threshold was configured.
|
|
484
|
+
Analyzers that did not run are silently omitted — no empty sections appear.
|
|
485
|
+
Control enabled scan types via `scans.coverage.types` in `config.yaml`.
|
|
486
|
+
|
|
487
|
+
### Public API
|
|
488
|
+
|
|
489
|
+
```ts
|
|
490
|
+
import { generateBuildSummary, generatePrSummary } from 'api-test-coverage-analyzer';
|
|
491
|
+
|
|
492
|
+
const { markdown, sections, json } = await generateBuildSummary({
|
|
493
|
+
results, // CoverageResult[] from any analyze* call
|
|
494
|
+
qualityGate, // QualityGateResult (optional)
|
|
495
|
+
thresholds, // Record<string, number> (optional)
|
|
496
|
+
projectName: 'my-api',
|
|
497
|
+
branch: 'main',
|
|
498
|
+
}, 'reports'); // optional output directory
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
```ts
|
|
502
|
+
type SummaryResult = {
|
|
503
|
+
markdown: string;
|
|
504
|
+
sections: Array<{
|
|
505
|
+
id: string; // e.g. "endpoint", "security-scan"
|
|
506
|
+
title: string; // human-readable heading
|
|
507
|
+
included: boolean;
|
|
508
|
+
gateEvaluated: boolean;
|
|
509
|
+
passed?: boolean;
|
|
510
|
+
markdown: string;
|
|
511
|
+
}>;
|
|
512
|
+
json: unknown; // machine-readable summary object
|
|
513
|
+
};
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
### GitHub Action outputs
|
|
517
|
+
|
|
518
|
+
| Output | Description |
|
|
519
|
+
|--------|-------------|
|
|
520
|
+
| `overallStatus` | `"passed"` or `"failed"` |
|
|
521
|
+
| `overallCoverage` | Average coverage % across all analyzed categories |
|
|
522
|
+
| `failedGates` | Comma-separated list of failed category names |
|
|
523
|
+
| `summaryPath` | Path to `build-summary.md` |
|
|
524
|
+
|
|
525
|
+
## UI Dashboard
|
|
526
|
+
|
|
527
|
+
A Vite + React dashboard is included for visualising reports:
|
|
528
|
+
|
|
529
|
+
```bash
|
|
530
|
+
cd dashboard
|
|
531
|
+
npm install
|
|
532
|
+
npm run dev # http://localhost:5173
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
Load any JSON report from `reports/` and explore:
|
|
536
|
+
|
|
537
|
+
- Overview, Endpoints, Parameters, Business Rules, Integration Flows
|
|
538
|
+
- Security, Errors, Performance/Resilience, Trends
|
|
539
|
+
|
|
540
|
+
## Documentation
|
|
541
|
+
|
|
542
|
+
Full documentation is available in the [`docs/`](docs/) directory and can be served locally:
|
|
543
|
+
|
|
544
|
+
```bash
|
|
545
|
+
npm run docs:dev # start dev server at http://localhost:5174
|
|
546
|
+
npm run docs:build # build static site → docs/.vitepress/dist/
|
|
547
|
+
npm run docs:check # validate sidebar, links, and assets, then build
|
|
548
|
+
npm run docs:preview # serve the built site (production preview)
|
|
549
|
+
npm run docs:test # run Cypress navigation/link tests
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
Documentation sections:
|
|
553
|
+
|
|
554
|
+
| Section | Description |
|
|
555
|
+
|---------|-------------|
|
|
556
|
+
| [Getting Started](docs/guide/getting-started.md) | First-run walkthrough |
|
|
557
|
+
| [Installation](docs/guide/installation.md) | Detailed setup steps |
|
|
558
|
+
| [CLI Reference](docs/reference/cli.md) | All commands and options |
|
|
559
|
+
| [Multi-Language Support](docs/guide/multi-language.md) | Java, Kotlin, Python, Ruby, Cucumber test suites |
|
|
560
|
+
| [Coverage Intelligence](docs/guide/coverage-intelligence.md) | Findings, risk scoring, missing test recommendations |
|
|
561
|
+
| [Architecture](docs/reference/architecture.md) | Module design and data flow |
|
|
562
|
+
| [CI/CD Integration](docs/guide/ci-cd.md) | GitHub Actions & Jenkins |
|
|
563
|
+
| [Interpreting Reports](docs/guide/interpreting-reports.md) | Reading each report type |
|
|
564
|
+
| [Writing Effective Tests](docs/guide/writing-tests.md) | Test best practices |
|
|
565
|
+
| [Extending via Plugins](docs/guide/plugins.md) | Custom coverage types |
|
|
566
|
+
| [Configuration Reference](docs/guides/configuration.md) | `config.yaml` field reference |
|
|
567
|
+
| [Troubleshooting](docs/guide/troubleshooting.md) | Common issues & FAQ |
|
|
568
|
+
| [Glossary](docs/guide/glossary.md) | Key terms |
|
|
569
|
+
| [Contributing](docs/reference/contributing.md) | How to contribute |
|
|
570
|
+
|
|
571
|
+
## TypeScript Example Project
|
|
572
|
+
|
|
573
|
+
A complete end-to-end example is available under [`examples/typescript/`](examples/typescript/).
|
|
574
|
+
|
|
575
|
+
This is a realistic **Wallets / Payments API** that demonstrates the analyzer in a real project context.
|
|
576
|
+
|
|
577
|
+
### Domain
|
|
578
|
+
|
|
579
|
+
| Concept | Description |
|
|
580
|
+
|---------|-------------|
|
|
581
|
+
| Wallets | Create, fund, debit, transfer, freeze/unfreeze, close |
|
|
582
|
+
| Payments | Create, process, refund, track status |
|
|
583
|
+
| Transactions | Ledger-style history |
|
|
584
|
+
| Risk / Limits | Daily limits, currency checks, idempotency |
|
|
585
|
+
| External deps | Payment processor + Fraud engine (nock-mocked) |
|
|
586
|
+
|
|
587
|
+
### Test layers
|
|
588
|
+
|
|
589
|
+
| Layer | Location | What it tests |
|
|
590
|
+
|-------|----------|---------------|
|
|
591
|
+
| Unit | `tests/unit/` | Service logic, risk rules, validation |
|
|
592
|
+
| Integration | `tests/integration/` | Routes, auth, supertest end-to-end |
|
|
593
|
+
| Blackbox | `tests/blackbox/` | Positive/negative/boundary/idempotency via HTTP |
|
|
594
|
+
| WireMock/nock | `tests/wiremock/` | External dependency healthy / failed / timeout |
|
|
595
|
+
|
|
596
|
+
### Running the example
|
|
597
|
+
|
|
598
|
+
```bash
|
|
599
|
+
cd examples/typescript
|
|
600
|
+
npm install
|
|
601
|
+
npm test # all 63 tests
|
|
602
|
+
npm run analyze # run the analyzer + generate reports
|
|
603
|
+
npm run screenshots # capture Playwright screenshots
|
|
604
|
+
```
|
|
605
|
+
|
|
606
|
+
### CI/CD demonstrations
|
|
607
|
+
|
|
608
|
+
| CI System | Location | What it does |
|
|
609
|
+
|-----------|----------|-------------|
|
|
610
|
+
| GitHub Actions | `.github/workflows/ci.yml` | Install, test, analyze, screenshots, upload artifacts |
|
|
611
|
+
| Jenkins | `ci/jenkins/Jenkinsfile` | Install, test, analyze, archive reports, surface gate failures |
|
|
612
|
+
|
|
613
|
+
### Observability
|
|
614
|
+
|
|
615
|
+
```bash
|
|
616
|
+
cd examples/typescript/observability
|
|
617
|
+
docker-compose up # starts Prometheus + Grafana
|
|
618
|
+
# Grafana at http://localhost:3000 — dashboards pre-configured
|
|
619
|
+
```
|
|
620
|
+
|
|
621
|
+
### Intentional coverage gaps
|
|
622
|
+
|
|
623
|
+
The example intentionally omits some test scenarios so the intelligence engine generates meaningful findings:
|
|
624
|
+
|
|
625
|
+
- Frozen wallet debit scenario (not tested)
|
|
626
|
+
- Daily $10,000 limit enforcement (not tested)
|
|
627
|
+
- Currency mismatch in transfer (not tested)
|
|
628
|
+
- Refund after 30-day window (not tested)
|
|
629
|
+
- Payment processor failure fallback (not tested)
|
|
630
|
+
|
|
631
|
+
## Self-Analysis
|
|
632
|
+
|
|
633
|
+
The analyzer is a **self-analyzing system**: on every build it runs all implemented metric types
|
|
634
|
+
against its own codebase, enforces 100% thresholds, and fails automatically if any metric falls
|
|
635
|
+
below threshold.
|
|
636
|
+
|
|
637
|
+
### Quick start
|
|
638
|
+
|
|
639
|
+
```bash
|
|
640
|
+
make install # Install dependencies
|
|
641
|
+
make build # Compile TypeScript
|
|
642
|
+
make self-analysis-all # Run all 8 metric types + intelligence engine
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
Or run the full CI pipeline:
|
|
646
|
+
|
|
647
|
+
```bash
|
|
648
|
+
make ci # install → build → test → self-analysis-all → summary
|
|
649
|
+
```
|
|
650
|
+
|
|
651
|
+
### Self-analysis input artifacts
|
|
652
|
+
|
|
653
|
+
| Artifact | Path | Purpose |
|
|
654
|
+
|---|---|---|
|
|
655
|
+
| OpenAPI spec | `openapi.self-analysis.yaml` | Analyzer CLI/library API surface |
|
|
656
|
+
| Business rules | `business-rules.self-analysis.yaml` | One rule per documented capability (19 rules) |
|
|
657
|
+
| Integration flows | `integration-flows.self-analysis.yaml` | Key usage sequences (5 flows) |
|
|
658
|
+
| Perf data | `load-results.self-analysis.json` | Reference data for performance metric |
|
|
659
|
+
| Config | `coverage.self-analysis.json` | 100% thresholds across all metrics |
|
|
660
|
+
|
|
661
|
+
### Reports
|
|
662
|
+
|
|
663
|
+
All reports are written to `reports/` after each run:
|
|
664
|
+
|
|
665
|
+
| File | Contents |
|
|
666
|
+
|---|---|
|
|
667
|
+
| `reports/endpoint-report.json/html` | Endpoint coverage |
|
|
668
|
+
| `reports/parameter-report.json/html` | Parameter coverage |
|
|
669
|
+
| `reports/business-report.json/html` | Business rule coverage |
|
|
670
|
+
| `reports/integration-report.json/html` | Integration flow coverage |
|
|
671
|
+
| `reports/error-report.json/html` | Error scenario coverage |
|
|
672
|
+
| `reports/security-report.json/html` | Security control coverage |
|
|
673
|
+
| `reports/perf-resilience-report.json/html` | Performance/resilience coverage |
|
|
674
|
+
| `reports/coverage-intelligence.json` | Intelligence findings + risk scores |
|
|
675
|
+
| `reports/pr-summary.md` | PR comment summary |
|
|
676
|
+
| `reports/build-summary.md` | Build log summary |
|
|
677
|
+
|
|
678
|
+
### Thresholds
|
|
679
|
+
|
|
680
|
+
All self-analysis thresholds default to **100%**. Override via environment variables for development:
|
|
681
|
+
|
|
682
|
+
```bash
|
|
683
|
+
THRESHOLD_ENDPOINT=80 make self-analysis-endpoint
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
See [docs/guides/thresholds.md](docs/guides/thresholds.md) for full threshold documentation.
|
|
687
|
+
|
|
688
|
+
### CI integration
|
|
689
|
+
|
|
690
|
+
The `.github/workflows/self-analysis.yml` workflow runs on every push and pull request.
|
|
691
|
+
All steps call Makefile targets. Pass/fail is governed by the analyzer's process exit code only.
|
|
692
|
+
|
|
693
|
+
See [docs/guides/self-analysis.md](docs/guides/self-analysis.md) for the full self-analysis guide.
|
|
694
|
+
|
|
695
|
+
## Contributing
|
|
696
|
+
|
|
697
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) and the [full contributing guide](docs/reference/contributing.md).
|
|
698
|
+
|
|
699
|
+
Please read our [Code of Conduct](CODE_OF_CONDUCT.md) before participating.
|
|
700
|
+
|
|
701
|
+
## License
|
|
702
|
+
|
|
703
|
+
MIT © [q-intel](https://github.com/q-intel)
|