@dailephd/my-dev-kit-lab 0.4.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +285 -12
- package/dist/scripts/audits/runAudit.js +28 -1
- package/dist/src/audits/core/auditConfig.js +16 -1
- package/dist/src/audits/core/auditRunner.js +6 -2
- package/dist/src/audits/report/auditReportModel.js +7 -0
- package/dist/src/audits/report/renderAuditJsonReport.js +1 -0
- package/dist/src/audits/report/renderAuditTextReport.js +37 -0
- package/dist/src/audits/security/androidAuditIntegration.js +145 -0
- package/dist/src/audits/security/mapAndroidSecurityFindingToAuditIssue.js +116 -0
- package/dist/src/audits/security/mapSecurityFindingToAuditIssue.js +15 -5
- package/dist/src/audits/security/securityAuditAdapter.js +20 -2
- package/dist/src/audits/security/securityAuditTypes.js +37 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,27 +1,300 @@
|
|
|
1
1
|
# my-dev-kit-lab
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
> Current release state: v0.4.1 is the latest published baseline. v0.4.0 delivered the Android validation MVP; v0.4.1 delivered advanced Android security. v0.4.2 is implemented and release-prepared (package metadata `0.4.2`) but not yet published. Manual pentest is post-v1 / version TBD.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
my-dev-kit-lab is the experiment, evidence, reporting, security-validation, and audit companion for my-dev-kit. It runs reproducible experiments that test whether my-dev-kit's graph-guided retrieval helps coding-agent workflows, collects metrics, renders reports, generates plots, captures screenshots, builds gallery outputs, performs automated CLI/package security validation, and runs the generic audit framework.
|
|
6
6
|
|
|
7
|
-
`v0.
|
|
7
|
+
The generic experiment-plugin framework was introduced in `v0.2.0`. Its first plugin, `context-strategy-comparison`, preserves the raw-full-file vs my-dev-kit-guided comparison through the plugin runner. Later releases added the generic audit framework, language-aware code-rot detectors for TypeScript/JavaScript, Python, Java, and Kotlin, a security-validation audit adapter, and Android validation. See [CHANGELOG.md](CHANGELOG.md) for the complete release history.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
my-dev-kit is most useful when the repository is larger than the task. It helps coding agents work with large codebases through reusable structural indexing, graph-guided retrieval, targeted source slices, and auditable context selection. Results are scoped evidence; the project does not claim that my-dev-kit always saves tokens.
|
|
10
|
+
|
|
11
|
+
**my-dev-kit** is the repo indexing and graph-guided retrieval engine.
|
|
12
|
+
**my-dev-kit-lab** is the separate lab layer that feeds it benchmark inputs and records evaluation outputs.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Current capabilities
|
|
17
|
+
|
|
18
|
+
- Benchmark projects at small, medium, and large complexity levels
|
|
19
|
+
- Project complexity metrics and benchmark case metadata with answer keys
|
|
20
|
+
- Prompt variant generation at `short`, `medium`, `long`, and `multi-step` complexity levels
|
|
21
|
+
- Fake-agent adapter for deterministic smoke and demo validation
|
|
22
|
+
- Codex and Claude adapters for real-agent campaigns
|
|
23
|
+
- Controlled experiment runner comparing `raw-full-file` vs `my-dev-kit-guided` strategies
|
|
24
|
+
- Deterministic correctness scoring from answer keys
|
|
25
|
+
- Token usage, duration, and status comparisons between matched strategy pairs
|
|
26
|
+
- HTML experiment report rendering
|
|
27
|
+
- Static SVG plot generation
|
|
28
|
+
- Optional PNG screenshot capture
|
|
29
|
+
- Gallery manifest and static gallery index output
|
|
30
|
+
- Visualization demos using my-dev-kit commands against benchmark projects
|
|
31
|
+
- Final demo workflow combining all pipeline stages
|
|
32
|
+
- Generic experiment-plugin command surface: `experiment:list`, `experiment:describe`, and `experiment:run`
|
|
33
|
+
- First experiment plugin: `context-strategy-comparison`
|
|
34
|
+
- Target-aware experiment execution for local projects via `experiment:run -- --target <path>`
|
|
35
|
+
- Plugin-aware JSON and HTML reports with plugin, target, variant, metric, artifact, warning, skip, and failure metadata
|
|
36
|
+
- Security validation framework: dependency audit, package tarball inspection, CLI adversarial tests, static scans (CodeQL/Semgrep), bounded fuzz smoke, attack-scenario checks, and structured verdict/report output — runnable against any local project via `security:validate --target <path>`. `security:validate` remains the standalone, focused security-validation command; it is not replaced by the audit adapter below.
|
|
37
|
+
- Generic audit framework: `npm run audit` runs conservative code-rot and security detectors and writes text/JSON reports under `reports/audits/` by default. `code-rot` and `security` are the currently implemented audit types (`--types code-rot`, `--types security`, or combined `--types code-rot,security`); `quality`, `project`, and `all` audit types are planned and fail cleanly instead of running. Audit does not modify target files and does not auto-fix issues. See [docs/COMMANDS.md](docs/COMMANDS.md) for full flags.
|
|
38
|
+
- Language-aware code-rot detectors for TypeScript/JavaScript, Python, Java, and Kotlin, built on a shared source-facts model and normalized language/file-role inventory. Java/Kotlin support is conservative static analysis only: no compiler parsing, type/classpath resolution, or Gradle/Maven execution.
|
|
39
|
+
- Security-validation audit adapter: `npm run audit -- --types security` runs the same `security:validate` internals through the shared audit/report surface, mapping findings into audit issues and linking back to the original `reports/security/` report. `security:validate`'s own checks and reports are unaffected.
|
|
40
|
+
- Android validation: `security:validate --profile android` runs nineteen static checks against an Android project (manifest, permissions, exported components, network security config, backup/release configuration, secrets, signing, WebView/FileProvider, sensitive storage/logging, and Firebase/Google services), with optional opt-in Gradle operations and external tools (Semgrep, OSV-Scanner, Android Lint, Dependency-Check). Default execution starts zero Gradle processes, zero external tools, and zero network operations.
|
|
41
|
+
- Android-aware generic audit: `npm run audit -- --types security --android` runs the same static Android validation through the audit adapter, mapping confirmed findings into audit issues while keeping review-only `CandidateEvidence` separate and never treated as a confirmed issue.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Architecture overview
|
|
46
|
+
|
|
47
|
+
```mermaid
|
|
48
|
+
flowchart TD
|
|
49
|
+
A[scripts/experiments] --> B[Experiment plugin registry + runner]
|
|
50
|
+
B --> C[context-strategy-comparison]
|
|
51
|
+
C --> D[Controlled experiment foundations]
|
|
52
|
+
D --> E[Plugin + legacy artifacts]
|
|
53
|
+
E --> F[Plugin-aware reports]
|
|
54
|
+
E --> G[Plots / screenshots / gallery]
|
|
55
|
+
H[scripts/security] --> I[Automated security validation]
|
|
56
|
+
I --> J[Security reports + verdict]
|
|
57
|
+
I --> K[Android validation]
|
|
58
|
+
L[scripts/audits] --> M[Generic audit framework]
|
|
59
|
+
M --> N[Code-rot detectors]
|
|
60
|
+
M --> I
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Quickstart
|
|
66
|
+
|
|
67
|
+
### Install
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
npm install
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The same command works in PowerShell and `cmd.exe`.
|
|
74
|
+
|
|
75
|
+
### Build
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
npm run build
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Verify the installation
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
npm run verify
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
### Run the fake-agent final demo (deterministic, no external CLIs required)
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
npm run run-final-demo -- \
|
|
91
|
+
--cases examples/token-savings-cases.json \
|
|
92
|
+
--out lab-output/final-demo \
|
|
93
|
+
--kit-command "node tests/fixtures/fake-my-dev-kit-cli.js" \
|
|
94
|
+
--agents fake-agent \
|
|
95
|
+
--complexities short \
|
|
96
|
+
--no-screenshot
|
|
97
|
+
```
|
|
10
98
|
|
|
11
99
|
```powershell
|
|
12
|
-
npm run
|
|
100
|
+
npm run run-final-demo -- `
|
|
101
|
+
--cases examples/token-savings-cases.json `
|
|
102
|
+
--out lab-output/final-demo `
|
|
103
|
+
--kit-command "node tests/fixtures/fake-my-dev-kit-cli.js" `
|
|
104
|
+
--agents fake-agent `
|
|
105
|
+
--complexities short `
|
|
106
|
+
--no-screenshot
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
```bat
|
|
110
|
+
npm run run-final-demo -- --cases examples/token-savings-cases.json --out lab-output/final-demo --kit-command "node tests/fixtures/fake-my-dev-kit-cli.js" --agents fake-agent --complexities short --no-screenshot
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
The lab resolves Windows `.cmd` and `.ps1` CLI shims, supports command paths with spaces, and keeps generated artifacts inside the requested output directory.
|
|
114
|
+
|
|
115
|
+
This runs a full pipeline: controlled experiment → report → plots → visualization demos → gallery.
|
|
116
|
+
|
|
117
|
+
### Run a real-agent campaign (requires Codex or Claude CLI)
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
npm run run-controlled-experiment -- \
|
|
121
|
+
--cases examples/real-agent-campaign-cases.json \
|
|
122
|
+
--agents codex,claude \
|
|
123
|
+
--strategies raw-full-file,my-dev-kit-guided \
|
|
124
|
+
--complexities medium,multi-step \
|
|
125
|
+
--out lab-output/real-agent-campaign \
|
|
126
|
+
--include-real-agents \
|
|
127
|
+
--continue-on-failure \
|
|
128
|
+
--timeout-ms 240000
|
|
13
129
|
```
|
|
14
130
|
|
|
15
|
-
|
|
131
|
+
Real-agent runs require local Codex or Claude CLI setup and available usage capacity. Runs that time out, produce invalid output, or hit session limits are recorded as structured outcomes rather than failures.
|
|
132
|
+
|
|
133
|
+
### List, describe, and run experiment plugins
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
npm run experiment:list
|
|
137
|
+
npm run experiment:describe -- --experiment context-strategy-comparison
|
|
138
|
+
npm run experiment:run -- \
|
|
139
|
+
--experiment context-strategy-comparison \
|
|
140
|
+
--target /path/to/local/project \
|
|
141
|
+
--agents fake-agent \
|
|
142
|
+
--complexities short \
|
|
143
|
+
--no-screenshot
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
```powershell
|
|
147
|
+
npm run experiment:list
|
|
148
|
+
npm run experiment:describe -- --experiment context-strategy-comparison
|
|
149
|
+
npm run experiment:run -- `
|
|
150
|
+
--experiment context-strategy-comparison `
|
|
151
|
+
--target "Z:\Users\newuser\Projects\my-dev-kit-v1" `
|
|
152
|
+
--agents fake-agent `
|
|
153
|
+
--complexities short `
|
|
154
|
+
--no-screenshot
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
When `--target` is omitted, the experiment runs in self mode against my-dev-kit-lab. When `--target <path>` is provided, the lab remains the tool root and the target project is inspected separately. Generated experiment outputs stay under lab-controlled output directories by default, not inside the target project.
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## Where to find outputs
|
|
162
|
+
|
|
163
|
+
| Artifact | Location |
|
|
164
|
+
|---|---|
|
|
165
|
+
| Experiment summary | `lab-output/<experiment>/experiment-summary.json` |
|
|
166
|
+
| All runs | `lab-output/<experiment>/experiment-runs.json` |
|
|
167
|
+
| Strategy comparisons | `lab-output/<experiment>/experiment-comparisons.json` |
|
|
168
|
+
| HTML report | `lab-output/<report>/experiment-report.html` |
|
|
169
|
+
| Report JSON | `lab-output/<report>/experiment-report.json` |
|
|
170
|
+
| Report screenshot | `lab-output/<report>/experiment-report.png` |
|
|
171
|
+
| Plugin experiment report JSON | `lab-output/experiments/<plugin>/<target>/<run>/report.json` |
|
|
172
|
+
| Plugin experiment report HTML | `lab-output/experiments/<plugin>/<target>/<run>/report.html` |
|
|
173
|
+
| Plot data | `lab-output/<plots>/plot-data.json` |
|
|
174
|
+
| SVG charts | `lab-output/<plots>/charts/*.svg` |
|
|
175
|
+
| Gallery manifest | `lab-output/<gallery>/gallery-manifest.json` |
|
|
176
|
+
| Gallery index | `lab-output/<gallery>/gallery-index.html` |
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## How to read the main report
|
|
181
|
+
|
|
182
|
+
Open `experiment-report.html` in a browser. The report shows:
|
|
183
|
+
|
|
184
|
+
- **Project profile** — benchmark project name, language mix, complexity score, and file tree
|
|
185
|
+
- **Benchmark tasks** — task descriptions and answer keys
|
|
186
|
+
- **Strategy comparisons** — paired `raw-full-file` vs `my-dev-kit-guided` runs per case
|
|
187
|
+
- **Correctness scores** — deterministic answer-key scoring (not semantic LLM judging)
|
|
188
|
+
- **Token usage** — estimated or reported token totals per run
|
|
189
|
+
- **Token savings** — positive means my-dev-kit used fewer tokens; negative means it used more
|
|
190
|
+
- **Duration** — wall-clock time per run
|
|
191
|
+
- **Status** — completed, timeout, invalid-output, or limit-reached
|
|
192
|
+
- **Warnings and limitations** — notes on missing token totals or partial results
|
|
193
|
+
|
|
194
|
+
See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
|
|
195
|
+
|
|
196
|
+
---
|
|
197
|
+
|
|
198
|
+
## Current limitations
|
|
199
|
+
|
|
200
|
+
- Token savings shown in fake-agent runs are based on estimated character counts, not provider billing telemetry
|
|
201
|
+
- Claude does not expose token totals; token savings comparisons are unavailable for Claude runs
|
|
202
|
+
- Codex may expose token totals but can produce timeouts or invalid-output runs
|
|
203
|
+
- Small projects may make raw-full-file cheaper than my-dev-kit-guided; larger localized tasks are where my-dev-kit is expected to become more useful
|
|
204
|
+
- The generic experiment-plugin framework currently ships one plugin, `context-strategy-comparison`; future plugins such as warm-index reuse, incremental-change, and context-window scaling are not implemented yet
|
|
205
|
+
- The current baseline does not prove token savings are guaranteed; it produces auditable evidence for specific cases, targets, agents, and strategies
|
|
206
|
+
- Provider telemetry dashboards, semantic LLM judging, and cloud API billing integration are not yet implemented
|
|
207
|
+
|
|
208
|
+
---
|
|
209
|
+
|
|
210
|
+
## Roadmap direction
|
|
211
|
+
|
|
212
|
+
my-dev-kit-lab is at a working baseline. The raw-vs-indexed experiment pipeline is fully implemented and produces reproducible artifacts, and real-agent campaign support exists for Codex and Claude. Recent releases added language-aware code-rot detectors (TypeScript/JavaScript, Python, Java, Kotlin), a security-validation audit adapter, and Android validation with an Android-aware extension of that same adapter.
|
|
213
|
+
|
|
214
|
+
- `v0.4.0` and `v0.4.1`: Android automated security validation (published)
|
|
215
|
+
- `v0.4.2`: Android-aware extension of the existing security audit adapter (see [CHANGELOG.md](CHANGELOG.md) for release status)
|
|
216
|
+
- manual pentest: post-v1 / version TBD
|
|
217
|
+
|
|
218
|
+
See [docs/ROADMAP.md](docs/ROADMAP.md) for the complete, versioned plan.
|
|
219
|
+
|
|
220
|
+
---
|
|
221
|
+
|
|
222
|
+
## Security validation
|
|
223
|
+
|
|
224
|
+
my-dev-kit-lab owns a release-security validation track for **my-dev-kit**. This work is separate from the experiment pipeline and does not replace the generic experiment-plugin roadmap. Its purpose is to generate release-validation evidence for the local CLI/package before release preparation.
|
|
225
|
+
|
|
226
|
+
`security:validate` remains the standalone, focused command for this validation and is unaffected by the current checked-out state. `npm run audit -- --types security` additionally adapts the same `security:validate` internals into the shared audit/report surface (mapped issues, a `securitySummary` field, and links to the original `reports/security/` output) — it does not replace or duplicate `security:validate`'s own checks or reports.
|
|
227
|
+
|
|
228
|
+
This is not a web application pentest framework. **my-dev-kit** is a local CLI/package, so the validation model is CLI/package adversarial testing focused on whether it remains:
|
|
229
|
+
|
|
230
|
+
- local-first
|
|
231
|
+
- deterministic
|
|
232
|
+
- read-only with respect to user source files
|
|
233
|
+
- network-free during normal CLI operation
|
|
234
|
+
- LLM-free
|
|
235
|
+
- database-free
|
|
236
|
+
- safe to run on local repositories
|
|
237
|
+
|
|
238
|
+
The automated validation gate is implemented. It combines static scans, dependency/package checks, adversarial CLI tests, bounded fuzz smoke tests, and attack-scenario checks with structured text/JSON reports and a four-category verdict.
|
|
239
|
+
|
|
240
|
+
### Security commands
|
|
241
|
+
|
|
242
|
+
| Command | Description |
|
|
243
|
+
|---|---|
|
|
244
|
+
| `npm run security:deps` | npm audit, OSV-Scanner (if available), outdated packages |
|
|
245
|
+
| `npm run security:package` | npm pack --dry-run, forbidden content detection |
|
|
246
|
+
| `npm run security:codeql` | CodeQL CLI availability check; skipped gracefully when absent |
|
|
247
|
+
| `npm run security:semgrep` | Semgrep scan via local binary or npx; skipped gracefully when both absent |
|
|
248
|
+
| `npm run test:security` | Automated security and adversarial CLI tests (path traversal, read-only boundaries, malformed artifacts, JSON safety, and related checks) |
|
|
249
|
+
| `npm run test:fuzz:smoke` | 9 bounded fuzz targets, seeded PRNG, completes in under 1 second |
|
|
250
|
+
| `npm run security:validate` | Runs selected security checks, applies profile-aware defaults when requested, and writes text/JSON reports according to `--format` |
|
|
251
|
+
|
|
252
|
+
CodeQL, Semgrep, and OSV-Scanner are optional. When unavailable locally, they are recorded as `skipped` in the report, not as passed. That can lead to `ready except optional manual checks`; it does not silently turn missing tooling into a clean pass.
|
|
253
|
+
|
|
254
|
+
Each security command can validate my-dev-kit-lab itself or another local project via `--target <path>`. When `--target` is omitted, the framework performs self-validation. Target projects are inspected in place: their source files are not modified by default, generated artifacts stay under `reports/security/` unless `--out` is used, and external-target reports identify the tool root, target root, target package metadata, target git metadata, command cwd, exit code, and stdout/stderr summaries.
|
|
255
|
+
|
|
256
|
+
`security:validate` supports `--checks`, `--profile`, `--format`, `--fail-on`, `--out`, and `--report-prefix`. The no-flag path remains backward compatible and still runs the classic implemented check groups: `deps`, `package`, `static`, `cli-adversarial`, and `fuzz`. Supplying `--profile` without `--checks` swaps in that profile's default checks; supplying explicit `--checks` always wins over profile defaults.
|
|
257
|
+
|
|
258
|
+
The current attack-scenario checks cover boundary, subprocess, secrets, and network assumptions. They are automated adversarial checks, not a manual pentest. Manual pentest is deferred until after `v1.0.0`. `verdictImpact` metadata from each registered scenario drives blocker categorization in report reasoning, and narrowed `--checks` runs are labeled as scoped rather than described as a full release gate.
|
|
259
|
+
|
|
260
|
+
JSON report poisoning/config-injection checks use a baseline-diff schema guard so legitimate additive JSON fields do not fail the guard while payload-created trusted top-level fields still do. Text reports are sanitized to strip ANSI/control-byte payloads before rendering.
|
|
261
|
+
|
|
262
|
+
Known limits remain intentionally explicit: secret scanning is bounded rather than exhaustive, the network/local-first check is a bounded static assumption check rather than proof of runtime isolation, package-boundary severity is still result-level rather than per-evidence-item, and profile-specific scenario behavior is limited to profile-based scenario selection/default checks.
|
|
263
|
+
|
|
264
|
+
For external-target validation, `security:validate` reads the target `package.json`, detects `scripts.test:security`, and runs `npm run test:security` in the target project root when that script exists. This behavior is validated both from the source checkout and from an installed packed tarball because published-package execution differs from local source execution.
|
|
265
|
+
|
|
266
|
+
Generated security reports under `reports/security/` are excluded from git by default. They are produced locally or in CI as release-gate evidence and are not committed to the repository.
|
|
267
|
+
|
|
268
|
+
See [docs/COMMANDS.md](docs/COMMANDS.md) for full command options and [docs/security-validation-framework.md](docs/security-validation-framework.md) for the security model, implemented modules, and release verdicts.
|
|
269
|
+
|
|
270
|
+
---
|
|
271
|
+
|
|
272
|
+
## License
|
|
273
|
+
|
|
274
|
+
MIT License. See [LICENSE](LICENSE) for the full text.
|
|
275
|
+
|
|
276
|
+
---
|
|
277
|
+
|
|
278
|
+
## Support
|
|
279
|
+
|
|
280
|
+
my-dev-kit-lab is an independent project by dailephd LLC, developed and maintained by Dai Le.
|
|
16
281
|
|
|
17
|
-
|
|
282
|
+
If this project helps your workflow, you can support continued development through GitHub Sponsors or PayPal:
|
|
18
283
|
|
|
19
|
-
|
|
284
|
+
- [Sponsor on GitHub](https://github.com/sponsors/dailephd)
|
|
285
|
+
- [Support via PayPal](https://paypal.me/daile88)
|
|
20
286
|
|
|
21
|
-
|
|
287
|
+
Support is optional and does not affect access to the project.
|
|
22
288
|
|
|
23
|
-
|
|
289
|
+
---
|
|
24
290
|
|
|
25
|
-
|
|
291
|
+
## Documentation
|
|
26
292
|
|
|
27
|
-
|
|
293
|
+
- [docs/PROJECT_OVERVIEW.md](docs/PROJECT_OVERVIEW.md) — product purpose and target users
|
|
294
|
+
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — current and future architecture
|
|
295
|
+
- [docs/WORKFLOWS.md](docs/WORKFLOWS.md) — step-by-step workflows with diagrams
|
|
296
|
+
- [docs/COMMANDS.md](docs/COMMANDS.md) — all commands with options and examples
|
|
297
|
+
- [docs/TUTORIAL.md](docs/TUTORIAL.md) — first-run walkthrough
|
|
298
|
+
- [docs/METRICS.md](docs/METRICS.md) — metric definitions and interpretation
|
|
299
|
+
- [docs/ROADMAP.md](docs/ROADMAP.md) — current baseline and future phases
|
|
300
|
+
- [docs/GALLERY.md](docs/GALLERY.md) — gallery output explained
|
|
@@ -20,9 +20,29 @@ import { writeAuditReports } from "../../src/audits/report/writeAuditReports.js"
|
|
|
20
20
|
// ---------------------------------------------------------------------------
|
|
21
21
|
const USAGE = "Usage: npm run audit -- [--target <path>] [--types code-rot,security] " +
|
|
22
22
|
"[--include docs,tests,package,architecture,cli] [--format text,json] " +
|
|
23
|
-
"[--fail-on blocker|high|medium|low|none] [--out <path>]"
|
|
23
|
+
"[--fail-on blocker|high|medium|low|none] [--out <path>] [--android]\n" +
|
|
24
|
+
"\n" +
|
|
25
|
+
" --android opts into Batch 2's programmatic Android security integration.\n" +
|
|
26
|
+
" Requires --types to include \"security\" (e.g. --types security or --types\n" +
|
|
27
|
+
" code-rot,security). Runs the same static, read-only, nineteen-check\n" +
|
|
28
|
+
" Android validation security:validate --profile android performs --\n" +
|
|
29
|
+
" detection, manifest parsing, and internal advanced security checks --\n" +
|
|
30
|
+
" through the existing security adapter, never a subprocess. Confirmed\n" +
|
|
31
|
+
" findings are mapped into the normal issue collection; CandidateEvidence\n" +
|
|
32
|
+
" remains a separate, bounded summary and is never treated as a confirmed\n" +
|
|
33
|
+
" issue. Starts zero Gradle operations, zero external tools, and zero\n" +
|
|
34
|
+
" network operations by default; --android does not expose any of those.\n" +
|
|
35
|
+
" Example: npm run audit -- --target \"<android-project-path>\" --types security --android --format text,json --fail-on none";
|
|
24
36
|
const rawArgs = process.argv.slice(2);
|
|
25
37
|
const toolRoot = resolveToolRoot(import.meta.url);
|
|
38
|
+
// Help deliberately wins over every other argument, mirroring
|
|
39
|
+
// scripts/security/validate.ts -- no config normalization, target
|
|
40
|
+
// resolution, audit execution, or report write.
|
|
41
|
+
if (rawArgs.includes("--help") || rawArgs.includes("-h")) {
|
|
42
|
+
console.log(USAGE);
|
|
43
|
+
process.exitCode = 0;
|
|
44
|
+
process.exit(0);
|
|
45
|
+
}
|
|
26
46
|
let config;
|
|
27
47
|
try {
|
|
28
48
|
const args = parseAuditArgs(rawArgs);
|
|
@@ -89,6 +109,13 @@ if (model.securitySummary.ran) {
|
|
|
89
109
|
console.log(` Full report: ${model.securitySummary.reportPaths.text}`);
|
|
90
110
|
}
|
|
91
111
|
}
|
|
112
|
+
if (model.androidSecurity.summary.requested) {
|
|
113
|
+
const android = model.androidSecurity.summary;
|
|
114
|
+
console.log(`\nAndroid security validation: status=${android.status} applicable=${android.applicable ?? "n/a"} verdict=${android.verdict ?? "(unknown)"} checks=${android.totalChecks} confirmed=${android.confirmedFindingCount} mappedIssues=${android.mappedIssueCount} candidates=${android.candidateSummary.totalCount}`);
|
|
115
|
+
if (android.reportPaths.text) {
|
|
116
|
+
console.log(` Full report: ${android.reportPaths.text}`);
|
|
117
|
+
}
|
|
118
|
+
}
|
|
92
119
|
console.log(`\nInventory: ${model.inventory.totalScannedFileCount} file(s) scanned, ${model.inventory.skippedFileCount} skipped`);
|
|
93
120
|
console.log(` source=${model.inventory.filesByCategory.source} tests=${model.inventory.filesByCategory.tests} docs=${model.inventory.filesByCategory.docs} package=${model.inventory.filesByCategory.package} config=${model.inventory.filesByCategory.config} scripts=${model.inventory.filesByCategory.scripts} ci=${model.inventory.filesByCategory.ci}`);
|
|
94
121
|
console.log(`Source of truth: package=${model.sourceOfTruth.packageName ?? "(none)"}@${model.sourceOfTruth.packageVersion ?? "?"} readme=${model.sourceOfTruth.hasReadme} changelog=${model.sourceOfTruth.hasChangelog} ciWorkflows=${model.sourceOfTruth.ciWorkflowCount}`);
|
|
@@ -8,10 +8,19 @@ const FLAGS_WITH_VALUE = {
|
|
|
8
8
|
"--fail-on": "failOn",
|
|
9
9
|
"--out": "out",
|
|
10
10
|
};
|
|
11
|
+
// v0.4.2 Batch 3 -- presence-only flags (no following value consumed).
|
|
12
|
+
const BOOLEAN_FLAGS = {
|
|
13
|
+
"--android": "android",
|
|
14
|
+
};
|
|
11
15
|
export function parseAuditArgs(argv) {
|
|
12
16
|
const result = {};
|
|
13
17
|
for (let i = 0; i < argv.length; i++) {
|
|
14
18
|
const arg = argv[i];
|
|
19
|
+
const booleanKey = BOOLEAN_FLAGS[arg];
|
|
20
|
+
if (booleanKey) {
|
|
21
|
+
result[booleanKey] = true;
|
|
22
|
+
continue;
|
|
23
|
+
}
|
|
15
24
|
const key = FLAGS_WITH_VALUE[arg];
|
|
16
25
|
if (!key)
|
|
17
26
|
continue;
|
|
@@ -28,6 +37,10 @@ export function normalizeAuditConfig(raw, toolRoot) {
|
|
|
28
37
|
const formats = parseFormatOption(raw.format);
|
|
29
38
|
const failOn = parseFailOnOption(raw.failOn);
|
|
30
39
|
const out = raw.out !== undefined ? resolveOutDir(raw.out) : defaultOutDir(toolRoot, types.value);
|
|
40
|
+
const android = raw.android === true;
|
|
41
|
+
if (android && !types.value.includes("security")) {
|
|
42
|
+
throw new Error(`Invalid combination: --android requires --types to include "security" (e.g. --types security or --types code-rot,security).`);
|
|
43
|
+
}
|
|
31
44
|
return {
|
|
32
45
|
targetPathArg: raw.target,
|
|
33
46
|
targetMode: raw.target === undefined ? "self" : "external",
|
|
@@ -46,7 +59,9 @@ export function normalizeAuditConfig(raw, toolRoot) {
|
|
|
46
59
|
raw.include === undefined &&
|
|
47
60
|
raw.format === undefined &&
|
|
48
61
|
raw.failOn === undefined &&
|
|
49
|
-
raw.out === undefined
|
|
62
|
+
raw.out === undefined &&
|
|
63
|
+
raw.android === undefined,
|
|
64
|
+
android,
|
|
50
65
|
};
|
|
51
66
|
}
|
|
52
67
|
function splitAndNormalizeList(raw, flagName) {
|
|
@@ -6,7 +6,7 @@ import { collectSourceOfTruth } from "./sourceOfTruth.js";
|
|
|
6
6
|
import { collectSourceFacts } from "./collectSourceFacts.js";
|
|
7
7
|
import { collectPythonProjectMetadata } from "./pythonProjectMetadata.js";
|
|
8
8
|
import { runSecurityAuditAdapter } from "../security/securityAuditAdapter.js";
|
|
9
|
-
import { SECURITY_AUDIT_NOT_RUN_SUMMARY } from "../security/securityAuditTypes.js";
|
|
9
|
+
import { ANDROID_AUDIT_NOT_REQUESTED_SUMMARY, SECURITY_AUDIT_NOT_RUN_SUMMARY, } from "../security/securityAuditTypes.js";
|
|
10
10
|
export async function runAudit(options) {
|
|
11
11
|
const { config, toolRoot } = options;
|
|
12
12
|
const registry = options.registry ?? DEFAULT_AUDIT_REGISTRY;
|
|
@@ -49,10 +49,12 @@ export async function runAudit(options) {
|
|
|
49
49
|
// code-rot detector issues (registry order) first, then security findings
|
|
50
50
|
// (security-validation check-execution order).
|
|
51
51
|
let securitySummary = SECURITY_AUDIT_NOT_RUN_SUMMARY;
|
|
52
|
+
let androidSummary = ANDROID_AUDIT_NOT_REQUESTED_SUMMARY;
|
|
52
53
|
if (config.types.includes("security")) {
|
|
53
|
-
const securityAudit = await runSecurityAuditAdapter({ toolRoot, config });
|
|
54
|
+
const securityAudit = await runSecurityAuditAdapter({ toolRoot, config, android: config.android ? { enabled: true } : undefined }, options.securityDependencies);
|
|
54
55
|
issues.push(...securityAudit.issues);
|
|
55
56
|
securitySummary = securityAudit.summary;
|
|
57
|
+
androidSummary = securityAudit.android;
|
|
56
58
|
}
|
|
57
59
|
const issueCounts = countIssuesBySeverity(issues);
|
|
58
60
|
const { exitCode, reason } = calculateAuditExitCode(issues, config.failOn);
|
|
@@ -64,6 +66,7 @@ export async function runAudit(options) {
|
|
|
64
66
|
failOn: config.failOn,
|
|
65
67
|
out: config.out,
|
|
66
68
|
isDefaultRun: config.isDefaultRun,
|
|
69
|
+
android: config.android,
|
|
67
70
|
},
|
|
68
71
|
targetSummary: {
|
|
69
72
|
rootPath: target.rootPath,
|
|
@@ -81,6 +84,7 @@ export async function runAudit(options) {
|
|
|
81
84
|
sourceFacts,
|
|
82
85
|
pythonProjectMetadata,
|
|
83
86
|
securitySummary,
|
|
87
|
+
androidSummary,
|
|
84
88
|
noDetectorsRegistered: selected.length === 0,
|
|
85
89
|
};
|
|
86
90
|
}
|
|
@@ -166,6 +166,7 @@ export function buildAuditReportModel(result, opts) {
|
|
|
166
166
|
failOn: result.configSummary.failOn,
|
|
167
167
|
out: result.configSummary.out,
|
|
168
168
|
isDefaultRun: result.configSummary.isDefaultRun,
|
|
169
|
+
android: result.configSummary.android,
|
|
169
170
|
},
|
|
170
171
|
summary: {
|
|
171
172
|
totalIssues: issues.length,
|
|
@@ -191,6 +192,10 @@ export function buildAuditReportModel(result, opts) {
|
|
|
191
192
|
sourceFacts: summarizeSourceFacts(result.sourceFacts),
|
|
192
193
|
pythonProjectMetadata: result.pythonProjectMetadata,
|
|
193
194
|
securitySummary: result.securitySummary,
|
|
195
|
+
androidSecurity: {
|
|
196
|
+
summary: result.androidSummary,
|
|
197
|
+
candidates: result.androidSummary.candidateSummary,
|
|
198
|
+
},
|
|
194
199
|
detectors,
|
|
195
200
|
issues,
|
|
196
201
|
skippedDetectors: result.skippedDetectors,
|
|
@@ -252,5 +257,7 @@ function reconstructCommand(result) {
|
|
|
252
257
|
parts.push("--format", cfg.formats.join(","));
|
|
253
258
|
parts.push("--fail-on", cfg.failOn);
|
|
254
259
|
parts.push("--out", cfg.out);
|
|
260
|
+
if (cfg.android)
|
|
261
|
+
parts.push("--android");
|
|
255
262
|
return parts.join(" ");
|
|
256
263
|
}
|
|
@@ -21,6 +21,7 @@ export function renderAuditJsonReport(model) {
|
|
|
21
21
|
sourceFacts: model.sourceFacts,
|
|
22
22
|
pythonProjectMetadata: model.pythonProjectMetadata,
|
|
23
23
|
securitySummary: model.securitySummary,
|
|
24
|
+
androidSecurity: model.androidSecurity,
|
|
24
25
|
detectors: model.detectors,
|
|
25
26
|
issues: model.issues,
|
|
26
27
|
skippedDetectors: model.skippedDetectors,
|
|
@@ -133,6 +133,43 @@ export function renderAuditTextReport(model) {
|
|
|
133
133
|
}
|
|
134
134
|
}
|
|
135
135
|
lines.push("");
|
|
136
|
+
// v0.4.2 Batch 3 -- bounded summary only, mirroring the "Security
|
|
137
|
+
// validation summary" section above exactly. Confirmed mapped Android
|
|
138
|
+
// findings are NOT re-listed here -- they already appear in the "Issues"
|
|
139
|
+
// section below like any other AuditIssue (see spec section 12: "the
|
|
140
|
+
// existing generic issues section remains the detailed presentation for
|
|
141
|
+
// confirmed mapped Android findings"). CandidateEvidence bodies never
|
|
142
|
+
// appear here either -- only counts, with a pointer to the full Android
|
|
143
|
+
// report for detail.
|
|
144
|
+
lines.push("Android security validation summary");
|
|
145
|
+
lines.push(divider("-"));
|
|
146
|
+
if (!model.androidSecurity.summary.requested) {
|
|
147
|
+
lines.push(" (not requested -- add --android alongside --types security to include it)");
|
|
148
|
+
}
|
|
149
|
+
else {
|
|
150
|
+
const android = model.androidSecurity.summary;
|
|
151
|
+
const candidates = model.androidSecurity.candidates;
|
|
152
|
+
const applicabilityNote = android.applicable === false ? " (target is not an Android project)" : "";
|
|
153
|
+
lines.push(` Status: ${android.status}${applicabilityNote}`);
|
|
154
|
+
lines.push(` Verdict: ${android.verdict ?? "(unknown)"}`);
|
|
155
|
+
lines.push(` Checks: total=${android.totalChecks} passed=${android.passedChecks} withFindings=${android.checksWithFindings} candidateOnly=${android.candidateOnlyChecks} skipped=${android.skippedChecks} inconclusive=${android.inconclusiveChecks} failed=${android.failedChecks}`);
|
|
156
|
+
lines.push(` Findings: confirmed=${android.confirmedFindingCount} mappedIssues=${android.mappedIssueCount}`);
|
|
157
|
+
lines.push(` Candidates (review evidence, not confirmed vulnerabilities): total=${candidates.totalCount} checksWithCandidates=${candidates.checksWithCandidates}`);
|
|
158
|
+
if (android.warnings.length > 0) {
|
|
159
|
+
lines.push(` Warnings: ${android.warnings.length}`);
|
|
160
|
+
}
|
|
161
|
+
if (android.errors.length > 0) {
|
|
162
|
+
lines.push(` Errors: ${android.errors.length}`);
|
|
163
|
+
for (const e of android.errors.slice(0, 5)) {
|
|
164
|
+
lines.push(` ${sanitizeLine(e)}`);
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
if (android.reportPaths.text || android.reportPaths.json) {
|
|
168
|
+
lines.push(` Full report: ${android.reportPaths.text ?? "(none)"} / ${android.reportPaths.json ?? "(none)"}`);
|
|
169
|
+
}
|
|
170
|
+
lines.push(" Static analysis only -- does not prove runtime behavior, APK/AAB validity, Play compliance, or pentest results.");
|
|
171
|
+
}
|
|
172
|
+
lines.push("");
|
|
136
173
|
lines.push("Issue summary");
|
|
137
174
|
lines.push(divider("-"));
|
|
138
175
|
lines.push(` Total issues: ${model.summary.totalIssues}`);
|
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
import path from "node:path";
|
|
2
|
+
import { validateAndroidTarget } from "../../mobile/android/validate/validateAndroidTarget.js";
|
|
3
|
+
import { toAndroidReportModel } from "../../mobile/android/report/model.js";
|
|
4
|
+
import { writeAndroidReportFiles } from "../../mobile/android/report/writeAndroidReportFiles.js";
|
|
5
|
+
import { reportFilenamePrefix } from "../../securityValidation/validate/resolveTarget.js";
|
|
6
|
+
import { mapAndroidSecurityFindingToAuditIssue } from "./mapAndroidSecurityFindingToAuditIssue.js";
|
|
7
|
+
import { ANDROID_AUDIT_NOT_REQUESTED_SUMMARY, EMPTY_ANDROID_AUDIT_CANDIDATE_SUMMARY, } from "./securityAuditTypes.js";
|
|
8
|
+
const ANDROID_REPORT_SUBDIR = "android";
|
|
9
|
+
// Bounded, redacted — never the thrown value's stack, never raw child-process
|
|
10
|
+
// output (there is none at this boundary; validateAndroidTarget itself never
|
|
11
|
+
// spawns a process unless Gradle/external-tool options are requested, which
|
|
12
|
+
// Batch 2 never supplies).
|
|
13
|
+
const MAX_ANDROID_ERROR_MESSAGE_LENGTH = 500;
|
|
14
|
+
function boundErrorMessage(message) {
|
|
15
|
+
return message.length <= MAX_ANDROID_ERROR_MESSAGE_LENGTH
|
|
16
|
+
? message
|
|
17
|
+
: `${message.slice(0, MAX_ANDROID_ERROR_MESSAGE_LENGTH)}... [truncated, ${message.length} chars total]`;
|
|
18
|
+
}
|
|
19
|
+
function failedSummary(errorMessage) {
|
|
20
|
+
return {
|
|
21
|
+
...ANDROID_AUDIT_NOT_REQUESTED_SUMMARY,
|
|
22
|
+
requested: true,
|
|
23
|
+
applicable: null,
|
|
24
|
+
status: "failed",
|
|
25
|
+
errors: [boundErrorMessage(errorMessage)],
|
|
26
|
+
};
|
|
27
|
+
}
|
|
28
|
+
// Exact-duplicate dedupe by AuditIssue id only, first occurrence wins —
|
|
29
|
+
// mirrors validateAndroidTarget's own findingsById convention. Iterates
|
|
30
|
+
// checks in their own fixed order (never re-sorted) and each check's own
|
|
31
|
+
// findings in their own order — never the top-level, already-flattened
|
|
32
|
+
// result.findings, which has lost per-check identity this mapper needs as
|
|
33
|
+
// context. Exported standalone so aggregation correctness (ordering,
|
|
34
|
+
// dedup, provenance) can be tested against small literal AndroidCheckResult
|
|
35
|
+
// fixtures without needing a full AndroidValidationResult.
|
|
36
|
+
export function aggregateAndroidIssues(checks, reportReference) {
|
|
37
|
+
const issuesById = new Map();
|
|
38
|
+
for (const check of checks) {
|
|
39
|
+
for (const finding of check.findings) {
|
|
40
|
+
const issue = mapAndroidSecurityFindingToAuditIssue(finding, {
|
|
41
|
+
checkId: check.id,
|
|
42
|
+
checkCategory: check.category,
|
|
43
|
+
checkConfidence: check.confidence,
|
|
44
|
+
reportReference,
|
|
45
|
+
});
|
|
46
|
+
if (!issuesById.has(issue.id)) {
|
|
47
|
+
issuesById.set(issue.id, issue);
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
return [...issuesById.values()];
|
|
52
|
+
}
|
|
53
|
+
export function buildCandidateSummary(checks) {
|
|
54
|
+
const byConfidence = { ...EMPTY_ANDROID_AUDIT_CANDIDATE_SUMMARY.byConfidence };
|
|
55
|
+
const byResolutionState = { ...EMPTY_ANDROID_AUDIT_CANDIDATE_SUMMARY.byResolutionState };
|
|
56
|
+
const byCategory = {};
|
|
57
|
+
let totalCount = 0;
|
|
58
|
+
let checksWithCandidates = 0;
|
|
59
|
+
for (const check of checks) {
|
|
60
|
+
const candidates = check.candidateEvidence ?? [];
|
|
61
|
+
if (candidates.length === 0)
|
|
62
|
+
continue;
|
|
63
|
+
checksWithCandidates += 1;
|
|
64
|
+
for (const candidate of candidates) {
|
|
65
|
+
totalCount += 1;
|
|
66
|
+
byConfidence[candidate.confidence] += 1;
|
|
67
|
+
byResolutionState[candidate.resolutionState] += 1;
|
|
68
|
+
byCategory[candidate.category] = (byCategory[candidate.category] ?? 0) + 1;
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
// Deterministic key order regardless of check/candidate traversal order.
|
|
72
|
+
const sortedByCategory = Object.fromEntries(Object.entries(byCategory).sort(([a], [b]) => a.localeCompare(b)));
|
|
73
|
+
return { totalCount, checksWithCandidates, byConfidence, byResolutionState, byCategory: sortedByCategory };
|
|
74
|
+
}
|
|
75
|
+
export async function runAndroidAuditIntegration(options) {
|
|
76
|
+
const runAndroidValidation = options.runAndroidValidation ?? validateAndroidTarget;
|
|
77
|
+
let result;
|
|
78
|
+
try {
|
|
79
|
+
result = await runAndroidValidation({
|
|
80
|
+
toolRoot: options.toolRoot,
|
|
81
|
+
targetPath: options.targetPathArg,
|
|
82
|
+
});
|
|
83
|
+
}
|
|
84
|
+
catch (error) {
|
|
85
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
86
|
+
return { issues: [], summary: failedSummary(`Android validation failed to run: ${message}`) };
|
|
87
|
+
}
|
|
88
|
+
const isApplicable = result.detection.projectKind !== "non-android";
|
|
89
|
+
const reportModel = toAndroidReportModel(result, { profile: "android" });
|
|
90
|
+
const reportDir = path.join(options.toolRoot, "reports", "security", ANDROID_REPORT_SUBDIR);
|
|
91
|
+
const prefix = reportFilenamePrefix({
|
|
92
|
+
isSelf: result.target.local.isSelf,
|
|
93
|
+
packageName: result.target.local.packageName,
|
|
94
|
+
packageVersion: result.target.local.packageVersion,
|
|
95
|
+
targetRoot: result.target.local.targetRoot,
|
|
96
|
+
});
|
|
97
|
+
let reportPaths;
|
|
98
|
+
let writeErrors = [];
|
|
99
|
+
try {
|
|
100
|
+
const written = writeAndroidReportFiles({ outDir: reportDir, prefix, report: reportModel, formats: ["text", "json"] });
|
|
101
|
+
reportPaths = { text: written.textPath, json: written.jsonPath };
|
|
102
|
+
}
|
|
103
|
+
catch (error) {
|
|
104
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
105
|
+
reportPaths = { text: null, json: null };
|
|
106
|
+
writeErrors = [boundErrorMessage(`Failed to write Android report files: ${message}`)];
|
|
107
|
+
}
|
|
108
|
+
const issues = aggregateAndroidIssues(result.checks, reportPaths);
|
|
109
|
+
const passedChecks = result.checks.filter((c) => c.status === "passed").length;
|
|
110
|
+
const skippedChecks = result.checks.filter((c) => c.status === "skipped").length;
|
|
111
|
+
const inconclusiveChecks = result.checks.filter((c) => c.status === "inconclusive").length;
|
|
112
|
+
const failedChecks = result.checks.filter((c) => c.status === "failed" || c.status === "error").length;
|
|
113
|
+
const checksWithFindings = result.checks.filter((c) => c.findings.length > 0).length;
|
|
114
|
+
const candidateOnlyChecks = result.checks.filter((c) => c.findings.length === 0 && (c.candidateEvidence?.length ?? 0) > 0).length;
|
|
115
|
+
const candidateEvidenceCount = result.checks.reduce((sum, c) => sum + (c.candidateEvidence?.length ?? 0), 0);
|
|
116
|
+
const summary = {
|
|
117
|
+
requested: true,
|
|
118
|
+
applicable: isApplicable,
|
|
119
|
+
status: "completed",
|
|
120
|
+
verdict: result.verdict,
|
|
121
|
+
totalChecks: result.checks.length,
|
|
122
|
+
passedChecks,
|
|
123
|
+
checksWithFindings,
|
|
124
|
+
candidateOnlyChecks,
|
|
125
|
+
skippedChecks,
|
|
126
|
+
inconclusiveChecks,
|
|
127
|
+
failedChecks,
|
|
128
|
+
confirmedFindingCount: result.findings.length,
|
|
129
|
+
mappedIssueCount: issues.length,
|
|
130
|
+
candidateEvidenceCount,
|
|
131
|
+
// No Gradle/external-tool programmatic passthrough exists yet in Batch 2
|
|
132
|
+
// (see securityAuditTypes.ts's AndroidAuditSummary comment) — the default
|
|
133
|
+
// request always executes zero of either.
|
|
134
|
+
requestedGradleOperationCount: 0,
|
|
135
|
+
executedGradleOperationCount: 0,
|
|
136
|
+
requestedExternalToolCount: 0,
|
|
137
|
+
executedExternalToolCount: 0,
|
|
138
|
+
skippedExternalToolCount: 0,
|
|
139
|
+
reportPaths,
|
|
140
|
+
warnings: result.warnings,
|
|
141
|
+
errors: [...result.errors, ...writeErrors],
|
|
142
|
+
candidateSummary: buildCandidateSummary(result.checks),
|
|
143
|
+
};
|
|
144
|
+
return { issues, summary };
|
|
145
|
+
}
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
import { makeAuditEvidence } from "../core/auditIssue.js";
|
|
2
|
+
import { isBlockingSecuritySeverity, mapSecuritySeverityToAuditSeverity } from "./mapSecurityFindingToAuditIssue.js";
|
|
3
|
+
// ---------------------------------------------------------------------------
|
|
4
|
+
// v0.4.2 Batch 1 — pure Android SecurityFinding -> AuditIssue mapper.
|
|
5
|
+
//
|
|
6
|
+
// Android checks already reuse SecurityFinding as-is (see
|
|
7
|
+
// src/mobile/android/audit/androidFinding.ts's own header comment) — there is
|
|
8
|
+
// no parallel "AndroidSecurityFinding" type. What Android findings lose by
|
|
9
|
+
// reusing the generic SecurityFinding shape is per-family taxonomy: every
|
|
10
|
+
// Android finding's own `category` is the single shared value "static-scan"
|
|
11
|
+
// (a SecurityCheckCategory), while the *specific* Android family (backup,
|
|
12
|
+
// secrets, webview, ...) only lives on the owning AndroidCheckResult.category.
|
|
13
|
+
// This mapper therefore takes a small AndroidAuditMappingContext alongside
|
|
14
|
+
// the finding — supplied by the (future Batch 2) caller that already has the
|
|
15
|
+
// owning AndroidCheckResult in hand — rather than trying to recover that
|
|
16
|
+
// context by parsing the finding's own opaque id/evidence text.
|
|
17
|
+
//
|
|
18
|
+
// Reuses mapSecurityFindingToAuditIssue's severity/blocking policy exactly
|
|
19
|
+
// (SecuritySeverity is the same shared type for Android and non-Android
|
|
20
|
+
// findings) instead of defining a second severity table. Pure, no I/O, no
|
|
21
|
+
// mutation, no CandidateEvidence handling — CandidateEvidence is a distinct
|
|
22
|
+
// type (see src/mobile/android/advancedSecurity/candidateEvidence.ts) and
|
|
23
|
+
// cannot be passed here at all: this function's parameter type is
|
|
24
|
+
// SecurityFinding, not CandidateEvidence.
|
|
25
|
+
// ---------------------------------------------------------------------------
|
|
26
|
+
export const ANDROID_SECURITY_AUDIT_DETECTOR_ID = "android-security-validation-adapter";
|
|
27
|
+
const DEFAULT_CONFIDENCE = "medium";
|
|
28
|
+
const DEFAULT_FALSE_POSITIVE_RISK = "medium";
|
|
29
|
+
// MobileConfidence carries an extra "unknown" state SecurityFinding-derived
|
|
30
|
+
// AuditConfidence has no equivalent for; mapped to the same conservative
|
|
31
|
+
// "medium" default the generic security mapper uses when no per-finding
|
|
32
|
+
// confidence exists at all, rather than inventing a fourth AuditConfidence
|
|
33
|
+
// value.
|
|
34
|
+
function mapCheckConfidenceToAuditConfidence(confidence) {
|
|
35
|
+
if (confidence === "high" || confidence === "medium" || confidence === "low")
|
|
36
|
+
return confidence;
|
|
37
|
+
return DEFAULT_CONFIDENCE;
|
|
38
|
+
}
|
|
39
|
+
// makeAndroidFinding (src/mobile/android/audit/androidFinding.ts) is the sole
|
|
40
|
+
// place Android findings compose their `evidence` string, always in the exact
|
|
41
|
+
// form `location=<relativePath>(:<line>)?` (optionally followed by further
|
|
42
|
+
// ` | `-joined detail segments). Extracting the line from that well-known,
|
|
43
|
+
// single-producer format is not fabricating a source position — it is
|
|
44
|
+
// reading a real value the same codebase already wrote, deterministically.
|
|
45
|
+
// Guarded so a value is only trusted when its path matches the finding's own
|
|
46
|
+
// affectedFiles[0]; anything else is left absent rather than guessed.
|
|
47
|
+
const LOCATION_EVIDENCE_PATTERN = /(?:^|\s\|\s)location=([^\s|]+)/;
|
|
48
|
+
function extractAndroidFindingLine(finding) {
|
|
49
|
+
if (!finding.evidence)
|
|
50
|
+
return undefined;
|
|
51
|
+
const relativePath = finding.affectedFiles?.[0];
|
|
52
|
+
if (!relativePath)
|
|
53
|
+
return undefined;
|
|
54
|
+
const match = LOCATION_EVIDENCE_PATTERN.exec(finding.evidence);
|
|
55
|
+
if (!match)
|
|
56
|
+
return undefined;
|
|
57
|
+
const [locationPath, lineText] = match[1].split(":");
|
|
58
|
+
if (locationPath !== relativePath || lineText === undefined)
|
|
59
|
+
return undefined;
|
|
60
|
+
const line = Number(lineText);
|
|
61
|
+
return Number.isInteger(line) && line > 0 ? line : undefined;
|
|
62
|
+
}
|
|
63
|
+
export function mapAndroidSecurityFindingToAuditIssue(finding, context) {
|
|
64
|
+
const severity = mapSecuritySeverityToAuditSeverity(finding.severity);
|
|
65
|
+
const isBlockingSeverity = isBlockingSecuritySeverity(finding.severity);
|
|
66
|
+
const relativePath = finding.affectedFiles?.[0];
|
|
67
|
+
const line = extractAndroidFindingLine(finding);
|
|
68
|
+
const provenanceSuffix = context.externalToolId ? `, tool=${context.externalToolId}` : "";
|
|
69
|
+
const evidence = finding.evidence
|
|
70
|
+
? [
|
|
71
|
+
makeAuditEvidence({
|
|
72
|
+
kind: "observation",
|
|
73
|
+
message: `Android security validation (${context.checkId}${provenanceSuffix}): ${finding.title}`,
|
|
74
|
+
excerpt: finding.evidence,
|
|
75
|
+
filePath: relativePath,
|
|
76
|
+
line,
|
|
77
|
+
source: `android-security-validation:${context.checkId}`,
|
|
78
|
+
confidence: mapCheckConfidenceToAuditConfidence(context.checkConfidence),
|
|
79
|
+
}),
|
|
80
|
+
]
|
|
81
|
+
: [];
|
|
82
|
+
const reportPath = context.reportReference?.json ?? context.reportReference?.text ?? undefined;
|
|
83
|
+
if (reportPath) {
|
|
84
|
+
evidence.push(makeAuditEvidence({
|
|
85
|
+
kind: "reference",
|
|
86
|
+
message: "Original Android security-validation report",
|
|
87
|
+
filePath: reportPath,
|
|
88
|
+
source: "android-security-validation:report",
|
|
89
|
+
confidence: DEFAULT_CONFIDENCE,
|
|
90
|
+
}));
|
|
91
|
+
}
|
|
92
|
+
return {
|
|
93
|
+
id: `android:${context.checkId}:${finding.id}`,
|
|
94
|
+
auditType: "security",
|
|
95
|
+
detectorId: ANDROID_SECURITY_AUDIT_DETECTOR_ID,
|
|
96
|
+
title: finding.title,
|
|
97
|
+
description: finding.description,
|
|
98
|
+
severity,
|
|
99
|
+
confidence: mapCheckConfidenceToAuditConfidence(context.checkConfidence),
|
|
100
|
+
falsePositiveRisk: DEFAULT_FALSE_POSITIVE_RISK,
|
|
101
|
+
// Deliberately the Android-specific check category (e.g.
|
|
102
|
+
// "android-backup-configuration"), not finding.category — every Android
|
|
103
|
+
// finding's own `category` is the single shared "static-scan" value, so
|
|
104
|
+
// reusing it here would make every mapped Android issue's category
|
|
105
|
+
// uselessly uniform. See module header comment.
|
|
106
|
+
category: context.checkCategory,
|
|
107
|
+
evidence,
|
|
108
|
+
affectedFiles: finding.affectedFiles ?? [],
|
|
109
|
+
recommendedAction: finding.recommendation ?? "Review the Android security validation report for remediation guidance.",
|
|
110
|
+
suggestedFixStrategy: finding.recommendation ?? "See the linked Android security validation report for details.",
|
|
111
|
+
validationCommands: ["npm run security:validate -- --profile android"],
|
|
112
|
+
releaseBlocking: isBlockingSeverity,
|
|
113
|
+
implementationBlocking: isBlockingSeverity,
|
|
114
|
+
autoFixEligible: false,
|
|
115
|
+
};
|
|
116
|
+
}
|
|
@@ -31,12 +31,22 @@ const SEVERITY_MAP = {
|
|
|
31
31
|
// false positives) nor discounts it as unreliable.
|
|
32
32
|
const DEFAULT_CONFIDENCE = "medium";
|
|
33
33
|
const DEFAULT_FALSE_POSITIVE_RISK = "medium";
|
|
34
|
+
// Exported so other SecurityFinding-consuming mappers (e.g. the Android
|
|
35
|
+
// audit-adapter mapper, v0.4.2 Batch 1) reuse the same conservative
|
|
36
|
+
// severity policy instead of defining a second one.
|
|
37
|
+
export function mapSecuritySeverityToAuditSeverity(severity) {
|
|
38
|
+
return SEVERITY_MAP[severity];
|
|
39
|
+
}
|
|
40
|
+
// Mirrors verdict.ts's calculateVerdict: blocker and major findings are what
|
|
41
|
+
// drive a "not-ready-security-blocker-remains" verdict — minor/informational
|
|
42
|
+
// findings never block release or implementation on their own. Exported for
|
|
43
|
+
// reuse by other SecurityFinding-consuming mappers.
|
|
44
|
+
export function isBlockingSecuritySeverity(severity) {
|
|
45
|
+
return severity === "blocker" || severity === "major";
|
|
46
|
+
}
|
|
34
47
|
export function mapSecurityFindingToAuditIssue(finding) {
|
|
35
|
-
const severity =
|
|
36
|
-
|
|
37
|
-
// what drive a "not-ready-security-blocker-remains" verdict — minor/
|
|
38
|
-
// informational findings never block release or implementation on their own.
|
|
39
|
-
const isBlockingSeverity = finding.severity === "blocker" || finding.severity === "major";
|
|
48
|
+
const severity = mapSecuritySeverityToAuditSeverity(finding.severity);
|
|
49
|
+
const isBlockingSeverity = isBlockingSecuritySeverity(finding.severity);
|
|
40
50
|
const evidence = finding.evidence
|
|
41
51
|
? [
|
|
42
52
|
makeAuditEvidence({
|
|
@@ -6,7 +6,9 @@ import { writeSecurityReportFiles } from "../../securityValidation/report/writeS
|
|
|
6
6
|
import { DEFAULT_SECURITY_CHECKS, DEFAULT_SECURITY_PROFILE } from "../../securityValidation/validate/cliOptions.js";
|
|
7
7
|
import { verdictToHumanLabel } from "../../securityValidation/validate/verdict.js";
|
|
8
8
|
import { mapSecurityFindingToAuditIssue } from "./mapSecurityFindingToAuditIssue.js";
|
|
9
|
-
|
|
9
|
+
import { runAndroidAuditIntegration } from "./androidAuditIntegration.js";
|
|
10
|
+
import { ANDROID_AUDIT_NOT_REQUESTED_SUMMARY, } from "./securityAuditTypes.js";
|
|
11
|
+
export async function runSecurityAuditAdapter(options, dependencies = {}) {
|
|
10
12
|
const { toolRoot, config } = options;
|
|
11
13
|
const summary = await runSecurityValidation({
|
|
12
14
|
cwd: toolRoot,
|
|
@@ -60,7 +62,23 @@ export async function runSecurityAuditAdapter(options) {
|
|
|
60
62
|
verdictReasonSummary: summary.verdictReasonSummary ?? null,
|
|
61
63
|
reportPaths: { text: textPath, json: jsonPath },
|
|
62
64
|
};
|
|
63
|
-
|
|
65
|
+
// v0.4.2 Batch 2 — Android integration is strictly additive: existing
|
|
66
|
+
// non-Android issues/summary above are computed identically whether or not
|
|
67
|
+
// Android is requested, and Android issues are always appended after them,
|
|
68
|
+
// never interleaved or reordered.
|
|
69
|
+
let androidIssues = [];
|
|
70
|
+
let androidSummary = ANDROID_AUDIT_NOT_REQUESTED_SUMMARY;
|
|
71
|
+
if (options.android?.enabled) {
|
|
72
|
+
const androidResult = await runAndroidAuditIntegration({
|
|
73
|
+
toolRoot,
|
|
74
|
+
targetPathArg: config.targetPathArg,
|
|
75
|
+
request: options.android,
|
|
76
|
+
runAndroidValidation: dependencies.runAndroidValidation,
|
|
77
|
+
});
|
|
78
|
+
androidIssues = androidResult.issues;
|
|
79
|
+
androidSummary = androidResult.summary;
|
|
80
|
+
}
|
|
81
|
+
return { issues: [...issues, ...androidIssues], summary: securitySummary, android: androidSummary };
|
|
64
82
|
}
|
|
65
83
|
function buildRunScopedReportSuffix(startedAt, finishedAt) {
|
|
66
84
|
return `${sanitizeTimestamp(startedAt)}-${sanitizeTimestamp(finishedAt)}-${process.pid}`;
|
|
@@ -15,3 +15,40 @@ export const SECURITY_AUDIT_NOT_RUN_SUMMARY = {
|
|
|
15
15
|
verdictReasonSummary: null,
|
|
16
16
|
reportPaths: { text: null, json: null },
|
|
17
17
|
};
|
|
18
|
+
// Mirrors SecurityAuditReportSummary's own `ran`-first, always-present
|
|
19
|
+
// convention: a status literal rather than inferring "did it run" from
|
|
20
|
+
// nullable fields, so callers/tests can switch on one value instead of
|
|
21
|
+
// reconstructing a state machine from several optional fields.
|
|
22
|
+
export const ANDROID_AUDIT_STATUSES = ["not-requested", "not-applicable", "completed", "failed"];
|
|
23
|
+
export const EMPTY_ANDROID_AUDIT_CANDIDATE_SUMMARY = {
|
|
24
|
+
totalCount: 0,
|
|
25
|
+
checksWithCandidates: 0,
|
|
26
|
+
byConfidence: { unknown: 0, low: 0, medium: 0, high: 0 },
|
|
27
|
+
byResolutionState: { resolved: 0, unresolved: 0, missing: 0, malformed: 0, unsupported: 0, "not-applicable": 0 },
|
|
28
|
+
byCategory: {},
|
|
29
|
+
};
|
|
30
|
+
export const ANDROID_AUDIT_NOT_REQUESTED_SUMMARY = {
|
|
31
|
+
requested: false,
|
|
32
|
+
applicable: null,
|
|
33
|
+
status: "not-requested",
|
|
34
|
+
verdict: null,
|
|
35
|
+
totalChecks: 0,
|
|
36
|
+
passedChecks: 0,
|
|
37
|
+
checksWithFindings: 0,
|
|
38
|
+
candidateOnlyChecks: 0,
|
|
39
|
+
skippedChecks: 0,
|
|
40
|
+
inconclusiveChecks: 0,
|
|
41
|
+
failedChecks: 0,
|
|
42
|
+
confirmedFindingCount: 0,
|
|
43
|
+
mappedIssueCount: 0,
|
|
44
|
+
candidateEvidenceCount: 0,
|
|
45
|
+
requestedGradleOperationCount: 0,
|
|
46
|
+
executedGradleOperationCount: 0,
|
|
47
|
+
requestedExternalToolCount: 0,
|
|
48
|
+
executedExternalToolCount: 0,
|
|
49
|
+
skippedExternalToolCount: 0,
|
|
50
|
+
reportPaths: { text: null, json: null },
|
|
51
|
+
warnings: [],
|
|
52
|
+
errors: [],
|
|
53
|
+
candidateSummary: EMPTY_ANDROID_AUDIT_CANDIDATE_SUMMARY,
|
|
54
|
+
};
|