icsf-cli 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- icsf_cli-0.1.0/PKG-INFO +1095 -0
- icsf_cli-0.1.0/README.md +1059 -0
- icsf_cli-0.1.0/backend/__init__.py +7 -0
- icsf_cli-0.1.0/backend/cli.py +202 -0
- icsf_cli-0.1.0/backend/cli_api.py +409 -0
- icsf_cli-0.1.0/backend/config.py +76 -0
- icsf_cli-0.1.0/backend/diag_auth.py +16 -0
- icsf_cli-0.1.0/backend/logging_config.py +18 -0
- icsf_cli-0.1.0/backend/main.py +1644 -0
- icsf_cli-0.1.0/icsf_cli.egg-info/PKG-INFO +1095 -0
- icsf_cli-0.1.0/icsf_cli.egg-info/SOURCES.txt +15 -0
- icsf_cli-0.1.0/icsf_cli.egg-info/dependency_links.txt +1 -0
- icsf_cli-0.1.0/icsf_cli.egg-info/entry_points.txt +2 -0
- icsf_cli-0.1.0/icsf_cli.egg-info/requires.txt +16 -0
- icsf_cli-0.1.0/icsf_cli.egg-info/top_level.txt +1 -0
- icsf_cli-0.1.0/pyproject.toml +55 -0
- icsf_cli-0.1.0/setup.cfg +4 -0
icsf_cli-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,1095 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: icsf-cli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: ICSF β Intelligent Code Security & Fixing Platform (CLI)
|
|
5
|
+
Author-email: Ramu Venkatesan <Ramu.Venkatesan@infoservices.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/icsf-testing/icsf-poc
|
|
8
|
+
Project-URL: Source, https://github.com/icsf-testing/icsf-poc
|
|
9
|
+
Project-URL: Issues, https://github.com/icsf-testing/icsf-poc/issues
|
|
10
|
+
Keywords: security,cli,github,java,maven,icsf
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Topic :: Security
|
|
17
|
+
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
Requires-Dist: fastapi<1.0.0,>=0.104.1
|
|
21
|
+
Requires-Dist: uvicorn[standard]<0.32.0,>=0.24.0
|
|
22
|
+
Requires-Dist: httpx<1,>=0.28.1
|
|
23
|
+
Requires-Dist: pydantic<3.0.0,>=2.10.0
|
|
24
|
+
Requires-Dist: pydantic[email]<3.0.0,>=2.10.0
|
|
25
|
+
Requires-Dist: python-dotenv==1.0.0
|
|
26
|
+
Requires-Dist: pandas<3.0.0,>=2.0.0
|
|
27
|
+
Requires-Dist: python-multipart<1.0.0,>=0.0.6
|
|
28
|
+
Requires-Dist: boto3<2.0.0,>=1.34.0
|
|
29
|
+
Requires-Dist: botocore<2.0.0,>=1.34.0
|
|
30
|
+
Requires-Dist: pyyaml<7.0.0,>=6.0.1
|
|
31
|
+
Requires-Dist: PyGithub<3.0.0,>=2.1.1
|
|
32
|
+
Requires-Dist: GitPython<4.0.0,>=3.1.40
|
|
33
|
+
Requires-Dist: numpy<3.0.0,>=1.26.0
|
|
34
|
+
Requires-Dist: streamlit<2.0.0,>=1.28.0
|
|
35
|
+
Requires-Dist: starlette<1.0.0,>=0.27.0
|
|
36
|
+
|
|
37
|
+
# ICSF β Intelligent Code Security & Fixing Platform
|
|
38
|
+
|
|
39
|
+
ICSF is a full-stack, AI-powered platform that automates the discovery, analysis, and remediation of security vulnerabilities in Java/Maven codebases. It combines a multi-agent cognitive fixing pipeline with an autonomous self-healing testing framework (**Atlas**) to deliver a closed-loop system: from vulnerability report β verified, PR-ready fix.
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## Table of Contents
|
|
44
|
+
|
|
45
|
+
- [Architecture Overview](#-architecture-overview)
|
|
46
|
+
- [End-to-End Application Flow](#-end-to-end-application-flow)
|
|
47
|
+
- [Backend Deep Dive](#-backend-deep-dive)
|
|
48
|
+
- [FastAPI Main (`main.py`)](#1-fastapi-main-mainpy--1636-lines)
|
|
49
|
+
- [Configuration & Credentials](#2-configuration--credentials)
|
|
50
|
+
- [Pydantic Data Models](#3-pydantic-data-models-modelsagent_modelspy)
|
|
51
|
+
- [Services Layer](#4-services-layer-services)
|
|
52
|
+
- [Agents Layer (Cognitive Fixing Loop)](#5-agents-layer-agents--cognitive-fixing-loop)
|
|
53
|
+
- [Atlas Subsystem (Self-Healing Testing)](#6-atlas-subsystem-atlas--self-healing-testing-framework)
|
|
54
|
+
- [Frontend Deep Dive](#-frontend-deep-dive)
|
|
55
|
+
- [RAG (Retrieval-Augmented Generation)](#-rag-retrieval-augmented-generation)
|
|
56
|
+
- [AI / LLM Integration](#-ai--llm-integration)
|
|
57
|
+
- [Input Requirements](#-input-requirements)
|
|
58
|
+
- [Technical Stack](#-technical-stack)
|
|
59
|
+
- [Getting Started](#-getting-started)
|
|
60
|
+
- [Project Structure](#-project-structure)
|
|
61
|
+
- [API Reference](#-api-reference)
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## ποΈ Architecture Overview
|
|
66
|
+
|
|
67
|
+
ICSF follows a layered, modular architecture:
|
|
68
|
+
|
|
69
|
+
```
|
|
70
|
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
71
|
+
β Frontend (Streamlit β 5676 lines) β
|
|
72
|
+
β Premium dark-mode dashboard Β· Real-time progress Β· Lineage graph β
|
|
73
|
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
|
|
74
|
+
β Backend (FastAPI β 1636 lines) β
|
|
75
|
+
β REST API Β· WebSocket/SSE Β· Request ID middleware β
|
|
76
|
+
ββββββββββββββ¬βββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ€
|
|
77
|
+
β Services β Agents β Atlas Subsystem β
|
|
78
|
+
β (14 files) β (Cognitive Loop) β (Self-Healing Testing) β
|
|
79
|
+
β β 5 agents + helpers β 14 sub-packages β
|
|
80
|
+
ββββββββββββββΌβββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββ€
|
|
81
|
+
β β AWS Bedrock (LLM) β SQLite RAG Store β
|
|
82
|
+
β β Claude 3.5 Sonnet β Titan Embeddings β
|
|
83
|
+
β β Titan Embeddings β Cosine Similarity Search β
|
|
84
|
+
ββββββββββββββ΄βββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
### Key Design Principles
|
|
88
|
+
|
|
89
|
+
| Principle | Implementation |
|
|
90
|
+
|---|---|
|
|
91
|
+
| **Single AI Provider** | All LLM/embedding calls route through AWS Bedrock only |
|
|
92
|
+
| **Multi-Agent Pipeline** | 5 specialized agents, each with a single responsibility |
|
|
93
|
+
| **Self-Healing** | Atlas auto-repairs build failures and test regressions |
|
|
94
|
+
| **Cross-Repo Awareness** | Dependency analysis spans across multiple repositories |
|
|
95
|
+
| **Cost Control** | `CostGuardService` enforces per-run budget limits |
|
|
96
|
+
| **Resilience** | Retry with exponential backoff + circuit breakers |
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## π End-to-End Application Flow
|
|
101
|
+
|
|
102
|
+
```mermaid
|
|
103
|
+
flowchart TD
|
|
104
|
+
%% Entry & Configuration
|
|
105
|
+
U((User)) -->|1. Setup| UI[Streamlit Dashboard]
|
|
106
|
+
UI -->|2. Upload CSV| BE[FastAPI Backend]
|
|
107
|
+
|
|
108
|
+
subgraph "Phase I: Discovery & Mapping"
|
|
109
|
+
direction TB
|
|
110
|
+
BE -->|3. Fetch Projects| GH[GitHub API]
|
|
111
|
+
GH -->|4. List Repos| REPO[(Repository Store)]
|
|
112
|
+
VS[Vulnerability Mapper] -->|5. Map Assets| FT[(Local Workspace)]
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
BE --> VS
|
|
116
|
+
|
|
117
|
+
subgraph "Phase II: Quality Baseline β Atlas"
|
|
118
|
+
direction LR
|
|
119
|
+
BASE[Lightweight Baseline] -->|6. Verify Build| COV[Capture Coverage]
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
VS --> BASE
|
|
123
|
+
|
|
124
|
+
subgraph "Phase III: Impact Analysis"
|
|
125
|
+
direction TB
|
|
126
|
+
DS[Dependency Service] -->|7. Map Blast Radius| MAP[Call Tree & Usage]
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
COV --> DS
|
|
130
|
+
|
|
131
|
+
subgraph "Phase IV: Cognitive Fixing Loop"
|
|
132
|
+
direction TB
|
|
133
|
+
CC[Code Context Agent] --> FS[Fix Strategy Agent]
|
|
134
|
+
FS --> CF[Code Fixer Agent]
|
|
135
|
+
CF --> SV[Safety Validator Agent]
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
MAP -->|8. Start Fix| CC
|
|
139
|
+
CC <-->|Rich Context| DS
|
|
140
|
+
CF -->|9. Apply Fix| FT
|
|
141
|
+
|
|
142
|
+
subgraph "Phase V: Self-Healing Pipeline β Atlas"
|
|
143
|
+
direction TB
|
|
144
|
+
BM[BuildMechanic] --> TH[TestHealer]
|
|
145
|
+
TH --> TG[AI Test Generator]
|
|
146
|
+
TG --> VR[Validation Report]
|
|
147
|
+
end
|
|
148
|
+
|
|
149
|
+
SV -->|10. Final Verify| BM
|
|
150
|
+
BM -->|Self-Heal Build| FT
|
|
151
|
+
FT --> TH
|
|
152
|
+
|
|
153
|
+
subgraph "Phase VI: Delivery & Sync"
|
|
154
|
+
direction TB
|
|
155
|
+
VR --> PR[Batch PR Manager]
|
|
156
|
+
PR -->|11. Create Sync PR| GH
|
|
157
|
+
PR -->|12. Update UI| UI
|
|
158
|
+
end
|
|
159
|
+
|
|
160
|
+
%% RAG Knowledge Loop
|
|
161
|
+
TG -.->|Save Success Patterns| RAG[(SQLite RAG Store)]
|
|
162
|
+
RAG -.->|Context Enrichment| CC
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
### Phase-by-Phase Walkthrough
|
|
166
|
+
|
|
167
|
+
| Phase | What Happens | Key Service |
|
|
168
|
+
|---|---|---|
|
|
169
|
+
| **I. Discovery** | Upload CSV β fetch repos from GitHub API β match vulnerability file paths to repositories using intelligent path normalization | `VulnerabilityService`, `GitHubService` |
|
|
170
|
+
| **II. Baseline** | Run `mvn compile test` on the unmodified code to establish Ground Truth coverage & build health | `AtlasService.run_baseline_only()` |
|
|
171
|
+
| **III. Impact Analysis** | Parse Java files, build global dependency graph, find cross-repo callers of the vulnerable method | `DependencyService` |
|
|
172
|
+
| **IV. Cognitive Fixing** | 4-agent pipeline: Analyze context β Plan strategy β Generate fix β Validate safety | `FixOrchestrator` + 4 Agents |
|
|
173
|
+
| **V. Self-Healing** | BuildMechanic auto-repairs compilation; TestHealer fixes broken tests; AI generates new security-targeted tests | Atlas pipeline |
|
|
174
|
+
| **VI. Delivery** | Aggregate all fixes into a single PR with rich markdown body, push to GitHub | `BatchPRService`, `PRManagerService` |
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## π§ Backend Deep Dive
|
|
179
|
+
|
|
180
|
+
### 1. FastAPI Main (`main.py`) β 1636 lines
|
|
181
|
+
|
|
182
|
+
The central orchestration hub. Defines the REST API, middleware, and all endpoint routes.
|
|
183
|
+
|
|
184
|
+
#### Startup & Middleware
|
|
185
|
+
|
|
186
|
+
| Component | Purpose |
|
|
187
|
+
|---|---|
|
|
188
|
+
| `_startup_validation()` | Smoke-checks AWS + GitHub credentials on boot |
|
|
189
|
+
| `RequestIDMiddleware` | Injects a UUID `X-Request-ID` header into every request for log correlation |
|
|
190
|
+
| CORS middleware | Configurable via `ALLOWED_ORIGINS` env var |
|
|
191
|
+
|
|
192
|
+
#### Pydantic Request/Response Models (inline)
|
|
193
|
+
|
|
194
|
+
| Model | Fields | Used By |
|
|
195
|
+
|---|---|---|
|
|
196
|
+
| `GitHubRepoRequest` | `username`, `email`, `token` | `POST /api/github/repos` |
|
|
197
|
+
| `Repository` | `id`, `name`, `full_name`, `clone_url`, `language`, etc. | All repo endpoints |
|
|
198
|
+
| `RepositoriesResponse` | `username`, `total_repos`, `repositories[]` | Repo listing |
|
|
199
|
+
| `TestingRequest` | `repo_url`, `repo_path`, `fixed_files`, `create_pr`, `vulnerability`, etc. | Testing pipeline |
|
|
200
|
+
| `Vulnerability` | `file_name`, `line_no` | Vulnerability mapping |
|
|
201
|
+
| `MappedVulnerability` | `repo: Repository`, `vulnerabilities[]` | Mapping results |
|
|
202
|
+
|
|
203
|
+
#### API Endpoints
|
|
204
|
+
|
|
205
|
+
| Method | Route | Description |
|
|
206
|
+
|---|---|---|
|
|
207
|
+
| `GET` | `/` | Root welcome |
|
|
208
|
+
| `GET` | `/api/health` | Health check for Docker/LB probes |
|
|
209
|
+
| `GET` | `/api/credentials/github` | Retrieve loaded GitHub credentials |
|
|
210
|
+
| `GET` | `/api/credentials/verify` | Debug credential loading |
|
|
211
|
+
| `POST` | `/api/github/repos` | Fetch repos (POST with body) |
|
|
212
|
+
| `GET` | `/api/github/repos` | Fetch repos (GET with query params) |
|
|
213
|
+
| `POST` | `/api/vulnerabilities/map` | Upload CSV + map vulnerabilities to repos |
|
|
214
|
+
| `POST` | `/api/dependencies/analyze` | Analyze dependencies for a single vulnerability |
|
|
215
|
+
| `POST` | `/api/dependencies/batch-analyze` | Batch dependency analysis for multiple vulnerabilities |
|
|
216
|
+
| `POST` | `/api/fix/orchestrate` | Run the full multi-agent fixing pipeline |
|
|
217
|
+
| `POST` | `/api/pr/create` | Create a single PR with fixed code |
|
|
218
|
+
| `POST` | `/api/testing/start` | Start async testing pipeline job |
|
|
219
|
+
| `GET` | `/api/testing/job/{job_id}` | Poll job status |
|
|
220
|
+
| `GET` | `/api/testing/stream/{job_id}` | SSE event stream for real-time progress |
|
|
221
|
+
| `GET` | `/api/testing/runs` | List recent pipeline runs |
|
|
222
|
+
| `POST` | `/api/testing/run` | Legacy sync testing endpoint |
|
|
223
|
+
| `POST` | `/api/fix/batch` | Batch fix multiple vulnerabilities |
|
|
224
|
+
| `POST` | `/api/pr/merge` | Merge PR with conflict resolution |
|
|
225
|
+
| `POST` | `/api/pr/check-mergeability` | Check PR mergeability |
|
|
226
|
+
| `POST` | `/api/pr/create-batch` | Create single aggregated PR for all fixes |
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
### 2. Configuration & Credentials
|
|
231
|
+
|
|
232
|
+
#### `config.py` β The Config Class
|
|
233
|
+
|
|
234
|
+
| Attribute | Source | Default |
|
|
235
|
+
|---|---|---|
|
|
236
|
+
| `AWS_ACCESS_KEY_ID` | `.env` | β |
|
|
237
|
+
| `AWS_SECRET_ACCESS_KEY` | `.env` | β |
|
|
238
|
+
| `AWS_REGION` | `.env` | `us-east-1` |
|
|
239
|
+
| `AWS_SESSION_TOKEN` | `.env` | `None` |
|
|
240
|
+
| `BEDROCK_MODEL_ID` | `.env` | `anthropic.claude-3-5-sonnet-20240620-v1:0` |
|
|
241
|
+
| `BEDROCK_EMBED_MODEL_ID` | `.env` | `amazon.titan-embed-text-v1` |
|
|
242
|
+
|
|
243
|
+
**Key Methods:**
|
|
244
|
+
|
|
245
|
+
- `get_github_credentials(force_reload=False)` β Reads `credentials.yaml` for GitHub PAT, username, email
|
|
246
|
+
- `validate_bedrock_credentials()` β Returns `(is_valid, error_msg)` tuple
|
|
247
|
+
- `get_bedrock_config()` β Returns dict with `access_key`, `secret_key`, `region`
|
|
248
|
+
|
|
249
|
+
#### `credentials.yaml`
|
|
250
|
+
|
|
251
|
+
```yaml
|
|
252
|
+
github:
|
|
253
|
+
token: ghp_xxxxx
|
|
254
|
+
username: your-username
|
|
255
|
+
email: your-email@example.com
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
---
|
|
259
|
+
|
|
260
|
+
### 3. Pydantic Data Models (`models/agent_models.py`)
|
|
261
|
+
|
|
262
|
+
These 10 models define the complete data flow through the multi-agent pipeline:
|
|
263
|
+
|
|
264
|
+
```mermaid
|
|
265
|
+
flowchart LR
|
|
266
|
+
VFR[VulnerabilityFixRequest] --> VA[VulnerabilityAnalysis]
|
|
267
|
+
VA --> CC[CodeContext]
|
|
268
|
+
CC --> FS[FixStrategy]
|
|
269
|
+
FS --> CF[CodeFix]
|
|
270
|
+
CF --> SV[SafetyValidation]
|
|
271
|
+
SV --> FE[FixExplanation]
|
|
272
|
+
VFR --> FOR[FixOrchestrationResult]
|
|
273
|
+
VA --> FOR
|
|
274
|
+
CC --> FOR
|
|
275
|
+
FS --> FOR
|
|
276
|
+
CF --> FOR
|
|
277
|
+
SV --> FOR
|
|
278
|
+
FE --> FOR
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
| Model | Role | Key Fields |
|
|
282
|
+
|---|---|---|
|
|
283
|
+
| `VulnerabilityFixRequest` | Input to pipeline | `vulnerability_type`, `file_path`, `line_number`, `repo_path` |
|
|
284
|
+
| `VulnerabilityAnalysis` | Agent 1 output | `severity`, `security_impact`, `root_causes`, `fix_category` |
|
|
285
|
+
| `CodeContext` | Agent 2 output | `code_snippet`, `class_name`, `dependent_files_intra/inter`, `data_flow` |
|
|
286
|
+
| `FixStrategy` | Agent 3 output | `fix_approach`, `code_changes_plan`, `files_to_modify_primary/secondary` |
|
|
287
|
+
| `CodeFix` | Agent 4 output | `fixed_code` (Dict[pathβcode]), `diff`, `change_summary`, `reasoning` |
|
|
288
|
+
| `SafetyValidation` | Agent 5 output | `validation_status`, `correctness_score`, `breaking_changes`, `issues_found` |
|
|
289
|
+
| `FixExplanation` | Agent 6 output | `vulnerability_summary`, `fix_explanation`, `markdown_report` |
|
|
290
|
+
| `FixOrchestrationResult` | Complete result | Aggregates all agent outputs + `overall_status`, `errors` |
|
|
291
|
+
| `VulnerabilitySeverity` | Enum | `CRITICAL`, `HIGH`, `MEDIUM`, `LOW`, `INFO` |
|
|
292
|
+
| `ValidationResult` | Enum | `APPROVED`, `REJECTED`, `NEEDS_REVIEW` |
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
### 4. Services Layer (`services/`)
|
|
297
|
+
|
|
298
|
+
The services layer contains 14 files providing the core business logic.
|
|
299
|
+
|
|
300
|
+
#### 4.1 `bedrock_service.py` β AWS Bedrock LLM Wrapper (439 lines)
|
|
301
|
+
|
|
302
|
+
The primary AI gateway used by the **Agents** layer.
|
|
303
|
+
|
|
304
|
+
| Method | Description |
|
|
305
|
+
|---|---|
|
|
306
|
+
| `invoke_claude(prompt, model_id, max_tokens, temperature, system_prompt)` | Synchronous Claude invocation via Bedrock `invoke_model` API |
|
|
307
|
+
| `ainvoke_claude(...)` | Async wrapper using `asyncio.to_thread` |
|
|
308
|
+
| `invoke_llama(prompt, ...)` | Llama 3 70B invocation (different payload format) |
|
|
309
|
+
| `ainvoke_llama(...)` | Async Llama wrapper |
|
|
310
|
+
| `embed_text(text, embed_model_id)` | Generate embeddings via Amazon Titan Embed |
|
|
311
|
+
| `invoke_model(model_id, prompt, ...)` | Generic dispatcher β auto-selects Claude/Llama based on model ID |
|
|
312
|
+
| `test_connection()` | Smoke test with simple prompt |
|
|
313
|
+
|
|
314
|
+
**Supported Model Constants:**
|
|
315
|
+
|
|
316
|
+
| Constant | Model ID |
|
|
317
|
+
|---|---|
|
|
318
|
+
| `CLAUDE_3_5_SONNET` | `anthropic.claude-3-5-sonnet-20240620-v1:0` |
|
|
319
|
+
| `CLAUDE_3_SONNET` | `anthropic.claude-3-sonnet-20240229-v1:0` |
|
|
320
|
+
| `LLAMA_3_70B` | `meta.llama3-70b-instruct-v1:0` |
|
|
321
|
+
|
|
322
|
+
#### 4.2 `github_service.py` β GitHub API Client (455 lines)
|
|
323
|
+
|
|
324
|
+
| Method | Description |
|
|
325
|
+
|---|---|
|
|
326
|
+
| `verify_token_and_get_user(username)` | Validate PAT + retrieve user info |
|
|
327
|
+
| `get_user_by_username(username)` | Public API user lookup |
|
|
328
|
+
| `get_username_from_email(email)` | Reverse email β username lookup |
|
|
329
|
+
| `get_user_organizations()` | List authenticated user's orgs |
|
|
330
|
+
| `get_organization_repositories(org_name)` | List all repos in an org (paginated) |
|
|
331
|
+
| `get_all_repositories(username, include_private, include_orgs)` | Aggregated repo fetch (user + org repos) |
|
|
332
|
+
| `get_repository_details(owner, repo_name)` | Single repo metadata |
|
|
333
|
+
| `get_repository_file_tree(owner, repo_name, branch)` | Recursive file tree via Git Tree API |
|
|
334
|
+
|
|
335
|
+
#### 4.3 `vulnerability_service.py` β CSV Parser & Repo Mapper (833 lines)
|
|
336
|
+
|
|
337
|
+
Parses vulnerability reports from Fortify, Checkmarx, SonarQube, Snyk, etc.
|
|
338
|
+
|
|
339
|
+
| Method | Description |
|
|
340
|
+
|---|---|
|
|
341
|
+
| `parse_csv_file(file_content, filename)` | Parse CSV into DataFrame; auto-detects column names |
|
|
342
|
+
| `extract_repo_name_from_url(url)` | Handles HTTPS, SSH, `.git` suffix URLs |
|
|
343
|
+
| `normalize_repo_identifier(repo_name, repo_url)` | Lowercase normalization for matching |
|
|
344
|
+
| `normalize_file_path(file_path)` | Cross-platform path normalization |
|
|
345
|
+
| `get_path_variations(file_path)` | Generates multiple path format variations for fuzzy matching |
|
|
346
|
+
| `match_file_in_repo(file_name, repo_files)` | Intelligent file matching with early-exit optimization |
|
|
347
|
+
| `clone_repository_and_get_files(repo_url, clone_dir)` | Git clone + file tree extraction |
|
|
348
|
+
| `map_vulnerabilities_to_repos(df, repositories, repo_files_map, clone_repos)` | Core mapping: CSV rows β repository + file matches |
|
|
349
|
+
|
|
350
|
+
#### 4.4 `dependency_service.py` β Java Dependency Graph Engine (2037 lines)
|
|
351
|
+
|
|
352
|
+
The largest service file. Performs static analysis of Java source code and Maven POM files.
|
|
353
|
+
|
|
354
|
+
| Method | Description |
|
|
355
|
+
|---|---|
|
|
356
|
+
| `parse_java_file(file_path)` | Extracts package, imports, classes, methods, interfaces, method calls via regex/AST parsing |
|
|
357
|
+
| `parse_pom_xml(pom_path)` | Extracts `groupId`, `artifactId`, `version`, dependencies, parent POM |
|
|
358
|
+
| `find_java_files(repo_path)` | Recursive `.java` file discovery |
|
|
359
|
+
| `find_pom_files(repo_path)` | Recursive `pom.xml` file discovery |
|
|
360
|
+
| `build_global_dependency_graph(all_repos, artifact_index)` | Builds both intra-repo and inter-repo dependency edges. Node identity = `(repo_name, file_path)` |
|
|
361
|
+
| `build_intra_repo_dependencies(repo_path)` | File-to-file dependencies within a single repo (import-graph) |
|
|
362
|
+
| `find_maven_artifact_for_file(file_path, repo_path)` | Map a `.java` file to its Maven artifact coordinates |
|
|
363
|
+
| `find_cross_repo_dependent_files(...)` | **Inter-repo blast radius**: finds files in other repos that depend on the vulnerable file |
|
|
364
|
+
| `_build_cross_repo_dependency_chains(...)` | Transitive dependency chain traversal across repos (up to `max_depth=5`) |
|
|
365
|
+
| `build_maven_artifact_index(all_repos)` | Maps `(groupId, artifactId)` β repository metadata |
|
|
366
|
+
|
|
367
|
+
#### 4.5 `fix_orchestrator.py` β Multi-Agent Pipeline Controller (564 lines)
|
|
368
|
+
|
|
369
|
+
Coordinates the sequential agent execution:
|
|
370
|
+
|
|
371
|
+
```
|
|
372
|
+
Agent 2 (Code Context) β Agent 3 (Fix Strategy) β Agent 4 (Code Fix) β Agent 5 (Safety Validator)
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
| Method | Description |
|
|
376
|
+
|---|---|
|
|
377
|
+
| `orchestrate_fix(request, stop_at_agent, validate_fix, max_validation_retries, all_repositories_info)` | Main entry point. Runs agents 2β5 sequentially, with optional validation loop |
|
|
378
|
+
| `get_orchestration_status(result)` | Human-readable status summary |
|
|
379
|
+
| `_create_skeleton_analysis(request)` | Generates a default `VulnerabilityAnalysis` from request data |
|
|
380
|
+
|
|
381
|
+
Supports `stop_at_agent` for incremental testing (e.g., run only agents 2-3).
|
|
382
|
+
|
|
383
|
+
#### 4.6 `batch_fix_service.py` β Batch Vulnerability Processing (753 lines)
|
|
384
|
+
|
|
385
|
+
Processes multiple vulnerabilities in sequence or with controlled concurrency.
|
|
386
|
+
|
|
387
|
+
| Method | Description |
|
|
388
|
+
|---|---|
|
|
389
|
+
| `_process_vulnerability_fix(current_idx, vuln_idx, vuln, ...)` | Process a single vulnerability with logging |
|
|
390
|
+
| `_run_testing_agent(repo_path, repo_name, phase, fixed_files)` | Run Atlas testing (baseline or validation phase) |
|
|
391
|
+
| `fix_single_vulnerability(vulnerability, repo_path, ...)` | Single fix with full orchestration |
|
|
392
|
+
| `fix_batch_vulnerabilities(vulnerabilities, repo_path, ..., max_concurrent, auto_create_pr, run_tests_after_fix)` | Main batch entry point. Runs baseline β sequential fixes β validation β optional PR |
|
|
393
|
+
|
|
394
|
+
Workflow: **Baseline β Fix each vulnerability β Run Atlas validation β Create aggregated PR**
|
|
395
|
+
|
|
396
|
+
#### 4.7 `pr_manager_service.py` β Git & PR Operations (1359 lines)
|
|
397
|
+
|
|
398
|
+
Complete Git workflow management.
|
|
399
|
+
|
|
400
|
+
| Method | Description |
|
|
401
|
+
|---|---|
|
|
402
|
+
| `_run_git_command(repo_path, command, timeout)` | Safe subprocess wrapper for git commands |
|
|
403
|
+
| `create_branch(repo_path, branch_name, base_branch)` | Create and checkout new branch |
|
|
404
|
+
| `commit_changes(repo_path, files_to_commit, commit_message, author_name, author_email)` | Stage + commit with configurable author |
|
|
405
|
+
| `push_branch(repo_path, branch_name, remote)` | Git push to remote |
|
|
406
|
+
| `create_pull_request(owner, repo, title, body, head_branch, base_branch)` | GitHub API PR creation |
|
|
407
|
+
| `_validate_compilation(repo_path, files_modified)` | Best-effort Maven/Gradle compilation check |
|
|
408
|
+
| `_clean_code_before_validation(code)` | Removes markdown artifacts, separator lines from LLM output |
|
|
409
|
+
| `_validate_java_code(code, file_path)` | Basic Java structure validation (package, class, brace matching) |
|
|
410
|
+
| `apply_fixed_code(repo_path, files_modified, fixed_code_map)` | Write fixed code to files with validation |
|
|
411
|
+
| `create_pr_for_fix(repo_path, repo_owner, ..., include_all_repo_changes)` | Complete workflow: apply β branch β commit β push β PR |
|
|
412
|
+
|
|
413
|
+
#### 4.8 `batch_pr_service.py` β Aggregated PR Creation (429 lines)
|
|
414
|
+
|
|
415
|
+
| Method | Description |
|
|
416
|
+
|---|---|
|
|
417
|
+
| `_extract_files_and_code(fix_result)` | Parse fix result into `files_modified` + `fixed_code_map` |
|
|
418
|
+
| `create_single_pr(fix_result, ...)` | Create PR for one vulnerability |
|
|
419
|
+
| `create_batch_prs(successful_fixes, ...)` | One PR per vulnerability |
|
|
420
|
+
| `create_single_batch_pr(successful_fixes, ..., test_results)` | **Single aggregated PR** combining all fixes + test results |
|
|
421
|
+
|
|
422
|
+
#### 4.9 `atlas_service.py` β Testing Pipeline FaΓ§ade (430 lines)
|
|
423
|
+
|
|
424
|
+
Bridges the backend API to the Atlas subsystem.
|
|
425
|
+
|
|
426
|
+
| Method | Description |
|
|
427
|
+
|---|---|
|
|
428
|
+
| `_check_required_tools()` | Validates `git`, `mvn`, `java` are on PATH |
|
|
429
|
+
| `run_testing_pipeline(repo_url, create_pr, job_id)` | Full pipeline on remote repo (clone β test β coverage β PR) |
|
|
430
|
+
| `run_testing_pipeline_local(repo_path, repo_url, fixed_files, ...)` | Full pipeline on already-cloned local repo |
|
|
431
|
+
| `run_baseline_only(repo_path, repo_url)` | Lightweight: build + existing tests + coverage β NO AI |
|
|
432
|
+
|
|
433
|
+
#### 4.10 `fix_validator_service.py` β Post-Fix Validation (277 lines)
|
|
434
|
+
|
|
435
|
+
| Method | Description |
|
|
436
|
+
|---|---|
|
|
437
|
+
| `validate_fix(repo_path, files_modified)` | Run Maven build + tests on the fixed repo. Uses `BuildMechanic` for auto-repair |
|
|
438
|
+
| `get_validation_feedback(validation_result)` | Generate feedback string for retry loop |
|
|
439
|
+
|
|
440
|
+
#### 4.11 `job_manager.py` β Async Job & SSE Streaming (86 lines)
|
|
441
|
+
|
|
442
|
+
| Method | Description |
|
|
443
|
+
|---|---|
|
|
444
|
+
| `create_job()` | Create UUID-identified job with `asyncio.Queue` |
|
|
445
|
+
| `update_job(job_id, status, message, progress)` | Update status + push to SSE queue |
|
|
446
|
+
| `end_job(job_id)` | Signal `[DONE]` to SSE stream |
|
|
447
|
+
| `stream_job_events(job_id)` | Async generator for `StreamingResponse` |
|
|
448
|
+
|
|
449
|
+
#### 4.12 `run_history.py` β SQLite Run Persistence (115 lines)
|
|
450
|
+
|
|
451
|
+
| Method | Description |
|
|
452
|
+
|---|---|
|
|
453
|
+
| `create_run(repo_url, repo_path)` | Insert new run record |
|
|
454
|
+
| `update_run(run_id, status, result_data, error_msg, cost)` | Update with test/coverage/regression/quality gate reports |
|
|
455
|
+
| `get_recent_runs(limit)` | Fetch recent runs with JSON report parsing |
|
|
456
|
+
|
|
457
|
+
Schema: `pipeline_runs(run_id, repo_url, repo_path, status, start_time, end_time, total_cost, test_report, coverage_report, regression_report, quality_gate_report, error_message)`
|
|
458
|
+
|
|
459
|
+
#### 4.13 `cost_guard.py` β LLM Cost Limiter (50 lines)
|
|
460
|
+
|
|
461
|
+
| Method | Description |
|
|
462
|
+
|---|---|
|
|
463
|
+
| `start_run(run_id)` | Initialize per-run cost tracking |
|
|
464
|
+
| `add_cost(run_id, prompt_tokens, completion_tokens, model_id)` | Accumulate cost; returns `False` if budget exceeded |
|
|
465
|
+
| `get_run_cost(run_id)` | Query accumulated cost |
|
|
466
|
+
|
|
467
|
+
Pricing: Claude 3.5 Sonnet β $0.003/1K prompt tokens, $0.015/1K completion tokens. Default budget: **$5.00/run**.
|
|
468
|
+
|
|
469
|
+
---
|
|
470
|
+
|
|
471
|
+
### 5. Agents Layer (`agents/`) β Cognitive Fixing Loop
|
|
472
|
+
|
|
473
|
+
#### 5.1 `code_context_agent.py` β Blast Radius Mapper (642 lines)
|
|
474
|
+
|
|
475
|
+
**Purpose:** Understand the full context around a vulnerability β local code, dependent files, data flow.
|
|
476
|
+
|
|
477
|
+
| Method | Logic |
|
|
478
|
+
|---|---|
|
|
479
|
+
| `_read_file_with_context(file_path, line_number, context_lines)` | Extract code snippet + surrounding context. Includes class/method even if vulnerability is in imports |
|
|
480
|
+
| `_extract_class_and_method(file_path, line_number, code_content)` | Regex-based Java class/method extraction |
|
|
481
|
+
| `_analyze_data_flow_and_usage(code_snippet, vulnerability_type, ...)` | **LLM call**: Analyze how user input flows from source β vulnerable sink |
|
|
482
|
+
| `_analyze_method_usage_in_dependents(vulnerable_class, ..., dependent_files)` | Check if the fix will break dependent files by analyzing their imports and usage |
|
|
483
|
+
| `_discover_other_repositories(current_repo_path)` | Scan `temp_cloned_repos/` directory for cross-repo analysis |
|
|
484
|
+
| `analyze(request, vulnerability_analysis, all_repositories_info)` | Main entry: reads file, finds dependents (intra + inter-repo), runs LLM data flow analysis |
|
|
485
|
+
|
|
486
|
+
#### 5.2 `fix_strategy_agent.py` β Surgical Planner (633 lines)
|
|
487
|
+
|
|
488
|
+
**Purpose:** Design a backward-compatible fix plan.
|
|
489
|
+
|
|
490
|
+
| Method | Logic |
|
|
491
|
+
|---|---|
|
|
492
|
+
| `_get_available_java_files(repo_path, max_files)` | Inventory of Java files for validation |
|
|
493
|
+
| `_analyze_file_imports_and_usage(repo_path, file_path)` | Static import analysis to find related files |
|
|
494
|
+
| `_build_strategy_prompt(request, analysis, context)` | Constructs a detailed LLM prompt with vulnerability info, dependents, constraint rules |
|
|
495
|
+
| `_parse_strategy_response(response_content)` | JSON extraction from LLM response |
|
|
496
|
+
| `analyze(request, vulnerability_analysis, code_context)` | **LLM call**: Generate fix strategy; categorizes files as Primary (logic change) or Secondary (impacted usage) |
|
|
497
|
+
|
|
498
|
+
**Key Decision Logic:**
|
|
499
|
+
- If a method is called by 50+ files β force **backward-compatible fix** (overloaded method, not breaking change)
|
|
500
|
+
- Uses `FrameworkDetector` for framework-specific recommendations (Spring Security, Jakarta, etc.)
|
|
501
|
+
|
|
502
|
+
#### 5.3 `code_fix_agent.py` β Multi-File Code Generator (1071 lines)
|
|
503
|
+
|
|
504
|
+
**Purpose:** Generate actual fixed Java code across multiple files.
|
|
505
|
+
|
|
506
|
+
| Method | Logic |
|
|
507
|
+
|---|---|
|
|
508
|
+
| `_read_file(file_path)` | Read source file |
|
|
509
|
+
| `_find_nearest_pom_xml(repo_path, file_rel_path)` | Walk up directories to find `pom.xml` |
|
|
510
|
+
| `_project_allows_spring_security(repo_path, file_rel_path, original_code)` | Check if Spring Security dependencies exist before generating SS code |
|
|
511
|
+
| `_dependency_constraints_text(repo_path, ...)` | Generate constraint text for LLM prompt |
|
|
512
|
+
| `_postprocess_for_project_dependencies(code, ...)` | **Deterministic safety net**: strip Spring Security constructs if project doesn't include it |
|
|
513
|
+
| `_generate_diff(original, fixed, file_path)` | Unified diff generation |
|
|
514
|
+
| `_clean_generated_code(code, file_path)` | Aggressive cleanup: removes markdown, `<thinking>` blocks, ensures valid Java |
|
|
515
|
+
| `_generate_fixed_code(original_code, request, ...)` | **LLM call**: Generate complete fixed file with prompt-chain reasoning |
|
|
516
|
+
| `fix_code(request, ..., fix_strategy)` | Main entry: fixes ALL files in `files_to_modify_primary`, runs post-processing |
|
|
517
|
+
|
|
518
|
+
Uses `ImportManager.add_missing_imports()` and `SyntaxValidator.validate()` for post-processing.
|
|
519
|
+
|
|
520
|
+
#### 5.4 `safety_validator_agent.py` β Logic Gate (371 lines)
|
|
521
|
+
|
|
522
|
+
**Purpose:** Verify the fix is correct, introduces no regressions.
|
|
523
|
+
|
|
524
|
+
| Method | Logic |
|
|
525
|
+
|---|---|
|
|
526
|
+
| `_format_fixed_code(fixed_code)` | Format code dict for display |
|
|
527
|
+
| `_format_dependent_files_for_validation(code_context)` | Format dependent files context |
|
|
528
|
+
| `_build_validation_prompt(request, ..., code_fix)` | Comprehensive validation prompt |
|
|
529
|
+
| `_parse_validation_response(response_content)` | Extract structured validation data |
|
|
530
|
+
| `_normalize_validation_data(parsed)` | Ensure correct types for downstream consumption |
|
|
531
|
+
| `validate(request, ..., code_fix)` | **LLM call**: Returns `APPROVED`/`REJECTED`/`NEEDS_REVIEW` with `correctness_score` (0-1) |
|
|
532
|
+
|
|
533
|
+
#### 5.5 `codebase_analysis_agent.py` β Repository Intelligence (594 lines)
|
|
534
|
+
|
|
535
|
+
**Purpose:** Deep structural analysis of the codebase (similar to AI coding assistants).
|
|
536
|
+
|
|
537
|
+
| Method | Logic |
|
|
538
|
+
|---|---|
|
|
539
|
+
| `analyze_codebase_structure(repo_path, focus_file)` | Full repo analysis with in-memory cache (TTL=300s) |
|
|
540
|
+
| `find_dependent_files(repo_path, target_file, max_depth)` | Find all files depending on target file |
|
|
541
|
+
| `analyze_code_flow(repo_path, file_path, line_number)` | Data flow analysis around a specific line |
|
|
542
|
+
| `_analyze_architecture(repo_path, java_files)` | Detect project layers (Controller, Service, DAO, etc.) |
|
|
543
|
+
| `_build_dependency_graph(repo_path, java_files)` | Build import-based dependency graph |
|
|
544
|
+
| `_detect_patterns(repo_path, java_files)` | Detect design patterns (Singleton, Factory, Builder, Observer) |
|
|
545
|
+
| `_parse_java_file(file_path)` | Extract package, imports, classes (regex-based) |
|
|
546
|
+
|
|
547
|
+
#### 5.6 `agent_improvements.py` β Helper Utilities (368 lines)
|
|
548
|
+
|
|
549
|
+
Four static helper classes:
|
|
550
|
+
|
|
551
|
+
| Class | Purpose |
|
|
552
|
+
|---|---|
|
|
553
|
+
| `ImportManager` | Auto-detect and add missing Java imports (maps common security classes to their import statements) |
|
|
554
|
+
| `SyntaxValidator` | Basic Java syntax validation (brace matching, package declaration, class structure) |
|
|
555
|
+
| `FrameworkDetector` | Detect frameworks in `pom.xml` (Spring Boot, Spring Security, JPA, Jackson, etc.) with framework-specific fix recommendations |
|
|
556
|
+
| `ContextEnhancer` | Extract full method/class definitions from source code for enhanced prompt context |
|
|
557
|
+
|
|
558
|
+
---
|
|
559
|
+
|
|
560
|
+
### 6. Atlas Subsystem (`atlas/`) β Self-Healing Testing Framework
|
|
561
|
+
|
|
562
|
+
Atlas is a comprehensive, autonomous testing and quality assurance pipeline with 14 sub-packages.
|
|
563
|
+
|
|
564
|
+
#### 6.1 `orchestrator/run_pipeline.py` β Pipeline Core (1412 lines)
|
|
565
|
+
|
|
566
|
+
The brain of Atlas. Orchestrates the entire testing lifecycle.
|
|
567
|
+
|
|
568
|
+
| Function | Description |
|
|
569
|
+
|---|---|
|
|
570
|
+
| `run_full_pipeline(repo_url, ...)` | Clone remote repo β full pipeline |
|
|
571
|
+
| `run_full_pipeline_local(repo_path, ...)` | Full pipeline on local repo |
|
|
572
|
+
| `run_baseline_only(repo_path, ...)` | Lightweight: build + test + coverage only |
|
|
573
|
+
| `_run_baseline_phase(repo_path, ...)` | Build (with restricted auto-fix) + existing tests + JaCoCo coverage |
|
|
574
|
+
| `_run_validation_phase(repo_path, ...)` | Diff-aware test generation, healing, regression detection (630+ lines) |
|
|
575
|
+
| `_run_full_pipeline_core(...)` | Core pipeline: baseline β validation β quality gate β PR |
|
|
576
|
+
| `evaluate_quality_gate(coverage, unit, min_coverage_pct, max_failures)` | Pass/fail decision on release readiness |
|
|
577
|
+
| `calculate_regression_report(state_mgr, ...)` | Compare current vs baseline to detect regressions/improvements |
|
|
578
|
+
| `_calculate_usage(llm)` | Compute estimated cost from Bedrock token metrics |
|
|
579
|
+
| `run_organization_pipeline(org_url, ...)` | Scan entire GitHub org: run pipeline on each Java/Maven repo |
|
|
580
|
+
|
|
581
|
+
#### 6.2 `agents/build_mechanic.py` β Build Failure Auto-Repair (1133 lines)
|
|
582
|
+
|
|
583
|
+
The SRE agent. Diagnoses and fixes compilation failures.
|
|
584
|
+
|
|
585
|
+
| Method | Description |
|
|
586
|
+
|---|---|
|
|
587
|
+
| `analyze(stdout, stderr)` | Parse Maven build output β `BuildDiagnosis` (root cause, confidence, hints) |
|
|
588
|
+
| `generate_fix(diagnosis, workspace_path, ...)` | **LLM call**: Generate concrete fix (file patches, POM changes, config files) |
|
|
589
|
+
|
|
590
|
+
**Domain Expertise:**
|
|
591
|
+
- Spring Security 6 migration patterns (`WebSecurityConfigurerAdapter` β lambda DSL)
|
|
592
|
+
- Deprecated API detection and deletion
|
|
593
|
+
- Missing dependency resolution (maps class names β Maven coordinates)
|
|
594
|
+
- `COMMON_TEST_HINTS` dictionary: 30+ patterns mapping class names to imports
|
|
595
|
+
- Test assertion guidelines (status codes, JSON paths, mock strategies)
|
|
596
|
+
|
|
597
|
+
#### 6.3 `agents/test_healer.py` β Test Failure Doctor (151 lines)
|
|
598
|
+
|
|
599
|
+
| Method | Description |
|
|
600
|
+
|---|---|
|
|
601
|
+
| `heal(failed_tests, workspace_path)` | Group failures by class β **LLM call**: generate fixed test file β `AgentFix` |
|
|
602
|
+
| `_find_test_file(root, classname)` | Locate `.java` test file by class name |
|
|
603
|
+
|
|
604
|
+
Processes top 10 failures, max 3 classes, max 5 failures per class.
|
|
605
|
+
|
|
606
|
+
#### 6.4 `rag/store.py` β SQLite Vector RAG Store (210 lines)
|
|
607
|
+
|
|
608
|
+
Lightweight persistent RAG store for test pattern learning.
|
|
609
|
+
|
|
610
|
+
| Method | Description |
|
|
611
|
+
|---|---|
|
|
612
|
+
| `upsert(id, kind, embedding, text, metadata)` | Insert/update with normalized float32 embedding blob |
|
|
613
|
+
| `query(embedding, top_k, kind, kinds, score_threshold, include_expired)` | Cosine similarity search via dot product |
|
|
614
|
+
| `get_by_id(id)` | Direct ID lookup |
|
|
615
|
+
| `count(kind)` | Count entries by kind |
|
|
616
|
+
| `evict_expired()` | TTL-based cleanup (default 30 days) |
|
|
617
|
+
|
|
618
|
+
**Schema:** `rag_items(id TEXT PK, kind TEXT, created_at INT, embedding BLOB, metadata_json TEXT, text TEXT)`
|
|
619
|
+
**Indexes:** `kind`, `created_at`
|
|
620
|
+
|
|
621
|
+
#### 6.5 `llm/bedrock.py` β Atlas Bedrock Client (163 lines)
|
|
622
|
+
|
|
623
|
+
Dedicated Bedrock client for the Atlas subsystem.
|
|
624
|
+
|
|
625
|
+
| Method | Description |
|
|
626
|
+
|---|---|
|
|
627
|
+
| `embed_text(text)` | Titan Embeddings: `inputText` β embedding vector |
|
|
628
|
+
| `generate_text(system, user, max_tokens)` | Claude Messages API via Bedrock `invoke_model` |
|
|
629
|
+
|
|
630
|
+
Tracks `total_input_tokens`, `total_output_tokens`, `total_embedding_tokens` for cost calculation.
|
|
631
|
+
**Security:** Permanent credentials (`AKIA*`) do NOT use session tokens; temporary (`ASIA*`) require them.
|
|
632
|
+
|
|
633
|
+
#### 6.6 `generation/java_unit_test_generator.py` β RAG-Enhanced Test Gen (441 lines)
|
|
634
|
+
|
|
635
|
+
| Method | Description |
|
|
636
|
+
|---|---|
|
|
637
|
+
| `generate_minimal_tests_for_repo(target_count, preferred_classes, ...)` | Discover main classes β prioritize by scoring β generate tests |
|
|
638
|
+
| `_generate_single_test(src, repo_path, ...)` | **LLM + RAG call**: Check fingerprint β query RAG for similar patterns β generate JUnit 5 test |
|
|
639
|
+
| `_set_fingerprint(class_key, sha, test_path)` | Store source hash in RAG for idempotent re-runs |
|
|
640
|
+
|
|
641
|
+
**Scoring heuristic for class prioritization:**
|
|
642
|
+
- +10 if in preferred classes list
|
|
643
|
+
- +5 for service/controller/repository classes
|
|
644
|
+
- +3 for `@RestController`/`@Service`/`@Repository` annotations
|
|
645
|
+
- β2 for test/config/model classes
|
|
646
|
+
|
|
647
|
+
Uses `RepoContractRegistry` for constructor/method signature validation in generated tests.
|
|
648
|
+
|
|
649
|
+
#### 6.7 `build/` β Build Infrastructure (5 files)
|
|
650
|
+
|
|
651
|
+
| File | Purpose |
|
|
652
|
+
|---|---|
|
|
653
|
+
| `maven.py` | Maven command runner (`mvn compile`, `mvn test`, etc.) with subprocess management |
|
|
654
|
+
| `jacoco_injector.py` | Inject JaCoCo Maven plugin into `pom.xml` for code coverage |
|
|
655
|
+
| `spring_test_injector.py` | Inject `spring-boot-starter-test` dependency |
|
|
656
|
+
| `failsafe_injector.py` | Inject Maven Failsafe plugin for integration tests |
|
|
657
|
+
| `dependency_governance.py` | Enforce dependency version governance (BOM alignment, conflict resolution) |
|
|
658
|
+
|
|
659
|
+
#### 6.8 `core/` β Core Infrastructure (5 files)
|
|
660
|
+
|
|
661
|
+
| File | Purpose |
|
|
662
|
+
|---|---|
|
|
663
|
+
| `config.py` | Atlas-specific configuration (data dirs, model IDs, etc.) |
|
|
664
|
+
| `logging.py` | `RunLogger` class for structured pipeline logging |
|
|
665
|
+
| `state.py` | `PipelineStateManager` β manages baseline/validation state persistence |
|
|
666
|
+
| `shell.py` | Safe shell command execution with timeout |
|
|
667
|
+
| `resilience.py` | **Retry with exponential backoff** (configurable attempts, jitter) + **Circuit Breaker** pattern (CLOSED/OPEN/HALF-OPEN states) + **Rate Limiter** |
|
|
668
|
+
|
|
669
|
+
#### 6.9 `analysis/` β Code Analysis (3 files)
|
|
670
|
+
|
|
671
|
+
| File | Purpose |
|
|
672
|
+
|---|---|
|
|
673
|
+
| `java_maven.py` | Java project analysis: `detect_repo_facts()`, `count_existing_tests()`, `find_domain_models()` |
|
|
674
|
+
| `contract_service.py` | `RepoContractRegistry`: extract class constructors, method signatures for test generation validation |
|
|
675
|
+
| `diff_analyzer.py` | Analyze git diffs to identify functional changes for targeted test generation |
|
|
676
|
+
|
|
677
|
+
#### 6.10 `reporting/` β Test Reporting (2 files)
|
|
678
|
+
|
|
679
|
+
| File | Purpose |
|
|
680
|
+
|---|---|
|
|
681
|
+
| `models.py` | Report dataclasses: `TestReport`, `CoverageReport`, `BreakageReport`, `GenerationReport`, `RegressionReport`, `QualityGateReport`, `UsageReport`, `FullRunReport` |
|
|
682
|
+
| `parsers.py` | Parse Surefire XML reports, JaCoCo CSV coverage data, classify test failures |
|
|
683
|
+
|
|
684
|
+
#### 6.11 `gitops/` β GitHub Integration (3 files)
|
|
685
|
+
|
|
686
|
+
| File | Purpose |
|
|
687
|
+
|---|---|
|
|
688
|
+
| `github_pr.py` | Create PRs for Atlas-generated tests |
|
|
689
|
+
| `github_issues.py` | Create GitHub issues for persistent test failures |
|
|
690
|
+
| `github_org.py` | List repos in a GitHub organization for org-wide scanning |
|
|
691
|
+
|
|
692
|
+
#### 6.12 `repo/` β Repository Management (2 files)
|
|
693
|
+
|
|
694
|
+
| File | Purpose |
|
|
695
|
+
|---|---|
|
|
696
|
+
| `cloner.py` | `RepoCloner`: Clone repos with token authentication |
|
|
697
|
+
| `history.py` | Run history tracking for the Atlas pipeline |
|
|
698
|
+
|
|
699
|
+
---
|
|
700
|
+
|
|
701
|
+
## π¨ Frontend Deep Dive
|
|
702
|
+
|
|
703
|
+
**Technology:** Streamlit (5676 lines, single `app.py` + utility modules)
|
|
704
|
+
|
|
705
|
+
### UI Components
|
|
706
|
+
|
|
707
|
+
The frontend is a premium dark-mode dashboard with glassmorphism styling, gradient headers, and micro-animations. Key CSS tokens:
|
|
708
|
+
|
|
709
|
+
- Background: `#0f172a` (dark slate), Secondary: `#1e293b`
|
|
710
|
+
- Accent: `linear-gradient(135deg, #3b82f6, #2dd4bf)` (blue β teal)
|
|
711
|
+
- Font: Inter (body), JetBrains Mono (code)
|
|
712
|
+
|
|
713
|
+
### Core Functions
|
|
714
|
+
|
|
715
|
+
| Function | Lines | Purpose |
|
|
716
|
+
|---|---|---|
|
|
717
|
+
| `main()` | 45 | Entry point: mode selector (Vulnerability Workflow vs Repository Explorer) |
|
|
718
|
+
| `display_vulnerability_workflow(api_url)` | ~600 | Streamlined flow: Upload β Map β Test β Fix β Verify |
|
|
719
|
+
| `display_repositories(data)` | ~2400 | Full repository explorer with vulnerability cards, dep trees, fix controls |
|
|
720
|
+
| `process_active_batch_fix(selected_repo_id, ...)` | ~1040 | Real-time batch fix processing with progress bars |
|
|
721
|
+
| `display_lineage_graph(result, repo_name, vuln_idx)` | ~275 | NetworkX-based dependency graph visualization |
|
|
722
|
+
| `fetch_repositories(api_base_url, ...)` | 30 | Call backend to fetch GitHub repos |
|
|
723
|
+
| `map_vulnerabilities(api_url, repositories_data, csv_file)` | 28 | Upload CSV and map vulnerabilities |
|
|
724
|
+
| `run_testing_agent(api_url, repo_url)` | 70 | SSE streaming of Atlas pipeline progress |
|
|
725
|
+
| `batch_fix_vulnerabilities(api_url, vulnerabilities, ...)` | ~210 | Call batch fix endpoint with progress callbacks |
|
|
726
|
+
| `display_lineage_graph._extract_paths(items)` | 10 | Extract file paths from dependent files list |
|
|
727
|
+
| `display_setup_progress(current_step)` | ~160 | Animated 3-step progress tracker (Upload β Fetch β Map) |
|
|
728
|
+
| `display_run_history(api_url)` | 70 | Fetch and display pipeline run history table |
|
|
729
|
+
|
|
730
|
+
### Frontend Utility Modules
|
|
731
|
+
|
|
732
|
+
| File | Purpose |
|
|
733
|
+
|---|---|
|
|
734
|
+
| `src/vulnerability_ui.py` (52KB) | Advanced vulnerability display: cards, severity badges, fix result rendering |
|
|
735
|
+
| `src/lineage.py` (10KB) | Lineage graph data transformations |
|
|
736
|
+
| `utils/atlas_report_comprehensive.py` (21KB) | Comprehensive Atlas report rendering |
|
|
737
|
+
| `utils/integrate_render.py` (4KB) | Report integration helpers |
|
|
738
|
+
|
|
739
|
+
---
|
|
740
|
+
|
|
741
|
+
## π§ RAG (Retrieval-Augmented Generation)
|
|
742
|
+
|
|
743
|
+
ICSF uses a custom RAG implementation for test pattern learning:
|
|
744
|
+
|
|
745
|
+
### Architecture
|
|
746
|
+
|
|
747
|
+
```
|
|
748
|
+
ββββββββββββββββββββ
|
|
749
|
+
β Titan Embed β
|
|
750
|
+
β (Bedrock) β
|
|
751
|
+
ββββββββββ¬ββββββββββ
|
|
752
|
+
β embedding vector
|
|
753
|
+
ββββββββββΌββββββββββ
|
|
754
|
+
β SqliteVectorRag β
|
|
755
|
+
β Store β
|
|
756
|
+
β (cosine search) β
|
|
757
|
+
ββββββββββ¬ββββββββββ
|
|
758
|
+
β similar patterns
|
|
759
|
+
ββββββββββΌββββββββββ
|
|
760
|
+
β Test Generator β
|
|
761
|
+
β (LLM prompt) β
|
|
762
|
+
ββββββββββββββββββββ
|
|
763
|
+
```
|
|
764
|
+
|
|
765
|
+
### How RAG is Used
|
|
766
|
+
|
|
767
|
+
1. **Fingerprint Check**: Before generating a test, hash the source file β query RAG for existing fingerprint β skip if unchanged
|
|
768
|
+
2. **Pattern Retrieval**: Query RAG store for similar test patterns (`kind=test_pattern`) with cosine similarity β₯ 0.25
|
|
769
|
+
3. **Context Injection**: Retrieved patterns are injected into the LLM prompt as examples
|
|
770
|
+
4. **Pattern Storage**: After successful test generation, store the pattern in RAG for future use
|
|
771
|
+
|
|
772
|
+
### RAG Store Configuration
|
|
773
|
+
|
|
774
|
+
| Setting | Value |
|
|
775
|
+
|---|---|
|
|
776
|
+
| **Database** | SQLite (`data/atlas_rag.db`) |
|
|
777
|
+
| **Embedding Model** | Amazon Titan Embed Text v1 |
|
|
778
|
+
| **Embedding Dimension** | 1536 (float32) |
|
|
779
|
+
| **Similarity Metric** | Cosine (via dot product on normalized vectors) |
|
|
780
|
+
| **TTL** | 30 days (auto-eviction of stale entries) |
|
|
781
|
+
| **Score Threshold** | 0.25 minimum cosine similarity |
|
|
782
|
+
|
|
783
|
+
---
|
|
784
|
+
|
|
785
|
+
## π€ AI / LLM Integration
|
|
786
|
+
|
|
787
|
+
### Models Used
|
|
788
|
+
|
|
789
|
+
| Model | Use Case | Provider |
|
|
790
|
+
|---|---|---|
|
|
791
|
+
| **Claude 3.5 Sonnet** | All reasoning: code analysis, fix generation, strategy planning, safety validation, build repair, test healing | AWS Bedrock |
|
|
792
|
+
| **Amazon Titan Embed Text v1** | Text embeddings for RAG store | AWS Bedrock |
|
|
793
|
+
| **Llama 3 70B** (optional) | Alternative generation model | AWS Bedrock |
|
|
794
|
+
|
|
795
|
+
### LLM Call Sites
|
|
796
|
+
|
|
797
|
+
| Component | # of LLM Calls | Purpose |
|
|
798
|
+
|---|---|---|
|
|
799
|
+
| `CodeContextAgent` | 1 | Data flow analysis |
|
|
800
|
+
| `FixStrategyAgent` | 1 | Fix strategy planning |
|
|
801
|
+
| `CodeFixAgent` | 1 per file | Code generation |
|
|
802
|
+
| `SafetyValidatorAgent` | 1 | Fix validation |
|
|
803
|
+
| `BuildMechanic` | 1β3 per build failure | Build error diagnosis + fix |
|
|
804
|
+
| `TestHealer` | 1 per test class | Test repair |
|
|
805
|
+
| `JavaUnitTestGenerator` | 1 per source class | Test generation |
|
|
806
|
+
| Total per vulnerability | ~6β12 | Depending on file count and failure iterations |
|
|
807
|
+
|
|
808
|
+
### Cost Management
|
|
809
|
+
|
|
810
|
+
- `CostGuardService` tracks cost per run with **$5.00 default budget**
|
|
811
|
+
- Pricing model: Claude 3.5 Sonnet @ $0.003/1K input, $0.015/1K output
|
|
812
|
+
- `_calculate_usage()` in the pipeline reports total tokens + estimated cost
|
|
813
|
+
- `BedrockClient` tracks `total_input_tokens`, `total_output_tokens`, `total_embedding_tokens`
|
|
814
|
+
|
|
815
|
+
---
|
|
816
|
+
|
|
817
|
+
## π₯ Input Requirements
|
|
818
|
+
|
|
819
|
+
### 1. Security Vulnerability Report (CSV)
|
|
820
|
+
|
|
821
|
+
Supported scanners: **Fortify**, **Checkmarx**, **SonarQube**, **Snyk**
|
|
822
|
+
|
|
823
|
+
| Required Column | Example |
|
|
824
|
+
|---|---|
|
|
825
|
+
| `vulnerability_type` or `category` | Cross-Site Scripting |
|
|
826
|
+
| `file_name` or `file_path` | `src/main/java/com/example/Controller.java` |
|
|
827
|
+
| `line_no` or `line_number` | `42` |
|
|
828
|
+
| `severity` | Critical / High / Medium / Low |
|
|
829
|
+
| `description` | User input is rendered without encoding |
|
|
830
|
+
| `recommendation` | Use OWASP encoder for output encoding |
|
|
831
|
+
| `repo_name` or `link` | `my-app` or `https://github.com/org/my-app` |
|
|
832
|
+
|
|
833
|
+
### 2. Version Control Credentials
|
|
834
|
+
|
|
835
|
+
- **GitHub PAT**: Requires `repo` and `read:user` scopes
|
|
836
|
+
- Stored in `backend/credentials.yaml`
|
|
837
|
+
|
|
838
|
+
### 3. AI Model Access (AWS Bedrock)
|
|
839
|
+
|
|
840
|
+
- **AWS credentials**: `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` in `.env`
|
|
841
|
+
- **Region**: `us-east-1` (default) or any Bedrock-enabled region
|
|
842
|
+
- **Model access**: Must have Claude 3.5 Sonnet + Titan Embeddings enabled in your AWS account
|
|
843
|
+
|
|
844
|
+
### 4. Build Environment
|
|
845
|
+
|
|
846
|
+
- **Java JDK 17+** on PATH
|
|
847
|
+
- **Maven** on PATH
|
|
848
|
+
- **Git** on PATH
|
|
849
|
+
|
|
850
|
+
---
|
|
851
|
+
|
|
852
|
+
## π οΈ Technical Stack
|
|
853
|
+
|
|
854
|
+
| Layer | Technology | Version |
|
|
855
|
+
|---|---|---|
|
|
856
|
+
| **Language** | Python | 3.10+ |
|
|
857
|
+
| **Backend Framework** | FastAPI | β₯0.104 |
|
|
858
|
+
| **Frontend Framework** | Streamlit | β₯1.28 |
|
|
859
|
+
| **LLM Provider** | AWS Bedrock (Boto3) | β₯1.34 |
|
|
860
|
+
| **Embedding Model** | Amazon Titan Embed Text v1 | β |
|
|
861
|
+
| **Reasoning Model** | Claude 3.5 Sonnet | β |
|
|
862
|
+
| **Database** | SQLite | (stdlib) |
|
|
863
|
+
| **HTTP Client** | httpx | β₯0.28 |
|
|
864
|
+
| **Data Processing** | pandas | β₯2.0 |
|
|
865
|
+
| **Version Control** | GitPython + GitHub API | β₯3.1 |
|
|
866
|
+
| **Graph Analysis** | NetworkX | β₯3.0 |
|
|
867
|
+
| **Validation** | Pydantic | β₯2.10 |
|
|
868
|
+
| **Containerization** | Docker Compose | 3.8 |
|
|
869
|
+
| **Build Tools** | Maven, JDK 17+ | β |
|
|
870
|
+
|
|
871
|
+
---
|
|
872
|
+
|
|
873
|
+
## π Getting Started
|
|
874
|
+
|
|
875
|
+
### Prerequisites
|
|
876
|
+
|
|
877
|
+
- **Git**, **Java JDK 17+**, and **Maven** installed and on PATH
|
|
878
|
+
- **Python 3.10+**
|
|
879
|
+
- **AWS credentials** with Bedrock access (Claude 3.5 Sonnet + Titan Embeddings enabled)
|
|
880
|
+
- **GitHub PAT** with `repo` and `read:user` scopes
|
|
881
|
+
|
|
882
|
+
### Environment Setup
|
|
883
|
+
|
|
884
|
+
1. **Create `.env`** in `backend/`:
|
|
885
|
+
|
|
886
|
+
```env
|
|
887
|
+
AWS_ACCESS_KEY_ID=your_key
|
|
888
|
+
AWS_SECRET_ACCESS_KEY=your_secret
|
|
889
|
+
AWS_REGION=us-east-1
|
|
890
|
+
BEDROCK_MODEL_ID=anthropic.claude-3-5-sonnet-20240620-v1:0
|
|
891
|
+
BEDROCK_EMBED_MODEL_ID=amazon.titan-embed-text-v1
|
|
892
|
+
```
|
|
893
|
+
|
|
894
|
+
2. **Create `credentials.yaml`** in `backend/`:
|
|
895
|
+
|
|
896
|
+
```yaml
|
|
897
|
+
github:
|
|
898
|
+
token: ghp_your_personal_access_token
|
|
899
|
+
username: your-github-username
|
|
900
|
+
email: your-email@example.com
|
|
901
|
+
```
|
|
902
|
+
|
|
903
|
+
### Run with Docker (Recommended)
|
|
904
|
+
|
|
905
|
+
```bash
|
|
906
|
+
docker-compose up --build
|
|
907
|
+
```
|
|
908
|
+
|
|
909
|
+
- **Backend**: http://localhost:8000
|
|
910
|
+
- **Frontend**: http://localhost:8501
|
|
911
|
+
- Backend has 4GB memory limit, frontend has 1GB
|
|
912
|
+
- Health checks are configured for both services
|
|
913
|
+
|
|
914
|
+
### Manual Installation
|
|
915
|
+
|
|
916
|
+
**Backend:**
|
|
917
|
+
```bash
|
|
918
|
+
cd backend
|
|
919
|
+
python -m venv venv
|
|
920
|
+
source venv/bin/activate # or venv\Scripts\activate on Windows
|
|
921
|
+
pip install -r requirements.txt
|
|
922
|
+
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
|
923
|
+
```
|
|
924
|
+
|
|
925
|
+
**Frontend:**
|
|
926
|
+
```bash
|
|
927
|
+
cd frontend
|
|
928
|
+
python -m venv venv
|
|
929
|
+
source venv/bin/activate # or venv\Scripts\activate on Windows
|
|
930
|
+
pip install -r requirements.txt
|
|
931
|
+
streamlit run app.py --server.port 8501
|
|
932
|
+
```
|
|
933
|
+
|
|
934
|
+
---
|
|
935
|
+
|
|
936
|
+
## π Project Structure
|
|
937
|
+
|
|
938
|
+
```
|
|
939
|
+
ICSF/
|
|
940
|
+
βββ backend/
|
|
941
|
+
β βββ main.py # FastAPI entrypoint (1636 lines, 20+ endpoints)
|
|
942
|
+
β βββ config.py # Config class (AWS, Bedrock, GitHub credentials)
|
|
943
|
+
β βββ credentials.yaml # GitHub PAT + user info
|
|
944
|
+
β βββ logging_config.py # Global logging configuration
|
|
945
|
+
β βββ .env # AWS credentials (not committed)
|
|
946
|
+
β β
|
|
947
|
+
β βββ models/
|
|
948
|
+
β β βββ agent_models.py # 10 Pydantic models for pipeline data flow
|
|
949
|
+
β β
|
|
950
|
+
β βββ services/ # 14 service files
|
|
951
|
+
β β βββ bedrock_service.py # AWS Bedrock LLM wrapper (Claude, Llama, Titan)
|
|
952
|
+
β β βββ github_service.py # GitHub API client (repos, orgs, file trees)
|
|
953
|
+
β β βββ vulnerability_service.py # CSV parsing & repo mapping (833 lines)
|
|
954
|
+
β β βββ dependency_service.py # Java dependency graph engine (2037 lines)
|
|
955
|
+
β β βββ fix_orchestrator.py # Multi-agent pipeline controller
|
|
956
|
+
β β βββ batch_fix_service.py # Batch vulnerability processing
|
|
957
|
+
β β βββ pr_manager_service.py # Git operations & PR creation (1359 lines)
|
|
958
|
+
β β βββ batch_pr_service.py # Aggregated PR creation
|
|
959
|
+
β β βββ atlas_service.py # Testing pipeline faΓ§ade
|
|
960
|
+
β β βββ fix_validator_service.py # Post-fix build/test validation
|
|
961
|
+
β β βββ job_manager.py # Async job & SSE streaming
|
|
962
|
+
β β βββ run_history.py # SQLite run persistence
|
|
963
|
+
β β βββ cost_guard.py # LLM cost limiter ($5/run default)
|
|
964
|
+
β β
|
|
965
|
+
β βββ agents/ # 7 agent files (Cognitive Fixing Loop)
|
|
966
|
+
β β βββ code_context_agent.py # Blast radius mapper (642 lines)
|
|
967
|
+
β β βββ fix_strategy_agent.py # Surgical planner (633 lines)
|
|
968
|
+
β β βββ code_fix_agent.py # Multi-file code generator (1071 lines)
|
|
969
|
+
β β βββ safety_validator_agent.py # Logic gate validator (371 lines)
|
|
970
|
+
β β βββ codebase_analysis_agent.py # Repository intelligence (594 lines)
|
|
971
|
+
β β βββ agent_improvements.py # Helpers: ImportManager, SyntaxValidator, etc.
|
|
972
|
+
β β
|
|
973
|
+
β βββ atlas/ # Self-Healing Testing Framework
|
|
974
|
+
β β βββ orchestrator/
|
|
975
|
+
β β β βββ run_pipeline.py # Pipeline core (1412 lines)
|
|
976
|
+
β β βββ agents/
|
|
977
|
+
β β β βββ build_mechanic.py # Build failure auto-repair (1133 lines)
|
|
978
|
+
β β β βββ test_healer.py # Test failure doctor (151 lines)
|
|
979
|
+
β β β βββ models.py # Agent data models
|
|
980
|
+
β β βββ rag/
|
|
981
|
+
β β β βββ store.py # SQLite vector RAG store (210 lines)
|
|
982
|
+
β β βββ llm/
|
|
983
|
+
β β β βββ bedrock.py # Atlas Bedrock client (163 lines)
|
|
984
|
+
β β βββ generation/
|
|
985
|
+
β β β βββ java_unit_test_generator.py # RAG-enhanced test gen (441 lines)
|
|
986
|
+
β β βββ build/
|
|
987
|
+
β β β βββ maven.py # Maven command runner
|
|
988
|
+
β β β βββ jacoco_injector.py # JaCoCo coverage plugin injection
|
|
989
|
+
β β β βββ spring_test_injector.py # Spring test dependency injection
|
|
990
|
+
β β β βββ failsafe_injector.py # Failsafe plugin injection
|
|
991
|
+
β β β βββ dependency_governance.py # Dependency version governance
|
|
992
|
+
β β βββ core/
|
|
993
|
+
β β β βββ config.py # Atlas configuration
|
|
994
|
+
β β β βββ logging.py # RunLogger
|
|
995
|
+
β β β βββ state.py # Pipeline state manager
|
|
996
|
+
β β β βββ shell.py # Safe shell execution
|
|
997
|
+
β β β βββ resilience.py # Retry, circuit breaker, rate limiter
|
|
998
|
+
β β βββ analysis/
|
|
999
|
+
β β β βββ java_maven.py # Java project fact detection
|
|
1000
|
+
β β β βββ contract_service.py # Constructor/method signature registry
|
|
1001
|
+
β β β βββ diff_analyzer.py # Git diff β functional change detection
|
|
1002
|
+
β β βββ reporting/
|
|
1003
|
+
β β β βββ models.py # Report dataclasses
|
|
1004
|
+
β β β βββ parsers.py # Surefire XML & JaCoCo CSV parsers
|
|
1005
|
+
β β βββ gitops/
|
|
1006
|
+
β β β βββ github_pr.py # PR creation for generated tests
|
|
1007
|
+
β β β βββ github_issues.py # Issue creation for failures
|
|
1008
|
+
β β β βββ github_org.py # Organization repo listing
|
|
1009
|
+
β β βββ repo/
|
|
1010
|
+
β β βββ cloner.py # Repository cloning
|
|
1011
|
+
β β βββ history.py # Run history tracking
|
|
1012
|
+
β β
|
|
1013
|
+
β βββ scripts/ # Utility & test scripts
|
|
1014
|
+
β β βββ test_bedrock_connection.py
|
|
1015
|
+
β β βββ test_cross_repo_dependencies.py
|
|
1016
|
+
β β βββ test_dependency_analysis.py
|
|
1017
|
+
β β βββ test_orchestrator.py
|
|
1018
|
+
β β βββ analyze_all_matched_files.py
|
|
1019
|
+
β β βββ visualize_dependency_mapping.py
|
|
1020
|
+
β β
|
|
1021
|
+
β βββ data/ # SQLite databases & logs
|
|
1022
|
+
β βββ runs.db # Pipeline run history
|
|
1023
|
+
β βββ atlas_rag.db # RAG vector store
|
|
1024
|
+
β
|
|
1025
|
+
βββ frontend/
|
|
1026
|
+
β βββ app.py # Streamlit UI (5676 lines)
|
|
1027
|
+
β βββ src/
|
|
1028
|
+
β β βββ vulnerability_ui.py # Vulnerability display components
|
|
1029
|
+
β β βββ lineage.py # Lineage graph data transforms
|
|
1030
|
+
β βββ utils/
|
|
1031
|
+
β β βββ atlas_report_comprehensive.py # Atlas report rendering
|
|
1032
|
+
β β βββ integrate_render.py # Report integration helpers
|
|
1033
|
+
β βββ requirements.txt
|
|
1034
|
+
β βββ Dockerfile
|
|
1035
|
+
β
|
|
1036
|
+
βββ docker-compose.yml # Multi-container setup
|
|
1037
|
+
βββ start_frontend.bat # Windows frontend launcher
|
|
1038
|
+
βββ start_frontend.sh # Linux/Mac frontend launcher
|
|
1039
|
+
```
|
|
1040
|
+
|
|
1041
|
+
---
|
|
1042
|
+
|
|
1043
|
+
## π‘ API Reference
|
|
1044
|
+
|
|
1045
|
+
### Health & Credentials
|
|
1046
|
+
|
|
1047
|
+
| Endpoint | Method | Description |
|
|
1048
|
+
|---|---|---|
|
|
1049
|
+
| `/api/health` | GET | Health check (Docker/LB probes) |
|
|
1050
|
+
| `/api/credentials/github` | GET | Retrieve loaded GitHub credentials |
|
|
1051
|
+
| `/api/credentials/verify` | GET | Debug credential loading |
|
|
1052
|
+
|
|
1053
|
+
### Repository Management
|
|
1054
|
+
|
|
1055
|
+
| Endpoint | Method | Description |
|
|
1056
|
+
|---|---|---|
|
|
1057
|
+
| `/api/github/repos` | POST/GET | Fetch GitHub repositories |
|
|
1058
|
+
|
|
1059
|
+
### Vulnerability Management
|
|
1060
|
+
|
|
1061
|
+
| Endpoint | Method | Description |
|
|
1062
|
+
|---|---|---|
|
|
1063
|
+
| `/api/vulnerabilities/map` | POST | Upload CSV + map vulnerabilities |
|
|
1064
|
+
| `/api/dependencies/analyze` | POST | Single vulnerability dependency analysis |
|
|
1065
|
+
| `/api/dependencies/batch-analyze` | POST | Batch dependency analysis |
|
|
1066
|
+
|
|
1067
|
+
### Fix Operations
|
|
1068
|
+
|
|
1069
|
+
| Endpoint | Method | Description |
|
|
1070
|
+
|---|---|---|
|
|
1071
|
+
| `/api/fix/orchestrate` | POST | Full multi-agent fix pipeline |
|
|
1072
|
+
| `/api/fix/batch` | POST | Batch fix multiple vulnerabilities |
|
|
1073
|
+
|
|
1074
|
+
### Testing Pipeline
|
|
1075
|
+
|
|
1076
|
+
| Endpoint | Method | Description |
|
|
1077
|
+
|---|---|---|
|
|
1078
|
+
| `/api/testing/start` | POST | Start async testing job |
|
|
1079
|
+
| `/api/testing/job/{job_id}` | GET | Poll job status |
|
|
1080
|
+
| `/api/testing/stream/{job_id}` | GET | SSE event stream |
|
|
1081
|
+
| `/api/testing/runs` | GET | Pipeline run history |
|
|
1082
|
+
| `/api/testing/run` | POST | Legacy sync testing |
|
|
1083
|
+
|
|
1084
|
+
### Pull Request Management
|
|
1085
|
+
|
|
1086
|
+
| Endpoint | Method | Description |
|
|
1087
|
+
|---|---|---|
|
|
1088
|
+
| `/api/pr/create` | POST | Create single PR |
|
|
1089
|
+
| `/api/pr/create-batch` | POST | Create aggregated PR |
|
|
1090
|
+
| `/api/pr/merge` | POST | Merge with conflict resolution |
|
|
1091
|
+
| `/api/pr/check-mergeability` | POST | Check PR mergeability |
|
|
1092
|
+
|
|
1093
|
+
---
|
|
1094
|
+
|
|
1095
|
+
*ICSF β Making code security intelligent, automated, and reliable.*
|