repo-security-scanner 0.1.0__tar.gz

repo_security_scanner-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 MR-UNKNOWN8014
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

repo_security_scanner-0.1.0/PKG-INFO

@@ -0,0 +1,475 @@
+Metadata-Version: 2.4
+Name: repo-security-scanner
+Version: 0.1.0
+Summary: Pre-clone security scanner for Git repositories: malicious code, secrets, vulnerable dependencies, and Dockerfile issues
+Author: MR-UNKNOWN8014
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/MR-UNKNOWN8014/repo-security-scanner
+Project-URL: Issues, https://github.com/MR-UNKNOWN8014/repo-security-scanner/issues
+Keywords: security,scanner,malware,secrets,supply-chain,sast,dockerfile
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Security
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: colorama>=0.4.6
+Requires-Dist: tqdm>=4.64.0
+Dynamic: license-file
+
+# Repository Security Scanner
+
+![CI](https://github.com/MR-UNKNOWN8014/repo-security-scanner/actions/workflows/tests.yml/badge.svg)
+![Python 3.8+](https://img.shields.io/badge/python-3.8%2B-blue)
+![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)
+![PyPI](https://img.shields.io/pypi/v/repo-security-scanner)
+![Stars](https://img.shields.io/github/stars/MR-UNKNOWN8014/repo-security-scanner?style=social)
+
+> A Python-based security tool that analyzes Git repositories for malicious code, backdoors, obfuscation, and suspicious patterns before you clone them. It provides risk scoring and actionable recommendations.
+
+The scanner performs static analysis on repository files, detecting:
+- Crypto miner patterns
+- Backdoor signatures
+- Data exfiltration code
+- Obfuscation techniques
+- Dangerous function calls
+- Vulnerable dependencies
+- High-entropy (encrypted/obfuscated) files
+- Hardcoded secrets and API keys
+- Insecure Dockerfile patterns
+
+---
+
+## Why?
+
+Cloning a repo to audit it is already too late — `postinstall` scripts, 
+Makefiles, and CI configs run before you ever open the code. This tool 
+scans **before** `git clone` completes, so you know what you're pulling 
+in before it can touch your machine.
+
+Built because existing tools (gitleaks, trufflehog) focus on *secrets*, 
+and SAST tools (semgrep, bandit) focus on *your* code. Neither answers 
+the question: "is this random GitHub repo safe to run?"
+
+---
+
+## Quick Start
+
+```bash
+# Clone the repository
+git clone https://github.com/MR-UNKNOWN8014/repo-security-scanner.git
+cd repo-security-scanner
+
+# Install dependencies
+./setup.sh          # Linux/macOS
+setup.bat           # Windows
+
+# Scan a repository
+python run_scanner.py https://github.com/user/repo.git
+
+```
+
+---
+## 1. Installation:
+
+**Needs Python 3.8 or Higher to work.**
+
+### 1.2 Linux / macOS
+
+```bash
+# Linux/Mac
+chmod +x setup.sh
+
+# Windows
+./setup.sh
+```
+
+**The installer will**:
+- Verify Python version
+- Install all required dependencies
+- Create the run script
+
+>[!TIP]
+>If Python is not in your PATH, the installer will prompt you to install Python 3.8+ from [python.org](https://python.org/).
+ 
+
+### 1.3 Manual Installation
+
+```bash
+# Create virtual environment (optional)
+python -m venv venv
+
+# Linux/Mac
+source venv/bin/activate 
+
+# Windows
+venv\Scripts\activate
+
+# Install dependencies
+pip install -r requirements.txt
+```
+
+### 1.4 Install as a Package
+
+The project also ships a `pyproject.toml`, so it can be installed as an editable package instead of running the scripts directly:
+
+```bash
+pip install -e .
+
+# now available as a command anywhere in the venv
+repo-scanner https://github.com/user/repo.git
+```
+
+----
+## 2. Usage
+
+### 2.1 Basic Commands
+
+```bash
+# Scan remote repository
+python run_scanner.py https://github.com/user/repo.git
+
+# Scan local repository
+python run_scanner.py /path/to/local/repo
+
+# Quick scan (limited files)
+python run_scanner.py https://github.com/user/repo.git --mode quick
+
+# Thorough scan with detailed output
+python run_scanner.py https://github.com/user/repo.git --mode thorough --detailed
+
+# Export report to JSON
+python run_scanner.py https://github.com/user/repo.git --output report.json
+
+# Check every pinned dependency against OSV.dev's live vulnerability database
+python run_scanner.py https://github.com/user/repo.git --check-vulns
+```
+
+### 2.2 Command Line Arguments
+
+| Option            | Short | Description                                                                     | Default  |
+| ----------------- | ----- | ------------------------------------------------------------------------------- | -------- |
+| `--mode`          | `-m`  | Scan mode: quick, balanced, thorough, smart (detail below on types)             | balanced |
+| `--output`        | `-o`  | Export report to JSON or CSV file                                               | None     |
+| `--verbose`       | `-v`  | Display detailed scan progress                                                  | False    |
+| `--detailed`      | `-d`  | Show detailed breakdown in report                                               | False    |
+| `--format`        | -     | Report format: simple, multi, categories, detailed, all (detail below on types) | all      |
+| `--keep-repo`     | -     | Keep cloned repository after scanning                                           | False    |
+| `--auto-decision` | -     | Automatically decide based on risk score                                        | False    |
+| `--check-vulns`   | -     | Query OSV.dev for live known vulnerabilities in every pinned dependency (sends dependency names/versions to a third-party service) | False    |
+
+### 2.3 Scan Modes
+
+| Mode         | Description                         | Files Scanned                                                                |
+| ------------ | ----------------------------------- | ---------------------------------------------------------------------------- |
+| **Quick**    | Fast scanning with size limits      | Files < 1MB, max 10,000                                                      |
+| **Balanced** | Moderate coverage with good speed   | All valid files                                                              |
+| **Thorough** | Complete scanning                   | All files without limits (will take some time if too many files or big size) |
+| **Smart**    | Prioritizes executable/script files | Scripts, binaries, executables                                               |
+
+>[!TIP]
+>Use `--mode quick` for CI/CD pipelines and `--mode thorough` for security audits.
+
+### 2.4 Ignoring Known False Positives
+
+Drop a `.reposecurityignore` file in the root of the repository being scanned to skip specific files or paths, one glob pattern per line:
+
+```
+# skip test fixtures with fake credentials
+tests/fixtures/*
+*.sample.env
+```
+
+Lines starting with `#` and blank lines are ignored. Patterns match against both the file's path relative to the repo root and its filename.
+
+----
+## 3. Report Formats
+
+The scanner supports four report formats, plus an "all" option that displays them sequentially.
+
+### 3.1 Simple Format
+
+Single percentage score with status bar and risk level.
+
+```bash
+============================================================
+  REPOSITORY: awesome-project
+  RISK SCORE: 23.5%  [████████░░░░░░░░░░░░]   LOW RISK
+============================================================
+```
+
+### 3.2 Multi-Factor Format
+
+Breaks down risk into four categories with individual scores.
+
+```bash
+============================================================
+  REPOSITORY: awesome-project
+  
+  MULTI-FACTOR ANALYSIS
+  Code Quality:  76.3%  [██████████░░░░░░]
+  Security:      76.5%  [██████████░░░░░░]
+  Obfuscation:   16.5%  [██░░░░░░░░░░░░░░]
+  Malicious:     11.8%  [█░░░░░░░░░░░░░░░]
+  
+  OVERALL: 23.5%  LOW RISK
+============================================================
+```
+
+### 3.3 Categories Format
+
+Displays risk category with file breakdown and recommendation.
+
+```bash
+============================================================
+  REPOSITORY: awesome-project
+  
+  [LOW] RISK
+  
+  Risk Level:    [LOW]
+  Score:         23.5%
+  Findings:      12 issues found
+  Files:         0 high, 3 medium, 9 low
+  Recommendation: REVIEW BEFORE CLONING
+============================================================
+```
+
+### 3.4 Detailed Format
+
+Complete report including all findings and statistics.
+
+```bash
+============================================================
+  REPOSITORY: awesome-project
+  
+  [LOW] RISK
+  
+  Risk Level:    [LOW]
+  Score:         23.5%
+  Findings:      12 issues found
+  Files:         0 high, 3 medium, 9 low
+  Recommendation: REVIEW BEFORE CLONING
+============================================================
+
+FILE BREAKDOWN:
+  Safe:        45
+  Low Risk:    9
+  Medium Risk: 3
+  High Risk:   0
+
+STATISTICS:
+  Total Files:     57
+  Scan Duration:   4.32s
+  Findings Found:  12
+
+TOP FINDINGS:
+  1. [MEDIUM] encoding: Multiple base64 strings: 8
+  2. [MEDIUM] vulnerable_dependency: NPM package 'lodash': Prototype pollution vulnerabilities
+  3. [LOW] dangerous_function: Dangerous function: eval
+  4. [LOW] network: Network activity: HTTP request
+```
+
+---
+
+## 4. Risk Scoring
+
+### 4.1 Risk Levels
+
+|Score|Level|Recommendation|
+|---|---|---|
+|0-9|SAFE|Safe to clone|
+|10-24|LOW RISK|Review before cloning|
+|25-49|MEDIUM RISK|Exercise caution|
+|50-74|HIGH RISK|Avoid cloning|
+|75-100|CRITICAL|Do not clone|
+
+### 4.2 Scoring Factors
+
+|Factor|Maximum Points|Description|
+|---|---|---|
+|Malicious Patterns|100|Crypto miners, backdoors, exfiltration|
+|Dangerous Functions|100|eval, exec, system, subprocess|
+|Entropy Analysis|20|High entropy indicates encryption/obfuscation|
+|Base64 Encoding|10|Large base64 strings may contain payloads|
+|File Size|10|Large files may contain hidden payloads|
+|Vulnerable Dependencies|15 per package (20 if confirmed via `--check-vulns`)|Known vulnerable packages|
+|Secret Detection|80 per secret|Hardcoded API keys, tokens, passwords, private keys|
+|Dockerfile Issues|3-20 per finding|Unpinned images, root user, curl-pipe-bash, hardcoded secrets|
+
+---
+
+## 5. Detection Capabilities
+
+### 5.1 Malicious Pattern Categories
+
+|Category|Example Patterns|
+|---|---|
+|Crypto Miners|cryptonight, stratum, xmrig, mining, cpuminer|
+|Backdoors|socket.bind, socket.connect, base64.b64decode|
+|Data Exfiltration|requests.post, discord webhook, smtplib, telegram|
+|Obfuscation|base64, zlib, exec(decode), String.fromCharCode|
+|Shell Commands|subprocess, os.system, popen, eval|
+|Network Activity|socket, requests, urllib, websocket, ftp|
+|File Operations|os.remove, shutil.rmtree, file.write, os.chmod|
+
+### 5.2 Language-Specific Detection
+
+|Language|Detected Functions|
+|---|---|
+|Python|exec, eval, compile, **import**, os.system, subprocess.call, subprocess.Popen, os.popen|
+|JavaScript|eval, Function, setTimeout, setInterval, document.write, innerHTML|
+|Bash|exec, eval, source, export, alias|
+|PHP|eval, system, exec, passthru, shell_exec, assert|
+|Ruby|eval, exec, system, ``, IO.popen, Open3|
+|Go|os/exec, syscall, reflect, unsafe|
+|Rust|std::process, std::fs, unsafe, std::mem|
+|Java|Runtime.exec, ProcessBuilder, System.load, Class.forName|
+
+### 5.3 Vulnerable Dependency Detection
+
+Offline (default): checks pinned versions of the packages below against a known-bad floor. A package is only flagged if the pinned version is actually below the fixed version; unpinned or unparseable version specs are skipped rather than guessed at.
+
+| Package Manager | Checked Packages                                  |
+| --------------- | ------------------------------------------------- |
+| npm             | crypto-js, node-fetch, axios, lodash, request     |
+| pip             | requests, urllib3, paramiko, cryptography, pyyaml |
+
+Online (`--check-vulns`): additionally queries the [OSV.dev](https://osv.dev/) API for every pinned dependency in `requirements.txt`/`package.json`, not just the packages above, against its live vulnerability database. Off by default since it sends dependency names and versions to a third-party service.
+
+### 5.4 Secret Detection
+
+Scans every text file for hardcoded credentials: AWS/GitHub/GitLab/Slack/Google/Stripe/Twilio/SendGrid keys, private key blocks, JWTs, and generic `api_key`/`secret`/`token`/`password` assignments. Matches are reported with a file and line number but the value itself is redacted (`AKIA************WXYZ`), never printed in full. Known placeholder values (`EXAMPLE`, `changeme`, `<your_key_here>`, etc.) are filtered out so docs and templates don't trip it. Any confirmed secret pushes that file's risk score into the CRITICAL range on its own.
+
+Add real fixtures containing fake secrets you need to keep to `.reposecurityignore` (see 2.4) to suppress them.
+
+### 5.5 Dockerfile Scanning
+
+Any `Dockerfile`, `Dockerfile.*`, or `*.dockerfile` is additionally checked for:
+
+| Check | What it catches |
+|---|---|
+| Unpinned base image | `FROM image:latest` or no tag at all |
+| Root user | No `USER` instruction, or an explicit `USER root` |
+| ADD vs COPY | `ADD` used for a local file instead of `COPY` (ADD's extra behavior is only needed for URLs/archives) |
+| Pipe to shell | `curl \| bash`, `wget \| sh`, etc. |
+| Insecure TLS | `curl -k`, `--no-check-certificate` |
+| Hardcoded secret | `ENV`/`ARG` setting a `PASSWORD`/`SECRET`/`TOKEN`/`API_KEY` directly |
+
+---
+
+## 6. Project Structure
+
+```bash
+repo-security-scanner/
+├── setup.sh                 # Linux/macOS installer
+├── setup.bat                # Windows installer
+├── requirements.txt         # Python dependencies
+├── pyproject.toml          # Packaging (pip install -e .)
+├── README.md                # Documentation
+├── LICENSE                  # MIT License
+├── .gitignore              # Git ignore file
+├── run_scanner.py          # Entry point
+├── main.py                 # CLI entry point (main())
+│
+├── .github/
+│   └── workflows/
+│       └── tests.yml       # CI: runs the test suite on push/PR
+│
+├── tests/                   # Unit tests (unittest)
+│   ├── test_entropy_calculator.py
+│   ├── test_pattern_matcher.py
+│   ├── test_dependency_checker.py
+│   ├── test_file_utils.py
+│   ├── test_scoring.py
+│   ├── test_secret_detector.py
+│   └── test_dockerfile_scanner.py
+│
+└── repo_scanner/           # Main package
+    ├── __init__.py
+    ├── config.py           # Configuration and constants
+    ├── cli/                # Command-line interface
+    │   ├── __init__.py
+    │   └── arguments.py    # Argument parsing
+    │
+    ├── scanner/            # Core scanning modules
+    │   ├── __init__.py
+    │   ├── core.py         # Scanner orchestrator
+    │   ├── file_analyzer.py # Individual file analysis
+    │   ├── pattern_matcher.py # Pattern detection engine
+    │   ├── dependency_checker.py # Dependency scanning (offline + OSV.dev)
+    │   ├── entropy_calculator.py # Entropy analysis
+    │   ├── secret_detector.py # Credential/API key/token detection
+    │   └── dockerfile_scanner.py # Dockerfile security linting
+    │
+    ├── report/             # Report generation
+    │   ├── __init__.py
+    │   ├── formatter.py    # Output formatting
+    │   └── exporters.py    # JSON/CSV export
+    │
+    ├── models/             # Data models
+    │   ├── __init__.py
+    │   └── scan_result.py  # Result data structures
+    │
+    └── utils/              # Utility functions
+        ├── __init__.py
+        ├── file_utils.py   # File operations, .reposecurityignore
+        └── git_utils.py    # Git operations
+```
+
+---
+
+## 7. Troubleshooting
+
+|Problem|Check|Solution|
+|---|---|---|
+|Python not found|Python installed?|Install Python 3.8+ from [python.org](https://python.org/)|
+|ModuleNotFoundError|Dependencies installed?|Run `pip install -r requirements.txt`|
+|Git clone failed|Git installed?|Install Git from [git-scm.com](https://git-scm.com/)|
+|Permission denied|File permissions?|`chmod +x setup.sh` (Linux/macOS)|
+|Scan takes too long|Large repository?|Use `--mode quick` or `--mode smart`|
+|Report not generated|Output path valid?|Check write permissions for output directory|
+|High entropy warnings|False positives?|Review the file manually, may be legitimate. Add it to `.reposecurityignore` to suppress it going forward|
+
+---
+
+## 8. Future Enhancements
+
+- [x] Secret detection - API keys, passwords, tokens, credentials
+- [x] Dockerfile scanning - insecure base images, root user, secrets in ENV/ARG, curl-pipe-bash
+- [ ] Git history analysis - scan commit history for secrets and keys
+- [ ] License checker - detect incompatible or restrictive licenses
+- [ ] Binary signature analysis - verify binaries from trusted sources
+- [ ] Database of known malicious repositories
+- [ ] Package registry scanning (PyPI, npm, RubyGems)
+
+
+-----
+
+## 9. Contributing and Bugs
+
+### 9.1 Pull Request Process
+
+1. Fork the repository
+2. Create a feature branch: `git checkout -b feature/AmazingFeature`
+3. Run the test suite locally: `python -m unittest discover -s tests -v`
+4. Commit your changes: `git commit -m 'Add some AmazingFeature'`
+5. Push to the branch: `git push origin feature/AmazingFeature`
+6. Submit a Pull Request
+
+> [!note]  
+> All pull requests must pass the CI checks (.github/workflows/tests.yml, run on Python 3.8 and 3.12) and maintain 100% compatibility with Python 3.8+.
+
+### 9.2 Reporting Issues
+
+When reporting issues on GitHub, include:
+
+- Description of the issue
+- Steps to reproduce
+- Expected behavior
+- Actual behavior
+- Environment (OS, Python version)
+- Logs or screenshots (if applicable)
+
+This all so I can work on those bugs and make the tool better.