skylos 2.1.2__tar.gz → 2.2.3__tar.gz

{skylos-2.1.2 → skylos-2.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: skylos
-Version: 2.1.2
+Version: 2.2.3
 Summary: A static analysis tool for Python codebases
 Author-email: oha <aaronoh2015@gmail.com>
 Requires-Python: >=3.9

{skylos-2.1.2 → skylos-2.2.3}/README.md

@@ -32,6 +32,7 @@
 - [Example Output](#example-output)
 - [Interactive Mode](#interactive-mode)
 - [Development](#development)
+- [CI/CD (Pre-commit & GitHub Actions)](#cicd-pre-commit--github-actions)
 - [FAQ](#faq)
 - [Limitations](#limitations)
 - [Troubleshooting](#troubleshooting)
@@ -51,6 +52,8 @@
 * **Unused Imports**: Identifies imports that are not used
 * **Folder Management**: Inclusion/exclusion of directories 
 * **Ignore Pragmas**: Skip lines tagged with `# pragma: no skylos`, `# pragma: no cover`, or `# noqa`
+**NEW** **Secrets Scanning (PoC, opt-in)**: Detects API keys & secrets (GitHub, GitLab, Slack, Stripe, AWS, Google, SendGrid, Twilio, private key blocks)
+
 
 ## Benchmark (You can find this benchmark test in `test` folder)
 
@@ -94,12 +97,17 @@ pip install .
 ```bash
 skylos /path/to/your/project
 
+skylos /path/to/your/project --secrets  ## include api key scan
+
 # To launch the front end
 skylos run
 
 # Interactive mode - select items to remove
 skylos --interactive /path/to/your/project 
 
+# Comment out items
+skylos . --interactive --comment-out
+
 # Dry run - see what would be removed
 skylos --interactive --dry-run /path/to/your/project 
 
@@ -193,7 +201,7 @@ When Skylos detects a test file, it by default, will apply a confidence penalty
 
 ## Ignoring Pragmas
 
-To ignore any warning, indicate `# pragma: no skylos` **ON THE SAME LINE** as the function/class you want to ignore
+1. To ignore any warning, indicate `# pragma: no skylos` **ON THE SAME LINE** as the function/class you want to ignore
 
 Example
 
@@ -203,6 +211,14 @@ Example
         return new_path
 ```
 
+2. To suppress a **secret** on a line, add: `# skylos: ignore[SKY-S101]`
+ 
+Example
+ ```python
+
+API_KEY = "ghp_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"  # skylos: ignore[SKY-S101]
+```
+
 ## Including & Excluding Files
 
 ### Default Exclusions
@@ -237,6 +253,7 @@ Options:
   --json                       Output raw JSON instead of formatted text  
   -o, --output FILE            Write output to file instead of stdout
   -v, --verbose                Enable verbose output
+  --version                    Checks version
   -i, --interactive            Interactively select items to remove
   --dry-run                    Show what would be removed without modifying files
   --exclude-folder FOLDER      Exclude a folder from analysis (can be used multiple times)
@@ -244,6 +261,7 @@ Options:
   --no-default-excludes        Don't exclude default folders (__pycache__, .git, venv, etc.)
   --list-default-excludes      List the default excluded folders and
   -c, --confidence LEVEL       Confidence threshold (0-100). Lower values will show more items.
+  -- secrets                   Scan for api keys/secrets
 ```
 
 ## Interactive Mode
@@ -254,6 +272,206 @@ The interactive mode lets you select specific functions and imports to remove:
 2. **Confirm changes**: Review selected items before applying
 3. **Auto-cleanup**: Files are automatically updated
 
+## CI/CD (Pre-commit & GitHub Actions)
+
+Pick **one** (or use **both**) 
+
+1. Pre-commit (local + CI): runs Skylos before commits/PRs.
+   - You must install pre-commit locally once. Skylos gets installed automatically by the hook.
+
+2. GitHub Actions: runs Skylos on pushes/PRs in CI.
+   - No local install needed
+
+### Option A — Pre-commit (local + CI)
+
+1. Create or edit `.pre-commit-config.yaml` at the repo root:
+
+**A: Skylos hook repo**
+```yaml
+## .pre-commit-config.yaml
+repos:
+  - repo: https://github.com/duriantaco/skylos
+    rev: v2.2.3
+    hooks:
+      - id: skylos-scan
+        name: skylos report
+        entry: python -m skylos.cli
+        language: python
+        types_or: [python]
+        pass_filenames: false
+        require_serial: true
+        args: [".", "--output", "report.json", "--confidence", "70"]
+
+  - repo: local
+    hooks:
+      - id: skylos-fail-on-findings
+        name: skylos
+        env:
+          SKYLOS_SOFT: "1"
+        language: python
+        language_version: python3
+        pass_filenames: false
+        require_serial: true
+        entry: >
+          python -c "import os, json, sys, pathlib;
+          p=pathlib.Path('report.json');
+
+          if not p.exists(): 
+            sys.exit(0);
+
+          data=json.loads(p.read_text(encoding='utf-8'));
+
+          count = 0
+          for v in data.values():
+            if isinstance(v, list):
+              count += len(v)
+
+          print(f'[skylos] findings: {count}');
+          sys.exit(0 if os.getenv('SKYLOS_SOFT') or count==0 else 1)"
+```
+**B: self-contained local hook**
+
+```yaml
+repos:
+  - repo: local
+    hooks:
+      - id: skylos-scan
+        name: skylos report
+        language: python
+        entry: python -m skylos.cli
+        pass_filenames: false
+        require_serial: true
+        additional_dependencies: [skylos==2.2.3]
+        args: [".", "--output", "report.json", "--confidence", "70"]
+
+      - id: skylos-fail-on-findings
+        name: skylos (soft)
+        language: python
+        language_version: python3
+        pass_filenames: false
+        require_serial: true
+        entry: >
+          python -c "import os, json, sys, pathlib;
+          p=pathlib.Path('report.json');
+
+          if not p.exists(): 
+            sys.exit(0);
+
+          data=json.loads(p.read_text(encoding='utf-8'));
+
+          count = 0
+          for v in data.values():
+            if isinstance(v, list):
+              count += len(v)
+
+          print(f'[skylos] findings: {count}');
+          sys.exit(0 if os.getenv('SKYLOS_SOFT') or count==0 else 1)"
+```
+
+**Install requirements:**
+
+You must install pre-commit locally once:
+```bash
+pip install pre-commit
+pre-commit install
+```
+
+2. pre-commit run --all-files
+
+
+3. Run the same hooks in CI (GitHub Actions): create .github/workflows/pre-commit.yml:
+
+```yaml
+name: pre-commit
+on: [push, pull_request]
+jobs:
+  run:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with: { python-version: "3.11", cache: "pip" }
+      - uses: pre-commit/action@v3.0.1
+        with: { extra_args: --all-files }
+```
+
+**Pre commit behavior:** the second hook is soft by default (SKYLOS_SOFT=1). This means that it prints findings and passes. You can remove the env/logic if you want pre-commit to block commits on finding
+
+### Option B — Github Actions
+
+1. Create .github/workflows/skylos.yml:
+
+```yaml
+name: Skylos Deadcode Scan
+
+on:
+  pull_request:
+  push:
+    branches: [ main, master ]
+  workflow_dispatch:
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    env:
+      SKYLOS_STRICT: ${{ vars.SKYLOS_STRICT || 'false' }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+          cache: 'pip'
+
+      - name: Install Skylos
+        run: pip install skylos
+
+      - name: Run Skylos
+        env:
+          REPORT: skylos_${{ github.run_number }}_${{ github.sha }}.json
+        run: |
+          echo "REPORT=$REPORT" >> "$GITHUB_OUTPUT"
+          skylos . --json > "$REPORT"
+        id: scan
+
+      - name: Fail if there are findings
+        continue-on-error: ${{ env.SKYLOS_STRICT != 'true' }}
+        env:
+          REPORT: ${{ steps.scan.outputs.REPORT }}
+        run: |
+            python - << 'PY'
+            import json, sys, os
+            report = os.environ["REPORT"]
+            data = json.load(open(report, "r", encoding="utf-8"))
+            count = 0
+            for value in data.values():
+                if isinstance(value, list):
+                    count += len(value)
+            print(f"Findings: {count}")
+            if count > 0:
+              print(f"::warning title=Skylos findings::{count} potential issues found. See {report}")
+            sys.exit(1 if count > 0 else 0)
+            PY
+
+      - name: Upload report artifact
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ steps.scan.outputs.REPORT }}
+          path: ${{ steps.scan.outputs.REPORT }}
+
+      - name: Summarize in job log
+        if: always()
+        run: |
+          echo "Skylos report: ${{ steps.scan.outputs.REPORT }}" >> $GITHUB_STEP_SUMMARY
+```
+
+**To make the job fail on findings (strict mode)**:
+
+1. Go to GitHub -> Settings -> Secrets and variables -> Actions -> Variables
+
+2. Add variable SKYLOS_STRICT with value true
+
 ## Development
 
 ### Prerequisites
@@ -307,6 +525,7 @@ A: Start with 60 (default) for safe cleanup. Use 30 for framework applications.
 - **Frameworks**: Django models, Flask, FastAPI routes may appear unused but aren't
 - **Test data**: Limited scenarios, your mileage may vary
 - **False positives**: Always manually review before deleting code
+- **Secrets PoC**: May emit both a provider hit and a generic high-entropy hit for the same token. All tokens are detected only in py files (`.py`, `.pyi`, `.pyw`).
 
 ## Troubleshooting
 
@@ -339,9 +558,10 @@ We welcome contributions! Please read our [Contributing Guidelines](CONTRIBUTING
 ## Roadmap
 - [x] Expand our test cases
 - [ ] Configuration file support 
-- [ ] Git hooks integration
-- [ ] CI/CD integration examples
+- [x] Git hooks integration
+- [x] CI/CD integration examples
 - [ ] Further optimization
+- [ ] Add new rules
 
 ## License
 

{skylos-2.1.2 → skylos-2.2.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "skylos"
-version = "2.1.2"
+version = "2.2.3"
 requires-python = ">=3.9"
 description = "A static analysis tool for Python codebases"
 authors = [{name = "oha", email = "aaronoh2015@gmail.com"}]

{skylos-2.1.2 → skylos-2.2.3}/setup.py

@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
 
 setup(
     name="skylos",
-    version="2.1.2",
+    version="2.2.3",
     packages=find_packages(),
     python_requires=">=3.9",
     install_requires=[

skylos-2.2.3/skylos/__init__.py

@@ -0,0 +1,10 @@
+__version__ = "2.2.3"
+
+def analyze(*args, **kwargs):
+    from .analyzer import analyze as _analyze
+    return _analyze(*args, **kwargs)
+
+def debug_test():
+    return "debug-ok"
+
+__all__ = ["analyze", "debug_test", "__version__"]

{skylos-2.1.2 → skylos-2.2.3}/skylos/analyzer.py

@@ -7,10 +7,11 @@ from pathlib import Path
 from collections import defaultdict
 from skylos.visitor import Visitor
 from skylos.constants import ( PENALTIES, AUTO_CALLED )
-from skylos.test_aware import TestAwareVisitor
+from skylos.visitors.test_aware import TestAwareVisitor
+from skylos.rules.secrets import scan_ctx as _secrets_scan_ctx
 import os
 import traceback
-from skylos.framework_aware import FrameworkAwareVisitor, detect_framework_usage
+from skylos.visitors.framework_aware import FrameworkAwareVisitor, detect_framework_usage
 
 logging.basicConfig(level=logging.INFO,format='%(asctime)s - %(levelname)s - %(message)s')
 logger=logging.getLogger('Skylos')
@@ -237,7 +238,7 @@ class Skylos:
                     if method.simple_name == "format" and cls.endswith("Formatter"):
                         method.references += 1
 
-    def analyze(self, path, thr=60, exclude_folders=None):
+    def analyze(self, path, thr=60, exclude_folders= None, enable_secrets = False):
         files, root = self._get_python_files(path, exclude_folders)
         
         if not files:
@@ -260,6 +261,7 @@ class Skylos:
         for f in files:
             modmap[f] = self._module(root, f)
         
+        all_secrets = []
         for file in files:
             mod = modmap[file]
             defs, refs, dyn, exports, test_flags, framework_flags = proc_file(file, mod)
@@ -272,6 +274,16 @@ class Skylos:
             self.dynamic.update(dyn)
             self.exports[mod].update(exports)
 
+            if enable_secrets and _secrets_scan_ctx is not None:
+                try:
+                    src_lines = Path(file).read_text(encoding="utf-8", errors="ignore").splitlines(True)
+                    ctx = {"relpath": str(file), "lines": src_lines, "tree": None}
+                    findings = list(_secrets_scan_ctx(ctx))
+                    if findings:
+                        all_secrets.extend(findings)
+                except Exception:
+                    pass
+
         self._mark_refs()
         self._apply_heuristics() 
         self._mark_exports()
@@ -303,6 +315,9 @@ class Skylos:
                 "excluded_folders": exclude_folders or [],
             }
         }
+
+        if enable_secrets and all_secrets:
+            result["secrets"] = all_secrets
         
         for u in unused:
             if u["type"] in ("function", "method"):
@@ -355,8 +370,8 @@ def proc_file(file_or_args, mod=None):
 
         return [], [], set(), set(), dummy_visitor, dummy_framework_visitor
 
-def analyze(path,conf=60, exclude_folders=None):
-    return Skylos().analyze(path,conf, exclude_folders)
+def analyze(path, conf=60, exclude_folders=None, enable_secrets=False):
+    return Skylos().analyze(path,conf, exclude_folders, enable_secrets)
 
 if __name__ == "__main__":
     if len(sys.argv)>1:

{skylos-2.1.2 → skylos-2.2.3}/skylos/cli.py

@@ -8,6 +8,8 @@ from skylos.analyzer import analyze as run_analyze
 from skylos.codemods import (
     remove_unused_import_cst,
     remove_unused_function_cst,
+    comment_out_unused_import_cst,
+    comment_out_unused_function_cst, 
 )
 import pathlib
 import skylos
@@ -80,6 +82,32 @@ def remove_unused_function(file_path, function_name, line_number):
     except Exception as e:
         logging.error(f"Failed to remove function {function_name} from {file_path}: {e}")
         return False
+    
+def comment_out_unused_import(file_path, import_name, line_number, marker="SKYLOS DEADCODE"):
+    path = pathlib.Path(file_path)
+    try:
+        src = path.read_text(encoding="utf-8")
+        new_code, changed = comment_out_unused_import_cst(src, import_name, line_number, marker=marker)
+        if not changed:
+            return False
+        path.write_text(new_code, encoding="utf-8")
+        return True
+    except Exception as e:
+        logging.error(f"Failed to comment out import {import_name} from {file_path}: {e}")
+        return False
+
+def comment_out_unused_function(file_path, function_name, line_number, marker="SKYLOS DEADCODE"):
+    path = pathlib.Path(file_path)
+    try:
+        src = path.read_text(encoding="utf-8")
+        new_code, changed = comment_out_unused_function_cst(src, function_name, line_number, marker=marker)
+        if not changed:
+            return False
+        path.write_text(new_code, encoding="utf-8")
+        return True
+    except Exception as e:
+        logging.error(f"Failed to comment out function {function_name} from {file_path}: {e}")
+        return False
 
 def interactive_selection(logger, unused_functions, unused_imports):
     if not INTERACTIVE_AVAILABLE:
@@ -110,7 +138,7 @@ def interactive_selection(logger, unused_functions, unused_imports):
             selected_functions = answers['functions']
     
     if unused_imports:
-        logger.info(f"\n{Colors.MAGENTA}{Colors.BOLD}Select unused imports to remove (hit spacebar to select):{Colors.RESET}")
+        logger.info(f"\n{Colors.MAGENTA}{Colors.BOLD}Select unused imports to act on (hit spacebar to select):{Colors.RESET}")
         
         import_choices = []
 
@@ -172,6 +200,13 @@ def main():
         action="store_true",
         help="Output raw JSON",
     )
+
+    parser.add_argument(
+        "--comment-out",
+        action="store_true",
+        help="Comment out selected dead code instead of deleting it",
+    )
+
     parser.add_argument(
         "--output",
         "-o",
@@ -232,12 +267,15 @@ def main():
         help="List the default excluded folders and exit."
     )
 
+    parser.add_argument("--secrets", action="store_true",
+                   help="Scan for API keys. Off by default.")
+    
     args = parser.parse_args()
 
     if args.list_default_excludes:
         print("Default excluded folders:")
         for folder in sorted(DEFAULT_EXCLUDE_FOLDERS):
-            print(f"  {folder}")
+            print(f" {folder}")
         print(f"\nTotal: {len(DEFAULT_EXCLUDE_FOLDERS)} folders")
         print("\nUse --no-default-excludes to disable these exclusions")
         print("Use --include-folder <folder> to force include specific folders")
@@ -266,7 +304,7 @@ def main():
             logger.info(f"{Colors.GREEN}📁 No folders excluded{Colors.RESET}")
 
     try:
-        result_json = run_analyze(args.path, conf=args.confidence, exclude_folders=list(final_exclude_folders))
+        result_json = run_analyze(args.path, conf=args.confidence, enable_secrets=bool(args.secrets), exclude_folders=list(final_exclude_folders))
         result = json.loads(result_json)
 
     except Exception as e:
@@ -282,6 +320,7 @@ def main():
     unused_parameters = result.get("unused_parameters", [])
     unused_variables = result.get("unused_variables", [])
     unused_classes = result.get("unused_classes", [])
+    secrets_findings = result.get("secrets", [])
     
     logger.info(f"{Colors.CYAN}{Colors.BOLD} Python Static Analysis Results{Colors.RESET}")
     logger.info(f"{Colors.CYAN}{'=' * 35}{Colors.RESET}")
@@ -292,49 +331,75 @@ def main():
     logger.info(f" * Unused parameters: {Colors.YELLOW}{len(unused_parameters)}{Colors.RESET}")
     logger.info(f" * Unused variables: {Colors.YELLOW}{len(unused_variables)}{Colors.RESET}")
     logger.info(f" * Unused classes: {Colors.YELLOW}{len(unused_classes)}{Colors.RESET}")
+    if secrets_findings:
+        logger.info(f" * Secrets: {Colors.RED}{len(secrets_findings)}{Colors.RESET}")
 
     if args.interactive and (unused_functions or unused_imports):
         logger.info(f"\n{Colors.BOLD}Interactive Mode:{Colors.RESET}")
         selected_functions, selected_imports = interactive_selection(logger, unused_functions, unused_imports)
         
         if selected_functions or selected_imports:
-            logger.info(f"\n{Colors.BOLD}Selected items to remove:{Colors.RESET}")
+            logger.info(f"\n{Colors.BOLD}Selected items to process:{Colors.RESET}")
             
             if selected_functions:
-                logger.info(f"  Functions: {len(selected_functions)}")
+                logger.info(f" Functions: {len(selected_functions)}")
                 for func in selected_functions:
-                    logger.info(f"    - {func['name']} ({func['file']}:{func['line']})")
+                    logger.info(f"  - {func['name']} ({func['file']}: {func['line']})")
             
             if selected_imports:
-                logger.info(f"  Imports: {len(selected_imports)}")
+                logger.info(f" Imports: {len(selected_imports)}")
                 for imp in selected_imports:
-                    logger.info(f"    - {imp['name']} ({imp['file']}:{imp['line']})")
+                    logger.info(f"  - {imp['name']} ({imp['file']}: {imp['line']})")
             
             if not args.dry_run:
+                if args.comment_out:
+                    confirm_verb = "comment out"
+                else:
+                    confirm_verb = "remove"
+
                 questions = [
                     inquirer.Confirm('confirm',
-                                   message="Are you sure you want to remove these items?",
+                                   message="Are you sure you want to process these items?",
                                    default=False)
                 ]
                 answers = inquirer.prompt(questions)
                 
                 if answers and answers['confirm']:
-                    logger.info(f"\n{Colors.YELLOW}Removing selected items...{Colors.RESET}")
-                    
+                    action = "Commenting out" if args.comment_out else "Removing"
+                    logger.info(f"\n{Colors.YELLOW}{action} selected items...{Colors.RESET}")
+
+                    action_func = comment_out_unused_function if args.comment_out else remove_unused_function
+                    if args.comment_out:
+                        action_past = "Commented out"
+                        action_verb = "comment out"
+                    else:
+                        action_past = "Removed"
+                        action_verb = "remove"
+
                     for func in selected_functions:
-                        success = remove_unused_function(func['file'], func['name'], func['line'])
+                        success = action_func(func['file'], func['name'], func['line'])
+                        
                         if success:
-                            logger.info(f"  {Colors.GREEN} {Colors.RESET} Removed function: {func['name']}")
+                            logger.info(f"  {Colors.GREEN} ✓ {Colors.RESET} {action_past} function: {func['name']}")
                         else:
-                            logger.error(f"  {Colors.RED} x {Colors.RESET} Failed to remove: {func['name']}")
-                    
+                            logger.error(f"  {Colors.RED} x {Colors.RESET} Failed to {action_verb}: {func['name']}")
+                            
+                    import_func = comment_out_unused_import if args.comment_out else remove_unused_import
+                    if args.comment_out:
+                        action_past = "Commented out"
+                        action_verb = "comment out"
+                    else:
+                        action_past = "Removed"
+                        action_verb = "remove"
+
                     for imp in selected_imports:
-                        success = remove_unused_import(imp['file'], imp['name'], imp['line'])
+                        success = import_func(imp['file'], imp['name'], imp['line'])
+                        
                         if success:
-                            logger.info(f"  {Colors.GREEN} {Colors.RESET} Removed import: {imp['name']}")
+                            logger.info(f"  {Colors.GREEN} ✓ {Colors.RESET} {action_past} import: {imp['name']}")
                         else:
-                            logger.error(f"  {Colors.RED} x {Colors.RESET} Failed to remove: {imp['name']}")
-                    
+                            logger.error(f"  {Colors.RED} x {Colors.RESET} Failed to {action_verb}: {imp['name']}")
+                            
                     logger.info(f"\n{Colors.GREEN}Cleanup complete!{Colors.RESET}")
                 else:
                     logger.info(f"\n{Colors.YELLOW}Operation cancelled.{Colors.RESET}")
@@ -389,6 +454,16 @@ def main():
         else:
             logger.info(f"\n{Colors.GREEN}✓ All classes are being used!{Colors.RESET}")
 
+        if secrets_findings:
+            logger.info(f"\n{Colors.RED}{Colors.BOLD} - Secrets{Colors.RESET}")
+            logger.info(f"{Colors.RED}{'=' * 9}{Colors.RESET}")
+            for i, s in enumerate(secrets_findings[:20], 1):
+                provider = s.get("provider", "generic")
+                where = f"{s.get('file','?')}:{s.get('line','?')}"
+                prev = s.get("preview", "****")
+                msg = s.get("message", "Secret detected")
+                logger.info(f"{Colors.GRAY}{i:2d}. {Colors.RESET}{msg} [{provider}] {Colors.GRAY}({where}){Colors.RESET} -> {prev}")
+
         dead_code_count = len(unused_functions) + len(unused_imports) + len(unused_variables) + len(unused_classes) + len(unused_parameters)
 
         print_badge(dead_code_count, logger)