skylos 2.1.1__tar.gz → 2.2.2__tar.gz

{skylos-2.1.1 → skylos-2.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: skylos
-Version: 2.1.1
+Version: 2.2.2
 Summary: A static analysis tool for Python codebases
 Author-email: oha <aaronoh2015@gmail.com>
 Requires-Python: >=3.9

{skylos-2.1.1 → skylos-2.2.2}/README.md

@@ -32,6 +32,7 @@
 - [Example Output](#example-output)
 - [Interactive Mode](#interactive-mode)
 - [Development](#development)
+- [CI/CD (Pre-commit & GitHub Actions)](#cicd-pre-commit--github-actions)
 - [FAQ](#faq)
 - [Limitations](#limitations)
 - [Troubleshooting](#troubleshooting)
@@ -51,6 +52,8 @@
 * **Unused Imports**: Identifies imports that are not used
 * **Folder Management**: Inclusion/exclusion of directories 
 * **Ignore Pragmas**: Skip lines tagged with `# pragma: no skylos`, `# pragma: no cover`, or `# noqa`
+**NEW** **Secrets Scanning (PoC, opt-in)**: Detects API keys & secrets (GitHub, GitLab, Slack, Stripe, AWS, Google, SendGrid, Twilio, private key blocks)
+
 
 ## Benchmark (You can find this benchmark test in `test` folder)
 
@@ -94,12 +97,17 @@ pip install .
 ```bash
 skylos /path/to/your/project
 
+skylos /path/to/your/project --secrets  ## include api key scan
+
 # To launch the front end
 skylos run
 
 # Interactive mode - select items to remove
 skylos --interactive /path/to/your/project 
 
+# Comment out items
+skylos . --interactive --comment-out
+
 # Dry run - see what would be removed
 skylos --interactive --dry-run /path/to/your/project 
 
@@ -193,7 +201,7 @@ When Skylos detects a test file, it by default, will apply a confidence penalty
 
 ## Ignoring Pragmas
 
-To ignore any warning, indicate `# pragma: no skylos` **ON THE SAME LINE** as the function/class you want to ignore
+1. To ignore any warning, indicate `# pragma: no skylos` **ON THE SAME LINE** as the function/class you want to ignore
 
 Example
 
@@ -203,6 +211,14 @@ Example
         return new_path
 ```
 
+2. To suppress a **secret** on a line, add: `# skylos: ignore[SKY-S101]`
+ 
+Example
+ ```python
+
+API_KEY = "ghp_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"  # skylos: ignore[SKY-S101]
+```
+
 ## Including & Excluding Files
 
 ### Default Exclusions
@@ -244,6 +260,7 @@ Options:
   --no-default-excludes        Don't exclude default folders (__pycache__, .git, venv, etc.)
   --list-default-excludes      List the default excluded folders and
   -c, --confidence LEVEL       Confidence threshold (0-100). Lower values will show more items.
+  -- secrets                   Scan for api keys/secrets
 ```
 
 ## Interactive Mode
@@ -254,6 +271,206 @@ The interactive mode lets you select specific functions and imports to remove:
 2. **Confirm changes**: Review selected items before applying
 3. **Auto-cleanup**: Files are automatically updated
 
+## CI/CD (Pre-commit & GitHub Actions)
+
+Pick **one** (or use **both**) 
+
+1. Pre-commit (local + CI): runs Skylos before commits/PRs.
+   - You must install pre-commit locally once. Skylos gets installed automatically by the hook.
+
+2. GitHub Actions: runs Skylos on pushes/PRs in CI.
+   - No local install needed
+
+### Option A — Pre-commit (local + CI)
+
+1. Create or edit `.pre-commit-config.yaml` at the repo root:
+
+**A: Skylos hook repo**
+```yaml
+## .pre-commit-config.yaml
+repos:
+  - repo: https://github.com/duriantaco/skylos
+    rev: v2.2.2
+    hooks:
+      - id: skylos-scan
+        name: skylos report
+        entry: python -m skylos.cli
+        language: python
+        types_or: [python]
+        pass_filenames: false
+        require_serial: true
+        args: [".", "--output", "report.json", "--confidence", "70"]
+
+  - repo: local
+    hooks:
+      - id: skylos-fail-on-findings
+        name: skylos
+        env:
+          SKYLOS_SOFT: "1"
+        language: python
+        language_version: python3
+        pass_filenames: false
+        require_serial: true
+        entry: >
+          python -c "import os, json, sys, pathlib;
+          p=pathlib.Path('report.json');
+
+          if not p.exists(): 
+            sys.exit(0);
+
+          data=json.loads(p.read_text(encoding='utf-8'));
+
+          count = 0
+          for v in data.values():
+            if isinstance(v, list):
+              count += len(v)
+
+          print(f'[skylos] findings: {count}');
+          sys.exit(0 if os.getenv('SKYLOS_SOFT') or count==0 else 1)"
+```
+**B: self-contained local hook**
+
+```yaml
+repos:
+  - repo: local
+    hooks:
+      - id: skylos-scan
+        name: skylos report
+        language: python
+        entry: python -m skylos.cli
+        pass_filenames: false
+        require_serial: true
+        additional_dependencies: [skylos==2.2.2]
+        args: [".", "--output", "report.json", "--confidence", "70"]
+
+      - id: skylos-fail-on-findings
+        name: skylos (soft)
+        language: python
+        language_version: python3
+        pass_filenames: false
+        require_serial: true
+        entry: >
+          python -c "import os, json, sys, pathlib;
+          p=pathlib.Path('report.json');
+
+          if not p.exists(): 
+            sys.exit(0);
+
+          data=json.loads(p.read_text(encoding='utf-8'));
+
+          count = 0
+          for v in data.values():
+            if isinstance(v, list):
+              count += len(v)
+
+          print(f'[skylos] findings: {count}');
+          sys.exit(0 if os.getenv('SKYLOS_SOFT') or count==0 else 1)"
+```
+
+**Install requirements:**
+
+You must install pre-commit locally once:
+```bash
+pip install pre-commit
+pre-commit install
+```
+
+2. pre-commit run --all-files
+
+
+3. Run the same hooks in CI (GitHub Actions): create .github/workflows/pre-commit.yml:
+
+```yaml
+name: pre-commit
+on: [push, pull_request]
+jobs:
+  run:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with: { python-version: "3.11", cache: "pip" }
+      - uses: pre-commit/action@v3.0.1
+        with: { extra_args: --all-files }
+```
+
+**Pre commit behavior:** the second hook is soft by default (SKYLOS_SOFT=1). This means that it prints findings and passes. You can remove the env/logic if you want pre-commit to block commits on finding
+
+### Option B — Github Actions
+
+1. Create .github/workflows/skylos.yml:
+
+```yaml
+name: Skylos Deadcode Scan
+
+on:
+  pull_request:
+  push:
+    branches: [ main, master ]
+  workflow_dispatch:
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    env:
+      SKYLOS_STRICT: ${{ vars.SKYLOS_STRICT || 'false' }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+          cache: 'pip'
+
+      - name: Install Skylos
+        run: pip install skylos
+
+      - name: Run Skylos
+        env:
+          REPORT: skylos_${{ github.run_number }}_${{ github.sha }}.json
+        run: |
+          echo "REPORT=$REPORT" >> "$GITHUB_OUTPUT"
+          skylos . --json > "$REPORT"
+        id: scan
+
+      - name: Fail if there are findings
+        continue-on-error: ${{ env.SKYLOS_STRICT != 'true' }}
+        env:
+          REPORT: ${{ steps.scan.outputs.REPORT }}
+        run: |
+            python - << 'PY'
+            import json, sys, os
+            report = os.environ["REPORT"]
+            data = json.load(open(report, "r", encoding="utf-8"))
+            count = 0
+            for value in data.values():
+                if isinstance(value, list):
+                    count += len(value)
+            print(f"Findings: {count}")
+            if count > 0:
+              print(f"::warning title=Skylos findings::{count} potential issues found. See {report}")
+            sys.exit(1 if count > 0 else 0)
+            PY
+
+      - name: Upload report artifact
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ steps.scan.outputs.REPORT }}
+          path: ${{ steps.scan.outputs.REPORT }}
+
+      - name: Summarize in job log
+        if: always()
+        run: |
+          echo "Skylos report: ${{ steps.scan.outputs.REPORT }}" >> $GITHUB_STEP_SUMMARY
+```
+
+**To make the job fail on findings (strict mode)**:
+
+1. Go to GitHub -> Settings -> Secrets and variables -> Actions -> Variables
+
+2. Add variable SKYLOS_STRICT with value true
+
 ## Development
 
 ### Prerequisites
@@ -307,6 +524,7 @@ A: Start with 60 (default) for safe cleanup. Use 30 for framework applications.
 - **Frameworks**: Django models, Flask, FastAPI routes may appear unused but aren't
 - **Test data**: Limited scenarios, your mileage may vary
 - **False positives**: Always manually review before deleting code
+- **Secrets PoC**: May emit both a provider hit and a generic high-entropy hit for the same token. All tokens are detected only in py files (`.py`, `.pyi`, `.pyw`).
 
 ## Troubleshooting
 
@@ -339,9 +557,10 @@ We welcome contributions! Please read our [Contributing Guidelines](CONTRIBUTING
 ## Roadmap
 - [x] Expand our test cases
 - [ ] Configuration file support 
-- [ ] Git hooks integration
-- [ ] CI/CD integration examples
+- [x] Git hooks integration
+- [x] CI/CD integration examples
 - [ ] Further optimization
+- [ ] Add new rules
 
 ## License
 

{skylos-2.1.1 → skylos-2.2.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "skylos"
-version = "2.1.1"
+version = "2.2.2"
 requires-python = ">=3.9"
 description = "A static analysis tool for Python codebases"
 authors = [{name = "oha", email = "aaronoh2015@gmail.com"}]

{skylos-2.1.1 → skylos-2.2.2}/setup.py

@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
 
 setup(
     name="skylos",
-    version="2.1.1",
+    version="2.2.2",
     packages=find_packages(),
     python_requires=">=3.9",
     install_requires=[

skylos-2.2.2/skylos/__init__.py

@@ -0,0 +1,10 @@
+__version__ = "2.2.2"
+
+def analyze(*args, **kwargs):
+    from .analyzer import analyze as _analyze
+    return _analyze(*args, **kwargs)
+
+def debug_test():
+    return "debug-ok"
+
+__all__ = ["analyze", "debug_test", "__version__"]

{skylos-2.1.1 → skylos-2.2.2}/skylos/analyzer.py

@@ -7,10 +7,11 @@ from pathlib import Path
 from collections import defaultdict
 from skylos.visitor import Visitor
 from skylos.constants import ( PENALTIES, AUTO_CALLED )
-from skylos.test_aware import TestAwareVisitor
+from skylos.visitors.test_aware import TestAwareVisitor
+from skylos.rules.secrets import scan_ctx as _secrets_scan_ctx
 import os
 import traceback
-from skylos.framework_aware import FrameworkAwareVisitor, detect_framework_usage
+from skylos.visitors.framework_aware import FrameworkAwareVisitor, detect_framework_usage
 
 logging.basicConfig(level=logging.INFO,format='%(asctime)s - %(levelname)s - %(message)s')
 logger=logging.getLogger('Skylos')
@@ -155,18 +156,40 @@ class Skylos:
     
     def _apply_penalties(self, def_obj, visitor, framework):
         confidence=100
+
+        if getattr(visitor, "ignore_lines", None) and def_obj.line in visitor.ignore_lines:
+            def_obj.confidence = 0
+            return
+
+        if def_obj.type == "variable" and def_obj.simple_name == "_":
+            def_obj.confidence = 0
+            return
+
         if def_obj.simple_name.startswith("_") and not def_obj.simple_name.startswith("__"):
             confidence -= PENALTIES["private_name"] 
+
         if def_obj.simple_name.startswith("__") and def_obj.simple_name.endswith("__"):
             confidence -= PENALTIES["dunder_or_magic"]
-        if def_obj.type == "variable" and def_obj.simple_name.isupper():
-            confidence = 0
+
         if def_obj.in_init and def_obj.type in ("function", "class"):
             confidence -= PENALTIES["in_init_file"]
+            
         if def_obj.name.split(".")[0] in self.dynamic:
             confidence -= PENALTIES["dynamic_module"]
+
         if visitor.is_test_file or def_obj.line in visitor.test_decorated_lines:
             confidence -= PENALTIES["test_related"]
+
+        if def_obj.type == "variable" and getattr(framework, "dataclass_fields", None):
+            if def_obj.name in framework.dataclass_fields:
+                def_obj.confidence = 0
+                return
+            
+        if def_obj.type == "variable":
+            fr = getattr(framework, "first_read_lineno", {}).get(def_obj.name)
+            if fr is not None and fr >= def_obj.line:
+                def_obj.confidence = 0
+                return
         
         framework_confidence = detect_framework_usage(def_obj, visitor=framework)
         if framework_confidence is not None:
@@ -215,7 +238,7 @@ class Skylos:
                     if method.simple_name == "format" and cls.endswith("Formatter"):
                         method.references += 1
 
-    def analyze(self, path, thr=60, exclude_folders=None):
+    def analyze(self, path, thr=60, exclude_folders= None, enable_secrets = False):
         files, root = self._get_python_files(path, exclude_folders)
         
         if not files:
@@ -238,6 +261,7 @@ class Skylos:
         for f in files:
             modmap[f] = self._module(root, f)
         
+        all_secrets = []
         for file in files:
             mod = modmap[file]
             defs, refs, dyn, exports, test_flags, framework_flags = proc_file(file, mod)
@@ -250,6 +274,16 @@ class Skylos:
             self.dynamic.update(dyn)
             self.exports[mod].update(exports)
 
+            if enable_secrets and _secrets_scan_ctx is not None:
+                try:
+                    src_lines = Path(file).read_text(encoding="utf-8", errors="ignore").splitlines(True)
+                    ctx = {"relpath": str(file), "lines": src_lines, "tree": None}
+                    findings = list(_secrets_scan_ctx(ctx))
+                    if findings:
+                        all_secrets.extend(findings)
+                except Exception:
+                    pass
+
         self._mark_refs()
         self._apply_heuristics() 
         self._mark_exports()
@@ -262,7 +296,7 @@ class Skylos:
         for d in sorted(self.defs.values(), key=def_sort_key):
             if shown >= 50:
                 break
-            print(f" {d.type:<8} refs={d.references:<2} conf={d.confidence:<3} exported={d.is_exported} line={d.line:<4} {d.name}")
+            print(f" type={d.type} refs={d.references} conf={d.confidence} exported={d.is_exported} line={d.line} name={d.name}")
             shown += 1
 
         unused = []
@@ -281,6 +315,9 @@ class Skylos:
                 "excluded_folders": exclude_folders or [],
             }
         }
+
+        if enable_secrets and all_secrets:
+            result["secrets"] = all_secrets
         
         for u in unused:
             if u["type"] in ("function", "method"):
@@ -304,10 +341,13 @@ def proc_file(file_or_args, mod=None):
 
     try:
         source = Path(file).read_text(encoding="utf-8")
-        tree   = ast.parse(source)
+        ignore_lines = {i for i, line in enumerate(source.splitlines(), start=1)
+                        if "pragma: no skylos" in line}
+        tree = ast.parse(source)
 
         tv = TestAwareVisitor(filename=file)
         tv.visit(tree)
+        tv.ignore_lines = ignore_lines
 
         fv = FrameworkAwareVisitor(filename=file)
         fv.visit(tree)
@@ -315,18 +355,23 @@ def proc_file(file_or_args, mod=None):
         v  = Visitor(mod, file)
         v.visit(tree)
 
+        fv.dataclass_fields = getattr(v, "dataclass_fields", set())
+        fv.first_read_lineno = getattr(v, "first_read_lineno", {})
+
         return v.defs, v.refs, v.dyn, v.exports, tv, fv
+    
     except Exception as e:
         logger.error(f"{file}: {e}")
         if os.getenv("SKYLOS_DEBUG"):
             logger.error(traceback.format_exc())
         dummy_visitor = TestAwareVisitor(filename=file)
+        dummy_visitor.ignore_lines = set()
         dummy_framework_visitor = FrameworkAwareVisitor(filename=file)
 
         return [], [], set(), set(), dummy_visitor, dummy_framework_visitor
 
-def analyze(path,conf=60, exclude_folders=None):
-    return Skylos().analyze(path,conf, exclude_folders)
+def analyze(path, conf=60, exclude_folders=None, enable_secrets=False):
+    return Skylos().analyze(path,conf, exclude_folders, enable_secrets)
 
 if __name__ == "__main__":
     if len(sys.argv)>1: