gitghost-scanner 1.2.0__tar.gz

gitghost_scanner-1.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 CyberM
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gitghost_scanner-1.2.0/PKG-INFO

@@ -0,0 +1,99 @@
+Metadata-Version: 2.2
+Name: gitghost-scanner
+Version: 1.2.0
+Summary: Find secrets in a GitHub account's public repos — including the ones still hiding in git history after being 'deleted'.
+Author: cy3erm
+License: MIT
+Project-URL: Homepage, https://github.com/cy3erm/gitghost
+Project-URL: Repository, https://github.com/cy3erm/gitghost
+Project-URL: Issues, https://github.com/cy3erm/gitghost/issues
+Keywords: security,secrets-detection,git,osint,devsecops,bug-bounty
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Security
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# gitghost
+![gitghost exposure dossier](preview.png)
+Finds secrets in a GitHub account's public repos — including the ones that got committed, then "deleted," but are still sitting in the git history where anyone can read them.
+
+I built this after noticing how often the real leak isn't in someone's current code — it's in a commit from eight months ago that they thought they'd cleaned up. You paste an API key, catch it, delete the line, and move on. The latest version looks fine. But the old commit still has the key, and `git log` hands it to anyone who clones the repo. Most scanners only look at your current files and miss this entirely. gitghost goes digging through history for exactly those, and then rolls everything up into a single exposure score so you can actually tell how bad things are at a glance.
+
+```
+$ gitghost cy3erm
+
+EXPOSURE SCORE: 96/100  [CRITICAL]  grade F
+  · 7 live secrets in current code
+  · 3 "deleted" secrets still recoverable from history
+  · author email exposed in commit metadata
+  · timezone inferable from commit times (UTC+0530)
+
+dossier written to gitghost-dossier.html
+```
+
+You get an HTML report you can open in a browser, not just a wall of terminal text. Every finding links straight to the spot — the exact file and line for live secrets, or the commit for the ones recovered from history — so you're not hunting for where the key actually is. And the report ends with a short how-to-fix guide, because the instinct when you see a leaked key (delete the file, nuke the repo) doesn't actually fix anything: the key's already been cloned. The real fix is to rotate it, then purge it from history, and the report walks you through both.
+
+## Running it
+
+You'll need Python 3.10+ and git. There are no runtime dependencies — it's all standard library.
+
+Install it as a command:
+
+```bash
+pipx install git+https://github.com/cy3erm/gitghost
+gitghost <username>
+```
+
+Or just clone and run it without installing:
+
+```bash
+git clone https://github.com/cy3erm/gitghost
+cd gitghost
+python3 -m gitghost <username>
+```
+
+If you'd rather not point it at a real person first, there's a demo. It builds a little repo with fake credentials — including one that gets "deleted" a few commits in — so you can watch the history recovery dig it back out:
+
+```bash
+bash demo/make_demo_repo.sh /tmp/demo
+python3 -m gitghost --local /tmp/demo --name demo
+```
+
+A few other flags:
+
+```bash
+python3 -m gitghost <username> --limit 50        # cap how many repos
+python3 -m gitghost <username> --out report.html # where to write the report
+python3 -m gitghost --local ./some-repo          # scan a checkout you have locally
+python3 -m gitghost --repo owner/name            # scan just one repo (URL or owner/name)
+```
+
+Scanning more than a couple of accounts? Set `GITHUB_TOKEN` (any token works, it doesn't need any scopes) so you don't run into GitHub's 60-requests-an-hour limit for anonymous calls.
+
+## What it looks for
+
+- Cloud and service keys — AWS, GCP, Stripe, GitHub, OpenAI, Slack, private keys, database URLs
+- The same keys pulled back out of history after they were removed from the current code
+- Metadata you might not realize you're sharing — the email in your commits, and a rough guess at your timezone and working hours from commit timestamps
+- Committed `.env` files, internal hostnames, hardcoded private IPs
+
+The score is deliberately one number so you can watch it move — run it, clean things up, run it again. A single live cloud key is enough to put an account in the red by itself; a pile of smaller stuff pushes it up from there.
+
+## Where it draws the line
+
+It only ever reads public repositories — things the account already chose to publish — and it's detection-only. It'll tell you a string *looks like* a credential and leave it at that. It won't try the key against the actual service to see if it still works, because quietly logging into someone else's account isn't the tool's job, and honestly it's not yours either. Findings in the report are shown as fingerprints, not the raw values, so you can share a report without leaking anything.
+
+Point it at yourself first. Most people turn up at least one thing they'd completely forgotten about — I did. (The first time I pushed this repo, GitHub's own secret scanner blocked me because the demo's fake keys looked real enough to trip it. Which is about the best proof of the premise I could ask for.)
+
+## Adding your own detections
+
+The patterns live in `gitghost/rules.py`. Each one is just a name, a regex, a severity, and a line of advice for fixing it — easy to add. If you write a good one, send a PR.
+
+## License
+
+MIT

gitghost_scanner-1.2.0/README.md

@@ -0,0 +1,79 @@
+# gitghost
+![gitghost exposure dossier](preview.png)
+Finds secrets in a GitHub account's public repos — including the ones that got committed, then "deleted," but are still sitting in the git history where anyone can read them.
+
+I built this after noticing how often the real leak isn't in someone's current code — it's in a commit from eight months ago that they thought they'd cleaned up. You paste an API key, catch it, delete the line, and move on. The latest version looks fine. But the old commit still has the key, and `git log` hands it to anyone who clones the repo. Most scanners only look at your current files and miss this entirely. gitghost goes digging through history for exactly those, and then rolls everything up into a single exposure score so you can actually tell how bad things are at a glance.
+
+```
+$ gitghost cy3erm
+
+EXPOSURE SCORE: 96/100  [CRITICAL]  grade F
+  · 7 live secrets in current code
+  · 3 "deleted" secrets still recoverable from history
+  · author email exposed in commit metadata
+  · timezone inferable from commit times (UTC+0530)
+
+dossier written to gitghost-dossier.html
+```
+
+You get an HTML report you can open in a browser, not just a wall of terminal text. Every finding links straight to the spot — the exact file and line for live secrets, or the commit for the ones recovered from history — so you're not hunting for where the key actually is. And the report ends with a short how-to-fix guide, because the instinct when you see a leaked key (delete the file, nuke the repo) doesn't actually fix anything: the key's already been cloned. The real fix is to rotate it, then purge it from history, and the report walks you through both.
+
+## Running it
+
+You'll need Python 3.10+ and git. There are no runtime dependencies — it's all standard library.
+
+Install it as a command:
+
+```bash
+pipx install git+https://github.com/cy3erm/gitghost
+gitghost <username>
+```
+
+Or just clone and run it without installing:
+
+```bash
+git clone https://github.com/cy3erm/gitghost
+cd gitghost
+python3 -m gitghost <username>
+```
+
+If you'd rather not point it at a real person first, there's a demo. It builds a little repo with fake credentials — including one that gets "deleted" a few commits in — so you can watch the history recovery dig it back out:
+
+```bash
+bash demo/make_demo_repo.sh /tmp/demo
+python3 -m gitghost --local /tmp/demo --name demo
+```
+
+A few other flags:
+
+```bash
+python3 -m gitghost <username> --limit 50        # cap how many repos
+python3 -m gitghost <username> --out report.html # where to write the report
+python3 -m gitghost --local ./some-repo          # scan a checkout you have locally
+python3 -m gitghost --repo owner/name            # scan just one repo (URL or owner/name)
+```
+
+Scanning more than a couple of accounts? Set `GITHUB_TOKEN` (any token works, it doesn't need any scopes) so you don't run into GitHub's 60-requests-an-hour limit for anonymous calls.
+
+## What it looks for
+
+- Cloud and service keys — AWS, GCP, Stripe, GitHub, OpenAI, Slack, private keys, database URLs
+- The same keys pulled back out of history after they were removed from the current code
+- Metadata you might not realize you're sharing — the email in your commits, and a rough guess at your timezone and working hours from commit timestamps
+- Committed `.env` files, internal hostnames, hardcoded private IPs
+
+The score is deliberately one number so you can watch it move — run it, clean things up, run it again. A single live cloud key is enough to put an account in the red by itself; a pile of smaller stuff pushes it up from there.
+
+## Where it draws the line
+
+It only ever reads public repositories — things the account already chose to publish — and it's detection-only. It'll tell you a string *looks like* a credential and leave it at that. It won't try the key against the actual service to see if it still works, because quietly logging into someone else's account isn't the tool's job, and honestly it's not yours either. Findings in the report are shown as fingerprints, not the raw values, so you can share a report without leaking anything.
+
+Point it at yourself first. Most people turn up at least one thing they'd completely forgotten about — I did. (The first time I pushed this repo, GitHub's own secret scanner blocked me because the demo's fake keys looked real enough to trip it. Which is about the best proof of the premise I could ask for.)
+
+## Adding your own detections
+
+The patterns live in `gitghost/rules.py`. Each one is just a name, a regex, a severity, and a line of advice for fixing it — easy to add. If you write a good one, send a PR.
+
+## License
+
+MIT

gitghost_scanner-1.2.0/gitghost/__init__.py

@@ -0,0 +1 @@
+__version__ = "1.2.0"

gitghost_scanner-1.2.0/gitghost/__main__.py

@@ -0,0 +1,2 @@
+from .cli import main
+main()

gitghost_scanner-1.2.0/gitghost/banner.py

@@ -0,0 +1,38 @@
+import sys
+
+from . import __version__
+
+_ART = r"""
+   ____ _(_) /_____ _/ /_  ____  _____/ /_
+  / __ `/ / __/ __ `/ __ \/ __ \/ ___/ __/
+ / /_/ / / /_/ /_/ / / / / /_/ (__  ) /_
+ \__, /_/\__/\__, /_/ /_/\____/____/\__/
+/____/      /____/
+"""
+
+_GHOST = r"""      .-.
+     (o o)
+     | O |
+      \_/
+"""
+
+
+def banner():
+    if not sys.stdout.isatty():
+        return ""
+    orange, dim, ghost_c, reset = "\033[38;5;208m", "\033[2m", "\033[38;5;250m", "\033[0m"
+    art = _ART.strip("\n").splitlines()
+    ghost = _GHOST.strip("\n").splitlines()
+    width = max(len(l) for l in art) + 4
+    out = []
+    for i, line in enumerate(art):
+        g = ghost[i] if i < len(ghost) else ""
+        out.append(orange + line.ljust(width) + reset + ghost_c + g + reset)
+    tag = f"{dim}  the secrets you deleted are still in git history{reset}   {orange}v{__version__}{reset}"
+    return "\n".join(out) + "\n" + tag + "\n"
+
+
+def print_banner():
+    b = banner()
+    if b:
+        print(b)

gitghost_scanner-1.2.0/gitghost/cli.py

@@ -0,0 +1,123 @@
+import argparse
+import sys
+import tempfile
+
+from . import github
+from .ghost import recover_ghosts
+from .metadata import analyze_metadata
+from .report import render_report
+from .rules import Finding
+from .scanner import scan_repo
+from .score import compute_score
+
+
+def _scan_one(root: str, name: str) -> tuple[list[Finding], object]:
+    findings = list(scan_repo(root, name).findings)
+    findings += recover_ghosts(root, name)
+    meta = analyze_metadata(root)
+    return findings, meta
+
+
+def run_local(path: str, name: str, out: str) -> None:
+    print(f"[*] scanning local repo: {name}")
+    findings, meta = _scan_one(path, name)
+    card = compute_score(findings, meta)
+    _emit(name, card, findings, meta, 1, out)
+
+
+def run_repo(url: str, out: str) -> None:
+    try:
+        repo = github.repo_from_url(url)
+    except ValueError as e:
+        sys.exit(f"[!] {e}")
+    print(f"[*] scanning single repo: {repo.full_name}")
+    with tempfile.TemporaryDirectory() as tmp:
+        dest = github.clone(repo, tmp)
+        if not dest:
+            sys.exit(f"[!] could not clone {repo.clone_url} (private, renamed, or network issue)")
+        findings, meta = _scan_one(dest, repo.name)
+        for finding in findings:
+            finding.repo_url = repo.html_url
+        card = compute_score(findings, meta)
+        _emit(repo.full_name, card, findings, meta, 1, out)
+
+
+def run_identity(identity: str, limit: int, out: str) -> None:
+    print(f"[*] enumerating public repos for @{identity} ...")
+    try:
+        repos = github.list_public_repos(identity, limit=limit)
+    except Exception as e:
+        sys.exit(f"[!] could not reach GitHub API: {e}\n    (set GITHUB_TOKEN to raise rate limits)")
+    if not repos:
+        sys.exit(f"[!] no public repos found for @{identity}")
+    print(f"[*] {len(repos)} repos. cloning + scanning (history included for ghost recovery)...")
+
+    all_findings: list[Finding] = []
+    merged_meta = None
+    with tempfile.TemporaryDirectory() as tmp:
+        for r in repos:
+            dest = github.clone(r, tmp)
+            if not dest:
+                print(f"    - skip {r.name} (clone failed)")
+                continue
+            f, m = _scan_one(dest, r.name)
+            for finding in f:
+                finding.repo_url = r.html_url
+            all_findings += f
+            merged_meta = _merge_meta(merged_meta, m)
+            tag = f"{len([x for x in f if x.kind=='secret' and not x.is_ghost])} live / {len([x for x in f if x.is_ghost])} ghost"
+            print(f"    - {r.name:<32} {tag}")
+
+    card = compute_score(all_findings, merged_meta)
+    _emit(identity, card, all_findings, merged_meta, len(repos), out)
+
+
+def _merge_meta(a, b):
+    if a is None:
+        return b
+    for e in b.emails:
+        if e not in a.emails:
+            a.emails.append(e)
+    a.commit_count += b.commit_count
+    a.dominant_utc_offset = a.dominant_utc_offset or b.dominant_utc_offset
+    a.likely_active_hours = a.likely_active_hours or b.likely_active_hours
+    return a
+
+
+def _emit(identity, card, findings, meta, repos_scanned, out):
+    html = render_report(identity, card, findings, meta, repos_scanned)
+    with open(out, "w", encoding="utf-8") as f:
+        f.write(html)
+    print(f"\n[=] EXPOSURE SCORE: {card.score}/100  [{card.band}]  grade {card.grade}")
+    for d in card.drivers:
+        print(f"    · {d}")
+    print(f"[=] dossier written to: {out}")
+
+
+def main() -> None:
+    p = argparse.ArgumentParser(prog="gitghost", description="GitHub exposure dossier (detection-only).")
+    from . import __version__
+    p.add_argument("--version", action="version", version=f"gitghost {__version__}")
+    p.add_argument("identity", nargs="?", help="GitHub username or org")
+    p.add_argument("--repo", help="scan a single repo by URL or owner/name")
+    p.add_argument("--local", help="scan a repo already on disk instead of GitHub")
+    p.add_argument("--name", default="local-repo", help="label for --local scans")
+    p.add_argument("--limit", type=int, default=30, help="max repos to scan")
+    p.add_argument("--out", default="gitghost-dossier.html", help="output HTML path")
+    args = p.parse_args()
+
+    from .banner import print_banner
+    print_banner()
+
+    if args.local:
+        run_local(args.local, args.name, args.out)
+    elif args.repo:
+        run_repo(args.repo, args.out)
+    elif args.identity:
+        run_identity(args.identity, args.limit, args.out)
+    else:
+        p.error("provide a GitHub username, --repo <url>, or --local <path>")
+
+
+if __name__ == "__main__":
+    main()

gitghost_scanner-1.2.0/gitghost/ghost.py

@@ -0,0 +1,81 @@
+import re
+import subprocess
+
+
+from .rules import Finding, scan_text
+
+
+def _git(root: str, *args: str) -> str:
+    return subprocess.run(
+        ["git", "-C", root, *args],
+        capture_output=True, text=True, errors="ignore",
+    ).stdout
+
+
+def _head_blobs(root: str) -> set[str]:
+    out = _git(root, "ls-tree", "-r", "HEAD")
+    blobs = set()
+    for line in out.splitlines():
+        parts = line.split()
+        if len(parts) >= 3 and parts[1] == "blob":
+            blobs.add(parts[2])
+    return blobs
+
+
+def _all_historical_blobs(root: str) -> list[tuple[str, str]]:
+    seen: dict[str, str] = {}
+    for line in _git(root, "rev-list", "--all", "--objects").splitlines():
+        parts = line.split(maxsplit=1)
+        if len(parts) == 2 and len(parts[0]) == 40:
+            seen.setdefault(parts[0], parts[1])
+
+    for line in _git(root, "fsck", "--unreachable", "--no-reflogs").splitlines():
+        parts = line.split()
+        if len(parts) == 3 and parts[1] == "blob":
+            seen.setdefault(parts[2], "")
+    return list(seen.items())
+
+
+_VENDOR = re.compile(r"(^|/)(node_modules|vendor|dist|build|\.next|bower_components|"
+                     r"third_party|site-packages|\.venv|venv)/|"
+                     r"(package-lock\.json|yarn\.lock|pnpm-lock\.yaml|"
+                     r"\.min\.js|\.min\.css|\.map)$", re.I)
+
+
+def _introducing_commit(root: str, blob: str) -> tuple[str, str]:
+    out = _git(root, "log", "--all", "--format=%H|%ci", "--find-object", blob, "--reverse")
+    for line in out.splitlines():
+        if "|" in line:
+            h, date = line.split("|", 1)
+            return h[:10], date.strip()[:10]
+    return "dangling", "unknown"
+
+
+def recover_ghosts(root: str, repo_name: str, max_blobs: int = 4000) -> list[Finding]:
+    head = _head_blobs(root)
+    ghosts: list[Finding] = []
+    seen_fp: set[tuple] = set()
+    checked = 0
+    for blob, path in _all_historical_blobs(root):
+        if blob in head:
+            continue
+        if path and _VENDOR.search(path):
+            continue
+        if checked >= max_blobs:
+            break
+        checked += 1
+        content = _git(root, "cat-file", "-p", blob)
+        if not content:
+            continue
+        for f in scan_text(content):
+            key = (f.rule_id, f.redacted)
+            if key in seen_fp:
+                continue
+            seen_fp.add(key)
+            commit, date = _introducing_commit(root, blob)
+            f.repo = repo_name
+            f.is_ghost = True
+            f.commit = commit
+            f.path = f"(history) entered {date}"
+            ghosts.append(f)
+    return ghosts

gitghost_scanner-1.2.0/gitghost/github.py

@@ -0,0 +1,80 @@
+import json
+import os
+import subprocess
+
+import urllib.request
+from dataclasses import dataclass
+
+
+API = "https://api.github.com"
+
+
+@dataclass
+class Repo:
+    name: str
+    full_name: str
+    clone_url: str
+    pushed_at: str
+    html_url: str = ""
+
+
+def _get(url: str) -> list | dict:
+    req = urllib.request.Request(url, headers={
+        "Accept": "application/vnd.github+json",
+        "User-Agent": "gitghost",
+    })
+    token = os.environ.get("GITHUB_TOKEN")
+    if token:
+        req.add_header("Authorization", f"Bearer {token}")
+    with urllib.request.urlopen(req, timeout=30) as r:
+        return json.loads(r.read().decode())
+
+
+def list_public_repos(identity: str, limit: int = 30) -> list[Repo]:
+    repos: list[Repo] = []
+    page = 1
+    while len(repos) < limit:
+
+        data = _get(f"{API}/users/{identity}/repos?per_page=100&page={page}&sort=pushed")
+        if not isinstance(data, list) or not data:
+            break
+        for d in data:
+            if d.get("fork"):
+                continue
+            repos.append(Repo(
+                name=d["name"], full_name=d["full_name"],
+                clone_url=d["clone_url"], pushed_at=d.get("pushed_at", ""),
+                html_url=d.get("html_url", ""),
+            ))
+            if len(repos) >= limit:
+                break
+        page += 1
+    return repos
+
+
+def repo_from_url(url: str) -> Repo:
+    u = url.strip().rstrip("/")
+    if u.endswith(".git"):
+        u = u[:-4]
+
+    for prefix in ("https://github.com/", "http://github.com/", "git@github.com:", "github.com/"):
+        if u.startswith(prefix):
+            u = u[len(prefix):]
+            break
+    parts = u.split("/")
+    if len(parts) < 2:
+        raise ValueError(f"not a valid repo reference: {url!r} (expected owner/name or a GitHub URL)")
+    owner, name = parts[0], parts[1]
+    full = f"{owner}/{name}"
+    return Repo(name=name, full_name=full,
+                clone_url=f"https://github.com/{full}.git",
+                pushed_at="", html_url=f"https://github.com/{full}")
+
+
+def clone(repo: Repo, dest_parent: str) -> str | None:
+    dest = os.path.join(dest_parent, repo.name)
+    r = subprocess.run(
+        ["git", "clone", "--quiet", repo.clone_url, dest],
+        capture_output=True, text=True,
+    )
+    return dest if r.returncode == 0 else None

gitghost_scanner-1.2.0/gitghost/metadata.py

@@ -0,0 +1,52 @@
+import re
+import subprocess
+from collections import Counter
+from dataclasses import dataclass, field
+
+
+@dataclass
+class MetadataReport:
+    emails: list[str] = field(default_factory=list)
+    noreply_emails: list[str] = field(default_factory=list)
+    dominant_utc_offset: str | None = None
+    likely_active_hours: str | None = None
+    commit_count: int = 0
+
+
+_NOREPLY = re.compile(r"noreply|users\.noreply\.github\.com", re.I)
+
+
+def analyze_metadata(root: str) -> MetadataReport:
+    out = subprocess.run(
+        ["git", "-C", root, "log", "--all", "--format=%ae|%ai"],
+        capture_output=True, text=True, errors="ignore",
+    ).stdout
+
+    emails: Counter[str] = Counter()
+    offsets: Counter[str] = Counter()
+    hours: Counter[int] = Counter()
+    count = 0
+
+    for line in out.splitlines():
+        if "|" not in line:
+            continue
+        email, ts = line.split("|", 1)
+        count += 1
+        emails[email.strip()] += 1
+
+        m = re.search(r"(\d{2}):\d{2}:\d{2}\s([+-]\d{4})", ts)
+        if m:
+            hours[int(m.group(1))] += 1
+            offsets[m.group(2)] += 1
+
+    report = MetadataReport(commit_count=count)
+    for email, _ in emails.most_common():
+        (report.noreply_emails if _NOREPLY.search(email) else report.emails).append(email)
+
+    if offsets:
+        report.dominant_utc_offset = offsets.most_common(1)[0][0]
+    if hours:
+        top = [h for h, _ in hours.most_common(6)]
+        lo, hi = min(top), max(top)
+        report.likely_active_hours = f"{lo:02d}:00–{hi:02d}:00 (local)"
+    return report