jmo-security 0.3.0__tar.gz → 0.3.2__tar.gz

{jmo_security-0.3.0 → jmo_security-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: jmo-security
-Version: 0.3.0
+Version: 0.3.2
 Summary: JMo Security Audit Suite (terminal-first, multi-tool, unified outputs)
 Author-email: James Moceri <general@jmogaming.com>
 License: MIT
@@ -13,6 +13,7 @@ License-File: LICENSE
 Provides-Extra: reporting
 Requires-Dist: PyYAML>=6.0; extra == "reporting"
 Requires-Dist: jsonschema>=4.0; extra == "reporting"
+Dynamic: license-file
 
 # JMo's Security Audit Tool Suite
 
@@ -20,14 +21,77 @@ Requires-Dist: jsonschema>=4.0; extra == "reporting"
    <img src="assets/jmo-logo.png" alt="JMo Security Audit Tool Suite" width="220" />
 </p>
 
+[![Tests](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml?query=branch%3Amain)
+[![Website](https://img.shields.io/website?url=https%3A%2F%2Fjmotools.com)](https://jmotools.com)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/Python-%E2%89%A53.8-3776AB?logo=python&logoColor=white)](#)
+[![Contributions welcome](https://img.shields.io/badge/Contributions-welcome-brightgreen.svg)](https://github.com/jimmy058910/jmo-security-repo/issues)
+[![Buy me an energy drink](https://img.shields.io/badge/Buy%20me%20an-energy%20drink-%23ff4d00)](https://ko-fi.com/jmogaming)
+
+<!-- CI/coverage/package badges (enable once configured)
+[![CI](https://img.shields.io/badge/CI-GitHub%20Actions-coming--soon-lightgrey)](#)
+<!-- If/when a workflow exists, switch to:
 [![Tests](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml/badge.svg)](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml)
+-->
+<!-- Codecov (enable after uploading coverage):
 [![codecov](https://codecov.io/gh/jimmy058910/jmo-security-repo/branch/main/graph/badge.svg)](https://codecov.io/gh/jimmy058910/jmo-security-repo)
+-->
+<!-- PyPI (enable after first release):
 [![PyPI - Version](https://img.shields.io/pypi/v/jmo-security)](https://pypi.org/project/jmo-security/)
+-->
 
 A terminal-first, cross-platform security audit toolkit that orchestrates multiple scanners (secrets, SAST, SBOM, IaC, Dockerfile) with a unified Python CLI, normalized outputs, and an HTML dashboard.
 
 👉 New here? Read the comprehensive User Guide: [docs/USER_GUIDE.md](docs/USER_GUIDE.md)
 Docs hub: [docs/index.md](docs/index.md)
+Project homepage: [jmotools.com](https://jmotools.com)
+
+> Origin story: This started as part of a Cybersecurity Capstone Project. It has since grown into a general-purpose toolkit. I’d love for folks with deeper expertise to jump in—issues and PRs are welcome!
+
+Thinking about contributing? See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, coding standards, and release steps.
+
+Roadmap & history:
+- Completed steps (summary): see [RELEASE_NOTES.md](RELEASE_NOTES.md) → “Roadmap Summary (Steps 1–13)”
+- Active/planned work: see [ROADMAP.md](ROADMAP.md)
+
+For scanning a list of repos from a TSV end-to-end (clone + unshallow + full toolchain), see: [docs/examples/scan_from_tsv.md](docs/examples/scan_from_tsv.md)
+
+## 🧪 New: Simple wrapper commands
+
+If you're just getting started, use the beginner-friendly wrapper:
+
+- `jmotools fast` — quick scan (fast profile)
+- `jmotools balanced` — default balanced scan
+- `jmotools full` — deep scan with more tools
+
+Examples:
+
+```bash
+# Fast profile over repos listed in a TSV (auto-clone) and open dashboard when done
+jmotools fast --tsv ai-search/candidates.tsv --dest ./repos-tsv
+
+# Full (deep) profile targeting an existing directory of repos
+jmotools full --repos-dir ./repos-tsv --allow-missing-tools
+```
+
+Under the hood, these commands verify your OS/tools, optionally clone from a TSV, run `jmo ci` with the appropriate profile, and open `dashboard.html` and `SUMMARY.md` on completion.
+
+Setup tools quickly:
+
+```bash
+# Check tools or auto-install on Linux/WSL (where supported)
+jmotools setup --check
+jmotools setup --auto-install         # or --print-commands, --force-reinstall
+```
+
+Makefile shortcuts:
+
+```bash
+make setup             # jmotools setup --check (installs package if needed)
+make fast DIR=~/repos  # jmotools fast --repos-dir ~/repos
+make balanced DIR=~/repos
+make full DIR=~/repos
+```
 
 ## 🎯 Overview
 
@@ -55,6 +119,12 @@ make verify-env      # check OS/WSL/macOS and tool availability
 make dev-deps        # install Python dev dependencies
 ```
 
+Optional: install the package locally to get `jmo` and `jmotools` commands on your PATH:
+
+```bash
+pip install -e .
+```
+
 ### Quick Start (Unified CLI)
 
 1) Verify your environment (Linux/WSL/macOS) and see install hints for optional tools:
@@ -76,7 +146,10 @@ make dev-deps
 python3 scripts/cli/jmo.py scan --repos-dir ~/repos --profile-name balanced --human-logs
 
 # Aggregate and write unified outputs to results/summaries
+# (positional or --results-dir are both accepted)
 python3 scripts/cli/jmo.py report ./results --profile --human-logs
+# or
+python3 scripts/cli/jmo.py report --results-dir ./results --profile --human-logs
 
 # Or do both in one step for CI with a failure threshold
 python3 scripts/cli/jmo.py ci --repos-dir ~/repos --profile-name fast --fail-on HIGH --profile --human-logs
@@ -128,6 +201,8 @@ cat ~/security-results-*/SUMMARY_REPORT.md
 # Linux: xdg-open ~/security-results-*/dashboard.html
 ```
 
+Looking for screenshots and how to capture them? See: [docs/screenshots/README.md](docs/screenshots/README.md)
+
 ## 📚 Documentation
 
 ### Workflow (at a glance)
@@ -559,6 +634,14 @@ The HTML dashboard provides:
 
 Contributions are welcome! Please feel free to submit pull requests or open issues for bugs and feature requests.
 
+## ❤️ Support
+
+If this toolkit saves you time, consider fueling development with an energy drink.
+
+- Prefer one-time tips? Ko‑fi: https://ko-fi.com/jmogaming
+- When you’re ready, replace the badge target with your preferred platform: GitHub Sponsors (industry standard), Open Collective, Ko-fi, or Stripe Checkout.
+- GitHub Sponsors integrates directly with your GitHub profile and repository sidebar once enabled.
+
 ## 📝 License
 
 MIT License. See LICENSE.

{jmo_security-0.3.0 → jmo_security-0.3.2}/README.md

@@ -4,14 +4,77 @@
    <img src="assets/jmo-logo.png" alt="JMo Security Audit Tool Suite" width="220" />
 </p>
 
+[![Tests](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml?query=branch%3Amain)
+[![Website](https://img.shields.io/website?url=https%3A%2F%2Fjmotools.com)](https://jmotools.com)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/Python-%E2%89%A53.8-3776AB?logo=python&logoColor=white)](#)
+[![Contributions welcome](https://img.shields.io/badge/Contributions-welcome-brightgreen.svg)](https://github.com/jimmy058910/jmo-security-repo/issues)
+[![Buy me an energy drink](https://img.shields.io/badge/Buy%20me%20an-energy%20drink-%23ff4d00)](https://ko-fi.com/jmogaming)
+
+<!-- CI/coverage/package badges (enable once configured)
+[![CI](https://img.shields.io/badge/CI-GitHub%20Actions-coming--soon-lightgrey)](#)
+<!-- If/when a workflow exists, switch to:
 [![Tests](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml/badge.svg)](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml)
+-->
+<!-- Codecov (enable after uploading coverage):
 [![codecov](https://codecov.io/gh/jimmy058910/jmo-security-repo/branch/main/graph/badge.svg)](https://codecov.io/gh/jimmy058910/jmo-security-repo)
+-->
+<!-- PyPI (enable after first release):
 [![PyPI - Version](https://img.shields.io/pypi/v/jmo-security)](https://pypi.org/project/jmo-security/)
+-->
 
 A terminal-first, cross-platform security audit toolkit that orchestrates multiple scanners (secrets, SAST, SBOM, IaC, Dockerfile) with a unified Python CLI, normalized outputs, and an HTML dashboard.
 
 👉 New here? Read the comprehensive User Guide: [docs/USER_GUIDE.md](docs/USER_GUIDE.md)
 Docs hub: [docs/index.md](docs/index.md)
+Project homepage: [jmotools.com](https://jmotools.com)
+
+> Origin story: This started as part of a Cybersecurity Capstone Project. It has since grown into a general-purpose toolkit. I’d love for folks with deeper expertise to jump in—issues and PRs are welcome!
+
+Thinking about contributing? See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, coding standards, and release steps.
+
+Roadmap & history:
+- Completed steps (summary): see [RELEASE_NOTES.md](RELEASE_NOTES.md) → “Roadmap Summary (Steps 1–13)”
+- Active/planned work: see [ROADMAP.md](ROADMAP.md)
+
+For scanning a list of repos from a TSV end-to-end (clone + unshallow + full toolchain), see: [docs/examples/scan_from_tsv.md](docs/examples/scan_from_tsv.md)
+
+## 🧪 New: Simple wrapper commands
+
+If you're just getting started, use the beginner-friendly wrapper:
+
+- `jmotools fast` — quick scan (fast profile)
+- `jmotools balanced` — default balanced scan
+- `jmotools full` — deep scan with more tools
+
+Examples:
+
+```bash
+# Fast profile over repos listed in a TSV (auto-clone) and open dashboard when done
+jmotools fast --tsv ai-search/candidates.tsv --dest ./repos-tsv
+
+# Full (deep) profile targeting an existing directory of repos
+jmotools full --repos-dir ./repos-tsv --allow-missing-tools
+```
+
+Under the hood, these commands verify your OS/tools, optionally clone from a TSV, run `jmo ci` with the appropriate profile, and open `dashboard.html` and `SUMMARY.md` on completion.
+
+Setup tools quickly:
+
+```bash
+# Check tools or auto-install on Linux/WSL (where supported)
+jmotools setup --check
+jmotools setup --auto-install         # or --print-commands, --force-reinstall
+```
+
+Makefile shortcuts:
+
+```bash
+make setup             # jmotools setup --check (installs package if needed)
+make fast DIR=~/repos  # jmotools fast --repos-dir ~/repos
+make balanced DIR=~/repos
+make full DIR=~/repos
+```
 
 ## 🎯 Overview
 
@@ -39,6 +102,12 @@ make verify-env      # check OS/WSL/macOS and tool availability
 make dev-deps        # install Python dev dependencies
 ```
 
+Optional: install the package locally to get `jmo` and `jmotools` commands on your PATH:
+
+```bash
+pip install -e .
+```
+
 ### Quick Start (Unified CLI)
 
 1) Verify your environment (Linux/WSL/macOS) and see install hints for optional tools:
@@ -60,7 +129,10 @@ make dev-deps
 python3 scripts/cli/jmo.py scan --repos-dir ~/repos --profile-name balanced --human-logs
 
 # Aggregate and write unified outputs to results/summaries
+# (positional or --results-dir are both accepted)
 python3 scripts/cli/jmo.py report ./results --profile --human-logs
+# or
+python3 scripts/cli/jmo.py report --results-dir ./results --profile --human-logs
 
 # Or do both in one step for CI with a failure threshold
 python3 scripts/cli/jmo.py ci --repos-dir ~/repos --profile-name fast --fail-on HIGH --profile --human-logs
@@ -112,6 +184,8 @@ cat ~/security-results-*/SUMMARY_REPORT.md
 # Linux: xdg-open ~/security-results-*/dashboard.html
 ```
 
+Looking for screenshots and how to capture them? See: [docs/screenshots/README.md](docs/screenshots/README.md)
+
 ## 📚 Documentation
 
 ### Workflow (at a glance)
@@ -543,6 +617,14 @@ The HTML dashboard provides:
 
 Contributions are welcome! Please feel free to submit pull requests or open issues for bugs and feature requests.
 
+## ❤️ Support
+
+If this toolkit saves you time, consider fueling development with an energy drink.
+
+- Prefer one-time tips? Ko‑fi: https://ko-fi.com/jmogaming
+- When you’re ready, replace the badge target with your preferred platform: GitHub Sponsors (industry standard), Open Collective, Ko-fi, or Stripe Checkout.
+- GitHub Sponsors integrates directly with your GitHub profile and repository sidebar once enabled.
+
 ## 📝 License
 
 MIT License. See LICENSE.

{jmo_security-0.3.0 → jmo_security-0.3.2}/jmo_security.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: jmo-security
-Version: 0.3.0
+Version: 0.3.2
 Summary: JMo Security Audit Suite (terminal-first, multi-tool, unified outputs)
 Author-email: James Moceri <general@jmogaming.com>
 License: MIT
@@ -13,6 +13,7 @@ License-File: LICENSE
 Provides-Extra: reporting
 Requires-Dist: PyYAML>=6.0; extra == "reporting"
 Requires-Dist: jsonschema>=4.0; extra == "reporting"
+Dynamic: license-file
 
 # JMo's Security Audit Tool Suite
 
@@ -20,14 +21,77 @@ Requires-Dist: jsonschema>=4.0; extra == "reporting"
    <img src="assets/jmo-logo.png" alt="JMo Security Audit Tool Suite" width="220" />
 </p>
 
+[![Tests](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml?query=branch%3Amain)
+[![Website](https://img.shields.io/website?url=https%3A%2F%2Fjmotools.com)](https://jmotools.com)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/Python-%E2%89%A53.8-3776AB?logo=python&logoColor=white)](#)
+[![Contributions welcome](https://img.shields.io/badge/Contributions-welcome-brightgreen.svg)](https://github.com/jimmy058910/jmo-security-repo/issues)
+[![Buy me an energy drink](https://img.shields.io/badge/Buy%20me%20an-energy%20drink-%23ff4d00)](https://ko-fi.com/jmogaming)
+
+<!-- CI/coverage/package badges (enable once configured)
+[![CI](https://img.shields.io/badge/CI-GitHub%20Actions-coming--soon-lightgrey)](#)
+<!-- If/when a workflow exists, switch to:
 [![Tests](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml/badge.svg)](https://github.com/jimmy058910/jmo-security-repo/actions/workflows/tests.yml)
+-->
+<!-- Codecov (enable after uploading coverage):
 [![codecov](https://codecov.io/gh/jimmy058910/jmo-security-repo/branch/main/graph/badge.svg)](https://codecov.io/gh/jimmy058910/jmo-security-repo)
+-->
+<!-- PyPI (enable after first release):
 [![PyPI - Version](https://img.shields.io/pypi/v/jmo-security)](https://pypi.org/project/jmo-security/)
+-->
 
 A terminal-first, cross-platform security audit toolkit that orchestrates multiple scanners (secrets, SAST, SBOM, IaC, Dockerfile) with a unified Python CLI, normalized outputs, and an HTML dashboard.
 
 👉 New here? Read the comprehensive User Guide: [docs/USER_GUIDE.md](docs/USER_GUIDE.md)
 Docs hub: [docs/index.md](docs/index.md)
+Project homepage: [jmotools.com](https://jmotools.com)
+
+> Origin story: This started as part of a Cybersecurity Capstone Project. It has since grown into a general-purpose toolkit. I’d love for folks with deeper expertise to jump in—issues and PRs are welcome!
+
+Thinking about contributing? See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, coding standards, and release steps.
+
+Roadmap & history:
+- Completed steps (summary): see [RELEASE_NOTES.md](RELEASE_NOTES.md) → “Roadmap Summary (Steps 1–13)”
+- Active/planned work: see [ROADMAP.md](ROADMAP.md)
+
+For scanning a list of repos from a TSV end-to-end (clone + unshallow + full toolchain), see: [docs/examples/scan_from_tsv.md](docs/examples/scan_from_tsv.md)
+
+## 🧪 New: Simple wrapper commands
+
+If you're just getting started, use the beginner-friendly wrapper:
+
+- `jmotools fast` — quick scan (fast profile)
+- `jmotools balanced` — default balanced scan
+- `jmotools full` — deep scan with more tools
+
+Examples:
+
+```bash
+# Fast profile over repos listed in a TSV (auto-clone) and open dashboard when done
+jmotools fast --tsv ai-search/candidates.tsv --dest ./repos-tsv
+
+# Full (deep) profile targeting an existing directory of repos
+jmotools full --repos-dir ./repos-tsv --allow-missing-tools
+```
+
+Under the hood, these commands verify your OS/tools, optionally clone from a TSV, run `jmo ci` with the appropriate profile, and open `dashboard.html` and `SUMMARY.md` on completion.
+
+Setup tools quickly:
+
+```bash
+# Check tools or auto-install on Linux/WSL (where supported)
+jmotools setup --check
+jmotools setup --auto-install         # or --print-commands, --force-reinstall
+```
+
+Makefile shortcuts:
+
+```bash
+make setup             # jmotools setup --check (installs package if needed)
+make fast DIR=~/repos  # jmotools fast --repos-dir ~/repos
+make balanced DIR=~/repos
+make full DIR=~/repos
+```
 
 ## 🎯 Overview
 
@@ -55,6 +119,12 @@ make verify-env      # check OS/WSL/macOS and tool availability
 make dev-deps        # install Python dev dependencies
 ```
 
+Optional: install the package locally to get `jmo` and `jmotools` commands on your PATH:
+
+```bash
+pip install -e .
+```
+
 ### Quick Start (Unified CLI)
 
 1) Verify your environment (Linux/WSL/macOS) and see install hints for optional tools:
@@ -76,7 +146,10 @@ make dev-deps
 python3 scripts/cli/jmo.py scan --repos-dir ~/repos --profile-name balanced --human-logs
 
 # Aggregate and write unified outputs to results/summaries
+# (positional or --results-dir are both accepted)
 python3 scripts/cli/jmo.py report ./results --profile --human-logs
+# or
+python3 scripts/cli/jmo.py report --results-dir ./results --profile --human-logs
 
 # Or do both in one step for CI with a failure threshold
 python3 scripts/cli/jmo.py ci --repos-dir ~/repos --profile-name fast --fail-on HIGH --profile --human-logs
@@ -128,6 +201,8 @@ cat ~/security-results-*/SUMMARY_REPORT.md
 # Linux: xdg-open ~/security-results-*/dashboard.html
 ```
 
+Looking for screenshots and how to capture them? See: [docs/screenshots/README.md](docs/screenshots/README.md)
+
 ## 📚 Documentation
 
 ### Workflow (at a glance)
@@ -559,6 +634,14 @@ The HTML dashboard provides:
 
 Contributions are welcome! Please feel free to submit pull requests or open issues for bugs and feature requests.
 
+## ❤️ Support
+
+If this toolkit saves you time, consider fueling development with an energy drink.
+
+- Prefer one-time tips? Ko‑fi: https://ko-fi.com/jmogaming
+- When you’re ready, replace the badge target with your preferred platform: GitHub Sponsors (industry standard), Open Collective, Ko-fi, or Stripe Checkout.
+- GitHub Sponsors integrates directly with your GitHub profile and repository sidebar once enabled.
+
 ## 📝 License
 
 MIT License. See LICENSE.

jmo_security-0.3.2/jmo_security.egg-info/SOURCES.txt

@@ -0,0 +1,39 @@
+LICENSE
+README.md
+pyproject.toml
+jmo_security.egg-info/PKG-INFO
+jmo_security.egg-info/SOURCES.txt
+jmo_security.egg-info/dependency_links.txt
+jmo_security.egg-info/entry_points.txt
+jmo_security.egg-info/requires.txt
+jmo_security.egg-info/top_level.txt
+scripts/__init__.py
+scripts/cli/__init__.py
+scripts/cli/clone_from_tsv.py
+scripts/cli/jmo.py
+scripts/cli/jmotools.py
+scripts/core/__init__.py
+scripts/core/common_finding.py
+scripts/core/config.py
+scripts/core/generate_dashboard.py
+scripts/core/normalize_and_report.py
+scripts/core/schema_utils.py
+scripts/core/suppress.py
+scripts/core/adapters/__init__.py
+scripts/core/adapters/bandit_adapter.py
+scripts/core/adapters/checkov_adapter.py
+scripts/core/adapters/gitleaks_adapter.py
+scripts/core/adapters/hadolint_adapter.py
+scripts/core/adapters/noseyparker_adapter.py
+scripts/core/adapters/osv_adapter.py
+scripts/core/adapters/semgrep_adapter.py
+scripts/core/adapters/syft_adapter.py
+scripts/core/adapters/tfsec_adapter.py
+scripts/core/adapters/trivy_adapter.py
+scripts/core/adapters/trufflehog_adapter.py
+scripts/core/reporters/__init__.py
+scripts/core/reporters/basic_reporter.py
+scripts/core/reporters/html_reporter.py
+scripts/core/reporters/sarif_reporter.py
+scripts/core/reporters/suppression_reporter.py
+scripts/core/reporters/yaml_reporter.py

{jmo_security-0.3.0 → jmo_security-0.3.2}/jmo_security.egg-info/entry_points.txt

@@ -1,2 +1,3 @@
 [console_scripts]
 jmo = scripts.cli.jmo:main
+jmotools = scripts.cli.jmotools:main

{jmo_security-0.3.0 → jmo_security-0.3.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "jmo-security"
-version = "0.3.0"
+version = "0.3.2"
 description = "JMo Security Audit Suite (terminal-first, multi-tool, unified outputs)"
 readme = "README.md"
 requires-python = ">=3.8"
@@ -22,7 +22,10 @@ reporting = ["PyYAML>=6.0", "jsonschema>=4.0"]
 
 [project.scripts]
 jmo = "scripts.cli.jmo:main"
+jmotools = "scripts.cli.jmotools:main"
 
 [tool.setuptools]
-packages = ["scripts"]
 include-package-data = true
+
+[tool.setuptools.packages.find]
+include = ["scripts*"]

jmo_security-0.3.2/scripts/cli/__init__.py

@@ -0,0 +1,6 @@
+"""CLI package for jmo-security.
+
+This file ensures setuptools treats `scripts/cli` as a proper Python package
+so that the console entry point `jmo = scripts.cli.jmo:main` resolves correctly
+when installed from PyPI.
+"""

jmo_security-0.3.2/scripts/cli/clone_from_tsv.py

@@ -0,0 +1,182 @@
+#!/usr/bin/env python3
+"""
+Clone repositories listed in a TSV and emit a targets file for jmo scan.
+
+Inputs
+- --tsv: Path to a TSV file with a header containing either 'url' or 'full_name' columns.
+  Example header: full_name	url	stars	language	description	created_at	updated_at	matched_by
+  Rows may have blank description; this script only cares about url/full_name.
+
+Outputs
+- Clones repos under --dest (default: ./repos-tsv/<owner>/<repo>)
+- Writes a newline-delimited list of absolute repo paths to --targets-out (default: results/targets.tsv.txt)
+
+Behavior
+- If a repo already exists, perform a fetch and ensure it is not shallow ("unshallowed")
+- Prefer the 'url' column when present; otherwise construct https://github.com/<full_name>.git
+- Limits cloning to --max (default: all)
+"""
+from __future__ import annotations
+
+import argparse
+import csv
+import subprocess
+import sys
+from pathlib import Path
+from typing import List, Optional, Tuple
+
+
+def log(msg: str, level: str = "INFO", human: bool = True) -> None:
+    level = level.upper()
+    if human:
+        color = {"DEBUG": "\x1b[36m", "INFO": "\x1b[32m", "WARN": "\x1b[33m", "ERROR": "\x1b[31m"}.get(level, "")
+        reset = "\x1b[0m"
+        sys.stderr.write(f"{color}{level:5}{reset} {msg}\n")
+    else:
+        import json
+        import datetime
+        rec = {"ts": datetime.datetime.utcnow().isoformat() + "Z", "level": level, "msg": msg}
+        sys.stderr.write(json.dumps(rec) + "\n")
+
+
+def run(cmd: List[str], cwd: Optional[Path] = None, ok_rcs: Tuple[int, ...] = (0,)) -> Tuple[int, str, str]:
+    try:
+        cp = subprocess.run(
+            cmd,
+            cwd=str(cwd) if cwd else None,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            text=True,
+            check=False,
+        )
+        rc = cp.returncode
+        if rc not in ok_rcs:
+            return rc, cp.stdout or "", cp.stderr or ""
+        return rc, cp.stdout or "", cp.stderr or ""
+    except FileNotFoundError as e:
+        return 127, "", str(e)
+
+
+def ensure_unshallowed(repo_dir: Path) -> None:
+    # Determine if shallow and unshallow if needed
+    rc, out, err = run(["git", "rev-parse", "--is-shallow-repository"], cwd=repo_dir)
+    if rc != 0:
+        log(f"git rev-parse failed for {repo_dir.name}: {err.strip()}", "WARN")
+        return
+    shallow = (out.strip().lower() == "true")
+    if shallow:
+        # Attempt to unshallow
+        rc, _, err = run(["git", "fetch", "--unshallow"], cwd=repo_dir)
+        if rc != 0:
+            # Some setups require specifying the remote; try origin
+            rc2, _, err2 = run(["git", "fetch", "origin", "--unshallow"], cwd=repo_dir)
+            if rc2 != 0:
+                log(f"Failed to unshallow {repo_dir.name}: {err.strip() or err2.strip()}", "WARN")
+        # Also fetch tags/prune to be thorough
+    run(["git", "fetch", "--all", "--tags", "--prune"], cwd=repo_dir)
+
+
+def clone_or_update(url: str, dest_root: Path) -> Optional[Path]:
+    # Derive a stable folder path like <dest_root>/<owner>/<repo>
+    try:
+        # Strip trailing .git for folder naming
+        stem = url.split("//", 1)[-1]
+        # stem like github.com/owner/repo(.git)
+        parts = stem.split("/")
+        owner, repo = parts[-2], parts[-1]
+        if repo.endswith(".git"):
+            repo = repo[:-4]
+    except Exception:
+        owner, repo = "misc", url.replace("://", "_").replace("/", "-")
+    target = dest_root / owner / repo
+    target.parent.mkdir(parents=True, exist_ok=True)
+    if target.exists():
+        log(f"Updating existing repo {owner}/{repo}")
+        rc, _, err = run(["git", "remote", "-v"], cwd=target)
+        if rc != 0:
+            log(f"Not a git repo at {target}: {err.strip()}", "ERROR")
+            return None
+        run(["git", "fetch", "--all", "--tags", "--prune"], cwd=target)
+        ensure_unshallowed(target)
+        return target
+    log(f"Cloning {owner}/{repo}")
+    rc, _, err = run(["git", "clone", url, str(target)])
+    if rc != 0:
+        log(f"Clone failed for {url}: {err.strip()}", "ERROR")
+        return None
+    ensure_unshallowed(target)
+    return target
+
+
+def parse_tsv(tsv_path: Path, max_count: Optional[int]) -> List[str]:
+    urls: List[str] = []
+    with tsv_path.open("r", encoding="utf-8") as f:
+        # Sniff delimiter; default to tab
+        sample = f.read(4096)
+        f.seek(0)
+        dialect = csv.Sniffer().sniff(sample, delimiters="\t,;") if sample else csv.excel_tab
+        reader = csv.DictReader(f, dialect=dialect)
+        cols = [c.strip().lower() for c in (reader.fieldnames or [])]
+        if not cols:
+            raise RuntimeError("TSV file has no header row")
+        use_url = "url" in cols
+        use_full = "full_name" in cols
+        if not use_url and not use_full:
+            raise RuntimeError("TSV must include either 'url' or 'full_name' columns")
+        for row in reader:
+            if max_count and len(urls) >= max_count:
+                break
+            u = (row.get("url") or "").strip() if use_url else ""
+            if not u and use_full:
+                fn = (row.get("full_name") or "").strip()
+                if fn:
+                    u = f"https://github.com/{fn}.git"
+            if u:
+                urls.append(u)
+    return urls
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    ap = argparse.ArgumentParser(description="Clone repos listed in a TSV and emit a jmo targets file")
+    ap.add_argument("--tsv", required=True, help="Path to candidates.tsv with 'url' or 'full_name' header")
+    ap.add_argument("--dest", default="repos-tsv", help="Destination directory to clone into (default: repos-tsv)")
+    ap.add_argument("--targets-out", default=str(Path("results") / "targets.tsv.txt"), help="Path to write newline-delimited repo paths (default: results/targets.tsv.txt)")
+    ap.add_argument("--max", type=int, default=None, help="Optional max number of repos to process")
+    ap.add_argument("--human-logs", action="store_true", help="Use human-friendly logs")
+    args = ap.parse_args(argv)
+
+    tsv_path = Path(args.tsv).expanduser().resolve()
+    dest = Path(args.dest).expanduser().resolve()
+    targets_out = Path(args.targets_out).expanduser().resolve()
+    targets_out.parent.mkdir(parents=True, exist_ok=True)
+
+    if not tsv_path.exists():
+        log(f"TSV not found: {tsv_path}", "ERROR")
+        return 2
+
+    urls = parse_tsv(tsv_path, args.max)
+    if not urls:
+        log("No repository URLs found in TSV", "ERROR")
+        return 2
+
+    paths: List[Path] = []
+    for url in urls:
+        p = clone_or_update(url, dest)
+        if p:
+            paths.append(p)
+
+    if not paths:
+        log("No repositories cloned/updated successfully", "ERROR")
+        return 1
+
+    # Write absolute paths to targets file
+    with targets_out.open("w", encoding="utf-8") as f:
+        for p in paths:
+            f.write(str(p) + "\n")
+    log(f"Wrote targets file: {targets_out}")
+    log(f"Cloned/updated {len(paths)} repos into {dest}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())