dump-github 0.2.0__tar.gz → 0.2.1__tar.gz

{dump_github-0.2.0 → dump_github-0.2.1}/CHANGELOG.md

@@ -5,6 +5,26 @@ All notable changes to this project are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/).
 
+## [0.2.1] - 2026-11-12
+
+### Fixed
+
+- Non-token mode silently returned no repositories when the anonymous GitHub API
+  hit its 60 requests/hour/IP rate limit (common on shared/NAT/CI networks).
+  Listing now falls back to the public GitHub web repository pages
+  (`github.com/{user}?tab=repositories`, `github.com/orgs/{org}/repositories`)
+  and prints a warning, so public backups work again.
+- GitHub API failures are no longer swallowed: the CLI reports the error and
+  exits with status 1 instead of exiting 0 with no output.
+- Zip downloads no longer assume `main` when the default branch is unknown
+  (web fallback); the `HEAD` archive is used, and token mode omits the ref so
+  GitHub resolves the default branch.
+
+### Tests
+
+- 27 tests, including the API rate-limit fallback, pagination, organisation
+  detection via the web listing, and the `HEAD` archive path.
+
 ## [0.2.0] - 2026-11-12
 
 ### Added

{dump_github-0.2.0/dump_github.egg-info → dump_github-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dump-github
-Version: 0.2.0
+Version: 0.2.1
 Summary: Back up users' GitHub repositories in case they need them later.
 Author-email: core2002 <core2002@aliyun.com>
 License-Expression: MIT
@@ -81,6 +81,15 @@ GitHub’s API requires a proper `Authorization: Bearer <token>` header. This to
 - **Private repositories**: Without a token, only public repos are listed and can be cloned or archived. With a token that has the right scopes, you can include private repos you are allowed to access (for example when backing up your own account).
 - **Rate limits**: Authenticated API requests get a much higher limit than anonymous requests.
 
+**Without a token**
+
+Only public repositories can be listed and downloaded. The anonymous GitHub API allows
+60 requests per hour per IP, which shared/NATed networks and CI runners exhaust quickly;
+if `api.github.com` is rate-limited or otherwise unavailable, the tool automatically
+falls back to the public GitHub web listing
+(`github.com/{user}?tab=repositories` / `github.com/orgs/{org}/repositories`), prints a
+warning and keeps going. Private repositories still require a token.
+
 **How the token is used**
 
 - **Listing repos**: sent as `Bearer` on the REST API and scoped strictly to the requested account:
@@ -97,9 +106,12 @@ Create a token under GitHub → **Settings** → **Developer settings** → **Pe
 **Examples**
 
 ```bash
-# Public repos only (no token)
+# Public repos only (no token; falls back to the web listing if the API is rate-limited)
 dump-github Core2002 -p
 
+# Public backup without any credentials
+dump-github Core2002 -d
+
 # Token on the command line (private + higher API limits)
 dump-github username --token ghp_xxxxxxxxxxxx
 

{dump_github-0.2.0 → dump_github-0.2.1}/README.md

@@ -49,6 +49,15 @@ GitHub’s API requires a proper `Authorization: Bearer <token>` header. This to
 - **Private repositories**: Without a token, only public repos are listed and can be cloned or archived. With a token that has the right scopes, you can include private repos you are allowed to access (for example when backing up your own account).
 - **Rate limits**: Authenticated API requests get a much higher limit than anonymous requests.
 
+**Without a token**
+
+Only public repositories can be listed and downloaded. The anonymous GitHub API allows
+60 requests per hour per IP, which shared/NATed networks and CI runners exhaust quickly;
+if `api.github.com` is rate-limited or otherwise unavailable, the tool automatically
+falls back to the public GitHub web listing
+(`github.com/{user}?tab=repositories` / `github.com/orgs/{org}/repositories`), prints a
+warning and keeps going. Private repositories still require a token.
+
 **How the token is used**
 
 - **Listing repos**: sent as `Bearer` on the REST API and scoped strictly to the requested account:
@@ -65,9 +74,12 @@ Create a token under GitHub → **Settings** → **Developer settings** → **Pe
 **Examples**
 
 ```bash
-# Public repos only (no token)
+# Public repos only (no token; falls back to the web listing if the API is rate-limited)
 dump-github Core2002 -p
 
+# Public backup without any credentials
+dump-github Core2002 -d
+
 # Token on the command line (private + higher API limits)
 dump-github username --token ghp_xxxxxxxxxxxx
 

dump_github-0.2.1/dump_github/_version.py

@@ -0,0 +1 @@
+__version__ = "0.2.1"

dump_github-0.2.1/dump_github/lib.py

@@ -0,0 +1,345 @@
+import re
+import warnings
+from urllib.parse import quote
+
+import requests
+
+GITHUB_API = "https://api.github.com"
+GITHUB_WEB = "https://github.com"
+PER_PAGE = 100
+WEB_USER_AGENT = "dump-github (+https://github.com/Core2002/dump_github)"
+
+
+class GitHubAPIError(RuntimeError):
+    """GitHub could not be read through the REST API (or the web fallback)."""
+
+    def __init__(self, status_code, message):
+        self.status_code = status_code
+        self.message = message
+        super().__init__(message)
+
+
+def _github_auth_headers(token):
+    """GitHub REST API expects Authorization: Bearer <token> (PAT classic / fine-grained)."""
+    if not token:
+        return {}
+    token = token.strip()
+    if not token:
+        return {}
+    if token.lower().startswith(("bearer ", "token ", "basic ")):
+        return {"Authorization": token}
+    return {"Authorization": "Bearer {}".format(token)}
+
+
+def _api_headers(token=""):
+    headers = {
+        "Accept": "application/vnd.github+json",
+        "X-GitHub-Api-Version": "2022-11-28",
+    }
+    headers.update(_github_auth_headers(token))
+    return headers
+
+
+def _api_get(path, token=""):
+    """GET one GitHub REST API path and return the decoded JSON.
+
+    Raises GitHubAPIError instead of silently returning an empty result, so
+    callers can distinguish "no repositories" from "request failed".
+    """
+    try:
+        response = requests.get(
+            GITHUB_API + path, headers=_api_headers(token), timeout=30
+        )
+    except requests.RequestException as exc:
+        raise GitHubAPIError(None, "request failed: {}".format(exc)) from exc
+    if response.status_code != 200:
+        message = "HTTP {}".format(response.status_code)
+        try:
+            payload = response.json()
+        except ValueError:
+            payload = None
+        if isinstance(payload, dict) and payload.get("message"):
+            message = "{} (HTTP {})".format(payload["message"], response.status_code)
+        raise GitHubAPIError(response.status_code, message)
+    try:
+        return response.json()
+    except ValueError as exc:
+        raise GitHubAPIError(
+            response.status_code, "invalid JSON response from GitHub"
+        ) from exc
+
+
+def get_authenticated_login(token=""):
+    """Return the login of the token owner, or None if there is no usable token."""
+    if not _github_auth_headers(token):
+        return None
+    try:
+        account = _api_get("/user", token)
+    except GitHubAPIError:
+        return None
+    if not isinstance(account, dict):
+        return None
+    return account.get("login") or None
+
+
+def get_account_type(username, token=""):
+    """Return ``"Organization"``/``"User"`` for ``username``, or None if unknown."""
+    quoted = quote(username, safe="")
+    try:
+        account = _api_get("/users/{}".format(quoted), token)
+    except GitHubAPIError:
+        account = None
+    if isinstance(account, dict) and account.get("type"):
+        return account["type"]
+    # ``/users/{org}`` is not guaranteed to resolve for organisations, so fall
+    # back to the org endpoint, which unambiguously identifies one.
+    try:
+        org = _api_get("/orgs/{}".format(quoted), token)
+    except GitHubAPIError:
+        org = None
+    if isinstance(org, dict) and org.get("login"):
+        return "Organization"
+    return None
+
+
+def _repos_path(username, page, authenticated_login, account_type):
+    """Build a listing API path that only contains repositories owned by ``username``.
+
+    ``GET /user/repos`` (used by an earlier version whenever a token was
+    supplied) lists every repository the token owner can see, which includes
+    repositories of the organisations the token owner is a member of, and it
+    ignores ``username`` entirely.  Listing must therefore be scoped to the
+    requested account:
+
+    * organisation -> ``/orgs/{org}/repos?type=all``; only repositories owned
+      by that organisation, including private ones the token may access
+    * own user account -> ``/user/repos?affiliation=owner``; keeps private
+      repos, drops organisation/collaborator repos
+    * any other user account -> ``/users/{username}/repos?type=owner``
+    """
+    quoted = quote(username, safe="")
+    if (account_type or "").lower() == "organization":
+        return "/orgs/{}/repos?type=all&per_page={}&page={}".format(
+            quoted, PER_PAGE, page
+        )
+    own_account = bool(authenticated_login) and (
+        authenticated_login.lower() == username.lower()
+    )
+    if own_account:
+        return "/user/repos?affiliation=owner&per_page={}&page={}".format(
+            PER_PAGE, page
+        )
+    return "/users/{}/repos?type=owner&per_page={}&page={}".format(
+        quoted, PER_PAGE, page
+    )
+
+
+def _is_owned_by(repo, username):
+    owner = repo.get("owner") or {}
+    login = owner.get("login") or ""
+    if login:
+        return login.lower() == username.lower()
+    full_name = repo.get("full_name") or ""
+    return full_name.split("/", 1)[0].lower() == username.lower()
+
+
+def _fetch_repos_page(username, page, token="", authenticated_login=None,
+                      account_type=None):
+    """Fetch one raw page of repositories owned by ``username``."""
+    path = _repos_path(username, page, authenticated_login, account_type)
+    repos = _api_get(path, token)
+    if not isinstance(repos, list):
+        return []
+    return [repo for repo in repos if isinstance(repo, dict)]
+
+
+def get_page_of_repos(username, page, token="", authenticated_login=None,
+                      account_type=None):
+    username = username.strip()
+    if account_type is None:
+        account_type = get_account_type(username, token)
+    if authenticated_login is None:
+        authenticated_login = get_authenticated_login(token)
+    repos = _fetch_repos_page(
+        username, page, token, authenticated_login, account_type
+    )
+    return [repo for repo in repos if _is_owned_by(repo, username)]
+
+
+# --- public web fallback -------------------------------------------------
+#
+# api.github.com allows only 60 unauthenticated requests per hour *per IP*.
+# Shared/NATed networks and CI runners exhaust that quickly, which used to make
+# ``dump-github user`` silently return nothing.  github.com's repository pages
+# are not subject to that API quota, so they are used as a read-only fallback
+# for public repositories when no token is configured.
+
+_USER_REPO_TAG_RE = re.compile(r'<a\b[^>]*itemprop="name codeRepository"[^>]*>')
+_WEB_REPO_HREF_RE = re.compile(r'href="/([^"/]+)/([^"/?#]+)"')
+_WEB_NEXT_RE = re.compile(r'rel="next"')
+_WEB_NON_REPO_PATHS = {
+    "followers",
+    "following",
+    "people",
+    "packages",
+    "projects",
+    "repositories",
+    "settings",
+    "sponsoring",
+}
+
+
+def _web_get(url):
+    try:
+        response = requests.get(
+            url, headers={"User-Agent": WEB_USER_AGENT}, timeout=30
+        )
+    except requests.RequestException as exc:
+        raise GitHubAPIError(None, "web request failed: {}".format(exc)) from exc
+    if response.status_code != 200:
+        raise GitHubAPIError(
+            response.status_code,
+            "github.com returned HTTP {}".format(response.status_code),
+        )
+    return response.text
+
+
+def _web_repo(owner, name):
+    return {
+        "name": name,
+        "full_name": "{}/{}".format(owner, name),
+        "owner": {"login": owner},
+        "clone_url": "{}/{}/{}.git".format(GITHUB_WEB, owner, name),
+        # The default branch is not exposed by the web listing; zip downloads
+        # then use the HEAD archive instead.
+    }
+
+
+def _parse_user_repos_page(html, username):
+    repos = []
+    seen = set()
+    for tag in _USER_REPO_TAG_RE.findall(html):
+        match = _WEB_REPO_HREF_RE.search(tag)
+        if not match:
+            continue
+        owner, name = match.group(1), match.group(2)
+        if owner.lower() != username.lower() or name in seen:
+            continue
+        seen.add(name)
+        repos.append(_web_repo(owner, name))
+    return repos
+
+
+def _parse_org_repos_page(html, org):
+    repos = []
+    seen = set()
+    for owner, name in _WEB_REPO_HREF_RE.findall(html):
+        if owner.lower() != org.lower():
+            continue
+        if name.lower() in _WEB_NON_REPO_PATHS or name in seen:
+            continue
+        seen.add(name)
+        repos.append(_web_repo(owner, name))
+    return repos
+
+
+def _web_page_url(username, page, is_org):
+    quoted = quote(username, safe="")
+    if is_org:
+        return "{}/orgs/{}/repositories?page={}".format(GITHUB_WEB, quoted, page)
+    return "{}/{}?tab=repositories&page={}".format(GITHUB_WEB, quoted, page)
+
+
+def _parse_web_page(html, username, is_org):
+    if is_org:
+        return _parse_org_repos_page(html, username)
+    return _parse_user_repos_page(html, username)
+
+
+def _search_via_web(username, account_type):
+    """List public repositories from github.com when the REST API is unavailable."""
+    is_org = (account_type or "").lower() == "organization"
+    first = _web_get(_web_page_url(username, 1, is_org))
+    repos = _parse_web_page(first, username, is_org)
+    if not repos and account_type is None:
+        # The account may be an organisation: its profile page redirects to
+        # github.com/{org} and does not contain the plain repository list.
+        try:
+            org_first = _web_get(_web_page_url(username, 1, True))
+        except GitHubAPIError as exc:
+            if exc.status_code == 404:
+                return []
+            raise
+        org_repos = _parse_org_repos_page(org_first, username)
+        if not org_repos:
+            return []
+        is_org = True
+        repos = org_repos
+        first = org_first
+    if not repos:
+        return []
+    page = 1
+    while _WEB_NEXT_RE.search(first):
+        page += 1
+        html = _web_get(_web_page_url(username, page, is_org))
+        page_repos = _parse_web_page(html, username, is_org)
+        if not page_repos:
+            break
+        repos.extend(page_repos)
+        first = html
+    return repos
+
+
+def _search_via_api(username, token, authenticated_login, account_type):
+    page = 0
+    res_repos = []
+    while True:
+        page += 1
+        repos = _fetch_repos_page(
+            username, page, token, authenticated_login, account_type
+        )
+        if len(repos) == 0:
+            break
+        res_repos += repos
+        if len(repos) < PER_PAGE:
+            break
+    return res_repos
+
+
+def search_user_repos(username, token=""):
+    username = username.strip()
+    authenticated_login = get_authenticated_login(token)
+    account_type = get_account_type(username, token)
+    try:
+        repos = _search_via_api(
+            username, token, authenticated_login, account_type
+        )
+    except GitHubAPIError as exc:
+        if token or exc.status_code == 404:
+            raise
+        warnings.warn(
+            "GitHub API request failed ({}); falling back to the public GitHub "
+            "web listing. Only public repositories can be listed without a "
+            "token.".format(exc),
+            RuntimeWarning,
+            stacklevel=2,
+        )
+        repos = _search_via_web(username, account_type)
+
+    # Never return repositories that belong to somebody else (e.g. a user repo
+    # returned for an organisation, or an organisation the token owner is in).
+    result = []
+    seen = set()
+    for repo in repos:
+        if not _is_owned_by(repo, username):
+            continue
+        key = (repo.get("full_name") or repo.get("name") or "").lower()
+        if key in seen:
+            continue
+        seen.add(key)
+        result.append(repo)
+    return result
+
+
+# Backward-compatible alias for the historical typo
+serach_user_repos = search_user_repos

{dump_github-0.2.0 → dump_github-0.2.1}/dump_github/main.py

@@ -1,9 +1,10 @@
 import argparse
 import os
 import subprocess
+import sys
 
 from ._version import __version__
-from .lib import search_user_repos
+from .lib import GitHubAPIError, search_user_repos
 
 # One-shot credential helper: git asks for credentials, the helper answers with
 # the token from the environment.  The token is therefore never part of the
@@ -67,7 +68,7 @@ def clone_repo(repo, token=""):
 
 
 def download_zip(user_name, reop, token="", limit_size=100):
-    ref = reop.get("default_branch") or "main"
+    ref = reop.get("default_branch")
     # Always archive the repository the API actually listed.  Using the
     # positional username here would point at the wrong owner whenever the
     # repo does not belong to that user.
@@ -88,7 +89,9 @@ def download_zip(user_name, reop, token="", limit_size=100):
     if max_size > 0:
         curl_cmd.extend(["--max-filesize", str(max_size)])
     if token:
-        url = "https://api.github.com/repos/{}/zipball/{}".format(full_name, ref)
+        url = "https://api.github.com/repos/{}/zipball".format(full_name)
+        if ref:
+            url += "/{}".format(ref)
         curl_cmd.extend(
             [
                 "-H",
@@ -101,7 +104,12 @@ def download_zip(user_name, reop, token="", limit_size=100):
             ]
         )
     else:
-        url = "https://github.com/{}/archive/refs/heads/{}.zip".format(full_name, ref)
+        # The web listing does not know the default branch; GitHub resolves
+        # HEAD to it.
+        if ref:
+            url = "https://github.com/{}/archive/refs/heads/{}.zip".format(full_name, ref)
+        else:
+            url = "https://github.com/{}/archive/HEAD.zip".format(full_name)
         curl_cmd.extend(["-o", file_name, url])
     print("Download {} : {}".format(reop["name"], url))
     subprocess.run(curl_cmd, check=False)
@@ -117,7 +125,20 @@ def download_zip(user_name, reop, token="", limit_size=100):
 def main(argv=None):
     args = parser.parse_args(argv)
     token = (args.token or "").strip() or (os.environ.get("GITHUB_TOKEN") or "").strip()
-    repos = search_user_repos(args.username, token)
+    try:
+        repos = search_user_repos(args.username, token)
+    except GitHubAPIError as exc:
+        print(
+            "error: cannot list repositories for {}: {}".format(args.username, exc),
+            file=sys.stderr,
+        )
+        raise SystemExit(1)
+    if not repos:
+        print(
+            "No public repositories found for {}.".format(args.username),
+            file=sys.stderr,
+        )
+        return
     for repo in repos:
         if args.print:
             print(repo["clone_url"])

{dump_github-0.2.0 → dump_github-0.2.1/dump_github.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dump-github
-Version: 0.2.0
+Version: 0.2.1
 Summary: Back up users' GitHub repositories in case they need them later.
 Author-email: core2002 <core2002@aliyun.com>
 License-Expression: MIT
@@ -81,6 +81,15 @@ GitHub’s API requires a proper `Authorization: Bearer <token>` header. This to
 - **Private repositories**: Without a token, only public repos are listed and can be cloned or archived. With a token that has the right scopes, you can include private repos you are allowed to access (for example when backing up your own account).
 - **Rate limits**: Authenticated API requests get a much higher limit than anonymous requests.
 
+**Without a token**
+
+Only public repositories can be listed and downloaded. The anonymous GitHub API allows
+60 requests per hour per IP, which shared/NATed networks and CI runners exhaust quickly;
+if `api.github.com` is rate-limited or otherwise unavailable, the tool automatically
+falls back to the public GitHub web listing
+(`github.com/{user}?tab=repositories` / `github.com/orgs/{org}/repositories`), prints a
+warning and keeps going. Private repositories still require a token.
+
 **How the token is used**
 
 - **Listing repos**: sent as `Bearer` on the REST API and scoped strictly to the requested account:
@@ -97,9 +106,12 @@ Create a token under GitHub → **Settings** → **Developer settings** → **Pe
 **Examples**
 
 ```bash
-# Public repos only (no token)
+# Public repos only (no token; falls back to the web listing if the API is rate-limited)
 dump-github Core2002 -p
 
+# Public backup without any credentials
+dump-github Core2002 -d
+
 # Token on the command line (private + higher API limits)
 dump-github username --token ghp_xxxxxxxxxxxx
 

{dump_github-0.2.0 → dump_github-0.2.1}/tests/test_lib.py

@@ -1,3 +1,4 @@
+import json
 import os
 import sys
 import unittest
@@ -12,6 +13,7 @@ class FakeResponse:
     def __init__(self, status_code, payload):
         self.status_code = status_code
         self._payload = payload
+        self.text = payload if isinstance(payload, str) else json.dumps(payload)
 
     def json(self):
         return self._payload
@@ -26,6 +28,37 @@ def make_repo(owner, name):
     }
 
 
+def url_matches(url, prefix):
+    return (
+        url == prefix
+        or url.startswith(prefix + "?")
+        or url.startswith(prefix + "/")
+    )
+
+
+def user_repos_html(names, next_page=False):
+    body = "".join(
+        '<a href="/alice/{}" itemprop="name codeRepository" >{}</a>'.format(
+            name, name
+        )
+        for name in names
+    )
+    if next_page:
+        body += '<a rel="next" href="/alice?tab=repositories&amp;page=2">Next</a>'
+    return body
+
+
+def org_repos_html(org, names, next_page=False):
+    body = "".join(
+        '<a href="/{}/{}">{}</a>'.format(org, name, name) for name in names
+    )
+    if next_page:
+        body += '<a rel="next" href="/orgs/{}/repositories?page=2">Next</a>'.format(
+            org
+        )
+    return body
+
+
 class ListingTests(unittest.TestCase):
     def setUp(self):
         self.calls = []
@@ -290,5 +323,132 @@ class ListingTests(unittest.TestCase):
         self.assertEqual([repo["name"] for repo in repos], ["public-repo"])
 
 
+class WebFallbackTests(unittest.TestCase):
+    def setUp(self):
+        self.calls = []
+
+    def fake_get(self, web):
+        def get(url, headers=None, timeout=None):
+            self.calls.append((url, headers or {}))
+            if url.startswith("https://api.github.com"):
+                return FakeResponse(403, {"message": "API rate limit exceeded"})
+            return web(url)
+
+        return get
+
+    def urls(self):
+        return [url for url, _ in self.calls]
+
+    def test_api_403_falls_back_to_web_listing_for_user(self):
+        def web(url):
+            if url.startswith("https://github.com/alice?tab=repositories"):
+                return FakeResponse(200, user_repos_html(["one", "two"]))
+            return FakeResponse(404, {"message": "Not Found"})
+
+        with mock.patch.object(lib.requests, "get", self.fake_get(web)):
+            with self.assertWarns(RuntimeWarning) as caught:
+                repos = lib.search_user_repos("alice")
+
+        self.assertEqual([repo["name"] for repo in repos], ["one", "two"])
+        self.assertIn("rate limit", str(caught.warning))
+        self.assertIn(
+            "https://github.com/alice?tab=repositories&page=1", self.urls()
+        )
+        self.assertTrue(
+            all("Authorization" not in headers for _, headers in self.calls)
+        )
+        self.assertEqual(
+            repos[0]["clone_url"], "https://github.com/alice/one.git"
+        )
+        self.assertNotIn("default_branch", repos[0])
+
+    def test_web_fallback_paginates_until_no_next_link(self):
+        first_page = ["repo-{}".format(i) for i in range(30)]
+        second_page = ["last-a", "last-b"]
+
+        def web(url):
+            if "page=1" in url:
+                return FakeResponse(200, user_repos_html(first_page, next_page=True))
+            if "page=2" in url:
+                return FakeResponse(200, user_repos_html(second_page))
+            return FakeResponse(404, {"message": "Not Found"})
+
+        with mock.patch.object(lib.requests, "get", self.fake_get(web)):
+            with self.assertWarns(RuntimeWarning):
+                repos = lib.search_user_repos("alice")
+
+        self.assertEqual(len(repos), 32)
+        self.assertEqual(repos[-1]["name"], "last-b")
+        self.assertFalse(any("page=3" in url for url in self.urls()))
+
+    def test_web_fallback_detects_organisation(self):
+        def web(url):
+            if url.startswith("https://github.com/acme-org?tab=repositories"):
+                return FakeResponse(200, "<html>organisation profile</html>")
+            if url.startswith("https://github.com/orgs/acme-org/repositories"):
+                return FakeResponse(200, org_repos_html("acme-org", ["org-a", "org-b"]))
+            return FakeResponse(404, {"message": "Not Found"})
+
+        with mock.patch.object(lib.requests, "get", self.fake_get(web)):
+            with self.assertWarns(RuntimeWarning):
+                repos = lib.search_user_repos("acme-org")
+
+        self.assertEqual([repo["name"] for repo in repos], ["org-a", "org-b"])
+        self.assertIn(
+            "https://github.com/orgs/acme-org/repositories?page=1", self.urls()
+        )
+        self.assertEqual(
+            repos[0]["full_name"], "acme-org/org-a"
+        )
+
+    def test_web_fallback_returns_empty_for_missing_account(self):
+        def web(url):
+            if url.startswith("https://github.com/nobody?tab=repositories"):
+                return FakeResponse(200, "<html></html>")
+            return FakeResponse(404, {"message": "Not Found"})
+
+        with mock.patch.object(lib.requests, "get", self.fake_get(web)):
+            with self.assertWarns(RuntimeWarning):
+                repos = lib.search_user_repos("nobody")
+
+        self.assertEqual(repos, [])
+
+    def test_token_mode_403_raises_and_does_not_scrape(self):
+        def fake_get(url, headers=None, timeout=None):
+            self.calls.append((url, headers or {}))
+            if url == "https://api.github.com/user":
+                return FakeResponse(200, {"login": "bob"})
+            if url == "https://api.github.com/users/alice":
+                return FakeResponse(200, {"login": "alice", "type": "User"})
+            if url.startswith("https://api.github.com/users/alice/repos"):
+                return FakeResponse(403, {"message": "API rate limit exceeded"})
+            return FakeResponse(200, user_repos_html(["should-not-be-used"]))
+
+        with mock.patch.object(lib.requests, "get", fake_get):
+            with self.assertRaises(lib.GitHubAPIError) as caught:
+                lib.search_user_repos("alice", token="ghp_test")
+
+        self.assertEqual(caught.exception.status_code, 403)
+        self.assertFalse(
+            any(url.startswith("https://github.com/") for url in self.urls())
+        )
+
+    def test_api_404_is_reported_instead_of_scraping(self):
+        def fake_get(url, headers=None, timeout=None):
+            self.calls.append((url, headers or {}))
+            if url.startswith("https://api.github.com"):
+                return FakeResponse(404, {"message": "Not Found"})
+            return FakeResponse(200, user_repos_html(["should-not-be-used"]))
+
+        with mock.patch.object(lib.requests, "get", fake_get):
+            with self.assertRaises(lib.GitHubAPIError) as caught:
+                lib.search_user_repos("ghost")
+
+        self.assertEqual(caught.exception.status_code, 404)
+        self.assertFalse(
+            any(url.startswith("https://github.com/") for url in self.urls())
+        )
+
+
 if __name__ == "__main__":
     unittest.main()

{dump_github-0.2.0 → dump_github-0.2.1}/tests/test_main.py

@@ -1,4 +1,6 @@
+import contextlib
 import importlib
+import io
 import os
 import sys
 import unittest
@@ -100,5 +102,75 @@ class CloneTests(unittest.TestCase):
         self.assertNotIn("DUMP_GITHUB_TOKEN", result.stdout)
 
 
+class CliBehaviourTests(unittest.TestCase):
+    def setUp(self):
+        self.calls = []
+
+    def fake_run(self, cmd, **kwargs):
+        self.calls.append((cmd, kwargs))
+        return mock.Mock(returncode=0)
+
+    def test_zip_without_token_uses_head_when_branch_unknown(self):
+        repo = {"name": "repo", "full_name": "alice/repo"}
+        with mock.patch.object(main.subprocess, "run", self.fake_run):
+            main.download_zip("alice", repo, limit_size=0)
+
+        cmd = self.calls[0][0]
+        self.assertEqual(
+            cmd[-1], "https://github.com/alice/repo/archive/HEAD.zip"
+        )
+
+    def test_zip_with_token_and_known_branch_uses_zipball_ref(self):
+        repo = {
+            "name": "repo",
+            "full_name": "alice/repo",
+            "default_branch": "main",
+        }
+        with mock.patch.object(main.subprocess, "run", self.fake_run):
+            main.download_zip("alice", repo, token="ghp_test", limit_size=0)
+
+        cmd = self.calls[0][0]
+        self.assertIn(
+            "https://api.github.com/repos/alice/repo/zipball/main", cmd
+        )
+        self.assertIn("Authorization: Bearer ghp_test", cmd)
+
+    def test_zip_with_token_and_unknown_branch_omits_ref(self):
+        repo = {"name": "repo", "full_name": "alice/repo"}
+        with mock.patch.object(main.subprocess, "run", self.fake_run):
+            main.download_zip("alice", repo, token="ghp_test", limit_size=0)
+
+        cmd = self.calls[0][0]
+        self.assertIn("https://api.github.com/repos/alice/repo/zipball", cmd)
+        self.assertFalse(
+            any("/zipball/" in part for part in cmd)
+        )
+
+    def test_listing_error_exits_nonzero_with_message(self):
+        err = io.StringIO()
+        with mock.patch.object(
+            main,
+            "search_user_repos",
+            side_effect=main.GitHubAPIError(
+                403, "API rate limit exceeded (HTTP 403)"
+            ),
+        ):
+            with self.assertRaises(SystemExit) as caught:
+                with contextlib.redirect_stderr(err):
+                    main.main(["alice"])
+
+        self.assertEqual(caught.exception.code, 1)
+        self.assertIn("alice", err.getvalue())
+        self.assertIn("rate limit", err.getvalue())
+
+    def test_empty_listing_reports_message(self):
+        err = io.StringIO()
+        with mock.patch.object(main, "search_user_repos", return_value=[]):
+            with contextlib.redirect_stderr(err):
+                main.main(["alice"])
+
+        self.assertIn("No public repositories found", err.getvalue())
+
+
 if __name__ == "__main__":
     unittest.main()

dump_github-0.2.0/dump_github/_version.py

@@ -1 +0,0 @@
-__version__ = "0.2.0"

dump_github-0.2.0/dump_github/lib.py

@@ -1,158 +0,0 @@
-from urllib.parse import quote
-
-import requests
-
-GITHUB_API = "https://api.github.com"
-PER_PAGE = 100
-
-
-def _github_auth_headers(token):
-    """GitHub REST API expects Authorization: Bearer <token> (PAT classic / fine-grained)."""
-    if not token:
-        return {}
-    token = token.strip()
-    if not token:
-        return {}
-    if token.lower().startswith(("bearer ", "token ", "basic ")):
-        return {"Authorization": token}
-    return {"Authorization": "Bearer {}".format(token)}
-
-
-def _api_headers(token=""):
-    headers = {
-        "Accept": "application/vnd.github+json",
-        "X-GitHub-Api-Version": "2022-11-28",
-    }
-    headers.update(_github_auth_headers(token))
-    return headers
-
-
-def _api_get(path, token=""):
-    """GET one GitHub REST API path and return the decoded JSON, or None on failure."""
-    try:
-        response = requests.get(
-            GITHUB_API + path, headers=_api_headers(token), timeout=30
-        )
-    except requests.RequestException:
-        return None
-    if response.status_code != 200:
-        return None
-    try:
-        return response.json()
-    except ValueError:
-        return None
-
-
-def get_authenticated_login(token=""):
-    """Return the login of the token owner, or None if there is no usable token."""
-    if not _github_auth_headers(token):
-        return None
-    account = _api_get("/user", token)
-    if not isinstance(account, dict):
-        return None
-    return account.get("login") or None
-
-
-def get_account_type(username, token=""):
-    """Return ``"Organization"``/``"User"`` for ``username``, or None if unknown."""
-    quoted = quote(username, safe="")
-    account = _api_get("/users/{}".format(quoted), token)
-    if isinstance(account, dict) and account.get("type"):
-        return account["type"]
-    # ``/users/{org}`` is not guaranteed to resolve for organisations, so fall
-    # back to the org endpoint, which unambiguously identifies one.
-    org = _api_get("/orgs/{}".format(quoted), token)
-    if isinstance(org, dict) and org.get("login"):
-        return "Organization"
-    return None
-
-
-def _repos_path(username, page, authenticated_login, account_type):
-    """Build a listing API path that only contains repositories owned by ``username``.
-
-    ``GET /user/repos`` (used by an earlier version whenever a token was
-    supplied) lists every repository the token owner can see, which includes
-    repositories of the organisations the token owner is a member of, and it
-    ignores ``username`` entirely.  Listing must therefore be scoped to the
-    requested account:
-
-    * organisation -> ``/orgs/{org}/repos?type=all``; only repositories owned
-      by that organisation, including private ones the token may access
-    * own user account -> ``/user/repos?affiliation=owner``; keeps private
-      repos, drops organisation/collaborator repos
-    * any other user account -> ``/users/{username}/repos?type=owner``
-    """
-    quoted = quote(username, safe="")
-    if (account_type or "").lower() == "organization":
-        return "/orgs/{}/repos?type=all&per_page={}&page={}".format(
-            quoted, PER_PAGE, page
-        )
-    own_account = bool(authenticated_login) and (
-        authenticated_login.lower() == username.lower()
-    )
-    if own_account:
-        return "/user/repos?affiliation=owner&per_page={}&page={}".format(
-            PER_PAGE, page
-        )
-    return "/users/{}/repos?type=owner&per_page={}&page={}".format(
-        quoted, PER_PAGE, page
-    )
-
-
-def _is_owned_by(repo, username):
-    owner = repo.get("owner") or {}
-    login = owner.get("login") or ""
-    if login:
-        return login.lower() == username.lower()
-    full_name = repo.get("full_name") or ""
-    return full_name.split("/", 1)[0].lower() == username.lower()
-
-
-def _fetch_repos_page(username, page, token="", authenticated_login=None,
-                      account_type=None):
-    """Fetch one raw page of repositories owned by ``username``."""
-    path = _repos_path(username, page, authenticated_login, account_type)
-    repos = _api_get(path, token)
-    if not isinstance(repos, list):
-        return []
-    return [repo for repo in repos if isinstance(repo, dict)]
-
-
-def get_page_of_repos(username, page, token="", authenticated_login=None,
-                      account_type=None):
-    username = username.strip()
-    if account_type is None:
-        account_type = get_account_type(username, token)
-    if authenticated_login is None:
-        authenticated_login = get_authenticated_login(token)
-    repos = _fetch_repos_page(
-        username, page, token, authenticated_login, account_type
-    )
-    return [repo for repo in repos if _is_owned_by(repo, username)]
-
-
-def search_user_repos(username, token=""):
-    username = username.strip()
-    authenticated_login = get_authenticated_login(token)
-    account_type = get_account_type(username, token)
-    page = 0
-    res_repos = []
-
-    while True:
-        page += 1
-        repos = _fetch_repos_page(
-            username, page, token, authenticated_login, account_type
-        )
-        if len(repos) == 0:
-            break
-        res_repos += repos
-        if len(repos) < PER_PAGE:
-            break
-
-    # Never return repositories that belong to somebody else (e.g. a user repo
-    # returned for an organisation, or an organisation the token owner is in).
-    return [repo for repo in res_repos if _is_owned_by(repo, username)]
-
-
-# Backward-compatible alias for the historical typo
-serach_user_repos = search_user_repos