pkgmap-py 0.1.0__tar.gz

pkgmap_py-0.1.0/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.3
+Name: pkgmap-py
+Version: 0.1.0
+Summary: Cross-distribution Linux package name resolution
+Author: Levizor
+Author-email: Levizor <levizor@disroot.org>
+License: MIT
+Requires-Dist: typer>=0.12.0
+Requires-Dist: zstandard>=0.22.0
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/Levizor/pkgmap
+Project-URL: Repository, https://github.com/Levizor/pkgmap
+Project-URL: Issues, https://github.com/Levizor/pkgmap/issues
+Description-Content-Type: text/markdown
+
+# pkgmap
+
+Cross-distribution Linux package name resolution.
+
+Different distributions often package the exact same software under different names (for example, `ninja` on Arch is `ninja-build` on Debian and Alpine; `bat` on Arch is `rust-bat` on Debian and Fedora; `fd` on Arch is `rust-fd-find` on Debian; Nixpkgs uses specific attribute paths). `pkgmap` provides an offline SQLite database and a companion Python library/CLI to translate package names between any pair of Linux distributions, resolve packages across all repositories, or look up native packages by upstream project name.
+
+Data is updated daily from [Repology](https://repology.org) and indexed for instant local lookups. On first run, `pkgmap` automatically downloads and decompresses the latest daily database release (`~/.cache/pkgmap/pkgmap.db`).
+
+---
+
+## CLI Usage
+
+### Translate between distributions
+
+Use `--from` (`-f`) and `--to` (`-t`):
+
+```bash
+# What is Arch's 'bat' called in Debian?
+pkgmap resolve bat --from arch --to debian
+# Output: rust-bat (0.22.1) [category: misc]
+
+# What is Arch's 'fd' in Nixpkgs?
+pkgmap resolve fd -f arch -t nixpkgs
+# Output: fd (10.5.0)
+```
+
+### Resolve across all distributions
+
+Omit `--to` to see equivalents across every indexed distribution:
+
+```bash
+pkgmap resolve bat --from arch
+```
+
+Output:
+```text
+alpine_3_20: bat (0.24.0)
+debian_12: rust-bat (0.22.1) [category: misc]
+fedora_rawhide: rust-bat (0.26.1)
+nix_unstable: bat (0.26.1)
+ubuntu_24_04: rust-bat (0.24.0)
+...
+```
+
+### Look up by upstream project name
+
+Omit `--from` (or pass `--project` / `-p`) to resolve packages directly from an upstream project name:
+
+```bash
+pkgmap resolve ripgrep --to debian
+# Output: rust-ripgrep (13.0.0) [category: misc]
+```
+
+### JSON output
+
+To output machine-readable JSON (great for automation and scripts), add `--json` / `-j`:
+
+```bash
+pkgmap resolve bat -f arch -t debian --json
+```
+
+```json
+[
+  {
+    "name": "rust-bat",
+    "version": "0.22.1",
+    "category": "misc",
+    "repo": "debian_12",
+    "repo_family": "debuntu",
+    "install_name": "rust-bat"
+  }
+]
+```
+
+### Other commands
+
+```bash
+# List available distributions and repository slugs
+pkgmap repos
+
+# Manually refresh the local database snapshot
+pkgmap update
+```
+
+---
+
+## Python API
+
+You can also use `pkgmap` as a Python library:
+
+```python
+from pkgmap import resolve, resolve_all, resolve_batch
+
+# 1. Single package resolution
+res = resolve("bat", from_repo="arch", to_repo="debian")
+if res:
+    print(res.install_name, res.version)
+    # Output: rust-bat 0.22.1
+
+# 2. Query all repositories for equivalents
+matches = resolve_all("bat", from_repo="arch")
+for r in matches:
+    print(f"{r.repo}: {r.install_name}")
+
+# 3. High-performance batch resolution (executed in a single SQL query)
+packages = ["bat", "fd", "ripgrep"]
+results = resolve_batch(packages, from_repo="arch", to_repo="nixpkgs")
+
+for pkg, match in results.items():
+    if match:
+        print(f"{pkg} -> {match.install_name}")
+```
+
+### API Methods
+
+- `resolve(package, from_repo=None, to_repo=None, db_path=None)`: Resolves a package to a single match.
+- `resolve_all(package, from_repo=None, to_repo=None, db_path=None)`: Returns all matching packages.
+- `resolve_batch(packages, from_repo=None, to_repo=None, db_path=None)`: Resolves a sequence of packages in a single batch query, returning `{package: PackageResolution | None}`.
+- `resolve_batch_all(packages, from_repo=None, to_repo=None, db_path=None)`: Batch resolution returning `{package: list[PackageResolution]}`.
+- `list_repositories(db_path=None)`: Returns all repositories with package counts and families.
+
+---
+
+## Configuration
+
+By default, the database is stored at:
+- `~/.cache/pkgmap/pkgmap.db`
+
+You can customize the location using:
+- The `PKGMAP_DB_PATH` environment variable:
+  ```bash
+  export PKGMAP_DB_PATH="/path/to/custom/pkgmap.db"
+  ```
+- Or the `--db-path` argument on the CLI / `db_path` parameter in Python.
+
+---
+
+## Attribution
+
+Data is sourced daily from [Repology](https://repology.org). Repology's data is published under the [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) license.

pkgmap_py-0.1.0/README.md

@@ -0,0 +1,140 @@
+# pkgmap
+
+Cross-distribution Linux package name resolution.
+
+Different distributions often package the exact same software under different names (for example, `ninja` on Arch is `ninja-build` on Debian and Alpine; `bat` on Arch is `rust-bat` on Debian and Fedora; `fd` on Arch is `rust-fd-find` on Debian; Nixpkgs uses specific attribute paths). `pkgmap` provides an offline SQLite database and a companion Python library/CLI to translate package names between any pair of Linux distributions, resolve packages across all repositories, or look up native packages by upstream project name.
+
+Data is updated daily from [Repology](https://repology.org) and indexed for instant local lookups. On first run, `pkgmap` automatically downloads and decompresses the latest daily database release (`~/.cache/pkgmap/pkgmap.db`).
+
+---
+
+## CLI Usage
+
+### Translate between distributions
+
+Use `--from` (`-f`) and `--to` (`-t`):
+
+```bash
+# What is Arch's 'bat' called in Debian?
+pkgmap resolve bat --from arch --to debian
+# Output: rust-bat (0.22.1) [category: misc]
+
+# What is Arch's 'fd' in Nixpkgs?
+pkgmap resolve fd -f arch -t nixpkgs
+# Output: fd (10.5.0)
+```
+
+### Resolve across all distributions
+
+Omit `--to` to see equivalents across every indexed distribution:
+
+```bash
+pkgmap resolve bat --from arch
+```
+
+Output:
+```text
+alpine_3_20: bat (0.24.0)
+debian_12: rust-bat (0.22.1) [category: misc]
+fedora_rawhide: rust-bat (0.26.1)
+nix_unstable: bat (0.26.1)
+ubuntu_24_04: rust-bat (0.24.0)
+...
+```
+
+### Look up by upstream project name
+
+Omit `--from` (or pass `--project` / `-p`) to resolve packages directly from an upstream project name:
+
+```bash
+pkgmap resolve ripgrep --to debian
+# Output: rust-ripgrep (13.0.0) [category: misc]
+```
+
+### JSON output
+
+To output machine-readable JSON (great for automation and scripts), add `--json` / `-j`:
+
+```bash
+pkgmap resolve bat -f arch -t debian --json
+```
+
+```json
+[
+  {
+    "name": "rust-bat",
+    "version": "0.22.1",
+    "category": "misc",
+    "repo": "debian_12",
+    "repo_family": "debuntu",
+    "install_name": "rust-bat"
+  }
+]
+```
+
+### Other commands
+
+```bash
+# List available distributions and repository slugs
+pkgmap repos
+
+# Manually refresh the local database snapshot
+pkgmap update
+```
+
+---
+
+## Python API
+
+You can also use `pkgmap` as a Python library:
+
+```python
+from pkgmap import resolve, resolve_all, resolve_batch
+
+# 1. Single package resolution
+res = resolve("bat", from_repo="arch", to_repo="debian")
+if res:
+    print(res.install_name, res.version)
+    # Output: rust-bat 0.22.1
+
+# 2. Query all repositories for equivalents
+matches = resolve_all("bat", from_repo="arch")
+for r in matches:
+    print(f"{r.repo}: {r.install_name}")
+
+# 3. High-performance batch resolution (executed in a single SQL query)
+packages = ["bat", "fd", "ripgrep"]
+results = resolve_batch(packages, from_repo="arch", to_repo="nixpkgs")
+
+for pkg, match in results.items():
+    if match:
+        print(f"{pkg} -> {match.install_name}")
+```
+
+### API Methods
+
+- `resolve(package, from_repo=None, to_repo=None, db_path=None)`: Resolves a package to a single match.
+- `resolve_all(package, from_repo=None, to_repo=None, db_path=None)`: Returns all matching packages.
+- `resolve_batch(packages, from_repo=None, to_repo=None, db_path=None)`: Resolves a sequence of packages in a single batch query, returning `{package: PackageResolution | None}`.
+- `resolve_batch_all(packages, from_repo=None, to_repo=None, db_path=None)`: Batch resolution returning `{package: list[PackageResolution]}`.
+- `list_repositories(db_path=None)`: Returns all repositories with package counts and families.
+
+---
+
+## Configuration
+
+By default, the database is stored at:
+- `~/.cache/pkgmap/pkgmap.db`
+
+You can customize the location using:
+- The `PKGMAP_DB_PATH` environment variable:
+  ```bash
+  export PKGMAP_DB_PATH="/path/to/custom/pkgmap.db"
+  ```
+- Or the `--db-path` argument on the CLI / `db_path` parameter in Python.
+
+---
+
+## Attribution
+
+Data is sourced daily from [Repology](https://repology.org). Repology's data is published under the [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) license.

pkgmap_py-0.1.0/pyproject.toml

@@ -0,0 +1,56 @@
+[project]
+name = "pkgmap-py"
+version = "0.1.0"
+description = "Cross-distribution Linux package name resolution"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "typer>=0.12.0",
+    "zstandard>=0.22.0",
+]
+
+[[project.authors]]
+name = "Levizor"
+email = "levizor@disroot.org"
+
+[project.license]
+text = "MIT"
+
+[project.urls]
+Homepage = "https://github.com/Levizor/pkgmap"
+Repository = "https://github.com/Levizor/pkgmap"
+Issues = "https://github.com/Levizor/pkgmap/issues"
+
+[project.scripts]
+pkgmap = "pkgmap.apps:cli_app"
+pkgmap-pipeline = "pkgmap.apps:pipeline_app"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0.0",
+    "ruff>=0.4.0",
+    "pre-commit",
+]
+
+[build-system]
+requires = ["uv_build>=0.9.0,<0.10.0"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+module-name = "pkgmap"
+
+[tool.ruff.lint]
+select = [
+    "E4",
+    "E7",
+    "E9",
+    "F",
+    "I",
+    "B",
+]
+
+[tool.ruff.lint.isort]
+combine-as-imports = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

pkgmap_py-0.1.0/pyproject.toml.orig

@@ -0,0 +1,44 @@
+[project]
+name = "pkgmap-py"
+version = "0.1.0"
+description = "Cross-distribution Linux package name resolution"
+authors = [{ name = "Levizor", email = "levizor@disroot.org" }]
+license = { text = "MIT" }
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "typer>=0.12.0",
+    "zstandard>=0.22.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/Levizor/pkgmap"
+Repository = "https://github.com/Levizor/pkgmap"
+Issues = "https://github.com/Levizor/pkgmap/issues"
+
+[project.scripts]
+pkgmap = "pkgmap.apps:cli_app"
+pkgmap-pipeline = "pkgmap.apps:pipeline_app"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0.0",
+    "ruff>=0.4.0",
+    "pre-commit",
+]
+
+[build-system]
+requires = ["uv_build>=0.9.0,<0.10.0"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+module-name = "pkgmap"
+
+[tool.ruff.lint]
+select = ["E4", "E7", "E9", "F", "I", "B"]
+
+[tool.ruff.lint.isort]
+combine-as-imports = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

pkgmap_py-0.1.0/src/pkgmap/__init__.py

@@ -0,0 +1,26 @@
+"""Cross-distribution Linux package name resolution library."""
+
+from pkgmap.db import ensure_database, get_database_path, update_database
+from pkgmap.resolver import (
+    PackageResolution,
+    list_repositories,
+    resolve,
+    resolve_all,
+    resolve_batch,
+    resolve_batch_all,
+    resolve_project,
+)
+
+__version__ = "0.1.0"
+__all__ = [
+    "PackageResolution",
+    "ensure_database",
+    "get_database_path",
+    "list_repositories",
+    "resolve",
+    "resolve_all",
+    "resolve_batch",
+    "resolve_batch_all",
+    "resolve_project",
+    "update_database",
+]

pkgmap_py-0.1.0/src/pkgmap/apps.py

@@ -0,0 +1,83 @@
+from pathlib import Path
+from typing import Annotated
+
+import typer
+
+from pkgmap import pipeline
+from pkgmap.cli import cli_app
+from pkgmap.pipeline import download_dump_file, set_github_output
+
+pipeline_app = typer.Typer(
+    name="pkgmap-pipeline",
+    help="Download dump, transform, and build package map database",
+    add_completion=False,
+)
+
+
+@pipeline_app.command(name="download")
+def pipeline_download(
+    link: Annotated[
+        str | None,
+        typer.Option("--link", "-l", help="Dump link to download from"),
+    ] = None,
+    dest_dir: Annotated[
+        Path | None,
+        typer.Option("--dest-dir", "-d", help="Directory where to save the dump file"),
+    ] = None,
+) -> None:
+    """Download the latest Repology database dump."""
+    dump_file, dump_date = download_dump_file(dump_link=link, dest_dir=dest_dir)
+    set_github_output(
+        date=dump_date,
+        tag=f"db-{dump_date}",
+        dump_path=str(dump_file.resolve()),
+    )
+    typer.echo(f"Downloaded Repology dump ({dump_date}) to {dump_file}")
+
+
+@pipeline_app.command(name="build")
+def pipeline_build(
+    input_dump: Annotated[
+        Path | None,
+        typer.Option(
+            "--input-dump",
+            "-i",
+            help="Dump file to take as an input to the processing pipeline",
+        ),
+    ] = None,
+    output_file: Annotated[
+        Path,
+        typer.Option(
+            "-o",
+            "--output-path",
+            help="Path for the output SQLite database",
+        ),
+    ] = Path("pkgmap.db"),
+    batch_size: Annotated[
+        int,
+        typer.Option(
+            "--batch-size",
+            help="Number of records per batch in the processing pipeline",
+        ),
+    ] = 50_000,
+    release_notes: Annotated[
+        Path | None,
+        typer.Option("--release-notes", help="Path to generate release notes markdown"),
+    ] = None,
+    dump_date: Annotated[
+        str | None,
+        typer.Option("--dump-date", help="Snapshot date (YYYY-MM-DD)"),
+    ] = None,
+) -> None:
+    """Process Repology dump and generate SQLite package map database."""
+    pipeline.main(
+        dump_file=input_dump,
+        output_file=output_file,
+        batch_size=batch_size,
+        release_notes=release_notes,
+        dump_date=dump_date,
+    )
+
+
+if __name__ == "__main__":
+    cli_app()

pkgmap_py-0.1.0/src/pkgmap/cli.py

@@ -0,0 +1,171 @@
+import json
+from pathlib import Path
+from typing import Annotated
+
+import typer
+
+from pkgmap.db import get_database_path, update_database
+from pkgmap.resolver import list_repositories, resolve_all, resolve_project
+
+cli_app = typer.Typer(
+    name="pkgmap",
+    help="Cross-distribution Linux package name resolution CLI",
+    add_completion=False,
+)
+
+
+@cli_app.callback()
+def cli_callback(
+    verbose: Annotated[
+        bool,
+        typer.Option("--verbose", "-v", help="Enable verbose debug logging"),
+    ] = False,
+) -> None:
+    if verbose:
+        import logging
+
+        for name in ("pkgmap", "pkgmap-db", "pkgmap-downloader", "pkgmap-consumer"):
+            log = logging.getLogger(name)
+            log.setLevel(logging.DEBUG)
+            for handler in log.handlers:
+                handler.setLevel(logging.DEBUG)
+
+
+@cli_app.command(name="resolve")
+def resolve_cmd(
+    package: Annotated[
+        str, typer.Argument(help="Package or upstream project name to resolve")
+    ],
+    to_repo: Annotated[
+        str | None,
+        typer.Option(
+            "--to",
+            "-t",
+            help="Target repository/distribution (e.g. nixpkgs, alpine). If omitted, matches across all repositories.",
+        ),
+    ] = None,
+    from_repo: Annotated[
+        str | None,
+        typer.Option(
+            "--from",
+            "-f",
+            help="Source repository/distribution (e.g. debian, arch). If omitted, package is resolved by upstream project name.",
+        ),
+    ] = None,
+    project: Annotated[
+        bool,
+        typer.Option(
+            "--project",
+            "-p",
+            help="Resolve package directly as an upstream project name",
+        ),
+    ] = False,
+    all_matches: Annotated[
+        bool,
+        typer.Option("--all", "-a", help="Show all matching target packages"),
+    ] = False,
+    json_output: Annotated[
+        bool,
+        typer.Option("--json", "-j", help="Output results in JSON format"),
+    ] = False,
+    db_path: Annotated[
+        Path | None,
+        typer.Option("--db-path", help="Custom path to pkgmap.db database"),
+    ] = None,
+) -> None:
+    """Resolve a package name across Linux distributions."""
+    if project and from_repo:
+        typer.secho(
+            "Error: Cannot specify both --from and --project.",
+            fg=typer.colors.RED,
+        )
+        raise typer.Exit(code=2)
+
+    if from_repo and not project:
+        results = resolve_all(
+            package, from_repo=from_repo, to_repo=to_repo, db_path=db_path
+        )
+    else:
+        results = resolve_project(package, to_repo=to_repo, db_path=db_path)
+
+    if not results:
+        if json_output:
+            typer.echo(json.dumps([]))
+        else:
+            if from_repo and not project:
+                if to_repo:
+                    msg = f"No equivalent package found for '{package}' from '{from_repo}' to '{to_repo}'."
+                else:
+                    msg = f"No equivalent package found for '{package}' from '{from_repo}'."
+            else:
+                if to_repo:
+                    msg = f"No equivalent package found for project '{package}' in '{to_repo}'."
+                else:
+                    msg = f"No equivalent package found for project '{package}'."
+            typer.secho(msg, fg=typer.colors.YELLOW)
+        raise typer.Exit(code=1)
+
+    items_to_show = results if (all_matches or to_repo is None) else results[:1]
+
+    if json_output:
+        data = [
+            {
+                "name": r.name,
+                "version": r.version,
+                "category": r.category,
+                "repo": r.repo,
+                "repo_family": r.repo_family,
+                "install_name": r.install_name,
+            }
+            for r in items_to_show
+        ]
+        typer.echo(json.dumps(data, indent=2))
+        return
+
+    for res in items_to_show:
+        version_str = f" ({res.version})" if res.version else ""
+        meta_parts = []
+        if res.category and res.repo_family != "nix":
+            meta_parts.append(f"category: {res.category}")
+        meta_str = f" [{', '.join(meta_parts)}]" if meta_parts else ""
+        repo_prefix = f"{res.repo}: " if to_repo is None else ""
+        typer.echo(f"{repo_prefix}{res.install_name}{version_str}{meta_str}")
+
+
+@cli_app.command(name="update")
+def update_cmd(
+    db_path: Annotated[
+        Path | None,
+        typer.Option("--db-path", help="Custom path to save pkgmap.db database"),
+    ] = None,
+) -> None:
+    """Explicitly download and update the local package map database."""
+    target_path = get_database_path(db_path)
+    typer.echo(f"Updating package database at {target_path}...")
+    update_database(db_path=target_path, show_progress=True)
+    typer.secho(
+        f"Successfully updated database at {target_path}", fg=typer.colors.GREEN
+    )
+
+
+@cli_app.command(name="repos")
+def repos_cmd(
+    db_path: Annotated[
+        Path | None,
+        typer.Option("--db-path", help="Custom path to pkgmap.db database"),
+    ] = None,
+) -> None:
+    """List all available distributions and repositories."""
+    repos = list_repositories(db_path=db_path)
+    typer.echo(f"{'Repository':<25} {'Family':<15} {'Packages':>10}")
+    typer.echo("-" * 52)
+    for r in sorted(repos, key=lambda x: x.name):
+        typer.echo(f"{r.name:<25} {r.family:<15} {r.package_count:>10,}")
+
+
+def main() -> None:
+    cli_app()
+
+
+if __name__ == "__main__":
+    main()