als-finder 1.0.1__tar.gz

als_finder-1.0.1/.agents/CONVERSATION_DUMP_Phase2.md

@@ -0,0 +1,21 @@
+# Antigravity State Dump: Phase 2 Execution Completion
+
+## Core Summary
+Completed the fundamental integration of **OpenTopography** handling alongside deep CLI parameter unification and Dry-Run `download` payload extraction tracking. The system is structurally completely ready to begin the **PDAL Normalization Pipeline (Phase 3)** smoothly bridging arrays globally.
+
+## Finalized Matrices
+1. **Double Date Drop Bug**: Fixed `cli.py` dropping generic temporal subset boundaries erroneously by moving `sort_date` tracking definitions directly prior to logical `<` and `>` exclusions natively cleanly opening subset extraction structurally `/2020-01-01`.
+2. **Tabular Size Render**: Re-coded `download.py` transforming strict iterative text output logs into beautiful explicit ASCII reporting geometries tracking `Tiles`, `True Size`, and native `Format` physically aggregating into formal estimates prior to active physical retrieval.
+3. **HTTP HEAD Parsing**: Intercepted OpenTopography Mode B bounding geometries naturally routing all indices natively through concurrent `ThreadPoolExecutor` abstractions probing the S3 `requests.head` to calculate literal sizes matching standard `Mode A` natively!
+4. **README Consistency**: Rewrote parameter sequences tracking CLI matrices enforcing identical arrays globally (`--roi`, `--name`, `--workspace`) replacing literal texts seamlessly. Cleanly explicitly scoped target Point Density `--density 2/10`. Expanded "federal registries" clause structurally to logically encompass generic "public multi-origin LiDAR repositories".
+5. **Distribution Infrastructure**: Established cross-platform robust package delivery explicitly.
+   - **PyPI**: Integrated automated Trusted Publishing via `.github/workflows/pypi-publish.yml` dynamically bridging GitHub Releases to PyPI directly natively.
+   - **Conda-Forge**: Finalized PR `#33018` for `als-finder-feedstock` via `staged-recipes`, comprehensively navigating pure Python `noarch` cross-platform edge-cases (implementing native `build.sh` and `bld.bat` explicitly fixing Windows `cmd.exe` limitations executing inline Bash environment variables resolving `SETUPTOOLS_SCM_PRETEND_VERSION`).
+## Current Project Space
+- `git` was fully synchronized and pushed cleanly under `feat: robust tabular payload tracking on dry-runs natively & OT HTTP HEAD evaluations`.
+- `scratch/` workspace was completely expunged securely.
+- **GitHub Issue**: Sourced a tracking PR explicitly bounding the dry-run CLI additions under Milestone `v1.1 (Big-Data Architecture)`.
+
+## Next Objective (Phase 3)
+1. Read the newly tracked parameters and verify `src/als_finder/` structures.
+2. Formally initiate native Phase 3 protocols tracking `.laz` normalization integrations gracefully (handling CRS Projections natively leveraging PDAL pipelines). We must natively map formats globally resolving variable geometries!

als_finder-1.0.1/.agents/rules.md

@@ -0,0 +1,123 @@
+---
+description: Documentation Maintenance Rules
+---
+
+# Documentation Rules
+
+## 1. Sync Logic
+-   **Code ↔ Docs**: If you change the code behavior (arguments, features, outputs), you MUST update the corresponding documentation immediately.
+-   **PLAN.md**: This is a living document. Mark tasks as `[x]` when completed. Add new constraints or steps as they are discovered.
+
+## 2. Required Documentation
+-   **README.md**: Must contain:
+    -   Project Title & Description
+    -   Installation Instructions (Docker & Local)
+    -   Basic Usage Examples
+    -   Current Status/badges
+-   **docs/**:
+    -   `PLAN.md`: Implementation roadmap.
+    -   `API.md`: (Future) Auto-generated or manual API reference.
+    -   `Architecture.md`: (Optional) Diagram/text explaining the provider -> download flow.
+
+## 3. Docstrings
+-   Code is not "done" until it is documented.
+-   See `python_standards.md` for format.
+---
+description: Rules for Version Control and Checkpointing
+---
+
+# Git & Checkpointing Rules
+
+## 1. Commit Granularity
+-   **Atomic Commits**: Commit one logical change at a time. Do not bundle refactors with feature additions.
+-   **Frequency**: "Checkpoint" (commit) often. Ideally, after every successful step or file creation/modification that reaches a stable state.
+-   **Safety**: Always commit before attempting a risky operation (e.g., large refactor, file deletion).
+
+## 2. Commit Message Convention
+Follow the Conventional Commits specification:
+`type(scope): description`
+
+-   **Types**:
+    -   `feat`: A new feature
+    -   `fix`: A bug fix
+    -   `docs`: Documentation only changes
+    -   `style`: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
+    -   `refactor`: A code change that neither fixes a bug nor adds a feature
+    -   `perf`: A code change that improves performance
+    -   `test`: Adding missing tests or correcting existing tests
+    -   `chore`: Changes to the build process or auxiliary tools and libraries
+    -   `wip`: Work in progress (use sparingly, amend later if possible)
+
+-   **Example**:
+    -   `feat(opentopo): implement search function for global datasets`
+    -   `docs(readme): update installation instructions`
+    -   `chore(deps): add pdal to requirements`
+
+## 3. Branching (If applicable)
+-   Main development happens on the current branch unless instructed otherwise.
+-   If experimenting, creating a temporary branch is encouraged.
+---
+description: Python Development Guidelines and Standards
+---
+
+# Python Development Rules
+
+## 1. Code Style & Formatting
+-   **Formatter**: Use `black` for code formatting.
+-   **Linter**: Use `ruff` or `flake8` for linting.
+-   **Imports**: Sort imports using `isort` (or `ruff`'s import sorting).
+-   **Line Length**: 88 characters (Black default).
+
+## 2. Type Hinting
+-   **Mandatory**: All function signatures (arguments and return types) MUST have type hints.
+-   **Static Analysis**: Code must pass `mypy` strict mode checks.
+-   **Variables**: Type hint complex variables where inference is ambiguous.
+
+## 3. Documentation
+-   **Docstrings**: All modules, classes, and public functions must have Google-style docstrings.
+    -   *Example*:
+        ```python
+        def fetch_data(roi: dict) -> pd.DataFrame:
+            """Fetches data for a given region.
+
+            Args:
+                roi (dict): The region of interest geometry.
+
+            Returns:
+                pd.DataFrame: Metadata of found datasets.
+            """
+        ```
+-   **Comments**: Use comments to explain *why*, not *what*.
+
+## 4. Testing
+-   **Framework**: Use `pytest`.
+-   **Coverage**: Aim for high test coverage, particularly for core logic in `providers/`.
+-   **Fixtures**: Use `conftest.py` for shared fixtures (e.g., sample ROIs).
+
+## 5. File Handling
+-   **Pathlib**: ALWAYS use `pathlib.Path` instead of `os.path` for file manipulation.
+-   **Context Managers**: Use `with` statements for opening files.
+---
+description: Project Directory Structure and Organization
+---
+
+# Project Structure
+
+## Top Level
+-   `src/`: **Source Code**. All application logic resides here.
+    -   `als_finder/`: The main package.
+-   `docs/`: **Documentation**. Plans, API docs, architecture notes.
+-   `.agent/`: **Agent Configuration**. Rules and workflows for the AI assistant.
+-   `scratch/`: **Scratch Space**. (Gitignored) Primary workspace for debugging scripts and local `tests/`.
+-   `data/`: **Local Data**. (Gitignored) Place to store input ROIs or output data during dev.
+-   `output/`: **Results**. (Gitignored) Default download destination.
+
+## Configuration Files
+-   `Dockerfile`: Container definition.
+-   `setup.py`: Package installation metadata.
+-   `requirements.txt`: Pinned dependencies.
+-   `.env`: (Gitignored) Secrets and keys.
+
+## Rules
+1.  **No Logic at Root**: Do not put python scripts at the top level. Use `src/` or `scripts/`.
+2.  **Clean Root**: Only standard config files (`.gitignore`, `README.md`, etc.) belong at the root.

als_finder-1.0.1/.agents/workflows/docker_build.md

@@ -0,0 +1,22 @@
+---
+description: Build and test the Docker container
+---
+
+# Docker Build Workflow
+
+## Steps
+
+1.  **Validate Requirements**
+    -   Ensure `requirements.txt` is up to date.
+    -   Ensure `Dockerfile` exists.
+
+2.  **Build Image**
+    -   Run: `docker build -t als-finder:latest .`
+
+3.  **Test Run**
+    -   Run help command to verify entrypoint:
+        `docker run --rm als-finder:latest --help`
+
+4.  **Integration Test (Optional)**
+    -   Run a small search to verify connectivity (requires API key):
+        `docker run --rm --env-file .env als-finder:latest search --roi tests/sample_roi.json`

als_finder-1.0.1/.agents/workflows/local_setup.md

@@ -0,0 +1,23 @@
+---
+description: Setup local development environment
+---
+
+# Local Development Setup
+
+## 1. Create Virtual Environment
+We use a specific venv named `als-finder-env` to avoid conflicts.
+
+```bash
+python3 -m venv als-finder-env
+source als-finder-env/bin/activate
+pip install --upgrade pip
+```
+
+## 2. Install Dependencies
+```bash
+pip install -r requirements.txt
+pip install -e .  # Install package in editable mode
+```
+
+## 3. VS Code Configuration
+Select the interpreter path: `./als-finder-env/bin/python`

als_finder-1.0.1/.agents/workflows/qa_checks.md

@@ -0,0 +1,21 @@
+---
+description: Run code quality checks and tests
+---
+
+# Test & Lint Workflow
+
+## Steps
+
+1.  **Format**
+    -   `black src/ tests/`
+
+2.  **Lint**
+    -   `ruff check src/ tests/`
+
+3.  **Type Check**
+    -   `mypy src/`
+
+4.  **Unit Tests**
+    -   `pytest tests/`
+
+// turbo-all

als_finder-1.0.1/.dockerignore

@@ -0,0 +1,14 @@
+__pycache__
+*.pyc
+*.pyo
+*.pyd
+.Python
+env/
+venv/
+.env
+.pytest_cache/
+.mypy_cache/
+.git/
+.vscode/
+output/
+data/

als_finder-1.0.1/.env.template

@@ -0,0 +1 @@
+OPENTOPOGRAPHY_API_KEY=your_api_key_here

als_finder-1.0.1/.github/workflows/docker-publish.yml

@@ -0,0 +1,49 @@
+name: Docker Build and Publish
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=raw,value=latest,enable={{is_default_branch}}
+            type=ref,event=branch
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          push: ${{ github.event_name != 'pull_request' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

als_finder-1.0.1/.github/workflows/pypi-publish.yml

@@ -0,0 +1,36 @@
+name: Upload Python Package to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  pypi-publish:
+    name: Build and publish to PyPI
+    runs-on: ubuntu-latest
+    
+    # Specifies that the job requires permission to request an OIDC token
+    permissions:
+      id-token: write
+      contents: read
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0 # Required for setuptools_scm to get git tags
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install build tools
+        run: python -m pip install --upgrade pip build
+
+      - name: Build sdist and wheel
+        run: python -m build
+
+      - name: Publish package to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        # No password or username needed. It uses Trusted Publishing (OIDC).

als_finder-1.0.1/.gitignore

@@ -0,0 +1,42 @@
+# Python
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+.Python
+env/
+venv/
+.venv/
+als-finder-env/
+.env
+
+# Build Artifacts
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# Testing
+.pytest_cache/
+.mypy_cache/
+.tox/
+.coverage
+htmlcov/
+
+# Local Workspaces
+scratch/
+data/
+output/
+test_workspace/

als_finder-1.0.1/ARCHITECTURE.md

@@ -0,0 +1,29 @@
+# ALS-Finder Architecture Roadmap
+
+**Purpose**: An API extraction engine designed to search, deduplicate, and securely download high-resolution LiDAR Point clouds (`.laz`) natively across isolated federal boundaries. Acts as the geospatial ingestion stage for the downstream `als-downscaler` HPC orchestration pipeline.
+
+***
+
+## Workflow A: Geospatial Search (`cli.py search`)
+- **Inputs**: 
+  - Local GeoJSON Region of Interest boundaries (`--roi`).
+  - Explicit permitted query providers (USGS STAC, OpenTopography, NOAA).
+- **Core Execution**:
+  1. Translates the ROI limits into raw `shapely.geometry` spatial intersections.
+  2. Dispatches asynchronous, federated API payload queries across the Microsoft Planetary Computer STAC (USGS 3DEP), NOAA AWS S3 Entwine index catalogs, and OpenTopography JSON-LD spatial environments.
+  3. Homogenizes resulting heterogeneous JSON responses natively into an internal python schema array.
+  4. Extracts dataset aliases to isolate and natively suppress functionally identical targets (Currently strictly bounded to literal string matching).
+- **Outputs**: 
+  - A highly structured JSON configuration manifest (`manifest.json`) securely mapping all mathematically viable `.laz` download arrays across the designated bounding box.
+
+***
+
+## Workflow B: Target Extraction (`cli.py download`)
+- **Inputs**: 
+  - The upstream generated JSON architecture manifest, or dynamic `--tile-url` injection parameters.
+- **Core Execution**:
+  1. Spools the native HTTP extraction clients to map against targeted provider logic (e.g., `USGSProvider`).
+  2. Actively probes the payload structures: if native Azure Blob URLs block access (`409 Public Access Blocked`), proactively dispatches identical authentication queries into the Microsoft Planetary `sas` overlay to construct an ephemeral **Shared Access Signature (SAS)** API bypass token.
+  3. Instantiates parallel or successive byte stream downloads into local scratch volumes.
+- **Outputs**: 
+  - Localized, uncompressed `.copc.laz` or legacy point cloud layers structured rigidly onto the `$SCRATCH` partitions for HPC intersection.

als_finder-1.0.1/Dockerfile

@@ -0,0 +1,36 @@
+# Use Micromamba to handle complex geospatial C++ dependencies (GDAL, PDAL) smoothly
+FROM mambaorg/micromamba:1.5-jammy
+
+# Set environment variables
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1
+
+# Copy requirements and install them via conda-forge
+COPY --chown=$MAMBA_USER:$MAMBA_USER requirements.txt /tmp/requirements.txt
+RUN micromamba install -y -n base -c conda-forge \
+    python=3.11 \
+    gdal \
+    pdal \
+    python-pdal \
+    --file /tmp/requirements.txt && \
+    micromamba clean --all --yes
+
+# Set the working directory in the container
+WORKDIR /app
+
+# Copy the application code
+COPY --chown=$MAMBA_USER:$MAMBA_USER src/ src/
+COPY --chown=$MAMBA_USER:$MAMBA_USER .agent/ .agent/
+COPY --chown=$MAMBA_USER:$MAMBA_USER docs/ docs/
+COPY --chown=$MAMBA_USER:$MAMBA_USER README.md .
+COPY --chown=$MAMBA_USER:$MAMBA_USER setup.py .
+
+# Install the package itself in editable mode via pip (since micromamba provides pip)
+# We use the micromamba python interpreter
+ARG MAMBA_DOCKERFILE_ACTIVATE=1
+ENV SETUPTOOLS_SCM_PRETEND_VERSION=0.1.0
+RUN pip install --no-cache-dir -e .
+
+# Define the entrypoint
+ENTRYPOINT ["/usr/local/bin/_entrypoint.sh", "python", "-m", "als_finder.cli"]
+CMD ["--help"]

als_finder-1.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jonathan Greenberg / University of Nevada, Reno
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

als_finder-1.0.1/MANIFEST.in

@@ -0,0 +1,4 @@
+include README.md
+include LICENSE
+recursive-include examples *
+recursive-include src/als_finder/data *