aquascope 0.4.0__tar.gz

aquascope-0.4.0/.github/FUNDING.yml

@@ -0,0 +1,14 @@
+# Supported funding platforms — see https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/displaying-a-sponsor-button-in-your-repository
+#
+# Uncomment + fill in the lines for any platform you want the "Sponsor" button to link to.
+# Multiple platforms can be enabled at the same time.
+
+github: [Rekin226]              # Apply at https://github.com/sponsors — button stays hidden until approved
+ko_fi: getaquascope             # ko-fi.com/getaquascope
+# buy_me_a_coffee: Rekin226    # buymeacoffee.com/<username>
+# open_collective: aquascope   # opencollective.com/<slug> — best for multi-maintainer transparency
+# patreon: Rekin226            # patreon.com/<username>
+# tidelift: pypi/aquascope     # for OSS maintainers via Tidelift Subscription
+# liberapay: Rekin226          # liberapay.com/<username>
+# polar: Rekin226              # polar.sh/<username> — pay for issues/features
+# custom: ['https://your-lab-page.example']   # any HTTPS URL (university lab, PayPal.me, Stripe, etc.)

aquascope-0.4.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,31 @@
+---
+name: Bug Report
+about: Report a problem with AquaScope
+title: "[BUG] "
+labels: bug
+assignees: ""
+---
+
+## Description
+A clear description of the bug.
+
+## Steps to Reproduce
+1. ...
+2. ...
+
+## Expected Behaviour
+
+
+## Actual Behaviour
+
+
+## Environment
+- OS: 
+- Python version: 
+- AquaScope version: 
+- Collector/source involved: 
+
+## Logs / Traceback
+```
+paste here
+```

aquascope-0.4.0/.github/ISSUE_TEMPLATE/new_data_source.md

@@ -0,0 +1,39 @@
+---
+name: New Data Source Request
+about: Propose a new water data API or dataset to integrate
+title: "[DATA SOURCE] "
+labels: enhancement, data-source
+assignees: ""
+---
+
+## Data Source Name
+
+
+## API / Download URL
+
+
+## Data Description
+What kind of water data does it provide? (water quality, hydrology, reservoir, groundwater, etc.)
+
+## Geographic Coverage
+(e.g., Japan, Europe, Global)
+
+## Update Frequency
+(e.g., real-time, daily, monthly, annual)
+
+## API Documentation Link
+
+
+## Authentication Required?
+- [ ] No (fully open)
+- [ ] Yes — free API key
+- [ ] Yes — institutional access
+
+## Sample Response
+```json
+paste a sample here
+```
+
+## Would you like to implement this collector?
+- [ ] Yes, I can submit a PR
+- [ ] No, I am requesting it

aquascope-0.4.0/.github/ISSUE_TEMPLATE/new_methodology.md

@@ -0,0 +1,35 @@
+---
+name: New Research Methodology
+about: Propose a new methodology to add to the AI recommender knowledge base
+title: "[METHODOLOGY] "
+labels: enhancement, methodology
+assignees: ""
+---
+
+## Methodology Name
+
+
+## Category
+(statistical / machine_learning / process_engineering / remote_sensing / hydrological_modelling / policy_analysis)
+
+## Description
+A concise description of the methodology.
+
+## Applicable Water Quality Parameters
+(e.g., DO, BOD5, COD, NH3-N, pH, Turbidity)
+
+## Data Requirements
+What kind of data does this method need?
+
+## Typical Scale
+(lab / pilot / field / regional / global)
+
+## Complexity
+(low / medium / high)
+
+## Key References
+1. 
+2. 
+
+## Tags
+(e.g., time-series, forecasting, deep-learning, wastewater)

aquascope-0.4.0/.github/copilot-instructions.md

@@ -0,0 +1,67 @@
+# Copilot Instructions for AquaScope
+
+AquaScope is a water data aggregation toolkit with AI-powered research methodology recommendations. It collects water-quality and hydrology data from 8 global sources, normalises them into unified Pydantic schemas, and uses an AI engine to recommend and auto-execute research methodologies.
+
+## Build, Test, and Lint
+
+```bash
+# Install with all optional dependencies (ML, viz, LLM, dev tools)
+pip install -e ".[dev,all]"
+
+# Run all tests
+pytest
+
+# Run a single test file
+pytest tests/test_collectors/test_taiwan_moenv.py
+
+# Run a single test by name
+pytest -k "test_normalise_produces_correct_records"
+
+# Lint
+ruff check aquascope/ tests/
+
+# Type check
+mypy aquascope/ --ignore-missing-imports
+```
+
+CI runs lint and type-check on Python 3.12; tests on 3.10, 3.11, and 3.12.
+
+## Architecture
+
+The data flow is a pipeline:
+
+```
+Collectors (fetch + normalise) → Pydantic Schemas → Analysis (EDA, Quality) → AI Recommender → Pipelines (execute)
+```
+
+- **`collectors/`** — 8 source modules, each subclassing `BaseCollector`. Implement `fetch_raw()` and `normalise()` to return unified schema objects.
+- **`schemas/water_data.py`** — Pydantic models (`WaterQualitySample`, `WaterLevelReading`, `ReservoirStatus`, `SDG6Indicator`) plus a `DataSource` enum.
+- **`analysis/`** — `eda.py` for dataset profiling, `quality.py` for assessment and preprocessing.
+- **`ai_engine/`** — `knowledge_base.py` defines 26 `ResearchMethodology` entries; `recommender.py` scores them against a `DatasetProfile` (rule-based + optional LLM).
+- **`pipelines/model_builder.py`** — 7 executable pipeline functions registered in `_PIPELINE_REGISTRY`, each returning a `PipelineResult`.
+- **`utils/`** — `http_client.py` (CachedHTTPClient with retries and rate limiting), `storage.py` (save to JSON/CSV).
+- **`cli.py`** — 7 argparse commands: `collect`, `recommend`, `eda`, `quality`, `run`, `list-methods`, `list-sources`. Uses lazy imports — modules are imported inside command functions.
+
+## Key Conventions
+
+- **Python 3.10+** — uses `X | None` union syntax and `list[str]` generics.
+- **`from __future__ import annotations`** at the top of every module.
+- **Pydantic for data schemas only** (`schemas/`); internal structures use `@dataclass` (`DatasetProfile`, `Recommendation`, `PipelineResult`, `QualityReport`, `EDAReport`).
+- **Google-style docstrings** with `Parameters`, `Returns`, `Raises` sections.
+- **Full type hints** on all public functions.
+- **One logger per module**: `logger = logging.getLogger(__name__)`.
+- **Ruff** enforces style — line length is 120; rules: E, F, I, N, W, UP.
+- **Tests mirror the package structure** (`tests/test_<module>/`). No `conftest.py`; tests are self-contained using `setup_method` and inline helper functions.
+
+## Adding a New Data Collector
+
+1. Create `aquascope/collectors/<source>.py` — subclass `BaseCollector`, implement `fetch_raw()` and `normalise()`.
+2. Add an entry to the `DataSource` enum in `schemas/water_data.py`.
+3. Export the class from `aquascope/collectors/__init__.py`.
+4. Register it in `cli.py`'s `cmd_collect` function.
+5. Add tests in `tests/test_collectors/`.
+
+## Adding a New Methodology / Pipeline
+
+- **Methodology only**: Add a `ResearchMethodology` instance to `METHODOLOGIES` in `ai_engine/knowledge_base.py`.
+- **Executable pipeline**: Add a `run_<method_id>(df, config=None) -> PipelineResult` function in `pipelines/model_builder.py` and register it in `_PIPELINE_REGISTRY`.

aquascope-0.4.0/.github/workflows/ci.yml

@@ -0,0 +1,44 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - run: pip install -e ".[dev,all]"
+      - run: ruff check aquascope/ tests/
+      - name: Type check (informational — does not fail build)
+        run: mypy aquascope/ --ignore-missing-imports
+        continue-on-error: true
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - run: pip install -e ".[dev,all]"
+      - run: pytest --cov=aquascope --cov-report=term-missing --cov-report=xml
+      - name: Upload coverage artifact
+        if: matrix.python-version == '3.12'
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-report
+          path: coverage.xml

aquascope-0.4.0/.github/workflows/publish.yml

@@ -0,0 +1,24 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  id-token: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install build tools
+        run: pip install build
+      - name: Build package
+        run: python -m build
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

aquascope-0.4.0/.gitignore

@@ -0,0 +1,40 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+dist/
+build/
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# Data cache (local only)
+data/cache/
+data/raw/
+data/processed/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+
+# mypy
+.mypy_cache/
+
+# Environment
+.env
+.env.local

aquascope-0.4.0/.pre-commit-config.yaml

@@ -0,0 +1,22 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-added-large-files
+        args: ['--maxkb=500']
+      - id: check-merge-conflict
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.3.0
+    hooks:
+      - id: ruff
+        args: [--fix]
+      - id: ruff-format
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.8.0
+    hooks:
+      - id: mypy
+        additional_dependencies: [pydantic>=2.0]
+        args: [--ignore-missing-imports]

aquascope-0.4.0/CHANGELOG.md

@@ -0,0 +1,70 @@
+# Changelog
+
+All notable changes to AquaScope are documented here.
+
+## [0.4.0] — 2026-04-01
+
+### Added
+- **Groundwater module** (`aquascope/groundwater/`) — GRACE satellite data integration, well monitoring, recharge estimation, and aquifer hydraulics analysis
+- **Climate projections module** (`aquascope/climate/`) — CMIP6 scenario analysis, statistical downscaling, Palmer Drought Severity Index (PDSI), and climate impact assessment
+- **JOSS paper** — Added `paper.md` and `paper.bib` for Journal of Open Source Software submission
+- **EU Water Framework Directive collector** (in progress) — European water body status and compliance data
+- **Japan MLIT collector** (in progress) — Japanese river and water quality monitoring data
+- **Korea WAMIS collector** (in progress) — Korean water resources management information
+- **15 data source collectors** total across global water monitoring networks
+- **New CLI commands**: `groundwater`, `climate` for the new modules
+- **New convenience API functions** in `aquascope.api` for streamlined programmatic access
+- **Agricultural water module** (`aquascope/agri/`) — crop water demand, ET₀ calculation, water balance, productivity benchmarking, and irrigation planning
+- **Alerts module** (`aquascope/alerts/`) — threshold-based monitoring, anomaly checking, and notification system
+- **Advanced analysis** — changepoint detection, copula modelling
+- **Hydrological modelling** (`aquascope/hydrology/`) — rainfall-runoff, routing, flood frequency, baseflow separation, CAMELS benchmarking
+- **AI agent and planner** — multi-step research planning and autonomous execution
+- **685+ tests** across all modules
+
+### Changed
+- Bumped version to 0.4.0
+- Expanded optional dependency groups: `forecast`, `copernicus`, `scientific`, `dashboard`, `spatial`
+- Added Python 3.13 classifier
+- GitHub Actions publish workflow for PyPI releases via trusted publishing
+
+## [0.2.0] — 2026-03-12
+
+### Added
+- **Analysis module** — Automated EDA (`aquascope eda`) with per-parameter statistics, outlier detection (IQR), correlation matrix, and completeness scoring
+- **Data quality pipeline** — Assessment + preprocessing (`aquascope quality --fix`) with duplicate removal, imputation, outlier filtering, normalization, and daily resampling
+- **7 model pipelines** — Auto-execute research methodologies via `aquascope run`:
+  - Mann-Kendall trend analysis
+  - Taiwan River Pollution Index (RPI)
+  - PCA + K-Means clustering
+  - Random Forest classification
+  - XGBoost regression
+  - ARIMA time-series forecasting
+  - Pearson correlation analysis
+- **3 new data collectors**:
+  - GEMStat (UNEP global freshwater quality via Zenodo)
+  - Taiwan Civil IoT (real-time SensorThings API)
+  - US Water Quality Portal (USGS + EPA + 400 agencies)
+- **13 new research methodologies** in the knowledge base (26 total), including: ARIMA forecasting, Transformer-based prediction, SWMM/QUAL2K process models, kriging spatial interpolation, isotope hydrology, paired watershed design, and more
+- **5 new CLI commands**: `eda`, `quality`, `run`, `list-methods`, `list-sources`
+- **Documentation guides**: Architecture, Adding a Data Source, Adding a Methodology, Running Pipelines
+- **Jupyter quickstart tutorial** (`notebooks/01_quickstart_tutorial.ipynb`)
+- **Comprehensive test suite** — 69 tests covering analysis, pipelines, collectors, AI engine
+
+### Changed
+- Bumped version to 0.2.0 (Beta status)
+- `pandas` and `numpy` are now core dependencies (not optional)
+- Updated `collect` CLI to support all 8 data sources
+- Expanded `pyproject.toml` with `viz`, `ml` optional dependency groups
+
+## [0.1.0] — 2026-03-10
+
+### Added
+- Initial release
+- 5 data collectors: Taiwan MOENV, Taiwan WRA (level + reservoir), USGS, UN SDG 6
+- Unified Pydantic schemas for water data
+- AI methodology recommender with 13 built-in methodologies
+- Rule-based scoring + optional LLM enhancement
+- CLI with `collect` and `recommend` commands
+- HTTP client with caching and rate limiting
+- 12 tests, ruff lint, GitHub Actions CI/CD
+- Contributing guide, MIT license

aquascope-0.4.0/CONTRIBUTING.md

@@ -0,0 +1,112 @@
+# Contributing to AquaScope
+
+Thank you for your interest in contributing to AquaScope! This project aims to be a community-driven resource for water researchers worldwide. Whether you are a hydrologist, environmental engineer, data scientist, or student — your contributions are welcome.
+
+## Ways to Contribute
+
+### 1. Add a New Data Source Collector
+
+We want to cover water APIs from every country. To add a new collector:
+
+1. Create a new file in `aquascope/collectors/` (e.g., `japan_mlit.py`).
+2. Subclass `BaseCollector` and implement `fetch_raw()` and `normalise()`.
+3. Map raw API fields to our unified schemas in `aquascope/schemas/water_data.py`.
+4. Add tests in `tests/test_collectors/`.
+5. Document the data source in your module docstring (API URL, required keys, datasets).
+
+```python
+from aquascope.collectors.base import BaseCollector
+from aquascope.schemas.water_data import WaterQualitySample, DataSource
+
+class JapanMLITCollector(BaseCollector):
+    name = "japan_mlit"
+
+    def fetch_raw(self, **kwargs):
+        return self.client.get_json("https://api.example.jp/water/v1/quality")
+
+    def normalise(self, raw):
+        # Convert raw records into WaterQualitySample instances
+        ...
+```
+
+### 2. Add a Research Methodology
+
+To expand the AI recommender's knowledge base:
+
+1. Open `aquascope/ai_engine/knowledge_base.py`.
+2. Add a new `ResearchMethodology` entry to the `METHODOLOGIES` list.
+3. Include: description, applicable parameters, data requirements, scale, complexity, references, and tags.
+4. Add a test in `tests/test_ai_engine/` to verify your methodology is findable.
+
+Or simply open an issue using the **New Research Methodology** template.
+
+### 3. Improve the AI Recommender
+
+The scoring algorithm in `aquascope/ai_engine/recommender.py` can be improved:
+
+- Better heuristics for data sufficiency scoring
+- Additional scoring dimensions (e.g., data frequency, parameter correlations)
+- LLM prompt improvements for the enhanced mode
+
+### 4. Add Notebooks / Tutorials
+
+Example Jupyter notebooks in `notebooks/` help new users get started. Contributions in English, French, Chinese, or any language are appreciated.
+
+### 5. Fix Bugs / Improve Docs
+
+Bug fixes and documentation improvements are always welcome.
+
+## Development Setup
+
+```bash
+# Clone the repo
+git clone https://github.com/Rekin226/aquascope.git
+cd aquascope
+
+# Create a virtual environment
+python -m venv .venv
+source .venv/bin/activate  # Linux/macOS
+# .venv\Scripts\activate   # Windows
+
+# Install in editable mode with dev dependencies
+pip install -e ".[dev,all]"
+
+# Run tests
+pytest
+
+# Run linter
+ruff check aquascope/ tests/
+
+# Type checking
+mypy aquascope/ --ignore-missing-imports
+```
+
+## Pull Request Guidelines
+
+1. **Fork** the repository and create a feature branch.
+2. Write tests for new functionality.
+3. Ensure `pytest`, `ruff check`, and `mypy` pass.
+4. Keep commits atomic and write clear commit messages.
+5. Update documentation if you change public APIs.
+6. Reference related issues in your PR description.
+
+## Code Style
+
+- Follow PEP 8; enforced by `ruff`.
+- Use type hints for all public functions.
+- Write docstrings (Google or NumPy style).
+- Keep modules focused — one collector per file, one concept per module.
+
+## Reporting Issues
+
+- **Bugs**: Use the Bug Report template.
+- **New data sources**: Use the New Data Source template.
+- **New methodologies**: Use the New Research Methodology template.
+
+## Code of Conduct
+
+Be respectful, inclusive, and constructive. We follow the [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the MIT License.

aquascope-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 AquaScope Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

aquascope-0.4.0/MANIFEST.in

@@ -0,0 +1,13 @@
+include LICENSE
+include README.md
+include CHANGELOG.md
+include CONTRIBUTING.md
+include paper.md
+include paper.bib
+recursive-include aquascope *.py
+recursive-include data *.json *.csv *.md
+recursive-include docs *.md *.rst
+recursive-include examples *.py *.md
+prune tests
+prune notebooks
+prune venv