property-shared 0.1.0__tar.gz

property_shared-0.1.0/.dockerignore

@@ -0,0 +1,52 @@
+# Git
+.git
+.gitignore
+
+# Python
+__pycache__
+*.py[cod]
+*$py.class
+*.so
+.Python
+.venv
+venv
+ENV
+env
+.eggs
+*.egg-info
+*.egg
+
+# Testing/Dev
+.pytest_cache
+.coverage
+htmlcov
+.tox
+.mypy_cache
+.ruff_cache
+
+# IDE
+.vscode
+.idea
+*.swp
+*.swo
+
+# Output/Logs
+output/
+*.log
+
+# Apify (not needed for Fly deployment)
+apify_actors/
+
+# Docs/Tests (not needed at runtime)
+tests/
+docs/
+*.md
+!README.md
+
+# Environment
+.env
+.env.*
+
+# Misc
+.DS_Store
+Thumbs.db

property_shared-0.1.0/.gitignore

@@ -0,0 +1,252 @@
+# Virtual envs and tooling
+.venv/
+.env
+.env.*
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+.pytest_cache/
+.mypy_cache/
+.coverage
+htmlcov/
+
+# Editors
+.vscode/
+.idea/
+.DS_Store
+
+# Bytecode and build
+build/
+dist/
+*.egg-info/
+
+# Local data/cache
+/data/
+*.db
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[codz]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#   Usually these files are written by a python script from a template
+#   before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+# Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+# uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+# poetry.lock
+# poetry.toml
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#   pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
+#   https://pdm-project.org/en/latest/usage/project/#working-with-version-control
+# pdm.lock
+# pdm.toml
+.pdm-python
+.pdm-build/
+
+# pixi
+#   Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
+# pixi.lock
+#   Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
+#   in the .venv directory. It is recommended not to include this directory in version control.
+.pixi
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# Redis
+*.rdb
+*.aof
+*.pid
+
+# RabbitMQ
+mnesia/
+rabbitmq/
+rabbitmq-data/
+
+# ActiveMQ
+activemq-data/
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.envrc
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#   JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#   be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#   and can be added to the global gitignore or merged into this file.  For a more nuclear
+#   option (not recommended) you can uncomment the following to ignore the entire idea folder.
+# .idea/
+
+# Abstra
+#   Abstra is an AI-powered process automation framework.
+#   Ignore directories containing user credentials, local state, and settings.
+#   Learn more at https://abstra.io/docs
+.abstra/
+
+# Visual Studio Code
+#   Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore 
+#   that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
+#   and can be added to the global gitignore or merged into this file. However, if you prefer, 
+#   you could uncomment the following to ignore the entire vscode folder
+# .vscode/
+
+# Ruff stuff:
+.ruff_cache/
+
+# PyPI configuration file
+.pypirc
+
+# Marimo
+marimo/_static/
+marimo/_lsp/
+__marimo__/
+
+# Streamlit
+.streamlit/secrets.toml
+
+# Planning scraper test output
+output/
+
+# Local/temp files
+AGENTS.md
+skills/
+verification_*.csv
+example_ref/

property_shared-0.1.0/.gitlab-ci.yml

@@ -0,0 +1,42 @@
+stages:
+  - test
+  - publish
+
+variables:
+  UV_CACHE_DIR: .uv-cache
+
+.uv-setup: &uv-setup
+  before_script:
+    - curl -LsSf https://astral.sh/uv/install.sh | sh
+    - source $HOME/.local/bin/env
+    - uv sync --extra dev
+
+cache:
+  paths:
+    - .uv-cache/
+
+test:
+  stage: test
+  image: python:3.12
+  <<: *uv-setup
+  script:
+    - uv run pytest -v
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+
+publish:
+  stage: publish
+  image: python:3.12
+  environment: pypi
+  id_tokens:
+    PYPI_ID_TOKEN:
+      aud: pypi
+  before_script:
+    - curl -LsSf https://astral.sh/uv/install.sh | sh
+    - source $HOME/.local/bin/env
+  script:
+    - uv build
+    - uv publish --trusted-publishing always
+  rules:
+    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/

property_shared-0.1.0/CLAUDE.md

@@ -0,0 +1,124 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Property Shared is a FastAPI service + pure-Python core library for UK property data. It integrates:
+- **PPD** (Price Paid Data) - Land Registry transactions via SPARQL/Linked Data API
+- **EPC** - Energy Performance Certificates (requires API credentials)
+- **Rightmove** - Property listings via scraping with built-in politeness
+- **Planning** - UK council planning applications via vision-guided browser automation (98 verified councils)
+
+## Commands
+
+# Start .venv
+
+``bash
+act 
+```
+
+```bash
+# Install dependencies (with dev extras)
+uv sync --extra dev
+
+# Run API server
+uv run property-api                              # production mode
+uv run uvicorn app.main:app --reload             # dev mode with reload
+
+# Run CLI (core mode - no server needed)
+uv run --extra cli property-cli meta
+uv run --extra cli property-cli ppd comps "SW1A 1AA" --months 24
+uv run --extra cli property-cli rightmove search-url "SW1A 1AA"
+
+# CLI targeting running API (add --api-url)
+uv run --extra cli property-cli ppd comps "SW1A 1AA" --api-url http://localhost:8000
+
+# Tests
+uv run --extra dev pytest                        # unit tests (mocked)
+RUN_LIVE_TESTS=1 uv run --extra dev pytest       # live network tests
+
+# Single test
+uv run --extra dev pytest tests/test_ppd_service_live.py -v
+```
+
+## Architecture
+
+```
+property_core/              # Pure Python library (no FastAPI, no DB assumptions)
+├── models/                 # Domain Pydantic models
+│   ├── ppd.py              # PPDTransaction, PPDCompsResponse, SubjectProperty, etc.
+│   ├── epc.py              # EPCData
+│   ├── rightmove.py        # RightmoveListing, RightmoveListingDetail
+│   └── report.py           # PropertyReport, SaleHistory, MarketAnalysis, etc.
+├── ppd_client.py           # Transport: Land Registry SPARQL + Linked Data API → dicts
+├── epc_client.py           # Transport: EPC registry (async) → dicts/tuples
+├── rightmove_scraper.py    # Transport: listings scraper (sync) → dataclasses
+├── rightmove_location.py   # Transport: search URL builder (sync)
+├── postcode_client.py      # Transport: postcodes.io → dicts
+├── ppd_service.py          # Domain service: SPARQL parsing → typed PPD models (sync)
+├── planning_service.py     # Domain service: council matching + URL building (sync)
+├── report_service.py       # Product pipeline: multi-source aggregation (async)
+├── rental_service.py       # Standalone rental analysis with yield calculation (async)
+├── enrichment.py           # EPC enrichment pipeline + compute_enriched_stats()
+├── planning_scraper.py     # Vision-guided planning portal scraper (Playwright + OpenAI)
+└── planning_councils.json  # Verified council database (98 councils, 6 system types)
+
+app/                        # FastAPI service (thin HTTP wrapper)
+├── api/v1/                 # Versioned routers (import services from property_core)
+├── services/               # API-specific adapters (async threading, config binding)
+│   ├── epc_service.py      # Config-binding wrapper around EPCClient
+│   └── rightmove_service.py # anyio.to_thread + PoliteLimiter
+├── schemas/                # API envelope models (import domain models from core)
+├── core/config.py          # Settings via pydantic-settings (reads .env)
+└── web/                    # Demo UI at /demo
+
+property_cli/               # Typer CLI (imports only from property_core)
+└── main.py                 # All commands; --api-url switches to HTTP mode
+```
+
+**Three-layer separation**:
+- Transport clients (raw HTTP/SPARQL → dicts)
+- Domain services (parsing + orchestration → typed Pydantic models)
+- API layer (envelopes, async threading, rate limiting)
+
+**Data flow**: API router → Core service (domain logic) → Core client (network)
+
+## Key Patterns
+
+- **Dual-mode CLI**: Commands call `property_core` directly by default (fast, offline-capable). Add `--api-url` to route through the HTTP API instead.
+- **Domain service guardrails**: `property_core/ppd_service.py` enforces limits (MAX_LIMIT=200, FORM_MAX_LIMIT=50) and normalizes responses. API routers are thin wrappers.
+- **`include_raw` pattern**: All endpoints normalize data by default. Pass `include_raw=true` to get the original source data alongside normalized fields. EPC, PPD (transactions/address-search), Rightmove (listings), and Planning (council-for-postcode) all support this.
+- **Area stats**: `PPDCompsResponse` includes `percentile_25`, `percentile_75` for price quartiles. When an address is provided and found, also includes `subject_price_percentile` (0-100) and `subject_vs_median_pct` (e.g., +10.8 means 10.8% above median).
+- **EPC enrichment**: PPD comps can be enriched with EPC floor area via `enrich_epc=true` on the comps endpoint (or `--enrich-epc` in CLI). Groups comps by postcode, fetches all EPC certs per postcode (one API call each), fuzzy-matches addresses, and attaches derived fields (`epc_floor_area_sqm`, `price_per_sqft`, `epc_rating`, etc.) plus the full matched cert (`epc_match`) and confidence score (`epc_match_score`). After enrichment, call `compute_enriched_stats()` to populate `median_price_per_sqft` and `epc_match_rate`.
+- **Standalone rental analysis**: `analyze_rentals(postcode, purchase_price=N)` returns rental market stats (median/average rent, listing count) with optional gross yield calculation. No full report needed.
+- **Live test gating**: Tests making real network calls check `RUN_LIVE_TESTS=1` and skip gracefully on 503 or missing credentials.
+
+## Environment Variables
+
+Copy `.env.example` to `.env`. Key variables:
+- `EPC_API_EMAIL` / `EPC_API_KEY` - Required for EPC endpoints
+- `RIGHTMOVE_DELAY_SECONDS` - Rate limit delay (default 0.6s)
+- `OPENAI_API_KEY` - Required for planning scraper (vision extraction)
+- `PLAYWRIGHT_PROXY_URL` - Optional residential proxy for planning scraper (councils block datacenter IPs)
+
+## Using as a Library
+
+Install in another project: `pip install /path/to/property_shared` or add to dependencies.
+
+```python
+# Domain services (typed models, no FastAPI needed)
+from property_core import PPDService, PlanningService, PropertyReportService
+
+# Transport clients (raw dicts)
+from property_core import PricePaidDataClient, EPCClient, RightmoveLocationAPI, fetch_listings, PostcodeClient
+from property_core import enrich_comps_with_epc, compute_enriched_stats, fetch_listing, analyze_rentals
+
+# Domain models
+from property_core.models.ppd import PPDTransaction, PPDCompsResponse
+from property_core.models.epc import EPCData
+from property_core.models.report import PropertyReport
+
+# Planning scraper (requires playwright, openai)
+from property_core.planning_scraper import scrape_planning_application, search_planning_by_postcode
+```

property_shared-0.1.0/Dockerfile

@@ -0,0 +1,33 @@
+FROM python:3.11-slim
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    UV_PROJECT_ENVIRONMENT=/opt/venv \
+    PATH="/opt/venv/bin:$PATH"
+
+WORKDIR /app
+
+# System deps for Playwright/Chromium
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 \
+    libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 \
+    libgbm1 libasound2 libpango-1.0-0 libcairo2 libatspi2.0-0 \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN pip install --no-cache-dir uv
+
+# Copy dependency manifests first for better layer caching
+COPY pyproject.toml uv.lock README.md ./
+RUN uv sync --frozen --no-dev --extra api
+
+# Install Playwright browsers (Chromium only to save space)
+RUN playwright install chromium
+
+# Copy application code
+COPY app ./app
+COPY property_core ./property_core
+
+EXPOSE 8080
+
+CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
+

property_shared-0.1.0/GUIDELINES.md

@@ -0,0 +1,55 @@
+# Development Guidelines
+
+## Architecture
+
+Three-layer pattern:
+
+1. **Transport clients** (`property_core/*_client.py`)
+   - Raw HTTP calls → dicts
+   - Handle rate limiting, retries, auth
+   - Examples: `epc_client.py`, `ppd_client.py`, `postcode_client.py`
+
+2. **Domain services** (`property_core/*_service.py`)
+   - Parse raw data → typed Pydantic models
+   - Business logic, validation, orchestration
+   - Examples: `ppd_service.py`, `planning_service.py`
+
+3. **API routers** (`app/api/v1/*.py`)
+   - Thin HTTP wrappers over services
+   - Request validation, response envelopes
+
+## Adding a New Data Source
+
+1. Create transport client in `property_core/new_client.py`
+2. Create domain service in `property_core/new_service.py` (if needed)
+3. Add models to `property_core/models/new.py`
+4. Export from `property_core/models/__init__.py`
+5. Export from `property_core/__init__.py`
+6. Add API router in `app/api/v1/new.py` (optional)
+
+## Error Handling
+
+- **Not found**: Return `None` (let caller decide)
+- **Invalid input**: Raise `ValueError` with helpful message
+- **Network errors**: Let bubble up or wrap in domain-specific exception
+- **Debugging**: Use `include_raw=True` instead of logging
+
+## Testing
+
+All tests are live integration tests:
+
+```bash
+RUN_LIVE_TESTS=1 uv run --extra dev pytest -v
+```
+
+Tests skip gracefully on 503/network errors.
+
+Env vars: `RIGHTMOVE_TEST_POSTCODE`, `EPC_API_EMAIL`, `EPC_API_KEY`
+
+## Code Style
+
+- Type hints on all functions
+- Docstrings on public functions
+- Private functions with `_` prefix
+- Pydantic models with `Field()` for defaults
+- `from __future__ import annotations` in all modules