usaspending-orm 0.7.0__tar.gz → 0.7.3__tar.gz

usaspending_orm-0.7.3/.gitignore

@@ -0,0 +1,6 @@
+.DS_Store
+.venv
+__pycache__
+.*_cache
+repomix*
+dist/

usaspending_orm-0.7.3/CHANGELOG.md

@@ -0,0 +1,81 @@
+# Changelog
+
+All notable changes to the USASpending ORM library are documented here.
+The format loosely follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and the project adheres to [Semantic Versioning](https://semver.org/).
+
+## [0.7.3] - 2026-07-05
+
+### Added
+
+- `AwardsSearch.object_classes(*codes)`: filter awards by federal object class codes (per Office of Management and Budget Circular A-11) on the `/search/spending_by_award/` and `/search/spending_by_award_count/` endpoints. Pass codes such as `"10"` or `"252"`, not names; multiple codes use OR logic.
+- `SubAwardsSearch.object_classes()` raises `ValidationError`. The USASpending API accepts the `object_classes` filter for award searches only and returns HTTP 422 when it is combined with `spending_level=subawards`.
+- `client.downloads.search()` emits a `UserWarning` when the query contains an `object_classes` filter, because the `/api/v2/download/search/` endpoint drops the key. The filter is still forwarded unmodified.
+- Known upstream limitation as of 2026-07-05: the production API accepts `object_classes` but the rollout is incomplete. The search endpoint returns zero results for any object class, the count endpoint does not narrow, and the subawards rejection is not yet enforced server side. The filter should begin returning data once USAspending finishes deploying the feature and populating its award index.
+- `tests/test_readme_examples.py`: integration tests that extract the README's Python code blocks and execute the runnable ones verbatim against the live API, so broken or rotted examples fail mechanically. A network-free guard in the default suite catches extraction drift. The live `client` fixture moved to `tests/conftest.py` and is now shared with `tests/test_integration.py`.
+
+### Documentation
+
+- Corrected the `api-docs-links.md` subawards entry: `SubAwardsSearch` POSTs `/api/v2/search/spending_by_award/` with `subawards=true` and `spending_level=subawards` (inheriting `AwardsSearch`) rather than calling `/api/v2/subawards/`.
+- Added missing `api-docs-links.md` entries for `/api/v2/idvs/awards/` (IDV child awards) and the TAS filter-tree endpoints (`/api/v2/references/filter_tree/tas/` and its `{toptier_code}/` and `{toptier_code}/{federal_account}/` variants).
+- An upstream API compatibility review of USASpending API releases from April through June 2026 found no breaking changes for this library. The 0.7.2 release already handles the `spending_by_award` Pydantic migration (HTTP 422 validation errors), the `program_activities` filter rename, and the new F001 through F010 assistance type codes.
+- Revalidated all README examples against live USAspending data. Fixed two broken chains (`location.full_address` is actually `location.formatted_address`; a subaward's place of performance is on the subaward, not its recipient), refreshed stale sample outputs, and added examples for the `client.spending` category rollups and the list-style `len(query)` interface.
+
+## [0.7.2] - 2026-06-05
+
+### Added
+
+- `client.downloads.search(query, ...)` backed by the new `/api/v2/download/search/` endpoint, which bundles award, transaction, and subaward data into a single download built from an awards search query. Supports `file_format`, `spending_level`, `columns`, and `limit`.
+- `QueryBuilder.to_filters_payload()`: public accessor returning the assembled API `filters` object for a query without executing a search.
+
+### Changed
+
+- HTTP 422 (Unprocessable Entity) validation responses are now surfaced as `APIError` with the response detail instead of a generic `HTTPError`. This aligns with the USASpending API's move to Pydantic request validation on `/search/spending_by_award/`.
+
+### Deprecated
+
+- `program_activity()` is deprecated and now forwards to `program_activities()`, emitting a `DeprecationWarning`. The USASpending API removed the `program_activity` filter from `/search/spending_by_award/` in favor of `program_activities`. Existing calls continue to work; integer codes are forwarded as string `code` values.
+
+## [0.7.1] - 2026-04-16
+
+### Added
+
+- Public export of `DetachedInstanceError` so callers can `from usaspending import DetachedInstanceError` as documented in the README.
+- `config.allowed_download_hosts`: an allow-list of hostnames the library will fetch binary downloads from. Defaults to `{"api.usaspending.gov", "files.usaspending.gov"}`.
+- `config.validate()` now rejects malformed `base_url`, CR/LF in `user_agent`, and empty `cache_dir`.
+
+### Changed
+
+- `ValidationError` now inherits from both `USASpendingError` and the built-in `ValueError`. Callers that previously caught `ValueError` for invalid parameters continue to work unchanged; new code can catch the library-specific `ValidationError`.
+- `RateLimiter` now raises `ValidationError` (rather than a bare `ValueError`) for invalid `max_calls`/`period` arguments. Non-breaking due to the dual inheritance above.
+- `RecipientsResource.find_by_recipient_id` is now annotated as non-Optional. This matches its long-standing behavior of raising on miss via the direct-ID lookup path; runtime behavior is unchanged. `find_by_duns` and `find_by_uei` remain Optional because they use the search-delegation path.
+- `Agency.get_obligations` return annotation corrected from `float | None` to `Decimal | None` to match actual runtime type.
+- `Agency.total_obligations` now applies `Decimal` coercion to stored values to match its declared return type.
+- `TransactionsSearch._award_id` and `FundingSearch._award_id` annotated as `str | None` rather than `str` (they are always `None` until set).
+- `time_period()` now rejects date ranges where `end_date` is earlier than `start_date`.
+
+### Fixed
+
+- `to_date()` now extracts the `date` portion from `datetime` inputs instead of passing them through unchanged.
+- `IDVChildAwardsSearch._clone()` no longer silently drops `_filter_objects`, `_max_pages`, `_order_by`, and `_order_direction` when chaining.
+- `SubAwardsSearch._clone()` now delegates to `super()._clone()` so base-class state changes propagate.
+- `AwardAccountsQuery._clone()` no longer copies a stale `_cached_count` across filter changes.
+- `Agency.latest_action_date` no longer crashes when the award-summary endpoint returns no data.
+- `round_to_millions()` cleaned up redundant disjunct in threshold check.
+- Removed unused `DEFAULT_KEEP_UPPERCASE` set, which also contained a latent missing-comma bug that concatenated two entries.
+- Fixed malformed `Raises:` block in `AwardResource.find_by_generated_id` docstring.
+- `Agency.id` was missing its return-type annotation.
+
+### Security
+
+- Binary downloads (`_download_binary_file`) now require `https` and a hostname on `config.allowed_download_hosts`, mitigating SSRF if API responses are tampered with.
+- File-backed cache directory is now created with `0700` and tightened to `0700` if it already exists with looser bits. Protects the pickle-backed cache on shared hosts.
+- ZIP extraction creates the extract directory with `0700`, detects absolute paths with stdlib `posixpath.isabs` / `ntpath.isabs` (covers Windows drive letters and UNC prefixes), and uses `normpath`-based prefix comparison for traversal detection.
+
+### Removed
+
+- Dead code: `get_past_fiscal_years`, `custom_titlecase_callback`, `parse_agency_type`, `parse_agency_tier`, `QueryBuilder._fetch_page`, empty `AwardsSearch.__init__`, redundant `AwardsSearch._clone` override, unreachable `try/except` in `_derived_award_identifier`.
+
+## [0.7.0] - 2026-04-12
+
+Previous release. See git history for details.

usaspending_orm-0.7.3/CLAUDE.md

@@ -0,0 +1,243 @@
+# claude.md
+
+## Project Overview
+
+USASpending ORM is a Python ORM library for the USAspending.gov API, providing a modern, client-centric interface with query builders and automatic pagination.
+
+## USASpending API Reference Documentation
+
+- See `api-docs-links.md` for official endpoint documentation links
+- Use `context7 MCP` to access the API documentation for the USASpending project and for any external libraries referenced in this codebase
+
+## Architecture Principles
+
+### Agency-agnostic implementation
+
+- Designed to work with any agency's data
+- No hardcoded agency logic
+- Provides generic filters and helper methods
+
+### Client-Centric Design
+
+- All operations flow through a central `USASpendingClient` client
+- Resources accessed as properties: `client.awards`, `client.recipients`, etc.
+- No global state or session variables
+- Thread-safe through instance-based design
+
+### Query Builder Pattern
+
+- Inspired by ActiveRecord and Django ORM syntax
+- Chainable, immutable query construction via `_clone()` method
+- Lazy evaluation - queries execute only when iterated
+- Standard python List-like interface for results will fire relevant API calls (e.g. `len(client.awards)` fires off an API call to a `count` endpoint, for example)
+
+### Resource Organization
+
+- Resources grouped in `resources/` directory
+- Each resource inherits from `BaseResource`
+- Resources create and return appropriate query builders
+- Clean separation between resources and queries
+
+### Models and Data Structures
+
+- Inspired by ActiveRecord and Django ORM structure
+- Data models in `models/` directory
+- Use composition for nested structures
+- Raw API data stored in internal `_data` attribute and exposed via `raw()` method in `BaseModel`
+- Properties provide access to structured data
+- Properties use the `_lazy_get()` method to fetch data lazily for models inheriting from `LazyRecord`
+- Models can chain their associations as QueryBuilder objects for one-to-many associations (e.g. `award.recipient.location.city`
+- Models load related one-to-one associations as related model objects (e.g. `award.recipient`)
+- Models provide a consistent API to underlying property names given the API's sometimes inconsistent naming conventions
+
+## Code Style and Standards
+
+### Python Guidelines
+
+- Follow PEP 8 guidelines
+- Follow code style in the linting settings in `pyproject.toml`
+- FOLLOW BEST PRACTICES OBJECT-ORIENTED DESIGN
+  - DON'T REPEAT YOURSELF (DRY)
+  - Single Responsibility Principle
+  - Don't over-engineer: keep it simple, stupid (KISS)
+
+### Type Hints
+
+- Use `from __future__ import annotations` for forward references
+- Add type hints to ALL function signatures
+- Use `TypeVar` and `Generic` for query builders
+- Prefer `Optional[T]` over `Union[T, None]`
+
+### Project Structure
+
+```
+src/usaspending/
+├── client.py              # Main USASpendingClient
+├── cli/                   # CLI tools (e.g., download_award)
+├── config.py              # Configuration settings
+├── download/              # Download job management
+├── logging_config.py      # Custom logger configuration
+├── exceptions.py          # Custom exceptions
+├── resources/            # Resource classes
+├── queries/              # Query builders
+├── models/               # Data models
+└── utils/                # Utilities (retry, rate limit)
+```
+
+## Testing Strategy
+
+### Test Organization
+
+```
+tests/
+├── conftest.py           # Shared fixtures
+├── test_client.py        # Client tests
+├── queries/              # Query builder tests
+├── models/               # Model tests
+├── resources/            # Resource tests
+├── utils/                # Utility tests
+├── mocks/                # Mock client and response builders
+└── fixtures/             # Real-world POST-header and response JSON data from USASpending API
+```
+
+### Testing Principles
+
+- Use `pytest` for test framework and implement pytest best practices
+- Always integrate fixture data from `tests/fixtures/` into tests
+- Aim for >80% test coverage
+- Mock API client using the `mock_usa_client()` method and other helps in `tests/mocks/`
+- Use helper methods to load fixtures and the mock client object in `tests/conftest.py`
+
+## Quick Commands
+
+- Run tests: `uv run pytest`
+- Run integration tests: `uv run pytest -m integration`
+- Lint: `uv run ruff check src/ tests/`
+- Format: `uv run ruff format src/ tests/`
+- No Justfile in this project
+
+## Implementation Patterns
+
+### Configuration
+
+- Uses a global `config` object
+- No environment variables in library code
+
+### Resource Classes
+
+- Lazy-loaded via property descriptors
+- Stored in `_resources` dict
+- Return query builder instances
+- Handle resource-specific logic
+
+### Query Builders
+
+- Hierarchy: `BaseQuery[T]` -> `QueryBuilder[T]` -> `SearchQueryBuilder[T]`; also `ClientSideQueryBuilder[T]` (extends `BaseQuery`)
+- `QueryBuilder[T]` handles paginated API queries; `ClientSideQueryBuilder[T]` handles client-side filtering
+- Implement abstract methods:
+  - `_endpoint()`: API endpoint
+  - `_build_payload()`: Request payload
+  - `_transform_result()`: Result transformation
+  - `_clone()`: Return cloned instance
+  - `__iter__()`: Iterable interface
+- All filter methods return cloned instances
+
+### Caching
+
+- Implemented using `cachier` python library via `cachier` decorator
+- Supports file (default, via pickle) and memory backends
+- Dynamically reconfigurable at runtime via observer pattern (`register_cache_settings_observer` in `config.py`); cachier decorator is rebuilt when settings change
+- Client uses cache-with-fallback: tries cached response first, falls back to uncached on failure
+- Cache configurable via the global `config` object set in `src/usaspending/config.py`:
+  - `cache_enabled`: Enable/disable caching (default: False)
+  - `cache_ttl`: `timedelta` TTL (default: 1 week); `configure()` accepts seconds as `int`/`float`
+  - `cache_timeout`: Seconds to wait for cache entry processing (default: 60)
+  - `cache_dir`: Directory for file-based cache (default: "~/.cache/usaspending")
+  - `cache_backend`: Backend type ('file' or 'memory', default: 'file')
+  - `cache_namespace`: Namespace for cache keys (default: 'usaspending-orm')
+
+### Logging
+
+- Implemented via custom `USASpendingLogger` class using Python's `logging` module
+- Log all API requests and responses, including total counts of API calls
+- Log query execution times
+- Implement variable log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+
+## Development Workflow
+
+### Adding New Features
+
+1. Define interface in resource class
+2. Create query builder if needed
+3. Write tests first (TDD)
+4. Implement with full type hints
+5. Add to `__all__` exports
+6. Document in docstrings
+
+### Code Review Checklist
+
+- [ ] All methods have type hints
+- [ ] Docstrings follow Google style
+- [ ] Proper exception handling
+- [ ] PEP 8 pythonic code style
+- [ ] Code is DRY, object-oriented, clear and maintainable
+- [ ] Passes `ruff` linting and formatting check
+
+## Error Handling
+
+### Exception Hierarchy
+
+- `USASpendingError` - Base exception
+- `APIError` - API response errors
+- `HTTPError` - Network/transport layer errors (retryable 5xx)
+- `RateLimitError` - Rate limit exceeded
+- `ValidationError` - Invalid parameters
+- `DetachedInstanceError` - Lazy-loading on closed/detached client
+- `ConfigurationError` - Invalid configuration settings
+- `DownloadError` - File download/extraction failures
+
+### Retry Logic
+
+- Implemented in `RetryHandler`
+- Exponential backoff
+- Configurable max attempts
+- Only retry on specific errors
+
+## Documentation Standards
+
+### Docstring Requirements
+
+- Google style format
+- Include parameter types
+- Document return values
+- Add usage examples
+- Note any side effects
+
+## Release Process
+
+### Quality Gates
+
+- 100% public API documented
+- Type hints pass mypy
+- Tests pass with >80% coverage
+- No TODO/FIXME in code
+
+### Versioning Strategy
+
+- Semantic versioning
+- Alpha releases initially
+- Document all changes
+- Migration guides for major versions
+
+## Special Instructions
+
+- **Use full agency names and not acronyms in examples and docstrings. For example always use `National Aeronautics and Space Administration` instead of `NASA` when referencing the space agency.**
+- **This file is committed to the repository and so should never include any secrets.**
+- **Always read `README.md` before making changes.**
+- **Cross-reference:** Also read `api-docs-links.md` for detailed contributor guidelines.
+- **When adding new features, update all relevant docs, tests, and requirements files.**
+- **All code must be Python 3.9+ compatible.**
+- **When in doubt, prefer explicit, readable code over cleverness.**
+- **Never use non-ASCII characters or the em dash.**
+- **When referencing/calling external libraries, use `context7 MCP` to get latest documentation.**
+- **All new code must have full type annotations and Google-style docstrings.**