pycharter 0.0.24__py3-none-any.whl → 0.0.25__py3-none-any.whl

pycharter/api/README.md

@@ -0,0 +1,340 @@
+# PyCharter API
+
+REST API wrapper for PyCharter services using FastAPI.
+
+## Overview
+
+The PyCharter API is located at the root level of the repository (`api/`) and provides HTTP endpoints for all core PyCharter services:
+- **Contract Parsing**: Parse data contract files into components
+- **Contract Building**: Reconstruct contracts from metadata store
+- **Metadata Storage**: Store and retrieve schemas, metadata, and rules
+- **Schema Generation**: Generate Pydantic models from JSON Schemas
+- **Schema Conversion**: Convert Pydantic models to JSON Schemas
+- **Runtime Validation**: Validate data against schemas
+- **Quality Assurance**: Run quality checks, query metrics and violations
+
+## Installation
+
+Install PyCharter with API dependencies:
+
+```bash
+pip install pycharter[api]
+```
+
+## Running the API Server
+
+### Using the CLI command:
+
+```bash
+pycharter api
+```
+
+### Using uvicorn directly:
+
+```bash
+uvicorn api.main:app --reload
+```
+
+### Using Python:
+
+```python
+from api.main import main
+main()
+```
+
+## Configuration
+
+The API requires a database connection for most endpoints. Configure it using environment variables:
+
+### Database URL Configuration
+
+The API uses `PYCHARTER_DATABASE_URL` environment variable to connect to the database. You can set it in several ways:
+
+#### Method 1: Export in Shell (Recommended)
+
+```bash
+# Set the environment variable
+export PYCHARTER_DATABASE_URL="postgresql://user:password@localhost:5432/pycharter"
+
+# Then run the API
+pycharter api
+```
+
+#### Method 2: Inline with Command
+
+```bash
+# Set and run in one line
+PYCHARTER_DATABASE_URL="postgresql://user:password@localhost:5432/pycharter" pycharter api
+```
+
+#### Method 3: Add to Shell Profile (Persistent)
+
+Add to your `~/.bashrc`, `~/.zshrc`, or `~/.profile`:
+
+```bash
+export PYCHARTER_DATABASE_URL="postgresql://user:password@localhost:5432/pycharter"
+```
+
+Then reload your shell:
+```bash
+source ~/.bashrc  # or source ~/.zshrc
+```
+
+#### Method 4: Using a .env File
+
+Create a `.env` file in your project root:
+
+```bash
+PYCHARTER_DATABASE_URL=postgresql://user:password@localhost:5432/pycharter
+```
+
+Then load it before running (requires `python-dotenv`):
+```bash
+export $(cat .env | xargs)
+pycharter api
+```
+
+Or use `dotenv-cli`:
+```bash
+pip install dotenv-cli
+dotenv run pycharter api
+```
+
+### Alternative Environment Variable
+
+PyCharter also supports Airflow-style configuration:
+
+```bash
+export PYCHARTER__DATABASE__SQL_ALCHEMY_CONN="postgresql://user:password@localhost:5432/pycharter"
+```
+
+### Metadata Store Configuration
+
+The API can be configured to use different metadata store backends:
+
+```bash
+# Metadata store type (in_memory, postgres, mongodb, redis)
+export PYCHARTER_API_STORE_TYPE=postgres
+
+# Connection string for database-backed stores
+export PYCHARTER_API_CONNECTION_STRING=postgresql://user:pass@localhost:5432/pycharter
+```
+
+**Note**: If `PYCHARTER_DATABASE_URL` is set, it will be used for both the database connection and metadata store (for PostgreSQL).
+
+## API Endpoints
+
+### Contract Endpoints
+
+#### `POST /api/v1/contracts/parse`
+Parse a data contract dictionary into components.
+
+**Request Body:**
+```json
+{
+  "contract": {
+    "schema": {...},
+    "metadata": {...},
+    "ownership": {...},
+    "governance_rules": {...}
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "schema": {...},
+  "metadata": {...},
+  "ownership": {...},
+  "governance_rules": {...},
+  "coercion_rules": {...},
+  "validation_rules": {...},
+  "versions": {...}
+}
+```
+
+#### `POST /api/v1/contracts/build`
+Build a complete contract from metadata store.
+
+**Request Body:**
+```json
+{
+  "schema_id": "user_schema",
+  "version": "1.0.0",
+  "include_metadata": true,
+  "include_ownership": true,
+  "include_governance": true
+}
+```
+
+#### `GET /api/v1/contracts`
+List all data contracts stored in the database.
+
+#### `GET /api/v1/contracts/{contract_id}`
+Get a specific data contract by ID.
+
+### Metadata Endpoints
+
+#### `GET /api/v1/metadata/schemas`
+List all schemas stored in the metadata store.
+
+#### `GET /api/v1/metadata/schemas/{schema_id}`
+Get a schema by ID (optional version parameter).
+
+#### `GET /api/v1/metadata/schemas/{schema_id}/complete`
+Get complete schema with coercion and validation rules merged.
+
+#### `POST /api/v1/metadata/schemas`
+Store a schema in the metadata store.
+
+#### `GET /api/v1/metadata/metadata/{schema_id}`
+Get metadata for a schema (optional version parameter).
+
+#### `POST /api/v1/metadata/metadata`
+Store metadata for a schema.
+
+#### `GET /api/v1/metadata/coercion-rules/{schema_id}`
+Get coercion rules for a schema (optional version parameter).
+
+#### `POST /api/v1/metadata/coercion-rules`
+Store coercion rules for a schema.
+
+#### `GET /api/v1/metadata/validation-rules/{schema_id}`
+Get validation rules for a schema (optional version parameter).
+
+#### `POST /api/v1/metadata/validation-rules`
+Store validation rules for a schema.
+
+### Quality Assurance Endpoints
+
+#### `GET /api/v1/quality/metrics`
+List quality metrics (optional filtering by schema_id, pagination).
+
+#### `GET /api/v1/quality/metrics/{metric_id}`
+Get a specific quality metric by ID.
+
+#### `GET /api/v1/quality/reports/{schema_id}`
+Get quality reports for a schema/data feed (optional data_source filter).
+
+#### `POST /api/v1/quality/check`
+Run a quality check against a data contract.
+
+#### `POST /api/v1/quality/violations`
+Query data quality violations.
+
+### Schema Endpoints
+
+#### `POST /api/v1/schemas/generate`
+Generate Pydantic model from JSON Schema.
+
+#### `POST /api/v1/schemas/convert`
+Convert Pydantic model to JSON Schema.
+
+### Validation Endpoints
+
+#### `POST /api/v1/validation/validate`
+Validate a single record against a schema.
+
+#### `POST /api/v1/validation/validate-batch`
+Validate a batch of records against a schema.
+
+## API Documentation
+
+Once the API server is running, you can access:
+
+- **Swagger UI**: http://localhost:8000/docs
+- **ReDoc**: http://localhost:8000/redoc
+- **OpenAPI JSON**: http://localhost:8000/openapi.json
+
+## Example Usage
+
+### Using curl:
+
+```bash
+# List all contracts
+curl http://localhost:8000/api/v1/contracts
+
+# Get a specific contract
+curl http://localhost:8000/api/v1/contracts/{contract_id}
+
+# List all schemas
+curl http://localhost:8000/api/v1/metadata/schemas
+
+# Get a schema
+curl http://localhost:8000/api/v1/metadata/schemas/{schema_id}
+
+# Get quality metrics
+curl http://localhost:8000/api/v1/quality/metrics
+
+# Get quality report for a schema
+curl http://localhost:8000/api/v1/quality/reports/{schema_id}
+```
+
+### Using Python requests:
+
+```python
+import requests
+
+# List contracts
+response = requests.get("http://localhost:8000/api/v1/contracts")
+contracts = response.json()
+
+# Get quality metrics
+response = requests.get("http://localhost:8000/api/v1/quality/metrics")
+metrics = response.json()
+```
+
+## Troubleshooting
+
+### Database Connection Issues
+
+If you see errors about database URL not being configured:
+
+1. **Check environment variable is set:**
+   ```bash
+   echo $PYCHARTER_DATABASE_URL
+   ```
+
+2. **Verify database is accessible:**
+   ```bash
+   psql $PYCHARTER_DATABASE_URL -c "SELECT 1;"
+   ```
+
+3. **Ensure database is initialized:**
+   ```bash
+   pycharter db init $PYCHARTER_DATABASE_URL
+   ```
+
+### API Not Starting
+
+If the API fails to start:
+
+1. **Check if uvicorn is installed:**
+   ```bash
+   pip install pycharter[api]
+   ```
+
+2. **Check for port conflicts:**
+   ```bash
+   # Use a different port
+   pycharter api --port 8001
+   ```
+
+3. **Check logs for errors:**
+   The API will print error messages to stderr if there are configuration issues.
+
+## Production Deployment
+
+For production, consider:
+
+1. **Use a process manager** (systemd, supervisor, etc.)
+2. **Set environment variables** in your deployment configuration
+3. **Use a reverse proxy** (nginx, traefik) in front of the API
+4. **Enable HTTPS** for secure connections
+5. **Configure CORS** appropriately in `api/main.py`
+6. **Use a production ASGI server** like Gunicorn with Uvicorn workers:
+
+```bash
+gunicorn api.main:app -w 4 -k uvicorn.workers.UvicornWorker
+```

{api → pycharter/api}/__init__.py

@@ -17,7 +17,7 @@ __version__ = "0.0.3"
 
 # Check if FastAPI is available
 try:
-    from api.main import app, create_application
+    from pycharter.api.main import app, create_application
     __all__ = ["app", "create_application"]
 except (ImportError, ModuleNotFoundError):
     # FastAPI not installed or api module not available

{api → pycharter/api}/dependencies/__init__.py

@@ -4,7 +4,7 @@ Dependencies for API routes.
 This module provides FastAPI dependency injection for shared resources.
 """
 
-from api.dependencies.database import get_db_session
-from api.dependencies.store import get_metadata_store
+from pycharter.api.dependencies.database import get_db_session
+from pycharter.api.dependencies.store import get_metadata_store
 
 __all__ = ["get_db_session", "get_metadata_store"]

{api → pycharter/api}/main.py

@@ -16,7 +16,7 @@ from fastapi.routing import APIRoute
 from pycharter import __version__ as pycharter_version
 
 # Import routers from v1
-from api.routes.v1 import (
+from pycharter.api.routes.v1 import (
     contracts,
     metadata,
     quality,
@@ -296,7 +296,7 @@ def main():
     import uvicorn
     
     uvicorn.run(
-        "api.main:app",
+        "pycharter.api.main:app",
         host="0.0.0.0",
         port=8000,
         reload=True,  # Set to False in production

{api → pycharter/api}/models/__init__.py

@@ -4,13 +4,13 @@ API request and response models.
 This module contains Pydantic models for API request/response validation.
 """
 
-from api.models.contracts import (
+from pycharter.api.models.contracts import (
     ContractParseRequest,
     ContractParseResponse,
     ContractBuildRequest,
     ContractBuildResponse,
 )
-from api.models.metadata import (
+from pycharter.api.models.metadata import (
     MetadataStoreRequest,
     MetadataStoreResponse,
     MetadataGetRequest,
@@ -20,13 +20,13 @@ from api.models.metadata import (
     SchemaGetRequest,
     SchemaGetResponse,
 )
-from api.models.schemas import (
+from pycharter.api.models.schemas import (
     SchemaGenerateRequest,
     SchemaGenerateResponse,
     SchemaConvertRequest,
     SchemaConvertResponse,
 )
-from api.models.validation import (
+from pycharter.api.models.validation import (
     ValidationRequest,
     ValidationResponse,
     ValidationBatchRequest,

{api → pycharter/api}/routes/v1/__init__.py

@@ -6,7 +6,7 @@ All routers defined here are automatically included in the main application
 with the `/api/v1` prefix.
 """
 
-from api.routes.v1 import (
+from pycharter.api.routes.v1 import (
     contracts,
     metadata,
     quality,

{api → pycharter/api}/routes/v1/contracts.py

@@ -15,9 +15,9 @@ from pycharter.contract_builder.builder import ContractArtifacts
 from pycharter.db.models import DataContractModel
 from pycharter.metadata_store import MetadataStoreClient
 
-from api.dependencies.database import get_db_session
-from api.dependencies.store import get_metadata_store
-from api.models.contracts import (
+from pycharter.api.dependencies.database import get_db_session
+from pycharter.api.dependencies.store import get_metadata_store
+from pycharter.api.models.contracts import (
     ContractBuildRequest,
     ContractBuildResponse,
     ContractCreateFromArtifactsRequest,
@@ -28,7 +28,7 @@ from api.models.contracts import (
     ContractParseResponse,
     ContractUpdateRequest,
 )
-from api.utils import ensure_uuid, get_by_id_or_404, safe_uuid_to_str
+from pycharter.api.utils import ensure_uuid, get_by_id_or_404, safe_uuid_to_str
 
 logger = logging.getLogger(__name__)
 router = APIRouter()

{api → pycharter/api}/routes/v1/docs.py

@@ -9,8 +9,8 @@ from typing import Optional
 
 from fastapi import APIRouter, Depends, HTTPException, Query
 
-from api.dependencies.store import get_metadata_store
-from api.models.docs import (
+from pycharter.api.dependencies.store import get_metadata_store
+from pycharter.api.models.docs import (
     DocsFormat,
     DocsRequest,
     DocsResponse,

{api → pycharter/api}/routes/v1/evolution.py

@@ -9,8 +9,8 @@ from typing import Optional
 
 from fastapi import APIRouter, Depends, HTTPException, Query
 
-from api.dependencies.store import get_metadata_store
-from api.models.evolution import (
+from pycharter.api.dependencies.store import get_metadata_store
+from pycharter.api.models.evolution import (
     ChangeTypeEnum,
     CompatibilityCheckRequest,
     CompatibilityCheckResponse,

{api → pycharter/api}/routes/v1/metadata.py

@@ -7,10 +7,10 @@ from typing import Any, Dict, List
 from fastapi import APIRouter, Depends, HTTPException, Query, status
 from sqlalchemy.orm import Session
 
-from api.dependencies.database import get_db_session
-from api.dependencies.store import get_metadata_store
-from api.utils import ensure_uuid, get_by_id_or_404, model_to_dict, safe_uuid_to_str
-from api.models.metadata import (
+from pycharter.api.dependencies.database import get_db_session
+from pycharter.api.dependencies.store import get_metadata_store
+from pycharter.api.utils import ensure_uuid, get_by_id_or_404, model_to_dict, safe_uuid_to_str
+from pycharter.api.models.metadata import (
     CoercionRulesStoreRequest,
     MetadataGetRequest,
     MetadataGetResponse,
@@ -26,7 +26,7 @@ from api.models.metadata import (
     SchemaStoreResponse,
     ValidationRulesStoreRequest,
 )
-from api.models.metadata_entities import (
+from pycharter.api.models.metadata_entities import (
     DomainCreateRequest,
     DomainUpdateRequest,
     EnvironmentCreateRequest,

{api → pycharter/api}/routes/v1/quality.py

@@ -18,9 +18,9 @@ from pycharter.quality import (
     QualityReport,
     QualityThresholds,
 )
-from api.dependencies.store import get_metadata_store
-from api.dependencies.database import get_db_session
-from api.models.quality import (
+from pycharter.api.dependencies.store import get_metadata_store
+from pycharter.api.dependencies.database import get_db_session
+from pycharter.api.models.quality import (
     FieldQualityMetricsResponse,
     QualityCheckRequest,
     QualityReportResponse,

{api → pycharter/api}/routes/v1/schemas.py

@@ -5,7 +5,7 @@ Route handlers for schema generation and conversion.
 from fastapi import APIRouter, HTTPException, status
 
 from pycharter import from_dict, to_dict
-from api.models.schemas import (
+from pycharter.api.models.schemas import (
     SchemaConvertRequest,
     SchemaConvertResponse,
     SchemaGenerateRequest,

{api → pycharter/api}/routes/v1/settings.py

@@ -13,7 +13,7 @@ from sqlalchemy.exc import SQLAlchemyError
 from sqlalchemy.orm import Session
 
 from pycharter.config import get_database_url
-from api.dependencies.database import get_db_session
+from pycharter.api.dependencies.database import get_db_session
 
 router = APIRouter()
 

{api → pycharter/api}/routes/v1/tracking.py

@@ -10,7 +10,7 @@ from typing import Optional
 
 from fastapi import APIRouter, HTTPException, Query
 
-from api.models.tracking import (
+from pycharter.api.models.tracking import (
     ExportFormat,
     ExportResponse,
     MetricsQueryResponse,

{api → pycharter/api}/routes/v1/validation.py

@@ -7,8 +7,8 @@ Supports the new ValidatorBuilder API with quality metrics.
 from fastapi import APIRouter, Depends, HTTPException, status
 
 from pycharter import Validator, ValidatorBuilder
-from api.dependencies.store import get_metadata_store
-from api.models.validation import (
+from pycharter.api.dependencies.store import get_metadata_store
+from pycharter.api.models.validation import (
     ValidationBatchRequest,
     ValidationBatchResponse,
     ValidationErrorDetail,

{api → pycharter/api}/routes/v1/validation_jobs.py

@@ -12,14 +12,14 @@ from fastapi import APIRouter, Depends, HTTPException, status
 from pydantic import BaseModel, Field
 
 if TYPE_CHECKING:
-    from worker.queue.redis_queue import ValidationJobQueue
+    from pycharter.worker.queue.redis_queue import ValidationJobQueue
 
-from api.dependencies.store import get_metadata_store
+from pycharter.api.dependencies.store import get_metadata_store
 from pycharter.metadata_store import MetadataStoreClient
 
 # Try to import worker components
 try:
-    from worker.queue.redis_queue import ValidationJobQueue
+    from pycharter.worker.queue.redis_queue import ValidationJobQueue
     WORKER_AVAILABLE = True
 except ImportError:
     WORKER_AVAILABLE = False

pycharter/cli.py

@@ -390,7 +390,7 @@ def main():
             os.environ["PYTHONPATH"] = pythonpath
 
         uvicorn.run(
-            "api.main:app",
+            "pycharter.api.main:app",
             host=args.host,
             port=args.port,
             reload=args.reload,
@@ -408,21 +408,19 @@ def main():
         ui_path = None
         possible_paths = []
 
-        # Check installed package location first (for pip installs)
+        # Check installed package location first (src layout: __file__ parent is package dir)
         try:
             import pycharter
-            pycharter_path = Path(pycharter.__file__).parent.parent
-            package_ui = pycharter_path / "ui"
+            package_ui = Path(pycharter.__file__).parent / "ui"
             possible_paths.append(package_ui)
         except (ImportError, AttributeError):
             pass
 
-        # Check development locations
+        # Check development locations (__file__ is src/pycharter/cli.py)
         possible_paths.extend([
-            Path(__file__).parent.parent.parent
-            / "ui",  # From pycharter/cli.py -> pycharter -> project root -> ui
-            Path.cwd() / "ui",  # Current working directory
-            Path(__file__).parent.parent / "ui",  # Alternative path
+            Path(__file__).parent / "ui",  # src/pycharter/ui
+            Path.cwd() / "src" / "pycharter" / "ui",
+            Path.cwd() / "ui",
         ])
 
         for path in possible_paths:
@@ -492,7 +490,7 @@ def main():
 
         if args.worker_command == "start":
             try:
-                from worker.cli import cmd_worker_start
+                from pycharter.worker.cli import cmd_worker_start
             except ImportError:
                 print(
                     "❌ Error: worker component not found.",