mocksql 0.1.0__tar.gz

mocksql-0.1.0/PKG-INFO

@@ -0,0 +1,232 @@
+Metadata-Version: 2.4
+Name: mocksql
+Version: 0.1.0
+Summary: TDD engine for Analytics Engineering — generate unit test data for SQL queries
+License: MIT
+Keywords: sql,testing,analytics,bigquery,duckdb,langgraph
+Author: Adel Skhiri
+Author-email: skhiriadel92@gmail.com
+Requires-Python: >=3.11,<3.14
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Testing
+Requires-Dist: bcrypt (>=4.2.0,<5.0.0)
+Requires-Dist: db-dtypes (>=1.2.0,<2.0.0)
+Requires-Dist: duckdb (>=1.5.0,<2.0.0)
+Requires-Dist: fastapi (>=0.115.0,<1.0.0)
+Requires-Dist: google-cloud-bigquery (>=3.41.0,<4.0.0)
+Requires-Dist: google-cloud-bigquery-storage (>=2.25.0,<3.0.0)
+Requires-Dist: google-cloud-secret-manager (>=2.20.1,<3.0.0)
+Requires-Dist: itsdangerous (>=2.2.0,<3.0.0)
+Requires-Dist: langchain (>=1.2.0,<2.0.0)
+Requires-Dist: langchain-classic (==1.0.3)
+Requires-Dist: langchain-google-genai (>=4.0.0,<5.0.0)
+Requires-Dist: langgraph (>=1.1.0,<2.0.0)
+Requires-Dist: pandas (>=2.2.2,<3.0.0)
+Requires-Dist: passlib (>=1.7.4,<2.0.0)
+Requires-Dist: pyarrow (>=22,<23)
+Requires-Dist: pydantic (>=2.0.0,<3.0.0)
+Requires-Dist: pyjwt (>=2.8.0,<3.0.0)
+Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
+Requires-Dist: pyyaml (>=6.0)
+Requires-Dist: sqlglot[c] (>=30.4.0,<31.0.0)
+Requires-Dist: sqlmesh (>=0.234.0,<0.235.0)
+Requires-Dist: typer[all] (>=0.24.2,<0.25.0)
+Requires-Dist: uvicorn (>=0.44.0,<0.45.0)
+Project-URL: Homepage, https://github.com/skadel/mocksql
+Project-URL: Repository, https://github.com/skadel/mocksql
+Description-Content-Type: text/markdown
+
+# MockSQL — Backend
+
+**FastAPI + LangGraph + CLI**
+
+Ce dossier contient le cœur de MockSQL : l'API REST, le graph LangGraph de génération de données, le CLI, et le package `mocksql-ui` (serveur web + assets React).
+
+---
+
+## Quickstart local
+
+> Prérequis : Python ≥ 3.11, Poetry ≥ 1.8, un projet Google Cloud avec la facturation activée.
+
+### 1. Authentification Google Cloud
+
+```bash
+gcloud auth application-default login
+gcloud config set project <PROJECT_ID>
+```
+
+### 2. Variables d'environnement
+
+```bash
+cp .env.example .env   # puis compléter les valeurs
+```
+
+Variables minimales requises :
+
+```dotenv
+PROJECT_ID=<votre-projet-gcp>
+GOOGLE_CLOUD_PROJECT=<votre-projet-gcp>
+BQ_SCHEMA_BILLING_PROJECT=<votre-projet-gcp>
+DEFAULT_MODEL_NAME=gemini-2.0-flash-lite
+DUCKDB_PATH=data/mocksql.duckdb
+SECRET_KEY=<générer avec : python -c "import secrets; print(secrets.token_hex(32))">
+API_SECRET_KEY=<idem>
+FRONT_URL=http://127.0.0.1:3000
+```
+
+### 3. Installation & lancement
+
+```bash
+python -m venv .venv
+
+# Linux / macOS
+source .venv/bin/activate
+
+# Windows
+.\.venv\Scripts\activate
+
+pip install poetry && poetry install
+
+# Lancer le serveur
+uvicorn server:app --port 8080 --reload
+```
+
+L'API est accessible sur **http://localhost:8080**, le frontend sur **http://localhost:3000** (voir [../front/README.md](../front/README.md)).
+
+---
+
+## Commandes de développement
+
+```bash
+make style    # ruff check + ruff format --check + vulture (code mort)
+make format   # ruff format + ruff check --fix (auto-correction)
+make test     # pytest
+make check    # style + test
+```
+
+Type checking :
+
+```bash
+poetry run mypy build_query/ app/
+```
+
+---
+
+## Packaging
+
+MockSQL produit deux wheels indépendants :
+
+| Wheel | Contenu |
+|-------|---------|
+| `mocksql-*.whl` | CLI + LangGraph core (sans UI) |
+| `mocksql_ui-*.whl` | Serveur web + assets React bundlés |
+
+### Builder les wheels
+
+```bash
+# CLI uniquement
+make build-cli
+
+# CLI + UI (build React inclus — Node.js 18+ requis)
+make build-ui
+```
+
+Les wheels sont générés dans `dist/`.
+
+### Lancer l'UI depuis les wheels
+
+```bash
+pip install dist/mocksql-*.whl dist/mocksql_ui-*.whl
+mocksql ui              # http://localhost:8080/static/
+mocksql ui --port 4000
+mocksql ui --no-browser
+```
+
+---
+
+## Déploiement
+
+### Variables d'environnement (production)
+
+Voir `.env.example` pour la liste complète. En production (Cloud Run), les secrets sont gérés via **Google Secret Manager** ou les variables d'environnement du service.
+
+### Conteneurisation (Docker)
+
+```bash
+docker build -t mocksql-backend .
+docker run -d -p 8080:8080 \
+  -e PROJECT_ID=<votre-projet-gcp> \
+  -e GOOGLE_APPLICATION_CREDENTIALS=/keys/sa.json \
+  -v ~/keys:/keys \
+  mocksql-backend
+```
+
+### Google Cloud Run
+
+#### 1. Build & push
+
+```bash
+gcloud builds submit --tag gcr.io/${PROJECT_ID}/mocksql-backend .
+```
+
+#### 2. Secrets
+
+```bash
+gcloud secrets create mocksql-env --data-file .env
+```
+
+#### 3. Déploiement
+
+```bash
+gcloud run deploy mocksql-backend \
+  --image gcr.io/${PROJECT_ID}/mocksql-backend \
+  --region europe-west1 \
+  --platform managed \
+  --allow-unauthenticated \
+  --memory 1Gi \
+  --service-account mocksql-sa@${PROJECT_ID}.iam.gserviceaccount.com
+```
+
+### Infrastructure Terraform
+
+```bash
+cd ../terraform
+cp variables.example.tfvars my_variables.tfvars
+# compléter project_id, region, etc.
+
+terraform init
+terraform apply --var-file=my_variables.tfvars
+```
+
+Rôles requis pour le compte de service Terraform :
+- Cloud SQL Admin
+- Project IAM Admin
+- Service Account Admin
+- Service Usage Admin
+- BigQuery Admin
+
+#### Permissions PostgreSQL (si Cloud SQL)
+
+```sql
+-- Base mocksql
+GRANT USAGE, CREATE ON SCHEMA public TO "mocksql@${project_id}.iam";
+GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO "mocksql@${project_id}.iam";
+CREATE EXTENSION IF NOT EXISTS vector;
+
+-- Base sqlmeshconf
+GRANT USAGE, CREATE ON SCHEMA public TO "mocksql@${project_id}.iam";
+GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO "mocksql@${project_id}.iam";
+```
+
+---
+
+## Licence
+
+Propriétaire — © 2025 Adel Skhiri. Contact : [skhiriadel92@gmail.com](mailto:skhiriadel92@gmail.com)
+

mocksql-0.1.0/README.md

@@ -0,0 +1,187 @@
+# MockSQL — Backend
+
+**FastAPI + LangGraph + CLI**
+
+Ce dossier contient le cœur de MockSQL : l'API REST, le graph LangGraph de génération de données, le CLI, et le package `mocksql-ui` (serveur web + assets React).
+
+---
+
+## Quickstart local
+
+> Prérequis : Python ≥ 3.11, Poetry ≥ 1.8, un projet Google Cloud avec la facturation activée.
+
+### 1. Authentification Google Cloud
+
+```bash
+gcloud auth application-default login
+gcloud config set project <PROJECT_ID>
+```
+
+### 2. Variables d'environnement
+
+```bash
+cp .env.example .env   # puis compléter les valeurs
+```
+
+Variables minimales requises :
+
+```dotenv
+PROJECT_ID=<votre-projet-gcp>
+GOOGLE_CLOUD_PROJECT=<votre-projet-gcp>
+BQ_SCHEMA_BILLING_PROJECT=<votre-projet-gcp>
+DEFAULT_MODEL_NAME=gemini-2.0-flash-lite
+DUCKDB_PATH=data/mocksql.duckdb
+SECRET_KEY=<générer avec : python -c "import secrets; print(secrets.token_hex(32))">
+API_SECRET_KEY=<idem>
+FRONT_URL=http://127.0.0.1:3000
+```
+
+### 3. Installation & lancement
+
+```bash
+python -m venv .venv
+
+# Linux / macOS
+source .venv/bin/activate
+
+# Windows
+.\.venv\Scripts\activate
+
+pip install poetry && poetry install
+
+# Lancer le serveur
+uvicorn server:app --port 8080 --reload
+```
+
+L'API est accessible sur **http://localhost:8080**, le frontend sur **http://localhost:3000** (voir [../front/README.md](../front/README.md)).
+
+---
+
+## Commandes de développement
+
+```bash
+make style    # ruff check + ruff format --check + vulture (code mort)
+make format   # ruff format + ruff check --fix (auto-correction)
+make test     # pytest
+make check    # style + test
+```
+
+Type checking :
+
+```bash
+poetry run mypy build_query/ app/
+```
+
+---
+
+## Packaging
+
+MockSQL produit deux wheels indépendants :
+
+| Wheel | Contenu |
+|-------|---------|
+| `mocksql-*.whl` | CLI + LangGraph core (sans UI) |
+| `mocksql_ui-*.whl` | Serveur web + assets React bundlés |
+
+### Builder les wheels
+
+```bash
+# CLI uniquement
+make build-cli
+
+# CLI + UI (build React inclus — Node.js 18+ requis)
+make build-ui
+```
+
+Les wheels sont générés dans `dist/`.
+
+### Lancer l'UI depuis les wheels
+
+```bash
+pip install dist/mocksql-*.whl dist/mocksql_ui-*.whl
+mocksql ui              # http://localhost:8080/static/
+mocksql ui --port 4000
+mocksql ui --no-browser
+```
+
+---
+
+## Déploiement
+
+### Variables d'environnement (production)
+
+Voir `.env.example` pour la liste complète. En production (Cloud Run), les secrets sont gérés via **Google Secret Manager** ou les variables d'environnement du service.
+
+### Conteneurisation (Docker)
+
+```bash
+docker build -t mocksql-backend .
+docker run -d -p 8080:8080 \
+  -e PROJECT_ID=<votre-projet-gcp> \
+  -e GOOGLE_APPLICATION_CREDENTIALS=/keys/sa.json \
+  -v ~/keys:/keys \
+  mocksql-backend
+```
+
+### Google Cloud Run
+
+#### 1. Build & push
+
+```bash
+gcloud builds submit --tag gcr.io/${PROJECT_ID}/mocksql-backend .
+```
+
+#### 2. Secrets
+
+```bash
+gcloud secrets create mocksql-env --data-file .env
+```
+
+#### 3. Déploiement
+
+```bash
+gcloud run deploy mocksql-backend \
+  --image gcr.io/${PROJECT_ID}/mocksql-backend \
+  --region europe-west1 \
+  --platform managed \
+  --allow-unauthenticated \
+  --memory 1Gi \
+  --service-account mocksql-sa@${PROJECT_ID}.iam.gserviceaccount.com
+```
+
+### Infrastructure Terraform
+
+```bash
+cd ../terraform
+cp variables.example.tfvars my_variables.tfvars
+# compléter project_id, region, etc.
+
+terraform init
+terraform apply --var-file=my_variables.tfvars
+```
+
+Rôles requis pour le compte de service Terraform :
+- Cloud SQL Admin
+- Project IAM Admin
+- Service Account Admin
+- Service Usage Admin
+- BigQuery Admin
+
+#### Permissions PostgreSQL (si Cloud SQL)
+
+```sql
+-- Base mocksql
+GRANT USAGE, CREATE ON SCHEMA public TO "mocksql@${project_id}.iam";
+GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO "mocksql@${project_id}.iam";
+CREATE EXTENSION IF NOT EXISTS vector;
+
+-- Base sqlmeshconf
+GRANT USAGE, CREATE ON SCHEMA public TO "mocksql@${project_id}.iam";
+GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO "mocksql@${project_id}.iam";
+```
+
+---
+
+## Licence
+
+Propriétaire — © 2025 Adel Skhiri. Contact : [skhiriadel92@gmail.com](mailto:skhiriadel92@gmail.com)

mocksql-0.1.0/app/api/endpoints/messages.py

@@ -0,0 +1,114 @@
+import logging
+from typing import List, Any, Optional
+
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+from storage.config import is_initialized
+from storage.test_repository import get_test, update_test
+from utils.saver import common_history_retriever
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter()
+
+
+class MessageRequest(BaseModel):
+    modelId: str
+
+
+class PatchTestsRequest(BaseModel):
+    sessionId: str
+    tests: List[Any]
+
+
+class PatchSqlRequest(BaseModel):
+    sessionId: str
+    sql: str
+    optimized_sql: str = ""
+    tests: Optional[List[Any]] = None
+    test_results: Optional[List[Any]] = None
+    restored_message_id: Optional[str] = None
+    last_error: Optional[str] = None
+
+
+@router.post("/getMessages")
+async def get_messages(body: MessageRequest):
+    if not is_initialized():
+        raise HTTPException(
+            status_code=400,
+            detail="Projet non initialisé. Lancez 'mocksql init' dans votre répertoire de travail pour commencer.",
+        )
+    try:
+        history = await common_history_retriever(body.modelId, filtered_types=[])
+        if history is None:
+            raise HTTPException(status_code=404, detail="Session not found")
+
+        test = get_test(body.modelId)
+        sql = test.get("sql") if test else None
+        optimized_sql = test.get("optimized_sql") if test else None
+        last_error = test.get("last_error") if test else ""
+        test_results = test.get("test_cases", []) if test else []
+        restored_message_id = test.get("restored_message_id") if test else None
+
+        # Fallback: extract from last results message in history
+        if not test_results:
+            import json
+
+            for msg in reversed(history):
+                if msg.additional_kwargs.get("type") == "results":
+                    try:
+                        test_results = json.loads(msg.content)
+                    except Exception:
+                        pass
+                    break
+
+        return {
+            "messages": history,
+            "sql": sql,
+            "optimized_sql": optimized_sql,
+            "test_results": test_results,
+            "restored_message_id": restored_message_id,
+            "last_error": last_error or "",
+            "sql_history": [],
+        }
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.exception(
+            "Erreur lors du chargement des messages pour la session %s", body.modelId
+        )
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.patch("/models/sql")
+async def patch_model_sql(body: PatchSqlRequest):
+    try:
+        updates: dict = {
+            "sql": body.sql,
+            "optimized_sql": body.optimized_sql,
+        }
+        if body.tests is not None:
+            updates["test_cases"] = body.tests
+        if body.test_results is not None:
+            updates["test_cases"] = body.test_results
+        if body.restored_message_id is not None:
+            updates["restored_message_id"] = body.restored_message_id or None
+        if body.last_error is not None:
+            updates["last_error"] = body.last_error
+
+        update_test(body.sessionId, updates)
+        return {"ok": True}
+    except Exception as e:
+        print(e)
+        raise HTTPException(status_code=500, detail="Internal Server Error")
+
+
+@router.patch("/models/tests")
+async def patch_model_tests(body: PatchTestsRequest):
+    try:
+        update_test(body.sessionId, {"test_cases": body.tests})
+        return {"ok": True}
+    except Exception as e:
+        print(e)
+        raise HTTPException(status_code=500, detail="Internal Server Error")

mocksql-0.1.0/app/api/endpoints/models.py

@@ -0,0 +1,117 @@
+from fastapi import APIRouter, HTTPException
+from fastapi.responses import JSONResponse
+from pydantic import BaseModel
+
+from storage.test_repository import (
+    list_models,
+    list_all_tests,
+    list_tests,
+    get_test,
+    create_test,
+    delete_test,
+    _test_path,
+    _read_json,
+)
+
+router = APIRouter()
+
+
+# ---------------------------------------------------------------------------
+# SQL Models (fichiers physiques dans models_path)
+# ---------------------------------------------------------------------------
+
+
+@router.get("/models")
+async def get_models():
+    """Liste les fichiers .sql disponibles dans models_path.
+
+    Each entry includes session_id / updated_at / test_name when a test already
+    exists for that model, so the frontend can redirect instead of regenerating.
+    Tested models are returned first (sorted by updated_at desc), then untested.
+    """
+    try:
+        sql_files = list_models()
+        tested = []
+        untested = []
+        for f in sql_files:
+            p = _test_path(f["name"])
+            if p.exists():
+                data = _read_json(p)
+                if data:
+                    tested.append(
+                        {
+                            **f,
+                            "session_id": data.get("test_id"),
+                            "updated_at": data.get("updated_at"),
+                            "test_name": data.get("test_name"),
+                        }
+                    )
+                    continue
+            untested.append(f)
+        tested.sort(key=lambda x: x.get("updated_at") or "", reverse=True)
+        return tested + untested
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.get("/tests/all")
+async def get_all_tests():
+    """Liste toutes les sessions de test (tous models confondus), triées par date décroissante."""
+    try:
+        return list_all_tests()
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+# ---------------------------------------------------------------------------
+# Tests (fichiers dans .mocksql/tests/)
+# ---------------------------------------------------------------------------
+
+
+class CreateTestRequest(BaseModel):
+    model_name: str
+
+
+@router.get("/tests")
+async def get_tests(model_name: str):
+    """Liste tous les tests pour un model donné."""
+    try:
+        return list_tests(model_name)
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.get("/test/{session_id}")
+async def get_test_route(session_id: str, model_name: str = None):
+    try:
+        test = get_test(session_id, model_name)
+        if test is None:
+            raise HTTPException(status_code=404, detail="Test not found")
+        return test
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.post("/tests")
+async def create_test_route(body: CreateTestRequest):
+    """Crée un nouveau test pour un model. Retourne le test_id (= session_id)."""
+    try:
+        test = create_test(body.model_name)
+        return JSONResponse(content=test, status_code=201)
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.delete("/tests/{session_id}")
+async def delete_test_route(session_id: str, model_name: str):
+    try:
+        ok = delete_test(session_id, model_name)
+        if not ok:
+            raise HTTPException(status_code=404, detail="Test not found")
+        return JSONResponse(status_code=204, content={"message": "Test deleted"})
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))