pyarchinit-mini 2.4.7__tar.gz → 2.7.1__tar.gz

{pyarchinit_mini-2.4.7 → pyarchinit_mini-2.7.1}/CHANGELOG.md

@@ -1,3 +1,154 @@
+## [2.7.1] - 2026-05-19
+
+### Fixed (IT)
+- **AI Matrix Import** post-merge hotfix: la 2.7.0 era inservibile in produzione per 4 bug scoperti al primo uso reale.
+  - `g.db_session` non viene mai popolato in produzione: ora si usa `_get_session()` context manager (stesso pattern di `yed_import_routes`).
+  - Le chiavi API AI vivono nella tabella `app_settings` (gestita dalla UI Admin), non solo in env: bootstrap automatico in `os.environ` prima di chiamare l'extractor.
+  - Schema `us_table` corretto: `descrizione` (Text) invece di `d_stratigrafica` (String); `fase_finale` invece del campo inventato `fase_recente`; cast int→str per le fasi (colonne TEXT).
+  - Schema `us_relationships_table` corretto: colonna `sito` singola (non `sito_from`/`sito_to`), `relationship_type` (non `tipo_relazione`), `id_relationship` (non `id_rel`).
+  - Endpoint redirect: `us_list` (bare route) invece di `us.list_us` (blueprint inesistente).
+  - Template (upload/preview/rejected) ora estendono `base.html` con styling Bootstrap, navbar e icone Font Awesome.
+
+### Fixed (EN)
+- **AI Matrix Import** post-merge hotfix: 2.7.0 was unusable in production due to 4 bugs caught on first real use.
+  - `g.db_session` was never populated in production: now uses `_get_session()` context manager (same pattern as `yed_import_routes`).
+  - AI API keys live in the `app_settings` table (managed via UI Admin), not only in env: automatic bootstrap into `os.environ` before extractor call.
+  - Corrected `us_table` schema: `descrizione` (Text) instead of `d_stratigrafica` (String); `fase_finale` instead of made-up `fase_recente`; int→str cast for phases (TEXT columns).
+  - Corrected `us_relationships_table` schema: single `sito` column (not `sito_from`/`sito_to`), `relationship_type` (not `tipo_relazione`), `id_relationship` (not `id_rel`).
+  - Redirect endpoint: `us_list` (bare route) instead of `us.list_us` (non-existent blueprint).
+  - Templates (upload/preview/rejected) now extend `base.html` with Bootstrap styling, navbar, and Font Awesome icons.
+
+## [2.7.0] - 2026-05-18
+
+### Added (IT)
+- **AI Matrix Import** (`/matrix-import/`): nuova feature che usa Vision API
+  (Anthropic Claude Sonnet 4.7 o OpenAI GPT-5.5) per estrarre US e relazioni
+  stratigrafiche da un'immagine di un Harris matrix. Include validation gate
+  (l'AI rifiuta immagini che non sono matrici), preview editabile prima del
+  commit, e auto-creazione del sito se nuovo.
+- L'immagine sorgente viene salvata come media legato al sito.
+- Toggle Anthropic / OpenAI nel form di upload.
+- Required env vars: `ANTHROPIC_API_KEY` e/o `OPENAI_API_KEY`, `AI_PROVIDER` opzionale.
+
+### Added (EN)
+- **AI Matrix Import** (`/matrix-import/`): new feature that uses Vision API
+  (Anthropic Claude Sonnet 4.7 or OpenAI GPT-5.5) to extract stratigraphic
+  units and relationships from an image of a Harris matrix. Includes a
+  validation gate (the AI rejects non-matrix images), editable preview
+  before commit, and auto-creation of the site if new.
+- Source image saved as media linked to the site.
+- Anthropic / OpenAI toggle in the upload form.
+- Required env vars: `ANTHROPIC_API_KEY` and/or `OPENAI_API_KEY`, optional `AI_PROVIDER`.
+
+## [2.6.0] - 2026-05-18
+
+### Changed (IT)
+- Editor Harris Matrix Creator: gli archi stratigrafici sono ora resi con
+  routing ortogonale (90°, `curve-style: 'taxi'`) invece che con curve
+  bezier diagonali. Allineato al `y:PolyLineEdge` di yEd usato da
+  pyarchinit QGIS plugin.
+- `harris_swimlane/harris_layout.py`: defaults aumentati a `v_gap=40`
+  (era 20) e `h_gap=50` (era 30) per dare respiro agli archi ortogonali.
+  Chiamanti che passano i parametri esplicitamente non sono toccati.
+
+### Changed (EN)
+- Harris Matrix Creator editor: stratigraphic edges now render with
+  right-angle routing (`curve-style: 'taxi'`) instead of bezier curves,
+  matching the `y:PolyLineEdge` style used by pyarchinit QGIS.
+- `harris_swimlane/harris_layout.py`: defaults bumped to `v_gap=40`
+  (was 20) and `h_gap=50` (was 30) to give orthogonal edges breathing
+  room. Explicit callers are unaffected.
+
+## [2.5.1] - 2026-05-18
+
+### Added (IT)
+- Pottery list: anteprima media in linea con mini-carousel CSS scroll-snap
+  (thumb 48x48 + GLightbox). N+1 evitato con pre-load batch via
+  `MediaService.get_media_for_entity_ids`.
+- Pottery filtri: nuovi input `id_number` e `anno` (esatti, integer cast
+  con guard `try/except`).
+- Pottery detail: galleria thumbnail inline (120x120, GLightbox) sopra i
+  campi della scheda.
+- Nuovo blueprint `pyarchinit_mini/i18n/lang_routes.py` con
+  `POST /set-language/<lang>` che salva session e fa redirect al referrer.
+  Lo switcher di lingua diventa `<form>` POST: preserva la query string
+  corrente (filtri/ricerca non si perdono più al cambio lingua).
+
+### Changed (IT)
+- `templates/components/language_switcher.html`: `<a>` sostituiti con form POST.
+- `PotteryService._FILTERABLE_INT` aggiunto con `id_number`, `anno`.
+- `pottery_routes.FILTER_KEYS` esteso con `id_number`, `anno`.
+
+### Added (EN)
+- Pottery list: inline media preview with CSS scroll-snap mini-carousel
+  (48x48 thumbs + GLightbox). Batch pre-load avoids N+1.
+- Pottery filters: new `id_number` + `anno` exact-match inputs with
+  integer cast guard.
+- Pottery detail: inline thumbnail gallery (120x120, GLightbox).
+- New `pyarchinit_mini/i18n/lang_routes.py` blueprint: `POST /set-language/<lang>`
+  saves session + redirects to referrer, preserving the URL query string
+  (filters/search no longer lost on language switch).
+
+### Changed (EN)
+- Language switcher templates: anchors replaced with POST forms.
+- `PotteryService._FILTERABLE_INT` introduces `id_number`, `anno`.
+- `pottery_routes.FILTER_KEYS` extended.
+
+## [2.5.0] - 2026-05-18
+
+### Added (IT)
+- Editor swimlane: dropdown "Group by" con 9 valori (period_phase | struttura
+  | attivita | settore | area | ambient | saggio | quad_par | none).
+  Cambiando valore le lane si ricostruiscono e i nodi US si ridistribuiscono
+  con un layout Harris-classico server-side.
+- Nuovo modulo `harris_swimlane/harris_layout.py` — ordinamento topologico
+  per dipendenze stratigrafiche (`overlies` / `is_after`), recent in alto.
+- Nuovo `graphml_io/yed_importer.py` — parse + build_import_plan +
+  apply_import_plan per round-trip da Extended Matrix → DB. Idempotente,
+  preview 2-fasi, upsert by `node_uuid`.
+- Nuova pagina `/import-graphml/` (upload + preview + apply). Sidebar entry
+  sia in navbar che laterale (sezione Tools).
+
+### Changed (IT)
+- `graphml_io/yed_writer.py` riscritto: emette file Extended Matrix
+  byte-compat con pyarchinit QGIS (38 keys d0..d37, `y:TableNode
+  YED_TABLE_NODE`, `pyarchinit.epochs_meta`, `pyarchinit.*` per ogni nodo
+  US). Vecchio `write_yed_graphml` mantenuto come wrapper deprecato.
+- API `/harris-creator/api/load/<site>` accetta `?group_by=...` query
+  param (default `period_phase`, retro-compat).
+- API `/harris-creator/api/export/<site>/yed-graphml` accetta `?group_by=...`
+  e popola `epochs_meta` da `periodizzazione_table`.
+- Path output cambia in `data/exports/harris_yed/<slug>-extmatrix.graphml`
+  (era `<slug>-harris-yed.graphml`).
+
+### Added (EN)
+- Editor "Group by" dropdown with 9 values; lanes rebuild + Harris-classic
+  re-layout on change.
+- New `harris_swimlane/harris_layout.py` — server-side topological
+  positioning.
+- New `graphml_io/yed_importer.py` — round-trip yEd → DB.
+- New `/import-graphml/` page (upload + preview + apply).
+
+### Changed (EN)
+- `graphml_io/yed_writer.py` rewritten — emits 38-key Extended Matrix
+  byte-compat with pyarchinit.
+- `/api/load` and `/api/export` accept `?group_by=...`.
+- Export output path renamed `*-extmatrix.graphml`.
+
+## [2.4.8] - 2026-05-18
+
+### Fixed (IT)
+- `loadExistingData` (legacy loader) ora deduplica le relationship triple
+  `(from_us, to_us, relationship)` e suffissa gli ID Cytoscape che
+  collidono quando la stessa coppia compare con tipi diversi. Risolve
+  l'errore "Can not create second element with ID `edge_131_136`" che
+  abortiva l'intero forEach e lasciava il canvas senza archi.
+
+### Fixed (EN)
+- Legacy `loadExistingData` now deduplicates relationship triples and
+  suffixes colliding edge IDs. Fixes "second element with ID
+  edge_X_Y" errors that previously aborted edge loading.
+
 ## [2.4.7] - 2026-05-18
 
 ### Fixed (IT)

{pyarchinit_mini-2.4.7/pyarchinit_mini.egg-info → pyarchinit_mini-2.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyarchinit-mini
-Version: 2.4.7
+Version: 2.7.1
 Summary: Lightweight archaeological data management system with REST API, Web UI, Desktop GUI, CLI, multi-user authentication, real-time collaboration, analytics dashboard, PyArchInit import/export, Heriverse/ATON export integration, and 3D CRUD viewer
 Home-page: https://github.com/enzococca/pyarchinit-mini
 Author: PyArchInit Team

{pyarchinit_mini-2.4.7 → pyarchinit_mini-2.7.1}/pyarchinit_mini/__init__.py

@@ -23,7 +23,7 @@ Features:
 - Scalable and modular architecture
 """
 
-__version__ = "2.4.7"
+__version__ = "2.7.1"
 __author__ = "PyArchInit Team"
 __email__ = "enzo.ccc@gmail.com"
 

pyarchinit_mini-2.7.1/pyarchinit_mini/ai_matrix/__init__.py

@@ -0,0 +1 @@
+"""AI Matrix Import — Vision API based stratigraphic matrix extraction."""

pyarchinit_mini-2.7.1/pyarchinit_mini/ai_matrix/apply.py

@@ -0,0 +1,102 @@
+"""Commit an AIPlan to the database: site auto-create, US dedupe, edge int guard."""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import datetime
+
+from sqlalchemy import text
+
+from pyarchinit_mini.ai_matrix.plan import AIPlan
+
+
+VALID_UNIT_TYPES = {"USM", "USR", "US"}
+VALID_REL_TYPES = {
+    "copre", "coperto da",
+    "taglia", "tagliato da",
+    "riempie", "riempito da",
+    "si appoggia", "gli si appoggia",
+    "uguale a",
+}
+
+
+@dataclass
+class ApplyResult:
+    us_imported: int = 0
+    us_skipped: int = 0
+    edges_imported: int = 0
+    edges_skipped: int = 0
+    site_created: bool = False
+
+
+def apply_ai_plan(plan: AIPlan, sito: str, db_session) -> ApplyResult:
+    now = datetime.utcnow()
+    result = ApplyResult()
+
+    # Step 1: site_table auto-create
+    existing = db_session.execute(
+        text("SELECT sito FROM site_table WHERE sito = :s"), {"s": sito}
+    ).fetchone()
+    if not existing:
+        db_session.execute(text("""
+            INSERT INTO site_table (sito, descrizione, created_at, updated_at)
+            VALUES (:sito, :desc, :now, :now)
+        """), {"sito": sito, "desc": "Auto-creato da AI Matrix Import", "now": now})
+        result.site_created = True
+
+    # Step 2: us_table
+    for u in plan.us:
+        if not u.us_num or not u.area or not u.unit_type:
+            result.us_skipped += 1
+            continue
+        unit_type = u.unit_type if u.unit_type in VALID_UNIT_TYPES else "US"
+        existing_us = db_session.execute(
+            text("SELECT us FROM us_table WHERE sito = :s AND us = :u"),
+            {"s": sito, "u": u.us_num},
+        ).fetchone()
+        if existing_us:
+            result.us_skipped += 1
+            continue
+        db_session.execute(text("""
+            INSERT INTO us_table
+                (sito, area, us, unita_tipo, descrizione,
+                 fase_finale, fase_iniziale, created_at, updated_at)
+            VALUES (:sito, :area, :us, :ut, :desc, :ffin, :fini, :now, :now)
+        """), {
+            "sito": sito, "area": u.area, "us": u.us_num, "ut": unit_type,
+            "desc": u.descrizione,
+            "ffin": str(u.fase_recente) if u.fase_recente is not None else None,
+            "fini": str(u.fase_iniziale) if u.fase_iniziale is not None else None,
+            "now": now,
+        })
+        result.us_imported += 1
+
+    # Step 3: us_relationships_table
+    # Schema: sito (single), us_from (int), us_to (int), relationship_type, created_at, updated_at
+    for e in plan.edges:
+        try:
+            us_from = int(e.us_from)
+            us_to = int(e.us_to)
+        except (ValueError, TypeError):
+            result.edges_skipped += 1
+            continue
+        if e.tipo not in VALID_REL_TYPES:
+            result.edges_skipped += 1
+            continue
+        existing_rel = db_session.execute(text("""
+            SELECT 1 FROM us_relationships_table
+            WHERE sito = :s
+              AND us_from = :uf AND us_to = :ut AND relationship_type = :t
+        """), {"s": sito, "uf": us_from, "ut": us_to, "t": e.tipo}).fetchone()
+        if existing_rel:
+            result.edges_skipped += 1
+            continue
+        db_session.execute(text("""
+            INSERT INTO us_relationships_table
+                (sito, us_from, us_to, relationship_type,
+                 created_at, updated_at)
+            VALUES (:s, :uf, :ut, :t, :now, :now)
+        """), {"s": sito, "uf": us_from, "ut": us_to, "t": e.tipo, "now": now})
+        result.edges_imported += 1
+
+    db_session.commit()
+    return result

pyarchinit_mini-2.7.1/pyarchinit_mini/ai_matrix/plan.py

@@ -0,0 +1,54 @@
+"""Dataclasses for AI Matrix Import plan + extraction results."""
+from __future__ import annotations
+
+from dataclasses import dataclass, field, asdict
+
+
+@dataclass
+class USRow:
+    us_num: str
+    area: str | None
+    unit_type: str
+    descrizione: str
+    fase_recente: int
+    fase_iniziale: int
+
+
+@dataclass
+class EdgeRow:
+    us_from: str
+    us_to: str
+    tipo: str
+
+
+@dataclass
+class AIPlan:
+    detected_site: str | None
+    detected_area: str | None
+    us: list[USRow] = field(default_factory=list)
+    edges: list[EdgeRow] = field(default_factory=list)
+
+    def as_dict(self) -> dict:
+        return {
+            "detected_site": self.detected_site,
+            "detected_area": self.detected_area,
+            "us": [asdict(u) for u in self.us],
+            "edges": [asdict(e) for e in self.edges],
+        }
+
+    @classmethod
+    def from_dict(cls, d: dict) -> "AIPlan":
+        return cls(
+            detected_site=d.get("detected_site"),
+            detected_area=d.get("detected_area"),
+            us=[USRow(**u) for u in d.get("us", [])],
+            edges=[EdgeRow(**e) for e in d.get("edges", [])],
+        )
+
+
+@dataclass
+class ExtractResult:
+    rejected: bool
+    reason: str
+    confidence: float
+    plan: AIPlan | None

pyarchinit_mini-2.7.1/pyarchinit_mini/ai_matrix/vision_extractor.py

@@ -0,0 +1,197 @@
+"""Vision API extractor for Harris matrix images."""
+from __future__ import annotations
+
+import base64
+import json
+import os
+
+from pyarchinit_mini.ai_matrix.plan import AIPlan, USRow, EdgeRow, ExtractResult
+
+
+CONFIDENCE_THRESHOLD = 0.7
+
+
+def _parse_response(raw: str) -> ExtractResult:
+    """Parse the AI's JSON response into an ExtractResult.
+
+    Applies the validation gate: rejects non-matrix images and low-confidence
+    results before constructing an AIPlan.
+    """
+    try:
+        data = json.loads(raw)
+    except json.JSONDecodeError as e:
+        return ExtractResult(
+            rejected=True,
+            reason=f"AI returned non-JSON response: {e}",
+            confidence=0.0,
+            plan=None,
+        )
+
+    try:
+        confidence = float(data.get("confidence", 0.0))
+        is_matrix = bool(data.get("is_harris_matrix", False))
+
+        if not is_matrix or confidence < CONFIDENCE_THRESHOLD:
+            return ExtractResult(
+                rejected=True,
+                reason=data.get("reason", "Immagine non riconosciuta come Harris matrix"),
+                confidence=confidence,
+                plan=None,
+            )
+
+        plan = AIPlan(
+            detected_site=data.get("detected_site"),
+            detected_area=data.get("detected_area"),
+            us=[USRow(
+                us_num=str(u.get("us_num", "")),
+                area=u.get("area"),
+                unit_type=str(u.get("unit_type", "US")),
+                descrizione=str(u.get("descrizione", "")),
+                fase_recente=int(u.get("fase_recente", 1)),
+                fase_iniziale=int(u.get("fase_iniziale", 1)),
+            ) for u in data.get("us", [])],
+            edges=[EdgeRow(
+                us_from=str(edge_e.get("us_from", "")),
+                us_to=str(edge_e.get("us_to", "")),
+                tipo=str(edge_e.get("tipo", "")),
+            ) for edge_e in data.get("edges", [])],
+        )
+        return ExtractResult(rejected=False, reason="OK", confidence=confidence, plan=plan)
+    except (ValueError, TypeError) as e:
+        return ExtractResult(
+            rejected=True,
+            reason=f"AI returned malformed values: {e}",
+            confidence=0.0,
+            plan=None,
+        )
+
+
+MAX_IMAGE_BYTES = 10 * 1024 * 1024  # 10MB
+API_TIMEOUT_S = 90
+
+
+SYSTEM_PROMPT = """You are an expert in archaeological stratigraphy and Harris matrix diagrams.
+
+Your task: analyze the attached image and determine if it depicts a Harris matrix (stratigraphic diagram). If yes, extract all stratigraphic units (US/SU) and their relationships.
+
+A Harris matrix is a diagram with:
+- Numbered rectangles/boxes (= stratigraphic units, US)
+- Lines/arrows connecting them (= temporal relationships)
+- Hierarchical layout: recent units on top, older on bottom
+
+Vocabulary for unit_type:
+- USM = muratura (masonry/wall)
+- USR = revestimento (covering/coating)
+- US  = stratigraphic deposit (default if ambiguous)
+
+Vocabulary for edge tipo (Italian stratigraphic vocabulary):
+- "copre"        = covers
+- "taglia"       = cuts
+- "riempie"      = fills
+- "si appoggia"  = leans on
+- "uguale a"     = same as
+
+OUTPUT: strict JSON only, no markdown, no commentary. Schema:
+{
+  "is_harris_matrix": bool,
+  "confidence": float 0..1,
+  "reason": str,
+  "detected_site": str | null,
+  "detected_area": str | null,
+  "us": [{"us_num": str, "area": str | null, "unit_type": str,
+          "descrizione": str, "fase_recente": int, "fase_iniziale": int}],
+  "edges": [{"us_from": str, "us_to": str, "tipo": str}]
+}
+
+If is_harris_matrix is false, return empty us and edges arrays and explain in 'reason' what you see instead."""
+
+
+def _detect_media_type(image_bytes: bytes) -> str:
+    """Return MIME type based on magic bytes. Defaults to PNG."""
+    if image_bytes[:3] == b"\xff\xd8\xff":
+        return "image/jpeg"
+    if image_bytes[:8] == b"\x89PNG\r\n\x1a\n":
+        return "image/png"
+    if image_bytes[:6] in (b"GIF87a", b"GIF89a"):
+        return "image/gif"
+    if image_bytes[:4] == b"RIFF" and image_bytes[8:12] == b"WEBP":
+        return "image/webp"
+    return "image/png"
+
+
+def extract(image_bytes: bytes, text: str | None, provider: str) -> ExtractResult:
+    """Send image to Vision API and return parsed ExtractResult.
+
+    Args:
+        image_bytes: raw image content (<=10MB)
+        text: optional user-provided context
+        provider: 'anthropic' or 'openai'
+    """
+    if len(image_bytes) > MAX_IMAGE_BYTES:
+        return ExtractResult(
+            rejected=True,
+            reason="Immagine troppo grande (max 10MB)",
+            confidence=0.0,
+            plan=None,
+        )
+    if provider == "anthropic":
+        raw = _call_anthropic_vision(image_bytes, text)
+    elif provider == "openai":
+        raw = _call_openai_vision(image_bytes, text)
+    else:
+        raise ValueError(f"unknown provider: {provider}")
+    return _parse_response(raw)
+
+
+def _call_anthropic_vision(image_bytes: bytes, text: str | None) -> str:
+    try:
+        import anthropic
+    except ImportError:
+        raise RuntimeError("Il package 'anthropic' non è installato. pip install anthropic")
+    api_key = os.environ.get("ANTHROPIC_API_KEY")
+    if not api_key:
+        raise RuntimeError("ANTHROPIC_API_KEY non configurata")
+    client = anthropic.Anthropic(api_key=api_key, timeout=API_TIMEOUT_S)
+    response = client.messages.create(
+        model="claude-sonnet-4-7",
+        max_tokens=4096,
+        system=SYSTEM_PROMPT,
+        messages=[{
+            "role": "user",
+            "content": [
+                {"type": "image", "source": {
+                    "type": "base64",
+                    "media_type": _detect_media_type(image_bytes),
+                    "data": base64.b64encode(image_bytes).decode(),
+                }},
+                {"type": "text", "text": text or "Analyze the matrix."},
+            ],
+        }],
+    )
+    return response.content[0].text
+
+
+def _call_openai_vision(image_bytes: bytes, text: str | None) -> str:
+    try:
+        import openai
+    except ImportError:
+        raise RuntimeError("Il package 'openai' non è installato. pip install openai")
+    api_key = os.environ.get("OPENAI_API_KEY")
+    if not api_key:
+        raise RuntimeError("OPENAI_API_KEY non configurata")
+    client = openai.OpenAI(api_key=api_key, timeout=API_TIMEOUT_S)
+    media_type = _detect_media_type(image_bytes)
+    response = client.chat.completions.create(
+        model="gpt-5.5",
+        response_format={"type": "json_object"},
+        messages=[
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user", "content": [
+                {"type": "image_url", "image_url": {
+                    "url": f"data:{media_type};base64,{base64.b64encode(image_bytes).decode()}"
+                }},
+                {"type": "text", "text": text or "Analyze the matrix."},
+            ]},
+        ],
+    )
+    return response.choices[0].message.content