arango-query-core 0.2.3__tar.gz → 0.2.5__tar.gz

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/CHANGELOG.md

@@ -6,6 +6,63 @@ exact versions (contextual-data-fabric CC-9) and re-run their golden /
 eval suites on every bump — pre-1.0, **minor** bumps may break the API
 and this file must say so explicitly.
 
+## [0.2.5] - 2026-10-26
+
+### Added
+
+- **Supertypes (label hierarchies).** A supertype is a label that stands for
+  other labels, such as LDBC's `Message`, which is every `Post` and every
+  `Comment`. It is mapped with one of two new entity styles,
+  `SUPERTYPE_STYLES`:
+  - `MULTI_LABEL`: the members share a collection and list their labels in
+    an array field (`labelsField`, `labelValue`);
+  - `UNION`: the members have their own collections (`members`).
+- **New resolver methods.** `MappingResolver.is_supertype()` and
+  `supertype_members()`. The latter flattens nested supertypes and refuses
+  unknown members and cycles with `INVALID_MAPPING`. A supertype without
+  properties of its own has its members' properties.
+- **`edge_constrains_target`** treats a supertype target as guaranteed when
+  every class an edge endpoint can be is a member: `HAS_CREATOR`'s domain
+  `[Post, Comment]` guarantees `Message`.
+- **OWL/Turtle** writes and reads the hierarchy as `rdfs:subClassOf`.
+  `turtle_to_mapping` no longer invents a `collectionName` for a `UNION`
+  supertype. The rdflib parser does not read `rdfs:subClassOf` yet.
+
+The change is additive: mappings without the new styles resolve exactly as
+before. A consumer that switches on `style` and does not handle the new
+values rejects them rather than misreading them.
+
+## [0.2.4] - 2026-10-21
+
+### Added
+
+- `QueryLanguageAdapter` may define `normalize(query) -> str`, an optional seam
+  `NLQueryEngine` calls on every candidate after the fence is stripped and
+  before `validate`. Use it for a deterministic repair that would otherwise
+  cost an LLM retry; nl2cypher uses it to rewrite labels the schema does not
+  have. What it returns is validated, quoted in retries and returned. A
+  `normalize` that raises is logged and the candidate kept. It is not a
+  protocol member, so existing adapters are unaffected. #8
+- `NLQueryEngine.generate(..., user_hint="")` appends a caller hint to the
+  first attempt's user message only, for example "your previous query
+  failed with …" when re-asking. It leaves the system prompt (the cacheable
+  prefix), few-shot retrieval and postcondition context untouched. #10
+- `NLQueryEngine(retry_provider_errors=True)`: a provider call that raises (a
+  timeout, a 5xx) uses one attempt of `max_retries`, and the loop goes on.
+  `NLResult.provider_errors` counts them. If no attempt returned a response,
+  the last exception is re-raised. The default (`False`) keeps today's
+  behaviour: the first exception propagates. #9
+
+### Fixed
+
+- The engine finds the query in the first fenced block anywhere in the
+  response. It used to strip a fence only when the response started with
+  one, so "Here is the query: ```…```" failed validation and spent a retry.
+  A response that opens a fence without closing it yields what follows the
+  fence, and a response with no fence is returned as is, as before. When a
+  response has two fenced blocks, the first is taken; it used to be
+  everything from the first opening fence to the last closing one. #7
+
 ## [0.2.3] - 2026-10-21
 
 ### Added

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: arango-query-core
-Version: 0.2.3
+Version: 0.2.5
 Summary: Shared conceptual-model and NL substrate for ArangoDB query transpilers (arango-cypher-py, arango-sparql-py)
 Project-URL: Repository, https://github.com/ArthurKeen/arango-query-core
 Project-URL: Changelog, https://github.com/ArthurKeen/arango-query-core/blob/main/CHANGELOG.md

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/arango_query_core/__init__.py

@@ -4,6 +4,7 @@ from .exec import AqlExecutor, explain_aql, safe_execute
 from .extensions import ExtensionPolicy, ExtensionRegistry
 from .mapping import (
     COLLECTION_NAME_RE,
+    SUPERTYPE_STYLES,
     IndexInfo,
     MappingBundle,
     MappingResolver,
@@ -23,6 +24,7 @@ except ImportError:
 
 __all__ = [
     "COLLECTION_NAME_RE",
+    "SUPERTYPE_STYLES",
     "AqlExecutor",
     "AqlFragment",
     "AqlQuery",

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/arango_query_core/mapping.py

@@ -10,6 +10,28 @@ from .errors import CoreError
 
 JsonObj = dict[str, Any]
 
+#: Entity mapping styles for a *supertype*: a label whose instances are the
+#: instances of other labels (LDBC's ``Message`` is every ``Post`` and every
+#: ``Comment``).
+#:
+#: * ``MULTI_LABEL`` — the members share one collection and every document
+#:   lists all its labels in an array field::
+#:
+#:       {"style": "MULTI_LABEL", "collectionName": "nodes",
+#:        "labelsField": "labels", "labelValue": "Message"}
+#:
+#:   matches documents of ``nodes`` whose ``labels`` contain ``"Message"``.
+#: * ``UNION`` — the members live in their own collections::
+#:
+#:       {"style": "UNION", "members": ["Post", "Comment"]}
+#:
+#: Members come from the physical ``members`` list or, failing that, the
+#: conceptual entity's ``subtypes``. They must be mapped labels; a member may
+#: itself be a supertype (members are flattened). Consumers that switch on
+#: ``style`` and do not know these values fail closed, which is why supertypes
+#: get styles of their own rather than extra fields on ``LABEL``/``COLLECTION``.
+SUPERTYPE_STYLES = frozenset({"MULTI_LABEL", "UNION"})
+
 
 # ArangoDB collection-name grammar (per the official docs):
 #   * starts with a letter or underscore
@@ -391,6 +413,13 @@ class MappingResolver:
             return {}
 
         props_raw = mapping.get("properties", {})
+        if not props_raw and mapping.get("style") in SUPERTYPE_STYLES:
+            # A supertype without its own property list has its members' properties.
+            merged: dict[str, PropertyInfo] = {}
+            for member in self.supertype_members(label_or_type):
+                for name, info in self.resolve_properties(member).items():
+                    merged.setdefault(name, info)
+            return merged
         if not isinstance(props_raw, dict):
             return {}
 
@@ -518,7 +547,16 @@ class MappingResolver:
         be in the set **and** every class in the union must map to the same
         physical collection (otherwise multiple collections are reachable and
         IS_SAME_COLLECTION is still needed).
+
+        A supertype target is guaranteed when every class the endpoint can be
+        is one of its members: ``HAS_CREATOR``'s domain ``[Post, Comment]``
+        guarantees ``Message``.
         """
+        if self.is_supertype(target_label):
+            classes = [endpoint] if isinstance(endpoint, str) else list(endpoint or [])
+            members = set(self.supertype_members(target_label))
+            return bool(classes) and all(c in members for c in classes)
+
         if isinstance(endpoint, str):
             return endpoint == target_label
 
@@ -645,6 +683,68 @@ class MappingResolver:
         """Check if any index on the relationship's edge collection has VCI enabled."""
         return any(idx.vci for idx in self.resolve_indexes(rel_type))
 
+    # ------------------------------------------------------------------
+    # Supertypes (label hierarchies)
+    # ------------------------------------------------------------------
+
+    def is_supertype(self, label: str) -> bool:
+        """True when *label* is mapped with a supertype style (see :data:`SUPERTYPE_STYLES`)."""
+        try:
+            return self.resolve_entity(label).get("style") in SUPERTYPE_STYLES
+        except CoreError:
+            return False
+
+    def _declared_members(self, label: str, emap: JsonObj) -> list[str]:
+        members = emap.get("members")
+        if members is None:
+            for ent in self.bundle.conceptual_schema.get("entities", []) or []:
+                if isinstance(ent, dict) and ent.get("name") == label:
+                    members = ent.get("subtypes")
+                    break
+        if not isinstance(members, list) or not members or not all(isinstance(m, str) and m for m in members):
+            raise CoreError(
+                f"Supertype {label!r} declares no members: give the physical mapping a 'members' list "
+                f"or the conceptual entity a 'subtypes' list",
+                code="INVALID_MAPPING",
+            )
+        return members
+
+    def supertype_members(self, label: str) -> list[str]:
+        """The concrete (non-supertype) labels a supertype stands for, in declaration order.
+
+        Nested supertypes are flattened. Raises ``INVALID_MAPPING`` for a
+        member that is not mapped or a cycle; raises ``MAPPING_NOT_FOUND`` if
+        *label* is not mapped, and ``INVALID_MAPPING`` if it is not a supertype.
+        """
+        out: list[str] = []
+
+        def visit(name: str, path: tuple[str, ...]) -> None:
+            if name in path:
+                raise CoreError(
+                    f"Supertype cycle: {' -> '.join((*path, name))}",
+                    code="INVALID_MAPPING",
+                )
+            try:
+                emap = self.resolve_entity(name)
+            except CoreError as e:
+                if path:
+                    raise CoreError(
+                        f"Supertype {path[-1]!r} lists member {name!r}, which is not mapped",
+                        code="INVALID_MAPPING",
+                    ) from e
+                raise
+            if emap.get("style") not in SUPERTYPE_STYLES:
+                if not path:
+                    raise CoreError(f"{name!r} is not a supertype", code="INVALID_MAPPING")
+                if name not in out:
+                    out.append(name)
+                return
+            for member in self._declared_members(name, emap):
+                visit(member, (*path, name))
+
+        visit(label, ())
+        return out
+
     def all_entity_labels(self) -> list[str]:
         """Return all entity type labels defined in the mapping."""
         entities = self.bundle.physical_mapping.get("entities", {})

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/arango_query_core/nl/engine.py

@@ -17,6 +17,8 @@ signature breaks.
 
 from __future__ import annotations
 
+import logging
+import re
 from collections.abc import Sequence
 from dataclasses import dataclass, field
 from typing import Any
@@ -33,6 +35,8 @@ from .seams import GuardrailVerdict, QueryLanguageAdapter, ValidationResult
 
 _USAGE_KEYS = ("prompt_tokens", "completion_tokens", "total_tokens", "cached_tokens")
 
+logger = logging.getLogger(__name__)
+
 
 @dataclass
 class NLResult:
@@ -58,6 +62,8 @@ class NLResult:
     guardrail: GuardrailVerdict | None = None
     validation: ValidationResult | None = None
     postcondition: PostconditionViolation | None = None
+    provider_errors: int = 0
+    """Provider calls that raised and were retried (``retry_provider_errors``)."""
 
 
 @dataclass
@@ -91,11 +97,26 @@ class NLQueryEngine:
     guardrail_context: dict[str, Any] = field(default_factory=dict)
     postconditions: Sequence[Postcondition] = ()
     postcondition_mapping: Any = None
+    retry_provider_errors: bool = False
+    """When True, a provider call that raises (a timeout, a 5xx) uses one
+    attempt of the ``max_retries`` budget and the loop goes on; the count is
+    on :attr:`NLResult.provider_errors`. If no attempt ever returned a
+    response, the last exception is re-raised. False (the default) lets the
+    first exception propagate, as before."""
+
+    def generate(self, question: str, *, schema_context: str = "", user_hint: str = "") -> NLResult:
+        """Generate a query for *question*.
 
-    def generate(self, question: str, *, schema_context: str = "") -> NLResult:
+        *user_hint* is appended to the **first** attempt's user message only —
+        e.g. "your previous query failed with ..." when a caller re-asks after a
+        failure. It leaves the system prompt (the cacheable prefix), few-shot
+        retrieval and postcondition context untouched; retries carry their own
+        "previous query was rejected" text.
+        """
         result = NLResult()
         system = self._system_prompt(question, schema_context)
-        user = question
+        user = f"{question}\n\n{user_hint}" if user_hint else question
+        last_provider_error: Exception | None = None
         failure: ValidationResult | None = None
         violation: PostconditionViolation | None = None
         candidate = ""
@@ -104,7 +125,7 @@ class NLQueryEngine:
         fallback: tuple[str, ValidationResult, PostconditionViolation] | None = None
 
         for attempt in range(self.max_retries + 1):
-            if attempt > 0:
+            if attempt > 0 and failure is not None:
                 # A postcondition rejection carries its own correction text;
                 # otherwise fall back to the adapter's syntactic repair hint.
                 if violation is not None:
@@ -119,11 +140,26 @@ class NLQueryEngine:
                     f"Problem: {hint}\n"
                     f"Return a corrected query only."
                 )
+            if attempt > 0:
                 result.retries = attempt
-            content, usage = self.provider.generate(system, user)
+            try:
+                content, usage = self.provider.generate(system, user)
+            except Exception as exc:
+                if not self.retry_provider_errors:
+                    raise
+                result.provider_errors += 1
+                last_provider_error = exc
+                logger.warning(
+                    "NL provider call failed (attempt %d/%d): %s: %s",
+                    attempt + 1,
+                    self.max_retries + 1,
+                    type(exc).__name__,
+                    exc,
+                )
+                continue
             for key in _USAGE_KEYS:
                 setattr(result, key, getattr(result, key) + int(usage.get(key, 0) or 0))
-            candidate = _strip_code_fence(content, self.adapter.language)
+            candidate = self._normalize(_strip_code_fence(content, self.adapter.language))
             failure = self.adapter.validate(candidate)
             result.validation = failure
             if not failure.ok:
@@ -156,6 +192,9 @@ class NLQueryEngine:
                 candidate, result.validation, result.postcondition = fallback
                 failure, violation = result.validation, None
 
+        if failure is None and last_provider_error is not None:
+            # Every attempt raised: nothing to report but the failure itself.
+            raise last_provider_error
         if failure is None or not failure.ok:
             result.error = failure.error if failure else "provider returned nothing"
             return result
@@ -176,6 +215,21 @@ class NLQueryEngine:
         result.ok = True
         return result
 
+    def _normalize(self, candidate: str) -> str:
+        """The adapter's optional deterministic repair (``normalize`` seam),
+        applied before validation; the result is what is validated, quoted in
+        retries and returned. A ``normalize`` that raises is logged and the
+        candidate is kept — a broken repair must not abort generation."""
+        normalize = getattr(self.adapter, "normalize", None)
+        if not callable(normalize):
+            return candidate
+        try:
+            normalized = normalize(candidate)
+        except Exception as exc:  # noqa: BLE001 - see docstring
+            logger.warning("adapter.normalize raised %s: %s; keeping the candidate", type(exc).__name__, exc)
+            return candidate
+        return normalized if isinstance(normalized, str) else candidate
+
     def _system_prompt(self, question: str, schema_context: str) -> str:
         sections = [self.adapter.grammar_prompt_section(schema_context)]
         index = self.adapter.few_shot_index()
@@ -208,14 +262,22 @@ class NLQueryEngine:
         return "\n\n".join(s for s in sections if s)
 
 
+_FENCED_BLOCK = re.compile(r"```[^\n`]*\n(.*?)```", re.DOTALL)
+
+
 def _strip_code_fence(content: str, language: str) -> str:
     """Extract the query from a fenced code block, tolerating bare text.
 
     LLMs frequently wrap the answer in ```<lang> fences despite the
-    output-format contract; accept fenced, language-tagged-fenced, and
-    bare responses uniformly so the validator sees only query text.
+    output-format contract, and often lead with prose ("Here is the
+    query:") or follow with an explanation. The first fenced block wherever
+    it appears is the query; a response that only opens a fence yields what
+    follows it; a response without a fence is returned as is.
     """
     text = content.strip()
+    block = _FENCED_BLOCK.search(text)
+    if block is not None:
+        return block.group(1).strip()
     if not text.startswith("```"):
         return text
     first_newline = text.find("\n")

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/arango_query_core/nl/seams.py

@@ -74,6 +74,15 @@ class QueryLanguageAdapter(Protocol):
     ``language`` is the lowercase tag used for fenced code blocks and
     telemetry (``"cypher"``, ``"sparql"``). All other members map
     one-to-one onto the numbered seams in the module docstring.
+
+    **Optional:** ``normalize(query: str) -> str``. When an adapter defines
+    it, :class:`~arango_query_core.nl.NLQueryEngine` calls it on every
+    candidate, after the code fence is stripped and before :meth:`validate`,
+    for a deterministic repair that would otherwise cost an LLM retry (e.g.
+    rewriting a label the schema does not have to the one it does). What it
+    returns is what is validated, quoted in a retry and returned. It is not a
+    protocol member, so existing adapters conform unchanged; a ``normalize``
+    that raises is logged and the candidate kept.
     """
 
     language: str

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/arango_query_core/owl_turtle.py

@@ -62,6 +62,13 @@ def mapping_to_turtle(bundle: MappingBundle) -> str:
                 lines.append(f':{name} phys:typeField "{tf}" .')
             if tv:
                 lines.append(f':{name} phys:typeValue "{tv}" .')
+        if style == "MULTI_LABEL":
+            for annot in ("labelsField", "labelValue"):
+                if emap.get(annot):
+                    lines.append(f':{name} phys:{annot} "{emap[annot]}" .')
+        # A supertype's members are its OWL subclasses.
+        for member in emap.get("members") or entity_def.get("subtypes") or []:
+            lines.append(f":{member} rdfs:subClassOf :{name} .")
 
         for prop in entity_def.get("properties", []):
             pname = prop.get("name", "")
@@ -127,6 +134,7 @@ def turtle_to_mapping(turtle: str) -> MappingBundle:
     obj_props: dict[str, dict[str, str]] = {}
     data_props: list[dict[str, Any]] = []
     annotations: dict[str, dict[str, str]] = {}
+    subclasses: dict[str, list[str]] = {}  # supertype -> member classes (rdfs:subClassOf)
 
     current_subject: str | None = None
     current_kind: str | None = None  # "class", "objprop", "dataprop"
@@ -150,6 +158,11 @@ def turtle_to_mapping(turtle: str) -> MappingBundle:
             obj_props.setdefault(current_subject, {})
             continue
 
+        m_sub = re.match(r":(\w+)\s+rdfs:subClassOf\s+:(\w+)\s*\.", stripped)
+        if m_sub:
+            subclasses.setdefault(m_sub.group(2), []).append(m_sub.group(1))
+            continue
+
         m_dataprop = re.match(r":(\w+)\s+a\s+owl:DatatypeProperty", stripped)
         if m_dataprop:
             current_subject = m_dataprop.group(1)
@@ -188,6 +201,8 @@ def turtle_to_mapping(turtle: str) -> MappingBundle:
             "typeValue",
             "edgeCollectionName",
             "attributeName",
+            "labelsField",
+            "labelValue",
         ):
             m_a = re.match(rf':(\w+)\s+phys:{annot}\s+"([^"]+)"', stripped)
             if m_a:
@@ -200,15 +215,24 @@ def turtle_to_mapping(turtle: str) -> MappingBundle:
     for name in classes:
         own = [dp for dp in data_props if name in dp["domains"]]
         props = [{"name": dp["name"], "type": _from_xsd_type(dp.get("range", "xsd:string"))} for dp in own]
-        entities_cs.append({"name": name, "labels": [name], "properties": props})
+        cs_entry: dict[str, Any] = {"name": name, "labels": [name], "properties": props}
+        if subclasses.get(name):
+            cs_entry["subtypes"] = list(subclasses[name])
+        entities_cs.append(cs_entry)
         ann = annotations.get(name, {})
-        pm_entry: dict[str, Any] = {
-            "style": ann.get("mappingStyle", "COLLECTION"),
-            "collectionName": ann.get("collectionName", name.lower() + "s"),
-        }
-        if pm_entry["style"] == "LABEL":
+        style = ann.get("mappingStyle", "COLLECTION")
+        pm_entry: dict[str, Any] = {"style": style}
+        if style == "UNION":
+            # A UNION supertype has no collection of its own; its members are its subclasses.
+            pm_entry["members"] = list(subclasses.get(name, []))
+        else:
+            pm_entry["collectionName"] = ann.get("collectionName", name.lower() + "s")
+        if style == "LABEL":
             pm_entry["typeField"] = ann.get("typeField", "type")
             pm_entry["typeValue"] = ann.get("typeValue", name)
+        if style == "MULTI_LABEL":
+            pm_entry["labelsField"] = ann.get("labelsField", "labels")
+            pm_entry["labelValue"] = ann.get("labelValue", name)
         fields = {
             dp["name"]: {"field": annotations[dp["name"]]["attributeName"]}
             for dp in own

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/docs/PRD.md

@@ -116,20 +116,20 @@ to prevent a three-way fork.
 
 | Module / subpackage | Purpose |
 | --- | --- |
-| `mapping` | `MappingBundle`, `MappingResolver`, `MappingSource`, `PropertyInfo`, `IndexInfo`, `RelationshipStats`; `mapping_from_wire_dict`, `mapping_hash`; collection-name grammar guard (`COLLECTION_NAME_RE`, `is_valid_collection_name`) |
+| `mapping` | `MappingBundle`, `MappingResolver` (incl. supertypes: `SUPERTYPE_STYLES`, `is_supertype`, `supertype_members`), `MappingSource`, `PropertyInfo`, `IndexInfo`, `RelationshipStats`; `mapping_from_wire_dict`, `mapping_hash`; collection-name grammar guard (`COLLECTION_NAME_RE`, `is_valid_collection_name`) |
 | `owl_turtle` / `owl_rdflib` | OWL/Turtle ⇄ MappingBundle round-trip; rdflib parser behind the `owl` extra (import degrades to `None` without it) |
 | `aql` | `AqlFragment`, `AqlQuery` value types |
 | `exec` | `AqlExecutor` protocol, `safe_execute` (bind-var reference checking), `explain_aql` |
 | `extensions` | `ExtensionRegistry` / `ExtensionPolicy` for `arango.*` extension functions |
 | `errors` | `CoreError` shared base |
-| `nl.engine` | `NLQueryEngine` — prompt assembly, few-shot retrieval, provider call, generate→validate→repair loop; `NLResult` |
+| `nl.engine` | `NLQueryEngine` — prompt assembly, few-shot retrieval, provider call, generate→validate→repair loop; `NLResult`. The query is the first fenced block anywhere in a response (prose around it is ignored), else the bare response. An adapter's optional `normalize` seam repairs each candidate before validation. `generate(user_hint=...)` appends a caller hint to the first user message only, leaving the cacheable system prefix alone. With `retry_provider_errors=True`, a provider call that raises uses one attempt of the budget (`NLResult.provider_errors`), and the last error is re-raised only if no attempt returned a response; the default propagates the first error. |
 | `nl.providers` | `LLMProvider` protocol; OpenAI / OpenRouter / Anthropic implementations (Anthropic with prompt-cache system-block splitting); env-driven `get_llm_provider` |
 | `nl.fewshot` | `FewShotIndex` over `(question, query)` pairs (legacy corpus keys accepted); `BM25Retriever` (rank_bm25, `nl` extra) and `DenseRetriever` (sentence-transformers, `dense` extra), selected via `mode="bm25"\|"dense"\|"auto"`; `cached_few_shot_index` memoizes the expensive dense build |
 | `nl.grounding` | Seam 6/7 retrieval: `LabelIndex`/`GroundedEntity` (entity grounding) and `PredicateIndex`/`GroundedPredicate` (predicate/schema-convention grounding, renders IRI local names not labels), sharing one token-substring scorer |
 | `nl.pathindex` | Seam 8: `ClassPathIndex`/`ClassPath` — bounded shortest predicate paths between two seam-6/7-anchored classes over the TBox's class-connectivity graph |
 | `nl.postconditions` | Caller-supplied semantic invariants (`Postcondition`, `PostconditionContext`, `PostconditionViolation`, `run_postconditions`) enforced through the engine's own retry budget; fails closed on exhaustion, unless a valid candidate drew only a soft violation (`PostconditionViolation.soft`); the first such candidate is returned, with the violation on `NLResult.postcondition` |
 | `nl.synthbank` | `SHAPE_CATALOG`/`ShapeTemplate` — 9 ontology-agnostic compositional query-shape templates for synthetic few-shot bank generation |
-| `nl.seams` | `QueryLanguageAdapter` — the eight language-specific seams: (1) grammar prompt section, (2) few-shot corpus, (3) syntax validator, (4) repair rules keyed on validator errors, (5) guardrail checks, (6) entity grounding index, (7) predicate grounding index, (8) relationship-path grounding index. Plus `ValidationResult`, `GuardrailVerdict` |
+| `nl.seams` | `QueryLanguageAdapter` — the eight language-specific seams: (1) grammar prompt section, (2) few-shot corpus, (3) syntax validator, (4) repair rules keyed on validator errors, (5) guardrail checks, (6) entity grounding index, (7) predicate grounding index, (8) relationship-path grounding index. Optional, outside the protocol: `normalize(query) -> str`, a deterministic repair applied before validation. Plus `ValidationResult`, `GuardrailVerdict` |
 
 ## 5. Requirements
 
@@ -149,6 +149,24 @@ to prevent a three-way fork.
 - **R5 — Deterministic contract types.** `mapping_hash` over a bundle
   is stable across processes (canonical JSON) so consumers can use it
   for cache keys and change detection.
+- **R7 — Label hierarchies (supertypes).** A supertype is a label whose
+  instances are the instances of other labels (LDBC's `Message` is every
+  `Post` and every `Comment`).
+  - **Styles.** It is mapped with one of two entity styles (`SUPERTYPE_STYLES`):
+    - `MULTI_LABEL`: the members share one collection, and documents list
+      their labels in an array field (`labelsField`, `labelValue`).
+    - `UNION`: the members have collections of their own (`members`).
+  - **Members** come from the physical `members` list or the conceptual
+    entity's `subtypes`.
+  - **Resolver.** `MappingResolver.supertype_members()` flattens nested
+    supertypes, and refuses unknown members and cycles (`INVALID_MAPPING`).
+    A supertype without properties of its own has its members' properties.
+    `edge_constrains_target` treats a supertype target as guaranteed when
+    every class an edge endpoint can be is a member.
+  - **New styles fail closed.** Supertypes get styles of their own, so a
+    consumer that does not handle them rejects the mapping instead of
+    misreading it.
+  - **OWL.** OWL/Turtle round-trips the hierarchy as `rdfs:subClassOf`.
 - **R6 — Quality gates.** ruff (lint + format), mypy, and pytest run in
   CI on every push/PR; all three must pass before a release is tagged.
 

{arango_query_core-0.2.3 → arango_query_core-0.2.5}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "arango-query-core"
-version = "0.2.3"
+version = "0.2.5"
 description = "Shared conceptual-model and NL substrate for ArangoDB query transpilers (arango-cypher-py, arango-sparql-py)"
 readme = "README.md"
 license = { file = "LICENSE" }

arango_query_core-0.2.5/tests/test_mapping_supertypes.py

@@ -0,0 +1,190 @@
+"""Supertypes (label hierarchies): the MULTI_LABEL and UNION entity styles."""
+
+from __future__ import annotations
+
+import pytest
+
+from arango_query_core import SUPERTYPE_STYLES, CoreError, MappingBundle, MappingResolver
+
+
+def _bundle(
+    entities: dict, relationships: dict | None = None, conceptual: dict | None = None
+) -> MappingBundle:
+    return MappingBundle(
+        conceptual_schema=conceptual or {"entities": [], "relationships": []},
+        physical_mapping={"entities": entities, "relationships": relationships or {}},
+        metadata={},
+    )
+
+
+_POST = {
+    "collectionName": "post",
+    "style": "COLLECTION",
+    "properties": {"id": {"field": "id"}, "imageFile": {}},
+}
+_COMMENT = {
+    "collectionName": "comment",
+    "style": "COLLECTION",
+    "properties": {"id": {"field": "id"}, "length": {}},
+}
+_HAS_CREATOR = {
+    "edgeCollectionName": "has_creator",
+    "style": "DEDICATED_COLLECTION",
+    "domain": ["Post", "Comment"],
+    "range": "Person",
+}
+
+
+def _pg(**extra) -> MappingResolver:
+    return MappingResolver(
+        _bundle(
+            {
+                "Post": _POST,
+                "Comment": _COMMENT,
+                "Person": {"collectionName": "person", "style": "COLLECTION"},
+                "Message": {"style": "UNION", "members": ["Post", "Comment"]},
+                **extra,
+            },
+            {"HAS_CREATOR": _HAS_CREATOR},
+        )
+    )
+
+
+def test_both_styles_are_supertype_styles():
+    assert {"MULTI_LABEL", "UNION"} == SUPERTYPE_STYLES
+
+
+def test_union_members_and_is_supertype():
+    r = _pg()
+    assert r.is_supertype("Message")
+    assert not r.is_supertype("Post")
+    assert not r.is_supertype("Nowhere")
+    assert r.supertype_members("Message") == ["Post", "Comment"]
+
+
+def test_members_may_come_from_the_conceptual_subtypes():
+    r = MappingResolver(
+        _bundle(
+            {
+                "Post": {
+                    "collectionName": "nodes",
+                    "style": "LABEL",
+                    "typeField": "type",
+                    "typeValue": "Post",
+                },
+                "Comment": {
+                    "collectionName": "nodes",
+                    "style": "LABEL",
+                    "typeField": "type",
+                    "typeValue": "Comment",
+                },
+                "Message": {
+                    "style": "MULTI_LABEL",
+                    "collectionName": "nodes",
+                    "labelsField": "labels",
+                    "labelValue": "Message",
+                },
+            },
+            conceptual={"entities": [{"name": "Message", "subtypes": ["Post", "Comment"]}]},
+        )
+    )
+    assert r.supertype_members("Message") == ["Post", "Comment"]
+
+
+def test_nested_supertypes_are_flattened_without_duplicates():
+    r = _pg(
+        Reply={"style": "UNION", "members": ["Comment"]},
+        Content={"style": "UNION", "members": ["Message", "Reply", "Post"]},
+    )
+    assert r.supertype_members("Content") == ["Post", "Comment"]
+
+
+@pytest.mark.parametrize(
+    "extra,needle",
+    [
+        ({"Message": {"style": "UNION", "members": ["Post", "Ghost"]}}, "'Ghost', which is not mapped"),
+        ({"Message": {"style": "UNION"}}, "declares no members"),
+        (
+            {
+                "Message": {"style": "UNION", "members": ["Thread"]},
+                "Thread": {"style": "UNION", "members": ["Message"]},
+            },
+            "cycle",
+        ),
+    ],
+)
+def test_invalid_hierarchies_are_refused(extra, needle):
+    r = _pg(**extra)
+    with pytest.raises(CoreError) as exc:
+        r.supertype_members("Message")
+    assert exc.value.code == "INVALID_MAPPING"
+    assert needle in str(exc.value)
+
+
+def test_a_concrete_label_is_not_a_supertype():
+    with pytest.raises(CoreError, match="is not a supertype"):
+        _pg().supertype_members("Post")
+
+
+def test_supertype_without_properties_has_its_members_properties():
+    props = _pg().resolve_properties("Message")
+    assert set(props) == {"id", "imageFile", "length"}
+
+
+def test_an_edge_whose_endpoints_are_all_members_guarantees_the_supertype():
+    r = _pg()
+    assert r.edge_constrains_target("HAS_CREATOR", "Message", "INBOUND")
+    assert r.edge_constrains_target("HAS_CREATOR", "Person", "OUTBOUND")
+
+
+def test_an_edge_that_can_reach_a_non_member_does_not():
+    r = _pg(Forum={"collectionName": "forum", "style": "COLLECTION"})
+    r.bundle.physical_mapping["relationships"]["HAS_TAG"] = {
+        "edgeCollectionName": "has_tag",
+        "style": "DEDICATED_COLLECTION",
+        "domain": ["Forum", "Post", "Comment"],
+        "range": "Tag",
+    }
+    assert not r.edge_constrains_target("HAS_TAG", "Message", "INBOUND")
+
+
+def test_existing_mappings_are_unaffected():
+    r = MappingResolver(_bundle({"Post": _POST}))
+    assert not r.is_supertype("Post")
+    assert set(r.resolve_properties("Post")) == {"id", "imageFile"}
+
+
+def test_union_and_multi_label_round_trip_through_owl_as_subclasses():
+    from arango_query_core import mapping_to_turtle, turtle_to_mapping
+
+    bundle = _bundle(
+        {
+            "Post": {"collectionName": "post", "style": "COLLECTION"},
+            "Comment": {"collectionName": "comment", "style": "COLLECTION"},
+            "Message": {"style": "UNION", "members": ["Post", "Comment"]},
+            "Thing": {
+                "style": "MULTI_LABEL",
+                "collectionName": "nodes",
+                "labelsField": "labels",
+                "labelValue": "Thing",
+            },
+        },
+        conceptual={
+            "entities": [
+                {"name": "Post"},
+                {"name": "Comment"},
+                {"name": "Message"},
+                {"name": "Thing", "subtypes": ["Post"]},
+            ],
+            "relationships": [],
+        },
+    )
+    turtle = mapping_to_turtle(bundle)
+    assert ":Post rdfs:subClassOf :Message ." in turtle
+    back = turtle_to_mapping(turtle)
+    pm = back.physical_mapping["entities"]
+    assert pm["Message"] == {"style": "UNION", "members": ["Post", "Comment"]}
+    assert pm["Thing"]["labelsField"] == "labels" and pm["Thing"]["labelValue"] == "Thing"
+    r = MappingResolver(back)
+    assert r.supertype_members("Message") == ["Post", "Comment"]
+    assert r.supertype_members("Thing") == ["Post"]

arango_query_core-0.2.5/tests/test_nl_engine_extraction_and_resilience.py

@@ -0,0 +1,165 @@
+"""Engine behaviour arango-cypher-py needed to drop its own NL loop
+(arango-solutions-org/arango-cypher#24), one issue each:
+
+* #7 — the query is found in a fenced block anywhere in the response;
+* #8 — an adapter's optional ``normalize`` seam repairs a candidate before
+  validation;
+* #9 — ``retry_provider_errors`` lets a provider call that raised use the
+  retry budget instead of aborting generation;
+* #10 — ``generate(user_hint=...)`` adds a hint to the first user message.
+
+Fakes follow ``tests/test_nl_engine.py``: a scripted provider and an adapter
+that accepts any query containing ``SELECT``.
+"""
+
+from __future__ import annotations
+
+import logging
+
+import pytest
+from test_nl_engine import FakeAdapter  # tests/ is on sys.path (pytest rootdir import)
+
+from arango_query_core.nl import NLQueryEngine
+from arango_query_core.nl.engine import _strip_code_fence
+
+GOOD = "SELECT ?s WHERE { ?s ?p ?o }"
+
+
+class ScriptedProvider:
+    """Answers in order; an ``Exception`` in the script is raised instead."""
+
+    def __init__(self, *answers: str | Exception) -> None:
+        self._answers = list(answers)
+        self.calls: list[tuple[str, str]] = []
+
+    def generate(self, system: str, user: str) -> tuple[str, dict[str, int]]:
+        self.calls.append((system, user))
+        answer = self._answers.pop(0)
+        if isinstance(answer, Exception):
+            raise answer
+        return answer, {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15, "cached_tokens": 0}
+
+
+# --- #7: the fenced query, wherever it is ---------------------------------------
+
+
+@pytest.mark.parametrize(
+    ("response", "query"),
+    [
+        (f"Here is the query:\n```sparql\n{GOOD}\n```\nIt lists every triple.", GOOD),
+        (f"```\n{GOOD}\n```", GOOD),
+        (f"First:\n```sparql\n{GOOD}\n```\nOr:\n```sparql\nASK {{}}\n```", GOOD),
+        (f"```sparql\n{GOOD}", GOOD),  # a fence that is never closed
+        (GOOD, GOOD),  # no fence at all
+    ],
+    ids=["prose-around", "untagged", "first-of-two", "unclosed", "bare"],
+)
+def test_the_query_is_found_wherever_the_fence_is(response: str, query: str) -> None:
+    assert _strip_code_fence(response, "sparql") == query
+
+
+def test_prose_before_the_fence_costs_no_retry() -> None:
+    provider = ScriptedProvider(f"Sure! Here you go:\n```sparql\n{GOOD}\n```")
+    result = NLQueryEngine(provider=provider, adapter=FakeAdapter()).generate("everything")
+    assert result.ok and result.query == GOOD and result.retries == 0
+    assert len(provider.calls) == 1
+
+
+# --- #8: the normalize seam ------------------------------------------------------
+
+
+class NormalizingAdapter(FakeAdapter):
+    """Repairs a lowercase keyword the validator would reject."""
+
+    def __init__(self, *, fail: bool = False) -> None:
+        super().__init__()
+        self.seen: list[str] = []
+        self._fail = fail
+
+    def normalize(self, query: str) -> str:
+        self.seen.append(query)
+        if self._fail:
+            raise ValueError("normalizer bug")
+        return query.replace("select", "SELECT")
+
+
+def test_a_normalized_candidate_is_what_is_validated_and_returned() -> None:
+    adapter = NormalizingAdapter()
+    provider = ScriptedProvider("```sparql\nselect ?s WHERE { ?s ?p ?o }\n```")
+    result = NLQueryEngine(provider=provider, adapter=adapter).generate("everything")
+    assert result.ok and result.query == GOOD and result.retries == 0
+    assert adapter.seen == ["select ?s WHERE { ?s ?p ?o }"]  # after the fence is stripped
+
+
+def test_a_retry_quotes_the_normalized_candidate() -> None:
+    adapter = NormalizingAdapter()
+    provider = ScriptedProvider("ask {}", GOOD)
+    NLQueryEngine(provider=provider, adapter=adapter, max_retries=1).generate("everything")
+    assert "```sparql\nask {}\n```" in provider.calls[1][1]
+
+
+def test_a_normalize_that_raises_keeps_the_candidate(caplog: pytest.LogCaptureFixture) -> None:
+    provider = ScriptedProvider(GOOD)
+    with caplog.at_level(logging.WARNING):
+        result = NLQueryEngine(provider=provider, adapter=NormalizingAdapter(fail=True)).generate(
+            "everything"
+        )
+    assert result.ok and result.query == GOOD
+    assert "normalizer bug" in caplog.text
+
+
+# --- #9: provider failures and the retry budget ---------------------------------
+
+
+def test_by_default_a_provider_failure_propagates() -> None:
+    provider = ScriptedProvider(TimeoutError("timed out"), GOOD)
+    with pytest.raises(TimeoutError):
+        NLQueryEngine(provider=provider, adapter=FakeAdapter()).generate("everything")
+    assert len(provider.calls) == 1
+
+
+def test_opted_in_a_failed_call_uses_one_attempt() -> None:
+    provider = ScriptedProvider(TimeoutError("timed out"), GOOD)
+    engine = NLQueryEngine(provider=provider, adapter=FakeAdapter(), retry_provider_errors=True)
+    result = engine.generate("everything")
+    assert result.ok and result.query == GOOD
+    assert (result.provider_errors, result.retries, result.total_tokens) == (1, 1, 15)
+    assert provider.calls[1][1] == "everything"  # nothing to repair yet: the plain question again
+
+
+def test_opted_in_every_call_failing_reraises_the_last_error() -> None:
+    provider = ScriptedProvider(TimeoutError("first"), ConnectionError("second"), RuntimeError("third"))
+    engine = NLQueryEngine(
+        provider=provider, adapter=FakeAdapter(), max_retries=2, retry_provider_errors=True
+    )
+    with pytest.raises(RuntimeError, match="third"):
+        engine.generate("everything")
+    assert len(provider.calls) == 3
+
+
+def test_opted_in_a_failure_after_a_bad_candidate_reports_that_candidate() -> None:
+    provider = ScriptedProvider("ask {}", TimeoutError("timed out"))
+    engine = NLQueryEngine(
+        provider=provider, adapter=FakeAdapter(), max_retries=1, retry_provider_errors=True
+    )
+    result = engine.generate("everything")
+    assert not result.ok and result.error == "not a SELECT" and result.provider_errors == 1
+
+
+# --- #10: a hint on the first user message ---------------------------------------
+
+
+def test_a_user_hint_rides_on_the_first_message_only() -> None:
+    provider = ScriptedProvider("ask {}", GOOD)
+    engine = NLQueryEngine(provider=provider, adapter=FakeAdapter(), max_retries=1)
+    result = engine.generate("everything", user_hint="Your previous query timed out. Add LIMIT.")
+    assert result.ok
+    assert provider.calls[0][1] == "everything\n\nYour previous query timed out. Add LIMIT."
+    assert "timed out" not in provider.calls[1][1]
+
+
+def test_a_user_hint_leaves_the_system_prompt_alone() -> None:
+    plain, hinted = ScriptedProvider(GOOD), ScriptedProvider(GOOD)
+    NLQueryEngine(provider=plain, adapter=FakeAdapter()).generate("everything")
+    NLQueryEngine(provider=hinted, adapter=FakeAdapter()).generate("everything", user_hint="hint")
+    assert plain.calls[0][0] == hinted.calls[0][0]

arango_query_core-0.2.3/.claude/settings.json

@@ -1,8 +0,0 @@
-{
-  "permissions": {
-    "allow": [
-      "Bash(gh run *)",
-      "Bash(git worktree *)"
-    ]
-  }
-}