symbolicai 1.1.1__tar.gz → 1.2.0__tar.gz

{symbolicai-1.1.1 → symbolicai-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: symbolicai
-Version: 1.1.1
+Version: 1.2.0
 Summary: A Neurosymbolic Perspective on Large Language Models
 Author-email: Marius-Constantin Dinu <marius@extensity.ai>, Leoveanu-Condrei Claudiu <leo@extensity.ai>
 License: BSD 3-Clause License

{symbolicai-1.1.1 → symbolicai-1.2.0}/docs/source/ENGINES/indexing_engine.md

@@ -58,24 +58,12 @@ os.environ["INDEXING_ENGINE_API_KEY"] = "your-api-key"
 
 ### Basic Usage
 
-The Qdrant engine automatically registers when a Qdrant server is available. Use it through the `Interface` abstraction:
-
-```python
-from symai.interfaces import Interface
-
-# Qdrant engine is used automatically when server is running
-db = Interface('naive_vectordb', index_name="my_collection")
-db("Hello world", operation="add")
-result = db("Hello", operation="search", top_k=5)
-print(result.value)  # list of relevant matches
-```
-
-### Advanced Usage: Direct Engine Access
-
-For more control, use the Qdrant engine directly:
+The Qdrant engine is used directly via the `QdrantIndexEngine` class:
 
 ```python
+import asyncio
 from symai.backend.engines.index.engine_qdrant import QdrantIndexEngine
+from symai import Symbol
 
 # Initialize engine
 engine = QdrantIndexEngine(
@@ -86,6 +74,33 @@ engine = QdrantIndexEngine(
     index_top_k=5,  # default top-k for searches
     index_metric="Cosine"  # Cosine, Dot, or Euclidean
 )
+
+async def basic_usage():
+    # Create a collection
+    await engine.create_collection("my_collection", vector_size=1536)
+
+    # Add documents using chunk_and_upsert
+    num_chunks = await engine.chunk_and_upsert(
+        collection_name="my_collection",
+        text="Hello world, this is a test document.",
+        metadata={"source": "example"}
+    )
+
+    # Search for similar documents
+    query = Symbol("Hello")
+    query_embedding = query.embedding
+    results = await engine.search(
+        collection_name="my_collection",
+        query_vector=query_embedding,
+        limit=5
+    )
+
+    # Print results
+    for result in results:
+        print(f"Score: {result.score}")
+        print(f"Text: {result.payload.get('text', '')}")
+
+asyncio.run(basic_usage())
 ```
 
 ### Collection Management

{symbolicai-1.1.1 → symbolicai-1.2.0}/docs/source/ENGINES/neurosymbolic_engine.md

@@ -8,6 +8,7 @@ Depending on which backend you configure (OpenAI/GPT, Claude, Gemini, Deepseek,
 * Local engines (llamacpp, HuggingFace) do *not* yet support token counting, JSON format enforcement, or vision inputs in the same way.
 * Groq engine requires a special format for the `NEUROSYMBOLIC_ENGINE_MODEL` key: `groq:model_id`. E.g., `groq:qwen/qwen3-32b`.
 * Cerebras engine requires a special format for the `NEUROSYMBOLIC_ENGINE_MODEL` key: `cerebras:model_id`. E.g., `cerebras:gpt-oss-120b`.
+* OpenAI Responses API engine requires the `responses:` prefix: `responses:model_id`. E.g., `responses:gpt-4.1`, `responses:o3-mini`. This uses OpenAI's newer `/v1/responses` endpoint instead of `/v1/chat/completions`.
 * Token‐truncation and streaming are handled automatically but may vary in behavior by engine.
 
 > ❗️**NOTE**❗️the most accurate documentation is the _code_, so be sure to check out the tests. Look for the `mandatory` mark since those are the features that were tested and are guaranteed to work.
@@ -97,9 +98,9 @@ assert blocks[0].name == "get_stock_price"
 
 ---
 
-## Thinking Trace (Claude, Gemini, Deepseek, Groq's reasoning models)
+## Thinking Trace (Claude, Gemini, Deepseek, Groq, OpenAI Responses)
 
-Some engines (Anthropic's Claude, Google's Gemini, Deepseek) can return an internal **thinking trace** that shows how they arrived at an answer. To get it, you must:
+Some engines (Anthropic's Claude, Google's Gemini, Deepseek, OpenAI Responses API with reasoning models) can return an internal **thinking trace** that shows how they arrived at an answer. To get it, you must:
 
 1. Pass `return_metadata=True`.
 2. Pass a `thinking=` configuration if required.
@@ -190,6 +191,24 @@ print(metadata["thinking"])
 
 For Cerebras backends, `symai` collects the reasoning trace from either the dedicated `reasoning` field on the message (when present) or from `<think>…</think>` blocks embedded in the content. In both cases the trace is exposed as `metadata["thinking"]` and removed from the final user-facing text.
 
+### OpenAI Responses API (reasoning models)
+
+```python
+from symai import Symbol
+
+# responses:o3-mini or responses:gpt-5
+res, metadata = Symbol("Topic: Disneyland") \
+    .query(
+      "Write a dystopic take on the topic.",
+      return_metadata=True,
+      reasoning={"effort": "medium"}  # optional: low, medium, or high
+    )
+print(res)
+print(metadata["thinking"])
+```
+
+For OpenAI Responses API with reasoning models (e.g., `o3-mini`, `o3`, `o4-mini`, `gpt-5`, `gpt-5.1`), the thinking trace is extracted from the reasoning summary in the response output.
+
 ---
 
 ## JSON‐Only Responses
@@ -305,7 +324,8 @@ aggregated_usage = RuntimeInfo(
     cached_tokens=0,
     total_calls=0,
     total_tokens=0,
-    cost_estimate=0
+    cost_estimate=0,
+    extras={}
 )
 for (engine_name, model_id), engine_data in usage_per_engine.items():
     pricing_key = (engine_name, model_id)
@@ -320,6 +340,21 @@ print(aggregated_usage)
 
 This approach provides a robust way to monitor and control costs associated with LLM API usage, especially when making multiple calls. Remember to update the `pricing` dictionary with the current rates for the models you are using. The `estimate_cost` function can also be customized to reflect complex pricing schemes (e.g., different rates for different models, image tokens, etc.).
 
+#### The `extras` Field
+
+`RuntimeInfo` includes an `extras` dictionary for engine-specific usage metrics that don't fit the standard token fields. For example, when using `ParallelEngine`, the tracker captures additional usage items like `sku_search` (number of search operations) and `sku_extract_excerpts` (number of excerpt extractions):
+
+```python
+# After tracking parallel search operations:
+usage_per_engine = RuntimeInfo.from_tracker(tracker, 0)
+for (engine_name, model_id), engine_data in usage_per_engine.items():
+    if engine_name == "ParallelEngine":
+        print(f"Search calls: {engine_data.extras.get('sku_search', 0)}")
+        print(f"Excerpt extractions: {engine_data.extras.get('sku_extract_excerpts', 0)}")
+```
+
+When aggregating `RuntimeInfo` objects with `+`, numeric values in `extras` are summed, while non-numeric values are overwritten.
+
 ---
 
 ## Preview Mode

{symbolicai-1.1.1 → symbolicai-1.2.0}/ruff.toml

@@ -45,10 +45,11 @@ select = [
   "RUF","T10","T20"
 ]
 ignore = [
-  "B028",  # no explicit stacklevel; enable if you rely on warnings for users
-  "PLW0603",  # global statement discouraged; often noisy in legacy code
-  "G004", #  logging-f-string
-  "A002", #  argument-shadowing-builtin
+  "B028", # no explicit stacklevel; enable if you rely on warnings for users
+  "PLW0603", # global statement discouraged; often noisy in legacy code
+  "G004", # logging-f-string
+  "A002", # argument-shadowing-builtin
+  "C901", # better to handle this manually case by case
 ]
 
 # Allow fix for all enabled rules (when `--fix`) is provided.
@@ -58,9 +59,6 @@ unfixable = []
 # Allow unused variables when underscore-prefixed.
 dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
 
-[lint.mccabe]
-max-complexity = 10
-
 [format]
 quote-style = "double"
 indent-style = "space"

{symbolicai-1.1.1 → symbolicai-1.2.0}/symai/__init__.py

@@ -33,7 +33,7 @@ os.environ["TOKENIZERS_PARALLELISM"] = "false"
 # Create singleton instance
 config_manager = settings.SymAIConfig()
 
-SYMAI_VERSION = "1.1.1"
+SYMAI_VERSION = "1.2.0"
 __version__ = SYMAI_VERSION
 __root_dir__ = config_manager.config_dir
 

{symbolicai-1.1.1 → symbolicai-1.2.0}/symai/backend/engines/index/engine_qdrant.py

@@ -22,7 +22,9 @@ try:
     from qdrant_client.http.models import (
         Distance,
         Filter,
+        NamedVector,
         PointStruct,
+        Query,
         ScoredPoint,
         VectorParams,
     )
@@ -33,6 +35,8 @@ except ImportError:
     VectorParams = None
     PointStruct = None
     Filter = None
+    Query = None
+    NamedVector = None
     ScoredPoint = None
 
 try:
@@ -322,6 +326,50 @@ class QdrantIndexEngine(Engine):
             # Reinitialize client to refresh collection list
             self._init_client()
 
+    def _build_query_filter(self, raw_filter: Any) -> Filter | None:
+        """Normalize various filter representations into a Qdrant Filter.
+
+        Supports:
+        - None: returns None
+        - Existing Filter instance: returned as-is
+        - Dict[str, Any]: converted to equality-based Filter over payload keys
+
+        The dict form is intentionally simple and maps directly to `payload.<key>`
+        equality conditions, which covers the majority of RAG use cases while
+        remaining easy to serialize and pass through higher-level APIs.
+        """
+        if raw_filter is None or Filter is None:
+            return None
+
+        # Already a Filter instance → use directly
+        if isinstance(raw_filter, Filter):
+            return raw_filter
+
+        # Simple dict → build equality-based must filter
+        if isinstance(raw_filter, dict):
+            if models is None:
+                UserMessage(
+                    "Qdrant filter models are not available. "
+                    "Please install `qdrant-client` to use filtering.",
+                    raise_with=ImportError,
+                )
+
+            conditions = []
+            for key, value in raw_filter.items():
+                # We keep semantics simple and robust: every entry is treated as an
+                # equality condition on the payload key (logical AND across keys).
+                conditions.append(
+                    models.FieldCondition(
+                        key=key,
+                        match=models.MatchValue(value=value),
+                    )
+                )
+
+            return Filter(must=conditions) if conditions else None
+
+        # Fallback: pass through other representations (e.g. already-built Filter-like)
+        return raw_filter
+
     def _prepare_points_for_upsert(
         self,
         embeddings: list | np.ndarray | Any,
@@ -338,7 +386,7 @@ class QdrantIndexEngine(Engine):
             embeddings = [embeddings]
 
         for i, vec in enumerate(embeddings):
-            point_id = ids[i] if ids and i < len(ids) else i
+            point_id = self._normalize_point_id(ids[i]) if ids and i < len(ids) else i
             payload = payloads[i] if payloads and i < len(payloads) else {}
             points.append(
                 PointStruct(id=point_id, vector=self._normalize_vector(vec), payload=payload)
@@ -349,6 +397,14 @@ class QdrantIndexEngine(Engine):
     def forward(self, argument):
         kwargs = argument.kwargs
         embedding = argument.prop.prepared_input
+        if embedding is None:
+            embedding = getattr(argument.prop, "prompt", None)
+        if embedding is None:
+            msg = (
+                "Qdrant forward() requires an embedding vector. "
+                "Provide it via prepared_input or prompt before calling forward()."
+            )
+            raise ValueError(msg)
         query = argument.prop.ori_query
         operation = argument.prop.operation
         collection_name = argument.prop.index_name if argument.prop.index_name else self.index_name
@@ -369,8 +425,20 @@ class QdrantIndexEngine(Engine):
             # Ensure collection exists - fail fast if it doesn't
             self._ensure_collection_exists(collection_name)
             index_top_k = kwargs.get("index_top_k", self.index_top_k)
-            # Use existing _query method
-            rsp = self._query(collection_name, embedding, index_top_k)
+            # Optional search parameters
+            score_threshold = kwargs.get("score_threshold")
+            # Accept both `query_filter` and `filter` for convenience
+            raw_filter = kwargs.get("query_filter", kwargs.get("filter"))
+            query_filter = self._build_query_filter(raw_filter)
+
+            # Use shared search helper that already handles retries and normalization
+            rsp = self._search_sync(
+                collection_name=collection_name,
+                query_vector=embedding,
+                limit=index_top_k,
+                score_threshold=score_threshold,
+                query_filter=query_filter,
+            )
         elif operation == "add":
             # Create collection if it doesn't exist (only for write operations)
             self._create_collection_sync(collection_name, collection_dims, self.index_metric)
@@ -446,14 +514,19 @@ class QdrantIndexEngine(Engine):
         )
         def _func():
             query_vector_normalized = self._normalize_vector(query_vector)
-            return self.client.search(
+            # For single vector collections, pass vector directly to query parameter
+            # For named vector collections, use Query(near_vector=NamedVector(name="vector_name", vector=...))
+            # query_points API uses query_filter (not filter) for filtering
+            response = self.client.query_points(
                 collection_name=collection_name,
-                query_vector=query_vector_normalized,
+                query=query_vector_normalized,
                 limit=top_k,
                 with_payload=True,
                 with_vectors=self.index_values,
                 **kwargs,
             )
+            # query_points returns QueryResponse with .points attribute, extract it
+            return response.points
 
         return _func()
 
@@ -563,16 +636,67 @@ class QdrantIndexEngine(Engine):
                     for name, vec in vector_config.items()
                 }
             }
-        return {
+        # Qdrant 1.16.1+ compatibility: vectors_count and indexed_vectors_count may not exist
+        # Use points_count as the primary count, and try to get vectors_count if available
+        result = {
             "name": collection_name,
-            "vectors_count": collection_info.vectors_count,
-            "indexed_vectors_count": collection_info.indexed_vectors_count,
             "points_count": collection_info.points_count,
             "config": {"params": {"vectors": vectors_info}},
         }
 
+        # Try to get vectors_count if available (for older Qdrant versions)
+        if hasattr(collection_info, "vectors_count"):
+            result["vectors_count"] = collection_info.vectors_count
+        else:
+            # In Qdrant 1.16.1+, vectors_count is not available, use points_count as approximation
+            result["vectors_count"] = collection_info.points_count
+
+        # Try to get indexed_vectors_count if available
+        if hasattr(collection_info, "indexed_vectors_count"):
+            result["indexed_vectors_count"] = collection_info.indexed_vectors_count
+        else:
+            # In Qdrant 1.16.1+, indexed_vectors_count may not be available
+            result["indexed_vectors_count"] = collection_info.points_count
+
+        return result
+
     # ==================== Point Operations ====================
 
+    def _normalize_point_id(self, point_id: Any) -> int | uuid.UUID:
+        """Normalize point ID to integer or UUID for Qdrant 1.16.1+ compatibility.
+
+        Qdrant 1.16.1+ requires point IDs to be either unsigned integers or UUIDs.
+        This function converts string IDs (like 'vec-1') to integers or UUIDs.
+        """
+        # If already int or UUID, return as-is
+        if isinstance(point_id, (int, uuid.UUID)):
+            return point_id
+
+        # If string, try to convert
+        if isinstance(point_id, str):
+            # Try to parse as integer first
+            try:
+                # Handle string IDs like "vec-1" by extracting the number
+                if point_id.startswith("vec-"):
+                    num_str = point_id.split("-", 1)[-1]
+                    return int(num_str)
+                # Try direct integer conversion
+                return int(point_id)
+            except (ValueError, AttributeError):
+                # If not a valid integer, try UUID
+                try:
+                    return uuid.UUID(point_id)
+                except (ValueError, AttributeError):
+                    # Fallback: generate UUID from string hash
+                    return uuid.uuid5(uuid.NAMESPACE_DNS, point_id)
+
+        # For other types, try to convert to int
+        try:
+            return int(point_id)
+        except (ValueError, TypeError):
+            # Last resort: generate UUID from string representation
+            return uuid.uuid5(uuid.NAMESPACE_DNS, str(point_id))
+
     def _upsert_points_sync(
         self,
         collection_name: str,
@@ -589,17 +713,17 @@ class QdrantIndexEngine(Engine):
         if isinstance(points[0], dict):
             points = [
                 PointStruct(
-                    id=point["id"],
+                    id=self._normalize_point_id(point["id"]),
                     vector=self._normalize_vector(point["vector"]),
                     payload=point.get("payload", {}),
                 )
                 for point in points
             ]
         else:
-            # Normalize vectors in existing PointStruct objects
+            # Normalize vectors and IDs in existing PointStruct objects
             points = [
                 PointStruct(
-                    id=point.id,
+                    id=self._normalize_point_id(point.id),
                     vector=self._normalize_vector(point.vector),
                     payload=point.payload,
                 )

{symbolicai-1.1.1 → symbolicai-1.2.0}/symai/backend/engines/neurosymbolic/__init__.py

@@ -11,6 +11,7 @@ from ...mixin import (
     GROQ_REASONING_MODELS,
     OPENAI_CHAT_MODELS,
     OPENAI_REASONING_MODELS,
+    OPENAI_RESPONSES_MODELS,
 )
 from .engine_anthropic_claudeX_chat import ClaudeXChatEngine
 from .engine_anthropic_claudeX_reasoning import ClaudeXReasoningEngine
@@ -20,6 +21,7 @@ from .engine_google_geminiX_reasoning import GeminiXReasoningEngine
 from .engine_groq import GroqEngine
 from .engine_openai_gptX_chat import GPTXChatEngine
 from .engine_openai_gptX_reasoning import GPTXReasoningEngine
+from .engine_openai_responses import OpenAIResponsesEngine
 
 # create the mapping
 ENGINE_MAPPING = {
@@ -31,6 +33,7 @@ ENGINE_MAPPING = {
     **dict.fromkeys(GOOGLE_REASONING_MODELS, GeminiXReasoningEngine),
     **dict.fromkeys(OPENAI_CHAT_MODELS, GPTXChatEngine),
     **dict.fromkeys(OPENAI_REASONING_MODELS, GPTXReasoningEngine),
+    **dict.fromkeys(OPENAI_RESPONSES_MODELS, OpenAIResponsesEngine),
     **dict.fromkeys(GROQ_CHAT_MODELS, GroqEngine),
     **dict.fromkeys(GROQ_REASONING_MODELS, GroqEngine),
 }
@@ -49,6 +52,7 @@ __all__ = [
     "GROQ_REASONING_MODELS",
     "OPENAI_CHAT_MODELS",
     "OPENAI_REASONING_MODELS",
+    "OPENAI_RESPONSES_MODELS",
     "ClaudeXChatEngine",
     "ClaudeXReasoningEngine",
     "DeepSeekXReasoningEngine",
@@ -56,4 +60,5 @@ __all__ = [
     "GPTXReasoningEngine",
     "GeminiXReasoningEngine",
     "GroqEngine",
+    "OpenAIResponsesEngine",
 ]