vectorizer-sdk 3.0.3__tar.gz → 3.5.0__tar.gz

{vectorizer_sdk-3.0.3/vectorizer_sdk.egg-info → vectorizer_sdk-3.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vectorizer_sdk
-Version: 3.0.3
+Version: 3.5.0
 Summary: Python SDK for Vectorizer - Semantic search, vector operations, and the VectorizerRPC binary transport (default in v3.x)
 Author-email: HiveLLM Team <team@hivellm.org>
 License: Apache-2.0
@@ -13,40 +13,39 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: Text Processing :: Indexing
 Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
-Requires-Python: >=3.8
+Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: aiohttp>=3.10.0
-Requires-Dist: msgpack>=1.1.0
+Requires-Dist: aiohttp>=3.10.11
+Requires-Dist: msgpack>=1.1.1
 Provides-Extra: umicp
 Requires-Dist: umicp-python>=0.1.3; extra == "umicp"
 Provides-Extra: dev
-Requires-Dist: pytest>=8.0.0; extra == "dev"
+Requires-Dist: pytest>=8.3.5; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.24.0; extra == "dev"
 Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
 Requires-Dist: black>=25.0.0; extra == "dev"
-Requires-Dist: isort>=5.13.0; extra == "dev"
-Requires-Dist: flake8>=7.0.0; extra == "dev"
-Requires-Dist: mypy>=1.11.0; extra == "dev"
+Requires-Dist: isort>=5.13.2; extra == "dev"
+Requires-Dist: flake8>=7.1.2; extra == "dev"
+Requires-Dist: mypy>=1.14.1; extra == "dev"
 Requires-Dist: pre-commit>=4.0.0; extra == "dev"
 Provides-Extra: docs
-Requires-Dist: sphinx>=7.0.0; extra == "docs"
-Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
-Requires-Dist: myst-parser>=3.0.0; extra == "docs"
+Requires-Dist: sphinx>=7.1.2; extra == "docs"
+Requires-Dist: sphinx-rtd-theme>=3.1.0; extra == "docs"
+Requires-Dist: myst-parser>=3.0.1; extra == "docs"
 Provides-Extra: test
-Requires-Dist: pytest>=8.0.0; extra == "test"
+Requires-Dist: pytest>=8.3.5; extra == "test"
 Requires-Dist: pytest-asyncio>=0.24.0; extra == "test"
 Requires-Dist: pytest-cov>=5.0.0; extra == "test"
-Requires-Dist: httpx>=0.27.0; extra == "test"
+Requires-Dist: httpx>=0.28.1; extra == "test"
 Dynamic: license-file
 
 # Vectorizer Python SDK
@@ -58,9 +57,47 @@ Dynamic: license-file
 A comprehensive Python SDK for the Vectorizer semantic search service.
 
 **Package**: `vectorizer_sdk` (PEP 625 compliant)
-**Version**: 3.0.0
+**Version**: 3.5.0
 **PyPI**: https://pypi.org/project/vectorizer-sdk/
 
+## v3.5 — server alignment (no client API changes)
+
+Version tracks the Vectorizer **3.5.0** server release: non-blocking
+search during batch inserts, PQ/Binary quantization wiring, SIMD
+quantize kernels, BM25-after-restart and WAL-durability fixes, and a
+security/dependency refresh. All server-internal — the client API is
+unchanged since **v3.3** (REST control-surface parity + dashboard
+metrics). See `CHANGELOG.md` for the full method surface.
+
+## v3.2 — backpressure-aware client (HTTP 429 + `Retry-After`)
+
+The REST `VectorizerClient` honors server-side bulk-upsert
+backpressure shipped in Vectorizer 3.2.0
+([#263](https://github.com/hivellm/vectorizer/issues/263)). On HTTP
+`429 Too Many Requests` the client parses `Retry-After` (seconds
+form, 1 s default, 30 s cap), sleeps, and retries up to 3 times
+before raising a typed `RateLimitError`. Pre-3.2.0 clients bounced
+429s into a generic 5xx and lost the retry budget. Identical
+semantics ship in every first-party SDK (Rust, Python, TypeScript,
+Go, C#) — see `tests/test_retry_after_parse.py`.
+
+## v3.1 — `/insert_vectors` + stable client-id upserts
+
+- `insert_vectors(collection, vectors, public_key=None)` — bulk-
+  insert pre-computed embeddings with caller-supplied vector ids.
+  Skips the embedding pipeline entirely.
+- `insert` / `insert_texts`: the request `id` is now used verbatim
+  as the stored `Vector.id` (non-chunked) or as `<id>#<chunk_index>`
+  (chunked). Re-running the same payload upserts in place instead
+  of duplicating.
+- Chunked vectors expose a flat payload layout (`{content,
+  file_path, chunk_index, parent_id, ...user_metadata}`). Legacy
+  nested payloads from ≤ 3.0.x stay readable during the deprecation
+  window.
+
+Client-id contract: non-empty, length ≤ 256, no leading/trailing
+whitespace, must not contain `#`.
+
 ## v3.0 — VectorizerRPC is the default transport
 
 Starting with v3.0, the recommended transport is **VectorizerRPC**: a
@@ -79,6 +116,8 @@ import vectorizer_sdk
 
 async def main():
     client = await vectorizer_sdk.connect_async("vectorizer://127.0.0.1:15503")
+    # `hello` and `search_basic` are RPC-only (not available on the legacy
+    # REST `VectorizerClient`).
     await client.hello(vectorizer_sdk.HelloPayload(client_name="my-app"))
     print(await client.list_collections())
     hits = await client.search_basic("docs", "vector database", limit=5)
@@ -140,7 +179,7 @@ for a runnable end-to-end example.
 pip install vectorizer-sdk
 
 # Or specific version
-pip install vectorizer-sdk==3.0.0
+pip install vectorizer-sdk==3.5.0
 ```
 
 ## Package Layout (v3.x)
@@ -477,6 +516,11 @@ related = await client.get_related_files(
 
 ### Summarization Operations
 
+> WARNING: The `/summarize/*` REST endpoints are documented but not yet wired
+> server-side (see `DOC_GAP_ANALYSIS`). The SDK methods below
+> (`summarize_text`, `summarize_context`) will fail until server wiring is
+> complete.
+
 #### Summarize Text
 Summarize text using various methods:
 
@@ -509,6 +553,11 @@ summary = await client.summarize_context(
 
 ### Workspace Management
 
+> WARNING: `add_workspace`, `list_workspaces`, and `remove_workspace` are
+> exposed via the REST transport through dynamic `__getattr__` delegation,
+> but they are not first-class SDK methods yet. They work at runtime but
+> won't autocomplete in IDEs. A future release will add explicit methods.
+
 #### Add Workspace
 Add a new workspace:
 
@@ -537,6 +586,11 @@ await client.remove_workspace(
 
 ### Backup Operations
 
+> WARNING: `create_backup`, `list_backups`, and `restore_backup` are
+> exposed via the REST transport through dynamic `__getattr__` delegation,
+> but they are not first-class SDK methods yet. They work at runtime but
+> won't autocomplete in IDEs. A future release will add explicit methods.
+
 #### Create Backup
 Create a backup of collections:
 
@@ -664,8 +718,20 @@ await client.create_collection("documents", dimension=768)
 await client.insert_texts("documents", [
     {"id": "doc1", "text": "Sample document", "metadata": {"source": "api"}}
 ])
-await client.update_vector("documents", "doc1", metadata={"updated": True})
-await client.delete_vector("documents", "doc1")
+# Update-via-reinsert: re-call `insert_texts` with the same id to replace the record.
+await client.insert_texts("documents", [
+    {"id": "doc1", "text": "Sample document (updated)", "metadata": {"updated": True}}
+])
+report = await client.delete_vectors("documents", ["doc1"])
+print(f"deleted={report.deleted} failed={report.failed}")
+
+# Tier demotion: move vectors between collections without re-embedding
+# (issue #265). Insert into dst lands BEFORE delete from src so a
+# mid-batch crash leaves a recoverable duplicate, never data loss.
+mv = await client.move_to_collection("hot", "warm", ["vec-1", "vec-2"])
+for row in mv.results:
+    if row.status != "ok":
+        print(f"move failed id={row.id!r} status={row.status} err={row.error!r}")
 
 # Reads automatically go to replicas (load balanced)
 results = await client.search_vectors("documents", query="sample", limit=10)
@@ -673,6 +739,212 @@ collections = await client.list_collections()
 vector = await client.get_vector("documents", "doc1")
 ```
 
+## Control surface (3.4)
+
+### Admin / observability
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Server health, uptime, collection/vector counts
+    stats = await client.get_stats()
+    print(f"Total vectors: {stats['total_vectors']}")
+
+    status = await client.get_status()
+    print(f"Server v{status['version']}, uptime: {status['uptime']}s")
+
+    # Recent logs
+    logs = await client.get_logs(lines=50, level="INFO")
+    for entry in logs:
+        print(f"{entry['timestamp']}: {entry['message']}")
+
+    # Per-collection indexing progress
+    progress = await client.get_indexing_progress()
+    for collection, pct in progress.items():
+        print(f"{collection}: {pct:.1f}% complete")
+
+    # Force flush one collection
+    await client.force_save_collection("my_docs")
+
+    # List and clean empty collections
+    empty = await client.list_empty_collections()
+    if empty:
+        report = await client.cleanup_empty_collections()
+        print(f"Cleaned up {report['deleted']} empty collections")
+
+    # List workspaces
+    workspaces = await client.list_workspaces()
+    print(f"Workspaces: {workspaces}")
+
+asyncio.run(main())
+```
+
+### Auth
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Current user info
+    me = await client.me()
+    print(f"Logged in as: {me['username']} (roles: {', '.join(me['roles'])})")
+
+    # Refresh token with extended TTL
+    token = await client.refresh_token()
+    print(f"Token refreshed, expires in: {token['expires_in']} seconds")
+
+    # Validate password before creating account
+    report = await client.validate_password("MySecure123!")
+    print(f"Valid: {report['valid']}, feedback: {', '.join(report['feedback'])}")
+
+    # Create API key for programmatic access
+    api_key = await client.create_api_key(
+        name="integration-key",
+        expires_in=86400 * 365  # 1 year
+    )
+    print(f"API Key: {api_key['api_key']}")
+
+    # List and revoke API keys
+    keys = await client.list_api_keys()
+    for key in keys:
+        print(f"Key: {key['id']} (expires: {key['expires_at']})")
+    await client.revoke_api_key(keys[0]['id'])
+
+    # Change password
+    await client.change_password("newPassword123!")
+
+    # Logout
+    await client.logout()
+
+asyncio.run(main())
+```
+
+### Replication
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Check replication role and status
+    status = await client.get_replication_status()
+    print(f"Role: {status['role']}, enabled: {status['enabled']}")
+
+    # Get replication statistics (lag, bytes synced)
+    stats = await client.get_replication_stats()
+    print(f"Bytes synced: {stats['bytes_synced']}")
+
+    # List all replicas connected to this master
+    replicas = await client.list_replicas()
+    for replica in replicas:
+        print(f"Replica: {replica['address']} (lag: {replica['lag_ms']}ms)")
+
+asyncio.run(main())
+```
+
+### Discovery pipeline
+
+The discovery pipeline chains six stages from broad search to final LLM-ready prompt:
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Stage 1: Broad discovery — multi-query search across all collections
+    broad = await client.broad_discovery(
+        query="machine learning algorithms",
+        max_results=20
+    )
+    print(f"Found {len(broad['results'])} broad results")
+
+    # Stage 2: Semantic focus — narrow search to top collection
+    focused = await client.semantic_focus(
+        query="neural networks",
+        collection="research",
+        max_results=10
+    )
+    print(f"Focused results: {len(focused['results'])}")
+
+    # Stage 3: Promote README — elevate high-quality chunks
+    promoted = await client.promote_readme(
+        results=focused['results'],
+        readme_boost=2.0
+    )
+
+    # Stage 4: Compress evidence — distill to bullet points
+    bullets = await client.compress_evidence(
+        chunks=promoted['results'],
+        max_bullets=15
+    )
+    print(f"Evidence bullets: {bullets['bullets']}")
+
+    # Stage 5: Build answer plan — organize bullets into sections
+    plan = await client.build_answer_plan(
+        evidence=bullets['bullets'],
+        max_sections=5
+    )
+    print(f"Sections: {plan['sections']}")
+
+    # Stage 6: Render LLM prompt — final markdown string for LLM
+    llm_prompt = await client.render_llm_prompt(
+        plan=plan,
+        style="formal"
+    )
+    print(f"LLM prompt:\n{llm_prompt['markdown']}")
+
+asyncio.run(main())
+```
+
+### Hub backups
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    user_id = "user-123"
+
+    # List user's backups
+    backups = await client.list_user_backups(user_id)
+    for backup in backups:
+        print(f"Backup: {backup['id']} (size: {backup['size_bytes']} bytes)")
+
+    # Create a new backup
+    new_backup = await client.create_user_backup(
+        user_id=user_id,
+        name="full-backup-2024-01",
+        description="January full backup",
+        collections=None  # backup all
+    )
+    print(f"Created backup: {new_backup['id']}")
+
+    # Restore a backup
+    await client.restore_user_backup(
+        user_id=user_id,
+        backup_id=new_backup['id']
+    )
+    print("Restore started")
+
+    # Delete old backup
+    await client.delete_user_backup(user_id, backups[0]['id'])
+
+asyncio.run(main())
+```
+
 #### Read Preferences
 
 | Preference | Description | Use Case |
@@ -702,7 +974,7 @@ The SDK automatically classifies operations:
 
 | Operation Type | Routed To | Methods |
 |---------------|-----------|---------|
-| **Writes** | Always Master | `insert_texts`, `insert_vectors`, `update_vector`, `delete_vector`, `create_collection`, `delete_collection` |
+| **Writes** | Always Master | `insert_texts`, `insert_vectors`, `delete_vectors`, `create_collection`, `delete_collection` |
 | **Reads** | Based on `read_preference` | `search_vectors`, `get_vector`, `list_collections`, `intelligent_search`, `semantic_search`, `hybrid_search` |
 
 #### Standalone Mode (Single Node)