vectorizer-sdk 3.2.0__tar.gz → 3.5.0__tar.gz

{vectorizer_sdk-3.2.0 → vectorizer_sdk-3.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vectorizer_sdk
-Version: 3.2.0
+Version: 3.5.0
 Summary: Python SDK for Vectorizer - Semantic search, vector operations, and the VectorizerRPC binary transport (default in v3.x)
 Author-email: HiveLLM Team <team@hivellm.org>
 License: Apache-2.0
@@ -13,40 +13,39 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: Text Processing :: Indexing
 Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
-Requires-Python: >=3.8
+Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: aiohttp>=3.10.0
-Requires-Dist: msgpack>=1.1.0
+Requires-Dist: aiohttp>=3.10.11
+Requires-Dist: msgpack>=1.1.1
 Provides-Extra: umicp
 Requires-Dist: umicp-python>=0.1.3; extra == "umicp"
 Provides-Extra: dev
-Requires-Dist: pytest>=8.0.0; extra == "dev"
+Requires-Dist: pytest>=8.3.5; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.24.0; extra == "dev"
 Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
 Requires-Dist: black>=25.0.0; extra == "dev"
-Requires-Dist: isort>=5.13.0; extra == "dev"
-Requires-Dist: flake8>=7.0.0; extra == "dev"
-Requires-Dist: mypy>=1.11.0; extra == "dev"
+Requires-Dist: isort>=5.13.2; extra == "dev"
+Requires-Dist: flake8>=7.1.2; extra == "dev"
+Requires-Dist: mypy>=1.14.1; extra == "dev"
 Requires-Dist: pre-commit>=4.0.0; extra == "dev"
 Provides-Extra: docs
-Requires-Dist: sphinx>=7.0.0; extra == "docs"
-Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
-Requires-Dist: myst-parser>=3.0.0; extra == "docs"
+Requires-Dist: sphinx>=7.1.2; extra == "docs"
+Requires-Dist: sphinx-rtd-theme>=3.1.0; extra == "docs"
+Requires-Dist: myst-parser>=3.0.1; extra == "docs"
 Provides-Extra: test
-Requires-Dist: pytest>=8.0.0; extra == "test"
+Requires-Dist: pytest>=8.3.5; extra == "test"
 Requires-Dist: pytest-asyncio>=0.24.0; extra == "test"
 Requires-Dist: pytest-cov>=5.0.0; extra == "test"
-Requires-Dist: httpx>=0.27.0; extra == "test"
+Requires-Dist: httpx>=0.28.1; extra == "test"
 Dynamic: license-file
 
 # Vectorizer Python SDK
@@ -58,9 +57,18 @@ Dynamic: license-file
 A comprehensive Python SDK for the Vectorizer semantic search service.
 
 **Package**: `vectorizer_sdk` (PEP 625 compliant)
-**Version**: 3.2.0
+**Version**: 3.5.0
 **PyPI**: https://pypi.org/project/vectorizer-sdk/
 
+## v3.5 — server alignment (no client API changes)
+
+Version tracks the Vectorizer **3.5.0** server release: non-blocking
+search during batch inserts, PQ/Binary quantization wiring, SIMD
+quantize kernels, BM25-after-restart and WAL-durability fixes, and a
+security/dependency refresh. All server-internal — the client API is
+unchanged since **v3.3** (REST control-surface parity + dashboard
+metrics). See `CHANGELOG.md` for the full method surface.
+
 ## v3.2 — backpressure-aware client (HTTP 429 + `Retry-After`)
 
 The REST `VectorizerClient` honors server-side bulk-upsert
@@ -171,7 +179,7 @@ for a runnable end-to-end example.
 pip install vectorizer-sdk
 
 # Or specific version
-pip install vectorizer-sdk==3.2.0
+pip install vectorizer-sdk==3.5.0
 ```
 
 ## Package Layout (v3.x)
@@ -714,7 +722,16 @@ await client.insert_texts("documents", [
 await client.insert_texts("documents", [
     {"id": "doc1", "text": "Sample document (updated)", "metadata": {"updated": True}}
 ])
-await client.delete_vectors("documents", ["doc1"])
+report = await client.delete_vectors("documents", ["doc1"])
+print(f"deleted={report.deleted} failed={report.failed}")
+
+# Tier demotion: move vectors between collections without re-embedding
+# (issue #265). Insert into dst lands BEFORE delete from src so a
+# mid-batch crash leaves a recoverable duplicate, never data loss.
+mv = await client.move_to_collection("hot", "warm", ["vec-1", "vec-2"])
+for row in mv.results:
+    if row.status != "ok":
+        print(f"move failed id={row.id!r} status={row.status} err={row.error!r}")
 
 # Reads automatically go to replicas (load balanced)
 results = await client.search_vectors("documents", query="sample", limit=10)
@@ -722,6 +739,212 @@ collections = await client.list_collections()
 vector = await client.get_vector("documents", "doc1")
 ```
 
+## Control surface (3.4)
+
+### Admin / observability
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Server health, uptime, collection/vector counts
+    stats = await client.get_stats()
+    print(f"Total vectors: {stats['total_vectors']}")
+
+    status = await client.get_status()
+    print(f"Server v{status['version']}, uptime: {status['uptime']}s")
+
+    # Recent logs
+    logs = await client.get_logs(lines=50, level="INFO")
+    for entry in logs:
+        print(f"{entry['timestamp']}: {entry['message']}")
+
+    # Per-collection indexing progress
+    progress = await client.get_indexing_progress()
+    for collection, pct in progress.items():
+        print(f"{collection}: {pct:.1f}% complete")
+
+    # Force flush one collection
+    await client.force_save_collection("my_docs")
+
+    # List and clean empty collections
+    empty = await client.list_empty_collections()
+    if empty:
+        report = await client.cleanup_empty_collections()
+        print(f"Cleaned up {report['deleted']} empty collections")
+
+    # List workspaces
+    workspaces = await client.list_workspaces()
+    print(f"Workspaces: {workspaces}")
+
+asyncio.run(main())
+```
+
+### Auth
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Current user info
+    me = await client.me()
+    print(f"Logged in as: {me['username']} (roles: {', '.join(me['roles'])})")
+
+    # Refresh token with extended TTL
+    token = await client.refresh_token()
+    print(f"Token refreshed, expires in: {token['expires_in']} seconds")
+
+    # Validate password before creating account
+    report = await client.validate_password("MySecure123!")
+    print(f"Valid: {report['valid']}, feedback: {', '.join(report['feedback'])}")
+
+    # Create API key for programmatic access
+    api_key = await client.create_api_key(
+        name="integration-key",
+        expires_in=86400 * 365  # 1 year
+    )
+    print(f"API Key: {api_key['api_key']}")
+
+    # List and revoke API keys
+    keys = await client.list_api_keys()
+    for key in keys:
+        print(f"Key: {key['id']} (expires: {key['expires_at']})")
+    await client.revoke_api_key(keys[0]['id'])
+
+    # Change password
+    await client.change_password("newPassword123!")
+
+    # Logout
+    await client.logout()
+
+asyncio.run(main())
+```
+
+### Replication
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Check replication role and status
+    status = await client.get_replication_status()
+    print(f"Role: {status['role']}, enabled: {status['enabled']}")
+
+    # Get replication statistics (lag, bytes synced)
+    stats = await client.get_replication_stats()
+    print(f"Bytes synced: {stats['bytes_synced']}")
+
+    # List all replicas connected to this master
+    replicas = await client.list_replicas()
+    for replica in replicas:
+        print(f"Replica: {replica['address']} (lag: {replica['lag_ms']}ms)")
+
+asyncio.run(main())
+```
+
+### Discovery pipeline
+
+The discovery pipeline chains six stages from broad search to final LLM-ready prompt:
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Stage 1: Broad discovery — multi-query search across all collections
+    broad = await client.broad_discovery(
+        query="machine learning algorithms",
+        max_results=20
+    )
+    print(f"Found {len(broad['results'])} broad results")
+
+    # Stage 2: Semantic focus — narrow search to top collection
+    focused = await client.semantic_focus(
+        query="neural networks",
+        collection="research",
+        max_results=10
+    )
+    print(f"Focused results: {len(focused['results'])}")
+
+    # Stage 3: Promote README — elevate high-quality chunks
+    promoted = await client.promote_readme(
+        results=focused['results'],
+        readme_boost=2.0
+    )
+
+    # Stage 4: Compress evidence — distill to bullet points
+    bullets = await client.compress_evidence(
+        chunks=promoted['results'],
+        max_bullets=15
+    )
+    print(f"Evidence bullets: {bullets['bullets']}")
+
+    # Stage 5: Build answer plan — organize bullets into sections
+    plan = await client.build_answer_plan(
+        evidence=bullets['bullets'],
+        max_sections=5
+    )
+    print(f"Sections: {plan['sections']}")
+
+    # Stage 6: Render LLM prompt — final markdown string for LLM
+    llm_prompt = await client.render_llm_prompt(
+        plan=plan,
+        style="formal"
+    )
+    print(f"LLM prompt:\n{llm_prompt['markdown']}")
+
+asyncio.run(main())
+```
+
+### Hub backups
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    user_id = "user-123"
+
+    # List user's backups
+    backups = await client.list_user_backups(user_id)
+    for backup in backups:
+        print(f"Backup: {backup['id']} (size: {backup['size_bytes']} bytes)")
+
+    # Create a new backup
+    new_backup = await client.create_user_backup(
+        user_id=user_id,
+        name="full-backup-2024-01",
+        description="January full backup",
+        collections=None  # backup all
+    )
+    print(f"Created backup: {new_backup['id']}")
+
+    # Restore a backup
+    await client.restore_user_backup(
+        user_id=user_id,
+        backup_id=new_backup['id']
+    )
+    print("Restore started")
+
+    # Delete old backup
+    await client.delete_user_backup(user_id, backups[0]['id'])
+
+asyncio.run(main())
+```
+
 #### Read Preferences
 
 | Preference | Description | Use Case |

vectorizer_sdk-3.2.0/vectorizer_sdk.egg-info/PKG-INFO → vectorizer_sdk-3.5.0/README.md

@@ -1,54 +1,3 @@
-Metadata-Version: 2.4
-Name: vectorizer_sdk
-Version: 3.2.0
-Summary: Python SDK for Vectorizer - Semantic search, vector operations, and the VectorizerRPC binary transport (default in v3.x)
-Author-email: HiveLLM Team <team@hivellm.org>
-License: Apache-2.0
-Project-URL: Homepage, https://github.com/hivellm/vectorizer
-Project-URL: Documentation, https://github.com/hivellm/vectorizer/tree/main/docs
-Project-URL: Repository, https://github.com/hivellm/vectorizer
-Project-URL: Issues, https://github.com/hivellm/vectorizer/issues
-Keywords: vectorizer,semantic-search,embeddings,machine-learning,ai,search,vectors,similarity,hivellm,umicp
-Classifier: Development Status :: 5 - Production/Stable
-Classifier: Intended Audience :: Developers
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Classifier: Topic :: Text Processing :: Indexing
-Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: aiohttp>=3.10.0
-Requires-Dist: msgpack>=1.1.0
-Provides-Extra: umicp
-Requires-Dist: umicp-python>=0.1.3; extra == "umicp"
-Provides-Extra: dev
-Requires-Dist: pytest>=8.0.0; extra == "dev"
-Requires-Dist: pytest-asyncio>=0.24.0; extra == "dev"
-Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
-Requires-Dist: black>=25.0.0; extra == "dev"
-Requires-Dist: isort>=5.13.0; extra == "dev"
-Requires-Dist: flake8>=7.0.0; extra == "dev"
-Requires-Dist: mypy>=1.11.0; extra == "dev"
-Requires-Dist: pre-commit>=4.0.0; extra == "dev"
-Provides-Extra: docs
-Requires-Dist: sphinx>=7.0.0; extra == "docs"
-Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
-Requires-Dist: myst-parser>=3.0.0; extra == "docs"
-Provides-Extra: test
-Requires-Dist: pytest>=8.0.0; extra == "test"
-Requires-Dist: pytest-asyncio>=0.24.0; extra == "test"
-Requires-Dist: pytest-cov>=5.0.0; extra == "test"
-Requires-Dist: httpx>=0.27.0; extra == "test"
-Dynamic: license-file
-
 # Vectorizer Python SDK
 
 [![PyPI version](https://badge.fury.io/py/vectorizer-sdk.svg)](https://pypi.org/project/vectorizer-sdk/)
@@ -58,9 +7,18 @@ Dynamic: license-file
 A comprehensive Python SDK for the Vectorizer semantic search service.
 
 **Package**: `vectorizer_sdk` (PEP 625 compliant)
-**Version**: 3.2.0
+**Version**: 3.5.0
 **PyPI**: https://pypi.org/project/vectorizer-sdk/
 
+## v3.5 — server alignment (no client API changes)
+
+Version tracks the Vectorizer **3.5.0** server release: non-blocking
+search during batch inserts, PQ/Binary quantization wiring, SIMD
+quantize kernels, BM25-after-restart and WAL-durability fixes, and a
+security/dependency refresh. All server-internal — the client API is
+unchanged since **v3.3** (REST control-surface parity + dashboard
+metrics). See `CHANGELOG.md` for the full method surface.
+
 ## v3.2 — backpressure-aware client (HTTP 429 + `Retry-After`)
 
 The REST `VectorizerClient` honors server-side bulk-upsert
@@ -171,7 +129,7 @@ for a runnable end-to-end example.
 pip install vectorizer-sdk
 
 # Or specific version
-pip install vectorizer-sdk==3.2.0
+pip install vectorizer-sdk==3.5.0
 ```
 
 ## Package Layout (v3.x)
@@ -714,7 +672,16 @@ await client.insert_texts("documents", [
 await client.insert_texts("documents", [
     {"id": "doc1", "text": "Sample document (updated)", "metadata": {"updated": True}}
 ])
-await client.delete_vectors("documents", ["doc1"])
+report = await client.delete_vectors("documents", ["doc1"])
+print(f"deleted={report.deleted} failed={report.failed}")
+
+# Tier demotion: move vectors between collections without re-embedding
+# (issue #265). Insert into dst lands BEFORE delete from src so a
+# mid-batch crash leaves a recoverable duplicate, never data loss.
+mv = await client.move_to_collection("hot", "warm", ["vec-1", "vec-2"])
+for row in mv.results:
+    if row.status != "ok":
+        print(f"move failed id={row.id!r} status={row.status} err={row.error!r}")
 
 # Reads automatically go to replicas (load balanced)
 results = await client.search_vectors("documents", query="sample", limit=10)
@@ -722,6 +689,212 @@ collections = await client.list_collections()
 vector = await client.get_vector("documents", "doc1")
 ```
 
+## Control surface (3.4)
+
+### Admin / observability
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Server health, uptime, collection/vector counts
+    stats = await client.get_stats()
+    print(f"Total vectors: {stats['total_vectors']}")
+
+    status = await client.get_status()
+    print(f"Server v{status['version']}, uptime: {status['uptime']}s")
+
+    # Recent logs
+    logs = await client.get_logs(lines=50, level="INFO")
+    for entry in logs:
+        print(f"{entry['timestamp']}: {entry['message']}")
+
+    # Per-collection indexing progress
+    progress = await client.get_indexing_progress()
+    for collection, pct in progress.items():
+        print(f"{collection}: {pct:.1f}% complete")
+
+    # Force flush one collection
+    await client.force_save_collection("my_docs")
+
+    # List and clean empty collections
+    empty = await client.list_empty_collections()
+    if empty:
+        report = await client.cleanup_empty_collections()
+        print(f"Cleaned up {report['deleted']} empty collections")
+
+    # List workspaces
+    workspaces = await client.list_workspaces()
+    print(f"Workspaces: {workspaces}")
+
+asyncio.run(main())
+```
+
+### Auth
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Current user info
+    me = await client.me()
+    print(f"Logged in as: {me['username']} (roles: {', '.join(me['roles'])})")
+
+    # Refresh token with extended TTL
+    token = await client.refresh_token()
+    print(f"Token refreshed, expires in: {token['expires_in']} seconds")
+
+    # Validate password before creating account
+    report = await client.validate_password("MySecure123!")
+    print(f"Valid: {report['valid']}, feedback: {', '.join(report['feedback'])}")
+
+    # Create API key for programmatic access
+    api_key = await client.create_api_key(
+        name="integration-key",
+        expires_in=86400 * 365  # 1 year
+    )
+    print(f"API Key: {api_key['api_key']}")
+
+    # List and revoke API keys
+    keys = await client.list_api_keys()
+    for key in keys:
+        print(f"Key: {key['id']} (expires: {key['expires_at']})")
+    await client.revoke_api_key(keys[0]['id'])
+
+    # Change password
+    await client.change_password("newPassword123!")
+
+    # Logout
+    await client.logout()
+
+asyncio.run(main())
+```
+
+### Replication
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Check replication role and status
+    status = await client.get_replication_status()
+    print(f"Role: {status['role']}, enabled: {status['enabled']}")
+
+    # Get replication statistics (lag, bytes synced)
+    stats = await client.get_replication_stats()
+    print(f"Bytes synced: {stats['bytes_synced']}")
+
+    # List all replicas connected to this master
+    replicas = await client.list_replicas()
+    for replica in replicas:
+        print(f"Replica: {replica['address']} (lag: {replica['lag_ms']}ms)")
+
+asyncio.run(main())
+```
+
+### Discovery pipeline
+
+The discovery pipeline chains six stages from broad search to final LLM-ready prompt:
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    # Stage 1: Broad discovery — multi-query search across all collections
+    broad = await client.broad_discovery(
+        query="machine learning algorithms",
+        max_results=20
+    )
+    print(f"Found {len(broad['results'])} broad results")
+
+    # Stage 2: Semantic focus — narrow search to top collection
+    focused = await client.semantic_focus(
+        query="neural networks",
+        collection="research",
+        max_results=10
+    )
+    print(f"Focused results: {len(focused['results'])}")
+
+    # Stage 3: Promote README — elevate high-quality chunks
+    promoted = await client.promote_readme(
+        results=focused['results'],
+        readme_boost=2.0
+    )
+
+    # Stage 4: Compress evidence — distill to bullet points
+    bullets = await client.compress_evidence(
+        chunks=promoted['results'],
+        max_bullets=15
+    )
+    print(f"Evidence bullets: {bullets['bullets']}")
+
+    # Stage 5: Build answer plan — organize bullets into sections
+    plan = await client.build_answer_plan(
+        evidence=bullets['bullets'],
+        max_sections=5
+    )
+    print(f"Sections: {plan['sections']}")
+
+    # Stage 6: Render LLM prompt — final markdown string for LLM
+    llm_prompt = await client.render_llm_prompt(
+        plan=plan,
+        style="formal"
+    )
+    print(f"LLM prompt:\n{llm_prompt['markdown']}")
+
+asyncio.run(main())
+```
+
+### Hub backups
+
+```python
+import asyncio
+from vectorizer_sdk import VectorizerClient
+
+async def main():
+    client = VectorizerClient(base_url="http://localhost:15002")
+
+    user_id = "user-123"
+
+    # List user's backups
+    backups = await client.list_user_backups(user_id)
+    for backup in backups:
+        print(f"Backup: {backup['id']} (size: {backup['size_bytes']} bytes)")
+
+    # Create a new backup
+    new_backup = await client.create_user_backup(
+        user_id=user_id,
+        name="full-backup-2024-01",
+        description="January full backup",
+        collections=None  # backup all
+    )
+    print(f"Created backup: {new_backup['id']}")
+
+    # Restore a backup
+    await client.restore_user_backup(
+        user_id=user_id,
+        backup_id=new_backup['id']
+    )
+    print("Restore started")
+
+    # Delete old backup
+    await client.delete_user_backup(user_id, backups[0]['id'])
+
+asyncio.run(main())
+```
+
 #### Read Preferences
 
 | Preference | Description | Use Case |