sf-queue-sdk 0.3.1b233__tar.gz → 0.3.1b255__tar.gz

sf_queue_sdk-0.3.1b233/README.md → sf_queue_sdk-0.3.1b255/PKG-INFO

@@ -1,3 +1,11 @@
+Metadata-Version: 2.4
+Name: sf-queue-sdk
+Version: 0.3.1b255
+Summary: Python SDK for sf-queue - Redis-based queue system
+Requires-Python: >=3.11
+Requires-Dist: redis>=5.0.0
+Description-Content-Type: text/markdown
+
 # sf-queue-sdk
 
 Python SDK for sf-queue. Enqueues emails via Redis Streams with optional blocking confirmation from the Go consumer service.
@@ -607,6 +615,7 @@ result = client.document_upload.send(
 | `folder_id` | `Optional[str]` | No | Folder for organization |
 | `upload_batch_id` | `Optional[str]` | No | Batch ID when multiple files uploaded together |
 | `youtube_title` | `Optional[str]` | No | Title override for YouTube video transcripts |
+| `skip_material_creation` | `Optional[bool]` | No | Skip folder and material creation when the material already exists (e.g. live lectures) |
 
 ### Response Type
 
@@ -662,11 +671,69 @@ print(f"Pipeline: {status.pipeline_type}")             # e.g. "pdf", "youtube",
 
 Status data is stored in Redis with a 1-hour TTL, reset on every update. If `get_status()` is called with an unknown request ID or after the TTL expires, it returns `status="unknown"`.
 
-## Document Persist (Deferred Note Generation)
+## Document Notes (Note Generation)
+
+Enqueue note generation jobs after clustering completes. In the three-queue architecture, Queue 1 (document-upload) handles extraction and clustering, Queue 2 (document-notes) generates notes and persists materials, and Queue 3 (document-persist) links materials to topics after user approval.
+
+Queue 1 automatically enqueues to document-notes after clustering. The notes queue generates notes, persists materials, and sends the callback that closes the upload modal.
+
+### Fire and forget
+
+```python
+result = client.document_notes.send(
+    uploaded_file_id="file_abc123",
+    study_set_id="set_xyz",
+    user_id="user_123",
+    stash_key="clustering-stash/file_abc123/1720000000.json",
+    is_deferred=True,
+    callback_url="https://api.example.com/completeCluster",
+    callback_api_key="secret",
+)
+
+print("Enqueued:", result.message_id)
+```
+
+### With notes disabled
+
+```python
+result = client.document_notes.send(
+    uploaded_file_id="file_abc123",
+    study_set_id="set_xyz",
+    user_id="user_123",
+    stash_key="clustering-stash/file_abc123/1720000000.json",
+    is_deferred=False,
+    generate_notes=False,
+    folder_id="folder_abc",
+    callback_url="https://api.example.com/completeCluster",
+    callback_api_key="secret",
+)
+```
+
+### Document Notes Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `uploaded_file_id` | `str` | Yes | ID of the uploaded file record |
+| `study_set_id` | `str` | Yes | Study set to persist materials into |
+| `user_id` | `str` | Yes | User who uploaded the document |
+| `stash_key` | `str` | Yes | GCS key of the clustering stash JSON |
+| `is_deferred` | `bool` | No | Whether the study set has existing topics requiring approval (default: `False`) |
+| `generate_notes` | `bool` | No | Whether to generate notes (default: `True`). Set to `False` for noNotes materials |
+| `callback_url` | `Optional[str]` | No | URL to POST results to after completion |
+| `callback_api_key` | `Optional[str]` | No | Auth key for the callback |
+| `folder_id` | `Optional[str]` | No | Pre-created folder ID for the materials |
+
+### Response Types
+
+| Method | Return Type | Fields |
+|--------|-------------|--------|
+| `send()` | `SendResult` | `message_id` |
+
+## Document Persist (Topic Linking)
 
-Enqueue deferred note generation + material persistence jobs. Used in the two-queue architecture where Queue 1 (document-upload) handles extraction and clustering, and Queue 2 (document-persist) handles note generation and persistence after user approval.
+Enqueue topic-linking jobs after user approval. In the three-queue architecture, Queue 3 (document-persist) loads the material map from the stash sidecar and links pre-generated materials to approved topics.
 
-For new study sets (no existing study path), Queue 1 auto-forwards to document-persist. For existing study sets, the clustering output is stashed and presented as pending changes for user review. After approval, the approve route enqueues to document-persist.
+For new study sets (no existing study path), Queue 2 handles everything. For existing study sets, the approve route enqueues to document-persist after the user reviews pending changes.
 
 ### Fire and forget
 

sf_queue_sdk-0.3.1b233/PKG-INFO → sf_queue_sdk-0.3.1b255/README.md

@@ -1,11 +1,3 @@
-Metadata-Version: 2.4
-Name: sf-queue-sdk
-Version: 0.3.1b233
-Summary: Python SDK for sf-queue - Redis-based queue system
-Requires-Python: >=3.11
-Requires-Dist: redis>=5.0.0
-Description-Content-Type: text/markdown
-
 # sf-queue-sdk
 
 Python SDK for sf-queue. Enqueues emails via Redis Streams with optional blocking confirmation from the Go consumer service.
@@ -615,6 +607,7 @@ result = client.document_upload.send(
 | `folder_id` | `Optional[str]` | No | Folder for organization |
 | `upload_batch_id` | `Optional[str]` | No | Batch ID when multiple files uploaded together |
 | `youtube_title` | `Optional[str]` | No | Title override for YouTube video transcripts |
+| `skip_material_creation` | `Optional[bool]` | No | Skip folder and material creation when the material already exists (e.g. live lectures) |
 
 ### Response Type
 
@@ -670,11 +663,69 @@ print(f"Pipeline: {status.pipeline_type}")             # e.g. "pdf", "youtube",
 
 Status data is stored in Redis with a 1-hour TTL, reset on every update. If `get_status()` is called with an unknown request ID or after the TTL expires, it returns `status="unknown"`.
 
-## Document Persist (Deferred Note Generation)
+## Document Notes (Note Generation)
+
+Enqueue note generation jobs after clustering completes. In the three-queue architecture, Queue 1 (document-upload) handles extraction and clustering, Queue 2 (document-notes) generates notes and persists materials, and Queue 3 (document-persist) links materials to topics after user approval.
+
+Queue 1 automatically enqueues to document-notes after clustering. The notes queue generates notes, persists materials, and sends the callback that closes the upload modal.
+
+### Fire and forget
+
+```python
+result = client.document_notes.send(
+    uploaded_file_id="file_abc123",
+    study_set_id="set_xyz",
+    user_id="user_123",
+    stash_key="clustering-stash/file_abc123/1720000000.json",
+    is_deferred=True,
+    callback_url="https://api.example.com/completeCluster",
+    callback_api_key="secret",
+)
+
+print("Enqueued:", result.message_id)
+```
+
+### With notes disabled
+
+```python
+result = client.document_notes.send(
+    uploaded_file_id="file_abc123",
+    study_set_id="set_xyz",
+    user_id="user_123",
+    stash_key="clustering-stash/file_abc123/1720000000.json",
+    is_deferred=False,
+    generate_notes=False,
+    folder_id="folder_abc",
+    callback_url="https://api.example.com/completeCluster",
+    callback_api_key="secret",
+)
+```
+
+### Document Notes Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `uploaded_file_id` | `str` | Yes | ID of the uploaded file record |
+| `study_set_id` | `str` | Yes | Study set to persist materials into |
+| `user_id` | `str` | Yes | User who uploaded the document |
+| `stash_key` | `str` | Yes | GCS key of the clustering stash JSON |
+| `is_deferred` | `bool` | No | Whether the study set has existing topics requiring approval (default: `False`) |
+| `generate_notes` | `bool` | No | Whether to generate notes (default: `True`). Set to `False` for noNotes materials |
+| `callback_url` | `Optional[str]` | No | URL to POST results to after completion |
+| `callback_api_key` | `Optional[str]` | No | Auth key for the callback |
+| `folder_id` | `Optional[str]` | No | Pre-created folder ID for the materials |
+
+### Response Types
+
+| Method | Return Type | Fields |
+|--------|-------------|--------|
+| `send()` | `SendResult` | `message_id` |
+
+## Document Persist (Topic Linking)
 
-Enqueue deferred note generation + material persistence jobs. Used in the two-queue architecture where Queue 1 (document-upload) handles extraction and clustering, and Queue 2 (document-persist) handles note generation and persistence after user approval.
+Enqueue topic-linking jobs after user approval. In the three-queue architecture, Queue 3 (document-persist) loads the material map from the stash sidecar and links pre-generated materials to approved topics.
 
-For new study sets (no existing study path), Queue 1 auto-forwards to document-persist. For existing study sets, the clustering output is stashed and presented as pending changes for user review. After approval, the approve route enqueues to document-persist.
+For new study sets (no existing study path), Queue 2 handles everything. For existing study sets, the approve route enqueues to document-persist after the user reviews pending changes.
 
 ### Fire and forget
 

{sf_queue_sdk-0.3.1b233 → sf_queue_sdk-0.3.1b255}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "sf-queue-sdk"
-version = "0.3.1b233"
+version = "0.3.1b255"
 description = "Python SDK for sf-queue - Redis-based queue system"
 readme = "README.md"
 requires-python = ">=3.11"

{sf_queue_sdk-0.3.1b233 → sf_queue_sdk-0.3.1b255}/queue_sdk/__init__.py

@@ -1,6 +1,6 @@
 """sf-queue Python SDK - Redis-based queue system."""
 
-__version__ = "0.3.1b233"
+__version__ = "0.3.1b255"
 
 from queue_sdk.client import QueueClient
 from queue_sdk.query import QueryClient
@@ -13,6 +13,7 @@ from queue_sdk.types import (
     BatchTopicAssignmentData,
     ConvertData,
     ConvertResponse,
+    DocumentNotesData,
     DocumentPersistData,
     DocumentPersistResponse,
     DocumentUploadData,
@@ -54,6 +55,7 @@ __all__ = [
     "BatchToolAssignmentData",
     "ConvertData",
     "ConvertResponse",
+    "DocumentNotesData",
     "DocumentPersistData",
     "DocumentPersistResponse",
     "DocumentUploadData",

{sf_queue_sdk-0.3.1b233 → sf_queue_sdk-0.3.1b255}/queue_sdk/client.py

@@ -1,6 +1,7 @@
 """QueueClient - main entry point for the Python SDK."""
 
 from queue_sdk.query import QueryClient
+from queue_sdk.queues.document_notes_queue import DocumentNotesQueue
 from queue_sdk.queues.document_persist_queue import DocumentPersistQueue
 from queue_sdk.queues.document_upload_queue import DocumentUploadQueue
 from queue_sdk.queues.email_queue import EmailQueue
@@ -72,6 +73,7 @@ class QueueClient:
         self.pdf_to_image = PdfToImageQueue(self._redis, self._environment)
         self.recommendations = RecommendationQueue(self._redis, self._environment)
         self.document_upload = DocumentUploadQueue(self._redis, self._environment)
+        self.document_notes = DocumentNotesQueue(self._redis, self._environment)
         self.document_persist = DocumentPersistQueue(self._redis, self._environment)
         self.prompt_ingest = PromptIngestQueue(self._redis, self._environment)
         self.query = QueryClient(self._redis)

sf_queue_sdk-0.3.1b255/queue_sdk/queues/document_notes_queue.py

@@ -0,0 +1,95 @@
+"""Document Notes queue for note generation after clustering."""
+
+from typing import Any, Optional
+
+import redis as redis_lib
+
+from queue_sdk.queues.base_queue import BaseQueue
+from queue_sdk.types import SendResult
+
+
+class DocumentNotesQueue(BaseQueue):
+    """Typed methods for enqueueing note generation jobs.
+
+    Queue 1 (cluster) enqueues these automatically after clustering
+    completes. The notes queue generates notes, persists materials,
+    and sends the callback to the frontend.
+    """
+
+    def __init__(self, redis_client: redis_lib.Redis, environment: str):
+        super().__init__(redis_client, environment, "document-notes")
+
+    def send(
+        self,
+        uploaded_file_id: str,
+        study_set_id: str,
+        user_id: str,
+        stash_key: str,
+        is_deferred: bool = False,
+        generate_notes: bool = True,
+        callback_url: Optional[str] = None,
+        callback_api_key: Optional[str] = None,
+        folder_id: Optional[str] = None,
+    ) -> SendResult:
+        """Enqueue a note generation request."""
+        self._validate(uploaded_file_id, study_set_id, user_id, stash_key)
+        payload = self._build_payload(
+            uploaded_file_id=uploaded_file_id,
+            study_set_id=study_set_id,
+            user_id=user_id,
+            stash_key=stash_key,
+            is_deferred=is_deferred,
+            generate_notes=generate_notes,
+            callback_url=callback_url,
+            callback_api_key=callback_api_key,
+            folder_id=folder_id,
+        )
+        message_id = self._enqueue(payload)
+
+        return SendResult(message_id=message_id)
+
+    @staticmethod
+    def _build_payload(
+        uploaded_file_id: str,
+        study_set_id: str,
+        user_id: str,
+        stash_key: str,
+        is_deferred: bool,
+        generate_notes: bool,
+        callback_url: Optional[str],
+        callback_api_key: Optional[str],
+        folder_id: Optional[str],
+    ) -> dict[str, Any]:
+        payload: dict[str, Any] = {
+            "uploadedFileId": uploaded_file_id,
+            "studySetId": study_set_id,
+            "userId": user_id,
+            "stashKey": stash_key,
+            "isDeferred": is_deferred,
+            "generateNotes": generate_notes,
+        }
+
+        if callback_url is not None:
+            payload["callbackUrl"] = callback_url
+        if callback_api_key is not None:
+            payload["callbackApiKey"] = callback_api_key
+        if folder_id is not None:
+            payload["folderId"] = folder_id
+
+        return payload
+
+    @staticmethod
+    def _validate(
+        uploaded_file_id: str,
+        study_set_id: str,
+        user_id: str,
+        stash_key: str,
+    ) -> None:
+        if not uploaded_file_id:
+            raise ValueError("uploaded_file_id is required")
+        if not study_set_id:
+            raise ValueError("study_set_id is required")
+        if not user_id:
+            raise ValueError("user_id is required")
+        if not stash_key:
+            raise ValueError("stash_key is required")

{sf_queue_sdk-0.3.1b233 → sf_queue_sdk-0.3.1b255}/queue_sdk/queues/document_upload_queue.py

@@ -50,6 +50,7 @@ class DocumentUploadQueue(BaseQueue):
         callback_url: Optional[str] = None,
         callback_api_key: Optional[str] = None,
         study_plan_hook: Optional[bool] = None,
+        skip_material_creation: Optional[bool] = None,
     ) -> SendResult:
         """Enqueue a document upload processing request (fire and forget)."""
         self._validate(uploaded_file_id, study_set_id, user_id, pdf_s3_key, filename)
@@ -84,6 +85,7 @@ class DocumentUploadQueue(BaseQueue):
             callback_url=callback_url,
             callback_api_key=callback_api_key,
             study_plan_hook=study_plan_hook,
+            skip_material_creation=skip_material_creation,
         )
         message_id = self._enqueue(payload)
         self._set_initial_status(message_id)
@@ -168,6 +170,7 @@ class DocumentUploadQueue(BaseQueue):
         callback_url: Optional[str],
         callback_api_key: Optional[str],
         study_plan_hook: Optional[bool],
+        skip_material_creation: Optional[bool],
     ) -> dict[str, Any]:
         """Build the document upload payload (snake_case -> camelCase)."""
         payload: dict[str, Any] = {
@@ -217,6 +220,8 @@ class DocumentUploadQueue(BaseQueue):
             payload["callbackApiKey"] = callback_api_key
         if study_plan_hook is not None:
             payload["studyPlanHook"] = study_plan_hook
+        if skip_material_creation is not None:
+            payload["skipMaterialCreation"] = skip_material_creation
 
         return payload
 

{sf_queue_sdk-0.3.1b233 → sf_queue_sdk-0.3.1b255}/queue_sdk/types.py

@@ -268,9 +268,24 @@ class PromptIngestResponse:
     processed_at: Optional[str] = None
 
 
+@dataclass
+class DocumentNotesData:
+    """Data for a note generation request after clustering."""
+
+    uploaded_file_id: str
+    study_set_id: str
+    user_id: str
+    stash_key: str
+    is_deferred: bool = False
+    generate_notes: bool = True
+    callback_url: Optional[str] = None
+    callback_api_key: Optional[str] = None
+    folder_id: Optional[str] = None
+
+
 @dataclass
 class DocumentPersistData:
-    """Data for a deferred note generation + persistence request."""
+    """Data for a topic-linking request after approval."""
 
     uploaded_file_id: str
     study_set_id: str