rosetta-cli 3.0.1__tar.gz → 3.0.4__tar.gz

{rosetta_cli-3.0.1/rosetta_cli.egg-info → rosetta_cli-3.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rosetta-cli
-Version: 3.0.1
+Version: 3.0.4
 Summary: Rosetta CLI for publishing knowledge base content to RAGFlow
 Author-email: Rosetta Team <rosetta-support@griddynamics.com>
 License-Expression: Apache-2.0
@@ -42,7 +42,7 @@ This directory contains the Python package for publishing knowledge base content
 
 - **🚀 Smart Publishing** - MD5 hash-based change detection (~77% faster republishing)
 - **🏗️ Modular Architecture** - Command pattern with service layer for maintainability
-- **🏷️ Tag-in-Title Format** - `[tag1][tag2] filename.ext` for powerful server-side filtering
+- **🏷️ Metadata Tags** - path-derived tags in document metadata for `--tags` filtering
 - **📊 Parse Status Tracking** - Monitor document parsing progress with visual indicators
 - **🔄 Upsert Semantics** - No duplicates, republishing updates existing documents
 - **⏱️ Performance Timing** - All commands show execution time
@@ -143,9 +143,9 @@ File: /instructions/agents/r1/agents.md
 
 Published as:
   Document ID: b0ec4d56-6cc5-5bbd-9868-5d49afa2a7d8 (UUID from path)
-  Title: [instructions][agents][r1] agents.md
+  Title: agents.md (path relative to the release folder, no tag prefixes)
   Dataset: aia-r1 (from template: aia-{release})
-  Tags: ["instructions", "agents", "r1"]  (in metadata)
+  Tags: ["instructions", "agents", "r1", "agents.md"]  (in metadata only)
   Domain: instructions (first folder)
   Release: r1 (auto-detected from path)
   Content Hash: abc123... (MD5 of content)
@@ -180,7 +180,7 @@ uvx rosetta-cli@latest list-dataset --dataset aia-r1
 ```
 
 **Output shows:**
-- Document title (with tag prefixes)
+- Document title (path relative to the release folder, no tag prefixes)
 - Document ID, file size, parse status, chunk count
 - Metadata (tags, domain, release, source path)
 
@@ -212,7 +212,7 @@ uvx rosetta-cli@latest cleanup-dataset --dataset aia-r1 --tags "r1,agents" --for
 ⚠️ **Warning:** Without `--prefix` or `--tags`, this deletes ALL documents. Use `--dry-run` first.
 
 **Filtering Options:**
-- `--prefix`: Match documents by title prefix (e.g., `"[instructions]"`)
+- `--prefix`: Match documents by literal title prefix (e.g., `"aqa-phase"`, `"core/skills"`)
 - `--tags`: Match documents by metadata tags (e.g., `"r1 agents"` or `"r1,agents"`)
   - Uses OR logic: finds documents with ANY of the specified tags
   - Server-side filtering for efficiency
@@ -360,7 +360,7 @@ results = publisher.publish(
     force=False,      # Skip unchanged files
     dry_run=False,    # Preview mode
     no_parse=False,   # Skip parsing after upload
-    parse_timeout=300 # Parse timeout in seconds
+    parse_timeout=1200 # Parse timeout in seconds
 )
 
 print(f"Published: {results.published_count}")
@@ -374,7 +374,7 @@ print(f"Failed: {results.failed_count}")
 File: /instructions/agents/r1/bootstrap.md
 
 Extracted:
-  Tags: ["instructions", "agents", "r1"]
+  Tags: ["instructions", "agents", "r1", "bootstrap.md"]
   Domain: instructions
   Release: r1
   Title: bootstrap.md
@@ -382,47 +382,50 @@ Extracted:
   Document ID: uuid-from-path
 ```
 
-## 🎯 Tag-in-Title Format
+## 🎯 Titles and Tags
 
-### What is Tag-in-Title?
+### Document Titles
 
-Documents are stored with tags as prefixes for server-side filtering:
+Titles carry no tag prefixes. A title is the document path relative to its
+release folder, or just the filename for files outside an `instructions/<release>` tree:
 
 ```
-Format: [tag1][tag2][tag3] filename.ext
-
-Examples:
-  [instructions][agents][r1] agents.md
-  [business][project] RFP.pdf
+File: /instructions/agents/r1/agents.md              → Title: agents.md
+File: /instructions/r3/core/skills/coding/SKILL.md   → Title: core/skills/coding/SKILL.md
+File: /business/project/RFP.pdf                      → Title: RFP.pdf
 ```
 
-### Why Two Locations?
-
-Tags are stored in **both title and metadata**:
+### Where Tags Live
 
-**Title:** Fast server-side keyword search
-**Metadata:** Precise client-side filtering with complex queries
+Tags are stored in document **metadata** (`meta_fields.tags`) only, never in the title.
 
 ### How Tags are Generated
 
-Tags come from **folder structure only**:
+Tags come from the file path, plus optional Markdown frontmatter `tags`:
 
 ```
-File: /instructions/agents/r1/bootstrap.md
-Folders: instructions / agents / r1 / (file)
-Tags: [instructions][agents][r1]
+File: /instructions/r3/core/skills/coding/SKILL.md
+Tags: instructions, r3, core, skills, coding, SKILL.md,
+      coding/SKILL.md, skills/coding/SKILL.md
 ```
 
+- Every folder in the path
+- The filename, with extension
+- The last two and last three path segments, joined with `/`
+- Any `tags:` values from Markdown frontmatter (`.md`/`.markdown` only)
+
 ### Using Tags for Filtering
 
 ```bash
 # Delete all instruction documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions]"
+uvx rosetta-cli@latest cleanup-dataset --tags "instructions"
 
-# Delete all r1 agent documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions][agents][r1]"
+# Delete documents tagged "agents" OR "r1"
+uvx rosetta-cli@latest cleanup-dataset --tags "agents,r1"
 ```
 
+Use `--prefix` for literal title-prefix matching instead, e.g. `--prefix "core/skills"`.
+
 ## 💻 Usage Examples
 
 ### Example 1: First-Time Setup
@@ -549,10 +552,11 @@ nano .env
 
 ### Documents Not Showing Tags
 
-Tags should appear in title with format `[tag1][tag2]`:
+Tags live in metadata, not in the title. `list-dataset` prints them on their own line:
 ```bash
 uvx rosetta-cli@latest list-dataset
-# Output: 1. [instructions][agents][r1] agents.md
+# Output: 1. agents.md
+#            Tags: instructions, agents, r1, agents.md
 ```
 
 ## 🚦 Performance Tips
@@ -591,7 +595,7 @@ RAGFLOW_CHUNK_TOKEN_NUM=1024
 
 ```bash
 # Fast: Delete specific documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions][agents]" --force
+uvx rosetta-cli@latest cleanup-dataset --tags "agents" --force
 
 # Slow: Delete and republish everything
 uvx rosetta-cli@latest cleanup-dataset --force

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/README.md

@@ -15,7 +15,7 @@ This directory contains the Python package for publishing knowledge base content
 
 - **🚀 Smart Publishing** - MD5 hash-based change detection (~77% faster republishing)
 - **🏗️ Modular Architecture** - Command pattern with service layer for maintainability
-- **🏷️ Tag-in-Title Format** - `[tag1][tag2] filename.ext` for powerful server-side filtering
+- **🏷️ Metadata Tags** - path-derived tags in document metadata for `--tags` filtering
 - **📊 Parse Status Tracking** - Monitor document parsing progress with visual indicators
 - **🔄 Upsert Semantics** - No duplicates, republishing updates existing documents
 - **⏱️ Performance Timing** - All commands show execution time
@@ -116,9 +116,9 @@ File: /instructions/agents/r1/agents.md
 
 Published as:
   Document ID: b0ec4d56-6cc5-5bbd-9868-5d49afa2a7d8 (UUID from path)
-  Title: [instructions][agents][r1] agents.md
+  Title: agents.md (path relative to the release folder, no tag prefixes)
   Dataset: aia-r1 (from template: aia-{release})
-  Tags: ["instructions", "agents", "r1"]  (in metadata)
+  Tags: ["instructions", "agents", "r1", "agents.md"]  (in metadata only)
   Domain: instructions (first folder)
   Release: r1 (auto-detected from path)
   Content Hash: abc123... (MD5 of content)
@@ -153,7 +153,7 @@ uvx rosetta-cli@latest list-dataset --dataset aia-r1
 ```
 
 **Output shows:**
-- Document title (with tag prefixes)
+- Document title (path relative to the release folder, no tag prefixes)
 - Document ID, file size, parse status, chunk count
 - Metadata (tags, domain, release, source path)
 
@@ -185,7 +185,7 @@ uvx rosetta-cli@latest cleanup-dataset --dataset aia-r1 --tags "r1,agents" --for
 ⚠️ **Warning:** Without `--prefix` or `--tags`, this deletes ALL documents. Use `--dry-run` first.
 
 **Filtering Options:**
-- `--prefix`: Match documents by title prefix (e.g., `"[instructions]"`)
+- `--prefix`: Match documents by literal title prefix (e.g., `"aqa-phase"`, `"core/skills"`)
 - `--tags`: Match documents by metadata tags (e.g., `"r1 agents"` or `"r1,agents"`)
   - Uses OR logic: finds documents with ANY of the specified tags
   - Server-side filtering for efficiency
@@ -333,7 +333,7 @@ results = publisher.publish(
     force=False,      # Skip unchanged files
     dry_run=False,    # Preview mode
     no_parse=False,   # Skip parsing after upload
-    parse_timeout=300 # Parse timeout in seconds
+    parse_timeout=1200 # Parse timeout in seconds
 )
 
 print(f"Published: {results.published_count}")
@@ -347,7 +347,7 @@ print(f"Failed: {results.failed_count}")
 File: /instructions/agents/r1/bootstrap.md
 
 Extracted:
-  Tags: ["instructions", "agents", "r1"]
+  Tags: ["instructions", "agents", "r1", "bootstrap.md"]
   Domain: instructions
   Release: r1
   Title: bootstrap.md
@@ -355,47 +355,50 @@ Extracted:
   Document ID: uuid-from-path
 ```
 
-## 🎯 Tag-in-Title Format
+## 🎯 Titles and Tags
 
-### What is Tag-in-Title?
+### Document Titles
 
-Documents are stored with tags as prefixes for server-side filtering:
+Titles carry no tag prefixes. A title is the document path relative to its
+release folder, or just the filename for files outside an `instructions/<release>` tree:
 
 ```
-Format: [tag1][tag2][tag3] filename.ext
-
-Examples:
-  [instructions][agents][r1] agents.md
-  [business][project] RFP.pdf
+File: /instructions/agents/r1/agents.md              → Title: agents.md
+File: /instructions/r3/core/skills/coding/SKILL.md   → Title: core/skills/coding/SKILL.md
+File: /business/project/RFP.pdf                      → Title: RFP.pdf
 ```
 
-### Why Two Locations?
-
-Tags are stored in **both title and metadata**:
+### Where Tags Live
 
-**Title:** Fast server-side keyword search
-**Metadata:** Precise client-side filtering with complex queries
+Tags are stored in document **metadata** (`meta_fields.tags`) only, never in the title.
 
 ### How Tags are Generated
 
-Tags come from **folder structure only**:
+Tags come from the file path, plus optional Markdown frontmatter `tags`:
 
 ```
-File: /instructions/agents/r1/bootstrap.md
-Folders: instructions / agents / r1 / (file)
-Tags: [instructions][agents][r1]
+File: /instructions/r3/core/skills/coding/SKILL.md
+Tags: instructions, r3, core, skills, coding, SKILL.md,
+      coding/SKILL.md, skills/coding/SKILL.md
 ```
 
+- Every folder in the path
+- The filename, with extension
+- The last two and last three path segments, joined with `/`
+- Any `tags:` values from Markdown frontmatter (`.md`/`.markdown` only)
+
 ### Using Tags for Filtering
 
 ```bash
 # Delete all instruction documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions]"
+uvx rosetta-cli@latest cleanup-dataset --tags "instructions"
 
-# Delete all r1 agent documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions][agents][r1]"
+# Delete documents tagged "agents" OR "r1"
+uvx rosetta-cli@latest cleanup-dataset --tags "agents,r1"
 ```
 
+Use `--prefix` for literal title-prefix matching instead, e.g. `--prefix "core/skills"`.
+
 ## 💻 Usage Examples
 
 ### Example 1: First-Time Setup
@@ -522,10 +525,11 @@ nano .env
 
 ### Documents Not Showing Tags
 
-Tags should appear in title with format `[tag1][tag2]`:
+Tags live in metadata, not in the title. `list-dataset` prints them on their own line:
 ```bash
 uvx rosetta-cli@latest list-dataset
-# Output: 1. [instructions][agents][r1] agents.md
+# Output: 1. agents.md
+#            Tags: instructions, agents, r1, agents.md
 ```
 
 ## 🚦 Performance Tips
@@ -564,7 +568,7 @@ RAGFLOW_CHUNK_TOKEN_NUM=1024
 
 ```bash
 # Fast: Delete specific documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions][agents]" --force
+uvx rosetta-cli@latest cleanup-dataset --tags "agents" --force
 
 # Slow: Delete and republish everything
 uvx rosetta-cli@latest cleanup-dataset --force

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/env.template

@@ -42,6 +42,8 @@ RAGFLOW_AUTO_KEYWORDS=0
 RAGFLOW_AUTO_QUESTIONS=0
 # Page size for listing operations (default: 1000)
 RAGFLOW_PAGE_SIZE=1000
-# Timeout for document parsing operations in seconds (default: 300 = 5 minutes)
-RAGFLOW_PARSE_TIMEOUT=300
+# Timeout for document parsing operations in seconds (default: 1200 = 20 minutes)
+RAGFLOW_PARSE_TIMEOUT=1200
+# HTTP request timeout for RAGFlow health checks in seconds (default: 30)
+RAGFLOW_TIMEOUT=30
 

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "rosetta-cli"
-version = "3.0.1"
+version = "3.0.4"
 description = "Rosetta CLI for publishing knowledge base content to RAGFlow"
 readme = "README.md"
 requires-python = ">=3.12"

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/cli.py

@@ -134,9 +134,10 @@ Performance Notes:
   - Change detection skips unchanged files (77% faster)
   - API key verification timing shown when applicable
 
-Tag-in-Title Format:
-  - Documents are published with tags in title: [tag1][tag2] filename.ext
-  - Example: [instructions][agents][r1] agents.md
+Titles and Tags:
+  - Titles carry no tag prefixes: path relative to the release folder, or the filename
+  - Example: core/skills/coding/SKILL.md
+  - Tags are stored in metadata only; filter with --tags (--prefix matches literal titles)
 
 Frontmatter Metadata (publish flow):
   - Supported keys: tags, sort_order
@@ -192,8 +193,8 @@ Frontmatter Metadata (publish flow):
     publish_parser.add_argument(
         '--parse-timeout',
         type=int,
-        default=300,
-        help='Timeout for parsing in seconds (default: 300)'
+        default=None,
+        help='Timeout for parsing in seconds (default: RAGFLOW_PARSE_TIMEOUT, or 1200)'
     )
     publish_parser.add_argument(
         '--env',
@@ -331,8 +332,8 @@ Frontmatter Metadata (publish flow):
     parse_parser.add_argument(
         '--parse-timeout',
         type=int,
-        default=300,
-        help='Timeout for parsing in seconds (default: 300)'
+        default=None,
+        help='Timeout for parsing in seconds (default: RAGFLOW_PARSE_TIMEOUT, or 1200)'
     )
     parse_parser.add_argument(
         '--env',
@@ -376,7 +377,8 @@ Frontmatter Metadata (publish flow):
             embedding_model=config.embedding_model,
             chunk_method=config.chunk_method,
             parser_config=config.parser_config,
-            page_size=config.page_size
+            page_size=config.page_size,
+            timeout=config.timeout
         )
         
         return execute_command(args.command, args, client, config)

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/commands/cleanup_command.py

@@ -94,13 +94,13 @@ class CleanupCommand(BaseCommand):
             # Filter by tags (metadata condition)
             tags_list = self._parse_tags(args.tags)
             filtered_documents = document_service.filter_documents_by_tags(
-                dataset, tags_list
+                dataset, tags_list, limit=self.config.page_size
             )
             print(f"\nFiltered {len(filtered_documents)} document(s) with tags: {', '.join(tags_list)}\n")
         elif args.prefix:
             # Filter by prefix
             filtered_documents = document_service.filter_documents_by_prefix(
-                dataset, args.prefix
+                dataset, args.prefix, limit=self.config.page_size
             )
             print(f"\nFiltered {len(filtered_documents)} document(s) matching prefix '{args.prefix}'\n")
         else:

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/commands/parse_command.py

@@ -25,8 +25,10 @@ class ParseCommand(BaseCommand):
         """Execute parse command."""
         self._start_timing()
 
-        # CLI flag must override config default for this run.
-        self.config.parse_timeout = args.parse_timeout
+        # Explicit CLI flag wins for this run; an unset flag keeps the
+        # configured (RAGFLOW_PARSE_TIMEOUT) value.
+        if getattr(args, "parse_timeout", None) is not None:
+            self.config.parse_timeout = args.parse_timeout
 
         # Verify authentication before any dataset auto-detection touches the API.
         AuthService.verify_or_exit(self.client, self.config)

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/commands/publish_command.py

@@ -25,13 +25,19 @@ class PublishCommand(BaseCommand):
         Execute publish command.
         
         Args:
-            args: Command arguments with path, dry_run, force, no_parse flags
+            args: Command arguments with path, dry_run, force, no_parse,
+                  parse_timeout flags
         
         Returns:
             0 if successful, 1 if failed
         """
         self._start_timing()
         
+        # Explicit CLI flag wins for this run; an unset flag keeps the
+        # configured (RAGFLOW_PARSE_TIMEOUT) value.
+        if getattr(args, "parse_timeout", None) is not None:
+            self.config.parse_timeout = args.parse_timeout
+        
         # Print header
         print(f"Publishing knowledge base content from: {args.path}")
         self._print_header_with_api_key()
@@ -47,7 +53,8 @@ class PublishCommand(BaseCommand):
             self.client,
             str(workspace_root),
             dataset_default=self.config.dataset_default,
-            dataset_template=self.config.dataset_template
+            dataset_template=self.config.dataset_template,
+            parse_timeout=self.config.parse_timeout
         )
         
         # Publish

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/ragflow_client.py

@@ -6,8 +6,8 @@ This module provides a wrapper around the ragflow-sdk for Rosetta-specific opera
 Key Features:
 - Dataset management with template resolution (aia-{release})
 - Document upload with change detection (MD5 hashing)
-- Tag-in-title format: [tag1][tag2][tag3] filename.ext
-- Two-stage filtering support (server keyword + client metadata)
+- Plain titles (path relative to the release folder, or the filename)
+- Tags stored in meta_fields for metadata filtering
 """
 
 import hashlib
@@ -90,7 +90,7 @@ class RAGFlowClient:
     Wrapper class for RAGFlow SDK operations.
     
     Provides high-level methods for dataset and document management
-    with Rosetta-specific functionality like tag-in-title format and
+    with Rosetta-specific functionality like metadata tagging and
     change detection.
     
     Usage:
@@ -406,7 +406,10 @@ class RAGFlowClient:
                 if cached is not None:
                     return cached
 
-                # Filter by name (RAGFlow does substring, we verify exact match)
+                # Filter by name (RAGFlow does substring, we verify exact match).
+                # page_size=10 is intentional and sufficient: dataset names are
+                # unique, so this lookup can return at most one exact match.
+                # Do not "fix" this to a larger page size or a pagination loop.
                 with _timed(f"list_datasets(name={name})"):
                     datasets = self._client.list_datasets(name=name, page_size=10)
                 # Filter for exact match

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/rosetta_config.py

@@ -19,6 +19,11 @@ logger = logging.getLogger(__name__)
 
 ENV_FILE_ENV_VAR = "ROSETTA_CLI_ENV_FILE"
 
+# Default timeouts in seconds. Single source of truth for the dataclass
+# defaults, the env-var fallbacks, and env.template.
+DEFAULT_TIMEOUT = 30
+DEFAULT_PARSE_TIMEOUT = 1200
+
 
 def _candidate_env_names(env_name: str | None) -> list[str]:
     names: list[str] = []
@@ -92,6 +97,7 @@ class RosettaConfig:
         RAGFLOW_AUTO_QUESTIONS: Auto-generate questions per chunk (default: 0)
         RAGFLOW_PAGE_SIZE: Page size for listing operations (default: 1000)
         RAGFLOW_PARSE_TIMEOUT: Timeout for parsing operations in seconds (default: 1200)
+        RAGFLOW_TIMEOUT: HTTP request timeout in seconds (default: 30)
         ENVIRONMENT: Environment name (default: "local")
     
     Examples:
@@ -112,7 +118,8 @@ class RosettaConfig:
     parser_config: JsonDict | None = None
     environment: str = "local"
     page_size: int = 1000
-    parse_timeout: int = 300
+    parse_timeout: int = DEFAULT_PARSE_TIMEOUT
+    timeout: int = DEFAULT_TIMEOUT
     
     @classmethod
     def from_env(
@@ -216,7 +223,8 @@ class RosettaConfig:
         
         # Pagination and timeout settings
         page_size = int(os.getenv("RAGFLOW_PAGE_SIZE", "1000"))
-        parse_timeout = int(os.getenv("RAGFLOW_PARSE_TIMEOUT", "1200"))
+        parse_timeout = int(os.getenv("RAGFLOW_PARSE_TIMEOUT", str(DEFAULT_PARSE_TIMEOUT)))
+        timeout = int(os.getenv("RAGFLOW_TIMEOUT", str(DEFAULT_TIMEOUT)))
         
         # Parser configuration for naive chunking
         parser_config: JsonDict | None = None
@@ -243,7 +251,8 @@ class RosettaConfig:
             parser_config=parser_config,
             environment=environment,
             page_size=page_size,
-            parse_timeout=parse_timeout
+            parse_timeout=parse_timeout,
+            timeout=timeout
         )
     
     def validate(self) -> bool:
@@ -268,7 +277,9 @@ class RosettaConfig:
             )
         
         if not self.api_key.startswith("ragflow-"):
-            logger.warning("API key should start with 'ragflow-', got: %s...", self.api_key[:10])
+            # Never log any part of the key: the "ragflow-" prefix is 8 chars, so a
+            # 10-char excerpt leaks secret material, and the caller knows their own key.
+            logger.warning("API key should start with 'ragflow-'")
         
         return True
     
@@ -321,7 +332,9 @@ class RosettaConfig:
     
     def __str__(self) -> str:
         """String representation (masks API key)"""
-        masked_key = f"{self.api_key[:10]}..." if len(self.api_key) > 10 else "***"
+        # Show only the 8-char "ragflow-" prefix. A 10-char excerpt would expose two
+        # characters of secret material, which the docstring's "masks API key" rules out.
+        masked_key = f"{self.api_key[:8]}..." if len(self.api_key) > 8 else "***"
         return (
             f"RosettaConfig(\n"
             f"  base_url={self.base_url}\n"

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/rosetta_publisher.py

@@ -6,9 +6,8 @@ from folder structure.
 
 Features:
 - RAGFlow SDK integration for document upload and management
-- Tag-in-title format: [tag1][tag2] filename.ext
-- Preserves dots in filenames (e.g., "agents.md" stays "agents.md")
-- Two-location tag storage: title + meta_fields for optimal search performance
+- Plain filename titles (e.g., "agents.md" stays "agents.md")
+- Tags stored in meta_fields for metadata filtering
 - Dataset-based organization
 - MD5 hash-based change detection
 """
@@ -26,6 +25,7 @@ from typing import cast
 from .services.document_service import DocumentService
 from .services.document_data import DocumentData
 from .ragflow_client import DocumentMetadata, RAGFlowClient, RAGFlowClientError
+from .rosetta_config import DEFAULT_PARSE_TIMEOUT
 from .typing_utils import DocumentLike, JsonDict
 
 # Extensions RAGFlow can actually parse (from ragflow source: api/utils/file_utils.py).
@@ -80,7 +80,8 @@ class ContentPublisher:
         dataset_default: str = "aia",
         dataset_template: str = "aia-{release}",
         enable_change_tracking: bool = True,
-        file_extensions: list[str] | None = None
+        file_extensions: list[str] | None = None,
+        parse_timeout: int = DEFAULT_PARSE_TIMEOUT
     ):
         """
         Initialize the publisher.
@@ -92,6 +93,7 @@ class ContentPublisher:
             dataset_template: Dataset name template (can use {release} placeholder)
             enable_change_tracking: Enable hash-based change detection (default: True)
             file_extensions: List of file extensions to publish (default: None = all files)
+            parse_timeout: Max seconds to wait for parsing (default: 1200)
         """
         self.client = client
         self.workspace_root = Path(workspace_root).resolve()
@@ -99,6 +101,7 @@ class ContentPublisher:
         self.dataset_template = dataset_template
         self.enable_change_tracking = enable_change_tracking
         self.file_extensions = file_extensions  # None = all files (no extension filter)
+        self.parse_timeout = parse_timeout
         self._skip_names = {'.DS_Store', 'Thumbs.db', '.gitkeep', '.mcp.json'}
         self._skip_folders = {'.cursor-plugin', '.claude-plugin'}
     
@@ -502,12 +505,12 @@ class ContentPublisher:
             if docs_to_wait:
                 # Use DocumentService for consistent waiting with progress bar
                 doc_service = DocumentService(self.client)
-                doc_service.wait_for_parsing(docs_to_wait)
+                doc_service.wait_for_parsing(docs_to_wait, self.parse_timeout)
     
     def _wait_for_all_parsing_with_progress(
         self,
         documents: list[JsonDict],
-        timeout: int = 300,  # 5 minutes
+        timeout: int | None = None,
         poll_interval: float = 0.5
     ) -> None:
         """
@@ -517,10 +520,12 @@ class ContentPublisher:
         
         Args:
             documents: List of {"id": doc_id, "name": name, "dataset_id": dataset_id, "folder": folder}
-            timeout: Max seconds to wait
+            timeout: Max seconds to wait (default: None = configured parse_timeout)
             poll_interval: Seconds between status checks (reduced to 0.5 for smoother progress)
         """ 
         doc_service = DocumentService(self.client)
+        if timeout is None:
+            timeout = self.parse_timeout
         doc_service.wait_for_parsing(documents, timeout, poll_interval)
     
     def _has_content_changed_cached(self, cache: DocumentData) -> bool:

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4}/rosetta_cli/services/document_service.py

@@ -10,6 +10,7 @@ from typing import cast
 
 from tqdm import tqdm
 from ..ragflow_client import RAGFlowClient
+from ..rosetta_config import DEFAULT_PARSE_TIMEOUT
 from ..typing_utils import DatasetLike, DocumentLike, JsonDict
 
 
@@ -227,7 +228,7 @@ class DocumentService:
     def wait_for_parsing(
         self,
         documents: list[JsonDict],
-        timeout: int = 300,
+        timeout: int = DEFAULT_PARSE_TIMEOUT,
         poll_interval: float = 0.5
     ) -> tuple[int, int]:
         """
@@ -235,7 +236,7 @@ class DocumentService:
         
         Args:
             documents: List of {"id": doc_id, "name": name, "dataset_id": dataset_id, "folder": folder}
-            timeout: Max seconds to wait (default: 300 = 5 minutes)
+            timeout: Max seconds to wait (default: DEFAULT_PARSE_TIMEOUT = 1200 = 20 minutes)
             poll_interval: Seconds between status checks (default: 0.5 for smooth progress)
         
         Returns:

{rosetta_cli-3.0.1 → rosetta_cli-3.0.4/rosetta_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rosetta-cli
-Version: 3.0.1
+Version: 3.0.4
 Summary: Rosetta CLI for publishing knowledge base content to RAGFlow
 Author-email: Rosetta Team <rosetta-support@griddynamics.com>
 License-Expression: Apache-2.0
@@ -42,7 +42,7 @@ This directory contains the Python package for publishing knowledge base content
 
 - **🚀 Smart Publishing** - MD5 hash-based change detection (~77% faster republishing)
 - **🏗️ Modular Architecture** - Command pattern with service layer for maintainability
-- **🏷️ Tag-in-Title Format** - `[tag1][tag2] filename.ext` for powerful server-side filtering
+- **🏷️ Metadata Tags** - path-derived tags in document metadata for `--tags` filtering
 - **📊 Parse Status Tracking** - Monitor document parsing progress with visual indicators
 - **🔄 Upsert Semantics** - No duplicates, republishing updates existing documents
 - **⏱️ Performance Timing** - All commands show execution time
@@ -143,9 +143,9 @@ File: /instructions/agents/r1/agents.md
 
 Published as:
   Document ID: b0ec4d56-6cc5-5bbd-9868-5d49afa2a7d8 (UUID from path)
-  Title: [instructions][agents][r1] agents.md
+  Title: agents.md (path relative to the release folder, no tag prefixes)
   Dataset: aia-r1 (from template: aia-{release})
-  Tags: ["instructions", "agents", "r1"]  (in metadata)
+  Tags: ["instructions", "agents", "r1", "agents.md"]  (in metadata only)
   Domain: instructions (first folder)
   Release: r1 (auto-detected from path)
   Content Hash: abc123... (MD5 of content)
@@ -180,7 +180,7 @@ uvx rosetta-cli@latest list-dataset --dataset aia-r1
 ```
 
 **Output shows:**
-- Document title (with tag prefixes)
+- Document title (path relative to the release folder, no tag prefixes)
 - Document ID, file size, parse status, chunk count
 - Metadata (tags, domain, release, source path)
 
@@ -212,7 +212,7 @@ uvx rosetta-cli@latest cleanup-dataset --dataset aia-r1 --tags "r1,agents" --for
 ⚠️ **Warning:** Without `--prefix` or `--tags`, this deletes ALL documents. Use `--dry-run` first.
 
 **Filtering Options:**
-- `--prefix`: Match documents by title prefix (e.g., `"[instructions]"`)
+- `--prefix`: Match documents by literal title prefix (e.g., `"aqa-phase"`, `"core/skills"`)
 - `--tags`: Match documents by metadata tags (e.g., `"r1 agents"` or `"r1,agents"`)
   - Uses OR logic: finds documents with ANY of the specified tags
   - Server-side filtering for efficiency
@@ -360,7 +360,7 @@ results = publisher.publish(
     force=False,      # Skip unchanged files
     dry_run=False,    # Preview mode
     no_parse=False,   # Skip parsing after upload
-    parse_timeout=300 # Parse timeout in seconds
+    parse_timeout=1200 # Parse timeout in seconds
 )
 
 print(f"Published: {results.published_count}")
@@ -374,7 +374,7 @@ print(f"Failed: {results.failed_count}")
 File: /instructions/agents/r1/bootstrap.md
 
 Extracted:
-  Tags: ["instructions", "agents", "r1"]
+  Tags: ["instructions", "agents", "r1", "bootstrap.md"]
   Domain: instructions
   Release: r1
   Title: bootstrap.md
@@ -382,47 +382,50 @@ Extracted:
   Document ID: uuid-from-path
 ```
 
-## 🎯 Tag-in-Title Format
+## 🎯 Titles and Tags
 
-### What is Tag-in-Title?
+### Document Titles
 
-Documents are stored with tags as prefixes for server-side filtering:
+Titles carry no tag prefixes. A title is the document path relative to its
+release folder, or just the filename for files outside an `instructions/<release>` tree:
 
 ```
-Format: [tag1][tag2][tag3] filename.ext
-
-Examples:
-  [instructions][agents][r1] agents.md
-  [business][project] RFP.pdf
+File: /instructions/agents/r1/agents.md              → Title: agents.md
+File: /instructions/r3/core/skills/coding/SKILL.md   → Title: core/skills/coding/SKILL.md
+File: /business/project/RFP.pdf                      → Title: RFP.pdf
 ```
 
-### Why Two Locations?
-
-Tags are stored in **both title and metadata**:
+### Where Tags Live
 
-**Title:** Fast server-side keyword search
-**Metadata:** Precise client-side filtering with complex queries
+Tags are stored in document **metadata** (`meta_fields.tags`) only, never in the title.
 
 ### How Tags are Generated
 
-Tags come from **folder structure only**:
+Tags come from the file path, plus optional Markdown frontmatter `tags`:
 
 ```
-File: /instructions/agents/r1/bootstrap.md
-Folders: instructions / agents / r1 / (file)
-Tags: [instructions][agents][r1]
+File: /instructions/r3/core/skills/coding/SKILL.md
+Tags: instructions, r3, core, skills, coding, SKILL.md,
+      coding/SKILL.md, skills/coding/SKILL.md
 ```
 
+- Every folder in the path
+- The filename, with extension
+- The last two and last three path segments, joined with `/`
+- Any `tags:` values from Markdown frontmatter (`.md`/`.markdown` only)
+
 ### Using Tags for Filtering
 
 ```bash
 # Delete all instruction documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions]"
+uvx rosetta-cli@latest cleanup-dataset --tags "instructions"
 
-# Delete all r1 agent documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions][agents][r1]"
+# Delete documents tagged "agents" OR "r1"
+uvx rosetta-cli@latest cleanup-dataset --tags "agents,r1"
 ```
 
+Use `--prefix` for literal title-prefix matching instead, e.g. `--prefix "core/skills"`.
+
 ## 💻 Usage Examples
 
 ### Example 1: First-Time Setup
@@ -549,10 +552,11 @@ nano .env
 
 ### Documents Not Showing Tags
 
-Tags should appear in title with format `[tag1][tag2]`:
+Tags live in metadata, not in the title. `list-dataset` prints them on their own line:
 ```bash
 uvx rosetta-cli@latest list-dataset
-# Output: 1. [instructions][agents][r1] agents.md
+# Output: 1. agents.md
+#            Tags: instructions, agents, r1, agents.md
 ```
 
 ## 🚦 Performance Tips
@@ -591,7 +595,7 @@ RAGFLOW_CHUNK_TOKEN_NUM=1024
 
 ```bash
 # Fast: Delete specific documents
-uvx rosetta-cli@latest cleanup-dataset --prefix "[instructions][agents]" --force
+uvx rosetta-cli@latest cleanup-dataset --tags "agents" --force
 
 # Slow: Delete and republish everything
 uvx rosetta-cli@latest cleanup-dataset --force