fruxon 0.13.2__tar.gz → 0.13.3__tar.gz

{fruxon-0.13.2 → fruxon-0.13.3}/HISTORY.md

@@ -1,5 +1,21 @@
 # History
 
+## Unreleased — A knowledge base can be consolidated
+
+Serial merge-per-conversation cannot repair a pair once both halves exist:
+each was written before the other was searchable. The backend now runs a
+pass over the whole corpus (fruxon-backend, `knowledgeBases/{kb}:consolidate`).
+
+- **New `fruxon knowledge-bases consolidate <kb>`** — embeds the base's
+  live documents, clusters those above a similarity threshold, and prints
+  the clusters. Report-only unless `--apply`, which has the base's
+  consolidator agent decide each cluster and merge it (survivor keeps the
+  union of tags, labels and source conversations; the rest are archived).
+  Waits for the report by default; `--no-wait` prints the queued run and
+  `fruxon knowledge-bases consolidation <kb> <run>` reads it later.
+- `agents executions list --trigger-type` accepts `KNOWLEDGE_CONSOLIDATION`,
+  the trigger the consolidator agent's runs carry.
+
 ## Unreleased — What a listing leaves out, and what a typo returns
 
 A patch release from a CLI trial run against a live workspace. Four defects,

{fruxon-0.13.2 → fruxon-0.13.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: fruxon
-Version: 0.13.2
+Version: 0.13.3
 Summary: The Fruxon SDK is a lightweight Python client for integrating with the Fruxon platform.
 Project-URL: bugs, https://github.com/fruxon-ai/fruxon-sdk/issues
 Project-URL: changelog, https://github.com/fruxon-ai/fruxon-sdk/blob/main/HISTORY.md
@@ -138,6 +138,7 @@ Integrations + tools:
 | `fruxon keys list/mint/revoke/delete/history/scopes` | Audit and revoke scoped tokens. Minting opens the dashboard so the secret never enters the CLI process. |
 | `fruxon llm-providers list/get/models` | Browse LLM providers and models supported at the tenant level. |
 | `fruxon assets create/list/get/wait/operations/delete` | Manage knowledge-base (RAG) assets a step can query — upload local files, wait for async ingestion, inspect operations, and use the ids for `assetConfig.assetIds`. `delete` confirms + refuses without `--yes` in agent mode. |
+| `fruxon knowledge-bases consolidate/consolidation` | Find the near-duplicate articles a per-conversation merge left in an editable knowledge base and, with `--apply`, have the base's consolidator agent fold each cluster into one (union of tags, labels and sources; the rest archived with a `merged_into` label). Report-only by default; waits for the report unless `--no-wait`. |
 | `fruxon assets search/documents/chunks` | Reproduce and inspect a retrieval. `search` runs the same query an agent's `search_assets` does (`--mode HYBRID/VECTOR/FULL_TEXT`, `--top-k`, fusion weights) — note `score` is on a different scale per mode, and in hybrid the top hit is always 1.0. `documents` lists what the index contains (a `chunkCount` of 0 means that file yielded no indexable text); `chunks` reads one document's chunks in order. The `documentId` these print is the **index's** id, which is what `chunks` takes — on a knowledge base's backing asset it is not the editable document's id, and it changes whenever a document's body is re-indexed. The `fileName` (`<knowledge document id>.md`) is the document a `knowledge_base` tool would edit. |
 | `fruxon metrics list` | Browse the evaluation-metric catalog — the ids a step's LLM-judge config (`judge.metrics[]`) binds, with a default weight each. |
 | `fruxon triggers list/get/create/update/delete/fire/bind/unbind` | Manage triggers — the schedule/event sources that fire agents. `create`/`update` take `--file` (+`--schema`); `create` also takes `--application`, the Application that will own the trigger and must own everything it fires. `bind`/`unbind` wire which agents fire; `fire` runs it now. `fire`/`delete`/`unbind` confirm + refuse without `--yes` in agent mode. The control plane for autonomy. `get` returns the full stored `workShape` and every `scheduleTimes` slot, so `get -o json | jq .workShape` produces a body `update --file` can post straight back — a PATCH replaces the whole shape, so edit what you read. |

{fruxon-0.13.2 → fruxon-0.13.3}/README.md

@@ -105,6 +105,7 @@ Integrations + tools:
 | `fruxon keys list/mint/revoke/delete/history/scopes` | Audit and revoke scoped tokens. Minting opens the dashboard so the secret never enters the CLI process. |
 | `fruxon llm-providers list/get/models` | Browse LLM providers and models supported at the tenant level. |
 | `fruxon assets create/list/get/wait/operations/delete` | Manage knowledge-base (RAG) assets a step can query — upload local files, wait for async ingestion, inspect operations, and use the ids for `assetConfig.assetIds`. `delete` confirms + refuses without `--yes` in agent mode. |
+| `fruxon knowledge-bases consolidate/consolidation` | Find the near-duplicate articles a per-conversation merge left in an editable knowledge base and, with `--apply`, have the base's consolidator agent fold each cluster into one (union of tags, labels and sources; the rest archived with a `merged_into` label). Report-only by default; waits for the report unless `--no-wait`. |
 | `fruxon assets search/documents/chunks` | Reproduce and inspect a retrieval. `search` runs the same query an agent's `search_assets` does (`--mode HYBRID/VECTOR/FULL_TEXT`, `--top-k`, fusion weights) — note `score` is on a different scale per mode, and in hybrid the top hit is always 1.0. `documents` lists what the index contains (a `chunkCount` of 0 means that file yielded no indexable text); `chunks` reads one document's chunks in order. The `documentId` these print is the **index's** id, which is what `chunks` takes — on a knowledge base's backing asset it is not the editable document's id, and it changes whenever a document's body is re-indexed. The `fileName` (`<knowledge document id>.md`) is the document a `knowledge_base` tool would edit. |
 | `fruxon metrics list` | Browse the evaluation-metric catalog — the ids a step's LLM-judge config (`judge.metrics[]`) binds, with a default weight each. |
 | `fruxon triggers list/get/create/update/delete/fire/bind/unbind` | Manage triggers — the schedule/event sources that fire agents. `create`/`update` take `--file` (+`--schema`); `create` also takes `--application`, the Application that will own the trigger and must own everything it fires. `bind`/`unbind` wire which agents fire; `fire` runs it now. `fire`/`delete`/`unbind` confirm + refuse without `--yes` in agent mode. The control plane for autonomy. `get` returns the full stored `workShape` and every `scheduleTimes` slot, so `get -o json | jq .workShape` produces a body `update --file` can post straight back — a PATCH replaces the whole shape, so edit what you read. |

{fruxon-0.13.2 → fruxon-0.13.3}/src/fruxon/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.13.2'
-__version_tuple__ = version_tuple = (0, 13, 2)
+__version__ = version = '0.13.3'
+__version_tuple__ = version_tuple = (0, 13, 3)
 
 __commit_id__ = commit_id = None

{fruxon-0.13.2 → fruxon-0.13.3}/src/fruxon/cli/__init__.py

@@ -599,6 +599,7 @@ from fruxon.cli import examples as _examples  # noqa: E402, F401
 from fruxon.cli import guides as _guides  # noqa: E402, F401
 from fruxon.cli import integrations as _integrations  # noqa: E402, F401
 from fruxon.cli import keys as _keys  # noqa: E402, F401
+from fruxon.cli import knowledge_bases as _knowledge_bases  # noqa: E402, F401
 from fruxon.cli import llm_providers as _llm_providers  # noqa: E402, F401
 from fruxon.cli import messages as _messages  # noqa: E402, F401
 from fruxon.cli import metrics as _metrics  # noqa: E402, F401

{fruxon-0.13.2 → fruxon-0.13.3}/src/fruxon/cli/agents_executions.py

@@ -81,6 +81,7 @@ _VALID_TRIGGER_TYPE = frozenset(
         "PARTICIPANT_REQUEST",
         "RELEVANCE_CHECK",
         "DOCUMENT_ENRICHMENT",
+        "KNOWLEDGE_CONSOLIDATION",
         "COLLECTION_ITEM",
         "TEST",
         "TOPIC_TURN",

fruxon-0.13.3/src/fruxon/cli/knowledge_bases.py

@@ -0,0 +1,266 @@
+"""``fruxon knowledge-bases`` — passes over an editable knowledge base.
+
+A *knowledge base* is the authored, reviewable corpus behind a
+``manual_knowledge`` asset; its documents reach search only when the base
+is published. This group carries the corpus-level operations that no
+single document verb can express. Today that is ``consolidate``: find the
+near-duplicate articles a per-item merge stage left behind and, on
+``--apply``, have the base's consolidator agent fold each cluster into one.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Annotated
+
+import typer
+
+from fruxon.cli import app
+from fruxon.cli._shared import build_client
+from fruxon.exceptions import FruxonError
+from fruxon.ui import (
+    EXIT_VALIDATION,
+    fail,
+    fail_from_api_error,
+    resolve_output_format,
+    say_info,
+    say_ok,
+    say_warn,
+    stderr,
+)
+
+knowledge_bases_app = typer.Typer(
+    help="Passes over an editable knowledge base (consolidate near-duplicate articles).",
+    no_args_is_help=True,
+)
+app.add_typer(knowledge_bases_app, name="knowledge-bases")
+
+_TERMINAL = {"COMPLETED", "FAILED"}
+
+
+@knowledge_bases_app.command("consolidate")
+def knowledge_bases_consolidate(
+    knowledge_base: Annotated[str, typer.Argument(help="Knowledge base id.")],
+    apply: Annotated[
+        bool,
+        typer.Option(
+            "--apply",
+            help=(
+                "Merge the clusters instead of only reporting them. Needs a consolidator agent on the "
+                "knowledge base; without one the pass reports whatever this says."
+            ),
+        ),
+    ] = False,
+    threshold: Annotated[
+        float | None,
+        typer.Option(
+            "--threshold", help="Cosine similarity at or above which two documents cluster (server default 0.86)."
+        ),
+    ] = None,
+    max_cluster_size: Annotated[
+        int | None,
+        typer.Option(
+            "--max-cluster-size", help="Largest cluster handed to the agent in one decision (server default 5)."
+        ),
+    ] = None,
+    recent_minutes: Annotated[
+        int | None,
+        typer.Option(
+            "--recent-minutes", help="Leave out documents modified within this many minutes (server default 10)."
+        ),
+    ] = None,
+    wait: Annotated[
+        bool,
+        typer.Option("--wait/--no-wait", help="Block until the run completes and print its report (default: wait)."),
+    ] = True,
+    timeout: Annotated[float, typer.Option("--timeout", help="Seconds before giving up on --wait.")] = 600.0,
+    poll_interval: Annotated[float, typer.Option("--poll-interval", help="Seconds between polls.")] = 3.0,
+    workspace: Annotated[str | None, typer.Option("--workspace", help="Workspace identifier override.")] = None,
+    base_url: Annotated[str | None, typer.Option("--base-url", help="API base URL override.")] = None,
+    output: Annotated[
+        str | None,
+        typer.Option("--output", "-o", help="Output format: text (default for humans), json (default in agent mode)."),
+    ] = None,
+):
+    """Queue a consolidation pass and, by default, wait for its report.
+
+    Report-only unless ``--apply`` is given: the pass embeds the base's
+    Draft and Published documents, clusters the ones above the threshold,
+    and lists each cluster. With ``--apply`` and a consolidator agent set
+    on the knowledge base, the agent decides each cluster and a merge
+    writes one article (union of tags, labels and source conversations)
+    and archives the rest. The pass runs on the worker; ``--no-wait``
+    prints the queued run and returns.
+
+    Examples:
+        fruxon knowledge-bases consolidate <kb>
+        fruxon knowledge-bases consolidate <kb> --apply
+        fruxon knowledge-bases consolidate <kb> --threshold 0.9 --no-wait -o json
+    """
+    output = resolve_output_format(output, human_default="text", agent_default="json")
+    if output not in {"text", "json"}:
+        fail(f"Unknown output format: [bold]{output}[/bold]", hint="Valid formats: text, json.", code=EXIT_VALIDATION)
+    if threshold is not None and not 0.5 <= threshold <= 1.0:
+        fail("--threshold must be between 0.5 and 1.0.", code=EXIT_VALIDATION)
+    if max_cluster_size is not None and max_cluster_size < 2:
+        fail("--max-cluster-size must be at least 2.", code=EXIT_VALIDATION)
+    if recent_minutes is not None and recent_minutes < 0:
+        fail("--recent-minutes cannot be negative.", code=EXIT_VALIDATION)
+    if timeout <= 0 or poll_interval <= 0:
+        fail("Timeout and poll interval must be positive.", code=EXIT_VALIDATION)
+
+    client = build_client(workspace, base_url)
+    try:
+        run = client.consolidate_knowledge_base(
+            knowledge_base,
+            report_only=not apply,
+            similarity_threshold=threshold,
+            max_cluster_size=max_cluster_size,
+            recent_write_window_minutes=recent_minutes,
+        )
+    except FruxonError as e:
+        fail_from_api_error(e, hint=_hint_for_error(e))
+
+    run_id = str(run.get("id", ""))
+    if not wait:
+        if output == "json":
+            print(json.dumps(run, indent=2))
+            return
+        say_ok(
+            f"Queued consolidation run [bold]{run_id}[/bold] on knowledge base [bold]{knowledge_base}[/bold].",
+            hint=f"Read it with: fruxon knowledge-bases consolidation {knowledge_base} {run_id}",
+        )
+        return
+
+    try:
+        if stderr.is_terminal and output == "text":
+            with stderr.status(f"[bold]Consolidating [cyan]{knowledge_base}[/cyan]…[/bold]"):
+                run = client.wait_for_knowledge_base_consolidation(
+                    knowledge_base, run_id, timeout_s=timeout, poll_interval_s=poll_interval
+                )
+        else:
+            run = client.wait_for_knowledge_base_consolidation(
+                knowledge_base, run_id, timeout_s=timeout, poll_interval_s=poll_interval
+            )
+    except TimeoutError as e:
+        fail(str(e), hint=f"Read it later with: fruxon knowledge-bases consolidation {knowledge_base} {run_id}")
+    except FruxonError as e:
+        fail_from_api_error(e, hint=_hint_for_error(e))
+
+    _emit_run(run, output)
+
+
+@knowledge_bases_app.command("consolidation")
+def knowledge_bases_consolidation(
+    knowledge_base: Annotated[str, typer.Argument(help="Knowledge base id.")],
+    run: Annotated[str, typer.Argument(help="Consolidation run id, as printed by `consolidate`.")],
+    workspace: Annotated[str | None, typer.Option("--workspace", help="Workspace identifier override.")] = None,
+    base_url: Annotated[str | None, typer.Option("--base-url", help="API base URL override.")] = None,
+    output: Annotated[
+        str | None,
+        typer.Option("--output", "-o", help="Output format: text (default for humans), json (default in agent mode)."),
+    ] = None,
+):
+    """Read one consolidation run — its status, and the report once it completed."""
+    output = resolve_output_format(output, human_default="text", agent_default="json")
+    if output not in {"text", "json"}:
+        fail(f"Unknown output format: [bold]{output}[/bold]", hint="Valid formats: text, json.", code=EXIT_VALIDATION)
+
+    client = build_client(workspace, base_url)
+    try:
+        current = client.get_knowledge_base_consolidation(knowledge_base, run)
+    except FruxonError as e:
+        fail_from_api_error(e, hint=_hint_for_error(e))
+
+    _emit_run(current, output)
+
+
+def _emit_run(run: dict, output: str) -> None:
+    if output == "json":
+        print(json.dumps(run, indent=2))
+        return
+
+    status = str(run.get("status", ""))
+    run_id = run.get("id", "")
+    if status == "FAILED":
+        fail(f"Consolidation run [bold]{run_id}[/bold] failed: {run.get('error') or 'no error recorded'}")
+    if status not in _TERMINAL:
+        say_info(f"Consolidation run [bold]{run_id}[/bold] is {status.lower() or 'pending'}; no report yet.")
+        return
+
+    report = run.get("report") or {}
+    _print_report(report, run)
+
+
+def _print_report(report: dict, run: dict) -> None:
+    from rich.markup import escape
+    from rich.table import Table
+
+    mode = "report only" if report.get("reportOnly", True) else "applied"
+    agent = report.get("consolidatorAgentId")
+    say_ok(
+        f"Consolidation run [bold]{run.get('id', '')}[/bold] completed ({mode}"
+        + (f", agent [bold]{escape(str(agent))}[/bold]" if agent else ", no consolidator agent")
+        + ")."
+    )
+    stderr.print(
+        f"  documents {report.get('documentsConsidered', 0)}"
+        f" · skipped as recently modified {report.get('documentsSkippedAsRecentlyModified', 0)}"
+        f" · embeddings computed {report.get('embeddingsComputed', 0)}"
+        f" / reused {report.get('embeddingsReused', 0)}"
+        + (f" · model {escape(str(report['embeddingModel']))}" if report.get("embeddingModel") else "")
+    )
+    stderr.print(
+        f"  clusters {report.get('clustersFound', 0)}"
+        f" · merged {report.get('clustersMerged', 0)}"
+        f" · kept separate {report.get('clustersKeptSeparate', 0)}"
+        f" · failed {report.get('clustersFailed', 0)}"
+        f" · documents archived {report.get('documentsArchived', 0)}"
+        f" · threshold {report.get('similarityThreshold', '')}"
+    )
+
+    clusters = report.get("clusters") or []
+    if not clusters:
+        say_info("No clusters above the threshold — nothing to consolidate.")
+        return
+
+    for cluster in clusters:
+        outcome = str(cluster.get("outcome", ""))
+        header = (
+            f"[bold]#{cluster.get('index', '')}[/bold] {outcome.lower().replace('_', ' ')}"
+            f" · similarity {cluster.get('minSimilarity', '')}–{cluster.get('maxSimilarity', '')}"
+        )
+        if cluster.get("survivorId"):
+            header += f" · survivor {cluster['survivorId']}"
+        stderr.print(header)
+        table = Table(show_header=True, header_style="bold", show_edge=False, pad_edge=False, box=None, padding=(0, 1))
+        table.add_column("document")
+        table.add_column("status")
+        table.add_column("sources", justify="right")
+        table.add_column("title")
+        archived = set(cluster.get("archivedIds") or [])
+        for member in cluster.get("members") or []:
+            member_id = str(member.get("id", ""))
+            status = str(member.get("status", ""))
+            if member_id in archived:
+                status = "ARCHIVED (merged)"
+            table.add_row(
+                member_id, status, str(member.get("sourceConversationCount", 0)), escape(str(member.get("title", "")))
+            )
+        stderr.print(table)
+        if cluster.get("note"):
+            stderr.print(f"  note: {escape(str(cluster['note']))}")
+    if report.get("clustersFailed", 0):
+        say_warn("Some clusters failed; their notes say why. Nothing was written for those.")
+
+
+def _hint_for_error(e: FruxonError) -> str | None:
+    status = getattr(e, "status", None)
+    if status == 404:
+        return (
+            "Check the knowledge base id: fruxon assets list shows its backing asset, "
+            "the id here is the knowledge base's own."
+        )
+    if status == 409:
+        return "A pass is already queued or running on this knowledge base, or the base is archived."
+    return None

{fruxon-0.13.2 → fruxon-0.13.3}/src/fruxon/fruxon.py

@@ -2437,6 +2437,76 @@ class FruxonClient:
         raw = self._get_json(self._assets_url(f"/{asset}/operations/{operation}"))
         return raw if isinstance(raw, dict) else {}
 
+    # ── Knowledge bases ──────────────────────────────────────────────────
+
+    def _knowledge_bases_url(self, action: str = "") -> str:
+        return f"{self._base_url}/v1/tenants/{self._workspace}/knowledgeBases{action}"
+
+    def consolidate_knowledge_base(
+        self,
+        knowledge_base: str,
+        *,
+        report_only: bool = True,
+        similarity_threshold: float | None = None,
+        max_cluster_size: int | None = None,
+        recent_write_window_minutes: int | None = None,
+    ) -> dict:
+        """Queue a consolidation pass over a knowledge base (``POST …:consolidate``).
+
+        The pass runs on the maintenance worker: the reply is the queued run,
+        not the report. Poll :meth:`get_knowledge_base_consolidation` or use
+        :meth:`wait_for_knowledge_base_consolidation`. ``report_only`` defaults
+        to True, matching the server — a pass merges nothing unless told to.
+        Omitted options fall back to the server's defaults, which the run
+        records.
+        """
+        body: dict[str, object] = {"reportOnly": report_only}
+        if similarity_threshold is not None:
+            body["similarityThreshold"] = similarity_threshold
+        if max_cluster_size is not None:
+            body["maxClusterSize"] = max_cluster_size
+        if recent_write_window_minutes is not None:
+            body["recentWriteWindowMinutes"] = recent_write_window_minutes
+        return self._post_json(self._knowledge_bases_url(f"/{knowledge_base}:consolidate"), body)
+
+    def get_knowledge_base_consolidation(self, knowledge_base: str, run: str) -> dict:
+        """Fetch one consolidation run; ``report`` is present once it completed."""
+        raw = self._get_json(self._knowledge_bases_url(f"/{knowledge_base}/consolidations/{run}"))
+        return raw if isinstance(raw, dict) else {}
+
+    def list_knowledge_base_consolidations(self, knowledge_base: str, *, all_pages: bool = True) -> list[dict]:
+        """List consolidation runs over a knowledge base, newest first."""
+        base_url = self._knowledge_bases_url(f"/{knowledge_base}/consolidations")
+        items = self._fetch_all_pages(base_url, []) if all_pages else self._fetch_page(base_url, [], page_token=None)[0]
+        return [item for item in items if isinstance(item, dict)]
+
+    def wait_for_knowledge_base_consolidation(
+        self,
+        knowledge_base: str,
+        run: str,
+        *,
+        timeout_s: float = 600.0,
+        poll_interval_s: float = 3.0,
+    ) -> dict:
+        """Poll a consolidation run until it is COMPLETED or FAILED.
+
+        Returns the terminal run (its ``report`` when completed, its ``error``
+        when failed). Raises :class:`TimeoutError` past ``timeout_s``.
+        """
+        deadline = time.monotonic() + timeout_s
+        last_status = None
+        while True:
+            current = self.get_knowledge_base_consolidation(knowledge_base, run)
+            last_status = current.get("status")
+            if last_status in {"COMPLETED", "FAILED"}:
+                return current
+            if time.monotonic() >= deadline:
+                raise TimeoutError(
+                    f"Timed out waiting for consolidation run {run} on knowledge base {knowledge_base}."
+                    f" Last status: {last_status}."
+                )
+            time.sleep(poll_interval_s)
+
     def list_asset_embedding_models(self) -> list[dict]:
         """List embedding models accepted by asset ingestion."""
         raw = self._get_json(self._assets_url("/embeddingModels"))