ellf-cli 18.1.1__tar.gz → 19.0.3__tar.gz

{ellf_cli-18.1.1/ellf_cli.egg-info → ellf_cli-19.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ellf-cli
-Version: 18.1.1
+Version: 19.0.3
 Summary: Ellf Command Line Interface
 Home-page: https://prodi.gy
 Author: ExplosionAI GmbH

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/README.md

@@ -603,30 +603,6 @@ Export all the examples from a dataset and save it in the designated file as JSO
 | `name_or_id` | `Union[str, UUID]` | Name or ID of the dataset |  |
 | `--output`, `-o` | `str` | JSON output path for data | `'-'` |
 
-### `ellf examples`
-
-Disable and activate annotated examples
-
-#### `ellf examples disable`
-
-Disable examples: supersede each with a copy answered "ignore". They stay in their dataset and remain their annotators' work
-
-| Argument | Type | Description | Default |
-| --- | --- | --- | --- |
-| `dataset` | `str` | Name of the dataset the examples belong to |  |
-| `example_ids` | `list[int]` | Comma-separated IDs of the examples |  |
-| `--json` | `bool` | Output the result as JSON | `False` |
-
-#### `ellf examples activate`
-
-Activate disabled examples: restore the answers they replaced
-
-| Argument | Type | Description | Default |
-| --- | --- | --- | --- |
-| `dataset` | `str` | Name of the dataset the examples belong to |  |
-| `example_ids` | `list[int]` | Comma-separated IDs of the examples |  |
-| `--json` | `bool` | Output the result as JSON | `False` |
-
 ### `ellf files`
 
 Manage files on the cluster. Your files are only ever sent to servers or buckets you control. They are never sent to our servers.
@@ -1331,23 +1307,23 @@ Mint a fresh 90-day token for an existing service. Issuing a new token does not
 
 ### `ellf sessions`
 
-View, disable and activate a task's annotation sessions
+View, exclude and activate a task's annotation sessions
 
 #### `ellf sessions list`
 
-List a task's annotation sessions, with how much each annotated and whether it was disabled
+List a task's annotation sessions, with how much each annotated, how much of that the dataset still holds, and whether it was excluded
 
 | Argument | Type | Description | Default |
 | --- | --- | --- | --- |
 | `name_or_id` | `Union[str, UUID]` | Name or ID of the task |  |
 | `--project` | `str` | Name, slug, or ID of task's project or the last project if not set | `None` |
 | `--cluster` | `str` | Name, slug, or ID of the cluster to search for task name (or the last cluster if not set) | `None` |
-| `--select` | `list[str]` | Comma-separated fields to select and show in output. Available: ['id', 'annotator', 'started', 'annotations', 'status'] | `['id', 'annotator', 'started', 'annotations', 'status']` |
+| `--select` | `list[str]` | Comma-separated fields to select and show in output. Available: ['id', 'annotator', 'started', 'annotated', 'active', 'status'] | `['id', 'annotator', 'started', 'annotated', 'active', 'status']` |
 | `--json` | `bool` | Output the result as JSON | `False` |
 
-#### `ellf sessions disable`
+#### `ellf sessions exclude`
 
-Disable an annotation session: take its examples out of the task's dataset, so training and exports leave them out and their tasks are served again. They still count as the annotator's work
+Exclude an annotation session: take its examples out of the task's dataset, so training and exports leave them out and their tasks are served again. They still count as the annotator's work
 
 | Argument | Type | Description | Default |
 | --- | --- | --- | --- |
@@ -1356,7 +1332,7 @@ Disable an annotation session: take its examples out of the task's dataset, so t
 
 #### `ellf sessions activate`
 
-Activate a disabled annotation session: put its examples back into the task's dataset
+Activate an excluded annotation session: put its examples back into the task's dataset
 
 | Argument | Type | Description | Default |
 | --- | --- | --- | --- |
@@ -1407,7 +1383,7 @@ List the tasks in a project. Uses your default project unless --project is given
 | Argument | Type | Description | Default |
 | --- | --- | --- | --- |
 | `--project` | `str` | Name, slug, or ID of task's project or the last project if not set | `None` |
-| `--select` | `list[str]` | Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'created_by_user', 'state'] | `['id', 'name', 'state']` |
+| `--select` | `list[str]` | Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'last_execution_started', 'created_by_user', 'state'] | `['id', 'name', 'state']` |
 | `--json` | `bool` | Output the result as JSON | `False` |
 
 #### `ellf tasks info`
@@ -1419,7 +1395,7 @@ Print information about a task on the cluster
 | `name_or_id` | `Union[str, UUID]` | Name or ID of the task |  |
 | `--project` | `str` | Name, slug, or ID of task's project or the last project if not set | `None` |
 | `--cluster` | `str` | Name, slug, or ID of the cluster to search for task name (or the last cluster if not set) | `None` |
-| `--select` | `list[str]` | Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'created_by_user', 'last_execution_id', 'related_actions', 'url_logs', 'url'] | `None` |
+| `--select` | `list[str]` | Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'last_execution_started', 'created_by_user', 'last_execution_id', 'related_actions', 'url_logs', 'url'] | `None` |
 | `--json` | `bool` | Output the result as JSON | `False` |
 
 #### `ellf tasks logs`

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/cli.py

@@ -110,7 +110,6 @@ cli.placeholder("auth", description=Messages.doc_auth)
 cli.placeholder("clusters", description=Messages.doc_clusters)
 cli.placeholder("config", description=Messages.doc_config)
 cli.placeholder("datasets", description=Messages.doc_datasets)
-cli.placeholder("examples", description=Messages.doc_examples)
 cli.placeholder("files", description=Messages.doc_files)
 cli.placeholder("gcp", description=Messages.doc_gcp)
 cli.placeholder("infra", description=Messages.doc_infra)

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/commands/datasets.py

@@ -164,6 +164,9 @@ def export(
     cluster_id = auth.cluster_id
 
     dataset = resolve_dataset(name_or_id, cluster_id=cluster_id)
+    legacy = auth.broker_client.datasets.total_example_count(dataset.name).legacy
+    if legacy:
+        raise CLIError(Messages.E066.format(name=dataset.name, count=legacy))
 
     def get_examples_page(page: int):
         return auth.broker_client.datasets.read_examples(

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/commands/sessions.py

@@ -1,10 +1,10 @@
-"""Annotation sessions and the examples in them.
+"""Annotation sessions.
 
 A task's annotations live in the cluster's Prodigy database, where every
 PAM session (one sitting of one annotator) has a dataset of its own named
-``{dataset}-{task_id}-{annotator_id}-{session_id}``. Disabling a session
+``{dataset}-{task_id}-{annotator_id}-{session_id}``. Excluding a session
 takes its examples out of the task's dataset while keeping them in that
-session dataset; disabling an example supersedes it with an ignored answer.
+session dataset.
 """
 
 import builtins
@@ -12,7 +12,7 @@ from collections.abc import Sequence
 from typing import Any, Optional, Union
 from uuid import UUID
 
-from radicli import Arg, get_list_converter
+from radicli import Arg
 
 from ..cli import cli
 from ..messages import Messages
@@ -20,7 +20,7 @@ from ..query import resolve_task
 from ..ui import print_mutation_result, print_table_with_select
 from ._state import get_auth_state
 
-SESSION_FIELDS = ["id", "annotator", "started", "annotations", "status"]
+SESSION_FIELDS = ["id", "annotator", "started", "annotated", "active", "status"]
 
 
 def _sitting_name(
@@ -45,31 +45,34 @@ def list(
     select: builtins.list[str] = SESSION_FIELDS,
     as_json: bool = False,
 ) -> Sequence[dict[str, Any]]:
-    """List a task's annotation sessions, with how much each annotated and
-    whether it was disabled"""
+    """List a task's annotation sessions, with how much each annotated, how
+    much of that the dataset still holds, and whether it was excluded"""
     auth = get_auth_state()
     task = resolve_task(name_or_id, cluster_id=cluster_id, project_id=project_id)
     dataset = auth.client.task.dataset(task.id)
     summary = auth.broker_client.datasets.session_counts(dataset.name)
     removed = set(summary.removed or [])
     rows = []
+    counts = summary.counts or {}
     for session in auth.client.session.all(task_id=task.id):
         name = _sitting_name(dataset.name, task.id, session.annotator_id, session.id)
+        count = counts.get(name)
         rows.append(
             {
                 "id": str(session.id),
                 "annotator": session.annotator_name,
                 "started": session.created.isoformat(timespec="seconds"),
-                "annotations": (summary.counts or {}).get(name, 0),
-                "status": "disabled" if name in removed else "active",
+                "annotated": count.annotated if count else 0,
+                "active": count.active if count else 0,
+                "status": "excluded" if name in removed else "active",
             }
         )
     print_table_with_select(rows, select=select, as_json=as_json)
     return rows
 
 
-def _set_session_disabled(
-    session_id: UUID, disabled: bool, as_json: bool
+def _set_session_excluded(
+    session_id: UUID, excluded: bool, as_json: bool
 ) -> dict[str, Any]:
     auth = get_auth_state()
     session = auth.client.session.read(id=session_id)
@@ -80,11 +83,11 @@ def _set_session_disabled(
     datasets = auth.broker_client.datasets
     change = (
         datasets.remove_session(dataset.name, name)
-        if disabled
+        if excluded
         else datasets.restore_session(dataset.name, name)
     )
     result = {"session": str(session_id), "examples": change.examples}
-    message = Messages.T072 if disabled else Messages.T073
+    message = Messages.T072 if excluded else Messages.T073
     print_mutation_result(
         result, message.format(id=session_id, count=change.examples), as_json=as_json
     )
@@ -93,15 +96,15 @@ def _set_session_disabled(
 
 @cli.subcommand(
     "sessions",
-    "disable",
+    "exclude",
     session_id=Arg(help=Messages.session_id),
     as_json=Arg("--json", help=Messages.as_json),
 )
-def disable(session_id: UUID, as_json: bool = False) -> dict[str, Any]:
-    """Disable an annotation session: take its examples out of the task's
+def exclude(session_id: UUID, as_json: bool = False) -> dict[str, Any]:
+    """Exclude an annotation session: take its examples out of the task's
     dataset, so training and exports leave them out and their tasks are served
     again. They still count as the annotator's work"""
-    return _set_session_disabled(session_id, True, as_json)
+    return _set_session_excluded(session_id, True, as_json)
 
 
 @cli.subcommand(
@@ -111,58 +114,6 @@ def disable(session_id: UUID, as_json: bool = False) -> dict[str, Any]:
     as_json=Arg("--json", help=Messages.as_json),
 )
 def activate(session_id: UUID, as_json: bool = False) -> dict[str, Any]:
-    """Activate a disabled annotation session: put its examples back into the
-    task's dataset"""
-    return _set_session_disabled(session_id, False, as_json)
-
-
-def _set_examples_disabled(
-    dataset: str, example_ids: builtins.list[int], disabled: bool, as_json: bool
-) -> dict[str, Any]:
-    examples = get_auth_state().broker_client.examples
-    replaced = (
-        examples.disable(dataset, example_ids)
-        if disabled
-        else examples.activate(dataset, example_ids)
-    )
-    result = {
-        "examples": [
-            {"old_example_id": r.old_example_id, "id": r.id} for r in replaced.examples
-        ]
-    }
-    message = Messages.T074 if disabled else Messages.T075
-    print_mutation_result(
-        result,
-        message.format(count=len(replaced.examples), name=dataset),
-        as_json=as_json,
-    )
-    return result
-
-
-@cli.subcommand(
-    "examples",
-    "disable",
-    dataset=Arg(help=Messages.example_dataset),
-    example_ids=Arg(help=Messages.example_ids, converter=get_list_converter(int)),
-    as_json=Arg("--json", help=Messages.as_json),
-)
-def disable_examples(
-    dataset: str, example_ids: builtins.list[int], as_json: bool = False
-) -> dict[str, Any]:
-    """Disable examples: supersede each with a copy answered "ignore". They stay
-    in their dataset and remain their annotators' work"""
-    return _set_examples_disabled(dataset, example_ids, True, as_json)
-
-
-@cli.subcommand(
-    "examples",
-    "activate",
-    dataset=Arg(help=Messages.example_dataset),
-    example_ids=Arg(help=Messages.example_ids, converter=get_list_converter(int)),
-    as_json=Arg("--json", help=Messages.as_json),
-)
-def activate_examples(
-    dataset: str, example_ids: builtins.list[int], as_json: bool = False
-) -> dict[str, Any]:
-    """Activate disabled examples: restore the answers they replaced"""
-    return _set_examples_disabled(dataset, example_ids, False, as_json)
+    """Activate an excluded annotation session: put its examples back into
+    the task's dataset"""
+    return _set_session_excluded(session_id, False, as_json)

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf.json

@@ -1,7 +1,7 @@
 {
   "prog": "ellf",
   "help": "Ellf Command Line Interface.",
-  "version": "18.1.1",
+  "version": "19.0.3",
   "extra_key": "_extra",
   "commands": {
     "actions": {
@@ -68,14 +68,6 @@
       "parent": null,
       "is_placeholder": true
     },
-    "examples": {
-      "name": "examples",
-      "args": [],
-      "description": "Disable and activate annotated examples",
-      "allow_extra": false,
-      "parent": null,
-      "is_placeholder": true
-    },
     "files": {
       "name": "files",
       "args": [],
@@ -167,7 +159,7 @@
     "sessions": {
       "name": "sessions",
       "args": [],
-      "description": "View, disable and activate a task's annotation sessions",
+      "description": "View, exclude and activate a task's annotation sessions",
       "allow_extra": false,
       "parent": null,
       "is_placeholder": true
@@ -7467,9 +7459,9 @@
             "id": "select",
             "option": "--select",
             "short": null,
-            "orig_help": "Comma-separated fields to select and show in output. Available: ['id', 'annotator', 'started', 'annotations', 'status']",
-            "default": "['id', 'annotator', 'started', 'annotations', 'status']",
-            "help": "Comma-separated fields to select and show in output. Available: ['id', 'annotator', 'started', 'annotations', 'status'] (list[str])",
+            "orig_help": "Comma-separated fields to select and show in output. Available: ['id', 'annotator', 'started', 'annotated', 'active', 'status']",
+            "default": "['id', 'annotator', 'started', 'annotated', 'active', 'status']",
+            "help": "Comma-separated fields to select and show in output. Available: ['id', 'annotator', 'started', 'annotated', 'active', 'status'] (list[str])",
             "action": null,
             "choices": null,
             "has_converter": true,
@@ -7490,13 +7482,13 @@
             "orig_type": "bool"
           }
         ],
-        "description": "List a task's annotation sessions, with how much each annotated and\nwhether it was disabled",
+        "description": "List a task's annotation sessions, with how much each annotated, how\nmuch of that the dataset still holds, and whether it was excluded",
         "allow_extra": false,
         "parent": "sessions",
         "is_placeholder": false
       },
-      "disable": {
-        "name": "disable",
+      "exclude": {
+        "name": "exclude",
         "args": [
           {
             "id": "session_id",
@@ -7525,7 +7517,7 @@
             "orig_type": "bool"
           }
         ],
-        "description": "Disable an annotation session: take its examples out of the task's\ndataset, so training and exports leave them out and their tasks are served\nagain. They still count as the annotator's work",
+        "description": "Exclude an annotation session: take its examples out of the task's\ndataset, so training and exports leave them out and their tasks are served\nagain. They still count as the annotator's work",
         "allow_extra": false,
         "parent": "sessions",
         "is_placeholder": false
@@ -7560,110 +7552,12 @@
             "orig_type": "bool"
           }
         ],
-        "description": "Activate a disabled annotation session: put its examples back into the\ntask's dataset",
+        "description": "Activate an excluded annotation session: put its examples back into\nthe task's dataset",
         "allow_extra": false,
         "parent": "sessions",
         "is_placeholder": false
       }
     },
-    "examples": {
-      "disable": {
-        "name": "disable",
-        "args": [
-          {
-            "id": "dataset",
-            "option": null,
-            "short": null,
-            "orig_help": "Name of the dataset the examples belong to",
-            "default": "==SUPPRESS==",
-            "help": "Name of the dataset the examples belong to (str)",
-            "action": null,
-            "choices": null,
-            "has_converter": false,
-            "type": "str",
-            "orig_type": "str"
-          },
-          {
-            "id": "example_ids",
-            "option": null,
-            "short": null,
-            "orig_help": "Comma-separated IDs of the examples",
-            "default": "==SUPPRESS==",
-            "help": "Comma-separated IDs of the examples (list[int])",
-            "action": null,
-            "choices": null,
-            "has_converter": true,
-            "type": "converter",
-            "orig_type": "list[int]"
-          },
-          {
-            "id": "as_json",
-            "option": "--json",
-            "short": null,
-            "orig_help": "Output the result as JSON",
-            "default": false,
-            "help": "Output the result as JSON (bool)",
-            "action": "store_true",
-            "choices": null,
-            "has_converter": false,
-            "type": null,
-            "orig_type": "bool"
-          }
-        ],
-        "description": "Disable examples: supersede each with a copy answered \"ignore\". They stay\nin their dataset and remain their annotators' work",
-        "allow_extra": false,
-        "parent": "examples",
-        "is_placeholder": false
-      },
-      "activate": {
-        "name": "activate",
-        "args": [
-          {
-            "id": "dataset",
-            "option": null,
-            "short": null,
-            "orig_help": "Name of the dataset the examples belong to",
-            "default": "==SUPPRESS==",
-            "help": "Name of the dataset the examples belong to (str)",
-            "action": null,
-            "choices": null,
-            "has_converter": false,
-            "type": "str",
-            "orig_type": "str"
-          },
-          {
-            "id": "example_ids",
-            "option": null,
-            "short": null,
-            "orig_help": "Comma-separated IDs of the examples",
-            "default": "==SUPPRESS==",
-            "help": "Comma-separated IDs of the examples (list[int])",
-            "action": null,
-            "choices": null,
-            "has_converter": true,
-            "type": "converter",
-            "orig_type": "list[int]"
-          },
-          {
-            "id": "as_json",
-            "option": "--json",
-            "short": null,
-            "orig_help": "Output the result as JSON",
-            "default": false,
-            "help": "Output the result as JSON (bool)",
-            "action": "store_true",
-            "choices": null,
-            "has_converter": false,
-            "type": null,
-            "orig_type": "bool"
-          }
-        ],
-        "description": "Activate disabled examples: restore the answers they replaced",
-        "allow_extra": false,
-        "parent": "examples",
-        "is_placeholder": false
-      }
-    },
     "support": {
       "create": {
         "name": "create",
@@ -7903,9 +7797,9 @@
             "id": "select",
             "option": "--select",
             "short": null,
-            "orig_help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'created_by_user', 'state']",
+            "orig_help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'last_execution_started', 'created_by_user', 'state']",
             "default": "['id', 'name', 'state']",
-            "help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'created_by_user', 'state'] (list[str])",
+            "help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'last_execution_started', 'created_by_user', 'state'] (list[str])",
             "action": null,
             "choices": null,
             "has_converter": true,
@@ -7977,9 +7871,9 @@
             "id": "select",
             "option": "--select",
             "short": null,
-            "orig_help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'created_by_user', 'last_execution_id', 'related_actions', 'url_logs', 'url']",
+            "orig_help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'last_execution_started', 'created_by_user', 'last_execution_id', 'related_actions', 'url_logs', 'url']",
             "default": null,
-            "help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'created_by_user', 'last_execution_id', 'related_actions', 'url_logs', 'url'] (list[str])",
+            "help": "Comma-separated fields to select and show in output. Available: ['id', 'created', 'updated', 'project_id', 'cluster_id', 'name', 'recipe_name', 'recipe_title', 'project_name', 'error', 'plan', 'job_type', 'cli_command', 'last_execution_started', 'created_by_user', 'last_execution_id', 'related_actions', 'url_logs', 'url'] (list[str])",
             "action": null,
             "choices": null,
             "has_converter": true,

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf_skills/skills/ellf-annotate.assistant/SKILL.md

@@ -168,4 +168,4 @@ When you finish:
 |------|---------------|--------------|
 | `${CLAUDE_SKILL_DIR}/references/annotation_audit.md` | Readiness checklist: objective, schema, recipe fit, data, dataset | Before every launch |
 | `${CLAUDE_SKILL_DIR}/references/builtin_ellf_annotation_recipes.md` | Built-in Ellf task and agent recipes with supported workflows | Recipe selection |
-| `${CLAUDE_SKILL_DIR}/references/label_schemes.md` | Label scheme (`labels` asset) format, design and use by tasks and agents | Labels need definitions, or writing guidelines |
+| `${CLAUDE_SKILL_DIR}/references/label_schemes.md` | Label scheme (`labels` asset) format, design and use by tasks and agents. Built-in `ner`, `spans` and `textcat` only | Labels need definitions, or writing guidelines |

{ellf_cli-18.1.1/ellf_cli/ellf_skills/skills/ellf-annotate.coding → ellf_cli-19.0.3/ellf_cli/ellf_skills/skills/ellf-annotate.assistant}/references/annotation_audit.md

@@ -41,7 +41,9 @@ passes.
 Labels that aren't self-evident need a written definition before launch: what the
 label covers, which look-alikes not to mark, and a few boundary examples. Keep them
 in a label scheme (a `labels` asset, see `label_schemes.md`) used by the task, so
-annotators see the descriptions and LLM agents get them in every prompt. If a label
+annotators see the descriptions and LLM agents get them in every prompt. Only the
+built-in `ner`, `spans` and `textcat` recipes (and custom recipes that implement it)
+take a scheme; for any other recipe, keep the definitions in the annotation guidelines. If a label
 can't be defined without "it depends", the schema isn't ready.
 
 ### Task complexity

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf_skills/skills/ellf-annotate.assistant/references/builtin_ellf_annotation_recipes.md

@@ -44,6 +44,11 @@ workflows.
 
 ## Label schemes
 
+Only `ner`, `spans` and `textcat` take a label scheme. No other built-in recipe accepts
+a `labels` asset, so don't suggest one for them: put the label definitions in the
+annotation instructions instead, or hand a custom recipe that takes a scheme off
+to the coding agent (`/ellf-handoff`).
+
 `ner`, `spans` and `textcat` take their labels as the **Labels** field, switched between **List** (typed labels) and **Asset** (a
 scheme): an asset of kind
 `labels` holding one JSON file with the labels in order and an optional definition

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf_skills/skills/ellf-annotate.assistant/references/label_schemes.md

@@ -8,6 +8,15 @@ Read this when designing labels for a project, when turning an agreed label set
 into something a task can use, or when writing annotation guidelines for labels
 that already exist.
 
+**Only some recipes take a scheme.** A scheme is a recipe argument, and the only
+built-in recipes that have it are `ner`, `spans` and `textcat` (`--label-scheme`, or
+**Asset** on the **Labels** field). A custom recipe can take one too, if it implements
+it with `props.label_scheme` (hand a new one off to the coding agent with `/ellf-handoff`). No other built-in recipe accepts a
+`labels` asset, so never suggest a scheme or `--label-scheme` for one of them. For
+those, write the label definitions into the annotation guidelines (an `instructions`
+asset) instead, or build a custom recipe if the definitions need to reach annotators
+and agents label by label.
+
 ## Format
 
 The file is a JSON object that maps each label to an object. The key order is the
@@ -89,13 +98,14 @@ Show it to the user and settle the labels before publishing.
   previewed from the Assets page and from the asset picker in recipe forms.
 - **Reuse:** check for an existing `labels` asset (`ellf assets list`, or the Assets
   page) before writing a new one. Use one scheme per label set, not one per task.
-- **Tasks:** the built-in `ner`, `spans` and `textcat` recipes take either a typed list
-  of labels (`--label A,B`) or a scheme (`--label-scheme <asset>`). In the web form, the
+- **Tasks:** only the built-in `ner`, `spans` and `textcat` recipes (and custom recipes
+  that implement it) take a scheme. They accept either a typed list of labels
+  (`--label A,B`) or a scheme (`--label-scheme <asset>`). In the web form, the
   **Labels** field switches between **List** and **Asset**. Prefer a scheme whenever
   the labels need explaining, are reused across tasks, or agents will annotate.
 - **Annotators** see a label's description when they hover the label in the annotation
   app (and on choice options for text classification).
-- **Agents:** a task passes its label definitions on to every LLM agent assigned to it
+- **Agents:** a task with a scheme passes its label definitions on to every LLM agent assigned to it
   (`gemini_agent`, `hf_llm_agent`), and they're included in each prompt: description,
   `do_not_mark` and the examples that fit the task type. An agent can have its own
   scheme on top (`--labels.scheme`), whose definitions replace the task's for the

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf_skills/skills/ellf-annotate.coding/SKILL.md

@@ -199,4 +199,4 @@ When you finish:
 | `${CLAUDE_SKILL_DIR}/references/annotation_audit.md` | Readiness checklist: objective, schema, recipe fit, data, dataset | Before every launch |
 | `${CLAUDE_SKILL_DIR}/references/builtin_ellf_annotation_recipes.md` | Built-in Ellf task and agent recipes with supported workflows | Recipe selection for cluster |
 | `${CLAUDE_SKILL_DIR}/references/builtin_prodigy_recipes.md` | Built-in standalone Prodigy recipes | Recipe selection for local |
-| `${CLAUDE_SKILL_DIR}/references/label_schemes.md` | Label scheme (`labels` asset) format, design and use by tasks and agents | Labels need definitions, building them from annotated data, or writing guidelines |
+| `${CLAUDE_SKILL_DIR}/references/label_schemes.md` | Label scheme (`labels` asset) format, design and use by tasks and agents. Built-in `ner`, `spans` and `textcat` only | Labels need definitions, building them from annotated data, or writing guidelines |

{ellf_cli-18.1.1/ellf_cli/ellf_skills/skills/ellf-annotate.assistant → ellf_cli-19.0.3/ellf_cli/ellf_skills/skills/ellf-annotate.coding}/references/annotation_audit.md

@@ -41,7 +41,9 @@ passes.
 Labels that aren't self-evident need a written definition before launch: what the
 label covers, which look-alikes not to mark, and a few boundary examples. Keep them
 in a label scheme (a `labels` asset, see `label_schemes.md`) used by the task, so
-annotators see the descriptions and LLM agents get them in every prompt. If a label
+annotators see the descriptions and LLM agents get them in every prompt. Only the
+built-in `ner`, `spans` and `textcat` recipes (and custom recipes that implement it)
+take a scheme; for any other recipe, keep the definitions in the annotation guidelines. If a label
 can't be defined without "it depends", the schema isn't ready.
 
 ### Task complexity

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf_skills/skills/ellf-annotate.coding/references/builtin_ellf_annotation_recipes.md

@@ -44,6 +44,11 @@ workflows.
 
 ## Label schemes
 
+Only `ner`, `spans` and `textcat` take a label scheme. No other built-in recipe accepts
+a `labels` asset, so don't suggest one for them: put the label definitions in the
+annotation instructions instead, or build a custom recipe that takes a scheme
+(`/ellf-prodigy`).
+
 `ner`, `spans` and `textcat` take their labels as `--label A,B` for a typed list, or `--label-scheme <asset>` for a scheme: an asset of kind
 `labels` holding one JSON file with the labels in order and an optional definition
 for each (description, look-alikes not to mark, examples). Prefer a scheme when the

{ellf_cli-18.1.1/ellf_cli/ellf_skills/skills/ellf-project.coding → ellf_cli-19.0.3/ellf_cli/ellf_skills/skills/ellf-annotate.coding}/references/label_schemes.md

@@ -8,6 +8,15 @@ Read this when designing labels for a project, when turning an agreed label set
 into something a task can use, or when writing annotation guidelines for labels
 that already exist.
 
+**Only some recipes take a scheme.** A scheme is a recipe argument, and the only
+built-in recipes that have it are `ner`, `spans` and `textcat` (`--label-scheme`, or
+**Asset** on the **Labels** field). A custom recipe can take one too, if it implements
+it with `props.label_scheme` (see `/ellf-prodigy`). No other built-in recipe accepts a
+`labels` asset, so never suggest a scheme or `--label-scheme` for one of them. For
+those, write the label definitions into the annotation guidelines (an `instructions`
+asset) instead, or build a custom recipe if the definitions need to reach annotators
+and agents label by label.
+
 ## Format
 
 The file is a JSON object that maps each label to an object. The key order is the
@@ -122,13 +131,14 @@ Show the draft to the user, settle it, then publish it and use it in the tasks.
   previewed from the Assets page and from the asset picker in recipe forms.
 - **Reuse:** check for an existing `labels` asset (`ellf assets list`, or the Assets
   page) before writing a new one. Use one scheme per label set, not one per task.
-- **Tasks:** the built-in `ner`, `spans` and `textcat` recipes take either a typed list
-  of labels (`--label A,B`) or a scheme (`--label-scheme <asset>`). In the web form, the
+- **Tasks:** only the built-in `ner`, `spans` and `textcat` recipes (and custom recipes
+  that implement it) take a scheme. They accept either a typed list of labels
+  (`--label A,B`) or a scheme (`--label-scheme <asset>`). In the web form, the
   **Labels** field switches between **List** and **Asset**. Prefer a scheme whenever
   the labels need explaining, are reused across tasks, or agents will annotate.
 - **Annotators** see a label's description when they hover the label in the annotation
   app (and on choice options for text classification).
-- **Agents:** a task passes its label definitions on to every LLM agent assigned to it
+- **Agents:** a task with a scheme passes its label definitions on to every LLM agent assigned to it
   (`gemini_agent`, `hf_llm_agent`), and they're included in each prompt: description,
   `do_not_mark` and the examples that fit the task type. An agent can have its own
   scheme on top (`--labels.scheme`), whose definitions replace the task's for the

{ellf_cli-18.1.1 → ellf_cli-19.0.3}/ellf_cli/ellf_skills/skills/ellf-ops.coding/references/data_infra_cli.md

@@ -131,7 +131,8 @@ This is the preferred way to upload local files to the cluster — it copies the
 
 Asset kinds that hold a single document: `instructions` (one Markdown file of
 annotation guidelines) and `labels` (one JSON label scheme, the labels of a task with
-their definitions). A label scheme is checked when a task or agent loads it, so
+their definitions; only the built-in `ner`, `spans` and `textcat` recipes, and custom
+recipes that implement it, take one). A label scheme is checked when a task or agent loads it, so
 validate the JSON locally before publishing:
 
 ```bash