codeer-cli 0.1.3__tar.gz → 0.1.5__tar.gz

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/API_REFERENCE.md

@@ -81,23 +81,46 @@ Base path: `/external/context-object-faqs`
 Create body:
 
 ```json
-{"context_object_id": 123, "question": "How do I reset billing?", "ranges": [{"start_line": 12, "end_line": 18}]}
+{
+  "context_object_id": 123,
+  "question": "How do I reset billing?",
+  "ranges": [
+    {
+      "start_line": 12,
+      "start_column": 0,
+      "end_line": 12,
+      "end_column": 42
+    }
+  ]
+}
 ```
 
 Update body accepts any of these fields:
 
 ```json
-{"context_object_id": 456, "question": "How do I update billing?", "ranges": [{"start_line": 20, "end_line": 24}]}
+{
+  "context_object_id": 456,
+  "question": "How do I update billing?",
+  "ranges": [
+    {
+      "start_line": 20,
+      "start_column": 0,
+      "end_line": 20,
+      "end_column": 39
+    }
+  ]
+}
 ```
 
 `context_object_id` is the KB file's `snapshot_object_id` from the KB node
 listing. `ranges` is optional; use it when the FAQ route should reserve chunks
-overlapping a stable line range inside that file. The compact CLI output includes
-the target id:
+overlapping a stable passage inside that file. Include both line and column
+positions so the Codeer UI can map the range onto rendered Markdown. The compact
+CLI output includes the target id:
 
 ```bash
 codeer kb files --kb-id <kb-id>
-codeer kb faq-create --context-object-id <snapshot-object-id> --question "..." --range 12:18 --dry-run
+codeer kb faq-create --context-object-id <snapshot-object-id> --question "..." --range 12:0-12:42 --dry-run
 ```
 
 ## Stage 3 — Live Test on a specific version
@@ -126,10 +149,14 @@ for the apply → test → publish workflow. Pass the draft `AgentHistory.id` fr
 
 | Method & path | Purpose |
 | --- | --- |
-| `POST /eval/cases` | Create case (`input`, `expected_output?`, `rubric?`); rubric = user-docs "Standard" |
+| `GET /eval/workspaces/{workspace_id}/case-labels` | List reusable eval case labels |
+| `POST /eval/workspaces/{workspace_id}/case-labels` | Create reusable eval case label (`name`, `color?`) |
+| `PUT /eval/case-labels/{label_id}` | Update eval case label name/color |
+| `DELETE /eval/case-labels/{label_id}` | Delete eval case label and clear associations |
+| `POST /eval/cases` | Create case (`input`, `expected_output?`, `rubric?`, `label_ids?`); rubric = user-docs "Standard" |
 | `GET /eval/agents/{agent_id}/cases` | List cases for an agent |
 | `GET /eval/cases/{case_id}` | Read one |
-| `PUT /eval/cases/{case_id}` | Update |
+| `PUT /eval/cases/{case_id}` | Update, including replacing labels via `label_ids` |
 | `DELETE /eval/cases/{case_id}` | Delete |
 | `POST /eval/cases/upload-csv` | Bulk import |
 | `POST /eval/cases/bulk` | Bulk delete |
@@ -142,6 +169,22 @@ for the apply → test → publish workflow. Pass the draft `AgentHistory.id` fr
 | `POST /eval/rubric` | Set/override the rubric for one (case, evaluator) — write-only |
 | `POST /eval/rubrics/batch` | **Read** rubrics for a batch of (case, evaluator) pairs |
 
+Eval case labels are workspace-scoped reusable objects. The case create/update
+payload uses `label_ids` (stringified label IDs), not freeform label names:
+
+```json
+{
+  "agent_id": "<agent_uuid>",
+  "input": "How do I return an item?",
+  "expected_output": "Explain the return policy.",
+  "label_ids": ["12", "13"]
+}
+```
+
+Send `label_ids: []` on update to clear all labels from a case. The
+`codeer eval cases-apply` manifest can resolve label names through a separate
+`labels` array; the legacy `label` field remains a local review/display label.
+
 ## Stage 6 — Diagnose + update
 
 | Method & path | Purpose |

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codeer-cli
-Version: 0.1.3
+Version: 0.1.5
 Summary: Command line tools for managing Codeer agents over the Codeer API.
 Project-URL: Homepage, https://www.codeer.ai
 Author: Codeer.AI
@@ -184,15 +184,19 @@ override matching JSON keys.
 
 Use Context Object FAQ entries to route high-value questions to a canonical KB
 file when semantic retrieval misses the right source. The FAQ target is a KB
-file's `snapshot_object_id`, shown by `codeer kb files`. Add `--range
-START_LINE:END_LINE` when the route should reserve chunks overlapping a stable
-line range inside that file.
+file's `snapshot_object_id`, shown by `codeer kb files`. Add `--range` when the
+route should reserve a stable passage inside that file. Ranges must include both
+line and column positions so the Codeer UI can map them onto rendered Markdown.
 
 ```bash
 codeer kb files --kb-id <kb-id>
 codeer kb faq-list --context-object-id <snapshot-object-id>
-codeer kb faq-create --context-object-id <snapshot-object-id> --question "..." --range 12:18 --dry-run
+codeer kb faq-create --context-object-id <snapshot-object-id> --question "..." --range 12:0-12:42 --dry-run
+codeer kb faq-update <faq-id> --range 12:0-12:42 --dry-run
 ```
 
+`--range` accepts `START_LINE:START_COLUMN-END_LINE:END_COLUMN`; repeat it to
+reserve multiple passages.
+
 After reviewing the dry-run output, rerun the create/update/delete command
 without `--dry-run` to apply it.

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/README.md

@@ -166,15 +166,19 @@ override matching JSON keys.
 
 Use Context Object FAQ entries to route high-value questions to a canonical KB
 file when semantic retrieval misses the right source. The FAQ target is a KB
-file's `snapshot_object_id`, shown by `codeer kb files`. Add `--range
-START_LINE:END_LINE` when the route should reserve chunks overlapping a stable
-line range inside that file.
+file's `snapshot_object_id`, shown by `codeer kb files`. Add `--range` when the
+route should reserve a stable passage inside that file. Ranges must include both
+line and column positions so the Codeer UI can map them onto rendered Markdown.
 
 ```bash
 codeer kb files --kb-id <kb-id>
 codeer kb faq-list --context-object-id <snapshot-object-id>
-codeer kb faq-create --context-object-id <snapshot-object-id> --question "..." --range 12:18 --dry-run
+codeer kb faq-create --context-object-id <snapshot-object-id> --question "..." --range 12:0-12:42 --dry-run
+codeer kb faq-update <faq-id> --range 12:0-12:42 --dry-run
 ```
 
+`--range` accepts `START_LINE:START_COLUMN-END_LINE:END_COLUMN`; repeat it to
+reserve multiple passages.
+
 After reviewing the dry-run output, rerun the create/update/delete command
 without `--dry-run` to apply it.

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "codeer-cli"
-version = "0.1.3"
+version = "0.1.5"
 description = "Command line tools for managing Codeer agents over the Codeer API."
 readme = "README.md"
 requires-python = ">=3.11"

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/src/codeer_cli/cli.py

@@ -3,7 +3,7 @@
     codeer check
     codeer agent list|get|apply|diff|versions
     codeer kb list|files|upload|faq-list|faq-get|faq-create|faq-update|faq-delete
-    codeer eval list|evaluators|evaluator-create|evaluator-update|run|export|reconcile|cases-apply|rubrics|rubrics-apply
+    codeer eval list|label-list|label-create|label-update|label-delete|case-update|case-delete|evaluators|evaluator-create|evaluator-update|run|export|reconcile|cases-apply|rubrics|rubrics-apply
     codeer history list|get|conversations|negative-feedback
 """
 
@@ -29,12 +29,17 @@ Safe workflow for coding agents:
   codeer agent get <agent-id> --full
   codeer kb list
   codeer eval list --agent <agent-id>
+  codeer eval label-list
+  codeer eval case-update --case <case-id> --input "..." --dry-run
   codeer eval evaluators
   codeer agent diff --agent <agent-id> --from-version <n> --to-version <n>
   codeer eval reconcile --agent <agent-id> --manifest .codeer/eval_cases.json
 
 Preview mutations before applying:
   codeer agent apply --payload agent.json --dry-run
+  codeer eval case-update --case <case-id> --input "..." --dry-run
+  codeer eval label-create --name "routing" --color "#0969da" --dry-run
+  codeer eval case-delete --case <case-id> --dry-run
   codeer eval cases-apply --agent <agent-id> --cases eval_cases.json --dry-run
   codeer eval rubrics-apply --rubrics rubrics.json --dry-run
   codeer kb upload --dir kb --name "Product KB" --dry-run

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/src/codeer_cli/commands/eval_cmd.py

@@ -46,6 +46,72 @@ def register(subparsers):
                    help="Write stripped full case payloads to this file; stdout stays compact unless --full.")
     p.set_defaults(func=run_list)
 
+    # codeer eval label-list/create/update/delete
+    p = sub.add_parser("label-list", help="List eval case labels in the workspace")
+    p.add_argument("--workspace", default=None, help="Workspace UUID (default: active API-key workspace)")
+    p.add_argument("--out", default=None)
+    p.set_defaults(func=run_label_list)
+
+    p = sub.add_parser("label-create", help="Create an eval case label; run --dry-run first")
+    p.add_argument("--name", required=True)
+    p.add_argument("--color", default=None, help="Hex color like #0969da (default: server default)")
+    p.add_argument("--workspace", default=None, help="Workspace UUID (default: active API-key workspace)")
+    p.add_argument("--dry-run", action="store_true")
+    p.add_argument("--out", default=None)
+    p.set_defaults(func=run_label_create)
+
+    p = sub.add_parser("label-update", help="Update an eval case label; run --dry-run first")
+    p.add_argument("--label", required=True, dest="label_id", help="Eval case label ID")
+    p.add_argument("--name", default=None)
+    p.add_argument("--color", default=None, help="Hex color like #0969da")
+    p.add_argument("--dry-run", action="store_true")
+    p.add_argument("--out", default=None)
+    p.set_defaults(func=run_label_update)
+
+    p = sub.add_parser("label-delete", help="Delete an eval case label; run --dry-run first")
+    p.add_argument("--label", required=True, dest="label_id", help="Eval case label ID")
+    p.add_argument("--dry-run", action="store_true")
+    p.add_argument("--out", default=None)
+    p.set_defaults(func=run_label_delete)
+
+    # codeer eval case-update
+    p = sub.add_parser("case-update", help="Update one eval case by UUID; run --dry-run first")
+    p.add_argument("--case", required=True, dest="case_id", help="Eval case UUID")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--input", help="New eval case input text")
+    g.add_argument("--input-file", help="Path to new eval case input text")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--expected-output", help="New expected_output text")
+    g.add_argument("--expected-output-file", help="Path to new expected_output text")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--rubric", help="New case-level rubric text")
+    g.add_argument("--rubric-file", help="Path to new case-level rubric text")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--note", help="New case note text")
+    g.add_argument("--note-file", help="Path to new case note text")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--meta-json", help="New case meta JSON object")
+    g.add_argument("--meta-file", help="Path to new case meta JSON object")
+    p.add_argument("--attachment-ids", default=None,
+                   help="Comma-separated file UUIDs to set as the case attachments")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--label-ids", default=None,
+                   help="Comma-separated eval case label IDs to set on the case")
+    g.add_argument("--clear-labels", action="store_true",
+                   help="Remove all labels from the case")
+    p.add_argument("--dry-run", action="store_true",
+                   help="Validate inputs and print intended mutation without writing server state.")
+    p.add_argument("--out", default=None)
+    p.set_defaults(func=run_case_update)
+
+    # codeer eval case-delete
+    p = sub.add_parser("case-delete", help="Delete one eval case by UUID; run --dry-run first")
+    p.add_argument("--case", required=True, dest="case_id", help="Eval case UUID")
+    p.add_argument("--dry-run", action="store_true",
+                   help="Print intended deletion without writing server state.")
+    p.add_argument("--out", default=None)
+    p.set_defaults(func=run_case_delete)
+
     # codeer eval evaluators
     p = sub.add_parser(
         "evaluators",
@@ -121,6 +187,8 @@ def register(subparsers):
     p.add_argument("--agent", required=True)
     p.add_argument("--attachments-dir", default=None, dest="attachments_dir")
     p.add_argument("--allow-duplicates", action="store_true")
+    p.add_argument("--create-labels", action="store_true",
+                   help="Create missing labels referenced by manifest case labels.")
     p.add_argument("--dry-run", action="store_true",
                    help="Validate manifest and print intended mutations without writing server state.")
     p.add_argument("--out", default=None)
@@ -151,6 +219,7 @@ def register(subparsers):
 # ---------------------------------------------------------------------------
 
 def _case_summary(case: dict, *, full: bool = False) -> dict:
+    labels = case.get("labels") or []
     row = {
         "id": case.get("id"),
         "input_preview": truncate(case.get("input") or "", 240 if full else 80),
@@ -158,6 +227,11 @@ def _case_summary(case: dict, *, full: bool = False) -> dict:
         "expected_output_chars": len(case.get("expected_output") or ""),
         "note_preview": truncate(case.get("note") or "", 180 if full else 100),
         "attachment_count": len(case.get("attachments") or case.get("attachment_ids") or []),
+        "labels": [
+            {"id": label.get("id"), "name": label.get("name"), "color": label.get("color")}
+            for label in labels
+            if isinstance(label, dict)
+        ],
     }
     if full:
         row["created_at"] = case.get("created_at")
@@ -206,6 +280,232 @@ def run_list(args, client) -> int:
     return 0
 
 
+# ---------------------------------------------------------------------------
+# eval case labels
+# ---------------------------------------------------------------------------
+
+def _workspace_arg_or_default(client, workspace_id: str | None) -> str:
+    if workspace_id:
+        return workspace_id
+    ws, _ = client.resolve_scope()
+    return ws
+
+
+def _label_summary(label: dict) -> dict:
+    return {
+        "id": label.get("id"),
+        "name": label.get("name"),
+        "color": label.get("color"),
+        "workspace_id": label.get("workspace_id"),
+    }
+
+
+def run_label_list(args, client) -> int:
+    workspace_id = _workspace_arg_or_default(client, args.workspace)
+    labels = eval_mod.list_case_labels(client, workspace_id=workspace_id)
+    out = {
+        "workspace_id": workspace_id,
+        "label_count": len(labels),
+        "labels": [_label_summary(label) for label in labels],
+    }
+    print_json(out)
+    write_json(args.out, out)
+    return 0
+
+
+def run_label_create(args, client) -> int:
+    workspace_id = _workspace_arg_or_default(client, args.workspace)
+    if args.dry_run:
+        out = {
+            "dry_run": True,
+            "operation": "label_create",
+            "method": "POST",
+            "path": f"/eval/workspaces/{workspace_id}/case-labels",
+            "workspace_id": workspace_id,
+            "name": args.name,
+            "color": args.color,
+            "would_write_server_state": True,
+            "next_step": "Review this summary, then rerun without --dry-run after approval.",
+        }
+        print_json(out)
+        write_json(args.out, out)
+        return 0
+
+    label = eval_mod.create_case_label(
+        client, workspace_id=workspace_id, name=args.name, color=args.color
+    )
+    out = _label_summary(strip_noisy_fields(label))
+    print_json(out)
+    write_json(args.out, out)
+    return 0
+
+
+def run_label_update(args, client) -> int:
+    if args.name is None and args.color is None:
+        log("error: provide --name and/or --color")
+        return 2
+
+    if args.dry_run:
+        out = {
+            "dry_run": True,
+            "operation": "label_update",
+            "method": "PUT",
+            "path": f"/eval/case-labels/{args.label_id}",
+            "label_id": args.label_id,
+            "updates": {"name": args.name, "color": args.color},
+            "would_write_server_state": True,
+            "next_step": "Review this summary, then rerun without --dry-run after approval.",
+        }
+        print_json(out)
+        write_json(args.out, out)
+        return 0
+
+    label = eval_mod.update_case_label(
+        client, label_id=args.label_id, name=args.name, color=args.color
+    )
+    out = _label_summary(strip_noisy_fields(label))
+    print_json(out)
+    write_json(args.out, out)
+    return 0
+
+
+def run_label_delete(args, client) -> int:
+    if args.dry_run:
+        out = {
+            "dry_run": True,
+            "operation": "label_delete",
+            "method": "DELETE",
+            "path": f"/eval/case-labels/{args.label_id}",
+            "label_id": args.label_id,
+            "would_write_server_state": True,
+            "next_step": "Review this summary, then rerun without --dry-run after approval.",
+        }
+        print_json(out)
+        write_json(args.out, out)
+        return 0
+
+    deleted = strip_noisy_fields(eval_mod.delete_case_label(client, label_id=args.label_id))
+    print_json(deleted)
+    write_json(args.out, deleted)
+    return 0
+
+
+# ---------------------------------------------------------------------------
+# eval case-update / case-delete
+# ---------------------------------------------------------------------------
+
+def _read_text_arg(value: str | None, file_path: str | None) -> str | None:
+    if file_path is not None:
+        return Path(file_path).read_text()
+    return value
+
+
+def _read_meta_arg(value: str | None, file_path: str | None) -> dict | None:
+    if file_path is not None:
+        raw = Path(file_path).read_text()
+    elif value is not None:
+        raw = value
+    else:
+        return None
+
+    meta = json.loads(raw)
+    if not isinstance(meta, dict):
+        raise ValueError("case meta must be a JSON object")
+    return meta
+
+
+def run_case_update(args, client) -> int:
+    try:
+        input_text = _read_text_arg(args.input, args.input_file)
+        expected_output = _read_text_arg(args.expected_output, args.expected_output_file)
+        rubric = _read_text_arg(args.rubric, args.rubric_file)
+        note = _read_text_arg(args.note, args.note_file)
+        meta = _read_meta_arg(args.meta_json, args.meta_file)
+    except (OSError, json.JSONDecodeError, ValueError) as e:
+        log(f"error: {e}")
+        return 2
+    attachment_ids = _ids(args.attachment_ids)
+    label_ids = [] if args.clear_labels else _ids(args.label_ids)
+
+    has_update = any(
+        value is not None
+        for value in (input_text, expected_output, rubric, note, meta, attachment_ids, label_ids)
+    ) or args.clear_labels
+    if not has_update:
+        log(
+            "error: provide at least one of --input, --input-file, --expected-output, "
+            "--expected-output-file, --rubric, --rubric-file, --note, --note-file, "
+            "--meta-json, --meta-file, --attachment-ids, --label-ids, --clear-labels"
+        )
+        return 2
+
+    if args.dry_run:
+        current = strip_noisy_fields(eval_mod.get_case(client, args.case_id))
+        out = {
+            "dry_run": True,
+            "operation": "case_update",
+            "method": "PUT",
+            "path": f"/external/eval/cases/{args.case_id}",
+            "case_id": args.case_id,
+            "current": _case_summary(current, full=True),
+            "updates": {
+                "input_chars": len(input_text) if input_text is not None else None,
+                "expected_output_chars": (
+                    len(expected_output) if expected_output is not None else None
+                ),
+                "rubric_chars": len(rubric) if rubric is not None else None,
+                "note_chars": len(note) if note is not None else None,
+                "meta": meta,
+                "attachment_ids": attachment_ids,
+                "label_ids": label_ids,
+            },
+            "would_write_server_state": True,
+            "next_step": "Review this summary, then rerun without --dry-run after approval.",
+        }
+        print_json(out)
+        write_json(args.out, out)
+        return 0
+
+    updated = eval_mod.update_case(
+        client,
+        args.case_id,
+        input=input_text,
+        expected_output=expected_output,
+        rubric=rubric,
+        attachment_ids=attachment_ids,
+        label_ids=label_ids,
+        meta=meta,
+        note=note,
+    )
+    out = strip_noisy_fields(updated)
+    print_json(out)
+    write_json(args.out, out)
+    return 0
+
+
+def run_case_delete(args, client) -> int:
+    if args.dry_run:
+        current = strip_noisy_fields(eval_mod.get_case(client, args.case_id))
+        out = {
+            "dry_run": True,
+            "operation": "case_delete",
+            "method": "DELETE",
+            "path": f"/external/eval/cases/{args.case_id}",
+            "case_id": args.case_id,
+            "current": _case_summary(current, full=True),
+            "would_write_server_state": True,
+            "next_step": "Review this summary, then rerun without --dry-run after approval.",
+        }
+        print_json(out)
+        write_json(args.out, out)
+        return 0
+
+    deleted = strip_noisy_fields(eval_mod.delete_case(client, args.case_id))
+    print_json(deleted)
+    write_json(args.out, deleted)
+    return 0
+
+
 # ---------------------------------------------------------------------------
 # eval evaluators
 # ---------------------------------------------------------------------------
@@ -795,6 +1095,50 @@ def _upload_attachment(client: CodeerClient, *, file_path: Path, workspace_id: s
     return uuid
 
 
+def _manifest_label_names(case: dict) -> list[str]:
+    raw = case.get("labels")
+    if raw is None:
+        return []
+    if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):
+        raise ValueError(f"case '{case.get('label')}' labels must be a list of label names")
+    return [item.strip() for item in raw if item.strip()]
+
+
+def _manifest_label_ids(case: dict) -> list[str] | None:
+    raw = case.get("label_ids")
+    if raw is None:
+        return None
+    if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):
+        raise ValueError(f"case '{case.get('label')}' label_ids must be a list of label ID strings")
+    return [item.strip() for item in raw if item.strip()]
+
+
+def _dedupe_preserve_order(items: list[str]) -> list[str]:
+    seen: set[str] = set()
+    out: list[str] = []
+    for item in items:
+        if item in seen:
+            continue
+        seen.add(item)
+        out.append(item)
+    return out
+
+
+def _resolve_case_label_ids(case: dict, labels_by_name: dict[str, dict]) -> tuple[list[str] | None, list[str]]:
+    explicit_ids = _manifest_label_ids(case)
+    label_names = _manifest_label_names(case)
+    if explicit_ids is None and not label_names:
+        return None, []
+
+    resolved_ids = list(explicit_ids or [])
+    for name in label_names:
+        label = labels_by_name.get(name.casefold())
+        if label is None:
+            raise ValueError(f"case '{case.get('label')}' references unknown label '{name}'")
+        resolved_ids.append(str(label["id"]))
+    return _dedupe_preserve_order(resolved_ids), label_names
+
+
 def run_cases_apply(args, client) -> int:
     payload = json.loads(Path(args.cases).read_text())
     cases = payload.get("cases") or []
@@ -815,6 +1159,53 @@ def run_cases_apply(args, client) -> int:
         return 2
 
     workspace_id, _ = client.resolve_scope()
+    try:
+        manifest_label_names = sorted({
+            name
+            for case in cases
+            for name in _manifest_label_names(case)
+        }, key=str.casefold)
+        for case in cases:
+            _manifest_label_ids(case)
+    except ValueError as e:
+        log(f"error: {e}")
+        return 2
+
+    labels_by_name: dict[str, dict] = {}
+    created_labels: list[dict] = []
+    would_create_labels: list[str] = []
+    if manifest_label_names:
+        labels_by_name = {
+            (label.get("name") or "").casefold(): label
+            for label in eval_mod.list_case_labels(client, workspace_id=workspace_id)
+            if label.get("name")
+        }
+        missing_label_names = [
+            name for name in manifest_label_names
+            if name.casefold() not in labels_by_name
+        ]
+        if missing_label_names and not args.create_labels:
+            log(
+                "error: manifest references missing labels: "
+                + ", ".join(missing_label_names)
+                + ". Create them first with `codeer eval label-create`, "
+                + "or rerun cases-apply with --create-labels."
+            )
+            return 2
+        if args.dry_run:
+            would_create_labels = missing_label_names
+            for name in missing_label_names:
+                labels_by_name[name.casefold()] = {
+                    "id": f"(new:{name})",
+                    "name": name,
+                    "color": "#0969da",
+                }
+        else:
+            for name in missing_label_names:
+                log(f"creating label: {name}")
+                label = eval_mod.create_case_label(client, workspace_id=workspace_id, name=name)
+                labels_by_name[name.casefold()] = label
+                created_labels.append(_label_summary(label))
 
     existing_by_input: dict[str, dict] = {}
     if not args.allow_duplicates:
@@ -838,6 +1229,11 @@ def run_cases_apply(args, client) -> int:
             return 2
 
         label = case.get("label", "(unlabeled)")
+        try:
+            case_label_ids, case_label_names = _resolve_case_label_ids(case, labels_by_name)
+        except ValueError as e:
+            log(f"error: {e}")
+            return 2
         attachment_ids: list[str] = []
         for fname in case.get("attachment_files") or []:
             fp = (attach_dir / fname).resolve() if attach_dir else None
@@ -866,16 +1262,26 @@ def run_cases_apply(args, client) -> int:
                         or attachment_ids
                         or case.get("meta") is not None
                         or case.get("note") is not None
+                        or case_label_ids is not None
                     ),
+                    "labels": case_label_names,
+                    "label_ids": case_label_ids,
                     "rubric_count": len(rubrics),
                 })
                 continue
             log(f"reusing existing case: {label} ({case_id[:8]})")
-            if case.get("expected_output") is not None or attachment_ids or case.get("meta") is not None or case.get("note") is not None:
+            if (
+                case.get("expected_output") is not None
+                or attachment_ids
+                or case.get("meta") is not None
+                or case.get("note") is not None
+                or case_label_ids is not None
+            ):
                 eval_mod.update_case(
                     client, case_id,
                     expected_output=case.get("expected_output"),
                     attachment_ids=attachment_ids or None,
+                    label_ids=case_label_ids,
                     meta=case.get("meta"),
                     note=case.get("note"),
                 )
@@ -894,6 +1300,8 @@ def run_cases_apply(args, client) -> int:
                 "input_chars": len(case.get("input") or ""),
                 "expected_output_chars": len(case.get("expected_output") or ""),
                 "attachment_count": len(attachment_ids),
+                "labels": case_label_names,
+                "label_ids": case_label_ids,
                 "rubric_count": len(rubrics),
             })
             continue
@@ -903,6 +1311,7 @@ def run_cases_apply(args, client) -> int:
             client, agent_id=args.agent, input=case["input"],
             expected_output=case.get("expected_output"),
             attachment_ids=attachment_ids or None,
+            label_ids=case_label_ids,
             rubrics_by_evaluator=rubrics, meta=case.get("meta"),
             note=case.get("note"),
         )
@@ -910,12 +1319,19 @@ def run_cases_apply(args, client) -> int:
         labels.append(label)
         created.append({"case_id": result["id"], "label": label})
 
-    out = {"case_ids": case_ids, "labels": labels, "created": created, "reused": reused}
+    out = {
+        "case_ids": case_ids,
+        "labels": labels,
+        "created": created,
+        "reused": reused,
+        "created_case_labels": created_labels,
+    }
     if args.dry_run:
         out.update({
             "dry_run": True,
             "operation": "cases_apply",
             "agent_id": args.agent,
+            "would_create_case_labels": would_create_labels,
             "updates": dry_run_updates,
             "would_write_server_state": True,
             "next_step": "Review this summary, then rerun without --dry-run after approval.",

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/src/codeer_cli/commands/kb.py

@@ -107,8 +107,7 @@ def register(subparsers):
     p.add_argument("--context-object-id", type=int, required=True,
                    help="KB file snapshot_object_id from `codeer kb files`")
     p.add_argument("--question", required=True)
-    p.add_argument("--range", dest="ranges", action="append", type=_parse_faq_range, default=None,
-                   help="Reserve matching chunks that overlap START_LINE:END_LINE; repeatable")
+    _add_faq_range_args(p)
     p.add_argument("--dry-run", action="store_true",
                    help="Print intended request without writing server state.")
     p.add_argument("--out", default=None, help="Write result JSON to this file too")
@@ -119,8 +118,7 @@ def register(subparsers):
     p.add_argument("--context-object-id", type=int, default=None,
                    help="Move FAQ to a different KB file snapshot_object_id")
     p.add_argument("--question", default=None)
-    p.add_argument("--range", dest="ranges", action="append", type=_parse_faq_range, default=None,
-                   help="Replace reserved ranges with START_LINE:END_LINE; repeatable")
+    _add_faq_range_args(p, verb="Replace")
     p.add_argument("--dry-run", action="store_true",
                    help="Print intended request without writing server state.")
     p.add_argument("--out", default=None, help="Write result JSON to this file too")
@@ -217,18 +215,50 @@ def _dry_run(path: str | None, result: dict) -> int:
     return 0
 
 
+def _add_faq_range_args(parser, *, verb: str = "Reserve") -> None:
+    parser.add_argument(
+        "--range",
+        dest="ranges",
+        action="append",
+        type=_parse_faq_range,
+        default=None,
+        help=(
+            f"{verb} matching passages as "
+            "START_LINE:START_COLUMN-END_LINE:END_COLUMN; repeatable"
+        ),
+    )
+
+
 def _parse_faq_range(value: str) -> dict[str, int]:
     try:
-        start_raw, end_raw = value.split(":", 1)
-        start_line = int(start_raw)
-        end_line = int(end_raw)
+        start_raw, end_raw = value.split("-", 1)
+        start_line_raw, start_column_raw = start_raw.split(":", 1)
+        end_line_raw, end_column_raw = end_raw.split(":", 1)
+        faq_range = {
+            "start_line": int(start_line_raw),
+            "start_column": int(start_column_raw),
+            "end_line": int(end_line_raw),
+            "end_column": int(end_column_raw),
+        }
     except ValueError as exc:
-        raise argparse.ArgumentTypeError("expected START_LINE:END_LINE") from exc
+        raise argparse.ArgumentTypeError(
+            "expected START_LINE:START_COLUMN-END_LINE:END_COLUMN"
+        ) from exc
+    _validate_faq_range_position(faq_range)
+    return faq_range
+
+
+def _validate_faq_range_position(faq_range: dict[str, int]) -> None:
+    start_line = faq_range["start_line"]
+    end_line = faq_range["end_line"]
+    start_column = faq_range["start_column"]
+    end_column = faq_range["end_column"]
     if start_line < 1 or end_line < 1:
         raise argparse.ArgumentTypeError("line numbers must be >= 1")
-    if end_line < start_line:
-        raise argparse.ArgumentTypeError("END_LINE must be >= START_LINE")
-    return {"start_line": start_line, "end_line": end_line}
+    if start_column < 0 or end_column < 0:
+        raise argparse.ArgumentTypeError("column numbers must be >= 0")
+    if (end_line, end_column) < (start_line, start_column):
+        raise argparse.ArgumentTypeError("end position must be >= start position")
 
 
 def _parse_config_json(config_json: str | None) -> dict | None:

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/src/codeer_cli/eval_.py

@@ -22,6 +22,7 @@ def create_case(
     expected_output: Optional[str] = None,
     rubric: Optional[str] = None,
     attachment_ids: Optional[List[str]] = None,
+    label_ids: Optional[List[str]] = None,
     meta: Optional[dict] = None,
     note: Optional[str] = None,
 ) -> dict:
@@ -41,6 +42,8 @@ def create_case(
         body["rubric"] = rubric
     if attachment_ids:
         body["attachment_ids"] = attachment_ids
+    if label_ids is not None:
+        body["label_ids"] = label_ids
     if meta:
         body["meta"] = meta
     if note is not None:
@@ -64,6 +67,7 @@ def update_case(
     expected_output: Optional[str] = None,
     rubric: Optional[str] = None,
     attachment_ids: Optional[List[str]] = None,
+    label_ids: Optional[List[str]] = None,
     meta: Optional[dict] = None,
     note: Optional[str] = None,
 ) -> dict:
@@ -76,6 +80,8 @@ def update_case(
         body["rubric"] = rubric
     if attachment_ids is not None:
         body["attachment_ids"] = attachment_ids
+    if label_ids is not None:
+        body["label_ids"] = label_ids
     if meta is not None:
         body["meta"] = meta
     if note is not None:
@@ -83,6 +89,48 @@ def update_case(
     return client.put(f"/external/eval/cases/{case_id}", json=body)
 
 
+def delete_case(client: CodeerClient, case_id: str) -> dict:
+    return client.delete(f"/external/eval/cases/{case_id}")
+
+
+# --- case labels -----------------------------------------------------------
+
+def list_case_labels(client: CodeerClient, *, workspace_id: str) -> list[dict]:
+    return client.get(f"/eval/workspaces/{workspace_id}/case-labels")
+
+
+def create_case_label(
+    client: CodeerClient,
+    *,
+    workspace_id: str,
+    name: str,
+    color: Optional[str] = None,
+) -> dict:
+    body: dict[str, Any] = {"name": name}
+    if color is not None:
+        body["color"] = color
+    return client.post(f"/eval/workspaces/{workspace_id}/case-labels", json=body)
+
+
+def update_case_label(
+    client: CodeerClient,
+    *,
+    label_id: str,
+    name: Optional[str] = None,
+    color: Optional[str] = None,
+) -> dict:
+    body: dict[str, Any] = {}
+    if name is not None:
+        body["name"] = name
+    if color is not None:
+        body["color"] = color
+    return client.put(f"/eval/case-labels/{label_id}", json=body)
+
+
+def delete_case_label(client: CodeerClient, *, label_id: str) -> dict:
+    return client.delete(f"/eval/case-labels/{label_id}")
+
+
 
 # --- evaluators -----------------------------------------------------------
 
@@ -393,6 +441,7 @@ def create_case_with_rubrics(
     rubrics_by_evaluator: dict[str, str],
     expected_output: Optional[str] = None,
     attachment_ids: Optional[List[str]] = None,
+    label_ids: Optional[List[str]] = None,
     meta: Optional[dict] = None,
     note: Optional[str] = None,
 ) -> dict:
@@ -412,6 +461,7 @@ def create_case_with_rubrics(
         input=input,
         expected_output=expected_output,
         attachment_ids=attachment_ids,
+        label_ids=label_ids,
         meta=meta,
         note=note,
     )

codeer_cli-0.1.5/tests/test_eval_labels.py

@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+import unittest
+
+from codeer_cli import eval_ as eval_mod
+from codeer_cli.commands.eval_cmd import _resolve_case_label_ids
+
+
+class FakeClient:
+    def __init__(self) -> None:
+        self.calls: list[tuple[str, str, dict]] = []
+
+    def get(self, path: str):
+        self.calls.append(("GET", path, {}))
+        return []
+
+    def post(self, path: str, **kwargs):
+        self.calls.append(("POST", path, kwargs))
+        return {"id": "created"}
+
+    def put(self, path: str, **kwargs):
+        self.calls.append(("PUT", path, kwargs))
+        return {"id": "updated"}
+
+    def delete(self, path: str, **kwargs):
+        self.calls.append(("DELETE", path, kwargs))
+        return {"ok": True}
+
+
+class EvalLabelClientTests(unittest.TestCase):
+    def test_create_case_sends_label_ids(self) -> None:
+        client = FakeClient()
+
+        eval_mod.create_case(
+            client,  # type: ignore[arg-type]
+            agent_id="agent-1",
+            input="How much?",
+            label_ids=["10", "11"],
+        )
+
+        self.assertEqual(client.calls[0][0], "POST")
+        self.assertEqual(client.calls[0][1], "/external/eval/cases")
+        self.assertEqual(client.calls[0][2]["json"]["label_ids"], ["10", "11"])
+
+    def test_update_case_sends_empty_label_ids_to_clear(self) -> None:
+        client = FakeClient()
+
+        eval_mod.update_case(
+            client,  # type: ignore[arg-type]
+            "case-1",
+            label_ids=[],
+        )
+
+        self.assertEqual(client.calls[0][0], "PUT")
+        self.assertEqual(client.calls[0][1], "/external/eval/cases/case-1")
+        self.assertEqual(client.calls[0][2]["json"]["label_ids"], [])
+
+    def test_case_label_crud_paths(self) -> None:
+        client = FakeClient()
+
+        eval_mod.list_case_labels(client, workspace_id="ws-1")  # type: ignore[arg-type]
+        eval_mod.create_case_label(client, workspace_id="ws-1", name="Routing")  # type: ignore[arg-type]
+        eval_mod.update_case_label(client, label_id="5", color="#0969da")  # type: ignore[arg-type]
+        eval_mod.delete_case_label(client, label_id="5")  # type: ignore[arg-type]
+
+        self.assertEqual(client.calls[0], ("GET", "/eval/workspaces/ws-1/case-labels", {}))
+        self.assertEqual(client.calls[1][0:2], ("POST", "/eval/workspaces/ws-1/case-labels"))
+        self.assertEqual(client.calls[1][2]["json"], {"name": "Routing"})
+        self.assertEqual(client.calls[2][0:2], ("PUT", "/eval/case-labels/5"))
+        self.assertEqual(client.calls[2][2]["json"], {"color": "#0969da"})
+        self.assertEqual(client.calls[3][0:2], ("DELETE", "/eval/case-labels/5"))
+
+
+class EvalLabelManifestTests(unittest.TestCase):
+    def test_resolve_case_label_ids_from_ids_and_names(self) -> None:
+        label_ids, label_names = _resolve_case_label_ids(
+            {"label_ids": ["7"], "labels": ["Routing", "Billing"]},
+            {
+                "routing": {"id": "8", "name": "Routing"},
+                "billing": {"id": "9", "name": "Billing"},
+            },
+        )
+
+        self.assertEqual(label_ids, ["7", "8", "9"])
+        self.assertEqual(label_names, ["Routing", "Billing"])
+
+    def test_resolve_case_label_ids_dedupes_preserving_order(self) -> None:
+        label_ids, _ = _resolve_case_label_ids(
+            {"label_ids": ["7", "8"], "labels": ["Routing"]},
+            {"routing": {"id": "8", "name": "Routing"}},
+        )
+
+        self.assertEqual(label_ids, ["7", "8"])
+
+    def test_unknown_label_name_is_error(self) -> None:
+        with self.assertRaises(ValueError):
+            _resolve_case_label_ids({"labels": ["Missing"]}, {})
+
+
+if __name__ == "__main__":
+    unittest.main()

codeer_cli-0.1.5/tests/test_kb_ranges.py

@@ -0,0 +1,35 @@
+from __future__ import annotations
+
+import argparse
+import unittest
+
+from codeer_cli.commands.kb import _parse_faq_range
+
+
+class FaqRangeTests(unittest.TestCase):
+    def test_parse_column_range(self) -> None:
+        self.assertEqual(
+            _parse_faq_range("47:2-48:9"),
+            {
+                "start_line": 47,
+                "start_column": 2,
+                "end_line": 48,
+                "end_column": 9,
+            },
+        )
+
+    def test_rejects_single_line_range(self) -> None:
+        with self.assertRaises(argparse.ArgumentTypeError):
+            _parse_faq_range("47")
+
+    def test_rejects_legacy_line_range(self) -> None:
+        with self.assertRaises(argparse.ArgumentTypeError):
+            _parse_faq_range("47:48")
+
+    def test_rejects_reversed_column_range(self) -> None:
+        with self.assertRaises(argparse.ArgumentTypeError):
+            _parse_faq_range("48:1-47:1")
+
+
+if __name__ == "__main__":
+    unittest.main()

{codeer_cli-0.1.3 → codeer_cli-0.1.5}/uv.lock

@@ -26,7 +26,7 @@ wheels = [
 
 [[package]]
 name = "codeer-cli"
-version = "0.1.3"
+version = "0.1.5"
 source = { editable = "." }
 dependencies = [
     { name = "httpx" },