@byteplus/ark-cli-darwin-x64 0.0.0-stage → 1.0.46-prerelease.dc921200a42d
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/bin/arkcli-darwin-amd64 +4 -0
- package/manifest.json +43 -0
- package/package.json +20 -4
- package/skills/arkcli-agent/SKILL.md +152 -0
- package/skills/arkcli-agent/references/agent.md +398 -0
- package/skills/arkcli-agent/references/byok.md +65 -0
- package/skills/arkcli-agent/references/context-management.md +42 -0
- package/skills/arkcli-agent/references/debug-export.md +26 -0
- package/skills/arkcli-agent/references/evals.md +178 -0
- package/skills/arkcli-agent/references/events-chat.md +148 -0
- package/skills/arkcli-agent/references/interfaces-gaps.md +52 -0
- package/skills/arkcli-agent/references/mcp-vault.md +111 -0
- package/skills/arkcli-agent/references/model-config.md +46 -0
- package/skills/arkcli-agent/references/self-hosted.md +47 -0
- package/skills/arkcli-agent/references/session-files.md +149 -0
- package/skills/arkcli-agent/references/session-upgrade.md +61 -0
- package/skills/arkcli-agent/references/skills.md +87 -0
- package/skills/arkcli-agent/references/todo-write.md +31 -0
- package/skills/arkcli-api-explorer/SKILL.md +188 -0
- package/skills/arkcli-api-explorer/references/arkcli-api.md +180 -0
- package/skills/arkcli-api-explorer/references/evals.md +176 -0
- package/skills/arkcli-auth/SKILL.md +184 -0
- package/skills/arkcli-auth/references/arkcli-auth-login.md +92 -0
- package/skills/arkcli-auth/references/arkcli-auth-status.md +109 -0
- package/skills/arkcli-auth/references/arkcli-auth-whoami.md +56 -0
- package/skills/arkcli-auth/references/auth-modes.md +78 -0
- package/skills/arkcli-auth/references/evals.md +82 -0
- package/skills/arkcli-auth/references/identity-resolution.md +33 -0
- package/skills/arkcli-auth/references/realname-gate.md +69 -0
- package/skills/arkcli-billing/SKILL.md +122 -0
- package/skills/arkcli-billing/references/arkcli-billing-list.md +259 -0
- package/skills/arkcli-billing/references/evals.md +107 -0
- package/skills/arkcli-chat/SKILL.md +114 -0
- package/skills/arkcli-chat/references/arkcli-chat.md +283 -0
- package/skills/arkcli-chat/references/caching-thinking.md +124 -0
- package/skills/arkcli-chat/references/chat-meta.md +77 -0
- package/skills/arkcli-chat/references/evals.md +34 -0
- package/skills/arkcli-chat/references/stream-events.md +124 -0
- package/skills/arkcli-chat/references/text-format.md +167 -0
- package/skills/arkcli-chat/references/tools.md +119 -0
- package/skills/arkcli-config/SKILL.md +197 -0
- package/skills/arkcli-config/references/arkcli-config-init.md +89 -0
- package/skills/arkcli-config/references/arkcli-config-profile.md +106 -0
- package/skills/arkcli-config/references/evals.md +154 -0
- package/skills/arkcli-connect/SKILL.md +196 -0
- package/skills/arkcli-connect/references/arkcli-connect.md +165 -0
- package/skills/arkcli-connect/references/evals.md +231 -0
- package/skills/arkcli-custommodel/SKILL.md +167 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-available-quantizations.md +29 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-delete.md +37 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-get.md +45 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-list.md +65 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-quantize.md +59 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-update.md +47 -0
- package/skills/arkcli-custommodel/references/arkcli-custommodel-upload.md +64 -0
- package/skills/arkcli-custommodel/references/evals.md +59 -0
- package/skills/arkcli-datasets/SKILL.md +39 -0
- package/skills/arkcli-datasets/references/commands.md +83 -0
- package/skills/arkcli-datasets/references/evals.md +31 -0
- package/skills/arkcli-deploy/SKILL.md +89 -0
- package/skills/arkcli-deploy/references/arkcli-deploy.md +141 -0
- package/skills/arkcli-deploy/references/evals.md +80 -0
- package/skills/arkcli-docs/SKILL.md +453 -0
- package/skills/arkcli-docs/references/commands.md +283 -0
- package/skills/arkcli-docs/references/evals.md +201 -0
- package/skills/arkcli-docs/references/failure-modes.md +85 -0
- package/skills/arkcli-doctor/CONTRIBUTING.md +54 -0
- package/skills/arkcli-doctor/SKILL.md +367 -0
- package/skills/arkcli-doctor/references/error-codes.md +162 -0
- package/skills/arkcli-doctor/references/evals.md +385 -0
- package/skills/arkcli-doctor/references/scope-account.md +94 -0
- package/skills/arkcli-doctor/references/scope-cli.md +88 -0
- package/skills/arkcli-doctor/references/scope-infer-endpoint.md +132 -0
- package/skills/arkcli-doctor/references/scope-metrics.md +157 -0
- package/skills/arkcli-doctor/references/scope-model.md +130 -0
- package/skills/arkcli-doctor/references/verify-origin.md +102 -0
- package/skills/arkcli-gen/SKILL.md +188 -0
- package/skills/arkcli-gen/references/arkcli-gen.md +230 -0
- package/skills/arkcli-gen/references/gen-meta.md +154 -0
- package/skills/arkcli-gen/references/image-stream.md +105 -0
- package/skills/arkcli-helper/SKILL.md +200 -0
- package/skills/arkcli-helper/references/arkcli-helper.md +298 -0
- package/skills/arkcli-helper/references/evals.md +334 -0
- package/skills/arkcli-iam/SKILL.md +46 -0
- package/skills/arkcli-iam/references/arkcli-iam-userid.md +54 -0
- package/skills/arkcli-iam/references/evals.md +24 -0
- package/skills/arkcli-infer-endpoint/SKILL.md +82 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-capability-get.md +50 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-create.md +121 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-delete.md +71 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-get.md +43 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-list.md +82 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-start.md +36 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-stop.md +36 -0
- package/skills/arkcli-infer-endpoint/references/arkcli-infer-endpoint-update.md +78 -0
- package/skills/arkcli-infer-endpoint/references/evals.md +24 -0
- package/skills/arkcli-models/SKILL.md +305 -0
- package/skills/arkcli-models/references/arkcli-models-activate.md +103 -0
- package/skills/arkcli-models/references/arkcli-models-get.md +94 -0
- package/skills/arkcli-models/references/arkcli-models-list.md +71 -0
- package/skills/arkcli-models/references/arkcli-models-scenario-table.md +75 -0
- package/skills/arkcli-models/references/arkcli-models-search.md +170 -0
- package/skills/arkcli-models/references/evals.md +57 -0
- package/skills/arkcli-onboard/SKILL.md +169 -0
- package/skills/arkcli-onboard/references/evals.md +156 -0
- package/skills/arkcli-plans/SKILL.md +135 -0
- package/skills/arkcli-plans/references/arkcli-plans-buy.md +125 -0
- package/skills/arkcli-plans/references/arkcli-plans-get.md +84 -0
- package/skills/arkcli-plans/references/arkcli-plans-model-apply.md +58 -0
- package/skills/arkcli-plans/references/arkcli-plans-model-list.md +114 -0
- package/skills/arkcli-plans/references/arkcli-plans-renew.md +125 -0
- package/skills/arkcli-plans/references/arkcli-plans-team-rotate-apikey.md +61 -0
- package/skills/arkcli-plans/references/arkcli-plans-team-seat-assign.md +147 -0
- package/skills/arkcli-plans/references/arkcli-plans-team-seat-list.md +100 -0
- package/skills/arkcli-plans/references/evals.md +178 -0
- package/skills/arkcli-pricing/SKILL.md +120 -0
- package/skills/arkcli-pricing/references/arkcli-pricing-models.md +258 -0
- package/skills/arkcli-pricing/references/arkcli-pricing-plans.md +139 -0
- package/skills/arkcli-profile/SKILL.md +168 -0
- package/skills/arkcli-profile/references/arkcli-profile-create.md +92 -0
- package/skills/arkcli-profile/references/arkcli-profile-keys.md +135 -0
- package/skills/arkcli-profile/references/arkcli-profile-set-default.md +81 -0
- package/skills/arkcli-profile/references/evals.md +93 -0
- package/skills/arkcli-resources/SKILL.md +112 -0
- package/skills/arkcli-resources/references/arkcli-resources-list.md +105 -0
- package/skills/arkcli-shared/SKILL.md +209 -0
- package/skills/arkcli-shared/references/global-flags.md +77 -0
- package/skills/arkcli-shared/references/profile-defaults.md +35 -0
- package/skills/arkcli-shared/references/troubleshooting.md +49 -0
- package/skills/arkcli-shared/references/update.md +117 -0
- package/skills/arkcli-train-finetune/SKILL.md +109 -0
- package/skills/arkcli-train-finetune/references/create.md +274 -0
- package/skills/arkcli-train-finetune/references/download.md +50 -0
- package/skills/arkcli-train-finetune/references/evals.md +59 -0
- package/skills/arkcli-train-finetune/references/export-deploy.md +179 -0
- package/skills/arkcli-train-finetune/references/list.md +79 -0
- package/skills/arkcli-train-finetune/references/manage.md +140 -0
- package/skills/arkcli-understand/SKILL.md +105 -0
- package/skills/arkcli-understand/references/arkcli-understand.md +249 -0
- package/skills/arkcli-understand/references/evals.md +109 -0
- package/skills/arkcli-understand/references/sub-skills.md +137 -0
- package/skills/arkcli-usage/SKILL.md +123 -0
- package/skills/arkcli-usage/references/arkcli-usage-balance.md +196 -0
- package/skills/arkcli-usage/references/arkcli-usage-plan.md +153 -0
- package/skills/arkcli-usage/references/arkcli-usage-seats.md +199 -0
- package/skills/arkcli-usage/references/arkcli-usage-stats.md +160 -0
- package/README.md +0 -3
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Models scenario recommendation table (curation layer / highest priority for intent ranking)
|
|
2
|
+
|
|
3
|
+
> **Prerequisite:** Read [`../arkcli-shared/SKILL.md`](../../arkcli-shared/SKILL.md) first to understand authentication, global parameters, and safety rules.
|
|
4
|
+
> **Snapshot alignment:** 2026-06. The "Version" column is a **curated snapshot**, not the source of truth. **Always use the complete model ID returned in real time by `arkcli models search`**. See the JOIN protocol below.
|
|
5
|
+
|
|
6
|
+
## What this table solves
|
|
7
|
+
|
|
8
|
+
Model search is not a recall problem; it is a ranking problem. The local cache contains **all models**, so every query has 100% recall. The difficult part is **accurate ranking**. Naming heuristics (generation / tier / update_time) can only guess capabilities from model names; they cannot determine whether "this model is suitable for translation or Agent tasks". What is missing is a **labeling system**.
|
|
9
|
+
|
|
10
|
+
This table is that labeling system: a manually curated `scenario label → recommended model` mapping that fixes the "intent → model" step. It is the **highest-priority signal for intent ranking**, above naming heuristics, but recommendations must be verified against the left table (local cache) before adoption.
|
|
11
|
+
|
|
12
|
+
## Usage rules
|
|
13
|
+
|
|
14
|
+
- The Agent recalls recommended models based on "scenario label + recommended model description".
|
|
15
|
+
- Recommended model descriptions include version differences, common requirement terms, legacy model aliases, and upgrade guidance to improve retrieval.
|
|
16
|
+
- When the user explicitly specifies a third-party, open-source, or historical model, return to `arkcli models search` and **do not forcibly replace it**.
|
|
17
|
+
- This table serves only "find a model by intent" (the search path). **Enumeration / inventory / statistics use `arkcli models list` without this table's reranking.**
|
|
18
|
+
|
|
19
|
+
## Model recommendation table
|
|
20
|
+
|
|
21
|
+
| Scenario label | Default recommended model | Version | Recommended model description |
|
|
22
|
+
| --- | --- | --- | --- |
|
|
23
|
+
| Text generation | Dola-Seed-2.1-turbo | dola-seed-2-1-turbo-260628 | General-purpose production model that balances quality, response speed, and cost. Suitable for everyday Q&A, content creation, copywriting, summarization, office assistance, and general text processing. Prefer it when the user has no special requirements or asks only for a general-purpose, default, or everyday model. |
|
|
24
|
+
| Text translation | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Suitable for multilingual translation of articles, product copy, technical materials, and everyday content, as well as polishing, rewriting, and cross-language content conversion. Recommend it when the user asks for a translation model, English polishing, or multilingual processing. |
|
|
25
|
+
| Information extraction | Dola-Seed-2.0-mini | seed-2-0-mini-260428 | Optimized for high throughput and lightweight calls. Suitable for field extraction, classification and summarization, label generation, format conversion, and batch structured processing. Prefer it when the user emphasizes high call volume, well-defined tasks, batch processing, or cost sensitivity. |
|
|
26
|
+
| Long-text understanding | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Suitable for analyzing long articles, reports, web content, and multi-turn context to perform summarization, Q&A, synthesis, and key-information extraction. Recommend it for long-form reading, long context, or report summarization. |
|
|
27
|
+
| Document understanding | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Suitable for reading documents, analyzing materials, summarizing reports, and answering questions based on file contents. Applicable to office work, research, and knowledge organization. Recommend it for PDF, file parsing, or document Q&A. |
|
|
28
|
+
| Complex reasoning | Dola-Seed-2.0-pro | seed-2-0-pro-260328 | Optimized for complex workflows and long-chain reasoning. Suitable for multi-step analysis, complex instructions, multi-constraint tasks, solution comparison, mathematical logic, and research assistance. Prefer it when the user prioritizes quality, high complexity, or strong reasoning. |
|
|
29
|
+
| Agent tasks | Dola-Seed-2.0-pro | seed-2-0-pro-260328 | Suitable for agents, automated tasks, and complex workflows; handles task decomposition, multi-turn planning, continuous decision-making, and long-chain execution. Prefer it for Agents, intelligent agents, complex task orchestration, or automated execution. |
|
|
30
|
+
| Tool calling | Dola-Seed-2.0-pro | seed-2-0-pro-260328 | Suitable for information queries, business operations, process automation, and complex tool orchestration through Function Calling, MCP, or external APIs. Recommend it when the user explicitly requires tools, MCP, function calling, or API orchestration. |
|
|
31
|
+
| Low-cost calls | Dola-Seed-2.0-mini | seed-2-0-mini-260428 | Optimized for inference throughput and deployment density. Suitable for budget-sensitive lightweight tasks such as classification, extraction, rewriting, format conversion, label generation, and batch content processing. Recommend it when the user asks for inexpensive, cost-saving, or low-cost options. |
|
|
32
|
+
| High-concurrency calls | Dola-Seed-2.0-mini | seed-2-0-mini-260428 | Optimized for high concurrency and batch generation. Suitable for quick responses, real-time classification, short-text processing, large-scale offline tasks, and high-frequency calls. Recommend it for high throughput, short waits, or batch processing. |
|
|
33
|
+
| Code generation | Dola-Seed-2.0-Code | seed-2-0-code-preview-260328 | Optimized for enterprise programming tasks. Suitable for code generation, completion, explanation, debugging, bug fixes, frontend development, multi-file changes, and project-level development. Prefer it for coding tasks, development agents, or repository-level changes. |
|
|
34
|
+
| Vision understanding | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Seed2.0 Lite natively supports unified understanding of text, images, audio, and video. Suitable for image Q&A, screenshot analysis, chart interpretation, visual information extraction, and mixed image-text input. Prefer it when the user asks to view or recognize images, analyze charts, or use a general multimodal model. |
|
|
35
|
+
| Complex visual reasoning | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Seed2.0 Lite supports cross-modal reasoning and can combine image, video, audio, and text information for complex analysis. Suitable for multi-image understanding, complex charts, visual logic reasoning, image-text integration, and cross-modal Q&A. |
|
|
36
|
+
| Video understanding | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Seed2.0 Lite enhances temporal video understanding and motion perception and can combine video frames, speech, and ambient sound for end-to-end audiovisual understanding. Suitable for video summarization, key-event analysis, action understanding, video Q&A, and extracting audio information from videos. |
|
|
37
|
+
| Audio understanding | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Seed2.0 Lite supports native audio input and understanding. It can analyze recordings, podcasts, meetings, and interviews for summarization, information extraction, and Q&A. Suitable for tasks that must combine speech, ambient sound, and context. |
|
|
38
|
+
| GUI Agent | Dola-Seed-2.0-lite | seed-2-0-lite-260428 | Seed2.0 Lite has upgraded GUI capabilities and is suitable for visual understanding, element recognition, visual localization, click execution, and workflow automation on web, desktop, and app interfaces. When the user mentions GUI Agent, UI-TARS, ui tars, Browser Use, Computer Use, or page automation, route to Seed2.0 Lite by default; query older models only when compatibility with a legacy integration is required. |
|
|
39
|
+
| Image generation | Dola-Seedream-5.0-pro | dola-seedream-5-0-pro-260628 | Next-generation image generation model supporting deep thinking and web search, with enhanced knowledge reasoning, instruction understanding, generation quality, editing responsiveness, and consistency. Suitable for posters, e-commerce materials, brand visuals, illustrations, concept art, and professional creative content. When the user asks about Seedream 4.0, Seedream 4.5, or an older image model without requiring legacy compatibility, prefer Seedream 5.0. |
|
|
40
|
+
| Knowledge-based image generation | Dola-Seedream-5.0-pro | dola-seedream-5-0-pro-260628 | Supports deep thinking and web search, making it suitable for generation that combines knowledge, real-time content, and complex instructions. Applicable to infographics, knowledge posters, professional design assets, current-affairs visualization, and image creation requiring factual understanding. |
|
|
41
|
+
| Image editing | Dola-Seedream-5.0-pro | dola-seedream-5-0-pro-260628 | Further improves editing responsiveness and consistency compared with Seedream 4.5. Suitable for local modifications, adding/removing elements, reference-image editing, style adjustments, image optimization, and subject consistency. When the user mentions Seedream 4.5, Seededit, or older image editing, route to Seedream 5.0 by default. |
|
|
42
|
+
| Multiple image references | Dola-Seedream-5.0-pro | dola-seedream-5-0-pro-260628 | Suitable for generation and editing with multiple reference images, including person consistency, product consistency, multi-image fusion, series assets, and reference-image transformations. Recommend it for multi-image editing, subject preservation, or series image generation. |
|
|
43
|
+
| Text rendering in images | Dola-Seedream-5.0-pro | dola-seedream-5-0-pro-260628 | Suitable for generating text in images, producing visual assets with copy, and creating knowledge-based images. Applicable to posters, covers, marketing images, infographics, and layout design. When the user mentions Seedream 4.5 text-rendering capabilities, prefer the newer Seedream 5.0. |
|
|
44
|
+
| Video generation | Dreamina-Seedance-2.0 | dreamina-seedance-2-0-260128 | Next-generation joint audio-video generation model supporting mixed text, image, audio, and video input and references for frames, actions, camera movements, visual effects, and sound. Suitable for ads, short films, creative videos, complex actions, video editing, and video extension. When the user mentions Seedance 1.5, SD1.5, Seedance 1.0, or an older video model, route to Seedance 2.0 by default and explain that it is a comprehensive upgrade. |
|
|
45
|
+
| Fast video generation | Dreamina-Seedance-2.0-fast | dreamina-seedance-2-0-fast-260128 | Fast version of Seedance 2.0, suitable for quickly generating video previews, batch-producing short videos, and high-frequency iteration. Recommend it when the user emphasizes fast output, reduced waiting, batch generation, or previewing results first. |
|
|
46
|
+
| Audiovisual video generation | Dreamina-Seedance-2.0 | dreamina-seedance-2-0-260128 | Uses a unified multimodal joint audio-video generation architecture and supports text, image, audio, and video input. Suitable for video creation with coordinated native soundtracks, dialogue, ambient sound, action, and camera work. Applicable to short dramas, ads, music videos, lip sync, and multimodal reference generation. |
|
|
47
|
+
| 3D generation | Hyper3D-Gen2 | hyper3d-gen2-260112 | Generates high-precision 3D assets from text/images. |
|
|
48
|
+
| Multimodal embeddings | skylark-embedding-vision-250615 | skylark-embedding-vision-251215 | Suitable for converting text, images, and videos into vector representations for semantic retrieval, similar-content recall, cross-modal search, recommendation systems, and multimodal knowledge indexing. Recommend it for embeddings, vector models, image search, or video retrieval. |
|
|
49
|
+
|
|
50
|
+
## JOIN / verification protocol (scenario table ⨝ full local cache)
|
|
51
|
+
|
|
52
|
+
**Core principle: the scenario table supplies intent, while `arkcli models search` supplies facts. Versions/full IDs in the table are only starting points; use the real-time search response for matches.**
|
|
53
|
+
|
|
54
|
+
**Layer boundary (do not mix layers): this table is the "model marketplace selection layer". Its data source is always `models search` (catalog), and it provides catalog-recommended model names.** "Whether the current profile can call it and the exact callable ID" belongs to the downstream availability layer (`resources list` → `models get` → `+gen`/`+chat`; arkcli-gen is implemented) and is **outside this table's responsibility**. Pure selection questions ("find/choose a model") use only `models search`; do not query `resources list`. Descend to resources list to resolve the current profile's actual ID only when the user truly wants to generate/invoke. Example: image-generation selection returns `dola-seedream-5-0` (catalog name); a plan profile may actually invoke `dola-seedream-5-0-pro`, which downstream `+gen` resolves.
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
① User intent → match scenario label (use requirement terms in "Recommended model description" to improve matching)
|
|
58
|
+
② Take the row's "Default recommended model" and derive the family stem (for example, Dola-Seed-2.0-lite → seed-2-0-lite)
|
|
59
|
+
③ arkcli models search <family stem> ← verify facts against the left table (required; cannot be skipped)
|
|
60
|
+
├─ exact family matches and version is consistent → concatenate <name>-<primary_version> and rank it first
|
|
61
|
+
├─ drift family matches but version differs → use the actual version returned by search (already sorted by gen/time; take the top)
|
|
62
|
+
│ and say "recommendation snapshot X, current Y"
|
|
63
|
+
├─ bare match with primary_version=None (voice/audio) → stop at marketplace discovery: only state that it is discoverable; do not concatenate a version or pass it downstream
|
|
64
|
+
└─ miss no match for the family stem → table ID may be stale; search again using a scenario term or shorter stem
|
|
65
|
+
④ Add 2–3 alternative candidates from search results with the same modality
|
|
66
|
+
⑤ User names a third-party/open-source/historical model (Seedream 4.5, glm…) → skip this table, search directly, and do not forcibly replace it
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Key boundaries (derived from actual testing; must read)
|
|
70
|
+
|
|
71
|
+
- **Do not pass the table's full ID directly to `+chat` / `+gen`, and do not add suffixes to model names from memory**. The actual image-generation model is `dola-seedream-5-0` (**without the `-lite` suffix; no variant with `-lite` exists in the cache, and an exact query returns no items**). Such naming drift recurs. **Search by family stem, always trust the returned result, and do not append suffixes such as `-lite` / `-pro` yourself.**
|
|
72
|
+
- **Search tolerates DisplayName**: `Dola-Seed-2.0-lite` (dots and case included) also matches `seed-2-0-lite`, but the **family stem (lowercase with hyphens) is the most reliable** and should be the preferred search keyword.
|
|
73
|
+
- **The six voice/audio entries follow the bare path and currently support only marketplace discovery in arkcli**: they appear in the models catalog but have `primary_version=None` and no enrichment (ctx/lifecycle are empty). After a match, state directly that the bare name is discoverable in the marketplace, but do not try to concatenate a version number and do not pass it to `+chat` / `+gen` / `+deploy` / `usage` / `pricing` / `onboard`. For arkcli, marketplace discovery does not mean callable, Endpoint-creatable, price-queryable, or usage-queryable.
|
|
74
|
+
- **Avoid `-internal` variants**: search may return internal variants such as `seed-2-0-lite-internal`; do not recommend them to users.
|
|
75
|
+
- **Lifecycle**: when a candidate is `Retiring`, proactively tell the user it is being retired and find a newer version in the same family (consistent with the main SKILL).
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
# models search
|
|
2
|
+
|
|
3
|
+
> **Prerequisite:** Read [`../arkcli-shared/SKILL.md`](../../arkcli-shared/SKILL.md) first to understand authentication, global parameters, and safety rules.
|
|
4
|
+
|
|
5
|
+
**Agent-first** model search: recall = full catalog; enrichment = ArkModels (context window / modalities / capabilities); filtering = structured conditions; ranking = four buckets + context_window weighting + recency. There is **no concept of pagination**, and all matches are returned by default.
|
|
6
|
+
|
|
7
|
+
## Commands
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
# Basic
|
|
11
|
+
arkcli models search # All entries sorted by UpdateTime descending
|
|
12
|
+
arkcli models search doubao # Keyword (or use --keyword doubao)
|
|
13
|
+
arkcli models search doubao --size 5 # Explicitly truncate to five entries
|
|
14
|
+
|
|
15
|
+
# Modality filtering (required for D3)
|
|
16
|
+
arkcli models search --modality video # Video generation models
|
|
17
|
+
arkcli models search --modality text # Text output models
|
|
18
|
+
arkcli models search --modality image # Image generation models
|
|
19
|
+
arkcli models search --input-modality text,image --output-modality text # VQA category
|
|
20
|
+
arkcli models search --multimodal # Models whose input supports multiple modalities
|
|
21
|
+
|
|
22
|
+
# Numeric capacity filtering
|
|
23
|
+
arkcli models search --min-context-window 200000
|
|
24
|
+
arkcli models search --max-context-window 1000000
|
|
25
|
+
arkcli models search --min-input-tokens 100000
|
|
26
|
+
arkcli models search --min-output-tokens 8192
|
|
27
|
+
|
|
28
|
+
# Capability filtering (repeatable, AND relationship)
|
|
29
|
+
arkcli models search --capability thinking
|
|
30
|
+
arkcli models search --capability mcp --capability functioncall
|
|
31
|
+
|
|
32
|
+
# Composite query: find the strongest thinking LLM with 200K+ context
|
|
33
|
+
arkcli models search --modality text --min-context-window 200000 --capability thinking --strict-filter
|
|
34
|
+
|
|
35
|
+
# Cache control
|
|
36
|
+
arkcli models search --refresh-cache # Force one synchronous refresh of ArkModels metadata
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## Parameters
|
|
40
|
+
|
|
41
|
+
| Parameter | Type | Description |
|
|
42
|
+
|------|------|------|
|
|
43
|
+
| `[keyword]` / `--keyword` | string | Keyword. Matches any name / display_name / short_name / description / introduction field (case-insensitive) |
|
|
44
|
+
| `--size` | int | **Omitted = no truncation** (the Agent receives all entries by default); explicitly pass N to truncate to N |
|
|
45
|
+
| `--modality` | string | Coarse-grained alias: `text` / `image` / `video` / `audio` (output modality shorthand) |
|
|
46
|
+
| `--input-modality` | string | Explicit input-modality filter, comma-separated (such as `text,image,video`) |
|
|
47
|
+
| `--output-modality` | string | Explicit output-modality filter |
|
|
48
|
+
| `--multimodal` | bool | Return only models with at least two input modalities |
|
|
49
|
+
| `--min-context-window` | int | Minimum context_window (tokens) |
|
|
50
|
+
| `--max-context-window` | int | Maximum context_window |
|
|
51
|
+
| `--min-input-tokens` | int | Minimum max_input_tokens |
|
|
52
|
+
| `--min-output-tokens` | int | Minimum max_completion_tokens |
|
|
53
|
+
| `--capability` | string (repeatable) | Required capabilities; multiple values use AND: `thinking` / `mcp` / `functioncall` / `web_browsing` / `knowledge_base` / `caching` / `response_format` / `reasoning_effort` |
|
|
54
|
+
| `--strict-filter` | bool | Directly exclude models without enrichment data (retained by default to avoid false negatives) |
|
|
55
|
+
| `--include-deprecated` | bool | Include models with `lifecycle_status=Shutdown` or `display_name` containing "deprecated/offline" (**filtered by default**, retaining only callable models) |
|
|
56
|
+
| `--refresh-cache` | bool | Synchronously refresh the ArkModels metadata cache (stale-while-revalidate asynchronous refresh by default) |
|
|
57
|
+
|
|
58
|
+
## Workflow
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
search [keyword] [filters]
|
|
62
|
+
│
|
|
63
|
+
1️⃣ ListFoundationModels(all, sort=UpdateTime DESC)
|
|
64
|
+
│ ↓ 152 candidates
|
|
65
|
+
2️⃣ EnrichWithMetadata
|
|
66
|
+
│ ├─ read cache: ~/.arkcli-bp/cache/<profile>/<region>/<project>/arkmodels-meta.json
|
|
67
|
+
│ ├─ hit → return immediately + fork detached subprocess for asynchronous refresh
|
|
68
|
+
│ └─ miss → synchronously call ArkModels(IsPrimaryVersionOnly:true) → write cache
|
|
69
|
+
│ (without SSO, enrichment is silently skipped and filtering automatically becomes a no-op)
|
|
70
|
+
│ ↓ add context_window / input_modalities / output_modalities / capabilities to each item
|
|
71
|
+
3️⃣ Modality fallback: derive from task_types when output_modalities is empty
|
|
72
|
+
│ (TextToVideo → out=[video], VisualQA → in=[text,image] out=[text], ...)
|
|
73
|
+
4️⃣ Keyword filtering (case-insensitive substring across name+display+short+description+intro)
|
|
74
|
+
5️⃣ Structured filtering (modality / context / tokens / capability, AND)
|
|
75
|
+
6️⃣ Four-bucket reranking + (context_window desc, update_time desc, name asc) tie-break
|
|
76
|
+
7️⃣ --size N truncation (no truncation by default)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Return fields
|
|
80
|
+
|
|
81
|
+
Each item:
|
|
82
|
+
|
|
83
|
+
```json
|
|
84
|
+
{
|
|
85
|
+
"name": "seed-2-0-pro",
|
|
86
|
+
"display_name": "Dola-Seed-2.0-pro",
|
|
87
|
+
"primary_version": "260328",
|
|
88
|
+
"update_time": "2026-03-28T...",
|
|
89
|
+
"foundation_model_tag": { "filter_task_types": [...], "task_types": [...] },
|
|
90
|
+
|
|
91
|
+
// —— enrichment (from ArkModels; null when SSO fails) ——
|
|
92
|
+
"context_window": 262144,
|
|
93
|
+
"max_input_tokens": 229376,
|
|
94
|
+
"max_completion_tokens": 65536,
|
|
95
|
+
"input_modalities": ["text", "image", "video"],
|
|
96
|
+
"output_modalities": ["text"],
|
|
97
|
+
"capabilities": {
|
|
98
|
+
"thinking": true, "mcp": true, "functioncall": true,
|
|
99
|
+
"web_browsing": true, "knowledge_base": true, "caching": true,
|
|
100
|
+
"response_format": false, "reasoning_effort": true
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Important behavior
|
|
106
|
+
|
|
107
|
+
### Handling missing data (StrictFilter)
|
|
108
|
+
|
|
109
|
+
- **Default `--strict-filter=false`**: with `--modality video`, models **without** modality data are also **retained** (the Agent sees candidates with low confidence).
|
|
110
|
+
- **`--strict-filter`**: missing data is treated as not satisfying the condition and is excluded directly (100% of the Agent's results are ground truth).
|
|
111
|
+
|
|
112
|
+
Modality queries are normally more accurate with `--strict-filter`; numeric queries (context window) are usually more reasonable with the default behavior (do not eliminate missing data).
|
|
113
|
+
|
|
114
|
+
### Recall no longer has a top-K limit
|
|
115
|
+
|
|
116
|
+
Unlike the old fuzzy API's default of nine entries, this command returns **all matches** by default. To limit results, pass `--size N`.
|
|
117
|
+
|
|
118
|
+
### Cache and SSO
|
|
119
|
+
|
|
120
|
+
- ArkModels is a Console BFF API and requires SSO credentials.
|
|
121
|
+
- Without SSO (AK/SK only) → enrichment is silently skipped, models are still returned, and filtering automatically becomes a no-op.
|
|
122
|
+
- Cache paths are separated by `(identity/account, profile, region, project)`. Persistent caching is disabled when no authoritative identity is available, preventing cross-account reuse.
|
|
123
|
+
- The default TTL is 5 minutes and can be adjusted with `ARKCLI_MODEL_CACHE_TTL`. Reads within the TTL use local cache only.
|
|
124
|
+
- After TTL expiry, stale-while-revalidate serves the current snapshot and starts a detached refresh (`arkcli models _refresh-cache`). Cold fills and refreshes are single-flighted across local processes for the same scope.
|
|
125
|
+
- `auth logout` and a fresh account switch remove all model caches for the current product.
|
|
126
|
+
- Every ArkModels Action passes through an account-scoped transport limiter with a 250ms minimum interval (about 4 QPS, leaving headroom below the backend's 5 QPS limit).
|
|
127
|
+
|
|
128
|
+
### Keyword matching scope
|
|
129
|
+
|
|
130
|
+
- name / display_name / short_name (match → reranking bucket 1)
|
|
131
|
+
- description / display_description / introduction (match → bucket 2)
|
|
132
|
+
- Substring matching, **case-insensitive**
|
|
133
|
+
- **No semantic/synonym expansion**
|
|
134
|
+
|
|
135
|
+
### Reranking details
|
|
136
|
+
|
|
137
|
+
| Bucket | Condition |
|
|
138
|
+
|---|------|
|
|
139
|
+
| 1 | Non-hidden + name matches keyword |
|
|
140
|
+
| 2 | Non-hidden + description matches keyword |
|
|
141
|
+
| 3 | Non-hidden + no keyword match (fallback) |
|
|
142
|
+
| 4 | Hidden (`customized_tags` contains `experience hiding`/`inference hiding`/`Square Hide`; these are legacy hidden-page tags from BytePlus ModelArk and are unrelated to this CLI command) |
|
|
143
|
+
|
|
144
|
+
Within a bucket: larger `context_window` first → newer `update_time` first → `name` lexicographically.
|
|
145
|
+
|
|
146
|
+
## Common errors
|
|
147
|
+
|
|
148
|
+
| Error | Cause | Handling |
|
|
149
|
+
|------|------|------|
|
|
150
|
+
| `--modality video` returns substantial noise | strict-filter=false by default, so models with missing data were not excluded | Add `--strict-filter` |
|
|
151
|
+
| Unexpected results after numeric filtering | Some ArkModels entries lack context_window | Accept the default retention (items with ctx=null may be inaccurate) |
|
|
152
|
+
| No thinking models | --capability is non-strict by default and results may be overwhelmed by noise | Add `--strict-filter` |
|
|
153
|
+
| All enrichment fields are null | Not logged in through SSO | Run `arkcli auth login` to re-establish BytePlus identity |
|
|
154
|
+
| Recall is clearly lower than expected | Cache is old, or ArkModels may be temporarily unavailable | Add `--refresh-cache` to force a synchronous refresh |
|
|
155
|
+
|
|
156
|
+
## Division of responsibility with `models list` / `models get`
|
|
157
|
+
|
|
158
|
+
| Use case | Recommendation |
|
|
159
|
+
|------|------|
|
|
160
|
+
| User provides a fuzzy keyword and wants candidates | **`search`** ✓ |
|
|
161
|
+
| Find the most recently released models (time-sensitive) | **`search`** (results sorted by update_time DESC) |
|
|
162
|
+
| Find by modality / context / capability | **`search`** + corresponding flag |
|
|
163
|
+
| Fully enumerate all models | `search` without a keyword (all 152 entries) |
|
|
164
|
+
| Exact match by name | `list --name foo` (exact) or `search foo` (includes fuzzy matching) |
|
|
165
|
+
| Get complete details for one model (pricing, rate limits, detailed capability descriptions) | `get <name>` |
|
|
166
|
+
|
|
167
|
+
## References
|
|
168
|
+
|
|
169
|
+
- [arkcli-models](../SKILL.md) — All models commands
|
|
170
|
+
- [arkcli-shared](../../arkcli-shared/SKILL.md) — Authentication and global parameters
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# arkcli-models evals
|
|
2
|
+
|
|
3
|
+
## Coverage goals
|
|
4
|
+
|
|
5
|
+
- Model search requests should use `arkcli models search`; avoid using `list` for capability recommendations.
|
|
6
|
+
- Model asset inventory requests should use `arkcli models list --page-all`; avoid jumping to Raw API Explorer.
|
|
7
|
+
- "Custom models from the past N days" should be filtered in local JSON by `create_time` and custom-model fields, without asking the user to use the console.
|
|
8
|
+
|
|
9
|
+
## Trigger / should trigger
|
|
10
|
+
|
|
11
|
+
- Trigger `arkcli-models` when the user explicitly asks for a model list, model details, model search, or model statistics.
|
|
12
|
+
- Trigger the `models list` path when the user asks for asset inventory such as "mine/custom/recently created".
|
|
13
|
+
|
|
14
|
+
## Anti-trigger / should not trigger
|
|
15
|
+
|
|
16
|
+
- When the user wants to chat, generate, or deploy directly, do not treat `arkcli-models` as the destination. Enter it briefly only when the model name is missing.
|
|
17
|
+
- When the user explicitly wants a Raw API Action, OpenAPI list, or low-level params, do not use this skill; route to `arkcli-api-explorer`.
|
|
18
|
+
|
|
19
|
+
## Guard
|
|
20
|
+
|
|
21
|
+
- When authentication fails, a 401 occurs, or login state is unknown, first check `arkcli auth status`.
|
|
22
|
+
- When profile, region, or base URL is abnormal, route to `arkcli-config` for configuration troubleshooting.
|
|
23
|
+
- A missing server-side time-filter flag is not a failure condition. First filter local JSON and explain the limitation only after confirming the request cannot be completed.
|
|
24
|
+
|
|
25
|
+
## Tested happy-path CLI commands
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
arkcli models list --page-all --sort-by CreateTime --sort-order Desc --format json
|
|
29
|
+
arkcli models search --modality text --min-context-window 200000 --capability thinking --strict-filter
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
## Regression cases
|
|
33
|
+
|
|
34
|
+
| case | prompt | Expected |
|
|
35
|
+
|------|--------|------|
|
|
36
|
+
| `uat-models-list-recent-custom-001` | How many custom models did I create in BytePlus in the past seven days? List them. | Use `arkcli models list --page-all --format json`, and locally filter `create_time` and Custom fields; forbid `arkcli api --list` and do not suggest using the console |
|
|
37
|
+
| `models-list-owned-custom` | Help me see which custom models I have, and output the count and model names. | Use `arkcli models list` for asset enumeration, then perform client-side filtering and statistics |
|
|
38
|
+
| `models-search-task-fit` | Help me find a text model that supports thinking and has more than 200K context for +chat. | Use `arkcli models search --min-context-window ... --capability thinking` |
|
|
39
|
+
| `models-search-speech-boundary` | Does the BytePlus ModelArk marketplace have TTS models? Can I deploy one directly? | Use `arkcli models search <tts/voice keyword>` for marketplace discovery; clearly state that voice models currently do not support `+deploy` / usage / pricing / onboard; do not recommend `+chat` or `+gen` |
|
|
40
|
+
|
|
41
|
+
## Model detail pricing migration cases
|
|
42
|
+
|
|
43
|
+
| case | prompt | Expected |
|
|
44
|
+
|------|--------|----------|
|
|
45
|
+
| `models-get-dimensional-price` | What is the input price for regular inference on this known model? Are there context tiers? | Read the get reference and use `pricing.prices`; explain service, label, dimensions, and units; never read old charge_items or select the first row |
|
|
46
|
+
| `models-get-missing-price` | Does a null price or empty array mean free? | Distinguish null, empty arrays, and zero; never infer free, inactive, or nonexistent status or fall back to old prices |
|
|
47
|
+
| `models-get-entitlements` | Show the price, activation state, and remaining free quota. | Read new prices from `pricing.prices`, state and quota from existing `pricing.state` / `pricing.inference_free_usage`; do not infer one from the other |
|
|
48
|
+
| `models-get-price-scope` | Is this a price for the specified version and region? | Explain foundation-model-name and global query scope; do not infer version-specific or region-specific rates |
|
|
49
|
+
|
|
50
|
+
## Key scoring points
|
|
51
|
+
|
|
52
|
+
- Must route to `arkcli-models`.
|
|
53
|
+
- Asset inventory must recommend `arkcli models list`, preferably with `--page-all` and `--format json`.
|
|
54
|
+
- Time filtering must explain client-side filtering or local JSON processing.
|
|
55
|
+
- Must not recommend `arkcli api --list` as the preferred path.
|
|
56
|
+
- Must not claim that the CLI lacks the capability or send the user to the console merely because a server-side time flag is missing.
|
|
57
|
+
- After a voice-model match, must stop at the marketplace discovery layer and must not continue recommending deployment, usage, or pricing queries.
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: arkcli-onboard
|
|
3
|
+
version: 0.2.0
|
|
4
|
+
description: "Guide a user through production integration of a BytePlus Ark model: authenticate, verify the account gate, select a deployable model, and reuse a matching running inference Endpoint or create one safely. Trigger for intent-level requests such as integrating a model into an application or service when the user has not already asked for a specific deployment command."
|
|
5
|
+
metadata:
|
|
6
|
+
requires:
|
|
7
|
+
bins: ["arkcli"]
|
|
8
|
+
cliHelp: "arkcli --help"
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# BytePlus Ark production onboarding
|
|
12
|
+
|
|
13
|
+
<!-- Future BytePlus code-example support:
|
|
14
|
+
Restore optional code-example generation after Endpoint creation and include
|
|
15
|
+
the generated example location in the completion contract.
|
|
16
|
+
-->
|
|
17
|
+
|
|
18
|
+
Before using this Skill, read
|
|
19
|
+
[`../arkcli-shared/SKILL.md`](../arkcli-shared/SKILL.md). This is an
|
|
20
|
+
orchestration Skill: it owns the order and branching between capabilities, not
|
|
21
|
+
the private flags or response schemas of each delegated command.
|
|
22
|
+
|
|
23
|
+
## When To Trigger
|
|
24
|
+
|
|
25
|
+
- "Integrate a BytePlus Ark model into my application."
|
|
26
|
+
- "Help my backend call this model in production."
|
|
27
|
+
- "I selected a model; help me obtain a reusable Endpoint"
|
|
28
|
+
- Similar intent-level production-integration requests that do not already
|
|
29
|
+
specify an exact deployment command.
|
|
30
|
+
|
|
31
|
+
## When NOT To Trigger
|
|
32
|
+
|
|
33
|
+
- The user explicitly asks to deploy or create an Endpoint: route directly to
|
|
34
|
+
[`../arkcli-deploy/SKILL.md`](../arkcli-deploy/SKILL.md).
|
|
35
|
+
- The user only wants a one-off trial, answer, image, or video: route to
|
|
36
|
+
[`../arkcli-chat/SKILL.md`](../arkcli-chat/SKILL.md) or
|
|
37
|
+
[`../arkcli-gen/SKILL.md`](../arkcli-gen/SKILL.md).
|
|
38
|
+
- The owning model Skill classifies the target as discoverable but not
|
|
39
|
+
deployable: stop after model discovery and explain the Ark CLI capability
|
|
40
|
+
boundary.
|
|
41
|
+
|
|
42
|
+
## Execution modes
|
|
43
|
+
|
|
44
|
+
### Normal workflow
|
|
45
|
+
|
|
46
|
+
Use remote read operations as needed, then ask for confirmation at the single
|
|
47
|
+
write boundary.
|
|
48
|
+
|
|
49
|
+
### Help-only or no-remote workflow
|
|
50
|
+
|
|
51
|
+
- If the user supplies a model and explicitly forbids remote queries, do not
|
|
52
|
+
run model search or Endpoint listing.
|
|
53
|
+
- To validate command selection without remote calls, use help only:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
arkcli models --help
|
|
57
|
+
arkcli infer endpoint list --mine --help
|
|
58
|
+
arkcli +deploy --help
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
- Help-only validation must not log in, activate a model, create an Endpoint,
|
|
62
|
+
or query account resources.
|
|
63
|
+
|
|
64
|
+
## Agent execution order
|
|
65
|
+
|
|
66
|
+
Use these steps in order:
|
|
67
|
+
|
|
68
|
+
1. Authenticate and apply the BytePlus account gate.
|
|
69
|
+
2. Select and confirm a deployable model.
|
|
70
|
+
3. Reuse an exact matching available Endpoint when one exists.
|
|
71
|
+
4. Otherwise, delegate guarded Endpoint creation to `arkcli-deploy`.
|
|
72
|
+
<!-- 5. Optionally generate code examples when BytePlus supports them. -->
|
|
73
|
+
5. Return the integration result.
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
Step 0 Authenticate and apply the BytePlus account gate
|
|
77
|
+
|
|
|
78
|
+
+--> arkcli auth status --format json
|
|
79
|
+
| |
|
|
80
|
+
| +--> session missing or expired: arkcli auth login, then resume
|
|
81
|
+
| +--> byteplus_sso.identity.verified == false:
|
|
82
|
+
| | stop at https://console.byteplus.com/user/basics/
|
|
83
|
+
| +--> verified == true: continue
|
|
84
|
+
| +--> verified missing: continue without claiming a result
|
|
85
|
+
|
|
|
86
|
+
Step 1 Select and confirm a deployable model
|
|
87
|
+
|
|
|
88
|
+
+--> delegate search/get to arkcli-models
|
|
89
|
+
+--> no model supplied: present candidates and ask the user to choose
|
|
90
|
+
+--> discoverable-only or unsupported target: stop
|
|
91
|
+
|
|
|
92
|
+
Step 2 Look for a reusable inference Endpoint
|
|
93
|
+
|
|
|
94
|
+
+--> arkcli infer endpoint list --mine --page-all
|
|
95
|
+
+--> match the exact model/version and an available status
|
|
96
|
+
|
|
|
97
|
+
+--> one match: reuse it
|
|
98
|
+
+--> multiple matches: ask the user which Endpoint to use
|
|
99
|
+
+--> no match: continue to Step 3
|
|
100
|
+
|
|
|
101
|
+
Step 3 Create an Endpoint
|
|
102
|
+
|
|
|
103
|
+
+--> delegate to arkcli-deploy
|
|
104
|
+
+--> use read-only checks, state that +deploy has no --dry-run, then obtain explicit confirmation
|
|
105
|
+
+--> this is the only write in the onboarding workflow
|
|
106
|
+
|
|
|
107
|
+
Step 4 Return the production integration result
|
|
108
|
+
endpoint ID + model ID + reused/created status
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Endpoint reuse rules
|
|
112
|
+
|
|
113
|
+
Reuse an Endpoint only when all of these are true:
|
|
114
|
+
|
|
115
|
+
- it belongs to the current SSO sub-user result set from `--mine`;
|
|
116
|
+
- it is bound to the exact target model and version;
|
|
117
|
+
- its status is available for inference, such as `Running`.
|
|
118
|
+
|
|
119
|
+
An Endpoint for another model is not a match. Do not silently choose among
|
|
120
|
+
multiple matches. Listing all pages is required so a reusable Endpoint is not
|
|
121
|
+
missed because of pagination.
|
|
122
|
+
|
|
123
|
+
## Write and verification guards
|
|
124
|
+
|
|
125
|
+
- This production workflow can lead to a billable write, so Step 0 applies the
|
|
126
|
+
combined BytePlus account-opening and payment-verification gate before model
|
|
127
|
+
activation or deployment.
|
|
128
|
+
- `byteplus_sso.identity.verified == false` does not reveal which underlying
|
|
129
|
+
check is incomplete. Use the single BytePlus account settings URL and do not
|
|
130
|
+
guess.
|
|
131
|
+
- A missing `verified` field is unknown, not success or failure. Continue and
|
|
132
|
+
preserve any later structured backend error and request ID.
|
|
133
|
+
- Step 3 must follow `arkcli-deploy` safety rules. Do not bypass confirmation
|
|
134
|
+
through an API Explorer action or an undocumented environment variable.
|
|
135
|
+
- Do not create a second Endpoint when Step 2 found a valid reusable one.
|
|
136
|
+
|
|
137
|
+
## Completion contract
|
|
138
|
+
|
|
139
|
+
The final response must return to the user's original integration goal and
|
|
140
|
+
include:
|
|
141
|
+
|
|
142
|
+
- the selected model ID;
|
|
143
|
+
- the Endpoint ID;
|
|
144
|
+
- whether the Endpoint was reused or created;
|
|
145
|
+
- any remaining user action, without inventing a console URL.
|
|
146
|
+
|
|
147
|
+
## Guard Checklist
|
|
148
|
+
|
|
149
|
+
- Read `arkcli-shared` before starting.
|
|
150
|
+
- Keep authentication and verification as intermediate gates, not final
|
|
151
|
+
outcomes.
|
|
152
|
+
- Delegate command-specific flags to the owning Skill.
|
|
153
|
+
- Use `--mine --page-all` for reusable Endpoint discovery.
|
|
154
|
+
- Confirm the exact model before deployment.
|
|
155
|
+
- `+deploy` has no Client Preview. Use read-only checks and explicit confirmation before the single write.
|
|
156
|
+
- Stop on a known failed BytePlus account gate.
|
|
157
|
+
- Do not turn a one-off trial into an Endpoint deployment.
|
|
158
|
+
|
|
159
|
+
## References
|
|
160
|
+
|
|
161
|
+
- [`../arkcli-auth/references/realname-gate.md`](../arkcli-auth/references/realname-gate.md)
|
|
162
|
+
- combined BytePlus account-opening and payment-verification contract
|
|
163
|
+
- [`../arkcli-models/SKILL.md`](../arkcli-models/SKILL.md) - model selection
|
|
164
|
+
- [`../arkcli-infer-endpoint/SKILL.md`](../arkcli-infer-endpoint/SKILL.md) -
|
|
165
|
+
Endpoint listing and exact-match semantics
|
|
166
|
+
- [`../arkcli-deploy/SKILL.md`](../arkcli-deploy/SKILL.md) - guarded Endpoint creation
|
|
167
|
+
|
|
168
|
+
- [`references/evals.md`](references/evals.md) - routing and workflow
|
|
169
|
+
evaluations
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# arkcli-onboard evaluations
|
|
2
|
+
|
|
3
|
+
Use these cases to verify intent routing, workflow order, exact Endpoint reuse,
|
|
4
|
+
verification behavior, and the write boundary.
|
|
5
|
+
|
|
6
|
+
## 1. Trigger: intent-level production integration
|
|
7
|
+
|
|
8
|
+
User request:
|
|
9
|
+
|
|
10
|
+
> Help me integrate a BytePlus Ark text model into my backend. I have not
|
|
11
|
+
> deployed anything yet.
|
|
12
|
+
|
|
13
|
+
Expected behavior:
|
|
14
|
+
|
|
15
|
+
- Route to `arkcli-onboard`.
|
|
16
|
+
- Run the sequence:
|
|
17
|
+
`auth status -> models search/get -> infer endpoint list --mine --page-all`.
|
|
18
|
+
- Continue to `arkcli-deploy` only when no exact reusable Endpoint exists.
|
|
19
|
+
- Keep command-specific deployment flags in `arkcli-deploy`.
|
|
20
|
+
|
|
21
|
+
## 2. Reuse one matching Endpoint
|
|
22
|
+
|
|
23
|
+
Precondition:
|
|
24
|
+
|
|
25
|
+
- One `Running` Endpoint owned by the current SSO sub-user is bound to the
|
|
26
|
+
selected model and version.
|
|
27
|
+
|
|
28
|
+
Expected behavior:
|
|
29
|
+
|
|
30
|
+
- Reuse that Endpoint ID.
|
|
31
|
+
- Do not run `+deploy`.
|
|
32
|
+
- Report that the Endpoint was reused.
|
|
33
|
+
|
|
34
|
+
## 3. Multiple matching Endpoints
|
|
35
|
+
|
|
36
|
+
Precondition:
|
|
37
|
+
|
|
38
|
+
- Two available Endpoints match the exact model and version.
|
|
39
|
+
|
|
40
|
+
Expected behavior:
|
|
41
|
+
|
|
42
|
+
- Present the matching Endpoint IDs and relevant names or statuses.
|
|
43
|
+
- Ask the user to select one.
|
|
44
|
+
- Do not silently choose and do not create another Endpoint.
|
|
45
|
+
|
|
46
|
+
## 4. Non-matching Endpoint
|
|
47
|
+
|
|
48
|
+
Precondition:
|
|
49
|
+
|
|
50
|
+
- The account has a `Running` Endpoint, but it is bound to a different model.
|
|
51
|
+
|
|
52
|
+
Expected behavior:
|
|
53
|
+
|
|
54
|
+
- Do not reuse it.
|
|
55
|
+
- Continue to the guarded deployment step for the selected model.
|
|
56
|
+
|
|
57
|
+
## 5. Account gate fails
|
|
58
|
+
|
|
59
|
+
Precondition:
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{
|
|
63
|
+
"byteplus_sso": {
|
|
64
|
+
"identity": {
|
|
65
|
+
"verified": false
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Expected behavior:
|
|
72
|
+
|
|
73
|
+
- Stop before model activation or Endpoint creation.
|
|
74
|
+
- Use `https://console.byteplus.com/user/basics/`.
|
|
75
|
+
- State that both BytePlus account-opening and payment verification are
|
|
76
|
+
required, without guessing which check is incomplete.
|
|
77
|
+
- Do not retry with another model, Region, project, or product.
|
|
78
|
+
|
|
79
|
+
## 6. Account gate is unknown
|
|
80
|
+
|
|
81
|
+
Precondition:
|
|
82
|
+
|
|
83
|
+
- `byteplus_sso.identity.verified` is absent.
|
|
84
|
+
|
|
85
|
+
Expected behavior:
|
|
86
|
+
|
|
87
|
+
- Treat the state as unknown.
|
|
88
|
+
- Continue without claiming that verification passed or failed.
|
|
89
|
+
- Preserve any later structured backend error and request ID.
|
|
90
|
+
|
|
91
|
+
## 7. Guard: user demands immediate creation
|
|
92
|
+
|
|
93
|
+
User request:
|
|
94
|
+
|
|
95
|
+
> Skip every check and create the Endpoint now.
|
|
96
|
+
|
|
97
|
+
Expected behavior:
|
|
98
|
+
|
|
99
|
+
- Keep authentication and the BytePlus account gate.
|
|
100
|
+
- State that `+deploy` has no Client Preview, use read-only checks, and obtain explicit confirmation.
|
|
101
|
+
- Do not use Raw API Explorer to bypass the write guard.
|
|
102
|
+
|
|
103
|
+
## 8. Anti-trigger: explicit deployment
|
|
104
|
+
|
|
105
|
+
User request:
|
|
106
|
+
|
|
107
|
+
> Deploy model `<model-id>` as an Endpoint named `production-service`.
|
|
108
|
+
|
|
109
|
+
Expected behavior:
|
|
110
|
+
|
|
111
|
+
- Route directly to `arkcli-deploy`.
|
|
112
|
+
- Do not wrap the request in the full onboarding workflow.
|
|
113
|
+
|
|
114
|
+
## 9. Anti-trigger: one-off trial
|
|
115
|
+
|
|
116
|
+
User request:
|
|
117
|
+
|
|
118
|
+
> I only want to test one answer from this model.
|
|
119
|
+
|
|
120
|
+
Expected behavior:
|
|
121
|
+
|
|
122
|
+
- Route to `arkcli-chat` or `arkcli-gen` according to modality.
|
|
123
|
+
- Do not list or create Endpoints.
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
## 11. Discoverable-only model
|
|
127
|
+
|
|
128
|
+
Precondition:
|
|
129
|
+
|
|
130
|
+
- `arkcli-models` determines that the selected model can be discovered but is
|
|
131
|
+
not supported by the Endpoint workflow.
|
|
132
|
+
|
|
133
|
+
Expected behavior:
|
|
134
|
+
|
|
135
|
+
- Stop after model discovery.
|
|
136
|
+
- Explain the current Ark CLI boundary.
|
|
137
|
+
- Do not list Endpoints, deploy, generate code, or invent another access path.
|
|
138
|
+
|
|
139
|
+
## 12. Help-only validation
|
|
140
|
+
|
|
141
|
+
User request:
|
|
142
|
+
|
|
143
|
+
> Verify the intended commands, but do not log in, query remote resources, or
|
|
144
|
+
> create anything.
|
|
145
|
+
|
|
146
|
+
Expected behavior:
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
arkcli models --help
|
|
150
|
+
arkcli infer endpoint list --mine --help
|
|
151
|
+
arkcli +deploy --help
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
- Do not run `auth login`, model search, Endpoint list, or deployment.
|
|
155
|
+
- Preserve `--mine` in the Endpoint help command so the intended ownership
|
|
156
|
+
scope remains explicit.
|