mlclaw 0.1.0 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. package/.agents/skills/mlclaw/SKILL.md +63 -18
  2. package/Dockerfile +21 -4
  3. package/README.md +70 -24
  4. package/assets/hf-logo.svg +8 -0
  5. package/assets/hf-tooling/manifest.json +50 -0
  6. package/assets/hf-tooling/skills/hf-cli/SKILL.md +218 -0
  7. package/assets/hf-tooling/skills/hf-mem/SKILL.md +79 -0
  8. package/assets/hf-tooling/skills/huggingface-best/SKILL.md +134 -0
  9. package/assets/hf-tooling/skills/huggingface-datasets/SKILL.md +107 -0
  10. package/assets/hf-tooling/skills/huggingface-gradio/SKILL.md +298 -0
  11. package/assets/hf-tooling/skills/huggingface-gradio/examples.md +613 -0
  12. package/assets/hf-tooling/skills/huggingface-local-models/SKILL.md +113 -0
  13. package/assets/hf-tooling/skills/huggingface-local-models/references/hardware.md +38 -0
  14. package/assets/hf-tooling/skills/huggingface-local-models/references/hub-discovery.md +178 -0
  15. package/assets/hf-tooling/skills/huggingface-local-models/references/quantization.md +256 -0
  16. package/assets/hf-tooling/skills/huggingface-papers/SKILL.md +239 -0
  17. package/assets/hf-tooling/skills/huggingface-spaces/SKILL.md +239 -0
  18. package/assets/hf-tooling/skills/huggingface-spaces/references/buckets.md +89 -0
  19. package/assets/hf-tooling/skills/huggingface-spaces/references/debugging.md +236 -0
  20. package/assets/hf-tooling/skills/huggingface-spaces/references/gradio.md +200 -0
  21. package/assets/hf-tooling/skills/huggingface-spaces/references/grants.md +56 -0
  22. package/assets/hf-tooling/skills/huggingface-spaces/references/inference-providers.md +85 -0
  23. package/assets/hf-tooling/skills/huggingface-spaces/references/known-errors.md +232 -0
  24. package/assets/hf-tooling/skills/huggingface-spaces/references/requirements.md +169 -0
  25. package/assets/hf-tooling/skills/huggingface-spaces/references/zerogpu.md +349 -0
  26. package/assets/hf-tooling/skills/huggingface-tool-builder/SKILL.md +120 -0
  27. package/assets/hf-tooling/skills/huggingface-tool-builder/references/baseline_hf_api.py +57 -0
  28. package/assets/hf-tooling/skills/huggingface-tool-builder/references/baseline_hf_api.sh +40 -0
  29. package/assets/hf-tooling/skills/huggingface-tool-builder/references/baseline_hf_api.tsx +57 -0
  30. package/assets/hf-tooling/skills/huggingface-tool-builder/references/find_models_by_paper.sh +230 -0
  31. package/assets/hf-tooling/skills/huggingface-tool-builder/references/hf_enrich_models.sh +96 -0
  32. package/assets/hf-tooling/skills/huggingface-tool-builder/references/hf_model_card_frontmatter.sh +188 -0
  33. package/assets/hf-tooling/skills/huggingface-tool-builder/references/hf_model_papers_auth.sh +171 -0
  34. package/assets/hf-tooling/skills/huggingface-zerogpu/SKILL.md +289 -0
  35. package/assets/hf-tooling/skills/huggingface-zerogpu/references/concurrency.md +79 -0
  36. package/assets/hf-tooling/skills/huggingface-zerogpu/references/cuda-and-deps.md +66 -0
  37. package/assets/hf-tooling/skills/huggingface-zerogpu/references/how-quota-works.md +74 -0
  38. package/assets/hf-tooling/skills/huggingface-zerogpu/references/how-zerogpu-works.md +50 -0
  39. package/assets/hf-tooling/templates/.agents/mcp/huggingface.json +11 -0
  40. package/assets/hf-tooling/templates/.env.example +5 -0
  41. package/assets/hf-tooling/templates/examples/huggingface/README.md +16 -0
  42. package/assets/hf-tooling/templates/examples/huggingface/bucket-sync.md +19 -0
  43. package/assets/hf-tooling/templates/examples/huggingface/dataset-upload.py +22 -0
  44. package/assets/hf-tooling/templates/examples/huggingface/hf-discover.md +13 -0
  45. package/assets/hf-tooling/templates/examples/huggingface/runtime-inspection.py +9 -0
  46. package/assets/mlclaw-control-ui/assets/index-D2TFes32.js +9 -0
  47. package/assets/mlclaw-control-ui/assets/index-DP72PFuv.css +1 -0
  48. package/assets/mlclaw-control-ui/index.html +13 -0
  49. package/assets/mlclaw.svg +298 -124
  50. package/dist/hf-tooling-seed.js +261 -0
  51. package/dist/mlclaw-space-runtime.js +6634 -3321
  52. package/dist/mlclaw.mjs +428 -178
  53. package/entrypoint.sh +4 -0
  54. package/mlclaw.ps1 +4 -2
  55. package/mlclaw.sh +5 -3
  56. package/openclaw.default.json +1 -0
  57. package/package.json +16 -2
  58. package/space/README.md +32 -2
  59. package/tsconfig.json +3 -2
@@ -0,0 +1,218 @@
1
+ ---
2
+ name: hf-cli
3
+ description: "Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`."
4
+ ---
5
+
6
+ Install: `curl -LsSf https://hf.co/cli/install.sh | bash -s`.
7
+
8
+ The Hugging Face Hub CLI tool `hf` is available. IMPORTANT: The `hf` command replaces the deprecated `huggingface-cli` command.
9
+
10
+ Use `hf --help` to view available functions. Note that auth commands are now all under `hf auth` e.g. `hf auth whoami`.
11
+
12
+ Generated with `huggingface_hub v1.19.0`. Run `hf skills add --force` to regenerate.
13
+
14
+ ## Commands
15
+
16
+ - `hf cp SRC` — Copy files between local paths, repositories, and buckets. `[--format [auto|human|agent|json|quiet]]`
17
+ - `hf download REPO_ID` — Download files from the Hub. `[--type [model|dataset|space] --revision TEXT --include TEXT --exclude TEXT --cache-dir TEXT --local-dir TEXT --force-download --dry-run --max-workers INTEGER --format [auto|human|agent|json|quiet]]`
18
+ - `hf env` — Print information about the environment. `[--format [auto|human|agent|json|quiet]]`
19
+ - `hf sync` — Sync files between local directory and a bucket. `[--delete --ignore-times --ignore-sizes --plan TEXT --apply TEXT --dry-run --include TEXT --exclude TEXT --filter-from TEXT --existing --ignore-existing --verbose --format [auto|human|agent|json|quiet]]`
20
+ - `hf update` — Update the `hf` CLI to the latest version. `[--format [auto|human|agent|json|quiet]]`
21
+ - `hf upload REPO_ID` — Upload a file or a folder to the Hub. Recommended for single-commit uploads. `[--type [model|dataset|space] --revision TEXT --private --include TEXT --exclude TEXT --delete TEXT --commit-message TEXT --commit-description TEXT --create-pr --every FLOAT --format [auto|human|agent|json|quiet]]`
22
+ - `hf upload-large-folder REPO_ID LOCAL_PATH` — Upload a large folder to the Hub. Recommended for resumable uploads. `[--type [model|dataset|space] --revision TEXT --private --include TEXT --exclude TEXT --num-workers INTEGER --no-report --no-bars --format [auto|human|agent|json|quiet]]`
23
+ - `hf version` — Print information about the hf version. `[--format [auto|human|agent|json|quiet]]`
24
+
25
+ ### `hf auth` — Manage authentication (login, logout, etc.).
26
+
27
+ - `hf auth list` — List all stored access tokens. `[--format [auto|human|agent|json|quiet]]`
28
+ - `hf auth login` — Login using a token from huggingface.co/settings/tokens. `[--add-to-git-credential --force --format [auto|human|agent|json|quiet]]`
29
+ - `hf auth logout` — Logout from a specific token. `[--token-name TEXT --format [auto|human|agent|json|quiet]]`
30
+ - `hf auth switch` — Switch between access tokens. `[--token-name TEXT --add-to-git-credential --format [auto|human|agent|json|quiet]]`
31
+ - `hf auth token` — Print the current access token to stdout. `[--format [auto|human|agent|json|quiet]]`
32
+ - `hf auth whoami` — Find out which huggingface.co account you are logged in as. `[--format [auto|human|agent|json|quiet]]`
33
+
34
+ ### `hf buckets` — Commands to interact with buckets.
35
+
36
+ - `hf buckets cp SRC` — Copy files between local paths, repositories, and buckets. `[--format [auto|human|agent|json|quiet]]`
37
+ - `hf buckets create BUCKET_ID` — Create a new bucket. `[--private --region [us|eu] --exist-ok --format [auto|human|agent|json|quiet]]`
38
+ - `hf buckets delete BUCKET_ID` — Delete a bucket. `[--yes --missing-ok --format [auto|human|agent|json|quiet]]`
39
+ - `hf buckets info BUCKET_ID` — Get info about a bucket. `[--format [auto|human|agent|json|quiet]]`
40
+ - `hf buckets list` — List buckets or files in a bucket. `[--human-readable --tree --recursive --search TEXT --format [auto|human|agent|json|quiet]]`
41
+ - `hf buckets move FROM_ID TO_ID` — Move (rename) a bucket to a new name or namespace. `[--format [auto|human|agent|json|quiet]]`
42
+ - `hf buckets remove ARGUMENT` — Remove files from a bucket. `[--recursive --yes --dry-run --include TEXT --exclude TEXT --format [auto|human|agent|json|quiet]]`
43
+ - `hf buckets sync` — Sync files between local directory and a bucket. `[--delete --ignore-times --ignore-sizes --plan TEXT --apply TEXT --dry-run --include TEXT --exclude TEXT --filter-from TEXT --existing --ignore-existing --verbose --format [auto|human|agent|json|quiet]]`
44
+
45
+ ### `hf cache` — Manage local cache directory.
46
+
47
+ - `hf cache list` — List cached repositories or revisions. `[--cache-dir TEXT --revisions --filter TEXT --sort [accessed|accessed:asc|accessed:desc|modified|modified:asc|modified:desc|name|name:asc|name:desc|size|size:asc|size:desc] --limit INTEGER --format [auto|human|agent|json|quiet]]`
48
+ - `hf cache prune` — Remove detached revisions from the cache. `[--cache-dir TEXT --yes --dry-run --format [auto|human|agent|json|quiet]]`
49
+ - `hf cache rm TARGETS` — Remove cached repositories or revisions. `[--cache-dir TEXT --yes --dry-run --format [auto|human|agent|json|quiet]]`
50
+ - `hf cache verify REPO_ID` — Verify checksums for a single repo revision from cache or a local directory. `[--type [model|dataset|space] --revision TEXT --cache-dir TEXT --local-dir TEXT --fail-on-missing-files --fail-on-extra-files --format [auto|human|agent|json|quiet]]`
51
+
52
+ ### `hf collections` — Interact with collections on the Hub.
53
+
54
+ - `hf collections add-item COLLECTION_SLUG ITEM_ID ITEM_TYPE` — Add an item to a collection. `[--note TEXT --exists-ok --format [auto|human|agent|json|quiet]]`
55
+ - `hf collections create TITLE` — Create a new collection on the Hub. `[--namespace TEXT --description TEXT --private --exists-ok --format [auto|human|agent|json|quiet]]`
56
+ - `hf collections delete COLLECTION_SLUG` — Delete a collection from the Hub. `[--missing-ok --format [auto|human|agent|json|quiet]]`
57
+ - `hf collections delete-item COLLECTION_SLUG ITEM_OBJECT_ID` — Delete an item from a collection. `[--missing-ok --format [auto|human|agent|json|quiet]]`
58
+ - `hf collections info COLLECTION_SLUG` — Get info about a collection on the Hub. `[--format [auto|human|agent|json|quiet]]`
59
+ - `hf collections list` — List collections on the Hub. `[--owner TEXT --item TEXT --sort [lastModified|trending|upvotes] --limit INTEGER --format [auto|human|agent|json|quiet]]`
60
+ - `hf collections update COLLECTION_SLUG` — Update a collection's metadata on the Hub. `[--title TEXT --description TEXT --position INTEGER --private --theme TEXT --format [auto|human|agent|json|quiet]]`
61
+ - `hf collections update-item COLLECTION_SLUG ITEM_OBJECT_ID` — Update an item in a collection. `[--note TEXT --position INTEGER --format [auto|human|agent|json|quiet]]`
62
+
63
+ ### `hf datasets` — Interact with datasets on the Hub.
64
+
65
+ - `hf datasets card DATASET_ID` — Get the dataset card (README) for a dataset on the Hub. `[--metadata --text --format [auto|human|agent|json|quiet]]`
66
+ - `hf datasets info DATASET_ID` — Get info about a dataset on the Hub. `[--revision TEXT --expand TEXT --format [auto|human|agent|json|quiet]]`
67
+ - `hf datasets leaderboard DATASET_ID` — List model scores from a dataset leaderboard. This command helps find the best models for a task or compare models by benchmark scores. Use 'hf datasets ls --filter benchmark:official' to list available leaderboards. `[--limit INTEGER --format [auto|human|agent|json|quiet]]`
68
+ - `hf datasets list` — List datasets on the Hub, or files in a dataset repo. `[--search TEXT --author TEXT --filter TEXT --sort [created_at|downloads|last_modified|likes|trending_score] --limit INTEGER --expand TEXT --human-readable --tree --recursive --revision TEXT --format [auto|human|agent|json|quiet]]`
69
+ - `hf datasets parquet DATASET_ID` — List parquet file URLs available for a dataset. `[--subset TEXT --split TEXT --format [auto|human|agent|json|quiet]]`
70
+ - `hf datasets sql SQL` — Execute a raw SQL query with DuckDB against dataset parquet URLs. `[--format [auto|human|agent|json|quiet]]`
71
+
72
+ ### `hf discussions` — Manage discussions and pull requests on the Hub.
73
+
74
+ - `hf discussions close REPO_ID NUM` — Close a discussion or pull request. `[--comment TEXT --yes --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
75
+ - `hf discussions comment REPO_ID NUM` — Comment on a discussion or pull request. `[--body TEXT --body-file PATH --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
76
+ - `hf discussions create REPO_ID --title TEXT` — Create a new discussion or pull request on a repo. `[--body TEXT --body-file PATH --pull-request --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
77
+ - `hf discussions diff REPO_ID NUM` — Show the diff of a pull request. `[--type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
78
+ - `hf discussions info REPO_ID NUM` — Get info about a discussion or pull request. `[--type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
79
+ - `hf discussions list REPO_ID` — List discussions and pull requests on a repo. `[--status [open|closed|merged|draft|all] --kind [all|discussion|pull_request] --author TEXT --limit INTEGER --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
80
+ - `hf discussions merge REPO_ID NUM` — Merge a pull request. `[--comment TEXT --yes --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
81
+ - `hf discussions rename REPO_ID NUM NEW_TITLE` — Rename a discussion or pull request. `[--type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
82
+ - `hf discussions reopen REPO_ID NUM` — Reopen a closed discussion or pull request. `[--comment TEXT --yes --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
83
+
84
+ ### `hf endpoints` — Manage Hugging Face Inference Endpoints.
85
+
86
+ - `hf endpoints catalog deploy --repo TEXT` — Deploy an Inference Endpoint from the Model Catalog. `[--name TEXT --accelerator TEXT --namespace TEXT --format [auto|human|agent|json|quiet]]`
87
+ - `hf endpoints catalog list` — List available Catalog models. `[--format [auto|human|agent|json|quiet]]`
88
+ - `hf endpoints delete NAME` — Delete an Inference Endpoint permanently. `[--namespace TEXT --yes --format [auto|human|agent|json|quiet]]`
89
+ - `hf endpoints deploy NAME --repo TEXT --framework TEXT --accelerator TEXT --instance-size TEXT --instance-type TEXT --region TEXT --vendor TEXT` — Deploy an Inference Endpoint from a Hub repository. `[--namespace TEXT --task TEXT --min-replica INTEGER --max-replica INTEGER --scale-to-zero-timeout INTEGER --scaling-metric [pendingRequests|hardwareUsage] --scaling-threshold FLOAT --format [auto|human|agent|json|quiet]]`
90
+ - `hf endpoints describe NAME` — Get information about an existing endpoint. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
91
+ - `hf endpoints list` — Lists all Inference Endpoints for the given namespace. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
92
+ - `hf endpoints pause NAME` — Pause an Inference Endpoint. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
93
+ - `hf endpoints resume NAME` — Resume an Inference Endpoint. `[--namespace TEXT --fail-if-already-running --format [auto|human|agent|json|quiet]]`
94
+ - `hf endpoints scale-to-zero NAME` — Scale an Inference Endpoint to zero. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
95
+ - `hf endpoints update NAME` — Update an existing endpoint. `[--namespace TEXT --repo TEXT --accelerator TEXT --instance-size TEXT --instance-type TEXT --framework TEXT --revision TEXT --task TEXT --min-replica INTEGER --max-replica INTEGER --scale-to-zero-timeout INTEGER --scaling-metric [pendingRequests|hardwareUsage] --scaling-threshold FLOAT --format [auto|human|agent|json|quiet]]`
96
+
97
+ ### `hf extensions` — Manage hf CLI extensions.
98
+
99
+ - `hf extensions exec NAME` — Execute an installed extension.
100
+ - `hf extensions install REPO_ID` — Install an extension from a public GitHub repository. `[--force --format [auto|human|agent|json|quiet]]`
101
+ - `hf extensions list` — List installed extension commands. `[--format [auto|human|agent|json|quiet]]`
102
+ - `hf extensions remove NAME` — Remove an installed extension. `[--format [auto|human|agent|json|quiet]]`
103
+ - `hf extensions search` — Search extensions available on GitHub (tagged with 'hf-extension' topic). `[--format [auto|human|agent|json|quiet]]`
104
+
105
+ ### `hf jobs` — Run and manage Jobs on the Hub.
106
+
107
+ - `hf jobs cancel JOB_ID` — Cancel a Job `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
108
+ - `hf jobs hardware` — List available hardware options for Jobs `[--format [auto|human|agent|json|quiet]]`
109
+ - `hf jobs inspect JOB_IDS` — Display detailed information on one or more Jobs `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
110
+ - `hf jobs labels JOB_ID` — Update labels on a Job. Replaces all existing labels. `[--label TEXT --clear --namespace TEXT --format [auto|human|agent|json|quiet]]`
111
+ - `hf jobs logs JOB_ID` — Fetch the logs of a Job. `[--follow --tail INTEGER --namespace TEXT --format [auto|human|agent|json|quiet]]`
112
+ - `hf jobs ps` — List Jobs. `[--all --namespace TEXT --filter TEXT --format [auto|human|agent|json|quiet]]`
113
+ - `hf jobs run IMAGE COMMAND` — Run a Job. `[--env TEXT --secrets TEXT --label TEXT --volume TEXT --env-file TEXT --secrets-file TEXT --flavor [cpu-basic|cpu-upgrade|cpu-performance|cpu-xl|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8|h200|h200x2|h200x4|h200x8|rtx-pro-6000|rtx-pro-6000x2|rtx-pro-6000x4|rtx-pro-6000x8] --timeout TEXT --detach --expose INTEGER --namespace TEXT]`
114
+ - `hf jobs scheduled delete SCHEDULED_JOB_ID` — Delete a scheduled Job. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
115
+ - `hf jobs scheduled inspect SCHEDULED_JOB_IDS` — Display detailed information on one or more scheduled Jobs `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
116
+ - `hf jobs scheduled labels SCHEDULED_JOB_ID` — Update labels on a scheduled Job. Replaces all existing labels. `[--label TEXT --clear --namespace TEXT --format [auto|human|agent|json|quiet]]`
117
+ - `hf jobs scheduled ps` — List scheduled Jobs `[--all --namespace TEXT --filter TEXT --format [auto|human|agent|json|quiet]]`
118
+ - `hf jobs scheduled resume SCHEDULED_JOB_ID` — Resume (unpause) a scheduled Job. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
119
+ - `hf jobs scheduled run SCHEDULE IMAGE COMMAND` — Schedule a Job. `[--suspend --concurrency --env TEXT --secrets TEXT --label TEXT --volume TEXT --env-file TEXT --secrets-file TEXT --flavor [cpu-basic|cpu-upgrade|cpu-performance|cpu-xl|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8|h200|h200x2|h200x4|h200x8|rtx-pro-6000|rtx-pro-6000x2|rtx-pro-6000x4|rtx-pro-6000x8] --timeout TEXT --expose INTEGER --namespace TEXT]`
120
+ - `hf jobs scheduled suspend SCHEDULED_JOB_ID` — Suspend (pause) a scheduled Job. `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
121
+ - `hf jobs scheduled uv run SCHEDULE SCRIPT` — Run a UV script (local file or URL) on HF infrastructure `[--suspend --concurrency --image TEXT --flavor [cpu-basic|cpu-upgrade|cpu-performance|cpu-xl|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8|h200|h200x2|h200x4|h200x8|rtx-pro-6000|rtx-pro-6000x2|rtx-pro-6000x4|rtx-pro-6000x8] --env TEXT --secrets TEXT --label TEXT --volume TEXT --env-file TEXT --secrets-file TEXT --timeout TEXT --expose INTEGER --namespace TEXT --with TEXT --python TEXT]`
122
+ - `hf jobs stats` — Fetch the resource usage statistics and metrics of Jobs `[--namespace TEXT --format [auto|human|agent|json|quiet]]`
123
+ - `hf jobs uv run SCRIPT` — Run a UV script (local file or URL) on HF infrastructure `[--image TEXT --flavor [cpu-basic|cpu-upgrade|cpu-performance|cpu-xl|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8|h200|h200x2|h200x4|h200x8|rtx-pro-6000|rtx-pro-6000x2|rtx-pro-6000x4|rtx-pro-6000x8] --env TEXT --secrets TEXT --label TEXT --volume TEXT --env-file TEXT --secrets-file TEXT --timeout TEXT --detach --expose INTEGER --namespace TEXT --with TEXT --python TEXT]`
124
+
125
+ ### `hf models` — Interact with models on the Hub.
126
+
127
+ - `hf models card MODEL_ID` — Get the model card (README) for a model on the Hub. `[--metadata --text --format [auto|human|agent|json|quiet]]`
128
+ - `hf models info MODEL_ID` — Get info about a model on the Hub. `[--revision TEXT --expand TEXT --format [auto|human|agent|json|quiet]]`
129
+ - `hf models list` — List models on the Hub, or files in a model repo. `[--search TEXT --author TEXT --filter TEXT --num-parameters TEXT --sort [created_at|downloads|last_modified|likes|trending_score] --limit INTEGER --expand TEXT --human-readable --tree --recursive --revision TEXT --format [auto|human|agent|json|quiet]]`
130
+
131
+ ### `hf papers` — Interact with papers on the Hub.
132
+
133
+ - `hf papers info PAPER_ID` — Get info about a paper on the Hub. `[--format [auto|human|agent|json|quiet]]`
134
+ - `hf papers list` — List daily papers on the Hub. `[--date TEXT --week TEXT --month TEXT --submitter TEXT --sort [publishedAt|trending] --limit INTEGER --format [auto|human|agent|json|quiet]]`
135
+ - `hf papers read PAPER_ID` — Read a paper as markdown. `[--format [auto|human|agent|json|quiet]]`
136
+ - `hf papers search QUERY` — Search papers on the Hub. `[--limit INTEGER --format [auto|human|agent|json|quiet]]`
137
+
138
+ ### `hf repos` — Manage repos on the Hub.
139
+
140
+ - `hf repos branch create REPO_ID BRANCH` — Create a new branch for a repo on the Hub. `[--revision TEXT --type [model|dataset|space] --exist-ok --format [auto|human|agent|json|quiet]]`
141
+ - `hf repos branch delete REPO_ID BRANCH` — Delete a branch from a repo on the Hub. `[--type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
142
+ - `hf repos cp SRC` — Copy files between local paths, repositories, and buckets. `[--format [auto|human|agent|json|quiet]]`
143
+ - `hf repos create REPO_ID` — Create a new repo on the Hub. `[--type [model|dataset|space] --space-sdk TEXT --private --public --protected --exist-ok --resource-group-id TEXT --region [us|eu] --flavor [cpu-basic|cpu-upgrade|zero-a10g|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8] --storage [small|medium|large] --sleep-time INTEGER --secrets TEXT --secrets-file TEXT --env TEXT --env-file TEXT --volume TEXT --format [auto|human|agent|json|quiet]]`
144
+ - `hf repos delete REPO_ID` — Delete a repo from the Hub. This is an irreversible operation. `[--type [model|dataset|space] --missing-ok --yes --format [auto|human|agent|json|quiet]]`
145
+ - `hf repos delete-files REPO_ID PATTERNS` — Delete files from a repo on the Hub. `[--type [model|dataset|space] --revision TEXT --commit-message TEXT --commit-description TEXT --create-pr --format [auto|human|agent|json|quiet]]`
146
+ - `hf repos duplicate FROM_ID` — Duplicate a repo on the Hub (model, dataset, or Space). `[--type [model|dataset|space] --private --public --protected --exist-ok --flavor [cpu-basic|cpu-upgrade|zero-a10g|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8] --storage [small|medium|large] --sleep-time INTEGER --secrets TEXT --secrets-file TEXT --env TEXT --env-file TEXT --volume TEXT --format [auto|human|agent|json|quiet]]`
147
+ - `hf repos list` — List all repos (models, datasets, spaces, buckets) with storage info. `[--namespace TEXT --type [model|dataset|space|bucket] --search TEXT --limit INTEGER --explore --format [auto|human|agent|json|quiet]]`
148
+ - `hf repos move FROM_ID TO_ID` — Move a repository from a namespace to another namespace. `[--type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
149
+ - `hf repos settings REPO_ID` — Update the settings of a repository. `[--gated [auto|manual|false] --private --public --protected --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
150
+ - `hf repos tag create REPO_ID TAG` — Create a tag for a repo. `[--message TEXT --revision TEXT --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
151
+ - `hf repos tag delete REPO_ID TAG` — Delete a tag for a repo. `[--yes --type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
152
+ - `hf repos tag list REPO_ID` — List tags for a repo. `[--type [model|dataset|space] --format [auto|human|agent|json|quiet]]`
153
+
154
+ ### `hf skills` — Manage skills for AI assistants.
155
+
156
+ - `hf skills add` — Download a Hugging Face skill and install it for an AI assistant. `[--claude --global --dest PATH --force --format [auto|human|agent|json|quiet]]`
157
+ - `hf skills list` — List available skills from the Hugging Face marketplace. `[--format [auto|human|agent|json|quiet]]`
158
+ - `hf skills preview` — Print the generated `hf-cli` SKILL.md to stdout. `[--format [auto|human|agent|json|quiet]]`
159
+ - `hf skills update` — Update installed Hugging Face marketplace skills. `[--claude --global --dest PATH --format [auto|human|agent|json|quiet]]`
160
+
161
+ ### `hf spaces` — Interact with spaces on the Hub.
162
+
163
+ - `hf spaces card SPACE_ID` — Get the Space card (README) for a Space on the Hub. `[--metadata --text --format [auto|human|agent|json|quiet]]`
164
+ - `hf spaces dev-mode SPACE_ID` — Enable or disable dev mode on a Space. `[--stop --format [auto|human|agent|json|quiet]]`
165
+ - `hf spaces hardware` — List available hardware options for Spaces. `[--format [auto|human|agent|json|quiet]]`
166
+ - `hf spaces hot-reload SPACE_ID` — Hot-reload any Python file of a Space without a full rebuild + restart. `[--local-file PATH --skip-checks --skip-summary --format [auto|human|agent|json|quiet]]`
167
+ - `hf spaces info SPACE_ID` — Get info about a space on the Hub. `[--revision TEXT --expand TEXT --format [auto|human|agent|json|quiet]]`
168
+ - `hf spaces list` — List spaces on the Hub, or files in a space repo. `[--search TEXT --author TEXT --filter TEXT --sort [created_at|last_modified|likes|trending_score] --limit INTEGER --expand TEXT --human-readable --tree --recursive --revision TEXT --format [auto|human|agent|json|quiet]]`
169
+ - `hf spaces logs SPACE_ID` — Fetch the run or build logs of a Space. `[--build --follow --tail INTEGER --format [auto|human|agent|json|quiet]]`
170
+ - `hf spaces pause SPACE_ID` — Pause a Space. `[--format [auto|human|agent|json|quiet]]`
171
+ - `hf spaces restart SPACE_ID` — Restart a Space. `[--factory-reboot --format [auto|human|agent|json|quiet]]`
172
+ - `hf spaces search QUERY` — Search spaces on the Hub using semantic search. `[--filter TEXT --sdk TEXT --include-non-running --description --limit INTEGER --format [auto|human|agent|json|quiet]]`
173
+ - `hf spaces secrets add SPACE_ID` — Add or update secrets for a Space. `[--secrets TEXT --secrets-file TEXT --format [auto|human|agent|json|quiet]]`
174
+ - `hf spaces secrets delete SPACE_ID KEY` — Remove a secret from a Space. `[--yes --format [auto|human|agent|json|quiet]]`
175
+ - `hf spaces secrets list SPACE_ID` — List secrets for a Space. Secret values are write-only and not returned. `[--format [auto|human|agent|json|quiet]]`
176
+ - `hf spaces settings SPACE_ID` — Update the settings of a Space. `[--sleep-time INTEGER --hardware [cpu-basic|cpu-upgrade|zero-a10g|t4-small|t4-medium|l4x1|l4x4|l40sx1|l40sx4|l40sx8|a10g-small|a10g-large|a10g-largex2|a10g-largex4|a100-large|a100x4|a100x8] --format [auto|human|agent|json|quiet]]`
177
+ - `hf spaces ssh SPACE_ID` — SSH into a Space's Dev Mode container. `[--identity-file PATH --dry-run --auto --format [auto|human|agent|json|quiet]]`
178
+ - `hf spaces variables add SPACE_ID` — Add or update environment variables for a Space. `[--env TEXT --env-file TEXT --format [auto|human|agent|json|quiet]]`
179
+ - `hf spaces variables delete SPACE_ID KEY` — Remove an environment variable from a Space. `[--yes --format [auto|human|agent|json|quiet]]`
180
+ - `hf spaces variables list SPACE_ID` — List environment variables for a Space. `[--format [auto|human|agent|json|quiet]]`
181
+ - `hf spaces volumes delete SPACE_ID` — Remove all volumes from a Space. `[--yes --format [auto|human|agent|json|quiet]]`
182
+ - `hf spaces volumes list SPACE_ID` — List volumes mounted in a Space. `[--format [auto|human|agent|json|quiet]]`
183
+ - `hf spaces volumes set SPACE_ID` — Set (replace) volumes for a Space. `[--volume TEXT --format [auto|human|agent|json|quiet]]`
184
+
185
+ ### `hf webhooks` — Manage webhooks on the Hub.
186
+
187
+ - `hf webhooks create --watch TEXT` — Create a new webhook. `[--url TEXT --job-id TEXT --domain [repo|discussions] --secret TEXT --format [auto|human|agent|json|quiet]]`
188
+ - `hf webhooks delete WEBHOOK_ID` — Delete a webhook permanently. `[--yes --format [auto|human|agent|json|quiet]]`
189
+ - `hf webhooks disable WEBHOOK_ID` — Disable an active webhook. `[--format [auto|human|agent|json|quiet]]`
190
+ - `hf webhooks enable WEBHOOK_ID` — Enable a disabled webhook. `[--format [auto|human|agent|json|quiet]]`
191
+ - `hf webhooks info WEBHOOK_ID` — Show full details for a single webhook. `[--format [auto|human|agent|json|quiet]]`
192
+ - `hf webhooks list` — List all webhooks for the current user. `[--format [auto|human|agent|json|quiet]]`
193
+ - `hf webhooks update WEBHOOK_ID` — Update an existing webhook. Only provided options are changed. `[--url TEXT --watch TEXT --domain [repo|discussions] --secret TEXT --format [auto|human|agent|json|quiet]]`
194
+
195
+ ## Common options
196
+
197
+ - `--format` — Output format: `--format json` (or `--json`) or `--format table` (default).
198
+ - `-q / --quiet` — Quiet output (one ID per line).
199
+ - `--revision` — Git revision id which can be a branch name, a tag, or a commit hash.
200
+ - `--token` — Use a User Access Token. Prefer setting `HF_TOKEN` env var instead of passing `--token`.
201
+ - `--type` — The type of repository (model, dataset, or space).
202
+
203
+ ## Mounting repos as local filesystems
204
+
205
+ To mount Hub repositories or buckets as local filesystems — no download, no copy, no waiting — use `hf-mount`. Files are fetched on demand. GitHub: https://github.com/huggingface/hf-mount
206
+
207
+ Install: `curl -fsSL https://raw.githubusercontent.com/huggingface/hf-mount/main/install.sh | sh`
208
+
209
+ Some command examples:
210
+ - `hf-mount start repo openai-community/gpt2 /tmp/gpt2` — mount a repo (read-only)
211
+ - `hf-mount start --hf-token $HF_TOKEN bucket myuser/my-bucket /tmp/data` — mount a bucket (read-write)
212
+ - `hf-mount status` / `hf-mount stop /tmp/data` — list or unmount
213
+
214
+ ## Tips
215
+
216
+ - Use `hf <command> --help` for full options, descriptions, usage, and real-world examples
217
+ - Authenticate with `HF_TOKEN` env var (recommended) or with `--token`
218
+ - Update the CLI with `hf update` (uses the correct command for the detected install method)
@@ -0,0 +1,79 @@
1
+ ---
2
+ name: hf-mem
3
+ description: Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub
4
+ ---
5
+
6
+ `hf_mem` estimates the required memory for inference, including model weights and an optional KV cache, for Safetensors and GGUF for models on the Hugging Face Hub using HTTP Range requests i.e., without downloading or loading any weights locally.
7
+
8
+ ## When to use?
9
+
10
+ - User asks how much VRAM or memory a model needs to run
11
+ - User wants to know if a model fits on their GPU or a given instance
12
+ - User references a Hugging Face model ID or URL and asks about inference requirements
13
+
14
+ ## What are the requirements?
15
+
16
+ - `uv` installed (for `uvx`)
17
+ - `HF_TOKEN` env var or `--hf-token` flag (for gated or private models only)
18
+
19
+ ## How to run?
20
+
21
+ Run with `--model-id` pointing to the Hugging Face Hub repository which will check that it either contains Safetensors (via `model.safetensors`, `model.safetensors.index.json` if sharded, or `model_index.json` for Diffusers) or GGUF model weights within.
22
+
23
+ ```bash
24
+ uvx hf-mem --model-id <model-id> --json-output
25
+ ```
26
+
27
+ If the repository contains GGUF model weights in multiple precisions / quantizations, the estimations will be on a per-file basis, whereas for inference you won't load all of those but rather only a single precision. This being said, for GGUF you might as well need to provide `--gguf-file` to target the specific file (or path if sharded) you want to run.
28
+
29
+ ```bash
30
+ uvx hf-mem --model-id <model-id> --gguf-file <file-or-path> --json-output
31
+ ```
32
+
33
+ Additionally, `hf-mem` comes with an `--experimental` flag that will also calculate the KV cache memory requirements too, useful for large-language models, meaning it applies to LLMs (`...ForCausalLM`), VLMs (`...ForConditionalGeneration`), and GGUF models.
34
+
35
+ As per the context window, it will be read from the default or overridden with `--max-model-len` a la vLLM. And, same goes for the KV cache precision, which will default to the model precision unless manually set via `--kv-cache-dtype` a la vLLM too.
36
+
37
+ For Safetensors use as:
38
+
39
+ ```bash
40
+ uvx hf-mem --model-id <model-id> --experimental [--max-model-len N] [--batch-size N] [--kv-cache-dtype auto|bfloat16|fp8|fp8_ds_mla|fp8_e4m3|fp8_e5m2|fp8_inc] --json-output
41
+ ```
42
+
43
+ And, for GGUF use as:
44
+
45
+ ```bash
46
+ uvx hf-mem --model-id <model-id> --gguf-file <file-or-path> --experimental [--max-model-len N] [--batch-size N] [--kv-cache-dtype auto|F32|F16|Q4_0|Q4_1|Q5_0|Q5_1|Q8_0|Q8_1|Q2_K|Q3_K|Q4_K|Q5_K|Q6_K|Q8_K|IQ2_XXS|IQ2_XS|IQ3_XXS|IQ1_S|IQ4_NL|IQ3_S|IQ2_S|IQ4_XS|I8|I16|I32|I64|F64|IQ1_M|BF16|TQ1_0|TQ2_0|MXFP4] --json-output
47
+ ```
48
+
49
+ ## Examples
50
+
51
+ For Transformers with Safetensors weights:
52
+
53
+ ```bash
54
+ uvx hf-mem --model-id MiniMaxAI/MiniMax-M2 --json-output
55
+ ```
56
+
57
+ For Diffusers with Safetensors weights:
58
+
59
+ ```bash
60
+ uvx hf-mem --model-id Qwen/Qwen-Image --json-output
61
+ ```
62
+
63
+ For Sentence Transformers with Safetensors weights:
64
+
65
+ ```bash
66
+ uvx hf-mem --model-id google/embeddinggemma-300m --json-output
67
+ ```
68
+
69
+ With `--experimental` to include the KV cache estimation for LLMs and VLMs:
70
+
71
+ ```bash
72
+ uvx hf-mem --model-id mistralai/Mistral-7B-v0.1 --experimental --json-output
73
+ ```
74
+
75
+ And, for LLMs or VLMs with GGUF weights:
76
+
77
+ ```bash
78
+ uvx hf-mem --model-id unsloth/Qwen3.5-397B-A17B-GGUF --gguf-file Q4_K_M --experimental --json-output
79
+ ```
@@ -0,0 +1,134 @@
1
+ ---
2
+ name: huggingface-best
3
+ description: >
4
+ Use when the user asks about finding the best, top, or recommended model for a task,
5
+ wants to know what AI model to use, or wants to compare models by benchmark scores.
6
+ Triggers on: "best model for X", "what model should I use for", "top models for [task]",
7
+ "which model runs on my laptop/machine/device", "recommend a model for", "what LLM should
8
+ I use for", "compare models for", "what's state of the art for", or any question about
9
+ choosing an AI model for a specific use case. Always use this skill when the user wants
10
+ model recommendations or comparisons, even if they don't explicitly mention HuggingFace
11
+ or benchmarks.
12
+ ---
13
+
14
+ # HuggingFace Best Model Finder
15
+
16
+ Finds the best models for a task by querying official HF benchmark leaderboards, enriching
17
+ results with model size data, filtering for what fits on the user's device, and returning a
18
+ comparison table with benchmark scores.
19
+
20
+ ---
21
+
22
+ ## Step 1: Parse the request
23
+
24
+ Extract from the user's message:
25
+ - **Task**: what they want the model to do (coding, math/reasoning, chat, OCR, RAG/retrieval, speech recognition, image classification, multimodal, agents, etc.)
26
+ - **Device**: hardware constraints (MacBook M-series 8/16/32/64GB unified memory, RTX GPU with VRAM amount, CPU-only, cloud/no constraint, etc.)
27
+
28
+ If device is not mentioned, skip filtering entirely and return the highest-performing models regardless of size. If the task is genuinely ambiguous, ask one clarifying question.
29
+
30
+ ### Device → max parameter budget
31
+
32
+ When a device is specified, extract its available memory (unified RAM for Apple Silicon, VRAM for discrete GPUs) and apply:
33
+
34
+ - **fp16 max params (B)** ≈ memory (GB) ÷ 2
35
+ - **Q4 max params (B)** ≈ memory (GB) × 2
36
+
37
+ Examples: 16GB → 8B fp16 / 32B Q4 — 24GB VRAM → 12B fp16 / 48B Q4 — 8GB → 4B fp16 / 16B Q4
38
+
39
+ ---
40
+
41
+ ## Step 2: Find relevant benchmark datasets
42
+
43
+ Fetch the full list of official HF benchmarks:
44
+
45
+ ```bash
46
+ curl -s -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
47
+ "https://huggingface.co/api/datasets?filter=benchmark:official&limit=500" | jq '[.[] | {id, tags, description}]'
48
+ ```
49
+
50
+ Read the returned list and select the datasets most relevant to the user's task — match on dataset id, tags, and description. Use your judgment; don't limit yourself to 2-3. Aim for comprehensive coverage: if 5 benchmarks clearly cover the task, use all 5.
51
+
52
+ ---
53
+
54
+ ## Step 3: Fetch top models from leaderboards
55
+
56
+ For each selected benchmark dataset:
57
+
58
+ ```bash
59
+ curl -s -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
60
+ "https://huggingface.co/api/datasets/<namespace>/<repo>/leaderboard" | jq '[.[:15] | .[] | {rank, modelId, value, verified}]'
61
+ ```
62
+
63
+ Collect model IDs and scores across all benchmarks. If a leaderboard returns an error (404, 401, etc.), skip it and note it in the output.
64
+
65
+ ---
66
+
67
+ ## Step 4: Enrich with model metadata
68
+
69
+ For the top 10-15 candidate model IDs, get model infos.
70
+
71
+ ```bash
72
+ # REST API
73
+ curl -s -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
74
+ "https://huggingface.co/api/models/org/model1" | jq '{safetensors, tags, cardData}'
75
+
76
+ # CLI (hf-cli)
77
+ hf models info org/model1 --json | jq '{safetensors, tags, cardData}'
78
+ ```
79
+
80
+ Extract from each response:
81
+ - **Parameters**: `safetensors.total` → convert to B (e.g., 7_241_748_480 → "7.2B")
82
+ - **License**: from model card tags (look for `license:apache-2.0`, `license:mit`, etc.)
83
+ - If `safetensors` is absent, parse size from the model name (look for "7b", "8b", "13b", "70b", "72b", etc.)
84
+
85
+ ---
86
+
87
+ ## Step 5: Filter and rank
88
+
89
+ **If a device was specified:**
90
+ 1. Remove models exceeding the fp16 parameter budget for the device
91
+ 2. Flag models that fit only with Q4 quantization (multiply budget by ~4 for Q4 capacity)
92
+ 3. If a highly-ranked model is slightly over budget, keep it with a "needs Q4" note — don't silently drop it
93
+
94
+ **If no device was mentioned:** skip all size filtering — just rank by benchmark score.
95
+
96
+ Then: rank by benchmark score (descending), keep top 5-8 models.
97
+
98
+ Include proprietary models (GPT-4, Claude, Gemini) if they appear on leaderboards, but flag them as "API only / not self-hostable". If the user explicitly asked for local/open models only, exclude them.
99
+
100
+ ---
101
+
102
+ ## Step 6: Output
103
+
104
+ ### Comparison table
105
+
106
+ ```markdown
107
+ | # | Model | Params | [Benchmark 1] | [Benchmark 2] | License | On device |
108
+ |---|-------|--------|--------------|--------------|---------|-----------|
109
+ | ⭐1 | [org/name](https://huggingface.co/org/name) | 7B | 85.2% | — | Apache 2.0 | Yes (fp16) |
110
+ | 2 | [org/name](https://huggingface.co/org/name) | 13B | 83.1% | 71.5% | MIT | Q4 only |
111
+ | 3 | [org/name](https://huggingface.co/org/name) | 70B | 90.0% | 81.0% | Llama | Too large |
112
+ ```
113
+
114
+ - Link model names to `https://huggingface.co/<model_id>`
115
+ - Use `—` for benchmarks where the model wasn't evaluated
116
+ - Star the top recommended pick with ⭐
117
+ - "On device" values: `Yes (fp16)`, `Q4 only`, `Too large`, `API only`
118
+
119
+ ### Follow-up
120
+
121
+ After presenting the table, ask the user: "Would you like to run **[top recommended model]**?"
122
+
123
+ If they say yes, ask whether they'd prefer to:
124
+ - **Run locally** — ask about their device if not already known, then give appropriate setup instructions
125
+ - **Run on HF Jobs** — point them to the HF Jobs guide: https://huggingface.co/docs/huggingface_hub/en/guides/jobs
126
+
127
+ ---
128
+
129
+ ## Error handling
130
+
131
+ - **Leaderboard not found**: skip, note "leaderboard unavailable" in output
132
+ - **Model missing from hub_repo_details**: fall back to parsing size from model name
133
+ - **No benchmarks found for task**: use the curated fallback table above, or try `hub_repo_search` with `filters=["<task>"]` sorted by `trendingScore`
134
+ - **All leaderboards fail**: fall back to `hub_repo_search` for popular models tagged with the task, note that results are by popularity rather than benchmark score
@@ -0,0 +1,107 @@
1
+ ---
2
+ name: huggingface-datasets
3
+ description: Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.
4
+ ---
5
+
6
+ # Hugging Face Dataset Viewer
7
+
8
+ Use this skill to execute read-only Dataset Viewer API calls for dataset exploration and extraction.
9
+
10
+ ## Core workflow
11
+
12
+ 1. Optionally validate dataset availability with `/is-valid`.
13
+ 2. Resolve `config` + `split` with `/splits`.
14
+ 3. Preview with `/first-rows`.
15
+ 4. Paginate content with `/rows` using `offset` and `length` (max 100).
16
+ 5. Use `/search` for text matching and `/filter` for row predicates.
17
+ 6. Retrieve parquet links via `/parquet` and totals/metadata via `/size` and `/statistics`.
18
+
19
+ ## Defaults
20
+
21
+ - Base URL: `https://datasets-server.huggingface.co`
22
+ - Default API method: `GET`
23
+ - Query params should be URL-encoded.
24
+ - `offset` is 0-based.
25
+ - `length` max is usually `100` for row-like endpoints.
26
+ - Gated/private datasets require `Authorization: Bearer <HF_TOKEN>`.
27
+
28
+ ## Dataset Viewer
29
+
30
+ - `Validate dataset`: `/is-valid?dataset=<namespace/repo>`
31
+ - `List subsets and splits`: `/splits?dataset=<namespace/repo>`
32
+ - `Preview first rows`: `/first-rows?dataset=<namespace/repo>&config=<config>&split=<split>`
33
+ - `Paginate rows`: `/rows?dataset=<namespace/repo>&config=<config>&split=<split>&offset=<int>&length=<int>`
34
+ - `Search text`: `/search?dataset=<namespace/repo>&config=<config>&split=<split>&query=<text>&offset=<int>&length=<int>`
35
+ - `Filter with predicates`: `/filter?dataset=<namespace/repo>&config=<config>&split=<split>&where=<predicate>&orderby=<sort>&offset=<int>&length=<int>`
36
+ - `List parquet shards`: `/parquet?dataset=<namespace/repo>`
37
+ - `Get size totals`: `/size?dataset=<namespace/repo>`
38
+ - `Get column statistics`: `/statistics?dataset=<namespace/repo>&config=<config>&split=<split>`
39
+ - `Get Croissant metadata (if available)`: `/croissant?dataset=<namespace/repo>`
40
+
41
+ Pagination pattern:
42
+
43
+ ```bash
44
+ curl "https://datasets-server.huggingface.co/rows?dataset=stanfordnlp/imdb&config=plain_text&split=train&offset=0&length=100"
45
+ curl "https://datasets-server.huggingface.co/rows?dataset=stanfordnlp/imdb&config=plain_text&split=train&offset=100&length=100"
46
+ ```
47
+
48
+ When pagination is partial, use response fields such as `num_rows_total`, `num_rows_per_page`, and `partial` to drive continuation logic.
49
+
50
+ Search/filter notes:
51
+
52
+ - `/search` matches string columns (full-text style behavior is internal to the API).
53
+ - `/filter` requires predicate syntax in `where` and optional sort in `orderby`.
54
+ - Keep filtering and searches read-only and side-effect free.
55
+
56
+ For CLI-based parquet URL discovery or SQL, use the `hf-cli` skill with `hf datasets parquet` and `hf datasets sql`.
57
+
58
+ ## Creating and Uploading Datasets
59
+
60
+ Use one of these flows depending on dependency constraints.
61
+
62
+ Zero local dependencies (Hub UI):
63
+
64
+ - Create dataset repo in browser: `https://huggingface.co/new-dataset`
65
+ - Upload parquet files in the repo "Files and versions" page.
66
+ - Verify shards appear in Dataset Viewer:
67
+
68
+ ```bash
69
+ curl -s "https://datasets-server.huggingface.co/parquet?dataset=<namespace>/<repo>"
70
+ ```
71
+
72
+ Low dependency CLI flow (`npx @huggingface/hub` / `hfjs`):
73
+
74
+ - Set auth token:
75
+
76
+ ```bash
77
+ export HF_TOKEN=<your_hf_token>
78
+ ```
79
+
80
+ - Upload parquet folder to a dataset repo (auto-creates repo if missing):
81
+
82
+ ```bash
83
+ npx -y @huggingface/hub upload datasets/<namespace>/<repo> ./local/parquet-folder data
84
+ ```
85
+
86
+ - Upload as private repo on creation:
87
+
88
+ ```bash
89
+ npx -y @huggingface/hub upload datasets/<namespace>/<repo> ./local/parquet-folder data --private
90
+ ```
91
+
92
+ After upload, call `/parquet` to discover `<config>/<split>/<shard>` values for querying with `@~parquet`.
93
+
94
+ ## Agent Traces
95
+
96
+ The Hub supports raw agent session traces from Claude Code, Codex, and Pi Agent. Upload them to Hugging Face Datasets as original JSONL files and the Hub can auto-detect the trace format, tag the dataset as `Traces`, and enable the trace viewer for browsing sessions, turns, tool calls, and model responses. Common local session directories:
97
+
98
+ - Claude Code: `~/.claude/projects`
99
+ - Codex: `~/.codex/sessions`
100
+ - Pi: `~/.pi/agent/sessions`
101
+
102
+ Default to private dataset repos because traces can contain prompts, file paths, tool outputs, secrets, or PII. Preserve the raw `.jsonl` files and nest them by project/cwd instead of uploading every session at the dataset root.
103
+
104
+ ```bash
105
+ hf repos create <namespace>/<repo> --type dataset --private --exist-ok
106
+ hf upload <namespace>/<repo> ~/.codex/sessions codex/<project-or-cwd> --type dataset
107
+ ```