@aiagenta2z/agtm 1.0.7 → 1.0.9

package/README.md

@@ -1,297 +1,273 @@
 
-### agtm: Command Line CLI Tool for AI Agent Registry in the Open Source AI Agent Marketplace
+### agtm: CLI Tool for AI Agent Management, Skills, Agent Registry, Benchmarks and Hints in AI Agent Marketplace
 
 [GitHub](https://github.com/aiagenta2z/agtm)|[AI Agent Marketplace CLI Doc](https://www.deepnlp.org/doc/ai_agent_marketplace)|[DeepNLP AI Agent Marketplace](https://www.deepnlp.org/store/ai-agent) | [OneKey Agent Router](https://www.deepnlp.org/agent/onekey-mcp-router) | [Agent MCP OneKey Router Ranking](https://www.deepnlp.org/agent/rankings) | [NodeJS agtm](https://www.npmjs.com/package/@aiagenta2z/agtm)
 
+`agtm` (AI Agent Management CLI) unifies skill management, agent registration, marketplace search, and provider CLI execution. Install skills from GitHub, log and rate skill runs, upload agent metadata to registries, query the public marketplace, and run agent toolchains with fuzzy hints.
 
-The command line `agtm upload` , `agtm search`,  `agtm add` helps users to register and publish their AI Agents meta api information from agent.json/agent.yaml file
-or a github repo URL in a few seconds.
+Features
 
-'agtm' means 'ai agent marketplace' or 'ai agent manager', you can use this command line to manage agent meta, skills add,installation, etc.
-To use the command line, you need to first install the package either using python or node environment
+*`agtm skills`*: Manage Skills, Add Skills, List Skills, Log Skills Performance, Skills performance Evaluator, compare to realworld benchmarks   
+*`agtm upload`*: AI Agent Registry, register local agent meta information of json or yaml format(agent.json/agent.yaml) or sync your github source meta including README.md     
+*`agtm search`*: Search the open source AI Agent Marketplace, including github community, huggingface community, product hunt community, deepnlp ai agent marketplace index, etc     
+*`agtm run`*: Run agent clis, don't need to remember, with the powerful hints and completion ability, just type a few characters and "--hint" will help you complete the command line.   
 
-## Quickstart
-
-Agtm CLI Options
-
-### `search`
-`agtm search` helps to search AI Agent MCP and skills marketplace by id or query keywords.
+Furthermore, `agtm` provides memory to track skill outputs and enables performance rating against industry job level benchmarks. This allows you to score each skill execution and assign a professional tier to your AI Agent's capabilities—for example, evaluating its performance as equivalent to that of an L3 or L5 software engineer, marketing prefessional, etc.
 
-Example Usage
 ```shell
-agtm search --q 'coding agent'
-agtm search --id 'google-maps/google-maps'
+skill_id             run_times  score  level
+-------------------  ---------  -----  -----
+code_success_skills  5          0.9     L3(100%)    
 ```
 
-### `upload`
-`agtm update` helps to update your local agent.json or agent.yaml meta information to DeepNLP AI Agent Marketplace Index.
-Example Usage
-```shell
-agtm upload --github https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent
-agtm upload --config ./agent.json
-agtm upload --config ./agent.yaml
-```
+## Quickstart
 
-### `add`
-`agtm add` Install Skills from Github
+###  Installation
 
-Example Usage
-```shell
-agtm add aiagenta2z/onekey-agent-router --skill google-maps     ## install to your project folder
-agtm add aiagenta2z/onekey-agent-router --skill google-maps -g  ## install to global cache folder
-agtm add https://github.com/aiagenta2z/onekey-agent-router --skill google-maps  ## install one skill
+**Node**
+``` NodeJS
+npm install -g @aiagenta2z/agtm
 ```
 
-Expected Results install google-maps
+Setup hint and skills benchmark 
 ```shell
-Installed skill 'google-maps' to antigravity at /path/to/project/.agent/skills/google-maps
-Installed skill 'google-maps' to codex at /path/to/project/.agents/skills/google-maps
-Installed skill 'google-maps' to claude-code at /path/to/project/.claude/skills/google-maps
+agtm setup --levels  ## Needed before `agtm rate`, to sync the benchmarks json to local folder 
+agtm setup --hint    ## Needed before `agtm run` 
 ```
 
+Agtm CLI Options
 
-## Tutorial 
+| CLI         | Command and Options                       | Document                       |
+|-------------|-------------------------------------------|--------------------------------|
+| agtm skills | add, list, log, rate                      | [Doc](../docs/skills/README.md)   |
+| agtm search | --q query                                 | [Doc](../docs/registry/README.md) |
+| agtm upload | --github  --config to local agent meta    | [Doc](../docs/registry/README.md) |
+| agtm run    | --hint agent-cli hint and auto completion | [Doc](../docs/run/README.md)      |
 
-###  1. Installation
+## `skills`  
 
-**Node**
-``` NodeJS
-npm install -g @aiagenta2z/agtm
-```
+### skills add
 
-note: run nodejs environment, you need to add -g to install, so you can just run command line `agtm` without the `npx agtm` prefix,
+Download and add skills to your agent directory.
 
-Command line to register your AI agent from GitHub, Local config(agent.json, agent.yaml) or search the AI agent marketplace.
-To see detailed usage, visit the GitHub of `agtm` at Documentation (https://github.com/aiagenta2z/ai-agent-marketplace)
+#### Usage
+```
+agtm skills add <unique_id>
+agtm skills add <github_url>
+agtm skills add <github_url> -a <agent_id>
+agtm skills add <owner_id/repo_id> -s <skill_id>
+```
 
-**Python**
+#### Example
 ```
-pip install ai-agent-marketplace
+agtm skills add anthropics/skills -a claude-code  ## install skills only to claude-codex
+agtm skills add msitarzewski/agency-agents
+agtm skills add aiagenta2z/onekey-gateway
+agtm skills add msitarzewski/agency-agents -s academic-anthropologist -a codex
+agtm skills add anthropics/skills -s skill-creator -a claude-code --global
 ```
 
-The cli bin will be copied to python path. Future version, only 'node' command line will be supported.
+### skills list
 
-### Setup
-Let's say you have create a repo of your open source AI Agent (e.g. https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent), And you want to 
-register and get a registered detailed page of AI Agent in the marketplace and monitor the traffic.
+#### Usage
+```
+agtm skills list
+agtm skills list --agent <agent_id> --global
+```
+Lists installed skills with `agent`, `skill_id`, `description`, install `path`, average `score`, and aggregated `level` (if ratings exist).
 
-**Setup Access Key**
 
-First you need to setup an access_key in the environment to authenticate.
-Register your AI Agent Marketplace [Access Key here](https://deepnlp.org/workspace/keys)
 
+#### Example
 ```
-export AI_AGENT_MARKETPLACE_ACCESS_KEY="{your_access_key}"
+agtm skills list
+agtm skills list --agent codex
+agtm skills list --agent claude-code --global
 ```
 
-You can also use the test key for validation, which is associated with a test-only account on deepnlp.org and aiagenta2z.com. 
 
+### skills log
+
+#### Usage
 ```
-export AI_AGENT_MARKETPLACE_ACCESS_KEY="TEST_KEY_AI_AGENT_REGISTRY"
+agtm skills log <skill_id> --data '<json_payload>'
 ```
+- Persists a run record at `.agtm/skills/log/<uuid>.json` (or the `--logDir` you supply).
+- `<json_payload>` must contain at least `input` and `output`; optional fields (meta, rating, level) are accepted.
 
-### 2. Skills 
+#### Example
+```
+agtm skills log <skill_id> --data '{"input":"write a website for store","output":"success"}'
+agtm skills log code_success_skills --data '{"input":"generate sql","output":"ok","meta":{"agent":"claude-code"}}'
+```
 
-Install skills from the `aiagenta2z/onekey-agent-router` Github repo (`user_name/repo_name`). The `google-maps` skill is a good starter example.
+### skills rate
 
-Under the `./skills/google-maps` folder of the target github repo.
+#### Setup
 
+To use the rate command, have to setup the benchmark levels configuration. save to `./agtm/levels/*.json` files
 ```shell
-agtm add aiagenta2z/onekey-agent-router --skill google-maps
+agtm setup --levels
 ```
 
-Install to a specific agent (codex):
-
-```shell
-agtm add aiagenta2z/onekey-agent-router --skill google-maps -a codex
+#### Usage
 ```
-
-Install to a specific agent (openclaw):
-
-```shell
-agtm add aiagenta2z/onekey-agent-router --skill google-maps -a openclaw
+agtm skills rate prepare --skill_id <skill_id> --prompt "<eval_prompt>" --benchmark <path/benchmark.json>
+agtm skills rate apply   --skill_id <skill_id> --result '<result_json>'
+agtm skills rate show    --skill_id <skill_id>
 ```
+- `prepare` exports logs plus the top benchmark slices (e.g., Google SWE L3–L7) for an external evaluator.
+- `apply` writes evaluator outputs (`rating`, `level`) back to each log.
+- `show` summarizes run counts, average score, and level distribution.
 
-Install to global paths instead of local project paths:
-
-```shell
-agtm add aiagenta2z/onekey-agent-router --skill google-maps -g
+#### Example
 ```
-
-
-### 3. AI Agent Registry
-
-**Registry AI Agent from your GitHub**
-
-The default registry provider endpoint includes: [DeepNLP AI Agent Registry Endpoint](https://www.deepnlp.org/api/ai_agent_marketplace/registry) | 
-[aiagenta2z.com Registry Endpoint](https://www.aiagenta2z.com/api/ai_agent_marketplace/registry)|[aiagenta2z.org Registry Endpoint](https://www.aiagenta2z.org/api/ai_agent_marketplace/registry), etc. 
-
-You can also set your customized registry endpoint via `--endpoint` parameter
-
-
+agtm skills rate prepare --skill_id code_success_skills --prompt "Evaluate the results" --benchmark path/customized_agent_benchmark.json
+agtm skills rate apply --skill_id code_success_skills --result '{"results":[{"log_id":"3679a3fe-4d97-4eb1-83bc-f83d711be195","rating":0.90,"level":"L4"}]}'
+agtm skills rate show --skill_id code_success_skills
 ```
-agtm upload --github https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent
+Sample output:
 ```
-
-
-**Registry AI Agent from agent.json or agent.yaml file**
-
-
+skill_id             run_times  score  level
+-------------------  ---------  -----  -----
+code_success_skills  3          0.88   L4(100%)
 ```
-
-## Register from local config of AI Agent meta
-agtm upload --config ./agent.json
-agtm upload --config ./agent.yaml
+Benchmarks live in `data/config/levels/*.json` and follow this structure:
 ```
+{
+  "software-engineering": {
+    "Google": [
+      { "level": "L3", "title": "Software Engineer II", "description": "Entry-level engineer. Delivers well-scoped tasks with guidance.", "signals": ["task execution","learning velocity","code quality basics"] },
+      { "level": "L4", "title": "Software Engineer III", "description": "Independent contributor. Owns small features end-to-end.", "signals": ["ownership","code quality","debugging ability"] }
+    ]
+  }
+}
+```
+The evaluator compares each `<input,output>` log to the benchmark definitions and assigns a rating. You can customize levels such as `poor`, `fair`, `good`, `excellent` in your benchmark file.
 
-You can download sample [agent.json](https://github.com/aiagenta2z/agtm/blob/main/agent.json) or [agent.yaml](https://github.com/aiagenta2z/agtm/blob/main/agent.yaml) file from github https://github.com/aiagenta2z/agtm
-See the explanation of the schema, please visit the [documentation](https://www.deepnlp.org/doc/ai_agent_marketplace).
+Partial List of Skills and Job Level Description 
 
+| Job Category      | Company | Levels Description (e.g. Google L3, Meta E4)                                              |
+|-------------------|---------|-------------------------------------------------------------------------------------------|
+| Marketing         | Apple   | [marketing_sales_apple_levels.json](data/config/levels/marketing_sales_apple_levels.json) |
+| Software Engineer | Google  | [software_engineer_google_levels.json](data/config/levels/software_engineer_google_levels.json)                                                                                         |
+| Software Engineer | Meta    | [software_engineer_meta_levels.json](data/config/levels/software_engineer_meta_levels.json)                                            |
+
+
+write your `customized_agent_benchmark.json` following the formats
 
-Demo agent.json file
 ```
 {
-    "name": "My First AI Coding Agent",
-    "content": "This AI Agent can do complicated programming work for humans",
-    "website": "https://www.my_first_agent.com",
-    "field": "AI AGENT",
-    "subfield": "Coding Agent",
-    "content_tag_list": "coding,python",
-    "github": "",
-    "thumbnail_picture": "https://avatars.githubusercontent.com/u/242328252?s=200&v=4",
-    "upload_image_files": "",
-    "api": "https://www.my_first_agent.com/agent",
-    "price_type": "PER_CALL",
-    "price_per_call_credit": 0.0,
-    "price_fixed_credit" : 0.0,
-    "price_subscription": "Basic: your basic plan introduction, Advanced: Your Advanced Plan introduction, etc."
+  "domain": {
+    "my_benchmark": [
+      { "level": "poor", "description": "the skill failed or the output is meaningless"},
+      { "level": "fair", "description": "the skills produces fair results, complete the task"},
+      { "level": "good", "description": "the skills output a report, an image is good in real life standards.."},
+    ]
+  }
 }
 ```
-Demo agent.yaml file
-```
 
-name: "My First AI Coding Agent"
-content: "This AI Agent can do complicated programming work for humans"
-website: "https://www.my_first_agent.com"
-field: "AI AGENT"
-subfield: "Coding Agent"
-content_tag_list: "coding,python"
-github: "https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent"
-thumbnail_picture: "https://avatars.githubusercontent.com/u/242328252?s=200&v=4"
-upload_image_files: ""
-api: "https://www.my_first_agent.com/agent"
-price_type: "API Call"
-price_per_call_credit: 0.0
-price_fixed_credit: 0.0
-price_subscription: "Basic: your basic plan introduction, Advanced: Your Advanced Plan introduction, etc."
 
-```
+## `Run`
 
-**Use your customized endpoint and schema definition**
+Execution of Agent Cli with hints and auto completion.
 
-You have the flexibility to use the AI Agent marketplace/manager cli `agtm` to submit customized ai agent schema to your customized endpoint.
+The `run` command executes agent workflows with interactive hints and fuzzy CLI completion. Typing a few characters surfaces suggested commands so you can finish the full invocation quickly (for example, typing `play` will suggest the Playwright runner).
 
-You need to define a customized `schema.json` (https://github.com/aiagenta2z/agtm/blob/main/schema.json) file similar to the default one below and your URL of `endpoint`.
-Then the package will POST the the data schema to your endpoint. 
+Let's say you want to run an agent command of Playwright to go to a URL and fetch a webpage. You don't need to remember the full command—type `play`, pick the provider, then pick the CLI action.
 
 
-Note: the keys in agent.json and schema.json should match. Package will select keys from the agent.json/agent.yaml files.
 
+### Usage
 
-`schema.json` should have two keys defined the required fields and optional fields you want to submit from the agent.json file. 
+Remember to setup hint before running the agent-cli
 
-Remember to keep the `access_key` in safe place, the post request will post the `access_key` as well as schema to the endpoint. 
+```shell
+agtm setup --hint
+```
 
-#### default schema.json definition
 ```
-{
-    "required": [
-        "name",
-        "content"
-    ],
-    "optional": [
-        "website", 
-        "field", 
-        "subfield", 
-        "content_tag_list", 
-        "github", 
-        "price_type", 
-        "api", 
-        "thumbnail_picture", 
-        "upload_image_files",
-        "sdk",
-        "package"
-    ]
-}
+agtm run <provider_unique_id> <agent_cli>
 ```
 
+### Example 
 
-Please visit the command line github package [agtm](https://github.com/aiagenta2z/agtm) and [DOC](https://www.deepnlp.org/doc/ai_agent_marketplace) detailed usage.
+```shell
+rockingdingo@rockingdingodeMacBook-Pro skills_cli % agtm run play
+DEBUG: Entering Human Mode | idArg play | commandArgs  | options [object Object] | hasHints true | hints [object Object]
 
+Skill ID suggestions:
+  1. microsoft/playwright-cli
+  2. googleworkspace/cli
+Skill ID suggestions: 1
+Command hints:
+  1. playwright-cli goto <url> # navigate to a url
+  2. playwright-cli open [url] # open browser, optionally navigate to url
 
-Use the test account and access 
-
+Select command (number or input custom): 1
+Final command [playwright-cli goto <url>]: playwright-cli goto https://www.github.com
+agtm run microsoft/playwright-cli playwright-cli goto https://www.github.com
 ```
-export AI_AGENT_MARKETPLACE_ACCESS_KEY="TEST_KEY_AI_AGENT_REGISTRY"
 
-agtm upload --config ./agent.json --endpoint https://www.deepnlp.org/api/ai_agent_marketplace/registry --schema ./schema.json
+Support CLI List, Please welcome to contrib
 
-agtm upload --config ./agent.json --endpoint https://www.aiagenta2z.com/api/ai_agent_marketplace/registry --schema ./schema.json
-```
+| unique_id | agent cli |
+| --- | --- |
+| microsoft/playwright-cli | playwright-cli open [url], playwright-cli goto <url>
+| googleworkspace/cli | gws drive files list --params
+| aiagenta2z/onekey-gateway | onekey agent <unique_id> <api_id> <data_json|@file>, onekey mcp <server_name> [--name config_name], onekey llm --provider <provider> --model <model> --messages <json|@file> [--temperature <num>] [--response-format <format>] [--options <json|@file>], onekey llm --payload <json|@file>
+| openai/codex-cli | codex, codex exec "[instruction]"
+| anthropic/claude-code | claude "[prompt]", claude --dangerously-skip-permissions
+| paul-gauthier/aider | aider --model [model_name], /test [command]
+| openinterpreter/open-interpreter | interpreter, interpreter --os
+| google-gemini/gemini-cli | gemini, gemini -p "[prompt] @[file/dir]", gemini --yolo, /memory add "[fact]", /mcp list, /restore
 
 
-```
-Setting Registry Endpoint to URL: https://www.deepnlp.org/api/ai_agent_marketplace/registry
-Customized Schema is enabled : ./schema.json
-Attempting to register agent from config file: ./agent.json
-✅ Loaded custom schema from: ./schema.json
-Using customized schema {"required": ["name", "content"], "optional": ["website", "field", "subfield", "content_tag_list", "github", "price_type", "api", "thumbnail_picture", "upload_image_files", "sdk", "package"]}
-WARN: Calling AddServiceAPI input param optional keys filled fields ['website', 'field', 'subfield', 'content_tag_list', 'github', 'price_type', 'api', 'thumbnail_picture', 'upload_image_files']|missing_fields []
-Submitting agent information to the marketplace...
+## AI Agent Registry
+### `search`
+`agtm search` helps to search AI Agent MCP and skills marketplace by id or query keywords.
 
-✅ Registration Successful!
-   URL: https://www.deepnlp.org/store/ai-agent/coding-agent/pub-test-agtm-registry/my-first-ai-coding-agent
-   Message: Content Updated Successfully|Visit URL at https://www.deepnlp.org/store/ai-agent/coding-agent/pub-test-agtm-registry/my-first-ai-coding-agent and Login to View the Pending status webpage...
-   Track its status at: https://www.deepnlp.org/store/ai-agent/coding-agent/pub-test-agtm-registry/my-first-ai-coding-agent
+Example Usage
+```shell
+agtm search --q 'coding agent'
+agtm search --id 'google-maps/google-maps'
 ```
 
+### `upload`
+`agtm update` helps to update your local agent.json or agent.yaml meta information to DeepNLP AI Agent Marketplace Index.
+Example Usage
+```shell
+agtm upload --github https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent
+agtm upload --config ./agent.json
+agtm upload --config ./agent.yaml
+```
 
-Use www.example.com as endpoint example 
+`schema.json` should have two keys defined the required fields and optional fields you want to submit from the agent.json file. 
 
+Remember to keep the `access_key` in safe place, the post request will post the `access_key` as well as schema to the endpoint. 
 
-```
-agtm upload --config ./agent.json --endpoint https://www.example.com --schema ./schema.json
+Please visit the command line github package [agtm](https://github.com/aiagenta2z/agtm) and [DOC](https://www.deepnlp.org/doc/ai_agent_marketplace) detailed usage.
 
-```
 
-It will return an error message showing the endpoint and data, https://www.example.com doesn't have endpoint handling
+Use the test account and access 
 
 ```
-❌ Registration Failed. Please check your endpoint https://www.example.com  and agent data {'name': 'My First AI Coding Agent', 'content': 'This AI Agent can do complicated programming work for humans', 'website': 'https://www.my_first_agent.com', 'field': 'AI AGENT', 'subfield': 'Coding Agent', 'content_tag_list': 'coding,python', 'github': '', 'thumbnail_picture': 'https://avatars.githubusercontent.com/u/242328252?s=200&v=4', 'upload_image_files': '', 'api': 'https://www.my_first_agent.com/agent', 'price_type': 'FREE', 'price_per_call_credit': 0.0, 'price_fixed_credit': 0.0, 'price_subscription': 'Basic: your basic plan introduction, Advanced: Your Advanced Plan introduction, etc.'}
-```
-
+export AI_AGENT_MARKETPLACE_ACCESS_KEY="TEST_KEY_AI_AGENT_REGISTRY"
 
-**Search AI Agent Marketplace**
+agtm upload --config ./agent.json --endpoint https://www.deepnlp.org/api/ai_agent_marketplace/registry --schema ./schema.json
 
+agtm upload --config ./agent.json --endpoint https://www.aiagenta2z.com/api/ai_agent_marketplace/registry --schema ./schema.json
 ```
 
-## Search Open Source AI Agent marketplace of deepnlp
+### Contributing
 
-agtm search --q 'coding agent'
-agtm search --id 'google-maps/google-maps'
-agtm search --id 'AI-Hub-Admin/My-First-AI-Coding-Agent'
-```
+#### Agent CLI List
+You are welcome to contrib your cli list to the agent cli hints json file to folder [hints](data/config/hints) or table README.md
 
-Search Result of query q="coding agent"
+#### Skill Run Benchmark 
 
-```
-[{'content_name': 'Cursor AI', 'publisher_id': 'pub-cursor', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-cursor/cursor-ai', 'website': 'https://docs.cursor.com/agent', 'github': '', 'review_cnt': '3', 'rating': '3.3', 'description': 'Cursor AI Coding Editor provides Agent mode, which is the default mode in Cursor, equipped with all tools to efficiently handle a wide range of coding tasks. Agent is the default mode in Cursor that is best for most tasks. It has all tools enabled to allow it to perform all operations necessary to complete the task at hand. Make sure to read chat overview to learn more about how modes work in Curs', 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'cursor/cursor-ai', 'content_tag_list': 'Coding Agent,AI AGENT', 'thumbnail_picture': 'https://static.aiagenta2z.com/scripts/img/ai_service_content/a7dd95e7eb419437ef4a91f4b0e853cb.jpg'}, {'content_name': 'trae ai', 'publisher_id': 'pub-trae-ai', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-trae-ai/trae-ai', 'website': 'https://www.trae.ai/', 'github': '', 'review_cnt': '2', 'rating': '4.0', 'description': 'Trae AI IDE is a cutting-edge AI-integrated development environment designed to enhance developer productivity by leveraging advanced AI models like Claude3.5 and GPT-4o. It offers a seamless coding experience with features such as intelligent code generation, real-time feedback, and multi-modal input support, making it a powerful tool for developers. Introduction Trae AI IDE, developed by ByteDan', 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'trae-ai/trae-ai', 'content_tag_list': 'CODING AGENT,AI AGENT', 'thumbnail_picture': 'https://static.aiagenta2z.com/scripts/img/ai_service_content/cc43d2dc25488b50b883781e0ae8c988.png'}, {'content_name': 'codex', 'publisher_id': 'pub-openai', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-openai/codex', 'website': 'https://github.com/openai/codex', 'github': 'https://github.com/openai/codex', 'review_cnt': '1', 'rating': '5.0', 'description': 'npm i -g @openai/codexor brew install --cask codex\n\nCodex CLI is a coding agent from OpenAI that runs locally on your computer.\n\nIf you want Codex in your code editor (VS Code, Cursor, Windsurf), install in your IDE\nIf you are looking for the cloud-based agent from OpenAI, Codex Web, go to chatgpt.com/codex\n\n\n  \n  \n\n---\n\n## Quickstart\n\n### Installing and running Codex CLI\n\nInstall globally with yo', 'subfield': 'CODING AGENT', 'field': 'AI AGENT', 'id': 'openai/codex', 'content_tag_list': 'GitHub 48.6k', 'thumbnail_picture': 'https://avatars.githubusercontent.com/u/14957082?v=4'}, {'content_name': 'fine dev', 'publisher_id': 'pub-fine-dev', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/ai-agent/pub-fine-dev/fine-dev', 'website': 'https://www.fine.dev', 'github': '', 'review_cnt': '1', 'rating': '4.0', 'description': 'Full-Stack AI Developer Agent for Startups Accelerate your startup’s release schedule and ship daily improvements. Fine is the AI Coding Agent built to act like another team member, getting work done. Try free or watch it work.', 'subfield': 'AI AGENT', 'field': 'AI AGENT', 'id': 'fine-dev/fine-dev', 'content_tag_list': ',AI AGENT', 'thumbnail_picture': 'https://www.fine.dev/_next/static/media/logo.76f79362.svg'}, {'content_name': 'claude code', 'publisher_id': 'pub-claude-code', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-claude-code/claude-code', 'website': 'https://www.anthropic.com/claude-code', 'github': '', 'review_cnt': '1', 'rating': '5.0', 'description': 'Claude Code by Anthropic is an AI-powered coding assistant designed to enhance developer productivity by providing intelligent code generation, debugging, and optimization capabilities. It integrates seamlessly with various developer tools, making it easy to use within existing workflows. Introduction Claude Code is a powerful AI coding assistant developed by Anthropic, aimed at helping developers', 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'claude-code/claude-code', 'content_tag_list': 'CODING AGENT,AI AGENT', 'thumbnail_picture': 'https://static.aiagenta2z.com/scripts/img/ai_service_content/7d6c735321396b42b5c2e46499e17a6e.png'}, {'content_name': 'qwen code', 'publisher_id': 'pub-qwen-code', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-qwen-code/qwen-code', 'website': '', 'github': '', 'review_cnt': '1', 'rating': '4.0', 'description': 'Qwen Coding Agent offers an advanced framework for developing applications utilizing large language models (LLMs), enhancing capabilities such as instruction following, tool usage, planning, and memory. Introduction Qwen Coding Agent provides a robust platform for building LLM applications, leveraging the advanced features of Qwen models (version 2.0 and above). It facilitates the development of i', 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'qwen-code/qwen-code', 'content_tag_list': ',AI AGENT', 'thumbnail_picture': 'https://camo.githubusercontent.com/db544c14e8419ecf289564a79841e005c518fc29cd418ac4a31215e9377eaa22/68747470733a2f2f7169616e77656e2d7265732e6f73732d636e2d6265696a696e672e616c6979756e63732e636f6d2f5177656e322e352f5177656e322e352d436f6465722f7177656e322e352d636f6465722d6c6f676f'}, {'content_name': 'Qoder', 'publisher_id': 'pub-qoder', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-qoder/qoder', 'website': 'https://qoder.com/download', 'github': '', 'review_cnt': '1', 'rating': '5.0', 'description': '\r\n## Qoder\r\n\r\nQoder is an agentic coding platform designed for real software development. It seamlessly integrates enhanced context engineering with intelligent agents to gain a comprehensive understanding of your codebase and systematically tackles software development tasks.\r\nIt goes beyond simple code completion-Qoder helps you think deeper, code smarter, and build better by automating intricat', 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'qoder/qoder', 'content_tag_list': 'coding,agent', 'thumbnail_picture': 'https://static.aiagenta2z.com/scripts/img/ai_service_content/6d7c68c43f520b76251fe2ac7c0061a8.jpg'}, {'content_name': 'CodeBuddy IDE Tencent', 'publisher_id': 'pub-tencent', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-tencent/codebuddy-ide-tencent', 'website': 'https://copilot.tencent.com/ide', 'github': '', 'review_cnt': '1', 'rating': '4.0', 'description': '\r\nCodeBuddy IDE Tencent AI 一站式工作台：从此用自然语言实现产品、设计、研发全流程\r\n\r\n\r\n\r\n## AI 智能代码补全\r\n所想即所码\r\nAI驱动的实时代码预测与智能补全，让编程如行云流水\r\n\r\n\r\n## AI 设计生成\r\n草图秒变原型\r\n手绘概念瞬间转化为高保真交互原型，创意无需等待\r\n\r\n## 设计稿转代码\r\n一稿生万码\r\nFigma设计 稿一键转换为生产级代码，还原度高达99.9%\r\n\r\n\r\n## AI 全栈开发\r\n你的AI编程搭档\r\n智能开发代理，自主完成多文件代码生成与重构\r\n\r\n\r\n## 内置后端服务\r\n后端开 发零门槛\r\n内置腾讯云 CloudBase 和 Supabase服务，数据库与用户认证一键搞定\r\n\r\n## 一键部署分享\r\n开发、部署、分享，一键搞定\r\n内置 Cloud Studio，EdgeOne Pages，从开发环境到在线演示，只需一秒钟', 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'tencent/codebuddy-ide-tencent', 'content_tag_list': 'ide,coding agent', 'thumbnail_picture': 'https://static.aiagenta2z.com/scripts/img/ai_service_content/3ef7d0e02ecbf36eb9dbd25590c7c87e.jpg'}, {'content_name': 'zencoder ai', 'publisher_id': 'pub-zencoder-ai', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/coding-agent/pub-zencoder-ai/zencoder-ai', 'website': 'https://zencoder.ai', 'github': '', 'review_cnt': '0', 'rating': '0.0', 'description': "Zencoder AI provides a range of AI agents designed to streamline and enhance various business processes through intelligent automation and advanced AI capabilities. Introduction Zencoder AI's AI agents are built to act as digital assistants, automating complex tasks and improving operational efficiency across multiple industries. These agents leverage cutting-edge AI technologies to handle tasks s", 'subfield': 'Coding Agent', 'field': 'AI AGENT', 'id': 'zencoder-ai/zencoder-ai', 'content_tag_list': ',AI AGENT', 'thumbnail_picture': 'https://zencoder.ai/favicon.ico'}, {'content_name': 'gpt-engineer', 'publisher_id': 'pub-antonosika', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/ai-agent/pub-antonosika/gpt-engineer', 'website': 'https://github.com/AntonOsika/gpt-engineer', 'github': 'https://github.com/AntonOsika/gpt-engineer', 'review_cnt': '0', 'rating': '0.0', 'description': '# gpt-engineer\n\n](https://github.com/gpt-engineer-org/gpt-engineer)\n](https://discord.gg/8tcDQ89Ej2)\n](https://github.com/gpt-engineer-org/gpt-engineer/blob/main/LICENSE)\n](https://github.com/gpt-engineer-org/gpt-engineer/issues)\n\n](https://twitter.com/antonosika)\n\nThe OG code genereation experimentation platform!\n\nIf you are looking for the evolution that is an opinionated, managed service – chec', 'subfield': 'AI AGENT', 'field': 'AI AGENT', 'id': 'antonosika/gpt-engineer', 'content_tag_list': 'GitHub 55.0k,ai,autonomous-agent,code-generation,codebase-generation,codegen,coding-assistant,gpt-4,gpt-engineer,openai,python', 'thumbnail_picture': 'https://avatars.githubusercontent.com/u/4467025?v=4'}]
-```
+You are welcome to contrib your own customized benchmarks of skills and unique levels system in folder [levels](data/config/levels). 
 
-Search Result of Retrieving registered project id 'AI-Hub-Admin/My-First-AI-Coding-Agent'
 
-```
-Retrieving specific agent with unique ID: AI-Hub-Admin/My-First-AI-Coding-Agent
 
-{'total_hits': 1, 'id': 'AI-Hub-Admin/My-First-AI-Coding-Agent', 'items': [{'content_name': 'My-First-AI-Coding-Agent', 'publisher_id': 'pub-ai-hub-admin', 'detail_url': 'https://www.deepnlp.org/store/ai-agent/ai-agent/pub-ai-hub-admin/my-first-ai-coding-agent', 'website': 'https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent', 'github': 'https://github.com/AI-Hub-Admin/My-First-AI-Coding-Agent', 'review_cnt': '0', 'rating': '0.0', 'description': '# My-First-AI-Coding-Agent\nAI Agent Registry Demo project\n\n', 'subfield': 'AI AGENT', 'field': 'AI AGENT', 'id': 'ai-hub-admin/my-first-ai-coding-agent', 'content_tag_list': 'GitHub 0', 'thumbnail_picture': 'https://avatars.githubusercontent.com/u/184629057?v=4'}]}
-```

package/data/config/hints/base_hints.json

@@ -0,0 +1,118 @@
+{
+  "microsoft/playwright-cli": {
+    "hints": [
+      {
+        "cli": "playwright-cli open [url]",
+        "hint": "open browser, optionally navigate to url"
+      },
+      {
+        "cli": "playwright-cli goto <url> ",
+        "hint": "navigate to a url"
+      }
+    ]
+  },
+  "googleworkspace/cli": {
+    "hints": [
+      {
+        "cli": "gws drive files list --params",
+        "hint": "List the 10 most recent files"
+      }
+    ]
+  },
+  "aiagenta2z/onekey-gateway": {
+    "hints": [
+      {
+        "cli": "onekey agent <unique_id> <api_id> <data_json|@file>",
+        "hint": "Invoke a specific agent by ID with structured JSON data or a file input"
+      },
+      {
+        "cli": "onekey mcp <server_name> [--name config_name]",
+        "hint": "Connect to a Model Context Protocol server to extend agent capabilities"
+      },
+      {
+        "cli": "onekey llm --provider <provider> --model <model> --messages <json|@file> [--temperature <num>] [--response-format <format>] [--options <json|@file>]",
+        "hint": "Execute a direct LLM call with granular control over provider, model, and sampling parameters"
+      },
+      {
+        "cli": "onekey llm --payload <json|@file>",
+        "hint": "Run a raw LLM request using a pre-formatted JSON payload or configuration file"
+      }
+    ]
+  },
+"openai/codex-cli": {
+    "hints": [
+      {
+        "cli": "codex",
+        "hint": "Start an interactive terminal UI (TUI) coding session"
+      },
+      {
+        "cli": "codex exec \"[instruction]\"",
+        "hint": "Run Codex non-interactively and stream results to stdout"
+      }
+    ]
+  },
+  "anthropic/claude-code": {
+    "hints": [
+      {
+        "cli": "claude \"[prompt]\"",
+        "hint": "Start an interactive session with an initial task"
+      },
+      {
+        "cli": "claude --dangerously-skip-permissions",
+        "hint": "Run without interrupting for tool execution permissions"
+      }
+    ]
+  },
+"paul-gauthier/aider": {
+    "hints": [
+      {
+        "cli": "aider --model [model_name]",
+        "hint": "Start aider with a specific LLM context"
+      },
+      {
+        "cli": "/test [command]",
+        "hint": "Run a test; aider will automatically attempt to fix code if it fails"
+      }
+    ]
+  },
+  "openinterpreter/open-interpreter": {
+    "hints": [
+      {
+        "cli": "interpreter",
+        "hint": "Launch the natural language interface to control your local computer"
+      },
+      {
+        "cli": "interpreter --os",
+        "hint": "Enable visual/OS-level control for mouse and keyboard automation"
+      }
+    ]
+  },
+  "google-gemini/gemini-cli": {
+    "hints": [
+      {
+        "cli": "gemini",
+        "hint": "Start an interactive agentic session in the current directory"
+      },
+      {
+        "cli": "gemini -p \"[prompt] @[file/dir]\"",
+        "hint": "Run a prompt with specific file or directory context injected"
+      },
+      {
+        "cli": "gemini --yolo",
+        "hint": "Enable 'YOLO mode' to auto-approve all tool and shell executions"
+      },
+      {
+        "cli": "/memory add \"[fact]\"",
+        "hint": "Save a persistent fact to the project's GEMINI.md hierarchical memory"
+      },
+      {
+        "cli": "/mcp list",
+        "hint": "List all active Model Context Protocol servers and available tools"
+      },
+      {
+        "cli": "/restore",
+        "hint": "List or restore project state checkpoints created before file edits"
+      }
+    ]
+  }
+}