@treasuredata/tdx 0.1.5 → 0.1.6

package/README.md

@@ -93,7 +93,7 @@ tdx chat --new "Tell me about data pipelines"
 1. **CLI flags** (highest priority) - `--database`, `--site`, etc.
 2. **Session context** - Shell-scoped overrides set with `tdx use`
 3. **Project config** - Per-project defaults in `tdx.json`
-4. **Profile config** - Account-specific settings in `~/.config/tdx/profiles/<name>/tdx.json`
+4. **Profile config** - Account-specific settings in `~/.config/tdx/tdx.json` or project `tdx.json`
 5. **Global config** - Fallback defaults in `~/.config/tdx/tdx.json`
 
 ### Profiles
@@ -104,41 +104,55 @@ Profiles store long-lived account configurations for easy switching between envi
 # List all profiles with details
 tdx profiles
 
-# Switch active profile
+# Set session profile (shell-scoped)
 tdx use profile production
 ```
 
 **Profile Structure:**
 
-```
-~/.config/tdx/profiles/
-  ├── production/
-  │   ├── tdx.json      # Configuration (site, database, llm_project, etc.)
-  │   └── .env          # Credentials (TD_API_KEY)
-  └── dev/
-      ├── tdx.json      # Development environment config
-      └── .env          # Development credentials
-```
+Profiles are defined in `tdx.json` files with a `profiles` object:
 
-**Example Production Profile** (`~/.config/tdx/profiles/production/tdx.json`):
 ```json
 {
-  "description": "Production environment for US region",
-  "site": "us01",
-  "database": "analytics",
-  "llm_project": "DataAnalytics"
+  "profiles": {
+    "production": {
+      "description": "Production environment for US region",
+      "site": "us01",
+      "database": "analytics",
+      "llm_project": "DataAnalytics"
+    },
+    "dev": {
+      "description": "Development and testing environment",
+      "site": "jp01",
+      "database": "dev_db"
+    }
+  }
 }
 ```
 
-**Example Development Profile** (`~/.config/tdx/profiles/dev/tdx.json`):
-```json
-{
-  "description": "Development and testing environment",
-  "site": "jp01",
-  "database": "dev_db"
-}
+**Profile Locations:**
+- **User-level**: `~/.config/tdx/tdx.json` - Shared across all projects
+- **Project-level**: `(project folder)/tdx.json` - Project-specific profiles
+
+**Credentials:**
+Store API keys separately from profiles:
+```
+~/.config/tdx/
+  ├── tdx.json              # User-level profiles
+  ├── .env.production       # Production credentials
+  └── .env.dev              # Development credentials
+```
+
+**Example credential file** (`~/.config/tdx/.env.production`):
+```
+TD_API_KEY=1234/abcdefg...
 ```
 
+**Profile Precedence:**
+1. Project `tdx.json` profiles (local)
+2. User `~/.config/tdx/tdx.json` profiles (user-level)
+3. Directory-based profiles (deprecated)
+
 **Note:** The `description` field is optional but recommended for documenting what each profile is for.
 
 ### Session Context
@@ -152,8 +166,11 @@ tdx use database mydb
 # Set session LLM project
 tdx use llm_project Analytics
 
+# View current context
+tdx context
+
 # Clear session context
-tdx use --clear
+tdx context --clear
 ```
 
 **How Session Scope Works:**
@@ -175,7 +192,7 @@ tdx tables  # Uses database: analytics
 tdx tables  # Uses default database (separate session)
 ```
 
-Session context persists until the shell is closed or explicitly cleared with `tdx use --clear`.
+Session context persists until the shell is closed or explicitly cleared with `tdx context --clear`.
 
 **Note:** For advanced session sharing across multiple processes or shells, see [Session ID Option](#session-id-option) in the Advanced Options section.
 
@@ -221,12 +238,12 @@ Example output:
 site: us01 (global: ~/.config/tdx/tdx.json)
 database: analytics (session)
 llm_project: DataAnalytics (profile: production)
-profile: production (active)
+profile: production (session)
 
 Configuration Files:
   Session: /Users/user/.config/tdx/sessions/12345.json ✓
-  Profile: /Users/user/.config/tdx/profiles/production/tdx.json ✓
   Global: /Users/user/.config/tdx/tdx.json ✓
+  Project: /Users/user/projects/myproject/tdx.json ✓
 ```
 
 ## Available Commands
@@ -292,7 +309,7 @@ Simplified top-level command for chatting with LLM agents.
 | Command | Description | Example |
 |---------|-------------|---------|
 | `chat <message>` | Chat (continues last session by default) | `tdx chat "Analyze my data"` |
-| `chat <message> --agent <ref>` | Chat with specific agent | `tdx chat "Hello" --agent "Project/Agent"` |
+| `chat <message> --agent <name>` | Chat with specific agent | `tdx chat "Hello" --agent "Data Analyst"` |
 | `chat <message> --model <name>` | Chat using specific model | `tdx chat "Help" --model "gpt-5"` |
 | `chat <message> --temperature <n>` | Set temperature (0.0-2.0, default: 0.7) | `tdx chat "Be creative" --temperature 1.2` |
 | `chat <message> --new` | Start a new chat session | `tdx chat --new "Fresh conversation"` |
@@ -304,33 +321,33 @@ tdx chat "Show me the top customers by revenue"
 
 # Continue previous conversation (default behavior)
 tdx chat "What about last quarter?"
-tdx chat "And the quarter before that?"
 
 # Start a new conversation
 tdx chat --new "Tell me about data modeling best practices"
 
-# Use specific agent
-tdx chat "Analyze churn" --agent "MyProject/Data Analyst"
-
-# Use specific model (reuses agent per model)
+# Use specific model (auto-creates/reuses agent)
 tdx chat "Help me write SQL" --model "gpt-5"
 
-# Adjust temperature for more creative responses
-tdx chat "Write a poem" --temperature 1.5
+# Use specific agent by name (uses current project context)
+tdx llm use "MyProject"
+tdx chat "Analyze churn" --agent "Data Analyst"
 
-# Combine options
-tdx chat "Be precise" --model "claude-4-sonnet" --temperature 0.3
+# Override project for agent lookup
+tdx chat "Hello" --agent "Analyst" --llm-project "OtherProject"
+
+# Adjust temperature
+tdx chat "Write a poem" --temperature 1.5
 
 # Verbose mode shows chat session ID
 tdx chat "Hello" --verbose
 ```
 
 **How it works**:
-- **Default behavior**: Continues last chat session automatically (or creates new if none exists)
+- **Default behavior**: Continues last chat session automatically
 - **Session persistence**: Last chat ID saved to `.cache/tdx/last_chat_id`
 - **--new flag**: Starts a fresh chat session
-- **--agent**: Use specific agent (requires `"project/agent"` format)
-- **--model**: Reuses or creates agent per model (e.g., `tdx_gpt-5`)
+- **--agent**: Use specific agent by name (resolves using project context)
+- **--model**: Auto-creates/reuses agent per model (e.g., `tdx_gpt-5`)
 - **--temperature**: Controls randomness (0.7 default, lower=focused, higher=creative)
 
 ### Database Commands
@@ -446,20 +463,20 @@ CDP (Customer Data Platform) segment management for audiences and activations.
 
 **Name Format Conventions:**
 - Parent segment: `parent_name` (e.g., `"My Audience"`)
-- Child segment: `parent_name.child_name` (e.g., `"My Audience.High Value Users"`) - uses dot notation
-- Folder: `parent_name.folder_name` (e.g., `"My Audience.Marketing"`)
-- Pattern: Commands accept `(parent_name)(.child_name)?` format
+- Child segment: `parent_name/child_name` (e.g., `"My Audience/High Value Users"`) - uses slash notation
+- Folder: `parent_name/folder_name` (e.g., `"My Audience/Marketing"`)
+- Pattern: Commands accept `(parent_name)(/child_name)?` format
 - **Note**: Names are case-sensitive
 
 | Command | Description | Example |
 |---------|-------------|---------|
 | `segments` | List all parent segments | `tdx segments` |
 | `segments <parent_name>` | List child segments under parent | `tdx segments "My Audience"` |
-| `segment describe <name>` (alias: `desc`) | Show parent or child segment details | `tdx segment describe "My Audience"` or `tdx segment desc "My Audience.Premium Users"` |
-| `segment show <name>` | Execute segment SQL query and show results | `tdx segment show "My Audience"` or `tdx segment show "My Audience.Premium Users" --limit 100` |
-| `segment sql <name>` | Get SQL query for segment | `tdx segment sql "My Audience"` or `tdx segment sql "My Audience.Premium Users"` |
+| `segment describe <name>` (alias: `desc`) | Show parent or child segment details | `tdx segment describe "My Audience"` or `tdx segment desc "My Audience/Premium Users"` |
+| `segment show <name>` | Execute segment SQL query and show results | `tdx segment show "My Audience"` or `tdx segment show "My Audience/Premium Users" --limit 100` |
+| `segment sql <name>` | Get SQL query for segment | `tdx segment sql "My Audience"` or `tdx segment sql "My Audience/Premium Users"` |
 | `segment folders <parent_name>` | List folders under parent | `tdx segment folders "My Audience"` |
-| `segment folder show <parent_name.folder_name>` | Show folder details | `tdx segment folder show "My Audience.Marketing"` |
+| `segment folder show <parent_name/folder_name>` | Show folder details | `tdx segment folder show "My Audience/Marketing"` |
 
 **Examples**:
 ```bash
@@ -474,14 +491,14 @@ tdx segment describe "My Audience"
 # or use the alias
 tdx segment desc "My Audience"
 
-# Show child segment metadata (dot notation)
-tdx segment describe "My Audience.High Value Users"
+# Show child segment metadata (slash notation)
+tdx segment describe "My Audience/High Value Users"
 
 # Execute parent segment query and show results (default: 40 rows)
 tdx segment show "My Audience"
 
-# Execute child segment query with custom limit (dot notation)
-tdx segment show "My Audience.Premium Users" --limit 100
+# Execute child segment query with custom limit (slash notation)
+tdx segment show "My Audience/Premium Users" --limit 100
 
 # Execute segment query and save as JSON
 tdx segment show "My Audience" --format json --output results.json
@@ -489,14 +506,14 @@ tdx segment show "My Audience" --format json --output results.json
 # Get SQL for parent segment
 tdx segment sql "My Audience" > parent-query.sql
 
-# Get SQL for child segment with filtering (dot notation)
-tdx segment sql "My Audience.High Value Users" > child-query.sql
+# Get SQL for child segment with filtering (slash notation)
+tdx segment sql "My Audience/High Value Users" > child-query.sql
 
 # List folders under parent
 tdx segment folders "My Audience"
 
-# Show folder details (parent.folder notation)
-tdx segment folder show "My Audience.Marketing"
+# Show folder details (parent/folder notation)
+tdx segment folder show "My Audience/Marketing"
 ```
 
 ### Activation Commands (CDP)
@@ -505,18 +522,18 @@ Manage CDP activations (syndications) for segments.
 
 | Command | Description | Example |
 |---------|-------------|---------|
-| `activations <parent_name.child_name>` | List activations for a child segment | `tdx activations "My Audience.Premium Users"` |
+| `activations <parent_name/child_name>` | List activations for a child segment | `tdx activations "My Audience/Premium Users"` |
 
 **Examples**:
 ```bash
-# List activations for a child segment (dot notation with names)
-tdx activations "My Audience.Premium Users"
+# List activations for a child segment (slash notation with names)
+tdx activations "My Audience/Premium Users"
 
 # Get activations as JSON
-tdx activations "My Audience.High Value Users" --format json
+tdx activations "My Audience/High Value Users" --format json
 
 # Using Tokyo region
-tdx activations "My Audience.Premium Users" --site jp01
+tdx activations "My Audience/Premium Users" --site jp01
 ```
 
 ### Workflow Commands
@@ -536,6 +553,9 @@ Manage Treasure Data workflows powered by Digdag. Monitor execution sessions, vi
 | `logs <attempt-id> <task-name>` | Show logs for a specific task | `tdx wf logs 67890 +step1` |
 | `kill <attempt-id>` | Kill a running attempt | `tdx wf kill 67890` |
 | `retry <session-id\|attempt-id>` | Retry a session or attempt | `tdx wf retry session:12345` or `tdx wf retry attempt:67890` |
+| `download <project-name> [output-dir]` | Download workflow project from TD console | `tdx wf download myproject` or `tdx wf download myproject ./workflows` |
+| `push [project-name-or-dir]` | Push workflow project to TD console (digdag-style) | `cd myproject && tdx wf push myproject` or `tdx wf push ./myproject` |
+| `delete <project-name-or-id>` | Delete workflow project from TD console | `tdx wf delete myproject` or `tdx wf delete 12345` |
 
 **Workflow-specific options:**
 
@@ -553,6 +573,14 @@ For `tasks` command:
 For `logs` command:
 - `--offset <number>`: Starting offset for logs (default: 0)
 
+For `download` command:
+- `--revision <revision>`: Download specific revision (default: latest)
+
+For `push` command:
+- `--name <name>`: Project name (default: directory name)
+- `--revision <revision>`: Revision name (default: timestamp)
+- `--skip-validation`: Skip validation of .dig files
+
 For `kill` command:
 - `--reason <text>`: Reason for killing the attempt
 - `-y, --yes`: Skip confirmation prompt
@@ -650,88 +678,149 @@ tdx wf retry attempt:67890 --params @params.json
 # Force retry without confirmation
 tdx wf retry attempt:67890 --force -y
 
+# Download workflow project
+tdx wf download myproject
+
+# Download to specific directory
+tdx wf download myproject ./workflows
+
+# Download specific revision
+tdx wf download myproject --revision v1.0.0
+
+# Push workflow project (digdag-style: cd into project directory first)
+cd myproject
+tdx wf push myproject
+
+# Push with custom revision
+cd myproject
+tdx wf push myproject --revision v1.0.0
+
+# Push from parent directory with path
+tdx wf push ./myproject
+
+# Push with --name override
+cd myproject
+tdx wf push myproject --name production_workflow
+
+# Push from current directory without specifying name
+cd myproject
+tdx wf push
+
+# Push without validation
+cd myproject
+tdx wf push myproject --skip-validation
+
+# Delete workflow project (with confirmation)
+tdx wf delete myproject
+
+# Delete by project ID
+tdx wf delete 12345
+
+# Delete without confirmation
+tdx wf delete myproject -y
+
 # Get workflow data as JSON
 tdx wf sessions --format json --output sessions.json
 ```
 
-### Agent Commands (LLM API)
+### LLM Commands
 
-Usage: `tdx agent <subcommand>` | `tdx agents`
+Usage: `tdx llm <subcommand>`
 
-Manage LLM agents and chat sessions with streaming support.
+Manage LLM projects, agents, and chat sessions with streaming support.
 
 | Command | Description | Example |
 |---------|-------------|---------|
-| `agent models` | List available LLM models | `tdx agent models` |
-| `agent projects` | List all projects | `tdx agent projects` |
-| `agents [project-name]` | List all agents (or filter by project) | `tdx agents` or `tdx agents "MyProject"` |
-| `agent get <project/agent>` | Get agent details | `tdx agent get "MyProject/Data Analyst"` |
-| `agent create <name>` | Create a new agent | `tdx agent create "My Agent" --project-id <id> --model <model>` |
-| `agent update <project/agent>` | Update an agent | `tdx agent update "MyProject/Agent" --name "New Name"` |
-| `agent remove <project/agent>` | Delete an agent | `tdx agent remove "MyProject/Agent"` |
-| `agent history [chat-id]` | List chat sessions or show messages | `tdx agent history` or `tdx agent history chat456` |
-
-**Agent Reference Format:**
-
-Agent commands require the full project-scoped format (since agent names are only unique within a project):
-- `"project-name/agent-name"` - Required format for agent operations
-
-**Agent Command Options:**
-
-For `create` command:
-- `--project-id <id>`: Project ID (required)
+| `llm use <project>` | Set current project context | `tdx llm use "MyProject"` |
+| `llm models` | List available LLM models | `tdx llm models` |
+| `llm projects [pattern]` | List all projects (with wildcard support) | `tdx llm projects` or `tdx llm projects "data*"` |
+| `llm project create <name>` | Create a new project | `tdx llm project create "MyProject"` |
+| `llm project delete <name>` | Delete a project | `tdx llm project delete "MyProject"` |
+| `llm agents [pattern]` | List agents in current project (with wildcard support) | `tdx llm agents` or `tdx llm agents "test*"` |
+| `llm agent show <name>` | Show agent details | `tdx llm agent show "Data Analyst"` |
+| `llm agent create <name>` | Create a new agent in current project | `tdx llm agent create "My Agent" --model <model>` |
+| `llm agent update <name>` | Update an agent | `tdx llm agent update "Agent" --name "New Name"` |
+| `llm agent delete <name>` | Delete an agent | `tdx llm agent delete "Agent"` |
+| `llm history [chat-id]` | List chat sessions or show messages | `tdx llm history` or `tdx llm history chat456` |
+
+**Project Resolution (Precedence Order):**
+
+Agent commands resolve projects in this order:
+1. `--llm-project` global option (highest priority)
+2. `tdx llm use <project>` context (session-only)
+3. Default project `tdx_default_<username>` (auto-created)
+
+**Command Options:**
+
+For `project create`:
+- `--description <text>`: Project description
+
+For `agent create`:
 - `--system-prompt <text>`: System prompt/instructions (default: empty)
 - `--model <name>`: Model type (default: claude-4.5-sonnet)
 - `--starter-message <text>`: Starter message for the agent
 - `--max-tool-iterations <n>`: Maximum tool iterations (default: 4)
 - `--temperature <n>`: Temperature 0.0-2.0 (default: 0.7)
 
-For `update` command:
+For `agent update`:
 - `--name <text>`: New agent name
 - `--prompt <text>`: New prompt/instructions
 - `--description <text>`: New description
 - `--starter-message <text>`: New starter message
 
-
 **Examples**:
 ```bash
 # List available models
-tdx agent models
+tdx llm models
 
 # List all projects
-tdx agent projects
+tdx llm projects
+
+# List projects matching pattern (wildcards: * = any chars, ? = single char)
+tdx llm projects "data*"     # Projects starting with 'data'
+tdx llm projects "*_prod"    # Projects ending with '_prod'
 
-# List all agents
-tdx agents
+# Create a new project
+tdx llm project create "MyProject" --description "My data analysis project"
 
-# List agents for a specific project (by project name)
-tdx agents "MyProject"
+# Set current project context (session-only)
+tdx llm use "MyProject"
 
-# Get agent details (requires project/agent format)
-tdx agent get "MyProject/Data Analyst"
+# Now all agent commands use "MyProject" by default:
+tdx llm agents                               # List agents in MyProject
+tdx llm agents "test*"                       # Filter agents by pattern
+tdx llm agent show "Data Analyst"            # Show agent from MyProject
+tdx llm agent create "Analyst" --model gpt-5 # Create in MyProject
+tdx llm agent update "Analyst" --name "Senior Analyst"
+tdx llm agent delete "Analyst"
 
-# Create a new agent
-tdx agent create "Data Analyst" \
-  --prompt "You are a helpful data analyst specialized in SQL and data visualization" \
-  --starter-message "Hello! I can help you analyze your data."
+# Override project with --llm-project option (highest priority)
+tdx llm agents --llm-project "OtherProject"
+tdx llm agent show "Agent" --llm-project "OtherProject"
 
-# Update agent (requires project/agent format)
-tdx agent update "MyProject/Data Analyst" --name "Senior Data Analyst"
+# Use default project (tdx_default_<username>) - no setup needed
+tdx llm agents              # Uses tdx_default_leo if no context set
+tdx llm agent create "Bot"  # Creates in default project
 
-# Update agent prompt
-tdx agent update "MyProject/Data Analyst" --prompt "New instructions"
+# Complete workflow: Create custom agent and chat with it
+tdx llm use "MyProject"
+tdx llm agent create "SQL Expert" \
+  --system-prompt "You are an expert in SQL and data analysis. Help users write efficient queries." \
+  --model "claude-4.5-sonnet"
 
-# Delete agent (requires project/agent format)
-tdx agent remove "MyProject/Data Analyst"
+# Now chat with your custom agent
+tdx chat "Help me analyze customer churn" --agent "SQL Expert"
+tdx chat "Show me the query for that"  # Continues conversation
 
-# List recent chat sessions
-tdx agent history
+# Delete project
+tdx llm project delete "OldProject"
 
-# Show specific chat message history
-tdx agent history chat456
+# Chat history
+tdx llm history              # List recent chat sessions
+tdx llm history chat456      # Show specific chat messages
 
-# Get agents as JSON
-tdx agents --format json --output agents.json
+# JSON output
+tdx llm agents --format json --output agents.json
 ```
 
 ## Advanced Query Features