aia 0.9.24 → 0.10.2

data/docs/cli-reference.md

@@ -48,6 +48,24 @@ aia --terse my_prompt
 aia --terse --chat
 ```
 
+### `--tokens`
+Display token usage information after each response in chat mode. Shows input tokens, output tokens, and model ID.
+
+```bash
+aia --chat --tokens
+aia --chat --tokens --model gpt-4
+```
+
+### `--cost`
+Include cost calculations with token usage. Automatically enables `--tokens`. Shows estimated cost based on the model's pricing.
+
+```bash
+aia --chat --cost
+aia --chat --cost --model gpt-4,claude-3-sonnet
+```
+
+**Note**: `--cost` implies `--tokens`, so you don't need to specify both.
+
 ## Adapter Options
 
 ### `--adapter ADAPTER`
@@ -59,24 +77,24 @@ aia --adapter ruby_llm
 
 **Valid adapters**: `ruby_llm`
 
-### `--available_models [QUERY]`
+### `--available-models [QUERY]`
 List (then exit) available models that match the optional query. Query is a comma-separated list of AND components.
 
 ```bash
 # List all models
-aia --available_models
+aia --available-models
 
 # Filter by provider
-aia --available_models openai
+aia --available-models openai
 
 # Filter by capability and provider
-aia --available_models openai,mini
+aia --available-models openai,mini
 
 # Filter by modality
-aia --available_models text_to_text
+aia --available-models text_to_text
 
 # Complex filter
-aia --available_models openai,gpt,text_to_image
+aia --available-models openai,gpt,text_to_image
 ```
 
 ## Model Options
@@ -122,47 +140,47 @@ aia --model "gpt-4,claude-3-sonnet" --consensus my_prompt
 aia --model "gpt-4,claude-3-sonnet" --no-consensus my_prompt
 ```
 
-### `--sm, --speech_model MODEL`
+### `--sm, --speech-model MODEL`
 Speech model to use for text-to-speech functionality.
 
 ```bash
-aia --speech_model tts-1 --speak my_prompt
+aia --speech-model tts-1 --speak my_prompt
 aia --sm tts-1-hd --speak my_prompt
 ```
 
-### `--tm, --transcription_model MODEL`
+### `--tm, --transcription-model MODEL`
 Transcription model to use for speech-to-text functionality.
 
 ```bash
-aia --transcription_model whisper-1 audio_file.wav
+aia --transcription-model whisper-1 audio_file.wav
 aia --tm whisper-1 my_audio.mp3
 ```
 
 ## File Options
 
-### `-c, --config_file FILE`
+### `-c, --config-file FILE`
 Load configuration from a specific file.
 
 ```bash
-aia --config_file /path/to/config.yml my_prompt
+aia --config-file /path/to/config.yml my_prompt
 aia -c ~/.aia/custom_config.yml my_prompt
 ```
 
-### `-o, --[no-]out_file [FILE]`
+### `-o, --[no-]output [FILE]`
 Output file for saving AI responses.
 
 ```bash
 # Save to default file (temp.md)
-aia --out_file my_prompt
+aia --output my_prompt
 
 # Save to specific file
-aia --out_file output.txt my_prompt
+aia --output output.txt my_prompt
 
 # Use absolute path
-aia --out_file /tmp/ai_response.md my_prompt
+aia --output /tmp/ai_response.md my_prompt
 
 # Disable file output
-aia --no-out_file my_prompt
+aia --no-output my_prompt
 ```
 
 ### `-a, --[no-]append`
@@ -170,24 +188,24 @@ Append to output file instead of overwriting.
 
 ```bash
 # Append mode
-aia --out_file log.md --append my_prompt
+aia --output log.md --append my_prompt
 
 # Overwrite mode (default)
-aia --out_file log.md --no-append my_prompt
+aia --output log.md --no-append my_prompt
 ```
 
-### `-l, --[no-]log_file [FILE]`
-Log file for AIA operations.
+### `--[no-]history-file [FILE]`
+Conversation history file for logging prompts and responses.
 
 ```bash
-# Enable logging to default location
-aia --log_file my_prompt
+# Enable history logging to default location
+aia --history-file my_prompt
 
 # Log to specific file
-aia --log_file /var/log/aia.log my_prompt
+aia --history-file /var/log/aia_history.log my_prompt
 
-# Disable logging
-aia --no-log_file my_prompt
+# Disable history logging
+aia --no-history-file my_prompt
 ```
 
 ### `--md, --[no-]markdown`
@@ -203,20 +221,20 @@ aia --no-markdown my_prompt
 
 ## Prompt Options
 
-### `--prompts_dir DIR`
+### `--prompts-dir DIR`
 Directory containing prompt files.
 
 ```bash
-aia --prompts_dir /custom/prompts my_prompt
-aia --prompts_dir ~/work/prompts my_prompt
+aia --prompts-dir /custom/prompts my_prompt
+aia --prompts-dir ~/work/prompts my_prompt
 ```
 
-### `--roles_prefix PREFIX`
+### `--roles-prefix PREFIX`
 Subdirectory name for role files (default: `roles`).
 
 ```bash
 # Use custom roles directory
-aia --roles_prefix personas --role expert
+aia --roles-prefix personas --role expert
 
 # Results in looking for roles in ~/.prompts/personas/expert.txt
 ```
@@ -260,12 +278,12 @@ aia --list-roles
 
 Roles are discovered from:
 - **Default location**: `~/.prompts/roles/`
-- **Custom location**: Set via `--prompts_dir` and `--roles_prefix`
+- **Custom location**: Set via `--prompts-dir` and `--roles-prefix`
 - **Nested directories**: Supports subdirectories like `roles/software/architect.txt`
 
 **Use case**: Discover available roles before using them with `--role` or inline `MODEL=ROLE` syntax.
 
-**See also**: `--role`, `--model`, `--prompts_dir`, `--roles_prefix`
+**See also**: `--role`, `--model`, `--prompts-dir`, `--roles-prefix`
 
 ### `-n, --next PROMPT_ID`
 Next prompt to process (can be used multiple times to build a pipeline).
@@ -294,12 +312,12 @@ aia --exec my_script_prompt
 aia --no-exec my_script_prompt
 ```
 
-### `--system_prompt PROMPT_ID`
+### `--system-prompt PROMPT_ID`
 System prompt ID to use for chat sessions.
 
 ```bash
-aia --system_prompt helpful_assistant --chat
-aia --system_prompt code_expert --chat my_code.py
+aia --system-prompt helpful_assistant --chat
+aia --system-prompt code_expert --chat my_code.py
 ```
 
 ### `--regex PATTERN`
@@ -329,42 +347,42 @@ aia --temperature 1.5 creative_writing
 aia -t 2.0 brainstorm_ideas
 ```
 
-### `--max_tokens TOKENS`
+### `--max-tokens TOKENS`
 Maximum tokens for text generation.
 
 ```bash
-aia --max_tokens 100 short_summary
-aia --max_tokens 4000 detailed_analysis
+aia --max-tokens 100 short_summary
+aia --max-tokens 4000 detailed_analysis
 ```
 
-### `--top_p VALUE`
+### `--top-p VALUE`
 Top-p sampling value (0.0 to 1.0). Alternative to temperature for controlling randomness.
 
 ```bash
-aia --top_p 0.1 precise_answer
-aia --top_p 0.9 creative_response
+aia --top-p 0.1 precise_answer
+aia --top-p 0.9 creative_response
 ```
 
-### `--frequency_penalty VALUE`
+### `--frequency-penalty VALUE`
 Frequency penalty (-2.0 to 2.0). Positive values discourage repetition.
 
 ```bash
 # Discourage repetition
-aia --frequency_penalty 0.5 my_prompt
+aia --frequency-penalty 0.5 my_prompt
 
 # Encourage repetition
-aia --frequency_penalty -0.5 my_prompt
+aia --frequency-penalty -0.5 my_prompt
 ```
 
-### `--presence_penalty VALUE`
+### `--presence-penalty VALUE`
 Presence penalty (-2.0 to 2.0). Positive values encourage discussing new topics.
 
 ```bash
 # Encourage new topics
-aia --presence_penalty 0.5 broad_discussion
+aia --presence-penalty 0.5 broad_discussion
 
 # Focus on current topics
-aia --presence_penalty -0.5 deep_dive
+aia --presence-penalty -0.5 deep_dive
 ```
 
 ## Audio/Image Options
@@ -389,32 +407,32 @@ aia --voice onyx --speak my_prompt
 aia --voice shimmer --speak my_prompt
 ```
 
-### `--is, --image_size SIZE`
+### `--is, --image-size SIZE`
 Image size for image generation.
 
 ```bash
-aia --image_size 1024x1024 image_prompt
+aia --image-size 1024x1024 image_prompt
 aia --is 1792x1024 wide_image
 aia --is 1024x1792 tall_image
 ```
 
 **Common sizes**: `256x256`, `512x512`, `1024x1024`, `1792x1024`, `1024x1792`
 
-### `--iq, --image_quality QUALITY`
+### `--iq, --image-quality QUALITY`
 Image quality for image generation.
 
 ```bash
-aia --image_quality standard image_prompt
+aia --image-quality standard image_prompt
 aia --iq hd high_quality_image
 ```
 
 **Values**: `standard`, `hd`
 
-### `--style, --image_style STYLE`
+### `--style, --image-style STYLE`
 Style for image generation.
 
 ```bash
-aia --image_style vivid colorful_image
+aia --image-style vivid colorful_image
 aia --style natural realistic_image
 ```
 
@@ -447,28 +465,37 @@ aia --tools ./tools/ my_prompt
 aia --tools "./tools/,./special_tool.rb" my_prompt
 ```
 
-### `--at, --allowed_tools TOOLS_LIST`
+### `--at, --allowed-tools TOOLS_LIST`
 Allow only these tools to be used. Security feature to restrict tool access.
 
 ```bash
 # Allow specific tools
-aia --allowed_tools "calculator,file_reader" my_prompt
+aia --allowed-tools "calculator,file_reader" my_prompt
 aia --at "web_scraper,data_analyzer" analysis_prompt
 ```
 
-### `--rt, --rejected_tools TOOLS_LIST`
+### `--rt, --rejected-tools TOOLS_LIST`
 Reject/block these tools from being used.
 
 ```bash
 # Block dangerous tools
-aia --rejected_tools "file_writer,system_command" my_prompt
+aia --rejected-tools "file_writer,system_command" my_prompt
 aia --rt "network_access" secure_prompt
 ```
 
 ## Utility Options
 
+### Log Level Options
+
+AIA provides multiple log level options to control logging verbosity. These options set the log level for all three loggers:
+- **aia**: Used within the AIA codebase for application-level logging
+- **llm**: Passed to the RubyLLM gem's configuration (`RubyLLM.logger`)
+- **mcp**: Passed to the RubyLLM::MCP process (`RubyLLM::MCP.logger`)
+
+Only one log level option should be used at a time.
+
 ### `-d, --debug`
-Enable debug output for troubleshooting.
+Enable debug output (most verbose) and set all loggers to DEBUG level. Also sets `$DEBUG_ME = true` for the debug_me gem.
 
 ```bash
 aia --debug my_prompt
@@ -476,12 +503,49 @@ aia -d --chat
 ```
 
 ### `--no-debug`
-Explicitly disable debug output.
+Explicitly disable debug output. Sets `$DEBUG_ME = false`.
 
 ```bash
 aia --no-debug my_prompt
 ```
 
+### `--info`
+Set all loggers to INFO level. Shows informational messages and above.
+
+```bash
+aia --info my_prompt
+aia --info --chat
+```
+
+### `--warn`
+Set all loggers to WARN level (this is the default). Shows warnings, errors, and fatal messages.
+
+```bash
+aia --warn my_prompt
+```
+
+### `--error`
+Set all loggers to ERROR level. Shows only errors and fatal messages.
+
+```bash
+aia --error my_prompt
+aia --error --chat
+```
+
+### `--fatal`
+Set all loggers to FATAL level (least verbose). Shows only critical/fatal messages.
+
+```bash
+aia --fatal my_prompt
+```
+
+**Log Level Hierarchy** (from most to least verbose):
+1. `debug` - All messages including detailed debugging information
+2. `info` - Informational messages and above
+3. `warn` - Warnings, errors, and fatal messages (default)
+4. `error` - Only errors and fatal messages
+5. `fatal` - Only critical/fatal messages
+
 ### `-v, --[no-]verbose`
 Enable/disable verbose output.
 
@@ -575,10 +639,10 @@ aia --model "gpt-4,claude-3-sonnet" --consensus analysis_prompt data.csv
 aia --model gpt-4 --temperature 1.2 --speak --voice nova story_prompt
 
 # Secure tool usage
-aia --tools ./safe_tools/ --allowed_tools "calculator,file_reader" --rejected_tools "system_command" analysis_prompt
+aia --tools ./safe_tools/ --allowed-tools "calculator,file_reader" --rejected-tools "system_command" analysis_prompt
 
 # Pipeline with custom configuration
-aia --pipeline "extract,analyze,summarize" --temperature 0.3 --max_tokens 2000 --out_file report.md data_source.txt
+aia --pipeline "extract,analyze,summarize" --temperature 0.3 --max-tokens 2000 --output report.md data_source.txt
 
 # Debug mode with verbose output
 aia --debug --verbose --model claude-3-sonnet problematic_prompt
@@ -588,13 +652,13 @@ aia --debug --verbose --model claude-3-sonnet problematic_prompt
 
 ```bash
 # Use custom configuration
-aia --config_file ./project_config.yml --prompts_dir ./project_prompts/ my_prompt
+aia --config-file ./project_config.yml --prompts-dir ./project_prompts/ my_prompt
 
 # Save output with markdown formatting
-aia --out_file analysis.md --markdown --append data_analysis dataset.csv
+aia --output analysis.md --markdown --append data_analysis dataset.csv
 
 # Audio processing
-aia --transcription_model whisper-1 --speech_model tts-1-hd --voice echo audio_prompt audio_file.wav
+aia --transcription-model whisper-1 --speech-model tts-1-hd --voice echo audio_prompt audio_file.wav
 ```
 
 ## Exit Codes
@@ -607,10 +671,11 @@ aia --transcription_model whisper-1 --speech_model tts-1-hd --voice echo audio_p
 
 ## Environment Variables
 
-Many CLI options have corresponding environment variables with the `AIA_` prefix:
+Many CLI options have corresponding environment variables with the `AIA_` prefix.
+Use double underscore (`__`) for nested configuration sections:
 
 ```bash
-# Basic model configuration
+# Model configuration (top-level)
 export AIA_MODEL="gpt-4"
 
 # Model with inline role syntax
@@ -619,11 +684,19 @@ export AIA_MODEL="gpt-4o=architect"
 # Multiple models with roles
 export AIA_MODEL="gpt-4o=architect,claude=security,gemini=performance"
 
-# Other configuration
-export AIA_TEMPERATURE="0.8"
-export AIA_PROMPTS_DIR="/custom/prompts"
-export AIA_VERBOSE="true"
-export AIA_DEBUG="false"
+# LLM settings (nested under llm:)
+export AIA_LLM__TEMPERATURE="0.8"
+
+# Prompts settings (nested under prompts:)
+export AIA_PROMPTS__DIR="/custom/prompts"
+
+# Flags (nested under flags:)
+export AIA_FLAGS__VERBOSE="true"
+export AIA_FLAGS__DEBUG="false"
+export AIA_FLAGS__CHAT="true"
+
+# Output settings (nested under output:)
+export AIA_OUTPUT__FILE="./output.md"
 ```
 
 **Note**: The `AIA_MODEL` environment variable supports the same inline `MODEL=ROLE` syntax as the `--model` CLI option.

data/docs/configuration.md

@@ -35,7 +35,7 @@ frequency_penalty: 0.0        # Repetition penalty (-2.0 to 2.0)
 presence_penalty: 0.0         # Topic penalty (-2.0 to 2.0)
 
 # Output Settings
-out_file: null                # Output file (null = no file output)
+output: null                  # Output file (null = no file output)
 append: false                 # Append to output file instead of overwriting
 markdown: true                # Format output with Markdown
 verbose: false                # Show detailed output
@@ -72,8 +72,23 @@ pipeline: []                  # Default prompt pipeline
 executable_prompt: false     # Run prompts as executables
 
 # Logging
-log_file: null               # Log file path
+history_file: null           # Conversation history file for logging prompts and responses
 refresh: 7                   # Model database refresh interval (days)
+
+# Logger Configuration (for detailed logging control)
+logger:
+  aia:                       # AIA application logging
+    file: STDOUT             # STDOUT, STDERR, or a file path
+    level: warn              # debug, info, warn, error, fatal
+    flush: true              # Immediate write (no buffering)
+  llm:                       # RubyLLM gem logging
+    file: STDOUT
+    level: warn
+    flush: true
+  mcp:                       # RubyLLM::MCP gem logging
+    file: STDOUT
+    level: warn
+    flush: true
 ```
 
 ### Model-Specific Configuration
@@ -95,27 +110,48 @@ max_tokens: 8000
 
 ## Environment Variables
 
-All configuration options can be set via environment variables with the `AIA_` prefix:
+All configuration options can be set via environment variables with the `AIA_` prefix.
+Use double underscore (`__`) for nested configuration sections:
 
 ```bash
-# Core settings
+# LLM settings (nested under llm:)
+export AIA_LLM__TEMPERATURE="0.8"
+export AIA_LLM__ADAPTER="ruby_llm"
+
+# Models (top-level array, supports MODEL=ROLE syntax)
 export AIA_MODEL="gpt-4"
-export AIA_TEMPERATURE="0.8"
-export AIA_PROMPTS_DIR="/path/to/my/prompts"
+export AIA_MODEL="gpt-4o=architect,claude=reviewer"
+
+# Prompts settings (nested under prompts:)
+export AIA_PROMPTS__DIR="/path/to/my/prompts"
+export AIA_PROMPTS__ROLES_PREFIX="roles"
 
 # API Keys (handled by RubyLLM)
 export OPENAI_API_KEY="your_key_here"
-export ANTHROPIC_API_KEY="your_key_here" 
+export ANTHROPIC_API_KEY="your_key_here"
 export GOOGLE_API_KEY="your_key_here"
 
-# Chat settings
-export AIA_CHAT="true"
-export AIA_VERBOSE="true"
-export AIA_DEBUG="false"
+# Flags (nested under flags:)
+export AIA_FLAGS__CHAT="true"
+export AIA_FLAGS__VERBOSE="true"
+export AIA_FLAGS__DEBUG="false"
+export AIA_FLAGS__TOKENS="true"
+export AIA_FLAGS__COST="true"
+
+# Output settings (nested under output:)
+export AIA_OUTPUT__FILE="/tmp/aia_output.md"
+export AIA_OUTPUT__MARKDOWN="true"
+export AIA_OUTPUT__LOG_FILE="~/.prompts/_prompts.log"
+
+# Tools settings (nested under tools:)
+export AIA_TOOLS__PATHS="/path/to/tools"
+export AIA_TOOLS__REJECTED="dangerous_tool"
 
-# Output settings
-export AIA_OUT_FILE="/tmp/aia_output.md"
-export AIA_MARKDOWN="true"
+# Registry settings (nested under registry:)
+export AIA_REGISTRY__REFRESH="7"
+
+# Paths settings (nested under paths:)
+export AIA_PATHS__AIA_DIR="~/.aia"
 ```
 
 ## Command Line Arguments
@@ -134,6 +170,107 @@ Prompts can contain configuration directives that override all other settings:
 Write a creative story about...
 ```
 
+## Logger Configuration
+
+AIA uses the Lumberjack gem for logging and manages three separate loggers:
+
+| Logger | Purpose |
+|--------|---------|
+| `aia` | Used within the AIA codebase for application-level logging |
+| `llm` | Passed to the RubyLLM gem's configuration (`RubyLLM.logger`) |
+| `mcp` | Passed to the RubyLLM::MCP process (`RubyLLM::MCP.logger`) |
+
+### Configuration File Settings
+
+Each logger can be configured independently in your `~/.aia/config.yml`:
+
+```yaml
+logger:
+  aia:
+    file: STDOUT           # STDOUT, STDERR, or a file path (e.g., ~/.aia/aia.log)
+    level: warn            # debug, info, warn, error, fatal
+    flush: true            # true = immediate write, false = buffered
+  llm:
+    file: STDOUT
+    level: warn
+    flush: true
+  mcp:
+    file: STDOUT
+    level: warn
+    flush: true
+```
+
+**Note**: All three loggers can safely write to the same file path. AIA handles multi-process safe file writes with automatic log file rotation (daily).
+
+### CLI Log Level Override
+
+Command-line log level options override the config file settings for ALL loggers:
+
+```bash
+# Set all loggers to debug level
+aia --debug my_prompt
+
+# Set all loggers to info level
+aia --info my_prompt
+
+# Set all loggers to warn level (default)
+aia --warn my_prompt
+
+# Set all loggers to error level
+aia --error my_prompt
+
+# Set all loggers to fatal level
+aia --fatal my_prompt
+```
+
+### Environment Variables
+
+Logger settings can also be configured via environment variables:
+
+```bash
+# AIA logger settings
+export AIA_LOGGER__AIA__FILE="~/.aia/aia.log"
+export AIA_LOGGER__AIA__LEVEL="debug"
+export AIA_LOGGER__AIA__FLUSH="true"
+
+# LLM logger settings
+export AIA_LOGGER__LLM__FILE="STDOUT"
+export AIA_LOGGER__LLM__LEVEL="info"
+
+# MCP logger settings
+export AIA_LOGGER__MCP__FILE="STDERR"
+export AIA_LOGGER__MCP__LEVEL="warn"
+```
+
+### Log Levels
+
+| Level | Description |
+|-------|-------------|
+| `debug` | Most verbose - all messages including detailed debugging info |
+| `info` | Informational messages and above |
+| `warn` | Warnings, errors, and fatal messages (default) |
+| `error` | Only errors and fatal messages |
+| `fatal` | Least verbose - only critical/fatal messages |
+
+### Example: File-Based Logging
+
+```yaml
+# ~/.aia/config.yml - Log everything to files
+logger:
+  aia:
+    file: ~/.aia/logs/aia.log
+    level: info
+    flush: true
+  llm:
+    file: ~/.aia/logs/llm.log
+    level: debug
+    flush: false
+  mcp:
+    file: ~/.aia/logs/mcp.log
+    level: warn
+    flush: true
+```
+
 ## Advanced Configuration
 
 ### Multi-Model Configuration
@@ -227,7 +364,7 @@ model: gpt-4
 temperature: 0.3
 verbose: true
 debug: true
-out_file: ./dev_output.md
+output: ./dev_output.md
 prompts_dir: ./prompts
 tool_paths: [./tools]
 ```
@@ -241,7 +378,7 @@ model: gpt-3.5-turbo
 temperature: 0.7
 verbose: false
 debug: false
-log_file: /var/log/aia.log
+history_file: /var/log/aia_history.log
 prompts_dir: /etc/aia/prompts
 tool_paths: [/usr/share/aia-tools]
 allowed_tools: [safe_calculator, file_reader]
@@ -258,7 +395,7 @@ max_tokens: 4000
 speak: true
 voice: nova
 markdown: true
-out_file: ~/writing/aia_output.md
+output: ~/writing/aia_output.md
 append: true
 ```
 
@@ -276,7 +413,7 @@ aia --dump config.yaml
 
 ```bash
 # Test model access
-aia --available_models
+aia --available-models
 
 # Test configuration
 aia --debug --verbose hello_world
@@ -289,7 +426,7 @@ aia --tools ./my_tools --debug test_prompt
 
 #### Model Not Found
 - Check your API keys are set
-- Verify the model name: `aia --available_models`
+- Verify the model name: `aia --available-models`
 - Check network connectivity
 
 #### Permission Errors