@altairalabs/promptarena 1.1.2 → 1.1.4

package/README.md

@@ -47,146 +47,69 @@ PromptKit Arena is a comprehensive testing framework for LLM-based applications.
 
 ## Quick Start
 
-1. Create a test configuration:
-
-```yaml
-# arena.yaml
-name: Customer Support Test
-prompts:
-  - name: support-agent
-    system_prompt: |
-      You are a helpful customer support agent.
-      Be professional and empathetic.
-
-conversations:
-  - name: refund-request
-    turns:
-      - role: user
-        content: "I'd like a refund for order #12345"
-      - role: assistant
-        expected_topics: ["refund", "order"]
-
-providers:
-  - type: openai
-    model: gpt-4
-    api_key: ${OPENAI_API_KEY}
-```
-
-2. Run the test:
+Get started in under 2 minutes:
 
 ```bash
-promptarena run -c arena.yaml
-```
-
-3. View the HTML report:
-
-```bash
-open out/report.html
-```
-
-## Features
-
-### Multi-Provider Testing
-
-Test the same prompts across different LLM providers:
-
-```yaml
-providers:
-  - type: openai
-    model: gpt-4
-  - type: anthropic
-    model: claude-3-5-sonnet-20241022
-  - type: google
-    model: gemini-1.5-pro
-```
+# Create a new project from a template
+npx @altairalabs/promptarena init my-test --quick
 
-### Automated Assertions
-
-Validate LLM responses automatically:
-
-```yaml
-turns:
-  - role: assistant
-    assertions:
-      - type: contains
-        value: "refund"
-      - type: tone
-        expected: professional
-      - type: length
-        min: 50
-        max: 500
-```
+# Navigate to your project
+cd my-test
 
-### Self-Play Mode
+# Set your API key (or use mock provider for testing)
+export OPENAI_API_KEY=your-key-here
 
-Let AI agents interact with each other:
+# Run your first test
+npx @altairalabs/promptarena run
 
-```yaml
-self_play:
-  enabled: true
-  rounds: 5
-  agents:
-    - role: customer
-      prompt: "Act as a frustrated customer"
-    - role: support
-      prompt: "Act as a patient support agent"
+# View the HTML report
+open out/report.html
 ```
 
-## How It Works
-
-This npm package downloads pre-built Go binaries from [GitHub Releases](https://github.com/AltairaLabs/PromptKit/releases) during installation. The binaries are:
-
-1. Downloaded for your specific OS and architecture
-2. Extracted from the release archive
-3. Made executable (Unix-like systems)
-4. Invoked through a thin Node.js wrapper
-
-No Go toolchain is required on your machine.
-
-## Supported Platforms
-
-- macOS (Intel and Apple Silicon)
-- Linux (x86_64 and arm64)
-- Windows (x86_64 and arm64)
+That's it! The template includes pre-configured scenarios, assertions, and examples to get you started.
 
-## Documentation
+### Browse Available Templates
 
-- [Full Documentation](https://github.com/AltairaLabs/PromptKit#readme)
-- [Examples](https://github.com/AltairaLabs/PromptKit/tree/main/examples)
-- [Configuration Reference](https://github.com/AltairaLabs/PromptKit/tree/main/docs)
+```bash
+# List all available templates
+npx @altairalabs/promptarena templates list
 
-## Troubleshooting
+# Create from a specific template
+npx @altairalabs/promptarena init my-project --template community/iot-maintenance-demo
 
-### Binary Download Fails
+# Interactive mode (choose template, provider, etc.)
+npx @altairalabs/promptarena init
+```
 
-If the postinstall script fails:
+## Key Features
 
-1. Check your internet connection
-2. Verify the version exists in [GitHub Releases](https://github.com/AltairaLabs/PromptKit/releases)
-3. Check npm proxy/registry settings
-4. Try manual installation:
+- 🎯 **Multi-Provider Testing** - Compare OpenAI, Anthropic, Google, and Azure side-by-side
+- 🔄 **Self-Play Mode** - AI agents simulate realistic user conversations with personas
+- ✅ **Turn-Level Assertions** - Validate individual responses (content, tone, length, JSON)
+- 📊 **Conversation Assertions** - Check patterns across entire conversations
+- 🎭 **Template & Persona System** - Dynamic prompts with variables and reusable personas
+- 🛡️ **Guardrail Testing** - Ensure tools and responses follow safety constraints
+- 📈 **HTML Reports** - Beautiful, detailed reports with cost tracking and metrics
 
-```bash
-# Download binary directly
-curl -L https://github.com/AltairaLabs/PromptKit/releases/download/v0.0.1/PromptKit_v0.0.1_Darwin_arm64.tar.gz -o promptarena.tar.gz
-tar -xzf promptarena.tar.gz promptarena
-chmod +x promptarena
-```
+## Learn More
 
-### Permission Denied
+### Assertion Types
 
-On Unix-like systems:
+- **Turn-Level**: `content_includes`, `content_matches`, `json_schema`, `jsonpath`, `llm_judge`, `tone`, `length`
+- **Conversation-Level**: `llm_judge_conversation`, `tools_not_called_with_args`, `max_tool_calls`
 
-```bash
-chmod +x node_modules/@altairalabs/promptarena/promptarena
-```
+See the [Assertions Guide](https://promptkit.altairalabs.ai/arena/tutorials/05-assertions/) for examples and best practices.
 
-## Alternative Installation Methods
+### Documentation
 
-- **Homebrew**: `brew install altairalabs/tap/promptkit`
-- **Go Install**: `go install github.com/AltairaLabs/PromptKit/tools/arena/cmd/promptarena@latest`
-- **Direct Download**: [GitHub Releases](https://github.com/AltairaLabs/PromptKit/releases)
-- **Build from Source**: Clone repo and run `make install-tools`
+- **[Full Documentation](https://promptkit.altairalabs.ai/)** - Comprehensive guides and tutorials
+- **[Configuration Reference](https://promptkit.altairalabs.ai/arena/reference/config-schema/)** - Complete schema documentation
+- **[Examples](https://github.com/AltairaLabs/PromptKit/tree/main/examples)** - Working examples:
+  - [Assertions Test](https://github.com/AltairaLabs/PromptKit/tree/main/examples/assertions-test) - Turn and conversation-level assertions
+  - [Customer Support](https://github.com/AltairaLabs/PromptKit/tree/main/examples/customer-support) - Self-play with personas
+  - [Variables Demo](https://github.com/AltairaLabs/PromptKit/tree/main/examples/variables-demo) - Template rendering
+  - [LLM Judge](https://github.com/AltairaLabs/PromptKit/tree/main/examples/llm-judge) - AI-powered evaluation
+- **[Multi-Turn Tutorial](https://promptkit.altairalabs.ai/arena/tutorials/03-multi-turn/)** - Self-play patterns
 
 ## License
 
@@ -199,4 +122,4 @@ Contributions welcome! See [CONTRIBUTING.md](https://github.com/AltairaLabs/Prom
 ## Support
 
 - [GitHub Issues](https://github.com/AltairaLabs/PromptKit/issues)
-- [Discussions](https://github.com/AltairaLabs/PromptKit/discussions)
+- [Discussions](https://github.com/AltairaLabs/PromptKit/discussions)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@altairalabs/promptarena",
-  "version": "1.1.2",
+  "version": "1.1.4",
   "type": "module",
   "description": "PromptKit Arena - Multi-turn conversation simulation and testing tool for LLM applications",
   "bin": {
@@ -8,7 +8,8 @@
   },
   "scripts": {
     "postinstall": "node postinstall.js",
-    "test": "node bin/promptarena.js --version"
+    "test": "node bin/promptarena.js --version",
+    "test:init": "node test-init.js"
   },
   "keywords": [
     "llm",