testronaut 1.2.0 → 1.2.2

package/README.md

@@ -1,13 +1,14 @@
 # 🧑‍🚀 Testronaut
 
-**Testronaut** is an autonomous testing framework powered by OpenAI function calling and browser automation. It allows you to define mission-based tests in plain English, then runs them through a real browser to validate UI workflows.
+**Testronaut** is an autonomous testing framework powered by **LLMs and Playwright**.  
+It lets you define *mission-based tests* in plain English, then runs them through a real browser to validate UI workflows — all while generating human-readable reports.
 
 ---
 
 ## 🌌 Join the Mission Control Community
 
-Got questions, ideas, or want to share your missions?  
-Join our Discord to connect with other Testronauts, get support, and help shape the framework’s future.  
+Got questions, ideas, or missions to share?  
+Join the Discord to connect with other Testronauts, get support, and help shape the framework’s future.
 
 [![Join Discord](https://img.shields.io/badge/Join%20Us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/pBfdef92ba)
 
@@ -15,46 +16,52 @@ Join our Discord to connect with other Testronauts, get support, and help shape
 
 ## 🚀 Features
 
-- Run real-world UI flows using missions written as plain strings or functions
-- Execute entire suites with `runMissions`
-- Uses GPT-4o + Playwright to reason and interact with your app
-- Dynamic DOM injection and retry logic
-- Built-in throttling for OpenAI rate limits
-- Modular design — define tools and memory handling as you like
+- Write tests in plain English — no brittle selectors  
+- Runs real browser sessions via **Playwright**  
+- Works with **multiple LLM providers** (OpenAI, Google Gemini, and more coming)  
+- Modular tool and DOM-reasoning system  
+- Dynamic rate-limit and token-tracking logic  
+- Generates JSON + HTML reports automatically  
 
 ---
 
 ## 📖 Documentation
 
 Looking for deeper guides, API references, and examples?  
-Head over to the official Testronaut Docs:
+Check the official docs:
 
 [![Read the Docs](https://img.shields.io/badge/Read%20the%20Docs-2E8555?style=for-the-badge&logo=readthedocs&logoColor=white)](https://docs.testronaut.app)
 
-The docs cover:
-- Getting started step-by-step
-- Writing advanced missions
-- CLI options and configuration
-- Integrating with Mission Control
-- Troubleshooting & FAQs
+Includes:
+- Quickstart and setup  
+- Writing advanced missions  
+- Configuring providers and models  
+- CLI options  
+- Mission Control integration  
+- Troubleshooting and FAQs  
 
 ---
 
 ## 📦 Installation
 
-```
+```bash
 npm install -g testronaut
 ```
-Then initialize with npx, following the instructions:
-```
+
+Then initialize your project:
+
+```bash
 npx testronaut --init
 ```
-run the welcome mission:
-```
+
+Run the sample mission:
+```bash
 npx testronaut welcome.mission.js
 ```
-📁 Directory Structure
-Your project should include a missions/ folder with mission files like:
+
+---
+
+## 📁 Project Structure
 
 ```
 missions/
@@ -62,142 +69,137 @@ missions/
 ├── logout.mission.js
 └── dashboard.mission.js
 ```
-Each file exports a mission string or function and invokes runMissions.
 
-### ✍️ Writing a Mission File
-Create a file in your missions/ directory, e.g. missions/login.mission.js:
+Each mission exports a string or function and calls `runMissions`.
 
-```
+---
+
+## ✍️ Example Mission
+
+```js
 import { runMissions } from 'testronaut';
 
 export const loginMission = `
 Visit ${process.env.URL}.
-Fill the username field with ${process.env.USERNAME}. 
+Fill the username field with ${process.env.USERNAME}.
 Fill the password field with ${process.env.PASSWORD}.
-Take a screenshot.
-Submit the form.
+Click the Login button.
 Wait for the dashboard to appear.
-Take another screenshot.
-Report SUCCESS if the dashboard is loaded, otherwise report FAILURE.
+Take a screenshot.
+Report SUCCESS if the dashboard is loaded, otherwise FAILURE.
 `;
 
 export async function executeMission() {
-  await runMissions({
-    mission: loginMission
-  }, "Login Mission");
+  await runMissions({ mission: loginMission }, "Login Mission");
 }
 ```
 
-Pass in credentials using an .env file. The .env file should also include your Open AI API key (permissions to the completions endpoint required):
+Create a `.env` file with your credentials **and LLM API key** (depending on your chosen provider):
+
+```bash
+# For OpenAI
+OPENAI_API_KEY=sk-...
+
+# Or for Gemini
+GEMINI_API_KEY=AIza...
 
-```
-OPENAI_API_KEY=sk-proj-############
 URL=https://example.com/login
 USERNAME=example@example.com
 PASSWORD=********
 ```
 
-You can chain multiple phases:
+---
 
-```
-import { runMissions } from 'testronaut';
-import { loginMission } from './login.mission.js';
-import { logoutMission } from './logout.mission.js';
-import { navigateToContactFormMission } from './navigateToContactForm.mission.js';
-
-export const fillContactFormMission = 
-`Input the text "example@example.com" in the email field.
-Input the phone number (555)555-5555 into the phone number field.
-Click the submit button.
-Upon submission success there should be a toast notification indicating the form information was saved successfully.
-Take a screenshot.
-If found, report SUCCESS. Otherwise, report FAILURE.`
+## 🧠 LLM Provider Support
 
-export async function executeMission() {
-  await runMissions({
-    preMission: [loginMission, navigateToContactFormMission],
-    mission: fillContactFormMission,
-    postMission: logoutMission,
-  }, "Fill Contact Form Mission");
-}
-```
+Testronaut is provider-agnostic.  
+Choose your preferred LLM at init or via environment variables.
 
-### 🏃 Running Missions
-Run all missions in the missions/ directory:
+```bash
+# During init
+npx testronaut --init
 
+# Or override anytime
+TESTRONAUT_PROVIDER=gemini TESTRONAUT_MODEL=gemini-2.5-pro npx testronaut
 ```
+
+Current supported providers:
+
+| Provider | Example Models |
+|-----------|----------------|
+| **OpenAI** | gpt-4o, gpt-4.1, o3, gpt-5, etc. |
+| **Google Gemini** | gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-8b |
+
+More providers coming soon (Anthropic, Mistral, etc.).
+
+---
+
+## 🏃 Running Missions
+
+Run all missions:
+```bash
 npx testronaut
 ```
-Run a specific file:
 
-```
+Run a specific mission:
+```bash
 npx testronaut login.mission.js
 ```
 
-🧰 Available Helpers
-
-```
-runMissions({ preMission, mission, postMission }) 
+Chain missions together:
+```js
+await runMissions({
+  preMission: [loginMission],
+  mission: fillContactFormMission,
+  postMission: logoutMission,
+}, "Contact Form Flow");
 ```
-– Runs test phases
+---
 
-```
-mission(name, fn) and objective(desc, workflow) 
-```
-– Optional suite syntax
+## 📋 Reports
+
+Testronaut generates both JSON and HTML reports automatically under:
 
 ```
-runSuite(objectives)
+missions/mission_reports/
 ```
-– Executes all registered test objectives
-
-### 📋 Reports
-HTML and JSON reports automatically created in the ```missions/mission_reports/``` directory for each testronaut run
 
-### 🧪 Under the Hood
-Uses Playwright for browser automation
+Each includes:
+- Steps executed
+- Token usage
+- Screenshots
+- Pass/Fail summaries
 
-Interacts with OpenAI’s GPT-4o via Function Calling
+---
 
-Supports modular tool definitions via CHROME_TOOL_MAP
+## 🧪 Under the Hood
 
-🔧 Developing Locally
-Clone and install locally:
+- **Playwright** for browser automation  
+- **LLMs** for reasoning, DOM parsing, and tool use  
+- **Token throttling** + adaptive cooldowns  
+- **Extensible architecture** for custom tools and workflows  
 
-```
-npm install
-npm link
-```
-Now you can use testronaut in any local project.
+---
 
-### 📄 License
-MIT
+## 🧭 Mission Control
 
---
+[Mission Control](https://mission.testronaut.app) lets you:
+- View all reports in one dashboard  
+- Track mission history and success rates  
+- Compare results across environments  
+- Access screenshots and step details anytime  
 
-### 📊 Track Your Missions Over Time
-Want to see your test results evolve, spot trends, and keep all your reports in one place?  
-Sign up for **Mission Control** — your personal dashboard for Testronaut.
+---
 
-With Mission Control you can:
-- 📈 Track your mission reports historically
-- 🔍 Drill into past runs with full step-by-step details
-- 🛠️ Compare results between different builds or environments
-- ☁️ Access reports from anywhere, without digging through local files
+## 📄 License
 
-Getting started is simple:
-1. [![Sign up for Mission Control](https://img.shields.io/badge/Sign%20Up%20for%20Mission%20Control-0A84FF?style=for-the-badge&logo=rocket&logoColor=white)](https://mission.testronaut.app)
-2. Link your account to Testronaut CLI
-3. Watch your mission history grow!
+MIT
 
 ---
 
 ## ☕ Support the Mission
-🤖 Built with ❤️ by [Shane Fast](https://github.com/scfast)
 
-If Testronaut has helped you save time or improve your testing, consider fueling the mission!  
-Your support helps cover hosting, development time, and the occasional coffee needed to debug at 2am.
+🤖 Built with ❤️ by [Shane Fast](https://github.com/scfast)
 
+If Testronaut saves you time, consider fueling the mission:  
 [![Donate](https://img.shields.io/badge/Donate-Coffee%20Fuel%20for%20Testronaut-ff813f?style=for-the-badge&logo=buy-me-a-coffee&logoColor=white)](https://buymeacoffee.com/testronaut)
-
----

package/core/redaction.js

@@ -1,78 +1,206 @@
-// core/redaction.js
+/**
+ * redaction.js
+ * -------------
+ * Purpose:
+ *   Centralized redaction utilities for Testronaut logs and plans.
+ *   - Single source of truth for sensitive keywords
+ *   - Masks tool args when targeting sensitive fields (e.g., #access, password inputs)
+ *   - Masks free-form instructions that reveal credentials
+ *
+ * Design goals:
+ *   - No side effects (pure functions)
+ *   - Easy to extend by editing one keyword list
+ *   - Unit testable via Vitest
+ *
+ * Related tests:
+ *   See `tests/core/redaction.test.js`
+ *
+ * Used by:
+ *   - CLI log/plan renderers
+ *   - Tool call serializers (type/fill)
+ *
+ * Example usage:
+ *   const safeArgs = redactArgs('fill', { selector: '#access', text: 'P@ssw0rd' });
+ *   const safePlan = redactPasswordInText(`Type "P@ssw0rd" into #access`);
+ */
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Sensitive Keywords (Single Source of Truth)
+// Add/remove terms here and every consumer updates automatically.
+// Hyphenated forms are expanded to also match `_` and spaces (api-key/api_key/api key).
+// ─────────────────────────────────────────────────────────────────────────────
+
+export const SENSITIVE_KEYWORDS = [
+  'pass',
+  'password',
+  'pwd',
+  'passwd',
+  'access',
+  'secret',
+  'token',
+  'api-key',
+  'apikey',
+  'bearer',
+  'auth',
+  'pin',
+  'otp',
+];
+
+/**
+ * Builds a single RegExp that treats hyphens as `[-_\s]?` to catch variants.
+ * @returns {RegExp}
+ */
+function buildSensitiveRegex() {
+  // Expand hyphen variants: "api-key" -> "api[-_\s]?key"
+  const parts = SENSITIVE_KEYWORDS.map(k => k.replace(/-/g, '[-_\\s]?'));
+  return new RegExp(`\\b(${parts.join('|')})\\b`, 'i');
+}
 
+const SENSITIVE_RE = buildSensitiveRegex();
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Core helpers
+// ─────────────────────────────────────────────────────────────────────────────
+
+/**
+ * Safe clone: prefer structuredClone, fallback to JSON.
+ * @template T
+ * @param {T} obj
+ * @returns {T}
+ */
 function clone(obj) {
   try { return structuredClone(obj); } catch { return JSON.parse(JSON.stringify(obj ?? {})); }
 }
 
+/**
+ * Returns a masked preview. If showLength is true, includes length for debugging.
+ * @param {unknown} val
+ * @param {boolean} [showLength=true]
+ * @returns {string}
+ */
 export function maskPreview(val, showLength = true) {
   const s = String(val ?? '');
   if (!s) return '••••••';
-  // Show length for debugging without leaking the value
   return showLength ? `•••••• (${s.length})` : '••••••';
 }
 
-function hasPasswordHint(str = '') {
-  return /\b(pass(word)?|pwd|passwd)\b/i.test(String(str || ''));
+/**
+ * True if the provided string contains sensitive hints (password, token, access, etc.).
+ * @param {string} [str='']
+ * @returns {boolean}
+ */
+function hasSensitiveHint(str = '') {
+  return SENSITIVE_RE.test(String(str || ''));
 }
 
-function isPasswordCall(fnName, args = {}) {
+/**
+ * True if an HTML input type itself is sensitive.
+ * @param {string} [t='']
+ * @returns {boolean}
+ */
+function isSensitiveInputType(t = '') {
+  return String(t || '').toLowerCase() === 'password';
+}
+
+/**
+ * Returns true if a tool call (type/fill) looks like it targets a sensitive field.
+ * We check selector/label/placeholder/name/role/testId OR inputType=password.
+ *
+ * @param {'type'|'fill'|string} fnName
+ * @param {object} [args={}]
+ * @returns {boolean}
+ */
+function isSensitiveCall(fnName, args = {}) {
   if (!(fnName === 'type' || fnName === 'fill')) return false;
-  const { selector, label, placeholder, name, inputType, role, testId } = args;
+  const { selector, label, placeholder, name, inputType, role, testId } = args || {};
   return (
-    hasPasswordHint(selector) ||
-    hasPasswordHint(label) ||
-    hasPasswordHint(placeholder) ||
-    hasPasswordHint(name) ||
-    String(inputType || '').toLowerCase() === 'password' ||
-    hasPasswordHint(role) ||
-    hasPasswordHint(testId)
+    hasSensitiveHint(selector) ||
+    hasSensitiveHint(label) ||
+    hasSensitiveHint(placeholder) ||
+    hasSensitiveHint(name) ||
+    isSensitiveInputType(inputType) ||
+    hasSensitiveHint(role) ||
+    hasSensitiveHint(testId)
   );
 }
 
+// ─────────────────────────────────────────────────────────────────────────────
+// Public API
+// ─────────────────────────────────────────────────────────────────────────────
+
 /**
- * Redact ONLY password values in tool args:
- * - Applies to type/fill on password-looking fields
- * - Also masks any arg property explicitly named password/pwd/passwd (shallow)
+ * Redact ONLY secret-like values in tool args (shallow).
+ * - Masks any property whose *key* name looks sensitive (e.g., apiKey, token)
+ * - If the call targets a sensitive field, masks common value props (text/value/input/keys)
+ *
+ * @param {'type'|'fill'|string} fnName
+ * @param {Record<string, unknown>} [args={}]
+ * @param {{ showLength?: boolean }} [opts]
+ * @returns {Record<string, unknown>}
  */
 export function redactArgs(fnName, args = {}, { showLength = true } = {}) {
   const out = clone(args);
 
-  // Mask direct props named password/pwd/passwd if present
+  // Mask direct props with sensitive-looking keys
   for (const k of Object.keys(out || {})) {
-    if (hasPasswordHint(k)) {
+    if (hasSensitiveHint(k)) {
       out[k] = maskPreview(out[k], showLength);
     }
   }
 
-  // For typing into a password field, mask the value
-  if (isPasswordCall(fnName, args)) {
-    if ('text'  in out) out.text  = maskPreview(out.text, showLength);
-    if ('value' in out) out.value = maskPreview(out.value, showLength);
-    if ('input' in out) out.input = maskPreview(out.input, showLength);
-    if ('keys'  in out) out.keys  = maskPreview(out.keys, showLength);
+  // Mask value-like props when the call targets a sensitive field
+  if (isSensitiveCall(fnName, args)) {
+    for (const key of ['text', 'value', 'input', 'keys']) {
+      if (key in out) out[key] = maskPreview(out[key], showLength);
+    }
   }
 
   return out;
 }
 
 /**
- * Redact password literals in freeform mission text.
- * Heuristic: look for the word "password" then the next quoted string OR
- * a token after "with|as|to|=" within ~80 chars.
+ * Redact sensitive literals in freeform mission text.
+ *
+ * Cases:
+ * 1) Sensitive-word ... "VALUE"
+ *    e.g., `password is "hunter2"`
+ *
+ * 2) Sensitive-word ... (with|as|to|=) VALUE
+ *    e.g., `token=abcd1234`, `auth with xyz`
+ *
+ * 3) "VALUE" ... into/in/on/to TARGET  (TARGET contains a sensitive hint)
+ *    e.g., `Type "Imp0st3r123!" into #access`
+ *
+ * @param {string} [text='']
+ * @param {{ showLength?: boolean }} [opts]
+ * @returns {string}
  */
 export function redactPasswordInText(text = '', { showLength = true } = {}) {
   let s = String(text ?? '');
 
-  // Case 1: password ... "VALUE"
+  // Build a reusable source pattern for SENSITIVE_KEYWORDS with hyphen expansion
+  const kwParts = SENSITIVE_KEYWORDS.map(k => k.replace(/-/g, '[-_\\s]?'));
+  const kwGroup = `(${kwParts.join('|')})`;
+
+  // Case 1: keyword ... "VALUE"
   s = s.replace(
-    /(\bpassword\b[^"'\\\n]{0,80}["'])([^"']+)(["'])/gi,
-    (_m, pre, pwd, post) => pre + maskPreview(pwd, showLength) + post
+    new RegExp(`(\\b${kwGroup}\\b[^"'\\\\\\n]{0,80}["'])([^"']+)(["'])`, 'gi'),
+    (_m, pre, _kw, secret, post) => pre + maskPreview(secret, showLength) + post
   );
 
-  // Case 2: password ... (with|as|to|=) VALUE
+  // Case 2: keyword ... (with|as|to|=) VALUE
   s = s.replace(
-    /(\bpassword\b[^.\n]{0,80}?\b(with|as|to|=)\s*)([^\s"'`]+)/gi,
-    (_m, pre, _kw, pwd) => pre + maskPreview(pwd, showLength)
+    new RegExp(`(\\b${kwGroup}\\b[^.\\n]{0,80}?\\b(with|as|to|=)\\s*)([^\\s"'\\\`]+)`, 'gi'),
+    (_m, pre, _kw1, _kw2, secret) => pre + maskPreview(secret, showLength)
+  );
+
+  // Case 3: "VALUE" ... into/in/on/to TARGET (TARGET looks sensitive)
+  s = s.replace(
+    /(["'`])([^"'`]+)\1\s+(?:into|in|on|to)\s+([^\s,.;:]+)/gi,
+    (_m, q, val, target) => {
+      if (hasSensitiveHint(target)) return q + maskPreview(val, showLength) + q + ' into ' + target;
+      return _m;
+    }
   );
 
   return s;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testronaut",
-  "version": "1.2.0",
+  "version": "1.2.2",
   "main": "./runner/testronaut.js",
   "bin": {
     "testronaut": "./bin/cli.js"

package/tests/coreTests/redation.test.js

@@ -0,0 +1,118 @@
+/**
+ * redaction.test.js
+ * -----------------
+ * Purpose:
+ *   Vitest unit tests for redaction utilities:
+ *   - Centralized keyword usage
+ *   - Tool arg masking (selector & key-name based)
+ *   - Free-text masking for plans/instructions
+ *
+ * Test runner:
+ *   Vitest (https://vitest.dev/)
+ *
+ * Location:
+ *   tests/core/redaction.test.js
+ */
+
+import { describe, it, expect } from 'vitest';
+import {
+  SENSITIVE_KEYWORDS,
+  maskPreview,
+  redactArgs,
+  redactPasswordInText,
+} from '../../core/redaction.js';
+
+describe('redaction utilities', () => {
+  describe('SENSITIVE_KEYWORDS (centralized list)', () => {
+    it('contains expected baseline keywords', () => {
+      const expected = ['password', 'token', 'access', 'api-key', 'auth'];
+      for (const k of expected) {
+        expect(SENSITIVE_KEYWORDS).toContain(k);
+      }
+    });
+  });
+
+  describe('maskPreview', () => {
+    it('masks and includes length by default', () => {
+      expect(maskPreview('hunter2')).toBe('•••••• (7)');
+    });
+
+    it('masks empty-ish values without length', () => {
+      expect(maskPreview('')).toBe('••••••');
+      expect(maskPreview(null)).toBe('••••••');
+      expect(maskPreview(undefined)).toBe('••••••');
+    });
+
+    it('respects showLength=false', () => {
+      expect(maskPreview('abcdef', false)).toBe('••••••');
+    });
+  });
+
+  describe('redactArgs', () => {
+    it('masks when selector contains #access', () => {
+      const args = { selector: '#access', text: 'Imp0st3r123!' };
+      const redacted = redactArgs('fill', args);
+      expect(redacted.text).toMatch(/^•••••• \(\d+\)$/);
+      expect(redacted.selector).toBe('#access'); // selector remains visible
+    });
+
+    it('masks when inputType is password', () => {
+      const args = { selector: '#login', inputType: 'password', value: 'superSecret' };
+      const redacted = redactArgs('type', args);
+      expect(redacted.value).toBe('•••••• (11)');
+    });
+
+    it('masks values by sensitive arg key names (e.g., apiKey, token)', () => {
+      const args = { apiKey: 'sk-live-abc', token: 'abcd1234', other: 'notSecret' };
+      const redacted = redactArgs('noop', args); // fnName not type/fill still masks by key name
+      expect(redacted.apiKey).toBe('•••••• (11)');
+      expect(redacted.token).toBe('•••••• (8)');
+      expect(redacted.other).toBe('notSecret');
+    });
+
+    it('does not mask non-sensitive calls', () => {
+      const args = { selector: '#search', text: 'hello world' };
+      const redacted = redactArgs('fill', args);
+      expect(redacted.text).toBe('hello world');
+    });
+  });
+
+  describe('redactPasswordInText', () => {
+    it('Case 1: keyword ... "VALUE"', () => {
+      const input = `My password is "hunter2".`;
+      const out = redactPasswordInText(input);
+      expect(out).toContain(`password is "•••••• (7)"`);
+    });
+
+    it('Case 2: keyword ... (with|as|to|=) VALUE', () => {
+      const input = `token=abcd1234 and auth with xyz`;
+      const out = redactPasswordInText(input);
+      expect(out).toContain(`token=•••••• (8)`);
+      expect(out).toContain(`auth with •••••• (3)`);
+    });
+
+    it('Case 3: "VALUE" into sensitive target (#access)', () => {
+      const input = `Type "Imp0st3r123!" into #access`;
+      const out = redactPasswordInText(input);
+      expect(out).toBe(`Type "•••••• (12)" into #access`);
+    });
+
+    it('non-sensitive text is unchanged', () => {
+      const input = `Click the button and type "hello" into #search`;
+      const out = redactPasswordInText(input);
+      expect(out).toBe(input);
+    });
+
+    it('handles api key forms with spaces/underscores/hyphens', () => {
+      const cases = [
+        `api-key "abcDEF123"`,
+        `api key "abcDEF123"`,
+        `api_key "abcDEF123"`,
+      ];
+      for (const text of cases) {
+        const out = redactPasswordInText(text);
+        expect(out).toContain(`"•••••• (9)"`);
+      }
+    });
+  });
+});