sentience-ts 0.10.5 → 0.11.0

package/LICENSE.md

@@ -0,0 +1,43 @@
+# Elastic License
+## Acceptance
+By using the software, you agree to all of the terms and conditions below.
+
+## Copyright License
+The licensor grants you a non-exclusive, royalty-free, worldwide, non-sublicensable, non-transferable license to use, copy, distribute, make available, and prepare derivative works of the software, in each case subject to the limitations and conditions below.
+
+## Limitations
+You may not provide the software to third parties as a hosted or managed service, where the service provides users with access to any substantial set of the features or functionality of the software.
+
+You may not move, change, disable, or circumvent the license key functionality in the software, and you may not remove or obscure any functionality in the software that is protected by the license key.
+
+You may not alter, remove, or obscure any licensing, copyright, or other notices of the licensor in the software. Any use of the licensor’s trademarks is subject to applicable law.
+
+## Patents
+The licensor grants you a license, under any patent claims the licensor can license, or becomes able to license, to make, have made, use, sell, offer for sale, import and have imported the software, in each case subject to the limitations and conditions in this license. This license does not cover any patent claims that you cause to be infringed by modifications or additions to the software. If you or your company make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
+
+## Notices
+You must ensure that anyone who gets a copy of any part of the software from you also gets a copy of these terms.
+
+If you modify the software, you must include in any modified copies of the software prominent notices stating that you have modified the software.
+
+## No Other Rights
+These terms do not imply any licenses other than those expressly granted in these terms.
+
+## Termination
+If you use the software in violation of these terms, such use is not licensed, and your licenses will automatically terminate. If the licensor provides you with a notice of your violation, and you cease all violation of this license no later than 30 days after you receive that notice, your licenses will be reinstated retroactively. However, if you violate these terms after such reinstatement, any additional violation of these terms will cause your licenses to terminate automatically and permanently.
+
+## No Liability
+As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.
+
+## Definitions
+The **licensor** is the entity offering these terms, and the **software** is the software the licensor makes available under these terms, including any portion of it.
+
+you refers to the individual or entity agreeing to these terms.
+
+**your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization. control means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
+
+**your licenses** are all the licenses granted to you for the software under these terms.
+
+**use** means anything you do with the software requiring one of your licenses.
+
+**trademark** means trademarks, service marks, and similar rights.

package/README.md

@@ -1,8 +1,6 @@
 # Sentience TypeScript SDK
 
-**📜 License**: Apache License 2.0
-
-TypeScript/JavaScript SDK for Sentience AI Agent Browser Automation. Build intelligent web automation agents that can see, understand, and interact with web pages like humans do.
+The SDK is open under ELv2; the core semantic geometry and reliability logic runs in Sentience-hosted services.
 
 ## Installation
 
@@ -14,7 +12,194 @@ npm run build
 npx playwright install chromium
 ```
 
-## Quick Start
+## Quick Start: Choose Your Abstraction Level
+
+Sentience SDK offers **4 levels of abstraction** - choose based on your needs:
+
+### 💬 Level 4: Conversational Agent (Highest Abstraction) - **NEW in v0.3.0**
+
+Complete automation with natural conversation. Just describe what you want, and the agent plans and executes everything:
+
+```typescript
+import { SentienceBrowser, ConversationalAgent, OpenAIProvider } from 'sentience-ts';
+
+const browser = await SentienceBrowser.create({ apiKey: process.env.SENTIENCE_API_KEY });
+const llm = new OpenAIProvider(process.env.OPENAI_API_KEY!, 'gpt-4o');
+const agent = new ConversationalAgent({ llmProvider: llm, browser });
+
+// Navigate to starting page
+await browser.getPage().goto('https://amazon.com');
+
+// ONE command does it all - automatic planning and execution!
+const response = await agent.execute(
+  "Search for 'wireless mouse' and tell me the price of the top result"
+);
+console.log(response); // "I found the top result for wireless mouse on Amazon. It's priced at $24.99..."
+
+// Follow-up questions maintain context
+const followUp = await agent.chat("Add it to cart");
+console.log(followUp);
+
+await browser.close();
+```
+
+**When to use:** Complex multi-step tasks, conversational interfaces, maximum convenience
+**Code reduction:** 99% less code - describe goals in natural language
+**Requirements:** OpenAI or Anthropic API key
+
+### 🤖 Level 3: Agent (Natural Language Commands) - **Recommended for Most Users**
+
+Zero coding knowledge needed. Just write what you want in plain English:
+
+```typescript
+import { SentienceBrowser, SentienceAgent, OpenAIProvider } from 'sentience-ts';
+
+const browser = await SentienceBrowser.create({ apiKey: process.env.SENTIENCE_API_KEY });
+const llm = new OpenAIProvider(process.env.OPENAI_API_KEY!, 'gpt-4o-mini');
+const agent = new SentienceAgent(browser, llm);
+
+await browser.getPage().goto('https://www.amazon.com');
+
+// Just natural language commands - agent handles everything!
+await agent.act('Click the search box');
+await agent.act("Type 'wireless mouse' into the search field");
+await agent.act('Press Enter key');
+await agent.act('Click the first product result');
+
+// Automatic token tracking
+console.log(`Tokens used: ${agent.getTokenStats().totalTokens}`);
+await browser.close();
+```
+
+**When to use:** Quick automation, non-technical users, rapid prototyping
+**Code reduction:** 95-98% less code vs manual approach
+**Requirements:** OpenAI API key (or Anthropic for Claude)
+
+### 🔧 Level 2: Direct SDK (Technical Control)
+
+Full control with semantic selectors. For technical users who want precision:
+
+```typescript
+import { SentienceBrowser, snapshot, find, click, typeText, press } from 'sentience-ts';
+
+const browser = await SentienceBrowser.create({ apiKey: process.env.SENTIENCE_API_KEY });
+await browser.getPage().goto('https://www.amazon.com');
+
+// Get semantic snapshot
+const snap = await snapshot(browser);
+
+// Find elements using query DSL
+const searchBox = find(snap, 'role=textbox text~"search"');
+await click(browser, searchBox!.id);
+
+// Type and submit
+await typeText(browser, searchBox!.id, 'wireless mouse');
+await press(browser, 'Enter');
+
+await browser.close();
+```
+
+**When to use:** Need precise control, debugging, custom workflows
+**Code reduction:** Still 80% less code vs raw Playwright
+**Requirements:** Only Sentience API key
+
+### ⚙️ Level 1: Raw Playwright (Maximum Control)
+
+For when you need complete low-level control (rare):
+
+```typescript
+import { chromium } from 'playwright';
+
+const browser = await chromium.launch();
+const page = await browser.newPage();
+await page.goto('https://www.amazon.com');
+await page.fill('#twotabsearchtextbox', 'wireless mouse');
+await page.press('#twotabsearchtextbox', 'Enter');
+await browser.close();
+```
+
+**When to use:** Very specific edge cases, custom browser configs
+**Tradeoffs:** No semantic intelligence, brittle selectors, more code
+
+---
+
+## Agent Layer Examples
+
+### Google Search (6 lines of code)
+
+```typescript
+import { SentienceBrowser, SentienceAgent, OpenAIProvider } from 'sentience-ts';
+
+const browser = await SentienceBrowser.create({ apiKey: apiKey });
+const llm = new OpenAIProvider(openaiKey, 'gpt-4o-mini');
+const agent = new SentienceAgent(browser, llm);
+
+await browser.getPage().goto('https://www.google.com');
+await agent.act('Click the search box');
+await agent.act("Type 'mechanical keyboards' into the search field");
+await agent.act('Press Enter key');
+await agent.act('Click the first non-ad search result');
+
+await browser.close();
+```
+
+**See full example:** [examples/agent-google-search.ts](examples/agent-google-search.ts)
+
+### Using Anthropic Claude Instead of GPT
+
+```typescript
+import { SentienceAgent, AnthropicProvider } from 'sentience-ts';
+
+// Swap OpenAI for Anthropic - same API!
+const llm = new AnthropicProvider(
+  process.env.ANTHROPIC_API_KEY!,
+  'claude-3-5-sonnet-20241022'
+);
+
+const agent = new SentienceAgent(browser, llm);
+await agent.act('Click the search button'); // Works exactly the same
+```
+
+**BYOB (Bring Your Own Brain):** OpenAI, Anthropic, or implement `LLMProvider` for any model.
+
+**See full example:** [examples/agent-with-anthropic.ts](examples/agent-with-anthropic.ts)
+
+### Amazon Shopping (98% code reduction)
+
+**Before (manual approach):** 350 lines
+**After (agent layer):** 6 lines
+
+```typescript
+await agent.act('Click the search box');
+await agent.act("Type 'wireless mouse' into the search field");
+await agent.act('Press Enter key');
+await agent.act('Click the first visible product in the search results');
+await agent.act("Click the 'Add to Cart' button");
+```
+
+**See full example:** [examples/agent-amazon-shopping.ts](examples/agent-amazon-shopping.ts)
+
+---
+
+## Installation for Agent Layer
+
+```bash
+# Install core SDK
+npm install sentience-ts
+
+# Install LLM provider (choose one or both)
+npm install openai              # For GPT-4, GPT-4o, GPT-4o-mini
+npm install @anthropic-ai/sdk   # For Claude 3.5 Sonnet
+
+# Set API keys
+export SENTIENCE_API_KEY="your-sentience-key"
+export OPENAI_API_KEY="your-openai-key"        # OR
+export ANTHROPIC_API_KEY="your-anthropic-key"
+```
+
+---
+
+## Direct SDK Quick Start
 
 ```typescript
 import { SentienceBrowser, snapshot, find, click } from './src';
@@ -351,6 +536,12 @@ element.z_index         // CSS stacking order
 
 See the `examples/` directory for complete working examples:
 
+### Agent Layer (Level 3 - Natural Language)
+- **`agent-google-search.ts`** - Google search automation with natural language commands
+- **`agent-amazon-shopping.ts`** - Amazon shopping bot (6 lines vs 350 lines manual code)
+- **`agent-with-anthropic.ts`** - Using Anthropic Claude instead of OpenAI GPT
+
+### Direct SDK (Level 2 - Technical Control)
 - **`hello.ts`** - Extension bridge verification
 - **`basic-agent.ts`** - Basic snapshot and element inspection
 - **`query-demo.ts`** - Query engine demonstrations
@@ -492,8 +683,21 @@ npm run example:hello
 
 ## License
 
-**📜 License**
+📜 **License**
+
+This SDK is licensed under the **Elastic License 2.0 (ELv2)**.
+
+The Elastic License 2.0 allows you to use, modify, and distribute this SDK for internal, research, and non-competitive purposes. It **does not permit offering this SDK or a derivative as a hosted or managed service**, nor using it to build a competing product or service.
+
+### Important Notes
+
+- This SDK is a **client-side library** that communicates with proprietary Sentience services and browser components.
+
+- The Sentience backend services (including semantic geometry grounding, ranking, visual cues, and trace processing) are **not open source** and are governed by Sentience’s Terms of Service.
+
+- Use of this SDK does **not** grant rights to operate, replicate, or reimplement Sentience’s hosted services.
+
+For commercial usage, hosted offerings, or enterprise deployments, please contact Sentience to obtain a commercial license.
 
-This SDK is licensed under the Apache License 2.0.
+See the full license text in [`LICENSE`](./LICENSE.md).
 
-Note: The SDK communicates with proprietary Sentience services and browser components that are not open source. Access to those components is governed by Sentience's Terms of Service.

package/dist/agent.d.ts

@@ -0,0 +1,145 @@
+/**
+ * Sentience Agent: High-level automation agent using LLM + SDK
+ * Implements observe-think-act loop for natural language commands
+ */
+import { SentienceBrowser } from './browser';
+import { SnapshotOptions } from './snapshot';
+import { LLMProvider } from './llm-provider';
+/**
+ * Execution result from agent.act()
+ */
+export interface AgentActResult {
+    success: boolean;
+    action?: string;
+    elementId?: number;
+    text?: string;
+    key?: string;
+    outcome?: string;
+    urlChanged?: boolean;
+    durationMs: number;
+    attempt: number;
+    goal: string;
+    error?: string;
+    message?: string;
+}
+/**
+ * History entry for executed action
+ */
+export interface HistoryEntry {
+    goal: string;
+    action: string;
+    result: AgentActResult;
+    success: boolean;
+    attempt: number;
+    durationMs: number;
+}
+/**
+ * Token usage statistics
+ */
+export interface TokenStats {
+    totalPromptTokens: number;
+    totalCompletionTokens: number;
+    totalTokens: number;
+    byAction: Array<{
+        goal: string;
+        promptTokens?: number;
+        completionTokens?: number;
+        totalTokens?: number;
+        model?: string;
+    }>;
+}
+/**
+ * High-level agent that combines Sentience SDK with any LLM provider.
+ *
+ * Uses observe-think-act loop to execute natural language commands:
+ * 1. OBSERVE: Get snapshot of current page state
+ * 2. THINK: Query LLM to decide next action
+ * 3. ACT: Execute action using SDK
+ *
+ * Example:
+ * ```typescript
+ * import { SentienceBrowser, SentienceAgent, OpenAIProvider } from 'sentience-ts';
+ *
+ * const browser = await SentienceBrowser.create({ apiKey: 'sentience_key' });
+ * const llm = new OpenAIProvider('openai_key', 'gpt-4o');
+ * const agent = new SentienceAgent(browser, llm);
+ *
+ * await browser.getPage().goto('https://google.com');
+ * await agent.act('Click the search box');
+ * await agent.act("Type 'magic mouse' into the search field");
+ * await agent.act('Press Enter key');
+ * ```
+ */
+export declare class SentienceAgent {
+    private browser;
+    private llm;
+    private snapshotLimit;
+    private verbose;
+    private history;
+    private tokenUsage;
+    /**
+     * Initialize Sentience Agent
+     * @param browser - SentienceBrowser instance
+     * @param llm - LLM provider (OpenAIProvider, AnthropicProvider, etc.)
+     * @param snapshotLimit - Maximum elements to include in context (default: 50)
+     * @param verbose - Print execution logs (default: true)
+     */
+    constructor(browser: SentienceBrowser, llm: LLMProvider, snapshotLimit?: number, verbose?: boolean);
+    /**
+     * Execute a high-level goal using observe → think → act loop
+     * @param goal - Natural language instruction (e.g., "Click the Sign In button")
+     * @param maxRetries - Number of retries on failure (default: 2)
+     * @param snapshotOptions - Optional snapshot parameters (limit, filter, etc.)
+     * @returns Result dict with status, action_taken, reasoning, and execution data
+     *
+     * Example:
+     * ```typescript
+     * const result = await agent.act('Click the search box');
+     * console.log(result);
+     * // { success: true, action: 'click', elementId: 42, ... }
+     * ```
+     */
+    act(goal: string, maxRetries?: number, snapshotOptions?: SnapshotOptions): Promise<AgentActResult>;
+    /**
+     * Filter elements from snapshot based on goal context.
+     * Applies goal-based keyword matching to boost relevant elements and filters out irrelevant ones.
+     */
+    private filterElements;
+    /**
+     * Extract meaningful keywords from goal text
+     */
+    private extractKeywords;
+    /**
+     * Convert snapshot elements to token-efficient prompt string
+     * Format: [ID] <role> "text" {cues} @ (x,y) (Imp:score)
+     * Note: elements are already filtered by filterElements() in act()
+     */
+    private buildContext;
+    /**
+     * Query LLM with standardized prompt template
+     */
+    private queryLLM;
+    /**
+     * Parse action string and execute SDK call
+     */
+    private executeAction;
+    /**
+     * Track token usage for analytics
+     */
+    private trackTokens;
+    /**
+     * Get token usage statistics
+     * @returns Dictionary with token usage breakdown
+     */
+    getTokenStats(): TokenStats;
+    /**
+     * Get execution history
+     * @returns List of all actions taken with results
+     */
+    getHistory(): HistoryEntry[];
+    /**
+     * Clear execution history and reset token counters
+     */
+    clearHistory(): void;
+}
+//# sourceMappingURL=agent.d.ts.map

package/dist/agent.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EAAE,gBAAgB,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,EAAY,eAAe,EAAE,MAAM,YAAY,CAAC;AAGvD,OAAO,EAAE,WAAW,EAAe,MAAM,gBAAgB,CAAC;AAE1D;;GAEG;AACH,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,OAAO,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,GAAG,CAAC,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,UAAU,CAAC,EAAE,OAAO,CAAC;IACrB,UAAU,EAAE,MAAM,CAAC;IACnB,OAAO,EAAE,MAAM,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;CAClB;AAED;;GAEG;AACH,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,cAAc,CAAC;IACvB,OAAO,EAAE,OAAO,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;CACpB;AAED;;GAEG;AACH,MAAM,WAAW,UAAU;IACzB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,qBAAqB,EAAE,MAAM,CAAC;IAC9B,WAAW,EAAE,MAAM,CAAC;IACpB,QAAQ,EAAE,KAAK,CAAC;QACd,IAAI,EAAE,MAAM,CAAC;QACb,YAAY,CAAC,EAAE,MAAM,CAAC;QACtB,gBAAgB,CAAC,EAAE,MAAM,CAAC;QAC1B,WAAW,CAAC,EAAE,MAAM,CAAC;QACrB,KAAK,CAAC,EAAE,MAAM,CAAC;KAChB,CAAC,CAAC;CACJ;AAED;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,OAAO,CAAmB;IAClC,OAAO,CAAC,GAAG,CAAc;IACzB,OAAO,CAAC,aAAa,CAAS;IAC9B,OAAO,CAAC,OAAO,CAAU;IACzB,OAAO,CAAC,OAAO,CAAiB;IAChC,OAAO,CAAC,UAAU,CAAa;IAE/B;;;;;;OAMG;gBAED,OAAO,EAAE,gBAAgB,EACzB,GAAG,EAAE,WAAW,EAChB,aAAa,GAAE,MAAW,EAC1B,OAAO,GAAE,OAAc;IAezB;;;;;;;;;;;;;OAaG;IACG,GAAG,CACP,IAAI,EAAE,MAAM,EACZ,UAAU,GAAE,MAAU,EACtB,eAAe,CAAC,EAAE,eAAe,GAChC,OAAO,CAAC,cAAc,CAAC;IAiG1B;;;OAGG;IACH,OAAO,CAAC,cAAc;IA+CtB;;OAEG;IACH,OAAO,CAAC,eAAe;IASvB;;;;OAIG;IACH,OAAO,CAAC,YAAY;IA0BpB;;OAEG;YACW,QAAQ;IAkCtB;;OAEG;YACW,aAAa;IAsE3B;;OAEG;IACH,OAAO,CAAC,WAAW;IAoBnB;;;OAGG;IACH,aAAa,IAAI,UAAU;IAI3B;;;OAGG;IACH,UAAU,IAAI,YAAY,EAAE;IAI5B;;OAEG;IACH,YAAY,IAAI,IAAI;CASrB"}