osborn 0.8.16 → 0.8.18

package/.claude/skills/browser-apply/SKILL.md

@@ -0,0 +1,114 @@
+# Skill: Browser Apply — Step-by-Step Workday Application
+
+Automate Workday job applications interactively, one step at a time. Each step takes a screenshot, confirms what's on screen, fills the current page, and waits before proceeding.
+
+**This skill uses the Playwright MCP tools** (`mcp__playwright__browser_*`) for direct browser control — no scripts needed.
+
+## When to Use
+- Any Workday ATS application (`*.wd1.myworkdayjobs.com`)
+- Any multi-step JS-heavy job application form
+- When you want visible, confirmable progress at each step
+
+## Key Principle: Step-by-Step, Not One Big Script
+
+Do NOT write a monolithic automation script. Instead:
+1. Navigate to the URL
+2. Take a screenshot — confirm what's on screen
+3. Fill only the current step's fields
+4. Take another screenshot — confirm fields filled correctly
+5. Ask the user "Ready for next step?" before clicking Next
+6. Click Next, wait for page load, screenshot again
+7. Repeat for each step
+
+This approach catches rendering issues, unexpected fields, and errors before they cascade.
+
+## Step-by-Step Execution Pattern
+
+### Step 0 — Open the browser
+Use: `mcp__playwright__browser_navigate` with the applyManually URL
+
+Then: `mcp__playwright__browser_take_screenshot` — show the user what loaded
+
+### Step 1 — Create Account / Sign In
+Take a snapshot with `mcp__playwright__browser_snapshot` to see element refs.
+Fill fields using `mcp__playwright__browser_fill_form` or individual `mcp__playwright__browser_type` calls.
+Screenshot to confirm. Then ask user before clicking Create Account / Sign In.
+
+### Step 2 — Start Application
+If "Start Your Application" screen appears with Apply Manually button:
+Screenshot it. Click "Apply Manually" using `mcp__playwright__browser_click`.
+Screenshot after.
+
+### Step 3 — My Information
+Snapshot → fill each field → screenshot → ask user before clicking Next.
+
+Fields to fill:
+- First Name, Last Name, Phone
+- Address, City, State (dropdown), Zip
+- Work authorization: Yes
+- Sponsorship: No
+
+### Step 4 — My Experience
+Snapshot → click Add for each job entry → fill title/company/dates/description → save each → screenshot.
+Then add education entries.
+Ask user before clicking Next.
+
+### Step 5 — Application Questions
+Snapshot to see all questions. Fill each one. **Always confirm salary expectation with user before filling** — never guess. Screenshot. Ask before Next.
+
+### Step 6 — Voluntary Disclosures
+Select "I do not wish to answer" / "Prefer not to disclose" for all. Screenshot. Ask before Next.
+
+### Step 7 — Self Identify
+Fill name and date. Select disability option. Screenshot. Ask before Next.
+
+### Step 8 — Review
+Screenshot the full review page. Confirm with user before clicking Submit.
+
+### Step 9 — Confirm submission
+Screenshot the confirmation dialog. Save it.
+
+## Candidate Data (Osborn Ojure)
+
+- Email: osbornojure@gmail.com
+- Password: Workday2026!
+- First: Osborn, Last: Ojure
+- Phone: 3127185561
+- Address: 1234 N Michigan Ave, Chicago, IL 60601
+
+Jobs:
+1. Meta API Consultant at Prehype / Audos — April 2024 to Present
+2. Full Stack Developer, Freelance — January 2016 to Present
+
+Education:
+1. A.S. Information Systems
+2. B.S. Psychology
+
+## Workday data-automation-id Selector Reference
+
+| Field | Selector |
+|---|---|
+| Email | `input[type="email"]` |
+| Password | `input[type="password"]` |
+| First name | `[data-automation-id="legalNameSection_firstName"]` |
+| Last name | `[data-automation-id="legalNameSection_lastName"]` |
+| Phone | `[data-automation-id="phone-number"]` |
+| Address | `[data-automation-id="addressSection_addressLine1"]` |
+| City | `[data-automation-id="addressSection_city"]` |
+| Zip | `[data-automation-id="addressSection_postalCode"]` |
+| Job title | `[data-automation-id="jobTitle"]` |
+| Company | `[data-automation-id="company"]` |
+| Description | `[data-automation-id="description"]` |
+| Next button | `[data-automation-id="bottom-navigation-next-btn"]` |
+| Create Account | `[data-automation-id="click_filter"][aria-label="Create Account"]` |
+
+## Critical Rules
+- headless: false always (Workday renders blank in headless)
+- Confirm salary with user before every submission — never auto-fill
+- After "You already applied to this job" error — that confirms a previous submission worked
+- Use `{ force: true }` on Workday buttons — overlay click filters block normal clicks
+- Always wait for networkidle or waitForSelector after navigation before interacting
+
+## Playwright Install Location
+Run scripts from: `/Users/newupgrade/Desktop/Developer/osborn/frontend`
+(playwright is in `node_modules` there)

package/.env.example

@@ -19,3 +19,10 @@ ANTHROPIC_API_KEY=sk-ant-...
 # Smithery (cloud-hosted MCP servers - YouTube, GitHub, etc.)
 # Get your key at: https://smithery.ai
 # SMITHERY_API_KEY=your-smithery-api-key
+
+# Frontend URL — used by the agent to upload workspace artifacts (resumes,
+# reports, PDFs, search indexes) to Supabase Storage via the frontend's
+# /api/upload route. Without this, the agent falls back to inlining file
+# content through the LiveKit data channel (capped at 30KB to avoid
+# corrupting the publisher connection). Local dev: http://localhost:3000
+# OSBORN_FRONTEND_URL=http://localhost:3000

package/dist/claude-llm.js

@@ -17,6 +17,13 @@ import { fileURLToPath } from 'node:url';
 // Directory of this module — used to locate co-located prompt files (e.g., turn-shape reminder).
 const __claudeLlmDir = dirname(fileURLToPath(import.meta.url));
 const TURN_SHAPE_REMINDER_PATH = join(__claudeLlmDir, 'prompts', 'turn-shape-reminder.md');
+// ≤3 direct tool call budget per turn. Reset on every UserPromptSubmit (new user message).
+// Enforced mechanically in PreToolUse — the model CANNOT exceed this regardless of JSONL history.
+// Task/Agent delegations are exempt (delegation is what we WANT). Sub-agent tool calls
+// (agent_type !== null) are exempt (they're inside a delegation). Only the main orchestrator
+// agent's direct tool calls count against the budget.
+let turnToolCallCount = 0;
+const TOOL_CALL_BUDGET = 3;
 /**
  * Strip markdown formatting for TTS (text-to-speech)
  * Removes **bold**, ##headers, ```code```, etc. so TTS doesn't read them literally
@@ -750,8 +757,8 @@ class ClaudeLLMStream extends llm.LLMStream {
                 cwd: this.#opts.workingDirectory,
                 permissionMode: this.#opts.permissionMode,
                 allowedTools,
-                model: this.#opts.model || 'haiku', // haiku for speed with limited tools, sonnet for full research capabilities (including tool use trace in response)
-                // model: this.#opts.model || 'claude-sonnet-4-6', // Sonnet orchestrator with named sub-agents (Haiku tested but ignored delegation rules)
+                // model: this.#opts.model || 'haiku', // haiku for speed with limited tools, sonnet for full research capabilities (including tool use trace in response)
+                model: this.#opts.model || 'claude-sonnet-4-6', // Sonnet orchestrator with named sub-agents (Haiku tested but ignored delegation rules)
                 enableFileCheckpointing: true,
                 extraArgs: { 'replay-user-messages': null },
                 ...(this.#abortController && { abortController: this.#abortController }),
@@ -824,6 +831,22 @@ class ClaudeLLMStream extends llm.LLMStream {
                                     const toolInput = input?.tool_input || {};
                                     const agentType = input?.agent_type || null;
                                     console.log(`🔍 PreToolUse: toolName=${toolName} agent_type=${agentType} agent_id=${input?.agent_id || 'none'} all_keys=[${Object.keys(input || {}).join(', ')}]`);
+                                    // ≤3 direct tool call budget enforcement.
+                                    // Only counts calls from the MAIN orchestrator agent (agent_type === null).
+                                    // Task/Agent delegations are exempt — delegation is the desired behavior.
+                                    // Sub-agent tool calls are exempt — they're inside a delegation.
+                                    if (!agentType && toolName !== 'Task' && toolName !== 'Agent') {
+                                        turnToolCallCount++;
+                                        if (turnToolCallCount > TOOL_CALL_BUDGET) {
+                                            console.log(`🛑 Tool budget exceeded (${turnToolCallCount}/${TOOL_CALL_BUDGET}) — DENYING ${toolName}. Must delegate via Task.`);
+                                            this.#eventEmitter.emit('tool_blocked', { name: toolName, reason: `Tool call budget exceeded (${turnToolCallCount}/${TOOL_CALL_BUDGET}). Delegate via Task.` });
+                                            return {
+                                                hookSpecificOutput: { hookEventName: 'PreToolUse', permissionDecision: 'deny' },
+                                                reason: `Hard limit: maximum ${TOOL_CALL_BUDGET} direct tool calls per turn (you are at ${turnToolCallCount}). Delegate the remaining work to a sub-agent via Task(subagent_type=\'researcher\'|\'writer\'|\'reasoner\', run_in_background: true). This is a system-enforced limit.`,
+                                            };
+                                        }
+                                        console.log(`🔧 Tool call ${turnToolCallCount}/${TOOL_CALL_BUDGET}: ${toolName}`);
+                                    }
                                     // Write/Edit/MultiEdit access control
                                     if (toolName === 'Write' || toolName === 'Edit' || toolName === 'MultiEdit') {
                                         // Writer sub-agent gets full write access everywhere
@@ -871,9 +894,11 @@ class ClaudeLLMStream extends llm.LLMStream {
                             matcher: '.*',
                             hooks: [async (input) => {
                                     try {
+                                        // Reset the per-turn tool call counter so the new turn starts fresh.
+                                        turnToolCallCount = 0;
                                         const reminder = readFileSync(TURN_SHAPE_REMINDER_PATH, 'utf-8');
                                         const promptPreview = String(input?.prompt || '').substring(0, 60).replace(/\n/g, ' ');
-                                        console.log(`📌 UserPromptSubmit: injected turn-shape reminder (${reminder.length} chars) for prompt="${promptPreview}..."`);
+                                        console.log(`📌 UserPromptSubmit: injected turn-shape reminder (${reminder.length} chars) for prompt="${promptPreview}..." [tool budget reset to 0/${TOOL_CALL_BUDGET}]`);
                                         return {
                                             hookSpecificOutput: {
                                                 hookEventName: 'UserPromptSubmit',

package/dist/index.js

@@ -491,6 +491,11 @@ async function main() {
     let userState = 'listening'; // Track user speech state for queue safety
     let currentVoiceMode = voiceMode; // Track active voice mode for data handlers
     let currentProvider = realtimeConfig.provider; // Track active realtime provider
+    // Authenticated Supabase userId from participant metadata. Used to scope
+    // workspace artifact uploads to the owner's prefix in Supabase Storage.
+    // Empty string = anonymous / unauthenticated; uploads fall back to a
+    // session-only path (no user prefix).
+    let currentUserId = '';
     // Track the active resume session ID across scopes (ParticipantConnected + DataReceived)
     // Updated by resume_session, session_selected, continue_session, switch_session handlers
     let currentResumeSessionId;
@@ -1776,6 +1781,15 @@ async function main() {
         try {
             const metadata = JSON.parse(participant.metadata || '{}');
             console.log(`📋 Participant metadata:`, metadata);
+            // userId from authenticated Supabase session — used to scope Supabase
+            // Storage uploads so each user's workspace artifacts live under their
+            // own prefix. Falls through to '' (anonymous) if not authenticated.
+            if (typeof metadata.userId === 'string' && metadata.userId.length > 0) {
+                currentUserId = metadata.userId;
+            }
+            else {
+                currentUserId = '';
+            }
             if (metadata.voiceArch === 'realtime' || metadata.voiceArch === 'direct' || metadata.voiceArch === 'pipeline') {
                 sessionVoiceMode = metadata.voiceArch;
                 console.log(`🎙️ Using voice mode from frontend: ${sessionVoiceMode}`);
@@ -2703,40 +2717,99 @@ async function main() {
                 if (filePath && (filePath.includes('/osb/') || filePath.includes('.osborn/sessions/') || filePath.includes('.osborn/research/'))) {
                     try {
                         const fs = await import('fs');
+                        const path = await import('path');
                         const fileName = filePath.split('/').pop() || '';
                         const ext = fileName.split('.').pop()?.toLowerCase() || '';
                         const isImage = ['png', 'jpg', 'jpeg', 'gif', 'webp'].includes(ext);
-                        // WebRTC SCTP data channel max message size is ~256KB. Sending
-                        // larger payloads corrupts the publisher transport, killing ALL
-                        // subsequent sends (publishData, streamBytes, publishTranscription)
-                        // with "could not establish publisher connection: timeout". This
-                        // was the root cause of the career-ops session bug: a 480KB
-                        // evaluation report blew through the limit on resume.
-                        // ⚠️ MUST be low — not just per-message but cumulative back-to-back pressure.
-                        // 12 artifact requests arrive simultaneously during session resume. Even if
-                        // each is individually "safe", flooding them kills the publisher PC. At 200KB
-                        // the search-index.txt (136KB) passed through and poisoned the connection.
-                        // 30KB catches search-index.txt (136KB), resume.pdf (233KB), and search-index-
-                        // meta.json (5.7KB passes). resume.html (14KB) also passes — acceptable.
-                        const MAX_DATA_CHANNEL_BYTES = 30_000; // 30KB max per artifact
-                        if (isImage) {
+                        const mimeByExt = {
+                            png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg',
+                            gif: 'image/gif', webp: 'image/webp', pdf: 'application/pdf',
+                            html: 'text/html', md: 'text/markdown', txt: 'text/plain',
+                            json: 'application/json',
+                        };
+                        const mimeType = mimeByExt[ext] || 'application/octet-stream';
+                        // Strategy: upload the file to Supabase Storage via the frontend's
+                        // /api/upload route and send back just the URL. This mirrors the
+                        // existing frontend→agent attachment flow (where the browser uploads
+                        // user attachments to Supabase and passes URLs to the agent). For
+                        // the reverse direction we do the same: URLs are ~100 bytes, so
+                        // the LiveKit data channel stays healthy regardless of file size.
+                        //
+                        // Fallback to inline send if OSBORN_FRONTEND_URL isn't configured
+                        // OR the upload fails — with a small size cap so we don't kill the
+                        // publisher PC with a 480KB payload (see earlier career-ops bug).
+                        const FRONTEND_URL = process.env.OSBORN_FRONTEND_URL || process.env.NEXT_PUBLIC_FRONTEND_URL || '';
+                        const MAX_INLINE_BYTES = 30_000; // fallback-only cap
+                        let uploadedUrl = null;
+                        if (FRONTEND_URL) {
+                            try {
+                                const buf = fs.readFileSync(filePath);
+                                const form = new FormData();
+                                form.append('file', new Blob([buf], { type: mimeType }), fileName);
+                                form.append('folder', 'artifacts');
+                                // Pass userId + sessionId so /api/upload can place the file
+                                // under `{userId}/{sessionId}/...` in Supabase Storage for
+                                // easy ownership queries and future RLS policies. Both are
+                                // optional — route falls back to `artifacts/...` if missing.
+                                if (currentUserId)
+                                    form.append('userId', currentUserId);
+                                // Prefer the live resume session id (updated by session
+                                // switches), fall back to whatever SDK session id the LLM
+                                // reports, fall back to empty.
+                                const uploadSessionId = currentResumeSessionId
+                                    || currentLLM?.sessionId
+                                    || '';
+                                if (uploadSessionId)
+                                    form.append('sessionId', uploadSessionId);
+                                const r = await fetch(`${FRONTEND_URL.replace(/\/$/, '')}/api/upload`, {
+                                    method: 'POST', body: form,
+                                    signal: AbortSignal.timeout(15_000),
+                                });
+                                if (r.ok) {
+                                    const j = await r.json();
+                                    if (j.success && j.url) {
+                                        uploadedUrl = j.url;
+                                        console.log(`☁️ Uploaded artifact to Supabase: ${fileName} (${(buf.length / 1024).toFixed(0)}KB) → ${j.url.substring(0, 80)}...`);
+                                    }
+                                    else {
+                                        console.warn(`⚠️ Upload failed for ${fileName}: ${j.error || 'unknown'}`);
+                                    }
+                                }
+                                else {
+                                    console.warn(`⚠️ Upload HTTP ${r.status} for ${fileName}`);
+                                }
+                            }
+                            catch (err) {
+                                console.warn(`⚠️ Upload threw for ${fileName}:`, err.message);
+                            }
+                        }
+                        if (uploadedUrl) {
+                            // Success path — send URL, no inline content.
+                            await sendToFrontend({
+                                type: 'research_artifact_content',
+                                filePath, fileName, url: uploadedUrl,
+                                isImage, mimeType,
+                            });
+                        }
+                        else if (isImage) {
+                            // Fallback: inline image (with size cap)
                             const stats = fs.statSync(filePath);
-                            const base64Size = Math.ceil(stats.size * 4 / 3); // base64 inflates ~33%
-                            if (base64Size > MAX_DATA_CHANNEL_BYTES) {
-                                console.log(`⚠️ Artifact too large for data channel: ${fileName} (${(base64Size / 1024).toFixed(0)}KB base64) — sending truncation notice`);
+                            const base64Size = Math.ceil(stats.size * 4 / 3);
+                            if (base64Size > MAX_INLINE_BYTES) {
+                                console.log(`⚠️ Artifact too large for inline fallback: ${fileName} (${(base64Size / 1024).toFixed(0)}KB base64) — sending truncation notice`);
                                 await sendToFrontend({ type: 'research_artifact_content', filePath, content: '', fileName, isImage: false, truncated: true, originalSize: stats.size });
                             }
                             else {
                                 const base64 = fs.readFileSync(filePath, 'base64');
-                                await sendToFrontend({ type: 'research_artifact_content', filePath, content: base64, fileName, isImage: true, mimeType: `image/${ext}` });
+                                await sendToFrontend({ type: 'research_artifact_content', filePath, content: base64, fileName, isImage: true, mimeType });
                             }
                         }
                         else {
+                            // Fallback: inline text (with size cap)
                             const content = fs.readFileSync(filePath, 'utf-8');
-                            if (Buffer.byteLength(content, 'utf-8') > MAX_DATA_CHANNEL_BYTES) {
-                                // Send a truncated preview + metadata so the frontend knows the file exists
-                                const truncated = content.substring(0, 5_000); // ~5KB preview (keep well under the 30KB limit)
-                                console.log(`⚠️ Artifact too large for data channel: ${fileName} (${(Buffer.byteLength(content, 'utf-8') / 1024).toFixed(0)}KB) — sending truncated preview`);
+                            if (Buffer.byteLength(content, 'utf-8') > MAX_INLINE_BYTES) {
+                                const truncated = content.substring(0, 5_000);
+                                console.log(`⚠️ Artifact too large for inline fallback: ${fileName} (${(Buffer.byteLength(content, 'utf-8') / 1024).toFixed(0)}KB) — sending truncated preview`);
                                 await sendToFrontend({ type: 'research_artifact_content', filePath, content: truncated, fileName, isImage: false, truncated: true, originalSize: Buffer.byteLength(content, 'utf-8') });
                             }
                             else {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "osborn",
-  "version": "0.8.16",
+  "version": "0.8.18",
   "description": "Voice AI coding assistant - local agent that connects to Osborn frontend",
   "type": "module",
   "bin": {