@ai-digital-factory/lmt-web-cli 0.1.1 → 0.1.3

package/README.md

@@ -5,6 +5,67 @@ browser interaction with the configured local or dev Live Market Terminal Web UR
 keeps browser dependencies and session lifetime out of the browser-free `lmt`
 HTTP CLI.
 
+## Agent guide (Dev only)
+
+Agents working with a deployed Live Market Terminal must use the **Dev** environment.
+Do not target production. The examples here use the published CLI through `npx` and
+open a headless desktop browser by default.
+
+Install Chromium once on the machine that will own the browser session:
+
+```sh
+npx --yes @ai-digital-factory/lmt-web-cli@latest install-browser
+```
+
+If Operator login needs the Gate Operator Directory, provision this CLI's SMS read
+credential once from a secure input source. The Web CLI keeps its credential
+separately from `lmt-cli` and does not use profiles:
+
+```sh
+npx --yes @ai-digital-factory/lmt-web-cli@latest credential provision --env dev --credential-stdin
+```
+
+Start a fresh desktop session against Dev and keep the returned `sessionId` for every
+later command:
+
+```sh
+npx --yes @ai-digital-factory/lmt-web-cli@latest start --env dev --device desktop
+```
+
+The session starts on the Home Page. Use the same session to log into the Operator;
+if a Zerodha login is required, start that flow and submit the current OTP through
+standard input only, using the `attemptId` returned by `zerodha-login`:
+
+```sh
+npx --yes @ai-digital-factory/lmt-web-cli@latest operator-login --session SESSION_ID
+npx --yes @ai-digital-factory/lmt-web-cli@latest zerodha-login --session SESSION_ID
+npx --yes @ai-digital-factory/lmt-web-cli@latest zerodha-otp --session SESSION_ID --attempt ATTEMPT_ID --stdin
+```
+
+The user supplies the OTP for the active attempt. Never put it in a command-line
+argument or ask the CLI to retrieve it. Inspect the latest JSON result after each
+command. Control references are short-lived: every new observation or action returns
+fresh references, so use only references from the most recent result.
+
+For example, to add an instrument to Market Watch, inspect the page, select `NSE`
+in an empty row, enter the symbol, then apply that row. Use the current row's control
+references from each immediately preceding result. `Apply` configures the Market
+Watch row; it does not submit a trade.
+
+Each command returns JSON with the page state and screenshot path. Review the result
+before continuing, especially if an action is `uncertain`; inspect the page instead
+of repeating that action automatically. When finished, stop only the session you
+started:
+
+```sh
+npx --yes @ai-digital-factory/lmt-web-cli@latest stop --session SESSION_ID
+```
+
+Stopping releases the browser process but does not log out the Operator or stop
+Broker activity. The CLI also supports `inspect`, `click`, `fill`, `select`, and
+`navigate`; run `npx --yes @ai-digital-factory/lmt-web-cli@latest --help` for the
+complete command list.
+
 ## Install
 
 Requires Node.js 22 or later, Chromium, and a macOS or Linux session owner.
@@ -54,6 +115,11 @@ Quitting the browser from its Dock menu or closing its session window also ends
 the browser session and its local owner. This does not log out the Operator or
 stop Broker activity. A later `inspect` reports `session_not_found`.
 
+On Dev Deployment, Dev Operator opens the Web UI with `?captureSafe=1` on the first
+navigation (for example `https://dev-live-market-terminal.piyush-arora.in/?captureSafe=1`)
+so Capture-safe Mode stays enabled for that tab through navigation and refresh.
+`lmt-web` screenshots still mask access-token value fields automatically without that flag.
+
 ```sh
 lmt-web start --env dev
 lmt-web inspect --session SESSION_ID
@@ -80,7 +146,10 @@ Each command writes one versioned JSON result to stdout and diagnostics to
 stderr. Results contain the action outcome, current URL, visible page text and
 controls, emulated device and actual viewport, and screenshot outcome. Images
 are written to unique absolute paths in an owner-only temporary directory;
-screenshots preserve the rendered page without added blur or masking. Field values
+screenshots solid-mask only Zerodha access-token value fields marked with
+`data-lmt-sensitive="zerodha-access-token"`; other controls and metadata stay visible.
+If protected capture fails, the screenshot outcome is `failed` with no unmasked fallback.
+Field values
 are omitted from JSON observations. Captures wait for document loading, visible
 session-loading or busy indicators, fonts, visible images, and finite animations
 to finish. The page must render text, controls, or visual content; an empty app
@@ -260,3 +329,32 @@ sets up its own Chromium runtime, verifies packaged runtime modules/configuratio
 and reuses all browser acceptance tests. Installed commands and their live owners
 run outside the checkout. Test-only fixture servers provide repeatable responses;
 this check performs no npm publication or live dev login.
+
+## Explicit values, equality, scrolling, and full content
+
+```sh
+lmt-web read --session ID --ref FIELD
+lmt-web clipboard --session ID
+lmt-web check --session ID --ref FIELD --stdin < /path/to/protected-expectation
+lmt-web clipboard-check --session ID --ref FIELD
+lmt-web scroll --session ID --ref FIELD
+lmt-web scroll --session ID --y 600
+lmt-web inspect --session ID --full-page
+```
+
+read returns an ordinary field value. clipboard returns content only when it matches
+a verified ordinary field on the current page; unknown or secret content is refused.
+check compares the field with one stdin-supplied expectation (an optional final newline
+is removed). clipboard-check compares clipboard and field inside the browser session.
+Comparisons return only matches: true or matches: false. An unreadable field/clipboard
+or an expired reference is an error without a matches result. Inspect/action observations
+continue to redact secrets, including revealed fields.
+
+Referenced scrolling brings the control into view through its ancestors. Pixel scrolling
+targets the largest scrolling content area. Results include before/after positions and
+moved, already_visible, or at_boundary. Each observation expires older references.
+
+Full-page capture temporarily expands inner scrolling content and restores its styles
+and positions afterward. A screenshot with unexpanded clipped content reports failed;
+browser-action and screenshot outcomes remain separate. Screenshots retain the existing
+rendered-page confidentiality behavior: protect their files as you would the visible UI.

package/dist/index.js

@@ -511,6 +511,13 @@ var ACTIONS = {
       return { action: "zerodha-otp", attemptId, otp: await input.readOtp() };
     }
   },
+  read: { control: true, flags: [], values: ["--ref"], build: (input) => ({ action: "read", reference: required(input, "--ref") }) },
+  clipboard: { control: true, flags: [], values: [], build: () => ({ action: "clipboard" }) },
+  check: { control: true, flags: ["--stdin"], values: ["--ref"], build: async (input) => {
+    if (!input.flags.has("--stdin")) throw new InvocationError("check requires --stdin.");
+    return { action: "check", reference: required(input, "--ref"), expected: await input.readInput() };
+  } },
+  "clipboard-check": { control: true, flags: [], values: ["--ref"], build: (input) => ({ action: "clipboard-check", reference: required(input, "--ref") }) },
   click: { control: true, flags: [], values: ["--ref"], build: (input) => ({ action: "click", reference: required(input, "--ref") }) },
   fill: {
     control: true,
@@ -614,6 +621,52 @@ import os from "node:os";
 import path4 from "node:path";
 import { chromium as chromium2 } from "playwright";
 
+// ../../packages/crawler/src/full-content.ts
+async function captureFullContent(browserPage, options) {
+  const page = browserPage;
+  if (!page.evaluate) throw new Error("Full-content capture is unavailable.");
+  try {
+    await page.evaluate(() => {
+      const state = [];
+      window.__lmtCapture = state;
+      const scrolling = Array.from(document.querySelectorAll("body *")).filter((element) => {
+        const style = getComputedStyle(element);
+        return /(auto|scroll)/.test(style.overflowY) && element.scrollHeight > element.clientHeight;
+      });
+      const expand = /* @__PURE__ */ new Set();
+      for (const element of scrolling) {
+        for (let parent = element; parent; parent = parent.parentElement) expand.add(parent);
+      }
+      for (const element of expand) {
+        state.push({ element, style: element.getAttribute("style"), x: element.scrollLeft, y: element.scrollTop });
+        element.style.setProperty("height", "auto", "important");
+        element.style.setProperty("max-height", "none", "important");
+        element.style.setProperty("overflow", "visible", "important");
+        element.style.setProperty("flex-shrink", "0", "important");
+        element.scrollTop = 0;
+      }
+    });
+    await page.evaluate(() => new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(() => resolve()))));
+    const complete = await page.evaluate(() => Array.from(document.querySelectorAll("body *")).every((element) => {
+      const style = getComputedStyle(element);
+      return style.display === "none" || element.clientHeight === 0 || !/(auto|scroll|hidden|clip)/.test(style.overflowY) || element.scrollHeight <= element.clientHeight + 1;
+    }));
+    if (!complete) throw new Error("Full-content capture could not expand all clipped content.");
+    return await page.screenshot({ ...options, fullPage: true });
+  } finally {
+    await page.evaluate(() => {
+      const owner = window;
+      for (const { element, style, x, y } of (owner.__lmtCapture ?? []).reverse()) {
+        if (style === null) element.removeAttribute("style");
+        else element.setAttribute("style", style);
+        element.scrollLeft = x;
+        element.scrollTop = y;
+      }
+      delete owner.__lmtCapture;
+    }).catch(() => void 0);
+  }
+}
+
 // ../../packages/crawler/src/controls.ts
 import { randomBytes } from "node:crypto";
 var ControlError = class extends Error {
@@ -623,6 +676,7 @@ var ControlError = class extends Error {
   }
   code;
 };
+var SENSITIVE_FIELD_PATTERN = "password|credential|secret|token|otp|one.time|username|user.?id|api.?key|pin";
 function controlSignature(node) {
   if (!node.isConnected) return null;
   return JSON.stringify([
@@ -657,8 +711,8 @@ var PageControls = class {
     return result;
   }
   async learnSecrets() {
-    const values = await this.page.evaluate(() => {
-      const sensitive = /password|credential|secret|token|otp|one.time|username|user.?id|api.?key|pin/i;
+    const values = await this.page.evaluate((pattern) => {
+      const sensitive = new RegExp(pattern, "i");
       const found = Array.from(document.querySelectorAll("input, textarea")).filter((element) => sensitive.test([
         element.outerHTML.replace(/value=("[^"]*"|'[^']*')/g, ""),
         ...Array.from(element.labels ?? []).map((label) => label.innerText)
@@ -667,15 +721,16 @@ var PageControls = class {
         if (match[1]) found.push(match[1]);
       }
       return found;
-    });
+    }, SENSITIVE_FIELD_PATTERN);
     this.registerSecrets(...values);
   }
   async screenshotMasks() {
+    const stable = this.page.locator('[data-lmt-sensitive="zerodha-access-token"]');
     try {
       await this.learnSecrets();
     } catch (error) {
       if (this.secrets.size) throw error;
-      else return [];
+      else return [stable];
     }
     await this.page.evaluate((secrets) => {
       document.querySelectorAll("[data-lmt-secret-mask]").forEach((element) => element.removeAttribute("data-lmt-secret-mask"));
@@ -685,7 +740,7 @@ var PageControls = class {
         if (secrets.some((secret) => directText.includes(secret) || (element.textContent ?? "").includes(secret) && !Array.from(element.children).some((child) => (child.textContent ?? "").includes(secret)))) element.setAttribute("data-lmt-secret-mask", "");
       }
     }, [...this.secrets]);
-    return [this.page.locator("[data-lmt-secret-mask]")];
+    return [stable, this.page.locator("[data-lmt-secret-mask]")];
   }
   async inspect() {
     await this.learnSecrets();
@@ -735,9 +790,52 @@ var PageControls = class {
   async act(command, timeoutMs, onSubmit = () => void 0) {
     this.page.setDefaultTimeout(timeoutMs);
     this.page.setDefaultNavigationTimeout(timeoutMs);
+    if (command.action === "read" || command.action === "check" || command.action === "clipboard-check" || command.action === "clipboard") {
+      await this.learnSecrets();
+      if (command.action === "check") this.registerSecrets(command.expected);
+      try {
+        const field = command.action === "clipboard" ? void 0 : await this.resolve(command.reference);
+        if (command.action === "clipboard-check" || command.action === "clipboard") await this.page.context().grantPermissions(["clipboard-read", "clipboard-write"], { origin: this.origin });
+        if (command.action === "clipboard-check") {
+          const matches = await field.evaluate(async (node) => {
+            if (!(node instanceof HTMLInputElement || node instanceof HTMLTextAreaElement || node instanceof HTMLSelectElement)) throw new Error("Unreadable field");
+            return await navigator.clipboard.readText() === node.value;
+          });
+          return { matches };
+        }
+        const value = command.action === "clipboard" ? await this.page.evaluate(() => navigator.clipboard.readText()) : await field.evaluate((node) => {
+          if (!(node instanceof HTMLInputElement || node instanceof HTMLTextAreaElement || node instanceof HTMLSelectElement)) throw new Error("Unreadable field");
+          return node.value;
+        });
+        if (command.action === "check") {
+          this.registerSecrets(value);
+          return { matches: value === command.expected };
+        }
+        const sensitive = field && await field.evaluate((node, pattern) => new RegExp(pattern, "i").test(node.outerHTML.replace(/value=("[^"]*"|'[^']*')/g, "") + Array.from(node.labels ?? []).map((label) => label.innerText).join(" ")), SENSITIVE_FIELD_PATTERN);
+        if (command.action === "clipboard") {
+          const ordinary = await this.page.evaluate(({ value: value2, pattern }) => {
+            const sensitive2 = new RegExp(pattern, "i");
+            return Array.from(document.querySelectorAll("input,textarea,select")).some((node) => {
+              if (!(node instanceof HTMLInputElement || node instanceof HTMLTextAreaElement || node instanceof HTMLSelectElement)) return false;
+              const description = node.outerHTML.replace(/value=("[^"]*"|'[^']*')/g, "") + Array.from(node.labels ?? []).map((label) => label.innerText).join(" ");
+              return !sensitive2.test(description) && node.value === value2;
+            });
+          }, { value, pattern: SENSITIVE_FIELD_PATTERN });
+          if (!ordinary) {
+            this.registerSecrets(value);
+            throw new ControlError("secret_read_refused", "Clipboard content has no verified ordinary field source. Use clipboard-check for secret Copy.");
+          }
+        }
+        if (sensitive || [...this.secrets].some((secret) => value.includes(secret))) throw new ControlError("secret_read_refused", "Use an equality check for secret values.");
+        return { value };
+      } catch (error) {
+        if (error instanceof ControlError) throw error;
+        throw new ControlError("value_unreadable", "The field or clipboard could not be read. No comparison result is available.");
+      }
+    }
     if (command.action === "fill") {
       const element = await this.resolve(command.reference);
-      const sensitive = await element.evaluate((node) => /password|credential|secret|token|otp|one.time|username|user.?id|api.?key|pin/i.test(node.outerHTML + Array.from(node.labels ?? []).map((label) => label.innerText).join(" ")));
+      const sensitive = await element.evaluate((node, pattern) => new RegExp(pattern, "i").test(node.outerHTML.replace(/value=("[^"]*"|'[^']*')/g, "") + Array.from(node.labels ?? []).map((label) => label.innerText).join(" ")), SENSITIVE_FIELD_PATTERN);
       if (command.secret || sensitive) this.registerSecrets(command.value);
       await element.fill(command.value, { timeout: timeoutMs });
       return;
@@ -747,6 +845,7 @@ var PageControls = class {
       return;
     }
     if (command.action === "click") {
+      await this.page.context().grantPermissions(["clipboard-read", "clipboard-write"], { origin: this.origin });
       const element = await this.resolve(command.reference);
       const deadlineAt = Date.now() + timeoutMs;
       await element.click({ trial: true, timeout: timeoutMs });
@@ -768,12 +867,33 @@ var PageControls = class {
     }
     if (command.action === "scroll") {
       if (!Number.isFinite(command.x) || !Number.isFinite(command.y)) throw new ControlError("invalid_invocation", "Scroll offsets must be finite pixel values.");
-      if (command.reference) {
-        await (await this.resolve(command.reference)).evaluate((node, offsets) => node.scrollBy({ left: offsets.x, top: offsets.y, behavior: "instant" }), command);
-      } else {
-        await this.page.evaluate((offsets) => window.scrollBy({ left: offsets.x, top: offsets.y, behavior: "instant" }), command);
-      }
+      const element = command.reference ? await this.resolve(command.reference) : void 0;
+      const scroll = await this.page.evaluate(({ element: node, offsets }) => {
+        const scrollable = (item) => {
+          const style = getComputedStyle(item);
+          return /(auto|scroll)/.test(style.overflow + style.overflowY + style.overflowX) && (item.scrollHeight > item.clientHeight || item.scrollWidth > item.clientWidth);
+        };
+        const containers = [];
+        if (node) {
+          for (let parent = node.parentElement; parent; parent = parent.parentElement) if (scrollable(parent)) containers.push(parent);
+          if (document.scrollingElement && !containers.includes(document.scrollingElement)) containers.push(document.scrollingElement);
+        } else {
+          const candidates = Array.from(document.querySelectorAll("body *")).filter(scrollable);
+          const root = document.scrollingElement;
+          if (root && !/(hidden|clip)/.test(getComputedStyle(root).overflowY) && (root.scrollHeight > root.clientHeight || root.scrollWidth > root.clientWidth)) candidates.push(root);
+          candidates.sort((a, b) => b.clientWidth * b.clientHeight - a.clientWidth * a.clientHeight);
+          containers.push(candidates[0] ?? document.scrollingElement);
+        }
+        const positions = () => containers.map((item) => ({ x: item.scrollLeft, y: item.scrollTop }));
+        const before = positions();
+        if (node) node.scrollIntoView({ block: "nearest", inline: "nearest", behavior: "instant" });
+        else containers[0].scrollBy({ left: offsets.x, top: offsets.y, behavior: "instant" });
+        const after = positions();
+        const moved = before.some((position, i) => position.x !== after[i].x || position.y !== after[i].y);
+        return { outcome: moved ? "moved" : node ? "already_visible" : "at_boundary", before, after };
+      }, { element: element ?? null, offsets: command });
       await this.page.evaluate(() => new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(() => resolve()))));
+      return { scroll };
     }
   }
 };
@@ -1915,6 +2035,13 @@ async function inspectPage(page, mode, crawler) {
     }
   };
 }
+var ACCESS_TOKEN_SCREENSHOT_MASK = '[data-lmt-sensitive="zerodha-access-token"]';
+async function screenshotMaskOptions(page) {
+  const playwrightPage = page;
+  const locator = playwrightPage.locator(ACCESS_TOKEN_SCREENSHOT_MASK);
+  if (!await locator.count()) return {};
+  return { mask: [locator], maskColor: "#000000" };
+}
 async function captureScreenshot(page, sessionId, action, fullPage, timeoutMs, onTimeout) {
   if (!page) return { outcome: "unavailable", reason: "No browser page was available." };
   if (!page.screenshot) return { outcome: "failed", error: "The browser does not support screenshots." };
@@ -1930,12 +2057,15 @@ async function captureScreenshot(page, sessionId, action, fullPage, timeoutMs, o
     await withDeadline(
       (async () => {
         await waitForRenderedPage(page);
-        return page.screenshot({
+        const maskOptions = await screenshotMaskOptions(page);
+        const options = {
           path: screenshotPath,
           fullPage,
           animations: "disabled",
-          timeout: timeoutMs
-        });
+          timeout: timeoutMs,
+          ...maskOptions
+        };
+        return fullPage ? captureFullContent(page, options) : page.screenshot(options);
       })(),
       timeoutMs,
       onTimeout
@@ -2189,11 +2319,12 @@ async function runSessionOwner(options) {
       guidedError = error2;
     }
     let inspected = null;
+    let controlResult;
     let actionError;
     try {
       if (isControlAction(request)) {
         const budget = Math.max(1, remainingMs(deadlineAt) - Math.min(2e3, request.timeoutMs / 4));
-        await withDeadline(crawler.controls().act(request, budget, () => {
+        controlResult = await withDeadline(crawler.controls().act(request, budget, () => {
           submitted = true;
         }), budget, () => void 0);
       }
@@ -2235,6 +2366,7 @@ async function runSessionOwner(options) {
       sessionId: options.sessionId,
       action: request.action,
       outcome: error instanceof ControlError && error.code === "action_uncertain" ? "uncertain" : error ? submitted && !(error instanceof OperatorActionError) && !(error instanceof LoginAttemptError) ? "uncertain" : "failure" : "success",
+      ...!actionError ? controlResult : {},
       ...loginAttempt ? { loginAttempt: { attemptId: loginAttempt.attemptId, status: loginAttempt.status } } : {},
       url: inspected?.url ?? (page ? safeUrl(crawler?.controls().redact(page.url()) ?? page.url()) : null),
       visibleState: inspected?.visibleState ?? null,
@@ -2387,6 +2519,10 @@ Usage:
   lmt-web inspect --session ID [--full-page] [--timeout 30s]
   lmt-web click --session ID --ref REF
   lmt-web fill --session ID --ref REF --value TEXT [--secret]
+  lmt-web read --session ID --ref REF
+  lmt-web clipboard --session ID
+  lmt-web check --session ID --ref REF --stdin
+  lmt-web clipboard-check --session ID --ref REF
   lmt-web select --session ID --ref REF --value OPTION
   lmt-web scroll --session ID [--ref REF] [--x PIXELS] [--y PIXELS]
   lmt-web navigate --session ID --url PATH_OR_URL
@@ -2474,7 +2610,7 @@ async function parseCommand(argv) {
   }
   let payload;
   try {
-    payload = await definition.build({ values, flags, readOtp: () => readOtp(timeoutMs) });
+    payload = await definition.build({ values, flags, readOtp: () => readOtp(timeoutMs), readInput: () => readInput(timeoutMs) });
   } catch (error) {
     throw new InvocationError(error instanceof Error ? error.message : "The action input could not be read.", error instanceof InvocationError ? error.code : "action_failed", action);
   }
@@ -2487,6 +2623,23 @@ function emit(result, diagnostic) {
   if (diagnostic) process.stderr.write(`${diagnostic}
 `);
 }
+async function readInput(timeoutMs) {
+  if (process.stdin.isTTY) throw new InvocationError("Pipe the expected value on stdin.");
+  let input = "";
+  const timer = setTimeout(() => process.stdin.destroy(new Error("Input deadline exceeded.")), timeoutMs);
+  try {
+    for await (const chunk of process.stdin) {
+      input += chunk.toString();
+      if (Buffer.byteLength(input) > 16384) throw new InvocationError("Expected value is too large.");
+    }
+    return input.replace(/\r?\n$/, "");
+  } catch {
+    throw new InvocationError("Expected value could not be read from stdin.");
+  } finally {
+    input = "";
+    clearTimeout(timer);
+  }
+}
 async function readOtp(timeoutMs) {
   let input = "";
   const timer = setTimeout(() => process.stdin.destroy(new Error("OTP input exceeded its deadline.")), timeoutMs);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ai-digital-factory/lmt-web-cli",
-  "version": "0.1.1",
+  "version": "0.1.3",
   "type": "module",
   "bin": {
     "lmt-web": "dist/index.js"