@isdk/web-fetcher 0.2.12 → 0.3.1

package/README.engine.cn.md

@@ -12,6 +12,14 @@
 
 > ℹ️ 该系统建立在 [Crawlee](https://crawlee.dev/) 库之上，并使用其强大的爬虫抽象。
 
+### Debug 模式与追踪 (Debug Mode & Tracing)
+
+当启用 `debug: true` 选项时，引擎会提供其内部操作的详细追踪信息：
+
+1. **详细日志追踪**：每个主要步骤（请求处理、元素选择、数据提取）都会记录到控制台，并带有 `[FetchEngine:id]` 前缀。
+2. **提取洞察**：在执行 `extract()` 期间，引擎会记录每个选择器匹配到的元素数量以及正在提取的具体值，这使得调试复杂或错位的 Schema 变得更加容易。
+3. **元数据 (Metadata)**：`FetchResponse` 将包含一个丰富的 `metadata` 对象，其中包含引擎详情、计时指标（如果可用）和代理信息。
+
 ---
 
 ## 🧩 2. 核心概念
@@ -20,13 +28,23 @@
 
 这是定义所有抓取引擎契约的抽象基类。
 
-* **职责**：为导航、内容检索和用户交互等操作提供一致的高级 API。
+* **职责**：为导航、内容检索和用户交互等操作提供一致的高级 API。它充当核心 **动作分发器 (Action Dispatcher)**，统一处理引擎无关的逻辑（如 `extract`、`pause`、`getContent`），并将其余动作委托给具体实现。
 * **关键抽象**：
   * **生命周期**：`initialize()` 和 `cleanup()` 方法。
   * **核心操作**：`goto()`、`getContent()`、`click()`、`fill()`、`submit()`、`waitFor()`、`extract()`。
+  * **DOM 原子操作 (Primitives)**: `_querySelectorAll()`、`_extractValue()`、`_parentElement()`、`_nextSiblingsUntil()`。
   * **配置与状态**：`headers()`、`cookies()`、`blockResources()`、`getState()`、`sessionPoolOptions`。
 * **静态注册表**：它维护所有可用引擎实现的静态注册表（`FetchEngine.register`），允许通过 `id` 或 `mode` 动态选择引擎。
 
+### `FetchElementScope`
+
+**`FetchElementScope`** 是引擎相关的 DOM 元素“句柄”或“上下文”。
+
+- 在 **`CheerioFetchEngine`** 中，它是 Cheerio API (`$`) 与当前选择 (`el`) 的组合。
+- 在 **`PlaywrightFetchEngine`** 中，它是 `Locator`。
+
+所有提取和交互的原子操作都基于此 Scope 运行，确保了在不同底层技术之上引用元素的统一方式。
+
 ### `FetchEngine.create(context, options)`
 
 此静态工厂方法是创建引擎实例的指定入口点。它会自动选择并初始化合适的引擎。
@@ -69,14 +87,14 @@ await session.executeAll([
 引擎支持在多次执行之间持久化和恢复会话状态（主要是 Cookie）。
 
 * **灵活的会话隔离与存储控制**：库提供了对会话数据存储和隔离的精细控制，通过 `storage` 配置实现：
-    * **`id`**：自定义存储标识符。
-        * **隔离（默认）**：如果省略，每个会话将获得一个唯一 ID，确保 `RequestQueue`、`KeyValueStore` 和 `SessionPool` 完全隔离。
-        * **共享**：在不同会话中提供相同的 `id` 可以让它们共享底层存储，适用于保持持久登录状态。
-    * **`persist`**：(boolean) 是否启用磁盘持久化（对应 Crawlee 的 `persistStorage`）。默认为 `false`（仅在内存中）。
-    * **`purge`**：(boolean) 会话关闭时是否删除存储（清理 `RequestQueue` 和 `KeyValueStore`）。默认为 `true`。
-        * 设置 `purge: false` 并配合固定的 `id`，可以创建跨应用重启依然存在的持久会话。
-    * **`config`**：允许向底层 Crawlee 实例传递原生配置。
-        * **注意**：当 `persist` 为 true 时，在 config 中使用 `localDataDirectory` 指定存储路径（例如：`storage: { persist: true, config: { localDataDirectory: './my-data' } }`）。
+  * **`id`**：自定义存储标识符。
+    * **隔离（默认）**：如果省略，每个会话将获得一个唯一 ID，确保 `RequestQueue`、`KeyValueStore` 和 `SessionPool` 完全隔离。
+    * **共享**：在不同会话中提供相同的 `id` 可以让它们共享底层存储，适用于保持持久登录状态。
+  * **`persist`**：(boolean) 是否启用磁盘持久化（对应 Crawlee 的 `persistStorage`）。默认为 `false`（仅在内存中）。
+  * **`purge`**：(boolean) 会话关闭时是否删除存储（清理 `RequestQueue` 和 `KeyValueStore`）。默认为 `true`。
+    * 设置 `purge: false` 并配合固定的 `id`，可以创建跨应用重启依然存在的持久会话。
+  * **`config`**：允许向底层 Crawlee 实例传递原生配置。
+    * **注意**：当 `persist` 为 true 时，在 config 中使用 `localDataDirectory` 指定存储路径（例如：`storage: { persist: true, config: { localDataDirectory: './my-data' } }`）。
 * **`sessionState`**: 一个完整的状态对象（源自 Crawlee 的 SessionPool），可用于完全恢复之前的会话。该状态会**自动包含在每个 `FetchResponse` 中**，方便进行持久化，并在以后初始化引擎时通过选项传回。
 * **`sessionPoolOptions`**: 允许对底层的 Crawlee `SessionPool` 进行高级配置（例如 `maxUsageCount`, `maxPoolSize`）。
 * **`overrideSessionState`**: 如果设置为 `true`，则强制引擎使用提供的 `sessionState` 覆盖存储中的任何现有持久化状态。当你希望确保会话以确切提供的状态启动，忽略持久化层中的任何陈旧数据时，这非常有用。
@@ -86,6 +104,7 @@ await session.executeAll([
 
 **优先级规则：**
 如果同时提供了 `sessionState` 和 `cookies`，引擎将采用**“合并并覆盖”**策略：
+
 1. 首先从 `sessionState` 恢复会话。
 2. 然后在之上应用显式的 `cookies`。
    * **结果**：`sessionState` 中任何冲突的 Cookie 都会被显式的 `cookies` **覆盖**。
@@ -114,8 +133,11 @@ await session.executeAll([
     * 页面上下文用于解析 `goto()` 调用返回的 `Promise`。
     * 页面被标记为“活动”状态 (`isPageActive = true`)。
     * 至关重要的是，在 `requestHandler` 返回之前，它会启动一个**动作循环** (`_executePendingActions`)。此循环通过监听 `EventEmitter` 上的事件，有效地**暂停 `requestHandler`**，从而保持页面上下文的存活。
+    * **严格顺序执行与重入保护**：循环使用内部队列确保所有动作按分发顺序执行。同时包含重入保护，允许组合动作调用原子动作而不产生死锁。
 5. **交互式动作 (`click`, `fill` 等)**：消费者现在可以调用 `await engine.click(...)`。此方法将一个动作分派到 `EventEmitter` 并返回一个新的 `Promise`。
-6. **动作执行**：仍在原始 `requestHandler` 作用域内运行的动作循环，会监听到该事件。因为它能访问页面上下文，所以可以执行*实际的*交互（例如 `page.click(...)`）。
+6. **动作执行**：仍在原始 `requestHandler` 作用域内运行的动作循环，会监听到该事件。
+    * **统一处理的动作**：像 `extract`、`pause` 和 `getContent` 这样的动作会直接由 `FetchEngine` 基类使用统一的逻辑进行处理。
+    * **委托执行的动作**：引擎相关的交互（如 `click`、`fill`）会委托给子类的 `executeAction` 实现。
 7. **健壮的清理**：当调用 `dispose()` 或 `cleanup()` 时：
     * 设置 `isEngineDisposed` 标志以阻止新动作。
     * 发出 `dispose` 信号以唤醒并终止动作循环。
@@ -134,8 +156,9 @@ await session.executeAll([
 * **机制**: 使用 `CheerioCrawler` 通过原始 HTTP 请求抓取页面并解析静态 HTML。
 * **行为**:
   * ✅ **快速轻量**：非常适合追求速度和低资源消耗的场景。
+  * ✅ **符合 HTTP 标准的重定向**：正确处理 301-303 和 307/308 重定向，按照 HTTP 规范保留方法/正文或转换为 GET。
   * ❌ **无 JavaScript 执行**：无法与客户端渲染的内容交互。
-  * ⚙️ **模拟交互**：像 `click` 和 `submit` 这样的动作是通过发起新的 HTTP 请求来模拟的。
+  * ⚙️ **模拟交互**：像 `click` 和 `submit` 这样的动作是通过发起新的 HTTP 请求来模拟的。**仅浏览器支持的动作**（如 `mouseMove`, `keyboardType`）将抛出 `not_supported` 错误。
 * **用例**: 抓取静态网站、服务器渲染页面或 API。
 
 ### `PlaywrightFetchEngine` (browser 模式)
@@ -160,17 +183,98 @@ await session.executeAll([
 * **用例**: 抓取受 Cloudflare 或其他高级机器人检测系统保护的网站。
 * **注意**: 此功能需要额外的依赖项（`camoufox-js`, `firefox`），并可能带来性能开销。
 
+#### 配置 (Configuration)
+
+您可以通过选项中的 `browser` 属性来配置浏览器引擎：
+
+*   `headless` (boolean): 是否以无头模式运行浏览器（默认：`true`）。
+*   `launchOptions` (object): 原生 Playwright [LaunchOptions](https://playwright.dev/docs/api/class-browsertype#browser-type-launch)，直接传递给浏览器启动器（例如 `slowMo`, `args`, `devtools`）。
+
+    ```typescript
+    const result = await fetchWeb({
+      url: 'https://example.com',
+      engine: 'browser',
+      browser: {
+        headless: false,
+        launchOptions: {
+          slowMo: 100, // 将操作减慢 100ms
+          args: ['--start-maximized'] // 传递自定义参数
+        }
+      }
+    });
+    ```
+
 ---
 
 ## 📊 5. 使用 `extract()` 进行数据提取
 
 `extract()` 方法提供了一种强大的声明式方式来从网页中提取结构化数据。它通过一个 **Schema (模式)** 来定义您期望的 JSON 输出结构,引擎会自动处理 DOM 遍历和数据提取。
 
-### 核心设计: Schema 规范化
+### 核心设计: 三层提取架构
+
+为了确保不同引擎之间的一致性并保持高质量,提取系统分为三个层次:
+
+1. **规范化层 (`src/core/normalize-extract-schema.ts`)**: 将用户提供的各种简写模式预处理为统一的内部格式,处理 CSS 筛选器的合并。
+2. **核心提取逻辑 (`src/core/extract.ts`)**: 引擎无关层,负责提取的工作流分发。它通过 **Dispatcher (`_extract`)** 将任务分发给 `_extractObject`、`_extractArray` 或 `_extractValue`。该层管理递归、严格模式/必填字段验证、锚点解析、性能优化的树操作（LCA、冒泡）以及顺序消费游标逻辑。
+3. **引擎接口层 (`IExtractEngine`)**: 在核心层定义,由各引擎实现,提供底层的 DOM 原子操作。
+
+#### `IExtractEngine` 实现准则
+
+为了保持跨引擎的一致性，所有实现必须遵循以下行为契约：
+
+- **`_querySelectorAll`**:
+  - 必须按 **DOM 文档顺序**返回匹配的元素。
+  - 必须检查 Scope 元素**自身**是否匹配选择器，并搜索其**后代**。
+- **`_nextSiblingsUntil`**:
+  - 必须返回一个平铺的兄弟节点列表，从起始锚点之后开始，到第一个匹配 `untilSelector` 的元素之前停止。
+- **`_isSameElement`**:
+  - 必须基于元素身份（Identity）进行比较，而不是内容。
+- **`_findClosestAncestor`**:
+  - 必须高效地查找在给定候选集中的最近祖先元素。
+  - 必须针对浏览器引擎进行优化，避免多次 IPC 调用。
+- **`_contains`**:
+  - 必须实现标准的 DOM `Node.contains()` 行为。
+  - 必须针对高频边界检查进行优化。
+- **`_findCommonAncestor`**:
+  - 必须查找两个元素的最近公共祖先 (Lowest Common Ancestor, LCA)。
+  - **性能关键**: 在浏览器引擎中，这必须在单次 `evaluate` 调用中执行，以最小化 IPC (进程间通信) 开销。
+- **`_findContainerChild`**:
+  - 必须在容器中查找包含特定后代元素的直系子元素。
+  - **性能关键**: 这取代了在 Node.js 环境中的手动“冒泡”循环，显著减少了深层 DOM 树的操作开销。
+- **`_bubbleUpToScope` (内部辅助方法)**:
+  - 实现从深层元素向上冒泡到当前作用域内直接祖先的逻辑。
+  - 支持可选的 `depth` 参数来限制向上遍历的层级。
+  - 必须包含最大深度限制 (默认 1000) 以防止无限循环。
+
+这种架构确保了诸如 **列对齐 (Columnar Alignment)**、**分段扫描 (Segmented Scanning)** 以及 **属性锚点跳转 (Anchor Jumping)** 等复杂功能在不同引擎下表现高度一致。
+
+这种解耦确保了诸如 **列对齐 (Columnar Alignment)**、**分段扫描 (Segmented Scanning)** 以及 **属性锚点跳转 (Anchor Jumping)** 等复杂功能,无论是在快速的 Cheerio 引擎还是完整的 Playwright 浏览器中,其行为都完全一致。
+
+### Schema 规范化
+
+为了提升易用性和灵活性,`extract` 方法在内部实现了一个**“规范化 (Normalization)”**层。这意味着您可以提供语义清晰的简写形式,在执行前,引擎会自动将其转换为标准的内部格式。
+
+#### 1. 简写规则
+
+- **字符串简写**: 一个简单的字符串（如 `'h1'`）会自动扩展为 `{ selector: 'h1', type: 'string', mode: 'text' }`。
+- **隐式对象简写**: 如果你提供一个没有显式 `type: 'object'` 的对象，它会被视为一个 `object` 模式，其中对象的键即为要提取的属性名。
+  - *示例*: `{ "title": "h1" }` 变为 `{ "type": "object", "properties": { "title": { "selector": "h1" } } }`。
+- **筛选器简写**: 如果在提供 `selector` 的同时提供了 `has` 或 `exclude`，它们会自动使用 `:has()` 和 `:not()` 伪类合并到 CSS 选择器中。
+  - *示例*: `{ "selector": "div", "has": "p" }` 变为 `{ "selector": "div:has(p)" }`。
+- **数组简写**: 在 `array` 模式下直接提供 `attribute` 会作为其 `items` 的简写。
+  - *示例*: `{ "type": "array", "attribute": "href" }` 变为 `{ "type": "array", "items": { "type": "string", "attribute": "href" } }`。
+
+#### 2. 配置与数据 (关键字分离)
+
+在**隐式对象 (Implicit Objects)** 中，引擎必须区分*配置* (去哪里找) 和*数据* (提取什么)。
+
+- **上下文配置关键字 (Context Keys)**：`selector`、`has`、`exclude`、`required`、`strict` 和 `depth` 是保留的，用于定义提取上下文和校验规则。它们保留在 Schema 的根部。
+- **数据关键字 (Data Keys)**：所有其他键 (包括 `items`、`attribute`、`mode`，甚至名为 `type` 的字段) 都会被移动到 `properties` 对象中，作为要提取的数据字段。
+- **冲突处理**：您可以安全地提取名为 `type` 的字段，只要其值不是保留的 Schema 类型关键字 (`string`、`number`、`boolean`、`html`、`object`、`array`)。
 
-为了提升易用性和灵活性,`extract` 方法在内部实现了一个**“规范化 (Normalization)”**层。这意味着您可以提供语义清晰的简写形式,在执行前,引擎会自动将其转换为标准的、更详细的内部格式。这使得编写复杂的提取规则变得简单直观。
+#### 3. 跨引擎一致性
 
-### Schema 结构
+规范化层确保了无论你使用的是 `cheerio` (http) 还是 `playwright` (browser) 引擎，复杂的简写逻辑行为都完全一致，提供了一个统一的“AI 友好”接口。
 
 一个 Schema 可以是以下三种类型之一:
 

package/README.engine.md

@@ -12,6 +12,14 @@ The `engine` directory contains the core logic for the web fetcher. Its primary
 
 > ℹ️ The system is built on top of the [Crawlee](https://crawlee.dev/) library, using its powerful crawler abstractions.
 
+### Debug Mode & Tracing
+
+When the `debug: true` option is enabled, the engine provides detailed tracing of its internal operations:
+
+1. **Detailed Tracing**: Every major step (request processing, element selection, data extraction) is logged to the console with a `[FetchEngine:id]` prefix.
+2. **Extraction Insights**: During `extract()`, the engine logs how many elements were matched for each selector and the specific values being extracted, making it easier to debug complex or misaligned schemas.
+3. **Metadata**: The `FetchResponse` will include an enriched `metadata` object containing engine details, timing metrics (where available), and proxy information.
+
 ---
 
 ## 🧩 2. Core Concepts
@@ -20,13 +28,23 @@ The `engine` directory contains the core logic for the web fetcher. Its primary
 
 This is the abstract base class that defines the contract for all fetch engines.
 
-* **Role**: To provide a consistent, high-level API for actions like navigation, content retrieval, and user interaction.
+* **Role**: To provide a consistent, high-level API for actions like navigation, content retrieval, and user interaction. It acts as the central **Action Dispatcher**, handling engine-agnostic logic (like `extract`, `pause`, `getContent`) and delegating others.
 * **Key Abstractions**:
   * **Lifecycle**: `initialize()` and `cleanup()` methods.
   * **Core Actions**: `goto()`, `getContent()`, `click()`, `fill()`, `submit()`, `waitFor()`, `extract()`.
+  * **DOM Primitives**: `_querySelectorAll()`, `_extractValue()`, `_parentElement()`, `_nextSiblingsUntil()`.
   * **Configuration & State**: `headers()`, `cookies()`, `blockResources()`, `getState()`, `sessionPoolOptions`.
 * **Static Registry**: It maintains a static registry of all available engine implementations (`FetchEngine.register`), allowing for dynamic selection by `id` or `mode`.
 
+### `FetchElementScope`
+
+The **`FetchElementScope`** is the engine-specific "handle" or "context" for a DOM element.
+
+- In **`CheerioFetchEngine`**, it is a combination of the Cheerio API (`$`) and the current selection (`el`).
+- In **`PlaywrightFetchEngine`**, it is a `Locator`.
+
+All extraction and interaction primitives operate on this scope, ensuring a unified way to reference elements across different underlying technologies.
+
 ### `FetchEngine.create(context, options)`
 
 This static factory method is the designated entry point for creating an engine instance. It automatically selects and initializes the appropriate engine.
@@ -69,14 +87,14 @@ await session.executeAll([
 The engine supports persisting and restoring session state (primarily cookies) between executions.
 
 * **Flexible Session Isolation & Storage**: The library provides fine-grained control over how session data is stored and isolated via the `storage` configuration:
-    * **`id`**: A custom string to identify the storage.
-        * **Isolation (Default)**: If omitted, each session gets a unique ID, ensuring complete isolation of `RequestQueue`, `KeyValueStore`, and `SessionPool`.
-        * **Sharing**: Providing the same `id` across sessions allows them to share the same underlying storage, useful for persistent login sessions.
-    * **`persist`**: (boolean) Whether to enable disk persistence (Crawlee's `persistStorage`). Defaults to `false` (in-memory).
-    * **`purge`**: (boolean) Whether to delete the storage (drop `RequestQueue` and `KeyValueStore`) when the session is closed. Defaults to `true`.
-        * Set `purge: false` and provide a fixed `id` to create a truly persistent session that survives across application restarts.
-    * **`config`**: Allows passing raw configuration to the underlying Crawlee instance.
-        * **Note**: When `persist` is true, use `localDataDirectory` in the config to specify the storage path (e.g., `storage: { persist: true, config: { localDataDirectory: './my-data' } }`).
+  * **`id`**: A custom string to identify the storage.
+    * **Isolation (Default)**: If omitted, each session gets a unique ID, ensuring complete isolation of `RequestQueue`, `KeyValueStore`, and `SessionPool`.
+    * **Sharing**: Providing the same `id` across sessions allows them to share the same underlying storage, useful for persistent login sessions.
+  * **`persist`**: (boolean) Whether to enable disk persistence (Crawlee's `persistStorage`). Defaults to `false` (in-memory).
+  * **`purge`**: (boolean) Whether to delete the storage (drop `RequestQueue` and `KeyValueStore`) when the session is closed. Defaults to `true`.
+    * Set `purge: false` and provide a fixed `id` to create a truly persistent session that survives across application restarts.
+  * **`config`**: Allows passing raw configuration to the underlying Crawlee instance.
+    * **Note**: When `persist` is true, use `localDataDirectory` in the config to specify the storage path (e.g., `storage: { persist: true, config: { localDataDirectory: './my-data' } }`).
 * **`sessionState`**: A comprehensive state object (derived from Crawlee's SessionPool) that can be used to fully restore a previous session. This state is **automatically included in every `FetchResponse`**, making it easy to persist and later provide back to the engine during initialization.
 * **`sessionPoolOptions`**: Allows advanced configuration of the underlying Crawlee `SessionPool` (e.g., `maxUsageCount`, `maxPoolSize`).
 * **`overrideSessionState`**: If set to `true`, it forces the engine to overwrite any existing persistent state in the storage with the provided `sessionState`. This is useful when you want to ensure the session starts with the exact state provided, ignoring any stale data in the persistence layer.
@@ -86,6 +104,7 @@ The engine supports persisting and restoring session state (primarily cookies) b
 
 **Precedence Rule:**
 If both `sessionState` and `cookies` are provided, the engine adopts a **"Merge and Override"** strategy:
+
 1. The session is first restored from the `sessionState`.
 2. The explicit `cookies` are then applied on top.
    * **Result:** Any conflicting cookies in `sessionState` will be **overwritten** by the explicit `cookies`.
@@ -114,8 +133,11 @@ Our engine solves this by creating a bridge between the external API calls and t
     * The page context is used to resolve the `Promise` from the `goto()` call.
     * The page is marked as "active" (`isPageActive = true`).
     * Crucially, before the `requestHandler` returns, it starts an **action loop** (`_executePendingActions`). This loop effectively **pauses the `requestHandler`** by listening for events on an `EventEmitter`, keeping the page context alive.
+    * **Strict Sequential Execution & Re-entrancy**: The loop uses an internal queue to ensure all actions are executed in the exact order they were dispatched. It also includes re-entrancy protection to allow composite actions to call atomic actions without deadlocking.
 5. **Interactive Actions (`click`, `fill`, etc.)**: The consumer can now call `await engine.click(...)`. This dispatches an action to the `EventEmitter` and returns a new `Promise`.
-6. **Action Execution**: The action loop, still running within the original `requestHandler`'s scope, hears the event. Because it has access to the page context, it can perform the *actual* interaction (e.g., `page.click(...)`).
+6. **Action Execution**: The action loop, still running within the original `requestHandler`'s scope, hears the event.
+    * **Centralized Actions**: Actions like `extract`, `pause`, and `getContent` are processed immediately by the `FetchEngine` base class using the unified logic.
+    * **Delegated Actions**: Engine-specific interactions (e.g., `click`, `fill`) are delegated to the subclass's `executeAction` implementation.
 7. **Robust Cleanup**: When `dispose()` or `cleanup()` is called:
     * An `isEngineDisposed` flag is set to prevent new actions.
     * A `dispose` signal is emitted to wake up and terminate the action loop.
@@ -134,8 +156,9 @@ There are two primary engine implementations:
 * **Mechanism**: Uses `CheerioCrawler` to fetch pages via raw HTTP and parse static HTML.
 * **Behavior**:
   * ✅ **Fast and Lightweight**: Ideal for speed and low resource consumption.
+  * ✅ **HTTP-Compliant Redirects**: Correctly handles 301-303 and 307/308 redirects, preserving methods/bodies or converting to GET as per HTTP specifications.
   * ❌ **No JavaScript Execution**: Cannot interact with client-side rendered content.
-  * ⚙️ **Simulated Interaction**: Actions like `click` and `submit` are simulated by making new HTTP requests.
+  * ⚙️ **Simulated Interaction**: Actions like `click` and `submit` are simulated by making new HTTP requests. **Browser-only actions** (e.g., `mouseMove`, `keyboardType`) will throw a `not_supported` error.
 * **Use Case**: Scraping static websites, server-rendered pages, or APIs.
 
 ### `PlaywrightFetchEngine` (browser mode)
@@ -160,17 +183,95 @@ To combat sophisticated anti-bot measures, the `PlaywrightFetchEngine` offers an
 * **Use Case**: Scraping websites protected by services like Cloudflare or other advanced bot-detection systems.
 * **Note**: This feature requires additional dependencies (`camoufox-js`, `firefox`) and may have a performance overhead.
 
+#### Configuration
+
+You can configure the browser engine via the `browser` property in options:
+
+*   `headless` (boolean): Whether to run browser in headless mode (default: `true`).
+*   `launchOptions` (object): Native Playwright [LaunchOptions](https://playwright.dev/docs/api/class-browsertype#browser-type-launch) passed directly to the browser launcher (e.g., `slowMo`, `args`, `devtools`).
+
+    ```typescript
+    const result = await fetchWeb({
+      url: 'https://example.com',
+      engine: 'browser',
+      browser: {
+        headless: false,
+        launchOptions: {
+          slowMo: 100, // Slow down operations by 100ms
+          args: ['--start-maximized'] // Pass custom arguments
+        }
+      }
+    });
+    ```
+
 ---
 
 ## 📊 5. Data Extraction with `extract()`
 
 The `extract()` method provides a powerful, declarative way to pull structured data from a web page. It uses a **Schema** to define the structure of your desired JSON output, and the engine automatically handles DOM traversal and data extraction.
 
-### Core Design: Schema Normalization
+### Core Design: The Three-Layer Architecture
+
+To ensure consistency across engines and maintain high quality, the extraction system is divided into three layers:
+
+1. **Normalization Layer (`src/core/normalize-extract-schema.ts`)**: Pre-processes user-provided schemas into a canonical internal format, handling CSS filter merging and implicit object detection.
+2. **Core Extraction Logic (`src/core/extract.ts`)**: An engine-agnostic layer responsible for the extraction workflow. It dispatches tasks to `_extractObject`, `_extractArray`, or `_extractValue` via a **Dispatcher (`_extract`)**. It manages recursion, strict mode, required field validation, anchor resolution, performance-optimized tree operations (LCA, bubbling), and sequential consumption cursors.
+3. **Engine Interface (`IExtractEngine`)**: Defined at the core layer and implemented by engines to provide low-level DOM primitives.
+
+#### Implementation Rules for `IExtractEngine`
+
+To maintain cross-engine consistency, all implementations MUST follow these behavior contracts:
+
+- **`_querySelectorAll`**:
+  - MUST return matching elements in **document order**.
+  - MUST check if the scope element(s) **themselves** match the selector and search their **descendants**.
+- **`_nextSiblingsUntil`**:
+  - MUST return a flat list of siblings starting *after* the anchor and stopping *before* the first element matching the `untilSelector`.
+- **`_isSameElement`**:
+  - MUST compare elements based on **identity**, not content.
+- **`_findClosestAncestor`**:
+  - MUST efficiently find the closest ancestor of an element that exists in a given set of candidates.
+  - MUST be optimized to avoid multiple IPC calls in browser-based engines.
+- **`_contains`**:
+  - MUST implement standard DOM `Node.contains()` behavior.
+  - MUST be optimized for high-frequency boundary checks.
+- **`_findCommonAncestor`**:
+  - MUST find the Lowest Common Ancestor (LCA) of two elements.
+  - **Performance Critical**: In browser engines, this MUST be executed within a single `evaluate` call to minimize IPC (Inter-Process Communication) overhead.
+- **`_findContainerChild`**:
+  - MUST find the direct child of a container that contains a specific descendant.
+  - **Performance Critical**: This replaces manual "bubble-up" loops in the Node.js context, significantly reducing overhead for deep DOM trees.
+- **`_bubbleUpToScope` (Internal Helper)**:
+  - Implements the logic to bubble up from a deep element to its direct ancestor in the current scope.
+  - Supports an optional `depth` parameter to limit how many parent levels to traverse.
+  - MUST include a maximum depth limit (default 1000) to prevent infinite loops.
+
+This architecture ensures that complex features like **Columnar Alignment**, **Segmented Scanning**, and **Anchor Jumping** behave identically across the fast Cheerio engine and the full Playwright browser.
+
+### Schema Normalization
+
+To enhance usability and flexibility, the `extract` method internally implements a **"Normalization"** layer. This allows you to provide semantically clear shorthands, which are automatically converted into a standardized internal format.
+
+#### 1. Shorthand Rules
+
+- **String Shorthand**: A simple string like `'h1'` is automatically expanded to `{ selector: 'h1', type: 'string', mode: 'text' }`.
+- **Implicit Object Shorthand**: If you provide an object without an explicit `type: 'object'`, it is automatically treated as an `object` schema where the keys are the property names.
+  - *Example*: `{ "title": "h1" }` becomes `{ "type": "object", "properties": { "title": { "selector": "h1" } } }`.
+- **Filter Shorthand**: If you provide `has` or `exclude` alongside a `selector`, they are automatically merged into the CSS selector using `:has()` and `:not()` pseudo-classes.
+- **Array Shorthand**: Providing `attribute` directly on an `array` schema acts as a shorthand for its `items`.
+  - *Example*: `{ "type": "array", "attribute": "href" }` becomes `{ "type": "array", "items": { "type": "string", "attribute": "href" } }`.
+
+#### 2. Context vs. Data (Keyword Separation)
+
+In **Implicit Objects**, the engine must distinguish between *configuration* (where to look) and *data* (what to extract).
+
+- **Context Keys**: The keys `selector`, `has`, `exclude`, `required`, `strict`, and `depth` are reserved for defining the extraction context and validation. They stay at the root of the schema.
+- **Data Keys**: All other keys (including `items`, `attribute`, `mode`, or even a field named `type`) are moved into the `properties` object as data fields to be extracted.
+- **Collision Handling**: You can safely extract a field named `type` as long as its value is not one of the reserved schema type keywords (`string`, `number`, `boolean`, `html`, `object`, `array`).
 
-To enhance usability and flexibility, the `extract` method internally implements a **"Normalization"** layer. This means you can provide semantically clear shorthands, and the engine will automatically convert them into a standard, more verbose internal format before execution. This makes writing complex extraction rules simple and intuitive.
+#### 3. Cross-Engine Consistency
 
-### Schema Structure
+This normalization layer ensures that regardless of whether you are using the `cheerio` (http) or `playwright` (browser) engine, the complex shorthand logic behaves identically, providing a consistent "AI-friendly" interface.
 
 A schema can be one of three types:
 

package/README.md

@@ -132,7 +132,7 @@ This is the main entry point for the library.
 * `url` (string): The initial URL to navigate to.
 * `engine` ('http' | 'browser' | 'auto'): The engine to use. Defaults to `auto`.
 * `proxy` (string | string[]): Proxy URL(s) to use for requests.
-* `debug` (boolean): Enable detailed execution metadata (timings, engine used, etc.) in response.
+* `debug` (boolean | string | string[]): Enable detailed execution metadata (timings, engine used, etc.) in response, or enable debug logs for specific categories (e.g., 'extract', 'submit', 'request').
 * `actions` (FetchActionOptions[]): An array of action objects to execute. (Supports `action`/`name` as alias for `id`, and `args` as alias for `params`)
 * `headers` (Record<string, string>): Headers to use for all requests.
 * `cookies` (Cookie[]): Array of cookies to use.
@@ -145,21 +145,30 @@ This is the main entry point for the library.
 * `output` (object): Controls the output fields in `FetchResponse`.
   * `cookies` (boolean): Whether to include cookies in the response (default: `true`).
   * `sessionState` (boolean): Whether to include session state in the response (default: `true`).
+* `browser` (object): Browser engine configuration.
+  * `headless` (boolean): Run in headless mode (default: `true`).
+  * `launchOptions` (object): Playwright launch options (e.g., `{ slowMo: 50, args: [...] }`).
 * `sessionPoolOptions` (SessionPoolOptions): Advanced configuration for the underlying Crawlee SessionPool.
 * ...and many other options for proxy, retries, etc.
 
 ### Built-in Actions
 
-Here are the essential built-in actions:
+The library provides a set of powerful built-in actions, many of which are engine-agnostic and handled centrally for consistency:
 
 * `goto`: Navigates to a new URL.
-* `click`: Clicks on an element specified by a selector.
-* `fill`: Fills an input field with a specified value.
-* `submit`: Submits a form.
-* `waitFor`: Pauses execution to wait for a specific condition (e.g., a timeout, a selector to appear, or network to be idle).
-* `pause`: Pauses execution for manual intervention (e.g., solving a CAPTCHA).
-* `getContent`: Retrieves the full content (HTML, text, etc.) of the current page state.
-* `extract`: Extracts any structured data from the page with ease using an expressive, declarative schema.
+* `click`: Clicks on an element (Engine-specific).
+* `fill`: Fills an input field (Engine-specific).
+* `submit`: Submits a form (Engine-specific).
+* `mouseMove`: Moves the mouse cursor to a specific coordinate or element (Bézier curve supported).
+* `mouseClick`: Triggers a mouse click at the current position or specified coordinates.
+* `keyboardType`: Simulates human-like typing into the currently focused element.
+* `keyboardPress`: Simulates pressing a single key or a key combination.
+* `trim`: Removes elements from the DOM to clean up the page.
+* `waitFor`: Pauses execution to wait for a specific condition (Supports fixed timeouts centrally).
+* `pause`: Pauses execution for manual intervention (Handled centrally).
+* `getContent`: Retrieves the full content of the current page (Handled centrally).
+* `evaluate`: Executes custom JavaScript within the page context.
+* `extract`: Extracts structured data using an engine-agnostic core logic and engine-specific DOM primitives. Supports `required` fields and `strict` validation.
 
 ### Response Structure
 
@@ -172,7 +181,7 @@ The `fetchWeb` function returns an object containing:
   * `cookies`: Array of cookies.
   * `sessionState`: Crawlee session state.
   * `text`, `html`: Page content.
-* `outputs` (Record<string, any>): Data extracted and stored via `storeAs`.
+* `outputs` (Record<string, any>): Data extracted and stored via `storeAs`. Note: When multiple actions store objects into the same key, they are merged instead of overwritten.
 
 ---