@isdk/web-fetcher 0.2.11 → 0.2.12

package/README.action.cn.md

@@ -222,19 +222,26 @@ await fetchWeb({
 
 ###### 1. 提取单个值
 
-最基础的提取，可以指定 **`selector`** (CSS 选择器), **`attribute`** (要提取的属性名), 以及 **`type`** (string, number, boolean, html)。
+最基础的提取，可以指定 **`selector`** (CSS 选择器), **`attribute`** (要提取的属性名), **`type`** (string, number, boolean, html), 以及 **`mode`** (text, innerText)。
 
 ```json
 {
   "id": "extract",
   "params": {
     "selector": "h1.main-title",
-    "type": "string"
+    "type": "string",
+    "mode": "innerText"
   }
 }
 ```
 
-> 上例将提取 class 为 `main-title` 的 `<h1>` 标签的文本内容。
+> **提取模式 (Extraction Modes):**
+>
+> * **`text`** (默认): 提取元素的 `textContent`。
+> * **`innerText`**: 提取渲染后的文本，尊重 CSS 样式并处理换行。
+> * **`html`**: 返回元素的 `innerHTML`。
+> * **`outerHTML`**: 返回包含标签自身的完整 HTML。对于保留元素结构非常有用。
+> 上例将使用 `innerText` 模式提取 class 为 `main-title` 的 `<h1>` 标签的文本内容。
 
 ###### 2. 提取对象
 
@@ -261,21 +268,13 @@ await fetchWeb({
 * **提取文本数组 (默认行为)**: 当您想提取一个文本列表时,只需提供选择器,省略 `items` 即可。这是最常见的用法。
 
     ```json
-
     {
-
       "id": "extract",
-
       "params": {
-
         "type": "array",
-
         "selector": ".tags li"
-
       }
-
     }
-
     ```
 
     > 上例将返回一个包含所有 `<li>` 标签文本的数组, 如 `["tech", "news"]`。
@@ -285,26 +284,88 @@ await fetchWeb({
     ```json
 
     {
-
       "id": "extract",
-
       "params": {
-
         "type": "array",
-
         "selector": ".gallery img",
-
         "attribute": "src"
-
       }
-
     }
-
     ```
 
     > 上例将返回一个包含所有 `<img>` 标签 `src` 属性的数组。
 
-###### 4. 精确筛选: `has` 和 `exclude`
+* **数组提取模式**: 在提取数组时，引擎支持不同的模式来处理各种 DOM 结构。
+
+  * **`nested`** (默认): `selector` 匹配每个项目的包裹元素。
+  * **`columnar`** (原 Zip 策略): `selector` 指向**容器**，`items` 中的字段是平行的列，按索引缝合在一起。
+  * **`segmented`**: `selector` 指向**容器**，项目通过“锚点”字段进行分段提取。
+
+###### 4. 列对齐模式 (Columnar Mode，原 Zip 策略)
+
+当 `selector` 指向一个**容器**（如结果列表）且项目数据以平行的独立列散落在其中时使用。
+
+```json
+{
+  "id": "extract",
+  "params": {
+    "type": "array",
+    "selector": "#search-results",
+    "mode": "columnar",
+    "items": {
+      "title": { "selector": ".item-title" },
+      "link": { "selector": "a.item-link", "attribute": "href" }
+    }
+  }
+}
+```
+
+> **启发式自动检测:** 如果省略了 `mode`，且 `selector` 恰好只匹配到一个元素，同时 `items` 中包含选择器，引擎会自动使用 **columnar** 模式。
+
+**Columnar 配置参数:**
+
+* **`strict`** (boolean, 默认: `true`): 如果为 `true`，当不同字段匹配到的数量不一致时将抛出错误。
+* **`inference`** (boolean, 默认: `false`): 如果为 `true`，当字段数量不匹配时，尝试通过 DOM 树自动寻找“包裹元素”来修复错位的列表。
+
+###### 5. 分段扫描模式 (Segmented Mode)
+
+适用于完全“平铺”且没有包裹元素的结构。它使用第一个字段（或指定的 `anchor`）作为锚点来对容器内容进行分段。
+
+```json
+{
+  "id": "extract",
+  "params": {
+    "type": "array",
+    "selector": "#flat-container",
+    "mode": { "type": "segmented", "anchor": "title" },
+    "items": {
+      "title": { "selector": "h3" },
+      "desc": { "selector": "p" }
+    }
+  }
+}
+```
+
+> 在此模式下，每当引擎发现一个 `h3`，就会开启一个新项目。随后找到的 `p` 标签（直到下一个 `h3` 出现前）都归属于该项目。
+
+###### 6. 隐式对象提取 (最简语法)
+
+为了让对象提取更简单，你可以省略 `type: 'object'` 和 `properties`。如果 schema 对象包含非保留关键字（如 `selector`, `attribute`, `type` 等）的键，它将被视为对象 schema，其中的键作为属性名。
+
+```json
+{
+  "id": "extract",
+  "params": {
+    "selector": ".author-bio",
+    "name": { "selector": ".author-name" },
+    "email": { "selector": "a.email", "attribute": "href" }
+  }
+}
+```
+
+> 这等同于示例 2，但更简洁。
+
+###### 6. 精确筛选: `has` 和 `exclude`
 
 您可以在任何包含 **`selector`** 的 Schema 中使用 **`has`** 和 **`exclude`** 字段来精确控制元素的选择。
 

package/README.action.md

@@ -224,19 +224,26 @@ The `params` object itself is a Schema that describes the data structure you wan
 
 ###### 1. Extracting a Single Value
 
-The most basic extraction. You can specify a `selector` (CSS selector), an `attribute` (the name of the attribute to extract), and a `type` (string, number, boolean, html).
+The most basic extraction. You can specify a `selector` (CSS selector), an `attribute` (the name of the attribute to extract), a `type` (string, number, boolean, html), and a `mode` (text, innerText).
 
 ```json
 {
   "id": "extract",
   "params": {
     "selector": "h1.main-title",
-    "type": "string"
+    "type": "string",
+    "mode": "innerText"
   }
 }
 ```
 
-> The example above will extract the text content of the `<h1>` tag with the class `main-title`.
+> **Extraction Modes:**
+>
+> * **`text`** (default): Extracts the `textContent` of the element.
+> * **`innerText`**: Extracts the rendered text, respecting CSS styling and line breaks.
+> * **`html`**: Returns the `innerHTML` of the element.
+> * **`outerHTML`**: Returns the HTML including the tag itself. Useful for preserving the full element structure.
+> The example above will extract the text content of the `<h1>` tag with the class `main-title` using the `innerText` mode.
 
 ###### 2. Extracting an Object
 
@@ -289,7 +296,77 @@ Extract a list using `type: 'array'`. To make the most common operations simpler
 
     > The example above will return an array of the `src` attributes from all `<img>` tags.
 
-###### 4. Precise Filtering: `has` and `exclude`
+* **Array Extraction Modes**: When extracting an array, the engine supports different modes to handle various DOM structures.
+
+  * **`nested`** (Default): The `selector` matches individual item wrappers.
+  * **`columnar`** (formerly Zip): The `selector` matches a **container**, and fields in `items` are parallel columns stitched together by index.
+  * **`segmented`**: The `selector` matches a **container**, and items are segmented by an "anchor" field.
+
+###### 4. Columnar Mode (formerly Zip Strategy)
+
+This mode is used when the `selector` points to a **container** (like a results list) and item data is scattered as separate columns.
+
+```json
+{
+  "id": "extract",
+  "params": {
+    "type": "array",
+    "selector": "#search-results",
+    "mode": "columnar",
+    "items": {
+      "title": { "selector": ".item-title" },
+      "link": { "selector": "a.item-link", "attribute": "href" }
+    }
+  }
+}
+```
+
+> **Heuristic Detection:** If `mode` is omitted and the `selector` matches exactly one element while `items` contains nested selectors, the engine automatically uses **columnar** mode.
+
+**Columnar Configuration:**
+
+* **`strict`** (boolean, default: `true`): If `true`, throws an error if fields have different match counts.
+* **`inference`** (boolean, default: `false`): If `true`, tries to automatically find the "item wrapper" elements to fix misaligned lists.
+
+###### 5. Segmented Mode (Anchor-based Scanning)
+
+This mode is ideal for "flat" structures where there are no item wrappers. It uses the first field (or a specified `anchor`) to segment the container's content.
+
+```json
+{
+  "id": "extract",
+  "params": {
+    "type": "array",
+    "selector": "#flat-container",
+    "mode": { "type": "segmented", "anchor": "title" },
+    "items": {
+      "title": { "selector": "h3" },
+      "desc": { "selector": "p" }
+    }
+  }
+}
+```
+
+> In this mode, every time the engine finds an `h3`, it starts a new item. Any `<p>` found after that `h3` (and before the next one) is assigned to that item.
+
+###### 6. Implicit Object Extraction (Simplest Syntax)
+
+For simpler object extraction, you can omit `type: 'object'` and `properties`. If the schema object contains keys that are not reserved keywords (like `selector`, `attribute`, `type`, etc.), it is treated as an object schema where keys are property names.
+
+```json
+{
+  "id": "extract",
+  "params": {
+    "selector": ".author-bio",
+    "name": { "selector": ".author-name" },
+    "email": { "selector": "a.email", "attribute": "href" }
+  }
+}
+```
+
+> This is equivalent to Example 2 but more concise.
+
+###### 6. Precise Filtering: `has` and `exclude`
 
 You can use the `has` and `exclude` fields in any schema that includes a `selector` to precisely control element selection.
 

package/README.engine.cn.md

@@ -31,18 +31,38 @@
 
 此静态工厂方法是创建引擎实例的指定入口点。它会自动选择并初始化合适的引擎。
 
-### `FetchSession(options, engine)`
+### `FetchSession(options)`
 
-`FetchSession` 类管理抓取操作的生命周期。它现在在构造函数中支持一个可选的 `engine` 参数，用于为该会话强制指定引擎实现，从而绕过任何自动检测或基于注册表的选择。
+`FetchSession` 类管理抓取操作的生命周期。您可以在 `options` 中指定 `engine` 来为该会话强制指定特定的引擎实现。
 
-### 引擎选择优先级
+```typescript
+const session = new FetchSession({ engine: 'browser' });
+```
+
+#### 引擎选择优先级 (Engine Selection Priority)
+
+引擎在执行第一个动作时延迟初始化，并在会话持续期间保持固定。选择遵循以下规则：
 
-当库决定使用哪个引擎时（通过内部的 `maybeCreateEngine`），它遵循以下优先级：
+1. **显式选项**: 如果在 `FetchSession` 的 `options.engine`（或 `executeAll` 的临时上下文覆盖）中提供了引擎且不为 `'auto'`。
+    * ⚠️ **快速失败 (Fail-Fast)**: 如果请求的引擎不可用（例如缺少依赖），将立即抛出错误。
+2. **站点注册表 (Site Registry)**: 如果设置为 `'auto'`（默认），系统会尝试根据目标 URL 匹配 `sites` 注册表。
+3. **智能升级 (Smart Upgrade)**: 如果启用，系统可能会根据响应特征（如机器人检测或大量 JS）动态从 `http` 升级到 `browser`。
+4. **默认**: 回退到 `'http'` (Cheerio)。
 
-1. **显式强制引擎**：如果在会话或引擎创建期间显式传递了引擎 ID（例如 `FetchSession` 中的新 `engine` 参数）。
-2. **配置引擎**：在 `FetcherOptions` 中定义的 `engine` 属性。
-3. **站点注册表**：如果目标 URL 匹配已配置 `sites` 注册表中的域名，则使用该站点首选的引擎。
-4. **默认值**：默认为 `auto`。如果启用了 `enableSmart`，它会智能地在 `http` 和 `browser`之间切换，否则默认为 `http`。
+#### 带有覆盖的批量执行 (Batch Execution with Overrides)
+
+您可以执行一系列动作，并指定临时的配置覆盖（例如 headers、timeout）。这些覆盖仅应用于当前批次，不会修改会话的全局状态。
+
+```typescript
+// 使用临时的自定义 header 和超时执行动作
+await session.executeAll([
+  { name: 'goto', params: { url: '...' } },
+  { name: 'extract', params: { ... } }
+], {
+  headers: { 'x-custom-priority': 'high' },
+  timeoutMs: 30000
+});
+```
 
 ### 会话管理与状态持久化
 

package/README.engine.md

@@ -31,18 +31,38 @@ This is the abstract base class that defines the contract for all fetch engines.
 
 This static factory method is the designated entry point for creating an engine instance. It automatically selects and initializes the appropriate engine.
 
-### `FetchSession(options, engine)`
+### `FetchSession(options)`
 
-The `FetchSession` class manages the lifecycle of a fetch operation. It now supports an optional `engine` parameter in its constructor to force a specific engine implementation for that session, bypassing any auto-detection or registry-based selection.
+The `FetchSession` class manages the lifecycle of a fetch operation. You can specify the `engine` in the `options` to force a specific engine implementation for that session.
 
-### Engine Selection Priority
+```typescript
+const session = new FetchSession({ engine: 'browser' });
+```
+
+#### Engine Selection Priority
+
+The engine is initialized lazily upon the first action execution and remains fixed for the duration of the session. The selection follows these rules:
 
-When the library determines which engine to use (via internal `maybeCreateEngine`), it follows this priority:
+1. **Explicit Option**: If `options.engine` (or temporary context override in `executeAll`) is provided and NOT set to `'auto'`.
+    * ⚠️ **Fail-Fast**: If the requested engine is unavailable (e.g., missing dependencies), an error is thrown immediately.
+2. **Site Registry**: If set to `'auto'` (default), the system attempts to match the target URL against the `sites` registry.
+3. **Smart Upgrade**: If enabled, the engine may be dynamically upgraded from `http` to `browser` based on response characteristics (e.g., bot detection or heavy JS).
+4. **Default**: Falls back to `'http'` (Cheerio).
 
-1. **Explicit Forced Engine**: If an engine ID is explicitly passed during session or engine creation (e.g., the new `engine` parameter in `FetchSession`).
-2. **Configuration Engine**: The `engine` property defined in `FetcherOptions`.
-3. **Site Registry**: If the target URL matches a domain in the configured `sites` registry, it uses the engine preferred for that site.
-4. **Default**: Defaults to `auto`, which intelligently switches between `http` and `browser` if `enableSmart` is enabled, or defaults to `http` otherwise.
+#### Batch Execution with Overrides
+
+You can execute a sequence of actions with temporary configuration overrides (e.g., headers, timeout) that apply only to that specific batch, without modifying the session's global state.
+
+```typescript
+// Execute actions with a temporary custom header and timeout
+await session.executeAll([
+  { name: 'goto', params: { url: '...' } },
+  { name: 'extract', params: { ... } }
+], {
+  headers: { 'x-custom-priority': 'high' },
+  timeoutMs: 30000
+});
+```
 
 ### Session Management & State Persistence