@things-factory/integration-headless 8.0.5 → 9.0.0-beta.12

package/helps/integration/connector/headless-connector.zh.md

@@ -1,57 +1,168 @@
-### 无头浏览器池连接器 (Headless Pool Connector)
+### 🏆 无头连接器 (Headless Connector)
 
-**无头浏览器池连接器**旨在管理无头浏览器实例的池。该连接器在执行需要多个无头浏览器会话同时运行的任务时非常有用，例如生成PDF或进行网页抓取。它能够有效地管理这些无头浏览器实例的创建、使用和销毁。
+**无头连接器** 利用无头浏览器执行 **自动登录**、**网页爬取**、**PDF 生成**、**表单提交**等任务。  
+它可以管理多个浏览器会话，支持会话保持、Cookie 处理、基于 API 的登录等功能。
 
-#### 端点
+---
 
-`无头浏览器池连接器`不要求特定的端点，但可以作为连接设置的一部分提供URI。此URI通常作为占位符使用，池在内部处理实际连接。也就是说，端点可以输入任意字符串。
+## 🚀 1. 端点（Endpoint）
 
-#### 参数
+无头连接器的端点需要填写 **目标网页的 URL**。  
+例如，可以输入 `https://example.com` 以抓取该网站的内容，或者使用 `localhost` 生成 PDF。
 
-- **min** (数字):
-  - 指定池中维持的最少无头浏览器实例数量。
-  - 默认值: `2`
-- **max** (数字):
-  - 指定池中可以处理的最大无头浏览器实例数量。
-  - 默认值: `10`
+---
 
-这些参数允许根据预期的工作负载调整池的大小。例如，如果应用程序经常同时需要多个浏览器实例，则可以增加`max`值。
+## 🔧 2. 参数（Parameters）
 
-#### 设置示例
+### ✅ 登录相关配置
+
+| 参数                 | 类型     | 描述                                         | 默认值        |
+| -------------------- | -------- | -------------------------------------------- | ------------- |
+| `username`           | `string` | 登录用户名（账户名）                         | `""`          |
+| `password`           | `string` | 登录密码                                     | `""`          |
+| `loginPagePath`      | `string` | 登录页面路径（如 `/login`）                  | `"/login"`    |
+| `loginApiUrl`        | `string` | 登录 API 端点（如果没有则 `null`）           | `null`        |
+| `usernameSelector`   | `string` | 登录表单的用户名输入框 CSS 选择器            | `"#username"` |
+| `passwordSelector`   | `string` | 登录表单的密码输入框 CSS 选择器              | `"#password"` |
+| `submitSelector`     | `string` | 登录按钮的 CSS 选择器                        | `"#submit"`   |
+| `successSelector`    | `string` | 用于判断登录成功的元素 CSS 选择器            | `null`        |
+| `shadowDomSelectors` | `string` | 处理 Shadow DOM 内部元素的选择器（逗号分隔） | `""`          |
+| `timeout`            | `number` | 登录及页面加载的超时时间（毫秒）             | `15000`       |
+| `retries`            | `number` | 登录重试次数                                 | `3`           |
+
+---
+
+## 🔍 3. `loginApiUrl` 参数的详细说明
+
+### 🔹 `loginApiUrl` 是什么？
+
+`loginApiUrl` 是指用于网站 **API 登录** 的接口地址。如果网站不使用 API 方式登录，则不需要设置此参数。
+
+### ✅ `loginApiUrl` 为空 (`null`)
+
+此时 **使用 HTML 表单提交的方式进行登录**。  
+无头浏览器会直接模拟用户操作，如填写用户名和密码，并点击提交按钮。
+
+#### 📌 示例（HTML 表单登录）
 
 ```json
 {
-  "name": "headless-browser-pool",
+  "endpoint": "https://example.com",
+  "params": {
+    "username": "user123",
+    "password": "securepassword",
+    "loginPagePath": "/login",
+    "usernameSelector": "#username",
+    "passwordSelector": "#password",
+    "submitSelector": "#submit-button",
+    "successSelector": ".dashboard"
+  }
+}
+```
+
+#### 🛠 处理过程：
+
+1. 访问 `https://example.com/login`
+2. 在 `#username` 和 `#password` 输入框中填入账户信息
+3. 点击 `#submit-button` 提交表单
+4. 通过 `.dashboard` 元素是否存在来判断是否登录成功
+
+---
+
+### ✅ `loginApiUrl` 设定时
+
+如果目标网站使用 **API 方式登录**，则 `loginApiUrl` 需要设置为该 API 的地址。  
+例如，如果 API 登录地址是 `/api/auth/login`，则需要手动设置该参数。
+
+#### 📌 示例（API 登录方式）
+
+```json
+{
+  "endpoint": "https://example.com",
+  "params": {
+    "username": "user123",
+    "password": "securepassword",
+    "loginPagePath": "/login",
+    "loginApiUrl": "/api/auth/login",
+    "usernameSelector": "#username",
+    "passwordSelector": "#password",
+    "submitSelector": "#submit-button",
+    "successSelector": ".dashboard"
+  }
+}
+```
+
+#### 🛠 处理过程：
+
+1. 发送 **HTTP POST 请求** 到 `https://example.com/api/auth/login`
+2. 服务器返回成功响应，浏览器存储会话 Cookie
+3. 访问主页面并检查 `.dashboard` 是否存在，以确认登录成功
+
+---
+
+## 🏗 4. 配置示例
+
+```json
+{
+  "name": "headless-browser-connector",
   "connector": "headless-connector",
-  "endpoint": "1",
+  "endpoint": "https://example.com",
   "params": {
-    "min": 5,
-    "max": 20
+    "username": "user123",
+    "password": "securepassword",
+    "loginPagePath": "/login",
+    "loginApiUrl": "/api/auth/login",
+    "usernameSelector": "#username",
+    "passwordSelector": "#password",
+    "submitSelector": "#submit",
+    "successSelector": ".dashboard",
+    "timeout": 20000,
+    "retries": 5
   }
 }
 ```
 
-在上述示例中，池至少保持5个无头浏览器实例，并且在需要时可以扩展到最多20个实例。
+---
+
+## 🔄 5. 连接生命周期（Connection Lifecycle）
+
+- **connect**
 
-#### 连接生命周期
+  - 根据参数执行登录流程，并保持会话。
+  - 存储 Cookie，以便后续请求使用。
 
-- **connect**:
+- **disconnect**
+  - 关闭浏览器会话，并清除 Cookie。
 
-  - 初始化连接，根据设定的参数(`min` 和 `max`)创建无头浏览器实例池。
-  - 池会自动管理这些实例的生命周期，必要时创建新的实例并重用现有的实例。
+---
 
-- **disconnect**:
-  - 销毁无头浏览器池，确保所有实例都被适当关闭并释放资源。
+## ⚡ 6. 支持的任务（Supported Tasks）
 
-#### 使用场景
+| 任务类型               | 描述                   |
+| ---------------------- | ---------------------- |
+| `headless-pdf-capture` | 将 HTML 转换为 PDF     |
+| `headless-scrap`       | 网页爬取，提取特定数据 |
+| `headless-post`        | 自动填写表单并提交     |
 
-`无头浏览器池连接器`主要用于需要在无头浏览器环境中执行任务的场景。例如，生成HTML内容的PDF、捕获网页截图或从网站抓取数据。
+#### 📌 任务示例
 
-#### 支持的任务
+### 1️⃣ 网页爬取（`headless-scrap`）
 
-`无头浏览器池连接器`支持以 `headless-pdf` 为前缀的任务。这些任务将使用由此连接器管理的无头浏览器实例。
+```json
+{
+  "name": "scrape-data",
+  "taskType": "headless-scrap",
+  "params": {
+    "path": "/products",
+    "selectors": [
+      { "text": "商品名称", "value": ".product-title" },
+      { "text": "价格", "value": ".product-price" }
+    ]
+  }
+}
+```
 
-#### 任务示例使用
+### 2️⃣ PDF 生成（`headless-pdf-capture`）
 
 ```json
 {
@@ -66,6 +177,42 @@
 }
 ```
 
-#### 说明
+### 3️⃣ 表单提交（`headless-post`）
+
+```json
+{
+  "name": "submit-form",
+  "taskType": "headless-post",
+  "params": {
+    "formPath": "/contact",
+    "fields": {
+      "#name": "张三",
+      "#email": "zhangsan@example.com",
+      "#message": "你好，我想咨询一下！"
+    },
+    "submitSelector": "#submit-button"
+  }
+}
+```
+
+---
+
+## 📢 7. 说明（Description）
+
+**无头连接器** 可用于 **自动化网页爬取、PDF 生成、表单提交** 等任务。  
+它支持 **HTML 登录** 和 **API 登录**，适用于各种不同的网页登录方式。
+
+---
+
+### 📌 功能总结
+
+| 功能              | 说明                                 |
+| ----------------- | ------------------------------------ |
+| **HTML 表单登录** | 直接模拟用户操作（适用于大部分网页） |
+| **API 登录**      | 通过 API 发送登录请求（更快）        |
+| **会话保持**      | 存储 Cookie 并维持登录状态           |
+| **网页爬取**      | 提取页面中的特定数据                 |
+| **PDF 生成**      | 将 HTML 页面转换为 PDF               |
+| **表单提交**      | 自动填充并提交表单                   |
 
-**无头浏览器池连接器**通过高效管理无头浏览器实例池，确保在高需求场景下执行PDF生成和网页抓取等任务时，资源利用的优化和可扩展性。
+📌 **通过无头连接器，你可以轻松地在不同网站上实现自动化操作！** 🚀

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@things-factory/integration-headless",
-  "version": "8.0.5",
+  "version": "9.0.0-beta.12",
   "main": "dist-server/index.js",
   "things-factory": true,
   "author": "heartyoh <heartyoh@hatiolab.com>",
@@ -22,12 +22,12 @@
     "clean": "npm run clean:server"
   },
   "dependencies": {
-    "@things-factory/attachment-base": "^8.0.5",
-    "@things-factory/board-service": "^8.0.5",
-    "@things-factory/integration-base": "^8.0.5",
-    "@things-factory/shell": "^8.0.2",
+    "@things-factory/attachment-base": "^9.0.0-beta.12",
+    "@things-factory/board-service": "^9.0.0-beta.12",
+    "@things-factory/integration-base": "^9.0.0-beta.12",
+    "@things-factory/shell": "^9.0.0-beta.12",
     "ejs": "^3.1.10",
     "pdf-lib": "^1.17.1"
   },
-  "gitHead": "9ab6fca18eeb58b9d2f3411661508a39d0ab6aee"
+  "gitHead": "5e9ade1c2d4b4c96b89396e36c3afa1caaf18ef0"
 }

package/dist-server/engine/connector/headless-connector.d.ts

@@ -1,14 +0,0 @@
-import { Connector } from '@things-factory/integration-base';
-export declare class HeadlessConnector implements Connector {
-    ready(connectionConfigs: any): Promise<void>;
-    connect(connection: any): Promise<void>;
-    disconnect(connection: any): Promise<void>;
-    get parameterSpec(): {
-        type: string;
-        name: string;
-        label: string;
-    }[];
-    get taskPrefixes(): string[];
-    get description(): string;
-    get help(): string;
-}

package/dist-server/engine/connector/headless-connector.js

@@ -1,54 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.HeadlessConnector = void 0;
-const integration_base_1 = require("@things-factory/integration-base");
-const headless_pool_1 = require("./headless-pool");
-class HeadlessConnector {
-    async ready(connectionConfigs) {
-        await Promise.all(connectionConfigs.map(this.connect.bind(this)));
-        integration_base_1.ConnectionManager.logger.info('headless-connector connections are ready');
-    }
-    async connect(connection) {
-        var { endpoint: uri = '1', params: { min = 2, max = 10 } } = connection;
-        const headlessPool = (0, headless_pool_1.createHeadlessPool)({
-            min,
-            max,
-            testOnBorrow: true,
-            acquireTimeoutMillis: 15000
-        });
-        integration_base_1.ConnectionManager.addConnectionInstance(connection, headlessPool);
-        integration_base_1.ConnectionManager.logger.info(`headless-connector connection(${connection.name}:${connection.endpoint}) is connected`);
-    }
-    async disconnect(connection) {
-        const headlessPool = integration_base_1.ConnectionManager.getConnectionInstance(connection);
-        (0, headless_pool_1.destroyHeadlessPool)(headlessPool);
-        integration_base_1.ConnectionManager.removeConnectionInstance(connection);
-        integration_base_1.ConnectionManager.logger.info(`headless-connector connection(${connection.name}) is disconnected`);
-    }
-    get parameterSpec() {
-        return [
-            {
-                type: 'number',
-                name: 'min',
-                label: 'pool-size-min'
-            },
-            {
-                type: 'number',
-                name: 'max',
-                label: 'pool-size-max'
-            }
-        ];
-    }
-    get taskPrefixes() {
-        return ['headless-pdf'];
-    }
-    get description() {
-        return 'Headless Pool Connector';
-    }
-    get help() {
-        return 'integration/connector/headless-connector';
-    }
-}
-exports.HeadlessConnector = HeadlessConnector;
-integration_base_1.ConnectionManager.registerConnector('headless-connector', new HeadlessConnector());
-//# sourceMappingURL=headless-connector.js.map

package/dist-server/engine/connector/headless-connector.js.map

@@ -1 +0,0 @@
-{"version":3,"file":"headless-connector.js","sourceRoot":"","sources":["../../../server/engine/connector/headless-connector.ts"],"names":[],"mappings":";;;AAAA,uEAA+E;AAC/E,mDAAyE;AAEzE,MAAa,iBAAiB;IAC5B,KAAK,CAAC,KAAK,CAAC,iBAAiB;QAC3B,MAAM,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAA;QAEjE,oCAAiB,CAAC,MAAM,CAAC,IAAI,CAAC,0CAA0C,CAAC,CAAA;IAC3E,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,UAAU;QACtB,IAAI,EACF,QAAQ,EAAE,GAAG,GAAG,GAAG,EACnB,MAAM,EAAE,EAAE,GAAG,GAAG,CAAC,EAAE,GAAG,GAAG,EAAE,EAAE,EAC9B,GAAG,UAAU,CAAA;QAEd,MAAM,YAAY,GAAG,IAAA,kCAAkB,EAAC;YACtC,GAAG;YACH,GAAG;YACH,YAAY,EAAE,IAAI;YAClB,oBAAoB,EAAE,KAAK;SAC5B,CAAC,CAAA;QAEF,oCAAiB,CAAC,qBAAqB,CAAC,UAAU,EAAE,YAAY,CAAC,CAAA;QAEjE,oCAAiB,CAAC,MAAM,CAAC,IAAI,CAC3B,iCAAiC,UAAU,CAAC,IAAI,IAAI,UAAU,CAAC,QAAQ,gBAAgB,CACxF,CAAA;IACH,CAAC;IAED,KAAK,CAAC,UAAU,CAAC,UAAU;QACzB,MAAM,YAAY,GAAG,oCAAiB,CAAC,qBAAqB,CAAC,UAAU,CAAC,CAAA;QACxE,IAAA,mCAAmB,EAAC,YAAY,CAAC,CAAA;QAEjC,oCAAiB,CAAC,wBAAwB,CAAC,UAAU,CAAC,CAAA;QAEtD,oCAAiB,CAAC,MAAM,CAAC,IAAI,CAAC,iCAAiC,UAAU,CAAC,IAAI,mBAAmB,CAAC,CAAA;IACpG,CAAC;IAED,IAAI,aAAa;QACf,OAAO;YACL;gBACE,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,KAAK;gBACX,KAAK,EAAE,eAAe;aACvB;YACD;gBACE,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,KAAK;gBACX,KAAK,EAAE,eAAe;aACvB;SACF,CAAA;IACH,CAAC;IAED,IAAI,YAAY;QACd,OAAO,CAAC,cAAc,CAAC,CAAA;IACzB,CAAC;IAED,IAAI,WAAW;QACb,OAAO,yBAAyB,CAAA;IAClC,CAAC;IAED,IAAI,IAAI;QACN,OAAO,0CAA0C,CAAA;IACnD,CAAC;CACF;AA9DD,8CA8DC;AAED,oCAAiB,CAAC,iBAAiB,CAAC,oBAAoB,EAAE,IAAI,iBAAiB,EAAE,CAAC,CAAA","sourcesContent":["import { ConnectionManager, Connector } from '@things-factory/integration-base'\nimport { createHeadlessPool, destroyHeadlessPool } from './headless-pool'\n\nexport class HeadlessConnector implements Connector {\n  async ready(connectionConfigs) {\n    await Promise.all(connectionConfigs.map(this.connect.bind(this)))\n\n    ConnectionManager.logger.info('headless-connector connections are ready')\n  }\n\n  async connect(connection) {\n    var {\n      endpoint: uri = '1',\n      params: { min = 2, max = 10 }\n    } = connection\n\n    const headlessPool = createHeadlessPool({\n      min,\n      max,\n      testOnBorrow: true,\n      acquireTimeoutMillis: 15000\n    })\n\n    ConnectionManager.addConnectionInstance(connection, headlessPool)\n\n    ConnectionManager.logger.info(\n      `headless-connector connection(${connection.name}:${connection.endpoint}) is connected`\n    )\n  }\n\n  async disconnect(connection) {\n    const headlessPool = ConnectionManager.getConnectionInstance(connection)\n    destroyHeadlessPool(headlessPool)\n\n    ConnectionManager.removeConnectionInstance(connection)\n\n    ConnectionManager.logger.info(`headless-connector connection(${connection.name}) is disconnected`)\n  }\n\n  get parameterSpec() {\n    return [\n      {\n        type: 'number',\n        name: 'min',\n        label: 'pool-size-min'\n      },\n      {\n        type: 'number',\n        name: 'max',\n        label: 'pool-size-max'\n      }\n    ]\n  }\n\n  get taskPrefixes() {\n    return ['headless-pdf']\n  }\n\n  get description() {\n    return 'Headless Pool Connector'\n  }\n\n  get help() {\n    return 'integration/connector/headless-connector'\n  }\n}\n\nConnectionManager.registerConnector('headless-connector', new HeadlessConnector())\n"]}

package/dist-server/engine/connector/headless-pool.d.ts

@@ -1,3 +0,0 @@
-import * as genericPool from 'generic-pool';
-export declare function createHeadlessPool(options: genericPool.Options): genericPool.Pool<any>;
-export declare function destroyHeadlessPool(headlessPool: genericPool.Pool<any>): Promise<void>;

package/dist-server/engine/connector/headless-pool.js

@@ -1,63 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.createHeadlessPool = createHeadlessPool;
-exports.destroyHeadlessPool = destroyHeadlessPool;
-const tslib_1 = require("tslib");
-const genericPool = tslib_1.__importStar(require("generic-pool"));
-const env_1 = require("@things-factory/env");
-try {
-    var puppeteer = require('puppeteer');
-}
-catch (err) {
-    env_1.logger.error(err);
-}
-function createHeadlessPool(options) {
-    return genericPool.createPool({
-        create() {
-            console.log('headless instance in headless-pool-integration about to create');
-            return initializeChromium();
-        },
-        validate(browser) {
-            return Promise.race([
-                new Promise(res => setTimeout(() => res(false), 1500)),
-                browser
-                    //@ts-ignore
-                    .version()
-                    .then(_ => true)
-                    .catch(_ => false)
-            ]);
-        },
-        destroy(browser) {
-            //@ts-ignore
-            return browser.close();
-        }
-    }, options);
-}
-async function destroyHeadlessPool(headlessPool) {
-    if (headlessPool) {
-        console.log('headless-pool-integration about to destroy');
-        await headlessPool.drain();
-        await headlessPool.clear();
-    }
-}
-const CHROMIUM_PATH = env_1.config.get('CHROMIUM_PATH');
-async function initializeChromium() {
-    try {
-        if (!puppeteer) {
-            return;
-        }
-        var launchSetting = {
-            args: ['--hide-scrollbars', '--mute-audio', '--no-sandbox', '--use-gl=egl'],
-            headless: 'shell'
-        };
-        if (CHROMIUM_PATH) {
-            launchSetting['executablePath'] = CHROMIUM_PATH;
-        }
-        const browser = await puppeteer.launch(launchSetting);
-        return browser;
-    }
-    catch (err) {
-        env_1.logger.error(err);
-    }
-}
-//# sourceMappingURL=headless-pool.js.map

package/dist-server/engine/connector/headless-pool.js.map

@@ -1 +0,0 @@
-{"version":3,"file":"headless-pool.js","sourceRoot":"","sources":["../../../server/engine/connector/headless-pool.ts"],"names":[],"mappings":";;AAUA,gDAwBC;AAED,kDAOC;;AA3CD,kEAA2C;AAE3C,6CAAoD;AAEpD,IAAI,CAAC;IACH,IAAI,SAAS,GAAG,OAAO,CAAC,WAAW,CAAC,CAAA;AACtC,CAAC;AAAC,OAAO,GAAG,EAAE,CAAC;IACb,YAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAA;AACnB,CAAC;AAED,SAAgB,kBAAkB,CAAC,OAA4B;IAC7D,OAAO,WAAW,CAAC,UAAU,CAC3B;QACE,MAAM;YACJ,OAAO,CAAC,GAAG,CAAC,gEAAgE,CAAC,CAAA;YAC7E,OAAO,kBAAkB,EAAE,CAAA;QAC7B,CAAC;QACD,QAAQ,CAAC,OAAO;YACd,OAAO,OAAO,CAAC,IAAI,CAAC;gBAClB,IAAI,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,UAAU,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,IAAI,CAAC,CAAC;gBACtD,OAAO;oBACL,YAAY;qBACX,OAAO,EAAE;qBACT,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC;qBACf,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,KAAK,CAAC;aACrB,CAAC,CAAA;QACJ,CAAC;QACD,OAAO,CAAC,OAAO;YACb,YAAY;YACZ,OAAO,OAAO,CAAC,KAAK,EAAE,CAAA;QACxB,CAAC;KAC0B,EAC7B,OAAO,CACR,CAAA;AACH,CAAC;AAEM,KAAK,UAAU,mBAAmB,CAAC,YAAmC;IAC3E,IAAI,YAAY,EAAE,CAAC;QACjB,OAAO,CAAC,GAAG,CAAC,4CAA4C,CAAC,CAAA;QAEzD,MAAM,YAAY,CAAC,KAAK,EAAE,CAAA;QAC1B,MAAM,YAAY,CAAC,KAAK,EAAE,CAAA;IAC5B,CAAC;AACH,CAAC;AAED,MAAM,aAAa,GAAG,YAAM,CAAC,GAAG,CAAC,eAAe,CAAC,CAAA;AAEjD,KAAK,UAAU,kBAAkB;IAC/B,IAAI,CAAC;QACH,IAAI,CAAC,SAAS,EAAE,CAAC;YACf,OAAM;QACR,CAAC;QAED,IAAI,aAAa,GAAG;YAClB,IAAI,EAAE,CAAC,mBAAmB,EAAE,cAAc,EAAE,cAAc,EAAE,cAAc,CAAC;YAC3E,QAAQ,EAAE,OAAO;SAClB,CAAA;QAED,IAAI,aAAa,EAAE,CAAC;YAClB,aAAa,CAAC,gBAAgB,CAAC,GAAG,aAAa,CAAA;QACjD,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,SAAS,CAAC,MAAM,CAAC,aAAa,CAAC,CAAA;QAErD,OAAO,OAAO,CAAA;IAChB,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,YAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAA;IACnB,CAAC;AACH,CAAC","sourcesContent":["import * as genericPool from 'generic-pool'\n\nimport { config, logger } from '@things-factory/env'\n\ntry {\n  var puppeteer = require('puppeteer')\n} catch (err) {\n  logger.error(err)\n}\n\nexport function createHeadlessPool(options: genericPool.Options) {\n  return genericPool.createPool(\n    {\n      create() {\n        console.log('headless instance in headless-pool-integration about to create')\n        return initializeChromium()\n      },\n      validate(browser) {\n        return Promise.race([\n          new Promise(res => setTimeout(() => res(false), 1500)),\n          browser\n            //@ts-ignore\n            .version()\n            .then(_ => true)\n            .catch(_ => false)\n        ])\n      },\n      destroy(browser) {\n        //@ts-ignore\n        return browser.close()\n      }\n    } as genericPool.Factory<any>,\n    options\n  )\n}\n\nexport async function destroyHeadlessPool(headlessPool: genericPool.Pool<any>) {\n  if (headlessPool) {\n    console.log('headless-pool-integration about to destroy')\n\n    await headlessPool.drain()\n    await headlessPool.clear()\n  }\n}\n\nconst CHROMIUM_PATH = config.get('CHROMIUM_PATH')\n\nasync function initializeChromium() {\n  try {\n    if (!puppeteer) {\n      return\n    }\n\n    var launchSetting = {\n      args: ['--hide-scrollbars', '--mute-audio', '--no-sandbox', '--use-gl=egl'],\n      headless: 'shell'\n    }\n\n    if (CHROMIUM_PATH) {\n      launchSetting['executablePath'] = CHROMIUM_PATH\n    }\n\n    const browser = await puppeteer.launch(launchSetting)\n\n    return browser\n  } catch (err) {\n    logger.error(err)\n  }\n}\n"]}

package/dist-server/engine/connector/index.d.ts

@@ -1 +0,0 @@
-import './headless-connector';

package/dist-server/engine/connector/index.js

@@ -1,4 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-require("./headless-connector");
-//# sourceMappingURL=index.js.map

package/dist-server/engine/connector/index.js.map

@@ -1 +0,0 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../server/engine/connector/index.ts"],"names":[],"mappings":";;AAAA,gCAA6B","sourcesContent":["import './headless-connector'\n"]}

package/server/engine/connector/headless-connector.ts

@@ -1,68 +0,0 @@
-import { ConnectionManager, Connector } from '@things-factory/integration-base'
-import { createHeadlessPool, destroyHeadlessPool } from './headless-pool'
-
-export class HeadlessConnector implements Connector {
-  async ready(connectionConfigs) {
-    await Promise.all(connectionConfigs.map(this.connect.bind(this)))
-
-    ConnectionManager.logger.info('headless-connector connections are ready')
-  }
-
-  async connect(connection) {
-    var {
-      endpoint: uri = '1',
-      params: { min = 2, max = 10 }
-    } = connection
-
-    const headlessPool = createHeadlessPool({
-      min,
-      max,
-      testOnBorrow: true,
-      acquireTimeoutMillis: 15000
-    })
-
-    ConnectionManager.addConnectionInstance(connection, headlessPool)
-
-    ConnectionManager.logger.info(
-      `headless-connector connection(${connection.name}:${connection.endpoint}) is connected`
-    )
-  }
-
-  async disconnect(connection) {
-    const headlessPool = ConnectionManager.getConnectionInstance(connection)
-    destroyHeadlessPool(headlessPool)
-
-    ConnectionManager.removeConnectionInstance(connection)
-
-    ConnectionManager.logger.info(`headless-connector connection(${connection.name}) is disconnected`)
-  }
-
-  get parameterSpec() {
-    return [
-      {
-        type: 'number',
-        name: 'min',
-        label: 'pool-size-min'
-      },
-      {
-        type: 'number',
-        name: 'max',
-        label: 'pool-size-max'
-      }
-    ]
-  }
-
-  get taskPrefixes() {
-    return ['headless-pdf']
-  }
-
-  get description() {
-    return 'Headless Pool Connector'
-  }
-
-  get help() {
-    return 'integration/connector/headless-connector'
-  }
-}
-
-ConnectionManager.registerConnector('headless-connector', new HeadlessConnector())

package/server/engine/connector/headless-pool.ts

@@ -1,69 +0,0 @@
-import * as genericPool from 'generic-pool'
-
-import { config, logger } from '@things-factory/env'
-
-try {
-  var puppeteer = require('puppeteer')
-} catch (err) {
-  logger.error(err)
-}
-
-export function createHeadlessPool(options: genericPool.Options) {
-  return genericPool.createPool(
-    {
-      create() {
-        console.log('headless instance in headless-pool-integration about to create')
-        return initializeChromium()
-      },
-      validate(browser) {
-        return Promise.race([
-          new Promise(res => setTimeout(() => res(false), 1500)),
-          browser
-            //@ts-ignore
-            .version()
-            .then(_ => true)
-            .catch(_ => false)
-        ])
-      },
-      destroy(browser) {
-        //@ts-ignore
-        return browser.close()
-      }
-    } as genericPool.Factory<any>,
-    options
-  )
-}
-
-export async function destroyHeadlessPool(headlessPool: genericPool.Pool<any>) {
-  if (headlessPool) {
-    console.log('headless-pool-integration about to destroy')
-
-    await headlessPool.drain()
-    await headlessPool.clear()
-  }
-}
-
-const CHROMIUM_PATH = config.get('CHROMIUM_PATH')
-
-async function initializeChromium() {
-  try {
-    if (!puppeteer) {
-      return
-    }
-
-    var launchSetting = {
-      args: ['--hide-scrollbars', '--mute-audio', '--no-sandbox', '--use-gl=egl'],
-      headless: 'shell'
-    }
-
-    if (CHROMIUM_PATH) {
-      launchSetting['executablePath'] = CHROMIUM_PATH
-    }
-
-    const browser = await puppeteer.launch(launchSetting)
-
-    return browser
-  } catch (err) {
-    logger.error(err)
-  }
-}

package/server/engine/connector/index.ts

@@ -1 +0,0 @@
-import './headless-connector'

package/server/engine/index.ts

@@ -1,2 +0,0 @@
-import './connector'
-import './task'