portadom 1.0.4 → 2.0.0

package/README.md

@@ -1,264 +1,134 @@
-# Portadom
+# portadom
 
-*Single DOM manipulation interface across Browser API, JSDOM, Cheerio, Playwright.*
+**One DOM interface. Any engine.**
 
-If you write web scrapers, you will know that you have multiple ways of parsing and manipulating the HTML / DOM:
-- Download the HTML and feed into JSDOM or Cheerio.
-- Through browser automation like Playwright, Puppeteer, or Selenium.
-- Or right from inside the DevTools console, if you need to test something out.
+Write scraping and extraction logic once. Run it on Cheerio, Playwright, or the Browser API. Swap engines when requirements change -- no rewrite needed.
 
-When I'm writing scrapers, my approach is usually:
-1. Define the transformations in DevTools with vanilla JS.
-2. Check if the HTML data can be extracted statically, just from the HTML (no JS).
-3. If static HTML is enough, then migrate vanilla JS to JSDOM or Cheerio.
-4. If I need JS runtime, migrate the vanilla JS to Playwright or other browser automation tool.
-
-Migrating from one to another can be prone to errors, and you may miss some features.
-
-Portadom takes care of this. Here's how you can move the same DOM manipulation logic from Cheerio to Playwright:
-
-Before:
-
-```js
-import { load as loadCheerio } from 'cheerio';
-import { cheerioPortadom } from 'portadom';
-
-// Loading step changes
-const html = `<div>
-  <a href="#">Click Me!</a>
-</div>`;
-const $ = loadCheerio(html);
-const dom = cheerioPortadom($.root(), url);
-
-// DOM manipulation remains the same
-const btn = dom.findOne('a');
-const btnText = await btn.text();
-// btnText == "Click Me!"
+```bash
+npm install portadom
 ```
 
-After:
-
-```js
-import { playwrightLocatorPortadom } from 'portadom';
+## Quick start
 
-// Loading step changes
-const page = await somehowLoadPage();
-const bodyLoc = page.locator('body');
-const dom = playwrightLocatorPortadom(bodyLoc, page);
+Wrap any DOM engine in a `Portadom` instance, then use the same API everywhere:
 
-// DOM manipulation remains the same
-const btn = dom.findOne('a');
-const btnText = await btn.text();
-// btnText == "Click Me!"
-```
+```typescript
+import * as cheerio from 'cheerio';
+import { cheerioPortadom } from 'portadom';
 
-## Installation
+// 1. Create a Portadom instance from a Cheerio root
+const $ = cheerio.load('<h1>Hello</h1><a href="/about">About</a>');
+const dom = cheerioPortadom($.root(), 'https://example.com');
 
-```sh
-npm install portadom
+// 2. Use the engine-agnostic API
+await dom.findOne('h1').text(); // "Hello"
+await dom.findOne('a').href(); // "https://example.com/about"
+await dom.findMany('li').map((el) => el.text()); // ["Item 1", "Item 2", ...]
 ```
 
-## How to use
-
-### Minimal example
+Switch engines by changing only the setup -- the extraction logic stays identical:
 
-```js
-const html = `<div>
-  <a href="#">Click Me!</a>
-</div>`;
-const $ = loadCheerio(html);
-const dom = cheerioPortadom($.root(), url);
-
-const btn = dom.findOne('a');
-const btnText = await btn.text();
-// btnText == "Click Me!"
+```typescript
+import { playwrightLocatorPortadom } from 'portadom';
 
-const btnProp = await btn.href();
-// btnProp == "https://example.com#"
-```
+// 1. Create a Portadom instance from a Playwright locator
+const dom = playwrightLocatorPortadom(page.locator('body'), page);
 
-### Full example
-
-```js
-const $ = loadCheerio(html);
-const dom = cheerioPortadom($.root(), url);
-// ...
-const rootEl = dom.root();
-const url = await dom.url();
-
-// Find and extract data
-const entries = await rootEl.findMany('.list-row:not(.native-agent):not(.reach-list)')
-  .mapAsyncSerial(async (el) => {
-  const employerName = await el.findOne('.employer').text();
-  const employerUrl = await el.findOne('.offer-company-logo-link').href();
-  const employerLogoUrl = await el.findOne('.offer-company-logo-link img').src();
-
-  const offerUrlEl = el.findOne('h2 a');
-  const offerUrl = await offerUrlEl.href();
-  const offerName = await offerUrlEl.text();
-  const offerId = offerUrl?.match(/O\d{2,}/)?.[0] ?? null;
-
-  const location = await el.findOne('.job-location').text();
-
-  const salaryText = await el.findOne('.label-group > a[data-dimension7="Salary label"]').text();
-
-  const labels = await el.findMany('.label-group > a:not([data-dimension7="Salary label"])')
-    .mapAsyncSerial((el) => el.text())
-    .then((arr) => arr.filter(Boolean) as string[]);
-
-  const footerInfoEl = el.findOne('.list-footer .info');
-  const lastChangeRelativeTimeEl = footerInfoEl.findOne('strong');
-  const lastChangeRelativeTime = await lastChangeRelativeTimeEl.text();
-  // Remove the element so it's easier to get the text content
-  await lastChangeRelativeTimeEl.remove();
-  const lastChangeTypeText = await footerInfoEl.textAsLower();
-  const lastChangeType = lastChangeTypeText === 'pridané' ? 'added' : 'modified';
-
-  return {
-    listingUrl: url,
-    employerName,
-    employerUrl,
-    employerLogoUrl,
-    offerName,
-    offerUrl,
-    offerId,
-    location,
-    labels,
-    lastChangeRelativeTime,
-    lastChangeType,
-  };
-});
+// 2. Exact same API -- no changes needed
+await dom.findOne('h1').text();
+await dom.findOne('a').href();
 ```
 
-### Loading
+## Why portadom?
 
-Here is how you can load DOM in different environments:
+### Write once. Run anywhere.
 
-#### Browser
+Cheerio, Playwright, and the Browser API have incompatible DOM APIs. Portadom wraps them behind a single `Portadom` interface. Your selectors, traversals, and data extraction stay the same regardless of engine.
 
-When working with browser [Document](https://developer.mozilla.org/en-US/docs/Web/API/Document), the `node` is an [Element](https://developer.mozilla.org/en-US/docs/Web/API/Element).
+### Chain without awaits.
 
-```js
-import { browserPortadom } from 'portadom';
+Traversal methods return `PortadomPromise` -- chain `findOne`, `closest`, `parent`, and more without intermediate `await`s. The promise resolves when you extract a value.
 
-const dom = browserPortadom(document.body);
-const btnNode = await dom.findOne('a').node;
+```typescript
+// Without portadom
+const root = await page.locator('#root').elementHandle();
+const heading = root ? await root.$('h1') : null;
+const text = heading ? await heading.textContent() : null;
 
-// Or
-const startNode = document.querySelector('...');
-const dom = browserPortadom(startNode);
-const btnNode = await dom.findOne('a').node;
+// With portadom
+const text = await dom.findOne('#root').findOne('h1').text();
 ```
 
-#### Cheerio
+### Collections that feel like arrays.
 
-In [Cheerio](https://cheerio.js.org/), the `node` is the Cheerio Element wrapper. [See DOM traversal with Cheerio](https://cheerio.js.org/docs/basics/traversing).
+`findMany()` returns `PortadomArrayPromise` -- the full `Array` API (`map`, `filter`, `find`, `slice`, `reduce`, ...) plus async helpers like `mapAsyncSerial` and `filterAsyncParallel`.
 
-```js
-import { cheerioPortadom } from 'portadom';
-import { load as loadCheerio } from 'cheerio';
-
-const $ = loadCheerio(html);
-const dom = cheerioPortadom($.root(), url);
-const btnNode = await dom.findOne('a').node;
-
-// Or
-const startNode = $('a');
-const dom = cheerioPortadom(startNode, url);
-const btnNode = await dom.findOne('a').node;
-
-// Set `null` if you don't have an URL for the HTML
-const dom = cheerioPortadom($.root(), null);
+```typescript
+const prices = await dom
+  .findMany('.product')
+  .filterAsyncParallel(async (el) => (await el.attr('in-stock')) === 'true')
+  .mapAsyncSerial(async (el) => await el.findOne('.price').textAsNumber({ mode: 'float' }));
 ```
 
-#### Playwright (using Locators)
+### URLs resolved automatically.
 
-In [Playwright](https://cheerio.js.org/), you can either work with the [Locators](https://playwright.dev/docs/api/class-locator) or the [ElementHandles](https://playwright.dev/docs/api/class-elementhandle).
+`href()` and `src()` resolve relative paths to absolute URLs using the page's base URL. No manual `new URL()` calls.
 
-When using Locators, the `node` is a Locator instance.
+<details>
+<summary>What portadom replaces (click to expand)</summary>
 
-```js
-import { playwrightLocatorPortadom } from 'portadom';
+**Cheerio -- extracting an href:**
 
-const page = await somehowLoadPage();
-const bodyLoc = page.locator('body');
-const dom = playwrightLocatorPortadom(bodyLoc, page);
-const btnNode = await dom.findOne('a').node;
+```typescript
+const rawHref = $('a').attr('href'); // "/about"
+const absHref = rawHref?.startsWith('/') ? new URL(rawHref, 'https://example.com').href : rawHref;
 ```
 
-#### Playwright (using Handles)
-
-When using ElementHandles, the `node` is an ElementHandle instance.
-
-NOTE: You can pass Locator to `playwrightHandlePortadom`, but this will be converted to JSHandle internally.
-
-```js
-import { playwrightHandlePortadom } from 'portadom';
-
-const page = await somehowLoadPage();
+**Portadom:**
 
-// Use `evaluateHandle` with page-side logic to query the target element
-const handle = await page.evaluateHandle(, () => document.body);
-const handle = await page.evaluateHandle(, () => document.querySelector('.myClass'));
-
-// Or use other helpers such as `getByText`
-const handle = await page.getByText('hello');
-
-// Or use locators
-const handle = page.locator('body');
-
-const dom = playwrightHandlePortadom(bodyLoc, page);
-const btnNode = await dom.findOne('a').node;
+```typescript
+await dom.findOne('a').href(); // "https://example.com/about"
 ```
 
-### Chaining
-
-For cross-compatibility, each method on a Portadom instance returns
-a Promise.
+</details>
 
-But this then leads to `then` / `await` hell when you need to call multiple methods in a row:
+## Supported engines
 
-```js
-const employerName = (await (await el.findOne('.employer'))?.text()) ?? null;
-```
-
-To get around that, the results are wrapped in chainable instance. This applies to each method that returns a Portadom instance, or an array of Portadom instances.
+| Factory function                           | Engine                | Element type               |
+| ------------------------------------------ | --------------------- | -------------------------- |
+| `cheerioPortadom(el, url)`                 | Cheerio               | `Cheerio<AnyNode>`         |
+| `playwrightLocatorPortadom(locator, page)` | Playwright (locators) | `Locator`                  |
+| `playwrightHandlePortadom(handle, page)`   | Playwright (handles)  | `Locator \| ElementHandle` |
+| `browserPortadom(element)`                 | Browser API           | `Element`                  |
 
-So instead, we can call:
+Install only the engine(s) you need:
 
-```js
-const employerName = await el.findOne('.employer').text();
+```bash
+npm install cheerio       # static HTML
+npm install playwright    # browser automation
 ```
 
-You don't have to chain the commands. Instead, you can access the associated promise under `promise` property. For example this:
+## Page-level operations (experimental)
 
-```js
-const mapPromises = await dom.findOne('ul')
-  .parent()
-  .findMany('li[data-id]')
-  .map((li) => li.attr('data-id'));
-const attrs = await Promise.all(mapResult);
-```
+`Portapage` provides page-level helpers like infinite scroll handling:
 
-Is the same as:
+```typescript
+import { playwrightPortapage } from 'portadom';
 
-```js
-const ul = await dom.findOne('ul').promise;
-const parent = await ul?.parent().promise;
-const idEls = await parent?.findMany('li[data-id]').promise;
-const mapPromises = idEls?.map((li) => li.attr('data-id')) ?? [];
-const attrs = await Promise.all(mapPromises);
+const portapage = await playwrightPortapage(page);
+await portapage.infiniteScroll('#feed', async (newItems, ctx, stop) => {
+  // Process each batch of new DOM elements as they load
+  if (shouldStop) stop();
+});
 ```
 
-## Reference
+## Documentation
 
-See the [full documentation here](./docs/typedoc/modules.md).
-- [Portadom](./docs/typedoc/interfaces/Portadom.md)
-- [Portapage](./docs/typedoc/interfaces/Portapage.md)
+| Document                                   | Description                      |
+| ------------------------------------------ | -------------------------------- |
+| [API reference](./docs/typedoc/globals.md) | Full typedoc-generated API docs. |
+| [Changelog](./CHANGELOG.md)                | Release history.                 |
 
-## Real life exampes
+## License
 
-- [Profesia.sk Scraper](https://github.com/JuroOravec/apify-actor-profesia-sk)
-  - [Example 1](https://github.com/JuroOravec/apify-actor-profesia-sk/blob/3793915632bd81dc257d36699808635c8bc3f87e/src/pageActions/jobListing.ts#L128)
-  - [Example 2](https://github.com/JuroOravec/apify-actor-profesia-sk/blob/3793915632bd81dc257d36699808635c8bc3f87e/src/pageActions/jobDetail.ts#L75)
-- [SKCRIS Scraper](https://github.com/JuroOravec/apify-actor-skcris/blob/9ce92f9bd55ffcde91f22744e49ba97b6b4f0e44/src/pageActions/detail.ts#L510)
+MIT

package/dist/{cjs/dom → dom}/dom.d.ts

@@ -1,19 +1,20 @@
-import type { AnyNode, Cheerio } from 'cheerio';
+import type { AnyNode } from 'domhandler';
+import type { Cheerio } from 'cheerio';
 import type { ElementHandle, Locator, Page } from 'playwright';
-import { type Portadom } from './types';
+import { type Portadom } from './types.js';
 /** Implementation of Portadom in browser (using Browser API) */
-export type BrowserPortadom<T extends Element = Element> = Portadom<T, Element>;
+export type BrowserPortadom<El extends Element = Element> = Portadom<El>;
 /** Implementation of Portadom in browser (using Browser API) */
-export declare const browserPortadom: <El extends Element>(node: El) => BrowserPortadom<El>;
+export declare const browserPortadom: <El extends Element = Element>(node: El) => BrowserPortadom<El>;
 /** Implementation of Portadom in Cheerio */
 export type CheerioPortadom<El extends Cheerio<AnyNode> = Cheerio<AnyNode>> = Portadom<El, Cheerio<AnyNode>>;
 /** Implementation of Portadom in Cheerio */
-export declare const cheerioPortadom: <El extends Cheerio<AnyNode>>(cheerioNode: El, srcUrl: string | null) => CheerioPortadom<El>;
+export declare const cheerioPortadom: <El extends Cheerio<AnyNode> = Cheerio<AnyNode>>(cheerioNode: El, srcUrl: string | null) => CheerioPortadom<El>;
 /** Implementation of Portadom in Playwright using Handles */
-export type PlaywrightHandlePortadom<El extends Locator | ElementHandle<Node> = Locator | ElementHandle<Node>> = Portadom<El, Locator | ElementHandle<Node>>;
+export type PlaywrightHandlePortadom<El extends Locator | ElementHandle = Locator | ElementHandle> = Portadom<El, Locator | ElementHandle>;
 /** Implementation of Portadom in Playwright using Handles */
-export declare const playwrightHandlePortadom: <El extends ElementHandle<Node> | Locator>(node: El, page: Page) => PlaywrightHandlePortadom<El>;
+export declare const playwrightHandlePortadom: <El extends Locator | ElementHandle = Locator | ElementHandle>(node: El, page: Page) => PlaywrightHandlePortadom<El>;
 /** Implementation of Portadom in Playwright using Locators */
-export type PlaywrightLocatorPortadom<El extends Locator = Locator> = Portadom<El, Locator>;
+export type PlaywrightLocatorPortadom<El extends Locator = Locator> = Portadom<El>;
 /** Implementation of Portadom in Playwright using Locators */
-export declare const playwrightLocatorPortadom: <El extends Locator>(node: El, page: Page) => PlaywrightLocatorPortadom<El>;
+export declare const playwrightLocatorPortadom: <El extends Locator = Locator>(node: El, page: Page) => PlaywrightLocatorPortadom<El>;

package/dist/{cjs/dom → dom}/domUtils.d.ts

@@ -1,6 +1,7 @@
-import type { AnyNode, Cheerio } from 'cheerio';
+import type { AnyNode } from 'domhandler';
+import type { Cheerio } from 'cheerio';
 import type { ElementHandle, JSHandle, Locator } from 'playwright';
-import type { MaybeArray, MaybePromise } from '../utils/types';
+import type { MaybeArray, MaybePromise } from '../utils/types.js';
 /**
  * Given a Cheerio selection, split it into an array of Cheerio selections,
  * where each has only one element.
@@ -9,7 +10,7 @@ import type { MaybeArray, MaybePromise } from '../utils/types';
  *
  * To `[Cheerio[el], Cheerio[el], Cheerio[el], Cheerio[el]]`
  */
-export declare const splitCheerioSelection: (cheerioSel: Cheerio<AnyNode>) => Cheerio<AnyNode>[];
+export declare const splitCheerioSelection: <T extends AnyNode>(cheerioSel: Cheerio<T>) => Cheerio<T>[];
 /**
  * Given a Playwright JSHandle that points to an array of Elements, split it into an array of
  * ElementHandles, where each has only one element.
@@ -18,12 +19,12 @@ export declare const splitCheerioSelection: (cheerioSel: Cheerio<AnyNode>) => Ch
  *
  * To `ElHandle(el), ElHandle(el), ElHandle(el)`
  */
-export declare const splitPlaywrightSelection: <T>(handle: JSHandle<T[]>) => Promise<import("playwright-core/types/structs").SmartHandle<T>[]>;
+export declare const splitPlaywrightSelection: <T>(handle: JSHandle<T[]>) => Promise<JSHandle<T>[]>;
 /** Any instance that is a Playwright Handle. */
-export type AnyHandle<T = any> = JSHandle<T> | ElementHandle<T>;
+export type AnyHandle = JSHandle | ElementHandle;
 /** Any instance that is a Playwright Handle, or can be converted to one. */
-export type HandleLike<T = any> = Locator | JSHandle<T> | ElementHandle<T>;
-export declare const handleIsLocator: <T>(h: HandleLike<T>) => h is Locator;
+export type HandleLike = Locator | JSHandle | ElementHandle;
+export declare const handleIsLocator: (h: HandleLike) => h is Locator;
 /**
  * Join several Locators and Handles in a single JSHandle.
  *
@@ -31,12 +32,12 @@ export declare const handleIsLocator: <T>(h: HandleLike<T>) => h is Locator;
  *
  * To override how Locators are resolved, supply own `locatorResolver` function.
  */
-export declare const mergeHandles: <T = any>(handles: HandleLike<T>[], options?: {
+export declare const mergeHandles: (handles: HandleLike[], options?: {
     /**
      * Configure how to process {@link Locator}s into {@link JSHandle}s.
      *
      * By default, Locator resolves to the first DOM Element it matches,
      * using {@link Locator.elementHandle}.
      */
-    locatorResolver?: ((loc: Locator) => MaybePromise<MaybeArray<AnyHandle<T>>>) | undefined;
-} | undefined) => Promise<JSHandle<T[]>>;
+    locatorResolver?: (loc: Locator) => MaybePromise<MaybeArray<AnyHandle>>;
+}) => Promise<JSHandle<Element[]>>;