tablefacts 0.1.0

package/src/instagram/README.md

@@ -0,0 +1,135 @@
+# Instagram photos
+
+Downloads a restaurant's Instagram photos through <https://toolzu.com>, driven by Playwright in **the user's own Edge window**. Use it to source real photos for a restaurant's site when it has no originals to send.
+
+| | |
+| --- | --- |
+| Photos of a whole profile | `--profile <link or @name>`, needs a toolzu account signed in to that browser |
+| Photos of specific posts | links as arguments or `--file links.txt`, saves every image of each post |
+| Videos, stories, highlights | not supported (videos on a profile are skipped) |
+
+**Rights first.** Only download what the restaurant owns or has allowed you to use. The restaurant sending you the originals, or using Instagram's own "Download your information" export, is better than this tool: full size, no scraping. Mention in the hand-off where each photo came from.
+
+## The browser: read this before running anything
+
+Toolzu guards its forms with a Cloudflare Turnstile check. **Headless browsers, Playwright's bundled Chromium, a fresh automated Edge and Playwright's `codegen` window all failed it** in testing. An ordinary Edge window that the user started with remote debugging passes it by itself, usually in 8 to 45 seconds. So the script does not drive a browser of its own: it **attaches to that window** (`--cdp http://localhost:9222`) and opens its own tab in it. **If nothing is listening on that port, the script starts Edge itself** (detached, so it stays open and keeps the toolzu login for the next run).
+
+Rules for agents:
+
+- **Never try to bypass, solve or fake the Cloudflare check** (no stealth plugins, header tricks, solvers). The script only waits for it (up to 2 minutes). If a "verify you are human" box appears in the window, **the user ticks it**. Tell them when you are waiting.
+- **Never kill Edge or its processes.** The window may be the user's, and the run holds a lock on its profile folder. If a run hangs, stop only the `node` process you started.
+- Leave the window open between runs. Closing it ends the session (toolzu login included).
+
+### Starting the window by hand (only if the script cannot)
+
+The script does this for you using the profile folder `C:\ig-edge` (`--edge-dir` changes it). If it reports that Edge did not open its debugging port, start it yourself:
+
+```powershell
+& "C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe" --remote-debugging-port=9222 --user-data-dir=C:\ig-edge
+```
+
+- The `&` is required in PowerShell (without it: `Unexpected token 'remote-debugging-port=9222'`).
+- **Close every other Edge window first**, or Edge ignores the debugging flag and nothing listens on 9222.
+- `--user-data-dir` must be a folder of its own (recent Edge refuses debugging on the default profile). `C:\ig-edge` keeps the toolzu login between sessions. Do not point it at `.tablefacts/instagram-profile`, which the script uses for its own fallback browser.
+- Check it is up: `curl http://localhost:9222/json/version` returns JSON. `ECONNREFUSED ::1:9222` means it is not running: ask the user to start it (it opens a window on their screen).
+- First time: in that window open toolzu.com, **sign in or sign up**, then leave it. The profile tool needs the account; the single-post tool does not.
+
+## Run
+
+```bash
+# one profile, first batch, listing only (writes nothing)
+tablefacts photos instagram --cdp http://localhost:9222 --google --profile https://www.instagram.com/<user>/ --out <folder> --dry-run
+
+# same, for real; several batches
+tablefacts photos instagram --cdp http://localhost:9222 --google --profile @<user> --pages 3 --out <folder>
+
+# specific posts
+tablefacts photos instagram --cdp http://localhost:9222 --out <folder> <post-link> <post-link>
+tablefacts photos instagram --cdp http://localhost:9222 --out <folder> --file links.txt
+```
+
+| Flag | Meaning |
+| --- | --- |
+| `--out <folder>` | required; created if missing. Use a scratch folder outside the site's public folder, then copy what you pick |
+| `--cdp <url>` | attach to the user's Edge, starting it if nothing answers (the supported way) |
+| `--edge-dir <dir>` | profile folder for that Edge, where the toolzu login lives (default `C:\ig-edge`) |
+| `--google` | reach toolzu through a Google search first, as the user does (first link only; falls back to the direct address). Library option: `viaGoogle` |
+| `--profile <link\|@name>` | download a whole profile's photos |
+| `--pages <n\|all>` | with `--profile`, batches of posts to load with NEXT (default 1) |
+| `--dry-run` | print what it would save, write nothing. Do this first |
+| `--debug` | post mode: save the results HTML in `<out>/_debug` to fix selectors |
+| `--browser msedge`, `--headed`, `--user-data-dir` | launch a browser instead of attaching. Fails the Cloudflare check unless a person ticks it; keep as a fallback |
+
+Links may carry tracking queries (`?utm_source=…`, `?img_index=1`); they are stripped and `img_index` is ignored: **every link saves all the images of its post**.
+
+Output: profile photos are `<user>-<instagram file name>.jpg`; post photos are `<shortcode>-<n>.jpg`. A re-run skips files already there. The summary prints saved, already there and failed; the exit code is non-zero if anything failed.
+
+## From code
+
+```js
+import { downloadInstagram } from 'tablefacts'
+
+const { saved, skipped, failed } = await downloadInstagram({
+  links: ['https://www.instagram.com/p/<shortcode>/'],   // and/or `file`, `profile`
+  out: 'photos/instagram',      // relative paths resolve against `projectDir`
+  cdp: 'http://localhost:9222',
+  viaGoogle: true,              // the CLI's --google
+  projectDir: '/path/to/project',
+  log: (message, level) => console.log(level ?? 'info', message),
+})
+for (const { item, reason } of failed) console.error(item, reason)   // the post or profile link that failed
+```
+
+Give at least one of `links`, `file` or `profile`. `out`, `file` and `userDataDir` resolve against `projectDir` (default
+`TABLEFACTS_PROJECT`, then the current folder), as does the fallback browser profile `.tablefacts/instagram-profile`.
+`log(message, level)` gets `'info'`, `'warn'` (skipped links, retries) or `'error'`; without it nothing is printed.
+Bad arguments throw a `TablefactsError` with code `EUSAGE` and `err.option` (`'out'`, `'links'`, `'profile'`); the CLI
+prints them with flag names and exits `2`. A missing Playwright is `EDEPENDENCY`. Post failures do not throw: they are
+in `failed`, and the CLI exits `1` when there are any.
+
+## What the script does (so you can fix it when toolzu changes)
+
+1. Optionally Google `toolzu instagram profile` (or `…photo downloader`) and click the result.
+2. Paste the link into `#instagramdownloaderform-search` and **click Download straight away**, like a person. If toolzu flags the box invalid because the check was not finished, wait for the check and click again (up to 3 times).
+3. Results land in `#ajax-results` as `.download-card` elements. A photo has `.fa-image`, a video `.fa-play-circle`; the card's button is `a[download]`. Only photo cards are used.
+4. **Both modes click each card's own Download button** and save what the browser receives (a real download, or an image opened in a new tab, which the script fetches with the browser's cookies). The click is a click event sent straight to the button: a real Playwright click scrolls each card into view and waits for the lazy-loading images to stop shifting, which made later batches about 3x slower. A real click is only the fallback. The `NEXT` button (`#viewer-next`) adds another batch to the list; each batch is saved before NEXT is pressed.
+5. It waits for the cards to appear, never for `networkidle` (toolzu's ads keep the network busy for tens of seconds). Waits 2.5 seconds between links.
+
+Everything toolzu-specific is `findImages`, `downloadProfile`, `profileImages`, `submitForm` and `waitForCheck` in `download.mjs`. Link parsing, saving, retries and the summary do not depend on toolzu.
+
+## Verified, and not
+
+Verified on the Zelavi profile and one carousel post, signed in to toolzu:
+- **Profile, `--pages 3`:** Google to toolzu, Download clicked right after pasting, then 27 + 32 + 48 = 107 photos saved with no failures in 1 minute 2 seconds (about 0.4 s per photo in every batch). All valid JPEGs at full resolution (about 1080 px wide). NEXT adds to the list and did not need another Cloudflare tick. The 4 videos on the first page were skipped.
+- **Post mode:** a 3-image carousel saved as `<shortcode>-1.jpg` to `-3.jpg`, all valid JPEGs; a re-run skipped all 3; a non-Instagram link was reported and skipped.
+- **Starting Edge:** the script started it when port 9222 was closed and reused it afterwards.
+
+**Not verified, so check before relying on them:**
+- A profile with hundreds of posts: `--pages all` has not been run, and toolzu may limit or slow later batches.
+- Private accounts, reels in post mode, and a toolzu account that is not signed in.
+- `record.mjs` (attach to the window and open Playwright's Inspector to record a flow): the user reported recording did not work. Ask them to describe or screenshot the steps instead.
+
+## Troubleshooting
+
+| Message | Cause and fix |
+| --- | --- |
+| `Edge did not open its debugging port` | another Edge is using the `C:\ig-edge` profile without debugging, or Edge is not in its usual folder; close that window or start Edge by hand (see above) |
+| `connect ECONNREFUSED ::1:9222` | `--cdp` points at a non-local address, or Edge died mid-run; run again |
+| `the Cloudflare check was not passed` | the user must tick the box in the window, or the window is not a normal Edge |
+| `not signed in to toolzu` | sign in in the debugging window; the profile tool may refuse otherwise |
+| `toolzu did not accept the link` | link is not a public profile or post, or toolzu is blocking the request |
+| `the tool returned no photos` / `no images` | markup changed or the account is private; run with `--debug` (post mode) and compare with the selectors above |
+| `clicking Download started nothing` / `Download navigated instead of saving` | the card button now behaves differently; check by hand what happens when a person clicks it |
+| `launchPersistentContext … closed` | another run still holds `.tablefacts/instagram-profile`; stop the `node` process you started |
+
+## After the download
+
+- Most of a restaurant's feed is event posters and guest photos with text. Look at every image before using it; posters with text rarely belong on the site.
+- Copy the chosen files into your site's public folder, and record real pixel `width`/`height` and an `alt` text for each.
+- In the hand-off list the profile, how many photos were taken, and whether the restaurant confirmed they may be used.
+
+## Shell notes for agents on Windows
+
+- Run the commands in PowerShell. In Git Bash, a long heredoc containing quotes can break and write nothing: put scripts in a file instead.
+- `pkill` does not exist in Git Bash; use `Get-CimInstance Win32_Process` in PowerShell and match the command line narrowly (a loose pattern also matches your own shell).
+- Playwright's ESM import needs `file:///C:/...` URLs for absolute paths on Windows.

package/src/instagram/download.mjs

@@ -0,0 +1,62 @@
+// Downloads the photos of Instagram posts through https://toolzu.com/downloader/instagram/photo/
+// Usage: tablefacts photos instagram --out <folder> [--cdp <url>] [--profile <link|@name>] [--file links.txt] [link...]
+// The work is in index.mjs (downloadInstagram); this file is only the command line.
+import { TablefactsError, cliMessage, exitCodeFor } from '../lib/errors.mjs'
+import { consoleLog } from '../lib/log.mjs'
+import { downloadInstagram } from './index.mjs'
+import { parseArgs } from './links.mjs'
+
+const HELP = `Download Instagram post photos through toolzu.com.
+
+  tablefacts photos instagram --out <folder> [options] [link...]
+
+  --out <folder>   where the images are saved (required, created if missing)
+  --file <path>    text file with one link per line (# comments and blank lines ignored)
+  --cdp <url>      attach to Edge on this debugging address, e.g. http://localhost:9222. If nothing is
+                   listening there the script starts Edge itself (the most reliable way past
+                   toolzu's Cloudflare check)
+  --edge-dir <dir> profile folder of that Edge, where the toolzu login is kept (default C:\ig-edge)
+  --google         reach toolzu through a Google search, like a person would
+  --browser <name> use an installed browser: msedge or chrome (default: bundled Chromium)
+  --user-data-dir <dir>  browser profile kept between runs (default: .tablefacts/instagram-profile)
+  --profile <link|@name> download the photos of a whole Instagram profile (videos are skipped);
+                   needs a toolzu account signed in to the browser
+  --pages <n|all>  with --profile, how many batches of posts to load with NEXT (default 1)
+  --headed         show the browser (to watch, or to pass a challenge by hand)
+  --dry-run        print the image URLs found, save nothing
+  --debug          save the page HTML after each submit into <folder>/_debug
+  --help           this text
+
+Every link saves all the images of its post; ?img_index=N in a link is ignored.
+
+Only download photos the restaurant owns or has allowed you to use.`
+
+const FLAGS = {
+  out: '--out <folder>',
+  file: '--file',
+  profile: '--profile',
+  links: 'the links',
+  cdp: '--cdp',
+  debug: '--debug',
+}
+
+let summary
+try {
+  const opts = parseArgs(process.argv.slice(2))
+  if (opts.help) {
+    console.log(HELP)
+    process.exit(0)
+  }
+  if (!opts.out) {
+    console.error('Missing --out <folder>.\n\n' + HELP)
+    process.exit(2)
+  }
+  summary = await downloadInstagram({ ...opts, log: consoleLog })
+} catch (err) {
+  if (!(err instanceof TablefactsError)) throw err
+  console.error(cliMessage(err, FLAGS))
+  process.exit(exitCodeFor(err))
+}
+console.log(`\nSaved ${summary.saved}, already there ${summary.skipped}, failed ${summary.failed.length}.`)
+for (const f of summary.failed) console.log(`  ${f.item}: ${cliMessage({ message: f.reason }, FLAGS)}`)
+process.exit(summary.failed.length ? 1 : 0)

package/src/instagram/index.mjs

@@ -0,0 +1,355 @@
+// Library entry for downloading Instagram post photos through https://toolzu.com/downloader/instagram/photo/
+// Silent unless a `log` function is given; importing this file loads nothing heavy (playwright is loaded on use).
+import { readFile, writeFile } from 'node:fs/promises'
+import { join } from 'node:path'
+import { ensureEdge } from '../lib/edge.mjs'
+import { TablefactsError, optionError } from '../lib/errors.mjs'
+import { assertImageResponse, writeImage } from '../lib/images.mjs'
+import { normalizeLog } from '../lib/log.mjs'
+import { DELAY_MS, newSummary, prepareOut, storeFile } from '../lib/photos.mjs'
+import { loadPlaywright } from '../lib/playwright.mjs'
+import { resolveIn, workDirIn } from '../lib/project.mjs'
+import { normalize, parseProfile } from './links.mjs'
+
+const TOOL = 'https://toolzu.com/downloader/instagram/photo/'
+const PROFILE_TOOL = 'https://toolzu.com/downloader/instagram/profile/'
+const IMAGE_HOSTS = /(cdninstagram\.com|fbcdn\.net)$/
+
+async function readLinks(links, file) {
+  const raw = [...links]
+  if (file) {
+    const text = await readFile(file, 'utf8')
+    for (const line of text.split(/\r?\n/)) {
+      const t = line.replace(/#.*/, '').trim()
+      if (t) raw.push(t)
+    }
+  }
+  const seen = new Map()
+  const bad = []
+  for (const r of raw) {
+    const n = normalize(r)
+    if (!n) bad.push(r)
+    else if (!seen.has(n.shortcode)) seen.set(n.shortcode, n)
+  }
+  return { posts: [...seen.values()], bad }
+}
+
+// Runs `attempt` once more if it fails the first time, telling `onRetry` why.
+async function retry(attempt, onRetry) {
+  try {
+    return await attempt()
+  } catch (err) {
+    onRetry(err)
+    return attempt()
+  }
+}
+
+// Toolzu guards its forms with a Cloudflare Turnstile that fills a hidden field when passed.
+// Headless fails it. In a real browser (cdp) it passes alone, sometimes after 40 seconds;
+// if it shows a box, tick it. We only wait for it, never try to solve it.
+async function waitForCheck(page, log, visible) {
+  if (visible) log('  waiting for the Cloudflare check: tick it in the browser window if it asks (2 min)')
+  await page
+    .waitForFunction(() => [...document.querySelectorAll('input[name*=turnstile]')].some((i) => i.value), null, { timeout: 120000 })
+    .catch(() => {
+      throw new TablefactsError('the Cloudflare check was not passed (use `cdp` with your own browser, or solve it by hand)', 'EFAILED')
+    })
+}
+
+// Clicks Download right after the link is pasted, as a person does. If toolzu answers that the
+// check was not finished (it marks the box invalid), wait for the check and click again.
+async function submitForm(page, path, log, visible) {
+  for (let attempt = 0; attempt < 3; attempt++) {
+    const reply = page
+      .waitForResponse((r) => r.url().includes(path) && r.request().method() === 'POST', { timeout: attempt ? 60000 : 10000 })
+      .catch(() => null)
+    await page.click('#downloader-form button[type=submit]')
+    const answered = await reply
+    await page.waitForTimeout(500)
+    if (answered && !(await page.locator('#instagramdownloaderform-search.is-invalid').count())) return
+    await waitForCheck(page, log, visible)
+  }
+  throw new TablefactsError('toolzu did not accept the link', 'EFAILED')
+}
+
+// Opens the tool from a Google search: type the query, click the toolzu result.
+// Falls back to the direct address if Google asks for its own check or shows no result.
+async function openFromGoogle(page, query, resultHref, log) {
+  try {
+    await page.goto('https://www.google.com/', { waitUntil: 'domcontentloaded' })
+    await page.getByRole('button', { name: /accept all|aceptar todo/i }).click({ timeout: 3000 }).catch(() => {})
+    const box = page.locator('textarea[name=q], input[name=q]').first()
+    await box.click({ timeout: 10000 })
+    await box.pressSequentially(query, { delay: 90 })
+    await page.keyboard.press('Enter')
+    await page.locator(`a[href*="${resultHref}"]`).first().click({ timeout: 20000 })
+    await page.waitForSelector('#instagramdownloaderform-search', { timeout: 20000 })
+    return true
+  } catch {
+    log('  Google did not lead to toolzu; opening it directly', 'warn')
+    return false
+  }
+}
+
+// Submits one post link to the tool and returns the image URLs it offers.
+async function findImages(page, post, debugDir, { log, visible, google }) {
+  const opened = google && (await openFromGoogle(page, 'toolzu instagram photo downloader', 'toolzu.com/downloader/instagram/photo', log))
+  if (!opened) await page.goto(TOOL, { waitUntil: 'domcontentloaded' })
+  await page.getByRole('button', { name: 'Got it' }).click({ timeout: 2000 }).catch(() => {})
+  await page.fill('#instagramdownloaderform-search', post.url)
+  await submitForm(page, '/downloader/instagram/photo', log, visible)
+  // The results are injected after the response; the cards are what we wait for.
+  await page.waitForSelector('#ajax-results .download-card', { timeout: 30000 }).catch(() => {})
+  if (debugDir) await writeFile(join(debugDir, `${post.shortcode}.html`), await page.content())
+
+  // Each result has a download button: <a download href="…cdninstagram.com/…">. Read those first
+  // and only fall back to every link and image on the page if the markup changes.
+  // `index` is the button's position among the page's Download buttons, so it can be clicked.
+  const { urls, buttons } = await page.evaluate(() => {
+    const buttons = [...document.querySelectorAll('#ajax-results a[download]')].map((a) => a.href)
+    if (buttons.length) return { urls: buttons, buttons: true }
+    return { urls: [...document.querySelectorAll('#ajax-results a[href], #ajax-results img[src]')].map((el) => el.href ?? el.src), buttons: false }
+  })
+  const images = []
+  for (const [index, url] of urls.entries()) {
+    let host
+    try {
+      host = new URL(url).hostname
+    } catch {
+      continue
+    }
+    if (IMAGE_HOSTS.test(host) && !images.some((i) => i.url === url)) images.push({ url, index: buttons ? index : null })
+  }
+  return images
+}
+
+// The photo cards currently on the profile results page; videos (play icon) are left out.
+function profileImages(page) {
+  return page.evaluate(() =>
+    [...document.querySelectorAll('#ajax-results .download-card')]
+      .filter((card) => card.querySelector('.fa-image'))
+      .map((card) => card.querySelector('a[download]')?.href)
+      .filter(Boolean),
+  )
+}
+
+// Loads a profile through the profile tool and calls `handle` with the photo cards of each
+// batch, following NEXT for `maxPages` batches. Returns how many photos it saw.
+async function downloadProfile(page, profile, maxPages, handle, { log, visible, google }) {
+  const opened = google && (await openFromGoogle(page, 'toolzu instagram profile', 'toolzu.com/downloader/instagram/profile', log))
+  if (!opened) await page.goto(PROFILE_TOOL, { waitUntil: 'domcontentloaded' })
+  await page.getByRole('button', { name: 'Got it' }).click({ timeout: 2000 }).catch(() => {})
+  if (!(await page.getByText('Logout').count())) log('  not signed in to toolzu: the profile tool may ask you to sign up', 'warn')
+  await page.fill('#instagramdownloaderform-search', profile.url)
+  await submitForm(page, '/downloader/instagram/profile', log, visible)
+  await page.waitForSelector('#ajax-results .download-card', { timeout: 60000 }).catch(() => {})
+
+  const found = new Set()
+  // Hands over the cards not seen yet, with their position among the photo cards on the page.
+  const take = async () => {
+    const fresh = (await profileImages(page)).map((url, index) => ({ url, index })).filter((c) => !found.has(c.url))
+    for (const c of fresh) found.add(c.url)
+    if (fresh.length) await handle(fresh)
+    return fresh.length
+  }
+  await take()
+  for (let n = 1; n < maxPages; n++) {
+    const next = page.locator('#viewer-next')
+    if (!(await next.count()) || !(await next.isVisible())) break
+    log(`  loading batch ${n + 1} (${found.size} photos so far)`)
+    const before = await page.locator('#ajax-results .download-card').count()
+    const firstHref = await page.locator('#ajax-results a[download]').first().getAttribute('href')
+    await next.click()
+    // The next batch is in once there are more cards than before, or the first one is a different
+    // photo (in case NEXT replaces the list instead of adding to it).
+    await page
+      .waitForFunction(
+        ({ n, first }) =>
+          document.querySelectorAll('#ajax-results .download-card').length > n ||
+          document.querySelector('#ajax-results a[download]')?.getAttribute('href') !== first,
+        { n: before, first: firstHref },
+        { timeout: 90000 },
+      )
+      .catch(() => log('  NEXT brought no new photos', 'warn'))
+    const added = await take()
+    if (!added && (await page.locator('#ajax-results .download-card').count()) <= before) break
+    await page.waitForTimeout(DELAY_MS)
+  }
+  return found.size
+}
+
+// Clicks a card's Download button and saves what the browser receives. Depending on the link it
+// arrives as a real download or opens the image in a new tab; both are handled.
+async function saveByClick(page, link, file) {
+  const context = page.context()
+  const before = page.url()
+  // First a click event sent straight to the button: nothing scrolls and nothing waits for the
+  // layout to settle, which on a long list of lazy-loading cards is what makes a real click slow.
+  // If that starts nothing, a real click (scroll, wait, click) is the fallback.
+  let got = null
+  for (const real of [false, true]) {
+    const timeout = real ? 20000 : 8000
+    const started = Promise.race([
+      page.waitForEvent('download', { timeout }).then((download) => ({ download })),
+      context.waitForEvent('page', { timeout }).then((tab) => ({ tab })),
+    ])
+    await (real ? link.click() : link.dispatchEvent('click'))
+    got = await started.catch(() => null)
+    if (got) break
+  }
+  if (!got) throw new TablefactsError('clicking Download started nothing', 'EFAILED')
+  if (got.download) return got.download.saveAs(file)
+  await got.tab.waitForURL((u) => u.toString() !== 'about:blank', { timeout: 15000 }).catch(() => {})
+  const url = got.tab.url()
+  await got.tab.close()
+  if (page.url() !== before) throw new TablefactsError('the results page was left; Download navigated instead of saving', 'EFAILED')
+  const res = await context.request.get(url)
+  const bytes = await res.body()
+  assertImageResponse({ ok: res.ok(), status: res.status(), contentType: res.headers()['content-type'], bytes })
+  await writeImage(file, bytes)
+}
+
+async function saveUrl(url, file) {
+  const res = await fetch(url)
+  const bytes = Buffer.from(await res.arrayBuffer())
+  assertImageResponse({ ok: res.ok, status: res.status, contentType: res.headers.get('content-type'), bytes })
+  await writeImage(file, bytes)
+}
+
+/**
+ * Downloads the photos of Instagram posts (and optionally a whole profile) through toolzu.com.
+ * At least one of `links`, `file` or `profile` is required. Resolves to a summary whose `failed`
+ * lists `{ item, reason }` and whose `found` is present only in a dry run. Throws a TablefactsError
+ * on invalid arguments (code 'EUSAGE', with `option`), a missing playwright ('EDEPENDENCY') or a
+ * browser that cannot be reached.
+ * `out`, `file` and `userDataDir` resolve against `projectDir` (default: TABLEFACTS_PROJECT or the
+ * current folder). `viaGoogle` reaches toolzu through a Google search. `log(message, level)` gets
+ * level 'info', 'warn' or 'error'.
+ * @param {import('../lib/types.mjs').InstagramOptions} options
+ * @returns {Promise<import('../lib/types.mjs').PhotoSummary>}
+ */
+export async function downloadInstagram({
+  links = [],
+  file,
+  out,
+  profile,
+  pages = 1,
+  cdp,
+  edgeDir,
+  viaGoogle = false,
+  browser,
+  userDataDir,
+  projectDir,
+  headed = false,
+  dryRun = false,
+  debug = false,
+  log: logOption,
+} = {}) {
+  const log = normalizeLog(logOption)
+  if (!out) throw optionError('out', '`out` is required')
+  const { posts, bad } = await readLinks(links, file && resolveIn(projectDir, file))
+  for (const b of bad) log(`Skipping, not an Instagram post link: ${b}`, 'warn')
+  const profileInfo = profile ? parseProfile(profile) : null
+  if (profile && !profileInfo) throw optionError('profile', `\`profile\` is not an Instagram profile: ${profile}`)
+  if (!posts.length && !profileInfo) throw optionError('links', 'no valid Instagram post links in `links`/`file`/`profile`')
+
+  const { outDir, debugDir } = await prepareOut({ out, dryRun, debug, projectDir })
+  const summary = newSummary({ dryRun })
+  const visible = headed || Boolean(cdp) || viaGoogle
+  // Through Google once, for the first page opened only; later ones open the tool directly.
+  let usedGoogle = false
+  const takeGoogle = () => viaGoogle && !usedGoogle && (usedGoogle = true)
+
+  const { chromium } = await loadPlaywright()
+  let context
+  let close
+  let page
+  try {
+    if (cdp) {
+      // Your own browser: our tab is opened and closed here, the browser itself is left running.
+      await ensureEdge(cdp, edgeDir)
+      const connection = await chromium.connectOverCDP(cdp)
+      context = connection.contexts()[0]
+      close = async () => {
+        try {
+          await page?.close()
+        } finally {
+          await connection.close()
+        }
+      }
+      page = await context.newPage()
+    } else {
+      // A persistent profile keeps cookies between runs, which helps with the Cloudflare check.
+      context = await chromium.launchPersistentContext(resolveIn(projectDir, userDataDir ?? workDirIn(projectDir, 'instagram-profile')), {
+        channel: browser,
+        headless: !headed,
+        args: ['--disable-blink-features=AutomationControlled'],
+      })
+      page = context.pages()[0] ?? (await context.newPage())
+      close = () => context.close()
+    }
+
+    // Saves one image unless the file is already there (or only reports it in a dry run).
+    // `link` is the locator of the card's own Download button, or null to fetch the URL directly.
+    const store = (name, url, link) =>
+      storeFile({
+        outDir,
+        name,
+        url,
+        dryRun,
+        summary,
+        log,
+        save: ({ file }) => (link ? saveByClick(page, link, file) : saveUrl(url, file)),
+      })
+
+    if (profileInfo) {
+      log(`Profile ${profileInfo.url}`)
+      try {
+        const maxPages = pages === 'all' ? Infinity : Math.max(1, Number(pages) || 1)
+        const handle = async (cards) => {
+          log(`  ${cards.length} photos`)
+          for (const { url, index } of cards) {
+            // The file name from Instagram is unique per photo, so a re-run skips what is saved.
+            const stem = new URL(url).pathname.split('/').pop().replace(/\.\w+$/, '').replace(/_n$/, '')
+            const name = `${profileInfo.user}-${stem}.jpg`
+            const before = summary.saved
+            // The card's own Download button, as a person clicks it.
+            await store(name, url, page.locator('#ajax-results .download-card:has(.fa-image) a[download]').nth(index))
+            if (summary.saved > before) await page.waitForTimeout(150)
+          }
+        }
+        const seen = await downloadProfile(page, profileInfo, maxPages, handle, { log, visible, google: takeGoogle() })
+        if (!seen) throw new TablefactsError('the tool returned no photos', 'EFAILED')
+      } catch (err) {
+        summary.failed.push({ item: profileInfo.url, reason: err.message })
+        log(`  failed: ${err.message}`, 'error')
+      }
+    }
+
+    for (const [i, post] of posts.entries()) {
+      log(`[${i + 1}/${posts.length}] ${post.url}`)
+      try {
+        const images = await retry(
+          async () => {
+            const found = await findImages(page, post, debugDir, { log, visible, google: takeGoogle() })
+            if (!found.length) throw new TablefactsError('the tool returned no images', 'EFAILED')
+            return found
+          },
+          (err) => log(`  retrying (${err.message})`, 'warn'),
+        )
+        for (const [n, { url, index }] of images.entries()) {
+          // The card's own Download button when we know which one it is, as a person clicks it.
+          const link = index === null ? null : page.locator('#ajax-results a[download]').nth(index)
+          await store(`${post.shortcode}-${n + 1}.jpg`, url, link)
+        }
+      } catch (err) {
+        summary.failed.push({ item: post.url, reason: err.message })
+        log(`  failed: ${err.message}`, 'error')
+      }
+      if (i < posts.length - 1) await page.waitForTimeout(DELAY_MS)
+    }
+  } finally {
+    await close?.().catch(() => {})
+  }
+  return summary
+}

package/src/instagram/links.mjs

@@ -0,0 +1,55 @@
+// The pure parts of download.mjs: reading the command line and the links it is given.
+// Split out so they can be tested without starting a browser.
+
+import { usageError } from '../lib/errors.mjs'
+
+export function parseArgs(argv) {
+  const opts = { links: [], headed: false, dryRun: false, debug: false }
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i]
+    if (a === '--out') opts.out = argv[++i]
+    else if (a === '--cdp') opts.cdp = argv[++i]
+    else if (a === '--edge-dir') opts.edgeDir = argv[++i]
+    else if (a === '--google') opts.viaGoogle = true
+    else if (a === '--file') opts.file = argv[++i]
+    else if (a === '--browser') opts.browser = argv[++i]
+    else if (a === '--profile') opts.profile = argv[++i]
+    else if (a === '--user-data-dir') opts.userDataDir = argv[++i]
+    else if (a === '--pages') opts.pages = argv[++i]
+    else if (a === '--headed') opts.headed = true
+    else if (a === '--dry-run') opts.dryRun = true
+    else if (a === '--debug') opts.debug = true
+    else if (a === '--help' || a === '-h') opts.help = true
+    else if (a.startsWith('--')) throw usageError(`Unknown option ${a}`)
+    else opts.links.push(a)
+  }
+  return opts
+}
+
+// Keeps only post and reel links and drops the tracking query, so duplicates collapse.
+export function normalize(raw) {
+  let url
+  try {
+    url = new URL(raw.trim())
+  } catch {
+    return null
+  }
+  if (!/(^|\.)instagram\.com$/.test(url.hostname)) return null
+  const m = url.pathname.match(/\/(p|reel|reels|tv)\/([\w-]+)/)
+  if (!m) return null
+  return { shortcode: m[2], url: `https://www.instagram.com/${m[1] === 'reels' ? 'reel' : m[1]}/${m[2]}/` }
+}
+
+// '@name', 'name' or a profile link -> { user, url }
+export function parseProfile(raw) {
+  let user = raw.trim()
+  if (/^https?:/i.test(user)) {
+    try {
+      user = new URL(user).pathname.split('/').filter(Boolean)[0] ?? ''
+    } catch {
+      user = ''
+    }
+  }
+  user = user.replace(/^@/, '')
+  return /^[\w.]+$/.test(user) ? { user, url: `https://www.instagram.com/${user}/` } : null
+}

package/src/instagram/record.mjs

@@ -0,0 +1,18 @@
+// Records what you do on a page, in your own browser, as Playwright code.
+// Start Edge with --remote-debugging-port=9222 first, then:
+//   node src/instagram/record.mjs [url]
+// In the Inspector window press Record, do the steps in the browser tab, copy the code.
+import { loadPlaywright } from '../lib/playwright.mjs'
+
+try {
+  const { chromium } = await loadPlaywright()
+  const url = process.argv[2] ?? 'https://www.google.com/'
+  const browser = await chromium.connectOverCDP('http://localhost:9222')
+  const page = await browser.contexts()[0].newPage()
+  await page.goto(url)
+  await page.pause()
+  await browser.close()
+} catch (err) {
+  console.error(err.message)
+  process.exitCode = 1
+}