lacspace-leads 1.7.1 → 1.8.0

package/LICENSE

@@ -0,0 +1,51 @@
+Lacspace Free Licence
+Version 1.0, August 2026
+
+Copyright (c) 2026 Lacspace
+
+PREAMBLE
+
+This software is published by Lacspace under the Lacspace Free Licence — a free,
+permissive licence that lets you use this software for any purpose, including in
+commercial products and services, at no cost. It grants the same freedoms as
+common permissive open-source licences; the only condition is that this notice
+travels with the software. The canonical, always-current text of this licence is
+maintained at https://lacspace.com/licenses/lacspace-free-1.0
+
+GRANT OF RIGHTS
+
+Permission is hereby granted, free of charge, to any person or organisation
+obtaining a copy of this software and its associated documentation and data files
+(the "Software"), to deal in the Software without restriction, including without
+limitation the rights to use, copy, modify, merge, publish, distribute,
+sublicense, and/or sell copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the conditions below. These rights are
+granted for any purpose, personal or commercial, and are perpetual, worldwide,
+non-exclusive, and royalty-free.
+
+CONDITIONS
+
+The above copyright notice, this permission notice, and the name of this licence
+("Lacspace Free Licence") shall be included in all copies or substantial portions
+of the Software.
+
+TRADEMARKS
+
+This licence does not grant permission to use the trade names, trademarks, service
+marks, logos, or product names of Lacspace, except as required to reproduce the
+notice above or to describe the origin of the Software in a truthful manner.
+
+DISCLAIMER OF WARRANTY AND LIMITATION OF LIABILITY
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN
+AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+---
+
+The Lacspace Free Licence is a source-available, permissive licence and is not (as
+of this version) an OSI-approved licence. In substance it grants the same freedoms
+as the MIT Licence. Learn more at https://lacspace.com/licenses

package/README.md

@@ -10,6 +10,7 @@ That opens a browser, searches Maps for *"restaurants in Baneshwor, Kathmandu"*,
 
 ## Why it's different
 
+- **Live file, row by row** — every lead is written to your file the moment it's found: CSV/NDJSON rows are appended, JSON stays a valid array, Excel is rewritten each time. Open it mid-run and the rows are there; stop with Ctrl-C and nothing collected is lost. `--resume` carries on from the file.
 - **Free & keyless** — uses a real browser (via [Playwright](https://playwright.dev)), not a paid Places API.
 - **Ask for a number, get that number** — one Google search stops at ~120 results; `--target 500` keeps searching (every area you name, then tiles of the map) until it has 500 unique businesses.
 - **Sweep a whole city** — comma-separate areas and it runs each search, then **merges and de-duplicates** into one list: `--area "Thamel,Baneshwor,Patan"`.
@@ -151,6 +152,8 @@ npx lacspace-leads --config campaign.json
 | Far fewer leads than the target | The map ran out — that really is everything Google lists for that query | More areas or cities, related `--type`s, or a bigger `--step` |
 | About 114 back from `--limit 500` | Google's per-search ceiling | Use `--target 500` instead |
 | No leads at all | Usually a CAPTCHA or a page change | Drop `--headless` to watch it, raise `--delay`, try again shortly |
+| Empty "Reviews" column | Google currently hides review counts from signed-out visitors (the rating still shows) | Nothing to fix on our side; sort by `rating` instead |
+| Excel file "can't be written" warning | The file is open in Excel, which locks it | Rows are kept and written as soon as you close it |
 | Empty address / website columns | `--no-details` was on, so listings were never opened | Remove it (`--no-website` and `--split` turn details back on for you) |
 | Everything grouped under "other" with `--split` | Same cause — no addresses to match against | Same fix |
 
@@ -361,6 +364,23 @@ npx lacspace-leads --config campaign.json
 
 Every search is run, merged and de-duplicated; the shared options, filters and output settings apply across the whole campaign. Programmatically: `runConfig(config)`.
 
+## Live output: watch the file fill up
+
+Ask for 100 leads and the file gets row 1 as soon as the first business is read, row 2 a few seconds later, and so on. Each lead is finished before it is written: website cleaned, email and socials found (in the background, while the browser opens the next listing), email verified, phone normalised, de-duplicated and checked against every filter. Each row is a final row, and a `--target 100` with filters stops at 100 *matching* leads.
+
+```
+  ✚ 37/100 Himalayan Java Coffee — 01-4422519 · himalayanjava.com  (14.2/min, ~5 min left)
+```
+
+| Format | While it runs |
+| --- | --- |
+| `csv` | header first, then one appended line per lead |
+| `ndjson` | one appended line per lead (also streams live to stdout with `-o -`) |
+| `json` | always a valid JSON array: the closing `]` is moved after each lead |
+| `xlsx` | the workbook is rewritten safely (temp file + rename) after each lead |
+
+When the run ends the file is rewritten once more in its final order (`--sort`, `--append` merges). Press **Ctrl-C** and it stops cleanly, saving everything collected. Then run the same command with `--resume`: it keeps the rows in the file, skips those listings and fills up to the `--target`. `--no-live` turns this off and writes only at the end.
+
 ## Resume a long sweep (`--resume`)
 
 Sweeping a whole city one neighbourhood at a time can take a while, and Google may throw a CAPTCHA halfway through. Add `--resume` and every completed sub-search is checkpointed to `<out>.checkpoint.json` — the leads gathered so far plus which searches are done. Re-run the exact same command and it **skips the areas already collected** and picks up where it stopped; when the sweep finishes cleanly the checkpoint is deleted.
@@ -372,7 +392,7 @@ npx lacspace-leads cafes --city Kathmandu \
 # crashes after Patan? just run it again — Thamel/Baneshwor/Patan are skipped.
 ```
 
-`--resume` applies to multi-search sweeps and `--config` campaigns (it's a no-op for a single search). The checkpoint key normalises case/whitespace, so the same area sweep always lines up.
+For multi-search sweeps and `--config` campaigns, `--resume` uses the checkpoint. For a single search or a `--target` run (since 1.8.0), it continues from the rows already in the output file. The checkpoint key normalises case/whitespace, so the same area sweep always lines up.
 
 ## Run summary (`--summary`)
 
@@ -565,7 +585,8 @@ const leads = await searchLeadsBatch(
 | `enrichContacts(website)` / `extractEmails` / `extractSocials` | Website enrichment, on tap. |
 | `cleanWebsite` / `normalizePhone` / `sortLeads` | Pure data-cleaning helpers (unit-tested). |
 | `haversineMeters` / `parseLatLngPair` / `parseDistance` | Pure geo helpers for radius search. |
-| `filterLeads` / `dedupeLeads` / `subtractLeads` | Pure post-processing — filter, in-list dedupe, and cross-file dedupe. |
+| `filterLeads` / `dedupeLeads` / `subtractLeads` / `dedupeKey` | Pure post-processing — filter, in-list dedupe, and cross-file dedupe. |
+| `createLiveWriter(options)` | Write leads to csv / ndjson / json / xlsx (or stdout) one by one as they arrive. |
 | `summarize(leads)` / `formatSummary(s)` | Rating bands, contact coverage %, top categories — the `--summary` engine. |
 | `parsePriceLevel` / `priceLevelValue` / `parseBusinessStatus` / `parseClaimed` / `parseOpenNow` / `parseCategoryTags` | Pure field parsers (unit-tested against HTML/aria snippets). |
 | `pipeToEnrich(handler)` / `leadsToEnrichInput` / `leadDomains` / `leadDomain` | Bridge collected leads to `lacspace-enrich` (no hard dep). |
@@ -573,7 +594,21 @@ const leads = await searchLeadsBatch(
 | `expandQueries` / `resolvePreset` / `FIELD_PRESETS` | Batch expansion + field presets. |
 | `composeQuery` / `mapsSearchUrl` / `normalizeFields` / `defaultFilename` | Query + helper utilities. |
 
-The `onProgress` and `onLead` callbacks stream progress and each lead as it's found — handy for live UIs or crash-safe writing. Everything is fully typed (`Lead`, `LeadField`, `SearchOptions`, `LeadStats`, `EmailStatus`, `SortKey`, `OutputFormat`, `Contacts` …) and ships dual **ESM + CJS**.
+Write leads as they're found (1.8.0):
+
+```ts
+import { searchLeads, createLiveWriter, DEFAULT_FIELDS } from "lacspace-leads";
+
+const writer = createLiveWriter({ file: "cafes.csv", format: "csv", fields: [...DEFAULT_FIELDS] });
+const leads = await searchLeads({
+  type: "cafes", city: "Kathmandu", limit: 100, headless: true,
+  onResult: (lead) => writer.add(lead),          // a finished row, the moment it's ready
+  remaining: () => 100 - writer.added,           // stop opening listings once there are 100
+});
+writer.finish(leads);
+```
+
+`onResult` gets each lead once it is final (enriched, verified, normalised, de-duplicated, filtered). `onLead` still gets the raw listing straight off the page, and `onProgress` streams status messages. `searchLeadsBatch` and target sweeps forward `onResult` across every search and tile, each business once. Everything is fully typed (`Lead`, `LeadField`, `SearchOptions`, `LeadStats`, `EmailStatus`, `SortKey`, `OutputFormat`, `Contacts` …) and ships dual **ESM + CJS**.
 
 ## Please use it responsibly