salesprompter-cli 0.1.82 → 0.1.83
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +49 -6
- package/dist/cli.js +48 -33
- package/dist/company-leads.js +144 -16
- package/dist/company-recovery.js +84 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -44,11 +44,54 @@ For headless or automation use, generate a CLI token in the app and run `salespr
|
|
|
44
44
|
|
|
45
45
|
`leads:at-companies` researches Director, Head-of, VP and C-level contacts across functions. It exports a shortlist for review without starting email enrichment or outreach.
|
|
46
46
|
|
|
47
|
-
The checkpoint, shortlist and review exports refresh after every
|
|
47
|
+
The checkpoint, shortlist and review exports refresh after every collected page, so long runs expose saved progress immediately. Company labels with punctuation, such as `(TKMS)`, are escaped safely in search queries.
|
|
48
48
|
|
|
49
49
|
Use `--report-only` to refresh saved reports without contacting LinkedIn. The workspace-bound checkpoint is still checked. `progress` separates unique shortlisted people, review rows, review people, review-only people, pending searches, partial/unknown coverage and unresolved companies. `collectionComplete` is false while any coverage gap remains; `nextActions` explains what is left. Unrestricted geography is labeled explicitly.
|
|
50
50
|
|
|
51
|
-
|
|
51
|
+
### Recover saved research without recollecting
|
|
52
|
+
|
|
53
|
+
Version 0.1.83 separates collection evidence from review decisions. Keep the original brief and output directory:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
# Recheck saved roles and write an actionable recovery.json; no LinkedIn requests.
|
|
57
|
+
salesprompter leads:at-companies --brief companies.json --out-dir ./research --report-only
|
|
58
|
+
# Verify canonical employer names against native company details for selected targets.
|
|
59
|
+
salesprompter leads:at-companies --brief companies.json --out-dir ./research \
|
|
60
|
+
--browser chrome --verify-employers --company 'Example (CH)'
|
|
61
|
+
# Continue partial searches until company shortlist ceilings are filled.
|
|
62
|
+
salesprompter leads:at-companies --brief companies.json --out-dir ./research \
|
|
63
|
+
--browser chrome --continue-partial quota --all
|
|
64
|
+
# Or request all accessible pages for one exact brief company name.
|
|
65
|
+
salesprompter leads:at-companies --brief companies.json --out-dir ./research \
|
|
66
|
+
--browser chrome --continue-partial exhaustive --company 'Example (CH)'
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`--company` is repeatable and limits network work, not the overall report. `--verify-employers` checks scoped employers with saved candidates and caches their canonical-name evidence. Only equivalent presentation/legal-suffix names (for example, `BioNTech` / `BioNTech SE`) qualify automatically; group, division and subsidiary differences still require review. The exact numeric current-employer ID check always applies.
|
|
70
|
+
|
|
71
|
+
`recovery.json` contains alias proposals, identity-search attempts and candidate evidence, explicit unresolved reasons, and a fingerprint-bound review template. Proposals are **not** approvals. Copy its `reviewFileTemplate` into a private JSON file and add only independently verified decisions:
|
|
72
|
+
|
|
73
|
+
```json
|
|
74
|
+
{
|
|
75
|
+
"fingerprint": "COPY_THE_64_CHARACTER_FINGERPRINT_FROM_RECOVERY_JSON",
|
|
76
|
+
"employerAliases": [
|
|
77
|
+
{ "companyId": "123", "name": "Exact source name", "evidenceUrl": "https://official.example/evidence" }
|
|
78
|
+
],
|
|
79
|
+
"identities": [
|
|
80
|
+
{ "targetName": "Exact name in brief", "companyId": "456", "canonicalName": "Verified source name", "evidenceUrl": "https://official.example/evidence" }
|
|
81
|
+
]
|
|
82
|
+
}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
salesprompter leads:at-companies --brief companies.json --out-dir ./research \
|
|
87
|
+
--review-file ./verified-decisions.json --report-only
|
|
88
|
+
# Then collect any newly resolved identities through the same checkpoint.
|
|
89
|
+
salesprompter leads:at-companies --brief companies.json --out-dir ./research --browser chrome --all
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
The CLI validates and persists the complete review file as `review-decisions.json`, checks its workspace/brief fingerprint, and rejects conflicting/shared identities. Supplied evidence URLs record human decisions; their contents are not independently verified by `--review-file`. Later runs reuse the decisions. An alias never approves an ambiguous role. For a new collection, `verifiedEmployerAliases` in the company brief remains supported; changing the original targeting brief still requires a new output directory.
|
|
93
|
+
|
|
94
|
+
Partial continuation checkpoints the raw next offset and deduplicates people. Old checkpoints without an offset replay the first page once before advancing. `quota` stops at the shortlist ceiling; `exhaustive` means attempting the accessible 2,500-result window, **not** a promise of exhaustive coverage. Count changes, empty/repeated pages and larger result sets remain explicitly incomplete. Stopped searches require investigation; the CLI does not automatically clear `stoppedReason`. `--all` controls the number of jobs, not these safety limits. Make a private copy of the output directory before comparing old and new role-policy results.
|
|
52
95
|
|
|
53
96
|
For standalone browser research, sign in once to the CLI-owned Chrome profile, then select Chrome:
|
|
54
97
|
|
|
@@ -74,7 +117,7 @@ For a name-only list, the executable end-to-end research mode is:
|
|
|
74
117
|
salesprompter leads:at-companies --brief companies.json --out-dir ./research --resolve-companies --all
|
|
75
118
|
```
|
|
76
119
|
|
|
77
|
-
The CLI first collects prepared IDs, then resolves missing IDs through paginated Account Search and immediately collects their contacts. It
|
|
120
|
+
The CLI first collects prepared IDs, then resolves missing IDs through paginated Account Search and immediately collects their contacts. It tries up to three name queries, normalizing only presentation/legal-suffix differences. Automatic resolution requires a unique matching identity in a complete result set of at most 1,000 companies plus matching native company-detail evidence. Fuzzy, shared-ID, oversized, ambiguous and composite parent/subsidiary targets remain unresolved with diagnostics in `recovery.json`. Resolutions and misses are saved in the workspace-bound checkpoint and `company-resolutions.json`, without editing the input brief. Repeat the command to resume. Add `--retry-unresolved --resolve-companies` to intentionally retry saved identity misses without repeating successful work; add `--company` to narrow the retry. `--all` removes the invocation's job-count limit, not LinkedIn pacing or per-function candidate limits. A working LinkedIn session is required for new requests; use `--browser chrome` to eliminate the external browser-worker dependency.
|
|
78
121
|
|
|
79
122
|
```bash
|
|
80
123
|
salesprompter leads:at-companies --brief companies.json --out-dir ./research \
|
|
@@ -102,13 +145,13 @@ The default six functions are Digital Marketing & CRM, Digital Product & UX, Sof
|
|
|
102
145
|
|
|
103
146
|
Contacts must match the current company ID, a senior title and the requested function. Selection alternates between functions, deduplicates profile URLs and never pads a company to its ceiling. The default ceiling is 20; each company can override it. Head-of roles classified as experienced managers are searched, but generic manager titles do not pass the final seniority check.
|
|
104
147
|
|
|
105
|
-
Default role matching includes talent acquisition, business intelligence, digital workplace and German department-head titles. Broad digital/transformation, communications and operational-technology matches go into `review.csv`, not the selected contacts. Mixed-role headlines cannot borrow seniority from an assistant or former role. Current-position evidence overrides a conflicting summary employer or headline.
|
|
148
|
+
Default role matching includes talent acquisition, business intelligence, digital workplace and German department-head titles. Role policy 2 requires software-specific context for Software Development and digital/UX context for Digital Product & UX; generic engineering, physical-vehicle product roles and technical flight/hydraulic data do not automatically qualify. Broad digital/transformation, communications and operational-technology matches go into `review.csv`, not the selected contacts. Mixed-role headlines cannot borrow seniority from an assistant or former role. Current-position evidence overrides a conflicting summary employer or headline. Reports include `rolePolicyVersion`; review rows retain every applicable `reasons` entry as well as the primary `reason`.
|
|
106
149
|
|
|
107
|
-
The export retains the source employer name.
|
|
150
|
+
The export retains the source employer name and accepted alias evidence. Unverified employer-name differences go to review: verify equivalent canonical names with `--verify-employers`, or supply an evidence-backed review decision for non-equivalent names. Parent/subsidiary scope is never inferred.
|
|
108
151
|
|
|
109
152
|
For custom functions, add `reviewTerms` (a subset of `terms`) for discovery terms that should require manual review: `{"name":"IT","terms":["it","digital"],"reviewTerms":["digital"]}`. A specific non-review term in the same senior-role clause can qualify a contact.
|
|
110
153
|
|
|
111
|
-
Outputs are private local `contacts.csv`, `review.csv`, `coverage.json` (including review candidates, rejected/unresolved matches and per-function counts), and `checkpoint.json`. Review counts are candidate/function pairs, not additional unique leads. A completed search batch is not exhaustive coverage:
|
|
154
|
+
Outputs are private local `contacts.csv`, `review.csv`, `coverage.json` (including review candidates, rejected/unresolved matches and per-function counts), `recovery.json`, `company-resolutions.json`, and `checkpoint.json`. Review counts are candidate/function pairs, not additional unique leads. A completed search batch is not exhaustive coverage: ordinary collection stops at `candidatesPerDepartment`; use `--continue-partial` explicitly to advance. The report records LinkedIn's reported total and any shortfall. Default `--max-searches 12` bounds collection jobs and resolution attempts; employer-detail verification is scoped separately with `--company`. Increase job limits explicitly for larger batches. HTTP 429/999 stops the run and preserves saved pages; resume only after cooldown.
|
|
112
155
|
|
|
113
156
|
Live research requires Salesprompter workspace login and a LinkedIn session. `--browser-relay-port` uses the existing signed-in browser relay. Checkpoints are bound to both the brief and workspace; changed criteria require a new output directory. A concurrent run is refused; after a crash, remove only `.research-lock` after confirming the process stopped. This command does not import the shortlist into the workspace, find emails, or create campaigns. Review company identity, role fit and coverage before downstream use.
|
|
114
157
|
|
package/dist/cli.js
CHANGED
|
@@ -16,6 +16,7 @@ import { Command } from "commander";
|
|
|
16
16
|
import { z } from "zod";
|
|
17
17
|
import { AffiliateCopyReviewSchema, renderAffiliateCopyReview } from "./affiliate-copy.js";
|
|
18
18
|
import { CompanyBriefSchema, planCompanySearches, runCompanyResearch } from "./company-leads.js";
|
|
19
|
+
import { CompanyReviewSchema, resolveCompanyIdentity } from "./company-recovery.js";
|
|
19
20
|
import { createLazySessionRecovery, waitForFreshSession, isFreshSessionForIdentity } from "./session-recovery.js";
|
|
20
21
|
import { createChromeResearchBrowser, ChromeResearchInterruptedError } from "./chrome-browser.js";
|
|
21
22
|
import { clearAuthSession, loginWithBrowserConnect, loginWithDeviceFlow, loginWithToken, readAuthSession, requireAuthSession, shouldBypassAuth, verifySession, writeAuthSession } from "./auth.js";
|
|
@@ -15160,6 +15161,10 @@ program
|
|
|
15160
15161
|
.option("--all", "Process all pending company/function searches in this invocation", false)
|
|
15161
15162
|
.option("--resolve-companies", "Resolve unique exact company names and collect their contacts in the same run", false)
|
|
15162
15163
|
.option("--retry-unresolved", "Retry cached identity misses without discarding completed searches (requires --resolve-companies)", false)
|
|
15164
|
+
.option("--review-file <path>", "Apply fingerprint-bound, evidence-backed identity/alias decisions without changing the brief")
|
|
15165
|
+
.option("--verify-employers", "Verify canonical employer names against LinkedIn company details and save safe aliases", false)
|
|
15166
|
+
.option("--continue-partial <mode>", "Resume saved partial pages: quota or exhaustive (2500-result window)")
|
|
15167
|
+
.option("--company <name>", "Limit recovery to an exact brief company name; repeat for multiple companies", (value, previous) => [...previous, value], [])
|
|
15163
15168
|
.option("--wait-for-session <seconds>", "Wait for a refreshed extension session after an auth failure; never retry rate limits", "0")
|
|
15164
15169
|
.option("--browser-relay-port <number>", "Use the signed-in Codex browser through a loopback relay")
|
|
15165
15170
|
.option("--browser <mode>", "Use chrome for CLI-owned browser research, or session for extension credentials", "session")
|
|
@@ -15174,10 +15179,16 @@ program
|
|
|
15174
15179
|
if (waitMs && options.browserRelayPort)
|
|
15175
15180
|
throw new Error("--wait-for-session refreshes extension credentials, not a browser relay. Use one connection mode.");
|
|
15176
15181
|
const brief = await readJsonFile(path.resolve(options.brief), CompanyBriefSchema);
|
|
15182
|
+
const review = options.reviewFile ? await readJsonFile(path.resolve(options.reviewFile), CompanyReviewSchema) : undefined;
|
|
15183
|
+
const continuePartial = options.continuePartial ? z.enum(["quota", "exhaustive"]).parse(options.continuePartial) : undefined;
|
|
15184
|
+
for (const name of options.company)
|
|
15185
|
+
if (!brief.companies.some(c => c.name === name))
|
|
15186
|
+
throw new Error(`Unknown target company: ${name}`);
|
|
15177
15187
|
const jobs = planCompanySearches(brief);
|
|
15178
15188
|
const maxSearches = options.all ? Number.MAX_SAFE_INTEGER : z.coerce.number().int().min(1).max(10000).parse(options.maxSearches);
|
|
15179
15189
|
if (options.dryRun) {
|
|
15180
|
-
|
|
15190
|
+
const inScope = (name) => !options.company.length || options.company.includes(name);
|
|
15191
|
+
printOutput({ status: "ok", dryRun: true, searches: jobs.filter(job => inScope(job.company.name)), unresolvedCompanies: brief.companies.filter(c => !c.companyId && inScope(c.name)), maxSearches, recovery: { companies: options.company, verifyEmployers: options.verifyEmployers, continuePartial: continuePartial ?? null, reviewFile: options.reviewFile ? path.resolve(options.reviewFile) : null, savedCheckpointNotRead: true }, outreachStarted: false, emailEnrichmentStarted: false });
|
|
15181
15192
|
return;
|
|
15182
15193
|
}
|
|
15183
15194
|
const session = await requireAuthSession();
|
|
@@ -15185,7 +15196,7 @@ program
|
|
|
15185
15196
|
if (!orgId)
|
|
15186
15197
|
throw new Error("Choose a Salesprompter workspace before company research.");
|
|
15187
15198
|
if (options.reportOnly) {
|
|
15188
|
-
const report = await runCompanyResearch({ brief, outDir: path.resolve(options.outDir), scope: `${session.apiBaseUrl}:${orgId}`, maxSearches: 0, search: async () => { throw new Error("Report-only cannot collect leads."); } });
|
|
15199
|
+
const report = await runCompanyResearch({ brief, review, onlyCompanies: options.company, outDir: path.resolve(options.outDir), scope: `${session.apiBaseUrl}:${orgId}`, maxSearches: 0, search: async () => { throw new Error("Report-only cannot collect leads."); } });
|
|
15189
15200
|
printOutput({ status: "ok", reportOnly: true, progress: report.progress, nextActions: report.nextActions, output: report.output, outreachStarted: false, emailEnrichmentStarted: false });
|
|
15190
15201
|
return;
|
|
15191
15202
|
}
|
|
@@ -15228,41 +15239,45 @@ program
|
|
|
15228
15239
|
},
|
|
15229
15240
|
});
|
|
15230
15241
|
let startedSearches = 0;
|
|
15242
|
+
const paced = async () => { if (startedSearches++ > 0)
|
|
15243
|
+
await delay(randomIntegerBetween(5000, 8000)); };
|
|
15244
|
+
const detailsCache = new Map();
|
|
15245
|
+
const details = async (id) => {
|
|
15246
|
+
if (detailsCache.has(id))
|
|
15247
|
+
return detailsCache.get(id);
|
|
15248
|
+
await paced();
|
|
15249
|
+
process.stderr.write(`Verifying employer ${id}\n`);
|
|
15250
|
+
const response = await recoverSession(config => fetchCliImportSalesNavigatorJson({ url: buildCliImportCompanyDetailsUrl([id]), config, browserRelay: relay, timeoutMs: 30000, label: "Company identity evidence", companyId: id }));
|
|
15251
|
+
const body = cliImportCompanyRecord(response.body?.results);
|
|
15252
|
+
const record = cliImportCompanyRecord(body?.[id]);
|
|
15253
|
+
const explicitId = record?.entityUrn ? extractLinkedInSalesCompanyIdFromUrn(record.entityUrn) : null;
|
|
15254
|
+
const evidence = response.status === 200 && typeof record?.name === "string" && (!explicitId || explicitId === id)
|
|
15255
|
+
? { companyId: id, name: record.name, website: typeof record.website === "string" ? record.website : undefined, evidenceUrl: `https://www.linkedin.com/sales/company/${id}` } : null;
|
|
15256
|
+
detailsCache.set(id, evidence);
|
|
15257
|
+
return evidence;
|
|
15258
|
+
};
|
|
15231
15259
|
try {
|
|
15232
|
-
const report = await runCompanyResearch({ brief, outDir: path.resolve(options.outDir), scope: `${session.apiBaseUrl}:${orgId}`, maxSearches, retryUnresolved: options.retryUnresolved,
|
|
15233
|
-
|
|
15234
|
-
|
|
15235
|
-
|
|
15236
|
-
|
|
15237
|
-
const
|
|
15238
|
-
const response = await recoverSession(
|
|
15260
|
+
const report = await runCompanyResearch({ brief, review, onlyCompanies: options.company, continuePartial, outDir: path.resolve(options.outDir), scope: `${session.apiBaseUrl}:${orgId}`, maxSearches, retryUnresolved: options.retryUnresolved,
|
|
15261
|
+
verifyEmployer: options.verifyEmployers ? details : undefined,
|
|
15262
|
+
diagnoseCompany: !options.resolveCompanies ? undefined : company => resolveCompanyIdentity(company.name, async (query, start) => {
|
|
15263
|
+
await paced();
|
|
15264
|
+
process.stderr.write(`Resolving ${company.name}: ${query}, offset ${start}\n`);
|
|
15265
|
+
const url = buildCliImportAccountSearchUrl(query).replace(/([?&])start=\d+/, `$1start=${start}`);
|
|
15266
|
+
const response = await recoverSession(config => fetchCliImportSalesNavigatorJson({ url, config, browserRelay: relay, timeoutMs: 30000, label: `Company identity for ${company.name}` }));
|
|
15239
15267
|
if (response.status !== 200 || !response.body)
|
|
15240
15268
|
throw new Error(`Company identity search failed (${response.status}).`);
|
|
15241
15269
|
const elements = extractLocalSalesNavigatorElements(response.body);
|
|
15242
|
-
|
|
15243
|
-
|
|
15244
|
-
|
|
15245
|
-
|
|
15246
|
-
|
|
15247
|
-
|
|
15248
|
-
const
|
|
15249
|
-
|
|
15250
|
-
|
|
15251
|
-
|
|
15252
|
-
|
|
15253
|
-
if (!rows.length)
|
|
15254
|
-
return null;
|
|
15255
|
-
elements.push(...rows);
|
|
15256
|
-
}
|
|
15257
|
-
if (elements.length < total)
|
|
15258
|
-
return null;
|
|
15259
|
-
const accounts = elements.map(element => normalizeLocalSalesNavigatorAccount(element, queryUrl)).filter(Boolean);
|
|
15260
|
-
const exact = accounts.filter(account => normalizeLooseMatchText(String(account.companyName ?? "")) === normalizeLooseMatchText(company.name));
|
|
15261
|
-
const ids = new Set(exact.map(account => String(account.companyId ?? "")).filter(id => /^[1-9]\d*$/.test(id)));
|
|
15262
|
-
if (ids.size !== 1)
|
|
15263
|
-
return null;
|
|
15264
|
-
const companyId = [...ids][0];
|
|
15265
|
-
return { companyId, companyName: company.name, evidenceUrl: `https://www.linkedin.com/sales/company/${companyId}` };
|
|
15270
|
+
return { rawCount: elements.length, total: extractLocalSalesNavigatorTotalResults(response.body), candidates: elements.map(e => normalizeLocalSalesNavigatorAccount(e, url)).filter(a => a && /^[1-9]\d*$/.test(String(a.companyId)) && typeof a.companyName === "string").map(a => ({ companyId: String(a.companyId), name: String(a.companyName), evidenceUrl: `https://www.linkedin.com/sales/company/${a.companyId}` })) };
|
|
15271
|
+
}, details),
|
|
15272
|
+
searchPage: async (job, start, count) => {
|
|
15273
|
+
await paced();
|
|
15274
|
+
process.stderr.write(`Researching ${job.company.name} — ${job.department.name}, offset ${start}\n`);
|
|
15275
|
+
return recoverSession(async (config) => {
|
|
15276
|
+
const request = relay ? { url: buildSalesNavigatorLeadApiUrlFromSearchUrl(job.queryUrl, count), headers: {} } : buildSalesNavigatorApiRequestFromSearchUrl(job.queryUrl, config, count);
|
|
15277
|
+
request.url = withSalesNavigatorPaging(request.url, start, count);
|
|
15278
|
+
const response = relay ? await relay.request(request) : await fetchLocalSalesNavigatorRequest(request, { maxRetries: 0, retryBaseDelayMs: 2000, retryMaxDelayMs: 10000 });
|
|
15279
|
+
return { people: normalizeLocalSalesNavigatorPeople(response.body, request.url), totalResults: extractLocalSalesNavigatorTotalResults(response.body), rawCount: extractLocalSalesNavigatorElements(response.body).length };
|
|
15280
|
+
});
|
|
15266
15281
|
},
|
|
15267
15282
|
search: async (job) => {
|
|
15268
15283
|
if (startedSearches++ > 0)
|
package/dist/company-leads.js
CHANGED
|
@@ -3,6 +3,7 @@ import { mkdir, readFile, rename, writeFile, chmod, rmdir } from "node:fs/promis
|
|
|
3
3
|
import path from "node:path";
|
|
4
4
|
import { z } from "zod";
|
|
5
5
|
import { buildSalesNavigatorPeopleSearchUrl } from "./sales-navigator.js";
|
|
6
|
+
import { CompanyReviewSchema, companyIdentityKey } from "./company-recovery.js";
|
|
6
7
|
export const defaultDepartments = [
|
|
7
8
|
{ name: "Digital Marketing & CRM", terms: ["marketing", "crm", "growth", "communication"], reviewTerms: ["communication"] },
|
|
8
9
|
{ name: "Digital Product & UX", terms: ["product", "ux", "design"], reviewTerms: [] },
|
|
@@ -63,7 +64,17 @@ export function classifyCompanyRole(title, department) {
|
|
|
63
64
|
if (!relevant.length)
|
|
64
65
|
return "seniority_not_verified";
|
|
65
66
|
const reviewTerms = new Set(department.reviewTerms.map(words));
|
|
66
|
-
|
|
67
|
+
const contextMatches = (clause) => {
|
|
68
|
+
const t = words(clause);
|
|
69
|
+
if (department.name === "Software Development")
|
|
70
|
+
return /\b(software|cto|developer|developers|devops|frontend|backend|full stack)\b/.test(t);
|
|
71
|
+
if (department.name === "Digital Product & UX")
|
|
72
|
+
return /\b(ux|user experience|digital product|software product|product design|digital design|ui)\b/.test(t) && !/\b(vehicles|mechanical|industrial design|hardware design)\b/.test(t);
|
|
73
|
+
if (department.name === "Data & AI" && /\b(technical data|hydraulic|flight control)\b/.test(t))
|
|
74
|
+
return /\b(analytics|artificial intelligence|machine learning|data science|data engineering)\b/.test(t);
|
|
75
|
+
return true;
|
|
76
|
+
};
|
|
77
|
+
if (relevant.some(clause => contextMatches(clause) && department.terms.some(term => !reviewTerms.has(words(term)) && hasTerm(clause, term))))
|
|
67
78
|
return "match";
|
|
68
79
|
if (relevant.some(clause => department.terms.some(term => hasTerm(clause, term))))
|
|
69
80
|
return "review";
|
|
@@ -123,7 +134,7 @@ export function shortlistCompanyLeads(brief, results) {
|
|
|
123
134
|
const key = `${company.companyId}:${job.department.name}:${profile}`;
|
|
124
135
|
if (!reviewSeen.has(key)) {
|
|
125
136
|
reviewSeen.add(key);
|
|
126
|
-
review.push({ ...row, reason: employerNameDiffers ? "employer_name_differs" : "ambiguous_role" });
|
|
137
|
+
review.push({ ...row, reason: employerNameDiffers ? "employer_name_differs" : "ambiguous_role", reasons: [...(employerNameDiffers ? ["employer_name_differs"] : []), ...(role === "review" ? ["ambiguous_role"] : [])] });
|
|
127
138
|
}
|
|
128
139
|
continue;
|
|
129
140
|
}
|
|
@@ -151,11 +162,11 @@ export function shortlistCompanyLeads(brief, results) {
|
|
|
151
162
|
selected.push(...local);
|
|
152
163
|
coverage.push({ companyName: company.name, companyId: company.companyId ?? null, selected: local.length, reviewRequired: review.filter(p => p.companyId === company.companyId).length, ceiling: company.maxContacts,
|
|
153
164
|
status: !company.companyId ? "unresolved_company" : jobs.some(j => !results[j.key]) ? "pending" : local.length ? "review_ready" : "no_verified_matches",
|
|
154
|
-
departments: jobs.map(j => ({ name: j.department.name, selected: local.filter(p => p.department === j.department.name).length, collected: results[j.key]?.people.length ?? null, reported: results[j.key]?.totalResults ?? null, searchComplete: results[j.key]?.totalResults != null ? results[j.key].people.length >= results[j.key].totalResults : null })) });
|
|
165
|
+
departments: jobs.map(j => ({ name: j.department.name, selected: local.filter(p => p.department === j.department.name).length, collected: results[j.key]?.people.length ?? null, reported: results[j.key]?.totalResults ?? null, stoppedReason: results[j.key]?.stoppedReason ?? null, searchComplete: results[j.key]?.totalResults != null ? !results[j.key].stoppedReason && results[j.key].people.length >= results[j.key].totalResults : null })) });
|
|
155
166
|
}
|
|
156
167
|
const jobs = planCompanySearches(brief);
|
|
157
168
|
const completed = jobs.filter(j => results[j.key]);
|
|
158
|
-
const incomplete = completed.filter(j => results[j.key].totalResults != null && results[j.key].people.length < results[j.key].totalResults);
|
|
169
|
+
const incomplete = completed.filter(j => results[j.key].stoppedReason || results[j.key].totalResults != null && results[j.key].people.length < results[j.key].totalResults);
|
|
159
170
|
const unknown = completed.filter(j => results[j.key].totalResults == null);
|
|
160
171
|
const selectedIds = new Set(selected.map(p => String(p.profileUrl)));
|
|
161
172
|
const reviewIds = new Set(review.map(p => String(p.profileUrl)));
|
|
@@ -171,14 +182,14 @@ export function shortlistCompanyLeads(brief, results) {
|
|
|
171
182
|
};
|
|
172
183
|
const nextActions = [
|
|
173
184
|
...(progress.remainingSearches ? ["Resume the unchanged brief and output directory to collect pending searches."] : []),
|
|
174
|
-
...(unresolved.length ? ["
|
|
175
|
-
...(incomplete.length || unknown.length ? ["
|
|
176
|
-
...(review.length ? ["
|
|
185
|
+
...(unresolved.length ? ["Inspect recovery.json; retry selected identities with --resolve-companies --retry-unresolved --company, or apply evidence-backed --review-file decisions."] : []),
|
|
186
|
+
...(incomplete.length || unknown.length ? ["Use --continue-partial quota or exhaustive to resume eligible pages; inspect stoppedReason before claiming exhaustive coverage."] : []),
|
|
187
|
+
...(review.length ? ["Use --verify-employers for canonical names, or evidence-backed --review-file aliases. Ambiguous roles remain review-only."] : []),
|
|
177
188
|
];
|
|
178
|
-
return { selected, review, rejected, coverage, progress, nextActions, outreachStarted: false, emailEnrichmentStarted: false };
|
|
189
|
+
return { selected, review, rejected, coverage, progress, nextActions, rolePolicyVersion: 2, outreachStarted: false, emailEnrichmentStarted: false };
|
|
179
190
|
}
|
|
180
191
|
export function companyLeadCsv(rows) {
|
|
181
|
-
const columns = ["companyName", "companyId", "fullName", "title", "department", "profileUrl", "location", "sourceQueryUrl", "observedAt", "sourceCompanyName", "reason", "employerAliasEvidence"];
|
|
192
|
+
const columns = ["companyName", "companyId", "fullName", "title", "department", "profileUrl", "location", "sourceQueryUrl", "observedAt", "sourceCompanyName", "reason", "reasons", "employerAliasEvidence"];
|
|
182
193
|
const escape = (value) => { let s = String(value ?? ""); if (/^[=+@\-\t\r]/.test(s))
|
|
183
194
|
s = "'" + s; return `"${s.replace(/"/g, '""')}"`; };
|
|
184
195
|
return [columns.join(","), ...rows.map(r => columns.map(c => escape(r[c])).join(","))].join("\n") + "\n";
|
|
@@ -217,42 +228,159 @@ async function runCompanyResearchLocked(input) {
|
|
|
217
228
|
throw e;
|
|
218
229
|
}
|
|
219
230
|
const save = async (file, value) => { await writeFile(file + ".tmp", value, { mode: 0o600 }); await chmod(file + ".tmp", 0o600); await rename(file + ".tmp", file); };
|
|
231
|
+
let review = input.review ? CompanyReviewSchema.parse(input.review) : undefined;
|
|
232
|
+
if (!review) {
|
|
233
|
+
try {
|
|
234
|
+
review = CompanyReviewSchema.parse(JSON.parse(await readFile(path.join(outDir, "review-decisions.json"), "utf8")));
|
|
235
|
+
}
|
|
236
|
+
catch (e) {
|
|
237
|
+
if (e.code !== "ENOENT")
|
|
238
|
+
throw e;
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
if (review && review.fingerprint !== fingerprint)
|
|
242
|
+
throw new Error("Review decisions belong to a different brief or workspace.");
|
|
243
|
+
for (const identity of review?.identities ?? []) {
|
|
244
|
+
const index = brief.companies.findIndex(c => c.name === identity.targetName);
|
|
245
|
+
if (index < 0 || brief.companies[index].companyId || state.resolutions?.[String(index)] && state.resolutions[String(index)].companyId !== identity.companyId)
|
|
246
|
+
throw new Error("Reviewed identity must target an unresolved company in this brief.");
|
|
247
|
+
const occupied = [...brief.companies.flatMap(c => c.companyId ? [c.companyId] : []), ...Object.entries(state.resolutions ?? {}).filter(([i]) => i !== String(index)).flatMap(([, r]) => r ? [r.companyId] : [])];
|
|
248
|
+
if (occupied.includes(identity.companyId))
|
|
249
|
+
throw new Error("Reviewed identity is already assigned to another target.");
|
|
250
|
+
state.resolutions ??= {};
|
|
251
|
+
state.resolutions[String(index)] = { companyId: identity.companyId, companyName: identity.targetName, canonicalName: identity.canonicalName, evidenceUrl: identity.evidenceUrl };
|
|
252
|
+
}
|
|
220
253
|
const effectiveBrief = () => CompanyBriefSchema.parse({ ...brief, companies: brief.companies.map((company, index) => {
|
|
221
254
|
const resolution = state.resolutions?.[String(index)];
|
|
222
|
-
|
|
255
|
+
const resolved = !company.companyId && resolution ? { ...company, companyId: resolution.companyId } : company;
|
|
256
|
+
const evidence = resolved.companyId ? state.employerEvidence?.[resolved.companyId] : undefined;
|
|
257
|
+
const aliases = [...(resolved.verifiedEmployerAliases ?? []), ...(review?.employerAliases.filter(a => a.companyId === resolved.companyId).map(({ name, evidenceUrl }) => ({ name, evidenceUrl })) ?? [])];
|
|
258
|
+
if (resolution?.canonicalName)
|
|
259
|
+
aliases.push({ name: resolution.canonicalName, evidenceUrl: resolution.evidenceUrl });
|
|
260
|
+
if (evidence && companyIdentityKey(evidence.name) === companyIdentityKey(resolved.name))
|
|
261
|
+
aliases.push({ name: evidence.name, evidenceUrl: evidence.evidenceUrl });
|
|
262
|
+
return { ...resolved, verifiedEmployerAliases: aliases };
|
|
223
263
|
}) });
|
|
224
|
-
const
|
|
264
|
+
const allowed = (name) => !input.onlyCompanies?.length || input.onlyCompanies.includes(name);
|
|
265
|
+
for (const name of input.onlyCompanies ?? [])
|
|
266
|
+
if (!brief.companies.some(c => c.name === name))
|
|
267
|
+
throw new Error(`Unknown target company: ${name}`);
|
|
268
|
+
for (const alias of review?.employerAliases ?? [])
|
|
269
|
+
if (!effectiveBrief().companies.some(c => c.companyId === alias.companyId))
|
|
270
|
+
throw new Error("Reviewed alias targets an unknown company ID.");
|
|
271
|
+
if (input.review) {
|
|
272
|
+
await save(path.join(outDir, "review-decisions.json"), JSON.stringify(review, null, 2));
|
|
273
|
+
await save(checkpoint, JSON.stringify(state));
|
|
274
|
+
}
|
|
275
|
+
const publish = async () => {
|
|
276
|
+
const report = shortlistCompanyLeads(effectiveBrief(), state.results);
|
|
277
|
+
const aliases = new Map();
|
|
278
|
+
for (const row of report.review.filter(r => r.reason === "employer_name_differs")) {
|
|
279
|
+
const id = String(row.companyId), name = String(row.sourceCompanyName);
|
|
280
|
+
aliases.set(`${id}:${name}`, { companyId: id, targetName: String(row.companyName), name, evidenceUrl: `https://www.linkedin.com/sales/company/${id}`, status: "needs_verification" });
|
|
281
|
+
}
|
|
282
|
+
await save(path.join(outDir, "contacts.csv"), companyLeadCsv(report.selected));
|
|
283
|
+
await save(path.join(outDir, "review.csv"), companyLeadCsv(report.review));
|
|
284
|
+
await save(path.join(outDir, "coverage.json"), JSON.stringify(report, null, 2));
|
|
285
|
+
await save(path.join(outDir, "company-resolutions.json"), JSON.stringify(state.resolutions ?? {}, null, 2));
|
|
286
|
+
await save(path.join(outDir, "recovery.json"), JSON.stringify({ fingerprint, aliases: [...aliases.values()], identities: brief.companies.flatMap((c, i) => !c.companyId && !state.resolutions?.[String(i)] ? [{ targetName: c.name, diagnostic: state.diagnostics?.[String(i)] ?? { reason: "legacy_unknown" } }] : []), reviewFileTemplate: { fingerprint, employerAliases: [], identities: [] } }, null, 2));
|
|
287
|
+
return report;
|
|
288
|
+
};
|
|
225
289
|
let performed = 0;
|
|
226
290
|
try {
|
|
291
|
+
if (input.verifyEmployer)
|
|
292
|
+
for (const company of effectiveBrief().companies) {
|
|
293
|
+
if (!company.companyId || !allowed(company.name) || state.employerEvidence?.[company.companyId])
|
|
294
|
+
continue;
|
|
295
|
+
if (!Object.values(state.results).some(result => result.people.some(p => String(p.companyId) === company.companyId)))
|
|
296
|
+
continue;
|
|
297
|
+
const evidence = await input.verifyEmployer(company.companyId);
|
|
298
|
+
if (evidence?.companyId === company.companyId) {
|
|
299
|
+
state.employerEvidence ??= {};
|
|
300
|
+
state.employerEvidence[company.companyId] = evidence;
|
|
301
|
+
await save(checkpoint, JSON.stringify(state));
|
|
302
|
+
await publish();
|
|
303
|
+
}
|
|
304
|
+
}
|
|
227
305
|
// Known IDs run first; an unresolved identity cannot block already prepared work.
|
|
228
306
|
const collect = async () => {
|
|
229
307
|
for (const job of planCompanySearches(effectiveBrief())) {
|
|
230
|
-
if (
|
|
308
|
+
if (!allowed(job.company.name))
|
|
309
|
+
continue;
|
|
310
|
+
const previous = state.results[job.key];
|
|
311
|
+
const defaultBudgetFinished = previous && (previous.nextOffset == null || previous.nextOffset >= brief.candidatesPerDepartment);
|
|
312
|
+
if (previous && (!input.continuePartial && defaultBudgetFinished || previous.totalResults != null && previous.people.length >= previous.totalResults || previous.stoppedReason))
|
|
313
|
+
continue;
|
|
314
|
+
if (previous && input.continuePartial === "quota" && shortlistCompanyLeads(effectiveBrief(), state.results).coverage.find(c => c.companyId === job.company.companyId).selected >= job.company.maxContacts)
|
|
231
315
|
continue;
|
|
232
316
|
if (performed >= input.maxSearches)
|
|
233
317
|
break;
|
|
234
|
-
state.results[job.key] = await input.search(job);
|
|
235
318
|
performed++;
|
|
319
|
+
if (input.searchPage) {
|
|
320
|
+
const result = previous ? { ...previous, people: [...previous.people] } : { people: [], totalResults: null, fetchedPages: 0, nextOffset: 0 };
|
|
321
|
+
// Old checkpoints lack a reliable raw offset; replay the first page once and deduplicate.
|
|
322
|
+
let start = result.nextOffset ?? 0;
|
|
323
|
+
const seen = new Set(result.people.map(p => canonicalProfile(p.profileUrl)));
|
|
324
|
+
const target = input.continuePartial ? 2500 : brief.candidatesPerDepartment;
|
|
325
|
+
while (start < target) {
|
|
326
|
+
const replayingLegacyFirstPage = previous != null && previous.nextOffset == null && start === 0;
|
|
327
|
+
const count = Math.min(100, target - start);
|
|
328
|
+
const page = await input.searchPage(job, start, count);
|
|
329
|
+
result.fetchedPages++;
|
|
330
|
+
if (result.totalResults != null && page.totalResults !== result.totalResults)
|
|
331
|
+
result.stoppedReason = "reported_total_changed";
|
|
332
|
+
result.totalResults ??= page.totalResults;
|
|
333
|
+
let added = 0;
|
|
334
|
+
for (const person of page.people) {
|
|
335
|
+
const key = canonicalProfile(person.profileUrl);
|
|
336
|
+
if (key && !seen.has(key)) {
|
|
337
|
+
seen.add(key);
|
|
338
|
+
result.people.push(person);
|
|
339
|
+
added++;
|
|
340
|
+
}
|
|
341
|
+
}
|
|
342
|
+
start += page.rawCount;
|
|
343
|
+
result.nextOffset = start;
|
|
344
|
+
if ((page.rawCount === 0 || !added && !replayingLegacyFirstPage) && (result.totalResults == null || result.people.length < result.totalResults))
|
|
345
|
+
result.stoppedReason = "empty_or_duplicate_page";
|
|
346
|
+
state.results[job.key] = result;
|
|
347
|
+
await save(checkpoint, JSON.stringify(state));
|
|
348
|
+
await publish();
|
|
349
|
+
if (result.stoppedReason || result.totalResults != null && start >= result.totalResults)
|
|
350
|
+
break;
|
|
351
|
+
if (input.continuePartial === "quota" && shortlistCompanyLeads(effectiveBrief(), state.results).coverage.find(c => c.companyId === job.company.companyId).selected >= job.company.maxContacts)
|
|
352
|
+
break;
|
|
353
|
+
}
|
|
354
|
+
}
|
|
355
|
+
else
|
|
356
|
+
state.results[job.key] = await input.search(job);
|
|
236
357
|
await save(checkpoint, JSON.stringify(state));
|
|
237
358
|
await publish();
|
|
238
359
|
}
|
|
239
360
|
};
|
|
240
361
|
await collect();
|
|
241
|
-
if (input.resolveCompany) {
|
|
362
|
+
if (input.resolveCompany || input.diagnoseCompany) {
|
|
242
363
|
let resolutionsPerformed = 0;
|
|
243
364
|
for (const [index, company] of brief.companies.entries()) {
|
|
244
365
|
if (performed >= input.maxSearches || resolutionsPerformed >= input.maxSearches)
|
|
245
366
|
break;
|
|
246
367
|
const savedResolution = state.resolutions?.[String(index)];
|
|
247
|
-
if (company.companyId || (Object.hasOwn(state.resolutions ?? {}, String(index)) && !(input.retryUnresolved && savedResolution === null)))
|
|
368
|
+
if (!allowed(company.name) || company.companyId || (Object.hasOwn(state.resolutions ?? {}, String(index)) && !(input.retryUnresolved && savedResolution === null)))
|
|
248
369
|
continue;
|
|
249
|
-
const
|
|
370
|
+
const diagnosed = input.diagnoseCompany ? await input.diagnoseCompany(company) : undefined;
|
|
371
|
+
const resolution = diagnosed ? diagnosed.resolution : await input.resolveCompany(company);
|
|
372
|
+
if (diagnosed) {
|
|
373
|
+
state.diagnostics ??= {};
|
|
374
|
+
state.diagnostics[String(index)] = diagnosed.diagnostic;
|
|
375
|
+
}
|
|
250
376
|
resolutionsPerformed++;
|
|
251
377
|
if (resolution && (!/^[1-9]\d*$/.test(resolution.companyId) || words(resolution.companyName) !== words(company.name)))
|
|
252
378
|
throw new Error("Company resolver returned a non-exact identity.");
|
|
253
379
|
const occupied = new Set(effectiveBrief().companies.map(c => c.companyId).filter(Boolean));
|
|
254
380
|
state.resolutions ??= {};
|
|
255
381
|
state.resolutions[String(index)] = resolution && !occupied.has(resolution.companyId) ? resolution : null;
|
|
382
|
+
if (resolution && occupied.has(resolution.companyId) && state.diagnostics?.[String(index)])
|
|
383
|
+
state.diagnostics[String(index)].reason = "shared_identity";
|
|
256
384
|
await save(checkpoint, JSON.stringify(state));
|
|
257
385
|
await publish();
|
|
258
386
|
await collect();
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
import { z } from "zod";
|
|
2
|
+
// Only presentation annotations and legal suffixes are interchangeable. Group,
|
|
3
|
+
// division, brand and subsidiary names remain different identities.
|
|
4
|
+
export function companyIdentityKey(name) {
|
|
5
|
+
let key = name.replace(/\s*\((?:CH|DE|AT|UK|US|USA)\)\s*$/i, "")
|
|
6
|
+
.normalize("NFKD").replace(/[\u0300-\u036f]/g, "").replace(/ß/g, "ss").toLowerCase()
|
|
7
|
+
.replace(/&/g, " and ").replace(/[^a-z0-9]+/g, " ")
|
|
8
|
+
.replace(/\s+/g, " ").trim();
|
|
9
|
+
// Do not erase a brand token in names such as AG Insurance or SE Ranking.
|
|
10
|
+
while (/\s(?:gmbh|kgaa|ag|se|ltd|limited|inc|incorporated|llc)$/.test(key))
|
|
11
|
+
key = key.replace(/\s\S+$/, "");
|
|
12
|
+
return key;
|
|
13
|
+
}
|
|
14
|
+
export function companyQueryVariants(name) {
|
|
15
|
+
const plain = name.replace(/\s*\([^)]*\)\s*/g, " ").trim();
|
|
16
|
+
const primary = plain.split(/\s+\/\s+/)[0].trim();
|
|
17
|
+
return [...new Set([name, plain, primary, companyIdentityKey(primary)].filter(Boolean))].slice(0, 3);
|
|
18
|
+
}
|
|
19
|
+
export async function resolveCompanyIdentity(name, search, details) {
|
|
20
|
+
const diagnostic = { reason: "no_exact_match", attempts: [], candidates: [], observedAt: new Date().toISOString() };
|
|
21
|
+
const candidates = new Map();
|
|
22
|
+
const proven = new Map();
|
|
23
|
+
let incomplete = false, oversized = false;
|
|
24
|
+
for (const query of companyQueryVariants(name)) {
|
|
25
|
+
const first = await search(query, 0);
|
|
26
|
+
let count = first.rawCount;
|
|
27
|
+
const total = first.total;
|
|
28
|
+
const queryCandidates = new Map(first.candidates.map(c => [c.companyId, c]));
|
|
29
|
+
let complete = total != null && total <= 1000;
|
|
30
|
+
for (const c of first.candidates)
|
|
31
|
+
candidates.set(c.companyId, c);
|
|
32
|
+
if (complete)
|
|
33
|
+
for (let start = 25; start < total; start += 25) {
|
|
34
|
+
const page = await search(query, start);
|
|
35
|
+
if (page.total !== total || page.rawCount === 0) {
|
|
36
|
+
complete = false;
|
|
37
|
+
break;
|
|
38
|
+
}
|
|
39
|
+
count += page.rawCount;
|
|
40
|
+
for (const c of page.candidates) {
|
|
41
|
+
candidates.set(c.companyId, c);
|
|
42
|
+
queryCandidates.set(c.companyId, c);
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
complete = complete && count >= total && queryCandidates.size >= total;
|
|
46
|
+
diagnostic.attempts.push({ query, total, collected: count, complete });
|
|
47
|
+
incomplete ||= !complete;
|
|
48
|
+
oversized ||= total != null && total > 1000;
|
|
49
|
+
// A complete exact query is sufficient; avoid repeating equivalent searches.
|
|
50
|
+
if (complete)
|
|
51
|
+
for (const c of queryCandidates.values())
|
|
52
|
+
proven.set(c.companyId, c);
|
|
53
|
+
if (complete && [...queryCandidates.values()].some(c => companyIdentityKey(c.name) === companyIdentityKey(name))) {
|
|
54
|
+
incomplete = false;
|
|
55
|
+
oversized = false;
|
|
56
|
+
break;
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
diagnostic.candidates = [...candidates.values()].slice(0, 100);
|
|
60
|
+
const exact = [...proven.values()].filter(c => companyIdentityKey(c.name) === companyIdentityKey(name));
|
|
61
|
+
if (/\/|\([^)]*(?:\/|,)[^)]*\)/.test(name))
|
|
62
|
+
diagnostic.reason = "entity_decision_required";
|
|
63
|
+
else if (oversized)
|
|
64
|
+
diagnostic.reason = "oversized";
|
|
65
|
+
else if (incomplete)
|
|
66
|
+
diagnostic.reason = "incomplete";
|
|
67
|
+
else if (exact.length > 1)
|
|
68
|
+
diagnostic.reason = "ambiguous";
|
|
69
|
+
else if (exact.length === 1) {
|
|
70
|
+
const evidence = await details(exact[0].companyId);
|
|
71
|
+
if (evidence?.companyId === exact[0].companyId && companyIdentityKey(evidence.name) === companyIdentityKey(name)) {
|
|
72
|
+
diagnostic.reason = "resolved";
|
|
73
|
+
return { resolution: { companyId: evidence.companyId, companyName: name, canonicalName: evidence.name, evidenceUrl: evidence.evidenceUrl }, diagnostic };
|
|
74
|
+
}
|
|
75
|
+
diagnostic.reason = "detail_mismatch";
|
|
76
|
+
}
|
|
77
|
+
return { resolution: null, diagnostic };
|
|
78
|
+
}
|
|
79
|
+
const evidenceUrl = z.url().refine(value => /^https:\/\//i.test(value), "Evidence must use HTTPS");
|
|
80
|
+
export const CompanyReviewSchema = z.object({
|
|
81
|
+
fingerprint: z.string().regex(/^[a-f0-9]{64}$/),
|
|
82
|
+
employerAliases: z.array(z.object({ companyId: z.string().regex(/^[1-9]\d*$/), name: z.string().trim().min(1), evidenceUrl }).strict()).default([]),
|
|
83
|
+
identities: z.array(z.object({ targetName: z.string().trim().min(1), companyId: z.string().regex(/^[1-9]\d*$/), canonicalName: z.string().trim().min(1), evidenceUrl }).strict()).default([]),
|
|
84
|
+
}).strict();
|
package/package.json
CHANGED