fullstackgtm 0.21.2 → 0.23.1

package/CHANGELOG.md

@@ -5,6 +5,86 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and the project adheres to [Semantic Versioning](https://semver.org/).
 The path to 1.0 is planned in [docs/roadmap-to-1.0.md](./docs/roadmap-to-1.0.md).
 
+## [0.23.1] — 2026-06-12
+
+### Fixed
+
+- **Literal `${vendorHeads}` leaked into expanded claim-group tables** — a
+  templating slip in the 0.22.0 narrative restructure left the vendor header
+  row unrendered (and masked a use-before-declaration). Fixed, plus a
+  regression guard: the HTML must contain no unrendered `${` placeholders
+  and expanded groups must carry real vendor headers.
+- Claims section heading renamed to "Market Claims".
+
+## [0.23.0] — 2026-06-12
+
+### Added
+
+- **The enrich layer (MVP)** — governed append/refresh of third-party data
+  into the CRM (spec: docs/enrich.md in the monorepo). Where every enrichment
+  vendor ships fire-and-forget writeback, enrich emits a diff you approve:
+  source data → deterministic matcher → fill-blanks patch plan → the
+  existing plans approve → apply chain, with the source payload stored as
+  `GtmEvidence` on the plan and `beforeValue` set on every operation for
+  apply-time compare-and-set.
+  - `enrich append [--source <id>] [--objects companies,contacts] [--save] [--config <path>]`
+    — pull (Apollo) or read staged ingest data (Clay), match via ordered
+    keys (unique-hit-wins, zero-hits-next-key, multi-hit → `onAmbiguous`
+    skip-with-candidates-recorded or `requires_human_record_selection`
+    placeholders into the suggest chain), emit a fill-blanks-only plan.
+    Without `--save`: dry-run diff, nothing written.
+  - `enrich refresh [--source <id>] [--stale-days <n>] [--save]` — work set
+    from run-store stamps older than the staleness window (per-field
+    `staleDays` → `policy.defaultStaleDays` → 90); operations only where the
+    source value actually changed, and only on fields the ledger proves
+    enrich stamped.
+  - `enrich ingest <file.csv|payload.json> --source <id> [--run-label <l>]`
+    — stage Clay CSV exports (dependency-free CSV parser) or webhook payload
+    JSON for a subsequent append/refresh.
+  - `enrich status [--runs] [--source <id>]` — last run per source, counts,
+    staleness distribution, interrupted-run cursor.
+  - `enrich.config.json` (sources / ordered match keys / field mappings /
+    policy) with strict up-front validation; the `system-only` and `always`
+    conflict-ladder rungs error as "not yet implemented (phase 2)" instead
+    of being silently accepted. MVP policy: `never` (fill blanks only).
+  - Apollo source client: raw `fetch` against the people-match /
+    organization-enrich endpoints, BYO key via `login apollo` (0600 cred
+    store) or `APOLLO_API_KEY`, and 429-aware retry with capped exponential
+    backoff honoring `Retry-After` — local to the Apollo client; the shared
+    connector contract is unchanged.
+  - Profile-scoped append-only run store
+    (`~/.fullstackgtm/profiles/<p>/enrich/runs/<runLabel>.json`): resume
+    checkpoint (cursor + already-paid-for payloads), per-record/per-field
+    `enrichedAt` staleness ledger, and the surface `enrich status` reads.
+    State stays local — no `fsgtm_enriched_at`-style properties are written
+    into the customer's portal.
+  - Every `enrich` subcommand catches `--help`/`-h` before config load,
+    credential resolution, or any network call. No scheduling/cron logic —
+    that is the horizontal schedule layer's job (docs/schedule.md).
+
+## [0.22.0] — 2026-06-12
+
+The report becomes a narrative: map → claims → where to attack.
+
+### Changed
+
+- **Reading order rebuilt around the insight.** The strategic map is now the
+  hero, directly under the header: contenders and their positions first.
+  The claim detail follows, then the report CLOSES on the reasoned takeaway.
+  The front-summary stat cards are gone (numbers without referents).
+- **"Where to attack"** — a generated closing section that walks the open
+  fronts as an argument: each open claim names its closest quiet contenders,
+  whether the anchor already ships it quietly (promote candidate) or it's
+  unclaimed (first-mover), plus held ground (anchor-owned fronts to defend)
+  and crowded ground (saturated fronts where message budget buys least).
+  Ends by pointing at `market overlay` for evidence-backed directives.
+- **Claims grouped and collapsed**: the matrix splits into Open / Contested /
+  Owned / Saturated `<details>` groups, default collapsed, each summary
+  carrying the skimmer's stats (count, definition of the state, anchor's
+  loud count within the group).
+- **Evidence appendix grouped by vendor**, collapsed — receipts on demand.
+  All groups auto-expand on print (beforeprint).
+
 ## [0.21.2] — 2026-06-12
 
 Scatter interactivity + honest sizing fallbacks.

package/README.md

@@ -126,6 +126,21 @@ The discipline matches the rest of the tool. Intensity readings are *proposals*
 
 `market axes` is for earning a strategic 2×2 instead of asserting one: PCA over the intensity matrix (PC1 = the category's own primary axis, PC2 = the most differentiating direction orthogonal to it), triangulation of your configured axes against the data, and an orthogonality screen that flags two axes that are secretly one. Axes are claim-scoring rubrics in the config; the report renders the primary pair as the strategic map. Captures and observations are profile-scoped (`~/.fullstackgtm/market/<category>`), so one client's category intel never bleeds into another's.
 
+## Governed enrichment: a diff you approve before third-party data touches your CRM
+
+Every enrichment vendor ships fire-and-forget writeback. The **enrich layer** inverts that: declare once which fields come from which source under which conflict policy (`enrich.config.json` — sources, ordered match keys, field mappings, policy), then `enrich append` fills the gaps and `enrich refresh` keeps them current — with every write passing through the normal dry-run → approval → apply contract, and every value traceable to the source payload that produced it.
+
+```bash
+echo "$APOLLO_API_KEY" | fullstackgtm login apollo          # BYO key, stored 0600
+fullstackgtm enrich append --provider hubspot               # pull → match → dry-run diff, writes NOTHING
+fullstackgtm enrich append --provider hubspot --save        # persist the plan (needs_approval) + run record
+fullstackgtm enrich ingest clay-export.csv --source clay    # stage a push-style source (Clay CSV / webhook JSON)
+fullstackgtm enrich refresh --source apollo --save          # re-check stale stamped fields; ops only where the source changed
+fullstackgtm enrich status --runs                           # last run per source, counts, staleness, interrupted-run cursor
+```
+
+Matching is deterministic: ordered keys, unique hit wins, zero hits falls through to the next key, and multiple hits are never guessed away — they skip (recorded with candidate ids) or flow into the existing `suggest` → `plans approve` chain as `requires_human_record_selection` placeholders. The MVP conflict policy is `never`: enrich only fills blank fields, and `refresh` only re-touches fields its own run-store ledger proves it stamped (per-record/per-field `enrichedAt`, profile-scoped, never written into your portal as custom properties). The `system-only` and `always` rungs of the ladder are phase 2 and are refused explicitly, not silently accepted. Recurring execution belongs to the scheduler — enrich owns no cron logic.
+
 ### Working across organizations
 
 Consultants and fractional operators hold credentials for several CRMs at once. A profile scopes stored logins *and* stored plans to one organization:

package/dist/bulkUpdate.d.ts

@@ -3,10 +3,23 @@ export type BulkUpdateOptions = {
     objectType: "account" | "contact" | "deal";
     /** raw --where expressions, AND-ed together; at least one is required */
     where: string[];
-    /** canonical field → new value; one action only */
+    /**
+     * canonical field → new value; one action only. A value of the form
+     * `from:<sourceField>` is resolved PER RECORD from the filter view at
+     * plan time (relational pseudo-fields like account.ownerId included);
+     * records whose source value is empty are skipped, not failed, and
+     * counted in the plan summary.
+     */
     set?: Record<string, string>;
     /** propose archive_record instead of field writes */
     archive?: boolean;
+    /**
+     * bypass the archive duplicate guard: by default --archive refuses when a
+     * matched account/contact shares its identity key (normalized domain /
+     * lowercased email) with another record — those are duplicates, and
+     * archiving a duplicate discards its data where merging preserves it
+     */
+    forceArchiveDuplicates?: boolean;
     /** propose create_task on each matched record with this subject/body text */
     createTask?: string;
     /** explicit preconditions (field=value), re-verified at apply time */
@@ -28,6 +41,8 @@ type WhereClause = {
     raw: string;
 };
 export declare function parseWhere(expr: string): WhereClause;
+/** True when `field` is filterable for this object type (relational pseudo-fields included). */
+export declare function isFilterableField(objectType: BulkUpdateOptions["objectType"], field: string): boolean;
 export declare function parseGuard(raw: string): PlanGuard;
 /** Ids of records matching a filter — used for apply-time filter re-verification. */
 export declare function eligibleIds(snapshot: CanonicalGtmSnapshot, objectType: BulkUpdateOptions["objectType"], where: string[]): Set<string>;

package/dist/bulkUpdate.js

@@ -27,6 +27,7 @@
  * `account.ownerId`, `account.contactCount`; accounts get `contactCount`
  * and `openDealCount`.
  */
+import { normalizeDomain } from "./merge.js";
 import { stableHash } from "./rules.js";
 const FIELD_PATTERN = "[a-zA-Z][a-zA-Z0-9_]*(?:\\.[a-zA-Z][a-zA-Z0-9_]*)?";
 export function parseWhere(expr) {
@@ -91,6 +92,10 @@ const VALID_FIELDS = {
     contact: new Set(["id", "crmId", "accountId", "firstName", "lastName", "email", "phone", "title", "ownerId", "lastSyncAt", ...RELATIONAL_FIELDS]),
     deal: new Set(["id", "crmId", "accountId", "ownerId", "name", "amount", "currency", "stage", "closeDate", "dealType", "forecastCategory", "nextStep", "probability", "isClosed", "isWon", "lastActivityAt", "lastSyncAt", ...RELATIONAL_FIELDS]),
 };
+/** True when `field` is filterable for this object type (relational pseudo-fields included). */
+export function isFilterableField(objectType, field) {
+    return VALID_FIELDS[objectType].has(field);
+}
 function assertValidFields(objectType, clauses, context) {
     for (const clause of clauses) {
         if (!VALID_FIELDS[objectType].has(clause.field)) {
@@ -192,16 +197,66 @@ export function buildBulkUpdatePlan(snapshot, options) {
     const clauses = options.where.map(parseWhere);
     assertValidFields(options.objectType, clauses, "--where");
     const WRITABLE_BLOCKLIST = new Set(["id", "crmId", "contactCount", "openDealCount", "openDealStages"]);
-    for (const field of Object.keys(options.set ?? {})) {
+    // `from:<sourceField>` values resolve per record from the filter view —
+    // the source is validated with the same strictness as filters (relational
+    // pseudo-fields allowed; the WRITTEN field still must be canonical).
+    const assignments = [];
+    for (const [field, value] of Object.entries(options.set ?? {})) {
         if (!VALID_FIELDS[options.objectType].has(field) || WRITABLE_BLOCKLIST.has(field) || field.includes(".")) {
             throw new Error(`Cannot --set "${field}" on ${options.objectType}s — not a writable canonical field.`);
         }
+        if (value.startsWith("from:")) {
+            const fromField = value.slice("from:".length);
+            if (!VALID_FIELDS[options.objectType].has(fromField)) {
+                throw new Error(`Cannot --set ${field}=from:${fromField} on ${options.objectType}s — unknown source field "${fromField}". Valid fields: ${[...VALID_FIELDS[options.objectType]].join(", ")}.`);
+            }
+            assignments.push({ field, fromField });
+        }
+        else {
+            assignments.push({ field, literal: value });
+        }
     }
     const views = buildViews(snapshot, options.objectType);
     const matched = views.filter(({ view }) => clauses.every((c) => matches(view, c)));
     if (matched.length > maxOperations) {
         throw new Error(`Filter matched ${matched.length} ${COLLECTIONS[options.objectType]} — above the ${maxOperations}-record safety cap. Narrow the --where filter or raise --max-operations explicitly.`);
     }
+    // Archive duplicate guard: archiving a record that shares its identity key
+    // with another active record discards data a merge would preserve. Refuse
+    // and point at `dedupe` unless explicitly overridden. Deals are exempt —
+    // they carry no identity key.
+    if (options.archive && options.objectType !== "deal" && !options.forceArchiveDuplicates) {
+        const keyName = options.objectType === "account" ? "domain" : "email";
+        const keyOf = (record) => options.objectType === "account"
+            ? normalizeDomain(record.domain)
+            : (record.email?.trim().toLowerCase() || undefined);
+        const allRecords = snapshot[COLLECTIONS[options.objectType]];
+        const byKey = new Map();
+        for (const record of allRecords) {
+            const key = keyOf(record);
+            if (!key)
+                continue;
+            const existing = byKey.get(key) ?? [];
+            existing.push(record);
+            byKey.set(key, existing);
+        }
+        const collisions = [];
+        for (const { record } of matched) {
+            const key = keyOf(record);
+            if (!key)
+                continue;
+            const others = (byKey.get(key) ?? []).filter((other) => other.id !== record.id);
+            if (others.length === 0)
+                continue;
+            const label = record.name ?? record.email ?? "";
+            collisions.push(`${options.objectType} ${record.id}${label ? ` "${label}"` : ""} shares ${keyName} "${key}" with ${others
+                .map((other) => `${other.id}${other.name ? ` "${other.name}"` : ""}`)
+                .join(", ")}`);
+        }
+        if (collisions.length > 0) {
+            throw new Error(`Refusing to archive: ${collisions.length} matched record(s) look like duplicates of other records — archiving a duplicate DISCARDS its data, merging preserves it. Use \`fullstackgtm dedupe ${options.objectType} --key ${keyName}\` (merge_records) instead, or pass --force-archive-duplicates to archive anyway.\n  - ${collisions.join("\n  - ")}`);
+        }
+    }
     // Preconditions: explicit --require, plus every equality filter on a real
     // (re-readable, non-relational) field. The premise the plan was built on
     // is re-verified per record at apply time.
@@ -236,7 +291,9 @@ export function buildBulkUpdatePlan(snapshot, options) {
             : `set ${Object.entries(options.set).map(([k, v]) => `${k}=${v}`).join(", ")}`;
     const reason = options.reason ?? `bulk-update: ${action} where ${whereText}`;
     const operations = [];
-    for (const { record } of matched) {
+    // records skipped because a from:<sourceField> value was empty, per source
+    const skippedBySource = new Map();
+    for (const { record, view } of matched) {
         const objectId = String(record.id);
         const groupId = `grp_${options.objectType}_${objectId}`;
         const preconditions = preconditionSpecs.map((p) => ({
@@ -279,7 +336,30 @@ export function buildBulkUpdatePlan(snapshot, options) {
             });
             continue;
         }
-        for (const [field, value] of Object.entries(options.set)) {
+        // Resolve every assignment for this record BEFORE emitting any of its
+        // operations: a record whose from:<sourceField> resolves empty is
+        // skipped whole (its operations share a groupId — half a record's
+        // updates is exactly what grouping exists to prevent).
+        const resolved = [];
+        let emptySource = null;
+        for (const assignment of assignments) {
+            if (assignment.fromField !== undefined) {
+                const value = fieldValue(view, assignment.fromField);
+                if (value === "") {
+                    emptySource = assignment.fromField;
+                    break;
+                }
+                resolved.push({ field: assignment.field, value });
+            }
+            else {
+                resolved.push({ field: assignment.field, value: assignment.literal });
+            }
+        }
+        if (emptySource !== null) {
+            skippedBySource.set(emptySource, (skippedBySource.get(emptySource) ?? 0) + 1);
+            continue;
+        }
+        for (const { field, value } of resolved) {
             operations.push({
                 ...shared,
                 id: `op_${stableHash(`bulk-set:${options.objectType}:${objectId}:${field}:${value}`)}`,
@@ -300,13 +380,16 @@ export function buildBulkUpdatePlan(snapshot, options) {
         if (failure)
             throw new Error(`${failure} The guard already fails against the current snapshot — the plan would never apply.`);
     }
+    const skippedText = [...skippedBySource.entries()]
+        .map(([sourceField, count]) => ` ${count} skipped: empty ${sourceField}.`)
+        .join("");
     return {
-        id: `patch_plan_${stableHash(`bulk:${snapshot.provider}:${snapshot.generatedAt}:${whereText}:${action}:${operations.length}`)}`,
+        id: `patch_plan_${stableHash(`bulk:${snapshot.provider}:${snapshot.generatedAt}:${options.objectType}:${whereText}:${action}:${operations.length}`)}`,
         title: `Bulk update: ${options.objectType}s where ${whereText}`,
         createdAt: snapshot.generatedAt,
         status: operations.length > 0 ? "needs_approval" : "draft",
         dryRun: true,
-        summary: `${matched.length} ${COLLECTIONS[options.objectType]} matched (${whereText}); ${operations.length} proposed dry-run operations (${action}).${guards.length > 0 ? ` ${guards.length} apply-time guard(s).` : ""}`,
+        summary: `${matched.length} ${COLLECTIONS[options.objectType]} matched (${whereText}); ${operations.length} proposed dry-run operations (${action}).${skippedText}${guards.length > 0 ? ` ${guards.length} apply-time guard(s).` : ""}`,
         findings: [],
         operations,
         filter: { objectType: options.objectType, where: options.where },