@installagent/sdk 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/dist/agent-templates.d.ts +36 -2
  2. package/dist/agent-templates.js +173 -3
  3. package/dist/client-linear-create.d.ts +26 -0
  4. package/dist/client-linear-create.js +46 -0
  5. package/dist/client-linear.d.ts +28 -9
  6. package/dist/client-linear.js +36 -14
  7. package/dist/docs-search.d.ts +19 -6
  8. package/dist/docs-search.js +32 -11
  9. package/dist/doctor.d.ts +9 -0
  10. package/dist/doctor.js +126 -23
  11. package/dist/env-fill.d.ts +13 -0
  12. package/dist/env-fill.js +34 -9
  13. package/dist/env-push.d.ts +3 -0
  14. package/dist/env-push.js +11 -6
  15. package/dist/env-registry.d.ts +25 -0
  16. package/dist/env-registry.js +69 -7
  17. package/dist/exec.js +15 -2
  18. package/dist/index.d.ts +14 -6
  19. package/dist/index.js +14 -6
  20. package/dist/linear-project.d.ts +2 -2
  21. package/dist/linear-project.js +2 -2
  22. package/dist/log.d.ts +28 -0
  23. package/dist/log.js +73 -0
  24. package/dist/n8n-executions.d.ts +85 -0
  25. package/dist/n8n-executions.js +259 -0
  26. package/dist/op-cli.d.ts +29 -0
  27. package/dist/op-cli.js +52 -0
  28. package/dist/platform.d.ts +35 -0
  29. package/dist/platform.js +39 -0
  30. package/dist/portal-admin.d.ts +40 -0
  31. package/dist/portal-admin.js +30 -0
  32. package/dist/standup.d.ts +88 -0
  33. package/dist/standup.js +235 -0
  34. package/dist/sync-repos.d.ts +47 -0
  35. package/dist/sync-repos.js +80 -0
  36. package/dist/template-sync.d.ts +10 -0
  37. package/dist/template-sync.js +11 -0
  38. package/dist/wizard.d.ts +15 -0
  39. package/dist/wizard.js +35 -56
  40. package/dist/workspace-docs.d.ts +44 -0
  41. package/dist/workspace-docs.js +73 -0
  42. package/package.json +2 -2
  43. package/templates/env.example +33 -23
  44. package/templates/workspace-root/AGENT.md +7 -5
  45. package/templates/workspace-root/ONBOARDING.md +65 -8
  46. package/templates/workspace-root/skills/.github/workflows/pr-to-slack.yml +102 -0
  47. package/templates/workspace-root/skills/CLAUDE.md +24 -6
  48. package/templates/workspace-root/skills/README.md +156 -0
  49. package/templates/workspace-root/skills/backtest-acceptance-gates/SKILL.md +293 -0
  50. package/templates/workspace-root/skills/backtest-acceptance-gates/backtest-acceptance-gates-guide.pdf +0 -0
  51. package/templates/workspace-root/skills/backtest-acceptance-gates/references/evidence-record-template.md +128 -0
  52. package/templates/workspace-root/skills/backtest-acceptance-gates/references/harness-checklist.md +90 -0
  53. package/templates/workspace-root/skills/client-latest/SKILL.md +301 -0
  54. package/templates/workspace-root/skills/client-latest/catchup.command.md +17 -0
  55. package/templates/workspace-root/skills/cloud-devops/SKILL.md +235 -0
  56. package/templates/workspace-root/skills/email-triage-rollout/SKILL.md +181 -0
  57. package/templates/workspace-root/skills/gcloud-cli-setup/SKILL.md +100 -0
  58. package/templates/workspace-root/skills/google-chat-app-setup/SKILL.md +136 -0
  59. package/templates/workspace-root/skills/google-chat-app-setup/google-chat-app-setup-guide.pdf +0 -0
  60. package/templates/workspace-root/skills/house-voice/SKILL.md +80 -4
  61. package/templates/workspace-root/skills/ia-cli/SKILL.md +19 -6
  62. package/templates/workspace-root/skills/installagent-brand-kit/SKILL.md +59 -10
  63. package/templates/workspace-root/skills/installagent-brand-kit/assets/installagent-deck-template.pptx +0 -0
  64. package/templates/workspace-root/skills/installagent-brand-kit/references/deck-guide.md +181 -0
  65. package/templates/workspace-root/skills/installagent-brand-kit/scripts/deck.py +652 -0
  66. package/templates/workspace-root/skills/installagent-brand-kit/scripts/deck_lint.py +201 -0
  67. package/templates/workspace-root/skills/installagent-brand-kit/scripts/logo_raster.py +102 -0
  68. package/templates/workspace-root/skills/installagent-brand-kit/scripts/make_deck_template.py +290 -0
  69. package/templates/workspace-root/skills/salesforce-api-snapshot/SKILL.md +379 -0
  70. package/templates/workspace-root/skills/salesforce-api-snapshot/references/schema-discovery-soql.md +153 -0
  71. package/templates/workspace-root/skills/salesforce-api-snapshot/references/snapshot-storage.md +122 -0
  72. package/templates/workspace-root/skills/salesforce-api-snapshot/salesforce-api-snapshot-guide.pdf +0 -0
  73. package/templates/workspace-root/skills/slack-voice/SKILL.md +158 -0
  74. package/templates/workspace-root/skills/sso-gated-client-app/SKILL.md +337 -0
  75. package/templates/workspace-root/skills/sso-gated-client-app/sso-gated-client-app-guide.pdf +0 -0
  76. package/templates/workspace-root/skills/tech-call-prep/SKILL.md +146 -47
  77. package/templates/workspace-root/skills/vendor-feed-ingestion/SKILL.md +323 -0
  78. package/templates/workspace-root/skills/vendor-feed-ingestion/references/delivery-contract-template.md +110 -0
  79. package/templates/workspace-root/skills/vendor-feed-ingestion/references/feed-guards.md +134 -0
  80. package/templates/workspace-root/skills/vendor-feed-ingestion/vendor-feed-ingestion-guide.pdf +0 -0
@@ -2,39 +2,75 @@
2
2
  name: tech-call-prep
3
3
  description: >-
4
4
  Verify a technician is actually prepared for an upcoming client call and
5
- generate the one-page per-client call sheet they run it from. Reads the
6
- registry (team-clients/clients/[slug]/) and the build repo
5
+ generate the per-client call agenda they run it from — the full
6
+ run-of-show (read-back facts, walkthrough plan, tailored questions,
7
+ access table, expectations, close checklist), not just a question list.
8
+ Reads the registry (team-clients/clients/[slug]/) and the build repo
7
9
  (clients/client-[slug]/ docs 01–04, onboarding.html), checks the
8
- come-prepared checklist (repo scaffolded, engagement docs drafted, proposals
9
- loaded), and writes docs/05-call-sheet.md with tailored questions, access
10
- gaps, and proposal talking points. Use this WHENEVER a discovery/scoping
11
- call is coming up, or someone says "prep me for the call with [client]",
12
- "generate the call sheet", "am I ready for the [client] call?", or you're
10
+ come-prepared checklist (repo scaffolded, engagement docs drafted,
11
+ proposals loaded), pulls in the auto-drafted agenda Doc at the root of
12
+ the client's Drive folder when one exists (the portal Lead Close Kickoff
13
+ creates it when sales closes the lead), and writes docs/05-call-agenda.md
14
+ (superseding the old 05-call-sheet.md) plus a Google Doc copy at the
15
+ root of the client's Drive folder. Use this WHENEVER a discovery/scoping call is coming
16
+ up, or someone says "prep me for the call with [client]", "generate the
17
+ call agenda / call sheet", "am I ready for the [client] call?", or you're
13
18
  working in a client-[slug]/ repo before a call. Reach for it even if the
14
19
  user never says "skill" — turning the registry profile + call history +
15
- engagement docs into a concrete plan for the next conversation. The static
16
- call script lives on the team hub (Techs → Discovery call); this skill
17
- produces the per-client half.
20
+ engagement docs into a concrete plan for the next conversation. The
21
+ static call script lives on the team hub (Techs → Discovery call); this
22
+ skill produces the per-client half.
18
23
  ---
19
24
 
20
- # Tech call prep — the per-client call sheet
25
+ # Tech call prep — the per-client call agenda
21
26
 
22
27
  The call script is static and lives on the team hub (`/docs/technicians/
23
- discovery-call`). This skill produces the other half: the **call sheet** —
24
- one page, specific to this client and this call, that the technician
25
- actually runs from. It also refuses to produce one if the prep behind it
26
- hasn't happened, because a call sheet over an empty repo is theater.
28
+ client-engagement/discovery-call`). This skill produces the other half: the
29
+ **call agenda** — the run-of-show, specific to this client and this call,
30
+ that the technician actually runs from. It also refuses to produce one if
31
+ the prep behind it hasn't happened, because an agenda over an empty repo is
32
+ theater.
33
+
34
+ The agenda earns its keep only if it's **on the table during the call** —
35
+ transcript evidence shows a written agenda that stayed closed while the two
36
+ blocks that most needed it (access audit, decisions) got dropped. Print it
37
+ or keep it on a second screen.
27
38
 
28
39
  ## Inputs
29
40
 
30
41
  - **Client slug** (required). Registry: `team-clients/clients/<slug>/`.
31
42
  Build repo: `clients/client-<slug>/`.
32
43
  - **Call type** (default: tech discovery call). Also works for monthly
33
- scoping calls — same sheet, shorter question list, current-month
34
- delivery status added.
44
+ scoping calls — same structure, shorter question list, current-month
45
+ delivery status added — and per-stakeholder calls (an operator/SME
46
+ rather than the buyer: keep the read-back, drop commercials entirely,
47
+ add "quantify so I can reprioritize the roadmap" questions).
35
48
 
36
49
  ## Workflow
37
50
 
51
+ ### 0. Get the calls from Google Calendar first — it is the source of truth
52
+
53
+ Before reading anything else, query **`angello@installagent.com`'s Google
54
+ Calendar across the surrounding week** (not just the target day). Nothing else
55
+ in this workspace reliably knows what is booked:
56
+
57
+ - **Fathom cannot prove a call didn't happen** — it only shows calls that were
58
+ held *and* recorded. Use it for what was *said*, never for what exists.
59
+ - **Client repos and Linear lag reality.** Calls booked through the GHL
60
+ booking widget (invite description contains `msgsndr.com/widget/booking`)
61
+ never reach a repo until someone writes them down by hand.
62
+ - A single-day query hides the shape of the week. Stakeholder rounds,
63
+ back-to-back calls that feed each other, and follow-ups all matter to the
64
+ agenda.
65
+
66
+ Check each attendee's email domain against the registry `PROFILE.md`. **An
67
+ attendee who appears nowhere in the repo is a prep gap, not a detail** — that
68
+ is a stakeholder you are about to meet cold.
69
+
70
+ Also read `created`/`updated` on events: a replacement event created shortly
71
+ after a missed slot means the earlier call was *rescheduled*, not skipped, and
72
+ its agenda carries forward intact.
73
+
38
74
  ### 1. Read everything first
39
75
 
40
76
  In order: registry `PROFILE.md` (including Internal notes — that's where
@@ -43,14 +79,24 @@ name mis-transcriptions and discrepancies live), every file in registry
43
79
  `onboarding.html` (Proposals tab + access table). For a monthly scoping
44
80
  call, add `docs/00-delivery-log.md` and `evidence/`.
45
81
 
82
+ **Check for the auto-drafted agenda.** The portal's Lead Close Kickoff
83
+ (sales runs it the moment a deal closes — team hub → Sales → Lead Close
84
+ Kickoff) drops a `<date>-tech-discovery-agenda` Google Doc at the **root
85
+ of the client's Drive folder**, drafted from the merged registry call
86
+ notes. If one exists, use it as the first draft — but verify every fact in it
87
+ against the registry and the sales-call notes before keeping it. It is a
88
+ draft from a transcript, not a source of truth; transcripts mangle names
89
+ and the model can mis-rank priorities. Never run a call from an unverified
90
+ auto-draft.
91
+
46
92
  Then derive the client's **vertical** from registry `PROFILE.md` §
47
93
  "Industry & business model" (primary industry / NAICS) and look for a
48
94
  matching playbook in
49
95
  `team-web/apps/docs/content/docs/sales/industry-playbooks/` (`ls` the
50
96
  directory; match on vertical, not exact wording — roofing contractor →
51
97
  `roofing.mdx`). If one matches, its default wedge + automation list
52
- feed the sheet's **Vertical context** block. If none matches, that goes
53
- on the sheet as a prep-gap line ("no industry playbook for
98
+ feed the agenda's **Vertical context** block. If none matches, that goes
99
+ on the agenda as a prep-gap line ("no industry playbook for
54
100
  \<vertical\>") — a real signal for the technician, never a silent skip.
55
101
 
56
102
  ### 2. Verify the come-prepared checklist
@@ -63,79 +109,132 @@ Check, honestly, against the build repo:
63
109
  | Engagement docs drafted | `docs/01`–`04` exist and reflect the *latest* registry call, not the template |
64
110
  | Proposals loaded | `onboarding.html` Proposals tab has ≥1 non-placeholder row per pitchable build, each grounded in a registry call file |
65
111
  | Access tracker real | `04-access-checklist.md` statuses are set from calls, not left as template defaults |
112
+ | Sales recording watched | The technician has watched the sales-call recording at 2x — the read-back must be checked against the *tape*, not just notes (wrong-stack and wrong-priority read-backs have blown up calls) |
66
113
 
67
- Anything failing goes into a **PREP GAPS** block at the top of the sheet
114
+ Anything failing goes into a **PREP GAPS** block at the top of the agenda
68
115
  with the fix (usually: run `engagement-spec`, or fill the Proposals tab).
69
116
  If the repo doesn't exist at all, stop and say so — scaffolding comes
70
- first (`client-template`), not a call sheet.
117
+ first (`client-template`), not a call agenda.
71
118
 
72
- ### 3. Write `docs/05-call-sheet.md`
119
+ ### 3. Write `docs/05-call-agenda.md`
73
120
 
74
- One page in the build repo, overwritten per call (it's a prep artifact,
75
- not a record — records are registry call notes). Structure:
121
+ One file in the build repo, overwritten per call (it's a prep artifact,
122
+ not a record — records are registry call notes). If an old
123
+ `05-call-sheet.md` exists from before the agenda format, this supersedes
124
+ it — remove the old file in the same change. Structure:
76
125
 
77
126
  ```markdown
78
- # Call sheet — <Client> — <call type> — <YYYY-MM-DD>
127
+ # Call agenda — <Client> — <call type> — <YYYY-MM-DD>
79
128
 
80
129
  > Generated <date> by tech-call-prep. Script: team hub → Techs → Discovery call.
130
+ > Logistics: designated recorder = <name> (ONE bot only) · red-dot check at
131
+ > client-join · room is hot from link-open.
81
132
 
82
133
  ## Prep gaps <!-- omit section if none -->
83
134
  - [ ] <gap> → <fix>
84
135
 
85
- ## Read-back facts <!-- 5–8 bullets max -->
86
- - <who they are, in one line — verified name spelling>
136
+ ## Read-back (open with this — confirm-and-correct, never "remind me")
137
+ "I've been through your call with <sales rep>, so correct me where I'm wrong:"
87
138
  - <the primary pain, in the client's own words, quoted from a call note>
88
- - <commercial state: trial/signed, what's been paid, per registry + Airtable>
89
- - <landmines: prior failed automation, name confusion, discrepancies>
139
+ - <their real stack, verified against the recording>
140
+ - <volume / trigger / commercial state fact>
141
+ - <landmines: prior failed automation, name spelling, discrepancies>
142
+ "What did I get wrong or miss?" → then the call contract, minute 2:
143
+ "My goal this hour: align on the highest-impact month-one builds and the
144
+ access I need. At the end we book the follow-ups."
90
145
 
91
146
  ## Vertical context <!-- from industry-playbooks/<vertical>.mdx; if no playbook, log in Prep gaps instead -->
92
147
  - Proven wedge: <the playbook's default wedge, one line>
93
148
  - Common automations: <2–3 from the playbook, most relevant to this client first>
149
+ - Prescription ready: <the 60-second "clients like you run…" answer + one reference build>
150
+
151
+ ## Walkthrough plan <!-- block 2 -->
152
+ - Open with: "Walk me through <the process> as if you were doing it for
153
+ yourself — be explicit about what softwares you're using."
154
+ - <which process to walk, who should drive, what artifact to ask them to share>
155
+
156
+ ## Questions to ask <!-- ONLY unanswered ones; use the script's question FORMS -->
157
+ - <question> → fills <02-tech-spec §X / trigger / volume / boundary / …>
94
158
 
95
159
  ## The month-one pitch
96
160
  - Lead: <primary candidate> — because <their stated cost>.
161
+ - Sequencing check: "Is the biggest one the *first* one?" <!-- biggest ≠ first -->
97
162
  - Park for later: <other CANDIDATE rows, one line each>.
98
163
  - Outcome sentence to get agreed: "By <date>, <process> goes from
99
164
  <current cost> to <target state>."
100
-
101
- ## Questions to ask <!-- ONLY unanswered ones -->
102
- - <question> → fills <02-tech-spec §X / trigger / volume / …>
165
+ - Elicit + read back their acceptance criterion: "what would make month
166
+ one worth it to you?"
103
167
 
104
168
  ## Access to chase <!-- from 04-access-checklist.md -->
105
- - <item> — MISSING/REQUESTED — <how they grant it, minimum role>
169
+ - <item> — MISSING/REQUESTED — <how they grant it, minimum role> —
170
+ <LIVE-on-call if grantable in <2 min, else working session>
171
+
172
+ ## Expectations to set
173
+ - Week 1 = permissions ("the more you give me, the more I can give you").
174
+ - <compliance line if regulated — WE raise it: BAA / no data at rest / admin-then-demote>
175
+ - Timeline shape: number + MVP definition + partial hedge.
176
+ - Deliverable: workflow diagram in 1–2 days → agree → only then build.
177
+
178
+ ## Decisions needed
179
+ - <decision> — <who makes it> — <default if unmade>
106
180
 
107
181
  ## Close checklist
182
+ - [ ] Month-one list enumerated OUT LOUD (not "these two")
108
183
  - [ ] Outcome sentence agreed out loud
109
- - [ ] Point of contact named
184
+ - [ ] Client acceptance criterion read back
185
+ - [ ] Checkpoint heard: some version of "based on these expectations, I'm good to move forward"
186
+ - [ ] Every actor named (assistant / approver / operators) + operator call booked
110
187
  - [ ] Working session booked (if any OAuth/2FA items)
111
188
  - [ ] LLM-setup session booked
112
- - [ ] Recap sent same day; docs 01–04 + onboarding.html updated
189
+ - [ ] Recap email same day (backup record); docs 01–04 + onboarding.html updated
113
190
  ```
114
191
 
192
+ ### 4. Copy the agenda to the client's Drive
193
+
194
+ Place a copy of the finished agenda at the **root of the client's Drive
195
+ folder**, named `<YYYY-MM-DD>-tech-discovery-agenda` (as a Google Doc, so
196
+ the technician can read it from any device mid-call). If the kickoff's
197
+ auto-drafted Doc from step 1 exists, update **that Doc in place** rather
198
+ than creating a second one — one agenda per call, everywhere. The Drive
199
+ copy is for running the call; the build-repo file remains the versioned
200
+ source.
201
+
115
202
  ## Quality rules
116
203
 
117
204
  - **Questions come from gaps, not templates.** A question already
118
- answered in a call note never appears on the sheet — the sheet's job
119
- is to prove to the client we listened. Every question names the doc
120
- section its answer fills.
205
+ answered in a call note never appears on the agenda — re-known facts go
206
+ in the read-back as statements to confirm, not questions to re-ask.
207
+ Every question names the doc section its answer fills.
121
208
  - **Facts get verified, not copied.** Transcripts mangle names; the
122
209
  registry's Internal notes section is authoritative for known
123
- discrepancies. Never propagate a mis-transcription onto a sheet the
124
- tech will read out loud.
125
- - **One page.** If it doesn't fit, the read-back facts are too long or
126
- the questions aren't prioritized. The tech glances at this mid-call.
127
- - **No PII beyond what the build repo already allows** — the sheet lives
210
+ discrepancies. Never propagate a mis-transcription onto an agenda the
211
+ tech will read out loud — company spelling included (a bad transcript
212
+ spelling once flowed into an email-address design).
213
+ - **Read-back ≤ 6 bullets, close checklist on the same page.** The agenda
214
+ may run two pages; the two blocks the tech needs mid-call (read-back,
215
+ close) must each be glanceable.
216
+ - **Commercials stay out.** The agenda never carries pricing or billing
217
+ mechanics — the tech call converts price into believed scope, it never
218
+ renegotiates price. If a pricing question is expected, the agenda's
219
+ Decisions block routes it to sales/proposal.
220
+ - **No PII beyond what the build repo already allows** — the agenda lives
128
221
  in `client-<slug>/docs/`, so contact names are fine, but nothing from
129
222
  the registry that the build repo's rules exclude (no personal
130
223
  emails/phones — point at the registry instead), and never credentials.
131
224
 
132
225
  ## Before you finish — checklist
133
226
 
134
- - [ ] All four prep checks ran against the real repo (not assumed).
227
+ - [ ] **Google Calendar queried across the week**, and every attendee's email
228
+ checked against the registry — unknown attendees logged as prep gaps.
229
+ - [ ] All five prep checks ran against the real repo (not assumed).
230
+ - [ ] Auto-drafted Drive agenda (if any) was found, verified, and merged —
231
+ not ignored, not trusted blind.
135
232
  - [ ] Prep gaps, if any, are at the top with fixes — not buried.
233
+ - [ ] Read-back facts are checked against the sales recording, not just notes.
136
234
  - [ ] Every question maps to an unfilled doc section.
137
- - [ ] Access list matches `04-access-checklist.md` exactly.
235
+ - [ ] Access list matches `04-access-checklist.md` exactly, each item
236
+ tagged LIVE-on-call or working-session.
138
237
  - [ ] Vertical context comes from a real playbook file — or its absence
139
238
  is a prep-gap line, not a silently missing section.
140
- - [ ] Sheet is ≤1 page; landmines (name spelling, prior failures) are in
141
- read-back facts.
239
+ - [ ] Old `05-call-sheet.md` removed if it existed.
240
+ - [ ] Drive copy placed/updated at the root of the client's Drive folder.
@@ -0,0 +1,323 @@
1
+ ---
2
+ name: vendor-feed-ingestion
3
+ description: >-
4
+ Get a client's vendor data arriving automatically every day — via our shared
5
+ SFTP drop (SFTPGo → S3) or the data@ ingest inbox — and make the pipeline
6
+ refuse to publish garbage. Covers the source-preference ladder (API →
7
+ scheduled vendor push → RPA last resort, with manual export as bootstrap
8
+ only), the delivery contract you hand the person building the push
9
+ (atomic write, one stable path, cadence, required columns, join key), the
10
+ format traps that eat a morning each (UTF-8 BOM, .xls named .csv, column-name
11
+ variants, invented status vocabularies, uploads landing in a nested path),
12
+ the difference between a snapshot export and a change-log export, and — the
13
+ part nobody builds until it's too late — two-tier feed guards calibrated
14
+ against real versioned deliveries, tuned so they never fire on a normal
15
+ morning and never blow away yesterday's good output. Use this WHENEVER an
16
+ engagement ingests a recurring file or export — "set up the daily feed",
17
+ "they'll send us a CSV", "pull their export", "the data looks wrong today",
18
+ "the workflow ran but the numbers are off", "onboard a new client feed", or
19
+ provisioning an SFTP user.
20
+ ---
21
+
22
+ # Vendor feed ingestion
23
+
24
+ Built and operated on CarVision (vAuto + DriveCentric daily feeds →
25
+ `clients/client-carvision/agent-pricing/`), running on the shared drop in
26
+ `team-infra/sftp-hosting/`. That README is the *infrastructure* reference — how
27
+ to deploy and operate the host. This is the *engagement* reference: how to take
28
+ a client from "someone exports a spreadsheet every morning" to a feed that runs
29
+ itself and tells you when it's lying to you.
30
+
31
+ **The thesis: the ingest is not the hard part. Knowing whether today's file is
32
+ trustworthy is the hard part**, and every hour saved by skipping it is repaid
33
+ with interest the first time a bad file silently produces a confident wrong
34
+ answer in front of the client.
35
+
36
+ ---
37
+
38
+ ## Part 1 — Choose the source, in this order
39
+
40
+ 1. **Native API.** Best when the vendor has a self-serve developer program.
41
+ 2. **Vendor scheduled push** — SFTP or secure email on a timer. The standard
42
+ fallback and, in practice, what most engagements ship on. Cheaper to set up
43
+ than an API partnership and it needs no ongoing auth maintenance.
44
+ 3. **RPA on a read-only login.** Last resort. Brittle, and frequently against
45
+ the vendor's ToS — flag it explicitly rather than sliding it in.
46
+
47
+ **A one-time manual export is a bootstrap, never the design.** Start building
48
+ on it so the scoring/logic layer progresses while authorizations clear, then
49
+ swap in the live feed with no rework. Say this out loud in the kickoff: "no one
50
+ at your company exports anything every morning" is a selling point *and* a
51
+ scope commitment.
52
+
53
+ **Vendor authorization has lead time.** A data agreement or partner-program
54
+ approval can take days to weeks and needs the *client* to flip the switch. Get
55
+ it started in week 1 and track it on `docs/04-access-checklist.md` — it is
56
+ routinely the longest pole and it isn't ours to shorten.
57
+
58
+ ---
59
+
60
+ ## Part 2 — Pick the transport
61
+
62
+ | | **SFTP drop** | **`data@installagent.com` inbox** |
63
+ |---|---|---|
64
+ | Best for | A vendor or client engineer who can script a push | A vendor that only offers "email me this report on a schedule" |
65
+ | Endpoint | `sftp.installagent.com:2022`, key-based, chrooted to `s3://<bucket>/<client>/<source>/` | An n8n Gmail trigger with a sender + subject filter |
66
+ | History | S3 versioning keeps every delivery | Mailbox retains the attachments |
67
+ | Setup | `team-infra/sftp-hosting/scripts/provision-client.sh` | A filter and a fetch node |
68
+ | Watch out for | Uploads landing in the wrong path | Attachment format lying about itself |
69
+
70
+ Both are live and both are in production use — CarVision's CRM feed rides SFTP
71
+ and its inventory feed rides email, in the same workflow.
72
+
73
+ **Provisioning an SFTP feed** (full walkthrough in
74
+ `team-infra/sftp-hosting/README.md`): one **write** user for the vendor to push
75
+ with, and optionally a **read** user (`-ro` suffix) for n8n to pull with, both
76
+ on the same S3 prefix. Re-running the provisioning script updates an existing
77
+ user, so it's safe to re-run. Keys are delivered through the client's 1Password
78
+ vault — not pasted into Slack, not emailed.
79
+
80
+ ---
81
+
82
+ ## Part 3 — Write the delivery contract down and hand it over
83
+
84
+ Whoever builds the push on the client's side needs one page, not a
85
+ conversation. Template in
86
+ [`references/delivery-contract-template.md`](references/delivery-contract-template.md).
87
+ The five clauses that are load-bearing:
88
+
89
+ **1. One stable path, not a dated filename.** `drivecentric-latest.csv` at the
90
+ **root of the session**, overwritten daily. S3 versioning keeps the history, so
91
+ you get both a stable read path and a full archive. A dated filename means the
92
+ workflow has to guess today's name and silently reads nothing when the vendor's
93
+ date format drifts.
94
+
95
+ > CarVision's first real push landed several directories deep
96
+ > (`…/<their-server-path>/<source>-latest.csv`) instead of at the session
97
+ > root. The workflow read the root path, found the
98
+ > previous file, and ran happily on stale data. **Spell out "no directory
99
+ > prefix" and verify the first delivery's actual S3 key** before declaring the
100
+ > feed live.
101
+
102
+ **2. Atomic write, required.**
103
+
104
+ ```
105
+ put <export>.csv feed-latest.csv.part
106
+ rename feed-latest.csv.part feed-latest.csv
107
+ ```
108
+
109
+ Without it the workflow eventually reads a half-written file. The write user
110
+ has `rename` permission specifically so this is possible.
111
+
112
+ **3. Cadence with a stated deadline, tied to our schedule.** "Completed before
113
+ 05:30 local; our workflow runs at 06:00." And say *what the file must cover* —
114
+ each push must be **that day's** export, not a re-upload of yesterday's.
115
+
116
+ **4. Required columns, and the join key called out as mandatory.** List every
117
+ column the logic reads and mark the join key **required, non-blank**. Rows
118
+ arriving with a blank join key are usually unrecoverable — on CarVision, 1 of
119
+ 162 blank-VIN lead rows had a stock number to recover from, and the rest were
120
+ simply lost. Cheaper to make it a contract clause than a repair project.
121
+
122
+ **5. One combined file when the client has multiple locations**, with a store
123
+ identifier column, rather than one file per site. The client concatenates on
124
+ their side; we read one stable path and never build merge logic that has to
125
+ know how many stores exist this quarter.
126
+
127
+ ---
128
+
129
+ ## Part 4 — Format traps
130
+
131
+ Each of these has cost a morning. Check them on the **first real file**, not
132
+ the sample.
133
+
134
+ - **UTF-8 BOM.** Vendor exports routinely carry one, and it corrupts the first
135
+ column's header (`VIN` becomes `VIN`, matching nothing). Set
136
+ `enableBOM: true` on every CSV parse node, unconditionally. It's free when
137
+ there's no BOM.
138
+ - **The extension lies.** A file named `.csv` that is really an Excel `.xls`,
139
+ or vice versa. Test-parse the first real delivery and look at the output, not
140
+ the filename.
141
+ - **Column-name variants.** `CurrentStage` vs `Current Stage` vs `current_stage`.
142
+ Read defensively — accept the known variants — and log which one matched, so
143
+ a silent switch is visible instead of producing zeros.
144
+ - **Invented status vocabularies.** Never encode rules against a status list
145
+ you assumed. CarVision's config assumed an `Engaged / Visit / Proposal`
146
+ funnel; the live feed emitted `Open / Dead / Delivered`. Pull the distinct
147
+ values from the first real file and write the rules against those, keeping
148
+ the finer vocabulary wired as forward-compatible no-ops if the client plans
149
+ to emit it later.
150
+ - **A coarse status is not the status you wanted.** When `Open` covers ~60% of
151
+ rows, a rule keyed on it is far blunter than the spec implies. Ship it if
152
+ it's conservative in the safe direction, and write down what the sharper
153
+ version needs.
154
+
155
+ ---
156
+
157
+ ## Part 5 — Snapshot vs change-log, and the window offset
158
+
159
+ Two questions to answer about **every** feed, in writing, before building logic
160
+ on it:
161
+
162
+ **Is this a snapshot or a change log?** A snapshot is "the state of everything
163
+ right now." A change log is "what changed." They demand different logic, and
164
+ vendors label neither. CarVision's appointment export turned out to be a change
165
+ log — so absence of a row means "nothing changed," not "nothing exists," and
166
+ fields like a no-show flag are *defaults* on rows where nothing was recorded,
167
+ not observations. Treating a change log as a snapshot produces confidently
168
+ wrong aggregates.
169
+
170
+ **What time window does the file actually cover?** Not "when was it sent" —
171
+ what period does the data span.
172
+
173
+ > CarVision's feed measured ~29 hours "stale" every single day. It looked like
174
+ > a broken export. It was a **one-day window offset**: the vendor's export
175
+ > covered the previous day and ours covered the day before that, both delivered
176
+ > the same morning. The data was fine; our understanding of the window was not.
177
+ >
178
+ > On the strength of the wrong reading we had told the client 20.3% of their
179
+ > stage data was wrong. It wasn't. Every one of the 38 contradicting records
180
+ > landed *after* our cutoff — our window, their data. **That retraction is the
181
+ > reason this section exists.** Measure the window before making a claim about
182
+ > someone else's data quality.
183
+
184
+ Compute the lag **in production** and record it (`lagHours` on every run), so
185
+ the number in a client conversation is measured, not inferred.
186
+
187
+ **And when a window changes, ask for a one-off catch-up file.** Flipping a
188
+ change-log export's window forward skips a day — and a skipped change-log day
189
+ is gone forever.
190
+
191
+ ---
192
+
193
+ ## Part 6 — Feed guards (the part that earns its keep)
194
+
195
+ A guard decides whether today's delivery is fit to publish. Get this wrong in
196
+ either direction and you lose: too loose and a degraded file produces a
197
+ confident wrong answer; too tight and the client gets no output on a normal
198
+ Monday.
199
+
200
+ ### Two tiers, for a reason
201
+
202
+ | Tier | Example | Needs | Available |
203
+ |---|---|---|---|
204
+ | **Absolute floors** | `minRowsHardFloor: 100`, required columns present | no state | from the very first run |
205
+ | **Relative** | share of the trailing 7-day median row count | run history | after ~3 runs |
206
+
207
+ The relative guards are the real instrument; the absolute floors are what you
208
+ have on day one. Ship both.
209
+
210
+ ### Calibrate against real deliveries, not intuition
211
+
212
+ Pull the versioned history out of S3 — every past delivery is still there — and
213
+ replay the candidate thresholds over it. The bar: **every known-degraded day
214
+ trips and every known-healthy day passes.** CarVision's were calibrated on 14
215
+ versioned snapshots and validated at 7 of 16 historical degraded deliveries
216
+ blocked with **zero false positives** on healthy days.
217
+
218
+ Record the calibration evidence *next to the thresholds*, in the config file
219
+ itself — what data it was calibrated on, which days were degraded, and a note
220
+ that absolute floors are scale-dependent and need re-checking if the client's
221
+ volume changes materially.
222
+
223
+ ### A guard that fires every morning is a guard that gets muted
224
+
225
+ Set the freshness threshold **above** the routine baseline. CarVision's content
226
+ lag ran ~29h every day; `maxLagHours` sits at **40**. What that catches is a
227
+ feed that has *stopped moving* — a re-uploaded stale file, a paused export —
228
+ not the routine lag, which is an upstream problem needing an upstream fix.
229
+ A threshold set at 30h would have fired daily and been ignored within a week.
230
+
231
+ ### Volume is seasonal — a flat floor is a trap
232
+
233
+ CarVision's row count tracks dealership activity ~2 days back: a weekday file
234
+ carries 2,300–2,900 rows, a Saturday ~1,000, a Sunday ~200. A flat
235
+ "50% of the trailing median" floor therefore trips **every Monday and Tuesday,
236
+ forever** — and the four "degraded deliveries" originally recorded turned out
237
+ to be two Mondays and two Tuesdays, i.e. the weekly cycle, not four incidents.
238
+
239
+ Two lessons, both general:
240
+
241
+ - **Plot the metric across a full week before setting a threshold on it.**
242
+ - **Low volume warns; only impossible volume trips.** Hard floors sit *below*
243
+ the smallest real delivery ever observed.
244
+
245
+ **Upper bounds stay hard trips.** A 27× history dump poisons any accumulated
246
+ state and is not recoverable on the next run, so `maxShareOfMedian` /
247
+ `maxRowsAbsolute` block outright.
248
+
249
+ ### A trip must not destroy yesterday's good output
250
+
251
+ The failure path matters more than the detection. On a trip:
252
+
253
+ - **Skip the destructive step** — do not clear the client-facing sheet/table.
254
+ Yesterday's good output survives.
255
+ - **Alert a human**, with the specific reason and number.
256
+ - **Make sure warnings actually go somewhere.** CarVision had a warning tier
257
+ whose alert text was computed and then sent nowhere for weeks. If a branch
258
+ has no destination node, it doesn't exist.
259
+ - **Tell the client's user the output is stale.** A person looking at yesterday's
260
+ worklist with no staleness marker cannot tell. Publish the produced-at
261
+ timestamp and a feed-health indicator on the surface they actually look at.
262
+
263
+ Concrete threshold shapes and the full guard set:
264
+ [`references/feed-guards.md`](references/feed-guards.md).
265
+
266
+ ---
267
+
268
+ ## Part 7 — Verify against ground truth, not against itself
269
+
270
+ **An audit of the feed against the feed is worth nothing.** CarVision's
271
+ lead-volume investigation concluded "ingestion is faithful, counts are correct"
272
+ — and was wrong, because it had no external reference. The later reconciliation
273
+ against the client's own CRM report found the feed carried **79%** of the real
274
+ leads, and that gap had been silently distorting output the whole time.
275
+
276
+ So: **get one export of the same data from a second path** — the client running
277
+ a report out of their UI is fine — and reconcile row counts, key coverage, and
278
+ a couple of aggregates. Do it once at go-live, and again whenever the numbers
279
+ start an argument.
280
+
281
+ Two corollaries that generalize past this pattern:
282
+
283
+ - **Don't ask the client what our own data answers.** Several questions we
284
+ queued up for a client SME were answerable from files already on disk. Check
285
+ first; ask what's left.
286
+ - **Before accusing a vendor's data of being wrong, prove the fault isn't in
287
+ your window, your join, or your parse.** See Part 5.
288
+
289
+ ---
290
+
291
+ ## Requirements checklist
292
+
293
+ - [ ] Source chosen against the ladder; manual export explicitly labelled bootstrap-only
294
+ - [ ] Vendor authorization started in week 1 and tracked on the access checklist
295
+ - [ ] Delivery contract handed over in writing: stable path, atomic write, cadence + coverage, required columns, mandatory join key
296
+ - [ ] **First real delivery's actual storage key verified** — not assumed from the contract
297
+ - [ ] `enableBOM: true` on every CSV parse node; first real file test-parsed and eyeballed
298
+ - [ ] Distinct values pulled for every status/category column the logic keys on
299
+ - [ ] Snapshot vs change-log determined and written in the tech spec
300
+ - [ ] Window coverage established and `lagHours` measured in production
301
+ - [ ] Absolute floors live from run 1; relative guards calibrated on versioned S3 history
302
+ - [ ] Calibration evidence recorded alongside the thresholds
303
+ - [ ] Freshness threshold set above the routine baseline
304
+ - [ ] Volume metric plotted across a full week before any share-of-median floor
305
+ - [ ] Trip path skips the destructive step, alerts a human, and every warning branch has a destination
306
+ - [ ] Staleness surfaced on the client-facing output
307
+ - [ ] Reconciled against a second, independent export of the same data
308
+
309
+ ## Known gaps to flag, not silently skip
310
+
311
+ - **Guards detect shape, not truth.** A file with the right row count, columns
312
+ and freshness can still be wrong. Guards buy you protection against the
313
+ obvious failures; Part 7 is what covers the rest.
314
+ - **Absolute floors are scale-dependent.** They encode the client's current
315
+ volume. Re-check them after any material change in the client's business, and
316
+ say so in the config comment.
317
+ - **The upstream fix is usually the real fix.** Guards manage the symptom of a
318
+ stale or thin feed; only the vendor can stop producing one. Keep the upstream
319
+ ask alive rather than letting a guard become the answer.
320
+ - **Local workflow JSON drifts from the live workflow.** Once a workflow has
321
+ been edited in the n8n UI, a repo copy is a historical artifact — never
322
+ re-import it over the live version. Patch node bodies deliberately and verify
323
+ by hash.