@pipeworx/mcp-ats-boards 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -10
- package/package.json +4 -4
- package/server.json +2 -2
- package/src/index.ts +631 -8
- package/src/server.ts +5 -2
package/README.md
CHANGED
|
@@ -1,13 +1,21 @@
|
|
|
1
1
|
# @pipeworx/ats-boards
|
|
2
2
|
|
|
3
|
-
Public job-board postings across
|
|
4
|
-
Lever, Ashby, SmartRecruiters, Recruitee, BambooHR, Workday,
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
3
|
+
Public job-board postings across fifteen applicant-tracking systems
|
|
4
|
+
(Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee, BambooHR, Workday,
|
|
5
|
+
Workable, Teamtailor, Personio, Jobvite, Breezy, Pinpoint, Rippling, iCIMS) —
|
|
6
|
+
detect which ATS a company publishes its careers page on, then read any
|
|
7
|
+
company's open roles without a key. This is NOT the `greenhouse`/`ashby`
|
|
8
|
+
packs, which read the caller's OWN private hiring data with their own API
|
|
9
|
+
key; this pack reads the same public board any browser can see, for any
|
|
10
|
+
company.
|
|
11
|
+
|
|
12
|
+
Part of [Pipeworx](https://pipeworx.io) — an MCP gateway connecting AI agents to 1847+ live data sources. This is an independent, unofficial integration — not affiliated with, endorsed by, or published by the upstream provider.
|
|
13
|
+
|
|
14
|
+
Dover was evaluated and dropped: its public board is a client-rendered SPA
|
|
15
|
+
behind a Cloudflare Turnstile challenge, and the underlying jobs API
|
|
16
|
+
(`app.dover.com/api/v1/jobs`) answers HTTP 401 "Authentication credentials
|
|
17
|
+
were not provided" for every query-param combination tried — there is no
|
|
18
|
+
discoverable keyless endpoint.
|
|
11
19
|
|
|
12
20
|
## Tools
|
|
13
21
|
|
|
@@ -31,7 +39,7 @@ Part of [Pipeworx](https://pipeworx.io) — an MCP gateway connecting AI agents
|
|
|
31
39
|
|
|
32
40
|
Keyless. Every endpoint below is the same public board data a browser sees on
|
|
33
41
|
a company's own careers page — no account, token or registration on any of
|
|
34
|
-
the
|
|
42
|
+
the fifteen supported ATS vendors.
|
|
35
43
|
|
|
36
44
|
The one exception is Workable: its public widget is currently blocked by
|
|
37
45
|
Cloudflare from this platform's edge (see "Data sources" below) — not an auth
|
|
@@ -49,6 +57,13 @@ rather than returning an empty result.
|
|
|
49
57
|
- `https://{board_token}.bamboohr.com/careers/list` — BambooHR's public careers-widget endpoint.
|
|
50
58
|
- `https://{tenant}.wd{N}.myworkdayjobs.com/wday/cxs/{tenant}/{site}/jobs` (POST) — Workday's public CXS jobs search. `board_token` encodes all three path parts as `"{tenant}.wd{N}/{site}"`, e.g. `"workday.wd5/Workday"` (read the tenant/wd-number/site straight off the company's own `myworkdayjobs.com` careers URL).
|
|
51
59
|
- `https://apply.workable.com/api/v1/widget/accounts/{board_token}` — Workable's public widget API. **Currently unreachable from this platform's edge**, confirmed live via a throwaway `wrangler dev --remote` Worker on Pipeworx's production Cloudflare account: every one of 5+ attempts (two endpoint shapes — the v1 widget and the v3 `/api/v3/accounts/{account}/jobs`, with and without an `Accept` header, against several real account names, several minutes apart) returned HTTP 429 with `Content-Type: text/plain` and body `error code: 1015` — Cloudflare's OWN bot-mitigation, not a Workable application response. That reads as apply.workable.com's Cloudflare WAF specifically rate-limiting Cloudflare-Worker-to-Cloudflare traffic. The adapter is implemented per Workable's documented widget shape and is ready the moment this changes; until then every live call throws loud, naming the 429, rather than returning an empty board.
|
|
60
|
+
- `https://{board_token}.teamtailor.com/jobs.json` — Teamtailor's keyless JSON Feed (jsonfeed.org v1.1).
|
|
61
|
+
- `https://{board_token}.jobs.personio.com/xml?language=en`, falling back to `.jobs.personio.de` on a 404 — Personio's public XML feed. A company lives on exactly one of the two TLDs with no redirect between them.
|
|
62
|
+
- `https://jobs.jobvite.com/{board_token}` — Jobvite's server-rendered HTML board (no JSON API; the vendor's own feed is per-customer and off by default). Parsed by regex from the `jv-job-list-*` table markup.
|
|
63
|
+
- `https://{board_token}.breezy.hr/json` — Breezy's keyless JSON array.
|
|
64
|
+
- `https://{board_token}.pinpointhq.com/postings.json` — Pinpoint's keyless, officially documented postings JSON.
|
|
65
|
+
- `https://ats.rippling.com/{board_token}/jobs` — Rippling has no separate JSON API (every guessed `/api/...` path 404s); the job list is parsed out of the page's `<script id="__NEXT_DATA__">` hydration data.
|
|
66
|
+
- `https://{board_token}.icims.com/jobs/search?ss=1&in_iframe=1` — iCIMS's documented Talent-Cloud API is partner-gated, but the public career-search page itself is plain server-rendered HTML with no login; parsed by regex. `board_token` is the FULL subdomain label (e.g. `"careers-acme"`) — iCIMS customers don't share one fixed prefix.
|
|
52
67
|
|
|
53
68
|
### The silent-zero shape, per ATS (each measured live, not assumed from docs)
|
|
54
69
|
|
|
@@ -65,6 +80,13 @@ from "real board, zero jobs right now" at all:
|
|
|
65
80
|
| BambooHR | HTTP 200 (not 404!), `Content-Type: text/html` — BambooHR's own marketing homepage, byte-identical across different nonsense subdomains (a wildcard catch-all, not a per-company 404) | HTTP 200 JSON `{"meta":{"totalCount":0},"result":[]}` (confirmed live: `helpscout`, `pantheon`) |
|
|
66
81
|
| Workday | HTTP 404 JSON `{"errorCode":"S21",...,"message":"not found: Job_Posting_Site_ID=..."}` | Not independently confirmed (every real tenant found had jobs); `limit` over 20 is HTTP 400, not an empty page; `offset` past 10,000 returns HTTP 200 with the SAME page as `offset:0` rather than erroring — this pack reports `capped:true` when either cap is hit |
|
|
67
82
|
| Workable | Blocked (HTTP 429, Cloudflare) before any board-shaped answer is reachable from this platform | — |
|
|
83
|
+
| Teamtailor | HTTP 404, `Content-Type: application/json`, body EMPTY (0 bytes) | HTTP 200 JSON Feed `{"items":[...]}` (confirmed live: `career`, `arsenalfc`) |
|
|
84
|
+
| Personio | HTTP 404 `text/html` on BOTH `.com` and `.de` | HTTP 200 XML `<workzag-jobs>` (confirmed live: `kertos` on `.de`, `1nce` on `.com`) |
|
|
85
|
+
| Jobvite | HTTP 200 but 4 silent redirects to a DIFFERENT host (`www.jobvite.com/support/...?invalid=1`) — `fetch`'s default `redirect:'follow'` hides this unless you check `res.redirected` | HTTP 200, no redirect, server-rendered job table (confirmed live: `bagleybagelbites`, 65 rows) |
|
|
86
|
+
| Breezy | HTTP 404, `Content-Type: text/html`, title "Career portal not found" | HTTP 200 JSON array (confirmed live: `breezy`, `duolingo`) |
|
|
87
|
+
| Pinpoint | HTTP 404, `Content-Type: text/html` (Pinpoint's own 404 page) | Not independently confirmed (every real board found had postings); expected HTTP 200 `{"data":[]}` by the same convention |
|
|
88
|
+
| Rippling | HTTP 404, Next.js's own 404 page, no `__NEXT_DATA__` job-posts query | HTTP 200 with a `job-posts` query present but `items:[]` — not independently confirmed (every real board found had postings) |
|
|
89
|
+
| iCIMS | HTTP 404, title `"gone: {subdomain}.icims.com : dc409"` | Not independently confirmed (every real portal found had postings) |
|
|
68
90
|
|
|
69
91
|
Treating any non-200 as "zero jobs," or SmartRecruiters' `totalFound:0` as a
|
|
70
92
|
confirmed empty board, is the exact silent-zero failure this pack exists to
|
|
@@ -127,7 +149,7 @@ directly, instead of just this one's:
|
|
|
127
149
|
}
|
|
128
150
|
```
|
|
129
151
|
|
|
130
|
-
Both URLs reach the same gateway and the same
|
|
152
|
+
Both URLs reach the same gateway and the same 1847+ data sources. The
|
|
131
153
|
only difference is which pack's tools are listed **directly**; `ask_pipeworx`
|
|
132
154
|
reaches all of them from either one.
|
|
133
155
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@pipeworx/mcp-ats-boards",
|
|
3
|
-
"version": "0.1.
|
|
4
|
-
"description": "ATS Boards MCP — public job-board postings across eight applicant-tracking",
|
|
3
|
+
"version": "0.1.2",
|
|
4
|
+
"description": "ATS Boards MCP — public job-board postings across eight applicant-tracking systems (keyless).",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.ts",
|
|
7
7
|
"types": "src/index.ts",
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
"@cloudflare/workers-types": "^4.20260405.1"
|
|
27
27
|
},
|
|
28
28
|
"pipeworx": {
|
|
29
|
-
"sourceHash": "v1-
|
|
30
|
-
"sourceCommit": "
|
|
29
|
+
"sourceHash": "v1-9c1752cc9563c412beb9efed47c3dad9cfd23f84a3077a322a1382e8c34b5739",
|
|
30
|
+
"sourceCommit": "54bf7d1c06a0775508a65d0c388dd10ba75174c3"
|
|
31
31
|
}
|
|
32
32
|
}
|
package/server.json
CHANGED
|
@@ -2,8 +2,8 @@
|
|
|
2
2
|
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
|
|
3
3
|
"name": "io.github.pipeworx-io/ats-boards",
|
|
4
4
|
"title": "Ats Boards",
|
|
5
|
-
"description": "ATS Boards MCP — public job-board postings across eight applicant-tracking",
|
|
6
|
-
"version": "0.1.
|
|
5
|
+
"description": "ATS Boards MCP — public job-board postings across eight applicant-tracking systems (keyless).",
|
|
6
|
+
"version": "0.1.2",
|
|
7
7
|
"websiteUrl": "https://pipeworx.io/packs/ats-boards",
|
|
8
8
|
"repository": {
|
|
9
9
|
"url": "https://github.com/pipeworx-io/mcp-ats-boards",
|
package/src/index.ts
CHANGED
|
@@ -597,14 +597,14 @@ function xmlFaultText(raw: string): string | null {
|
|
|
597
597
|
* and a leak is a leak regardless of which branch produced it.
|
|
598
598
|
*/
|
|
599
599
|
function stripMarkup(s: string): string {
|
|
600
|
-
return collapse(decodeEntities(s.replace(/<[^>]*>/g, ' ')).replace(/[<>]/g, ' '));
|
|
600
|
+
return collapse(decodeEntities$shared(s.replace(/<[^>]*>/g, ' ')).replace(/[<>]/g, ' '));
|
|
601
601
|
}
|
|
602
602
|
|
|
603
603
|
/** The handful of entities that show up in error-page titles. Decoded AFTER
|
|
604
604
|
* tags are stripped and BEFORE the angle-bracket sweep, so `<script>`
|
|
605
605
|
* in a title cannot decode into markup that survives — EMBL-EBI's ChEMBL 500
|
|
606
606
|
* page renders as `500 Internal Server Error < EMBL-EBI` otherwise. */
|
|
607
|
-
function decodeEntities(s: string): string {
|
|
607
|
+
function decodeEntities$shared(s: string): string {
|
|
608
608
|
return s
|
|
609
609
|
.replace(/&(?:amp|#0*38);/gi, '&')
|
|
610
610
|
.replace(/&(?:lt|#0*60);/gi, '<')
|
|
@@ -671,6 +671,254 @@ function collapse(s: string): string {
|
|
|
671
671
|
return s.replace(/\s+/g, ' ').trim();
|
|
672
672
|
}
|
|
673
673
|
|
|
674
|
+
// ── Bounded readers — peak memory, not just wall time (fleet #3322) ──────
|
|
675
|
+
//
|
|
676
|
+
// `parseJson` above does `await res.text()` then `JSON.parse(raw)`, so a
|
|
677
|
+
// multi-megabyte upstream body is held TWICE at peak: once as the raw string
|
|
678
|
+
// and once as the object graph (which for an array of small objects runs
|
|
679
|
+
// several times the byte size). Nothing in `workers/gateway` can see that
|
|
680
|
+
// peak — it happens inside the pack, before any gateway code holds a payload —
|
|
681
|
+
// which is why the gateway-side payload caps of #3293 and #3320 did not move
|
|
682
|
+
// the 503/1102 rate at all.
|
|
683
|
+
//
|
|
684
|
+
// Measured on the live gateway, 2026-10-19, sha fa5de5c34, direct tools/call:
|
|
685
|
+
// statcan_list_cubes 8,049,566 B <- one upstream array, no limit argument
|
|
686
|
+
// gnomad_gene BRCA1 2,732,706 B
|
|
687
|
+
// Five of those in flight at the quality runner's width is ~40 MB of parse
|
|
688
|
+
// peak against a 128 MB isolate; Cloudflare kills the isolate and every
|
|
689
|
+
// request in flight returns 503/1102, which is why the victims look random.
|
|
690
|
+
//
|
|
691
|
+
// `readJsonArrayPrefix` is the fix for the dominant shape: an upstream that
|
|
692
|
+
// returns one enormous top-level JSON array with no server-side paging. It
|
|
693
|
+
// parses elements off the stream and DISCARDS the consumed bytes, so peak
|
|
694
|
+
// retained memory is one element plus one chunk rather than the whole body,
|
|
695
|
+
// and it cancels the body as soon as it has what it was asked for.
|
|
696
|
+
|
|
697
|
+
/** Default ceiling for a single upstream JSON body read into an isolate.
|
|
698
|
+
* Chosen so the known tail (statcan 8.0 MB, gnomad 2.7 MB) is bounded well
|
|
699
|
+
* below the point where a handful of concurrent reads exhausts 128 MB. */
|
|
700
|
+
const DEFAULT_MAX_JSON_BYTES = 1_500_000;
|
|
701
|
+
|
|
702
|
+
interface JsonArrayPrefix<T> {
|
|
703
|
+
/** The decoded elements, at most `maxItems` of them. */
|
|
704
|
+
items: T[];
|
|
705
|
+
/** True when the upstream array had more elements than we read. */
|
|
706
|
+
truncated: boolean;
|
|
707
|
+
/** Bytes actually pulled off the socket — the number that bounds memory. */
|
|
708
|
+
bytesRead: number;
|
|
709
|
+
/** `Content-Length` if the upstream declared one, else null. */
|
|
710
|
+
declaredBytes: number | null;
|
|
711
|
+
}
|
|
712
|
+
|
|
713
|
+
/**
|
|
714
|
+
* Read the FIRST `maxItems` elements of a top-level JSON array off a response
|
|
715
|
+
* stream without ever materialising the whole body.
|
|
716
|
+
*
|
|
717
|
+
* Use this instead of `parseJson` when the upstream returns one big array and
|
|
718
|
+
* offers no `limit`/`offset` of its own. Peak retained memory is one element
|
|
719
|
+
* plus the current chunk; the body is cancelled as soon as `maxItems` is
|
|
720
|
+
* reached, so the remaining megabytes are never transferred at all.
|
|
721
|
+
*
|
|
722
|
+
* const { items, truncated } = await readJsonArrayPrefix<Cube>(
|
|
723
|
+
* res, 'StatCan WDS', { maxItems: limit + offset });
|
|
724
|
+
*
|
|
725
|
+
* `maxBytes` is a second, independent backstop for a pathological element (a
|
|
726
|
+
* single 50 MB object): hitting it stops the read and sets `truncated`, rather
|
|
727
|
+
* than throwing, because a partial answer plus an explicit truncated flag is
|
|
728
|
+
* more useful to a caller than an error.
|
|
729
|
+
*/
|
|
730
|
+
async function readJsonArrayPrefix<T>(
|
|
731
|
+
res: Response,
|
|
732
|
+
name: string,
|
|
733
|
+
opts: { maxItems: number; maxBytes?: number },
|
|
734
|
+
): Promise<JsonArrayPrefix<T>> {
|
|
735
|
+
const maxBytes = opts.maxBytes ?? DEFAULT_MAX_JSON_BYTES;
|
|
736
|
+
const declaredRaw = Number(res.headers.get('content-length'));
|
|
737
|
+
const declaredBytes = Number.isFinite(declaredRaw) && declaredRaw > 0 ? declaredRaw : null;
|
|
738
|
+
|
|
739
|
+
// No streaming body available (a mock, or an already-buffered response):
|
|
740
|
+
// fall back to the whole-text path so behaviour stays correct, just unbounded.
|
|
741
|
+
if (!res.body) {
|
|
742
|
+
const all = await parseJson<T[]>(res, name);
|
|
743
|
+
if (!Array.isArray(all)) throw new Error(`upstream_down: ${name} returned ${typeof all} where a JSON array was expected.`);
|
|
744
|
+
return {
|
|
745
|
+
items: all.slice(0, opts.maxItems),
|
|
746
|
+
truncated: all.length > opts.maxItems,
|
|
747
|
+
bytesRead: declaredBytes ?? 0,
|
|
748
|
+
declaredBytes,
|
|
749
|
+
};
|
|
750
|
+
}
|
|
751
|
+
|
|
752
|
+
const reader = res.body.getReader();
|
|
753
|
+
const decoder = new TextDecoder();
|
|
754
|
+
const items: T[] = [];
|
|
755
|
+
|
|
756
|
+
let buf = '';
|
|
757
|
+
let scan = 0;
|
|
758
|
+
let bytesRead = 0;
|
|
759
|
+
let started = false; // consumed the opening '['
|
|
760
|
+
let arrayEnded = false;
|
|
761
|
+
let depth = 0;
|
|
762
|
+
let inStr = false;
|
|
763
|
+
let esc = false;
|
|
764
|
+
let elemStart = -1;
|
|
765
|
+
let truncated = false;
|
|
766
|
+
|
|
767
|
+
const push = (text: string) => {
|
|
768
|
+
const t = text.trim();
|
|
769
|
+
if (!t) return;
|
|
770
|
+
items.push(JSON.parse(t) as T);
|
|
771
|
+
};
|
|
772
|
+
|
|
773
|
+
try {
|
|
774
|
+
outer: while (items.length < opts.maxItems && !arrayEnded) {
|
|
775
|
+
const { value, done } = await reader.read();
|
|
776
|
+
if (done) break;
|
|
777
|
+
bytesRead += value.byteLength;
|
|
778
|
+
buf += decoder.decode(value, { stream: true });
|
|
779
|
+
if (bytesRead > maxBytes) { truncated = true; break; }
|
|
780
|
+
|
|
781
|
+
for (; scan < buf.length; scan++) {
|
|
782
|
+
const ch = buf[scan];
|
|
783
|
+
|
|
784
|
+
if (inStr) {
|
|
785
|
+
if (esc) { esc = false; continue; }
|
|
786
|
+
if (ch === '\\') { esc = true; continue; }
|
|
787
|
+
if (ch === '"') inStr = false;
|
|
788
|
+
continue;
|
|
789
|
+
}
|
|
790
|
+
|
|
791
|
+
if (!started) {
|
|
792
|
+
if (ch === '[') { started = true; continue; }
|
|
793
|
+
if (ch.trim() === '') continue;
|
|
794
|
+
throw new Error(
|
|
795
|
+
`upstream_down: ${name} returned a body that does not begin with a JSON array (saw ${JSON.stringify(ch)}). ` +
|
|
796
|
+
'This reader is only for upstreams that answer with one top-level array.',
|
|
797
|
+
);
|
|
798
|
+
}
|
|
799
|
+
|
|
800
|
+
if (ch === '"') { if (depth === 0 && elemStart < 0) elemStart = scan; inStr = true; continue; }
|
|
801
|
+
|
|
802
|
+
if (ch === '{' || ch === '[') {
|
|
803
|
+
if (depth === 0 && elemStart < 0) elemStart = scan;
|
|
804
|
+
depth++;
|
|
805
|
+
continue;
|
|
806
|
+
}
|
|
807
|
+
|
|
808
|
+
if (ch === '}' || ch === ']') {
|
|
809
|
+
if (depth > 0) {
|
|
810
|
+
depth--;
|
|
811
|
+
if (depth === 0 && elemStart >= 0) {
|
|
812
|
+
push(buf.slice(elemStart, scan + 1));
|
|
813
|
+
elemStart = -1;
|
|
814
|
+
if (items.length >= opts.maxItems) { truncated = true; scan++; break outer; }
|
|
815
|
+
}
|
|
816
|
+
continue;
|
|
817
|
+
}
|
|
818
|
+
// depth 0 and a closing bracket: the end of the top-level array.
|
|
819
|
+
if (elemStart >= 0) { push(buf.slice(elemStart, scan)); elemStart = -1; }
|
|
820
|
+
arrayEnded = true;
|
|
821
|
+
break outer;
|
|
822
|
+
}
|
|
823
|
+
|
|
824
|
+
if (ch === ',' && depth === 0) {
|
|
825
|
+
if (elemStart >= 0) {
|
|
826
|
+
push(buf.slice(elemStart, scan));
|
|
827
|
+
elemStart = -1;
|
|
828
|
+
if (items.length >= opts.maxItems) { truncated = true; scan++; break outer; }
|
|
829
|
+
}
|
|
830
|
+
continue;
|
|
831
|
+
}
|
|
832
|
+
|
|
833
|
+
// A bare scalar element (number / true / false / null).
|
|
834
|
+
if (depth === 0 && elemStart < 0 && ch.trim() !== '') elemStart = scan;
|
|
835
|
+
}
|
|
836
|
+
|
|
837
|
+
// Drop everything already consumed. THIS is what bounds memory: without
|
|
838
|
+
// it `buf` grows to the full body and the reader is streaming in name only.
|
|
839
|
+
if (elemStart >= 0) {
|
|
840
|
+
buf = buf.slice(elemStart);
|
|
841
|
+
scan -= elemStart;
|
|
842
|
+
elemStart = 0;
|
|
843
|
+
} else {
|
|
844
|
+
buf = buf.slice(scan);
|
|
845
|
+
scan = 0;
|
|
846
|
+
}
|
|
847
|
+
}
|
|
848
|
+
} finally {
|
|
849
|
+
// Cancel rather than drain: the point is that the remaining megabytes are
|
|
850
|
+
// never transferred. A cancel on an already-finished body is a no-op.
|
|
851
|
+
await reader.cancel().catch(() => {});
|
|
852
|
+
}
|
|
853
|
+
|
|
854
|
+
return { items, truncated, bytesRead, declaredBytes };
|
|
855
|
+
}
|
|
856
|
+
|
|
857
|
+
/**
|
|
858
|
+
* `parseJson`, with a hard ceiling on how many bytes are read.
|
|
859
|
+
*
|
|
860
|
+
* For an upstream whose body is not a single array (so `readJsonArrayPrefix`
|
|
861
|
+
* does not apply) but which can still answer with megabytes. Unlike the array
|
|
862
|
+
* reader this one THROWS when the cap is hit, because a truncated prefix of an
|
|
863
|
+
* arbitrary JSON document is not parseable and there is no partial answer to
|
|
864
|
+
* hand back. A declared `Content-Length` over the cap is refused before a
|
|
865
|
+
* single chunk is read.
|
|
866
|
+
*/
|
|
867
|
+
async function parseJsonBounded<T>(
|
|
868
|
+
res: Response,
|
|
869
|
+
name: string,
|
|
870
|
+
maxBytes: number = DEFAULT_MAX_JSON_BYTES,
|
|
871
|
+
): Promise<T> {
|
|
872
|
+
const declaredRaw = Number(res.headers.get('content-length'));
|
|
873
|
+
const declared = Number.isFinite(declaredRaw) && declaredRaw > 0 ? declaredRaw : null;
|
|
874
|
+
|
|
875
|
+
const tooBig = (n: number) =>
|
|
876
|
+
new Error(
|
|
877
|
+
`upstream_down: ${name} answered with ${n.toLocaleString()} bytes, over this pack's ${maxBytes.toLocaleString()}-byte ceiling. ` +
|
|
878
|
+
'The body is not read past the ceiling on purpose — a few concurrent reads of a body that size exhaust the ' +
|
|
879
|
+
'Worker isolate and take every other in-flight request down with them. Narrow the request (a smaller range, ' +
|
|
880
|
+
'a specific id, or a lower limit) and retry.',
|
|
881
|
+
);
|
|
882
|
+
|
|
883
|
+
if (declared !== null && declared > maxBytes) {
|
|
884
|
+
await res.body?.cancel().catch(() => {});
|
|
885
|
+
throw tooBig(declared);
|
|
886
|
+
}
|
|
887
|
+
if (!res.body) return parseJson<T>(res, name);
|
|
888
|
+
|
|
889
|
+
const reader = res.body.getReader();
|
|
890
|
+
const decoder = new TextDecoder();
|
|
891
|
+
let raw = '';
|
|
892
|
+
let bytesRead = 0;
|
|
893
|
+
try {
|
|
894
|
+
for (;;) {
|
|
895
|
+
const { value, done } = await reader.read();
|
|
896
|
+
if (done) break;
|
|
897
|
+
bytesRead += value.byteLength;
|
|
898
|
+
if (bytesRead > maxBytes) throw tooBig(bytesRead);
|
|
899
|
+
raw += decoder.decode(value, { stream: true });
|
|
900
|
+
}
|
|
901
|
+
raw += decoder.decode();
|
|
902
|
+
} finally {
|
|
903
|
+
await reader.cancel().catch(() => {});
|
|
904
|
+
}
|
|
905
|
+
|
|
906
|
+
if (!raw.trim()) {
|
|
907
|
+
throw new Error(
|
|
908
|
+
`upstream_down: ${name} answered HTTP ${res.status} with an EMPTY body where JSON was expected. ` +
|
|
909
|
+
'Nothing about the request can cause this — it is an upstream fault, and the same call may well work on retry.',
|
|
910
|
+
);
|
|
911
|
+
}
|
|
912
|
+
try {
|
|
913
|
+
return JSON.parse(raw) as T;
|
|
914
|
+
} catch {
|
|
915
|
+
throw new Error(
|
|
916
|
+
`upstream_down: ${name} answered HTTP ${res.status} with a body that is not valid JSON. ` +
|
|
917
|
+
`It begins: ${stripMarkup(raw).slice(0, 120) || '(unreadable)'}`,
|
|
918
|
+
);
|
|
919
|
+
}
|
|
920
|
+
}
|
|
921
|
+
|
|
674
922
|
|
|
675
923
|
/**
|
|
676
924
|
* SSRF guard for fetching user- or registry-supplied URLs.
|
|
@@ -1380,6 +1628,83 @@ async function safeFetchExtended(
|
|
|
1380
1628
|
* list. Do not spend more probing time rediscovering this; it is
|
|
1381
1629
|
* consistent and documented.
|
|
1382
1630
|
*
|
|
1631
|
+
* ── Second wave (fleet #3086, 2026-10-14) — Teamtailor, Personio, Jobvite,
|
|
1632
|
+
* Breezy, Pinpoint, Rippling, iCIMS added; Dover dropped ──────────────────
|
|
1633
|
+
* Measured live via curl from a laptop AND a throwaway Worker (see
|
|
1634
|
+
* `_edge_probe_worker` note below). Endpoints:
|
|
1635
|
+
*
|
|
1636
|
+
* TEAMTAILOR GET {token}.teamtailor.com/jobs.json — a keyless JSON Feed
|
|
1637
|
+
* (jsonfeed.org v1.1). Unknown subdomain -> HTTP 404, Content-Type
|
|
1638
|
+
* application/json, body EMPTY (0 bytes, not even `{}`) — status is the
|
|
1639
|
+
* only signal, never parse-then-check. Confirmed real w/ jobs:
|
|
1640
|
+
* career.teamtailor.com (Teamtailor itself) and arsenalfc.teamtailor.com.
|
|
1641
|
+
* Each item carries a schema.org `_jobposting.jobLocation[0].address`;
|
|
1642
|
+
* no department/remote field in this feed.
|
|
1643
|
+
*
|
|
1644
|
+
* PERSONIO GET {token}.jobs.personio.com/xml?language=en, falling back to
|
|
1645
|
+
* .jobs.personio.de on a 404 — a company is hosted on exactly one of the
|
|
1646
|
+
* two TLDs and there is no redirect between them, confirmed live
|
|
1647
|
+
* (1nce -> .com; kertos -> .de; hometogo.com 404'd, never probed .de).
|
|
1648
|
+
* Unknown subdomain (either TLD) -> HTTP 404 text/html. Real feed is XML
|
|
1649
|
+
* (`<workzag-jobs><position>...`), parsed by regex (Workers have no
|
|
1650
|
+
* DOMParser) with basic entity decoding. No per-job URL field; this pack
|
|
1651
|
+
* builds one as `{token}.jobs.personio.{tld}/job/{id}?language=en`,
|
|
1652
|
+
* matching the pattern on every indexed Personio job link.
|
|
1653
|
+
*
|
|
1654
|
+
* JOBVITE GET jobs.jobvite.com/{token} — server-rendered HTML table
|
|
1655
|
+
* (`<td class="jv-job-list-name"><a href="/{token}/job/{code}">`), no
|
|
1656
|
+
* JSON API (the vendor's own feed is per-customer and off by default).
|
|
1657
|
+
* Unknown token -> HTTP 200 but 4 silent redirects to
|
|
1658
|
+
* www.jobvite.com/support/job-seeker-support/?invalid=1 — a DIFFERENT
|
|
1659
|
+
* host, confirmed live (`num_redirects:4`). `fetch()`'s default
|
|
1660
|
+
* `redirect:'follow'` makes this invisible unless you check
|
|
1661
|
+
* `res.redirected`; this pack treats any redirect as board_not_found.
|
|
1662
|
+
* Confirmed real w/ jobs: jobs.jobvite.com/bagleybagelbites (65 rows).
|
|
1663
|
+
*
|
|
1664
|
+
* BREEZY GET {token}.breezy.hr/json — keyless JSON array (not wrapped).
|
|
1665
|
+
* Unknown subdomain -> HTTP 404, Content-Type text/html, title "Career
|
|
1666
|
+
* portal not found | Breezy HR". Confirmed real w/ jobs: breezy.breezy.hr
|
|
1667
|
+
* and duolingo.breezy.hr.
|
|
1668
|
+
*
|
|
1669
|
+
* PINPOINT GET {token}.pinpointhq.com/postings.json — keyless JSON
|
|
1670
|
+
* `{data:[...]}`, per Pinpoint's own published docs. Unknown subdomain ->
|
|
1671
|
+
* HTTP 404, Content-Type text/html (Pinpoint's own 404 page, not JSON).
|
|
1672
|
+
* Confirmed real w/ jobs: workwithus.pinpointhq.com. A real board with
|
|
1673
|
+
* zero current postings would be HTTP 200 `{data:[]}` by the same
|
|
1674
|
+
* 200-success-envelope convention every sibling ATS here uses — NOT
|
|
1675
|
+
* independently confirmed live (every real board probed had postings),
|
|
1676
|
+
* flagged as such like Recruitee's empty-board case above.
|
|
1677
|
+
*
|
|
1678
|
+
* RIPPLING GET ats.rippling.com/{token}/jobs — a Next.js page with NO
|
|
1679
|
+
* separate JSON API; every guessed `/api/...` path 404'd. The job data is
|
|
1680
|
+
* server-rendered into a `<script id="__NEXT_DATA__" type="application/
|
|
1681
|
+
* json">` tag as hydrated react-query state — this pack parses that
|
|
1682
|
+
* script tag and reads the query whose `queryKey` is
|
|
1683
|
+
* `["board", token, "job-posts"]`. Unknown token -> HTTP 404,
|
|
1684
|
+
* Content-Type text/html, Next.js's own 404 page (no `__NEXT_DATA__`
|
|
1685
|
+
* job-posts query to find). Confirmed real w/ jobs:
|
|
1686
|
+
* ats.rippling.com/skillable-careers/jobs and .../rearc/jobs.
|
|
1687
|
+
*
|
|
1688
|
+
* ICIMS GET {token}.icims.com/jobs/search?ss=1&in_iframe=1 — iCIMS's
|
|
1689
|
+
* documented Talent-Cloud API is gated behind a partner application, but
|
|
1690
|
+
* the public career-SEARCH page itself is plain server-rendered HTML
|
|
1691
|
+
* with no login (`<li class="iCIMS_JobCardItem">` blocks: title via
|
|
1692
|
+
* `<h3>`, location via the "Job Locations" field, department via the
|
|
1693
|
+
* "Category" dt/dd pair) — parsed by regex, same category as this pack's
|
|
1694
|
+
* existing HTML scraping for ats_detect. `token` is the FULL subdomain
|
|
1695
|
+
* label a company's careers portal uses (e.g. "careers-intrepidsolutions"
|
|
1696
|
+
* — iCIMS customers don't share one fixed prefix). Unknown subdomain ->
|
|
1697
|
+
* HTTP 404, title "gone: {subdomain}.icims.com : dc409". Confirmed real
|
|
1698
|
+
* w/ jobs: careers-intrepidsolutions.icims.com (14 postings).
|
|
1699
|
+
*
|
|
1700
|
+
* DOVER — DROPPED. app.dover.com/jobs/{slug} is a client-rendered SPA
|
|
1701
|
+
* behind a Cloudflare Turnstile challenge (the page ships
|
|
1702
|
+
* `challenges.cloudflare.com/turnstile/v0/api.js`); its underlying jobs
|
|
1703
|
+
* API, app.dover.com/api/v1/jobs, answers HTTP 401 `{"detail":
|
|
1704
|
+
* "Authentication credentials were not provided."}` for every query-
|
|
1705
|
+
* param combination tried (bare, `?company=`, `?careers_page_slug=`) —
|
|
1706
|
+
* there is no discoverable keyless public endpoint, only a walled one.
|
|
1707
|
+
*
|
|
1383
1708
|
* ── Persistence (fleet #2754: never module scope) ──────────────────────────
|
|
1384
1709
|
* The gateway injects its one shared KV namespace as `_cacheKv`
|
|
1385
1710
|
* (`injectCacheKv: true` on this pack's pack-manifest.json row — same
|
|
@@ -1494,7 +1819,23 @@ export type AtsJobsResult = AtsJobsFound | AtsJobsNotFound;
|
|
|
1494
1819
|
// re-fetching boards themselves — see workers/scraper/src/subscriptions.ts'
|
|
1495
1820
|
// evaluateJobPost, fleet #2858) can validate/enumerate the same board types
|
|
1496
1821
|
// without duplicating this list.
|
|
1497
|
-
export const VALID_ATS = [
|
|
1822
|
+
export const VALID_ATS = [
|
|
1823
|
+
'greenhouse',
|
|
1824
|
+
'lever',
|
|
1825
|
+
'ashby',
|
|
1826
|
+
'smartrecruiters',
|
|
1827
|
+
'recruitee',
|
|
1828
|
+
'bamboohr',
|
|
1829
|
+
'workday',
|
|
1830
|
+
'workable',
|
|
1831
|
+
'teamtailor',
|
|
1832
|
+
'personio',
|
|
1833
|
+
'jobvite',
|
|
1834
|
+
'breezy',
|
|
1835
|
+
'pinpoint',
|
|
1836
|
+
'rippling',
|
|
1837
|
+
'icims',
|
|
1838
|
+
] as const;
|
|
1498
1839
|
export type Ats = (typeof VALID_ATS)[number];
|
|
1499
1840
|
|
|
1500
1841
|
function requireAts(v: unknown): Ats {
|
|
@@ -1804,6 +2145,267 @@ async function fetchWorkable(token: string): Promise<AtsJobsResult> {
|
|
|
1804
2145
|
return { found: true, ats: 'workable', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
1805
2146
|
}
|
|
1806
2147
|
|
|
2148
|
+
// -- Minimal entity decoder for regex-parsed HTML/XML (Workers have no DOMParser) --
|
|
2149
|
+
function decodeEntities(s: string): string {
|
|
2150
|
+
return s
|
|
2151
|
+
.replace(/&/g, '&')
|
|
2152
|
+
.replace(/</g, '<')
|
|
2153
|
+
.replace(/>/g, '>')
|
|
2154
|
+
.replace(/"/g, '"')
|
|
2155
|
+
.replace(/'|'/g, "'")
|
|
2156
|
+
.replace(/&#x([0-9a-f]+);/gi, (_, hex) => String.fromCodePoint(parseInt(hex, 16)))
|
|
2157
|
+
.replace(/&#(\d+);/g, (_, dec) => String.fromCodePoint(parseInt(dec, 10)))
|
|
2158
|
+
.trim();
|
|
2159
|
+
}
|
|
2160
|
+
|
|
2161
|
+
async function fetchTeamtailor(token: string): Promise<AtsJobsResult> {
|
|
2162
|
+
const sub = assertHostLabel(token, 'board_token');
|
|
2163
|
+
const res = await pwFetch(`https://${sub}.teamtailor.com/jobs.json`, { headers: { Accept: 'application/json' } });
|
|
2164
|
+
if (res.status === 404) {
|
|
2165
|
+
return { found: false, ats: 'teamtailor', board_token: token, reason: 'board_not_found', evidence: 'HTTP 404, empty body — Teamtailor serves no jobs.json for an unknown subdomain.', data_as_of: nowIso() };
|
|
2166
|
+
}
|
|
2167
|
+
if (!res.ok) {
|
|
2168
|
+
throw new Error(`Teamtailor board "${token}" returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2169
|
+
}
|
|
2170
|
+
const data = (await res.json()) as { items?: unknown[] };
|
|
2171
|
+
const jobs: NormalizedJob[] = (data.items ?? []).map((raw) => {
|
|
2172
|
+
const j = raw as Record<string, unknown>;
|
|
2173
|
+
const jp = j._jobposting as Record<string, unknown> | undefined;
|
|
2174
|
+
const locs = jp?.jobLocation as Array<Record<string, unknown>> | undefined;
|
|
2175
|
+
const addr = locs?.[0]?.address as Record<string, unknown> | undefined;
|
|
2176
|
+
const locationParts = [addr?.addressLocality, addr?.addressRegion].filter((v) => typeof v === 'string' && v);
|
|
2177
|
+
return {
|
|
2178
|
+
title: String(j.title ?? ''),
|
|
2179
|
+
location: locationParts.length ? locationParts.join(', ') : null,
|
|
2180
|
+
dept: null,
|
|
2181
|
+
remote: null,
|
|
2182
|
+
comp: null,
|
|
2183
|
+
posted_at: (j.date_published as string) ?? null,
|
|
2184
|
+
url: (j.url as string) ?? null,
|
|
2185
|
+
};
|
|
2186
|
+
});
|
|
2187
|
+
return { found: true, ats: 'teamtailor', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2188
|
+
}
|
|
2189
|
+
|
|
2190
|
+
function parsePersonioXml(xml: string, token: string, tld: 'com' | 'de'): NormalizedJob[] {
|
|
2191
|
+
const positions = xml.match(/<position>[\s\S]*?<\/position>/g) ?? [];
|
|
2192
|
+
return positions.map((block) => {
|
|
2193
|
+
const field = (name: string): string | null => {
|
|
2194
|
+
const m = new RegExp(`<${name}>([\\s\\S]*?)<\\/${name}>`).exec(block);
|
|
2195
|
+
return m ? decodeEntities(m[1]) : null;
|
|
2196
|
+
};
|
|
2197
|
+
const id = field('id');
|
|
2198
|
+
const office = field('office');
|
|
2199
|
+
const additional = Array.from(block.matchAll(/<additionalOffices>([\s\S]*?)<\/additionalOffices>/g)).flatMap((m) =>
|
|
2200
|
+
Array.from(m[1].matchAll(/<office>([\s\S]*?)<\/office>/g)).map((o) => decodeEntities(o[1])),
|
|
2201
|
+
);
|
|
2202
|
+
const location = [office, ...additional].filter(Boolean).join(', ') || null;
|
|
2203
|
+
return {
|
|
2204
|
+
title: field('name') ?? '',
|
|
2205
|
+
location,
|
|
2206
|
+
dept: field('department'),
|
|
2207
|
+
remote: location ? /remote/i.test(location) : null,
|
|
2208
|
+
comp: null,
|
|
2209
|
+
posted_at: field('createdAt'),
|
|
2210
|
+
url: id ? `https://${token}.jobs.personio.${tld}/job/${id}?language=en` : null,
|
|
2211
|
+
};
|
|
2212
|
+
});
|
|
2213
|
+
}
|
|
2214
|
+
|
|
2215
|
+
async function fetchPersonio(token: string): Promise<AtsJobsResult> {
|
|
2216
|
+
const sub = assertHostLabel(token, 'board_token');
|
|
2217
|
+
for (const tld of ['com', 'de'] as const) {
|
|
2218
|
+
const res = await pwFetch(`https://${sub}.jobs.personio.${tld}/xml?language=en`, { headers: { Accept: 'text/xml' } });
|
|
2219
|
+
if (res.status === 404) continue; // try the other TLD — a company lives on exactly one
|
|
2220
|
+
if (!res.ok) {
|
|
2221
|
+
throw new Error(`Personio board "${token}" (.${tld}) returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2222
|
+
}
|
|
2223
|
+
const xml = await res.text();
|
|
2224
|
+
const jobs = parsePersonioXml(xml, sub, tld);
|
|
2225
|
+
return { found: true, ats: 'personio', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2226
|
+
}
|
|
2227
|
+
return { found: false, ats: 'personio', board_token: token, reason: 'board_not_found', evidence: 'HTTP 404 on both jobs.personio.com and jobs.personio.de — Personio hosts a company on exactly one TLD with no redirect between them, and neither had this subdomain.', data_as_of: nowIso() };
|
|
2228
|
+
}
|
|
2229
|
+
|
|
2230
|
+
async function fetchJobvite(token: string): Promise<AtsJobsResult> {
|
|
2231
|
+
const slug = assertHostLabel(token, 'board_token');
|
|
2232
|
+
const res = await pwFetch(`https://jobs.jobvite.com/${slug}`, { headers: { Accept: 'text/html' } });
|
|
2233
|
+
if (res.redirected) {
|
|
2234
|
+
// CONFIRMED LIVE: an unknown slug 4-hop redirects to
|
|
2235
|
+
// www.jobvite.com/support/job-seeker-support/?invalid=1 — a different
|
|
2236
|
+
// host — rather than answering 404 on jobs.jobvite.com itself.
|
|
2237
|
+
return { found: false, ats: 'jobvite', board_token: token, reason: 'board_not_found', evidence: `Request redirected to ${res.url} instead of serving the board — Jobvite's "invalid slug" signal.`, data_as_of: nowIso() };
|
|
2238
|
+
}
|
|
2239
|
+
if (!res.ok) {
|
|
2240
|
+
throw new Error(`Jobvite board "${token}" returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2241
|
+
}
|
|
2242
|
+
const html = await res.text();
|
|
2243
|
+
const rows = Array.from(
|
|
2244
|
+
html.matchAll(/<tr>\s*<td class="jv-job-list-name">\s*<a href="([^"]+)"[^>]*>([^<]*)<\/a>\s*<\/td>\s*<td class="jv-job-list-location">([\s\S]*?)<\/td>\s*<\/tr>/g),
|
|
2245
|
+
);
|
|
2246
|
+
const jobs: NormalizedJob[] = rows.map((m) => ({
|
|
2247
|
+
title: decodeEntities(m[2]),
|
|
2248
|
+
location: decodeEntities(m[3].replace(/\s+/g, ' ')) || null,
|
|
2249
|
+
dept: null,
|
|
2250
|
+
remote: null,
|
|
2251
|
+
comp: null,
|
|
2252
|
+
posted_at: null,
|
|
2253
|
+
url: m[1].startsWith('http') ? m[1] : `https://jobs.jobvite.com${m[1]}`,
|
|
2254
|
+
}));
|
|
2255
|
+
return { found: true, ats: 'jobvite', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2256
|
+
}
|
|
2257
|
+
|
|
2258
|
+
async function fetchBreezy(token: string): Promise<AtsJobsResult> {
|
|
2259
|
+
const sub = assertHostLabel(token, 'board_token');
|
|
2260
|
+
const res = await pwFetch(`https://${sub}.breezy.hr/json`, { headers: { Accept: 'application/json' } });
|
|
2261
|
+
if (res.status === 404) {
|
|
2262
|
+
return { found: false, ats: 'breezy', board_token: token, reason: 'board_not_found', evidence: 'HTTP 404, Content-Type text/html — Breezy: "Career portal not found".', data_as_of: nowIso() };
|
|
2263
|
+
}
|
|
2264
|
+
if (!res.ok) {
|
|
2265
|
+
throw new Error(`Breezy board "${token}" returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2266
|
+
}
|
|
2267
|
+
const data = (await res.json()) as unknown[];
|
|
2268
|
+
const jobs: NormalizedJob[] = (Array.isArray(data) ? data : []).map((raw) => {
|
|
2269
|
+
const j = raw as Record<string, unknown>;
|
|
2270
|
+
const location = j.location as Record<string, unknown> | undefined;
|
|
2271
|
+
const salary = typeof j.salary === 'string' ? j.salary.trim() : '';
|
|
2272
|
+
return {
|
|
2273
|
+
title: String(j.name ?? ''),
|
|
2274
|
+
location: (location?.name as string) ?? null,
|
|
2275
|
+
dept: (j.department as string) ?? null,
|
|
2276
|
+
remote: typeof location?.is_remote === 'boolean' ? (location.is_remote as boolean) : null,
|
|
2277
|
+
comp: salary || null,
|
|
2278
|
+
posted_at: (j.published_date as string) ?? null,
|
|
2279
|
+
url: (j.url as string) ?? null,
|
|
2280
|
+
};
|
|
2281
|
+
});
|
|
2282
|
+
return { found: true, ats: 'breezy', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2283
|
+
}
|
|
2284
|
+
|
|
2285
|
+
async function fetchPinpoint(token: string): Promise<AtsJobsResult> {
|
|
2286
|
+
const sub = assertHostLabel(token, 'board_token');
|
|
2287
|
+
const res = await pwFetch(`https://${sub}.pinpointhq.com/postings.json`, { headers: { Accept: 'application/json' } });
|
|
2288
|
+
if (res.status === 404) {
|
|
2289
|
+
return { found: false, ats: 'pinpoint', board_token: token, reason: 'board_not_found', evidence: 'HTTP 404, Content-Type text/html — Pinpoint\'s own 404 page, not a JSON response.', data_as_of: nowIso() };
|
|
2290
|
+
}
|
|
2291
|
+
if (!res.ok) {
|
|
2292
|
+
throw new Error(`Pinpoint board "${token}" returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2293
|
+
}
|
|
2294
|
+
const data = (await res.json()) as { data?: unknown[] };
|
|
2295
|
+
const jobs: NormalizedJob[] = (data.data ?? []).map((raw) => {
|
|
2296
|
+
const j = raw as Record<string, unknown>;
|
|
2297
|
+
const location = j.location as Record<string, unknown> | undefined;
|
|
2298
|
+
const job = j.job as Record<string, unknown> | undefined;
|
|
2299
|
+
const dept = job?.department as Record<string, unknown> | undefined;
|
|
2300
|
+
const workplaceType = typeof j.workplace_type === 'string' ? j.workplace_type.toLowerCase() : undefined;
|
|
2301
|
+
const min = j.compensation_minimum as number | undefined;
|
|
2302
|
+
const max = j.compensation_maximum as number | undefined;
|
|
2303
|
+
const currency = j.compensation_currency as string | undefined;
|
|
2304
|
+
const comp = j.compensation_visible && (min || max) ? `${currency ?? ''} ${min ?? ''}${max ? `-${max}` : ''}`.trim() : null;
|
|
2305
|
+
return {
|
|
2306
|
+
title: String(j.title ?? ''),
|
|
2307
|
+
location: (location?.city as string) ?? (location?.name as string) ?? null,
|
|
2308
|
+
dept: (dept?.name as string) ?? null,
|
|
2309
|
+
remote: workplaceType ? workplaceType === 'remote' : null,
|
|
2310
|
+
comp,
|
|
2311
|
+
posted_at: null, // not confirmed live — Pinpoint's postings.json carries no date field in any probed response
|
|
2312
|
+
url: (j.url as string) ?? null,
|
|
2313
|
+
};
|
|
2314
|
+
});
|
|
2315
|
+
// Not independently confirmed live for a real board with ZERO current
|
|
2316
|
+
// postings (every probed board had jobs); documented by analogy with every
|
|
2317
|
+
// sibling ATS's 200-success-envelope convention, same as Recruitee above.
|
|
2318
|
+
return { found: true, ats: 'pinpoint', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2319
|
+
}
|
|
2320
|
+
|
|
2321
|
+
interface RippleQuery {
|
|
2322
|
+
queryKey?: unknown[];
|
|
2323
|
+
state?: { data?: { items?: unknown[] } };
|
|
2324
|
+
}
|
|
2325
|
+
|
|
2326
|
+
function parseRipplingNextData(html: string): unknown[] | null {
|
|
2327
|
+
const m = /<script id="__NEXT_DATA__" type="application\/json">([\s\S]*?)<\/script>/.exec(html);
|
|
2328
|
+
if (!m) return null;
|
|
2329
|
+
let parsed: unknown;
|
|
2330
|
+
try {
|
|
2331
|
+
parsed = JSON.parse(m[1]);
|
|
2332
|
+
} catch {
|
|
2333
|
+
return null;
|
|
2334
|
+
}
|
|
2335
|
+
const queries = (parsed as { props?: { pageProps?: { dehydratedState?: { queries?: RippleQuery[] } } } })?.props?.pageProps?.dehydratedState?.queries ?? [];
|
|
2336
|
+
const jobPosts = queries.find((q) => Array.isArray(q.queryKey) && q.queryKey[0] === 'board' && q.queryKey[2] === 'job-posts');
|
|
2337
|
+
return jobPosts?.state?.data?.items ?? null;
|
|
2338
|
+
}
|
|
2339
|
+
|
|
2340
|
+
async function fetchRippling(token: string): Promise<AtsJobsResult> {
|
|
2341
|
+
const slug = assertHostLabel(token, 'board_token');
|
|
2342
|
+
const res = await pwFetch(`https://ats.rippling.com/${slug}/jobs`, { headers: { Accept: 'text/html' } });
|
|
2343
|
+
if (res.status === 404) {
|
|
2344
|
+
return { found: false, ats: 'rippling', board_token: token, reason: 'board_not_found', evidence: 'HTTP 404 — Rippling\'s own Next.js 404 page, no __NEXT_DATA__ job-posts query present.', data_as_of: nowIso() };
|
|
2345
|
+
}
|
|
2346
|
+
if (!res.ok) {
|
|
2347
|
+
throw new Error(`Rippling board "${token}" returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2348
|
+
}
|
|
2349
|
+
const html = await res.text();
|
|
2350
|
+
const items = parseRipplingNextData(html);
|
|
2351
|
+
if (items === null) {
|
|
2352
|
+
throw new Error(`Rippling board "${token}" returned HTTP 200 but no parseable __NEXT_DATA__ job-posts query was found — the page shape may have changed.`);
|
|
2353
|
+
}
|
|
2354
|
+
const jobs: NormalizedJob[] = items.map((raw) => {
|
|
2355
|
+
const j = raw as Record<string, unknown>;
|
|
2356
|
+
const dept = j.department as Record<string, unknown> | undefined;
|
|
2357
|
+
const locs = j.locations as Array<Record<string, unknown>> | undefined;
|
|
2358
|
+
const loc0 = locs?.[0];
|
|
2359
|
+
return {
|
|
2360
|
+
title: String(j.name ?? ''),
|
|
2361
|
+
location: (loc0?.name as string) ?? null,
|
|
2362
|
+
dept: (dept?.name as string) ?? null,
|
|
2363
|
+
remote: typeof loc0?.workplaceType === 'string' ? (loc0.workplaceType as string).toUpperCase() === 'REMOTE' : null,
|
|
2364
|
+
comp: null,
|
|
2365
|
+
posted_at: null,
|
|
2366
|
+
url: (j.url as string) ?? null,
|
|
2367
|
+
};
|
|
2368
|
+
});
|
|
2369
|
+
return { found: true, ats: 'rippling', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2370
|
+
}
|
|
2371
|
+
|
|
2372
|
+
function parseIcimsCards(html: string): NormalizedJob[] {
|
|
2373
|
+
const cards = html.split('<li class="iCIMS_JobCardItem">').slice(1);
|
|
2374
|
+
return cards.map((card) => {
|
|
2375
|
+
const titleMatch = /<h3[^>]*>\s*([^<]+?)\s*<\/h3>/.exec(card);
|
|
2376
|
+
const urlMatch = /<a href="([^"]+)" class="iCIMS_Anchor"/.exec(card);
|
|
2377
|
+
const locMatch = /Job Locations<\/span>\s*<span[^>]*>\s*([^<]*?)\s*<\/span>/.exec(card);
|
|
2378
|
+
const deptMatch = /<dt class="iCIMS_JobHeaderField">Category<\/dt>\s*<dd class="iCIMS_JobHeaderData"><span[^>]*>\s*([^<]*?)\s*<\/span>/.exec(card);
|
|
2379
|
+
const rawUrl = urlMatch ? urlMatch[1] : null;
|
|
2380
|
+
return {
|
|
2381
|
+
title: titleMatch ? decodeEntities(titleMatch[1]) : '',
|
|
2382
|
+
location: locMatch && locMatch[1] ? decodeEntities(locMatch[1]) : null,
|
|
2383
|
+
dept: deptMatch && deptMatch[1] ? decodeEntities(deptMatch[1]) : null,
|
|
2384
|
+
remote: null,
|
|
2385
|
+
comp: null,
|
|
2386
|
+
posted_at: null,
|
|
2387
|
+
url: rawUrl ? rawUrl.replace(/\?in_iframe=1$/, '') : null,
|
|
2388
|
+
};
|
|
2389
|
+
});
|
|
2390
|
+
}
|
|
2391
|
+
|
|
2392
|
+
async function fetchIcims(token: string): Promise<AtsJobsResult> {
|
|
2393
|
+
const sub = assertHostLabel(token, 'board_token');
|
|
2394
|
+
const res = await pwFetch(`https://${sub}.icims.com/jobs/search?ss=1&in_iframe=1`, { headers: { Accept: 'text/html' } });
|
|
2395
|
+
if (res.status === 404) {
|
|
2396
|
+
return { found: false, ats: 'icims', board_token: token, reason: 'board_not_found', evidence: `HTTP 404 — iCIMS: "gone: ${sub}.icims.com : dc409".`, data_as_of: nowIso() };
|
|
2397
|
+
}
|
|
2398
|
+
if (!res.ok) {
|
|
2399
|
+
throw new Error(`iCIMS board "${token}" returned HTTP ${res.status}: ${summarizeErrorBody(await res.text())}`);
|
|
2400
|
+
}
|
|
2401
|
+
const html = await res.text();
|
|
2402
|
+
const jobs = parseIcimsCards(html);
|
|
2403
|
+
// Not independently confirmed live for a real portal with ZERO current
|
|
2404
|
+
// postings — every probed portal had jobs. Documented by analogy, same
|
|
2405
|
+
// disclaimer as Pinpoint/Recruitee above.
|
|
2406
|
+
return { found: true, ats: 'icims', board_token: token, total: jobs.length, jobs, data_as_of: nowIso() };
|
|
2407
|
+
}
|
|
2408
|
+
|
|
1807
2409
|
async function fetchRaw(ats: Ats, token: string, filters: Record<string, unknown> = {}): Promise<AtsJobsResult> {
|
|
1808
2410
|
switch (ats) {
|
|
1809
2411
|
case 'greenhouse':
|
|
@@ -1822,6 +2424,20 @@ async function fetchRaw(ats: Ats, token: string, filters: Record<string, unknown
|
|
|
1822
2424
|
return fetchWorkday(token, filters as { title_query?: string; limit?: number; offset?: number });
|
|
1823
2425
|
case 'workable':
|
|
1824
2426
|
return fetchWorkable(token);
|
|
2427
|
+
case 'teamtailor':
|
|
2428
|
+
return fetchTeamtailor(token);
|
|
2429
|
+
case 'personio':
|
|
2430
|
+
return fetchPersonio(token);
|
|
2431
|
+
case 'jobvite':
|
|
2432
|
+
return fetchJobvite(token);
|
|
2433
|
+
case 'breezy':
|
|
2434
|
+
return fetchBreezy(token);
|
|
2435
|
+
case 'pinpoint':
|
|
2436
|
+
return fetchPinpoint(token);
|
|
2437
|
+
case 'rippling':
|
|
2438
|
+
return fetchRippling(token);
|
|
2439
|
+
case 'icims':
|
|
2440
|
+
return fetchIcims(token);
|
|
1825
2441
|
}
|
|
1826
2442
|
}
|
|
1827
2443
|
|
|
@@ -1873,6 +2489,13 @@ function scanHtmlForAts(html: string): DetectCandidate[] {
|
|
|
1873
2489
|
{ ats: 'bamboohr', re: /([a-z0-9][a-z0-9-]{0,61})\.bamboohr\.com/i },
|
|
1874
2490
|
{ ats: 'workable', re: /apply\.workable\.com\/(?:api\/v\d\/widget\/accounts\/|j\/)?([a-z0-9][a-z0-9_-]{0,63})/i },
|
|
1875
2491
|
{ ats: 'workday', re: /([a-z0-9-]+)\.(wd\d{1,2})\.myworkdayjobs\.com\/(?:[a-z]{2}-[A-Z]{2}\/)?([A-Za-z0-9_.-]+)/i },
|
|
2492
|
+
{ ats: 'teamtailor', re: /([a-z0-9][a-z0-9-]{0,61})\.teamtailor\.com/i },
|
|
2493
|
+
{ ats: 'personio', re: /([a-z0-9][a-z0-9-]{0,61})\.jobs\.personio\.(?:com|de)/i },
|
|
2494
|
+
{ ats: 'jobvite', re: /jobs\.jobvite\.com\/([a-z0-9][a-z0-9_-]{0,63})/i },
|
|
2495
|
+
{ ats: 'breezy', re: /([a-z0-9][a-z0-9-]{0,61})\.breezy\.hr/i },
|
|
2496
|
+
{ ats: 'pinpoint', re: /([a-z0-9][a-z0-9-]{0,61})\.pinpointhq\.com/i },
|
|
2497
|
+
{ ats: 'rippling', re: /ats\.rippling\.com\/([a-z0-9][a-z0-9_-]{0,63})/i },
|
|
2498
|
+
{ ats: 'icims', re: /([a-z0-9][a-z0-9-]{0,61})\.icims\.com/i },
|
|
1876
2499
|
];
|
|
1877
2500
|
for (const { ats, re } of patterns) {
|
|
1878
2501
|
const m = re.exec(html);
|
|
@@ -1952,7 +2575,7 @@ async function atsDetect(args: Record<string, unknown>, kv: KvLike | undefined):
|
|
|
1952
2575
|
|
|
1953
2576
|
if (domain) {
|
|
1954
2577
|
const slug = domain.split('.')[0];
|
|
1955
|
-
const probeOrder: Ats[] = ['greenhouse', 'lever', 'ashby', 'smartrecruiters', 'recruitee', 'bamboohr'];
|
|
2578
|
+
const probeOrder: Ats[] = ['greenhouse', 'lever', 'ashby', 'smartrecruiters', 'recruitee', 'bamboohr', 'teamtailor', 'personio', 'breezy', 'pinpoint'];
|
|
1956
2579
|
for (const ats of probeOrder) {
|
|
1957
2580
|
if (await probeSlug(ats, slug)) {
|
|
1958
2581
|
const result = {
|
|
@@ -1972,7 +2595,7 @@ async function atsDetect(args: Record<string, unknown>, kv: KvLike | undefined):
|
|
|
1972
2595
|
reason: 'no_ats_detected',
|
|
1973
2596
|
checked_scrape: candidates.map((c) => `${c.ats}:${c.board_token}`),
|
|
1974
2597
|
checked_probe: probeOrder.map((a) => `${a}:${slug}`),
|
|
1975
|
-
note: 'Scraped the careers page and the homepage for ATS links, then slug-probed Greenhouse/Lever/Ashby/SmartRecruiters/Recruitee/BambooHR using the domain label as the board token. None matched. Workday needs a tenant + wd-number + site slug that cannot be guessed from a domain alone, so it is not blind-probed. Workable is not blind-probed either — its public widget is currently blocked by Cloudflare from this platform\'s edge (see ats_jobs with ats:"workable" for the measured 429).',
|
|
2598
|
+
note: 'Scraped the careers page and the homepage for ATS links, then slug-probed Greenhouse/Lever/Ashby/SmartRecruiters/Recruitee/BambooHR/Teamtailor/Personio/Breezy/Pinpoint using the domain label as the board token. None matched. Workday needs a tenant + wd-number + site slug that cannot be guessed from a domain alone, so it is not blind-probed. Jobvite, Rippling and iCIMS use slugs/subdomain prefixes that often do not match the bare domain label either, so they are not blind-probed — pass board_token directly if you already know it. Workable is not blind-probed either — its public widget is currently blocked by Cloudflare from this platform\'s edge (see ats_jobs with ats:"workable" for the measured 429).',
|
|
1976
2599
|
data_as_of: nowIso(),
|
|
1977
2600
|
};
|
|
1978
2601
|
}
|
|
@@ -1986,7 +2609,7 @@ const tools: McpToolExport['tools'] = [
|
|
|
1986
2609
|
{
|
|
1987
2610
|
name: 'ats_detect',
|
|
1988
2611
|
description:
|
|
1989
|
-
'Detect which applicant-tracking system (ATS) a company publishes its public job board on, and its board token. Reads the company\'s careers page (or a given careers_url) for a Greenhouse/Lever/Ashby/SmartRecruiters/Recruitee/BambooHR/Workday/Workable link; falls back to slug-probing the domain name against each board API. Does not read any private/Harvest-key ATS data — only what a public job board already exposes. Discoveries persist so repeat lookups for the same domain are instant.',
|
|
2612
|
+
'Detect which applicant-tracking system (ATS) a company publishes its public job board on, and its board token. Reads the company\'s careers page (or a given careers_url) for a Greenhouse/Lever/Ashby/SmartRecruiters/Recruitee/BambooHR/Workday/Workable/Teamtailor/Personio/Jobvite/Breezy/Pinpoint/Rippling/iCIMS link; falls back to slug-probing the domain name against each board API that supports it. Does not read any private/Harvest-key ATS data — only what a public job board already exposes. Discoveries persist so repeat lookups for the same domain are instant.',
|
|
1990
2613
|
inputSchema: {
|
|
1991
2614
|
type: 'object' as const,
|
|
1992
2615
|
properties: {
|
|
@@ -1998,7 +2621,7 @@ const tools: McpToolExport['tools'] = [
|
|
|
1998
2621
|
{
|
|
1999
2622
|
name: 'ats_jobs',
|
|
2000
2623
|
description:
|
|
2001
|
-
'Fetch normalized job postings from a PUBLIC ATS board you already have the ats + board_token for (use ats_detect or ats_company_jobs if you only have a domain). Returns {found:false, reason:"board_not_found"} for an unknown board — never an empty jobs[] for that case. Lever/Ashby/BambooHR/Recruitee DO distinguish a real zero-job board (found:true, jobs:[]) from an unknown one; SmartRecruiters cannot
|
|
2624
|
+
'Fetch normalized job postings from a PUBLIC ATS board you already have the ats + board_token for (use ats_detect or ats_company_jobs if you only have a domain). Returns {found:false, reason:"board_not_found"} for an unknown board — never an empty jobs[] for that case. Lever/Ashby/BambooHR/Recruitee/Teamtailor/Breezy DO distinguish a real zero-job board (found:true, jobs:[]) from an unknown one; SmartRecruiters cannot (ambiguous:true); Pinpoint and iCIMS have not had a confirmed-empty real board observed live, so a 0-job result from those two is NOT independently confirmed to mean "real board, no jobs" versus a mis-detected token. Workday board_token is "<tenant>.wd<N>/<site>" (e.g. "workday.wd5/Workday"); its response states capped:true if your limit/offset hit Workday\'s own pagination limits (limit<=20, offset<=10000). Workable is currently blocked by Cloudflare from this platform\'s edge and will throw a loud error naming the 429, not an empty result. Dover is NOT supported: its public board is a Cloudflare-Turnstile-gated SPA with no keyless jobs endpoint.',
|
|
2002
2625
|
inputSchema: {
|
|
2003
2626
|
type: 'object' as const,
|
|
2004
2627
|
properties: {
|
|
@@ -2006,7 +2629,7 @@ const tools: McpToolExport['tools'] = [
|
|
|
2006
2629
|
board_token: {
|
|
2007
2630
|
type: 'string',
|
|
2008
2631
|
description:
|
|
2009
|
-
'The board identifier: Greenhouse/Lever/Ashby/SmartRecruiters company slug or
|
|
2632
|
+
'The board identifier: Greenhouse/Lever/Ashby/SmartRecruiters/Teamtailor/Personio/Breezy/Pinpoint company slug or subdomain label; Recruitee/BambooHR subdomain label; Jobvite slug (path segment after jobs.jobvite.com/); Rippling slug (path segment after ats.rippling.com/); iCIMS full subdomain label before .icims.com (e.g. "careers-acme" — prefixes vary by customer); Workable account slug; Workday "<tenant>.wd<N>/<site>".',
|
|
2010
2633
|
},
|
|
2011
2634
|
filters: {
|
|
2012
2635
|
type: 'object',
|
package/src/server.ts
CHANGED
|
@@ -8,8 +8,9 @@ import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
|
|
|
8
8
|
import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js';
|
|
9
9
|
import pack from './index.js';
|
|
10
10
|
|
|
11
|
+
|
|
11
12
|
const server = new Server(
|
|
12
|
-
{ name: '@pipeworx/mcp-ats-boards', version: '0.1.
|
|
13
|
+
{ name: '@pipeworx/mcp-ats-boards', version: '0.1.2' },
|
|
13
14
|
{ capabilities: { tools: {} } },
|
|
14
15
|
);
|
|
15
16
|
|
|
@@ -24,7 +25,9 @@ server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
|
24
25
|
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
25
26
|
const { name, arguments: args } = request.params;
|
|
26
27
|
try {
|
|
27
|
-
const
|
|
28
|
+
const callArgs = { ...((args ?? {}) as Record<string, unknown>) };
|
|
29
|
+
|
|
30
|
+
const result = await pack.callTool(name, callArgs);
|
|
28
31
|
return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] };
|
|
29
32
|
} catch (err) {
|
|
30
33
|
return {
|