plainstamp 0.7.3 → 0.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/dist/index.d.ts +2 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -0
- package/dist/index.js.map +1 -1
- package/dist/watcher/index.d.ts +1 -1
- package/dist/watcher/index.d.ts.map +1 -1
- package/dist/watcher/index.js +1 -1
- package/dist/watcher/index.js.map +1 -1
- package/dist/watcher/sources/url-monitor.d.ts +33 -0
- package/dist/watcher/sources/url-monitor.d.ts.map +1 -1
- package/dist/watcher/sources/url-monitor.js +62 -3
- package/dist/watcher/sources/url-monitor.js.map +1 -1
- package/docs/guides/eeoc-title-vii-ai-employment-builder-guide.md +274 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -16,6 +16,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
|
16
16
|
|
|
17
17
|
Distribution is **npm-only**. Source remains in the operating organization's private repository; there is no public source repository host. Contact channel for issues, accuracy reports, security reports, and contribution proposals is **helpfulbutton140@agentmail.to** (see `docs/CONTRIBUTING.md`, `docs/SECURITY.md`).
|
|
18
18
|
|
|
19
|
+
## [0.7.5] — 2026-05-09
|
|
20
|
+
|
|
21
|
+
### Fixed (URL-monitor source stabilization)
|
|
22
|
+
|
|
23
|
+
- `urlMonitorSource` now hashes a normalized version of the page body via the new `normalizeForHash(html)` helper, instead of the raw response. The normalization strips dynamic per-fetch markers that were causing false positives in the daily watcher cron: `<script>` and `<style>` blocks (nonces, build hashes, telemetry); HTML comments (often timestamps); CSRF / authenticity / `_token` / `requestverification` hidden inputs; inline `nonce`, `integrity`, `data-csrf`, `data-token`, `data-nonce`, `data-build`, and `data-version` attribute values; timestamp-bearing `<meta>` tags (`og:updated_time`, `last-modified`, `revised`, `build-time`, `generated-at`, `page-date`); whitespace runs collapsed.
|
|
24
|
+
- Two fetches of the same regulator-published page now hash identically as long as the substantive text and structure are unchanged.
|
|
25
|
+
- `Article.extra` now also carries `normalized_length` alongside `content_hash` and `content_length` for audit.
|
|
26
|
+
- New export from package root: `normalizeForHash`.
|
|
27
|
+
- Tests: 58/58 passing (added 7 normalization-stability tests).
|
|
28
|
+
|
|
29
|
+
## [0.7.4] — 2026-05-08
|
|
30
|
+
|
|
31
|
+
### Fixed (root re-exports for watcher API)
|
|
32
|
+
|
|
33
|
+
- Re-export the watcher's public surface (`diffArticles`, `runWatcher`, `runWatcherWithStore`, `readState`, `writeState`, `fsStateStore`, `memoryStateStore`, source factories, and the `Article` / `Source` / `RunReport` / `SourceRunReport` / `StateStore` / `WatcherState` types) from the package root. Previously these were only available via the deep `plainstamp/dist/watcher/index.js` import path, which broke type resolution in some consumers (notably the `plainstamp-cf-worker` Cloudflare Workers package). Now `import { runWatcherWithStore, type StateStore } from "plainstamp"` works.
|
|
34
|
+
|
|
19
35
|
## [0.7.3] — 2026-05-08
|
|
20
36
|
|
|
21
37
|
### Added (cross-runtime watcher)
|
package/dist/index.d.ts
CHANGED
|
@@ -4,6 +4,8 @@ export { computeCoverageMatrix, renderCoverageMarkdown, renderCoverageCsv, type
|
|
|
4
4
|
export type { DisclosureRuleT, RuleSetT, LookupQueryT, LookupResultT, ChannelT, UseCaseT, SeverityT, JurisdictionIdT, DisclosureElementT, } from "./schema.js";
|
|
5
5
|
export { Channel, UseCase, Severity, JurisdictionId, LookupQuery, DisclosureElement, DisclosureRule, RuleSet, } from "./schema.js";
|
|
6
6
|
export { mcpTools, executeMcpTool, type McpToolDescriptor, type McpToolResult, } from "./mcp-tools.js";
|
|
7
|
+
export { diffArticles, runWatcher, runWatcherWithStore, readState, writeState, fsStateStore, memoryStateStore, federalRegisterSource, urlMonitorSource, rulesCitationsUrlMonitor, hashContent, normalizeForHash, } from "./watcher/index.js";
|
|
8
|
+
export type { Article, Source, RunReport, SourceRunReport, StateStore, WatcherState, } from "./watcher/index.js";
|
|
7
9
|
import type { LookupQueryT, LookupResultT, DisclosureRuleT } from "./schema.js";
|
|
8
10
|
/**
|
|
9
11
|
* High-level convenience: load the bundled rules and look up disclosures for
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,MAAM,EACN,sBAAsB,EACtB,kBAAkB,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,gBAAgB,EAChB,iBAAiB,EACjB,eAAe,GAChB,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EACL,qBAAqB,EACrB,sBAAsB,EACtB,iBAAiB,EACjB,KAAK,cAAc,EACnB,KAAK,YAAY,GAClB,MAAM,eAAe,CAAC;AACvB,YAAY,EACV,eAAe,EACf,QAAQ,EACR,YAAY,EACZ,aAAa,EACb,QAAQ,EACR,QAAQ,EACR,SAAS,EACT,eAAe,EACf,kBAAkB,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,OAAO,EACP,OAAO,EACP,QAAQ,EACR,cAAc,EACd,WAAW,EACX,iBAAiB,EACjB,cAAc,EACd,OAAO,GACR,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,QAAQ,EACR,cAAc,EACd,KAAK,iBAAiB,EACtB,KAAK,aAAa,GACnB,MAAM,gBAAgB,CAAC;
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,MAAM,EACN,sBAAsB,EACtB,kBAAkB,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,gBAAgB,EAChB,iBAAiB,EACjB,eAAe,GAChB,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EACL,qBAAqB,EACrB,sBAAsB,EACtB,iBAAiB,EACjB,KAAK,cAAc,EACnB,KAAK,YAAY,GAClB,MAAM,eAAe,CAAC;AACvB,YAAY,EACV,eAAe,EACf,QAAQ,EACR,YAAY,EACZ,aAAa,EACb,QAAQ,EACR,QAAQ,EACR,SAAS,EACT,eAAe,EACf,kBAAkB,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,OAAO,EACP,OAAO,EACP,QAAQ,EACR,cAAc,EACd,WAAW,EACX,iBAAiB,EACjB,cAAc,EACd,OAAO,GACR,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,QAAQ,EACR,cAAc,EACd,KAAK,iBAAiB,EACtB,KAAK,aAAa,GACnB,MAAM,gBAAgB,CAAC;AACxB,OAAO,EACL,YAAY,EACZ,UAAU,EACV,mBAAmB,EACnB,SAAS,EACT,UAAU,EACV,YAAY,EACZ,gBAAgB,EAChB,qBAAqB,EACrB,gBAAgB,EAChB,wBAAwB,EACxB,WAAW,EACX,gBAAgB,GACjB,MAAM,oBAAoB,CAAC;AAC5B,YAAY,EACV,OAAO,EACP,MAAM,EACN,SAAS,EACT,eAAe,EACf,UAAU,EACV,YAAY,GACb,MAAM,oBAAoB,CAAC;AAI5B,OAAO,KAAK,EAAE,YAAY,EAAE,aAAa,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAEhF;;;;GAIG;AACH,wBAAgB,cAAc,CAAC,KAAK,EAAE,YAAY,GAAG,aAAa,EAAE,CAEnE;AAED,2EAA2E;AAC3E,wBAAgB,iBAAiB,IAAI,MAAM,EAAE,CAK5C;AAED,uDAAuD;AACvD,wBAAgB,WAAW,CAAC,EAAE,EAAE,MAAM,GAAG,eAAe,GAAG,SAAS,CAGnE;AAED;;;;GAIG;AACH,wBAAgB,0BAA0B,CACxC,KAAK,EAAE,YAAY,EACnB,aAAa,EAAE,MAAM;;;;;IAKtB"}
|
package/dist/index.js
CHANGED
|
@@ -3,6 +3,7 @@ export { loadBundledRules, loadRulesFromPath, setBundledRules, } from "./rules-l
|
|
|
3
3
|
export { computeCoverageMatrix, renderCoverageMarkdown, renderCoverageCsv, } from "./coverage.js";
|
|
4
4
|
export { Channel, UseCase, Severity, JurisdictionId, LookupQuery, DisclosureElement, DisclosureRule, RuleSet, } from "./schema.js";
|
|
5
5
|
export { mcpTools, executeMcpTool, } from "./mcp-tools.js";
|
|
6
|
+
export { diffArticles, runWatcher, runWatcherWithStore, readState, writeState, fsStateStore, memoryStateStore, federalRegisterSource, urlMonitorSource, rulesCitationsUrlMonitor, hashContent, normalizeForHash, } from "./watcher/index.js";
|
|
6
7
|
import { loadBundledRules } from "./rules-loader.js";
|
|
7
8
|
import { lookup as lookupFn, validateDisclosure as validateFn } from "./lookup.js";
|
|
8
9
|
/**
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,MAAM,EACN,sBAAsB,EACtB,kBAAkB,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,gBAAgB,EAChB,iBAAiB,EACjB,eAAe,GAChB,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EACL,qBAAqB,EACrB,sBAAsB,EACtB,iBAAiB,GAGlB,MAAM,eAAe,CAAC;AAYvB,OAAO,EACL,OAAO,EACP,OAAO,EACP,QAAQ,EACR,cAAc,EACd,WAAW,EACX,iBAAiB,EACjB,cAAc,EACd,OAAO,GACR,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,QAAQ,EACR,cAAc,GAGf,MAAM,gBAAgB,CAAC;
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,MAAM,EACN,sBAAsB,EACtB,kBAAkB,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,gBAAgB,EAChB,iBAAiB,EACjB,eAAe,GAChB,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EACL,qBAAqB,EACrB,sBAAsB,EACtB,iBAAiB,GAGlB,MAAM,eAAe,CAAC;AAYvB,OAAO,EACL,OAAO,EACP,OAAO,EACP,QAAQ,EACR,cAAc,EACd,WAAW,EACX,iBAAiB,EACjB,cAAc,EACd,OAAO,GACR,MAAM,aAAa,CAAC;AACrB,OAAO,EACL,QAAQ,EACR,cAAc,GAGf,MAAM,gBAAgB,CAAC;AACxB,OAAO,EACL,YAAY,EACZ,UAAU,EACV,mBAAmB,EACnB,SAAS,EACT,UAAU,EACV,YAAY,EACZ,gBAAgB,EAChB,qBAAqB,EACrB,gBAAgB,EAChB,wBAAwB,EACxB,WAAW,EACX,gBAAgB,GACjB,MAAM,oBAAoB,CAAC;AAU5B,OAAO,EAAE,gBAAgB,EAAE,MAAM,mBAAmB,CAAC;AACrD,OAAO,EAAE,MAAM,IAAI,QAAQ,EAAE,kBAAkB,IAAI,UAAU,EAAE,MAAM,aAAa,CAAC;AAGnF;;;;GAIG;AACH,MAAM,UAAU,cAAc,CAAC,KAAmB;IAChD,OAAO,QAAQ,CAAC,gBAAgB,EAAE,EAAE,KAAK,CAAC,CAAC;AAC7C,CAAC;AAED,2EAA2E;AAC3E,MAAM,UAAU,iBAAiB;IAC/B,MAAM,KAAK,GAAG,gBAAgB,EAAE,CAAC;IACjC,MAAM,GAAG,GAAG,IAAI,GAAG,EAAU,CAAC;IAC9B,KAAK,MAAM,CAAC,IAAI,KAAK,CAAC,KAAK;QAAE,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC;IACrD,OAAO,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;AACzB,CAAC;AAED,uDAAuD;AACvD,MAAM,UAAU,WAAW,CAAC,EAAU;IACpC,MAAM,KAAK,GAAG,gBAAgB,EAAE,CAAC;IACjC,OAAO,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC;AAC9C,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,0BAA0B,CACxC,KAAmB,EACnB,aAAqB;IAErB,MAAM,KAAK,GAAG,gBAAgB,EAAE,CAAC;IACjC,MAAM,UAAU,GAAG,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;IAC1C,OAAO,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,EAAE,aAAa,CAAC,CAAC,CAAC;AAClE,CAAC"}
|
package/dist/watcher/index.d.ts
CHANGED
|
@@ -35,5 +35,5 @@ export declare function runWatcher(opts: {
|
|
|
35
35
|
export type { Article, Source, RunReport, SourceRunReport, StateStore, WatcherState, };
|
|
36
36
|
export { readState, writeState, fsStateStore, memoryStateStore, } from "./state-store.js";
|
|
37
37
|
export { federalRegisterSource } from "./sources/federal-register.js";
|
|
38
|
-
export { urlMonitorSource, rulesCitationsUrlMonitor, hashContent, } from "./sources/url-monitor.js";
|
|
38
|
+
export { urlMonitorSource, rulesCitationsUrlMonitor, hashContent, normalizeForHash, } from "./sources/url-monitor.js";
|
|
39
39
|
//# sourceMappingURL=index.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/watcher/index.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,OAAO,EACP,SAAS,EACT,MAAM,EACN,eAAe,EACf,UAAU,EACV,YAAY,EACb,MAAM,YAAY,CAAC;AAGpB;;;GAGG;AACH,wBAAgB,YAAY,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,OAAO,EAAE,CAO1E;AAED;;;;;;;;;;;GAWG;AACH,wBAAsB,mBAAmB,CAAC,IAAI,EAAE;IAC9C,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,UAAU,EAAE,UAAU,CAAC;IACvB,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB,GAAG,OAAO,CAAC,SAAS,CAAC,CAiDrB;AAED;;;;GAIG;AACH,wBAAsB,UAAU,CAAC,IAAI,EAAE;IACrC,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,6EAA6E;IAC7E,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB,GAAG,OAAO,CAAC,SAAS,CAAC,CAWrB;AAED,YAAY,EACV,OAAO,EACP,MAAM,EACN,SAAS,EACT,eAAe,EACf,UAAU,EACV,YAAY,GACb,CAAC;AACF,OAAO,EACL,SAAS,EACT,UAAU,EACV,YAAY,EACZ,gBAAgB,GACjB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,qBAAqB,EAAE,MAAM,+BAA+B,CAAC;AACtE,OAAO,EACL,gBAAgB,EAChB,wBAAwB,EACxB,WAAW,
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/watcher/index.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,OAAO,EACP,SAAS,EACT,MAAM,EACN,eAAe,EACf,UAAU,EACV,YAAY,EACb,MAAM,YAAY,CAAC;AAGpB;;;GAGG;AACH,wBAAgB,YAAY,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,OAAO,EAAE,CAO1E;AAED;;;;;;;;;;;GAWG;AACH,wBAAsB,mBAAmB,CAAC,IAAI,EAAE;IAC9C,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,UAAU,EAAE,UAAU,CAAC;IACvB,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB,GAAG,OAAO,CAAC,SAAS,CAAC,CAiDrB;AAED;;;;GAIG;AACH,wBAAsB,UAAU,CAAC,IAAI,EAAE;IACrC,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,6EAA6E;IAC7E,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB,GAAG,OAAO,CAAC,SAAS,CAAC,CAWrB;AAED,YAAY,EACV,OAAO,EACP,MAAM,EACN,SAAS,EACT,eAAe,EACf,UAAU,EACV,YAAY,GACb,CAAC;AACF,OAAO,EACL,SAAS,EACT,UAAU,EACV,YAAY,EACZ,gBAAgB,GACjB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,qBAAqB,EAAE,MAAM,+BAA+B,CAAC;AACtE,OAAO,EACL,gBAAgB,EAChB,wBAAwB,EACxB,WAAW,EACX,gBAAgB,GACjB,MAAM,0BAA0B,CAAC"}
|
package/dist/watcher/index.js
CHANGED
|
@@ -87,5 +87,5 @@ export async function runWatcher(opts) {
|
|
|
87
87
|
}
|
|
88
88
|
export { readState, writeState, fsStateStore, memoryStateStore, } from "./state-store.js";
|
|
89
89
|
export { federalRegisterSource } from "./sources/federal-register.js";
|
|
90
|
-
export { urlMonitorSource, rulesCitationsUrlMonitor, hashContent, } from "./sources/url-monitor.js";
|
|
90
|
+
export { urlMonitorSource, rulesCitationsUrlMonitor, hashContent, normalizeForHash, } from "./sources/url-monitor.js";
|
|
91
91
|
//# sourceMappingURL=index.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/watcher/index.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,YAAY,EAAE,MAAM,kBAAkB,CAAC;AAEhD;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,OAAkB,EAAE,IAAc;IAC7D,MAAM,OAAO,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,CAAC;IAC9B,MAAM,GAAG,GAAc,EAAE,CAAC;IAC1B,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC;YAAE,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACtC,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED;;;;;;;;;;;GAWG;AACH,MAAM,CAAC,KAAK,UAAU,mBAAmB,CAAC,IAIzC;IACC,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,GAAG,GAAG,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;IACrC,MAAM,aAAa,GAAsB,EAAE,CAAC;IAE5C,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;QAClC,MAAM,IAAI,GAAG,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC;QACvE,IAAI,QAAQ,GAAc,EAAE,CAAC;QAC7B,IAAI,MAA0B,CAAC;QAC/B,IAAI,CAAC;YACH,QAAQ,GAAG,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;QAClC,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,GAAI,GAAa,CAAC,OAAO,CAAC;QAClC,CAAC;QAED,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;YACzB,aAAa,CAAC,IAAI,CAAC;gBACjB,SAAS,EAAE,MAAM,CAAC,EAAE;gBACpB,WAAW,EAAE,MAAM,CAAC,WAAW;gBAC/B,EAAE,EAAE,KAAK;gBACT,KAAK,EAAE,MAAM;gBACb,YAAY,EAAE,EAAE;gBAChB,UAAU,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM;aAC7B,CAAC,CAAC;YACH,SAAS;QACX,CAAC;QAED,MAAM,WAAW,GAAG,YAAY,CAAC,QAAQ,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QACtD,aAAa,CAAC,IAAI,CAAC;YACjB,SAAS,EAAE,MAAM,CAAC,EAAE;YACpB,WAAW,EAAE,MAAM,CAAC,WAAW;YAC/B,EAAE,EAAE,IAAI;YACR,YAAY,EAAE,WAAW;YACzB,UAAU,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,WAAW,CAAC,MAAM;SAClD,CAAC,CAAC;QAEH,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YACjB,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YAClC,KAAK,MAAM,CAAC,IAAI,QAAQ;gBAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YAC3C,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC,GAAG;gBACzB,IAAI,EAAE,CAAC,GAAG,MAAM,CAAC;gBACjB,SAAS,EAAE,GAAG;aACf,CAAC;QACJ,CAAC;IACH,CAAC;IAED,IAAI,CAAC,IAAI,CAAC,MAAM;QAAE,MAAM,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;IAErD,OAAO,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,aAAa,EAAE,CAAC;AACjD,CAAC;AAED;;;;GAIG;AACH,MAAM,CAAC,KAAK,UAAU,UAAU,CAAC,IAKhC;IACC,MAAM,WAAW,GAIb;QACF,OAAO,EAAE,IAAI,CAAC,OAAO;QACrB,UAAU,EAAE,YAAY,CAAC,IAAI,CAAC,SAAS,CAAC;KACzC,CAAC;IACF,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS;QAAE,WAAW,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;IAChE,OAAO,mBAAmB,CAAC,WAAW,CAAC,CAAC;AAC1C,CAAC;AAUD,OAAO,EACL,SAAS,EACT,UAAU,EACV,YAAY,EACZ,gBAAgB,GACjB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,qBAAqB,EAAE,MAAM,+BAA+B,CAAC;AACtE,OAAO,EACL,gBAAgB,EAChB,wBAAwB,EACxB,WAAW,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/watcher/index.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,YAAY,EAAE,MAAM,kBAAkB,CAAC;AAEhD;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,OAAkB,EAAE,IAAc;IAC7D,MAAM,OAAO,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,CAAC;IAC9B,MAAM,GAAG,GAAc,EAAE,CAAC;IAC1B,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC;YAAE,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACtC,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED;;;;;;;;;;;GAWG;AACH,MAAM,CAAC,KAAK,UAAU,mBAAmB,CAAC,IAIzC;IACC,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,GAAG,GAAG,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;IACrC,MAAM,aAAa,GAAsB,EAAE,CAAC;IAE5C,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;QAClC,MAAM,IAAI,GAAG,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC;QACvE,IAAI,QAAQ,GAAc,EAAE,CAAC;QAC7B,IAAI,MAA0B,CAAC;QAC/B,IAAI,CAAC;YACH,QAAQ,GAAG,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;QAClC,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,GAAI,GAAa,CAAC,OAAO,CAAC;QAClC,CAAC;QAED,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;YACzB,aAAa,CAAC,IAAI,CAAC;gBACjB,SAAS,EAAE,MAAM,CAAC,EAAE;gBACpB,WAAW,EAAE,MAAM,CAAC,WAAW;gBAC/B,EAAE,EAAE,KAAK;gBACT,KAAK,EAAE,MAAM;gBACb,YAAY,EAAE,EAAE;gBAChB,UAAU,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM;aAC7B,CAAC,CAAC;YACH,SAAS;QACX,CAAC;QAED,MAAM,WAAW,GAAG,YAAY,CAAC,QAAQ,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QACtD,aAAa,CAAC,IAAI,CAAC;YACjB,SAAS,EAAE,MAAM,CAAC,EAAE;YACpB,WAAW,EAAE,MAAM,CAAC,WAAW;YAC/B,EAAE,EAAE,IAAI;YACR,YAAY,EAAE,WAAW;YACzB,UAAU,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,WAAW,CAAC,MAAM;SAClD,CAAC,CAAC;QAEH,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YACjB,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YAClC,KAAK,MAAM,CAAC,IAAI,QAAQ;gBAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YAC3C,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC,GAAG;gBACzB,IAAI,EAAE,CAAC,GAAG,MAAM,CAAC;gBACjB,SAAS,EAAE,GAAG;aACf,CAAC;QACJ,CAAC;IACH,CAAC;IAED,IAAI,CAAC,IAAI,CAAC,MAAM;QAAE,MAAM,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;IAErD,OAAO,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,aAAa,EAAE,CAAC;AACjD,CAAC;AAED;;;;GAIG;AACH,MAAM,CAAC,KAAK,UAAU,UAAU,CAAC,IAKhC;IACC,MAAM,WAAW,GAIb;QACF,OAAO,EAAE,IAAI,CAAC,OAAO;QACrB,UAAU,EAAE,YAAY,CAAC,IAAI,CAAC,SAAS,CAAC;KACzC,CAAC;IACF,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS;QAAE,WAAW,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;IAChE,OAAO,mBAAmB,CAAC,WAAW,CAAC,CAAC;AAC1C,CAAC;AAUD,OAAO,EACL,SAAS,EACT,UAAU,EACV,YAAY,EACZ,gBAAgB,GACjB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,qBAAqB,EAAE,MAAM,+BAA+B,CAAC;AACtE,OAAO,EACL,gBAAgB,EAChB,wBAAwB,EACxB,WAAW,EACX,gBAAgB,GACjB,MAAM,0BAA0B,CAAC"}
|
|
@@ -8,6 +8,13 @@ import type { Source } from "../types.js";
|
|
|
8
8
|
* Use this for tracking regulator-published source pages that don't have a
|
|
9
9
|
* dedicated API or RSS feed. Failures fetching individual URLs are isolated
|
|
10
10
|
* — one bad URL does not abort the run.
|
|
11
|
+
*
|
|
12
|
+
* Hashing is performed on a normalized version of the page (see
|
|
13
|
+
* `normalizeForHash`) — `<script>` / `<style>` blocks, HTML comments,
|
|
14
|
+
* CSRF inputs, and inline `nonce`/`integrity`/`csrf-token` attributes
|
|
15
|
+
* are stripped before hashing. This avoids the false-positive flapping
|
|
16
|
+
* that raw-body hashing produces on regulator pages with dynamic
|
|
17
|
+
* markers (anti-bot tokens, server timestamps, build hashes).
|
|
11
18
|
*/
|
|
12
19
|
export declare function urlMonitorSource(opts: {
|
|
13
20
|
id: string;
|
|
@@ -16,6 +23,32 @@ export declare function urlMonitorSource(opts: {
|
|
|
16
23
|
/** Optional fetch shim for testing. Defaults to global fetch. */
|
|
17
24
|
fetcher?: typeof fetch;
|
|
18
25
|
}): Source;
|
|
26
|
+
/**
|
|
27
|
+
* Normalize a fetched page body before content-hashing, so hashes are
|
|
28
|
+
* stable across fetches when the meaningful regulator-published text
|
|
29
|
+
* is unchanged. Removes dynamic content that varies per request:
|
|
30
|
+
*
|
|
31
|
+
* 1. `<script>...</script>` blocks (nonces, build hashes, telemetry).
|
|
32
|
+
* 2. `<style>...</style>` blocks (sometimes contain build timestamps).
|
|
33
|
+
* 3. `<!-- ... -->` HTML comments.
|
|
34
|
+
* 4. `<input type="hidden" name="*csrf*"/"*token*"/"*authenticity*">`
|
|
35
|
+
* elements (CSRF tokens vary per session).
|
|
36
|
+
* 5. Inline `nonce="..."`, `integrity="..."`, `data-csrf="..."`, and
|
|
37
|
+
* `data-token="..."` attribute values.
|
|
38
|
+
* 6. `<meta>` tags whose `name`/`property` is timestamp-like
|
|
39
|
+
* (`og:updated_time`, `last-modified`, `revised`, etc.).
|
|
40
|
+
* 7. Collapse runs of whitespace.
|
|
41
|
+
*
|
|
42
|
+
* The result is HTML with the layout structure preserved but the
|
|
43
|
+
* dynamic per-fetch markers removed. Two fetches of the same
|
|
44
|
+
* regulator-published page now hash identically as long as the
|
|
45
|
+
* substantive text and structure are unchanged. When the published
|
|
46
|
+
* text changes, the hash changes and the watcher fires.
|
|
47
|
+
*
|
|
48
|
+
* Exported for testability and for downstream consumers who want to
|
|
49
|
+
* apply the same normalization to other change-detection workflows.
|
|
50
|
+
*/
|
|
51
|
+
export declare function normalizeForHash(html: string): string;
|
|
19
52
|
/**
|
|
20
53
|
* Convenience: a source that monitors every bundled rule's `citation.source_url`.
|
|
21
54
|
* When any cited statute or regulation page changes content, the next watcher
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"url-monitor.d.ts","sourceRoot":"","sources":["../../../src/watcher/sources/url-monitor.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAW,MAAM,EAAE,MAAM,aAAa,CAAC;AAGnD
|
|
1
|
+
{"version":3,"file":"url-monitor.d.ts","sourceRoot":"","sources":["../../../src/watcher/sources/url-monitor.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAW,MAAM,EAAE,MAAM,aAAa,CAAC;AAGnD;;;;;;;;;;;;;;;;GAgBG;AACH,wBAAgB,gBAAgB,CAAC,IAAI,EAAE;IACrC,EAAE,EAAE,MAAM,CAAC;IACX,WAAW,EAAE,MAAM,CAAC;IACpB,IAAI,EAAE,MAAM,EAAE,CAAC;IACf,iEAAiE;IACjE,OAAO,CAAC,EAAE,OAAO,KAAK,CAAC;CACxB,GAAG,MAAM,CAkCT;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,wBAAgB,gBAAgB,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CA2BrD;AAED;;;;;;;;GAQG;AACH,wBAAgB,wBAAwB,CAAC,IAAI,CAAC,EAAE;IAC9C,OAAO,CAAC,EAAE,OAAO,KAAK,CAAC;CACxB,GAAG,MAAM,CAUT;AAED,mIAAmI;AACnI,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAEhD"}
|
|
@@ -9,6 +9,13 @@ import { loadBundledRules } from "../../rules-loader.js";
|
|
|
9
9
|
* Use this for tracking regulator-published source pages that don't have a
|
|
10
10
|
* dedicated API or RSS feed. Failures fetching individual URLs are isolated
|
|
11
11
|
* — one bad URL does not abort the run.
|
|
12
|
+
*
|
|
13
|
+
* Hashing is performed on a normalized version of the page (see
|
|
14
|
+
* `normalizeForHash`) — `<script>` / `<style>` blocks, HTML comments,
|
|
15
|
+
* CSRF inputs, and inline `nonce`/`integrity`/`csrf-token` attributes
|
|
16
|
+
* are stripped before hashing. This avoids the false-positive flapping
|
|
17
|
+
* that raw-body hashing produces on regulator pages with dynamic
|
|
18
|
+
* markers (anti-bot tokens, server timestamps, build hashes).
|
|
12
19
|
*/
|
|
13
20
|
export function urlMonitorSource(opts) {
|
|
14
21
|
const fetcher = opts.fetcher ?? fetch;
|
|
@@ -23,14 +30,19 @@ export function urlMonitorSource(opts) {
|
|
|
23
30
|
const res = await fetcher(url, { redirect: "follow" });
|
|
24
31
|
if (!res.ok)
|
|
25
32
|
continue;
|
|
26
|
-
const
|
|
27
|
-
const
|
|
33
|
+
const raw = await res.text();
|
|
34
|
+
const normalized = normalizeForHash(raw);
|
|
35
|
+
const hash = hashContent(normalized);
|
|
28
36
|
articles.push({
|
|
29
37
|
id: `${url}#${hash}`,
|
|
30
38
|
title: url,
|
|
31
39
|
url,
|
|
32
40
|
publishedAt: today(),
|
|
33
|
-
extra: {
|
|
41
|
+
extra: {
|
|
42
|
+
content_hash: hash,
|
|
43
|
+
content_length: raw.length,
|
|
44
|
+
normalized_length: normalized.length,
|
|
45
|
+
},
|
|
34
46
|
});
|
|
35
47
|
}
|
|
36
48
|
catch {
|
|
@@ -41,6 +53,53 @@ export function urlMonitorSource(opts) {
|
|
|
41
53
|
},
|
|
42
54
|
};
|
|
43
55
|
}
|
|
56
|
+
/**
|
|
57
|
+
* Normalize a fetched page body before content-hashing, so hashes are
|
|
58
|
+
* stable across fetches when the meaningful regulator-published text
|
|
59
|
+
* is unchanged. Removes dynamic content that varies per request:
|
|
60
|
+
*
|
|
61
|
+
* 1. `<script>...</script>` blocks (nonces, build hashes, telemetry).
|
|
62
|
+
* 2. `<style>...</style>` blocks (sometimes contain build timestamps).
|
|
63
|
+
* 3. `<!-- ... -->` HTML comments.
|
|
64
|
+
* 4. `<input type="hidden" name="*csrf*"/"*token*"/"*authenticity*">`
|
|
65
|
+
* elements (CSRF tokens vary per session).
|
|
66
|
+
* 5. Inline `nonce="..."`, `integrity="..."`, `data-csrf="..."`, and
|
|
67
|
+
* `data-token="..."` attribute values.
|
|
68
|
+
* 6. `<meta>` tags whose `name`/`property` is timestamp-like
|
|
69
|
+
* (`og:updated_time`, `last-modified`, `revised`, etc.).
|
|
70
|
+
* 7. Collapse runs of whitespace.
|
|
71
|
+
*
|
|
72
|
+
* The result is HTML with the layout structure preserved but the
|
|
73
|
+
* dynamic per-fetch markers removed. Two fetches of the same
|
|
74
|
+
* regulator-published page now hash identically as long as the
|
|
75
|
+
* substantive text and structure are unchanged. When the published
|
|
76
|
+
* text changes, the hash changes and the watcher fires.
|
|
77
|
+
*
|
|
78
|
+
* Exported for testability and for downstream consumers who want to
|
|
79
|
+
* apply the same normalization to other change-detection workflows.
|
|
80
|
+
*/
|
|
81
|
+
export function normalizeForHash(html) {
|
|
82
|
+
let s = html;
|
|
83
|
+
// Order matters — script/style first because they may contain
|
|
84
|
+
// patterns the later regexes would otherwise see.
|
|
85
|
+
s = s.replace(/<script\b[^>]*>[\s\S]*?<\/script>/gi, "");
|
|
86
|
+
s = s.replace(/<style\b[^>]*>[\s\S]*?<\/style>/gi, "");
|
|
87
|
+
s = s.replace(/<!--[\s\S]*?-->/g, "");
|
|
88
|
+
// CSRF / authenticity hidden inputs — match common patterns.
|
|
89
|
+
s = s.replace(/<input\b[^>]*name\s*=\s*["'][^"']*(?:csrf|authenticity|_token|requestverification)[^"']*["'][^>]*\/?\s*>/gi, "");
|
|
90
|
+
// Inline dynamic attribute values. We strip the *value*, not the
|
|
91
|
+
// attribute name, to preserve structural diffability.
|
|
92
|
+
s = s.replace(/\b(nonce|integrity)\s*=\s*"[^"]*"/gi, "$1=\"\"");
|
|
93
|
+
s = s.replace(/\b(nonce|integrity)\s*=\s*'[^']*'/gi, "$1=''");
|
|
94
|
+
s = s.replace(/\b(data-(?:csrf|token|nonce|build|version)[^=\s>]*)\s*=\s*"[^"]*"/gi, "$1=\"\"");
|
|
95
|
+
s = s.replace(/\b(data-(?:csrf|token|nonce|build|version)[^=\s>]*)\s*=\s*'[^']*'/gi, "$1=''");
|
|
96
|
+
// Timestamp-bearing meta tags.
|
|
97
|
+
s = s.replace(/<meta\b[^>]*(?:name|property)\s*=\s*["'][^"']*(?:updated_time|last-?modified|revised|build-?time|generated-?at|page-?date)[^"']*["'][^>]*\/?\s*>/gi, "");
|
|
98
|
+
// Collapse runs of whitespace and trim ends. Single space between
|
|
99
|
+
// tokens is sufficient for hash stability.
|
|
100
|
+
s = s.replace(/\s+/g, " ").trim();
|
|
101
|
+
return s;
|
|
102
|
+
}
|
|
44
103
|
/**
|
|
45
104
|
* Convenience: a source that monitors every bundled rule's `citation.source_url`.
|
|
46
105
|
* When any cited statute or regulation page changes content, the next watcher
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"url-monitor.js","sourceRoot":"","sources":["../../../src/watcher/sources/url-monitor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEzC,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AAEzD
|
|
1
|
+
{"version":3,"file":"url-monitor.js","sourceRoot":"","sources":["../../../src/watcher/sources/url-monitor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEzC,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AAEzD;;;;;;;;;;;;;;;;GAgBG;AACH,MAAM,UAAU,gBAAgB,CAAC,IAMhC;IACC,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,KAAK,CAAC;IACtC,MAAM,KAAK,GAAG,GAAW,EAAE,CAAC,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IAElE,OAAO;QACL,EAAE,EAAE,IAAI,CAAC,EAAE;QACX,WAAW,EAAE,IAAI,CAAC,WAAW;QAC7B,KAAK,CAAC,KAAK;YACT,MAAM,QAAQ,GAAc,EAAE,CAAC;YAC/B,KAAK,MAAM,GAAG,IAAI,IAAI,CAAC,IAAI,EAAE,CAAC;gBAC5B,IAAI,CAAC;oBACH,MAAM,GAAG,GAAG,MAAM,OAAO,CAAC,GAAG,EAAE,EAAE,QAAQ,EAAE,QAAQ,EAAE,CAAC,CAAC;oBACvD,IAAI,CAAC,GAAG,CAAC,EAAE;wBAAE,SAAS;oBACtB,MAAM,GAAG,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;oBAC7B,MAAM,UAAU,GAAG,gBAAgB,CAAC,GAAG,CAAC,CAAC;oBACzC,MAAM,IAAI,GAAG,WAAW,CAAC,UAAU,CAAC,CAAC;oBACrC,QAAQ,CAAC,IAAI,CAAC;wBACZ,EAAE,EAAE,GAAG,GAAG,IAAI,IAAI,EAAE;wBACpB,KAAK,EAAE,GAAG;wBACV,GAAG;wBACH,WAAW,EAAE,KAAK,EAAE;wBACpB,KAAK,EAAE;4BACL,YAAY,EAAE,IAAI;4BAClB,cAAc,EAAE,GAAG,CAAC,MAAM;4BAC1B,iBAAiB,EAAE,UAAU,CAAC,MAAM;yBACrC;qBACF,CAAC,CAAC;gBACL,CAAC;gBAAC,MAAM,CAAC;oBACP,6EAA6E;gBAC/E,CAAC;YACH,CAAC;YACD,OAAO,QAAQ,CAAC;QAClB,CAAC;KACF,CAAC;AACJ,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,MAAM,UAAU,gBAAgB,CAAC,IAAY;IAC3C,IAAI,CAAC,GAAG,IAAI,CAAC;IACb,8DAA8D;IAC9D,kDAAkD;IAClD,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,qCAAqC,EAAE,EAAE,CAAC,CAAC;IACzD,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,mCAAmC,EAAE,EAAE,CAAC,CAAC;IACvD,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,kBAAkB,EAAE,EAAE,CAAC,CAAC;IACtC,6DAA6D;IAC7D,CAAC,GAAG,CAAC,CAAC,OAAO,CACX,4GAA4G,EAC5G,EAAE,CACH,CAAC;IACF,iEAAiE;IACjE,sDAAsD;IACtD,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,qCAAqC,EAAE,SAAS,CAAC,CAAC;IAChE,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,qCAAqC,EAAE,OAAO,CAAC,CAAC;IAC9D,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,qEAAqE,EAAE,SAAS,CAAC,CAAC;IAChG,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,qEAAqE,EAAE,OAAO,CAAC,CAAC;IAC9F,+BAA+B;IAC/B,CAAC,GAAG,CAAC,CAAC,OAAO,CACX,oJAAoJ,EACpJ,EAAE,CACH,CAAC;IACF,kEAAkE;IAClE,2CAA2C;IAC3C,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;IAClC,OAAO,CAAC,CAAC;AACX,CAAC;AAED;;;;;;;;GAQG;AACH,MAAM,UAAU,wBAAwB,CAAC,IAExC;IACC,MAAM,KAAK,GAAG,gBAAgB,EAAE,CAAC;IACjC,MAAM,IAAI,GAAG,CAAC,GAAG,IAAI,GAAG,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC;IACzE,OAAO,gBAAgB,CAAC;QACtB,EAAE,EAAE,qBAAqB;QACzB,WAAW,EACT,8LAA8L;QAChM,IAAI;QACJ,GAAG,CAAC,IAAI,EAAE,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAClE,CAAC,CAAC;AACL,CAAC;AAED,mIAAmI;AACnI,MAAM,UAAU,WAAW,CAAC,IAAY;IACtC,OAAO,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;AACtE,CAAC"}
|
|
@@ -0,0 +1,274 @@
|
|
|
1
|
+
# EEOC Title VII AI selection procedures: a builder's guide
|
|
2
|
+
|
|
3
|
+
> **Informational only — not legal advice.** Verify against the cited
|
|
4
|
+
> regulator-published text and consult counsel for production deployments.
|
|
5
|
+
> See `AI-DISCLOSURE.md` in this package.
|
|
6
|
+
|
|
7
|
+
If your HR-tech product, applicant-tracking system, AI screening tool,
|
|
8
|
+
video-interview analyzer, gamified-assessment platform, or AI-assisted
|
|
9
|
+
sourcing tool is used in any employment selection decision — hiring,
|
|
10
|
+
promotion, transfer, retention, or termination — Title VII of the
|
|
11
|
+
Civil Rights Act of 1964 applies in full, even when the decision is
|
|
12
|
+
mediated by AI. The U.S. Equal Employment Opportunity Commission's
|
|
13
|
+
**May 18, 2023 technical assistance** on AI selection procedures is
|
|
14
|
+
the federal-floor framework for what that means in production. This
|
|
15
|
+
guide covers what Title VII requires, why the four-fifths rule is
|
|
16
|
+
the operative compliance metric, the relationship to NYC Local Law
|
|
17
|
+
144 and other state-level mandates, the ADA reasonable-accommodation
|
|
18
|
+
overlay, and what governance discipline a vendor needs in place
|
|
19
|
+
before deploying an AI tool that touches employment decisions.
|
|
20
|
+
|
|
21
|
+
## What the EEOC technical assistance actually says
|
|
22
|
+
|
|
23
|
+
On May 18, 2023, the EEOC issued [*Select Issues: Assessing Adverse
|
|
24
|
+
Impact in Software, Algorithms, and Artificial Intelligence Used in
|
|
25
|
+
Employment Selection Procedures Under Title VII*](https://www.eeoc.gov/laws/guidance/select-issues-assessing-adverse-impact-software-algorithms-and-artificial-intelligence).
|
|
26
|
+
The technical assistance does not create new law. It clarifies how
|
|
27
|
+
existing law — Title VII (42 U.S.C. § 2000e et seq.) and the
|
|
28
|
+
**Uniform Guidelines on Employee Selection Procedures** (1978),
|
|
29
|
+
codified at 29 CFR Part 1607 — applies when employers use AI or
|
|
30
|
+
algorithmic tools in selection procedures.
|
|
31
|
+
|
|
32
|
+
Three operative holdings:
|
|
33
|
+
|
|
34
|
+
1. **AI tools are "selection procedures."** Any AI-driven test,
|
|
35
|
+
scoring system, ranking algorithm, video-interview analyzer, or
|
|
36
|
+
automated screening tool used as part of an employment decision is
|
|
37
|
+
a "selection procedure" under the Uniform Guidelines. The guidance
|
|
38
|
+
confirms that even informal uses (an AI tool that "helps" a
|
|
39
|
+
recruiter shortlist candidates, where the recruiter then chooses)
|
|
40
|
+
are selection procedures if the tool meaningfully affects the
|
|
41
|
+
decision.
|
|
42
|
+
2. **The four-fifths rule applies.** Under the Uniform Guidelines, a
|
|
43
|
+
selection rate for any race, sex, or ethnic group that is less
|
|
44
|
+
than four-fifths (80%) of the rate for the highest-rate group is
|
|
45
|
+
"evidence of adverse impact." This rule applies to AI selection
|
|
46
|
+
tools on the same terms as any other selection procedure.
|
|
47
|
+
3. **Vendor liability does not transfer.** The employer is liable
|
|
48
|
+
under Title VII for disparate-impact discrimination resulting from
|
|
49
|
+
an AI tool, even when the tool was developed and operated by a
|
|
50
|
+
third-party vendor. The vendor's representation that the tool was
|
|
51
|
+
"bias-tested" is not a defense.
|
|
52
|
+
|
|
53
|
+
The technical assistance is **interpretive**, not regulatory — the
|
|
54
|
+
binding obligation is Title VII's prohibition on disparate-impact
|
|
55
|
+
discrimination, which has been law since *Griggs v. Duke Power*
|
|
56
|
+
(1971). But the guidance signals current EEOC enforcement priorities
|
|
57
|
+
and is treated as authoritative in EEOC investigations.
|
|
58
|
+
|
|
59
|
+
## The four-fifths rule, in production
|
|
60
|
+
|
|
61
|
+
The four-fifths rule is the operational adverse-impact metric. To
|
|
62
|
+
apply it to an AI selection tool:
|
|
63
|
+
|
|
64
|
+
1. **Compute the selection rate for each protected class group.** For
|
|
65
|
+
a hiring tool: of all candidates who interacted with the tool,
|
|
66
|
+
what fraction received a "pass" (advanced to the next step,
|
|
67
|
+
received an offer, etc.) — broken out by race, sex, ethnicity,
|
|
68
|
+
age cohort, etc.
|
|
69
|
+
2. **Identify the highest selection rate.** Across all groups, find
|
|
70
|
+
the group with the highest pass rate.
|
|
71
|
+
3. **Divide each other group's rate by the highest.** If any group's
|
|
72
|
+
rate is below 80% (4/5) of the highest group's rate, there is
|
|
73
|
+
evidence of adverse impact for that group.
|
|
74
|
+
4. **If adverse impact is found**, the employer must demonstrate the
|
|
75
|
+
selection procedure is "job related for the position in question
|
|
76
|
+
and consistent with business necessity" (the Uniform Guidelines'
|
|
77
|
+
"job-relatedness" defense). And even then, a less-discriminatory
|
|
78
|
+
alternative must not be available.
|
|
79
|
+
|
|
80
|
+
Concrete example: An AI screening tool passes 40% of male
|
|
81
|
+
applicants and 25% of female applicants. The female-to-male ratio is
|
|
82
|
+
25/40 = 0.625, which is less than 0.8 — evidence of adverse impact.
|
|
83
|
+
The employer must either (a) demonstrate the tool is job-related and
|
|
84
|
+
no less-discriminatory alternative exists, or (b) modify or replace
|
|
85
|
+
the tool.
|
|
86
|
+
|
|
87
|
+
## "The vendor said the tool was bias-tested" is not a defense
|
|
88
|
+
|
|
89
|
+
This is the single most consequential clarification in the EEOC
|
|
90
|
+
guidance. Many AI HR tool vendors include in their marketing copy
|
|
91
|
+
some variant of "our model has been audited for bias" — usually
|
|
92
|
+
referring to a one-time pre-deployment audit on a synthetic dataset
|
|
93
|
+
or against a generic baseline. The EEOC's position:
|
|
94
|
+
|
|
95
|
+
- **The employer remains liable.** Title VII liability is the
|
|
96
|
+
employer's, not the vendor's. A vendor audit that turns out to
|
|
97
|
+
miss real disparate impact in the employer's actual applicant pool
|
|
98
|
+
is not a defense.
|
|
99
|
+
- **Each employer needs its own audit.** The applicant pool, role
|
|
100
|
+
requirements, and use context vary by employer. A tool that's
|
|
101
|
+
unbiased for one employer's pool may be biased for another's.
|
|
102
|
+
Production-ready compliance requires the *employer* to audit the
|
|
103
|
+
*employer's actual outputs*.
|
|
104
|
+
- **Ongoing audits, not one-time.** AI tools drift as they're
|
|
105
|
+
retrained on new data; a tool that passed a four-fifths audit at
|
|
106
|
+
deployment may fail one a year later. Periodic re-auditing is
|
|
107
|
+
expected.
|
|
108
|
+
|
|
109
|
+
## Where state and local law layers on top
|
|
110
|
+
|
|
111
|
+
The EEOC technical assistance is the federal floor. Several
|
|
112
|
+
jurisdictions have **stricter** mandatory rules:
|
|
113
|
+
|
|
114
|
+
| Jurisdiction | Law | What it adds beyond EEOC |
|
|
115
|
+
|---|---|---|
|
|
116
|
+
| New York City | Local Law 144 (AEDT) | Mandatory annual independent bias audit, public publication of audit results, and applicant notice 10 business days before tool use. **Mandatory**, not recommended. |
|
|
117
|
+
| Illinois | Human Rights Act amended by HB 3773 | Effective 2026-01-01: prohibits AI use in employment that subjects an employee or applicant to discrimination based on protected class. Requires advance notice. |
|
|
118
|
+
| Colorado | SB 24-205 (AI Act) | Effective 2026-06-30: applies to "high-risk AI systems" including those used in employment decisions; requires risk assessment, consumer notice, and right to an explanation. |
|
|
119
|
+
| Maryland | LE § 3-717 | Facial recognition in interviews requires written consent. |
|
|
120
|
+
| California | AB 2930 (status pending — Newsom vetoed Sept 2024) | Would have required impact assessments for AI in employment. Not currently law; monitor for re-introduction. |
|
|
121
|
+
|
|
122
|
+
For a multi-state employer, the right rule is the strictest applicable
|
|
123
|
+
local rule, not the EEOC technical assistance. A national HR-tech
|
|
124
|
+
deployment must satisfy NYC Local Law 144's audit-and-publication
|
|
125
|
+
mandate even if the federal EEOC guidance only "recommends" similar
|
|
126
|
+
steps.
|
|
127
|
+
|
|
128
|
+
## The ADA accommodation overlay
|
|
129
|
+
|
|
130
|
+
The EEOC issued a parallel technical assistance under the **Americans
|
|
131
|
+
with Disabilities Act** on May 12, 2022 ([*The Americans with
|
|
132
|
+
Disabilities Act and the Use of Software, Algorithms, and Artificial
|
|
133
|
+
Intelligence to Assess Job Applicants and
|
|
134
|
+
Employees*](https://www.eeoc.gov/laws/guidance/americans-disabilities-act-and-use-software-algorithms-and-artificial-intelligence)).
|
|
135
|
+
Three operative principles:
|
|
136
|
+
|
|
137
|
+
1. **AI tools must be accessible.** A tool that screens out candidates
|
|
138
|
+
with disabilities — for example, a video-interview analyzer that
|
|
139
|
+
penalizes candidates with speech disabilities, or a gamified
|
|
140
|
+
assessment that's inaccessible to candidates with motor
|
|
141
|
+
disabilities — violates the ADA when no reasonable accommodation
|
|
142
|
+
is offered.
|
|
143
|
+
2. **Reasonable-accommodation obligation.** Employers must provide
|
|
144
|
+
alternative selection procedures, additional time, or modified
|
|
145
|
+
formats on request — the same obligation that applies to
|
|
146
|
+
traditional tests.
|
|
147
|
+
3. **Pre-employment medical inquiry rules apply.** AI tools that
|
|
148
|
+
probe for disability status (even indirectly, through behavioral
|
|
149
|
+
patterns) trigger the ADA's prohibition on pre-offer medical
|
|
150
|
+
inquiry.
|
|
151
|
+
|
|
152
|
+
The notice template needs to address both the Title VII concerns
|
|
153
|
+
(adverse impact, alternative procedure) and the ADA concerns
|
|
154
|
+
(accommodations).
|
|
155
|
+
|
|
156
|
+
## What an EEOC-compliant AI selection notice looks like
|
|
157
|
+
|
|
158
|
+
The EEOC's technical assistance recommends — but does not strictly
|
|
159
|
+
mandate — applicant notice and an alternative-procedure pathway. A
|
|
160
|
+
notice template that meets the federal floor and most state-level
|
|
161
|
+
overlays:
|
|
162
|
+
|
|
163
|
+
> Notice: This employer uses an automated decision-making (AI) tool
|
|
164
|
+
> to assist in evaluating applications and employment decisions. The
|
|
165
|
+
> tool's outputs are reviewed by human decision-makers and are
|
|
166
|
+
> subject to the federal Title VII non-discrimination requirements.
|
|
167
|
+
> If you would prefer an alternative, non-AI selection process, or
|
|
168
|
+
> require a reasonable accommodation under the Americans with
|
|
169
|
+
> Disabilities Act, please contact our human resources team at
|
|
170
|
+
> [contact].
|
|
171
|
+
|
|
172
|
+
For NYC Local Law 144 compliance, the notice must additionally:
|
|
173
|
+
- Be delivered at least 10 business days before AEDT use.
|
|
174
|
+
- Include the data the AEDT will use about the candidate.
|
|
175
|
+
- Reference where the bias audit results are publicly posted.
|
|
176
|
+
|
|
177
|
+
For Illinois HB 3773 compliance (post-2026-01-01), the notice must
|
|
178
|
+
identify the AI tool's role in the decision and the protected
|
|
179
|
+
classes considered.
|
|
180
|
+
|
|
181
|
+
## Common compliance failure patterns
|
|
182
|
+
|
|
183
|
+
- **No applicant notice.** The AI tool is used silently; applicants
|
|
184
|
+
don't know they were screened by AI. Gives applicants ground for
|
|
185
|
+
Title VII claim alleging procedural unfairness; in NYC, an AEDT
|
|
186
|
+
Local Law 144 violation per use.
|
|
187
|
+
- **Vendor audit treated as employer audit.** Employer relies on
|
|
188
|
+
vendor's marketing claim that the tool is "bias-tested" without
|
|
189
|
+
conducting its own four-fifths analysis on its actual output.
|
|
190
|
+
- **No alternative-procedure pathway.** Applicants have no way to
|
|
191
|
+
request a non-AI process or an ADA accommodation. Direct ADA
|
|
192
|
+
reasonable-accommodation violation; under stricter state rules,
|
|
193
|
+
per-incident liability.
|
|
194
|
+
- **No periodic re-audit.** Initial audit at deployment, no follow-up
|
|
195
|
+
as the tool retrains. The deployed tool may now fail four-fifths
|
|
196
|
+
even though the audit-on-record passed.
|
|
197
|
+
- **Adverse-impact data not segregated by tool.** Employer collects
|
|
198
|
+
EEO-1 data on hires but does not break out outcomes by which
|
|
199
|
+
candidates were AI-screened. Cannot detect AI-tool-specific bias.
|
|
200
|
+
- **NYC noncompliance for cross-state employers.** A national
|
|
201
|
+
employer applies a single national policy that doesn't meet NYC
|
|
202
|
+
Local Law 144's audit-and-publication mandate. Each NYC AEDT use
|
|
203
|
+
is a violation.
|
|
204
|
+
|
|
205
|
+
## How plainstamp helps
|
|
206
|
+
|
|
207
|
+
`plainstamp` ships a `us-eeoc-title-vii-ai-employment-2023` rule
|
|
208
|
+
that returns the recommended applicant notice elements, plain-
|
|
209
|
+
language and formal-language templates, citation back to Title VII +
|
|
210
|
+
the Uniform Guidelines + the EEOC technical assistance, and a
|
|
211
|
+
`last_verified` date. Lookup:
|
|
212
|
+
|
|
213
|
+
```bash
|
|
214
|
+
npx plainstamp lookup --jurisdiction us \
|
|
215
|
+
--channel ai-generated-content \
|
|
216
|
+
--use-case employment-decisions
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
For multi-state HR-tech, query each state's jurisdiction in parallel
|
|
220
|
+
to get the additional mandatory overlays:
|
|
221
|
+
|
|
222
|
+
```bash
|
|
223
|
+
npx plainstamp lookup --jurisdiction us-il --channel ai-generated-content --use-case employment-decisions
|
|
224
|
+
npx plainstamp lookup --jurisdiction us-ny-nyc --channel ai-generated-content --use-case employment-decisions
|
|
225
|
+
npx plainstamp lookup --jurisdiction us-co --channel ai-generated-content --use-case employment-decisions
|
|
226
|
+
npx plainstamp lookup --jurisdiction us-md --channel ai-generated-content --use-case employment-decisions
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
The disclosure copy must satisfy each applicable layer. The strictest
|
|
230
|
+
state rule typically governs.
|
|
231
|
+
|
|
232
|
+
## The minimum viable compliance posture
|
|
233
|
+
|
|
234
|
+
If your AI HR-tech deployment is starting from zero on EEOC + Title
|
|
235
|
+
VII compliance, ship these five artifacts in order:
|
|
236
|
+
|
|
237
|
+
1. **Applicant notice template.** Notice under the federal EEOC
|
|
238
|
+
guidance + the strictest applicable state/city rule. Includes the
|
|
239
|
+
alternative-procedure pathway and ADA accommodation contact.
|
|
240
|
+
2. **Four-fifths audit pipeline.** A monthly or quarterly job that
|
|
241
|
+
computes selection rates by protected class and surfaces any group
|
|
242
|
+
below four-fifths of the highest. Owned by HR compliance, not the
|
|
243
|
+
tool vendor.
|
|
244
|
+
3. **Alternative-procedure workflow.** A non-AI fallback selection
|
|
245
|
+
procedure, with documented decision criteria, that any candidate
|
|
246
|
+
can request without adverse consequence.
|
|
247
|
+
4. **ADA accommodation pathway.** A documented intake for
|
|
248
|
+
accommodation requests with clear SLAs for response, alternative
|
|
249
|
+
procedures, and modified formats.
|
|
250
|
+
5. **NYC Local Law 144 compliance** (if any NYC employees). Annual
|
|
251
|
+
independent bias audit by an "independent auditor" as defined in
|
|
252
|
+
the rule, public publication of audit summary on the employer's
|
|
253
|
+
website, and 10-business-day-advance applicant notice.
|
|
254
|
+
|
|
255
|
+
Then layer the higher-fidelity work — disparate-treatment risk
|
|
256
|
+
analysis, model-card transparency, employee interaction with the
|
|
257
|
+
tool — onto the higher-stakes use cases first.
|
|
258
|
+
|
|
259
|
+
## Source-of-truth links
|
|
260
|
+
|
|
261
|
+
- **EEOC Title VII technical assistance (May 2023)** ([eeoc.gov](https://www.eeoc.gov/laws/guidance/select-issues-assessing-adverse-impact-software-algorithms-and-artificial-intelligence))
|
|
262
|
+
- **EEOC ADA technical assistance (May 2022)** ([eeoc.gov](https://www.eeoc.gov/laws/guidance/americans-disabilities-act-and-use-software-algorithms-and-artificial-intelligence))
|
|
263
|
+
- **Title VII of the Civil Rights Act, 42 U.S.C. § 2000e** ([uscode.house.gov](https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title42-section2000e&num=0&edition=prelim))
|
|
264
|
+
- **Uniform Guidelines on Employee Selection Procedures, 29 CFR Part 1607** ([ecfr.gov](https://www.ecfr.gov/current/title-29/subtitle-B/chapter-XIV/part-1607))
|
|
265
|
+
- **NYC Local Law 144 (AEDT)** ([nyc.gov](https://rules.cityofnewyork.us/wp-content/uploads/2023/04/DCWP-NOA-for-Use-of-Automated-Employment-Decisionmaking-Tools-2.pdf))
|
|
266
|
+
- **Illinois HB 3773** ([ilga.gov](https://www.ilga.gov/legislation/billstatus.asp?DocNum=3773&GAID=17&DocTypeID=HB&LegID=152525&SessionID=112&GA=103))
|
|
267
|
+
|
|
268
|
+
`plainstamp` is maintained by an autonomous AI agent operating under
|
|
269
|
+
KS Elevated Solutions LLC. Accuracy reports, rule-update suggestions,
|
|
270
|
+
and security disclosures: [helpfulbutton140@agentmail.to](mailto:helpfulbutton140@agentmail.to).
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
[`← Back to plainstamp`](https://plainstamp.pages.dev/)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "plainstamp",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.5",
|
|
4
4
|
"description": "AI disclosure compliance assistant — generates legally-grounded AI disclosure text per (jurisdiction × channel × use-case) and tracks regulatory updates. Operated by an autonomous AI agent under KS Elevated Solutions LLC.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|