@1aboveio/skills 0.20.2 → 0.20.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. package/README.md +14 -6
  2. package/package.json +1 -1
  3. package/runtime/skills/distribution/generated/recipes.json +220 -25
  4. package/runtime/skills/distribution/scripts/bundles.mjs +167 -48
  5. package/runtime/skills/engineering/engineering-runtime/scripts/workflow-coherence.mjs +4 -0
  6. package/runtime/skills/engineering/engineering-runtime/scripts/workflow-policy.mjs +1 -1
  7. package/skills/compliance/compliance-audit/LICENSE +3 -0
  8. package/skills/compliance/compliance-audit/SKILL.md +466 -0
  9. package/skills/compliance/compliance-audit/references/findings-schema.md +290 -0
  10. package/skills/compliance/compliance-audit/references/vars-checklist.md +1068 -0
  11. package/skills/compliance/compliance-audit/references/vars-findings-schema.md +395 -0
  12. package/skills/compliance/compliance-audit/scripts/reconcile_tracker.py +454 -0
  13. package/skills/compliance/compliance-audit/scripts/test_validate_tracker.py +90 -0
  14. package/skills/compliance/compliance-audit/scripts/validate_tracker.py +752 -0
  15. package/skills/compliance/doc-authenticity/LICENSE +3 -0
  16. package/skills/compliance/doc-authenticity/SKILL.md +158 -0
  17. package/skills/compliance/doc-authenticity/references/agreement.md +66 -0
  18. package/skills/compliance/doc-authenticity/references/bank-statement.md +150 -0
  19. package/skills/compliance/doc-authenticity/references/forensic-signals.md +157 -0
  20. package/skills/compliance/doc-authenticity/references/image-signals.md +114 -0
  21. package/skills/compliance/doc-authenticity/references/invoice.md +66 -0
  22. package/skills/compliance/doc-authenticity/references/issuer-profiles.md +86 -0
  23. package/skills/compliance/doc-authenticity/references/verdict-and-report.md +95 -0
  24. package/skills/compliance/doc-authenticity/references/verdict.schema.json +81 -0
  25. package/skills/compliance/doc-authenticity/scripts/image_forensics.py +294 -0
  26. package/skills/compliance/doc-authenticity/scripts/pdf_forensics.py +401 -0
  27. package/skills/compliance/doc-authenticity/scripts/validate_codes.py +224 -0
  28. package/skills/compliance/doc-authenticity/scripts/validate_verdict.py +308 -0
  29. package/skills/compliance/report-writing/LICENSE +3 -0
  30. package/skills/compliance/report-writing/SKILL.md +206 -0
  31. package/skills/compliance/report-writing/references/audit-report-template.md +121 -0
  32. package/skills/compliance/report-writing/references/external-assessment-template.md +167 -0
  33. package/skills/compliance/report-writing/references/management-report-template.md +145 -0
  34. package/skills/compliance/report-writing/references/vars-report-template.md +224 -0
  35. package/skills/document-utils/doc-to-md/LICENSE +3 -0
  36. package/skills/document-utils/doc-to-md/SKILL.md +77 -0
  37. package/skills/document-utils/doc-to-md/scripts/convert.py +157 -0
  38. package/skills/document-utils/reg-parser/LICENSE +3 -0
  39. package/skills/document-utils/reg-parser/SKILL.md +201 -0
  40. package/skills/document-utils/reg-parser/references/checklist-schema.md +223 -0
  41. package/skills/document-utils/reg-parser/references/checklists/hk-amlo-checklist.json +1306 -0
  42. package/skills/document-utils/reg-parser/references/checklists/hk-amlo-checklist.md +469 -0
  43. package/skills/document-utils/reg-parser/references/checklists/hk-fi-corporate-onboarding-cdd-checklist.json +634 -0
  44. package/skills/document-utils/reg-parser/references/checklists/hk-fi-corporate-onboarding-cdd-checklist.md +233 -0
  45. package/skills/document-utils/reg-parser/references/checklists/hk-mso-guideline-checklist.json +2194 -0
  46. package/skills/document-utils/reg-parser/references/checklists/hk-mso-guideline-checklist.md +771 -0
  47. package/skills/document-utils/reg-parser/references/checklists/kyc-aml-checklist.md +258 -0
  48. package/skills/document-utils/reg-parser/references/checklists/visa-vars-checklist.md +1058 -0
  49. package/skills/document-utils/reg-parser/references/mandates/hk-amlo-cap615.md +389 -0
  50. package/skills/document-utils/reg-parser/references/mandates/visa-vars.md +205 -0
  51. package/skills/document-utils/reg-parser/scripts/compare_checklist.py +228 -0
  52. package/skills/document-utils/reg-parser/scripts/dump_checklist.py +348 -0
  53. package/skills/document-utils/reg-parser/scripts/preprocess-mandate.sh +171 -0
  54. package/skills/document-utils/reg-parser/scripts/test_quarantine.sh +102 -0
  55. package/skills/document-utils/reg-parser/scripts/validate_checklist.py +532 -0
  56. package/skills/document-utils/reg-parser/scripts/validate_mandate.py +242 -0
  57. package/skills/engineering/engineering-runtime/coherence/workflow.json +16 -16
  58. package/skills/engineering/engineering-runtime/scripts/workflow-coherence.mjs +4 -0
  59. package/skills/engineering/engineering-runtime/scripts/workflow-policy.mjs +1 -1
  60. package/skills/payment/similar-domain/LICENSE +3 -0
  61. package/skills/payment/similar-domain/SKILL.md +178 -0
  62. package/skills/payment/similar-domain/references/entities.md +58 -0
  63. package/skills/payment/similar-domain/references/fingerprints.md +82 -0
  64. package/skills/payment/similar-domain/references/pivots.md +66 -0
  65. package/skills/payment/similar-domain/references/report-template.md +76 -0
  66. package/skills/payment/similar-domain/references/scoring.md +61 -0
  67. package/skills/payment/similar-domain/references/terminology.md +37 -0
  68. package/skills/payment/similar-domain/references/workflow.md +79 -0
  69. package/skills/payment/tld-detection/SKILL.md +76 -21
  70. package/skills/payment/tld-detection/references/clean.md +1 -1
  71. package/skills/payment/tld-detection/references/clustering.md +21 -6
  72. package/skills/payment/tld-detection/references/decision.md +7 -2
  73. package/skills/payment/tld-detection/references/fx-conversion.md +229 -0
  74. package/skills/payment/tld-detection/references/report-template.md +63 -27
  75. package/skills/payment/tld-detection/references/streams.md +90 -0
  76. package/skills/payment/tld-detection/references/terminology.md +15 -7
  77. package/skills/payment/tld-detection/references/workflow.md +1 -1
  78. package/skills/payment/tld-detection/scripts/detect_streams.py +542 -0
  79. package/skills/engineering/ensure-coverage/evals/evals.json +0 -125
  80. package/skills/engineering/rush-issues/evals/evals.json +0 -65
  81. package/skills/engineering/rush-release/evals/evals.json +0 -58
  82. package/skills/fullstack/shadcn/evals/evals.json +0 -90
  83. package/skills/payment/fraud-analysis/evals/evals.json +0 -40
  84. package/skills/payment/tld-detection/evals/evals.json +0 -107
  85. package/skills/payment/tld-detection/tests/test_detect_ladders.py +0 -132
@@ -0,0 +1,242 @@
1
+ #!/usr/bin/env python3
2
+ """Validate a mandate (Mode A — Clause Annotation Extract) Markdown file.
3
+
4
+ Catches the failure modes specific to mandate extracts produced by the
5
+ `reg-parser` skill:
6
+
7
+ - Agent left the raw template scaffolding behind (Status: RAW, HTML
8
+ comment placeholders, the embedded raw-text fenced block).
9
+ - One of the four canonical sections is missing or silently empty.
10
+ - Header metadata (Source attribution, jurisdiction/version/date) is
11
+ missing.
12
+ - Notes section is empty or stub.
13
+
14
+ Optional --source <converted-md> enables citation-existence warnings:
15
+ each `§<ref>` cited in the extract is fuzzy-matched against the source
16
+ markdown. Missing citations emit warnings, not errors (refs are often
17
+ restyled during extraction).
18
+
19
+ Usage:
20
+ python3 validate_mandate.py <mandate.md> [--source <converted-md>]
21
+
22
+ Exit code 0 = valid. Non-zero = errors found.
23
+ """
24
+
25
+ import argparse
26
+ import re
27
+ import sys
28
+ import os
29
+
30
+ REQUIRED_SECTIONS = [
31
+ "Mandatory Downstream Agreement Clauses",
32
+ "Prohibited Conduct",
33
+ "Thresholds & Triggers",
34
+ "Audit & Oversight Rights",
35
+ "Notes",
36
+ ]
37
+
38
+ # Markers an agent may use to indicate a section legitimately has no entries.
39
+ EMPTY_MARKERS = ("_none_", "(none)", "n/a", "not applicable", "none.")
40
+
41
+ # Header metadata keywords — at least one must appear in the header block
42
+ # (between H1 and the first H2). Matches the conventions in the existing
43
+ # extracts (visa-vars.md, hk-amlo-cap615.md).
44
+ HEADER_KEYWORDS = ("source", "jurisdiction", "version", "effective", "updated", "date")
45
+
46
+ # Patterns that indicate the agent left raw template artifacts behind.
47
+ RAW_STATUS_RE = re.compile(r"^\s*>\s*\*?\*?Status:\*?\*?\s*RAW", re.IGNORECASE)
48
+ RAW_TEXT_HEADING_RE = re.compile(r"^##\s+Raw Text\b", re.IGNORECASE)
49
+ SCAFFOLDING_COMMENT_RES = [
50
+ re.compile(r"<!--\s*Agent:\s*replace", re.IGNORECASE),
51
+ re.compile(r"<!--\s*Format:\s*\*\*§\[ref\]"),
52
+ re.compile(r"<!--\s*Clauses the counterparty"),
53
+ re.compile(r"<!--\s*Explicitly prohibited"),
54
+ re.compile(r"<!--\s*Numeric thresholds"),
55
+ re.compile(r"<!--\s*Rights granted"),
56
+ re.compile(r"<!--\s*Jurisdiction scope, effective date"),
57
+ ]
58
+
59
+ # A §-style citation. Captures common forms: §AACQ.C2.1, §76E,
60
+ # §Sched.2 §2(1)(a), § 14(3)(b). The capturing group keeps just the
61
+ # alphanumeric/punctuation token; the leading § is consumed.
62
+ CITATION_RE = re.compile(r"§\s*([A-Za-z0-9][A-Za-z0-9.\-]*(?:\([^)]+\))*)")
63
+
64
+ H1_RE = re.compile(r"^#\s+(.+?)\s*$")
65
+ H2_RE = re.compile(r"^##\s+(.+?)\s*$")
66
+
67
+
68
+ def split_sections(lines: list[str]) -> tuple[list[str], dict[str, list[str]]]:
69
+ """Return (header_lines, {section_title: body_lines})."""
70
+ header: list[str] = []
71
+ sections: dict[str, list[str]] = {}
72
+ current: str | None = None
73
+ seen_h1 = False
74
+
75
+ for line in lines:
76
+ if not seen_h1 and H1_RE.match(line):
77
+ seen_h1 = True
78
+ continue
79
+ h2 = H2_RE.match(line)
80
+ if h2:
81
+ current = h2.group(1).strip()
82
+ sections.setdefault(current, [])
83
+ continue
84
+ if current is None:
85
+ header.append(line)
86
+ else:
87
+ sections[current].append(line)
88
+
89
+ return header, sections
90
+
91
+
92
+ def section_is_empty(body: list[str]) -> bool:
93
+ """A section is empty if it has no §-bullet entries AND no explicit
94
+ 'none' marker."""
95
+ text = "\n".join(body).strip()
96
+ if not text:
97
+ return True
98
+ lower = text.lower()
99
+ if any(marker in lower for marker in EMPTY_MARKERS):
100
+ return False
101
+ if "**§" in text or re.search(r"^\s*§", text, re.MULTILINE):
102
+ return False
103
+ if "§" in text and re.search(r"§\s*[A-Za-z0-9]", text):
104
+ return False
105
+ return True
106
+
107
+
108
+ def extract_citations(text: str) -> set[str]:
109
+ """Pull §-style citation tokens out of arbitrary text."""
110
+ return {m.group(1) for m in CITATION_RE.finditer(text)}
111
+
112
+
113
+ def fuzzy_present(token: str, haystack_lower: str) -> bool:
114
+ """Best-effort: token appears in source if either the literal token
115
+ or a heavily-normalized form (lowercased, punctuation stripped) does.
116
+ """
117
+ if token.lower() in haystack_lower:
118
+ return True
119
+ norm = re.sub(r"[^a-z0-9]", "", token.lower())
120
+ if not norm:
121
+ return False
122
+ norm_haystack = re.sub(r"[^a-z0-9]", "", haystack_lower)
123
+ return norm in norm_haystack
124
+
125
+
126
+ def validate(filepath: str, source_path: str | None) -> tuple[list[str], list[str]]:
127
+ """Return (errors, warnings)."""
128
+ errors: list[str] = []
129
+ warnings: list[str] = []
130
+
131
+ if not os.path.isfile(filepath):
132
+ return [f"File not found: {filepath}"], []
133
+
134
+ with open(filepath, "r", encoding="utf-8") as f:
135
+ raw = f.read()
136
+ lines = raw.splitlines()
137
+
138
+ # --- Raw template artifacts ---
139
+ for line_num, line in enumerate(lines, 1):
140
+ if RAW_STATUS_RE.match(line):
141
+ errors.append(
142
+ f"Line {line_num}: 'Status: RAW' marker still present — "
143
+ f"agent extraction not completed"
144
+ )
145
+ if RAW_TEXT_HEADING_RE.match(line):
146
+ errors.append(
147
+ f"Line {line_num}: '## Raw Text' section still present — "
148
+ f"remove the embedded raw-text dump after extraction"
149
+ )
150
+ for pat in SCAFFOLDING_COMMENT_RES:
151
+ if pat.search(line):
152
+ errors.append(
153
+ f"Line {line_num}: leftover template scaffolding comment"
154
+ )
155
+ break
156
+
157
+ # --- Structure ---
158
+ header_lines, sections = split_sections(lines)
159
+
160
+ if not any(H1_RE.match(line) for line in lines):
161
+ errors.append("Missing H1 title (expected '# <Source> — Contract Obligation Extract')")
162
+
163
+ # Header metadata: at least one of Source/Jurisdiction/Version/etc.
164
+ header_text = "\n".join(header_lines).lower()
165
+ if not any(kw in header_text for kw in HEADER_KEYWORDS):
166
+ errors.append(
167
+ "Header metadata block missing — expected blockquote with "
168
+ "Source / Jurisdiction / Version / Effective Date"
169
+ )
170
+
171
+ # The Source attribution specifically is critical.
172
+ if "source" not in header_text:
173
+ errors.append("Header missing 'Source:' attribution")
174
+
175
+ # Required sections present
176
+ for required in REQUIRED_SECTIONS:
177
+ if required not in sections:
178
+ errors.append(f"Missing required section: ## {required}")
179
+
180
+ # Required content sections non-empty
181
+ for required in REQUIRED_SECTIONS[:-1]: # everything except Notes
182
+ body = sections.get(required)
183
+ if body is not None and section_is_empty(body):
184
+ errors.append(
185
+ f"Section '## {required}' is empty — if there are no "
186
+ f"items, mark explicitly as '_None_'"
187
+ )
188
+
189
+ # Notes substance check
190
+ notes_body = sections.get("Notes")
191
+ if notes_body is not None:
192
+ notes_text = "\n".join(notes_body).strip()
193
+ if len(notes_text) < 50:
194
+ errors.append(
195
+ "Section '## Notes' is too short — include jurisdiction "
196
+ "scope, effective date, version, and caveats"
197
+ )
198
+
199
+ # --- Optional: citation existence check ---
200
+ if source_path:
201
+ if not os.path.isfile(source_path):
202
+ warnings.append(f"--source file not found: {source_path}")
203
+ else:
204
+ with open(source_path, "r", encoding="utf-8") as sf:
205
+ source_lower = sf.read().lower()
206
+ cites = extract_citations(raw)
207
+ missing = sorted(c for c in cites if not fuzzy_present(c, source_lower))
208
+ if missing:
209
+ preview = ", ".join(f"§{c}" for c in missing[:8])
210
+ more = "" if len(missing) <= 8 else f" (+{len(missing) - 8} more)"
211
+ warnings.append(
212
+ f"{len(missing)} citation(s) not found in source: {preview}{more}"
213
+ )
214
+
215
+ return errors, warnings
216
+
217
+
218
+ def main() -> None:
219
+ p = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
220
+ p.add_argument("filepath", help="Path to mandate Markdown file")
221
+ p.add_argument(
222
+ "--source",
223
+ help="Optional path to the converted source Markdown for citation checks",
224
+ )
225
+ args = p.parse_args()
226
+
227
+ errors, warnings = validate(args.filepath, args.source)
228
+
229
+ for w in warnings:
230
+ print(f" ⚠️ {w}", file=sys.stderr)
231
+
232
+ if errors:
233
+ print(f"❌ Validation failed for {args.filepath}:", file=sys.stderr)
234
+ for err in errors:
235
+ print(f" • {err}", file=sys.stderr)
236
+ sys.exit(1)
237
+ print(f"✅ {args.filepath} is valid.")
238
+ sys.exit(0)
239
+
240
+
241
+ if __name__ == "__main__":
242
+ main()
@@ -342,7 +342,7 @@
342
342
  "id": "first-party",
343
343
  "type": "first-party",
344
344
  "package": "@1aboveio/skills",
345
- "version": "0.20.2"
345
+ "version": "0.20.4"
346
346
  },
347
347
  "contentDigest": "eff6c7b5931bce5b2265a619bccddd371a89df6ce7bc2edc74f11d00060a71dd",
348
348
  "digestExcludes": []
@@ -355,7 +355,7 @@
355
355
  "id": "first-party",
356
356
  "type": "first-party",
357
357
  "package": "@1aboveio/skills",
358
- "version": "0.20.2"
358
+ "version": "0.20.4"
359
359
  },
360
360
  "contentDigest": "d97abeff7bdc5ae2e4be2f631d09dba46d4753ce1e67334793364329af59f066",
361
361
  "digestExcludes": []
@@ -368,9 +368,9 @@
368
368
  "id": "first-party",
369
369
  "type": "first-party",
370
370
  "package": "@1aboveio/skills",
371
- "version": "0.20.2"
371
+ "version": "0.20.4"
372
372
  },
373
- "contentDigest": "00cfdd3e5bfbc3e37b1369ad93aae91161e671829760a8867ea9be25047ba461",
373
+ "contentDigest": "fe2ec3bae2bfec2dbef19db30ab0272391d7c1d9b6893341a76a9f339adefc57",
374
374
  "digestExcludes": []
375
375
  },
376
376
  {
@@ -381,7 +381,7 @@
381
381
  "id": "first-party",
382
382
  "type": "first-party",
383
383
  "package": "@1aboveio/skills",
384
- "version": "0.20.2"
384
+ "version": "0.20.4"
385
385
  },
386
386
  "contentDigest": "58b0556228a9271cf9e33727a05fc08a4fdfd29512ec0936b169f0a55d60e6ef",
387
387
  "digestExcludes": []
@@ -394,9 +394,9 @@
394
394
  "id": "first-party",
395
395
  "type": "first-party",
396
396
  "package": "@1aboveio/skills",
397
- "version": "0.20.2"
397
+ "version": "0.20.4"
398
398
  },
399
- "contentDigest": "4664873230f22de2fcf46b7b529570cfedabdc850620031d8f96baf89fcdc5b4",
399
+ "contentDigest": "d28dd4d3c4394df0cc7e6b48d0e5daa7d2277701ea459e39fba99acf1d8ff670",
400
400
  "digestExcludes": []
401
401
  },
402
402
  {
@@ -407,9 +407,9 @@
407
407
  "id": "first-party",
408
408
  "type": "first-party",
409
409
  "package": "@1aboveio/skills",
410
- "version": "0.20.2"
410
+ "version": "0.20.4"
411
411
  },
412
- "contentDigest": "cd9f398f6d8d1df7c2d944d3bec014e788535638014f327783f074757bfb4f49",
412
+ "contentDigest": "56fb5843788c9073b0d5a65ffa72bd7d2fa80dc6a1ff06fb0e19fab513c51502",
413
413
  "digestExcludes": []
414
414
  },
415
415
  {
@@ -420,7 +420,7 @@
420
420
  "id": "first-party",
421
421
  "type": "first-party",
422
422
  "package": "@1aboveio/skills",
423
- "version": "0.20.2"
423
+ "version": "0.20.4"
424
424
  },
425
425
  "contentDigest": "f048fd00c69f2dc666fc7a3096cfeee6dfba73933bfb69602f3f0e26159798cc",
426
426
  "digestExcludes": []
@@ -433,7 +433,7 @@
433
433
  "id": "first-party",
434
434
  "type": "first-party",
435
435
  "package": "@1aboveio/skills",
436
- "version": "0.20.2"
436
+ "version": "0.20.4"
437
437
  },
438
438
  "contentDigest": "d533684db4483cfc90be2aa7161980b6548057ef18a4b54f0cdbcd53c79dd46d",
439
439
  "digestExcludes": []
@@ -446,15 +446,15 @@
446
446
  "id": "first-party",
447
447
  "type": "first-party",
448
448
  "package": "@1aboveio/skills",
449
- "version": "0.20.2"
449
+ "version": "0.20.4"
450
450
  },
451
- "contentDigest": "f7f56f7e33b2b12c3043df0f4729447dfccd181daa79936ff62da38df91d5175",
451
+ "contentDigest": "86a91b567eabe6143a4ecac3cbec7271da252978202f292e287bb3a8da95a797",
452
452
  "digestExcludes": [
453
453
  "coherence/workflow.json"
454
454
  ]
455
455
  }
456
456
  ],
457
- "releaseIdentity": "455424f618fddc62d553f7c89282887d6b449b278eb0335b72bf415f760f1b34",
457
+ "releaseIdentity": "ffa735d74d402c79fbd8cbca332b415422884ee88f914412d4aa88c2d1353cb9",
458
458
  "lifecycleAuthority": "native-skills-cli",
459
459
  "repairRecipe": {
460
460
  "id": "engineering-workflow-dependency-first",
@@ -512,7 +512,7 @@
512
512
  "sourceId": "first-party",
513
513
  "sourceType": "first-party",
514
514
  "package": "@1aboveio/skills",
515
- "version": "0.20.2",
515
+ "version": "0.20.4",
516
516
  "installPath": null,
517
517
  "members": [
518
518
  "harness-runtime",
@@ -528,7 +528,7 @@
528
528
  "commands": [
529
529
  {
530
530
  "transport": "npm",
531
- "command": "npx @1aboveio/skills@0.20.2 install --group engineering-workflow --yes"
531
+ "command": "npx @1aboveio/skills@0.20.4 install --group engineering-workflow --yes"
532
532
  }
533
533
  ],
534
534
  "onFailure": {
@@ -154,6 +154,10 @@ const TEST_FIXTURE = /\.fixtures?\.[^.]+$/;
154
154
  export function publishedContentExcluded(name) {
155
155
  return name.startsWith('.')
156
156
  || name === 'node_modules'
157
+ || name === '__pycache__'
158
+ || name === 'evals'
159
+ || name === 'samples'
160
+ || name === 'tests'
157
161
  || name.endsWith('-workspace')
158
162
  || name.endsWith('.test.mjs')
159
163
  || TEST_FIXTURE.test(name);
@@ -86,7 +86,7 @@ export const WORKFLOW_TRUSTED_SOURCES = deepFreeze({
86
86
  id: 'first-party',
87
87
  type: 'first-party',
88
88
  package: '@1aboveio/skills',
89
- version: '0.20.2',
89
+ version: '0.20.4',
90
90
  },
91
91
  matt: {
92
92
  id: 'matt-pocock',
@@ -0,0 +1,3 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 1AboveIO
@@ -0,0 +1,178 @@
1
+ ---
2
+ name: similar-domain
3
+ description: >
4
+ Find the domains, mirrors, and operating entities related to one or more seed
5
+ domains: collect technical fingerprints (analytics IDs, block-page templates,
6
+ DNS, mail, certificates, registration), reverse-pivot on each fingerprint,
7
+ score every link, and expand until the cluster stops growing. Use whenever
8
+ the user gives a domain or URL and asks who is behind it, whether several
9
+ sites are related or the same operator, for sister sites, mirror domains,
10
+ the cluster, or a merchant-website background check — even if they never say
11
+ similar-domain. Stops at websites, mirrors and operators: anything inside
12
+ transaction data (streams, preset menus, amount ladders) is tld-detection,
13
+ and transaction-level fraud rules are fraud-analysis.
14
+ ---
15
+
16
+ # Similar Domain
17
+
18
+ Start from a seed domain and find every other domain that the same people run.
19
+ A cluster is defined by **shared infrastructure**, not by shared branding or
20
+ shared industry. Corporate records (operator, licence, affiliate program)
21
+ confirm and name the cluster; they do not create it on their own.
22
+
23
+ The method is a loop: read fingerprints off a domain, ask "who else has this?"
24
+ for each fingerprint, score what comes back, and feed every new domain back
25
+ into the loop.
26
+
27
+ **Produce:** a scored cluster with an evidence matrix, the entity chain behind
28
+ it, and a ≤3-sentence verdict, under `reports/similar-domain/`.
29
+ **Do not produce:** an ownership claim with no corporate evidence, a cluster
30
+ built from one weak signal, or a list of every site on the same CDN.
31
+
32
+ Paths under `references/` are relative to this skill.
33
+
34
+ ## Workflow
35
+
36
+ Follow these steps in order. Detail:
37
+ [references/workflow.md](references/workflow.md).
38
+
39
+ | Step | Do | Gate |
40
+ |---|---|---|
41
+ | 0 Pre-flight | Seeds, question, jurisdiction, access limits | Stop if there is no resolvable domain; note geo-blocks and challenge pages |
42
+ | 1 Fingerprint | Record every non-brand identifier on each domain | Every fingerprint has a source and a timestamp; local resolver artefacts are flagged |
43
+ | 2 Pivot | Reverse-search each fingerprint for other domains | Each pivot is logged with source class and hit count; high-noise pivots are filtered by a second fingerprint |
44
+ | 3 Expand names | Enumerate mirror and numbering variants of every brand found | Candidates are only kept if they share a fingerprint from step 1 |
45
+ | 4 Entity | Operator, licence, address, affiliate program, terms page | Each entity fact cites a page; conflicting operator names are kept, not resolved by guess |
46
+ | 5 Score | Weight every link; label each domain | Two independent strong signals for Confirmed; one for Probable; nothing weaker than Possible enters the cluster table |
47
+ | 6 Loop | Send every new Confirmed or Probable domain back to step 1 | Stop when a full pass adds no new domain |
48
+ | 7 Report | Verdict, cluster table, evidence matrix, entity chain, outliers, pivot log | ≤3-sentence verdict; no ownership wording without an entity citation |
49
+
50
+ ### 0 — Pre-flight
51
+
52
+ Record the seed domains, what the user actually wants to know (same
53
+ operator? mirrors? merchant background?), and any jurisdiction or time limits.
54
+ Note what you cannot see: geo-blocks, bot challenges, private WHOIS. A blocked
55
+ site still yields a block page, DNS, certificates, and history, so blocking is
56
+ a finding, not a dead end. Grain and limits:
57
+ [workflow.md](references/workflow.md).
58
+
59
+ ### 1 — Fingerprint
60
+
61
+ [references/fingerprints.md](references/fingerprints.md). Collect
62
+ everything about the domain that is not the brand name: registration facts,
63
+ DNS and mail records, certificate history, analytics and pixel IDs, the
64
+ block-page or error-page template, support-address pattern, hard-coded
65
+ partner hosts, and naming scheme. Record each with where it came from.
66
+ Fingerprint the seed fully before pivoting; a half-collected profile makes
67
+ later pivots look independent when they are not.
68
+
69
+ ### 2 — Pivot
70
+
71
+ [references/pivots.md](references/pivots.md). For each fingerprint, query the
72
+ source class that can answer "who else has this": page-source indexes for
73
+ analytics IDs and templates, passive DNS for nameservers and mail records,
74
+ certificate transparency for subdomains and naming schemes, reverse WHOIS for
75
+ registration batches. Ordinary web search does not index page source; do not
76
+ treat an empty web-search result as a negative. Filter noisy pivots
77
+ (shared CDN nameservers, common mail includes) by requiring a second
78
+ fingerprint before a hit becomes a candidate.
79
+
80
+ ### 3 — Expand names
81
+
82
+ Brands that rotate mirrors follow a scheme: numbered suffixes, hyphenated
83
+ digits, double-dash prefixes, country TLDs. Enumerate the scheme for every
84
+ brand found and keep only the variants that share a step-1 fingerprint.
85
+ A name match alone is a candidate, never a member.
86
+
87
+ ### 4 — Entity
88
+
89
+ [references/entities.md](references/entities.md). Once a brand name exists,
90
+ collect the legal entity, licence number, address, and affiliate program from
91
+ the site's own terms page and from review aggregators, then reverse each of
92
+ those: licence registers list every domain under one licence, affiliate
93
+ programs list every brand they pay for, entity names appear in other sites'
94
+ terms pages. Keep conflicting operator names side by side with dates; operators
95
+ migrate and the conflict is often the history.
96
+
97
+ ### 5 — Score
98
+
99
+ [references/scoring.md](references/scoring.md). Weight each link by how hard
100
+ it is to share by accident. One analytics account or one custom template is
101
+ very strong; one registrar or one CDN nameserver pair is medium. Two signals
102
+ from the same vendor account count once. Label each domain Confirmed,
103
+ Probable, Possible, or Unrelated, and keep the reasoning visible in the
104
+ evidence matrix.
105
+
106
+ ### 6 — Loop
107
+
108
+ Every Confirmed or Probable domain becomes a new seed and goes through steps 1
109
+ to 5. Possible domains are parked with the fingerprint that would promote them.
110
+ Stop when a complete pass finds nothing new, or when the user's question is
111
+ answered and further expansion would only add mirrors.
112
+
113
+ ### 7 — Report
114
+
115
+ [references/report-template.md](references/report-template.md). Terms:
116
+ [references/terminology.md](references/terminology.md).
117
+
118
+ ```text
119
+ reports/similar-domain/<seed>_cluster.md
120
+ ```
121
+
122
+ Lead with the verdict, then the cluster table sorted Confirmed → Probable →
123
+ Possible, the evidence matrix (domain × fingerprint), the entity chain with
124
+ dates, outliers that were checked and rejected, coverage caveats, and the
125
+ pivot log. Grep banned strings after writing.
126
+
127
+ ## Progress
128
+
129
+ Track a task plan. One item is `in_progress` at a time.
130
+
131
+ 1. Pass pre-flight
132
+ 2. Fingerprint every seed
133
+ 3. Run and log pivots
134
+ 4. Expand naming schemes
135
+ 5. Collect entity chain
136
+ 6. Score and label
137
+ 7. Loop until no new domain
138
+ 8. Write report and proofread
139
+
140
+ ## Boundary
141
+
142
+ | Need | Skill |
143
+ |---|---|
144
+ | Related domains, mirrors, sister sites, who runs a website, merchant-site background | `similar-domain` |
145
+ | Anything inside a transaction file: upstream streams behind a descriptor, wallet currency, preset menus, frozen-rate batches, retail constants, currency-amount ladders | `tld-detection` (`scripts/detect_streams.py`, then the ladder run) |
146
+ | Supervised fraud labels, holdout precision/recall, rule packages | `fraud-analysis` |
147
+ | Volume, journey, auth rate, decline mix | `payment-analysis` |
148
+
149
+ A domain cluster from this skill can feed a merchant-risk decision; it is not
150
+ itself a fraud finding. When the user also has authorization data for the
151
+ merchant, hand that part to `tld-detection`: this skill stops at the websites,
152
+ mirrors and operators.
153
+
154
+ ## Reference index
155
+
156
+ | Need | Read |
157
+ |---|---|
158
+ | Inputs, limits, order that must not move | [workflow.md](references/workflow.md) |
159
+ | What to collect per domain and why it identifies an operator | [fingerprints.md](references/fingerprints.md) |
160
+ | Which source class answers each "who else" question, and the traps | [pivots.md](references/pivots.md) |
161
+ | Operator, licence, affiliate, terms-page pivots | [entities.md](references/entities.md) |
162
+ | Link weights, independence, labels | [scoring.md](references/scoring.md) |
163
+ | Report chapters | [report-template.md](references/report-template.md) |
164
+ | Language and banned strings | [terminology.md](references/terminology.md) |
165
+
166
+ ## Sanity check
167
+
168
+ - [ ] Seeds, question, and access limits are stated; blocked sites are still fingerprinted
169
+ - [ ] Every fingerprint has a source and a timestamp; local resolver artefacts are excluded
170
+ - [ ] Every pivot is logged with source class, query, and hit count, including empty ones
171
+ - [ ] Noisy pivots (CDN nameservers, common mail includes) never add a domain on their own
172
+ - [ ] Name-scheme candidates are kept only when they share a fingerprint
173
+ - [ ] Entity facts cite the page they came from; conflicting operator names are dated, not merged
174
+ - [ ] Two signals from one vendor account are counted once
175
+ - [ ] Every cluster member has a label and the matrix shows why
176
+ - [ ] Loop ran until a full pass added nothing, or the stop reason is stated
177
+ - [ ] Verdict is ≤3 sentences; no "owned by" without an entity citation
178
+ - [ ] Outliers checked and rejected are listed with the reason
@@ -0,0 +1,58 @@
1
+ # Entities
2
+
3
+ Step 4 names and dates the cluster. It runs after at least one brand name
4
+ is known and after the technical links exist; it explains the cluster, it
5
+ does not replace the evidence.
6
+
7
+ ## What to collect per brand
8
+
9
+ | Fact | Where it usually is |
10
+ |---|---|
11
+ | Legal operator name and company number | Site footer, terms and conditions, "about" page, review aggregators |
12
+ | Licence authority and number | Footer seal, validator link, regulator register |
13
+ | Registered address | Terms page, regulator register |
14
+ | Launch year and relaunches | Review aggregators, archives, certificate history |
15
+ | Affiliate program | Footer "Affiliates" link, affiliate directories |
16
+ | Platform vendor | Footer, script hosts, review sites |
17
+ | Previous operators | Older reviews, forum threads, archive snapshots |
18
+ | Complaint patterns | Review aggregators, complaint forums |
19
+
20
+ Record each fact with its page and the date the page was written or
21
+ captured. Review aggregators disagree with each other because operators
22
+ migrate; keep every version with its date rather than picking one.
23
+
24
+ ## Entity pivots
25
+
26
+ | Have | Reverse into | Yields |
27
+ |---|---|---|
28
+ | Operator name | Aggregator "owner" pages, licence register, web search of terms-page text | Brand list under that operator |
29
+ | Licence number | Regulator register | Every licensed domain, including ones with no technical link yet |
30
+ | Affiliate program | Affiliate directories and forums | Brands marketed together |
31
+ | Address | Web search | Other entities at the address; treat corporate-service addresses as noise |
32
+ | Platform vendor | Vendor's client list, review sites | Vendor-level neighbours; filter, do not add |
33
+ | Previous operator | Same as operator name | The migration path (for example Curaçao entity → transitional entity → Anjouan entity) |
34
+
35
+ ## Migration patterns worth expecting
36
+
37
+ Offshore operators reorganise often. Common shapes:
38
+
39
+ - **Entity swap, same platform** — the brand keeps its template and
40
+ tokens; only the footer entity and licence change. Technical evidence
41
+ stays continuous; corporate evidence shows a break.
42
+ - **Brand acquisition** — an old, once-reputable domain is bought and
43
+ relaunched on a new operator's platform. Registration date is old;
44
+ everything else matches the new cluster. Weight the technical evidence.
45
+ - **Mirror farms** — numbered or hyphenated domains registered in batches,
46
+ each redirecting or serving the parent brand. They inherit every
47
+ fingerprint and are usually Confirmed on the first pass.
48
+ - **White-label tenants** — several unrelated operators on one vendor
49
+ platform share vendor-level fingerprints but not operator-level ones.
50
+ This is the main source of false clusters.
51
+
52
+ ## Adding a domain from entity evidence alone
53
+
54
+ Allowed only when the source is authoritative for domain membership: a
55
+ regulator's licence register, or the operator's own terms page listing its
56
+ domains. Label such domains Probable until a technical fingerprint
57
+ confirms them. Aggregator sister-site lists are hypotheses to test, not
58
+ grounds for membership.