@1aboveio/skills 1.0.10 → 1.0.11

package/skills/risk-compliance/tld-detection/references/report-template.md

@@ -1,272 +1,49 @@
-# Report template
-
-Save:
-
-```text
-reports/tld-detection/<scope>_clusters.md
-reports/tld-detection/<scope>_clusters.zh.md
-reports/tld-detection/<scope>_clusters.zh.html
-reports/tld-detection/<scope>_evidence.md
-reports/tld-detection/<scope>_evidence.zh.md
-reports/tld-detection/<scope>_evidence.zh.html
-reports/tld-detection/<scope>_methodology.md
-reports/tld-detection/<scope>_methodology.zh.md
-reports/tld-detection/<scope>_methodology.zh.html
-```
-
-HTML is preferred when there are charts (Chart.js bars). No Markdown bold in report body.
-The main report (`_clusters`), complete evidence (`_evidence`), and methodology
-must link to one another. These are distinct reading layers, not three copies
-of the same explanation. Use a new revision-qualified directory or scope for
-each delivered revision. Never overwrite prior reports, evidence, methodology,
-or chart assets; record the previous revision and pin links to the intended
-revision. A presentation-only revision reuses accepted saved results without
-rerunning inference, detection, or risk assignment. Preserve memberships, risk
-tiers, baselines, exposure, and all statistics. After
-writing, run `editorial-review`, then `polish-cn` on all Chinese deliverables as
-specified in [chinese-polish.md](chinese-polish.md).
-Verify unchanged facts, synchronize Chinese Markdown and HTML, then grep
-banned strings in [terminology.md](terminology.md).
-
-English Markdown headings on the left; Chinese HTML / `.zh.md` on the right.
-
-```markdown
-# Potential transaction laundering (TLD) currency-amount pairs — <window>
-# 潜在交易清洗（TLD）币种金额对 — <window>
-
-## Period Conclusion                    # 本期结论（月报用“本月结论”）
-[Plain-language decision summary]       # [先查谁、为什么、目前不能判断什么]
-[Evidence and methodology links]        # [完整证据与方法链接]
-## Main Findings                        # 主要发现
-### <Exact descriptor and finding>      # <账单名称与发现>
-## Next Checks                          # 下一步核查
-```
-
-Use exactly these three top-level sections in the main report. Do not repeat
-a closing Verdict or require a table for every descriptor. Include only
-supported observations that explain a conclusion or a bounded next check.
-Retain uncertainty and negative evidence needed to interpret the finding.
-For each finding: observed evidence with period and denominator, then supported
-payment-pattern hypothesis and limitations, then its reconciled relationship
-to exact amount clusters. Do not lead with raw detector labels.
-
-The complete evidence document contains payment-pattern tables, the High-risk
-inventory, by-tier/by-descriptor and overall exposure, every cluster profile,
-anchor rates, country breakdowns, and secondary charts. Keep the detailed
-requirements below here, not in the main report. No evidence is deleted merely
-because it is not selected for the main narrative.
-
-The separate methodology document uses this structure:
-
-```markdown
-# Transaction laundering (TLD) currency-amount-pair detection methodology — <scope/window>
-# 交易清洗（TLD）币种金额对识别方法 — <范围/时段>
-
-## Universe and cleaning                # 分析总体与清洗
-## Descriptor partition logic           # Descriptor 分区逻辑
-## Payment-pattern inference            # 收款模式推断
-## Currency-amount-pair logic           # 币种金额对逻辑
-## Baseline and audit details           # 基线与审计明细
-```
-
-## Chapter requirements
-
-### Period conclusion / 本期结论
-
-Answer these three questions directly and in this order:
-
-1. What is the highest investigation-priority risk tier found?
-2. Which clusters are High, Medium, and Low risk; what characteristics support
-   each tier; and what share do they represent inside their descriptors?
-3. What is the exact-member exposure by risk tier, descriptor, and across all
-   authorization attempts?
-
-Risk tiers prioritize investigation and do not confirm TLD. Summarize the
-answers in ordinary language; keep the full inventories and exposure tables in
-evidence. Order findings `High → Medium → Low`, then by evidence strength and
-attempts inside each tier, retaining stable IDs. The opening summary has at
-most three short sentences; do not repeat it at the end.
-
-### Separate methodology / 独立方法文档
-
-Document the logic and values actually used for this run, not generic skill
-instructions. State the authorization grain and window; give the census from
-[clean.md](clean.md), BIN-country and currency mix, and portfolio
-constants.
-
-Partition logic defines the normalized descriptor key, normalization rules,
-the unknown bucket, descriptor counts, and shares. Currency-amount-pair
-logic explains within-descriptor ranking, adjacent currency-specific
-absolute residual caps, sensitivity evidence, the multi-anchor support floor,
-exact final membership, and the ≥100-attempt threshold. BIN country is
-post-formation attribution, not a partition or membership key.
-
-The ranking table contains partition, currency, exact amount, attempts,
-share of the descriptor, share of all authorization attempts, and selected
-cluster id or `baseline`. Chart attempt bars within the most relevant
-descriptors, sorted by count descending.
-
-Audit details contain drop counts, currency caps, sensitivity results,
-multi-anchor support floor, ≥100 cluster attempts, min-n for
-cluster-versus-baseline rate ranking (not the definition anchor table),
-descriptor-local high-count rule, baseline definitions, below-threshold
-families, risk-tier rules, all run-specific materiality thresholds, and files
-used. No skill name in the HTML footer; filenames only.
-
-### Payment patterns in complete evidence / 完整证据中的收款模式
-
-Use Direct top-up pattern / 直接充值模式, FX-converted top-up pattern /
-换汇充值模式, E-commerce pattern (Retail pattern) / 电商模式, or Other pattern /
-其他模式. Include this caption: names are hypotheses based on authorization
-data, not verified businesses or successful settlements. No raw menu/frozen/
-off-menu labels in reader-facing headings; retain them in methodology.
-
-One table per descriptor, with pattern, inferred top-up currency, actual charge
-currency, attempts, share of descriptor, top issuing countries, approval rate,
-suspected-fraud response share, and relevant repeat/retry behavior. Split display
-rows where charge currencies imply different direct/FX top-up subtypes, while
-preserving counts and all other/insufficient-sample attempts. Several inferred
-top-up currencies may be listed separately without inventing several cashiers.
-
-Show fixed amounts / 定额 as a supplementary characteristic, not an exclusive
-pattern. Record whether source-to-top-up rate consistency was tested; a raw
-`live` result is not proof of live conversion. Do not call fixed amounts generated
-traffic or infer tax, fees, or verified goods from a fitted retail constant.
-
-Explain the clearest observed differences and the unresolved business question
-in ordinary language. Patterns do not partition canonical detection; a cluster
-may span several patterns. Describe any pattern allocation only where supported
-by a recorded row-level reconciliation, never by guessing from matching labels.
-Methodology holds raw labels, parameters, cache, coverage, drift, alternatives,
-and excluded/missing inference coverage; those rows remain in canonical detection.
-When a finding includes a supported menu whose presets do not double at each
-step, explain the distinction in that finding: the menu describes a pricing
-hypothesis, while the three-level, fixed-2x, 100-attempt gate selects formal
-amount clusters. Showing two different lists or saying they overlap is not
-an explanation of why both can be valid. Preserve their separate counts and
-row-reconciled intersection; never force menu prices into cluster anchors.
-
-### High-risk inventory in complete evidence / 完整证据中的高风险组合
-
-Include High-risk clusters only. For each, show descriptor, representative
-anchors, attempts, share of its descriptor, share of all authorization attempts,
-risk basis, standout evidence, and the clearest independent corroborating
-characteristic. Do not describe the tier as confirmation that TLD occurred.
-
-### Exposure in complete evidence / 完整证据中的风险敞口
-
-Exposure is the disjoint exact-member attempt count of all identified clusters,
-separated into High, Medium, and Low risk. The by-tier-and-descriptor table
-reports descriptor attempts, cluster attempts, share of descriptor, share of
-all authorization attempts, risk tier, and cluster IDs. The overall view has
-one row per risk tier plus `Baseline` / `基线`, and the rows conserve all
-authorization attempts. Never combine the three tiers into a single “confirmed”
-or “supported TLD” impact figure.
-
-### Cluster analysis in complete evidence / 完整证据中的金额组合
-
-One subsection per identified currency-amount pair. Show the risk tier and
-concise basis immediately below the cluster heading, then use one consolidated
-profile/comparison table. Do not emit separate Definition, Characteristics,
-Specification, and Metrics tables because they repeat descriptor, attempts,
-and cluster context.
-
-The consolidated table has columns `Section`, `Item`, `Cluster / value`, and
-`Same-descriptor baseline`. It contains each fact exactly once:
-
-| Section | Items |
-|---|---|
-| Definition | Descriptor, seed, representative anchors |
-| Population | Descriptor attempts, cluster attempts, cluster share of descriptor |
-| Specification | MCC |
-| Metrics | Authorization success rate, suspected fraud as share of attempts, Top 3 issuing banks, Top 3 BIN countries |
-
-Use `—` in the baseline column for definition/specification values that have no
-comparison. The baseline population is the same descriptor after removing the
-cluster's exact keys. Express anchors as `USD 11.7 → USD 23.39* → USD 46.78`.
-
-Follow the consolidated table with one anchor table whose columns are anchor,
-attempts, authorization success rate, and suspected fraud as share of attempts.
-Always compute both rates for every theoretical level from that level's exact
-member attempts, including levels with fewer than the contrast min-n (default
-30). Do not write `n/a` on this table because the rung is small. For a level
-containing multiple nearby exact values, display its highest-count amount with
-`*`; the table totals attempts and rates across that level and explains `*`.
-
-For every Top 3 cluster BIN country, add a table with country, cluster attempts,
-share of cluster, total attempts in the same descriptor-country slice, and
-cluster coverage of that slice. In canonical original-currency results only,
-mark exact 100% single-country composition as a standout. Treat `Mixed` as
-neutral. Do not assign a threshold or risk points to country-slice coverage,
-and do not promote any composition below 100% merely because it is high. In FX
-results, caption even 100% concentration as descriptive because BIN country
-constructs target currency; it cannot raise the FX tier.
-
-Follow the tables with transaction-date and transaction-time charts comparing
-the cluster with the same-descriptor baseline. These are characteristics, not
-cluster membership conditions.
-
-When clustering finds no qualifying family, omit currency-amount-pair subsections,
-publish the ranking and below-threshold candidates in the methodology
-document, show zero identified-cluster exposure, and still write the three
-main-report sections. Candidate rankings are not risk hits; do not invent
-cluster charts or promote descriptor-level leads to formal tiers.
-
-For an FX sensitivity, keep the same consolidated layout, label anchors as
-base-currency anchors, and link separate `<scope>_bin_base_currency_clusters*`
-and methodology files. Define `|P_FX|` as the exact-date-converted attempts in
-the current frozen canonical descriptor partition. Use only `P_FX` for FX
-high-count/rung-pool thresholds and the same-descriptor baseline; keep excluded
-rows only in exclusion coverage and reconciliation to canonical `U`. State both
-converted-universe and all-authorization exposure denominators.
-
-### Main findings and next checks / 主要发现与下一步核查
-
-Follow [decision.md](decision.md). The opening summary is limited to three
-short sentences, not the whole report. Explain whom to review and why in Main
-findings, using only established evidence. Put specific checks against orders,
-products, sellers, websites, or fulfillment in Next checks and connect each to
-an observed finding. State when evidence cannot support a priority or useful
-check rather than inventing one.
-Explain observations in business language and retain uncertainty. In a zero-
-cluster run, state zero identified-cluster exposure without implying low risk;
-descriptor-level leads remain separate from formal cluster tiers. For a period
-focus, separate current activity from historical findings. The final chat
-handoff follows the same structure rather than leading with detector IDs.
-
-## Charts
-
-Preserve useful original charts in the concise report. Put the most important
-amount and cluster-versus-baseline rate charts inline beside the relevant
-finding. Use expandable sections for date/hour views, secondary clusters, and
-candidate rankings, with full evidence links. Keep all valid original charts
-available in the main report, not only linked elsewhere. Zero-cluster reports
-may reuse candidate rankings with an explicit not-risk-hits caption; never
-invent cluster charts. Canonical and FX charts stay in separate reports.
-
-For presentation revisions, inventory original chart IDs and compare payloads
-before and after: chart type, labels, series, values, axes, orientation,
-normalization, denominators, currency, and timezone semantics. Retain local
-chart dependencies. Initialize hidden charts only when visible and resize
-on expansion. Verify nonblank canvases, readable labels, expansion behavior,
-and no page horizontal overflow at desktop and mobile widths. For Markdown,
-retain chart captions and links to the corresponding HTML views. Correct an
-invalid source chart only as an explicit analytical correction in a separate
-revision with the reason recorded, never silently during presentation work.
-
-| View | Mark |
-|---|---|
-| Representative-anchor attempts | Horizontal bar |
-| Cluster vs same-descriptor baseline rates | Grouped bar |
-| Transaction date trend, cluster vs same-descriptor baseline | Two-series line, each series normalized to its own attempt total |
-| Transaction time by BIN-country local hour, cluster vs same-descriptor baseline | Two-series line, each series normalized to its own attempt total |
-
-Label axes with the full metric (`授权笔数`, `比率 %`, `每日授权占比 %`,
-`每小时授权占比 %`). State that transaction time uses the BIN country's local
-hour and UTC when the BIN country has no configured timezone.
-
-## What the report is not
-
-Not a payment overview (`payment-analysis`). Not a holdout rule pack
-(`fraud-analysis`). Not a recommendation to file a SAR or to terminate a
-merchant.
+# Reporting
+
+Lead with the key observations, investigation priorities, uncertainty and next
+checks. State the period and populations, keep currency views separate, and label
+all denominators. Read `{house-style baseDir}/references/analytical-reporting.md` and
+its required references. For each analytical topic, put its supported point near its
+chart and show the evidence and decision-relevant limitations. Do not impose a fixed
+finding quota. When evidence cannot support a point, name the concrete shortfall and
+what it prevents the reader from concluding or doing. Use the requested language and
+format; no fixed bilingual or file-count requirement applies.
+
+## Evidence and methodology
+
+Retain exact group IDs and amount keys, group/out-group profiles, daily and hourly
+counts, status coverage, and ID unions for combined totals. Record source files,
+cleaning census, aggregation rules, parameters, timezone, FX mapping/cache and
+exclusion reconciliation. Link these details from the main report when stored
+separately. The saved profile and offline charts use the same group/out-group
+interface for concentration, escalation and pricing; see the
+[metric contract](metric-contract.md) for schemas and formulas. Keep sensitive
+identifiers in the authorized evidence destination.
+
+## Revisions
+
+Save each delivered revision to a new path under `reports/tld-detection/` and
+link the intended evidence, methodology and prior revision. For presentation-only
+changes, reuse accepted membership, metrics, tiers and valid chart payloads;
+do not rerun analysis. Request missing artifacts and disclose unverified rendering.
+
+## Language and review
+
+A descriptor is a billing name, not a verified merchant identity. BIN country is
+the card-issuing country, not customer location. Suspected fraud is not confirmed
+fraud; an authorization decline is not a chargeback. Spell out transaction
+laundering (TLD) on first use.
+
+For Chinese, use 账单名称, 授权成功, 疑似欺诈 and 调查优先级; preserve exact
+identifiers and never translate an authorization decline as 拒付.
+For review, reuse the shared analytical rules already read in unchanged context; reload them
+after context loss. Give their resolved paths and this template to `write-report`, which
+coordinates editorial review and requires delegated reviewers to read them. Return remaining
+domain checks to this workflow without repeating a completed writing pass. For Chinese output,
+pass this template's Chinese-polish step to the writer when `polish-cn` is available; that is
+a domain selection, not the writer's generic optional default. Reuse its completed pass.
+Do not send raw transactions, card identifiers or
+credentials to polishing services. Protect amounts, dates, currencies, IDs,
+denominators, tiers and FX provenance; verify each remains attached to the correct
+population and period. Missing facts remain unknown. Disclose any review or
+polishing step that could not run rather than claiming it passed.

package/skills/risk-compliance/tld-detection/scripts/detect_ladders.py

@@ -9,6 +9,7 @@ import re
 from collections import Counter, defaultdict
 from dataclasses import dataclass
 from decimal import Decimal
+from fractions import Fraction
 from pathlib import Path
 from typing import Iterable, Mapping, Sequence
 
@@ -30,8 +31,38 @@ def _rank(keys: Iterable[Pair], counts: Mapping[Pair, int]) -> list[Pair]:
     return sorted(keys, key=lambda key: (-counts[key], key[1], key[0]))
 
 
-def _residual(lower: Pair, upper: Pair, multiplier: Decimal) -> Decimal:
-    return abs(upper[1] - lower[1] * multiplier)
+def _residual(lower: Pair, upper: Pair, multiplier: Decimal) -> Fraction:
+    return abs(Fraction(upper[1]) - Fraction(lower[1]) * Fraction(multiplier))
+
+
+def _decimal_exact(value: Fraction) -> Decimal:
+    # Residuals of finite decimals also have finite decimal representations.
+    denominator = value.denominator
+    twos = fives = 0
+    while denominator % 2 == 0:
+        denominator //= 2
+        twos += 1
+    while denominator % 5 == 0:
+        denominator //= 5
+        fives += 1
+    if denominator != 1:
+        raise ValueError("residual is not a finite decimal")
+    scale = max(twos, fives)
+    integer = value.numerator * 2 ** (scale - twos) * 5 ** (scale - fives)
+    return Decimal((int(integer < 0), tuple(map(int, str(abs(integer)))), -scale))
+
+
+def _adjacent(lower: Pair, upper: Pair, multiplier: Decimal, cap: Decimal) -> bool:
+    """Require distinct positive rungs and an unambiguous doubling fit.
+
+    The absolute cap controls price noise. A candidate must also be closer to
+    2x than to 1x or 3x, so a large cap cannot manufacture tiny-price ladders.
+    """
+    return (
+        Decimal(0) < lower[1] < upper[1]
+        and _residual(lower, upper, multiplier) <= cap
+        and _residual(lower, upper, multiplier) < Fraction(lower[1]) / 2
+    )
 
 
 def _expand_levels(
@@ -53,22 +84,22 @@ def _expand_levels(
                     candidate
                     for candidate in available - claimed
                     if candidate[0] == seed[0]
-                    and any(_residual(anchor, candidate, multiplier) <= cap for anchor in previous)
+                    and any(_adjacent(anchor, candidate, multiplier, cap) for anchor in previous)
                 }
             else:
                 candidates = {
                     candidate
                     for candidate in available - claimed
                     if candidate[0] == seed[0]
-                    and any(_residual(candidate, anchor, multiplier) <= cap for anchor in previous)
+                    and any(_adjacent(candidate, anchor, multiplier, cap) for anchor in previous)
                 }
             if not candidates:
                 break
 
-            def candidate_residual(candidate: Pair) -> Decimal:
+            def candidate_residual(candidate: Pair) -> Fraction:
                 if direction == 1:
-                    return min(_residual(anchor, candidate, multiplier) for anchor in previous)
-                return min(_residual(candidate, anchor, multiplier) for anchor in previous)
+                    return min(_residual(anchor, candidate, multiplier) for anchor in previous if anchor[1] < candidate[1])
+                return min(_residual(candidate, anchor, multiplier) for anchor in previous if candidate[1] < anchor[1])
 
             primary = min(candidates, key=lambda candidate: (-counts[candidate], candidate_residual(candidate), candidate[1]))
             next_level = {primary} | {
@@ -80,13 +111,13 @@ def _expand_levels(
     return levels
 
 
-def _fit_residual(levels: Mapping[int, set[Pair]], multiplier: Decimal) -> Decimal:
-    total = Decimal("0")
+def _fit_residual(levels: Mapping[int, set[Pair]], multiplier: Decimal) -> Fraction:
+    total = Fraction(0)
     for level, anchors in levels.items():
         if level > 0:
-            total += sum(min(_residual(lower, anchor, multiplier) for lower in levels[level - 1]) for anchor in anchors)
+            total += sum(min(_residual(lower, anchor, multiplier) for lower in levels[level - 1] if lower[1] < anchor[1]) for anchor in anchors)
         elif level < 0:
-            total += sum(min(_residual(anchor, upper, multiplier) for upper in levels[level + 1]) for anchor in anchors)
+            total += sum(min(_residual(anchor, upper, multiplier) for upper in levels[level + 1] if anchor[1] < upper[1]) for anchor in anchors)
     return total
 
 
@@ -120,7 +151,7 @@ def detect_ladders(
     for seed in _rank(rung_pool, normalized):
         if seed not in unused:
             continue
-        candidates: list[tuple[tuple[object, ...], dict[int, set[Pair]], Decimal, Decimal]] = []
+        candidates: list[tuple[tuple[object, ...], dict[int, set[Pair]], Decimal, Fraction]] = []
         for multiplier in (CLUSTER_MULTIPLIER,):
             cap = config.absolute_caps[seed[0]]
             levels = _expand_levels(
@@ -153,7 +184,7 @@ def detect_ladders(
             "pairs": family,
             "rule": "adjacent-absolute-residual",
             "absolute_cap": config.absolute_caps[seed[0]],
-            "fit_residual": fit_residual,
+            "fit_residual": _decimal_exact(fit_residual),
             "multi_anchor_levels": {level: anchors for level, anchors in levels.items() if len(anchors) > 1},
             "alternatives": {},
         }
@@ -249,6 +280,7 @@ def sensitivity_analysis(
                 multi_anchor_min_attempts=config.multi_anchor_min_attempts,
             ),
         )
+        ladders = [ladder for ladder in ladders if ladder["seed"][0] == currency]
         output.append(
             {
                 "currency": currency,
@@ -283,6 +315,7 @@ def partitioned_sensitivity_analysis(
                 multi_anchor_min_attempts=config.multi_anchor_min_attempts,
             ),
         )
+        ladders = [ladder for ladder in ladders if ladder["seed"][0] == currency]
         output.append(
             {
                 "currency": currency,