sieve-scope 1.3.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,168 @@
1
+ Metadata-Version: 2.4
2
+ Name: sieve-scope
3
+ Version: 1.3.0
4
+ Summary: Deterministic and Explainable Code Plagiarism Detection Engine with 7-bit Observation Space
5
+ Author: Kai IWASAKI
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/neguseatama/sieve-scope
8
+ Project-URL: Repository, https://github.com/neguseatama/sieve-scope.git
9
+ Project-URL: Bug Tracker, https://github.com/neguseatama/sieve-scope/issues
10
+ Classifier: Development Status :: 5 - Production/Stable
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: Intended Audience :: Education
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.9
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
20
+ Classifier: Topic :: Text Processing :: Linguistic
21
+ Requires-Python: >=3.9
22
+ Description-Content-Type: text/markdown
23
+ License-File: LICENSE
24
+ Dynamic: license-file
25
+
26
+ # Sieve Scope v1.3
27
+
28
+ **English** | [日本語](README.ja.md)
29
+
30
+ > **"See through the formatting, and look into the pure logic of the algorithm."**
31
+ > A fully local, zero-dependency, deterministic (100% reproducible) code plagiarism and structural similarity detection engine.
32
+ > Extends [Sieve-Core](https://github.com/neguseatama/sieve-core)'s deterministic similarity engine and [Sieve-Referee](https://github.com/neguseatama/sieve-referee)'s multi-hypothesis LUT judgment philosophy to the **source code domain**.
33
+
34
+ [![CI](https://github.com/neguseatama/sieve-scope/actions/workflows/test.yml/badge.svg)](https://github.com/neguseatama/sieve-scope/actions/workflows/test.yml)
35
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
36
+ [![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/)
37
+
38
+ ---
39
+
40
+ ## 💡 Concept & Philosophy
41
+
42
+ The starting point of this project is **Sieve-Core**, a core engine that applies the "sieve of set theory" design philosophy of **Sieve-AI**—inspired by the Sieve of Eratosthenes—to the more general domain of text and data processing.
43
+
44
+ Next, **Sieve Referee**, which adds a "sieve of combinatorics" to Sieve-Core, is a zero-dependency Python library that extracts reliable information from noise based on corroboration from multiple independent sources.
45
+
46
+ Then, **Sieve Scope** further adds a two-axis evaluation and redesigns the approach, making it an explainable code plagiarism detection engine that observes the **surface format** (variable names, comments, whitespace) and the **essential logic** (algorithm structure) separately.
47
+
48
+ Traditional string comparison and simple similarity measures struggle to distinguish between:
49
+ - Plagiarism by merely renaming variables (**variable laundering**)
50
+ - Coincidental matches due to shared templates (**boilerplate**)
51
+
52
+ `Sieve Scope` observes two axes independently:
53
+ - **AST (Abstract Syntax Tree)** — extracting the "logic skeleton" with identifiers removed
54
+ - **tokenize** — extracting "identifier vocabulary and order"
55
+
56
+ By inheriting the hypothesis mask (H1–H7) and LUT from Sieve-Referee, it fully explains why a judgment was made.
57
+
58
+ > The name "Sieve Scope" comes from the design philosophy of seeing through the surface of code (format) and comparing only the pure logic, like a microscope.
59
+
60
+ ---
61
+
62
+ ## 🔥 Key Features
63
+
64
+ - **7-bit Observation Space (H1–H7)**
65
+ Independently evaluates AST skeleton, identifier vocabulary, positional/order (g_POS), constants (g_CONST), and vocabulary context (g_FREQ).
66
+ - **v0_mask compatibility**
67
+ The legacy 4-bit v0_mask is preserved and regression-tested.
68
+ - **Strict Cluster Safety Invariant**
69
+ H4 is promoted to 1 only when the cluster size is at least 3 **and** there is direct similarity evidence (H2∨H3). Indirect "telephone game" chains are never mixed into direct judgments.
70
+ - **Short-Circuit Optimization**
71
+ Pairs with no structural or vocabulary similarity (H2=0 ∧ H3=0) are immediately returned as `1000-000`, skipping expensive hash computations.
72
+ - **Zero Dependencies & Deterministic**
73
+ Implemented using only the Python standard library. No external packages, no API costs, no probabilistic elements.
74
+
75
+ ---
76
+
77
+ ## 📐 7-bit Observation Space
78
+
79
+ The evaluation function `f_v1.3(C1, C2)` projects a code pair into seven boolean hypotheses `(H1, H2, H3, H4, H5, H6, H7)`.
80
+
81
+ | Hypothesis | Name | What it measures | Criterion / Threshold |
82
+ | :--- | :--- | :--- | :--- |
83
+ | **H1** | Parseability | Can both be parsed as Python syntax? | Success/failure of AST construction |
84
+ | **H2** | AST Skeleton | Do the AST node type sequences match exactly? | Exact match of skeleton strings |
85
+ | **H3** | Identifier Similarity | Jaccard + sequence similarity of identifiers | Score `> 0.50` |
86
+ | **H4** | Cluster Safety | Direct similarity + membership in a valid cluster (size ≥3) | Invariant condition satisfied |
87
+ | **H5** | g_POS | SHA-256 hash of recursive AST field representation | Exact hash match |
88
+ | **H6** | g_CONST | Constants / literals set match | Sorted constant list exact match |
89
+ | **H7** | g_FREQ | Weighted identifier bi-gram Dice coefficient | Score `>= 0.60` |
90
+
91
+ Mask notation is `v0_mask (4-bit) - extension (3-bit)`.
92
+ Example: `1100-011` → H1=1, H2=1, H3=0, H4=0 / H5=0, H6=1, H7=1
93
+
94
+ ---
95
+
96
+ ## 📊 Representative LUT Patterns
97
+
98
+ | v0_mask | Extended Example | Judgment Label | Description |
99
+ |:---:|:---:|:---|:---|
100
+ | `1110` | `1110-111` | `EXACT_CLONE` | Observation-space exact match |
101
+ | `1100` | `1100-110` | `VARIABLE_LAUNDERING` | Logic matches but surface differs |
102
+ | `1010` | `1010-011` | `COINCIDENTAL_FRAMEWORK_MATCH` | Logic differs but surface matches |
103
+ | `1000` | `1000-000` | `INDEPENDENT_LOGIC` | No direct similarity (short-circuit) |
104
+ | `1110` | `1110-101` | `CONSTANT_MODIFIED_CLONE` | Only constants differ |
105
+ | `1110` | `1110-011` | `PERMUTATED_CLONE` | Non-commutative order reversed |
106
+
107
+ ---
108
+
109
+ ## 💻 Quick Start
110
+
111
+ ```python
112
+ from sieve_scope import SieveScopeEngineV1_3, SieveClusterAnalyzerV1_3
113
+
114
+ engine = SieveScopeEngineV1_3(jaccard_threshold=0.50, freq_dice_threshold=0.60)
115
+
116
+ code_a = "def calc_sum(a, b):\n return a + b"
117
+ code_b = "def calc_sum(x, y):\n return x + y"
118
+
119
+ result = engine.evaluate_pair(code_a, code_b)
120
+
121
+ print(result["v0_mask"]) # e.g., 1100
122
+ print(result["mask"]) # e.g., 1100-110
123
+
124
+ ---
125
+
126
+ ### Configuration / Thresholds
127
+
128
+ - `jaccard_threshold` (default: `0.50`): Threshold for H3 Multiset Jaccard + sequence similarity.
129
+ - `freq_dice_threshold` (default: `0.60`): Threshold for H7 Weighted identifier bi-gram Dice coefficient.
130
+
131
+ > **Note on Calibration & Defaults (v1.3)**:
132
+ > Exhaustive grid search experiments on synthetic benchmark datasets demonstrate optimal class separation ($F1 = 1.0000$) across a wide threshold range of `0.35` to `0.75`. However, the default threshold parameters are strictly maintained at **`0.50` / `0.60`** in order to preserve backwards compatibility with v0, maintain regression test stability, and prevent potential false positives caused by shared domain vocabulary in production codebases.
133
+
134
+ ---
135
+
136
+ ## 🔬 Testing & Verification
137
+ Cross-Python Validation Summary
138
+ 24 regression cases (test_v1_3_full_suite.py): PASSED
139
+
140
+ 5 compatibility tests (test_v1_3_regression.py): PASSED
141
+
142
+ 6 edge-case & parallel determinism tests (test_v1_3_edge_cases.py): PASSED
143
+
144
+ 81 synthetic threshold configurations (threshold_tuning.py): PASSED
145
+
146
+ All test suites and synthetic experiments passed completely without errors across Python 3.9, 3.10, 3.11, and 3.12.
147
+
148
+ Threshold Calibration & Synthetic Validation
149
+ The threshold parameters (H3 Multiset Jaccard: 0.50, H7 Weighted Bi-gram Dice: 0.60) were designed and calibrated using a synthetic validation pair set (Positive 6 pairs / Negative 3 pairs).
150
+ The engine's 1110-111 indicates an exact match in the current 7-dimensional observation space (Observation Clones), and does not guarantee complete semantic equivalence (Semantic Clones).
151
+
152
+ ---
153
+
154
+ ## 📚 Documentation
155
+
156
+ - 🏛️ Architecture Specification
157
+
158
+ - 🔌 API & Integration Guide
159
+
160
+ - 🚀 Deployment Guide
161
+
162
+ - 🔐 CI/CD & Security Guide
163
+
164
+ ## 📄 License
165
+ MIT License. See LICENSE for details.
166
+
167
+ ## 👤 Author
168
+ Kai IWASAKI
@@ -0,0 +1,6 @@
1
+ sieve_scope.py,sha256=xts1AlZ9csOvbg8aCz8X-n26nBG2oFbjnSQi5Cg7Rfs,12871
2
+ sieve_scope-1.3.0.dist-info/licenses/LICENSE,sha256=gkCel7djhBCn21V3Nwq1EM5HN6y53y-xoNZ3fRsfz5Y,1067
3
+ sieve_scope-1.3.0.dist-info/METADATA,sha256=V2EP_sgXTxSD5eiO2pYtdc3T9ORZGIW1Sp4uXR4yBuo,8313
4
+ sieve_scope-1.3.0.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
5
+ sieve_scope-1.3.0.dist-info/top_level.txt,sha256=4PpNdcIQ5n6Bfc9wqsyuDfzitVcIGwpRh0yaBhBhl10,12
6
+ sieve_scope-1.3.0.dist-info/RECORD,,
@@ -0,0 +1,5 @@
1
+ Wheel-Version: 1.0
2
+ Generator: setuptools (82.0.1)
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
5
+
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Kai IWASAKI
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ sieve_scope
sieve_scope.py ADDED
@@ -0,0 +1,326 @@
1
+ """
2
+ Sieve Scope Engine v1.3 (Production Ready)
3
+ - 7-bit Observation Space (H1 - H7)
4
+ - Short-Circuit Evaluation
5
+ - Tokenize-based Pure Identifier Extraction
6
+ - Weighted Bi-gram Dice Coefficient
7
+ - Parallel Pair Evaluation
8
+ - Indirect Chain Detection
9
+ """
10
+
11
+ import ast
12
+ import difflib
13
+ import hashlib
14
+ import io
15
+ import keyword
16
+ import tokenize
17
+ from collections import Counter, defaultdict, deque
18
+ from concurrent.futures import ProcessPoolExecutor, as_completed
19
+ from typing import Any, Dict, List, Set, Tuple, Optional
20
+
21
+ # v0完全互換の除外識別子集合
22
+ IGNORED_WORDS_V0: Set[str] = {
23
+ 'print', 'len', 'range', 'sum', 'int', 'str', 'self', '__init__'
24
+ }
25
+
26
+
27
+ def extract_pure_identifiers(code: str, ignored_words: Set[str] = IGNORED_WORDS_V0) -> List[str]:
28
+ """
29
+ v0完全互換: tokenizeベースで非キーワードNAMEトークンを順序通り抽出する。
30
+ 関数名・クラス名・変数名・属性名をすべて識別子として捕捉する。
31
+ """
32
+ identifiers = []
33
+ try:
34
+ tokens = tokenize.generate_tokens(io.StringIO(code).readline)
35
+ for toknum, tokval, _, _, _ in tokens:
36
+ if toknum == tokenize.NAME and not keyword.iskeyword(tokval):
37
+ if tokval not in ignored_words:
38
+ identifiers.append(tokval)
39
+ except Exception:
40
+ pass
41
+ return identifiers
42
+
43
+
44
+ def build_pos_tuple(node: Any) -> Any:
45
+ """
46
+ v1.2完全互換: ASTフィールド構造タプルを生成する。
47
+ value / id / attr フィールドのうち非AST値(定数・識別子名)は除外するが、
48
+ arg などのその他スカラーフィールドは保持する。
49
+ """
50
+ if not isinstance(node, ast.AST):
51
+ if isinstance(node, list):
52
+ return tuple(build_pos_tuple(x) for x in node)
53
+ return str(node)
54
+
55
+ field_tuples = []
56
+ for field, value in ast.iter_fields(node):
57
+ if field in ("value", "id", "attr") and not isinstance(value, ast.AST):
58
+ continue
59
+ field_tuples.append((field, build_pos_tuple(value)))
60
+ return (type(node).__name__, tuple(field_tuples))
61
+
62
+
63
+ class ASTSkeletonVisitor(ast.NodeVisitor):
64
+ """
65
+ H2: AST構造骨格の抽出。
66
+ generic_visit はすべてのノード型名を記録する(Load, Store, Del, 演算子ノードを含む)。
67
+ """
68
+ def __init__(self):
69
+ self.skeleton = []
70
+
71
+ def generic_visit(self, node):
72
+ self.skeleton.append(type(node).__name__)
73
+ super().generic_visit(node)
74
+
75
+
76
+ class ASTConstantVisitor(ast.NodeVisitor):
77
+ """H6: g_CONST 定数・リテラルの抽出"""
78
+ def __init__(self):
79
+ self.constants = []
80
+
81
+ def visit_Constant(self, node):
82
+ self.constants.append(str(node.value))
83
+ self.generic_visit(node)
84
+
85
+
86
+ class SieveScopeEngineV1_3:
87
+ """7次元観測空間評価エンジン"""
88
+
89
+ def __init__(self, jaccard_threshold: float = 0.50, freq_dice_threshold: float = 0.60):
90
+ self.jaccard_threshold = jaccard_threshold
91
+ self.freq_dice_threshold = freq_dice_threshold
92
+
93
+ def evaluate_pair(self, code1: str, code2: str) -> Dict[str, Any]:
94
+ # H1: Parseability
95
+ try:
96
+ tree1 = ast.parse(code1)
97
+ tree2 = ast.parse(code2)
98
+ h1 = 1
99
+ except Exception:
100
+ return {
101
+ "h_states": {"H1": 0, "H2": 0, "H3": 0, "H4": 0, "H5": 0, "H6": 0, "H7": 0},
102
+ "v0_mask": "0000",
103
+ "mask": "0000-000",
104
+ "short_circuited": True,
105
+ }
106
+
107
+ # H2: AST Skeleton(全ノード型の列を比較)
108
+ vis1, vis2 = ASTSkeletonVisitor(), ASTSkeletonVisitor()
109
+ vis1.visit(tree1)
110
+ vis2.visit(tree2)
111
+ skeleton1 = "_".join(vis1.skeleton)
112
+ skeleton2 = "_".join(vis2.skeleton)
113
+ h2 = 1 if skeleton1 == skeleton2 else 0
114
+
115
+ # H3: 純粋識別子の類似度(Multiset Jaccard + SequenceMatcher)
116
+ ids1 = extract_pure_identifiers(code1)
117
+ ids2 = extract_pure_identifiers(code2)
118
+
119
+ if not ids1 and not ids2:
120
+ h3_score = 1.0
121
+ elif not ids1 or not ids2:
122
+ h3_score = 0.0
123
+ else:
124
+ c1, c2 = Counter(ids1), Counter(ids2)
125
+ intersection = sum((c1 & c2).values())
126
+ union = sum((c1 | c2).values())
127
+ multiset_jaccard = intersection / union if union > 0 else 0.0
128
+
129
+ seq_ratio = difflib.SequenceMatcher(None, ids1, ids2).ratio()
130
+ h3_score = 0.6 * multiset_jaccard + 0.4 * seq_ratio
131
+
132
+ # v0互換: 厳密な '>' 判定
133
+ h3 = 1 if h3_score > self.jaccard_threshold else 0
134
+
135
+ # ショートサーキット: H2=0 かつ H3=0 の場合のみ高コスト計算をスキップ
136
+ if h2 == 0 and h3 == 0:
137
+ return {
138
+ "h_states": {"H1": 1, "H2": 0, "H3": 0, "H4": 0, "H5": 0, "H6": 0, "H7": 0},
139
+ "v0_mask": "1000",
140
+ "mask": "1000-000",
141
+ "short_circuited": True,
142
+ "scores": {"h3_score": h3_score},
143
+ }
144
+
145
+ # H5: g_POS(build_pos_tuple による決定論的ハッシュ)
146
+ pos_tuple1 = build_pos_tuple(tree1)
147
+ pos_tuple2 = build_pos_tuple(tree2)
148
+ pos_hash1 = hashlib.sha256(str(pos_tuple1).encode('utf-8')).hexdigest()
149
+ pos_hash2 = hashlib.sha256(str(pos_tuple2).encode('utf-8')).hexdigest()
150
+ h5 = 1 if pos_hash1 == pos_hash2 else 0
151
+
152
+ # H6: g_CONST(定数集合の一致)
153
+ c_vis1, c_vis2 = ASTConstantVisitor(), ASTConstantVisitor()
154
+ c_vis1.visit(tree1)
155
+ c_vis2.visit(tree2)
156
+ h6 = 1 if sorted(c_vis1.constants) == sorted(c_vis2.constants) else 0
157
+
158
+ # H7: g_FREQ(重み付き Bi-gram Dice 係数)
159
+ bigrams1 = self._get_weighted_bigrams(ids1)
160
+ bigrams2 = self._get_weighted_bigrams(ids2)
161
+
162
+ common_keys = set(bigrams1.keys()) & set(bigrams2.keys())
163
+ common_weight = sum(min(bigrams1[bg], bigrams2[bg]) for bg in common_keys)
164
+ total_weight = sum(bigrams1.values()) + sum(bigrams2.values())
165
+
166
+ dice_score = (2.0 * common_weight) / total_weight if total_weight > 0 else 0.0
167
+ h7 = 1 if dice_score >= self.freq_dice_threshold else 0
168
+
169
+ # H4: ペア評価では常に0(クラスタ分析時のみ昇格)
170
+ h4 = 0
171
+
172
+ v0_mask = f"{h1}{h2}{h3}{h4}"
173
+ ext_mask = f"{h5}{h6}{h7}"
174
+
175
+ return {
176
+ "h_states": {"H1": h1, "H2": h2, "H3": h3, "H4": h4, "H5": h5, "H6": h6, "H7": h7},
177
+ "v0_mask": v0_mask,
178
+ "mask": f"{v0_mask}-{ext_mask}",
179
+ "short_circuited": False,
180
+ "scores": {
181
+ "h3_score": h3_score,
182
+ "h7_dice_score": dice_score,
183
+ },
184
+ }
185
+
186
+ @staticmethod
187
+ def _get_weighted_bigrams(identifiers: List[str]) -> Dict[str, float]:
188
+ """識別子列から Bi-gram と出現頻度(重み)を生成"""
189
+ bigrams = defaultdict(float)
190
+ for i in range(len(identifiers) - 1):
191
+ bg = f"{identifiers[i]}___{identifiers[i+1]}"
192
+ bigrams[bg] += 1.0
193
+ return bigrams
194
+
195
+
196
+ def _pair_worker_func(args):
197
+ """ProcessPoolExecutor用のトップレベル並列ワーカー関数"""
198
+ key1, code1, key2, code2, jaccard_th, freq_th = args
199
+ engine = SieveScopeEngineV1_3(jaccard_threshold=jaccard_th, freq_dice_threshold=freq_th)
200
+ result = engine.evaluate_pair(code1, code2)
201
+ return (key1, key2), result
202
+
203
+
204
+ class SieveClusterAnalyzerV1_3:
205
+ """クラスタ解析・安全不変式・並列処理・間接伝搬警告エンジン"""
206
+
207
+ def __init__(self, engine: SieveScopeEngineV1_3, min_cluster_size: int = 3):
208
+ self.engine = engine
209
+ self.min_cluster_size = min_cluster_size
210
+
211
+ def analyze_corpus(self, corpus: Dict[str, str], max_workers: Optional[int] = None) -> Dict[str, Any]:
212
+ keys = sorted(list(corpus.keys()))
213
+ pair_tasks = []
214
+
215
+ for i in range(len(keys)):
216
+ for j in range(i + 1, len(keys)):
217
+ k1, k2 = keys[i], keys[j]
218
+ pair_tasks.append((
219
+ k1, corpus[k1], k2, corpus[k2],
220
+ self.engine.jaccard_threshold,
221
+ self.engine.freq_dice_threshold,
222
+ ))
223
+
224
+ raw_pair_results = {}
225
+
226
+ if len(pair_tasks) > 10 and max_workers != 1:
227
+ with ProcessPoolExecutor(max_workers=max_workers) as executor:
228
+ futures = [executor.submit(_pair_worker_func, task) for task in pair_tasks]
229
+ for future in as_completed(futures):
230
+ pair_key, res = future.result()
231
+ raw_pair_results[pair_key] = res
232
+ else:
233
+ for task in pair_tasks:
234
+ pair_key, res = _pair_worker_func(task)
235
+ raw_pair_results[pair_key] = res
236
+
237
+ # 決定論的順序の保証
238
+ pair_results = dict(sorted(raw_pair_results.items()))
239
+
240
+ # 直接類似エッジの構築(H2=1 または H3=1)
241
+ direct_adj = defaultdict(set)
242
+ for (k1, k2), res in pair_results.items():
243
+ if res["h_states"]["H2"] == 1 or res["h_states"]["H3"] == 1:
244
+ direct_adj[k1].add(k2)
245
+ direct_adj[k2].add(k1)
246
+
247
+ # 連結成分抽出
248
+ visited = set()
249
+ valid_clusters = []
250
+ all_clusters = []
251
+
252
+ for node in keys:
253
+ if node not in visited:
254
+ component = set()
255
+ queue = deque([node])
256
+ visited.add(node)
257
+ while queue:
258
+ curr = queue.popleft()
259
+ component.add(curr)
260
+ for neighbor in sorted(list(direct_adj[curr])):
261
+ if neighbor not in visited:
262
+ visited.add(neighbor)
263
+ queue.append(neighbor)
264
+
265
+ all_clusters.append(component)
266
+ if len(component) >= self.min_cluster_size:
267
+ valid_clusters.append(component)
268
+
269
+ # H4 安全不変式の適用(有効クラスタ内・直接類似のあるペアのみ昇格)
270
+ valid_nodes = set().union(*valid_clusters) if valid_clusters else set()
271
+
272
+ for (k1, k2), res in pair_results.items():
273
+ is_direct = (res["h_states"]["H2"] == 1 or res["h_states"]["H3"] == 1)
274
+ in_valid_cluster = (k1 in valid_nodes and k2 in valid_nodes and
275
+ any(k1 in cl and k2 in cl for cl in valid_clusters))
276
+
277
+ if in_valid_cluster and is_direct:
278
+ res["h_states"]["H4"] = 1
279
+ h = res["h_states"]
280
+ res["v0_mask"] = f"{h['H1']}{h['H2']}{h['H3']}1"
281
+ res["mask"] = f"{res['v0_mask']}-{h['H5']}{h['H6']}{h['H7']}"
282
+
283
+ # 間接伝搬チェーン警告の検出
284
+ warnings = self.detect_indirect_chains(keys, direct_adj, valid_clusters)
285
+
286
+ return {
287
+ "valid_clusters": [sorted(list(c)) for c in valid_clusters],
288
+ "all_clusters": [sorted(list(c)) for c in all_clusters],
289
+ "pair_results": pair_results,
290
+ "warnings": warnings,
291
+ }
292
+
293
+ @staticmethod
294
+ def detect_indirect_chains(nodes, direct_adj, valid_clusters):
295
+ warnings = []
296
+ for cluster in valid_clusters:
297
+ cluster_nodes = sorted(list(cluster))
298
+ for i in range(len(cluster_nodes)):
299
+ for j in range(i + 1, len(cluster_nodes)):
300
+ u, v = cluster_nodes[i], cluster_nodes[j]
301
+ if v not in direct_adj[u]:
302
+ queue = deque([[u]])
303
+ visited = {u}
304
+ found_path = None
305
+ while queue:
306
+ path = queue.popleft()
307
+ curr = path[-1]
308
+ if curr == v:
309
+ found_path = path
310
+ break
311
+ for neighbor in sorted(list(direct_adj[curr])):
312
+ if neighbor not in visited and neighbor in cluster:
313
+ visited.add(neighbor)
314
+ queue.append(path + [neighbor])
315
+ if found_path and len(found_path) > 2:
316
+ warnings.append({
317
+ "code": "INDIRECT_PLAGIARISM_CHAIN_WARNING",
318
+ "pair": (u, v),
319
+ "message": (
320
+ f"Direct similarity not confirmed between '{u}' and '{v}', "
321
+ f"but indirect chain detected via {len(found_path)-1} hops."
322
+ ),
323
+ "propagation_path": " -> ".join(found_path),
324
+ "hops": len(found_path) - 1,
325
+ })
326
+ return warnings