codeclone 1.1.0__tar.gz → 1.2.1__tar.gz

{codeclone-1.1.0 → codeclone-1.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codeclone
-Version: 1.1.0
+Version: 1.2.1
 Summary: AST and CFG-based code clone detector for Python focused on architectural duplication
 Author-email: Den Rozhnovskiy <pytelemonbot@mail.ru>
 Maintainer-email: Den Rozhnovskiy <pytelemonbot@mail.ru>
@@ -10,11 +10,10 @@ Project-URL: Repository, https://github.com/orenlab/codeclone
 Project-URL: Issues, https://github.com/orenlab/codeclone/issues
 Project-URL: Changelog, https://github.com/orenlab/codeclone/releases
 Project-URL: Documentation, https://github.com/orenlab/codeclone/tree/main/docs
-Keywords: python,ast,code-clone,duplication,static-analysis,ci,architecture
+Keywords: python,ast,cfg,code-clone,duplication,static-analysis,architecture,control-flow,ci
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Topic :: Software Development :: Quality Assurance
-Classifier: Topic :: Software Development :: Code Generators
 Classifier: Topic :: Software Development :: Testing
 Classifier: Typing :: Typed
 Classifier: License :: OSI Approved :: MIT License
@@ -23,26 +22,31 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Operating System :: OS Independent
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: pygments>=2.19.2
+Requires-Dist: rich>=14.3.2
 Provides-Extra: dev
 Requires-Dist: pytest>=9.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=6.1.0; extra == "dev"
 Requires-Dist: build>=1.2.0; extra == "dev"
 Requires-Dist: twine>=5.0.0; extra == "dev"
 Requires-Dist: mypy>=1.19.1; extra == "dev"
+Requires-Dist: ruff>=0.12.0; extra == "dev"
 Dynamic: license-file
 
 # CodeClone
 
 [![PyPI](https://img.shields.io/pypi/v/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![Downloads](https://img.shields.io/pypi/dm/codeclone.svg)](https://pypi.org/project/codeclone/)
+[![tests](https://github.com/orenlab/codeclone/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/orenlab/codeclone/actions/workflows/tests.yml)
 [![Python](https://img.shields.io/pypi/pyversions/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![License](https://img.shields.io/pypi/l/codeclone.svg)](LICENSE)
 
-**CodeClone** is a Python code clone detector based on **normalized AST and control-flow graphs (CFG)**.
+**CodeClone** is a Python code clone detector based on **normalized Python AST and Control Flow Graphs (CFG)**.
 It helps teams discover architectural duplication and prevent new copy-paste from entering the codebase via CI.
 
 CodeClone is designed to help teams:
@@ -51,15 +55,16 @@ CodeClone is designed to help teams:
 - identify architectural hotspots,
 - prevent *new* duplication via CI and pre-commit hooks.
 
-Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against renaming,
-formatting, and minor refactoring.
+Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against
+renaming, formatting, and minor refactoring.
 
 ---
 
 ## Why CodeClone?
 
 Most existing tools detect *textual* duplication.
-CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or architectural drift.
+CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or
+architectural drift.
 
 Typical use cases:
 
@@ -77,11 +82,11 @@ Typical use cases:
 - Detects functions and methods with identical **control-flow structure**.
 - Based on **Control Flow Graph (CFG)** fingerprinting.
 - Robust to:
-  - variable renaming,
-  - constant changes,
-  - attribute renaming,
-  - formatting differences,
-  - docstrings and type annotations.
+    - variable renaming,
+    - constant changes,
+    - attribute renaming,
+    - formatting differences,
+    - docstrings and type annotations.
 - Ideal for spotting architectural duplication across layers.
 
 ### Block-level clone detection (Type-3-lite)
@@ -89,24 +94,29 @@ Typical use cases:
 - Detects repeated **statement blocks** inside larger functions.
 - Uses sliding windows over CFG-normalized statement sequences.
 - Targets:
-  - validation blocks,
-  - guard clauses,
-  - repeated orchestration logic.
+    - validation blocks,
+    - guard clauses,
+    - repeated orchestration logic.
 - Carefully filtered to reduce noise:
-  - no overlapping windows,
-  - no clones inside the same function,
-  - no `__init__` noise,
-  - size and statement-count thresholds.
+    - no overlapping windows,
+    - no clones inside the same function,
+    - no `__init__` noise,
+    - size and statement-count thresholds.
 
 ### Control-Flow Awareness (CFG v1)
 
 - Each function is converted into a **Control Flow Graph**.
 - CFG nodes contain normalized AST statements.
-- CFG edges represent structural control flow (`if`, `for`, `while`).
+- CFG edges represent structural control flow:
+    - `if` / `else`
+    - `for` / `async for` / `while`
+    - `try` / `except` / `finally`
+    - `with` / `async with`
+    - `match` / `case` (Python 3.10+)
 - Current CFG semantics (v1):
-  - `break` and `continue` are treated as statements (no jump targets),
-  - after-blocks are explicit and always present,
-  - focus is on **structural similarity**, not precise runtime semantics.
+    - `break` and `continue` are treated as statements (no jump targets),
+    - after-blocks are explicit and always present,
+    - focus is on **structural similarity**, not precise runtime semantics.
 
 This design keeps clone detection **stable, deterministic, and low-noise**.
 
@@ -115,6 +125,7 @@ This design keeps clone detection **stable, deterministic, and low-noise**.
 - AST + CFG normalization instead of token matching.
 - Conservative defaults tuned for real-world Python projects.
 - Explicit thresholds for size and statement count.
+- No probabilistic scoring or heuristic similarity thresholds.
 - Focus on *architectural duplication*, not micro-similarities.
 
 ### CI-friendly baseline mode
@@ -154,14 +165,14 @@ Generate reports:
 
 ```bash
 codeclone . \
-  --json-out .cache/codeclone/report.json \
-  --text-out .cache/codeclone/report.txt
+  --json .cache/codeclone/report.json \
+  --text .cache/codeclone/report.txt
 ```
 
 Generate an HTML report:
 
 ```bash
-codeclone . --html-out .cache/codeclone/report.html
+codeclone . --html .cache/codeclone/report.html
 ```
 
 ---
@@ -181,14 +192,26 @@ Commit the generated baseline file to the repository.
 ### 2. Use in CI
 
 ```bash
-codeclone . --fail-on-new
+codeclone . --fail-on-new --no-progress
 ```
 
 Behavior:
 
-- ✅ existing clones are allowed,
-- ❌ build fails if *new* clones appear,
-- ✅ refactoring that removes duplication is always allowed.
+- existing clones are allowed,
+- the build fails if *new* clones appear,
+- refactoring that removes duplication is always allowed.
+
+`--fail-on-new` exits with a non-zero code when new clones are detected.
+
+### Python Version Consistency for Baseline Checks
+
+Due to inherent differences in Python’s AST between interpreter versions, baseline
+generation and verification must be performed using the same Python version.
+
+This ensures deterministic and reproducible clone detection results.
+
+CI checks therefore pin baseline verification to a single Python version, while the
+test matrix continues to validate compatibility across Python 3.10–3.14.
 
 ---
 
@@ -196,14 +219,14 @@ Behavior:
 
 ```yaml
 repos:
--   repo: local
+  - repo: local
     hooks:
-    -   id: codeclone
+      - id: codeclone
         name: CodeClone
         entry: codeclone
         language: python
-        args: [".", "--fail-on-new"]
-        types: [python]
+        args: [ ".", "--fail-on-new" ]
+        types: [ python ]
 ```
 
 ---
@@ -235,6 +258,10 @@ repos:
 5. Detect function-level and block-level clones.
 6. Apply conservative filters to suppress noise.
 
+See the architectural overview:
+
+- [docs/architecture.md](docs/architecture.md)
+
 ---
 
 ## Control Flow Graph (CFG)
@@ -245,6 +272,7 @@ to improve structural clone detection robustness.
 The CFG is a **structural abstraction**, not a runtime execution model.
 
 See full design and semantics:
+
 - [docs/cfg.md](docs/cfg.md)
 
 ---

{codeclone-1.1.0 → codeclone-1.2.1}/README.md

@@ -2,10 +2,11 @@
 
 [![PyPI](https://img.shields.io/pypi/v/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![Downloads](https://img.shields.io/pypi/dm/codeclone.svg)](https://pypi.org/project/codeclone/)
+[![tests](https://github.com/orenlab/codeclone/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/orenlab/codeclone/actions/workflows/tests.yml)
 [![Python](https://img.shields.io/pypi/pyversions/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![License](https://img.shields.io/pypi/l/codeclone.svg)](LICENSE)
 
-**CodeClone** is a Python code clone detector based on **normalized AST and control-flow graphs (CFG)**.
+**CodeClone** is a Python code clone detector based on **normalized Python AST and Control Flow Graphs (CFG)**.
 It helps teams discover architectural duplication and prevent new copy-paste from entering the codebase via CI.
 
 CodeClone is designed to help teams:
@@ -14,15 +15,16 @@ CodeClone is designed to help teams:
 - identify architectural hotspots,
 - prevent *new* duplication via CI and pre-commit hooks.
 
-Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against renaming,
-formatting, and minor refactoring.
+Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against
+renaming, formatting, and minor refactoring.
 
 ---
 
 ## Why CodeClone?
 
 Most existing tools detect *textual* duplication.
-CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or architectural drift.
+CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or
+architectural drift.
 
 Typical use cases:
 
@@ -40,11 +42,11 @@ Typical use cases:
 - Detects functions and methods with identical **control-flow structure**.
 - Based on **Control Flow Graph (CFG)** fingerprinting.
 - Robust to:
-  - variable renaming,
-  - constant changes,
-  - attribute renaming,
-  - formatting differences,
-  - docstrings and type annotations.
+    - variable renaming,
+    - constant changes,
+    - attribute renaming,
+    - formatting differences,
+    - docstrings and type annotations.
 - Ideal for spotting architectural duplication across layers.
 
 ### Block-level clone detection (Type-3-lite)
@@ -52,24 +54,29 @@ Typical use cases:
 - Detects repeated **statement blocks** inside larger functions.
 - Uses sliding windows over CFG-normalized statement sequences.
 - Targets:
-  - validation blocks,
-  - guard clauses,
-  - repeated orchestration logic.
+    - validation blocks,
+    - guard clauses,
+    - repeated orchestration logic.
 - Carefully filtered to reduce noise:
-  - no overlapping windows,
-  - no clones inside the same function,
-  - no `__init__` noise,
-  - size and statement-count thresholds.
+    - no overlapping windows,
+    - no clones inside the same function,
+    - no `__init__` noise,
+    - size and statement-count thresholds.
 
 ### Control-Flow Awareness (CFG v1)
 
 - Each function is converted into a **Control Flow Graph**.
 - CFG nodes contain normalized AST statements.
-- CFG edges represent structural control flow (`if`, `for`, `while`).
+- CFG edges represent structural control flow:
+    - `if` / `else`
+    - `for` / `async for` / `while`
+    - `try` / `except` / `finally`
+    - `with` / `async with`
+    - `match` / `case` (Python 3.10+)
 - Current CFG semantics (v1):
-  - `break` and `continue` are treated as statements (no jump targets),
-  - after-blocks are explicit and always present,
-  - focus is on **structural similarity**, not precise runtime semantics.
+    - `break` and `continue` are treated as statements (no jump targets),
+    - after-blocks are explicit and always present,
+    - focus is on **structural similarity**, not precise runtime semantics.
 
 This design keeps clone detection **stable, deterministic, and low-noise**.
 
@@ -78,6 +85,7 @@ This design keeps clone detection **stable, deterministic, and low-noise**.
 - AST + CFG normalization instead of token matching.
 - Conservative defaults tuned for real-world Python projects.
 - Explicit thresholds for size and statement count.
+- No probabilistic scoring or heuristic similarity thresholds.
 - Focus on *architectural duplication*, not micro-similarities.
 
 ### CI-friendly baseline mode
@@ -117,14 +125,14 @@ Generate reports:
 
 ```bash
 codeclone . \
-  --json-out .cache/codeclone/report.json \
-  --text-out .cache/codeclone/report.txt
+  --json .cache/codeclone/report.json \
+  --text .cache/codeclone/report.txt
 ```
 
 Generate an HTML report:
 
 ```bash
-codeclone . --html-out .cache/codeclone/report.html
+codeclone . --html .cache/codeclone/report.html
 ```
 
 ---
@@ -144,14 +152,26 @@ Commit the generated baseline file to the repository.
 ### 2. Use in CI
 
 ```bash
-codeclone . --fail-on-new
+codeclone . --fail-on-new --no-progress
 ```
 
 Behavior:
 
-- ✅ existing clones are allowed,
-- ❌ build fails if *new* clones appear,
-- ✅ refactoring that removes duplication is always allowed.
+- existing clones are allowed,
+- the build fails if *new* clones appear,
+- refactoring that removes duplication is always allowed.
+
+`--fail-on-new` exits with a non-zero code when new clones are detected.
+
+### Python Version Consistency for Baseline Checks
+
+Due to inherent differences in Python’s AST between interpreter versions, baseline
+generation and verification must be performed using the same Python version.
+
+This ensures deterministic and reproducible clone detection results.
+
+CI checks therefore pin baseline verification to a single Python version, while the
+test matrix continues to validate compatibility across Python 3.10–3.14.
 
 ---
 
@@ -159,14 +179,14 @@ Behavior:
 
 ```yaml
 repos:
--   repo: local
+  - repo: local
     hooks:
-    -   id: codeclone
+      - id: codeclone
         name: CodeClone
         entry: codeclone
         language: python
-        args: [".", "--fail-on-new"]
-        types: [python]
+        args: [ ".", "--fail-on-new" ]
+        types: [ python ]
 ```
 
 ---
@@ -198,6 +218,10 @@ repos:
 5. Detect function-level and block-level clones.
 6. Apply conservative filters to suppress noise.
 
+See the architectural overview:
+
+- [docs/architecture.md](docs/architecture.md)
+
 ---
 
 ## Control Flow Graph (CFG)
@@ -208,6 +232,7 @@ to improve structural clone detection robustness.
 The CFG is a **structural abstraction**, not a runtime execution model.
 
 See full design and semantics:
+
 - [docs/cfg.md](docs/cfg.md)
 
 ---

{codeclone-1.1.0 → codeclone-1.2.1}/codeclone/__init__.py

@@ -6,7 +6,7 @@ Copyright (c) 2026 Den Rozhnovskiy
 Licensed under the MIT License.
 """
 
-from importlib.metadata import version, PackageNotFoundError
+from importlib.metadata import PackageNotFoundError, version
 
 try:
     __version__ = version("codeclone")

codeclone-1.2.1/codeclone/baseline.py

@@ -0,0 +1,84 @@
+"""
+CodeClone — AST and CFG-based code clone detector for Python
+focused on architectural duplication.
+
+Copyright (c) 2026 Den Rozhnovskiy
+Licensed under the MIT License.
+"""
+
+from __future__ import annotations
+
+import json
+from collections.abc import Mapping
+from pathlib import Path
+from typing import Any
+
+
+class Baseline:
+    __slots__ = ("blocks", "functions", "path", "python_version")
+
+    def __init__(self, path: str | Path):
+        self.path = Path(path)
+        self.functions: set[str] = set()
+        self.blocks: set[str] = set()
+        self.python_version: str | None = None
+
+    def load(self) -> None:
+        if not self.path.exists():
+            return
+
+        try:
+            data = json.loads(self.path.read_text("utf-8"))
+            self.functions = set(data.get("functions", []))
+            self.blocks = set(data.get("blocks", []))
+            python_version = data.get("python_version")
+            self.python_version = (
+                python_version if isinstance(python_version, str) else None
+            )
+        except json.JSONDecodeError as e:
+            raise ValueError(f"Corrupted baseline file at {self.path}: {e}") from e
+
+    def save(self) -> None:
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        self.path.write_text(
+            json.dumps(
+                _baseline_payload(self.functions, self.blocks, self.python_version),
+                indent=2,
+                ensure_ascii=False,
+            ),
+            "utf-8",
+        )
+
+    @staticmethod
+    def from_groups(
+        func_groups: Mapping[str, object],
+        block_groups: Mapping[str, object],
+        path: str | Path = "",
+        python_version: str | None = None,
+    ) -> Baseline:
+        bl = Baseline(path)
+        bl.functions = set(func_groups.keys())
+        bl.blocks = set(block_groups.keys())
+        bl.python_version = python_version
+        return bl
+
+    def diff(
+        self, func_groups: Mapping[str, object], block_groups: Mapping[str, object]
+    ) -> tuple[set[str], set[str]]:
+        new_funcs = set(func_groups.keys()) - self.functions
+        new_blocks = set(block_groups.keys()) - self.blocks
+        return new_funcs, new_blocks
+
+
+def _baseline_payload(
+    functions: set[str],
+    blocks: set[str],
+    python_version: str | None,
+) -> dict[str, Any]:
+    payload: dict[str, Any] = {
+        "functions": sorted(functions),
+        "blocks": sorted(blocks),
+    }
+    if python_version:
+        payload["python_version"] = python_version
+    return payload

{codeclone-1.1.0 → codeclone-1.2.1}/codeclone/blockhash.py

@@ -11,7 +11,7 @@ from __future__ import annotations
 import ast
 import hashlib
 
-from .normalize import NormalizationConfig, AstNormalizer
+from .normalize import AstNormalizer, NormalizationConfig
 
 
 def stmt_hash(stmt: ast.stmt, cfg: NormalizationConfig) -> str:

{codeclone-1.1.0 → codeclone-1.2.1}/codeclone/blocks.py

@@ -15,7 +15,7 @@ from .blockhash import stmt_hash
 from .normalize import NormalizationConfig
 
 
-@dataclass(frozen=True)
+@dataclass(frozen=True, slots=True)
 class BlockUnit:
     block_hash: str
     filepath: str
@@ -42,7 +42,8 @@ def extract_blocks(
 
     blocks: list[BlockUnit] = []
     last_start: int | None = None
-    MIN_LINE_DISTANCE = 5  # suppress overlapping windows
+    # Allow some overlap (50%), but at least 3 lines apart
+    min_line_distance = max(block_size // 2, 3)
 
     for i in range(len(stmt_hashes) - block_size + 1):
         start = getattr(body[i], "lineno", None)
@@ -50,7 +51,7 @@ def extract_blocks(
         if not start or not end:
             continue
 
-        if last_start is not None and start - last_start < MIN_LINE_DISTANCE:
+        if last_start is not None and start - last_start < min_line_distance:
             continue
 
         bh = "|".join(stmt_hashes[i : i + block_size])