codeclone 1.2.0__tar.gz → 1.2.1__tar.gz

{codeclone-1.2.0 → codeclone-1.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codeclone
-Version: 1.2.0
+Version: 1.2.1
 Summary: AST and CFG-based code clone detector for Python focused on architectural duplication
 Author-email: Den Rozhnovskiy <pytelemonbot@mail.ru>
 Maintainer-email: Den Rozhnovskiy <pytelemonbot@mail.ru>
@@ -10,11 +10,10 @@ Project-URL: Repository, https://github.com/orenlab/codeclone
 Project-URL: Issues, https://github.com/orenlab/codeclone/issues
 Project-URL: Changelog, https://github.com/orenlab/codeclone/releases
 Project-URL: Documentation, https://github.com/orenlab/codeclone/tree/main/docs
-Keywords: python,ast,code-clone,duplication,static-analysis,ci,architecture
+Keywords: python,ast,cfg,code-clone,duplication,static-analysis,architecture,control-flow,ci
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Topic :: Software Development :: Quality Assurance
-Classifier: Topic :: Software Development :: Code Generators
 Classifier: Topic :: Software Development :: Testing
 Classifier: Typing :: Typed
 Classifier: License :: OSI Approved :: MIT License
@@ -32,19 +31,22 @@ Requires-Dist: pygments>=2.19.2
 Requires-Dist: rich>=14.3.2
 Provides-Extra: dev
 Requires-Dist: pytest>=9.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=6.1.0; extra == "dev"
 Requires-Dist: build>=1.2.0; extra == "dev"
 Requires-Dist: twine>=5.0.0; extra == "dev"
 Requires-Dist: mypy>=1.19.1; extra == "dev"
+Requires-Dist: ruff>=0.12.0; extra == "dev"
 Dynamic: license-file
 
 # CodeClone
 
 [![PyPI](https://img.shields.io/pypi/v/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![Downloads](https://img.shields.io/pypi/dm/codeclone.svg)](https://pypi.org/project/codeclone/)
+[![tests](https://github.com/orenlab/codeclone/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/orenlab/codeclone/actions/workflows/tests.yml)
 [![Python](https://img.shields.io/pypi/pyversions/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![License](https://img.shields.io/pypi/l/codeclone.svg)](LICENSE)
 
-**CodeClone** is a Python code clone detector based on **normalized AST and control-flow graphs (CFG)**.
+**CodeClone** is a Python code clone detector based on **normalized Python AST and Control Flow Graphs (CFG)**.
 It helps teams discover architectural duplication and prevent new copy-paste from entering the codebase via CI.
 
 CodeClone is designed to help teams:
@@ -53,15 +55,16 @@ CodeClone is designed to help teams:
 - identify architectural hotspots,
 - prevent *new* duplication via CI and pre-commit hooks.
 
-Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against renaming,
-formatting, and minor refactoring.
+Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against
+renaming, formatting, and minor refactoring.
 
 ---
 
 ## Why CodeClone?
 
 Most existing tools detect *textual* duplication.
-CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or architectural drift.
+CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or
+architectural drift.
 
 Typical use cases:
 
@@ -79,11 +82,11 @@ Typical use cases:
 - Detects functions and methods with identical **control-flow structure**.
 - Based on **Control Flow Graph (CFG)** fingerprinting.
 - Robust to:
-  - variable renaming,
-  - constant changes,
-  - attribute renaming,
-  - formatting differences,
-  - docstrings and type annotations.
+    - variable renaming,
+    - constant changes,
+    - attribute renaming,
+    - formatting differences,
+    - docstrings and type annotations.
 - Ideal for spotting architectural duplication across layers.
 
 ### Block-level clone detection (Type-3-lite)
@@ -91,29 +94,29 @@ Typical use cases:
 - Detects repeated **statement blocks** inside larger functions.
 - Uses sliding windows over CFG-normalized statement sequences.
 - Targets:
-  - validation blocks,
-  - guard clauses,
-  - repeated orchestration logic.
+    - validation blocks,
+    - guard clauses,
+    - repeated orchestration logic.
 - Carefully filtered to reduce noise:
-  - no overlapping windows,
-  - no clones inside the same function,
-  - no `__init__` noise,
-  - size and statement-count thresholds.
+    - no overlapping windows,
+    - no clones inside the same function,
+    - no `__init__` noise,
+    - size and statement-count thresholds.
 
 ### Control-Flow Awareness (CFG v1)
 
 - Each function is converted into a **Control Flow Graph**.
 - CFG nodes contain normalized AST statements.
 - CFG edges represent structural control flow:
-  - `if` / `else`
-  - `for` / `async for` / `while`
-  - `try` / `except` / `finally`
-  - `with` / `async with`
-  - `match` / `case` (Python 3.10+)
+    - `if` / `else`
+    - `for` / `async for` / `while`
+    - `try` / `except` / `finally`
+    - `with` / `async with`
+    - `match` / `case` (Python 3.10+)
 - Current CFG semantics (v1):
-  - `break` and `continue` are treated as statements (no jump targets),
-  - after-blocks are explicit and always present,
-  - focus is on **structural similarity**, not precise runtime semantics.
+    - `break` and `continue` are treated as statements (no jump targets),
+    - after-blocks are explicit and always present,
+    - focus is on **structural similarity**, not precise runtime semantics.
 
 This design keeps clone detection **stable, deterministic, and low-noise**.
 
@@ -122,6 +125,7 @@ This design keeps clone detection **stable, deterministic, and low-noise**.
 - AST + CFG normalization instead of token matching.
 - Conservative defaults tuned for real-world Python projects.
 - Explicit thresholds for size and statement count.
+- No probabilistic scoring or heuristic similarity thresholds.
 - Focus on *architectural duplication*, not micro-similarities.
 
 ### CI-friendly baseline mode
@@ -188,14 +192,26 @@ Commit the generated baseline file to the repository.
 ### 2. Use in CI
 
 ```bash
-codeclone . --fail-on-new
+codeclone . --fail-on-new --no-progress
 ```
 
 Behavior:
 
-- ✅ existing clones are allowed,
-- ❌ build fails if *new* clones appear,
-- ✅ refactoring that removes duplication is always allowed.
+- existing clones are allowed,
+- the build fails if *new* clones appear,
+- refactoring that removes duplication is always allowed.
+
+`--fail-on-new` exits with a non-zero code when new clones are detected.
+
+### Python Version Consistency for Baseline Checks
+
+Due to inherent differences in Python’s AST between interpreter versions, baseline
+generation and verification must be performed using the same Python version.
+
+This ensures deterministic and reproducible clone detection results.
+
+CI checks therefore pin baseline verification to a single Python version, while the
+test matrix continues to validate compatibility across Python 3.10–3.14.
 
 ---
 
@@ -203,14 +219,14 @@ Behavior:
 
 ```yaml
 repos:
--   repo: local
+  - repo: local
     hooks:
-    -   id: codeclone
+      - id: codeclone
         name: CodeClone
         entry: codeclone
         language: python
-        args: [".", "--fail-on-new"]
-        types: [python]
+        args: [ ".", "--fail-on-new" ]
+        types: [ python ]
 ```
 
 ---
@@ -243,6 +259,7 @@ repos:
 6. Apply conservative filters to suppress noise.
 
 See the architectural overview:
+
 - [docs/architecture.md](docs/architecture.md)
 
 ---
@@ -255,6 +272,7 @@ to improve structural clone detection robustness.
 The CFG is a **structural abstraction**, not a runtime execution model.
 
 See full design and semantics:
+
 - [docs/cfg.md](docs/cfg.md)
 
 ---

{codeclone-1.2.0 → codeclone-1.2.1}/README.md

@@ -2,10 +2,11 @@
 
 [![PyPI](https://img.shields.io/pypi/v/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![Downloads](https://img.shields.io/pypi/dm/codeclone.svg)](https://pypi.org/project/codeclone/)
+[![tests](https://github.com/orenlab/codeclone/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/orenlab/codeclone/actions/workflows/tests.yml)
 [![Python](https://img.shields.io/pypi/pyversions/codeclone.svg)](https://pypi.org/project/codeclone/)
 [![License](https://img.shields.io/pypi/l/codeclone.svg)](LICENSE)
 
-**CodeClone** is a Python code clone detector based on **normalized AST and control-flow graphs (CFG)**.
+**CodeClone** is a Python code clone detector based on **normalized Python AST and Control Flow Graphs (CFG)**.
 It helps teams discover architectural duplication and prevent new copy-paste from entering the codebase via CI.
 
 CodeClone is designed to help teams:
@@ -14,15 +15,16 @@ CodeClone is designed to help teams:
 - identify architectural hotspots,
 - prevent *new* duplication via CI and pre-commit hooks.
 
-Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against renaming,
-formatting, and minor refactoring.
+Unlike token- or text-based tools, CodeClone operates on **normalized Python AST and CFG**, making it robust against
+renaming, formatting, and minor refactoring.
 
 ---
 
 ## Why CodeClone?
 
 Most existing tools detect *textual* duplication.
-CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or architectural drift.
+CodeClone detects **structural and block-level duplication**, which usually signals missing abstractions or
+architectural drift.
 
 Typical use cases:
 
@@ -40,11 +42,11 @@ Typical use cases:
 - Detects functions and methods with identical **control-flow structure**.
 - Based on **Control Flow Graph (CFG)** fingerprinting.
 - Robust to:
-  - variable renaming,
-  - constant changes,
-  - attribute renaming,
-  - formatting differences,
-  - docstrings and type annotations.
+    - variable renaming,
+    - constant changes,
+    - attribute renaming,
+    - formatting differences,
+    - docstrings and type annotations.
 - Ideal for spotting architectural duplication across layers.
 
 ### Block-level clone detection (Type-3-lite)
@@ -52,29 +54,29 @@ Typical use cases:
 - Detects repeated **statement blocks** inside larger functions.
 - Uses sliding windows over CFG-normalized statement sequences.
 - Targets:
-  - validation blocks,
-  - guard clauses,
-  - repeated orchestration logic.
+    - validation blocks,
+    - guard clauses,
+    - repeated orchestration logic.
 - Carefully filtered to reduce noise:
-  - no overlapping windows,
-  - no clones inside the same function,
-  - no `__init__` noise,
-  - size and statement-count thresholds.
+    - no overlapping windows,
+    - no clones inside the same function,
+    - no `__init__` noise,
+    - size and statement-count thresholds.
 
 ### Control-Flow Awareness (CFG v1)
 
 - Each function is converted into a **Control Flow Graph**.
 - CFG nodes contain normalized AST statements.
 - CFG edges represent structural control flow:
-  - `if` / `else`
-  - `for` / `async for` / `while`
-  - `try` / `except` / `finally`
-  - `with` / `async with`
-  - `match` / `case` (Python 3.10+)
+    - `if` / `else`
+    - `for` / `async for` / `while`
+    - `try` / `except` / `finally`
+    - `with` / `async with`
+    - `match` / `case` (Python 3.10+)
 - Current CFG semantics (v1):
-  - `break` and `continue` are treated as statements (no jump targets),
-  - after-blocks are explicit and always present,
-  - focus is on **structural similarity**, not precise runtime semantics.
+    - `break` and `continue` are treated as statements (no jump targets),
+    - after-blocks are explicit and always present,
+    - focus is on **structural similarity**, not precise runtime semantics.
 
 This design keeps clone detection **stable, deterministic, and low-noise**.
 
@@ -83,6 +85,7 @@ This design keeps clone detection **stable, deterministic, and low-noise**.
 - AST + CFG normalization instead of token matching.
 - Conservative defaults tuned for real-world Python projects.
 - Explicit thresholds for size and statement count.
+- No probabilistic scoring or heuristic similarity thresholds.
 - Focus on *architectural duplication*, not micro-similarities.
 
 ### CI-friendly baseline mode
@@ -149,14 +152,26 @@ Commit the generated baseline file to the repository.
 ### 2. Use in CI
 
 ```bash
-codeclone . --fail-on-new
+codeclone . --fail-on-new --no-progress
 ```
 
 Behavior:
 
-- ✅ existing clones are allowed,
-- ❌ build fails if *new* clones appear,
-- ✅ refactoring that removes duplication is always allowed.
+- existing clones are allowed,
+- the build fails if *new* clones appear,
+- refactoring that removes duplication is always allowed.
+
+`--fail-on-new` exits with a non-zero code when new clones are detected.
+
+### Python Version Consistency for Baseline Checks
+
+Due to inherent differences in Python’s AST between interpreter versions, baseline
+generation and verification must be performed using the same Python version.
+
+This ensures deterministic and reproducible clone detection results.
+
+CI checks therefore pin baseline verification to a single Python version, while the
+test matrix continues to validate compatibility across Python 3.10–3.14.
 
 ---
 
@@ -164,14 +179,14 @@ Behavior:
 
 ```yaml
 repos:
--   repo: local
+  - repo: local
     hooks:
-    -   id: codeclone
+      - id: codeclone
         name: CodeClone
         entry: codeclone
         language: python
-        args: [".", "--fail-on-new"]
-        types: [python]
+        args: [ ".", "--fail-on-new" ]
+        types: [ python ]
 ```
 
 ---
@@ -204,6 +219,7 @@ repos:
 6. Apply conservative filters to suppress noise.
 
 See the architectural overview:
+
 - [docs/architecture.md](docs/architecture.md)
 
 ---
@@ -216,6 +232,7 @@ to improve structural clone detection robustness.
 The CFG is a **structural abstraction**, not a runtime execution model.
 
 See full design and semantics:
+
 - [docs/cfg.md](docs/cfg.md)
 
 ---

{codeclone-1.2.0 → codeclone-1.2.1}/codeclone/__init__.py

@@ -6,7 +6,7 @@ Copyright (c) 2026 Den Rozhnovskiy
 Licensed under the MIT License.
 """
 
-from importlib.metadata import version, PackageNotFoundError
+from importlib.metadata import PackageNotFoundError, version
 
 try:
     __version__ = version("codeclone")

{codeclone-1.2.0 → codeclone-1.2.1}/codeclone/baseline.py

@@ -9,14 +9,19 @@ Licensed under the MIT License.
 from __future__ import annotations
 
 import json
+from collections.abc import Mapping
 from pathlib import Path
+from typing import Any
 
 
 class Baseline:
+    __slots__ = ("blocks", "functions", "path", "python_version")
+
     def __init__(self, path: str | Path):
         self.path = Path(path)
         self.functions: set[str] = set()
         self.blocks: set[str] = set()
+        self.python_version: str | None = None
 
     def load(self) -> None:
         if not self.path.exists():
@@ -26,6 +31,10 @@ class Baseline:
             data = json.loads(self.path.read_text("utf-8"))
             self.functions = set(data.get("functions", []))
             self.blocks = set(data.get("blocks", []))
+            python_version = data.get("python_version")
+            self.python_version = (
+                python_version if isinstance(python_version, str) else None
+            )
         except json.JSONDecodeError as e:
             raise ValueError(f"Corrupted baseline file at {self.path}: {e}") from e
 
@@ -33,10 +42,7 @@ class Baseline:
         self.path.parent.mkdir(parents=True, exist_ok=True)
         self.path.write_text(
             json.dumps(
-                {
-                    "functions": sorted(self.functions),
-                    "blocks": sorted(self.blocks),
-                },
+                _baseline_payload(self.functions, self.blocks, self.python_version),
                 indent=2,
                 ensure_ascii=False,
             ),
@@ -45,14 +51,34 @@ class Baseline:
 
     @staticmethod
     def from_groups(
-        func_groups: dict, block_groups: dict, path: str | Path = ""
-    ) -> "Baseline":
+        func_groups: Mapping[str, object],
+        block_groups: Mapping[str, object],
+        path: str | Path = "",
+        python_version: str | None = None,
+    ) -> Baseline:
         bl = Baseline(path)
         bl.functions = set(func_groups.keys())
         bl.blocks = set(block_groups.keys())
+        bl.python_version = python_version
         return bl
 
-    def diff(self, func_groups: dict, block_groups: dict) -> tuple[set, set]:
+    def diff(
+        self, func_groups: Mapping[str, object], block_groups: Mapping[str, object]
+    ) -> tuple[set[str], set[str]]:
         new_funcs = set(func_groups.keys()) - self.functions
         new_blocks = set(block_groups.keys()) - self.blocks
         return new_funcs, new_blocks
+
+
+def _baseline_payload(
+    functions: set[str],
+    blocks: set[str],
+    python_version: str | None,
+) -> dict[str, Any]:
+    payload: dict[str, Any] = {
+        "functions": sorted(functions),
+        "blocks": sorted(blocks),
+    }
+    if python_version:
+        payload["python_version"] = python_version
+    return payload

{codeclone-1.2.0 → codeclone-1.2.1}/codeclone/blockhash.py

@@ -11,7 +11,7 @@ from __future__ import annotations
 import ast
 import hashlib
 
-from .normalize import NormalizationConfig, AstNormalizer
+from .normalize import AstNormalizer, NormalizationConfig
 
 
 def stmt_hash(stmt: ast.stmt, cfg: NormalizationConfig) -> str:

{codeclone-1.2.0 → codeclone-1.2.1}/codeclone/blocks.py

@@ -15,7 +15,7 @@ from .blockhash import stmt_hash
 from .normalize import NormalizationConfig
 
 
-@dataclass(frozen=True)
+@dataclass(frozen=True, slots=True)
 class BlockUnit:
     block_hash: str
     filepath: str
@@ -42,7 +42,8 @@ def extract_blocks(
 
     blocks: list[BlockUnit] = []
     last_start: int | None = None
-    MIN_LINE_DISTANCE = 5  # suppress overlapping windows
+    # Allow some overlap (50%), but at least 3 lines apart
+    min_line_distance = max(block_size // 2, 3)
 
     for i in range(len(stmt_hashes) - block_size + 1):
         start = getattr(body[i], "lineno", None)
@@ -50,7 +51,7 @@ def extract_blocks(
         if not start or not end:
             continue
 
-        if last_start is not None and start - last_start < MIN_LINE_DISTANCE:
+        if last_start is not None and start - last_start < min_line_distance:
             continue
 
         bh = "|".join(stmt_hashes[i : i + block_size])

codeclone-1.2.1/codeclone/cache.py

@@ -0,0 +1,187 @@
+"""
+CodeClone — AST and CFG-based code clone detector for Python
+focused on architectural duplication.
+
+Copyright (c) 2026 Den Rozhnovskiy
+Licensed under the MIT License.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import hmac
+import json
+import os
+import secrets
+from collections.abc import Mapping
+from dataclasses import asdict
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, TypedDict, cast
+
+if TYPE_CHECKING:
+    from .blocks import BlockUnit
+    from .extractor import Unit
+
+from .errors import CacheError
+
+
+class FileStat(TypedDict):
+    mtime_ns: int
+    size: int
+
+
+class UnitDict(TypedDict):
+    qualname: str
+    filepath: str
+    start_line: int
+    end_line: int
+    loc: int
+    stmt_count: int
+    fingerprint: str
+    loc_bucket: str
+
+
+class BlockDict(TypedDict):
+    block_hash: str
+    filepath: str
+    qualname: str
+    start_line: int
+    end_line: int
+    size: int
+
+
+class CacheEntry(TypedDict):
+    stat: FileStat
+    units: list[UnitDict]
+    blocks: list[BlockDict]
+
+
+class CacheData(TypedDict):
+    version: str
+    files: dict[str, CacheEntry]
+
+
+class Cache:
+    __slots__ = ("data", "load_warning", "path", "secret")
+    CACHE_VERSION = "1.0"
+
+    def __init__(self, path: str | Path):
+        self.path = Path(path)
+        self.data: CacheData = {"version": self.CACHE_VERSION, "files": {}}
+        self.secret = self._load_secret()
+        self.load_warning: str | None = None
+
+    def _load_secret(self) -> bytes:
+        """Load or create cache signing secret."""
+        # Store secret in the same directory as the cache file, named .cache_secret
+        # If cache is at ~/.cache/codeclone/cache.json, secret is
+        # ~/.cache/codeclone/.cache_secret
+        secret_path = self.path.parent / ".cache_secret"
+        if secret_path.exists():
+            return secret_path.read_bytes()
+        else:
+            secret = secrets.token_bytes(32)
+            try:
+                self.path.parent.mkdir(parents=True, exist_ok=True)
+                secret_path.write_bytes(secret)
+                # Set restrictive permissions on secret file (Unix only)
+                if os.name == "posix":
+                    secret_path.chmod(0o600)
+            except OSError:
+                pass
+            return secret
+
+    def _sign_data(self, data: Mapping[str, Any]) -> str:
+        """Create HMAC signature of cache data."""
+        # Sort keys for deterministic JSON serialization
+        data_str = json.dumps(data, sort_keys=True)
+        return hmac.new(self.secret, data_str.encode(), hashlib.sha256).hexdigest()
+
+    def load(self) -> None:
+        if not self.path.exists():
+            return
+
+        try:
+            raw = json.loads(self.path.read_text("utf-8"))
+            stored_sig = raw.get("_signature")
+
+            # Extract data without signature for verification
+            data = {k: v for k, v in raw.items() if k != "_signature"}
+
+            # Verify signature
+            expected_sig = self._sign_data(data)
+            if stored_sig != expected_sig:
+                self.load_warning = "Cache signature mismatch; ignoring cache."
+                self.data = {"version": self.CACHE_VERSION, "files": {}}
+                return
+
+            if data.get("version") != self.CACHE_VERSION:
+                self.load_warning = (
+                    "Cache version mismatch "
+                    f"(found {data.get('version')}); ignoring cache."
+                )
+                self.data = {"version": self.CACHE_VERSION, "files": {}}
+                return
+
+            # Basic structure check
+            if not isinstance(data.get("files"), dict):
+                self.load_warning = "Cache format invalid; ignoring cache."
+                self.data = {"version": self.CACHE_VERSION, "files": {}}
+                return
+
+            self.data = cast(CacheData, data)
+            self.load_warning = None
+
+        except (json.JSONDecodeError, ValueError):
+            self.load_warning = "Cache corrupted; ignoring cache."
+            self.data = {"version": self.CACHE_VERSION, "files": {}}
+
+    def save(self) -> None:
+        try:
+            self.path.parent.mkdir(parents=True, exist_ok=True)
+
+            # Add signature
+            data_with_sig = {**self.data, "_signature": self._sign_data(self.data)}
+
+            self.path.write_text(
+                json.dumps(data_with_sig, ensure_ascii=False, indent=2),
+                "utf-8",
+            )
+        except OSError as e:
+            raise CacheError(f"Failed to save cache: {e}") from e
+
+    def get_file_entry(self, filepath: str) -> CacheEntry | None:
+        entry = self.data["files"].get(filepath)
+
+        if entry is None:
+            return None
+
+        if not isinstance(entry, dict):
+            return None
+
+        required = {"stat", "units", "blocks"}
+        if not required.issubset(entry.keys()):
+            return None
+
+        return entry
+
+    def put_file_entry(
+        self,
+        filepath: str,
+        stat_sig: FileStat,
+        units: list[Unit],
+        blocks: list[BlockUnit],
+    ) -> None:
+        self.data["files"][filepath] = {
+            "stat": stat_sig,
+            "units": cast(list[UnitDict], cast(object, [asdict(u) for u in units])),
+            "blocks": cast(list[BlockDict], cast(object, [asdict(b) for b in blocks])),
+        }
+
+
+def file_stat_signature(path: str) -> FileStat:
+    st = os.stat(path)
+    return {
+        "mtime_ns": st.st_mtime_ns,
+        "size": st.st_size,
+    }