sourcecode 1.4.0__tar.gz → 1.6.0__tar.gz

sourcecode-1.6.0/.continue-here.md

@@ -0,0 +1,105 @@
+# Continue Here — sourcecode Java/Spring Enterprise Bug Fixes v1.5.x
+**Created:** 2026-05-08  
+**Session context:** Ad-hoc benchmark-driven bug fixes (not a GSD phase execution)
+
+---
+
+## What Was Done This Session
+
+Applied 5 fixes targeting regressions in v1.5.0 benchmark report on saint-server (3681 .java, 460 *RestController*, 52 *Mapper.xml, Spring Boot 2.7.18 DDD+legacy layout).
+
+### Root Causes Found and Fixed
+
+| ID | File(s) | Root Cause | Fix |
+|----|---------|-----------|-----|
+| N1 | `detectors/java.py` | Scanner `depth >= max_depth=10` cuts `ddd/*/infraestructure/rest/` files silently. Path `src/main/java/com/m3/saint/ddd/domain/infraestructure/rest/` = 10 parts → excluded from file_tree → not in entry_points. | New `_augment_deep_java_controllers()`: direct `os.walk` of `src/main/java/` for `*Controller*.java` files not in file_tree |
+| G3 | `contract_pipeline.py` | Mapper.xml score=0.3 (no `.xml` suffix boost) < Java score=0.4. 3000+ files compete for 2500-file cap — all 52 Mapper.xml cut. | `_is_priority()`: Mapper.xml gets same priority as entry_points in cap sort |
+| T11 | `contract_pipeline.py` | Centrality sort key ignored `is_entrypoint`. REST controllers have low fan_in → ranked below JPA entities. | Added `-c.is_entrypoint` as primary key in centrality sort |
+| P3-B | `cli.py` | Auto-switch to centrality when DDD architecture detected — actively hurt REST controller visibility. | Removed 5-line P3-B block |
+| N2 | `serializer.py` | `hotspots` removed from top-level output in v1.5.0, only in `semantic_summary`. Breaking for consumers using `d["hotspots"]`. | Restore `result["hotspots"]` at top level when `semantic_summary.hotspots` exists |
+
+### Modified Files (uncommitted — user requested no commits)
+
+```
+src/sourcecode/cli.py               # P3-B removed (DDD auto-centrality switch)
+src/sourcecode/contract_pipeline.py # G3: _is_priority() + T11: centrality sort key
+src/sourcecode/detectors/java.py    # N1: _augment_deep_java_controllers() method
+src/sourcecode/serializer.py        # N2: result["hotspots"] top-level restored
+```
+
+### Test Status
+- 651 passed, 3 skipped (subset excluding pre-existing failures) — zero regressions
+
+---
+
+## What Remains
+
+### Not Fixed (deferred)
+
+| ID | Description |
+|----|-------------|
+| M1 | `prepare-context` from monorepo root detects Angular instead of saint-server Java. Requires monorepo subproject routing in `prepare_context.py` → `TaskContextBuilder` / `RepoClassifier`. |
+| M2 | `@M3FiltroSeguridad` custom security annotation not aggregated into security map |
+| M3 | Spring profiles detected but not linked to `application-{profile}.yml` files |
+
+### Verification Commands (run against saint-server after commit)
+
+```bash
+# N1: DDD controllers now captured (need --depth 12+ for tree, augment works at any depth)
+sourcecode saint-server --depth 12 --entrypoints-only --format json | python -c "
+import sys, json
+d = json.load(sys.stdin)
+rest = [x for x in d.get('contracts', []) if 'RestController' in x.get('path','')]
+ddd  = [x for x in rest if '/ddd/' in x['path'] or '/infraestructure/' in x['path']]
+print(f'REST total: {len(rest)} | DDD: {len(ddd)}')"
+
+# G3: MyBatis XML in contracts
+sourcecode saint-server --depth 10 --format json | python -c "
+import sys, json
+d = json.load(sys.stdin)
+xml = [x for x in d.get('contracts', []) if x.get('language') == 'mybatis-xml']
+print(f'MyBatis contracts: {len(xml)}')"
+
+# N2: hotspots at top level restored
+sourcecode saint-server --semantics --format json | python -c "
+import sys, json
+d = json.load(sys.stdin)
+print('top-level hotspots:', len(d.get('hotspots', [])))
+print('semantic_summary.hotspots:', len(d.get('semantic_summary',{}).get('hotspots',[])))"
+
+# T11: centrality now surfaces REST controllers
+sourcecode saint-server --rank-by centrality --depth 10 --format json | python -c "
+import sys, json
+d = json.load(sys.stdin)
+rest = [x for x in d.get('contracts', []) if 'RestController' in x.get('path','')]
+print(f'REST in centrality: {len(rest)}')"
+```
+
+**Note on N1 depth:** `_augment_deep_java_controllers` bypasses the scanner depth limit and always walks `src/main/java/` directly. DDD controllers will be detected as entry_points regardless of `--depth`. However, for these same files to appear in the general file_paths (not just entry_points), the scanner itself needs `--depth >= 11`.
+
+---
+
+## Previous Session Fixes (already committed)
+
+C1/C2/C3/G1/G2/G3 from v1.4.0→v1.5.0:
+- `--mode standard` key `file_contracts` → `contracts`
+- Controller-first scan, cap 500→1000
+- DDD bounded contexts from package structure
+- Maven BOM limitation warning
+- MyBatis XML extractor (code existed but files were cut by cap — fixed in this session)
+
+---
+
+## GSD Project State (unchanged)
+
+All 13 phases complete (see `.planning/STATE.md`). Both sessions were ad-hoc bug work from benchmark reports, not planned phase executions.
+
+---
+
+## Resume Instructions
+
+1. `git diff --stat` — verify 4 uncommitted files
+2. `python -m pytest tests/ --ignore=tests/test_block2_coverage.py --ignore=tests/test_dependency_analyzer_node_python.py --ignore=tests/test_pipeline_integrity.py -q`
+3. Commit the fixes
+4. Run verification commands above against saint-server
+5. If tackling M1, entry point is `prepare_context.py:TaskContextBuilder.build()` + `repo_classifier.py`

{sourcecode-1.4.0 → sourcecode-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sourcecode
-Version: 1.4.0
+Version: 1.6.0
 Summary: Deterministic codebase context for AI coding agents
 License:                                  Apache License
                                    Version 2.0, January 2004

{sourcecode-1.4.0 → sourcecode-1.6.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sourcecode"
-version = "1.4.0"
+version = "1.6.0"
 description = "Deterministic codebase context for AI coding agents"
 readme = "README.md"
 requires-python = ">=3.9"

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/__init__.py

@@ -1,3 +1,3 @@
 """sourcecode — Deterministic codebase context maps for AI coding agents."""
 
-__version__ = "1.4.0"
+__version__ = "1.6.0"

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/architecture_analyzer.py

@@ -182,8 +182,19 @@ class ArchitectureAnalyzer:
         ddd_result = self._detect_ddd(sm.file_paths)
         if ddd_result is not None:
             ddd_pattern, ddd_layers, ddd_contexts, ddd_layer_names = ddd_result
-            domains_for_ddd = self._cluster_domains(filtered) if len(filtered) >= 2 else []
             module_files = self._build_ddd_module_files(sm.file_paths, ddd_contexts)
+            # Use DDD bounded context names as domains so --architecture shows each
+            # context as a distinct domain instead of collapsing all files under
+            # the Maven path segment (e.g. "java").
+            domains_for_ddd = [
+                ArchitectureDomain(
+                    name=n,
+                    files=module_files.get(n, []),
+                    role="DDD bounded context",
+                    confidence="high",
+                )
+                for n in ddd_contexts
+            ]
             bc_list = [
                 BoundedContext(name=n, modules=module_files.get(n, []), confidence="high")
                 for n in ddd_contexts

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/ast_extractor.py

@@ -1172,6 +1172,51 @@ def _detect_role(path: str, contract: FileContract) -> str:
     return "util"
 
 
+# ---------------------------------------------------------------------------
+# MyBatis XML mapper extractor
+# ---------------------------------------------------------------------------
+
+def _extract_mybatis_xml(rel_path: str, source: str) -> FileContract:
+    """Extract namespace and SQL operations from a MyBatis *Mapper.xml file."""
+    import re as _re
+    from xml.etree import ElementTree
+
+    _NS_STRIP = _re.compile(r"\{[^}]+\}")
+    _SQL_OPS = frozenset({"select", "insert", "update", "delete"})
+
+    exports: list[ExportRecord] = []
+    namespace: str | None = None
+
+    try:
+        root_elem = ElementTree.fromstring(source.encode("utf-8"))
+        namespace = root_elem.get("namespace") or None
+        for elem in root_elem:
+            tag = _NS_STRIP.sub("", elem.tag).lower()
+            if tag in _SQL_OPS:
+                op_id = (elem.get("id") or "").strip()
+                if op_id:
+                    # type_ref carries select/insert/update/delete for the serializer
+                    exports.append(ExportRecord(kind="query", name=op_id, type_ref=tag))
+    except Exception:
+        return FileContract(
+            path=rel_path,
+            language="mybatis-xml",
+            role="mybatis-mapper",
+            extraction_method="heuristic",
+            limitations=["xml_parse_error: failed to parse mapper XML"],
+        )
+
+    deps = [f"namespace:{namespace}"] if namespace else []
+    return FileContract(
+        path=rel_path,
+        language="mybatis-xml",
+        role="mybatis-mapper",
+        exports=exports,
+        dependencies=deps,
+        extraction_method="heuristic",
+    )
+
+
 # ---------------------------------------------------------------------------
 # AstExtractor public class
 # ---------------------------------------------------------------------------
@@ -1191,6 +1236,16 @@ class AstExtractor:
         return self._ts_ok
 
     def extract(self, path: Path, root: Optional[Path] = None) -> Optional[FileContract]:
+        # MyBatis mapper XML — handled before the language map lookup so .xml
+        # files are only processed when they match the mapper naming convention.
+        if path.suffix.lower() == ".xml" and path.name.endswith("Mapper.xml"):
+            try:
+                source = path.read_text(encoding="utf-8", errors="replace")
+            except OSError:
+                return None
+            rel_path = str(path.relative_to(root)).replace("\\", "/") if root else path.name
+            return _extract_mybatis_xml(rel_path, source)
+
         ext = path.suffix.lower()
         language = _LANGUAGE_MAP.get(ext)
         if language is None:

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/cli.py

@@ -1399,12 +1399,6 @@ def main(
                         ))
         sm = _replace(sm, pipeline_trace=_trace.build_trace())
 
-    # P3-B: Auto-switch to centrality ranking when DDD layout detected
-    if (rank_by == "relevance"
-            and sm.architecture is not None
-            and sm.architecture.pattern == "ddd"):
-        rank_by = "centrality"
-
     # Contract pipeline — runs for mode=contract|standard|deep|hybrid (skip for raw)
     _is_contract_mode = mode in ("contract", "standard")
     _pipeline_error = False

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/contract_pipeline.py

@@ -219,9 +219,18 @@ class ContractPipeline:
             fname = Path(pn).name
             return any(fname.startswith(pat) or f".{pat.strip('.')}" in fname for pat in _TEST_PATTERNS)
 
+        def _is_extractable(p: str) -> bool:
+            suf = Path(p).suffix.lower()
+            if suf in _SRC_EXTENSIONS:
+                return True
+            # MyBatis mapper XML files — only *Mapper.xml, not all XML
+            if suf == ".xml" and p.endswith("Mapper.xml"):
+                return True
+            return False
+
         src_paths = [
             p for p in file_paths
-            if Path(p).suffix.lower() in _SRC_EXTENSIONS
+            if _is_extractable(p)
             and not scorer.is_noise(p)
             and (symbol is not None or changed_only or not _is_test(p))
         ]
@@ -231,10 +240,18 @@ class ContractPipeline:
 
         # Apply max_files cap — bypass when symbol search to ensure defining files are found.
         # A symbol query over a large repo needs all files; result set is small after filtering.
+        # MyBatis Mapper.xml contracts rank below Java files on path score alone (.xml has no
+        # suffix boost). Give them the same priority slot as entry_points so they survive the cap.
+        def _is_priority(p: str) -> bool:
+            if p in entry_paths:
+                return True
+            name = p.rsplit("/", 1)[-1]
+            return name.lower().endswith("mapper.xml")
+
         if symbol is None and len(src_paths) > self.max_files:
             src_paths = sorted(
                 src_paths,
-                key=lambda p: (p in entry_paths, scorer.score(p)),
+                key=lambda p: (_is_priority(p), scorer.score(p)),
                 reverse=True,
             )[:self.max_files]
 
@@ -353,7 +370,9 @@ class ContractPipeline:
 
     def _rank(self, contracts: list[FileContract], rank_by: RankStrategy) -> list[FileContract]:
         if rank_by == "centrality":
-            return sorted(contracts, key=lambda c: (-(c.fan_in + c.fan_out), c.path))
+            # Entrypoints (REST controllers, main classes) surface first even in centrality mode:
+            # they have low fan_in (not imported) but are the primary API surface.
+            return sorted(contracts, key=lambda c: (-c.is_entrypoint, -(c.fan_in + c.fan_out), c.path))
         if rank_by == "git-churn":
             return sorted(contracts, key=lambda c: (-c.is_changed, -c.relevance_score, c.path))
         # Default: relevance — path breaks ties deterministically

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/dependency_analyzer.py

@@ -1191,6 +1191,18 @@ class DependencyAnalyzer:
         limitations: list[str] = []
         if not records:
             limitations.append("java: pom.xml sin dependencias parseables (puede usar BOM o propiedades)")
+
+        # Warn when Spring Boot BOM manages transitive deps — they can't be resolved statically.
+        parent_artifact_local = (
+            root_elem.findtext(f"{ns}parent/{ns}artifactId") or ""
+        ).strip() if parent_elem is not None else ""
+        if parent_artifact_local == "spring-boot-starter-parent" and parent_version:
+            limitations.append(
+                f"spring_boot_bom_detected: transitive deps managed by Spring Boot BOM "
+                f"v{parent_version}, not resolved statically. "
+                "Run 'mvn dependency:tree' for the full transitive tree."
+            )
+
         return records, limitations
 
     def _analyze_gradle(self, root: Path) -> tuple[list[DependencyRecord], list[str]]:

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/detectors/java.py

@@ -18,7 +18,7 @@ _NS_TAG_RE = re.compile(r"\{[^}]+\}")
 
 _MAX_FILE_SIZE = 256 * 1024  # 256 KB
 _MAX_JAVA_ENTRY_SCAN = 1000
-_MAX_ANNOTATION_ENTRY_POINTS = 500
+_MAX_ANNOTATION_ENTRY_POINTS = 1000
 
 _REST_CONTROLLER_RE = re.compile(r'@RestController\b')
 _MVC_CONTROLLER_RE = re.compile(r'@Controller\b')
@@ -225,6 +225,12 @@ class JavaDetector(AbstractDetector):
         all_paths = flatten_file_tree(context.file_tree)
         all_java = [p for p in all_paths if p.endswith(".java")]
 
+        # Augment with a direct scan of standard Java source roots for Controller-named
+        # files that the depth-limited file_tree scanner may have missed.
+        # DDD layouts place REST controllers at depth 10+ (e.g.
+        # src/main/java/com/org/app/ddd/domain/infraestructure/rest/XxxRestController.java).
+        self._augment_deep_java_controllers(context, all_java)
+
         # 1. @SpringBootApplication entry: Application.java / Main.java by name
         app_candidates = [
             p for p in all_java
@@ -236,10 +242,12 @@ class JavaDetector(AbstractDetector):
         ]
 
         # 2. Annotation-based scan: @RestController, @WebFilter, FilterRegistrationBean
-        scan_candidates = [
-            p for p in all_java
-            if "/test/" not in p and "/tests/" not in p
-        ][:_MAX_JAVA_ENTRY_SCAN]
+        # Prioritize Controller-named files so all REST controllers are detected
+        # even in large codebases where total files > _MAX_JAVA_ENTRY_SCAN.
+        _non_test = [p for p in all_java if "/test/" not in p and "/tests/" not in p]
+        _ctrl_files = [p for p in _non_test if "Controller" in p]
+        _other_files = [p for p in _non_test if "Controller" not in p]
+        scan_candidates = _ctrl_files + _other_files[:max(0, _MAX_JAVA_ENTRY_SCAN - len(_ctrl_files))]
 
         annotation_eps: list[EntryPoint] = []
         for rel_path in scan_candidates:
@@ -268,6 +276,40 @@ class JavaDetector(AbstractDetector):
                 unique_eps.append(ep)
         return unique_eps
 
+    def _augment_deep_java_controllers(self, context: DetectionContext, all_java: list[str]) -> None:
+        """Scan standard Java source roots for *Controller*.java files not in all_java.
+
+        The depth-limited file_tree scanner misses files at depth >= max_depth.
+        DDD layouts place REST controllers deep (e.g. depth 10+), so we supplement
+        with a direct filesystem walk scoped to the standard Maven/Gradle source root.
+        """
+        import os as _os
+        existing = set(all_java)
+        # Standard Java source root candidates (Maven first, then Gradle/other)
+        _SRC_ROOTS = ("src/main/java", "src/main/kotlin", "src/java", "src")
+        for src_root_name in _SRC_ROOTS:
+            src_root = context.root / src_root_name
+            if not src_root.is_dir():
+                continue
+            try:
+                for dirpath, _dirs, filenames in _os.walk(str(src_root)):
+                    for fname in filenames:
+                        if "Controller" not in fname or not fname.endswith(".java"):
+                            continue
+                        full = Path(dirpath) / fname
+                        if full.is_symlink():
+                            continue
+                        try:
+                            rel = str(full.relative_to(context.root)).replace("\\", "/")
+                            if rel not in existing:
+                                all_java.append(rel)
+                                existing.add(rel)
+                        except ValueError:
+                            pass
+            except OSError:
+                pass
+            return  # use only first matching source root
+
     def _scan_java_file_for_entry_points(self, abs_path: Path, rel_path: str) -> list[EntryPoint]:
         try:
             if abs_path.stat().st_size > _MAX_FILE_SIZE:

{sourcecode-1.4.0 → sourcecode-1.6.0}/src/sourcecode/serializer.py

@@ -1110,6 +1110,10 @@ def standard_view(sm: SourceMap, *, include_tree: bool = False) -> dict[str, Any
 
     if sm.semantic_summary is not None and sm.semantic_summary.requested:
         result["semantic_summary"] = asdict(sm.semantic_summary)
+        # Backward compat: also emit hotspots at top level (moved to semantic_summary in v1.5.0).
+        # Consumers reading d["hotspots"] directly still work.
+        if sm.semantic_summary.hotspots:
+            result["hotspots"] = sm.semantic_summary.hotspots[:10]
         # Defensive filter: never emit objects with null required fields.
         # A null entry in these arrays is worse than a shorter array — it causes
         # agents to misinterpret the analysis as valid when it is not.
@@ -1439,10 +1443,31 @@ def _serialize_contract_java(c: Any) -> dict[str, Any]:
     return item
 
 
+def _serialize_contract_mybatis_xml(c: Any) -> dict[str, Any]:
+    """Serialize a MyBatis *Mapper.xml contract."""
+    item: dict[str, Any] = {"path": c.path, "language": "mybatis-xml"}
+    # Extract namespace stored as "namespace:<fqn>" in dependencies
+    for dep in (c.dependencies or []):
+        if dep.startswith("namespace:"):
+            item["namespace"] = dep[len("namespace:"):]
+            break
+    exports_out: list[dict] = []
+    for e in c.exports:
+        entry: dict = {"kind": e.kind, "name": e.name}
+        if getattr(e, "type_ref", None):
+            entry["type"] = e.type_ref
+        exports_out.append(entry)
+    if exports_out:
+        item["exports"] = exports_out
+    return item
+
+
 def _serialize_contract_minimal(c: Any) -> dict[str, Any]:
     """Serialize one FileContract to minimal format."""
     if getattr(c, "language", None) == "java":
         return _serialize_contract_java(c)
+    if getattr(c, "language", None) == "mybatis-xml":
+        return _serialize_contract_mybatis_xml(c)
     item: dict[str, Any] = {"path": c.path, "role": c.role}
 
     if c.is_changed:
@@ -1560,6 +1585,11 @@ def _contract_view_standard(
     if contracts:
         serialized: list[dict[str, Any]] = []
         for c in contracts:
+            if getattr(c, "language", None) == "mybatis-xml":
+                item = _serialize_contract_mybatis_xml(c)
+                item["relevance_score"] = round(c.relevance_score, 3)
+                serialized.append(item)
+                continue
             item: dict[str, Any] = {
                 "path": c.path,
                 "language": c.language,
@@ -1609,7 +1639,7 @@ def _contract_view_standard(
                     item["ranking_reasons"] = non_trivial
             item["method"] = c.extraction_method
             serialized.append(item)
-        result["file_contracts"] = serialized
+        result["contracts"] = serialized
 
     # Optional analysis sections (deep mode or when analyzers ran)
     if include_optional:

{sourcecode-1.4.0 → sourcecode-1.6.0}/tests/test_contract_pipeline.py

@@ -188,7 +188,7 @@ def test_cli_raw_mode_preserves_standard_output(tmp_path: Path) -> None:
     data = json.loads(result.output)
     assert "metadata" in data
     assert data["metadata"]["schema_version"] == "1.0"
-    assert "file_contracts" not in data
+    assert "contracts" not in data
 
 
 def test_cli_max_symbols_flag(tmp_path: Path) -> None:
@@ -198,11 +198,11 @@ def test_cli_max_symbols_flag(tmp_path: Path) -> None:
             f"def func{i}a(): pass\ndef func{i}b(): pass\n"
         )
 
-    # Use --mode standard to get file_contracts with full per-symbol detail
+    # Use --mode standard to get contracts with full per-symbol detail
     result = runner.invoke(app, ["--mode", "standard", "--max-symbols", "5", str(tmp_path)])
     assert result.exit_code == 0, result.output
     data = json.loads(result.output)
-    contracts = data.get("file_contracts", [])
+    contracts = data.get("contracts", [])
     total = sum(
         len(c.get("exports", [])) + len(c.get("functions", [])) + len(c.get("types", []))
         for c in contracts
@@ -250,7 +250,7 @@ def test_cli_standard_mode_includes_detail_fields(tmp_path: Path) -> None:
     assert "schema_version" in data
     assert "stacks" in data
     assert "entry_points" in data
-    assert "file_contracts" in data
+    assert "contracts" in data
 
 
 def test_cli_invalid_mode_exits_nonzero(tmp_path: Path) -> None: