sourcecode 2.3.0__tar.gz → 2.5.0__tar.gz

{sourcecode-2.3.0 → sourcecode-2.5.0}/CHANGELOG.md

@@ -1,5 +1,111 @@
 # Changelog
 
+## [2.5.0] — 2026-07-13
+
+**Semantic Retrieval Engine v1 — a declarative, agent-ready query layer over ASK's
+already-built knowledge.** Retrieval turns *"ask a typed question of the repository"*
+into a first-class, reusable operation (Layer 3.5, above the Knowledge Layer, below any
+presentation). It **queries** existing structured knowledge — it never extracts, parses,
+or computes new evidence: every answer is a composition of existing engines acquired
+through one shared, cached `KnowledgeContext`. Frozen as v1 in ADR-0004.
+
+### Added
+
+- **Semantic Retrieval Engine** (`src/sourcecode/retrieval/`): a typed `RetrievalRequest`
+  → `IntentResolver` → `QueryPlanner` → declarative `KnowledgeQuery` (a closed, serializable
+  Step algebra) → generic `QueryPlanRuntime` → structured `KnowledgeResult` (entities,
+  evidence, observations, confidence, structured explanation, provenance, executed plan).
+  Adding an intent is enum + plan + registered steps — the generic Runtime is unchanged;
+  this held across 11 phases and **29 intents / 9 domains**.
+- **29 experimental `ask retrieve …` intents** across Architecture, Impact, Transactions,
+  Security, Interfaces/Validation, Integrations, Subsystems, Dependencies, and Graph — each
+  a pure composition of an existing engine (ArchetypeClassifier, `compute_blast_radius`,
+  `SpringSemanticModel`, `run_tx_audit`, `run_security_audit`, `build_validation_surface`,
+  `SemanticIntegrationEngine`, `run_impact_chain`, the module graph). Cross-pack step reuse
+  (`ResolveTarget`, `ResolveEndpointRef`, `ResolveScope`, generic `BuildResult`).
+- **Consolidation (B1–B4):** lazy SourceMap acquisition; per-acquisition timing surfaced in
+  the runtime report; impact-pack evidence parity; the `limit` option activated (default
+  unbounded). `ADR-0004` + `DESIGN-knowledge-provider-integration.md` document the frozen
+  contract and the pattern for adding future knowledge providers.
+- **User guide — architecture chapters (EN + ES):** *How ASK works*, *The semantic engine*,
+  *Architectural intelligence*, *AI workflows* — the Semantic IR / Context Graph / Knowledge
+  Layer / evidence model that the guides previously did not cover.
+
+### Fixed
+
+- **Cold-path performance / apparent hang:** `ContextGraph.relations()`/`symbols()` were
+  re-converting every edge and re-sorting every node on each call, and per-file consumers
+  call them once per file — an O(files × edges) blow-up that made cold `ask --agent` take
+  minutes on large repositories. Memoizing the projections cut jobrunr cold ~21 s → ~4 s and
+  BroadleafCommerce cold `--agent` from a multi-minute hang to ~54 s. Benefits every command
+  that reads the graph.
+- **`archetype` library de-bias (P0):** consumable libraries were confidently labeled
+  `engine` (their internals are engine-like). Non-runnable artifacts (no app/server/servlet
+  entry) are now correctly distinguished from runnable engines — jobrunr `engine(high)` →
+  `library(low)`; real engines (Neo4j) unchanged. Implemented as transparent, auditable
+  evidence.
+- **`fix-bug` argument consistency (P0):** now accepts the sibling-consistent
+  `fix-bug <symptom> <path>` form (mirroring `impact <target> <path>`) instead of failing
+  with a raw usage error; `--symptom` and path-only forms remain unchanged.
+
+### Removed
+
+- All references to a proprietary third-party security annotation were scrubbed from code,
+  tests, and docs (ASK references nothing vendor-proprietary).
+
+## [2.4.0] — 2026-07-12
+
+**AI Context Cache — a provider-agnostic cache for ASK's reusable Knowledge
+Context.** This is *not* a prompt cache: it never stores prompts, generated text,
+or LLM responses. It caches the structured knowledge ASK derives from source —
+so expensive commands stop re-parsing the same repository on every run. The cache
+belongs entirely to ASK and is consumed identically by any AI provider (GPT,
+Claude, Gemini, local models) or none at all; the LLM never participates in
+producing or reading it.
+
+### Added
+
+- **`context_cache.py` — the AI Context Cache component.** Operates at the
+  *knowledge* level: it caches the reusable Canonical IR (the expensive Java
+  parse) keyed **only by knowledge state**, never by command or prompt. The key
+  is `worktree_signature + KL_SCHEMA_VERSION + PRODUCER_VERSION + scope`, where
+  `worktree_signature` is the committed HEAD folded with a hash of the working-tree
+  diff when dirty — so a new commit *or* an uncommitted edit deterministically
+  invalidates the entry rather than serving stale context. Invalidation is
+  state-based, never TTL-based for correctness (TTL and a max-entry cap exist only
+  as cleanup). Entries are gzip'd, inspectable envelopes (no opaque blobs);
+  persistence survives across executions under `~/.sourcecode/context-cache/`.
+- **`get_or_build_cir()` — the reusable-knowledge entry point.** Builds the
+  Canonical IR once and, on a hit, rebuilds the full CIR cheaply from the cached
+  raw IR via `ir_dict_to_canonical` — skipping the parse entirely, with the
+  `cir_hash` verified equal so the rebuilt CIR is provably faithful. Because the
+  key is command-agnostic, every command that needs a CIR shares one entry: once
+  wired, `impact` / `review-pr` / `modernize` reuse it for free.
+- **`CachedContext`** — an inspectable structured record (`metadata`, `entities`,
+  `evidence`, `observations`, `summaries`, `graph_metrics`, `payload`,
+  `timestamps`, `producer_version`). Advisory metrics are tracked from day one:
+  hits, misses, invalidations, stores, hit ratio, average lookup/build time, bytes
+  stored, contexts reused.
+- **`ask cache context-stats` / `ask cache context-clear`** — observability and
+  management for the context cache, kept distinct from the existing snapshot-cache
+  `cache status` / `cache clear` so neither clobbers the other.
+- **`ask explain` is the first consumer.** It fetches its CIR through
+  `get_or_build_cir` and derives the class explanation cheaply from the shared
+  knowledge; each run reports `Context cache: HIT/MISS` with timing. Output is
+  byte-identical whether served from cache or a fresh build.
+
+### Notes
+
+- **Only `explain` is wired** — a deliberate reference integration to validate the
+  design before connecting `impact` / `onboard` / `prepare-context` / `archetype` /
+  `review-pr` / `modernize`. The SourceMap-based commands (`onboard`,
+  `prepare-context`, `archetype`) will use a separate cache scope.
+- No AI-provider-specific logic, no embeddings, no RAG, no prompt/response caching.
+- Env vars: `SOURCECODE_CONTEXT_CACHE_{DIR,DISABLE,TTL_DAYS,MAX_ENTRIES}`.
+- Tests: `tests/test_context_cache.py` adds 23 tests (hit/miss, commit/dirty/version
+  invalidation, persistence, corruption recovery, concurrency, TTL GC, CIR
+  round-trip fidelity, cross-command reuse). No report or behaviour cutover.
+
 ## [2.3.0] — 2026-07-12
 
 **Module graph promoted to a first-class Evidence Provider for architectural

{sourcecode-2.3.0 → sourcecode-2.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sourcecode
-Version: 2.3.0
+Version: 2.5.0
 Summary: Persistent structural context and ultra-fast repeated analysis for AI coding agents
 License-File: LICENSE
 Keywords: agents,ai,codebase,context,developer-tools,llm
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
 
 **Context · Impact · Migration · Architecture · Review — everything from one structural model.**
 
-![Version](https://img.shields.io/badge/version-2.3.0-blue)
+![Version](https://img.shields.io/badge/version-2.5.0-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 > **ASK Engine** is the product. The CLI command is **`ask`**. The legacy **`sourcecode`**
@@ -84,7 +84,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 2.3.0
+ask version                    # ask 2.5.0
 ```
 
 > **Package vs. command.** The install package is named `sourcecode` this release
@@ -218,7 +218,7 @@ repo root (otherwise they report `policy: "none_detected"`):
 ```json
 {
   "customSecurityAnnotations": [
-    { "fullyQualifiedName": "com.example.security.M3FiltroSeguridad", "shortName": "M3FiltroSeguridad" }
+    { "fullyQualifiedName": "com.example.security.CustomSecurityAnnotation", "shortName": "CustomSecurityAnnotation" }
   ]
 }
 ```

{sourcecode-2.3.0 → sourcecode-2.5.0}/README.md

@@ -4,7 +4,7 @@
 
 **Context · Impact · Migration · Architecture · Review — everything from one structural model.**
 
-![Version](https://img.shields.io/badge/version-2.3.0-blue)
+![Version](https://img.shields.io/badge/version-2.5.0-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 > **ASK Engine** is the product. The CLI command is **`ask`**. The legacy **`sourcecode`**
@@ -46,7 +46,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 2.3.0
+ask version                    # ask 2.5.0
 ```
 
 > **Package vs. command.** The install package is named `sourcecode` this release
@@ -180,7 +180,7 @@ repo root (otherwise they report `policy: "none_detected"`):
 ```json
 {
   "customSecurityAnnotations": [
-    { "fullyQualifiedName": "com.example.security.M3FiltroSeguridad", "shortName": "M3FiltroSeguridad" }
+    { "fullyQualifiedName": "com.example.security.CustomSecurityAnnotation", "shortName": "CustomSecurityAnnotation" }
   ]
 }
 ```

{sourcecode-2.3.0 → sourcecode-2.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sourcecode"
-version = "2.3.0"
+version = "2.5.0"
 description = "Persistent structural context and ultra-fast repeated analysis for AI coding agents"
 readme = "README.md"
 requires-python = ">=3.9"

{sourcecode-2.3.0 → sourcecode-2.5.0}/src/sourcecode/__init__.py

@@ -4,4 +4,4 @@ ASK Engine is the product. ``ask`` is the canonical CLI command; ``sourcecode``
 the legacy compatibility alias and the Python/PyPI package name. See
 docs/PRODUCT_IDENTITY.md (normative)."""
 
-__version__ = "2.3.0"
+__version__ = "2.5.0"

{sourcecode-2.3.0 → sourcecode-2.5.0}/src/sourcecode/archetype.py

@@ -80,6 +80,17 @@ _MIN_PRIMARY_SCORE = 0.05
 # A winner must clear this absolute score (not just the top-2 margin) to earn
 # medium/high confidence — a near-floor lone candidate stays "low".
 _MIN_CONFIDENT_SCORE = 0.5
+# Fraction of engine's positive contribution discounted for a non-runnable artifact
+# (P0 library de-bias): an engine-like internal core is expected of a library, so it is not
+# discriminating. Tuned so a clear library (no app/server/servlet entry) is not confidently
+# labeled an engine, while runnable systems (which never trigger the gate) are untouched.
+_LIBRARY_ENGINE_DISCOUNT = 0.55
+# Entry kinds that make a repo a runnable system (an app/server/servlet/daemon) rather
+# than a consumable library. Used to gate the library de-bias (P0).
+_RUNNABLE_ENTRY_KINDS = frozenset({
+    "application", "bootstrap", "main", "server", "daemon", "worker",
+    "mvc_controller", "jax_rs_controller", "servlet", "security_filter", "filter",
+})
 
 
 @dataclass
@@ -389,6 +400,41 @@ class ArchetypeClassifier:
             f"{f.module_count} modules; distributed mass ≈ {cm['distributed']:.0%}",
             1.5, multi, max(cm["distributed"], 0.1))
         self._apply_graph_evidence("software_archetype", cands, f)
+        # ── Library de-bias (P0) ────────────────────────────────────────────────
+        # A consumable JVM artifact — `jar` packaging with no server/bootstrap/application
+        # entry — is a library, not a runnable system. But a library naturally exhibits
+        # engine-like topology: a few core API types with high fan-in, tightly-coupled
+        # internals (SCCs), hub concentration. Those graph metrics (`graph_*`) otherwise
+        # label every library an "engine" (observed: jobrunr → engine(high), library 0.3).
+        # Correct it transparently: (1) add a packaging-grounded library signal that does
+        # not depend on an `api/` package existing; (2) append a NAMED negative evidence row
+        # that discounts most of engine's graph-derived score, since topological hubness in
+        # a library is expected and therefore not discriminating for "engine".
+        # A runnable/web entry of ANY kind (an app bootstrap, an HTTP controller, a
+        # servlet, a daemon) means the repo is a system you run, not a library you depend
+        # on. Packaging is unreliable here (gradle libs report no packaging; maven
+        # aggregators report `pom`), so the entry-kind surface is the robust signal.
+        is_runnable = any(k in f.entry_kinds for k in _RUNNABLE_ENTRY_KINDS)
+        is_library_shaped = not is_runnable
+        if is_library_shaped:
+            add("library", "no_runnable_entry",
+                f"entry kinds {sorted(f.entry_kinds) or 'none'} — no app/server/servlet "
+                "bootstrap; a consumable artifact, not a runnable system",
+                3.0, 1.0, 0.7)
+            # A library legitimately owns an engine-like core (storage/kernel mass, hub
+            # topology, SCCs) — that is what a library *is*. Discount most of engine's
+            # positive evidence for a non-runnable artifact so its internal machinery does
+            # not, alone, promote it to "engine". Transparent: a named negative row.
+            engine_positive = sum(max(0.0, e.contribution) for e in cands["engine"])
+            if engine_positive > 0:
+                discount = engine_positive * _LIBRARY_ENGINE_DISCOUNT
+                cands["engine"].append(Evidence(
+                    "library_core_discount",
+                    "non-runnable artifact: an engine-like internal core (storage/kernel "
+                    "mass, hubs, SCCs) is expected of a library and is not, alone, evidence "
+                    "of an engine product",
+                    1.0, 1.0, 1.0, -discount,
+                ))
         return self._resolve("software_archetype", cands, f)
 
     def _architectural_style(self, f: _Features) -> DimensionResult: