victor-codegraph 0.8.1__tar.gz → 0.10.0__tar.gz

victor_codegraph-0.10.0/PKG-INFO

@@ -0,0 +1,174 @@
+Metadata-Version: 2.4
+Name: victor-codegraph
+Version: 0.10.0
+Summary: Repository-to-Code-Semantic-Graph compiler with stable identity, bounded chunks, incremental deltas, and ProximaRecord projection.
+Author-email: Vijaykumar Singh <vijay@anvaiops.com>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/anvai-labs/victor
+Project-URL: Repository, https://github.com/anvai-labs/victor
+Keywords: code-graph,cpg,semantic-graph,chunking,tree-sitter,ast,embeddings,rag
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Provides-Extra: embeddings
+Requires-Dist: sentence-transformers<6,>=3; extra == "embeddings"
+Provides-Extra: treesitter
+Requires-Dist: tree-sitter>=0.26.0; extra == "treesitter"
+Requires-Dist: tree-sitter-language-pack>=1.21.3; extra == "treesitter"
+Provides-Extra: contracts
+Requires-Dist: victor-contracts<1.0,>=0.9.1; extra == "contracts"
+Provides-Extra: dev
+Requires-Dist: pytest>=9.1.1; extra == "dev"
+Requires-Dist: ruff>=0.16.7; extra == "dev"
+Requires-Dist: tree-sitter>=0.26.0; extra == "dev"
+Requires-Dist: tree-sitter-language-pack>=1.21.3; extra == "dev"
+
+# victor-codegraph
+
+Shared **repository → Code Semantic Graph compiler**: symbol and relation extraction,
+hard-bounded embeddable chunks, repository-wide resolution/deltas, diagnostics, and a
+`ProximaRecord` projection. One compiler, three
+consumers — Victor (owner), the ProximaDB SDK (`[codegraph]` extra), and AnvaiOps (SaaS
+code-graph vertical).
+
+> Design: ProximaDB `ADR-029` (authoritative) · Victor `ADR-014` (owner/donor) ·
+> AnvaiOps `ADR-0018` (consumer). This package is the **TD-CG1** scaffold.
+
+## Why
+
+The same tree-sitter code→symbol+relation chunker existed twice (ProximaDB SDK `code.py`
+and Victor `victor-coding`) and was about to be written a third time in AnvaiOps. This
+package is the single neutral home. It merges the best of both donors and fixes their two
+gaps:
+
+- **Size-capping** — ProximaDB's `code.py` emitted one chunk per symbol with *no* size
+  bound (a huge function became a huge chunk). Here, oversized symbols are body-split with
+  overlap (LlamaIndex `CodeSplitter` discipline). See `sizing.py`.
+- **Real JS/TS** — the donor JS/TS parser was a stub returning no symbols. Here JS/TS get a
+  real tree-sitter extractor (functions, classes, methods, `const … = () =>`, imports).
+
+## Install
+
+Published on PyPI as [`victor-codegraph`](https://pypi.org/project/victor-codegraph/).
+This source tree is version **0.10.0**; consumers should use a compatible `<1.0` pin.
+
+```bash
+# from PyPI
+pip install victor-codegraph                 # Python-only (stdlib ast) path, zero native deps
+pip install "victor-codegraph[treesitter]"   # + multi-language tree-sitter grammars
+
+# dev (monorepo): editable, with tree-sitter grammars + test tooling
+make -C victor-codegraph dev          # = pip install -e ../victor-contracts && pip install -e ".[dev]"
+pip install -e ./victor-codegraph     # minimal editable install
+```
+
+### Releasing
+
+CI: `.github/workflows/ci-codegraph.yml` runs the suite (editable install, grammars on) for every
+PR touching `victor-codegraph/**`. Publishing: push a tag `victor-codegraph-v0.1.0` to trigger
+`.github/workflows/release-codegraph.yml`, which builds and publishes via **PyPI Trusted Publishing**
+(OIDC — no API token). Configure the publisher once on PyPI (owner `vjsingh1984`, repo `victor`,
+workflow `release-codegraph.yml`, environments `pypi` / `testpypi`); see the header of that workflow.
+
+## 0.9 consumer qualification and migration
+
+| Surface | Contract / limit |
+|---|---|
+| Python references | Exact import aliases and dotted modules; lexical scope precedes arity. Unknown receivers, wildcard ambiguity and shadowed bindings remain explicit unresolved references. No runtime execution/type inference. |
+| ES modules | Relative named/default/namespace imports, explicit named re-exports and exported arrow functions. Type-only imports produce dependencies, never runtime call evidence. Re-export traversal is cycle-safe and bounded to 64 bindings. Package/tsconfig aliases, star exports and dynamic exports remain unresolved. |
+| Object-literal identity | Methods include structural object scopes, including anonymous callback ancestry. Whitespace/comments and checkout root do not affect IDs; inserting/reordering anonymous siblings can change structural ordinals. |
+| Source evidence | Python call spans include UTF-8 byte offsets/lengths and end coordinates. New binding metadata is diagnostic evidence, not runtime authorization or a complete language semantic model. |
+| Victor persistence | The canonical snapshot supplies symbols. CLI distinguishes attempted upserts from persisted totals, exposes partial parses and accepts exact opaque IDs; ambiguous display names are not chosen arbitrarily. Default file exclusions can differ from the package walker. |
+
+Rebuild derived indexes after upgrading from older parser versions; an unchanged
+source hash alone does not migrate older IDs/edges. Victor users can run
+`victor graph index --force --no-ccg --no-embeddings`. Consumers must stage a new
+complete generation and switch readers after validation; never mix old and new
+symbol identities. The AnvaiOps adapter/generation migration remains consumer-owned.
+The package adds no commercial runtime, account, entitlement or billing dependency.
+
+
+## Use
+
+```python
+from victor_codegraph import chunk, parse, parse_repo, to_proxima_records, ChunkConfig
+
+# Size-capped, embeddable chunks:
+chunks = chunk(source, file_path="app/service.py", config=ChunkConfig(max_chunk_tokens=512))
+
+# Symbols + relations:
+parsed = parse(source, file_path="app/service.py")
+
+# Project to the ProximaDB substrate-keystone record shape (one symbol = row+node+vector):
+records = to_proxima_records(parsed, repo_graph_id="myrepo", branch_id="main",
+                             embedder=my_embed_fn)  # embedder optional
+
+# Repository-wide resolution + file/import/containment graph:
+repository = parse_repo(".", repo_id="myrepo")
+records = to_proxima_records(repository, repo_graph_id="myrepo")
+```
+
+## Design principles (the "best posture" this encodes)
+
+1. Chunk at **symbol** granularity (not statement, not fixed-size).
+2. **Newline-preferred and hard-bounded** — forced character splits are explicit; an optional
+   tokenizer callback enforces the real model limit.
+3. Extract a **Code Semantic Graph** (CALLS/EXTENDS/IMPLEMENTS/CONTAINS/IMPORTS). Full
+   intra-procedural CFG/CDG/DDG is a separate optional layer, not implied here.
+4. **One structural v2 ID** across symbols, chunks, relations, and records; line IDs remain aliases.
+5. **Deletion-complete deltas** — incremental output can be checked against a full rebuild.
+6. **Observable fallback** — every parse reports status, capability tier, and diagnostics.
+7. **Explicit uncertainty** — unresolved targets become structured external-reference nodes.
+
+## Status
+
+`0.10.0` source version. Python (`ast`) is the full, offline frontend. JS/TS, Go, Rust,
+Java, C, and C++ have conformance fixtures; other mapped tree-sitter languages report
+symbols-tier or fallback status rather than implying full fidelity.
+
+### Rust graph fidelity
+
+Rust indexing retains inline module scopes and discriminates same-named trait
+implementations and conditional definitions in stable identities. Repository
+resolution follows declared library modules and explicit top-level `use` paths,
+including grouped imports, aliases, and local Cargo path dependencies. Hyphenated
+Cargo dependency names are normalized to their Rust underscore names.
+
+Binary-owned or ambiguous files, conditional items/imports, `#[path]` overrides,
+workspace-inherited dependencies, re-export chains, macros and unknown receiver
+types retain unresolved references. Glob imports retain a module dependency but
+do not guess call targets. This is a conservative static graph, not rustc name
+resolution or evidence that a call executes. Publish this package before a Victor
+release that requires its exact repository graph contract.
+
+
+### Embedding conformance (0.10.0)
+
+The accepted FEP-0040 supplies store-neutral
+`EmbeddingProfile`, `SymbolEmbedding`, `embed_symbols` and an optional
+`SentenceTransformerEncoder`. The base compiler stays dependency-free. The
+`embeddings` extra loads sentence-transformers only when explicitly selected;
+private source is not sent to a remote model by default.
+
+The contract uses canonical compiler IDs and `symbol-summary-v1` (name,
+signature and documentation), with an immutable model revision, dimensions,
+token limit, normalization, query prefix and separate input/source hashes.
+Inference errors propagate; hash placeholders, zero vectors and malformed batches
+are not semantic evidence. File/container symbols marked `embedding_excluded`
+remain graph evidence only.
+
+These APIs are introduced in 0.10.0 and are unavailable in 0.9.0. Install the
+exact published `victor-codegraph[treesitter,embeddings]==0.10.0` when selecting
+local model execution; the base package has no mandatory model dependency.
+Current evidence covers
+synthetic document-vector persistence parity and local LanceDB collection/query
+profile enforcement. Labeled relevance and hosted lifecycle remain separate
+acceptance gates. Victor binds a new LanceDB table to the complete profile in its
+Arrow schema and rejects incompatible query encoders and mixed/orphan row markers.
+Legacy tables require a fresh destination; they are never silently relabeled.

victor_codegraph-0.10.0/README.md

@@ -0,0 +1,143 @@
+# victor-codegraph
+
+Shared **repository → Code Semantic Graph compiler**: symbol and relation extraction,
+hard-bounded embeddable chunks, repository-wide resolution/deltas, diagnostics, and a
+`ProximaRecord` projection. One compiler, three
+consumers — Victor (owner), the ProximaDB SDK (`[codegraph]` extra), and AnvaiOps (SaaS
+code-graph vertical).
+
+> Design: ProximaDB `ADR-029` (authoritative) · Victor `ADR-014` (owner/donor) ·
+> AnvaiOps `ADR-0018` (consumer). This package is the **TD-CG1** scaffold.
+
+## Why
+
+The same tree-sitter code→symbol+relation chunker existed twice (ProximaDB SDK `code.py`
+and Victor `victor-coding`) and was about to be written a third time in AnvaiOps. This
+package is the single neutral home. It merges the best of both donors and fixes their two
+gaps:
+
+- **Size-capping** — ProximaDB's `code.py` emitted one chunk per symbol with *no* size
+  bound (a huge function became a huge chunk). Here, oversized symbols are body-split with
+  overlap (LlamaIndex `CodeSplitter` discipline). See `sizing.py`.
+- **Real JS/TS** — the donor JS/TS parser was a stub returning no symbols. Here JS/TS get a
+  real tree-sitter extractor (functions, classes, methods, `const … = () =>`, imports).
+
+## Install
+
+Published on PyPI as [`victor-codegraph`](https://pypi.org/project/victor-codegraph/).
+This source tree is version **0.10.0**; consumers should use a compatible `<1.0` pin.
+
+```bash
+# from PyPI
+pip install victor-codegraph                 # Python-only (stdlib ast) path, zero native deps
+pip install "victor-codegraph[treesitter]"   # + multi-language tree-sitter grammars
+
+# dev (monorepo): editable, with tree-sitter grammars + test tooling
+make -C victor-codegraph dev          # = pip install -e ../victor-contracts && pip install -e ".[dev]"
+pip install -e ./victor-codegraph     # minimal editable install
+```
+
+### Releasing
+
+CI: `.github/workflows/ci-codegraph.yml` runs the suite (editable install, grammars on) for every
+PR touching `victor-codegraph/**`. Publishing: push a tag `victor-codegraph-v0.1.0` to trigger
+`.github/workflows/release-codegraph.yml`, which builds and publishes via **PyPI Trusted Publishing**
+(OIDC — no API token). Configure the publisher once on PyPI (owner `vjsingh1984`, repo `victor`,
+workflow `release-codegraph.yml`, environments `pypi` / `testpypi`); see the header of that workflow.
+
+## 0.9 consumer qualification and migration
+
+| Surface | Contract / limit |
+|---|---|
+| Python references | Exact import aliases and dotted modules; lexical scope precedes arity. Unknown receivers, wildcard ambiguity and shadowed bindings remain explicit unresolved references. No runtime execution/type inference. |
+| ES modules | Relative named/default/namespace imports, explicit named re-exports and exported arrow functions. Type-only imports produce dependencies, never runtime call evidence. Re-export traversal is cycle-safe and bounded to 64 bindings. Package/tsconfig aliases, star exports and dynamic exports remain unresolved. |
+| Object-literal identity | Methods include structural object scopes, including anonymous callback ancestry. Whitespace/comments and checkout root do not affect IDs; inserting/reordering anonymous siblings can change structural ordinals. |
+| Source evidence | Python call spans include UTF-8 byte offsets/lengths and end coordinates. New binding metadata is diagnostic evidence, not runtime authorization or a complete language semantic model. |
+| Victor persistence | The canonical snapshot supplies symbols. CLI distinguishes attempted upserts from persisted totals, exposes partial parses and accepts exact opaque IDs; ambiguous display names are not chosen arbitrarily. Default file exclusions can differ from the package walker. |
+
+Rebuild derived indexes after upgrading from older parser versions; an unchanged
+source hash alone does not migrate older IDs/edges. Victor users can run
+`victor graph index --force --no-ccg --no-embeddings`. Consumers must stage a new
+complete generation and switch readers after validation; never mix old and new
+symbol identities. The AnvaiOps adapter/generation migration remains consumer-owned.
+The package adds no commercial runtime, account, entitlement or billing dependency.
+
+
+## Use
+
+```python
+from victor_codegraph import chunk, parse, parse_repo, to_proxima_records, ChunkConfig
+
+# Size-capped, embeddable chunks:
+chunks = chunk(source, file_path="app/service.py", config=ChunkConfig(max_chunk_tokens=512))
+
+# Symbols + relations:
+parsed = parse(source, file_path="app/service.py")
+
+# Project to the ProximaDB substrate-keystone record shape (one symbol = row+node+vector):
+records = to_proxima_records(parsed, repo_graph_id="myrepo", branch_id="main",
+                             embedder=my_embed_fn)  # embedder optional
+
+# Repository-wide resolution + file/import/containment graph:
+repository = parse_repo(".", repo_id="myrepo")
+records = to_proxima_records(repository, repo_graph_id="myrepo")
+```
+
+## Design principles (the "best posture" this encodes)
+
+1. Chunk at **symbol** granularity (not statement, not fixed-size).
+2. **Newline-preferred and hard-bounded** — forced character splits are explicit; an optional
+   tokenizer callback enforces the real model limit.
+3. Extract a **Code Semantic Graph** (CALLS/EXTENDS/IMPLEMENTS/CONTAINS/IMPORTS). Full
+   intra-procedural CFG/CDG/DDG is a separate optional layer, not implied here.
+4. **One structural v2 ID** across symbols, chunks, relations, and records; line IDs remain aliases.
+5. **Deletion-complete deltas** — incremental output can be checked against a full rebuild.
+6. **Observable fallback** — every parse reports status, capability tier, and diagnostics.
+7. **Explicit uncertainty** — unresolved targets become structured external-reference nodes.
+
+## Status
+
+`0.10.0` source version. Python (`ast`) is the full, offline frontend. JS/TS, Go, Rust,
+Java, C, and C++ have conformance fixtures; other mapped tree-sitter languages report
+symbols-tier or fallback status rather than implying full fidelity.
+
+### Rust graph fidelity
+
+Rust indexing retains inline module scopes and discriminates same-named trait
+implementations and conditional definitions in stable identities. Repository
+resolution follows declared library modules and explicit top-level `use` paths,
+including grouped imports, aliases, and local Cargo path dependencies. Hyphenated
+Cargo dependency names are normalized to their Rust underscore names.
+
+Binary-owned or ambiguous files, conditional items/imports, `#[path]` overrides,
+workspace-inherited dependencies, re-export chains, macros and unknown receiver
+types retain unresolved references. Glob imports retain a module dependency but
+do not guess call targets. This is a conservative static graph, not rustc name
+resolution or evidence that a call executes. Publish this package before a Victor
+release that requires its exact repository graph contract.
+
+
+### Embedding conformance (0.10.0)
+
+The accepted FEP-0040 supplies store-neutral
+`EmbeddingProfile`, `SymbolEmbedding`, `embed_symbols` and an optional
+`SentenceTransformerEncoder`. The base compiler stays dependency-free. The
+`embeddings` extra loads sentence-transformers only when explicitly selected;
+private source is not sent to a remote model by default.
+
+The contract uses canonical compiler IDs and `symbol-summary-v1` (name,
+signature and documentation), with an immutable model revision, dimensions,
+token limit, normalization, query prefix and separate input/source hashes.
+Inference errors propagate; hash placeholders, zero vectors and malformed batches
+are not semantic evidence. File/container symbols marked `embedding_excluded`
+remain graph evidence only.
+
+These APIs are introduced in 0.10.0 and are unavailable in 0.9.0. Install the
+exact published `victor-codegraph[treesitter,embeddings]==0.10.0` when selecting
+local model execution; the base package has no mandatory model dependency.
+Current evidence covers
+synthetic document-vector persistence parity and local LanceDB collection/query
+profile enforcement. Labeled relevance and hosted lifecycle remain separate
+acceptance gates. Victor binds a new LanceDB table to the complete profile in its
+Arrow schema and rejects incompatible query encoders and mixed/orphan row markers.
+Legacy tables require a fresh destination; they are never silently relabeled.

{victor_codegraph-0.8.1 → victor_codegraph-0.10.0}/pyproject.toml

@@ -1,20 +1,21 @@
 [build-system]
-requires = ["setuptools>=61.0", "wheel"]
+requires = ["setuptools>=84.0.0", "wheel"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "victor-codegraph"
-version = "0.8.1"
-description = "Code->CPG chunker: tree-sitter symbol + relation extraction, size-capped chunks, ProximaRecord projection. Shared by Victor, ProximaDB SDK, and AnvaiOps."
+version = "0.10.0"
+description = "Repository-to-Code-Semantic-Graph compiler with stable identity, bounded chunks, incremental deltas, and ProximaRecord projection."
 readme = "README.md"
-requires-python = ">=3.10"
+requires-python = ">=3.12"
 license = { text = "Apache-2.0" }
 authors = [
-    { name = "Vijaykumar Singh", email = "singhvjd@gmail.com" },
+    { name = "Vijaykumar Singh", email = "vijay@anvaiops.com" },
 ]
 keywords = [
     "code-graph",
     "cpg",
+    "semantic-graph",
     "chunking",
     "tree-sitter",
     "ast",
@@ -26,9 +27,8 @@ classifiers = [
     "Intended Audience :: Developers",
     "License :: OSI Approved :: Apache Software License",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.10",
-    "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
     "Topic :: Software Development :: Libraries",
 ]
 
@@ -40,18 +40,19 @@ classifiers = [
 dependencies = []
 
 [project.optional-dependencies]
+embeddings = ["sentence-transformers>=3,<6"]
 treesitter = [
-    "tree-sitter>=0.23",
-    "tree-sitter-language-pack>=1.0",
+    "tree-sitter>=0.26.0",
+    "tree-sitter-language-pack>=1.21.3",
 ]
 contracts = [
-    "victor-contracts>=0.7.0,<1.0",
+    "victor-contracts>=0.9.1,<1.0",
 ]
 dev = [
-    "pytest>=8.0",
-    "ruff>=0.5",
-    "tree-sitter>=0.23",
-    "tree-sitter-language-pack>=1.0",
+    "pytest>=9.1.1",
+    "ruff>=0.16.7",
+    "tree-sitter>=0.26.0",
+    "tree-sitter-language-pack>=1.21.3",
 ]
 
 [project.urls]
@@ -64,7 +65,7 @@ include = ["victor_codegraph*"]
 
 [tool.ruff]
 line-length = 100
-target-version = "py310"
+target-version = "py312"
 
 # Pin the rule set explicitly: the CI workflow installs the latest ruff, whose
 # evolving defaults must not be able to red a path-filtered pipeline overnight.
@@ -77,3 +78,6 @@ select = ["E4", "E7", "E9", "F"]
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 addopts = ""
+
+[tool.setuptools.package-data]
+victor_codegraph = ["py.typed"]

victor_codegraph-0.10.0/tests/test_adapter_validation_v2.py

@@ -0,0 +1,42 @@
+from __future__ import annotations
+
+import pytest
+
+from victor_codegraph import parse, to_proxima_records
+
+
+def test_batch_embedder_cardinality_mismatch_fails():
+    parsed = parse("def a(): pass\ndef b(): pass\n", file_path="m.py")
+    with pytest.raises(ValueError, match="one vector per symbol"):
+        to_proxima_records(parsed, "repo", batch_embedder=lambda texts: [[0.0] * 3], dim=3)
+
+
+def test_embedding_dimension_mismatch_fails():
+    parsed = parse("def a(): pass\n", file_path="m.py")
+    with pytest.raises(ValueError, match="dimension"):
+        to_proxima_records(parsed, "repo", embedder=lambda text: [0.0, 1.0], dim=3)
+
+
+def test_non_finite_embedding_fails_before_storage():
+    parsed = parse("def a(): pass\n", file_path="m.py")
+    with pytest.raises(ValueError, match="finite numeric"):
+        to_proxima_records(parsed, "repo", embedder=lambda text: [0.0, float("nan")], dim=2)
+
+
+def test_repeated_calls_are_one_edge_with_all_call_sites():
+    parsed = parse("def a():\n    b()\n    b()\n\ndef b(): pass\n", file_path="m.py")
+    records = to_proxima_records(parsed, "repo")
+    calls = [
+        r for r in records if "graph_edge" in r["labels"] and r["edge"]["edge_type"] == "CALLS"
+    ]
+    assert len(calls) == 1
+    assert [site["line"] for site in calls[0]["props"]["call_sites"]] == [2, 3]
+
+
+def test_legacy_oid_mode_keeps_edge_endpoints_attached():
+    parsed = parse("def a(): return b()\n\ndef b(): return 1\n", file_path="m.py")
+    records = to_proxima_records(parsed, "repo", stable_oid=False)
+    nodes = {r["oid"] for r in records if "graph_node" in r["labels"]}
+    edge = next(r for r in records if "graph_edge" in r["labels"])
+    assert edge["edge"]["from_oid"] in nodes
+    assert edge["edge"]["to_oid"] in nodes

victor_codegraph-0.10.0/tests/test_chunk_contract_v2.py

@@ -0,0 +1,75 @@
+from __future__ import annotations
+
+import pytest
+
+from victor_codegraph import ChunkConfig, chunk
+
+
+def test_fallback_caps_a_single_oversized_line():
+    config = ChunkConfig(max_chunk_tokens=10, chunk_overlap_tokens=2, chars_per_token=1)
+    chunks = chunk("x" * 100, language="text", file_path="notes.txt", config=config)
+    assert len(chunks) > 1
+    assert all(len(c.text) <= config.max_chunk_chars for c in chunks)
+
+
+def test_python_chunks_report_exact_nonzero_byte_spans():
+    source = "prefix = 1\n\ndef f():\n    return 1\n"
+    function = next(
+        c for c in chunk(source, file_path="m.py") if c.metadata.get("simple_name") == "f"
+    )
+    assert function.start_pos == source.index("def f")
+    assert function.end_pos == function.start_pos + len(function.text.encode())
+
+
+def test_chunking_preserves_module_level_semantic_content_without_container_duplication():
+    source = """import os
+GLOBAL = configure()
+
+class Service:
+    def run(self):
+        return work()
+
+main()
+"""
+    chunks = chunk(source, file_path="m.py", config=ChunkConfig(max_chunk_tokens=1000))
+    text = "\n".join(c.text for c in chunks)
+    assert "import os" in text
+    assert "GLOBAL = configure()" in text
+    assert "main()" in text
+    assert sum(c.text.count("return work()") for c in chunks) == 1
+
+
+def test_include_tests_and_language_filters_are_effective():
+    assert (
+        chunk(
+            "def test_it():\n    pass\n",
+            file_path="test_demo.py",
+            config=ChunkConfig(include_tests=False),
+        )
+        == []
+    )
+    assert (
+        chunk(
+            "def f():\n    pass\n",
+            file_path="m.py",
+            config=ChunkConfig(languages=["rust"]),
+        )
+        == []
+    )
+
+
+def test_custom_token_counter_is_a_hard_limit():
+    source = "def f():\n" + "\n".join(f"    value_{i} = {i}" for i in range(30))
+    config = ChunkConfig(
+        max_chunk_tokens=8,
+        chunk_overlap_tokens=0,
+        token_counter=lambda text: len(text.split()),
+    )
+    chunks = chunk(source, file_path="m.py", config=config)
+    assert all(config.estimate_tokens(c.text) <= config.max_chunk_tokens for c in chunks)
+
+
+def test_impossible_token_counter_fails_instead_of_emitting_over_budget_chunk():
+    config = ChunkConfig(max_chunk_tokens=1, token_counter=lambda text: 2 if text else 0)
+    with pytest.raises(ValueError, match="cannot fit"):
+        chunk("x", language="text", file_path="notes.txt", config=config)

victor_codegraph-0.10.0/tests/test_diagnostics_v2.py

@@ -0,0 +1,26 @@
+from __future__ import annotations
+
+from victor_codegraph import CapabilityTier, ChunkConfig, ParseStatus, parse
+
+
+def test_syntax_error_reports_fallback_diagnostic():
+    parsed = parse("def broken(:\n", file_path="bad.py")
+    assert parsed.status == ParseStatus.FALLBACK
+    assert parsed.diagnostics
+    assert parsed.diagnostics[0].code == "syntax_error"
+
+
+def test_unknown_language_reports_fallback_instead_of_silent_success():
+    parsed = parse("plain text", language="unknown", file_path="x.unknown")
+    assert parsed.status == ParseStatus.FALLBACK
+    assert parsed.capability_tier == CapabilityTier.FALLBACK
+    assert parsed.diagnostics
+
+
+def test_extract_relations_config_is_effective():
+    parsed = parse(
+        "def f():\n    return g()\n",
+        file_path="m.py",
+        config=ChunkConfig(extract_relations=False),
+    )
+    assert parsed.relations == []