ai-docs-toolkit 0.3.3__tar.gz → 0.4.0__tar.gz

{ai_docs_toolkit-0.3.3 → ai_docs_toolkit-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ai-docs-toolkit
-Version: 0.3.3
+Version: 0.4.0
 Summary: AI-oriented documentation toolkit.
 Requires-Python: >=3.14
 Description-Content-Type: text/markdown
@@ -30,13 +30,14 @@ The core idea is that documentation should not be a secondary artifact created a
 
 ## Current Status
 
-Version 0.3.3 includes:
+Version 0.4.0 includes:
 
 - schema and structure validation for Markdown documents with YAML front matter;
 - document graph output;
 - impact analysis with linked test cases and changed-path discovery coverage;
 - optional saved review evidence and the read-only `ai-docs review-check` command;
 - an optional convention for documentation links beside key code;
+- the opt-in `ai-docs code-links-check` command for Python comment paths and primary IDs;
 - test-evidence review connecting requirements to assertions and proportionate sensitivity checks;
 - context bundle output;
 - an executable `ai-docs` CLI;
@@ -47,26 +48,41 @@ Version 0.3.3 includes:
 Install this version with Python 3.14 or newer once it is available on PyPI:
 
 ```bash
-python -m pip install ai-docs-toolkit==0.3.3
+python -m pip install ai-docs-toolkit==0.4.0
 ai-docs --version
 ```
 
 For ephemeral execution with `uvx`:
 
 ```bash
-uvx --python 3.14 --from ai-docs-toolkit==0.3.3 ai-docs --version
+uvx --python 3.14 --from ai-docs-toolkit==0.4.0 ai-docs --version
 ```
 
 For the optional local MCP runtime:
 
 ```bash
-python -m pip install "ai-docs-toolkit[mcp]==0.3.3"
+python -m pip install "ai-docs-toolkit[mcp]==0.4.0"
 ai-docs-mcp --project-root .
 ```
 
 For the latest stable release, use `python -m pip install --upgrade ai-docs-toolkit`
 without an exact version pin.
 
+## Release 0.4.0
+
+`ai-docs code-links-check FILE... --format json` checks documentation paths and
+primary IDs in real Python comments without importing code or changing files.
+Explicit files, repository boundaries and unverified inputs remain visible;
+strings/docstrings are ignored and links remain optional. Semantic agreement
+between documentation, comments, implementation and tests still requires review.
+The command does not add a gate to ordinary validation or new graph edges.
+
+After upgrading from 0.3.3, inspect `init --check` and `--dry-run`, then refresh the
+existing authoring reference with `init`. Preserve local rules, project documents
+and review history; no schema or bulk annotation migration is required. Minimal
+profile needs no reference refresh. See “Migration 0.3.3 To 0.4.0” in the packaged
+`ai_docs_toolkit/docs/agent-installation.md` for commands, checks and rollback.
+
 ## Release 0.3.3
 
 Documentation change review now traces material requirements to test inputs and
@@ -305,6 +321,12 @@ Check optional saved review evidence:
 ai-docs review-check review.yaml --format json
 ```
 
+Check existing documentation links in selected Python files:
+
+```bash
+ai-docs code-links-check src/example.py --format json
+```
+
 Prepare an agent context bundle for changed files:
 
 ```bash

{ai_docs_toolkit-0.3.3 → ai_docs_toolkit-0.4.0}/README.md

@@ -2,7 +2,7 @@
 
 AI Docs Toolkit is a documentation toolkit for AI-assisted software development.
 
-It treats documentation as an engineering interface between humans, AI agents, code, tests and change management. Source version 0.3.3 refines test-evidence review, alongside optional code documentation links, rollback guidance, linked test cases, changed-path coverage and optional review evidence checks. Python 3.14 or newer is required.
+It treats documentation as an engineering interface between humans, AI agents, code, tests and change management. Version 0.4.0 adds an optional checker for documentation IDs and paths in Python comments, alongside test-evidence review, rollback guidance, linked test cases, changed-path coverage and optional review evidence checks. Python 3.14 or newer is required.
 
 ## Purpose
 
@@ -83,9 +83,8 @@ pipx install ai-docs-toolkit
 uvx --from ai-docs-toolkit ai-docs validate
 ```
 
-For upgrades from 0.3.2 to 0.3.3, follow the [consumer migration guide](src/ai_docs_toolkit/docs/agent-installation.md#migration-032-to-033).
-Install the exact version only after it is available on PyPI; a source preparation
-commit does not publish the package. Consumers on 0.3.1 must also read the
+For upgrades from 0.3.3 to 0.4.0, follow the [consumer migration guide](src/ai_docs_toolkit/docs/agent-installation.md#migration-033-to-040).
+Consumers on 0.3.1 must also read the
 [0.3.2 migration](src/ai_docs_toolkit/docs/agent-installation.md#migration-031-to-032).
 Consumers on 0.3.0 must also read the
 [0.3.1 migration](src/ai_docs_toolkit/docs/agent-installation.md#migration-030-to-031).
@@ -97,8 +96,7 @@ Use Python 3.14+, review managed-file changes and preserve existing project rule
 and task records before refreshing. Completed standalone tasks are archived;
 tasks in a plan remain in place until the whole plan is archived.
 
-Version 0.3.3 is prepared in source and awaits publication approval; the latest
-published version remains 0.3.2. The
+Version 0.4.0 is prepared for publication; [0.3.3 remains available on Public PyPI](https://pypi.org/project/ai-docs-toolkit/0.3.3/). The
 [test-evidence review rule](.ai-docs/reference/documentation-change-review-skill.md#test-evidence-review)
 connects material requirements to test setup and assertions. A concrete risk of
 masked violations calls for a proportionate check within existing authority.
@@ -181,6 +179,15 @@ impact candidates. It does not approve semantic correctness. See the
 [review record contract](.ai-docs/reference/documentation-change-review-skill.md#advisory-checker)
 for YAML/Markdown input, tool identity and exit codes.
 
+Version 0.4.0 provides an optional check for existing Python code links:
+
+```bash
+ai-docs code-links-check src/example.py --format json
+```
+
+Run from the Git root with explicit files. This checks paths and primary IDs,
+without importing code or assessing semantic agreement. See the [code-link contract](.ai-docs/reference/documentation-cascade-authoring-skill.md#code-link-checker).
+
 Prepare an agent context bundle for changed files:
 
 ```bash

{ai_docs_toolkit-0.3.3 → ai_docs_toolkit-0.4.0}/packaging/pypi/README.md

@@ -16,13 +16,14 @@ The core idea is that documentation should not be a secondary artifact created a
 
 ## Current Status
 
-Version 0.3.3 includes:
+Version 0.4.0 includes:
 
 - schema and structure validation for Markdown documents with YAML front matter;
 - document graph output;
 - impact analysis with linked test cases and changed-path discovery coverage;
 - optional saved review evidence and the read-only `ai-docs review-check` command;
 - an optional convention for documentation links beside key code;
+- the opt-in `ai-docs code-links-check` command for Python comment paths and primary IDs;
 - test-evidence review connecting requirements to assertions and proportionate sensitivity checks;
 - context bundle output;
 - an executable `ai-docs` CLI;
@@ -33,26 +34,41 @@ Version 0.3.3 includes:
 Install this version with Python 3.14 or newer once it is available on PyPI:
 
 ```bash
-python -m pip install ai-docs-toolkit==0.3.3
+python -m pip install ai-docs-toolkit==0.4.0
 ai-docs --version
 ```
 
 For ephemeral execution with `uvx`:
 
 ```bash
-uvx --python 3.14 --from ai-docs-toolkit==0.3.3 ai-docs --version
+uvx --python 3.14 --from ai-docs-toolkit==0.4.0 ai-docs --version
 ```
 
 For the optional local MCP runtime:
 
 ```bash
-python -m pip install "ai-docs-toolkit[mcp]==0.3.3"
+python -m pip install "ai-docs-toolkit[mcp]==0.4.0"
 ai-docs-mcp --project-root .
 ```
 
 For the latest stable release, use `python -m pip install --upgrade ai-docs-toolkit`
 without an exact version pin.
 
+## Release 0.4.0
+
+`ai-docs code-links-check FILE... --format json` checks documentation paths and
+primary IDs in real Python comments without importing code or changing files.
+Explicit files, repository boundaries and unverified inputs remain visible;
+strings/docstrings are ignored and links remain optional. Semantic agreement
+between documentation, comments, implementation and tests still requires review.
+The command does not add a gate to ordinary validation or new graph edges.
+
+After upgrading from 0.3.3, inspect `init --check` and `--dry-run`, then refresh the
+existing authoring reference with `init`. Preserve local rules, project documents
+and review history; no schema or bulk annotation migration is required. Minimal
+profile needs no reference refresh. See “Migration 0.3.3 To 0.4.0” in the packaged
+`ai_docs_toolkit/docs/agent-installation.md` for commands, checks and rollback.
+
 ## Release 0.3.3
 
 Documentation change review now traces material requirements to test inputs and
@@ -291,6 +307,12 @@ Check optional saved review evidence:
 ai-docs review-check review.yaml --format json
 ```
 
+Check existing documentation links in selected Python files:
+
+```bash
+ai-docs code-links-check src/example.py --format json
+```
+
 Prepare an agent context bundle for changed files:
 
 ```bash

{ai_docs_toolkit-0.3.3 → ai_docs_toolkit-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ai-docs-toolkit"
-version = "0.3.3"
+version = "0.4.0"
 description = "AI-oriented documentation toolkit."
 readme = "packaging/pypi/README.md"
 requires-python = ">=3.14"

{ai_docs_toolkit-0.3.3 → ai_docs_toolkit-0.4.0}/src/ai_docs_toolkit/__init__.py

@@ -1,3 +1,3 @@
 """AI Docs Toolkit package baseline."""
 
-__version__ = "0.3.3"
+__version__ = "0.4.0"

{ai_docs_toolkit-0.3.3 → ai_docs_toolkit-0.4.0}/src/ai_docs_toolkit/cli/__init__.py

@@ -91,6 +91,7 @@ def _help_text() -> str:
             "  ai-docs tasks push --provider azure-devops (--task TASK-ID | --stage STAGE-ID | --plan PLAN-ID) --organization ORG --project PROJECT [--dry-run] [--write-back] [--format json]",
             "  ai-docs graph --format json [--type TYPE] [--module MODULE] [--id ID]",
             "  ai-docs impact (--id ID | --changed [--base REF]) [--format json]",
+            "  ai-docs code-links-check FILE... [--format human|json|--json]",
             "  ai-docs review-check RECORD [--record N] [--format json|--json]",
             "  ai-docs context (--id ID | --module MODULE | --feature FEATURE | --changed) [--format json] [--history exclude|include|legacy] [--body-budget-bytes N] [--depth N]",
             "  ai-docs mcp status [--project-root PATH] [--config FILE]",
@@ -117,6 +118,21 @@ def main(argv: list[str] | None = None) -> int:
         print(__version__)
         return 0
 
+    if args[0] == "code-links-check":
+        import json
+        import sys
+        from ai_docs_toolkit.core.code_links import check_code_links, format_code_links_human
+
+        parsed = _parse_code_links_args(args[1:])
+        if parsed is None:
+            print("Usage: ai-docs code-links-check FILE... [--format human|json|--json]", file=sys.stderr)
+            return 3
+        paths, output_format = parsed
+        result = check_code_links(paths, ".")
+        print(json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True)
+              if output_format == "json" else format_code_links_human(result))
+        return result["exit_code"]
+
     if args[0] == "review-check":
         import json
         import sys
@@ -1194,3 +1210,27 @@ def _parse_review_args(args):
             return None
         number = int(number)
     return args[1], number, 'json' if '--format' in options or '--json' in options else 'human'
+
+
+def _parse_code_links_args(args: list[str]) -> tuple[list[str], str] | None:
+    paths = []
+    output_format = None
+    index = 0
+    while index < len(args):
+        item = args[index]
+        if item in ("--json", "--format"):
+            if output_format is not None:
+                return None
+            if item == "--json":
+                output_format = "json"
+            else:
+                index += 1
+                if index >= len(args) or args[index] not in ("human", "json"):
+                    return None
+                output_format = args[index]
+        elif item.startswith("-"):
+            return None
+        else:
+            paths.append(item)
+        index += 1
+    return (paths, output_format or "human") if paths else None

ai_docs_toolkit-0.4.0/src/ai_docs_toolkit/core/code_links.py

@@ -0,0 +1,369 @@
+"""Opt-in, read-only checks for documentation links in Python comments."""
+
+from __future__ import annotations
+
+import io
+import os
+from pathlib import Path, PurePosixPath, PureWindowsPath
+import re
+import stat
+import subprocess
+import tokenize
+from typing import Any
+
+import yaml
+
+from ai_docs_toolkit.core.config import load_project_config
+from ai_docs_toolkit.core.frontmatter import parse_markdown_text
+from ai_docs_toolkit.core.registry import (
+    build_document_registry,
+    registry_document_from_parsed,
+)
+
+LIMITATIONS = [
+    "Only explicitly selected Python files are checked; no full source coverage is claimed.",
+    "Paths and primary IDs do not prove semantic consistency, implementation or test completeness.",
+    "Run against a stable tree; the scan is not an atomic snapshot or a concurrent-filesystem sandbox.",
+]
+_MARKER = re.compile(r"^#\s*Docs:")
+_LINK = re.compile(r"^#\s*Docs:\s*([^\s|]+)\s*\|\s*([^|]+?)\s*$")
+
+
+class _UniqueKeys(yaml.SafeLoader):
+    pass
+
+
+def _mapping(loader, node, deep=False):
+    loader.flatten_mapping(node)
+    result = {}
+    for key_node, value_node in node.value:
+        key = loader.construct_object(key_node, deep=deep)
+        if key in result:
+            raise ValueError("Duplicate YAML mapping key.")
+        result[key] = loader.construct_object(value_node, deep=deep)
+    return result
+
+
+_UniqueKeys.add_constructor(yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, _mapping)
+
+
+class _UnsupportedPath(ValueError):
+    pass
+
+
+def _local_path(root: Path, value: str) -> Path:
+    if (
+        not value
+        or any(ord(c) < 32 or ord(c) == 127 for c in value)
+        or "\\" in value
+        or PureWindowsPath(value).drive
+        or PurePosixPath(value).is_absolute()
+        or any(p in ("", ".", "..", ".git") for p in value.split("/"))
+    ):
+        raise _UnsupportedPath("Expected a canonical repository-relative POSIX path.")
+    current = root
+    for part in value.split("/"):
+        current = current / part
+        if current.is_symlink():
+            raise _UnsupportedPath("Symlink traversal is unsupported.")
+        if current.is_dir() and (
+            (current / ".git").exists() or (current / ".git").is_symlink()
+        ):
+            raise _UnsupportedPath("Path belongs to a nested Git repository.")
+    return current
+
+
+def _regular(path: Path) -> bool:
+    return stat.S_ISREG(path.stat().st_mode)
+
+
+# Check navigation only; semantic agreement remains a separate review.
+# Docs: REF-DOCUMENTATION-CASCADE-AUTHORING-SKILL-001 | .ai-docs/reference/documentation-cascade-authoring-skill.md
+def check_code_links(
+    paths: list[str], project_root: str | Path = "."
+) -> dict[str, Any]:
+    root = Path(project_root).resolve()
+    requested = sorted(set(paths))
+    observed: dict[str, Any] = dict(
+        requested_files=requested,
+        checked_files=[],
+        unverified_files=[],
+        documents_scanned=0,
+        links_found=0,
+        links=[],
+    )
+    findings: list[dict[str, Any]] = []
+    unavailable = False
+
+    def add(rule, message, path=None, line=None, document_id=None, target=None):
+        findings.append(
+            dict(
+                rule_id=rule,
+                severity="error",
+                message=message,
+                path=path,
+                line=line,
+                document_id=document_id,
+                target=target,
+            )
+        )
+
+    def finish(status=None):
+        observed["unverified_files"] = sorted(
+            set(requested) - set(observed["checked_files"])
+        )
+        ordered = sorted(
+            findings,
+            key=lambda f: (
+                f["path"] or "",
+                f["line"] or 0,
+                f["rule_id"],
+                f["document_id"] or "",
+                f["message"],
+            ),
+        )
+        status = status or (
+            "unavailable" if unavailable else "findings" if ordered else "checked"
+        )
+        code = {
+            "checked": 0,
+            "findings": 1,
+            "invalid_config": 2,
+            "usage_error": 3,
+            "unavailable": 4,
+        }[status]
+        observed["links_found"] = len(observed["links"])
+        return dict(
+            ok=code == 0,
+            status=status,
+            exit_code=code,
+            findings=ordered,
+            observed=observed,
+            limitations=LIMITATIONS.copy(),
+        )
+
+    if not requested:
+        add("code_link.arguments_invalid", "Supply at least one Python file.")
+        return finish("usage_error")
+    try:
+        git = subprocess.run(
+            ["git", "-C", str(root), "rev-parse", "--show-toplevel"],
+            env={**os.environ, "GIT_OPTIONAL_LOCKS": "0"},
+            capture_output=True,
+            text=True,
+            timeout=10,
+        )
+    except OSError, UnicodeError, subprocess.TimeoutExpired:
+        add("code_link.git_unavailable", "Git repository root could not be determined.")
+        return finish("unavailable")
+    if git.returncode or Path(git.stdout.rstrip("\n")).resolve() != root:
+        add("code_link.repository_root", "Run from the root of the Git repository.")
+        return finish("invalid_config")
+    try:
+        config_path = _local_path(root, "ai-docs.config.yaml")
+        if config_path.exists() and not _regular(config_path):
+            raise _UnsupportedPath("Configuration is not a regular file.")
+        # Validate original paths before the config loader resolves away symlinks.
+        raw_config = (
+            yaml.safe_load(config_path.read_bytes().decode("utf-8"))
+            if config_path.exists()
+            else {}
+        )
+        project = (
+            raw_config.get("project", {}) if isinstance(raw_config, dict) else None
+        )
+        roots = (
+            project.get("documentation_roots", ["ai-docs"])
+            if isinstance(project, dict)
+            else None
+        )
+        if isinstance(roots, list):
+            for value in roots:
+                if isinstance(value, str) and value != ".":
+                    _local_path(root, value)
+        loaded = load_project_config(root)
+    except yaml.YAMLError:
+        add(
+            "code_link.config_invalid",
+            "Invalid YAML configuration.",
+            "ai-docs.config.yaml",
+        )
+        return finish("invalid_config")
+    except (OSError, ValueError, UnicodeError, RecursionError) as exc:
+        add("code_link.config_unavailable", str(exc), "ai-docs.config.yaml")
+        return finish("unavailable")
+    if not loaded.ok:
+        for error in loaded.errors:
+            add("code_link.config_invalid", error.message, "ai-docs.config.yaml")
+        return finish("invalid_config")
+    config = loaded.config
+    assert config is not None
+
+    # Inventory without following symlinks; reuse primary/alias identity rules.
+    inventory: set[Path] = set()
+    visited: set[Path] = set()
+
+    def walk(directory: Path):
+        nonlocal unavailable
+        label = directory.relative_to(root).as_posix()
+        try:
+            if directory != root:
+                _local_path(root, label)
+            if directory in visited:
+                return
+            visited.add(directory)
+            for child in sorted(directory.iterdir()):
+                if child.name in config.excluded_directories:
+                    continue
+                name = child.relative_to(root).as_posix()
+                try:
+                    safe = _local_path(root, name)
+                    if safe.is_dir():
+                        walk(safe)
+                    elif safe.suffix.lower() == ".md":
+                        if not _regular(safe):
+                            raise _UnsupportedPath(
+                                "Markdown inventory entry is not a regular file."
+                            )
+                        inventory.add(safe)
+                except (OSError, ValueError) as exc:
+                    unavailable = True
+                    add("code_link.inventory_unavailable", str(exc), name)
+        except (OSError, ValueError) as exc:
+            unavailable = True
+            add("code_link.inventory_unavailable", str(exc), label)
+
+    for directory in config.documentation_roots:
+        try:
+            label = directory.relative_to(root).as_posix()
+            if directory != root:
+                _local_path(root, label)
+        except (OSError, ValueError) as exc:
+            unavailable = True
+            add("code_link.inventory_unavailable", str(exc))
+            continue
+        walk(directory)
+
+    documents = []
+    observed["documents_scanned"] = len(inventory)
+    for path in sorted(inventory):
+        label = path.relative_to(root).as_posix()
+        try:
+            text = path.read_bytes().decode("utf-8")
+        except (OSError, UnicodeError) as exc:
+            unavailable = True
+            add("code_link.document_unavailable", str(exc), label)
+            continue
+        try:
+            parsed = parse_markdown_text(text, path=str(path))
+        except RecursionError, ValueError, TypeError:
+            add("code_link.document_invalid", "Front matter cannot be parsed.", label)
+            continue
+        if not parsed.ok:
+            for error in parsed.errors:
+                add("code_link.document_invalid", error.message, label, error.line)
+            continue
+        assert parsed.document is not None
+        try:
+            yaml.load(parsed.document.raw_frontmatter, Loader=_UniqueKeys)
+        except yaml.YAMLError, ValueError, TypeError, RecursionError:
+            add(
+                "code_link.document_invalid",
+                "Front matter has invalid or duplicate mapping keys.",
+                label,
+            )
+            continue
+        documents.append(registry_document_from_parsed(parsed.document))
+    registry = build_document_registry(documents)
+    for issue in registry.issues:
+        for path in issue.paths:
+            add(
+                issue.rule_id,
+                issue.message,
+                path.relative_to(root).as_posix(),
+                document_id=issue.document_id,
+            )
+
+    for name in requested:
+        try:
+            source = _local_path(root, name)
+            if source.suffix != ".py" or not _regular(source):
+                raise _UnsupportedPath("Source must be a regular .py file.")
+            tokens = list(tokenize.tokenize(io.BytesIO(source.read_bytes()).readline))
+            if any(
+                t.type == tokenize.ERRORTOKEN and not t.string.isspace() for t in tokens
+            ):
+                raise ValueError("Invalid Python token.")
+        except (
+            OSError,
+            ValueError,
+            SyntaxError,
+            LookupError,
+            tokenize.TokenError,
+        ) as exc:
+            unavailable = True
+            observed["unverified_files"].append(name)
+            add("code_link.source_unavailable", str(exc), name)
+            continue
+        observed["checked_files"].append(name)
+        for token in tokens:
+            if token.type != tokenize.COMMENT or not _MARKER.match(token.string):
+                continue
+            match = _LINK.fullmatch(token.string)
+            if not match or not match[2].strip():
+                add(
+                    "code_link.marker_invalid",
+                    "Expected Docs: <primary ID> | <repository-relative path>.",
+                    name,
+                    token.start[0],
+                )
+                continue
+            document_id, target = match[1], match[2].strip()
+            link = dict(
+                path=name, line=token.start[0], document_id=document_id, target=target
+            )
+            observed["links"].append(link)
+            try:
+                target_path = _local_path(root, target)
+                if not target_path.exists():
+                    add(
+                        "code_link.target_missing",
+                        "Documentation target does not exist.",
+                        **link,
+                    )
+                    continue
+                if target_path not in inventory:
+                    raise _UnsupportedPath(
+                        "Target is not a Markdown file in the configured inventory."
+                    )
+            except (OSError, ValueError) as exc:
+                unavailable = True
+                add("code_link.target_unsupported", str(exc), **link)
+                continue
+            document = registry.get_by_path(target_path)
+            if document is None:
+                # Parsing/read findings already identify why this target is unverified.
+                continue
+            if document.document_id != document_id:
+                add(
+                    "code_link.id_mismatch",
+                    f"Target primary ID is {document.document_id!r}; comment has {document_id!r}.",
+                    **link,
+                )
+    return finish()
+
+
+def format_code_links_human(result: dict[str, Any]) -> str:
+    observed = result["observed"]
+    lines = [
+        f"Code links: {result['status']} (exit {result['exit_code']})",
+        f"Files checked: {len(observed['checked_files'])}/{len(observed['requested_files'])}; "
+        f"links: {observed['links_found']}; unverified files: {len(observed['unverified_files'])}",
+    ]
+    for finding in result["findings"]:
+        location = finding["path"] or "(project)"
+        if finding["line"] is not None:
+            location += f":{finding['line']}"
+        lines.append(f"- {location} {finding['rule_id']}: {finding['message']}")
+    lines.extend(["Limitations:", *("- " + text for text in result["limitations"])])
+    return "\n".join(lines)