rm-contextos 0.1.0__tar.gz

rm_contextos-0.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,32 @@
+---
+name: Bug report
+about: Report a reproducible problem in ContextOS
+title: "[Bug]: "
+labels: bug
+assignees: ""
+---
+
+## What happened?
+
+
+## Steps to reproduce
+
+1. 
+2. 
+3. 
+
+## Expected behavior
+
+
+## Environment
+
+- ContextOS version:
+- Python version:
+- OS:
+- Install method:
+
+## Logs or output
+
+```text
+
+```

rm_contextos-0.1.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+blank_issues_enabled: true

rm_contextos-0.1.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: Feature request
+about: Suggest an improvement for ContextOS
+title: "[Feature]: "
+labels: enhancement
+assignees: ""
+---
+
+## Problem
+
+
+## Proposed solution
+
+
+## Alternatives considered
+
+
+## Additional context
+

rm_contextos-0.1.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,17 @@
+## Summary
+
+- 
+
+## Validation
+
+- [ ] `pytest`
+- [ ] `ruff check .`
+- [ ] `mypy .`
+- [ ] `ruff format --check contextos/ tests/`
+
+## Safety Checklist
+
+- [ ] No unrelated behavior changes.
+- [ ] Secret handling remains safe by default.
+- [ ] Filesystem reads/writes are scoped and intentional.
+- [ ] Documentation or changelog updated when user-visible behavior changes.

rm_contextos-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,42 @@
+name: CI
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+jobs:
+  test:
+    name: Test (Python ${{ matrix.python-version }}, ${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+        os: [ubuntu-latest, macos-latest]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]"
+
+      - name: Lint (ruff)
+        run: ruff check contextos/ tests/
+
+      - name: Format check (ruff)
+        run: ruff format --check contextos/ tests/
+
+      - name: Type check (mypy)
+        run: mypy contextos/
+
+      - name: Run tests
+        run: pytest --cov=contextos --cov-report=term-missing --cov-fail-under=80

rm_contextos-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,45 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  build:
+    name: Build distribution
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install build
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Upload dist artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    name: Publish to PyPI
+    needs: build
+    runs-on: ubuntu-latest
+    steps:
+      - name: Download dist artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          password: ${{ secrets.PYPI_API_TOKEN }}

rm_contextos-0.1.0/.gitignore

@@ -0,0 +1,15 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.venv/
+venv/
+.coverage
+.coverage.*
+htmlcov/
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.contextos/
+*.md.bak

rm_contextos-0.1.0/ARCHITECTURE.md

@@ -0,0 +1,257 @@
+# ContextOS Architecture
+
+## Overview
+
+ContextOS is a pipeline: a repository enters one end, a context pack exits the other. Each stage is a pure transformation — no side effects, no network, no mutation of the source repo.
+
+```
+repo path
+    │
+    ▼
+┌──────────┐
+│  Scanner │  walks files, respects .gitignore, classifies by language
+└──────────┘
+    │ FileIndex
+    ▼
+┌─────────────┐
+│ Intelligence│  extracts symbols, imports, call edges → RepoGraph
+└─────────────┘
+    │ RepoGraph
+    ▼
+┌──────────┐
+│ Selector │  takes task + budget → ranks and clips chunks
+└──────────┘
+    │ ContextBundle
+    ▼
+┌────────────┐
+│ Compressor │  optional Headroom pass — shrinks tokens further
+└────────────┘
+    │ CompressedBundle
+    ▼
+┌──────────┐
+│ Exporter │  formats for target agent
+└──────────┘
+    │
+    ▼
+context pack (file on disk)
+```
+
+## Data Models (`models.py`)
+
+```python
+@dataclass
+class FileNode:
+    path: Path           # absolute path
+    rel_path: Path       # relative to repo root
+    language: str        # "python", "typescript", etc.
+    size_bytes: int
+    lines: int
+    content: str
+    tokens: int          # estimated at scan time
+
+@dataclass
+class ChunkNode:
+    file: FileNode
+    start_line: int
+    end_line: int
+    chunk_type: str      # "function", "class", "module", "block"
+    name: str | None     # symbol name if extractable
+    content: str
+    tokens: int
+    score: float         # set by Selector; 0.0 at creation
+
+@dataclass
+class RepoGraph:
+    files: list[FileNode]
+    chunks: list[ChunkNode]
+    imports: dict[Path, list[Path]]    # file → files it imports
+    symbols: dict[str, list[ChunkNode]] # symbol name → defining chunks
+    call_edges: list[tuple[str, str]]  # (caller_symbol, callee_symbol)
+
+@dataclass
+class ContextBundle:
+    task: str
+    repo_root: Path
+    chunks: list[ChunkNode]           # selected, ordered
+    total_tokens: int
+    budget: int
+    strategy: str
+    created_at: str                   # ISO 8601, deterministic
+
+@dataclass
+class CompressedBundle:
+    bundle: ContextBundle
+    compressed_content: str
+    original_tokens: int
+    compressed_tokens: int
+    compression_ratio: float
+```
+
+## Module Breakdown
+
+### `scanner/`
+
+**`walker.py`** — `RepoWalker`
+- Walks a directory tree using `pathlib.Path.rglob`
+- Reads `.gitignore` via `gitignore-parser` (or built-in simple parser)
+- Skips: binary files, files over a size limit (default 512 KB), lock files, build artifacts
+- Returns an ordered list of `FileNode` (ordered by `rel_path` for determinism)
+
+**`classifier.py`** — `LanguageClassifier`
+- Maps file extensions to language names
+- Detects shebangs for extensionless scripts
+- Returns language string or `"unknown"`
+
+**`index.py`** — `FileIndex`
+- Thin wrapper over `list[FileNode]`
+- Provides lookup by path, by language, by glob pattern
+- Computes aggregate stats (total files, total tokens, language breakdown)
+
+### `intelligence/`
+
+**`chunker.py`** — `Chunker`
+- Splits files into `ChunkNode` objects
+- Two modes:
+  - **AST mode** (tree-sitter, optional): function/class/method-level chunks
+  - **Line mode** (default): fixed-size overlapping windows (e.g. 50 lines, 10-line overlap)
+- Chunk size is bounded: minimum 5 lines, maximum 200 lines
+- Always produces at least one chunk per file (the whole file if small)
+
+**`extractor.py`** — `SymbolExtractor`
+- Runs in AST mode only
+- Extracts: function names, class names, import statements, exported symbols
+- Falls back to regex-based extraction for unsupported languages
+
+**`graph.py`** — `GraphBuilder`
+- Builds `RepoGraph` from `FileIndex` + extracted symbols
+- Import resolution: converts `from foo.bar import baz` → `Path` edges
+- Does not execute code; static analysis only
+- Cycle detection: records import cycles as metadata, does not fail on them
+
+### `selector/`
+
+**`ranker.py`** — `Ranker`
+- Assigns a relevance score to each `ChunkNode` given a task string
+- Scoring is additive; no single signal dominates:
+
+| Signal | Weight | Description |
+|--------|--------|-------------|
+| Keyword overlap | 0.40 | Task tokens that appear in chunk content |
+| Symbol match | 0.25 | Task tokens matching extracted symbol names |
+| Import centrality | 0.20 | Files imported by many others score higher |
+| Recency proxy | 0.15 | Files modified recently score slightly higher |
+
+- Scores are normalized to [0.0, 1.0]
+- Scoring is purely local: no embeddings, no LLM calls
+
+**`budget.py`** — `BudgetEnforcer`
+- Takes ranked `ChunkNode` list and a token budget
+- Greedy selection: add highest-scoring chunks until budget is exhausted
+- Two modes:
+  - `greedy`: pure score-descending fill
+  - `balanced`: ensures at least one chunk from each file that appears in the top N
+
+**`strategy.py`** — `SelectionStrategy`
+- Named presets that combine Ranker weights + BudgetEnforcer mode
+- Built-in: `default`, `deep-file` (fewer files, deeper), `wide-repo` (more files, shallower)
+- Custom strategies via TOML config
+
+### `compressor/`
+
+**`headroom.py`** — `HeadroomCompressor`
+- Optional; only active when `headroom-ai` is installed and proxy is reachable
+- Sends `ContextBundle` content to local Headroom proxy at `http://127.0.0.1:8787`
+- Returns `CompressedBundle` with compressed text and token delta
+- Fails gracefully: if proxy is unreachable, returns original bundle unchanged
+- No data leaves the machine; proxy is local
+
+### `exporter/`
+
+Each exporter takes a `ContextBundle | CompressedBundle` and returns a `str`.
+
+**`base.py`** — `BaseExporter` (abstract)
+- `render(bundle) -> str`
+- `filename() -> str`
+
+**`markdown.py`** — renders as fenced code blocks with file headers
+
+**`xml.py`** — renders as `<file path="...">` XML tags (Claude-style)
+
+**`claude.py`** — XML blocks + a task preamble formatted for Claude Code's system prompt
+
+**`codex.py`** — plain text blocks formatted for `AGENTS.md` injection
+
+**`cursor.py`** — splits into `.cursorrules` (repo summary) + `context.md` (file blocks)
+
+**`aider.py`** — tab-separated file list + content blocks for Aider's `--read` input
+
+**`json_.py`** — machine-readable JSON: metadata + chunks as structured objects
+
+### `cli.py`
+
+Built with Typer. Three top-level commands:
+
+```
+contextos scan   <repo>                          # index and summarize
+contextos pack   <repo> --task --budget --format # select and export
+contextos graph  <repo>                          # print import graph (optional)
+```
+
+All commands are read-only on the source repo. Output goes to stdout or `--out <path>`.
+
+## Token Counting
+
+Token counting uses a priority chain:
+
+1. `tiktoken` with the model's actual tokenizer (if installed and model known)
+2. `tiktoken` with `cl100k_base` as a universal approximation
+3. `len(content) // 4` as a last-resort fallback
+
+The counting method is reported in bundle metadata so consumers know which approximation was used.
+
+## Determinism Contract
+
+Given identical inputs (repo state, task string, budget, strategy), ContextOS always produces byte-for-byte identical output. This is guaranteed by:
+
+- File iteration order: `sorted(path.rglob(...))` — lexicographic, OS-independent
+- Chunk ordering: deterministic by `(file.rel_path, start_line)`
+- Score tie-breaking: ties broken by `(rel_path, start_line)` not by dict insertion order
+- Timestamps: `created_at` in output can be suppressed with `--no-timestamp` for reproducible diffs
+
+## Extension Points
+
+| Extension | Mechanism |
+|-----------|-----------|
+| New language chunker | Register in `chunker.py` dispatch table |
+| New ranking signal | Add signal function to `ranker.py`, assign weight |
+| New exporter | Subclass `BaseExporter`, register in `cli.py` |
+| Custom selection strategy | TOML config file, loaded by `strategy.py` |
+| Alternative compressor | Implement `Compressor` protocol in `compressor/` |
+
+## Security Model
+
+- **Read-only**: ContextOS never writes to the source repo
+- **No execution**: no `eval`, no subprocess of repo code, no dynamic imports from repo
+- **No network by default**: all network features gated behind explicit flags
+- **Path traversal**: all paths validated to stay within the declared repo root
+- **Output only to declared destinations**: `--out` flag or stdout; no implicit file writes
+
+## Dependencies
+
+### Required
+- `typer` — CLI
+- `pathlib` (stdlib) — path handling
+- `dataclasses` (stdlib) — models
+- `tomllib` (stdlib, Python 3.11+) — config parsing
+
+### Optional
+- `tiktoken` — accurate token counting
+- `tree-sitter` + language grammars — AST chunking
+- `gitignore-parser` — full `.gitignore` spec support
+- `headroom-ai` — compression
+
+### Dev only
+- `pytest` — tests
+- `pytest-cov` — coverage
+- `ruff` — lint + format
+- `mypy` — type checking

rm_contextos-0.1.0/AUDIT_REPORT.md

@@ -0,0 +1,200 @@
+# ContextOS Maintainer Audit Report
+
+Date: 2026-06-29
+
+Scope: strict open-source maintainer review of the Python package, Typer CLI, scanner,
+context pack builder, secret redaction, tests, packaging metadata, and README.
+
+## Executive Summary
+
+ContextOS is small, well-tested, and generally conservative about filesystem access. The
+main high-priority issue found during this audit was that the repository advertises a
+strict mypy configuration, but `mypy .` did not pass. That is a release/CI quality gate
+failure, especially for a typed CLI package with dynamic exporter and compression
+boundaries.
+
+I fixed only that high-priority issue. Lower-priority documentation and packaging gaps
+are recorded below but intentionally left unfixed.
+
+## High-Priority Findings Fixed
+
+### HP-1: Configured mypy check failed
+
+Severity: High
+
+Area: typing, CI/release confidence, dynamic plugin boundaries
+
+Evidence:
+
+- `pyproject.toml` enables `[tool.mypy] strict = true`.
+- Baseline `mypy .` failed with 27 errors across source and tests.
+- Source-level failures included untyped dynamic provider/exporter calls:
+  - `contextos/core/compression.py`
+  - `contextos/core/headroom_adapter.py`
+  - `contextos/cli/commands/export.py`
+  - `contextos/core/secret_detector.py`
+
+Impact:
+
+- Contributors cannot run the configured quality gate successfully.
+- Dynamic compression/exporter boundaries were opaque to mypy, which weakens confidence
+  in CLI code paths that are meant to be extensible.
+
+Fix:
+
+- Added narrow protocols/casts around dynamic exporter and Headroom imports.
+- Removed a dead helper in secret redaction.
+- Tightened test fixture/type annotations and corrected test imports that referenced a
+  helper through the wrong module.
+
+Status: Fixed. `mypy .` now passes.
+
+## Other Findings Not Fixed In This Pass
+
+### README project structure is stale
+
+Severity: Medium
+
+Area: README clarity
+
+Details:
+
+The README's `Project Structure` section lists an older layout such as `cli.py`,
+`scanner/`, `intelligence/`, `selector/`, `compressor/`, and `models.py`. The current
+repository uses `contextos/cli/commands/`, `contextos/core/`, and `contextos/exporters/`.
+
+Risk:
+
+New contributors will look for files that do not exist.
+
+Status: Not fixed. Documentation-only and not high priority for this pass.
+
+### MIT license declared but no LICENSE file present
+
+Severity: Medium
+
+Area: packaging, open-source hygiene
+
+Details:
+
+`pyproject.toml` declares `license = { text = "MIT" }`, but the repository does not
+include a `LICENSE` file.
+
+Risk:
+
+Package metadata is installable, but GitHub/open-source consumers do not get the full
+license text in the repository.
+
+Status: Not fixed. Adding license text should be done by the project owner with the
+correct copyright holder.
+
+### Package metadata is minimal
+
+Severity: Low
+
+Area: packaging
+
+Details:
+
+The package has name, version, description, readme, Python requirement, dependencies,
+and console script configured. It does not include classifiers, project URLs, authors,
+or keywords.
+
+Risk:
+
+Lower discoverability and weaker PyPI presentation, but no runtime breakage.
+
+Status: Not fixed. Not high priority.
+
+## Hardening Follow-Up
+
+Additional real-user hardening completed after the initial audit:
+
+- `.contextos/` is now always excluded from scans so generated packs and indexes do not
+  feed back into later scans or packs.
+- `contextos scan --max-files` now enforces the documented indexing limit instead of
+  only changing the displayed count.
+- Scanner relative paths are normalized to POSIX separators for stable output across
+  Windows, macOS, and Linux.
+- Generated/noisy notebook and minified asset files are skipped before content indexing.
+- `contextos pack` now regenerates scan data when `.contextos/file_summaries.json` or
+  `.contextos/dependency_graph.json` is corrupted or missing.
+- Test-file detection now handles Windows-style separators.
+
+Regression coverage added for empty repos, repos without README files, monorepo-shaped
+trees, binary files, large lockfiles, minified JS, notebooks, symlinks, permission errors,
+invalid Unicode, corrupted `.contextos`, repeated init/scan/pack workflows, Windows path
+separators, nested git metadata, and missing optional dependencies.
+
+### `--out` writes exactly the user-provided path
+
+Severity: Low
+
+Area: file handling
+
+Details:
+
+`contextos pack --out` and `contextos export ... --out` write to the provided path
+without creating parent directories and without restricting the path to the repository.
+
+Risk:
+
+This is normal CLI behavior for an explicit output flag, but errors may be less friendly
+when a parent directory is missing. It is not an unsafe implicit write because the user
+must provide `--out`.
+
+Status: Not fixed. Not high priority.
+
+## Areas Reviewed
+
+### CLI behavior
+
+- Typer command registration is straightforward.
+- Invalid repo and invalid budget paths return controlled `typer.Exit(code=1)`.
+- Tests cover command help, init, scan, task, memory, pack, export, invalid formats,
+  missing repos, and secret-related flags.
+
+### Unsafe file handling
+
+- Scanner uses `os.walk(..., followlinks=False)`.
+- Symlinks escaping the repo root are rejected.
+- Binary files, oversized files, permission errors, encoding errors, `.git`, virtualenvs,
+  build artifacts, and cache directories are skipped.
+- `init --force` overwrites `.contextos` templates, but that behavior is explicit and
+  documented.
+
+### Secret leakage risks
+
+- Secret-looking filenames are excluded from context selection.
+- Raw content is redacted before pack/export output unless `--allow-sensitive` is used.
+- `--allow-sensitive` is visibly marked dangerous.
+- Memory and decision commands reject note text that looks like embedded credentials.
+
+### Platform issues
+
+- The code primarily uses `pathlib`, UTF-8 text IO, and Typer/Rich.
+- Windows-specific chmod semantics are skipped in tests.
+- No high-priority Mac/Linux/Windows runtime issue was found in the reviewed code.
+
+### Abstractions and complexity
+
+- The separation between scanner, summarizer, selector, pack builder, exporters, and CLI
+  commands is reasonable for the package size.
+- Dynamic exporter/provider loading needed clearer typing; fixed.
+- No large unnecessary abstraction was introduced in this pass.
+
+## Validation
+
+Commands run after fixes:
+
+```bash
+.venv/bin/pytest
+.venv/bin/ruff check .
+.venv/bin/mypy .
+```
+
+Final status after hardening:
+
+- `pytest`: 907 passed, 1 skipped
+- `ruff`: passed
+- `mypy`: passed, 49 source files checked

rm_contextos-0.1.0/CHANGELOG.md

@@ -0,0 +1,33 @@
+# Changelog
+
+All notable changes to ContextOS are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2026-06-29
+
+### Added
+
+- Initial ContextOS CLI with `init`, `scan`, `task`, `memory`, `pack`, and `export` commands.
+- Deterministic repository scanning, summarization, dependency graphing, and task-aware context selection.
+- Context pack rendering for Markdown/JSON plus Claude Code, Codex, Cursor, and Aider exporters.
+- Secret filename exclusion and content redaction, with explicit `--allow-sensitive` override.
+- Optional Headroom compression provider integration.
+- Project memory and decision-log helpers under `.contextos/`.
+- CI, release documentation, community health files, and issue/PR templates.
+
+### Hardened
+
+- Exclude generated `.contextos/` outputs from future scans.
+- Enforce `scan --max-files` in written scan output.
+- Skip notebooks and minified generated assets by default.
+- Recover from corrupted `.contextos` scan data during pack generation.
+- Normalize internal scan paths for cross-platform stability.
+
+### Verified
+
+- `pytest`: 907 passed, 1 skipped.
+- `ruff check .`: passing.
+- `mypy .`: passing.
+- Local wheel build, install, and installed CLI smoke test.