lackpy 0.1.0__tar.gz

lackpy-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,57 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+    tags: ["v*"]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install
+        run: pip install -e ".[dev]"
+      - name: Run tests
+        run: pytest tests/ -v
+
+  build:
+    if: startsWith(github.ref, 'refs/tags/v')
+    needs: test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install build tools
+        run: pip install hatch
+      - name: Build package
+        run: hatch build
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - name: Download artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

lackpy-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.eggs/
+*.egg
+.pytest_cache/
+.mypy_cache/

lackpy-0.1.0/.readthedocs.yaml

@@ -0,0 +1,16 @@
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
+mkdocs:
+  configuration: mkdocs.yml
+
+python:
+  install:
+    - method: pip
+      path: .
+      extra_requirements:
+        - docs

lackpy-0.1.0/PKG-INFO

@@ -0,0 +1,34 @@
+Metadata-Version: 2.4
+Name: lackpy
+Version: 0.1.0
+Summary: Python that lacks most of Python. Restricted program generation and execution for tool composition.
+Project-URL: Documentation, https://lackpy.readthedocs.io
+Project-URL: Repository, https://github.com/teague/lackpy
+Requires-Python: >=3.10
+Provides-Extra: anthropic
+Requires-Dist: anthropic>=0.40; extra == 'anthropic'
+Provides-Extra: blq
+Requires-Dist: blq-cli; extra == 'blq'
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-gen-files>=0.5; extra == 'docs'
+Requires-Dist: mkdocs-literate-nav>=0.6; extra == 'docs'
+Requires-Dist: mkdocs-material>=9; extra == 'docs'
+Requires-Dist: mkdocstrings[python]>=0.24; extra == 'docs'
+Provides-Extra: fledgling
+Requires-Dist: fledgling; extra == 'fledgling'
+Provides-Extra: full
+Requires-Dist: anthropic>=0.40; extra == 'full'
+Requires-Dist: blq-cli; extra == 'full'
+Requires-Dist: fledgling; extra == 'full'
+Requires-Dist: mcp[cli]; extra == 'full'
+Requires-Dist: nsjail-python; extra == 'full'
+Requires-Dist: ollama>=0.4; extra == 'full'
+Provides-Extra: mcp
+Requires-Dist: mcp[cli]; extra == 'mcp'
+Provides-Extra: ollama
+Requires-Dist: ollama>=0.4; extra == 'ollama'
+Provides-Extra: sandbox
+Requires-Dist: nsjail-python; extra == 'sandbox'

lackpy-0.1.0/docs/concepts/architecture.md

@@ -0,0 +1,122 @@
+# Architecture
+
+## Pipeline
+
+A `delegate()` call traverses four stages in sequence:
+
+```
+  ┌────────────────────────────────────────────────────────────────┐
+  │  LackpyService                                                 │
+  │                                                                │
+  │  1. Kit resolution                                             │
+  │     kit name/list/dict ──► ResolvedKit (tools + callables)    │
+  │                                                                │
+  │  2. Inference                                                  │
+  │     intent + namespace_desc                                    │
+  │     ──► InferenceDispatcher                                    │
+  │          tier 0: TemplatesProvider (regex pattern match)       │
+  │          tier 1: RulesProvider    (keyword rules)              │
+  │          tier 2: OllamaProvider   (local LLM, optional)        │
+  │          tier 3: AnthropicProvider (cloud LLM, optional)       │
+  │     ◄── GenerationResult (program + provider_name + time_ms)  │
+  │                                                                │
+  │  3. Validation                                                 │
+  │     program + allowed_names + extra_rules                      │
+  │     ──► validate() (AST walk)                                  │
+  │     ◄── ValidationResult (valid, errors, calls, variables)     │
+  │                                                                │
+  │  4. Execution                                                  │
+  │     program + resolved.callables + param_values                │
+  │     ──► RestrictedRunner.run()                                 │
+  │     ◄── ExecutionResult (success, output, trace, variables)    │
+  └────────────────────────────────────────────────────────────────┘
+```
+
+Validation is also performed inside `InferenceDispatcher` after each provider attempt. If a provider returns an invalid program, the dispatcher feeds the errors back to the provider for a retry before moving to the next tier.
+
+---
+
+## Modules
+
+| Module | Responsibility | Key dependencies |
+|--------|---------------|-----------------|
+| `lackpy.service` | Unified service layer; wires all components together | All modules |
+| `lackpy.config` | Load `config.toml` from `.lackpy/` | stdlib `tomllib` / `tomli` |
+| `lackpy.lang.grammar` | `ALLOWED_NODES`, `FORBIDDEN_NODES`, `FORBIDDEN_NAMES`, `ALLOWED_BUILTINS` | `ast` |
+| `lackpy.lang.validator` | AST walk + rule application → `ValidationResult` | `lang.grammar` |
+| `lackpy.lang.grader` | `Grade(w, d)` computation from tool specs | none |
+| `lackpy.lang.rules` | Built-in custom rule callables | `ast` |
+| `lackpy.lang.spec` | Machine-readable grammar spec (used by `lackpy spec`) | `lang.grammar` |
+| `lackpy.kit.toolbox` | `Toolbox` — provider registry + tool resolution | none |
+| `lackpy.kit.registry` | `resolve_kit()` — name/list/dict → `ResolvedKit` | `kit.toolbox`, `lang.grader` |
+| `lackpy.kit.providers.builtin` | Built-in tools: `read`, `glob`, `write`, `edit` | `pathlib` |
+| `lackpy.kit.providers.python` | Wrap any importable function as a tool | `importlib` |
+| `lackpy.run.trace` | `Trace`, `TraceEntry`, `make_traced()` | `inspect`, `time` |
+| `lackpy.run.base` | `ExecutionResult`, `Executor` protocol | `run.trace` |
+| `lackpy.run.runner` | `RestrictedRunner` — compile + exec with traced namespace | `lang.grammar`, `run.trace` |
+| `lackpy.infer.dispatch` | `InferenceDispatcher` — priority-ordered provider loop | `lang.validator`, `infer.sanitize` |
+| `lackpy.infer.prompt` | `build_system_prompt()`, `format_params_description()` | `lang.grammar` |
+| `lackpy.infer.sanitize` | Strip model artifacts (markdown fences, preambles) | none |
+| `lackpy.infer.providers.*` | `TemplatesProvider`, `RulesProvider`, `OllamaProvider`, `AnthropicProvider` | `infer.prompt` |
+| `lackpy.cli` | `argparse`-based CLI; calls `LackpyService` | `service` |
+| `lackpy.mcp` | MCP server exposing the service as tools | `service` |
+
+---
+
+## Service layer role
+
+`LackpyService` is the single entry point. Both the CLI (`lackpy.cli`) and the MCP server (`lackpy.mcp`) call the service methods rather than accessing lower-level modules directly. This means:
+
+- The MCP server and CLI always have identical behaviour.
+- Third-party code using the Python API benefits from the same validation, tracing, and grade computation as the built-in interfaces.
+- Configuration is loaded once at `LackpyService.__init__` and propagated automatically.
+
+---
+
+## Security model
+
+lackpy uses three layers of defence in depth:
+
+### Layer 1 — AST validation (primary)
+
+Every program is parsed and walked before it is compiled or executed. `ALLOWED_NODES` is a whitelist: if a node type is not in the set, the program is rejected. This means:
+
+- `import` / `from ... import` → structurally impossible
+- `def` / `class` / `lambda` → structurally impossible
+- `while` / `try` / `except` → structurally impossible
+
+The validator also checks:
+
+- All function calls are to names in the kit or `ALLOWED_BUILTINS`
+- No name in `FORBIDDEN_NAMES` appears anywhere
+- No string constant contains `__` (prevents dunder access via `getattr`)
+- `for` loops must iterate over a function call or a variable (not a literal)
+
+### Layer 2 — Restricted execution namespace (secondary)
+
+`RestrictedRunner` executes programs with `__builtins__` set to `{}` (the empty dict). The only names available at runtime are:
+
+- Kit tools (wrapped in tracing callables)
+- `ALLOWED_BUILTINS` (direct references from the `builtins` module)
+- Parameter values
+
+Even if a program somehow bypassed the AST check, it could not call `eval`, `exec`, `compile`, or any other dangerous builtin — they are simply not in scope.
+
+### Layer 3 — nsjail (v2, planned)
+
+A future `sandbox` tier will use nsjail for process-level isolation with configurable memory and time limits. The `sandbox_enabled` config flag and `sandbox` parameter on `delegate`/`run_program` are already wired; the nsjail integration is slated for v2.
+
+---
+
+## Grade system
+
+Every kit has a `Grade(w, d)` computed from its tools:
+
+| Field | Meaning | Scale |
+|-------|---------|-------|
+| `w` | World coupling | 0 = pure, 1 = pinhole read, 2 = scoped exec, 3 = scoped write |
+| `d` | Effects ceiling | 0–3, higher = more side effects |
+
+`compute_grade()` takes the element-wise maximum across all tools in the kit. The grade is reported in every `delegate()` result and `kit_info()` response so callers can decide whether a given kit is acceptable for their context.
+
+Tool authors set `grade_w` and `effects_ceiling` on their `ToolSpec`. The built-in tools default to `grade_w=3, effects_ceiling=3` (conservative).

lackpy-0.1.0/docs/concepts/inference.md

@@ -0,0 +1,170 @@
+# Inference Pipeline
+
+## Tiers
+
+| Tier | Provider | Plugin | Latency | Requires |
+|------|----------|--------|---------|----------|
+| 0 | `TemplatesProvider` | built-in | ~0 ms | `.lackpy/templates/*.tmpl` files |
+| 1 | `RulesProvider` | built-in | ~0 ms | nothing |
+| 2 | `OllamaProvider` | `ollama` | 200–2000 ms | `pip install lackpy[ollama]`, running Ollama |
+| 3 | `AnthropicProvider` | `anthropic` | 500–3000 ms | `pip install lackpy[full]`, `ANTHROPIC_API_KEY` |
+
+The dispatcher tries each available provider in priority order. A provider is skipped if `available()` returns `False` (e.g. the `ollama` package is not installed). If a provider returns a syntactically valid program that fails AST validation, the dispatcher feeds the errors back for one retry before moving on.
+
+---
+
+## Tier 0 — Templates
+
+Templates are `.tmpl` files in `.lackpy/templates/`. Each file contains a frontmatter block and a program body:
+
+```
+---
+name: read-file
+pattern: "read the file {path}"
+success_count: 12
+fail_count: 0
+---
+content = read('{path}')
+content
+```
+
+The `pattern` field is a mini-template: `{name}` placeholders are converted to named regex groups. The intent is matched case-insensitively. On match, placeholders in the program body are substituted with the captured values.
+
+Templates are checked in sorted filename order. The first match wins.
+
+---
+
+## Tier 1 — Rules
+
+The rules tier uses direct regex matching for common intents. It handles:
+
+- `read (the )? file <path>` → `content = read('<path>')\ncontent`
+- `find (the )? definition(s)? (of|for) <name>` → `results = find_definitions('<name>')\nresults`
+- `find (all)? callers|usages|references (of|for) <name>` → `results = find_callers('<name>')\nresults`
+- `(find|list) all <ext> files` → `files = glob('**/*.<ext>')\nfiles`
+- `glob <pattern>` → `files = glob('<pattern>')\nfiles`
+
+Rules are only applied if the corresponding tool name appears in the namespace description. The rules tier always returns `available() = True`.
+
+---
+
+## Tier 2 — Ollama
+
+The Ollama provider sends a structured system prompt + user intent to a local model. The system prompt describes:
+
+- The available tools and their signatures
+- The `ALLOWED_BUILTINS`
+- Any pre-set parameter variables
+- The constraints (no `import`, `def`, `class`, etc.)
+
+If the first generation fails validation, the errors are appended to the user message and the model is called again once.
+
+---
+
+## Tier 3 — Anthropic
+
+The Anthropic provider works identically to the Ollama provider but calls the Anthropic Messages API. It is intended as a high-quality fallback for intents that a small local model cannot handle.
+
+---
+
+## Dispatch flow
+
+```
+for provider in providers:
+    if not provider.available():
+        continue
+
+    raw = await provider.generate(intent, namespace_desc)
+    program = sanitize_output(raw)
+    result = validate(program, allowed_names, extra_rules)
+
+    if result.valid:
+        return GenerationResult(program, provider.name, elapsed_ms)
+
+    # One retry with error feedback
+    raw = await provider.generate(intent, namespace_desc, error_feedback=result.errors)
+    program = sanitize_output(raw)
+    result = validate(program, allowed_names, extra_rules)
+
+    if result.valid:
+        return GenerationResult(program, provider.name, elapsed_ms)
+
+raise RuntimeError("All providers failed")
+```
+
+---
+
+## Config example
+
+```toml
+[inference]
+order = ["templates", "rules", "ollama-local", "anthropic-fallback"]
+
+[inference.providers.ollama-local]
+plugin = "ollama"
+host = "http://localhost:11434"
+model = "qwen2.5-coder:1.5b"
+temperature = 0.2
+keep_alive = "30m"
+
+[inference.providers.anthropic-fallback]
+plugin = "anthropic"
+model = "claude-haiku-4-5-20251001"
+```
+
+The `order` list controls priority. Built-in providers (`templates`, `rules`) are always prepended regardless of their position in `order`.
+
+---
+
+## The ratchet
+
+The ratchet pattern is a workflow built on top of the template tier:
+
+1. Issue `delegate` — the intent is handled by rules or an LLM on the first call.
+2. Verify the result is correct.
+3. Issue `create` to save the validated program as a template with an intent pattern.
+4. Subsequent `delegate` calls with matching intents hit tier 0 — zero latency, guaranteed valid.
+
+Over time, the template library grows and LLM calls become less frequent. The template tier acts as a ratchet: once an intent is captured, it stays captured.
+
+```bash
+# Step 1: first run (rules tier)
+lackpy delegate "read the file pyproject.toml" --kit read
+
+# Step 2: save as template
+cat > read_pyproject.py << 'EOF'
+content = read('pyproject.toml')
+content
+EOF
+lackpy create read_pyproject.py --name read-pyproject --kit read
+
+# Step 3: future runs hit tier 0
+lackpy delegate "read the file pyproject.toml" --kit read
+# generation_tier: "templates"
+```
+
+---
+
+## Custom providers
+
+Inference providers implement a simple protocol. See [Extending: Inference Providers](../extending/inference-providers.md) for the full guide.
+
+The minimum interface is:
+
+```python
+class MyProvider:
+    @property
+    def name(self) -> str: ...
+
+    def available(self) -> bool: ...
+
+    async def generate(
+        self,
+        intent: str,
+        namespace_desc: str,
+        config: dict | None = None,
+        error_feedback: list[str] | None = None,
+    ) -> str | None: ...
+```
+
+Register the provider on the service's dispatcher by appending it to `svc._inference_providers` before calling `delegate` or `generate`.

lackpy-0.1.0/docs/concepts/kits.md

@@ -0,0 +1,142 @@
+# Kits & Toolbox
+
+## Toolbox vs Kits
+
+| Concept | What it is | Scope |
+|---------|------------|-------|
+| **Toolbox** | The global registry of all available tools and their providers | Service-wide |
+| **Kit** | A named subset of toolbox tools for a specific task | Per-request |
+
+The `Toolbox` holds every tool that has been registered across all providers. A `Kit` is the subset of those tools that a particular program may call — it defines the allowed namespace for validation and the callable namespace for execution.
+
+---
+
+## ToolSpec fields
+
+`ToolSpec` is the metadata record for a single tool:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `name` | `str` | The function name used in lackpy programs |
+| `provider` | `str` | Provider name that resolves this tool (e.g. `"builtin"`, `"python"`) |
+| `provider_config` | `dict` | Provider-specific config (e.g. `module`, `function` for the `python` provider) |
+| `description` | `str` | Human-readable description, shown to LLMs in the system prompt |
+| `args` | `list[ArgSpec]` | Argument names, types, and descriptions |
+| `returns` | `str` | Return type annotation string |
+| `grade_w` | `int` | World coupling (0–3) |
+| `effects_ceiling` | `int` | Effects ceiling (0–3) |
+
+`ArgSpec` fields: `name`, `type` (string), `description`.
+
+---
+
+## Registering tools
+
+Tools are registered by adding a `ToolSpec` to the `Toolbox` and ensuring a matching provider is also registered:
+
+```python
+from lackpy import LackpyService
+from lackpy.kit.toolbox import ToolSpec, ArgSpec
+
+svc = LackpyService()
+
+# Register a custom tool backed by a Python function
+svc.toolbox.register_tool(ToolSpec(
+    name="count_lines",
+    provider="python",
+    provider_config={
+        "module": "my_tools",
+        "function": "count_lines",
+    },
+    description="Count the number of lines in a file",
+    args=[ArgSpec(name="path", type="str", description="File path")],
+    returns="int",
+    grade_w=1,
+    effects_ceiling=0,
+))
+```
+
+The `python` provider is always registered. It resolves tools by importing the named module and looking up the function.
+
+---
+
+## Provider table
+
+| Provider | Name | How it resolves tools |
+|----------|------|----------------------|
+| `BuiltinProvider` | `"builtin"` | Hardcoded implementations for `read`, `glob`, `write`, `edit` |
+| `PythonProvider` | `"python"` | `importlib.import_module(module)` then `getattr(module, function)` |
+| Custom | any string | Implement the provider protocol (see [Tool Providers](../extending/tool-providers.md)) |
+
+---
+
+## Kit parameter forms
+
+`resolve_kit()` accepts three kit forms:
+
+| Form | Type | Example | Behaviour |
+|------|------|---------|-----------|
+| Named kit | `str` | `"filesystem"` | Loads `.lackpy/kits/filesystem.kit` |
+| Tool list | `list[str]` | `["read", "glob"]` | Uses tool names directly as aliases |
+| Tool mapping | `dict` | `{"find": "glob"}` | Alias → actual tool name |
+| Nested dict | `dict` | `{"r": {"tool": "read"}}` | Dict entry with `"tool"` key |
+
+With the tool mapping form, the program sees `find(...)` but the toolbox resolves it to the `glob` implementation.
+
+---
+
+## Kit file format
+
+Named kits are stored as `.kit` files in `.lackpy/kits/`:
+
+```
+---
+name: filesystem
+description: Read, write, and search files
+---
+read
+glob
+write
+edit
+```
+
+- The YAML-like frontmatter between `---` lines is metadata.
+- Lines after the closing `---` are tool names, one per line.
+- Lines starting with `#` are treated as comments.
+- The frontmatter is not validated against a schema; only `name` and `description` are used.
+
+---
+
+## CLI management
+
+```bash
+# List all kits in .lackpy/kits/
+lackpy kit list
+
+# Show tools and grade for a kit
+lackpy kit info filesystem
+
+# Show tools and grade for an ad-hoc list
+lackpy kit info read,glob,write
+
+# Create a new kit
+lackpy kit create mykit --tools read glob --description "Read-only tools"
+```
+
+---
+
+## Grade computation
+
+`compute_grade(tools)` takes a dict of `{name: {"grade_w": int, "effects_ceiling": int}}` and returns the element-wise maximum across all tools:
+
+```python
+from lackpy import compute_grade
+
+grade = compute_grade({
+    "read":  {"grade_w": 1, "effects_ceiling": 0},
+    "write": {"grade_w": 3, "effects_ceiling": 3},
+})
+# Grade(w=3, d=3)
+```
+
+This grade is attached to every `ResolvedKit` and reported in `delegate()` results. The grade is informational — lackpy does not block execution based on grade values, but callers can use it to gate access in security-sensitive contexts.