@bilalimamoglu/sift 0.1.0 → 0.2.1

package/README.md

@@ -1,38 +1,18 @@
 # sift
 
-`sift` is a small wrapper for agent workflows.
+`sift` is a small command-output reducer for agent workflows.
 
-Instead of giving a model the full output of `pytest`, `git diff`, `npm audit`, or `terraform plan`, you run the command through `sift`. `sift` captures the output, trims the noise, and returns a much smaller answer.
+Instead of feeding a model the full output of `pytest`, `git diff`, `npm audit`, `tsc --noEmit`, `eslint .`, or `terraform plan`, you run the command through `sift`. It captures the output, trims the noise, and returns a much smaller answer.
 
-That answer can be short text or structured JSON.
+Best fit:
+- non-interactive shell commands
+- agents that need short answers instead of full logs
+- CI checks where a command may succeed but still produce a blocking result
 
-## What it is
-
-- a command-output reducer for agents
-- best used with `sift exec ... -- <command>`
-- designed for non-interactive shell commands
-- compatible with OpenAI-style APIs
-
-## What it is not
-
-- not a native Codex tool
-- not an MCP server
-- not a replacement for raw shell output when exact logs matter
-- not meant for TUI or interactive password/confirmation flows
-
-## Why use it
-
-Large shell output is expensive and noisy.
-
-If an agent only needs to know:
-- did tests pass
-- what changed
-- are there critical vulnerabilities
-- is this infra plan risky
-
-then sending the full raw output to a large model is wasteful.
-
-`sift` keeps the shell command, but shrinks what the model has to read.
+Not a fit:
+- exact raw log inspection
+- TUI tools
+- password/confirmation prompts
 
 ## Installation
 
@@ -44,70 +24,84 @@ npm install -g @bilalimamoglu/sift
 
 ## One-time setup
 
-Set credentials once in your shell:
+For OpenAI-hosted models:
 
 ```bash
+export SIFT_PROVIDER=openai
 export SIFT_BASE_URL=https://api.openai.com/v1
-export SIFT_API_KEY=your_api_key
-export SIFT_MODEL=gpt-4.1-mini
+export SIFT_MODEL=gpt-5-nano
+export OPENAI_API_KEY=your_openai_api_key
 ```
 
-Or write them to a config file:
+Or generate a config file:
 
 ```bash
 sift config init
 ```
 
-`sift` is remote-first today. The safe path is to set `SIFT_API_KEY`, `SIFT_BASE_URL`, and `SIFT_MODEL` once, then run `sift` normally.
+If you use a different OpenAI-compatible endpoint, switch to `provider: openai-compatible` and use either the endpoint's native API key env var or the generic fallback:
+
+```bash
+export SIFT_PROVIDER_API_KEY=your_provider_api_key
+```
+
+Common compatible env fallbacks:
+- `OPENROUTER_API_KEY`
+- `TOGETHER_API_KEY`
+- `GROQ_API_KEY`
 
 ## Quick start
 
 ```bash
 sift exec "what changed?" -- git diff
 sift exec --preset test-status -- pytest
+sift exec --preset typecheck-summary -- tsc --noEmit
+sift exec --preset lint-failures -- eslint .
 sift exec --preset audit-critical -- npm audit
 sift exec --preset infra-risk -- terraform plan
+sift exec --preset audit-critical --fail-on -- npm audit
+sift exec --preset infra-risk --fail-on -- terraform plan
 ```
 
 ## Main workflow
 
-`sift exec` is the main path:
+`sift exec` is the default path:
 
 ```bash
 sift exec "did tests pass?" -- pytest
-sift exec "what changed?" -- git diff
-sift exec --preset infra-risk -- terraform plan
+sift exec --dry-run "what changed?" -- git diff
 ```
 
-What happens:
+What it does:
+1. runs the command
+2. captures `stdout` and `stderr`
+3. sanitizes, optionally redacts, and truncates the output
+4. sends the reduced input to a smaller model
+5. prints a short answer or JSON
+6. preserves the wrapped command's exit code
 
-1. `sift` runs the command.
-2. It captures `stdout` and `stderr`.
-3. It sanitizes, optionally redacts, and truncates the result.
-4. It sends the reduced input to a smaller model.
-5. It prints a short answer or JSON.
-6. It preserves the wrapped command's exit code.
+Use `--dry-run` to inspect the reduced input and prompt without calling the provider.
 
-## Pipe mode
+Use `--fail-on` when a built-in semantic preset should turn a technically successful command into a CI failure. Supported presets:
+- `infra-risk`
+- `audit-critical`
 
-If the output already exists in a pipeline, pipe mode still works:
+Pipe mode still works when output already exists:
 
 ```bash
 git diff 2>&1 | sift "what changed?"
 ```
 
-Use pipe mode when the command is already being produced elsewhere.
+## Built-in presets
 
-## Presets
-
-Built-in presets:
-
-- `test-status`
-- `audit-critical`
-- `diff-summary`
-- `build-failure`
-- `log-errors`
-- `infra-risk`
+- `test-status`: summarize test results
+- `typecheck-summary`: group blocking type errors by root cause
+- `lint-failures`: group repeated lint violations and highlight the files or rules that matter
+- `audit-critical`: extract only high and critical vulnerabilities
+- `infra-risk`: return a safety verdict for infra changes
+- `diff-summary`: summarize code changes and risks
+- `build-failure`: explain the most likely build failure
+- `log-errors`: extract the most relevant error signals
 
 Inspect them with:
 
@@ -118,135 +112,89 @@ sift presets show audit-critical
 
 ## Output modes
 
-- `brief`: short plain-text answer
-- `bullets`: short bullet list
-- `json`: structured JSON
-- `verdict`: `{ verdict, reason, evidence }`
+- `brief`
+- `bullets`
+- `json`
+- `verdict`
 
-Some built-in presets also use local heuristics before calling a model. For example, `infra-risk` can mark obvious destructive plans as `fail` without sending the whole decision to the model.
+Built-in JSON and verdict flows return strict error objects on provider or model failure.
 
 ## Config
 
-Generate an example config:
+Useful commands:
 
 ```bash
 sift config init
+sift config show
+sift config validate
+sift doctor
 ```
 
-`sift config show` masks secret values by default. Use `sift config show --show-secrets` only when you explicitly need the raw values.
+`sift config show` masks secrets by default. Use `--show-secrets` only when you explicitly need raw values.
 
 Resolution order:
-
 1. CLI flags
 2. environment variables
 3. `sift.config.yaml` or `sift.config.yml`
 4. `~/.config/sift/config.yaml` or `~/.config/sift/config.yml`
 5. built-in defaults
 
-If you pass `--config <path>`, that path is treated strictly. Missing explicit config paths are errors; `sift` does not silently fall back to defaults in that case.
-
-Supported environment variables:
-
-- `SIFT_PROVIDER`
-- `SIFT_MODEL`
-- `SIFT_BASE_URL`
-- `SIFT_API_KEY`
-- `SIFT_MAX_CAPTURE_CHARS`
-- `SIFT_TIMEOUT_MS`
-- `SIFT_MAX_INPUT_CHARS`
+If you pass `--config <path>`, that path is strict. Missing explicit config paths are errors.
 
-Example config:
+Minimal example:
 
 ```yaml
 provider:
-  provider: openai-compatible
-  model: gpt-4.1-mini
+  provider: openai
+  model: gpt-5-nano
   baseUrl: https://api.openai.com/v1
   apiKey: YOUR_API_KEY
-  timeoutMs: 20000
-  temperature: 0.1
-  maxOutputTokens: 220
 
 input:
   stripAnsi: true
-  redact: true
-  redactStrict: false
-  maxCaptureChars: 250000
-  maxInputChars: 20000
-  headChars: 6000
-  tailChars: 6000
+  redact: false
+  maxCaptureChars: 400000
+  maxInputChars: 60000
 
 runtime:
   rawFallback: true
-  verbose: false
-```
-
-## Commands
-
-```bash
-sift [question]
-sift preset <name>
-sift exec [question] -- <program> [args...]
-sift exec --preset <name> -- <program> [args...]
-sift exec [question] --shell "<command string>"
-sift exec --preset <name> --shell "<command string>"
-sift config init
-sift config show
-sift config validate
-sift doctor
-sift presets list
-sift presets show <name>
 ```
 
-## Using it with Codex
-
-`sift` does not install itself into Codex. The normal setup is:
+## Agent usage
 
-1. put credentials in your shell environment or `sift.config.yaml`
-2. add a short rule to `~/.codex/AGENTS.md`
+For Claude Code, add a short rule to `CLAUDE.md`.
 
-That way Codex inherits credentials safely. It should not pass API keys inline on every command.
+For Codex, add the same rule to `~/.codex/AGENTS.md`.
 
-Example:
-
-```md
-Prefer `sift exec` for non-interactive shell commands whose output will be read or summarized.
-Use pipe mode only when the output already exists from another pipeline.
-Do not use `sift` when exact raw output is required.
-Do not use `sift` for interactive or TUI workflows.
-```
-
-That gives the agent a simple habit:
-
-- run command through `sift exec` when a summary is enough
-- skip `sift` when exact output matters
+The important part is simple:
+- prefer `sift exec` for noisy shell commands
+- skip `sift` when exact raw output matters
+- keep credentials in your shell env or `sift.config.yaml`, never inline in prompts or agent instructions
 
 ## Safety and limits
 
-- Redaction is optional and regex-based.
-- Redaction is off by default. If command output may contain secrets, enable `--redact` or set it in config before sending output to a provider.
-- Built-in JSON and verdict flows return strict error objects on provider/model failure.
-- `sift exec` detects simple prompt-like output such as `[y/N]` or `password:` and skips reduction instead of guessing.
-- Pipe mode does not preserve upstream shell pipeline failures; use `set -o pipefail` if you need that behavior.
-- `sift exec` mirrors the wrapped command's exit code.
-- `sift doctor` is a conservative local config check. For the default OpenAI-compatible path it requires `baseUrl`, `model`, and `apiKey`.
+- redaction is optional and regex-based
+- retriable provider failures such as `429`, timeouts, and `5xx` are retried once
+- `sift exec` detects simple prompt-like output such as `[y/N]` or `password:` and skips reduction
+- pipe mode does not preserve upstream shell pipeline failures; use `set -o pipefail` if you need that behavior
 
-## Current scope
+## Releasing
 
-`sift` is intentionally small.
+This repo uses a manual GitHub Actions release workflow with npm trusted publishing.
 
-Today it supports:
-- OpenAI-compatible providers
-- agent-first `exec` mode
-- pipe mode
-- presets
-- local redaction and truncation
-- strict JSON/verdict fallbacks
+Release flow:
+1. bump `package.json`
+2. merge to `main`
+3. run the `release` workflow manually
 
-It does not try to be a full agent platform.
+The workflow:
+1. installs dependencies
+2. runs typecheck, tests, and build
+3. packs and smoke-tests the tarball
+4. publishes to npm
+5. creates and pushes the `vX.Y.Z` tag
+6. creates a GitHub Release
 
 ## License
 
 MIT
-
-The top-level MIT license is the licensing surface for this repo. Per-file license headers are not required unless code is copied or adapted from another source that needs separate notice or attribution.