python-skills 1.0.0__py3-none-any.whl

skills/engineering/application_logging.md

@@ -0,0 +1,195 @@
+# Engineering: Logging (Application)
+
+**Purpose**: Application-level logging setup and patterns.
+
+**When to use**: Setting up logging for applications (not libraries).
+
+---
+
+## Core Rules
+
+### Application vs Library Logging
+- **Application**: Configures logging (handlers, formatters, levels)
+- **Library**: Only uses `logging.getLogger(__name__)`, adds `NullHandler`
+
+### Structured Logging (JSON)
+```python
+import logging
+import json
+import sys
+from datetime import datetime
+
+class JSONFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        log_data = {
+            "timestamp": datetime.fromtimestamp(record.created).isoformat(),
+            "level": record.levelname,
+            "logger": record.name,
+            "message": record.getMessage(),
+            "module": record.module,
+            "function": record.funcName,
+            "line": record.lineno,
+        }
+        
+        # Add extra fields
+        for key, value in record.__dict__.items():
+            if key not in {"name", "msg", "args", "created", "filename", "funcName",
+                          "levelname", "levelno", "lineno", "module", "msecs",
+                          "message", "name", "pathname", "process", "processName",
+                          "relativeCreated", "thread", "threadName", "exc_info",
+                          "exc_text", "stack_info", "getMessage"}:
+                log_data[key] = value
+        
+        if record.exc_info:
+            log_data["exception"] = self.formatException(record.exc_info)
+        
+        return json.dumps(log_data, default=str)
+
+# Setup
+handler = logging.StreamHandler(sys.stdout)
+handler.setFormatter(JSONFormatter())
+root = logging.getLogger()
+root.addHandler(handler)
+root.setLevel(logging.INFO)
+```
+
+### Structlog (Better Structured Logging)
+```python
+import structlog
+
+structlog.configure(
+    processors=[
+        structlog.contextvars.merge_contextvars,
+        structlog.processors.add_log_level,
+        structlog.processors.TimeStamper(fmt="iso"),
+        structlog.dev.ConsoleRenderer() if dev else structlog.processors.JSONRenderer(),
+    ],
+    wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
+    context_class=dict,
+    logger_factory=structlog.stdlib.LoggerFactory(),
+    cache_logger_on_first_use=True,
+)
+
+log = structlog.get_logger()
+
+# Usage
+log.info("user_login", user_id=123, ip="1.2.3.4")
+log.error("db_failed", error=str(e), query="SELECT ...")
+```
+
+### Context Injection
+```python
+import contextvars
+
+request_id_var: contextvars.ContextVar[str | None] = contextvars.ContextVar("request_id", default=None)
+
+class RequestIDFilter(logging.Filter):
+    def filter(self, record: logging.LogRecord) -> bool:
+        record.request_id = request_id_var.get()
+        return True
+
+# In middleware
+request_id_var.set("req-123")
+log.info("processing")  # Includes request_id
+```
+
+### Log Levels Guide
+| Level | Use Case |
+|-------|----------|
+| DEBUG | Detailed diagnostic (dev only) |
+| INFO | General operations (requests, startup) |
+| WARNING | Unexpected but handled (retry, fallback) |
+| ERROR | Operation failed (5xx, failed request) |
+| CRITICAL | System may stop (OOM, disk full) |
+
+---
+
+## Decision Rules
+
+| Situation | Setup |
+|-----------|-------|
+| Simple app | `logging.basicConfig` + JSON formatter |
+| Production service | structlog + JSON + log aggregation |
+| Library | `getLogger(__name__)` + `NullHandler` |
+| CLI tool | Rich handler for pretty output |
+
+---
+
+## Preferred Patterns
+
+```python
+# Centralized setup (call once at startup)
+def setup_logging(
+    level: str = "INFO",
+    json_format: bool = False,
+    dev_mode: bool = False,
+) -> None:
+    import logging
+    import sys
+    
+    # Clear existing
+    root = logging.getLogger()
+    for h in root.handlers[:]:
+        root.removeHandler(h)
+    
+    handler = logging.StreamHandler(sys.stdout)
+    
+    if json_format and not dev_mode:
+        handler.setFormatter(JSONFormatter())
+    elif dev_mode:
+        import rich.logging
+        handler = rich.logging.RichHandler(rich_tracebacks=True)
+    else:
+        handler.setFormatter(logging.Formatter(
+            "%(asctime)s [%(levelname)s] %(name)s: %(message)s"
+        ))
+    
+    root.addHandler(handler)
+    root.setLevel(level)
+    
+    # Quiet noisy libraries
+    logging.getLogger("httpx").setLevel(logging.WARNING)
+    logging.getLogger("httpcore").setLevel(logging.WARNING)
+    logging.getLogger("asyncio").setLevel(logging.WARNING)
+
+# Usage in app
+log = logging.getLogger(__name__)
+
+def process_order(order_id: str):
+    log.info("order_started", order_id=order_id)
+    try:
+        do_work()
+        log.info("order_completed", order_id=order_id)
+    except Exception as e:
+        log.exception("order_failed", order_id=order_id, error=str(e))
+        raise
+```
+
+---
+
+## Avoid
+
+- `print()` in production code
+- Logging sensitive data (passwords, tokens, PII)
+- Excessive DEBUG in production
+- No log rotation (use `RotatingFileHandler` or external)
+- Blocking I/O in logging (use `QueueHandler` + `QueueListener` for high volume)
+
+---
+
+## Validation Considerations
+
+- Test log output format
+- Verify log levels in different environments
+- Check structured fields are present
+- Ensure no PII in logs
+- Test high-volume logging doesn't block
+
+---
+
+## Related Skills
+
+- `stdlib/logging.md`
+- `engineering/configuration.md`
+- `security/secrets.md`
+- `generation/error_handling.md`

skills/engineering/cli_apps.md

@@ -0,0 +1,207 @@
+# Engineering: CLI Applications
+
+**Purpose**: Building command-line interfaces with Python.
+
+**When to use**: Creating CLI tools, scripts, or application entry points.
+
+---
+
+## Core Rules
+
+### Entry Points
+```toml
+# pyproject.toml
+[project.scripts]
+my-cli = "mypackage.cli:main"
+my-other = "mypackage.other:run"
+```
+
+### Click (Recommended for Complex CLIs)
+```python
+import click
+from pathlib import Path
+
+@click.group()
+@click.version_option()
+@click.option("-v", "--verbose", count=True)
+@click.pass_context
+def cli(ctx: click.Context, verbose: int):
+    """My CLI tool."""
+    ctx.ensure_object(dict)
+    ctx.obj["verbose"] = verbose
+
+@cli.command()
+@click.argument("input", type=click.Path(exists=True, path_type=Path))
+@click.option("-o", "--output", type=click.Path(path_type=Path))
+@click.pass_context
+def process(ctx: click.Context, input: Path, output: Path | None):
+    """Process a file."""
+    verbose = ctx.obj["verbose"]
+    if verbose:
+        click.echo(f"Processing {input}")
+    # ...
+
+@cli.command()
+@click.option("--force", is_flag=True)
+def clean(force: bool):
+    """Clean cache."""
+    if not force:
+        click.confirm("Delete all cache?", abort=True)
+    # ...
+
+if __name__ == "__main__":
+    cli()
+```
+
+### Typer (Modern, Type-Hint Based)
+```python
+import typer
+from pathlib import Path
+from typing_extensions import Annotated
+
+app = typer.Typer()
+
+@app.command()
+def process(
+    input: Annotated[Path, typer.Argument(exists=True, help="Input file")],
+    output: Annotated[Path | None, typer.Option("-o", "--output")] = None,
+    verbose: Annotated[int, typer.Option("-v", "--verbose", count=True)] = 0,
+):
+    """Process a file."""
+    if verbose:
+        typer.echo(f"Processing {input}")
+
+@app.command()
+def clean(force: Annotated[bool, typer.Option("--force", help="Skip confirmation")] = False):
+    """Clean cache."""
+    if not force:
+        typer.confirm("Delete all cache?", abort=True)
+
+if __name__ == "__main__":
+    app()
+```
+
+### argparse (Stdlib Only)
+```python
+import argparse
+from pathlib import Path
+
+def create_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="my-cli",
+        description="Description",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    parser.add_argument("input", type=Path)
+    parser.add_argument("-o", "--output", type=Path)
+    parser.add_argument("-v", "--verbose", action="count", default=0)
+    return parser
+
+def main(argv: list[str] | None = None) -> int:
+    parser = create_parser()
+    args = parser.parse_args(argv)
+    # ...
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())
+```
+
+### Rich (Beautiful Output)
+```python
+from rich.console import Console
+from rich.table import Table
+from rich.progress import Progress
+
+console = Console()
+
+console.print("[bold green]Success![/bold green]")
+console.print("[red]Error:[/red] Something failed")
+
+table = Table(title="Results")
+table.add_column("Name")
+table.add_column("Status")
+table.add_row("Item 1", "[green]OK[/green]")
+table.add_row("Item 2", "[red]FAIL[/red]")
+console.print(table)
+
+with Progress() as progress:
+    task = progress.add_task("Processing...", total=100)
+    for i in range(100):
+        progress.update(task, advance=1)
+```
+
+---
+
+## Decision Rules
+
+| Complexity | Tool |
+|------------|------|
+| Simple (1-3 commands) | `argparse` (stdlib) |
+| Multiple commands, options | `click` |
+| Modern, type-hint focused | `typer` |
+| Beautiful output needed | `rich` (with any) |
+| No dependencies allowed | `argparse` |
+
+---
+
+## Preferred Patterns
+
+```python
+# Common structure
+# src/mypackage/cli.py
+
+def main(argv: list[str] | None = None) -> int:
+    """Entry point for setuptools console_scripts."""
+    try:
+        # Parse args, run logic
+        return 0
+    except KeyboardInterrupt:
+        return 130
+    except Exception as e:
+        logger.exception("Fatal error")
+        return 1
+
+if __name__ == "__main__":
+    sys.exit(main())
+```
+
+### Subcommand Organization
+```
+cli.py          # Main entry, creates command group
+commands/
+  __init__.py   # Imports and registers subcommands
+  process.py    # @cli.command() or @app.command()
+  clean.py
+  config.py
+```
+
+---
+
+## Avoid
+
+- Logic in `__main__.py` (use entry point function)
+- Global state in CLI module
+- Printing directly (use `click.echo`, `typer.echo`, `console.print`)
+- No `--help` / `--version`
+- Exit codes not meaningful (0=success, 1=error, 130=interrupt)
+
+---
+
+## Validation Considerations
+
+- `my-cli --help` works
+- `my-cli --version` shows version
+- Invalid args show usage + exit 2
+- Tests for each command
+- Shell completion (`click`/`typer` support)
+
+---
+
+## Related Skills
+
+- `stdlib/argparse.md`
+- `engineering/configuration.md`
+- `engineering/packaging.md`
+- `generation/error_handling.md`
+- `testing/organization.md`

skills/engineering/configuration.md

@@ -0,0 +1,218 @@
+# Engineering: Configuration
+
+**Purpose**: Application configuration management patterns.
+
+**When to use**: Any application needing configuration (CLI, server, library).
+
+---
+
+## Core Rules
+
+### Configuration Sources (Priority Order)
+1. **Defaults** — Code defaults (lowest)
+2. **Config file** — TOML/YAML/JSON file
+3. **Environment variables** — Override config file
+4. **CLI arguments** — Highest priority (for apps)
+
+### Layered Configuration
+```python
+from dataclasses import dataclass, field
+from typing import Optional
+import os
+from pathlib import Path
+
+@dataclass
+class Config:
+    # Required (no default)
+    database_url: str
+    
+    # Optional with defaults
+    host: str = "0.0.0.0"
+    port: int = 8000
+    debug: bool = False
+    log_level: str = "INFO"
+    
+    # Nested config
+    redis: "RedisConfig" = field(default_factory=RedisConfig)
+
+@dataclass
+class RedisConfig:
+    host: str = "localhost"
+    port: int = 6379
+    db: int = 0
+    password: Optional[str] = None
+```
+
+### Environment Variable Mapping
+```python
+import os
+from dataclasses import fields
+
+def load_from_env(config: Config, prefix: str = "APP_") -> Config:
+    """Update config from environment variables."""
+    for f in fields(config):
+        env_key = f"{prefix}{f.name.upper()}"
+        if env_key in os.environ:
+            value = os.environ[env_key]
+            # Type conversion
+            if f.type == bool:
+                value = value.lower() in ("1", "true", "yes", "on")
+            elif f.type == int:
+                value = int(value)
+            elif f.type == list[str]:
+                value = value.split(",")
+            setattr(config, f.name, value)
+    return config
+```
+
+### Config File Loading (TOML)
+```python
+import tomllib  # Python 3.11+
+
+def load_config_file(path: Path) -> dict:
+    with path.open("rb") as f:
+        return tomllib.load(f)
+
+# For older Python: pip install tomli
+```
+
+### Complete Loader
+```python
+def load_config(
+    config_path: Path | None = None,
+    env_prefix: str = "APP_",
+    cli_overrides: dict | None = None,
+) -> Config:
+    # 1. Defaults
+    config = Config(database_url="")  # Will be validated
+    
+    # 2. Config file
+    if config_path and config_path.exists():
+        file_config = load_config_file(config_path)
+        apply_dict(config, file_config)
+    
+    # 3. Environment
+    load_from_env(config, env_prefix)
+    
+    # 4. CLI overrides
+    if cli_overrides:
+        apply_dict(config, cli_overrides)
+    
+    # 5. Validate
+    validate_config(config)
+    
+    return config
+
+def apply_dict(config: Config, data: dict) -> None:
+    for key, value in data.items():
+        if hasattr(config, key):
+            setattr(config, key, value)
+        elif hasattr(config, key.replace("-", "_")):
+            setattr(config, key.replace("-", "_"), value)
+
+def validate_config(config: Config) -> None:
+    if not config.database_url:
+        raise ValueError("database_url is required")
+    if not 1 <= config.port <= 65535:
+        raise ValueError("Invalid port")
+```
+
+### Pydantic Settings (Alternative)
+```python
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+class Settings(BaseSettings):
+    database_url: str
+    host: str = "0.0.0.0"
+    port: int = 8000
+    debug: bool = False
+    
+    model_config = SettingsConfigDict(
+        env_file=".env",
+        env_file_encoding="utf-8",
+        env_prefix="APP_",
+        case_sensitive=False,
+        extra="ignore",
+    )
+
+settings = Settings()  # Auto-loads from env + .env
+```
+
+---
+
+## Decision Rules
+
+| Need | Approach |
+|------|----------|
+| Simple app | Dataclass + env vars |
+| Complex validation | Pydantic Settings |
+| Multiple environments | Config files per env + env vars |
+| Secrets | Environment variables only (never files) |
+| Feature flags | Config + env override |
+
+---
+
+## Preferred Patterns
+
+```python
+# Central config instance
+_config: Config | None = None
+
+def get_config() -> Config:
+    global _config
+    if _config is None:
+        _config = load_config()
+    return _config
+
+# Reset for testing
+def reset_config() -> None:
+    global _config
+    _config = None
+```
+
+---
+
+## Secrets Management
+
+```python
+# NEVER hardcode secrets
+# NEVER commit .env files with real secrets
+# Use: Environment variables, secret managers (AWS Secrets, Vault, etc.)
+
+# .env.example (committed)
+DATABASE_URL=postgresql://user:pass@localhost/db
+API_KEY=
+
+# .env (gitignored, local only)
+DATABASE_URL=postgresql://prod:secret@db/prod
+API_KEY=sk-live-...
+```
+
+---
+
+## Avoid
+
+- Global mutable config (use frozen dataclass or singleton getter)
+- Config file with secrets
+- Complex inheritance in config
+- Silent failures on missing required config
+- Type confusion (env vars are strings)
+
+---
+
+## Validation Considerations
+
+- Test config loading with various sources
+- Validate required fields
+- Test type conversion edge cases
+- Ensure secrets never logged
+
+---
+
+## Related Skills
+
+- `engineering/pyproject_toml.md`
+- `engineering/cli_apps.md`
+- `security/secrets.md`
+- `generation/type_hints.md`
+- `generation/error_handling.md`