python-skills 1.0.0__py3-none-any.whl

skills/stdlib/json.md

@@ -0,0 +1,162 @@
+# Stdlib: json
+
+**Purpose**: JSON encoding/decoding with Python objects.
+
+**When to use**: Serialization, config, APIs, data exchange.
+
+---
+
+## Core Rules
+
+### Basic Usage
+```python
+import json
+
+# Encode
+json_str = json.dumps(obj, indent=2, ensure_ascii=False)
+json_bytes = json.dumps(obj).encode("utf-8")
+
+# Decode
+obj = json.loads(json_str)
+obj = json.load(file_obj)
+```
+
+### Default Serialization
+| Python | JSON |
+|--------|------|
+| `dict` | object |
+| `list`, `tuple` | array |
+| `str` | string |
+| `int`, `float` | number |
+| `True`, `False` | `true`, `false` |
+| `None` | `null` |
+
+### Custom Serialization
+```python
+# default= callable for unsupported types
+json.dumps(obj, default=lambda o: o.isoformat() if hasattr(o, "isoformat") else str(o))
+
+# Or subclass JSONEncoder
+class CustomEncoder(json.JSONEncoder):
+    def default(self, obj):
+        if isinstance(obj, datetime):
+            return obj.isoformat()
+        if isinstance(obj, UUID):
+            return str(obj)
+        return super().default(obj)
+
+json.dumps(obj, cls=CustomEncoder)
+```
+
+### Custom Deserialization
+```python
+# object_hook for dicts
+def decode_datetime(d):
+    for key, value in d.items():
+        if key.endswith("_at") and isinstance(value, str):
+            try:
+                d[key] = datetime.fromisoformat(value)
+            except ValueError:
+                pass
+    return d
+
+json.loads(json_str, object_hook=decode_datetime)
+
+# object_pairs_hook for ordered pairs (preserves duplicate keys)
+json.loads(json_str, object_pairs_hook=OrderedDict)
+```
+
+### Streaming (Large Data)
+```python
+# Incremental encoding
+encoder = json.JSONEncoder()
+for chunk in encoder.iterencode(large_obj):
+    write(chunk)
+
+# Incremental decoding
+decoder = json.JSONDecoder()
+for obj in decoder.raw_decode(stream):
+    ...
+```
+
+---
+
+## Decision Rules
+
+| Situation | Approach |
+|-----------|----------|
+| Simple dict/list | `json.dumps` / `json.loads` |
+| Datetime, UUID, Decimal | Custom `JSONEncoder` + `object_hook` |
+| Large data (streaming) | `JSONEncoder.iterencode` / `JSONDecoder.raw_decode` |
+| Preserve order | Default (Python 3.7+) or `object_pairs_hook` |
+| Non-ASCII chars | `ensure_ascii=False` |
+| Compact output | `separators=(",", ":")` |
+| Human-readable | `indent=2` |
+
+---
+
+## Preferred Patterns
+
+```python
+# Standard API response
+def to_json(data: Any) -> str:
+    return json.dumps(
+        data,
+        default=json_default,
+        ensure_ascii=False,
+        separators=(",", ":"),
+    )
+
+def json_default(obj):
+    if isinstance(obj, (datetime, date)):
+        return obj.isoformat()
+    if isinstance(obj, UUID):
+        return str(obj)
+    if isinstance(obj, Decimal):
+        return str(obj)
+    if hasattr(obj, "__dict__"):
+        return obj.__dict__
+    raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")
+
+# Safe loading with schema validation
+def load_json(path: Path, schema: type) -> Any:
+    data = json.loads(path.read_text(encoding="utf-8"))
+    # Validate with pydantic, attrs, or manual check
+    return validate(data, schema)
+```
+
+---
+
+## Security
+
+- **Never** use `json.load` on untrusted input without validation
+- `json` module is safe (no code execution) unlike `pickle`/`yaml`
+- Large inputs can cause DoS (billions of nested objects) — consider `defusedxml`-style limits
+- `object_hook` runs on every dict — avoid expensive operations
+
+---
+
+## Avoid
+
+- `pickle` for data exchange (unsafe, Python-specific)
+- `yaml.load` without `Loader=yaml.SafeLoader` (unsafe)
+- Custom encoders that mutate input
+- `default=str` (silently converts everything to string, loses type info)
+- Large `indent` in production (wastes bandwidth)
+
+---
+
+## Validation Considerations
+
+- Schema validation after decode (pydantic, jsonschema, manual)
+- Round-trip test: `json.loads(json.dumps(obj)) == obj`
+- `json.JSONDecodeError` for parse errors (includes position)
+
+---
+
+## Related Skills
+
+- `security/unsafe_deserialization.md`
+- `engineering/configuration.md`
+- `stdlib/datetime.md`
+- `generation/type_hints.md` (TypedDict for JSON shapes)

skills/stdlib/logging.md

@@ -0,0 +1,185 @@
+# Stdlib: logging
+
+**Purpose**: Structured application logging.
+
+**When to use**: All application logging. Never `print()` in production code.
+
+---
+
+## Core Rules
+
+### Basic Setup
+```python
+import logging
+
+# Module-level logger (standard pattern)
+logger = logging.getLogger(__name__)
+
+# Logging levels
+logger.debug("Detailed diagnostic")
+logger.info("General operation")
+logger.warning("Unexpected, but handled")
+logger.error("Function failed")
+logger.critical("System may fail")
+
+# With context
+logger.info("User login", extra={"user_id": 123, "ip": "1.2.3.4"})
+```
+
+### Configuration (Application Entry Point)
+```python
+import logging
+import logging.config
+
+LOGGING_CONFIG = {
+    "version": 1,
+    "disable_existing_loggers": False,
+    "formatters": {
+        "standard": {
+            "format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+            "datefmt": "%Y-%m-%d %H:%M:%S",
+        },
+        "json": {
+            "()": "pythonjsonlogger.jsonlogger.JsonFormatter",
+            "format": "%(asctime)s %(levelname)s %(name)s %(message)s",
+        },
+    },
+    "handlers": {
+        "console": {
+            "class": "logging.StreamHandler",
+            "formatter": "standard",
+            "stream": "ext://sys.stdout",
+        },
+        "file": {
+            "class": "logging.handlers.RotatingFileHandler",
+            "formatter": "json",
+            "filename": "app.log",
+            "maxBytes": 10_000_000,
+            "backupCount": 5,
+        },
+    },
+    "root": {
+        "level": "INFO",
+        "handlers": ["console", "file"],
+    },
+    "loggers": {
+        "myapp": {"level": "DEBUG", "propagate": True},
+        "httpx": {"level": "WARNING"},  # Quiet noisy libs
+    },
+}
+
+logging.config.dictConfig(LOGGING_CONFIG)
+```
+
+### Structured Logging (Recommended)
+```python
+# Use extra for structured data
+logger.info("Order processed", extra={
+    "order_id": "ORD-123",
+    "amount": 99.99,
+    "currency": "USD",
+    "customer_id": "CUST-456",
+})
+
+# Or use a structured logging library (structlog)
+import structlog
+
+log = structlog.get_logger()
+log.info("order_processed", order_id="ORD-123", amount=99.99)
+```
+
+### Exception Logging
+```python
+try:
+    risky()
+except Exception:
+    logger.exception("Operation failed")  # Includes traceback
+    # Or:
+    logger.error("Operation failed", exc_info=True)
+```
+
+### Lazy Evaluation (Performance)
+```python
+# Arguments evaluated only if level enabled
+logger.debug("Expensive: %s", expensive_computation)  # Not f-string!
+
+# For complex objects
+logger.debug("Data: %s", lambda: repr(large_object))
+```
+
+---
+
+## Decision Rules
+
+| Situation | Approach |
+|-----------|----------|
+| Application code | `logger = logging.getLogger(__name__)` |
+| Library code | Same, but don't configure — let app configure |
+| Structured data | `extra={...}` or `structlog` |
+| Exceptions | `logger.exception()` or `exc_info=True` |
+| High-volume debug | Lazy `%s` formatting |
+| Production | JSON format, file rotation |
+| Development | Human-readable, console |
+
+---
+
+## Preferred Patterns
+
+```python
+# Library: just get logger, don't configure
+# mylib/__init__.py
+import logging
+log = logging.getLogger(__name__)
+
+# Application: configure once at startup
+# main.py
+def setup_logging(level: str = "INFO", json_format: bool = False):
+    import logging.config
+    # ... dictConfig as above
+
+# Context injection (middleware, decorators)
+def with_context(func):
+    @functools.wraps(func)
+    def wrapper(*args, **kwargs):
+        old_factory = logging.getLogRecordFactory()
+        def factory(*a, **kw):
+            record = old_factory(*a, **kw)
+            record.request_id = getattr(request_context, "id", None)
+            return record
+        logging.setLogRecordFactory(factory)
+        try:
+            return func(*args, **kwargs)
+        finally:
+            logging.setLogRecordFactory(old_factory)
+    return wrapper
+```
+
+---
+
+## Avoid
+
+- `print()` in production code
+- `logging.basicConfig()` in libraries (configures root globally)
+- `logging.getLogger()` without name (root logger)
+- f-strings in logging calls (evaluates always)
+- Logging sensitive data (secrets, PII, tokens)
+- Excessive DEBUG in production
+- Catching and logging then re-raising without context
+
+---
+
+## Validation Considerations
+
+- Test log output format
+- Verify log levels in different environments
+- Check log rotation works
+- Ensure no PII in logs (automated scanning)
+
+---
+
+## Related Skills
+
+- `engineering/configuration.md`
+- `security/secrets.md`
+- `stdlib/os_sys.md` (environment config)
+- `generation/error_handling.md` (exception logging)

skills/stdlib/os_sys.md

@@ -0,0 +1,184 @@
+# Stdlib: os and sys
+
+**Purpose**: Operating system interfaces and Python runtime.
+
+**When to use**: Process environment, filesystem, Python internals. Prefer `pathlib` for paths.
+
+---
+
+## Core Rules
+
+### os — Environment and Process
+```python
+import os
+
+# Environment variables
+os.environ              # Mapping (str -> str)
+os.getenv("KEY", "default")
+os.environ["KEY"] = "value"   # Set
+del os.environ["KEY"]         # Unset
+
+# Process
+os.getpid()             # Current PID
+os.getppid()            # Parent PID
+os.cpu_count()          # CPU count
+os.urandom(n)           # Cryptographic random bytes
+
+# Working directory
+os.getcwd()             # Str
+os.chdir(path)          # Change
+
+# Filesystem (prefer pathlib)
+os.listdir(path)        # Names only
+os.scandir(path)        # DirEntry objects (efficient)
+os.makedirs(path, exist_ok=True)
+os.remove(path)
+os.rmdir(path)
+os.rename(src, dst)
+os.replace(src, dst)    # Atomic
+os.symlink(src, dst)
+os.readlink(path)
+os.stat(path)           # stat_result
+os.path.isfile(path)    # Legacy — use pathlib
+
+# Path manipulation (legacy — use pathlib)
+os.path.join(*parts)
+os.path.split(path)
+os.path.dirname(path)
+os.path.basename(path)
+os.path.splitext(path)
+os.path.abspath(path)
+os.path.relpath(path, start)
+os.path.normpath(path)
+os.path.expanduser("~/path")
+os.path.expandvars("$VAR/path")
+```
+
+### sys — Python Runtime
+```python
+import sys
+
+# Version
+sys.version             # String
+sys.version_info        # Named tuple (major, minor, micro, ...)
+sys.version_info >= (3, 12)
+
+# Paths
+sys.path                # Module search path (list)
+sys.prefix              # Install prefix
+sys.exec_prefix         # Platform-specific prefix
+sys.executable          # Python interpreter path
+
+# Arguments
+sys.argv                # CLI args (list[str])
+sys.flags               # Interpreter flags
+
+# I/O
+sys.stdin               # TextIOWrapper
+sys.stdout
+sys.stderr
+sys.stdin.buffer        # Binary
+sys.stdout.buffer
+sys.stderr.buffer
+
+# Exit
+sys.exit(code)          # SystemExit
+sys.exit("message")     # Prints to stderr, exit 1
+
+# Modules
+sys.modules             # Loaded modules dict
+sys.builtin_module_names
+```
+
+### Platform Detection
+```python
+import sys, os
+
+sys.platform            # "linux", "darwin", "win32", "cygwin"
+os.name                 # "posix", "nt", "java"
+
+# Better: platform module
+import platform
+platform.system()       # "Linux", "Darwin", "Windows"
+platform.release()      # Kernel version
+platform.machine()      # "x86_64", "arm64"
+platform.python_implementation()  # "CPython", "PyPy"
+```
+
+---
+
+## Decision Rules
+
+| Need | Module | Function |
+|------|--------|----------|
+| Env vars | `os` | `os.getenv`, `os.environ` |
+| CWD | `os` | `os.getcwd`, `os.chdir` |
+| CLI args | `sys` | `sys.argv` |
+| Python version | `sys` | `sys.version_info` |
+| Module path | `sys` | `sys.path` |
+| Std I/O | `sys` | `sys.std*` |
+| Exit | `sys` | `sys.exit` |
+| Random bytes | `os` | `os.urandom` |
+| Platform | `platform` | `platform.system()` |
+
+---
+
+## Preferred Patterns
+
+```python
+# Environment-based config
+def get_config() -> Config:
+    return Config(
+        database_url=os.getenv("DATABASE_URL", "sqlite:///local.db"),
+        debug=os.getenv("DEBUG", "false").lower() == "true",
+        port=int(os.getenv("PORT", "8000")),
+    )
+
+# Platform-specific paths
+def get_data_dir() -> Path:
+    if sys.platform == "win32":
+        base = Path(os.getenv("LOCALAPPDATA", Path.home() / "AppData" / "Local"))
+    elif sys.platform == "darwin":
+        base = Path.home() / "Library" / "Application Support"
+    else:  # Linux/Unix
+        base = Path(os.getenv("XDG_DATA_HOME", Path.home() / ".local" / "share"))
+    return base / "myapp"
+
+# Safe subprocess with clean env
+def run_clean(cmd: list[str]):
+    clean_env = {
+        "PATH": "/usr/bin:/bin",
+        "LANG": "C.UTF-8",
+        "HOME": "/tmp",
+    }
+    return subprocess.run(cmd, env=clean_env, ...)
+```
+
+---
+
+## Avoid
+
+- `os.path` for new code (use `pathlib`)
+- `os.system`, `os.popen` (use `subprocess`)
+- Modifying `sys.path` at runtime (use proper packaging)
+- `sys.exit()` in library code (raise exceptions)
+- Hardcoding paths (`/tmp`, `C:\\Temp`)
+- Assuming Unix paths on Windows
+
+---
+
+## Validation Considerations
+
+- Test on target platforms (Windows vs Unix)
+- Verify env var handling with missing/empty values
+- Check path handling with spaces, unicode
+
+---
+
+## Related Skills
+
+- `stdlib/pathlib.md`
+- `stdlib/subprocess.md`
+- `engineering/configuration.md`
+- `engineering/virtual_environments.md`
+- `security/secrets.md`