python-skills 1.0.0__py3-none-any.whl

skills/stdlib/pathlib.md

@@ -0,0 +1,218 @@
+# Stdlib: pathlib
+
+**Purpose**: Modern, object-oriented filesystem paths.
+
+**When to use**: All filesystem operations. Replaces `os.path`, `os.walk`, `glob`.
+
+---
+
+## Core Rules
+
+### Path Objects
+```python
+from pathlib import Path
+
+# Creation
+Path("relative/path")
+Path("/absolute/path")
+Path.home() / "documents"
+Path.cwd()
+
+# From parts
+Path("a", "b", "c")  # a/b/c
+```
+
+### Properties
+```python
+p = Path("a/b/c.txt")
+
+p.name        # "c.txt"
+p.stem        # "c"
+p.suffix      # ".txt"
+p.suffixes    # [".txt"]
+p.parent      # Path("a/b")
+p.parents     # [Path("a/b"), Path("a"), Path(".")]
+p.anchor      # "" (or "/" on Unix, "C:\\" on Windows)
+p.root        # "/" or "C:\\"
+p.drive       # "" or "C:"
+```
+
+### Navigation
+```python
+p / "subdir" / "file.txt"     # Join
+p.joinpath("sub", "file.txt") # Join (multiple args)
+p.resolve()                    # Absolute, resolve symlinks
+p.absolute()                   # Absolute (no symlink resolve)
+p.relative_to(Path("a"))       # Relative path (raises if not subpath)
+p.is_relative_to(Path("a"))    # Boolean check (Python 3.9+)
+```
+
+### Filesystem Operations
+```python
+p.exists()           # Exists (file, dir, symlink)
+p.is_file()          # Regular file
+p.is_dir()           # Directory
+p.is_symlink()       # Symlink
+p.is_absolute()      # Absolute path
+p.is_relative()      # Relative path (Python 3.13+)
+
+p.stat()             # os.stat_result
+p.lstat()            # Don't follow symlinks
+p.owner()            # Owner name
+p.group()            # Group name
+
+# Times (float seconds since epoch)
+p.stat().st_mtime    # Modified
+p.stat().st_atime    # Accessed
+p.stat().st_ctime    # Created (Unix: metadata change)
+```
+
+### Reading/Writing
+```python
+# Text
+text = p.read_text(encoding="utf-8")
+p.write_text("content", encoding="utf-8")
+
+# Binary
+data = p.read_bytes()
+p.write_bytes(b"content")
+
+# Open (returns file object)
+with p.open("r", encoding="utf-8") as f:
+    ...
+
+# Append
+p.write_text("more", encoding="utf-8", mode="a")
+```
+
+### Directory Operations
+```python
+p.mkdir(parents=True, exist_ok=True)  # Create dir (mkdir -p)
+p.rmdir()                              # Remove empty dir
+p.unlink(missing_ok=True)              # Remove file (missing_ok Python 3.8+)
+p.replace(target)                      # Atomic replace
+p.rename(target)                       # Rename (not atomic across fs)
+p.symlink_to(target)                   # Create symlink
+p.hardlink_to(target)                  # Create hardlink
+p.touch(exist_ok=True)                 # Create empty file / update mtime
+```
+
+### Iteration and Globbing
+```python
+# Iterate directory
+for child in p.iterdir():
+    ...
+
+# Glob patterns
+for match in p.glob("*.py"):           # Non-recursive
+    ...
+for match in p.rglob("*.py"):          # Recursive
+    ...
+
+# Walk (like os.walk)
+for root, dirs, files in p.walk():
+    ...
+```
+
+### Path Matching
+```python
+p.match("*.py")              # Match final component
+p.match("*/*.py")            # Match relative pattern
+```
+
+### Temporary Files
+```python
+import tempfile
+
+with tempfile.NamedTemporaryFile(mode="w", delete=False) as f:
+    path = Path(f.name)
+    # Use path
+# Cleanup manual or use context manager
+
+# Better: tempfile.TemporaryDirectory
+with tempfile.TemporaryDirectory() as tmp:
+    path = Path(tmp) / "file.txt"
+    ...
+# Auto cleanup
+```
+
+---
+
+## Decision Rules
+
+| Operation | pathlib Method |
+|-----------|----------------|
+| Join paths | `p / "name"` or `p.joinpath()` |
+| Get parent | `p.parent` |
+| Get filename | `p.name` |
+| Get extension | `p.suffix` / `p.suffixes` |
+| Read text file | `p.read_text(encoding="utf-8")` |
+| Write text file | `p.write_text(content, encoding="utf-8")` |
+| Read binary | `p.read_bytes()` |
+| Write binary | `p.write_bytes(data)` |
+| Create dirs | `p.mkdir(parents=True, exist_ok=True)` |
+| Delete file | `p.unlink(missing_ok=True)` |
+| Delete empty dir | `p.rmdir()` |
+| List directory | `p.iterdir()` |
+| Find files | `p.glob()` / `p.rglob()` |
+| Walk tree | `p.walk()` |
+| Atomic write | Write to temp, `p.replace(target)` |
+
+---
+
+## Preferred Patterns
+
+```python
+# Atomic write (prevents partial reads)
+def atomic_write(path: Path, content: str, encoding: str = "utf-8") -> None:
+    tmp = path.with_suffix(path.suffix + ".tmp")
+    tmp.write_text(content, encoding=encoding)
+    tmp.replace(path)
+
+# Safe config loading
+def load_config(path: Path) -> dict:
+    if not path.is_file():
+        return {}
+    return json.loads(path.read_text(encoding="utf-8"))
+
+# Recursive find with filter
+py_files = list(project_root.rglob("*.py"))
+```
+
+---
+
+## Avoid
+
+- `os.path` functions (use pathlib equivalents)
+- String manipulation for paths (`+` or `os.path.join`)
+- `glob.glob()` (use `Path.glob()` / `rglob()`)
+- `os.walk()` (use `Path.walk()`)
+- Not specifying `encoding` in text operations
+- `missing_ok=False` (default) — raises on missing file
+
+---
+
+## Windows Notes
+
+- `Path` handles `\` and `/` correctly
+- `p.drive` returns `"C:"` on Windows
+- `p.anchor` returns `"C:\\"` on Windows
+- Symlinks require admin or Developer Mode
+- Case-insensitive filesystem
+
+---
+
+## Validation Considerations
+
+- Type checkers understand `Path` methods
+- `Path` implements `os.PathLike` — works with stdlib APIs
+- `resolve()` may raise on broken symlinks
+
+---
+
+## Related Skills
+
+- `stdlib/os_sys.md`
+- `engineering/configuration.md`
+- `security/path_traversal.md`
+- `security/file_handling.md`

skills/stdlib/re.md

@@ -0,0 +1,171 @@
+# Stdlib: re (Regular Expressions)
+
+**Purpose**: Pattern matching and text processing.
+
+**When to use**: String validation, extraction, transformation. Not for parsing structured formats (HTML, JSON, etc.).
+
+---
+
+## Core Rules
+
+### Basic Usage
+```python
+import re
+
+# Compile once, use many (performance)
+PATTERN = re.compile(r"\d{4}-\d{2}-\d{2}")
+
+# Match at start
+PATTERN.match("2024-01-15")      # Match object or None
+PATTERN.match("2024-01-15 text") # Matches!
+
+# Search anywhere
+PATTERN.search("Date: 2024-01-15")  # Match object or None
+
+# Full match (entire string)
+PATTERN.fullmatch("2024-01-15")    # Match object or None
+
+# Find all
+PATTERN.findall("2024-01-15 and 2024-02-20")  # List of strings/groups
+PATTERN.finditer("...")           # Iterator of Match objects
+
+# Substitute
+PATTERN.sub("REDACTED", text)     # Replace all
+PATTERN.subn("REDACTED", text)    # (new_string, count)
+
+# Split
+PATTERN.split("a2024-01-15b")     # ["a", "b"]
+```
+
+### Match Object
+```python
+m = PATTERN.search("Date: 2024-01-15")
+if m:
+    m.group()        # Entire match
+    m.group(1)       # Group 1
+    m.groups()       # All groups tuple
+    m.groupdict()    # Named groups dict
+    m.start(), m.end()  # Span
+    m.span(1)        # Span of group 1
+```
+
+### Common Patterns
+```python
+# Email (simplified)
+r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"
+
+# URL
+r"https?://[^\s/$.?#].[^\s]*"
+
+# IP address
+r"\b(?:\d{1,3}\.){3}\d{1,3}\b"
+
+# UUID
+r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
+
+# ISO datetime
+r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?"
+```
+
+### Flags
+```python
+re.IGNORECASE / re.I     # Case insensitive
+re.MULTILINE / re.M      # ^ $ match line start/end
+re.DOTALL / re.S         # . matches newline
+re.VERBOSE / re.X        # Verbose mode (whitespace ignored, comments)
+re.ASCII / re.A          # \w, \d, \s ASCII only
+```
+
+### Verbose Mode (Readable)
+```python
+EMAIL_PATTERN = re.compile(r"""
+    [a-zA-Z0-9._%+-]+   # Local part
+    @                   # Separator
+    [a-zA-Z0-9.-]+      # Domain
+    \.[a-zA-Z]{2,}      # TLD
+""", re.VERBOSE)
+```
+
+### Raw Strings (Critical)
+```python
+# ALWAYS use raw strings for regex
+r"\d+"     # Correct: backslash preserved
+"\d+"      # Wrong: "\d" = "d" in string literal
+```
+
+---
+
+## Decision Rules
+
+| Need | Method |
+|------|--------|
+| Validate entire string | `fullmatch` |
+| Find first occurrence | `search` |
+| Check prefix | `match` |
+| Extract all occurrences | `findall` / `finditer` |
+| Replace all | `sub` |
+| Replace with function | `sub(lambda m: ...)` |
+| Split on pattern | `split` |
+
+---
+
+## Preferred Patterns
+
+```python
+# Compile at module level (not in function)
+ISO_DATE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
+
+def is_valid_date(s: str) -> bool:
+    return ISO_DATE.fullmatch(s) is not None
+
+# Named groups for clarity
+LOG_PATTERN = re.compile(
+    r"(?P<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})"
+    r"\s+"
+    r"(?P<level>\w+)"
+    r"\s+"
+    r"(?P<message>.*)"
+)
+
+for match in LOG_PATTERN.finditer(log_text):
+    print(match.groupdict())
+
+# Substitution with function
+def redact_emails(text: str) -> str:
+    return EMAIL_PATTERN.sub(lambda m: f"{m.group(1)}@[REDACTED]", text)
+```
+
+---
+
+## Avoid
+
+- Regex for parsing HTML/XML/JSON (use proper parsers)
+- Complex regex without `re.VERBOSE` and comments
+- `.*` greedy matching when `.*?` non-greedy needed
+- Not using raw strings (`r"..."`)
+- Compiling regex inside hot loops
+- Catastrophic backtracking (nested quantifiers: `(a+)+`)
+
+---
+
+## Performance
+
+- Compile once, reuse
+- `fullmatch` > `match` + `$` > `search` + `^...$`
+- `finditer` > `findall` for large data (memory)
+- Specific patterns faster than generic
+
+---
+
+## Validation Considerations
+
+- `re.compile` validates pattern syntax at compile time
+- Type checkers don't validate regex patterns
+- Test edge cases: empty string, no match, multiple matches
+
+---
+
+## Related Skills
+
+- `security/input_validation.md`
+- `generation/type_hints.md` (Pattern type)

skills/stdlib/statistics.md

@@ -0,0 +1,112 @@
+# Stdlib: statistics
+
+**Purpose**: Basic statistical operations on numeric data.
+
+**When to use**: Simple statistics without NumPy/pandas dependency.
+
+---
+
+## Core Rules
+
+### Measures of Central Tendency
+```python
+import statistics
+
+data = [1, 2, 3, 4, 5, 6, 7, 8, 9]
+
+statistics.mean(data)       # Arithmetic mean: 5
+statistics.fmean(data)      # Fast float mean (3.8+): 5.0
+statistics.median(data)     # Median: 5
+statistics.median_low(data) # Low median: 5
+statistics.median_high(data) # High median: 5
+statistics.median_grouped(data, interval=1) # Grouped median
+statistics.mode(data)       # Single mode (raises if multimodal)
+statistics.multimode(data)  # List of modes (3.8+)
+statistics.harmonic_mean(data)  # Harmonic mean
+statistics.geometric_mean(data) # Geometric mean (3.8+)
+```
+
+### Measures of Spread
+```python
+statistics.stdev(data)      # Sample standard deviation
+statistics.pstdev(data)     # Population standard deviation
+statistics.variance(data)   # Sample variance
+statistics.pvariance(data)  # Population variance
+```
+
+### Requirements
+- Input: iterable of numeric (int, float, Decimal, Fraction)
+- At least 2 data points for stdev/variance
+- At least 1 for mean/median
+- Raises `StatisticsError` for insufficient data
+
+---
+
+## Decision Rules
+
+| Need | Function |
+|------|----------|
+| Average | `mean` / `fmean` |
+| Middle value | `median` |
+| Most common | `mode` / `multimode` |
+| Spread (sample) | `stdev` / `variance` |
+| Spread (population) | `pstdev` / `pvariance` |
+| Weighted average | Manual or NumPy |
+
+---
+
+## Preferred Patterns
+
+```python
+def summarize(values: list[float]) -> dict:
+    if not values:
+        return {"count": 0}
+    if len(values) == 1:
+        return {"count": 1, "mean": values[0], "median": values[0]}
+    
+    return {
+        "count": len(values),
+        "mean": statistics.fmean(values),
+        "median": statistics.median(values),
+        "stdev": statistics.stdev(values),
+        "min": min(values),
+        "max": max(values),
+    }
+
+# Streaming (large data) — use running calculation
+def running_stats():
+    n = 0
+    mean = 0.0
+    m2 = 0.0  # Sum of squares of differences
+    for x in data_stream:
+        n += 1
+        delta = x - mean
+        mean += delta / n
+        m2 += delta * (x - mean)
+        if n > 1:
+            yield {"mean": mean, "stdev": (m2 / (n - 1)) ** 0.5}
+```
+
+---
+
+## Avoid
+
+- Using for large datasets (use NumPy/pandas)
+- Calling on empty iterables (raises)
+- Expecting weighted statistics (not supported)
+- Using `mode` on multimodal data (raises; use `multimode`)
+
+---
+
+## Validation Considerations
+
+- Test with edge cases: empty, single element, two elements
+- Verify numeric precision for large values
+- Compare with NumPy for correctness if available
+
+---
+
+## Related Skills
+
+- `generation/type_hints.md`
+- `engineering/dependency_management.md` (when to add NumPy)

skills/stdlib/subprocess.md

@@ -0,0 +1,211 @@
+# Stdlib: subprocess
+
+**Purpose**: Safe subprocess execution and process management.
+
+**When to use**: Running external commands. Prefer stdlib APIs when possible.
+
+---
+
+## Core Rules
+
+### Basic Usage (Python 3.5+)
+```python
+import subprocess
+from pathlib import Path
+
+# Simple run (recommended)
+result = subprocess.run(
+    ["cmd", "arg1", "arg2"],
+    capture_output=True,  # stdout, stderr as bytes
+    text=True,            # Decode as str (encoding=locale)
+    encoding="utf-8",     # Explicit encoding
+    timeout=30,           # Seconds
+    check=False,          # Don't raise on non-zero
+    cwd=Path("/work"),
+    env={"VAR": "value"}, # Or None to inherit
+)
+
+result.returncode
+result.stdout
+result.stderr
+```
+
+### Safe Patterns
+```python
+# ALWAYS use list form (not shell=True)
+subprocess.run(["git", "status"])  # Safe
+subprocess.run("git status", shell=True)  # UNSAFE - shell injection!
+
+# With input
+result = subprocess.run(
+    ["grep", "pattern"],
+    input="data\nto\nsearch",
+    capture_output=True,
+    text=True,
+)
+
+# Streaming (large output)
+proc = subprocess.Popen(
+    ["cmd", "arg"],
+    stdout=subprocess.PIPE,
+    stderr=subprocess.PIPE,
+    text=True,
+)
+for line in proc.stdout:
+    process(line)
+proc.wait()
+```
+
+### Error Handling
+```python
+try:
+    result = subprocess.run(cmd, check=True, capture_output=True, text=True)
+except subprocess.CalledProcessError as e:
+    # e.returncode, e.stdout, e.stderr, e.cmd
+    logger.error("Command failed", extra={"cmd": e.cmd, "stderr": e.stderr})
+    raise
+except subprocess.TimeoutExpired as e:
+    # e.stdout, e.stderr (partial)
+    logger.error("Command timed out")
+    raise
+except FileNotFoundError:
+    logger.error("Command not found", extra={"cmd": cmd[0]})
+    raise
+```
+
+### Pipes and Chaining
+```python
+# Safe pipeline without shell
+p1 = subprocess.Popen(["cmd1"], stdout=subprocess.PIPE)
+p2 = subprocess.Popen(["cmd2"], stdin=p1.stdout, stdout=subprocess.PIPE)
+p1.stdout.close()  # Allow p1 to receive SIGPIPE
+output = p2.communicate()[0]
+```
+
+### Environment
+```python
+# Inherit + modify
+env = os.environ.copy()
+env["CUSTOM_VAR"] = "value"
+subprocess.run(cmd, env=env)
+
+# Clean environment
+subprocess.run(cmd, env={"PATH": "/usr/bin", "HOME": "/tmp"})
+```
+
+---
+
+## Security Rules (Critical)
+
+### NEVER use `shell=True` with untrusted input
+```python
+# DANGEROUS
+subprocess.run(f"echo {user_input}", shell=True)
+# User input: "; rm -rf /"
+
+# SAFE
+subprocess.run(["echo", user_input])
+```
+
+### Validate/Allowlist Commands
+```python
+ALLOWED_COMMANDS = {"git", "docker", "kubectl"}
+
+def safe_run(cmd: list[str], **kwargs):
+    if cmd[0] not in ALLOWED_COMMANDS:
+        raise ValueError(f"Command not allowed: {cmd[0]}")
+    return subprocess.run(cmd, **kwargs)
+```
+
+### Path Safety
+```python
+# Resolve paths, prevent traversal
+safe_path = Path(user_input).resolve()
+if not safe_path.is_relative_to(ALLOWED_ROOT):
+    raise ValueError("Path traversal attempt")
+subprocess.run(["process", str(safe_path)])
+```
+
+---
+
+## Decision Rules
+
+| Situation | Approach |
+|-----------|----------|
+| Simple command, wait for result | `subprocess.run()` |
+| Need streaming/large output | `subprocess.Popen` |
+| Pipeline | `Popen` chain (no shell) |
+| Fire and forget | `Popen` + `detach()` or `start_new_session=True` |
+| Need shell features (glob, vars) | Avoid; reimplement in Python |
+| Untrusted input in command | List form, validate/allowlist |
+
+---
+
+## Preferred Patterns
+
+```python
+def run_command(
+    cmd: list[str],
+    *,
+    cwd: Path | None = None,
+    timeout: float = 30.0,
+    env: dict[str, str] | None = None,
+    input_data: str | None = None,
+) -> subprocess.CompletedProcess:
+    """Safe subprocess wrapper."""
+    # Validate command
+    if not cmd or not isinstance(cmd, list):
+        raise ValueError("Command must be non-empty list")
+    
+    # Allowlist check (optional but recommended)
+    # if cmd[0] not in ALLOWED_COMMANDS: ...
+    
+    return subprocess.run(
+        cmd,
+        capture_output=True,
+        text=True,
+        encoding="utf-8",
+        errors="replace",  # Handle encoding issues
+        timeout=timeout,
+        check=False,       # Handle returncode manually
+        cwd=cwd,
+        env=env,
+        input=input_data,
+    )
+
+# Usage
+result = run_command(["git", "diff", "--name-only"])
+if result.returncode != 0:
+    logger.warning("Git diff failed", stderr=result.stderr)
+files = result.stdout.strip().splitlines()
+```
+
+---
+
+## Avoid
+
+- `shell=True` (except trusted, static commands)
+- String commands (always use list)
+- Unvalidated user input in command args
+- `os.system()`, `os.popen()` (legacy, unsafe)
+- Ignoring `returncode`
+- No timeout (hangs indefinitely)
+- Inheriting env without consideration (leaks secrets)
+
+---
+
+## Validation Considerations
+
+- Test with malicious input (injection attempts)
+- Verify timeout behavior
+- Check encoding handling
+- Audit command allowlists
+
+---
+
+## Related Skills
+
+- `security/command_injection.md`
+- `security/path_traversal.md`
+- `stdlib/os_sys.md`
+- `engineering/configuration.md`