elliot-stack 1.0.13 → 1.0.14

package/skills/estack-flight-planner/references/shuttle_schedules.md

@@ -0,0 +1,98 @@
+# Shuttle Schedules — Template & Format Reference
+
+This skill optionally pairs flights with a ground shuttle service. Pairing is **off by default** — the skill only runs it if the user's config has a `shuttle_service` block set.
+
+This file describes:
+1. How to configure your shuttle service in `~/.flight-planner/config.json`
+2. The JSON format the pairing script (`scripts/pair_shuttles.py`) expects
+3. How the skill builds that JSON from your shuttle company's website each run
+
+## 1. Config block
+
+Add this to `~/.flight-planner/config.json` if you regularly use a shuttle:
+
+```json
+"shuttle_service": {
+  "name": "Acme Airport Express",
+  "schedule_urls": [
+    "https://acmeairport.example.com/schedules/ord",
+    "https://acmeairport.example.com/schedules/mdw"
+  ],
+  "costs": {
+    "ORD": 60,
+    "MDW": 55
+  },
+  "home_timezone": "America/New_York",
+  "airport_timezones": {
+    "ORD": "America/Chicago",
+    "MDW": "America/Chicago"
+  }
+}
+```
+
+**Fields:**
+
+- `name` — display name for the company
+- `schedule_urls` — one or more URLs the skill will fetch with WebFetch each run. Should be public schedule pages.
+- `costs` — one-way USD cost per destination airport (IATA code)
+- `home_timezone` — IANA timezone string for the city/town the shuttle picks you up from
+- `airport_timezones` — IANA timezone string per destination airport
+
+Set `shuttle_service: null` (or omit the field entirely) if you don't use a shuttle.
+
+## 2. JSON the pairing script expects
+
+After fetching schedule URLs, the skill assembles this JSON and saves it as `shuttles.json`:
+
+```json
+{
+  "shuttles": [
+    {
+      "company": "Acme Airport Express",
+      "from": "Home",
+      "to": "ORD",
+      "pickup_location": "Downtown station",
+      "departs_local": "06:00",
+      "arrives_local": "08:30"
+    },
+    {
+      "company": "Acme Airport Express",
+      "from": "Home",
+      "to": "ORD",
+      "pickup_location": "Downtown station",
+      "departs_local": "10:00",
+      "arrives_local": "12:30"
+    }
+  ]
+}
+```
+
+**Field rules:**
+
+- `from` — descriptive label for the pickup region (not used for matching)
+- `to` — destination airport IATA code (used to match flights)
+- `departs_local` — pickup time in **home timezone**, HH:MM 24-hour
+- `arrives_local` — arrival time in **destination airport's local timezone**, HH:MM 24-hour
+- `pickup_location` — optional, shown in output
+
+The pairing script uses `--tz-offsets` to translate `arrives_local` into the home timezone for buffer math. The skill computes those offsets from `home_timezone` and `airport_timezones` in your config.
+
+## 3. Extracting schedules from a website
+
+Most shuttle companies post fixed weekly schedules on their site. To turn a schedule page into the JSON above:
+
+1. Use WebFetch on each URL in `schedule_urls`. Prompt: "extract every shuttle run with pickup time, destination airport, and arrival time."
+2. For each run, build one object with `from`, `to`, `departs_local`, `arrives_local`.
+3. Append all objects to a single `"shuttles"` array.
+
+If the company has multiple runs to the same airport on different days, treat each as a separate entry. If schedules change on weekends, fetch the relevant day's schedule when the user's flight date is a weekend.
+
+## 4. Cost overrides for a single run
+
+If the user wants to override shuttle costs for one run only (e.g., a friend is driving them for free), the skill passes `--shuttle-costs "ORD:0"` instead of the config value. This doesn't modify the saved config.
+
+## 5. Tightness tuning
+
+`--min-buffer-min` (default 90) sets the floor for "viable" — flights with less buffer than this between shuttle arrival and flight departure are marked TOO_TIGHT. Increase to 120 if your airport has long security lines, decrease to 60 if you trust your shuttle and have TSA PreCheck.
+
+`--max-wait-min` (default 240) caps how much pre-flight wait time is acceptable. Pairings beyond this get a LONG_WAIT label but still appear in the output.

package/skills/estack-flight-planner/scripts/check_setup.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# Deterministic setup check. Run at skill load time via the ```! fence in SKILL.md.
+# Outputs a human-readable report of:
+#   - whether ~/.flight-planner/config.json exists and its contents (key masked)
+#   - whether ~/.flight-planner/flight_history.json exists and its entry count
+#   - today's date (for resolving relative dates in Phase 1)
+#   - whether SERPAPI_KEY env var is set
+# No side effects. Safe to run on every skill invocation.
+
+set -u
+
+CONFIG_DIR="$HOME/.flight-planner"
+CONFIG_FILE="$CONFIG_DIR/config.json"
+HISTORY_FILE="$CONFIG_DIR/flight_history.json"
+
+echo "=== Flight Planner Setup ==="
+echo "Today: $(date +%Y-%m-%d)  (timezone: $(date +%Z))"
+echo ""
+
+# --- Config ---
+if [ -f "$CONFIG_FILE" ]; then
+  echo "Config:      $CONFIG_FILE (exists)"
+  echo ""
+  echo "--- Saved preferences ---"
+  # Mask serpapi_key value: show as "set" or "null" but never the actual key
+  python -c "
+import json, sys
+try:
+    with open(r'''$CONFIG_FILE''', encoding='utf-8') as f:
+        c = json.load(f)
+except Exception as e:
+    print(f'  ERROR reading config: {e}')
+    sys.exit(0)
+
+key = c.get('serpapi_key')
+print('  SerpAPI key:           ' + ('set' if key else 'null (will use WebSearch fallback)'))
+print('  Budget:                \$' + str(c.get('budget_usd', '?')) + ' (' + str(c.get('budget_strength', '?')) + ')')
+airlines = c.get('airline_preferences') or []
+print('  Airlines:              ' + (', '.join(airlines) if airlines else 'any') + ' (' + str(c.get('airline_preference_strength', '?')) + ')')
+np = c.get('nonstop_preference', '?')
+ns = c.get('nonstop_strength', '?')
+print('  Nonstop:               ' + str(np) + ' (' + str(ns) + ')')
+bands = c.get('time_priority_bands') or []
+print('  Time priority:         ' + (', '.join(bands) if bands else 'none') + ' (' + str(c.get('time_priority_strength', '?')) + ')')
+print('  Home airport:          ' + str(c.get('home_airport') or 'not set'))
+freq = c.get('frequent_destinations') or []
+print('  Frequent destinations: ' + (', '.join(freq) if freq else 'not set'))
+shuttle = c.get('shuttle_service')
+if shuttle:
+    name = shuttle.get('name', '?')
+    costs = shuttle.get('costs', {}) or {}
+    cost_str = ', '.join(f'{k}: \${v}' for k, v in costs.items()) if costs else 'no costs configured'
+    print('  Shuttle service:       ' + str(name) + ' (' + cost_str + ')')
+else:
+    print('  Shuttle service:       none (pairing step will be skipped)')
+" 2>&1 || echo "  (python not available; cannot parse config — read the file directly)"
+else
+  echo "Config:      $CONFIG_FILE (NOT FOUND)"
+  echo ""
+  echo "First-run setup is needed. The skill will walk the user through Phase 2"
+  echo "to create this file."
+fi
+
+echo ""
+
+# --- Environment SERPAPI_KEY ---
+if [ -n "${SERPAPI_KEY:-}" ]; then
+  echo "SERPAPI_KEY: set in environment (will be used if config key is null)"
+else
+  echo "SERPAPI_KEY: not set in environment"
+fi
+
+echo ""
+
+# --- Flight history ---
+if [ -f "$HISTORY_FILE" ]; then
+  count=$(python -c "
+import json
+try:
+    with open(r'''$HISTORY_FILE''', encoding='utf-8') as f:
+        data = json.load(f)
+    print(len(data) if isinstance(data, list) else 0)
+except Exception:
+    print(0)
+" 2>/dev/null || echo "?")
+  echo "History:     $HISTORY_FILE ($count entries)"
+else
+  echo "History:     $HISTORY_FILE (not created yet — first search will create it)"
+fi

package/skills/estack-flight-planner/scripts/fetch_flights.py

@@ -0,0 +1,99 @@
+"""
+Fetch flight data from SerpAPI Google Flights.
+
+Saves raw JSON responses to a platform-aware temp directory, one file per
+route x date combination. Prints the temp directory path on stdout so the
+caller can pipe it to filter_flights.py.
+
+Usage:
+    python fetch_flights.py --dates 2026-05-09,2026-05-10 --routes IND-EWR,ORD-LGA
+    python fetch_flights.py --dates 2026-05-09 --routes IND-EWR --airlines UA,DL --stops 1
+    python fetch_flights.py --dates 2026-05-09 --routes IND-EWR --output-dir /tmp/x
+
+Args:
+    --airlines    Optional comma-separated IATA airline codes (e.g. UA,DL). Omit for any airline.
+    --stops       Optional. 0=any (default), 1=nonstop only, 2=one-stop-max.
+
+Auth:
+    Reads SERPAPI_KEY from environment. Override with --api-key.
+"""
+import argparse
+import os
+import sys
+import tempfile
+import urllib.parse
+import urllib.request
+from pathlib import Path
+
+ENDPOINT = "https://serpapi.com/search"
+BASE_PARAMS = {
+    "engine": "google_flights",
+    "type": "2",          # one-way
+    "sort_by": "2",       # price
+    "currency": "USD",
+}
+
+
+def default_output_dir() -> Path:
+    base = Path(tempfile.gettempdir()) / "flight-planner"
+    base.mkdir(parents=True, exist_ok=True)
+    return base
+
+
+def fetch_one(api_key: str, dep: str, arr: str, date: str, out_path: Path,
+              airlines: str = None, stops: str = None) -> int:
+    params = {**BASE_PARAMS, "departure_id": dep, "arrival_id": arr,
+              "outbound_date": date, "api_key": api_key}
+    if airlines:
+        params["include_airlines"] = airlines
+    if stops and stops != "0":
+        params["stops"] = stops
+    url = f"{ENDPOINT}?{urllib.parse.urlencode(params)}"
+    with urllib.request.urlopen(url, timeout=60) as resp:
+        data = resp.read()
+    out_path.write_bytes(data)
+    return len(data)
+
+
+def main() -> int:
+    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    p.add_argument("--dates", required=True, help="Comma-separated YYYY-MM-DD")
+    p.add_argument("--routes", required=True, help="Comma-separated DEP-ARR (e.g. IND-EWR,ORD-LGA)")
+    p.add_argument("--airlines", default=None,
+                   help="Optional comma-separated IATA codes (e.g. UA,DL). Omit for any airline.")
+    p.add_argument("--stops", default="0",
+                   help="0=any (default), 1=nonstop only, 2=one-stop-max")
+    p.add_argument("--output-dir", default=None, help="Override temp dir")
+    p.add_argument("--api-key", default=None, help="SerpAPI key (else uses $SERPAPI_KEY)")
+    args = p.parse_args()
+
+    api_key = args.api_key or os.environ.get("SERPAPI_KEY")
+    if not api_key:
+        print("ERROR: provide --api-key or set SERPAPI_KEY env var", file=sys.stderr)
+        return 2
+
+    out_dir = Path(args.output_dir) if args.output_dir else default_output_dir()
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    dates = [d.strip() for d in args.dates.split(",") if d.strip()]
+    routes = [tuple(r.strip().split("-")) for r in args.routes.split(",") if r.strip()]
+
+    total = 0
+    for dep, arr in routes:
+        for date in dates:
+            fname = out_dir / f"{dep}_{arr}_{date}.json"
+            try:
+                size = fetch_one(api_key, dep, arr, date, fname,
+                                 airlines=args.airlines, stops=args.stops)
+                print(f"  {dep}->{arr} {date}: {size} bytes", file=sys.stderr)
+                total += 1
+            except Exception as e:
+                print(f"  {dep}->{arr} {date}: FAILED ({e})", file=sys.stderr)
+
+    print(f"Fetched {total} files to:", file=sys.stderr)
+    print(out_dir)  # stdout = the path, for piping
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/skills/estack-flight-planner/scripts/filter_flights.py

@@ -0,0 +1,265 @@
+"""
+Filter and rank flights from saved SerpAPI JSON files.
+
+Reads JSON files written by fetch_flights.py, applies filters (price, time
+window, route, airlines, nonstop), ranks results, and outputs a JSON list to
+stdout. Filters can be marked "soft" via --soft-filters — soft filters include
+non-matching flights but flag them with `soft_filter_violations` and rank them
+lower.
+
+With --cluster-analysis, prints a constraint-impact report instead of
+filtering: shows which constraint(s) eliminated which flight counts, plus a
+price distribution. The skill uses this to propose specific relaxations when
+hard filters return zero results.
+
+Usage:
+    python filter_flights.py --json-dir /tmp/flight-planner/ --max-price 200 --from IND --to EWR
+    python filter_flights.py --json-dir /tmp/flight-planner/ --max-price 200 \\
+        --soft-filters max-price,time-priority
+    python filter_flights.py --json-dir /tmp/flight-planner/ --cluster-analysis
+
+All filter args (--max-price, --time-priority, --from, --to, --airlines, --nonstop)
+are required for normal filter mode (no defaults — caller passes config values).
+Cluster-analysis mode echoes back impact of each provided filter.
+"""
+import argparse
+import glob
+import json
+import sys
+import tempfile
+from pathlib import Path
+
+
+SOFT_FILTER_NAMES = {"max-price", "time-priority", "airlines", "nonstop", "route"}
+
+
+def parse_band(b: str):
+    a, c = b.split("-")
+    return a.strip(), c.strip()
+
+
+def in_band(t: str, lo: str, hi: str) -> bool:
+    return lo <= t <= hi
+
+
+def parse_set(csv: str):
+    if not csv:
+        return set()
+    return {x.strip() for x in csv.split(",") if x.strip()}
+
+
+def parse_bands(csv: str):
+    if not csv:
+        return []
+    return [parse_band(b) for b in csv.split(",") if b.strip()]
+
+
+def load_flights(json_dir: Path):
+    flights = []
+    for fname in sorted(glob.glob(str(json_dir / "*.json"))):
+        parts = Path(fname).stem.split("_")
+        if len(parts) < 3:
+            continue
+        dep, arr = parts[0], parts[1]
+        date = "_".join(parts[2:])
+        try:
+            with open(fname, encoding="utf-8") as f:
+                data = json.load(f)
+        except Exception:
+            continue
+        for fg in data.get("best_flights", []) + data.get("other_flights", []):
+            legs = fg.get("flights", [])
+            if not legs:
+                continue
+            leg = legs[0]
+            last_leg = legs[-1]
+            dep_dt = leg.get("departure_airport", {}).get("time", "")
+            arr_dt = last_leg.get("arrival_airport", {}).get("time", "")
+            airline_code = leg.get("airline", "") or ""
+            airline_iata = (leg.get("airline_logo", "") or "")[-6:-4] if "airline_logo" in leg else ""
+            flights.append({
+                "date": date, "from": dep, "to": arr,
+                "flight": leg.get("flight_number", ""),
+                "airline": airline_code,
+                "airline_iata": (leg.get("flight_number", "") or "").split()[0] if leg.get("flight_number") else "",
+                "departs": dep_dt[-5:] if dep_dt else "",
+                "arrives": arr_dt[-5:] if arr_dt else "",
+                "departs_full": dep_dt, "arrives_full": arr_dt,
+                "price": fg.get("price"),
+                "duration_min": fg.get("total_duration", 0),
+                "aircraft": leg.get("airplane", ""),
+                "delayed": fg.get("often_delayed_by_over_30_min", False),
+                "stops": max(0, len(legs) - 1),
+            })
+    return flights
+
+
+def violations_for(f, max_price, bands, origins, dests, airlines, nonstop_required):
+    """Return a list of filter names this flight violates, or [] if it passes all."""
+    v = []
+    if origins and f["from"] not in origins:
+        v.append("route")
+    if dests and f["to"] not in dests:
+        if "route" not in v:
+            v.append("route")
+    if max_price is not None and (f["price"] is None or f["price"] > max_price):
+        v.append("max-price")
+    if bands:
+        if not f["departs"]:
+            v.append("time-priority")
+        else:
+            in_any = any(in_band(f["departs"], lo, hi) for lo, hi in bands)
+            if not in_any:
+                v.append("time-priority")
+    if airlines and f.get("airline_iata") and f["airline_iata"] not in airlines:
+        v.append("airlines")
+    if nonstop_required and f.get("stops", 0) > 0:
+        v.append("nonstop")
+    return v
+
+
+def priority_band(f, bands):
+    """Return index of first band the departure time falls into, or len(bands) if outside all."""
+    if not bands or not f["departs"]:
+        return 0
+    for i, (lo, hi) in enumerate(bands):
+        if in_band(f["departs"], lo, hi):
+            return i
+    return len(bands)
+
+
+def cluster_analysis(flights, max_price, bands, origins, dests, airlines, nonstop_required):
+    """Report which constraint eliminated which flight counts + price distribution."""
+    prices = sorted({f["price"] for f in flights if f["price"] is not None})
+
+    # Per-constraint impact: how many would be eliminated by this filter alone
+    constraint_impact = {}
+    for name in ("route", "max-price", "time-priority", "airlines", "nonstop"):
+        eliminated = 0
+        for f in flights:
+            v = violations_for(f, max_price, bands, origins, dests, airlines, nonstop_required)
+            if name in v:
+                eliminated += 1
+        constraint_impact[name] = {
+            "eliminated": eliminated,
+            "of_total": len(flights),
+        }
+
+    # All-filters combined: how many pass everything
+    passing_all = sum(
+        1 for f in flights
+        if not violations_for(f, max_price, bands, origins, dests, airlines, nonstop_required)
+    )
+
+    # Price distribution + natural gaps
+    gaps = []
+    if prices:
+        for i in range(1, len(prices)):
+            gap = prices[i] - prices[i - 1]
+            if gap >= 20:
+                gaps.append({"after": prices[i - 1], "before": prices[i], "gap": gap})
+
+    recommended_budgets = []
+    if prices:
+        if gaps:
+            recommended_budgets.append(gaps[0]["after"])
+            if len(gaps) > 1:
+                recommended_budgets.append(gaps[1]["after"])
+        else:
+            recommended_budgets.append(prices[len(prices) // 2])
+
+    return {
+        "total_flights": len(flights),
+        "passing_all_filters": passing_all,
+        "constraint_impact": constraint_impact,
+        "price_distribution": {
+            "min": prices[0] if prices else None,
+            "max": prices[-1] if prices else None,
+            "unique_prices": len(prices),
+            "natural_gaps": gaps[:5],
+            "recommended_budgets": recommended_budgets,
+        },
+        "applied_filters": {
+            "max_price": max_price,
+            "time_priority_bands": [f"{a}-{b}" for a, b in bands],
+            "origins": sorted(origins) if origins else None,
+            "destinations": sorted(dests) if dests else None,
+            "airlines": sorted(airlines) if airlines else None,
+            "nonstop_required": nonstop_required,
+        },
+    }
+
+
+def main() -> int:
+    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    default_dir = Path(tempfile.gettempdir()) / "flight-planner"
+    p.add_argument("--json-dir", default=str(default_dir))
+    p.add_argument("--max-price", type=int, default=None,
+                   help="Max price in USD. Omit for no price filter.")
+    p.add_argument("--time-priority", default="",
+                   help="Comma-separated priority bands HH:MM-HH:MM (e.g. 11:00-14:00,14:00-22:00). Empty = no time filter.")
+    p.add_argument("--from", dest="origins", default="",
+                   help="Comma-separated origin IATA codes. Empty = any origin.")
+    p.add_argument("--to", dest="dests", default="",
+                   help="Comma-separated destination IATA codes. Empty = any destination.")
+    p.add_argument("--airlines", default="",
+                   help="Comma-separated airline IATA codes (e.g. UA,DL). Empty = any airline.")
+    p.add_argument("--nonstop", action="store_true",
+                   help="Require nonstop only (stops==0).")
+    p.add_argument("--soft-filters", default="",
+                   help="Comma-separated filter names treated as soft (rank, don't exclude). "
+                        "Valid: max-price, time-priority, airlines, nonstop, route.")
+    p.add_argument("--cluster-analysis", action="store_true",
+                   help="Print constraint-impact + price distribution report instead of filtering.")
+    args = p.parse_args()
+
+    json_dir = Path(args.json_dir)
+    if not json_dir.exists():
+        print(f"ERROR: dir not found: {json_dir}", file=sys.stderr)
+        return 2
+
+    flights = load_flights(json_dir)
+    if not flights:
+        print(f"ERROR: no flights parsed from {json_dir}", file=sys.stderr)
+        return 2
+
+    origins = parse_set(args.origins)
+    dests = parse_set(args.dests)
+    airlines = parse_set(args.airlines)
+    bands = parse_bands(args.time_priority)
+    soft = parse_set(args.soft_filters)
+    bad_soft = soft - SOFT_FILTER_NAMES
+    if bad_soft:
+        print(f"ERROR: unknown --soft-filters values: {sorted(bad_soft)}. "
+              f"Valid: {sorted(SOFT_FILTER_NAMES)}", file=sys.stderr)
+        return 2
+
+    if args.cluster_analysis:
+        report = cluster_analysis(flights, args.max_price, bands, origins, dests,
+                                  airlines, args.nonstop)
+        print(json.dumps(report, indent=2))
+        return 0
+
+    out = []
+    for f in flights:
+        v = violations_for(f, args.max_price, bands, origins, dests, airlines, args.nonstop)
+        hard_v = [name for name in v if name not in soft]
+        if hard_v:
+            continue  # excluded by a hard filter
+        f["priority_band"] = priority_band(f, bands)
+        f["soft_filter_violations"] = v  # may be empty
+        out.append(f)
+
+    # Sort: # of soft violations asc, then priority band, then price, then departs
+    out.sort(key=lambda f: (
+        len(f.get("soft_filter_violations", [])),
+        f.get("priority_band", 0),
+        f.get("price") if f.get("price") is not None else 10**9,
+        f.get("departs", ""),
+    ))
+    print(json.dumps(out, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())