spark-connect-cli 0.2.0__py3-none-any.whl

spark_connect_cli/SKILL.md

@@ -0,0 +1,151 @@
+---
+name: spark-connect-cli
+description: >-
+  Query Spark / Hive from the shell with the `scq` CLI over Spark Connect, and
+  run long Spark jobs (e.g. Hive->ClickHouse syncs) without blocking. Use
+  whenever the user wants to read Hive/Spark data, explore databases/tables/
+  schema, run a Spark SQL analysis, or sync a Hive table somewhere. Triggers:
+  Hive, Spark, Spark SQL, 查 Hive, 跑个 Spark SQL, 看下这个表, 同步到 ClickHouse,
+  sync table.
+---
+
+# scq — Spark Connect from the shell
+
+`scq` queries a Spark Connect server (JSON-first, **read-only by default**) and
+manages **async long jobs** so you never sit in a blocking tool call.
+
+## Discover before you query
+
+Don't guess names. Discover them first:
+
+1. `scq databases` — list databases.
+2. `scq tables [DB] --like '%keyword%'` — list tables.
+3. `scq describe <db.table>` — list columns (name, type, comment).
+4. `scq query "SELECT ..."` — run it once you know the schema.
+
+## Reading output
+
+- **stdout** carries data. Default is **JSONEachRow** (NDJSON — one JSON object
+  per line). Other formats: `--format json|csv|tsv|table`.
+- **stderr** carries errors as one JSON object `{"error": ..., "code": ...}`.
+- `query` caps at `SCQ_MAX_ROWS` (default 10k); a `{"warning": ...}` on stderr
+  means it was truncated — add `LIMIT`/filters or raise `--max-rows`.
+
+## Branch on the exit code
+
+`0` ok · `1` query error (fix the SQL) · `2` connection error (check
+`$SPARK_REMOTE`) · `3` read-only guard blocked it · `4` job-control error.
+
+## Read-only by default
+
+`scq query` allows only SELECT/SHOW/DESCRIBE/EXPLAIN/WITH. Writes and DDL exit
+with code `3` unless you pass `--allow-ddl`. **Only** add `--allow-ddl` when the
+user explicitly asked to modify data or schema.
+
+## Long jobs — submit, then poll. NEVER block.
+
+A full-table sync is a multi-minute Spark job. **Do not** run it in the
+foreground and wait. Submit it, tell the user the job id, and hand control back:
+
+```bash
+scq sync ods.orders --to clickhouse      # prints {"job_id": "...", "state": "running"}
+```
+
+Then, *only when the user asks* "how's it going" / after a natural pause:
+
+```bash
+scq jobs status j-20260625-...           # state, source_rows, written_rows, exit_code
+scq jobs logs   j-20260625-... --tail 40 # recent progress
+scq jobs cancel j-20260625-...           # stop it
+```
+
+Etiquette:
+- After submitting, reply with the job id and a one-line "I'll check when you
+  want." Don't loop on `status` in a tight wait — let the user drive, or poll on
+  a relaxed cadence.
+- Report terminal state plainly: `succeeded` with `written_rows`, or `failed`
+  with the tail of the log.
+
+## Hive → ClickHouse sync workflow
+
+When the user says "同步 X 表到 ClickHouse":
+
+1. `scq describe <src>` — get the Hive schema.
+2. Decide the **target database and table**. `--target` takes `db.table`:
+   - If the user names a database (e.g. `class_db`), pass it **qualified**:
+     `--target class_db.class`. A **bare table name lands in the connection's
+     default database** (`default`) — don't let data silently go there.
+   - The **database must already exist** (auto-create makes the table, not the
+     database). Ensure it first: `chsql query --allow-ddl "CREATE DATABASE IF NOT
+     EXISTS class_db"`.
+3. Make sure the target table is good:
+   - For a quick/one-off sync, let `scq sync` auto-create it — but pass
+     `--order-by <key>` so it gets a real sort key (otherwise it is created with
+     `ORDER BY tuple()`, no primary index).
+   - For a production table, **pre-create it** with `chsql query --allow-ddl
+     "CREATE TABLE class_db.class (...) ENGINE = MergeTree ORDER BY (...)"` (full
+     control over engine, keys, partitioning), then sync.
+4. Submit: `scq sync <src> --target db.table [--order-by key] [--where ...]`.
+5. Hand back the job id. Verify with row counts when it finishes.
+
+The ClickHouse JDBC connection (`$SCQ_CH_JDBC`) is preconfigured — you do **not**
+pass credentials; just choose the `db.table` with `--target`.
+
+### Spark/Hive → ClickHouse type mapping
+
+| Spark/Hive | ClickHouse |
+|------------|------------|
+| boolean | Bool |
+| tinyint / smallint / int / bigint | Int8 / Int16 / Int32 / Int64 |
+| float / double | Float32 / Float64 |
+| decimal(p,s) | Decimal(p,s) |
+| string / varchar / char / binary | String |
+| date | Date32 |
+| timestamp | DateTime64(3) |
+
+Nullable columns map to `Nullable(T)`. Nested/complex types default to `String`
+(JSON) — confirm with the user before relying on them.
+
+## Metadata & execution introspection
+
+Two general primitives — don't hand-stitch many queries.
+
+**Table metadata → `scq meta db.table`** — one JSON: schema (+ ClickHouse type
+mapping), created time, owner, format, HDFS location, partition columns +
+partition list/count, file count/total size, and min/max file modification time
+(i.e. "when did the data arrive"). Add `--count` for an exact row count (runs a
+`count(*)`, so only when asked). For ad-hoc bits you can still use
+`scq query "DESCRIBE EXTENDED t"` / `"SHOW PARTITIONS t"`.
+
+**Execution metadata → `scq exec <path>`** — read-only passthrough to the Spark
+REST API (auto-discovers the app, GET-only). The model reads the JSON, so any
+runtime question is the same command with a different path:
+
+```bash
+scq exec stages?status=active          # what's running now
+scq exec sql                           # each query's plan + metrics
+scq exec executors                     # cores / memory / GC / shuffle
+scq exec jobs
+scq exec stages/<id>/<attempt>/taskSummary?quantiles=0.5,0.95,1.0
+```
+
+- **Data skew**: pull a stage's `taskSummary` and compare a metric's **max vs
+  median** (`executorRunTime`, `shuffleReadBytes`, `shuffleReadRecords`). A large
+  `max/median` ratio = a straggler / skewed partition. `…?details=true` on a
+  stage lists every task to find the hot one.
+- Stage/job lists can be long — filter (`?status=active`) or fetch one id.
+- For the *plan before running*, use `scq query "EXPLAIN FORMATTED SELECT ..."`.
+
+## Connection
+
+`scq --remote sc://host:15002 ...` or set `$SPARK_REMOTE`. No Kerberos or JVM is
+needed on this side — the Spark Connect server does the auth.
+
+## Recipes
+
+```bash
+scq --format table tables analytics --like '%event%'
+scq query --format table "SELECT count(*) FROM analytics.events"
+scq query --max-rows 0 "SELECT * FROM small_dim"        # no cap
+scq sync analytics.events --to clickhouse --where "dt='2026-06-25'"
+```

spark_connect_cli/__init__.py

@@ -0,0 +1,2 @@
+"""spark-connect-cli — an agent-friendly Spark Connect CLI."""
+__version__ = "0.2.0"

spark_connect_cli/__main__.py

@@ -0,0 +1,4 @@
+from .cli import main
+
+if __name__ == "__main__":
+    main()

spark_connect_cli/cli.py

@@ -0,0 +1,138 @@
+"""Argument parsing and dispatch for `scq` / `spark-connect-cli`."""
+from __future__ import annotations
+
+import json
+import os
+import sys
+
+from . import jobs, meta, query, rest
+from .session import DEFAULT_REMOTE
+
+
+def cmd_sync(args) -> None:
+    # The JDBC URL carries the ClickHouse password, so it must NOT land in argv
+    # (argv is persisted in the job registry's meta.json). Pass it to the worker
+    # through the environment instead — submit() copies os.environ to the child.
+    if args.ch_jdbc:
+        os.environ["SCQ_CH_JDBC"] = args.ch_jdbc
+    argv = [args.source, "--to", args.to, "--remote", args.remote, "--mode", args.mode]
+    if args.target:
+        argv += ["--target", args.target]
+    if args.where:
+        argv += ["--where", args.where]
+    if args.limit:
+        argv += ["--limit", str(args.limit)]
+    if args.batchsize:
+        argv += ["--batchsize", str(args.batchsize)]
+    if args.num_partitions:
+        argv += ["--num-partitions", str(args.num_partitions)]
+    if args.order_by:
+        argv += ["--order-by", args.order_by]
+    if args.engine:
+        argv += ["--engine", args.engine]
+    job_id = jobs.submit("sync", argv,
+                         {"source": args.source, "target": args.target or "", "to": args.to})
+    print(json.dumps({
+        "job_id": job_id, "state": "running",
+        "message": f"sync of {args.source} -> {args.to} submitted; "
+                   f"poll with `scq jobs status {job_id}`",
+    }))
+
+
+def cmd_skill_install(args) -> None:
+    """Write the bundled SKILL.md into an agent skills directory (mirrors
+    `chsql skill install`)."""
+    import importlib.resources as ir
+    from pathlib import Path
+    root = Path(args.dir or os.environ.get("SKILLS_DIR")
+                or (Path.home() / ".agents" / "skills"))
+    dest = root / "spark-connect-cli"
+    dest.mkdir(parents=True, exist_ok=True)
+    content = ir.files("spark_connect_cli").joinpath("SKILL.md").read_text()
+    (dest / "SKILL.md").write_text(content)
+    print(json.dumps({"installed": str(dest / "SKILL.md")}))
+
+
+def build_parser():
+    import argparse
+    ap = argparse.ArgumentParser(
+        prog="scq",
+        description="Agent-friendly Spark Connect CLI: read-only querying + "
+                    "async long-job control. No JVM, no Kerberos on the client.")
+    ap.add_argument("--remote", default=DEFAULT_REMOTE,
+                    help=f"Spark Connect endpoint (default {DEFAULT_REMOTE} / $SPARK_REMOTE)")
+    ap.add_argument("--format", default="jsoneachrow",
+                    choices=["jsoneachrow", "json", "csv", "tsv", "table"])
+    sub = ap.add_subparsers(dest="cmd", required=True)
+
+    sub.add_parser("databases", help="List databases").set_defaults(func=query.cmd_databases)
+
+    pt = sub.add_parser("tables", help="List tables in a database")
+    pt.add_argument("database", nargs="?", default=None)
+    pt.add_argument("--like", default=None)
+    pt.set_defaults(func=query.cmd_tables)
+
+    pd = sub.add_parser("describe", help="Show a table's columns")
+    pd.add_argument("table")
+    pd.set_defaults(func=query.cmd_describe)
+
+    pq = sub.add_parser("query", help="Run SQL (read-only unless --allow-ddl)")
+    pq.add_argument("sql")
+    pq.add_argument("--allow-ddl", action="store_true")
+    pq.add_argument("--max-rows", type=int, default=None)
+    pq.set_defaults(func=query.cmd_query)
+
+    ps = sub.add_parser("sync", help="Submit an async Hive->ClickHouse sync (returns a job id)")
+    ps.add_argument("source", help="Hive table, e.g. db.table")
+    ps.add_argument("--to", default="clickhouse")
+    ps.add_argument("--mode", choices=["auto", "parallel", "single"], default="auto")
+    ps.add_argument("--ch-jdbc", default=None, help="ClickHouse JDBC URL (or $SCQ_CH_JDBC)")
+    ps.add_argument("--target", default=None, help="ClickHouse target as db.table (or bare table = default db)")
+    ps.add_argument("--where", default=None)
+    ps.add_argument("--limit", type=int, default=0)
+    ps.add_argument("--batchsize", type=int, default=None)
+    ps.add_argument("--num-partitions", type=int, default=None)
+    ps.add_argument("--order-by", default=None, help="ORDER BY key for an auto-created table, e.g. 'id'")
+    ps.add_argument("--engine", default=None, help="Engine for an auto-created table (default MergeTree)")
+    ps.set_defaults(func=cmd_sync)
+
+    pm = sub.add_parser("meta", help="One JSON bundle of a table's metadata")
+    pm.add_argument("table", help="db.table")
+    pm.add_argument("--count", action="store_true", help="include exact row count (runs count(*))")
+    pm.set_defaults(func=meta.cmd_meta)
+
+    pe = sub.add_parser("exec", help="Read-only Spark execution metadata (REST API passthrough)")
+    pe.add_argument("path", nargs="?", default="",
+                    help="REST subpath, e.g. 'stages' or 'stages/61/0/taskSummary?quantiles=0.5,0.95,1.0'")
+    pe.add_argument("--rm", default=None, help="YARN RM base URL (or $SCQ_YARN_RM)")
+    pe.add_argument("--compact", action="store_true", help="compact JSON output")
+    pe.set_defaults(func=rest.cmd_exec)
+
+    psk = sub.add_parser("skill", help="Manage the agent skill")
+    sksub = psk.add_subparsers(dest="skcmd", required=True)
+    ski = sksub.add_parser("install", help="Write SKILL.md into the skills dir")
+    ski.add_argument("--dir", default=None, help="Skills dir (or $SKILLS_DIR)")
+    ski.set_defaults(func=cmd_skill_install)
+
+    pj = sub.add_parser("jobs", help="Manage async jobs")
+    js = pj.add_subparsers(dest="jcmd", required=True)
+    js.add_parser("list", help="List jobs").set_defaults(func=jobs.cmd_list)
+    s = js.add_parser("status", help="Show a job's full status")
+    s.add_argument("id"); s.set_defaults(func=jobs.cmd_status)
+    lg = js.add_parser("logs", help="Show a job's log (tail by default)")
+    lg.add_argument("id"); lg.add_argument("--tail", type=int, default=40)
+    lg.add_argument("--full", action="store_true"); lg.set_defaults(func=jobs.cmd_logs)
+    c = js.add_parser("cancel", help="Cancel a running job")
+    c.add_argument("id"); c.set_defaults(func=jobs.cmd_cancel)
+
+    return ap
+
+
+def main(argv=None) -> None:
+    argv = list(sys.argv[1:] if argv is None else argv)
+    # Internal entrypoint for the detached worker.
+    if argv and argv[0] == "__run-job":
+        jobs.run_worker(argv[1])
+        return
+    args = build_parser().parse_args(argv)
+    args.func(args)

spark_connect_cli/jobs.py

@@ -0,0 +1,221 @@
+"""Layer A — async background-job control.
+
+The point: long Spark jobs must not block the caller (an LLM agent should not sit
+in a 30-minute tool call). `submit()` spawns the work detached in its own process
+group, records a durable handle on disk, and returns immediately. The handle
+survives a process/container restart because it lives in a file registry, not in
+memory.
+
+A job is generic: `kind="exec"` runs an arbitrary argv (used for tests and ad-hoc
+long commands); `kind="sync"` runs the Hive->ClickHouse mover. New kinds plug in
+via `_dispatch`.
+"""
+from __future__ import annotations
+
+import json
+import os
+import signal
+import subprocess
+import sys
+import time
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+from .session import EXIT_JOB_ERR, err
+
+JOBS_DIR = Path(os.environ.get(
+    "SCQ_JOBS_DIR", str(Path.home() / ".spark-connect-cli" / "jobs")))
+
+
+def _now() -> str:
+    return datetime.now(timezone.utc).astimezone().isoformat(timespec="seconds")
+
+
+def _job_dir(job_id: str) -> Path:
+    return JOBS_DIR / job_id
+
+
+def _meta_path(job_id: str) -> Path:
+    return _job_dir(job_id) / "meta.json"
+
+
+def _log_path(job_id: str) -> Path:
+    return _job_dir(job_id) / "out.log"
+
+
+def read_meta(job_id: str) -> dict:
+    p = _meta_path(job_id)
+    if not p.exists():
+        err(f"no such job: {job_id}", EXIT_JOB_ERR)
+    return json.loads(p.read_text())
+
+
+def write_meta(job_id: str, meta: dict) -> None:
+    tmp = _meta_path(job_id).with_suffix(".tmp")
+    tmp.write_text(json.dumps(meta, indent=2, default=str))
+    tmp.replace(_meta_path(job_id))
+
+
+def _pid_alive(pid: int) -> bool:
+    if not pid:
+        return False
+    try:
+        os.kill(pid, 0)
+    except ProcessLookupError:
+        return False
+    except PermissionError:
+        return True
+    # Signal-0 succeeds for a zombie (terminated but not yet reaped by its
+    # parent), but a zombie is not doing work — treat it as dead. On Linux the
+    # process state is the char right after the ")" that closes comm in stat.
+    try:
+        with open(f"/proc/{pid}/stat") as f:
+            stat = f.read()
+        if stat[stat.rfind(")") + 2] == "Z":
+            return False
+    except (FileNotFoundError, ProcessLookupError, IndexError):
+        return False
+    return True
+
+
+def reconcile(meta: dict) -> dict:
+    """A 'running' job whose process is gone but that never recorded an end has
+    crashed — mark it failed so status never lies."""
+    if meta.get("state") == "running" and not _pid_alive(meta.get("pid", 0)):
+        meta["state"] = "failed"
+        meta["ended_at"] = _now()
+        if meta.get("exit_code") is None:
+            meta["exit_code"] = -1
+        meta["error"] = meta.get("error") or "process exited without recording completion"
+        write_meta(meta["id"], meta)
+    return meta
+
+
+def _new_job_id() -> str:
+    return f"j-{datetime.now().strftime('%Y%m%d-%H%M%S')}-{uuid.uuid4().hex[:4]}"
+
+
+def submit(kind: str, argv: list[str], descr: dict | None = None) -> str:
+    """Spawn a detached worker for this job and return its id immediately."""
+    JOBS_DIR.mkdir(parents=True, exist_ok=True)
+    job_id = _new_job_id()
+    _job_dir(job_id).mkdir(parents=True)
+    meta = {
+        "id": job_id, "kind": kind, "state": "submitted",
+        "submitted_at": _now(), "started_at": None, "ended_at": None,
+        "pid": None, "pgid": None, "exit_code": None, "argv": argv,
+        **(descr or {}),
+    }
+    write_meta(job_id, meta)
+
+    log = open(_log_path(job_id), "ab", buffering=0)
+    proc = subprocess.Popen(
+        [sys.executable, "-m", "spark_connect_cli", "__run-job", job_id],
+        stdout=log, stderr=subprocess.STDOUT, stdin=subprocess.DEVNULL,
+        start_new_session=True,  # own process group -> kill the whole tree
+        env=os.environ.copy(),
+    )
+    meta["pid"] = proc.pid
+    meta["pgid"] = os.getpgid(proc.pid)
+    meta["state"] = "running"
+    meta["started_at"] = _now()
+    write_meta(job_id, meta)
+    return job_id
+
+
+def _dispatch(meta: dict) -> int:
+    kind = meta["kind"]
+    argv = meta.get("argv", [])
+    if kind == "exec":
+        return subprocess.run(argv).returncode
+    if kind == "sync":
+        from .sync import run as sync_run
+        return sync_run(argv, meta)
+    print(f"[scq] unknown job kind: {kind}", flush=True)
+    return 2
+
+
+def run_worker(job_id: str) -> None:
+    """Runs INSIDE the detached child. Executes the work, then records a terminal
+    state. Never raises out — always writes a final meta."""
+    meta = read_meta(job_id)
+    rc = 0
+    try:
+        rc = _dispatch(meta)
+    except SystemExit as e:
+        rc = int(e.code) if isinstance(e.code, int) else 1
+    except Exception as e:  # noqa: BLE001
+        meta = read_meta(job_id)
+        meta["error"] = str(e)
+        print(f"[scq] job failed: {e}", flush=True)
+        rc = 1
+    meta = read_meta(job_id)  # re-read: the body may have updated counters
+    meta["state"] = "succeeded" if rc == 0 else "failed"
+    meta["exit_code"] = rc
+    meta["ended_at"] = _now()
+    write_meta(job_id, meta)
+    sys.exit(rc)
+
+
+# -- agent-facing commands -------------------------------------------------
+
+def cmd_list(args) -> None:
+    if not JOBS_DIR.exists():
+        print(json.dumps({"jobs": []}))
+        return
+    out = []
+    for d in sorted(JOBS_DIR.iterdir(), reverse=True):
+        if not (d / "meta.json").exists():
+            continue
+        m = reconcile(json.loads((d / "meta.json").read_text()))
+        out.append({k: m.get(k) for k in
+                    ("id", "kind", "state", "source", "target", "submitted_at",
+                     "source_rows", "written_rows")})
+    if args.format == "table":
+        from .session import emit_rows
+        emit_rows([(j["id"], j["kind"], j["state"], j.get("source") or "",
+                    str(j.get("written_rows") if j.get("written_rows") is not None else ""))
+                   for j in out],
+                  ["id", "kind", "state", "source", "written"], "table")
+    else:
+        print(json.dumps({"jobs": out}, default=str))
+
+
+def cmd_status(args) -> None:
+    print(json.dumps(reconcile(read_meta(args.id)), indent=2, default=str))
+
+
+def cmd_logs(args) -> None:
+    read_meta(args.id)  # validate existence
+    p = _log_path(args.id)
+    if not p.exists():
+        print("")
+        return
+    data = p.read_text(errors="replace")
+    if args.full:
+        sys.stdout.write(data)
+        return
+    lines = data.splitlines()
+    sys.stdout.write("\n".join(lines[-args.tail:]) + ("\n" if lines else ""))
+
+
+def cmd_cancel(args) -> None:
+    meta = reconcile(read_meta(args.id))
+    if meta["state"] not in ("running", "submitted"):
+        print(json.dumps({"id": args.id, "state": meta["state"],
+                          "message": "job already finished"}))
+        return
+    pgid = meta.get("pgid")
+    if pgid:
+        try:
+            os.killpg(pgid, signal.SIGTERM)
+            time.sleep(2)
+            if _pid_alive(meta.get("pid", 0)):
+                os.killpg(pgid, signal.SIGKILL)
+        except ProcessLookupError:
+            pass
+    meta["state"] = "cancelled"
+    meta["ended_at"] = _now()
+    write_meta(args.id, meta)
+    print(json.dumps({"id": args.id, "state": "cancelled"}))

spark_connect_cli/meta.py

@@ -0,0 +1,95 @@
+"""scq meta — one structured metadata document for a table.
+
+Bundles what is otherwise spread across DESCRIBE EXTENDED + SHOW PARTITIONS +
+the Spark `_metadata` hidden column, so an agent gets the whole picture of a
+table in a single call instead of stitching several queries.
+"""
+from __future__ import annotations
+
+import json
+
+from .session import EXIT_QUERY_ERR, err, get_spark
+from .sync import map_type
+
+
+def _describe_extended(spark, table):
+    """Parse DESCRIBE EXTENDED into (columns, partition_columns, details)."""
+    rows = spark.sql(f"DESCRIBE EXTENDED {table}").collect()
+    cols, part_cols, details = [], [], {}
+    section = "cols"
+    for r in rows:
+        name = (r[0] or "").strip()
+        val = (r[1] or "")
+        if name.startswith("# Partition Information"):
+            section = "partcols"
+            continue
+        if name.startswith("# Detailed Table Information"):
+            section = "details"
+            continue
+        if name.startswith("#") or not name:
+            continue
+        if section == "cols":
+            cols.append({"name": name, "type": val, "clickhouse": map_type(val),
+                         "comment": r[2]})
+        elif section == "partcols":
+            part_cols.append(name)
+        else:
+            details[name] = val.strip()
+    return cols, part_cols, details
+
+
+def _file_stats(spark, table):
+    """Aggregate per-file size/mtime from the _metadata hidden column. Returns
+    None if the source doesn't expose _metadata."""
+    try:
+        row = spark.sql(
+            f"SELECT count(*) AS num_files, sum(sz) AS total_bytes, "
+            f"min(mt) AS first_modified, max(mt) AS last_modified FROM ("
+            f"  SELECT _metadata.file_path AS p, max(_metadata.file_size) AS sz, "
+            f"  max(_metadata.file_modification_time) AS mt FROM {table} GROUP BY 1)"
+        ).collect()[0]
+        return {"numFiles": row["num_files"], "totalBytes": row["total_bytes"],
+                "firstModified": str(row["first_modified"]),
+                "lastModified": str(row["last_modified"])}
+    except Exception:  # noqa: BLE001 — _metadata unsupported / empty table
+        return None
+
+
+def cmd_meta(args) -> None:
+    spark = get_spark(args.remote)
+    table = args.table
+    try:
+        cols, part_cols, details = _describe_extended(spark, table)
+    except Exception as e:  # noqa: BLE001
+        err(f"describe failed: {e}", EXIT_QUERY_ERR)
+
+    out = {
+        "table": table,
+        "createdTime": details.get("Created Time"),
+        "lastAccess": details.get("Last Access"),
+        "owner": details.get("Owner"),
+        "createdBy": details.get("Created By"),
+        "provider": details.get("Provider"),
+        "type": details.get("Type"),
+        "location": details.get("Location"),
+        "statistics": details.get("Statistics"),
+        "partitionColumns": part_cols,
+        "columns": cols,
+    }
+
+    if part_cols:
+        try:
+            parts = [r[0] for r in spark.sql(f"SHOW PARTITIONS {table}").collect()]
+            out["partitionCount"] = len(parts)
+            out["partitions"] = parts if len(parts) <= 200 else parts[:200] + ["…"]
+        except Exception:  # noqa: BLE001
+            pass
+
+    files = _file_stats(spark, table)
+    if files:
+        out["files"] = files
+
+    if args.count:
+        out["rowCount"] = spark.sql(f"SELECT count(*) c FROM {table}").collect()[0]["c"]
+
+    print(json.dumps(out, ensure_ascii=False, indent=2, default=str))

spark_connect_cli/query.py

@@ -0,0 +1,57 @@
+"""Read-only / discovery commands over Spark Connect."""
+from __future__ import annotations
+
+import json
+import sys
+
+from .session import (DEFAULT_MAX_ROWS, EXIT_BLOCKED, EXIT_QUERY_ERR, emit_rows,
+                      err, get_spark, is_read_only)
+
+
+def cmd_databases(args) -> None:
+    spark = get_spark(args.remote)
+    rows = spark.sql("SHOW DATABASES").collect()
+    emit_rows([(r[0],) for r in rows], ["database"], args.format)
+
+
+def cmd_tables(args) -> None:
+    spark = get_spark(args.remote)
+    db = args.database or "default"
+    rows = spark.sql(f"SHOW TABLES IN `{db}`").collect()
+    out = [(r["namespace"], r["tableName"]) for r in rows]
+    if args.like:
+        pat = args.like.replace("%", "").lower()
+        out = [t for t in out if pat in t[1].lower()]
+    emit_rows(out, ["database", "table"], args.format)
+
+
+def cmd_describe(args) -> None:
+    spark = get_spark(args.remote)
+    rows = spark.sql(f"DESCRIBE TABLE {args.table}").collect()
+    emit_rows([(r[0], r[1], r[2]) for r in rows],
+              ["col_name", "data_type", "comment"], args.format)
+
+
+def cmd_query(args) -> None:
+    sql = args.sql
+    if not args.allow_ddl and not is_read_only(sql):
+        err("write/DDL blocked by read-only guard; pass --allow-ddl to override",
+            EXIT_BLOCKED)
+    spark = get_spark(args.remote)
+    try:
+        df = spark.sql(sql)
+    except Exception as e:  # noqa: BLE001
+        err(f"query failed: {e}", EXIT_QUERY_ERR)
+    if not is_read_only(sql):
+        print(json.dumps({"ok": True}))
+        return
+    max_rows = args.max_rows if args.max_rows is not None else DEFAULT_MAX_ROWS
+    columns = df.columns
+    limited = df.limit(max_rows + 1).collect() if max_rows > 0 else df.collect()
+    truncated = max_rows > 0 and len(limited) > max_rows
+    rows = limited[:max_rows] if truncated else limited
+    emit_rows([tuple(r) for r in rows], columns, args.format)
+    if truncated:
+        print(json.dumps({"warning": f"result capped at {max_rows} rows; "
+                                     "add LIMIT/filters or raise --max-rows"}),
+              file=sys.stderr)

spark_connect_cli/rest.py

@@ -0,0 +1,53 @@
+"""scq exec — read-only passthrough to the Spark REST API.
+
+One generic accessor over the Spark monitoring REST API instead of a pile of
+single-purpose diagnostics. The model composes the path and interprets the JSON,
+so skew, slow stages, shuffle spill, executor GC/OOM, etc. are all the same
+command with different readings.
+
+Access path: the Spark UI redirects to the YARN ResourceManager web proxy, so we
+discover the running Spark application via the RM REST and go through the proxy.
+Pure-Python (urllib) — no curl, no manual app id. GET-only against /api/v1.
+"""
+from __future__ import annotations
+
+import json
+import os
+import urllib.request
+
+from .session import EXIT_CONN_ERR, err
+
+DEFAULT_RM = os.environ.get("SCQ_YARN_RM", "http://namenode.hive-net:8088")
+
+
+def _get_json(url: str, timeout: int = 15):
+    with urllib.request.urlopen(url, timeout=timeout) as r:  # noqa: S310 — fixed RM base
+        return json.load(r)
+
+
+def _discover_app(rm: str) -> str | None:
+    apps = _get_json(f"{rm}/ws/v1/cluster/apps?states=RUNNING&applicationTypes=SPARK")
+    lst = ((apps.get("apps") or {}).get("app")) or []
+    for a in lst:  # prefer the Spark Connect Server
+        if "connect" in (a.get("name") or "").lower():
+            return a["id"]
+    return lst[0]["id"] if lst else None
+
+
+def cmd_exec(args) -> None:
+    rm = (args.rm or DEFAULT_RM).rstrip("/")
+    path = (args.path or "").lstrip("/")
+    if ".." in path:
+        err("path must not contain '..'", EXIT_CONN_ERR)
+    try:
+        app = _discover_app(rm)
+        if not app:
+            err("no RUNNING Spark application found via the YARN RM", EXIT_CONN_ERR)
+        base = f"{rm}/proxy/{app}/api/v1/applications/{app}"
+        data = _get_json(f"{base}/{path}" if path else base)
+    except SystemExit:
+        raise
+    except Exception as e:  # noqa: BLE001
+        err(f"Spark REST request failed: {e}", EXIT_CONN_ERR)
+    print(json.dumps(data, ensure_ascii=False,
+                     indent=None if args.compact else 2, default=str))

spark_connect_cli/session.py

@@ -0,0 +1,72 @@
+"""Spark Connect session, read-only guard, and output formatting.
+
+Connecting to Spark Connect needs NO Kerberos and NO JVM on the client side —
+the server runs under its own keytab and does the auth. The endpoint is a plain
+gRPC address, e.g. sc://spark-connect:15002.
+"""
+from __future__ import annotations
+
+import json
+import os
+import sys
+
+DEFAULT_REMOTE = os.environ.get("SPARK_REMOTE", "sc://localhost:15002")
+DEFAULT_MAX_ROWS = int(os.environ.get("SCQ_MAX_ROWS", "10000"))
+
+# Exit codes — stable contract so an agent can branch on them.
+EXIT_OK = 0
+EXIT_QUERY_ERR = 1
+EXIT_CONN_ERR = 2
+EXIT_BLOCKED = 3      # read-only guard tripped
+EXIT_JOB_ERR = 4      # job-control error (no such job, etc.)
+
+READ_ONLY_LEADERS = ("select", "show", "describe", "desc", "explain", "with")
+
+
+def err(msg: object, code: int) -> None:
+    """Emit a single JSON error object on stderr and exit."""
+    print(json.dumps({"error": str(msg), "code": code}), file=sys.stderr)
+    sys.exit(code)
+
+
+def is_read_only(sql: str) -> bool:
+    """True if the statement only reads (SELECT/SHOW/DESCRIBE/EXPLAIN/WITH)."""
+    leader = sql.strip().lstrip("(").split(None, 1)
+    return bool(leader) and leader[0].lower() in READ_ONLY_LEADERS
+
+
+def get_spark(remote: str):
+    try:
+        from pyspark.sql import SparkSession
+    except ModuleNotFoundError:
+        err("pyspark is not installed; `pip install 'pyspark[connect]>=3.5,<4'`", EXIT_CONN_ERR)
+    try:
+        return SparkSession.builder.remote(remote).getOrCreate()
+    except Exception as e:  # noqa: BLE001
+        err(f"could not connect to Spark Connect at {remote}: {e}", EXIT_CONN_ERR)
+
+
+def emit_rows(rows, columns, fmt: str) -> None:
+    """Render a result set. JSON-first (JSONEachRow) by default."""
+    if fmt == "jsoneachrow":
+        for r in rows:
+            print(json.dumps(dict(zip(columns, r)), default=str, ensure_ascii=False))
+    elif fmt == "json":
+        print(json.dumps(
+            {"meta": list(columns), "data": [list(r) for r in rows], "rows": len(rows)},
+            default=str, ensure_ascii=False))
+    elif fmt in ("csv", "tsv"):
+        sep = "," if fmt == "csv" else "\t"
+        print(sep.join(columns))
+        for r in rows:
+            print(sep.join("" if v is None else str(v) for v in r))
+    elif fmt == "table":
+        widths = [len(c) for c in columns]
+        srows = [["" if v is None else str(v) for v in r] for r in rows]
+        for r in srows:
+            for i, v in enumerate(r):
+                widths[i] = max(widths[i], len(v))
+        print(" | ".join(c.ljust(widths[i]) for i, c in enumerate(columns)))
+        print("-+-".join("-" * w for w in widths))
+        for r in srows:
+            print(" | ".join(v.ljust(widths[i]) for i, v in enumerate(r)))

spark_connect_cli/sync.py

@@ -0,0 +1,146 @@
+"""Hive -> ClickHouse sync — one feature built on the async job subsystem.
+
+Runs inside a detached job worker, so all output goes to the job log and never
+into the agent's context. The data path is **Spark direct write**: a Spark
+Connect job reads the Hive table and writes to ClickHouse over JDBC. The write
+happens on the executors (in the cluster), so rows never pass through this
+process or the agent.
+
+Requirements (the "path A wiring"):
+  - clickhouse-jdbc on the Spark Connect server classpath (/opt/spark/jars/),
+  - network egress from the cluster to ClickHouse,
+  - a JDBC URL with credentials (`--ch-jdbc` / $SCQ_CH_JDBC),
+  - the target ClickHouse table already created with a suitable engine
+    (Spark `append` does not create a usable MergeTree table for you).
+
+Modes:
+  single   — one JDBC connection (numPartitions=1). Best for small tables.
+  parallel — N partitions write concurrently. Best for large tables.
+  auto     — single under --auto-threshold rows, else parallel.
+"""
+from __future__ import annotations
+
+import argparse
+import os
+import re
+
+from .jobs import write_meta
+from .session import DEFAULT_REMOTE, get_spark
+
+# Spark/Hive -> ClickHouse type mapping. The SKILL carries the authoritative
+# table the agent reasons with; this is just for the descriptive log line.
+SPARK_TO_CH = {
+    "boolean": "Bool", "tinyint": "Int8", "smallint": "Int16", "int": "Int32",
+    "integer": "Int32", "bigint": "Int64", "float": "Float32", "double": "Float64",
+    "string": "String", "varchar": "String", "char": "String", "binary": "String",
+    "date": "Date32", "timestamp": "DateTime64(3)",
+}
+
+AUTO_THRESHOLD = int(os.environ.get("SCQ_AUTO_THRESHOLD", "1000000"))
+DEFAULT_BATCHSIZE = int(os.environ.get("SCQ_BATCHSIZE", "100000"))
+DEFAULT_NUM_PARTITIONS = int(os.environ.get("SCQ_NUM_PARTITIONS", "8"))
+
+
+def map_type(spark_type: str) -> str:
+    t = spark_type.lower().strip()
+    if t.startswith("decimal"):
+        return t.replace("decimal", "Decimal")
+    base = re.split(r"[(<]", t, 1)[0]
+    return SPARK_TO_CH.get(base, "String")
+
+
+def _parse(argv: list[str]):
+    p = argparse.ArgumentParser(prog="scq sync")
+    p.add_argument("source")
+    p.add_argument("--to", default="clickhouse")
+    p.add_argument("--remote", default=DEFAULT_REMOTE)
+    p.add_argument("--mode", choices=["auto", "parallel", "single"], default="auto")
+    p.add_argument("--ch-jdbc", default=os.environ.get("SCQ_CH_JDBC", ""))
+    p.add_argument("--target", default=None)
+    p.add_argument("--where", default=None)
+    p.add_argument("--limit", type=int, default=0)
+    p.add_argument("--batchsize", type=int, default=DEFAULT_BATCHSIZE)
+    p.add_argument("--num-partitions", type=int, default=DEFAULT_NUM_PARTITIONS)
+    p.add_argument("--auto-threshold", type=int, default=AUTO_THRESHOLD)
+    # Auto-create control: when the target table doesn't exist, Spark creates it.
+    # Without an explicit sort key ClickHouse defaults to ORDER BY tuple() (no
+    # primary index). --order-by injects a real key via createTableOptions.
+    p.add_argument("--order-by", default=None, help="ORDER BY key(s) for auto-created table, e.g. 'id' or 'id,dt'")
+    p.add_argument("--engine", default="MergeTree", help="Engine for auto-created table (default MergeTree)")
+    return p.parse_args(argv)
+
+
+def run(argv: list[str], meta: dict) -> int:
+    a = _parse(argv)
+    if not a.ch_jdbc:
+        print("[scq] no --ch-jdbc / SCQ_CH_JDBC set — cannot write to ClickHouse",
+              flush=True)
+        return 2
+
+    print(f"[scq] sync start: {a.source} -> {a.to}  mode={a.mode}", flush=True)
+    spark = get_spark(a.remote)
+
+    # 1. discover schema (informational; the target table must already exist)
+    desc = spark.sql(f"DESCRIBE TABLE {a.source}").collect()
+    cols = [(r[0], r[1]) for r in desc if r[0] and not r[0].startswith("#")]
+    print(f"[scq] {len(cols)} columns: "
+          + ", ".join(f"{c}:{t}->{map_type(t)}" for c, t in cols), flush=True)
+
+    src_count = spark.sql(f"SELECT count(*) c FROM {a.source}").collect()[0]["c"]
+    # Target may be `db.table` (explicit database) or a bare table name (lands in
+    # the JDBC connection's default database). Keep the landing spot explicit.
+    target = a.target or a.source.split(".")[-1]
+    qualified = "." in target
+    meta["source_rows"] = src_count
+    meta["target"] = target
+    write_meta(meta["id"], meta)
+    hint = "" if qualified else "  (connection default database; pass --target db.table to choose one)"
+    print(f"[scq] source rows: {src_count} -> target {target}{hint}", flush=True)
+
+    # 2. build the read
+    sel = f"SELECT * FROM {a.source}"
+    if a.where:
+        sel += f" WHERE {a.where}"
+    if a.limit:
+        sel += f" LIMIT {a.limit}"
+    df = spark.sql(sel)
+
+    # 3. choose write parallelism
+    mode = a.mode
+    if mode == "auto":
+        mode = "parallel" if src_count >= a.auto_threshold else "single"
+    num_partitions = 1 if mode == "single" else max(1, a.num_partitions)
+    if num_partitions == 1:
+        df = df.coalesce(1)
+    else:
+        df = df.repartition(num_partitions)
+    print(f"[scq] writing via JDBC: mode={mode} numPartitions={num_partitions} "
+          f"batchsize={a.batchsize}", flush=True)
+
+    # 4. Spark direct write to ClickHouse. Rows are written by the executors;
+    # nothing flows through this process.
+    writer = (df.write.format("jdbc")
+              .option("url", a.ch_jdbc)
+              .option("dbtable", target)
+              .option("driver", "com.clickhouse.jdbc.ClickHouseDriver")
+              .option("batchsize", a.batchsize)
+              .option("isolationLevel", "NONE"))  # ClickHouse has no txns
+    # createTableOptions only affects auto-create (when the table is missing); it
+    # is ignored when the table already exists.
+    if a.order_by:
+        writer = writer.option("createTableOptions",
+                               f"ENGINE = {a.engine} ORDER BY ({a.order_by})")
+        print(f"[scq] auto-create (if needed): ENGINE = {a.engine} ORDER BY ({a.order_by})", flush=True)
+    else:
+        print("[scq] note: an auto-created target uses ORDER BY tuple() (no sort key) — "
+              "pass --order-by for a real key, or pre-create the table", flush=True)
+    try:
+        writer.mode("append").save()
+    except Exception as e:  # noqa: BLE001
+        print(f"[scq] JDBC write failed: {e}", flush=True)
+        return 1
+
+    print(f"[scq] done: wrote {src_count} rows to {target}", flush=True)
+    meta["written_rows"] = src_count
+    write_meta(meta["id"], meta)
+    return 0

spark_connect_cli-0.2.0.dist-info/METADATA

@@ -0,0 +1,156 @@
+Metadata-Version: 2.4
+Name: spark-connect-cli
+Version: 0.2.0
+Summary: Agent-friendly Spark Connect CLI: read-only querying + async long-job control. No JVM, no Kerberos on the client.
+Project-URL: Homepage, https://github.com/dengshu2/spark-connect-cli
+Project-URL: Issues, https://github.com/dengshu2/spark-connect-cli/issues
+Author: dengshu
+License: MIT
+License-File: LICENSE
+Keywords: agent,cli,clickhouse,hive,llm,spark,spark-connect
+Classifier: Environment :: Console
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Database :: Front-Ends
+Requires-Python: >=3.9
+Requires-Dist: pyspark[connect]<4,>=3.5
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# spark-connect-cli (`scq`)
+
+An agent-friendly [Spark Connect](https://spark.apache.org/spark-connect/) CLI —
+**read-only querying** plus **async control for long-running jobs**.
+
+Built for LLM agents and humans who live in a shell. Unlike `spark-sql` /
+`spark-submit`, the client is a thin **pure-Python gRPC client**: no JVM, and
+**no Kerberos on the client side** — the Spark Connect server authenticates with
+its own keytab, so you just point at `sc://host:15002` and go.
+
+## Why
+
+- **JSON-first, read-only by default.** Safe for an agent to call for
+  exploration; writes/DDL are blocked unless you opt in (`--allow-ddl`).
+- **Long jobs don't block you.** A multi-minute Spark job shouldn't trap an agent
+  in a 30-minute tool call. `scq` submits the job, hands back a durable **job
+  id**, and returns immediately. Poll it whenever you like; the handle survives a
+  client/container restart because it lives in an on-disk registry.
+- **Stable exit codes** so a caller can branch without scraping text.
+
+## Install
+
+```bash
+pip install spark-connect-cli         # once published
+# or, from source:
+pip install -e .
+```
+
+## Quick start
+
+```bash
+export SPARK_REMOTE=sc://localhost:15002   # your Spark Connect endpoint
+
+scq databases
+scq tables mydb --like '%orders%'
+scq describe mydb.orders
+scq query "SELECT id, name FROM mydb.orders LIMIT 10"
+```
+
+Output is **JSONEachRow** (one JSON object per line) by default; pick another with
+`--format json|csv|tsv|table`.
+
+### Read-only guard
+
+`scq query` allows only `SELECT/SHOW/DESCRIBE/EXPLAIN/WITH`. Anything else exits
+with code **3** unless you pass `--allow-ddl`.
+
+| exit | meaning |
+|------|---------|
+| 0 | success |
+| 1 | query error (bad SQL) |
+| 2 | connection error |
+| 3 | blocked by the read-only guard |
+| 4 | job-control error (no such job, …) |
+
+## Async jobs (Layer A)
+
+Long work runs detached and is tracked by a file-based registry under
+`$SCQ_JOBS_DIR` (default `~/.spark-connect-cli/jobs`).
+
+```bash
+# submit — returns a job id immediately, does NOT block
+scq sync ods.orders --to clickhouse
+# {"job_id": "j-20260625-...", "state": "running", "message": "... poll with ..."}
+
+scq jobs list                       # all jobs + state
+scq jobs status j-20260625-...      # full status (rows, timings, pid, exit code)
+scq jobs logs   j-20260625-... --tail 40
+scq jobs cancel j-20260625-...      # kills the whole process group
+```
+
+Design: each job is a directory with `meta.json` (state machine:
+`submitted → running → succeeded|failed|cancelled`) and `out.log`. The worker
+runs in its **own process group**, so cancel kills the entire tree (no orphans).
+A `running` job whose process has vanished is reconciled to `failed` on the next
+status read, so status never lies.
+
+## Hive → ClickHouse sync
+
+`scq sync` is one job kind built on the async subsystem. It uses **Spark direct
+write**: a Spark Connect job reads the Hive table and writes to ClickHouse over
+JDBC. The write runs on the executors, so rows never pass through this process or
+the agent.
+
+Modes control write parallelism — `single` (one connection, small tables),
+`parallel` (N partitions, large tables), `auto` (picks by row count).
+
+Requires:
+- `clickhouse-jdbc` on the Spark Connect server classpath (`/opt/spark/jars/`),
+- cluster→ClickHouse network egress,
+- a JDBC URL with credentials via `--ch-jdbc` / `$SCQ_CH_JDBC`,
+- the **target ClickHouse table created beforehand** with a suitable engine
+  (Spark `append` won't build a usable MergeTree table for you — create it first,
+  e.g. with the `chsql` skill).
+
+## Introspection
+
+```bash
+scq meta db.table            # one JSON: schema, created time, location,
+                             # partitions, file count/size, mtime range
+scq meta db.table --count    # also run an exact count(*)
+
+scq exec stages?status=active            # read-only Spark REST passthrough
+scq exec executors
+scq exec stages/<id>/<attempt>/taskSummary?quantiles=0.5,0.95,1.0   # skew: max/median
+```
+
+`scq exec` auto-discovers the running Spark app via the YARN ResourceManager and
+proxies its monitoring REST API (GET-only). Set the RM base with `$SCQ_YARN_RM`.
+
+**Reading `scq exec executors`** — the `maxMemory` field is Spark's
+**storage/cache pool** (`(heap − 300 MB reserved) × 0.6`), *not* the executor's
+total memory: a 512 MB executor reports ~93 MB, a 1536 MB driver ~741 MB. The
+real heap is `spark.executor.memory` (+ off-heap overhead). The `driver` row has
+0 cores and runs no tasks. With dynamic allocation, idle executors are released —
+so the list may show only the driver when nothing is running.
+
+## Configuration
+
+| env | default | meaning |
+|-----|---------|---------|
+| `SPARK_REMOTE` | `sc://localhost:15002` | Spark Connect endpoint |
+| `SCQ_JOBS_DIR` | `~/.spark-connect-cli/jobs` | job registry (put on a persistent volume) |
+| `SCQ_MAX_ROWS` | `10000` | default row cap for `query` |
+| `SCQ_CH_JDBC` | — | ClickHouse JDBC URL for `sync` path A |
+| `SCQ_YARN_RM` | `http://namenode.hive-net:8088` | YARN RM base for `scq exec` |
+
+## Use with an LLM agent
+
+`SKILL.md` ships a ready-made skill (discover-before-query workflow, async-job
+etiquette, type-mapping table). Drop it into your agent's skills directory and
+the agent drives `scq` through a shell/Bash tool.
+
+## License
+
+MIT

spark_connect_cli-0.2.0.dist-info/RECORD

@@ -0,0 +1,15 @@
+spark_connect_cli/__init__.py,sha256=d_xkc721_-_mG3JGkeXU7BspJ1x_-yufyS9D6dLgR-k,87
+spark_connect_cli/__main__.py,sha256=MSmt_5Xg84uHqzTN38JwgseJK8rsJn_11A8WD99VtEo,61
+spark_connect_cli/cli.py,sha256=8VLGgQIVgTr0OccAT3uLnBpFCsO0i8IYTq1fScYn-9U,6379
+spark_connect_cli/jobs.py,sha256=t55-tBFXd5jNMhiSWL7StrTlgCGPjU_XsYGr5qew7gY,7233
+spark_connect_cli/meta.py,sha256=u1-RaKxP-Y7wbKXppeSuh2d9UCRk1HZ1U-GaFOJ4M9g,3473
+spark_connect_cli/query.py,sha256=Ej0M0yW6RAdpShhEudTGTOJhL2ezSzwovFDPgRhCWH8,2103
+spark_connect_cli/rest.py,sha256=JueQlHgTFhNvl8KCTum-1LGVxJWDs441NVDsU1GjBOE,2021
+spark_connect_cli/session.py,sha256=xSXsSUZTwERj0uGKDyuQQUNveH6ox9LqsAurNiD7AyY,2756
+spark_connect_cli/sync.py,sha256=fNCfqbYO6ClCfBo0Y_CDIfnCbGuEX26bNilKMqieM-4,6554
+spark_connect_cli/SKILL.md,sha256=Rm_9aT_4G24iu2gM8MR6YHp4G_mLLbKMLPANU3-9qSA,6450
+spark_connect_cli-0.2.0.dist-info/METADATA,sha256=RuIoJAQxqZl1Y21Msioj1JrqpTD_kLlpyvSMnbM1BdI,6081
+spark_connect_cli-0.2.0.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
+spark_connect_cli-0.2.0.dist-info/entry_points.txt,sha256=xweUT4medkn5exc6YvKPNBWlOV9qWBz5hxjZsoRdLK4,98
+spark_connect_cli-0.2.0.dist-info/licenses/LICENSE,sha256=CQKNwxelRbTkzP7qdNam0XmENhKgOC6PgWd3ljvRiVM,1064
+spark_connect_cli-0.2.0.dist-info/RECORD,,

spark_connect_cli-0.2.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.30.1
+Root-Is-Purelib: true
+Tag: py3-none-any

spark_connect_cli-0.2.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+scq = spark_connect_cli.cli:main
+spark-connect-cli = spark_connect_cli.cli:main

spark_connect_cli-0.2.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 dengshu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.