spark-connect-cli 0.2.0__tar.gz → 0.2.1__tar.gz

{spark_connect_cli-0.2.0 → spark_connect_cli-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: spark-connect-cli
-Version: 0.2.0
+Version: 0.2.1
 Summary: Agent-friendly Spark Connect CLI: read-only querying + async long-job control. No JVM, no Kerberos on the client.
 Project-URL: Homepage, https://github.com/dengshu2/spark-connect-cli
 Project-URL: Issues, https://github.com/dengshu2/spark-connect-cli/issues
@@ -151,6 +151,14 @@ so the list may show only the driver when nothing is running.
 etiquette, type-mapping table). Drop it into your agent's skills directory and
 the agent drives `scq` through a shell/Bash tool.
 
+## Roadmap
+
+- Clarify in `SKILL.md` that `scq exec executors` `maxMemory` is the storage
+  pool, not total memory (already noted above).
+- `scq cluster` — optional read-only passthrough to the YARN ResourceManager
+  REST (apps / queues / nodes), rounding out the introspection plane.
+- Vendored/offline install path (bundle wheels) for air-gapped deployments.
+
 ## License
 
 MIT

{spark_connect_cli-0.2.0 → spark_connect_cli-0.2.1}/README.md

@@ -131,6 +131,14 @@ so the list may show only the driver when nothing is running.
 etiquette, type-mapping table). Drop it into your agent's skills directory and
 the agent drives `scq` through a shell/Bash tool.
 
+## Roadmap
+
+- Clarify in `SKILL.md` that `scq exec executors` `maxMemory` is the storage
+  pool, not total memory (already noted above).
+- `scq cluster` — optional read-only passthrough to the YARN ResourceManager
+  REST (apps / queues / nodes), rounding out the introspection plane.
+- Vendored/offline install path (bundle wheels) for air-gapped deployments.
+
 ## License
 
 MIT

{spark_connect_cli-0.2.0 → spark_connect_cli-0.2.1}/SKILL.md

@@ -129,6 +129,10 @@ scq exec jobs
 scq exec stages/<id>/<attempt>/taskSummary?quantiles=0.5,0.95,1.0
 ```
 
+- **`executors` memory**: `maxMemory` / `memoryUsed` are the **storage/cache
+  pool** (roughly `(heap − 300MB) × 0.6`), **not** the executor's total memory.
+  A ~100MB `maxMemory` does **not** mean a tiny executor — total heap is set by
+  `spark.executor.memory`. Don't report the cache pool as the executor size.
 - **Data skew**: pull a stage's `taskSummary` and compare a metric's **max vs
   median** (`executorRunTime`, `shuffleReadBytes`, `shuffleReadRecords`). A large
   `max/median` ratio = a straggler / skewed partition. `…?details=true` on a

{spark_connect_cli-0.2.0 → spark_connect_cli-0.2.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "spark-connect-cli"
-version = "0.2.0"
+version = "0.2.1"
 description = "Agent-friendly Spark Connect CLI: read-only querying + async long-job control. No JVM, no Kerberos on the client."
 readme = "README.md"
 requires-python = ">=3.9"

{spark_connect_cli-0.2.0 → spark_connect_cli-0.2.1}/src/spark_connect_cli/__init__.py

@@ -1,2 +1,2 @@
 """spark-connect-cli — an agent-friendly Spark Connect CLI."""
-__version__ = "0.2.0"
+__version__ = "0.2.1"