dataframe-toolkit 0.11.2__tar.gz → 0.12.0__tar.gz

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: dataframe-toolkit
-Version: 0.11.2
+Version: 0.12.0
 Summary: A command-line toolkit for DataFrame analysis and manipulation
 Keywords: dataframe,statistics,data analysis,CLI,TSV,pandas
 Author: Gunnar Pálsson
@@ -20,6 +20,7 @@ Requires-Dist: statsmodels>=0.14
 Requires-Dist: patsy>=0.5.6
 Requires-Dist: duckdb>=0.8
 Requires-Dist: matplotlib>=3.7
+Requires-Dist: pillow>=13.0
 Requires-Dist: seaborn>=0.12
 Requires-Dist: tqdm>=4.65
 Requires-Python: >=3.12
@@ -76,17 +77,20 @@ uv tool install .
 
 ### Skill for coding agents
 
-`skills/dftk/` is an agent skill: a short guide that coding agents such as
-Codex and Claude Code load when a task involves dftk. It covers the piping
+dftk ships an agent skill: a short guide that coding agents such as Codex
+and Claude Code load when a task involves dftk. It covers the piping
 conventions, which command to use for what, and patterns for Snakemake and
-Make. Copy it into your agent's skills directory, for example:
+Make. Install it with:
 
 ```bash
-cp -r skills/dftk ~/.codex/skills/     # Codex
-cp -r skills/dftk ~/.claude/skills/    # Claude Code
+dftk skill install            # every agent found: ~/.claude and/or ~/.codex
+dftk skill install --codex    # only one (--claude or --codex; creates the folder)
+dftk skill install --dir DIR  # another agent's skills folder
 ```
 
-Copy it again after upgrading dftk so it matches the installed version.
+It prints where it installed the skill. Run it again after upgrading dftk
+so the skill matches the installed version; it replaces only the earlier
+dftk skill. The source is in `src/dftk/skills/dftk/`.
 
 ## Quick example
 
@@ -94,15 +98,15 @@ Copy it again after upgrading dftk so it matches the installed version.
 dftk dataset tips | dftk stat - -c tip total_bill -g day
 ```
 
-Chain commands with `-o` (write to a temp parquet, print its path) and `...`
-(read a parquet path from stdin) instead of piping TSV text directly — faster
-for large data, and the recommended pattern once a pipeline has more than one
-step:
+Chain commands with a bare `-o` (write a temp parquet, print its path)
+instead of piping TSV text: the next command sees the path on stdin and reads
+that file. It's faster for large data, keeps column types exact, and is the
+recommended pattern once a pipeline has more than one step:
 
 ```bash
 dftk dataset tips -o \
-  | dftk pivot ... -i day -v tip -f mean -o \
-  | dftk print ...
+  | dftk pivot -i day -v tip -f mean -o \
+  | dftk print
 ```
 
 ## Subcommands
@@ -162,7 +166,7 @@ publication-quality output.
 | `sample`   | Random row sampling (with or without replacement, by count or fraction)   |
 | `split`    | Split a dataframe into one file per group                                 |
 | `annotate` | Read and write provenance metadata (genome, source, …) in parquet files  |
-| `print`    | Read any dftk input (TSV, stdin, `...` parquet pipe) and write TSV     |
+| `print`    | Read any dftk input (TSV, parquet, stdin) and write TSV                |
 | `clean`    | Remove leftover temp parquet pipe files from interrupted pipelines       |
 | `help`     | List all subcommands or show full help for one: `dftk help stat`       |
 
@@ -173,8 +177,8 @@ publication-quality output.
 ```bash
 # z-score sepal_length within species, then fit against petal_length
 dftk dataset iris -o \
-  | dftk scale ... -c sepal_length -g species -o \
-  | dftk fit ... -f "sepal_length_scaled ~ petal_length" -g species
+  | dftk scale -c sepal_length -g species -o \
+  | dftk fit -f "sepal_length_scaled ~ petal_length" -g species
 ```
 
 ### Clean up spreadsheet headers
@@ -183,8 +187,8 @@ Headers such as `Country Name` or `1960` need quoting in SQL. `cleannames` turns
 
 ```bash
 dftk dataset fertility -o \
-  | dftk cleannames ... -o \
-  | dftk query ... --sql "SELECT country_name, x1960, x2000 FROM data LIMIT 3"
+  | dftk cleannames -o \
+  | dftk query --sql "SELECT country_name, x1960, x2000 FROM data LIMIT 3"
 ```
 
 For your own export, `dftk cleannames data.tsv --backend duckdb -o data.parquet` also keeps integer columns with empty cells as integers.
@@ -195,7 +199,7 @@ With tens of thousands of points a scatter plot is a solid blob. `--hexbin` coun
 
 ```bash
 dftk dataset diamonds -o \
-  | dftk scat ... -x carat -y price --hexbin --hexlog --logx --logy -f diamonds.png
+  | dftk scat -x carat -y price --hexbin --hexlog --logx --logy -f diamonds.png
 ```
 
 `--colorcol COL` colours each hexagon by the mean of `COL` instead of the count; with `--subgraphcol` all panels share one colour scale.
@@ -204,8 +208,8 @@ dftk dataset diamonds -o \
 
 ```bash
 dftk dataset tips -o \
-  | dftk stat ... -c tip -g day -o \
-  | dftk line ... -x day -y mean --yerr sem -f fig.png
+  | dftk stat -c tip -g day -o \
+  | dftk line -x day -y mean --yerr sem -f fig.png
 ```
 
 ### Wide-to-long then plot overlaid histograms
@@ -249,18 +253,18 @@ suffix `_r` from `merge`; fill from it, then drop it. Here the defaults are
 each species' median body mass:
 
 ```bash
-dftk dataset penguins -o | dftk pivot ... -i species -v body_mass_g \
+dftk dataset penguins -o | dftk pivot -i species -v body_mass_g \
   -f median --move median_body_mass_g:body_mass_g -o defaults.parquet
 dftk dataset penguins -o \
-  | dftk merge ... -r defaults.parquet -k species -t left -o \
-  | dftk eval ... -f "body_mass_g = body_mass_g.fillna(body_mass_g_r)" \
+  | dftk merge -r defaults.parquet -k species -t left -o \
+  | dftk eval -f "body_mass_g = body_mass_g.fillna(body_mass_g_r)" \
       --drop body_mass_g_r
 ```
 
 **From neighbouring rows** (`co2` is a weekly series with gaps):
 
 ```bash
-dftk dataset co2 -o | dftk eval ... -f "ffill = co2.ffill()" \
+dftk dataset co2 -o | dftk eval -f "ffill = co2.ffill()" \
   -f "bfill = co2.bfill()" -f "linear = co2.interpolate()"
 ```
 
@@ -278,9 +282,9 @@ to be next to each other, but within each group they must be in order:
 
 ```bash
 dftk dataset healthexp -o \
-  | dftk eval ... -f "le = Life_Expectancy.where(Year % 10 != 5)" \
+  | dftk eval -f "le = Life_Expectancy.where(Year % 10 != 5)" \
       --sortasc Country Year -o \
-  | dftk eval ... -f "le_filled = le.groupby(Country).ffill()"
+  | dftk eval -f "le_filled = le.groupby(Country).ffill()"
 ```
 
 `interpolate()` has no grouped form. Once each group's rows are together,
@@ -290,14 +294,78 @@ with something before and after them in the same group:
 
 ```bash
 dftk dataset healthexp -o \
-  | dftk eval ... -f "le = Life_Expectancy.where(Year % 10 != 5)" \
+  | dftk eval -f "le = Life_Expectancy.where(Year % 10 != 5)" \
       --sortasc Country Year -o \
-  | dftk eval ... -f "before = le.groupby(Country).ffill().notna()" \
+  | dftk eval -f "before = le.groupby(Country).ffill().notna()" \
       -f "after = le.groupby(Country).bfill().notna()" \
       -f "le_linear = le.interpolate().where(before & after)" \
       --drop before after
 ```
 
+### Differences between rows
+
+For the change in a column from one row to the next, `eval -f` takes
+pandas `diff()` and `pct_change()`, also within groups. They use the rows
+in their order, so sort first, in a step of its own:
+
+```bash
+dftk dataset healthexp -o | dftk eval --sortasc Country Year -o \
+  | dftk eval -f "change = Spending_USD.groupby(Country).diff().round(1)" \
+      -f "pct = (Spending_USD.groupby(Country).pct_change() * 100).round(1)" \
+      -f "years = Year.groupby(Country).diff()"
+```
+
+```text
+Year  Country  Spending_USD  Life_Expectancy  change  pct   years
+1971  Canada   313.391       72.8
+1976  Canada   543.337       73.8             229.9   73.4  5.0
+1979  Canada   692.269       75.1             148.9   27.4  3.0
+1980  Canada   791.812       75.2             99.5    14.4  1.0
+```
+
+The first row of each country gets an empty cell. Without
+`groupby(Country)`, it would be compared with the last row of the
+previous country.
+
+To compare a value with one in another column of the previous or next row, for example the
+gap between segments given by `Chr`, `Begin`, `End`, use SQL's `LAG()` /
+`LEAD()` in `query --sql`:
+
+```bash
+dftk query segments.tsv --sql \
+  'SELECT *, "Begin" - LAG("End") OVER (PARTITION BY Chr ORDER BY "Begin")
+     AS gap
+   FROM data ORDER BY Chr, "Begin"'
+```
+
+```text
+Chr   Begin  End  Id  gap
+chr1  100    200  a
+chr1  250    400  b   50
+chr1  390    500  c   -10
+chr2  50     80   d
+chr2  100    300  e   20
+```
+
+A negative gap is an overlap, and the first segment on each chromosome
+gets an empty cell. `lead("Begin") OVER (...) - "End"` gives the gap to
+the next segment instead.
+
+- `Begin` and `End` are SQL keywords: quote such column names with double
+  quotes and the whole statement with single quotes.
+- No sorting is needed first; `ORDER BY` inside `OVER` sets the order.
+- Integers stay integers.
+
+`eval -f` can do the same with pandas `shift`, but it goes by row order, so
+sort by `Chr` and `Begin` first (results are floats):
+
+```bash
+dftk eval segments.tsv -f 'gap = Begin - End.groupby(Chr).shift()'
+```
+
+Without `groupby(Chr)` the first segment of a chromosome is compared with
+the last one of the previous chromosome.
+
 ### Bootstrap confidence intervals
 
 Bootstrap mode repeats the full stat computation N times on resampled data
@@ -307,9 +375,9 @@ confidence interval:
 
 ```bash
 dftk dataset tips -o \
-  | dftk stat ... -c tip -g day --bootstrap 1000 --randomseed 42 -o \
-  | dftk pivot ... -i day -v mean -f mean cilo cihi -o \
-  | dftk print ...
+  | dftk stat -c tip -g day --bootstrap 1000 --randomseed 42 -o \
+  | dftk pivot -i day -v mean -f mean cilo cihi -o \
+  | dftk print
 ```
 
 ## Input/output
@@ -317,15 +385,21 @@ dftk dataset tips -o \
 All commands accept:
 
 - A TSV filename as a positional argument
-- `-` to read TSV from stdin
-- `...` to receive a parquet path from stdin (written by a previous `-o` command)
 - A `.parquet` filename to read a named parquet file directly
+- Nothing, or `-`, to read stdin: either TSV, or the parquet path printed by
+  a previous command's bare `-o` (that temp file is then read and deleted)
+- A pipe as the filename, e.g. `<(grep -v '^#' data.tsv)` or `<(dftk … -o)`,
+  read the same way as stdin
+
+```bash
+dftk dataset tips -o | dftk stat -c tip -g day        # parquet path on stdin
+dftk dataset tips | dftk stat -c tip -g day           # TSV on stdin
+dftk merge -l tips.tsv -r <(dftk pivot tips.tsv -i day -v tip -f mean -o) -k day
+```
 
-> **`-` vs `...`** — these are not interchangeable. Use `-` when the previous
-> command in the pipe wrote plain TSV to stdout (the default). Use `...` when
-> the previous command used bare `-o` (pipe mode), which writes a temp
-> parquet and prints *its path* to stdout — `-` would try to parse that path
-> as TSV and fail.
+Older scripts give `...` as the file to read a parquet path from stdin; that
+still works. In zsh with Oh My Zsh an unquoted `...` becomes `../..`, so
+leave it out (or use `-`) in scripts you share.
 
 ### Comment lines and preambles
 

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/README.md

@@ -47,17 +47,20 @@ uv tool install .
 
 ### Skill for coding agents
 
-`skills/dftk/` is an agent skill: a short guide that coding agents such as
-Codex and Claude Code load when a task involves dftk. It covers the piping
+dftk ships an agent skill: a short guide that coding agents such as Codex
+and Claude Code load when a task involves dftk. It covers the piping
 conventions, which command to use for what, and patterns for Snakemake and
-Make. Copy it into your agent's skills directory, for example:
+Make. Install it with:
 
 ```bash
-cp -r skills/dftk ~/.codex/skills/     # Codex
-cp -r skills/dftk ~/.claude/skills/    # Claude Code
+dftk skill install            # every agent found: ~/.claude and/or ~/.codex
+dftk skill install --codex    # only one (--claude or --codex; creates the folder)
+dftk skill install --dir DIR  # another agent's skills folder
 ```
 
-Copy it again after upgrading dftk so it matches the installed version.
+It prints where it installed the skill. Run it again after upgrading dftk
+so the skill matches the installed version; it replaces only the earlier
+dftk skill. The source is in `src/dftk/skills/dftk/`.
 
 ## Quick example
 
@@ -65,15 +68,15 @@ Copy it again after upgrading dftk so it matches the installed version.
 dftk dataset tips | dftk stat - -c tip total_bill -g day
 ```
 
-Chain commands with `-o` (write to a temp parquet, print its path) and `...`
-(read a parquet path from stdin) instead of piping TSV text directly — faster
-for large data, and the recommended pattern once a pipeline has more than one
-step:
+Chain commands with a bare `-o` (write a temp parquet, print its path)
+instead of piping TSV text: the next command sees the path on stdin and reads
+that file. It's faster for large data, keeps column types exact, and is the
+recommended pattern once a pipeline has more than one step:
 
 ```bash
 dftk dataset tips -o \
-  | dftk pivot ... -i day -v tip -f mean -o \
-  | dftk print ...
+  | dftk pivot -i day -v tip -f mean -o \
+  | dftk print
 ```
 
 ## Subcommands
@@ -133,7 +136,7 @@ publication-quality output.
 | `sample`   | Random row sampling (with or without replacement, by count or fraction)   |
 | `split`    | Split a dataframe into one file per group                                 |
 | `annotate` | Read and write provenance metadata (genome, source, …) in parquet files  |
-| `print`    | Read any dftk input (TSV, stdin, `...` parquet pipe) and write TSV     |
+| `print`    | Read any dftk input (TSV, parquet, stdin) and write TSV                |
 | `clean`    | Remove leftover temp parquet pipe files from interrupted pipelines       |
 | `help`     | List all subcommands or show full help for one: `dftk help stat`       |
 
@@ -144,8 +147,8 @@ publication-quality output.
 ```bash
 # z-score sepal_length within species, then fit against petal_length
 dftk dataset iris -o \
-  | dftk scale ... -c sepal_length -g species -o \
-  | dftk fit ... -f "sepal_length_scaled ~ petal_length" -g species
+  | dftk scale -c sepal_length -g species -o \
+  | dftk fit -f "sepal_length_scaled ~ petal_length" -g species
 ```
 
 ### Clean up spreadsheet headers
@@ -154,8 +157,8 @@ Headers such as `Country Name` or `1960` need quoting in SQL. `cleannames` turns
 
 ```bash
 dftk dataset fertility -o \
-  | dftk cleannames ... -o \
-  | dftk query ... --sql "SELECT country_name, x1960, x2000 FROM data LIMIT 3"
+  | dftk cleannames -o \
+  | dftk query --sql "SELECT country_name, x1960, x2000 FROM data LIMIT 3"
 ```
 
 For your own export, `dftk cleannames data.tsv --backend duckdb -o data.parquet` also keeps integer columns with empty cells as integers.
@@ -166,7 +169,7 @@ With tens of thousands of points a scatter plot is a solid blob. `--hexbin` coun
 
 ```bash
 dftk dataset diamonds -o \
-  | dftk scat ... -x carat -y price --hexbin --hexlog --logx --logy -f diamonds.png
+  | dftk scat -x carat -y price --hexbin --hexlog --logx --logy -f diamonds.png
 ```
 
 `--colorcol COL` colours each hexagon by the mean of `COL` instead of the count; with `--subgraphcol` all panels share one colour scale.
@@ -175,8 +178,8 @@ dftk dataset diamonds -o \
 
 ```bash
 dftk dataset tips -o \
-  | dftk stat ... -c tip -g day -o \
-  | dftk line ... -x day -y mean --yerr sem -f fig.png
+  | dftk stat -c tip -g day -o \
+  | dftk line -x day -y mean --yerr sem -f fig.png
 ```
 
 ### Wide-to-long then plot overlaid histograms
@@ -220,18 +223,18 @@ suffix `_r` from `merge`; fill from it, then drop it. Here the defaults are
 each species' median body mass:
 
 ```bash
-dftk dataset penguins -o | dftk pivot ... -i species -v body_mass_g \
+dftk dataset penguins -o | dftk pivot -i species -v body_mass_g \
   -f median --move median_body_mass_g:body_mass_g -o defaults.parquet
 dftk dataset penguins -o \
-  | dftk merge ... -r defaults.parquet -k species -t left -o \
-  | dftk eval ... -f "body_mass_g = body_mass_g.fillna(body_mass_g_r)" \
+  | dftk merge -r defaults.parquet -k species -t left -o \
+  | dftk eval -f "body_mass_g = body_mass_g.fillna(body_mass_g_r)" \
       --drop body_mass_g_r
 ```
 
 **From neighbouring rows** (`co2` is a weekly series with gaps):
 
 ```bash
-dftk dataset co2 -o | dftk eval ... -f "ffill = co2.ffill()" \
+dftk dataset co2 -o | dftk eval -f "ffill = co2.ffill()" \
   -f "bfill = co2.bfill()" -f "linear = co2.interpolate()"
 ```
 
@@ -249,9 +252,9 @@ to be next to each other, but within each group they must be in order:
 
 ```bash
 dftk dataset healthexp -o \
-  | dftk eval ... -f "le = Life_Expectancy.where(Year % 10 != 5)" \
+  | dftk eval -f "le = Life_Expectancy.where(Year % 10 != 5)" \
       --sortasc Country Year -o \
-  | dftk eval ... -f "le_filled = le.groupby(Country).ffill()"
+  | dftk eval -f "le_filled = le.groupby(Country).ffill()"
 ```
 
 `interpolate()` has no grouped form. Once each group's rows are together,
@@ -261,14 +264,78 @@ with something before and after them in the same group:
 
 ```bash
 dftk dataset healthexp -o \
-  | dftk eval ... -f "le = Life_Expectancy.where(Year % 10 != 5)" \
+  | dftk eval -f "le = Life_Expectancy.where(Year % 10 != 5)" \
       --sortasc Country Year -o \
-  | dftk eval ... -f "before = le.groupby(Country).ffill().notna()" \
+  | dftk eval -f "before = le.groupby(Country).ffill().notna()" \
       -f "after = le.groupby(Country).bfill().notna()" \
       -f "le_linear = le.interpolate().where(before & after)" \
       --drop before after
 ```
 
+### Differences between rows
+
+For the change in a column from one row to the next, `eval -f` takes
+pandas `diff()` and `pct_change()`, also within groups. They use the rows
+in their order, so sort first, in a step of its own:
+
+```bash
+dftk dataset healthexp -o | dftk eval --sortasc Country Year -o \
+  | dftk eval -f "change = Spending_USD.groupby(Country).diff().round(1)" \
+      -f "pct = (Spending_USD.groupby(Country).pct_change() * 100).round(1)" \
+      -f "years = Year.groupby(Country).diff()"
+```
+
+```text
+Year  Country  Spending_USD  Life_Expectancy  change  pct   years
+1971  Canada   313.391       72.8
+1976  Canada   543.337       73.8             229.9   73.4  5.0
+1979  Canada   692.269       75.1             148.9   27.4  3.0
+1980  Canada   791.812       75.2             99.5    14.4  1.0
+```
+
+The first row of each country gets an empty cell. Without
+`groupby(Country)`, it would be compared with the last row of the
+previous country.
+
+To compare a value with one in another column of the previous or next row, for example the
+gap between segments given by `Chr`, `Begin`, `End`, use SQL's `LAG()` /
+`LEAD()` in `query --sql`:
+
+```bash
+dftk query segments.tsv --sql \
+  'SELECT *, "Begin" - LAG("End") OVER (PARTITION BY Chr ORDER BY "Begin")
+     AS gap
+   FROM data ORDER BY Chr, "Begin"'
+```
+
+```text
+Chr   Begin  End  Id  gap
+chr1  100    200  a
+chr1  250    400  b   50
+chr1  390    500  c   -10
+chr2  50     80   d
+chr2  100    300  e   20
+```
+
+A negative gap is an overlap, and the first segment on each chromosome
+gets an empty cell. `lead("Begin") OVER (...) - "End"` gives the gap to
+the next segment instead.
+
+- `Begin` and `End` are SQL keywords: quote such column names with double
+  quotes and the whole statement with single quotes.
+- No sorting is needed first; `ORDER BY` inside `OVER` sets the order.
+- Integers stay integers.
+
+`eval -f` can do the same with pandas `shift`, but it goes by row order, so
+sort by `Chr` and `Begin` first (results are floats):
+
+```bash
+dftk eval segments.tsv -f 'gap = Begin - End.groupby(Chr).shift()'
+```
+
+Without `groupby(Chr)` the first segment of a chromosome is compared with
+the last one of the previous chromosome.
+
 ### Bootstrap confidence intervals
 
 Bootstrap mode repeats the full stat computation N times on resampled data
@@ -278,9 +345,9 @@ confidence interval:
 
 ```bash
 dftk dataset tips -o \
-  | dftk stat ... -c tip -g day --bootstrap 1000 --randomseed 42 -o \
-  | dftk pivot ... -i day -v mean -f mean cilo cihi -o \
-  | dftk print ...
+  | dftk stat -c tip -g day --bootstrap 1000 --randomseed 42 -o \
+  | dftk pivot -i day -v mean -f mean cilo cihi -o \
+  | dftk print
 ```
 
 ## Input/output
@@ -288,15 +355,21 @@ dftk dataset tips -o \
 All commands accept:
 
 - A TSV filename as a positional argument
-- `-` to read TSV from stdin
-- `...` to receive a parquet path from stdin (written by a previous `-o` command)
 - A `.parquet` filename to read a named parquet file directly
+- Nothing, or `-`, to read stdin: either TSV, or the parquet path printed by
+  a previous command's bare `-o` (that temp file is then read and deleted)
+- A pipe as the filename, e.g. `<(grep -v '^#' data.tsv)` or `<(dftk … -o)`,
+  read the same way as stdin
+
+```bash
+dftk dataset tips -o | dftk stat -c tip -g day        # parquet path on stdin
+dftk dataset tips | dftk stat -c tip -g day           # TSV on stdin
+dftk merge -l tips.tsv -r <(dftk pivot tips.tsv -i day -v tip -f mean -o) -k day
+```
 
-> **`-` vs `...`** — these are not interchangeable. Use `-` when the previous
-> command in the pipe wrote plain TSV to stdout (the default). Use `...` when
-> the previous command used bare `-o` (pipe mode), which writes a temp
-> parquet and prints *its path* to stdout — `-` would try to parse that path
-> as TSV and fail.
+Older scripts give `...` as the file to read a parquet path from stdin; that
+still works. In zsh with Oh My Zsh an unquoted `...` becomes `../..`, so
+leave it out (or use `-`) in scripts you share.
 
 ### Comment lines and preambles
 

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dataframe-toolkit"
-version = "0.11.2"
+version = "0.12.0"
 description = "A command-line toolkit for DataFrame analysis and manipulation"
 readme = "README.md"
 keywords = [
@@ -29,6 +29,7 @@ dependencies = [
     "patsy>=0.5.6",
     "duckdb>=0.8",
     "matplotlib>=3.7",
+    "pillow>=13.0",
     "seaborn>=0.12",
     "tqdm>=4.65",
 ]

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "dataframe-toolkit"
-version = "0.11.2"
+version = "0.12.0"
 description = "A command-line toolkit for DataFrame analysis and manipulation"
 readme = "README.md"
 authors = [
@@ -26,6 +26,7 @@ dependencies = [
     "patsy>=0.5.6",
     "duckdb>=0.8",
     "matplotlib>=3.7",
+    "pillow>=13.0",  # not used directly (matplotlib needs it); floor for security fixes
     "seaborn>=0.12",
     "tqdm>=4.65",
 ]

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/src/dftk/commands/__init__.py

@@ -73,6 +73,7 @@ COMMAND_GROUPS: list[tuple[str, list[tuple[str, str]]]] = [
             ("annotate", "annotate_cmd:AnnotateCommand"),
             ("print", "print_cmd:PrintCommand"),
             ("clean", "clean_cmd:CleanCommand"),
+            ("skill", "skill_cmd:SkillCommand"),
         ],
     ),
 ]

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/src/dftk/commands/annotate_cmd.py

@@ -26,7 +26,7 @@ Usage examples:
       --meta genome=hg38 --meta source=my_pipeline
 
   # Same, built from a bundled dataset:
-  dftk dataset iris -o | dftk eval ... -f "z = sepal_length + petal_length" \\
+  dftk dataset iris -o | dftk eval -f "z = sepal_length + petal_length" \\
       -o result.parquet --meta genome=hg38 --meta source=my_pipeline
 
 Annotations propagate through the parquet pipe: a file annotated at

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/src/dftk/commands/binx_cmd.py

@@ -74,10 +74,10 @@ USEVALUE (--usevalue)
 EXAMPLES
 --------
   dftk binx data.tsv -c age -b 0:100:10
-  dftk dataset tips -o | dftk binx ... -c total_bill -b 0:60:10
-  dftk dataset tips -o | dftk binx ... -c tip -b 0,2,4,6,10 -d tip_bracket --usevalue m
-  dftk dataset tips -o | dftk binx ... -c total_bill -b 0:60:10 -d bill_bin -o \\
-      | dftk stat ... -c tip -g bill_bin
+  dftk dataset tips -o | dftk binx -c total_bill -b 0:60:10
+  dftk dataset tips -o | dftk binx -c tip -b 0,2,4,6,10 -d tip_bracket --usevalue m
+  dftk dataset tips -o | dftk binx -c total_bill -b 0:60:10 -d bill_bin -o \\
+      | dftk stat -c tip -g bill_bin
 """
 
 

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/src/dftk/commands/chi_cmd.py

@@ -116,21 +116,21 @@ EXAMPLES
 Basic chi-plot:
 
   dftk chi data.tsv -x recomb_rate -y gc_content
-  dftk dataset tips -o | dftk chi ... -x total_bill -y tip
+  dftk dataset tips -o | dftk chi -x total_bill -y tip
 
 Split into a subplot grid by category, to check whether tail dependence is
 consistent across the whole dataset or driven by a few subgroups (e.g.
 chromosome, sector, region — here, diamond cut quality):
 
-  dftk dataset diamonds -o | dftk chi ... -x carat -y price --subgraphcol cut
+  dftk dataset diamonds -o | dftk chi -x carat -y price --subgraphcol cut
 
 Colour-code by group instead of splitting into subplots:
 
-  dftk dataset diamonds -o | dftk chi ... -x carat -y price -g cut
+  dftk dataset diamonds -o | dftk chi -x carat -y price -g cut
 
 Publication figure (Nature single column, PDF):
 
-  dftk dataset tips -o | dftk chi ... -x total_bill -y tip \\
+  dftk dataset tips -o | dftk chi -x total_bill -y tip \\
       --size single --fontsize publication -f fig.pdf
 """
 

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/src/dftk/commands/cleannames_cmd.py

@@ -57,13 +57,13 @@ column per year (1960, 1961, ...); after cleaning they can be used
 unquoted in SQL:
 
   dftk dataset fertility -o \\
-      | dftk cleannames ... -o \\
-      | dftk query ... --sql "SELECT country_name, x1960, x2000 FROM data LIMIT 3"
+      | dftk cleannames -o \\
+      | dftk query --sql "SELECT country_name, x1960, x2000 FROM data LIMIT 3"
 
 Write the old and new names to a TSV file, e.g. for a data dictionary:
 
   dftk dataset fertility -o \\
-      | dftk cleannames ... --mapping names.tsv -o fertility.parquet
+      | dftk cleannames --mapping names.tsv -o fertility.parquet
 """
 
 # Letters that Unicode does not decompose into ASCII letter + accent

{dataframe_toolkit-0.11.2 → dataframe_toolkit-0.12.0}/src/dftk/commands/concat_cmd.py

@@ -12,7 +12,7 @@ Examples
     dftk concat penguins.tsv penguins.tsv --sourcecol source
     dftk dataset penguins -o penguins.parquet
     dftk concat penguins.parquet penguins.parquet -o
-    dftk dataset penguins -o | dftk eval ... -f "x=1" -o | dftk concat ... penguins.tsv
+    dftk dataset penguins -o | dftk eval -f "x=1" -o | dftk concat - penguins.tsv
 """
 
 import argparse
@@ -51,7 +51,7 @@ EXAMPLES
   dftk concat penguins.tsv penguins.tsv --sourcecol source
   dftk dataset penguins -o penguins.parquet
   dftk concat penguins.parquet penguins.parquet -o result.parquet
-  dftk dataset penguins -o | dftk eval ... -f "x=1" -o | dftk concat ... penguins.tsv
+  dftk dataset penguins -o | dftk eval -f "x=1" -o | dftk concat - penguins.tsv
 
 Mismatched columns are outer-joined and filled (NaN by default, or --fill):
 
@@ -65,7 +65,7 @@ Mismatched columns are outer-joined and filled (NaN by default, or --fill):
             metavar="FILE",
             help=(
                 "Input files to concatenate (TSV or .parquet).  "
-                "Use - for TSV stdin or ... for a parquet pipe path on stdin.  "
+                "Use - for stdin (TSV, or a parquet path from -o).  "
                 "At most one stdin source is allowed."
             ),
         )