dataframe-toolkit 0.4.1__tar.gz → 0.5.1__tar.gz

{dataframe_toolkit-0.4.1 → dataframe_toolkit-0.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: dataframe-toolkit
-Version: 0.4.1
+Version: 0.5.1
 Summary: A command-line toolkit for DataFrame analysis and manipulation
 Keywords: dataframe,statistics,data analysis,CLI,TSV,pandas
 Author: Gunnar Pálsson
@@ -38,6 +38,14 @@ for analysts who live in the terminal and want a consistent, pipeable toolkit
 for the full data analysis pipeline — from initial exploration through
 statistical modelling and publication-quality figures.
 
+**No data file handy?** Every example below uses `dftk dataset` to pull in a
+small, real dataset, so you can copy-paste any of them and run them right
+now. Run `dftk dataset --list` to browse everything available (dozens of
+datasets from seaborn, statsmodels, and pydataset). `dftk dataset NAME`
+searches all three sources in that order and returns the first match, so
+`--source` is optional — pass it only to disambiguate a name that exists in
+more than one source, or to restrict the search.
+
 ## Installation
 
 ```bash
@@ -68,28 +76,38 @@ uv tool install .
 ## Quick example
 
 ```bash
-dftk stat data.tsv -c height weight -g sex
-dftk stat data.tsv -c value -g group -o \
-  | dftk pivot -i group -v value_mean -o \
-  | dftk print
+dftk dataset tips | dftk stat - -c tip total_bill -g day
+```
+
+Chain commands with `-o` (write to a temp parquet, print its path) and `...`
+(read a parquet path from stdin) instead of piping TSV text directly — faster
+for large data, and the recommended pattern once a pipeline has more than one
+step:
+
+```bash
+dftk dataset tips -o \
+  | dftk pivot ... -i day -v tip -f mean -o \
+  | dftk print ...
 ```
 
 ## Subcommands
 
 ### Data transformation
 
-| Command   | Description                                                                          |
-|-----------|--------------------------------------------------------------------------------------|
-| `eval`    | Add or modify columns: eval expressions, string/path functions, statistical ops      |
-| `query`   | Filter rows with pandas query expressions or SQL (`--sql` via DuckDB)                |
-| `merge`   | Join two tables on key columns (inner/left/right/outer)                              |
-| `concat`  | Concatenate two or more tables row-wise                                              |
-| `melt`    | Reshape wide-to-long (`pd.melt`)                                                     |
-| `pivot`   | Reshape long-to-wide with per-cell aggregation                                       |
-| `func`    | Column transforms: cumsum, group mean/sum/min/max/count/median/std, rank, qcut:N     |
-| `scale`   | Normalise: z-score, min-shift, sum/max/mean scaling, Blom rank, regression residuals |
-| `interp`  | Interpolate values from a reference curve into a table (1-D lookup)                  |
-| `binx`    | Assign bin indices to a column based on explicit or generated edges                  |
+| Command     | Description                                                                          |
+|-------------|---------------------------------------------------------------------------------------|
+| `eval`      | Add or modify columns: eval expressions, string/path functions, statistical ops      |
+| `query`     | Filter rows with pandas query expressions or SQL (`--sql` via DuckDB)                |
+| `merge`     | Join two tables on key columns (inner/left/right/outer)                              |
+| `concat`    | Concatenate two or more tables row-wise                                              |
+| `melt`      | Reshape wide-to-long (`pd.melt`)                                                     |
+| `pivot`     | Reshape long-to-wide with per-cell aggregation                                       |
+| `func`      | Column transforms: cumsum, group mean/sum/min/max/count/median/std, rank, qcut:N     |
+| `scale`     | Normalise: z-score, min-shift, sum/max/mean scaling, Blom rank, regression residuals |
+| `interp`    | Interpolate values from a reference curve into a table (1-D lookup)                  |
+| `binx`      | Assign bin indices to a column based on explicit or generated edges                  |
+| `transpose` | Flip rows and columns; original column names land in a new `--keycol` column         |
+| `segid`     | Assign a segment ID that increments on each value change in a column                 |
 
 ### Statistics
 
@@ -101,6 +119,7 @@ dftk stat data.tsv -c value -g group -o \
 | `test`     | P-values between column pairs: t-test, Mann-Whitney, Wilcoxon, KS, correlations, bootstrap; groups  |
 | `corr`     | Pairwise column correlations (Pearson/Spearman/Kendall) with optional BCa bootstrap CI              |
 | `describe` | Quick column-level summary (dtype, n, n_unique, n_null, sample values)                              |
+| `info`     | Per-column dtype, null counts, and memory usage; `--summary` for dataset-level totals               |
 | `randvar`  | Sample from a distribution and append as a new column (norm, alpha, beta, …)                        |
 
 ### Plots
@@ -133,38 +152,48 @@ publication-quality output.
 ### Chaining commands
 
 ```bash
-# z-score within group, then fit a model
-dftk scale data.tsv -c expr -g condition -o \
-  | dftk fit - -f "expr_z ~ time + batch" -g condition
+# z-score sepal_length within species, then fit against petal_length
+dftk dataset iris -o \
+  | dftk scale ... -c sepal_length -g species -o \
+  | dftk fit ... -f "sepal_length_scaled ~ petal_length" -g species
 ```
 
 ### Group summary then plot
 
 ```bash
-dftk stat results.tsv -c value -g group -o \
-  | dftk line - -x group -y value_mean --yerr value_sem -f fig.png
+dftk dataset tips -o \
+  | dftk stat ... -c tip -g day -o \
+  | dftk line ... -x day -y mean --yerr sem -f fig.png
 ```
 
 ### Wide-to-long then plot overlaid histograms
 
 ```bash
-dftk melt data.tsv -i sample -d gene -v expression \
-  | dftk hist - -x expression -g gene -k -f dist.png
+dftk dataset iris \
+  | dftk melt - -i species -d measurement -v value \
+  | dftk hist - -x value -g measurement -k -f dist.png
 ```
 
 ### Interpolation (standard curve lookup)
 
 ```bash
-dftk interp samples.tsv --ref stdcurve.tsv \
-    -x fluorescence -v concentration -d conc_ng_ul
+printf "conc\tfluor\n0\t5\n10\t52\n20\t98\n30\t151\n" > stdcurve.tsv
+printf "sample\tfluor\ns1\t60\ns2\t110\n" \
+  | dftk interp - --ref stdcurve.tsv -x fluor --refx fluor -v conc -d conc_ng_ul
 ```
 
 ### Bootstrap confidence intervals
 
+Bootstrap mode repeats the full stat computation N times on resampled data
+and emits one row per resample (tagged with `samplenum`) — pipe into `pivot`
+with percentile aggfuncs (`cilo`/`cihi`) to collapse that into an empirical
+confidence interval:
+
 ```bash
-dftk stat data.tsv -c value -g group --bootstrap 1000 --randomseed 42 -o \
-  | dftk pivot -i group -v value_mean -f mean -o \
-  | dftk stat - -c group_A group_B
+dftk dataset tips -o \
+  | dftk stat ... -c tip -g day --bootstrap 1000 --randomseed 42 -o \
+  | dftk pivot ... -i day -v mean -f mean cilo cihi -o \
+  | dftk print ...
 ```
 
 ## Input/output
@@ -176,9 +205,19 @@ All commands accept:
 - `...` to receive a parquet path from stdin (written by a previous `-o` command)
 - A `.parquet` filename to read a named parquet file directly
 
+> **`-` vs `...`** — these are not interchangeable. Use `-` when the previous
+> command in the pipe wrote plain TSV to stdout (the default). Use `...` when
+> the previous command used bare `-o` (pipe mode), which writes a temp
+> parquet and prints *its path* to stdout — `-` would try to parse that path
+> as TSV and fail.
+
 Standard output options (available on all tabular commands):
 
-- `-o` / `--output` — write a temp parquet for the next piped command instead of TSV to stdout
+- `-o` / `--output` — controls where output goes; takes an *optional* value:
+  - `-o` alone — write a temp parquet to pipe into the next command, printing its path to stdout (auto-deleted once read)
+  - `-o FILE.parquet` — write a named, reusable parquet, printing its path to stdout
+  - `-o FILE` (no `.parquet` extension) — write TSV to `FILE`, with no stdout output
+  - omit `-o` entirely — write TSV to stdout (the default)
 - `--select col1 col2 …` — keep only these columns
 - `--drop col1 col2 …` — remove these columns
 - `--round N` — round numeric output
@@ -193,20 +232,19 @@ any new `--meta` values.
 
 ```bash
 # Tag a file at creation
-dftk eval raw.tsv -f "z = x + y" -o results.parquet \
-    --meta genome=hg38 --meta source=gwas_2024
+dftk dataset iris -o iris.parquet --meta source=seaborn --meta species_col=species
 
 # Inspect annotations
-dftk annotate results.parquet
-# genome   hg38
-# source   gwas_2024
+dftk annotate iris.parquet
+# source        seaborn
+# species_col   species
 
 # Add or update an annotation in-place
-dftk annotate results.parquet --set step=qc_filtered
+dftk annotate iris.parquet --set step=raw
 
 # Annotations survive piping
-dftk eval results.parquet -f "z_scaled = z / 2" -o | dftk scale ... -c z -o scaled.parquet
-dftk annotate scaled.parquet   # genome and source still present
+dftk eval iris.parquet -f "petal_area = petal_length * petal_width" -o iris2.parquet
+dftk annotate iris2.parquet   # source, species_col, and step all still present
 ```
 
 ## Figure options (plot commands)

{dataframe_toolkit-0.4.1 → dataframe_toolkit-0.5.1}/README.md

@@ -10,6 +10,14 @@ for analysts who live in the terminal and want a consistent, pipeable toolkit
 for the full data analysis pipeline — from initial exploration through
 statistical modelling and publication-quality figures.
 
+**No data file handy?** Every example below uses `dftk dataset` to pull in a
+small, real dataset, so you can copy-paste any of them and run them right
+now. Run `dftk dataset --list` to browse everything available (dozens of
+datasets from seaborn, statsmodels, and pydataset). `dftk dataset NAME`
+searches all three sources in that order and returns the first match, so
+`--source` is optional — pass it only to disambiguate a name that exists in
+more than one source, or to restrict the search.
+
 ## Installation
 
 ```bash
@@ -40,28 +48,38 @@ uv tool install .
 ## Quick example
 
 ```bash
-dftk stat data.tsv -c height weight -g sex
-dftk stat data.tsv -c value -g group -o \
-  | dftk pivot -i group -v value_mean -o \
-  | dftk print
+dftk dataset tips | dftk stat - -c tip total_bill -g day
+```
+
+Chain commands with `-o` (write to a temp parquet, print its path) and `...`
+(read a parquet path from stdin) instead of piping TSV text directly — faster
+for large data, and the recommended pattern once a pipeline has more than one
+step:
+
+```bash
+dftk dataset tips -o \
+  | dftk pivot ... -i day -v tip -f mean -o \
+  | dftk print ...
 ```
 
 ## Subcommands
 
 ### Data transformation
 
-| Command   | Description                                                                          |
-|-----------|--------------------------------------------------------------------------------------|
-| `eval`    | Add or modify columns: eval expressions, string/path functions, statistical ops      |
-| `query`   | Filter rows with pandas query expressions or SQL (`--sql` via DuckDB)                |
-| `merge`   | Join two tables on key columns (inner/left/right/outer)                              |
-| `concat`  | Concatenate two or more tables row-wise                                              |
-| `melt`    | Reshape wide-to-long (`pd.melt`)                                                     |
-| `pivot`   | Reshape long-to-wide with per-cell aggregation                                       |
-| `func`    | Column transforms: cumsum, group mean/sum/min/max/count/median/std, rank, qcut:N     |
-| `scale`   | Normalise: z-score, min-shift, sum/max/mean scaling, Blom rank, regression residuals |
-| `interp`  | Interpolate values from a reference curve into a table (1-D lookup)                  |
-| `binx`    | Assign bin indices to a column based on explicit or generated edges                  |
+| Command     | Description                                                                          |
+|-------------|---------------------------------------------------------------------------------------|
+| `eval`      | Add or modify columns: eval expressions, string/path functions, statistical ops      |
+| `query`     | Filter rows with pandas query expressions or SQL (`--sql` via DuckDB)                |
+| `merge`     | Join two tables on key columns (inner/left/right/outer)                              |
+| `concat`    | Concatenate two or more tables row-wise                                              |
+| `melt`      | Reshape wide-to-long (`pd.melt`)                                                     |
+| `pivot`     | Reshape long-to-wide with per-cell aggregation                                       |
+| `func`      | Column transforms: cumsum, group mean/sum/min/max/count/median/std, rank, qcut:N     |
+| `scale`     | Normalise: z-score, min-shift, sum/max/mean scaling, Blom rank, regression residuals |
+| `interp`    | Interpolate values from a reference curve into a table (1-D lookup)                  |
+| `binx`      | Assign bin indices to a column based on explicit or generated edges                  |
+| `transpose` | Flip rows and columns; original column names land in a new `--keycol` column         |
+| `segid`     | Assign a segment ID that increments on each value change in a column                 |
 
 ### Statistics
 
@@ -73,6 +91,7 @@ dftk stat data.tsv -c value -g group -o \
 | `test`     | P-values between column pairs: t-test, Mann-Whitney, Wilcoxon, KS, correlations, bootstrap; groups  |
 | `corr`     | Pairwise column correlations (Pearson/Spearman/Kendall) with optional BCa bootstrap CI              |
 | `describe` | Quick column-level summary (dtype, n, n_unique, n_null, sample values)                              |
+| `info`     | Per-column dtype, null counts, and memory usage; `--summary` for dataset-level totals               |
 | `randvar`  | Sample from a distribution and append as a new column (norm, alpha, beta, …)                        |
 
 ### Plots
@@ -105,38 +124,48 @@ publication-quality output.
 ### Chaining commands
 
 ```bash
-# z-score within group, then fit a model
-dftk scale data.tsv -c expr -g condition -o \
-  | dftk fit - -f "expr_z ~ time + batch" -g condition
+# z-score sepal_length within species, then fit against petal_length
+dftk dataset iris -o \
+  | dftk scale ... -c sepal_length -g species -o \
+  | dftk fit ... -f "sepal_length_scaled ~ petal_length" -g species
 ```
 
 ### Group summary then plot
 
 ```bash
-dftk stat results.tsv -c value -g group -o \
-  | dftk line - -x group -y value_mean --yerr value_sem -f fig.png
+dftk dataset tips -o \
+  | dftk stat ... -c tip -g day -o \
+  | dftk line ... -x day -y mean --yerr sem -f fig.png
 ```
 
 ### Wide-to-long then plot overlaid histograms
 
 ```bash
-dftk melt data.tsv -i sample -d gene -v expression \
-  | dftk hist - -x expression -g gene -k -f dist.png
+dftk dataset iris \
+  | dftk melt - -i species -d measurement -v value \
+  | dftk hist - -x value -g measurement -k -f dist.png
 ```
 
 ### Interpolation (standard curve lookup)
 
 ```bash
-dftk interp samples.tsv --ref stdcurve.tsv \
-    -x fluorescence -v concentration -d conc_ng_ul
+printf "conc\tfluor\n0\t5\n10\t52\n20\t98\n30\t151\n" > stdcurve.tsv
+printf "sample\tfluor\ns1\t60\ns2\t110\n" \
+  | dftk interp - --ref stdcurve.tsv -x fluor --refx fluor -v conc -d conc_ng_ul
 ```
 
 ### Bootstrap confidence intervals
 
+Bootstrap mode repeats the full stat computation N times on resampled data
+and emits one row per resample (tagged with `samplenum`) — pipe into `pivot`
+with percentile aggfuncs (`cilo`/`cihi`) to collapse that into an empirical
+confidence interval:
+
 ```bash
-dftk stat data.tsv -c value -g group --bootstrap 1000 --randomseed 42 -o \
-  | dftk pivot -i group -v value_mean -f mean -o \
-  | dftk stat - -c group_A group_B
+dftk dataset tips -o \
+  | dftk stat ... -c tip -g day --bootstrap 1000 --randomseed 42 -o \
+  | dftk pivot ... -i day -v mean -f mean cilo cihi -o \
+  | dftk print ...
 ```
 
 ## Input/output
@@ -148,9 +177,19 @@ All commands accept:
 - `...` to receive a parquet path from stdin (written by a previous `-o` command)
 - A `.parquet` filename to read a named parquet file directly
 
+> **`-` vs `...`** — these are not interchangeable. Use `-` when the previous
+> command in the pipe wrote plain TSV to stdout (the default). Use `...` when
+> the previous command used bare `-o` (pipe mode), which writes a temp
+> parquet and prints *its path* to stdout — `-` would try to parse that path
+> as TSV and fail.
+
 Standard output options (available on all tabular commands):
 
-- `-o` / `--output` — write a temp parquet for the next piped command instead of TSV to stdout
+- `-o` / `--output` — controls where output goes; takes an *optional* value:
+  - `-o` alone — write a temp parquet to pipe into the next command, printing its path to stdout (auto-deleted once read)
+  - `-o FILE.parquet` — write a named, reusable parquet, printing its path to stdout
+  - `-o FILE` (no `.parquet` extension) — write TSV to `FILE`, with no stdout output
+  - omit `-o` entirely — write TSV to stdout (the default)
 - `--select col1 col2 …` — keep only these columns
 - `--drop col1 col2 …` — remove these columns
 - `--round N` — round numeric output
@@ -165,20 +204,19 @@ any new `--meta` values.
 
 ```bash
 # Tag a file at creation
-dftk eval raw.tsv -f "z = x + y" -o results.parquet \
-    --meta genome=hg38 --meta source=gwas_2024
+dftk dataset iris -o iris.parquet --meta source=seaborn --meta species_col=species
 
 # Inspect annotations
-dftk annotate results.parquet
-# genome   hg38
-# source   gwas_2024
+dftk annotate iris.parquet
+# source        seaborn
+# species_col   species
 
 # Add or update an annotation in-place
-dftk annotate results.parquet --set step=qc_filtered
+dftk annotate iris.parquet --set step=raw
 
 # Annotations survive piping
-dftk eval results.parquet -f "z_scaled = z / 2" -o | dftk scale ... -c z -o scaled.parquet
-dftk annotate scaled.parquet   # genome and source still present
+dftk eval iris.parquet -f "petal_area = petal_length * petal_width" -o iris2.parquet
+dftk annotate iris2.parquet   # source, species_col, and step all still present
 ```
 
 ## Figure options (plot commands)

{dataframe_toolkit-0.4.1 → dataframe_toolkit-0.5.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dataframe-toolkit"
-version = "0.4.1"
+version = "0.5.1"
 description = "A command-line toolkit for DataFrame analysis and manipulation"
 readme = "README.md"
 authors = [

{dataframe_toolkit-0.4.1 → dataframe_toolkit-0.5.1}/src/dftk/commands/__init__.py

@@ -18,6 +18,7 @@ from dftk.commands.fit_cmd import FitCommand
 from dftk.commands.func_cmd import FuncCommand
 from dftk.commands.help import HelpCommand
 from dftk.commands.hist_cmd import HistCommand
+from dftk.commands.info_cmd import InfoCommand
 from dftk.commands.interp_cmd import InterpCommand
 from dftk.commands.line_cmd import LineCommand
 from dftk.commands.melt_cmd import MeltCommand
@@ -29,15 +30,14 @@ from dftk.commands.randvar_cmd import RandvarCommand
 from dftk.commands.sample_cmd import SampleCommand
 from dftk.commands.scale_cmd import ScaleCommand
 from dftk.commands.scat_cmd import ScatCommand
+from dftk.commands.segid_cmd import SegidCommand
 from dftk.commands.split_cmd import SplitCommand
 from dftk.commands.stat_cmd import StatCommand
 from dftk.commands.test_cmd import TestCommand
+from dftk.commands.transpose_cmd import TransposeCommand
 from dftk.commands.wstat_cmd import WstatCommand
 
 # Ported subcommands — uncomment as each one is added:
-# from dftk.commands.segid     import SegidCommand
-# from dftk.commands.info      import InfoCommand
-# from dftk.commands.transpose import TransposeCommand
 # from dftk.commands.color     import ColorCommand
 # from dftk.commands.wavelet   import WaveletCommand
 # from dftk.commands.fisher    import FisherCommand  # superseded by eval's
@@ -57,6 +57,8 @@ COMMAND_GROUPS: list[tuple[str, list]] = [
             ScaleCommand(),
             InterpCommand(),
             BinxCommand(),
+            TransposeCommand(),
+            SegidCommand(),
         ],
     ),
     (
@@ -68,6 +70,7 @@ COMMAND_GROUPS: list[tuple[str, list]] = [
             TestCommand(),
             CorrCommand(),
             DescribeCommand(),
+            InfoCommand(),
             RandvarCommand(),
         ],
     ),

dataframe_toolkit-0.5.1/src/dftk/commands/info_cmd.py

@@ -0,0 +1,74 @@
+"""
+dftk.commands.info_cmd — dftk info subcommand.
+
+Lightweight structural overview of a DataFrame: per-column dtype, null
+counts, and memory usage. Complements `describe`, which profiles
+statistical distributions and quality flags instead.
+
+Example
+-------
+    dftk info data.tsv
+    dftk info data.tsv --summary
+"""
+
+import argparse
+import sys
+
+import pandas as pd
+
+from dftk.commands.base import BaseCommand
+from dftk.common.io import io
+
+
+def _info(df: pd.DataFrame) -> pd.DataFrame:
+    mem = df.memory_usage(deep=True, index=False)
+    return pd.DataFrame(
+        [
+            dict(
+                name=col,
+                dtype=str(df[col].dtype),
+                non_null=int(df[col].notna().sum()),
+                null=int(df[col].isna().sum()),
+                memory_bytes=int(mem[col]),
+            )
+            for col in df.columns
+        ]
+    )
+
+
+def _summary_line(df: pd.DataFrame) -> str:
+    n_rows, n_cols = df.shape
+    total_mem = int(df.memory_usage(deep=True).sum())
+    n_dupes = int(df.duplicated().sum())
+    return (
+        f"Dataset: {n_rows} rows x {n_cols} columns, "
+        f"{total_mem} bytes, {n_dupes} duplicated row(s)"
+    )
+
+
+class InfoCommand(BaseCommand):
+    @property
+    def name(self) -> str:
+        return "info"
+
+    @property
+    def help(self) -> str:
+        return "Structural overview: per-column dtype, null counts, memory usage"
+
+    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
+        self.add_io_arguments(parser)
+
+        g = parser.add_argument_group("info options")
+        g.add_argument(
+            "--summary",
+            action="store_true",
+            help="Print dataset totals (shape, memory, duplicate rows) to stderr.",
+        )
+
+    def execute(self, args: argparse.Namespace) -> None:
+        df = io.read(args)
+
+        if getattr(args, "summary", False):
+            print(_summary_line(df), file=sys.stderr)
+
+        io.printdf(_info(df), args)

dataframe_toolkit-0.5.1/src/dftk/commands/segid_cmd.py

@@ -0,0 +1,118 @@
+"""
+dftk.commands.segid_cmd — dftk segid subcommand.
+
+Port of dfsegid.py: assign a segment ID that increments each time the
+value in a column changes, useful for grouping contiguous runs of
+identical values (e.g. constant-value stretches in a sorted column).
+
+Rows matching --ignore are excluded from the run comparison and always get
+segid 0; they don't break a run spanning across them. Redesigned from the
+legacy script's magic sentinel default ('1415926535', digits of pi) to a
+plain optional flag — omit --ignore to include every row.
+"""
+
+import argparse
+
+import pandas as pd
+
+from dftk.commands.base import BaseCommand
+from dftk.common.io import check_cols, io
+
+_EPILOG = """\
+EXAMPLE
+-------
+  col
+  A
+  A
+  B
+  B
+  A
+
+  dftk segid data.tsv -c col
+
+  col  segid
+  A    1
+  A    1
+  B    2
+  B    2
+  A    3
+
+With --ignore X, rows where col == X get segid 0 and are skipped when
+detecting value changes, so a run continues across them:
+
+  col
+  A
+  X
+  A
+  B
+
+  dftk segid data.tsv -c col --ignore X
+
+  col  segid
+  A    1
+  X    0
+  A    1
+  B    2
+"""
+
+
+def _segid(df: pd.DataFrame, args: argparse.Namespace) -> pd.DataFrame:
+    col = args.col
+    dest = args.destcol
+
+    df[dest] = 0
+
+    if args.ignore is not None:
+        ignore_value = pd.Series([args.ignore]).astype(df[col].dtype).iloc[0]
+        mask = df[col] != ignore_value
+    else:
+        mask = pd.Series(True, index=df.index)
+
+    subseries = df.loc[mask, col]
+    df.loc[mask, dest] = subseries.ne(subseries.shift()).cumsum()
+    return df
+
+
+class SegidCommand(BaseCommand):
+    @property
+    def name(self) -> str:
+        return "segid"
+
+    @property
+    def help(self) -> str:
+        return "Assign a segment ID that increments on each value change"
+
+    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
+        parser.formatter_class = argparse.RawDescriptionHelpFormatter
+        parser.epilog = _EPILOG
+
+        self.add_io_arguments(parser)
+
+        g = parser.add_argument_group("segid options")
+        g.add_argument(
+            "-c",
+            "--col",
+            required=True,
+            metavar="COL",
+            help="Column whose value changes define segment boundaries.",
+        )
+        g.add_argument(
+            "-d",
+            "--destcol",
+            default="segid",
+            metavar="DESTCOL",
+            help="Name of the output segment-ID column (default: segid).",
+        )
+        g.add_argument(
+            "--ignore",
+            default=None,
+            metavar="VALUE",
+            help="Column value to exclude from segmenting (gets segid 0, "
+            "doesn't break a run spanning across it). Default: none.",
+        )
+
+    def execute(self, args: argparse.Namespace) -> None:
+        df = io.read(args)
+        check_cols(df, [args.col], "-c/--col")
+        result = _segid(df, args)
+        io.printdf(result, args)

{dataframe_toolkit-0.4.1 → dataframe_toolkit-0.5.1}/src/dftk/commands/stat_cmd.py

@@ -21,9 +21,11 @@ into ``dftk pivot`` to aggregate across samples.
 Example
 -------
     dftk stat data.tsv -c height weight -g sex
+
+    # Empirical bootstrap CI: resample, then collapse with percentile aggfuncs
     dftk stat data.tsv -c value -g group --bootstrap 1000 --randomseed 42 -o \\
-        | dftk pivot -i group -v value_mean -o \\
-        | dftk print
+        | dftk pivot ... -i group -v mean -f mean cilo cihi -o \\
+        | dftk print ...
 """
 
 import argparse
@@ -278,9 +280,12 @@ class StatCommand(BaseCommand):
                 if args.groupcol is None:
                     sample = df.sample(**sample_kwargs)
                 else:
-                    sample = df.groupby(args.groupcol, group_keys=False).apply(
-                        lambda x: x.sample(**sample_kwargs)
-                    )
+                    # [df.columns.tolist()] re-selection works around a pandas 3.0
+                    # groupby(...).apply() change that otherwise drops the
+                    # grouping column from the result.
+                    sample = df.groupby(args.groupcol, group_keys=False)[
+                        df.columns.tolist()
+                    ].apply(lambda x: x.sample(**sample_kwargs))
 
                 result = _compute_stats(sample, args)
                 result["samplenum"] = i

dataframe_toolkit-0.5.1/src/dftk/commands/transpose_cmd.py

@@ -0,0 +1,57 @@
+"""
+dftk.commands.transpose_cmd — dftk transpose subcommand.
+
+Port of dftpos.py: flips rows and columns. Original column names become
+values in a new key column (default name: "column"); original row
+positions become the new column headers.
+
+Example
+-------
+Input:
+
+  sample  x  y
+  s1      1  2
+  s2      3  4
+
+  dftk transpose data.tsv
+
+Output:
+
+  column  0  1
+  sample  s1 s2
+  x       1  3
+  y       2  4
+"""
+
+import argparse
+
+from dftk.commands.base import BaseCommand
+from dftk.common.io import io
+
+
+class TransposeCommand(BaseCommand):
+    @property
+    def name(self) -> str:
+        return "transpose"
+
+    @property
+    def help(self) -> str:
+        return "Transpose rows and columns"
+
+    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
+        self.add_io_arguments(parser)
+
+        g = parser.add_argument_group("transpose options")
+        g.add_argument(
+            "--keycol",
+            default="column",
+            metavar="NAME",
+            help="Name for the new column holding the original column names "
+            "(default: column).",
+        )
+
+    def execute(self, args: argparse.Namespace) -> None:
+        df = io.read(args)
+        result = df.transpose().reset_index(names=args.keycol)
+        result.columns = [str(c) for c in result.columns]
+        io.printdf(result, args)