dataframe-textual 2.47.0__tar.gz → 3.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. dataframe_textual-3.0.0/PKG-INFO +1161 -0
  2. dataframe_textual-3.0.0/README.md +1121 -0
  3. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/pyproject.toml +6 -4
  4. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/__init__.py +1 -1
  5. dataframe_textual-3.0.0/src/dataframe_textual/commands.py +457 -0
  6. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/common.py +153 -93
  7. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/console_panel.py +12 -12
  8. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/data_frame_table.py +2866 -1280
  9. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/data_frame_viewer.py +415 -197
  10. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/file_picker_screen.py +7 -4
  11. dataframe_textual-2.47.0/src/dataframe_textual/data_frame_help_panel.py → dataframe_textual-3.0.0/src/dataframe_textual/help_panel.py +34 -10
  12. dataframe_textual-3.0.0/src/dataframe_textual/keybindings.py +399 -0
  13. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/loading_screen.py +24 -2
  14. dataframe_textual-3.0.0/src/dataframe_textual/run_command_screen.py +53 -0
  15. dataframe_textual-3.0.0/src/dataframe_textual/table_screen.py +1539 -0
  16. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/text_screen.py +9 -0
  17. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/theme_screen.py +17 -1
  18. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/yes_no_screen.py +597 -153
  19. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/uv.lock +431 -409
  20. dataframe_textual-2.47.0/PKG-INFO +0 -984
  21. dataframe_textual-2.47.0/README.md +0 -945
  22. dataframe_textual-2.47.0/src/dataframe_textual/table_screen.py +0 -1169
  23. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/.gitignore +0 -0
  24. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/LICENSE +0 -0
  25. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/main.py +0 -0
  26. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/__main__.py +0 -0
  27. {dataframe_textual-2.47.0 → dataframe_textual-3.0.0}/src/dataframe_textual/status_bar.py +0 -0
@@ -0,0 +1,1161 @@
1
+ Metadata-Version: 2.4
2
+ Name: dataframe-textual
3
+ Version: 3.0.0
4
+ Summary: Interactive terminal viewer/editor (TUI) for CSV/TSV, Excel, Parquet, JSONL, etc.
5
+ Project-URL: Homepage, https://github.com/need47/dataframe-textual
6
+ Project-URL: Repository, https://github.com/need47/dataframe-textual.git
7
+ Project-URL: Documentation, https://github.com/need47/dataframe-textual#readme
8
+ Project-URL: Bug Tracker, https://github.com/need47/dataframe-textual/issues
9
+ Author-email: Tiejun Cheng <need47@gmail.com>
10
+ License: MIT
11
+ License-File: LICENSE
12
+ Keywords: csv,data-analysis,editor,excel,interactive,jsonl,parquet,polars,terminal,textual,tsv,tui,viewer
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Intended Audience :: End Users/Desktop
17
+ Classifier: License :: OSI Approved :: MIT License
18
+ Classifier: Natural Language :: English
19
+ Classifier: Operating System :: MacOS
20
+ Classifier: Operating System :: POSIX
21
+ Classifier: Operating System :: Unix
22
+ Classifier: Programming Language :: Python :: 3
23
+ Classifier: Programming Language :: Python :: 3.11
24
+ Classifier: Programming Language :: Python :: 3.12
25
+ Classifier: Programming Language :: Python :: 3.13
26
+ Classifier: Programming Language :: Python :: 3.14
27
+ Classifier: Topic :: Office/Business
28
+ Classifier: Topic :: Utilities
29
+ Classifier: Typing :: Typed
30
+ Requires-Python: >=3.11
31
+ Requires-Dist: fastexcel>=0.16.0
32
+ Requires-Dist: polars>=1.41.0
33
+ Requires-Dist: textual[syntax]>=8.0.2
34
+ Requires-Dist: xlsxwriter>=3.2.9
35
+ Provides-Extra: dev
36
+ Requires-Dist: textual-dev>=1.8.0; extra == 'dev'
37
+ Provides-Extra: vortex
38
+ Requires-Dist: vortex-data>=0.75.0; extra == 'vortex'
39
+ Description-Content-Type: text/markdown
40
+
41
+ # DataFrame Textual
42
+
43
+ A powerful, interactive terminal-based viewer/editor for CSV/TSV/Excel/[Parquet](https://parquet.apache.org/)/[Vortex](https://vortex.dev/)/JSON/[NDJSON](https://jsonlines.org/) built with Python, [Polars](https://pola.rs/), and [Textual](https://textual.textualize.io/). Inspired by [VisiData](https://www.visidata.org/), this tool provides smooth keyboard navigation, data manipulation, and a clean interface for exploring and analyzing tabular data directly in terminal with multi-tab support for multiple files!
44
+
45
+ ![Screenshot](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/main-table.png)
46
+
47
+ ## Features
48
+
49
+ ### Data Viewing
50
+
51
+ - 🚀 **Fast Loading** - Powered by Polars for efficient batch data handling
52
+ - 🎨 **Rich Terminal UI** - Beautiful, color-coded columns with auto-detected data types (e.g., integer, float, string)
53
+ - ⌨️ **Comprehensive Keyboard Navigation** - Intuitive controls with fully customizable key bindings (mostly VisiData compatible)
54
+ - 📊 **Flexible Input** - Read from files and/or stdin (pipes/redirects) in various formats
55
+ - 🔄 **Smart Pagination** - Lazy load rows on demand for handling large datasets
56
+
57
+ ### Data Manipulation
58
+
59
+ - 📝 **Data Editing** - Edit cells, delete rows, reorder columns, and beyond
60
+ - 🧹 **Duplicate Removal** - Remove duplicate rows
61
+ - 🔍 **Search & Filter** - Find values, highlight matches, and filter selected rows
62
+ - ↔️ **Column/Row Reordering** - Move columns and rows with simple keyboard shortcuts
63
+ - 📈 **Sorting & Statistics** - Multi-column sorting, frequency distribution, and histogram analysis
64
+ - 💾 **Save & Undo** - Save edits back to file with full undo/redo support
65
+
66
+ ### Advanced Features
67
+
68
+ - 📂 **Multi-File Support** - Open multiple files in separate tabs
69
+ - 🔄 **Tab Management** - Seamlessly switch between open files with keyboard shortcuts
70
+ - 🔗 **Table Joins** - Join two tables into a new tab
71
+ - 📑 **Duplicate Tab** - Create a copy of the current tab with the same data
72
+ - 🐍 **Embedded Python Console** - Inspect and transform the active table directly in app
73
+ - 📌 **Freeze Rows/Columns** - Keep important rows and columns visible while scrolling
74
+ - 📸 **Take Screenshot** - Capture terminal view as a SVG image
75
+
76
+ ## Installation
77
+
78
+ ### Using pip
79
+
80
+ ```bash
81
+ # Install from PyPI
82
+ pip install dataframe-textual
83
+ ```
84
+
85
+ This installs an executable `dv`.
86
+
87
+ ### Using [uv](https://docs.astral.sh/uv/)
88
+
89
+ ```bash
90
+ # Install from PyPI
91
+ uv tool install dataframe-textual
92
+
93
+ # Install from GitHub
94
+ uv tool install https://github.com/need47/dataframe-textual.git
95
+
96
+ # Run once with uvx without installing locally
97
+ uvx dataframe-textual <csvfile>
98
+ ```
99
+
100
+ ### Development installation
101
+
102
+ ```bash
103
+ # Clone the repository
104
+ git clone https://github.com/need47/dataframe-textual.git
105
+ cd dataframe-textual
106
+
107
+ # Install from local source with development dependencies
108
+ pip install -e ".[dev]"
109
+ ```
110
+
111
+ ## Usage
112
+
113
+ ### Basic Usage - Single File
114
+
115
+ ```bash
116
+ # Open one file
117
+ dv pokemon.csv
118
+ ```
119
+
120
+ ### Multi-File Usage - Multiple Tabs
121
+
122
+ ```bash
123
+ # Open multiple files in tabs
124
+ dv file1.csv file2.csv file3.csv
125
+
126
+ # Open multiple sheets in an Excel file as separate tabs
127
+ dv file.xlsx
128
+
129
+ # Mix files and stdin
130
+ dv data1.tsv < data2.tsv
131
+ ```
132
+
133
+ ### Multi-File Usage - Single Tab
134
+
135
+ ```bash
136
+
137
+ # Read all parquet files (must be of same format and same structure) into one single table
138
+ dv *.parquet --all-in-one
139
+ ```
140
+
141
+ ## Command Line Options
142
+
143
+ ```
144
+ usage: dv [-h] [-V] [-d DELIMITER] [-f FORMAT] [-F [FIELDS ...]] [-H [HEADER ...]] [-L [N]] [-I] [-T] [-E] [-C [C]] [-Q [C]] [-K N] [-A N] [-M [N]] [-N NULL [NULL ...]] [--theme [THEME]] [--all-in-one]
145
+ [--expr EXPR] [--sql SQL] [-o OUTPUT]
146
+ [files ...]
147
+
148
+ TUI viewer/editor for tabular data (e.g., CSV/Excel).
149
+
150
+ positional arguments:
151
+ files Files to view (or read from stdin)
152
+
153
+ options:
154
+ -h, --help show this help message and exit
155
+ -V, --version show program's version number and exit
156
+ -d, --delimiter DELIMITER
157
+ Specify the delimiter of the input files (must be a single character, e.g., `|` or `;`). By default, the delimiter is inferred from the file extension. If reading from stdin, the
158
+ delimiter must be specified unless it is tab delimited.
159
+ -f, --format FORMAT Specify the format of the input files (e.g., `csv` or `excel`). By default, the format is inferred from the file extension. If reading from stdin, the format must be specified
160
+ unless it is tab delimited.
161
+ -F, --fields [FIELDS ...]
162
+ When used without values, list available fields. Otherwise, read only specified fields.
163
+ -H, --header [HEADER ...]
164
+ Specify header info. When reading CSV/TSV. If used without values, assumes no header. Otherwise, use provided values as column header (e.g., `-H col1 col2 col3`).
165
+ -L, --infer_schema_length [N]
166
+ Number of rows to use for inferring schema when reading CSV/TSV. Defaults to 100. When used without value, uses all rows for schema inference (can be slow for large files).
167
+ -I, --no-inference Do not infer data types when reading CSV/TSV. All values will be of string type.
168
+ -T, --truncate-ragged-lines
169
+ Truncate ragged lines when reading CSV/TSV
170
+ -E, --ignore-errors Ignore errors when reading CSV/TSV
171
+ -C, --comment-prefix [C]
172
+ Skip comment lines starting with `C` when reading CSV/TSV
173
+ -Q, --quote-char [C] Use `C` as quote character for reading CSV/TSV. When used without value, disables special handling of quote characters.
174
+ -K, --skip-lines N Skip first N lines when reading CSV/TSV
175
+ -A, --skip-rows-after-header N
176
+ Skip N rows after header when reading CSV/TSV
177
+ -M, --n-rows [N] Read maximum rows
178
+ -N, --null NULL [NULL ...]
179
+ Values to interpret as null values when reading CSV/TSV
180
+ --theme [THEME] Set the theme for the application. If used without value, show available themes.
181
+ --all-in-one, --aio, --one
182
+ Read all files (must be of same format and same structure) into one single table.
183
+ --expr EXPR Specify a Polars expression to filter data (e.g., $age > 30)
184
+ --sql SQL Specify a SQL query to execute on the input file (e.g., to select and filter data)
185
+ -o, --output OUTPUT Output file (optionally modified) with specified format, which is inferred from file extension (e.g., .csv, .xlsx).
186
+ ```
187
+
188
+ ### CLI Examples
189
+
190
+ ```bash
191
+ # Open a file
192
+ dv data.csv
193
+
194
+ # Gzipped files are supported
195
+ dv data.csv.gz
196
+
197
+ # Read from stdin (defaults to TSV)
198
+ cat data.tsv | dv
199
+ dv < data.tsv
200
+
201
+ # Specify delimiter
202
+ dv data.txt -d '|'
203
+
204
+ # Specify format
205
+ dv data.json -f ndjson
206
+
207
+ # View headless CSV file
208
+ dv data_no_header.csv -H
209
+
210
+ # View headless CSV file with provided header
211
+ dv data_no_header.csv -H country ip requests
212
+
213
+ # Skip first 3 rows (e.g., metadata)
214
+ dv data_with_meta.csv -K 3
215
+
216
+ # Skip 1 row after header (e.g., units row)
217
+ dv data_with_units.csv -A 1
218
+
219
+ # Skip 3 rows before header and 1 row after
220
+ dv messy_scientific_data.csv -K 3 -A 1
221
+
222
+ # Skip comment lines (or just -C)
223
+ dv commented_data.csv -C '#'
224
+
225
+ # Disable type inference for faster loading
226
+ dv large_data.csv -I
227
+
228
+ # Ignore parsing errors in malformed CSV
229
+ dv data_with_errors.csv -E
230
+
231
+ # Treat specific values as null (e.g., 'NA', 'N/A')
232
+ dv data.csv -N NA N/A
233
+
234
+ # Use different quote character (e.g., single quote for CSV)
235
+ dv data.csv -Q "'"
236
+
237
+ # Disable quote character processing for TSV with embedded quotes
238
+ dv data.tsv -Q
239
+
240
+ # Choose the `monokai` theme
241
+ dv data.csv --theme monokai
242
+
243
+ # Show column headers
244
+ dv data.csv -F
245
+
246
+ # Read only specific columns: 'name', 'age', first column, and last column
247
+ dv data.csv -F name age 1 -1
248
+
249
+ # Read all files (must be of same format and same structure) into one single table
250
+ dv data-1.csv data-2.csv --all-in-one
251
+
252
+ # Filter rows before opening the TUI using a Polars expression
253
+ dv data.csv --expr '$age > 30'
254
+
255
+ # Filter data using SQL query (use 'self' as the table name)
256
+ dv data.csv --sql 'SELECT * FROM self WHERE age > 30'
257
+
258
+ # Convert to other format
259
+ dv data.csv -o data.parquet
260
+ ```
261
+
262
+ ## Keyboard Shortcuts
263
+
264
+ Shortcuts are a single key, a modifier combo (e.g., `Shift+G`), or a **leader sequence** starting with `g` or `z` (e.g., `g/`, `g_`, `zQ`).
265
+
266
+ **How Leader Mode Works:**
267
+
268
+ - Press the leader key `g` or `z` to activate the mode — a 3-second timeout begins
269
+ - Press the next key within the timeout to execute the combined command
270
+ - If no second key is pressed within 3 seconds, leader mode is cancelled.
271
+
272
+ **Important:** press `z` then `Ctrl+H` to open a **Commands** tab, where all commands and keybindings can be viewed and modified. In this tab, edit a `Leader` or `Key` cell with `e` or double-click to capture the next key press as the binding. Save the Commands tab to persist changes.
273
+
274
+ ![Commands](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/commands.png)
275
+
276
+ ### Memorize `g` vs `z` Quickly
277
+
278
+ Use this as a practical memory aid (not a strict rule):
279
+
280
+ - `g` often means broader scope, global-style behavior, or toward left/up.
281
+ - `z` is often the opposite: narrower/specific variants, toward right/down, or transform-style/less common actions.
282
+
283
+ Why this helps:
284
+
285
+ - In alphabet order, `g` comes before `z`, so it is easier to remember `g` as global/earlier/wider and `z` as later/narrower.
286
+ - This pattern appears repeatedly in the command registry and default keybinding registry.
287
+
288
+ Useful examples from current bindings:
289
+
290
+ - Scope widening:
291
+ - `,` selects matching rows in current column, while `g,` selects in all columns.
292
+ - `/` searches current column, while `g/` searches all columns.
293
+ - `?` searches backward in current column, while `g?` searches backward in all columns.
294
+ - Left/up vs right/down:
295
+ - `g*` delete current column and those before (left), while `z*` deletes current column and those after (right).
296
+ - `gd` delete current row and those above, while `zd` deletes current row and those below.
297
+ - `gb` move tab left, while `zb` moves tab right.
298
+ - Specific/transform-style variants under `z`:
299
+ - `z/` and `z?` search with cursor value (specialized variant).
300
+ - `z:` joins selected columns.
301
+ - `zT` transposes rows/columns.
302
+ - `za` adds a link column from a URL template.
303
+
304
+ **Tip**: Learn the base key first, then its `g` and `z` variants as a small family.
305
+
306
+ ### App-Level Controls
307
+
308
+ #### File & Tab Management
309
+
310
+ | Key | Action |
311
+ | -------------- | ---------------------------------------------------------- |
312
+ | `q` | Quit current tab (prompts to save unsaved changes) or view |
313
+ | `gq` | Quit all tabs then app (prompts to save unsaved changes) |
314
+ | `Ctrl+Q` | Force quit app (discards unsaved changes) |
315
+ | `gB` | Toggle tab bar visibility |
316
+ | `B` | Previous tab |
317
+ | `zB` | Jump to last focused tab |
318
+ | `b` | Next tab |
319
+ | `gb` | Move current tab left (wrap to last) |
320
+ | `zb` | Move current tab right (wrap to first) |
321
+ | `Ctrl+T` | Save current tab (or current view) to file |
322
+ | `Ctrl+S` | Save all tabs to file |
323
+ | `w` | Save current tab to file (overwrite without prompt) |
324
+ | `gw` | Save all tabs to file (overwrite without prompt) |
325
+ | `Ctrl+D` | Duplicate current tab |
326
+ | `Ctrl+O` | Open file in a new tab |
327
+ | `Ctrl+N` | Create new tab from Polars expression |
328
+ | `Double-click` | Rename tab |
329
+
330
+ **Tips:**
331
+
332
+ - Tabs with unsaved changes are indicated with a bright background
333
+ - Closing a tab with unsaved changes triggers a save prompt
334
+
335
+ #### View & Settings
336
+
337
+ | Key | Action |
338
+ | ------------------------- | ------------------------------------ |
339
+ | `F1` | Toggle help panel |
340
+ | `S` | Show information for all open tabs |
341
+ | `z Ctrl+H` / `zBackspace` | Show all commands and key bindings |
342
+ | `Ctrl+P` -> `Screenshot` | Capture terminal view as a SVG image |
343
+ | `gT` | Select theme |
344
+
345
+ #### Join Tables
346
+
347
+ | Key | Action |
348
+ | --- | ------------------------------ |
349
+ | `&` | Join two tables into a new tab |
350
+
351
+ ---
352
+
353
+ ### Table-Level Controls
354
+
355
+ #### Undo/Redo/Reset
356
+
357
+ | Key | Action |
358
+ | ---- | ----------------------- |
359
+ | `u` / `U` | Undo last action |
360
+ | `R` | Redo last undone action |
361
+ | `gU` | Reset to initial state |
362
+
363
+ #### Navigation
364
+
365
+ | Key | Action |
366
+ | ------------------------ | ----------------------------------- |
367
+ | `gg` | Go to first row |
368
+ | `G` | Go to last row |
369
+ | `Ctrl+G` | Go to specific row |
370
+ | `←` / `↓` / `↑` / `→` | Move left/down/up/right |
371
+ | `h` / `j` / `k` / `l` | Move left/down/up/right (Vim-style) |
372
+ | `gh` / `g←` | Scroll to leftmost column |
373
+ | `gj` / `g↓` | Scroll to last row |
374
+ | `gk` / `g↑` | Scroll to first row |
375
+ | `gl` / `g→` | Scroll to rightmost column |
376
+ | `Home` / `End` | Go to first/last column |
377
+ | `Ctrl+Home` / `Ctrl+End` | Go to page top/bottom |
378
+ | `PageUp` / `Ctrl+B` | Page backward |
379
+ | `PageDown` / `Ctrl+F` | Page forward |
380
+
381
+ #### Display
382
+
383
+ | Key | Action |
384
+ | ----------------- | -------------------------------------------------------------- |
385
+ | `Enter` | Show details for the current row as two-column key–value pairs |
386
+ | `Tab` | Show current cell details |
387
+ | `C` | Show metadata for all columns (name and data type) |
388
+ | `F` | Show frequency distribution for current or selected columns |
389
+ | `I` | Show statistics for current column |
390
+ | `gI` | Show statistics for all columns |
391
+ | `=` | Show histogram for current column |
392
+ | `g=` | Show histogram for current column with custom bins |
393
+ | `z=` | Toggle inline bar chart display for current numeric column |
394
+ | `-` (minus) | Hide selected columns or current column |
395
+ | `g-` (minus) | Hide current column and all columns before it |
396
+ | `z-` (minus) | Hide current column and all columns after it |
397
+ | `gv` | Show all hidden columns |
398
+ | `$` | Toggle 1-based column index prefixes |
399
+ | `z!` | Toggle freeze rows and/or columns |
400
+ | `_` (underscore) | Toggle column full width for current column |
401
+ | `g_` (underscore) | Toggle column full width for all string/list columns |
402
+ | `z_` (underscore) | Resize current column |
403
+ | `z,` | Toggle thousand separator for current column |
404
+ | `<` | Decrease float precision for current column |
405
+ | `>` | Increase float precision for current column |
406
+ | `g^` | Set current row as the new header row |
407
+ | `Shift+Tab` | Cycle cursor type (cell -> row -> column) |
408
+
409
+ #### Editing
410
+
411
+ | Key | Action |
412
+ | -------------- | ------------------------------------------------------------------------- |
413
+ | `Double-click` | Edit cell or rename column header |
414
+ | `Delete` | Clear current cell (set to NULL) |
415
+ | `Shift+Delete` | Clear current column (set matching cells to NULL) |
416
+ | `e` | Edit current cell (respects data type) |
417
+ | `E` | Edit entire column with value/expression |
418
+ | `a` | Add empty column after current |
419
+ | `A` | Add column with name and value/expression |
420
+ | `i` | Add index column after current |
421
+ | `za` | Add a link column from URL template |
422
+ | `^` | Rename current column |
423
+ | `*` | Delete selected columns or current column |
424
+ | `g*` | Delete current column and all columns before it |
425
+ | `z*` | Delete current column and all columns after it |
426
+ | `d` | Delete current row |
427
+ | `gd` | Delete current row and all those above |
428
+ | `zd` | Delete current row and all those below |
429
+ | `D` | Duplicate current row |
430
+ | `zD` | Duplicate current column |
431
+ | `zU` | Remove duplicate rows (keep first occurrence) |
432
+ | `:` | Split current string column into a new column by delimiter |
433
+ | `g:` | Glue items of a list column into a string column by delimiter |
434
+ | `z:` | Join all selected columns into a new string column by delimiter |
435
+ | `Ctrl+U` | Convert current or selected string column(s) to uppercase |
436
+ | `Ctrl+L` | Convert current or selected string column(s) to lowercase |
437
+ | `zSpace` | Strip leading and trailing whitespaces in current string column |
438
+ | `zf` | Fill null values in current column with a value |
439
+ | `gf` | Fill null values in all columns with a value |
440
+ | `(` | Expand current list column into indexed columns (e.g. `col[1]`, `col[2]`) |
441
+ | `)` | Contract indexed sibling columns (`col[N]`) back into a list column |
442
+ | `z(` | Explode current list column into multiple rows |
443
+ | `z)` | Implode current column into a list column |
444
+ | `zT` | Transpose table (swap rows and columns) |
445
+
446
+ #### Row/Column Selection
447
+
448
+ | Key | Action |
449
+ | ---------------- | ------------------------------------------------------------------------------ |
450
+ | `,` | Select rows with cell matches or those matching cursor value in current column |
451
+ | `g,` | Select rows with cell matches or those matching cursor value in all columns |
452
+ | `\|` (pipe) | Select rows where expression matches in current column |
453
+ | `g\|` | Select rows where expression matches in all columns |
454
+ | `\` | Unselect selected rows where expression matches in current column |
455
+ | `g\` | Unselect selected rows where expression matches in all columns |
456
+ | `{` | Go to previous selected row |
457
+ | `}` | Go to next selected row |
458
+ | `s` | Select/deselect current row |
459
+ | `gs` | Select current row and all rows above |
460
+ | `zs` | Select current row and all rows below |
461
+ | `gu` | Unselect all rows |
462
+ | `'` (apostrophe) | Select/deselect current column |
463
+ | `g'` | Select current column and all columns to the left |
464
+ | `z'` | Select current column and all columns to the right |
465
+ | `t` | Toggle row selections (invert) |
466
+ | `T` | Clear all row/column selections and cell matches |
467
+
468
+ #### Find & Replace
469
+
470
+ | Key | Action |
471
+ | ---- | --------------------------------------------------------------- |
472
+ | `/` | Search forward in current column with expression |
473
+ | `g/` | Search forward in all columns with expression |
474
+ | `z/` | Search forward in current column with cursor value |
475
+ | `?` | Search backward in current column with expression |
476
+ | `g?` | Search backward in all columns with expression |
477
+ | `z?` | Search backward in current column with cursor value |
478
+ | `;` | Search cursor value in all columns and go to first match |
479
+ | `n` | Go to next matching cell |
480
+ | `N` | Go to previous matching cell |
481
+ | `r` | Find and replace in current column (interactive or replace all) |
482
+ | `gr` | Find and replace in all columns (interactive or replace all) |
483
+
484
+ #### Filter & Collect
485
+
486
+ | Key | Action |
487
+ | ------------------ | ------------------------------------------------------ |
488
+ | `v` | Filter rows with cursor value in the current column |
489
+ | `V` | Filter rows with specified value or expression |
490
+ | `f` | Filter rows using values in the current column |
491
+ | `.` | Filter rows with non-null values in the current column |
492
+ | `z.` | Filter rows with null values in the current column |
493
+ | `"` (double quote) | Collect rows/columns to a new tab |
494
+
495
+ #### Sorting (supporting multiple columns)
496
+
497
+ | Key | Action |
498
+ | --- | ------------------------------ |
499
+ | `[` | Sort current column ascending |
500
+ | `]` | Sort current column descending |
501
+
502
+ #### Reordering
503
+
504
+ | Key | Action |
505
+ | ----------------- | ------------------------------ |
506
+ | `H` / `Shift+←` | Move current column left |
507
+ | `J` / `Shift+↓` | Move current row down |
508
+ | `K` / `Shift+↑` | Move current row up |
509
+ | `L` / `Shift+→` | Move current column right |
510
+ | `gH` / `gShift+←` | Move column to start |
511
+ | `gJ` / `gShift+↓` | Move row to bottom |
512
+ | `gK` / `gShift+↑` | Move row to top |
513
+ | `gL` / `gShift+→` | Move column to end |
514
+ | `!` | Pin column to start and freeze |
515
+
516
+ #### Type Casting
517
+
518
+ | Key | Action |
519
+ | ---- | ------------------------------ |
520
+ | `~` | Cast current column to string |
521
+ | `@` | Cast current column to date |
522
+ | `#` | Cast current column to integer |
523
+ | `%` | Cast current column to float |
524
+ | `z#` | Cast current column to boolean |
525
+
526
+ #### Copy
527
+
528
+ | Key | Action |
529
+ | -------- | ------------------------------------- |
530
+ | `c` | Copy current cell to clipboard |
531
+ | `Ctrl+C` | Copy column to clipboard |
532
+ | `Ctrl+R` | Copy row to clipboard (tab-separated) |
533
+
534
+ #### SQL & Advanced
535
+
536
+ | Key | Action |
537
+ | -------- | ---------------------------------------------------------- |
538
+ | `Q` | SQL query interface (full SQL query with syntax highlight) |
539
+ | `zQ` | SQL query interface (select columns & where clause) |
540
+ | `Space` | Toggle Python console |
541
+ | `gSpace` | Run a command by name with optional arguments |
542
+
543
+ ## Features in Detail
544
+
545
+ ### 1. Undo/Redo/Reset
546
+
547
+ These actions are the fastest way to get out of trouble after a mistaken edit, delete, sort, filter, or other table change.
548
+
549
+ - `U`: Undo the last action and restore the previous state.
550
+ - `R`: Redo the last undone action.
551
+ - `gU`: Reset the table to its original loaded state if you want to start over.
552
+
553
+ ### 2. Display & UI
554
+
555
+ Columns are automatically styled based on their data types (auto-inferred):
556
+
557
+ | Data Type | Text Color | Alignment |
558
+ | --------- | ---------- | --------- |
559
+ | integer | Cyan | right |
560
+ | float | Yellow | right |
561
+ | string | Green | left |
562
+ | boolean | Blue | centered |
563
+ | temporal | Magenta | centered |
564
+
565
+ These controls change how the table is shown without changing the underlying data.
566
+
567
+ - `-`: Hide selected columns, or the current column if nothing is selected.
568
+ - `g-`: Hide the current column and all columns before it.
569
+ - `z-`: Hide the current column and all columns after it.
570
+ - `gv`: Show all hidden columns.
571
+ - `z!`: Freeze rows and/or columns to keep important areas visible while scrolling.
572
+ - `$`: Toggle a 1-based index prefix in visible column headers such as `1_colname`.
573
+ - `_`: Toggle full width for the current column.
574
+ - `g_`: Toggle full width for all string/list columns.
575
+ - `z_`: Resize the current column from a prompt.
576
+ - `z,`: Toggle the thousand separator for the current numeric column.
577
+ - `<` / `>`: Decrease or increase float precision for the current float column. Each column keeps its own precision setting, and `0` means the default full display.
578
+ - `z=`: Toggle inline bar chart display for the current numeric column. When active, each cell is rendered as a Rich `Bar`, normalized to that column's min/max range. Press `z=` again to restore normal value display.
579
+
580
+ ### 3. Modal Screen
581
+
582
+ Several features open a **modal screen** (an overlay table) for inspection or interaction. The following modals share a common set of keyboard shortcuts:
583
+
584
+ | Modal Screen | Opened With | Purpose |
585
+ | --------------- | ----------- | ------------------------------------------ |
586
+ | Sheets Overview | `S` | Summary of all open tabs |
587
+ | Column Metadata | `C` | Column name, data type, and width |
588
+ | Row Detail | `Enter` | All column values for one row |
589
+ | Cell Detail | `Tab` | Drill into a single cell value |
590
+ | Frequency | `F` | Value distribution for a column |
591
+ | Statistics | `I` / `gI` | Summary statistics for column or dataframe |
592
+ | Histogram | `=` / `g=` | Numeric distribution as histogram |
593
+
594
+ **Common keys available in all modal screens:**
595
+
596
+ | Key | Action |
597
+ | -------------- | -------------------------------------------- |
598
+ | `q` / `Escape` | Close the modal |
599
+ | `gg` | Scroll to top |
600
+ | `G` | Scroll to bottom |
601
+ | `[` | Sort by current column ascending |
602
+ | `]` | Sort by current column descending |
603
+ | `,` | Toggle thousand separator for numeric values |
604
+ | `Shift+Tab` | Cycle cursor type (cell → row → column) |
605
+ | `T` | Open modal data as a new tab |
606
+ | `Ctrl+S` | Save the modal table to file |
607
+
608
+ Individual modals may add extra keys on top of these (documented in each subsection below).
609
+
610
+ ### 4. Sheets Overview
611
+
612
+ Press `S` to open a modal providing a summary view of all currently opened tabs.
613
+
614
+ ![Sheets](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/sheets.png)
615
+
616
+ The modal displays a table with the following columns:
617
+
618
+ | Column | Description |
619
+ | ------------ | ----------------------- |
620
+ | **Tab** | Display name of the tab |
621
+ | **#Rows** | Total number of rows |
622
+ | **#Cols** | Number of columns |
623
+ | **Filename** | Source file path |
624
+
625
+ **Keys inside the modal:**
626
+
627
+ | Key | Action |
628
+ | ------- | ----------------------------------------------------------- |
629
+ | `Enter` | Close the modal and switch to the tab under the cursor |
630
+ | `e` | Rename the tab under the cursor |
631
+ | `d` | Close the tab under the cursor (prompts if unsaved changes) |
632
+ | `s` | Select or deselect the tab under the cursor |
633
+ | `&` | Join exactly two selected tabs into a new tab |
634
+
635
+ This is useful for quickly navigating between tabs, reviewing file sizes at a glance, closing tabs you no longer need without switching to them first, or selecting two tabs and pressing `&` to open the join-table modal.
636
+
637
+ ### 5. Column Overview
638
+
639
+ Press `C` to open a modal displaying details for all columns:
640
+
641
+ - **Column** - Column name
642
+ - **Type** - Data type (e.g., Int64, String, Float64, Boolean)
643
+ - **Width** - Column width
644
+
645
+ **Keys inside the modal**
646
+
647
+ - Press `Enter` to jump to the selected column in the main table and close the modal
648
+ - Press `F` to show the frequency table for the selected column
649
+ - Press `I` to show the statistics table for the selected column
650
+ - Press `J` or `Shift+↓` to move the selected column right (and move the metadata row down)
651
+ - Press `K` or `Shift+↑` to move the selected column left (and move the metadata row up)
652
+ - Press `e` to rename, resize, or cast the selected column
653
+ - Press `d` to delete the selected column from the main table
654
+
655
+ ### 6. Column Statistics
656
+
657
+ Show summary statistics such as count, unique count, null count, mean, median, standard deviation, min, max, sum, and etc.
658
+
659
+ - `I` shows statistics for the current column
660
+ - `gI` shows statistics for all columns in the dataframe
661
+
662
+ ![Column statistics](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/column-statistics.png)
663
+
664
+ This is useful for:
665
+
666
+ - Understanding data distributions and overall column behavior
667
+ - Identifying outliers and anomalies
668
+ - Checking data quality quickly
669
+ - Reviewing summary statistics
670
+ - Comparing columns at a glance
671
+
672
+ ### 7. Row Detail View
673
+
674
+ Press `Enter` on any row to open a modal showing all column values for that row.
675
+ Useful for examining wide table where columns don't fit well on screen.
676
+
677
+ ![Row detail](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/row-detail.png)
678
+
679
+ **Keys inside the modal**:
680
+
681
+ - Press `v` to **filter** all rows containing the selected column value
682
+ - Press `"` to **collect** all rows containing the selected column value to a new tab
683
+ - Press `{` to show the previous row
684
+ - Press `}` to show the next row
685
+ - Press `F` to show the frequency table for the selected column
686
+ - Press `I` to show the statistics table for the selected column
687
+ - Press `Tab` to open a cell-detail modal for the selected field
688
+
689
+ ### 8. Cell Detail View
690
+
691
+ Press `Tab` in the main table to inspect the current cell in its own modal for complex data or long text
692
+
693
+ You can also press `Tab` from the Row Detail modal to drill into the selected field.
694
+
695
+ Inside the cell-detail modal, press `Tab` again on the selected row/column to keep drilling into nested values.
696
+
697
+ - Scalar values are displayed inline. For long text, press `Tab` again to view the full content in a multi-line modal.
698
+ - String values are split into multiple rows using `|` by default
699
+ - List-like values are expanded into a one-column table
700
+ - Dict-like values are shown as key/value columns
701
+
702
+ ### 9. Frequency Distribution
703
+
704
+ Press `F` to see value distributions for the current column. If multiple columns are selected, it shows frequency of value combinations across those selected columns.
705
+
706
+ ![Frequency distribution](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/frequency-distribution.png)
707
+
708
+ **Keys inside the modal**:
709
+
710
+ - Press `v` to **filter** rows matching the selected value (or selected value combinations)
711
+ - Press `"` to **collect** rows matching the selected value (or selected value combinations) to a new tab
712
+ - Press `s` to select or deselect the current frequency row
713
+
714
+ This is useful for:
715
+
716
+ - Understanding value distributions
717
+ - Quickly filtering to specific values
718
+ - Identifying rare or common values
719
+ - Finding the most/least frequent entries
720
+
721
+ ### 10. Row/Column Selection
722
+
723
+ The application provides multiple ways to select rows (for filtering or collecting) and columns (for hiding or deleting):
724
+
725
+ - `,` - Select rows with cell matches or those matching cursor value in current column (respects data type)
726
+ - `g,` - Select rows with cell matches or those matching cursor value in all columns
727
+ - `|` - Select rows where expression matches in the current column
728
+ - `g|` - Select rows where expression matches in all columns
729
+ - `\` - Unselect currently selected rows where expression matches in the current column
730
+ - `g\` - Unselect currently selected rows where expression matches in all columns
731
+ - `s` - Select/deselect current row
732
+ - `gs` - Select current row and all rows above
733
+ - `zs` - Select current row and all rows below
734
+ - `u` - Unselect the current row
735
+ - `gu` - Unselect all rows
736
+ - `'` (apostrophe) - Select/deselect current column
737
+ - `g'` - Select current column and all columns to the left
738
+ - `z'` - Select current column and all columns to the right
739
+ - `t` - Flip selections of all rows
740
+ - `T` - Clear all row selections, column selections, and cell matches
741
+ - `{` - Go to previous selected row
742
+ - `}` - Go to next selected row
743
+
744
+ **Advanced Options**:
745
+
746
+ When searching, you can use checkboxes in the dialog to enable:
747
+
748
+ - Match option `Nocase` for case-insensitive matching
749
+ - Match option `Whole` to match full text
750
+ - Match option `Literal` to ignore special regex characters
751
+ - Match option `Reverse` to perform reverse match
752
+
753
+ These options work with plain text searches. Use Polars regex patterns in expressions for more control. For example, use `(?i)` prefix in regex (e.g., `(?i)john`) for case-insensitive matching.
754
+
755
+ **Quick Tips:**
756
+
757
+ - Search results highlight matching rows in **red**
758
+ - Use expression for advanced selection (e.g., $attack > $defense)
759
+ - Type-aware matching automatically converts values. Resort to string comparison if conversion fails
760
+
761
+ ### 11. Find & Replace
762
+
763
+ Find by value/expression and highlight matching cells:
764
+
765
+ - `/` - Search forward in current column with expression
766
+ - `g/` - Search forward in all columns with expression
767
+ - `z/` - Search forward in current column with cursor value
768
+ - `?` - Search backward in current column with expression
769
+ - `g?` - Search backward in all columns with expression
770
+ - `z?` - Search backward in current column with cursor value
771
+ - `;` - Search cursor value in all columns and go to first match
772
+ - `n` - Go to next matching cell
773
+ - `N` - Go to previous matching cell
774
+
775
+ Replace values in current column (`r`) or in all columns (`gr`).
776
+
777
+ **How It Works:**
778
+
779
+ When you press `r` or `gr`, enter:
780
+
781
+ 1. **Find term**: Value or expression to search for (done by string value)
782
+ 2. **Replace term**: Replacement value
783
+ 3. **Matching options**:
784
+ - `Nocase` for case-insensitive matching
785
+ - `Whole` to match full text
786
+ - `Literal` to ignore special regex characters
787
+ - `Reverse` to perform reverse match
788
+ 4. **Replace mode**: All at once or interactive review
789
+
790
+ **Replace All**:
791
+
792
+ - Replaces all matches with one operation
793
+ - Shows confirmation with match count
794
+
795
+ **Replace Interactive**:
796
+
797
+ - Review each match one at a time (confirm, skip, or cancel)
798
+ - Shows progress
799
+
800
+ **Tips:**
801
+
802
+ - Search are done by string value (i.e., ignoring data type)
803
+ - Type `NULL` to find or replace null values
804
+
805
+ ### 12. Filter & Collect
806
+
807
+ Both actions work on a subset of the original dataframe, but they serve different workflows.
808
+
809
+ **Filtering options**:
810
+
811
+ **Basic Filter** (`v`):
812
+
813
+ - Opens the chose subset as a derived view inside the current workflow
814
+ - Edits made in the filtered view still apply to the original dataframe
815
+ - Press `Ctrl+T` to save the current view to a file
816
+ - Press `q` to leave the filtered view and return to the main table
817
+
818
+ **Advanced Filter** (`V`):
819
+
820
+ - Opens a dialog for value-based or expression-based filtering
821
+ - Useful when you want to define the subset directly
822
+
823
+ ![Advanced Filter](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/advanced-filter.png)
824
+
825
+ **Column Filter** (`f`):
826
+
827
+ - Opens a type-aware filter dialog for the current column
828
+ - Numeric columns support `=`, `!=`, `<`, `<=`, `>=`, and `>`
829
+ - String columns support exact match, prefix, suffix, contains, and regex matching
830
+ - Boolean columns support true, false, and null filtering
831
+ - Temporal columns support the same comparison operators as numeric columns
832
+ - List columns support exact-list matching and item membership checks such as "contains"
833
+
834
+ ![Column Filter](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/column-filter.png)
835
+
836
+ **Collect** (`"`):
837
+
838
+ - Creates a separate tab containing only the chosen rows/columns
839
+ - The collected tab is independent from the source dataframe
840
+ - Edits in the collected tab do not modify the original table
841
+
842
+ For **Basic Filter** (`v`) and **Collect** (`"`), rows are chosen in this order:
843
+
844
+ - Use selected rows if any are present
845
+ - Otherwise, use rows with active matches from search or find
846
+ - Otherwise, use rows whose current-column value matches the current cell
847
+
848
+ ### 13. Sorting
849
+
850
+ - Press `[` to sort current column ascending
851
+ - Press `]` to sort current column descending
852
+ - Multi-column sorting supported (press multiple times on different columns)
853
+ - Press same key twice to remove the current column from sorting
854
+
855
+ ![Sorting](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/sort-column.png)
856
+
857
+ ### 14. Editing
858
+
859
+ Editing covers cell updates, structural table changes, and quick cleanup.
860
+
861
+ - `e` or **Double-click**: Edit the current cell with type-aware validation.
862
+ - `^` or **Double-click** column header: Rename the current column.
863
+ - `d`: Delete the selected rows, or the current row if nothing is selected.
864
+ - `gd`: Delete the current row and all rows above it.
865
+ - `zd`: Delete the current row and all rows below it.
866
+ - `*`: Delete selected columns, or the current column if nothing is selected.
867
+ - `g*`: Delete the current column and all columns to its left.
868
+ - `z*`: Delete the current column and all columns to its right.
869
+ - `a`: Add an empty column after the current column.
870
+ - `A`: Add a column after the current column using a value or expression such as `$age * 2`.
871
+ - `i`: Insert an index column after the current column.
872
+ - `D`: Duplicate the current row.
873
+ - `zD`: Duplicate the current column using a `_copy` suffix.
874
+ - `zU`: Remove duplicate rows while keeping the first occurrence, based on visible-column values.
875
+ - `:`: Split the current string column into a new list column using a delimiter.
876
+ - `(`: Expand the current list column into indexed columns named like `colname[1]`, `colname[2]`, etc.
877
+ - `)`: Contract those indexed sibling columns back into a single list column. Position the cursor on any sibling (e.g. `colname[2]`) and press `)` to merge all `colname[N]` columns back into `colname`.
878
+ - `Ctrl+U`: Convert the current column, or all selected columns that are string type, to uppercase.
879
+ - `Ctrl+L`: Convert the current column, or all selected columns that are string type, to lowercase.
880
+ - `zSpace`: Strip leading and trailing whitespaces in the current string column.
881
+
882
+ ### 15. Column & Row Reordering
883
+
884
+ **Move Columns**: `Shift+←` and `Shift+→`
885
+
886
+ - Swaps adjacent columns
887
+ - `H` and `L` provide the same left/right movement
888
+ - `gH` / `gShift+←` moves column to start
889
+ - `gL` / `gShift+→` moves column to end
890
+ - Reorder is preserved when saving
891
+
892
+ **Move Rows**: `Shift+↑` and `Shift+↓`
893
+
894
+ - Swaps adjacent rows
895
+ - `J` and `K` provide the same down/up movement
896
+ - `gK` / `gShift+↑` moves row to top
897
+ - `gJ` / `gShift+↓` moves row to bottom
898
+ - Reorder is preserved when saving
899
+
900
+ **Pin Column**: `!`
901
+
902
+ - Moves the current column to the leftmost unfrozen position and freezes it
903
+ - If columns are already frozen, appends to the frozen set (increments `fixed_columns`)
904
+
905
+ ### 16. Save File
906
+
907
+ The application provides save actions for the current tab (or active view) and all tabs.
908
+
909
+ **Save Current Tab** (`Ctrl+T`):
910
+
911
+ - Saves the active tab to file
912
+ - If currently in a derived/filtered view, saves the current view instead
913
+ - Useful when you want to export only the dataframe you are currently working on
914
+
915
+ **Save All Tabs** (`Ctrl+S`):
916
+
917
+ - Saves every open tab to file
918
+ - Useful after editing multiple datasets in the same session
919
+
920
+ The output format is determined by the file extension, making it easy to convert between formats such as CSV, TSV, Parquet, or Excel.
921
+
922
+ ![Save File](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/save-file.png)
923
+
924
+ ### 17. Clipboard Operations
925
+
926
+ Copies value to system clipboard with `pbcopy` on macOS and `xclip` on Linux.
927
+
928
+ **Note**: may require a X server to work.
929
+
930
+ - Press `c` to copy cursor value
931
+ - Press `Ctrl+C` to copy column values
932
+ - Press `Ctrl+R` to copy row values (delimited by tab)
933
+ - Hold `Shift` to select with mouse
934
+
935
+ ### 18. [Polars Expressions](https://docs.pola.rs/api/python/stable/reference/expressions/index.html)
936
+
937
+ Complex values, filters, and advanced operations can be specified via Polars expressions, with the following adaptions for convenience:
938
+
939
+ **Column References:**
940
+
941
+ - `$_` - Current column (based on cursor position)
942
+ - `$1`, `$2`, etc. - Column by 1-based index
943
+ - `$age`, `$salary` - Column by name (use actual column names)
944
+ - `` $`col name` `` - Column by name with spaces (backtick quoted)
945
+
946
+ **Row References:**
947
+
948
+ - `$#` - Current row index (1-based)
949
+
950
+ **DataFrame References:**
951
+
952
+ - `self` - Current dataframe
953
+
954
+ **Basic Comparisons:**
955
+
956
+ - `$_ > 50` - Current column greater than 50
957
+ - `$salary >= 100000` - Salary at least 100,000
958
+ - `$age < 30` - Age less than 30
959
+ - `$status == 'active'` - Status exactly matches 'active'
960
+ - `$name != 'Unknown'` - Name is not 'Unknown'
961
+ - `$# <= 10` - Top 10 rows
962
+
963
+ **Logical Operators:**
964
+
965
+ - `&` - AND
966
+ - `|` - OR
967
+ - `~` - NOT
968
+
969
+ **Practical Examples:**
970
+
971
+ - `($age < 30) & ($status == 'active')` - Age less than 30 AND status is active
972
+ - `($name == 'Alice') | ($name == 'Bob')` - Name is Alice or Bob
973
+ - `$salary / 1000 >= 50` - Salary divided by 1,000 is at least 50
974
+ - `($department == 'Sales') & ($bonus > 5000)` - Sales department with bonus over 5,000
975
+ - `($score >= 80) & ($score <= 90)` - Score between 80 and 90
976
+ - `~($status == 'inactive')` - Status is not inactive
977
+ - `$revenue > $expenses` - Revenue exceeds expenses
978
+ - ``$`product id` > 100`` - Product ID with spaces in column name greater than 100
979
+ - `self.drop(RID).is_duplicated()` - Duplicate rows (note: the internal RID column must be excluded)
980
+
981
+ **String Operations:** ([Polars string API reference](https://docs.pola.rs/api/python/stable/reference/series/string.html))
982
+
983
+ - `$name.str.contains("John")` - Name contains "John" (case-sensitive)
984
+ - `$name.str.contains("(?i)john")` - Name contains "john" (case-insensitive)
985
+ - `$email.str.ends_with("@company.com")` - Email ends with domain
986
+ - `$code.str.starts_with("ABC")` - Code starts with "ABC"
987
+ - `$name.str.len_chars() < 7` - Length of name shorter than 7
988
+
989
+ **Number Operations:**
990
+
991
+ - `$age * 2 > 100` - Double age greater than 100
992
+ - `($salary + $bonus) > 150000` - Total compensation over 150,000
993
+ - `$percentage >= 50` - Percentage at least 50%
994
+
995
+ **Null Handling:**
996
+
997
+ - `$column.is_null()` - Find null values
998
+ - `$column.is_not_null()` - Find non-null values
999
+ - `NULL` - a value to represent null for convenience
1000
+
1001
+ **Tips:**
1002
+
1003
+ - Use column indices (e.g., `$1`, `$2`) for faster column access.
1004
+ - Use column names that match exactly (case-sensitive)
1005
+ - Use parentheses to clarify complex expressions: `($a & $b) | ($c & $d)`
1006
+
1007
+ ### 19. Link Column Creation
1008
+
1009
+ Press `za` to create a new column containing dynamically generated URLs using template. Links are typically clickable in a terminal emulator using Ctrl+Click.
1010
+
1011
+ **Template Placeholders:**
1012
+
1013
+ The link template supports multiple placeholder types for maximum flexibility:
1014
+
1015
+ - **`$_`** - Current column, e.g., `https://example.com/search/$_` - Uses values from the current column
1016
+ - **`$1`, `$2`, `$3`, etc.** - Column by 1-based position index, e.g., `https://example.com/product/$1/details/$2` - Uses 1st and 2nd columns
1017
+ - **`$name`** - Column by name (use actual column names), e.g., `https://example.com/$region/$city/data` - Uses `region` and `city` columns
1018
+
1019
+ **Features:**
1020
+
1021
+ - **Multiple Placeholders**: Mix and match placeholders in a single template
1022
+ - **URL Prefix**: Automatically prepends `https://` if URL doesn't start with `http://` or `https://`
1023
+
1024
+ ### 20. Join Tables
1025
+
1026
+ Press `&` from the main table to open the join-table modal. The modal lets you choose the left and right tables, select matching key columns from each side, choose the join type, and create the joined result as a new tab.
1027
+
1028
+ You can also start from the Sheets Overview: press `S`, select exactly two tabs with `s`, then press `&` to open the same join-table modal with those two tables pre-selected.
1029
+
1030
+ Supported join types include inner, left, right, full, semi, and anti joins. Select the same number of key columns on both sides before pressing **Join**.
1031
+
1032
+ ![Join Table](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/join-table.png)
1033
+
1034
+ ### 21. SQL Interface
1035
+
1036
+ The SQL interface provides two modes for querying your dataframe:
1037
+
1038
+ #### Advanced SQL Interface (`Q`)
1039
+
1040
+ Execute complete SQL queries for advanced data manipulation:
1041
+
1042
+ - Write full SQL queries with standard [SQL syntax](https://docs.pola.rs/api/python/stable/reference/sql/index.html)
1043
+ - Access to all SQL capabilities for complex transformations
1044
+ - Always use `self` as the table name
1045
+ - Syntax highlighted
1046
+
1047
+ ![SQL Interface](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/sql-query.png)
1048
+
1049
+ #### Simple SQL Interface (`zQ`)
1050
+
1051
+ SELECT specific columns and apply WHERE conditions without writing full SQL:
1052
+
1053
+ - Choose which columns to include in results
1054
+ - Specify WHERE clause for filtering
1055
+ - Ideal for quick filtering and column selection
1056
+
1057
+ ![SQL Interface](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/sql-simple.png)
1058
+
1059
+ **Examples:**
1060
+
1061
+ ```sql
1062
+ -- Filter and select specific rows and/or columns
1063
+ SELECT name, age
1064
+ FROM self
1065
+ WHERE age > 30
1066
+
1067
+ -- Use backticks (`) for column names with spaces
1068
+ SELECT *
1069
+ FROM self
1070
+ WHERE `product id` = 7
1071
+ ```
1072
+
1073
+ ### 22. Python Console
1074
+
1075
+ Use the built-in Python console for quick interactive transformations without leaving the TUI.
1076
+
1077
+ - Open/close console: `Space`
1078
+ - Run shell commands: prefix with `!` (example: `!ls`)
1079
+ - In console: `clear` or `cls` clears console output
1080
+ - In console: `Esc` closes the console panel
1081
+ - Available names: `df` (active DataFrame), `self` (active table), `app` (viewer app), `pl` (Polars)
1082
+ - Assign a DataFrame or Series back to `df` to refresh the current table immediately
1083
+
1084
+ ![Python Console](https://raw.githubusercontent.com/need47/dataframe-textual/refs/heads/main/screenshots/python-console.png)
1085
+
1086
+ ## Handling Loading Errors
1087
+
1088
+ Most loading failures come from malformed CSV/TSV input, quoting issues, or mixed column types. When this happens, the application prints a hint with a suggested retry option.
1089
+
1090
+ **Common fixes:**
1091
+
1092
+ - Use `-Q` if quote characters are mismatched, improperly escaped, or should be ignored entirely
1093
+ - Use `-T` when rows contain more fields than expected and you want to truncate ragged lines
1094
+ - Use `-L` to increase the number of rows used for schema inference when early rows do not represent the full column types
1095
+ - Use `-I` to disable type inference and read CSV/TSV values as strings
1096
+ - Check the delimiter and format options if the input appears empty or unreadable
1097
+ - Use `-E` as a last resort to ignore recoverable parsing errors
1098
+
1099
+ **Typical cases:**
1100
+
1101
+ **Malformed CSV or broken quoting**:
1102
+
1103
+ - Symptom: errors mentioning malformed CSV, mismatched quotes, or improperly escaped fields
1104
+ - Try: `-Q` to disable quoting or choose a different quote character
1105
+
1106
+ **Ragged lines or inconsistent field counts**:
1107
+
1108
+ - Symptom: errors saying the input has more fields than defined in the schema
1109
+ - Try: `-T` to truncate ragged lines
1110
+
1111
+ **Mixed data types in one column**:
1112
+
1113
+ - Symptom: errors saying a value could not be parsed as an integer, float, or other inferred type at a specific column
1114
+ - Try: `-L` first, then `-I` if the column is genuinely mixed
1115
+
1116
+ **No data could be loaded**:
1117
+
1118
+ - Symptom: errors indicating that no data was available to load
1119
+ - Check: whether the file is empty, the format is correct, and the delimiter matches the input
1120
+
1121
+ **Fallback option**:
1122
+
1123
+ - If none of the above helps and the file is mostly usable, retry with `-E` to ignore parsing errors
1124
+
1125
+ **Examples:**
1126
+
1127
+ ```bash
1128
+ # Disable quote handling when CSV quoting is broken
1129
+ dv bad.csv -Q
1130
+
1131
+ # Truncate ragged lines
1132
+ dv messy.tsv -T
1133
+
1134
+ # Increase schema inference depth
1135
+ dv mixed_types.csv -L 1000
1136
+
1137
+ # Disable type inference entirely
1138
+ dv mixed_types.csv -I
1139
+
1140
+ # Ignore recoverable parsing errors
1141
+ dv partially_broken.csv -E
1142
+ ```
1143
+
1144
+ ## Dependencies
1145
+
1146
+ - **polars**: Fast DataFrame library for data loading/processing
1147
+ - **textual**: Terminal UI framework
1148
+ - **fastexcel**: Read Excel files
1149
+ - **xlsxwriter**: Write Excel files
1150
+ - **vortex-data**: Read/Write Vortex files (optional)
1151
+
1152
+ ## Requirements
1153
+
1154
+ - Python 3.11+
1155
+ - POSIX-compatible terminal (macOS, Linux, WSL)
1156
+ - Terminal supporting ANSI escape sequences and mouse events
1157
+
1158
+ ## Acknowledgments
1159
+
1160
+ - Inspired by [VisiData](https://visidata.org/)
1161
+ - Built with [Textual](https://textual.textualize.io/) and [Polars](https://www.pola.rs/)