dataframe-textual 0.1.0__py3-none-any.whl → 1.1.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of dataframe-textual might be problematic. Click here for more details.
- dataframe_textual/__main__.py +65 -0
- dataframe_textual/common.py +340 -0
- {dataframe_viewer → dataframe_textual}/data_frame_help_panel.py +22 -4
- dataframe_textual/data_frame_table.py +2768 -0
- dataframe_textual/data_frame_viewer.py +472 -0
- dataframe_textual/table_screen.py +490 -0
- dataframe_textual/yes_no_screen.py +672 -0
- dataframe_textual-1.1.0.dist-info/METADATA +726 -0
- dataframe_textual-1.1.0.dist-info/RECORD +13 -0
- dataframe_textual-1.1.0.dist-info/entry_points.txt +2 -0
- dataframe_textual-0.1.0.dist-info/METADATA +0 -522
- dataframe_textual-0.1.0.dist-info/RECORD +0 -13
- dataframe_textual-0.1.0.dist-info/entry_points.txt +0 -2
- dataframe_viewer/__main__.py +0 -48
- dataframe_viewer/common.py +0 -204
- dataframe_viewer/data_frame_table.py +0 -1395
- dataframe_viewer/data_frame_viewer.py +0 -320
- dataframe_viewer/table_screen.py +0 -311
- dataframe_viewer/yes_no_screen.py +0 -409
- {dataframe_viewer → dataframe_textual}/__init__.py +0 -0
- {dataframe_textual-0.1.0.dist-info → dataframe_textual-1.1.0.dist-info}/WHEEL +0 -0
- {dataframe_textual-0.1.0.dist-info → dataframe_textual-1.1.0.dist-info}/licenses/LICENSE +0 -0
|
@@ -0,0 +1,726 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dataframe-textual
|
|
3
|
+
Version: 1.1.0
|
|
4
|
+
Summary: Interactive terminal viewer/editor for tabular data
|
|
5
|
+
Project-URL: Homepage, https://github.com/need47/dataframe-textual
|
|
6
|
+
Project-URL: Repository, https://github.com/need47/dataframe-textual.git
|
|
7
|
+
Project-URL: Documentation, https://github.com/need47/dataframe-textual#readme
|
|
8
|
+
Project-URL: Bug Tracker, https://github.com/need47/dataframe-textual/issues
|
|
9
|
+
Author-email: Tiejun Cheng <need47@gmail.com>
|
|
10
|
+
License: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Keywords: csv,data-analysis,editor,excel,interactive,polars,terminal,textual,tui,viewer
|
|
13
|
+
Classifier: Development Status :: 3 - Alpha
|
|
14
|
+
Classifier: Environment :: Console
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Intended Audience :: End Users/Desktop
|
|
17
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
18
|
+
Classifier: Natural Language :: English
|
|
19
|
+
Classifier: Operating System :: MacOS
|
|
20
|
+
Classifier: Operating System :: POSIX
|
|
21
|
+
Classifier: Operating System :: Unix
|
|
22
|
+
Classifier: Programming Language :: Python :: 3
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
24
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
25
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
26
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
27
|
+
Classifier: Topic :: Office/Business
|
|
28
|
+
Classifier: Topic :: Utilities
|
|
29
|
+
Classifier: Typing :: Typed
|
|
30
|
+
Requires-Python: >=3.11
|
|
31
|
+
Requires-Dist: polars>=1.34.0
|
|
32
|
+
Requires-Dist: textual>=6.5.0
|
|
33
|
+
Provides-Extra: dev
|
|
34
|
+
Requires-Dist: textual-dev>=1.8.0; extra == 'dev'
|
|
35
|
+
Provides-Extra: excel
|
|
36
|
+
Requires-Dist: fastexcel>=0.16.0; extra == 'excel'
|
|
37
|
+
Requires-Dist: xlsxwriter>=3.2.9; extra == 'excel'
|
|
38
|
+
Description-Content-Type: text/markdown
|
|
39
|
+
|
|
40
|
+
# DataFrame Textual
|
|
41
|
+
|
|
42
|
+
A powerful, interactive terminal-based viewer/editor for CSV/TSV/Excel/Parquet/JSON/NDJSON built with Python, [Polars](https://pola.rs/), and [Textual](https://textual.textualize.io/). Inspired by [VisiData](https://www.visidata.org/), this tool provides smooth keyboard navigation, data manipulation, and a clean interface for exploring tabular data directly in terminal with multi-tab support for multiple files!
|
|
43
|
+
|
|
44
|
+

|
|
45
|
+
|
|
46
|
+
## Features
|
|
47
|
+
|
|
48
|
+
### Core Data Viewing
|
|
49
|
+
- 🚀 **Fast Loading** - Powered by Polars for efficient data handling
|
|
50
|
+
- 🎨 **Rich Terminal UI** - Beautiful, color-coded columns with automatic type detection
|
|
51
|
+
- ⌨️ **Comprehensive Keyboard Navigation** - Intuitive controls for browsing, editing, and manipulating data
|
|
52
|
+
- 📊 **Flexible Input** - Read from files or stdin (pipes/redirects)
|
|
53
|
+
- 🔄 **Smart Pagination** - Lazy load rows on demand for handling large datasets
|
|
54
|
+
|
|
55
|
+
### Data Manipulation
|
|
56
|
+
- 📝 **Data Editing** - Edit cells, delete rows, and remove columns
|
|
57
|
+
- 🔍 **Search & Filter** - Find values, highlight matches, and filter selected rows
|
|
58
|
+
- ↔️ **Column/Row Reordering** - Move columns and rows with simple keyboard shortcuts
|
|
59
|
+
- 📈 **Sorting & Statistics** - Multi-column sorting and frequency distribution analysis
|
|
60
|
+
- 💾 **Save & Undo** - Save edits back to file with full undo/redo support
|
|
61
|
+
|
|
62
|
+
### Advanced Features
|
|
63
|
+
- 📂 **Multi-File Support** - Open multiple files in tabs for side-by-side comparison
|
|
64
|
+
- 🔄 **Tab Management** - Seamlessly switch between open files with keyboard shortcuts
|
|
65
|
+
- 📌 **Freeze Rows/Columns** - Keep important rows and columns visible while scrolling
|
|
66
|
+
- 🎯 **Cursor Type Cycling** - Switch between cell, row, and column selection modes
|
|
67
|
+
|
|
68
|
+
## Installation
|
|
69
|
+
|
|
70
|
+
### Using pip
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
# Install from PyPI
|
|
74
|
+
pip install dataframe-textual
|
|
75
|
+
|
|
76
|
+
# With Excel support (fastexcel, xlsxwriter)
|
|
77
|
+
pip install dataframe-textual[excel]
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
This installs an executable `dv`.
|
|
81
|
+
|
|
82
|
+
Then run:
|
|
83
|
+
```bash
|
|
84
|
+
dv <csv_file>
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
### Using [uv](https://docs.astral.sh/uv/)
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
# Quick run using uvx without installation
|
|
91
|
+
uvx https://github.com/need47/dataframe-textual.git <csvfile>
|
|
92
|
+
|
|
93
|
+
# Clone or download the project
|
|
94
|
+
cd dataframe-textual
|
|
95
|
+
|
|
96
|
+
# Run directly with uv
|
|
97
|
+
uv run dv <csv_file>
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### Development installation
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
# Clone the repository
|
|
104
|
+
git clone https://github.com/need47/dataframe-textual.git
|
|
105
|
+
cd dataframe-textual
|
|
106
|
+
|
|
107
|
+
# Install from local source
|
|
108
|
+
pip install -e .
|
|
109
|
+
|
|
110
|
+
# Or with development dependencies
|
|
111
|
+
pip install -e ".[excel,dev]"
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## Usage
|
|
115
|
+
|
|
116
|
+
### Basic Usage - Single File
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
# After pip install dataframe-textual
|
|
120
|
+
dv pokemon.csv
|
|
121
|
+
|
|
122
|
+
# Or if running from source
|
|
123
|
+
python main.py pokemon.csv
|
|
124
|
+
|
|
125
|
+
# Or with uv
|
|
126
|
+
uv run python main.py pokemon.csv
|
|
127
|
+
|
|
128
|
+
# Read from stdin (auto-detects format; defaults to TSV if not recognized)
|
|
129
|
+
cat data.tsv | dv
|
|
130
|
+
dv < data.tsv
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
### Multi-File Usage - Multiple Tabs
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
# Open multiple files in tabs
|
|
137
|
+
dv file1.csv file2.csv file3.csv
|
|
138
|
+
|
|
139
|
+
# Open multiple sheets in tabs in an Excel file
|
|
140
|
+
dv file.xlsx
|
|
141
|
+
|
|
142
|
+
# Mix files and stdin (read from stdin, then open file)
|
|
143
|
+
dv data1.tsv < data2.tsv
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
When multiple files are opened:
|
|
147
|
+
- Each file appears as a separate tab at the top
|
|
148
|
+
- Switch between tabs using `>` (next) or `<` (previous)
|
|
149
|
+
- Open additional files with `Ctrl+O`
|
|
150
|
+
- Close the current tab with `Ctrl+W`
|
|
151
|
+
- Each file maintains its own state (edits, sort order, selections, history, etc.)
|
|
152
|
+
|
|
153
|
+
## Keyboard Shortcuts
|
|
154
|
+
|
|
155
|
+
### App-Level Controls
|
|
156
|
+
|
|
157
|
+
#### File & Tab Management
|
|
158
|
+
|
|
159
|
+
| Key | Action |
|
|
160
|
+
|-----|--------|
|
|
161
|
+
| `Ctrl+O` | Open file in a new tab |
|
|
162
|
+
| `Ctrl+W` | Close current tab |
|
|
163
|
+
| `Ctrl+Shift+S` | Save all open tabs to Excel file |
|
|
164
|
+
| `>` or `b` | Move to next tab |
|
|
165
|
+
| `<` | Move to previous tab |
|
|
166
|
+
| `B` | Toggle tab bar visibility |
|
|
167
|
+
| `q` | Quit the application |
|
|
168
|
+
|
|
169
|
+
#### View & Settings
|
|
170
|
+
|
|
171
|
+
| Key | Action |
|
|
172
|
+
|-----|--------|
|
|
173
|
+
| `Ctrl+H` | Toggle help panel |
|
|
174
|
+
| `k` | Cycle through themes |
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
### Table-Level Controls
|
|
179
|
+
|
|
180
|
+
#### Navigation
|
|
181
|
+
|
|
182
|
+
| Key | Action |
|
|
183
|
+
|-----|--------|
|
|
184
|
+
| `g` | Jump to first row |
|
|
185
|
+
| `G` | Jump to last row (loads all remaining rows) |
|
|
186
|
+
| `↑` / `↓` | Move up/down one row |
|
|
187
|
+
| `←` / `→` | Move left/right one column |
|
|
188
|
+
| `Home` / `End` | Jump to first/last column in current row |
|
|
189
|
+
| `Ctrl + Home` / `Ctrl + End` | Jump to top/bottom in current page |
|
|
190
|
+
| `PageDown` / `PageUp` | Scroll down/up one page |
|
|
191
|
+
|
|
192
|
+
#### Viewing & Display
|
|
193
|
+
|
|
194
|
+
| Key | Action |
|
|
195
|
+
|-----|--------|
|
|
196
|
+
| `Enter` | View full details of current row in modal |
|
|
197
|
+
| `F` | Show frequency distribution for column |
|
|
198
|
+
| `s` | Show statistics for current column |
|
|
199
|
+
| `S` | Show statistics for entire dataframe |
|
|
200
|
+
| `K` | Cycle cursor type: cell → row → column → cell |
|
|
201
|
+
| `~` | Toggle row labels |
|
|
202
|
+
|
|
203
|
+
#### Data Editing
|
|
204
|
+
|
|
205
|
+
| Key | Action |
|
|
206
|
+
|-----|--------|
|
|
207
|
+
| `Double-click` | Edit cell or rename column header |
|
|
208
|
+
| `X` | Clear current cell (set to NULL) |
|
|
209
|
+
| `e` | Edit current cell (respects data type) |
|
|
210
|
+
| `E` | Edit entire column with expression |
|
|
211
|
+
| `a` | Add empty column after current |
|
|
212
|
+
| `A` | Add column with name and value/expression |
|
|
213
|
+
| `-` (minus) | Delete current column |
|
|
214
|
+
| `x` | Delete current row |
|
|
215
|
+
| `d` | Duplicate current column (appends '_copy' suffix) |
|
|
216
|
+
| `D` | Duplicate current row |
|
|
217
|
+
| `h` | Hide current column |
|
|
218
|
+
| `H` | Show all hidden columns |
|
|
219
|
+
|
|
220
|
+
#### Searching & Filtering
|
|
221
|
+
|
|
222
|
+
| Key | Action |
|
|
223
|
+
|-----|--------|
|
|
224
|
+
| `\` | Search in current column using cursor value and select rows |
|
|
225
|
+
| `\|` (pipe) | Search in current column with expression and select rows |
|
|
226
|
+
| `/` | Find in current column with cursor value and highlight matches |
|
|
227
|
+
| `?` | Find in current column with expression and highlight matches |
|
|
228
|
+
| `n` | Go to next match |
|
|
229
|
+
| `N` | Go to previous match |
|
|
230
|
+
| `{` | Go to previous selected row |
|
|
231
|
+
| `}` | Go to next selected row |
|
|
232
|
+
| `'` | Select/deselect current row |
|
|
233
|
+
| `t` | Toggle selected rows (invert) |
|
|
234
|
+
| `T` | Clear all selected rows and/or matches |
|
|
235
|
+
| `"` (quote) | Filter to selected rows only |
|
|
236
|
+
| `v` | View only rows by selected rows and/or matches or cursor value |
|
|
237
|
+
| `V` | View only rows by expression |
|
|
238
|
+
|
|
239
|
+
#### Find & Replace
|
|
240
|
+
|
|
241
|
+
| Key | Action |
|
|
242
|
+
|-----|--------|
|
|
243
|
+
| `f` | Find across all columns with cursor value |
|
|
244
|
+
| `Ctrl+F` | Find across all columns with expression |
|
|
245
|
+
| `r` | Find and replace in current column (interactive or replace all) |
|
|
246
|
+
| `R` | Find and replace across all columns (interactive or replace all) |
|
|
247
|
+
|
|
248
|
+
#### Sorting
|
|
249
|
+
|
|
250
|
+
| Key | Action |
|
|
251
|
+
|-----|--------|
|
|
252
|
+
| `[` | Sort current column ascending |
|
|
253
|
+
| `]` | Sort current column descending |
|
|
254
|
+
|
|
255
|
+
#### Reordering
|
|
256
|
+
|
|
257
|
+
| Key | Action |
|
|
258
|
+
|-----|--------|
|
|
259
|
+
| `Shift+↑` | Move current row up |
|
|
260
|
+
| `Shift+↓` | Move current row down |
|
|
261
|
+
| `Shift+←` | Move current column left |
|
|
262
|
+
| `Shift+→` | Move current column right |
|
|
263
|
+
|
|
264
|
+
#### Type Conversion
|
|
265
|
+
|
|
266
|
+
| Key | Action |
|
|
267
|
+
|-----|--------|
|
|
268
|
+
| `#` | Cast current column to integer (Int64) |
|
|
269
|
+
| `%` | Cast current column to float (Float64) |
|
|
270
|
+
| `!` | Cast current column to boolean |
|
|
271
|
+
| `$` | Cast current column to string |
|
|
272
|
+
| `@` | Make URLs in current column clickable with Ctrl/Cmd + click|
|
|
273
|
+
|
|
274
|
+
#### Data Management
|
|
275
|
+
|
|
276
|
+
| Key | Action |
|
|
277
|
+
|-----|--------|
|
|
278
|
+
| `z` | Freeze rows and columns |
|
|
279
|
+
| `,` | Toggle thousand separator for numeric display |
|
|
280
|
+
| `c` | Copy current cell to clipboard |
|
|
281
|
+
| `Ctrl+C` | Copy column to clipboard |
|
|
282
|
+
| `Ctrl+R` | Copy row to clipboard (tab-separated) |
|
|
283
|
+
| `Ctrl+S` | Save current tab to file |
|
|
284
|
+
| `u` | Undo last action |
|
|
285
|
+
| `U` | Reset to original data |
|
|
286
|
+
|
|
287
|
+
## Features in Detail
|
|
288
|
+
|
|
289
|
+
### 1. Color-Coded Data Types
|
|
290
|
+
|
|
291
|
+
Columns are automatically styled based on their data type:
|
|
292
|
+
- **integer**: Cyan text, right-aligned
|
|
293
|
+
- **float**: Magenta text, right-aligned
|
|
294
|
+
- **string**: Green text, left-aligned
|
|
295
|
+
- **boolean**: Blue text, centered
|
|
296
|
+
- **temporal**: Yellow text, centered
|
|
297
|
+
|
|
298
|
+
### 2. Row Detail View
|
|
299
|
+
|
|
300
|
+
Press `Enter` on any row to open a modal showing all column values for that row.
|
|
301
|
+
Useful for examining wide datasets where columns don't fit on screen.
|
|
302
|
+
|
|
303
|
+
**In the Row Detail Modal**:
|
|
304
|
+
- Press `v` to **view** the main table to show only rows with the selected column value
|
|
305
|
+
- Press `"` to **filter** all rows containing the selected column value
|
|
306
|
+
- Press `q` or `Escape` to close the modal
|
|
307
|
+
|
|
308
|
+
### 3. Search & Filtering
|
|
309
|
+
|
|
310
|
+
The application provides multiple search modes for different use cases:
|
|
311
|
+
|
|
312
|
+
**Search Operations** - Direct value/expression matching in current column:
|
|
313
|
+
- **`|` - Column Expression Search**: Opens dialog to search current column with custom expression
|
|
314
|
+
- **`\` - Column Cursor Search**: Instantly search current column using the cursor value
|
|
315
|
+
|
|
316
|
+
**Find Operations** - Find by value/expression:
|
|
317
|
+
- **`/` - Column Find**: Find cursor value within current column
|
|
318
|
+
- **`?` - Column Expression Find**: Open dialog to search current column with expression
|
|
319
|
+
- **`f` - Global Find**: Find cursor value across all columns
|
|
320
|
+
- **`Ctrl+f` - Global Expression Find**: Open dialog to search all columns with expression
|
|
321
|
+
|
|
322
|
+
**Selection & Filtering**:
|
|
323
|
+
- **`'` - Toggle Row Selection**: Select/deselect current row (marks it for filtering)
|
|
324
|
+
- **`t` - Invert Selections**: Flip selection state of all rows at once
|
|
325
|
+
- **`T` - Clear Selections**: Remove all row selections and matches
|
|
326
|
+
- **`"` - Filter Selected**: Display only the selected rows and remove others
|
|
327
|
+
- **`v` - View by Value**: Filter/view rows by selected rows or cursor value (others hidden but preserved)
|
|
328
|
+
- **`V` - View by Expression**: Filter/view rows using custom Polars expression (others hidden but preserved)
|
|
329
|
+
|
|
330
|
+
**Advanced Matching Options**:
|
|
331
|
+
|
|
332
|
+
When searching or finding, you can use checkboxes in the dialog to enable:
|
|
333
|
+
- **Match Nocase**: Ignore case differences (e.g., "john", "John", "JOHN" all match)
|
|
334
|
+
- **Match Whole**: Match complete value, not partial substrings or words (e.g., "cat" won't match in "catfish")
|
|
335
|
+
|
|
336
|
+
These options work with plain text searches. Use Polars regex patterns in expressions for more control:
|
|
337
|
+
- **Case-insensitive matching in expressions**: Use `(?i)` prefix in regex (e.g., `(?i)john`)
|
|
338
|
+
- **Word boundaries in expressions**: Use `\b` in regex (e.g., `\bjohn\b` matches whole word)
|
|
339
|
+
|
|
340
|
+
**Quick Tips:**
|
|
341
|
+
- Search results highlight matching rows/cells in **red**
|
|
342
|
+
- Multiple searches **accumulate selections** - each new search adds to the selections
|
|
343
|
+
- Type-aware matching automatically converts values. Resort to string comparison if conversion fails
|
|
344
|
+
- Use `u` to undo any search or filter
|
|
345
|
+
|
|
346
|
+
### 3b. Find & Replace
|
|
347
|
+
|
|
348
|
+
The application provides powerful find and replace functionality for both single-column and global replacements.
|
|
349
|
+
|
|
350
|
+
**Replace Operations**:
|
|
351
|
+
- **`r` - Column Replace**: Replace values in the current column
|
|
352
|
+
- **`R` - Global Replace**: Replace values across all columns
|
|
353
|
+
|
|
354
|
+
**How It Works:**
|
|
355
|
+
|
|
356
|
+
When you press `r` or `R`, a dialog opens where you can enter:
|
|
357
|
+
1. **Find term**: The value or expression to search for
|
|
358
|
+
2. **Replace term**: What to replace matches with
|
|
359
|
+
3. **Matching options**:
|
|
360
|
+
- **Match Nocase**: Ignore case differences when matching (unchecked by default)
|
|
361
|
+
- **Match Whole**: Match complete words only, not partial words (unchecked by default)
|
|
362
|
+
4. **Replace option**:
|
|
363
|
+
- Choose **"Replace All"** to replace all matches at once (with confirmation)
|
|
364
|
+
- Otherwise, review and confirm each match individually
|
|
365
|
+
|
|
366
|
+
**Replace All** (`r` or `R` → Choose "Replace All"):
|
|
367
|
+
- Shows a confirmation dialog with the number of matches and replacements
|
|
368
|
+
- Replaces all matches with a single operation
|
|
369
|
+
- Full undo support with `u`
|
|
370
|
+
- Useful for bulk replacements when you're confident about the change
|
|
371
|
+
|
|
372
|
+
**Replace Interactive** (`r` or `R` → Choose "Replace Interactive"):
|
|
373
|
+
- Shows each match one at a time with a preview of the replacement
|
|
374
|
+
- For each match, press:
|
|
375
|
+
- `Enter` or press the `Yes` button - **Replace this occurrence** and move to next
|
|
376
|
+
- Press the `Skip` button - **Skip this occurrence** and move to next
|
|
377
|
+
- `Escape` or press the `No` button - **Cancel** remaining replacements (but keep already-made replacements)
|
|
378
|
+
- Displays progress: `Occurrence X of Y` (Y = total matches, X = current)
|
|
379
|
+
- Shows the value that will be replaced and what it will become
|
|
380
|
+
- Useful for careful replacements where you want to review each change
|
|
381
|
+
|
|
382
|
+
**Search Term Types:**
|
|
383
|
+
- **Plain text**: Exact string match (e.g., "John" finds "John")
|
|
384
|
+
- Use **Match Nocase** checkbox to match regardless of case (e.g., find "john", "John", "JOHN")
|
|
385
|
+
- Use **Match Whole** checkbox to match complete words only (e.g., find "cat" but not in "catfish")
|
|
386
|
+
- **NULL**: Replace null/missing values (type `NULL`)
|
|
387
|
+
- **Expression**: Polars expressions for complex matching (e.g., `$_ > 50` for column replace)
|
|
388
|
+
- **Regex patterns**: Use Polars regex syntax for advanced matching
|
|
389
|
+
- Case-insensitive: Use `(?i)` prefix (e.g., `(?i)john`)
|
|
390
|
+
- Whole word: Use `\b` boundary markers (e.g., `\bjohn\b`)
|
|
391
|
+
|
|
392
|
+
**Examples:**
|
|
393
|
+
|
|
394
|
+
```
|
|
395
|
+
Find: "John"
|
|
396
|
+
Replace: "Jane"
|
|
397
|
+
→ All occurrences of "John" become "Jane"
|
|
398
|
+
|
|
399
|
+
Find: "john"
|
|
400
|
+
Replace: "jane"
|
|
401
|
+
Match Nocase: ✓ (checked)
|
|
402
|
+
→ "John", "JOHN", "john" all become "jane"
|
|
403
|
+
|
|
404
|
+
Find: "cat"
|
|
405
|
+
Replace: "dog"
|
|
406
|
+
Match Whole: ✓ (checked)
|
|
407
|
+
→ "cat" becomes "dog", but "catfish" is not matched
|
|
408
|
+
|
|
409
|
+
Find: "NULL"
|
|
410
|
+
Replace: "Unknown"
|
|
411
|
+
→ All null/missing values become "Unknown"
|
|
412
|
+
|
|
413
|
+
Find: "(?i)active" # Case-insensitive
|
|
414
|
+
Replace: "inactive"
|
|
415
|
+
→ "Active", "ACTIVE", "active" all become "inactive"
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
**For Global Replace (`R`)**:
|
|
419
|
+
- Searches and replaces across all columns simultaneously
|
|
420
|
+
- Each column can have different matching behavior (string matching for text, numeric for numbers)
|
|
421
|
+
- Preview shows which columns contain matches before replacement
|
|
422
|
+
- Useful for standardizing values across multiple columns
|
|
423
|
+
|
|
424
|
+
**Features:**
|
|
425
|
+
- **Full history support**: Use `u` (undo) to revert any replacement
|
|
426
|
+
- **Visual feedback**: Matching cells are highlighted before you choose replacement mode
|
|
427
|
+
- **Safe operations**: Requires confirmation before replacing
|
|
428
|
+
- **Progress tracking**: Shows how many replacements have been made during interactive mode
|
|
429
|
+
- **Type-aware**: Respects column data types when matching and replacing
|
|
430
|
+
- **Flexible matching**: Support for case-insensitive and whole-word matching
|
|
431
|
+
|
|
432
|
+
**Tips:**
|
|
433
|
+
- Use interactive mode for one-time replacements to be absolutely sure
|
|
434
|
+
- Use "Replace All" for routine replacements (e.g., fixing typos, standardizing formats)
|
|
435
|
+
- Use **Match Nocase** for matching variations of names or titles
|
|
436
|
+
- Use **Match Whole** to avoid unintended partial replacements
|
|
437
|
+
- Use `u` immediately if you accidentally replace something wrong
|
|
438
|
+
- For complex replacements, use Polars expressions or regex patterns in the find term
|
|
439
|
+
- Test with a small dataset first before large replacements
|
|
440
|
+
|
|
441
|
+
### 4. [Polars Expressions](https://docs.pola.rs/api/python/stable/reference/expressions/index.html)
|
|
442
|
+
|
|
443
|
+
Complex values or filters can be specified via Polars expressions, with the following adaptions for convenience:
|
|
444
|
+
|
|
445
|
+
**Column References:**
|
|
446
|
+
- `$_` - Current column (based on cursor position)
|
|
447
|
+
- `$1`, `$2`, etc. - Column by 1-based index
|
|
448
|
+
- `$age`, `$salary` - Column by name (use actual column names)
|
|
449
|
+
|
|
450
|
+
**Row References:**
|
|
451
|
+
- `$#` - Current row index (1-based)
|
|
452
|
+
|
|
453
|
+
**Basic Comparisons:**
|
|
454
|
+
- `$_ > 50` - Current column greater than 50
|
|
455
|
+
- `$salary >= 100000` - Salary at least 100,000
|
|
456
|
+
- `$age < 30` - Age less than 30
|
|
457
|
+
- `$status == 'active'` - Status exactly matches 'active'
|
|
458
|
+
- `$name != 'Unknown'` - Name is not 'Unknown'
|
|
459
|
+
|
|
460
|
+
**Logical Operators:**
|
|
461
|
+
- `&` - AND
|
|
462
|
+
- `|` - OR
|
|
463
|
+
- `~` - NOT
|
|
464
|
+
|
|
465
|
+
**Practical Examples:**
|
|
466
|
+
- `($age < 30) & ($status == 'active')` - Age less than 30 AND status is active
|
|
467
|
+
- `($name == 'Alice') | ($name == 'Bob')` - Name is Alice or Bob
|
|
468
|
+
- `$salary / 1000 >= 50` - Salary divided by 1,000 is at least 50
|
|
469
|
+
- `($department == 'Sales') & ($bonus > 5000)` - Sales department with bonus over 5,000
|
|
470
|
+
- `($score >= 80) & ($score <= 90)` - Score between 80 and 90
|
|
471
|
+
- `~($status == 'inactive')` - Status is not inactive
|
|
472
|
+
- `$revenue > $expenses` - Revenue exceeds expenses
|
|
473
|
+
|
|
474
|
+
**String Matching:**
|
|
475
|
+
- `$name.str.contains("John")` - Name contains "John" (case-sensitive)
|
|
476
|
+
- `$name.str.contains("(?i)john")` - Name contains "john" (case-insensitive)
|
|
477
|
+
- `$email.str.ends_with("@company.com")` - Email ends with domain
|
|
478
|
+
- `$code.str.starts_with("ABC")` - Code starts with "ABC"
|
|
479
|
+
- `$age.cast(pl.String).str.starts_with("7")` - Age (cast to string first) starts with "7"
|
|
480
|
+
|
|
481
|
+
**Number Operations:**
|
|
482
|
+
- `$age * 2 > 100` - Double age greater than 100
|
|
483
|
+
- `($salary + $bonus) > 150000` - Total compensation over 150,000
|
|
484
|
+
- `$percentage >= 50` - Percentage at least 50%
|
|
485
|
+
|
|
486
|
+
**Null Handling:**
|
|
487
|
+
- `$column.is_null()` - Find null/missing values
|
|
488
|
+
- `$column.is_not_null()` - Find non-null values
|
|
489
|
+
- `NULL` - a value to represent null for convenience
|
|
490
|
+
|
|
491
|
+
**Tips:**
|
|
492
|
+
- Use column names that match exactly (case-sensitive)
|
|
493
|
+
- Use parentheses to clarify complex expressions: `($a & $b) | ($c & $d)`
|
|
494
|
+
|
|
495
|
+
### 5. Sorting
|
|
496
|
+
|
|
497
|
+
- Press `[` to sort current column ascending
|
|
498
|
+
- Press `]` to sort current column descending
|
|
499
|
+
- Multi-column sorting supported (press multiple times on different columns)
|
|
500
|
+
- Press same key twice to remove the column from sorting
|
|
501
|
+
|
|
502
|
+
### 6. Frequency Distribution
|
|
503
|
+
|
|
504
|
+
Press `F` to see how many times each value appears in the current column. The modal shows:
|
|
505
|
+
- Value
|
|
506
|
+
- Count
|
|
507
|
+
- Percentage
|
|
508
|
+
- Histogram
|
|
509
|
+
- **Total row** at the bottom
|
|
510
|
+
|
|
511
|
+
**In the Frequency Table**:
|
|
512
|
+
- Press `[` and `]` to sort by any column (value, count, or percentage)
|
|
513
|
+
- Press `v` to **filter** the main table to show only rows with the selected value
|
|
514
|
+
- Press `"` to **highlight** all rows containing the selected value
|
|
515
|
+
- Press `q` or `Escape` to close the frequency table
|
|
516
|
+
|
|
517
|
+
This is useful for:
|
|
518
|
+
- Understanding value distributions
|
|
519
|
+
- Quickly filtering to specific values
|
|
520
|
+
- Identifying rare or common values
|
|
521
|
+
- Finding the most/least frequent entries
|
|
522
|
+
|
|
523
|
+
### 7. Column & Dataframe Statistics
|
|
524
|
+
|
|
525
|
+
Press `s` to see summary statistics for the current column, or press `S` for statistics across the entire dataframe.
|
|
526
|
+
|
|
527
|
+
**Column Statistics** (`s`):
|
|
528
|
+
- Shows calculated statistics using Polars' `describe()` method
|
|
529
|
+
- Displays: count, null count, mean, median, std, min, max, etc.
|
|
530
|
+
- Values are color-coded according to their data type
|
|
531
|
+
- Statistics label column has no styling for clarity
|
|
532
|
+
|
|
533
|
+
**Dataframe Statistics** (`S`):
|
|
534
|
+
- Shows statistics for all numeric and applicable columns simultaneously
|
|
535
|
+
- Data columns are color-coded by their data type (integer, float, string, etc.)
|
|
536
|
+
|
|
537
|
+
**In the Statistics Modal**:
|
|
538
|
+
- Press `q` or `Escape` to close the statistics table
|
|
539
|
+
- Use arrow keys to navigate
|
|
540
|
+
- Useful for quick data validation and summary reviews
|
|
541
|
+
|
|
542
|
+
This is useful for:
|
|
543
|
+
- Understanding data distributions and characteristics
|
|
544
|
+
- Identifying outliers and anomalies
|
|
545
|
+
- Data quality assessment
|
|
546
|
+
- Quick statistical summaries without external tools
|
|
547
|
+
- Comparing statistics across columns
|
|
548
|
+
|
|
549
|
+
### 8. Data Editing
|
|
550
|
+
|
|
551
|
+
**Edit Cell** (`e` or **Double-click**):
|
|
552
|
+
- Opens modal for editing current cell
|
|
553
|
+
- Validates input based on column data type
|
|
554
|
+
|
|
555
|
+
**Rename Column Header** (**Double-click** column header):
|
|
556
|
+
- Quick rename by double-clicking the column header
|
|
557
|
+
|
|
558
|
+
**Delete Row** (`x`):
|
|
559
|
+
- Delete all selected rows (if any) at once
|
|
560
|
+
- Or delete single row at cursor
|
|
561
|
+
|
|
562
|
+
**Delete Column** (`-`):
|
|
563
|
+
- Removes the entire column from view and dataframe
|
|
564
|
+
|
|
565
|
+
### 9. Hide & Show Columns
|
|
566
|
+
|
|
567
|
+
**Hide Column** (`h`):
|
|
568
|
+
- Temporarily hides the current column from display
|
|
569
|
+
- Column data is preserved in the dataframe
|
|
570
|
+
- Hidden columns are included in saves
|
|
571
|
+
|
|
572
|
+
**Show Hidden Columns** (`H`):
|
|
573
|
+
- Restores all previously hidden columns to the display
|
|
574
|
+
- Returns table to full column view
|
|
575
|
+
|
|
576
|
+
This is useful for:
|
|
577
|
+
- Focusing on specific columns without deleting data
|
|
578
|
+
- Temporarily removing cluttered or unnecessary columns
|
|
579
|
+
|
|
580
|
+
### 10. Duplicate Column
|
|
581
|
+
|
|
582
|
+
Press `d` to duplicate the current column:
|
|
583
|
+
- Creates a new column immediately after the current column
|
|
584
|
+
- New column has '_copy' suffix (e.g., 'price' → 'price_copy')
|
|
585
|
+
- Duplicate preserves all data from original column
|
|
586
|
+
- New column is inserted into the dataframe
|
|
587
|
+
|
|
588
|
+
This is useful for:
|
|
589
|
+
- Creating backup copies of columns before transformation
|
|
590
|
+
- Working with alternative versions of column data
|
|
591
|
+
- Comparing original vs. processed column values side-by-side
|
|
592
|
+
|
|
593
|
+
### 11. Duplicate Row
|
|
594
|
+
|
|
595
|
+
Press `D` to duplicate the current row:
|
|
596
|
+
- Creates a new row immediately after the current row
|
|
597
|
+
- Duplicate preserves all data from original row
|
|
598
|
+
- New row is inserted into the dataframe
|
|
599
|
+
|
|
600
|
+
This is useful for:
|
|
601
|
+
- Creating variations of existing data records
|
|
602
|
+
- Batch adding similar rows with modifications
|
|
603
|
+
|
|
604
|
+
### 12. Column & Row Reordering
|
|
605
|
+
|
|
606
|
+
**Move Columns**: `Shift+←` and `Shift+→`
|
|
607
|
+
- Swaps adjacent columns
|
|
608
|
+
- Reorder is preserved when saving
|
|
609
|
+
|
|
610
|
+
**Move Rows**: `Shift+↑` and `Shift+↓`
|
|
611
|
+
- Swaps adjacent rows
|
|
612
|
+
- Reorder is preserved when saving
|
|
613
|
+
|
|
614
|
+
### 13. Freeze Rows and Columns
|
|
615
|
+
|
|
616
|
+
Press `z` to open the dialog:
|
|
617
|
+
- Enter number of fixed rows and/or columns to keep top rows/columns visible while scrolling
|
|
618
|
+
|
|
619
|
+
### 13.5. Thousand Separator Toggle
|
|
620
|
+
|
|
621
|
+
Press `,` to toggle thousand separator formatting for numeric data:
|
|
622
|
+
- Applies to **integer** and **float** columns
|
|
623
|
+
- Formats large numbers with commas for readability (e.g., `1000000` → `1,000,000`)
|
|
624
|
+
- Works across all numeric columns in the table
|
|
625
|
+
- Toggle on/off as needed for different viewing preferences
|
|
626
|
+
- Display-only: does not modify underlying data in the dataframe
|
|
627
|
+
- State persists during the session
|
|
628
|
+
|
|
629
|
+
### 14. Save File
|
|
630
|
+
|
|
631
|
+
Press `Ctrl+S` to save:
|
|
632
|
+
- Save filtered, edited, or sorted data back to file
|
|
633
|
+
- Choose filename in modal dialog
|
|
634
|
+
- Confirm if file already exists
|
|
635
|
+
|
|
636
|
+
### 15. Undo/Redo
|
|
637
|
+
|
|
638
|
+
Press `u` to undo:
|
|
639
|
+
- Reverts last action with full state restoration
|
|
640
|
+
- Works for edits, deletions, sorts, searches, etc.
|
|
641
|
+
- Shows description of reverted action
|
|
642
|
+
|
|
643
|
+
Press `U` to revert to when data was initially loaded
|
|
644
|
+
|
|
645
|
+
### 16. Column Type Conversion
|
|
646
|
+
|
|
647
|
+
Press the type conversion keys to instantly cast the current column to a different data type:
|
|
648
|
+
|
|
649
|
+
**Type Conversion Shortcuts**:
|
|
650
|
+
- `#` - Cast to **integer**
|
|
651
|
+
- `%` - Cast to **float**
|
|
652
|
+
- `!` - Cast to **boolean**
|
|
653
|
+
- `$` - Cast to **string**
|
|
654
|
+
|
|
655
|
+
**Features**:
|
|
656
|
+
- Instant conversion with visual feedback
|
|
657
|
+
- Full undo support - press `u` to revert
|
|
658
|
+
- Leverage Polars' robust type casting
|
|
659
|
+
|
|
660
|
+
**Note**: Type conversion attempts to preserve data where possible. Conversions may lose data (e.g., float to int rounding).
|
|
661
|
+
|
|
662
|
+
### 17. Cursor Type Cycling
|
|
663
|
+
|
|
664
|
+
Press `K` to cycle through selection modes:
|
|
665
|
+
1. **Cell mode**: Highlight individual cell (and its row/column headers)
|
|
666
|
+
2. **Row mode**: Highlight entire row
|
|
667
|
+
3. **Column mode**: Highlight entire column
|
|
668
|
+
|
|
669
|
+
### 18. URL Handling
|
|
670
|
+
|
|
671
|
+
Press `@` to make URLs in the current column clickable:
|
|
672
|
+
- **Ctrl/Cmd + click** on URLs to open them in your default browser
|
|
673
|
+
- **Scans** all cells in the current column for URLs starting with `http://` or `https://`
|
|
674
|
+
- **Applies** link styling to make them clickable and dataframe remains unchanged
|
|
675
|
+
|
|
676
|
+
### 19. Clipboard Operations
|
|
677
|
+
|
|
678
|
+
Copies value to system clipboard with `pbcopy` on macOS and `xclip` on Linux
|
|
679
|
+
|
|
680
|
+
Press `Ctrl+C` to copy:
|
|
681
|
+
- Press `c` to copy cursor value
|
|
682
|
+
- Press `Ctrl+C` to copy column values
|
|
683
|
+
- Press `Ctrl+R` to copy row values (delimited by tab)
|
|
684
|
+
|
|
685
|
+
## Examples
|
|
686
|
+
|
|
687
|
+
### Single File Examples
|
|
688
|
+
|
|
689
|
+
```bash
|
|
690
|
+
# View Pokemon dataset
|
|
691
|
+
dv pokemon.csv
|
|
692
|
+
|
|
693
|
+
# Chain with other command and specify input file format
|
|
694
|
+
cut -d',' -f1,2,3 pokemon.csv | dv -f csv
|
|
695
|
+
```
|
|
696
|
+
|
|
697
|
+
### Multi-File/Tab Examples
|
|
698
|
+
|
|
699
|
+
```bash
|
|
700
|
+
# Open multiple sheets as tabs in a single Excel
|
|
701
|
+
dv sales.xlsx
|
|
702
|
+
|
|
703
|
+
# Open multiple files as tabs
|
|
704
|
+
dv pokemon.csv titanic.csv
|
|
705
|
+
|
|
706
|
+
# Start with one file, then open others using Ctrl+O
|
|
707
|
+
dv initial_data.csv
|
|
708
|
+
```
|
|
709
|
+
|
|
710
|
+
## Dependencies
|
|
711
|
+
|
|
712
|
+
- **polars**: Fast DataFrame library for data loading/processing
|
|
713
|
+
- **textual**: Terminal UI framework
|
|
714
|
+
- **fastexcel**: Read Excel files
|
|
715
|
+
- **xlsxwriter**: Write Excel files
|
|
716
|
+
|
|
717
|
+
## Requirements
|
|
718
|
+
|
|
719
|
+
- Python 3.11+
|
|
720
|
+
- POSIX-compatible terminal (macOS, Linux, WSL)
|
|
721
|
+
- Terminal supporting ANSI escape sequences and mouse events
|
|
722
|
+
|
|
723
|
+
## Acknowledgments
|
|
724
|
+
|
|
725
|
+
- Inspired by [VisiData](https://visidata.org/)
|
|
726
|
+
- Built with [Textual](https://textual.textualize.io/) and [Polars](https://www.pola.rs/)
|