listpick 0.1.16.0__tar.gz → 0.1.16.3__tar.gz

{listpick-0.1.16.0 → listpick-0.1.16.3}/CHANGELOG.md

@@ -1,10 +1,26 @@
 # CHANGELOG.md
 Note that the changes between 0.1.11.0 and 1.1.12.0 are listed under 0.1.11
 
-## [0.1.16.0] 2025-09-05
- - Data generation is now multithreaded.
-   - ~5x quicker when getting data for video files
+## [0.1.16.2] 2025-09-07
+ - Fixed some f-strings with nested quotes.
+
+## [0.1.16.1] 2025-09-07
+ - Massive improvements to Picker data generation.
+   - Data generation is now multithreaded.
+     - ~5x quicker when getting information on video files
+   - Data generation is now done **asynchronously**.
+     - generate_picker_data sets the header and items of the Picker with only the Files column filled. The rest of the cells have "..." as a placeholder. Multiple threads are created and they generate data for each of the cells.
+     - The Picker can receive user interaction while the data loads.
+   - Implemented a task priority queue which the threads utilise to determine their next task.
+     - Cells that are in view are prioritised in the queue and are generated first.
+   - Created generate_picker_data_from_file() function and separated the generation-specific code into the original generate_picker_data() function.
+     - This allows one to utilise the new generation capabilities by simply passing a list of functions and filenames to the generate_picker_data() function--no toml file necessary.
+       - This has been implemented in lpfman to display file attributes.
+ - Bug fixes:
+    - Static footer string was not displayed in some cases.
 
+## [0.1.16.1] 2025-09-05
+ - Data generation is now multithreaded.
 
 ## [0.1.15.20] 2025-08-31
  - Fixed screen refresh function for default options selector.

{listpick-0.1.16.0/src/listpick.egg-info → listpick-0.1.16.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: listpick
-Version: 0.1.16.0
+Version: 0.1.16.3
 Summary: Listpick is a powerful TUI data tool for creating TUI apps or viewing/comparing tabulated data.
 Home-page: https://github.com/grimandgreedy/listpick
 Author: Grim
@@ -45,7 +45,7 @@ Dynamic: summary
 
 listpick is a TUI tool which displays a tabulated list of rows and allows the user to operate upon these rows--select, copy, pipe. A very simple concept but also, I hope, a powerful tool that will make it easier for people to develop TUI apps.
 
-Rows of data can be viewed, selected, generated, saved, loaded, refreshed, modified or copied to the clipboard. Easy to integrate into your project by creating a `menu = Picker(stdscr, items:list[list[str]])` and then the menu will be displayed by running `menu.run()`.
+Rows of data can be viewed, selected, generated, saved, loaded, refreshed, modified or copied to the clipboard. Easy to integrate into your project by creating a `menu = Picker(stdscr: curses.window, items: list[list[str]])` and then the menu will be displayed by running `menu.run()`.
 
 It works great as the backend for a TUI application and can also be used as a standalone data viewer.
 
@@ -53,13 +53,13 @@ It works great as the backend for a TUI application and can also be used as a st
 
 # Quickstart
 
-Install listpick
+Install listpick:
 
 ```
 python -m pip installl "listpick[full]"
 ```
 
-Create a picker object to display rows:
+Create a Picker:
 
 ```
 from listpick.listpick_app import Picker, start_curses, close_curses
@@ -79,7 +79,7 @@ x.run()
 close_curses(stdscr)
 
 ```
-Use the listpick binary to generate and display rows based on a list of commands:
+Or use the listpick binary to generate and display rows based on a list of commands:
 
 ```
 wget https://raw.githubusercontent.com/grimandgreedy/listpick/refs/heads/master/examples/data_generation/list_files.toml

{listpick-0.1.16.0 → listpick-0.1.16.3}/README.md

@@ -2,7 +2,7 @@
 
 listpick is a TUI tool which displays a tabulated list of rows and allows the user to operate upon these rows--select, copy, pipe. A very simple concept but also, I hope, a powerful tool that will make it easier for people to develop TUI apps.
 
-Rows of data can be viewed, selected, generated, saved, loaded, refreshed, modified or copied to the clipboard. Easy to integrate into your project by creating a `menu = Picker(stdscr, items:list[list[str]])` and then the menu will be displayed by running `menu.run()`.
+Rows of data can be viewed, selected, generated, saved, loaded, refreshed, modified or copied to the clipboard. Easy to integrate into your project by creating a `menu = Picker(stdscr: curses.window, items: list[list[str]])` and then the menu will be displayed by running `menu.run()`.
 
 It works great as the backend for a TUI application and can also be used as a standalone data viewer.
 
@@ -10,13 +10,13 @@ It works great as the backend for a TUI application and can also be used as a st
 
 # Quickstart
 
-Install listpick
+Install listpick:
 
 ```
 python -m pip installl "listpick[full]"
 ```
 
-Create a picker object to display rows:
+Create a Picker:
 
 ```
 from listpick.listpick_app import Picker, start_curses, close_curses
@@ -36,7 +36,7 @@ x.run()
 close_curses(stdscr)
 
 ```
-Use the listpick binary to generate and display rows based on a list of commands:
+Or use the listpick binary to generate and display rows based on a list of commands:
 
 ```
 wget https://raw.githubusercontent.com/grimandgreedy/listpick/refs/heads/master/examples/data_generation/list_files.toml

{listpick-0.1.16.0 → listpick-0.1.16.3}/TODO.md

@@ -28,6 +28,12 @@
 > - [x] Selected column isn't highlighted properly when cell beginning is offscreen
 > - [x] Header rows not aligned properly sometimes...
 > - [ ] Check leftmost_char after resizing
+> - [ ] Kill threads when we exit the picker.
+> - [ ] Data locks when data is still being generated. No modifications.
+> - [ ] (!!!) Cell cursor value with hidden columns
+>   - [ ] The selected column can be a hidden column
+>   - [ ] Hide all columns and toggle cell cursor mode. >> Hangs
+>   - [ ] Toggle last visible column in cell cursor mode. >> Hangs
 
 
 

{listpick-0.1.16.0 → listpick-0.1.16.3}/examples/data_generation/video_duplicates.toml

@@ -7,31 +7,42 @@
 
 
 [environment]
-cwd="~/Downloads/new"
-# cwd="~/Videos/"
+cwd="~/Videos/"
 
 [data]
 # files_command = "eza -1 --no-quotes | grep -E mp4$"
 
 commands = [
+  # """find . -iname '*.m4v' -or -iname '*.webm'""",
   """eza -1 --no-quotes | grep -E mp4$""",
+  """du -hs {} | awk '{{ print $1 }}'""",
+
   # """sha1sum {} | awk '{{print $1}}'""",
+
+  ## Duration, width, height methods
+  """mediainfo --Inform="Video;%Duration/String%" {}""",
+  """mediainfo --Inform="Video;%Width%" {}""",
+  """mediainfo --Inform="Video;%Height%" {}""",
+  """mediainfo --Inform="Video;%BitRate/String%" {}""",
+  """mediainfo --Inform="Video;%CodecID%" {}""",
+  """mediainfo --Inform="Video;%BitDepth%" {}""",
+
+  ## Alternate duration, width, height methods
   # """ffprobe -show_entries format=duration -v quiet -of csv="p=0" -i {}""",
   # """ffprobe -v error -select_streams v:0 -show_entries stream=width -of csv=s=x:p=0 {}""",
   # """ffprobe -v error -select_streams v:0 -show_entries stream=height -of csv=s=x:p=0 {}""",
-  """mediainfo --Inform="Video;%Duration%" {}""",
-  """mediainfo --Inform="Video;%Width%" {}""",
-  """mediainfo --Inform="Video;%Height%" {}""",
-  """du -hs {} | awk '{{ print $1 }}'""",
 ]
 
 header = [
-  "file",
+  "File",
+  "Size",
   # "sha1",
-  "duration",
-  "width",
-  "height",
-  "size",
+  "Duration",
+  "Width",
+  "Height",
+  "Bit Rate",
+  "Codec",
+  "Bit Depth",
 ]
 
 # run = [

{listpick-0.1.16.0 → listpick-0.1.16.3}/setup.py

@@ -16,7 +16,7 @@ with open("README.md", "r", encoding = "utf-8") as fh:
 
 setuptools.setup(
     name = "listpick",
-    version = "0.1.16.0",
+    version = "0.1.16.3",
     author = "Grim",
     author_email = "grimandgreedy@protonmail.com",
     description = "Listpick is a powerful TUI data tool for creating TUI apps or viewing/comparing tabulated data.",

{listpick-0.1.16.0 → listpick-0.1.16.3}/src/listpick/listpick_app.py

@@ -36,7 +36,8 @@ from listpick.utils.paste_operations import *
 from listpick.utils.searching import search
 from listpick.ui.help_screen import help_lines
 from listpick.ui.keys import picker_keys, notification_keys, options_keys, help_keys
-from listpick.utils.generate_data import generate_picker_data
+# from listpick.utils.generate_data import generate_picker_data
+from listpick.utils.generate_data_multithreaded import generate_picker_data_from_file
 from listpick.utils.dump import dump_state, load_state, dump_data
 from listpick.ui.build_help import build_help_rows
 from listpick.ui.footer import StandardFooter, CompactFooter, NoFooter
@@ -77,7 +78,7 @@ class Picker:
         timer: float = 5,
 
         get_new_data: bool =False,                          # Whether we can get new data
-        refresh_function: Optional[Callable] = lambda: [],  # The function with which we get new data
+        refresh_function: Optional[Callable] = lambda items, header, visible_rows_indices, getting_data: None,  # The function with which we get new data
         get_data_startup: bool =False,                      # Whether we should get data at statrup
         track_entries_upon_refresh: bool = True,
         pin_cursor: bool = False,
@@ -146,7 +147,7 @@ class Picker:
         footer_style: int = 0,
         footer_string: str="",
         footer_string_auto_refresh: bool=False,
-        footer_string_refresh_function: Optional[Callable] = None,
+        footer_string_refresh_function: Optional[Callable] = lambda : None,
         footer_timer: float=1,
         get_footer_string_startup=False,
         unicode_char_width: bool = True,
@@ -197,6 +198,8 @@ class Picker:
         right_panes: list = [],
         right_pane_index: int = 0,
 
+        # getting_data: threading.Event = threading.Event(),
+
     ):
         self.stdscr = stdscr
         self.items = items
@@ -282,7 +285,7 @@ class Picker:
         self.footer_string_auto_refresh = footer_string_auto_refresh
         self.footer_string_refresh_function = footer_string_refresh_function
         self.footer_timer = footer_timer
-        self.get_footer_string_startup = get_footer_string_startup,
+        self.get_footer_string_startup = get_footer_string_startup
         self.unicode_char_width = unicode_char_width
 
 
@@ -314,7 +317,6 @@ class Picker:
 
 
         # Refresh function variables
-        self.data_refreshed = False
         self.refreshing_data = False
         self.data_lock = threading.Lock()
         self.data_ready = False
@@ -349,12 +351,22 @@ class Picker:
         self.split_right = split_right
         self.right_panes = right_panes
         self.right_pane_index = right_pane_index
+        self.visible_rows_indices = []
+
+
+
+
         self.initialise_picker_state(reset_colours=self.reset_colours)
 
         # Note: We have to set the footer after initialising the picker state so that the footer can use the get_function_data method
         self.footer_options = [StandardFooter(self.stdscr, colours_start, self.get_function_data), CompactFooter(self.stdscr, colours_start, self.get_function_data), NoFooter(self.stdscr, colours_start, self.get_function_data)]
         self.footer = self.footer_options[self.footer_style]
-        self.__version__ = "1.0"
+
+
+
+        # getting_data.is_set() is True when we are getting data
+        self.getting_data = threading.Event()
+        self.getting_data.set()
 
     def __sizeof__(self):
 
@@ -463,8 +475,11 @@ class Picker:
             end_index = min(start_index + self.items_per_page, len(self.indexed_items))
         if len(self.indexed_items) == 0: start_index, end_index = 0, 0
 
-        rows = [v[1] for v in self.indexed_items[start_index:end_index]] if len(self.indexed_items) else self.items
-        return rows
+        self.visible_rows = [v[1] for v in self.indexed_items[start_index:end_index]] if len(self.indexed_items) else self.items
+        # self.visible_rows_indices = [v[0] for v in self.indexed_items[start_index:end_index]] if len(self.indexed_items) else []
+        self.visible_rows_indices.clear()
+        self.visible_rows_indices.extend([v[0] for v in self.indexed_items[start_index:end_index]])
+        return self.visible_rows
 
     def initialise_picker_state(self, reset_colours=False) -> None:
         """ Initialise state variables for the picker. These are: debugging and colours. """
@@ -558,7 +573,10 @@ class Picker:
                     self.cursor_pos_id = self.indexed_items[self.cursor_pos][1][self.id_column]
                     self.cursor_pos_prev = self.cursor_pos
         
-            self.items, self.header = self.refresh_function()
+
+
+            self.getting_data.clear()
+            self.refresh_function(self.items, self.header, self.visible_rows_indices, self.getting_data)
 
         self.items = pad_lists_to_same_length(self.items)
                     
@@ -566,7 +584,7 @@ class Picker:
         ## Ensure that items is a List[List[Str]] object
         if len(self.items) > 0 and not isinstance(self.items[0], list):
             self.items = [[item] for item in self.items]
-        self.items = [[str(cell) for cell in row] for row in self.items]
+        # self.items = [[str(cell) for cell in row] for row in self.items]
 
 
         # Ensure that header is of the same length as the rows
@@ -820,8 +838,9 @@ class Picker:
         # Determine widths based only on the currently indexed rows
         # rows = [v[1] for v in self.indexed_items] if len(self.indexed_items) else self.items
         # Determine widths based only on the currently displayed indexed rows
-        rows = [v[1] for v in self.indexed_items[start_index:end_index]] if len(self.indexed_items) else self.items
-        self.column_widths = get_column_widths(rows, header=self.header, max_column_width=self.max_column_width, number_columns=self.number_columns, max_total_width=self.rows_w, unicode_char_width=self.unicode_char_width)
+        # rows = [v[1] for v in self.indexed_items[start_index:end_index]] if len(self.indexed_items) else self.items
+        self.get_visible_rows()
+        self.column_widths = get_column_widths(self.visible_rows, header=self.header, max_column_width=self.max_column_width, number_columns=self.number_columns, max_total_width=self.rows_w, unicode_char_width=self.unicode_char_width)
         visible_column_widths = [c for i,c in enumerate(self.column_widths) if i not in self.hidden_columns]
         visible_columns_total_width = sum(visible_column_widths) + len(self.separator)*(len(visible_column_widths)-1)
 
@@ -1199,6 +1218,11 @@ class Picker:
             else:
                 self.stdscr.addstr(0,self.term_w-3,"  ", curses.color_pair(self.colours_start+23) | curses.A_BOLD)
 
+        # Display data fetch symbol
+        if not self.getting_data.is_set():
+            self.stdscr.addstr(0,self.term_w-3,"  ", curses.color_pair(self.colours_start+21) | curses.A_BOLD)
+            # self.stdscr.addstr(0,self.term_w-6,"⏳", curses.color_pair(self.colours_start+21) | curses.A_BOLD)
+
         ## Display footer
         if self.show_footer:
             # self.footer = NoFooter(self.stdscr, self.colours_start, self.get_function_data)
@@ -1592,6 +1616,7 @@ class Picker:
                 "split_right": False,
                 "cell_cursor": False,
                 "crosshair_cursor": False,
+                "show_header": False,
 
             }
             OptionPicker = Picker(submenu_win, **notification_data)
@@ -2123,7 +2148,7 @@ class Picker:
 
                 self.stdscr = tmp
 
-                self.notification(self.stdscr, f"{repr(file_to_load)} has been loaded!")
+                # self.notification(self.stdscr, f"{repr(file_to_load)} has been loaded!")
 
                 self.set_function_data({}, reset_absent_variables=True)
                 self.load_file(self.loaded_file)
@@ -2146,7 +2171,9 @@ class Picker:
     def fetch_data(self) -> None:
         """ Refesh data asynchronously. When data has been fetched self.data_ready is set to True. """
         self.logger.info(f"function: fetch_data()")
-        tmp_items, tmp_header = self.refresh_function()
+        tmp_items, tmp_header = [], []
+        self.getting_data.clear()
+        self.refresh_function(tmp_items, tmp_header, self.visible_rows_indices, self.getting_data)
         if self.track_entries_upon_refresh:
             selected_indices = get_selected_indices(self.selections)
             self.ids = [item[self.id_column] for i, item in enumerate(self.items) if i in selected_indices]
@@ -2424,6 +2451,10 @@ class Picker:
             else: return False
 
         COLS, LINES = os.get_terminal_size()
+
+
+        getting_data_prev = False
+
         # Main loop
         while True:
             # key = self.stdscr.getch()
@@ -2432,6 +2463,16 @@ class Picker:
             if key != -1:
                 self.logger.info(f"key={key}")
 
+            # Ensure that
+
+            if not self.getting_data.is_set():
+                self.initialise_variables()
+                getting_data_prev = True
+            elif getting_data_prev:
+                ## Ensure that we reinitialise one final time after all data is retrieved.
+                self.initialise_variables()
+                getting_data_prev = False
+
             self.term_resize_event = terminal_resized(COLS, LINES)
             COLS, LINES = os.get_terminal_size()
             if self.term_resize_event: 
@@ -2950,8 +2991,8 @@ class Picker:
 
 
                 ## Scroll with column select
-                rows = self.get_visible_rows()
-                self.column_widths = get_column_widths(rows, header=self.header, max_column_width=self.max_column_width, number_columns=self.number_columns, max_total_width=self.rows_w, unicode_char_width=self.unicode_char_width)
+                self.get_visible_rows()
+                self.column_widths = get_column_widths(self.visible_rows, header=self.header, max_column_width=self.max_column_width, number_columns=self.number_columns, max_total_width=self.rows_w, unicode_char_width=self.unicode_char_width)
                 visible_column_widths = [c for i,c in enumerate(self.column_widths) if i not in self.hidden_columns]
                 column_set_width = sum(visible_column_widths)+len(self.separator)*len(visible_column_widths)
                 start_of_cell = sum(visible_column_widths[:self.selected_column])+len(self.separator)*self.selected_column
@@ -2977,8 +3018,8 @@ class Picker:
                 #     curses.flash()
 
                 ## Scroll with column select
-                rows = self.get_visible_rows()
-                self.column_widths = get_column_widths(rows, header=self.header, max_column_width=self.max_column_width, number_columns=self.number_columns, max_total_width=self.rows_w, unicode_char_width=self.unicode_char_width)
+                self.get_visible_rows()
+                self.column_widths = get_column_widths(self.visible_rows, header=self.header, max_column_width=self.max_column_width, number_columns=self.number_columns, max_total_width=self.rows_w, unicode_char_width=self.unicode_char_width)
                 visible_column_widths = [c for i,c in enumerate(self.column_widths) if i not in self.hidden_columns]
                 column_set_width = sum(visible_column_widths)+len(self.separator)*len(visible_column_widths)
                 start_of_cell = sum(visible_column_widths[:self.selected_column])+len(self.separator)*self.selected_column
@@ -3708,7 +3749,7 @@ def parse_arguments() -> Tuple[argparse.Namespace, dict]:
     #     input_arg = args.filename
 
     elif args.generate:
-        function_data["refresh_function"] = lambda : generate_picker_data(args.generate)
+        function_data["refresh_function"] = lambda items, header, visible_rows_indices, getting_data: generate_picker_data_from_file(args.generate, items, header, visible_rows_indices, getting_data)
         function_data["get_data_startup"] = True
         function_data["get_new_data"] = True
         return args, function_data

{listpick-0.1.16.0 → listpick-0.1.16.3}/src/listpick/ui/footer.py

@@ -160,7 +160,7 @@ class StandardFooter(Footer):
         if state["footer_string"]:
             footer_string_width = min(w-1, len(state["footer_string"])+2)
 
-            disp_string = f"{state["footer_string"][:footer_string_width]}"
+            disp_string = f"{state['footer_string'][:footer_string_width]}"
             disp_string = f" {disp_string:>{footer_string_width-2}} "
             self.stdscr.addstr(self.footer_string_y, w-footer_string_width-1, " "*footer_string_width, curses.color_pair(self.colours_start+24))
             self.stdscr.addstr(self.footer_string_y, w-footer_string_width-1, f"{disp_string}", curses.color_pair(self.colours_start+24))
@@ -259,7 +259,7 @@ class CompactFooter(Footer):
         if state["footer_string"]:
             footer_string_width = min(w-1, len(state["footer_string"])+2)
 
-            disp_string = f"{state["footer_string"][:footer_string_width]}"
+            disp_string = f"{state['footer_string'][:footer_string_width]}"
             disp_string = f" {disp_string:>{footer_string_width-2}} "
             self.stdscr.addstr(h - 1, w-footer_string_width-1, " "*footer_string_width, curses.color_pair(self.colours_start+24))
             self.stdscr.addstr(h - 1, w-footer_string_width-1, f"{disp_string}", curses.color_pair(self.colours_start+24))
@@ -318,7 +318,7 @@ class NoFooter(Footer):
 
         if state["footer_string"]:
             footer_string_width = min(w-1, len(state["footer_string"])+2)
-            disp_string = f"{state["footer_string"][:footer_string_width]}"
+            disp_string = f"{state['footer_string'][:footer_string_width]}"
             disp_string = f" {disp_string:>{footer_string_width-2}} "
             self.stdscr.addstr(h - 1, w-footer_string_width-1, " "*footer_string_width, curses.color_pair(self.colours_start+24))
             self.stdscr.addstr(h - 1, w-footer_string_width-1, f"{disp_string}", curses.color_pair(self.colours_start+24))

{listpick-0.1.16.0 → listpick-0.1.16.3}/src/listpick/utils/generate_data.py

@@ -35,6 +35,23 @@ def generate_columns(funcs: list, files: list) -> list[list[str]]:
     return results
 
 
+def generate_columns_multithread(funcs: list, files: list) -> list[list[str]]:
+    """
+    Takes a list of functions and a list of files. 
+    Tasks are run in parallel using concurrent.futures.
+    """
+    logger.info("function: generate_columns (generate_data.py)")
+    
+    results = []
+    # Create a future object for each combination of func and file
+    with concurrent.futures.ThreadPoolExecutor() as executor:
+        futures = [[executor.submit(func, file) for func in funcs] for file in files]
+        
+        for file_futures in futures:
+            result = [future.result() for future in file_futures]
+            results.append(result)
+    return results
+
 def generate_columns_single_thread(funcs: list, files: list) -> list[list[str]]:
     """
     Takes a list of functions and a list of files. Each function is run for each file and a list of lists is returned.

listpick-0.1.16.3/src/listpick/utils/generate_data_multithreaded.py

@@ -0,0 +1,202 @@
+#!/bin/python
+# -*- coding: utf-8 -*-
+"""
+generate_data_multithreaded.py
+Generate data for listpick Picker.
+
+
+1. Read toml file.
+2. Set environment variables.
+3. Get files from first command.
+4. Create items with "..." for cells to be filled.
+5. Create a priority queue which determines which cells are to be filled first.
+6. Create a queue updater which increases the priorty of cells which are on screen which does so each second.
+7. Create threads to start generating data for cells.
+
+Author: GrimAndGreedy
+License: MIT
+"""
+
+import subprocess
+import os
+from typing import Tuple, Callable
+import toml
+import logging
+import threading
+from queue import PriorityQueue
+import time
+
+logger = logging.getLogger('picker_log')
+
+def generate_columns_worker(
+    funcs: list,
+    files: list,
+    items: list[list[str]],
+    getting_data: threading.Event,
+    task_queue: PriorityQueue,
+    completed_cells: set,
+) -> None:
+    """ Get a task from the priorty queue and fill the data for that cell."""
+    while task_queue.qsize() > 0:
+        _, (i, j) = task_queue.get()
+
+        if (i, j) in completed_cells:
+            task_queue.task_done()
+            continue
+            
+        generate_cell(
+            func=funcs[j],
+            file=files[i],
+            items=items,
+            row=i,
+            col=j+1,
+        )
+        completed_cells.add((i, j))
+        task_queue.task_done()
+    getting_data.set()
+
+def generate_cell(func: Callable, file: str, items: list[list[str]], row: int, col: int) -> None:
+    """
+    Takes a function, file and a file and then sets items[row][col] to the result.
+    """
+    
+    items[row][col] = func(file).strip()
+
+def update_queue(task_queue: PriorityQueue, visible_rows_indices: list[int], rows: int, cols: int):
+    """ Increase the priority of getting the data for the cells that are currently visible. """
+    while task_queue.qsize() > 0:
+        time.sleep(0.1)
+        for row in visible_rows_indices:
+            for col in range(cols):
+                if 0 <= row < rows:
+                    task_queue.put((1, (row, col)))
+
+        # Delay
+        time.sleep(0.9)
+
+def command_to_func(command: str) -> Callable:
+    """
+    Convert a command string to a function that will run the command.
+
+    E.g.,
+        mediainfo {} | grep -i format
+        mediainfo {} | grep -i format | head -n 1 | awk '{{print $3}}'
+    """
+    logger.info("function: command_to_func (generate_data.py)")
+    
+    func = lambda arg: subprocess.run(command.format(repr(arg)), shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.decode("utf-8").strip()
+    return func
+
+def load_environment(envs:dict):
+    """
+    Load environment variables from an envs dict.
+    """
+    logger.info("function: load_environment (generate_data.py)")
+
+    if "cwd" in envs:
+        os.chdir(os.path.expandvars(os.path.expanduser(envs["cwd"])))
+
+
+def read_toml(file_path) -> Tuple[dict, list, list]:
+    """
+    Read toml file and return the environment, commands and header sections.
+    """
+    logger.info("function: read_toml (generate_data.py)")
+    with open(file_path, 'r') as file:
+        config = toml.load(file)
+
+    environment = config['environment'] if 'environment' in config else {}
+    data = config['data'] if 'data' in config else {}
+    commands = [command.strip() for command in data['commands']] if 'commands' in data else []
+    header = [header for header in data['header']]  if 'header' in data else []
+    return environment, commands, header
+    
+    
+def generate_picker_data_from_file(
+    file_path: str,
+    items,
+    header,
+    visible_rows_indices,
+    getting_data
+) -> None:
+    """
+    Generate data for Picker based upon the toml file commands.
+    """
+    logger.info("function: generate_picker_data (generate_data.py)")
+
+
+    environment, lines, hdr = read_toml(file_path)
+
+    # Load any environment variables from the toml file
+    if environment:
+        load_environment(environment)
+
+
+    # Get list of files to be displayed in the first column.
+    get_files_command = lines[0]
+    files = subprocess.run(get_files_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.decode("utf-8").strip().split("\n")
+    files = [file.strip() for file in files if files]
+    
+    commands_list = [line.strip() for line in lines[1:]]
+    command_funcs = [command_to_func(command) for command in commands_list]
+
+    generate_picker_data(
+        files = files,
+        column_functions = command_funcs,
+        data_header = hdr,
+        items = items,
+        picker_header = header,
+        visible_rows_indices = visible_rows_indices,
+        getting_data = getting_data,
+    )
+
+def generate_picker_data(
+    files: list[str],
+    column_functions: list[Callable],
+    data_header,
+    items,
+    picker_header,
+    visible_rows_indices,
+    getting_data
+) -> None:
+    """
+    Generate data from a list of files and a list of column functions which will be used to 
+        generate subsequent columns.
+
+    This function is performed asynchronously with os.cpu_count() threads.
+
+    data_header: header list to be set for the picker
+    picker_header: the picker header will be passed in so that it can be set for the class
+
+    """
+    logger.info("function: generate_picker_data (generate_data.py)")
+
+    items.clear()
+    items.extend([[file] + ["..." for _ in column_functions] for file in files])
+    # items[:] = [[file] + ["..." for _ in column_functions] for file in files]
+    picker_header[:] = data_header
+
+
+    task_queue = PriorityQueue()
+    for i in range(len(files)):
+        for j in range(len(column_functions)):
+            task_queue.put((10, (i, j)))
+
+    num_workers = os.cpu_count()
+    if num_workers in [None, -1]: num_workers = 4
+    completed_cells = set()
+
+    for _ in range(num_workers):
+        gen_items_thread = threading.Thread(
+            target=generate_columns_worker,
+            args=(column_functions, files, items, getting_data, task_queue, completed_cells),
+        )
+        gen_items_thread.daemon = True
+        gen_items_thread.start()
+
+    update_queue_thread = threading.Thread(
+        target=update_queue,
+        args=(task_queue, visible_rows_indices, len(files), len(column_functions)),
+    )
+    update_queue_thread.daemon = True
+    update_queue_thread.start()

{listpick-0.1.16.0 → listpick-0.1.16.3/src/listpick.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: listpick
-Version: 0.1.16.0
+Version: 0.1.16.3
 Summary: Listpick is a powerful TUI data tool for creating TUI apps or viewing/comparing tabulated data.
 Home-page: https://github.com/grimandgreedy/listpick
 Author: Grim
@@ -45,7 +45,7 @@ Dynamic: summary
 
 listpick is a TUI tool which displays a tabulated list of rows and allows the user to operate upon these rows--select, copy, pipe. A very simple concept but also, I hope, a powerful tool that will make it easier for people to develop TUI apps.
 
-Rows of data can be viewed, selected, generated, saved, loaded, refreshed, modified or copied to the clipboard. Easy to integrate into your project by creating a `menu = Picker(stdscr, items:list[list[str]])` and then the menu will be displayed by running `menu.run()`.
+Rows of data can be viewed, selected, generated, saved, loaded, refreshed, modified or copied to the clipboard. Easy to integrate into your project by creating a `menu = Picker(stdscr: curses.window, items: list[list[str]])` and then the menu will be displayed by running `menu.run()`.
 
 It works great as the backend for a TUI application and can also be used as a standalone data viewer.
 
@@ -53,13 +53,13 @@ It works great as the backend for a TUI application and can also be used as a st
 
 # Quickstart
 
-Install listpick
+Install listpick:
 
 ```
 python -m pip installl "listpick[full]"
 ```
 
-Create a picker object to display rows:
+Create a Picker:
 
 ```
 from listpick.listpick_app import Picker, start_curses, close_curses
@@ -79,7 +79,7 @@ x.run()
 close_curses(stdscr)
 
 ```
-Use the listpick binary to generate and display rows based on a list of commands:
+Or use the listpick binary to generate and display rows based on a list of commands:
 
 ```
 wget https://raw.githubusercontent.com/grimandgreedy/listpick/refs/heads/master/examples/data_generation/list_files.toml

{listpick-0.1.16.0 → listpick-0.1.16.3}/src/listpick.egg-info/SOURCES.txt

@@ -46,6 +46,7 @@ src/listpick/utils/config.py
 src/listpick/utils/dump.py
 src/listpick/utils/filtering.py
 src/listpick/utils/generate_data.py
+src/listpick/utils/generate_data_multithreaded.py
 src/listpick/utils/graphing.py
 src/listpick/utils/keycodes.py
 src/listpick/utils/options_selectors.py