nettracer3d 0.5.3__tar.gz → 0.5.4__tar.gz

{nettracer3d-0.5.3/src/nettracer3d.egg-info → nettracer3d-0.5.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: nettracer3d
-Version: 0.5.3
+Version: 0.5.4
 Summary: Scripts for intializing and analyzing networks from segmentations of three dimensional images.
 Author-email: Liam McLaughlin <mclaughlinliam99@gmail.com>
 Project-URL: User_Tutorial, https://www.youtube.com/watch?v=cRatn5VTWDY
@@ -44,9 +44,10 @@ NetTracer3D is free to use/fork for academic/nonprofit use so long as citation i
 
 NetTracer3D was developed by Liam McLaughlin while working under Dr. Sanjay Jain at Washington University School of Medicine.
 
--- Version 0.5.3 updates --
+-- Version 0.5.4 updates --
 
-1. Improved calculate volumes method. Previous method used np.argwhere() to count voxels of labeled objects in parallel which was quite strenuous in large arrays with many objects. New method uses np.bincount() which uses optimized numpy C libraries to do the same.
-2. scipy.ndimage.find_objects() method was replaced as the method to find bounding boxes for objects when searching for object neighborhoods for the morphological proximity network and the edge < > node interaction quantification. This new version should be substantially faster in big arrays with many labels. (Depending on how well this improves performance, I may reimplement the secondary network search algorithm, as a side-option, which uses the same parallel-search within subarray strategies, as opposed to the primary network search algorithm that uses distance transforms).
-3. Image viewer window can now load in .nii format images, as well as .jpeg, .jpg, and .png. The nibabel library was added to the dependencies to enable .nii loading, although this is currently all it is used for (and the gui will still run without nibabel).
-4. Fixed bug regarding deleting edge objects. 
+1. Added new function to GUI in image -> overlays -> color nodes/edges. Generates a rgb array corresponding to the nodes/edge labels where each node/edge (depending which array is selected) is randomly assigned a unique rgb color in an overlay channel. This can be used, for example, to color code labeled branches for easy identification of which branch is which.
+
+2. Improved highlight overlay general functionality (for selecting nodes/edges). Previously selecting a node/edge had the program attempting to create an equal sized array as an overlay, find all objects corresponding to the selected ones, fill those into the new highlight overlay, then overlay that image. This was understandably quite slow in big arrays where the system was wasting a lot of time searching the entire array every time something was selected. New version retains this functionality for arrays below 125 million voxels, since search time is rather manageable at that size. For larger arrays, it instead draws the highlight for the selected objects only into the current slice, rendering a new slice whenever the user scrolls in the stack (although the entire highlight overlay is still initialized as a placeholder). Functions that require the use of the entire highlight overlay (such as masking) are correspondingly updated to draw the entirety of the highlight overlay before executing (when the system has up until that point been drawing slices one at a time). This will likely be the retained behavior moving forward, although to eliminate this behavior, one can open nettracer_gui.py and set self.mini_thresh to some comically large value. The new highlight overlay seems to work effectively the same but faster in my testing although it is possible a bug slipped through, which I will fix if informed about (or if I find it myself).
+
+3. For the machine learning segmenter, changed the system to attempt to segment the image by chunking the array into the largest possible chunks that can be divided across all CPU cores. Previously the system split the array into 64^3 voxel sized chunks and passed those to the CPU cores until everything was processed. I am not sure which version is more efficient/faster so this is somewhat of a test. In theory the new behavior could be faster because it asking Python to interpret less stuff. 

nettracer3d-0.5.4/README.md

@@ -0,0 +1,17 @@
+NetTracer3D is a python package developed for both 2D and 3D analysis of microscopic images in the .tif file format. It supports generation of 3D networks showing the relationships between objects (or nodes) in three dimensional space, either based on their own proximity or connectivity via connecting objects such as nerves or blood vessels. In addition to these functionalities are several advanced 3D data processing algorithms, such as labeling of branched structures or abstraction of branched structures into networks. Note that nettracer3d uses segmented data, which can be segmented from other softwares such as ImageJ and imported into NetTracer3D, although it does offer its own segmentation via intensity and volumetric thresholding, or random forest machine learning segmentation. NetTracer3D currently has a fully functional GUI. To use the GUI, after installing the nettracer3d package via pip, enter the command 'nettracer3d' in your command prompt:
+
+
+This gui is built from the PyQt6 package and therefore may not function on dockers or virtual envs that are unable to support PyQt6 displays. More advanced documentation is coming down the line, but for now please see: https://www.youtube.com/watch?v=cRatn5VTWDY
+for a video tutorial on using the GUI.
+
+NetTracer3D is free to use/fork for academic/nonprofit use so long as citation is provided, and is available for commercial use at a fee (see license file for information).
+
+NetTracer3D was developed by Liam McLaughlin while working under Dr. Sanjay Jain at Washington University School of Medicine.
+
+-- Version 0.5.4 updates --
+
+1. Added new function to GUI in image -> overlays -> color nodes/edges. Generates a rgb array corresponding to the nodes/edge labels where each node/edge (depending which array is selected) is randomly assigned a unique rgb color in an overlay channel. This can be used, for example, to color code labeled branches for easy identification of which branch is which.
+
+2. Improved highlight overlay general functionality (for selecting nodes/edges). Previously selecting a node/edge had the program attempting to create an equal sized array as an overlay, find all objects corresponding to the selected ones, fill those into the new highlight overlay, then overlay that image. This was understandably quite slow in big arrays where the system was wasting a lot of time searching the entire array every time something was selected. New version retains this functionality for arrays below 125 million voxels, since search time is rather manageable at that size. For larger arrays, it instead draws the highlight for the selected objects only into the current slice, rendering a new slice whenever the user scrolls in the stack (although the entire highlight overlay is still initialized as a placeholder). Functions that require the use of the entire highlight overlay (such as masking) are correspondingly updated to draw the entirety of the highlight overlay before executing (when the system has up until that point been drawing slices one at a time). This will likely be the retained behavior moving forward, although to eliminate this behavior, one can open nettracer_gui.py and set self.mini_thresh to some comically large value. The new highlight overlay seems to work effectively the same but faster in my testing although it is possible a bug slipped through, which I will fix if informed about (or if I find it myself).
+
+3. For the machine learning segmenter, changed the system to attempt to segment the image by chunking the array into the largest possible chunks that can be divided across all CPU cores. Previously the system split the array into 64^3 voxel sized chunks and passed those to the CPU cores until everything was processed. I am not sure which version is more efficient/faster so this is somewhat of a test. In theory the new behavior could be faster because it asking Python to interpret less stuff. 

{nettracer3d-0.5.3 → nettracer3d-0.5.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "nettracer3d"
-version = "0.5.3"
+version = "0.5.4"
 authors = [
   { name="Liam McLaughlin", email="mclaughlinliam99@gmail.com" },
 ]

{nettracer3d-0.5.3 → nettracer3d-0.5.4}/src/nettracer3d/community_extractor.py

@@ -9,6 +9,7 @@ from scipy import ndimage
 from scipy.ndimage import zoom
 from networkx.algorithms import community
 from community import community_louvain
+import random
 from . import node_draw
 
 
@@ -781,6 +782,47 @@ def generate_distinct_colors(n_colors: int) -> List[Tuple[int, int, int]]:
         colors.append(rgb)
     return colors
 
+def assign_node_colors(node_list: List[int], labeled_array: np.ndarray) -> Tuple[np.ndarray, Dict[int, str]]:
+    """
+    Assign distinct colors to nodes and create an RGBA image.
+    
+    Args:
+        node_list: List of node IDs
+        labeled_array: 3D numpy array with labels corresponding to node IDs
+    
+    Returns:
+        Tuple of (RGBA-coded numpy array (H, W, D, 4), dictionary mapping nodes to color names)
+    """
+    
+    # Sort communities by size (descending)
+    sorted_nodes= sorted(node_list, reverse=True)
+    
+    # Generate distinct colors
+    colors = generate_distinct_colors(len(node_list))
+    random.shuffle(colors) #Randomly sorted to make adjacent structures likely stand out
+    
+    # Convert RGB colors to RGBA by adding alpha channel
+    colors_rgba = [(r, g, b, 255) for r, g, b in colors]  # Full opacity for colored regions
+    
+    # Create mapping from community to color
+    node_to_color = {node: colors_rgba[i] for i, node in enumerate(sorted_nodes)}
+    
+    # Create RGBA array (initialize with transparent background)
+    rgba_array = np.zeros((*labeled_array.shape, 4), dtype=np.uint8)
+    
+    # Assign colors to each voxel based on its label
+    for label in np.unique(labeled_array):
+        if label in node_to_color:  # Skip background (usually label 0)
+            mask = labeled_array == label
+            for i in range(4):  # RGBA channels
+                rgba_array[mask, i] = node_to_color[label][i]
+
+    # Convert the RGB portion of community_to_color back to RGB for color naming
+    node_to_color_rgb = {k: tuple(v[:3]) for k, v in node_to_color.items()}
+    node_to_color_names = convert_node_colors_to_names(node_to_color_rgb)
+    
+    return rgba_array, node_to_color_names
+
 def assign_community_colors(community_dict: Dict[int, int], labeled_array: np.ndarray) -> Tuple[np.ndarray, Dict[int, str]]:
     """
     Assign distinct colors to communities and create an RGBA image.

{nettracer3d-0.5.3 → nettracer3d-0.5.4}/src/nettracer3d/nettracer.py

@@ -4091,6 +4091,29 @@ class Network_3D:
 
 
         return image, output
+
+    def node_to_color(self, down_factor = None, mode = 0):
+
+        if mode == 0:
+            array = self._nodes
+        elif mode == 1:
+            array = self._edges
+
+        items = list(np.unique(array))
+        if 0 in items:
+            del items[0]
+
+
+        if down_factor is not None:
+            original_shape = array.shape
+            array = downsample(array, down_factor)
+
+        array, output = community_extractor.assign_node_colors(items, array)
+
+        if down_factor is not None:
+            array = upsample_with_padding(array, down_factor, original_shape)
+
+        return array, output
         
 
 

{nettracer3d-0.5.3 → nettracer3d-0.5.4}/src/nettracer3d/nettracer_gui.py

@@ -397,6 +397,8 @@ class ImageViewerWindow(QMainWindow):
         # Initialize highlight overlay
         self.highlight_overlay = None
         self.highlight_bounds = None  # Store bounds for positioning
+        self.mini_overlay = False # If the program is currently drawing the overlay by frame this will be true
+        self.mini_thresh = (500*500*500) # Array volume to start using mini overlays for
 
     def start_left_scroll(self):
         """Start scrolling left when left arrow is pressed."""
@@ -443,6 +445,8 @@ class ImageViewerWindow(QMainWindow):
             edge_indices (list): List of edge indices to highlight
         """
 
+        self.mini_overlay = False #If this method is ever being called, it means we are rendering the entire overlay so mini overlay needs to reset.
+
         def process_chunk(chunk_data, indices_to_check):
             """Process a single chunk of the array to create highlight mask"""
             mask = np.isin(chunk_data, indices_to_check)
@@ -463,6 +467,9 @@ class ImageViewerWindow(QMainWindow):
         if overlay1_indices is not None:
             if 0 in overlay1_indices:
                 overlay1_indices.remove(0)
+        if overlay2_indices is not None:
+            if 0 in overlay2_indices:
+                overlay2_indices.remove(0)
 
         if node_indices is None:
             node_indices = []
@@ -628,6 +635,110 @@ class ImageViewerWindow(QMainWindow):
         # Update display
         self.update_display(preserve_zoom=(current_xlim, current_ylim), called = True)
 
+    def create_mini_overlay(self, node_indices = None, edge_indices = None):
+
+        """
+        Create a binary overlay highlighting specific nodes and/or edges using parallel processing, one slice at a time for efficiency.
+        
+        Args:
+            node_indices (list): List of node indices to highlight
+            edge_indices (list): List of edge indices to highlight
+        """
+
+        def process_chunk(chunk_data, indices_to_check):
+            """Process a single chunk of the array to create highlight mask"""
+            mask = np.isin(chunk_data, indices_to_check)
+            return mask * 255
+
+
+        if node_indices is not None:
+            if 0 in node_indices:
+                node_indices.remove(0)
+        if edge_indices is not None:
+            if 0 in edge_indices:
+                edge_indices.remove(0)
+
+
+        if node_indices is None:
+            node_indices = []
+        if edge_indices is None:
+            edge_indices = []
+
+            
+        current_xlim = self.ax.get_xlim() if hasattr(self, 'ax') and self.ax.get_xlim() != (0, 1) else None
+        current_ylim = self.ax.get_ylim() if hasattr(self, 'ax') and self.ax.get_ylim() != (0, 1) else None
+        
+        if not node_indices and not edge_indices:
+            self.highlight_overlay = None
+            self.update_display(preserve_zoom=(current_xlim, current_ylim))
+            return
+            
+        # Get the shape of the full array from any existing channel
+        for channel in self.channel_data:
+            if channel is not None:
+                full_shape = channel.shape
+                break
+        else:
+            return  # No valid channels to get shape from
+            
+        # Initialize full-size overlay
+        if self.highlight_overlay is None:
+            self.highlight_overlay = np.zeros(full_shape, dtype=np.uint8)
+        
+        # Get number of CPU cores
+        num_cores = mp.cpu_count()
+        
+        # Calculate chunk size along y-axis
+        chunk_size = full_shape[1] // num_cores
+        if chunk_size < 1:
+            chunk_size = 1
+        
+        def process_channel(channel_data, indices, array_shape):
+            if channel_data is None or not indices:
+                return None
+                
+            # Create chunks
+            chunks = []
+            for i in range(0, array_shape[1], chunk_size):
+                end = min(i + chunk_size, array_shape[1])
+                chunks.append(channel_data[i:end, :])
+                
+            # Process chunks in parallel using ThreadPoolExecutor
+            process_func = partial(process_chunk, indices_to_check=indices)
+
+            
+            with ThreadPoolExecutor(max_workers=num_cores) as executor:
+                chunk_results = list(executor.map(process_func, chunks))
+                
+            # Reassemble the chunks
+            return np.concatenate(chunk_results, axis=0)
+        
+        # Process nodes and edges in parallel using multiprocessing
+        with ThreadPoolExecutor(max_workers=num_cores) as executor:
+            try:
+                slice_node = self.channel_data[0][self.current_slice, :, :] #This is the only major difference to the big highlight... we are only looking at this
+                future_nodes = executor.submit(process_channel, slice_node, node_indices, full_shape)
+                node_overlay = future_nodes.result()
+            except:
+                node_overlay = None
+            try:
+                slice_edge = self.channel_data[1][self.current_slice, :, :]
+                future_edges = executor.submit(process_channel, slice_edge, edge_indices, full_shape)
+                edge_overlay = future_edges.result()
+            except:
+                edge_overlay = None
+                        
+        # Combine results
+        self.highlight_overlay[self.current_slice, :, :] = np.zeros_like(self.highlight_overlay[self.current_slice, :, :])
+        if node_overlay is not None:
+            self.highlight_overlay[self.current_slice, :, :] = np.maximum(self.highlight_overlay[self.current_slice, :, :], node_overlay)
+        if edge_overlay is not None:
+            self.highlight_overlay[self.current_slice, :, :] = np.maximum(self.highlight_overlay[self.current_slice, :, :], edge_overlay)
+
+                
+        # Update display
+        self.update_display(preserve_zoom=(current_xlim, current_ylim))
+
 
 
         
@@ -890,14 +1001,23 @@ class ImageViewerWindow(QMainWindow):
                     edge_indices = filtered_df.iloc[:, 2].unique().tolist()
                     self.clicked_values['edges'] = edge_indices
 
-                    self.create_highlight_overlay(
-                        node_indices=self.clicked_values['nodes'], 
-                        edge_indices=self.clicked_values['edges']
-                    )
+                    if self.channel_data[1].shape[0] * self.channel_data[1].shape[1] * self.channel_data[1].shape[2] > self.mini_thresh:
+                        self.mini_overlay = True
+                        self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                    else:
+                        self.create_highlight_overlay(
+                            node_indices=self.clicked_values['nodes'], 
+                            edge_indices=self.clicked_values['edges']
+                        )
                 else:
-                    self.create_highlight_overlay(
-                        node_indices=self.clicked_values['nodes']
-                )
+                    if self.channel_data[0].shape[0] * self.channel_data[0].shape[1] * self.channel_data[0].shape[2] > self.mini_thresh:
+                        self.mini_overlay = True
+                        self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                    else:
+                        self.create_highlight_overlay(
+                            node_indices=self.clicked_values['nodes'],
+                            edge_indices = self.clicked_values['edges']
+                    )
             
                 
         except Exception as e:
@@ -972,14 +1092,23 @@ class ImageViewerWindow(QMainWindow):
             if edges:
                 edge_indices = filtered_df.iloc[:, 2].unique().tolist()
                 self.clicked_values['edges'] = edge_indices
-                self.create_highlight_overlay(
-                    node_indices=self.clicked_values['nodes'],
-                    edge_indices=edge_indices
-                )
+                if self.channel_data[1].shape[0] * self.channel_data[1].shape[1] * self.channel_data[1].shape[2] > self.mini_thresh:
+                    self.mini_overlay = True
+                    self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                else:
+                    self.create_highlight_overlay(
+                        node_indices=self.clicked_values['nodes'],
+                        edge_indices=edge_indices
+                    )
             else:
-                self.create_highlight_overlay(
-                    node_indices = self.clicked_values['nodes']
-            )
+                if self.channel_data[0].shape[0] * self.channel_data[0].shape[1] * self.channel_data[0].shape[2] > self.mini_thresh:
+                    self.mini_overlay = True
+                    self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                else:
+                    self.create_highlight_overlay(
+                        node_indices = self.clicked_values['nodes'],
+                        edge_indices = self.clicked_values['edges']
+                )
 
         except Exception as e:
 
@@ -1043,15 +1172,24 @@ class ImageViewerWindow(QMainWindow):
                 if edges:
                     edge_indices = filtered_df.iloc[:, 2].unique().tolist()
                     self.clicked_values['edges'] = edge_indices
-                    self.create_highlight_overlay(
-                        node_indices=nodes,
-                        edge_indices=edge_indices
-                    )
+                    if self.channel_data[1].shape[0] * self.channel_data[1].shape[1] * self.channel_data[1].shape[2] > self.mini_thresh:
+                        self.mini_overlay = True
+                        self.create_mini_overlay(node_indices = nodes, edge_indices = edge_indices)
+                    else:
+                        self.create_highlight_overlay(
+                            node_indices=nodes,
+                            edge_indices=edge_indices
+                        )
                     self.clicked_values['nodes'] = nodes
                 else:
-                    self.create_highlight_overlay(
-                        node_indices = nodes
-                )
+                    if self.channel_data[0].shape[0] * self.channel_data[0].shape[1] * self.channel_data[0].shape[2] > self.mini_thresh:
+                        self.mini_overlay = True
+                        self.create_mini_overlay(node_indices = nodes, edge_indices = self.clicked_values['edges'])
+                    else:
+                        self.create_highlight_overlay(
+                            node_indices = nodes,
+                            edge_indices = self.clicked_values['edges']
+                    )
                     self.clicked_values['nodes'] = nodes
 
         except Exception as e:
@@ -1095,9 +1233,15 @@ class ImageViewerWindow(QMainWindow):
 
             print(f"Found {len(filtered_df)} direct connections between nodes of ID {sort} and their neighbors (of any ID)")
 
-            self.create_highlight_overlay(
-                node_indices= nodes
-                )
+            if self.channel_data[0].shape[0] * self.channel_data[0].shape[1] * self.channel_data[0].shape[2] > self.mini_thresh:
+                self.mini_overlay = True
+                self.create_mini_overlay(node_indices = nodes, edge_indices = self.clicked_values['edges'])
+            else:
+                self.create_highlight_overlay(
+                    node_indices = nodes,
+                    edge_indices = self.clicked_values['edges']
+                    )
+            self.clicked_values['nodes'] = nodes
 
         except Exception as e:
             print(f"Error showing identities: {e}")
@@ -1393,8 +1537,6 @@ class ImageViewerWindow(QMainWindow):
 
             pairs = list(combinations(nodes, 2))
 
-            print(pairs)
-
 
             for i in range(len(my_network.network_lists[0]) - 1, -1, -1):
                 print((my_network.network_lists[0][i], my_network.network_lists[1][i]))
@@ -1891,7 +2033,11 @@ class ImageViewerWindow(QMainWindow):
                         self.clicked_values['nodes'].extend(selected_values)
                         # Remove duplicates while preserving order
                         self.clicked_values['nodes'] = list(dict.fromkeys(self.clicked_values['nodes']))
-                        self.create_highlight_overlay(node_indices=self.clicked_values['nodes'])
+                        if self.channel_data[0].shape[0] * self.channel_data[0].shape[1] * self.channel_data[0].shape[2] > self.mini_thresh:
+                            self.mini_overlay = True
+                            self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                        else:
+                            self.create_highlight_overlay(node_indices=self.clicked_values['nodes'])
                         
                         # Try to highlight the last selected value in tables
                         if self.clicked_values['nodes']:
@@ -1903,7 +2049,11 @@ class ImageViewerWindow(QMainWindow):
                         self.clicked_values['edges'].extend(selected_values)
                         # Remove duplicates while preserving order
                         self.clicked_values['edges'] = list(dict.fromkeys(self.clicked_values['edges']))
-                        self.create_highlight_overlay(edge_indices=self.clicked_values['edges'])
+                        if self.channel_data[1].shape[0] * self.channel_data[1].shape[1] * self.channel_data[1].shape[2] > self.mini_thresh:
+                            self.mini_overlay = True
+                            self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                        else:
+                            self.create_highlight_overlay(edge_indices=self.clicked_values['edges'])
                         
                         # Try to highlight the last selected value in tables
                         if self.clicked_values['edges']:
@@ -2144,9 +2294,17 @@ class ImageViewerWindow(QMainWindow):
 
                     # Highlight the clicked element in the image using the stored lists                
                     if self.active_channel == 0 and (starting_vals['nodes']) != (self.clicked_values['nodes']):
-                        self.create_highlight_overlay(node_indices=self.clicked_values['nodes'], edge_indices=self.clicked_values['edges'])
+                        if self.channel_data[0].shape[0] * self.channel_data[0].shape[1] * self.channel_data[0].shape[2] > self.mini_thresh:
+                            self.mini_overlay = True
+                            self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                        else:
+                            self.create_highlight_overlay(node_indices=self.clicked_values['nodes'], edge_indices=self.clicked_values['edges'])
                     elif self.active_channel == 1 and starting_vals['edges'] != self.clicked_values['edges']:
-                        self.create_highlight_overlay(node_indices=self.clicked_values['nodes'], edge_indices=self.clicked_values['edges'])
+                        if self.channel_data[1].shape[0] * self.channel_data[1].shape[1] * self.channel_data[1].shape[2] > self.mini_thresh:
+                            self.mini_overlay = True
+                            self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
+                        else:
+                            self.create_highlight_overlay(node_indices=self.clicked_values['nodes'], edge_indices=self.clicked_values['edges'])
 
                                 
                 except IndexError:
@@ -2298,6 +2456,8 @@ class ImageViewerWindow(QMainWindow):
         netoverlay_action.triggered.connect(self.show_netoverlay_dialog)
         idoverlay_action = overlay_menu.addAction("Create ID Overlay")
         idoverlay_action.triggered.connect(self.show_idoverlay_dialog)
+        coloroverlay_action = overlay_menu.addAction("Color Nodes (or Edges)")
+        coloroverlay_action.triggered.connect(self.show_coloroverlay_dialog)
         searchoverlay_action = overlay_menu.addAction("Show Search Regions")
         searchoverlay_action.triggered.connect(self.show_search_dialog)
         shuffle_action = overlay_menu.addAction("Shuffle")
@@ -2578,6 +2738,11 @@ class ImageViewerWindow(QMainWindow):
         dialog = IdOverlayDialog(self)
         dialog.exec()
 
+    def show_coloroverlay_dialog(self):
+        """show the color overlay dialog"""
+        dialog = ColorOverlayDialog(self)
+        dialog.exec()
+
     def show_search_dialog(self):
         """Show the search dialog"""
         dialog = SearchOverlayDialog(self)
@@ -3001,14 +3166,20 @@ class ImageViewerWindow(QMainWindow):
             else:
                 self.channel_data[channel_index] = channel_data
 
-            if len(self.channel_data[channel_index].shape) == 3:  # potentially 2D RGB
-                if self.channel_data[channel_index].shape[-1] in (3, 4):  # last dim is 3 or 4
-                    if self.confirm_rgb_dialog():
-                        # User confirmed it's 2D RGB, expand to 4D
-                        self.channel_data[channel_index] = np.expand_dims(self.channel_data[channel_index], axis=0)
-                        
-            if len(self.channel_data[channel_index].shape) == 4 and (channel_index == 0 or channel_index == 1):
-                self.channel_data[channel_index] = self.reduce_rgb_dimension(self.channel_data[channel_index])
+            try:
+                if len(self.channel_data[channel_index].shape) == 3:  # potentially 2D RGB
+                    if self.channel_data[channel_index].shape[-1] in (3, 4):  # last dim is 3 or 4
+                        if self.confirm_rgb_dialog():
+                            # User confirmed it's 2D RGB, expand to 4D
+                            self.channel_data[channel_index] = np.expand_dims(self.channel_data[channel_index], axis=0)
+            except:
+                pass
+            
+            try:
+                if len(self.channel_data[channel_index].shape) == 4 and (channel_index == 0 or channel_index == 1):
+                    self.channel_data[channel_index] = self.reduce_rgb_dimension(self.channel_data[channel_index])
+            except:
+                pass
 
             reset_resize = False
 
@@ -3050,27 +3221,30 @@ class ImageViewerWindow(QMainWindow):
                 self.active_channel_combo.setEnabled(True)
             
             # Update slider range if this is the first channel loaded
-            if len(self.channel_data[channel_index].shape) == 3 or len(self.channel_data[channel_index].shape) == 4:
-                if not self.slice_slider.isEnabled():
-                    self.slice_slider.setEnabled(True)
-                    self.slice_slider.setMinimum(0)
-                    self.slice_slider.setMaximum(self.channel_data[channel_index].shape[0] - 1)
-                    if self.slice_slider.value() < self.channel_data[channel_index].shape[0] - 1:
-                        self.current_slice = self.slice_slider.value()
+            try:
+                if len(self.channel_data[channel_index].shape) == 3 or len(self.channel_data[channel_index].shape) == 4:
+                    if not self.slice_slider.isEnabled():
+                        self.slice_slider.setEnabled(True)
+                        self.slice_slider.setMinimum(0)
+                        self.slice_slider.setMaximum(self.channel_data[channel_index].shape[0] - 1)
+                        if self.slice_slider.value() < self.channel_data[channel_index].shape[0] - 1:
+                            self.current_slice = self.slice_slider.value()
+                        else:
+                            self.slice_slider.setValue(0)
+                            self.current_slice = 0
                     else:
-                        self.slice_slider.setValue(0)
-                        self.current_slice = 0
+                        self.slice_slider.setEnabled(True)
+                        self.slice_slider.setMinimum(0)
+                        self.slice_slider.setMaximum(self.channel_data[channel_index].shape[0] - 1)
+                        if self.slice_slider.value() < self.channel_data[channel_index].shape[0] - 1:
+                            self.current_slice = self.slice_slider.value()
+                        else:
+                            self.current_slice = 0
+                            self.slice_slider.setValue(0)
                 else:
-                    self.slice_slider.setEnabled(True)
-                    self.slice_slider.setMinimum(0)
-                    self.slice_slider.setMaximum(self.channel_data[channel_index].shape[0] - 1)
-                    if self.slice_slider.value() < self.channel_data[channel_index].shape[0] - 1:
-                        self.current_slice = self.slice_slider.value()
-                    else:
-                        self.current_slice = 0
-                        self.slice_slider.setValue(0)
-            else:
-                self.slice_slider.setEnabled(False)
+                    self.slice_slider.setEnabled(False)
+            except:
+                pass
 
             
             # If this is the first channel loaded, make it active
@@ -3083,13 +3257,16 @@ class ImageViewerWindow(QMainWindow):
             self.min_max[channel_index][1] = np.max(self.channel_data[channel_index])
             self.volume_dict[channel_index] = None #reset volumes
 
-            if assign_shape: #keep original shape tracked to undo resampling.
-                if self.original_shape is None:
-                    self.original_shape = self.channel_data[channel_index].shape
-                elif self.original_shape[0] < self.channel_data[channel_index].shape[0] or self.original_shape[1] < self.channel_data[channel_index].shape[1] or self.original_shape[2] < self.channel_data[channel_index].shape[2]:
-                    self.original_shape = self.channel_data[channel_index].shape
-                if len(self.original_shape) == 4:
-                    self.original_shape = (self.original_shape[0], self.original_shape[1], self.original_shape[2])
+            try:
+                if assign_shape: #keep original shape tracked to undo resampling.
+                    if self.original_shape is None:
+                        self.original_shape = self.channel_data[channel_index].shape
+                    elif self.original_shape[0] < self.channel_data[channel_index].shape[0] or self.original_shape[1] < self.channel_data[channel_index].shape[1] or self.original_shape[2] < self.channel_data[channel_index].shape[2]:
+                        self.original_shape = self.channel_data[channel_index].shape
+                    if len(self.original_shape) == 4:
+                        self.original_shape = (self.original_shape[0], self.original_shape[1], self.original_shape[2])
+            except:
+                pass
 
             self.update_display(reset_resize = reset_resize)
 
@@ -3262,6 +3439,8 @@ class ImageViewerWindow(QMainWindow):
                 elif ch_index == 3:
                     my_network.save_id_overlay(filename=filename)
                 elif ch_index == 4:
+                    if self.mini_overlay == True:
+                        self.create_highlight_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
                     if filename == None:
                         filename = "Highlighted_Element.tif"
                     tifffile.imwrite(f"{filename}", self.highlight_overlay)
@@ -3306,6 +3485,8 @@ class ImageViewerWindow(QMainWindow):
         if self.pending_slice is not None:
             slice_value, view_settings = self.pending_slice
             self.current_slice = slice_value
+            if self.mini_overlay == True: #If we are rendering the highlight overlay for selected values one at a time.
+                self.create_mini_overlay(node_indices = self.clicked_values['nodes'], edge_indices = self.clicked_values['edges'])
             self.update_display(preserve_zoom=view_settings)
             self.pending_slice = None
 
@@ -3968,7 +4149,13 @@ class CustomTableView(QTableView):
                         # Navigate to the Z-slice
                         self.parent.slice_slider.setValue(int(centroid[0]))
                         print(f"Found node {value} at Z-slice {centroid[0]}")
-                        self.parent.create_highlight_overlay(node_indices=[value])
+                        if self.parent.channel_data[0].shape[0] * self.parent.channel_data[0].shape[1] * self.parent.channel_data[0].shape[2] > self.parent.mini_thresh:
+                            self.parent.mini_overlay = True
+                            self.parent.create_mini_overlay(node_indices = [value])
+                        else:
+                            self.parent.create_highlight_overlay(node_indices=[value])
+                        self.parent.clicked_values['nodes'] = []
+                        self.parent.clicked_values['edges'] = []
                         self.parent.clicked_values['nodes'].append(value)
                         
                         # Highlight the value in both tables if it exists
@@ -3992,7 +4179,13 @@ class CustomTableView(QTableView):
                         # Navigate to the Z-slice
                         self.parent.slice_slider.setValue(int(centroid[0]))
                         print(f"Found edge {value} at Z-slice {centroid[0]}")
-                        self.parent.create_highlight_overlay(edge_indices=[value])
+                        if self.parent.channel_data[1].shape[0] * self.parent.channel_data[1].shape[1] * self.parent.channel_data[1].shape[2] > self.parent.mini_thresh:
+                            self.parent.mini_overlay = True
+                            self.parent.create_mini_overlay(edge_indices = [value])
+                        else:
+                            self.parent.create_highlight_overlay(edge_indices=[value])
+                        self.parent.clicked_values['nodes'] = []
+                        self.parent.clicked_values['edges'] = []
                         self.parent.clicked_values['edges'].append(value)
 
                         # Highlight the value in both tables if it exists
@@ -4020,12 +4213,24 @@ class CustomTableView(QTableView):
                 self.parent.slice_slider.setValue(int(centroid1[0]))
                 print(f"Found node pair {value[0]} and {value[1]} at Z-slices {centroid1[0]} and {centroid2[0]}, respectively")
                 try:
-                    self.parent.create_highlight_overlay(node_indices=[int(value[0]), int(value[1])], edge_indices = int(value[2]))
+                    if self.parent.channel_data[0].shape[0] * self.parent.channel_data[0].shape[1] * self.parent.channel_data[0].shape[2] > self.parent.mini_thresh:
+                        self.parent.mini_overlay = True
+                        self.parent.create_mini_overlay(node_indices=[int(value[0]), int(value[1])], edge_indices = int(value[2]))
+                    else:
+                        self.parent.create_highlight_overlay(node_indices=[int(value[0]), int(value[1])], edge_indices = int(value[2]))
+                    self.parent.clicked_values['nodes'] = []
+                    self.parent.clicked_values['edges'] = []
                     self.parent.clicked_values['edges'].append(value[2])
                     self.parent.clicked_values['nodes'].append(value[0])
                     self.parent.clicked_values['nodes'].append(value[1])
                 except:
-                    self.parent.create_highlight_overlay(node_indices=[int(value[0]), int(value[1])])
+                    if self.parent.channel_data[0].shape[0] * self.parent.channel_data[0].shape[1] * self.parent.channel_data[0].shape[2] > self.parent.mini_thresh:
+                        self.parent.mini_overlay = True
+                        self.parent.create_mini_overlay(node_indices=[int(value[0]), int(value[1])])
+                    else:
+                        self.parent.create_highlight_overlay(node_indices=[int(value[0]), int(value[1])])
+                    self.parent.clicked_values['nodes'] = []
+                    self.parent.clicked_values['edges'] = []
                     self.parent.clicked_values['nodes'].append(value[0])
                     self.parent.clicked_values['nodes'].append(value[1])
 
@@ -4571,6 +4776,8 @@ class Show3dDialog(QDialog):
                             arrays_4d.append(channel)
 
             if self.parent().highlight_overlay is not None:
+                if self.parent().mini_overlay == True:
+                    self.parent().create_highlight_overlay(node_indices = self.parent().clicked_values['nodes'], edge_indices = self.parent().clicked_values['edges'])
                 arrays_3d.append(self.parent().highlight_overlay)
                 colors.append(color_template[4])
         
@@ -4705,6 +4912,45 @@ class IdOverlayDialog(QDialog):
 
         self.accept()
 
+class ColorOverlayDialog(QDialog):
+
+    def __init__(self, parent=None):
+
+        super().__init__(parent)
+        self.setWindowTitle("Generate Node (or Edge) -> Color Overlay?")
+        self.setModal(True)
+
+        layout = QFormLayout(self)
+
+        self.down_factor = QLineEdit("")
+        layout.addRow("down_factor (for speeding up overlay generation - optional):", self.down_factor)
+
+        # Add Run button
+        run_button = QPushButton("Generate (Will go to Overlay 2)")
+        run_button.clicked.connect(self.coloroverlay)
+        layout.addWidget(run_button)
+
+    def coloroverlay(self):
+
+        down_factor = float(self.down_factor.text()) if self.down_factor.text().strip() else None
+
+        if self.parent().active_channel == 0:
+            mode = 0
+            self.sort = 'Node'
+        else:
+            mode = 1
+            self.sort = 'Edge'
+
+
+        result, legend = my_network.node_to_color(down_factor = down_factor, mode = mode)
+
+        self.parent().format_for_upperright_table(legend, f'{self.sort} Id', f'Encoding Val: {self.sort}', 'Legend')
+
+
+        self.parent().load_channel(3, channel_data = result, data = True)
+
+        self.accept()
+
 
 class ShuffleDialog(QDialog):
 
@@ -4745,11 +4991,15 @@ class ShuffleDialog(QDialog):
             accepted_target = self.target_selector.currentIndex()
 
             if accepted_mode == 4:
+                if self.parent().mini_overlay == True:
+                    self.parent().create_highlight_overlay(node_indices = self.parent().clicked_values['nodes'], edge_indices = self.parent().clicked_values['edges'])
                 active_data = self.parent().highlight_overlay
             else:
                 active_data = self.parent().channel_data[accepted_mode]
 
             if accepted_target == 4:
+                if self.parent().mini_overlay == True:
+                    self.parent().create_highlight_overlay(node_indices = self.parent().clicked_values['nodes'], edge_indices = self.parent().clicked_values['edges'])
                 target_data = self.parent().highlight_overlay
             else:
                 target_data = self.parent().channel_data[accepted_target]
@@ -6851,6 +7101,8 @@ class MaskDialog(QDialog):
             output_target = self.output_selector.currentIndex()
 
             if accepted_mode == 4:
+                if self.parent().mini_overlay == True:
+                    self.parent().create_highlight_overlay(node_indices = self.parent().clicked_values['nodes'], edge_indices = self.parent().clicked_values['edges'])
                 active_data = self.parent().highlight_overlay
             else:
                 active_data = self.parent().channel_data[accepted_mode]

{nettracer3d-0.5.3 → nettracer3d-0.5.4}/src/nettracer3d/segmenter.py

@@ -401,7 +401,7 @@ class InteractiveSegmenter:
         
         return foreground, background
 
-    def segment_volume(self, chunk_size=64, gpu=False):
+    def segment_volume(self, chunk_size=None, gpu=False):
         """Segment volume using parallel processing of chunks with vectorized chunk creation"""
         #Change the above chunk size to None to have it auto-compute largest chunks (not sure which is faster, 64 seems reasonable in test cases)
 

{nettracer3d-0.5.3 → nettracer3d-0.5.4/src/nettracer3d.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: nettracer3d
-Version: 0.5.3
+Version: 0.5.4
 Summary: Scripts for intializing and analyzing networks from segmentations of three dimensional images.
 Author-email: Liam McLaughlin <mclaughlinliam99@gmail.com>
 Project-URL: User_Tutorial, https://www.youtube.com/watch?v=cRatn5VTWDY
@@ -44,9 +44,10 @@ NetTracer3D is free to use/fork for academic/nonprofit use so long as citation i
 
 NetTracer3D was developed by Liam McLaughlin while working under Dr. Sanjay Jain at Washington University School of Medicine.
 
--- Version 0.5.3 updates --
+-- Version 0.5.4 updates --
 
-1. Improved calculate volumes method. Previous method used np.argwhere() to count voxels of labeled objects in parallel which was quite strenuous in large arrays with many objects. New method uses np.bincount() which uses optimized numpy C libraries to do the same.
-2. scipy.ndimage.find_objects() method was replaced as the method to find bounding boxes for objects when searching for object neighborhoods for the morphological proximity network and the edge < > node interaction quantification. This new version should be substantially faster in big arrays with many labels. (Depending on how well this improves performance, I may reimplement the secondary network search algorithm, as a side-option, which uses the same parallel-search within subarray strategies, as opposed to the primary network search algorithm that uses distance transforms).
-3. Image viewer window can now load in .nii format images, as well as .jpeg, .jpg, and .png. The nibabel library was added to the dependencies to enable .nii loading, although this is currently all it is used for (and the gui will still run without nibabel).
-4. Fixed bug regarding deleting edge objects. 
+1. Added new function to GUI in image -> overlays -> color nodes/edges. Generates a rgb array corresponding to the nodes/edge labels where each node/edge (depending which array is selected) is randomly assigned a unique rgb color in an overlay channel. This can be used, for example, to color code labeled branches for easy identification of which branch is which.
+
+2. Improved highlight overlay general functionality (for selecting nodes/edges). Previously selecting a node/edge had the program attempting to create an equal sized array as an overlay, find all objects corresponding to the selected ones, fill those into the new highlight overlay, then overlay that image. This was understandably quite slow in big arrays where the system was wasting a lot of time searching the entire array every time something was selected. New version retains this functionality for arrays below 125 million voxels, since search time is rather manageable at that size. For larger arrays, it instead draws the highlight for the selected objects only into the current slice, rendering a new slice whenever the user scrolls in the stack (although the entire highlight overlay is still initialized as a placeholder). Functions that require the use of the entire highlight overlay (such as masking) are correspondingly updated to draw the entirety of the highlight overlay before executing (when the system has up until that point been drawing slices one at a time). This will likely be the retained behavior moving forward, although to eliminate this behavior, one can open nettracer_gui.py and set self.mini_thresh to some comically large value. The new highlight overlay seems to work effectively the same but faster in my testing although it is possible a bug slipped through, which I will fix if informed about (or if I find it myself).
+
+3. For the machine learning segmenter, changed the system to attempt to segment the image by chunking the array into the largest possible chunks that can be divided across all CPU cores. Previously the system split the array into 64^3 voxel sized chunks and passed those to the CPU cores until everything was processed. I am not sure which version is more efficient/faster so this is somewhat of a test. In theory the new behavior could be faster because it asking Python to interpret less stuff. 

nettracer3d-0.5.3/README.md

@@ -1,16 +0,0 @@
-NetTracer3D is a python package developed for both 2D and 3D analysis of microscopic images in the .tif file format. It supports generation of 3D networks showing the relationships between objects (or nodes) in three dimensional space, either based on their own proximity or connectivity via connecting objects such as nerves or blood vessels. In addition to these functionalities are several advanced 3D data processing algorithms, such as labeling of branched structures or abstraction of branched structures into networks. Note that nettracer3d uses segmented data, which can be segmented from other softwares such as ImageJ and imported into NetTracer3D, although it does offer its own segmentation via intensity and volumetric thresholding, or random forest machine learning segmentation. NetTracer3D currently has a fully functional GUI. To use the GUI, after installing the nettracer3d package via pip, enter the command 'nettracer3d' in your command prompt:
-
-
-This gui is built from the PyQt6 package and therefore may not function on dockers or virtual envs that are unable to support PyQt6 displays. More advanced documentation is coming down the line, but for now please see: https://www.youtube.com/watch?v=cRatn5VTWDY
-for a video tutorial on using the GUI.
-
-NetTracer3D is free to use/fork for academic/nonprofit use so long as citation is provided, and is available for commercial use at a fee (see license file for information).
-
-NetTracer3D was developed by Liam McLaughlin while working under Dr. Sanjay Jain at Washington University School of Medicine.
-
--- Version 0.5.3 updates --
-
-1. Improved calculate volumes method. Previous method used np.argwhere() to count voxels of labeled objects in parallel which was quite strenuous in large arrays with many objects. New method uses np.bincount() which uses optimized numpy C libraries to do the same.
-2. scipy.ndimage.find_objects() method was replaced as the method to find bounding boxes for objects when searching for object neighborhoods for the morphological proximity network and the edge < > node interaction quantification. This new version should be substantially faster in big arrays with many labels. (Depending on how well this improves performance, I may reimplement the secondary network search algorithm, as a side-option, which uses the same parallel-search within subarray strategies, as opposed to the primary network search algorithm that uses distance transforms).
-3. Image viewer window can now load in .nii format images, as well as .jpeg, .jpg, and .png. The nibabel library was added to the dependencies to enable .nii loading, although this is currently all it is used for (and the gui will still run without nibabel).
-4. Fixed bug regarding deleting edge objects.