contrek 1.0.3 → 1.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e3520e328bbf96ac5cb581ebfeedd1a1e70da62937bb1544d83c869a73b468da
-  data.tar.gz: 341b3852da6c311be99d0483cc4e99bb63e4669db6b68eda697081282d7d9c99
+  metadata.gz: d96b6c7ae60aefc5e7afed98c4619cf4e77cd0a226f2994d4dbc4f91952b315c
+  data.tar.gz: 336e0c2ca0f36f4608fd1547bdbcb27b5d78624a288d329a003e54412d6ce287
 SHA512:
-  metadata.gz: 05ae4b575e7a207b1cea114c45cac1fee1082e403f9e051a8db8f0b64b7b603e7fc8e6a1f083b29cc36032d4d42191884452942faf0c91d4fea9f48b52eb5224
-  data.tar.gz: 44b9b82008cb944ffd8aa598a72b08a9a09247f948f8d23eeec76f1fca6867a514dc32cfcc510359f90abf19ffd418a16d1f33aa1cd645a959444e8f00e8bdc7
+  metadata.gz: 615ae9e725c5fa149bb3f9f30be365863d348962f26fbda8610285a8e096b80dc6d564438480a8b783dc607c5bc4b3c66bb9bc5ca808d2783c494b74e3b3b7be
+  data.tar.gz: 1d2fc4088a58c1ffe2dbe46c08b67a728bf3eacce02ead907aa7b871565982e3161681205930de777bc7af3e702310dd3030fb961c0cbd71fb15280704e37da3

data/Gemfile.lock

@@ -1,15 +1,19 @@
 PATH
   remote: .
   specs:
-    contrek (1.0.2)
+    contrek (1.0.4)
       chunky_png (~> 1.4)
+      concurrent-ruby (~> 1.3.5)
       rice (~> 4.5)
 
 GEM
   remote: https://rubygems.org/
   specs:
     ast (2.4.3)
+    base64 (0.3.0)
     chunky_png (1.4.0)
+    concurrent-ruby (1.3.5)
+    curses (1.5.3)
     diff-lcs (1.6.1)
     json (2.11.3)
     language_server-protocol (3.17.0.4)
@@ -54,6 +58,8 @@ GEM
       lint_roller (~> 1.1)
       rubocop (>= 1.75.0, < 2.0)
       rubocop-ast (>= 1.38.0, < 2.0)
+    ruby-prof (1.7.2)
+      base64
     ruby-progressbar (1.13.0)
     standard (1.51.1)
       language_server-protocol (~> 3.17.0.2)
@@ -77,7 +83,9 @@ PLATFORMS
 
 DEPENDENCIES
   contrek!
+  curses (>= 1.5.3)
   rspec (~> 3.12)
+  ruby-prof (>= 1.7.2)
   standard (~> 1.51)
 
 BUNDLED WITH

data/README.md

@@ -1,8 +1,8 @@
 # Contrek
-Contrek is a Ruby library (C++ powered) to trace png bitmap areas polygonal contours. Manages png images usign png++ and picoPNG (version 20101224) libraries. 
+Contrek is a Ruby library (C++ powered) to trace png bitmap areas polygonal contours. Manages png images usign png++ and picoPNG (version 20101224) libraries and may work multithreading.
 
 ## About Contrek library
-Contrek (contour trekking) simply scans your png bitmap and returns shape contour as close polygonal lines, both for the external and internal sides. It can compute the nesting level of the polygons found with a tree structure. It supports various levels and modes of compression and approximation of the found coordinates.
+Contrek (**con**tour **trek**king) simply scans your png bitmap and returns shape contour as close polygonal lines, both for the external and internal sides. It can compute the nesting level of the polygons found with a tree structure. It supports various levels and modes of compression and approximation of the found coordinates. It is capable of multithreaded processing (currently only on the Ruby side), splitting the image into vertical strips and recombining the coordinates in pairs.
 
 In the following image all the non-white pixels have been examined and the result is the red polygon for the outer contour and the green one for the inner one
 ![alt text](contrek.png "Contour tracing")
@@ -43,7 +43,8 @@ result = Contrek.contour!(
   png_file_path: "labyrinth3.png",
   options: {
     class: "value_not_matcher",
-    color: {r: 255, g: 0, b: 0, a: 255}
+    color: {r: 255, g: 0, b: 0, a: 255},
+    finder: {treemap: true}
   }
 )
 ```
@@ -85,6 +86,67 @@ You can also read base64 png images
 png_bitmap = CPPRemotePngBitMap.new("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+P+/HgAFhAJ/wlseKgAAAABJRU5ErkJggg==")
 ```
 
+Multithreaded contour processing is supported. However, on Ruby MRI (the standard Ruby implementation, at least up to 3.x), the Global Interpreter Lock (GIL) prevents more than one thread from executing Ruby code simultaneously. As a consequence, execution remains effectively serialized even on multicore systems, unless the gem is used under JRuby or TruffleRuby (not tested).
+
+The actual multithreaded implementation effort is therefore focused on the native C++ component, which is currently under development and is expected to provide substantially higher performance.
+
+```ruby
+result = Contrek.contour!(
+  png_file_path: "./spec/files/images/rectangle_8x8.png",
+  options: {
+    number_of_threads: 2,
+    native: false,
+    class: "value_not_matcher",
+    color: {r: 255, g: 255, b: 255, a: 255},
+    finder: {number_of_tiles: 2, compress: {uniq: true, linear: true}}
+  }
+)
+```
+The example uses 2 threads, and the image has been divided into 2 vertical bands. It is also possible to rely on the system to determine the maximum number of threads supported by your CPU (cores) by passing the appropriate parameter.
+
+```ruby
+  number_of_threads: nil
+```
+
+Regarding multithreading:
+
+- The algorithm splits the contour-detection workflow into multiple phases that can be executed in parallel. The initial contour extraction on each band and the subsequent merging of coordinates between adjacent bands—performed pairwise, recursively, and in a non-deterministic order—results in a final output that is not idempotent. Idempotence is guaranteed only when the exact same merging sequence is repeated.
+
+- The treemap option is currently ignored (multithreaded treemap support will be introduced in upcoming revisions).
+
+
+## Multithreaded approach
+
+The multithreaded contour-tracing implementation operates as follows:
+
+The input image is divided into multiple vertical bands—at least two—where each band shares a one-pixel-wide vertical strip with its adjacent bands.
+For example, if the image is 20 pixels wide and split into two bands, the first band may span from x=0 to x=9 (10 pixels), and the second from x=9 to x=19 (11 pixels).
+The vertical column at x=9 is therefore the shared region.
+
+```md
+01234567890123456789
+---------*----------
+```
+
+Each band is processed independently as if it were a standalone image, with its contour-tracing assigned to a dedicated thread.
+Whenever two adjacent bands have completed processing, they are assigned to an available thread that performs the merge of their coordinates.
+
+For example, with three bands
+B1 – B2 – B3
+the merging sequence might be:
+
+B1 + B2 → (B1+B2) + B3
+or
+
+B2 + B3 → B1 + (B2+B3)
+
+The order is therefore nondeterministic, constrained only by the possible combinations derived from the initial number of bands.
+
+The merging algorithm operates by splitting polygons that intersect the shared band into sequential subsets of coordinates—called parts—distinguishing between segments inside the shared region and those outside it.
+The first stage merges the outer boundary; the second stage merges the disconnected inner parts, which are inserted where needed into the outer sequences produced in the first stage.
+
+This process is applied recursively, merging bands until a single final band remains, upon which the coordinate-compression algorithms are executed.
+
 ## Performances native vs pure
 One of the most complex test you can find under the spec folder is named "scans poly 1200x800", scans this [image](spec/files/images/sample_1200x800.png) computing coordinates to draw polygons drawn in this [result](spec/files/stored_samples/sample_1200x800.png).
 On pure ruby implementation kept time

data/contrek.gemspec

@@ -19,7 +19,11 @@ Gem::Specification.new do |s|
 
   s.add_development_dependency "rspec", "~> 3.12"
   s.add_development_dependency "standard", "~> 1.51"
+  s.add_development_dependency "curses", "~> 1.5", ">= 1.5.3"
+  s.add_development_dependency "ruby-prof", "~> 1.7", ">= 1.7.2"
+
   s.add_dependency "chunky_png", "~> 1.4"
+  s.add_dependency "concurrent-ruby", "~> 1.3.5"
   s.add_dependency "rice", "~> 4.5"
 
   s.required_ruby_version = ">= 3.0.0"

data/lib/contrek/bitmaps/bitmap.rb

@@ -3,19 +3,40 @@ module Contrek
     class Bitmap
       include Painting
 
-      def scan
-        x = 0
+      def scan(start_x: 0, end_x: w)
+        x = start_x
         y = 0
         loop do
           yield x, y, value_at(x, y)
           x += 1
-          if x == w
-            x = 0
+          if x == end_x
+            x = start_x
             y += 1
             break if y == h
           end
         end
       end
+
+      def copy_rect(src_bitmap:, src_x:, src_y:, dest_x:, dest_y:, rect_w:, rect_h:)
+        rect_h.times do |hp|
+          rect_w.times do |wp|
+            src_pos_x = src_x + wp
+            src_pos_y = src_y + hp
+            src_color = src_bitmap.value_at(src_pos_x, src_pos_y)
+            dest_pos_x = dest_x + wp
+            dest_pos_y = dest_y + hp
+            value_set(dest_pos_x, dest_pos_y, src_color)
+          end
+        end
+      end
+
+      def clear!(color: " ")
+        w.times do |x|
+          h.times do |y|
+            value_set(x, y, color)
+          end
+        end
+      end
     end
   end
 end

data/lib/contrek/bitmaps/chunky_bitmap.rb

@@ -28,6 +28,35 @@ module Contrek
 
         @raw[y * @module + x] = value
       end
+
+      def dup!
+        ChunkyBitmap.new(@raw, @module)
+      end
+
+      def draw_polygons(polygons)
+        polygons.each do |polygon|
+          [[:outer, "o"], [:inner, "i"]].each do |side, color|
+            sequences = polygon[side]
+            sequences = [sequences] if side == :outer
+            sequences.each do |sequence|
+              sequence.each do |position|
+                value_set(position[:x], position[:y], color)
+              end
+            end
+          end
+        end
+      end
+
+      def to_terminal
+        puts "  " + (0...@module).map { |i| (i % 10).to_s }.join
+        n = 0
+        @raw.scan(/.{1,#{@module}}/).each do |line|
+          puts n.to_s + " " + line
+          n += 1
+          n = 0 if n >= 10
+        end
+        puts
+      end
     end
   end
 end

data/lib/contrek/bitmaps/custom_bitmap.rb

@@ -0,0 +1,9 @@
+module Contrek
+  module Bitmaps
+    class CustomBitmap < PngBitmap
+      def initialize(w:, h:)
+        @image = ChunkyPNG::Image.new(w, h, ChunkyPNG::Color::TRANSPARENT)
+      end
+    end
+  end
+end

data/lib/contrek/bitmaps/png_bitmap.rb

@@ -57,6 +57,10 @@ module Contrek
       def inspect
         "PngBitMap"
       end
+
+      def clear
+        @image = ChunkyPNG::Image.new(w, h, ChunkyPNG::Color.rgba(255, 255, 255, 255))
+      end
     end
   end
 end

data/lib/contrek/finder/concurrent/clipped_polygon_finder.rb

@@ -0,0 +1,13 @@
+module Contrek
+  module Concurrent
+    class ClippedPolygonFinder < Finder::PolygonFinder
+      attr_reader :start_x, :end_x
+
+      def initialize(bitmap:, matcher:, start_x:, end_x:, options: {})
+        @start_x = start_x
+        @end_x = end_x
+        super(bitmap, matcher, nil, options)
+      end
+    end
+  end
+end

data/lib/contrek/finder/concurrent/cluster.rb

@@ -0,0 +1,86 @@
+module Contrek
+  module Concurrent
+    class Cluster
+      attr_reader :tiles, :hub
+
+      def initialize(finder:, height:, width:)
+        @finder = finder
+        @tiles = []
+        @hub = Hub.new(height:, width:)
+      end
+
+      def add(tile)
+        last_tile = @tiles.last
+        last_tile.next = last_tile.circular_next = tile if last_tile
+        @tiles << tile
+        tile.prev = last_tile
+        tile.circular_next = last_tile
+        tile.cluster = self
+      end
+
+      # Here, two tiles are merged into a single one. The tile is assigned new shapes composed of the detected
+      # outer sequences and both the old and new inner ones. Shapes that are not in the contact area are directly
+      # reassigned unchanged to the new shape.
+      def merge_tiles!
+        tot_inner = 0
+        tot_outer = 0
+
+        new_shapes = []
+        tot_outer += Benchmark.measure do
+          @tiles.each do |tile|
+            tile.shapes.each do |shape|
+              next if shape.outer_polyline.on?(Polyline::TRACKED_OUTER) || shape.outer_polyline.width == 0
+              if shape.outer_polyline.boundary?
+                shape.outer_polyline.partition!
+                shape.outer_polyline.precalc!
+              end
+            end
+          end
+        end.real
+
+        @tiles.each do |tile|
+          tile.shapes.each do |shape|
+            next if shape.outer_polyline.on?(Polyline::TRACKED_OUTER) || shape.outer_polyline.width == 0
+
+            if shape.outer_polyline.boundary? && shape.outer_polyline.next_tile_eligible_shapes.any?
+              new_outer = nil
+              new_inners = shape.inner_polylines
+              cursor = Cursor.new(cluster: self, shape: shape)
+
+              tot_outer += Benchmark.measure do
+                new_outer = cursor.join_outers!
+              end.real
+              tot_inner += Benchmark.measure do
+                new_inner_seq = cursor.join_inners!(new_outer)
+                new_inners += new_inner_seq if new_inner_seq.any?
+                new_inners += cursor.orphan_inners
+              end.real
+
+              new_shapes << Shape.new.tap do |shape|
+                polyline = Polyline.new(tile: tile, polygon: new_outer.to_a)
+                shape.outer_polyline = polyline
+                polyline.shape = shape
+                shape.inner_polylines = new_inners
+              end
+            else
+              new_shapes << shape
+            end
+          end
+        end
+        past_tot_outer = @tiles.first.benchmarks[:outer] + @tiles.last.benchmarks[:outer]
+        past_tot_inner = @tiles.first.benchmarks[:inner] + @tiles.last.benchmarks[:inner]
+
+        tile = Tile.new(
+          finder: @finder,
+          start_x: @tiles.first.start_x,
+          end_x: @tiles.last.end_x,
+          benchmarks: {outer: tot_outer + past_tot_outer, inner: tot_inner + past_tot_inner},
+          name: @tiles.first.name + @tiles.last.name
+        )
+
+        tile.assign_shapes!(new_shapes)
+        tile
+      end
+    end
+  end
+end

data/lib/contrek/finder/concurrent/cursor.rb

@@ -0,0 +1,219 @@
+module Contrek
+  module Concurrent
+    class Cursor
+      attr_reader :orphan_inners
+      def initialize(cluster:, shape:)
+        @polylines_sequence = []
+        @cluster = cluster
+        @outer_polyline = shape.outer_polyline
+        @orphan_inners = []
+        @shapes = [shape]
+      end
+
+      def inspect
+        self.class
+      end
+
+      # Given the initial polyline, draw its outer boundary, possibly extending into
+      # adjacent polylines, and then connect them. At the end, @polylines_sequence
+      # contains the merged polylines. Returns a new resulting polyline.
+      def join_outers!
+        seq_log = []
+        @polylines_sequence << @outer_polyline
+
+        outer_joined_polyline = Sequence.new
+
+        traverse_outer(@outer_polyline.parts.first,
+          seq_log,
+          @polylines_sequence,
+          @shapes,
+          outer_joined_polyline)
+        outer_joined_polyline.pop! if outer_joined_polyline.head.payload == outer_joined_polyline.tail.payload &&
+          @cluster.tiles.first.left? && @cluster.tiles.last.right?
+
+        @polylines_sequence.uniq!
+
+        @polylines_sequence.each { |c| c.turn_on(Polyline::TRACKED_OUTER) }
+        (@shapes - [@outer_polyline.shape]).each do |shape|
+          @orphan_inners += shape.inner_polylines
+          shape.clear_inner!
+        end
+
+        outer_joined_polyline
+      end
+
+      def join_inners!(outer_seq)
+        # search for missing sequence to sew
+        missing_shapes = []
+        @cluster.tiles.each do |tile|
+          tile.shapes.each do |shape|
+            next if shape.outer_polyline.on?(Polyline::TRACKED_OUTER) ||
+              shape.outer_polyline.on?(Polyline::TRACKED_INNER) ||
+              !shape.outer_polyline.boundary? ||
+              @polylines_sequence.include?(shape.outer_polyline)
+            missing_shapes << shape
+          end
+        end
+        @polylines_sequence.each do |polyline|
+          missing_shapes.each do |missing_shape|
+            outer_polyline = missing_shape.outer_polyline
+            if (intersection = polyline.intersection(outer_polyline)).any?
+              inject_sequences_left, inject_sequences_right = polyline.sew!(intersection, outer_polyline)
+              combine!(inject_sequences_right, inject_sequences_left).each do |sewn_sequence|
+                sewn_sequence.uniq!
+                @orphan_inners << sewn_sequence if sewn_sequence.size > 1 && sewn_sequence.map { |c| c[:x] }.uniq.size > 1 # segmenti non sono ammessi, solo aree
+              end
+              outer_polyline.clear!
+              outer_polyline.turn_on(Polyline::TRACKED_OUTER)
+              outer_polyline.turn_on(Polyline::TRACKED_INNER)
+              @orphan_inners += missing_shape.inner_polylines
+            end
+          end
+        end
+
+        retme = collect_inner_sequences(outer_seq)
+
+        @polylines_sequence.each do |polyline|
+          polyline.turn_on(Polyline::TRACKED_INNER)
+        end
+        retme
+      end
+
+      private
+
+      # rubocop:disable Lint/NonLocalExitFromIterator
+      def traverse_outer(act_part, all_parts, polylines, shapes, outer_joined_polyline)
+        all_parts << act_part if all_parts.last != act_part
+        shapes << act_part.polyline.shape if !shapes.include?(act_part.polyline.shape)
+
+        if act_part.is?(Part::EXCLUSIVE)
+          return if act_part.size == 0
+          while (position = act_part.next_position)
+            return if outer_joined_polyline.size > 1 &&
+              outer_joined_polyline.head.payload == position.payload &&
+              act_part == all_parts.first
+            outer_joined_polyline.add(position)
+          end
+        else
+          while (new_position = act_part.iterator)
+            return if outer_joined_polyline.size > 1 &&
+              outer_joined_polyline.head.payload == new_position.payload &&
+              act_part == all_parts.first
+            outer_joined_polyline.add(Position.new(position: new_position.payload, hub: @cluster.hub))
+
+            act_part.polyline.next_tile_eligible_shapes.each do |shape|
+              if (part = shape.outer_polyline.find_first_part_by_position(new_position))
+                if all_parts[-2] != part
+                  if all_parts.size >= 2
+                    map = all_parts[-2..].map(&:type).uniq
+                    break if map.size == 1 && map.first == Part::SEAM
+                  end
+                  polylines << part.polyline.shape.outer_polyline
+                  part.next_position(new_position.payload)
+                  traverse_outer(part, all_parts, polylines, shapes, outer_joined_polyline)
+                  return
+                end
+              end
+            end
+            act_part.passes += 1
+            act_part.next_position
+          end
+        end
+        next_part = act_part.circular_next
+        next_part.rewind!
+        traverse_outer(act_part.circular_next, all_parts, polylines, shapes, outer_joined_polyline)
+      end
+
+      def collect_inner_sequences(outer_seq)
+        return_sequences = []
+        @polylines_sequence.each do |polyline|
+          polyline.parts.each do |part|
+            if part.innerable?
+              all_parts = []
+              bounds = {min: polyline.max_y, max: 0}
+              traverse_inner(part, all_parts, bounds)
+              range_of_bounds = (bounds[:min]..bounds[:max])
+
+              retme_sequence = Sequence.new
+              all_parts.map do |part|
+                part.touch!
+
+                retme_sequence.move_from(part) do |pos|
+                  next false if part.is?(Part::ADDED) && !(range_of_bounds === pos.payload[:y])
+                  !(polyline.tile.tg_border?(pos.payload) && pos.end_point.queues.include?(outer_seq))
+                end
+              end.flatten
+              raw_sequence = retme_sequence.to_a
+              return_sequences << raw_sequence if raw_sequence.map { _1[:x] }.uniq.take(2).size > 1
+            end
+          end
+        end
+        return_sequences
+      end
+
+      def traverse_inner(act_part, all_parts, bounds)
+        return if act_part == all_parts.first
+
+        if act_part.size > 0
+          min_y, max_y = act_part.to_a.minmax_by { |p| p[:y] }
+          bounds[:min] = [bounds[:min], min_y[:y]].min
+          bounds[:max] = [bounds[:max], max_y[:y]].max
+        end
+        if act_part.innerable?
+          all_parts << act_part
+          while (act_part = act_part.next)
+            if act_part.innerable?
+              all_parts << act_part
+            else
+              act_part.polyline.next_tile_eligible_shapes.each do |shape|
+                shape.outer_polyline.parts.each do |dest_part|
+                  next if dest_part.trasmuted
+                  if dest_part.intersect_part?(act_part)
+                    link_seq = duplicates_intersection(dest_part, act_part)
+                    if link_seq.any?
+                      ins_part = Part.new(Part::ADDED, act_part.polyline)
+                      link_seq.each { |pos| ins_part.add(Position.new(position: pos, hub: @cluster.hub)) }
+                      all_parts << ins_part
+                    end
+                    traverse_inner(dest_part.circular_next, all_parts, bounds)
+                    return
+                  end
+                end
+              end
+              all_parts << act_part if act_part.is?(Part::SEAM)
+            end
+          end
+        elsif act_part.next
+          traverse_inner(act_part.next, all_parts, bounds)
+        end
+      end
+      # rubocop:enable Lint/NonLocalExitFromIterator
+
+      # The sequences should contain inverted parts, e.g., 4,5,6,6,5,4. These parts must be
+      # removed from the sequences, and the remaining elements are compared. Only the
+      # elements that appear once between the two sequences are kept. This represents
+      # a connection between parts inserted afterwards.
+      # TODO evaluate the adoption of remove_adjacent_pairs!
+      def duplicates_intersection(part_a, part_b)
+        a1 = part_a.inverts ? part_a.remove_adjacent_pairs : part_a.to_a
+        b1 = part_b.inverts ? part_b.remove_adjacent_pairs : part_b.to_a
+
+        (a1 - b1) + (b1 - a1)
+      end
+
+      # example
+      # a = [[A],[B,C]]
+      # b = [[D],[E,F]]
+      # res = [[D,B,C],[E,F,A]]
+      def combine!(seqa, seqb)
+        rets = []
+        [seqa.count, seqb.count].min.times do
+          last = seqa.pop
+          first = seqb.shift
+          rets << first + last
+        end
+        rets
+      end
+    end
+  end
+end

data/lib/contrek/finder/concurrent/end_point.rb

@@ -0,0 +1,14 @@
+module Contrek
+  module Concurrent
+    class EndPoint
+      attr_reader :queues
+      def initialize
+        @queues = []
+      end
+
+      def inspect
+        "EndPoint[#{@queues.size}]"
+      end
+    end
+  end
+end

data/lib/contrek/finder/concurrent/fake_cluster.rb

@@ -0,0 +1,10 @@
+module Contrek
+  module Concurrent
+    class FakeCluster < Finder::NodeCluster
+      def initialize(polygons, options)
+        @options = options
+        @polygons = polygons
+      end
+    end
+  end
+end