diarize-ruby 0.3.3 → 0.3.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 24c441a25ed2d6138eb4afe303016301d4abad6e
-  data.tar.gz: cdb304d830b7e1c1fd1d561b200c3a52177a35e6
+  metadata.gz: ccd52691b1b5e219ccea4aaba625f30afc3f16db
+  data.tar.gz: d7d61fdae578b67ac0c9d2adec42c70fd51fb64d
 SHA512:
-  metadata.gz: 071bff7fbba3e729b3edac28b8ed2a37153d8de4e7ba1f561c60490200b8ad0ae8f00cfca44e40a5adec4c4de19e9344c72358364b0e29475865868c1d5afc47
-  data.tar.gz: 1398a7144d95c87d9dbbe66318ffe614e6871fd39f94904e1c5cc8bc07948bab170ddbf3881d166a6b6268dd519af6ab0b012ee374ace46e2d0561d6cb16702e
+  metadata.gz: af259a8ab28a1b6e0dccd9a0699262dd33032d1f21997f78d0762622ee1c714aa5df4affae9ff942e6a21583f9ef9a8d89f3f9127d1d149fd6d532a5d050a8bf
+  data.tar.gz: f688d8d78b3eb163d7a604cdb0dac05202841137d40c65c3ddcc016d8303bcbcd8fa1303eaec8a6459edd2a93fed39874cf73b6b60040f04cfb08786b818afec

data/CHANGELOG.md

@@ -1,3 +1,8 @@
+## [v0.3.4] - 2016-11-16
+
+- Add diarize binary
+- DRb server
+
 ## [v0.3.3] - 2016-11-14
 
 - Remove require for audio-playback

data/README.md

@@ -1,17 +1,8 @@
 # diarize-ruby
 
-This library provides an easy-to-use toolkit for speaker segmentation (diarization) and identification from audio.
+This library provides an easy-to-use toolkit for speaker segmentation (diarization) and identification from audio. It was adopted from [diarize-jruby](https://github.com/bbc/diarize-jruby), being used within the BBC R&D World Service.
 
-This library was adopted from [diarize-jruby](https://github.com/bbc/diarize-jruby), being used within the BBC R&D World Service.
-
-The main reason from deviating from the original library is to have a universal gem that works with either Ruby interpreter. It uses [Ruby Java Bridge](http://rjb.rubyforge.org) instead of [JRuby](http://jruby.org).
-
-Work to be done:
-
-* Universal gem that works on JRuby and various Ruby implementations (MRI) and versions
-* Use performant math packages tuned to either Ruby implementation
-* Add support for alternative diarization tools
-* Add CI tool
+The main reason from deviating from the original is to provide support for Ruby MRI. It uses [Ruby Java Bridge](http://rjb.rubyforge.org) instead of [JRuby](http://jruby.org).
 
 ## Speaker Diarization
 
@@ -47,21 +38,65 @@ If you are using a different version of LIUM than what is bundled in the `bin` f
 
 ## Examples
 
-    $ irb -I lib
-    > require "diarize"
-    > audio = Diarize::Audio.new URI.join('file:///', File.join(File.expand_path(File.dirname(__FILE__)), "test", "data", "will-and-juergen.wav"))
-    > audio.analyze!
-    > audio.segments
-    > audio.speakers
-    > audio.to_rdf
-    > speakers = audio.speakers
-    > speakers.first.gender
-    > speakers.first.model.mean_log_likelihood
-    > speakers.first.model.components.size
-    > ...
-    > speakers ||= other_speakers
-    > Diarize::Speaker.match(speakers)
+### Get Segments and Speakers
+
+From Ruby:
+
+    $ diarize console
+
+```ruby
+    audio = Diarize::Audio.new(URI.join('file:///', File.join(File.expand_path(File.dirname(__FILE__)), "test", "data", "will-and-juergen.wav")))
+
+    audio.analyze!
+    audio.segments
+    audio.speakers
+    audio.to_rdf
+    speakers = audio.speakers
+    speakers.first.gender
+    speakers.first.model.mean_log_likelihood
+    speakers.first.model.components.size
+    ...
+    speakers ||= other_speakers
+    Diarize::Speaker.match(speakers)
+```
+
+From bash:
+
+    $ diarize audio speaker example.wav
+
+### Start Server
+
+Some Java implementations (i.e. OpenJDK on Linux) are causing trouble running [Rjb](http://rjb.rubyforge.org) on threaded environments (e.g. [Celluloid](https://github.com/celluloid/celluloid), [Sidekick](https://github.com/mperham/sidekiq), [Shoryuken](https://github.com/phstc/shoryuken)) leading to instability. One workaround is to start diarize as server [DRb](http://ruby-doc.org/stdlib-2.0.0/libdoc/drb/rdoc/DRb.html) by a client proxy.
 
+Start the diarizer in a separate process as a server:
+
+    $ diarize server -P 9999 -H localhost
+    Drb server
+    diarize-ruby 0.3.4
+    Listening on druby://localhost:9999, CTRL+C to stop
+
+### Client
+
+From bash:
+
+    $ diarize remote audio segment example.wav
+
+From Ruby:
+
+```ruby
+    require "diarize"
+    require "drb/drb"
+
+    server_uri = "druby://localhost:9999"
+    DRb.start_service
+    client = DRbObject.new_with_uri(server_uri)
+
+    audio_uri = URI.join('file:///', File.join(File.expand_path(File.dirname(__FILE__)), "test", "data", "will-and-juergen.wav"))
+    audio = client.new_audio(audio_uri)
+    audio.analyze!
+    audio.segments
+    ...
+```
 
 ## Running tests
 
@@ -103,6 +138,13 @@ is licensed under the GNU General Public License version 2. See
 http://lium3.univ-lemans.fr/diarization/doku.php/licence for more
 information.
 
+## TODOs
+
+* Universal gem that works on JRuby and various Ruby implementations (MRI) and versions
+* Use performant math packages tuned to either Ruby implementation
+* Add support for alternative diarization tools
+* Add CI tool
+
 ## Developer Resources
 
 * [Connecting Ruby to Java and vice versa](http://nofail.de/2010/04/ruby-in-java-java-in-ruby-jruby-or-ruby-java-bridge/)

data/bin/diarize

@@ -0,0 +1,191 @@
+#!/usr/bin/env ruby
+require "diarize"
+require "gli"
+require "uri"
+require "drb/drb"
+require "byebug"
+
+include GLI::App
+
+def uri_from_args(args)
+  url_or_file_name = args.first
+  uri = URI.parse(url_or_file_name)
+  if uri.scheme && uri.scheme.match(/^(http|https|file)$/)
+    uri = url_or_file_name
+  else
+    uri = if url_or_file_name[0] == "/"
+      URI.join('file:///', url_or_file_name)
+    else
+      URI.join('file:///', File.join(File.expand_path(Dir.pwd), url_or_file_name))
+    end
+  end
+  uri
+end
+
+def build_audio_from_args(args, options = {})
+  uri = uri_from_args(args)
+  $stdout.puts uri.to_s if options[:verbose]
+  audio = Diarize::Audio.new(uri)
+  audio.analyze!
+  audio
+end
+
+def build_remote_audio_from_args(args, options = {})
+  remote = client(options)
+  audio_uri = uri_from_args(args)
+
+  $stdout.puts uri.to_s if options[:verbose]
+  audio = remote.new_audio(audio_uri)
+  audio.analyze!
+  audio
+rescue DRb::DRbConnError => ex
+  $stdout.puts 'Connection error, start server with "diarize server"'
+end
+
+def client(options = {})
+  uri = "druby://#{options[:host]}:#{options[:port]}"
+  DRb.start_service
+  DRbObject.new_with_uri(uri)
+end
+
+program_desc 'Diarize is an easy-to-use toolkit for speaker segmentation.'
+
+version Diarize::VERSION
+
+subcommand_option_handling :normal
+arguments :strict
+
+#-v, --verbose
+desc 'Print more verbose output'
+switch [:v, :verbose]
+
+#-H, --host HOSTNAME
+desc 'Host, e.g. "www.example.com"'
+default_value 'localhost'
+arg_name 'HOSTNAME'
+flag [:H, :host]
+
+#-P, --port PORT
+desc 'Port number'
+default_value 9999
+arg_name 'PORT'
+flag [:P, :port]
+
+desc 'start console'
+command [:c, :console] do |console|
+  console.action do |global_options, options, args|
+    require 'irb'
+
+    ARGV.clear
+    IRB.start
+  end
+end
+
+command [:r, :remote] do |remote|
+  remote.command [:a, :audio] do |audio|
+    audio.desc 'diarize remote audio segments FILENAME'
+    audio.arg_name 'FILENAME', :multiple
+    audio.command [:se, :segment, :segments] do |segments|
+      segments.action do |global_options, options, args|
+        audio = build_remote_audio_from_args(args, options.merge(global_options))
+        audio.segments.each do |segment|
+          $stdout.puts segment.uri
+        end if audio
+      end
+    end
+
+    audio.desc 'diarize remote audio speakers FILENAME'
+    audio.arg_name 'FILENAME', :multiple
+    audio.command [:sp, :speaker, :speakers] do |speakers|
+      speakers.action do |global_options, options, args|
+        audio = build_remote_audio_from_args(args, options.merge(global_options))
+        audio.speakers.each do |speaker|
+          $stdout.puts speaker.uri
+        end if audio
+      end
+    end
+  end
+end
+
+command [:a, :audio] do |audio|
+  audio.desc 'diarize audio segments FILENAME'
+  audio.arg_name 'FILENAME', :multiple
+  audio.command [:se, :segment, :segments] do |segments|
+    segments.action do |global_options, options, args|
+      audio = build_audio_from_args(args, global_options)
+      audio.segments.each do |segment|
+        $stdout.puts segment.uri
+      end
+    end
+  end
+
+  audio.desc 'diarize audio speakers FILENAME'
+  audio.arg_name 'FILENAME', :multiple
+  audio.command [:sp, :speaker, :speakers] do |speakers|
+    speakers.action do |global_options, options, args|
+      audio = build_audio_from_args(args, global_options)
+      audio.speakers.each do |speaker|
+        $stdout.puts speaker.uri
+      end
+    end
+  end
+end
+
+desc 'start server'
+command [:s, :server] do |server|
+  server.action do |global_options, options, args|
+    merged_options = options.merge(global_options)
+    argv = merged_options.reject {|k, v| !k.is_a?(String)}
+    argv = argv.inject({}) {|r, h| r[h[0].length > 1 ? "--#{h[0]}" : "-#{h[0]}"] = h.last; r}
+    argv = argv.reject {|k, v| !v}
+    argv = argv.to_a.flatten
+
+    uri = "druby://#{merged_options[:host]}:#{merged_options[:port]}"
+    $stdout.puts "Drb server"
+    $stdout.puts "diarize-ruby #{Diarize::VERSION}"
+    $stdout.puts "Arguments: #{argv.inspect}" if global_options[:verbose]
+    $stdout.puts "Listening on #{uri}, CTRL+C to stop"
+    server = Diarize::Server.new
+    DRb.start_service(uri, server)
+    DRb.thread.join
+  end
+
+  #-H, --host HOSTNAME
+  server.desc 'Host, e.g. "www.example.com"'
+  server.default_value 'localhost'
+  server.arg_name 'HOSTNAME'
+  server.flag [:H, :host]
+
+  #-P, --port PORT
+  server.desc 'Port number'
+  server.default_value 9999
+  server.arg_name 'PORT'
+  server.flag [:P, :port]
+end
+
+pre do |global, command, options, args|
+  # Pre logic here
+  # Return true to proceed; false to abort and not call the
+  # chosen command
+  # Use skips_pre before a command to skip this block
+  # on that command only
+  true
+end
+
+post do |global,command,options,args|
+  # Post logic here
+  # Use skips_post before a command to skip this
+  # block on that command only
+end
+
+on_error do |exception|
+  # Error logic here
+  # return false to skip default error handling
+  true
+end
+
+begin
+  exit run(ARGV)
+rescue
+  $stdout.puts "Bye."
+end

data/diarize-ruby.gemspec

@@ -17,7 +17,7 @@ Gem::Specification.new do |spec|
 
   spec.files         = `git ls-files -z`.split("\x0")
   spec.bindir        = 'bin'
-  spec.executables   = []
+  spec.executables   = ["diarize"]
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ["lib"]
 
@@ -29,4 +29,6 @@ Gem::Specification.new do |spec|
   spec.add_dependency "rjb", "~> 1.5"
   spec.add_dependency "to-rdf", "~> 0"
   spec.add_dependency "jblas-ruby", "~> 1.1"
+
+  spec.add_runtime_dependency('gli', '~> 2.13.0')
 end

data/lib/diarize.rb

@@ -6,12 +6,19 @@ RJB_OPTIONS   = ['-Xms16m', '-Xmx1024m']
 Rjb::load(RJB_LOAD_PATH, RJB_OPTIONS)
 
 require "matrix"
+
+require "to_rdf"
+require "uri"
+require "open-uri"
+require "digest/md5"
+
 require "diarize/version"
-require "diarize/lium"
 require "diarize/audio"
 require "diarize/segment"
+require "diarize/speaker"
 require "diarize/segmentation"
 require "diarize/super_vector"
+require "diarize/server"
 
 # Extenions to the {Ruby-Java Bridge}[http://rjb.rubyforge.org/] module that
 # adds a generic Java object wrapper class.

data/lib/diarize/audio.rb

@@ -1,17 +1,5 @@
-require File.join(File.expand_path(File.dirname(__FILE__)), 'lium')
-require File.join(File.expand_path(File.dirname(__FILE__)), 'segmentation')
-require File.join(File.expand_path(File.dirname(__FILE__)), 'speaker')
-
-require 'rubygems'
-require 'to_rdf'
-require 'uri'
-require 'open-uri'
-require 'digest/md5'
-
 module Diarize
-
   class Audio
-
     attr_reader :path, :file, :uri
 
     def initialize(url_or_uri)
@@ -65,18 +53,18 @@ module Diarize
 
     def speakers
       return @speakers if @speakers
-      @speakers = segments.map { |segment| segment.speaker }.uniq
+      @speakers = segments.map {|segment| segment.speaker}.uniq
     end
 
     def segments_by_speaker(speaker)
-      segments.select { |segment| segment.speaker == speaker }
+      segments.select {|segment| segment.speaker == speaker}
     end
 
     def duration_by_speaker(speaker)
       return unless speaker
       segments = segments_by_speaker(speaker)
       duration = 0.0
-      segments.each { |segment| duration += segment.duration }
+      segments.each {|segment| duration += segment.duration}
       duration
     end
 
@@ -191,6 +179,5 @@ module Diarize
       Rjb::JavaObjectWrapper.new(clusters)
     end
 
-  end
-
+  end # Audio
 end

data/lib/diarize/segment.rb

@@ -1,7 +1,3 @@
-require 'rubygems'
-require 'to_rdf'
-require 'uri'
-
 module Diarize
   class Segment
     include ToRdf

data/lib/diarize/server.rb

@@ -0,0 +1,9 @@
+module Diarize
+  class Server
+
+    def new_audio(url_or_uri)
+      Audio.new(url_or_uri)
+    end
+
+  end
+end

data/lib/diarize/speaker.rb

@@ -1,15 +1,7 @@
-require 'rubygems'
-require 'to_rdf'
-# require 'jblas'
-
 module Diarize
-
   class Speaker
-
-    # include JBLAS
-
     @@log_likelihood_threshold = -33
-    @@detection_threshold = 0.2
+    @@detection_threshold      = 0.2
 
     @@speakers = {}
 
@@ -168,7 +160,5 @@ module Diarize
       Rjb::import('fr.lium.spkDiarization.libModel.ModelIO').writerGMMContainer(output, gmmlist.java_object)
       output.close
     end
-
-  end
-
+  end # Speaker
 end

data/lib/diarize/super_vector.rb

@@ -1,5 +1,4 @@
 module Diarize
-
   class SuperVector
     attr_reader :vector
 
@@ -72,6 +71,5 @@ module Diarize
       @vector.hash
     end
 
-  end
-
+  end # SuperVector
 end

data/lib/diarize/version.rb

@@ -1,3 +1,3 @@
 module Diarize
-  VERSION = "0.3.3"
+  VERSION = "0.3.4"
 end

data/test/server_test.rb

@@ -0,0 +1,11 @@
+require 'test_helper'
+
+class ServerTest < Test::Unit::TestCase
+
+  def test_new_audio_server_wrapper
+    server = Diarize::Server.new
+    audio = server.new_audio(URI('file:' + File.join(File.dirname(__FILE__), 'data', 'foo.wav')))
+    assert_equal Diarize::Audio, audio.class
+  end
+
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: diarize-ruby
 version: !ruby/object:Gem::Version
-  version: 0.3.3
+  version: 0.3.4
 platform: ruby
 authors:
 - Yves Raimond
@@ -109,11 +109,26 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.1'
+- !ruby/object:Gem::Dependency
+  name: gli
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.13.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.13.0
 description: A library for Ruby wrapping the LIUM Speaker Diarization and including
   a few extra tools
 email:
 - jfesslmeier@gmail.com
-executables: []
+executables:
+- diarize
 extensions: []
 extra_rdoc_files: []
 files:
@@ -127,12 +142,13 @@ files:
 - README.md
 - Rakefile
 - bin/LIUM_SpkDiarization-4.2.jar
+- bin/diarize
 - diarize-ruby.gemspec
 - lib/diarize.rb
 - lib/diarize/audio.rb
-- lib/diarize/lium.rb
 - lib/diarize/segment.rb
 - lib/diarize/segmentation.rb
+- lib/diarize/server.rb
 - lib/diarize/speaker.rb
 - lib/diarize/super_vector.rb
 - lib/diarize/ubm.gmm
@@ -143,6 +159,7 @@ files:
 - test/data/will-and-juergen.wav
 - test/segment_test.rb
 - test/segmentation_test.rb
+- test/server_test.rb
 - test/speaker_test.rb
 - test/super_vector_test.rb
 - test/test_helper.rb
@@ -177,6 +194,7 @@ test_files:
 - test/data/will-and-juergen.wav
 - test/segment_test.rb
 - test/segmentation_test.rb
+- test/server_test.rb
 - test/speaker_test.rb
 - test/super_vector_test.rb
 - test/test_helper.rb

data/lib/diarize/lium.rb

@@ -1,5 +0,0 @@
-# require File.join(File.expand_path(File.dirname(__FILE__)), 'LIUM_SpkDiarization-4.2.jar')
-
-def fr
-  Java::Fr
-end