flvedit 0.6.1

data/CHANGELOG.rdoc

@@ -0,0 +1,5 @@
+= flvedit --- History
+
+== Version 0.6 - April 21, 2009
+
+=== Initial release.

data/LICENSE

@@ -0,0 +1,24 @@
+FLVEdit - Copyright (c) 2009 Marc-André Lafortune
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+3. The name of the author may not be used to endorse or promote products
+   derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.rdoc

@@ -0,0 +1,90 @@
+= FLVEdit - Flash video manipulation
+
+FLVEdit is a tool that will read or edit FLV files to:
+* generate meta data
+* add/remove cue points
+* join files
+* inspect files
+
+It is meant as an improved FLVTool2, fixing the shortfalls that prompted others to write FLVTool++ & FLVMeta (see comparison chart)
+
+FLVEdit can be used either as a command-line command or as a ruby library.
+
+== This is not a stable release
+<b>Warning:</b> The basic functionality is there, but I'll be improving features and documentation. Until version 1.0, there will most likely be many changes to the command line interface and the library API.
+
+Comments & requests welcome...
+
+== Installation
+
++flvedit+ is a gem mirrored on Rubyforge and can thus be installed with:
+
+  sudo gem install flvedit
+
+flvedit is compatible with Ruby 1.8 and 1.9 
+
+== Command-line tool
+
+Type 'flvedit' for description of commands.
+
+== Library
+
+FLVEdit is written in ruby and divided in two layers. 
+
+=== FLV file format
+The FLV layer handles the FLV file format. It makes reading and writing FLV files a breeze:
+
+	FLV::File.open("example.flv") {|f| f.to_a }  # ==> [<#FLV::Header...>, <#FLV::Tag...>, ...]
+
+The main class is FLV::Tag with its different possible bodies: FLV::Audio, FLV::Video and most importantly FLV::Event for all meta data related information (onMetaData, onCuePoint, ...)
+
+The data packing and unpacking relies on the packable[http://github.com/marcandre/packable] library.
+
+=== FLV::Edit tool
+
+The FLV::Edit layer is the command-line tool itself. The FLV::Edit::Runner class parses the options and builds a chain of processors to apply to some flv files. Processors all derive from FLV::Edit::Processor::Base and only need to specify what to do for the type of data it wants to process. A simplistic example to use this level:
+
+	class CountCuePoints < FLV::Edit::Processor::Base
+		attr_writer :count
+		def on_cue_point(cue)
+			@count ||= 0
+			@count += 1
+		end
+	end
+
+	# Call manually
+	count =
+		FLV::File.open("example.flv") do |f|
+		  CountCuePoints.new(f).process_all.count
+		end
+
+	# Chain with other commands:
+	count = FLV::Edit::Runner.new([CountCuePoints, FLV::Edit::Processor::Debug], :files => "example.swf").run.count
+
+See FLV::Edit::Processor::Base for details on how the processing works
+
+== Comparisons with existing tools
+
+=== FLVTool2
+
+Features:
+* Can join (concat) flv files
+* Won't load the whole files in memory all at once
+* Won't choke on read-only files
+* Supports extented timestamps (for flv over... 4 hours!)
+
+Code:
+* Complete rewrite
+* More ruby-oriented
+* Commented
+* Unit tests
+* Easier to use library
+* Easily expandable with your own processing
+
+=== Compared to FLVTool++
+* Handles cue points
+* Usable as a library
+<to be completed>
+
+=== Compared to FLVMeta
+<to be completed>

data/Rakefile

@@ -0,0 +1,137 @@
+# encoding: utf-8
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+require 'rcov/rcovtask'
+
+# bench
+begin
+  desc "Benchmark"
+  task :bench do
+    require File.dirname(__FILE__)+"/lib/flv/edit"
+    SHORT_FLV = File.dirname(__FILE__) + "/test/fixtures/short.flv"
+    require 'benchmark' 
+    include Benchmark
+    runner = FLV::Edit::Runner.new([SHORT_FLV, SHORT_FLV, '--Join'])
+    bm(6) do |x| 
+      x.report("test") { 20.times { runner.run } } 
+    end 
+  end
+end
+
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "flvedit"
+    gem.summary = "Command line tool & library to handle FLV files"
+    gem.email = "github@marc-andre.ca"
+    gem.homepage = "http://github.com/marcandre/flvedit"
+    gem.description = <<-EOS
+      flvedit allows you to:
+      * compute metadata for FLV files
+      * merge, split or cut FLVs
+      * insert / remote cue points or other events
+      
+      flvedit is meant as a replacement for FLVTool2, FLVMeta, FLVTool++
+      It can be used as a command line tool or as a Ruby library.
+    EOS
+    gem.authors = ["Marc-André Lafortune"]
+    gem.rubyforge_project = "flvedit"
+    gem.add_dependency "packable", ">=1.2"
+    gem.add_dependency "backports"
+    gem.has_rdoc = true
+    gem.rdoc_options << '--title' << 'FLV::Edit' <<
+                           '--main' << 'README.rdoc' <<
+                           '--line-numbers' << '--inline-source'
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end unless RUBY_VERSION >= "1.9"
+
+Rake::TestTask.new do |t|
+  t.libs << 'lib'
+  t.pattern = 'test/**/test_*.rb'
+  t.verbose = false
+end
+
+Rake::RDocTask.new do |rdoc|
+  if File.exist?('VERSION.yml')
+    config = YAML.load(File.read('VERSION.yml'))
+    version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
+  else
+    version = ""
+  end
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = "FLVEdit #{version}"
+  rdoc.options << '--line-numbers' << '--inline-source'
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+Rcov::RcovTask.new do |t|
+  t.libs << 'test'
+  t.test_files = FileList['test/**/*_test.rb']
+  t.verbose = true
+end
+
+task :default => :rcov
+
+# stats
+begin
+  gem 'rails'
+  require 'code_statistics'
+  namespace :spec do
+    desc "Use Rails's rake:stats task for a gem"
+    task :statsetup do
+      class CodeStatistics
+        def calculate_statistics
+          @pairs.inject({}) do |stats, pair|
+            if 3 == pair.size
+              stats[pair.first] = calculate_directory_statistics(pair[1], pair[2]); stats
+            else
+              stats[pair.first] = calculate_directory_statistics(pair.last); stats
+            end
+          end
+        end
+      end
+      ::STATS_DIRECTORIES = [['Libraries',   'lib',  /.(sql|rhtml|erb|rb|yml)$/],
+                   ['Tests',     'test', /.(sql|rhtml|erb|rb|yml)$/]]
+      ::CodeStatistics::TEST_TYPES << "Tests"
+    end
+  end
+  desc "Report code statistics (KLOCs, etc) from the application"
+  task :stats => "spec:statsetup" do
+    CodeStatistics.new(*STATS_DIRECTORIES).to_s
+  end
+rescue Gem::LoadError => le
+  task :stats do
+    raise RuntimeError, "‘rails’ gem not found - you must install it in order to use this task.n"
+  end
+end
+
+begin
+  require 'rake/contrib/sshpublisher'
+  namespace :rubyforge do
+    
+    desc "Release gem and RDoc documentation to RubyForge"
+    task :release => ["rubyforge:release:gem", "rubyforge:release:docs"]
+    
+    namespace :release do
+      desc "Publish RDoc to RubyForge."
+      task :docs => [:rdoc] do
+        config = YAML.load(
+            File.read(File.expand_path('~/.rubyforge/user-config.yml'))
+        )
+
+        host = "#{config['username']}@rubyforge.org"
+        remote_dir = "/var/www/gforge-projects/flvedit/"
+        local_dir = 'rdoc'
+
+        Rake::SshDirPublisher.new(host, remote_dir, local_dir).upload
+      end
+    end
+  end
+rescue LoadError
+  puts "Rake SshDirPublisher is unavailable or your rubyforge environment is not configured."
+end

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:major: 0
+:minor: 6
+:patch: 1

data/bin/flvedit

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+ 
+#--
+# Copyright (c) 2009 Marc-Andre Lafortune
+# Release under the (modified) BSD License (see gem's LICENSE file)
+#++
+ 
+begin
+  require 'flv/edit'
+rescue LoadError
+  require 'rubygems'
+  require 'flv/edit'
+end
+FLV::Edit::Runner.new(ARGV).run

data/lib/flv/audio.rb

@@ -0,0 +1,66 @@
+module FLV
+  # The body of an audio tag.
+  # The data is quite complex stuff. We make no attempt to understand it all
+  # or to be able to modify any of it. We simply consider it a complex string
+  # and read the interesting bits to give more info.
+  class Audio < String
+    include Body
+
+    FORMATS = Hash.new{|h, key| "Unknown audio format: #{key}"}.merge(
+      0  => :"Linear PCM, platform endian"  ,
+      1  => :ADPCM                          ,
+      2  => :MP3                            ,
+      3  => :"Linear PCM, little endian"    ,
+      4  => :"Nellymoser 16-kHz mono"       ,
+      5  => :"Nellymoser 8-kHz mono"        ,
+      6  => :Nellymoser                     ,
+      7  => :"G.711 A-law logarithmic PCM"  ,
+      8  => :"G.711 mu-law logarithmic PCM" ,
+      10 => :AAC                            ,
+      11 => :Speex                          ,
+      14 => :"MP3 8-kHz"                    ,
+      15 => :"Device-specific sound"
+    ).freeze
+    
+    EXCEPTIONS = Hash.new({}).merge(
+      :"Nellymoser 8-kHz mono"  => {:channel => :mono,  :rate => 8000},
+      :"Nellymoser 16-kHz mono" => {:channel => :mono,  :rate => 16000},
+      :AAC                      => {:channel => :stereo,:rate => 44000},
+      :"MP3 8-kHz"              => {:rate => 8000}
+    ).freeze
+    
+    CHANNELS = { 0 => :mono, 1 => :stereo}.freeze
+
+    def codec_id
+      read_bits(0..3)
+    end
+    
+    # Returns the format (see Audio::FORMATS for list)
+    def format
+      FORMATS[codec_id]
+    end
+    
+    # returns :mono or :stereo
+    def channel
+      EXCEPTIONS[format][:channel] ||
+        CHANNELS[read_bits(7)]
+    end
+
+    # Returns the sampling rate (in Hz)
+    def rate
+      EXCEPTIONS[format][:rate] ||
+        5500 << read_bits(4..5)
+    end
+    
+    # Returns the sample size (in bits)
+    def sample_size
+      EXCEPTIONS[format][:sample_size] ||
+        8 << read_bits(6)
+    end
+    
+    def is?(what)
+      format.to_s.downcase == what.to_s.downcase || super
+    end
+    
+  end
+end

data/lib/flv/base.rb

@@ -0,0 +1,38 @@
+module FLV
+  # Common fonctionality to both FLV::Header, FLV::Tag & FLV::Body
+  module Base
+    def self.included(base)
+      base.class_eval do
+        include Packable
+      end
+    end
+    
+    # returns the instance methods
+    # that are proper (i.e. not redefinitions)
+    # and that don't require any argument.
+    def getters(of=self)
+      of.class.ancestors.
+        map{|k| k.instance_methods(false)}.
+        inject{|proper, methods| proper -= methods}.  # tricky: first ancestor is of.class, so that's what we start with
+        select{|m| of.class.instance_method(m).arity.between?(-1,0)}
+    end
+    
+    def to_h(attributes = getters)
+      Hash[attributes.map do |a|
+        a = a.to_s.delete("@").to_sym
+        [a, send(a)]
+      end]
+    end
+    
+    def is?(what)
+      kn = self.class.name.downcase
+      [kn, kn.sub("flv::","")].include?(what.to_s.downcase)
+    end
+
+    def size
+      StringIO.new.packed.write(self)
+    end
+    
+  end
+  
+end

data/lib/flv/body.rb

@@ -0,0 +1,57 @@
+module FLV
+  
+  # Common fonctionality to all types of Bodies (FLV::Audio, FLV::Video & FLV::Event)
+  module Body
+    def self.included(base)
+      # Caution: order is important; InstanceMethods::is? relies on Base::is?
+      base.class_eval do
+        include Base
+        include InstanceMethods
+        include InstanceMethodsWhenString
+      end
+    end
+    
+    module InstanceMethods # :nodoc:
+      def debug(format, *) #:nodoc
+        format.values(to_h)
+      end
+
+      def is?(what)
+        case what
+          when String, Symbol
+            super(what.to_s.downcase.gsub!(/_tag$/, "") || :never_match_on_class_name_unless_string_ends_with_tag)
+          else
+            super
+        end
+      end
+
+      def similar_to?(other_body)
+        getters.each{|getter| return false unless send(getter) == other_body.send(getter)}
+        true
+      end
+    
+      def title
+        self.class.name + " tag"
+      end
+    end 
+    
+    module InstanceMethodsWhenString # :nodoc:
+      # Returns an +Integer+ computed from bits specified by +which+.
+      # The 0th bit is the most significant bit of the first character.
+      # +which+ can designate a range of bits or a single bit
+      def read_bits(which)
+        which = which..which if which.is_a? Integer
+        first_byte, last_byte = which.first >> 3, which.max >> 3
+        return (getbyte(first_byte) >> (7 & ~which.max)) & ~(~1 << which.max-which.first) if(first_byte == last_byte)
+        mid = last_byte << 3
+        read_bits(which.first...mid) << (which.max - mid + 1)  |  read_bits(mid..which.max)
+      end
+
+      # We need to redefine this, since we want to end up with a Body, not a String
+      def read_packed(io, options) #:nodoc:
+        replace(io.read(String, options))
+      end
+    end  
+  end
+  
+end

data/lib/flv/edit/options.rb

@@ -0,0 +1,162 @@
+# encoding: utf-8
+require 'optparse'
+
+module FLV
+  module Edit
+    class Options
+      attr_reader :commands, :options
+      def initialize(argv)
+        @commands, @options = parse(argv)
+      end
+      
+      def to_a
+        [@commands, @options]
+      end
+      
+    private
+      def parse(argv)
+        commands = []
+        options = {}
+        parser = OptionParser.new do |parser|
+          parser.banner = banner
+          parser.separator ""
+          parser.separator "Commands:"
+          Processor::Base.registry.sort_by(&:to_s).
+            each do |command, desc, cmd_opt|
+              name = command.name.split('::').last.downcase
+              option = name.downcase.to_sym
+              shortcut = cmd_opt[:shortcut] || name[0..0]
+              if param = cmd_opt[:param]
+                name << " " << (param[:name] || param[:class].to_s.upcase)
+                desc = [param[:class], *desc]  if param[:class]
+              end
+              parser.on("-#{shortcut}", "--#{name}", *desc) do |v|
+                options[option] = v
+                commands << command
+              end
+            end
+           
+          parser.separator ""
+          parser.separator "Switches:"
+          [
+            [:keyframe_mode,      "Keyframe mode slides on_cue_point(navigation) tags added by the",
+                                  "add command to nearest keyframe position"]
+          ].each do |switch, *desc|
+            shortcut = desc.first.is_a?(Symbol) ? desc.shift.to_s : switch.to_s[0..0]
+            full = desc.first.is_a?(Class) ? "--#{switch.to_s} N" : "--#{switch.to_s}"
+            parser.on("-#{shortcut}", full, *desc) { |v| options[switch] = v }
+          end
+              
+          parser.separator "Common:"
+          parser.on("-h", "--help", "Show this message") {}
+          parser.on("--version", "Show version") do
+            puts "Current version: #{FLV::Edit.version}"
+            exit
+          end
+          
+          parser.separator ""
+          parser.separator "flvedit (#{FLV::Edit.version}), copyright (c) 2009 Marc-André Lafortune"
+          parser.separator "This program is published under the BSD license."
+        end
+        options[:files] = parser.parse!(argv)
+        if commands.empty?
+          puts parser
+          exit
+        end
+        return commands, options
+      end
+      
+      def banner
+        <<-EOS
+Usage: flvedit#{(RUBY_PLATFORM =~ /win32/) ? '.exe' : ''} [--input] files --processing --more_processing ... [--save [PATH]]
+******
+*NOTE*: THIS IS NOT A STABLE RELEASE. API WILL CHANGE!
+******
+flvedit will apply the given processing to the input files. 
+
+Examples:
+  # Printout of the first 42 tags:
+    flvedit example.flv --debug 42
+
+  # Extract the first 2 seconds of a.flv and b.flv, join them and update the meta data.
+    flvedit a.flv b.flv --cut 0-2 --join --update --save out.flv
+
+Option formats:
+  TIMESTAMP:  Given in seconds with decimals for fractions of seconds.
+              Use 'h' and 'm' or ':' for hours and minutes separators.
+  Examples:   1:00:01 (for 1 hour and 1 second), 1h1 (same),
+              2m3.004 (2 minutes, 3 seconds and 4 milliseconds), 123.004 (same)
+  RANGE:      Expressed as BEGIN-END, where BEGIN or END are timestamps
+              that can be let empty for no limit
+  Examples:   1m-2m (second minute), 1m- (all but first minute), -1m (first minute)
+  RANGE/TS:   RANGE or TIMESTAMP.
+        EOS
+        #s.split("\n").collect!(&:strip).join("\n")
+      end
+    end
+  end
+end
+
+#todo: explicit Input command
+#todo: conditions for implicit Save
+  
+        # options[:metadatacreator] = "inlet media FLVTool2 v#{PROGRAMM_VERSION.join('.')} - http://www.inlet-media.de/flvtool2"
+        # options[:metadata] = {}
+
+          # when /^-([a-zA-Z0-9]+?):(.+)$/
+          #   options[:metadata][$1] = $2
+          # when /^-([a-zA-Z0-9]+?)#([0-9]+)$/
+          #   options[:metadata][$1] = $2.to_f
+          # when /^-([a-zA-Z0-9]+?)@(\d{4,})-(\d{2,})-(\d{2,}) (\d{2,}):(\d{2,}):(\d{2,})$/
+          #   options[:metadata][$1] = Time.local($2, $3, $4, $5, $6, $7)
+        #   when /^([^-].*)$/
+        #     if options[:in_path].nil? 
+        #       options[:in_path] = $1
+        #       if options[:in_path].downcase =~ /stdin|pipe/
+        #         options[:in_pipe] = true
+        #         options[:in_path] = 'pipe'
+        #       else
+        #         options[:in_path] = File.expand_path( options[:in_path] )
+        #       end
+        #     else
+        #       options[:out_path] = $1
+        #       if options[:out_path].downcase =~ /stdout|pipe/
+        #         options[:out_pipe] = true
+        #         options[:out_path] = 'pipe'
+        #       else
+        #         options[:out_path] = File.expand_path( options[:out_path] )
+        #       end
+        #     end
+        #   end
+        # end
+    
+  
+      # def self.validate_options( options )
+      #    if options[:commands].empty?
+      #      show_usage
+      #      exit 0
+      #    end
+      #    
+      #    options[:commands].each do |command|  
+      #      case command
+      #      when :print
+      #        if options[:out_pipe]
+      #          throw_error "Could not use print command in conjunction with output piping or redirection"
+      #          exit 1
+      #        end
+      #      when :debug
+      #        if options[:out_pipe]
+      #          throw_error "Could not use debug command in conjunction with output piping or redirection"
+      #          exit 1
+      #        end
+      #      when :help
+      #        show_usage
+      #        exit 0
+      #      when :version
+      #        show_version
+      #        exit 0
+      #      end
+      #    end
+      #    options
+      #  end
+      #  

data/lib/flv/edit/processor/add.rb

@@ -0,0 +1,67 @@
+require 'yaml'
+require 'rexml/document'
+require 'optparse'
+
+module FLV
+  module Edit
+    module Processor
+
+      class Add < Base
+        desc "Adds tags from the xml or yaml file PATH (default: tags.yaml/xml)", :param => {:name => "[PATH]"}
+        
+        def initialize(*)
+          super
+          @add = (options[:add_tags] || read_tag_file).sort_by(&:timestamp)
+        end
+        
+        def on_tag(tag)
+          insert_before tag, @add
+        end
+        
+        # def on_keyframe(tag)
+        #   insert_before tag, @add_before_key_frame
+        # end
+        
+      private
+        def insert_before(tag, list_to_insert)
+          stop_at = list_to_insert.find_index{|i| i.timestamp > tag.timestamp} || list_to_insert.length
+          dispatch_instead(*(list_to_insert.slice!(0...stop_at) << tag))   if stop_at > 0
+        end
+      
+        DEFAULT = ["tags.yaml", "tags.xml"]
+        #todo: overwrite, navigation
+        #todo: default event
+        def read_tag_file
+          filename = options[:add]
+          DEFAULT.each{|fn| filename ||= fn if ::File.exists?(fn)}
+          raise "You must either specify a tag file or have a file named #{DEFAULT.join(' or ')}" unless filename
+          ::File.open(filename) {|f| read_tags(f)}
+        end
+        
+        def read_tags(file)
+          tags =
+            unless file.path.downcase.end_with? ".xml"
+              YAML.load(file)
+            else
+              xml = REXML::Document.new(file, :ignore_whitespace_nodes => :all) 
+              xml.root.elements.to_a("metatag").map do |tag|
+                {
+                  :event => tag.attributes['event'],
+                  :overwrite => tag.attributes['overwrite'],
+                  :timestamp => (tag.elements['timestamp'].text rescue 0),
+                  :navigation => (tag.elements['type'].text == "navigation" rescue false),
+                  :arguments => Hash[tag.elements['parameters'].map{|param| [param.name, param.text]}]
+                }
+              end
+            end
+          tags = [tags] unless tags.is_a? Array
+          tags.map do |info|
+            info.symbolize_keys!
+            Tag.new(info[:timestamp], Event.new(info[:event], info[:arguments]))
+          end
+        end
+      end
+
+    end
+  end
+end