abrizer 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b31f79c3371e0fed7e639734c2a5480274946c14
-  data.tar.gz: aad270f032d907ac4a949d0b62ac5a8571bf7db8
+  metadata.gz: fc9378a163fb93bef80e0850ac0133674ac9106f
+  data.tar.gz: 633f89376f94e84237122b36a4aba365d34ef3db
 SHA512:
-  metadata.gz: 862184366c07a709617e33fe30ffa0dc1e1337b42813a862797620a624182210473870ef94094b0ab7e466d4a8d1c76f501b44507656a41e3b56af927ca6886a
-  data.tar.gz: b64b33fd21d00ffb2e63634eda4e9351f3e9b4fbbccb096213dce957b776fb45ac6e3560738e6422ad40f5f11a2bce8cdb16a9d279cf1065d94e8285f2fe2a24
+  metadata.gz: 5a9aff673d64f3ae930cdffc748aed9a844f82414dd1fd12bb93ddbcc0ae9624096c5a869a5a930ad75f90481aa34e12a8f3b12e0662bb68e6751fa6dbba5891
+  data.tar.gz: 3ef0a0a10a9b21f7645fb9f869f6df5b3a34aff48a87f02928077eb29d0259226fa7860d5c69b9acb41bdfab7382d51b01ed9075853587c4830621c380de2881

data/README.md

@@ -1,6 +1,6 @@
 # Abrizer
 
-Abrizer takes a source video and creates various derivatives for delivery including adaptive bitrate formats like DASH and HLS.
+Abrizer takes a source video and creates various derivatives for delivery including adaptive bitrate formats like DASH and HLS. An opinionated work in progress.
 
 ## Requirements
 
@@ -26,6 +26,8 @@ Or install it yourself as:
 
     $ gem install abrizer
 
+See [Vagrant](#vagrant) below for one way to install dependencies and run the scripts.
+
 ## Usage
 
 Abrizer knows how to run various processes which can take a master or mezzanine video and create DASH and HLS streaming formats along with other derivatives like a fallback MP4 and WebM. The gem is opinionated about what formats to create and what settings to use. The intention is to provide a relatively complete but simple solution for delivering video over HTTP.
@@ -38,9 +40,10 @@ Some steps must be run after others as they have preconditions in order for subs
 - `mp4`: Process a progressive download MP4 from original
 - `vp9`: Process a progressive download WebM VP9 from original
 - `sprites`: Create video sprites and metadata WebVTT file retaining all the images in order for a human to later pick a poster image
+- `poster`: Copies over a temporary poster image from the output of the sprites
 - `clean`: Clean out the intermediate and log files including MP4 files used for packaging but not required for delivery
 
-Every command requires the path to the original video file and an output directory.
+All of the above commands require the path to the original video file and an output directory.
 
 ### Command Line
 
@@ -76,12 +79,29 @@ Abrizer::All.new(video_path, output_directory).run
 
 Take a look at `lib/abrizer/all.rb` for how to use the various classes provided. You can also see more examples in `lib/abrizer/cli.rb`. All classes expect to be passed the fully expanded path.
 
+### Canvas
+
+Experimental support is provided for creating a IIIF Canvas. Note that this work is done in advance of a draft standard for including video in a IIIF manifest. It can be created after other steps with `abrizer help canvas` for more information.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
+## Vagrant
+
+The easiest way to do development on Abrizer or even try out the scripts is to use a Vagrant machine. If you have Vagrant and Virtualbox installed just run `vagrant up` and all dependencies will be installed.
+
+This includes a web server that can be used for local testing of streams and videos. After `vagrant up` visit http://localhost:8088/v/ to see the contents of the project's `tmp` directory. Within the virtual machine (`vagrant ssh`) you can process a test video with:
+
+```shell
+cd /vagrant
+bundle exec exe/abrizer all test/videos/FullHDCinemaCountdown720p-8sec.mp4 tmp/countdown http://localhost:8088/v
+```
+
+Now visit http://localhost:8088/v/countdown/ to see the files that were created. You can then test any of the videos or streams.
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/jronallo/abrizer.

data/Vagrantfile

@@ -5,6 +5,8 @@ Vagrant.configure("2") do |config|
   config.vm.box = "boxcutter/centos72"
 
   config.vm.synced_folder '.', '/vagrant' #, type: 'nfs', mount_options: ['nolock', 'rw', 'vers=3', 'tcp', 'actimeo=2']
+  config.vm.network "forwarded_port", guest: 80, host: 8088,
+      auto_correct: true
 
   config.vm.provider "virtualbox" do |vb|
     vb.linked_clone = true

data/abrizer.gemspec

@@ -26,6 +26,7 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "bundler-audit"
 
   spec.add_dependency "thor"
-  spec.add_dependency 'json'
   spec.add_dependency 'video_sprites', '0.2.0'
+  spec.add_dependency 'jbuilder'
+  spec.add_dependency 'yajl-ruby'
 end

data/ansible/development-playbook.yml

@@ -7,6 +7,7 @@
   gather_facts: yes
   roles:
     - role: basic-setup
+    - role: apache
     - role: franklinkim.environment
       environment_config:
         PATH: /usr/local/bento4/bin:/usr/bin:/usr/local/bin

data/ansible/roles/apache/tasks/main.yml

@@ -0,0 +1,23 @@
+---
+- name: install apache
+  yum:
+    name: httpd
+    state: present
+- name: enable cors
+  lineinfile:
+    dest: /etc/httpd/conf/httpd.conf
+    line: 'Header set Access-Control-Allow-Origin "*"'
+    # regexp: '<Directory "/var/www/html">'
+    insertafter: '<Directory "/var/www/html">'
+- name: remove firewall
+  yum:
+    name: firewalld
+    state: removed
+- name: symlink /var/www/html/v to /vagrant/tmp
+  file:
+    path: /var/www/html/v
+    src: /vagrant/tmp
+    state: link
+    force: yes
+- name: restart httpd
+  service: name=httpd state=restarted

data/ansible/roles/basic-setup/tasks/main.yml

@@ -37,3 +37,4 @@
     - perl-devel
     - zlib-devel
     - tree
+    - jq

data/lib/abrizer.rb

@@ -1,4 +1,8 @@
-require 'json'
+require 'yajl'
+require 'multi_json'
+MultiJson.use :yajl
+MultiJson.dump_options = {pretty: true}
+require 'jbuilder'
 require 'video_sprites'
 require "abrizer/version"
 require 'abrizer/filepath_helpers'
@@ -15,8 +19,13 @@ require 'abrizer/cleaner'
 require 'abrizer/progressive_mp4'
 require 'abrizer/progressive_vp9'
 require 'abrizer/sprites'
+require 'abrizer/captions'
+require 'abrizer/canvas'
+require 'abrizer/temporary_poster'
 require 'abrizer/all'
 
+
+
 module Abrizer
   # Your code goes here...
 end

data/lib/abrizer/all.rb

@@ -1,9 +1,10 @@
 module Abrizer
   class All
 
-    def initialize(filename, output_dir=nil)
+    def initialize(filename, output_dir, base_url)
       @filename = filename
       @output_directory = output_dir
+      @base_url = base_url
     end
 
     def run
@@ -12,7 +13,10 @@ module Abrizer
       Abrizer::ProgressiveVp9.new(@filename, @output_directory).create
       Abrizer::PackageDashBento.new(@filename, @output_directory).package
       Abrizer::PackageHlsBento.new(@filename, @output_directory).package
+      Abrizer::Captions.new(@filename, @output_directory).copy
       Abrizer::Sprites.new(@filename, @output_directory).create
+      Abrizer::TemporaryPoster.new(@output_directory).copy
+      Abrizer::Canvas.new(@filename, @output_directory, @base_url).create
       Abrizer::Cleaner.new(@filename, @output_directory).clean
     end
 

data/lib/abrizer/canvas.rb

@@ -0,0 +1,216 @@
+module Abrizer
+  # Creates a IIIF Canvas JSON-LD document.
+  class Canvas
+
+    include FilepathHelpers
+
+    # TODO: allow control of items/versions listed on canvas
+    def initialize(filepath, output_directory, base_url)
+      @filename = filepath
+      @output_directory = output_directory
+      @base_url = base_url
+      finder = AdaptationFinder.new(@filename)
+      @adaptations = finder.adaptations
+    end
+
+    def create
+      File.open(canvas_filepath, 'w') do |fh|
+        fh.puts create_json
+      end
+    end
+
+    def create_json
+      Jbuilder.encode do |json|
+        json.set! '@context', 'http://iiif.io/api/presentation/3/context.json'
+        json.id canvas_id
+        json.type "Canvas"
+        json.width max_width
+        json.height max_height
+        json.duration duration
+        thumbnail_json(json)
+        media_json(json)
+      end
+    end
+
+    def thumbnail_json(json)
+      if File.exist? poster_image_filepath
+        json.thumbnail do
+          json.id poster_id
+          json.type 'Image'
+          json.format 'image/jpeg'
+        end
+      end
+    end
+
+    def media_json(json)
+      json.media do
+        json.child! do
+          json.type "Annotation"
+          json.motivation 'painting'
+          json.target canvas_id
+          json.body do
+            json.type "Choice"
+            json.items do
+              mpd_item(json)
+              hlsts_item(json)
+              vp9_item(json)
+              mp4_item(json)
+            end
+            json.seeAlso do
+              # TODO: Allow for adding more than one captions/subtitle file
+              captions_seealso(json)
+              sprites_seealso(json)
+            end
+          end
+        end
+      end
+    end
+
+    def mpd_item(json)
+      if File.exist? mpd_filepath
+        json.child! do
+          json.id mpd_id
+          json.type "Video"
+          json.format "application/dash+xml"
+          json.width max_width
+          json.height max_height
+        end
+      end
+    end
+
+    def hlsts_item(json)
+      if File.exist? hlsts_filepath
+        json.child! do
+          json.id hlsts_id
+          json.type "Video"
+          # TODO: or "vnd.apple.mpegURL"
+          json.format "application/x-mpegURL"
+          json.width max_width
+          json.height max_height
+        end
+      end
+    end
+
+    def vp9_item(json)
+      if File.exist? vp9_filepath
+        json.child! do
+          json.id vp9_id
+          json.type "Video"
+          #TODO: add webm codecs
+          json.format "video/webm"
+          json.width max_width
+          json.height max_height
+        end
+      end
+    end
+
+    def mp4_item(json)
+      if File.exist? mp4_filepath
+        json.child! do
+          json.id mp4_id
+          json.type "Video"
+          #TODO: add mp4 codecs
+          json.format "video/mp4"
+          json.width mp4_width
+          json.height mp4_height
+        end
+      end
+    end
+
+    def captions_seealso(json)
+      # TODO: update captions seeAlso for multiple captions
+      captions_file = File.join output_directory, 'vtt/captions.vtt'
+      if File.exist? captions_file
+        json.child! do
+          json.id vtt_id
+          json.format 'application/webvtt'
+          json.label 'English captions'
+          json.language 'en'
+          json._comments "How make explicit how whether to use these as captions or subtitles?"
+        end
+      end
+    end
+
+    def sprites_seealso(json)
+      if File.exist? sprites_filepath
+        json.child! do
+          json.id sprites_id
+          json.format 'application/webvtt'
+          json.label 'image sprite metadata'
+          json._comments "How to include resources like video image sprites like those created by https://github.com/jronallo/video_sprites and used by various players?"
+        end
+      end
+    end
+
+    def media_base_url
+      File.join @base_url, output_directory_basename
+    end
+
+    def canvas_id
+      File.join media_base_url, canvas_partial_filepath
+    end
+
+    def poster_id
+      File.join media_base_url, poster_partial_filepath
+    end
+
+    def mpd_id
+      File.join media_base_url, mpd_partial_filepath
+    end
+
+    def hlsts_id
+      File.join media_base_url, hlsts_partial_filepath
+    end
+
+    def mp4_id
+      File.join media_base_url, mp4_partial_filepath
+    end
+
+    def vp9_id
+      File.join media_base_url, vp9_partial_filepath
+    end
+
+    def vtt_id
+      File.join media_base_url, 'vtt/captions.vtt'
+    end
+
+    def sprites_id
+      File.join media_base_url, sprites_partial_filepath
+    end
+
+    def duration
+      informer = Abrizer::FfprobeInformer.new(mp4_filename)
+      informer.duration.to_f
+    end
+
+    def max_width
+      @adaptations.last.width
+    end
+
+    def min_width
+      @adaptations.first.width
+    end
+
+    def max_height
+      @adaptations.last.height
+    end
+
+    def min_height
+      @adaptations.first.height
+    end
+
+    def mp4_width
+      @adaptations[-2].width
+    end
+
+    def mp4_height
+      @adaptations[-2].height
+    end
+
+    # TODO: DRY up with progressive_mp4.rb
+    def mp4_filename
+      File.join output_directory, "progressive.mp4"
+    end
+
+  end
+end

data/lib/abrizer/captions.rb

@@ -0,0 +1,41 @@
+module Abrizer
+  # Copies over any WebVTT captions that are beside the original video resource
+  # into the destination folder.
+  # TODO: This may only be needed if fMP4 derivatives aren't created since
+  # captions are copied over.
+  # TODO: Allow for more than one captions/subtitle file to be copied over.
+  class Captions
+
+    include FilepathHelpers
+
+    def initialize(filename, output_dir)
+      @filename = filename
+      @output_directory = output_dir
+      if vtt_dir_glob.length > 0
+        FileUtils.mkdir_p vtt_output_directory
+      end
+    end
+
+    def copy
+      vtt_dir_glob.each do |vtt|
+        # vtt_basename = File.basename vtt
+        vtt_filename = File.join vtt_output_directory, 'captions.vtt'
+        FileUtils.cp vtt, vtt_filename
+      end
+    end
+
+    def vtt_dir_glob
+      Dir.glob vtt_file_glob
+    end
+
+    # TODO: actually search for more than one VTT file
+    def vtt_file_glob
+      File.join filename_directory, "#{basename}.vtt"
+    end
+
+    def vtt_output_directory
+      File.join @output_directory, 'vtt'
+    end
+
+  end
+end

data/lib/abrizer/cli.rb

@@ -2,11 +2,11 @@ require 'thor'
 module Abrizer
   class CLI < Thor
 
-    desc 'all <filepath> <output_directory>', 'Runn all processes including creating ABR streams, progressive download versions, and images and video sprites'
-    def all(filepath, output_dir=nil)
+    desc 'all <filepath> <output_directory> <base_url>', 'Run all processes including creating ABR streams, progressive download versions, and images and video sprites.'
+    def all(filepath, output_dir, base_url)
       filepath = File.expand_path filepath
       output_dir = File.expand_path output_dir
-      Abrizer::All.new(filepath, output_dir).run
+      Abrizer::All.new(filepath, output_dir, base_url).run
     end
 
     desc 'abr <filepath> <output_directory>', 'From file create ABR streams, includes processing MP4 adaptations for packaging'
@@ -76,6 +76,26 @@ module Abrizer
       Abrizer::Sprites.new(filepath, output_dir).create
     end
 
+    desc 'poster <output_directory>', 'Copy over a temporary poster image based on the sprite images'
+    def poster(output_dir=nil)
+      output_dir = File.expand_path output_dir
+      Abrizer::TemporaryPoster.new(output_dir).copy
+    end
+
+    desc 'captions <filepath> <output_directory>', 'Captions and subtitles files with the same basename as the video file and with a .vtt extension are copied over into the output directory'
+    def captions(filepath, output_dir=nil)
+      filepath = File.expand_path filepath
+      output_dir = File.expand_path output_dir
+      Abrizer::Captions.new(filepath, output_dir).copy
+    end
+
+    desc 'canvas <filepath> <output_directory> <base_url>', 'Creates a IIIF Canvas JSON-LD document as an API into the resources'
+    def canvas(filepath, output_directory, base_url)
+      filepath = File.expand_path filepath
+      output_directory = File.expand_path output_directory
+      Abrizer::Canvas.new(filepath, output_directory, base_url).create
+    end
+
     desc 'clean <filepath> <output_directory>', 'Clean up intermediary files'
     def clean(filepath, output_dir=nil)
       Abrizer::Cleaner.new(filepath, output_dir).clean

data/lib/abrizer/ffprobe_informer.rb

@@ -8,7 +8,7 @@ module Abrizer
 
     def get_info
       @json_result = `#{ffmpeg_info_cmd}`
-      @info = JSON.parse @json_result
+      @info = MultiJson.load @json_result
     end
 
     def width
@@ -19,6 +19,10 @@ module Abrizer
       video_stream['height'] if video_stream
     end
 
+    def duration
+      video_stream['duration'] if video_stream
+    end
+
     def display_aspect_ratio #dar
       dar = video_stream['display_aspect_ratio']
       sar = video_stream['sample_aspect_ratio']

data/lib/abrizer/filepath_helpers.rb

@@ -20,6 +20,10 @@ module Abrizer
       end
     end
 
+    def output_directory_basename
+      File.basename output_directory
+    end
+
     def filename_directory
       File.dirname @filename
     end
@@ -29,5 +33,66 @@ module Abrizer
       extname = File.extname @filename
       File.basename @filename, extname
     end
+
+    def poster_partial_filepath
+      'poster.jpg'
+    end
+
+    def mpd_partial_filepath
+      'fmp4/stream.mpd'
+    end
+
+    def mpd_filepath
+      File.join output_directory, mpd_partial_filepath
+    end
+
+    def hlsts_partial_filepath
+      'hls/master.m3u8'
+    end
+
+    def hlsts_filepath
+      File.join output_directory, hlsts_partial_filepath
+    end
+
+    def mp4_partial_filepath
+      'progressive.mp4'
+    end
+
+    def mp4_filepath
+      File.join output_directory, mp4_partial_filepath
+    end
+
+    def vp9_partial_filepath
+      'progressive-vp9.webm'
+    end
+
+    def vp9_filepath
+      File.join output_directory, vp9_partial_filepath
+    end
+
+    def canvas_partial_filepath
+      'canvas.json'
+    end
+
+    def canvas_filepath
+      File.join output_directory, canvas_partial_filepath
+    end
+
+    def sprites_partial_filepath
+      'sprites/sprites.vtt'
+    end
+
+    def sprites_filepath
+      File.join output_directory, sprites_partial_filepath
+    end
+
+    def first_image_filepath
+      File.join output_directory, 'sprites/images/img-00001.jpg'
+    end
+
+    def poster_image_filepath
+      File.join output_directory, 'poster.jpg'
+    end
+
   end
 end

data/lib/abrizer/temporary_poster.rb

@@ -0,0 +1,20 @@
+module Abrizer
+  # It is usually better for a human to select a poster image, but we try to
+  # select a temporary one.
+  # Creating sprites leaves some images around so that is a prerequisite for
+  # this step.
+  # TODO: selection of a temporary poster image could be improved
+  class TemporaryPoster
+
+    include FilepathHelpers
+
+    def initialize(output_dir)
+      @output_directory = output_dir
+    end
+
+    def copy
+      FileUtils.cp first_image_filepath, poster_image_filepath
+    end
+
+  end
+end

data/lib/abrizer/version.rb

@@ -1,3 +1,3 @@
 module Abrizer
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: abrizer
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Jason Ronallo
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-01-29 00:00:00.000000000 Z
+date: 2017-02-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -95,7 +95,21 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: json
+  name: video_sprites
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.2.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.2.0
+- !ruby/object:Gem::Dependency
+  name: jbuilder
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -109,19 +123,19 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: video_sprites
+  name: yajl-ruby
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '='
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 0.2.0
+        version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '='
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 0.2.0
+        version: '0'
 description: Creates adaptive bitrate streams and other delivery derivatives.
 email:
 - jronallo@gmail.com
@@ -141,6 +155,7 @@ files:
 - ansible/development-playbook.retry
 - ansible/development-playbook.yml
 - ansible/development.ini
+- ansible/roles/apache/tasks/main.yml
 - ansible/roles/basic-setup/tasks/main.yml
 - ansible/roles/bento4/tasks/main.yml
 - ansible/roles/ffmpeg/defaults/main.yml
@@ -191,6 +206,8 @@ files:
 - lib/abrizer/adaptation.rb
 - lib/abrizer/adaptation_finder.rb
 - lib/abrizer/all.rb
+- lib/abrizer/canvas.rb
+- lib/abrizer/captions.rb
 - lib/abrizer/cleaner.rb
 - lib/abrizer/cli.rb
 - lib/abrizer/ffmpeg_processor.rb
@@ -204,6 +221,7 @@ files:
 - lib/abrizer/progressive_mp4.rb
 - lib/abrizer/progressive_vp9.rb
 - lib/abrizer/sprites.rb
+- lib/abrizer/temporary_poster.rb
 - lib/abrizer/version.rb
 homepage: https://github.com/jronallo/abrizer
 licenses:
@@ -225,7 +243,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.2
+rubygems_version: 2.6.6
 signing_key: 
 specification_version: 4
 summary: Creates MPEG-DASH and HLS streams from a video file.