video_transcoding 0.21.2 → 0.22.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 89811ccde1116e1e2eb8005841225587b2e9ce21dd2407d8164b081ed4da7694
-  data.tar.gz: 652fde3679a6176b5cb587c76a44a095bda1721de984d5f1adaff9f5f6c5685f
+  metadata.gz: 2ec7d9f2674f60b914a959df7c76debc47249ae57b925606b1d2ca86937a6f35
+  data.tar.gz: 5db202054bf388c995005cd9752ae6203551e2aec5680acbdb3a887b5d1d38ed
 SHA512:
-  metadata.gz: 33621c9e885597c9dd0dcfe6662e17fd6e5700a3c5069c3700e4a67b5f473a0d76fec00308fe30790126e5789c98796d8b213174d6ea2af63a3a1be163c0197d
-  data.tar.gz: 2033fb19aeaf1f169dbd841f4a80497b23bdaaac751d8e54b75fdb2082ad87e3d7b4112ab66ce3e2ebc7a8368b4e2abdd77e4c1b0b6483d9e4a060983b338bbc
+  metadata.gz: 27dc2c42fee463759a08d50bafe115d43b42a904cb89ff9e9cd0fe59bb8be14f98bbd3f6cd23a08b7b40952191ee5c1f6f5d47c539f9fd178f929203a76b35c6
+  data.tar.gz: 548d0b737f9d976daf3a5e4f710837b0c2a807cead897b0d2c7ee7eed2509fdd1ca0044595c1b41fa29d90ecfaaf2f43d71503ae541f6b7df2074169eca62ca9

data/README.md

@@ -6,7 +6,7 @@ Tools to transcode, inspect and convert videos.
 
 Hi, I'm [Don Melton](http://donmelton.com/). I created these tools to transcode my collection of Blu-ray Discs and DVDs into a smaller, more portable format while remaining high enough quality to be mistaken for the originals.
 
-What makes these tools unique are the [two ratecontrol systems](#explanation) which achieve those goals.
+What makes these tools unique are the [ratecontrol systems](#explanation) which achieve those goals.
 
 This package is based on my original collection of [Video Transcoding Scripts](https://github.com/donmelton/video-transcoding-scripts) written in Bash. While still available online, those scripts are no longer in active development. Users are encouraged to install this Ruby Gem instead.
 
@@ -559,83 +559,49 @@ These examples are written in Bash and only supply crop values. But almost any s
 
 ## Explanation
 
-### A tale of two ratecontrol systems
+### Ratecontrol systems
 
-What is a ratecontrol sytem? It's how a video encoder decides on the amount of bits to allocate for a specific frame.
+What is a ratecontrol system? It's how a video encoder decides on the amount of bits to allocate for a specific frame.
 
-My `transcode-video` tool has two different ratecontrol systems available to control the size and quality of output video. The special, or default, ratecontrol system is designed to deliver consistent quality, while the average bitrate (ABR) ratecontrol system, enabled via the `--abr` option, is designed to produce a predictable output size.
+My `transcode-video` tool has three different ratecontrol systems which manipulate the x264 and x265 software-based video encoders built into `HandBrakeCLI`.
 
-Both ratecontrol systems are modified versions of what is commonly called a constrained variable bitrate (CVBR) mode. Which means they both allow bitrate to vary per frame but still constrain that bitrate.
+Additionally, `transcode-video` allows access to hardware-based video encoders which have their own ratecontrol systems.
 
-My special ratecontrol system leverages the constant quality ratecontrol system already within the x264 video encoder, an algorithm which uses a constant ratefactor (CRF) to target a specific quality instead of a bitrate.
+All of these ratecontrol systems, mine and those built into hardware, target a specific video bitrate.
 
-My average bitrate (ABR) ratecontrol system modifies the ABR algorithm already within x264 which targets a specific bitrate, constraining it to produce better overall quality.
+The target video bitrate for all of these systems is automatically determined by `transcode-video` using the resolution of the input. For example, the default target for 1080p output is `6000` Kbps, which is about one-fifth the video bitrate found on a typical Blu-ray Disc.
 
-The target video bitrate for both systems is automatically determined by `transcode-video` using the resolution of the input. For example, the default target for 1080p output is `6000` Kbps, which is about one-fifth the video bitrate found on a typical Blu-ray Disc.
+### How my simple and special ratecontrol systems work
 
-While both systems deliver high quality, they sometimes have different visual characteristics.
+My simple and special ratecontrol systems attempt to produce the highest possible video quality near a target bitrate using a constant ratefactor (CRF) to specify quality. A CRF is represented by a number from `0` to `51` with lower values indicating higher quality. The special value of `0` is for lossless output.
 
-### How my special ratecontrol system works
-
-When using `transcode-video`, you might notice two lines in the console output containing something like this:
-
-```
-options: vbv-maxrate=6000:vbv-bufsize=12000:crf-max=25:qpmax=34
-
-quality: 1.00 (RF)
-```
-
-These are actually the settings used by my special ratecontrol system to configure the x264 video encoder within HandBrake.
-
-This system attempts to produce the highest possible video quality near a target bitrate using a constant ratefactor (CRF) to specify quality. A CRF is represented by a number from `0` to `51` with lower values indicating higher quality. The special value of `0` is for lossless output.
-
-Unfortunately, the output bitrate is extremely unpredictable when using the x264's default CRF-based system. Typically, people pick a middle-level CRF value as their quality target and just hope for the best. This is what most of the presets built into HandBrake do, choosing a CRF of `20` or `22`.
+Unfortunately, the output bitrate is extremely unpredictable when using the default CRF-based system in x264 or x265. Typically, people pick a middle-level CRF value as their quality target and just hope for the best. This is what most of the presets built into HandBrake do, choosing a CRF of `20` or `22`.
 
 But such a strategy can result in output larger than its input or, worse, output too low in quality to be mistaken for that input.
 
-So I set the target CRF value to `1`, the best possible "lossy" quality. Normally this would produce a huge output bitrate but I also manipulate the video buffering verifier (VBV) model within x264 to constrain that bitrate.
+So I set the target CRF value to `1`, the best possible "lossy" quality. Normally this would produce a huge output bitrate but I also reduce the encoder's maximum bitrate to my target, e.g. `6000` Kbps for 1080p output, by manipulating an option called `vbv-maxrate`.
 
-Typically, the VBV model limits the output bitrate to a generous `25000` Kbps for video playback on devices like the Apple TV or Roku. But I reduce the VBV maximum bitrate (`vbv-maxrate`) to my target, e.g. `6000` Kbps for 1080p output.
+With this approach, the encoder chooses the lowest CRF value, and therefore the highest quality, which fits below that maximum bitrate ceiling, even if that's usually not a a CRF value of `1`.
 
-With this approach, x264 chooses the lowest CRF value, and therefore the highest quality, which fits below that ceiling, even if that's usually not a a CRF value of `1`.
+And this fully describes the behavior of `transcode-video` when using the `--simple` option.
 
-But manipulating only CRF and `vbv-maxrate` will not produce high enough quality output in some cases. Why? Sometimes you need a much higher bitrate for complex or difficult to encode passages than what is allowed by `vbv-maxrate`.
+However, my special, or default, ratecontrol system also sets a maximum CRF (`crf-max`) value of `25`, raising the minimum quality. This allows `vbv-maxrate` to become a "soft" ceiling so that the output bitrate can exceed the target when necessary to maintain that quality.
 
-Along with the target CRF value of `1`, I set a maximum CRF (`crf-max`) value of `25`, raising the minimum quality. This allows `vbv-maxrate` to become a "soft" ceiling so that the output bitrate can exceed the target when necessary to maintain that quality.
+Unfortunately, this internal tug of war can cause the encoder to sometimes generate a few very low quality frames. 
 
-But just adding a maximum CRF value is not enough. When under pressure to fit within all these constraints, x264 will sometimes generate a single, but still noticeable, very low quality frame. Why? Even though `crf-max` is set to `25`, individual frames can still use a higher quantizer value (QP) of much less quality.
-
-As part of the encoding process, x264 calculates a quantizer value (QP) for each macroblock within a frame of video. A QP is represented by a number from `0` to `69` with lower values indicating higher quality.
+As part of the encoding process, a quantizer value (QP) is calculated for each macroblock within a frame of video. A QP is represented by a number from `0` to `69` with lower values indicating higher quality.
 
 So I set a maximum quantizer (`qpmax`) value of `34`, again raising the minimum quality. The occasional bad frame is still there, but it's no longer noticeable because it's now of sufficient quality to blend in with the others.
 
-There's a final change required for the VBV model. I need to set the VBV buffer size (`vbv-bufsize`) so that my previous adjustment of `vbv-maxrate` is honored by x264. Otherwise the encoder will just ignore the VBV.
-
-It's safe to set `vbv-bufsize` anywhere in the range from one half to twice that of `vbv-maxrate`. However, that larger `vbv-bufsize` value produces an output bitrate closest to, on average, that of the target. So, if `vbv-maxrate` is `6000` Kbps, then I set `vbv-bufsize` to `12000` Kbps.
-
 ### How my average bitrate (ABR) ratecontrol system works
 
-When using `transcode-video` with the `--abr` option, you might notice two lines in the console output containing something like this:
-
-```
-options: vbv-maxrate=9000:vbv-bufsize=12000:nal-hrd=vbr
-
-bitrate: 6000 kbps, pass: 0
-```
+My average bitrate (ABR) ratecontrol system, selected via the `--abr` option, is based on the ABR algorithm already within x264 and x265 which targets a specific bitrate. 
 
-This ABR ratecontrol system attempts to produce a predictable output size while still maintaining high quality by manipulating the video buffering verifier (VBV) model within the x264 video encoder.
-
-As mentioned before, the VBV model typically allows bitrates to peak as high as `25000` Kbps during playback on most devices. But I constrain the VBV maximum bitrate (`vbv-maxrate`) to only 1.5 times that of the target, i.e. to just `9000` Kbps when the target bitrate is `6000` Kbps for 1080p output.
+But I constrain the maximum bitrate (`vbv-maxrate`) to only 1.5 times that of the target, i.e. to just `9000` Kbps when the target bitrate is `6000` Kbps for 1080p output.
 
 It seems counterintuitive, but constraining the maximum bitrate prevents too much bitrate being wasted on complex or difficult to encode passages at the expense of quality elsewhere. This is because with an average bitrate algorithm, when the peaks get too high then the valleys get too low.
 
-As with the default ratecontrol system, I need to set the VBV buffer size (`vbv-bufsize`) so that my previous adjustment of `vbv-maxrate` won't be ignored by x264. So, if `vbv-maxrate` is `9000` Kbps, then I set `vbv-bufsize` to `12000` Kbps.
-
-This VBV model manipulation is exactly the same strategy used by streaming services such as Netflix.
-
-The final setting, `nal-hrd=vbr`, doesn't actually affect ratecontrol. This is a x264 option signaling Hypothetical Reference Decoder (HRD) information, meaning that it adds the VBV maximum bitrate value as metadata to the output video. Which is useful for certain streaming environments and media tools.
-
-And this information is safe to include since my ABR ratecontrol implementation will, by design, never exceed the maximum bitrate. Which is something the default ratecontrol system cannot promise.
+And this manipulation is exactly the same strategy used by streaming services such as Netflix.
 
 ## FAQ
 
@@ -651,23 +617,25 @@ Plus, I wouldn't use a GUI for these tasks. And it's a bad idea to develop softw
 
 ### When will you add support for H.265 video?
 
-[High Efficiency Video Coding](https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding) or H.265  is the likely successor to [H.264](https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC), which is the format currently output by `transcode-video`. HandBrake has supported H.265 ever since it included the [x265 video encoder](http://x265.org/).
+HandBrake has supported [High Efficiency Video Coding](https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding) or H.265 ever since it included the [x265 video encoder](http://x265.org/).
+
+You can try HEVC transcoding now using the `--encoder` option:
 
-My ratecontrol system couldn't be applied with the x265 encoder in older versions of HandBrake because it didn't allow access to `qpmax`, critical for maintaining quality in certain situations. The latest version of HandBrake includes a new encoder which fixes that.
+    transcode-video --encoder x265 "/path/to/Movie.mkv"
 
-But support also requires equivalent quality at a smaller size when using my rate control system. And performance is an issue. While speed continues to improve, the x265 encoder is still considerably slower than the current H.264 system.
+While speed continues to improve, x265 is still considerably slower than the default x264 encoder.
 
-You can try _experimental_ HEVC transcoding now with an advanced option:
+### What about hardware-based video transcoding?
 
-    transcode-video --handbrake-option encoder=x265 "/path/to/Movie.mkv"
+Hardware-based encoders, like those in [Intel Quick Sync Video](https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video), are often considerably faster than x264 or x265. Some are available in recent versions of HandBrake.
 
-But use this _only_ with `HandBrakeCLI` version 1.0.0 or later.
+Check the output of `HandBrakeCLI --help` to find out if your platform has any of these video encoders available. The names of these encoders all end with "`_h264`" (for H.264) or "`_h265`" (for HEVC).
 
-### What about hardware-based video transcoding?
+You can try hardware-based transcoding now using the `--encoder` option. On macOS, select the H.264 encoder this way:
 
-Using hardware with [Intel Quick Sync Video](https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video) instead of software like x264 is certainly faster. HandBrake even supports that hardware on some platforms. However, my default ratecontrol system can't be applied to existing hardware encoders because they lack API to change the necessary settings.
+    transcode-video --encoder vt_h264 "/path/to/Movie.mkv"
 
-Also, keep in mind that hardware encoders are typically designed for realtime video chat or other similar duties. To maintain that performance, they often take shortcuts with video quality like reducing reference frames, lowering subpixel motion estimation, etc. Such an approach is the equivalent of using the `veryfast` preset with a software encoder. That's fine for video chat but I wouldn't recommend it for transcoding your disc collection.
+You can also use the `--target` option with these encoders.
 
 ### How do you assess video transcoding quality?
 
@@ -689,17 +657,29 @@ For a few problematic videos, I have to apply options like `--force-rate 23.976
 
 ## History
 
+### [0.22.0](https://github.com/donmelton/video_transcoding/releases/tag/0.22.0)
+
+Saturday, December 15, 2018
+
+* Add an `--encoder` option to `transcode-video` so `--encoder x265` will work the same as the much longer and harder to type `--handbrake-option encoder=x265`.
+* Add a `--simple` ratecontrol option to `transcode-video` (via [ #211](https://github.com/donmelton/video_transcoding/issues/211)) which:
+    * Works like my special, or default, ratecontrol system but won't emit those annoying `VBV underflow` warnings because it's only constrained by the target bitrate and not also a minimum quality.
+    * Signals Hypothetical Reference Decoder (HRD) information in metadata like my average bitrate (ABR) ratecontrol system.
+    * Produces output similar in appearance to that from hardware-based encoders but is less prone to color banding.
+* Modify `transcode-video` to not pass the target video bitrate to hardware-based encoders when a CRF value is also specified, e.g. via something like `--handbrake-option quality=20`. Currently, this is only applicable to encoders such as `nvenc_h264`, `nvenc_h265`, `vce_h264` and `vce_h265`.
+* Revise both the H.265 video and hardware-based video transcoding answers in the "FAQ" section of the "README" document.
+* Update and simplify the "Explanation" section of the "README" document.
+
 ### [0.21.2](https://github.com/donmelton/video_transcoding/releases/tag/0.21.2)
 
-Sunday, December 4, 2018
+Tuesday, December 4, 2018
 
 * Modify `transcode-video` to pass the target video bitrate to hardware-based encoders available in HandBrake for Windows and Linux as well as HandBrake nightly builds for macOS:
     * Check the output of `HandBrakeCLI --help` from one of those builds to find out if your platform has any of these video encoders available.
     * The names of these encoders all end with "`_h264`" (for H.264) or "`_h265`" (for HEVC).
-    * On macOS, adding `--handbrake-option encoder=vt_h264` is all that's needed to enable hardware-based H.264 transcoding. Use `vt_h264` for HEVC.
+    * On macOS, adding `--handbrake-option encoder=vt_h264` is all that's needed to enable hardware-based H.264 transcoding. Use `vt_h265` for HEVC.
     * On Windows and Linux, use `qsv_h264` or `qsv_h265`. Other encoders might be available as well in nightly builds.
     * WARNING: If you request an encoder that is _not_ available, `HandBrakeCLI` may fail or it may just fallback to a software-based encoder. Check your console output while transcoding to be certain.
-    * With the encoders I've tested so far, `--preset` and `--encoder-option` have no effect no matter what you pass.
     * These hardware-based encoders are far faster than the software-based `x264` and `x265` encoders while still delivering _reasonable_ quality. Of course, your mileage (and perception) may vary.
 
 ### [0.21.1](https://github.com/donmelton/video_transcoding/releases/tag/0.21.1)

data/bin/transcode-video

@@ -56,8 +56,12 @@ Output options:
 -n, --dry-run       don't transcode, just show `HandBrakeCLI` command and exit
 
 Quality options:
+    --encoder NAME  use named video encoder (default: x264)
+                      (refer to `HandBrakeCLI --help` for available encoders)
     --abr           use constrained average bitrate (ABR) ratecontrol
                       (predictable size with different quality than default)
+    --simple        use simple constrained ratecontrol
+                      (limited size with different quality than default)
     --target big|small
                     apply video bitrate target macro for all input resolutions
                       (`big` trades some size for increased quality)
@@ -69,12 +73,12 @@ Quality options:
                       (can be used multiple times)
     --quick         increase encoding speed by 70-80%
                       with no easily perceptible loss in video quality
-                      (avoids quality problems associated with x264 presets)
+                      (avoids quality problems with some encoder presets)
     --veryquick     increase encoding speed by 90-125%
                       with little easily perceptible loss in video quality
                       (unlike `--quick`, output size is larger than default)
     --preset veryfast|faster|fast|slow|slower|veryslow
-                    apply x264 encoder preset
+                    apply video encoder preset
 
 Video options:
     --crop T:B:L:R  set video crop values (default: 0:0:0:0)
@@ -192,11 +196,10 @@ External subtitle options:
 
 Advanced options:
 -E, --encoder-option NAME=VALUE|_NAME
-                    pass x264 video encoder option by name with value
+                    pass video encoder option by name with value
                       or disable use of option by prefixing name with "_"
                       (e.g.: `-E vbv-bufsize=8000`)
                       (e.g.: `-E _crf-max`)
-                      (refer to `x264 --fullhelp` for more information)
                       (can be used multiple times)
 -H, --handbrake-option NAME[=VALUE]|_NAME
                     pass `HandBrakeCLI` option by name or by name with value
@@ -228,6 +231,7 @@ HERE
       @log                        = true
       @dry_run                    = false
       @use_abr                    = false
+      @use_simple                 = false
       @target_bitrate_2160p       = 12000
       @target_bitrate_1080p       = 6000
       @target_bitrate_720p        = 3000
@@ -317,8 +321,18 @@ HERE
       opts.on('--no-log')             { @log = false }
       opts.on('-n', '--dry-run')      { @dry_run = true }
 
+      opts.on '--encoder ARG' do |arg|
+        force_handbrake_option 'encoder', arg
+      end
+
       opts.on '--abr' do
-        @use_abr = true
+        @use_abr    = true
+        @use_simple = false
+      end
+
+      opts.on '--simple' do
+        @use_simple = true
+        @use_abr    = false
       end
 
       opts.on '--target ARG' do |arg|
@@ -933,7 +947,7 @@ HERE
 
       encoder = @handbrake_options.fetch('encoder', 'x264')
 
-      if encoder =~ /_h26[45]$/
+      if encoder =~ /_h26[45]$/ and not @handbrake_options.has_key? 'quality'
         handbrake_options['vb'] = bitrate.to_s
       end
 
@@ -960,6 +974,12 @@ HERE
         encoder_options['vbv-bufsize']  = ((bitrate * 2).to_i).to_s
         encoder_options['nal-hrd']      = 'vbr' if encoder =~ /^x264(?:_10bit)?$/
         encoder_options['hrd']          = '1'   if encoder =~ /^x265(?:_1[02]bit)?$/
+      elsif @use_simple
+        handbrake_options['quality']    = '1'
+        encoder_options['vbv-maxrate']  = bitrate.to_s
+        encoder_options['vbv-bufsize']  = bitrate.to_s
+        encoder_options['nal-hrd']      = 'vbr' if encoder =~ /^x264(?:_10bit)?$/
+        encoder_options['hrd']          = '1'   if encoder =~ /^x265(?:_1[02]bit)?$/
       else
         handbrake_options['quality']    = '1'
         encoder_options['vbv-maxrate']  = bitrate.to_s

data/lib/video_transcoding/version.rb

@@ -5,5 +5,5 @@
 #
 
 module VideoTranscoding
-  VERSION = '0.21.2'
+  VERSION = '0.22.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: video_transcoding
 version: !ruby/object:Gem::Version
-  version: 0.21.2
+  version: 0.22.0
 platform: ruby
 authors:
 - Don Melton
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-12-04 00:00:00.000000000 Z
+date: 2018-12-16 00:00:00.000000000 Z
 dependencies: []
 description: |2
       Video Transcoding is a package of tools to transcode, inspect