yt 0.32.6 → 0.33.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e19662ddb23d57f544dd42ab0ceb974e0cf7b7a0aadb2137a298c9a5f0afe2cf
-  data.tar.gz: c02517119190c7eda5f532305330b148979ca2bd04b38485915717fb41c5125c
+  metadata.gz: 7f3d794d74e724d49a7a96cf3d27dcfbeabcc0c42a7a9766deaae7e01be367c0
+  data.tar.gz: bf836aa2984cb6c6903881d56a6f60aad68504a5cb7013c276e6466ee075cf71
 SHA512:
-  metadata.gz: 3d19c4c4c947855487e23d07fd89f6b11719af3cb11b8bc3d73594f4703be7f7990652050ac03449359d6861be02cd43eea2599839e2f9b8c0f304449bf084cf
-  data.tar.gz: d794dfbbe1eddcaa98bf0ec0daa36983b7e69b97c8aafa060418999ed5a5806dbf3bbec18fe772a2383f28033f87b8f1364c4742f74abdc6b1a201a1209935a4
+  metadata.gz: 74662b9a898592295377a2e3bd51295682a67d07c773abc27945706c858860e74ce4ce4c77ca20455a674a746090603ef49261bce21b40418984430f93f5d18c
+  data.tar.gz: 74f82bc61b9bde908766aa6c1e19afe7da9fbeb42faeead0a37d2cc31def646b71f333c0e3dde43589027b28cf3e781d04d1824b58be7db99af3b31914debc0d

data/.rspec

@@ -1,6 +1,3 @@
---format documentation
 --color
---tag ~rate_limited
---tag ~flaky
---tag ~extended_permissions
 --exclude_pattern spec/requests/as_content_owner/*_spec.rb
+--tag=~slow

data/CHANGELOG.md

@@ -6,6 +6,25 @@ For more information about changelogs, check
 [Keep a Changelog](http://keepachangelog.com) and
 [Vandamme](http://tech-angels.github.io/vandamme).
 
+## 0.33.0 - 2020-04-10
+
+If your code calls reports methods such as `views`, `likes`, or `reports`,
+do not include `by: :week` option since `7DayTotals` dimension will no longer be
+supported by YouTube API as of [April 15, 2020](https://developers.google.com/youtube/analytics/revision_history#october-15,-2019).
+
+Use `by: :day` option instead and add up the numbers from each day.
+
+If you keep using `by: :week` option after this change it will raise an error
+(before the gem upgrade) or it will run with `day` dimension instead (after
+the gem upgrade, like any other random input).
+
+* [REMOVAL] Remove `by: :week` option for reports.
+* [FEATURE] Add back the option of initializing a resource by its URL.
+* [BUGFIX] Limit retries on refreshing tokens
+
+**Breaking change**
+
+If your code is using constant `Yt::URL::CHANNEL_PATTERNS` etc then it's moved to `Yt::Resource::CHANNEL_PATTERNS`, `Yt::Resource::VIDEO_PATTERNS`, and `Yt::Resource::PLAYLIST_PATTERNS`.
 
 ## 0.32.6 - 2020-02-07
 

data/README.md

@@ -526,44 +526,34 @@ If a variable is set in both places, then `Yt.configure` takes precedence.
 How to test
 ===========
 
-Yt comes with two different sets of tests:
-
-1. tests in `spec/models`, `spec/collections` and `spec/errors` **do not hit** the YouTube API
-1. tests in `spec/requests` **hit** the YouTube API and require authentication
-
-To only run tests against models, collections and errors (which do not hit the API), type:
+To run tests:
 
 ```bash
-rspec spec/models spec/collections spec/errors
+rspec
 ```
 
-To also run live-tests against the YouTube API, type:
+We recommend RSpec >= 3.8.
 
-```bash
-rspec
-```
+Yt comes with two different sets of tests:
+
+1. Unit tests in `spec/models`, `spec/collections` and `spec/errors`
+2. Legacy integration tests in `spec/requests`
+
+Coming soon will be a new set of high-level integration tests.
+
+Integration tests are recorded with VCR. Some of the tests refer to
+fixture data that an arbitrary account may not have access to. If you
+need to modify one of these tests or re-record the cassette, we'd
+suggest working against your own version of the testing setup. Then in
+your pull request, we can help canonize your test/fixtures.
+
+Some of the integration tests require authentication. These can be set
+with the following environment variables:
 
-This will fail unless you have set up a test YouTube application and some
-tests YouTube accounts (with appropriate fixture data) to hit the API.
-Furthermore, tests that require authentication are divided into three
-roles, which correspond to each directory in `spec/requests`:
-
-* Account-based tests, which require a valid refresh token along with
-  the application-level credentials the refresh token was created with
-  (`YT_TEST_DEVICE_REFRESH_TOKEN`, `YT_TEST_DEVICE_CLIENT_ID`, and
-  `YT_TEST_DEVICE_CLIENT_SECRET` respectively).
-* Server application tests, which use a server API key
-  (`YT_TEST_SERVER_API_KEY`).
-* Tests that excercise YouTube's partner functionality. This requires an
-  a partner channel id (`YT_TEST_CONTENT_OWNER_NAME`), a refresh token
-  that's authenticated with that channel
-  (`YT_TEST_CONTENT_OWNER_REFRESH_TOKEN`), and the corresponding
-  application (`YT_TEST_PARTNER_CLIENT_ID` and
-  (`YT_TEST_PARTNER_CLIENT_SECRET`).
-
-The refresh tokens need to be generated with the `youtube`,
-`yt-analytics` and `userinfo.profile` permissions in order for tests to
-pass.
+* `YT_TEST_CLIENT_ID`
+* `YT_TEST_CLIENT_SECRET`
+* `YT_TEST_API_KEY`
+* `YT_TEST_REFRESH_TOKEN`
 
 How to release new versions
 ===========================

data/YOUTUBE_IT.md

@@ -141,7 +141,7 @@ client = YouTubeIt::Client.new
 client.videos_by(:query => "penguin",  :author => "liz")
 # with yt: the 'author' filter was removed from YouTube API V3, so the
 # request must be done using the channel of the requested author
-channel = Yt::Channel.new id: 'UCxxxxxxxxx'
+channel = Yt::Channel.new url: 'youtube.com/liz'
 channel.videos.where(q: 'penguin')
 ```
 
@@ -176,7 +176,7 @@ client = YouTubeIt::Client.new
 client.videos_by(:user => 'liz')
 # with yt: the 'author' filter was removed from YouTube API V3, so the
 # request must be done using the channel of the requested author
-channel = Yt::Channel.new id: 'UCxxxxxxxxx'
+channel = Yt::Channel.new url: 'youtube.com/liz'
 channel.videos.where(q: 'penguin')
 ```
 
@@ -188,7 +188,7 @@ client = YouTubeIt::Client.new
 client.videos_by(:favorites, :user => 'liz')
 # with yt: note that only *old* channels have a "Favorites" playlist, since
 # "Favorites" has been deprecated by YouTube in favor of "Liked Videos".
-channel = Yt::Channel.new id: 'UCxxxxxxxxx'
+channel = Yt::Channel.new url: 'youtube.com/liz'
 channel.related_playlists.find{|p| p.title == 'Favorites'}
 ```
 
@@ -832,4 +832,4 @@ TODO
 
  $ client.activity(user) #default current user
 
--->
+-->

data/lib/yt.rb

@@ -13,7 +13,6 @@ require 'yt/models/video_group'
 require 'yt/models/comment_thread'
 require 'yt/models/ownership'
 require 'yt/models/advertising_options_set'
-require 'yt/models/url'
 
 # An object-oriented Ruby client for YouTube.
 # Helps creating applications that need to interact with YouTube objects.

data/lib/yt/associations/has_reports.rb

@@ -10,7 +10,7 @@ module Yt
       #   @option options [Array<Symbol>] :only The metrics to generate reports
       #     for.
       #   @option options [Symbol] :by (:day) The dimension to collect metrics
-      #     by. Accepted values are: +:day+, +:week+, +:month+.
+      #     by. Accepted values are: +:day+, +:month+.
       #   @option options [#to_date] :since The first day of the time-range.
       #     Also aliased as +:from+.
       #   @option options [#to_date] :until The last day of the time-range.
@@ -40,11 +40,6 @@ module Yt
       #   @example Get the $1 for this and last month:
       #     resource.$1 since: 1.month.ago, by: :month
       #     # => {Wed, 01 Apr 2014..Thu, 30 Apr 2014 => 12.0, Fri, 01 May 2014..Sun, 31 May 2014 => 34.0, …}
-      #   @return [Hash<Range<Date, Date>, $2>] if grouped by week, the $1
-      #     for each week in the time-range.
-      #   @example Get the $1 for this and last week:
-      #     resource.$1 since: 1.week.ago, by: :week
-      #     # => {Wed, 01 Apr 2014..Tue, 07 Apr 2014 => 20.0, Wed, 08 Apr 2014..Tue, 14 Apr 2014 => 13.0, …}
       #   @macro report
 
       # @!macro [new] report_with_range
@@ -75,19 +70,19 @@ module Yt
 
       # @!macro [new] report_by_day
       #   @option options [Symbol] :by (:day) The dimension to collect $1 by.
-      #     Accepted values are: +:day+, +:week+, +:month+.
+      #     Accepted values are: +:day+, +:month+.
       #   @macro report_with_day
 
       # @!macro [new] report_by_day_and_country
       #   @option options [Symbol] :by (:day) The dimension to collect $1 by.
-      #     Accepted values are: +:day+, +:week+, +:month+, :+range+.
+      #     Accepted values are: +:day+, +:month+, :+range+.
       #   @macro report_with_day
       #   @macro report_with_range
       #   @macro report_with_country
 
       # @!macro [new] report_by_day_and_state
       #   @option options [Symbol] :by (:day) The dimension to collect $1 by.
-      #     Accepted values are: +:day+, +:week+, +:month+, :+range+.
+      #     Accepted values are: +:day+, +:month+, :+range+.
       #   @macro report_with_day
       #   @macro report_with_range
       #   @macro report_with_country_and_state
@@ -128,7 +123,7 @@ module Yt
 
       # @!macro [new] report_by_video_dimensions
       #   @option options [Symbol] :by (:day) The dimension to collect $1 by.
-      #     Accepted values are: +:day+, +:week+, +:month+, +:range+,
+      #     Accepted values are: +:day+, +:month+, +:range+,
       #     +:traffic_source+,+:search_term+, +:playback_location+,
       #     +:related_video+, +:embedded_player_location+.
       #   @option options [Array<Symbol>] :includes ([:id]) if grouped by
@@ -162,7 +157,7 @@ module Yt
 
       # @!macro [new] report_by_channel_dimensions
       #   @option options [Symbol] :by (:day) The dimension to collect $1 by.
-      #     Accepted values are: +:day+, +:week+, +:month+, +:range+,
+      #     Accepted values are: +:day+, +:month+, +:range+,
       #     +:traffic_source+, +:search_term+, +:playback_location+, +:video+,
       #     +:related_video+, +:playlist+, +:embedded_player_location+.
       #   @return [Hash<Symbol, $2>] if grouped by embedded player location,
@@ -175,7 +170,7 @@ module Yt
 
       # @!macro [new] report_by_playlist_dimensions
       #   @option options [Symbol] :by (:day) The dimension to collect $1 by.
-      #     Accepted values are: +:day+, +:week+, +:month+, +:range+,
+      #     Accepted values are: +:day+, +:month+, +:range+,
       #     +:traffic_source+, +:playback_location+, +:related_video+, +:video+,
       #     +:playlist+.
       #   @macro report_with_channel_dimensions
@@ -228,7 +223,7 @@ module Yt
       def define_reports_method(metric, type)
         (@metrics ||= {})[metric] = type
         define_method :reports do |options = {}|
-          from = options[:since] || options[:from] || (options[:by].in?([:day, :week, :month]) ? 5.days.ago : '2005-02-01')
+          from = options[:since] || options[:from] || (options[:by].in?([:day, :month]) ? 5.days.ago : '2005-02-01')
           to = options[:until] || options[:to] || Date.today
           location = options[:in]
           country = location.is_a?(Hash) ? location[:country] : location
@@ -252,7 +247,7 @@ module Yt
 
       def define_metric_method(metric)
         define_method metric do |options = {}|
-          from = options[:since] || options[:from] || (options[:by].in?([:day, :week, :month]) ? 5.days.ago : '2005-02-01')
+          from = options[:since] || options[:from] || (options[:by].in?([:day, :month]) ? 5.days.ago : '2005-02-01')
           to = options[:until] || options[:to] || Date.today
           location = options[:in]
           country = location.is_a?(Hash) ? location[:country] : location

data/lib/yt/collections/reports.rb

@@ -6,7 +6,6 @@ module Yt
     class Reports < Base
       DIMENSIONS = Hash.new({name: 'day', parse: ->(day, *values) { @metrics.keys.zip(values.map{|v| {Date.iso8601(day) => v}}).to_h} }).tap do |hash|
         hash[:month] = {name: 'month', parse: ->(month, *values) { @metrics.keys.zip(values.map{|v| {Range.new(Date.strptime(month, '%Y-%m').beginning_of_month, Date.strptime(month, '%Y-%m').end_of_month) => v} }).to_h} }
-        hash[:week] = {name: '7DayTotals', parse: ->(last_day_of_week, *values) { @metrics.keys.zip(values.map{|v| {Range.new(Date.strptime(last_day_of_week) - 6, Date.strptime(last_day_of_week)) => v} }).to_h} }
         hash[:range] = {parse: ->(*values) { @metrics.keys.zip(values.map{|v| {total: v}}).to_h } }
         hash[:traffic_source] = {name: 'insightTrafficSourceType', parse: ->(source, *values) { @metrics.keys.zip(values.map{|v| {TRAFFIC_SOURCES.key(source) => v}}).to_h} }
         hash[:playback_location] = {name: 'insightPlaybackLocationType', parse: ->(location, *values) { @metrics.keys.zip(values.map{|v| {PLAYBACK_LOCATIONS.key(location) => v}}).to_h} }
@@ -140,11 +139,6 @@ module Yt
           end
           if dimension == :month
             hash = hash.transform_values{|h| h.sort_by{|range, v| range.first}.to_h}
-          elsif dimension == :week
-            hash = hash.transform_values do |h|
-              h.select{|range, v| range.last.wday == days_range.last.wday}.
-              sort_by{|range, v| range.first}.to_h
-            end
           elsif dimension == :day
             hash = hash.transform_values{|h| h.sort_by{|day, v| day}.to_h}
           elsif dimension.in? [:traffic_source, :country, :state, :playback_location, :device_type, :operating_system, :subscribed_status]
@@ -159,11 +153,15 @@ module Yt
       # same query is a workaround that works and can hardly cause any damage.
       # Similarly, once in while YouTube responds with a random 503 error.
       rescue Yt::Error => e
-        (max_retries > 0) && rescue?(e) ? sleep(3) && within(days_range, country, state, dimension, videos, historical, max_retries - 1) : raise
+        (max_retries > 0) && rescue?(e) ? sleep(retry_time) && within(days_range, country, state, dimension, videos, historical, max_retries - 1) : raise
       end
 
     private
 
+      def retry_time
+        3
+      end
+
       def type_cast(value, type)
         case [type]
           when [Integer] then value.to_i if value

data/lib/yt/models/resource.rb

@@ -12,7 +12,9 @@ module Yt
 
       # @!attribute [r] id
       #   @return [String] the ID that YouTube uses to identify each resource.
-      attr_reader :id
+      def id
+        @id ||= @match['id'] || fetch_channel_id
+      end
 
     ### STATUS ###
 
@@ -42,7 +44,12 @@ module Yt
 
       # @private
       def initialize(options = {})
-        @id = options[:id]
+        if options[:url]
+          @url = options[:url]
+          @match = find_pattern_match
+        else
+          @id = options[:id]
+        end
         @auth = options[:auth]
         @snippet = Snippet.new(data: options[:snippet]) if options[:snippet]
         @status = Status.new(data: options[:status]) if options[:status]
@@ -50,7 +57,11 @@ module Yt
 
       # @private
       def kind
-        self.class.to_s.demodulize.underscore
+        if @url
+          @match[:kind].to_s
+        else
+          self.class.to_s.demodulize.underscore
+        end
       end
 
       # @private
@@ -66,8 +77,63 @@ module Yt
         end
       end
 
+      # @return [Array<Regexp>] patterns matching URLs of YouTube playlists.
+      PLAYLIST_PATTERNS = [
+         %r{^(?:https?://)?(?:www\.)?youtube\.com/playlist/?\?list=(?<id>[a-zA-Z0-9_-]+)},
+      ]
+
+      # @return [Array<Regexp>] patterns matching URLs of YouTube videos.
+      VIDEO_PATTERNS = [
+        %r{^(?:https?://)?(?:www\.)?youtube\.com/watch\?v=(?<id>[a-zA-Z0-9_-]{11})},
+        %r{^(?:https?://)?(?:www\.)?youtu\.be/(?<id>[a-zA-Z0-9_-]{11})},
+        %r{^(?:https?://)?(?:www\.)?youtube\.com/embed/(?<id>[a-zA-Z0-9_-]{11})},
+        %r{^(?:https?://)?(?:www\.)?youtube\.com/v/(?<id>[a-zA-Z0-9_-]{11})},
+      ]
+
+      # @return [Array<Regexp>] patterns matching URLs of YouTube channels.
+      CHANNEL_PATTERNS = [
+        %r{^(?:https?://)?(?:www\.)?youtube\.com/channel/(?<id>UC[a-zA-Z0-9_-]{22})},
+        %r{^(?:https?://)?(?:www\.)?youtube\.com/(?<format>c/|user/)?(?<name>[a-zA-Z0-9_-]+)}
+      ]
+
     private
 
+      def find_pattern_match
+        patterns.find do |kind, regex|
+          if data = @url.match(regex)
+            # Note: With Ruby 2.4, the following is data.named_captures
+            break data.names.zip(data.captures).to_h.merge kind: kind
+          end
+        end || {kind: :unknown}
+      end
+
+      def patterns
+        # @note: :channel *must* be the last since one of its regex eats the
+        # remaining patterns. In short, don't change the following order.
+        Enumerator.new do |patterns|
+          VIDEO_PATTERNS.each {|regex| patterns << [:video, regex]}
+          PLAYLIST_PATTERNS.each {|regex| patterns << [:playlist, regex]}
+          CHANNEL_PATTERNS.each {|regex| patterns << [:channel, regex]}
+        end
+      end
+
+      def fetch_channel_id
+        response = Net::HTTP.start 'www.youtube.com', 443, use_ssl: true do |http|
+          http.request Net::HTTP::Get.new("/#{@match['format']}#{@match['name']}")
+        end
+        if response.is_a?(Net::HTTPRedirection)
+          response = Net::HTTP.start 'www.youtube.com', 443, use_ssl: true do |http|
+            http.request Net::HTTP::Get.new(response['location'])
+          end
+        end
+        regex = %r{<meta itemprop="channelId" content="(?<id>UC[a-zA-Z0-9_-]{22})">}
+        if data = response.body.match(regex)
+          data[:id]
+        else
+          raise Yt::Errors::NoItems
+        end
+      end
+
       # Since YouTube API only returns tags on Videos#list, the memoized
       # `@snippet` is erased if the video was instantiated through Video#search
       # (e.g., by calling account.videos or channel.videos), so that the full

data/lib/yt/models/url.rb

@@ -14,18 +14,17 @@ module Yt
       # @param [String] text the name or URL of a YouTube resource (in any form).
       def initialize(text)
         @text = text.to_s.strip
-        @match = find_pattern_match
       end
 
       # @return [Symbol] the kind of YouTube resource matching the URL.
       # Possible values are: +:playlist+, +:video+, +:channel+, and +:unknown:.
       def kind
-        @match[:kind]
+        Resource.new(url: @text).kind.to_sym
       end
 
       # @return [<String, nil>] the ID of the YouTube resource matching the URL.
       def id
-        @match['id'] ||= fetch_id
+        Resource.new(url: @text).id
       end
 
       # @return [<Yt::Channel>] the resource associated with the URL
@@ -37,63 +36,6 @@ module Yt
           else raise Yt::Errors::NoItems
         end.new options.merge(id: id)
       end
-
-      # @return [Array<Regexp>] patterns matching URLs of YouTube playlists.
-      PLAYLIST_PATTERNS = [
-         %r{^(?:https?://)?(?:www\.)?youtube\.com/playlist/?\?list=(?<id>[a-zA-Z0-9_-]+)},
-      ]
-
-      # @return [Array<Regexp>] patterns matching URLs of YouTube videos.
-      VIDEO_PATTERNS = [
-        %r{^(?:https?://)?(?:www\.)?youtube\.com/watch\?v=(?<id>[a-zA-Z0-9_-]{11})},
-        %r{^(?:https?://)?(?:www\.)?youtu\.be/(?<id>[a-zA-Z0-9_-]{11})},
-        %r{^(?:https?://)?(?:www\.)?youtube\.com/embed/(?<id>[a-zA-Z0-9_-]{11})},
-        %r{^(?:https?://)?(?:www\.)?youtube\.com/v/(?<id>[a-zA-Z0-9_-]{11})},
-      ]
-
-      # @return [Array<Regexp>] patterns matching URLs of YouTube channels.
-      CHANNEL_PATTERNS = [
-        %r{^(?:https?://)?(?:www\.)?youtube\.com/channel/(?<id>UC[a-zA-Z0-9_-]{22})},
-        %r{^(?:https?://)?(?:www\.)?youtube\.com/(?<format>c/|user/)?(?<name>[a-zA-Z0-9_-]+)}
-      ]
-
-    private
-
-      def find_pattern_match
-        patterns.find(-> {{kind: :unknown}}) do |kind, regex|
-          if data = @text.match(regex)
-            # Note: With Ruby 2.4, the following is data.named_captures
-            break data.names.zip(data.captures).to_h.merge kind: kind
-          end
-        end
-      end
-
-      def patterns
-        # @note: :channel *must* be the last since one of its regex eats the
-        # remaining patterns. In short, don't change the following order.
-        Enumerator.new do |patterns|
-          VIDEO_PATTERNS.each {|regex| patterns << [:video, regex]}
-          PLAYLIST_PATTERNS.each {|regex| patterns << [:playlist, regex]}
-          CHANNEL_PATTERNS.each {|regex| patterns << [:channel, regex]}
-        end
-      end
-
-      def fetch_id
-        response = Net::HTTP.start 'www.youtube.com', 443, use_ssl: true do |http|
-          http.request Net::HTTP::Get.new("/#{@match['format']}#{@match['name']}")
-        end
-        if response.is_a?(Net::HTTPRedirection)
-          response = Net::HTTP.start 'www.youtube.com', 443, use_ssl: true do |http|
-            http.request Net::HTTP::Get.new(response['location'])
-          end
-        end
-        regex = %r{<meta itemprop="channelId" content="(?<id>UC[a-zA-Z0-9_-]{22})">}
-        if data = response.body.match(regex)
-          data[:id]
-        else
-          raise Yt::Errors::NoItems
-        end
-      end
     end
   end
 end