jekyll-open-sdg-plugins 1.0.0.rc6 → 1.0.0.rc7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8007d07bf097d9475a171e78389891a753f69338de2a859de6754013fc84ec17
-  data.tar.gz: 6bad1a307d617c391b03970471310aa8be6f04571f17340391caaee50a21033e
+  metadata.gz: 2f9696653a01838967c46fbc75ba9f42ae3407e23fbea42f5577fff00c812400
+  data.tar.gz: 222958568c169871c68f6812bdb2e47f074186514c28de2d7090dbccdae416cb
 SHA512:
-  metadata.gz: 9ebe2ef38212facd07500e1d02976bc14f34fbab8624dfe324e6f696c0abb37571e6aec6dcef7f84422c5eb5b27ac646111b4949f694b5db14ec794d9abbcd1e
-  data.tar.gz: 1a4070ac2858484f3819be479b0041095eb3a7f4ee3b002cd36ef7656dc8639022a453c4fbf244a9bc076e67954651d3696a7665343b010ab7d866d234392798
+  metadata.gz: 9002444b28bfaf97393a3920ea25191d462fffecd3f53cc054948bd627ee45a3090fbd592203ac570b897eef110bb2aaa5059f92c005b270adb4000e4c123c50
+  data.tar.gz: 615e228587be863c9729300966e784515834d072adf5fe34acdb39d76fdaf7a3004c1607964a9cc8bb37d01dabc38811a310d586767bddae38c59dfeb915a75d

data/README.md

@@ -2,152 +2,6 @@
 
 > 💎 Jekyll plugins for use with the Open SDG platform
 
-This plugin provides some Jekyll functionality for the [Open SDG](https://github.com/open-sdg/open-sdg) platform.
+This plugin provides required Jekyll functionality for the [Open SDG](https://github.com/open-sdg/open-sdg) platform.
 
-The functionality provided consists of:
-
-## 1. A "t" Liquid filter for translating strings.
-
-Usage example:
-
-```
-{{ my_variable | t }}
-```
-
-## 2. Fill in missing fields to allow partially translated metadata.
-
-This allows metadata to be translated, field-by-field as needed, in subfolders in the data repository.
-
-## 3. Automatically create goal pages based on data.
-
-This creates goal pages automatically based on the data, so that the site repository does not need to maintain a `_goals` folder. It depends on a `_config.yml` setting.
-
-Usage example (in `_config.yml`):
-```
-create_goals:
-  # This determines which layout is used to render the pages.
-  layout: goal
-```
-
-## 4. Automatically create indicator pages based on data.
-
-This creates indicator pages automatically based on the data, so that the site repository does not need to maintain a `_indicators` folder. It depends on a `_config.yml` setting.
-
-Usage example (in `_config.yml`):
-```
-create_indicators:
-  # This determines which layout is used to render the pages.
-  layout: indicator
-```
-
-## 5. Automatically create 4 required pages.
-
-This automates the creation of 4 required pages that all implementations of Open SDG will need. These consist of:
-
-* the home page: /
-* the indicators json page: /indicators.json
-* the search results page: /search
-* the reporting status page: /reporting-status
-
-There are advanced options for overriding the location of these pages (see creae_pages.rb). But see the example below to use the defaults.
-
-Usage example (in `_config.yml`):
-```
-create_pages: true
-```
-
-## 6. Automatically fetch remote data and translations.
-
-This automates the fetching of the remote data (from the "data repository") and any remote translations.
-
-Note: This feature is disabled if "jekyll_get_json" is in the site config. This was the older (more verbose) way to do this.
-
-Usage example (in `_config.yml`):
-```
-remote_data_prefix: https://mygithuborg.github.io/my-data-repository
-remote_translations:
-  - https://open-sdg.github.io/sdg-translations/translations-0.6.0.json
-
-```
-
-For those interested in switching to this convenience feature, note that this makes the "jekyll_get_json" and "remotedatabaseurl" settings obsolete; so they can be removed.
-
-## 7. Provide standard variables on all pages, for use in templates
-
-This feature provides access to Hashes for goals, targets, and indicators. Each
-contains the following keys:
-* number (eg, "1" for a goal, "1.1" for a target, "1.1.1" for an indicator)
-* slug (eg, "1" for a goal, "1-1", for a target, "1-1-1" for an indicator)
-* name (the fully-translated name of the goal/target/indicator
-* sort (a string suitable for use in sorting the goals/targets/indicators)
-* global (an equivalent Hash containing specifically "global" versions)
-
-Additionally, indicators contain:
-* url (the URL of that indicator's page)
-* goal_number (the number of that indicator's goal)
-* target_number (the number of that indicator's target)
-* [all the indicator's metadata fields]
-
-Additionally, targets contain:
-* goal_number (the number of that target's goal)
-
-Additionally, goals contain:
-* url (the URL of that goal's page)
-* icon (the URL of that goal's icon)
-* short (the short version of the goal name, translated)
-
-The following variables can be used on ALL pages:
-
-* goals : Array of goals
-* targets : Array of targets
-* indicators : Array of indicators
-* baseurl: A page-specific version of site.baseurl, taking language into account
-* language: The language code for the current page
-* language_public: The "public" language code, which may be different from the
-  language code.
-* t: A hash of all the compiled translations in the current language
-
-The following variables can be used on all indicator pages:
-
-* goal : the current goal
-* target : the current target
-* indicator : the current indicator
-
-The following variables can be used on all goal pages:
-
-* goal : the current goal
-
-Examples of usage:
-
-Printing titles for all available indicators in Goal 2:
-```
-{% assign indicators = page.indicators | where: "goal_number", "2" %}
-{% for indicator in indicators %}
-  {{ indicator.name }}
-{% endfor %}
-```
-
-Printing the short name for the current goal, on a goal page:
-```
-{{ page.goal.short }}
-```
-
-Printing the name of all targets in a particular, on that goal page:
-```
-{% assign targets = page.targets | where: "goal_number", page.goal.number %}
-{% for target in targets %}
-  {{ target.name }}
-{% endfor %}
-
-## 8. Lookup goals/targets/indicators by ID
-
-The hashes detailed above in #7 can also be looked up by id, with the `sdg_lookup` filter.
-
-Examples of usage:
-
-Looking up target 7.1 and printing its name
-
-```
-{% assign target = '7.1' | sdg_lookup %}
-{{ target.name }}
-```
+The details on this functionality can be found throughout the [Open SDG documentation](https://open-sdg.readthedocs.io/en/latest/).

data/lib/jekyll-open-sdg-plugins/create_goals.rb

@@ -1,4 +1,5 @@
 require "jekyll"
+require_relative "helpers"
 
 module JekyllOpenSdgPlugins
   class CreateGoals < Jekyll::Generator
@@ -10,7 +11,18 @@ module JekyllOpenSdgPlugins
       if site.config['languages'] and site.config['create_goals']
         # Compile the list of goals.
         goals = {}
-        site.data['meta'].each do |inid, meta|
+        # Are we using translated builds?
+        metadata = {}
+        if opensdg_translated_builds(site)
+          # If we are using translated builds, the 'meta' data is underneath
+          # language codes. We just use the first language.
+          default_language = site.config['languages'][0]
+          metadata = site.data[default_language]['meta']
+        else
+          # Otherwise the 'meta' data is not underneath any language code.
+          metadata = site.data['meta']
+        end
+        metadata.each do |inid, indicator|
           goal = inid.split('-')[0].to_i
           goals[goal] = true
         end

data/lib/jekyll-open-sdg-plugins/create_indicators.rb

@@ -1,4 +1,5 @@
 require "jekyll"
+require_relative "helpers"
 
 module JekyllOpenSdgPlugins
   class CreateIndicators < Jekyll::Generator
@@ -25,8 +26,17 @@ module JekyllOpenSdgPlugins
           if languages_public[language]
             language_public = languages_public[language]
           end
+          metadata = {}
+          if opensdg_translated_builds(site)
+            # If we are using translated builds, the metadata is underneath a
+            # language code.
+            metadata = site.data[language]['meta']
+          else
+            # Otherwise the 'meta' data is not underneath any language code.
+            metadata = site.data['meta']
+          end
           # Loop through the indicators (using metadata as a list).
-          site.data['meta'].each do |inid, meta|
+          metadata.each do |inid, meta|
             # Add the language subfolder for all except the default (first) language.
             dir = index == 0 ? inid : File.join(language_public, inid)
             # Create the indicator page.

data/lib/jekyll-open-sdg-plugins/create_pages.rb

@@ -30,6 +30,7 @@ module JekyllOpenSdgPlugins
       #
       # create_pages: true
       if site.config['languages'] and site.config['create_pages']
+
         default_pages = [
           {
             'folder' => '/',

data/lib/jekyll-open-sdg-plugins/fetch_remote_data.rb

@@ -2,61 +2,109 @@ require "jekyll"
 require 'json'
 require 'deep_merge'
 require 'open-uri'
+require_relative "helpers"
 
 module JekyllOpenSdgPlugins
   class FetchRemoteData < Jekyll::Generator
     safe true
     priority :highest
 
+    def download_build(prefix)
+
+      endpoints = {
+        'meta' => 'meta/all.json',
+        'headlines' => 'headline/all.json',
+        'schema' => 'meta/schema.json',
+        'reporting' => 'stats/reporting.json',
+        'translations' => 'translations/translations.json'
+      }
+      build = {}
+      endpoints.each do |key, value|
+        endpoint = prefix + '/' + value
+        begin
+          source = JSON.load(open(endpoint))
+          build[key] = source
+        rescue StandardError => e
+          puts e.message
+          # For backwards compatibility, we allow 'translations' to be missing.
+          if key != 'translations'
+            abort 'Unable to fetch remote data from: ' + endpoint
+          end
+        end
+      end
+      build
+    end
+
     def generate(site)
-      # Backwards compatibility - only do this if 'jekyll_get_json' is absent.
-      # (This is the older style of remote data fetching.)
-      if !site.config['jekyll_get_json']
-        # First try to grab the remote data.
-        if site.config['remote_data_prefix']
-          prefix = site.config['remote_data_prefix']
-          # Set the required remotedatabaseurl config setting.
-          site.config['remotedatabaseurl'] = prefix
-          # Fetch remote data for each of our endpoints.
-          endpoints = {
-            'meta' => 'meta/all.json',
-            'headlines' => 'headline/all.json',
-            'schema' => 'meta/schema.json',
-            'reporting' => 'stats/reporting.json'
-          }
-          endpoints.each do |key, value|
-            target = site.data[key]
-            endpoint = prefix + '/' + value
-            begin
-              source = JSON.load(open(endpoint))
-              if target
-                target.deep_merge(source)
-              else
-                site.data[key] = source
-              end
-            rescue StandardError => e
-              puts e.message
-              abort 'Unable to fetch remote data from: ' + endpoint
+
+      if site.config['remote_data_prefix']
+        prefix = site.config['remote_data_prefix']
+
+        # For below, make sure there is at least an empty hash at
+        # site.data.translations.
+        if !site.data.has_key?('translations')
+          site.data['translations'] = {}
+        end
+
+        # How do we tell, before data has been fetched, whether the site is
+        # using translated builds? Quick and dirty way - attempt a download
+        # of the non-tranlsated build. If it fails, assume translated builds.
+        translated_builds = false
+        begin
+          JSON.load(open(prefix + '/meta/all.json'))
+        rescue StandardError => e
+          translated_builds = true
+        end
+
+        if translated_builds
+          # For translated builds, we download a build for each language, and
+          # place them in "subfolders" (so to speak) of site.data.
+          site.config['languages'].each do |language|
+            data_target = site.data[language]
+            data_source = download_build(prefix + '/' + language)
+            if data_target
+              data_target.deep_merge(data_source)
+            else
+              site.data[language] = data_source
+            end
+            # Additionally, we move the language-specific translations to the
+            # site.data.translations location, where all translations are kept.
+            translation_target = site.data['translations'][language]
+            translation_source = site.data[language]['translations']
+            if translation_target
+              translation_target.deep_merge(translation_source)
+            else
+              site.data['translations'][language] = translation_source
             end
           end
+        else
+          # For untranslated builds, we download one build only, and place it
+          # in the "root" (so to speak) of site.data. Nothing else is needed.
+          target = site.data
+          source = download_build(prefix)
+          target.deep_merge(source)
         end
+      else
+        abort 'The "remote_data_prefix" configuration setting is missing.'
+      end
 
-        # Next try to grab any remote translations.
-        if site.config['remote_translations']
-          key = 'translations'
-          target = site.data[key]
-          site.config['remote_translations'].each do |endpoint|
-            begin
-              source = JSON.load(open(endpoint))
-              if target
-                target.deep_merge(source)
-              else
-                site.data[key] = source
-              end
-            rescue StandardError => e
-              puts e.message
-              abort 'Unable to fetch remote translation from: ' + endpoint
+      # Finally support the deprecated 'remote_translations' option.
+      # This is deprecated because translations should now be in the
+      # data repository, where they will be fetched in download_build().
+      if site.config['remote_translations']
+        key = 'translations'
+        target = site.data[key]
+        site.config['remote_translations'].each do |endpoint|
+          begin
+            source = JSON.load(open(endpoint))
+            if target
+              target.deep_merge(source)
+            else
+              site.data[key] = source
             end
+          rescue StandardError => e
+            puts e.message
+            abort 'Unable to fetch remote translation from: ' + endpoint
           end
         end
       end

data/lib/jekyll-open-sdg-plugins/helpers.rb

@@ -50,3 +50,17 @@ def opensdg_translate_key(key, translations, language)
   # Otherwise we must have drilled all they way.
   return drilled
 end
+
+# Takes a site object and decides whether it is using translated builds.
+def opensdg_translated_builds(site)
+  # Assume the site is using translated builds.
+  translated_builds = true
+  site.config['languages'].each do |language|
+    # If any languages don't have a key in site.data, the site is not using
+    # translated builds.
+    if !site.data.has_key? language
+      translated_builds = false
+    end
+  end
+  return translated_builds
+end

data/lib/jekyll-open-sdg-plugins/sdg_variables.rb

@@ -34,33 +34,44 @@ module JekyllOpenSdgPlugins
       sort_order
     end
 
+    # The Jekyll baseurl is user-configured, and can be inconsistent. This
+    # ensure it is consistent in whether it starts/ends with a slash.
+    def normalize_baseurl(baseurl)
+      if baseurl == ''
+        baseurl = '/'
+      end
+      if !baseurl.start_with? '/'
+        baseurl = '/' + baseurl
+      end
+      if !baseurl.end_with? '/'
+        baseurl = baseurl + '/'
+      end
+      baseurl
+    end
+
     # Compute a URL for an item, given it's number.
     def get_url(baseurl, language, number, languages, languages_public)
 
+      baseurl = normalize_baseurl(baseurl)
+
       default_language = languages[0]
       language_public = language
       if languages_public && languages_public[language]
         language_public = languages_public[language]
       end
-      if baseurl == ''
-        baseurl = '/'
-      end
       if default_language != language
         baseurl += language_public + '/'
       end
-      if !baseurl.start_with? '/'
-        baseurl = '/' + baseurl
-      end
-      if !baseurl.end_with? '/'
-        baseurl = baseurl + '/'
-      end
 
       number = number.gsub('.', '-')
       baseurl + number
     end
 
     # Get a Hash of all the URLs based on one particular one.
-    def get_all_urls(url, language, languages, languages_public)
+    def get_all_urls(url, language, languages, languages_public, baseurl)
+
+      baseurl = normalize_baseurl(baseurl)
+
       language_public = language
       if languages_public && languages_public[language]
         language_public = languages_public[language]
@@ -78,7 +89,7 @@ module JekyllOpenSdgPlugins
         language => url
       }
       if language != default_language
-        urls[default_language] = url_without_language
+        urls[default_language] = baseurl + url_without_language
       end
       languages.each do |other_language|
         if other_language == language
@@ -91,7 +102,7 @@ module JekyllOpenSdgPlugins
         if languages_public && languages_public[other_language]
           other_language_public = languages_public[other_language]
         end
-        urls[other_language] = '/' + other_language_public + url_without_language
+        urls[other_language] = baseurl + other_language_public + url_without_language
       end
       urls
     end
@@ -210,7 +221,7 @@ module JekyllOpenSdgPlugins
           meta_key = indicator_number.gsub('.', '-')
           # The location of the metadata is different depending on whether we are
           # using "translated_builds" or not.
-          if site.config['translated_builds']
+          if opensdg_translated_builds(site)
             meta = site.data[language]['meta'][meta_key]
           else
             meta = site.data['meta'][meta_key]
@@ -299,9 +310,15 @@ module JekyllOpenSdgPlugins
           doc.data['goals'] = available_goals[language]
           doc.data['targets'] = available_targets[language]
           doc.data['indicators'] = available_indicators[language]
-          doc.data['t'] = site.data['translations'][language]
           doc.data['baseurl'] = get_url(baseurl, language, '', languages, languages_public)
-          doc.data['url_by_language'] = get_all_urls(doc.url, language, languages, languages_public)
+          doc.data['url_by_language'] = get_all_urls(doc.url, language, languages, languages_public, baseurl)
+          doc.data['t'] = site.data['translations'][language]
+
+          # Set the remote_data_prefix for this indicator.
+          doc.data['remote_data_prefix'] = site.config['remote_data_prefix']
+          if opensdg_translated_builds(site)
+            doc.data['remote_data_prefix'] += '/' + language
+          end
 
           if collection == 'indicators'
             # For indicators we also set the current indicator/target/goal.

data/lib/jekyll-open-sdg-plugins/version.rb

@@ -1,3 +1,3 @@
 module JekyllOpenSdgPlugins
-  VERSION = "1.0.0.rc6".freeze
+  VERSION = "1.0.0.rc7".freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: jekyll-open-sdg-plugins
 version: !ruby/object:Gem::Version
-  version: 1.0.0.rc6
+  version: 1.0.0.rc7
 platform: ruby
 authors:
 - Brock Fanning
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-11-16 00:00:00.000000000 Z
+date: 2019-11-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: jekyll
@@ -55,9 +55,7 @@ files:
 - lib/jekyll-open-sdg-plugins/create_indicators.rb
 - lib/jekyll-open-sdg-plugins/create_pages.rb
 - lib/jekyll-open-sdg-plugins/fetch_remote_data.rb
-- lib/jekyll-open-sdg-plugins/get_indicator_name.rb
 - lib/jekyll-open-sdg-plugins/helpers.rb
-- lib/jekyll-open-sdg-plugins/multilingual_metadata.rb
 - lib/jekyll-open-sdg-plugins/sdg_variables.rb
 - lib/jekyll-open-sdg-plugins/translate_key.rb
 - lib/jekyll-open-sdg-plugins/version.rb

data/lib/jekyll-open-sdg-plugins/get_indicator_name.rb

@@ -1,146 +0,0 @@
-require "jekyll"
-require_relative "helpers"
-
-# This plugin will be removed before version 1.0.0.
-# Do not rely on this!
-
-module Jekyll
-  module IndicatorName
-
-    # Takes an indicator ID (dot-delimited or dash-delimited) and returns the
-    # translated indicator name (according to the current language). This lookup
-    # is as forgiving as possible, to make sure that something is always there.
-    #
-    # The order of preference in the lookup is:
-    #
-    # 1. "indicator_name_national" in translated metadata - subfolder approach
-    # 2. "indicator_name_national" in translated metadata - translation key approach
-    # 3. If the default language, "indicator_name_national" in non-translated metadata
-    # 4. If a global indicator, translated global indicator name
-    # 5. "indicator_name" in translated metadata - subfolder approach
-    # 6. "indicator_name" in translated metadata - translation key approach
-    # 7. "indicator_name" in non-translated metadata
-    # 8. Finally, fall back to the indicator ID
-
-    def get_indicator_name(inid)
-
-      # Safety code - abort now if id is nil.
-      if inid.nil?
-        return ""
-      end
-
-      # Also make sure it is a string, and otherwise just return it.
-      if not inid.is_a? String
-        return inid
-      end
-
-      # More safety code - abort now if inid is empty.
-      if inid.empty?
-        return ""
-      end
-
-      # Normalize around dash-delimited inids.
-      inid = inid.gsub('.', '-')
-
-      # Some variables to help our lookups later.
-      page = @context.environments.first['page']
-      language = page['language']
-      languages = @context.registers[:site].config['languages']
-      data = @context.registers[:site].data
-      translations = data['translations']
-      meta = data['meta'][inid]
-
-      # The metadata fields that we'll seek, first "override" then "default".
-      override_field = 'indicator_name_national'
-      default_field = 'indicator_name'
-
-      name = false
-
-      # 1. Is there a subfolder translation of the override field?
-      if meta and meta.has_key? language
-        if !name and meta[language].has_key? override_field
-          name = meta[language][override_field]
-        end
-      end
-
-      # 2. Is the override field actually a "translation key"?
-      if !name
-        if meta and meta.has_key? override_field
-          untranslated = meta[override_field]
-          translated = opensdg_translate_key(untranslated, translations, language)
-          if untranslated != translated
-            # If the opensdg_translate_key() function returned something else,
-            # that means it was an actual "translation key".
-            name = translated
-          end
-        end
-      end
-
-      # 3. If this is the default language, use the non-translated override
-      # field, if available.
-      if !name
-        if language == languages[0]
-          if meta and meta.has_key? override_field
-            name = meta[override_field]
-          end
-        end
-      end
-
-      # 4. Is this a global indicator with a translation?
-      if !name
-        title_key = inid + '-title'
-        # For backwards compatibility, look for both dot and dash-delimited keys.
-        title_key_dots = inid.gsub('-', '.') + '-title'
-        if translations.has_key? language
-          if translations[language].has_key? 'global_indicators'
-            if translations[language]['global_indicators'].has_key? title_key
-              name = translations[language]['global_indicators'][title_key]
-            elsif translations[language]['global_indicators'].has_key? title_key_dots
-              name = translations[language]['global_indicators'][title_key_dots]
-            end
-          end
-        end
-      end
-
-      # 5. Is there a subfolder translation of the default field?
-      if !name
-        if meta and meta.has_key? language
-          if !name and meta[language].has_key? default_field
-            name = meta[language][default_field]
-          end
-        end
-      end
-
-      # 6. Is the default field actually a "translation key"?
-      if !name
-        if meta and meta.has_key? default_field
-          untranslated = meta[default_field]
-          translated = opensdg_translate_key(untranslated, translations, language)
-          if untranslated != translated
-            # If the opensdg_translate_key() function returned something else,
-            # that means it was an actual "translation key".
-            name = translated
-          end
-        end
-      end
-
-      # 7. Use the non-translated default field, if available.
-      if !name
-        if meta and meta.has_key? default_field
-          name = meta[default_field]
-        end
-      end
-
-      # 8. Still here? Just return the inid.
-      if !name
-        name = inid
-      end
-
-      # Finally return the name with key translation for good measure.
-      return opensdg_translate_key(name, translations, language)
-
-    end
-  end
-end
-
-Liquid::Template.register_filter(Jekyll::IndicatorName)

data/lib/jekyll-open-sdg-plugins/multilingual_metadata.rb

@@ -1,32 +0,0 @@
-require "jekyll"
-
-# This plugin will be removed before version 1.0.0.
-# Do not rely on this!
-
-module JekyllOpenSdgPlugins
-  class MultilingualMetadataGenerator < Jekyll::Generator
-    safe true
-    priority :low
-
-    def generate(site)
-      # Make sure that the translated metadata contains a complete set of values
-      # including non-translated metadata as a fallback. This allows us to treat
-      # the translated metadata as complete when used in layouts and includes.
-      if site.config['languages']
-        site.config['languages'].each do |language|
-          site.data['meta'].each do |indicator_id, meta|
-            if meta[language]
-              meta.each do |meta_key, meta_value|
-                unless site.config['languages'].include? meta_key
-                  if !meta[language][meta_key]
-                    meta[language][meta_key] = meta_value
-                  end
-                end
-              end
-            end
-          end
-        end
-      end
-    end
-  end
-end