jekyll_pages_api 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 4485de15e66a09dedf59a42575b00ae341f442e9
+  data.tar.gz: f1063959c1529374a696d1be62521fa451de8b71
+SHA512:
+  metadata.gz: a589f6459b68d956a086a1c81a0cc6736115d6e403c38733626d6e90cc5ff601a42fb751031d3d017b62b9dad17038fced9ce7c1bcf46113510e03baff01f3b7
+  data.tar.gz: 0f791a9cfe8bf63305e66d0248fba0372ce911425edbbc35ab357497b7143413807ece67b91ebb1821dfc46ac39090b9d95102e0d1a322dce7fd074d021a9f5c

data/.gitignore

@@ -0,0 +1,14 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--require spec_helper

data/.travis.yml

@@ -0,0 +1,4 @@
+language: ruby
+rvm:
+- 2.0.0
+script: bundle exec rspec

data/CONTRIBUTING.md

@@ -0,0 +1,22 @@
+## Welcome!
+
+We're so glad you're thinking about contributing to an 18F open source project! If you're unsure or afraid of anything, just ask or submit the issue or pull request anyways. The worst that can happen is that you'll be politely asked to change something. We appreciate any sort of contribution, and don't want a wall of rules to get in the way of that.
+
+Before contributing, we encourage you to read our CONTRIBUTING policy (you are here), our LICENSE, and our README, all of which should be in this repository. If you have any questions, or want to read more about our underlying policies, you can consult the 18F Open Source Policy GitHub repository at https://github.com/18f/open-source-policy, or just shoot us an email/official government letterhead note to [18f@gsa.gov](mailto:18f@gsa.gov).
+
+## Running tests
+
+```bash
+bundle
+bundle exec rspec
+```
+
+## Public domain
+
+This project is in the public domain within the United States, and
+copyright and related rights in the work worldwide are waived through
+the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
+
+All contributions to this project will be released under the CC0
+dedication. By submitting a pull request, you are agreeing to comply
+with this waiver of copyright interest.

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in jekyll_pages_api.gemspec
+gemspec

data/LICENSE.md

@@ -0,0 +1,31 @@
+As a work of the United States Government, this project is in the
+public domain within the United States.
+
+Additionally, we waive copyright and related rights in the work
+worldwide through the CC0 1.0 Universal public domain dedication.
+
+## CC0 1.0 Universal Summary
+
+This is a human-readable summary of the [Legal Code (read the full text)](https://creativecommons.org/publicdomain/zero/1.0/legalcode).
+
+### No Copyright
+
+The person who associated a work with this deed has dedicated the work to
+the public domain by waiving all of his or her rights to the work worldwide
+under copyright law, including all related and neighboring rights, to the
+extent allowed by law.
+
+You can copy, modify, distribute and perform the work, even for commercial
+purposes, all without asking permission.
+
+### Other Information
+
+In no way are the patent or trademark rights of any person affected by CC0,
+nor are the rights that other persons may have in the work or in how the
+work is used, such as publicity or privacy rights.
+
+Unless expressly stated otherwise, the person who associated a work with
+this deed makes no warranties about the work, and disclaims liability for
+all uses of the work, to the fullest extent permitted by applicable law.
+When using or citing the work, you should not imply endorsement by the
+author or the affirmer.

data/README.md

@@ -0,0 +1,37 @@
+# Jekyll Pages API [![Build Status](https://travis-ci.org/18F/jekyll_pages_api.svg?branch=master)](https://travis-ci.org/18F/jekyll_pages_api)
+
+Jekyll Pages API is a [Jekyll Plugin](http://jekyllrb.com/docs/plugins/) gem that generates a JSON file with data for all the Pages in your Site. [Jekyll](http://jekyllrb.com), if you're not familiar, is a static website generator written in Ruby.
+
+## Usage
+
+Add this line to your application's Gemfile:
+
+```ruby
+group :jekyll_plugins do
+  gem 'jekyll_pages_api'
+end
+```
+
+And then execute:
+
+```bash
+bundle
+bundle exec jekyll serve
+```
+
+You can then see the generated JSON file at http://localhost:4000/api/v1/pages.json, which will look something like this:
+
+```json
+{
+  "entries": [
+    {
+      "title": "18F Hub",
+      "url": "/",
+      "body": "18F is a digital services team within GSA..."
+    },
+    ...
+  ]
+}
+```
+
+This endpoint will be re-generated any time your site it rebuilt.

data/Rakefile

@@ -0,0 +1,2 @@
+require "bundler/gem_tasks"
+

data/jekyll_pages_api.gemspec

@@ -0,0 +1,26 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'jekyll_pages_api/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "jekyll_pages_api"
+  spec.version       = JekyllPagesApi::VERSION
+  spec.authors       = ["Aidan Feldman"]
+  spec.email         = ["aidan.feldman@gsa.gov"]
+  spec.summary       = %q{A Jekyll Plugin that generates a JSON file with data for all the Pages in your Site.}
+  spec.homepage      = "https://github.com/18F/jekyll_pages_api"
+  spec.license       = "CC0-1.0"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "htmlentities", "~> 4.3"
+  spec.add_dependency "jekyll", "~> 2.0"
+
+  spec.add_development_dependency "bundler", "~> 1.7"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+end

data/lib/jekyll/site.rb

@@ -0,0 +1,17 @@
+require 'jekyll/site'
+require_relative '../jekyll_pages_api/generator'
+
+module Jekyll
+  class Site
+    # monkey-patch to make an after_render hook
+    alias_method :orig_render, :render
+    def render
+      orig_render
+      after_render
+    end
+
+    def after_render
+      JekyllPagesApi::Generator.new(self).generate
+    end
+  end
+end

data/lib/jekyll_pages_api/filters.rb

@@ -0,0 +1,38 @@
+require 'htmlentities'
+require 'liquid'
+
+module JekyllPagesApi
+  # This is a hack to allow the module functions to be used
+  class Filters
+    include Liquid::StandardFilters
+
+    def decode_html(str)
+      html_decoder.decode(str)
+    end
+
+    # Slight tweak of
+    # https://github.com/Shopify/liquid/blob/v2.6.1/lib/liquid/standardfilters.rb#L71-L74
+    # to replace newlines with spaces.
+    def condense(str)
+      str.to_s.gsub(/\s+/m, ' '.freeze).strip
+    end
+
+    def text_only(str)
+      # apply each filter in order
+      [
+        :strip_html,
+        :condense,
+        :decode_html
+      ].reduce(str) do |result, filter|
+        self.send(filter, result)
+      end
+    end
+
+
+    private
+
+    def html_decoder
+      @html_decoder = HTMLEntities.new
+    end
+  end
+end

data/lib/jekyll_pages_api/generator.rb

@@ -0,0 +1,44 @@
+require_relative 'page'
+require_relative 'page_without_a_file'
+
+module JekyllPagesApi
+  class Generator
+    attr_reader :site
+
+    def initialize(site)
+      @site = site
+    end
+
+    def pages
+      self.site.pages.map{|page| Page.new(page) }.select(&:html?)
+    end
+
+    def pages_data
+      self.pages.map(&:to_json)
+    end
+
+    def data
+      {
+        entries: pages_data
+      }
+    end
+
+    def dest_dir
+      File.join('api', 'v1')
+    end
+
+    def page
+      # based on https://github.com/jekyll/jekyll-sitemap/blob/v0.7.0/lib/jekyll-sitemap.rb#L51-L54
+      page = PageWithoutAFile.new(self.site, File.dirname(__FILE__), self.dest_dir, 'pages.json')
+      page.content = self.data.to_json
+      page.data['layout'] = nil
+      page.render(Hash.new, self.site.site_payload)
+
+      page
+    end
+
+    def generate
+      self.site.pages << self.page
+    end
+  end
+end

data/lib/jekyll_pages_api/page.rb

@@ -0,0 +1,44 @@
+require_relative 'filters'
+
+module JekyllPagesApi
+  # wrapper for a Jekyll::Page
+  class Page
+    attr_reader :page
+
+    def initialize(page)
+      @page = page
+    end
+
+    def html?
+      %w(.html .md).include?(self.page.ext)
+    end
+
+    def filterer
+      @filterer ||= Filters.new
+    end
+
+    def title
+      self.filterer.decode_html(self.page.data['title'] || '')
+    end
+
+    def base_url
+      self.page.site.baseurl
+    end
+
+    def url
+      [self.base_url, self.page.url].join
+    end
+
+    def body_text
+      self.filterer.text_only(self.page.content)
+    end
+
+    def to_json
+      {
+        title: self.title,
+        url: self.url,
+        body: self.body_text
+      }
+    end
+  end
+end

data/lib/jekyll_pages_api/page_without_a_file.rb

@@ -0,0 +1,10 @@
+require 'jekyll'
+
+# https://github.com/jekyll/jekyll-sitemap/blob/v0.7.0/lib/jekyll-sitemap.rb#L3-L8
+module JekyllPagesApi
+  class PageWithoutAFile < Jekyll::Page
+    def read_yaml(*)
+      @data ||= {}
+    end
+  end
+end

data/lib/jekyll_pages_api/version.rb

@@ -0,0 +1,3 @@
+module JekyllPagesApi
+  VERSION = '0.1.0'
+end

data/lib/jekyll_pages_api.rb

@@ -0,0 +1,2 @@
+require 'jekyll_pages_api/version'
+require_relative 'jekyll/site'

data/spec/filters_spec.rb

@@ -0,0 +1,7 @@
+describe JekyllPagesApi::Filters do
+  describe '#condense' do
+    it "removes line breaks" do
+      expect(subject.condense("foo\n   bar")).to eq('foo bar')
+    end
+  end
+end

data/spec/integration_spec.rb

@@ -0,0 +1,62 @@
+require 'json'
+require_relative 'support/shell'
+
+describe "integration" do
+  BUILD_DIR = File.join(Dir.pwd, 'spec', 'site')
+  JSON_PATH = File.join(BUILD_DIR, '_site', 'api', 'v1', 'pages.json')
+
+  def read_json(path)
+    contents = File.read(path)
+    JSON.parse(contents)
+  end
+
+  def entries_data
+    json = read_json(JSON_PATH)
+    json['entries']
+  end
+
+  def homepage_data
+    entries_data.find{|page| page['url'] == '/' }
+  end
+
+  before(:context) do
+    # http://bundler.io/man/bundle-exec.1.html#Shelling-out
+    Bundler.with_clean_env do
+      Dir.chdir(BUILD_DIR) do
+        run_cmd('bundle')
+        run_cmd('bundle exec jekyll build')
+      end
+    end
+  end
+
+  it "generates the JSON file" do
+    expect(File.exist?(JSON_PATH)).to be_truthy
+  end
+
+  it "includes an entry for every page" do
+    urls = entries_data.map{|page| page['url'] }
+    expect(urls).to eq(%w(
+      /about/
+      /index.html
+    ))
+  end
+
+  it "removes liquid tags" do
+    entries_data.each do |page|
+      expect(page['body']).to_not include('{%')
+      expect(page['body']).to_not include('{{')
+    end
+  end
+
+  it "removes HTML tags" do
+    entries_data.each do |page|
+      expect(page['body']).to_not include('<')
+    end
+  end
+
+  it "condenses the content" do
+    entries_data.each do |page|
+      expect(page['body']).to_not match(/\s{2,}/m)
+    end
+  end
+end

data/spec/page_spec.rb

@@ -0,0 +1,19 @@
+describe JekyllPagesApi::Page do
+  describe '#url' do
+    it "returns the path" do
+      site = instance_double(Jekyll::Site, baseurl: Jekyll::Configuration::DEFAULTS['baseurl'])
+      jekyll_page = instance_double(Jekyll::Page, site: site, url: '/foo/')
+      page = JekyllPagesApi::Page.new(jekyll_page)
+
+      expect(page.url).to eq('/foo/')
+    end
+
+    it "prepends the baseurl" do
+      site = instance_double(Jekyll::Site, baseurl: '/base')
+      jekyll_page = instance_double(Jekyll::Page, site: site, url: '/foo/')
+      page = JekyllPagesApi::Page.new(jekyll_page)
+
+      expect(page.url).to eq('/base/foo/')
+    end
+  end
+end

data/spec/site/.gitignore

@@ -0,0 +1,3 @@
+_site
+.sass-cache
+Gemfile.lock

data/spec/site/Gemfile

@@ -0,0 +1,7 @@
+source 'https://rubygems.org'
+
+gem 'jekyll'
+
+group :jekyll_plugins do
+  gem 'jekyll_pages_api', path: '../..'
+end

data/spec/site/_config.yml

@@ -0,0 +1,14 @@
+# Site settings
+title: Your awesome title
+email: your-email@domain.com
+description: > # this means to ignore newlines until "baseurl:"
+  Write an awesome description for your new site here. You can edit this
+  line in _config.yml. It will appear in your document head meta (for
+  Google search results) and in your feed.xml site description.
+baseurl: "" # the subpath of your site, e.g. /blog/
+url: "http://yourdomain.com" # the base hostname & protocol for your site
+twitter_username: jekyllrb
+github_username:  jekyll
+
+# Build settings
+markdown: kramdown

data/spec/site/_includes/footer.html

@@ -0,0 +1,55 @@
+<footer class="site-footer">
+
+  <div class="wrapper">
+
+    <h2 class="footer-heading">{{ site.title }}</h2>
+
+    <div class="footer-col-wrapper">
+      <div class="footer-col  footer-col-1">
+        <ul class="contact-list">
+          <li>{{ site.title }}</li>
+          <li><a href="mailto:{{ site.email }}">{{ site.email }}</a></li>
+        </ul>
+      </div>
+
+      <div class="footer-col  footer-col-2">
+        <ul class="social-media-list">
+          {% if site.github_username %}
+          <li>
+            <a href="https://github.com/{{ site.github_username }}">
+              <span class="icon  icon--github">
+                <svg viewBox="0 0 16 16">
+                  <path fill="#828282" d="M7.999,0.431c-4.285,0-7.76,3.474-7.76,7.761 c0,3.428,2.223,6.337,5.307,7.363c0.388,0.071,0.53-0.168,0.53-0.374c0-0.184-0.007-0.672-0.01-1.32 c-2.159,0.469-2.614-1.04-2.614-1.04c-0.353-0.896-0.862-1.135-0.862-1.135c-0.705-0.481,0.053-0.472,0.053-0.472 c0.779,0.055,1.189,0.8,1.189,0.8c0.692,1.186,1.816,0.843,2.258,0.645c0.071-0.502,0.271-0.843,0.493-1.037 C4.86,11.425,3.049,10.76,3.049,7.786c0-0.847,0.302-1.54,0.799-2.082C3.768,5.507,3.501,4.718,3.924,3.65 c0,0,0.652-0.209,2.134,0.796C6.677,4.273,7.34,4.187,8,4.184c0.659,0.003,1.323,0.089,1.943,0.261 c1.482-1.004,2.132-0.796,2.132-0.796c0.423,1.068,0.157,1.857,0.077,2.054c0.497,0.542,0.798,1.235,0.798,2.082 c0,2.981-1.814,3.637-3.543,3.829c0.279,0.24,0.527,0.713,0.527,1.437c0,1.037-0.01,1.874-0.01,2.129 c0,0.208,0.14,0.449,0.534,0.373c3.081-1.028,5.302-3.935,5.302-7.362C15.76,3.906,12.285,0.431,7.999,0.431z"/>
+                </svg>
+              </span>
+
+              <span class="username">{{ site.github_username }}</span>
+            </a>
+          </li>
+          {% endif %}
+
+          {% if site.twitter_username %}
+          <li>
+            <a href="https://twitter.com/{{ site.twitter_username }}">
+              <span class="icon  icon--twitter">
+                <svg viewBox="0 0 16 16">
+                  <path fill="#828282" d="M15.969,3.058c-0.586,0.26-1.217,0.436-1.878,0.515c0.675-0.405,1.194-1.045,1.438-1.809
+                  c-0.632,0.375-1.332,0.647-2.076,0.793c-0.596-0.636-1.446-1.033-2.387-1.033c-1.806,0-3.27,1.464-3.27,3.27 c0,0.256,0.029,0.506,0.085,0.745C5.163,5.404,2.753,4.102,1.14,2.124C0.859,2.607,0.698,3.168,0.698,3.767 c0,1.134,0.577,2.135,1.455,2.722C1.616,6.472,1.112,6.325,0.671,6.08c0,0.014,0,0.027,0,0.041c0,1.584,1.127,2.906,2.623,3.206 C3.02,9.402,2.731,9.442,2.433,9.442c-0.211,0-0.416-0.021-0.615-0.059c0.416,1.299,1.624,2.245,3.055,2.271 c-1.119,0.877-2.529,1.4-4.061,1.4c-0.264,0-0.524-0.015-0.78-0.046c1.447,0.928,3.166,1.469,5.013,1.469 c6.015,0,9.304-4.983,9.304-9.304c0-0.142-0.003-0.283-0.009-0.423C14.976,4.29,15.531,3.714,15.969,3.058z"/>
+                </svg>
+              </span>
+
+              <span class="username">{{ site.twitter_username }}</span>
+            </a>
+          </li>
+          {% endif %}
+        </ul>
+      </div>
+
+      <div class="footer-col  footer-col-3">
+        <p class="text">{{ site.description }}</p>
+      </div>
+    </div>
+
+  </div>
+
+</footer>

data/spec/site/_includes/head.html

@@ -0,0 +1,12 @@
+<head>
+  <meta charset="utf-8">
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+
+  <title>{% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}</title>
+  <meta name="description" content="{% if page.excerpt %}{{ page.excerpt | strip_html | strip_newlines | truncate: 160 }}{% else %}{{ site.description }}{% endif %}">
+
+  <link rel="stylesheet" href="{{ "/css/main.css" | prepend: site.baseurl }}">
+  <link rel="canonical" href="{{ page.url | replace:'index.html','' | prepend: site.baseurl | prepend: site.url }}">
+  <link rel="alternate" type="application/rss+xml" title="{{ site.title }}" href="{{ "/feed.xml" | prepend: site.baseurl | prepend: site.url }}" />
+</head>

data/spec/site/_includes/header.html

@@ -0,0 +1,27 @@
+<header class="site-header">
+
+  <div class="wrapper">
+
+    <a class="site-title" href="{{ site.baseurl }}/">{{ site.title }}</a>
+
+    <nav class="site-nav">
+      <a href="#" class="menu-icon">
+        <svg viewBox="0 0 18 15">
+          <path fill="#424242" d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.031C17.335,0,18,0.665,18,1.484L18,1.484z"/>
+          <path fill="#424242" d="M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0c0-0.82,0.665-1.484,1.484-1.484 h15.031C17.335,6.031,18,6.696,18,7.516L18,7.516z"/>
+          <path fill="#424242" d="M18,13.516C18,14.335,17.335,15,16.516,15H1.484C0.665,15,0,14.335,0,13.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.031C17.335,12.031,18,12.696,18,13.516L18,13.516z"/>
+        </svg>
+      </a>
+
+      <div class="trigger">
+        {% for page in site.pages %}
+          {% if page.title %}
+          <a class="page-link" href="{{ page.url | prepend: site.baseurl }}">{{ page.title }}</a>
+          {% endif %}
+        {% endfor %}
+      </div>
+    </nav>
+
+  </div>
+
+</header>

data/spec/site/_layouts/default.html

@@ -0,0 +1,20 @@
+<!DOCTYPE html>
+<html>
+
+  {% include head.html %}
+
+  <body>
+
+    {% include header.html %}
+
+    <div class="page-content">
+      <div class="wrapper">
+        {{ content }}
+      </div>
+    </div>
+
+    {% include footer.html %}
+
+  </body>
+
+</html>

data/spec/site/_layouts/page.html

@@ -0,0 +1,14 @@
+---
+layout: default
+---
+<div class="post">
+
+  <header class="post-header">
+    <h1 class="post-title">{{ page.title }}</h1>
+  </header>
+
+  <article class="post-content">
+    {{ content }}
+  </article>
+
+</div>

data/spec/site/_layouts/post.html

@@ -0,0 +1,15 @@
+---
+layout: default
+---
+<div class="post">
+
+  <header class="post-header">
+    <h1 class="post-title">{{ page.title }}</h1>
+    <p class="post-meta">{{ page.date | date: "%b %-d, %Y" }}{% if page.author %} • {{ page.author }}{% endif %}{% if page.meta %} • {{ page.meta }}{% endif %}</p>
+  </header>
+
+  <article class="post-content">
+    {{ content }}
+  </article>
+
+</div>

data/spec/site/_posts/2015-01-26-welcome-to-jekyll.markdown

@@ -0,0 +1,25 @@
+---
+layout: post
+title:  "Welcome to Jekyll!"
+date:   2015-01-26 02:53:40
+categories: jekyll update
+---
+You’ll find this post in your `_posts` directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different ways, but the most common way is to run `jekyll serve`, which launches a web server and auto-regenerates your site when a file is updated.
+
+To add new posts, simply add a file in the `_posts` directory that follows the convention `YYYY-MM-DD-name-of-post.ext` and includes the necessary front matter. Take a look at the source for this post to get an idea about how it works.
+
+Jekyll also offers powerful support for code snippets:
+
+{% highlight ruby %}
+def print_hi(name)
+  puts "Hi, #{name}"
+end
+print_hi('Tom')
+#=> prints 'Hi, Tom' to STDOUT.
+{% endhighlight %}
+
+Check out the [Jekyll docs][jekyll] for more info on how to get the most out of Jekyll. File all bugs/feature requests at [Jekyll’s GitHub repo][jekyll-gh]. If you have questions, you can ask them on [Jekyll’s dedicated Help repository][jekyll-help].
+
+[jekyll]:      http://jekyllrb.com
+[jekyll-gh]:   https://github.com/jekyll/jekyll
+[jekyll-help]: https://github.com/jekyll/jekyll-help