congress_forms 0.1.1

data/README.md

@@ -0,0 +1,152 @@
+# CongressForms
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'congress_forms'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install congress_forms
+
+
+### Program Dependencies
+
+  * google-chrome, git
+
+
+## Usage
+
+To send a message to Congress, begin by creating a form object. Senators should be identified by their [BioGuide ID](https://www.congress.gov/help/field-values/member-bioguide-ids), while Representatives are identified by an office code H*XXYY*, where XX is their state and YY is their district.
+
+```ruby
+# Form for Senator Kamala Harris (BioGuide H001075)
+CongressForms::Form.find("H001075")
+
+# Form for CA-13 Represenative Barbara Lee (office code HCA13)
+CongressForms::Form.find("HCA13")
+```
+
+Each Senator's office may require different fields, provide different options for select menus, etc. You can query the fields required by a particular form by calling `CongressForms::Form#required_params`.
+
+```ruby
+# List required message parameters
+irb(main)> CongressForms::Form.find("H001075").required_params
+[
+  # Required text fields
+  { :value => "$NAME_FIRST", :max_length => nil },
+  { :value => "$NAME_LAST", :max_length => nil },
+
+  # Required multiple choice field
+  { :value => "$NAME_PREFIX", :options => ["Mr.", "Ms.", "Mrs.", ...] },
+
+  # Required multiple choice field with distinct labels and values
+  {
+    :value => "$TOPIC", :options => {
+      "Abortion" => "943AD4D7-5056-A066-60A5-D652A671D70E",
+      "Agriculture" => "943AD58A-5056-A066-60BD-A9DBEE1187A1",
+      "Animal Welfare" => "943AD622-5056-A066-6065-1B45E2F6F45D",
+    }
+  },
+ ...
+]
+...
+```
+
+Pass the required values, in a hash, to `CongressForms::Form#fill` to send the message.
+
+```ruby
+form = CongressForms::Form.find("H001075")
+
+form.fill(
+  "$NAME_FIRST" => "...",
+  "$NAME_LAST" => "...",
+  "$MESSAGE" => "...",
+  ...
+)
+```
+
+For Senate offices, this will fill out the representative's contact form with a headless instance of Google Chrome. For House offices, messages are submitted through the Communicating with Congress (CWC) API.
+
+
+### CLI Usage
+
+You can also send messages from the command line:
+
+```
+$ bin/congress_forms --help
+Usage: congress_forms [options]
+    -i, --rep_id REP_ID              ID of the representative to message
+    -r, --repo DIR                   Location for unitedstates/contact_congress repository
+    -p, --param KEY=VALUE            e.g. -p NAME_FIRST=Badger
+```
+
+
+## Operation and Configuration
+
+Senate messages rely on contact form details tracked by the [unitedstates/contact-congress](https://github.com/unitedstates/contact-congress) project. This repo is cloned into a temporary directory by default. You can configure CongressForms to use an existing/persistent direcory with
+
+```ruby
+CongressForms.contact_congress_repository = "data/contact_congress"
+```
+
+A `git pull` is performed every now and then in this direcory, to keep the form details up to date. You can disable this behavior with
+
+```ruby
+CongressForms.auto_update_contact_congress = false
+```
+
+House messages are submitted through the [Communicating with Congress](https://www.house.gov/doing-business-with-the-house/communicating-with-congress-cwc) API. You will need to complete the vendor application process, then configure the API client with
+
+```ruby
+Cwc::Client.configure(
+  api_key: ENV["CWC_API_KEY"],
+  host: ENV["CWC_HOST"],
+  delivery_agent: ENV["CWC_DELIVERY_AGENT"],
+  delivery_agent_ack_email: ENV["CWC_DELIVERY_AGENT_ACK_EMAIL"],
+  delivery_agent_contact_name: ENV["CWC_DELIVERY_AGENT_CONTACT_NAME"],
+  delivery_agent_contact_email: ENV["CWC_DELIVERY_AGENT_CONTACT_EMAIL"],
+  delivery_agent_contact_phone: ENV["CWC_DELIVERY_AGENT_CONTACT_PHONE"]
+)
+```
+
+### CWC Concerns
+
+The CWC API requires that you connect from a whitelisted IP address. This is true even for the test endpoint, which makes development and testing of the API client tricky.
+
+If you have a whitelisted IP, you can use SSH port forwarding to tunnel requests to CWC through the approved server. Keep this command running in a console:
+
+```
+$ ssh -L [port]:test-cwc.house.gov:443 [server]
+```
+
+Use `https://localhost:[port]/` as your CWC host, and define these environment variables:
+
+```
+CWC_VERIFY_SSL=false
+CWC_HOST_HEADER=test-cwc.house.gov
+```
+
+(substitute `[server]` and `[port]` with your own values)
+
+### Disabling headless mode
+
+Chrome can be run in windowed mode by setting the environment variable `HEADLESS=0`.
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/efforg/congress_forms.
+
+
+## License
+
+The gem is available as open source under the terms of the [GPLv3 License](https://github.com/EFForg/congress_forms/blob/master/LICENSE.txt).
+

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/congress_forms

@@ -0,0 +1,100 @@
+#!/usr/bin/env ruby
+
+require "optparse"
+
+require "bundler/setup"
+require "congress_forms"
+
+require "dotenv/load"
+
+unless ENV["CWC_API_KEY"].nil?
+  Cwc::Client.configure(
+    api_key: ENV["CWC_API_KEY"],
+    host: ENV["CWC_HOST"],
+    delivery_agent: ENV["CWC_DELIVERY_AGENT"],
+    delivery_agent_ack_email: ENV["CWC_DELIVERY_AGENT_ACK_EMAIL"],
+    delivery_agent_contact_name: ENV["CWC_DELIVERY_AGENT_CONTACT_NAME"],
+    delivery_agent_contact_email: ENV["CWC_DELIVERY_AGENT_CONTACT_EMAIL"],
+    delivery_agent_contact_phone: ENV["CWC_DELIVERY_AGENT_CONTACT_PHONE"]
+  )
+end
+
+options, params = { submit: true }, {}
+
+ENV.each do |k, v|
+  next unless m = k.match(/CONGRESS_FORMS_(.+)/)
+  params["$#{m[1]}"] = v
+end
+
+opts = OptionParser.new do |opts|
+  opts.on("--rep_id REP_ID", "-i", "ID of the representative to message") do |id|
+    options[:rep_id] = id
+  end
+
+  opts.on("--repo DIR", "-r", "Location for unitedstates/contact_congress repository") do |dir|
+    CongressForms.contact_congress_repository = dir
+  end
+
+  opts.on("--param KEY=VALUE", "-p", "e.g. -p NAME_FIRST=Badger") do |pair|
+    key, value = pair.split("=", 2)
+    params["$#{key}"] = value
+  end
+
+  opts.on("--no-submit", "Fill out the form without submitting") do
+    options[:submit] = false
+  end
+
+  opts.on("--debug", "Debug mode (HEADLESS=0, binding.pry if an exception is raised)") do
+    ENV["HEADLESS"] = "0"
+    options[:debug] = true
+  end
+end.tap(&:parse!)
+
+unless options[:rep_id]
+  warn opts.help
+  exit(1)
+end
+
+form = CongressForms::Form.find(options[:rep_id])
+
+form.required_params.each do |param|
+  next if params[param[:value]]
+
+  print("#{param[:value]}: ")
+
+  if param[:options]
+    choices =
+      if param[:options].is_a?(Array)
+        param[:options].zip(param[:options]).to_h
+      else
+        param[:options]
+      end
+
+    puts
+    choices.each_with_index do |choice, i|
+      puts("#{i+1}. #{choice[0]}")
+    end
+
+    puts
+    print("Choice: ")
+    i = $stdin.gets or exit(1)
+
+    choice = choices.to_a[i.to_i-1][0]
+    params[param[:value]] = choices[choice]
+    puts
+  else
+    params[param[:value]] = $stdin.gets or exit(1)
+    params[param[:value]].chomp!
+  end
+end
+
+begin
+  form.fill(params, submit: options[:submit])
+rescue Exception => e
+  if options[:debug]
+    require "pry"
+    binding.pry
+  else
+    raise e
+  end
+end

data/bin/console

@@ -0,0 +1,9 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "dotenv/load"
+
+require "congress_forms"
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/congress_forms.gemspec

@@ -0,0 +1,30 @@
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'congress_forms/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "congress_forms"
+  spec.version       = CongressForms::VERSION
+  spec.authors       = ["Peter Woo"]
+  spec.email         = ["peterw@eff.org"]
+
+  spec.summary       = %q{...}
+  spec.homepage      = "https://github.com/efforg/congress_forms"
+  spec.license       = "GPL-3.0"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = "bin"
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib", "cwc/lib"]
+
+  spec.add_development_dependency "pry", "~> 0.11"
+  spec.add_development_dependency "bundler", "~> 1.12"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.add_development_dependency "dotenv", "~> 2.5"
+
+  spec.add_dependency "capybara-selenium", "~> 0.0.6"
+  spec.add_dependency "chromedriver-helper", "~> 1.2"
+  spec.add_dependency "nokogiri", ">= 1.8.2"
+  spec.add_dependency "rest-client", "~> 2.0"
+end

data/lib/congress_forms/actions.rb

@@ -0,0 +1,160 @@
+module CongressForms
+  module Actions
+    DEFAULT_FIND_WAIT_TIME = 5
+
+    def self.build(step)
+      key = step.keys.first
+
+      const_name = key.capitalize.gsub(/_(\w)/){ |m| m[1].upcase }
+      klass = const_get(const_name, false)
+
+      if Visit == klass
+        Array(klass.new("value" => step[key]))
+      else
+        step[key].map do |params|
+          klass.new(params)
+        end
+      end
+    end
+
+    class Base
+      attr_accessor :selector, :value, :options, :required
+      alias :required? :required
+
+      def initialize(params = {})
+        self.selector = params["selector"]
+        self.value = params["value"]
+        self.options = params["options"] || {}
+        self.required = !!params["required"]
+      end
+
+      def max_length
+        options.is_a?(Hash) ? options["max_length"] : nil
+      end
+
+      def select_options
+        [Choose, Select].include?(self.class) ? options : nil
+      end
+
+      def placeholder_value?
+        value[0, 1] == "$"
+      end
+
+      def escape_css_attribute(v)
+        v.gsub('"', '\"')
+      end
+
+      def submit?
+        "#{value} #{selector}".match(/submit/i)
+      end
+
+      def inspect
+        s = "#{self.class.name.sub(/^CongressForms::Actions::/, '')}("
+        s << "#{selector.inspect}, " unless selector.nil?
+        s << value.inspect << ")"
+      end
+    end
+
+    class Visit < Base
+      def perform(browser, params={})
+        browser.visit(value)
+      end
+    end
+
+    class Wait < Base
+      def perform(browser, params={})
+        sleep(value.to_i)
+      end
+    end
+
+    class FillIn < Base
+      def perform(browser, params={})
+        if placeholder_value?
+          value = params.fetch(self.value).gsub("\t", "    ")
+
+          maxl = options["max_length"]
+          value = value[0, (0.95 * maxl).floor] if maxl
+        else
+          value = self.value
+        end
+
+        browser.find(selector).set(value)
+      end
+    end
+
+    class Select < Base
+      def perform(browser, params={})
+        user_value = params[value]
+
+        browser.within(selector) do
+          if placeholder_value?
+            option_value = user_value
+          else
+            option_value = value
+          end
+
+          begin
+            elem = browser.first('option[value="' + escape_css_attribute(option_value) + '"]')
+          rescue Capybara::ElementNotFound
+            elem = browser.first('option', text: Regexp.compile("^" + Regexp.escape(option_value) + "(\\W|$)"))
+          end
+
+          elem.select_option
+        end
+      rescue Capybara::ElementNotFound => e
+        raise e, e.message unless options == "DEPENDENT"
+      end
+    end
+
+    class ClickOn < Base
+      def perform(browser, params={})
+        browser.find(selector).click
+      end
+    end
+
+    class Find < Base
+      def perform(browser, params={})
+        wait_val = options["wait"] || DEFAULT_FIND_WAIT_TIME
+
+        if value.nil?
+          browser.find(selector, wait: wait_val)
+        else
+          browser.find(selector, text: Regexp.compile(value),
+                                 wait: wait_val)
+        end
+      end
+    end
+
+    class Check < Base
+      def perform(browser, params={})
+        browser.find(selector).set(true)
+      end
+    end
+
+    class Uncheck < Base
+      def perform(browser, params={})
+        browser.find(selector).set(false)
+      end
+    end
+
+    class Choose < Base
+      def perform(browser, params={})
+        if options.any?
+          user_value = params[value]
+
+          browser.
+            find(selector + '[value="' + escape_css_attribute(user_value) + '"]').
+            set(true)
+        else
+          browser.find(selector).set(true)
+        end
+      end
+    end
+
+    class Javascript < Base
+      def perform(browser, params={})
+        browser.driver.evaluate_script(value)
+      end
+    end
+  end
+end

data/lib/congress_forms/cwc_form.rb

@@ -0,0 +1,62 @@
+module CongressForms
+  class CwcForm < Form
+    attr_accessor :office_code
+
+    def initialize(office_code)
+      self.office_code = office_code
+    end
+
+    def updated_at
+      Time.at(0)
+    end
+
+    def required_params
+      Cwc::RequiredJson.fetch("required_actions").map(&:dup)
+    end
+
+    def fill(values, campaign_tag: nil, organization: nil, browser: nil, validate_only: false)
+      params = {
+        campaign_id: campaign_tag || SecureRandom.hex(16),
+
+        recipient: { member_office: office_code },
+
+        constituent: {
+          prefix:	      values["$NAME_PREFIX"],
+          first_name:	      values["$NAME_FIRST"],
+          last_name:	      values["$NAME_LAST"],
+          address:	      Array(values["$ADDRESS_STREET"]),
+          city:		      values["$ADDRESS_CITY"],
+          state_abbreviation: values["$ADDRESS_STATE_POSTAL_ABBREV"],
+          zip:		      values["$ADDRESS_ZIP5"],
+          email:	      values["$EMAIL"]
+        },
+
+        message: {
+          subject:                    values["$SUBJECT"],
+          library_of_congress_topics: Array(values["$TOPIC"])
+        }
+      }
+
+      if organization
+        params[:organization] = organization
+      end
+
+      if values["$STATEMENT"]
+        params[:message][:organization_statement] = values["$STATEMENT"]
+      end
+
+      if values["$MESSAGE"] && values["$MESSAGE"] != values["$STATEMENT"]
+        params[:message][:constituent_message] = values["$MESSAGE"]
+      end
+
+      cwc_client = Cwc::Client.new
+      message = cwc_client.create_message(params)
+
+      if validate_only
+        cwc_client.validate(message)
+      else
+        cwc_client.deliver(message)
+      end
+    end
+  end
+end

data/lib/congress_forms/form.rb

@@ -0,0 +1,36 @@
+module CongressForms
+  class Form
+    @@repo = nil
+
+    def self.repo
+      @@repo ||=
+        Repo.new(CongressForms.contact_congress_remote).tap do |repo|
+          repo.location = CongressForms.contact_congress_repository
+          repo.auto_update = CongressForms.auto_update_contact_congress?
+        end
+    end
+
+    def self.find(form_id)
+      if Cwc::Client.new.office_supported?(form_id)
+        CwcForm.new(form_id)
+      else
+        content, timestamp = repo.find("members/#{form_id}.yaml")
+        WebForm.parse(content, updated_at: timestamp)
+      end
+    rescue Errno::ENOENT => e
+      nil
+    end
+
+    def missing_required_params(params)
+      missing_parameters = []
+
+      required_params.each do |field|
+        unless params.include?(field[:value])
+          missing_parameters << field[:value]
+        end
+      end
+
+      missing_parameters.empty? ? nil : missing_parameters.any
+    end
+  end
+end

data/lib/congress_forms/repo.rb

@@ -0,0 +1,95 @@
+require "tmpdir"
+require "fileutils"
+
+module CongressForms
+  class Repo
+    attr_reader :remote
+    attr_accessor :auto_update
+
+    alias :auto_update? :auto_update
+
+    def initialize(remote)
+      @remote = remote
+      @semaphore = Mutex.new
+      self.auto_update = true
+    end
+
+    def location
+      @location ||= Pathname.new(Dir.mktmpdir).tap do |tmpdir|
+        Kernel.at_exit{ FileUtils.rm_r(tmpdir) }
+      end
+    end
+
+    def location=(loc)
+      @location = loc ? Pathname.new(loc) : nil
+    end
+
+    def clone
+      system(
+        "git",
+        "clone",
+        "--quiet",
+        "--depth", "1",
+        remote,
+        location.to_s
+      ) or raise Error, "Error cloning repo at #{remote}"
+    end
+
+    def initialized?
+      File.exists?(git_dir)
+    end
+
+    def update!
+      system(
+        "git",
+        "--git-dir", git_dir.to_s,
+        "pull",
+        "--quiet",
+        "--ff-only"
+      ) or raise Error, "Error updating git repo at #{location}"
+    end
+
+    def update
+      begin
+        update!
+      rescue
+      end
+    end
+
+    def age
+      repo_touched_at = File.mtime(git_dir.join("HEAD"))
+      Time.now - repo_touched_at
+    end
+
+    def find(file)
+      lock do
+        clone unless initialized?
+
+        update if auto_update? && age > 5*60 # update every 5m
+
+        repo_file = system(
+          "git",
+          "--git-dir", git_dir.to_s,
+          "ls-files", "--error-unmatch",
+          "--", file
+        )
+
+        raise Errno::ENOENT, file unless repo_file
+
+        path = location.join(file).to_s
+
+        [File.read(path), File.mtime(path)]
+      end
+    end
+
+    def git_dir
+      location.join(".git")
+    end
+
+    private
+
+    def lock(&block)
+      @semaphore.synchronize(&block)
+    end
+  end
+end

data/lib/congress_forms/version.rb

@@ -0,0 +1,3 @@
+module CongressForms
+  VERSION = "0.1.1"
+end