extended_email_reply_parser 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 57e1fcc1b76d54e57566cf2c775967d99cc7a991
+  data.tar.gz: 52f31cfb6238fe4c2322b4bef77b0e7cc63c2ed7
+SHA512:
+  metadata.gz: b1a103c7bd67ec89fa1d0cbdf9f15a10f50a733aa9f58ad4b2d960abded59759732fb9b237834e05042be006c38ce76c43ef553a34a09fa430dbfa368c2e0011
+  data.tar.gz: cd1994764092fc0c9b45b6410fbf4d5e5187595fe400f1d238dc28f7abd9b8b81e8fe9daa097e5e19c3fbb7ebc88364ee2795f87a436c33a2f895a4ffe5787cd

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.1
+before_install: gem install bundler -v 1.12.5

data/CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+This project adheres to [Semantic Versioning](http://semver.org/).
+
+## master (unreleased)
+### Security
+### Added
+### Deprecated
+### Removed
+### Fixed
+
+## ExtendedEmailReplyParser 0.1.0 (2016-07-22)
+### Added
+- `ExtendedEmailReplyParser.read "/path/to/email.eml"` returns the corresponding `Mail::Message` object.
+- `ExtendedEmailReplyParser.read("/path/to/email.eml").extract_text` returns the email body text in utf-8. See also: http://stackoverflow.com/a/15818886/2066546
+- Methods for parsing emails: `ExtendedEmailReplyParser.parse("/path/to/email.eml")`, `ExtendedEmailReplyParser.parse(message)`, `ExtendedEmailReplyParser.parse(body_text)`, `Mail::Message#parse`.
+- Allowing to chain custom parsers. A parser inherits from `ExtendedEmailReplyParser::Parsers::Base` and needs to implement a `parse` method. The parser is automatically chained along with the others when calling `ExtendedEmailReplyParser.parse` or `MailMessage#parse`.
+- Wrapping the original [github/email_reply_parser](https://github.com/github/email_reply_parser) in `ExtendedEmailReplyParser::Parsers::Github`.

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in extended_email_reply_parser.gemspec
+gemspec

data/MIT-LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Sebastian Fiedlschuster
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,116 @@
+# ExtendedEmailReplyParser
+
+[![Build Status](https://travis-ci.org/fiedl/extended_email_reply_parser.svg?branch=master)](https://travis-ci.org/fiedl/extended_email_reply_parser)
+
+When implementing a "reply or comment by email" feature, it's neccessary to filter out signatures and the previous conversation. One needs to extract just the relevant parts for the conversation or comment section of the application. This is what this [ruby](https://www.ruby-lang.org) gem helps to do.
+
+This gem is an extended version of [github's `email_reply_parser`](https://github.com/github/email_reply_parser). It wraps the original email_reply_parser and allows to build extensions such as support for i18n and detecting previous conversation that is not properly marked as quotation by the sender's mail client.
+
+## Usage
+
+### Parsing incoming emails
+
+To extract the relevant text of an email reply, call `ExtendedEmailReplyParser.parse`, which accepts either a path to the email file, or a `Mail::Message` object, or the email body text itself, and returns the parsed, i.e. relevant text, which can be used as comment in the application
+
+```ruby
+ExtendedEmailReplyParser.parse "/path/to/email.eml"
+ExtendedEmailReplyParser.parse message
+ExtendedEmailReplyParser.parse body_text             # => parsed text as String
+```
+
+Or, if you prefer to call `#parse` on the `Mail::Message`:
+
+```ruby
+message.parse  # => parsed text as String
+```
+
+For example, for a incoming `Mail::Message`, `message`:
+
+```ruby
+@comment.author = User.where(email: message.from.first).first
+@comment.text = ExtendedEmailReplyParser.parse message
+@comment.save!
+```
+
+### Extracting the body text
+
+This gem extends [Mail::Message](https://github.com/mikel/mail/blob/master/lib/mail/message.rb) to  conveniently extract the text body as utf-8.
+
+```ruby
+message.extract_text
+```
+
+There's also a convenience method to extract the body text from an email file:
+
+```ruby
+ExtendedEmailReplyParser.read "/path/to/email.eml"  # => Mail::Message
+ExtendedEmailReplyParser.read("/path/to/email.eml").extract_text  # => String
+```
+
+### Writing a parser
+
+The parsing system allows you to add your own parser to the parsing chain. Just define a class inheriting from `ExtendedEmailReplyParser::Parsers::Base` and implement a `parse` method. The text before parsing is accessed via `text`.
+
+```ruby
+# app/models/email_parsers/my_parser.rb
+class EmailParsers::ShoutParser < ExtendedEmailReplyParser::Parsers::Base
+
+  # This parser SHOUTS THE WHOLE EMAIL.
+  # The text before parsing is accessed by `text`.
+  #
+  def parse
+    text.upcase
+  end
+end
+```
+
+### Selecting parsers to use
+
+By default, all parsers that inherit from `ExtendedEmailReplyParser::Parsers::Base` are used. One simply has to call:
+
+```ruby
+ExtendedEmailReplyParser.parse message
+```
+
+In order to select specific parsers, just chain them yourself:
+
+```ruby
+EmailParsers::ShoutParser.parse \
+  ExtendedEmailReplyParser::Parsers::Github.parse \
+  message
+```
+
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+# Gemfile
+gem 'extended_email_reply_parser'
+```
+
+And then execute:
+
+    ➜ bundle
+
+Or install it yourself as:
+
+    ➜ gem install extended_email_reply_parser
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/fiedl/extended_email_reply_parser.
+
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](MIT-LICENSE).
+

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "extended_email_reply_parser"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/extended_email_reply_parser.gemspec

@@ -0,0 +1,35 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'extended_email_reply_parser/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "extended_email_reply_parser"
+  spec.version       = ExtendedEmailReplyParser::VERSION
+  spec.authors       = ["Sebastian Fiedlschuster"]
+  spec.email         = ["sebastian@fiedlschuster.de"]
+
+  spec.summary       = "This is an extended version of github's email_reply_parser."
+  spec.description   = spec.description
+  spec.homepage      = "https://github.com/fiedl/extended_email_reply_parser"
+  spec.license       = "MIT"
+
+  if spec.respond_to?(:metadata)
+    spec.metadata['allowed_push_host'] = "https://rubygems.org"
+  else
+    raise "RubyGems 2.0 or newer is required to protect against public gem pushes."
+  end
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.12"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.add_development_dependency 'pry'
+
+  spec.add_dependency 'email_reply_parser', '~> 0.5.9'
+  spec.add_dependency 'mail'
+  spec.add_dependency 'charlock_holmes'
+  spec.add_dependency 'activesupport'
+end

data/lib/extended_email_reply_parser/mail/message.rb

@@ -0,0 +1,28 @@
+# This extends the `Mail::Message` object of https://github.com/mikel/mail
+# in order to allow:
+#
+#     (message.html_part || message.text_part || message).body_in_utf8
+#
+# See also: http://stackoverflow.com/a/15818886/2066546
+#
+module Mail
+  class Message
+    def body_in_utf8
+      require 'charlock_holmes/string'
+      body = self.body.decoded
+      if body.present?
+        encoding = body.detect_encoding[:encoding]
+        body = body.force_encoding(encoding).encode('UTF-8')
+      end
+      return body
+    end
+
+    def extract_text
+      (self.text_part || (self if self.content_type.include?('text/plain'))).body_in_utf8
+    end
+
+    def parse
+      ExtendedEmailReplyParser.parse self
+    end
+  end
+end

data/lib/extended_email_reply_parser/parsers/base.rb

@@ -0,0 +1,19 @@
+module ExtendedEmailReplyParser
+  class Parsers::Base
+
+    attr_accessor :text
+
+    def initialize(text_before_parsing)
+      self.text = text_before_parsing
+    end
+
+    def parse
+      return text
+    end
+
+
+    def self.subclasses
+      ObjectSpace.each_object(Class).select { |klass| klass < self }
+    end
+  end
+end

data/lib/extended_email_reply_parser/parsers/github.rb

@@ -0,0 +1,9 @@
+module ExtendedEmailReplyParser
+  class Parsers::Github < Parsers::Base
+
+    def parse
+      EmailReplyParser.parse_reply text
+    end
+
+  end
+end

data/lib/extended_email_reply_parser/parsers.rb

@@ -0,0 +1,4 @@
+module ExtendedEmailReplyParser
+  module Parsers
+  end
+end

data/lib/extended_email_reply_parser/version.rb

@@ -0,0 +1,3 @@
+module ExtendedEmailReplyParser
+  VERSION = "0.1.0"
+end

data/lib/extended_email_reply_parser.rb

@@ -0,0 +1,73 @@
+require 'mail'
+require 'charlock_holmes'
+require 'email_reply_parser'
+require 'active_support'
+require 'active_support/core_ext/object/blank'
+require 'extended_email_reply_parser/mail/message'
+require 'extended_email_reply_parser/parsers'
+require 'extended_email_reply_parser/parsers/base'
+Dir.glob(File.dirname(File.absolute_path(__FILE__)) + "/extended_email_reply_parser/parsers/*") { |file| require file }
+
+require 'extended_email_reply_parser/version'
+
+# Usage:
+#
+#     ExtendedEmailReplyParser.parse "/path/to/email.eml"
+#     ExtendedEmailReplyParser.parse mail_message
+#     ExtendedEmailReplyParser.parse MyReplyModel.find(123).text_content
+#
+module ExtendedEmailReplyParser
+
+  # Read an email file and return a parsable `Mail::Message` object.
+  # `Mail::Message` is defined in https://github.com/mikel/mail and
+  # slightly extended in this gem.
+  #
+  # Examples:
+  #
+  #     ExtendedEmailReplyParser.read "/path/to/email.eml"
+  #     ExtendedEmailReplyParser.read("/path/to/email.eml").parse
+  #     ExtendedEmailReplyParser.read("/path/to/email.eml").class   # => Mail::Message
+  #
+  # or, maybe, you are looking for this:
+  #
+  #     ExtendedEmailReplyParser.parse "/path/to/email.eml"
+  #
+  def self.read(email_file_path)
+    Mail.read email_file_path
+  end
+
+  # This parses the given object, i.e. removes quoted replies etc.
+  #
+  # Examples:
+  #
+  #     ExtendedEmailReplyParser.parse "/path/to/email.eml"
+  #     ExtendedEmailReplyParser.parse mail_message
+  #     ExtendedEmailReplyParser.parse MyReplyModel.find(123).text_content
+  #
+  def self.parse(object)
+    if object.kind_of? String and File.file? object
+      self.parse_file object
+    elsif object.kind_of? String
+      self.parse_text object
+    elsif object.kind_of? Mail::Message
+      self.parse_message object
+    end
+  end
+
+  def self.parse_file(file_path)
+    self.parse_message Mail.read file_path
+  end
+
+  def self.parse_message(message)
+    self.parse_text message.extract_text
+  end
+
+  def self.parse_text(text)
+    parsed_text = text
+    Parsers::Base.subclasses.each do |parser_class|
+      parsed_text = parser_class.new(parsed_text).parse
+    end
+    parsed_text
+  end
+
+end

metadata

@@ -0,0 +1,174 @@
+--- !ruby/object:Gem::Specification
+name: extended_email_reply_parser
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Sebastian Fiedlschuster
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-07-22 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.12'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: pry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: email_reply_parser
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.5.9
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.5.9
+- !ruby/object:Gem::Dependency
+  name: mail
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: charlock_holmes
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: ''
+email:
+- sebastian@fiedlschuster.de
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".travis.yml"
+- CHANGELOG.md
+- Gemfile
+- MIT-LICENSE
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- extended_email_reply_parser.gemspec
+- lib/extended_email_reply_parser.rb
+- lib/extended_email_reply_parser/mail/message.rb
+- lib/extended_email_reply_parser/parsers.rb
+- lib/extended_email_reply_parser/parsers/base.rb
+- lib/extended_email_reply_parser/parsers/github.rb
+- lib/extended_email_reply_parser/version.rb
+homepage: https://github.com/fiedl/extended_email_reply_parser
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.5.1
+signing_key: 
+specification_version: 4
+summary: This is an extended version of github's email_reply_parser.
+test_files: []