schleuder-gitlab-ticketing 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b59177cf964bf88408cc0d315ba893e8c5a8379fc250b1df318f36c136d1a2b0
+  data.tar.gz: 7b407e3792310e646fe68c949eb7d1b71d15bd617b89e2c85178a29492046cab
+SHA512:
+  metadata.gz: 1b448ce81e1cac8ae9ddeb6fa66d7517bc927fa5e3bafa283caa51fee703aca240870848aa6d205b6a2171614834780e250997770f1f2f90e52885e88270e2d6
+  data.tar.gz: 26164f54b6bca5e1fdfd5abb23208e87281028e4c5798279b2bdb98da3a8df7b3e391b9ed9e7fb6fc57d630cab243be3b629d32d45e6b42593f983ebac5ad7e8

data/README.md

@@ -0,0 +1,191 @@
+Schleuder Gitlab Ticketing
+==========================
+
+A schleuder filter/plugin to hook lists into the issue tracker of a gitlab project.
+
+Background
+----------
+
+Schleuderlists are not only a helpful tool for communication within groups, they can also be
+used for newsletters, or by using its famous remailer capabilities to be used as a contact
+address for a project or as a help desk.
+
+In the latter two cases keeping an overview of the different interactions can become somewhat
+cumbersome, especially in a bigger collective where not everybody is able to dedicate the same
+amount of time/attention to the contact address / help desk. Hence it is easy that a certain
+thread (and so task) on the list is getting lost. Having some kind of ticketing system in place
+can help to easily keep track of what is done, what is in progress and what should be looked at.
+
+Additionally, folks do not like to double the work and hence as much as possible should be done
+while emails flow over the list.
+
+This schleuder filter / plugin allows to map emails through an id in the subject to issues in
+a gitlab project. Additionally, it opens an issue on new emails or closes issues if the subject
+indicates that the job is done.
+
+The gitlab issues are mainly used to keep the state of a certain email thread and contains only
+as much plaintext information as was transmitted in plaintext over the wire. This means it only
+stores From:, Subject: and Message-Id: as issue comments.
+
+How it works
+------------
+
+The filter / plugin hooks into the filter process list in schleuder. First schleuder will check
+whether there is a configuration for the current list, as well as whether this configuration
+is correct. Otherwise the filter is immediately skipped.
+
+Schleuder then dispatches the processing to the specific list instance, which checks whether
+the email comes from a sender that shall be ignored (e.g. well known spammers or announce lists)
+or whether the subject matches any of the configured subject filters. The email is also ignored
+if the well known `X-Spam-Flag` header is set to `yes`.
+
+If none of that matches, the email is being processed and first the email's subject is analyzed
+to get an already existing ticket id. The plugin expects something like `Subject: my subject [gp#123]`,
+where gp is a configureable project identifier and 123 matches an issue id in gitlab. The brackets
+enclose the ticket identifier and can be anywhere within the subject.
+
+If no ticket id can be found or no issue can be found in gitlab a new issue will be created in
+gitlab. The ticket id will be appended to the subject, a faulty ticket id will be replaced.
+
+If a ticket can be found, a comment is added to the issue, that includes the sender email address,
+as well as the Subject and Message-ID of the email.
+
+Additionally, if the email is sent from an email address, that is a member of the gitlab project
+the ticket will be assigned to said user. Furthermore, the label `inprocess` is added to ticket
+to indicate that someone of the project is working on that thread.
+
+If the ticket is already closed it will be re-opened.
+
+If the subject contains: `[ok]` the ticket will be closed and the label `inprocess` removed. An
+already closed ticket stays closed.
+
+That's it.
+
+Prerequisits
+------------
+
+* ruby >=2.1
+* schleuder >= 3.3.0
+* A working schleuder installation and a schleuder list
+* A gitlab project and a dedicated user and its API token
+
+Installation
+------------
+
+* Install the gem (depends on the gitlab gem)
+* Link `lib/schleuder/filters/post_decrpytion/99_gitlab_ticketing.rb` into `/usr/local/lib/schleuder/filters/post_decryption/`,
+  so the plugin gets loaded.
+
+Configuration
+-------------
+
+The plugin is looking for a configuration in `/etc/schleuder/gitlab.yml`. The configuration file
+expects a `gitlab` and a lists `configuration` section:
+
+```
+gitlab:
+  endpoint: https://gitlab.example.com/api/v4
+  token: xxxx
+
+lists:
+  test@schleuder.example.com:
+    project: tickets
+    namespace: support
+```
+
+The key of the lists configuration indicates the listname on schleuder side, which matches its
+email address. A list's setting requires 2 configuration options: `project` & `namespace` which
+map to the path within gitlab, where the project exists. Namespace can either be the group or
+the users name, depending of where a project lives.
+
+The plugin supports working for multiple lists, as well as allows individual gitlab configuration
+for different lists:
+
+```
+gitlab:
+  endpoint: https://gitlab.example.com/api/v4
+  token: xxxx
+lists:
+  test@schleuder.example.com:
+    project: tickets
+    namespace: support
+  test2@schleuder.example.com:
+    gitlab:
+      endpoint: https://gitlab2.example.com/api/v4
+      token: aaaa
+```
+
+Additionally, you can specify filters based on the sender or the subject. They must be valid ruby
+regexps and can also be specified on a global and local level.
+
+```
+gitlab:
+  endpoint: https://gitlab.example.com/api/v4
+  token: xxxx
+
+subject_filters:
+  - '\[announce\]'
+sender_filters:
+  - '^spammer@example\.com$'
+
+lists:
+  test@schleuder.example.com:
+    project: tickets
+    namespace: support
+    subject_filters:
+      - 'ignore me'
+  test2@schleuder.example.com:
+    gitlab:
+      endpoint: https://gitlab2.example.com/api/v4
+      token: aaaa
+    sender_filters:
+      - 'noreply@example\.com'
+
+```
+
+Additionally, you can specify a specifc identifier to more easily identify a particular ticket id in
+the subject:
+
+```
+lists:
+  test@schleuder.example.com:
+    project: tickets
+    namespace: support
+    ticket_prefix: ourid
+```
+
+This will look for the following string in a subject: `[ourid#\d+]`. By default it would look for:
+`[gl/test#\d+]`
+
+Todo
+----
+
+This plugin is in a working state and has been tested by multiple use cases for nearly a year.
+
+Some more ideas:
+
+* Use schleuder's verification capabilities to further prove that the email comes from a list member.
+* Hook into schleuder's plugin capabilities, to allow more ticket actions (e.g. assign, labels, ...)
+
+Contributing
+------------
+
+Please see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+
+Mission statement
+-----------------
+
+Please see [MISSION_STATEMENT.md](MISSION_STATEMENT.md).
+
+
+Code of Conduct
+---------------
+
+We adopted a code of conduct. Please read [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).
+
+
+License
+-------
+
+GNU GPL 3.0. Please see [LICENSE.txt](LICENSE.txt).

data/lib/schleuder-gitlab-ticketing.rb

@@ -0,0 +1,6 @@
+
+module SchleuderGitlabTicketing
+end
+require 'schleuder-gitlab-ticketing/gitlab_config'
+require 'schleuder-gitlab-ticketing/config'
+require 'schleuder-gitlab-ticketing/list'

data/lib/schleuder-gitlab-ticketing/config.rb

@@ -0,0 +1,45 @@
+module SchleuderGitlabTicketing
+  class Config
+    include SchleuderGitlabTicketing::GitlabConfig
+
+    def initialize(config_path='/etc/schleuder/gitlab.yml')
+      @config = if File.exists?(config_path)
+                  YAML.load_file(config_path)
+                else
+                  {}
+                end
+    end
+
+    # returns true if mail was handled
+    # returns 'config-error' if list-config is not properly
+    # returns nil if list is not configured to handle
+    # gitlab plugin
+    def process_list(list_name, mail)
+      if l = lists[list_name]
+        if l.configured?
+          l.process(mail)
+        else
+          'config-error'
+        end
+      else
+        nil
+      end
+    end
+
+    def lists
+      @lists ||= read_lists
+    end
+
+    private
+    def read_lists
+      Hash(@config['lists']).inject({}) do |res,a|
+        n,v = a
+        v['gitlab'] ||= gitlab
+        v['subject_filters'] = Array(v['subject_filters']) + Array(@config['subject_filters'])
+        v['sender_filters'] = Array(v['sender_filters']) + Array(@config['sender_filters'])
+        res[n] = SchleuderGitlabTicketing::List.new(n,v)
+        res
+      end
+    end
+  end
+end

data/lib/schleuder-gitlab-ticketing/gitlab_config.rb

@@ -0,0 +1,11 @@
+module SchleuderGitlabTicketing
+  module GitlabConfig
+    def gitlab
+      @gitlab ||= if @config['gitlab']['endpoint'] && @config['gitlab']['token']
+        Gitlab.client(endpoint: @config['gitlab']['endpoint'], private_token: @config['gitlab']['token'])
+      else
+        nil
+      end
+    end
+  end
+end

data/lib/schleuder-gitlab-ticketing/list.rb

@@ -0,0 +1,198 @@
+require 'yaml'
+require 'gitlab'
+require 'set'
+module SchleuderGitlabTicketing
+  class List
+    include SchleuderGitlabTicketing::GitlabConfig
+
+    def initialize(name, config)
+      @name = name
+      @config = config
+    end
+
+    def configured?
+      unless @config['project'] && @config['namespace'] && @config['gitlab'] && gitlab && project
+        return false
+      end
+      true
+    end
+
+    def process(mail)
+      return false if ignore_mail?(mail)
+      ticket_id = get_ticket_id(mail.subject)
+      ticket = ticket_id ? get_ticket(ticket_id) : nil
+
+      if (title = clean_subject(mail.subject)).nil? || title.empty?
+        title = "Request from #{mail.from.first}"
+      end
+      desc = desc(mail)
+      updates = {}
+      labels = Set.new
+
+      if !ticket
+        ticket = create_ticket(title, desc)
+        ticket_id = ticket.iid
+        mail.subject = update_subject(mail.subject, ticket_id)
+      else
+        labels = Set.new(ticket.labels)
+        comment_ticket(ticket_id,desc)
+        assignee_id = team_member_id(mail.from.first)
+        if assignee_id && ((as = ticket.assignee).nil? || as.id != assignee_id)
+          updates[:assignee_id] = assignee_id
+        end
+      end
+
+      bc = be_closed?(mail.subject)
+      tc = ticket_closed?(ticket)
+
+      if !tc && bc
+        labels.delete('inprocess')
+        updates[:state_event] = 'close'
+      elsif !tc && !bc
+        labels << 'inprocess' if updates[:assignee_id]
+      elsif tc && !bc
+        labels << 'inprocess'
+        updates[:state_event] = 'reopen'
+      end
+      if labels.empty? && (updates[:state_event] == 'close')
+        # make sure we remove the inprocess label
+        updates[:labels] = ''
+      elsif !labels.empty? && (labels.to_a.sort != ticket.labels.sort)
+        updates[:labels] = labels.to_a.join(',')
+      end
+      update_ticket(ticket_id, updates)
+      true
+    end
+
+    def gitlab
+      @gitlab ||= if @config['gitlab'].is_a?(Gitlab::Client)
+        @config['gitlab']
+      else
+        super
+      end
+    end
+
+    def sender_filters
+      Array(@config['sender_filters'])
+    end
+    def subject_filters
+      Array(@config['subject_filters'])
+    end
+
+    private
+    def desc(mail)
+      res = []
+      res << "From: #{mail.from.first}"
+      res << "Message-Id: #{mail.message_id}"
+      res << "Subject: #{mail.subject || '[not set]'}"
+      res.join("\n\n")
+    end
+
+    def be_closed?(subject)
+      !(subject =~ /\[ok\]/).nil?
+    end
+
+    def ticket_closed?(ticket)
+      ticket.state == 'closed'
+    end
+
+    def update_ticket(ticket_id,updates)
+      return if updates.empty?
+      gitlab.edit_issue(project.id, ticket_id, updates)
+    rescue Gitlab::Error::NotFound => e
+      nil
+    end
+
+    def create_ticket(title, desc)
+      opts = { description: desc }
+      gitlab.create_issue(project.id, title, opts)
+    end
+
+    def comment_ticket(ticket_id, desc)
+      gitlab.create_issue_note(project.id, ticket_id, desc)
+    end
+
+    def get_ticket(id)
+      gitlab.issue(project.id, id)
+    rescue Gitlab::Error::NotFound => e
+      nil
+    end
+
+    def get_ticket_id(str)
+      if str && (m = str.match(/.*\[#{Regexp.escape(ticket_prefix)}\#(\d+)\].*/)) && m[1]
+        m[1].to_i
+      else
+        nil
+      end
+    end
+
+    def project
+      @project ||= gitlab.search_projects(@config['project']).find{|p| p.namespace.name == @config['namespace'] && p.name == @config['project'] }
+    rescue Gitlab::Error::NotFound => e
+      nil
+    end
+
+    def issues
+      gitlab.issues(p.id)
+    end
+
+    def team_member_id(email)
+      if user_id = find_user_id(email)
+        user_id if project_member?(project, user_id)
+      else
+        nil
+      end
+    rescue Gitlab::Error::NotFound => e
+      nil
+    end
+
+    def project_member?(project, user_id)
+      begin
+        gitlab.team_member(project.id, user_id)
+      rescue Gitlab::Error::NotFound
+        gitlab.group_member(project.namespace.id, user_id)
+      end
+    rescue Gitlab::Error::NotFound => e
+      nil
+    end
+
+    def find_user_id(username_or_email)
+      users = gitlab.user_search(username_or_email)
+      # did we look for an email => exact match
+      # or by username?
+      user = if username_or_email =~ /@/
+        users.first
+      else
+        users.find do |u|
+          u.name == username_or_email || u.username == username_or_email
+        end
+      end
+      user.nil? ? nil : user.id
+    rescue Gitlab::Error::NotFound => e
+      nil
+    end
+
+    def ignore_mail?(mail)
+      return true if mail.respond_to?(:[]) && mail['X-Spam-Flag'].to_s.downcase == 'yes'
+      return true if sender_filters.any?{|s| mail.from.first =~ /#{s}/ }
+      return true if subject_filters.any?{|s| mail.subject =~ /#{s}/ }
+      false
+    end
+
+    def ticket_prefix
+      @ticket_prefix ||= (@config['ticket_prefix'] || "gl/#{@config['project']}")
+    end
+
+    def clean_subject(subj)
+      subj.nil? ? nil : subj.gsub(/^(Re|Fw):\s*/,'').gsub(/\[[<>]?#{list_subj}\]\s*/,'').gsub(/\s*\[#{Regexp.escape(ticket_prefix)}#\d+\](\s*$)?/,'').gsub(/\s*\[ok\](\s*$)?/,'').strip
+    end
+
+    def list_subj
+      @list_subj ||= (@config['list_subj'] || @name.split('@',2).first)
+    end
+
+    def update_subject(subject,ticket_id)
+      [subject.nil? ? nil : subject.gsub(/(\s)?\[#{Regexp.escape(ticket_prefix)}#\d+\]/,''), "[#{ticket_prefix}##{ticket_id}]"].compact.join(' ')
+    end
+  end
+end

data/lib/schleuder-gitlab-ticketing/version.rb

@@ -0,0 +1,3 @@
+module SchleuderGitlabTicketing
+  VERSION = '1.0.0'
+end

data/lib/schleuder/filters/post_decryption/99_gitlab_ticketing.rb

@@ -0,0 +1,25 @@
+module Schleuder
+  module Filters
+    def self.gitlab_ticketing(list, mail)
+      @gt_config ||= begin
+        require 'schleuder-gitlab-ticketing'
+        SchleuderGitlabTicketing::Config.new
+      end
+
+      res = @gt_config.process_list(list.email, mail)
+      if res.nil?
+        list.logger.debug('No gitlab ticketing enabled for list')
+      elsif res == 'config-error'
+        list.logger.error('gitlab ticketing not correctly configured')
+      elsif res == false
+        list.logger.debug('Email skipped by list configuration')
+      end
+      nil
+    # make sure we catch any error with that filter
+    # so we don't affect list processing
+    rescue Exception => e
+      list.logger.notify_admin "Unable to process the following mail with gitlab-ticketing:\n\n#{e}\n\n#{e.backtrace.join("\n")}", mail.original_message, 'Error while processing mail with gitlab-ticketing'
+      nil
+    end
+  end
+end

metadata

@@ -0,0 +1,104 @@
+--- !ruby/object:Gem::Specification
+name: schleuder-gitlab-ticketing
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- schleuder dev team
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2019-02-21 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: gitlab
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.8'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.8'
+- !ruby/object:Gem::Dependency
+  name: schleuder
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.3'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.5'
+description: Schleuder Gitlab Ticketing combines a schleuder list with the issue tracker
+  of a gitlab project and operates as a state tracker of threads on the list. This
+  allows one to keep an overview on the state of various requests on a help desk powered
+  by schleuder
+email: team@schleuder.org
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/schleuder-gitlab-ticketing.rb
+- lib/schleuder-gitlab-ticketing/config.rb
+- lib/schleuder-gitlab-ticketing/gitlab_config.rb
+- lib/schleuder-gitlab-ticketing/list.rb
+- lib/schleuder-gitlab-ticketing/version.rb
+- lib/schleuder/filters/post_decryption/99_gitlab_ticketing.rb
+homepage: https://schleuder.org/
+licenses:
+- GPL-3.0
+metadata:
+  homepage_uri: https://schleuder.org/
+  documentation_uri: https://schleuder.org/docs/
+  changelog_uri: https://0xacab.org/schleuder/schleuder-gitlab-ticketing/blob/master/CHANGELOG.md
+  source_code_uri: https://0xacab.org/schleuder/schleuder-gitlab-ticketing/
+  bug_tracker_uri: https://0xacab.org/schleuder/schleuder-gitlab-ticketing/issues
+  mailing_list_uri: https://lists.nadir.org/mailman/listinfo/schleuder-announce/
+post_install_message: "\n\n    To activate the filter plugin you will need to link\n
+  \   lib/schleuder/filters/post_decryption/99_gitlab_ticketing.rb\n    to /usr/local/lib/schleuder/filters/post_decryption/\n\n
+  \ "
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.1.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: "[none]"
+rubygems_version: 2.7.8
+signing_key: 
+specification_version: 4
+summary: Schleuder Gitlab Ticketing is a filter plugin for schleuder to hook a list
+  into a gitlab issue tracker
+test_files: []