git_sme 0.1.0

data/README.md

@@ -0,0 +1,78 @@
+# Git SME
+
+Git SME allows you to analyze your git repository and identify subject matter experts for any file
+or directory that you would like to know more about.  It does this by analyzing all commits made to
+all files in your git repository over time and finding out who made the most changes to each file
+and directory.
+
+Commits are weighted so recent commits are more significant than past commits which should mitigate
+the effect a legacy coder would have on these reports.
+
+## Installation
+
+Install the gem for commandline usage in the appropriate version of ruby:
+
+    $ gem install git_sme
+    
+This will install the git-sme command which should now be available from everywhere assuming your
+PATH is setup appropriately.
+
+## Usage
+
+Basic usage of git-sme is as follows:
+
+    git-sme </path/to/repository> [flags]
+
+This will throw an error if the path is not a git repository.  As a rule, you don't have to point to
+the `.git` folder in your checked out code because `git-sme` will know to look for that folder as a
+child of the folder you **do** provide it.
+
+`git-sme` will output a list of paths (files and directories) and the list of users who it thinks
+are the subject matter experts on each of those paths.  Users are listed in decreasing order of
+expertise:
+
+    $ bundle exec git-sme ~/rails/dinghy --file ~/rails/dinghy/cli/dinghy/preferences.rb ~/rails/dinghy/cli/dinghy/machine.rb
+    Repository: /Users/sjaveed/rails/dinghy
+    Analyzed: 317 (0/s) 100.00% Time: 00:00:00 |==============================================================================|
+    
+    /: brianp, ryan, brian, adrian.falleiro, sally, dev, markse, fgrehm, kallin.nagelberg, matt
+    /cli: brianp, ryan, markse, sally, adrian.falleiro, fgrehm, matt, kallin.nagelberg, brian, dev
+    /cli/dinghy: brianp, markse, ryan, sally, fgrehm, matt, adrian.falleiro, brian, aisipos, paul.moelders
+    /cli/dinghy/machine.rb: brianp, markse, sally, brian, fgrehm, ryan, robertc
+    /cli/dinghy/preferences.rb: brianp
+    /cli/dinghy/machine: brianp, ryan, fgrehm
+    /dinghy: brianp
+
+Based on analysis of a checked out copy of dinghy, I can see that, for the files I'm interested in,
+brianp would be a subject matter expert but I'll probably find some useful information from ryan and
+markse as well since it looks like ryan has touched enough files in the `/cli/dinghy/machine`
+directory
+
+### Flags
+
+Flag | Description
+-----|------------
+`--branch <branch>` | The branch you want to analyze on the given repository.  Defaults to 'master'.
+`--user <username1 [username2 ...]>` | An optional list of users to whom you'd like to restrict the analysis.  This allows you to see e.g. who might know more about a file given their history of working with it over time.
+`--file </path/to/file [/path/to/other/file ...]` | An optional list of files/directories for which you'd like analysis.  Defaults to /.  The analysis will also include all directories between a subdirectory and the root of the repository.  All file paths are relative to the repository root.
+`--cache` | This is a default specification which caches all commits that the tool loads for a git repository.  This allows you to e.g. `git pull` on a large repository and only incur the cost of loading the additional commits from the repository while previously seen commits are loaded a lot quicker from a cache.
+`--no-cache` | Specify this if you do *not* want caching.  You'll probably never need to use this.
+`--results <count>` | The number of subject matter experts you'd like to see for each path.  Defaults to 10.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/sjaveed/git_sme. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [GPLv3 License](https://www.gnu.org/licenses/gpl-3.0.en.html).
+
+## Code of Conduct
+
+Everyone interacting in the GitSme project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/sjaveed/git_sme/blob/master/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "git_sme"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/exe/git-sme

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+
+require 'git_sme/cli'
+
+# Prepend analyze so we don't have to keep typing that on the commandline
+GitSme::CLI.start(ARGV.dup.unshift('analyze'))

data/git_sme.gemspec

@@ -0,0 +1,31 @@
+# coding: utf-8
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "git_sme/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "git_sme"
+  spec.version       = GitSme::VERSION
+  spec.authors       = ["Shahbaz Javeed"]
+  spec.email         = ["sjaveed@gmail.com"]
+
+  spec.summary       = %q{Identify subject matter experts by analyzing your git repository}
+  spec.description   = %q{Analyze your git repository and determine subject matter experts by identifying everyone who has touched a file with preference given to recent touches}
+  spec.homepage      = "https://github.com/sjaveed/git_sme"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.15"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.2"
+
+  spec.add_dependency 'ruby-progressbar'
+  spec.add_dependency 'rugged'
+  spec.add_dependency 'thor'
+end

data/lib/git_sme.rb

@@ -0,0 +1,9 @@
+require 'git_sme/version'
+require 'git_sme/preferences'
+require 'git_sme/commit_loader'
+require 'git_sme/commit_analyzer'
+require 'git_sme/analysis_presenter'
+
+module GitSme
+  # Your code goes here...
+end

data/lib/git_sme/analysis_presenter.rb

@@ -0,0 +1,71 @@
+module GitSme
+  class AnalysisPresenter
+    attr_reader :valid, :error_message
+
+    alias_method :valid?, :valid
+
+    def initialize(commit_analyzer, users = [], files = [])
+      @commit_analyzer = commit_analyzer
+      @users = users
+      @files = files
+      @files = ['/'] unless @users.any? || @files.any?
+
+      @valid = @commit_analyzer.valid?
+      @error_message = @commit_analyzer.error_message
+    end
+
+    def get_relevant_analyses(results_to_show = 10)
+      @commit_analyzer.analyze unless @commit_analyzer.analyzed?
+
+      users_to_match = @users.any? ? get_matching_keys(@commit_analyzer.analysis[:by_user].keys, @users) : []
+      files_to_match = @files.any? ? get_matching_keys(@commit_analyzer.analysis[:by_file].keys, @files) : []
+      presentable_data = []
+
+      if users_to_match.any? && files_to_match.any?
+        users_to_match.each do |user|
+          user_data = @commit_analyzer.analysis[:by_user][user].select { |k, v| files_to_match.include?(k) }
+          presentable_data << presentable_file_or_user({ user => user_data }, user, results_to_show: results_to_show)
+        end
+
+        puts
+
+        files_to_match.each do |file|
+          user_data = @commit_analyzer.analysis[:by_file][file].select { |k, v| users_to_match.include?(k) }
+          presentable_data << presentable_file_or_user({ file => user_data }, file)
+        end
+      elsif users_to_match.any?
+        get_matching_keys(@commit_analyzer.analysis[:by_user].keys, users_to_match).each do |user|
+          presentable_data << presentable_file_or_user(@commit_analyzer.analysis[:by_user], user)
+        end
+      elsif files_to_match.any?
+        get_matching_keys(@commit_analyzer.analysis[:by_file].keys, files_to_match).each do |path|
+          presentable_data << presentable_file_or_user(@commit_analyzer.analysis[:by_file], path)
+        end
+      end
+
+      presentable_data.compact
+    end
+
+    private
+
+    def presentable_file_or_user(data, key, results_to_show: 10)
+      stats = data[key]
+      info_to_show = sort_keys_by_value(stats).first(results_to_show)
+      return if info_to_show.empty?
+
+      {
+        key => info_to_show
+      }
+    end
+
+    def sort_keys_by_value(data)
+      data.keys.sort_by { |k| data[k] }.reverse
+    end
+
+    def get_matching_keys(all_keys, keys_to_match)
+      all_keys.select do |key|
+        keys_to_match.map { |matcher| matcher.match?(key) }.any? { |val| val }
+      end
+    end
+  end
+end

data/lib/git_sme/cache.rb

@@ -0,0 +1,51 @@
+require 'fileutils'
+
+require_relative 'preferences'
+
+module GitSme
+  class Cache
+    def initialize(name, enabled: true, directory: 'cache', file_prefix: '', file_suffix: '')
+      raise "Invalid cache name: [#{name}]" if name.nil? || name =~ /^\s+$/
+
+      @name = name.gsub(/[^a-zA-Z-]/, '').strip
+      @enabled = enabled
+      @cache_directory = File.join(PREFERENCES_HOME, directory)
+      @file_prefix = file_prefix
+      @file_suffix = file_suffix
+
+      FileUtils.mkdir_p(@cache_directory) unless File.exist?(@cache_directory)
+    end
+
+    def load
+      return [] unless @enabled && File.exist?(cache_filename)
+
+      YAML.load(File.read(cache_filename))
+    end
+
+    def save(data)
+      return unless @enabled
+
+      File.open(cache_filename, 'w') { |f| f.write(YAML.dump(data)) }
+    end
+
+    private
+
+    def prefix
+      return '' if @file_prefix =~ /^\s*$/
+
+      "#{@file_prefix}-"
+    end
+
+    def suffix
+      return '' if @file_suffix =~ /^\s*$/
+
+      "-#{@file_suffix}"
+    end
+
+    def cache_filename
+      filename = @name
+
+      File.join(@cache_directory, "#{prefix}#{filename}#{suffix}.yml")
+    end
+  end
+end

data/lib/git_sme/cli.rb

@@ -0,0 +1,60 @@
+require 'thor'
+require 'ruby-progressbar'
+require 'git_sme'
+
+module GitSme
+  class CLI < Thor
+    desc 'analyze <repository> [--branch <branch>] [--user <username>] [--file </path/to/file>] [--cache | --no-cache] [--results <count>]',
+      'Analyze the repository and determine the subject matter experts for the given files, limiting them to the users provided if needed'
+
+    method_option :branch, type: :string, default: 'master'
+    method_option :user, type: :array, default: []
+    method_option :file, type: :array, default: ['/']
+    method_option :cache, type: :boolean, default: true
+    method_option :results, type: :numeric, default: 10
+
+    def analyze(repository)
+      loader = GitSme::CommitLoader.new(repository, branch: options[:branch], enable_cache: options[:cache])
+      unless loader.valid?
+        puts "Error: #{loader.error_message}"
+        return
+      end
+
+      puts "Repository: #{loader.repo.path.gsub('/.git/', '')}"
+
+      loader_progress = ProgressBar.create(starting_at: 0, format: 'Loaded: %c (%R/s) %P%% %f |%B|')
+      loader.load do |new_commit_count, processed_commit_count, all_commit_count|
+        loader_progress.total = all_commit_count
+        loader_progress.increment
+      end
+
+      analyzer = GitSme::CommitAnalyzer.new(loader, enable_cache: false)
+      unless analyzer.valid?
+        puts "Error: #{analyzer.error_message}"
+        return
+      end
+
+      analyzer_progress = ProgressBar.create(starting_at: 0, total: loader.commits.size, format: 'Analyzed: %c (%R/s) %P%% %f |%B|')
+      analyzer.analyze do |commit_count, total_commits|
+        analyzer_progress.increment
+      end
+
+      presenter = AnalysisPresenter.new(analyzer, options[:user], options[:file])
+      analyses = presenter.get_relevant_analyses(options[:results].to_i)
+
+      puts
+
+      if !analyses.empty?
+        analyses.each do |result|
+          result.each do |path, users|
+            puts "#{path}: #{users.join(', ')}"
+          end
+        end
+      else
+        puts 'No data found!'
+      end
+    end
+
+    default_task :analyze
+  end
+end

data/lib/git_sme/commit_analyzer.rb

@@ -0,0 +1,125 @@
+module GitSme
+  class CommitAnalyzer
+    attr_reader :valid, :error_message, :analysis, :analyzed
+
+    alias_method :valid?, :valid
+    alias_method :analyzed?, :analyzed
+
+    def initialize(commit_loader, enable_cache: true)
+      @enable_cache = true
+      @commit_loader = commit_loader
+      @analyzed = false
+      @valid = @commit_loader.valid?
+      @error_message = @commit_loader.error_message
+
+      @analysis = {}
+      @cache = GitSme::Cache.new(@commit_loader.repo.path.gsub('/.git/', ''),
+        enabled: @enable_cache, file_suffix: "#{@commit_loader.branch}-analysis"
+      )
+    end
+
+    def analyze(force: false)
+      return unless valid?
+      return if analyzed? && !force
+
+      @commit_loader.load
+      @analysis = @cache.load
+      new_analysis = []
+
+      if !@commit_loader.new_commits? || !@analysis.any?
+        if block_given?
+          @analysis = analyze_new_commits(@commit_loader.commits) do |commit_count, total_commits|
+            yield(commit_count, total_commits)
+          end
+        else
+          @analysis = analyze_new_commits(@commit_loader.commits)
+        end
+      elsif @commit_loader.new_commits?
+        new_analysis = if block_given?
+          analyze_new_commits(@commit_loader.new_commits) do |commit_count, total_commits|
+            yield(commit_count, total_commits)
+          end
+        else
+          analyze_new_commits(@commit_loader.new_commits)
+        end
+
+        summed_merge(@analysis[:by_user], new_analysis[:by_user])
+        summed_merge(@analysis[:by_file], new_analysis[:by_file])
+      end
+
+      @cache.save(@analysis)
+      @analyzed = true
+    end
+
+    private
+
+    def analyze_new_commits(commits_to_process)
+      user_stats = {}
+      file_stats = {}
+      now = Time.now.to_i
+      commit_count = commits_to_process.size
+
+      commits_to_process.each_with_index do |commit, current_commit_idx|
+        author = commit[:author]
+        time_delta = now - commit[:timestamp]
+
+        commit[:file_changes].each do |filename, change_details|
+          all_affected_paths(filename).each do |path|
+            change_value = weighted_value(change_details[:changes], time_delta)
+
+            user_stats[author] = {} unless user_stats.key?(author)
+            user_stats[author][path] = 0 unless user_stats[author].key?(path)
+
+            file_stats[path] = {} unless file_stats.key?(path)
+            file_stats[path][author] = 0 unless file_stats[path].key?(author)
+
+            user_stats[author][path] += change_value
+            file_stats[path][author] += change_value
+          end
+        end
+
+        if block_given?
+          yield(current_commit_idx, commit_count)
+        end
+      end
+
+      {
+        by_user: user_stats,
+        by_file: file_stats
+      }
+    end
+
+    def summed_merge(cached_data, new_data)
+      return if new_data.nil? || cached_data.nil?
+
+      new_data.each do |key, value_hash|
+        if cached_data.key?(key)
+          value_hash.each do |value_key, value|
+            if cached_data[key].key?(value_key)
+              cached_data[key][value_key] += value
+            else
+              cached_data[key][value_key] = value
+            end
+          end
+        else
+          cached_data[key] = value_hash
+        end
+      end
+    end
+
+    def all_affected_paths(filename)
+      ['/'] + filename.split('/').each_with_object([]) do |path_part, path_list|
+        path_list << [path_list[-1], path_part].join('/')
+      end
+    end
+
+    def weighted_value(value, time_delta)
+      # value_attenuator = 1.0
+      # value_attenuation = value_attenuator * time_delta / time_delta
+      value_attenuation = time_delta > 0 ? time_delta ** (-1/3) : 1
+
+      (value * value_attenuation).to_f
+    end
+
+  end
+end