flay 1.1.0 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
Binary file
@@ -1,13 +1,26 @@
1
+ === 1.2.0 / 2009-03-09
2
+
3
+ * 2 major enhancements:
4
+
5
+ * Added flay_task.rb
6
+ * Added plugin system (any flay_(c,java,js,etc).rb files).
7
+
8
+ * 4 minor enhancements:
9
+
10
+ * Added expand_dirs_to_files and made dirs valid arguments.
11
+ * Added flay_erb.rb plugin.
12
+ * Added optparse option processing.
13
+ * Refactored to make using w/in rake and other CI systems clean and easy.
14
+
1
15
  === 1.1.0 / 2009-01-20
2
16
 
3
- * 8 minor enhancement:
17
+ * 7 minor enhancement:
4
18
 
5
19
  * Added -v verbose mode to print out N-way diff of the detected code.
6
20
  * Added identical node scoring and reporting.
7
- * Added the start of copy/paste+edit detection, not even close yet
21
+ * Added the start of copy/paste+edit detection, not even close yet.
8
22
  * Added more tests.
9
23
  * Added rcov tasks
10
- * Added the start of copy/paste+edit detection
11
24
  * Clarified output a bit
12
25
  * Refactored process_sexps to make doing other languages/systems easier.
13
26
 
@@ -16,4 +29,3 @@
16
29
  * 1 major enhancement
17
30
 
18
31
  * Birthday!
19
-
@@ -4,4 +4,6 @@ README.txt
4
4
  Rakefile
5
5
  bin/flay
6
6
  lib/flay.rb
7
- test/test_flay.rb
7
+ lib/flay_erb.rb
8
+ lib/flay_task.rb
9
+ test/test_flay.rb
data/README.txt CHANGED
@@ -5,23 +5,26 @@
5
5
 
6
6
  == DESCRIPTION:
7
7
 
8
- Flay analyzes ruby code for structural similarities. Differences in
9
- literal values, variable, class, method names, whitespace, programming
10
- style, braces vs do/end, etc are all ignored. Making this totally rad.
8
+ Flay analyzes code for structural similarities. Differences in literal
9
+ values, variable, class, method names, whitespace, programming style,
10
+ braces vs do/end, etc are all ignored. Making this totally rad.
11
11
 
12
12
  == FEATURES/PROBLEMS:
13
13
 
14
+ * Plugin system allows other languages to be flayed.
15
+ * Ships with .rb and .erb. javascript and others will be available separately.
16
+ * Includes FlayTask for Rakefiles.
14
17
  * Differences in literal values, variable, class, and method names are ignored.
15
18
  * Differences in whitespace, programming style, braces vs do/end, etc are ignored.
16
19
  * Works across files.
17
20
  * Reports differences at any level of code.
21
+ * Totally rad.
18
22
  * Adds a score multiplier to identical nodes.
19
23
  * Run verbose to see an N-way diff of the code.
20
24
 
21
25
  == TODO:
22
26
 
23
27
  * Editor integration (emacs, textmate, other contributions welcome).
24
- * UI improvement suggestions welcome. :)
25
28
  * Score sequence fragments (a;b;c;d;e) vs (b;c;d) etc.
26
29
 
27
30
  == SYNOPSIS:
@@ -77,7 +80,7 @@ style, braces vs do/end, etc are all ignored. Making this totally rad.
77
80
 
78
81
  (The MIT License)
79
82
 
80
- Copyright (c) 2008 Ryan Davis, Seattle.rb
83
+ Copyright (c) 2008-2009 Ryan Davis, Seattle.rb
81
84
 
82
85
  Permission is hereby granted, free of charge, to any person obtaining
83
86
  a copy of this software and associated documentation files (the
data/Rakefile CHANGED
@@ -12,6 +12,8 @@ Hoe.new('flay', Flay::VERSION) do |flay|
12
12
  flay.rubyforge_name = 'seattlerb'
13
13
  flay.developer('Ryan Davis', 'ryand-ruby@zenspider.com')
14
14
 
15
+ flay.flay_threshold = 250
16
+
15
17
  flay.extra_deps << ['sexp_processor', '>= 3.0.0']
16
18
  flay.extra_deps << ['ruby_parser', '>= 1.1.0']
17
19
  end
data/bin/flay CHANGED
@@ -1,7 +1,11 @@
1
- #!/usr/bin/ruby -s
1
+ #!/usr/bin/ruby
2
2
 
3
3
  require 'flay'
4
4
 
5
- flay = Flay.new($m.to_i)
6
- flay.process(*ARGV)
5
+ flay = Flay.new Flay.parse_options
6
+
7
+ ARGV << '.' if ARGV.empty?
8
+ files = Flay.expand_dirs_to_files(*ARGV)
9
+
10
+ flay.process(*files)
7
11
  flay.report
@@ -1,44 +1,144 @@
1
1
  #!/usr/bin/env ruby -w
2
2
 
3
3
  $: << "../../ruby_parser/dev/lib"
4
+ $: << "../../ruby2ruby/dev/lib"
4
5
 
6
+ require 'optparse'
5
7
  require 'rubygems'
6
8
  require 'sexp_processor'
7
9
  require 'ruby_parser'
8
- require 'pp' # TODO: remove
9
10
 
10
- $m ||= 16
11
- $v ||= false
12
- $f ||= false
13
-
14
- if $v then
15
- $: << "../../ruby2ruby/dev/lib"
16
- require 'ruby2ruby'
17
- require 'tempfile'
18
- end
11
+ abort "update rubygems to >= 1.3.1" unless Gem.respond_to? :find_files
19
12
 
20
13
  class Flay
21
- VERSION = '1.1.0'
14
+ VERSION = '1.2.0'
15
+
16
+ def self.default_options
17
+ {
18
+ :fuzzy => false,
19
+ :verbose => false,
20
+ :mass => 16,
21
+ }
22
+ end
23
+
24
+ def self.parse_options
25
+ options = self.default_options
26
+
27
+ OptionParser.new do |opts|
28
+ opts.banner = 'flay [options] files_or_dirs'
29
+ opts.version = Flay::VERSION
30
+
31
+ opts.separator ""
32
+ opts.separator "Specific options:"
33
+ opts.separator ""
34
+
35
+ opts.on('-h', '--help', 'Display this help.') do
36
+ puts opts
37
+ exit
38
+ end
39
+
40
+ opts.on('-f', '--fuzzy', "Attempt to do fuzzy similarities. (SLOW)") do
41
+ options[:fuzzy] = true
42
+ end
43
+
44
+ opts.on('-m', '--mass MASS', Integer, "Sets mass threshold") do |m|
45
+ options[:mass] = m.to_i
46
+ end
22
47
 
23
- attr_accessor :mass_threshold
24
- attr_reader :hashes
48
+ opts.on('-v', '--verbose', "Verbose. Display N-Way diff for ruby.") do
49
+ options[:verbose] = true
50
+ end
51
+
52
+ extensions = ['rb'] + Flay.load_plugins
53
+
54
+ opts.separator ""
55
+ opts.separator "Known extensions: #{extensions.join(', ')}"
56
+ end.parse!
57
+
58
+ options
59
+ end
60
+
61
+ def self.expand_dirs_to_files *dirs
62
+ extensions = ['rb'] + Flay.load_plugins
63
+
64
+ dirs.flatten.map { |p|
65
+ if File.directory? p then
66
+ Dir[File.join(p, '**', "*.{#{extensions.join(',')}}")]
67
+ else
68
+ p
69
+ end
70
+ }.flatten
71
+ end
72
+
73
+ def self.load_plugins
74
+ plugins = Gem.find_files("flay_*.rb").reject { |path| path =~ /flay_task/ }
75
+
76
+ plugins.each do |plugin|
77
+ begin
78
+ load plugin
79
+ rescue LoadError => e
80
+ warn "error loading #{plugin.inspect}: #{e.message}. skipping..."
81
+ end
82
+ end
25
83
 
26
- def initialize(mass = 16)
84
+ plugins.map { |f| File.basename(f, '.rb').sub(/^flay_/, '') }
85
+ end
86
+
87
+ attr_accessor :mass_threshold, :total, :identical, :masses
88
+ attr_reader :hashes, :option
89
+
90
+ def initialize option = nil
91
+ @option = option || Flay.default_options
27
92
  @hashes = Hash.new { |h,k| h[k] = [] }
28
- @mass_threshold = mass
93
+
94
+ self.identical = {}
95
+ self.masses = {}
96
+ self.total = 0
97
+ self.mass_threshold = @option[:mass]
98
+
99
+ require 'ruby2ruby' if @option[:verbose]
29
100
  end
30
101
 
31
102
  def process(*files)
32
103
  files.each do |file|
33
104
  warn "Processing #{file}"
34
105
 
35
- pt = RubyParser.new.process(File.read(file), file)
36
- next unless pt # empty files... hahaha, suck.
106
+ ext = File.extname(file).sub(/^\./, '')
107
+ ext = "rb" if ext.nil? || ext.empty?
108
+ msg = "process_#{ext}"
109
+
110
+ unless respond_to? msg then
111
+ warn " Unknown file type: #{ext}, defaulting to ruby"
112
+ msg = "process_rb"
113
+ end
114
+
115
+ sexp = begin
116
+ send msg, file
117
+ rescue => e
118
+ warn " #{e.message.strip}"
119
+ warn " skipping #{file}"
120
+ nil
121
+ end
122
+
123
+ next unless sexp
37
124
 
38
- process_sexp pt
125
+ process_sexp sexp
39
126
  end
40
127
 
41
- process_fuzzy_similarities if $f
128
+ process_fuzzy_similarities if option[:fuzzy]
129
+
130
+ self.prune
131
+
132
+ self.hashes.each do |hash,nodes|
133
+ identical[hash] = nodes[1..-1].all? { |n| n == nodes.first }
134
+ masses[hash] = nodes.first.mass * nodes.size
135
+ masses[hash] *= (nodes.size) if identical[hash]
136
+ self.total += masses[hash]
137
+ end
138
+ end
139
+
140
+ def process_rb file
141
+ RubyParser.new.process(File.read(file), file)
42
142
  end
43
143
 
44
144
  def process_sexp pt
@@ -136,19 +236,11 @@ class Flay
136
236
  end
137
237
 
138
238
  def report prune = nil
139
- self.prune
140
-
141
- identical = {}
142
- masses = {}
143
-
144
- self.hashes.each do |hash,nodes|
145
- identical[hash] = nodes[1..-1].all? { |n| n == nodes.first }
146
- masses[hash] = nodes.first.mass * nodes.size
147
- masses[hash] *= (nodes.size) if identical[hash]
148
- end
239
+ puts "Total score (lower is better) = #{self.total}"
240
+ puts
149
241
 
150
242
  count = 0
151
- masses.sort_by { |h,m| [-m, hashes[h].first.file] }.each do |hash,mass|
243
+ masses.sort_by { |h,m| [-m, hashes[h].first.file] }.each do |hash, mass|
152
244
  nodes = hashes[hash]
153
245
  next unless nodes.first.first == prune if prune
154
246
  puts
@@ -167,7 +259,7 @@ class Flay
167
259
  [count, match, node.first, bonus, mass]
168
260
 
169
261
  nodes.each_with_index do |node, i|
170
- if $v then
262
+ if option[:verbose] then
171
263
  c = (?A + i).chr
172
264
  puts " #{c}: #{node.file}:#{node.line}"
173
265
  else
@@ -175,7 +267,7 @@ class Flay
175
267
  end
176
268
  end
177
269
 
178
- if $v then
270
+ if option[:verbose] then
179
271
  puts
180
272
  r2r = Ruby2Ruby.new
181
273
  puts n_way_diff(*nodes.map { |s| r2r.process(s.deep_clone) })
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/ruby
2
+
3
+ require 'rubygems'
4
+ require 'flay'
5
+ require 'erb'
6
+
7
+ class Flay
8
+ def process_erb file
9
+ erb = File.read file
10
+
11
+ src = ERB.new(erb).src
12
+ RubyParser.new.process(src, file)
13
+ end
14
+ end
@@ -0,0 +1,31 @@
1
+ class FlayTask < Rake::TaskLib
2
+ attr_accessor :name
3
+ attr_accessor :dirs
4
+ attr_accessor :threshold
5
+ attr_accessor :verbose
6
+
7
+ def initialize name = :flay, threshold = 200, dirs = nil
8
+ @name = name
9
+ @dirs = dirs || %w(app lib test spec)
10
+ @threshold = threshold
11
+ @verbose = Rake.application.options.trace
12
+
13
+ yield self if block_given?
14
+
15
+ @dirs.reject! { |f| ! File.directory? f }
16
+
17
+ define
18
+ end
19
+
20
+ def define
21
+ desc "Analyze for code duplication in: #{dirs.join(', ')}"
22
+ task name do
23
+ flay = Flay.new
24
+ flay.process(*Flay.expand_dirs_to_files(dirs))
25
+ flay.report if verbose
26
+
27
+ raise "Flay total too high! #{flay.total} > #{threshold}" if flay.total > threshold
28
+ end
29
+ self
30
+ end
31
+ end
@@ -107,7 +107,7 @@ class TestSexp < Test::Unit::TestCase
107
107
  end
108
108
 
109
109
  def test_process_sexp_full
110
- flay = Flay.new(1)
110
+ flay = Flay.new(:mass => 1)
111
111
 
112
112
  s = RubyParser.new.process <<-RUBY
113
113
  def x(n)
@@ -137,14 +137,14 @@ class TestSexp < Test::Unit::TestCase
137
137
  end
138
138
 
139
139
  def test_process_sexp_no_structure
140
- flay = Flay.new(1)
140
+ flay = Flay.new(:mass => 1)
141
141
  flay.process_sexp s(:lit, 1)
142
142
 
143
143
  assert flay.hashes.empty?
144
144
  end
145
145
 
146
146
  def test_process_fuzzy_similarities
147
- flay = Flay.new 7
147
+ flay = Flay.new :mass => 7
148
148
 
149
149
  s1 = RubyParser.new.process("def w(n); a; b; c; d; e; end")
150
150
  s2 = RubyParser.new.process("def x(n); a; c; e; end")
@@ -161,7 +161,7 @@ class TestSexp < Test::Unit::TestCase
161
161
  end
162
162
 
163
163
  def test_process_fuzzy_similarities_2
164
- flay = Flay.new 7
164
+ flay = Flay.new :mass => 7
165
165
 
166
166
  s1 = RubyParser.new.process("def w(n); a; b; c; d; e; end")
167
167
  s2 = RubyParser.new.process("def x(n); a; c; e; end")
@@ -181,7 +181,7 @@ class TestSexp < Test::Unit::TestCase
181
181
  end
182
182
 
183
183
  def test_process_fuzzy_similarities_3
184
- flay = Flay.new 7
184
+ flay = Flay.new :mass => 7
185
185
 
186
186
  s1 = RubyParser.new.process("def w (n); a; b; c; d; e; end")
187
187
  s2 = RubyParser.new.process("def x (n); a; c; e; end")
metadata CHANGED
@@ -1,15 +1,36 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: flay
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ryan Davis
8
8
  autorequire:
9
9
  bindir: bin
10
- cert_chain: []
10
+ cert_chain:
11
+ - |
12
+ -----BEGIN CERTIFICATE-----
13
+ MIIDPjCCAiagAwIBAgIBADANBgkqhkiG9w0BAQUFADBFMRMwEQYDVQQDDApyeWFu
14
+ ZC1ydWJ5MRkwFwYKCZImiZPyLGQBGRYJemVuc3BpZGVyMRMwEQYKCZImiZPyLGQB
15
+ GRYDY29tMB4XDTA5MDMwNjE4NTMxNVoXDTEwMDMwNjE4NTMxNVowRTETMBEGA1UE
16
+ AwwKcnlhbmQtcnVieTEZMBcGCgmSJomT8ixkARkWCXplbnNwaWRlcjETMBEGCgmS
17
+ JomT8ixkARkWA2NvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALda
18
+ b9DCgK+627gPJkB6XfjZ1itoOQvpqH1EXScSaba9/S2VF22VYQbXU1xQXL/WzCkx
19
+ taCPaLmfYIaFcHHCSY4hYDJijRQkLxPeB3xbOfzfLoBDbjvx5JxgJxUjmGa7xhcT
20
+ oOvjtt5P8+GSK9zLzxQP0gVLS/D0FmoE44XuDr3iQkVS2ujU5zZL84mMNqNB1znh
21
+ GiadM9GHRaDiaxuX0cIUBj19T01mVE2iymf9I6bEsiayK/n6QujtyCbTWsAS9Rqt
22
+ qhtV7HJxNKuPj/JFH0D2cswvzznE/a5FOYO68g+YCuFi5L8wZuuM8zzdwjrWHqSV
23
+ gBEfoTEGr7Zii72cx+sCAwEAAaM5MDcwCQYDVR0TBAIwADALBgNVHQ8EBAMCBLAw
24
+ HQYDVR0OBBYEFEfFe9md/r/tj/Wmwpy+MI8d9k/hMA0GCSqGSIb3DQEBBQUAA4IB
25
+ AQAY59gYvDxqSqgC92nAP9P8dnGgfZgLxP237xS6XxFGJSghdz/nI6pusfCWKM8m
26
+ vzjjH2wUMSSf3tNudQ3rCGLf2epkcU13/rguI88wO6MrE0wi4ZqLQX+eZQFskJb/
27
+ w6x9W1ur8eR01s397LSMexySDBrJOh34cm2AlfKr/jokKCTwcM0OvVZnAutaovC0
28
+ l1SVZ0ecg88bsWHA0Yhh7NFxK1utWoIhtB6AFC/+trM0FQEB/jZkIS8SaNzn96Rl
29
+ n0sZEf77FLf5peR8TP/PtmIg7Cyqz23sLM4mCOoTGIy5OcZ8TdyiyINUHtb5ej/T
30
+ FBHgymkyj/AOSqKRIpXPhjC6
31
+ -----END CERTIFICATE-----
11
32
 
12
- date: 2009-01-20 00:00:00 -08:00
33
+ date: 2009-03-09 00:00:00 -07:00
13
34
  default_executable:
14
35
  dependencies:
15
36
  - !ruby/object:Gem::Dependency
@@ -40,9 +61,9 @@ dependencies:
40
61
  requirements:
41
62
  - - ">="
42
63
  - !ruby/object:Gem::Version
43
- version: 1.8.2
64
+ version: 1.10.0
44
65
  version:
45
- description: Flay analyzes ruby code for structural similarities. Differences in literal values, variable, class, method names, whitespace, programming style, braces vs do/end, etc are all ignored. Making this totally rad.
66
+ description: Flay analyzes code for structural similarities. Differences in literal values, variable, class, method names, whitespace, programming style, braces vs do/end, etc are all ignored. Making this totally rad.
46
67
  email:
47
68
  - ryand-ruby@zenspider.com
48
69
  executables:
@@ -60,6 +81,8 @@ files:
60
81
  - Rakefile
61
82
  - bin/flay
62
83
  - lib/flay.rb
84
+ - lib/flay_erb.rb
85
+ - lib/flay_task.rb
63
86
  - test/test_flay.rb
64
87
  has_rdoc: true
65
88
  homepage: http://ruby.sadi.st/
@@ -87,6 +110,6 @@ rubyforge_project: seattlerb
87
110
  rubygems_version: 1.3.1
88
111
  signing_key:
89
112
  specification_version: 2
90
- summary: Flay analyzes ruby code for structural similarities
113
+ summary: Flay analyzes code for structural similarities
91
114
  test_files:
92
115
  - test/test_flay.rb
@@ -0,0 +1 @@
1
+ �B�쑊 �u�4`. �,�*s�O