starscope 1.1.2 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2235e7d803ca938cc48e75eee57d14eb6b5b7a9d
-  data.tar.gz: 20c2ff4775299f5b0fa8b96aa38d260a62a48149
+  metadata.gz: 63556d4e16d22bf281211c311fa2ca379b7ec631
+  data.tar.gz: c48c940ea99cd383e0aa6f90a19b0cc2f9df6d96
 SHA512:
-  metadata.gz: 900973601a54f6383c1b58f2302d5c9f6f37e6333540b4ff36167bff8523038d842bf78f6b2337b518208381e2b395634cf93e429750ec879a1b828b4825ef5e
-  data.tar.gz: 6570238f423d05dcf5e17f3b133380f2d7e77f542520d7d65e3572062de9f93c87ac8f97350c8b47903a82e881ba0361c19a4358769d9f87ce78311e70a22647
+  metadata.gz: ad88485b560c0b4f56aa48c76b7b42451d1e91e9f72ff65689207db6ffc1b85cc2c84966399965f4cdf08489d9f5f771bcefda1bc2f61dd83be38f59564f3346
+  data.tar.gz: e03b09376ac960ea2316e5436a9963af358862cebaf51bee7565607bd06b4f60c89df4e5bc5e4802345cd3f7d43f282eac321b853327de8c2ca2959b059dfb8a

data/CHANGELOG.md

@@ -1,7 +1,25 @@
 Changelog
 =========
 
-v1.1.2 (trunk)
+v1.2.0 (trunk)
+--------------------
+
+Improvements:
+ * You can now specify `--export` multiple times with different formats in a
+   single run.
+ * Language extractors are now individually versioned. When an extractor is
+   upgraded, files it owns will be automatically re-parsed.
+ * Deduplicated some metadata, shrinking database size. This comes with a
+   related reduction in read/write time and memory usage.
+
+Bug Fixes:
+ * Proper handling of golang string literal escapes.
+
+Misc:
+ * Lots of internal refactoring and test suite improvements for better
+   maintainability going forward.
+
+v1.1.2 (2014-07-29)
 --------------------
 
 Bug Fixes:
@@ -105,7 +123,7 @@ v0.1.10 (2014-02-24)
 --------------------
 
 Improvements:
- * Import new ruby parser version and make necessary changes so that StarScope
+ * Import new ruby parser version and make necessary changes so that Starscope
    now runs on older Ruby versions (1.9.2 and 1.8.7)
 
 v0.1.9 (2014-02-22)

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    starscope (1.1.2)
+    starscope (1.2.0)
       oj (~> 2.9)
       parser (~> 2.1)
       ruby-progressbar (~> 1.5)
@@ -11,8 +11,11 @@ GEM
   specs:
     ast (2.0.0)
     coderay (1.1.0)
+    metaclass (0.0.4)
     method_source (0.8.2)
     minitest (5.4.0)
+    mocha (1.1.0)
+      metaclass (~> 0.0.1)
     oj (2.9.9)
     parser (2.1.9)
       ast (>= 1.1, < 3.0)
@@ -31,6 +34,7 @@ PLATFORMS
 DEPENDENCIES
   bundler (~> 1.5)
   minitest
+  mocha
   pry
   rake
   starscope!

data/README.md

@@ -1,4 +1,4 @@
-StarScope
+Starscope
 =========
 
 [![Gem Version](https://badge.fury.io/rb/starscope.png)](http://badge.fury.io/rb/starscope)
@@ -8,10 +8,10 @@ Anyone who has done much programming in C (or C++) on a Unix-based OS has come
 across the fantastic [Cscope](http://cscope.sourceforge.net/) tool. Sadly, it
 only works for C (and sort of works for C++).
 
-StarScope is a similar tool for [Ruby](https://www.ruby-lang.org/) and
+Starscope is a similar tool for [Ruby](https://www.ruby-lang.org/) and
 [Golang](http://golang.org/), with a design intended to make it easy to add
 [support for other languages](doc/LANGUAGE_SUPPORT.md) within the same framework
-(thus the name StarScope, i.e. \*scope).
+(thus the name Starscope, i.e. \*scope).
 
 Install it as a gem:
 ```
@@ -29,14 +29,12 @@ Ask it things directly:
 $ starscope -q calls,new # Lists all callers of new
 ```
 
-Export it to various formats for use with your editor:
+Export it to various existing formats for automatic integration with your editor:
 ```
 $ starscope -e ctags
 $ starscope -e cscope
 ```
 
-More flags and options are available, run `starscope --help` for the complete list.
-
 More Documentation
 ------------------
 
@@ -48,5 +46,5 @@ More Documentation
 Other Uses
 ----------
 
-StarScope is a supported backend for
+Starscope is a supported backend for
 [CodeQuery](https://github.com/ruben2020/codequery).

data/Rakefile

@@ -2,7 +2,7 @@ require 'bundler/gem_tasks'
 require 'rake/testtask'
 
 Rake::TestTask.new do |t|
-  t.test_files = FileList['test/unit/test_*.rb', 'test/functional/test_*.rb']
+  t.test_files = FileList['test/**/*_test.rb']
 end
 
 desc 'Run tests'

data/bin/starscope

@@ -8,14 +8,13 @@ require 'readline'
 require 'starscope'
 
 DEFAULT_DB='.starscope.db'
-DEFAULT_CTAGS='tags'
-DEFAULT_CSCOPE='cscope.out'
 
 options = {:read => true,
            :write => true,
            :update => true,
            :output => :normal,
-           :db => DEFAULT_DB
+           :export => [],
+           :db => DEFAULT_DB,
 }
 
 # Options Parsing
@@ -25,7 +24,7 @@ Usage: starscope [options] [PATHS]
 
 The default database is `#{DEFAULT_DB}` if you don't specify one with -f.
 The default behaviour is to read and update the database.
-If no database exists and no PATHS are specified, StarScope builds a new
+If no database exists and no PATHS are specified, Starscope builds a new
 database by recursing in the current directory.
 
 Scoped queries must use `::` as the scope separator, even for languages which
@@ -51,7 +50,7 @@ END
 
   opts.separator "\nDatabase Management"
   opts.on("-e", "--export FORMAT[,PATH]", "Export in FORMAT to PATH, (see EXPORTING)") do |export|
-    options[:export] = export
+    options[:export] << export
   end
   opts.on("-f", "--file FILE", "Use FILE instead of `#{DEFAULT_DB}`") do |path|
     options[:db] = path
@@ -72,7 +71,7 @@ END
 
   opts.separator "\nMisc"
   opts.on("-v", "--version", "Print the version number") do
-    puts StarScope::VERSION
+    puts Starscope::VERSION
     exit
   end
   opts.on("--verbose", "Print extra status messages") do
@@ -93,22 +92,26 @@ END
   opts.separator <<END
 \nEXPORTING
     At the moment two export formats are supported: 'ctags' and 'cscope'. If
-    you don't specify a path, the output is written to the file '#{DEFAULT_CTAGS}' (for
-    ctags) or '#{DEFAULT_CSCOPE}' (for cscope) in the current directory.
+    you don't specify a path, the output is written to the file '#{Starscope::Export::CTAGS_DEFAULT_PATH}' (for
+    ctags) or '#{Starscope::Export::CSCOPE_DEFAULT_PATH}' (for cscope) in the current directory.
 END
 
 end.parse!
 
 def print_summary(db)
-  summary = db.summary
-  puts "No tables" if summary.empty?
-  summary.each do |name, count|
-    printf("%-8s %5d records\n", name, count)
+  tables = db.tables
+  puts "No tables" if tables.empty?
+  tables.sort {|a,b| a.to_s <=> b.to_s}.each do |table|
+    printf("%-8s %5d records\n", table, db.records(table).length)
   end
 end
 
+def format_record(db, rec)
+  "#{rec[:name].join " "} -- #{rec[:file]}:#{rec[:line_no]} (#{db.line_for_record(rec).strip})"
+end
+
 def run_query(db, table, value)
-  if not value
+  if !value
     $stderr.puts "Invalid input - no query found."
     return false
   end
@@ -117,55 +120,60 @@ def run_query(db, table, value)
     puts "No results found."
   else
     results.sort_by {|x| x[:name].join(' ')}.each do |rec|
-      puts StarScope::Record.format(rec)
+      puts format_record(db, rec)
     end
   end
   return true
-rescue StarScope::DB::NoTableError
+rescue Starscope::DB::NoTableError
   $stderr.puts "Table '#{table}' doesn't exist."
   return false
 end
 
+def dump_table(db, table)
+  records = db.records(table).sort {|a,b|
+    a[:name][-1].to_s.downcase <=> b[:name][-1].to_s.downcase
+  }
+
+  puts "== Table: #{table} =="
+  puts "No records" if records.empty?
+
+  records.each do |record|
+    puts format_record(db, record)
+  end
+end
+
 def dump(db, table)
-  if table.nil?
-    db.dump_all
-  elsif table.start_with?('_')
-    db.dump_meta(table[1..-1].to_sym)
+
+  case table
+  when nil
+    db.tables.each {|t| dump_table(db, t)}
+  when "_meta"
+    puts db.metadata
+  when /^_/
+    puts db.metadata(table[1..-1].to_sym)
   else
-    db.dump_table(table.to_sym)
+    dump_table(db, table.to_sym)
   end
+
   return true
-rescue StarScope::DB::NoTableError
+rescue Starscope::DB::NoTableError
   $stderr.puts "Table '#{table}' doesn't exist."
   return false
 end
 
-def export(db, param, output)
+def export(db, param)
   format, path = param.split(',', 2)
-
-  case format
-  when 'ctags'; path ||= DEFAULT_CTAGS
-  when 'cscope'; path ||= DEFAULT_CSCOPE
-  else
-    $stderr.puts "Unrecognized export format \"#{format}\""
-    return
-  end
-
-  print "Exporting to '#{path}' in format '#{format}'... " unless output == :quiet
-  File.open(path, 'w') do |file|
-    case format
-    when 'ctags'; db.export_ctags(file)
-    when 'cscope'; db.export_cscope(file)
-    end
-  end
-  puts "done" unless output == :quiet
+  db.export(format.to_sym, path)
+rescue Starscope::Export::UnknownExportFormatError
+  $stderr.puts "Unrecognized export format \"#{format}\""
 end
 
-db = StarScope::DB.new(options[:output])
+output = Starscope::Output.new(options[:output])
+db = Starscope::DB.new(output)
 
 db_exists = File.exists?(options[:db])
 
-if options[:read] and db_exists
+if options[:read] && db_exists
   new_data = db.load(options[:db])
 else
   # no need to run an update if we didn't read any old data
@@ -177,11 +185,11 @@ if options[:exclude]
   new_data = true
 end
 
-if not ARGV.empty?
+if !ARGV.empty?
   # paths specified, add them
   db.add_paths(ARGV)
   new_data = true
-elsif not (options[:read] and db_exists)
+elsif !(options[:read] && db_exists)
   # no paths were specified and the database was not read or did not exist;
   # default to building a new DB in the current directory
   db.add_paths(['.'])
@@ -191,9 +199,9 @@ end
 updated = db.update() if options[:update]
 new_data ||= updated
 
-db.save(options[:db]) if options[:write] and (new_data or not db_exists)
+db.save(options[:db]) if options[:write] && (new_data || !db_exists)
 
-export(db, options[:export], options[:output]) if options[:export]
+options[:export].each { |target| export(db, target) }
 
 if options[:query]
   table, query = options[:query].split(',', 2)
@@ -237,7 +245,7 @@ if options[:linemode]
           dump(db, param)
         when "export"
           if param
-            export(db, param, :normal)
+            export(db, param)
           else
             puts "!export requires an argument"
           end
@@ -245,11 +253,11 @@ if options[:linemode]
           print_summary(db)
         when "update"
           changed = db.update
-          db.save(options[:db]) if options[:write] and changed
+          db.save(options[:db]) if options[:write] && changed
         when "help"
           puts linemode_help
         when "version"
-          puts StarScope::VERSION
+          puts Starscope::VERSION
         when "quit"
           exit
         else

data/doc/DB_FORMAT.md

@@ -1,7 +1,7 @@
 Database Format
 ===============
 
-The StarScope database format has gone through a couple of iterations over the
+The Starscope database format has gone through a couple of iterations over the
 last year, but the current one is pretty nice and I don't see it changing again
 in the near future. We magically read old formats and convert them, so we only
 document the current version here.
@@ -17,8 +17,8 @@ compression, the file has three lines:
 Version Number
 --------------
 
-This is just the ascii character '5'. If the format changes signficantly, it
-will be increased, but that is unlikely.
+This is just the ASCII character '5'. If the format changes significantly, it
+will be incremented, but that is unlikely.
 
 Metadata
 --------
@@ -28,7 +28,7 @@ This is a [JSON](https://en.wikipedia.org/wiki/Json) object. Keys include:
  * `:excludes` - the paths and patterns to exclude from scanning
  * `:files` - the files previously scanned (including things like last-modified
    time)
- * `:version` - The StarScope version which wrote the database (not the same as
+ * `:version` - The Starscope version which wrote the database (not the same as
    the database format version).
 
 Databases

data/doc/LANGUAGE_SUPPORT.md

@@ -20,8 +20,9 @@ For this doc, we're going to pretend to add support for a language called
 drop the following template in:
 
 ```ruby
-module StarScope::Lang
+module Starscope::Lang
   module Mylanguage
+    VERSION = 1
 
     def self.match_file(name)
       name.end_with?(".mylang")
@@ -30,13 +31,16 @@ module StarScope::Lang
     def self.extract(file)
       # TODO
     end
-
   end
 end
 ```
 
 This code is pretty simple: we define a module called
-`StarScope::Lang::Mylanguage` and give it two public module methods:
+`Starscope::Lang::Mylanguage` and give it one constant and two public module
+methods:
+ * `VERSION` is a constant integer defining the current version of the
+   extractor. It should be incremented when the extractor has changed enough
+   that any existing files should be re-parsed with the new version.
  * `match_file` takes the name of the file and returns a boolean if that file is
    written in MyLanguage or not. This can be as simple as checking the file
    extension (which the sample code does) or looking for a shell #! line, or
@@ -74,7 +78,7 @@ you want. Some existing ones that you may want to use include:
  * `col` for the column in the line where the name occurs
  * `type` for the type of definition (`:func`, `:class`, etc)
 
-And that's it! Parse your files, yield your records, and the StarScope engine
+And that's it! Parse your files, yield your records, and the Starscope engine
 takes care of everything else for you. If you've added support for a language
 that you think others might find useful, please contribute it (with tests!) via
 pull request.

data/doc/USER_GUIDE.md

@@ -1,6 +1,17 @@
-StarScope User Guide
+Starscope User Guide
 ====================
 
+* [About](#about)
+* [Installation](#installation)
+* [Quick Start](#quick-start)
+* [Editor and Workflow Integration](#editor-and-workflow-integration)
+* [Database Options](#database-options)
+* [Paths](#paths)
+* [Queries](#queries)
+* [Exporting](#exporting)
+* [Line-Mode](#line-mode)
+* [Miscellaneous](#miscellaneous)
+
 About
 -----
 
@@ -8,15 +19,15 @@ Anyone who has done much programming in C (or C++) on a Unix-based OS has come
 across the fantastic [Cscope](http://cscope.sourceforge.net/) tool. Sadly, it
 only works for C (and sort of works for C++).
 
-StarScope is a similar tool for [Ruby](https://www.ruby-lang.org/) and
+Starscope is a similar tool for [Ruby](https://www.ruby-lang.org/) and
 [Golang](http://golang.org/), with a design intended to make it easy to add
-[support for other languages](doc/LANGUAGE_SUPPORT.md) within the same framework
-(thus the name StarScope, i.e. \*scope).
+[support for other languages](LANGUAGE_SUPPORT.md) within the same framework
+(thus the name Starscope, i.e. \*scope).
 
 Installation
 ------------
 
-StarScope is a ruby gem available at https://rubygems.org/gems/starscope.
+Starscope is a ruby gem available at https://rubygems.org/gems/starscope.
 Install it with:
 ```
 $ gem install starscope
@@ -38,12 +49,28 @@ Ask it things directly:
 $ starscope -q calls,new # Lists all callers of new
 ```
 
-Export it to various formats for use with your editor:
+Export it to various existing formats for automatic integration with your editor:
 ```
 $ starscope -e ctags
 $ starscope -e cscope
 ```
 
+Editor and Workflow Integration
+-------------------------------
+
+While I don't know of any editors that interface natively with Starscope
+(if you find one, let me know!) there is a much simpler solution. Almost
+all modern editors know how to read Cscope and Ctags files, and Starscope
+can export to both of those file formats. Simply export to the appropriate
+file (see the section on [Exporting](#exporting) below), then use your
+editor's existing integration and it should just work.
+
+Many people also like to have this kind of tool automatically run when
+certain events happen, such as a `git commit`. Tim Pope has an excellent
+article on [how to do this with Ctags](http://tbaggery.com/2011/08/08/effortless-ctags-with-git.html)
+and with Starscope it is even simpler. Just place the line
+`starscope --quiet -e cscope &` into the hooks documented by Tim.
+
 Database Options
 ----------------
 
@@ -60,9 +87,9 @@ flag.
 Paths
 -----
 
-StarScope has powerful options with sane defaults for managing which paths get
+Starscope has powerful options with sane defaults for managing which paths get
 scanned for files and which do not. By default when creating a new database,
-StarScope will scan all files recursively in the current directory. To scan
+Starscope will scan all files recursively in the current directory. To scan
 specific paths or files instead, pass them as arguments (so `starscope myfolder`
 would only scan files in `myfolder/`).
 
@@ -100,7 +127,7 @@ scope separator).
 Exporting
 ---------
 
-StarScope can export its database into two other formats for use with
+Starscope can export its database into two other formats for use with
 third-party tools:
  * cscope format (default path: `cscope.out`)
  * ctags format (default path: `tags`)