reapack-index 1.0beta2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a9af1c4bbcd9cb1cf695d80ee3a492b32c93445e
+  data.tar.gz: b97e2c07c313617286159908c8adce89ecc783cc
+SHA512:
+  metadata.gz: 85bde223e337b5aefcad7e6a6c9f657ae27ad9994c4e78a96ba355a2acaf7975b7c54116013423984e43e7a86fd461f9407d4d123b8791ea63a59164b6ffb488
+  data.tar.gz: 5421422da2210cb6502471db950d97f17476c5dd3cee39b334dfb9117d00439f0ce237cd80c7da2b0939d43091e782c5895831880c303f4754505e6c08c5ca7a

data/.gitignore

@@ -0,0 +1,3 @@
+coverage/
+pkg/
+Gemfile.lock

data/.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+rvm:
+  - 2.2.3
+before_install:
+  - gem update bundler
+  - sudo apt-get -qq update
+  - sudo apt-get install -y pandoc
+cache: bundler

data/Gemfile

@@ -0,0 +1,3 @@
+source "https://rubygems.org"
+
+gemspec

data/README.md

@@ -0,0 +1,167 @@
+# Package indexer for ReaPack-based repositories
+
+Parent project: [https://github.com/cfillion/reapack](https://github.com/cfillion/reapack)  
+Subproject: [https://github.com/cfillion/metaheader](https://github.com/cfillion/metaheader)
+
+[![Gem Version](https://badge.fury.io/rb/reapack-index.svg)](http://badge.fury.io/rb/reapack-index)
+[![Build Status](https://travis-ci.org/cfillion/reapack-index.svg?branch=master)](https://travis-ci.org/cfillion/reapack-index)
+[![Dependency Status](https://gemnasium.com/cfillion/reapack-index.svg)](https://gemnasium.com/cfillion/reapack-index)
+[![Coverage Status](https://coveralls.io/repos/cfillion/reapack-index/badge.svg?branch=master&service=github)](https://coveralls.io/github/cfillion/reapack-index?branch=master)
+
+### Installation
+
+Ruby 2 need to be installed on your computer and ready to be used from a command
+prompt. Run the following command from a command prompt (cmd.exe, Terminal.app,
+XTerm) to install reapack-index on your computer:
+
+```
+gem install reapack-index
+```
+
+### Usage
+
+```
+reascript-indexer [options] [path-to-your-reascript-repository]
+```
+
+```
+Options:
+    -a, --[no-]amend                 Reindex existing versions
+    -c, --check                      Test every package including uncommited changes and exit
+    -o, --output FILE=./index.xml    Set the output filename and path for the index
+    -l, --link LINK                  Add or remove a website link
+        --donation-link LINK         Add or remove a donation link
+        --ls-links                   Display the link list then exit
+    -A, --about=FILE                 Set the about content from a file
+        --remove-about               Remove the about content from the index
+        --dump-about                 Dump the raw about content in RTF and exit
+        --[no-]progress              Enable or disable progress information
+    -V, --[no-]verbose               Activate diagnosis messages
+    -C, --[no-]commit                Select whether to commit the modified index
+        --prompt-commit              Ask at runtime whether to commit the index
+    -W, --warnings                   Enable warnings
+    -w, --no-warnings                Turn off warnings
+    -q, --[no-]quiet                 Disable almost all output
+        --no-config                  Bypass the configuration files
+    -v, --version                    Display version information
+    -h, --help                       Prints this help
+```
+
+### Configuration
+
+Options can be specified from the command line or stored in configuration files.
+The syntax is the same as the command line, but with a single option per line.
+
+The settings are applied in the following order:
+
+- ~/.reapack-index.conf (`~` = home directory)
+- ./.reapack-index.conf (`.` = repository root)
+- command line
+
+## Packaging Documentation
+
+This indexer uses metadata found at the start of the files to generate the
+database in ReaPack format.
+See also [MetaHeader](https://github.com/cfillion/metaheader)'s documentation.
+
+Tag not explicitly marked as required are optional.
+
+### Package type by extension:
+
+- `.lua`, `.eel`, `.py`: ReaScripts – the package file itself will be used as a source.
+- `.ext`: For REAPER native extensions
+
+### Package Tags
+
+These tags affects an entire package. Changes to any of those tags are
+applied immediately and may affect released versions.
+
+**@noindex**
+
+Disable indexing for this file. Set this on included files that
+should not be distributed alone.
+
+```
+@noindex
+
+NoIndex: true
+```
+
+**@version** [required]
+
+The current package version.
+Value must contain between one and four groups of digits.
+
+```
+@version 1.0
+@version 1.2pre3
+
+Version: 0.2015.12.25
+```
+
+### Version Tags
+
+These tags are specific to a single package version. You may still edit them
+after a release by running the indexer with the `--amend` option.
+
+**@author**
+
+```
+@author cfillion
+
+Author: Christian Fillion
+```
+
+**@changelog**
+
+```
+@changelog
+  Documented the metadata syntax
+  Added support for deleted scripts
+
+Changelog:
+  Added an alternate syntax for metadata tags
+```
+
+**@provides**
+
+Add additional files to the package. This is also used to add platform restrictions
+or set a custom download url (by default the download url is based on the "origin"
+git remote).  These files will be installed/updated together with the package.
+
+```
+@provides unicode.dat
+
+Provides:
+  Images/background.png
+  Images/fader_small.png
+  Images/fader_big.png
+
+@provides
+  [windows] reaper_extension.dll http://mysite.com/download/$version/$path
+```
+
+List of supported platform strings:
+- `windows`: All versions of Windows
+- `win32`: Windows 32-bit
+- `win64`: Windows 64-bit
+- `darwin`: All versions of OS X
+- `darwin32`: OS X 32-bit
+- `darwin64`: OS X 64-bit
+
+The following variables will be interpolated if found in the URL:
+- `$path`: The path of the file relative to the package
+- `$commit`: The hash of the commit being indexed or "master" if unavailable
+- `$version`: The version of the package being indexed
+
+Platform restriction and custom url can be set for the package itself,
+either by using its file name or a dot:
+
+```
+-- this is a lua script named `hello_osx.lua`
+-- @provides
+--   [darwin] hello_osx.lua
+
+-- @provides
+--   [darwin] .
+```

data/Rakefile

@@ -0,0 +1,7 @@
+require "bundler/gem_tasks"
+require 'rake/testtask'
+
+task :default => [:test]
+Rake::TestTask.new do |t|
+  t.test_files = FileList['test/test_*.rb']
+end

data/bin/reapack-index

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+
+Signal.trap('INT') { abort }
+
+require 'reapack/index'
+
+String.disable_colorization = !STDOUT.tty? || !STDERR.tty?
+exit !!ReaPack::Index::CLI.new(ARGV).run

data/lib/reapack/index.rb

@@ -0,0 +1,351 @@
+require 'reapack/index/gem_version'
+
+require 'colorize'
+require 'fileutils'
+require 'io/console'
+require 'metaheader'
+require 'nokogiri'
+require 'optparse'
+require 'pandoc-ruby'
+require 'pathname'
+require 'rugged'
+require 'shellwords'
+require 'time'
+require 'uri'
+
+require 'reapack/index/cli'
+require 'reapack/index/metadata'
+require 'reapack/index/named_node'
+require 'reapack/index/package'
+require 'reapack/index/parsers'
+require 'reapack/index/version'
+
+class ReaPack::Index
+  Error = Class.new RuntimeError
+
+  FILE_TYPES = {
+    'lua' => :script,
+    'eel' => :script,
+    'py'  => :script,
+    'ext' => :extension,
+  }.freeze
+
+  PROVIDES_VALIDATOR = proc {|value|
+    begin
+      files = value.lines.map {|l|
+        m = l.chomp.match PROVIDES_REGEX
+        Source.validate_platform m[:platform]
+        [m[:platform], m[:file]]
+      }
+      dup = files.detect {|f| files.count(f) > 1 }
+      "duplicate file (%s)" % dup[1] if dup
+    rescue Error => e
+      e.message
+    end
+  }.freeze
+
+  HEADER_RULES = {
+    # package-wide tags
+    :version => /\A(?:[^\d]*\d{1,4}[^\d]*){1,4}\z/,
+
+    # version-specific tags
+    :author => [MetaHeader::OPTIONAL, /\A[^\n]+\z/],
+    :changelog => [MetaHeader::OPTIONAL, /.+/],
+    :provides => [MetaHeader::OPTIONAL, PROVIDES_VALIDATOR]
+  }.freeze
+
+  SOURCE_PATTERNS = {
+    /\Agit@github\.com:([^\/]+)\/(.+)\.git\z/ =>
+      'https://github.com/\1/\2/raw/$commit/$path',
+    /\Ahttps:\/\/github\.com\/([^\/]+)\/(.+)\.git\z/ =>
+      'https://github.com/\1/\2/raw/$commit/$path',
+  }.freeze
+
+  ROOT = File.expand_path('/').freeze
+
+  PROVIDES_REGEX = /
+    \A
+    ( \[ \s* (?<platform> .+? ) \s* \] )?
+    \s*
+    (?<file> .+?)
+    ( \s+ (?<url> (?:file|https?):\/\/.+ ) )?
+    \z
+  /x.freeze
+
+  WITH_MAIN = [:script].freeze
+
+  attr_reader :path, :source_pattern
+  attr_accessor :amend, :files, :time
+
+  def self.type_of(path)
+    ext = File.extname(path)[1..-1]
+    FILE_TYPES[ext]
+  end
+
+  def self.source_for(url)
+    SOURCE_PATTERNS.each_pair {|regex, pattern|
+      return url.gsub regex, pattern if url =~ regex
+    }
+
+    nil
+  end
+
+  def self.validate_file(path)
+    mh = MetaHeader.from_file path
+    return if mh[:noindex]
+
+    mh.validate HEADER_RULES
+  end
+
+  def initialize(path)
+    @amend = false
+    @changes = {}
+    @files = []
+    @path = path
+
+    if File.exist? path
+      # noblanks: don't preserve the original white spaces
+      # so we always output a neat document
+      @doc = Nokogiri::XML File.open(path), &:noblanks
+    else
+      @dirty = true
+      @is_new = true
+
+      @doc = Nokogiri::XML::Document.new
+      @doc.root = Nokogiri::XML::Node.new 'index', @doc
+    end
+
+    @doc.root[:version] = 1
+    @doc.encoding = 'utf-8'
+
+    @metadata = Metadata.new @doc.root
+  end
+
+  def scan(path, contents)
+    type = self.class.type_of path
+    return unless type
+
+    mh = MetaHeader.new contents
+
+    backup = @doc.root.dup
+
+    if mh[:noindex]
+      remove path
+      return
+    end
+
+    if errors = mh.validate(HEADER_RULES)
+      prefix = errors.size == 1 ? "\x20" : "\n\x20\x20"
+      raise Error, "invalid metadata:%s" %
+        [prefix + errors.join(prefix)]
+    end
+
+    cat, pkg = find path
+    pkg.type = type.to_s
+
+    pkg.version mh[:version] do |ver|
+      next unless ver.is_new? || @amend
+
+      ver.author = mh[:author]
+      ver.time = @time if @time
+      ver.changelog = mh[:changelog]
+
+      ver.replace_sources do
+        sources = parse_provides mh[:provides], path
+
+        if WITH_MAIN.include?(type) && sources.none? {|src| src.file.nil? }
+          # add the package itself as a source
+          sources.unshift Source.new nil, nil, url_for(path)
+        end
+
+        sources.each {|src|
+          # the $path variable is interpolated elsewhere
+          # (in url_for for generated urls and make_sources for explicit urls)
+          src.url.sub! '$commit', commit || 'master'
+          src.url.sub! '$version', ver.name
+          ver.add_source src
+        }
+      end
+    end
+
+    log_change 'new category', 'new categories' if cat.is_new?
+
+    if pkg.is_new?
+      log_change 'new package'
+    elsif pkg.modified?
+      log_change 'modified package'
+    end
+
+    pkg.versions.each {|ver|
+      if ver.is_new?
+        log_change 'new version'
+      elsif ver.modified?
+        log_change 'modified version'
+      end
+    }
+  rescue Error
+    @doc.root = backup
+    raise
+  end
+  
+  def remove(path)
+    cat, pkg = find path, false
+    return unless pkg
+
+    pkg.remove
+    cat.remove if cat.empty?
+
+    log_change 'removed package'
+  end
+
+  def links(type)
+    @metadata.links type
+  end
+
+  def eval_link(type, string)
+    if string.index('-') == 0
+      @metadata.remove_link type, string[1..-1]
+      log_change "removed #{type} link"
+      return
+    end
+
+    link = @metadata.push_link type, *string.split('=', 2)
+
+    if link.is_new?
+      log_change "new #{type} link"
+    elsif link.modified?
+      log_change "modified #{type} link"
+    end
+  end
+
+  def description
+    @metadata.description
+  end
+
+  def description=(content)
+    old = @metadata.description
+    @metadata.description = content
+
+    log_change 'modified metadata' if old != @metadata.description
+  end
+
+  def source_pattern=(pattern)
+    if pattern.nil?
+      raise ArgumentError, 'Cannot use nil as a source pattern'
+    elsif not pattern.include? '$path'
+      raise ArgumentError, '$path not in source pattern'
+    end
+
+    @source_pattern = pattern
+  end
+
+  def version
+    @doc.root[:version].to_i
+  end
+
+  def commit
+    @doc.root[:commit]
+  end
+
+  def commit=(sha1)
+    if sha1.nil?
+      @doc.root.remove_attribute 'commit'
+    else
+      @doc.root['commit'] = sha1
+    end
+  end
+
+  def write(path)
+    FileUtils.mkdir_p File.dirname(path)
+    File.write path, @doc.to_xml
+  end
+
+  def write!
+    write @path
+
+    @is_new = @dirty = false
+    @changes.clear
+  end
+
+  def modified?
+    !!@dirty
+  end
+
+  def changelog
+    list = []
+
+    @changes.each_pair {|type, data|
+      count, plural = data
+      list << "#{count} #{count != 1 ? plural : type}"
+    }
+
+    list << 'empty index' if @is_new && Category.find_all(@doc.root).empty?
+
+    list.join ', '
+  end
+
+private
+  def log_change(desc, plural = nil)
+    @dirty = true
+
+    @changes[desc] ||= [0, plural || desc + 's']
+    @changes[desc][0] += 1
+  end
+
+  def dirname(path)
+    name = File.dirname path
+    name == '.' ? nil : name
+  end
+
+  def find(path, create = true)
+    cat_name = dirname(path) || 'Other'
+    pkg_name = File.basename path
+
+    cat = Category.get cat_name, @doc.root, create
+    pkg = Package.get pkg_name, cat && cat.node, create
+
+    [cat, pkg]
+  end
+
+  def url_for(path)
+    unless @source_pattern
+      raise Error, "source pattern is unset " \
+        "and the package doesn't specify its source url"
+    end
+
+    unless @files.include? path
+      raise Error, "#{path}: No such file or directory"
+    end
+
+    # other variables are interpolated in scan()
+    @source_pattern.sub('$path', path)
+  end
+
+  def parse_provides(provides, base)
+    this_file = File.basename base
+    basedir = dirname base
+
+    provides.to_s.lines.map {|line|
+      line.chomp!
+
+      m = line.match PROVIDES_REGEX
+
+      platform, file, url = m[:platform], m[:file], m[:url]
+      file = nil if file == this_file || file == '.'
+
+      if url.nil?
+        if file.nil?
+          url = url_for base
+        else
+          path = File.expand_path file, ROOT + basedir.to_s
+          url = url_for path[ROOT.size..-1]
+        end
+      else
+        # for explicit urls which don't go through url_for
+        url.sub! '$path', file || base
+      end
+
+      Source.new platform, file, url
+    }
+  end
+end