reapack-index 1.0beta2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: a9af1c4bbcd9cb1cf695d80ee3a492b32c93445e
4
+ data.tar.gz: b97e2c07c313617286159908c8adce89ecc783cc
5
+ SHA512:
6
+ metadata.gz: 85bde223e337b5aefcad7e6a6c9f657ae27ad9994c4e78a96ba355a2acaf7975b7c54116013423984e43e7a86fd461f9407d4d123b8791ea63a59164b6ffb488
7
+ data.tar.gz: 5421422da2210cb6502471db950d97f17476c5dd3cee39b334dfb9117d00439f0ce237cd80c7da2b0939d43091e782c5895831880c303f4754505e6c08c5ca7a
@@ -0,0 +1,3 @@
1
+ coverage/
2
+ pkg/
3
+ Gemfile.lock
@@ -0,0 +1,8 @@
1
+ language: ruby
2
+ rvm:
3
+ - 2.2.3
4
+ before_install:
5
+ - gem update bundler
6
+ - sudo apt-get -qq update
7
+ - sudo apt-get install -y pandoc
8
+ cache: bundler
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "https://rubygems.org"
2
+
3
+ gemspec
@@ -0,0 +1,167 @@
1
+ # Package indexer for ReaPack-based repositories
2
+
3
+ Parent project: [https://github.com/cfillion/reapack](https://github.com/cfillion/reapack)
4
+ Subproject: [https://github.com/cfillion/metaheader](https://github.com/cfillion/metaheader)
5
+
6
+ [![Gem Version](https://badge.fury.io/rb/reapack-index.svg)](http://badge.fury.io/rb/reapack-index)
7
+ [![Build Status](https://travis-ci.org/cfillion/reapack-index.svg?branch=master)](https://travis-ci.org/cfillion/reapack-index)
8
+ [![Dependency Status](https://gemnasium.com/cfillion/reapack-index.svg)](https://gemnasium.com/cfillion/reapack-index)
9
+ [![Coverage Status](https://coveralls.io/repos/cfillion/reapack-index/badge.svg?branch=master&service=github)](https://coveralls.io/github/cfillion/reapack-index?branch=master)
10
+
11
+ ### Installation
12
+
13
+ Ruby 2 need to be installed on your computer and ready to be used from a command
14
+ prompt. Run the following command from a command prompt (cmd.exe, Terminal.app,
15
+ XTerm) to install reapack-index on your computer:
16
+
17
+ ```
18
+ gem install reapack-index
19
+ ```
20
+
21
+ ### Usage
22
+
23
+ ```
24
+ reascript-indexer [options] [path-to-your-reascript-repository]
25
+ ```
26
+
27
+ ```
28
+ Options:
29
+ -a, --[no-]amend Reindex existing versions
30
+ -c, --check Test every package including uncommited changes and exit
31
+ -o, --output FILE=./index.xml Set the output filename and path for the index
32
+ -l, --link LINK Add or remove a website link
33
+ --donation-link LINK Add or remove a donation link
34
+ --ls-links Display the link list then exit
35
+ -A, --about=FILE Set the about content from a file
36
+ --remove-about Remove the about content from the index
37
+ --dump-about Dump the raw about content in RTF and exit
38
+ --[no-]progress Enable or disable progress information
39
+ -V, --[no-]verbose Activate diagnosis messages
40
+ -C, --[no-]commit Select whether to commit the modified index
41
+ --prompt-commit Ask at runtime whether to commit the index
42
+ -W, --warnings Enable warnings
43
+ -w, --no-warnings Turn off warnings
44
+ -q, --[no-]quiet Disable almost all output
45
+ --no-config Bypass the configuration files
46
+ -v, --version Display version information
47
+ -h, --help Prints this help
48
+ ```
49
+
50
+ ### Configuration
51
+
52
+ Options can be specified from the command line or stored in configuration files.
53
+ The syntax is the same as the command line, but with a single option per line.
54
+
55
+ The settings are applied in the following order:
56
+
57
+ - ~/.reapack-index.conf (`~` = home directory)
58
+ - ./.reapack-index.conf (`.` = repository root)
59
+ - command line
60
+
61
+ ## Packaging Documentation
62
+
63
+ This indexer uses metadata found at the start of the files to generate the
64
+ database in ReaPack format.
65
+ See also [MetaHeader](https://github.com/cfillion/metaheader)'s documentation.
66
+
67
+ Tag not explicitly marked as required are optional.
68
+
69
+ ### Package type by extension:
70
+
71
+ - `.lua`, `.eel`, `.py`: ReaScripts – the package file itself will be used as a source.
72
+ - `.ext`: For REAPER native extensions
73
+
74
+ ### Package Tags
75
+
76
+ These tags affects an entire package. Changes to any of those tags are
77
+ applied immediately and may affect released versions.
78
+
79
+ **@noindex**
80
+
81
+ Disable indexing for this file. Set this on included files that
82
+ should not be distributed alone.
83
+
84
+ ```
85
+ @noindex
86
+
87
+ NoIndex: true
88
+ ```
89
+
90
+ **@version** [required]
91
+
92
+ The current package version.
93
+ Value must contain between one and four groups of digits.
94
+
95
+ ```
96
+ @version 1.0
97
+ @version 1.2pre3
98
+
99
+ Version: 0.2015.12.25
100
+ ```
101
+
102
+ ### Version Tags
103
+
104
+ These tags are specific to a single package version. You may still edit them
105
+ after a release by running the indexer with the `--amend` option.
106
+
107
+ **@author**
108
+
109
+ ```
110
+ @author cfillion
111
+
112
+ Author: Christian Fillion
113
+ ```
114
+
115
+ **@changelog**
116
+
117
+ ```
118
+ @changelog
119
+ Documented the metadata syntax
120
+ Added support for deleted scripts
121
+
122
+ Changelog:
123
+ Added an alternate syntax for metadata tags
124
+ ```
125
+
126
+ **@provides**
127
+
128
+ Add additional files to the package. This is also used to add platform restrictions
129
+ or set a custom download url (by default the download url is based on the "origin"
130
+ git remote). These files will be installed/updated together with the package.
131
+
132
+ ```
133
+ @provides unicode.dat
134
+
135
+ Provides:
136
+ Images/background.png
137
+ Images/fader_small.png
138
+ Images/fader_big.png
139
+
140
+ @provides
141
+ [windows] reaper_extension.dll http://mysite.com/download/$version/$path
142
+ ```
143
+
144
+ List of supported platform strings:
145
+ - `windows`: All versions of Windows
146
+ - `win32`: Windows 32-bit
147
+ - `win64`: Windows 64-bit
148
+ - `darwin`: All versions of OS X
149
+ - `darwin32`: OS X 32-bit
150
+ - `darwin64`: OS X 64-bit
151
+
152
+ The following variables will be interpolated if found in the URL:
153
+ - `$path`: The path of the file relative to the package
154
+ - `$commit`: The hash of the commit being indexed or "master" if unavailable
155
+ - `$version`: The version of the package being indexed
156
+
157
+ Platform restriction and custom url can be set for the package itself,
158
+ either by using its file name or a dot:
159
+
160
+ ```
161
+ -- this is a lua script named `hello_osx.lua`
162
+ -- @provides
163
+ -- [darwin] hello_osx.lua
164
+
165
+ -- @provides
166
+ -- [darwin] .
167
+ ```
@@ -0,0 +1,7 @@
1
+ require "bundler/gem_tasks"
2
+ require 'rake/testtask'
3
+
4
+ task :default => [:test]
5
+ Rake::TestTask.new do |t|
6
+ t.test_files = FileList['test/test_*.rb']
7
+ end
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ Signal.trap('INT') { abort }
4
+
5
+ require 'reapack/index'
6
+
7
+ String.disable_colorization = !STDOUT.tty? || !STDERR.tty?
8
+ exit !!ReaPack::Index::CLI.new(ARGV).run
@@ -0,0 +1,351 @@
1
+ require 'reapack/index/gem_version'
2
+
3
+ require 'colorize'
4
+ require 'fileutils'
5
+ require 'io/console'
6
+ require 'metaheader'
7
+ require 'nokogiri'
8
+ require 'optparse'
9
+ require 'pandoc-ruby'
10
+ require 'pathname'
11
+ require 'rugged'
12
+ require 'shellwords'
13
+ require 'time'
14
+ require 'uri'
15
+
16
+ require 'reapack/index/cli'
17
+ require 'reapack/index/metadata'
18
+ require 'reapack/index/named_node'
19
+ require 'reapack/index/package'
20
+ require 'reapack/index/parsers'
21
+ require 'reapack/index/version'
22
+
23
+ class ReaPack::Index
24
+ Error = Class.new RuntimeError
25
+
26
+ FILE_TYPES = {
27
+ 'lua' => :script,
28
+ 'eel' => :script,
29
+ 'py' => :script,
30
+ 'ext' => :extension,
31
+ }.freeze
32
+
33
+ PROVIDES_VALIDATOR = proc {|value|
34
+ begin
35
+ files = value.lines.map {|l|
36
+ m = l.chomp.match PROVIDES_REGEX
37
+ Source.validate_platform m[:platform]
38
+ [m[:platform], m[:file]]
39
+ }
40
+ dup = files.detect {|f| files.count(f) > 1 }
41
+ "duplicate file (%s)" % dup[1] if dup
42
+ rescue Error => e
43
+ e.message
44
+ end
45
+ }.freeze
46
+
47
+ HEADER_RULES = {
48
+ # package-wide tags
49
+ :version => /\A(?:[^\d]*\d{1,4}[^\d]*){1,4}\z/,
50
+
51
+ # version-specific tags
52
+ :author => [MetaHeader::OPTIONAL, /\A[^\n]+\z/],
53
+ :changelog => [MetaHeader::OPTIONAL, /.+/],
54
+ :provides => [MetaHeader::OPTIONAL, PROVIDES_VALIDATOR]
55
+ }.freeze
56
+
57
+ SOURCE_PATTERNS = {
58
+ /\Agit@github\.com:([^\/]+)\/(.+)\.git\z/ =>
59
+ 'https://github.com/\1/\2/raw/$commit/$path',
60
+ /\Ahttps:\/\/github\.com\/([^\/]+)\/(.+)\.git\z/ =>
61
+ 'https://github.com/\1/\2/raw/$commit/$path',
62
+ }.freeze
63
+
64
+ ROOT = File.expand_path('/').freeze
65
+
66
+ PROVIDES_REGEX = /
67
+ \A
68
+ ( \[ \s* (?<platform> .+? ) \s* \] )?
69
+ \s*
70
+ (?<file> .+?)
71
+ ( \s+ (?<url> (?:file|https?):\/\/.+ ) )?
72
+ \z
73
+ /x.freeze
74
+
75
+ WITH_MAIN = [:script].freeze
76
+
77
+ attr_reader :path, :source_pattern
78
+ attr_accessor :amend, :files, :time
79
+
80
+ def self.type_of(path)
81
+ ext = File.extname(path)[1..-1]
82
+ FILE_TYPES[ext]
83
+ end
84
+
85
+ def self.source_for(url)
86
+ SOURCE_PATTERNS.each_pair {|regex, pattern|
87
+ return url.gsub regex, pattern if url =~ regex
88
+ }
89
+
90
+ nil
91
+ end
92
+
93
+ def self.validate_file(path)
94
+ mh = MetaHeader.from_file path
95
+ return if mh[:noindex]
96
+
97
+ mh.validate HEADER_RULES
98
+ end
99
+
100
+ def initialize(path)
101
+ @amend = false
102
+ @changes = {}
103
+ @files = []
104
+ @path = path
105
+
106
+ if File.exist? path
107
+ # noblanks: don't preserve the original white spaces
108
+ # so we always output a neat document
109
+ @doc = Nokogiri::XML File.open(path), &:noblanks
110
+ else
111
+ @dirty = true
112
+ @is_new = true
113
+
114
+ @doc = Nokogiri::XML::Document.new
115
+ @doc.root = Nokogiri::XML::Node.new 'index', @doc
116
+ end
117
+
118
+ @doc.root[:version] = 1
119
+ @doc.encoding = 'utf-8'
120
+
121
+ @metadata = Metadata.new @doc.root
122
+ end
123
+
124
+ def scan(path, contents)
125
+ type = self.class.type_of path
126
+ return unless type
127
+
128
+ mh = MetaHeader.new contents
129
+
130
+ backup = @doc.root.dup
131
+
132
+ if mh[:noindex]
133
+ remove path
134
+ return
135
+ end
136
+
137
+ if errors = mh.validate(HEADER_RULES)
138
+ prefix = errors.size == 1 ? "\x20" : "\n\x20\x20"
139
+ raise Error, "invalid metadata:%s" %
140
+ [prefix + errors.join(prefix)]
141
+ end
142
+
143
+ cat, pkg = find path
144
+ pkg.type = type.to_s
145
+
146
+ pkg.version mh[:version] do |ver|
147
+ next unless ver.is_new? || @amend
148
+
149
+ ver.author = mh[:author]
150
+ ver.time = @time if @time
151
+ ver.changelog = mh[:changelog]
152
+
153
+ ver.replace_sources do
154
+ sources = parse_provides mh[:provides], path
155
+
156
+ if WITH_MAIN.include?(type) && sources.none? {|src| src.file.nil? }
157
+ # add the package itself as a source
158
+ sources.unshift Source.new nil, nil, url_for(path)
159
+ end
160
+
161
+ sources.each {|src|
162
+ # the $path variable is interpolated elsewhere
163
+ # (in url_for for generated urls and make_sources for explicit urls)
164
+ src.url.sub! '$commit', commit || 'master'
165
+ src.url.sub! '$version', ver.name
166
+ ver.add_source src
167
+ }
168
+ end
169
+ end
170
+
171
+ log_change 'new category', 'new categories' if cat.is_new?
172
+
173
+ if pkg.is_new?
174
+ log_change 'new package'
175
+ elsif pkg.modified?
176
+ log_change 'modified package'
177
+ end
178
+
179
+ pkg.versions.each {|ver|
180
+ if ver.is_new?
181
+ log_change 'new version'
182
+ elsif ver.modified?
183
+ log_change 'modified version'
184
+ end
185
+ }
186
+ rescue Error
187
+ @doc.root = backup
188
+ raise
189
+ end
190
+
191
+ def remove(path)
192
+ cat, pkg = find path, false
193
+ return unless pkg
194
+
195
+ pkg.remove
196
+ cat.remove if cat.empty?
197
+
198
+ log_change 'removed package'
199
+ end
200
+
201
+ def links(type)
202
+ @metadata.links type
203
+ end
204
+
205
+ def eval_link(type, string)
206
+ if string.index('-') == 0
207
+ @metadata.remove_link type, string[1..-1]
208
+ log_change "removed #{type} link"
209
+ return
210
+ end
211
+
212
+ link = @metadata.push_link type, *string.split('=', 2)
213
+
214
+ if link.is_new?
215
+ log_change "new #{type} link"
216
+ elsif link.modified?
217
+ log_change "modified #{type} link"
218
+ end
219
+ end
220
+
221
+ def description
222
+ @metadata.description
223
+ end
224
+
225
+ def description=(content)
226
+ old = @metadata.description
227
+ @metadata.description = content
228
+
229
+ log_change 'modified metadata' if old != @metadata.description
230
+ end
231
+
232
+ def source_pattern=(pattern)
233
+ if pattern.nil?
234
+ raise ArgumentError, 'Cannot use nil as a source pattern'
235
+ elsif not pattern.include? '$path'
236
+ raise ArgumentError, '$path not in source pattern'
237
+ end
238
+
239
+ @source_pattern = pattern
240
+ end
241
+
242
+ def version
243
+ @doc.root[:version].to_i
244
+ end
245
+
246
+ def commit
247
+ @doc.root[:commit]
248
+ end
249
+
250
+ def commit=(sha1)
251
+ if sha1.nil?
252
+ @doc.root.remove_attribute 'commit'
253
+ else
254
+ @doc.root['commit'] = sha1
255
+ end
256
+ end
257
+
258
+ def write(path)
259
+ FileUtils.mkdir_p File.dirname(path)
260
+ File.write path, @doc.to_xml
261
+ end
262
+
263
+ def write!
264
+ write @path
265
+
266
+ @is_new = @dirty = false
267
+ @changes.clear
268
+ end
269
+
270
+ def modified?
271
+ !!@dirty
272
+ end
273
+
274
+ def changelog
275
+ list = []
276
+
277
+ @changes.each_pair {|type, data|
278
+ count, plural = data
279
+ list << "#{count} #{count != 1 ? plural : type}"
280
+ }
281
+
282
+ list << 'empty index' if @is_new && Category.find_all(@doc.root).empty?
283
+
284
+ list.join ', '
285
+ end
286
+
287
+ private
288
+ def log_change(desc, plural = nil)
289
+ @dirty = true
290
+
291
+ @changes[desc] ||= [0, plural || desc + 's']
292
+ @changes[desc][0] += 1
293
+ end
294
+
295
+ def dirname(path)
296
+ name = File.dirname path
297
+ name == '.' ? nil : name
298
+ end
299
+
300
+ def find(path, create = true)
301
+ cat_name = dirname(path) || 'Other'
302
+ pkg_name = File.basename path
303
+
304
+ cat = Category.get cat_name, @doc.root, create
305
+ pkg = Package.get pkg_name, cat && cat.node, create
306
+
307
+ [cat, pkg]
308
+ end
309
+
310
+ def url_for(path)
311
+ unless @source_pattern
312
+ raise Error, "source pattern is unset " \
313
+ "and the package doesn't specify its source url"
314
+ end
315
+
316
+ unless @files.include? path
317
+ raise Error, "#{path}: No such file or directory"
318
+ end
319
+
320
+ # other variables are interpolated in scan()
321
+ @source_pattern.sub('$path', path)
322
+ end
323
+
324
+ def parse_provides(provides, base)
325
+ this_file = File.basename base
326
+ basedir = dirname base
327
+
328
+ provides.to_s.lines.map {|line|
329
+ line.chomp!
330
+
331
+ m = line.match PROVIDES_REGEX
332
+
333
+ platform, file, url = m[:platform], m[:file], m[:url]
334
+ file = nil if file == this_file || file == '.'
335
+
336
+ if url.nil?
337
+ if file.nil?
338
+ url = url_for base
339
+ else
340
+ path = File.expand_path file, ROOT + basedir.to_s
341
+ url = url_for path[ROOT.size..-1]
342
+ end
343
+ else
344
+ # for explicit urls which don't go through url_for
345
+ url.sub! '$path', file || base
346
+ end
347
+
348
+ Source.new platform, file, url
349
+ }
350
+ end
351
+ end