reapack-index 1.0beta2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +3 -0
- data/.travis.yml +8 -0
- data/Gemfile +3 -0
- data/README.md +167 -0
- data/Rakefile +7 -0
- data/bin/reapack-index +8 -0
- data/lib/reapack/index.rb +351 -0
- data/lib/reapack/index/cli.rb +437 -0
- data/lib/reapack/index/gem_version.rb +5 -0
- data/lib/reapack/index/metadata.rb +185 -0
- data/lib/reapack/index/named_node.rb +68 -0
- data/lib/reapack/index/package.rb +65 -0
- data/lib/reapack/index/parsers.rb +35 -0
- data/lib/reapack/index/version.rb +178 -0
- data/reapack-index.gemspec +33 -0
- data/setup/.gitignore +1 -0
- data/setup/reapack-index.nsi +148 -0
- data/test/data/index.xml +11 -0
- data/test/data/noindex.lua +1 -0
- data/test/helper.rb +25 -0
- data/test/test_changelog.rb +76 -0
- data/test/test_cli.rb +846 -0
- data/test/test_index.rb +817 -0
- data/test/test_metadata.rb +403 -0
- data/test/test_named_node.rb +100 -0
- data/test/test_package.rb +95 -0
- data/test/test_parsers.rb +35 -0
- data/test/test_version.rb +215 -0
- metadata +238 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: a9af1c4bbcd9cb1cf695d80ee3a492b32c93445e
|
4
|
+
data.tar.gz: b97e2c07c313617286159908c8adce89ecc783cc
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 85bde223e337b5aefcad7e6a6c9f657ae27ad9994c4e78a96ba355a2acaf7975b7c54116013423984e43e7a86fd461f9407d4d123b8791ea63a59164b6ffb488
|
7
|
+
data.tar.gz: 5421422da2210cb6502471db950d97f17476c5dd3cee39b334dfb9117d00439f0ce237cd80c7da2b0939d43091e782c5895831880c303f4754505e6c08c5ca7a
|
data/.gitignore
ADDED
data/.travis.yml
ADDED
data/Gemfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,167 @@
|
|
1
|
+
# Package indexer for ReaPack-based repositories
|
2
|
+
|
3
|
+
Parent project: [https://github.com/cfillion/reapack](https://github.com/cfillion/reapack)
|
4
|
+
Subproject: [https://github.com/cfillion/metaheader](https://github.com/cfillion/metaheader)
|
5
|
+
|
6
|
+
[![Gem Version](https://badge.fury.io/rb/reapack-index.svg)](http://badge.fury.io/rb/reapack-index)
|
7
|
+
[![Build Status](https://travis-ci.org/cfillion/reapack-index.svg?branch=master)](https://travis-ci.org/cfillion/reapack-index)
|
8
|
+
[![Dependency Status](https://gemnasium.com/cfillion/reapack-index.svg)](https://gemnasium.com/cfillion/reapack-index)
|
9
|
+
[![Coverage Status](https://coveralls.io/repos/cfillion/reapack-index/badge.svg?branch=master&service=github)](https://coveralls.io/github/cfillion/reapack-index?branch=master)
|
10
|
+
|
11
|
+
### Installation
|
12
|
+
|
13
|
+
Ruby 2 need to be installed on your computer and ready to be used from a command
|
14
|
+
prompt. Run the following command from a command prompt (cmd.exe, Terminal.app,
|
15
|
+
XTerm) to install reapack-index on your computer:
|
16
|
+
|
17
|
+
```
|
18
|
+
gem install reapack-index
|
19
|
+
```
|
20
|
+
|
21
|
+
### Usage
|
22
|
+
|
23
|
+
```
|
24
|
+
reascript-indexer [options] [path-to-your-reascript-repository]
|
25
|
+
```
|
26
|
+
|
27
|
+
```
|
28
|
+
Options:
|
29
|
+
-a, --[no-]amend Reindex existing versions
|
30
|
+
-c, --check Test every package including uncommited changes and exit
|
31
|
+
-o, --output FILE=./index.xml Set the output filename and path for the index
|
32
|
+
-l, --link LINK Add or remove a website link
|
33
|
+
--donation-link LINK Add or remove a donation link
|
34
|
+
--ls-links Display the link list then exit
|
35
|
+
-A, --about=FILE Set the about content from a file
|
36
|
+
--remove-about Remove the about content from the index
|
37
|
+
--dump-about Dump the raw about content in RTF and exit
|
38
|
+
--[no-]progress Enable or disable progress information
|
39
|
+
-V, --[no-]verbose Activate diagnosis messages
|
40
|
+
-C, --[no-]commit Select whether to commit the modified index
|
41
|
+
--prompt-commit Ask at runtime whether to commit the index
|
42
|
+
-W, --warnings Enable warnings
|
43
|
+
-w, --no-warnings Turn off warnings
|
44
|
+
-q, --[no-]quiet Disable almost all output
|
45
|
+
--no-config Bypass the configuration files
|
46
|
+
-v, --version Display version information
|
47
|
+
-h, --help Prints this help
|
48
|
+
```
|
49
|
+
|
50
|
+
### Configuration
|
51
|
+
|
52
|
+
Options can be specified from the command line or stored in configuration files.
|
53
|
+
The syntax is the same as the command line, but with a single option per line.
|
54
|
+
|
55
|
+
The settings are applied in the following order:
|
56
|
+
|
57
|
+
- ~/.reapack-index.conf (`~` = home directory)
|
58
|
+
- ./.reapack-index.conf (`.` = repository root)
|
59
|
+
- command line
|
60
|
+
|
61
|
+
## Packaging Documentation
|
62
|
+
|
63
|
+
This indexer uses metadata found at the start of the files to generate the
|
64
|
+
database in ReaPack format.
|
65
|
+
See also [MetaHeader](https://github.com/cfillion/metaheader)'s documentation.
|
66
|
+
|
67
|
+
Tag not explicitly marked as required are optional.
|
68
|
+
|
69
|
+
### Package type by extension:
|
70
|
+
|
71
|
+
- `.lua`, `.eel`, `.py`: ReaScripts – the package file itself will be used as a source.
|
72
|
+
- `.ext`: For REAPER native extensions
|
73
|
+
|
74
|
+
### Package Tags
|
75
|
+
|
76
|
+
These tags affects an entire package. Changes to any of those tags are
|
77
|
+
applied immediately and may affect released versions.
|
78
|
+
|
79
|
+
**@noindex**
|
80
|
+
|
81
|
+
Disable indexing for this file. Set this on included files that
|
82
|
+
should not be distributed alone.
|
83
|
+
|
84
|
+
```
|
85
|
+
@noindex
|
86
|
+
|
87
|
+
NoIndex: true
|
88
|
+
```
|
89
|
+
|
90
|
+
**@version** [required]
|
91
|
+
|
92
|
+
The current package version.
|
93
|
+
Value must contain between one and four groups of digits.
|
94
|
+
|
95
|
+
```
|
96
|
+
@version 1.0
|
97
|
+
@version 1.2pre3
|
98
|
+
|
99
|
+
Version: 0.2015.12.25
|
100
|
+
```
|
101
|
+
|
102
|
+
### Version Tags
|
103
|
+
|
104
|
+
These tags are specific to a single package version. You may still edit them
|
105
|
+
after a release by running the indexer with the `--amend` option.
|
106
|
+
|
107
|
+
**@author**
|
108
|
+
|
109
|
+
```
|
110
|
+
@author cfillion
|
111
|
+
|
112
|
+
Author: Christian Fillion
|
113
|
+
```
|
114
|
+
|
115
|
+
**@changelog**
|
116
|
+
|
117
|
+
```
|
118
|
+
@changelog
|
119
|
+
Documented the metadata syntax
|
120
|
+
Added support for deleted scripts
|
121
|
+
|
122
|
+
Changelog:
|
123
|
+
Added an alternate syntax for metadata tags
|
124
|
+
```
|
125
|
+
|
126
|
+
**@provides**
|
127
|
+
|
128
|
+
Add additional files to the package. This is also used to add platform restrictions
|
129
|
+
or set a custom download url (by default the download url is based on the "origin"
|
130
|
+
git remote). These files will be installed/updated together with the package.
|
131
|
+
|
132
|
+
```
|
133
|
+
@provides unicode.dat
|
134
|
+
|
135
|
+
Provides:
|
136
|
+
Images/background.png
|
137
|
+
Images/fader_small.png
|
138
|
+
Images/fader_big.png
|
139
|
+
|
140
|
+
@provides
|
141
|
+
[windows] reaper_extension.dll http://mysite.com/download/$version/$path
|
142
|
+
```
|
143
|
+
|
144
|
+
List of supported platform strings:
|
145
|
+
- `windows`: All versions of Windows
|
146
|
+
- `win32`: Windows 32-bit
|
147
|
+
- `win64`: Windows 64-bit
|
148
|
+
- `darwin`: All versions of OS X
|
149
|
+
- `darwin32`: OS X 32-bit
|
150
|
+
- `darwin64`: OS X 64-bit
|
151
|
+
|
152
|
+
The following variables will be interpolated if found in the URL:
|
153
|
+
- `$path`: The path of the file relative to the package
|
154
|
+
- `$commit`: The hash of the commit being indexed or "master" if unavailable
|
155
|
+
- `$version`: The version of the package being indexed
|
156
|
+
|
157
|
+
Platform restriction and custom url can be set for the package itself,
|
158
|
+
either by using its file name or a dot:
|
159
|
+
|
160
|
+
```
|
161
|
+
-- this is a lua script named `hello_osx.lua`
|
162
|
+
-- @provides
|
163
|
+
-- [darwin] hello_osx.lua
|
164
|
+
|
165
|
+
-- @provides
|
166
|
+
-- [darwin] .
|
167
|
+
```
|
data/Rakefile
ADDED
data/bin/reapack-index
ADDED
@@ -0,0 +1,351 @@
|
|
1
|
+
require 'reapack/index/gem_version'
|
2
|
+
|
3
|
+
require 'colorize'
|
4
|
+
require 'fileutils'
|
5
|
+
require 'io/console'
|
6
|
+
require 'metaheader'
|
7
|
+
require 'nokogiri'
|
8
|
+
require 'optparse'
|
9
|
+
require 'pandoc-ruby'
|
10
|
+
require 'pathname'
|
11
|
+
require 'rugged'
|
12
|
+
require 'shellwords'
|
13
|
+
require 'time'
|
14
|
+
require 'uri'
|
15
|
+
|
16
|
+
require 'reapack/index/cli'
|
17
|
+
require 'reapack/index/metadata'
|
18
|
+
require 'reapack/index/named_node'
|
19
|
+
require 'reapack/index/package'
|
20
|
+
require 'reapack/index/parsers'
|
21
|
+
require 'reapack/index/version'
|
22
|
+
|
23
|
+
class ReaPack::Index
|
24
|
+
Error = Class.new RuntimeError
|
25
|
+
|
26
|
+
FILE_TYPES = {
|
27
|
+
'lua' => :script,
|
28
|
+
'eel' => :script,
|
29
|
+
'py' => :script,
|
30
|
+
'ext' => :extension,
|
31
|
+
}.freeze
|
32
|
+
|
33
|
+
PROVIDES_VALIDATOR = proc {|value|
|
34
|
+
begin
|
35
|
+
files = value.lines.map {|l|
|
36
|
+
m = l.chomp.match PROVIDES_REGEX
|
37
|
+
Source.validate_platform m[:platform]
|
38
|
+
[m[:platform], m[:file]]
|
39
|
+
}
|
40
|
+
dup = files.detect {|f| files.count(f) > 1 }
|
41
|
+
"duplicate file (%s)" % dup[1] if dup
|
42
|
+
rescue Error => e
|
43
|
+
e.message
|
44
|
+
end
|
45
|
+
}.freeze
|
46
|
+
|
47
|
+
HEADER_RULES = {
|
48
|
+
# package-wide tags
|
49
|
+
:version => /\A(?:[^\d]*\d{1,4}[^\d]*){1,4}\z/,
|
50
|
+
|
51
|
+
# version-specific tags
|
52
|
+
:author => [MetaHeader::OPTIONAL, /\A[^\n]+\z/],
|
53
|
+
:changelog => [MetaHeader::OPTIONAL, /.+/],
|
54
|
+
:provides => [MetaHeader::OPTIONAL, PROVIDES_VALIDATOR]
|
55
|
+
}.freeze
|
56
|
+
|
57
|
+
SOURCE_PATTERNS = {
|
58
|
+
/\Agit@github\.com:([^\/]+)\/(.+)\.git\z/ =>
|
59
|
+
'https://github.com/\1/\2/raw/$commit/$path',
|
60
|
+
/\Ahttps:\/\/github\.com\/([^\/]+)\/(.+)\.git\z/ =>
|
61
|
+
'https://github.com/\1/\2/raw/$commit/$path',
|
62
|
+
}.freeze
|
63
|
+
|
64
|
+
ROOT = File.expand_path('/').freeze
|
65
|
+
|
66
|
+
PROVIDES_REGEX = /
|
67
|
+
\A
|
68
|
+
( \[ \s* (?<platform> .+? ) \s* \] )?
|
69
|
+
\s*
|
70
|
+
(?<file> .+?)
|
71
|
+
( \s+ (?<url> (?:file|https?):\/\/.+ ) )?
|
72
|
+
\z
|
73
|
+
/x.freeze
|
74
|
+
|
75
|
+
WITH_MAIN = [:script].freeze
|
76
|
+
|
77
|
+
attr_reader :path, :source_pattern
|
78
|
+
attr_accessor :amend, :files, :time
|
79
|
+
|
80
|
+
def self.type_of(path)
|
81
|
+
ext = File.extname(path)[1..-1]
|
82
|
+
FILE_TYPES[ext]
|
83
|
+
end
|
84
|
+
|
85
|
+
def self.source_for(url)
|
86
|
+
SOURCE_PATTERNS.each_pair {|regex, pattern|
|
87
|
+
return url.gsub regex, pattern if url =~ regex
|
88
|
+
}
|
89
|
+
|
90
|
+
nil
|
91
|
+
end
|
92
|
+
|
93
|
+
def self.validate_file(path)
|
94
|
+
mh = MetaHeader.from_file path
|
95
|
+
return if mh[:noindex]
|
96
|
+
|
97
|
+
mh.validate HEADER_RULES
|
98
|
+
end
|
99
|
+
|
100
|
+
def initialize(path)
|
101
|
+
@amend = false
|
102
|
+
@changes = {}
|
103
|
+
@files = []
|
104
|
+
@path = path
|
105
|
+
|
106
|
+
if File.exist? path
|
107
|
+
# noblanks: don't preserve the original white spaces
|
108
|
+
# so we always output a neat document
|
109
|
+
@doc = Nokogiri::XML File.open(path), &:noblanks
|
110
|
+
else
|
111
|
+
@dirty = true
|
112
|
+
@is_new = true
|
113
|
+
|
114
|
+
@doc = Nokogiri::XML::Document.new
|
115
|
+
@doc.root = Nokogiri::XML::Node.new 'index', @doc
|
116
|
+
end
|
117
|
+
|
118
|
+
@doc.root[:version] = 1
|
119
|
+
@doc.encoding = 'utf-8'
|
120
|
+
|
121
|
+
@metadata = Metadata.new @doc.root
|
122
|
+
end
|
123
|
+
|
124
|
+
def scan(path, contents)
|
125
|
+
type = self.class.type_of path
|
126
|
+
return unless type
|
127
|
+
|
128
|
+
mh = MetaHeader.new contents
|
129
|
+
|
130
|
+
backup = @doc.root.dup
|
131
|
+
|
132
|
+
if mh[:noindex]
|
133
|
+
remove path
|
134
|
+
return
|
135
|
+
end
|
136
|
+
|
137
|
+
if errors = mh.validate(HEADER_RULES)
|
138
|
+
prefix = errors.size == 1 ? "\x20" : "\n\x20\x20"
|
139
|
+
raise Error, "invalid metadata:%s" %
|
140
|
+
[prefix + errors.join(prefix)]
|
141
|
+
end
|
142
|
+
|
143
|
+
cat, pkg = find path
|
144
|
+
pkg.type = type.to_s
|
145
|
+
|
146
|
+
pkg.version mh[:version] do |ver|
|
147
|
+
next unless ver.is_new? || @amend
|
148
|
+
|
149
|
+
ver.author = mh[:author]
|
150
|
+
ver.time = @time if @time
|
151
|
+
ver.changelog = mh[:changelog]
|
152
|
+
|
153
|
+
ver.replace_sources do
|
154
|
+
sources = parse_provides mh[:provides], path
|
155
|
+
|
156
|
+
if WITH_MAIN.include?(type) && sources.none? {|src| src.file.nil? }
|
157
|
+
# add the package itself as a source
|
158
|
+
sources.unshift Source.new nil, nil, url_for(path)
|
159
|
+
end
|
160
|
+
|
161
|
+
sources.each {|src|
|
162
|
+
# the $path variable is interpolated elsewhere
|
163
|
+
# (in url_for for generated urls and make_sources for explicit urls)
|
164
|
+
src.url.sub! '$commit', commit || 'master'
|
165
|
+
src.url.sub! '$version', ver.name
|
166
|
+
ver.add_source src
|
167
|
+
}
|
168
|
+
end
|
169
|
+
end
|
170
|
+
|
171
|
+
log_change 'new category', 'new categories' if cat.is_new?
|
172
|
+
|
173
|
+
if pkg.is_new?
|
174
|
+
log_change 'new package'
|
175
|
+
elsif pkg.modified?
|
176
|
+
log_change 'modified package'
|
177
|
+
end
|
178
|
+
|
179
|
+
pkg.versions.each {|ver|
|
180
|
+
if ver.is_new?
|
181
|
+
log_change 'new version'
|
182
|
+
elsif ver.modified?
|
183
|
+
log_change 'modified version'
|
184
|
+
end
|
185
|
+
}
|
186
|
+
rescue Error
|
187
|
+
@doc.root = backup
|
188
|
+
raise
|
189
|
+
end
|
190
|
+
|
191
|
+
def remove(path)
|
192
|
+
cat, pkg = find path, false
|
193
|
+
return unless pkg
|
194
|
+
|
195
|
+
pkg.remove
|
196
|
+
cat.remove if cat.empty?
|
197
|
+
|
198
|
+
log_change 'removed package'
|
199
|
+
end
|
200
|
+
|
201
|
+
def links(type)
|
202
|
+
@metadata.links type
|
203
|
+
end
|
204
|
+
|
205
|
+
def eval_link(type, string)
|
206
|
+
if string.index('-') == 0
|
207
|
+
@metadata.remove_link type, string[1..-1]
|
208
|
+
log_change "removed #{type} link"
|
209
|
+
return
|
210
|
+
end
|
211
|
+
|
212
|
+
link = @metadata.push_link type, *string.split('=', 2)
|
213
|
+
|
214
|
+
if link.is_new?
|
215
|
+
log_change "new #{type} link"
|
216
|
+
elsif link.modified?
|
217
|
+
log_change "modified #{type} link"
|
218
|
+
end
|
219
|
+
end
|
220
|
+
|
221
|
+
def description
|
222
|
+
@metadata.description
|
223
|
+
end
|
224
|
+
|
225
|
+
def description=(content)
|
226
|
+
old = @metadata.description
|
227
|
+
@metadata.description = content
|
228
|
+
|
229
|
+
log_change 'modified metadata' if old != @metadata.description
|
230
|
+
end
|
231
|
+
|
232
|
+
def source_pattern=(pattern)
|
233
|
+
if pattern.nil?
|
234
|
+
raise ArgumentError, 'Cannot use nil as a source pattern'
|
235
|
+
elsif not pattern.include? '$path'
|
236
|
+
raise ArgumentError, '$path not in source pattern'
|
237
|
+
end
|
238
|
+
|
239
|
+
@source_pattern = pattern
|
240
|
+
end
|
241
|
+
|
242
|
+
def version
|
243
|
+
@doc.root[:version].to_i
|
244
|
+
end
|
245
|
+
|
246
|
+
def commit
|
247
|
+
@doc.root[:commit]
|
248
|
+
end
|
249
|
+
|
250
|
+
def commit=(sha1)
|
251
|
+
if sha1.nil?
|
252
|
+
@doc.root.remove_attribute 'commit'
|
253
|
+
else
|
254
|
+
@doc.root['commit'] = sha1
|
255
|
+
end
|
256
|
+
end
|
257
|
+
|
258
|
+
def write(path)
|
259
|
+
FileUtils.mkdir_p File.dirname(path)
|
260
|
+
File.write path, @doc.to_xml
|
261
|
+
end
|
262
|
+
|
263
|
+
def write!
|
264
|
+
write @path
|
265
|
+
|
266
|
+
@is_new = @dirty = false
|
267
|
+
@changes.clear
|
268
|
+
end
|
269
|
+
|
270
|
+
def modified?
|
271
|
+
!!@dirty
|
272
|
+
end
|
273
|
+
|
274
|
+
def changelog
|
275
|
+
list = []
|
276
|
+
|
277
|
+
@changes.each_pair {|type, data|
|
278
|
+
count, plural = data
|
279
|
+
list << "#{count} #{count != 1 ? plural : type}"
|
280
|
+
}
|
281
|
+
|
282
|
+
list << 'empty index' if @is_new && Category.find_all(@doc.root).empty?
|
283
|
+
|
284
|
+
list.join ', '
|
285
|
+
end
|
286
|
+
|
287
|
+
private
|
288
|
+
def log_change(desc, plural = nil)
|
289
|
+
@dirty = true
|
290
|
+
|
291
|
+
@changes[desc] ||= [0, plural || desc + 's']
|
292
|
+
@changes[desc][0] += 1
|
293
|
+
end
|
294
|
+
|
295
|
+
def dirname(path)
|
296
|
+
name = File.dirname path
|
297
|
+
name == '.' ? nil : name
|
298
|
+
end
|
299
|
+
|
300
|
+
def find(path, create = true)
|
301
|
+
cat_name = dirname(path) || 'Other'
|
302
|
+
pkg_name = File.basename path
|
303
|
+
|
304
|
+
cat = Category.get cat_name, @doc.root, create
|
305
|
+
pkg = Package.get pkg_name, cat && cat.node, create
|
306
|
+
|
307
|
+
[cat, pkg]
|
308
|
+
end
|
309
|
+
|
310
|
+
def url_for(path)
|
311
|
+
unless @source_pattern
|
312
|
+
raise Error, "source pattern is unset " \
|
313
|
+
"and the package doesn't specify its source url"
|
314
|
+
end
|
315
|
+
|
316
|
+
unless @files.include? path
|
317
|
+
raise Error, "#{path}: No such file or directory"
|
318
|
+
end
|
319
|
+
|
320
|
+
# other variables are interpolated in scan()
|
321
|
+
@source_pattern.sub('$path', path)
|
322
|
+
end
|
323
|
+
|
324
|
+
def parse_provides(provides, base)
|
325
|
+
this_file = File.basename base
|
326
|
+
basedir = dirname base
|
327
|
+
|
328
|
+
provides.to_s.lines.map {|line|
|
329
|
+
line.chomp!
|
330
|
+
|
331
|
+
m = line.match PROVIDES_REGEX
|
332
|
+
|
333
|
+
platform, file, url = m[:platform], m[:file], m[:url]
|
334
|
+
file = nil if file == this_file || file == '.'
|
335
|
+
|
336
|
+
if url.nil?
|
337
|
+
if file.nil?
|
338
|
+
url = url_for base
|
339
|
+
else
|
340
|
+
path = File.expand_path file, ROOT + basedir.to_s
|
341
|
+
url = url_for path[ROOT.size..-1]
|
342
|
+
end
|
343
|
+
else
|
344
|
+
# for explicit urls which don't go through url_for
|
345
|
+
url.sub! '$path', file || base
|
346
|
+
end
|
347
|
+
|
348
|
+
Source.new platform, file, url
|
349
|
+
}
|
350
|
+
end
|
351
|
+
end
|