Icarus-Mod-Tools 3.1.0 → 3.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +10 -0
- data/Gemfile.lock +1 -1
- data/README.md +14 -0
- data/lib/icarus/mod/cli/add.rb +18 -2
- data/lib/icarus/mod/cli/validate.rb +22 -0
- data/lib/icarus/mod/github.rb +9 -0
- data/lib/icarus/mod/tools/entry_links.rb +32 -0
- data/lib/icarus/mod/tools/link_probe.rb +116 -0
- data/lib/icarus/mod/tools/repo_check.rb +140 -0
- data/lib/icarus/mod/tools/repo_report.rb +51 -0
- data/lib/icarus/mod/tools/sync/modinfo_list.rb +4 -4
- data/lib/icarus/mod/tools/sync/toolinfo_list.rb +4 -4
- data/lib/icarus/mod/version.rb +1 -1
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f71d742763b16a7a6cb096f6decf06568f415909ee0830f392315523b7ab307d
|
|
4
|
+
data.tar.gz: f5afb462e86cc8452508dad64b319d4c2246cad20a5963741efa43f2b76d873a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 96c885457a5b1e1998eea60852d62474a48c4b1b66ec4ccd4aede80f8300314bde8eb2e9cca637cdae7d6630e00d83fa33b0157af899ef50283e3a97760c25ee
|
|
7
|
+
data.tar.gz: deac9280f7a685695dbe3704ae48b990298a4559b3dd1d5cd5277bb51fef6fe92ab71f0fc8bfb05dda43a6a170c430729c4fac7b5e5f0f291d29f627ead1dba9
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
## History (reverse chronological order)
|
|
6
6
|
|
|
7
|
+
### v3.2.1 - 2026-10-31
|
|
8
|
+
|
|
9
|
+
- Fix `imt add repos owner/repo` storing a bare `owner/repo`, which stage 1 of sync never searched (it only looked at entries containing "github"); it now stores the canonical `https://github.com/owner/repo` for any GitHub form, refuses non-GitHub repositories, and does nothing when the repository is already listed under any spelling
|
|
10
|
+
- Stage 1 of sync (`imt sync modinfo`/`toolinfo`) now also searches bare `owner/repo` entries already in `meta/repos`
|
|
11
|
+
|
|
12
|
+
### v3.2.0 - 2026-10-30
|
|
13
|
+
|
|
14
|
+
- Add `imt validate repo OWNER/REPO [--json]`, a read-only pre-check for a repo before it is added to `meta/repos` (repo state, info files, entry validation and link checks; exits 0/1/2)
|
|
15
|
+
- Link checks never request loopback, private, link-local or other non-public addresses (checked on every redirect, and the request goes to the checked address), since the links come from the repo being vetted
|
|
16
|
+
|
|
7
17
|
### v3.1.0 - 2026-10-30
|
|
8
18
|
|
|
9
19
|
- `imt sync mods`/`tools --dry-run` now lists the entries a real run would delete, and warns when the mass-delete brake would refuse (a real run would exit 1 without `--force`)
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -160,6 +160,20 @@ Options:
|
|
|
160
160
|
|
|
161
161
|
**Note**: Sync operations automatically convert GitHub URLs to the correct `raw.githubusercontent.com` format for direct downloads. If you're using GitHub URLs in `modinfo.json` or `toolinfo.json` files, you can use either `/blob/` or `/raw/` formats and they will be automatically converted. Warnings will be displayed when URLs are auto-fixed so you can update your source files.
|
|
162
162
|
|
|
163
|
+
#### `imt validate`
|
|
164
|
+
|
|
165
|
+
```sh
|
|
166
|
+
Commands:
|
|
167
|
+
imt validate help [COMMAND] # Describe subcommands or one specific subcommand
|
|
168
|
+
imt validate modinfo # Reads modinfo data from 'meta/modinfo/list' and Validates syntax of modfiles
|
|
169
|
+
imt validate repo OWNER/REPO # Vets a GitHub repo before it is added to 'meta/repos/list'
|
|
170
|
+
imt validate toolinfo # Reads Toolinfo data from 'meta/toolinfo/list' and Validates syntax of toolfiles
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
`imt validate repo OWNER/REPO [--json]` (a `github.com` URL also works) vets a repo someone asked to have added, before you add it. It is read-only, never touches Firestore, and needs only the GitHub token from the config. It checks that the repo exists, is public and is not archived; finds `modinfo.json` and/or `toolinfo.json` (at least one is required); validates every entry; and requests every file, image and readme link (following up to 5 redirects) to confirm it resolves. A broken download link is an error, a broken image or readme link is a warning, and a download hosted outside GitHub or a tool shipping an `.exe` is a warning. Use `-v`/`-vv` for more detail, or `--json` to print the machine-readable result (`repo`, `exists`, `private`, `archived`, `default_branch`, `info_files`, `problems`, `entries`, `valid`) and nothing else on stdout.
|
|
174
|
+
|
|
175
|
+
**Exit status**: 0 when the repo is valid, 1 when it is not, 2 on usage, config or GitHub API errors (for example a rate limit or bad token).
|
|
176
|
+
|
|
163
177
|
## Development
|
|
164
178
|
|
|
165
179
|
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
data/lib/icarus/mod/cli/add.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "firestore"
|
|
4
|
+
require "github"
|
|
4
5
|
require "tools/modinfo"
|
|
5
6
|
|
|
6
7
|
module Icarus
|
|
@@ -22,7 +23,16 @@ module Icarus
|
|
|
22
23
|
|
|
23
24
|
desc "repos", "Adds an entry to 'meta/repos/list'"
|
|
24
25
|
def repos(item)
|
|
25
|
-
|
|
26
|
+
# Stage 1 of sync searches entries in the canonical URL form, so store that whatever form we were given
|
|
27
|
+
url = Github.canonical_url(item)
|
|
28
|
+
unless url
|
|
29
|
+
warn Paint["#{item} is not a GitHub repository (use owner/repo or a github.com URL)", :red]
|
|
30
|
+
exit 1
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
add_to_list(:repositories, :repositories, url) do |existing|
|
|
34
|
+
existing.any? { |entry| Github.canonical_url(entry)&.casecmp?(url) }
|
|
35
|
+
end
|
|
26
36
|
end
|
|
27
37
|
|
|
28
38
|
desc "mod", "Adds an entry to 'mods'"
|
|
@@ -78,7 +88,13 @@ module Icarus
|
|
|
78
88
|
end
|
|
79
89
|
|
|
80
90
|
firestore = Firestore.new
|
|
81
|
-
|
|
91
|
+
existing = firestore.send(collection)
|
|
92
|
+
if block_given? && yield(existing)
|
|
93
|
+
puts "#{item} is already listed"
|
|
94
|
+
return
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
payload = [existing, item].flatten.compact
|
|
82
98
|
|
|
83
99
|
exit 1 unless report(firestore.update(type, payload, merge: true))
|
|
84
100
|
end
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "tools/validator"
|
|
4
|
+
require "tools/repo_check"
|
|
5
|
+
require "tools/repo_report"
|
|
4
6
|
|
|
5
7
|
module Icarus
|
|
6
8
|
module Mod
|
|
@@ -17,6 +19,26 @@ module Icarus
|
|
|
17
19
|
validate(:toolinfo)
|
|
18
20
|
end
|
|
19
21
|
|
|
22
|
+
desc "repo OWNER/REPO", "Vets a GitHub repo (and its modinfo/toolinfo links) before it is added to 'meta/repos/list'; read-only"
|
|
23
|
+
method_option :json, type: :boolean, default: false, desc: "Print the result as JSON"
|
|
24
|
+
long_desc <<~DESC
|
|
25
|
+
Exits 0 when the repo is valid, 1 when it is not, and 2 on usage, config or GitHub API errors.
|
|
26
|
+
Never writes to Firestore and does not need it; only the GitHub token from the config is used.
|
|
27
|
+
DESC
|
|
28
|
+
def repo(name)
|
|
29
|
+
result = Icarus::Mod::Tools::RepoCheck.new(name).call
|
|
30
|
+
|
|
31
|
+
if options[:json]
|
|
32
|
+
puts JSON.generate(result)
|
|
33
|
+
else
|
|
34
|
+
Icarus::Mod::Tools::RepoReport.new(result, verbose:).print
|
|
35
|
+
end
|
|
36
|
+
exit(result[:valid] ? 0 : 1)
|
|
37
|
+
rescue Icarus::Mod::Tools::RepoCheck::Error => e
|
|
38
|
+
warn "Error: #{e.message}"
|
|
39
|
+
exit 2
|
|
40
|
+
end
|
|
41
|
+
|
|
20
42
|
no_commands do
|
|
21
43
|
def validate(type)
|
|
22
44
|
exit_code = 0
|
data/lib/icarus/mod/github.rb
CHANGED
|
@@ -8,6 +8,15 @@ module Icarus
|
|
|
8
8
|
class Github
|
|
9
9
|
attr_reader :client, :resources
|
|
10
10
|
|
|
11
|
+
# owner/repo, optionally as a github.com URL (with .git, a trailing slash or a deeper path)
|
|
12
|
+
REPO_PATTERN = %r{\A(?:(?:https?://)?(?:www\.)?github\.com/)?([A-Za-z0-9][\w.-]*)/([\w.-]+?)(?:\.git)?(?:[/?#].*)?\z}
|
|
13
|
+
|
|
14
|
+
# The form meta/repos stores ("https://github.com/owner/repo"), or nil when +repo+ is not a GitHub repository
|
|
15
|
+
def self.canonical_url(repo)
|
|
16
|
+
match = REPO_PATTERN.match(repo.to_s.strip)
|
|
17
|
+
match && "https://github.com/#{match[1]}/#{match[2]}"
|
|
18
|
+
end
|
|
19
|
+
|
|
11
20
|
def initialize(repo = nil)
|
|
12
21
|
self.repository = repo if repo
|
|
13
22
|
@client = Octokit::Client.new(access_token: Config.github.token)
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "uri"
|
|
4
|
+
require "tools/modinfo"
|
|
5
|
+
|
|
6
|
+
module Icarus
|
|
7
|
+
module Mod
|
|
8
|
+
module Tools
|
|
9
|
+
# The links inside a Modinfo/Toolinfo entry that RepoCheck probes
|
|
10
|
+
module EntryLinks
|
|
11
|
+
module_function
|
|
12
|
+
|
|
13
|
+
# [field, url, download?] for every non-blank string link in the entry
|
|
14
|
+
def for(info)
|
|
15
|
+
links = info.is_a?(Modinfo) ? mod_links(info) : [["fileURL", info.fileURL, true]]
|
|
16
|
+
links += [["imageURL", info.imageURL, false], ["readmeURL", info.readmeURL, false]]
|
|
17
|
+
links.select { |_, url, _| url.is_a?(String) && !url.strip.empty? }
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def mod_links(info)
|
|
21
|
+
info.file_urls.zip(info.file_types).map { |url, name| ["files.#{name}", url, true] }
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def windows_tool?(info)
|
|
25
|
+
info.fileType.to_s.match?(/\A\.?exe\z/i) || URI.parse(info.fileURL.to_s).path.to_s.match?(/\.exe\z/i)
|
|
26
|
+
rescue URI::InvalidURIError
|
|
27
|
+
false
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
end
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "ipaddr"
|
|
4
|
+
require "resolv"
|
|
5
|
+
require "uri"
|
|
6
|
+
require "net/http"
|
|
7
|
+
require "tools/sync/helpers"
|
|
8
|
+
|
|
9
|
+
module Icarus
|
|
10
|
+
module Mod
|
|
11
|
+
module Tools
|
|
12
|
+
# Requests a URL (following redirects) without downloading it, to see where it ends up
|
|
13
|
+
class LinkProbe
|
|
14
|
+
MAX_REDIRECTS = 5
|
|
15
|
+
REDIRECT_CODES = [301, 302, 303, 307, 308].freeze
|
|
16
|
+
# Statuses that mean "HEAD is not allowed here", so try a ranged GET instead
|
|
17
|
+
HEAD_REFUSED = [403, 405, 501].freeze
|
|
18
|
+
OK_STATUSES = [200, 206].freeze
|
|
19
|
+
|
|
20
|
+
# Links come from a stranger's modinfo.json and are probed from the maintainer's network, and the status
|
|
21
|
+
# ends up in a public comment: never request loopback, private, link-local or other non-public addresses
|
|
22
|
+
NON_PUBLIC = %w[0.0.0.0/8 100.64.0.0/10 192.0.0.0/24 198.18.0.0/15 224.0.0.0/3 ::/128 64:ff9b::/96 ff00::/8]
|
|
23
|
+
.map { |range| IPAddr.new(range) }.freeze
|
|
24
|
+
|
|
25
|
+
# http: a callable (uri, verb, ip) -> [status, location]; resolver: a callable (host) -> [addresses].
|
|
26
|
+
# Both are replaceable so specs need no network.
|
|
27
|
+
def initialize(http: nil, resolver: nil)
|
|
28
|
+
@http = http || method(:net_request)
|
|
29
|
+
@resolver = resolver || Resolv.method(:getaddresses)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def self.github_host?(host)
|
|
33
|
+
host == "github.com" || host.to_s.end_with?(".github.com") || host == "githubusercontent.com" ||
|
|
34
|
+
host.to_s.end_with?(".githubusercontent.com")
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# Follows up to MAX_REDIRECTS redirects; returns [status, final_host, problem]
|
|
38
|
+
def call(url)
|
|
39
|
+
uri = URI.parse(url)
|
|
40
|
+
status = nil
|
|
41
|
+
(MAX_REDIRECTS + 1).times do
|
|
42
|
+
status, location, refused = hop(uri)
|
|
43
|
+
return [status, uri.host, refused] if refused || !(REDIRECT_CODES.include?(status) && location)
|
|
44
|
+
|
|
45
|
+
uri = URI.join(uri, location)
|
|
46
|
+
end
|
|
47
|
+
[status, uri.host, "too many redirects (more than #{MAX_REDIRECTS})"]
|
|
48
|
+
rescue URI::Error, *Sync::Helpers::NETWORK_ERRORS => e
|
|
49
|
+
[nil, nil, "request failed (#{e.message})"]
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
private
|
|
53
|
+
|
|
54
|
+
# One request, after checking where it would go: [status, location] or [nil, nil, refusal]
|
|
55
|
+
def hop(uri)
|
|
56
|
+
ips, refused = public_addresses(uri)
|
|
57
|
+
refused ? [nil, nil, refused] : request(uri, ips)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
# The addresses to connect to (IPv4 first), or a reason to refuse. Every address the host resolves to must be
|
|
61
|
+
# public, and the request then goes to a checked address, so a DNS answer that changes in between cannot
|
|
62
|
+
# redirect it.
|
|
63
|
+
def public_addresses(uri)
|
|
64
|
+
return [nil, "refused: only http and https links are checked"] unless %w[http https].include?(uri.scheme)
|
|
65
|
+
|
|
66
|
+
host = uri.hostname
|
|
67
|
+
addresses = ip_literal?(host) ? [host] : @resolver.call(host)
|
|
68
|
+
return [nil, "refused: #{uri.host} does not resolve"] if addresses.empty?
|
|
69
|
+
return [nil, "refused: #{uri.host} resolves to a non-public address"] unless addresses.all? { |a| public_ip?(a) }
|
|
70
|
+
|
|
71
|
+
[addresses.partition { |a| IPAddr.new(a).ipv4? }.flatten, nil]
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
def ip_literal?(host)
|
|
75
|
+
IPAddr.new(host)
|
|
76
|
+
true
|
|
77
|
+
rescue IPAddr::InvalidAddressError
|
|
78
|
+
false
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def public_ip?(address)
|
|
82
|
+
ip = IPAddr.new(address)
|
|
83
|
+
ip = ip.native if ip.ipv4_mapped?
|
|
84
|
+
!(ip.private? || ip.loopback? || ip.link_local? || NON_PUBLIC.any? { |range| range.include?(ip) })
|
|
85
|
+
rescue IPAddr::InvalidAddressError
|
|
86
|
+
false
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
# HEAD, then a one-byte ranged GET if HEAD is refused. An unreachable address (no IPv6 route, say) falls back
|
|
90
|
+
# to the next checked one; the last failure propagates.
|
|
91
|
+
def request(uri, ips)
|
|
92
|
+
ips.each_with_index do |ip, index|
|
|
93
|
+
result = @http.call(uri, "HEAD", ip)
|
|
94
|
+
return HEAD_REFUSED.include?(result.first) ? @http.call(uri, "GET", ip) : result
|
|
95
|
+
rescue *Sync::Helpers::NETWORK_ERRORS
|
|
96
|
+
raise if index == ips.size - 1
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
def net_request(uri, verb, ip)
|
|
101
|
+
req = verb == "GET" ? Net::HTTP::Get.new(uri.request_uri, "Range" => "bytes=0-0") : Net::HTTP::Head.new(uri.request_uri)
|
|
102
|
+
# Yield from the block so a server that ignores Range never has its whole body downloaded
|
|
103
|
+
connection(uri, ip).start { |conn| conn.request(req) { |res| return [res.code.to_i, res["location"]] } }
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def connection(uri, ip)
|
|
107
|
+
Net::HTTP.new(uri.host, uri.port).tap do |http|
|
|
108
|
+
http.ipaddr = ip
|
|
109
|
+
http.use_ssl = uri.scheme == "https"
|
|
110
|
+
http.open_timeout = http.read_timeout = 15
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
end
|
|
116
|
+
end
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "github"
|
|
4
|
+
require "tools/modinfo"
|
|
5
|
+
require "tools/toolinfo"
|
|
6
|
+
require "tools/link_probe"
|
|
7
|
+
require "tools/entry_links"
|
|
8
|
+
require "tools/sync/helpers"
|
|
9
|
+
|
|
10
|
+
module Icarus
|
|
11
|
+
module Mod
|
|
12
|
+
module Tools
|
|
13
|
+
# Read-only vetting of a GitHub repo before it is added to meta/repos: checks the repo itself, its
|
|
14
|
+
# modinfo.json/toolinfo.json and every link they contain. Needs only the GitHub token, never Firestore.
|
|
15
|
+
class RepoCheck
|
|
16
|
+
include Sync::Helpers
|
|
17
|
+
|
|
18
|
+
# Usage or GitHub API failure (as opposed to a repo that simply fails validation)
|
|
19
|
+
class Error < StandardError; end
|
|
20
|
+
|
|
21
|
+
TYPES = {
|
|
22
|
+
modinfo: { file: "modinfo.json", type: "mod", key: :mods, entity: Modinfo },
|
|
23
|
+
toolinfo: { file: "toolinfo.json", type: "tool", key: :tools, entity: Toolinfo }
|
|
24
|
+
}.freeze
|
|
25
|
+
|
|
26
|
+
# http/resolver: see LinkProbe; replaceable so specs need no network
|
|
27
|
+
def initialize(repo, github: nil, http: nil, resolver: nil)
|
|
28
|
+
@github = github || Github.new
|
|
29
|
+
@github.repository = repo
|
|
30
|
+
@probe = LinkProbe.new(http:, resolver:)
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def call
|
|
34
|
+
result = blank_result(repo_name)
|
|
35
|
+
check_repo(result[:repo], result)
|
|
36
|
+
result[:valid] = valid?(result)
|
|
37
|
+
result
|
|
38
|
+
rescue Octokit::Error => e
|
|
39
|
+
raise Error, "GitHub API error: #{e.class.name.split("::").last}: #{e.message.lines.first&.strip}"
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
private
|
|
43
|
+
|
|
44
|
+
def repo_name
|
|
45
|
+
repo = @github.repository.to_s.delete_suffix("/").delete_suffix(".git")
|
|
46
|
+
raise Error, "Repository must be given as OWNER/REPO" unless %r{\A[\w.-]+/[\w.-]+\z}.match?(repo)
|
|
47
|
+
|
|
48
|
+
repo
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def blank_result(repo)
|
|
52
|
+
{ repo:, exists: false, private: false, archived: false, default_branch: nil,
|
|
53
|
+
info_files: { modinfo: nil, toolinfo: nil }, problems: [], entries: [], valid: false }
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def check_repo(repo, result)
|
|
57
|
+
record_repo(@github.client.repository(repo), result)
|
|
58
|
+
check_info_files(result) unless result[:private] || result[:archived]
|
|
59
|
+
rescue Octokit::NotFound
|
|
60
|
+
result[:problems] << "repository not found"
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def record_repo(data, result)
|
|
64
|
+
result.merge!(exists: true, private: data[:private] ? true : false, archived: data[:archived] ? true : false,
|
|
65
|
+
default_branch: data[:default_branch])
|
|
66
|
+
result[:problems] << "repository is private" if result[:private]
|
|
67
|
+
result[:problems] << "repository is archived" if result[:archived]
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def check_info_files(result)
|
|
71
|
+
TYPES.each do |kind, meta|
|
|
72
|
+
file = @github.find(meta[:file])
|
|
73
|
+
next unless file
|
|
74
|
+
|
|
75
|
+
result[:info_files][kind] = file[:download_url]
|
|
76
|
+
check_info_file(meta, file[:download_url], result)
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
result[:problems] << "no modinfo.json or toolinfo.json found" if result[:info_files].values.none?
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def check_info_file(meta, url, result)
|
|
83
|
+
entries = entries_from(meta, retrieve_from_url(url))
|
|
84
|
+
entries.each_with_index do |data, index|
|
|
85
|
+
if usable_name?(data)
|
|
86
|
+
result[:entries] << check_entry(meta, data)
|
|
87
|
+
else
|
|
88
|
+
result[:problems] << "#{meta[:file]}: entry #{index + 1} has no usable name (sync would skip it)"
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
rescue Sync::RequestFailed, JSON::ParserError => e
|
|
92
|
+
result[:problems] << "#{meta[:file]}: #{e.message}"
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
# Same rules as Sync::Entities#entries_from
|
|
96
|
+
def entries_from(meta, body)
|
|
97
|
+
entries = body[meta[:key]] if body.is_a?(Hash)
|
|
98
|
+
raise JSON::ParserError, "no '#{meta[:key]}' array" unless entries.is_a?(Array)
|
|
99
|
+
raise JSON::ParserError, "'#{meta[:key]}' entries must be objects" unless entries.all?(Hash)
|
|
100
|
+
|
|
101
|
+
entries
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def usable_name?(data)
|
|
105
|
+
data[:name].is_a?(String) && /[a-z0-9]+/i.match?(data[:name])
|
|
106
|
+
end
|
|
107
|
+
|
|
108
|
+
def check_entry(meta, data)
|
|
109
|
+
info = meta[:entity].new(data)
|
|
110
|
+
entry = entry_for(meta[:type], info)
|
|
111
|
+
|
|
112
|
+
entry[:warnings] << "tool ships a Windows executable" if meta[:type] == "tool" && EntryLinks.windows_tool?(info)
|
|
113
|
+
EntryLinks.for(info).each { |field, url, download| check_link(entry, field, url, download) }
|
|
114
|
+
entry
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def entry_for(type, info)
|
|
118
|
+
{ type:, name: info.name, author: info.author, version: info.version,
|
|
119
|
+
errors: info.errors.dup, warnings: info.warnings.dup, links: [] }
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
def check_link(entry, field, url, download)
|
|
123
|
+
status, host, problem = @probe.call(url)
|
|
124
|
+
entry[:links] << { field:, url:, status:, final_host: host }
|
|
125
|
+
|
|
126
|
+
if problem || !LinkProbe::OK_STATUSES.include?(status)
|
|
127
|
+
entry[download ? :errors : :warnings] << "#{field}: #{problem || "HTTP #{status}"} (#{url})"
|
|
128
|
+
elsif download && !LinkProbe.github_host?(host)
|
|
129
|
+
entry[:warnings] << "download hosted off GitHub: #{host}"
|
|
130
|
+
end
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
def valid?(result)
|
|
134
|
+
result[:exists] && result[:info_files].values.any? && result[:problems].empty? &&
|
|
135
|
+
result[:entries].none? { |entry| entry[:errors].any? }
|
|
136
|
+
end
|
|
137
|
+
end
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
end
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "paint"
|
|
4
|
+
|
|
5
|
+
module Icarus
|
|
6
|
+
module Mod
|
|
7
|
+
module Tools
|
|
8
|
+
# Human-readable output for a RepoCheck result, in the style of `imt validate modinfo`
|
|
9
|
+
class RepoReport
|
|
10
|
+
def initialize(result, verbose: 1)
|
|
11
|
+
@result = result
|
|
12
|
+
@verbose = verbose
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def print
|
|
16
|
+
print_header
|
|
17
|
+
@result[:problems].each { |problem| puts Paint["#{problem} in #{@result[:repo]}", :red] }
|
|
18
|
+
@result[:entries].each { |entry| print_entry(entry) }
|
|
19
|
+
print_verdict if @verbose.positive?
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
private
|
|
23
|
+
|
|
24
|
+
def print_header
|
|
25
|
+
puts Paint["Validating #{@result[:repo]}", :cyan, :bright] if @verbose.positive?
|
|
26
|
+
return unless @verbose > 1
|
|
27
|
+
|
|
28
|
+
puts Paint["Default branch: #{@result[:default_branch]}", :cyan]
|
|
29
|
+
@result[:info_files].each { |kind, url| puts "#{kind}: #{url || "not found"}" }
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def print_verdict
|
|
33
|
+
valid = @result[:valid]
|
|
34
|
+
puts Paint[valid ? "VALID" : "NOT VALID", valid ? :green : :red, :bright]
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def print_links(entry, name)
|
|
38
|
+
puts Paint["Checked #{entry[:type]} #{name}", :cyan, :bright]
|
|
39
|
+
entry[:links].each { |link| puts " #{link[:field]}: #{link[:status] || "failed"} #{link[:final_host]}" }
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def print_entry(entry)
|
|
43
|
+
name = "#{entry[:author].to_s.strip}/#{entry[:name].to_s.strip}"
|
|
44
|
+
print_links(entry, name) if @verbose > 1
|
|
45
|
+
entry[:errors].each { |error| puts Paint["#{error} in #{name}", :red] }
|
|
46
|
+
entry[:warnings].each { |warning| puts Paint["#{warning} in #{name}", :yellow] }
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
end
|
|
@@ -41,11 +41,11 @@ module Icarus
|
|
|
41
41
|
repositories.map do |repo|
|
|
42
42
|
print "searching #{repo}..." if verbose
|
|
43
43
|
|
|
44
|
-
|
|
45
|
-
|
|
44
|
+
# A GitHub URL, or a bare owner/repo (how `imt add repos` stored entries before 3.2.1)
|
|
45
|
+
if Github.canonical_url(repo)
|
|
46
46
|
find_modinfo(repo, verbose:)
|
|
47
|
-
|
|
48
|
-
puts "Skipped...repository type not supported yet"
|
|
47
|
+
elsif verbose
|
|
48
|
+
puts "Skipped...repository type not supported yet"
|
|
49
49
|
end
|
|
50
50
|
end.compact
|
|
51
51
|
end
|
|
@@ -41,11 +41,11 @@ module Icarus
|
|
|
41
41
|
repositories.map do |repo|
|
|
42
42
|
print "searching #{repo}..." if verbose
|
|
43
43
|
|
|
44
|
-
|
|
45
|
-
|
|
44
|
+
# A GitHub URL, or a bare owner/repo (how `imt add repos` stored entries before 3.2.1)
|
|
45
|
+
if Github.canonical_url(repo)
|
|
46
46
|
find_toolinfo(repo, verbose:)
|
|
47
|
-
|
|
48
|
-
puts "Skipped...repository type not supported yet"
|
|
47
|
+
elsif verbose
|
|
48
|
+
puts "Skipped...repository type not supported yet"
|
|
49
49
|
end
|
|
50
50
|
end.compact
|
|
51
51
|
end
|
data/lib/icarus/mod/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: Icarus-Mod-Tools
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.1
|
|
4
|
+
version: 3.2.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Donovan Young
|
|
@@ -116,7 +116,11 @@ files:
|
|
|
116
116
|
- lib/icarus/mod/github.rb
|
|
117
117
|
- lib/icarus/mod/tools.rb
|
|
118
118
|
- lib/icarus/mod/tools/baseinfo.rb
|
|
119
|
+
- lib/icarus/mod/tools/entry_links.rb
|
|
120
|
+
- lib/icarus/mod/tools/link_probe.rb
|
|
119
121
|
- lib/icarus/mod/tools/modinfo.rb
|
|
122
|
+
- lib/icarus/mod/tools/repo_check.rb
|
|
123
|
+
- lib/icarus/mod/tools/repo_report.rb
|
|
120
124
|
- lib/icarus/mod/tools/sync/entities.rb
|
|
121
125
|
- lib/icarus/mod/tools/sync/helpers.rb
|
|
122
126
|
- lib/icarus/mod/tools/sync/modinfo_list.rb
|