Icarus-Mod-Tools 2.7.0 → 3.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.rubocop.yml +55 -0
- data/.rubocop_todo.yml +142 -0
- data/.ruby-version +1 -1
- data/CHANGELOG.md +41 -0
- data/CLAUDE.md +4 -4
- data/Gemfile +4 -6
- data/Gemfile.lock +90 -71
- data/Guardfile +2 -0
- data/README.md +8 -2
- data/icarus-mod-tools.gemspec +4 -3
- data/lib/icarus/mod/cli/add.rb +53 -19
- data/lib/icarus/mod/cli/base.rb +4 -4
- data/lib/icarus/mod/cli/command.rb +6 -1
- data/lib/icarus/mod/cli/list.rb +19 -7
- data/lib/icarus/mod/cli/remove.rb +64 -62
- data/lib/icarus/mod/cli/subcommand_base.rb +2 -2
- data/lib/icarus/mod/cli/sync.rb +68 -23
- data/lib/icarus/mod/cli/validate.rb +1 -4
- data/lib/icarus/mod/config.rb +2 -2
- data/lib/icarus/mod/firestore.rb +38 -21
- data/lib/icarus/mod/github.rb +39 -22
- data/lib/icarus/mod/tools/baseinfo.rb +84 -22
- data/lib/icarus/mod/tools/modinfo.rb +1 -1
- data/lib/icarus/mod/tools/sync/entities.rb +102 -0
- data/lib/icarus/mod/tools/sync/helpers.rb +6 -2
- data/lib/icarus/mod/tools/sync/mods.rb +6 -79
- data/lib/icarus/mod/tools/sync/tools.rb +6 -81
- data/lib/icarus/mod/tools/toolinfo.rb +12 -8
- data/lib/icarus/mod/tools/validator.rb +7 -7
- data/lib/icarus/mod/version.rb +1 -1
- metadata +24 -8
- data/sig/database/sync/sync.rbs +0 -15
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1ce21bb90f71e739bad403f4f8c30a29d4f1c10ecd374ecd4012d2273a49d583
|
|
4
|
+
data.tar.gz: ff4adcb3bf12dae9bee5c0570c1c9b93c0ce062888014ee916ebf52fcf66e3ff
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '08bb649e40e165fa13072434fb0f70bb62d7f67a8309f02b7ad06b2b40986510a7176771f711dcbaec0c0b727643b31a3e6d95e877e0e53cfc37f67a761153fb'
|
|
7
|
+
data.tar.gz: 765209817e047371f78c2df064a6294c0ffbb30c40454ad1771cdabba0c93bf5d0c2384d446c38aafd3f4d6eaae587b2f12af09e42a1ccc1f7ea6d19721d2a23
|
data/.rubocop.yml
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
inherit_from: .rubocop_todo.yml
|
|
2
|
+
|
|
3
|
+
plugins:
|
|
4
|
+
- rubocop-rspec
|
|
5
|
+
|
|
6
|
+
AllCops:
|
|
7
|
+
TargetRubyVersion: 4.0
|
|
8
|
+
NewCops: enable
|
|
9
|
+
SuggestExtensions: false
|
|
10
|
+
Exclude:
|
|
11
|
+
- "pkg/**/*"
|
|
12
|
+
- "vendor/**/*"
|
|
13
|
+
|
|
14
|
+
Style/StringLiterals:
|
|
15
|
+
EnforcedStyle: double_quotes
|
|
16
|
+
|
|
17
|
+
Style/StringLiteralsInInterpolation:
|
|
18
|
+
EnforcedStyle: double_quotes
|
|
19
|
+
|
|
20
|
+
Layout/LineLength:
|
|
21
|
+
Max: 140
|
|
22
|
+
|
|
23
|
+
# The specs deliberately share many `let`s, group several expectations per example, mix `have_received`
|
|
24
|
+
# and `receive`, stub with `allow_any_instance_of` for the HTTP helpers, and build configs from OpenStruct
|
|
25
|
+
# (as Config does). These cops would force a rewrite of the suite for no behavior gain.
|
|
26
|
+
RSpec/MultipleMemoizedHelpers:
|
|
27
|
+
Enabled: false
|
|
28
|
+
RSpec/MultipleExpectations:
|
|
29
|
+
Enabled: false
|
|
30
|
+
RSpec/ExampleLength:
|
|
31
|
+
Enabled: false
|
|
32
|
+
RSpec/NestedGroups:
|
|
33
|
+
Enabled: false
|
|
34
|
+
RSpec/MessageSpies:
|
|
35
|
+
Enabled: false
|
|
36
|
+
RSpec/AnyInstance:
|
|
37
|
+
Enabled: false
|
|
38
|
+
RSpec/IndexedLet:
|
|
39
|
+
Enabled: false
|
|
40
|
+
RSpec/ReceiveMessages:
|
|
41
|
+
Enabled: false
|
|
42
|
+
RSpec/VerifiedDoubles:
|
|
43
|
+
Enabled: false
|
|
44
|
+
RSpec/SubjectStub:
|
|
45
|
+
Enabled: false
|
|
46
|
+
RSpec/ContextWording:
|
|
47
|
+
Enabled: false
|
|
48
|
+
RSpec/ExpectOutput:
|
|
49
|
+
Enabled: false
|
|
50
|
+
RSpec/InstanceVariable:
|
|
51
|
+
Enabled: false
|
|
52
|
+
|
|
53
|
+
# Config parses the JSON config into OpenStruct on purpose (dot access to nested settings)
|
|
54
|
+
Style/OpenStructUse:
|
|
55
|
+
Enabled: false
|
data/.rubocop_todo.yml
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# This configuration was generated by
|
|
2
|
+
# `rubocop --auto-gen-config --no-exclude-limit`
|
|
3
|
+
# on 2026-10-31 01:35:19 UTC using RuboCop version 1.92.2.
|
|
4
|
+
# The point is for the user to remove these configuration records
|
|
5
|
+
# one by one as the offenses are removed from the code base.
|
|
6
|
+
# Note that changes in the inspected code, or installation of new
|
|
7
|
+
# versions of RuboCop, may require this file to be generated again.
|
|
8
|
+
|
|
9
|
+
# Offense count: 3
|
|
10
|
+
# This cop supports safe autocorrection (--autocorrect).
|
|
11
|
+
# Configuration parameters: Max, AllowHeredoc, AllowURI, AllowQualifiedName, URISchemes, AllowRBSInlineAnnotation, AllowCopDirectives, AllowedPatterns, SplitStrings.
|
|
12
|
+
# URISchemes: http, https
|
|
13
|
+
Layout/LineLength:
|
|
14
|
+
Exclude:
|
|
15
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
16
|
+
- 'spec/icarus/mod/cli/remove_spec.rb'
|
|
17
|
+
- 'spec/icarus/mod/tools/baseinfo_spec.rb'
|
|
18
|
+
|
|
19
|
+
# Offense count: 24
|
|
20
|
+
# Configuration parameters: AllowedMethods, AllowedPatterns, CountRepeatedAttributes, Max.
|
|
21
|
+
Metrics/AbcSize:
|
|
22
|
+
Exclude:
|
|
23
|
+
- 'lib/icarus/mod/cli/add.rb'
|
|
24
|
+
- 'lib/icarus/mod/cli/command.rb'
|
|
25
|
+
- 'lib/icarus/mod/cli/list.rb'
|
|
26
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
27
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
28
|
+
- 'lib/icarus/mod/cli/validate.rb'
|
|
29
|
+
- 'lib/icarus/mod/firestore.rb'
|
|
30
|
+
- 'lib/icarus/mod/tools/baseinfo.rb'
|
|
31
|
+
- 'lib/icarus/mod/tools/sync/entities.rb'
|
|
32
|
+
- 'lib/icarus/mod/tools/sync/helpers.rb'
|
|
33
|
+
|
|
34
|
+
# Offense count: 2
|
|
35
|
+
# Configuration parameters: CountComments, Max, CountAsOne, AllowedMethods, AllowedPatterns, inherit_mode.
|
|
36
|
+
# AllowedMethods: refine
|
|
37
|
+
Metrics/BlockLength:
|
|
38
|
+
Exclude:
|
|
39
|
+
- 'lib/icarus/mod/cli/list.rb'
|
|
40
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
41
|
+
|
|
42
|
+
# Offense count: 4
|
|
43
|
+
# Configuration parameters: CountComments, Max, CountAsOne.
|
|
44
|
+
Metrics/ClassLength:
|
|
45
|
+
Exclude:
|
|
46
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
47
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
48
|
+
- 'lib/icarus/mod/firestore.rb'
|
|
49
|
+
- 'lib/icarus/mod/tools/baseinfo.rb'
|
|
50
|
+
|
|
51
|
+
# Offense count: 16
|
|
52
|
+
# Configuration parameters: AllowedMethods, AllowedPatterns, Max.
|
|
53
|
+
Metrics/CyclomaticComplexity:
|
|
54
|
+
Exclude:
|
|
55
|
+
- 'lib/icarus/mod/cli/add.rb'
|
|
56
|
+
- 'lib/icarus/mod/cli/list.rb'
|
|
57
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
58
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
59
|
+
- 'lib/icarus/mod/cli/validate.rb'
|
|
60
|
+
- 'lib/icarus/mod/firestore.rb'
|
|
61
|
+
- 'lib/icarus/mod/github.rb'
|
|
62
|
+
- 'lib/icarus/mod/tools/baseinfo.rb'
|
|
63
|
+
- 'lib/icarus/mod/tools/sync/entities.rb'
|
|
64
|
+
- 'lib/icarus/mod/tools/sync/helpers.rb'
|
|
65
|
+
|
|
66
|
+
# Offense count: 25
|
|
67
|
+
# Configuration parameters: CountComments, Max, CountAsOne, AllowedMethods, AllowedPatterns.
|
|
68
|
+
Metrics/MethodLength:
|
|
69
|
+
Exclude:
|
|
70
|
+
- 'lib/icarus/mod/cli/add.rb'
|
|
71
|
+
- 'lib/icarus/mod/cli/command.rb'
|
|
72
|
+
- 'lib/icarus/mod/cli/list.rb'
|
|
73
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
74
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
75
|
+
- 'lib/icarus/mod/cli/validate.rb'
|
|
76
|
+
- 'lib/icarus/mod/firestore.rb'
|
|
77
|
+
- 'lib/icarus/mod/github.rb'
|
|
78
|
+
- 'lib/icarus/mod/tools/sync/entities.rb'
|
|
79
|
+
- 'lib/icarus/mod/tools/sync/helpers.rb'
|
|
80
|
+
- 'lib/icarus/mod/tools/sync/modinfo_list.rb'
|
|
81
|
+
- 'lib/icarus/mod/tools/sync/toolinfo_list.rb'
|
|
82
|
+
|
|
83
|
+
# Offense count: 2
|
|
84
|
+
# Configuration parameters: Max, CountKeywordArgs, MaxOptionalParameters.
|
|
85
|
+
Metrics/ParameterLists:
|
|
86
|
+
Exclude:
|
|
87
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
88
|
+
|
|
89
|
+
# Offense count: 12
|
|
90
|
+
# Configuration parameters: AllowedMethods, AllowedPatterns, Max.
|
|
91
|
+
Metrics/PerceivedComplexity:
|
|
92
|
+
Exclude:
|
|
93
|
+
- 'lib/icarus/mod/cli/add.rb'
|
|
94
|
+
- 'lib/icarus/mod/cli/list.rb'
|
|
95
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
96
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
97
|
+
- 'lib/icarus/mod/cli/validate.rb'
|
|
98
|
+
- 'lib/icarus/mod/github.rb'
|
|
99
|
+
- 'lib/icarus/mod/tools/sync/helpers.rb'
|
|
100
|
+
|
|
101
|
+
# Offense count: 3
|
|
102
|
+
# Configuration parameters: Mode, AllowedMethods, AllowedPatterns, AllowBangMethods, WaywardPredicates.
|
|
103
|
+
# AllowedMethods: call
|
|
104
|
+
# WaywardPredicates: infinite?, nonzero?
|
|
105
|
+
Naming/PredicateMethod:
|
|
106
|
+
Exclude:
|
|
107
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
108
|
+
- 'lib/icarus/mod/firestore.rb'
|
|
109
|
+
|
|
110
|
+
# Offense count: 4
|
|
111
|
+
# This cop supports unsafe autocorrection (--autocorrect-all).
|
|
112
|
+
Style/DirectiveScope:
|
|
113
|
+
Exclude:
|
|
114
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
|
115
|
+
- 'lib/icarus/mod/cli/subcommand_base.rb'
|
|
116
|
+
- 'lib/icarus/mod/cli/sync.rb'
|
|
117
|
+
- 'lib/icarus/mod/tools/toolinfo.rb'
|
|
118
|
+
|
|
119
|
+
# Offense count: 4
|
|
120
|
+
# Configuration parameters: AllowedVariables.
|
|
121
|
+
Style/GlobalVars:
|
|
122
|
+
Exclude:
|
|
123
|
+
- 'spec/icarus/mod/cli/remove_spec.rb'
|
|
124
|
+
- 'spec/icarus/mod/cli/sync_spec.rb'
|
|
125
|
+
|
|
126
|
+
# Offense count: 1
|
|
127
|
+
Style/MultilineBlockChain:
|
|
128
|
+
Exclude:
|
|
129
|
+
- 'lib/icarus/mod/tools/sync/entities.rb'
|
|
130
|
+
|
|
131
|
+
# Offense count: 2
|
|
132
|
+
# This cop supports unsafe autocorrection (--autocorrect-all).
|
|
133
|
+
Style/ReduceToHash:
|
|
134
|
+
Exclude:
|
|
135
|
+
- 'lib/icarus/mod/tools/baseinfo.rb'
|
|
136
|
+
- 'lib/icarus/mod/tools/toolinfo.rb'
|
|
137
|
+
|
|
138
|
+
# Offense count: 2
|
|
139
|
+
# This cop supports unsafe autocorrection (--autocorrect-all).
|
|
140
|
+
Style/SelectByRegexp:
|
|
141
|
+
Exclude:
|
|
142
|
+
- 'lib/icarus/mod/cli/remove.rb'
|
data/.ruby-version
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
4.0
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,47 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
## History (reverse chronological order)
|
|
6
6
|
|
|
7
|
+
### v3.1.0 - 2026-10-30
|
|
8
|
+
|
|
9
|
+
- `imt sync mods`/`tools --dry-run` now lists the entries a real run would delete, and warns when the mass-delete brake would refuse (a real run would exit 1 without `--force`)
|
|
10
|
+
- Document `imt add --dry-run` in the README
|
|
11
|
+
- Update `google-cloud-firestore` to 3.x and `octokit` to 10.x (no API changes affect `imt`)
|
|
12
|
+
- Update development tools (`pry` 0.16, `guard`, `rake`)
|
|
13
|
+
|
|
14
|
+
### v3.0.0 - 2026-10-30
|
|
15
|
+
|
|
16
|
+
**Breaking:** requires Ruby 4.0 or greater. Sync commands now exit 1 on failure, and `imt sync mods`/`tools` refuse a mass delete without `--force`.
|
|
17
|
+
|
|
18
|
+
- Require Ruby 4.0 or greater (was 3.1)
|
|
19
|
+
- Fixes `imt sync` crashing during Firestore authentication with `wrong number of arguments (given 2, expected 1)` on Ruby 3.1 once `json` 3.x is installed; `multi_json` releases that support `json` 3.x (1.21.2+) need Ruby 3.2 or newer
|
|
20
|
+
- Update `multi_json` to 1.21.2
|
|
21
|
+
- Declare `ostruct` as a runtime dependency; it is no longer a default gem on Ruby 4.0, so a plain `gem install` could fail to load the config
|
|
22
|
+
- Update `faraday`, `jwt`, `addressable` and `concurrent-ruby` to clear known security advisories
|
|
23
|
+
- Remove the unused, stale RBS signatures under `sig/`
|
|
24
|
+
- Fix `imt sync all` not creating mods/tools for a `modinfo.json`/`toolinfo.json` found earlier in the same run; the new URL was only picked up on the next run
|
|
25
|
+
- Fix two sync hosts (or a sync and `imt add`/`remove`) running at once overwriting each other's changes to `meta/modinfo`, `meta/toolinfo`, `meta/repos` and the source maps
|
|
26
|
+
- Add a mass-delete brake: `imt sync mods`/`tools` refuses to delete more than 25% of a collection (at least 5 entries), or to empty it, unless run with `--force`
|
|
27
|
+
- Fix `imt sync` commands exiting 0 on failure; they now exit 1, and `imt sync all` runs every stage before exiting 1
|
|
28
|
+
- Fix `imt sync mods`/`tools` always printing "Success" at `-vv`, even when an update or delete failed
|
|
29
|
+
- Fix one `modinfo.json`/`toolinfo.json` whose body is not a JSON object (or whose entries are not objects) aborting the whole sync; it is now treated like invalid JSON and its existing entries are kept
|
|
30
|
+
- Fix a crash on a first run against an empty database (missing `meta` documents)
|
|
31
|
+
- Fix source URLs losing their query string, and a malformed URL crashing the sync instead of being skipped
|
|
32
|
+
- Save recorded sources even when a sync fails partway through, so the next run can still protect those entries
|
|
33
|
+
- Fix `imt remove repos` removing the wrong entries
|
|
34
|
+
- Matches a repo's `modinfo.json`/`toolinfo.json` URLs only on the GitHub host and exact `owner/repo` (case-insensitive); before, a same-named subdirectory in someone else's repo matched
|
|
35
|
+
- Deletes only the mods and tools that `meta/mod_sources`/`meta/tool_sources` attribute to the repo's URLs (falling back to name and author for entries with no recorded source), so a fork publishing the same name and author keeps its entry
|
|
36
|
+
- Cascades for repos stored as full URLs too
|
|
37
|
+
- Keeps the repo entry and exits 1 when any delete or fetch fails, so the removal can be retried
|
|
38
|
+
- Tighten validation: file types, URLs (http/https with a host) and versions are now matched exactly; input the old checks accepted (`notzipx`, `javascript:` URLs, `1.0 garbage`) is a warning for now and will become an error in a future release
|
|
39
|
+
- Fix non-string values (`"version": 1.0`, `"name": 5`, numeric URLs, `files` that is not an object) crashing validation; they are now reported as errors
|
|
40
|
+
- Fix `Toolinfo#validate` not being public, and unknown attribute names on mod/tool entries silently returning nil instead of raising
|
|
41
|
+
- Fix `imt list mods`/`tools` crashing on `--filter` values containing regex characters, on empty or non-text fields, on `--sort files` or fields missing from some entries, and on entries with no update time; filters are now case-insensitive substring matches and errors go to stderr
|
|
42
|
+
- Fix `imt validate` crashing on an empty list, and `imt` crashing on a malformed config file (it now prints an error and exits 1)
|
|
43
|
+
- Fix the GitHub file cache returning a top-level listing for recursive or sub-path lookups, and `find("modinfo.json")` matching names like `my_modinfo.json.bak`
|
|
44
|
+
- CI now also runs RuboCop and `bundler-audit`, and pins its GitHub Actions to commit SHAs; add a `.rubocop.yml` matching the project's style (existing size/complexity offenses are recorded in `.rubocop_todo.yml`)
|
|
45
|
+
- Update `rubocop-rspec` to 3.x (2.x no longer loads with current RuboCop)
|
|
46
|
+
- Fix `imt add`: `mod` validates every entry before writing anything and reports bad JSON or a missing `mods` list; every `add` command exits 1 on failure; add `--dry-run`
|
|
47
|
+
|
|
7
48
|
### v2.7.0 - 2026-10-30
|
|
8
49
|
|
|
9
50
|
- Fix sync deleting a source's mods and tools when its `modinfo.json`/`toolinfo.json` could not be read (#16)
|
data/CLAUDE.md
CHANGED
|
@@ -24,6 +24,7 @@ bundle exec rspec # Run tests only (with Fuubar formatter)
|
|
|
24
24
|
bundle exec rspec spec/path/to/file_spec.rb # Run single test file
|
|
25
25
|
bundle exec rspec spec/path/to/file_spec.rb:42 # Run single test at line 42
|
|
26
26
|
guard # Auto-run tests on file changes
|
|
27
|
+
bundle exec rubocop # Lint (CI runs it; .rubocop_todo.yml holds the existing metrics offenses)
|
|
27
28
|
```
|
|
28
29
|
|
|
29
30
|
### Console & Debugging
|
|
@@ -50,7 +51,7 @@ The tool implements a two-stage sync pipeline:
|
|
|
50
51
|
|
|
51
52
|
All operations support `--dry-run` for safe testing.
|
|
52
53
|
|
|
53
|
-
Deletion contract (#16): stage 2 deletes a doc only when no info file still lists it. `Sync::Helpers#retrieve_from_url` raises `RequestGone` for 404/410 (author removed the file: delete) and `RequestTransient` for everything else; invalid JSON also counts as transient. Docs do not record their source URL; instead `meta/mod_sources` / `meta/tool_sources` (Firestore, shared across sync hosts) map doc ID => source URL, written by `Sync::Mods/Tools#update` + `save_sources` and pruned by `Firestore#delete`. A transient failure protects only the docs mapped to that URL; unmapped docs are kept while any URL failed. Stage 1 never drops list entries (`Firestore#update` unions)
|
|
54
|
+
Deletion contract (#16): stage 2 deletes a doc only when no info file still lists it. `Sync::Helpers#retrieve_from_url` raises `RequestGone` for 404/410 (author removed the file: delete) and `RequestTransient` for everything else; invalid JSON also counts as transient. Docs do not record their source URL; instead `meta/mod_sources` / `meta/tool_sources` (Firestore, shared across sync hosts) map doc ID => source URL, written by `Sync::Mods/Tools#update` + `save_sources` and pruned by `Firestore#delete`. A transient failure protects only the docs mapped to that URL; unmapped docs are kept while any URL failed. Stage 1 never drops list entries (`Firestore#update` unions server-side with `FieldValue.array_union`; source maps are written per key with `merge: true`, so concurrent hosts don't overwrite each other). Stage 2 refuses to delete more than 25% of a collection (min 5) or empty it without `--force`, and a repo whose GitHub search fails transiently is skipped. New docs get a stable ID (`Firestore#stable_id`); `imt sync cleanup` keeps the oldest duplicate and copies the newest data onto it.
|
|
54
55
|
|
|
55
56
|
### Core Components
|
|
56
57
|
|
|
@@ -72,8 +73,7 @@ Deletion contract (#16): stage 2 deletes a doc only when no info file still list
|
|
|
72
73
|
|
|
73
74
|
- `ModinfoList`: GitHub → meta/modinfo sync
|
|
74
75
|
- `ToolinfoList`: GitHub → meta/toolinfo sync
|
|
75
|
-
- `
|
|
76
|
-
- `Tools`: meta/toolinfo → tools sync
|
|
76
|
+
- `Entities`: shared meta → entity sync; `Mods` (meta/modinfo → mods) and `Tools` (meta/toolinfo → tools) only set `TYPE`/`INFO`/`ENTITY`
|
|
77
77
|
- `Helpers`: HTTP utilities with custom SSL verification for Ruby 3.4+ CRL issues
|
|
78
78
|
|
|
79
79
|
**External Services** (`lib/icarus/mod/`)
|
|
@@ -148,7 +148,7 @@ Ruby 3.4+ with newer OpenSSL may fail CRL checking. The codebase handles this in
|
|
|
148
148
|
|
|
149
149
|
- **Firestore credentials**: Must be the full JSON object in config, not a file path
|
|
150
150
|
- **GitHub repo format**: `owner/repo` format (URL prefixes are stripped automatically)
|
|
151
|
-
- **Ruby version**:
|
|
151
|
+
- **Ruby version**: 4.0+ required
|
|
152
152
|
- **Platform**: WSL2/Linux recommended; not tested on Windows
|
|
153
153
|
- **Require paths**: The gemspec sets `require_paths` to both `lib` and `lib/icarus/mod`, so files use relative requires like `require "cli/base"` instead of `require_relative`
|
|
154
154
|
- **Verbosity**: `-v` and `-vv` flags control output detail in subcommands
|
data/Gemfile
CHANGED
|
@@ -7,14 +7,12 @@ gemspec
|
|
|
7
7
|
|
|
8
8
|
gem "rake", "~> 13.0"
|
|
9
9
|
|
|
10
|
-
group :
|
|
10
|
+
group :development do
|
|
11
11
|
gem "fuubar"
|
|
12
12
|
gem "guard", "~> 2.18"
|
|
13
13
|
gem "guard-rspec", "~> 4.7"
|
|
14
|
-
gem "pry", "~> 0.
|
|
14
|
+
gem "pry", "~> 0.16"
|
|
15
15
|
gem "rspec", "~> 3.12"
|
|
16
|
-
gem "rubocop", "~> 1.
|
|
17
|
-
gem "rubocop-rspec", "~>
|
|
16
|
+
gem "rubocop", "~> 1.92"
|
|
17
|
+
gem "rubocop-rspec", "~> 3.10", require: false
|
|
18
18
|
end
|
|
19
|
-
|
|
20
|
-
gem "ostruct", "~> 0.6.3"
|
data/Gemfile.lock
CHANGED
|
@@ -1,28 +1,29 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
Icarus-Mod-Tools (
|
|
5
|
-
google-cloud-firestore (~>
|
|
6
|
-
octokit (~>
|
|
4
|
+
Icarus-Mod-Tools (3.1.0)
|
|
5
|
+
google-cloud-firestore (~> 3.3)
|
|
6
|
+
octokit (~> 10.0)
|
|
7
|
+
ostruct (~> 0.6)
|
|
7
8
|
paint (~> 2.3)
|
|
8
9
|
thor (~> 1.2)
|
|
9
10
|
|
|
10
11
|
GEM
|
|
11
12
|
remote: https://rubygems.org/
|
|
12
13
|
specs:
|
|
13
|
-
addressable (2.
|
|
14
|
+
addressable (2.9.1)
|
|
14
15
|
public_suffix (>= 2.0.2, < 8.0)
|
|
15
16
|
ast (2.4.3)
|
|
16
17
|
base64 (0.3.0)
|
|
17
18
|
bigdecimal (3.3.1)
|
|
18
19
|
coderay (1.1.3)
|
|
19
|
-
concurrent-ruby (1.3.
|
|
20
|
+
concurrent-ruby (1.3.8)
|
|
20
21
|
diff-lcs (1.6.2)
|
|
21
|
-
faraday (2.
|
|
22
|
-
faraday-net_http (>= 2.0, < 3.
|
|
22
|
+
faraday (2.15.1)
|
|
23
|
+
faraday-net_http (>= 2.0, < 3.6)
|
|
23
24
|
json
|
|
24
25
|
logger
|
|
25
|
-
faraday-net_http (3.
|
|
26
|
+
faraday-net_http (3.5.0)
|
|
26
27
|
net-http (~> 0.5)
|
|
27
28
|
faraday-retry (2.4.0)
|
|
28
29
|
faraday (~> 2.0)
|
|
@@ -42,7 +43,7 @@ GEM
|
|
|
42
43
|
fuubar (2.5.1)
|
|
43
44
|
rspec-core (~> 3.0)
|
|
44
45
|
ruby-progressbar (~> 1.4)
|
|
45
|
-
gapic-common (1.
|
|
46
|
+
gapic-common (1.6.1)
|
|
46
47
|
faraday (>= 1.9, < 3.a)
|
|
47
48
|
faraday-retry (>= 1.0, < 3.a)
|
|
48
49
|
google-cloud-env (~> 2.2)
|
|
@@ -51,73 +52,97 @@ GEM
|
|
|
51
52
|
googleapis-common-protos (~> 1.6)
|
|
52
53
|
googleapis-common-protos-types (~> 1.15)
|
|
53
54
|
googleauth (~> 1.12)
|
|
54
|
-
grpc (
|
|
55
|
-
google-cloud-core (1.
|
|
55
|
+
grpc (>= 1.84.1, < 2.a)
|
|
56
|
+
google-cloud-core (1.9.1)
|
|
56
57
|
google-cloud-env (>= 1.0, < 3.a)
|
|
57
58
|
google-cloud-errors (~> 1.0)
|
|
58
|
-
google-cloud-env (2.
|
|
59
|
+
google-cloud-env (2.4.1)
|
|
59
60
|
base64 (~> 0.2)
|
|
60
61
|
faraday (>= 1.0, < 3.a)
|
|
61
|
-
google-cloud-errors (1.
|
|
62
|
-
google-cloud-firestore (
|
|
62
|
+
google-cloud-errors (1.7.1)
|
|
63
|
+
google-cloud-firestore (3.3.0)
|
|
63
64
|
bigdecimal (~> 3.0)
|
|
64
65
|
concurrent-ruby (~> 1.0)
|
|
65
|
-
google-cloud-core (~> 1.
|
|
66
|
-
google-cloud-firestore-v1 (
|
|
66
|
+
google-cloud-core (~> 1.7)
|
|
67
|
+
google-cloud-firestore-v1 (~> 2.0)
|
|
67
68
|
rbtree (~> 0.4.2)
|
|
68
|
-
google-cloud-firestore-v1 (
|
|
69
|
-
gapic-common (
|
|
69
|
+
google-cloud-firestore-v1 (2.6.0)
|
|
70
|
+
gapic-common (~> 1.6)
|
|
70
71
|
google-cloud-errors (~> 1.0)
|
|
71
|
-
google-cloud-location (
|
|
72
|
-
google-cloud-location (1.
|
|
73
|
-
gapic-common (~> 1.
|
|
72
|
+
google-cloud-location (~> 1.0)
|
|
73
|
+
google-cloud-location (1.6.0)
|
|
74
|
+
gapic-common (~> 1.6)
|
|
74
75
|
google-cloud-errors (~> 1.0)
|
|
75
|
-
google-logging-utils (0.2.
|
|
76
|
-
google-protobuf (4.
|
|
76
|
+
google-logging-utils (0.2.2)
|
|
77
|
+
google-protobuf (4.36.2)
|
|
77
78
|
bigdecimal
|
|
78
|
-
rake (
|
|
79
|
-
|
|
79
|
+
rake (~> 13.3)
|
|
80
|
+
google-protobuf (4.36.2-aarch64-linux-gnu)
|
|
81
|
+
bigdecimal
|
|
82
|
+
rake (~> 13.3)
|
|
83
|
+
google-protobuf (4.36.2-aarch64-linux-musl)
|
|
84
|
+
bigdecimal
|
|
85
|
+
rake (~> 13.3)
|
|
86
|
+
google-protobuf (4.36.2-arm64-darwin)
|
|
87
|
+
bigdecimal
|
|
88
|
+
rake (~> 13.3)
|
|
89
|
+
google-protobuf (4.36.2-x86-linux-gnu)
|
|
90
|
+
bigdecimal
|
|
91
|
+
rake (~> 13.3)
|
|
92
|
+
google-protobuf (4.36.2-x86-linux-musl)
|
|
93
|
+
bigdecimal
|
|
94
|
+
rake (~> 13.3)
|
|
95
|
+
google-protobuf (4.36.2-x86_64-darwin)
|
|
96
|
+
bigdecimal
|
|
97
|
+
rake (~> 13.3)
|
|
98
|
+
google-protobuf (4.36.2-x86_64-linux-gnu)
|
|
99
|
+
bigdecimal
|
|
100
|
+
rake (~> 13.3)
|
|
101
|
+
google-protobuf (4.36.2-x86_64-linux-musl)
|
|
102
|
+
bigdecimal
|
|
103
|
+
rake (~> 13.3)
|
|
104
|
+
googleapis-common-protos (1.11.0)
|
|
80
105
|
google-protobuf (~> 4.26)
|
|
81
106
|
googleapis-common-protos-types (~> 1.21)
|
|
82
|
-
grpc (
|
|
83
|
-
googleapis-common-protos-types (1.
|
|
107
|
+
grpc (>= 1.84.1, < 2.a)
|
|
108
|
+
googleapis-common-protos-types (1.24.0)
|
|
84
109
|
google-protobuf (~> 4.26)
|
|
85
|
-
googleauth (1.
|
|
110
|
+
googleauth (1.17.7)
|
|
86
111
|
faraday (>= 1.0, < 3.a)
|
|
87
112
|
google-cloud-env (~> 2.2)
|
|
88
113
|
google-logging-utils (~> 0.1)
|
|
89
114
|
jwt (>= 1.4, < 4.0)
|
|
90
|
-
multi_json (~> 1.11)
|
|
91
115
|
os (>= 0.9, < 2.0)
|
|
116
|
+
pstore (~> 0.1)
|
|
92
117
|
signet (>= 0.16, < 2.a)
|
|
93
|
-
grpc (1.
|
|
118
|
+
grpc (1.85.0)
|
|
94
119
|
google-protobuf (>= 3.25, < 5.0)
|
|
95
120
|
googleapis-common-protos-types (~> 1.0)
|
|
96
|
-
grpc (1.
|
|
121
|
+
grpc (1.85.0-aarch64-linux-gnu)
|
|
97
122
|
google-protobuf (>= 3.25, < 5.0)
|
|
98
123
|
googleapis-common-protos-types (~> 1.0)
|
|
99
|
-
grpc (1.
|
|
124
|
+
grpc (1.85.0-aarch64-linux-musl)
|
|
100
125
|
google-protobuf (>= 3.25, < 5.0)
|
|
101
126
|
googleapis-common-protos-types (~> 1.0)
|
|
102
|
-
grpc (1.
|
|
127
|
+
grpc (1.85.0-arm64-darwin)
|
|
103
128
|
google-protobuf (>= 3.25, < 5.0)
|
|
104
129
|
googleapis-common-protos-types (~> 1.0)
|
|
105
|
-
grpc (1.
|
|
130
|
+
grpc (1.85.0-x86-linux-gnu)
|
|
106
131
|
google-protobuf (>= 3.25, < 5.0)
|
|
107
132
|
googleapis-common-protos-types (~> 1.0)
|
|
108
|
-
grpc (1.
|
|
133
|
+
grpc (1.85.0-x86-linux-musl)
|
|
109
134
|
google-protobuf (>= 3.25, < 5.0)
|
|
110
135
|
googleapis-common-protos-types (~> 1.0)
|
|
111
|
-
grpc (1.
|
|
136
|
+
grpc (1.85.0-x86_64-darwin)
|
|
112
137
|
google-protobuf (>= 3.25, < 5.0)
|
|
113
138
|
googleapis-common-protos-types (~> 1.0)
|
|
114
|
-
grpc (1.
|
|
139
|
+
grpc (1.85.0-x86_64-linux-gnu)
|
|
115
140
|
google-protobuf (>= 3.25, < 5.0)
|
|
116
141
|
googleapis-common-protos-types (~> 1.0)
|
|
117
|
-
grpc (1.
|
|
142
|
+
grpc (1.85.0-x86_64-linux-musl)
|
|
118
143
|
google-protobuf (>= 3.25, < 5.0)
|
|
119
144
|
googleapis-common-protos-types (~> 1.0)
|
|
120
|
-
guard (2.20.
|
|
145
|
+
guard (2.20.2)
|
|
121
146
|
formatador (>= 0.2.4)
|
|
122
147
|
listen (>= 2.7, < 4.0)
|
|
123
148
|
logger (~> 1.6)
|
|
@@ -133,8 +158,8 @@ GEM
|
|
|
133
158
|
guard-compat (~> 1.1)
|
|
134
159
|
rspec (>= 2.99.0, < 4.0)
|
|
135
160
|
io-console (0.8.2)
|
|
136
|
-
json (
|
|
137
|
-
jwt (3.
|
|
161
|
+
json (3.1.0)
|
|
162
|
+
jwt (3.3.0)
|
|
138
163
|
base64
|
|
139
164
|
language_server-protocol (3.17.0.5)
|
|
140
165
|
lint_roller (1.1.0)
|
|
@@ -145,14 +170,16 @@ GEM
|
|
|
145
170
|
logger (1.7.0)
|
|
146
171
|
lumberjack (1.4.2)
|
|
147
172
|
method_source (1.1.0)
|
|
148
|
-
multi_json (1.19.1)
|
|
149
173
|
nenv (0.3.0)
|
|
150
|
-
net-http (0.
|
|
174
|
+
net-http (0.10.0)
|
|
175
|
+
net-protocol (>= 0.3.0)
|
|
151
176
|
uri (>= 0.11.1)
|
|
177
|
+
net-protocol (0.4.0)
|
|
178
|
+
timeout
|
|
152
179
|
notiffany (0.1.3)
|
|
153
180
|
nenv (~> 0.1)
|
|
154
181
|
shellany (~> 0.0)
|
|
155
|
-
octokit (
|
|
182
|
+
octokit (10.0.0)
|
|
156
183
|
faraday (>= 1, < 3)
|
|
157
184
|
sawyer (~> 0.9)
|
|
158
185
|
os (1.1.4)
|
|
@@ -163,17 +190,19 @@ GEM
|
|
|
163
190
|
ast (~> 2.4.1)
|
|
164
191
|
racc
|
|
165
192
|
prism (1.9.0)
|
|
166
|
-
pry (0.
|
|
193
|
+
pry (0.16.1)
|
|
167
194
|
coderay (~> 1.1)
|
|
168
195
|
method_source (~> 1.0)
|
|
169
|
-
|
|
196
|
+
reline (>= 0.6.0)
|
|
197
|
+
pstore (0.3.0)
|
|
198
|
+
public_suffix (7.0.5)
|
|
170
199
|
racc (1.8.1)
|
|
171
200
|
rainbow (3.1.1)
|
|
172
|
-
rake (13.
|
|
201
|
+
rake (13.4.2)
|
|
173
202
|
rb-fsevent (0.11.2)
|
|
174
203
|
rb-inotify (0.11.1)
|
|
175
204
|
ffi (~> 1.0)
|
|
176
|
-
rbtree (0.4.
|
|
205
|
+
rbtree (0.4.7)
|
|
177
206
|
regexp_parser (2.11.3)
|
|
178
207
|
reline (0.6.3)
|
|
179
208
|
io-console (~> 0.5)
|
|
@@ -190,48 +219,39 @@ GEM
|
|
|
190
219
|
diff-lcs (>= 1.2.0, < 2.0)
|
|
191
220
|
rspec-support (~> 3.13.0)
|
|
192
221
|
rspec-support (3.13.7)
|
|
193
|
-
rubocop (1.
|
|
194
|
-
json (
|
|
222
|
+
rubocop (1.92.2)
|
|
223
|
+
json (>= 2.3)
|
|
195
224
|
language_server-protocol (~> 3.17.0.2)
|
|
196
225
|
lint_roller (~> 1.1.0)
|
|
197
|
-
parallel (
|
|
226
|
+
parallel (>= 1.10)
|
|
198
227
|
parser (>= 3.3.0.2)
|
|
199
228
|
rainbow (>= 2.2.2, < 4.0)
|
|
200
|
-
regexp_parser (>= 2.
|
|
229
|
+
regexp_parser (>= 2.11.3, < 3.0)
|
|
201
230
|
rubocop-ast (>= 1.49.0, < 2.0)
|
|
202
231
|
ruby-progressbar (~> 1.7)
|
|
203
232
|
unicode-display_width (>= 2.4.0, < 4.0)
|
|
204
233
|
rubocop-ast (1.49.0)
|
|
205
234
|
parser (>= 3.3.7.2)
|
|
206
235
|
prism (~> 1.7)
|
|
207
|
-
rubocop-
|
|
208
|
-
lint_roller (~> 1.1)
|
|
209
|
-
rubocop (~> 1.72, >= 1.72.1)
|
|
210
|
-
rubocop-factory_bot (2.28.0)
|
|
236
|
+
rubocop-rspec (3.10.2)
|
|
211
237
|
lint_roller (~> 1.1)
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
rubocop (~> 1.40)
|
|
215
|
-
rubocop-capybara (~> 2.17)
|
|
216
|
-
rubocop-factory_bot (~> 2.22)
|
|
217
|
-
rubocop-rspec_rails (~> 2.28)
|
|
218
|
-
rubocop-rspec_rails (2.29.1)
|
|
219
|
-
rubocop (~> 1.61)
|
|
238
|
+
regexp_parser (>= 2.0)
|
|
239
|
+
rubocop (~> 1.86, >= 1.86.2)
|
|
220
240
|
ruby-progressbar (1.13.0)
|
|
221
241
|
sawyer (0.9.3)
|
|
222
242
|
addressable (>= 2.3.5)
|
|
223
243
|
faraday (>= 0.17.3, < 3)
|
|
224
244
|
shellany (0.0.1)
|
|
225
|
-
signet (0.
|
|
245
|
+
signet (0.22.2)
|
|
226
246
|
addressable (~> 2.8)
|
|
227
247
|
faraday (>= 0.17.5, < 3.a)
|
|
228
248
|
jwt (>= 1.5, < 4.0)
|
|
229
|
-
multi_json (~> 1.10)
|
|
230
249
|
thor (1.5.0)
|
|
250
|
+
timeout (0.6.1)
|
|
231
251
|
unicode-display_width (3.2.0)
|
|
232
252
|
unicode-emoji (~> 4.1)
|
|
233
253
|
unicode-emoji (4.2.0)
|
|
234
|
-
uri (1.1.
|
|
254
|
+
uri (1.1.2)
|
|
235
255
|
|
|
236
256
|
PLATFORMS
|
|
237
257
|
aarch64-linux-gnu
|
|
@@ -252,12 +272,11 @@ DEPENDENCIES
|
|
|
252
272
|
fuubar
|
|
253
273
|
guard (~> 2.18)
|
|
254
274
|
guard-rspec (~> 4.7)
|
|
255
|
-
|
|
256
|
-
pry (~> 0.14.1)
|
|
275
|
+
pry (~> 0.16)
|
|
257
276
|
rake (~> 13.0)
|
|
258
277
|
rspec (~> 3.12)
|
|
259
|
-
rubocop (~> 1.
|
|
260
|
-
rubocop-rspec (~>
|
|
278
|
+
rubocop (~> 1.92)
|
|
279
|
+
rubocop-rspec (~> 3.10)
|
|
261
280
|
|
|
262
281
|
BUNDLED WITH
|
|
263
282
|
4.0.5
|
data/Guardfile
CHANGED
data/README.md
CHANGED
|
@@ -8,7 +8,7 @@ To use this app, you'll need to obtain the following:
|
|
|
8
8
|
|
|
9
9
|
- A Github ACCESS_TOKEN (doesn't need access to any repos, this is used purely to make API calls)
|
|
10
10
|
- A Google Cloud Platform credentials `keyfile.json`
|
|
11
|
-
- Ruby
|
|
11
|
+
- Ruby 4.0 (or greater)
|
|
12
12
|
|
|
13
13
|
If you aren't sure how to obtain these credentials, please see:
|
|
14
14
|
|
|
@@ -84,8 +84,11 @@ Options:
|
|
|
84
84
|
-V, [--version], [--no-version] # Print the version and exit
|
|
85
85
|
-v, [--verbose], [--no-verbose] # Increase verbosity. May be repeated for even more verbosity.
|
|
86
86
|
# Default: [true]
|
|
87
|
+
[--dry-run], [--no-dry-run] # Validate and show what would be added without writing
|
|
87
88
|
```
|
|
88
89
|
|
|
90
|
+
`imt add mod` validates every entry in the file before writing any of them, and each `add` command exits 1 on failure.
|
|
91
|
+
|
|
89
92
|
#### `imt list`
|
|
90
93
|
|
|
91
94
|
```sh
|
|
@@ -146,9 +149,12 @@ Options:
|
|
|
146
149
|
-v, [--verbose], [--no-verbose] # Increase verbosity. May be repeated for even more verbosity.
|
|
147
150
|
# Default: [true]
|
|
148
151
|
[--dry-run], [--no-dry-run] # Dry run (no changes will be made)
|
|
152
|
+
[--force] # Let mods/tools sync delete over 25% of a collection
|
|
149
153
|
```
|
|
150
154
|
|
|
151
|
-
**Removal**: Mod and tool authors control their own entries. If a `modinfo.json` or `toolinfo.json` is removed or renamed (so it returns HTTP 404 or 410), `imt sync mods`/`tools` deletes its entries. If a file cannot be read for any other reason (timeout, network error, server error, rate limit, or invalid JSON), that file's entries are kept on that run and a warning names them; other sources are unaffected. Sync tracks which file each entry came from in the internal `meta/mod_sources` and `meta/tool_sources` documents (override with `collections.meta.mod_sources` / `tool_sources` in the config). An entry with no recorded source yet is kept while any source fails.
|
|
155
|
+
**Removal**: Mod and tool authors control their own entries. If a `modinfo.json` or `toolinfo.json` is removed or renamed (so it returns HTTP 404 or 410), `imt sync mods`/`tools` deletes its entries. If a file cannot be read for any other reason (timeout, network error, server error, rate limit, or invalid JSON), that file's entries are kept on that run and a warning names them; other sources are unaffected. Sync tracks which file each entry came from in the internal `meta/mod_sources` and `meta/tool_sources` documents (override with `collections.meta.mod_sources` / `tool_sources` in the config). An entry with no recorded source yet is kept while any source fails. As a brake on a wave of 404s (a GitHub outage, a mass rename), `imt sync mods`/`tools` refuses to delete more than 25% of a collection (at least 5 entries), or to empty it, and exits 1; rerun with `--force` once you have checked the sources. `--dry-run` lists what a real run would delete and warns if the brake would stop it.
|
|
156
|
+
|
|
157
|
+
**Exit status**: each `imt sync` command exits 1 when it fails. `imt sync all` runs every stage even if one fails, then exits 1.
|
|
152
158
|
|
|
153
159
|
**IDs**: New mods and tools get a document ID derived from their type, author and name, so an entry that is deleted and later re-added gets the same ID. `imt sync cleanup` keeps the oldest of any duplicates (the ID clients already have), copies the newest duplicate's data onto it, and deletes the rest.
|
|
154
160
|
|