automatic 26.08 → 26.09
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +80 -46
- data/VERSION +1 -1
- data/assets/siteinfo/items_all.json +60300 -52138
- data/automatic.gemspec +4 -3
- data/bin/automatic +1 -1
- data/doc/AI_TUTORIAL.md +64 -40
- data/doc/BASIC_DESIGN.md +31 -0
- data/doc/DEPLOYMENT.md +53 -47
- data/doc/PLUGINS.md +231 -91
- data/doc/POLICY.md +149 -44
- data/doc/QUICKSTART.md +19 -15
- data/doc/RELEASING.md +20 -7
- data/doc/REQUIREMENTS.md +40 -10
- data/doc/VERSIONS +112 -54
- data/lib/automatic/cli.rb +40 -18
- data/lib/automatic/environment.rb +1 -1
- data/lib/automatic/feed_maker.rb +57 -18
- data/lib/automatic/feed_parser.rb +1 -1
- data/lib/automatic/http.rb +1 -1
- data/lib/automatic/log.rb +1 -1
- data/lib/automatic/pipeline.rb +15 -5
- data/lib/automatic/recipe.rb +45 -1
- data/lib/automatic/version.rb +3 -3
- data/lib/automatic.rb +18 -3
- data/plugins/custom_feed/web.rb +1 -1
- data/plugins/filter/absolute_uri.rb +1 -1
- data/plugins/filter/batch.rb +97 -0
- data/plugins/filter/claude.rb +1 -1
- data/plugins/filter/clear.rb +1 -1
- data/plugins/filter/description_link.rb +20 -3
- data/plugins/filter/full_feed.rb +28 -16
- data/plugins/filter/gemini.rb +1 -1
- data/plugins/filter/ignore.rb +1 -1
- data/plugins/filter/image.rb +1 -1
- data/plugins/filter/image_source.rb +20 -3
- data/plugins/filter/join.rb +4 -6
- data/plugins/filter/kimi.rb +216 -0
- data/plugins/filter/limit.rb +57 -0
- data/plugins/filter/open_ai.rb +1 -1
- data/plugins/filter/present.rb +75 -0
- data/plugins/filter/sakura_ai.rb +1 -1
- data/plugins/filter/sanitize.rb +1 -1
- data/plugins/filter/sort.rb +1 -1
- data/plugins/filter/tumblr_resize.rb +1 -1
- data/plugins/notify/ikachan.rb +1 -1
- data/plugins/provide/fluentd.rb +1 -1
- data/plugins/publish/amazon_s3.rb +9 -3
- data/plugins/publish/console.rb +1 -1
- data/plugins/publish/fluentd.rb +1 -1
- data/plugins/publish/hatena_bookmark.rb +1 -1
- data/plugins/publish/markdown.rb +1 -1
- data/plugins/publish/memcached.rb +1 -1
- data/plugins/store/digest.rb +1 -1
- data/plugins/store/file.rb +11 -3
- data/plugins/store/full_text.rb +9 -13
- data/plugins/store/permalink.rb +1 -1
- data/plugins/subscription/feed.rb +1 -1
- data/plugins/subscription/link.rb +1 -1
- data/plugins/subscription/text.rb +13 -4
- data/plugins/subscription/tumblr.rb +1 -1
- data/plugins/subscription/xml.rb +1 -1
- metadata +6 -2
data/automatic.gemspec
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# -*- coding: utf-8 -*-
|
|
2
2
|
# Name:: automatic.gemspec
|
|
3
|
-
# Author: id774 (More info:
|
|
3
|
+
# Author: id774 (More info: https://id774.net)
|
|
4
4
|
# Source Code:: https://github.com/id774/automaticruby
|
|
5
5
|
# License:: The GPL version 3, or LGPL version 3 (Dual License).
|
|
6
6
|
# Contact:: idnanashi@gmail.com
|
|
@@ -87,8 +87,9 @@ Gem::Specification.new do |spec|
|
|
|
87
87
|
spec.extra_rdoc_files = ['README.md', 'doc/LICENSE.md']
|
|
88
88
|
|
|
89
89
|
# Runtime dependencies: what the framework in lib/ requires, and nothing
|
|
90
|
-
# else. Requiring `automatic`, loading a Recipe, loading a plugin,
|
|
91
|
-
# pipeline and the CLI's own work are what these
|
|
90
|
+
# else. Requiring `automatic`, loading a Recipe, loading a plugin,
|
|
91
|
+
# running a pipeline and the CLI's own work are what these dependencies are
|
|
92
|
+
# for.
|
|
92
93
|
#
|
|
93
94
|
# A gem needed by a plugin is NOT declared here, however useful that plugin
|
|
94
95
|
# is. It is required inside the plugin's own file and installed by the
|
data/bin/automatic
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env ruby
|
|
2
2
|
# -*- coding: utf-8 -*-
|
|
3
3
|
# Name:: automatic
|
|
4
|
-
# Author: id774 (More info:
|
|
4
|
+
# Author: id774 (More info: https://id774.net)
|
|
5
5
|
# Source Code:: https://github.com/id774/automaticruby
|
|
6
6
|
# License:: The GPL version 3, or LGPL version 3 (Dual License).
|
|
7
7
|
# Contact:: idnanashi@gmail.com
|
data/doc/AI_TUTORIAL.md
CHANGED
|
@@ -41,17 +41,21 @@ FilterSakuraAI ask one question about that text
|
|
|
41
41
|
PublishMarkdown write the answer to a document
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
|
|
44
|
+
Each plugin does one thing and hands its result to the next. The
|
|
45
45
|
Recipe that expresses it is in section 4, and nothing in it is a special case:
|
|
46
46
|
every entry is a plugin the framework loads by name, and the order of the list
|
|
47
47
|
is the order of the work.
|
|
48
48
|
|
|
49
49
|
The store plugin's position is not an aesthetic choice. `FilterJoin` produces
|
|
50
50
|
one item with **no link** — it is several articles at once, so there is no page
|
|
51
|
-
it points at
|
|
52
|
-
without
|
|
53
|
-
|
|
54
|
-
|
|
51
|
+
it points at. `StorePermalink` and `StoreFullText` use the shared link-based
|
|
52
|
+
store path and drop an item without a link, so both belong **before**
|
|
53
|
+
`FilterJoin`. `StoreDigest` is different: it identifies content from the
|
|
54
|
+
configured fields and can store an item whose link is nil. In this tutorial it
|
|
55
|
+
still belongs before `FilterJoin`, because the job is to record each source
|
|
56
|
+
article before they are joined and before later work is repeated. Putting
|
|
57
|
+
`StoreDigest` after `FilterJoin` changes its meaning to de-duplicating the whole
|
|
58
|
+
joined digest rather than the individual articles.
|
|
55
59
|
|
|
56
60
|
## 2. Build it without AI first
|
|
57
61
|
|
|
@@ -133,7 +137,7 @@ gem install activerecord sqlite3
|
|
|
133
137
|
gem install sanitize
|
|
134
138
|
```
|
|
135
139
|
|
|
136
|
-
In a checkout the
|
|
140
|
+
In a checkout, select the groups required by those plugins together:
|
|
137
141
|
|
|
138
142
|
```sh
|
|
139
143
|
bundle config set --local with "html store sanitize"
|
|
@@ -153,22 +157,28 @@ link, its date and whatever text the pipeline could get. No AI service has been
|
|
|
153
157
|
contacted, no credential exists yet, and the Recipe is complete in itself: a
|
|
154
158
|
person who only ever wanted this can stop here and put it in `cron`.
|
|
155
159
|
|
|
156
|
-
**
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
+
**Read the `FilterFullFeed` log rather than assuming bundled siteinfo
|
|
161
|
+
coverage.** The shipped siteinfo file is a snapshot. If a record matches an
|
|
162
|
+
article URL, `FilterFullFeed` fetches the page and applies that record's
|
|
163
|
+
XPath. If no record matches, the existing item is preserved and the log says:
|
|
160
164
|
|
|
161
165
|
```text
|
|
162
|
-
Fulltext SITEINFO not found: https://
|
|
166
|
+
Fulltext SITEINFO not found: https://example.com/article
|
|
163
167
|
```
|
|
164
168
|
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
index page read without a `description_selector` carries no description for
|
|
169
|
-
`FilterFullFeed` to have improved on.
|
|
169
|
+
A Recipe that relies on full article bodies should supply siteinfo for the
|
|
170
|
+
sites it relies on under `~/.automatic/assets/siteinfo/`, rather than treating
|
|
171
|
+
the bundled snapshot as a guarantee of coverage.
|
|
170
172
|
|
|
171
|
-
|
|
173
|
+
Where no record matches, the plugin leaves the item exactly as it arrived and
|
|
174
|
+
the run continues. If the index page also provides no description, that item
|
|
175
|
+
therefore has metadata but no article body. If a bundled or user-supplied
|
|
176
|
+
siteinfo record does match, `FilterFullFeed` may replace the description with
|
|
177
|
+
the selected article body. The tutorial relies on that conditional behavior,
|
|
178
|
+
not on either outcome being guaranteed for a particular site.
|
|
179
|
+
|
|
180
|
+
Text can enter these items from the index page or from the article page,
|
|
181
|
+
and both are
|
|
172
182
|
configuration rather than code:
|
|
173
183
|
|
|
174
184
|
- **From the index page.** Where the listing prints a summary, an
|
|
@@ -241,7 +251,7 @@ Now the AI filter goes between `FilterJoin` and `PublishMarkdown`:
|
|
|
241
251
|
- module: FilterSakuraAI
|
|
242
252
|
config:
|
|
243
253
|
token: YOUR_SAKURA_AI_TOKEN
|
|
244
|
-
model:
|
|
254
|
+
model: <Sakura AI Engine model>
|
|
245
255
|
prompt: |
|
|
246
256
|
以下の記事群について、個別記事の要約を羅列するのではなく、
|
|
247
257
|
全体を一つのダイジェストとして日本語で要約してください。
|
|
@@ -291,7 +301,7 @@ plugins:
|
|
|
291
301
|
- module: FilterSakuraAI
|
|
292
302
|
config:
|
|
293
303
|
token: YOUR_SAKURA_AI_TOKEN
|
|
294
|
-
model:
|
|
304
|
+
model: <Sakura AI Engine model>
|
|
295
305
|
prompt: |
|
|
296
306
|
以下の記事群について、個別記事の要約を羅列するのではなく、
|
|
297
307
|
全体を一つのダイジェストとして日本語で要約してください。
|
|
@@ -341,7 +351,7 @@ Move the AI filter to before `FilterJoin` and change nothing else:
|
|
|
341
351
|
- module: FilterSakuraAI
|
|
342
352
|
config:
|
|
343
353
|
token: YOUR_SAKURA_AI_TOKEN
|
|
344
|
-
model:
|
|
354
|
+
model: <Sakura AI Engine model>
|
|
345
355
|
prompt: |
|
|
346
356
|
以下の記事を日本語で三行に要約してください。
|
|
347
357
|
retry: 2
|
|
@@ -374,16 +384,16 @@ and that no plugin had to be changed or configured to express it.
|
|
|
374
384
|
|
|
375
385
|
## 6. Change the service by changing one line
|
|
376
386
|
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
387
|
+
Automatic Ruby uses separate service-specific AI filters rather than one
|
|
388
|
+
plugin with a `provider` setting. Their current catalogue and settings are
|
|
389
|
+
maintained in [`PLUGINS.md`](PLUGINS.md) section 6.3. Each replaces an item's description with what its service answers, so
|
|
380
390
|
in a Recipe they are interchangeable at the same position:
|
|
381
391
|
|
|
382
392
|
```yaml
|
|
383
393
|
- module: FilterOpenAI
|
|
384
394
|
config:
|
|
385
395
|
token: YOUR_OPENAI_API_KEY
|
|
386
|
-
model:
|
|
396
|
+
model: <OpenAI model>
|
|
387
397
|
prompt: |
|
|
388
398
|
以下の記事群を一つのダイジェストとして日本語で要約してください。
|
|
389
399
|
retry: 2
|
|
@@ -394,7 +404,7 @@ in a Recipe they are interchangeable at the same position:
|
|
|
394
404
|
- module: FilterClaude
|
|
395
405
|
config:
|
|
396
406
|
token: YOUR_ANTHROPIC_API_KEY
|
|
397
|
-
model:
|
|
407
|
+
model: <Anthropic model>
|
|
398
408
|
prompt: |
|
|
399
409
|
以下の記事群を一つのダイジェストとして日本語で要約してください。
|
|
400
410
|
max_tokens: 2048
|
|
@@ -406,27 +416,39 @@ in a Recipe they are interchangeable at the same position:
|
|
|
406
416
|
- module: FilterGemini
|
|
407
417
|
config:
|
|
408
418
|
token: YOUR_GEMINI_API_KEY
|
|
409
|
-
model:
|
|
419
|
+
model: <Gemini model>
|
|
420
|
+
prompt: |
|
|
421
|
+
以下の記事群を一つのダイジェストとして日本語で要約してください。
|
|
422
|
+
retry: 2
|
|
423
|
+
interval: 2
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
```yaml
|
|
427
|
+
- module: FilterKimi
|
|
428
|
+
config:
|
|
429
|
+
token: YOUR_KIMI_API_KEY
|
|
430
|
+
model: kimi-k3
|
|
410
431
|
prompt: |
|
|
411
432
|
以下の記事群を一つのダイジェストとして日本語で要約してください。
|
|
412
433
|
retry: 2
|
|
413
434
|
interval: 2
|
|
414
435
|
```
|
|
415
436
|
|
|
416
|
-
Nothing before or after the swapped entry changes. `token`, `model` and
|
|
417
|
-
are required
|
|
418
|
-
requires it. Model names move with the services, so take them from the provider
|
|
437
|
+
Nothing before or after the swapped entry changes. `token`, `model` and
|
|
438
|
+
`prompt` are shared required settings of the AI filters; `max_tokens` exists
|
|
439
|
+
only for Claude because that API requires it. Model names move with the services, so take them from the provider
|
|
419
440
|
you are using rather than from this document, and see
|
|
420
441
|
[`PLUGINS.md`](PLUGINS.md) section 6.3 for each plugin's endpoint,
|
|
421
442
|
authentication and answer handling.
|
|
422
443
|
|
|
423
|
-
|
|
444
|
+
A Recipe naming the service on its face is the reason for keeping
|
|
445
|
+
service-specific plugins. A
|
|
424
446
|
line reading `FilterClaude` says where the text is going, which is a question
|
|
425
447
|
worth being able to answer by reading the Recipe.
|
|
426
448
|
|
|
427
449
|
## 7. Change the prompt, and the Recipe does another job
|
|
428
450
|
|
|
429
|
-
None of the
|
|
451
|
+
None of the AI filters is a summarizer. The prompt is the instruction and the
|
|
430
452
|
item's description is the text it applies to, so summarizing, translating,
|
|
431
453
|
extracting and classifying are the same plugin with different words in one
|
|
432
454
|
setting. Keeping the Recipe of section 4 and replacing only the `prompt`:
|
|
@@ -457,15 +479,17 @@ still worth remembering what the input is: pages fetched from the open web. A
|
|
|
457
479
|
prompt that states what to do when the text does not contain what was asked for
|
|
458
480
|
is more robust than one that assumes it does.
|
|
459
481
|
|
|
460
|
-
**Long input has a limit that belongs to the model, not to the framework.**
|
|
461
|
-
joined text
|
|
462
|
-
|
|
463
|
-
`
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
482
|
+
**Long input has a limit that belongs to the model, not to the framework.**
|
|
483
|
+
A joined text can exceed what a model accepts, and the service then returns an
|
|
484
|
+
error that ends the run. `FilterLimit` can cap how many source items proceed,
|
|
485
|
+
and `FilterBatch` can group source items into fixed-size item batches before
|
|
486
|
+
an AI filter so that one request need not contain the whole pipeline. Their
|
|
487
|
+
exact behavior is specified in [`PLUGINS.md`](PLUGINS.md) section 6.3.
|
|
488
|
+
|
|
489
|
+
`FilterBatch` does not split one already-joined text by character or token
|
|
490
|
+
length, and Automatic Ruby does not perform recursive summarization or a
|
|
491
|
+
final synthesis automatically. A Recipe that needs those operations still
|
|
492
|
+
needs a different pipeline design or a purpose-built component.
|
|
469
493
|
|
|
470
494
|
## 8. Why it is built this way
|
|
471
495
|
|
data/doc/BASIC_DESIGN.md
CHANGED
|
@@ -17,6 +17,12 @@ repository.
|
|
|
17
17
|
|
|
18
18
|
## 2. Design policy
|
|
19
19
|
|
|
20
|
+
Automatic Ruby is a general-purpose composition framework, not host
|
|
21
|
+
infrastructure and not a purpose-built application. The design therefore
|
|
22
|
+
protects the small set of contracts that make composition possible without
|
|
23
|
+
treating every framework implementation detail or every shipped plugin as
|
|
24
|
+
equally permanent.
|
|
25
|
+
|
|
20
26
|
- **The framework is the small part.** It loads a Recipe, finds classes by name,
|
|
21
27
|
and calls them in order. It has no domain knowledge, and gaining some would be
|
|
22
28
|
a design error rather than a feature.
|
|
@@ -32,6 +38,25 @@ repository.
|
|
|
32
38
|
- **The library never exits and never prints.** Exit status is decided by the
|
|
33
39
|
entry point; user-facing text is written by the entry point or logged.
|
|
34
40
|
|
|
41
|
+
Maintenance strength follows the architectural layer:
|
|
42
|
+
|
|
43
|
+
- **Core contracts are strongly protected.** The Recipe format, plugin
|
|
44
|
+
contract, single pipeline shape, lookup and override semantics, execution
|
|
45
|
+
order and established CLI behaviour are depended on outside this repository.
|
|
46
|
+
- **Framework internals may improve inside those contracts.** The loader, CLI,
|
|
47
|
+
helpers and internal structure are not frozen merely because they are old,
|
|
48
|
+
provided the invariants and public contracts remain intact.
|
|
49
|
+
- **Plugins are intentionally replaceable.** They may be added, repaired,
|
|
50
|
+
replaced or removed as their external systems change. A plugin whose service
|
|
51
|
+
or interface no longer exists is not preserved by simulation merely to retain
|
|
52
|
+
catalogue size.
|
|
53
|
+
|
|
54
|
+
Composability is an architectural invariant, not an optimization preference.
|
|
55
|
+
It is the architectural property that defines the framework: a small core,
|
|
56
|
+
independent plugins, one pipeline representation and Recipe-level composition.
|
|
57
|
+
A change that replaces those properties changes the identity of the system and
|
|
58
|
+
is judged as an architecture change.
|
|
59
|
+
|
|
35
60
|
## 3. Composition
|
|
36
61
|
|
|
37
62
|
A typical run is a short, linear flow. Markdown is one publisher at the edge,
|
|
@@ -280,6 +305,12 @@ The categories are a convention with one mechanical consequence — the director
|
|
|
280
305
|
name is part of the lookup key (section 4.6) — and no other. Nothing enforces
|
|
281
306
|
that a `Filter` does not reach the network.
|
|
282
307
|
|
|
308
|
+
Replaceability is part of this boundary. A plugin is not given the same
|
|
309
|
+
permanence as the Recipe format or the plugin contract itself. The framework
|
|
310
|
+
preserves the rules that let plugins compose; it does not preserve a plugin
|
|
311
|
+
whose external purpose has disappeared, and it does not move a plugin's domain
|
|
312
|
+
behaviour into the core merely to make that behaviour permanent.
|
|
313
|
+
|
|
283
314
|
**`Publish` is the boundary at which the pipeline meets a representation that is
|
|
284
315
|
not the pipeline's.** A publishing plugin reads the value described in section
|
|
285
316
|
4.8, writes it out in whatever form its destination wants — a line on a
|
data/doc/DEPLOYMENT.md
CHANGED
|
@@ -28,15 +28,16 @@ nothing to stop.
|
|
|
28
28
|
|
|
29
29
|
- A Unix-like system. GNU/Linux and macOS are what this is used on; Windows is
|
|
30
30
|
not supported.
|
|
31
|
-
- **Ruby 3.3 through 4.0.** Check with `ruby -v`.
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
31
|
+
- **Ruby 3.3 through 4.0.** Check with `ruby -v`. The continuously validated
|
|
32
|
+
versions are the matrix in [`.github/workflows/ci.yml`](../.github/workflows/ci.yml);
|
|
33
|
+
a supported version absent from that matrix is simply not checked on every
|
|
34
|
+
commit, and a Ruby newer than the matrix is permitted rather than refused.
|
|
35
|
+
See [`REQUIREMENTS.md`](REQUIREMENTS.md) section 20.
|
|
35
36
|
- Optional gems for particular plugins, listed in the table under
|
|
36
37
|
"Optional plugin dependencies" below. None is needed to install Automatic
|
|
37
38
|
Ruby, or to run a Recipe that does not use the plugin; a Recipe that does
|
|
38
|
-
name one installs it as a step of its own, which is what the
|
|
39
|
-
|
|
39
|
+
name one installs it as a step of its own, which is what the "Install what
|
|
40
|
+
the Recipe needs" section of [`QUICKSTART.md`](QUICKSTART.md) demonstrates.
|
|
40
41
|
- A build environment may be needed for one of those optional gems — `nokogiri`
|
|
41
42
|
and `sqlite3` build a native extension where no binary package matches your
|
|
42
43
|
platform. The framework's own dependencies are pure Ruby, so the normal
|
|
@@ -52,9 +53,9 @@ gem install automatic
|
|
|
52
53
|
automatic --version
|
|
53
54
|
```
|
|
54
55
|
|
|
55
|
-
That installs the framework, the `automatic` command and
|
|
56
|
-
|
|
57
|
-
|
|
56
|
+
That installs the framework, the `automatic` command and the runtime
|
|
57
|
+
dependencies declared in `automatic.gemspec`. That gemspec is the source of
|
|
58
|
+
truth for the framework's runtime dependency set. No HTML parser, no database, no service client — a gem needed by
|
|
58
59
|
one plugin is installed by the operator who uses that plugin, so installing
|
|
59
60
|
Automatic Ruby does not install what your Recipes do not use.
|
|
60
61
|
|
|
@@ -71,7 +72,7 @@ which, and is the list to check before adding anything.
|
|
|
71
72
|
### From a checkout
|
|
72
73
|
|
|
73
74
|
For working on the framework, or for running a version that is not released.
|
|
74
|
-
|
|
75
|
+
The supported checkout setups are shown below; start with the minimal setup.
|
|
75
76
|
|
|
76
77
|
**Minimal — the framework and its test suite.** What you want for running the
|
|
77
78
|
checkout, and for developing the framework itself:
|
|
@@ -99,8 +100,10 @@ bundle install
|
|
|
99
100
|
bundle exec rake
|
|
100
101
|
```
|
|
101
102
|
|
|
102
|
-
That
|
|
103
|
-
|
|
103
|
+
That installs the gems declared in the `plugins` group of `Gemfile`, and
|
|
104
|
+
their specs then run as part of the ordinary suite. The
|
|
105
|
+
[Optional plugin dependencies](#optional-plugin-dependencies) table maps
|
|
106
|
+
those dependencies to the plugins and tools that use them. The setting
|
|
104
107
|
is written to the checkout's own `.bundle/config`, which is not committed;
|
|
105
108
|
`bundle config unset --local with` returns the checkout to the minimum, and
|
|
106
109
|
`bundle install` afterwards.
|
|
@@ -115,7 +118,7 @@ bundle install
|
|
|
115
118
|
```
|
|
116
119
|
|
|
117
120
|
Several at once are space-separated: `bundle config set --local with "store
|
|
118
|
-
html"`, and a Recipe using plugins from
|
|
121
|
+
html"`, and a Recipe using plugins from multiple groups needs exactly that. The group
|
|
119
122
|
names are in the table below, and "Working out what a Recipe needs, in a
|
|
120
123
|
checkout" takes one Recipe through choosing them.
|
|
121
124
|
|
|
@@ -162,8 +165,8 @@ full path runs without it.
|
|
|
162
165
|
automatic -c ~/.automatic/config/example/feed2console.yml
|
|
163
166
|
```
|
|
164
167
|
|
|
165
|
-
That fetches one feed and prints its items.
|
|
166
|
-
|
|
168
|
+
That fetches one feed and prints its items. Distinguish an unreachable feed
|
|
169
|
+
from an optional-dependency failure:
|
|
167
170
|
|
|
168
171
|
- A message about the feed being unreachable means the network or that
|
|
169
172
|
particular feed, not the installation.
|
|
@@ -230,12 +233,12 @@ plugins:
|
|
|
230
233
|
interval: 3
|
|
231
234
|
```
|
|
232
235
|
|
|
233
|
-
That Recipe needs
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
dependencies
|
|
236
|
+
That Recipe needs `nokogiri` for `FilterImageSource`, and `activerecord` plus
|
|
237
|
+
`sqlite3` for `StorePermalink`, because of the plugins it names rather than
|
|
238
|
+
because of the framework. The canonical mapping is the
|
|
239
|
+
[Optional plugin dependencies](#optional-plugin-dependencies) table.
|
|
237
240
|
|
|
238
|
-
|
|
241
|
+
The operational advice illustrated by that Recipe is:
|
|
239
242
|
|
|
240
243
|
- **`StorePermalink` before the plugin with the effect.** It records what has
|
|
241
244
|
been seen and passes on only what has not, which is what makes the Recipe safe
|
|
@@ -318,8 +321,7 @@ mkdir -p ~/notes && chmod 700 ~/notes
|
|
|
318
321
|
|
|
319
322
|
**Keeping the document clean.** With no `file`, the document goes to standard
|
|
320
323
|
output — and so does the log ([`REQUIREMENTS.md`](REQUIREMENTS.md) section 14),
|
|
321
|
-
so a redirect that collects one collects the other.
|
|
322
|
-
chooses:
|
|
324
|
+
so a redirect that collects one collects the other. Either disable log output in the Recipe or give `PublishMarkdown` a `file`:
|
|
323
325
|
|
|
324
326
|
```yaml
|
|
325
327
|
global:
|
|
@@ -446,8 +448,10 @@ Recipe is stated there in terms you can act on.
|
|
|
446
448
|
|
|
447
449
|
## Optional plugin dependencies
|
|
448
450
|
|
|
449
|
-
This table is the
|
|
450
|
-
|
|
451
|
+
This table is the operator-facing source of truth for which plugin or tool
|
|
452
|
+
needs which optional dependency and how to install it. Current plugin support
|
|
453
|
+
status is maintained separately in
|
|
454
|
+
[`PLUGINS.md`](PLUGINS.md#6-the-plugins).
|
|
451
455
|
|
|
452
456
|
None of these gems is installed by `gem install automatic` or by a default
|
|
453
457
|
`bundle install`. Install one only if you use the plugin.
|
|
@@ -456,26 +460,27 @@ None of these gems is installed by `gem install automatic` or by a default
|
|
|
456
460
|
--local with <group>` and `bundle install`, because `bundle exec` sees only the
|
|
457
461
|
bundle. `plugins` is every group in the first block at once.
|
|
458
462
|
|
|
459
|
-
| Plugin | Needs | Installed gem | Checkout group |
|
|
460
|
-
| --- | --- | --- | --- |
|
|
461
|
-
| `StorePermalink`, `StoreFullText`, `StoreDigest` | `activerecord`, `sqlite3` | `gem install activerecord sqlite3` | `store` |
|
|
462
|
-
| `FilterImageSource`, `FilterDescriptionLink`, `SubscriptionLink`, `SubscriptionTumblr`, `CustomFeedWeb` | `nokogiri` | `gem install nokogiri` | `html` |
|
|
463
|
-
| `PublishMarkdown` | `nokogiri`, for HTML bodies only | `gem install nokogiri` | `html` |
|
|
464
|
-
| `FilterSanitize` | `sanitize` | `gem install sanitize` | `sanitize` |
|
|
465
|
-
| `autodiscovery` and `inspect` subcommands | `feedbag` | `gem install feedbag` | `autodiscovery` |
|
|
466
|
-
| `FilterFullFeed` | `nokogiri`, and a siteinfo file | `gem install nokogiri` | `html` |
|
|
467
|
-
| `CustomFeedSVNLog` | the `svn` command; no gem | — | — |
|
|
468
|
-
| `ProvideFluentd`, `PublishFluentd` | `fluent-logger`, and a Fluentd instance | `gem install fluent-logger` | `fluentd` |
|
|
469
|
-
| `PublishMemcached` | `dalli`, and a memcached server | `gem install dalli` | `memcached` |
|
|
470
|
-
| `PublishAmazonS3`, `StoreFile` S3 path | `aws-sdk-s3`, and a bucket | `gem install aws-sdk-s3` | `s3` |
|
|
471
|
-
| `PublishInstapaper` | an Instapaper account; no gem | — | — |
|
|
472
|
-
| `PublishEject` | the `eject` or `drutil` command | — | — |
|
|
473
|
-
| `NotifyIkachan` | an `ikachan` gateway you run | — | — |
|
|
474
|
-
| `FilterOpenAI`, `FilterClaude`, `FilterGemini`, `FilterSakuraAI` | an account and an API token with that one service; no gem | — | — |
|
|
475
|
-
| `PublishHatenaBookmark` | the current Hatena API, which it does not speak | — | — |
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
463
|
+
| Plugin | Needs | Installed gem | Checkout group |
|
|
464
|
+
| --- | --- | --- | --- |
|
|
465
|
+
| `StorePermalink`, `StoreFullText`, `StoreDigest` | `activerecord`, `sqlite3` | `gem install activerecord sqlite3` | `store` |
|
|
466
|
+
| `FilterImageSource`, `FilterDescriptionLink`, `SubscriptionLink`, `SubscriptionTumblr`, `CustomFeedWeb` | `nokogiri` | `gem install nokogiri` | `html` |
|
|
467
|
+
| `PublishMarkdown` | `nokogiri`, for HTML bodies only | `gem install nokogiri` | `html` |
|
|
468
|
+
| `FilterSanitize` | `sanitize` | `gem install sanitize` | `sanitize` |
|
|
469
|
+
| `autodiscovery` and `inspect` subcommands | `feedbag` | `gem install feedbag` | `autodiscovery` |
|
|
470
|
+
| `FilterFullFeed` | `nokogiri`, and a siteinfo file | `gem install nokogiri` | `html` |
|
|
471
|
+
| `CustomFeedSVNLog` | the `svn` command; no gem | — | — |
|
|
472
|
+
| `ProvideFluentd`, `PublishFluentd` | `fluent-logger`, and a Fluentd instance | `gem install fluent-logger` | `fluentd` |
|
|
473
|
+
| `PublishMemcached` | `dalli`, and a memcached server | `gem install dalli` | `memcached` |
|
|
474
|
+
| `PublishAmazonS3`, `StoreFile` S3 path | `aws-sdk-s3`, and a bucket | `gem install aws-sdk-s3` | `s3` |
|
|
475
|
+
| `PublishInstapaper` | an Instapaper account; no gem | — | — |
|
|
476
|
+
| `PublishEject` | the `eject` or `drutil` command | — | — |
|
|
477
|
+
| `NotifyIkachan` | an `ikachan` gateway you run | — | — |
|
|
478
|
+
| `FilterOpenAI`, `FilterClaude`, `FilterGemini`, `FilterSakuraAI`, `FilterKimi` | an account and an API token with that one service; no gem | — | — |
|
|
479
|
+
| `PublishHatenaBookmark` | the current Hatena API, which it does not speak | — | — |
|
|
480
|
+
|
|
481
|
+
Membership of the aggregate `plugins` group is defined in `Gemfile`. It
|
|
482
|
+
contains the optional gems whose corresponding specs need no external
|
|
483
|
+
service; the table above maps those dependencies to their consumers. The gems below it are in their own groups
|
|
479
484
|
only, because each of those plugins also needs a service, a bucket or a
|
|
480
485
|
command, and installing a gem alone would not make the plugin — or its spec —
|
|
481
486
|
work.
|
|
@@ -546,7 +551,7 @@ plugins:
|
|
|
546
551
|
|
|
547
552
|
[`QUICKSTART.md`](QUICKSTART.md) runs that Recipe end to end, with the sites it
|
|
548
553
|
watches and what each plugin does. What follows is the part a checkout does
|
|
549
|
-
differently: turning
|
|
554
|
+
differently: turning the Recipe's plugins into a bundle that can run them.
|
|
550
555
|
|
|
551
556
|
**1. List the plugins the Recipe names.** They are the `module` lines, in order:
|
|
552
557
|
`CustomFeedWeb`, `StoreDigest`, `PublishMarkdown`.
|
|
@@ -693,7 +698,7 @@ bundle exec ruby -ractive_record -e 'puts ActiveRecord::VERSION::STRING'
|
|
|
693
698
|
bundle exec ruby -rsqlite3 -e 'puts SQLite3::VERSION'
|
|
694
699
|
```
|
|
695
700
|
|
|
696
|
-
Those
|
|
701
|
+
Those commands `require` the libraries the way the plugins do — note
|
|
697
702
|
`active_record` for the `activerecord` gem — under the same bundle the Recipe
|
|
698
703
|
will run under. `gem list` answers a different question and is the one to
|
|
699
704
|
distrust here: it lists what RubyGems has, which in a checkout is neither what
|
|
@@ -701,7 +706,8 @@ the plugins will load nor what a missing-gem message is about.
|
|
|
701
706
|
|
|
702
707
|
### All of the optional gems, or only the ones a Recipe names
|
|
703
708
|
|
|
704
|
-
|
|
709
|
+
Choose the aggregate `plugins` group for plugin development, or
|
|
710
|
+
purpose-specific groups for the Recipe being run:
|
|
705
711
|
|
|
706
712
|
```sh
|
|
707
713
|
bundle config set --local with plugins # all of them, at once
|