automatic 26.08 → 26.09
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +80 -46
- data/VERSION +1 -1
- data/assets/siteinfo/items_all.json +60300 -52138
- data/automatic.gemspec +4 -3
- data/bin/automatic +1 -1
- data/doc/AI_TUTORIAL.md +64 -40
- data/doc/BASIC_DESIGN.md +31 -0
- data/doc/DEPLOYMENT.md +53 -47
- data/doc/PLUGINS.md +231 -91
- data/doc/POLICY.md +149 -44
- data/doc/QUICKSTART.md +19 -15
- data/doc/RELEASING.md +20 -7
- data/doc/REQUIREMENTS.md +40 -10
- data/doc/VERSIONS +112 -54
- data/lib/automatic/cli.rb +40 -18
- data/lib/automatic/environment.rb +1 -1
- data/lib/automatic/feed_maker.rb +57 -18
- data/lib/automatic/feed_parser.rb +1 -1
- data/lib/automatic/http.rb +1 -1
- data/lib/automatic/log.rb +1 -1
- data/lib/automatic/pipeline.rb +15 -5
- data/lib/automatic/recipe.rb +45 -1
- data/lib/automatic/version.rb +3 -3
- data/lib/automatic.rb +18 -3
- data/plugins/custom_feed/web.rb +1 -1
- data/plugins/filter/absolute_uri.rb +1 -1
- data/plugins/filter/batch.rb +97 -0
- data/plugins/filter/claude.rb +1 -1
- data/plugins/filter/clear.rb +1 -1
- data/plugins/filter/description_link.rb +20 -3
- data/plugins/filter/full_feed.rb +28 -16
- data/plugins/filter/gemini.rb +1 -1
- data/plugins/filter/ignore.rb +1 -1
- data/plugins/filter/image.rb +1 -1
- data/plugins/filter/image_source.rb +20 -3
- data/plugins/filter/join.rb +4 -6
- data/plugins/filter/kimi.rb +216 -0
- data/plugins/filter/limit.rb +57 -0
- data/plugins/filter/open_ai.rb +1 -1
- data/plugins/filter/present.rb +75 -0
- data/plugins/filter/sakura_ai.rb +1 -1
- data/plugins/filter/sanitize.rb +1 -1
- data/plugins/filter/sort.rb +1 -1
- data/plugins/filter/tumblr_resize.rb +1 -1
- data/plugins/notify/ikachan.rb +1 -1
- data/plugins/provide/fluentd.rb +1 -1
- data/plugins/publish/amazon_s3.rb +9 -3
- data/plugins/publish/console.rb +1 -1
- data/plugins/publish/fluentd.rb +1 -1
- data/plugins/publish/hatena_bookmark.rb +1 -1
- data/plugins/publish/markdown.rb +1 -1
- data/plugins/publish/memcached.rb +1 -1
- data/plugins/store/digest.rb +1 -1
- data/plugins/store/file.rb +11 -3
- data/plugins/store/full_text.rb +9 -13
- data/plugins/store/permalink.rb +1 -1
- data/plugins/subscription/feed.rb +1 -1
- data/plugins/subscription/link.rb +1 -1
- data/plugins/subscription/text.rb +13 -4
- data/plugins/subscription/tumblr.rb +1 -1
- data/plugins/subscription/xml.rb +1 -1
- metadata +6 -2
data/doc/PLUGINS.md
CHANGED
|
@@ -158,8 +158,8 @@ The value is stored and is read correctly, because plugins read their settings
|
|
|
158
158
|
by string key rather than as a property. This is noise, not breakage, and a
|
|
159
159
|
Recipe using such a key needs no change.
|
|
160
160
|
|
|
161
|
-
|
|
162
|
-
|
|
161
|
+
The existing collisions retained for compatibility are `FilterSort`'s `sort`
|
|
162
|
+
and `PublishMemcached`'s `key`. Renaming them
|
|
163
163
|
would break every Recipe using them, which is not a trade worth making for a
|
|
164
164
|
warning. **A new plugin should not introduce one**: prefer `max_length` to
|
|
165
165
|
`max`, `item_count` to `count`, `cache_key` to `key`.
|
|
@@ -265,7 +265,7 @@ found.
|
|
|
265
265
|
|
|
266
266
|
### 3.3 Discovery and precedence
|
|
267
267
|
|
|
268
|
-
|
|
268
|
+
Search roots, in this order:
|
|
269
269
|
|
|
270
270
|
1. `~/.automatic/plugins/<category>/<rest>.rb`
|
|
271
271
|
2. `<installation>/plugins/<category>/<rest>.rb`
|
|
@@ -276,7 +276,7 @@ plugin's behaviour without editing the installation.
|
|
|
276
276
|
|
|
277
277
|
Creating a new category is creating a directory. `~/.automatic/plugins/mine/`
|
|
278
278
|
plus a class named `MineSomething` works with no change to the framework, though
|
|
279
|
-
staying inside the
|
|
279
|
+
staying inside the established categories listed in section 3.2 is preferred, because their names tell a
|
|
280
280
|
reader where in a pipeline the plugin belongs.
|
|
281
281
|
|
|
282
282
|
Loading is lazy: the loader registers an `autoload`, so the file is read when
|
|
@@ -313,9 +313,10 @@ Rules that follow from the shape:
|
|
|
313
313
|
|
|
314
314
|
- **Return the shape, always.** Returning `nil`, a string or a bare array of
|
|
315
315
|
items ends the pipeline for everything after it.
|
|
316
|
-
- **`link` may be `nil`, and so may any other field.**
|
|
317
|
-
|
|
318
|
-
|
|
316
|
+
- **`link` may be `nil`, and so may any other field.** A missing field is data,
|
|
317
|
+
not a framework-wide drop signal: `FeedMaker.create_pipeline` preserves an
|
|
318
|
+
item whose link is `nil`. A plugin that requires a link must check it itself;
|
|
319
|
+
a plugin that does not may keep processing the item.
|
|
319
320
|
- **Guard the feed itself.** `@pipeline.each { |feeds| next if feeds.nil? }` is
|
|
320
321
|
the prevailing idiom, because a subscription plugin that failed may have put a
|
|
321
322
|
`nil` in the array.
|
|
@@ -558,11 +559,11 @@ Section 6 lists every plugin shipped in the gem. Each carries a status:
|
|
|
558
559
|
| **Supported (external)** | The plugin is current, but it needs something the operator provides — a running service, an installed command, a credential, a data file. |
|
|
559
560
|
| **Needs rework** | The service and the capability still exist, but this plugin speaks an interface that has been replaced. It will not work as written, and restoring it is a self-contained piece of work. |
|
|
560
561
|
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
562
|
+
No status is used for an integration whose service or capability is
|
|
563
|
+
permanently unavailable. Such plugins are removed rather than retained; see
|
|
564
|
+
section 8.
|
|
564
565
|
|
|
565
|
-
|
|
566
|
+
The rules that govern this table are the reason it exists at all:
|
|
566
567
|
|
|
567
568
|
- **Nothing is faked.** A plugin is not stubbed, mocked or simulated to make a
|
|
568
569
|
test pass or a catalogue entry look better. Where a plugin's gem is absent
|
|
@@ -584,15 +585,13 @@ its spec as part of the ordinary suite, which is also what the separate
|
|
|
584
585
|
plugin is not demoted for needing a gem, and is not promoted by a test that CI
|
|
585
586
|
never executes.
|
|
586
587
|
|
|
587
|
-
**
|
|
588
|
-
|
|
589
|
-
The statuses that depend on an outside service can change without any commit
|
|
590
|
-
here. Where a status was reached from published information rather than from a
|
|
588
|
+
**Statuses involving an outside service are snapshots of external state.**
|
|
589
|
+
They can change without any commit here. Where a status was reached from published information rather than from a
|
|
591
590
|
live check, the entry says so. To verify one yourself, run its Recipe from
|
|
592
591
|
`test/integration` by hand; those are not part of CI and never will be.
|
|
593
592
|
|
|
594
|
-
Restoring
|
|
595
|
-
a good first contribution.
|
|
593
|
+
Restoring a plugin classified as **Needs rework** is a self-contained piece
|
|
594
|
+
of work and a good first contribution.
|
|
596
595
|
|
|
597
596
|
---
|
|
598
597
|
|
|
@@ -670,10 +669,13 @@ no network, which makes it the plugin to test a Recipe's later half with.
|
|
|
670
669
|
| `titles` | sequence | One item per title, no link |
|
|
671
670
|
| `urls` | sequence | One item per URL, no title |
|
|
672
671
|
| `feeds` | sequence | Mappings of `title`, `url`, `description`, `author`, `comments` |
|
|
673
|
-
| `files` | sequence | TSV paths; columns are title, url, description, author, comments |
|
|
672
|
+
| `files` | sequence | UTF-8 TSV paths; positional columns are title, url, description, author, comments |
|
|
674
673
|
|
|
675
|
-
The TSV separator is a tab
|
|
676
|
-
|
|
674
|
+
The TSV separator is a tab and `~` is expanded. Empty columns supply no value
|
|
675
|
+
for that field without shifting later columns; blank or all-empty rows produce
|
|
676
|
+
no item. Only the first five columns are used. Line endings are removed before
|
|
677
|
+
splitting, but other field whitespace is preserved. Any combination of the four
|
|
678
|
+
keys may be given.
|
|
677
679
|
|
|
678
680
|
#### SubscriptionTumblr — **Supported (external)**
|
|
679
681
|
|
|
@@ -866,6 +868,34 @@ match. Same three keys, same substring rule. An item whose field is missing is
|
|
|
866
868
|
not matched, and says so; it used to end the run with a `NoMethodError`, which
|
|
867
869
|
is not what its complement does with the same item.
|
|
868
870
|
|
|
871
|
+
#### FilterPresent — **Supported**
|
|
872
|
+
|
|
873
|
+
`filter/present.rb`. Keeps only items for which every configured field is
|
|
874
|
+
present. It is an AND filter over field presence, not a keyword matcher.
|
|
875
|
+
|
|
876
|
+
Recipe:
|
|
877
|
+
|
|
878
|
+
```yaml
|
|
879
|
+
- module: FilterPresent
|
|
880
|
+
config:
|
|
881
|
+
fields:
|
|
882
|
+
- title
|
|
883
|
+
- description
|
|
884
|
+
```
|
|
885
|
+
|
|
886
|
+
| Key | Type | Meaning |
|
|
887
|
+
| --- | --- | --- |
|
|
888
|
+
| `fields` | sequence | Fields that must all be present. Required and non-empty. |
|
|
889
|
+
|
|
890
|
+
The fields that may be checked are `title`, `link`, `description`, `author`,
|
|
891
|
+
`comments`, `source` and `content_encoded`. A field the item has no accessor
|
|
892
|
+
for, a `nil` value, an empty string and a whitespace-only string are all
|
|
893
|
+
absent; an RSS field that responds to `#content` — a parsed source, for
|
|
894
|
+
instance — is judged on that content rather than on the element itself. An
|
|
895
|
+
item is kept only when every configured field is present. An unknown field or
|
|
896
|
+
a field named twice is a settings error, raised when the plugin is
|
|
897
|
+
constructed. No network access, and no dependency on another plugin.
|
|
898
|
+
|
|
869
899
|
#### FilterSort — **Supported**
|
|
870
900
|
|
|
871
901
|
`filter/sort.rb`. Sorts each feed's items by date.
|
|
@@ -887,6 +917,31 @@ setting works; see section 2.6.1.
|
|
|
887
917
|
| --- | --- | --- |
|
|
888
918
|
| `pick` | string | `last` takes the last item. Anything else, including absent, takes the first. |
|
|
889
919
|
|
|
920
|
+
#### FilterLimit — **Supported**
|
|
921
|
+
|
|
922
|
+
`filter/limit.rb`. Limits how many items the whole pipeline passes downstream.
|
|
923
|
+
The limit is shared across feeds rather than applied once per feed.
|
|
924
|
+
|
|
925
|
+
Recipe:
|
|
926
|
+
|
|
927
|
+
```yaml
|
|
928
|
+
- module: FilterLimit
|
|
929
|
+
config:
|
|
930
|
+
max_items: 20
|
|
931
|
+
```
|
|
932
|
+
|
|
933
|
+
| Key | Type | Meaning |
|
|
934
|
+
| --- | --- | --- |
|
|
935
|
+
| `max_items` | integer | Maximum items passed by the whole pipeline. Required; must be greater than zero. |
|
|
936
|
+
|
|
937
|
+
Items are selected in feed order, then item order, up to `max_items`; the
|
|
938
|
+
grouping of the input feeds is kept in the output, and a feed that contributed
|
|
939
|
+
no item under the limit is left out of it. Once the limit is reached, later
|
|
940
|
+
feeds are not walked at all. Anything other than a positive integer —
|
|
941
|
+
including zero, a negative number, and a non-numeric or fractional string — is
|
|
942
|
+
a settings error, raised when the plugin is constructed; a numeric string such
|
|
943
|
+
as `"20"` is accepted. No network access, and no dependency on another plugin.
|
|
944
|
+
|
|
890
945
|
#### FilterRand — **Supported**
|
|
891
946
|
|
|
892
947
|
`filter/rand.rb`. Shuffles each feed's items. Combined with `FilterOne`, picks
|
|
@@ -899,9 +954,10 @@ plugin has done the work, so that later plugins publish nothing. No settings.
|
|
|
899
954
|
|
|
900
955
|
#### FilterImage — **Supported**
|
|
901
956
|
|
|
902
|
-
`filter/image.rb`. Sets `link` to `nil` unless it names an image.
|
|
903
|
-
|
|
904
|
-
|
|
957
|
+
`filter/image.rb`. Sets `link` to `nil` unless it names an image. It does not
|
|
958
|
+
remove the item: a later plugin that requires a link skips it under that
|
|
959
|
+
plugin's own rules, while a link-independent plugin may continue to use it. No
|
|
960
|
+
settings.
|
|
905
961
|
|
|
906
962
|
The extensions are `.jpg`, `.jpeg`, `.gif`, `.png`, `.tif`, `.tiff`, `.webp`
|
|
907
963
|
and `.avif`, and the test is on the URL's **path**. Both of those changed:
|
|
@@ -914,7 +970,16 @@ this filter will therefore keep links it used to blank.
|
|
|
914
970
|
|
|
915
971
|
`filter/image_source.rb`. Replaces each item with one item per image found: the
|
|
916
972
|
images in the description, or, if there are none, the images on the page the
|
|
917
|
-
link points at. Fetching pages means network access.
|
|
973
|
+
link points at. Fetching pages means network access.
|
|
974
|
+
|
|
975
|
+
| Key | Type | Meaning |
|
|
976
|
+
| --- | --- | --- |
|
|
977
|
+
| `interval` | integer | Seconds to wait after each page fetch attempt. Default `0`. |
|
|
978
|
+
|
|
979
|
+
An item whose images come from the description is never fetched, and
|
|
980
|
+
`interval` does not apply to it. Where a page is fetched, the wait follows the
|
|
981
|
+
actual fetch attempt whether it succeeded or failed. A non-positive or
|
|
982
|
+
non-numeric `interval` — including it being absent — means no wait at all.
|
|
918
983
|
|
|
919
984
|
Needs `nokogiri`: `gem install nokogiri`, or the `html` group in a checkout.
|
|
920
985
|
|
|
@@ -974,9 +1039,14 @@ the body.
|
|
|
974
1039
|
| --- | --- | --- |
|
|
975
1040
|
| `clear_description` | `1` | Empty the description afterwards. Any other value leaves it. |
|
|
976
1041
|
| `get_title` | `1` | Fetch the new link and use its `<title>`. Any other value skips it. |
|
|
1042
|
+
| `interval` | integer | Seconds to wait after each title-page fetch attempt when `get_title` is `1`. Default `0`. |
|
|
977
1043
|
|
|
978
1044
|
`get_title` makes one request per item; use `FilterOne` or a store plugin before
|
|
979
|
-
it on a large feed.
|
|
1045
|
+
it on a large feed. When `get_title` is not `1`, no title page is fetched and
|
|
1046
|
+
`interval` does not apply. A URL that is not fetchable is not read either, and
|
|
1047
|
+
does not wait. Where a title page is read, the wait follows the actual fetch
|
|
1048
|
+
attempt whether it succeeded or failed. A non-positive or non-numeric
|
|
1049
|
+
`interval` — including it being absent — means no wait at all.
|
|
980
1050
|
|
|
981
1051
|
**Both settings were being ignored in every real run.** The test that guarded
|
|
982
1052
|
them asked whether the settings mapping was a `Hash`, and the framework hands a
|
|
@@ -999,37 +1069,36 @@ page.
|
|
|
999
1069
|
| Key | Type | Meaning |
|
|
1000
1070
|
| --- | --- | --- |
|
|
1001
1071
|
| `siteinfo` | string | File name under the assets directory. Required. |
|
|
1072
|
+
| `interval` | integer | Seconds to wait after each article-page fetch attempt. Default `0`. |
|
|
1073
|
+
|
|
1074
|
+
An item whose link matches no siteinfo record is never fetched, and `interval`
|
|
1075
|
+
does not apply to it. Where a link does match, the wait follows the actual
|
|
1076
|
+
fetch attempt whether it succeeded or failed. A non-positive or non-numeric
|
|
1077
|
+
`interval` — including it being absent — means no wait at all.
|
|
1002
1078
|
|
|
1003
1079
|
Needs `nokogiri`: `gem install nokogiri`, or the `html` group in a checkout.
|
|
1004
1080
|
|
|
1005
|
-
The shipped `assets/siteinfo/items_all.json` is a
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
|
|
1009
|
-
|
|
1010
|
-
`~/.automatic/assets/siteinfo
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
- **A link matches under either scheme.**
|
|
1016
|
-
|
|
1017
|
-
|
|
1018
|
-
|
|
1019
|
-
|
|
1020
|
-
|
|
1021
|
-
the
|
|
1022
|
-
- **
|
|
1023
|
-
|
|
1024
|
-
|
|
1025
|
-
|
|
1026
|
-
the XPath that missed.
|
|
1027
|
-
- **The page's own encoding is believed before the record's.** The page is
|
|
1028
|
-
parsed from the stream, so a charset in a `meta` tag is read even when the
|
|
1029
|
-
response declared none. A record's `enc` is the fallback for a page that
|
|
1030
|
-
declares nothing anywhere — 1,186 records carry one, mostly EUC-JP and
|
|
1031
|
-
Shift_JIS — and an `enc` naming an encoding Ruby does not have is ignored
|
|
1032
|
-
rather than raised. What comes out is UTF-8 either way.
|
|
1081
|
+
The shipped `assets/siteinfo/items_all.json` is a bundled LDRFullFeed-compatible
|
|
1082
|
+
siteinfo snapshot. Its coverage is a snapshot rather than a guarantee that a
|
|
1083
|
+
site is present or still uses the recorded layout. The bundled file may be
|
|
1084
|
+
replaced from a newer upstream snapshot, as recorded in
|
|
1085
|
+
[`VERSIONS`](VERSIONS); an operator can also supply a siteinfo file under
|
|
1086
|
+
`~/.automatic/assets/siteinfo/`.
|
|
1087
|
+
|
|
1088
|
+
The plugin's behavior does not depend on the bundled snapshot having a
|
|
1089
|
+
particular record count, date, scheme distribution, or encoding distribution:
|
|
1090
|
+
|
|
1091
|
+
- **A link matches under either HTTP scheme.** A stored URL pattern can name
|
|
1092
|
+
`http://` while a current feed supplies `https://`, or the reverse. Matching
|
|
1093
|
+
therefore tries both schemes while fetching the page from the original item
|
|
1094
|
+
link.
|
|
1095
|
+
- **A record that selects nothing leaves the summary alone.** If the recorded
|
|
1096
|
+
XPath no longer selects content, the existing description is preserved and
|
|
1097
|
+
the miss is logged at `warn`.
|
|
1098
|
+
- **The page's own encoding is believed before the record's.** Response or
|
|
1099
|
+
document declarations take precedence; the record's `enc` is only a fallback,
|
|
1100
|
+
and an encoding Ruby does not recognize is ignored rather than raised. The
|
|
1101
|
+
selected body is normalized to UTF-8.
|
|
1033
1102
|
|
|
1034
1103
|
A record with no URL pattern, no XPath, or a pattern that is not a regular
|
|
1035
1104
|
expression is dropped when the file is loaded rather than being allowed to fail
|
|
@@ -1045,6 +1114,35 @@ pipeline expects. Needed because GitHub publishes Atom, not RSS. No settings.
|
|
|
1045
1114
|
A field that is already a string is taken as it stands, so a pipeline that has
|
|
1046
1115
|
been through another filter first is no longer a `NoMethodError`.
|
|
1047
1116
|
|
|
1117
|
+
#### FilterBatch — **Supported**
|
|
1118
|
+
|
|
1119
|
+
`filter/batch.rb`. Groups all items in the pipeline into fixed-size batches.
|
|
1120
|
+
Feed boundaries are intentionally discarded; each batch becomes one item in one
|
|
1121
|
+
output feed.
|
|
1122
|
+
|
|
1123
|
+
Recipe:
|
|
1124
|
+
|
|
1125
|
+
```yaml
|
|
1126
|
+
- module: FilterBatch
|
|
1127
|
+
config:
|
|
1128
|
+
batch_items: 5
|
|
1129
|
+
```
|
|
1130
|
+
|
|
1131
|
+
| Key | Type | Meaning |
|
|
1132
|
+
| --- | --- | --- |
|
|
1133
|
+
| `batch_items` | integer | Maximum source items in one batch. Required; must be greater than zero. |
|
|
1134
|
+
|
|
1135
|
+
The whole pipeline is collected in feed order, then item order, and sliced
|
|
1136
|
+
into batches of `batch_items` source items. Each batch becomes one item titled
|
|
1137
|
+
`Batch N`; a batch item carries no link. Its description lists the source
|
|
1138
|
+
items as `ARTICLE N`, `Title:`, `URL:` and the body, with the `ARTICLE`
|
|
1139
|
+
numbering starting over at 1 in every batch. An empty pipeline produces an
|
|
1140
|
+
empty pipeline. Anything other than a positive integer — including zero, a
|
|
1141
|
+
negative number, and a non-numeric or fractional string — is a settings error,
|
|
1142
|
+
raised when the plugin is constructed; a numeric string such as `"5"` is
|
|
1143
|
+
accepted. It is independent of `FilterJoin` — it does not require it, call it,
|
|
1144
|
+
or depend on it in any way — and reaches no network and no external service.
|
|
1145
|
+
|
|
1048
1146
|
#### FilterJoin — **Supported**
|
|
1049
1147
|
|
|
1050
1148
|
`filter/join.rb`. Joins every item in the pipeline into one item. Many items
|
|
@@ -1101,9 +1199,10 @@ next.
|
|
|
1101
1199
|
title: Daily Digest
|
|
1102
1200
|
```
|
|
1103
1201
|
|
|
1104
|
-
**The
|
|
1202
|
+
**The AI filters.** The plugins that follow each send an item's
|
|
1105
1203
|
description to one AI service and put the answer back in its place. They are
|
|
1106
|
-
|
|
1204
|
+
separate service-specific plugins rather than one plugin with a
|
|
1205
|
+
`provider` setting, and that is the design
|
|
1107
1206
|
rather than an accident: the services differ in endpoint, authentication,
|
|
1108
1207
|
request body, answer shape, error format and available models; each of those
|
|
1109
1208
|
moves without asking the others; and a Recipe naming `FilterClaude` says on its
|
|
@@ -1114,11 +1213,10 @@ Changing service is changing that one line.
|
|
|
1114
1213
|
the item's description is the text it applies to, so summarizing, translating,
|
|
1115
1214
|
extracting, reformatting and classifying are the same plugin with a different
|
|
1116
1215
|
prompt. There is no default prompt: a Recipe without one is refused with an
|
|
1117
|
-
`ArgumentError` rather than being given a purpose it did not ask for. The
|
|
1118
|
-
are sent as separate fields — a system instruction and a user turn — so that
|
|
1216
|
+
`ArgumentError` rather than being given a purpose it did not ask for. The instruction and the item text are sent as separate fields — a system instruction and a user turn — so that
|
|
1119
1217
|
what an article says is text to be worked on, never an instruction to obey.
|
|
1120
1218
|
|
|
1121
|
-
What the
|
|
1219
|
+
What the AI filters have in common:
|
|
1122
1220
|
|
|
1123
1221
|
| Point | What it is |
|
|
1124
1222
|
| --- | --- |
|
|
@@ -1156,16 +1254,16 @@ for new integrations, and authenticates with the token as a bearer token.
|
|
|
1156
1254
|
| `retry` | integer | Attempts after a failure. Default `0`. |
|
|
1157
1255
|
| `interval` | integer | Seconds between attempts. Default `0`. |
|
|
1158
1256
|
|
|
1159
|
-
The endpoint is
|
|
1160
|
-
plugin that talks to a different host, and a
|
|
1161
|
-
send the token somewhere else. The answer is read out of the typed `output`
|
|
1257
|
+
The endpoint is fixed by this plugin rather than exposed as a setting. An
|
|
1258
|
+
operator has no version of this plugin that talks to a different host, and a
|
|
1259
|
+
setting for the endpoint would provide a way to send the token somewhere else. The answer is read out of the typed `output`
|
|
1162
1260
|
array, from the `output_text` of the assistant's message.
|
|
1163
1261
|
|
|
1164
1262
|
```yaml
|
|
1165
1263
|
- module: FilterOpenAI
|
|
1166
1264
|
config:
|
|
1167
1265
|
token: sk-...
|
|
1168
|
-
model:
|
|
1266
|
+
model: <OpenAI model>
|
|
1169
1267
|
prompt: |
|
|
1170
1268
|
Summarize the following articles as one digest, in Japanese.
|
|
1171
1269
|
retry: 2
|
|
@@ -1178,7 +1276,8 @@ array, from the `output_text` of the assistant's message.
|
|
|
1178
1276
|
API, `https://api.anthropic.com/v1/messages`, and replaces it with the answer.
|
|
1179
1277
|
Anthropic authenticates with an `x-api-key` header rather than a bearer token,
|
|
1180
1278
|
requires an API version header, and requires a `max_tokens` — so this plugin
|
|
1181
|
-
sends
|
|
1279
|
+
sends those required values and exposes the required `max_tokens` setting in
|
|
1280
|
+
addition to the shared settings.
|
|
1182
1281
|
|
|
1183
1282
|
| Key | Type | Meaning |
|
|
1184
1283
|
| --- | --- | --- |
|
|
@@ -1198,7 +1297,7 @@ blocks of other kinds are passed over.
|
|
|
1198
1297
|
- module: FilterClaude
|
|
1199
1298
|
config:
|
|
1200
1299
|
token: sk-ant-...
|
|
1201
|
-
model:
|
|
1300
|
+
model: <Anthropic model>
|
|
1202
1301
|
prompt: |
|
|
1203
1302
|
Summarize the following articles as one digest, in Japanese.
|
|
1204
1303
|
max_tokens: 2048
|
|
@@ -1219,7 +1318,7 @@ URL and out of anything that logs one.
|
|
|
1219
1318
|
| Key | Type | Meaning |
|
|
1220
1319
|
| --- | --- | --- |
|
|
1221
1320
|
| `token` | string | Gemini API key, sent as `x-goog-api-key`. Required. |
|
|
1222
|
-
| `model` | string |
|
|
1321
|
+
| `model` | string | Bare model name without the `models/` prefix. Required. |
|
|
1223
1322
|
| `prompt` | string | The instruction, sent as `system_instruction`. Required. |
|
|
1224
1323
|
| `retry` | integer | Attempts after a failure. Default `0`. |
|
|
1225
1324
|
| `interval` | integer | Seconds between attempts. Default `0`. |
|
|
@@ -1234,7 +1333,7 @@ description.
|
|
|
1234
1333
|
- module: FilterGemini
|
|
1235
1334
|
config:
|
|
1236
1335
|
token: AIza...
|
|
1237
|
-
model:
|
|
1336
|
+
model: <Gemini model>
|
|
1238
1337
|
prompt: |
|
|
1239
1338
|
Summarize the following articles as one digest, in Japanese.
|
|
1240
1339
|
retry: 2
|
|
@@ -1266,7 +1365,46 @@ Recipe that says where the text goes for a Recipe that does not.
|
|
|
1266
1365
|
- module: FilterSakuraAI
|
|
1267
1366
|
config:
|
|
1268
1367
|
token: ...
|
|
1269
|
-
model:
|
|
1368
|
+
model: <Sakura AI Engine model>
|
|
1369
|
+
prompt: |
|
|
1370
|
+
以下の記事群について、個別記事の要約を羅列するのではなく、
|
|
1371
|
+
全体を一つのダイジェストとして日本語で要約してください。
|
|
1372
|
+
retry: 2
|
|
1373
|
+
interval: 2
|
|
1374
|
+
```
|
|
1375
|
+
|
|
1376
|
+
#### FilterKimi — **Supported (external)**
|
|
1377
|
+
|
|
1378
|
+
`filter/kimi.rb`. Sends each item's description to Moonshot AI's Kimi,
|
|
1379
|
+
`https://api.moonshot.ai/v1/chat/completions`, and replaces it with the
|
|
1380
|
+
answer. The token is a bearer token, and the request is the chat completions
|
|
1381
|
+
form: the prompt as a `system` message, the description as a `user` message.
|
|
1382
|
+
|
|
1383
|
+
| Key | Type | Meaning |
|
|
1384
|
+
| --- | --- | --- |
|
|
1385
|
+
| `token` | string | Kimi API key. Required. |
|
|
1386
|
+
| `model` | string | Model name, as Moonshot names it (e.g. `kimi-k3`). Required. |
|
|
1387
|
+
| `prompt` | string | The instruction, sent as the `system` message. Required. |
|
|
1388
|
+
| `retry` | integer | Attempts after a failure. Default `0`. |
|
|
1389
|
+
| `interval` | integer | Seconds between attempts. Default `0`. |
|
|
1390
|
+
|
|
1391
|
+
**This interface is OpenAI-compatible, and this is still its own plugin.** It is
|
|
1392
|
+
a different service: a different endpoint, a different account, a different set
|
|
1393
|
+
of models, its own limits and its own errors, any of which may move without
|
|
1394
|
+
OpenAI moving. Folding it into `FilterOpenAI` behind a setting would trade a
|
|
1395
|
+
Recipe that says where the text goes for a Recipe that does not, the same
|
|
1396
|
+
reasoning that already keeps Sakura AI in a plugin of its own.
|
|
1397
|
+
|
|
1398
|
+
Kimi may return a `reasoning_content` alongside the answer's `content`. That
|
|
1399
|
+
reasoning is never read, logged or written anywhere; only `content` is used,
|
|
1400
|
+
and only once `finish_reason` is `"stop"` -- any other value, or a missing or
|
|
1401
|
+
empty `content`, is an error rather than an empty description.
|
|
1402
|
+
|
|
1403
|
+
```yaml
|
|
1404
|
+
- module: FilterKimi
|
|
1405
|
+
config:
|
|
1406
|
+
token: YOUR_KIMI_API_KEY
|
|
1407
|
+
model: kimi-k3
|
|
1270
1408
|
prompt: |
|
|
1271
1409
|
以下の記事群について、個別記事の要約を羅列するのではなく、
|
|
1272
1410
|
全体を一つのダイジェストとして日本語で要約してください。
|
|
@@ -1301,7 +1439,7 @@ plugins:
|
|
|
1301
1439
|
- module: FilterSakuraAI
|
|
1302
1440
|
config:
|
|
1303
1441
|
token: ...
|
|
1304
|
-
model:
|
|
1442
|
+
model: <Sakura AI Engine model>
|
|
1305
1443
|
prompt: |
|
|
1306
1444
|
以下の記事群について、個別記事の要約を羅列するのではなく、
|
|
1307
1445
|
全体を一つのダイジェストとして日本語で要約してください。
|
|
@@ -1315,8 +1453,8 @@ plugins:
|
|
|
1315
1453
|
```
|
|
1316
1454
|
|
|
1317
1455
|
Changing service is changing the one entry: `FilterSakuraAI` for
|
|
1318
|
-
`FilterOpenAI`, `FilterClaude` or `
|
|
1319
|
-
settings. Nothing before or after it changes.
|
|
1456
|
+
`FilterOpenAI`, `FilterClaude`, `FilterGemini` or `FilterKimi`, with that
|
|
1457
|
+
plugin's own settings. Nothing before or after it changes.
|
|
1320
1458
|
|
|
1321
1459
|
### 6.4 Store
|
|
1322
1460
|
|
|
@@ -1350,9 +1488,10 @@ look new again.
|
|
|
1350
1488
|
#### StoreFullText — **Supported**
|
|
1351
1489
|
|
|
1352
1490
|
`store/full_text.rb`. Records title, link, description and `content_encoded`,
|
|
1353
|
-
and passes on only
|
|
1354
|
-
republished article with a new URL is not stored
|
|
1355
|
-
|
|
1491
|
+
and passes on only items whose new record was saved successfully. Deduplicates
|
|
1492
|
+
on link **or** title, so a republished article with a new URL is not stored
|
|
1493
|
+
twice. A database write failure ends the run; the unsaved item is not passed
|
|
1494
|
+
downstream. Pair with `FilterFullFeed` to archive article bodies.
|
|
1356
1495
|
|
|
1357
1496
|
| Key | Type | Meaning |
|
|
1358
1497
|
| --- | --- | --- |
|
|
@@ -1478,12 +1617,14 @@ headline edited between runs no longer republishes the article.
|
|
|
1478
1617
|
|
|
1479
1618
|
#### StoreFile — **Supported**
|
|
1480
1619
|
|
|
1481
|
-
`store/file.rb`. Downloads what each link points at and rewrites the link to
|
|
1482
|
-
`file
|
|
1620
|
+
`store/file.rb`. Downloads what each link points at and rewrites the link to an
|
|
1621
|
+
absolute `file:` URI for the saved file, which is how `PublishAmazonS3` later
|
|
1622
|
+
knows it has a local file. URI-reserved characters in the saved path are
|
|
1623
|
+
percent-encoded.
|
|
1483
1624
|
|
|
1484
1625
|
| Key | Type | Meaning |
|
|
1485
1626
|
| --- | --- | --- |
|
|
1486
|
-
| `path` | string | Directory to save into; created if absent. Required. |
|
|
1627
|
+
| `path` | string | Directory to save into; relative paths use the process working directory; created if absent. Required. |
|
|
1487
1628
|
| `retry` | integer | Attempts after the first. Default `0`. |
|
|
1488
1629
|
| `interval` | integer | Seconds between downloads. Default `0`. |
|
|
1489
1630
|
| `access_key` | string | S3 only. Omit to use the SDK's own credential chain. |
|
|
@@ -1782,8 +1923,9 @@ timeout, so an unanswered request ends rather than hanging a `cron` job.
|
|
|
1782
1923
|
|
|
1783
1924
|
#### PublishAmazonS3 — **Supported (external)**
|
|
1784
1925
|
|
|
1785
|
-
`publish/amazon_s3.rb`. Uploads files whose link is a `file
|
|
1786
|
-
normally after `StoreFile`.
|
|
1926
|
+
`publish/amazon_s3.rb`. Uploads files whose link is a `file:` URI to S3,
|
|
1927
|
+
normally after `StoreFile`. A percent-encoded URI path is decoded back to the
|
|
1928
|
+
local filesystem path before upload.
|
|
1787
1929
|
|
|
1788
1930
|
| Key | Type | Meaning |
|
|
1789
1931
|
| --- | --- | --- |
|
|
@@ -1832,21 +1974,19 @@ is a claim that the plugin works.
|
|
|
1832
1974
|
|
|
1833
1975
|
---
|
|
1834
1976
|
|
|
1835
|
-
## 7.
|
|
1836
|
-
|
|
1837
|
-
| Status | Count | Plugins |
|
|
1838
|
-
| --- | --- | --- |
|
|
1839
|
-
| Supported | 26 | `SubscriptionFeed`, `SubscriptionLink`, `SubscriptionXml`, `SubscriptionText`, `CustomFeedWeb`, `FilterIgnore`, `FilterAccept`, `FilterSort`, `FilterOne`, `FilterRand`, `FilterClear`, `FilterImage`, `FilterImageSource`, `FilterAbsoluteURI`, `FilterSanitize`, `FilterTumblrResize`, `FilterDescriptionLink`, `FilterGithubFeed`, `FilterJoin`, `StorePermalink`, `StoreFullText`, `StoreDigest`, `StoreFile`, `PublishMarkdown`, `PublishConsole`, `PublishConsoleLink` |
|
|
1840
|
-
| Supported (external) | 14 | `SubscriptionTumblr`, `CustomFeedSVNLog`, `FilterFullFeed`, `FilterOpenAI`, `FilterClaude`, `FilterGemini`, `FilterSakuraAI`, `ProvideFluentd`, `NotifyIkachan`, `PublishEject`, `PublishMemcached`, `PublishFluentd`, `PublishInstapaper`, `PublishAmazonS3` |
|
|
1841
|
-
| Needs rework | 1 | `PublishHatenaBookmark` |
|
|
1977
|
+
## 7. Catalogue maintenance
|
|
1842
1978
|
|
|
1843
|
-
|
|
1844
|
-
|
|
1845
|
-
|
|
1979
|
+
Section 6 is the single source of truth for the set of plugins that ship and
|
|
1980
|
+
for each plugin's current status. Current plugin totals, per-status totals and
|
|
1981
|
+
duplicate current plugin-name lists are not maintained here or in `README.md`;
|
|
1982
|
+
adding or removing a plugin changes its implementation, its specification and
|
|
1983
|
+
its Section 6 catalogue entry, not a second summary that has to be kept in
|
|
1984
|
+
sync.
|
|
1846
1985
|
|
|
1847
|
-
`spec/doc/plugins_catalogue_spec.rb`
|
|
1848
|
-
|
|
1849
|
-
|
|
1986
|
+
`spec/doc/plugins_catalogue_spec.rb` verifies that every shipped plugin has
|
|
1987
|
+
exactly one Section 6 entry, every Section 6 entry has a shipped plugin at the
|
|
1988
|
+
loader-derived path, and every entry uses one of the statuses defined in
|
|
1989
|
+
section 5.
|
|
1850
1990
|
|
|
1851
1991
|
## 8. Plugins that were removed
|
|
1852
1992
|
|