automatic 12.3.0 → 12.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/Gemfile +1 -0
- data/README.md +54 -36
- data/Rakefile +14 -0
- data/VERSION +1 -1
- data/automatic.gemspec +17 -5
- data/bin/automatic +37 -3
- data/bin/automatic-config +77 -0
- data/config/default.yml +9 -12
- data/doc/ChangeLog +32 -8
- data/doc/PLUGINS +205 -0
- data/doc/PLUGINS.ja +2 -3
- data/doc/README +488 -0
- data/doc/README.ja +195 -131
- data/lib/automatic/feed_parser.rb +1 -9
- data/lib/automatic/log.rb +1 -9
- data/lib/automatic/opml.rb +239 -0
- data/lib/automatic/pipeline.rb +16 -10
- data/lib/automatic/recipe.rb +3 -4
- data/lib/automatic.rb +32 -38
- data/lib/config/validator.rb +83 -0
- data/plugins/custom_feed/svn_log.rb +1 -1
- data/plugins/filter/ignore.rb +9 -1
- data/plugins/notify/ikachan.rb +7 -6
- data/plugins/publish/hatena_bookmark.rb +6 -9
- data/script/build +63 -0
- data/spec/lib/automatic/pipeline_spec.rb +55 -0
- data/spec/lib/automatic_spec.rb +77 -0
- data/spec/lib/pipeline_spec.rb +67 -0
- data/spec/plugins/filter/ignore_spec.rb +16 -0
- data/spec/plugins/filter/image_spec.rb +4 -4
- data/spec/plugins/filter/tumblr_resize_spec.rb +4 -4
- data/spec/plugins/notify/ikachan_spec.rb +30 -0
- data/spec/plugins/publish/console_spec.rb +1 -2
- data/spec/plugins/publish/hatena_bookmark_spec.rb +36 -1
- data/spec/plugins/store/full_text_spec.rb +0 -2
- data/spec/plugins/store/permalink_spec.rb +0 -1
- data/spec/plugins/store/target_link_spec.rb +0 -1
- data/spec/plugins/subscription/feed_spec.rb +0 -1
- data/spec/spec_helper.rb +6 -4
- data/spec/user_dir/plugins/store/mock.rb +12 -0
- data/test/fixtures/sampleOPML.xml +11 -0
- data/test/integration/test_activerecord.yml +2 -2
- data/test/integration/test_fulltext.yml +3 -3
- data/test/integration/test_hatenabookmark.yml +6 -2
- data/test/integration/test_ignore.yml +4 -1
- data/test/integration/test_ignore2.yml +1 -4
- data/test/integration/test_image2local.yml +3 -5
- data/test/integration/test_svnlog.yml +2 -1
- data/test/integration/test_tumblr2local.yml +3 -3
- metadata +43 -22
- data/utils/auto_discovery.rb +0 -18
- data/utils/opml_parser.rb +0 -247
data/doc/README
ADDED
@@ -0,0 +1,488 @@
|
|
1
|
+
automaticruby
|
2
|
+
|
3
|
+
Name
|
4
|
+
Automatic Ruby - Ruby General Automation Framework
|
5
|
+
|
6
|
+
Syntax
|
7
|
+
$ automatic
|
8
|
+
|
9
|
+
Specify the start of any recipe
|
10
|
+
$ automatic -c <recipe>
|
11
|
+
|
12
|
+
To view the version number
|
13
|
+
$ automatic -v
|
14
|
+
|
15
|
+
Description
|
16
|
+
This is a general-purpose automatic processing
|
17
|
+
ruby framework which can extend the functionality
|
18
|
+
by plug-ins.
|
19
|
+
|
20
|
+
|
21
|
+
============
|
22
|
+
Installation
|
23
|
+
============
|
24
|
+
|
25
|
+
[Stable]
|
26
|
+
$ gem install automatic
|
27
|
+
|
28
|
+
[Development]
|
29
|
+
$ git clone git://github.com/id774/automaticruby.git
|
30
|
+
|
31
|
+
|
32
|
+
If the ruby 1.9, put following command.
|
33
|
+
$ script/bootstrap
|
34
|
+
|
35
|
+
If the ruby 1.8, necessary to manually install the gems in the Gemfile.
|
36
|
+
|
37
|
+
|
38
|
+
===========
|
39
|
+
Get Started
|
40
|
+
===========
|
41
|
+
|
42
|
+
To create of any recipe (see below).
|
43
|
+
$ automatic -c your_recipe.yml
|
44
|
+
|
45
|
+
-c option to run with the manual. If there is no problem
|
46
|
+
to automate, register it to cron.
|
47
|
+
|
48
|
+
|
49
|
+
This is the easy recipe to simply output blog's feed to
|
50
|
+
only console. It can be used to check the operation of
|
51
|
+
this software.
|
52
|
+
$ automatic -c config/feed2console.yml
|
53
|
+
|
54
|
+
|
55
|
+
=======
|
56
|
+
Recipes
|
57
|
+
=======
|
58
|
+
|
59
|
+
Automatic Ruby parses configuration file that was written
|
60
|
+
in the form of YAML which including variety of information
|
61
|
+
of associated plug-ins.
|
62
|
+
|
63
|
+
This YAML file is called "Recipe".
|
64
|
+
|
65
|
+
When you start automatic ruby without argument -c option,
|
66
|
+
the config/default.yml is called. You can use -c option for
|
67
|
+
specify a file name.
|
68
|
+
|
69
|
+
[Example]
|
70
|
+
$ automatic -c ~/recipes/your_recipe.yml
|
71
|
+
|
72
|
+
|
73
|
+
=====================
|
74
|
+
How to write a Recipe
|
75
|
+
=====================
|
76
|
+
|
77
|
+
The Recipe has an implicit naming convention.
|
78
|
+
|
79
|
+
[Syntax]
|
80
|
+
|
81
|
+
plugins:
|
82
|
+
- module: MODULE_NAME
|
83
|
+
config:
|
84
|
+
VARIABLES
|
85
|
+
|
86
|
+
|
87
|
+
The following is a example.
|
88
|
+
|
89
|
+
plugins:
|
90
|
+
- module: SubscriptionFeed
|
91
|
+
config:
|
92
|
+
feeds:
|
93
|
+
- http://reretlet.tumblr.com/rss
|
94
|
+
|
95
|
+
- module: StorePermalink
|
96
|
+
config:
|
97
|
+
db: tumblr.db
|
98
|
+
|
99
|
+
- module: FilterImage
|
100
|
+
|
101
|
+
- module: FilterTumblrResize
|
102
|
+
|
103
|
+
- module: StoreTargetLink
|
104
|
+
config:
|
105
|
+
path: /Users/yourname/Desktop/
|
106
|
+
interval: 1
|
107
|
+
|
108
|
+
In the case of sample this recipe.
|
109
|
+
1. Subscribe to RSS feeds of Tumblr by SubScriptionFeed.
|
110
|
+
2. Save permalink to database by StorePermalink.
|
111
|
+
3. Specify the URL of the image by FilterImage and FilterTumblrResize.
|
112
|
+
4. Downloading image file of Tumblr by StoreTargetLink.
|
113
|
+
|
114
|
+
|
115
|
+
Showing another example as follows.
|
116
|
+
|
117
|
+
plugins:
|
118
|
+
- module: SubscriptionFeed
|
119
|
+
config:
|
120
|
+
feeds:
|
121
|
+
- http://example.com/rss2
|
122
|
+
- http://hogefuga.com/feed
|
123
|
+
|
124
|
+
- module: FilterIgnore
|
125
|
+
config:
|
126
|
+
link:
|
127
|
+
- hoge
|
128
|
+
- fuga
|
129
|
+
|
130
|
+
- module: StorePermalink
|
131
|
+
config:
|
132
|
+
db: permalink.db
|
133
|
+
|
134
|
+
- module: PublishHatenaBookmark
|
135
|
+
config:
|
136
|
+
username: your_hatena_id
|
137
|
+
password: your_password
|
138
|
+
interval: 5
|
139
|
+
|
140
|
+
In the case of sample this recipe.
|
141
|
+
1. Subscribe to 2 RSS feeds by SubScriptionFeed.
|
142
|
+
2. Feed ignore the keyword by FilterIgnore.
|
143
|
+
3. Save permalink to database by StorePermalink.
|
144
|
+
4. Social Bookmarking by PublishHatenaBookmark.
|
145
|
+
|
146
|
+
(Hatena Bookmark is a most popular social bookmark service
|
147
|
+
in Japan.)
|
148
|
+
|
149
|
+
|
150
|
+
These have been realized by a combination of plug-ins
|
151
|
+
a series of processes.
|
152
|
+
|
153
|
+
In this way, by simply writing to the recipe and
|
154
|
+
configuration information for plug-ins, free processing can
|
155
|
+
be achieved by a combination of multiple plug-ins.
|
156
|
+
|
157
|
+
The possibility of infinite depending on the combination
|
158
|
+
of plug-ins can be realized.
|
159
|
+
|
160
|
+
All depending on whether you make a plug-in looks like!
|
161
|
+
|
162
|
+
|
163
|
+
=======
|
164
|
+
Plug-in
|
165
|
+
=======
|
166
|
+
|
167
|
+
Plug-ins from the plugins directory (see below) will be loaded.
|
168
|
+
|
169
|
+
If ~ /.automatic/plugins exists, is loaded into them as well.
|
170
|
+
|
171
|
+
Put your own plug-ins in your home directory.
|
172
|
+
(How to make the plug-in will be described later.)
|
173
|
+
|
174
|
+
You can automatically generate a sub-directory for your own plug-ins
|
175
|
+
with the following command.
|
176
|
+
|
177
|
+
$ automatic-config scaffold
|
178
|
+
(For a description of automatic-config will be described later.)
|
179
|
+
|
180
|
+
|
181
|
+
============================
|
182
|
+
Directory and file structure
|
183
|
+
============================
|
184
|
+
.
|
185
|
+
|
|
186
|
+
+-+ bin
|
187
|
+
| |
|
188
|
+
| +-- automatic
|
189
|
+
| | The main file for run.
|
190
|
+
| |
|
191
|
+
| +-- automatic-config
|
192
|
+
| The tool for the auxiliary.
|
193
|
+
|
|
194
|
+
+-+ config
|
195
|
+
| |
|
196
|
+
| +-- default.yml
|
197
|
+
| To describe the recipe information in the form of yaml.
|
198
|
+
| They called in the argument -c option of Automatic Ruby.
|
199
|
+
| If you write a new recipe, locate it in the config directory.
|
200
|
+
|
|
201
|
+
|
|
202
|
+
+-+ plugins
|
203
|
+
| |
|
204
|
+
| +-- subscription
|
205
|
+
| | Plug-ins to subscribe to feeds.
|
206
|
+
| |
|
207
|
+
| +-- customfeed
|
208
|
+
| | Plug-ins to generate a custom feed.
|
209
|
+
| |
|
210
|
+
| +-- filter
|
211
|
+
| | Plug-ins to filter the information.
|
212
|
+
| |
|
213
|
+
| +-- store
|
214
|
+
| | Plug-ins to store the information to internally.
|
215
|
+
| |
|
216
|
+
| +-- notify
|
217
|
+
| | Plug-ins to notify the information.
|
218
|
+
| |
|
219
|
+
| +-- publish
|
220
|
+
| Plug-ins to send information to external.
|
221
|
+
|
|
222
|
+
| If you write a new plug-in, locate it in the plugins directory.
|
223
|
+
|
|
224
|
+
|
|
225
|
+
+-+ lib
|
226
|
+
| |
|
227
|
+
| +-- automatic.rb
|
228
|
+
| | To the definition of automatic module.
|
229
|
+
| |
|
230
|
+
| +-+ automatic
|
231
|
+
| | |
|
232
|
+
| | +-- environment.rb
|
233
|
+
| | | Read the environmental information.
|
234
|
+
| | |
|
235
|
+
| | +-- pipeline.rb
|
236
|
+
| | | To use an object called a pipeline
|
237
|
+
| | | sequentially processing plug-ins in recipes.
|
238
|
+
| | |
|
239
|
+
| | +-- feed_parser.rb
|
240
|
+
| | | To parse the feed.
|
241
|
+
| | |
|
242
|
+
| | +-- log.rb
|
243
|
+
| | | To output logs.
|
244
|
+
| | |
|
245
|
+
| | +-- recipe.rb
|
246
|
+
| | Read a recipes.
|
247
|
+
| |
|
248
|
+
| +-- config
|
249
|
+
|
|
250
|
+
+-+ db
|
251
|
+
| |
|
252
|
+
| +-- permalink.db
|
253
|
+
| Have been collected in the plug-in Store::Permalink.
|
254
|
+
|
|
255
|
+
+-+ script
|
256
|
+
| |
|
257
|
+
| +-- bootstrap
|
258
|
+
| To set the environment, such as bundle install.
|
259
|
+
| (Only more than ruby 1.9)
|
260
|
+
|
|
261
|
+
+-- spec
|
262
|
+
| Directory for unit tests (Use RSpec).
|
263
|
+
|
|
264
|
+
+-+ test
|
265
|
+
| |
|
266
|
+
| +-- integration
|
267
|
+
| Directory for the Integration Test.
|
268
|
+
|
|
269
|
+
+-+ vendor
|
270
|
+
| |
|
271
|
+
| +-- gems
|
272
|
+
| Bundle installed gem packages.
|
273
|
+
|
|
274
|
+
+---+ doc
|
275
|
+
|
|
276
|
+
+-- COPYING
|
277
|
+
| The license for this software.
|
278
|
+
|
|
279
|
+
+-- PLUGINS
|
280
|
+
| Documentation for plug-ins.
|
281
|
+
|
|
282
|
+
+-- README
|
283
|
+
This document.
|
284
|
+
|
285
|
+
|
286
|
+
|
287
|
+
===========
|
288
|
+
Development
|
289
|
+
===========
|
290
|
+
|
291
|
+
Repository
|
292
|
+
https://github.com/id774/automaticruby
|
293
|
+
|
294
|
+
Issues
|
295
|
+
https://github.com/id774/automaticruby/issues
|
296
|
+
|
297
|
+
RubyForge
|
298
|
+
http://rubyforge.org/projects/automatic/
|
299
|
+
|
300
|
+
CI
|
301
|
+
http://id774.net/jenkins/
|
302
|
+
|
303
|
+
|
304
|
+
===========
|
305
|
+
How to Join
|
306
|
+
===========
|
307
|
+
|
308
|
+
1. Fork the repository.
|
309
|
+
2. Write new plugin or existing code improvement.
|
310
|
+
3. Send pull request!
|
311
|
+
|
312
|
+
After that, we will give you the right to commit for a quick fix finely.
|
313
|
+
|
314
|
+
|
315
|
+
===========
|
316
|
+
Coding Rule
|
317
|
+
===========
|
318
|
+
|
319
|
+
2 tabs (Soft Tabs).
|
320
|
+
Remove trailing spaces.
|
321
|
+
Namespace and conventions, see the existing code.
|
322
|
+
{} is recommended than do end (To avoid end end end,,).
|
323
|
+
RDoc Header is written by creator of the file.
|
324
|
+
Write tests with RSPec.
|
325
|
+
Aim at 100% coverage.
|
326
|
+
|
327
|
+
|
328
|
+
===================
|
329
|
+
Automatic::Pipeline
|
330
|
+
===================
|
331
|
+
|
332
|
+
This framework sequentially analyze a plug-ins on yaml recipe.
|
333
|
+
At this time, pipeline passed instantance as an argument. And then,
|
334
|
+
pipeline backs again as the return value.
|
335
|
+
|
336
|
+
Code is as follows.
|
337
|
+
|
338
|
+
pipeline = []
|
339
|
+
recipe.each_plugin { |plugin|
|
340
|
+
load_plugin(plugin.module)
|
341
|
+
klass = Automatic::Plugin.const_get(plugin.module)
|
342
|
+
pipeline = klass.new(plugin.config, pipeline).run
|
343
|
+
}
|
344
|
+
|
345
|
+
In this iteration, sequential processing can be realized.
|
346
|
+
|
347
|
+
This mechanism called "Automatic::Pipeline".
|
348
|
+
|
349
|
+
|
350
|
+
|
351
|
+
=============================
|
352
|
+
Implementation of the plug-in
|
353
|
+
=============================
|
354
|
+
|
355
|
+
Plug-in is loaded by the constructor of YAML.load
|
356
|
+
Class is recieved the array of hashes as the instance
|
357
|
+
variable of pipelines object.
|
358
|
+
|
359
|
+
Example implementation of the constructor that is recommended
|
360
|
+
is as follows.
|
361
|
+
|
362
|
+
def initialize(config, pipeline=[])
|
363
|
+
@config = config
|
364
|
+
@pipeline = pipeline
|
365
|
+
end
|
366
|
+
|
367
|
+
Run an instance method is called automatically. The return value
|
368
|
+
is on the instance variable @pipeline. And then, be handed over
|
369
|
+
to the next plug-in in the recipe.
|
370
|
+
|
371
|
+
|
372
|
+
=========
|
373
|
+
Unit Test
|
374
|
+
=========
|
375
|
+
|
376
|
+
Unit testing is performed in RSpec.
|
377
|
+
|
378
|
+
Test of the plug-in, pass the pipeline which will be the argument,
|
379
|
+
to check the return value.
|
380
|
+
|
381
|
+
|
382
|
+
============
|
383
|
+
Binding Test
|
384
|
+
============
|
385
|
+
|
386
|
+
To combine more than one recipe under the test/integration.
|
387
|
+
Put the recipe for the binding test.
|
388
|
+
|
389
|
+
Before you check-in to the repository, run following
|
390
|
+
command with ruby 1.9.
|
391
|
+
$ script/build
|
392
|
+
|
393
|
+
By this, all RSpec tests and binding tests will be conducted.
|
394
|
+
You can check-in to the repository after all test successed.
|
395
|
+
Otherwise, CI will fail.
|
396
|
+
|
397
|
+
|
398
|
+
======================
|
399
|
+
Continuous Integration
|
400
|
+
======================
|
401
|
+
|
402
|
+
CI is performed in Jenkins.
|
403
|
+
http://id774.net/jenkins/
|
404
|
+
|
405
|
+
|
406
|
+
===========================
|
407
|
+
Description of the plug-ins
|
408
|
+
===========================
|
409
|
+
|
410
|
+
See doc/PLUGINS.
|
411
|
+
|
412
|
+
|
413
|
+
|
414
|
+
================
|
415
|
+
automatic-config
|
416
|
+
================
|
417
|
+
|
418
|
+
automatic-config is the auxiliary tool.
|
419
|
+
|
420
|
+
[Syntax]
|
421
|
+
$ automatic-config
|
422
|
+
To view a list of sub-command.
|
423
|
+
|
424
|
+
$ automatic-config scaffold
|
425
|
+
To generate a ~/.automatic directory in your home directory.
|
426
|
+
|
427
|
+
$ automatic-config autodiscovery <url>
|
428
|
+
Return the URL of the feed of target detected by auto discovery.
|
429
|
+
(ex.)
|
430
|
+
$ automatic-config autodiscovery http://id774.net/blog
|
431
|
+
["http://id774.net/blog/feed/", "http://id774.net/blog/comments/feed/"]
|
432
|
+
|
433
|
+
$ automatic-config opmlparser <opml path>
|
434
|
+
To output the URLs to parse the OPML file.
|
435
|
+
(ex.)
|
436
|
+
$ automatic-config opmlparser opml.xml > feeds.txt
|
437
|
+
|
438
|
+
$ automatic-config feedparser <url>
|
439
|
+
Return the contents to parse the feed.
|
440
|
+
|
441
|
+
$ automatic-config log <level> <message>
|
442
|
+
To output log messages in the form of Automatic Ruby.
|
443
|
+
|
444
|
+
|
445
|
+
|
446
|
+
==============
|
447
|
+
Update History
|
448
|
+
==============
|
449
|
+
|
450
|
+
See doc/ChangeLog.
|
451
|
+
|
452
|
+
|
453
|
+
====
|
454
|
+
TODO
|
455
|
+
====
|
456
|
+
|
457
|
+
Refer to the issues of github
|
458
|
+
https://github.com/id774/automaticruby/issues
|
459
|
+
|
460
|
+
|
461
|
+
===========
|
462
|
+
Environment
|
463
|
+
===========
|
464
|
+
|
465
|
+
ruby 1.8 or later
|
466
|
+
|
467
|
+
|
468
|
+
The following packages depend on the gem.
|
469
|
+
|
470
|
+
sqlite3
|
471
|
+
activesupport
|
472
|
+
hashie
|
473
|
+
activerecord
|
474
|
+
gcalapi
|
475
|
+
xml-simple
|
476
|
+
feedbag
|
477
|
+
|
478
|
+
For more information, see Gemfile.
|
479
|
+
|
480
|
+
|
481
|
+
=====
|
482
|
+
Notes
|
483
|
+
=====
|
484
|
+
|
485
|
+
To avoid excessive scraping, use in the bounds of common sense.
|
486
|
+
This software licensed under the GNU GENERAL PUBLIC LICENSE, Version 3.0.
|
487
|
+
|
488
|
+
|