automatic 12.3.0 → 12.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. data/Gemfile +1 -0
  2. data/README.md +54 -36
  3. data/Rakefile +14 -0
  4. data/VERSION +1 -1
  5. data/automatic.gemspec +17 -5
  6. data/bin/automatic +37 -3
  7. data/bin/automatic-config +77 -0
  8. data/config/default.yml +9 -12
  9. data/doc/ChangeLog +32 -8
  10. data/doc/PLUGINS +205 -0
  11. data/doc/PLUGINS.ja +2 -3
  12. data/doc/README +488 -0
  13. data/doc/README.ja +195 -131
  14. data/lib/automatic/feed_parser.rb +1 -9
  15. data/lib/automatic/log.rb +1 -9
  16. data/lib/automatic/opml.rb +239 -0
  17. data/lib/automatic/pipeline.rb +16 -10
  18. data/lib/automatic/recipe.rb +3 -4
  19. data/lib/automatic.rb +32 -38
  20. data/lib/config/validator.rb +83 -0
  21. data/plugins/custom_feed/svn_log.rb +1 -1
  22. data/plugins/filter/ignore.rb +9 -1
  23. data/plugins/notify/ikachan.rb +7 -6
  24. data/plugins/publish/hatena_bookmark.rb +6 -9
  25. data/script/build +63 -0
  26. data/spec/lib/automatic/pipeline_spec.rb +55 -0
  27. data/spec/lib/automatic_spec.rb +77 -0
  28. data/spec/lib/pipeline_spec.rb +67 -0
  29. data/spec/plugins/filter/ignore_spec.rb +16 -0
  30. data/spec/plugins/filter/image_spec.rb +4 -4
  31. data/spec/plugins/filter/tumblr_resize_spec.rb +4 -4
  32. data/spec/plugins/notify/ikachan_spec.rb +30 -0
  33. data/spec/plugins/publish/console_spec.rb +1 -2
  34. data/spec/plugins/publish/hatena_bookmark_spec.rb +36 -1
  35. data/spec/plugins/store/full_text_spec.rb +0 -2
  36. data/spec/plugins/store/permalink_spec.rb +0 -1
  37. data/spec/plugins/store/target_link_spec.rb +0 -1
  38. data/spec/plugins/subscription/feed_spec.rb +0 -1
  39. data/spec/spec_helper.rb +6 -4
  40. data/spec/user_dir/plugins/store/mock.rb +12 -0
  41. data/test/fixtures/sampleOPML.xml +11 -0
  42. data/test/integration/test_activerecord.yml +2 -2
  43. data/test/integration/test_fulltext.yml +3 -3
  44. data/test/integration/test_hatenabookmark.yml +6 -2
  45. data/test/integration/test_ignore.yml +4 -1
  46. data/test/integration/test_ignore2.yml +1 -4
  47. data/test/integration/test_image2local.yml +3 -5
  48. data/test/integration/test_svnlog.yml +2 -1
  49. data/test/integration/test_tumblr2local.yml +3 -3
  50. metadata +43 -22
  51. data/utils/auto_discovery.rb +0 -18
  52. data/utils/opml_parser.rb +0 -247
data/doc/README ADDED
@@ -0,0 +1,488 @@
1
+ automaticruby
2
+
3
+ Name
4
+ Automatic Ruby - Ruby General Automation Framework
5
+
6
+ Syntax
7
+ $ automatic
8
+
9
+ Specify the start of any recipe
10
+ $ automatic -c <recipe>
11
+
12
+ To view the version number
13
+ $ automatic -v
14
+
15
+ Description
16
+ This is a general-purpose automatic processing
17
+ ruby framework which can extend the functionality
18
+ by plug-ins.
19
+
20
+
21
+ ============
22
+ Installation
23
+ ============
24
+
25
+ [Stable]
26
+ $ gem install automatic
27
+
28
+ [Development]
29
+ $ git clone git://github.com/id774/automaticruby.git
30
+
31
+
32
+ If the ruby 1.9, put following command.
33
+ $ script/bootstrap
34
+
35
+ If the ruby 1.8, necessary to manually install the gems in the Gemfile.
36
+
37
+
38
+ ===========
39
+ Get Started
40
+ ===========
41
+
42
+ To create of any recipe (see below).
43
+ $ automatic -c your_recipe.yml
44
+
45
+ -c option to run with the manual. If there is no problem
46
+ to automate, register it to cron.
47
+
48
+
49
+ This is the easy recipe to simply output blog's feed to
50
+ only console. It can be used to check the operation of
51
+ this software.
52
+ $ automatic -c config/feed2console.yml
53
+
54
+
55
+ =======
56
+ Recipes
57
+ =======
58
+
59
+ Automatic Ruby parses configuration file that was written
60
+ in the form of YAML which including variety of information
61
+ of associated plug-ins.
62
+
63
+ This YAML file is called "Recipe".
64
+
65
+ When you start automatic ruby without argument -c option,
66
+ the config/default.yml is called. You can use -c option for
67
+ specify a file name.
68
+
69
+ [Example]
70
+ $ automatic -c ~/recipes/your_recipe.yml
71
+
72
+
73
+ =====================
74
+ How to write a Recipe
75
+ =====================
76
+
77
+ The Recipe has an implicit naming convention.
78
+
79
+ [Syntax]
80
+
81
+ plugins:
82
+ - module: MODULE_NAME
83
+ config:
84
+ VARIABLES
85
+
86
+
87
+ The following is a example.
88
+
89
+ plugins:
90
+ - module: SubscriptionFeed
91
+ config:
92
+ feeds:
93
+ - http://reretlet.tumblr.com/rss
94
+
95
+ - module: StorePermalink
96
+ config:
97
+ db: tumblr.db
98
+
99
+ - module: FilterImage
100
+
101
+ - module: FilterTumblrResize
102
+
103
+ - module: StoreTargetLink
104
+ config:
105
+ path: /Users/yourname/Desktop/
106
+ interval: 1
107
+
108
+ In the case of sample this recipe.
109
+ 1. Subscribe to RSS feeds of Tumblr by SubScriptionFeed.
110
+ 2. Save permalink to database by StorePermalink.
111
+ 3. Specify the URL of the image by FilterImage and FilterTumblrResize.
112
+ 4. Downloading image file of Tumblr by StoreTargetLink.
113
+
114
+
115
+ Showing another example as follows.
116
+
117
+ plugins:
118
+ - module: SubscriptionFeed
119
+ config:
120
+ feeds:
121
+ - http://example.com/rss2
122
+ - http://hogefuga.com/feed
123
+
124
+ - module: FilterIgnore
125
+ config:
126
+ link:
127
+ - hoge
128
+ - fuga
129
+
130
+ - module: StorePermalink
131
+ config:
132
+ db: permalink.db
133
+
134
+ - module: PublishHatenaBookmark
135
+ config:
136
+ username: your_hatena_id
137
+ password: your_password
138
+ interval: 5
139
+
140
+ In the case of sample this recipe.
141
+ 1. Subscribe to 2 RSS feeds by SubScriptionFeed.
142
+ 2. Feed ignore the keyword by FilterIgnore.
143
+ 3. Save permalink to database by StorePermalink.
144
+ 4. Social Bookmarking by PublishHatenaBookmark.
145
+
146
+ (Hatena Bookmark is a most popular social bookmark service
147
+ in Japan.)
148
+
149
+
150
+ These have been realized by a combination of plug-ins
151
+ a series of processes.
152
+
153
+ In this way, by simply writing to the recipe and
154
+ configuration information for plug-ins, free processing can
155
+ be achieved by a combination of multiple plug-ins.
156
+
157
+ The possibility of infinite depending on the combination
158
+ of plug-ins can be realized.
159
+
160
+ All depending on whether you make a plug-in looks like!
161
+
162
+
163
+ =======
164
+ Plug-in
165
+ =======
166
+
167
+ Plug-ins from the plugins directory (see below) will be loaded.
168
+
169
+ If ~ /.automatic/plugins exists, is loaded into them as well.
170
+
171
+ Put your own plug-ins in your home directory.
172
+ (How to make the plug-in will be described later.)
173
+
174
+ You can automatically generate a sub-directory for your own plug-ins
175
+ with the following command.
176
+
177
+ $ automatic-config scaffold
178
+ (For a description of automatic-config will be described later.)
179
+
180
+
181
+ ============================
182
+ Directory and file structure
183
+ ============================
184
+ .
185
+ |
186
+ +-+ bin
187
+ | |
188
+ | +-- automatic
189
+ | | The main file for run.
190
+ | |
191
+ | +-- automatic-config
192
+ | The tool for the auxiliary.
193
+ |
194
+ +-+ config
195
+ | |
196
+ | +-- default.yml
197
+ | To describe the recipe information in the form of yaml.
198
+ | They called in the argument -c option of Automatic Ruby.
199
+ | If you write a new recipe, locate it in the config directory.
200
+ |
201
+ |
202
+ +-+ plugins
203
+ | |
204
+ | +-- subscription
205
+ | | Plug-ins to subscribe to feeds.
206
+ | |
207
+ | +-- customfeed
208
+ | | Plug-ins to generate a custom feed.
209
+ | |
210
+ | +-- filter
211
+ | | Plug-ins to filter the information.
212
+ | |
213
+ | +-- store
214
+ | | Plug-ins to store the information to internally.
215
+ | |
216
+ | +-- notify
217
+ | | Plug-ins to notify the information.
218
+ | |
219
+ | +-- publish
220
+ | Plug-ins to send information to external.
221
+ |
222
+ | If you write a new plug-in, locate it in the plugins directory.
223
+ |
224
+ |
225
+ +-+ lib
226
+ | |
227
+ | +-- automatic.rb
228
+ | | To the definition of automatic module.
229
+ | |
230
+ | +-+ automatic
231
+ | | |
232
+ | | +-- environment.rb
233
+ | | | Read the environmental information.
234
+ | | |
235
+ | | +-- pipeline.rb
236
+ | | | To use an object called a pipeline
237
+ | | | sequentially processing plug-ins in recipes.
238
+ | | |
239
+ | | +-- feed_parser.rb
240
+ | | | To parse the feed.
241
+ | | |
242
+ | | +-- log.rb
243
+ | | | To output logs.
244
+ | | |
245
+ | | +-- recipe.rb
246
+ | | Read a recipes.
247
+ | |
248
+ | +-- config
249
+ |
250
+ +-+ db
251
+ | |
252
+ | +-- permalink.db
253
+ | Have been collected in the plug-in Store::Permalink.
254
+ |
255
+ +-+ script
256
+ | |
257
+ | +-- bootstrap
258
+ | To set the environment, such as bundle install.
259
+ | (Only more than ruby 1.9)
260
+ |
261
+ +-- spec
262
+ | Directory for unit tests (Use RSpec).
263
+ |
264
+ +-+ test
265
+ | |
266
+ | +-- integration
267
+ | Directory for the Integration Test.
268
+ |
269
+ +-+ vendor
270
+ | |
271
+ | +-- gems
272
+ | Bundle installed gem packages.
273
+ |
274
+ +---+ doc
275
+ |
276
+ +-- COPYING
277
+ | The license for this software.
278
+ |
279
+ +-- PLUGINS
280
+ | Documentation for plug-ins.
281
+ |
282
+ +-- README
283
+ This document.
284
+
285
+
286
+
287
+ ===========
288
+ Development
289
+ ===========
290
+
291
+ Repository
292
+ https://github.com/id774/automaticruby
293
+
294
+ Issues
295
+ https://github.com/id774/automaticruby/issues
296
+
297
+ RubyForge
298
+ http://rubyforge.org/projects/automatic/
299
+
300
+ CI
301
+ http://id774.net/jenkins/
302
+
303
+
304
+ ===========
305
+ How to Join
306
+ ===========
307
+
308
+ 1. Fork the repository.
309
+ 2. Write new plugin or existing code improvement.
310
+ 3. Send pull request!
311
+
312
+ After that, we will give you the right to commit for a quick fix finely.
313
+
314
+
315
+ ===========
316
+ Coding Rule
317
+ ===========
318
+
319
+ 2 tabs (Soft Tabs).
320
+ Remove trailing spaces.
321
+ Namespace and conventions, see the existing code.
322
+ {} is recommended than do end (To avoid end end end,,).
323
+ RDoc Header is written by creator of the file.
324
+ Write tests with RSPec.
325
+ Aim at 100% coverage.
326
+
327
+
328
+ ===================
329
+ Automatic::Pipeline
330
+ ===================
331
+
332
+ This framework sequentially analyze a plug-ins on yaml recipe.
333
+ At this time, pipeline passed instantance as an argument. And then,
334
+ pipeline backs again as the return value.
335
+
336
+ Code is as follows.
337
+
338
+ pipeline = []
339
+ recipe.each_plugin { |plugin|
340
+ load_plugin(plugin.module)
341
+ klass = Automatic::Plugin.const_get(plugin.module)
342
+ pipeline = klass.new(plugin.config, pipeline).run
343
+ }
344
+
345
+ In this iteration, sequential processing can be realized.
346
+
347
+ This mechanism called "Automatic::Pipeline".
348
+
349
+
350
+
351
+ =============================
352
+ Implementation of the plug-in
353
+ =============================
354
+
355
+ Plug-in is loaded by the constructor of YAML.load
356
+ Class is recieved the array of hashes as the instance
357
+ variable of pipelines object.
358
+
359
+ Example implementation of the constructor that is recommended
360
+ is as follows.
361
+
362
+ def initialize(config, pipeline=[])
363
+ @config = config
364
+ @pipeline = pipeline
365
+ end
366
+
367
+ Run an instance method is called automatically. The return value
368
+ is on the instance variable @pipeline. And then, be handed over
369
+ to the next plug-in in the recipe.
370
+
371
+
372
+ =========
373
+ Unit Test
374
+ =========
375
+
376
+ Unit testing is performed in RSpec.
377
+
378
+ Test of the plug-in, pass the pipeline which will be the argument,
379
+ to check the return value.
380
+
381
+
382
+ ============
383
+ Binding Test
384
+ ============
385
+
386
+ To combine more than one recipe under the test/integration.
387
+ Put the recipe for the binding test.
388
+
389
+ Before you check-in to the repository, run following
390
+ command with ruby 1.9.
391
+ $ script/build
392
+
393
+ By this, all RSpec tests and binding tests will be conducted.
394
+ You can check-in to the repository after all test successed.
395
+ Otherwise, CI will fail.
396
+
397
+
398
+ ======================
399
+ Continuous Integration
400
+ ======================
401
+
402
+ CI is performed in Jenkins.
403
+ http://id774.net/jenkins/
404
+
405
+
406
+ ===========================
407
+ Description of the plug-ins
408
+ ===========================
409
+
410
+ See doc/PLUGINS.
411
+
412
+
413
+
414
+ ================
415
+ automatic-config
416
+ ================
417
+
418
+ automatic-config is the auxiliary tool.
419
+
420
+ [Syntax]
421
+ $ automatic-config
422
+ To view a list of sub-command.
423
+
424
+ $ automatic-config scaffold
425
+ To generate a ~/.automatic directory in your home directory.
426
+
427
+ $ automatic-config autodiscovery <url>
428
+ Return the URL of the feed of target detected by auto discovery.
429
+ (ex.)
430
+ $ automatic-config autodiscovery http://id774.net/blog
431
+ ["http://id774.net/blog/feed/", "http://id774.net/blog/comments/feed/"]
432
+
433
+ $ automatic-config opmlparser <opml path>
434
+ To output the URLs to parse the OPML file.
435
+ (ex.)
436
+ $ automatic-config opmlparser opml.xml > feeds.txt
437
+
438
+ $ automatic-config feedparser <url>
439
+ Return the contents to parse the feed.
440
+
441
+ $ automatic-config log <level> <message>
442
+ To output log messages in the form of Automatic Ruby.
443
+
444
+
445
+
446
+ ==============
447
+ Update History
448
+ ==============
449
+
450
+ See doc/ChangeLog.
451
+
452
+
453
+ ====
454
+ TODO
455
+ ====
456
+
457
+ Refer to the issues of github
458
+ https://github.com/id774/automaticruby/issues
459
+
460
+
461
+ ===========
462
+ Environment
463
+ ===========
464
+
465
+ ruby 1.8 or later
466
+
467
+
468
+ The following packages depend on the gem.
469
+
470
+ sqlite3
471
+ activesupport
472
+ hashie
473
+ activerecord
474
+ gcalapi
475
+ xml-simple
476
+ feedbag
477
+
478
+ For more information, see Gemfile.
479
+
480
+
481
+ =====
482
+ Notes
483
+ =====
484
+
485
+ To avoid excessive scraping, use in the bounds of common sense.
486
+ This software licensed under the GNU GENERAL PUBLIC LICENSE, Version 3.0.
487
+
488
+