automatic 12.3.0 → 12.3.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (52) hide show
  1. data/Gemfile +1 -0
  2. data/README.md +54 -36
  3. data/Rakefile +14 -0
  4. data/VERSION +1 -1
  5. data/automatic.gemspec +17 -5
  6. data/bin/automatic +37 -3
  7. data/bin/automatic-config +77 -0
  8. data/config/default.yml +9 -12
  9. data/doc/ChangeLog +32 -8
  10. data/doc/PLUGINS +205 -0
  11. data/doc/PLUGINS.ja +2 -3
  12. data/doc/README +488 -0
  13. data/doc/README.ja +195 -131
  14. data/lib/automatic/feed_parser.rb +1 -9
  15. data/lib/automatic/log.rb +1 -9
  16. data/lib/automatic/opml.rb +239 -0
  17. data/lib/automatic/pipeline.rb +16 -10
  18. data/lib/automatic/recipe.rb +3 -4
  19. data/lib/automatic.rb +32 -38
  20. data/lib/config/validator.rb +83 -0
  21. data/plugins/custom_feed/svn_log.rb +1 -1
  22. data/plugins/filter/ignore.rb +9 -1
  23. data/plugins/notify/ikachan.rb +7 -6
  24. data/plugins/publish/hatena_bookmark.rb +6 -9
  25. data/script/build +63 -0
  26. data/spec/lib/automatic/pipeline_spec.rb +55 -0
  27. data/spec/lib/automatic_spec.rb +77 -0
  28. data/spec/lib/pipeline_spec.rb +67 -0
  29. data/spec/plugins/filter/ignore_spec.rb +16 -0
  30. data/spec/plugins/filter/image_spec.rb +4 -4
  31. data/spec/plugins/filter/tumblr_resize_spec.rb +4 -4
  32. data/spec/plugins/notify/ikachan_spec.rb +30 -0
  33. data/spec/plugins/publish/console_spec.rb +1 -2
  34. data/spec/plugins/publish/hatena_bookmark_spec.rb +36 -1
  35. data/spec/plugins/store/full_text_spec.rb +0 -2
  36. data/spec/plugins/store/permalink_spec.rb +0 -1
  37. data/spec/plugins/store/target_link_spec.rb +0 -1
  38. data/spec/plugins/subscription/feed_spec.rb +0 -1
  39. data/spec/spec_helper.rb +6 -4
  40. data/spec/user_dir/plugins/store/mock.rb +12 -0
  41. data/test/fixtures/sampleOPML.xml +11 -0
  42. data/test/integration/test_activerecord.yml +2 -2
  43. data/test/integration/test_fulltext.yml +3 -3
  44. data/test/integration/test_hatenabookmark.yml +6 -2
  45. data/test/integration/test_ignore.yml +4 -1
  46. data/test/integration/test_ignore2.yml +1 -4
  47. data/test/integration/test_image2local.yml +3 -5
  48. data/test/integration/test_svnlog.yml +2 -1
  49. data/test/integration/test_tumblr2local.yml +3 -3
  50. metadata +43 -22
  51. data/utils/auto_discovery.rb +0 -18
  52. data/utils/opml_parser.rb +0 -247
data/doc/README ADDED
@@ -0,0 +1,488 @@
1
+ automaticruby
2
+
3
+ Name
4
+ Automatic Ruby - Ruby General Automation Framework
5
+
6
+ Syntax
7
+ $ automatic
8
+
9
+ Specify the start of any recipe
10
+ $ automatic -c <recipe>
11
+
12
+ To view the version number
13
+ $ automatic -v
14
+
15
+ Description
16
+ This is a general-purpose automatic processing
17
+ ruby framework which can extend the functionality
18
+ by plug-ins.
19
+
20
+
21
+ ============
22
+ Installation
23
+ ============
24
+
25
+ [Stable]
26
+ $ gem install automatic
27
+
28
+ [Development]
29
+ $ git clone git://github.com/id774/automaticruby.git
30
+
31
+
32
+ If the ruby 1.9, put following command.
33
+ $ script/bootstrap
34
+
35
+ If the ruby 1.8, necessary to manually install the gems in the Gemfile.
36
+
37
+
38
+ ===========
39
+ Get Started
40
+ ===========
41
+
42
+ To create of any recipe (see below).
43
+ $ automatic -c your_recipe.yml
44
+
45
+ -c option to run with the manual. If there is no problem
46
+ to automate, register it to cron.
47
+
48
+
49
+ This is the easy recipe to simply output blog's feed to
50
+ only console. It can be used to check the operation of
51
+ this software.
52
+ $ automatic -c config/feed2console.yml
53
+
54
+
55
+ =======
56
+ Recipes
57
+ =======
58
+
59
+ Automatic Ruby parses configuration file that was written
60
+ in the form of YAML which including variety of information
61
+ of associated plug-ins.
62
+
63
+ This YAML file is called "Recipe".
64
+
65
+ When you start automatic ruby without argument -c option,
66
+ the config/default.yml is called. You can use -c option for
67
+ specify a file name.
68
+
69
+ [Example]
70
+ $ automatic -c ~/recipes/your_recipe.yml
71
+
72
+
73
+ =====================
74
+ How to write a Recipe
75
+ =====================
76
+
77
+ The Recipe has an implicit naming convention.
78
+
79
+ [Syntax]
80
+
81
+ plugins:
82
+ - module: MODULE_NAME
83
+ config:
84
+ VARIABLES
85
+
86
+
87
+ The following is a example.
88
+
89
+ plugins:
90
+ - module: SubscriptionFeed
91
+ config:
92
+ feeds:
93
+ - http://reretlet.tumblr.com/rss
94
+
95
+ - module: StorePermalink
96
+ config:
97
+ db: tumblr.db
98
+
99
+ - module: FilterImage
100
+
101
+ - module: FilterTumblrResize
102
+
103
+ - module: StoreTargetLink
104
+ config:
105
+ path: /Users/yourname/Desktop/
106
+ interval: 1
107
+
108
+ In the case of sample this recipe.
109
+ 1. Subscribe to RSS feeds of Tumblr by SubScriptionFeed.
110
+ 2. Save permalink to database by StorePermalink.
111
+ 3. Specify the URL of the image by FilterImage and FilterTumblrResize.
112
+ 4. Downloading image file of Tumblr by StoreTargetLink.
113
+
114
+
115
+ Showing another example as follows.
116
+
117
+ plugins:
118
+ - module: SubscriptionFeed
119
+ config:
120
+ feeds:
121
+ - http://example.com/rss2
122
+ - http://hogefuga.com/feed
123
+
124
+ - module: FilterIgnore
125
+ config:
126
+ link:
127
+ - hoge
128
+ - fuga
129
+
130
+ - module: StorePermalink
131
+ config:
132
+ db: permalink.db
133
+
134
+ - module: PublishHatenaBookmark
135
+ config:
136
+ username: your_hatena_id
137
+ password: your_password
138
+ interval: 5
139
+
140
+ In the case of sample this recipe.
141
+ 1. Subscribe to 2 RSS feeds by SubScriptionFeed.
142
+ 2. Feed ignore the keyword by FilterIgnore.
143
+ 3. Save permalink to database by StorePermalink.
144
+ 4. Social Bookmarking by PublishHatenaBookmark.
145
+
146
+ (Hatena Bookmark is a most popular social bookmark service
147
+ in Japan.)
148
+
149
+
150
+ These have been realized by a combination of plug-ins
151
+ a series of processes.
152
+
153
+ In this way, by simply writing to the recipe and
154
+ configuration information for plug-ins, free processing can
155
+ be achieved by a combination of multiple plug-ins.
156
+
157
+ The possibility of infinite depending on the combination
158
+ of plug-ins can be realized.
159
+
160
+ All depending on whether you make a plug-in looks like!
161
+
162
+
163
+ =======
164
+ Plug-in
165
+ =======
166
+
167
+ Plug-ins from the plugins directory (see below) will be loaded.
168
+
169
+ If ~ /.automatic/plugins exists, is loaded into them as well.
170
+
171
+ Put your own plug-ins in your home directory.
172
+ (How to make the plug-in will be described later.)
173
+
174
+ You can automatically generate a sub-directory for your own plug-ins
175
+ with the following command.
176
+
177
+ $ automatic-config scaffold
178
+ (For a description of automatic-config will be described later.)
179
+
180
+
181
+ ============================
182
+ Directory and file structure
183
+ ============================
184
+ .
185
+ |
186
+ +-+ bin
187
+ | |
188
+ | +-- automatic
189
+ | | The main file for run.
190
+ | |
191
+ | +-- automatic-config
192
+ | The tool for the auxiliary.
193
+ |
194
+ +-+ config
195
+ | |
196
+ | +-- default.yml
197
+ | To describe the recipe information in the form of yaml.
198
+ | They called in the argument -c option of Automatic Ruby.
199
+ | If you write a new recipe, locate it in the config directory.
200
+ |
201
+ |
202
+ +-+ plugins
203
+ | |
204
+ | +-- subscription
205
+ | | Plug-ins to subscribe to feeds.
206
+ | |
207
+ | +-- customfeed
208
+ | | Plug-ins to generate a custom feed.
209
+ | |
210
+ | +-- filter
211
+ | | Plug-ins to filter the information.
212
+ | |
213
+ | +-- store
214
+ | | Plug-ins to store the information to internally.
215
+ | |
216
+ | +-- notify
217
+ | | Plug-ins to notify the information.
218
+ | |
219
+ | +-- publish
220
+ | Plug-ins to send information to external.
221
+ |
222
+ | If you write a new plug-in, locate it in the plugins directory.
223
+ |
224
+ |
225
+ +-+ lib
226
+ | |
227
+ | +-- automatic.rb
228
+ | | To the definition of automatic module.
229
+ | |
230
+ | +-+ automatic
231
+ | | |
232
+ | | +-- environment.rb
233
+ | | | Read the environmental information.
234
+ | | |
235
+ | | +-- pipeline.rb
236
+ | | | To use an object called a pipeline
237
+ | | | sequentially processing plug-ins in recipes.
238
+ | | |
239
+ | | +-- feed_parser.rb
240
+ | | | To parse the feed.
241
+ | | |
242
+ | | +-- log.rb
243
+ | | | To output logs.
244
+ | | |
245
+ | | +-- recipe.rb
246
+ | | Read a recipes.
247
+ | |
248
+ | +-- config
249
+ |
250
+ +-+ db
251
+ | |
252
+ | +-- permalink.db
253
+ | Have been collected in the plug-in Store::Permalink.
254
+ |
255
+ +-+ script
256
+ | |
257
+ | +-- bootstrap
258
+ | To set the environment, such as bundle install.
259
+ | (Only more than ruby 1.9)
260
+ |
261
+ +-- spec
262
+ | Directory for unit tests (Use RSpec).
263
+ |
264
+ +-+ test
265
+ | |
266
+ | +-- integration
267
+ | Directory for the Integration Test.
268
+ |
269
+ +-+ vendor
270
+ | |
271
+ | +-- gems
272
+ | Bundle installed gem packages.
273
+ |
274
+ +---+ doc
275
+ |
276
+ +-- COPYING
277
+ | The license for this software.
278
+ |
279
+ +-- PLUGINS
280
+ | Documentation for plug-ins.
281
+ |
282
+ +-- README
283
+ This document.
284
+
285
+
286
+
287
+ ===========
288
+ Development
289
+ ===========
290
+
291
+ Repository
292
+ https://github.com/id774/automaticruby
293
+
294
+ Issues
295
+ https://github.com/id774/automaticruby/issues
296
+
297
+ RubyForge
298
+ http://rubyforge.org/projects/automatic/
299
+
300
+ CI
301
+ http://id774.net/jenkins/
302
+
303
+
304
+ ===========
305
+ How to Join
306
+ ===========
307
+
308
+ 1. Fork the repository.
309
+ 2. Write new plugin or existing code improvement.
310
+ 3. Send pull request!
311
+
312
+ After that, we will give you the right to commit for a quick fix finely.
313
+
314
+
315
+ ===========
316
+ Coding Rule
317
+ ===========
318
+
319
+ 2 tabs (Soft Tabs).
320
+ Remove trailing spaces.
321
+ Namespace and conventions, see the existing code.
322
+ {} is recommended than do end (To avoid end end end,,).
323
+ RDoc Header is written by creator of the file.
324
+ Write tests with RSPec.
325
+ Aim at 100% coverage.
326
+
327
+
328
+ ===================
329
+ Automatic::Pipeline
330
+ ===================
331
+
332
+ This framework sequentially analyze a plug-ins on yaml recipe.
333
+ At this time, pipeline passed instantance as an argument. And then,
334
+ pipeline backs again as the return value.
335
+
336
+ Code is as follows.
337
+
338
+ pipeline = []
339
+ recipe.each_plugin { |plugin|
340
+ load_plugin(plugin.module)
341
+ klass = Automatic::Plugin.const_get(plugin.module)
342
+ pipeline = klass.new(plugin.config, pipeline).run
343
+ }
344
+
345
+ In this iteration, sequential processing can be realized.
346
+
347
+ This mechanism called "Automatic::Pipeline".
348
+
349
+
350
+
351
+ =============================
352
+ Implementation of the plug-in
353
+ =============================
354
+
355
+ Plug-in is loaded by the constructor of YAML.load
356
+ Class is recieved the array of hashes as the instance
357
+ variable of pipelines object.
358
+
359
+ Example implementation of the constructor that is recommended
360
+ is as follows.
361
+
362
+ def initialize(config, pipeline=[])
363
+ @config = config
364
+ @pipeline = pipeline
365
+ end
366
+
367
+ Run an instance method is called automatically. The return value
368
+ is on the instance variable @pipeline. And then, be handed over
369
+ to the next plug-in in the recipe.
370
+
371
+
372
+ =========
373
+ Unit Test
374
+ =========
375
+
376
+ Unit testing is performed in RSpec.
377
+
378
+ Test of the plug-in, pass the pipeline which will be the argument,
379
+ to check the return value.
380
+
381
+
382
+ ============
383
+ Binding Test
384
+ ============
385
+
386
+ To combine more than one recipe under the test/integration.
387
+ Put the recipe for the binding test.
388
+
389
+ Before you check-in to the repository, run following
390
+ command with ruby 1.9.
391
+ $ script/build
392
+
393
+ By this, all RSpec tests and binding tests will be conducted.
394
+ You can check-in to the repository after all test successed.
395
+ Otherwise, CI will fail.
396
+
397
+
398
+ ======================
399
+ Continuous Integration
400
+ ======================
401
+
402
+ CI is performed in Jenkins.
403
+ http://id774.net/jenkins/
404
+
405
+
406
+ ===========================
407
+ Description of the plug-ins
408
+ ===========================
409
+
410
+ See doc/PLUGINS.
411
+
412
+
413
+
414
+ ================
415
+ automatic-config
416
+ ================
417
+
418
+ automatic-config is the auxiliary tool.
419
+
420
+ [Syntax]
421
+ $ automatic-config
422
+ To view a list of sub-command.
423
+
424
+ $ automatic-config scaffold
425
+ To generate a ~/.automatic directory in your home directory.
426
+
427
+ $ automatic-config autodiscovery <url>
428
+ Return the URL of the feed of target detected by auto discovery.
429
+ (ex.)
430
+ $ automatic-config autodiscovery http://id774.net/blog
431
+ ["http://id774.net/blog/feed/", "http://id774.net/blog/comments/feed/"]
432
+
433
+ $ automatic-config opmlparser <opml path>
434
+ To output the URLs to parse the OPML file.
435
+ (ex.)
436
+ $ automatic-config opmlparser opml.xml > feeds.txt
437
+
438
+ $ automatic-config feedparser <url>
439
+ Return the contents to parse the feed.
440
+
441
+ $ automatic-config log <level> <message>
442
+ To output log messages in the form of Automatic Ruby.
443
+
444
+
445
+
446
+ ==============
447
+ Update History
448
+ ==============
449
+
450
+ See doc/ChangeLog.
451
+
452
+
453
+ ====
454
+ TODO
455
+ ====
456
+
457
+ Refer to the issues of github
458
+ https://github.com/id774/automaticruby/issues
459
+
460
+
461
+ ===========
462
+ Environment
463
+ ===========
464
+
465
+ ruby 1.8 or later
466
+
467
+
468
+ The following packages depend on the gem.
469
+
470
+ sqlite3
471
+ activesupport
472
+ hashie
473
+ activerecord
474
+ gcalapi
475
+ xml-simple
476
+ feedbag
477
+
478
+ For more information, see Gemfile.
479
+
480
+
481
+ =====
482
+ Notes
483
+ =====
484
+
485
+ To avoid excessive scraping, use in the bounds of common sense.
486
+ This software licensed under the GNU GENERAL PUBLIC LICENSE, Version 3.0.
487
+
488
+