aws_pipes 0.0.5 → 0.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  ## Overview
2
2
 
3
- ### Communicating
3
+ ### Communication
4
4
 
5
5
  Send messages between Amazon EC2 instances through Unix pipes.
6
6
 
@@ -30,6 +30,18 @@ top of Amazon [SimpleDB](http://aws.amazon.com/simpledb/).
30
30
  - Save as much log history as you want, the storage is virtually
31
31
  unlimited.
32
32
 
33
+ ### Saving Datasets
34
+
35
+ Save data across EC2 instances with scalable throughput.
36
+ Data archival in aws_pipes is built on top of Amazon
37
+ [DynamoDB](http://aws.amazon.com/dynamodb/).
38
+
39
+ - Store data centrally.
40
+ - Automatically scale throughput and space.
41
+ - Query results (albeit not relationally).
42
+ - Monitor data acquisition through web control panel.
43
+ - Can export to S3.
44
+
33
45
  ## Usage
34
46
 
35
47
  ### aws_queue
@@ -72,6 +84,11 @@ ss-Substitution):
72
84
  # filling in columns a, b, and c
73
85
  your_program | aws_db foo a b c
74
86
 
87
+ DynamoDB tables have adjustable read- and write-throughput settings to
88
+ scale as needed. The `aws_db` command will automatically re-provision
89
+ write throughput if writing starts getting throttled. This makes
90
+ `aws_db` (when run in parallel) a way to save virtually unlimited
91
+ amounts of data as quickly as necessary.
75
92
 
76
93
  ## Installation
77
94
 
data/bin/aws_db CHANGED
@@ -32,8 +32,14 @@ db = AWS::DynamoDB.new(
32
32
  )
33
33
 
34
34
  $stdin.sync = true
35
- table = db.tables[table_name]
36
- table.load_schema
35
+ begin
36
+ table = db.tables[table_name]
37
+ table.load_schema
38
+ rescue AWS::DynamoDB::Errors::ResourceNotFoundException
39
+ warn "There is no DynamoDB table named \"#{table_name}\" for your AWS account."
40
+ warn "To create this and other tables, visit https://console.aws.amazon.com/dynamodb/"
41
+ exit 1
42
+ end
37
43
 
38
44
  while row = $stdin.gets
39
45
  begin
@@ -51,5 +57,8 @@ while row = $stdin.gets
51
57
  retry
52
58
  rescue Interrupt
53
59
  exit 0
60
+ rescue Timeout::Error
61
+ warn "aws_db retrying after Timeout::Error"
62
+ retry
54
63
  end
55
64
  end
@@ -71,4 +71,7 @@ begin
71
71
  end
72
72
  rescue Interrupt
73
73
  exit 0
74
+ rescue Timeout::Error
75
+ warn "aws_log retrying after Timeout::Error"
76
+ retry
74
77
  end
@@ -51,4 +51,7 @@ rescue AWS::SQS::Errors::NonExistentQueue
51
51
  exit 1
52
52
  rescue Interrupt
53
53
  exit 0
54
+ rescue Timeout::Error
55
+ warn "aws_queue retrying after Timeout::Error"
56
+ retry
54
57
  end
@@ -1,3 +1,3 @@
1
1
  module AwsPipes
2
- VERSION = "0.0.5"
2
+ VERSION = "0.0.6"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: aws_pipes
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.5
4
+ version: 0.0.6
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2012-12-28 00:00:00.000000000 Z
12
+ date: 2013-01-02 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: aws-sdk