raca 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/README.markdown +123 -0
- data/Rakefile +9 -0
- data/lib/raca.rb +5 -0
- data/lib/raca/account.rb +129 -0
- data/lib/raca/container.rb +309 -0
- data/lib/raca/containers.rb +79 -0
- data/lib/raca/server.rb +223 -0
- data/lib/raca/servers.rb +11 -0
- metadata +111 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 41701304d4625b6d3f99be830e6b5aa1facf6875
|
4
|
+
data.tar.gz: eadc909441d33e2b46270ff08f1c7762d424393b
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: d52aca857ce76407dd7f3fc83ff1eb044627eb3054e397eee46787f467eacd1764147750847b6a5e3be635b63b4c120a7e9b6b8b4bf3d7e9f8bfeb95253a78b1
|
7
|
+
data.tar.gz: a46a22b4169f9a7d888633cc402b68634fae716d007a10bcca21cc9c2cd72c5ac5468289ecdb911b0d1b230596e5765ff148a937b50a23fa7a079c5f266a95a7
|
data/README.markdown
ADDED
@@ -0,0 +1,123 @@
|
|
1
|
+
# Raca
|
2
|
+
|
3
|
+
A simple gem for interacting with Rackspace cloud APIs. The following APIs are
|
4
|
+
supported:
|
5
|
+
|
6
|
+
* Identity
|
7
|
+
* Cloud Files
|
8
|
+
* Cloud Servers
|
9
|
+
|
10
|
+
Raca intentionally has no dependencies outside the ruby standard library.
|
11
|
+
|
12
|
+
If loaded alongside Rails, it will utilise the rails cache to avoid repeated
|
13
|
+
requests to the rackspace identity API.
|
14
|
+
|
15
|
+
## Installation
|
16
|
+
|
17
|
+
gem install raca
|
18
|
+
|
19
|
+
## Usage
|
20
|
+
|
21
|
+
For full usage details check the documentation for each class, but here's
|
22
|
+
a taste of the basics.
|
23
|
+
|
24
|
+
### Regions
|
25
|
+
|
26
|
+
Many of the Rackspace cloud products are available in multiple regions. When
|
27
|
+
required, you can specify a region using a symbol with the 3-letter region code.
|
28
|
+
|
29
|
+
Currently, the following regions are valid:
|
30
|
+
|
31
|
+
* :ord - Chicago
|
32
|
+
* :iad - Northern Virginia
|
33
|
+
* :syd - Sydney
|
34
|
+
* :dfw - Dallas-Fort Worth
|
35
|
+
* :hkg - Hong Kong
|
36
|
+
|
37
|
+
### Identity
|
38
|
+
|
39
|
+
To authenticate and begin any interaction with rackspace, you must create a
|
40
|
+
Raca::Account instance.
|
41
|
+
|
42
|
+
account = Raca::Account.new("username", "api_key")
|
43
|
+
|
44
|
+
You can view the token that will be used for subsequent requests:
|
45
|
+
|
46
|
+
puts account.auth_token
|
47
|
+
|
48
|
+
Or you can view the URLs for each rackspace cloud API:
|
49
|
+
|
50
|
+
puts account.public_endpoint("cloudFiles", :ord)
|
51
|
+
puts account.service_endpoint("cloudFiles", :ord)
|
52
|
+
|
53
|
+
### Cloud Files
|
54
|
+
|
55
|
+
Using an existing Raca::Account object, retrieve a collection of Cloud Files
|
56
|
+
containers in a region like so:
|
57
|
+
|
58
|
+
ord_containers = account.containers(:ord)
|
59
|
+
|
60
|
+
You can retrieve a single container from the collection:
|
61
|
+
|
62
|
+
dir = ord_containers.get("container_name")
|
63
|
+
|
64
|
+
Retrieve some metadata on the collection:
|
65
|
+
|
66
|
+
put ord_containers.metadata
|
67
|
+
|
68
|
+
With a single container, you can perform a range of operations on the container
|
69
|
+
and objects inside it.
|
70
|
+
|
71
|
+
dir = ord_containers.get("container_name")
|
72
|
+
|
73
|
+
Download a file:
|
74
|
+
|
75
|
+
dir.download("remote_key.txt", "/home/jh/local_file.txt")
|
76
|
+
|
77
|
+
Upload a file:
|
78
|
+
|
79
|
+
dir.upload("target_path.txt", "/home/jh/local_file.txt")
|
80
|
+
|
81
|
+
List keys in the container, optionally limiting the results to those
|
82
|
+
starting with a prefix:
|
83
|
+
|
84
|
+
puts dir.list
|
85
|
+
puts dir.list(prefix: "subdir/")
|
86
|
+
|
87
|
+
Delete an object:
|
88
|
+
|
89
|
+
dir.delete("target_path.txt")
|
90
|
+
|
91
|
+
View metadata on the container:
|
92
|
+
|
93
|
+
puts dir.metadata
|
94
|
+
puts dir.cdn_metadata
|
95
|
+
|
96
|
+
Enable access to the container contents via a public CDN. Use this with caution, it will make *all* objects public!
|
97
|
+
|
98
|
+
It accepts an argument telling the CDN edge nodes how long they can cache each object for (in seconds).
|
99
|
+
|
100
|
+
dir.cdn_enable(60 * 60 * 24) # 1 day
|
101
|
+
|
102
|
+
Purge an object from the CDN:
|
103
|
+
|
104
|
+
dir.purge_from_akamai("target_path.txt", "notify@example.com")
|
105
|
+
|
106
|
+
Generate a public URL to an object in a private container. The second argument
|
107
|
+
is the temp URL key that can be set using Raca::Containers#set_temp_url_key
|
108
|
+
|
109
|
+
ord_containers = account.containers(:ord)
|
110
|
+
ord_containers.set_temp_url_key("secret")
|
111
|
+
dir = ord_containers.get("container_name")
|
112
|
+
puts dir.expiring_url("remote_key.txt", "secret", Time.now.to_i + 60)
|
113
|
+
|
114
|
+
### Cloud Servers
|
115
|
+
|
116
|
+
account = Raca::Account.new("username", "api_key")
|
117
|
+
server = account.servers(:ord).get("foo")
|
118
|
+
puts server.public_addresses
|
119
|
+
|
120
|
+
## Compatibility
|
121
|
+
|
122
|
+
The Raca version number is 0.0.x because it's highly unstable. Until we release
|
123
|
+
a 1.0.0, consider the API of this gem to be unstable.
|
data/Rakefile
ADDED
data/lib/raca.rb
ADDED
data/lib/raca/account.rb
ADDED
@@ -0,0 +1,129 @@
|
|
1
|
+
require 'yaml'
|
2
|
+
require 'json'
|
3
|
+
|
4
|
+
module Raca
|
5
|
+
|
6
|
+
# The rackspace auth API accepts a username and API key and returns a range
|
7
|
+
# of settings that are used for interacting with their other APIS. Think
|
8
|
+
# auth tokens, hostnames, paths, etc.
|
9
|
+
#
|
10
|
+
# This class caches these settings so we don't have to continually use our
|
11
|
+
# username/key to retrieve them.
|
12
|
+
#
|
13
|
+
class Account
|
14
|
+
|
15
|
+
def initialize(username, key, cache = nil)
|
16
|
+
@username, @key, @cache = username, key, cache
|
17
|
+
@cache ||= if defined?(Rails)
|
18
|
+
Rails.cache
|
19
|
+
else
|
20
|
+
{}
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
def auth_token
|
25
|
+
extract_value(cloudfiles_data, "access", "token", "id")
|
26
|
+
end
|
27
|
+
|
28
|
+
def public_endpoint(service_name, region)
|
29
|
+
region = region.to_s.upcase
|
30
|
+
endpoints = service_endpoints(service_name)
|
31
|
+
regional_endpoint = endpoints.detect { |e| e["region"] == region } || {}
|
32
|
+
regional_endpoint["publicURL"]
|
33
|
+
end
|
34
|
+
|
35
|
+
def containers(region)
|
36
|
+
Raca::Containers.new(self, region)
|
37
|
+
end
|
38
|
+
|
39
|
+
def servers(region)
|
40
|
+
Raca::Servers.new(self, region)
|
41
|
+
end
|
42
|
+
|
43
|
+
def refresh_cache
|
44
|
+
Net::HTTP.new('identity.api.rackspacecloud.com', 443).tap {|http|
|
45
|
+
http.use_ssl = true
|
46
|
+
}.start {|http|
|
47
|
+
payload = {
|
48
|
+
auth: {
|
49
|
+
'RAX-KSKEY:apiKeyCredentials' => {
|
50
|
+
username: @username,
|
51
|
+
apiKey: @key
|
52
|
+
}
|
53
|
+
}
|
54
|
+
}
|
55
|
+
response = http.post(
|
56
|
+
'/v2.0/tokens',
|
57
|
+
JSON.dump(payload),
|
58
|
+
{'Content-Type' => 'application/json'},
|
59
|
+
)
|
60
|
+
if response.is_a? Net::HTTPSuccess
|
61
|
+
cache_write(cache_key, JSON.load(response.body))
|
62
|
+
end
|
63
|
+
}
|
64
|
+
end
|
65
|
+
|
66
|
+
private
|
67
|
+
|
68
|
+
# This method is opaque, but it was the best I could come up with using just
|
69
|
+
# the standard library. Sorry.
|
70
|
+
#
|
71
|
+
# Use this to safely extract values from nested hashes:
|
72
|
+
#
|
73
|
+
# data = {a: {b: {c: 1}}}
|
74
|
+
# extract_value(data, :a, :b, :c)
|
75
|
+
# => 1
|
76
|
+
#
|
77
|
+
# extract_value(data, :a, :b, :d)
|
78
|
+
# => nil
|
79
|
+
#
|
80
|
+
# extract_value(data, :d)
|
81
|
+
# => nil
|
82
|
+
#
|
83
|
+
def extract_value(data, *keys)
|
84
|
+
if keys.empty?
|
85
|
+
data
|
86
|
+
elsif data.respond_to?(:[]) && data[keys.first]
|
87
|
+
extract_value(data[keys.first], *keys.slice(1,100))
|
88
|
+
else
|
89
|
+
nil
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
# An array of all the endpoints for a particular service (like cloud files,
|
94
|
+
# cloud servers, dns, etc)
|
95
|
+
#
|
96
|
+
def service_endpoints(service_name)
|
97
|
+
catalog = extract_value(cloudfiles_data, "access", "serviceCatalog") || {}
|
98
|
+
service = catalog.detect { |s| s["name"] == service_name } || {}
|
99
|
+
service["endpoints"] || []
|
100
|
+
end
|
101
|
+
|
102
|
+
def cache_read(key)
|
103
|
+
if @cache.respond_to?(:read) # rails cache
|
104
|
+
@cache.read(key)
|
105
|
+
else
|
106
|
+
@cache[key]
|
107
|
+
end
|
108
|
+
end
|
109
|
+
|
110
|
+
def cache_write(key, value)
|
111
|
+
if @cache.respond_to?(:write) # rails cache
|
112
|
+
@cache.write(key, value)
|
113
|
+
else
|
114
|
+
@cache[key] = value
|
115
|
+
end
|
116
|
+
end
|
117
|
+
|
118
|
+
def cloudfiles_data
|
119
|
+
refresh_cache unless cache_read(cache_key)
|
120
|
+
|
121
|
+
cache_read(cache_key) || {}
|
122
|
+
end
|
123
|
+
|
124
|
+
def cache_key
|
125
|
+
"raca-#{@username}"
|
126
|
+
end
|
127
|
+
|
128
|
+
end
|
129
|
+
end
|
@@ -0,0 +1,309 @@
|
|
1
|
+
require 'net/http'
|
2
|
+
require 'digest/md5'
|
3
|
+
require 'openssl'
|
4
|
+
require 'uri'
|
5
|
+
|
6
|
+
module Raca
|
7
|
+
# Handy abstraction for interacting with a single Cloud Files container. We
|
8
|
+
# could use fog or similar, but this ~200 line class is simple and does
|
9
|
+
# everything we need.
|
10
|
+
class Container
|
11
|
+
MAX_ITEMS_PER_LIST = 10_000
|
12
|
+
LARGE_FILE_THRESHOLD = 5_368_709_120 # 5 Gb
|
13
|
+
LARGE_FILE_SEGMENT_SIZE = 104_857_600 # 100 Mb
|
14
|
+
|
15
|
+
attr_reader :container_name
|
16
|
+
|
17
|
+
def initialize(account, region, container_name, opts = {})
|
18
|
+
raise ArgumentError, "The container name must not contain '/'." if container_name['/']
|
19
|
+
@account, @region, @container_name = account, region, container_name
|
20
|
+
@storage_url = @account.public_endpoint("cloudFiles", region)
|
21
|
+
@cdn_url = @account.public_endpoint("cloudFilesCDN", region)
|
22
|
+
@logger = opts[:logger]
|
23
|
+
@logger ||= Rails.logger if defined?(Rails)
|
24
|
+
end
|
25
|
+
|
26
|
+
# Upload data_or_path (which may be a filename or an IO) to the container, as key.
|
27
|
+
def upload(key, data_or_path)
|
28
|
+
case data_or_path
|
29
|
+
when StringIO, File
|
30
|
+
upload_io(key, data_or_path, data_or_path.size)
|
31
|
+
when String
|
32
|
+
File.open(data_or_path, "rb") do |io|
|
33
|
+
upload_io(key, io, io.stat.size)
|
34
|
+
end
|
35
|
+
else
|
36
|
+
raise ArgumentError, "data_or_path must be an IO with data or filename string"
|
37
|
+
end
|
38
|
+
end
|
39
|
+
|
40
|
+
# Delete +key+ from the container. If the container is on the CDN, the object will
|
41
|
+
# still be served from the CDN until the TTL expires.
|
42
|
+
def delete(key)
|
43
|
+
log "deleting #{key} from #{container_path}"
|
44
|
+
storage_request(Net::HTTP::Delete.new(File.join(container_path, key)))
|
45
|
+
end
|
46
|
+
|
47
|
+
# Remove +key+ from the CDN edge nodes on which it is currently cached. The object is
|
48
|
+
# not deleted from the container: as the URL is re-requested, the edge cache will be
|
49
|
+
# re-filled with the object currently in the container.
|
50
|
+
#
|
51
|
+
# This shouldn't be used except when it's really required (e.g. when a piece has to be
|
52
|
+
# taken down) because it's expensive: it lodges a support ticket at Akamai. (!)
|
53
|
+
def purge_from_akamai(key, email_address)
|
54
|
+
log "Requesting #{File.join(container_path, key)} to be purged from the CDN"
|
55
|
+
cdn_request(Net::HTTP::Delete.new(
|
56
|
+
File.join(container_path, key),
|
57
|
+
'X-Purge-Email' => email_address
|
58
|
+
))
|
59
|
+
end
|
60
|
+
|
61
|
+
def download(key, filepath)
|
62
|
+
log "downloading #{key} from #{container_path}"
|
63
|
+
storage_request(Net::HTTP::Get.new(File.join(container_path, key))) do |response|
|
64
|
+
File.open(filepath, 'wb') do |io|
|
65
|
+
response.read_body do |chunk|
|
66
|
+
io.write(chunk)
|
67
|
+
end
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
# Return an array of files in the container.
|
73
|
+
#
|
74
|
+
# Supported options
|
75
|
+
#
|
76
|
+
# max - the maximum number of items to return
|
77
|
+
# marker - return items alphabetically after this key. Useful for pagination
|
78
|
+
# prefix - only return items that start with this string
|
79
|
+
def list(options = {})
|
80
|
+
max = options.fetch(:max, MAX_ITEMS_PER_LIST)
|
81
|
+
marker = options.fetch(:marker, nil)
|
82
|
+
prefix = options.fetch(:prefix, nil)
|
83
|
+
limit = [max, MAX_ITEMS_PER_LIST].min
|
84
|
+
log "retrieving up to #{limit} of #{max} items from #{container_path}"
|
85
|
+
query_string = "limit=#{limit}"
|
86
|
+
query_string += "&marker=#{marker}" if marker
|
87
|
+
query_string += "&prefix=#{prefix}" if prefix
|
88
|
+
request = Net::HTTP::Get.new(container_path + "?#{query_string}")
|
89
|
+
result = storage_request(request).body || ""
|
90
|
+
result.split("\n").tap {|items|
|
91
|
+
if max <= limit
|
92
|
+
log "Got #{items.length} items; we don't need any more."
|
93
|
+
elsif items.length < limit
|
94
|
+
log "Got #{items.length} items; there can't be any more."
|
95
|
+
else
|
96
|
+
log "Got #{items.length} items; requesting #{max - limit} more."
|
97
|
+
items.concat list(max: max - limit, marker: items.last, prefix: prefix)
|
98
|
+
end
|
99
|
+
}
|
100
|
+
end
|
101
|
+
|
102
|
+
def search(prefix)
|
103
|
+
log "retrieving container listing from #{container_path} items starting with #{prefix}"
|
104
|
+
list(prefix: prefix)
|
105
|
+
end
|
106
|
+
|
107
|
+
def metadata
|
108
|
+
log "retrieving container metadata from #{container_path}"
|
109
|
+
response = storage_request(Net::HTTP::Head.new(container_path))
|
110
|
+
{
|
111
|
+
:objects => response["X-Container-Object-Count"].to_i,
|
112
|
+
:bytes => response["X-Container-Bytes-Used"].to_i
|
113
|
+
}
|
114
|
+
end
|
115
|
+
|
116
|
+
def cdn_metadata
|
117
|
+
log "retrieving container CDN metadata from #{container_path}"
|
118
|
+
response = cdn_request(Net::HTTP::Head.new(container_path))
|
119
|
+
{
|
120
|
+
:cdn_enabled => response["X-CDN-Enabled"] == "True",
|
121
|
+
:host => response["X-CDN-URI"],
|
122
|
+
:ssl_host => response["X-CDN-SSL-URI"],
|
123
|
+
:streaming_host => response["X-CDN-STREAMING-URI"],
|
124
|
+
:ttl => response["X-TTL"].to_i,
|
125
|
+
:log_retention => response["X-Log-Retention"] == "True"
|
126
|
+
}
|
127
|
+
end
|
128
|
+
|
129
|
+
# use this with caution, it will make EVERY object in the container publicly available
|
130
|
+
# via the CDN. CDN enabling can be done via the web UI but only with a TTL of 72 hours.
|
131
|
+
# Using the API it's possible to set a TTL of 50 years.
|
132
|
+
#
|
133
|
+
def cdn_enable(ttl = 72.hours.to_i)
|
134
|
+
log "enabling CDN access to #{container_path} with a cache expiry of #{ttl / 60} minutes"
|
135
|
+
|
136
|
+
cdn_request Net::HTTP::Put.new(container_path, "X-TTL" => ttl.to_i.to_s)
|
137
|
+
end
|
138
|
+
|
139
|
+
# Generate a expiring URL for a file that is otherwise private. useful for providing temporary
|
140
|
+
# access to files.
|
141
|
+
#
|
142
|
+
def expiring_url(object_key, temp_url_key, expires_at = Time.now.to_i + 60)
|
143
|
+
digest = OpenSSL::Digest::Digest.new('sha1')
|
144
|
+
|
145
|
+
method = 'GET'
|
146
|
+
expires = expires_at.to_i
|
147
|
+
path = File.join(container_path, object_key)
|
148
|
+
data = "#{method}\n#{expires}\n#{path}"
|
149
|
+
|
150
|
+
hmac = OpenSSL::HMAC.new(temp_url_key, digest)
|
151
|
+
hmac << data
|
152
|
+
|
153
|
+
"https://#{storage_host}#{path}?temp_url_sig=#{hmac.hexdigest}&temp_url_expires=#{expires}"
|
154
|
+
end
|
155
|
+
|
156
|
+
private
|
157
|
+
|
158
|
+
def upload_io(key, io, byte_count)
|
159
|
+
if byte_count <= LARGE_FILE_THRESHOLD
|
160
|
+
upload_io_standard(key, io, byte_count)
|
161
|
+
else
|
162
|
+
upload_io_large(key, io, byte_count)
|
163
|
+
end
|
164
|
+
end
|
165
|
+
|
166
|
+
def upload_io_standard(key, io, byte_count)
|
167
|
+
full_path = File.join(container_path, key)
|
168
|
+
|
169
|
+
headers = {}
|
170
|
+
headers['Content-Type'] = extension_content_type(full_path)
|
171
|
+
if io.respond_to?(:path)
|
172
|
+
headers['Content-Type'] ||= extension_content_type(io.path)
|
173
|
+
headers['Content-Type'] ||= file_content_type(io.path)
|
174
|
+
headers['Etag'] = md5(io.path)
|
175
|
+
end
|
176
|
+
headers['Content-Type'] ||= "application/octet-stream"
|
177
|
+
if content_type_needs_cors(key)
|
178
|
+
headers['Access-Control-Allow-Origin'] = "*"
|
179
|
+
end
|
180
|
+
|
181
|
+
log "uploading #{byte_count} bytes to #{full_path}"
|
182
|
+
|
183
|
+
request = Net::HTTP::Put.new(full_path, headers)
|
184
|
+
request.body_stream = io
|
185
|
+
request.content_length = byte_count
|
186
|
+
storage_request(request)
|
187
|
+
end
|
188
|
+
|
189
|
+
def upload_io_large(key, io, byte_count)
|
190
|
+
segment_count = (byte_count.to_f / LARGE_FILE_SEGMENT_SIZE).ceil
|
191
|
+
segments = []
|
192
|
+
while segments.size < segment_count
|
193
|
+
start_pos = 0 + (LARGE_FILE_SEGMENT_SIZE * segments.size)
|
194
|
+
segment_key = "%s.%03d" % [key, segments.size]
|
195
|
+
io.seek(start_pos)
|
196
|
+
segment_io = StringIO.new(io.read(LARGE_FILE_SEGMENT_SIZE))
|
197
|
+
result = upload_io_standard(segment_key, segment_io, segment_io.size)
|
198
|
+
segments << {path: "#{@container_name}/#{segment_key}", etag: result["ETag"], size_bytes: segment_io.size}
|
199
|
+
end
|
200
|
+
manifest_key = "#{key}?multipart-manifest=put"
|
201
|
+
manifest_body = StringIO.new(JSON.dump(segments))
|
202
|
+
upload_io_standard(manifest_key, manifest_body, manifest_body.size)
|
203
|
+
end
|
204
|
+
|
205
|
+
def cdn_request(request, &block)
|
206
|
+
cloud_request(request, cdn_host, &block)
|
207
|
+
end
|
208
|
+
|
209
|
+
def storage_request(request, &block)
|
210
|
+
cloud_request(request, storage_host, &block)
|
211
|
+
end
|
212
|
+
|
213
|
+
def cloud_request(request, hostname, retries = 0, &block)
|
214
|
+
cloud_http(hostname) do |http|
|
215
|
+
request['X-Auth-Token'] = @account.auth_token
|
216
|
+
http.request(request, &block)
|
217
|
+
end
|
218
|
+
rescue Timeout::Error
|
219
|
+
if retries >= 3
|
220
|
+
raise "Timeout from Rackspace at #{Time.now} while trying #{request.class} to #{request.path}"
|
221
|
+
end
|
222
|
+
|
223
|
+
unless defined?(Rails) && Rails.env.test?
|
224
|
+
retry_interval = 5 + (retries.to_i * 5) # Retry after 5, 10, 15 and 20 seconds
|
225
|
+
log "Rackspace timed out: retrying after #{retry_interval}s"
|
226
|
+
sleep(retry_interval)
|
227
|
+
end
|
228
|
+
|
229
|
+
cloud_request(request, hostname, retries + 1, &block)
|
230
|
+
end
|
231
|
+
|
232
|
+
def cloud_http(hostname, &block)
|
233
|
+
Net::HTTP.new(hostname, 443).tap {|http|
|
234
|
+
http.use_ssl = true
|
235
|
+
http.read_timeout = 70
|
236
|
+
}.start do |http|
|
237
|
+
response = block.call http
|
238
|
+
if response.is_a?(Net::HTTPUnauthorized)
|
239
|
+
log "Rackspace returned HTTP 401; refreshing auth before retrying."
|
240
|
+
@account.refresh_cache
|
241
|
+
response = block.call http
|
242
|
+
end
|
243
|
+
raise "Failure: Rackspace returned #{response.inspect}" unless response.is_a?(Net::HTTPSuccess)
|
244
|
+
response
|
245
|
+
end
|
246
|
+
end
|
247
|
+
|
248
|
+
def log(msg)
|
249
|
+
if @logger.respond_to?(:debug)
|
250
|
+
@logger.debug msg
|
251
|
+
end
|
252
|
+
end
|
253
|
+
|
254
|
+
def storage_host
|
255
|
+
URI.parse(@storage_url).host
|
256
|
+
end
|
257
|
+
|
258
|
+
def storage_path
|
259
|
+
URI.parse(@storage_url).path
|
260
|
+
end
|
261
|
+
|
262
|
+
def cdn_host
|
263
|
+
URI.parse(@cdn_url).host
|
264
|
+
end
|
265
|
+
|
266
|
+
def cdn_path
|
267
|
+
URI.parse(@cdn_url).path
|
268
|
+
end
|
269
|
+
|
270
|
+
def container_path
|
271
|
+
@container_path ||= File.join(storage_path, container_name)
|
272
|
+
end
|
273
|
+
|
274
|
+
def file_content_type(path)
|
275
|
+
`file -b --mime-type \"#{path.gsub('"', '\"')}\"`.chomp
|
276
|
+
end
|
277
|
+
|
278
|
+
def extension_content_type(path)
|
279
|
+
{
|
280
|
+
".css" => "text/css",
|
281
|
+
".eot" => "application/vnd.ms-fontobject",
|
282
|
+
".html" => "text/html",
|
283
|
+
".js" => "application/javascript",
|
284
|
+
".png" => "image/png",
|
285
|
+
".jpg" => "image/jpeg",
|
286
|
+
".txt" => "text/plain",
|
287
|
+
".woff" => "font/woff",
|
288
|
+
".zip" => "application/zip"
|
289
|
+
}[File.extname(path)]
|
290
|
+
end
|
291
|
+
|
292
|
+
# Fonts need to be served with CORS headers to work in IE and FF
|
293
|
+
#
|
294
|
+
def content_type_needs_cors(path)
|
295
|
+
[".eot",".ttf",".woff"].include?(File.extname(path))
|
296
|
+
end
|
297
|
+
|
298
|
+
def md5(path)
|
299
|
+
digest = Digest::MD5.new
|
300
|
+
File.open(path, 'rb') do |f|
|
301
|
+
# read in 128K chunks
|
302
|
+
f.each(1024 * 128) do |chunk|
|
303
|
+
digest << chunk
|
304
|
+
end
|
305
|
+
end
|
306
|
+
digest.hexdigest
|
307
|
+
end
|
308
|
+
end
|
309
|
+
end
|
@@ -0,0 +1,79 @@
|
|
1
|
+
module Raca
|
2
|
+
class Containers
|
3
|
+
def initialize(account, region, opts = {})
|
4
|
+
@account, @region = account, region
|
5
|
+
@storage_url = @account.public_endpoint("cloudFiles", region)
|
6
|
+
@logger = opts[:logger]
|
7
|
+
@logger ||= Rails.logger if defined?(Rails)
|
8
|
+
end
|
9
|
+
|
10
|
+
def get(container_name)
|
11
|
+
Raca::Container.new(@account, @region, container_name)
|
12
|
+
end
|
13
|
+
|
14
|
+
# Return metadata on all containers
|
15
|
+
#
|
16
|
+
def metadata
|
17
|
+
log "retrieving containers metadata from #{storage_path}"
|
18
|
+
response = storage_request(Net::HTTP::Head.new(storage_path))
|
19
|
+
{
|
20
|
+
:containers => response["X-Account-Container-Count"].to_i,
|
21
|
+
:objects => response["X-Account-Object-Count"].to_i,
|
22
|
+
:bytes => response["X-Account-Bytes-Used"].to_i
|
23
|
+
}
|
24
|
+
end
|
25
|
+
|
26
|
+
# Set the secret key that will be used to generate expiring URLs for all cloud files containers on the current account. This value should be passed to the expiring_url() method.
|
27
|
+
#
|
28
|
+
# Use this with caution, this will invalidate all previously generated expiring URLS *FOR THE ENTIRE ACCOUNT*
|
29
|
+
#
|
30
|
+
def set_temp_url_key(secret)
|
31
|
+
log "setting Account Temp URL Key on #{storage_path}"
|
32
|
+
|
33
|
+
storage_request Net::HTTP::Post.new(storage_path, "X-Account-Meta-Temp-Url-Key" => secret.to_s)
|
34
|
+
end
|
35
|
+
|
36
|
+
private
|
37
|
+
|
38
|
+
def storage_host
|
39
|
+
URI.parse(@storage_url).host
|
40
|
+
end
|
41
|
+
|
42
|
+
def storage_path
|
43
|
+
URI.parse(@storage_url).path
|
44
|
+
end
|
45
|
+
|
46
|
+
def storage_request(request, &block)
|
47
|
+
cloud_request(request, storage_host, &block)
|
48
|
+
end
|
49
|
+
|
50
|
+
def cloud_request(request, hostname, &block)
|
51
|
+
cloud_http(hostname) do |http|
|
52
|
+
request['X-Auth-Token'] = @account.auth_token
|
53
|
+
http.request(request, &block)
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
def cloud_http(hostname, &block)
|
58
|
+
Net::HTTP.new(hostname, 443).tap {|http|
|
59
|
+
http.use_ssl = true
|
60
|
+
http.read_timeout = 70
|
61
|
+
}.start do |http|
|
62
|
+
response = block.call http
|
63
|
+
if response.is_a?(Net::HTTPUnauthorized)
|
64
|
+
log "Rackspace returned HTTP 401; refreshing auth before retrying."
|
65
|
+
@account.refresh_cache
|
66
|
+
response = block.call http
|
67
|
+
end
|
68
|
+
raise "Failure: Rackspace returned #{response.inspect}" unless response.is_a?(Net::HTTPSuccess)
|
69
|
+
response
|
70
|
+
end
|
71
|
+
end
|
72
|
+
|
73
|
+
def log(msg)
|
74
|
+
if @logger.respond_to?(:debug)
|
75
|
+
@logger.debug msg
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
79
|
+
end
|
data/lib/raca/server.rb
ADDED
@@ -0,0 +1,223 @@
|
|
1
|
+
require 'json'
|
2
|
+
require 'base64'
|
3
|
+
require 'net/http'
|
4
|
+
|
5
|
+
module Raca
|
6
|
+
# Handy abstraction for interacting with a single rackspace Clpud Server. We
|
7
|
+
# could use fog or similar, but this ~200 line class is simple and does
|
8
|
+
# everything we need.
|
9
|
+
class Server
|
10
|
+
|
11
|
+
attr_reader :server_name, :server_id
|
12
|
+
|
13
|
+
def initialize(account, region, server_name)
|
14
|
+
@account = account
|
15
|
+
@region = region
|
16
|
+
@servers_url = @account.public_endpoint("cloudServersOpenStack", region)
|
17
|
+
@server_name = server_name
|
18
|
+
@server_id = find_server_id(server_name)
|
19
|
+
end
|
20
|
+
|
21
|
+
# return true if this server exists on Rackspace
|
22
|
+
#
|
23
|
+
def exists?
|
24
|
+
@server_id != nil
|
25
|
+
end
|
26
|
+
|
27
|
+
# create this server on Rackspace.
|
28
|
+
#
|
29
|
+
# flavor_name is a string that describes the amount of RAM. If you enter
|
30
|
+
# an invalid option a list of valid options will be raised.
|
31
|
+
#
|
32
|
+
# image_name is a string that describes the OS image to use. If you enter
|
33
|
+
# an invalid option a list of valid options will be raised. I suggest
|
34
|
+
# starting with 'Ubuntu 10.04 LTS'
|
35
|
+
#
|
36
|
+
# files is an optional Hash of path to blobs. Use it to place a file on the
|
37
|
+
# disk of the new server.
|
38
|
+
#
|
39
|
+
# Use it like this:
|
40
|
+
#
|
41
|
+
# server.create(512, "Ubuntu 10.04 LTS", "/root/.ssh/authorised_keys" => File.read("/foo"))
|
42
|
+
#
|
43
|
+
def create(flavor_name, image_name, files = {})
|
44
|
+
raise ArgumentError, "server already exists" if exists?
|
45
|
+
|
46
|
+
request = {
|
47
|
+
"server" => {
|
48
|
+
"name" => @server_name,
|
49
|
+
"imageRef" => image_name_to_id(image_name),
|
50
|
+
"flavorRef" => flavor_name_to_id(flavor_name),
|
51
|
+
}
|
52
|
+
}
|
53
|
+
files.each do |path, blob|
|
54
|
+
request['server']['personality'] ||= []
|
55
|
+
request['server']['personality'] << {
|
56
|
+
'path' => path,
|
57
|
+
'contents' => Base64.encode64(blob)
|
58
|
+
}
|
59
|
+
end
|
60
|
+
|
61
|
+
data = cloud_request(Net::HTTP::Post.new(servers_path), JSON.dump(request)).body
|
62
|
+
data = JSON.parse(data)['server']
|
63
|
+
@server_id = data['id']
|
64
|
+
end
|
65
|
+
|
66
|
+
def delete!
|
67
|
+
raise ArgumentError, "server doesn't exist" unless exists?
|
68
|
+
|
69
|
+
response = cloud_request(Net::HTTP::Delete.new(server_path))
|
70
|
+
response.is_a? Net::HTTPSuccess
|
71
|
+
end
|
72
|
+
|
73
|
+
# Poll Rackspace and return once a server is in an active state. Useful after
|
74
|
+
# creating a new server
|
75
|
+
#
|
76
|
+
def wait_for_active
|
77
|
+
until details['status'] == 'ACTIVE'
|
78
|
+
log "Not online yet. Waiting..."
|
79
|
+
sleep 10
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
# An array of private IP addresses for the server. They can be ipv4 or ipv6
|
84
|
+
#
|
85
|
+
def private_addresses
|
86
|
+
details['addresses']['private'].map { |i| i["addr"] }
|
87
|
+
end
|
88
|
+
|
89
|
+
# An array of public IP addresses for the server. They can be ipv4 or ipv6
|
90
|
+
#
|
91
|
+
def public_addresses
|
92
|
+
details['addresses']['public'].map { |i| i["addr"] }
|
93
|
+
end
|
94
|
+
|
95
|
+
# A Hash of various matadata about the server
|
96
|
+
#
|
97
|
+
def details
|
98
|
+
raise ArgumentError, "server doesn't exist" unless exists?
|
99
|
+
|
100
|
+
data = cloud_request(Net::HTTP::Get.new(server_path)).body
|
101
|
+
JSON.parse(data)['server']
|
102
|
+
end
|
103
|
+
|
104
|
+
private
|
105
|
+
|
106
|
+
def list
|
107
|
+
json = cloud_request(Net::HTTP::Get.new(servers_path)).body
|
108
|
+
JSON.parse(json)['servers']
|
109
|
+
end
|
110
|
+
|
111
|
+
def find_server_id(server_name)
|
112
|
+
server = list.detect {|row|
|
113
|
+
row["name"] == server_name
|
114
|
+
}
|
115
|
+
server ? server["id"] : nil
|
116
|
+
end
|
117
|
+
|
118
|
+
def flavors_path
|
119
|
+
@flavors_path ||= File.join(account_path, "flavors")
|
120
|
+
end
|
121
|
+
|
122
|
+
def images_path
|
123
|
+
@images_path ||= File.join(account_path, "images")
|
124
|
+
end
|
125
|
+
|
126
|
+
def server_path
|
127
|
+
@server_path ||= File.join(account_path, "servers", @server_id.to_s)
|
128
|
+
end
|
129
|
+
|
130
|
+
def servers_path
|
131
|
+
@servers_path ||= File.join(account_path, "servers")
|
132
|
+
end
|
133
|
+
|
134
|
+
def servers_host
|
135
|
+
@servers_host ||= URI.parse(@servers_url).host
|
136
|
+
end
|
137
|
+
|
138
|
+
def account_path
|
139
|
+
@account_path ||= URI.parse(@servers_url).path
|
140
|
+
end
|
141
|
+
|
142
|
+
def flavors
|
143
|
+
@flavors ||= begin
|
144
|
+
data = cloud_request(Net::HTTP::Get.new(flavors_path)).body
|
145
|
+
JSON.parse(data)['flavors']
|
146
|
+
end
|
147
|
+
end
|
148
|
+
|
149
|
+
def flavor_names
|
150
|
+
flavors.map {|row| row['name'] }
|
151
|
+
end
|
152
|
+
|
153
|
+
def flavor_name_to_id(str)
|
154
|
+
flavor = flavors.detect {|row|
|
155
|
+
row['name'].downcase.include?(str.to_s.downcase)
|
156
|
+
}
|
157
|
+
if flavor
|
158
|
+
flavor['id']
|
159
|
+
else
|
160
|
+
raise ArgumentError, "valid flavors are: #{flavor_names.join(', ')}"
|
161
|
+
end
|
162
|
+
end
|
163
|
+
|
164
|
+
def images
|
165
|
+
@images ||= begin
|
166
|
+
data = cloud_request(Net::HTTP::Get.new(images_path)).body
|
167
|
+
JSON.parse(data)['images']
|
168
|
+
end
|
169
|
+
end
|
170
|
+
|
171
|
+
def image_names
|
172
|
+
images.map {|row| row['name'] }
|
173
|
+
end
|
174
|
+
|
175
|
+
def image_name_to_id(str)
|
176
|
+
image = images.detect {|row|
|
177
|
+
row['name'].downcase.include?(str.to_s.downcase)
|
178
|
+
}
|
179
|
+
if image
|
180
|
+
image['id']
|
181
|
+
else
|
182
|
+
raise ArgumentError, "valid images are: #{image_names.join(', ')}"
|
183
|
+
end
|
184
|
+
end
|
185
|
+
|
186
|
+
def cloud_request(request, body = nil)
|
187
|
+
request['X-Auth-Token'] = @account.auth_token
|
188
|
+
request['Content-Type'] = 'application/json'
|
189
|
+
request['Accept'] = 'application/json'
|
190
|
+
cloud_http(servers_host) do |http|
|
191
|
+
http.request(request, body)
|
192
|
+
end
|
193
|
+
end
|
194
|
+
|
195
|
+
def cloud_http(hostname, retries = 3, &block)
|
196
|
+
http = Net::HTTP.new(hostname, 443)
|
197
|
+
http.use_ssl = true
|
198
|
+
http.start do |http|
|
199
|
+
response = block.call http
|
200
|
+
if response.is_a?(Net::HTTPUnauthorized)
|
201
|
+
log "Rackspace returned HTTP 401; refreshing auth before retrying."
|
202
|
+
@account.refresh_cache
|
203
|
+
response = block.call http
|
204
|
+
end
|
205
|
+
raise "Failure: Rackspace returned #{response.inspect}" unless response.is_a?(Net::HTTPSuccess)
|
206
|
+
response
|
207
|
+
end
|
208
|
+
rescue Timeout::Error => e
|
209
|
+
raise e if retries <= 0
|
210
|
+
|
211
|
+
cloud_http(hostname, retries - 1, &block)
|
212
|
+
end
|
213
|
+
|
214
|
+
def log(msg)
|
215
|
+
if defined?(Rails)
|
216
|
+
Rails.logger.info msg
|
217
|
+
else
|
218
|
+
puts msg
|
219
|
+
end
|
220
|
+
end
|
221
|
+
|
222
|
+
end
|
223
|
+
end
|
data/lib/raca/servers.rb
ADDED
metadata
ADDED
@@ -0,0 +1,111 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: raca
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- James Healy
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2014-02-24 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: rake
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '10.0'
|
20
|
+
type: :development
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '10.0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rspec
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '2.0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '2.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: webmock
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: ir_b
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - ">="
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ">="
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
69
|
+
description: A simple wrapper for the Rackspace Cloud API with no dependencies
|
70
|
+
email:
|
71
|
+
- james.healy@theconversation.edu.au
|
72
|
+
executables: []
|
73
|
+
extensions: []
|
74
|
+
extra_rdoc_files: []
|
75
|
+
files:
|
76
|
+
- README.markdown
|
77
|
+
- Rakefile
|
78
|
+
- lib/raca.rb
|
79
|
+
- lib/raca/account.rb
|
80
|
+
- lib/raca/container.rb
|
81
|
+
- lib/raca/containers.rb
|
82
|
+
- lib/raca/server.rb
|
83
|
+
- lib/raca/servers.rb
|
84
|
+
homepage: http://github.com/conversation/raca
|
85
|
+
licenses:
|
86
|
+
- MIT
|
87
|
+
metadata: {}
|
88
|
+
post_install_message:
|
89
|
+
rdoc_options:
|
90
|
+
- "--title"
|
91
|
+
- Raca
|
92
|
+
- "--line-numbers"
|
93
|
+
require_paths:
|
94
|
+
- lib
|
95
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
96
|
+
requirements:
|
97
|
+
- - ">="
|
98
|
+
- !ruby/object:Gem::Version
|
99
|
+
version: '0'
|
100
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
101
|
+
requirements:
|
102
|
+
- - ">="
|
103
|
+
- !ruby/object:Gem::Version
|
104
|
+
version: '0'
|
105
|
+
requirements: []
|
106
|
+
rubyforge_project:
|
107
|
+
rubygems_version: 2.2.0
|
108
|
+
signing_key:
|
109
|
+
specification_version: 4
|
110
|
+
summary: A simple wrapper for the Rackspace Cloud API with no dependencies
|
111
|
+
test_files: []
|