s3cp 1.1.0.pre.1 → 1.1.0.pre.2
Sign up to get free protection for your applications and to get access to all the features.
- data/History.txt +55 -38
- data/lib/s3cp/s3cp.rb +5 -0
- data/lib/s3cp/utils.rb +1 -7
- data/lib/s3cp/version.rb +1 -1
- metadata +4 -4
data/History.txt
CHANGED
@@ -1,19 +1,36 @@
|
|
1
|
-
=== 1.1.0 / (Pending)
|
1
|
+
=== 1.1.0.pre.2 / (Pending)
|
2
|
+
|
3
|
+
* Added: 's3cp --version' now reports version information.
|
4
|
+
|
5
|
+
* Changed: S3CP uses the default credential provider from aws-sdk which makes
|
6
|
+
a best effort to locate your AWS credentials. It checks a variety of
|
7
|
+
locations in the following order:
|
8
|
+
|
9
|
+
* Static credentials from AWS.config (e.g. AWS.config.access_key_id,
|
10
|
+
AWS.config.secret_access_key)
|
11
|
+
|
12
|
+
* The environment (e.g. ENV['AWS_ACCESS_KEY_ID'] or
|
13
|
+
ENV['AMAZON_ACCESS_KEY_ID'])
|
14
|
+
|
15
|
+
* EC2 metadata service (checks for credentials provided by
|
16
|
+
roles for instances).
|
17
|
+
|
18
|
+
=== 1.1.0.pre.1 / (2012-07-23)
|
2
19
|
|
3
20
|
* Changed: Underlying AWS library has been changed from 'right-aws' to the official
|
4
|
-
|
5
|
-
|
6
|
-
|
21
|
+
AWS SDK ('aws-sdk'). The main driver for this is to stay current on the
|
22
|
+
latest features, including instance-specific roles, temporary credentials,
|
23
|
+
etc.
|
7
24
|
|
8
25
|
* Added: New 's3buckets' command displays all S3 buckets associated with the
|
9
|
-
|
26
|
+
account.
|
10
27
|
|
11
28
|
* Added: New '--acl PERMISSION' option to 's3cp' and 's3up' commands.
|
12
29
|
|
13
30
|
=== 1.0.6 / (2012-07-23)
|
14
31
|
|
15
32
|
* Fixed: s3cp would exit with errorlevel 0 despite errors when running in
|
16
|
-
|
33
|
+
interactive mode.
|
17
34
|
|
18
35
|
=== 1.0.5 / (2012-06-25)
|
19
36
|
|
@@ -27,15 +44,15 @@
|
|
27
44
|
|
28
45
|
* Added: Exponential backoff for `s3cp` command based on the formula:
|
29
46
|
|
30
|
-
|
47
|
+
delay = initial_retry_delay * (factor ^ retries)
|
31
48
|
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
49
|
+
where:
|
50
|
+
- `initial_retry_delay` is 1 second by default.
|
51
|
+
- `factor` is 1.4142 by default
|
52
|
+
(retry delay doubles every two retries)
|
53
|
+
- max. number of retries is now 20 by default.
|
37
54
|
|
38
|
-
|
55
|
+
(which means s3cp retries for roughly 58 minutes by default)
|
39
56
|
|
40
57
|
* Fixed: `s3ls` and `s3du` now properly handle Errno::EPIPE
|
41
58
|
|
@@ -45,18 +62,18 @@
|
|
45
62
|
|
46
63
|
* Added: `s3up` now outputs uploaded file size to STDERR, e.g.,
|
47
64
|
|
48
|
-
|
49
|
-
|
65
|
+
% ls | s3up s3://bucket/path/to/file
|
66
|
+
s3://bucket/path/to/file => 214B
|
50
67
|
|
51
68
|
=== 1.0.1 / 2012-03-9
|
52
69
|
|
53
70
|
* Added: New command `s3up` to upload STDIN to S3
|
54
|
-
|
71
|
+
e.g. some_cmd | s3up s3://bucket/path/to/destination
|
55
72
|
|
56
73
|
Note: `s3up` does not yet directly stream data to S3 since right_aws gem
|
57
|
-
|
58
|
-
|
59
|
-
|
74
|
+
requires the upload size to be known in advance. `s3up` currently
|
75
|
+
persists STDIN into a temp file although this may change in the
|
76
|
+
future.
|
60
77
|
|
61
78
|
=== 1.0.0 / 2012-03-9
|
62
79
|
|
@@ -70,27 +87,27 @@ bugs -- but at least all known bugs have been addressed.
|
|
70
87
|
Probably worth mentioning: there are two outstanding experimental features,
|
71
88
|
|
72
89
|
1) s3cp --sync: Does not handle missing files. It's basically
|
73
|
-
|
90
|
+
a faster copy; not a true "sync" at this point.
|
74
91
|
|
75
92
|
2) Bash command-line completion is still very rough.
|
76
93
|
|
77
94
|
=== 0.2.7 / 2012-03-09
|
78
95
|
|
79
96
|
* Fixed #3: s3cp adds extra slash when copying to the root of a bucket.
|
80
|
-
|
97
|
+
e.g. s3://myBucket//someFile or s3://myBucket//someFolderToCopy/someFile
|
81
98
|
|
82
99
|
=== 0.2.6 / 2012-03-06
|
83
100
|
|
84
101
|
* Fixed: Possible division-by-zero error in s3du if it encounters
|
85
|
-
|
102
|
+
zero-length files.
|
86
103
|
|
87
104
|
=== 0.2.5 / 2012-03-02
|
88
105
|
|
89
106
|
* Added: "s3du" command to calculate disk usage.
|
90
|
-
|
107
|
+
Supports --depth, --regex, --unit parameters and more!
|
91
108
|
|
92
109
|
* Changed: "s3ls -l" command now accepts --unit and --precision to configure
|
93
|
-
|
110
|
+
file size display. Uses "smart" unit by default.
|
94
111
|
|
95
112
|
* Changed: "s3ls -l" will now use S3CP_DATE_FORMAT environment if set.
|
96
113
|
|
@@ -132,12 +149,12 @@ Probably worth mentioning: there are two outstanding experimental features,
|
|
132
149
|
* Added: Progress bars during upload/download if $stdout.isatty
|
133
150
|
|
134
151
|
* Fixed: s3cat now handles broken pipes properly
|
135
|
-
|
152
|
+
e.g. "s3cat bucket:some/file | head" will now terminate early.
|
136
153
|
|
137
154
|
=== 0.2.1 / (2012-02-20)
|
138
155
|
|
139
156
|
* Added: Bash completion now supports exclusions through S3CP_EXCLUDES
|
140
|
-
|
157
|
+
and defaults to excluding keys containing "_$folder$".
|
141
158
|
|
142
159
|
* Changed: s3dir and s3ls --delimiter now display both directories and files
|
143
160
|
|
@@ -146,15 +163,15 @@ Probably worth mentioning: there are two outstanding experimental features,
|
|
146
163
|
* Added: s3stat command to display S3 object properties
|
147
164
|
|
148
165
|
* Added: s3dir as a shortcut for "s3ls --delimiter / ..."
|
149
|
-
|
166
|
+
(default to character in S3CP_DELIMITER environment variable or "/" if not defined)
|
150
167
|
|
151
168
|
* Added: s3cp defaults can now be set using environment variables
|
152
|
-
|
169
|
+
S3CP_OVERWRITE, S3CP_CHECKSUM, S3CP_RETRIES, S3CP_RETRY_DELAY
|
153
170
|
|
154
171
|
* Added: Support for Bash command-line completion of S3 URLs (see below).
|
155
172
|
|
156
173
|
* Fixed: Skip checksum verification for S3 objects with invalid MD5's
|
157
|
-
|
174
|
+
(details @ https://forums.aws.amazon.com/message.jspa?messageID=234538)
|
158
175
|
|
159
176
|
To install Bash completion for S3 URLs, add the following to ~/.bashrc:
|
160
177
|
|
@@ -165,10 +182,10 @@ To install Bash completion for S3 URLs, add the following to ~/.bashrc:
|
|
165
182
|
=== 0.1.15 / (2012-02-17)
|
166
183
|
|
167
184
|
* Added: s3cp now automatically checks MD5 checksums during download/upload
|
168
|
-
|
169
|
-
|
170
|
-
|
171
|
-
|
185
|
+
and retries up to 5 times by default if the checksum fails.
|
186
|
+
The number of attempts may be configured using --max-attempts,
|
187
|
+
the retry delay may be changed with --retry-delay and the check
|
188
|
+
may be disabled completely using --no-checksum.
|
172
189
|
|
173
190
|
=== 0.1.14 / (2012-02-09)
|
174
191
|
|
@@ -190,7 +207,7 @@ To install Bash completion for S3 URLs, add the following to ~/.bashrc:
|
|
190
207
|
|
191
208
|
* Fixed: --max-keys now works correctly with --delimiter
|
192
209
|
* Fixed: do not display any keys if there are no common-prefix
|
193
|
-
|
210
|
+
delimiter-matching keys
|
194
211
|
|
195
212
|
=== 0.1.9 / (2012-01-23)
|
196
213
|
|
@@ -199,18 +216,18 @@ To install Bash completion for S3 URLs, add the following to ~/.bashrc:
|
|
199
216
|
=== 0.1.8 / (2011-12-29)
|
200
217
|
|
201
218
|
* Fixed: Apparently, File.new(path, File::CREAT|File::WRONLY) does not
|
202
|
-
|
219
|
+
truncate existing files; use File.new(path, "wb") instead.
|
203
220
|
|
204
221
|
=== 0.1.7 / (2011-12-29)
|
205
222
|
|
206
223
|
* Fixed: s3cp would not truncate existing files when overwriting, possibly
|
207
|
-
|
224
|
+
resulting in corrupted files.
|
208
225
|
|
209
226
|
=== 0.1.6 / (2011-12-16)
|
210
227
|
|
211
228
|
* Changed: s3rm now uses multi-object delete operation for faster deletes
|
212
229
|
* Changed: dependency on 'aboisvert-aws' instead of 'right-aws' until
|
213
|
-
|
230
|
+
it supports multi-object delete.
|
214
231
|
|
215
232
|
=== 0.1.5 / 2011-10-17
|
216
233
|
|
@@ -223,7 +240,7 @@ To install Bash completion for S3 URLs, add the following to ~/.bashrc:
|
|
223
240
|
=== 0.1.3 / 2011-09-29
|
224
241
|
|
225
242
|
* Fixed: s3cp --headers names are now converted to lowercase since underlying
|
226
|
-
|
243
|
+
RightAWS gem expects lowercase header names.
|
227
244
|
|
228
245
|
=== 0.1.2 / 2011-09-29
|
229
246
|
|
data/lib/s3cp/s3cp.rb
CHANGED
@@ -119,6 +119,11 @@ op = OptionParser.new do |opts|
|
|
119
119
|
options[:debug] = true
|
120
120
|
end
|
121
121
|
|
122
|
+
opts.on("--version", "Display version information") do
|
123
|
+
puts S3CP::VERSION
|
124
|
+
exit
|
125
|
+
end
|
126
|
+
|
122
127
|
opts.on_tail("-h", "--help", "Show this message") do
|
123
128
|
puts op
|
124
129
|
exit
|
data/lib/s3cp/utils.rb
CHANGED
@@ -39,13 +39,7 @@ module S3CP
|
|
39
39
|
|
40
40
|
# Connect to AWS S3
|
41
41
|
def connect()
|
42
|
-
|
43
|
-
secret_key = ENV["AWS_SECRET_ACCESS_KEY"] || raise("Missing environment variable AWS_SECRET_ACCESS_KEY")
|
44
|
-
|
45
|
-
::AWS::S3.new(
|
46
|
-
:access_key_id => access_key,
|
47
|
-
:secret_access_key => secret_key
|
48
|
-
)
|
42
|
+
::AWS::S3.new()
|
49
43
|
end
|
50
44
|
|
51
45
|
# Parse URL and return bucket and key.
|
data/lib/s3cp/version.rb
CHANGED
metadata
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: s3cp
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash: -
|
4
|
+
hash: -3585171968
|
5
5
|
prerelease: 6
|
6
6
|
segments:
|
7
7
|
- 1
|
8
8
|
- 1
|
9
9
|
- 0
|
10
10
|
- pre
|
11
|
-
-
|
12
|
-
version: 1.1.0.pre.
|
11
|
+
- 2
|
12
|
+
version: 1.1.0.pre.2
|
13
13
|
platform: ruby
|
14
14
|
authors:
|
15
15
|
- Alex Boisvert
|
@@ -17,7 +17,7 @@ autorequire:
|
|
17
17
|
bindir: bin
|
18
18
|
cert_chain: []
|
19
19
|
|
20
|
-
date: 2012-
|
20
|
+
date: 2012-08-09 00:00:00 Z
|
21
21
|
dependencies:
|
22
22
|
- !ruby/object:Gem::Dependency
|
23
23
|
prerelease: false
|