gcloud 0.0.4 → 0.0.6

data/packages/gsutil/boto/docs/source/security_groups.rst

@@ -0,0 +1,82 @@
+.. _security_groups:
+
+===================
+EC2 Security Groups
+===================
+
+Amazon defines a security group as:
+
+"A security group is a named collection of access rules.  These access rules
+ specify which ingress, i.e. incoming, network traffic should be delivered
+ to your instance."
+
+To get a listing of all currently defined security groups::
+
+    >>> rs = conn.get_all_security_groups()
+    >>> print rs
+    [SecurityGroup:appserver, SecurityGroup:default, SecurityGroup:vnc, SecurityGroup:webserver]
+
+Each security group can have an arbitrary number of rules which represent
+different network ports which are being enabled.  To find the rules for a
+particular security group, use the rules attribute::
+
+    >>> sg = rs[1]
+    >>> sg.name
+    u'default'
+    >>> sg.rules
+    [IPPermissions:tcp(0-65535),
+     IPPermissions:udp(0-65535),
+     IPPermissions:icmp(-1--1),
+     IPPermissions:tcp(22-22),
+     IPPermissions:tcp(80-80)]
+
+In addition to listing the available security groups you can also create
+a new security group.  I'll follow through the "Three Tier Web Service"
+example included in the EC2 Developer's Guide for an example of how to
+create security groups and add rules to them.
+
+First, let's create a group for our Apache web servers that allows HTTP
+access to the world::
+
+    >>> web = conn.create_security_group('apache', 'Our Apache Group')
+    >>> web
+    SecurityGroup:apache
+    >>> web.authorize('tcp', 80, 80, '0.0.0.0/0')
+    True
+
+The first argument is the ip protocol which can be one of; tcp, udp or icmp.
+The second argument is the FromPort or the beginning port in the range, the
+third argument is the ToPort or the ending port in the range and the last
+argument is the CIDR IP range to authorize access to.
+
+Next we create another group for the app servers::
+
+    >>> app = conn.create_security_group('appserver', 'The application tier')
+
+We then want to grant access between the web server group and the app
+server group.  So, rather than specifying an IP address as we did in the
+last example, this time we will specify another SecurityGroup object.:
+
+    >>> app.authorize(src_group=web)
+    True
+
+Now, to verify that the web group now has access to the app servers, we want to
+temporarily allow SSH access to the web servers from our computer.  Let's
+say that our IP address is 192.168.1.130 as it is in the EC2 Developer
+Guide.  To enable that access::
+
+    >>> web.authorize(ip_protocol='tcp', from_port=22, to_port=22, cidr_ip='192.168.1.130/32')
+    True
+
+Now that this access is authorized, we could ssh into an instance running in
+the web group and then try to telnet to specific ports on servers in the
+appserver group, as shown in the EC2 Developer's Guide.  When this testing is
+complete, we would want to revoke SSH access to the web server group, like this::
+
+    >>> web.rules
+    [IPPermissions:tcp(80-80),
+     IPPermissions:tcp(22-22)]
+    >>> web.revoke('tcp', 22, 22, cidr_ip='192.168.1.130/32')
+    True
+    >>> web.rules
+    [IPPermissions:tcp(80-80)]

data/packages/gsutil/boto/docs/source/ses_tut.rst

@@ -0,0 +1,171 @@
+.. ses_tut:
+
+=============================
+Simple Email Service Tutorial
+=============================
+
+This tutorial focuses on the boto interface to AWS' `Simple Email Service (SES) <ses>`_.
+This tutorial assumes that you have boto already downloaded and installed.
+
+.. _SES: http://aws.amazon.com/ses/
+
+Creating a Connection
+---------------------
+
+The first step in accessing SES is to create a connection to the service.
+To do so, the most straight forward way is the following::
+
+    >>> import boto
+    >>> conn = boto.connect_ses(
+            aws_access_key_id='<YOUR_AWS_KEY_ID>',
+            aws_secret_access_key='<YOUR_AWS_SECRET_KEY>')
+    >>> conn
+    SESConnection:email.us-east-1.amazonaws.com
+
+Bear in mind that if you have your credentials in boto config in your home
+directory, the two keyword arguments in the call above are not needed. More
+details on configuration can be fond in :doc:`boto_config_tut`.
+
+The :py:func:`boto.connect_ses` functions returns a
+:py:class:`boto.ses.connection.SESConnection` instance, which is a the boto API
+for working with SES.
+
+Notes on Sending
+----------------
+
+It is important to keep in mind that while emails appear to come "from" the
+address that you specify via Reply-To, the sending is done through Amazon.
+Some clients do pick up on this disparity, and leave a note on emails.
+
+Verifying a Sender Email Address
+--------------------------------
+
+Before you can send email "from" an address, you must prove that you have
+access to the account. When you send a validation request, an email is sent
+to the address with a link in it. Clicking on the link validates the address
+and adds it to your SES account. Here's how to send the validation email::
+
+    >>> conn.verify_email_address('some@address.com')
+    {
+        'VerifyEmailAddressResponse': {
+            'ResponseMetadata': {
+                'RequestId': '4a974fd5-56c2-11e1-ad4c-c1f08c91d554'
+            }
+        }
+    }
+
+After a short amount of time, you'll find an email with the validation
+link inside. Click it, and this address may be used to send emails.
+
+Listing Verified Addresses
+--------------------------
+
+If you'd like to list the addresses that are currently verified on your
+SES account, use
+:py:meth:`list_verified_email_addresses <boto.ses.connection.SESConnection.list_verified_email_addresses>`::
+
+    >>> conn.list_verified_email_addresses()
+    {
+        'ListVerifiedEmailAddressesResponse': {
+            'ListVerifiedEmailAddressesResult': {
+                'VerifiedEmailAddresses': [
+                    'some@address.com',
+                    'another@address.com'
+                ]
+            },
+            'ResponseMetadata': {
+                'RequestId': '2ab45c18-56c3-11e1-be66-ffd2a4549d70'
+            }
+        }
+    }
+
+Deleting a Verified Address
+---------------------------
+
+In the event that you'd like to remove an email address from your account,
+use
+:py:meth:`delete_verified_email_address <boto.ses.connection.SESConnection.delete_verified_email_address>`::
+
+    >>> conn.delete_verified_email_address('another@address.com')
+
+Sending an Email
+----------------
+
+Sending an email is done via
+:py:meth:`send_email <boto.ses.connection.SESConnection.send_email>`::
+
+    >>> conn.send_email(
+            'some@address.com',
+            'Your subject',
+            'Body here',
+            ['recipient-address-1@gmail.com'])
+    {
+        'SendEmailResponse': {
+            'ResponseMetadata': {
+                'RequestId': '4743c2b7-56c3-11e1-bccd-c99bd68002fd'
+            },
+            'SendEmailResult': {
+                'MessageId': '000001357a177192-7b894025-147a-4705-8455-7c880b0c8270-000000'
+            }
+        }
+    }
+
+If you're wanting to send a multipart MIME email, see the reference for
+:py:meth:`send_raw_email <boto.ses.connection.SESConnection.send_raw_email>`,
+which is a bit more of a low-level alternative.
+
+Checking your Send Quota
+------------------------
+
+Staying within your quota is critical, since the upper limit is a hard cap.
+Once you have hit your quota, no further email may be sent until enough
+time elapses to where your 24 hour email count (rolling continuously) is
+within acceptable ranges. Use
+:py:meth:`get_send_quota <boto.ses.connection.SESConnection.get_send_quota>`::
+
+    >>> conn.get_send_quota()
+    {
+        'GetSendQuotaResponse': {
+            'GetSendQuotaResult': {
+                'Max24HourSend': '100000.0',
+                'SentLast24Hours': '181.0',
+                'MaxSendRate': '28.0'
+            },
+            'ResponseMetadata': {
+                'RequestId': u'8a629245-56c4-11e1-9c53-9d5f4d2cc8d3'
+            }
+        }
+    }
+
+Checking your Send Statistics
+-----------------------------
+
+In order to fight spammers and ensure quality mail is being sent from SES,
+Amazon tracks bounces, rejections, and complaints. This is done via
+:py:meth:`get_send_statistics <boto.ses.connection.SESConnection.get_send_statistics>`.
+Please be warned that the output is extremely verbose, to the point
+where we'll just show a short excerpt here::
+
+    >>> conn.get_send_statistics()
+    {
+        'GetSendStatisticsResponse': {
+            'GetSendStatisticsResult': {
+                'SendDataPoints': [
+                    {
+                        'Complaints': '0',
+                        'Timestamp': '2012-02-13T05:02:00Z',
+                        'DeliveryAttempts': '8',
+                        'Bounces': '0',
+                        'Rejects': '0'
+                    },
+                    {
+                        'Complaints': '0',
+                        'Timestamp': '2012-02-13T05:17:00Z',
+                        'DeliveryAttempts': '12',
+                        'Bounces': '0',
+                        'Rejects': '0'
+                    }
+                ]
+            }
+        }
+    }

data/packages/gsutil/boto/docs/source/simpledb_tut.rst

@@ -0,0 +1,188 @@
+.. simpledb_tut:
+
+============================================
+An Introduction to boto's SimpleDB interface
+============================================
+This tutorial focuses on the boto interface to AWS' SimpleDB_. This tutorial
+assumes that you have boto already downloaded and installed.
+
+.. _SimpleDB: http://aws.amazon.com/simpledb/
+
+Creating a Connection
+---------------------
+The first step in accessing SimpleDB is to create a connection to the service.
+To do so, the most straight forward way is the following::
+
+    >>> import boto
+    >>> conn = boto.connect_sdb(aws_access_key_id='<YOUR_AWS_KEY_ID>',aws_secret_access_key='<YOUR_AWS_SECRET_KEY>')
+    >>> conn
+    SDBConnection:sdb.amazonaws.com
+    >>>
+
+Bear in mind that if you have your credentials in boto config in your home
+directory, the two keyword arguments in the call above are not needed. Also
+important to note is that just as any other AWS service, SimpleDB is
+region-specific and as such you might want to specify which region to connect
+to, by default, it'll connect to the US-EAST-1 region.
+
+Creating Domains
+----------------
+Arguably, once you have your connection established, you'll want to create one or more dmains.
+Creating new domains is a fairly straight forward operation. To do so, you can proceed as follows::
+
+    >>> conn.create_domain('test-domain')
+    Domain:test-domain
+    >>>
+    >>> conn.create_domain('test-domain-2')
+    Domain:test-domain
+    >>>
+
+Please note that SimpleDB, unlike its newest sibling DynamoDB, is truly and completely schema-less. 
+Thus, there's no need specify domain keys or ranges.
+
+Listing All Domains
+-------------------
+Unlike DynamoDB or other database systems, SimpleDB uses the concept of 'domains' instead of tables.
+So, to list all your domains for your account in a region, you can simply do as follows::
+
+    >>> domains = conn.get_all_domains()
+    >>> domains
+    [Domain:test-domain, Domain:test-domain-2]
+    >>>
+
+The get_all_domains() method returns a :py:class:`boto.resultset.ResultSet` containing
+all :py:class:`boto.sdb.domain.Domain` objects associated with
+this connection's Access Key ID for that region.
+
+Retrieving a Domain (by name)
+-----------------------------
+If you wish to retrieve a specific domain whose name is known, you can do so as follows::
+
+    >>> dom = conn.get_domain('test-domain')
+    >>> dom
+    Domain:test-domain
+    >>>
+
+The get_domain call has an optional validate parameter, which defaults to True. This will make sure to raise
+an exception if the domain you are looking for doesn't exist. If you set it to false, it will return a 
+:py:class:`Domain <boto.sdb.domain.Domain>` object blindly regardless of its existence. 
+
+Getting Domain Metadata
+------------------------
+There are times when you might want to know your domains' machine usage, aprox. item count and other such data.
+To this end, boto offers a simple and convenient way to do so as shown below::
+
+    >>> domain_meta = conn.domain_metadata(dom)
+    >>> domain_meta
+    <boto.sdb.domain.DomainMetaData instance at 0x23cd440>
+    >>> dir(domain_meta)
+    ['BoxUsage', 'DomainMetadataResponse', 'DomainMetadataResult', 'RequestId', 'ResponseMetadata', 
+    '__doc__', '__init__', '__module__', 'attr_name_count', 'attr_names_size', 'attr_value_count', 'attr_values_size', 
+    'domain', 'endElement', 'item_count', 'item_names_size', 'startElement', 'timestamp']
+    >>> domain_meta.item_count
+    0
+    >>>
+
+Please bear in mind that while in the example above we used a previously retrieved domain object as the parameter, you
+can retrieve the domain metadata via its name (string).
+
+Adding Items (and attributes)
+-----------------------------
+Once you have your domain setup, presumably, you'll want to start adding items to it.
+In its most straight forward form, you need to provide a name for the item -- think of it 
+as a record id -- and a collection of the attributes you want to store in the item (often a Dictionary-like object). 
+So, adding an item to a domain looks as follows::
+
+    >>> item_name = 'ABC_123'
+    >>> item_attrs = {'Artist': 'The Jackson 5', 'Genera':'Pop'}
+    >>> dom.put_attributes(item_name, item_attrs)
+    True
+    >>>
+
+Now let's check if it worked::
+
+    >>> domain_meta = conn.domain_metadata(dom)
+    >>> domain_meta.item_count
+    1
+    >>>
+
+
+Batch Adding Items (and attributes)
+-----------------------------------
+You can also add a number of items at the same time in a similar fashion. All you have to provide to the batch_put_attributes() method 
+is a Dictionary-like object with your items and their respective attributes, as follows::
+
+    >>> items = {'item1':{'attr1':'val1'},'item2':{'attr2':'val2'}}
+    >>> dom.batch_put_attributes(items)
+    True
+    >>>
+
+Now, let's check the item count once again::
+
+    >>> domain_meta = conn.domain_metadata(dom)
+    >>> domain_meta.item_count
+    3
+    >>>
+
+A few words of warning: both batch_put_attributes() and put_item(), by default, will overwrite the values of the attributes if both 
+the item and attribute already exist. If the item exists, but not the attributes, it will append the new attributes to the 
+attribute list of that item. If you do not wish these methods to behave in that manner, simply supply them with a 'replace=False'
+parameter.
+
+
+Retrieving Items
+-----------------
+To retrieve an item along with its attributes is a fairly straight forward operation and can be accomplished as follows::
+
+    >>> dom.get_item('item1')
+    {u'attr1': u'val1'}
+    >>>
+
+Since SimpleDB works in an "eventual consistency" manner, we can also request a forced consistent read (though this will 
+invariably adversely affect read performance). The way to accomplish that is as shown below::
+
+    >>> dom.get_item('item1', consistent_read=True)
+    {u'attr1': u'val1'}
+    >>>
+
+Retrieving One or More Items
+----------------------------
+Another way to retrieve items is through boto's select() method. This method, at the bare minimum, requires a standard SQL select query string 
+and you would do something along the lines of::
+
+    >>> query = 'select * from `test-domain` where attr1="val1"'
+    >>> rs = dom.select(query)
+    >>> for j in rs:
+    ...   print 'o hai'
+    ... 
+    o hai
+    >>>
+
+This method returns a ResultSet collection you can iterate over.
+
+Updating Item Attributes
+------------------------
+The easiest way to modify an item's attributes is by manipulating the item's attributes and then saving those changes. For example::
+
+    >>> item = dom.get_item('item1')
+    >>> item['attr1'] = 'val_changed'
+    >>> item.save()
+
+
+Deleting Items (and its attributes)
+-----------------------------------
+Deleting an item is a very simple operation. All you are required to provide is either the name of the item or an item object to the 
+delete_item() method, boto will take care of the rest::
+
+    >>>dom.delete_item(item)
+    >>>True
+
+    
+
+Deleting Domains
+-----------------------------------
+To delete a domain and all items under it (i.e. be very careful), you can do it as follows::
+
+    >>> conn.delete_domain('test-domain')
+    True
+    >>>

data/packages/gsutil/boto/docs/source/sqs_tut.rst

@@ -0,0 +1,246 @@
+.. _sqs_tut:
+
+=======================================
+An Introduction to boto's SQS interface
+=======================================
+
+This tutorial focuses on the boto interface to the Simple Queue Service
+from Amazon Web Services.  This tutorial assumes that you have boto already
+downloaded and installed.
+
+Creating a Connection
+---------------------
+The first step in accessing SQS is to create a connection to the service.
+The recommended method of doing this is as follows::
+
+    >>> import boto.sqs
+    >>> conn = boto.sqs.connect_to_region(
+    ...     "us-east-1",
+    ...     aws_access_key_id='<aws access key'>,
+    ...     aws_secret_access_key='<aws secret key>')
+
+At this point the variable conn will point to an SQSConnection object in the
+US-EAST-1 region. Bear in mind that just as any other AWS service, SQS is
+region-specific. In this example, the AWS access key and AWS secret key are
+passed in to the method explicitely. Alternatively, you can set the environment
+variables:
+
+* ``AWS_ACCESS_KEY_ID`` - Your AWS Access Key ID
+* ``AWS_SECRET_ACCESS_KEY`` - Your AWS Secret Access Key
+
+and then simply call::
+
+    >>> import boto.sqs
+    >>> conn = boto.sqs.connect_to_region("us-east-1")
+
+In either case, conn will point to an SQSConnection object which we will
+use throughout the remainder of this tutorial.
+
+Creating a Queue
+----------------
+Once you have a connection established with SQS, you will probably want to
+create a queue.  In its simplest form, that can be accomplished as follows::
+
+    >>> q = conn.create_queue('myqueue')
+
+The create_queue method will create (and return) the requested queue if it does not
+exist or will return the existing queue if it does.  There is an
+optional parameter to create_queue called visibility_timeout.  This basically
+controls how long a message will remain invisible to other queue readers
+once it has been read (see SQS documentation for more detailed explanation).
+If this is not explicitly specified the queue will be created with whatever
+default value SQS provides (currently 30 seconds).  If you would like to
+specify another value, you could do so like this::
+
+    >>> q = conn.create_queue('myqueue', 120)
+
+This would establish a default visibility timeout for this queue of 120
+seconds.  As you will see later on, this default value for the queue can
+also be overridden each time a message is read from the queue.  If you want
+to check what the default visibility timeout is for a queue::
+
+    >>> q.get_timeout()
+    30
+
+Listing all Queues
+------------------
+
+To retrieve a list of the queues for your account in the current region::
+
+    >>> conn.get_all_queues()
+    [
+        Queue(https://queue.amazonaws.com/411358162645/myqueue),
+        Queue(https://queue.amazonaws.com/411358162645/another_queue),
+        Queue(https://queue.amazonaws.com/411358162645/another_queue2)
+    ]
+
+This will leave you with a list of all of your :py:class:`boto.sqs.queue.Queue`
+instances. Alternatively, if you wanted to only list the queues that started
+with ``'another'``::
+
+    >>> conn.get_all_queues(prefix='another')
+    [
+        Queue(https://queue.amazonaws.com/411358162645/another_queue),
+        Queue(https://queue.amazonaws.com/411358162645/another_queue2)
+    ]
+
+Getting a Queue (by name)
+-------------------------
+If you wish to explicitly retrieve an existing queue and the name of the queue is known,
+you can retrieve the queue as follows::
+
+    >>> my_queue = conn.get_queue('myqueue')
+    Queue(https://queue.amazonaws.com/411358162645/myqueue)
+
+This leaves you with a single :py:class:`boto.sqs.queue.Queue`, which abstracts
+the SQS Queue named 'myqueue'.
+
+Writing Messages
+----------------
+Once you have a queue setup, presumably you will want to write some messages
+to it.  SQS doesn't care what kind of information you store in your messages
+or what format you use to store it.  As long as the amount of data per
+message is less than or equal to 256Kb, SQS won't complain.
+
+So, first we need to create a Message object::
+
+>>> from boto.sqs.message import Message
+>>> m = Message()
+>>> m.set_body('This is my first message.')
+>>> status = q.write(m)
+
+The write method returns a True if everything went well.  If the write
+didn't succeed it will either return a False (meaning SQS simply chose
+not to write the message for some reason) or an exception if there was
+some sort of problem with the request.
+
+Writing Messages (Custom Format)
+--------------------------------
+The technique above will work only if you use boto's default Message payload format;
+however, you may have a lot of specific requirements around the format of
+the message data.  For example, you may want to store one big string or you might
+want to store something that looks more like RFC822 messages or you might want
+to store a binary payload such as pickled Python objects.
+
+The way boto deals with this issue is to define a simple Message object that
+treats the message data as one big string which you can set and get.  If that
+Message object meets your needs, you're good to go.  However, if you need to
+incorporate different behavior in your message or handle different types of
+data you can create your own Message class.  You just need to register that
+class with the boto queue object so that it knows that, when you read a message from the
+queue, it should create one of your message objects rather than the
+default boto Message object.  To register your message class, you would::
+
+>>> import MyMessage
+>>> q.set_message_class(MyMessage)
+>>> m = MyMessage()
+>>> m.set_body('This is my first message.')
+>>> status = q.write(m)
+
+where MyMessage is the class definition for your message class.  Your
+message class should subclass the boto Message because there is a small
+bit of Python magic happening in the __setattr__ method of the boto Message
+class.
+
+Reading Messages
+----------------
+
+So, now we have a message in our queue.  How would we go about reading it?
+Here's one way:
+
+>>> rs = q.get_messages()
+>>> len(rs)
+1
+>>> m = rs[0]
+>>> m.get_body()
+u'This is my first message'
+
+The get_messages method also returns a ResultSet object as described
+above.  In addition to the special attributes that we already talked
+about the ResultSet object also contains any results returned by the
+request.  To get at the results you can treat the ResultSet as a
+sequence object (e.g. a list).  We can check the length (how many results)
+and access particular items within the list using the slice notation
+familiar to Python programmers.
+
+At this point, we have read the message from the queue and SQS will make
+sure that this message remains invisible to other readers of the queue
+until the visibility timeout period for the queue expires.  If you delete
+the message before the timeout period expires then no one else will ever see
+the message again.  However, if you don't delete it (maybe because your reader crashed
+or failed in some way, for example) it will magically reappear in my queue
+for someone else to read.  If you aren't happy with the default visibility
+timeout defined for the queue, you can override it when you read a message:
+
+>>> q.get_messages(visibility_timeout=60)
+
+This means that regardless of what the default visibility timeout is for
+the queue, this message will remain invisible to other readers for 60
+seconds.
+
+The get_messages method can also return more than a single message.  By
+passing a num_messages parameter (defaults to 1) you can control the maximum
+number of messages that will be returned by the method.  To show this
+feature off, first let's load up a few more messages.
+
+>>> for i in range(1, 11):
+...   m = Message()
+...   m.set_body('This is message %d' % i)
+...   q.write(m)
+...
+>>> rs = q.get_messages(10)
+>>> len(rs)
+10
+
+Don't be alarmed if the length of the result set returned by the get_messages
+call is less than 10.  Sometimes it takes some time for new messages to become
+visible in the queue.  Give it a minute or two and they will all show up.
+
+If you want a slightly simpler way to read messages from a queue, you
+can use the read method.  It will either return the message read or
+it will return None if no messages were available.  You can also pass
+a visibility_timeout parameter to read, if you desire:
+
+>>> m = q.read(60)
+>>> m.get_body()
+u'This is my first message'
+
+Deleting Messages and Queues
+----------------------------
+As stated above, messages are never deleted by the queue unless explicitly told to do so.
+To remove a message from a queue:
+
+>>> q.delete_message(m)
+[]
+
+If I want to delete the entire queue, I would use:
+
+>>> conn.delete_queue(q)
+
+However, and this is a good safe guard, this won't succeed unless the queue is empty.
+
+Additional Information
+----------------------
+The above tutorial covers the basic operations of creating queues, writing messages,
+reading messages, deleting messages, and deleting queues.  There are a
+few utility methods in boto that might be useful as well.  For example,
+to count the number of messages in a queue:
+
+>>> q.count()
+10
+
+This can be handy but is command as well as the other two utility methods
+I'll describe in a minute are inefficient and should be used with caution
+on queues with lots of messages (e.g. many hundreds or more).  Similarly,
+you can clear (delete) all messages in a queue with:
+
+>>> q.clear()
+
+Be REAL careful with that one!  Finally, if you want to dump all of the
+messages in a queue to a local file:
+
+>>> q.dump('messages.txt', sep='\n------------------\n')
+
+This will read all of the messages in the queue and write the bodies of
+each of the messages to the file messages.txt.  The option sep argument
+is a separator that will be printed between each message body in the file.