gcloud 0.0.7 → 0.1.0

data/packages/gsutil/boto/docs/source/ec2_tut.rst

@@ -1,86 +0,0 @@
-.. _ec2_tut:
-
-=======================================
-An Introduction to boto's EC2 interface
-=======================================
-
-This tutorial focuses on the boto interface to the Elastic Compute Cloud
-from Amazon Web Services.  This tutorial assumes that you have already
-downloaded and installed boto.
-
-Creating a Connection
----------------------
-
-The first step in accessing EC2 is to create a connection to the service.
-The recommended way of doing this in boto is::
-
-    >>> from boto.ec2
-    >>> conn = boto.ec2.connect_to_region("us-east-1",
-    ...    aws_access_key_id='<aws access key>',
-    ...    aws_secret_access_key='<aws secret key>')
-
-At this point the variable ``conn`` will point to an EC2Connection object.  In
-this example, the AWS access key and AWS secret key are passed in to the method
-explicitly.  Alternatively, you can set the boto config environment variables
-and then simply specify which region you want as follows::
-
-    >>> conn = boto.ec2.connect_to_region("us-east-1")
-
-In either case, conn will point to an EC2Connection object which we will
-use throughout the remainder of this tutorial.
-
-Launching Instances
--------------------
-
-Possibly, the most important and common task you'll use EC2 for is to launch,
-stop and terminate instances. In its most primitive form, you can launch an
-instance as follows::
-
-    >>> conn.run_instances('<ami-image-id>')
-
-This will launch an instance in the specified region with the default parameters.
-You will not be able to SSH into this machine, as it doesn't have a security
-group set. See :doc:`security_groups` for details on creating one.
-
-Now, let's say that you already have a key pair, want a specific type of
-instance, and you have your :doc:`security group <security_groups>` all setup.
-In this case we can use the keyword arguments to accomplish that::
-
-    >>> conn.run_instances(
-            '<ami-image-id>',
-            key_name='myKey',
-            instance_type='c1.xlarge',
-            security_groups=['your-security-group-here'])
-
-The main caveat with the above call is that it is possible to request an
-instance type that is not compatible with the provided AMI (for example, the
-instance was created for a 64-bit instance and you choose a m1.small instance_type).
-For more details on the plethora of possible keyword parameters, be sure to
-check out boto's :doc:`EC2 API reference <ref/ec2>`.
-
-Stopping Instances
-------------------
-Once you have your instances up and running, you might wish to shut them down
-if they're not in use. Please note that this will only de-allocate virtual
-hardware resources (as well as instance store drives), but won't destroy your
-EBS volumes -- this means you'll pay nominal provisioned EBS storage fees
-even if your instance is stopped. To do this, you can do so as follows::
-
-    >>> conn.stop_instances(instance_ids=['instance-id-1','instance-id-2', ...])
-
-This will request a 'graceful' stop of each of the specified instances. If you
-wish to request the equivalent of unplugging your instance(s), simply add
-``force=True`` keyword argument to the call above. Please note that stop
-instance is not allowed with Spot instances.
-
-Terminating Instances
----------------------
-Once you are completely done with your instance and wish to surrender both
-virtual hardware, root EBS volume and all other underlying components
-you can request instance termination. To do so you can use the call bellow::
-
-    >>> conn.terminate_instances(instance_ids=['instance-id-1','instance-id-2', ...])
-
-Please use with care since once you request termination for an instance there
-is no turning back.
-

data/packages/gsutil/boto/docs/source/elb_tut.rst

@@ -1,257 +0,0 @@
-.. _elb_tut:
-
-==========================================================
-An Introduction to boto's Elastic Load Balancing interface
-==========================================================
-
-This tutorial focuses on the boto interface for `Elastic Load Balancing`_
-from Amazon Web Services. This tutorial assumes that you have already
-downloaded and installed boto, and are familiar with the boto ec2 interface.
-
-.. _Elastic Load Balancing: http://aws.amazon.com/elasticloadbalancing/
-
-Elastic Load Balancing Concepts
--------------------------------
-`Elastic Load Balancing`_ (ELB) is intimately connected with Amazon's `Elastic
-Compute Cloud`_ (EC2) service. Using the ELB service allows you to create a load
-balancer - a DNS endpoint and set of ports that distributes incoming requests
-to a set of EC2 instances. The advantages of using a load balancer is that it
-allows you to truly scale up or down a set of backend instances without
-disrupting service. Before the ELB service, you had to do this manually by
-launching an EC2 instance and installing load balancer software on it (nginx,
-haproxy, perlbal, etc.) to distribute traffic to other EC2 instances.
-
-Recall that the EC2 service is split into Regions, which are further
-divided into Availability Zones (AZ).
-For example, the US-East region is divided into us-east-1a, us-east-1b,
-us-east-1c, us-east-1d, and us-east-1e. You can think of AZs as data centers -
-each runs off a different set of ISP backbones and power providers.
-ELB load balancers can span multiple AZs but cannot span multiple regions.
-That means that if you'd like to create a set of instances spanning both the
-US and Europe Regions you'd have to create two load balancers and have some
-sort of other means of distributing requests between the two load balancers.
-An example of this could be using GeoIP techniques to choose the correct load
-balancer, or perhaps DNS round robin. Keep in mind also that traffic is
-distributed equally over all AZs the ELB balancer spans. This means you should
-have an equal number of instances in each AZ if you want to equally distribute
-load amongst all your instances.
-
-.. _Elastic Compute Cloud: http://aws.amazon.com/ec2/
-
-Creating a Connection
----------------------
-
-The first step in accessing ELB is to create a connection to the service.
-
->>> import boto
->>> conn = boto.connect_elb(
-        aws_access_key_id='YOUR-KEY-ID-HERE',
-        aws_secret_access_key='YOUR-SECRET-HERE'
-    )
-
-
-A Note About Regions and Endpoints
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Like EC2, the ELB service has a different endpoint for each region. By default
-the US East endpoint is used. To choose a specific region, instantiate the
-ELBConnection object with that region's information.
-
->>> from boto.regioninfo import RegionInfo
->>> reg = RegionInfo(
-        name='eu-west-1',
-        endpoint='elasticloadbalancing.eu-west-1.amazonaws.com'
-    )
->>> conn = boto.connect_elb(
-        aws_access_key_id='YOUR-KEY-ID-HERE',
-        aws_secret_access_key='YOUR-SECRET-HERE',
-        region=reg
-    )
-
-Another way to connect to an alternative region is like this:
-
->>> import boto.ec2.elb
->>> elb = boto.ec2.elb.connect_to_region('eu-west-1')
-
-Here's yet another way to discover what regions are available and then
-connect to one:
-
->>> import boto.ec2.elb
->>> regions = boto.ec2.elb.regions()
->>> regions
-[RegionInfo:us-east-1,
- RegionInfo:ap-northeast-1,
- RegionInfo:us-west-1,
- RegionInfo:ap-southeast-1,
- RegionInfo:eu-west-1]
->>> elb = regions[-1].connect()
-
-Alternatively, edit your boto.cfg with the default ELB endpoint to use::
-
-    [Boto]
-    elb_region_name = eu-west-1
-    elb_region_endpoint = elasticloadbalancing.eu-west-1.amazonaws.com
-
-Getting Existing Load Balancers
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To retrieve any exiting load balancers:
-
->>> conn.get_all_load_balancers()
-[LoadBalancer:load-balancer-prod, LoadBalancer:load-balancer-staging]
-
-You can also filter by name
-
->>> conn.get_all_load_balancers(load_balancer_names=['load-balancer-prod'])
-[LoadBalancer:load-balancer-prod]
-
-:py:meth:`get_all_load_balancers <boto.ec2.elb.ELBConnection.get_all_load_balancers>`
-returns a :py:class:`boto.resultset.ResultSet` that contains instances
-of :class:`boto.ec2.elb.loadbalancer.LoadBalancer`, each of which abstracts
-access to a load balancer. :py:class:`ResultSet <boto.resultset.ResultSet>`
-works very much like a list.
-
->>> balancers = conn.get_all_load_balancers()
->>> balancers[0]
-[LoadBalancer:load-balancer-prod]
-
-Creating a Load Balancer
-------------------------
-To create a load balancer you need the following:
- #. The specific **ports and protocols** you want to load balancer over, and what port
-    you want to connect to all instances.
- #. A **health check** - the ELB concept of a *heart beat* or *ping*. ELB will use this health
-    check to see whether your instances are up or down. If they go down, the load balancer
-    will no longer send requests to them.
- #. A **list of Availability Zones** you'd like to create your load balancer over.
-
-Ports and Protocols
-^^^^^^^^^^^^^^^^^^^
-An incoming connection to your load balancer will come on one or more ports -
-for example 80 (HTTP) and 443 (HTTPS). Each can be using a protocol -
-currently, the supported protocols are TCP and HTTP.  We also need to tell the
-load balancer which port to route connects *to* on each instance.  For example,
-to create a load balancer for a website that accepts connections on 80 and 443,
-and that routes connections to port 8080 and 8443 on each instance, you would
-specify that the load balancer ports and protocols are:
-
- * 80, 8080, HTTP
- * 443, 8443, TCP
-
-This says that the load balancer will listen on two ports - 80 and 443.
-Connections on 80 will use an HTTP load balancer to forward connections to port
-8080 on instances. Likewise, the load balancer will listen on 443 to forward
-connections to 8443 on each instance using the TCP balancer. We need to
-use TCP for the HTTPS port because it is encrypted at the application
-layer. Of course, we could specify the load balancer use TCP for port 80,
-however specifying HTTP allows you to let ELB handle some work for you -
-for example HTTP header parsing.
-
-.. _elb-configuring-a-health-check:
-
-Configuring a Health Check
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-A health check allows ELB to determine which instances are alive and able to
-respond to requests. A health check is essentially a tuple consisting of:
-
- * *Target*: What to check on an instance. For a TCP check this is comprised of::
-
-        TCP:PORT_TO_CHECK
-
-   Which attempts to open a connection on PORT_TO_CHECK. If the connection opens
-   successfully, that specific instance is deemed healthy, otherwise it is marked
-   temporarily as unhealthy. For HTTP, the situation is slightly different::
-
-        HTTP:PORT_TO_CHECK/RESOURCE
-
-   This means that the health check will connect to the resource /RESOURCE on
-   PORT_TO_CHECK. If an HTTP 200 status is returned the instance is deemed healthy.
- * *Interval*: How often the check is made. This is given in seconds and defaults
-   to 30. The valid range of intervals goes from 5 seconds to 600 seconds.
- * *Timeout*: The number of seconds the load balancer will wait for a check to
-   return a result.
- * *Unhealthy threshold*: The number of consecutive failed checks to deem the
-   instance as being dead. The default is 5, and the range of valid values lies
-   from 2 to 10.
-
-The following example creates a health check called *instance_health* that
-simply checks instances every 20 seconds on port 80 over HTTP at the
-resource /health for 200 successes.
-
->>> from boto.ec2.elb import HealthCheck
->>> hc = HealthCheck(
-        interval=20,
-        healthy_threshold=3,
-        unhealthy_threshold=5,
-        target='HTTP:8080/health'
-    )
-
-Putting It All Together
-^^^^^^^^^^^^^^^^^^^^^^^
-
-Finally, let's create a load balancer in the US region that listens on ports
-80 and 443 and distributes requests to instances on 8080 and 8443 over HTTP
-and TCP. We want the load balancer to span the availability zones
-*us-east-1a* and *us-east-1b*:
-
->>> regions = ['us-east-1a', 'us-east-1b']
->>> ports = [(80, 8080, 'http'), (443, 8443, 'tcp')]
->>> lb = conn.create_load_balancer('my-lb', regions, ports)
->>> # This is from the previous section.
->>> lb.configure_health_check(hc)
-
-The load balancer has been created. To see where you can actually connect to
-it, do:
-
->>> print lb.dns_name
-my_elb-123456789.us-east-1.elb.amazonaws.com
-
-You can then CNAME map a better name, i.e. www.MYWEBSITE.com to the
-above address.
-
-Adding Instances To a Load Balancer
------------------------------------
-
-Now that the load balancer has been created, there are two ways to add
-instances to it:
-
- #. Manually, adding each instance in turn.
- #. Mapping an autoscale group to the load balancer. Please see the
-    :doc:`Autoscale tutorial <autoscale_tut>` for information on how to do this.
-
-Manually Adding and Removing Instances
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Assuming you have a list of instance ids, you can add them to the load balancer
-
->>> instance_ids = ['i-4f8cf126', 'i-0bb7ca62']
->>> lb.register_instances(instance_ids)
-
-Keep in mind that these instances should be in Security Groups that match the
-internal ports of the load balancer you just created (for this example, they
-should allow incoming connections on 8080 and 8443).
-
-To remove instances:
-
->>> lb.deregister_instances(instance_ids)
-
-Modifying Availability Zones for a Load Balancer
-------------------------------------------------
-
-If you wanted to disable one or more zones from an existing load balancer:
-
->>> lb.disable_zones(['us-east-1a'])
-
-You can then terminate each instance in the disabled zone and then deregister then from your load
-balancer.
-
-To enable zones:
-
->>> lb.enable_zones(['us-east-1c'])
-
-Deleting a Load Balancer
-------------------------
-
->>> lb.delete()
-
-

data/packages/gsutil/boto/docs/source/emr_tut.rst

@@ -1,108 +0,0 @@
-.. _emr_tut:
-
-=====================================================
-An Introduction to boto's Elastic Mapreduce interface
-=====================================================
-
-This tutorial focuses on the boto interface to Elastic Mapreduce from
-Amazon Web Services.  This tutorial assumes that you have already
-downloaded and installed boto.
-
-Creating a Connection
----------------------
-The first step in accessing Elastic Mapreduce is to create a connection
-to the service.  There are two ways to do this in boto.  The first is:
-
->>> from boto.emr.connection import EmrConnection
->>> conn = EmrConnection('<aws access key>', '<aws secret key>')
-
-At this point the variable conn will point to an EmrConnection object.
-In this example, the AWS access key and AWS secret key are passed in to
-the method explicitly.  Alternatively, you can set the environment variables:
-
-AWS_ACCESS_KEY_ID - Your AWS Access Key ID \
-AWS_SECRET_ACCESS_KEY - Your AWS Secret Access Key
-
-and then call the constructor without any arguments, like this:
-
->>> conn = EmrConnection()
-
-There is also a shortcut function in the boto package called connect_emr
-that may provide a slightly easier means of creating a connection:
-
->>> import boto
->>> conn = boto.connect_emr()
-
-In either case, conn points to an EmrConnection object which we will use
-throughout the remainder of this tutorial.
-
-Creating Streaming JobFlow Steps
---------------------------------
-Upon creating a connection to Elastic Mapreduce you will next 
-want to create one or more jobflow steps.  There are two types of steps, streaming
-and custom jar, both of which have a class in the boto Elastic Mapreduce implementation.
-
-Creating a streaming step that runs the AWS wordcount example, itself written in Python, can be accomplished by:
-
->>> from boto.emr.step import StreamingStep
->>> step = StreamingStep(name='My wordcount example',
-...                      mapper='s3n://elasticmapreduce/samples/wordcount/wordSplitter.py',
-...                      reducer='aggregate',
-...                      input='s3n://elasticmapreduce/samples/wordcount/input',
-...                      output='s3n://<my output bucket>/output/wordcount_output')
-
-where <my output bucket> is a bucket you have created in S3.
-
-Note that this statement does not run the step, that is accomplished later when we create a jobflow.  
-
-Additional arguments of note to the streaming jobflow step are cache_files, cache_archive and step_args.  The options cache_files and cache_archive enable you to use the Hadoops distributed cache to share files amongst the instances that run the step.  The argument step_args allows one to pass additional arguments to Hadoop streaming, for example modifications to the Hadoop job configuration.
-
-Creating Custom Jar Job Flow Steps
-----------------------------------
-
-The second type of jobflow step executes tasks written with a custom jar.  Creating a custom jar step for the AWS CloudBurst example can be accomplished by:
-
->>> from boto.emr.step import JarStep
->>> step = JarStep(name='Coudburst example',
-...                jar='s3n://elasticmapreduce/samples/cloudburst/cloudburst.jar',
-...                step_args=['s3n://elasticmapreduce/samples/cloudburst/input/s_suis.br',
-...                           's3n://elasticmapreduce/samples/cloudburst/input/100k.br',
-...                           's3n://<my output bucket>/output/cloudfront_output',
-...                            36, 3, 0, 1, 240, 48, 24, 24, 128, 16])
-
-Note that this statement does not actually run the step, that is accomplished later when we create a jobflow.  Also note that this JarStep does not include a main_class argument since the jar MANIFEST.MF has a Main-Class entry.
-
-Creating JobFlows
------------------
-Once you have created one or more jobflow steps, you will next want to create and run a jobflow.  Creating a jobflow that executes either of the steps we created above can be accomplished by:
-
->>> import boto
->>> conn = boto.connect_emr()
->>> jobid = conn.run_jobflow(name='My jobflow', 
-...                          log_uri='s3://<my log uri>/jobflow_logs', 
-...                          steps=[step])
-
-The method will not block for the completion of the jobflow, but will immediately return.  The status of the jobflow can be determined by:
-
->>> status = conn.describe_jobflow(jobid)
->>> status.state
-u'STARTING'
-
-One can then use this state to block for a jobflow to complete.  Valid jobflow states currently defined in the AWS API are COMPLETED, FAILED, TERMINATED, RUNNING, SHUTTING_DOWN, STARTING and WAITING.
-
-In some cases you may not have built all of the steps prior to running the jobflow.  In these cases additional steps can be added to a jobflow by running:
-
->>> conn.add_jobflow_steps(jobid, [second_step])
-
-If you wish to add additional steps to a running jobflow you may want to set the keep_alive parameter to True in run_jobflow so that the jobflow does not automatically terminate when the first step completes.
-
-The run_jobflow method has a number of important parameters that are worth investigating.  They include parameters to change the number and type of EC2 instances on which the jobflow is executed, set a SSH key for manual debugging and enable AWS console debugging.
-
-Terminating JobFlows
---------------------
-By default when all the steps of a jobflow have finished or failed the jobflow terminates.  However, if you set the keep_alive parameter to True or just want to halt the execution of a jobflow early you can terminate a jobflow by:
-
->>> import boto
->>> conn = boto.connect_emr()
->>> conn.terminate_jobflow('<jobflow id>') 
-