amps-python-client 5.3.4.1__zip → 5.3.4.2__zip

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/amps-statistics.rst.txt

@@ -1,358 +0,0 @@
-.. _amps-statistics:
-
-The AMPS Statistics Database
-============================
-
-AMPS provides the ability to record the statistics gathered from the
-AMPS instance and the host machine. 
-
-The AMPS statistics database is stored in sqlite3 format and can be
-used with any of the standard sqlite3 tools. This appendix assumes that
-you are using the standard ``sqlite3`` package installed on your local
-computer. While you may be able to run the SQL examples in this guide
-using other packages, this guide will assume that all SQL commands will
-be executed with ``sqlite3``.
-
-Notice that the statistics subsystem is independent of the other
-subsystems in AMPS, and is the only part of AMPS that uses the sqlite3
-format. You cannot use sqlite3 tools with SOW files, journal files or
-.ack files: these files use formats specifically designed for high
-performance messaging.
-
-Working with the AMPS statistics database is described in more detail 
-in the following sections.
-
-Configuring AMPS to Persist Statistics
---------------------------------------
-
-By default, AMPS maintains statistics in memory. To configure AMPS to
-record the statistics to a file, the following configuration options are
-available in the AMPS configuration file to update the location and
-frequency of the statistics database file.
-
-.. code-block:: xml
-
-    <AMPSConfig>
-        
-        ...
-
-        <Admin>
-            <InetAddr>localhost:9090</InetAddr>
-            <FileName>./stats.db</FileName>
-            <Interval>5s</Interval>
-        </Admin>
-     
-        ...
-
-    </AMPSConfig>
-
-In the example above, the AMPS administration interface is set to
-collect statistics every 5 seconds as indicated by the ``<Interval>``
-tag. The AMPS administration interface is additionally configured to
-save the statistics in the ``stats.db`` file, which will be created
-in the directory where AMPS was started.
-
-Introduction to SQLite3
-------------------------
-
-This section is a quick reference to sqlite3. It is intended to help in 
-getting started with examining the statistics provided by AMPS. While
-this guide will be sufficient to execute the examples listed, a more 
-comprehensive guide of the sqlite3 command line tool is available at
-http://www.sqlite.org/sqlite.html.
-
-Starting SQLite3
-^^^^^^^^^^^^^^^^^
-
-To start sqlite3 with the stats.db file simply type:
-
-.. code-block:: bash
-
-     $> sqlite3 ./stats.db
-
-This will create a command prompt that looks like the following:
-
-.. code-block:: bash
-
-    $> sqlite3 ./stats.db
-    SQLite version 3.7.3
-    Enter ".help" for instructions
-    Enter SQL statements terminated with a ";"
-    sqlite>
-
-To exit the sqlite3 prompt at any time, use the Ctrl+d sequence.
-
-Simple SQLite3 Commands
-^^^^^^^^^^^^^^^^^^^^^^^
-
-Tables
-~~~~~~
-
-To get a listing of all available tables in the sqlite database type the
-``.table`` command.
-
-.. build note: the documentation repository contains a bash script that takes a database and produces nicely-formatted output of the tables in the script
-
-.. code-block:: bash
-
-    sqlite> .table
-    HCPUS_DYNAMIC                   IMEMORY_CACHES_DYNAMIC
-    HCPUS_STATIC                    IMEMORY_CACHES_STATIC
-    HDISKS_DYNAMIC                  IMEMORY_DYNAMIC
-    HDISKS_STATIC                   IMEMORY_STATIC
-    HMEMORY_DYNAMIC                 IPROCESSORS_DYNAMIC
-    HMEMORY_STATIC                  IPROCESSORS_STATIC
-    HNET_DYNAMIC                    IQUEUES_DYNAMIC
-    HNET_STATIC                     IQUEUES_STATIC
-    ICLIENTS_DYNAMIC                IREPLICATIONS_DYNAMIC
-    ICLIENTS_STATIC                 IREPLICATIONS_STATIC
-    ICONFLATEDTOPICS_DYNAMIC        ISOW_DYNAMIC
-    ICONFLATEDTOPICS_STATIC         ISOW_STATIC
-    ICONSOLE_LOGGERS_DYNAMIC        ISTATISTICS_DYNAMIC
-    ICONSOLE_LOGGERS_STATIC         ISTATISTICS_STATIC
-    ICPUS_DYNAMIC                   ISUBSCRIPTIONS_DYNAMIC
-    ICPUS_STATIC                    ISUBSCRIPTIONS_STATIC
-    IFILE_LOGGERS_DYNAMIC           ISYSLOG_LOGGERS_DYNAMIC
-    IFILE_LOGGERS_STATIC            ISYSLOG_LOGGERS_STATIC
-    IGLOBALS_DYNAMIC                ITRANSPORTS_DYNAMIC
-    IGLOBALS_STATIC                 ITRANSPORTS_STATIC
-    ILIFETIMES_DYNAMIC              IVIEWS_DYNAMIC
-    ILIFETIMES_STATIC               IVIEWS_STATIC
-
-
-Schema
-~~~~~~
-
-To view the schema for any table, type the ``.schema <table name>`` command 
-where ``<table name>`` is the name of the table to inspect.
-
-.. code-block:: bash
-
-    sqlite> .schema IFILE_LOGGERS_DYNAMIC
-
-    CREATE TABLE IFILE_LOGGERS_DYNAMIC( timestamp integer,
-    static_id integer, bytes_written integer, PRIMARY
-    KEY( timestamp, static_id ) );
-
-Statistics Table Design
--------------------------
-
-This section describes the philosophy of how the AMPS tables are
-designed within the statistics database. This chapter also includes some
-examples of some useful queries which can give an administrator more
-information than just the raw data would normally give them. Such
-information can be a powerful tool in diagnosing perceived problems in
-AMPS.
-
-Table Naming Scheme
-^^^^^^^^^^^^^^^^^^^
-
-Tables in the database use the following naming scheme:
-
-.. code-block:: bash
-
-    <I|H><STAT>_<STATIC|DYNAMIC>
-    
-    Where:
-    I = AMPS instance statistics
-    H = Host statistics
-    STAT = The statistics that are collected (MEMORY, CPUs,
-    SUBSCRIPTIONS, etc)
-    STATIC = Attributes that rarely change for an object
-    (such as client name, CPU #)
-    DYNAMIC = Stats that are expected to change on every
-    sample (rates, counters, and so on)
-
-Example Queries
-^^^^^^^^^^^^^^^^
-
-To view which clients have fallen behind at one time, run:
-
-.. code-block:: bash
-
-    sqlite> SELECT s.client_name, MAX(d.queue_max_latency),
-    MAX(queued_bytes_out) FROM ICLIENTS_DYNAMIC d
-    JOIN ICLIENTS_STATIC s ON (s.static_id=d.static_id)
-    GROUP BY s.client_name;
-
-To view clients that are behind in the latest sample:
-
-.. code-block:: bash
-
-    sqlite> SELECT s.client_name, d.queue_max_latency,
-    queued_bytes_out FROM ICLIENTS_DYNAMIC d
-    JOIN ICLIENTS_STATIC s ON (s.static_id=d.static_id)
-    WHERE d.timestamp = (SELECT MAX(d.timestamp)
-    FROM ICLIENTS_DYNAMIC d) AND d.queue_max_latency > 0;
-
-Using the amps-sqlite3 Script
------------------------------
-
-The AMPS distribution includes a convenience script, ``amps-sqlite3``,
-for easily running queries against a statistics database. This script
-requires a Python 2.6 or 2.7 interpreter that includes the sqlite3
-module. Most Linux distributions meet this requirement in the default
-installation.
-
-The script takes two parameters, as shown below:
-
-.. list-table:: *amps-sqlite3 parameters*
-     :header-rows: 1
-
-     * - **Parameter**
-
-       - **Description**
-
-     * - .. code::
-         
-           database
-
-       - The sqlite3 database file to query.
-         
-
-     * - .. code::
-         
-           query
-
-       - The query to run. Notice that the query must be enclosed
-         in quotes, since this is a command-line program run by
-         the Linux shell.
-         
-
-
-The ``amps-sqlite3`` script joins the ``STATIC`` and ``DYNAMIC`` tables
-together, making a single table that is easier to query on. For example,
-the script joins the ``ICLIENTS_DYNAMIC`` and ``ICLIENTS_STATIC`` tables
-together into a single ``ICLIENTS`` table.
-
-The ``amps-sqlite3`` wrapper also provides a set of convenience
-functions that can be included in the query. These functions are
-evaluated before the query is presented to the sqlite3 database engine.
-
-.. list-table:: *amps-sqlite3 convenience functions*
-     :header-rows: 1
-
-     * - **Option**
-
-       - **Description**
-
-     * - ISO8601(*timestamp*)
-
-       - Convert ``timestamp`` to an ISO8601 format string.
-
-     * - ISO8601_local(*timestamp*)
-         
-
-       - Convert ``timestamp`` to an ISO8601 format
-         string in the local timezone.
-
-     * - timestamp(*string*)
-
-       - Convert the provided ISO8601 format ``string`` to a timestamp.
-
-
-
-To use the ``amps-sqlite3`` script, simply provide the file name of the
-database to query and the query to run. For example, the following query
-returns the set of samples AMPS has recorded for the ``system_percent``
-consumed on each CPU while the instance has been running:
-
-.. code-block:: bash
-
-    $ amps-sqlite3 stats.db "select iso8601(timestamp),system_percent from hcpus order by timestamp"
-
-SQLite Tips and Troubleshooting
---------------------------------
-
-This section includes information on SQLite tasks that may not be
-immediately obvious and troubleshooting information on SQLite.
-
-Converting AMPS Statistics Time to an ISO8601 Datetime
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-This Python function shows how to convert an AMPS timestamp to an
-ISO8601 datetime. You can use the equivalent in your language of choice
-to convert between the timestamps recorded in the statistics database
-and ISO8601 timestamps.
-
-.. code-block:: bash
-
-    def iso8601_time(amps_time):
-    """
-    Converts AMPS stats time into an ISO8601 datetime.
-    """
-    pt = float(amps_time)/1000 - 210866803200 # subtract the unix epoch
-    it = int(pt)
-    ft = pt-it
-    return time.strftime("%Y%m%dT%H%M%S",time.localtime(it)) + ("%.6f" % ft)[1:]                
-
-
-Shrinking an AMPS Statistics Database
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-If the retention policy for an AMPS statistics database has changed such that
-there is unused space in the file at the maximum retention size, it may be
-helpful to shrink the size of the statistics database.
-
-Running this procedure will only reduce the size of the database if the
-database has been truncated.
-
-To do this:
-
-1. Take the AMPS instance offline.
-
-2. (Optional, but recommended) Make a backup copy of the AMPS statistics database
-   on another device.
-
-3. Run the sqlite ``VACUUM`` command to shrink the database::
-
-      sqlite3 stats.db 'VACUUM'
-
-   This operation may require free disk space equal to the current size
-   of the statistics database. If the operation fails, the vacuum rolls back without
-   changing the database.
-
-A database *should not* be vacuumed while AMPS is running.
-
-Troubleshooting "Database Disk Image is Malformed"
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To repair this error, you need to extract the data from the SQLite
-datastore and create a new datastore. To do this:
-
-1. Open the sqlite datastore. For example, if the database store is
-   named ``stats.db``, the command would be:
-
-.. code-block:: xml
-
-    sqlite3 stats.db
-
-2. Dump the data into a SQL script.
-
-.. code-block:: bash
-
-    .mode insert
-    .output stats_data.sql
-    .dump
-    .exit
-
-This creates a series of SQL commands that recreate the data in the
-database.
-
-3. Make sure that the script commits updates (depending on the version
-   of sqlite3 and the state of the database, the script may roll back
-   the updates rather than committing them without this step).
-
-.. code-block:: bash
-
-   sed -i 's/^ROLLBACK;/COMMIT;/ig' stats_data.sql
-
-4. Now create a new database file using the SQL commands.
-
-.. code-block:: bash
-
-    sqlite3 good.db < stats_data.sql
-
-Finally, adjust the configuration of the Admin server to use the new
-database (in this example, ``good.db``) or copy the new database over
-the old database.

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/amps_expressions.rst.txt

@@ -1,83 +0,0 @@
-.. index:: Content filtering, Query; filters, SOW; content filters, SOW;
-   query filters, Filters, expressions,
-
-.. _#ug-amps-expressions:
-
-.. _amps_expressions:
-
-AMPS Expressions
-====================
-
-AMPS includes an expression language that combines elements of XPath and
-SQL-92's ``WHERE`` clause. This expression language is used whenever the
-AMPS server refers to the contents of a message, including:
-
--  Content filtering
-
--  Constructing fields for message enrichment
-
--  Creating projected fields for views
-
-AMPS uses a common syntax for each of these purposes, and provides a
-common set of operators and functions. AMPS also provides special
-directives for message enrichment, and aggregation functions for
-projecting views.
-
-For example, when an expression is used as a content filter, any message
-for which the expression returns ``true`` matches the content filter.
-When an expression is used to construct a field for message enrichment
-or view projection, the expression is evaluated and the result that the
-expression returns is used as the content of the field.
-
-Expressions Overview
---------------------
-
-The quickest way to learn AMPS expressions is to think of each as a
-combination of identifiers that tell AMPS where to find data in a
-message, and operators that tell AMPS what to do with that data. Each
-AMPS expression produces a value. The way AMPS uses that value depends
-on where the expression is used. For example, in a content filter, AMPS
-uses the value of the expression to determine whether a message matches
-the filter. When constructing a field, AMPS uses the value of the
-expression as the contents of the field.
-
-Consider a simple example of an expression used as a filter. Imagine
-AMPS receives the following JSON message:
-
-.. code-block:: javascript
-
-    {"name":"Gyro", "job":"kitten"}
-
-Using an AMPS expression, you can easily construct a content filter that
-matches the message:
-
-.. code::
-
-    /name = 'Gyro'
-
-There are three parts to this expression. The first part, ``/name``, is
-an *identifier* that tells AMPS to look for the contents of the ``name``
-field at the top level of the JSON document. The second part of the
-filter, ``=``, is the equality *operator*, which tells AMPS to compare
-the values on either side of the operator and return ``true`` if the
-values match. The final part of the filter, ``'Gyro'``, is a string
-*literal* for the equality operator to use in the comparison. When an
-expression is used in a content filter, a message matches the filter
-when the expression returns ``true``. The expression returns ``true``
-for the sample message, so the sample message matches the filter.
-
-The identifier syntax is a subset of XPath, as described in
-:ref:`Identifiers<#ug-expression-identifiers>`. The comparison syntax
-is similar to SQL-92.
-
-Notice that AMPS makes no rigid guarantees as to the number of times
-a given expression is evaluated or when that evaluation will take
-place. AMPS will evaluate the expression as needed.
-
-.. include:: ./syntax.inc
-
-.. include:: ./regular_expressions.inc
-
-
-.. include:: ./perf.inc
-

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/amps_functions.rst.txt

@@ -1,11 +0,0 @@
-.. index:: Content filtering, functions; functions; creating fields
-
-.. _#ug-amps-functions:
-
-.. _amps_functions:
-
-AMPS Functions
-====================
-
-.. include:: ./builtin_functions.inc
-

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/auxiliary_modules.rst.txt

@@ -1,86 +0,0 @@
-.. _#ug-appendix-auxilliary-modules:
-
-.. _auxiliary_modules:
-
-Auxiliary Modules
-=================
-
-The AMPS distribution provides several modules that extend AMPS with
-optional behavior. These modules are *not* loaded by default.
-
-In this release, AMPS includes auxiliary modules that provide the
-following functionality:
-
--  **User-Defined Functions**
-
-   AMPS includes the ``libamps_udf_legacy_compatibility`` module that
-   provides date and time handling functions similar to those provided
-   by legacy messaging systems.
-
-   AMPS provides a ``libamps_udf_experimental`` module that contains
-   special purpose functions.
-
--  **SOW Key Generation**
-
-   AMPS includes the ``libamps_id_chaining_generator`` module that
-   provides chained SOW key generation.
-
--  **Authentication and Entitlement**
-
-   AMPS includes several modules to help you implement an authentication
-   and entitlement system.
-
-   * The ``libamps_http_entitlement`` module makes requests to an
-     external web service to validate authentication credentials and
-     retrieve the set of applicable entitlements. With this module,
-     you can provide both authentication and entitlement infrastructure.
-
-   * The ``libamps_multi_authentication`` module supports multi-mechanism
-     authentication. In this release, the module supports both LDAP
-     and Kerberos. This module does not provide an entitlements
-     implementation.
-
-   * The ``libamps_simple_access_entitlement`` module restricts access
-     to specific resources. This module is most often used to provide
-     limited access to the AMPS Admin console. This module does not
-     provide an authentication implementation, and does not consider
-     individual user entitlements -- the restrictions applied by
-     this module apply to all users.
-
--  **Authenticators**
-
-   * AMPS includes a module, ``libamps_multi_authenticator``,  to provide
-     credentials for outgoing replication connections. Notice that the default
-     authenticator module, which is automatically loaded, can also be
-     configured to provide credentials.
-
-   * AMPS includes a module, ``libamps_exec_authenticator``,  that 
-     executes an external application and provides the output of
-     that process as the credentials for an outgoing replication
-     connection.
-
-The auxiliary modules are described in more detail in the following sections.
-
-.. include:: ./legacy_messaging_functions_module.inc
-
-.. include:: ./experimental_udf.inc
-
-.. include:: ./chaining_key_generator_module.inc
-
-.. _#ug-auth-module:
-
-.. include:: ./http_auth_module.inc
-
-..  _#ug-multi-authentication:
-.. include:: ./multi_authentication_module.inc
-
-.. include:: ./simple_access_module.inc
-
-.. _#ug-multi-authenticator:
-
-.. include:: ./multi_authenticator_module.inc
-
-..  _#ug-exec-authenticator:
-
-.. include:: ./exec_authenticator_module.inc
-

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/config-file-shortcuts.rst.txt

@@ -1,9 +0,0 @@
-.. _config-file-shortcuts:
-
-Configuration File Shortcuts
-=================================
-
-This appendix describes features that AMPS provides for simplifying configuration.
-
-.. include:: ../../common/chapters/configuration/sow-units-env.inc
-

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/conflated_topics.rst.txt

@@ -1,111 +0,0 @@
-.. index:: AMPS; Conflated Topic, Conflated Topic, conflation,
-
-.. _#ug-topic_replicas:
-
-.. _#ug-conflated_topics:
-
-.. _conflated_topics:
-
-
-Conflated Topics
-================
-
-AMPS provides the ability for the server to *conflate* updates
-to a SOW topic by defining a *conflated topic* for that
-SOW topic.
-
-A conflated topic will retain messages published to the underlying
-topic for a certain period of time, and provide the latest update
-for each distinct message in the underlying topic at the end of
-that period of time. In effect, AMPS guarantees that a subscriber
-will receive *no more than* one update for a given message
-per conflation interval.
-
-A conflated topic provides a way to reduce the bandwidth and
-processing for subscribers in cases where the subscriber
-needs periodic updates with the current state of the message
-rather than a rapid set of updates with each individual change
-to the message.
-
-For example, an application that presents a user interface to display
-rapidly changing data often uses conflation, since the value of a
-record may change more rapidly than the user interface is able
-to be refreshed. For this application, creating a conflated topic
-and then having the application subscribe to that topic can
-reduce network traffic, reduce processing load, and provide a
-more responsive user interface than if the application were to
-subscribe to the underlying topic and try to process every update.
-
-AMPS provides the ability to conflate messages for an individual
-subscription, as described in :ref:`#ug-pub-sub-conflation`. 
-When a single subscriber requires
-conflation, requesting conflation for that subscription is a reasonable
-approach to take. In cases where all instances of an application can
-benefit from conflation, *conflated topics* are a more efficient and
-scalable approach. A conflated topic is a copy of one SOW topic into
-another with the ability to control the update interval. In this case,
-AMPS maintains conflation for the entire topic. There is no need for
-subscribers to independently request conflation, and AMPS does not need
-to spend resources processing conflation for each subscriber
-individually.
-
-The underlying topic for a conflated topic can be a ``Topic``, a
-``View``, or another ``ConflatedTopic``.
-
-To better see the value in a conflated topic, imagine a SOW topic called
-``ORDER_STATE`` exists in an AMPS instance. ``ORDER_STATE`` messages are
-published frequently to the topic. Meanwhile, there are several
-subscribing clients that are watching updates to this topic and
-displaying the latest state in a GUI front-end.
-
-If this GUI front-end only needs updates in five second intervals from
-the ``ORDER_STATE`` topic, then more frequent updates would be wasteful
-of network and client-side processing resources. To reduce network
-congestion, a conflated topic for the ``ORDER_STATE`` topic can be
-created which will contain a copy of ``ORDER_STATE`` updated in five
-second intervals. Only the changed records from ``ORDER_STATE`` will be
-copied to the conflated topic and then sent to the subscribing clients.
-Those records with multiple updates within the time interval will have
-their latest updated values copied to the conflated topic, and only
-those conflated values are sent to the clients. This results in
-substantial savings in bandwidth for records with high update rates.
-This can also result in substantial savings in processing overhead for a
-client.
-
-AMPS treats the conflated topic as a conflated version of the underlying
-topic. Applications cannot publish directly to the conflated topic.
-Likewise, AMPS does not recalculate the SOW key for messages delivered
-from the conflated topic: these messages have the same SOW key as the
-corresponding message in the underlying topic.
-
-AMPS indexes conflated topics in the same way that it indexes the
-underlying topic in the SOW. When a query uses a given
-field, AMPS will automatically create a memo index for that field. A
-configuration can also declare one or more ``HashIndex`` indexes for
-a conflated topic.
-
-AMPS uses the following conflation strategy:
-
-* When a message arrives for a given key, if no message is already
-  pending for that key, begin the conflation interval.
-
-* If an update arrives during the conflation interval for a given
-  key, *replace* the pending message with the update for a subscription,
-  or merge the update into the pending message for a delta subscription.
-  All of the metadata on the message is fully replaced.
-
-* If the update is an ``oof`` notification, and the key has not
-  previously been delivered to the subscription, note that the
-  message should not be delivered.
-
-* At the end of the conflation interval, deliver the current version
-  of the message unless the current message is an ``oof`` that
-  should not be delivered. Remove the message from the set of
-  messages being conflated.
-
-This means that, when conflation is used, the first update for a given
-key will arrive after the conflation interval. Further, the application must
-expect that any given update may be delayed by up to the conflation
-interval.
-
-.. include:: ../../configuration/templates/conflatedtopic.inc