amps-python-client 5.3.4.1__zip

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/constructing-fields.rst.txt

@@ -0,0 +1,269 @@
+
+.. include:: ../macros.inc
+
+.. _#ug-constructing-fields:
+
+.. _constructing-fields:
+
+Constructing Fields
+===================
+
+For views, aggregated subscriptions, and SOW topic enrichment, AMPS
+allows you to construct new fields based on existing data.
+
+When you construct a field, there are two components required:
+
+1. A *source expression* that produces a value. This expression can
+   include XPath identifiers that extract values from a message, literal
+   values, operators, and functions.
+
+2. A *destination identifier* that specifies the identifier where the
+   message type will serialize the value produced by the source
+   expression.
+
+The source expression and the destination identifier are separated by
+the ``AS`` keyword. The format for a field construction expression is as
+follows:
+
+.. code::
+
+    <source expression>
+    AS 
+    <destination identifier>
+
+For example, to create a field in a view that calculates the total value
+of an order by multiplying the ``/price`` field times the ``/qty``
+field, construct the field as shown below:
+
+.. code-block:: xml
+
+    <Field>/price * /qty AS /total</Field>
+
+This constructs a field using ``/price * /qty`` as the source
+expression. Both ``/price`` and ``/qty`` are taken from the incoming
+message. When the result of this expression is computed, the value will
+be produced with the XPath identifier ``/total`` as the destination.
+That value will then be serialized to a message (with the exact format
+and syntax determined by the message type).
+
+Notice that the grammar for constructing fields does not specify
+precisely how the field is represented in the message. AMPS constructs
+the value and provides the XPath identifier to the message type. The
+message type itself is responsible for serializing the value into the
+correct representation and structure for that message type.
+
+All of the AMPS operators and functions that are available for filters
+are available to use in source expressions, including any user-defined
+functions loaded into the instance.
+
+Depending on the context for field construction, there are additional
+capabilities available when constructing fields, as described in the
+following sections.
+
+
+.. _#construct-preprocessing-fields:
+
+Constructing Preprocessing Fields
+----------------------------------
+
+Preprocessing field constructors operate on a single message and
+construct fields based on that message. The results of the preprocessing
+field constructor are merged into the incoming message. Any field in the
+source message that is not changed or removed during preprocessing is
+left unchanged, so it is not necessary to include all fields in the
+message in the ``Preprocessing`` block.
+
+Since preprocessing fields apply to a specific message, preprocessing
+fields cannot specify the topic or message type in an XPath identifier.
+All identifiers in the source expression are evaluated as identifiers
+in the message being preprocessed. Preprocessing fields are evaluated
+during the preprocessing phase, so they cannot refer to the previous state
+of a message.
+
+Using HINT to Control Field Construction
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Preprocessing can be used to remove fields from a message. By default,
+AMPS serializes any field that has an empty string or ``NULL`` value
+after preprocessing. Preprocessing fields can include a directive that
+specifies that a field that contains a ``NULL`` value should be removed
+from the set of fields rather than serialized with a ``NULL`` value. The
+directive ``HINT OPTIONAL`` applied to the XPath identifier specifies
+that if the result of the source expression is ``NULL``, AMPS does not
+provide the value for the message type to serialize. For example, the
+following field constructor removes the ``/source`` field from the
+message if the value provided is not in a specific list of values:
+
+.. code-block:: xml
+
+    <Field>IF(/source IN ('a','e','f'), /source, NULL)
+           AS /source HINT OPTIONAL</Field>
+
+By default, AMPS considers the results of field construction (the
+processed message) to be distinct from the current message. AMPS
+rewrites the current message *after* preprocessing is completed. This
+means that, by default, the results of fields constructed during
+preprocessing are not available to other fields within preprocessing.
+The ``HINT SET_CURRENT`` option immediately inserts or updates values in
+the current message, which makes the new value available to all
+subsequent ``Field`` declarations.
+
+In the sample below, AMPS enriches the message by performing an
+expensive operation (implemented as a user-defined function) on two
+input fields, and immediately updates the current message with the
+output of that operation. AMPS then sets other fields in the processed
+message using the updated value in the current message.
+
+.. code-block:: xml
+
+    <Field>EXPENSIVE_UDF_CALL(/dataSet1, /dataSet2)
+           AS /processedData HINT SET_CURRENT</Field>
+    <Field>IF(/processedData > 1000000,
+               'A',
+               'B') AS /resultClass</Field> 
+
+Notice that using ``HINT SET_CURRENT`` requires AMPS to process
+``Field`` declarations in order, which may prevent future optimizations.
+
+Hints can be combined as follows:
+
+.. code-block:: xml
+
+    <Field>EXPENSIVE_UDF_CALL(/dataSet1, /dataSet2)
+           AS /processedData HINT SET_CURRENT,OPTIONAL
+    </Field>
+
+In this case, if the projected field would be ``NULL``, the field is
+removed from the current message.
+
+.. _#construct-enrichment-fields:
+
+Constructing Enrichment Fields
+-------------------------------
+
+Enrichment field constructors operate on a single message and construct
+fields based on that message. Enrichment expressions operate on the
+current message and change the current message. The results of the
+enrichment directives are merged into the incoming message. Any field in
+the source message that is not changed or removed during preprocessing
+is left unchanged, so it is not necessary to include all fields in the
+message in the ``Enrichment`` directive.
+
+Since enrichment fields apply to a specific message, enrichment fields
+cannot specify the topic or message type in an XPath identifier. All
+identifiers in the source expression are evaluated as identifiers in
+the message being enriched.
+
+Enrichment fields are constructed during the enrichment phase, so enrichment
+fields can refer to the previous state of a message. Within an enrichment
+expression, AMPS provides two special modifiers for XPath identifiers that
+specify whether an XPath identifier refers to the current incoming message or
+the previous state of the message. These modifiers apply only to the source
+expression, and cannot be used in the destination identifier.  The
+modifiers are as follows:
+
+.. list-table:: *XPath identifier modifiers for enrichment*
+     :header-rows: 1
+
+     * - Modifier
+
+       - Description
+
+     * - ``OF CURRENT``
+         
+
+       - Specify that the XPath identifier refers to the
+         incoming message.
+
+     * - ``OF PREVIOUS``
+         
+
+       - Specify that the XPath identifier refers to the
+         previous state of the message in the SOW. If there
+         is no record in the SOW for this message, all
+         identifiers that specify ``OF PREVIOUS`` return
+         ``NULL``.
+
+Using HINT to Control Field Construction
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Enrichment can be used to remove fields from a message. By default, AMPS
+serializes any field that has an empty string or ``NULL`` value after
+enrichment. Enrichment ``Field`` elements can include a directive that
+specifies that a field that contains a ``NULL`` value should be removed
+from the message rather than serialized with a ``NULL`` value. The
+directive ``HINT OPTIONAL`` applied to the XPath identifier specifies
+that if the result of the source expression is ``NULL``, AMPS does not
+provide the value for the message type to serialize. For example, the
+following field constructor removes the ``/source`` field from the
+message if the value provided is not in a specific list of values:
+
+.. code-block:: xml
+
+    <Field>IF(/source IN ('a','e','f'), /source, NULL)
+           AS /source HINT OPTIONAL</Field>
+
+By default, AMPS considers the results of field construction (the
+enriched message) to be distinct from the current message. AMPS rewrites
+the current message *after* enrichment is completed. This means that, by
+default, the results of fields constructed during enrichment are not
+available to other fields within enrichment. The ``HINT SET_CURRENT``
+option immediately inserts or updates values in the current message,
+which makes the new value available to all subsequent ``Field``
+declarations.
+
+In the sample below, AMPS enriches the message by performing an
+expensive operation (implemented as a user-defined function) on two
+input fields, and immediately updates the current message with the
+output of that operation. AMPS then sets other fields in the processed
+message using the updated value in the current message.
+
+.. code-block:: xml
+
+    <Field>EXPENSIVE_UDF_CALL(/dataSet1, /dataSet2)
+           AS /processedData HINT SET_CURRENT</Field>
+    <Field>IF(/processedData > 1000000,
+               'A',
+               'B') AS /resultClass</Field> 
+
+Notice that using ``HINT SET_CURRENT`` requires AMPS to process
+``Field`` declarations in order, which may prevent future optimizations.
+
+Hints can be combined as follows:
+
+.. code-block:: xml
+
+    <Field>EXPENSIVE_UDF_CALL(/dataSet1, /dataSet2)
+           AS /processedData HINT SET_CURRENT,OPTIONAL
+    </Field>
+
+In this case, if the projected field would be ``NULL``, the field is
+removed from the current message.
+
+.. _#ug-constructing-view-fields:
+
+Constructing View Fields
+-------------------------
+
+View field constructors operate over groups of messages, and construct a
+single output message for each distinct group, as specified by the
+``Grouping`` element in the ``View`` configuration.
+
+When constructing a field in a view, all identifiers used in the source
+expression must be in one of the underlying topics for the view. When
+the view uses a ``Join``, the identifiers must include the topic
+identifier. If the topics in the ``Join`` are of different message types,
+the identifiers must include both the message type and the topic
+identifier.
+
+For example, the following ``Field`` definition multiplies the
+``/quantity`` from the NVFIX topic ``orders`` by the ``/price`` from the
+JSON topic ``items``, and projects the result into the ``/total`` field
+of the view.
+
+.. code-block:: xml
+
+    <Field>[nvfix].[orders]./quantity * [json].[items]./price AS /total</Field>
+
+.. include:: ./aggregate_functions.inc
+

amps-python-client-5.3.4.1/docs/user-guide/html/_sources/chapters/daemon.rst.txt

@@ -0,0 +1,232 @@
+.. index:: daemon,
+
+.. _#ug-daemon:
+
+.. _daemon:
+
+Running AMPS as a Linux Service
+====================================
+
+AMPS is designed to be able to easily integrate into your existing
+infrastructure. AMPS includes all of the dependencies it needs to run
+and is configured easily with a single configuration file. Some
+deployments integrate AMPS into a third-party service management
+infrastructure. For those deployments, the needs of that infrastructure
+determine how to install AMPS.
+
+More typically, AMPS runs as a Linux service. This chapter describes how
+to install AMPS as a service.
+
+Installing the Service
+----------------------
+
+AMPS includes a shell script that installs the service. The shell script
+is included in the ``bin`` directory of your AMPS installation. Run the
+script with root permission, as follows:
+
+.. code::
+
+    $ sudo ./install-amps-daemon.sh
+
+This script does the following installation work:
+
+-  Installs the AMPS distribution into ``/opt/amps``.
+
+-  Creates the ``/opt/etc/amps`` directory if it does not already exist.
+   By default, the daemon uses an AMPS configuration file at
+   ``/opt/etc/amps/config.xml``.
+
+-  Installs the service management scripts. Depending on the init system
+   the script detects on your system, this will either be a System V
+   style script located at ``/etc/init.d/amps`` or a SystemD service
+   definition file named ``amps.service`` installed under
+   ``/usr/lib/systemd/``.
+
+-  Updates the service management infrastructure to register AMPS as a
+   service and configure the service to start on startup. The exact
+   steps that the script takes to do this depend on the init system
+   detected.
+
+In addition, you must copy the AMPS configuration file for the instance
+to: ``/opt/etc/amps/config.xml``.
+
+You can only run one instance of AMPS as a service on a system at a
+given time using this script. AMPS does not enforce any restriction on
+how many instances can be run on the system at the same time through
+other means, but this script is designed to manage a single instance
+running as a service.
+
+Configuring the Service
+------------------------
+
+When running as a service, the following considerations apply to the
+configuration file:
+
+AMPS Logging
+^^^^^^^^^^^^
+
+60East recommends logging the most important AMPS messages to syslog
+when running as a service. For example, the following configuration file
+snippet logs messages of warning level and above to the system log:
+
+.. code-block:: xml
+
+    <Logging>
+        <Target>
+            <Protocol>syslog</Protocol>
+            <Level>warning</Level>
+            <Ident>amps</Ident>
+            <Options>LOG_CONS,LOG_NDELAY,LOG_PID</Options>
+            <Facility>LOG_USER</Facility>
+        </Target>
+    </Logging>
+
+60East does not recommend logging a level lower than warning to syslog,
+since an active AMPS instance can produce a large volume of messages.
+
+File Paths
+^^^^^^^^^^
+
+When running as a service, file paths in the configuration file also
+require attention. In particular:
+
+-  For simplicity, use absolute paths for all file paths in the
+   configuration file.
+
+-  Consider startup order, and ensure that any devices that AMPS uses
+   are mounted before AMPS starts.
+
+-  As with any other AMPS installation, it's also important to estimate
+   the amount of storage space AMPS requires and ensure that the device
+   where AMPS stores files has the needed capacity.
+
+Configuration File Location
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The AMPS service scripts require the configuration file to be located at:
+``/opt/etc/amps/config.xml``.
+
+Managing the Service
+---------------------
+
+The scripts that AMPS installs provide management functions for the AMPS
+service. The scripts are used in the same way scripts for other Linux
+services are used.
+
+Starting the AMPS Service
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To start the AMPS service, use the following command if your system uses
+System V init scripts:
+
+.. code-block:: bash
+
+    sudo /etc/init.d/amps start
+
+Many systems that use System V init scripts also provide convenience
+commands (such as ``service``) to locate and run commands for working
+with daemons. Check your distribution's documentation for details.
+
+If your system uses SystemD, you can use a command like:
+
+.. code-block:: bash
+
+    sudo systemctl start amps
+
+Stopping the AMPS Service
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To stop the AMPS service, use the following command if your system uses
+System V init scripts:
+
+.. code-block:: bash
+
+    sudo /etc/init.d/amps stop
+
+Many distributions that use System V init scripts also provide
+convenience commands (such as the ``service`` program) for working with
+daemons. Check your distribution's documentation for details.
+
+If your system uses SystemD, you can use a command like:
+
+.. code-block:: bash
+
+    sudo systemctl stop amps
+
+Restarting the AMPS Service
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To restart the AMPS service, use the following command if your system
+uses System V init scripts:
+
+.. code-block:: bash
+
+    sudo /etc/init.d/amps restart
+
+Many distributions that use System V init scripts also provide
+convenience commands (such as the ``service`` program) for working with
+daemons. Check your distribution's documentation for details.
+
+If your distribution uses SystemD, you can use a command like:
+
+.. code-block:: bash
+
+    sudo systemctl restart amps
+
+View Status for the AMPS Service
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To see the status of the AMPS service, use the following command if your
+system uses System V init scripts:
+
+.. code-block:: bash
+
+    sudo /etc/init.d/amps status
+
+Many distributions that use System V init scripts also provide
+convenience commands (such as the ``service`` program) for working with
+daemons. Check your distribution's documentation for details.
+
+If your distribution uses SystemD, you can use a command like:
+
+.. code-block:: bash
+
+    sudo systemctl status amps
+
+Uninstalling the Service
+------------------------
+
+AMPS includes a script that uninstalls AMPS as a service. The script
+reverses the changes that the install script makes to your system. Run
+the script with root permission, as follows:
+
+.. code-block:: bash
+
+    $ sudo ./uninstall-amps-daemon.sh
+
+The uninstall script does not remove the configuration file or any files
+or data that AMPS creates at runtime.
+
+Upgrading the Service
+---------------------
+
+To upgrade the service to a new version of AMPS, follow these steps:
+
+1. Stop the service.
+
+2. Uninstall the previous version of the service using the uninstall
+   script included with that version.
+
+3. If necessary, upgrade any data files or configuration files that you
+   want to retain. Note that data files, such as AMPS journal files and 
+   SOW files, as of version 5.0.0.0, are compatible with later releases 
+   and do not need to be upgraded.
+
+4. Install the new version of the service using the install script
+   included with the new version. Ensure that the configuration file is
+   at the appropriate path for the new installation.
+
+5. Start the service.
+
+For AMPS instances that participate in failover, you must coordinate
+your upgrades as you would for a standalone AMPS instance.