PyPI - amps-python-client - Versions diffs - 5.3.4.1__zip - Mend

amps-python-client 5.3.4.1__zip

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (550) hide show

amps-python-client-5.3.4.1/docs/user-guide/html/chapters/sow_queries.html ADDED Viewed

@@ -0,0 +1,621 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+  <head>
+    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+    <title>8. SOW Queries &#8212; AMPS User Guide develop documentation</title>
+    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
+    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+    <script type="text/javascript">
+      var DOCUMENTATION_OPTIONS = {
+        URL_ROOT:    '../',
+        VERSION:     'develop',
+        COLLAPSE_INDEX: false,
+        FILE_SUFFIX: '.html',
+        HAS_SOURCE:  true,
+        SOURCELINK_SUFFIX: '.txt'
+      };
+    </script>
+    <script type="text/javascript" src="../_static/jquery.js"></script>
+    <script type="text/javascript" src="../_static/underscore.js"></script>
+    <script type="text/javascript" src="../_static/doctools.js"></script>
+    <link rel="index" title="Index" href="../genindex.html" />
+    <link rel="search" title="Search" href="../search.html" />
+    <link rel="next" title="9. Out-of-Focus Messages (OOF)" href="oof.html" />
+    <link rel="prev" title="7. State of the World (SOW)" href="sow.html" />
+  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
+  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
+  </head>
+  <body role="document">
+  <div class="document">
+      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+        <div class="sphinxsidebarwrapper">
+            <p class="logo"><a href="../index.html">
+              <img class="logo" src="../_static/flag_logo.png" alt="Logo"/>
+            </a></p>
+  <h3><a href="../index.html">Table Of Contents</a></h3>
+  <ul>
+<li><a class="reference internal" href="#">8. SOW Queries</a><ul>
+<li><a class="reference internal" href="#overview-of-sow-queries">Overview of SOW Queries</a></li>
+<li><a class="reference internal" href="#historical-sow-queries">Historical SOW Queries</a><ul>
+<li><a class="reference internal" href="#when-to-use-a-historical-sow-query">When to Use a Historical SOW Query</a></li>
+<li><a class="reference internal" href="#configuring-the-topic-window-and-granularity">Configuring the Topic: Window and Granularity</a></li>
+<li><a class="reference internal" href="#message-sequence-flow">Message Sequence Flow</a></li>
+<li><a class="reference internal" href="#pagination-with-historical-sow-queries">Pagination with Historical SOW Queries</a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#sow-query-and-subscribe">SOW Query and Subscribe</a><ul>
+<li><a class="reference internal" href="#historical-sow-query-and-subscribe">Historical SOW Query and Subscribe</a></li>
+<li><a class="reference internal" href="#conflated-subscriptions-with-sow-and-subscribe">Conflated Subscriptions with SOW and Subscribe</a></li>
+<li><a class="reference internal" href="#replacing-subscriptions-with-sow-and-subscribe">Replacing Subscriptions with SOW and Subscribe</a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#sow-query-response-batching">SOW Query Response Batching</a></li>
+<li><a class="reference internal" href="#configuring-sow-query-result-sets">Configuring SOW Query Result Sets</a></li>
+<li><a class="reference internal" href="#paginated-sow-and-subscribe">Paginated SOW and Subscribe</a></li>
+<li><a class="reference internal" href="#aggregated-sow-queries">Aggregated SOW Queries</a></li>
+</ul>
+</li>
+</ul>
+<div class="relations">
+<h3>Related Topics</h3>
+<ul>
+  <li><a href="../index.html">Documentation overview</a><ul>
+      <li>Previous: <a href="sow.html" title="previous chapter">7. State of the World (SOW)</a></li>
+      <li>Next: <a href="oof.html" title="next chapter">9. Out-of-Focus Messages (OOF)</a></li>
+  </ul></li>
+</ul>
+</div>
+<div id="searchbox" style="display: none" role="search">
+  <h3>Quick search</h3>
+    <form class="search" action="../search.html" method="get">
+      <div><input type="text" name="q" /></div>
+      <div><input type="submit" value="Go" /></div>
+      <input type="hidden" name="check_keywords" value="yes" />
+      <input type="hidden" name="area" value="default" />
+    </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+        </div>
+      </div>
+      <div class="documentwrapper">
+        <div class="bodywrapper">
+          <div class="body" role="main">
+  <div class="section" id="sow-queries">
+<span id="ug-sow-queries"></span><span id="index-0"></span><span id="id1"></span><h1>8. SOW Queries<a class="headerlink" href="#sow-queries" title="Permalink to this headline">¶</a></h1>
+<p>When SOW topics are configured inside an AMPS instance, clients can
+issue SOW queries to AMPS to retrieve all of the messages matching a
+given topic and content filter. When a query is executed, AMPS will test
+each message in the SOW against the content filter specified and all
+messages matching the filter will be returned to the client. The topic
+can be a straight topic or a regular expression pattern.</p>
+<div class="section" id="overview-of-sow-queries">
+<span id="sow-queries-simple"></span><h2>Overview of SOW Queries<a class="headerlink" href="#overview-of-sow-queries" title="Permalink to this headline">¶</a></h2>
+<p>A client can issue a query by sending AMPS a <code class="docutils literal"><span class="pre">sow</span></code> command and
+specifying an AMPS topic. Optionally a filter can be used to further
+refine the query results. AMPS also allows you to restrict the query to
+a specific set of messages identified by a set of SowKeys. When AMPS
+receives the <code class="docutils literal"><span class="pre">sow</span></code> command request, it will validate the filter and
+start executing the query. When returning a query result back to the
+client, AMPS will package the <code class="docutils literal"><span class="pre">sow</span></code> results into a <code class="docutils literal"><span class="pre">sow</span></code> record
+group by first sending a <code class="docutils literal"><span class="pre">group_begin</span></code> message followed by the
+matching SOW records, if any, and finally indicating that all records
+have been sent by terminating with a <code class="docutils literal"><span class="pre">group_end</span></code> message. AMPS returns
+the results for a SOW query in a single, atomic operation. Any messages
+for the client that arrive during the SOW query are delivered after the
+SOW results.</p>
+<p>The sequence diagram below illustrates the message flow for a SOW query.</p>
+<p>For purposes of correlating a query request to its result, each query
+command can specify a <code class="docutils literal"><span class="pre">QueryId</span></code>. The <code class="docutils literal"><span class="pre">QueryId</span></code> specified will be
+returned as part of the response that is delivered back to the client.
+The <code class="docutils literal"><span class="pre">group_begin</span></code> and <code class="docutils literal"><span class="pre">group_end</span></code> messages will have the <code class="docutils literal"><span class="pre">QueryId</span></code>
+attribute set to the value provided by the client. The client specified
+<code class="docutils literal"><span class="pre">QueryId</span></code> is what the client can use to correlate query commands and
+responses coming from the AMPS engine.</p>
+<p>AMPS does not allow a <code class="docutils literal"><span class="pre">sow</span></code> command on topics that do not have a SOW
+enabled. If a client queries a topic that does not have a SOW enabled,
+AMPS returns an error.</p>
+<div class="admonition caution">
+<p class="first admonition-title">Caution</p>
+<p class="last">The ordering of records returned by a SOW query is undefined by
+default. You can include an OrderBy parameter on the query
+to specify a particular ordering based on the contents of the
+messages.</p>
+</div>
+<div class="figure" id="fig-sow-queries-sequence-diagram">
+<a class="reference internal image-reference" href="../_images/SOWQuery1.svg"><img alt="SOW Query Sequence Diagram" src="../_images/SOWQuery1.svg" width="75.0%" /></a>
+</div>
+</div>
+<div class="section" id="historical-sow-queries">
+<span id="sow-queries-historical"></span><h2>Historical SOW Queries<a class="headerlink" href="#historical-sow-queries" title="Permalink to this headline">¶</a></h2>
+<p>Topics in the State of the World can also be configured to include historical
+snapshots of messages, which allows subscribers to retrieve the contents of
+the topic at a particular point in time.</p>
+<p>As with simple queries, a client can issue a query by sending AMPS a
+<code class="docutils literal"><span class="pre">sow</span></code> command and specifying an AMPS topic. For a historical query,
+the client also adds a timestamp that includes the point in time for the
+query in the <code class="docutils literal"><span class="pre">Bookmark</span></code> header of the command. A filter can be used to
+further refine the query results based on the message content.</p>
+<div class="section" id="when-to-use-a-historical-sow-query">
+<h3>When to Use a Historical SOW Query<a class="headerlink" href="#when-to-use-a-historical-sow-query" title="Permalink to this headline">¶</a></h3>
+<p>Use a historical SOW query when it is important to get a snapshot of the
+state of messages in a topic as they existed at a specific point in time
+(that is, if it is important for an application to be able to query the
+state of the world at a point in time).</p>
+<p>If an application needs to replay the exact sequence of messages delivered
+to a topic, but does not need to be able to query the values that were current
+at a specific point in time, record the topic in the transaction log and
+replay from the transaction log.</p>
+<p>If an application needs to <em>both</em> retrieve a snapshot of the values that
+were current at a specific point in time and replay the exact sequence of
+messages from that point forward, use a historical SOW query and record
+the topic in the transaction log.</p>
+</div>
+<div class="section" id="configuring-the-topic-window-and-granularity">
+<h3>Configuring the Topic: Window and Granularity<a class="headerlink" href="#configuring-the-topic-window-and-granularity" title="Permalink to this headline">¶</a></h3>
+<p>By default, AMPS does not maintain history for a topic in the State of
+the World. To enable history (and historical query) for the topic,
+add the <code class="docutils literal"><span class="pre">History</span></code> element to the <code class="docutils literal"><span class="pre">Topic</span></code> configuration. This element
+configures how much information AMPS stores for enabling historical
+queries.</p>
+<p>There are two options that control how AMPS stores data for historical
+queries:</p>
+<ol class="arabic simple">
+<li>The <code class="docutils literal"><span class="pre">Window</span></code> option sets the amount of time that AMPS will retain
+historical versions of messages. AMPS will remove the historical state
+of the message from the SOW topic once that historical state is older
+than the specified window. (If the message has been deleted, and the
+delete command is older than the specified window, AMPS may remove the
+message from the SOW topic entirely). AMPS always retains
+the most current state of a message, even if that state was published
+earlier than the specified <code class="docutils literal"><span class="pre">Window</span></code>.</li>
+<li>The <code class="docutils literal"><span class="pre">Granularity</span></code> option sets the interval at which AMPS retains a
+historical copy of a message in the SOW. For example, if the
+<code class="docutils literal"><span class="pre">Granularity</span></code> is set to <code class="docutils literal"><span class="pre">10m</span></code>, AMPS stores a historical copy of the
+message no more frequently than every 10 minutes, regardless of how many
+times the message is updated in that 10 minute interval. AMPS stores the
+copies when a new message arrives to update the SOW. This means that
+AMPS always returns a valid SOW state that reflects a published message,
+but &#8211; as with a conflated topic &#8211; the SOW may not reflect all of the
+states that a message passes through. This also means that AMPS uses SOW
+space efficiently. If no updates have arrived for a message, since the
+last time a historical message was saved, AMPS has no need to save
+another copy of the message.</li>
+</ol>
+<p>When a message is deleted from a topic that maintains history, AMPS
+saves the fact that the message has been deleted, and queries as of
+that point in time will not return the message. However, previously
+saved states of the message within the <code class="docutils literal"><span class="pre">Window</span></code> are still present,
+and can still be queried.</p>
+<p>Likewise, if an application queries at a point in time earlier than the
+<code class="docutils literal"><span class="pre">Window</span></code>, AMPS will return an empty result set (even if messages had
+actually been present in the topic at that point), since the SOW state is
+only retained for the period in the <code class="docutils literal"><span class="pre">Window</span></code>.</p>
+<div class="admonition tip">
+<p class="first admonition-title">Tip</p>
+<p class="last">The <code class="docutils literal"><span class="pre">Granularity</span></code> for a topic is always specified as a duration. If
+your application requires that a query be able to return the exact
+state of the SOW exactly as AMPS would have represented it at that time
+(with no tolerance for the granularity), you can specify that AMPS
+keep every message during the <code class="docutils literal"><span class="pre">Window</span></code> by specifying a <code class="docutils literal"><span class="pre">Granularity</span></code>
+of <code class="docutils literal"><span class="pre">0s</span></code>. Notice that this is <em>not</em> required to replay every message
+after a point-in-time query, since replay is delivered from the
+transaction log rather than the stored State of the World.</p>
+</div>
+<p>When a historical SOW and Subscribe query is entered, and the topic is
+covered by a transaction log, AMPS returns the state of the SOW adjusted
+to the next oldest granularity, then replays messages from that point.
+In other words, AMPS returns the same results as a historical SOW query,
+then replays the full sequence of messages from that point forward.</p>
+<p>The transaction log and the SOW topic are maintained separately, and have
+separate views of history. When a version of the message is removed from
+the SOW topic (because it is older than the specified <code class="docutils literal"><span class="pre">Window</span></code>), the
+message remains in the transaction log, but will not be returned by a SOW
+query.</p>
+</div>
+<div class="section" id="message-sequence-flow">
+<h3>Message Sequence Flow<a class="headerlink" href="#message-sequence-flow" title="Permalink to this headline">¶</a></h3>
+<p>The message sequence flow is the same as a simple SOW query flow. Once
+AMPS has transmitted the messages that were in the SOW as of the
+timestamp of the query, the query ends. Notice that the query will
+include messages that have been subsequently deleted from the SOW, but which
+were the current state of the message as of that timestamp.</p>
+</div>
+<div class="section" id="pagination-with-historical-sow-queries">
+<h3>Pagination with Historical SOW Queries<a class="headerlink" href="#pagination-with-historical-sow-queries" title="Permalink to this headline">¶</a></h3>
+<p>Topics that maintain <code class="docutils literal"><span class="pre">History</span></code> in the SOW support paginated queries from a
+point in time. When the topic is also covered by the transaction log, the
+<code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command also supports paginated subscriptions from
+a point in time.  See <a class="reference internal" href="#sow-query-pagination"><span class="std std-ref">Paginated SOW and Subscribe</span></a> for details.</p>
+</div>
+</div>
+<div class="section" id="sow-query-and-subscribe">
+<span id="id3"></span><h2>SOW Query and Subscribe<a class="headerlink" href="#sow-query-and-subscribe" title="Permalink to this headline">¶</a></h2>
+<p>AMPS has a special command that will execute a query and place a
+subscription at the same time to prevent a gap between the query and
+subscription where messages can be lost. Without a command like this, it
+is difficult to reproduce the SOW state locally on a client without
+creating complex code to reconcile incoming messages and state.</p>
+<p>For example, this command is useful for recreating part of the SOW in
+a local cache and keeping it up to date. Without a special command to
+place the query and subscription at the same moment, a client is left
+with two options:</p>
+<ol class="arabic simple">
+<li>Issue the query request, process the query results, and then place
+the subscription, which misses any records published between the time
+when the query and subscription were placed; or</li>
+<li>Place the subscription and then issue the query request, which could
+send messages placed between the subscription and query twice.</li>
+</ol>
+<p>Instead of requiring every program to work around these options, the
+AMPS <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command allows clients to place a query and
+get the streaming updates to matching messages in a single command.</p>
+<p>In a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command, AMPS behaves as if the SOW command
+and subscription are placed at the exact same moment. The SOW query will
+be sent before any messages from the subscription are sent to the
+client. Additionally, any new publishes that come into AMPS that match
+the <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> filtering criteria and come in after the query
+started will be sent after the query finishes (and the query will not
+include those messages.) As with a simple SOW query, any other messages
+that arrive for the client while the SOW query is running will also be
+delivered after the query results.</p>
+<p>AMPS allows a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command on topics that do not have a
+SOW enabled. In this case, AMPS simply returns no messages between
+<code class="docutils literal"><span class="pre">group_begin</span></code> and <code class="docutils literal"><span class="pre">group_end</span></code>.</p>
+<p>The sequence diagram below illustrates the message flow for <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code>
+commands:</p>
+<div class="figure" id="fig-sow-queries-subscribe-sequence">
+<a class="reference internal image-reference" href="../_images/SOWQuery2.svg"><img alt="SOW-And-Subscribe Query Sequence Diagram" src="../_images/SOWQuery2.svg" width="75.0%" /></a>
+</div>
+<p><em>SOW-And-Subscribe Query Sequence Diagram</em></p>
+<div class="section" id="historical-sow-query-and-subscribe">
+<h3>Historical SOW Query and Subscribe<a class="headerlink" href="#historical-sow-query-and-subscribe" title="Permalink to this headline">¶</a></h3>
+<p>For topics that have <code class="docutils literal"><span class="pre">History</span></code> configured,
+AMPS SOW Query and Subscribe also allows you to begin the subscription
+with a historical SOW query. For historical SOW queries, the
+subscription begins at the point of the query with the results of the
+SOW query. The subscription then replays messages from the transaction
+log. Once messages from the transaction log have been replayed, the
+subscription then provides messages as AMPS publishes them.</p>
+<p>In effect, a SOW Query and Subscribe with a historical query allows you
+to recreate the client state and processing as though the client had
+issued a SOW Query and Subscribe at the point in time of the historical
+query.</p>
+<p>A historical SOW and Subscribe requires that the SOW topic is recorded
+in the transaction log and that history is enabled on the SOW. If
+history is not enabled for the topic, a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command
+returns the current state of the SOW and the subscription begins
+atomically at the point in time when AMPS processes the command.</p>
+</div>
+<div class="section" id="conflated-subscriptions-with-sow-and-subscribe">
+<h3>Conflated Subscriptions with SOW and Subscribe<a class="headerlink" href="#conflated-subscriptions-with-sow-and-subscribe" title="Permalink to this headline">¶</a></h3>
+<p>A <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command can include the conflation interval and conflation key
+options for server side conflation (as described in <a class="reference internal" href="pub_sub.html#ug-pub-sub-conflation"><span class="std std-ref">Conflated Subscriptions</span></a>),
+just as a regular subscription can. When the command requests conflation, the
+results of the SOW query are not conflated. Conflation only applies to the subscription.</p>
+</div>
+<div class="section" id="replacing-subscriptions-with-sow-and-subscribe">
+<span id="ug-sow-and-subscribe-replace"></span><h3>Replacing Subscriptions with SOW and Subscribe<a class="headerlink" href="#replacing-subscriptions-with-sow-and-subscribe" title="Permalink to this headline">¶</a></h3>
+<p>As described in <a class="reference internal" href="pub_sub.html#ug-pub-sub-replace"><span class="std std-ref">Replacing Subscriptions</span></a>,
+AMPS allows you to replace an existing subscription. When the subscription is entered with
+the <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command, AMPS will re-run the SOW query
+delivering the messages that are in scope with the new filter but which
+were not previously delivered. If the subscription requests out-of-focus
+(OOF) messages, AMPS will deliver out of focus messages for messages
+that matched the previous filter but do not match the new filter. As
+with the initial Query and Subscribe, AMPS guarantees to deliver any
+changes to the SOW that match the filter and occur after the point of
+the query.</p>
+<div class="figure">
+<a class="reference internal image-reference" href="../_images/SOWAndSubscribeAndReplace.svg"><img alt="SOW And Subscribe Replace Sequence Diagram" src="../_images/SOWAndSubscribeAndReplace.svg" width="75.0%" /></a>
+</div>
+<p><em>SOW And Subscribe Replace Sequence Diagram</em></p>
+</div>
+</div>
+<div class="section" id="sow-query-response-batching">
+<span id="sow-queries-batching"></span><h2>SOW Query Response Batching<a class="headerlink" href="#sow-query-response-batching" title="Permalink to this headline">¶</a></h2>
+<p>When processing a SOW query, AMPS has the ability to combine messages
+into batches for more efficient network usage. The maximum number of
+messages in a batch is determined by the <code class="docutils literal"><span class="pre">BatchSize</span></code> parameter on the
+SOW query command. AMPS defaults to a <code class="docutils literal"><span class="pre">BatchSize</span></code> value of 1, meaning
+AMPS sends one message per batch in the response. The <code class="docutils literal"><span class="pre">BatchSize</span></code> is
+the maximum number of records that will be returned within a single
+response payload. Each AMPS response for the query contains a
+<code class="docutils literal"><span class="pre">BatchSize</span></code> value in its header to indicate the number of messages in
+the batch. This number will be anywhere from 1 to <code class="docutils literal"><span class="pre">BatchSize</span></code>.</p>
+<p>Current versions of the AMPS client libraries set a batch size of 10
+when using the named convenience methods (for example,
+<code class="docutils literal"><span class="pre">sowAndSubscribe</span></code>) if no other batch size is specified.</p>
+<p>Notice that the format of messages returned from AMPS may be different
+depending on the message type requested. However, the information
+contained in the messages is the same for all message types.</p>
+<div class="admonition tip">
+<p class="first admonition-title">Tip</p>
+<p class="last">When issuing a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command AMPS will return a
+<code class="docutils literal"><span class="pre">group_begin</span></code> and <code class="docutils literal"><span class="pre">group_end</span></code> segment of messages before
+beginning the live subscription sequence of the query. This is
+also true when a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command is issued against
+a non-SOW topic. When the topic is not in the State of the World,
+no messages will be delivered between the <code class="docutils literal"><span class="pre">group_begin</span></code> and
+<code class="docutils literal"><span class="pre">group_end</span></code> messages.</p>
+</div>
+<p>Using a <code class="docutils literal"><span class="pre">BatchSize</span></code> greater than 1 can yield greater performance,
+particularly when querying a large number of small records. In general,
+60East recommends using a <code class="docutils literal"><span class="pre">BatchSize</span></code> that provides good network
+utilization without consuming excessive server memory. Most applications
+that use small messages set a batch size designed to create batches that
+fit well into the maximum transmission unit (MTU) for the network.
+AMPS reports an error if an application requests a batch size larger
+than 10,000 records (this value is orders of magnitude larger than the
+typical <code class="docutils literal"><span class="pre">BatchSize</span></code> used by applications).</p>
+<p>For applications that return a large number of messages that are
+larger than the MTU, 60East recommends testing performance with
+a variety of batch sizes. Because the client libraries parse the
+AMPS headers common to each message once per batch, a batch size
+larger than <cite>1</cite> can improve processing performance on the client
+side, particularly if the client message handling is efficient.
+Likewise, because the AMPS server only has to serialize the common
+headers once per batch, a batch size larger than <cite>1</cite> can improve
+performance at the server side (as well as reduce the overall
+bandwidth for a group of messages). At the same time, the server
+will hold a batch of messages until the batch can be transmitted
+together (or until the query is complete), so providing large values
+for the batch size can introduce latency in receiving results, and
+reduce performance if the total size of the batch is very large.</p>
+<p>In general, the default client value is a good compromise for
+many application patterns if the messages are larger than will fit
+into the MTU of the network. For smaller messages, or if it is
+important to tune performance, 60East recommends testing with
+a variety of batch sizes.</p>
+<div class="admonition tip">
+<p class="first admonition-title">Tip</p>
+<p class="last">Using an appropriate <code class="docutils literal"><span class="pre">BatchSize</span></code> parameter is critical to
+achieve the maximum query performance with a large number of
+messages when many messages will fit into the MTU for your
+network. For larger messages, tune the batch size based on
+performance testing with a variety of batch sizes.</p>
+</div>
+<div class="admonition caution">
+<p class="first admonition-title">Caution</p>
+<p>AMPS treats queries as a single, atomic operation. All results
+from a query are sent to a client before the results of any
+subsequent commands.  Use care when issuing queries that
+return a result set large enough to take several seconds or
+more to transmit over the network.</p>
+<p class="last">When planning for large queries, please see the information
+on how AMPS handles a situation where messages are produced
+faster than the client or network can consume them. This
+is discussed in the section called
+<a class="reference internal" href="operation.html#operations-and-deployment-best-practices-slow-clients"><span class="std std-ref">Slow Clients</span></a>
+for more information.</p>
+</div>
+<p>For more information on executing queries, please see the Developer
+Guide for the AMPS client of your choice, available from the 60East
+documentation site at <a class="reference external" href="http://docs.crankuptheamps.com/">http://docs.crankuptheamps.com/</a>.</p>
+</div>
+<div class="section" id="configuring-sow-query-result-sets">
+<span id="sow-query-results-sets"></span><h2>Configuring SOW Query Result Sets<a class="headerlink" href="#configuring-sow-query-result-sets" title="Permalink to this headline">¶</a></h2>
+<p>AMPS allows you to control the results returned by a SOW query by
+including the following options and header on the query:</p>
+<table border="1" class="docutils" id="id4">
+<caption><span class="caption-text"><em>SOW Query Options</em></span><a class="headerlink" href="#id4" title="Permalink to this table">¶</a></caption>
+<colgroup>
+<col width="50%" />
+<col width="50%" />
+</colgroup>
+<thead valign="bottom">
+<tr class="row-odd"><th class="head">Option /
+Header</th>
+<th class="head">Result</th>
+</tr>
+</thead>
+<tbody valign="top">
+<tr class="row-even"><td>top_n
+(option)</td>
+<td><p class="first">Limits the results returned to the number of messages specified.</p>
+<p class="last">When a <code class="docutils literal"><span class="pre">skip_n</span></code> option is also provided for a subscription,
+AMPS creates a paginated subscription. Otherwise, this option
+applies only to the SOW query part of a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code>
+or <code class="docutils literal"><span class="pre">sow_and_delta_subscribe</span></code> command.</p>
+</td>
+</tr>
+<tr class="row-odd"><td>skip_n
+(option)</td>
+<td>Skips the number of messages specified before returning results.
+A command that provides this option must also provide a <code class="docutils literal"><span class="pre">top_n</span></code>
+option.</td>
+</tr>
+<tr class="row-even"><td>OrderBy
+(header)</td>
+<td><p class="first">Orders the results returned as specified. Requires a
+comma-separated list of identifiers of the form</p>
+<div class="code highlight-default"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">field</span> <span class="p">[</span><span class="n">ASC</span> <span class="o">|</span> <span class="n">DESC</span><span class="p">]</span> <span class="p">[</span> <span class="n">TEXT</span> <span class="p">]</span>
+</pre></div>
+</div>
+<p>The <code class="docutils literal"><span class="pre">ASC</span></code> directive specifies that AMPS sort the results in
+ascending order (the default). <code class="docutils literal"><span class="pre">DESC</span></code> specifies that AMPS
+sort the results in descending order.</p>
+<p>The <code class="docutils literal"><span class="pre">TEXT</span></code> hint specifies that AMPS will sort the
+column according to the textual representation of the column.
+This can be helpful in cases where the column represents a
+string value, but where some values could be interpreted as
+numeric values.</p>
+<p>For example, to sort in descending order by <code class="docutils literal"><span class="pre">orderDate</span></code> so that
+the most recent orders are first, and ascending order by
+<code class="docutils literal"><span class="pre">customerName</span></code> for orders with the same date, you might use a
+specifier such as:</p>
+<div class="code highlight-default"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">orderDate</span> <span class="n">DESC</span><span class="p">,</span> <span class="o">/</span><span class="n">customerName</span> <span class="n">ASC</span>
+</pre></div>
+</div>
+<p>As another example, the following specifier will sort the
+<code class="docutils literal"><span class="pre">orderId</span></code> field as a string, with the <code class="docutils literal"><span class="pre">updateTimestamp</span></code>
+sorted in descending order for orders with the same <code class="docutils literal"><span class="pre">orderId</span></code>.</p>
+<div class="code highlight-default"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">orderId</span> <span class="n">TEXT</span><span class="p">,</span> <span class="o">/</span><span class="n">updateTimestamp</span> <span class="n">DESC</span>
+</pre></div>
+</div>
+<p class="last">If no sort order is specified for an identifier, AMPS defaults to
+ascending order. If no type hint is specified for an identifier,
+AMPS defaults to using a mixed-type sort.</p>
+</td>
+</tr>
+</tbody>
+</table>
+<p>For details on how to submit these options with a SOW query, see the
+documentation for the AMPS client library your application uses.</p>
+<p>When replacing a subscription that uses <code class="docutils literal"><span class="pre">top_n</span></code>, <code class="docutils literal"><span class="pre">skip_n</span></code>, or
+<code class="docutils literal"><span class="pre">OrderBy</span></code>, any of these options specified on the original command
+must be provided on the replacement command. In other words,
+<code class="docutils literal"><span class="pre">sow_and_subscribe</span></code> command that specifies <code class="docutils literal"><span class="pre">top_n=10,skip_n=20</span></code>
+must provide both <code class="docutils literal"><span class="pre">top_n</span></code> and <code class="docutils literal"><span class="pre">skip_n</span></code> on a replacement command.</p>
+</div>
+<div class="section" id="paginated-sow-and-subscribe">
+<span id="sow-query-pagination"></span><h2>Paginated SOW and Subscribe<a class="headerlink" href="#paginated-sow-and-subscribe" title="Permalink to this headline">¶</a></h2>
+<p>When <code class="docutils literal"><span class="pre">top_n</span></code> and <code class="docutils literal"><span class="pre">skip_n</span></code> are specified on a <code class="docutils literal"><span class="pre">sow_and_subscribe</span></code>
+command, AMPS creates a <em>paginated subscription</em>.  (Both <code class="docutils literal"><span class="pre">top_n</span></code> and
+<code class="docutils literal"><span class="pre">skip_n</span></code> must be provided to create a paginated subscription.)</p>
+<p id="index-1">With a paginated subscription, AMPS maintains a list of the set
+of results for the SOW query, and delivers only results that
+fall between the first record after the <cite>skip_n</cite> number and
+within the number of records specified by the <cite>top_n</cite> number. This
+allows applications that only need a subset of the results
+returned by a filter to work with only those results. This is
+commonly used for interactive applications, where
+a user interface shows a small number of records at a time in the
+interface.</p>
+<p>When the subscription specifies an <code class="docutils literal"><span class="pre">OrderBy</span></code>, that header specifies the
+order in which records are sorted within the paginated subscription.
+If no <code class="docutils literal"><span class="pre">OrderBy</span></code> is specified, the results are sorted by the <code class="docutils literal"><span class="pre">SowKey</span></code>
+generated by AMPS (effectively, an arbitrary but stable order).</p>
+<p>From a subscriber point of view, paginated subscriptions behave as
+though only the messages in the pagination window are present in
+AMPS. For example, when out-of-focus notifications are enabled
+and a message in the topic is deleted, subscribers receive an
+<code class="docutils literal"><span class="pre">oof</span></code> notification <em>only</em> if the deleted message was in the
+pagination window. Likewise, if a message that was previously in
+the pagination window falls outside of the window due to an insert
+or delete, the message that is now outside of the window will be
+out of focus, and will generate an <code class="docutils literal"><span class="pre">oof</span></code> notification.</p>
+<p>For example, consider the following topic in the SOW, where
+the topic uses the <code class="docutils literal"><span class="pre">/id</span></code> field as a key.</p>
+<div class="figure">
+<img alt="../_images/paginated_subs_sub.png" src="../_images/paginated_subs_sub.png" />
+</div>
+<p>With a <code class="docutils literal"><span class="pre">top_n</span></code> of <code class="docutils literal"><span class="pre">2</span></code>, a <code class="docutils literal"><span class="pre">skip_n</span></code> of <code class="docutils literal"><span class="pre">1</span></code>, and an
+<code class="docutils literal"><span class="pre">OrderBy</span></code> of <code class="docutils literal"><span class="pre">/id</span></code>, the results for the subscription will include
+the records with <code class="docutils literal"><span class="pre">id</span></code> of <code class="docutils literal"><span class="pre">2</span></code> and <code class="docutils literal"><span class="pre">id</span></code> of <code class="docutils literal"><span class="pre">5</span></code>.</p>
+<p>Now a new message is published with an <code class="docutils literal"><span class="pre">id</span></code> of <code class="docutils literal"><span class="pre">4</span></code>, as shown
+below:</p>
+<div class="figure">
+<img alt="../_images/paginated_subs_pub.png" src="../_images/paginated_subs_pub.png" />
+</div>
+<p>Since the new message falls within the pagination window, the
+message is published to the subscriber. Given that the message with
+the <code class="docutils literal"><span class="pre">id</span></code> of <code class="docutils literal"><span class="pre">5</span></code> is no longer within the pagination window,
+the subscriber will receive an <code class="docutils literal"><span class="pre">oof</span></code> message for the message
+with an <code class="docutils literal"><span class="pre">id</span></code> of <code class="docutils literal"><span class="pre">5</span></code> if the subscriber has requested
+out-of-focus notifications.</p>
+<p>While a paginated subscription is active, AMPS maintains a list of
+the messages that match the subscription in memory (but does not,
+as of version 5.3.2, maintain the entire sorted result set in memory).
+For efficiency, when more than one subscription uses the same topic,
+these subscriptions will use the same result set in memory. The memory
+used counts as part of the configured <code class="docutils literal"><span class="pre">MessageMemoryLimit</span></code>. Each
+connection that uses the result set is counted as consuming a
+portion of the memory retained. For example, if 5 connections use
+the same result set, each of those connections is counted as using 1/5 of the
+memory for the result set.</p>
+<p>In addition, each paginated subscription requires that AMPS maintain
+state for the window for that subscription: this memory is not shared,
+and is counted for that client.</p>
+</div>
+<div class="section" id="aggregated-sow-queries">
+<span id="sow-query-aggregation"></span><h2>Aggregated SOW Queries<a class="headerlink" href="#aggregated-sow-queries" title="Permalink to this headline">¶</a></h2>
+<p>AMPS provides the ability to aggregate the results of a SOW query. The results of an aggregated
+SOW query are the same as the results of querying a <code class="docutils literal"><span class="pre">View</span></code> with the same definition.</p>
+<p>To request an aggregated SOW query, provide the <code class="docutils literal"><span class="pre">grouping</span></code> and <code class="docutils literal"><span class="pre">projection</span></code> options
+with the <code class="docutils literal"><span class="pre">sow</span></code> query.</p>
+<table border="1" class="docutils" id="id5">
+<caption><span class="caption-text"><em>Aggregated SOW Query Options</em></span><a class="headerlink" href="#id5" title="Permalink to this table">¶</a></caption>
+<colgroup>
+<col width="50%" />
+<col width="50%" />
+</colgroup>
+<thead valign="bottom">
+<tr class="row-odd"><th class="head">Option</th>
+<th class="head">Description</th>
+</tr>
+</thead>
+<tbody valign="top">
+<tr class="row-even"><td><code class="docutils literal"><span class="pre">grouping=[keys]</span></code></td>
+<td><p class="first">For use with aggregated SOW queries.</p>
+<p>The format of this option is a comma-delimited
+list of XPath identifiers within brackets. For
+example, to aggregate entries based on their
+<code class="docutils literal"><span class="pre">/description</span></code> (producing one record in the
+aggregation for each distinct value in
+<code class="docutils literal"><span class="pre">/description</span></code>), you would use the following
+option:</p>
+<div class="code highlight-default"><div class="highlight"><pre><span></span><span class="n">grouping</span><span class="o">=</span><span class="p">[</span><span class="o">/</span><span class="n">description</span><span class="p">]</span>
+</pre></div>
+</div>
+<p>When this option is provided, a <code class="docutils literal"><span class="pre">projection</span></code>
+must also be provided.</p>
+<p class="last">When the topic has <code class="docutils literal"><span class="pre">History</span></code> enabled, this
+option can be used with a bookmark to aggregate
+the historical state of the SOW.</p>
+</td>
+</tr>
+<tr class="row-odd"><td><code class="docutils literal"><span class="pre">projection=[fields]</span></code></td>
+<td><p class="first">For use with aggregated SOW queries.</p>
+<p>Specifies a comma-delimited set of fields to
+project, within brackets. Each entry has the
+format described in the AMPS User Guide.</p>
+<p>This option must contain an entry for every
+field in the aggregated message. If there is no
+entry for a field in this option, that field
+will not appear in the aggregated message, even
+if the field is in the underlying message.</p>
+<p>There is no default for this option. When this
+option is provided, a <code class="docutils literal"><span class="pre">grouping</span></code> must also be
+provided.</p>
+<p class="last">When the topic has <code class="docutils literal"><span class="pre">History</span></code> enabled, this
+option can be used with a bookmark to aggregate
+the historical state of the SOW.</p>
+</td>
+</tr>
+</tbody>
+</table>
+</div>
+</div>
+          </div>
+        </div>
+      </div>
+    <div class="clearer"></div>
+  </div>
+    <div class="footer">
+      &copy;2023 60East Technologies, Inc. (version develop).
+      |
+      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.5.3</a>
+      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
+    </div>
+  </body>
+</html>