.. index:: AMPS; Conflated Topic, Conflated Topic, conflation,

.. _#ug-topic_replicas:

.. _#ug-conflated_topics:

.. _conflated_topics:


Conflated Topics
================

AMPS provides the ability for the server to *conflate* updates
to a SOW topic by defining a *conflated topic* for that
SOW topic.

A conflated topic will retain messages published to the underlying
topic for a certain period of time, and provide the latest update
for each distinct message in the underlying topic at the end of
that period of time. In effect, AMPS guarantees that a subscriber
will receive *no more than* one update for a given message
per conflation interval.

A conflated topic provides a way to reduce the bandwidth and
processing for subscribers in cases where the subscriber
needs periodic updates with the current state of the message
rather than a rapid set of updates with each individual change
to the message.

For example, an application that presents a user interface to display
rapidly changing data often uses conflation, since the value of a
record may change more rapidly than the user interface is able
to be refreshed. For this application, creating a conflated topic
and then having the application subscribe to that topic can
reduce network traffic, reduce processing load, and provide a
more responsive user interface than if the application were to
subscribe to the underlying topic and try to process every update.

AMPS provides the ability to conflate messages for an individual
subscription, as described in :ref:`#ug-pub-sub-conflation`. 
When a single subscriber requires
conflation, requesting conflation for that subscription is a reasonable
approach to take. In cases where all instances of an application can
benefit from conflation, *conflated topics* are a more efficient and
scalable approach. A conflated topic is a copy of one SOW topic into
another with the ability to control the update interval. In this case,
AMPS maintains conflation for the entire topic. There is no need for
subscribers to independently request conflation, and AMPS does not need
to spend resources processing conflation for each subscriber
individually.

The underlying topic for a conflated topic can be a ``Topic``, a
``View``, or another ``ConflatedTopic``.

To better see the value in a conflated topic, imagine a SOW topic called
``ORDER_STATE`` exists in an AMPS instance. ``ORDER_STATE`` messages are
published frequently to the topic. Meanwhile, there are several
subscribing clients that are watching updates to this topic and
displaying the latest state in a GUI front-end.

If this GUI front-end only needs updates in five second intervals from
the ``ORDER_STATE`` topic, then more frequent updates would be wasteful
of network and client-side processing resources. To reduce network
congestion, a conflated topic for the ``ORDER_STATE`` topic can be
created which will contain a copy of ``ORDER_STATE`` updated in five
second intervals. Only the changed records from ``ORDER_STATE`` will be
copied to the conflated topic and then sent to the subscribing clients.
Those records with multiple updates within the time interval will have
their latest updated values copied to the conflated topic, and only
those conflated values are sent to the clients. This results in
substantial savings in bandwidth for records with high update rates.
This can also result in substantial savings in processing overhead for a
client.

AMPS treats the conflated topic as a conflated version of the underlying
topic. Applications cannot publish directly to the conflated topic.
Likewise, AMPS does not recalculate the SOW key for messages delivered
from the conflated topic: these messages have the same SOW key as the
corresponding message in the underlying topic.

AMPS indexes conflated topics in the same way that it indexes the
underlying topic in the SOW. When a query uses a given
field, AMPS will automatically create a memo index for that field. A
configuration can also declare one or more ``HashIndex`` indexes for
a conflated topic.

AMPS uses the following conflation strategy:

* When a message arrives for a given key, if no message is already
  pending for that key, begin the conflation interval.

* If an update arrives during the conflation interval for a given
  key, *replace* the pending message with the update for a subscription,
  or merge the update into the pending message for a delta subscription.
  All of the metadata on the message is fully replaced.

* If the update is an ``oof`` notification, and the key has not
  previously been delivered to the subscription, note that the
  message should not be delivered.

* At the end of the conflation interval, deliver the current version
  of the message unless the current message is an ``oof`` that
  should not be delivered. Remove the message from the set of
  messages being conflated.

This means that, when conflation is used, the first update for a given
key will arrive after the conflation interval. Further, the application must
expect that any given update may be delayed by up to the conflation
interval.

.. include:: ../../configuration/templates/conflatedtopic.inc
