Presence Map and Stop Bit

The Presence Map (Pmap) indicates which fields in the template have data present and which fields have data implied. A Pmap is a sequence of the encoded bits with each bit representing a template field according to sequence. Fields with data present have the Pmap bit set to ‘1’. Fields with data implied have the Pmap bit set to ‘0’ (Exception: fields with a Constant operator in which the bit is set to ‘1’ for an implied state).

Presence Map Rules

Pmap rules determine when a FAST template field required a corresponding Pmap bit. A field does not require a bit in the Pmap when it meets any of the following criteria:

•      The field is defined as mandatory without a field operator – a value will always be present.

•      The field is defined as mandatory with a constant operator – a value should always be instantiated in the decoded message based on the value in the template.

•      The field is defined as mandatory with a delta operator – a delta value is always present.

•      The field is defined as optional without a field operator – either a value or NULL will always be present.

•      The field is defined as optional with a delta field operator – a delta value or NULL will always be present.

See the FAST v1.1 specification for the complete information on presence map rules.

Rules for Determining if a Presence Map Bit is Required

Message Structure

Note: The Market Data Incremental Refresh (tag 35-MsgType = X) message is used for example purposes throughout this document; not all FAST messages follow this format.

The FIX Market Data Incremental Refresh (tag 35-MsgType = X) message is made up of three components: a Header, a Body and a set of one or more Market Data Entries as shown in the diagram below. The Header carries transmission details. The Body carries information pertinent to all entries in the message such as TradeDate. The Market Data Entry carries specific instructions for updating the book or recording trades. The templates provided by CQG conform to this general structure and use only fields that are part of the Market Data Incremental Refresh (tag 35-MsgType = X) message.

Example: How to Read a Template Field

Field Operators and Data Types appear within the Template as follows:

Template ID Usage

Each template is assigned a Template ID to uniquely describe the format of an encoded message. A Template ID is carried in every encoded message to provide a reference to the correct template for decoding purposes.

The Template ID is an unsigned integer carried as the first data field following the first Pmap of every message allowing the decoding system to apply the correct template to the message upon receiving it.

Example: Template ID

CQG will not send the same Template ID more than once in a UDP packet. When the packed message is received, the sequence to begin decoding a message is:

1.  Locate and read the first Pmap field, which begins in the first byte of the application data preceding the Quotes Direct message Header.

2.  Treat the next field as the Template ID (the first logical field of the message).

3.  Retrieve the template specified by the Template ID.

4.  Decode the remainder of the message.

Optional vs. Mandatory Fields

A field is defined as optional when it is possible for a value to not be present in the decoded message. When a field is optional, the wire representation “null” indicates the field is not present. A field is defined as mandatory when that field must be present in the decoded message. Note that a field may not be present (“coded away”) in the wire formatted message but can be present in the decoded message as the result of an encoding operation (e.g. default) that removes data for efficiency but continues to imply data is present through the Pmap.

Optional fields are useful when a generic template is used to provide multiple market data types such as book updates and trades. In this situation, there may be fields which are present in one occurrence of the repeating group but not in another within a given message.

For example, the MDEntries repeating group that is present in the MDIncRefresh template shown above can be used to express a trade, bid, ask, high, low, etc. within a single message. A trade entry will not use tag 346, NumberOfOrders, which will then be defined as optional. However, FAST requires that this field be accounted for in the encoded message if specified in the template. This is done through the use of a reserved value of NULL to indicate that the field is not present in the decoded data. In the serialization layer, FAST reserves binary zeros to indicate a NULL value which tells the decoder that no data is present for this Template Distribution.

Rules for Determining if a Field is Nullable

XML Template Example

A template consists of Field Instructions that define the fields contained in the message. Field Instructions specify the field name, tag number, data type, field operator, and presence attribute that indicates if a field is optional or mandatory.

A sample market data template is shown below. The syntax is standard XML and can be parsed using a variety of open source tools. Valid template syntax is determined by the FAST Template Schema, which is available in the FAST v1.1 specification.

The real-time feed templates are based on the Market Data Incremental Refresh message type.

The information contained in a template is passed to the client FAST decoder at run-time such that the decoder recognizes how each field is encoded in terms of data type representation (Transfer) and data redundancy removal (Field).

The template is constructed of several sections including Template Identification, Header, Body and Sequence.

•      Template Identification provides the template name and identifier.

•      The Header includes FIX header fields such as ApplVerID (tag 1128), MsgType (tag 35), and SendingTime (tag 52).

•      The Body provides information common across all repeating groups.

•      Sequence represents a repeating group with a corresponding length field and a set of repeating group fields which carry the detailed entry information.

Refer to the Appendix for a decomposition of the template above with an explanation for each instruction.

Refer to the Appendix for a decomposition of the template above with an explanation for each instruction.

Note: CQG requires that client systems use an API layer to load templates rather than implement hard coded templates. Since templates are subject to change, this will facilitate template modification in production environments.

Templates are available from the FTP site at: ftp://develop.cqg.com