MQ Internals Explained (a little bit)

IBM likes to market IBM MQ (aka WebSphere MQ, MQSeries) as a simple and complete solution for business’s messaging needs. Going with the KISS approach is wise (very wise in some cases).

IBM talks about the 2 different messaging models that IBM MQ supports: Point-to-Point (P2P) and Publish/Subscribe (Pub/Sub).

IBM will:
• show you how to put messages to local/alias/cluster/remote queues (P2P) or get messages from local/alias/cluster queues (P2P).
• show you how to publish messages to a topic string or object (Pub/Sub) or subscribe to a subscription (or topic) to retrieve messages from a topic string or object (Pub/Sub).
• show you how to create an alias queue that can point to (1) a local queue (P2P) or (2) a topic (Pub/Sub) and if you use a topic, how to cross messages from the P2P model to the Pub/Sub model.
• show you how to create a subscription that can point to (1) a local queue (P2P) or (2) a topic (Pub/Sub) and if you use a local or alias queue, how to cross messages from the Pub/Sub model to the P2P model.

IBM likes to show the world that IBM MQ use of Point-to-Point and Publish/Subscribe is a single messaging model. While IBM markets IBM MQ as a single messaging model is fine (aka the KISS approach), it sometimes makes my life a little difficult when explaining either MQ Message Encryption or MQ Auditor to customers (although, less so with MQ Auditor, because by default, it collects information from both messaging models).

Here’s an example: A customer recently sent me a picture of a setup they were trying to implement with MQ Message Encryption (MQME) and they were confused why MQME did not do its job of encrypting messages when the messages were put/published at different points in the setup.

Here’s the picture of their setup:

For those who prefer MQSC definitions, here they are:

DEFINE QALIAS('TEST_ALIAS') TARGET('TEST_TOPIC') TARGTYPE(TOPIC)
DEFINE QREMOTE('TEST_QUEUE_REM') RQMNAME('QMGR2') RNAME('TEST_QUEUE') XMITQ('QMGR2')
DEFINE QLOCAL('TEST_QUEUE_LOC')
DEFINE TOPIC('TEST_TOPIC') TOPICSTR('MQME/TEST')
DEFINE SUB('TEST_SUB_1') TOPICSTR('') TOPICOBJ('TEST_TOPIC') DEST('TEST_QUEUE_REM')
DEFINE SUB('TEST_SUB_2') TOPICSTR('') TOPICOBJ('TEST_TOPIC') DEST('TEST_QUEUE_LOC')

When they put a message to the alias queue, everything was fine but when they published a message to the topic ‘MQME/TEST’ then the message was not encrypted.

In the user’s MQME IniFile, they only had set the protected queue(s) of ‘TEST*’. They did NOT have any protected topic definitions.

There are many things I need to explain that goes on under the covers with MQ. I will do my best to explain it in a logical order but I know that I’m about to confuse many people.

Things to remember:

(1) When messages flow through P2P model/environment, the message can be a plain message, JMS message (aka MQRFH2), CICS message, IMS message, SAP message, etc.

Layout for 5 example messages for P2P:

{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}
{CICS header}{message payload}
{IMS header}{message payload}
{SAP header}{message payload}

(2) When messages flow through Pub/Sub model/environment, the message MUST be a JMS message (aka MQRFH2) formatted message. Now a JMS message can contain a CICS message, IMS message, SAP message, etc.

Layout for 4 example messages for Pub/Sub:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{CICS header}{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{IMS header}{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{SAP header}{message payload}

Note: The user can change the queue’s property attribute (PROPCTL) or subscription’s property attribute (PSPROP) to ‘NONE’, so that MQ removes the message properties (aka named properties) and folders when the message is retrieved by the application.

Item #1:
When an application opens an alias queue, internally that object type is called MQOT_Q. Hence, MQ is operating in the P2P model. Since, it is the P2P model, MQME will check its protect queue definitions for the a match. If MQME finds a match then all messages put to or retrieved from that object handle (aka opened queue) will be encrypted or decrypted (assuming permissions are correct).

When the user ran an application to publish a message to topic ‘MQME/TEST’, the topic was opened and the object type was MQOT_TOPIC. Since, it is the Pub/Sub model, MQME will check its protect topic definitions for the a match. In this particular case, MQME did not find a match, so it will not encrypt the messages or decrypt the messages for subscribers.

Now if the user adds a protect topic definition to the MQME IniFile with something like ‘MQME/TEST’ or ‘MQME/*’ or ‘MQME/TEST*’ then MQME would find a match and encrypt or decrypt messages of that particular topic.

Here’s the analogy that I like to use with customers, think of IBM MQ as a brain (human brain if you will). Each person has a single brain. IBM MQ provides a single messaging infrastructure. But if you look closely at the brain, you will see that it is made up of 2 hemispheres and if you look closely, you will see that IBM MQ is made up of 2 different messaging models: Point-to-Point and Publish/Subscribe.

So, how an application opens a queue or topic will dictate which side of the brain (aka messaging system) MQ will use to handle the getting or putting of messages. And if companies have a configuration where messages flow back and forth between the 2 messaging systems then the users of MQME need to beware of the point of contact with the MQ (aka the brain).

Item #2:
Did you know that when a message hops between P2P and Pub/Sub that the message actually changes! Also, for those of you that are using MQ’s Pub/Sub for message replication, did you know that not all MQMD fields of the original message are replicated to the copied messages!

Example:

Using the above customer setup, if you start MQ Explorer and put a test message to the alias queue ‘TEST_ALIAS’ then the following is what happens to the message:

– Putting a plain message to an alias queue

{message payload}

– When MQ transfers the message to a topic object or subscription then the message becomes

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

– When the message is put to the XMITQ queue it looks like

{XMIT header}{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

– When it is put to the local queue of the local queue manager, it is still in the format:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

– When it is put to the local queue of the remote queue manager, it is still in the format:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

Note: For those newbies out there, MQRFH2 is the internal name for a JMS message. Also, yes, I know on the subscription, the user can change the Property attribute (PSPROP) to ‘NONE’, so that MQ removes the message properties (aka named properties) which in theory turns the message back to a regular message.

Users of MQ should really be aware of what goes on under the covers with MQ, so that they don’t blame MQ when they don’t get the results they want.

Also, there are far simpler solutions for handling message replication. i.e. MQ Message Replication

If you want, you can sub-title this blog posting as ‘your brain on MQ!’ 🙂

Regards,
Roger Lacroix
Capitalware Inc.