MQGem Tools TLS Configuration

This blog post is going to cover how to set up TLS secured client channels when using one of our command line tools, such as QLOAD, Q or MQSCX, to connect to your queue managers. Before I start, I just want to say up front that there is nothing special about how you do this. Our tools are all ‘C’ language MQI applications and so setting up TLS for them is just the same as setting it up for the IBM supplied ‘C’ language sample amqsputc as an example. If you know how to do that, then you know how to set it up for our command line tools.

If you want similar instructions for our GUI tools MO71 and MQEdit, please see this post.

Certificates

In order to have TLS secured connectivity between your MQGem tool and your queue manager, your queue manager must have a certificate, hopefully not a self-signed certificate; and you, the MQGem tool user, must also have a certificate identifying you. It is likely that this certificate identifying you will be used for connectivity from various different tools, so it does not need to be specific to the tool you are using. This blog post is not going to cover setting up the queue manager certificate as that has been amply covered in IBM Docs. What you need at your machine where you are running an MQGem tool is a signed certificate identifying you, and a copy of the CA certificate that signed the queue manager’s certificate. It is possible that you have several of the latter item depending on how your queue managers are setup. One hopes that you have many less of these than the number of queue managers you need to manage.

Key Database File

MQGem tools are ‘C’ language MQI applications. This means that the IBM MQ Client code expects it to use a KDB file with a matching named stashed file. Unlike Java programs, you do not provide the password to the certificate store (KDB file) anywhere, the IBM MQ Client code finds the stash file that contains the password due to the fact that it has the same name as the KDB file. Here is my set of files:-

 Directory of C:\MQGem

06/07/2021  16:43                88 MQGemClient.crl
06/07/2021  16:43            20,088 MQGemClient.kdb
06/07/2021  16:43                88 MQGemClient.rdb
06/07/2021  16:43               193 MQGemClient.sth
               4 File(s)         20,457 bytes

Later you will see when setting up your certificate configuration that you use the name up to, but not including, the file extension. The IBM MQ Client code can construct the name of the key repository and password stash file by appending the “.kdb” and “.sth” file extensions respectively.

As you follow the instructions in this blog post, you are going to create a number of files, the above KDB and associated ancillary files, a CCDT and an mqclient.ini file. It is useful to first create a directory to keep all these files together.

If you don’t already have a KDB file supplied for your use, you should create one now. Click on the twisty to see the details for each step that you need to do.

Step 1: Create a KDB file

IBM MQ supplies tools for you to create the files you need. In this case we show examples of using the command line tool runmqakm. There is also a GUI tool that can do the same job. Here is the command to create your KDB file. In my example, the KDB file (and accompanying stash file etc) will be created in the C:\MQGem directory, using names MQGemClient.* – you may wish to change this. In addition, you probably want a stronger password too!

runmqakm -keydb -create -db C:\MQGem\MQGemClient.kdb -pw passw0rd -type cms -stash

If you want, you can use the runmqakm tool to create a password for you:-

runmqakm -random -create -length 16 -strong

The KDB file, once created, is the place to put the various certificates you need. As mentioned above, you’ll need to have the CA certificate that signed your queue manager(s) certificates. This is so that the certificates coming from the queue managers at connection time can be validated by the IBM MQ Client as part of the TLS handshake.

If you have one or more files containing the CA certs to allow you to validate connections to your queue manager(s), add them to your KDB now. If you only have a single certificate file (a PKCS#12 file) that also contains the CA certs, skip Step 2.

Step 2: Add CA certificates to KDB

Here is the command to add a CA certificate to your KDB. As before, you are likely using a different location in your file system for the KDB file. Note that I am using the -stashed parameter from now on, so I never have to type in the password again. The label you use when you add a CA certificate doesn’t matter. Just make it something that is helpful in remembering what this certificate is. In my example MQGemProdCA is the label I have used for the production environment CA certificate.

runmqakm -cert -add -db C:\MQGem\MQGemClient.kdb -stashed -label MQGemProdCA -file MQGemProdCA.arm -format ascii -trust enable

Finally you need to have a certificate identifying you, the user of an MQGem tool. You may have been given a file already containing your signed certificate, and have to add that to your KDB, or you may need to generate a certificate request, get it signed by your company CA, and then when you get it back, receive it into the same KDB where you created the request. Choose either Step 3a or 3b below depending on how your company does this.

DO THIS: Step 3a: Add PKCS#12 private certificate into KDB

This special certificate file type is one that contains both your certificate and the private key for the certificate. It is very important to keep the private key safe, so when transporting this around in a PKCS#12 file, this particular type of file has a password. Other files that only contain certificates without private keys don’t have passwords.

Here is the command to add a PKCS#12 certificate file to your KDB. This command will also add the CA certificates that are in the file. Note that I am using the -stashed parameter for the KDB password, so I never have to type in the password again. The password on this command is the one that goes with the PKCS#12 file you have been given.

runmqakm -cert -import -file MoragCertificate.p12 -type pkcs12 -pw pkcs12passw0rd -target c:\MQGem\MQGemClient.kdb -stashed

OR THIS: Step 3b: Create a Certificate Request

To make a certificate request, issue the following command, using the KDB file that you just created.

runmqakm -certreq -create -db C:\MQGem\MQGemClient.kdb -stashed -label MQAdminProd -dn "CN=Morag Hughson,OU=Prod,O=MQGem Software" -file Clnt_Prod_Request.csr

Now send the certificate request file off to the CA that signs certificates for your environment. When you receive the signed certificate file back from them, receive it into the same KDB as before with the following command.

runmqakm -cert -receive -db C:\MQGem\MQGemClient.kdb -stashed -file Clnt_Prod_Signed.arm

As a final check that everything got added ok, list the contents of your KDB with the following command.

runmqakm -cert -list all -db C:\MQGem\MQGemClient.kdb -stashed

You should see output something like this. There should be at least one trusted certificate (shown with a !) and at least one personal certificate (shown with a -). You will need the label of the personal certificate when you come to set up your certificate configuration in a moment, so remember that.

5724-H72 (C) Copyright IBM Corp. 1994, 2020.
Certificates found
* default, - personal, ! trusted, # secret key
!       MQGemProdCA
-       MQAdminProd

Client-connection Channel

MQGem command line tools generally rely on the use of a Client Channel Definition Table (CCDT) to provide the client-connection channel definition details, rather than having lots of command line parameters to cover all the possible attributes. Use of a CCDT also makes it simpler to use the same channel definition details across a range of tools instead of having to specify the attributes individually for use with each tool. It is therefore a highly recommended method. If you don’t already have a CCDT with your client-connection channel details you should create one now.

Step 4: Create TLS-secured CLNTCONN channel in CCDT

In modern versions of IBM MQ, the CCDT can be created on the client machine using either runmqsc or our MQSCX tool, starting the tool with the -n flag to indicate you wish to create a CCDT.

Both runmqsc and MQSCX will pay attention to the MQCHLLIB and MQCHLTAB environment variables to determine the location where the CCDT file will be written. Additionally, MQSCX has a -t flag that you can use to specify the full path and file name you want instead.

If you want to use a JSON CCDT, MQSCX can be used to create one of those by using the -t flag. See Working with JSON CCDT files blog post for more details.

Once your MQSC tool of choice is running in CCDT mode, create the channel with a command like the following.

Your company may choose to use a different CipherSpec (the SSLCIPH) parameter. I have chosen to use the more recently introduced ANY_TLS12. If you can’t use one of the “ANY” values then you should ensure that the same value is used here, and in the SVRCONN channel (next step).

DEFINE CHANNEL(MQGEM.SVRCONN.TLS) CHLTYPE(CLNTCONN) TRPTYPE(TCP) CONNAME('GEMWIN3.mqgem.com(1701)') SSLCIPH(ANY_TLS12) QMNAME(MQG1)

Server-connection Channel

You’re also going to need to create a TLS-secured server-connection (SVRCONN) channel on the queue manager if you don’t already have one in place.

Step 5: Create TLS-secured SVRCONN channel

Here’s an example MQSC command to define such a channel. The name you use for your channel may well be different. Whatever it is, remember it for the next step.

Your company may choose to use a different CipherSpec (the SSLCIPH) parameter. I have chosen to use the more recently introduced ANY_TLS12. If you can’t use one of the “ANY” values then you should ensure that the same value is used here, and in the CLNTCONN channel (previous step).

DEFINE CHANNEL(MQGEM.SVRCONN.TLS) CHLTYPE(SVRCONN) TRPTYPE(TCP) SSLCIPH(ANY_TLS12) SSLCAUTH(REQUIRED)

mqclient.ini configuration

When setting things up outside of the MQGem tool itself, you can either use environment variables or the mqclient.ini file to point to all the various settings. We’ll be showing you the latter today. If you’re not familiar with why the mqclient.ini file is our recommendation, you might find this blog post interesting.

Step 6: Create mqclient.ini file

In your working directory where you have your KDB and CCDT files, create a text file with the name mqclient.ini. Edit this file to add the following contents.

Note the ‘.’ to mean “this directory” in the value for the ChannelDefinitionDirectory. Alternatively you can fully spell out your directory.

CHANNELS:
  ChannelDefinitionDirectory=.
  ChannelDefinitionFile=MQGEM.TAB
SSL:
  SSLKeyRepository=C:\MQGem\MQGemClient
  CertificateLabel=MQAdminProd

Test Connectivity

Now that you have your certificates set up, your client channel definition created, and your mqclient.ini file pulling all the information together, you can run your command line MQGem tool to pick up and use all these things you have configured.

The easiest way to do so is to ensure you are in the directory containing the mqclient.ini file before you run the command invocation. The IBM MQ client will pick up the mqclient.ini that is in the current directory by preference. However, if you want to do something different, this page shows all the different mechanisms.

Step 7: Run Q using TLS connection

From in your directory containing the mqclient.ini file, invoke Q with a command like the following.

The -l mqic flag tells Q to make a client connection rather than a locally bound connection, and the rest of the invocation is up to you. In our example we are just opening queue Q1 for output.

q -m MQG1 -l mqic -o Q1

Step 7: Run QLOAD using TLS connection

From in your directory containing the mqclient.ini file, invoke QLOAD with a command like the following.

The -l mqic flag tells QLOAD to make a client connection rather than a locally bound connection, and the rest of the invocation is up to you. In our example we are just browsing queue Q1 to a file.

qload -m MQG1 -l mqic -i Q1 -f q1.qld

Step 7: Run MQSCX using TLS connection

From in your directory containing the mqclient.ini file, invoke QLOAD with a command like the following.

The -l flag tells MQSCX to make a client connection rather than a locally bound connection.

mqscx -m MQG1 -l

If you receive an error when trying to connect, remember to use the client error log, found in <MQ_DATA_PATH>/errors/AMQERR01.LOG. Scroll to the bottom for the most recent error.

Summary

It is hoped that this post will help you set up TLS-secured connectivity between MQGem command line tools and your queue manager. Remember that the same KDB file and certificates will also work with the IBM-supplied samples such as amqsputc. If you have any issues, it may be helpful to try things out with that very simple sample first. It is also good advice if you get validation problems to change the SVRCONN channel to use SSLCAUTH(OPTIONAL) and get it working that way first. Once it does then go back to SSLCAUTH(REQUIRED) knowing that the problem is that you are either not sending, or the queue manager is not validating, the client side certificate.

If you have IBM MQ V9.3.0 (or later) as your client installation you can also now make use of a new feature in that release which allows you to avoid having a stash file. See this post for more details.

If you have situations where you are trying to set up TLS-secured connectivity for MQGem tools that are not covered in this post, please do get in touch, either in a comment below, or through any of the usual channels (see About page).

Advertisement

File names in JCL

As a result of reading Lyn Elkins’ posts about how she struggled with the quotation marks required to specify a file name as a parameter in a JCL PARM string, I thought I would write a little summary. I personally prefer to use DD cards for my file names whenever possible as these avoid the need for quotation marks and leave your eye less likely to be squinting by the time you are done. I will show the fully qualified example right at the end for anyone who really wants it though. I am using QLOAD as an example program here, but any application that simply does an fopen() can make use of the same syntax. For example, when supplying a script in a file using MQSCX, providing an input file to load onto a queue in QLOAD, or some string data to become message data in the Q program. Any parameter that expects a file name can also instead take a string of the format “DD:card-name” (note that you MUST use capital letter D’s here). All of these use the same syntax as illustrated here.

//QLOAD    JOB
//*********************************************************************
//* QLOAD Load messages onto Q1
//*********************************************************************
//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f DD:INFILE1')
//STEPLIB  DD DSN=GEMUSER.USER.LOAD,DISP=SHR                         
//         DD DSN=IBM.MQ.SCSQAUTH,DISP=SHR                           
//         DD DSN=IBM.MQ.SCSQANLE,DISP=SHR                           
//SYSPRINT DD SYSOUT=*  
//SYSOUT   DD SYSOUT=*      
//INFILE1  DD DSN=GEMUSER.DATA.MSGFILES(EXMPL1),DISP=SHR
//INFILE2  DD DSN=GEMUSER.DATA.OUTPUTQ1,DISP=SHR 
//INFILE3  DD PATH='/u/gemuser/Q1.qld',PATHOPTS=(ORDONLY)          
//MQGEML   DD DSN=GEMUSER.MQGEM.LIC,DISP=SHR
//

Here you can see that the JCL PARM string does not need any extra quotes at all, just the string DD:INFILE1 where INFILE1 is the name of a DD card in the JCL job. The DD card can then specify a fully qualified PDS member, MVS file, or HFS file as the three examples show.

To do the same job as shown in these three examples by providing the file name inline in the JCL PARM string, things start getting complicated by the fact that extra quotation makes are needed to escape the quotes since the whole JCL PARM string is surrounded by single quotes itself. This is what makes it so problematic, and why I always use DD cards instead.

If you take advantage of that the fact that MVS knows your TSO login and will pre-pend it to the MVS file name you supply, you can avoid quite a number of the quotation marks needed. The following two EXEC statements are equivalent when the TSO login is GEMUSER. The way you tell MVS not to pre-pend your TSO login is to surround the name you supply with single quotes, which turn into doubled up single quotes when used inside a JCL PARM string! The whole thing also needs to be enclosed in double quotes because of the slashes which are considered a delimiter to separate a JCL PARM string into run-time options and parameters to pass to the program.

//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f"//DATA.MSGFILES(EXMPL1)"')

//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f"//''GEMUSER.DATA.MSGFILES(EXMPL1)''"')

The two slashes shown in both the above example JCL PARM strings are how you tell fopen() that you are referencing an MVS file (to distinguish it from an HFS file). Here is an example EXEC statement using an HFS file instead. Again, this string needs to be surrounded by double quotes because of the slashes. The second example has no slashes, and will end up in your current directory. In fact any string that does not start with DD: or a slash (or two) will be interpreted as an HFS file in your current directory, and is why you might end up with an HFS file called “dd:SUMMARY” if you forgot to use capital letter D’s.

//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f"/u/gemuser/Q1.qld"')

//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -fQ1.qld')

Well, I hope that helps, and also probably convinces you that DD cards are the way to go.

I didn’t invent this stuff, I’ve just spent a lot of time using it. Here is some reference material that helped me get to grips with it. Perhaps it can help you too.

• Using fopen() > Using a DDname
• Using fopen() > Using a data set name

On a separate note, if you ever have a long JCL PARM string, don’t struggle with line continuations, move over to using PARMDD – see JCL with long PARM strings

Is there a way to clear an MQ queue of all messages?

We were recently asked how to clear a queue of all messages within one of our tools.

In this post we cover how to do this using MQGem tools as several of them offer this capability. Open up the twisty for each tool to read how to do this.

Using MO71

There are two options for clearing a queue of all messages when using MO71. One directly interacts with the messages in a Message List dialog, and the other uses the Clear Queue command via the command server. The latter can fail if someone else has the queue open.

MO71: Using the Message List dialog

In MO71 you can browse a list of messages on a queue, then in the context menu of that dialog choose Message Selection → Apply to all Messages. Then press the Delete All button, or select Delete All from the right-mouse button context menu and all the messages will be deleted.

Delete all messages using MO71

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the message(s) to a separate holding queue, just in case the message proves to be important after all. MO71 makes it simple to Move messages too. Just fill in the holding queue name, and then press the Move All, rather than Delete All, button.

MO71: Using the Clear Queue command

From a Queue List dialog in MO71, you can choose to clear a queue from the context menu. As a destructive command, this will show a confirmation dialog before it goes ahead.

Delete all messages using the Clear command in MO71

Remember that this command can fail if the queue is open by another application at the time.

Using the Q program

You can use the Q program to quickly destructively get off all the messages on a queue, to ensure it is empty before you begin using it for something else. This can work even when the queue is in use (unless it is exclusively in use of course) and the CLEAR QLOCAL command can’t be used.

Here is an example which destructively gets (-I) from Q1 and uses a zero-length buffer and accepts that the messages will be truncated (-=t0):-

q -m MQG1 -I Q1 -=t0

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the messages to a separate holding queue, just in case they prove to be important after all. This is simple to achieve using a command like the following:

q -m MQG1 -I Q1 -o HOLDING.Q

Using QLOAD

You can use QLOAD to quickly destructively get off all the messages on a queue, to ensure it is empty before you begin using it for something else. This can work even when the queue is in use (unless it is exclusively in use of course) and the CLEAR QLOCAL command can’t be used.

Here is an example which destructively gets (-I) from Q1 and discards them (-f null), aka sends them to the null file destination:-

qload -m MQG1 -I Q1 -f null

Take care when deleting messages from a production queue manager. We would recommend using QLOAD to actually move the messages to an output file, in case you change your mind about deleting them!

qload -m MQG1 -I Q1 -f c:\temp\Q1deletedmsgs.qld 

Using MQEdit

In MQEdit you can browse a list of messages on a queue, then in the context menu of that dialog choose Message Operations → Apply to all Messages. Then press the Delete All button, or select Delete All from the right-mouse button context menu and all the messages will be deleted.

Delete all messages using MQEdit

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the message(s) to a separate holding queue, just in case the message proves to be important after all. MQEdit makes it simple to Move messages too. Just fill in the holding queue name, and then press the Move All, rather than Delete All, button, or simply Drag and Drop the messages onto the new queue.

Using MQSCX

You can of course issue the MQSC CLEAR QLOCAL(q-name) command from MQSCX. However, it can go one better than this. The CLEAR QLOCAL command is problematic because it can fail when someone else has the queue open, even if it is not open in a way that stops you getting messages. Instead you can issue the following command in MQSCX:-

=clear qlocal(Q1)

then MQSCX will first attempt the CLEAR QLOCAL command and if that fails with "object in use" then it will switch to getting the messages off the queue instead. You can read more about this in MQSCX: Clearly a better way.

---

If you don't have any of these tools, but would like to try any or all of them out, please contact support@mqgem.com and a 1-month trial licence will be sent to you with no obligation to buy. You can download the tools from our website.

Finding the reason why a message was sent to the Dead Letter Queue (DLQ) in IBM MQ

IBM Support recently released a Support Doc which was titled:-

Summary

Finding the reason why a message was sent to the Dead Letter Queue (DLQ) in IBM MQ

The document described several options, which often required you to look at the message in hex and, when required, byte swap the value to find the reason code. In this post we cover how to do this using MQGem tools as several of them offer this capability, and our tools format out the Dead Letter Header so you don’t need to do any manual byte swapping. Open up the twisty for each tool to read how to do this.

Using MO71

In MO71 you can browse the messages on the Dead Letter queue using a Message List. If you choose to show the “Message Summary” column on your list dialog (see Configuring List Dialogs in MO71 for how to change the columns in a list dialog), you will immediately be able to see the reason code in the Dead-Letter header, as that is one of the useful things that Message Summary displays look for.

A message List dialog will summarise the Dead Letter header of a message for you

Alternatively you can double click on a message to look at the full details of an individual message, and you will see the full Dead-Letter header (MQDLH) is formatted out for you to see.

The full message display shows all of the Dead Letter Header formatted out

Using the Q program

To view your messages on the Dead Letter queue, and have them formatted out, you need to use the -df option which also has three levels of detail. For full detail use -df3. Here’s an example command:-

q -m MQG1 -i MQG1.DEAD.LETTER.QUEUE -df3

Which will format the Dead Letter header like this making it really easy to understand the reason why the message was placed on the DLQ.

[  181 bytes] Message Content
[  181 bytes] Dead Letter Queue Header (MQDLH)
StrucId      :'DLH '
Version      :1
Reason       :2085 (Unknown object name)
Dest. Queue  :'TARGET.QUEUE                                    '
Dest. QMgr   :'MQG1                                            '
MQEncoding   :0x'222' (Reversed)
CCSID        :437 (IBM PC)
Format       :'MQSTR   ' (String)
PutApplType  :11 (Windows NT)
PutApplName  :':\mqm9250\bin64\amqrmppa.exe'
Put Date     :'20220411'
Put Time     :'05021009'
Some text

Using MQEdit

When you look at the messages on a queue using MQEdit, the message list will show you a summary of the message in the “Message” column, which for Dead Letter header messages will show you the reason from the header. This makes it very easy to see the reasons why messages are on your Dead Letter queue.

Selecting a message from the list will display it in the pane below with the Dead Letter header formatted out also showing you the reason why the message ended up on the DLQ.

MQEdit shows a summary of the Dead Letter header in the message list, and the fully formatted message in the lower panel

Using QLOAD (honourable mention)

QLOAD isn’t a message formatting tool, so it will not help you to view the reason code in the Dead Letter header of a message. However, it does get an honourable mention in this list because it can help you to move, or unload messages from your Dead Letter queue that have a specific reason code in the Dead Letter header. This can be very helpful if you want to siphon off some specific errors to handle in a different way.

Use a command like this to copy (-i is browse) all the DLQ messages for MQRC_UNKNOWN_OBJECT_NAME (2085) to a file.

qload -m MQG1 -i MQG1.DEAD.LETTER.QUEUE -Gr:2085 -fUnknownObjs.qld

When you’re using this feature, QLOAD will report the message selection that you have active:-

Message selection active:
  Messages with DLQ Reason 2085 Unknown object name

Read    - Files:   0  Messages:     3  Bytes:      5616
Written - Files:   1  Messages:     1  Bytes:       181

---

If you don’t have any of these tools, but would like to try any or all of them out, please contact support@mqgem.com and a 1-month trial licence will be sent to you with no obligation to buy. You can download the tools from our website.

Q on z/OS

MQGem Software is pleased to announce that the Q Utility, is now available natively on z/OS.

Back when Q was a SupportPac, it was available on the z/OS platform, and we are pleased to be able to once again provide it on z/OS.

Q has evolved somewhat from the days when it was a SupportPac, and there have been a number of features added since then. These features are now also available natively on z/OS. Some of the highlights of those additional features include:

• New help features
  To make it easier to find the option you are looking for since the Q program does have a lot of flags!
• Minor flag enhancements
  See Announcing Q from MQGem for full details.
• Message formatters added
  Formatters for JSON, EDIFACT, CSV and FIX message formats have been added.
• Character substitution
  This will display text such as &gt; as the character ‘>’.
• New message summary display
  Get a one line summary of the messages on your queue. See Q version 9.2.0 is released for an example of this and a few other formatting enhancements.

To enable Q to run on z/OS you require a z/OS specific licence. A distributed platform Q licence will not enable Q on z/OS to run.

---

You can download Q for z/OS from the Q Download Page. If you’d like to try out Q on z/OS, please email support@mqgem.com and a 1-month trial licence will be sent to you.

Q: message operation menu

The Q program can do many different operations, many of which you will be familiar with, or which will be obvious just by glancing at the usage text of the program. There is a separate menu of options available at the prompt where you type in message data. I briefly mentioned this in the previous post about the Q program. I thought it might be worth taking a more detailed look at this part of the Q program today.

If you invoke the Q program like this:-

q -m MQG1 -o Q1

then at the prompt, you can type a ‘?’ to see the full usage text for this prompt:

Connecting ...connected to 'MQG1'.
>?
Formatted message :
 #end     = End program

 #[!][x][X][i][I][c[g[Count]]][p]Reps[/Size[/Delay[/Commits]]]
  [!]     = Don't send # string
  [x]     = Message is full range of hex characters
  [X]     = Message is indexed full range of hex characters
  [i]     = Index the messages
  [I]     = Index the messages using words
  [c]     = Checksum the messages
  [g]     = Checksum group - followed by optional count
  [p]     = Change properties
  Reps    = Number of messages to MQPUT
  Size    = Size of message (-ve range)
  Delay   = Number of milliseconds delay between MQPUTs (-ve range)
  Commits = Commit after n messages
>

Multiple Messages with one command

The simplest use of this menu is just using it to put multiple messages with one command string. So for example, if you type the following at the prompt:

#!20/1024/1000/5 My message text

You will be putting 20 messages, each of size 1K, with a 1 second delay between messages, and in transactions of 5 messages at a time, and the message text will start with “My message text”, i.e. because of the ! in the command, it won’t include the #!20/1024/1000/5 part of the command.

As you can imagine, this can be useful for having a trickle feed of messages appearing on a queue to test many aspects of an application. You can also randomise the size and delay by simply making the number negative. A message size of 1024 means you want messages exactly of that size. A message size of -1024 means you want messages to have a range of sizes up to 1024 bytes.

Random content

In the above example, we have a 1K message, but only a very short string of text. You might instead want to have some random content to fill your message. This can be done by adopting the first half of the check-summing feature which is explained in a moment. You can use a command like:

#!c20/1024

What’s not obvious here, that I should explicitly mention, is that there is a space character on the end of this command. It is this that ensures that the Q program will know to not put the #-command in as your message content, since you have a space on the end of the command. So long as you have the space, your message will contain 1K of random data.

Check-summing messages

If you want the ability to run messages through your system and test that they are being received complete and without corruption, you might choose to make use of the check-summing facility of the Q program. This has two parts. First the Q program generates messages with random content, and then calculates the checksum by adding up the numerical values of each byte in the message multiplied by its offset in the message. It will then print the checksum of the message to the screen.

Connecting ...connected to 'MQG1'.
>#c1/1024
[Chk.-683127]

Now, at the other end of the transmission of this message across your network, you can use the Q program to do a simple get of the message and it will print out the checksum again.

Connecting ...connected to 'MQG1'.
(1024 bytes) [Chk.-683127] #c1/1024)#.äßl.«R.I±±.Td.ª.<ç.>.$^....G.

Indexing messages

If you’re demonstrating a series of messages and what happens to them, for example when looking at how clustering round-robins your messages to different queue managers, you might find the indexing feature of the Q program useful. This feature allows you to very easily have your messages count up where they are in the sequence. Here’s an example (with a space on the end):-

Connecting ...connected to 'MQG1'.
>#!i10

which results in the following series of messages:-

Msg 1 of 10: Source Q1/MQG1
Msg 2 of 10: Source Q1/MQG1
Msg 3 of 10: Source Q1/MQG1
Msg 4 of 10: Source Q1/MQG1
Msg 5 of 10: Source Q1/MQG1
Msg 6 of 10: Source Q1/MQG1
Msg 7 of 10: Source Q1/MQG1
Msg 8 of 10: Source Q1/MQG1
Msg 9 of 10: Source Q1/MQG1
Msg 10 of 10: Source Q1/MQG1

Alternatively, you can have the number in words, by using the capital I alternative:

Connecting ...connected to 'MQG1'.
>#!I10

which results in the following series of messages:-

One
Two
Three
Four
Five
Six
Seven
Eight
Nine
Ten

The Q program can count pretty high too! 🙂

For those who like the ultimate in flexibility, you can build your own message string here as well, for example:-

Connecting ...connected to 'MQG1'.
>#!i10 Msg num %i (%I) out of %l (%L) put onto queue %q on qmgr %m

which results in the following series of messages:-

Msg num 1 (One) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 2 (Two) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 3 (Three) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 4 (Four) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 5 (Five) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 6 (Six) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 7 (Seven) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 8 (Eight) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 9 (Nine) out of 10 (Ten) put onto queue Q1 on qmgr MQG1
Msg num 10 (Ten) out of 10 (Ten) put onto queue Q1 on qmgr MQG1

So, if you have testing requirements, the Q program has many options for you. If you have more specific requirements for what your message content should look like when testing, remember that the QLOAD program can help you load up real message data at any rate you require.

---

Read more about the Q program, and download a copy, from the Q Web Page. If you would like to try out MQGem’s Q program before deciding whether to buy then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

Tasks we use the Q program for

The Q program was (almost^[1]) the first Support Pac (MA01) that Paul wrote, and he still uses it on an almost daily basis. Here are some of the tasks that we use it for on a quite regular basis.

Format Out Messages

If you have messages on a queue with a built-in IBM MQ format, then the Q program can format those out. For example, an event message, or messages on your Dead-letter queue. Sometimes just a quick look with Q is all you need to figure out what that message has to say.

Here is an example which browses (-i) the DLQ and formats (-df) the messages using detail level 3 (-df3):-

q -m MQG1 -i SYSTEM.DEAD.LETTER.QUEUE -df3

Empty a queue

You can use the Q program to quickly destructively get off all the messages on a queue, to ensure it is empty before you begin using it for something else. This is a common task we use it for. This works even when the queue is in use and the CLEAR QLOCAL command can’t be used.

Here is an example which destructively gets (-I) from Q1 and does not print the messages to the screen (-q), aka ‘quiet’ mode:-

q -m MQG1 -I Q1 -q

Copy the contents from one queue to another

When working with test messages that you have saved off for reuse, being able to quickly copy them onto your working queue is very handy. You can even do this from one queue manager to another.

Here is an example which copies messages, that is browses them (-i) from Q1 on MQG1, and puts them (-o) to Q1 on MQG2. It uses a shared blocking connection (-xb) to allow locally bound connections to be made to two queue managers in the same application:-

q -m MQG1 -xb -i Q1 -m MQG2 -o Q1

Put test messages

Sometimes you just need to have some test messages to work with.

• I need a lot of messages
  Here is an example which puts 1000 simple test messages:-

  q -m MQG1 -o Q1 -M"#1000 Test Message"

  Alternatively, you can invoke the Q program like this:-

  q -m MQG1 -o Q1

  and then at the prompt, type in the # string.

  Connecting ...connected to 'MQG1'.
  >#1000 Test Message
  >

• I need a big message
  Here is an example which puts only one, 10000 byte message:-

  q -m MQG1 -o Q1 -M"#1/10000 Test Message"

• I need a continual trickle feed of messages
  Here is an example which puts 1000 messages, with a random size up 2000 bytes, with 100 ms delay between each message put:-

  q -m MQG1 -o Q1 -M"#1000/-2000/100 Test Message"

If you’ve not played with the # string in the Q program before, remember that you can type ‘?’ at the prompt to see the full help. There’s also another blog post all about the # menu here.

Hold open a transaction

It’s surprising how often this one comes up, but being able to start a transaction, and not commit it yet, can be very useful.

Here’s an example where the Q program will commit a transaction after 5 messages are put in the transaction (-p 5):-

q -m MQG1 -o Q1 -p5

And then at the prompt, you can put four messages, and now you have a transaction that contains work but is not yet committed:-

Connecting ...connected to 'MQG1'.
>#4 Test Message
>

Echo program

You can use the Q program to do a simple echo. That is, it will send any message it reads from a queue, back to the ReplyToQ in the message. You can also use the Q program to send messages with a ReplyToQ set.

Here’s the echo program (-e) which get-waits for 120 seconds (-w120) on queue Q1:-

q -m MQG1 -IQ1 -w120 -e

And here’s the driver program, putting to Q1 with a ReplyToQ set (-rREPLY.QUEUE) which is also waiting for 120 seconds (-w120) on the messages coming back to the ReplyToQ (the ‘+’ in -r+REPLY.QUEUE):-

q -m MQG1 -oQ1 -r+REPLY.QUEUE -w120

Create messages in a different code page

A very simple way to create messages in a different code page, is to first create some messages in your own codepage on a queue, and then get the Q program to convert them and put them onto another queue.

Here is an example where the Q program is reading messages from Q1 and converting them into USA EBCDIC (-c37) before putting them onto Q2:-

q -m MQG1 -iQ1 -c37 -oQ2

Do a simple publish to a topic string

The Q program has the full selection of both publish and subscribe options using topic strings and/or topic objects. The simple publish to a topic string is the most commonly used one for us.

Here is an example where the Q program is publishing a retained message (-Tr) to the topic string (-Ts:) 'Price/Fruit/Apples':-

q -m MQG1 -Ts:"Price/Fruit/Apples" -Tr -M"Test Message"

Summary

This is just a small selection of the things that the Q program can do. It is an invaluable tool to have in your IBM MQ toolbox!

---

Read more about the Q program, and download a copy, from the Q Web Page. If you would like to try out MQGem’s Q program before deciding whether to buy then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

---

Footnotes:

1. The very first Support Pac Paul wrote was MO00. Its purpose was to show how it was possible to write an MQI application in Win 3.1. In particular it showed how it was possible to issue an MQGET(Wait) on a non-pre-emptive O/S and ensure your windows could still re-paint themselves while you were waiting for a response message to arrive on a queue.

Is there a way to delete a single message from an MQ queue?

IBM Support recently released a Support Doc which posed the question:-

Question

Is there a way to delete a single message from an MQ queue (arbitrary position in the queue)?

The document described a couple of options. In this post we cover how to do this using MQGem tools as several of them offer this capability. Open up the twisty for each tool to read how to do this.

Using MO71

In MO71 you can browse a list of messages on a queue, then from that list, select an individual message (or multiple messages if you need to). Then press the Delete button, or select Delete from the right-mouse button context menu and the selected message(s) will be deleted.

MO71 uses the MQ facility, get by message token, so only the exact message(s) that you select from the list will be deleted. While the position of the message on the queue is shown, this is not used for deletion and neither is Message Id, since there is no guarantee that this will be unique either. Each message on a queue is guaranteed to have a unique Message Token.

Alternatively, you can double click on a message from the list, to view the whole message individually. Then from that display, having checked all the details, and made sure it is the message you want to delete, you can press the Delete button, or select Delete from the context menu.

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the message(s) to a separate holding queue, just in case the message proves to be important after all. MO71 makes it simple to Move messages too. Just fill in the holding queue name, and then press the Move, rather than Delete, button.

Using the Q program

To delete a single message from an MQ queue using the Q program, you should first discover the message ID of the message you wish to delete, to ensure you are addressing the correct message that you intend to delete. Do this by displaying the messages on the queue like this:-

q -m MQG1 -i Q1 -dd

This will browse (-i) the messages on the queue and display the MQMD after the MQGET (-dd) allowing you to see the Message ID of each message on the queue. You will see output like this for each message:-

==============================================================
----- MQMD after MQGET -----
[  324 bytes] Message Descriptor (MQMD)
Report       :00000000
Message Type :8 (Datagram)
Format       :'MQSTR   ' (String)
Priority     :0 (Lowest)
Persistence  :0 (Not Persistent)
Message Id   :414D51204D514731202020202020202099FA4A6020B74B02
              A M Q   M Q G 1                 . ú J `   · K .
ReplyToQ     :'                                                '
ReplyToQMgr  :'MQG1                                            '

Now you can copy that message ID and use it on another invocation of the Q program to delete that specific message from the queue.

q -m MQG1 -IQ1 -gxm:414D51204D514731202020202020202099FA4A6020B74B02

Bear in mind that message IDs are likely to be unique, but this is not guaranteed. It would be safer to move the messages to a separate queue, before deletion. This is simple to achieve using a command like the following:

q -m MQG1 -IQ1 -o HOLDING.Q 
-gxm:414D51204D514731202020202020202099FA4A6020B74B02

Using QLOAD

If you have already browsed the queue contents and output them to a file, you can find the message Id of the specific message you are attempting to address by looking for the Message Descriptor attribute “MSI”. Here is a snippet from a QLOAD file to show an example.

A FMT MQSTR
A PRI 0
A PER 0
A MSI 414D51204D514731202020202020202099FA4A6020B74B02
A COI 000000000000000000000000000000000000000000000000

Now you can copy that message ID and use it on another invocation of the QLOAD program to delete that specific message from the queue. We would recommend using QLOAD to actually move that message to an output file, in case you change your mind about deleting it!

qload -m MQG1 -IQ1 -f c:\temp\deletedmsg.qld 
-gxm414D51204D514731202020202020202099FA4A6020B74B02

Alternatively, you can use the very flexible search and filtering capabilities of QLOAD to narrow down your search of messages on the queue until you find the one you wish to delete, and then offload it to a file, or delete it entirely.

For example, this invocation removes all messages on a queue that are older than one week, and moves them instead to the named file:-

qload -m MQG1 -I Q1 -T7:0:0 -f c:\temp\oldmsgs.qld

Another example, here we remove messages larger than 1MB from a queue:-

qload -m MQG1 -I Q1 -z 1M -f c:\temp\bigmsgs.qld

A third example, here we remove message containing the string “Acme Limited”:-

qload -m MQG1 -I Q1 -s "Acme Limited" -f c:\temp\acmemsgs.qld

Using MQEdit

In MQEdit you can browse a list of messages on a queue, then from that list, select an individual message (or multiple messages if you need to). Then press the Delete button, or select Delete from the right-mouse button context menu.

MQEdit uses the MQ facility, get by message token, so only the exact message that you select from the list will be deleted. While the position of the message on the queue is shown, this is not used for deletion and neither is Message Id, since there is no guarantee that this will be unique either. Each message on a queue is guaranteed to have a unique Message Token.

Alternatively, you can double click on a message from the list, to view the whole message individually in the pane below, and then from that pane, having checked all the details, and made sure it is the message you want to delete, you can select “Delete Message” from the context menu.

Particularly for production queue managers, we would always suggest that it is safer to move message(s) to a holding queue rather than deleting them immediately. In MQEdit it is very simple to move a message between queues, all you need do is drag and drop the required message(s) from one queue to another. You can even copy and move messages between queue managers this way.

---

If you don’t have any of these tools, but would like to try any or all of them out, please contact support@mqgem.com and a 1-month trial licence will be sent to you with no obligation to buy. You can download the tools from our website.

Using User IDs with Q

There are a number of different way that you might interact with user ids when using the Q program. This blog post will provide examples of each.

Providing a User ID and Password at Connect time

q -m MQG1 -u mqgemusr -o Q1

Since no password has been provided with -U on the command line, Q will prompt for the password and what you type in will be echoed to the screen as asterisks.

Please enter password for QM(MQG1) User(mqgemusr) >********

Setting the user ID in the Message Descriptor

When putting a message, to a queue or topic, it is possible to set the various context fields in the Message Descriptor. To do this, you indicate that you need to set either Identity context (-CI) or All context (-CA) and then set the values for the various context fields you need to set. To setting the User Identifier, which is part of Identity context, invoke Q like this:

q -m MQG1 -o Q1 -Au:mqgemusr -CI -M"Test Message"

Using Alternate User ID when opening a queue/topic/sub

An IBM MQ application can MQOPEN a queue or topic, or make an MQSUB using an Alternate User id for the authorization checks.

To open a queue using Alternate User ID, invoke Q like this:

q -m MQG1 -o Q1 -au:mqgemusr

To open a topic using Alternate User ID, invoke Q like this:

q -m MQG1 -To:TOPIC1 -au:mqgemusr

To create (and close after) a durable subscription using Alternate User ID, and wait for messages, invoke Q like this:

q -m MQG1 -SdDo:TOPIC1 -Sn:SUB.TO.TOPIC1 -au:mqgemusr -w60

---

Note: When Q became an MQGem product, we took the opportunity to “tidy up” some of the flags. There is a chapter in the Q User Guide titled “Migrating from the MA01 IBM Support Pac” which details exactly all the changes. This may mean that some of the above examples differ from the way you would have done things with MA01.

---

Read more about the Q product, and download a copy, from the Q Web Page. If you would like to try out MQGem’s Q program before deciding whether to buy then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

Q version 9.2.0 is released

MQGem Software is pleased to announce that a new release of our Q Utility is now available.

This release was created to deliver various display features requested by customers, and after an extended beta period with those individual customers, it is now generally available. The new features are:-

Message Display Formatting

This version of Q now has a new formatting engine based on wide-characters on Windows and Linux. This means that Q has better display support for DBCS, MBCS and variant characters. You should see less dots in your output display now (dots are how non-printable characters are shown).

q -m MQG1 -i Q1 -df

[    7 bytes] Message Content
Z.rich

Prior version of Q

[    7 bytes] Message Content
Zürich

Q V9.2.0

New message summary display

Option -dy will tell Q to output just a one line summary of the messages on the queue.

q -m MQG1 -iSYSTEM.ADMIN.QMGR.EVENT -dy

Connecting ...connected to 'MQG1'.
  1.[ 320 bytes] QMgr Event - Reason(Alias queue type error) QMgr Name:'MQG...
  2.[ 320 bytes] QMgr Event - Reason(Unknown alias base queue) QMgr Name:'M...
  3.[ 320 bytes] QMgr Event - Reason(Unknown alias base queue) QMgr Name:'M...
  4.[ 236 bytes] QMgr Event - Reason(Unknown object name) QMgr Name:'MQG1' ...
No more messages.

New formatted Put Date/Time option in ISO 8601 format

The new Put Time format is displayed based on the -dz and -dZ option which display the time in UTC or local time respectively.

q -m MQG1 -iQ3 -dz

Connecting ...connected to 'MQG1'.
PutDateTime  : 2020-09-07T03:17:45Z
Hello MQ World
No more messages.

UTC time display

q -m MQG1 -iQ3 -dZ

Connecting ...connected to 'MQG1'.
PutDateTime  : 2020-09-07T15:17:45+12:00
Hello MQ World
No more messages.

Local time display

Message limiting

The message limit parameter -L has been enhanced to contain both an optional start and end limit. For example -L3: will display just the 3rd message. -L5:7 will display messages 5,6 and 7. As before -L10 will display the first 10 messages.

XML formatting

By default XML leaf tags will be displayed on a single line. The previous method of displayed them on three lines can be obtained using option -dL.

q -m MQG1 -i PRODUCT.DATA -dfL

<Product>
  <Name>
    MQEV
  </Name>
  <WebpageURL>
    https://www.mqgem.com/mqev.html
  </WebpageURL>
</Product>

Prior release behaviour

q -m MQG1 -i PRODUCT.DATA -df

<Product>
  <Name>MQEV</Name>
  <WebpageURL>https://www.mqgem.com/mqev.html</WebpageURL>
</Product>

Q V9.2.0 default behaviour

PCF Groups

By default Q will now display the boundaries of PCF groups. This can be suppressed using the -dB option.

q -m MQG1 -iSYSTEM.ADMIN.COMMAND.EVENT -df1

[ 544 bytes] Message Content
Command :99 (Command event)
Reason :2413 (PCF Command)
Parameter Id :8001 (Group Command Context)
Parameter Id :3045 (Event User Identifier)
Value :'mqgemusr '
Parameter Id :1011 (Event Origin)
Value :3 [0x'3']
Parameter Id :3047 (Event Queue Manager)
Value :'MQG1 '
Parameter Id :3050 (Event Appl Name)
Value :'MQGem Software MQSCX '
Parameter Id :1021 (Command)
Value :8 [0x'8']
----------------------------------------- PCF group (Group Command Context) --
Parameter Id :8002 (Group Command Data)
Parameter Id :2016 (Q Name)
Value :'Q1 '
Parameter Id :20 (Q Type)
Value :1 [0x'1']
Parameter Id :15 (Max Q Depth)
Value :99999 [0x'1869F']
-------------------------------------------- PCF group (Group Command Data) --
--------------------------------------------------- Command (Command event) --

A command event, trimmed a little for brevity, showing two PCF groups

---

The new version can be downloaded from the Q Download Page. Any current licensed users of Q can run the new version on their existing licence. If you don’t have a licence and would like to try out Q then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.