QLOAD – Display Summary update

A minor update has been made to QLOAD to make some enhancements to the Display Summary output. This view gives you a snapshot of the queue, the sizes of the messages on the queue and the ages of the messages on the queue.

The enhancements added average age and average size to those pre-existing sections, and also a new section that summarises the queue size.

Message Ages

This section has had the average age added to the output. Of course, this is just a snapshot of what is on the queue at the moment when you run the qload command. If you want a broader view of the average age of messages on a queue, queue statistics would offer a better perspective.

Youngest Message : 3 Days 22 Hrs 43 Mins 50 Secs
Oldest   Message : 247 Weeks 2 Days 58 Mins 30 Secs
Average  Message : 133 Weeks 6 Days 7 Hrs 6 Mins 7 Secs

Message Sizes

This section has had the average size added to the output. As with the ages, this is just a point in time statement and queue statistics collection would aid in getting a better history.

Smallest Message : 4 bytes
Biggest  Message : 1.1 KBs
Average  Message : 195 bytes

Queue Size Summary

As a result of a recent customer interaction which I described here, we decided to enhance the QLOAD Display summary output with this additional report. The “size” of your queue varies depending on what you consider.

You might consider the size of your queue to be the number of messages on that queue – after all that’s what causes queue high and queue full events.

You might consider the size of your queue to be the quantity of message data that is stored on that queue. Some will consider that this should include the size of the MQMDs.

Or you might consider the size of your queue to be the amount of disk space used by the queue file.

Queue Size Summary
==================
                     Current     Maximum     Full
                    ---------   ---------  -------
Messages                12000       20000   60.0 %
Message Bytes       67.38 KBs    1.99 TBs    0.0 %
Data Bytes           4.23 MBs    1.99 TBs    0.0 %
Queue File Size         6 MBs    1.99 TBs    0.0 %

This table shows the count of all those possible “queue sizes” and the percentage full of the equivalent maximum size.

On the “Messages” row, the maximum is the maximum depth configured on the queue.

On the other three rows, the maximum is the maximum file size configured on the queue obtained from an Inquire Queue Status command.

If you have a queue manager that does not have this feature, which was added on distributed queue managers in V9.1.5, and is not applicable to z/OS, or your user ID does not have the authority to issue the Inquire Queue Status PCF command, then the final two rows are not shown, and the maximum value of the “Message Bytes” row is Max Depth x Max Message Length.


We hope that these enhancements to the summary display will make it easier to understand the utilisation of your queues.

You can run this command as a User Command in MO71 – see this blog post for more details.


Read more about the QLOAD program, and download a copy, from the QLOAD Download Page. If you don’t have a licence and would like to try out QLOAD then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

QLOAD: Using Non-QLOAD input files

We have just updated QLOAD with a couple of minor enhancements to improve your experience when using non-QLOAD files as input.

The ability to use non-QLOAD files as input to QLOAD has been around for a while, you can read more about it here. We have now enhanced this feature to allow it to operate with indexed file names which were already available for QLOAD-files.

An indexed file name is one with either %n or %N in the file name.

n Current File number
N Current File number padded to 5 characters eg.000001

So imagine you have a set of files, thus:-

msgs1.txt
msgs2.txt
msgs3.txt
msgs4.txt
msgs5.txt
msgs6.txt

each containing one or more messages, delimited say by XML tags. You can now present those files to QLOAD with the following command.

qload -m MQG1 -o Q1 -f msgs%n.txt -Cd -nns:"<msg>" -nne:"</msg>"

QLOAD will read through the files and prior to putting the messages, will report what it found and check that this is what you expected.

Loading 'msgs1.txt' : 4 messages found
Loading 'msgs2.txt' : 2 messages found
Loading 'msgs3.txt' : 2 messages found
Loading 'msgs4.txt' : 2 messages found
Loading 'msgs5.txt' : 3 messages found
Loading 'msgs6.txt' : 3 messages found
Do you want to continue? (y/n)

In addition to allowing multiple non-QLOAD files to be processed in one command, we have also added some over-rides for Message Descriptor (MQMD) attributes.

When you use QLOAD files, the MQMD attributes are all saved away in the file to allow you to reload the messages exactly as they were. If you need to you can edit the MQMD attributes to change these values before reloading.

When you use non-QLOAD files there are no MQMD attributes in the files and you would just get a default MQMD put to the queue with the message data.

With this new enhancement, you can set a number of the MQMD attributes when you put the data from your non-QLOAD file, or indeed over-ride the data in the QLOAD file to save you having to make numerous edits.

Here’s an example of loading up our multiple files and setting the reply-to-queue in the MQMD. This also sets the message type to MQMT_REQUEST as well.

qload -m MQG1 -o Q1 -f msgs%n.txt -Cd -nns:"<msg>" -nne:"</msg>" -Ar:MY.REPLY.Q

If we look at the message descriptor of the message we just put it looks like this:

[  324 bytes] Message Descriptor (MQMD)
Report       :00000000
Message Type :1 (Request msg)
Format       :'MQSTR   ' (String)
Priority     :2 (Low)
Persistence  :0 (Not Persistent)
Message Id   :414D51204D514731202020202020202037C10B650FF80140
              A M Q   M Q G 1                 7 ┴ . e . ° . @
ReplyToQ     :'MY.REPLY.Q                                      '
ReplyToQMgr  :'MQG1                                            '

The full set of MQMD attributes that you can set is as follows:

flag MQMD Attribute
-Ac:<ccsid> Message Codepage
-Ae:<expiry> Message Expiry (in tenths of a second)
-Af:<format> Message Format
-Ap:<priority> Message Priority
-Ar:<ReplyQ> Message Reply Queue and optionally Reply Queue Manager
-Ar:REPLYQ
-Ar:QM1/REPLYQ
-At:<Msg Type> Message Type: REQUEST, REPLY, DATAGRAM, REPORT

We hope this will allow you to use QLOAD to load up your message content that is currently held in files, into queues on your queue manager.


The new version can be downloaded from the QLOAD Download Page. Any current licensed users of QLOAD can run the new version on their existing licence. If you don’t have a licence and would like to try out QLOAD then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

MQGem Tools TLS Configuration

This blog post is going to cover how to set up TLS secured client channels when using one of our command line tools, such as QLOAD, Q or MQSCX, to connect to your queue managers. Before I start, I just want to say up front that there is nothing special about how you do this. Our tools are all ‘C’ language MQI applications and so setting up TLS for them is just the same as setting it up for the IBM supplied ‘C’ language sample amqsputc as an example. If you know how to do that, then you know how to set it up for our command line tools.

If you want similar instructions for our GUI tools MO71 and MQEdit, please see this post.

MO71 TLS Connection to Queue Manager

Certificates

In order to have TLS secured connectivity between your MQGem tool and your queue manager, your queue manager must have a certificate, hopefully not a self-signed certificate; and you, the MQGem tool user, must also have a certificate identifying you. It is likely that this certificate identifying you will be used for connectivity from various different tools, so it does not need to be specific to the tool you are using. This blog post is not going to cover setting up the queue manager certificate as that has been amply covered in IBM Docs. What you need at your machine where you are running an MQGem tool is a signed certificate identifying you, and a copy of the CA certificate that signed the queue manager’s certificate. It is possible that you have several of the latter item depending on how your queue managers are setup. One hopes that you have many less of these than the number of queue managers you need to manage.

Key Database File

MQGem tools are ‘C’ language MQI applications. This means that the IBM MQ Client code expects it to use a KDB file with a matching named stashed file. Unlike Java programs, you do not provide the password to the certificate store (KDB file) anywhere, the IBM MQ Client code finds the stash file that contains the password due to the fact that it has the same name as the KDB file. Here is my set of files:-

 Directory of C:\MQGem

06/07/2021  16:43                88 MQGemClient.crl
06/07/2021  16:43            20,088 MQGemClient.kdb
06/07/2021  16:43                88 MQGemClient.rdb
06/07/2021  16:43               193 MQGemClient.sth
               4 File(s)         20,457 bytes

Later you will see when setting up your certificate configuration that you use the name up to, but not including, the file extension. The IBM MQ Client code can construct the name of the key repository and password stash file by appending the “.kdb” and “.sth” file extensions respectively.

As you follow the instructions in this blog post, you are going to create a number of files, the above KDB and associated ancillary files, a CCDT and an mqclient.ini file. It is useful to first create a directory to keep all these files together.

If you don’t already have a KDB file supplied for your use, you should create one now. Click on the twisty to see the details for each step that you need to do.

Step 1: Create a KDB file

IBM MQ supplies tools for you to create the files you need. In this case we show examples of using the command line tool runmqakm. There is also a GUI tool that can do the same job. Here is the command to create your KDB file. In my example, the KDB file (and accompanying stash file etc) will be created in the C:\MQGem directory, using names MQGemClient.* – you may wish to change this. In addition, you probably want a stronger password too!

runmqakm -keydb -create -db C:\MQGem\MQGemClient.kdb -pw passw0rd -type cms -stash

If you want, you can use the runmqakm tool to create a password for you:-

runmqakm -random -create -length 16 -strong

The KDB file, once created, is the place to put the various certificates you need. As mentioned above, you’ll need to have the CA certificate that signed your queue manager(s) certificates. This is so that the certificates coming from the queue managers at connection time can be validated by the IBM MQ Client as part of the TLS handshake.

If you have one or more files containing the CA certs to allow you to validate connections to your queue manager(s), add them to your KDB now. If you only have a single certificate file (a PKCS#12 file) that also contains the CA certs, skip Step 2.

Step 2: Add CA certificates to KDB

Here is the command to add a CA certificate to your KDB. As before, you are likely using a different location in your file system for the KDB file. Note that I am using the -stashed parameter from now on, so I never have to type in the password again. The label you use when you add a CA certificate doesn’t matter. Just make it something that is helpful in remembering what this certificate is. In my example MQGemProdCA is the label I have used for the production environment CA certificate.

runmqakm -cert -add -db C:\MQGem\MQGemClient.kdb -stashed -label MQGemProdCA -file MQGemProdCA.arm -format ascii -trust enable

Finally you need to have a certificate identifying you, the user of an MQGem tool. You may have been given a file already containing your signed certificate, and have to add that to your KDB, or you may need to generate a certificate request, get it signed by your company CA, and then when you get it back, receive it into the same KDB where you created the request. Choose either Step 3a or 3b below depending on how your company does this.

DO THIS: Step 3a: Add PKCS#12 private certificate into KDB

This special certificate file type is one that contains both your certificate and the private key for the certificate. It is very important to keep the private key safe, so when transporting this around in a PKCS#12 file, this particular type of file has a password. Other files that only contain certificates without private keys don’t have passwords.

Here is the command to add a PKCS#12 certificate file to your KDB. This command will also add the CA certificates that are in the file. Note that I am using the -stashed parameter for the KDB password, so I never have to type in the password again. The password on this command is the one that goes with the PKCS#12 file you have been given.

runmqakm -cert -import -file MoragCertificate.p12 -type pkcs12 -pw pkcs12passw0rd -target c:\MQGem\MQGemClient.kdb -stashed
OR THIS: Step 3b: Create a Certificate Request

To make a certificate request, issue the following command, using the KDB file that you just created.

runmqakm -certreq -create -db C:\MQGem\MQGemClient.kdb -stashed -label MQAdminProd -dn "CN=Morag Hughson,OU=Prod,O=MQGem Software" -file Clnt_Prod_Request.csr

Now send the certificate request file off to the CA that signs certificates for your environment. When you receive the signed certificate file back from them, receive it into the same KDB as before with the following command.

runmqakm -cert -receive -db C:\MQGem\MQGemClient.kdb -stashed -file Clnt_Prod_Signed.arm

As a final check that everything got added ok, list the contents of your KDB with the following command.

runmqakm -cert -list all -db C:\MQGem\MQGemClient.kdb -stashed

You should see output something like this. There should be at least one trusted certificate (shown with a !) and at least one personal certificate (shown with a -). You will need the label of the personal certificate when you come to set up your certificate configuration in a moment, so remember that.

5724-H72 (C) Copyright IBM Corp. 1994, 2020.
Certificates found
* default, - personal, ! trusted, # secret key
!       MQGemProdCA
-       MQAdminProd

Client-connection Channel

MQGem command line tools generally rely on the use of a Client Channel Definition Table (CCDT) to provide the client-connection channel definition details, rather than having lots of command line parameters to cover all the possible attributes. Use of a CCDT also makes it simpler to use the same channel definition details across a range of tools instead of having to specify the attributes individually for use with each tool. It is therefore a highly recommended method. If you don’t already have a CCDT with your client-connection channel details you should create one now.

Step 4: Create TLS-secured CLNTCONN channel in CCDT
In modern versions of IBM MQ, the CCDT can be created on the client machine using either runmqsc or our MQSCX tool, starting the tool with the -n flag to indicate you wish to create a CCDT.

Both runmqsc and MQSCX will pay attention to the MQCHLLIB and MQCHLTAB environment variables to determine the location where the CCDT file will be written. Additionally, MQSCX has a -t flag that you can use to specify the full path and file name you want instead.

If you want to use a JSON CCDT, MQSCX can be used to create one of those by using the -t flag. See Working with JSON CCDT files blog post for more details.

Once your MQSC tool of choice is running in CCDT mode, create the channel with a command like the following.

Your company may choose to use a different CipherSpec (the SSLCIPH) parameter. I have chosen to use the more recently introduced ANY_TLS12. If you can’t use one of the “ANY” values then you should ensure that the same value is used here, and in the SVRCONN channel (next step).

DEFINE CHANNEL(MQGEM.SVRCONN.TLS) CHLTYPE(CLNTCONN) TRPTYPE(TCP) CONNAME('GEMWIN3.mqgem.com(1701)') SSLCIPH(ANY_TLS12) QMNAME(MQG1)

Server-connection Channel

You’re also going to need to create a TLS-secured server-connection (SVRCONN) channel on the queue manager if you don’t already have one in place.

Step 5: Create TLS-secured SVRCONN channel
Here’s an example MQSC command to define such a channel. The name you use for your channel may well be different. Whatever it is, remember it for the next step.

Your company may choose to use a different CipherSpec (the SSLCIPH) parameter. I have chosen to use the more recently introduced ANY_TLS12. If you can’t use one of the “ANY” values then you should ensure that the same value is used here, and in the CLNTCONN channel (previous step).

DEFINE CHANNEL(MQGEM.SVRCONN.TLS) CHLTYPE(SVRCONN) TRPTYPE(TCP) SSLCIPH(ANY_TLS12) SSLCAUTH(REQUIRED)

mqclient.ini configuration

When setting things up outside of the MQGem tool itself, you can either use environment variables or the mqclient.ini file to point to all the various settings. We’ll be showing you the latter today. If you’re not familiar with why the mqclient.ini file is our recommendation, you might find this blog post interesting.

Step 6: Create mqclient.ini file
In your working directory where you have your KDB and CCDT files, create a text file with the name mqclient.ini. Edit this file to add the following contents.

Note the ‘.’ to mean “this directory” in the value for the ChannelDefinitionDirectory. Alternatively you can fully spell out your directory.

CHANNELS:
  ChannelDefinitionDirectory=.
  ChannelDefinitionFile=MQGEM.TAB
SSL:
  SSLKeyRepository=C:\MQGem\MQGemClient
  CertificateLabel=MQAdminProd

Test Connectivity

Now that you have your certificates set up, your client channel definition created, and your mqclient.ini file pulling all the information together, you can run your command line MQGem tool to pick up and use all these things you have configured.

The easiest way to do so is to ensure you are in the directory containing the mqclient.ini file before you run the command invocation. The IBM MQ client will pick up the mqclient.ini that is in the current directory by preference. However, if you want to do something different, this page shows all the different mechanisms.

Step 7: Run Q using TLS connection
From in your directory containing the mqclient.ini file, invoke Q with a command like the following.

The -l mqic flag tells Q to make a client connection rather than a locally bound connection, and the rest of the invocation is up to you. In our example we are just opening queue Q1 for output.

q -m MQG1 -l mqic -o Q1
Step 7: Run QLOAD using TLS connection
From in your directory containing the mqclient.ini file, invoke QLOAD with a command like the following.

The -l mqic flag tells QLOAD to make a client connection rather than a locally bound connection, and the rest of the invocation is up to you. In our example we are just browsing queue Q1 to a file.

qload -m MQG1 -l mqic -i Q1 -f q1.qld
Step 7: Run MQSCX using TLS connection
From in your directory containing the mqclient.ini file, invoke QLOAD with a command like the following.

The -l flag tells MQSCX to make a client connection rather than a locally bound connection.

mqscx -m MQG1 -l

If you receive an error when trying to connect, remember to use the client error log, found in <MQ_DATA_PATH>/errors/AMQERR01.LOG. Scroll to the bottom for the most recent error.

Summary

It is hoped that this post will help you set up TLS-secured connectivity between MQGem command line tools and your queue manager. Remember that the same KDB file and certificates will also work with the IBM-supplied samples such as amqsputc. If you have any issues, it may be helpful to try things out with that very simple sample first. It is also good advice if you get validation problems to change the SVRCONN channel to use SSLCAUTH(OPTIONAL) and get it working that way first. Once it does then go back to SSLCAUTH(REQUIRED) knowing that the problem is that you are either not sending, or the queue manager is not validating, the client side certificate.

If you have IBM MQ V9.3.0 (or later) as your client installation you can also now make use of a new feature in that release which allows you to avoid having a stash file. See this post for more details.

If you have situations where you are trying to set up TLS-secured connectivity for MQGem tools that are not covered in this post, please do get in touch, either in a comment below, or through any of the usual channels (see About page).

File names in JCL

As a result of reading Lyn Elkins’ posts about how she struggled with the quotation marks required to specify a file name as a parameter in a JCL PARM string, I thought I would write a little summary. I personally prefer to use DD cards for my file names whenever possible as these avoid the need for quotation marks and leave your eye less likely to be squinting by the time you are done. I will show the fully qualified example right at the end for anyone who really wants it though. I am using QLOAD as an example program here, but any application that simply does an fopen() can make use of the same syntax. For example, when supplying a script in a file using MQSCX, providing an input file to load onto a queue in QLOAD, or some string data to become message data in the Q program. Any parameter that expects a file name can also instead take a string of the format “DD:card-name” (note that you MUST use capital letter D’s here). All of these use the same syntax as illustrated here.

//QLOAD    JOB
//*********************************************************************
//* QLOAD Load messages onto Q1
//*********************************************************************
//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f DD:INFILE1')
//STEPLIB  DD DSN=GEMUSER.USER.LOAD,DISP=SHR                         
//         DD DSN=IBM.MQ.SCSQAUTH,DISP=SHR                           
//         DD DSN=IBM.MQ.SCSQANLE,DISP=SHR                           
//SYSPRINT DD SYSOUT=*  
//SYSOUT   DD SYSOUT=*      
//INFILE1  DD DSN=GEMUSER.DATA.MSGFILES(EXMPL1),DISP=SHR
//INFILE2  DD DSN=GEMUSER.DATA.OUTPUTQ1,DISP=SHR 
//INFILE3  DD PATH='/u/gemuser/Q1.qld',PATHOPTS=(ORDONLY)          
//MQGEML   DD DSN=GEMUSER.MQGEM.LIC,DISP=SHR
//

Here you can see that the JCL PARM string does not need any extra quotes at all, just the string DD:INFILE1 where INFILE1 is the name of a DD card in the JCL job. The DD card can then specify a fully qualified PDS member, MVS file, or HFS file as the three examples show.

To do the same job as shown in these three examples by providing the file name inline in the JCL PARM string, things start getting complicated by the fact that extra quotation makes are needed to escape the quotes since the whole JCL PARM string is surrounded by single quotes itself. This is what makes it so problematic, and why I always use DD cards instead.

If you take advantage of that the fact that MVS knows your TSO login and will pre-pend it to the MVS file name you supply, you can avoid quite a number of the quotation marks needed. The following two EXEC statements are equivalent when the TSO login is GEMUSER. The way you tell MVS not to pre-pend your TSO login is to surround the name you supply with single quotes, which turn into doubled up single quotes when used inside a JCL PARM string! The whole thing also needs to be enclosed in double quotes because of the slashes which are considered a delimiter to separate a JCL PARM string into run-time options and parameters to pass to the program.

//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f"//DATA.MSGFILES(EXMPL1)"')
//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f"//''GEMUSER.DATA.MSGFILES(EXMPL1)''"')

The two slashes shown in both the above example JCL PARM strings are how you tell fopen() that you are referencing an MVS file (to distinguish it from an HFS file). Here is an example EXEC statement using an HFS file instead. Again, this string needs to be surrounded by double quotes because of the slashes. The second example has no slashes, and will end up in your current directory. In fact any string that does not start with DD: or a slash (or two) will be interpreted as an HFS file in your current directory, and is why you might end up with an HFS file called “dd:SUMMARY” if you forgot to use capital letter D’s.

//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -f"/u/gemuser/Q1.qld"')
//LOADTOQ  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o Q1 -fQ1.qld')

Well, I hope that helps, and also probably convinces you that DD cards are the way to go.

I didn’t invent this stuff, I’ve just spent a lot of time using it. Here is some reference material that helped me get to grips with it. Perhaps it can help you too.

On a separate note, if you ever have a long JCL PARM string, don’t struggle with line continuations, move over to using PARMDD – see JCL with long PARM strings

Loading files to multiple queues

If you have unloaded MQ message data in a file that you need to load onto a queue, you can use QLOAD with an invocation something like this:

qload -m MQG1 -o Q1 -f msgtext.qld

If you have files containing message data that wasn’t written by QLOAD, you can still load it up again with QLOAD. See QLOAD: Using delimited files. The mechanisms in this blog post can also be combined with those types of message data files.

Since QLOAD V8.0.2 you have also been able to load a file up onto multiple queues like this:

qload -m MQG1 -o Q1 -o Q2 -o Q3 -o Q4 -f msgtext.qld

In other words you can repeat the -o parameter as often as required. This operation will make use of distribution lists in IBM MQ unless you force the use of individual MQOPENs with the -OI flag. When running QLOAD on z/OS, this latter mechanism is always used.

The queue names specified on the -o parameters can be fully qualified destinations, for example MQG2/Q1, that is Q1 on queue manager MQG2.

If you have quite a lot of queues and you don’t want to specify lots of -o parameters, you can instead put the list of queue names into a destination file and tell QLOAD to use that, with an invocation something like this:

qload -m MQG1 -o file:mydestinations.dst -f msgtext.qld

Your destination file might look something like this, illustrating comment lines and the fact that you can also include other destination files:

* This file contains queue destinations used by QLOAD
MQG1/Q1
MQG2/Q1
MQG3/Q1

* You can even load other files
file:moredestinations.dst

mydestinations.dst

If you’re running QLOAD on z/OS, you could make use of an inline DD card to keep all the listed queue names in a single piece of JCL, or you might reference another file if you are using the same list of queue names over and over.

//QLOAD    JOB
//*********************************************************************
//* QLOAD Load to many queues
//*********************************************************************
//PUTMANY  EXEC PGM=QLOAD,
//         PARM=('-m MQG1 -o "file:DD:QLIST1" -f DD:MSGTEXT')
//STEPLIB  DD DSN=GEMUSER.USER.LOAD,DISP=SHR                         
//         DD DSN=IBM.MQ.SCSQAUTH,DISP=SHR                           
//         DD DSN=IBM.MQ.SCSQANLE,DISP=SHR                           
//SYSPRINT DD SYSOUT=*  
//SYSOUT   DD SYSOUT=*      
//MSGTEXT  DD DSN=GEMUSER.MSGFILES(EXMPL1),DISP=SHR
//MQGEML   DD DSN=GEMUSER.MQGEM.LIC,DISP=SHR
//QLIST1   DD *
Q1
Q2
Q3
Q4
/*
//

This is not a new feature of QLOAD – it has been there since January 2016, but I was prompted to write about it because of a question from someone who didn’t know about the destination files.


Read more about the QLOAD program, and download a copy, from the QLOAD Download Page. If you don’t have a licence and would like to try out QLOAD then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

Is there a way to clear an MQ queue of all messages?

We were recently asked how to clear a queue of all messages within one of our tools.

In this post we cover how to do this using MQGem tools as several of them offer this capability. Open up the twisty for each tool to read how to do this.

Using MO71

There are two options for clearing a queue of all messages when using MO71. One directly interacts with the messages in a Message List dialog, and the other uses the Clear Queue command via the command server. The latter can fail if someone else has the queue open.

MO71: Using the Message List dialog

In MO71 you can browse a list of messages on a queue, then in the context menu of that dialog choose Message Selection → Apply to all Messages. Then press the Delete All button, or select Delete All from the right-mouse button context menu and all the messages will be deleted.

Delete all messages using MO71

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the message(s) to a separate holding queue, just in case the message proves to be important after all. MO71 makes it simple to Move messages too. Just fill in the holding queue name, and then press the Move All, rather than Delete All, button.

MO71: Using the Clear Queue command

From a Queue List dialog in MO71, you can choose to clear a queue from the context menu. As a destructive command, this will show a confirmation dialog before it goes ahead.

Delete all messages using the Clear command in MO71

Remember that this command can fail if the queue is open by another application at the time.

Using the Q program

You can use the Q program to quickly destructively get off all the messages on a queue, to ensure it is empty before you begin using it for something else. This can work even when the queue is in use (unless it is exclusively in use of course) and the CLEAR QLOCAL command can’t be used.

Here is an example which destructively gets (-I) from Q1 and uses a zero-length buffer and accepts that the messages will be truncated (-=t0):-

q -m MQG1 -I Q1 -=t0

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the messages to a separate holding queue, just in case they prove to be important after all. This is simple to achieve using a command like the following:

q -m MQG1 -I Q1 -o HOLDING.Q
Using QLOAD

You can use QLOAD to quickly destructively get off all the messages on a queue, to ensure it is empty before you begin using it for something else. This can work even when the queue is in use (unless it is exclusively in use of course) and the CLEAR QLOCAL command can’t be used.

Here is an example which destructively gets (-I) from Q1 and discards them (-f null), aka sends them to the null file destination:-

qload -m MQG1 -I Q1 -f null

Take care when deleting messages from a production queue manager. We would recommend using QLOAD to actually move the messages to an output file, in case you change your mind about deleting them!

qload -m MQG1 -I Q1 -f c:\temp\Q1deletedmsgs.qld 
Using MQEdit

In MQEdit you can browse a list of messages on a queue, then in the context menu of that dialog choose Message Operations → Apply to all Messages. Then press the Delete All button, or select Delete All from the right-mouse button context menu and all the messages will be deleted.

Delete all messages using MQEdit

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the message(s) to a separate holding queue, just in case the message proves to be important after all. MQEdit makes it simple to Move messages too. Just fill in the holding queue name, and then press the Move All, rather than Delete All, button, or simply Drag and Drop the messages onto the new queue.

Using MQSCX

You can of course issue the MQSC CLEAR QLOCAL(q-name) command from MQSCX. However, it can go one better than this. The CLEAR QLOCAL command is problematic because it can fail when someone else has the queue open, even if it is not open in a way that stops you getting messages. Instead you can issue the following command in MQSCX:-

=clear qlocal(Q1)

then MQSCX will first attempt the CLEAR QLOCAL command and if that fails with "object in use" then it will switch to getting the messages off the queue instead. You can read more about this in MQSCX: Clearly a better way.


If you don't have any of these tools, but would like to try any or all of them out, please contact support@mqgem.com and a 1-month trial licence will be sent to you with no obligation to buy. You can download the tools from our website.

Loading Messages with the correct context

One of the benefits of using QLOAD to unload and then reload your messages is that the full MQMD is also saved and restored, so you don’t lose the context fields of your messages. They still show the original putting application and user ID just as before they were unloaded, and the put date and time fields stay the same as well.

What are the context fields?

In case you don’t remember, here’s a quick summary of the MQMD Context fields. There are two sets of context fields, the Identity Context and the Origin Context. It is possible to manipulate just the Identity Context fields or all the fields as we’ll see in a moment.

Identity Context Fields Origin Context Fields
UserIdentifier PutApplType
AccountingToken PutApplName
ApplIdentityData PutDate
  PutTime
  ApplOriginData

If you want a quick refresher on how authorisations work with these fields, check out this Morag’s Quirks blog post.

Manipulating Context fields

By default QLOAD will do the puts when you load messages onto a queue using “Set All Context” which means all the fields that are saved in the unloaded file are put back in the newly loaded up messages.

If you prefer to upload the messages and record new context fields to show that they are from you, then you can tell QLOAD you use a different context option. The following command will set all the context fields to identify QLOAD as the application that put the messages, and the user that is running QLOAD as the User Identifier. This is known as “Default Context” and is indicated using the -Cd flag (Context – default).

qload -m MQG1 -o Q1 -f Q1-messages.qld -Cd

If you want to just keep the identity context fields as they were in the unload file, and have the origin context fields reflect what’s happening now you can use the “Set Identity Context” option, -CI flag (Context – Identity).

qload -m MQG1 -o Q1 -f Q1-messages.qld -CI

Another option that can be very useful is to have all these MQMD fields be exactly as they were when unloaded with the exception of the Put Date and Time fields. This isn’t an MQ option, it’s just something that QLOAD offers you. It uses “Set All Context” and grabs a new timestamp for your Put Date and Time fields just before it puts the message to the queue. You can do this by using the -Ct flag (Context – time).

qload -m MQG1 -o Q1 -f Q1-messages.qld -Ct

This particular option can be very useful if you save off message data to replay in testing scenarios. Everything else stays the same, but the messages reflect the fact that they have been put right now.

There are a number of other context options for when you are moving messages not from a file, but from a queue. These can be found in the QLOAD User Guide, or by reading the QLOAD program help.

qload -C?

QLOAD can be downloaded from our webite. If you don’t have a licence and would like to try out QLOAD then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

Finding the reason why a message was sent to the Dead Letter Queue (DLQ) in IBM MQ

IBM Support recently released a Support Doc which was titled:-

Summary

Finding the reason why a message was sent to the Dead Letter Queue (DLQ) in IBM MQ

The document described several options, which often required you to look at the message in hex and, when required, byte swap the value to find the reason code. In this post we cover how to do this using MQGem tools as several of them offer this capability, and our tools format out the Dead Letter Header so you don’t need to do any manual byte swapping. Open up the twisty for each tool to read how to do this.

Using MO71

In MO71 you can browse the messages on the Dead Letter queue using a Message List. If you choose to show the “Message Summary” column on your list dialog (see Configuring List Dialogs in MO71 for how to change the columns in a list dialog), you will immediately be able to see the reason code in the Dead-Letter header, as that is one of the useful things that Message Summary displays look for.

A message List dialog will summarise the Dead Letter header of a message for you

Alternatively you can double click on a message to look at the full details of an individual message, and you will see the full Dead-Letter header (MQDLH) is formatted out for you to see.

The full message display shows all of the Dead Letter Header formatted out

Using the Q program

To view your messages on the Dead Letter queue, and have them formatted out, you need to use the -df option which also has three levels of detail. For full detail use -df3. Here’s an example command:-

q -m MQG1 -i MQG1.DEAD.LETTER.QUEUE -df3

Which will format the Dead Letter header like this making it really easy to understand the reason why the message was placed on the DLQ.

[  181 bytes] Message Content
[  181 bytes] Dead Letter Queue Header (MQDLH)
StrucId      :'DLH '
Version      :1
Reason       :2085 (Unknown object name)
Dest. Queue  :'TARGET.QUEUE                                    '
Dest. QMgr   :'MQG1                                            '
MQEncoding   :0x'222' (Reversed)
CCSID        :437 (IBM PC)
Format       :'MQSTR   ' (String)
PutApplType  :11 (Windows NT)
PutApplName  :':\mqm9250\bin64\amqrmppa.exe'
Put Date     :'20220411'
Put Time     :'05021009'
Some text
Using MQEdit

When you look at the messages on a queue using MQEdit, the message list will show you a summary of the message in the “Message” column, which for Dead Letter header messages will show you the reason from the header. This makes it very easy to see the reasons why messages are on your Dead Letter queue.

Selecting a message from the list will display it in the pane below with the Dead Letter header formatted out also showing you the reason why the message ended up on the DLQ.

MQEdit shows a summary of the Dead Letter header in the message list, and the fully formatted message in the lower panel

Using QLOAD (honourable mention)

QLOAD isn’t a message formatting tool, so it will not help you to view the reason code in the Dead Letter header of a message. However, it does get an honourable mention in this list because it can help you to move, or unload messages from your Dead Letter queue that have a specific reason code in the Dead Letter header. This can be very helpful if you want to siphon off some specific errors to handle in a different way.

Use a command like this to copy (-i is browse) all the DLQ messages for MQRC_UNKNOWN_OBJECT_NAME (2085) to a file.

qload -m MQG1 -i MQG1.DEAD.LETTER.QUEUE -Gr:2085 -fUnknownObjs.qld

When you’re using this feature, QLOAD will report the message selection that you have active:-

Message selection active:
  Messages with DLQ Reason 2085 Unknown object name

Read    - Files:   0  Messages:     3  Bytes:      5616
Written - Files:   1  Messages:     1  Bytes:       181

If you don’t have any of these tools, but would like to try any or all of them out, please contact support@mqgem.com and a 1-month trial licence will be sent to you with no obligation to buy. You can download the tools from our website.

QLOAD Summary display

When you have a few messages on a queue, and you’re not sure why, one of the questions you may ask yourself is how old these messages are.

The IBM MQ DISPLAY QSTATUS command can tell you how old the oldest message on the queue is when you look at the MSGAGE attribute, but that doesn’t tell you about the other not quite so old messages. You can also see the depth of the queue using MQ administrative commands, but you don’t get any information about the size of the messages.

The QLOAD Summary display can give you a snapshot of the various messages on your queue. Invoke QLOAD with a command like the following:-

qload -m MQG1 -i<queue-name> -ds

to see a report something like this:-

Message Ages
============

< 2  secs < 10 secs < 1   min < 1  hour < 1   day < 1  week
--------- --------- --------- --------- --------- ---------
        0         0         0         0         0         2

< 2 weeks < 4 weeks < 3 month < 6 month < 1  year > 1  year
--------- --------- --------- --------- --------- ---------
        2         2        13       110         0         0

Youngest Message : 4 Days 2 Hrs 29 Mins 28 Secs

Oldest   Message : 19 Weeks 3 Days 3 Hrs 26 Mins 7 Secs

Message Sizes
=============

< 100b <  1Kb <  4Kb <100Kb <  1Mb < 10Mb < 50Mb > 50Mb
------ ------ ------ ------ ------ ------ ------ ------
     1    128      0      0      0      0      0      0

Smallest Message : 4 bytes

Biggest  Message : 368 bytes

This is not a new feature of QLOAD – it had been there a very long time, and was enhanced in the V9.1.1 which was released in August 2019, but I was prompted to write about it because of a question from someone who didn’t know about this report.


Read more about the QLOAD program, and download a copy, from the QLOAD Download Page. If you don’t have a licence and would like to try out QLOAD then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

Is there a way to delete a single message from an MQ queue?

IBM Support recently released a Support Doc which posed the question:-

Question

Is there a way to delete a single message from an MQ queue (arbitrary position in the queue)?

The document described a couple of options. In this post we cover how to do this using MQGem tools as several of them offer this capability. Open up the twisty for each tool to read how to do this.

Using MO71

In MO71 you can browse a list of messages on a queue, then from that list, select an individual message (or multiple messages if you need to). Then press the Delete button, or select Delete from the right-mouse button context menu and the selected message(s) will be deleted.

MO71 uses the MQ facility, get by message token, so only the exact message(s) that you select from the list will be deleted. While the position of the message on the queue is shown, this is not used for deletion and neither is Message Id, since there is no guarantee that this will be unique either. Each message on a queue is guaranteed to have a unique Message Token.

Alternatively, you can double click on a message from the list, to view the whole message individually. Then from that display, having checked all the details, and made sure it is the message you want to delete, you can press the Delete button, or select Delete from the context menu.

Take care when deleting messages from a production queue manager. Consider that it might be safer to Move the message(s) to a separate holding queue, just in case the message proves to be important after all. MO71 makes it simple to Move messages too. Just fill in the holding queue name, and then press the Move, rather than Delete, button.

Using the Q program

To delete a single message from an MQ queue using the Q program, you should first discover the message ID of the message you wish to delete, to ensure you are addressing the correct message that you intend to delete. Do this by displaying the messages on the queue like this:-

q -m MQG1 -i Q1 -dd

This will browse (-i) the messages on the queue and display the MQMD after the MQGET (-dd) allowing you to see the Message ID of each message on the queue. You will see output like this for each message:-

==============================================================
----- MQMD after MQGET -----
[  324 bytes] Message Descriptor (MQMD)
Report       :00000000
Message Type :8 (Datagram)
Format       :'MQSTR   ' (String)
Priority     :0 (Lowest)
Persistence  :0 (Not Persistent)
Message Id   :414D51204D514731202020202020202099FA4A6020B74B02
              A M Q   M Q G 1                 . ú J `   · K .
ReplyToQ     :'                                                '
ReplyToQMgr  :'MQG1                                            '

Now you can copy that message ID and use it on another invocation of the Q program to delete that specific message from the queue.

q -m MQG1 -IQ1 -gxm:414D51204D514731202020202020202099FA4A6020B74B02

Bear in mind that message IDs are likely to be unique, but this is not guaranteed. It would be safer to move the messages to a separate queue, before deletion. This is simple to achieve using a command like the following:

q -m MQG1 -IQ1 -o HOLDING.Q 
-gxm:414D51204D514731202020202020202099FA4A6020B74B02
Using QLOAD

If you have already browsed the queue contents and output them to a file, you can find the message Id of the specific message you are attempting to address by looking for the Message Descriptor attribute “MSI”. Here is a snippet from a QLOAD file to show an example.

A FMT MQSTR
A PRI 0
A PER 0
A MSI 414D51204D514731202020202020202099FA4A6020B74B02
A COI 000000000000000000000000000000000000000000000000

Now you can copy that message ID and use it on another invocation of the QLOAD program to delete that specific message from the queue. We would recommend using QLOAD to actually move that message to an output file, in case you change your mind about deleting it!

qload -m MQG1 -IQ1 -f c:\temp\deletedmsg.qld 
-gxm414D51204D514731202020202020202099FA4A6020B74B02

Alternatively, you can use the very flexible search and filtering capabilities of QLOAD to narrow down your search of messages on the queue until you find the one you wish to delete, and then offload it to a file, or delete it entirely.

For example, this invocation removes all messages on a queue that are older than one week, and moves them instead to the named file:-

qload -m MQG1 -I Q1 -T7:0:0 -f c:\temp\oldmsgs.qld

Another example, here we remove messages larger than 1MB from a queue:-

qload -m MQG1 -I Q1 -z 1M -f c:\temp\bigmsgs.qld

A third example, here we remove message containing the string “Acme Limited”:-

qload -m MQG1 -I Q1 -s "Acme Limited" -f c:\temp\acmemsgs.qld
Using MQEdit

In MQEdit you can browse a list of messages on a queue, then from that list, select an individual message (or multiple messages if you need to). Then press the Delete button, or select Delete from the right-mouse button context menu.

MQEdit uses the MQ facility, get by message token, so only the exact message that you select from the list will be deleted. While the position of the message on the queue is shown, this is not used for deletion and neither is Message Id, since there is no guarantee that this will be unique either. Each message on a queue is guaranteed to have a unique Message Token.

Alternatively, you can double click on a message from the list, to view the whole message individually in the pane below, and then from that pane, having checked all the details, and made sure it is the message you want to delete, you can select “Delete Message” from the context menu.

Particularly for production queue managers, we would always suggest that it is safer to move message(s) to a holding queue rather than deleting them immediately. In MQEdit it is very simple to move a message between queues, all you need do is drag and drop the required message(s) from one queue to another. You can even copy and move messages between queue managers this way.


If you don’t have any of these tools, but would like to try any or all of them out, please contact support@mqgem.com and a 1-month trial licence will be sent to you with no obligation to buy. You can download the tools from our website.