Announcing Q from MQGem

SupportPac MA01 was written by Paul Clarke back in June 1995. It was one of the first SupportPacs for MQ as you can tell by its number. Paul wrote it because the amqsput, amqsget and amqsbcg samples didn’t give him the flexibility he wanted. It became a very handy tool in the pocket of many an MQ administrator over the years.

When Paul left IBM to form MQGem Software, the MA01 SupportPac was discontinued, and it’s source code put on GitHub for anyone who wanted to build it and run it themselves. However, many MQ users don’t want the complication and expense of looking after home-built tools, and would prefer a fully supported product. As such, a few of our customers have asked whether we would be willing to continue to maintain and enhance the Q program, and so here it is.

Q program pocket knife

A veritable pocket knife of features

The Q program from MQGem has moved on a little from the last version that was the MA01 SupportPac, after all we’ve continued to use it as a very helpful tool here inside MQGem Software. Some of the new features are:-

  • IBM MQ Multi-version Support
    Q will load the MQ libraries from the place identified by setmqenv.
  • New help features
    To make it easier to find the option you are looking for since the Q program does have a lot of flags!.
  • Message formatters added
    Formatters for JSON, EDIFACT, CSV and FIX message formats have been added.
  • Character substitution
    This will display text such as > as the character ‘>’.
  • 64-bit executable
    The Q program is now 64-bit across all platforms.
  • Minor flag enhancements
    1. The special message format string which starts with the ‘#’ character previously had a 40 character limit. This has been lifted.
    2. MQRO_PASS_CORREL_ID has been added to the confirm options on the -n flag as -n[passc].
    3. You can use -dt to print out the offsets of a message
    4. The prompt menu used by -xc to set various client connection channel settings such as TLS and exits has been updated to only request valid exits for a client channel, and to use a more modern CipherSpec by default.
    5. The use of truncation on an MQGET has been made safer by requiring the user explicitly select MQGMO_ACCEPT_TRUNCATED_MESSAGE when using the -=> flag by using the optional ‘t‘ flag, to give -=t<length>.
    6. The Commit interval (-p) flag has been extended to also take an optional commit interval after which an incomplete transaction, that is one that has not reached the requested total number of messages, will be committed anyway.
    7. The subscribe call created with the -S flag can be additionally configured to use MQSO_SET_CORREL_ID by using the -gc:CorrelId flag.
  • Flags renamed
    Some minor flags have been renamed to improve usability and consistency. These are listed in full in the User Guide. The majority of the flags are exactly the same as they were in the Support Pac.

Even if you don’t intend to purchase a licence, you may find value in downloading the user guide. At long last we have a manual for Q! As we said above, the parameters are not absolutely identical to the SupportPac, but the majority are the same, and the vast majority of the manual is still applicable to the SupportPac. You may well discover features you never knew that Q had.


Read more about the new product, and download a copy, from the Q Web Page. If you would like to try out MQGem’s Q program before deciding whether to buy then send an email to support@mqgem.com and a 1-month trial licence will be sent to you.

Advertisements

Behaviour changes in MQ V9.0.4 – CONNAUTH/CHLAUTH

UserID and PasswordIBM recently released it’s latest Continuous Delivery release (MQ V9.0.4). This has made some changes to the default behaviours for CONNAUTH and CHLAUTH. You can read all the new changes in V9.0.4 here, but I wanted to highlight a few I thought were very important.

Adopt Context is YES by default

From the introduction of Connection Authentication in IBM MQ V8, the default value of ADOPTCTX was NO. I am delighted to see that the default has now been changed to YES. This is probably the most common configuration problem we help customers with around the use of Connection Authentication. It’ll take a while to percolate through, because there are plenty of existing queue managers out there with ADOPTCTX(NO) but it will definitely improve things.

qm.ini ChlauthEarlyAdopt=Y is now the default

The qm.ini ChlauthEarlyAdopt attribute was added in IBM MQ V8 FixPack 5 to allow users to revert the behaviour back to the way it was prior to another change that was made – i.e. back to the designed IBM MQ V8 GA behaviour. I am very happy to see that IBM has now reverted this behaviour to be there by default for everyone.

Java clients use user ID and password in the same way as ‘C’ clients

Due to the historical use by the Java client of the FAP flow to send a user ID and password (as described in this blog post) a compatibility setting had to be provided at MQ V8 GA in case any Java Client connections into queue managers were relying on this behaviour for their security exits. This meant that Java clients and ‘C’ Clients operated differently by default. Now, as of V9.0.4, the Java client uses the MQCSP to send its user ID and password just as the ‘C’ client does and they both work the same way. This is very good news.

Creating a CCDT for any version

You may have read an earlier post where we described being able to determine what version of CCDT you had in your hand.

CCDT Version

How often have you had a CCDT file in your hand and wondered what version it was and whether you can give it to some of your known back-level client machines to use.

MQSCX can help you determine this. Open up your CCDT using the mqscx -n mode and then you can quite simply display the version number of all your client channels therein.

What you may not have realised from that post however, was that not only can MQSCX help to investigate what version number your CCDT is made for, it can also make a CCDT for the correct version as well. If you have back-level clients, it can be a real pain having to keep a queue manager of the same level around just to be able to create a CCDT that it will understand. Well, you can ditch that queue manager and use MQSCX instead. It’s really easy to do as well.

To use MQSCX to work with a CCDT, you need to use the -n parameter. This will then look for the CCDT file in the location specified by the MQCHLLIB and MQCHLTAB environment variables unless you provide the -t parameter to give it a specific file name. If one doesn’t exist, it will make a new one for you, and if one does exist it will read it and allow you to update it. In order to control the version of CCDT you are creating, you should additionally use the -V parameter which allows you to specify the version the CCDT file should be written as.

Here’s an example, run the MQSCX program like this:

mqscx -n -t C:\MQGem\CCDT\MQGEM.TAB -V7.0

And then you can use it to make DISPLAY, ALTER and DELETE commands.

MQSCX Extended MQSC Program – Version 8.0.0

CCDT commands directed to file ‘C:\MQGem\CCDT\MQGEM.TAB’

Licenced to Paul Clarke

Licence Location: Head Office

[12:02:10] DISPLAY CHANNEL(*) CONNAME VERSION

_________________________________________________

CHANNEL(MQG1.SVRCONN) CHLTYPE(CLNTCONN)

CONNAME(win12.mqgem.com(1602)) VERSION(8.0)

_________________________________________________

CHANNEL(MQG2.SVRCONN) CHLTYPE(CLNTCONN)

CONNAME(aix5.mqgem.com(4231)) VERSION(8.0)

_________________________________________________

CHANNEL(MQG3.SVRCONN) CHLTYPE(CLNTCONN)

CONNAME(mvs1.mqgem.com(1255)) VERSION(8.0)

_________________________________________________

Total display responses – Received:3

>

As you can see, at the moment all the channels in this CCDT are at V8.0 which means my V7.0 client won’t be able to read them. I need to make a change to each record to ensure MQSCX will write it out at version V7.0 as I have indicated on my start command. Helpfully, I can do that in one single command:-

ALTER CHANNEL(*)

This makes no actual change to the attributes of the channel definition, but does ‘touch’ each record to ensure that it gets the new version. Displaying the records again as above will show that the version number for each channel mentioned by the ALTER command (in this example all of them), now indicates it is at version V7.0, just what my back-level client application needs.

Exiting MQSCX and re-running it will show you that this earlier version of the CCDT has indeed been hardened.

Note that if you had been using some attributes introduced in later versions than V7.0, this information would be lost when altering the channel definition to be an earlier version.


If you’d like to try out MQSCX, please email support@mqgem.com to request a trial licence.

Visibility of QMgr’s Server in MO71

The MO71 product allows you to administer queue managers that are both local and remote to the machine where it runs. When remote, you can connect in via mode, or in client mode. Since WebSphere MQ V7.1 was released, queue managers could be configured as multi-instance, and client connections could be made with a comma-separated list of hostnames (and ports) in the CONNAME field.

This comma-separated CONNAME has been available to use in MO71’s direct client configuration since that time. You can also make MO71 client connections using a CCDT if you prefer.

When using a comma-separated list of hostnames, it is useful to know which of the hostnames you actually managed to connect to, perhaps so you can have a quick view of which machine is currently the primary in order to log into the machine directly, or because you have a preferential Primary and Backup machine and you need to know when the Backup is in use.

MO71 Client configuration

MO71 Client configuration for Multi-instance queue manager

MO71 showing multi-instance annotation

MO71 showing multi-instance annotation

When configuring the list of hostnames in MO71’s client connection dialog, you can now optionally add a string to be displayed when that hostname is the one successfully connected to. You don’t have to include the string for each hostname. You may prefer to only have the annotation on the screen when the Backup server is in use and so would omit the string for the primary hostname and add it to the secondary hostname.

Here are a few examples

Connection Name : win9.mqgem.com(1701) “Primary”, win12.mqgem.com(1701) “Secondary”
Connection Name : win9.mqgem.com(1701), win12.mqgem.com(1701) “**Backup**”

Note that you must configure the client channel directly for this annotation to work. You cannot, for example, use a channel defined in the Client Channel Definition Table (CCDT).


Comma-separated CONNAME has been in MO71 for a while, but the recent 9.0.2 release added the ability to see which of the instances is in use.

Be sure of your CCDT Version

CCDT VersionWe all know that you can’t use a CCDT file with a client that is older than the CCDT version. For example, a version 6 client can’t understand what the channel definitions in a V8 produced CCDT mean.

How often have you had a CCDT file in your hand and wondered what version it was and whether you can give it to some of your known back-level client machines to use.

MQSCX can help you determine this. Open up your CCDT using the mqscx -n mode and then you can quite simply display the version number of all your client channels therein.

DISPLAY CHANNEL(*) VERSION

CHANNEL(MQGEM.SVRCONN)            CHLTYPE(CLNTCONN)   VERSION(8.0)
CHANNEL(MQGEM.SVRCONN.SSL)        CHLTYPE(CLNTCONN)   VERSION(9.0)
CHANNEL(MQGEM.SVRCONN.ADMIN)      CHLTYPE(CLNTCONN)   VERSION(7.0)

Alternatively, you might like to write a little script that could iterate through all the channels in the table and tell you the minimum and maximum versions in use within that CCDT.

if (_ccdtmode)
  @minVer = 999999999
  @maxVer = 0
  foreach (DISPLAY CHANNEL(*) VERSION)
    if (VERSION < @minVer)
      @minVer = VERSION
    endif
    if (VERSION > @maxVer)
      @maxVer = VERSION
    endif
  endfor
  if (_numEach)
    print "Channel version in",_ccdt
    print "Minimum:",@minVer,"Maximum:",@maxVer
  else
    print "No channels found in",_ccdt
  endif
else
  print "Run this script with MQSCX in CCDT mode"
endif

Running this against a multi-version CCDT might produce output such as:-

Channel version in C:\MQGem\CCDT\MQGEM101.TAB
Minimum: 7.00 Maximum: 9.00

foreachThis type of CCDT analysis is possible due to the recent addition of the foreach construct to the CCDT processing in MQSCX.

If you are a current MQSCX licence holder, you can simply download the new version of MQSCX and start using it. If you’re not a current licence holder, and you’d like to try out MQSCX, please email support@mqgem.com to request a trial licence.

Scripts using foreach on the CCDT

foreachThere are different types of users who use MQSCX. Some like the interactive experience, with tab auto-complete of commands, keywords and objects names. Others like the ability to create and edit CCDT files suitable for any required version of IBM MQ. Others again like the powerful control language which makes writing scripts to interrogate your queue manager a breeze.

Freaking awesome, Paul! I wrote several reporting scripts for a customer a couple of months back. They declined to purchase MQSCX so I was forced to do much of the logic in the script, giving me I have a good basis for comparison of both approaches. The differential in lines of code, complexity and amount of additional billable time I spent would have paid for a site license for several years. The ROI is now more than doubled, possibly even 5x what it was.

User comment on MQSCX – see more at What our customers say

Sometimes those different use cases come together. The control language has a for loop concept where you can easily iterate over all the queue manager objects that are returned by the command server as the answer to an MQSC command, with a script something like this:-

@total = 0
foreach(DISPLAY QLOCAL(*) WHERE(CURDEPTH GT 0))
  @total = @total + CURDEPTH
endfor
print 'Total CURDEPTH =',@total

You can also write scripts that operate, not on queue manager objects, but on the contents of a CCDT file.

With the latest version of MQSCX, you can use the foreach construct on the items in your CCDT file in just same way as above. Here’s a small example:-

@ssl = 0;
foreach(DISPLAY CHANNEL(*) SSLCIPH)
  if (SSLCIPH)
    @ssl = @ssl + 1
  endif
endfor
print 'Found',@ssl,'SSL Channels out of',_numEach

Now with MQSCX V9.0.0 you can use the powerful control language to analyse and manipulate your CCDT files. Another example of using the foreach construct on a CCDT file can be see in Be sure of your CCDT Version


If you are a current MQSCX licence holder, you can simply download the new version of MQSCX and start using it. If you’re not a current licence holder, and you’d like to try out MQSCX, please email support@mqgem.com to request a trial licence.

Using the CCDT URL

MQGem recently delivered new versions of MO71 and MQSCX that support the new IBM MQ V9 release. As well as support for the new command level, they support other new IBM MQ V9 features. One such feature is the ability to have your Client Channel Definition Table (CCDT) hosted somewhere centralised, such as on an FTP server or web server. Prior to IBM MQ V9, this was only available to Java clients due to the fact that the language gave you the capability whenever you needed to specify a file name URI. Now in IBM MQ V9, ‘C’ clients (and unmanaged .NET clients) also have this feature.

Jon Rumsey has a great write-up of this feature on the MQDev Blog, MQ V9 Client Channel Table Enhancements – URL retrieval.

MO71

You can either set the MQCCDTURL environment variable and the whole MO71 application will take note of it, or you can set it individually for specific locations by providing the CCDT URL in the location dialog. Open the Location, ensure the Client checkbox is ticked which enables two things, the Configure button – which is for defining your channel definition manually through the MQCNO, and the CCDT URL entry field, which is where you can put the URL of your hosted CCDT file. You don’t need both of course, and MQ defines a precedence order of which is used if you do specify both.

MO71 Location Dialog showing CCDT URL field in use

Specify your CCDT URL in the MO71 Location Dialog.
Please note, the URL shown is for demonstration purposes only. There is no CCDT at the shown URL!

MQSCX

As with MO71, you can either set the MQCCDTURL environment variable and the whole MQSCX application will take note of it, or you can set it individually for specific locations by providing the ccdturl() on the =conn command.

MQSCX Extended MQSC Program – Version 9.0.0

Licenced to Paul Clarke

Licence Location: Head Quarters

> =conn qm(MQ900) client ccdturl(http://www.mqgem.com/MQGEM.TAB)

MQSCX Extended MQSC Program – Version 9.0.0

Licenced to Paul Clarke

Licence Location: Head Quarters

[14:05:31] =conn qm(MQ900) client ccdturl(http://www.mqgem.com/MQ

Connected to ‘MQ900’

MQ900>


If this feature interests you and you’d like to try it out for yourself on either of these products, and you are not currently a licence holder, you may email support@mqgem.com to request a trial licence.