Retaining Events with MQEV

How long should you keep event messages for?

That’s one of those “It depends” questions.

• Is it worth keeping command events for DISPLAY commands at all?
• Is it worth keeping channel retrying events once the channel is up and running again?
• Would you keep “not authorized” events for longer than logger events?

MQEV stores your IBM MQ event data on streams. By default all events are stored on the $EVENTS stream which has a retention interval of 90 days. Streams are your control point for how long to hold onto this data. You might decide that authority events should be kept for much longer, but that channel events can be kept for a shorter time. In which case you might create some new streams thus:-

DEFINE EVSTREAM(CHANNEL) TYPE(EVENTS) RETINTVL(30)
DEFINE EVSTREAM(SECURITY) TYPE(EVENTS) RETINTVL(120)

And then send different event types to their appropriate streams with the following in your MQEVEvent() function in the mqev.mqx script.

*********************************************************************
* Function for processing an event                                  *
*********************************************************************
func MQEVEvent()
  if (event.evtype = AUTHOR)
    _stream = "SECURITY"
  endif;
  if (event.evtype = CHANNEL)
    _stream = "CHANNEL"
  endif;
endfunc

You can still display all your events across these different streams with the usual DISPLAY EVENTS(*) command or the MO71 events list dialog, but if you need to, you can now also filter them by the stream that they are stored in.

MQEV will take care of deleting these events once they pass the retention interval defined on the stream they are stored in.

Of course, this is a very simplistic example. You could separate events out further, for example, by a specific reason. Some channel events might be worth keeping with the authority events, for example “Channel is Blocked” events. Let’s make a small change to the MQEVEvent() function to achieve that.

*********************************************************************
* Function for processing an event                                  *
*********************************************************************
func MQEVEvent()
  if (event.evtype = AUTHOR)
    _stream = "SECURITY"
  endif;
  if (event.evtype = CHANNEL)
    if (event.cfhreason = const.MQRC_CHANNEL_BLOCKED)
      _stream = "SECURITY"
    else
      _stream = "CHANNEL"
    endif;
  endif;
endfunc

And you are not limited there – you can use any data in the event message to determine which stream it should be stored on. For example, you might have a very important application whose retention interval is much higher than everything else, and so make use of the object name in the event, in order to save it off to the appropriate stream. This is where having a good naming convention helps greatly.

First Steps with MQEV and MQ Events

Our newly released product MQEV, can be used to collect, process, store and search the data generated by your IBM MQ Queue Manager as MQ Event messages. In this post, we will take you through the first steps of using MQEV to look at event messages.

We assume that you have downloaded the MQEV program from our website, and also grabbed the latest version of either MQSCX or MO71 to interact with MQEV. You don’t need to have a licence file for either of these products in order to use them with MQEV – they are covered by the MQEV licence, and will work for free without any licence until June 30th 2020. Installation of MQGem products is really easy – just unzip/untar into a location that is in your path and off you go.

Make sure your queue manager is running and then run the provided script file config.mqx to define the two queues that the MQEV product requires in order to work (MQGEM.MQEV.COMMAND.QUEUE and MQGEM.MQEV.DATA.QUEUE).

Now run the MQEV program with the following command (assuming your queue manager is called MQG1).

mqev -m MQG1

Now MQEV is running, but it’s not capturing any events yet – first you have to tell it what you’re interested in. Let’s assume you want to process command events and configuration events to begin with. (I’ve chosen these because they are the easiest events to generate so you can immediately see some data in MQEV). First turn them on in your queue manager with the following MQSC command (or it’s equivalent action in your favourite GUI).

ALTER QMGR CMDEV(NODISPLAY) CONFIGEV(ENABLED)

(Wondering about CMDEV(NODISPLAY)? Read more here).

Now your queue manager will generate command and configuration events, and the next step is to tell MQEV to listen out for them. You can do this using a command line tool such as MQSCX (in =mqev mode) or using the MO71 GUI. Open one of the twisties below to see instructions.

Command Line: MQSCX

Issue the following commands:

=mqev
RESUME EVQ(SYSTEM.ADMIN.COMMAND.EVENT)
RESUME EVQ(SYSTEM.ADMIN.CONFIG.EVENT)

If you now issue the following command your will see that the two queues you resumed above, now show STATUS(ACTIVE).

DISPLAY EVQ(*)

Using MO71 GUI

Right click on the queue manager in the main window and from the context menu choose MQ EV Events -> MQ Event Queue List …
Press the “Refresh” button to populate the list.
You will be presented with a list of the known event queue names (these are pre-configured to MQEV for you, but all in a suspended state so that you control when to start reading from them). Select the following two queues, the right click and choose “Resume” from the context menu.

• SYSTEM.ADMIN.COMMAND.EVENT
• SYSTEM.ADMIN.CONFIG.EVENT

Resume context menu on a list of MQEV event queues

You will see the dialog list automatically refresh and show that your two queues now have a status of “Active” instead of “Suspended”.

The first thing you will notice is that you are immediately given some events. This is because you just issued a command to MQ and made a change to the configuration of your queue manager – just by turning on the event switches! Take a look at these events now. Open one of the twisties below to see instructions.

Command Line: MQSCX

Issue the following commands:

=mqev
DISPLAY EVENTS(*)

You will see output something like this:-

[20:36:55] =mqev

Commands directed to MQEV

[20:37:07] DISPLAY EVENTS(*) EVENTID

_________________________________________________________________________________

EVQMGR(MQG1) EVTIME(2020-01-03 20:36:10 (Local)) EVENTID(00000002)

SUMMARY(Config – Change Object – Qmgr:MQG1 – CONFIGEV[DISABLED -> ENABLED] CMDEV[

DISABLED -> NODISPLAY])

_________________________________________________________________________________

EVQMGR(MQG1) EVTIME(2020-01-03 20:36:10 (Local)) EVENTID(00000001)

SUMMARY(Command – Change Qmgr – Qmgr:MQG1))

_________________________________________________________________________________

Total display responses – Received:2

MQEV:MQG1>

Using MO71 GUI

Right click on the queue manager in the main window and from the context menu choose MQ EV Events -> MQ Event List …
Press the “Refresh” button to populate the list.
You’ll see two events in the list like the screenshot below.

MO71 showing events captured by MQEV

This just shows a summary of the event details. Now let’s look at all the contents of one of the events.

Command Line: MQSCX

Issue the following commands to look at the Change Qmgr configuration event from the list.

DISPLAY EVENTS($EVENTS) EVQMGR(MQG1) EVENTID(2) DISTYPE(CONDENSE) ALL

You’ll see output like the following. This display type is a good way to look at change events because you don’t get swamped with all the rest of the attributes from the object (a queue manager object has quite a lot of attributes), and instead are only shown the ones that have changed. Experiment without the display type to see the difference.

[20:47:07] DISPLAY EVENTS($EVENTS) EVQMGR(MQG1) EVENTID(2) DISTYPE(CONDENSE) ALL

EVQMGR(MQG1) EVENTS($EVENTS) EVTIME(2020-01-03 20:36:10 (Local))

EVREASON(CFGCHGOBJ) EVTYPE(CONFIG) EVUSERID(mqgemusr) EVOBJNAME(MQG1)

EVOBJTYPE(QMGR) EVENTID(00000002) CFHCMD(43) CFHREASON(2368)

SUMMARY(Config – Change Object – Qmgr:MQG1 – CONFIGEV[DISABLED -> ENABLED] CMDEV[

DISABLED -> NODISPLAY])

BEFORE:

CONFIGEV(DISABLED) CMDEV(DISABLED)

AFTER:

CONFIGEV(ENABLED) CMDEV(NODISPLAY)

Total display responses – Received:1

MQEV:MQG1>

In this output you can see two small sections at the end, a “BEFORE:” section and an “AFTER:” section showing the values of the attributes that have been changed.

Using MO71 GUI

Double click on the “Config – Change Object” event to take a look at the details. This dialog will look very similar to the Queue Manager object dialog that you might well be familiar with from making changes to your queue manager. It has the same recognisable set of tabs along the top. Switch to the “Monitoring” tab and you’ll see the attributes that were changed and are reported in this event.

MO71 display of a single configuration change event from MQEV

Now create some other objects, or make changes to pre-existing objects to see more examples of command and configuration events and use the above commands or variations on them to look at the details.

We chose command and configuration events because they are so easy to generate. Other events are sometimes a bit harder to cause to happen, so to make it easier for you to look at what these other events contain, we have also provided a QLOAD / dmpmqmsg file containing a number of other events that you can just load onto one of your event queues (for example one of the two that MQEV is already reading) and then have a good old look around them.

Issue the following command:

qload -m MQG1 -o SYSTEM.ADMIN.COMMAND.EVENT -f TestEventMsgs.qld -CI

If you’re not a QLOAD licensee, you can, in this case, use dmpmqmsg with the same parameters.

---

The product is free for you to download and try out until June 30th 2020. You don’t even need to register any details to play with it. You can download it from our website.

All we ask in return is that in a week or two when you’ve played with it you send us an email telling us what you liked and what you didn’t like about it and perhaps what features you’d like to see in the future.

We look forward to hearing your comments. Please send them to support@mqgem.com.

Announcing the MQGem Event Processor!

MQGem Software is pleased to announce that it has just released a new product, MQEV, which makes processing Event, Accounting and Statistics messages a breeze. What’s more the product is absolutely free to use for the first 6 months to enable you to verify that it is useful to you.

Many installations don’t make full use of the Event, Accounting and Statistics messages that MQ can generate because the messages they generate are hard to analyse. This is a huge waste of valuable information which could be very useful in your planning or problem diagnosis tasks.

• Have you ever asked yourself any of the following?
• Did anything unusual happen with my Queue Manager over the weekend?
• Is queue or channel XYZ still being used? …and, if so, by who/what?
• What type of MQ clients are using my Queue Manager? JMS ? C ? Java ? C++ ?
• What version of MQ clients are connecting to my Queue Manager? Are any of the clients old and need updating?
• What channels are being used and what is the relative split of persistent message traffic?
• How efficient are my channels? What is the average batch size in use over time?
• Are my channel exits efficient? What time is being spent in them?
• How is my network performing? How does my network round-trip time vary over time?
• How does the activity/depth of queue X vary over the day? What are my peak traffic times?
• How does the activity/depth of queue X vary over the last 3 months? Am I seeing an increase of traffic?
• Will my MQ infrastructure be able to cope with future demands?
• Are my channels having to ever retry their puts to deliver messages?
• Are any of my applications misbehaving? Do we see excessive amounts of failed MQI calls such as MQGETs, short connections, or are they opening/closing the queue for each message?
• How is my processing speed varying over time? How long are messages languishing on queues?
• Am I delivering on my Service Level Agreements (SLA) ?
• Are my applications publishing to topics with no subscribers?
• What is the maximum depth my queue has reached during a particular period?

These types of questions, and many more, are the type of questions you can answer if you have a history of your event, accounting and statistics messages. With MQEV all you need to have is a couple of queues on your Queue Manager – no need for an expensive and complicated database.

To view, search and aggregrate the data collected and stored by MQEV, you can use command line commands through MQSCX, or a GUI interface through MO71. To use either of MQSCX or MO71 for MQEV purposes, no further licence is required beyond what is needed for MQEV. Until June 30th 2020, no licence is needed at all.

MO71 GUI displaying MQ Event messages in MQEV

MQSCX command line displaying MQ Event messages in MQEV

The product is free for you to download and try out until June 30th 2020. You don’t even need to register any details to play with it. The product is free to download from our website.

All we ask in return is that in a week or two when you’ve played with it you send us an email telling us what you liked and what you didn’t like about it and perhaps what features you’d like to see in the future.

We look forward to hearing your comments. Please send them to support@mqgem.com.