Weird Problem related to a Channel Security Exit

March 1, 2019 Roger Lacroix

I posted the following weird channel security exit problem on the MQSeries ListServer.

I received a lot of emails (both public and private) offering help to the problem. Thanks. It was not an SELinux issue nor was it a corrupt shared library issue. It was an issue related to permissions for a mount point.

Here is the description to the problem followed by the solution:

I’ve got a really weird problem, that the internet seems to say that it is a permission issue with “/tmp” but I’m at a loss.

– Brand new: Red Hat Enterprise Linux Server release 7.6 (Maipo)
– Fresh install of MQ v9.1.0.1 64-bit

Installed MQAUSX into /var/mqm/exits64/ and yes it is a working version that works perfectly on other Linux distributions.

Here is the MQ error message:

08/02/19 12:46:45 - Process(51014.254) User(roger) Program(amqrmppa)
                    Host(someserver) Installation(Installation1)
                    VRMF(9.1.0.1) QMgr(MQA1)
                    Time(2019-02-08T11:46:45.514Z)
                    ArithInsert1(536895861)
                    CommentInsert1(/var/mqm/exits64/mqausx)
                    CommentInsert2(/var/mqm/exits64/mqausx: failed to map segment from shared object: Operation not permitted)
                    CommentInsert3(64)

AMQ6175E: The system could not dynamically load the shared library
'/var/mqm/exits64/mqausx'. The system returned error message
'/var/mqm/exits64/mqausx: failed to map segment from shared object: Operation
not permitted'.

EXPLANATION:
This message applies to UNIX systems. The shared library '/var/mqm/exits64/mqausx' 
failed to load correctly due to a problem with the library.

ACTION:
Check the file access permissions and that the file has not been corrupted.

“mqausx” is a standard Unix/Linux shared library (It is running on tons of Linux servers without issue). Even the “ldd” command gives a weird result:

$ ldd /var/mqm/exits64/mqausx
ldd: warning: you do not have execution permission for `/var/mqm/exits64/mqausx'
        not a dynamic executable

The permissions are set as follows:

chown mqm:mqm mqausx
chmod 750 mqausx

I even tried 777 for permissions. i.e.

-rwxrwxrwx  1 mqm  mqm  247087 Feb  7 15:11 mqausx

It still failed. Its a shared library, what is or how is a shared library “dynamic executable”?

Here is the solution that Josh suggested that worked:

First, issue the following command against your shared library:

$ df /var/mqm/exits64
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/aaa/bbb           5107712   3967288   1140424  78% /var/mqm

Use whatever is under “Mounted on” (i.e. /var/mqm – you may have something else like /var ) in the following command:

$ mount | grep /var/mqm

In my customer’s particular case, their mount point was on /var and the results of the command was:

/dev/aaa/bbb on /var type xfs (rw,nosuid,nodev,noexec,relatime,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)

The source of the issue was that the mount point had “noexec”permission. Hence, once that option was removed then everything went back to normal.

I hope this helps someone else in the future.

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, Linux, MQ Auditor, MQ Authenticate User Security Exit, MQ Channel Connection Inspector, MQ Channel Encryption, MQ Channel Throttler, MQ Enterprise Security Suite, MQ Message Encryption, MQ Message Replication, MQ Standard Security Exit, Unix Comments Off on Weird Problem related to a Channel Security Exit

Best Practices for an IBM MQ Channel’s SHARECNV Parameter

March 1, 2019 Roger Lacroix

I was chatting with Josh the other day and he asked me if I knew that IBM now does not recommend uses the default value of 10 for a channel’s SHARECNV (shared conversation) value. He said that the MQ Knowledge Center recommends for distributed queue managers a value of 1 for a channel’s SHARECNV.

Of course, I said WTF and where did you read that. He emailed me the link and sure enough, that is what it says.

The default settings for client and server connection channels changed in Version 7.0 to use shared conversations. Performance enhancements for distributed severs were then introduced in Version 8.0. To benefit from the new features that were introduced alongside shared conversations, without the performance impact on the distributed server, set SHARECNV to 1 on your Version 8.0 or later server connection channels.

And further down it says:

However, for distributed servers, processing messages on channels that use the default configuration of 10 shared conversations is on average 15% slower than on channels that do not use shared conversations.

A 15% performance hit for using the default value for SHARCNV on distributed queue managers!!! OMG!

From a guy who writes MQ exits, I have always hated the SHARECNV parameter when it has a value greater than 1 because in my opinion, IBM did a horrible job in their implementation of it as it relates to a channel security exit.

When SHARECNV parameter is set to 1, a channel security exit is invoked with the following ExitReason values:

  • MQXR_INIT
  • MQXR_INIT_SEC
  • MQXR_SEC_MSG <- optional
  • MQXR_SEC_PARMS
  • MQXR_TERM

    • When SHARECNV parameter is set to a value greater than 1, i.e. 10, a channel security exit is invoked with the following ExitReason values:

  • MQXR_INIT
  • MQXR_INIT_SEC
  • MQXR_SEC_MSG <- optional
  • MQXR_SEC_PARMS – for session # 1
  • MQXR_SEC_PARMS – for session # 2
  • MQXR_SEC_PARMS – for session # 3
  • MQXR_SEC_PARMS – for session # 4
  • MQXR_SEC_PARMS – for session # 5
  • MQXR_SEC_PARMS – for session # 6
  • MQXR_SEC_PARMS – for session # 7
  • MQXR_SEC_PARMS – for session # 8
  • MQXR_SEC_PARMS – for session # 9
  • MQXR_SEC_PARMS – for session # 10
  • MQXR_TERM

    • Everything looks EXACTLY the same for the 1st session but for session #2 to session #10, the calling sequence is totally different.

    Secondly, did you happen to notice that when all but the last session ends, the channel security exit is NOT made aware of this fact!!! The channel security exit is only invoked with MQXR_TERM when the last session, whichever number that is, terminates (never for the other sessions ending). I have no idea who dreamt this up, but man it is a bad design.

    I am so happy that Josh told me about this little tidbit. Now I get to tell my customers that IBM’s best practices for a channel’s SHARECNV parameter on distributed platforms, is to set it to 1 rather than use the default value of 10.

    It would be interesting to know the performance for z/OS queue managers whose channel’s SHARECNV is set to 10 vs 1.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    HPE NonStop, IBM i (OS/400), IBM MQ, IBM MQ Appliance, Linux, Unix, Windows Comments Off on Best Practices for an IBM MQ Channel’s SHARECNV Parameter

    RIP: MQ Technical Conference

    February 19, 2019 Roger Lacroix

    Capitalware Inc. is a small software company specializing in IBM MQ. Capitalware used to be a vendor at IBM’s Transaction and Messaging Technical Conferences (T&M Conference). IBM merged several conferences including the T&M Conference into Impact 2007. I went as a vendor to Impact 2007 but found that most attendees were not involved in MQ. So, I decided that it would be the last conference I would attend as a vendor. I starting bugging IBM PartnerWorld, marketing and education people to bring back IBM’s T&M conference. This went on for 6 years with IBM people always saying “that is not the direction we are going in”.

    So, in earlier 2013, after many years of frustration, I decided to stick my neck out and create MQ Technical Conference (MQTC). I wanted a technical conference to be focused on IBM MQ (plus IBM products that use MQ) and thought 3 days was a good length. Since, my wife and I have 5 kids, I decided the conference should be at a family-friendly hotel/resort. I had already been to many conferences at Kalahari Resorts in Sandusky, Ohio, so decided it was a good location for MQTC. When I signed the contract with Kalahari Resorts, I had to personally guarantee to pay them $24,000 USD (for food and facilities) regardless if anyone showed up or not (plus penalties for minimum room guarantee). I was pretty nervous about it and hoped attendees would show up.

    It is a lot of work to create a conference, coordinate with vendors, food, audio/video, scheduling of sessions, etc. but I always enjoyed the conference and I believe most attendees found it valuable.

    Last year, Ray Daniel of IBM came to MQTC v2.0.1.8 as part of IBM sponsorship team. He so enjoyed the conference that he decided IBM needed to create its own conference modeled after MQTC. He created a 3-day technical conference called Integration Technical Conference (ITC). Like MQTC, it will have 5 session rooms, like MQTC, it will be heavy into IBM MQ and like MQTC, it will be at a family-friendly venue. The big difference, between ITC and MQTC is that ITC will be FREE to attend. Last year, MQTC v2.0.1.8 costs more than $65,000 USD and I cannot possibly offer a free conference.

    Therefore, I have decided to suspend MQTC as IBM now offers a conference for MQ users which is free.

    I do hope that Ray Daniel continues the tradition of having the cheesecake bar and ice-cream social at ITC. 🙂

    I would like to thank:

  • All of the attendees who came to the 6 MQTC conferences
  • All of the various speakers from a wide variety of industries who spoke at the 6 MQTC conferences
  • All the sponsors who supported the 6 MQTC conferences.

    • Thank you.

    Finally, I would like to especially thank T.Rob Wyatt for his support and help with MQTC.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    P.S. I would like to hear feedback from ITC attendees on core and non-code MQ sessions.

    Capitalware, Education, IBM MQ, MQ Technical Conference 10 Comments

    How to Compile and Link MQ Programs on Linux

    February 13, 2019 Roger Lacroix

    Someone on StackOverflow asked “How to compile a C program without knowing the include files”.

    JasonE gave a really good answer, so I thought I would repeat it here:

  • To compile and link 64-bit MQ applications on Linux, follow the instructions here
  • To compile and link 32-bit MQ applications on Linux, follow the instructions here

    • In summary:

  • -I is for the product includes, which are (For Linux) usually in /opt/mqm/inc
  • -L is the path to the libraries in your example which are (For Linux) usually in /opt/mqm/lib (for 32 bit applications) and /opt/mqm/lib64 (for 64 bit applications)
  • -l (lower case L) is for the required library/libraries

    • and the actual library you need is either:

  • mqm – server bound C applications (ie -lmqm, which links with libmqm.so)
  • mqic – client bound C applications (ie -lmqic, which links with libmqic.so)

    • and a suffix of _r if you are building as a threaded application (ie you are linking with -lpthread as well, ie providing -lmqm_r or -lmqic_r which in effect links with libmqm_r.so or libmqic.so)

    cmqc.h is the name of the main header file, and there are other cmq*.h headers you can optionally include as well.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    C, C++, IBM MQ, Linux, Programming Comments Off on How to Compile and Link MQ Programs on Linux

    Knock knock

    February 11, 2019 Roger Lacroix

    IBM: Knock knock
    User: Who’s there?
    IBM: MQ
    User: MQ who?
    IBM: IBM MQ Client for macOS

    In case you didn’t hear, IBM has released “IBM MQ macOS toolkit for developers” a couple of days ago.

    I downloaded and installed it. I simply uncompressed it into my home directory. I played around with a bunch of commands (dspmqver, amqsputc, amqgetc, etc.) and everything worked as expected for an MQ client install.

    I do C and Java development on a whole variety of platforms: AIX, HP-UX, Linux, IBM i (OS/400), Solaris, Windows and z/OS. I’ve only ever done Java development on macOS, so I thought I would have some fun and compile & link some C samples on macOS.

    I did some internet searches and people said that you can use a Linux makefile with some tweaks on macOS. So, I copied the amqsput0.c and amsget0.c from a Linux server and also copied over a makefile I have on a Linux server to my macOS. After a couple of tweaks, I got it to compile and link those 2 MQ sample files.

    Here is the makefile I used:

    #
# makefile for MQ applications on macOS
#
CC = clang
CFLAGS = -std=c99 -pedantic -Wall

MQINSTALLPATH = $(HOME)/IBM-MQ-Client-Mac-x64-9.1.1.0
MQINC = -I$(MQINSTALLPATH)/inc
MQLIBPATH64 = -L$(MQINSTALLPATH)/lib64
MQLIB = -lmqic_r

all: amqsgetc amqsputc

amqsget0.o: amqsget0.c
	$(CC) $(CFLAGS) $(MQINC) -c amqsget0.c

amqsgetc: amqsget0.o
	$(CC) amqsget0.o -o amqsget0 $(MQLIBPATH64) $(MQLIB)

amqsput0.o: amqsput0.c
	$(CC) $(CFLAGS) $(MQINC) -c amqsput0.c

amqsputc: amqsput0.o
	$(CC) amqsput0.o -o amqsput0 $(MQLIBPATH64) $(MQLIB)

clean:
	rm -f *.o amqsgetc amqsputc

    When I tried to run either of the 2 programs, I got the following error:

    dyld: Library not loaded: @rpath/libmqic_r.dylib

    After a few internet searches, I discovered that I needed to set DYLD_LIBRARY_PATH environment variable. It is like LD_LIBRARY_PATH on Linux.

    export DYLD_LIBRARY_PATH=$HOME/IBM-MQ-Client-Mac-x64-9.1.1.0/lib64

    When I did that, then everything worked as expected.

    Now for those that are paying attention, the first question should be, how did the amqsputc and amqsgetc that were included in the {MQ_INSTALL_DIR}/samp/bin/ directory work? It is because the IBM developer who compiled & linked those MQ samples explicitly set the “rpath” linker setting. Since they control where the dynamic libraries are in relation to the MQ samples, they simply set rpath as “../../lib64/”.

    If you want to do the same then your makefile will look like:

    #
# makefile for MQ applications on macOS
#
CC = clang
CFLAGS = -std=c99 -pedantic -Wall

MQINSTALLPATH = $(HOME)/IBM-MQ-Client-Mac-x64-9.1.1.0
MQINC = -I$(MQINSTALLPATH)/inc
MQLIBPATH64 = -L$(MQINSTALLPATH)/lib64
MQLIB = -lmqic_r

RTL64 = -Wl,-rpath,$(MQINSTALLPATH)/lib64

all: amqsgetc amqsputc

amqsget0.o: amqsget0.c
	$(CC) $(CFLAGS) $(MQINC) -c amqsget0.c

amqsgetc: amqsget0.o
	$(CC) amqsget0.o -o amqsget0 $(MQLIBPATH64) $(MQLIB) $(RTL64)

amqsput0.o: amqsput0.c
	$(CC) $(CFLAGS) $(MQINC) -c amqsput0.c

amqsputc: amqsput0.o
	$(CC) amqsput0.o -o amqsput0 $(MQLIBPATH64) $(MQLIB) $(RTL64)

clean:
	rm -f *.o amqsgetc amqsputc

    I don’t recommend you do this unless you have tight control over where the MQ Client installation will be located. I think it is easier to simply set the DYLD_LIBRARY_PATH environment variable.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    C, IBM MQ, macOS (Mac OS X), Programming Comments Off on Knock knock

    SQLite v3.27.1 Released

    February 8, 2019 Roger Lacroix

    D. Richard Hipp has just released SQLite v3.27.1.
    https://www.sqlite.org/news.html

    SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    C, Database, IBM i (OS/400), Linux, macOS (Mac OS X), Open Source, Programming, Unix, Windows, z/OS Comments Off on SQLite v3.27.1 Released

    IBM MQ V9.1.2 Announced

    February 7, 2019 Roger Lacroix

    IBM has announced IBM MQ V9.1.2:
    http://www.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_ca/1/877/ENUSZP19-0031/index.html

      Highlights:

    • MQ V9.1.2 base automatically balances a pool of connected applications across a set of available queue managers.
    • MQ V9.1.2 base provides enhanced REST Messaging performance with connection pools.
    • MQ Advanced V9.1.2 includes the capabilities in MQ V9.1.2 base and new REST API calls for MQ Managed File Transfer (MQ MFT) administration.
    • MQ Appliance V9.1.2 firmware includes the capabilities in MQ V9.1.2, and also the capability to stream logs off the MQ Appliance. Version 9.1.2 firmware is available for the following appliances:
      • MQ Appliance M2000
      • MQ Appliance M2001
      • MQ Appliance M2002

    Planned availability for IBM MQ V9.1.2 is March 21, 2019 for Electronic software delivery.

    IBM MQ (aka WebSphere MQ) homepage
    https://www.ibm.com/products/mq

    Regards,
    Roger Lacroix
    Capitalware Inc.

    Fix Packs for MQ, IBM MQ, IBM MQ Appliance, Linux, Unix, Windows, z/OS Comments Off on IBM MQ V9.1.2 Announced

    IBM MQ macOS toolkit for developers

    February 7, 2019 Roger Lacroix

    IBM releases IBM MQ macOS toolkit for developers. I’m shocked and wildly surprised that IBM did this.
    https://developer.ibm.com/messaging/2019/02/05/ibm-mq-macos-toolkit-for-developers/

    This toolkit provides the ability to develop and interact directly with MQ from your Mac, for developers using the language interfaces; such as C, C++, COBOL, GoLang, and Node.js; which are built on the platform-native C library (libmqm) . This extends the capability we already provide for Java developers on Mac, with the IBM MQ com.ibm.mq.allclient.jar.

    I downloaded and installed it. You are getting the IBM MQ Client components for macOS. For some strange reason, MQ Explorer is still not available for macOS (that’s the only downside).

    Hopefully, sometime in the future, IBM will port the MQ server components to macOS.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    C, C++, IBM MQ, Java, JMS, macOS (Mac OS X), Programming Comments Off on IBM MQ macOS toolkit for developers

    NBR: Privacy Policies: What are we really agreeing to?

    February 7, 2019 Roger Lacroix

    I have been watching NBR (Nightly Business Report) for a very, very long time. Last night, it had an awesome segment on data privacy policies. They explained the data privacy issues so that the average user will understand it.

    Everyone should watch this 5 minute segment:

    Regards,
    Roger Lacroix
    Capitalware Inc.

    Security Comments Off on NBR: Privacy Policies: What are we really agreeing to?

    More Spam Garbage

    February 4, 2019 Roger Lacroix

    For the last week, I have been receiving the following emails (and variations) to a variety of email accounts for the different domains I have. I have been laughing about it, deleting them and moving on. But today is Monday and I’m in no mood for this crap.

    Hey dumb ass spammers, I’ll let some of it slide but don’t annoy me before my Monday morning coffee!!!

    The email is an interesting threat but there are so many problems with it, I figured I would go through a bunch of them:

    (1) You didn’t hack my email account (or accounts since I have received many of these emails). First thing I did was turn on “All Headers” in Thunderbird. The information was:

    Message Id: rpapfg.h4vyn05g8k1ilhh@mail.rainvac.com
Return Path: marketing@rainvac.com
Received:
from server.rainvac.com ([67.225.188.99]:56969)
from 90-148-158-212.bluetone.cz ([212.158.148.90]:59400

    So, an email account is hacked but it ain’t mine. Someone from the Czech Republic is logging into the marketing email account at “Rainbow Vacuum Specialists” and spamming the world with this garbage.

    If anyone from “Rainbow Vacuum Specialists” reads this emails, would you please KICK your IT department’s CTO/director’s chair and tell them to get off there ass and fix the hacked account(s). Also, remove the malware that is installed too!!

    (2) I don’t go to porn sites. Hence, no malware was installed on my PC. I also keep AVG software up to date plus I run TDSKiller at least once a week. My PC is my whole life, so I can’t allow anything to happen to it.

    (3) I do NOT have a web cam attached to my PC. I just don’t have a need for a web cam. I’m not into video chatting. Hey, I’m a boring guy!! Also, if I was going to video chat then I would use my iPhone’s FaceTime. D’Oh!

    (4) The nonsense about Facebook Pixel makes me laugh because it is just so dam funny. For those computer geeks out there, here’s the funny part:

    Content-Type: text/plain; charset=UTF-8

    For those non geeks, basically the header says that the email is a Plain Text message (i.e. not HTML aka web) hence there could not be any tracking in the email. i.e. not like a web page.

    So, here is the dumb ass’ spam email:

    You may not know me and you are probably wondering why you are getting this e mail, right?
    I’m a hacker who cracked your email and devices a few months ago.

    Do not try to contact me or find me, it is impossible, since I sent you an email from YOUR hacked account.
    I setup a malware on the adult vids (porno) web-site and guess what, you visited this site to have fun (you know what I mean).
    While you were watching videos, your internet browser started out functioning as a RDP (Remote Control) having a keylogger which gave me accessibility to your screen and web cam.
    After that, my software program obtained all information.

    You entered a passwords on the websites you visited, and I intercepted it.
    Of course you can will change it, or already changed it.
    But it doesn’t matter, my malware updated it every time.
    What did I do?

    I backuped device. All files and contacts.
    I created a double-screen video. 1st part shows the video you were watching (you’ve got a good taste haha . . .), and 2nd part shows the recording of your web cam.
    exactly what should you do?
    Well, in my opinion, $1000 (USD) is a fair price for our little secret. You’ll make the payment by Bitcoin (if you do not know this, search “how to buy bitcoin” in Google).
    My Bitcoin wallet Address:
    1CJRgpn87v1YU55xL5uzLQ8ekssiuHR7CE
    (It is cAsE sensitive, so copy and paste it)

    Important:
    You have 48 hour in order to make the payment. (I’ve a unique pixel in this e mail, and at this moment I know that you have read through this email message).
    To track the reading of a message and the actions in it, I use the facebook pixel.
    Thanks to them. (Everything that is used for the authorities can help us.) If I do not get the BitCoins, I will certainly send out your video recording to all of your contacts including relatives, coworkers, and so on.

    Ok, now I’m going to drink my coffee in peace.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    Capitalware Comments Off on More Spam Garbage