dlsym and dlerror Weirdness on Solaris

Lately, I have been complaining about Microsoft’s Visual C++, so I figured I better write about some recent Solaris weirdness.

A customer installed MQAUSX on Solaris v10 that has WebSphere MQ v7.0.1.9 installed. They configured MQAUSX to authenticate the incoming MQ client connections against a remote LDAP server. Everything was working perfectly fine. Here’s what the calling sequence looks like:

MCA (amqrmppa) -> mqausx -> mqausxldap64.so -> client LDAP libraries ———> remote LDAP server

The customer decided to install WebSphere MQ v7.5 along side WebSphere MQ v7.0.1.9 (multi-installation) which is perfectly fine because MQAUSX can work with any version of MQ. Once they configured the WMQ v7.5 queue manager to use MQAUSX, for the first connection, MQAUSX would successfully connect to the LDAP server for authentication but for any other connection, MQAUSX would log the following dlsym error:

ld.so.1: amqrmppa: fatal: _ex_unwind: can't find symbol

I have absolutely no idea what “_ex_unwind” is. The same MQAUSX code runs on AIX, HP-UX, Linux and Solaris but it is only Solaris that has this issue.

If the customer restarts the queue manager then the first connection was fine but all others would get the same dlsym error. My first thought was now what is different in WMQ v7.5 compared to WMQ v7.0.1.9. I did a bunch of searches but found nothing related to WMQ.

After I banged my head against the wall for a while, I decided to do some generic searches about Solaris, dlsym and _ex_unwind. I got a lot of hits but not a lot of information until I found some hits related to an application called “brltty”. brltty had the same issue, worked on the first load of the shared library but failed on all the others. The developer determined that the “if” statement with the dlerror function after the dlsym call was giving a false result.

handle = dlopen ("/var/mqm/exits64/mqausxldap64.so", RTLD_NOW | RTLD_GLOBAL);
if (handle == NULL)
{
   /* log dlopen error */
}
else
{
   myFunc = dlsym(handle, "AuthLDAP");
   if ((error = dlerror()) != NULL)
   {
      /* log dlsym error */
   }
   else
   {
      rcode = (*myFunc)(pFuncParms);
   }

   dlclose(handle);
}

The solution they gave was to add a call to dlerror before dlsym to clear any errors from a previous call (unrelated to the current call sequence). They said that if you do not clear dlerror then it may give a false result because dlerror holds the “last error” and not the “error from last dl*** call”.

handle = dlopen ("/var/mqm/exits64/mqausxldap64.so", RTLD_NOW | RTLD_GLOBAL);
if (handle == NULL)
{
   /* log dlopen error */
}
else
{
   dlerror();    /* Clear any existing error */
   myFunc = dlsym(handle, "AuthLDAP");
   if ((error = dlerror()) != NULL)
   {
      /* log dlsym error */
   }
   else
   {
      rcode = (*myFunc)(pFuncParms);
   }

   dlclose(handle);
}

I do not know if this is true (need to call dlerror before dlsym) for all Unix/Linux platforms but it does appear to be true for Solaris. Anyway, I implemented the solution, gave the update to the customer and now the customer is happy. 🙂

Regards,
Roger Lacroix
Capitalware Inc.