Its All Fun & Games Until Your Message Is Empty!

I’ve collected a bunch of JSON sample files from the internet for testing with MQ Visual Edit. It is always good to have a variety of samples when testing features of a program rather than making up a couple of your own.

I was doing some testing with MQ Visual Edit and the JSON samples when I happen to select a particular JSON sample, clicked the “Format” button (pretty-print format) then clicked “Save to Queue” button. The message on the queue was empty (data length of zero). At first, I thought I did some wrong. I did the test again a couple of times, sure enough, every time the message was empty.

I looked at MQ Visual Edit’s logfile and it had an IOException. The IOException getMessage method error text was:

Input length = 1

And the IOException getClause method error text was null.

I shock my head because the first character of the JSON sample was a bracket. i.e. {
And the second character was a new-line. The error message made no sense. I ran the code in the Eclipse debugger and everything looked fine. The code extracts the data from the window panel and as it does the MQMessage writeString method, it throws the IOException.

Again more head shaking and thinking WTF!! Why doesn’t MQ like the JSON data. I went through all of the other samples, and not a problem. How can MQ not like a particular JSON sample. More specifically, I can load the same unformatted JSON sample without issue, it was only when I performed the pretty-print formatting that there was an issue.

The formatted JSON sample is 146 lines long, so I started to rip it apart. After several iteration, I found an interesting difference.

Here is a line from the unformatted JSON sample:

"album_information":"<p>Funk-punk party starters !!! performs live at KEXP\u2019s \u201cBean Room\u201d stage during our broadcast at the Capitol Hill Block Party 2010.\u00a0\u00a0\u00a0 <br \/><\/p>",

And here is the same line from the formatted JSON sample:

"album_information": "<p>Funk-punk party starters !!! performs live at KEXP’s “Bean Room” stage during our broadcast at the Capitol Hill Block Party 2010.    <br /></p>",

MQ Visual Edit uses the GSON library to perform the pretty-print formatting. If you look near the end of the unformatted line you will see 3 \u00a0 characters. For some strange reason, the GSON library converts \u00a0 to the real character of 0xA0 and MQMessage writeString method throws an exception when it encounters those characters. Weird, very weird.

So I put together a little hack that fixes the issue:

boolean flag = false;
for (int i=0; i < panelData.length(); i++)
{
   if (Character.UnicodeBlock.of(panelData.charAt(i)) != Character.UnicodeBlock.BASIC_LATIN)
   {
      flag = true;
      break;
   }
}

if (flag)  //found weird character
   mqMsg.write(panelData.getBytes());
else
   mqMsg.writeString(panelData);

If anyone has a better way of doing it then I’m all ears. 🙂

Now, I did do some searches on turning off Unicode conversion in the GSON library but all I found was for disabling HTML conversion. This is what my GSON code looks like for pretty-print formatting:

Gson gson = new GsonBuilder().disableHtmlEscaping().setPrettyPrinting().create();
JsonParser jp = new JsonParser();
JsonElement je = jp.parse(origMsgData);  // original message data

panel.setText(gson.toJson(je));

Again, if anyone has a better way of doing it then I’m all ears. 🙂

Regards,
Roger Lacroix
Capitalware Inc.