Welcome to

Plazko IO

Home / Technology / Paypal Express API Control Character Bug (failed due to Paypal Express Payment failed (10004): Invalid character in parameter. Invalid characters include 0x0, 0x1, 0x3, 0x5, 0x7, 0x8, 0xb, 0x1a, 0x1b for example)

Paypal Express API Control Character Bug (failed due to Paypal Express Payment failed (10004): Invalid character in parameter. Invalid characters include 0x0, 0x1, 0x3, 0x5, 0x7, 0x8, 0xb, 0x1a, 0x1b for example)

We recently had an issue come up at Plazko.com that was one of those issues that frustrates you because you don’t even really know the best way to diagnose the cause.  When the problem first occurred, the customer managed to fix it themselves and since no one else reported it also, I put the issue to rest.  Then a few weeks later the problem occurred again.  Since the problem affects a customer’s ability to place an order, it became crucial to resolve the issue.

The Problem

The customer is unable to checkout and is given only a vague message about the payment failing.  There is no known issue with the payment method.

What We Saw Internally

We don’t yet have a ton of UI’s that consolidate and display useful troubleshooting information.  I have a few screens that show me what a customer has in their cart and general events that have occurred with the transaction.  But, generally speaking, if I need to get to the dirty details of something I need to look at disparate information in database tables and server logs and form a conclusion based on them.  In this case though, the problem was seemingly easy to determine.  I could see in my panel that the payment was failing and Paypal was kind enough to tell me what the problem was.

Paypal Express Payment failed (10004): Invalid character in parameter. Invalid characters include 0x0, 0x1, 0x3, 0x5, 0x7, 0x8, 0xb, 0x1a, 0x1b for example

Finding the Invalid Character

That seemed simple enough to solve.  I just had to determine where the funky characters were coming from and then eliminate them.  Since I knew none of our data had any weird control characters or non-English Unicode in it, I assumed that the issue must be with some customer entered data such as the shipping or billing address.  Although I did recall from the first time this problem occurred that there was nothing noticeably weird about those fields, with this latest customer there was indeed some slightly unusual stuff in the address.  It was a New Zealand address which had forward slashes and apostrophe’s in it.  So I though, “oh, this must be the problem.  We must have just missed it the first time.”

I jumped on my dev server and used the customer’s address, verbatim, to place a test order against PayPal’s sandbox.  This was one of those times where you hoped the error would occur, because that would indicate definitive proof of the cause and therefor provide a solution.  Unfortunately, the order succeeded without any issues.  So, we were back to trying to find the bogus data.

At this moment I realized that we recently started storing the raw data (minus any private info) that we send to Paypal for the purpose of debugging issues just like this.  I excitedly went to look at that log, so I could see plainly exactly what we sent to Paypal, and what it didn’t like about it.  I was dismayed when I got to the log only to realize that I only logged this information for transactions that did not use Paypal Express, as this transaction did.  So, I had no log of the raw data I sent to Paypal.

We recalled that the first customer with this issue managed to fix it themselves, and that they had modified the contents of their cart before the problem stopped occurring.  So, based on that, our next hypothesis was that somehow the bad data was being transmitted as part of a product’s info, despite my confidence that that data did not actually contain those invalid characters.  Visually, the product information (title, description, PSID) looked fine; all readable ASCII.  I was a bit frustrated.

I wrote a SQL query that would pull up the product information for the product’s that had been ordered.  I ran the query in MySQL Workbench and the results did not reveal anything.  However, I was cognoscente of the fact that I wouldn’t be seeing non-printable characters.  I had hoped that MySQL Workbench might give me some sort of clue about the non-standard text.  It did not.  So, I copied the results by right-clicking on the data table and choosing “Copy Row (With Names)” and pasted them into Notepad++.  In Notepad++ I enabled the symbols view by choosing View > Show Symbol > Show All Characters from the menu.  With this option on, Notepad++ elegantly shows little icons in place of any non-printable characters so you can see everything that is in the document.  I was really happy and excited when I saw the Vertical Tab Icon in Notepad++ character in the description of several products!

Notepad++ showing vertical tabs in Plazko product data

Product data with offending descriptions displayed in Notepad++ with the Show all Symbols feature enabled

The Solution

I now knew where the bogus data lived.  I tried to figure out how it got there.  I Googled around a bit and did not find any reasonable way that this character could have been typed in from our US 104 ASCII keyboards.  So we concluded that the characters probably originated from our original product import in which we imported all of our old Excel spreadsheet product data into our modern database that we use for Plazko.com.  Whatever the case, I was content to just fix the problem and move on.  I know we won’t be doing any more Excel imports as that was a one-time migration task.  So, I simply removed the vertical tabs from the product descriptions using this simple SQL statement.

MySql Workbench Vertical Tab Replacement

I used an image to demonstrate how Notepad++ neatly shows a graphic for the Vertical Tab character. It would be nice if they would also show these characters in query results.

Now, none of our products have vertical tabs in the description, and the orders can be successfully sent to PayPal’s API.

What Would Have Been Better

This would have all gone more smoothly if a couple of things had occurred:

  • Paypal should offer a more descriptive error message.  Something like “Invalid character 0xB (Vertical Tab) detected in one or more of the product descriptions that were passed”
  • MySQL Workbench should offer an option similar to Notepadd++’s Show All Symbols feature that would allow me to see non-printable characters in query results

 

If you recently made an order with any of the affected products and the order failed, this was likely the issue that you faced.  The issue is now fully resolved.

 

 

Sharing is Caring (tm)