US Postal Service (USPS) 2011 Failure – What We Can Learn From It
It’s only a few days into the new year, and the US Postal Service (USPS) has managed to cause some major issues for many ecommerce merchants. What looked like a simple, straightforward update on January 2nd turned into a big mess for shopping carts that relied on real-time rates from USPS.
As they like to type on Twitter:
#fail (translated: utter and complete failure)
What did USPS break?
On January 2nd USPS was updating its rates and adding some new features to their API for real-time rates and labels. Their email that went out in mid December was quite detailed, and the changes weren’t supposed to impact shipping options in rate requests. Unfortunately, this was not the case.
To explain what happened, I thought it would be easier if I did my best to demonstrate this in the form of a Dilbert-esque description:
Boss: “Hey IT guy! I noticed the shipping services we’re returning for international rates do not have our registered trademark included in the text. It’s very important to have this. Fix it.”
IT guy: “Sir, doing this will cause some problems for the XML respo…” (abruptly cut-off)
Boss: “I don’t want to hear about techno mumbo jumbo. You know this is my pet peeve and what I spend hours each week doing. The trademark symbol has to be in all the responses now. Oh, and make sure it’s in superscript font. That one really gets my goat.”
IT guy: (dejected) “Superscript will be even more of a proble… nevermind.”
So what was originally passed back to applications:
<strong><span style="color: #008000;"><SvcDescription>Priority Mail International</SvcDescription></span></strong>
Now looks like this:
<strong><span style="color: #008000;"><SvcDescription>Priority Mail&lt;sup&gt;&amp;reg;&lt;/sup&gt; International</SvcDescription></span></strong>
In other words, a jumbled mess that carts (like ShopSite) do not like.
USPS as quiet as a mouse
On Sunday afternoon, Twitter, forums and emails starting lighting up the internet about problems with USPS. Lots of people were talking, except for the USPS. No update on their website, no Twitter or Facebook presence to calm the masses, nothing. Just ecommerce merchants and providers scrambling to figure out what actually changed, and then how to patch the software to work with these un-announced changes.
USPS did have a phone number and email support, but both were not very responsive, and quickly overwhelmed. Conflicting reports were being issued by USPS to complicate matters.
* NOTE: We have now patched all hosted ShopSite version 10 SP2 ecommerce stores to correct this problem.
A few things to take away
- 1. The US Postal Service is a dinosaur
Slow, out of touch, and becoming extinct. ok, it’s a bit unfair, but I’m venting.
- 2. Disclose *all* changes
If there is a remote chance a change you will make may affect your customers, then make sure you publish this fact in your email, newsletter, changelog, etc…
- 3. Have a public status somewhere (preferably more than one)
Whether that’s your website, mailing list, Twitter, and/or Facebook, make sure you can get the word out to your customers when problems arise.
- 4. Have resources ready when major updates occur
The USPS API support department was overwhelmed by support requests. They did not put enough resources in place to deal with the major changes they implemented. Don’t make this mistake, as you cannot absorb these type of disasters like the US government can.
It’s been a “fun” week thanks to the USPS. All of this could have been avoided if they just told people about this text change *before* it happened. Everyone here at LexiConn worked hard to patch our own modules and get the ShopSite patches applied to affected stores. At least we learned our emergency response system works as it should.
Were you affected by this USPS snafu? Feel free to vent share in the comments below.