Pages

Tuesday, August 27, 2013

Loops in Message Routing/Graphs

This topic seems to be so beat down even to me that it was somewhat surprising to actually come across a real life example where a major corporation seemed to have not paid attention to it. Even more surprising if this happens to a company which's whole business is about routing: USPS.

I recently ordered something online that was supposed to be delivered by USPS, which was tasked to just complete "the last mile". Now I made a mistake and had the package shipped to my office but accidentally had typed in the zip code of my home address (which are only about 7 miles apart from each other). Looking at the online tracking app, I could not believe what is happening:


Location Date Activity
SAN FRANCISCO, CA, United States 08/24/2013 4:52 A.M. Received by the local post office
SAN FRANCISCO, CA, United States 08/23/2013 2:51 P.M. Package processed by Post Office
SAN FRANCISCO, CA, United States 08/20/2013 9:03 A.M. Package routed to wrong local post office. Package will be transferred to correct post office for delivery.
08/20/2013 5:16 A.M. Received by the local post office
08/20/2013 1:34 A.M. Package processed by Post Office
SAN FRANCISCO, CA, United States 08/19/2013 6:38 P.M. Package processed by Post Office
08/19/2013 10:52 A.M. Package routed to wrong local post office. Package will be transferred to correct post office for delivery.
08/19/2013 6:50 A.M. Received by the local post office
SAN FRANCISCO, CA, United States 08/17/2013 8:54 P.M. Package processed by Post Office
08/17/2013 11:28 A.M. Local post office attempted delivery, package is undeliverable as addressed. Contact sender for further assistance.
08/17/2013 5:30 A.M. Received by the local post office
SAN FRANCISCO, CA, United States 08/16/2013 4:47 P.M. Package processed by Post Office
08/16/2013 10:39 A.M. Package routed to wrong local post office. Package will be transferred to correct post office for delivery.
08/16/2013 6:28 A.M. Received by the local post office
SAN FRANCISCO, CA, United States 08/15/2013 4:05 P.M. Package processed by Post Office
SAN FRANCISCO, CA, United States 08/14/2013 11:02 A.M. Package routed to wrong local post office. Package will be transferred to correct post office for delivery.
08/14/2013 8:28 A.M. Package out for local post office delivery
08/14/2013 8:18 A.M. Package sorted by local post office
08/14/2013 5:22 A.M. Received by the local post office
SAN FRANCISCO, CA, United States 08/13/2013 10:49 A.M. Electronic Shipment Information Received for Package by Post Office


It seems evident no human is involved in the process, at least not on one end of the equation. I can only guess (missing some details of the actual route the package is taking) that the package got routed to the post office responsible for the zip code. There it was determined that the address is outside its area and was being sent (hopefully) to the post office serving the street address mentioned on the package. That post office noticed the wrong zip code and sent it back to the first one. Now the shipment is stuck in an infinite loop going back and force between the 2 locations.

Off the top of my head I can see solutions to deal with this type of routing/traversal problem:

1. Add meta data to the message implying the nodes that have been passed. Simple implementation could be adding counters for all nodes that have been reached. On a certain threshold, break out of the automated processing and require manual interaction. In the above case adding PostOfficeA:1, PostOfficeB:1 to the message would do the trick. If PostOfficeA receives a message (package) it has already seen before continuing the standard process will most likely fail again so trying it again will probably not work.

2. If it can be determined that the destination is wrong, the destination is actually part of the message itself and the correct destination can be determined, update the destination on the message (fix address on the package) instead of sending the message unchanged to the new destination. If the receiver does some kind of a sanity check first (as obviously it is done case of my shipment) you possibly end up in an infinite loop.

Without any kind of exception handling the system is open to denial of service attacks as injecting a critical number of faulty messages could keep the system so busy that it can't do its job (I wonder if sending enough packages with this kind of wrong address would actually bring down USPS :-D).

Learnings:
  • When you allow messages/graph traversals to pass through a set of nodes randomly, make sure you can't get stuck in an infinite loops and consider edge cases and exceptions.
Update: Literally when I finished this post my package arrived. Looks like USPS does track repeated occurrences of a package at the same node. Just the number of times a package is allowed to travel through the same node is surprisingly high.

1 comment:

  1. A package I mailed over three weeks ago is still looping. I can't tell (unless the package is returned to me) whether I wrote the wrong ZIP code on it or whether the clerk who prepared the postage label typed it wrong. You can track the fun here. I suspect the USPS equivalent of IP time-to-live is to let a package fall apart from wear and tear.

    ReplyDelete