Important Difference Between urllib and urllib2, Handling of Internal Server Errors

To be honest, I always ignored learning the difference between urllib and urllib2. I always thought that as long as I use urllib2 I'm okay. However, when I helped a friend debug code. I found a difference which I think might trip some newbie Python developers.

My friend used the urllib.urlopen function. At first we assumed that this function would raise an exception when it gets an Internal Server Error. The code uses the exception to automatically retry an http request, because the API server sometimes returns an Internal Server Error when its overloaded. After, noticing that requests were not being retried, I became suspicious that an exception is not raised. True enough, after some test, urllib.urlopen does not raise and exception when it gets an Internal Server Error. Luckily, the urllib2.urlopen function raises and exception when it gets an Internal Server Error. So the solution was to simply use urllib2.