zeitbach.com

Urllib3 retries requests with expired auth tokens

If you are using urllib3’s Retry utility1 on requests that authenticate with expiring tokens, you need to handle authentication errors separately, because the request headers are fixed between retries and once your auth token has expired, each retry will fail.

For example, your client is caching a token that is valid for two hours. Before each request you make sure that the token has not expired yet, otherwise you get yourself a new one. Let’s assume your token is still valid for 2 minutes, so you make a request. Now there is some error for which you have configured a retry with exponential backoff. On the second or third attempt, your token has expired, but the retry keeps using the same Authorization header and returns a 401 (Unauthorized). If you have 401 on your status_forcelist, you’ll just keep repeating the request with the same expired token until a MaxRetryError is raised. So you need to handle this error by yourself, or you need to extend urllib3’s retry logic.

There are two open issues on the urllib3 repo that address this problem, but the maintainers are worried that the Retry class becomes more bloated than it already is and would prefer a partial rewrite, including a change of the public API, which is unlikely to happen any time soon.


  1. In case you’re using the requests lib, they use urllib3’s retries!↩︎