While doing some refactoring I noticed a package that I was only using for an auth backend. I moved the module to the main application package, ran the test-suite and then deployed the changes to the live server.
I then noticed that all responses from the server were error 500s. The trace included in the email alert referred to the package name of the old auth backend. Did I miss something? Did the deploy fail?
I SSHed into the server and a quick check suggested that the deployment was successful, and that there were no references to the old auth backend.
The problem was that the auth backend my user had authenticated with was stored in the session. Once I had cleared the session table, the site was operational again.