python - Log all errors to console or file on Django site

Question

How can I get Django 1.0 to write all errors to the console or a log file when running runserver in debug mode?

I've tried using a middleware class with process_exception function as described in the accepted answer to this question:

How do you log server errors on django sites

The process_exception function is called for some exceptions (eg: assert(False) in views.py) but process_exception is not getting called for other errors like ImportErrors (eg: import thisclassdoesnotexist in urs.py). I'm new to Django/Python. Is this because of some distinction between run-time and compile-time errors? But then I would expect runserver to complain if it was a compile-time error and it doesn't.

I've watched Simon Willison's fantastic presentation on Django debugging (http://simonwillison.net/2008/May/22/debugging/) but I didn't see an option that would work well for me.

In case it's relevant, I'm writing a Facebook app and Facebook masks HTTP 500 errors with their own message rather than showing Django's awesomely informative 500 page. So I need a way for all types of errors to be written to the console or file.

Edit: I guess my expectation is that if Django can return a 500 error page with lots of detail when I have a bad import (ImportError) in urls.py, it should be able to write the same detail to the console or a file without having to add any additional exception handling to the code. I've never seen exception handling around import statements.

Thanks, Jeff

Answer 1

It's a bit extreme, but for debugging purposes, you can turn on the DEBUG_PROPAGATE_EXCEPTIONS setting. This will allow you to set up your own error handling. The easiest way to set up said error handling would be to override sys.excepthook. This will terminate your application, but it will work. There may be things you can do to make this not kill your app, but this will depend on what platform you're deploying this for. At any rate, never use this in production!

For production, you're pretty much going to have to have extensive error handling in place. One technique I've used is something like this:

>>> def log_error(func):
...     def _call_func(*args, **argd):
...         try:
...             func(*args, **argd)
...         except:
...             print "error" #substitute your own error handling
...     return _call_func
...
>>> @log_error
... def foo(a):
...     raise AttributeError
...
>>> foo(1)
error

If you use log_error as a decorator on your view, it will automatically handle whatever errors happened within it.

The process_exception function is called for some exceptions (eg: assert(False) in views.py) but process_exception is not getting called for other errors like ImportErrors (eg: import thisclassdoesnotexist in urs.py). I'm new to Django/Python. Is this because of some distinction between run-time and compile-time errors?

In Python, all errors are run-time errors. The reason why this is causing problems is because these errors occur immediately when the module is imported before your view is ever called. The first method I posted will catch errors like these for debugging. You might be able to figure something out for production, but I'd argue that you have worse problems if you're getting ImportErrors in a production app (and you're not doing any dynamic importing).

A tool like pylint can help you eliminate these kinds of problems though.