python - Django logging on Heroku

Question

I know this question was already asked several times, but I just can't get it to work. I already spent half a day trying dozens of combinations, and again now and it is still not working.

In my code, I am logging at several parts, like within a try-except or to log some infos from management commands. I'm doing just very normal stuff, that is working on several local installs and on some Nginx servers.

A python file like this one :

import logging
logger = logging.getLogger(__name__)
logger.info('some important infos')

With following as minimal settings.py (I tried without stream indication, without the loggers specified, with named loggers, almost all possible combinations and I also tried much more complex ones)

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'stream': sys.stdout
        }
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'propagate': True,
            'level': 'INFO',
        },
        '': {
            'handlers': ['console'],
            'level': 'INFO',
        }
    }
}

Then also simply tested from the shell heroku run python

import logging
level = logging.INFO
handler = logging.StreamHandler()
handler.setLevel(level)
handler.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))
logger = logging.getLogger('info')
logger.addHandler(handler)
logger.setLevel(level) #even if not required...
logger.info('logging test')

The last one may show up as "print" statement in the console, but neither here nor from the command or the server, nothing never shows up in heroku logs....

EDIT: actually I have some entries showing up, application logs like following, just not mine:

2013-09-20T15:00:16.405492+00:00 heroku[run.2036]: Process exited with status 0
2013-09-20T15:00:17+00:00 app[heroku-postgres]: source=HEROKU_POSTGRESQL_OLIVE sample[...]
2013-09-20T14:59:47.403049+00:00 heroku[router]: at=info method=GET path=/
2013-09-20T14:59:16.304397+00:00 heroku[web.1]: source=web.1 dyno=heroku [...]

I also tried to look for entries using several addons. I had for a moment at the beginning newrelic, which I then also deactivated from the WSGI start-up. I don't remember if at that time it worked well, the free test period of newrelic is rather short.

Well, I don't know what I could try else... Thanks for any hint

Answer 1

Logging on Heroku from Django can be tricky at first, but it's actually not that horrible to get set up.

This following logging definition (goes into your settings file) defined two formatters. The verbose one matches the logging format Heroku itself uses. It also defines two handlers, a null handler (shouldn't need to be used) and a console handler - the console handler is what you want to use with Heroku. The reason for this is that logging on Heroku works by a simple stream logger, logging any output made to stdout/stderr. Lastly, I've defined one logger called testlogger - this part of the logging definition goes like normal for logging definitions.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': ('%(asctime)s [%(process)d] [%(levelname)s] ' +
                       'pathname=%(pathname)s lineno=%(lineno)s ' +
                       'funcname=%(funcName)s %(message)s'),
            'datefmt': '%Y-%m-%d %H:%M:%S'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        }
    },
    'handlers': {
        'null': {
            'level': 'DEBUG',
            'class': 'logging.NullHandler',
        },
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose'
        }
    },
    'loggers': {
        'testlogger': {
            'handlers': ['console'],
            'level': 'INFO',
        }
    }
}

Next up, how to use this. In this simple case, you can do the following in any other file in your Django project, to write to this specific logger we defined (testlogger). Remember that by the logger definition in our settings file, any log message INFO or above will be output.

import logging
logger = logging.getLogger('testlogger')
logger.info('This is a simple log message')