Usage

Since you’re using Elasticsearch to store log data, you probably don’t want to just store messages as strings. Good! Lumberjack supports (and encourages) logging dicts to be stored directly in Elasticsearch as JSON:

a_logger.info({'message': 'User did something.',
               'username': 'rory',
               'uid': 23})

Note: the ‘type’ of the document in Elasticsearch is determined by the logger’s name. For a_logger that’s __name__. This means that it’s a good idea to register several sub-loggers, one for each type of event being logged, and then attach the handler to the parent for all of them.

Schemas

Lumberjack also abstracts away creating mappings and templates for your data.

Before you log any events, it’s a good idea to tell Elasticsearch what kind of data to expect in them. To do this you use the register_schema method:

Schemas correspond closely with mappings in Elasticsearch, but are processed by Lumberjack to include some sensible defaults. An example schema might be:

{
    '_source': True,
    'properties': {
        'ip_address': {'type': 'ip'},
        'user_id': {'type': 'long'},
        'username': {'type': 'string'}
    }
}

This method should be called once per schema to register; it’s probably a good idea to call it at the same time as attaching your handler to a logging.Logger object.

A complete example

So now we’ve covered an introduction, basic usage and how to use schemas, let’s put it all together.

Suppose we are writing a web app, and we want to log logins and searches:

In the general initialisation we put:

from logging import getLogger, INFO
from lumberjack import Lumberjack

lj = Lumberjack(hosts=[{'host': 'localhost', 'port': 9200}])

# Register a schema for logins
lj.register_schema('appname.login',
    {
        'properties': {
            'ip_address': {'type': 'ip'},
            'username': {'type': 'string'},
            'successful': {'type': 'boolean'},
            'attempt_number': {'type': 'short'}
        }
    })

# Register a schema for searches
lj.register_schema('appname.search',
    {
        'properties': {
            'ip_address': {'type': 'ip'},
            'query': {
                'type': 'string',
                'index': 'analyzed'
            }
        }
    })

logger = getLogger('appname')
handler = lj.get_handler()

logger.addHandler(handler)
logger.setLevel(INFO)

In the login handling function we put:

from logging import getLogger

logger = getLogger('appname.login')

log_event = {'ip_address': some_ip_address_variable,
             'username': the_username,
             'successful': login_ok,
             'attempt_number': attempt_number}
logger.info(log_event)

And in the search handling function we put:

from logging import getLogger

logger = getLogger('appname.search')

log_event = {'ip_address': some_ip_address_variable,
             'query': query_string}
logger.info(log_event)

Now we’ve integrated elasticsearch logging into our web application.

Next steps

At the minimum, you should read the index prefixes bit of the Configuration section.