Configuration

The default config is included at the end of this file for reference.

Lumberjack is configured using a dict of config options passed to the Lumberjack object on instantiation. A safely-modifiable copy of the default config dict can be obtained by calling lumberjack.get_default_config():

from lumberjack import Lumberjack, get_default_config
lj = Lumberjack(hosts=[...], config=get_default_config())

With the exception of index_prefix, these defaults should be sensible for production. You should change index_prefix to something different for each of your applications.

The config returned is a deep copy of the default one, so it can be modified as you like without breaking subsequent calls to lumberjack.get_default_config():

from lumberjack import Lumberjack, get_default_config

my_config = get_default_config()
my_config['index_prefix'] = 'a-special-prefix-'

lj = Lumberjack(hosts=[...], config=my_config)

The index prefix

This configures the prefix for the created elasticsearch indices. Indices are created with a constant prefix and a time-based suffix, so might be named generic-logging-2014.10 for log entries from an unconfigured Lumberjack instance in October 2014.

The default mapping

This contains the basis for generating mappings in Elasticsearch. Its values are overridden by the values in the schema dict passed to lumberjack.Lumberjack.register_schema(). It contains keys like _source, _all, and _ttl.

Note that special processing happens to the properties key: instead of being overwritten by the schema’s properties key, the fields provided to Elasticsearch are the union of the two, with the schema’s fields taking precedence.

Default properties for types

When a field is given a particular type in the schema, Lumberjack automatically adds some properties to the field. For example, for string type fields, Lumberjack disables analysis on them. (The reason for this is that while analysis is a powerful Elasticsearch feature when dealing with natural language documents, for log data it makes little sense.)

The interval

This is the (maximum) amount of time to wait between flushes of the log event queue to Elasticsearch. It is an integer or floating-point value in seconds.

The maximum queue length

This is the maximum length the queue can grow to before a flush is triggered automatically.

The fallback log file

If Lumberjack experiences an error when indexing to Elasticsearch, it will fall back to dumping JSON to the file given in this variable.

The default config

# -*- coding: utf-8 -*-
#
# This file is part of Lumberjack.
# Copyright 2014 CERN.
#
# Lumberjack is free software: you can redistribute it and/or modify it under
# the terms of the GNU Lesser General Public License as published by the Free
# Software Foundation, either version 3 of the License, or (at your option) any
# later version.
#
# Lumberjack is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
# details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with Lumberjack.  If not, see <http://www.gnu.org/licenses/>.

"""The default configuration for Lumberjack."""

from __future__ import absolute_import
from copy import deepcopy

DEFAULT_CONFIG = {
    'default_mapping': {
        '_source': {'enabled': True},
        '_all': {'enabled': False},
        '_ttl': {'enabled': True},
        'properties': {
            'message': {
                'type': 'string'
            },
            '@timestamp': {
                'type': 'date',
            },
            'level': {
                'type': 'integer'
            }
        }
    },
    'default_type_properties': {
        'string': {
            'index': 'not_analyzed',
            'norms': {'enabled': False}
        },
        'date': {
            'format': 'dateOptionalTime'
        }
    },
    'index_prefix': 'generic-logging-',
    'interval': 30,
    'max_queue_length': None,
    'fallback_log_file': '/tmp/lumberjack_fallback.log'
}


def get_default_config():
    """Get a copy of the default config.

    The copy can be modified in-place without affecting the default config
    itself.

    """
    return deepcopy(DEFAULT_CONFIG)