Examples¶
This chapter is intended to give you a taste of realistic usage of structlog.
Flask and Thread Local Data¶
In the simplest case, you bind a unique request ID to every incoming request so you can easily see which log entries belong to which request.
import uuid
import flask
import structlog
from .some_module import some_function
logger = structlog.get_logger()
app = flask.Flask(__name__)
@app.route('/login', methods=['POST', 'GET'])
def some_route():
log = logger.new(
request_id=str(uuid.uuid4()),
)
# do something
# ...
log.info('user logged in', user='test-user')
# gives you:
# event='user logged in' request_id='ffcdc44f-b952-4b5f-95e6-0f1f3a9ee5fd' user='test-user'
# ...
some_function()
# ...
if __name__ == "__main__":
structlog.configure(
processors=[
structlog.processors.KeyValueRenderer(
key_order=['event', 'request_id'],
),
],
context_class=structlog.threadlocal.wrap_dict(dict),
logger_factory=structlog.stdlib.LoggerFactory(),
)
app.run()
some_module.py
from structlog import get_logger
logger = get_logger()
def some_function():
# later then:
logger.error('user did something', something='shot_in_foot')
# gives you:
# event='user did something 'request_id='ffcdc44f-b952-4b5f-95e6-0f1f3a9ee5fd' something='shot_in_foot'
While wrapped loggers are immutable by default, this example demonstrates how to circumvent that using a thread local dict implementation for context data for convenience (hence the requirement for using new() for re-initializing the logger).
Please note that structlog.stdlib.LoggerFactory
is a totally magic-free class that just deduces the name of the caller’s module and does a logging.getLogger(). with it.
It’s used by structlog.get_logger()
to rid you of logging boilerplate in application code.
If you prefer to name your standard library loggers explicitly, a positional argument to get_logger()
gets passed to the factory and used as the name.
Twisted, and Logging Out Objects¶
If you prefer to log less but with more context in each entry, you can bind everything important to your logger and log it out with each log entry.
import sys
import uuid
import structlog
import twisted
from twisted.internet import protocol, reactor
logger = structlog.getLogger()
class Counter(object):
i = 0
def inc(self):
self.i += 1
def __repr__(self):
return str(self.i)
class Echo(protocol.Protocol):
def connectionMade(self):
self._counter = Counter()
self._log = logger.new(
connection_id=str(uuid.uuid4()),
peer=self.transport.getPeer().host,
count=self._counter,
)
def dataReceived(self, data):
self._counter.inc()
log = self._log.bind(data=data)
self.transport.write(data)
log.msg('echoed data!')
if __name__ == "__main__":
structlog.configure(
processors=[structlog.twisted.EventAdapter()],
logger_factory=structlog.twisted.LoggerFactory(),
)
twisted.python.log.startLogging(sys.stderr)
reactor.listenTCP(1234, protocol.Factory.forProtocol(Echo))
reactor.run()
gives you something like:
... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=1 data='123\n' event='echoed data!'
... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=2 data='456\n' event='echoed data!'
... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=3 data='foo\n' event='echoed data!'
... peer='10.10.0.1' connection_id='85234511-...' count=1 data='cba\n' event='echoed data!'
... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=4 data='bar\n' event='echoed data!'
Since Twisted’s logging system is a bit peculiar, structlog ships with an adapter
so it keeps behaving like you’d expect it to behave.
I’d also like to point out the Counter class that doesn’t do anything spectacular but gets bound once per connection to the logger and since its repr is the number itself, it’s logged out correctly for each event. This shows off the strength of keeping a dict of objects for context instead of passing around serialized strings.
Processors¶
Processors are a both simple and powerful feature of structlog.
So you want timestamps as part of the structure of the log entry, censor passwords, filter out log entries below your log level before they even get rendered, and get your output as JSON for convenient parsing? Here you go:
>>> import datetime, logging, sys
>>> from structlog import wrap_logger
>>> from structlog.processors import JSONRenderer
>>> from structlog.stdlib import filter_by_level
>>> logging.basicConfig(stream=sys.stdout, format='%(message)s')
>>> def add_timestamp(_, __, event_dict):
... event_dict['timestamp'] = datetime.datetime.utcnow()
... return event_dict
>>> def censor_password(_, __, event_dict):
... pw = event_dict.get('password')
... if pw:
... event_dict['password'] = '*CENSORED*'
... return event_dict
>>> log = wrap_logger(
... logging.getLogger(__name__),
... processors=[
... filter_by_level,
... add_timestamp,
... censor_password,
... JSONRenderer(indent=1, sort_keys=True)
... ]
... )
>>> log.info('something.filtered')
>>> log.warning('something.not_filtered', password='secret')
{
"event": "something.not_filtered",
"password": "*CENSORED*",
"timestamp": "datetime.datetime(..., ..., ..., ..., ...)"
}
structlog comes with many handy processors build right in – for a list of shipped processors, check out the API documentation.