Here are a few hints how to get the best performance out of structlog in production:
Use structlog’s native BoundLogger (created using
structlog.make_filtering_bound_logger()) if you want to use level-based filtering.
return Noneis hard to beat.
Avoid (frequently) calling log methods on loggers you get back from
structlog.wrap_logger(). Since those functions are usually called in module scope and thus before you are able to configure them, they return a proxy object that assembles the correct logger on demand.
Create a local logger if you expect to log frequently without binding:
logger = structlog.get_logger() def f(): log = logger.bind() for i in range(1000000000): log.info("iterated", i=i)
Since global scope lookups are expensive in Python, it’s generally a good idea to copy frequently-used symbols into local scope.
Set the cache_logger_on_first_use option to
Trueso the aforementioned on-demand loggers will be assembled only once and cached for future uses:
This has two drawbacks:
Avoid sending your log entries through the standard library if you can: its dynamic nature and flexibility make it a major bottleneck. Instead use
structlog.WriteLoggerFactoryor – if your serializer returns bytes (e.g. orjson) –
You can still configure
loggingfor packages that you don’t control, but avoid it for your own log entries.
Be conscious about whether and how you use structlog’s asyncio support. While it’s true that moving log processing into separate threads prevents your application from hanging, it also comes with a performance cost.
Decide judiciously whether or not you’re willing to pay that price. If your processor chain has a good and predictable performance without external dependencies (as it should), it might not be worth it.
Here’s an example for a production-ready non-asyncio structlog configuration that’s as fast as it gets:
import logging import structlog structlog.configure( cache_logger_on_first_use=True, wrapper_class=structlog.make_filtering_bound_logger(logging.INFO), processors=[ structlog.contextvars.merge_contextvars, structlog.processors.add_log_level, structlog.processors.format_exc_info, structlog.processors.TimeStamper(fmt="iso", utc=True), structlog.processors.JSONRenderer(serializer=orjson.dumps), ], logger_factory=structlog.BytesLoggerFactory(), )
It has the following properties:
Caches all loggers on first use.
Filters all log entries below the
infolog level very efficiently. The
debugmethod literally consists of
Supports Context Variables (thread-local contexts).
Adds the log level name.
Adds an ISO 8601 timestamp under the
timestampkey in the UTC timezone.
Renders the log entries as JSON using orjson which is faster than plain logging in
structlog.BytesLoggerFactorybecause orjson returns bytes. That saves encoding ping-pong.
Therefore a log entry might look like this: