ds_common_logger_py_lib.formatter¶

File: formatter.py Region: ds_common_logger_py_lib

Description¶

Defines a custom logging formatter that appends non-standard LogRecord fields (i.e. values passed via the extra=... argument) to the formatted log output.

Example

>>> import logging
>>> from ds_common_logger_py_lib.formatter import ExtraFieldsFormatter
>>>
>>> formatter = ExtraFieldsFormatter()
>>> handler = logging.StreamHandler()
>>> handler.setFormatter(formatter)
>>>
>>> logger = logging.getLogger("test")
>>> logger.addHandler(handler)
>>> logger.setLevel(logging.INFO)
>>>
>>> logger.info("Test message", extra={"user_id": 123})
[2024-01-15T10:30:45][test][INFO][formatter.py:26]: Test message | extra: {"user_id": 123}

Classes¶

`LoggerFilter`	Filter to only allow logs from specified logger name prefixes.
`ExtraFieldsFormatter`	Custom formatter that includes extra fields in log output.

Module Contents¶

class ds_common_logger_py_lib.formatter.LoggerFilter(allowed_prefixes: set[str] | None = None, managed_loggers: set[str] | None = None)[source]¶

Bases: logging.Filter

Filter to only allow logs from specified logger name prefixes.

Simple whitelist approach: if a logger name starts with (or equals) any allowed prefix, it’s allowed. Everything else is filtered out.

Loggers in the managed_loggers set are automatically allowed, regardless of allowed_prefixes.

allowed_prefixes = None¶

managed_loggers¶

filter(record: logging.LogRecord) → bool[source]¶

Filter log records - return True to allow, False to exclude.

Loggers in managed_loggers are automatically allowed. If allowed_prefixes is None or empty, only managed loggers are allowed. Otherwise, managed loggers plus logs whose name starts with (or equals) an allowed prefix are allowed.

Parameters:: record – The log record to filter.
Returns:: True if the log should be shown, False if it should be excluded.

class ds_common_logger_py_lib.formatter.ExtraFieldsFormatter(fmt: str | None = None, datefmt: str | None = None, template_vars: dict[str, str] | None = None)[source]¶

Bases: logging.Formatter

Custom formatter that includes extra fields in log output.

This formatter extends the standard logging.Formatter to properly handle extra fields passed via the extra parameter in logging calls. Extra fields are serialized as JSON and appended to the log message.

The formatter also supports template variables in format strings, such as {prefix}, which are replaced at runtime with values provided by the caller.

Parameters:

fmt – Format string for the log message. May contain template variables like {prefix} that will be replaced at runtime.
datefmt – Date format string.
template_vars – Optional dictionary of template variables to replace in the format string (e.g., {“prefix”: “MyApp”}).

Returns:

Formatter instance that handles extra fields and template variables.

Example

>>> import logging
>>> formatter = ExtraFieldsFormatter(
...     fmt="[%(asctime)s][{prefix}][%(name)s]: %(message)s",
...     template_vars={"prefix": "MyApp"}
... )
>>> handler = logging.StreamHandler()
>>> handler.setFormatter(formatter)
>>> logger = logging.getLogger("test")
>>> logger.addHandler(handler)
>>> logger.setLevel(logging.INFO)
>>> logger.info("Test message", extra={"user_id": 123})
[2024-01-15T10:30:45][MyApp][test]: Test message | extra: {"user_id": 123}

_STANDARD_ATTRS: ClassVar[set[str]]¶

template_vars¶

_resolve_template(fmt: str) → str[source]¶

Resolve template variables in the format string. Remove empty bracket pairs: [] and optional space after it This handles patterns like “[] “ or “[]” at the start/middle/end of format string

Parameters:: fmt – Format string with potential template variables.
Returns:: Format string with template variables replaced.

format(record: logging.LogRecord) → str[source]¶

Format the log record, including extra fields.

Parameters:: record – The LogRecord instance to format.
Returns:: Formatted log message string with extra fields appended if present.

Example

>>> import logging
>>> formatter = ExtraFieldsFormatter()
>>> handler = logging.StreamHandler()
>>> handler.setFormatter(formatter)
>>> logger = logging.getLogger("test")
>>> logger.addHandler(handler)
>>> logger.setLevel(logging.INFO)
>>> logger.info("Test", extra={"user_id": 123})
[2024-01-15T10:30:45][test][INFO][test.py:1]: Test | extra: {"user_id": 123}