kumomta/docs/userguide/configuration/logging.md

Configuring Logging

By default, KumoMTA writes to a (zstd compressed) JSON log format, the details of which can be found on the Logging Reference Page,

Basic Log Configuration

The simplest logging configuration, added to the init event, is as follows:

kumo.configure_local_logs {
  log_dir = '/var/log/kumomta',

  -- We recommend setting this when you're getting started;
  -- this option is discussed in more detail below
  max_segment_duration = '10 seconds',
}

For multiple log files, the configure_local_logs function can be called multiple times with different parameters.

!!!note Logs can also be published as webhooks. See the Publishing Log Events Via Webhooks chapter.

OS Considerations

The log directory should be isolated to its own partition, in order to prevent a full log partition from affecting the overall server. For best performance, the log directory should be on a separate disk from the spool. The log partition should be monitored to ensure that the disk does not fill to 100% capacity.

Compression and Rotation

The server writes logs as a series of zstd compressed files, resulting in high storage efficiency as logs are written, instead of having to write large files to disk and compress them during file rotation.

By default, files are rotated after every 1Gb of uncompressed log data, resulting in files on disk that are approximately 50MB in size. The maximum size is configurable, see the Logging Reference Page.

Logs can be viewed in real time using the tailer utility:

$ /opt/kumomta/sbin/tailer --tail /var/log/kumomta
/var/tmp/kumo-logs/20231013-003826
{"type":"Reception","id":"d69b7572696011eebb51227d27bbd7ab",...}
{"type":"TransientFailure","id":"d69b7572696011eebb51227d27bbd7ab",...}
waiting for more files

Logs will appear as the segment files are flushed, which happens as zstd accumulates enough data to write out a block, or when the max_file_size is reached, or when shutting down the server.

It is possible to configure KumoMTA to rotate logs on a time interval basis, so that you can see log records emerge more quickly when you are first experimenting with kumomta and have very little load:

kumo.configure_local_logs {
  -- ..
  max_segment_duration = '10 seconds',
}

See the Logging Reference Page for more information on this setting.

Logging Message Headers

It's a common practice to encode important per-user or per-campaign information in message headers, or to use a message Subject line as an identifier in reporting. This use-case requires logging the headers, which can be achieved very simply by specifying the desired headers in the configuration:

kumo.configure_local_logs {
  -- This is convenient, but costly! Prefer to capture the
  -- headers into meta and log those instead, as shown in
  -- the example below!
  headers = { 'Subject', 'X-Client-ID' },
}

Our recommendation is to capture the headers into the message metadata to make the system run faster and more efficiently:

kumo.on('init', function()
  kumo.configure_local_logs {
    -- ..
    meta = { 'subject', 'x_client_id' },
  }
end)

kumo.on('smtp_server_message_received', function(msg, conn_meta)
  -- Arrange to log the subject header in the most
  -- efficient way, by capturing it into the message
  -- metadata when we receive the message.
  -- The `msg:import_x_headers` method will capture non-x-header
  -- names when header names are explicitly passed like this:
  msg:import_x_headers { 'subject', 'x-client-id' }
end)

Customizing the log format

If a non-JSON format is needed for the logs, the template option can be used:

{% raw %}

kumo.configure_local_logs {
  -- ..
  per_record = {
    Bounce = {
      -- Instead of logging the json record, evaluate this
      -- template string and log the result.
      template = [[Bounce! id={{ id }}, from={{ sender }} code={{ code }} age={{ timestamp - created }}]],
    },
  },
}

{% endraw %}

The Mini Jinja templating engine, with some supplemental extensions, is used to evaluate logging templates. The syntax and extensions are documented here.

Any key present in the default log format can be used in the templating engine.

Configuring Individual Record Types

Sometimes it is necessary to configure logging on a more granular basis, especially when using custom log formats. KumoMTA supports this using the per_record option:

{% raw %}

kumo.configure_local_logs {
  per_record = {
    Reception = {
      -- use names like "20230306-022811_recv" for reception logs
      suffix = '_recv',
    },

    Delivery = {
      -- put delivery logs in a different directory
      log_dir = '/var/log/kumo/delivery',
    },

    TransientFailure = {
      -- Don't log transient failures
      enable = false,
    },

    Bounce = {
      -- Instead of logging the json record, evaluate this
      -- template string and log the result.
      template = [[Bounce! id={{ id }}, from={{ sender }} code={{ code }} age={{ timestamp - created }}]],
    },

    -- For any record type not explicitly listed, apply these settings.
    -- This effectively turns off all other log records
    Any = {
      enable = false,
    },
  },
}

{% endraw %}

This can be used to override paths, disable logs, or customize the format of specific event types.