We've been hoping that mkdocs-material will ship the much anticipated
search enhancements for some time, but it's time to recognize that
we need to do something to improve the search results with how
things work right now.
This is a big commit that changes the titles of the various pages
from the code-annotated synopsis to just the name of the function.
This makes it much easier now to match things like `kumo.reject`
directly, but `reject` remains awkward to find.
I think this is the best that we can do at this time.
A few functions have been annotated with the `status: deprecated` to
show as deprecated in the toc/nav (shows with a little trash can next
to the name).
TL;DR: you can easily halve your system performance by logging headers
vs. logging meta.
This is one of those things that is easy to overlook or forget,
but: whenever you need to operate on the message data, rather
than its metadata, the aggregate cost is high.
In this case, we were recently troubleshooting a system where
the CPU was bogged down and we traced it to the logging configuration: a
number of message headers were being logged in a configuration that
made heavy use of throttles and limits in its traffic shaping, and
thus had a large number of Delayed and TransientFailure events being
written to the logs.
When logging headers, each one of those events requires loading
the message from the spool and parsing out the headers. When the
average message size is ~100KB this imposes a notable overhead
on the CPU and IO utilization of the system.
What we recommend instead of logging headers directly is capturing
the information that you want to log into the message metadata
at the time that the message is received.
The message meta is usually already loaded, but is also typically
much smaller and easier to decode than the full message content
in the cases where it is not loaded.
As a result, it is much cheaper to log meta than to log headers.
This commit adds some warnings and cross links to help folks
be aware of this, and to generally navigate related meta and logging
topics more easily via tags.
Augments our queue name format to be
`campaign:tenant@domain!routing_domain`.
The routing_domain is optional. If the routing_domain is not set, its
effective value is that of the recipient domain.
You can `msg:set_meta('routing_domain', 'bar.com')` to set the
routing_domain for a message, so if the original recipient was
`user@foo.com`, that would cause the computed queue name for it to be
`foo.com!bar.com`.
The routing_domain is used when deciding on the ready_queue name
and destination MXs, so continuing our example, instead of resolving
`foo.com` MX records we'd resolve `bar.com` and deliver to that site.
The `get_egress_path_config` event `domain` parameter is redefined to be
the effective `routing_domain`.
The `get_queue_config` event `domain` parameter is the regular recipient
domain. The `routing_domain` is not currently made available to
`get_queue_config`. If/when we expose it, it will likely be via a
queue name object instead of adding an additional parameter. That would
be a breaking change.
The consequence of not exposing this parameter is that per-message
routing scenarios for the same domain (but different routing domains)
cannot vary the scheduled queue parmeters (eg: retry intervals). Even
though they would have separate scheduled queue instances, those
instances would have the same scheduled queue parameters. If you need
to be able to do that, then explicitly setting the domain portion of the
queue name would be a way to do that: `msg:set_meta('queue',
'foo.com-via-bar.com!bar.com')`. `get_queue_config` would then be
called with `domain='foo.com-via-bar.com'` and your policy could then
respond accordingly.
We don't set received_via for http because axum/hyper make it a
bit weird to access that information and it will require some
effort to figure out how to get that without breaking the
existing proxy-aware client IP determination.
Wez Furlong
Mike Hillyer