Minor mechanical fixes. Capitalization and punctuation mostly. Also testing defined meta descriptions.

This commit is contained in:
Mike Hillyer
2026-07-09 14:30:41 -04:00
parent 3788f838d3
commit d878f95021
46 changed files with 121 additions and 117 deletions
+4
View File
@@ -1,3 +1,7 @@
---
description: Step-by-step guide to configuring KumoMTA — set up listeners, DKIM signing, traffic shaping, and logging for high-volume sending.
---
# Configuring KumoMTA
The KumoMTA configuration is entirely written in [Lua](https://www.lua.org/home.html). Lua is a powerful embedded scripting language that is easy to read and code, and is very powerful. It is used for custom scripts in Cisco security appliances, Roblox, World of Warcraft, and really awesome MTAs. You can read more about how we leverage Lua [here](../userguide/configuration/concepts.md).
+2 -2
View File
@@ -97,8 +97,8 @@ an appropriate source IP address. This approach results in
a simpler configuration overall and makes it easier to load
balance and scale the deployment up or down.
We support our own SOCKS-5 based proxy implementation as well
as ha-proxy.
KumoMTA provides its own SOCKS5-based proxy implementation, and
also supports HAProxy.
```mermaid
flowchart LR
+3 -3
View File
@@ -8,16 +8,16 @@ KumoMTA supports several methods commonly used for event data aggregating:
KumoMTA can publish log information to HTTP endpoints in the form of webhooks, which can be received and consumed by log processing applications.
For more information see the [webhooks page](../operation/webhooks.md) of the userguide.
For more information see the [webhooks page](../operation/webhooks.md) of the User Guide.
## AMQP
KumoMTA also supports relaying log data via AMQP.
For more information see the [AMQP page](../policy/amqp.md) of the userguide.
For more information see the [AMQP page](../policy/amqp.md) of the User Guide.
## Kafka
KumoMTA supports publishing log events (as well as other messages) via Apache Kafka.
For more information see the [Kafka page](../policy/kafka.md) of the userguide.
For more information see the [Kafka page](../policy/kafka.md) of the User Guide.
+4 -4
View File
@@ -6,7 +6,7 @@ To address these needs, KumoMTA has several features and integrations designed a
## Implementation Approach
The KumoMTA team works from a philosophy of "don't re-invent the wheel". To the degree possible KumoMTA is designed to work with existing solutions rather than implement our own version of existing tools.
The KumoMTA team works from a philosophy of "don't re-invent the wheel". To the degree possible, KumoMTA is designed to work with existing solutions rather than implementing its own versions of established tools.
For example: KumoMTA does not provide a configuration distribution or versioning tool because there are numerous existing options including Git, Puppet, Chef, K8s, etc.
@@ -16,13 +16,13 @@ While many existing users coming from commercial MTAs with per-node licensing te
Vertical scaling with high-performance hardware can see over ten million messages per hour on a single node, where horizontal scaling environments tend to aim for between two to four million messages per hour per node. Those leveraging orchestration solutions such as Kubernetes will likely prefer a horizontal scaling architecture.
## Shared Vs. Node-Specific Configuration
## Shared vs Node-Specific Configuration
While it is possible to configure KumoMTA nodes in a cluster with distinct configurations on a per-node basis, the traffic shaping approach used in KumoMTA is most effective when all nodes share the same IP configuration using proxies.
In this approach the nodes share a common configuration (implemented using the preferred method of the user) with the `Egress_Source` configured to use a proxy for sending. Alternatively Reverse NAT or port forwarding could also be used.
KumoMTA supports both HAPROXY and SOCKS5, see [the Proxy page](../operation/proxy.md) for information on using KumoMTA with outbound proxies.
KumoMTA supports both HAProxy and SOCKS5; see [the Proxy page](../operation/proxy.md) for information on using KumoMTA with outbound proxies.
KumoMTA includes a SOCKS5 proxy implementation, see the [KumoProxy page](../operation/kumo-proxy.md) for more information.
@@ -30,4 +30,4 @@ KumoMTA includes a SOCKS5 proxy implementation, see the [KumoProxy page](../oper
One of the most straightforward ways to manage secrets such as DKIM keys and authentication credentials is using a Vault.
For more information on using Vault for shared secrets see our [Storing Secrets in Hashicorp Vault](../policy/hashicorp_vault.md) page.
For more information on using Vault for shared secrets see the [Storing Secrets in HashiCorp Vault](../policy/hashicorp_vault.md) page.
+1 -1
View File
@@ -27,7 +27,7 @@ The following environment variables influence the Node ID:
* `KUMO_NODE_ID_PATH` - this can be set to an alternative location into which
the nodeid should be stored. If this is not set, the path is assumed to be
`/opt/kumomta/etc/.nodeid`. If the path is not writable for some reason
(eg: permission denied), then a fallback nodeid will be computed.
(e.g., permission denied), then a fallback nodeid will be computed.
## Fallback Node ID
+3 -3
View File
@@ -106,7 +106,7 @@ KPrbKH5ubT9V9pLKawIDAQAB
```
Once the public and private keys have been generated, create a DNS text record
for `<SELECTOR>._domainkey.<DOMAIN>` (IE: `dkim1024._domainkey.example.com`). The
for `<SELECTOR>._domainkey.<DOMAIN>` (e.g., `dkim1024._domainkey.example.com`). The
DNS record contains several DKIM "tag=value" pairs and should be similar
to the record shown below:
@@ -254,6 +254,6 @@ local signer = kumo.dkim.rsa_sha256_signer {
}
```
Where you want to enable dkim signing, simply call that signer in policy.
Where you want to enable DKIM signing, call that signer in policy.
IE: `msg:dkim_sign(signer)`
For example: `msg:dkim_sign(signer)`
+1 -1
View File
@@ -72,7 +72,7 @@ log_hooks:new_json {
-- Configure queue management settings. These are not throttles, but instead
-- control how messages flow through the queues.
-- WARNING: ENSURE THAT WEBHOOKS AND SHAPING ARE SETUP BEFORE THE QUEUE HELPER FOR PROPER OPERATION
-- WARNING: ENSURE THAT WEBHOOKS AND SHAPING ARE SET UP BEFORE THE QUEUE HELPER FOR PROPER OPERATION
-- WARNING: THIS WILL NOT LOAD WITHOUT the queues.toml FILE IN PLACE
-- See https://docs.kumomta.com/userguide/configuration/queuemanagement/
local queue_helper =
+2 -2
View File
@@ -40,7 +40,7 @@ The server writes logs as a series of
resulting in high storage efficiency as logs are written, instead of having to
write large files to disk and compress them during file rotation.
By default, files are rotated after every 1Gb of uncompressed log data,
By default, files are rotated after every 1GB of uncompressed log data,
resulting in files on disk that are approximately 50MB in size. The maximum
size is configurable, see the [Logging Reference
Page](../../reference/kumo/configure_local_logs/max_file_size.md).
@@ -61,7 +61,7 @@ reached, or when shutting down the server.
It is possible to configure KumoMTA to rotate logs on a time interval basis,
so that you can see log records emerge more quickly when you are first
experimenting with kumomta and have very little load:
experimenting with KumoMTA and have very little load:
```lua
kumo.configure_local_logs {
@@ -4,7 +4,7 @@ KumoMTA is designed around the concept of configuration as code, where all confi
Configuration as code offers numerous advantages, including late loading of config for lower memory consumption and minimal reloads and direct data source connectivity to make your KumoMTA instances a well-integrated part of your DevOps environment rather than a black box that requires automated config file updates and reload commands to be issued.
Configuration as code enables advanced use cases such as [storing your DKIM signing keys in HashiCorp Vault](../policy/hashicorp_vault.md) for realtime DKIM signing and checking SMTP Authentication credentials against a live data source.
Configuration as code enables advanced use cases such as [storing your DKIM signing keys in HashiCorp Vault](../policy/hashicorp_vault.md) for real-time DKIM signing and checking SMTP Authentication credentials against a live data source.
While configuration as code provides extreme flexibility and deep integration capabilities, that can come at the cost of complexity. In order to make KumoMTA more accessible for those who are accustomed to a static configuration file and don't need deeper integration, we have developed a set of policy helpers. These helpers are premade Lua policy scripts that implement common use cases by reading formatted TOML and JSON files to configure KumoMTA.
@@ -101,7 +101,7 @@ retry_interval = '17 mins'
There is no throttling configured at the Scheduled Queue level, instead, the
Scheduled Queue is where messages are evaluated when retries are needed,
meaning that at the Scheduled Queue level we configure settings such as the
meaning that at the Scheduled Queue level you configure settings such as the
time between retries and the maximum age of a message.
The settings for retry interval and message age are typically set globally and
@@ -204,7 +204,7 @@ incorrect `X-Tenant` header.
## Throttling The Scheduled Queue
By default KumoMTA moves messages from the Scheduled Queue to the Ready Queue as quickly as there is room available in the Ready Queue, with the rate impacted only by how quickly the Ready queues have available space as limited by traffic shaping throttles or sending reputation.
By default KumoMTA moves messages from the Scheduled Queue to the Ready Queue as quickly as there is room available in the Ready Queue, with the rate impacted only by how quickly the Ready Queues have available space as limited by traffic shaping throttles or sending reputation.
Under certain circumstances it can be beneficial to throttle the flow of messages from the Scheduled Queue to the Ready Queue. Some examples of such use cases include:
@@ -220,9 +220,9 @@ max_age = '24 hours'
max_message_rate = '100/s'
{% endcall %}
Note that the `max_message_rate` option applies on a per-queue basis even when configured for a less specific scope, where a queue is defined as campaign@tenant:domain. This means that if you set the `max_message_rate` option for a given tenant, it does **not** limit the tenant to a given rate, it limits _every created queue for that tenant_ to the specified rate.
Note that the `max_message_rate` option applies on a per-queue basis even when configured for a less specific scope, where a queue is defined as campaign:tenant@domain. This means that if you set the `max_message_rate` option for a given tenant, it does **not** limit the tenant to a given rate, it limits _every created queue for that tenant_ to the specified rate.
To limit the _collective_ set of queues for a given scope use the `overall_max_message_rate` option. This will enforce the limit across all Scheduled queues for the defined scope:
To limit the _collective_ set of queues for a given scope use the `overall_max_message_rate` option. This will enforce the limit across all Scheduled Queues for the defined scope:
{% call toml_data() %}
[tenant.'mytenant']
@@ -230,4 +230,4 @@ max_age = '24 hours'
overall_max_message_rate = '100/s'
{% endcall %}
In this example the listed `mytenant` tenant will be throttled to 100 messages per second across all Scheduled queues, or in other words regardless of campaign or destination domain.
In this example the listed `mytenant` tenant will be throttled to 100 messages per second across all Scheduled Queues, or in other words regardless of campaign or destination domain.
+8 -8
View File
@@ -1,6 +1,6 @@
# Configuring Queue Rollup
By default, KumoMTA will perform automatic "rollup" of the ready queue based on
By default, KumoMTA will perform automatic "rollup" of the Ready Queue based on
the _site name_ that it derives from the MX records for the destination domain,
which means that your shaping rules will automatically apply across sites that
share the same MX hosts with no additional configuration required.
@@ -47,7 +47,7 @@ outlook-com.olc.protection.outlook.com
```
What this means is that, by default, mail sent to outlook and hotmail will
egress through separate ready queues (because the site names are different),
egress through separate Ready Queues (because the site names are different),
and be subject to separate shaping rules and separate limits/throttles, which
may increase the chances of exceeding the sending rate desired by that
destination site.
@@ -72,7 +72,7 @@ end)
```
With this configuration in place mail sent to outlook and hotmail will now
egress through the same ready queues; hotmail will be treated as though it was
egress through the same Ready Queues; hotmail will be treated as though it was
`outlook.com` because its `routing_domain` will be set to `outlook.com`. This
configuration will match ANY destination domains whose MX hosts all end with
`.olc.protection.outlook.com`, not just hotmail and outlook. Your
@@ -80,7 +80,7 @@ configuration will match ANY destination domains whose MX hosts all end with
matched and rerouted by this module.
You will see the routing manifest in the queue names when you look at the metrics
and/or `kcli queue-summary` output; instead of the scheduled queue name being
and/or `kcli queue-summary` output; instead of the Scheduled Queue name being
`hotmail.com` it will appear as `hotmail.com!outlook.com` to indicate that it
will be routed via `outlook.com`.
@@ -103,7 +103,7 @@ Messages in that queue will be relayed to the set of IP addresses from that
first MX host name.
All domains that have the same matching MX hostname suffix and set of IP
addresses will egress through the ready queue and be subject to the same
addresses will egress through the same Ready Queue and be subject to the same
shaping rules.
```lua
@@ -143,7 +143,7 @@ $ dig +short a foo-com.mail.protection.outlook.com.
what will happen is:
* The message will be queued to a _scheduled queue_ named `foo.com!outlook.ip_rollup`
* The message will be queued to a _Scheduled Queue_ named `foo.com!outlook.ip_rollup`
* The ready queue that it will use to egress will be named `mx_list:[104.47.24.36],[104.47.25.36]`.
For some other domain, for example, a hypothetical `bar.com` domain whose
@@ -168,14 +168,14 @@ ready queue and have the shaping from the `outlook.ip_rollup` section of your
There are a number of important caveats with this particular IP rollup approach:
* We will never try anything beyond the highest priority MX for the matching domains
* KumoMTA will never try anything beyond the highest priority MX for the matching domains
* The IP addresses for the destination domains can vary over time. The
scheduled queue name (`foo.com!outlook.ip_rollup`) will remain the same, but
the list of IP addresses in the ready queue name will adjust to match the
evolving DNS. That means that during a transition there may be multiple ready
queues with different names as the IPs rotate. This can reduce the efficacy
of the rollup, but is a necessary function in order to handle the situation
where the destination site has an outage and we need to stop using the old IPs.
where the destination site has an outage and the server needs to stop using the old IPs.
* If the destination site is using round-robin DNS for load balancing purposes,
this approach is not useful as the resulting `mx_list` in the site name
will vary too frequently for there to be a meaningful or useful rolling up
+4 -4
View File
@@ -6,16 +6,16 @@ this is acceptable, but best practices recommend separating mail streams into
their own IP addresses in order to isolate reputation and enable larger
sending volumes than would be possible on a single IP address.
KumoMTA has the concept of pools of IPs. A given scheduled queue can be
KumoMTA has the concept of pools of IPs. A given Scheduled Queue can be
associated with a pool and it will then use *[Weighted Round
Robin](http://kb.linuxvirtualserver.org/wiki/Weighted_Round-Robin_Scheduling)*
(WRR) to distribute sends from that scheduled queue across the IPs contained
within its associated pool. When a scheduled queue is idle for approximately
(WRR) to distribute sends from that Scheduled Queue across the IPs contained
within its associated pool. When a Scheduled Queue is idle for approximately
10 minutes, it will idle out and the round robin state will be reset for the
next send.
!!! note
The _Weighted Round Robin_ implementation in kumomta is considered to be
The _Weighted Round Robin_ implementation in KumoMTA is considered to be
**probabilistic**, achieving the configured distribution only when the rate
of sending is sufficiently high (at least 1 message to a given site every
few minutes), and is scoped per-_scheduled_-queue. There is no whole-machine
+1 -1
View File
@@ -1,6 +1,6 @@
# Credits
While KumoMTA Corporation owns all copyrights in the KumoMTA software and the KumoMTA manual, we wish to thank the following developers, contributors, and supporters who have helped make KumoMTA what it is today.
While Kumo Corp owns all copyrights in the KumoMTA software and the KumoMTA manual, we wish to thank the following developers, contributors, and supporters who have helped make KumoMTA what it is today.
## Developers
+1 -1
View File
@@ -4,7 +4,7 @@ This is the Documentation for the KumoMTA Message Transfer Agent.
## Legal Notices
KumoMTA and this documentation are Copyright 2023 Kumo Corp.
KumoMTA and this documentation are Copyright 2023-2026 Kumo Corp.
Both are licensed under the Apache License, Version 2.0 (the "License");
Full [Copyright](https://github.com/KumoCorp/kumomta/blob/main/COPYRIGHT) and
+2 -2
View File
@@ -45,8 +45,8 @@ kumo.on('smtp_server_message_received', function(msg)
end)
```
When we launch the image, we want to mount our `init.lua` file into the image
and tell it to use it. The default location for this is `/opt/kumomta/etc/policy`:
When you launch the container, mount your `init.lua` file into it so that
kumod can use it. The default location for this is `/opt/kumomta/etc/policy`:
```console
$ sudo docker run --rm -p 2025:25 \
+1 -1
View File
@@ -5,7 +5,7 @@ KumoMTA can be installed in a physical server, public cloud, or private cloud.
* AWS
* Azure
* GCP
* VMWare
* VMware
* bare metal
* IBM Cloud
* Oracle Cloud
+1 -1
View File
@@ -6,7 +6,7 @@ KumoMTA is a performance MTA that will leverage every bit of power you provide.
## RAM and Storage
At an absolute minimum, you will need 4 CPU cores, 4Gb RAM, and 20Gb Storage. KumoMTA makes heavy use of both resources and response time is going to be a factor. For high performance systems you will want to select storage with the fastest IOPS and lowest latency, so local disk is going to be much better than NAS or SAN. Likewise, you can benefit from faster RAM if it is available. In AWS, a t2.medium is adequate for a minimal install. If you are installing a Docker Image, the same guidance applies. See the [Performance chapter](../performance/testing.md) for sample performance test results.
At an absolute minimum, you will need 4 CPU cores, 4GB RAM, and 20GB storage. KumoMTA makes heavy use of both resources and response time is going to be a factor. For high performance systems you will want to select storage with the fastest IOPS and lowest latency, so local disk is going to be much better than NAS or SAN. Likewise, you can benefit from faster RAM if it is available. In AWS, a t2.medium is adequate for a minimal install. If you are installing a Docker Image, the same guidance applies. See the [Performance chapter](../performance/testing.md) for sample performance test results.
## Network Interfaces
+1 -1
View File
@@ -95,7 +95,7 @@ The default configuration in the shaping helper for outgoing SMTP is to enable
advertises it, but only if the certificate is trusted.
Unfortunately, there are a large number of destination sites with poorly
maintained TLS, so many kumomta users choose to deploy with
maintained TLS, so many KumoMTA users choose to deploy with
`OpportunisticInsecure` TLS as a default, which will try to use TLS if
available, but will allow communicating in clear text if there are any issues
trying to establish the connection. That rationale for this choice is that
+2 -2
View File
@@ -236,8 +236,8 @@ instead of running as root.
## Running from your source directory
!!! note
This section is intended for people that are developing kumomta
itself, rather than people that just want to install and use kumomta
This section is intended for people who are developing KumoMTA
itself, rather than people who just want to install and use KumoMTA
This command will bring `kumod` up to date (in case you made changes), and then try to launch it:
+2 -1
View File
@@ -36,7 +36,8 @@ sudo apt-get -y upgrade
* Install basic testing and support tools like firewalld tree telnet git bind (or bind9) bind-utils (or bind9-utils)
Note that installing a caching name server is absolutely critical when you are using a high performance mail engine. Please do yourself a favour and install bind (or some other caching name server) and test it now.
!!! note
Installing a caching name server is absolutely critical when you are using a high performance mail engine. Please do yourself a favor and install bind (or some other caching name server) and test it now.
```bash
sudo apt install bind9 -y
+2 -2
View File
@@ -8,13 +8,13 @@
1. Sign up for EmailElement account at the portal, selecting a tier that works best for you.
[https://portal.emailelement.com/signup](https://portal.emailelement.com/signup)
Be sure to “Confirm Registration” after you sign up from the email you will receive.
2. Log in after you have confirmed your registration, and set-up a Brand and
2. Log in after you have confirmed your registration, and set up a Brand and
Sender Domain. If you need help with any of those items you can find some
documentation on it
[here](https://app.guidde.com/share/playlists/wkMVASZnJ82Af5EL6YmEny?active=0).
3. Go to `Send Integrations` from the navigation menu and select `Create Send Integration`.
4. Select `SMTP` from the `Type of Integration` dropdown, then choose Kumo and proceed.
5. Enter the required fields and save your integration.
6. Click `Run Test` to ensure everything is set-up correctly and youll be ready to start sending!
6. Click `Run Test` to ensure everything is set up correctly and youll be ready to start sending.
![EmailElement UI](../../assets/images/EmailElement.png)
+1 -4
View File
@@ -45,7 +45,4 @@ Follow the setup [instructions
here](../operation/status.md#setting-up-a-grafana-dashboard) to configure the
prometheus feed, and get the sample Grafana dashboard.
If you have done everything right, you should be able to see your data feed in Grafana within seconds.
You should see your data feed in Grafana within seconds.
@@ -4,7 +4,7 @@
This integration makes **Hornetsecurity Email Protection** and the
**Hornetsecurity Filter Engine** available for KumoMTA to scan messages in
real-time.
real time.
Only the "scan" function is implemented for in-line use with KumoMTA.
@@ -14,7 +14,7 @@ If you have not already done so, contact
[Hornetsecurity](https://www.hornetsecurity.com/) for documentation, binary and
license. Configure Hornetsecurity Email Protection as per their documentation.
EG:
For example:
```console
$ sudo dpkg -i hornetsecurity-emailprotection_5.0.0_amd64.deb
@@ -59,7 +59,7 @@ These API functions are not necessary and are not directly supported within Kumo
### hornet:scan
To scan a message, use `result = hornet:scan(hornethost,extraparams,msg)` in any event that can access the full message content. EG: `smtp_server_message_received`
To scan a message, use `result = hornet:scan(hornethost,extraparams,msg)` in any event that can access the full message content, e.g. `smtp_server_message_received`.
Note that `hornet:connect` must be called prior to `hornet:scan`.
```txt
@@ -72,13 +72,14 @@ Inputs:
msg: The KumoMTA message variable
Returns: array containing the result of the scan
IE: "200 OK: {"state":1,"score":250,"verdict":"spam:low","spamcause":"gggr...omh","elapsed":"14ms"}"
e.g. "200 OK: {"state":1,"score":250,"verdict":"spam:low","spamcause":"gggr...omh","elapsed":"14ms"}"
```
If the extra parameter `addheaders` = `true`, then the scan result headers will be added directly to the message before delivery.
For example:
```txt
IE:
X-Hornet-spamcause: gggr...omh
X-Hornet-verdict: malware
X-Hornet-elapsed: 6ms
+1 -1
View File
@@ -3,7 +3,7 @@
This chapter is focused on partner and adjacent systems integrations.
Some of the third-party integrations could involve a separate external fee, while others are entirely FOSS (Free and Open Source Software).
We are available to provide paid professional services to assist you with any of these integrations.
Paid professional services are available to assist with any of these integrations.
```mermaid
+1 -1
View File
@@ -48,4 +48,4 @@ If you have Mautic version 5 or newer, then the above is _likely_ still true,
but you may need to configure settings to specify the DSN as described
[here](https://docs.mautic.org/en/5.x/configuration/settings.html#smtp-transport) or use this [community contributed webhook callback](https://github.com/jos0405/mautickumomtacallback).
If you have done everything right, you should now be able to send messages from Mautic through KumoMTA with SMTP.
You should now be able to send messages from Mautic through KumoMTA via SMTP.
+3 -3
View File
@@ -2,7 +2,7 @@
## Introduction
[Ongage](https://www.ongage.com/) is an email marketing platform that allows you to create and manage content and campaigns then deliver them through your favourite sending engine. This integration describes how to use KumoMTA as the delivery engine for Ongage.
[Ongage](https://www.ongage.com/) is an email marketing platform that allows you to create and manage content and campaigns, then deliver them through your favorite sending engine. This integration describes how to use KumoMTA as the delivery engine for Ongage.
## Instructions
@@ -54,7 +54,7 @@ Now fill in all the required fields and test.
* Select YES for “Use TLS”
* The SMTP Port should match what your listener is listening on. Remember to update your public firewall too.
If you have done everything right, you will see a positive notification that credentials were verified.
You should see a notification that credentials were verified.
Pat yourself on the back then start using Ongage with KumoMTA.
You can now start using Ongage with KumoMTA.
+7 -7
View File
@@ -22,9 +22,9 @@ Ensure that you can inject and deliver mail before proceeding.
### Get Prometheus
First, **Install Node-Exporter** to collect typical system metrics to push to
Prometheus. If you are configuring a cluster, node-exporter should be installed
on each node.
First, **Install Node Exporter** to expose typical system metrics for
Prometheus to collect. If you are configuring a cluster, Node Exporter should
be installed on each node.
There is also a very handy step-by-step guide [here](https://prometheus.io/docs/guides/node-exporter/)
@@ -42,9 +42,9 @@ want to add that to your start up processes.
You can test this with a simple curl : `curl http://localhost:9100/metrics`
Now, **Install Prometheus** for system reporting. Get an appropriate version
from [here](https://prometheus.io/download/). Theoretically this can be set up
on a remote server to collect metrics from your whole cluster. In this
document we will install version 2.47.2 on localhost.
from [here](https://prometheus.io/download/). Prometheus can run on
a remote server to collect metrics from your whole cluster; this example
installs version 2.47.2 on localhost.
```console
$ cd
@@ -88,7 +88,7 @@ Remember to update your firewall to securely access port 9090
The UI is wide open by default so take measures to lock down access with usual network tools.
If you have done this all correctly, you should be able to access the target
You should now be able to access the target
status with `http://<your_domain>:9090/targets` in the Prometheus webUI. The
result should look something like this:
+1 -1
View File
@@ -34,7 +34,7 @@ log_hooks:new_json {
5) If you need to, you can modify the `log_parameters` as per [https://docs.kumomta.com/reference/kumo/configure_log_hook/](https://docs.kumomta.com/reference/kumo/configure_log_hook/)
If you have done everything right, you should see your data fill the Tatami Monitor feed almost immediately.
You should see your data fill the Tatami Monitor feed almost immediately.
+1 -1
View File
@@ -1,4 +1,4 @@
# Cancelling Queued Messages
# Canceling Queued Messages
Occasionally, you will have a bad send or for some reason need to cancel a send
quickly. OOPS! It happens. KumoMTA offers an API specifically for
@@ -4,17 +4,17 @@ KumoMTA comes with several utilities that are useful for debugging or supporting
# Utilities list
* validate-shaping - Used for validating the syntax of your custom shaping files. Using this tool is as simple as providing the shaping file to the utility on the command line. IE: `/opt/kumomta/sbin/validate-shaping /opt/kumomta/etc/policy/shaping.toml`
* resolve-shaping-domain - Used for identifying which traffic shaping rules will be applied to a given destination domain. IE: `/opt/kumomta/sbin/resolve-shaping-domain mosaicco.com`
* validate-shaping - Used for validating the syntax of your custom shaping files. Using this tool is as simple as providing the shaping file to the utility on the command line. For example: `/opt/kumomta/sbin/validate-shaping /opt/kumomta/etc/policy/custom-shaping.toml`
* resolve-shaping-domain - Used for identifying which traffic shaping rules will be applied to a given destination domain. For example: `/opt/kumomta/sbin/resolve-shaping-domain mosaicco.com`
* tsa-daemon - The TSA Daemon is a tool that can provide centralized traffic shaping data for your entire cluster even across data centers, providing the KumoMTA nodes can connect to it over TCP. This is typically launched from KumoMTA directives as documented [here](../trafficshaping/automation.md#configure-the-tsa_initlua-file)
* traffic-gen - TrafficGen is a handy performance testing tool that uses core KumoMTA speed to generate high-volume injection testing SMTP messages. Usage instructions are available with `/opt/kumomta/sbin/traffic-gen --help`
* tailer - Tailer provides a flexible command line tool for tracing log activity in real-time without having to `tail -f` the actual logs. It allows you to filter for specific patterns or evaluate a specific batch size of log lines. Usage instructions are available with `/opt/kumomta/sbin/tailer --help` More details can be found [here](./logs.md#using-tailer).
* proxy-server - KumoProxy is a functional socks5 proxy server that can run independently from KumoMTA. Usage instructions are available with `/opt/kumomta/sbin/proxy-server --help`
* tailer - Tailer provides a flexible command line tool for tracing log activity in real time without having to `tail -f` the actual logs. It allows you to filter for specific patterns or evaluate a specific batch size of log lines. Usage instructions are available with `/opt/kumomta/sbin/tailer --help` More details can be found [here](./logs.md#using-tailer).
* proxy-server - KumoProxy is a functional SOCKS5 proxy server that can run independently from KumoMTA. Usage instructions are available with `/opt/kumomta/sbin/proxy-server --help`
* accounting.sh - a helpful tool to show volumes of messages sent and received in a calendar month. This can be helpful for your own purposes in tracking monthly and annual volume.
* toml2jsonc - As implied by the name, will convert a TOML file to JSON.
* explain-throttle - explains how a throttle is interpreted by Kumomta
* explain-throttle - explains how a throttle is interpreted by KumoMTA
* resolve-queue-config - similar to `resolve-shaping-domain` but will show queue settings for that particular domain or MX
* resolve-site-name - provides the result of how KumoMTA sees the MX-Rollup for the target domain. For instance `/opt/kumomta/sbin/resolve-site-name kumomta.com` results in `smtp.google.com`
* tls-probe can be used in two ways. With the `probe` option, if can test if an MX supports STARTTLS. With the list-rustls-cipher-suites options, it will show all cipher suites supported by rustls.
* tls-probe can be used in two ways. With the `probe` option, it can test if an MX supports STARTTLS. With the `list-rustls-cipher-suites` option, it will show all cipher suites supported by rustls.
* kcli - KumoMTA Command Line Interface (KCLI) is a useful tool for accessing the HTTP API directly from the command line. Usage instructions are available with `/opt/kumomta/sbin/kcli --help` More details can be found [here](./kcli.md).
* kumod - this is the actual KumoMTA daemon and is just listed here for completeness.
+6 -6
View File
@@ -49,19 +49,19 @@ SITE SOURCE PROTO D T C
The output is presented in two sections:
1. The ready queues
2. The scheduled queues
1. The Ready Queues
2. The Scheduled Queues
The ready queue data is presented in columns that are mostly self explanatory, but the numeric counts are labelled with single character labels:
The Ready Queue data is presented in columns that are mostly self-explanatory, but the numeric counts are labeled with single character labels:
* D - the total number of delivered messages
* T - the total number of transiently failed messages
* C - the number of open connections
* Q - the number of ready messages in the queue
Note that the ready queue counter values reset whenever the ready queue is reaped, which occurs within a few minutes of the ready queue being idle, so those numbers are only useful to get a sense of recent/current activity. Accurate accounting must be performed using the delivery logs and not via this utility.
Note that the Ready Queue counter values reset whenever the Ready Queue is reaped, which occurs within a few minutes of the Ready Queue being idle, so those numbers are only useful to get a sense of recent/current activity. Accurate accounting must be performed using the delivery logs and not via this utility.
The scheduled queue data is presented in two columns; the queue name and the number of messages in that queue.
The Scheduled Queue data is presented in two columns; the queue name and the number of messages in that queue.
## Managing Bounces
@@ -183,7 +183,7 @@ Reference Manual for more information.
## Monitoring Inbound SMTP handshaking
When debugging, it is often helpful to monitor the full SMTP handshaking process in real-time. The kcli client enables that for inbound connections with the `trace-smtp-server` function:
When debugging, it is often helpful to monitor the full SMTP handshaking process in real time. The kcli client enables that for inbound connections with the `trace-smtp-server` function:
```console
$ kcli trace-smtp-server
+3 -3
View File
@@ -1,6 +1,6 @@
# Using the KumoProxy Socks5 proxy utility
# Using the KumoProxy SOCKS5 proxy utility
KumoMTA comes with a socks5 proxy of our own design to assist with deployment of cluster environments.
KumoMTA comes with its own SOCKS5 proxy server to assist with deployment of cluster environments.
The binary is located at `/opt/kumomta/sbin/proxy-server` and can be operated independently from KumoMTA.
@@ -43,7 +43,7 @@ Next, create an environment file where you can add your system variables.
Populate with your IP and Port
IE:
For example:
```txt
PROXY_IP="172.31.37.164"
PROXY_PORT="5000"
+3 -3
View File
@@ -38,7 +38,7 @@ nicely in this documentation. The actual log records are not output with wrappin
## Manually
We can take a look at a specific log by decompressing it and since these are
You can view a specific log by decompressing it: since these are
[zstd compressed](https://github.com/facebook/zstd#readme), you can view all
but the current one with zstdcat. ZSTD is a streaming compression utility so
the current file cannot be accessed until it is flushed and closed. You can
@@ -51,8 +51,8 @@ you are regularly wanting to inspect the logs on a live system.
[kumo.configure_local_logs](../../reference/kumo/configure_local_logs/index.md)
has all of the available logging configuration options.
Using the example above, we can see the content of the newest file after
stopping KumoMTA with a `zstdcat /var/log/kumomta/20230314-181435`.
Using the example above, you can see the content of the newest file after
stopping KumoMTA with `zstdcat /var/log/kumomta/20230314-181435`.
If you have not done so already, you will want to install `zstd` with a (`dnf`
or) `apt install zstd`. Below there is a sample of a decompressed received log:
+2 -2
View File
@@ -66,7 +66,7 @@ Oct 19 21:53:01 localhost.localdomain kumod[902]: 2023-10-20T01:53:01.221509Z I
In this example the `-f` option tells journalctl to follow the log, in other words to tail or continuously read the file, the `-n 50` option tells journalctl to start by reading the previous 50 lines, and the `-u` option tells journalctl to filter by a specific unit, in this case the `kumomta.service` unit.
A common issue with new installs is ownership of the spool directory. When the spool is provisioned as a separate volume, it will not be owned by the `kumod` user. In this example we change ownership of the `/var/spool/kumomta` directory, then attempt to start the kumomta service, then read the system journal to identify the issue:
A common issue with new installs is ownership of the spool directory. When the spool is provisioned as a separate volume, it will not be owned by the `kumod` user. The following example deliberately reproduces the problem by changing ownership of the `/var/spool/kumomta` directory to root, then attempts to start the kumomta service and reads the system journal to identify the issue:
```console
$ sudo systemctl stop kumomta
@@ -168,4 +168,4 @@ The log levels available, in order from least to most verbose are:
* Trace
!!! warning
The lower, more verbose levels of log levels can be very verbose, especially the `trace` level. These levels should not be enabled permanently as they can lead to a full disk in a short period of time.
The lower log levels can be very verbose, especially the `trace` level. These levels should not be enabled permanently as they can lead to a full disk in a short period of time.
+2 -2
View File
@@ -150,8 +150,8 @@ above.
When in batch mode, the connection will receive a batch consisting of
1 or more messages, up to the `batch_size` that you configured. The batch
can be less than the `batch_size`; the connection will pop off up-to the
configured number of messages from the _ready queue_. That queue holds
can be less than the `batch_size`; the connection will pop off up to the
configured number of messages from the _Ready Queue_. That queue holds
only a finite number of messages that are immediately ready for delivery.
The popping process does not artificially delay to encourage a larger
batch size. It will grab whatever is immediately ready and send it
+1 -1
View File
@@ -58,4 +58,4 @@ Some of the components that should be offloaded to their own servers include:
* Redis - Redis is used in multi-node environments for common throttles and counters and should not run on the same hardware as individual nodes.
* Injectors - A common architecture in many sending environments is to run message generators on the MTA nodes. This approach can lead to resource contention between the generators and the KumoMTA daemon and should be avoided.
* Log Processors - Another common architecture choice is to run log processors locally on the MTA nodes, with the processor reading the logs and pushing the data to a destination of choice. We recommend configuring KumoMTA either with log hooks to deliver data via HTTP/AMQP/Kafka, or to operate the minimum tooling necessary to move log files to a separate system for processing. Note that any log processing done locally will potentially lead to resource contention and can take disk IOPS away from the KumoMTA daemon.
* DNS - While we do recommend running a local caching Bind 9 service to ensure DNS performance, you do not need to run it on your MTA hardware. KumoMTA has its own resolver and caching in-process, DNS should be on its own server local to the KumoMTA nodes.
* DNS - While we do recommend running a local caching BIND 9 service to ensure DNS performance, you do not need to run it on your MTA hardware. KumoMTA has its own resolver and caching in-process, DNS should be on its own server local to the KumoMTA nodes.
+2 -2
View File
@@ -8,6 +8,6 @@ DNS performance is critical, and if DNS performance is poor it can slow down the
Due to the large volume of queries issued by KumoMTA it is strongly recommended that you use a local caching DNS resolver. Relying on external DNS providers introduces excess latency and potential service outages.
On most Linux distributions the default resolver is [Bind](https://en.wikipedia.org/wiki/BIND). Not all distributions maintain a current version of Bind, updating to the latest stable release is strongly recommended.
On most Linux distributions the default resolver is [BIND](https://en.wikipedia.org/wiki/BIND). Not all distributions maintain a current version of BIND; updating to the latest stable release is strongly recommended.
Tuning Bind for performance is beyond the scope of this document but is recommended.
Tuning BIND for performance is recommended, but is beyond the scope of this document.
+3 -3
View File
@@ -22,11 +22,11 @@ The **key concept** in this flow is that when you enable shipping logs via HTTP/
This means that when planning server capacity you must take into account the events that will be associated with each message and count them toward the capacity goal.
For example: if your target is to relay 1,000,000 messages per hour through a server, and you plan on delivering log data via HTTP as webhooks, and you estimate that on average each message will have three log events (reception, one transient failure (4xx), and one delivered or permanent failure (5XX)), then your required server throughput is actually 4,000,000 messages per hour. Inversely, if your server is designed to relay 1,000,000 messages per hour it will in fact only be able to relay 250,000 outbound messages and 750,000 log events.
For example: if your target is to relay 1,000,000 messages per hour through a server, and you plan on delivering log data via HTTP as webhooks, and you estimate that on average each message will have three log events (reception, one transient failure (4xx), and one delivered or permanent failure (5xx)), then your required server throughput is actually 4,000,000 messages per hour. Inversely, if your server is designed to relay 1,000,000 messages per hour it will in fact only be able to relay 250,000 outbound messages and 750,000 log events.
## Traffic Shaping For Log Queues
KumoMTA's default traffic shaping rules are configured to prevent reputation damage when sending by respecting the expectation of MailBox Providers(MBPs).
KumoMTA's default traffic shaping rules are configured to prevent reputation damage when sending by respecting the expectations of mailbox providers (MBPs).
While those defaults can help prevent throttling, they are overly restrictive when sending messages to internal HTTP/AMQP/Kafka endpoints, and can result in queue backpressure.
@@ -49,7 +49,7 @@ log_hooks:new_json {
```
!!! note
In the preceding example we use the `meta` option instead of the `headers` option because logging headers requires the message body to be processed during logging. This can be combined with the [msg:import_x_headers](../../reference/message/import_x_headers.md) function to push the relevant headers into the message metadata for efficient logging.
The preceding example uses the `meta` option instead of the `headers` option because logging headers requires the message body to be processed during logging. This can be combined with the [msg:import_x_headers](../../reference/message/import_x_headers.md) function to push the relevant headers into the message metadata for efficient logging.
Given the log hook name of `webhook` the following should be added to the shaping helper TOML configuration file:
+1 -1
View File
@@ -8,7 +8,7 @@ The process to queue log events and make them available for sending via `custom_
## Configuring A Queue Handler for AMQP
When a message is ready to be queued, the `get_queue_config` event is fired, at which point we can specify the protocol of the queue, in this case `custom_lua`. In the example below, we check whether the message is queued to the `amqp` queue and act accordingly:
When a message is ready to be queued, the `get_queue_config` event is fired, at which point you can specify the protocol of the queue, in this case `custom_lua`. The example below checks whether the message is queued to the `amqp` queue and acts accordingly:
```lua
kumo.on('get_queue_config', function(domain, tenant, campaign, routing_domain)
+3 -3
View File
@@ -6,11 +6,11 @@
Vault helps you keep passwords and other secrets separated from running code to help reduce the possibility of security leaks such as accidentally saving your API key in a GitHub repo.
## Configuring KumoMTA to use Hashicorp Vault
## Configuring KumoMTA to use HashiCorp Vault
The documentation [in the reference manual](https://docs.kumomta.com/reference/keysource/?h=hashi#hashicorp-vault) is straightforward, but does have some nuance.
In the example shown there and below, we are storing the DKIM signing key as a file in vault so it can be called dynamically, but including the vault token in the script is not a particularly secure way of doing things. It is recommended to place the vault address and token in environment variables that are accessible to KumoMTA. In most cases, that will mean modifying the systemd unit service file.
In the example shown there and below, the DKIM signing key is stored as a file in Vault so it can be loaded dynamically. Including the Vault token in the script, however, is not particularly secure. It is recommended to place the vault address and token in environment variables that are accessible to KumoMTA. In most cases, that will mean modifying the systemd unit service file.
```lua
local vault_signer = kumo.dkim.rsa_sha256_signer {
@@ -25,7 +25,7 @@ local vault_signer = kumo.dkim.rsa_sha256_signer {
}
```
To modify the systemd service file, use the built in edit command in systemctl. The [man page is here](https://man7.org/linux/man-pages/man1/systemctl.1.html), but Digital Ocean has an excellent [tutorial](https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units) that explains it in plain English.
To modify the systemd service file, use the built in edit command in systemctl. The [man page is here](https://man7.org/linux/man-pages/man1/systemctl.1.html), but DigitalOcean has an excellent [tutorial](https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units) that explains it in plain English.
The short version is that you can use `systemctl edit` to edit the file and add "Environment" values under the `[Service]` section so that those values will be available when the system service daemon starts KumoMTA. The example below modified the FULL service config. The `--full` option can be removed to modify a snippet instead of the full config.
+2 -2
View File
@@ -1,6 +1,6 @@
# Checking Inbound SMTP Authentication
When hosting relay users it is important to protect your infrastructure from malicious senders, often without the ability to whitelist the IP addresses of legitimate users. In such environments, it is critical to set up SMTP Authentication to validate injecting hosts before relaying their mail.
When hosting relay users it is important to protect your infrastructure from malicious senders, often without the ability to allowlist the IP addresses of legitimate users. In such environments, it is critical to set up SMTP Authentication to validate injecting hosts before relaying their mail.
!!! note
Authentication in KumoMTA can only occur on a TLS protected connection after `STARTTLS` has successfully been processed. This is because AUTH PLAIN credentials can be decoded and should not be sent over an open connection.
@@ -56,7 +56,7 @@ end)
## Querying a Keystore for Authentication
A more secure option for storing authentication credentials for checking is Hashicorp Vault. See the [Storing Secrets in Hashicorp Vault](./hashicorp_vault.md) page for more information on how to populate the credentials in the Vault as well as how to secure the connection credentials.
A more secure option for storing authentication credentials for checking is HashiCorp Vault. See the [Storing Secrets in HashiCorp Vault](./hashicorp_vault.md) page for more information on how to populate the credentials in the Vault as well as how to secure the connection credentials.
```lua
function vault_auth_check(user, password)
+1 -1
View File
@@ -10,7 +10,7 @@ The process to queue log events and make them available for sending via `custom_
## Configuring A Queue Handler for Kafka
When a message is ready to be queued, the `get_queue_config` event is fired, at which point we can specify the protocol of the queue, in this case, `custom_lua`. In the example below, we check whether the message is queued to the `kafka` queue and act accordingly:
When a message is ready to be queued, the `get_queue_config` event is fired, at which point you can specify the protocol of the queue, in this case, `custom_lua`. The example below checks whether the message is queued to the `kafka` queue and acts accordingly:
```lua
kumo.on('get_queue_config', function(domain, tenant, campaign, routing_domain)
+4 -3
View File
@@ -14,9 +14,10 @@ User Guide.
## Configuring A Queue Handler for NATS
When a message is ready to be queued, the `get_queue_config` event is fired, at
which point we can specify the protocol of the queue, in this case,
`custom_lua`. In the example below, we check whether the message is queued to
the `nats` queue and act accordingly:
which point you can specify the protocol of the queue, in this case,
`custom_lua`.
The example below checks whether the message is queued to the `nats` queue and acts accordingly:
```lua
kumo.on('init', function()
+1 -1
View File
@@ -2,7 +2,7 @@
Many of the largest MailBox Providers (MBPs) operate platforms that provide feedback to senders through their response codes during the SMTP conversation. This feedback will include information related to the traffic shaping patterns in use by the sender, including bounces for too many connections, too many messages per connection, sending rate, and sender reputation.
To ensure optimum throughput and deliverability, KumoMTA features Traffic Shaping Automation (TSA) that monitors responses from the MBPs and adjusts traffic shaping rules on a granular level to ensure compliance with the guidelines of the MBPs in realtime.
To ensure optimum throughput and deliverability, KumoMTA features Traffic Shaping Automation (TSA) that monitors responses from the MBPs and adjusts traffic shaping rules on a granular level to ensure compliance with the guidelines of the MBPs in real time.
## TSA Architecture
+3 -3
View File
@@ -2,7 +2,7 @@
<iframe width="560" height="315" src="https://www.youtube.com/embed/Vxbe5ExMOXk?si=2SC7o8FObyvWqavl" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
By default, the KumoMTA server will deliver messages as quickly as possible to each destination, with few restrictions regarding number of connections, number of messages per connection, or number of messages per second. Because unthrottled sending is unwelcome by most MailBox Providers (MBPs) it is highly recommended that KumoMTA users implement traffic shaping in order to limit sending speeds to something more aligned with the expectations of the individual MBPs.
By default, the KumoMTA server will deliver messages as quickly as possible to each destination, with few restrictions regarding number of connections, number of messages per connection, or number of messages per second. Because unthrottled sending is unwelcome to most MailBox Providers (MBPs) it is highly recommended that KumoMTA users implement traffic shaping in order to limit sending speeds to something more aligned with the expectations of the individual MBPs.
Common throttles include concurrent connection limits, messages per connection, rate of opening connections, and rate of sending messages. In addition, users can set options for various timeouts, queue sizes, and what encryption rules to use when communicating with remote hosts.
@@ -30,9 +30,9 @@ The remainder of this chapter is focused on the use of the `shaping.lua` helper.
## Traffic Shaping Automation
Many of the largest MailBox Providers (MBPs) operate platforms that provide feedback to senders through their response codes during the SMTP conversation. This feedback will include information related to the traffic shaping patterns in use by the sender, including bounces for too many connections, too many messages per connection, sending rate, and sender reputation.
Many of the largest MBPs operate platforms that provide feedback to senders through their response codes during the SMTP conversation. This feedback will include information related to the traffic shaping patterns in use by the sender, including bounces for too many connections, too many messages per connection, sending rate, and sender reputation.
To ensure optimum throughput and deliverability, KumoMTA features Traffic Shaping Automation (TSA) that monitors responses from remote hosts and adjusts traffic shaping rules on a granular level in realtime.
To ensure optimum throughput and deliverability, KumoMTA features Traffic Shaping Automation (TSA) that monitors responses from remote hosts and adjusts traffic shaping rules on a granular level in real time.
## In This Chapter
+6 -6
View File
@@ -39,7 +39,7 @@ max_connection_rate = "100/min"
max_message_rate = "100/s"
{% endcall %}
The `mx_rollup` option indicates whether or not the settings should apply to the domain or the site_name. In the example above, even though foo.com is hosted by Yahoo! we want to override the message throttle for the foo.com domain. The `mx_rollup` option is true by default and only needs to be specified for domains that override the main site name entry.
The `mx_rollup` option indicates whether or not the settings should apply to the domain or the site_name. In the example above, even though foo.com is hosted by Yahoo! the goal is to override the message throttle for the foo.com domain. The `mx_rollup` option is true by default and only needs to be specified for domains that override the main site name entry.
While the default max_deliveries_per_connection is 100, it is overridden for yahoo.com (and all domains that share the same site name as the yahoo.com domain) to 20. The foo.com domain is part of the same site name as yahoo.com, but because mx_rollup is set to false the foo.com domain is treated separately and instead is set to 50. Because there is a sources entry for IP-1, the max_deliveries_per_connection is further overridden to 5 for that source's traffic in particular.
@@ -63,12 +63,12 @@ $ dig +short mx outlook.com
5 outlook-com.olc.protection.outlook.com.
```
We can see that the individual MX hostnames have the same
`.olc.protection.outlook.com` suffix, so we can use that to identify the consumer MXes.
You can see that the individual MX hostnames have the same
`.olc.protection.outlook.com` suffix, so you can use that suffix to identify the consumer MXes.
In addition, Microsoft has recently announced a change to the MX hostnames used by Office 365, where existing MX records will end in `.mail.protection.outlook.com` but any user who wishes to activate DANE to enhance security is to use an MX hostname that ends in `.mx.microsoft`.
To address these three scenarios, we can use the following provider blocks:
To address these three scenarios, you can use the following provider blocks:
{% call toml_data() %}
[provider."outlook"]
@@ -133,7 +133,7 @@ domain that blends multiple different providers together.
!!! warning
When a provider is defined, it does **_not_** merge the various `site_name` queues covered by the provider together, which means that the `connection_limit` and `max_message_rate` options will not be enforced across all matching queues, but will be applied separately to each ready queue covered by the provider block.
The provider block introduces two new options: `provider_connection_limit` and `provider_max_message_rate`. When the `provider_connection_limit` and `provider_max_message_rate` options are set, the throttles defined will be enforced across all matching site_name ready queues for that provider. This is typically the desired behavior. One example of a scenario where the provider_ options would not be used is Mimecast: each regional MX pattern used by Mimecast is a separate set of servers in that region, but traffic shaping expectations are the same for all regions. To address this we use a provider block without the `provider_` throttles:
The provider block introduces two new options: `provider_connection_limit` and `provider_max_message_rate`. When the `provider_connection_limit` and `provider_max_message_rate` options are set, the throttles defined will be enforced across all matching site_name ready queues for that provider. This is typically the desired behavior. One example of a scenario where the provider_ options would not be used is Mimecast: each regional MX pattern used by Mimecast is a separate set of servers in that region, but traffic shaping expectations are the same for all regions. To address this, use a provider block without the `provider_` throttles:
{% call toml_data() %}
[provider."mimecast"]
@@ -142,7 +142,7 @@ max_deliveries_per_connection = 100
connection_limit = 10
{% endcall %}
In this case we can define traffic shaping rules that apply to Mimecast globally, but which are still enforced by each region's ready queue without limiting worldwide traffic.
In this case you can define traffic shaping rules that apply to Mimecast globally, but which are still enforced by each region's ready queue without limiting worldwide traffic.
!!! note
Both the `provider_` and regular throttles can be set, where `connection_limit` would be for the individual site names, and `provider_connection_limit` would cap the overall connection count. The same would apply for `max_message_rate` and `provider_max_message_rate`. Combining the `provider_` and regular throttles should be done with care, as it is easy to over-constrain the server if the settings are not aligned correctly.