Commit Graph
58 Commits
Author SHA1 Message Date
Wez Furlong ce7dece927 move domains out of listener, replace with get_listener_domain event
This commit also includes a policy helper
`policy_extras.listener_domains` to make it convenient to define
listener domains in toml and/or json files.

To facilitate this, the DomainMap rust structure has been exposed
to lua code via the new `kumo.domain_map.new` function.
2023-05-31 12:29:44 -07:00
Wez Furlong b7c557c31a define_egress_(pool|source) -> make_egress_(pool|source)
Switch the configuration plumbing for pools and sources to be pull-based
rather than push based.

In other words, rather than defining them in the `init` event,
you now need to supply them to the new `get_egress_pool` and
`get_egress_source` events.

Data is cached by default for 1 minute. This allows for new sources
to come into being on-demand, and for data to age out and change
over time, without requiring that the server be restarted.

This commit updates the reference section, but there is some content
in the user guide that refers to the old style of configuration that
will need to be updated.

refs: https://github.com/KumoCorp/kumomta/issues/13
2023-05-30 14:08:07 -07:00
Mike Hillyer 0d85aea133 Adding SMTP AUTH to the user guide. 2023-05-17 17:39:23 -04:00
Wez Furlong 9d3411431f Add outbound SMTP auth support
refs: https://github.com/KumoCorp/kumomta/issues/10
2023-05-15 17:38:17 -07:00
Mike Hillyer 3e3483b281 Automated doc formatting fix 2023-05-10 17:44:47 +00:00
Mike Hillyer 88390291b3 Adding Webhook section to the configuration chapter. 2023-05-10 13:42:38 -04:00
Wez Furlong 6fdee41600 Add HA Proxy support
I used a config like this for testing:

```
global
    log stdout  format raw  local0  debug

defaults
    timeout connect 10s
    timeout client 30s
    timeout server 30s
    log global

listen outboundsmtp
    log global
    bind 0:2526 accept-proxy
    mode tcp
    use-server v4 if { src 0.0.0.0/0 }
    use-server v6 if { src ::/0 }
    server v4 0.0.0.0 source 0.0.0.0 usesrc clientip
    server v6 ::: source ::: usesrc clientip
```

Launched via:

```console
$ sudo haproxy -f assets/haproxy.conf -V
```

with the source:

```lua
  kumo.define_egress_source {
    name = 'w00t',
    ha_proxy_server = '127.0.0.1:2526',
    ha_proxy_source_address = '192.168.1.92',
  }
```

refs: #19
2023-05-10 09:50:06 -07:00
Wez Furlong 14291df014 docs: fix missing note admonish 2023-05-09 11:06:01 -07:00
Wez Furlong 927721fc0c Add toml parsing/encoding functions to lua
refs: https://github.com/KumoCorp/kumomta/issues/43
2023-05-09 08:40:37 -07:00
Wez Furlong cceaccf06a docs: webhook/lua delivery reference
refs: https://github.com/KumoCorp/kumomta/issues/36
refs: https://github.com/KumoCorp/kumomta/issues/18
2023-05-08 13:51:06 -07:00
Wez Furlong 65c7a9e5a6 Add reception and delivery protocol to log record
closes: https://github.com/KumoCorp/kumomta/issues/38
2023-05-08 08:39:40 -07:00
Tom Mairs b435e210bb Add Note about CIDR strictness 2023-04-28 09:03:24 -07:00
Wez Furlong ef2abc7031 allow comments in json parsed by kumo.json_load, kumo.json_parse 2023-04-26 15:08:29 -07:00
Mike Hillyer 969d079eb6 Finish FBL and Bounce chapters, add chapter on Sending IPs. 2023-04-24 10:17:51 -04:00
Mike Hillyer e2be05804a Build out FBL and Domain Config chapters, tweak SMTP Listener content. 2023-04-22 12:09:44 -04:00
Mike Hillyer 2e2a283692 Update bounce file name in TOC 2023-04-08 13:56:56 -04:00
Mike Hillyer d619a8eb63 Introduce bounce & fbl chapter, initial fill of bounce section. 2023-04-08 13:56:56 -04:00
Mike Hillyer 475e28fbb5 Update logging documentation. 2023-04-07 21:30:40 -04:00
Wez Furlong e87a5b18bb Improve performance of finding CRLF.CRLF for large messages by 20x
Using a better tuned algorithm for finding a substring match removes
a CPU bottleneck and improves message throughput by 20x at 100Kb
message sizes.
2023-03-27 09:39:51 -07:00
Wez Furlong 6740a68fad add kumo.set_diagnostic_log_filter for adjusting log levels via config 2023-03-24 09:02:10 -07:00
Wez Furlong 36de652dd2 docs: fix indent 2023-03-22 17:05:03 -07:00
Wez Furlong 0167041ed0 add kumo.memoize as a way to add caching to any lua function 2023-03-22 17:01:07 -07:00
Wez Furlong be0fd76d0b add ehlo_domain config option 2023-03-14 22:26:07 -07:00
Wez Furlong e1f0939d81 add json functions 2023-03-14 20:07:40 -07:00
Wez Furlong 4c71090669 docs: standardize the header before the generated list of functions 2023-03-14 10:17:55 -07:00
Wez Furlong b56bed5060 docs: fix admonish syntax, explain the global variable thing better 2023-03-11 23:06:59 -07:00
Wez Furlong 7054fb31c9 switch to using human time for the queue related time intervals 2023-03-06 19:33:05 -07:00
Wez Furlong 00c253a9dd add maildir support
This is to facilitate end-to-end testing to validate that the
right bits come out at the end.
2023-03-06 19:05:49 -07:00
Wez Furlong cd5f7efbb0 docs: link to ARF rfc, add note about extensions: 2023-03-05 20:37:45 -07:00
Wez Furlong c6292440d0 docs: fix a typo and add a missing paragraph 2023-03-05 20:34:29 -07:00
Wez Furlong 1bfa3a42af Add custom logging 2023-03-05 20:29:14 -07:00
Wez Furlong a0e73b72b1 allow logging selected meta and headers to the logs 2023-03-05 15:58:19 -07:00
Wez Furlong 1ac929e6b2 add max_segment_duration option to the logger 2023-03-05 15:25:08 -07:00
Wez Furlong 0ca585bd5a Optionally use redis for throttles 2023-03-05 10:39:56 -07:00
Wez Furlong c184214253 allow loading TLS key+cert from hashicorp vault
refactor to allow reusing the same key loader logic for dkim, smtp
and http.
2023-03-04 11:01:35 -07:00
Wez Furlong c93c9af2eb refine default list of meta fields for supplemental trace header 2023-03-04 10:21:45 -07:00
Wez Furlong 52f7ab6150 add prohibited_hosts and skip_hosts for outbound resolving
While in that part of the code, handle null MX destination domains
(such as `example.com) more nicely.
2023-03-04 10:09:17 -07:00
Wez Furlong 7eb33d2ca5 make smtp server read timeout configurable
client_timeout is the config, and it defaults to 1m
2023-03-04 07:52:53 -07:00
Wez Furlong 6204642270 add trace headers configuration and link to ARF parsing 2023-03-04 07:22:02 -07:00
Wez Furlong 61f974d066 adjust listener domains config to support more relaying options
a domain can now specify:

* relay_from: mail from the domain is allowed to relay anywhere
  if the peer matches the cidr list
* relay_to: mail from anywhere is allowed to relay to the domain
* log_arf: ARF FBL reports are allowed to be received and their
  contents will be logged
* log_oob: oob bounces are allowed to be received and their
  contents will be logged
2023-03-03 22:53:56 -07:00
Wez Furlong 03c3484cd7 rename fbl -> arf
The spec is talking about ARF, so let's use that terminology
2023-03-03 14:34:06 -07:00
Wez Furlong ef7fa59355 synthesize OOB log records from OOB reports
* When logging the reception of a message, if we can parse it
  as an rfc3464 oob report, then synthesize OOB records for
  each recipient
* Add config to allow relaying for domains for which bounces
  are received, so that we can accept their message content.
* If the domain config allows bounces but not relaying, then
  the message will not be spooled or queued after reception;
  it will be dropped with no additional logging.
2023-03-03 06:53:33 -07:00
Wez Furlong d5b13a263b Add kumo.configure_bounce_classifier
and connect it to the logger
2023-03-02 08:59:03 -07:00
Wez Furlong cd7ba3a2bb make smtp client timeouts configurable by command 2023-03-01 12:19:45 -07:00
Wez Furlong 557fff650d implement max_deliveries_per_connection 2023-02-28 15:12:15 -07:00
Wez Furlong a691d7afcc docs: note about the unspecified pool as the default 2023-02-28 13:58:50 -07:00
Wez Furlong c275c696ff implement message and connection rate throttles
These use the local/in-process throttles.
Seems to work in my very light testing so far.
2023-02-28 13:56:58 -07:00
Wez Furlong 50eabe6f98 docs: update get_egress_path_config parameters 2023-02-28 11:47:54 -07:00
Wez Furlong f912fab58b docs: egress source, egress path etc. 2023-02-28 10:31:00 -07:00
Wez Furlong 9ab1727792 rename: dest_site -> egress path 2023-02-28 09:56:20 -07:00