diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 9539534c..0ccb969b 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -52,7 +52,7 @@ jobs: - name: Build env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: CARDS=true ./docs/build.sh + run: CI=true CARDS=true ./docs/build.sh - name: Upload artifact uses: actions/upload-pages-artifact@v1 with: diff --git a/.gitignore b/.gitignore index d779992c..1d2d9fcb 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,4 @@ dhat-heap.json .vscode/ **/*.deb **/*.rpm +/__pycache__/ diff --git a/docs/build.sh b/docs/build.sh index 3ba5f89b..5ca8bad2 100755 --- a/docs/build.sh +++ b/docs/build.sh @@ -37,7 +37,7 @@ mdbook-linkcheck --standalone docs # Adjust path to pick up pip-installed binaries PATH="$HOME/.local/bin;$PATH" -pip install --quiet mkdocs-material mkdocs-git-revision-date-localized-plugin black mkdocs-exclude +pip install --quiet mkdocs-material mkdocs-git-revision-date-localized-plugin black mkdocs-exclude mkdocs-macros-plugin if test -n "${CARDS}" ; then pip install --quiet pillow cairosvg fi diff --git a/docs/generate-toc.py b/docs/generate-toc.py index 958e485d..fd3083c6 100755 --- a/docs/generate-toc.py +++ b/docs/generate-toc.py @@ -281,6 +281,10 @@ TOC = [ "object: addressheader", "reference/addressheader", ), + Page( + "object: connectionmeta", + "reference/connectionmeta.md", + ), Page( "object: keysource", "reference/keysource.md", diff --git a/docs/reference/connectionmeta.md b/docs/reference/connectionmeta.md new file mode 100644 index 00000000..6016ed8d --- /dev/null +++ b/docs/reference/connectionmeta.md @@ -0,0 +1,43 @@ +# Connection Metadata Object + +{{since('dev')}} + +This object represents a collection of metadata keys and values that +are associated with an established incoming SMTP connection. + +KumoMTA populates a small number of predefined fields (see below), and allows +your policy scripts the ability to read those values as well as write (and +read!) additional values as needed by the local policy. For instance, you may +decide to compute a value after EHLO has been processed by +[smtp_server_ehlo](events/smtp_server_ehlo.md) and access that value in a +later SMTP event handler. + +Prior to calling [smtp_server_message_received](events/smtp_server_message_received.md), +KumoMTA will copy the values from the connection metadata and use those to populate +the [message](message/index.md) metadata. + +The `get_meta` and `set_meta` methods shown below are used to read and write +metadata values. + +## Predefined Connection Metadata Values + +The following values are predefined by KumoMTA: + +|Name|Purpose|Since| +|----|-------|-----| +|`reception_protocol`|indicates the reception protocol, such as `ESMTP`|{{since('dev', inline=True)}}| +|`received_via`|indicates the IP:port of the KumoMTA listener that is handling this session|{{since('dev', inline=True)}}| +|`received_from`|indicates the IP:port of the sending or peer machine in this session|{{since('dev', inline=True)}}| + +## Available Methods + +### `conn_meta:get_meta(name)` + +Returns the value associated with *name*, or `nil` if no such value has been defined. +Values may be predefined by KumoMTA, or may be set by policy scripts using `conn_meta:set_meta()`. + + +### `conn_meta:set_meta(name, value)` + +Sets the value associated with *name* to *value*. Value must be serializable as JSON; it can be simple +strings or numbers, but may also be an array or object value. diff --git a/docs/reference/events/get_listener_domain.md b/docs/reference/events/get_listener_domain.md index 7c2d0fdc..6851c1d5 100644 --- a/docs/reference/events/get_listener_domain.md +++ b/docs/reference/events/get_listener_domain.md @@ -1,4 +1,4 @@ -# `kumo.on('get_listener_domain', function(domain, listener))` +# `kumo.on('get_listener_domain', function(domain, listener, conn_meta))` This event is triggered by the SMTP server to retrieve information about either a source or destination domain to help determine whether the message @@ -11,6 +11,12 @@ listener address. When the SMTP `RCPT TO` command is issued by the client, the destination domain is passed as the *domain* parameter to this event. +{{since('dev', indent=True)}} + The *conn_meta* parameter represents the connection metadata and + can be used to share state between the various SMTP listener + event handlers. See [Connection Metadata](../connectionmeta.md) + for more information. + The event is expected to return a listener-domain object constructed by a call to [kumo.make_listener_domain](../kumo/make_listener_domain.md), or a `nil` value to indicate that there is no explicit configuration. diff --git a/docs/reference/events/smtp_server_auth_plain.md b/docs/reference/events/smtp_server_auth_plain.md index d6eb4406..701048c5 100644 --- a/docs/reference/events/smtp_server_auth_plain.md +++ b/docs/reference/events/smtp_server_auth_plain.md @@ -1,4 +1,4 @@ -# `kumo.on('smtp_server_auth_plain', function(authz, authc, password))` +# `kumo.on('smtp_server_auth_plain', function(authz, authc, password, conn_meta))` Called by the ESMTP server in response to the client issuing an `"AUTH PLAIN"` authentication attempt. @@ -17,6 +17,12 @@ The event handler receives the following parameters: with the *password* parameter. * *password* - the password which belongs to the claimed *authc* +{{since('dev', indent=True)}} + The *conn_meta* parameter represents the connection metadata and + can be used to share state between the various SMTP listener + event handlers. See [Connection Metadata](../connectionmeta.md) + for more information. + Note that [PLAIN SASL](https://www.rfc-editor.org/rfc/rfc4616) allows for *authz* to be empty. KumoMTA will assume the same value as *authc* in that case, so this event will always be triggered with that value. diff --git a/docs/reference/events/smtp_server_ehlo.md b/docs/reference/events/smtp_server_ehlo.md index 2888b5dc..db32f40a 100644 --- a/docs/reference/events/smtp_server_ehlo.md +++ b/docs/reference/events/smtp_server_ehlo.md @@ -1,9 +1,15 @@ -# `kumo.on('smtp_server_ehlo', function(domain))` +# `kumo.on('smtp_server_ehlo', function(domain, conn_meta))` Called by the ESMTP server in response to the client issuing either a `"HELO"` or `"EHLO"` command. The event handler is passed the *domain* parameter from the `HELO/EHLO` command. +{{since('dev', indent=True)}} + The *conn_meta* parameter represents the connection metadata and + can be used to share state between the various SMTP listener + event handlers. See [Connection Metadata](../connectionmeta.md) + for more information. + You may choose to reject the connection via [kumo.reject](../kumo/reject.md). ```lua diff --git a/docs/reference/events/smtp_server_mail_from.md b/docs/reference/events/smtp_server_mail_from.md index 9d4a909c..070ebe13 100644 --- a/docs/reference/events/smtp_server_mail_from.md +++ b/docs/reference/events/smtp_server_mail_from.md @@ -1,4 +1,4 @@ -# `kumo.on('smtp_server_mail_from', function(sender))` +# `kumo.on('smtp_server_mail_from', function(sender, conn_meta))` Called by the ESMTP server in response to the client issuing a `"MAIL FROM"` command. The event handler is passed the *sender address* parameter from @@ -6,6 +6,12 @@ the `MAIL FROM` command. The *sender address* is an [EnvelopeAddress](../address/index.md) object. +{{since('dev', indent=True)}} + The *conn_meta* parameter represents the connection metadata and + can be used to share state between the various SMTP listener + event handlers. See [Connection Metadata](../connectionmeta.md) + for more information. + You may choose to reject the connection via [kumo.reject](../kumo/reject.md). ```lua diff --git a/docs/reference/events/smtp_server_message_received.md b/docs/reference/events/smtp_server_message_received.md index 04c6c9d2..e0474c68 100644 --- a/docs/reference/events/smtp_server_message_received.md +++ b/docs/reference/events/smtp_server_message_received.md @@ -1,4 +1,4 @@ -# `kumo.on('smtp_server_message_received', function(message))` +# `kumo.on('smtp_server_message_received', function(message, conn_meta))` Called by the ESMTP server after receiving the message data. @@ -10,6 +10,12 @@ The event handler will be passed a [Message](../message/index.md) object. The Message will always have a `Received` header prepended that captures trace information about the sender. +{{since('dev', indent=True)}} + The *conn_meta* parameter represents the connection metadata and + can be used to share state between the various SMTP listener + event handlers. See [Connection Metadata](../connectionmeta.md) + for more information. + This event is the best place to carry out a number of important policy decisions: * DKIM signing via [message:dkim_sign](../message/dkim_sign.md). diff --git a/docs/reference/events/smtp_server_rcpt_to.md b/docs/reference/events/smtp_server_rcpt_to.md index 47984c17..5dd77d49 100644 --- a/docs/reference/events/smtp_server_rcpt_to.md +++ b/docs/reference/events/smtp_server_rcpt_to.md @@ -1,4 +1,4 @@ -# `kumo.on('smtp_server_rcpt_to', function(recipient))` +# `kumo.on('smtp_server_rcpt_to', function(recipient, conn_meta))` Called by the ESMTP server in response to the client issuing a `"RCPT TO"` command. The event handler is passed the *recipient address* parameter from @@ -6,6 +6,12 @@ the `RCPT TO` command. The *recipient address* is an [EnvelopeAddress](../address/index.md) object. +{{since('dev', indent=True)}} + The *conn_meta* parameter represents the connection metadata and + can be used to share state between the various SMTP listener + event handlers. See [Connection Metadata](../connectionmeta.md) + for more information. + You may choose to reject the connection via [kumo.reject](../kumo/reject.md). ```lua diff --git a/docs/reference/kumo/configure_local_logs.md b/docs/reference/kumo/configure_local_logs.md index 6d4e677a..b2ce9ddc 100644 --- a/docs/reference/kumo/configure_local_logs.md +++ b/docs/reference/kumo/configure_local_logs.md @@ -124,6 +124,7 @@ kumo.configure_local_logs { Allows configuring per-record type logging. +{% raw %} ```lua kumo.configure_local_logs { per_record = { @@ -156,6 +157,7 @@ kumo.configure_local_logs { }, } ``` +{% endraw %} The keys of the `per_record` table must correspond to one of the record types listed below, or the special `Any` key which can be used diff --git a/docs/userguide/configuration/logging.md b/docs/userguide/configuration/logging.md index 6e4a010a..37a154fe 100644 --- a/docs/userguide/configuration/logging.md +++ b/docs/userguide/configuration/logging.md @@ -49,12 +49,14 @@ kumo.configure_local_logs { If a non-JSON format is needed for the logs, the *template* option can be used: +{% raw %} ```lua kumo.configure_local_logs { -- .. template = [[{{type}} id={{ id }}, from={{ sender }} code={{ code }} age={{ timestamp - created }}]], } ``` +{% endraw %} The [Mini Jinja](https://docs.rs/minijinja/latest/minijinja/) templating engine is used to evalute logging templates. The full supported syntax is [documented @@ -64,6 +66,7 @@ here](https://docs.rs/minijinja/latest/minijinja/syntax/index.html). Any key pre Sometimes it is necessary to configure logging on a more granular basis, especially when using custom log formats. KumoMTA supports this using the *per_record* option: +{% raw %} ```lua kumo.configure_local_logs { per_record = { @@ -96,5 +99,6 @@ kumo.configure_local_logs { }, } ``` +{% endraw %} This can be used to override paths, disable logs, or customize the format of specific event types. diff --git a/mkdocs-base.yml b/mkdocs-base.yml index 324bb977..8632e21e 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -45,8 +45,11 @@ plugins: - social: cards: !ENV [CARDS, false] - git-revision-date-localized: + enabled: !ENV [CI, false] enable_creation_date: true type: timeago + - macros: + module_name: mkdocs_macros - exclude: glob: - "**/_index.md" diff --git a/mkdocs_macros.py b/mkdocs_macros.py new file mode 100644 index 00000000..62f9d8e6 --- /dev/null +++ b/mkdocs_macros.py @@ -0,0 +1,46 @@ +import json + +# https://mkdocs-macros-plugin.readthedocs.io/en/latest/macros/ +def define_env(env): + @env.macro + # Set indent=True when you want to define a box containing version-specific info. + # + # Set inline=True when you want to define a simple inline version indicator, + # such as when emitting information into a table row. + def since(vers, indent=False, inline=False): + + scope = "section" + expanded = "" + expander = "???" + if indent: + scope = "outlined box" + expander = "!!!" + + if vers == "dev": + first_line = "*Since: Dev Builds Only*" + if scope != "section": + expanded = "+" + blurb = f""" + *The functionality described in this {scope} requires a dev build of KumoMTA. + You can obtain a dev build by following the instructions in the + [Installation](/userguide/installation/linux.md) section.* +""" + else: + first_line = f"*Since: Version {vers}*" + blurb = f""" + *The functionality described in this {scope} requires version {vers} of KumoMTA, + or a more recent version.* +""" + + if inline: + return f"({first_line})" + + # If we're not expandable, don't emit the expanded marker + if expander == "!!!": + expanded = "" + + return f""" +{expander}{expanded} info "{first_line}" +{blurb} +
+"""