Commit Graph

860 Commits

Author SHA1 Message Date
Wez Furlong b13ab592e5 kumoproxy: improve diagnostics around connectivity issues
Previously we were very tight-lipped. We now will log some
context to the journal for the proxy service for issues that
we couldn't propagate back to the client.
2024-07-15 11:55:00 -07:00
Wez Furlong d385a1a684 logs: expand source_address to include proxy information
Change the field from a SocketAddr to a struct with distinct fields:

```json
    // For SMTP delivery, the source address (and port) that was used.
    // (*Since: Dev Builds Only*)
    "source_address": {
        // The source address. The port number may be unknown and reported
        // as zero when using a proxy protocol.
        "address": "10.0.0.1:53210",
        // If a proxy protocol was used, this field will be
        // set to its name. It may be null/not set for no proxy,
        // "haproxy" or "socks5".
        "protocol": "socks5",
        // If a proxy protocol was used, this field will be
        // set to the proxy server address. It will be null/not set
        // when no proxy was used.
        "server": "192.168.1.1:5000"
    },
```

In #154, the request was to log configuration information here, but I
opted against this as there can be a number of different configuration
fields and the combinatorics for future changes make me uncomfortable
from a code maintainance perspective--it will already be heavy to
try to pass thu all of the existing config information, and as we
add more options in the future it will be awful not just to look at,
but also from a memory and storage overhead.

The approach taken here is to make a little struct that is flexible
enough to convey the desired information without it being too much of a
burden.

closes: #40
closes: #154
2024-07-12 09:22:46 -07:00
Wez Furlong 9a9443be71 logs: add source_address field to SMTP client logs
Previously we would log the pool and source name, but it is
desirable to also log the underlying socket address information,
so here we go!

refs: #40
2024-07-12 08:39:15 -07:00
Wez Furlong b003e49c9c rfc5321: split out banner_timeout from connect_timeout
The motivation for this is:

My test environment is not permitted to reach outbound port 25.
If I run an ad-hoc test without setting up an explicit sink,
I end up with messages that try to reach the public internet.
Since they are blocked at a firewall, each of the MX hosts in
the connection plan is subject to a 60s wait before trying the next
thing.

In addition, this can cause the shutdown to take longer while
we wait for the in-flight delivery attempts to complete.

Making a separate configuration option allows the local administrator
to decide how to split the time waiting for a connection from
the time waiting for the banner.

refs: https://github.com/KumoCorp/kumomta/issues/196
2024-07-12 07:54:43 -07:00
Wez Furlong a00e8d0f0a NEW: add tracing to smtp client, kcli trace-smtp-client
It is now possible to trace outbound SMTP sessions, filtering
by a variety of properties.

Details are in `kcli trace-smtp-client --help` and also in
the docs at /reference/kcli/trace-smtp-client.md

refs: #87
2024-07-11 08:56:25 -07:00
Wez Furlong 4369bd8355 docs: document rollup module
closes: https://github.com/KumoCorp/kumomta/issues/213
2024-07-10 10:35:00 -07:00
Wez Furlong c1565d2478 docs: forgot to format 2024-07-10 10:34:21 -07:00
Wez Furlong af103457a7 docs: remove low-level webhook example
There are subtle edge cases that are handled by the helper,
so I want everyone to use the helper in order to reduce potential
headaches for everyone.
2024-07-10 08:27:55 -07:00
Wez Furlong 9ea02c3968 docs: cover --validate mode
closes: #211
2024-07-10 08:16:31 -07:00
Wez Furlong f898e53502 docs: show how to setup up grafana dashboard
closes: https://github.com/KumoCorp/kumomta/issues/215
2024-07-10 07:10:05 -07:00
Wez Furlong fdf7fcf249 docs: show how you can compose kumo.encode with kumo.uuid 2024-07-10 06:29:02 -07:00
Wez Furlong f99f852747 mod-sqlite: add :close method 2024-07-09 16:50:48 -07:00
Wez Furlong cde3ba66cb mod-kafka: add explicit close method 2024-07-09 14:09:23 -07:00
Wez Furlong ec795b71d3 mod-http: add client:close() method
Allows explicit closure of any cached connections in the client
object.

Make use of this in the log_hooks:new_json method.
2024-07-09 13:57:46 -07:00
Wez Furlong 73b8b5b4e0 docs: add missing page for configure_unbound_resolver 2024-07-09 13:42:01 -07:00
Wez Furlong c659fe6d4f new: add kumo.uuid lua module 2024-07-09 13:25:32 -07:00
Wez Furlong bd43edee14 lua_deliver: add close method
This is to facilitate more precise control over the lifetime of
connection objects.

You can now define `function sender:close()` on the connection
object that is returned from your lua protocol constructor.

This method will be called when the underlying
QueueDispatcher::close_connection method is called.

Adjust the logic in ready_queue to explicitly call
QueueDispatcher::close_connection in the cases where we're closing the
connection; previously we'd just leave it to the Drop handler to take
care of this.

Adjust the amqp docs to show how to bubble the close call through
to the associated client.
2024-07-09 09:55:10 -07:00
Wez Furlong 58f53687e6 mod-amqp: add explicit client close method
This allows explicit and clean shutdown of a connection.
2024-07-09 09:55:10 -07:00
Tom Mairs 6d4ef9a303 fix typo 2024-07-09 15:09:09 +00:00
Tom Mairs 53b16cefb4 fix typo 2024-07-08 20:40:32 +00:00
Wez Furlong 722f19a078 docs: fix name of requeue_message event in the example 2024-07-08 09:11:34 -07:00
Mike Hillyer cf7f5843ab Add a stronger warning that the quickstart does not produce production-ready installs. 2024-07-05 10:58:37 -04:00
Mike Hillyer c42536ee77 Updating the quickstart tutorial. 2024-07-05 09:48:20 -04:00
Wez Furlong f7604a3f7f docs: cut over toml examples to toml_data data macro
refs: https://github.com/KumoCorp/kumomta/issues/212
2024-07-03 16:09:52 -07:00
Wez Furlong aa22e788c9 mkdocs: make build the default command, rather than serve
This makes it easier to run in woodpecker
2024-07-03 15:37:27 -07:00
Wez Furlong d416b31af9 docs: add toml_data macro
This macro allows embedding TOML data into the docs,
and showing it in a tab that has both the TOML and JSON
representation of that data.

It works by executing the toml2jsonc helper that was added
in an earlier commit.

There's some machinery here to compile that utility to run
in the context of the mkdocs docker image; that works
locally, let's see how well it works in CI!

Usage is simple; before:

```toml
["something"]
foo = "bar"
```

after:

{% call toml_data() %}
["something"]
foo = "bar"
{% endcall %}

The first page to get switched over to this is https://docs.kumomta.com/tutorial/configuring_kumomta/

refs: https://github.com/KumoCorp/kumomta/issues/212
2024-07-03 14:59:52 -07:00
Wez Furlong e49985ff54 logging: capture incoming SMTP command line for Rejections 2024-07-03 06:46:01 -07:00
Wez Furlong acdb8e36d4 docs: add some diagrams to the deployment architecture page
I noticed that this page was totally blank; let's add some
useful content here!

refs: https://github.com/KumoCorp/kumomta/issues/214
2024-07-01 09:50:24 -07:00
Wez Furlong ca3339cebb docs: remove "building docker image" from "using docker" docs
It doesn't fit there, and the topic is a bit more complex
than that brief summary.
2024-07-01 07:21:08 -07:00
Wez Furlong ef8cbea354 docs: fixup links to the rapidoc http page 2024-06-27 19:56:53 -07:00
Wez Furlong 49facb4d0a docs: fixup amqp docs some more 2024-06-27 19:51:09 -07:00
Wez Furlong 5b0d936ef7 docs: improve markdown output for the kcli help 2024-06-27 15:40:39 -07:00
Wez Furlong fa2c6225e8 docs: add kcli reference docs
These are automatically extracted from `kcli --help`.
2024-06-27 15:18:07 -07:00
Wez Furlong 5d7cf4005f docs: fixup AMQP example 2024-06-27 09:42:36 -07:00
Wez Furlong d66d6bd750 docs: fix link 2024-06-27 08:39:03 -07:00
Wez Furlong e683fed556 docs: move Log Record to its own reference page
This makes it a bit easier to find and link to.
2024-06-27 08:28:45 -07:00
Wez Furlong 7330721766 docs: changelog for #211 and kumod --validate 2024-06-27 07:02:10 -07:00
Wez Furlong f8c3429738 serde: move json/toml to new kumo.serde module. Added YAML.
This helps to avoid bloating kumo-server-common, and makes it
easier to group related functions together in the docs.

The original names are still usable; we'll remove them in a
future release.
2024-06-26 11:08:15 -07:00
Wez Furlong 4083afa672 docs: add faq entry for queue flushing 2024-06-24 15:04:12 -07:00
Wez Furlong a2d76df7a9 NEW: rebind API and kcli subcommand
refs: https://github.com/KumoCorp/kumomta/issues/209
2024-06-24 12:26:06 -07:00
Wez Furlong 10bb7f2717 Add some string and regex functions
closes: https://github.com/KumoCorp/kumomta/issues/220
2024-06-21 17:20:04 -07:00
Mike Hillyer 53820402ba Killing off some orphans. 2024-06-20 17:55:52 -04:00
Wez Furlong 6f83f69236 docs: markdown <ol> is tricky, just use bullets/<ul> 2024-06-20 08:38:56 -07:00
Wez Furlong 7fc8dd97e6 docs: discuss creating an always-suspended queue
closes: https://github.com/KumoCorp/kumomta/issues/210
2024-06-20 08:35:17 -07:00
Mike Hillyer 77f9830513 Fix warning in the webhooks page 2024-06-19 16:55:52 -04:00
Mike Hillyer f957d812c1 Add the response rewrite page to the TOC. 2024-06-19 16:35:53 -04:00
Mike Hillyer 43e7023e72 Updated rewriting page for the User Guide. 2024-06-19 15:39:31 -04:00
giuliano-mendez 2c307d4e41 Fix typo 2024-06-19 08:39:23 -07:00
Wez Furlong 015fad5e4f Revert "Add page on rewriting remote server responses to the User Guide."
This reverts commit 82a258a92e0e0eef6c7e483ab25f71146cdfeea3; that new
page is incomplete and has invalid syntax.

Let's take it out for now to unblock the doc build.
2024-06-19 08:37:11 -07:00
Mike Hillyer 82a258a92e Add page on rewriting remote server responses to the User Guide. 2024-06-19 10:13:40 -04:00