Commit Graph
2131 Commits
Author SHA1 Message Date
Wez Furlong 8f87f66ea6 proxy: refine error reporting a little bit more 2024-07-15 16:04:59 -07:00
Wez Furlong b13ab592e5 kumoproxy: improve diagnostics around connectivity issues
Previously we were very tight-lipped. We now will log some
context to the journal for the proxy service for issues that
we couldn't propagate back to the client.
2024-07-15 11:55:00 -07:00
Wez Furlong d385a1a684 logs: expand source_address to include proxy information
Change the field from a SocketAddr to a struct with distinct fields:

```json
    // For SMTP delivery, the source address (and port) that was used.
    // (*Since: Dev Builds Only*)
    "source_address": {
        // The source address. The port number may be unknown and reported
        // as zero when using a proxy protocol.
        "address": "10.0.0.1:53210",
        // If a proxy protocol was used, this field will be
        // set to its name. It may be null/not set for no proxy,
        // "haproxy" or "socks5".
        "protocol": "socks5",
        // If a proxy protocol was used, this field will be
        // set to the proxy server address. It will be null/not set
        // when no proxy was used.
        "server": "192.168.1.1:5000"
    },
```

In #154, the request was to log configuration information here, but I
opted against this as there can be a number of different configuration
fields and the combinatorics for future changes make me uncomfortable
from a code maintainance perspective--it will already be heavy to
try to pass thu all of the existing config information, and as we
add more options in the future it will be awful not just to look at,
but also from a memory and storage overhead.

The approach taken here is to make a little struct that is flexible
enough to convey the desired information without it being too much of a
burden.

closes: #40
closes: #154
2024-07-12 09:22:46 -07:00
Wez Furlong 9a9443be71 logs: add source_address field to SMTP client logs
Previously we would log the pool and source name, but it is
desirable to also log the underlying socket address information,
so here we go!

refs: #40
2024-07-12 08:39:15 -07:00
Wez Furlong b003e49c9c rfc5321: split out banner_timeout from connect_timeout
The motivation for this is:

My test environment is not permitted to reach outbound port 25.
If I run an ad-hoc test without setting up an explicit sink,
I end up with messages that try to reach the public internet.
Since they are blocked at a firewall, each of the MX hosts in
the connection plan is subject to a 60s wait before trying the next
thing.

In addition, this can cause the shutdown to take longer while
we wait for the in-flight delivery attempts to complete.

Making a separate configuration option allows the local administrator
to decide how to split the time waiting for a connection from
the time waiting for the banner.

refs: https://github.com/KumoCorp/kumomta/issues/196
2024-07-12 07:54:43 -07:00
Wez Furlong d2128ec5bf ci: restore restore mtime
I need to get good at using `git rebase --ignore-date` and
`git commit --amend --date=now` to solve the underlying issue
that I had.
2024-07-11 09:19:08 -07:00
Wez Furlong 1006b56a3f ci: disable restore-mtime logic
Something seems to be caching overly strongly in CI, and I
suspect this.
2024-07-11 09:06:08 -07:00
Wez Furlong a00e8d0f0a NEW: add tracing to smtp client, kcli trace-smtp-client
It is now possible to trace outbound SMTP sessions, filtering
by a variety of properties.

Details are in `kcli trace-smtp-client --help` and also in
the docs at /reference/kcli/trace-smtp-client.md

refs: #87
2024-07-11 08:56:25 -07:00
Wez Furlong 4369bd8355 docs: document rollup module
closes: https://github.com/KumoCorp/kumomta/issues/213
2024-07-10 10:35:00 -07:00
Wez Furlong c1565d2478 docs: forgot to format 2024-07-10 10:34:21 -07:00
Wez Furlong af103457a7 docs: remove low-level webhook example
There are subtle edge cases that are handled by the helper,
so I want everyone to use the helper in order to reduce potential
headaches for everyone.
2024-07-10 08:27:55 -07:00
Wez Furlong 9ea02c3968 docs: cover --validate mode
closes: #211
2024-07-10 08:16:31 -07:00
Wez Furlong be5d5a8c23 run-lua-test: prefer local builds over installed build 2024-07-10 07:59:32 -07:00
Wez Furlong 97286058a1 run-lua-test: prefer debug kumod over release kumod
When we're running tests, we're generally running the debug build, so it
doesn't make sense to run the (potentially cached and not yet up to
date) release build.
2024-07-10 07:56:18 -07:00
Wez Furlong 8c06f7d462 kumod: add --script and --script-args parameters
These are primarily for my benefit; they make it a bit easier
to run kumod for one-off scripts without running the full service,
with less "noise" than you'd get for hacking around it as I was
doing previously in the resolve-site-name script and in the lua
test runner.

The idea is that you can now define a script like:

```lua
local kumo = require 'kumo'

kumo.on('main', function(domain)
  local mx = kumo.dns.lookup_mx(domain)
  print(mx.site_name)
end)
```

And (more) conveniently run it like this:

```
$ kumod --policy /path/to/script --script --script-args outlook.com
outlook-com.olc.protection.outlook.com
```
2024-07-10 07:39:35 -07:00
Wez Furlong f898e53502 docs: show how to setup up grafana dashboard
closes: https://github.com/KumoCorp/kumomta/issues/215
2024-07-10 07:10:05 -07:00
Wez Furlong 835660aa61 docs: make since() macro a bit smarter
Each time we add a new page in `main`, we get an INFO line like this,
because we didn't know the relative link:

INFO    -  Doc file 'reference/string/psl_domain.md' contains an absolute link '/userguide/installation/linux/', it was left as is. Did you mean
           '../../userguide/installation/linux.md'?

this commit adds some logic to compute the appropriate relative
path and make the doc build a bit less noisy.
2024-07-10 06:51:57 -07:00
Wez Furlong fdf7fcf249 docs: show how you can compose kumo.encode with kumo.uuid 2024-07-10 06:29:02 -07:00
Wez Furlong f99f852747 mod-sqlite: add :close method 2024-07-09 16:50:48 -07:00
Wez Furlong 1d4e20c6ee deps: make gethostname a workspace dep
No functional change
2024-07-09 16:34:37 -07:00
Wez Furlong bb7456359c deps: make nix a workspace dep
No functional change
2024-07-09 16:31:01 -07:00
Wez Furlong cde3ba66cb mod-kafka: add explicit close method 2024-07-09 14:09:23 -07:00
Wez Furlong f03dc85287 cargo update 2024-07-09 14:00:44 -07:00
Wez Furlong ec795b71d3 mod-http: add client:close() method
Allows explicit closure of any cached connections in the client
object.

Make use of this in the log_hooks:new_json method.
2024-07-09 13:57:46 -07:00
Wez Furlong 73b8b5b4e0 docs: add missing page for configure_unbound_resolver 2024-07-09 13:42:01 -07:00
Wez Furlong c659fe6d4f new: add kumo.uuid lua module 2024-07-09 13:25:32 -07:00
Wez Furlong bd43edee14 lua_deliver: add close method
This is to facilitate more precise control over the lifetime of
connection objects.

You can now define `function sender:close()` on the connection
object that is returned from your lua protocol constructor.

This method will be called when the underlying
QueueDispatcher::close_connection method is called.

Adjust the logic in ready_queue to explicitly call
QueueDispatcher::close_connection in the cases where we're closing the
connection; previously we'd just leave it to the Drop handler to take
care of this.

Adjust the amqp docs to show how to bubble the close call through
to the associated client.
2024-07-09 09:55:10 -07:00
Wez Furlong 58f53687e6 mod-amqp: add explicit client close method
This allows explicit and clean shutdown of a connection.
2024-07-09 09:55:10 -07:00
Tom Mairs 6d4ef9a303 fix typo 2024-07-09 15:09:09 +00:00
Tom Mairs 53b16cefb4 fix typo 2024-07-08 20:40:32 +00:00
Wez Furlong 902a157616 logging: raise an error when multiple log hooks have the same name 2024-07-08 09:40:42 -07:00
Wez Furlong 722f19a078 docs: fix name of requeue_message event in the example 2024-07-08 09:11:34 -07:00
Wez Furlong 55841daffa ci: fixup amazon arm build 2024-07-05 08:16:08 -07:00
Mike Hillyer cf7f5843ab Add a stronger warning that the quickstart does not produce production-ready installs. 2024-07-05 10:58:37 -04:00
Mike Hillyer c42536ee77 Updating the quickstart tutorial. 2024-07-05 09:48:20 -04:00
Wez Furlong f8b6acbd2f ci: update ubuntu builder for gh upload 2024-07-04 23:10:20 -07:00
Wez Furlong 29ad723c9a ci: explicitly use the ubuntu image for the gh upload 2024-07-04 22:31:13 -07:00
Wez Furlong d70f87593e ci: shift release creation to woodpecker, upload pkgs to release 2024-07-04 17:03:12 -07:00
Wez Furlong 072fc2ea30 ci: add arch to deb filename, fixup gh release upload for arm 2024-07-04 16:45:51 -07:00
Wez Furlong af7d19cb91 ci: try uploading main builds to a new dev GH release
Might need to add a different token for this action to succeed,
but let's give this a go.
2024-07-04 15:42:15 -07:00
Wez Furlong 1e29c1b459 ci: add gpg explicitly
This is needed to install the gh cli
2024-07-04 14:23:17 -07:00
Wez Furlong 7ad26cdac9 ci: install GH cli in the ubuntu image
This time for real; I edited the wrong script last time
2024-07-04 12:31:56 -07:00
Wez Furlong 2a33ed1101 ci: install GH cli in the ubuntu image 2024-07-04 08:54:53 -07:00
Wez Furlong f7604a3f7f docs: cut over toml examples to toml_data data macro
refs: https://github.com/KumoCorp/kumomta/issues/212
2024-07-03 16:09:52 -07:00
Wez Furlong c8de15f9ab ci: remove some stale cruft 2024-07-03 15:44:12 -07:00
Wez Furlong aa22e788c9 mkdocs: make build the default command, rather than serve
This makes it easier to run in woodpecker
2024-07-03 15:37:27 -07:00
Wez Furlong c8530dbe85 ci: maybe run our new published mkdocs image 2024-07-03 15:28:51 -07:00
Wez Furlong 68e01e00cf ci: build mkdocs image periodically 2024-07-03 15:26:48 -07:00
Wez Furlong 32330758a9 docs: fixup repo/tag for mkdocs image 2024-07-03 15:18:59 -07:00
Wez Furlong 24caa792fc docs: maybe build and run our own mkdocs image 2024-07-03 15:15:15 -07:00