Commit Graph
2059 Commits
Author SHA1 Message Date
Wez Furlong e63d1ca15c typing.lua: improve error reporting
Make the error messages a bit more regular and a bit easier
to read and trace.
2024-06-26 22:49:32 -07:00
Wez Furlong 245caaea6e dkim_sign.lua: make typing more succinct
This is a bit more legible
2024-06-26 17:48:33 -07:00
Wez Furlong aaaa96e7fb typing.lua: improve tests 2024-06-26 17:38:34 -07:00
Wez Furlong 7392ccf3d8 typing.lua: report errors from the caller's location
Don't attribute the error to the metamethods in the typing
module, attribute them to the calling code.
2024-06-26 17:04:28 -07:00
Wez Furlong d7a4b7a334 Add kumo.traceback
This function captures information about the lua call stack.
This is intended to be used when debugging lua module issues
and shouldn't be used in normal operation, as it can be relatively
expensive.
2024-06-26 17:03:34 -07:00
Wez Furlong b9dccf1cc3 add policy-extras.typing lua module
The primary purpose here is to allow defining typed records
for use in our helpers.

The dkim_sign module has been updated to make use of this
to ensure that the correct shape of data has been loaded.
2024-06-26 16:38:22 -07:00
Wez Furlong f8c3429738 serde: move json/toml to new kumo.serde module. Added YAML.
This helps to avoid bloating kumo-server-common, and makes it
easier to group related functions together in the docs.

The original names are still usable; we'll remove them in a
future release.
2024-06-26 11:08:15 -07:00
Wez Furlong 58ee029e33 kumod: disable INFO log level when in --validate mode
This keeps the output focused on any errors raised by
the validation stage.

refs: https://github.com/KumoCorp/kumomta/issues/211
2024-06-26 07:14:48 -07:00
Wez Furlong b1c04968fe metrics: add a couple of scheduled queue rollups
In addition to tracking queue depth by each scheduled queue,
track metrics for:

* domain
* tenant
* tenant-campaign
2024-06-25 19:14:52 -07:00
Wez Furlong 39818c1e36 Add validation for dkim_sign helper
refs: https://github.com/KumoCorp/kumomta/issues/211
2024-06-25 12:08:37 -07:00
Wez Furlong 471b44ae02 sources.lua: fixup to iterate file names correctly when validating 2024-06-25 11:19:56 -07:00
Wez Furlong 3139be2cfd Add new kumod --validate flag
The purpose is to provide a deeper, offline validation pass
of the policy configuration, prior to deploying and making it
live.

The system behavior changes when in `--validate` mode:

* Listeners, spool and spawned tasks will be silently skipped;
  the parameters will be validated but the primary functions
  of those things will be skipped silently.
* After triggering the `init` event, an additional new `validate_config`
  event (which can be registered multiple times) will be triggered
  to allow lua modules to perform extended validation.
* A module can either raise an error via `error` to immediately report
  a problem, or instead call a new, preferred, `kumo.validation_failed()`
  function to flag validation as failed but allow additional validation
  to be performed and summarized all together.
* Once the `validate_config` event returns, the process will terminate
  with either exit code 0 for a successful validation, or non-zero
  to indicate that something failed.

Validation errors are reported in a human readable form.

This commit adds validate_config event handlers for the following
helper modules:

* `shaping` - any warnings reported by the underlying rust code
  will be reported here and cause validation to fail. This is
  functionally equivalent to using the `validate-shaping` binary,
  except that it will automatically be passed the set of shaping
  files defined by your `init.lua`

  If the `sources` helper is also configured, the list of sources
  referenced by the shaping config will be cross-checked against
  the sources data to confirm that all possible sources are defined.

* `sources` - each listed source and pool will be validated by
  calling `kumo.make_egress_source` or `kumo.make_egress_pool`
  respectively.

  Pool membership will be validated to confirm that every
  listed pool is defined in the sources data.

* `queues` - each domain and tenant that references an egress_pool
  will be cross-checked with the `sources` helper, if the sources
  helper has been configured.

It is now an error to attempt to setup any of the above helpers
more than once.

refs: https://github.com/KumoCorp/kumomta/issues/211
2024-06-25 11:05:04 -07:00
Wez Furlong 6bb74f88da fix some warnings
These are all harmless; we do rely on printing these things
in Debug impls when under test.
2024-06-25 07:41:40 -07:00
Wez Furlong 785feb4510 cidr: update dep and revise error handling
The upstream cidr crate has a released a version that exposes the more
relaxed parser, so refactor the prior work from
7eb5bdcb5f (which was pointing to a
temporary development branch) to reference its "final form".

This is done by exposing a `parse_cidr` function from our cidr-map
crate, and standardizing all callers to route through that.

refs: https://github.com/stbuehler/rust-cidr/issues/8
2024-06-25 07:34:18 -07:00
Wez Furlong 4083afa672 docs: add faq entry for queue flushing 2024-06-24 15:04:12 -07:00
Wez Furlong a2d76df7a9 NEW: rebind API and kcli subcommand
refs: https://github.com/KumoCorp/kumomta/issues/209
2024-06-24 12:26:06 -07:00
Wez Furlong c415cdf4ee cargo update 2024-06-24 07:32:03 -07:00
dependabot[bot] e9b42e8a1f build(deps): bump docker/build-push-action from 5 to 6
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-24 06:50:46 -07:00
Wez Furlong 10bb7f2717 Add some string and regex functions
closes: https://github.com/KumoCorp/kumomta/issues/220
2024-06-21 17:20:04 -07:00
Mike Hillyer 53820402ba Killing off some orphans. 2024-06-20 17:55:52 -04:00
Wez Furlong 6f83f69236 docs: markdown <ol> is tricky, just use bullets/<ul> 2024-06-20 08:38:56 -07:00
Wez Furlong 7fc8dd97e6 docs: discuss creating an always-suspended queue
closes: https://github.com/KumoCorp/kumomta/issues/210
2024-06-20 08:35:17 -07:00
Wez Furlong 7eb5bdcb5f cidr: improve diagnostics for invalid cidr
This is done in two parts:

1. Showing the list of bad cidrs and their corresponding error messages
2. Upgrading to a yet-to-be-released branch of the underlying cidr
   library that can provide a helpful suggestion of how to fix it

refs: https://github.com/stbuehler/rust-cidr/issues/8
2024-06-20 07:57:34 -07:00
Mike Hillyer 77f9830513 Fix warning in the webhooks page 2024-06-19 16:55:52 -04:00
Mike Hillyer f957d812c1 Add the response rewrite page to the TOC. 2024-06-19 16:35:53 -04:00
Wez Furlong 0f39f7e306 take a run at packaging the new resolve-site-name script 2024-06-19 13:30:34 -07:00
Wez Furlong ca048031ad Add script that can resolve and print the site-name
The guts of this hvae been shared on discord a couple of times.
Let's put this in a convenient script.
We should arrange to install this in our official packages,
but for now, let's get it in the repo.
2024-06-19 13:07:45 -07:00
Mike Hillyer 43e7023e72 Updated rewriting page for the User Guide. 2024-06-19 15:39:31 -04:00
giuliano-mendez 2c307d4e41 Fix typo 2024-06-19 08:39:23 -07:00
Wez Furlong 015fad5e4f Revert "Add page on rewriting remote server responses to the User Guide."
This reverts commit 82a258a92e0e0eef6c7e483ab25f71146cdfeea3; that new
page is incomplete and has invalid syntax.

Let's take it out for now to unblock the doc build.
2024-06-19 08:37:11 -07:00
Wez Furlong 4ad21257bd ci: fixup failure in PR mode
We need to avoid recursively copying any symlinks-to-dirs as
directories. We can do that with cp, but the parallel xcp utility
doesn't support the --no-dereference flag at this time.

refs: https://github.com/tarka/xcp/issues/52
2024-06-19 08:34:22 -07:00
Mike Hillyer 82a258a92e Add page on rewriting remote server responses to the User Guide. 2024-06-19 10:13:40 -04:00
Wez Furlong 308d430160 cargo update 2024-06-18 15:43:08 -07:00
Wez Furlong e1d0eb60fd docs: clarify conformance issue description in changelog 2024-06-18 07:38:15 -07:00
Wez Furlong a20ba45520 docs: changelog for #217 2024-06-18 07:35:09 -07:00
ncai e6d10875e4 use nanoseconds in redis lease durations 2024-06-18 07:26:46 -07:00
Wez Furlong 626a7608e4 Fix bad message-id breaking message rebuild
The issue only triggered when there were issues that required
rebuilding the message body in addition to the bad header.

The fix here is to silently ignore any header parsing errors
during a rebuild.

closes: #216
2024-06-17 13:50:53 -07:00
Mike Hillyer 275f82ae58 Update README.md
Add site reference.
2024-06-17 10:37:57 -04:00
Mike Hillyer 376903e9a1 Update README.md
Adding reference to paid support to readme.
2024-06-17 10:32:07 -04:00
Wez Furlong b5ca1e9d9c docs: avoid throwing error when using cached rustdoc 2024-06-12 15:21:15 -07:00
Wez Furlong 0e401a4b9b ci: fixup docs build
The symlink switcheroo needs to happen sooner
2024-06-12 15:17:54 -07:00
Wez Furlong 43e2e34eaa ci: make the docker commit sha a bit shorter 2024-06-12 15:16:42 -07:00
Wez Furlong 7db3bb5eae docs: mention multiarch docker image in the changelog 2024-06-12 15:09:55 -07:00
Wez Furlong c76b621028 ci: fix template for -dev multiarch builds 2024-06-12 15:06:24 -07:00
Wez Furlong 47b7fb645f ci: ostensibly build multiarch images for releases
This uses the same pattern as for dev builds.
Won't know if this really works until we cut the next tag.
2024-06-12 15:05:04 -07:00
Wez Furlong b0e98c88a1 ci: re-organize -dev docker image building
This groups the related steps together better, and changes
the tags to be `commit-HASH-ARCH`.
2024-06-12 14:57:48 -07:00
Wez Furlong 96ff9902df docker: add label to container kumomta image 2024-06-12 14:52:23 -07:00
Wez Furlong 9bd199159c ci: try again 2024-06-12 14:36:27 -07:00
Wez Furlong d0f300e06a ci: try to workaround legacy drone/woodpecker stuff 2024-06-12 14:23:22 -07:00
Wez Furlong 90952fc660 ci: fixup typo 2024-06-12 14:11:02 -07:00