Ordinarily, the logic that calls the requeue_message event is
responsible for increasing the number of attempts, computing the revised
due time, and processing message expiration.
When using XFER to move a message to another node, we capture the
scheduling state at the point that the xfer was initiated.
If we do this inside the requeue_message event callback then we will
capture the scheduling information *prior* to the normal adjustments
that would happen later.
This commit adds kumo.xfer.xfer_in_requeue to handle this case; you pass
through the scheduling information parameters that were added to the
requeue_message event in the prior commit, and it will compute any
revised scheduling for the message before wrapping the message up
in the XFER encapsulation and moving the message to the xfer queue.
This adjusted process ignores message expiration, so if the message
is on its last retry it will be xfer'd to the target host which then
might choose to perform the expiration. We can't handle the expiration
in the context of `requeue_message` because the event handler doesn't
own the message and has no way to signal that it has reached the end of
its life.
Folks doing xfer in requeue_message will generally be moving the message
well in advance of it being expired, so this minor limitation is not
expected to be a practical problem.
I factored out the sample outputs and used an include to pull
in that content.
In the future, we could automate creating those sample data files
based on the live instance, however, it will need some thought
as many metrics are instantiated dynamically. That means that
simply launching the server to dump the initial counters could
miss a large number of the main production counters that are
the most interesting. Likely we'll need something smarter than
that approach. But that is an issue for the future!
A number of manually authored HTTP api docs can now be replaced
by the versions generated from code, making them more accurate
and less likely to drift away from the implementation as
future changes are made, so that's what this commit does.
Define, by convention, a tag like `kcli:bounce` to indicate that
a jsonschema path definition is linked to the `kcli bounce` subcommand.
This commit adds appropriate tags to each command, and teaches
the jsonschematodocs utility how to generate a boilerplate
paragraph containing the link.
This commit changes how the various http endpoints are registered;
previously we had to define the handler function in one place,
then in another, explicitly register that handler with a path
and operation type with the router. If we wanted to also export
documentation about that method, we would need to annotate the handler
with a duplicate of the path and method type.
There was a decent amount of boiler plate and it was very easy to omit
some of that registration information, or for it to potentially drift
or be mistranscribed.
This commit eliminates most of the boiler plate; now we require that
every endpoint be annotated with the utoipa::path macro, which
centralizes the handler, path and method type definition in one place.
To register the route and docs we now have a much simpler
`router_and_docs!` macro that can register both the with the axum router
and with the openapi docs machinery.
DRY when it comes to listing out components, request and response
bodies. We don't need to centrally list those if we're good
at annotating the path macros.
This commit removes a couple of schemas from the docs; that's
ok because the content of those was already inlined into the
respective endpoint docs anyway, so we're not losing anything,
just eliminating a redundant copy of the same information
on a second location.
We currently use a rather hacky embedding of rapidoc to provide a
generic browser around the jsonschema export from our API interface.
It's not great for a couple of reasons:
* The font sizes are tiny
* The documentation rapidoc produces is not indexable, being
generated by javascript when the browser loads. This also
prevents making proper links to the various doc pages
This commit introduces a little utility that we can use during
the doc build to translate the schema into documentation files
that can then be processed as normal by the build.
This commit does this just for kumod at this time, but we could
also add tsa daemon in the future if we expand its API surface.
These annotations allow the jsonschema to round-trip through the
utoipa json schema parser, which is important for a follow up
commit that will facilitate improved docs.
Previously, if setting `content.headers["To"]` we'd generate one header
for that value, and one for the per-recipient `To` header value.
We now check to see if the former is present; if so, we'll skip
generating the per-recipient `To` header value.
This option was named `--no-splice` on the command line because that
was the easiest UX there. The negative logic makes it harder to
understand how to enable/disable, so let's flip it and default
use_splice=true.
refs: #459
Add TLS encryption and username/password authentication support for
the KumoProxy SOCKS5 server, with full Lua configuration capabilities.
Proxy Server Changes:
- Add kumo.start_proxy_listener() Lua function with TLS support
- Add proxy_server_auth_rfc1929 event for Lua-based auth validation
- Return AuthInfo from auth for ACL system integration
- Support optional and required authentication modes
- Maintain backwards-compatible legacy CLI mode (--listen, --timeout-seconds)
Breaking Changes:
- Cache renamed from rfc5321_rustls_config to rustls_client_config
Refs: #451
Closes: 459
This commit doesn't change any behavior, it just adds an explicit
test for exceeding request_body_limit, and expands on the docs
for request_body_limit to clarify the response when the limit
is exceeded.
Replace the older type with the newer one.
Serialize a copy of auth_info when handling deferred generation,
so that we have a lossless representation of that state when
we eventually process the request. That doesn't change really
anything today, but will enable more granular ACL checks in
the future.
Pass the auth_info through to the http_message_generated and
xfer_message_received events to enable more granular access
control policies to be scripted.
This function sets up the accounting log which records authentication
and authorization events to a local log file.
Add some test assertions that we're not seeing any failed authn/authz
in a number of integration tests.
Add explicit check for failed a request to kcli's streaming metrics
parser, which previously would ignore the request status.
This commit allows the policy to return a richer representation of
the authentication information, which can include multiple identities
and group membership information.
This commit is a step towards some general improvements around
our handling of authentication and authorization.
This commit is focused primarily on authorization, but there are
some adjustments to how we track authentication as part of enabling
that.
We now have a separate AuthInfo type that holds the overall
authentication information/context associated with an inbound
SMTP or HTTP session.
It is populated with the peer_address as a fact rather than a statement
of trust.
If authentication via the appropriate lua auth callback is successful,
then the AuthInfo has additional identities added.
There are some types and events for loading access control lists and
matching their rules against an AuthInfo.
There is now a system default ACL that is equivalent to the prior
hard-coded access policy that was encoded into each HTTP endpoint.
This change makes it possible to replace the ACL with a
user-defined ACL.
Later will be some work on authentication to allow more options
for HTTP auth.
* Add AWS SigV4 signature module
- Implements complete AWS Signature Version 4 algorithm
- Supports all AWS services (S3, SNS, SQS, Kinesis Firehose, etc.)
- Includes SHA256 hashing and HMAC-SHA256 signing
- Secure key management via KeySource (files, Vault, inline)
- Session token support for temporary credentials
- Comprehensive documentation and examples
- 5 unit tests and 5 integration tests (all passing)
Exposes kumo.aws.sign_v4() function to Lua for signing AWS API
requests. This enables KumoMTA to integrate with AWS services for
streaming logs to Kinesis Firehose, storing data in S3, sending
notifications via SNS, and more.
Tested with:
- S3 GET/PUT requests
- SNS POST requests
- SQS with query parameters
- Kinesis Firehose PutRecord
updating README.md file
Co-authored-by: Aditya Ganti <adityaganti@mac.mynetworksettings.com>
kay ozaki
Wez Furlong
Kevin Vu
Mike Hillyer
Aditya Ganti