Wez Furlong d30a6718cb tsa: improve publish throughput by 2.3x ...

On my test system, running `k6 run tsa-torture.js` (included in this
commit) could achieve ~26,000 qps.

With the changes in this commit I see ~60,000 qps.

The gains are realized by introducing batching and transactions in the
publish code path, and allowing sqlite to avoid a synchronous flush.

Turning down the tracing::info! that was logging when rules
were matched made a non-trivial impact on throughput;
35k qps with info!, 60k qps with it at the current debug!, so I've
put it at debug.

2024-11-23 12:36:16 -07:00

..

bounce-classify

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

cidr-map

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

config

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

data-loader

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

dkim

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

dns-resolver

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

domain-map

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

duration-serde

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

integration-tests

test/retry_schedule: make test expectation clearer

2024-11-20 15:53:18 -07:00

kcli

Drop direct tungstenite dependency in favor of tokio-tungstenite

2024-11-13 13:50:27 -07:00

kumo-api-types

add session_id to LogRecord and thread through

2024-11-19 08:12:13 -07:00

kumo-chrono-helper

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

kumo-dmarc

dmarc: allow dead_code for now, to suppress warnings

2024-11-20 16:01:18 -07:00

kumo-log-types

adopt more compact representation of ResolvedAddress in logs/diagnostics

2024-11-22 08:54:12 -07:00

kumo-prometheus

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

kumo-server-common

Remove unused dependencies

2024-11-13 13:50:27 -07:00

kumo-server-lifecycle

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

kumo-server-memory

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

kumo-server-runtime

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

kumo-spf

dmarc: add support for generating reports

2024-11-20 12:33:50 -07:00

kumod

adopt more compact representation of ResolvedAddress in logs/diagnostics

2024-11-22 08:54:12 -07:00

lruttl

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

maildir

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mailparsing

Make named lifetimes explicit

2024-11-13 13:50:27 -07:00

message

Remove unused dependencies

2024-11-13 13:50:27 -07:00

mod-amqp

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-digest

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-dns-resolver

update unbound, fixup DNS port number propagation

2024-11-19 10:57:49 -07:00

mod-encode

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-filesystem

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-http

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-kafka

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-memoize

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-redis

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-regex

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-serde

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-sqlite

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-string

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mod-uuid

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

mta-sts

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

proxy-server

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

regex-set-map

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

rfc5321

smtp client: belt and suspenders

2024-11-15 13:42:51 -07:00

spool

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

spool-util

Remove unused dependencies

2024-11-13 13:50:27 -07:00

tailer

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

throttle

throttle: make tolerance <= to reduce flakes

2024-11-19 12:52:49 -07:00

timeq

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

tls-probe

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

toml2jsonc

docs: add toml_data macro

2024-07-03 14:59:52 -07:00

traffic-gen

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

tsa-daemon

tsa: improve publish throughput by 2.3x

2024-11-23 12:36:16 -07:00

uuid-helper

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

validate-bounces

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

validate-shaping

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00

version-info

Manage all external dependencies at the workspace level

2024-11-13 13:50:27 -07:00