Commit Graph

429 Commits

Author SHA1 Message Date
Mike Hillyer 9c16e45a92 Working on clarifying the scopes used in Kumo's shaping.lua. 2025-08-28 11:46:12 -04:00
Daniel Schaaff 3a42354289 fix prettier removing indentation on note section
Signed-off-by: Daniel Schaaff <daniel@danielschaaff.com>
2025-08-28 10:21:44 -04:00
Daniel Schaaff 1bcb2c31ac fix minor typo in logging doc 2025-08-28 10:21:44 -04:00
Tom Mairs 634a00ec6f Add new tested cloud environments 2025-07-27 22:50:10 +00:00
Wez Furlong 91884d3892 docs: format 2025-07-26 13:30:00 +01:00
Pankaj Rathi eae61a90fe Add support for custom key names in HashiCorp Vault secrets (#399)
* Add support for custom key names in HashiCorp Vault secrets

Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-07-25 06:02:21 +01:00
Tom Mairs 88b182d3c3 removed duplication 2025-07-22 23:08:46 +00:00
Tom Mairs 7064344b04 Update list of command line tools 2025-07-22 22:49:12 +00:00
Mike Hillyer 58e9770206 We miss you Mister Rogers. 2025-07-18 12:11:14 -04:00
Tom Mairs 535fc00e2e Add Mint to OS list 2025-07-11 02:52:19 +00:00
Mike Hillyer 34ceef78dd Update User Guide references to log_arf and log_oob to use the new LogThenRelay and LogThenDrop directives. 2025-07-09 15:11:42 -04:00
Bjarn Bronsveld 440fe285b2 Update user guide to replace 'log_fbl' with 'log_arf' in example 2025-06-29 09:24:09 +01:00
Tom Mairs f498cf3482 fix formatting 2025-06-20 20:03:56 +00:00
Tom Mairs a8157812de Add systemd service instructions for KumoProxy 2025-06-20 19:38:23 +00:00
Mike Hillyer 9dba9c5b5f Update traffic shaping defaults to require TLS for Google as per conversation with Gmail admins. 2025-06-10 09:26:07 -04:00
Mike Hillyer ac67fdf71e Linter complaint 2025-05-22 14:39:40 -04:00
Mike Hillyer 02537213aa Typo fix. 2025-05-22 14:27:29 -04:00
Mike Hillyer 168fb14e67 Correction. 2025-05-21 09:53:41 -04:00
Wez Furlong 48f1e48e6d docs: expand get_egress_path_config event docs 2025-05-16 06:28:13 -07:00
Mike Hillyer 41d8cd92a1 Add Postmastery integration. 2025-05-09 13:01:36 -04:00
Mike Hillyer 9249fbc482 Remove extra references. 2025-05-05 16:10:09 -04:00
Wez Furlong 9641bcf1ff docs: note about domain-suffix when using iptables 2025-05-01 10:23:01 -07:00
Wez Furlong db1f3c3773 docs: wrap iptables command 2025-05-01 10:20:01 -07:00
Wez Furlong c3b65667a6 docs: expand discussion around iptables 2025-05-01 10:19:15 -07:00
Wez Furlong 3732bc0fef docs: code formatting 2025-05-01 10:09:01 -07:00
Mike Hillyer d57953ba87 Formatting and spelling fixes. 2025-05-01 12:55:44 -04:00
Mike Hillyer 35e49a4312 Typo fix. 2025-05-01 12:31:05 -04:00
Mike Hillyer 366b45c2e5 Formatting update. 2025-05-01 12:31:05 -04:00
Mike Hillyer e08111beee Implement Wez's feedback to use meta instead of header in the logging example. 2025-05-01 12:31:05 -04:00
Mike Hillyer 1a68d526f5 Update to linux tuning to show possible parameters but not values to prevent copypasta. 2025-05-01 12:31:05 -04:00
Mike Hillyer c978a23a8c Update Linux tuning to have less examples so that people don't blindly implement them. 2025-05-01 12:31:05 -04:00
Mike Hillyer 1ef1cf4cc3 Update docs/userguide/performance/testing.md
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-05-01 12:31:05 -04:00
Mike Hillyer 50a274fa83 Update docs/userguide/performance/loghooks.md
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-05-01 12:31:05 -04:00
Mike Hillyer 1a645681fd Update docs/userguide/performance/loghooks.md
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-05-01 12:31:05 -04:00
Mike Hillyer 163f826fa1 Update docs/userguide/performance/architecture.md
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-05-01 12:31:05 -04:00
Mike Hillyer 7b5b655a09 Update docs/userguide/performance/architecture.md
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-05-01 12:31:05 -04:00
Mike Hillyer 62a48d4787 Added the architecture page, further tweaks. 2025-05-01 12:31:05 -04:00
Mike Hillyer c086a08c03 Linter complaints. 2025-05-01 12:31:05 -04:00
Mike Hillyer 08224b223d DNS and loghook pages 2025-05-01 12:31:05 -04:00
Mike Hillyer dd4a2acf60 Updates to the performance chapter of the userguide. 2025-05-01 12:31:05 -04:00
Mike Hillyer 01cd7985ae Update Message flow page to go deeper on some of the events and factors that will need to be tuned. 2025-05-01 12:31:05 -04:00
Mike Hillyer 27447c8e0b Updated testing page to use the sink container rather than the sink init.lua so that testers will be encouraged to have more realistic tests. 2025-05-01 12:31:05 -04:00
Mike Hillyer 66a377bae6 Initial re-arranging of existing performance related content from the Operation chapter to the Performance chapter. 2025-05-01 12:31:05 -04:00
Mike Hillyer ab40de0920 Initial commit just to test my changes. 2025-05-01 12:31:05 -04:00
Wez Furlong b8310da8be docs: add warning about logging headers
TL;DR: you can easily halve your system performance by logging headers
vs. logging meta.

This is one of those things that is easy to overlook or forget,
but: whenever you need to operate on the message data, rather
than its metadata, the aggregate cost is high.

In this case, we were recently troubleshooting a system where
the CPU was bogged down and we traced it to the logging configuration: a
number of message headers were being logged in a configuration that
made heavy use of throttles and limits in its traffic shaping, and
thus had a large number of Delayed and TransientFailure events being
written to the logs.

When logging headers, each one of those events requires loading
the message from the spool and parsing out the headers.  When the
average message size is ~100KB this imposes a notable overhead
on the CPU and IO utilization of the system.

What we recommend instead of logging headers directly is capturing
the information that you want to log into the message metadata
at the time that the message is received.

The message meta is usually already loaded, but is also typically
much smaller and easier to decode than the full message content
in the cases where it is not loaded.

As a result, it is much cheaper to log meta than to log headers.

This commit adds some warnings and cross links to help folks
be aware of this, and to generally navigate related meta and logging
topics more easily via tags.
2025-04-25 05:52:44 -07:00
Wez Furlong 448fa55afb docs: update for docker dev build naming change 2025-04-24 09:13:20 -07:00
Wez Furlong 02eee83cb5 docs: add explicit chmod to the gpg key 2025-04-22 09:32:12 -07:00
Wez Furlong d21b98ab45 docs: format logging.md 2025-03-21 16:56:12 -07:00
Ryan Bonnell 0d7a0b4bd6 Clarify example of log format template usage for customization 2025-03-20 08:26:31 -07:00
Blake Patton 0befdaf665 Complete the missing part of the command 2025-03-11 10:00:13 -07:00