mirror of
https://github.com/cfal/shoes.git
synced 2026-11-06 08:14:11 +00:00
Add automatic URL-test upstream groups (#221)
Add URL-scoped health and delay-based upstream selection so new connections can use healthy, transport-compatible proxies without requiring a controller listener. This introduces strict client_group.type configuration in place of strategy, bounded automatic probes with lazy activity tracking and reload fencing, controller reporting, and configuration documentation. It also preserves local failure attribution through outbound HTTP/2 adapters so local resource limits do not incorrectly penalize proxy health.
This commit is contained in:
@@ -2,6 +2,8 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Breaking: client groups now use `type` instead of `strategy`, with no legacy alias. Replace `strategy: selector` with `type: selector` and `strategy: round-robin` with `type: round-robin`; configurations containing `strategy` are rejected.
|
||||
- Added `type: url-test` client groups with automatic URL-scoped health/delay selection, tolerance, lazy checks, and supervised probes without a controller listener. See [upstream group configuration](CONFIG.md#clash-controller) and the [URL-test example](examples/url_test.yaml).
|
||||
- Breaking: file-backed inbound TLS/QUIC identities, including legacy `cert`/`key` paths and file-backed named PEMs, now rotate live and require currently valid, leaf-to-issuer certificate chains. Expired or misordered bundles can fail startup after upgrade, and expired sources stop serving new handshakes even to clients using fingerprint pinning or disabled verification. Renew certificates, remove expired cross-signs and certificates outside that chain, and synchronize the system clock before starting Shoes. Child-to-issuer signatures must use TLS-provider-supported algorithms (no SHA-1; RSA issuer keys of at least 2048 bits); reissue certificates with unsupported signatures. Inline-only pairs retain their previous behavior. See [certificate sources](CONFIG.md#certificate-sources).
|
||||
|
||||
## v0.3.4
|
||||
|
||||
@@ -328,7 +328,7 @@ Opt into manual selection explicitly:
|
||||
|
||||
```yaml
|
||||
- client_group: outbound
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: east
|
||||
client_proxies:
|
||||
- name: east
|
||||
@@ -341,7 +341,8 @@ Opt into manual selection explicitly:
|
||||
|
||||
Rules continue to use `client_chain: outbound`. A selector chooses an immediate
|
||||
named member or referenced group; inline selector members require `name`.
|
||||
Other groups default to `strategy: round-robin`. Nested round-robin weights,
|
||||
The group discriminant is `type`: `round-robin` (the default), `selector`, or
|
||||
`url-test`. The removed `strategy` field and unknown types are errors. Nested round-robin weights,
|
||||
duplicate membership, and existing UDP filtering are preserved. A selected
|
||||
branch without UDP support fails new UDP flows instead of silently falling back.
|
||||
Selection changes affect new flows only; existing streams, UDP destinations,
|
||||
@@ -349,6 +350,93 @@ DNS connections, and WireGuard associations remain pinned until reopened.
|
||||
Optional client and server/TUN `name` fields provide stable dashboard identities.
|
||||
Generated anonymous names are stable only for unchanged definition sites.
|
||||
|
||||
Automatic selection does not require a controller listener or a cache file:
|
||||
|
||||
```yaml
|
||||
- client_group: fastest
|
||||
type: url-test
|
||||
url: "https://probe.example.com/generate_204"
|
||||
interval: 300 # seconds, 1..86400
|
||||
timeout: 5000 # milliseconds, 1..60000, no longer than interval
|
||||
tolerance: 50 # milliseconds, 0..65535
|
||||
lazy: true
|
||||
client_proxies:
|
||||
- name: east
|
||||
address: "east.example.com:1080"
|
||||
protocol: {type: socks}
|
||||
- name: west
|
||||
address: "west.example.com:1080"
|
||||
protocol: {type: socks}
|
||||
```
|
||||
|
||||
Only `url` is required for `url-test`; the values above are the other defaults.
|
||||
Literal URL addresses denied by probe policy (including loopback, link-local,
|
||||
multicast, unspecified, and synthetic benchmark addresses) are configuration errors.
|
||||
Locally resolved hostnames are subject to the same destination policy at probe time.
|
||||
Literal URLs overlapping configured or retained fake-IP ranges are also rejected.
|
||||
`default` is selector-only. `url`, `interval`, `timeout`, `tolerance`, and `lazy`
|
||||
are url-test-only. Members must be nonempty and distinct; references to nested
|
||||
groups are supported. Explicit names are recommended for automatic inline members.
|
||||
|
||||
Selection uses the fastest fresh healthy result for the configured URL, among
|
||||
members eligible for that hop and transport. A healthy incumbent stays selected
|
||||
unless another member improves its delay by more than `tolerance`. TCP and UDP
|
||||
have separate incumbents; an automatic group can choose a UDP-capable alternative
|
||||
even when its fastest TCP member does not support UDP. Nested manual selectors
|
||||
still restrict eligibility, and intermediate chain hops only need TCP support.
|
||||
Changing selection affects new connections, not established streams or associations.
|
||||
|
||||
There is one initial check after activation, including for lazy groups. Thereafter
|
||||
`lazy: true` checks only groups used by new forwarding connections during the last
|
||||
interval. Probes, dashboard reads, and WireGuard carrier maintenance never count as use.
|
||||
New outbound connections through a WireGuard endpoint wake its carrier groups,
|
||||
including nested carriers, without changing their existing associations. Inbound-only
|
||||
endpoints and long-lived flows need `lazy: false` for continuous checks. `lazy: false` checks idle
|
||||
groups too. Samples older than `2 * interval + timeout` are stale and cannot win
|
||||
on delay. Without fresh healthy results (startup, all dead, stale, or locally
|
||||
unavailable probes), retain the eligible incumbent, or choose the first eligible
|
||||
member in configuration order. This is best-effort routing, not fail-closed routing.
|
||||
|
||||
Unambiguous remote connection errors request a rate-limited recheck (at most once
|
||||
per second); they do not mark the configured URL dead based on a different user
|
||||
destination. Local overload and unsupported operations do not penalize health.
|
||||
HTTP/2 adapters preserve matching transport-error causes for this classification.
|
||||
This is best effort when a protocol replaces a failure with a generic cancellation:
|
||||
for example, XHTTP upload cancellation can lose the original cause and count as a
|
||||
remote failure.
|
||||
User payloads are never retried across candidates. Probes use HTTP HEAD and accept
|
||||
any HTTP response status, without redirects; HTTPS certificates are verified.
|
||||
Local probe lookups use guarded system DNS, not a server's configured DNS group.
|
||||
API probes with the same normalized URL and wildcard/default expected status also
|
||||
update selection. Other URLs and explicit status expectations cannot overwrite it.
|
||||
|
||||
Periodic and API probes share a 16-probe limit. Each periodic round's timeout
|
||||
includes admission/queue time; members not admitted before the deadline retain
|
||||
their previous health. A deadline expiry after queueing is inconclusive and also
|
||||
preserves prior health; successful probes and explicit remote errors still update it.
|
||||
Members with no full-budget attempt, then the oldest full-budget attempts, lead each
|
||||
round, independently of health. Local failures cannot monopolize these slots, and
|
||||
queued attempts can receive a full budget on a later round. Tied members rotate on admission.
|
||||
Due groups with the oldest unqueued admission get priority, with stable catalog order
|
||||
breaking ties.
|
||||
Under sustained saturation, reduce group size, increase
|
||||
timeout/interval, or expect unprobed/stale members. Scheduling has one-second
|
||||
granularity and does not overlap rounds within a group; other groups can run while
|
||||
a slow round is in flight. Activation/reload cancels old work and rejects obsolete
|
||||
results; an interrupted round is immediately eligible when its generation resumes.
|
||||
New committed generations start with unprobed health.
|
||||
|
||||
The controller reports `type: URLTest`, effective TCP `now`, optional `udpNow`,
|
||||
`testUrl`, and selected-member `health` (`unprobed`, `healthy`, `dead`, or `stale`).
|
||||
Its `alive` is true only for fresh healthy results. Manual selection PUTs for
|
||||
automatic groups are rejected; automatic choices are neither pinned nor persisted.
|
||||
Leaf histories remain bounded to ten samples per URL and eight URLs. Automatic
|
||||
selection retains its configured-URL samples and full-budget attempt timestamps
|
||||
independently of API eviction; other attempt URLs are capped at eight per member
|
||||
for each of filtered and unfiltered probes.
|
||||
Automatic checks test each immediate member as a standalone outbound, not every
|
||||
surrounding chain or destination. URL-test groups are not supported by FFI.
|
||||
|
||||
Only one `cache_file_path` is allowed across all input files. Its absence disables
|
||||
persistence; relative paths resolve against the process working directory.
|
||||
The file is Shoes-specific versioned JSON, not a sing-box/Mihomo database. Parent
|
||||
@@ -394,7 +482,7 @@ are exempt from header idle expiry and have bounded queues/write deadlines.
|
||||
Responses are limited to 32 MiB. An oversized connection snapshot closes its
|
||||
WebSocket with code 1009 and reason `response exceeds 32 MiB`; tracking continues
|
||||
without truncating the connection list.
|
||||
Configuring a controller, `cache_file_path`, or any selector group enables the
|
||||
Configuring a controller, `cache_file_path`, or any selector/url-test group enables the
|
||||
catalog and its 16,384-node limit. Names must be nonempty, unique, at most 256
|
||||
bytes, and contain no control characters. `DIRECT`, `REJECT`, and
|
||||
`GLOBAL` are reserved. Configuration preparation rejects `/rules` or base
|
||||
|
||||
@@ -60,7 +60,7 @@ same cipher and run v0.3.3 or newer for AES-256 or ChaCha20.
|
||||
- **TLS fingerprint authentication**: Certificate pinning for TLS/QUIC
|
||||
- **Hot reloading**: Apply config changes without restart
|
||||
- **Unix socket support**: Bind to Unix domain sockets
|
||||
- **Optional Clash controller**: Named selectors, connection monitoring/cancellation, outbound delay tests, and optional selector persistence ([configuration](CONFIG.md#clash-controller), [example](examples/clash_api.yaml))
|
||||
- **Upstream groups and optional Clash controller**: Round-robin, manual selectors, and automatic URL tests (without an API listener); connection monitoring/cancellation and optional selector persistence ([configuration](CONFIG.md#clash-controller), [example](examples/clash_api.yaml), [URL-test example](examples/url_test.yaml))
|
||||
|
||||
For advanced access control (IP allowlist/blocklists), see [tobaru](https://github.com/cfal/tobaru).
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
- cache_file_path: "./cache.db"
|
||||
|
||||
- client_group: outbound
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: us-east
|
||||
client_proxies:
|
||||
- name: us-east
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# Replace the probe URL and upstream addresses with your own endpoints.
|
||||
- client_group: fastest
|
||||
type: url-test
|
||||
url: "https://probe.example.com/generate_204"
|
||||
interval: 300
|
||||
timeout: 5000
|
||||
tolerance: 50
|
||||
lazy: true
|
||||
client_proxies:
|
||||
- name: east
|
||||
address: "east.example.com:443"
|
||||
protocol:
|
||||
type: vless
|
||||
user_id: "550e8400-e29b-41d4-a716-446655440000"
|
||||
- name: west
|
||||
address: "west.example.com:443"
|
||||
protocol:
|
||||
type: vless
|
||||
user_id: "550e8400-e29b-41d4-a716-446655440000"
|
||||
|
||||
- address: "127.0.0.1:1080"
|
||||
protocol: {type: socks}
|
||||
rules:
|
||||
- mask: "0.0.0.0/0"
|
||||
action: allow
|
||||
client_chain: fastest
|
||||
+113
-27
@@ -1,11 +1,12 @@
|
||||
use serde_json::{Value, json};
|
||||
use std::collections::HashMap;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::io;
|
||||
|
||||
use crate::config::{ClientChain, ClientChainHop, ConfigSelection, GroupStrategy};
|
||||
use crate::config::{ClientChain, ClientChainHop, ConfigSelection, GroupType};
|
||||
use crate::control::catalog::{Catalog, CatalogNode, protocol_type};
|
||||
use crate::control::runtime::Generation;
|
||||
use crate::control::selection::Choices;
|
||||
use crate::control::url_test::AutomaticGroup;
|
||||
use crate::option_util::OneOrSome;
|
||||
|
||||
pub fn leaf_udp_support(catalog: &Catalog) -> io::Result<HashMap<String, bool>> {
|
||||
@@ -27,16 +28,20 @@ pub fn leaf_udp_support(catalog: &Catalog) -> io::Result<HashMap<String, bool>>
|
||||
|
||||
pub fn proxy(generation: &Generation, choices: &Choices, name: &str) -> Option<Value> {
|
||||
let node = generation.catalog.nodes.get(name)?;
|
||||
let udp = supports_udp(generation, choices, name, 0);
|
||||
let udp = supports_udp(generation, choices, name);
|
||||
let mut value = match node {
|
||||
CatalogNode::Leaf(client) => {
|
||||
json!({"name":name,"type":protocol_type(&client.protocol),"udp":udp})
|
||||
}
|
||||
CatalogNode::Reject => json!({"name":name,"type":"Reject","udp":false}),
|
||||
CatalogNode::Group { members, strategy } => {
|
||||
let proxy_type = match strategy {
|
||||
GroupStrategy::Selector => "Selector",
|
||||
GroupStrategy::RoundRobin => "LoadBalance",
|
||||
CatalogNode::Group {
|
||||
members,
|
||||
group_type,
|
||||
} => {
|
||||
let proxy_type = match group_type {
|
||||
GroupType::Selector => "Selector",
|
||||
GroupType::RoundRobin => "LoadBalance",
|
||||
GroupType::UrlTest(_) => "URLTest",
|
||||
};
|
||||
let mut value = json!({
|
||||
"name": name,
|
||||
@@ -51,31 +56,66 @@ pub fn proxy(generation: &Generation, choices: &Choices, name: &str) -> Option<V
|
||||
}
|
||||
};
|
||||
generation.probes.decorate(name, &mut value);
|
||||
if let Some(group) = generation.catalog.selection.automatic.0.get(name) {
|
||||
let selected = automatic_member(generation, choices, group, false)?;
|
||||
let health = group.health(&selected);
|
||||
value["now"] = json!(selected);
|
||||
value["udpNow"] = json!(automatic_member(generation, choices, group, true));
|
||||
value["testUrl"] = json!(group.config.url);
|
||||
value["health"] = json!(health);
|
||||
value["alive"] = json!(health == "healthy");
|
||||
value["history"] = generation.probes.url_history(&selected, &group.config.url);
|
||||
}
|
||||
Some(value)
|
||||
}
|
||||
|
||||
fn supports_udp(generation: &Generation, choices: &Choices, name: &str, depth: usize) -> bool {
|
||||
if depth > 128 {
|
||||
return false;
|
||||
}
|
||||
match generation.catalog.nodes.get(name) {
|
||||
Some(CatalogNode::Group { members, strategy }) => {
|
||||
if *strategy == GroupStrategy::Selector {
|
||||
choices
|
||||
.get(name)
|
||||
.is_some_and(|name| supports_udp(generation, choices, name, depth + 1))
|
||||
} else {
|
||||
members
|
||||
.iter()
|
||||
.any(|name| supports_udp(generation, choices, name, depth + 1))
|
||||
fn automatic_member(
|
||||
generation: &Generation,
|
||||
choices: &Choices,
|
||||
group: &AutomaticGroup,
|
||||
udp: bool,
|
||||
) -> Option<String> {
|
||||
let eligible = group
|
||||
.members
|
||||
.iter()
|
||||
.filter(|name| !udp || supports_udp(generation, choices, name))
|
||||
.map(String::as_str)
|
||||
.collect();
|
||||
group.choose(&eligible, udp)
|
||||
}
|
||||
|
||||
fn supports_udp(generation: &Generation, choices: &Choices, name: &str) -> bool {
|
||||
let mut pending = vec![name];
|
||||
let mut visited = HashSet::new();
|
||||
while let Some(name) = pending.pop() {
|
||||
if !visited.insert(name) {
|
||||
continue;
|
||||
}
|
||||
match generation.catalog.nodes.get(name) {
|
||||
Some(CatalogNode::Group {
|
||||
members,
|
||||
group_type,
|
||||
}) => {
|
||||
if *group_type == GroupType::Selector {
|
||||
if let Some(selected) = choices.get(name) {
|
||||
pending.push(selected);
|
||||
}
|
||||
} else {
|
||||
pending.extend(members.iter().map(String::as_str));
|
||||
}
|
||||
}
|
||||
_ if generation
|
||||
.leaf_udp_support
|
||||
.get(name)
|
||||
.copied()
|
||||
.unwrap_or(false) =>
|
||||
{
|
||||
return true;
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
_ => generation
|
||||
.leaf_udp_support
|
||||
.get(name)
|
||||
.copied()
|
||||
.unwrap_or(false),
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
pub fn proxies(generation: &Generation) -> Value {
|
||||
@@ -107,6 +147,52 @@ mod tests {
|
||||
use crate::control::selection::SelectionOwner;
|
||||
use std::sync::Arc;
|
||||
|
||||
#[test]
|
||||
fn url_test_dto_reports_transport_eligible_members_and_url_health() {
|
||||
let configs = serde_json::from_value(json!([
|
||||
{"client_group":"manual", "type":"selector", "default":"tcp", "client_proxies":[
|
||||
"direct", {"name":"tcp", "address":"proxy.invalid:80", "protocol":{"type":"http"}}
|
||||
]},
|
||||
{"client_group":"auto", "type":"url-test", "url":"http://probe.invalid/", "client_proxies":[
|
||||
"manual", {"name":"udp", "protocol":{"type":"direct"}}
|
||||
]}
|
||||
])).unwrap();
|
||||
let catalog = crate::config::create_server_configs(configs)
|
||||
.unwrap()
|
||||
.catalog
|
||||
.unwrap();
|
||||
let generation = Generation {
|
||||
id: 1,
|
||||
leaf_udp_support: leaf_udp_support(&catalog).unwrap(),
|
||||
catalog,
|
||||
rules: Vec::new(),
|
||||
probes: super::super::probes::ProbeState::new(
|
||||
Arc::new(crate::resolver::NativeResolver::new()),
|
||||
Vec::new(),
|
||||
),
|
||||
};
|
||||
let mut owner = SelectionOwner::new(None).unwrap();
|
||||
owner
|
||||
.activate(generation.catalog.selection.clone())
|
||||
.unwrap();
|
||||
let node = proxies(&generation)["proxies"]["auto"].clone();
|
||||
assert_eq!(node["now"], "manual");
|
||||
assert_eq!(node["udpNow"], "udp");
|
||||
assert_eq!(node["health"], "unprobed");
|
||||
assert_eq!(node["alive"], false);
|
||||
let automatic = &generation.catalog.selection.automatic;
|
||||
automatic.record("manual", "http://probe.invalid/", Some(5));
|
||||
automatic.record("udp", "http://probe.invalid/", Some(100));
|
||||
let node = proxies(&generation)["proxies"]["auto"].clone();
|
||||
assert_eq!(node["type"], "URLTest");
|
||||
assert_eq!(node["now"], "manual");
|
||||
assert_eq!(node["udpNow"], "udp");
|
||||
assert_eq!(node["udp"], true);
|
||||
assert_eq!(node["health"], "healthy");
|
||||
owner.select("manual", "DIRECT").unwrap();
|
||||
assert_eq!(proxies(&generation)["proxies"]["auto"]["udpNow"], "manual");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn catalog_dtos_preserve_group_types_members_and_selected_udp_support() {
|
||||
let configs = serde_json::from_value(json!([
|
||||
@@ -114,7 +200,7 @@ mod tests {
|
||||
{"name":"east", "address":"east.invalid:1080", "protocol":{"type":"socks"}},
|
||||
"direct"
|
||||
]},
|
||||
{"client_group":"outbound", "strategy":"selector", "default":"child", "client_proxies":[
|
||||
{"client_group":"outbound", "type":"selector", "default":"child", "client_proxies":[
|
||||
"child",
|
||||
{"name":"west", "address":"west.invalid:1080", "protocol":{"type":"socks"}}
|
||||
]}
|
||||
|
||||
+1820
-39
File diff suppressed because it is too large
Load Diff
@@ -9,7 +9,7 @@ use serde_json::json;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
use super::{ApiState, Body, auth, dto, error, full, no_content, probes, streams};
|
||||
use crate::config::{ClashApiConfig, GroupStrategy};
|
||||
use crate::config::{ClashApiConfig, GroupType};
|
||||
use crate::control::catalog::CatalogNode;
|
||||
|
||||
pub async fn serve(
|
||||
@@ -228,7 +228,7 @@ async fn route(
|
||||
match generation.catalog.nodes.get(*name) {
|
||||
None => return error(StatusCode::NOT_FOUND, "Not found"),
|
||||
Some(CatalogNode::Group {
|
||||
strategy: GroupStrategy::Selector,
|
||||
group_type: GroupType::Selector,
|
||||
..
|
||||
}) => {}
|
||||
_ => return error(StatusCode::BAD_REQUEST, "Must be a Selector"),
|
||||
|
||||
+233
-10
@@ -67,6 +67,12 @@ impl std::fmt::Debug for InitialHopEntry {
|
||||
}
|
||||
|
||||
impl InitialHopEntry {
|
||||
fn socket(&self) -> &Arc<dyn SocketConnector> {
|
||||
match self {
|
||||
Self::Direct(socket) | Self::Wireguard(socket) | Self::Proxy { socket, .. } => socket,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns true if this entry supports UDP.
|
||||
pub fn supports_udp(&self) -> bool {
|
||||
match self {
|
||||
@@ -142,11 +148,7 @@ impl ClientProxyChain {
|
||||
.map(|(i, hop)| hop[indices[i + 1]].clone())
|
||||
.collect();
|
||||
SelectedRoute {
|
||||
socket: match self.initial_hop[initial_index].as_ref() {
|
||||
InitialHopEntry::Direct(socket)
|
||||
| InitialHopEntry::Wireguard(socket)
|
||||
| InitialHopEntry::Proxy { socket, .. } => socket.clone(),
|
||||
},
|
||||
socket: self.initial_hop[initial_index].socket().clone(),
|
||||
initial: Some(self.initial_hop[initial_index].clone()),
|
||||
subsequent,
|
||||
resolver: resolver.clone(),
|
||||
@@ -203,6 +205,25 @@ impl ClientProxyChain {
|
||||
.is_none_or(|origin| origin.allowed(choices))
|
||||
})
|
||||
};
|
||||
if let Some(selection) = &self.selection
|
||||
&& !selection.automatic.0.is_empty()
|
||||
&& choices.is_some()
|
||||
{
|
||||
let mut candidates: Vec<_> =
|
||||
(0..length).filter(allowed).map(candidate).collect();
|
||||
selection
|
||||
.automatic
|
||||
.filter(&self.origins[hop], &mut candidates, final_udp);
|
||||
if candidates.is_empty() {
|
||||
return Err(std::io::Error::new(
|
||||
std::io::ErrorKind::Unsupported,
|
||||
"selected branch cannot carry this flow",
|
||||
));
|
||||
}
|
||||
let ordinal =
|
||||
counter.fetch_add(1, Ordering::Relaxed) as usize % candidates.len();
|
||||
return Ok(candidates[ordinal]);
|
||||
}
|
||||
let count = if choices.is_some() {
|
||||
(0..length).filter(allowed).count()
|
||||
} else {
|
||||
@@ -228,6 +249,57 @@ impl ClientProxyChain {
|
||||
self.origins.get(hop)?.get(index)?.as_ref()
|
||||
}
|
||||
|
||||
fn touch_automatic(&self, indices: &[usize], failed: bool) {
|
||||
if self
|
||||
.selection
|
||||
.as_ref()
|
||||
.is_none_or(|selection| selection.automatic.0.is_empty())
|
||||
{
|
||||
return;
|
||||
}
|
||||
for (hop, &index) in indices.iter().enumerate() {
|
||||
if let Some(origin) = self.origin(hop, index) {
|
||||
for (group, _) in &origin.path {
|
||||
origin.selection.automatic.touch(group, failed);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn with_route_activity<T>(
|
||||
&self,
|
||||
indices: &[usize],
|
||||
connect: impl Future<Output = std::io::Result<T>>,
|
||||
) -> std::io::Result<T> {
|
||||
self.touch_automatic(indices, false);
|
||||
self.initial_hop[indices[0]].socket().record_demand();
|
||||
let result = connect.await;
|
||||
if let Err(error) = &result
|
||||
&& crate::control::url_test::remote_failure(error)
|
||||
{
|
||||
self.touch_automatic(indices, true);
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
pub(crate) fn record_carrier_demand(&self) {
|
||||
if crate::control::url_test::is_background() {
|
||||
return;
|
||||
}
|
||||
// Shared carriers can have several pinned associations. Wake their groups
|
||||
// without selecting a new route or attributing a destination failure to them.
|
||||
for origin in self.origins.iter().flatten().flatten() {
|
||||
for (group, _) in &origin.path {
|
||||
origin.selection.automatic.touch(group, false);
|
||||
}
|
||||
}
|
||||
for entry in &self.initial_hop {
|
||||
if let InitialHopEntry::Wireguard(socket) = entry.as_ref() {
|
||||
socket.record_demand();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn route_names(&self, indices: &[usize]) -> Vec<String> {
|
||||
indices
|
||||
.iter()
|
||||
@@ -352,9 +424,22 @@ impl ClientProxyChain {
|
||||
if let Some(path) = path {
|
||||
*path = self.route_names(&indices);
|
||||
}
|
||||
self.with_route_activity(
|
||||
&indices,
|
||||
self.connect_tcp_selected(remote_location, resolver, &indices),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn connect_tcp_selected(
|
||||
&self,
|
||||
remote_location: ResolvedLocation,
|
||||
resolver: &Arc<dyn Resolver>,
|
||||
indices: &[usize],
|
||||
) -> std::io::Result<TcpClientSetupResult> {
|
||||
if self.needs_carrier_dialer() {
|
||||
return self
|
||||
.selected_route(resolver, &indices)
|
||||
.selected_route(resolver, indices)
|
||||
.connect(remote_location)
|
||||
.await;
|
||||
}
|
||||
@@ -362,7 +447,7 @@ impl ClientProxyChain {
|
||||
let entry = self.initial_hop[indices[0]].as_ref();
|
||||
|
||||
// Select proxy connectors for subsequent hops
|
||||
let subsequent_proxies = self.select_subsequent_proxies(&indices);
|
||||
let subsequent_proxies = self.select_subsequent_proxies(indices);
|
||||
|
||||
debug!(
|
||||
"Chain TCP connect: 1 initial + {} subsequent hop(s) -> {}",
|
||||
@@ -485,8 +570,21 @@ impl ClientProxyChain {
|
||||
if let Some(path) = path {
|
||||
*path = self.route_names(&indices);
|
||||
}
|
||||
self.with_route_activity(
|
||||
&indices,
|
||||
self.connect_udp_selected(resolver, target, &indices),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn connect_udp_selected(
|
||||
&self,
|
||||
resolver: &Arc<dyn Resolver>,
|
||||
target: ResolvedLocation,
|
||||
indices: &[usize],
|
||||
) -> std::io::Result<Box<dyn AsyncMessageStream>> {
|
||||
if self.needs_carrier_dialer() {
|
||||
let route = self.selected_route(resolver, &indices);
|
||||
let route = self.selected_route(resolver, indices);
|
||||
return match route.last_proxy() {
|
||||
Some(proxy) => {
|
||||
proxy
|
||||
@@ -693,6 +791,130 @@ mod carrier_tests {
|
||||
|
||||
const TRANSPORT_DATA: &[u8] = b"transport";
|
||||
|
||||
#[test]
|
||||
fn url_test_nested_groups_and_each_chain_hop_preserve_udp_eligibility() {
|
||||
for route in ["outer", "[inner, inner]"] {
|
||||
let yaml = format!(
|
||||
"
|
||||
- client_group: inner
|
||||
type: url-test
|
||||
url: http://inner.invalid/
|
||||
client_proxies:
|
||||
- name: http
|
||||
address: 127.0.0.1:8000
|
||||
protocol: {{type: http}}
|
||||
- name: udp
|
||||
address: 127.0.0.1:8001
|
||||
protocol: {{type: vless, user_id: 550e8400-e29b-41d4-a716-446655440000}}
|
||||
- client_group: manual
|
||||
type: selector
|
||||
client_proxies: [inner, direct]
|
||||
- client_group: outer
|
||||
type: url-test
|
||||
url: http://outer.invalid/
|
||||
client_proxies: [manual, direct]
|
||||
- address: 127.0.0.1:1080
|
||||
protocol: {{type: socks}}
|
||||
rules: [{{mask: 0.0.0.0/0, action: allow, client_chain: {route}}}]
|
||||
"
|
||||
);
|
||||
let mut validated =
|
||||
crate::config::create_server_configs(serde_yaml::from_str(&yaml).unwrap()).unwrap();
|
||||
assert!(validated.control.listeners.is_empty());
|
||||
let catalog = validated.catalog.take().unwrap();
|
||||
let crate::config::Config::Server(server) = validated.configs.remove(0) else {
|
||||
panic!()
|
||||
};
|
||||
let crate::config::ConfigSelection::Config(rule) = server.rules.into_vec().remove(0)
|
||||
else {
|
||||
panic!()
|
||||
};
|
||||
let crate::config::RuleActionConfig::Allow { client_chains, .. } = rule.action else {
|
||||
panic!()
|
||||
};
|
||||
let resolver: Arc<dyn Resolver> = Arc::new(crate::resolver::NativeResolver::new());
|
||||
let group =
|
||||
crate::tcp::chain_builder::try_build_client_chain_group(client_chains, resolver)
|
||||
.unwrap();
|
||||
let chain = &group.chains[0];
|
||||
let auto = &catalog.selection.automatic;
|
||||
auto.record("http", "http://inner.invalid/", Some(10));
|
||||
auto.record("udp", "http://inner.invalid/", Some(100));
|
||||
auto.record("manual", "http://outer.invalid/", Some(10));
|
||||
auto.record("DIRECT", "http://outer.invalid/", Some(200));
|
||||
if route == "outer" {
|
||||
assert_eq!(chain.select_indices(false).unwrap(), [0]);
|
||||
assert_eq!(chain.select_indices(true).unwrap(), [1]);
|
||||
let mut owner = crate::control::selection::SelectionOwner::new(None).unwrap();
|
||||
owner.activate(catalog.selection.clone()).unwrap();
|
||||
assert!(owner.select("inner", "udp").is_err());
|
||||
owner.select("manual", "DIRECT").unwrap();
|
||||
assert_eq!(chain.select_indices(false).unwrap(), [2]);
|
||||
assert_eq!(chain.select_indices(true).unwrap(), [2]);
|
||||
} else {
|
||||
assert_eq!(chain.select_indices(false).unwrap(), [0, 0]);
|
||||
assert_eq!(chain.select_indices(true).unwrap(), [0, 1]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_test_deep_udp_selection_survives_dashboard_reads() {
|
||||
use serde_json::json;
|
||||
let mut configs: Vec<_> = (0..128)
|
||||
.map(|index| {
|
||||
let member = if index == 127 {
|
||||
"direct".into()
|
||||
} else {
|
||||
format!("deep-{}", index + 1)
|
||||
};
|
||||
json!({
|
||||
"client_group": format!("deep-{index}"),
|
||||
"client_proxies": [member]
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
configs.push(json!({
|
||||
"client_group":"auto", "type":"url-test", "url":"http://probe.invalid/",
|
||||
"client_proxies":["deep-0", {"name":"other", "protocol":{"type":"direct"}}]
|
||||
}));
|
||||
configs.push(json!({
|
||||
"address":"127.0.0.1:1080", "protocol":{"type":"socks"},
|
||||
"rules":[{"mask":"0.0.0.0/0", "action":"allow", "client_chain":"auto"}]
|
||||
}));
|
||||
let mut validated =
|
||||
crate::config::create_server_configs(serde_json::from_value(json!(configs)).unwrap())
|
||||
.unwrap();
|
||||
let catalog = validated.catalog.take().unwrap();
|
||||
let crate::config::Config::Server(server) = validated.configs.remove(0) else {
|
||||
panic!()
|
||||
};
|
||||
let crate::config::ConfigSelection::Config(rule) = server.rules.into_vec().remove(0) else {
|
||||
panic!()
|
||||
};
|
||||
let crate::config::RuleActionConfig::Allow { client_chains, .. } = rule.action else {
|
||||
panic!()
|
||||
};
|
||||
let resolver: Arc<dyn Resolver> = Arc::new(crate::resolver::NativeResolver::new());
|
||||
let group = crate::tcp::chain_builder::try_build_client_chain_group(
|
||||
client_chains,
|
||||
resolver.clone(),
|
||||
)
|
||||
.unwrap();
|
||||
let generation = crate::control::runtime::Generation {
|
||||
id: 1,
|
||||
leaf_udp_support: crate::clash_api::dto::leaf_udp_support(&catalog).unwrap(),
|
||||
catalog,
|
||||
rules: Vec::new(),
|
||||
probes: crate::clash_api::probes::ProbeState::new(resolver, Vec::new()),
|
||||
};
|
||||
let chain = &group.chains[0];
|
||||
assert_eq!(chain.select_indices(true).unwrap(), [0]);
|
||||
let nodes = crate::clash_api::dto::proxies(&generation);
|
||||
assert_eq!(chain.select_indices(true).unwrap(), [0]);
|
||||
assert_eq!(nodes["proxies"]["auto"]["udpNow"], "deep-0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unselected_large_pools_keep_constant_time_round_robin() {
|
||||
let events = Arc::new(Mutex::new(Vec::new()));
|
||||
@@ -723,7 +945,7 @@ mod carrier_tests {
|
||||
- client_group: weighted
|
||||
client_proxies: [leaves, leaves, direct]
|
||||
- client_group: choose
|
||||
strategy: selector
|
||||
type: selector
|
||||
client_proxies: [weighted, direct]
|
||||
- address: 127.0.0.1:1080
|
||||
protocol: {type: socks}
|
||||
@@ -763,7 +985,7 @@ mod carrier_tests {
|
||||
fn udp_selection_rejects_ineligible_member_without_falling_back() {
|
||||
let yaml = "
|
||||
- client_group: leaves
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: http
|
||||
client_proxies:
|
||||
- name: http
|
||||
@@ -1226,6 +1448,7 @@ impl ClientChainGroup {
|
||||
let index = self.next_udp_index.fetch_add(1, Ordering::Relaxed) as usize;
|
||||
let chain = &self.chains[index % self.chains.len()];
|
||||
let indices = chain.select_indices(true)?;
|
||||
chain.touch_automatic(&indices, false);
|
||||
let InitialHopEntry::Direct(socket) = chain.initial_hop[indices[0]].as_ref() else {
|
||||
unreachable!("direct-only chain checked above")
|
||||
};
|
||||
|
||||
@@ -167,6 +167,19 @@ pub(super) fn validate(config: &mut ValidatedConfigs) -> io::Result<()> {
|
||||
|
||||
impl ValidatedConfigs {
|
||||
pub(crate) fn validate_network_endpoints(&mut self, ranges: &[IpNetwork]) -> io::Result<()> {
|
||||
if let Some(catalog) = &self.catalog {
|
||||
for group in catalog.selection.automatic.0.values() {
|
||||
let url = url::Url::parse(&group.config.url).map_err(io::Error::other)?;
|
||||
let ip = match url.host() {
|
||||
Some(url::Host::Ipv4(ip)) => Some(IpAddr::V4(ip)),
|
||||
Some(url::Host::Ipv6(ip)) => Some(IpAddr::V6(ip)),
|
||||
_ => None,
|
||||
};
|
||||
if let Some(ip) = ip {
|
||||
reject(ranges, ip, "url-test URL")?;
|
||||
}
|
||||
}
|
||||
}
|
||||
for config in &self.configs {
|
||||
match config {
|
||||
Config::Server(server) => {
|
||||
@@ -219,6 +232,38 @@ mod tests {
|
||||
super::super::create_server_configs(serde_yaml::from_str(yaml).map_err(io::Error::other)?)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_test_literals_reject_active_and_retained_custom_fake_ranges() {
|
||||
use serde_json::json;
|
||||
|
||||
for url in [
|
||||
"http://100.64.0.2/",
|
||||
"http://[::ffff:100.64.0.2]/",
|
||||
"http://[fd00::2]/",
|
||||
] {
|
||||
let mut configs = json!([
|
||||
{"dns_group":"fake", "dns_servers":"system", "fake_ip":{"ipv4_range":"100.64.0.0/10", "ipv6_range":"fd00::/64"}},
|
||||
{"client_group":"auto", "type":"url-test", "url":url, "client_proxies":["direct"]}
|
||||
]);
|
||||
let error = validate_yaml(&configs.to_string())
|
||||
.err()
|
||||
.expect("active probe range must be rejected");
|
||||
assert!(error.to_string().contains("url-test URL"), "{error}");
|
||||
configs[0]["fake_ip"] = false.into();
|
||||
let mut validated = validate_yaml(&configs.to_string()).unwrap();
|
||||
let error = validated
|
||||
.validate_network_endpoints(&[
|
||||
"100.64.0.0/10".parse().unwrap(),
|
||||
"fd00::/64".parse().unwrap(),
|
||||
])
|
||||
.unwrap_err();
|
||||
assert!(error.to_string().contains("url-test URL"), "{error}");
|
||||
validated
|
||||
.validate_network_endpoints(&["192.0.2.0/24".parse().unwrap()])
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tun_endpoint_iteration_preserves_order_duplicates_and_address_form() {
|
||||
let configs = serde_yaml::from_str::<Vec<Config>>(
|
||||
|
||||
+218
-9
@@ -17,23 +17,232 @@ use super::tun::TunConfig;
|
||||
/// Groups can reference other groups using `ConfigSelection::GroupName`.
|
||||
/// Group references are resolved using topological sort during validation.
|
||||
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
#[serde(try_from = "RawClientGroup")]
|
||||
pub struct ClientConfigGroup {
|
||||
pub client_group: String,
|
||||
#[serde(default)]
|
||||
pub strategy: GroupStrategy,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
#[serde(flatten)]
|
||||
pub group_type: GroupType,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub default: Option<String>,
|
||||
#[serde(alias = "client_proxy")]
|
||||
pub client_proxies: OneOrSome<ConfigSelection<ClientConfig>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub enum GroupStrategy {
|
||||
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize)]
|
||||
#[serde(tag = "type", rename_all = "kebab-case")]
|
||||
pub enum GroupType {
|
||||
#[default]
|
||||
RoundRobin,
|
||||
Selector,
|
||||
UrlTest(UrlTestConfig),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
|
||||
pub struct UrlTestConfig {
|
||||
pub url: String,
|
||||
pub interval: u64,
|
||||
pub timeout: u64,
|
||||
pub tolerance: u16,
|
||||
pub lazy: bool,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct RawClientGroup {
|
||||
client_group: String,
|
||||
#[serde(rename = "type", default = "default_group_type")]
|
||||
kind: String,
|
||||
default: Option<String>,
|
||||
#[serde(alias = "client_proxy")]
|
||||
client_proxies: OneOrSome<ConfigSelection<ClientConfig>>,
|
||||
url: Option<String>,
|
||||
interval: Option<u64>,
|
||||
timeout: Option<u64>,
|
||||
tolerance: Option<u16>,
|
||||
lazy: Option<bool>,
|
||||
}
|
||||
|
||||
fn default_group_type() -> String {
|
||||
"round-robin".into()
|
||||
}
|
||||
|
||||
impl TryFrom<RawClientGroup> for ClientConfigGroup {
|
||||
type Error = String;
|
||||
|
||||
fn try_from(raw: RawClientGroup) -> Result<Self, Self::Error> {
|
||||
let kind = raw.kind.as_str();
|
||||
if kind != "selector" && raw.default.is_some() {
|
||||
return Err("default is only valid for selector groups".into());
|
||||
}
|
||||
if kind != "url-test"
|
||||
&& (raw.url.is_some()
|
||||
|| raw.interval.is_some()
|
||||
|| raw.timeout.is_some()
|
||||
|| raw.tolerance.is_some()
|
||||
|| raw.lazy.is_some())
|
||||
{
|
||||
return Err("url, interval, timeout, tolerance and lazy require type: url-test".into());
|
||||
}
|
||||
let group_type = match kind {
|
||||
"round-robin" => GroupType::RoundRobin,
|
||||
"selector" => GroupType::Selector,
|
||||
"url-test" => {
|
||||
let config = UrlTestConfig {
|
||||
url: raw.url.ok_or("url-test requires url")?,
|
||||
interval: raw.interval.unwrap_or(300),
|
||||
timeout: raw.timeout.unwrap_or(5000),
|
||||
tolerance: raw.tolerance.unwrap_or(50),
|
||||
lazy: raw.lazy.unwrap_or(true),
|
||||
};
|
||||
config.validate().map_err(|error| error.to_string())?;
|
||||
GroupType::UrlTest(config)
|
||||
}
|
||||
_ => return Err(format!("unknown client group type: {kind}")),
|
||||
};
|
||||
Ok(Self {
|
||||
client_group: raw.client_group,
|
||||
group_type,
|
||||
default: raw.default,
|
||||
client_proxies: raw.client_proxies,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl UrlTestConfig {
|
||||
pub fn validate(&self) -> std::io::Result<()> {
|
||||
if !(1..=86_400).contains(&self.interval)
|
||||
|| !(1..=60_000).contains(&self.timeout)
|
||||
|| self.timeout > self.interval * 1000
|
||||
{
|
||||
return Err(std::io::Error::new(
|
||||
std::io::ErrorKind::InvalidInput,
|
||||
"url-test interval must be 1..86400 seconds; timeout 1..60000 ms and no longer than interval",
|
||||
));
|
||||
}
|
||||
crate::clash_api::probes::ProbeQuery::parse(
|
||||
&std::collections::HashMap::from([
|
||||
("url".into(), self.url.clone()),
|
||||
("timeout".into(), self.timeout.to_string()),
|
||||
]),
|
||||
true,
|
||||
)?
|
||||
.validate_automatic_target()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod group_type_tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn url_test_defaults_and_serialization_use_only_type() {
|
||||
let raw = json!({"client_group":"auto", "type":"url-test", "url":"http://probe.invalid/", "client_proxies":["direct"]});
|
||||
let group: ClientConfigGroup = serde_json::from_value(raw).unwrap();
|
||||
assert_eq!(
|
||||
group.group_type,
|
||||
GroupType::UrlTest(UrlTestConfig {
|
||||
url: "http://probe.invalid/".into(),
|
||||
interval: 300,
|
||||
timeout: 5000,
|
||||
tolerance: 50,
|
||||
lazy: true,
|
||||
})
|
||||
);
|
||||
let encoded = serde_json::to_value(&group).unwrap();
|
||||
assert_eq!(encoded["type"], "url-test");
|
||||
assert!(encoded.get("strategy").is_none());
|
||||
assert_eq!(
|
||||
serde_json::from_value::<ClientConfigGroup>(encoded)
|
||||
.unwrap()
|
||||
.group_type,
|
||||
group.group_type
|
||||
);
|
||||
let group: ClientConfigGroup =
|
||||
serde_json::from_value(json!({"client_group":"rr", "client_proxies":["direct"]}))
|
||||
.unwrap();
|
||||
assert_eq!(group.group_type, GroupType::RoundRobin);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn client_proxy_alias_preserves_canonical_serialization() {
|
||||
let group: ClientConfigGroup = serde_json::from_value(json!({
|
||||
"client_group": "rr",
|
||||
"client_proxy": ["direct"]
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(group.group_type, GroupType::RoundRobin);
|
||||
assert!(group.default.is_none());
|
||||
assert_eq!(
|
||||
serde_json::to_value(group).unwrap(),
|
||||
json!({
|
||||
"client_group": "rr",
|
||||
"type": "round-robin",
|
||||
"client_proxies": ["direct"]
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_test_rejects_removed_unknown_and_incompatible_group_settings() {
|
||||
let base = json!({"client_group":"auto", "type":"url-test", "url":"http://probe.invalid/", "client_proxies":["direct"]});
|
||||
for (field, value) in [
|
||||
("strategy", json!("url-test")),
|
||||
("type", json!("urltest")),
|
||||
("type", json!(null)),
|
||||
("type", json!("fallback")),
|
||||
("default", json!("DIRECT")),
|
||||
("url", json!("ftp://probe.invalid/")),
|
||||
("url", json!("http://u:p@probe.invalid/")),
|
||||
("url", json!("http://127.0.0.1/")),
|
||||
("url", json!("http://[::ffff:127.0.0.1]/")),
|
||||
("url", json!("http://169.254.1.1/")),
|
||||
("url", json!("http://198.18.0.1/")),
|
||||
("url", json!("http://[::]/")),
|
||||
("url", json!("http://224.0.0.1/")),
|
||||
("interval", json!(0)),
|
||||
("interval", json!(86401)),
|
||||
("interval", json!(1)),
|
||||
("timeout", json!(0)),
|
||||
("timeout", json!(60001)),
|
||||
("tolerance", json!(-1)),
|
||||
("tolerance", json!(65536)),
|
||||
("lazy", json!("true")),
|
||||
("typo", json!(true)),
|
||||
] {
|
||||
let mut raw = base.clone();
|
||||
raw[field] = value;
|
||||
assert!(
|
||||
serde_json::from_value::<ClientConfigGroup>(raw.clone()).is_err(),
|
||||
"{raw}"
|
||||
);
|
||||
}
|
||||
for kind in ["round-robin", "selector"] {
|
||||
for (field, value) in [
|
||||
("url", json!("http://probe.invalid/")),
|
||||
("interval", json!(10)),
|
||||
("timeout", json!(10)),
|
||||
("tolerance", json!(0)),
|
||||
("lazy", json!(false)),
|
||||
("strategy", json!(kind)),
|
||||
] {
|
||||
let mut raw =
|
||||
json!({"client_group":"test", "type":kind, "client_proxies":["direct"]});
|
||||
raw[field] = value;
|
||||
assert!(
|
||||
serde_json::from_value::<ClientConfigGroup>(raw.clone()).is_err(),
|
||||
"{raw}"
|
||||
);
|
||||
}
|
||||
}
|
||||
for members in [json!([]), json!(["direct", "direct"])] {
|
||||
let mut raw = base.clone();
|
||||
raw["client_proxies"] = members;
|
||||
let configs = serde_json::from_value(json!([raw]));
|
||||
assert!(
|
||||
configs.is_err() || crate::config::create_server_configs(configs.unwrap()).is_err()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||
@@ -422,7 +631,7 @@ mod tests {
|
||||
#[test]
|
||||
fn test_client_config_group() {
|
||||
let original = vec![Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-client-group".to_string(),
|
||||
client_proxies: OneOrSome::Some(vec![
|
||||
|
||||
@@ -40,7 +40,9 @@ pub use client::{
|
||||
pub use common::DEFAULT_REALITY_SHORT_ID;
|
||||
pub use dns::{DnsConfig, DnsConfigGroup, DnsServerSpec, ExpandedDnsGroup, ExpandedDnsSpec};
|
||||
pub use global::GlobalLimits;
|
||||
pub use groups::{ClientConfigGroup, Config, GroupStrategy, NamedPem, PemSource, RuleGroupRules};
|
||||
pub use groups::{
|
||||
ClientConfigGroup, Config, GroupType, NamedPem, PemSource, RuleGroupRules, UrlTestConfig,
|
||||
};
|
||||
pub use rules::{ClientChain, ClientChainHop, RuleActionConfig, RuleConfig};
|
||||
pub use selection::ConfigSelection;
|
||||
pub use server::{
|
||||
|
||||
+20
-20
@@ -2961,7 +2961,7 @@ mod tests {
|
||||
))),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-group".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
@@ -2978,13 +2978,13 @@ mod tests {
|
||||
// group-b has no dependencies, group-a references group-b
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-a".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::GroupName("group-b".to_string())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-b".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
@@ -3000,13 +3000,13 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-a".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::GroupName("group-b".to_string())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-b".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::GroupName("group-a".to_string())),
|
||||
@@ -3027,7 +3027,7 @@ mod tests {
|
||||
use crate::config::types::ClientConfigGroup;
|
||||
|
||||
let configs = vec![Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-a".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::GroupName("nonexistent".to_string())),
|
||||
@@ -3049,25 +3049,25 @@ mod tests {
|
||||
// Diamond: a -> b, a -> c, b -> d, c -> d
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-d".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-c".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::GroupName("group-d".to_string())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-b".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::GroupName("group-d".to_string())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "group-a".to_string(),
|
||||
client_proxies: OneOrSome::Some(vec![
|
||||
@@ -3086,19 +3086,19 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "us-proxies".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "eu-proxies".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
}),
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "all-proxies".to_string(),
|
||||
client_proxies: OneOrSome::Some(vec![
|
||||
@@ -3577,7 +3577,7 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-proxy".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(socks_config)),
|
||||
@@ -3616,7 +3616,7 @@ mod tests {
|
||||
// Create a direct chain - should be allowed for UDP (for bind_interface).
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "direct-chain".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
@@ -3657,7 +3657,7 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-proxy".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(socks_config)),
|
||||
@@ -3696,7 +3696,7 @@ mod tests {
|
||||
// Create a direct chain - should be allowed for H3 (for bind_interface).
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "direct-chain".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
@@ -3728,7 +3728,7 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-proxy".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
@@ -3761,7 +3761,7 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-proxy".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
@@ -3794,7 +3794,7 @@ mod tests {
|
||||
|
||||
let configs = vec![
|
||||
Config::ClientConfigGroup(ClientConfigGroup {
|
||||
strategy: Default::default(),
|
||||
group_type: Default::default(),
|
||||
default: None,
|
||||
client_group: "test-proxy".to_string(),
|
||||
client_proxies: OneOrSome::One(ConfigSelection::Config(ClientConfig::default())),
|
||||
|
||||
@@ -970,8 +970,8 @@ mod tests {
|
||||
let mut configs = json!([
|
||||
{"clash_api_address":"127.0.0.1:9090","secret":"test"},
|
||||
{"client_group":"vpn","client_proxies":[client()]},
|
||||
{"client_group":"outbound","strategy":"selector","client_proxies":["vpn","direct"]},
|
||||
{"client_group":"another","strategy":"selector","client_proxies":["outbound","vpn"]}
|
||||
{"client_group":"outbound","type":"selector","client_proxies":["vpn","direct"]},
|
||||
{"client_group":"another","type":"selector","client_proxies":["outbound","vpn"]}
|
||||
]);
|
||||
assert!(
|
||||
validate(configs.clone())
|
||||
|
||||
+28
-15
@@ -8,7 +8,7 @@ use super::selection::{
|
||||
};
|
||||
use super::store::valid_name;
|
||||
use crate::config::{
|
||||
ClientChainHop, ClientConfig, ClientProxyConfig, Config, ConfigSelection, GroupStrategy,
|
||||
ClientChainHop, ClientConfig, ClientProxyConfig, Config, ConfigSelection, GroupType,
|
||||
RuleActionConfig,
|
||||
};
|
||||
|
||||
@@ -20,16 +20,19 @@ pub struct MemberOrigin {
|
||||
}
|
||||
|
||||
impl MemberOrigin {
|
||||
fn manages(&self, group: &str) -> bool {
|
||||
self.selection.definitions().contains_key(group)
|
||||
|| self.selection.automatic.0.contains_key(group)
|
||||
}
|
||||
|
||||
pub fn has_selector(&self) -> bool {
|
||||
self.path
|
||||
.iter()
|
||||
.any(|(group, _)| self.selection.definitions().contains_key(group))
|
||||
self.path.iter().any(|(group, _)| self.manages(group))
|
||||
}
|
||||
|
||||
pub fn selector_path(&self) -> Vec<(String, String)> {
|
||||
self.path
|
||||
.iter()
|
||||
.filter(|(group, _)| self.selection.definitions().contains_key(group))
|
||||
.filter(|(group, _)| self.manages(group))
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
@@ -77,7 +80,7 @@ pub struct RuleDescription {
|
||||
pub enum CatalogNode {
|
||||
Group {
|
||||
members: Vec<String>,
|
||||
strategy: GroupStrategy,
|
||||
group_type: GroupType,
|
||||
},
|
||||
Leaf(Box<ClientConfig>),
|
||||
Reject,
|
||||
@@ -159,7 +162,7 @@ impl Catalog {
|
||||
pub fn capture(configs: &mut [Config]) -> io::Result<Option<Self>> {
|
||||
for config in configs.iter() {
|
||||
if let Config::ClientConfigGroup(group) = config
|
||||
&& group.strategy != GroupStrategy::Selector
|
||||
&& group.group_type != GroupType::Selector
|
||||
&& group.default.is_some()
|
||||
{
|
||||
return Err(invalid("default is only valid for selector groups"));
|
||||
@@ -167,7 +170,7 @@ impl Catalog {
|
||||
}
|
||||
let enabled = configs.iter().any(|config| match config {
|
||||
Config::ClashApi(_) | Config::CacheFile(_) => true,
|
||||
Config::ClientConfigGroup(group) => group.strategy == GroupStrategy::Selector,
|
||||
Config::ClientConfigGroup(group) => group.group_type != GroupType::RoundRobin,
|
||||
_ => false,
|
||||
});
|
||||
if !enabled {
|
||||
@@ -199,7 +202,7 @@ impl Catalog {
|
||||
group.client_group.clone(),
|
||||
CatalogNode::Group {
|
||||
members: Vec::new(),
|
||||
strategy: group.strategy,
|
||||
group_type: group.group_type.clone(),
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -246,10 +249,11 @@ impl Catalog {
|
||||
)?;
|
||||
}
|
||||
let mut definitions = SelectorDefinitions::new();
|
||||
let mut automatic = super::url_test::AutomaticGroups::default();
|
||||
for config in configs.iter() {
|
||||
if let Config::ClientConfigGroup(group) = config {
|
||||
let members: Vec<_> = group.client_proxies.iter().map(member_name).collect();
|
||||
if group.strategy == GroupStrategy::Selector {
|
||||
if group.group_type == GroupType::Selector {
|
||||
let has_unnamed_member = group.client_proxies.iter().any(|member| {
|
||||
matches!(member, ConfigSelection::Config(client) if client.name.is_none())
|
||||
});
|
||||
@@ -270,16 +274,24 @@ impl Catalog {
|
||||
},
|
||||
);
|
||||
}
|
||||
if let GroupType::UrlTest(config) = &group.group_type {
|
||||
automatic.insert(
|
||||
group.client_group.clone(),
|
||||
config.clone(),
|
||||
members.clone(),
|
||||
)?;
|
||||
}
|
||||
nodes.insert(
|
||||
group.client_group.clone(),
|
||||
CatalogNode::Group {
|
||||
members,
|
||||
strategy: group.strategy,
|
||||
group_type: group.group_type.clone(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
let selection = SelectionOwner::new(None)?.prepare(definitions)?;
|
||||
let mut selection = SelectionOwner::new(None)?.prepare(definitions)?;
|
||||
Arc::get_mut(&mut selection).unwrap().automatic = automatic;
|
||||
crate::config::client_visit::visit_configs(configs, &mut |client| {
|
||||
if let Some(origin) = &mut client.control {
|
||||
origin.selection = selection.clone();
|
||||
@@ -411,7 +423,7 @@ mod tests {
|
||||
address: west.example:1080
|
||||
protocol: {type: socks}
|
||||
- client_group: outbound
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: child
|
||||
client_proxies: [child, direct]
|
||||
- address: 127.0.0.1:1080
|
||||
@@ -529,11 +541,12 @@ mod tests {
|
||||
CONFIG.replace("[child, direct]", "[child, child]"),
|
||||
CONFIG.replace("name: west", "name: east"),
|
||||
CONFIG.replace("name: west", "name: GLOBAL"),
|
||||
CONFIG.replace("strategy: selector", "strategy: round-robin"),
|
||||
CONFIG.replace("type: selector", "type: round-robin"),
|
||||
CONFIG.replace("[child, direct]", "[{protocol: {type: direct}}]"),
|
||||
] {
|
||||
let configs = serde_yaml::from_str(&yaml);
|
||||
assert!(
|
||||
crate::config::create_server_configs(serde_yaml::from_str(&yaml).unwrap()).is_err()
|
||||
configs.is_err() || crate::config::create_server_configs(configs.unwrap()).is_err()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,4 +6,5 @@ pub mod runtime;
|
||||
pub mod selection;
|
||||
pub mod store;
|
||||
pub mod tracked_stream;
|
||||
pub mod url_test;
|
||||
pub mod validation;
|
||||
|
||||
+74
-6
@@ -6,6 +6,7 @@ use std::sync::{Arc, Weak};
|
||||
use parking_lot::RwLock;
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tokio_util::task::AbortOnDropHandle;
|
||||
|
||||
use super::catalog::Catalog;
|
||||
use super::flows::FlowRegistry;
|
||||
@@ -30,6 +31,7 @@ pub struct ControlRuntime {
|
||||
pub probes: Arc<tokio::sync::Semaphore>,
|
||||
pub logs: Arc<super::logs::LogHub>,
|
||||
active: RwLock<Option<Arc<Generation>>>,
|
||||
probe_epoch: RwLock<CancellationToken>,
|
||||
next_generation: AtomicU64,
|
||||
commands: mpsc::Sender<Command>,
|
||||
selections: Arc<tokio::sync::Semaphore>,
|
||||
@@ -86,6 +88,7 @@ impl ControlService {
|
||||
probes: Arc::new(tokio::sync::Semaphore::new(16)),
|
||||
logs: Arc::new(super::logs::LogHub::new()),
|
||||
active: RwLock::new(None),
|
||||
probe_epoch: RwLock::new(CancellationToken::new()),
|
||||
next_generation: AtomicU64::new(1),
|
||||
commands,
|
||||
selections: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
@@ -108,6 +111,34 @@ impl ControlService {
|
||||
}
|
||||
|
||||
impl ControlRuntime {
|
||||
pub(crate) fn probe_epoch(&self) -> CancellationToken {
|
||||
self.probe_epoch.read().clone()
|
||||
}
|
||||
|
||||
fn cancel_probes(&self) {
|
||||
let mut epoch = self.probe_epoch.write();
|
||||
epoch.cancel();
|
||||
*epoch = CancellationToken::new();
|
||||
}
|
||||
|
||||
pub(crate) fn with_probe_generation<T>(
|
||||
&self,
|
||||
generation: u64,
|
||||
apply: impl FnOnce() -> T,
|
||||
) -> io::Result<T> {
|
||||
let active = self.active.read();
|
||||
if self.shutdown.is_cancelled()
|
||||
|| self.activating.load(Ordering::Acquire)
|
||||
|| active.as_ref().is_none_or(|active| active.id != generation)
|
||||
{
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::Interrupted,
|
||||
"probe generation is no longer active",
|
||||
));
|
||||
}
|
||||
Ok(apply())
|
||||
}
|
||||
|
||||
async fn request(&self, make: impl FnOnce(Reply) -> Command) -> io::Result<()> {
|
||||
let (reply, received) = oneshot::channel();
|
||||
tokio::select! {
|
||||
@@ -162,10 +193,14 @@ impl ControlRuntime {
|
||||
"controller listeners, credentials, origins, and cache_file_path require a restart",
|
||||
));
|
||||
}
|
||||
if let Some(catalog) = &validated.catalog {
|
||||
self.request(|reply| Command::Prepare(catalog.selection.clone(), reply))
|
||||
.await?;
|
||||
}
|
||||
// Removing the last managed group still needs an empty publication to
|
||||
// retire the previous generation's automatic checks.
|
||||
let catalog = validated.catalog.get_or_insert_with(|| Catalog {
|
||||
nodes: Default::default(),
|
||||
selection: Default::default(),
|
||||
});
|
||||
self.request(|reply| Command::Prepare(catalog.selection.clone(), reply))
|
||||
.await?;
|
||||
let id = self.next_generation.fetch_add(1, Ordering::Relaxed);
|
||||
let rules = super::rules::attach(
|
||||
validated,
|
||||
@@ -176,6 +211,13 @@ impl ControlRuntime {
|
||||
}
|
||||
}
|
||||
|
||||
async fn stop_automatic(task: Option<AbortOnDropHandle<()>>) {
|
||||
if let Some(task) = task {
|
||||
task.abort();
|
||||
let _ = task.await;
|
||||
}
|
||||
}
|
||||
|
||||
async fn run_owner(
|
||||
mut owner: SelectionOwner,
|
||||
mut receiver: mpsc::Receiver<Command>,
|
||||
@@ -184,6 +226,7 @@ async fn run_owner(
|
||||
) {
|
||||
let mut paused = false;
|
||||
let mut pending: VecDeque<SelectionCommand> = VecDeque::new();
|
||||
let mut automatic: Option<AbortOnDropHandle<()>> = None;
|
||||
loop {
|
||||
if shutdown.is_cancelled() {
|
||||
break;
|
||||
@@ -199,6 +242,13 @@ async fn run_owner(
|
||||
let command = tokio::select! {
|
||||
_ = shutdown.cancelled() => break,
|
||||
command = receiver.recv() => command,
|
||||
result = async { automatic.as_mut().unwrap().await }, if automatic.is_some() => {
|
||||
automatic = None;
|
||||
if let Err(error) = result {
|
||||
log::error!("Automatic group probe supervisor stopped: {error}");
|
||||
}
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let Some(command) = command else {
|
||||
break;
|
||||
@@ -214,8 +264,11 @@ async fn run_owner(
|
||||
if result.is_ok() {
|
||||
paused = true;
|
||||
if let Some(runtime) = runtime.upgrade() {
|
||||
let _active = runtime.active.write();
|
||||
runtime.activating.store(true, Ordering::Release);
|
||||
runtime.cancel_probes();
|
||||
}
|
||||
stop_automatic(automatic.take()).await;
|
||||
}
|
||||
let _ = reply.send(result);
|
||||
}
|
||||
@@ -224,14 +277,28 @@ async fn run_owner(
|
||||
if result.is_ok()
|
||||
&& let Some(runtime) = runtime.upgrade()
|
||||
{
|
||||
*runtime.active.write() = Some(generation);
|
||||
let mut active = runtime.active.write();
|
||||
runtime.cancel_probes();
|
||||
*active = Some(generation);
|
||||
}
|
||||
let _ = reply.send(result);
|
||||
}
|
||||
Command::Resume(reply) => {
|
||||
paused = false;
|
||||
stop_automatic(automatic.take()).await;
|
||||
if let Some(runtime) = runtime.upgrade() {
|
||||
runtime.activating.store(false, Ordering::Release);
|
||||
if let Ok(generation) = runtime.generation()
|
||||
&& !generation.catalog.selection.automatic.0.is_empty()
|
||||
{
|
||||
automatic = Some(AbortOnDropHandle::new(tokio::spawn(
|
||||
crate::clash_api::probes::run_automatic(
|
||||
Arc::downgrade(&runtime),
|
||||
generation,
|
||||
runtime.probe_epoch(),
|
||||
),
|
||||
)));
|
||||
}
|
||||
}
|
||||
let _ = reply.send(Ok(()));
|
||||
}
|
||||
@@ -270,6 +337,7 @@ async fn run_owner(
|
||||
}
|
||||
}
|
||||
}
|
||||
stop_automatic(automatic).await;
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -309,7 +377,7 @@ mod tests {
|
||||
let directory = tempfile::tempdir().unwrap();
|
||||
let cache = directory.path().join("cache.db");
|
||||
let mut configs = serde_json::json!([
|
||||
{"client_group":"outbound", "strategy":if mode == "selector" { "selector" } else { "round-robin" }, "client_proxies":[
|
||||
{"client_group":"outbound", "type":if mode == "selector" { "selector" } else { "round-robin" }, "client_proxies":[
|
||||
{"name":"proxy", "address":"proxy.invalid:8080", "protocol":{"type":"http"}}
|
||||
]},
|
||||
{"name":"entry", "address":"127.0.0.1:0", "protocol":{"type":"socks"}, "rules":[
|
||||
|
||||
@@ -20,6 +20,7 @@ pub type Choices = BTreeMap<String, String>;
|
||||
pub struct SelectionView {
|
||||
definitions: SelectorDefinitions,
|
||||
choices: RwLock<Arc<Choices>>,
|
||||
pub automatic: super::url_test::AutomaticGroups,
|
||||
}
|
||||
|
||||
impl SelectionView {
|
||||
@@ -60,6 +61,7 @@ impl SelectionOwner {
|
||||
Ok(Arc::new(SelectionView {
|
||||
choices: RwLock::new(Arc::new(effective(&definitions, &self.explicit))),
|
||||
definitions,
|
||||
automatic: Default::default(),
|
||||
}))
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,370 @@
|
||||
use std::collections::{BTreeMap, HashMap, HashSet};
|
||||
use std::io;
|
||||
use std::time::Duration;
|
||||
|
||||
use parking_lot::Mutex;
|
||||
use tokio::time::Instant;
|
||||
|
||||
use crate::config::UrlTestConfig;
|
||||
|
||||
tokio::task_local! {
|
||||
pub static BACKGROUND: ();
|
||||
}
|
||||
|
||||
pub fn is_background() -> bool {
|
||||
BACKGROUND.try_with(|_| ()).is_ok()
|
||||
}
|
||||
|
||||
// Only unambiguous transport failures request a recheck. A user destination's
|
||||
// failure is not evidence that the configured probe URL is unreachable.
|
||||
pub fn remote_failure(error: &io::Error) -> bool {
|
||||
if crate::resources::is_exhausted(error) {
|
||||
return false;
|
||||
}
|
||||
matches!(
|
||||
error.kind(),
|
||||
io::ErrorKind::ConnectionRefused
|
||||
| io::ErrorKind::ConnectionReset
|
||||
| io::ErrorKind::ConnectionAborted
|
||||
| io::ErrorKind::NotConnected
|
||||
| io::ErrorKind::TimedOut
|
||||
| io::ErrorKind::UnexpectedEof
|
||||
| io::ErrorKind::BrokenPipe
|
||||
| io::ErrorKind::NetworkUnreachable
|
||||
| io::ErrorKind::HostUnreachable
|
||||
)
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
struct Sample {
|
||||
at: Instant,
|
||||
delay: Option<u16>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct State {
|
||||
samples: HashMap<String, Sample>,
|
||||
incumbent: [Option<String>; 2],
|
||||
last_touch: Option<Instant>,
|
||||
last_check: Option<Instant>,
|
||||
last_unqueued_admission: Option<Instant>,
|
||||
recheck: bool,
|
||||
cursor: usize,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct AutomaticGroup {
|
||||
pub config: UrlTestConfig,
|
||||
pub members: Vec<String>,
|
||||
state: Mutex<State>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct AutomaticGroups(pub BTreeMap<String, AutomaticGroup>);
|
||||
|
||||
impl AutomaticGroups {
|
||||
pub fn filter(
|
||||
&self,
|
||||
origins: &[Option<super::catalog::MemberOrigin>],
|
||||
candidates: &mut Vec<usize>,
|
||||
udp: bool,
|
||||
) {
|
||||
let mut depths = HashMap::<&str, usize>::new();
|
||||
for &index in candidates.iter() {
|
||||
let Some(origin) = &origins[index] else {
|
||||
continue;
|
||||
};
|
||||
for (depth, (group, _)) in origin.path.iter().enumerate() {
|
||||
if self.0.contains_key(group) {
|
||||
depths
|
||||
.entry(group)
|
||||
.and_modify(|value| *value = (*value).max(depth))
|
||||
.or_insert(depth);
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut groups: Vec<_> = depths.into_iter().collect();
|
||||
groups.sort_unstable_by_key(|(_, depth)| std::cmp::Reverse(*depth));
|
||||
for (group, _) in groups {
|
||||
let eligible = candidates
|
||||
.iter()
|
||||
.filter_map(|&index| origins[index].as_ref())
|
||||
.flat_map(|origin| &origin.path)
|
||||
.filter(|(name, _)| name == group)
|
||||
.map(|(_, member)| member.as_str())
|
||||
.collect();
|
||||
if let Some(selected) = self.0[group].choose(&eligible, udp) {
|
||||
candidates.retain(|&index| {
|
||||
origins[index].as_ref().is_none_or(|origin| {
|
||||
origin
|
||||
.path
|
||||
.iter()
|
||||
.all(|(name, member)| name != group || member == &selected)
|
||||
})
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn insert(
|
||||
&mut self,
|
||||
name: String,
|
||||
mut config: UrlTestConfig,
|
||||
members: Vec<String>,
|
||||
) -> io::Result<()> {
|
||||
config.validate()?;
|
||||
if members.is_empty() || members.iter().collect::<HashSet<_>>().len() != members.len() {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::InvalidInput,
|
||||
"url-test requires nonempty, distinct members",
|
||||
));
|
||||
}
|
||||
config.url = url::Url::parse(&config.url)
|
||||
.map_err(io::Error::other)?
|
||||
.to_string();
|
||||
self.0.insert(
|
||||
name,
|
||||
AutomaticGroup {
|
||||
config,
|
||||
members,
|
||||
state: Mutex::default(),
|
||||
},
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn record(&self, member: &str, url: &str, delay: Option<u16>) {
|
||||
for group in self.0.values() {
|
||||
if group.config.url == url && group.members.iter().any(|name| name == member) {
|
||||
group.state.lock().samples.insert(
|
||||
member.into(),
|
||||
Sample {
|
||||
at: Instant::now(),
|
||||
delay,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn touch(&self, name: &str, failed: bool) {
|
||||
if !is_background()
|
||||
&& let Some(group) = self.0.get(name)
|
||||
{
|
||||
let mut state = group.state.lock();
|
||||
state.last_touch = Some(Instant::now());
|
||||
state.recheck |= failed;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AutomaticGroup {
|
||||
fn fresh(&self, sample: &Sample, now: Instant) -> bool {
|
||||
now.duration_since(sample.at)
|
||||
<= Duration::from_secs(self.config.interval * 2)
|
||||
+ Duration::from_millis(self.config.timeout)
|
||||
}
|
||||
|
||||
pub fn choose(&self, eligible: &HashSet<&str>, udp: bool) -> Option<String> {
|
||||
let mut state = self.state.lock();
|
||||
let now = Instant::now();
|
||||
let delay = |name: &str| {
|
||||
state
|
||||
.samples
|
||||
.get(name)
|
||||
.filter(|sample| self.fresh(sample, now))
|
||||
.and_then(|sample| sample.delay)
|
||||
};
|
||||
let best = self
|
||||
.members
|
||||
.iter()
|
||||
.filter(|name| eligible.contains(name.as_str()))
|
||||
.filter_map(|name| delay(name).map(|delay| (name, delay)))
|
||||
.min_by_key(|(_, delay)| *delay);
|
||||
let incumbent = state.incumbent[usize::from(udp)]
|
||||
.as_ref()
|
||||
.filter(|name| eligible.contains(name.as_str()));
|
||||
let selected = match (best, incumbent) {
|
||||
(Some((best, best_delay)), Some(current)) => {
|
||||
if delay(current).is_some_and(|delay| {
|
||||
u32::from(delay) <= u32::from(best_delay) + u32::from(self.config.tolerance)
|
||||
}) {
|
||||
Some(current.clone())
|
||||
} else {
|
||||
Some(best.clone())
|
||||
}
|
||||
}
|
||||
(Some((best, _)), None) => Some(best.clone()),
|
||||
(None, Some(current)) => Some(current.clone()),
|
||||
(None, None) => self
|
||||
.members
|
||||
.iter()
|
||||
.find(|name| eligible.contains(name.as_str()))
|
||||
.cloned(),
|
||||
};
|
||||
state.incumbent[usize::from(udp)] = selected.clone();
|
||||
selected
|
||||
}
|
||||
|
||||
pub fn health(&self, member: &str) -> &'static str {
|
||||
let state = self.state.lock();
|
||||
match state.samples.get(member) {
|
||||
None => "unprobed",
|
||||
Some(sample) if !self.fresh(sample, Instant::now()) => "stale",
|
||||
Some(Sample { delay: Some(_), .. }) => "healthy",
|
||||
Some(_) => "dead",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn due_members(&self, now: Instant) -> Option<Vec<usize>> {
|
||||
let mut state = self.state.lock();
|
||||
let interval = Duration::from_secs(self.config.interval);
|
||||
if let Some(last_check) = state.last_check {
|
||||
let elapsed = now.duration_since(last_check);
|
||||
let recheck_due = state.recheck && elapsed >= Duration::from_secs(1);
|
||||
if !recheck_due {
|
||||
if elapsed < interval {
|
||||
return None;
|
||||
}
|
||||
let idle = state
|
||||
.last_touch
|
||||
.is_none_or(|touch| now.duration_since(touch) >= interval);
|
||||
if self.config.lazy && idle {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
}
|
||||
state.last_check = Some(now);
|
||||
state.recheck = false;
|
||||
let members = (state.cursor..self.members.len())
|
||||
.chain(0..state.cursor)
|
||||
.collect();
|
||||
Some(members)
|
||||
}
|
||||
|
||||
pub fn admitted(&self, index: usize, queued: bool) {
|
||||
let mut state = self.state.lock();
|
||||
state.cursor = (index + 1) % self.members.len();
|
||||
if !queued {
|
||||
state.last_unqueued_admission = Some(Instant::now());
|
||||
}
|
||||
}
|
||||
|
||||
pub fn last_unqueued_admission(&self) -> Option<Instant> {
|
||||
self.state.lock().last_unqueued_admission
|
||||
}
|
||||
|
||||
pub fn cancel_check(&self) {
|
||||
self.state.lock().last_check = None;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn groups() -> AutomaticGroups {
|
||||
let mut groups = AutomaticGroups::default();
|
||||
for (name, url) in [
|
||||
("one", "http://one.invalid/"),
|
||||
("two", "http://two.invalid/"),
|
||||
] {
|
||||
groups
|
||||
.insert(
|
||||
name.into(),
|
||||
UrlTestConfig {
|
||||
url: url.into(),
|
||||
interval: 10,
|
||||
timeout: 1000,
|
||||
tolerance: 50,
|
||||
lazy: true,
|
||||
},
|
||||
vec!["a".into(), "b".into()],
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
groups
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn selection_health_tolerance_failure_recovery_and_staleness_are_url_scoped() {
|
||||
let groups = groups();
|
||||
let group = &groups.0["one"];
|
||||
let both = HashSet::from(["a", "b"]);
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("a"));
|
||||
assert_eq!(group.health("a"), "unprobed");
|
||||
groups.record("b", "http://two.invalid/", Some(1));
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("a"));
|
||||
assert_eq!(groups.0["two"].choose(&both, false).as_deref(), Some("b"));
|
||||
groups.record("a", &group.config.url, Some(100));
|
||||
groups.record("b", &group.config.url, Some(50));
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("a"));
|
||||
groups.record("b", &group.config.url, Some(49));
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("b"));
|
||||
groups.record("b", &group.config.url, None);
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("a"));
|
||||
groups.record("b", &group.config.url, Some(1));
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("b"));
|
||||
assert_eq!(
|
||||
group.choose(&HashSet::from(["a"]), true).as_deref(),
|
||||
Some("a")
|
||||
);
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("b"));
|
||||
groups.record("a", &group.config.url, None);
|
||||
groups.record("b", &group.config.url, None);
|
||||
assert_eq!(group.health("b"), "dead");
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("b"));
|
||||
groups.record("b", &group.config.url, Some(1));
|
||||
tokio::time::advance(Duration::from_secs(22)).await;
|
||||
assert_eq!(group.health("b"), "stale");
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("b"));
|
||||
assert!(group.choose(&HashSet::new(), true).is_none());
|
||||
groups.record("a", &group.config.url, Some(150));
|
||||
assert_eq!(group.choose(&both, false).as_deref(), Some("a"));
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn lazy_startup_probe_suppression_touch_and_rate_limited_recheck() {
|
||||
let groups = groups();
|
||||
let group = &groups.0["one"];
|
||||
assert!(group.due_members(Instant::now()).is_some());
|
||||
tokio::time::advance(Duration::from_secs(10)).await;
|
||||
BACKGROUND
|
||||
.scope((), async {
|
||||
groups.touch("one", true);
|
||||
})
|
||||
.await;
|
||||
assert!(group.due_members(Instant::now()).is_none());
|
||||
groups.touch("one", false);
|
||||
assert!(group.due_members(Instant::now()).is_some());
|
||||
groups.touch("one", true);
|
||||
assert!(group.due_members(Instant::now()).is_none());
|
||||
tokio::time::advance(Duration::from_secs(1)).await;
|
||||
assert!(group.due_members(Instant::now()).is_some());
|
||||
tokio::time::advance(Duration::from_secs(10)).await;
|
||||
assert!(group.due_members(Instant::now()).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn local_errors_do_not_request_remote_health_changes() {
|
||||
for kind in [
|
||||
io::ErrorKind::WouldBlock,
|
||||
io::ErrorKind::Unsupported,
|
||||
io::ErrorKind::OutOfMemory,
|
||||
io::ErrorKind::ResourceBusy,
|
||||
io::ErrorKind::InvalidInput,
|
||||
io::ErrorKind::PermissionDenied,
|
||||
io::ErrorKind::Other,
|
||||
] {
|
||||
assert!(!remote_failure(&io::Error::from(kind)), "{kind:?}");
|
||||
}
|
||||
for kind in [
|
||||
io::ErrorKind::ConnectionRefused,
|
||||
io::ErrorKind::ConnectionReset,
|
||||
io::ErrorKind::TimedOut,
|
||||
io::ErrorKind::BrokenPipe,
|
||||
] {
|
||||
assert!(remote_failure(&io::Error::from(kind)), "{kind:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
+2
-2
@@ -1906,11 +1906,11 @@ mod tests {
|
||||
- client_group: west
|
||||
client_proxies: [{name: west-direct, protocol: {type: direct}, bind_interface: west}]
|
||||
- client_group: first
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: east
|
||||
client_proxies: [east, west]
|
||||
- client_group: second
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: west
|
||||
client_proxies: [east, west]
|
||||
- dns_group: selected
|
||||
|
||||
+13
-2
@@ -12,6 +12,17 @@ struct RealResolver {
|
||||
forbidden: Arc<[IpNetwork]>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct SyntheticDnsAnswer;
|
||||
|
||||
impl std::fmt::Display for SyntheticDnsAnswer {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
formatter.write_str("real DNS returned a synthetic address (possible DNS loop)")
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for SyntheticDnsAnswer {}
|
||||
|
||||
pub(crate) fn real_only(
|
||||
inner: Arc<dyn Resolver>,
|
||||
forbidden: Arc<[IpNetwork]>,
|
||||
@@ -36,7 +47,7 @@ impl Resolver for RealResolver {
|
||||
}) {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::InvalidData,
|
||||
"real DNS returned a synthetic address (possible DNS loop)",
|
||||
SyntheticDnsAnswer,
|
||||
));
|
||||
}
|
||||
Ok(addresses)
|
||||
@@ -69,7 +80,7 @@ impl RealResolver {
|
||||
}) {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::InvalidData,
|
||||
"real DNS returned a synthetic address (possible DNS loop)",
|
||||
SyntheticDnsAnswer,
|
||||
));
|
||||
}
|
||||
Ok(bytes)
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ mod parsed;
|
||||
mod proxy_runtime;
|
||||
mod system;
|
||||
|
||||
pub(crate) use guard::real_only;
|
||||
pub(crate) use guard::{SyntheticDnsAnswer, real_only};
|
||||
|
||||
/// Binds a TCP listener and a UDP socket on one loopback port, as DNS servers listen. A
|
||||
/// free TCP port does not imply a free UDP port, so retry with new ports.
|
||||
|
||||
@@ -273,7 +273,7 @@ mod tests {
|
||||
async fn new_dns_sockets_follow_direct_selector_changes() {
|
||||
let config = r#"
|
||||
- client_group: dns-route
|
||||
strategy: selector
|
||||
type: selector
|
||||
default: usable
|
||||
client_proxies:
|
||||
- name: unavailable
|
||||
|
||||
@@ -217,6 +217,16 @@ pub async fn start_from_config(
|
||||
|
||||
let mut validated = crate::config::compile_configs(configs).await?;
|
||||
validated.control.reject_ffi()?;
|
||||
if validated
|
||||
.catalog
|
||||
.as_ref()
|
||||
.is_some_and(|catalog| !catalog.selection.automatic.0.is_empty())
|
||||
{
|
||||
return Err(std::io::Error::new(
|
||||
std::io::ErrorKind::Unsupported,
|
||||
"url-test groups require the process runtime and are not supported by FFI",
|
||||
));
|
||||
}
|
||||
let mut certificates = crate::certificates::Registry::prepare(&mut validated, None).await?;
|
||||
#[cfg(unix)]
|
||||
let wireguard = crate::wireguard::Prepared::new(&mut validated).await?;
|
||||
@@ -355,6 +365,20 @@ mod tests {
|
||||
use std::sync::mpsc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
#[tokio::test]
|
||||
async fn url_test_configuration_is_rejected_without_a_process_supervisor() {
|
||||
let config = r#"
|
||||
- client_group: automatic
|
||||
type: url-test
|
||||
url: http://probe.invalid/
|
||||
client_proxies: [direct]
|
||||
"#;
|
||||
let (_shutdown, shutdown) = oneshot::channel();
|
||||
let error = start_from_config(config, shutdown).await.unwrap_err();
|
||||
assert_eq!(error.kind(), std::io::ErrorKind::Unsupported);
|
||||
assert!(error.to_string().contains("url-test groups"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lifecycle_requires_a_non_runtime_thread() {
|
||||
assert!(is_lifecycle_thread());
|
||||
|
||||
@@ -18,6 +18,7 @@ use tokio::time::interval;
|
||||
|
||||
use crate::address::NetLocation;
|
||||
use crate::async_stream::AsyncStream;
|
||||
use crate::http2::transport::TransportErrors;
|
||||
|
||||
use super::H2MuxOptions;
|
||||
use super::activity_tracker::{
|
||||
@@ -40,6 +41,7 @@ const MAX_FRAME_SIZE: u32 = 16 * 1024;
|
||||
/// - Driver ownership shared by sessions and active streams
|
||||
#[derive(Clone)]
|
||||
pub struct H2MuxClientSession {
|
||||
pub(super) errors: TransportErrors,
|
||||
send_request: h2::client::SendRequest<Bytes>,
|
||||
_driver_handle: Arc<DriverHandle>,
|
||||
padding_enabled: bool,
|
||||
@@ -104,14 +106,16 @@ impl H2MuxClientSession {
|
||||
where
|
||||
IO: AsyncRead + AsyncWrite + Unpin + Send + 'static,
|
||||
{
|
||||
let errors = TransportErrors::default();
|
||||
let (send_request, mut connection) = h2::client::Builder::new()
|
||||
.initial_window_size(STREAM_WINDOW_SIZE)
|
||||
.initial_connection_window_size(CONNECTION_WINDOW_SIZE)
|
||||
.max_frame_size(MAX_FRAME_SIZE)
|
||||
.max_concurrent_streams(1024)
|
||||
.handshake(conn)
|
||||
.handshake(errors.wrap(conn))
|
||||
.await
|
||||
.map_err(|e| io::Error::other(format!("H2 client handshake failed: {}", e)))?;
|
||||
.map_err(|e| errors.map_with("H2 client handshake failed", e))?;
|
||||
errors.start_driver();
|
||||
|
||||
// Take ping_pong handle before spawning - can only be called once
|
||||
let ping_pong = connection.ping_pong();
|
||||
@@ -142,6 +146,7 @@ impl H2MuxClientSession {
|
||||
debug!("H2MuxClientSession: ready for multiplexing");
|
||||
|
||||
Ok(Self {
|
||||
errors,
|
||||
send_request,
|
||||
_driver_handle: Arc::new(DriverHandle {
|
||||
_drain: drain_tx,
|
||||
@@ -276,7 +281,7 @@ impl H2MuxClientSession {
|
||||
let (response_future, send_stream) = self
|
||||
.send_request
|
||||
.send_request(http_request, false)
|
||||
.map_err(|e| io::Error::other(format!("Failed to send CONNECT: {}", e)))?;
|
||||
.map_err(|e| self.errors.map_with("Failed to send CONNECT", e))?;
|
||||
|
||||
// Create unified client stream with lazy response resolution
|
||||
let client_stream = H2MuxClientStream::new(
|
||||
@@ -300,6 +305,62 @@ mod tests {
|
||||
use super::*;
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_errors_preserve_transport_causes_with_and_without_padding() {
|
||||
use crate::http2::transport::tests::{Fault, cases};
|
||||
tokio::time::timeout(std::time::Duration::from_secs(5), async {
|
||||
for padding in [false, true] {
|
||||
for lazy in [false, true] {
|
||||
for (make_error, local) in cases() {
|
||||
let fault = Fault::new(make_error);
|
||||
let (client, peer) = tokio::io::duplex(8192);
|
||||
let (accepted_tx, accepted_rx) = oneshot::channel();
|
||||
let server = tokio::spawn(async move {
|
||||
let mut session =
|
||||
super::super::H2MuxServerSession::new(peer).await.unwrap();
|
||||
let inbound = session.accept().await.unwrap();
|
||||
accepted_tx.send(inbound).ok().unwrap();
|
||||
while session.accept().await.is_some() {}
|
||||
});
|
||||
let options = H2MuxOptions {
|
||||
padding,
|
||||
..Default::default()
|
||||
};
|
||||
let mut session = H2MuxClientSession::new(fault.wrap(client), &options)
|
||||
.await
|
||||
.unwrap();
|
||||
let mut stream = session
|
||||
.open_tcp(&NetLocation::from_str("example.com:80", None).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let inbound = if lazy {
|
||||
None
|
||||
} else {
|
||||
stream.write_all(b"x").await.unwrap();
|
||||
let mut inbound = accepted_rx.await.unwrap();
|
||||
inbound.stream.write_all(b"y").await.unwrap();
|
||||
assert_eq!(stream.read_u8().await.unwrap(), b'y');
|
||||
Some(inbound)
|
||||
};
|
||||
fault.inject();
|
||||
let error = stream.read_u8().await.unwrap_err();
|
||||
assert_eq!(
|
||||
crate::clash_api::probes::local_failure(&error),
|
||||
local,
|
||||
"padding={padding} lazy={lazy}: {error:?}"
|
||||
);
|
||||
drop(inbound);
|
||||
drop(stream);
|
||||
drop(session);
|
||||
server.abort();
|
||||
let _ = server.await;
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn streams_own_driver_until_last_stream_is_dropped() {
|
||||
for _ in 0..16 {
|
||||
|
||||
@@ -107,9 +107,10 @@ impl H2MuxClientStream {
|
||||
}
|
||||
Poll::Ready(Err(e)) => {
|
||||
self.recv_pending = None;
|
||||
Poll::Ready(Err(io::Error::other(format!(
|
||||
"CONNECT response error: {e}"
|
||||
))))
|
||||
Poll::Ready(Err(self
|
||||
.session
|
||||
.errors
|
||||
.map_with("CONNECT response error", e)))
|
||||
}
|
||||
Poll::Pending => Poll::Pending,
|
||||
}
|
||||
@@ -213,7 +214,7 @@ impl H2MuxClientStream {
|
||||
Poll::Ready(Ok(Some(data)))
|
||||
}
|
||||
Poll::Ready(Some(Err(e))) => {
|
||||
Poll::Ready(Err(io::Error::other(format!("H2 recv error: {e}"))))
|
||||
Poll::Ready(Err(self.session.errors.map_with("H2 recv error", e)))
|
||||
}
|
||||
Poll::Ready(None) => Poll::Ready(Ok(None)),
|
||||
Poll::Pending => Poll::Pending,
|
||||
@@ -263,12 +264,13 @@ impl H2MuxClientStream {
|
||||
let to_send = buf.len().min(capacity);
|
||||
self.send
|
||||
.send_data(Bytes::copy_from_slice(&buf[..to_send]), false)
|
||||
.map_err(|e| io::Error::other(format!("H2 send_data failed: {e}")))?;
|
||||
.map_err(|e| self.session.errors.map_with("H2 send_data failed", e))?;
|
||||
Poll::Ready(Ok(to_send))
|
||||
}
|
||||
Poll::Ready(Some(Err(e))) => Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 poll_capacity error: {e}"
|
||||
)))),
|
||||
Poll::Ready(Some(Err(e))) => Poll::Ready(Err(self
|
||||
.session
|
||||
.errors
|
||||
.map_with("H2 poll_capacity error", e))),
|
||||
Poll::Ready(None) => Poll::Ready(Err(io::Error::new(
|
||||
io::ErrorKind::BrokenPipe,
|
||||
"H2 stream closed",
|
||||
@@ -380,7 +382,7 @@ impl AsyncWrite for H2MuxClientStream {
|
||||
let to_send = remaining.len().min(capacity);
|
||||
self.send
|
||||
.send_data(pending_data.slice(sent..sent + to_send), false)
|
||||
.map_err(|e| io::Error::other(format!("H2 send_data failed: {e}")))?;
|
||||
.map_err(|e| self.session.errors.map_with("H2 send_data failed", e))?;
|
||||
|
||||
let new_sent = sent + to_send;
|
||||
if new_sent < pending_data.len() {
|
||||
@@ -392,9 +394,10 @@ impl AsyncWrite for H2MuxClientStream {
|
||||
return Poll::Ready(Ok(user_len));
|
||||
}
|
||||
Poll::Ready(Some(Err(e))) => {
|
||||
return Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 poll_capacity error: {e}"
|
||||
))));
|
||||
return Poll::Ready(Err(self
|
||||
.session
|
||||
.errors
|
||||
.map_with("H2 poll_capacity error", e)));
|
||||
}
|
||||
Poll::Ready(None) => {
|
||||
return Poll::Ready(Err(io::Error::new(
|
||||
@@ -427,7 +430,7 @@ impl AsyncWrite for H2MuxClientStream {
|
||||
let to_send = combined.len().min(capacity);
|
||||
self.send
|
||||
.send_data(combined.slice(..to_send), false)
|
||||
.map_err(|e| io::Error::other(format!("H2 send_data failed: {e}")))?;
|
||||
.map_err(|e| self.session.errors.map_with("H2 send_data failed", e))?;
|
||||
|
||||
if to_send < combined.len() {
|
||||
// Partial write - track remaining data
|
||||
@@ -439,9 +442,10 @@ impl AsyncWrite for H2MuxClientStream {
|
||||
Poll::Ready(Ok(buf.len()))
|
||||
}
|
||||
}
|
||||
Poll::Ready(Some(Err(e))) => Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 poll_capacity error: {e}"
|
||||
)))),
|
||||
Poll::Ready(Some(Err(e))) => Poll::Ready(Err(self
|
||||
.session
|
||||
.errors
|
||||
.map_with("H2 poll_capacity error", e))),
|
||||
Poll::Ready(None) => Poll::Ready(Err(io::Error::new(
|
||||
io::ErrorKind::BrokenPipe,
|
||||
"H2 stream closed",
|
||||
@@ -466,9 +470,10 @@ impl AsyncWrite for H2MuxClientStream {
|
||||
match self.send.send_data(Bytes::new(), true) {
|
||||
Ok(()) => self.shutdown_sent = true,
|
||||
Err(e) => {
|
||||
return Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 send END_STREAM failed: {e}"
|
||||
))));
|
||||
return Poll::Ready(Err(self
|
||||
.session
|
||||
.errors
|
||||
.map_with("H2 send END_STREAM failed", e)));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+31
-10
@@ -13,6 +13,7 @@ use crate::address::{Address, NetLocation};
|
||||
use crate::async_stream::AsyncStream;
|
||||
|
||||
use super::stream::H2MultiStream;
|
||||
use super::transport::TransportErrors;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct H2ClientSession {
|
||||
@@ -21,6 +22,7 @@ pub struct H2ClientSession {
|
||||
}
|
||||
|
||||
struct DriverHandle {
|
||||
errors: TransportErrors,
|
||||
driver: tokio::task::AbortHandle,
|
||||
_drain: oneshot::Sender<()>,
|
||||
active: Arc<ActiveStreams>,
|
||||
@@ -56,11 +58,22 @@ impl Drop for StreamReservation {
|
||||
pub(crate) struct OpenError {
|
||||
pub error: h2::Error,
|
||||
pub sent: bool,
|
||||
cause: Option<Arc<io::Error>>,
|
||||
}
|
||||
|
||||
impl OpenError {
|
||||
fn new(error: h2::Error, sent: bool, errors: &TransportErrors) -> Self {
|
||||
Self {
|
||||
cause: errors.cause(&error),
|
||||
error,
|
||||
sent,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<OpenError> for io::Error {
|
||||
fn from(error: OpenError) -> Self {
|
||||
io::Error::other(error.error)
|
||||
TransportErrors::map_captured(error.error, error.cause)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -68,6 +81,7 @@ impl H2ClientSession {
|
||||
pub async fn new(stream: Box<dyn AsyncStream>) -> io::Result<Self> {
|
||||
const MAX_FRAME_SIZE: u32 = 16 * 1024;
|
||||
|
||||
let errors = TransportErrors::default();
|
||||
let (send_request, mut connection) = h2::client::Builder::new()
|
||||
.initial_window_size(super::STREAM_WINDOW)
|
||||
.initial_connection_window_size(super::CONNECTION_WINDOW)
|
||||
@@ -75,9 +89,10 @@ impl H2ClientSession {
|
||||
.max_frame_size(MAX_FRAME_SIZE)
|
||||
.max_header_list_size(16 * 1024)
|
||||
.max_concurrent_streams(1024)
|
||||
.handshake(stream)
|
||||
.handshake(errors.wrap(stream))
|
||||
.await
|
||||
.map_err(|e| io::Error::other(format!("H2 client handshake failed: {}", e)))?;
|
||||
.map_err(|e| errors.map_with("H2 client handshake failed", e))?;
|
||||
errors.start_driver();
|
||||
|
||||
let ping = connection.ping_pong().unwrap();
|
||||
let (drain_tx, mut drain_rx) = oneshot::channel();
|
||||
@@ -117,6 +132,7 @@ impl H2ClientSession {
|
||||
Ok(Self {
|
||||
send_request,
|
||||
driver_handle: Arc::new(DriverHandle {
|
||||
errors,
|
||||
driver: abort_handle,
|
||||
_drain: drain_tx,
|
||||
active,
|
||||
@@ -192,28 +208,33 @@ async fn keep_alive(mut ping: h2::PingPong) -> io::Result<()> {
|
||||
}
|
||||
|
||||
impl StreamReservation {
|
||||
pub(super) fn errors(&self) -> &TransportErrors {
|
||||
&self.0.driver_handle.errors
|
||||
}
|
||||
|
||||
pub async fn open(
|
||||
mut self,
|
||||
request: Request<()>,
|
||||
) -> Result<(http::response::Parts, H2MultiStream), OpenError> {
|
||||
std::future::poll_fn(|cx| self.0.send_request.poll_ready(cx))
|
||||
.await
|
||||
.map_err(|error| OpenError { error, sent: false })?;
|
||||
.map_err(|error| OpenError::new(error, false, self.errors()))?;
|
||||
// A reservation may predate keepalive retirement; it must not restart idle draining.
|
||||
if self.0.driver_handle.retired.load(Ordering::SeqCst) {
|
||||
return Err(OpenError {
|
||||
error: h2::Reason::REFUSED_STREAM.into(),
|
||||
sent: false,
|
||||
});
|
||||
return Err(OpenError::new(
|
||||
h2::Reason::REFUSED_STREAM.into(),
|
||||
false,
|
||||
self.errors(),
|
||||
));
|
||||
}
|
||||
let (response, send) = self
|
||||
.0
|
||||
.send_request
|
||||
.send_request(request, false)
|
||||
.map_err(|error| OpenError { error, sent: false })?;
|
||||
.map_err(|error| OpenError::new(error, false, self.errors()))?;
|
||||
let response = response
|
||||
.await
|
||||
.map_err(|error| OpenError { error, sent: true })?;
|
||||
.map_err(|error| OpenError::new(error, true, self.errors()))?;
|
||||
let (parts, recv) = response.into_parts();
|
||||
let mut stream = H2MultiStream::new(send, recv);
|
||||
stream.set_reservation(self);
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
pub(crate) mod client;
|
||||
pub(crate) mod server;
|
||||
pub(crate) mod stream;
|
||||
pub(crate) mod transport;
|
||||
|
||||
pub(crate) const STREAM_WINDOW: u32 = 256 * 1024;
|
||||
pub(crate) const CONNECTION_WINDOW: u32 = 2 * 1024 * 1024;
|
||||
|
||||
+18
-7
@@ -23,6 +23,13 @@ pub struct H2MultiStream {
|
||||
}
|
||||
|
||||
impl H2MultiStream {
|
||||
fn map_error(&self, error: h2::Error) -> io::Error {
|
||||
match &self.reservation {
|
||||
Some(reservation) => reservation.errors().map(error),
|
||||
None => io::Error::other(error),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn new(send: h2::SendStream<Bytes>, recv: h2::RecvStream) -> Self {
|
||||
Self {
|
||||
send: Arc::new(parking_lot::Mutex::new(send)),
|
||||
@@ -61,12 +68,12 @@ impl AsyncRead for H2MultiStream {
|
||||
self.recv
|
||||
.flow_control()
|
||||
.release_capacity(count)
|
||||
.map_err(io::Error::other)?;
|
||||
.map_err(|error| self.map_error(error))?;
|
||||
return Poll::Ready(Ok(()));
|
||||
}
|
||||
match std::task::ready!(self.recv.poll_data(cx)) {
|
||||
Some(Ok(data)) => self.recv_buf = data,
|
||||
Some(Err(e)) => return Poll::Ready(Err(io::Error::other(e))),
|
||||
Some(Err(e)) => return Poll::Ready(Err(self.map_error(e))),
|
||||
None => return Poll::Ready(Ok(())),
|
||||
}
|
||||
}
|
||||
@@ -95,12 +102,12 @@ impl AsyncWrite for H2MultiStream {
|
||||
if capacity != 0 {
|
||||
let count = wanted.min(capacity);
|
||||
send.send_data(Bytes::copy_from_slice(&buf[..count]), false)
|
||||
.map_err(io::Error::other)?;
|
||||
.map_err(|error| self.map_error(error))?;
|
||||
return Poll::Ready(Ok(count));
|
||||
}
|
||||
match std::task::ready!(send.poll_capacity(cx)) {
|
||||
Some(Ok(_)) => continue,
|
||||
Some(Err(e)) => return Poll::Ready(Err(io::Error::other(e))),
|
||||
Some(Err(e)) => return Poll::Ready(Err(self.map_error(e))),
|
||||
None => return Poll::Ready(Err(io::ErrorKind::BrokenPipe.into())),
|
||||
}
|
||||
}
|
||||
@@ -117,9 +124,13 @@ impl AsyncWrite for H2MultiStream {
|
||||
match result {
|
||||
Ok(()) => self.shutdown_sent = true,
|
||||
Err(e) => {
|
||||
return Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 send END_STREAM failed: {e}"
|
||||
))));
|
||||
let error = match &self.reservation {
|
||||
Some(reservation) => reservation
|
||||
.errors()
|
||||
.map_with("H2 send END_STREAM failed", e),
|
||||
None => io::Error::other(format!("H2 send END_STREAM failed: {e}")),
|
||||
};
|
||||
return Poll::Ready(Err(error));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
use std::{
|
||||
error::Error,
|
||||
fmt, io,
|
||||
pin::Pin,
|
||||
sync::{
|
||||
Arc, OnceLock,
|
||||
atomic::{AtomicBool, Ordering},
|
||||
},
|
||||
task::{Context, Poll},
|
||||
};
|
||||
|
||||
use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
|
||||
|
||||
#[cfg(test)]
|
||||
#[path = "transport_tests.rs"]
|
||||
pub(crate) mod tests;
|
||||
|
||||
/// h2 copies transport errors as kind and text, discarding errno and typed causes.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub(crate) struct TransportErrors(Arc<State>);
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct State {
|
||||
running: AtomicBool,
|
||||
terminal: OnceLock<Option<Arc<io::Error>>>,
|
||||
}
|
||||
|
||||
impl TransportErrors {
|
||||
pub(crate) fn wrap<T>(&self, inner: T) -> ObservedIo<T> {
|
||||
ObservedIo {
|
||||
inner,
|
||||
errors: self.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn start_driver(&self) {
|
||||
// The preface write retries Interrupted; only driver errors are terminal.
|
||||
self.0.running.store(true, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
fn record(&self, error: io::Error) -> io::Error {
|
||||
if !self.0.running.load(Ordering::Relaxed) || self.0.terminal.get().is_some() {
|
||||
return error;
|
||||
}
|
||||
let copy = match (error.raw_os_error(), error.get_ref()) {
|
||||
(Some(code), _) => io::Error::from_raw_os_error(code),
|
||||
(None, Some(inner)) => io::Error::new(error.kind(), inner.to_string()),
|
||||
(None, None) => error.kind().into(),
|
||||
};
|
||||
let _ = self.0.terminal.set(Some(Arc::new(error)));
|
||||
copy
|
||||
}
|
||||
|
||||
fn closed(&self) {
|
||||
if self.0.running.load(Ordering::Relaxed) {
|
||||
let _ = self.0.terminal.set(None);
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn cause(&self, error: &h2::Error) -> Option<Arc<io::Error>> {
|
||||
let reported = error.get_io()?;
|
||||
let original = self.0.terminal.get()?.as_ref()?;
|
||||
let text = match original.get_ref() {
|
||||
Some(inner) => inner.to_string(),
|
||||
None => io::Error::from(original.kind()).to_string(),
|
||||
};
|
||||
let matches = reported.kind() == original.kind() && reported.to_string() == text;
|
||||
// h2's PING handle reports a bare BrokenPipe when the failed driver exits.
|
||||
let closed = reported.kind() == io::ErrorKind::BrokenPipe && reported.get_ref().is_none();
|
||||
if matches || closed {
|
||||
Some(original.clone())
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn map(&self, error: h2::Error) -> io::Error {
|
||||
let cause = self.cause(&error);
|
||||
Self::map_captured(error, cause)
|
||||
}
|
||||
|
||||
pub(crate) fn map_captured(error: h2::Error, cause: Option<Arc<io::Error>>) -> io::Error {
|
||||
match cause {
|
||||
Some(cause) => io::Error::other(RecoveredError {
|
||||
error,
|
||||
cause: Some(cause),
|
||||
label: None,
|
||||
}),
|
||||
None => io::Error::other(error),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn map_with(&self, label: &'static str, error: h2::Error) -> io::Error {
|
||||
if !error.is_io() {
|
||||
return io::Error::other(format!("{label}: {error}"));
|
||||
}
|
||||
io::Error::other(RecoveredError {
|
||||
cause: self.cause(&error),
|
||||
error,
|
||||
label: Some(label),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct RecoveredError {
|
||||
pub(crate) error: h2::Error,
|
||||
cause: Option<Arc<io::Error>>,
|
||||
label: Option<&'static str>,
|
||||
}
|
||||
|
||||
impl fmt::Display for RecoveredError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
if let Some(label) = self.label {
|
||||
write!(f, "{label}: ")?;
|
||||
}
|
||||
self.error.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl Error for RecoveredError {
|
||||
fn source(&self) -> Option<&(dyn Error + 'static)> {
|
||||
match &self.cause {
|
||||
Some(cause) => Some(cause.as_ref()),
|
||||
None => self.error.get_io().map(|error| error as &dyn Error),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct ObservedIo<T> {
|
||||
inner: T,
|
||||
errors: TransportErrors,
|
||||
}
|
||||
|
||||
impl<T: AsyncRead + Unpin> AsyncRead for ObservedIo<T> {
|
||||
fn poll_read(
|
||||
self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
buf: &mut ReadBuf<'_>,
|
||||
) -> Poll<io::Result<()>> {
|
||||
let this = self.get_mut();
|
||||
let filled = buf.filled().len();
|
||||
let remaining = buf.remaining();
|
||||
let result = Pin::new(&mut this.inner).poll_read(cx, buf);
|
||||
if matches!(result, Poll::Ready(Ok(()))) && remaining > 0 && buf.filled().len() == filled {
|
||||
this.errors.closed();
|
||||
}
|
||||
result.map_err(|error| this.errors.record(error))
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: AsyncWrite + Unpin> AsyncWrite for ObservedIo<T> {
|
||||
fn poll_write(
|
||||
self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
buf: &[u8],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
let this = self.get_mut();
|
||||
let result = Pin::new(&mut this.inner).poll_write(cx, buf);
|
||||
if matches!(result, Poll::Ready(Ok(0))) && !buf.is_empty() {
|
||||
this.errors.closed();
|
||||
}
|
||||
result.map_err(|error| this.errors.record(error))
|
||||
}
|
||||
|
||||
fn poll_write_vectored(
|
||||
self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
bufs: &[io::IoSlice<'_>],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
let this = self.get_mut();
|
||||
let result = Pin::new(&mut this.inner).poll_write_vectored(cx, bufs);
|
||||
if matches!(result, Poll::Ready(Ok(0))) && bufs.iter().any(|buf| !buf.is_empty()) {
|
||||
this.errors.closed();
|
||||
}
|
||||
result.map_err(|error| this.errors.record(error))
|
||||
}
|
||||
|
||||
fn is_write_vectored(&self) -> bool {
|
||||
self.inner.is_write_vectored()
|
||||
}
|
||||
|
||||
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
let this = self.get_mut();
|
||||
Pin::new(&mut this.inner)
|
||||
.poll_flush(cx)
|
||||
.map_err(|error| this.errors.record(error))
|
||||
}
|
||||
|
||||
fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
let this = self.get_mut();
|
||||
Pin::new(&mut this.inner)
|
||||
.poll_shutdown(cx)
|
||||
.map_err(|error| this.errors.record(error))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
use std::{
|
||||
io,
|
||||
pin::Pin,
|
||||
sync::{
|
||||
Arc,
|
||||
atomic::{AtomicBool, Ordering},
|
||||
},
|
||||
task::{Context, Poll},
|
||||
};
|
||||
|
||||
use bytes::Bytes;
|
||||
use futures::task::AtomicWaker;
|
||||
use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, ReadBuf};
|
||||
|
||||
use super::TransportErrors;
|
||||
use crate::async_stream::{AsyncPing, AsyncStream};
|
||||
|
||||
struct ErrorIo {
|
||||
error: fn() -> io::Error,
|
||||
calls: Vec<&'static str>,
|
||||
closed: bool,
|
||||
interrupt_preface: bool,
|
||||
}
|
||||
|
||||
impl ErrorIo {
|
||||
fn new(error: fn() -> io::Error) -> Self {
|
||||
Self {
|
||||
error,
|
||||
calls: Vec::new(),
|
||||
closed: false,
|
||||
interrupt_preface: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AsyncRead for ErrorIo {
|
||||
fn poll_read(
|
||||
mut self: Pin<&mut Self>,
|
||||
_: &mut Context<'_>,
|
||||
buf: &mut ReadBuf<'_>,
|
||||
) -> Poll<io::Result<()>> {
|
||||
self.calls.push("read");
|
||||
if self.closed || buf.remaining() == 0 {
|
||||
Poll::Ready(Ok(()))
|
||||
} else {
|
||||
Poll::Ready(Err((self.error)()))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AsyncWrite for ErrorIo {
|
||||
fn poll_write(
|
||||
mut self: Pin<&mut Self>,
|
||||
_: &mut Context<'_>,
|
||||
buf: &[u8],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
self.calls.push("write");
|
||||
if self.interrupt_preface {
|
||||
self.interrupt_preface = false;
|
||||
return Poll::Ready(Err(io::ErrorKind::Interrupted.into()));
|
||||
}
|
||||
if self.closed || buf.is_empty() {
|
||||
Poll::Ready(Ok(0))
|
||||
} else {
|
||||
Poll::Ready(Err((self.error)()))
|
||||
}
|
||||
}
|
||||
fn is_write_vectored(&self) -> bool {
|
||||
true
|
||||
}
|
||||
fn poll_write_vectored(
|
||||
mut self: Pin<&mut Self>,
|
||||
_: &mut Context<'_>,
|
||||
_: &[io::IoSlice<'_>],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
self.calls.push("vectored");
|
||||
Poll::Ready(Err((self.error)()))
|
||||
}
|
||||
fn poll_flush(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
self.calls.push("flush");
|
||||
Poll::Ready(Err((self.error)()))
|
||||
}
|
||||
fn poll_shutdown(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
self.calls.push("shutdown");
|
||||
Poll::Ready(Err((self.error)()))
|
||||
}
|
||||
}
|
||||
|
||||
async fn h2_copy(error: &io::Error) -> h2::Error {
|
||||
struct PrefaceError(Option<io::Error>);
|
||||
impl AsyncRead for PrefaceError {
|
||||
fn poll_read(
|
||||
self: Pin<&mut Self>,
|
||||
_: &mut Context<'_>,
|
||||
_: &mut ReadBuf<'_>,
|
||||
) -> Poll<io::Result<()>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl AsyncWrite for PrefaceError {
|
||||
fn poll_write(
|
||||
mut self: Pin<&mut Self>,
|
||||
_: &mut Context<'_>,
|
||||
_: &[u8],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
Poll::Ready(Err(self.0.take().unwrap()))
|
||||
}
|
||||
fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
unreachable!()
|
||||
}
|
||||
fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
let copy = match error.get_ref() {
|
||||
Some(inner) => io::Error::new(error.kind(), inner.to_string()),
|
||||
None => io::Error::from(error.kind()),
|
||||
};
|
||||
h2::client::handshake(PrefaceError(Some(copy)))
|
||||
.await
|
||||
.err()
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn observer_delegates_and_recovers_causes_without_changing_h2_messages() {
|
||||
for (make_error, local) in cases() {
|
||||
for operation in ["read", "write", "vectored", "flush", "shutdown"] {
|
||||
let errors = TransportErrors::default();
|
||||
errors.start_driver();
|
||||
let mut stream = errors.wrap(ErrorIo::new(make_error));
|
||||
let mut cx = Context::from_waker(std::task::Waker::noop());
|
||||
let mut bytes = [0; 1];
|
||||
assert!(stream.is_write_vectored());
|
||||
let result = match operation {
|
||||
"read" => Pin::new(&mut stream).poll_read(&mut cx, &mut ReadBuf::new(&mut bytes)),
|
||||
"write" => Pin::new(&mut stream)
|
||||
.poll_write(&mut cx, b"x")
|
||||
.map_ok(|_| ()),
|
||||
"vectored" => Pin::new(&mut stream)
|
||||
.poll_write_vectored(&mut cx, &[io::IoSlice::new(b"x")])
|
||||
.map_ok(|_| ()),
|
||||
"flush" => Pin::new(&mut stream).poll_flush(&mut cx),
|
||||
"shutdown" => Pin::new(&mut stream).poll_shutdown(&mut cx),
|
||||
_ => unreachable!(),
|
||||
};
|
||||
let Poll::Ready(Err(returned)) = result else {
|
||||
panic!("missing error")
|
||||
};
|
||||
let original = make_error();
|
||||
assert_eq!(returned.kind(), original.kind());
|
||||
assert_eq!(returned.raw_os_error(), original.raw_os_error());
|
||||
assert_eq!(returned.to_string(), original.to_string());
|
||||
assert_eq!(stream.inner.calls, [operation]);
|
||||
for label in [None, Some("H2 recv error")] {
|
||||
let h2_error = h2_copy(&returned).await;
|
||||
let display = match label {
|
||||
Some(label) => format!("{label}: {h2_error}"),
|
||||
None => h2_error.to_string(),
|
||||
};
|
||||
let mapped = match label {
|
||||
Some(label) => errors.map_with(label, h2_error),
|
||||
None => errors.map(h2_error),
|
||||
};
|
||||
assert_eq!(mapped.kind(), io::ErrorKind::Other);
|
||||
assert_eq!(mapped.to_string(), display);
|
||||
assert_eq!(
|
||||
crate::clash_api::probes::local_failure(&mapped),
|
||||
local,
|
||||
"{mapped:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn first_terminal_event_wins_and_recovery_is_connection_scoped() {
|
||||
for (make_error, local) in cases() {
|
||||
let errors = TransportErrors::default();
|
||||
errors.start_driver();
|
||||
let first = errors.record(make_error());
|
||||
let _ = errors.record(io::ErrorKind::OutOfMemory.into());
|
||||
for _ in 0..2 {
|
||||
let mapped = errors.map(h2_copy(&first).await);
|
||||
assert_eq!(crate::clash_api::probes::local_failure(&mapped), local);
|
||||
}
|
||||
let closed = errors.map(h2_copy(&io::ErrorKind::BrokenPipe.into()).await);
|
||||
assert_eq!(crate::clash_api::probes::local_failure(&closed), local);
|
||||
let reset = errors.map(h2::Reason::REFUSED_STREAM.into());
|
||||
assert!(reset.get_ref().unwrap().is::<h2::Error>());
|
||||
assert!(!crate::clash_api::probes::local_failure(&reset));
|
||||
let mismatched =
|
||||
errors.map(h2_copy(&io::Error::other("unrelated transport failure")).await);
|
||||
assert!(mismatched.get_ref().unwrap().is::<h2::Error>());
|
||||
let fresh = TransportErrors::default().map(h2_copy(&first).await);
|
||||
assert!(fresh.get_ref().unwrap().is::<h2::Error>());
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn empty_buffers_do_not_close_observation_but_eof_and_write_zero_do() {
|
||||
for read in [false, true] {
|
||||
for empty in [false, true] {
|
||||
let errors = TransportErrors::default();
|
||||
errors.start_driver();
|
||||
let mut inner = ErrorIo::new(crate::resources::exhausted);
|
||||
inner.closed = true;
|
||||
let mut stream = errors.wrap(inner);
|
||||
let mut cx = Context::from_waker(std::task::Waker::noop());
|
||||
let mut bytes = vec![0; usize::from(!empty)];
|
||||
if read {
|
||||
assert!(
|
||||
Pin::new(&mut stream)
|
||||
.poll_read(&mut cx, &mut ReadBuf::new(&mut bytes))
|
||||
.is_ready()
|
||||
);
|
||||
} else {
|
||||
assert!(Pin::new(&mut stream).poll_write(&mut cx, &bytes).is_ready());
|
||||
}
|
||||
let returned = errors.record(crate::resources::exhausted());
|
||||
let mapped = errors.map(h2_copy(&returned).await);
|
||||
assert_eq!(crate::clash_api::probes::local_failure(&mapped), empty);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn handshake_errors_keep_sources_without_recording_retried_interruptions() {
|
||||
for (make_error, local) in cases() {
|
||||
let errors = TransportErrors::default();
|
||||
let mut inner = ErrorIo::new(make_error);
|
||||
inner.interrupt_preface = true;
|
||||
let error = h2::client::handshake(errors.wrap(inner))
|
||||
.await
|
||||
.err()
|
||||
.unwrap();
|
||||
assert!(errors.0.terminal.get().is_none());
|
||||
let mapped = errors.map_with("H2 client handshake failed", error);
|
||||
assert_eq!(
|
||||
crate::clash_api::probes::local_failure(&mapped),
|
||||
local,
|
||||
"{mapped:?}"
|
||||
);
|
||||
errors.start_driver();
|
||||
let returned = errors.record(io::ErrorKind::ConnectionReset.into());
|
||||
assert!(!crate::clash_api::probes::local_failure(
|
||||
&errors.map(h2_copy(&returned).await)
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub(crate) struct Fault {
|
||||
failed: Arc<AtomicBool>,
|
||||
waker: Arc<AtomicWaker>,
|
||||
error: fn() -> io::Error,
|
||||
}
|
||||
|
||||
impl Fault {
|
||||
pub(crate) fn new(error: fn() -> io::Error) -> Self {
|
||||
Self {
|
||||
failed: Arc::new(AtomicBool::new(false)),
|
||||
waker: Arc::new(AtomicWaker::new()),
|
||||
error,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn inject(&self) {
|
||||
self.failed.store(true, Ordering::SeqCst);
|
||||
self.waker.wake();
|
||||
}
|
||||
|
||||
pub(crate) fn wrap<T>(&self, inner: T) -> FaultIo<T> {
|
||||
FaultIo {
|
||||
inner,
|
||||
fault: self.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct FaultIo<T> {
|
||||
inner: T,
|
||||
fault: Fault,
|
||||
}
|
||||
|
||||
impl<T: AsyncRead + Unpin> AsyncRead for FaultIo<T> {
|
||||
fn poll_read(
|
||||
mut self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
buf: &mut ReadBuf<'_>,
|
||||
) -> Poll<io::Result<()>> {
|
||||
self.fault.waker.register(cx.waker());
|
||||
if self.fault.failed.load(Ordering::SeqCst) {
|
||||
Poll::Ready(Err((self.fault.error)()))
|
||||
} else {
|
||||
Pin::new(&mut self.inner).poll_read(cx, buf)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: AsyncWrite + Unpin> AsyncWrite for FaultIo<T> {
|
||||
fn poll_write(
|
||||
mut self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
buf: &[u8],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
Pin::new(&mut self.inner).poll_write(cx, buf)
|
||||
}
|
||||
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
Pin::new(&mut self.inner).poll_flush(cx)
|
||||
}
|
||||
fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
Pin::new(&mut self.inner).poll_shutdown(cx)
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: AsyncStream> AsyncPing for FaultIo<T> {
|
||||
fn supports_ping(&self) -> bool {
|
||||
false
|
||||
}
|
||||
fn poll_write_ping(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<bool>> {
|
||||
Poll::Ready(Ok(false))
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: AsyncStream> AsyncStream for FaultIo<T> {}
|
||||
|
||||
pub(crate) fn peer(
|
||||
error: fn() -> io::Error,
|
||||
) -> (
|
||||
FaultIo<tokio::io::DuplexStream>,
|
||||
Fault,
|
||||
tokio::sync::mpsc::UnboundedReceiver<h2::SendStream<Bytes>>,
|
||||
tokio::task::JoinHandle<()>,
|
||||
) {
|
||||
let fault = Fault::new(error);
|
||||
let (client, peer) = tokio::io::duplex(8192);
|
||||
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
|
||||
let driver = tokio::spawn(async move {
|
||||
let mut connection = h2::server::handshake(peer).await.unwrap();
|
||||
let mut bodies = tokio::task::JoinSet::new();
|
||||
while let Some(Ok((request, mut respond))) = connection.accept().await {
|
||||
let mut send = respond
|
||||
.send_response(http::Response::new(()), false)
|
||||
.unwrap();
|
||||
send.send_data(Bytes::from_static(b"x"), false).unwrap();
|
||||
// Retain the request body so dropping it cannot reset a live CONNECT.
|
||||
bodies.spawn(async move {
|
||||
let mut body = request.into_body();
|
||||
while let Some(Ok(data)) = body.data().await {
|
||||
let _ = body.flow_control().release_capacity(data.len());
|
||||
}
|
||||
});
|
||||
if tx.send(send).is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
(fault.wrap(client), fault, rx, driver)
|
||||
}
|
||||
|
||||
pub(crate) fn cases() -> Vec<(fn() -> io::Error, bool)> {
|
||||
vec![
|
||||
(crate::resources::exhausted, true),
|
||||
#[cfg(unix)]
|
||||
(|| io::Error::from_raw_os_error(libc::EMFILE), true),
|
||||
(|| io::ErrorKind::ConnectionReset.into(), false),
|
||||
]
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn established_streams_share_transport_cause_but_not_remote_resets() {
|
||||
tokio::time::timeout(std::time::Duration::from_secs(5), async {
|
||||
for (make_error, local) in cases() {
|
||||
let (client, fault, mut sends, driver) = peer(make_error);
|
||||
let mut session = crate::http2::client::H2ClientSession::new(Box::new(client))
|
||||
.await
|
||||
.unwrap();
|
||||
let mut streams = Vec::new();
|
||||
let mut peers = Vec::new();
|
||||
for _ in 0..3 {
|
||||
let request = http::Request::builder()
|
||||
.method("CONNECT")
|
||||
.uri("example.com:80")
|
||||
.body(())
|
||||
.unwrap();
|
||||
let (_, mut stream) = session.open(request).await.unwrap();
|
||||
assert_eq!(stream.read_u8().await.unwrap(), b'x');
|
||||
streams.push(stream);
|
||||
peers.push(sends.recv().await.unwrap());
|
||||
}
|
||||
peers[0].send_reset(h2::Reason::REFUSED_STREAM);
|
||||
let reset = streams[0].read_u8().await.unwrap_err();
|
||||
assert!(!crate::clash_api::probes::local_failure(&reset));
|
||||
assert!(reset.get_ref().unwrap().is::<h2::Error>());
|
||||
fault.inject();
|
||||
for stream in &mut streams[1..] {
|
||||
let error = stream.read_u8().await.unwrap_err();
|
||||
assert_eq!(
|
||||
crate::clash_api::probes::local_failure(&error),
|
||||
local,
|
||||
"{error:?}"
|
||||
);
|
||||
}
|
||||
assert!(!crate::clash_api::probes::local_failure(&reset));
|
||||
drop(streams);
|
||||
drop(session);
|
||||
driver.abort();
|
||||
let _ = driver.await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
@@ -23,6 +23,13 @@ pub struct H2MultiStream {
|
||||
}
|
||||
|
||||
impl H2MultiStream {
|
||||
fn map_error(&self, label: &'static str, error: h2::Error) -> io::Error {
|
||||
match &self.session_owner {
|
||||
Some(session) => session.errors.map_with(label, error),
|
||||
None => io::Error::other(format!("{label}: {error}")),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn new(send: h2::SendStream<Bytes>, recv: h2::RecvStream) -> Self {
|
||||
Self {
|
||||
send,
|
||||
@@ -65,9 +72,7 @@ impl AsyncRead for H2MultiStream {
|
||||
|
||||
Poll::Ready(Ok(()))
|
||||
}
|
||||
Poll::Ready(Some(Err(e))) => {
|
||||
Poll::Ready(Err(io::Error::other(format!("H2 recv error: {e}"))))
|
||||
}
|
||||
Poll::Ready(Some(Err(e))) => Poll::Ready(Err(self.map_error("H2 recv error", e))),
|
||||
Poll::Ready(None) => Poll::Ready(Ok(())),
|
||||
Poll::Pending => Poll::Pending,
|
||||
}
|
||||
@@ -95,15 +100,13 @@ impl AsyncWrite for H2MultiStream {
|
||||
let to_send = buf.len().min(capacity);
|
||||
self.send
|
||||
.send_data(Bytes::copy_from_slice(&buf[..to_send]), false)
|
||||
.map_err(|e| io::Error::other(format!("H2 send_data failed: {e}")))?;
|
||||
.map_err(|e| self.map_error("H2 send_data failed", e))?;
|
||||
return Poll::Ready(Ok(to_send));
|
||||
}
|
||||
match std::task::ready!(self.send.poll_capacity(cx)) {
|
||||
Some(Ok(_)) => continue,
|
||||
Some(Err(e)) => {
|
||||
return Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 poll_capacity error: {e}"
|
||||
))));
|
||||
return Poll::Ready(Err(self.map_error("H2 poll_capacity error", e)));
|
||||
}
|
||||
None => {
|
||||
return Poll::Ready(Err(io::Error::new(
|
||||
@@ -125,9 +128,7 @@ impl AsyncWrite for H2MultiStream {
|
||||
match self.send.send_data(Bytes::new(), true) {
|
||||
Ok(()) => self.shutdown_sent = true,
|
||||
Err(e) => {
|
||||
return Poll::Ready(Err(io::Error::other(format!(
|
||||
"H2 send END_STREAM failed: {e}"
|
||||
))));
|
||||
return Poll::Ready(Err(self.map_error("H2 send END_STREAM failed", e)));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,6 +26,7 @@ use tokio::sync::oneshot;
|
||||
|
||||
use crate::address::{Address, NetLocation};
|
||||
use crate::async_stream::AsyncStream;
|
||||
use crate::http2::transport::TransportErrors;
|
||||
|
||||
use super::h2_multi_stream::H2MultiStream;
|
||||
use super::naive_padding_stream::{
|
||||
@@ -41,6 +42,7 @@ use super::naive_padding_stream::{
|
||||
/// H2 connection (via h2's internal `Arc<Mutex<...>>`).
|
||||
#[derive(Clone)]
|
||||
pub struct NaiveClientSession {
|
||||
pub(super) errors: TransportErrors,
|
||||
/// The SendRequest handle - has internal Arc, cheap to clone
|
||||
send_request: h2::client::SendRequest<Bytes>,
|
||||
driver_handle: Arc<DriverHandle>,
|
||||
@@ -61,14 +63,16 @@ impl NaiveClientSession {
|
||||
const WINDOW_SIZE: u32 = 256 * 1024; // 256 KB (was 16 MB)
|
||||
const MAX_FRAME_SIZE: u32 = 16 * 1024;
|
||||
|
||||
let errors = TransportErrors::default();
|
||||
let (send_request, mut connection) = h2::client::Builder::new()
|
||||
.initial_window_size(WINDOW_SIZE)
|
||||
.initial_connection_window_size(WINDOW_SIZE)
|
||||
.max_frame_size(MAX_FRAME_SIZE)
|
||||
.max_concurrent_streams(1024)
|
||||
.handshake(stream)
|
||||
.handshake(errors.wrap(stream))
|
||||
.await
|
||||
.map_err(|e| io::Error::other(format!("H2 client handshake failed: {}", e)))?;
|
||||
.map_err(|e| errors.map_with("H2 client handshake failed", e))?;
|
||||
errors.start_driver();
|
||||
|
||||
let (drain_tx, drain_rx) = oneshot::channel();
|
||||
let abort_handle = tokio::spawn(async move {
|
||||
@@ -86,6 +90,7 @@ impl NaiveClientSession {
|
||||
debug!("NaiveClientSession: H2 handshake complete, session ready for multiplexing");
|
||||
|
||||
Ok(Self {
|
||||
errors,
|
||||
send_request,
|
||||
driver_handle: Arc::new(DriverHandle {
|
||||
driver: abort_handle,
|
||||
@@ -132,15 +137,15 @@ impl NaiveClientSession {
|
||||
|
||||
std::future::poll_fn(|cx| self.send_request.poll_ready(cx))
|
||||
.await
|
||||
.map_err(|e| io::Error::other(format!("H2 session closed: {e}")))?;
|
||||
.map_err(|e| self.errors.map_with("H2 session closed", e))?;
|
||||
let (response_future, send_stream) = self
|
||||
.send_request
|
||||
.send_request(request, false)
|
||||
.map_err(|e| io::Error::other(format!("Failed to send CONNECT: {}", e)))?;
|
||||
.map_err(|e| self.errors.map_with("Failed to send CONNECT", e))?;
|
||||
|
||||
let response = response_future
|
||||
.await
|
||||
.map_err(|e| io::Error::other(format!("CONNECT response error: {}", e)))?;
|
||||
.map_err(|e| self.errors.map_with("CONNECT response error", e))?;
|
||||
|
||||
debug!(
|
||||
"NaiveClientSession: CONNECT response: status={}, headers={:?}",
|
||||
@@ -203,6 +208,47 @@ fn format_authority(location: &NetLocation) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod transport_tests {
|
||||
use super::*;
|
||||
use crate::http2::transport::tests::{cases, peer};
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
#[tokio::test]
|
||||
async fn established_stream_preserves_transport_causes() {
|
||||
tokio::time::timeout(std::time::Duration::from_secs(5), async {
|
||||
for (make_error, local) in cases() {
|
||||
let (client, fault, mut sends, driver) = peer(make_error);
|
||||
let mut session = NaiveClientSession::new(Box::new(client)).await.unwrap();
|
||||
let mut stream = session
|
||||
.open_stream(
|
||||
&NetLocation::from_str("example.com:80", None).unwrap(),
|
||||
"Basic test",
|
||||
false,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let _send = sends.recv().await.unwrap();
|
||||
assert_eq!(stream.read_u8().await.unwrap(), b'x');
|
||||
fault.inject();
|
||||
let error = stream.read_u8().await.unwrap_err();
|
||||
assert!(error.to_string().starts_with("H2 recv error: "));
|
||||
assert_eq!(
|
||||
crate::clash_api::probes::local_failure(&error),
|
||||
local,
|
||||
"{error:?}"
|
||||
);
|
||||
drop(stream);
|
||||
drop(session);
|
||||
driver.abort();
|
||||
let _ = driver.await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
+43
-3
@@ -183,10 +183,36 @@ pub(crate) fn try_stream() -> Option<BudgetPermit> {
|
||||
STREAMS.acquire(1)
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct ResourceExhausted(String);
|
||||
|
||||
impl std::fmt::Display for ResourceExhausted {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
formatter.write_str(&self.0)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for ResourceExhausted {}
|
||||
|
||||
pub(crate) fn is_exhausted(mut error: &(dyn std::error::Error + 'static)) -> bool {
|
||||
loop {
|
||||
if error.is::<ResourceExhausted>() {
|
||||
return true;
|
||||
}
|
||||
// io::Error::source can skip the wrapped error itself.
|
||||
let source = match error.downcast_ref::<std::io::Error>() {
|
||||
Some(error) => error.get_ref().map(|inner| inner as &dyn std::error::Error),
|
||||
None => error.source(),
|
||||
};
|
||||
let Some(source) = source else { return false };
|
||||
error = source;
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn exhausted() -> std::io::Error {
|
||||
std::io::Error::new(
|
||||
std::io::ErrorKind::ConnectionRefused,
|
||||
"process resource budget exhausted",
|
||||
ResourceExhausted("process resource budget exhausted".into()),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -204,10 +230,10 @@ pub(crate) fn quic_memory_exhausted(bytes: usize) -> std::io::Error {
|
||||
.map_or_else(|| "unlimited".to_owned(), |limit| limit.to_string());
|
||||
std::io::Error::new(
|
||||
std::io::ErrorKind::ConnectionRefused,
|
||||
format!(
|
||||
ResourceExhausted(format!(
|
||||
"QUIC memory budget exhausted: requested {bytes} bytes, active {} bytes, live cap {limit}",
|
||||
state.active,
|
||||
),
|
||||
)),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -331,6 +357,20 @@ impl Drop for ResourceReporter {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn local_budget_errors_remain_identifiable_through_wrappers() {
|
||||
let error = exhausted();
|
||||
assert_eq!(error.kind(), std::io::ErrorKind::ConnectionRefused);
|
||||
assert_eq!(error.to_string(), "process resource budget exhausted");
|
||||
assert!(is_exhausted(&error));
|
||||
let error = std::io::Error::other(std::io::Error::other(error));
|
||||
assert!(is_exhausted(&error));
|
||||
assert!(!is_exhausted(&std::io::Error::new(
|
||||
std::io::ErrorKind::ConnectionRefused,
|
||||
"process resource budget exhausted",
|
||||
)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unlimited_budget_tracks_usage_without_a_semaphore_ceiling() {
|
||||
let budget = Arc::new(Budget::new(None));
|
||||
|
||||
@@ -34,6 +34,9 @@ use crate::resolver::Resolver;
|
||||
reason = "async-trait adds must_use to boxed futures"
|
||||
)]
|
||||
pub trait SocketConnector: Send + Sync + Debug {
|
||||
/// Propagate new forwarding demand into a shared carrier without redialing it.
|
||||
fn record_demand(&self) {}
|
||||
|
||||
async fn connect_quic(
|
||||
&self,
|
||||
_resolver: &Arc<dyn Resolver>,
|
||||
|
||||
+130
-1
@@ -111,7 +111,7 @@ pub(super) fn start(
|
||||
incoming: incoming.clone(),
|
||||
drops: drops.clone(),
|
||||
};
|
||||
workers.spawn(worker.run());
|
||||
workers.spawn(crate::control::url_test::BACKGROUND.scope((), worker.run()));
|
||||
}
|
||||
drop(incoming);
|
||||
let shutdown = cancel.clone();
|
||||
@@ -349,6 +349,135 @@ mod tests {
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn url_test_idle_reconnects_do_not_activate_lazy_groups() {
|
||||
#[derive(Debug)]
|
||||
struct FailingSocket(Arc<AtomicU64>);
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl SocketConnector for FailingSocket {
|
||||
async fn connect(
|
||||
&self,
|
||||
_: &Arc<dyn Resolver>,
|
||||
_: &ResolvedLocation,
|
||||
) -> io::Result<Box<dyn AsyncStream>> {
|
||||
unreachable!()
|
||||
}
|
||||
|
||||
async fn connect_udp_bidirectional(
|
||||
&self,
|
||||
_: &Arc<dyn Resolver>,
|
||||
_: ResolvedLocation,
|
||||
) -> io::Result<Box<dyn AsyncMessageStream>> {
|
||||
self.0.fetch_add(1, Ordering::Relaxed);
|
||||
Err(io::ErrorKind::ConnectionRefused.into())
|
||||
}
|
||||
|
||||
fn bind_interface(&self) -> Option<&str> {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
let mut selection = crate::control::selection::SelectionView::default();
|
||||
selection
|
||||
.automatic
|
||||
.insert(
|
||||
"carrier".into(),
|
||||
crate::config::UrlTestConfig {
|
||||
url: "http://probe.invalid/".into(),
|
||||
interval: 10,
|
||||
timeout: 1000,
|
||||
tolerance: 50,
|
||||
lazy: true,
|
||||
},
|
||||
vec!["peer".into()],
|
||||
)
|
||||
.unwrap();
|
||||
let selection = Arc::new(selection);
|
||||
let attempts = Arc::new(AtomicU64::new(0));
|
||||
let chain = Arc::new(
|
||||
ClientProxyChain::new(
|
||||
vec![InitialHopEntry::Direct(Arc::new(FailingSocket(
|
||||
attempts.clone(),
|
||||
)))],
|
||||
vec![],
|
||||
)
|
||||
.with_origins(vec![vec![Some(crate::control::catalog::MemberOrigin {
|
||||
name: Arc::from("peer"),
|
||||
path: vec![("carrier".into(), "peer".into())],
|
||||
selection: selection.clone(),
|
||||
})]]),
|
||||
);
|
||||
let group = &selection.automatic.0["carrier"];
|
||||
assert!(group.due_members(Instant::now()).is_some());
|
||||
let (_sender, receiver, released) = start(
|
||||
"idle".into(),
|
||||
chain.clone(),
|
||||
Arc::new(NativeResolver::new()),
|
||||
["127.0.0.1:51820".parse().unwrap()].into_iter(),
|
||||
Arc::new(AtomicU64::new(0)),
|
||||
);
|
||||
for _ in 0..20 {
|
||||
tokio::time::advance(Duration::from_secs(1)).await;
|
||||
tokio::task::yield_now().await;
|
||||
}
|
||||
assert!(attempts.load(Ordering::Relaxed) > 1);
|
||||
assert!(group.due_members(Instant::now()).is_none());
|
||||
drop(receiver);
|
||||
released.await.unwrap();
|
||||
|
||||
let stack = crate::ip_stack::driver::PacketStack::new(
|
||||
vec!["10.0.0.2".parse().unwrap()],
|
||||
1420,
|
||||
false,
|
||||
Default::default(),
|
||||
)
|
||||
.unwrap();
|
||||
let connector = |carrier| {
|
||||
Arc::new(crate::wireguard::Connector {
|
||||
name: "demand".into(),
|
||||
active: Mutex::new(Some(crate::wireguard::ActiveEndpoint {
|
||||
handle: stack.handle.clone(),
|
||||
carrier: Some(carrier),
|
||||
})),
|
||||
addresses: vec!["10.0.0.2".parse().unwrap()],
|
||||
routes: vec!["0.0.0.0/0".parse().unwrap()],
|
||||
})
|
||||
};
|
||||
let outer = connector(chain);
|
||||
let nested = connector(Arc::new(ClientProxyChain::new(
|
||||
vec![InitialHopEntry::Wireguard(outer)],
|
||||
vec![],
|
||||
)));
|
||||
let resolver: Arc<dyn Resolver> = Arc::new(NativeResolver::new());
|
||||
let target: ResolvedLocation =
|
||||
NetLocation::from_ip_addr("10.0.0.1".parse().unwrap(), 80).into();
|
||||
for udp in [false, true] {
|
||||
for background in [true, false] {
|
||||
let connect = async {
|
||||
if udp {
|
||||
let attempt = nested.connect_udp_bidirectional(&resolver, target.clone());
|
||||
tokio::pin!(attempt);
|
||||
let _ = futures::poll!(&mut attempt);
|
||||
} else {
|
||||
let attempt = nested.connect(&resolver, &target);
|
||||
tokio::pin!(attempt);
|
||||
let _ = futures::poll!(&mut attempt);
|
||||
}
|
||||
};
|
||||
if background {
|
||||
crate::control::url_test::BACKGROUND
|
||||
.scope((), connect)
|
||||
.await;
|
||||
} else {
|
||||
connect.await;
|
||||
}
|
||||
assert_eq!(group.due_members(Instant::now()).is_some(), !background);
|
||||
tokio::time::advance(Duration::from_secs(11)).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn read(stream: &mut dyn AsyncMessageStream) -> Vec<u8> {
|
||||
let mut bytes = vec![0; 65536];
|
||||
let mut buffer = ReadBuf::new(&mut bytes);
|
||||
|
||||
+32
-9
@@ -44,10 +44,16 @@ fn canonical_socket_addr(mut address: SocketAddr) -> SocketAddr {
|
||||
|
||||
struct Connector {
|
||||
name: String,
|
||||
active: Mutex<Option<StackHandle>>,
|
||||
active: Mutex<Option<ActiveEndpoint>>,
|
||||
addresses: Vec<IpAddr>,
|
||||
routes: Vec<IpNetwork>,
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct ActiveEndpoint {
|
||||
handle: StackHandle,
|
||||
carrier: Option<Arc<crate::client_proxy_chain::ClientProxyChain>>,
|
||||
}
|
||||
impl fmt::Debug for Connector {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.debug_struct("WireguardConnector")
|
||||
@@ -57,12 +63,16 @@ impl fmt::Debug for Connector {
|
||||
}
|
||||
impl Connector {
|
||||
fn handle(&self) -> stdio::Result<StackHandle> {
|
||||
self.active.lock().clone().ok_or_else(|| {
|
||||
let active = self.active.lock().clone().ok_or_else(|| {
|
||||
stdio::Error::new(
|
||||
stdio::ErrorKind::NotConnected,
|
||||
format!("WireGuard endpoint {} is not active", self.name),
|
||||
)
|
||||
})
|
||||
})?;
|
||||
if let Some(carrier) = active.carrier {
|
||||
carrier.record_carrier_demand();
|
||||
}
|
||||
Ok(active.handle)
|
||||
}
|
||||
fn reachable(&self, address: SocketAddr) -> bool {
|
||||
self.addresses
|
||||
@@ -99,6 +109,17 @@ impl Connector {
|
||||
}
|
||||
#[async_trait]
|
||||
impl SocketConnector for Connector {
|
||||
fn record_demand(&self) {
|
||||
let carrier = self
|
||||
.active
|
||||
.lock()
|
||||
.as_ref()
|
||||
.and_then(|active| active.carrier.clone());
|
||||
if let Some(carrier) = carrier {
|
||||
carrier.record_carrier_demand();
|
||||
}
|
||||
}
|
||||
|
||||
async fn connect(
|
||||
&self,
|
||||
resolver: &Arc<dyn Resolver>,
|
||||
@@ -262,9 +283,7 @@ impl Prepared {
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
if !validated.control.listeners.is_empty()
|
||||
&& let Some(catalog) = &mut validated.catalog
|
||||
{
|
||||
if let Some(catalog) = &mut validated.catalog {
|
||||
for node in catalog.nodes.values_mut() {
|
||||
if let crate::control::catalog::CatalogNode::Leaf(client) = node {
|
||||
crate::config::client_visit::visit_client(client, &mut |client| {
|
||||
@@ -417,11 +436,12 @@ impl PreparedEndpoint {
|
||||
)?;
|
||||
let resolver = self.outer_resolver.clone();
|
||||
// Rebuilding after preflight gives each start fresh pool indices and handler lifetimes.
|
||||
let (udp, released, peers) = if let Some(chain) = self.build_outer_chain()? {
|
||||
let outer_chain = self.build_outer_chain()?;
|
||||
let (udp, released, peers) = if let Some(chain) = &outer_chain {
|
||||
let peers = self.peers_for_socket(None)?;
|
||||
let (sender, receiver, released) = detour::start(
|
||||
config.id.clone(),
|
||||
chain,
|
||||
chain.clone(),
|
||||
resolver,
|
||||
peers.iter().map(|peer| peer.endpoint.unwrap()),
|
||||
stack.drops.clone(),
|
||||
@@ -456,7 +476,10 @@ impl PreparedEndpoint {
|
||||
.map_err(|error| {
|
||||
stdio::Error::other(format!("WireGuard device startup failed: {error}"))
|
||||
})?;
|
||||
*self.connector.active.lock() = Some(stack.handle.clone());
|
||||
*self.connector.active.lock() = Some(ActiveEndpoint {
|
||||
handle: stack.handle.clone(),
|
||||
carrier: outer_chain,
|
||||
});
|
||||
let connector = self.connector.clone();
|
||||
let name = config.id.clone();
|
||||
let cancel = CancellationToken::new();
|
||||
|
||||
+88
-28
@@ -16,6 +16,7 @@ use tokio::io::AsyncWriteExt;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
use super::stream::{Body, Memory, Reader, Tasks, Upload, UploadSignal, UploadWatch, Writer, pipe};
|
||||
use crate::http2::transport::TransportErrors;
|
||||
use crate::{
|
||||
async_stream::AsyncStream, config::xhttp::XhttpVersion, tcp::carrier_dialer::QuicCarrier,
|
||||
};
|
||||
@@ -80,7 +81,10 @@ impl Timeouts {
|
||||
|
||||
enum Sender {
|
||||
H1(tokio::sync::Mutex<hyper::client::conn::http1::SendRequest<Body>>),
|
||||
H2(h2::client::SendRequest<Bytes>),
|
||||
H2 {
|
||||
sender: h2::client::SendRequest<Bytes>,
|
||||
errors: TransportErrors,
|
||||
},
|
||||
H3 {
|
||||
sender: H3Sender,
|
||||
opener: super::h3::Opener,
|
||||
@@ -131,13 +135,15 @@ impl Carrier {
|
||||
});
|
||||
Sender::H1(tokio::sync::Mutex::new(sender))
|
||||
} else {
|
||||
let errors = TransportErrors::default();
|
||||
let (sender, mut connection) = h2::client::Builder::new()
|
||||
.initial_window_size(stream_window)
|
||||
.initial_connection_window_size(connection_window)
|
||||
.max_header_list_size(header_limit as u32)
|
||||
.handshake(stream)
|
||||
.handshake(errors.wrap(stream))
|
||||
.await
|
||||
.map_err(io::Error::other)?;
|
||||
.map_err(|error| errors.map(error))?;
|
||||
errors.start_driver();
|
||||
let mut ping = connection.ping_pong().unwrap();
|
||||
tasks.spawn(async move {
|
||||
if let Err(error) = connection.await {
|
||||
@@ -147,8 +153,8 @@ impl Carrier {
|
||||
// The handshake alone does not read peer SETTINGS. PONG follows their application.
|
||||
ping.ping(h2::Ping::opaque())
|
||||
.await
|
||||
.map_err(io::Error::other)?;
|
||||
Sender::H2(sender)
|
||||
.map_err(|error| errors.map(error))?;
|
||||
Sender::H2 { sender, errors }
|
||||
};
|
||||
Ok(Arc::new(Self {
|
||||
sender,
|
||||
@@ -191,7 +197,7 @@ impl Carrier {
|
||||
}
|
||||
match &self.sender {
|
||||
Sender::H1(sender) => sender.try_lock().is_ok_and(|s| s.is_closed()),
|
||||
Sender::H2(sender) => matches!(
|
||||
Sender::H2 { sender, .. } => matches!(
|
||||
sender
|
||||
.clone()
|
||||
.poll_ready(&mut std::task::Context::from_waker(std::task::Waker::noop())),
|
||||
@@ -214,7 +220,7 @@ impl Carrier {
|
||||
pub fn capacity(&self, limit: usize) -> usize {
|
||||
match &self.sender {
|
||||
Sender::H1(_) => 1,
|
||||
Sender::H2(sender) => limit.min(sender.current_max_send_streams()),
|
||||
Sender::H2 { sender, .. } => limit.min(sender.current_max_send_streams()),
|
||||
Sender::H3 { .. } => limit,
|
||||
}
|
||||
}
|
||||
@@ -347,27 +353,28 @@ impl Carrier {
|
||||
});
|
||||
Response::from_parts(parts, reader)
|
||||
}
|
||||
Sender::H2(sender) => {
|
||||
Sender::H2 { sender, errors } => {
|
||||
let (parts, body) = request.into_parts();
|
||||
let empty = hyper::body::Body::is_end_stream(&body);
|
||||
let streaming = matches!(&body, Body::Stream { .. });
|
||||
let mut sender = tokio::time::timeout(timeouts.headers, sender.clone().ready())
|
||||
.await?
|
||||
.map_err(io::Error::other)?;
|
||||
.map_err(|error| errors.map(error))?;
|
||||
let (response, mut send) = sender
|
||||
.send_request(Request::from_parts(parts, ()), empty)
|
||||
.map_err(io::Error::other)?;
|
||||
.map_err(|error| errors.map(error))?;
|
||||
let request_cancel = tasks.cancel.child_token();
|
||||
let guard = request_cancel.clone().drop_guard();
|
||||
if !empty {
|
||||
let carrier = self.clone();
|
||||
let cancel = tasks.cancel.clone();
|
||||
let errors = errors.clone();
|
||||
tasks.spawn(async move {
|
||||
let _carrier = carrier;
|
||||
let result = tokio::select! {
|
||||
biased;
|
||||
_ = request_cancel.cancelled() => Err(H2SendError::Local(io::ErrorKind::ConnectionAborted.into())),
|
||||
result = send_h2_tracked(body, &mut send, upload) => result,
|
||||
result = send_h2_tracked(body, &mut send, upload, &errors) => result,
|
||||
};
|
||||
if let Err(error) = result {
|
||||
// RFC 9113 section 8.1 allows an early response followed by NO_ERROR.
|
||||
@@ -386,7 +393,7 @@ impl Carrier {
|
||||
}
|
||||
let response = timeouts
|
||||
.response(
|
||||
async { response.await.map_err(io::Error::other) },
|
||||
async { response.await.map_err(|error| errors.map(error)) },
|
||||
watch,
|
||||
&tasks.cancel,
|
||||
)
|
||||
@@ -400,9 +407,10 @@ impl Carrier {
|
||||
guard.disarm();
|
||||
let (parts, incoming) = response.into_parts();
|
||||
let carrier = self.clone();
|
||||
let errors = errors.clone();
|
||||
tasks.spawn(async move {
|
||||
let _carrier = carrier;
|
||||
receive_h2(incoming, writer).await;
|
||||
receive_h2_with_errors(incoming, writer, &errors).await;
|
||||
});
|
||||
Response::from_parts(parts, reader)
|
||||
}
|
||||
@@ -539,14 +547,22 @@ pub async fn receive_hyper(
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn receive_h2(mut body: h2::RecvStream, mut writer: Writer) {
|
||||
pub async fn receive_h2(body: h2::RecvStream, writer: Writer) {
|
||||
receive_h2_with_errors(body, writer, &TransportErrors::default()).await;
|
||||
}
|
||||
|
||||
async fn receive_h2_with_errors(
|
||||
mut body: h2::RecvStream,
|
||||
mut writer: Writer,
|
||||
errors: &TransportErrors,
|
||||
) {
|
||||
let result: io::Result<()> = async {
|
||||
while let Some(data) = body.data().await {
|
||||
let data = data.map_err(io::Error::other)?;
|
||||
let data = data.map_err(|error| errors.map(error))?;
|
||||
writer.write_all(&data).await?;
|
||||
body.flow_control()
|
||||
.release_capacity(data.len())
|
||||
.map_err(io::Error::other)?;
|
||||
.map_err(|error| errors.map(error))?;
|
||||
}
|
||||
writer.shutdown().await
|
||||
}
|
||||
@@ -558,7 +574,7 @@ pub async fn receive_h2(mut body: h2::RecvStream, mut writer: Writer) {
|
||||
}
|
||||
|
||||
pub async fn send_h2(body: Body, mut send: h2::SendStream<Bytes>) -> io::Result<()> {
|
||||
send_h2_tracked(body, &mut send, None)
|
||||
send_h2_tracked(body, &mut send, None, &TransportErrors::default())
|
||||
.await
|
||||
.map_err(H2SendError::into_io)
|
||||
}
|
||||
@@ -587,16 +603,11 @@ impl H2SendError {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<h2::Error> for H2SendError {
|
||||
fn from(error: h2::Error) -> Self {
|
||||
Self::Stream(io::Error::other(error))
|
||||
}
|
||||
}
|
||||
|
||||
async fn send_h2_tracked(
|
||||
mut body: Body,
|
||||
send: &mut h2::SendStream<Bytes>,
|
||||
mut upload: Option<Upload>,
|
||||
errors: &TransportErrors,
|
||||
) -> Result<(), H2SendError> {
|
||||
'frames: loop {
|
||||
let frame = tokio::select! {
|
||||
@@ -604,7 +615,8 @@ async fn send_h2_tracked(
|
||||
reset = futures::future::poll_fn(|cx| send.poll_reset(cx)) => return Err(H2SendError::Stream(io::Error::other(format!("XHTTP H2 stream reset: {reset:?}")))),
|
||||
};
|
||||
let Some(frame) = frame else {
|
||||
send.send_data(Bytes::new(), true)?;
|
||||
send.send_data(Bytes::new(), true)
|
||||
.map_err(|error| H2SendError::Stream(errors.map(error)))?;
|
||||
break;
|
||||
};
|
||||
let Ok(mut data) = frame.map_err(H2SendError::Local)?.into_data() else {
|
||||
@@ -614,14 +626,16 @@ async fn send_h2_tracked(
|
||||
send.reserve_capacity(data.len());
|
||||
let capacity = futures::future::poll_fn(|cx| send.poll_capacity(cx))
|
||||
.await
|
||||
.ok_or_else(|| H2SendError::Stream(io::ErrorKind::BrokenPipe.into()))??;
|
||||
.ok_or_else(|| H2SendError::Stream(io::ErrorKind::BrokenPipe.into()))?
|
||||
.map_err(|error| H2SendError::Stream(errors.map(error)))?;
|
||||
let len = capacity.min(data.len());
|
||||
if len == 0 {
|
||||
continue;
|
||||
}
|
||||
// Known-length peers may respond and reset immediately after the final bytes.
|
||||
let end = len == data.len() && hyper::body::Body::is_end_stream(&body);
|
||||
send.send_data(data.split_to(len), end)?;
|
||||
send.send_data(data.split_to(len), end)
|
||||
.map_err(|error| H2SendError::Stream(errors.map(error)))?;
|
||||
if let Some(upload) = &upload {
|
||||
upload.activity.touch();
|
||||
}
|
||||
@@ -642,6 +656,50 @@ mod tests {
|
||||
use crate::resources::Budget;
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
#[tokio::test]
|
||||
async fn h2_readiness_and_established_receive_preserve_transport_causes() {
|
||||
use crate::http2::transport::tests::{cases, peer};
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
for ready in [false, true] {
|
||||
for (make_error, local) in cases() {
|
||||
let (client, fault, mut sends, driver) = peer(make_error);
|
||||
if !ready {
|
||||
fault.inject();
|
||||
}
|
||||
let result = Carrier::tcp(Box::new(client), XhttpVersion::H2, 8192).await;
|
||||
let error = if ready {
|
||||
let carrier = result.unwrap();
|
||||
let tasks = Tasks::new();
|
||||
let memory = Memory::new(Arc::new(Budget::new(Some(16384))), Some(16384));
|
||||
let request = Request::get("http://example.com/")
|
||||
.body(Body::Empty)
|
||||
.unwrap();
|
||||
let mut response = carrier
|
||||
.request(request, memory, &tasks, Duration::from_secs(1))
|
||||
.await
|
||||
.unwrap()
|
||||
.into_body();
|
||||
let _send = sends.recv().await.unwrap();
|
||||
assert_eq!(response.read_u8().await.unwrap(), b'x');
|
||||
fault.inject();
|
||||
response.read_u8().await.unwrap_err()
|
||||
} else {
|
||||
result.err().expect("readiness PING must fail")
|
||||
};
|
||||
assert_eq!(
|
||||
crate::clash_api::probes::local_failure(&error),
|
||||
local,
|
||||
"ready={ready}: {error:?}"
|
||||
);
|
||||
driver.abort();
|
||||
let _ = driver.await;
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
async fn h2_peer() -> (
|
||||
Arc<Carrier>,
|
||||
tokio::sync::mpsc::UnboundedReceiver<(
|
||||
@@ -939,7 +997,7 @@ mod tests {
|
||||
async fn h2_no_error_does_not_hide_body_errors_or_finish_server_sends() {
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
let (carrier, mut requests, _server) = h2_peer().await;
|
||||
let Sender::H2(sender) = &carrier.sender else {
|
||||
let Sender::H2 { sender, .. } = &carrier.sender else {
|
||||
unreachable!()
|
||||
};
|
||||
let mut sender = sender.clone().ready().await.unwrap();
|
||||
@@ -967,7 +1025,9 @@ mod tests {
|
||||
let error = H2SendError::Stream(io::ErrorKind::BrokenPipe.into());
|
||||
assert!(error.peer_sent_no_error(&mut send));
|
||||
// A reset can arrive between the body EOF poll and sending the final DATA frame.
|
||||
let error = H2SendError::from(send.send_data(Bytes::new(), true).unwrap_err());
|
||||
let error = H2SendError::Stream(io::Error::other(
|
||||
send.send_data(Bytes::new(), true).unwrap_err(),
|
||||
));
|
||||
assert!(error.peer_sent_no_error(&mut send));
|
||||
let error = H2SendError::Local(io::ErrorKind::UnexpectedEof.into());
|
||||
assert!(!error.peer_sent_no_error(&mut send));
|
||||
|
||||
+262
-68
@@ -1593,78 +1593,272 @@ async fn h1_waiting_packet_ack_does_not_consume_another_sessions_connections() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn h1_packet_connection_close_redials_without_replaying_packets() {
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
let listener = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap();
|
||||
let address = ([127, 0, 0, 1], listener.local_addr().unwrap().port()).into();
|
||||
let received = Arc::new(parking_lot::Mutex::new(Vec::new()));
|
||||
let packets = received.clone();
|
||||
let tasks = Tasks::new();
|
||||
tasks.spawn(async move {
|
||||
let mut connections = tokio::task::JoinSet::new();
|
||||
loop {
|
||||
let (socket, _) = tokio::select! {
|
||||
accepted = listener.accept() => accepted.unwrap(),
|
||||
_ = connections.join_next(), if !connections.is_empty() => continue,
|
||||
async fn url_test_pooled_h2_sessions_record_carrier_demand() {
|
||||
use crate::{
|
||||
address::ResolvedLocation,
|
||||
async_stream::AsyncMessageStream,
|
||||
client_proxy_chain::{ClientProxyChain, InitialHopEntry},
|
||||
control::url_test::{AutomaticGroups, BACKGROUND},
|
||||
resolver::{NativeResolver, Resolver},
|
||||
tcp::{proxy_connector_impl::ProxyConnectorImpl, socket_connector::SocketConnector},
|
||||
};
|
||||
|
||||
#[derive(Debug)]
|
||||
struct ActivitySocket {
|
||||
address: SocketAddr,
|
||||
opens: AtomicUsize,
|
||||
groups: AutomaticGroups,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl SocketConnector for ActivitySocket {
|
||||
fn record_demand(&self) {
|
||||
self.groups.touch("carrier", false);
|
||||
}
|
||||
|
||||
async fn connect(
|
||||
&self,
|
||||
_: &Arc<dyn Resolver>,
|
||||
_: &ResolvedLocation,
|
||||
) -> io::Result<Box<dyn AsyncStream>> {
|
||||
self.record_demand();
|
||||
self.opens.fetch_add(1, Ordering::Relaxed);
|
||||
Ok(Box::new(
|
||||
tokio::net::TcpStream::connect(self.address).await?,
|
||||
))
|
||||
}
|
||||
|
||||
async fn connect_udp_bidirectional(
|
||||
&self,
|
||||
_: &Arc<dyn Resolver>,
|
||||
_: ResolvedLocation,
|
||||
) -> io::Result<Box<dyn AsyncMessageStream>> {
|
||||
unreachable!("XHTTP uses the TCP carrier")
|
||||
}
|
||||
|
||||
fn bind_interface(&self) -> Option<&str> {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
for udp in [false, true] {
|
||||
tokio::time::timeout(Duration::from_secs(60), async {
|
||||
let (address, _server) = echo_peer(XhttpVersion::H2, false).await;
|
||||
let resolver: Arc<dyn Resolver> = Arc::new(NativeResolver::new());
|
||||
let config = serde_json::from_value(serde_json::json!({
|
||||
"address": address.to_string(),
|
||||
"protocol": {
|
||||
"type": "xhttp", "http_version": "h2", "mode": "stream-one",
|
||||
"pool": {"idle_timeout_secs": 600},
|
||||
"protocol": {"type": "vless", "user_id": "550e8400-e29b-41d4-a716-446655440000"}
|
||||
}
|
||||
}))
|
||||
.unwrap();
|
||||
let proxy = ProxyConnectorImpl::from_config(config, resolver.clone())
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let mut groups = AutomaticGroups::default();
|
||||
groups
|
||||
.insert(
|
||||
"carrier".into(),
|
||||
crate::config::UrlTestConfig {
|
||||
url: "http://probe.invalid/".into(),
|
||||
interval: 10,
|
||||
timeout: 1000,
|
||||
tolerance: 50,
|
||||
lazy: true,
|
||||
},
|
||||
vec!["peer".into()],
|
||||
)
|
||||
.unwrap();
|
||||
let socket = Arc::new(ActivitySocket {
|
||||
address,
|
||||
opens: AtomicUsize::new(0),
|
||||
groups,
|
||||
});
|
||||
let chain = ClientProxyChain::new(
|
||||
vec![InitialHopEntry::Wireguard(socket.clone())],
|
||||
vec![vec![Box::new(proxy)]],
|
||||
);
|
||||
let target: ResolvedLocation =
|
||||
NetLocation::from_str("192.0.2.1:443", None).unwrap().into();
|
||||
let first = chain.connect_tcp(target.clone(), &resolver).await.unwrap();
|
||||
let group = &socket.groups.0["carrier"];
|
||||
assert_eq!(socket.opens.load(Ordering::Relaxed), 1);
|
||||
assert!(group.due_members(tokio::time::Instant::now()).is_some());
|
||||
tokio::time::pause();
|
||||
tokio::time::advance(Duration::from_secs(11)).await;
|
||||
assert!(group.due_members(tokio::time::Instant::now()).is_none());
|
||||
for background in [true, false] {
|
||||
let session = async {
|
||||
if udp {
|
||||
drop(
|
||||
chain
|
||||
.connect_udp_bidirectional(&resolver, target.clone())
|
||||
.await
|
||||
.unwrap(),
|
||||
);
|
||||
} else {
|
||||
drop(chain.connect_tcp(target.clone(), &resolver).await.unwrap());
|
||||
}
|
||||
};
|
||||
let packets = packets.clone();
|
||||
connections.spawn(async move {
|
||||
let service = hyper::service::service_fn(
|
||||
move |request: Request<hyper::body::Incoming>| {
|
||||
let packets = packets.clone();
|
||||
async move {
|
||||
if request.method() == http::Method::GET {
|
||||
let memory =
|
||||
Memory::new(Arc::new(Budget::new(Some(1024))), Some(1024));
|
||||
let (writer, reader) = pipe(memory);
|
||||
return Ok::<_, io::Error>(Response::new(
|
||||
Body::stream(reader).with_guard(writer),
|
||||
));
|
||||
if background {
|
||||
BACKGROUND.scope((), session).await;
|
||||
} else {
|
||||
session.await;
|
||||
}
|
||||
assert_eq!(
|
||||
socket.opens.load(Ordering::Relaxed),
|
||||
1,
|
||||
"carrier was redialed"
|
||||
);
|
||||
assert_eq!(
|
||||
group.due_members(tokio::time::Instant::now()).is_some(),
|
||||
!background,
|
||||
"udp={udp}, background={background}",
|
||||
);
|
||||
}
|
||||
tokio::time::resume();
|
||||
drop(first);
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn url_test_h1_packet_redials_preserve_activity_context_without_replay() {
|
||||
#[derive(Debug)]
|
||||
struct ActivityDialer {
|
||||
inner: Dialer,
|
||||
groups: crate::control::url_test::AutomaticGroups,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl CarrierDialer for ActivityDialer {
|
||||
fn route_key(&self) -> &RouteKey {
|
||||
self.inner.route_key()
|
||||
}
|
||||
|
||||
async fn open_stream(&self) -> io::Result<Box<dyn AsyncStream>> {
|
||||
self.groups.touch("carrier", false);
|
||||
self.inner.open_stream().await
|
||||
}
|
||||
}
|
||||
|
||||
for background in [false, true] {
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
let listener = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap();
|
||||
let address = ([127, 0, 0, 1], listener.local_addr().unwrap().port()).into();
|
||||
let received = Arc::new(parking_lot::Mutex::new(Vec::new()));
|
||||
let packets = received.clone();
|
||||
let tasks = Tasks::new();
|
||||
tasks.spawn(async move {
|
||||
let mut connections = tokio::task::JoinSet::new();
|
||||
loop {
|
||||
let (socket, _) = tokio::select! {
|
||||
accepted = listener.accept() => accepted.unwrap(),
|
||||
_ = connections.join_next(), if !connections.is_empty() => continue,
|
||||
};
|
||||
let packets = packets.clone();
|
||||
connections.spawn(async move {
|
||||
let service = hyper::service::service_fn(
|
||||
move |request: Request<hyper::body::Incoming>| {
|
||||
let packets = packets.clone();
|
||||
async move {
|
||||
if request.method() == http::Method::GET {
|
||||
let memory = Memory::new(
|
||||
Arc::new(Budget::new(Some(1024))),
|
||||
Some(1024),
|
||||
);
|
||||
let (writer, reader) = pipe(memory);
|
||||
return Ok::<_, io::Error>(Response::new(
|
||||
Body::stream(reader).with_guard(writer),
|
||||
));
|
||||
}
|
||||
let path = request.uri().path().to_owned();
|
||||
let data =
|
||||
request.into_body().collect().await.unwrap().to_bytes();
|
||||
packets.lock().push((path, data));
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
Ok(Response::builder()
|
||||
.header("connection", "close")
|
||||
.body(Body::Empty)
|
||||
.unwrap())
|
||||
}
|
||||
let path = request.uri().path().to_owned();
|
||||
let data = request.into_body().collect().await.unwrap().to_bytes();
|
||||
packets.lock().push((path, data));
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
Ok(Response::builder()
|
||||
.header("connection", "close")
|
||||
.body(Body::Empty)
|
||||
.unwrap())
|
||||
}
|
||||
},
|
||||
);
|
||||
let _ = hyper::server::conn::http1::Builder::new()
|
||||
.serve_connection(hyper_util::rt::TokioIo::new(socket), service)
|
||||
.await;
|
||||
});
|
||||
},
|
||||
);
|
||||
let _ = hyper::server::conn::http1::Builder::new()
|
||||
.serve_connection(hyper_util::rt::TokioIo::new(socket), service)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
});
|
||||
let config =
|
||||
serde_json::from_value(serde_json::json!({"protocol":{"type":"socks"}})).unwrap();
|
||||
let client = configured_client(config, XhttpVersion::H1, XhttpMode::PacketUp);
|
||||
let mut groups = crate::control::url_test::AutomaticGroups::default();
|
||||
groups
|
||||
.insert(
|
||||
"carrier".into(),
|
||||
crate::config::UrlTestConfig {
|
||||
url: "http://probe.invalid/".into(),
|
||||
interval: 10,
|
||||
timeout: 1000,
|
||||
tolerance: 50,
|
||||
lazy: true,
|
||||
},
|
||||
vec!["peer".into()],
|
||||
)
|
||||
.unwrap();
|
||||
assert!(
|
||||
groups.0["carrier"]
|
||||
.due_members(tokio::time::Instant::now() - Duration::from_secs(11))
|
||||
.is_some()
|
||||
);
|
||||
let dialer = Arc::new(ActivityDialer {
|
||||
groups,
|
||||
inner: Dialer {
|
||||
address,
|
||||
key: RouteKey(vec![1]),
|
||||
opens: AtomicUsize::new(0),
|
||||
},
|
||||
});
|
||||
let session = async {
|
||||
let mut stream = client.open(dialer.clone()).await.unwrap();
|
||||
for byte in 0..3 {
|
||||
stream.write_all(&[byte]).await.unwrap();
|
||||
stream.flush().await.unwrap();
|
||||
while received.lock().len() <= byte as usize {
|
||||
tokio::task::yield_now().await;
|
||||
}
|
||||
}
|
||||
stream.shutdown().await.unwrap();
|
||||
};
|
||||
if background {
|
||||
crate::control::url_test::BACKGROUND
|
||||
.scope((), session)
|
||||
.await;
|
||||
} else {
|
||||
session.await;
|
||||
}
|
||||
});
|
||||
let config =
|
||||
serde_json::from_value(serde_json::json!({"protocol":{"type":"socks"}})).unwrap();
|
||||
let client = configured_client(config, XhttpVersion::H1, XhttpMode::PacketUp);
|
||||
let dialer = Arc::new(Dialer {
|
||||
address,
|
||||
key: RouteKey(vec![1]),
|
||||
opens: AtomicUsize::new(0),
|
||||
});
|
||||
let mut stream = client.open(dialer.clone()).await.unwrap();
|
||||
for byte in 0..3 {
|
||||
stream.write_all(&[byte]).await.unwrap();
|
||||
stream.flush().await.unwrap();
|
||||
while received.lock().len() <= byte as usize {
|
||||
tokio::task::yield_now().await;
|
||||
assert_eq!(dialer.inner.opens.load(Ordering::Relaxed), 4);
|
||||
assert_eq!(
|
||||
dialer.groups.0["carrier"]
|
||||
.due_members(tokio::time::Instant::now())
|
||||
.is_some(),
|
||||
!background
|
||||
);
|
||||
let received = received.lock();
|
||||
assert_eq!(received.len(), 3);
|
||||
for (sequence, (path, bytes)) in received.iter().enumerate() {
|
||||
assert!(path.ends_with(&format!("/{sequence}")));
|
||||
assert_eq!(bytes.as_ref(), &[sequence as u8]);
|
||||
}
|
||||
}
|
||||
stream.shutdown().await.unwrap();
|
||||
assert_eq!(dialer.opens.load(Ordering::Relaxed), 4);
|
||||
let received = received.lock();
|
||||
assert_eq!(received.len(), 3);
|
||||
for (sequence, (path, bytes)) in received.iter().enumerate() {
|
||||
assert!(path.ends_with(&format!("/{sequence}")));
|
||||
assert_eq!(bytes.as_ref(), &[sequence as u8]);
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
+8
-2
@@ -531,9 +531,15 @@ impl Tasks {
|
||||
}
|
||||
pub fn spawn(&self, future: impl Future<Output = ()> + Send + 'static) {
|
||||
let cancel = self.cancel.clone();
|
||||
self.tracker.spawn(async move {
|
||||
let task = async move {
|
||||
tokio::select! { biased; _ = cancel.cancelled() => {}, _ = future => {} }
|
||||
});
|
||||
};
|
||||
if crate::control::url_test::is_background() {
|
||||
self.tracker
|
||||
.spawn(crate::control::url_test::BACKGROUND.scope((), task));
|
||||
} else {
|
||||
self.tracker.spawn(task);
|
||||
}
|
||||
}
|
||||
pub async fn wait(&self) {
|
||||
self.tracker.close();
|
||||
|
||||
@@ -177,7 +177,7 @@ async fn clash_api_pinned_mihomo_contract() {
|
||||
&shoes_config,
|
||||
serde_yaml::to_string(&json!([
|
||||
{"clash_api_address":format!("0.0.0.0:{}", shoes.port()), "secret":SECRET},
|
||||
{"client_group":"outbound", "strategy":"selector", "client_proxies":["direct"]}
|
||||
{"client_group":"outbound", "type":"selector", "client_proxies":["direct"]}
|
||||
]))
|
||||
.unwrap(),
|
||||
)
|
||||
|
||||
@@ -290,7 +290,7 @@ impl Fixture {
|
||||
{"clash_api_address":format!("0.0.0.0:{}", api.port()), "secret":SECRET, "cors_origins":["https://dashboard.example"]},
|
||||
{"clash_api_path":directory.path().join("api.sock"), "secret":SECRET},
|
||||
{"cache_file_path":directory.path().join("cache.db")},
|
||||
{"client_group":"outbound", "strategy":"selector", "default":"east", "client_proxies":[
|
||||
{"client_group":"outbound", "type":"selector", "default":"east", "client_proxies":[
|
||||
{"name":"east", "address":east.to_string(), "protocol":{"type":"socks"}},
|
||||
{"name":"west", "address":west.to_string(), "protocol":{"type":"socks"}}
|
||||
]},
|
||||
@@ -338,6 +338,51 @@ impl Fixture {
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn url_test_routes_without_controller_and_can_be_removed_on_reload() {
|
||||
let mut fixture = Fixture::new().await;
|
||||
fixture
|
||||
.config
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.retain(|entry| entry.get("client_group").is_some() || entry.get("rules").is_some());
|
||||
let group = fixture.outbound_mut();
|
||||
group.as_object_mut().unwrap().remove("default");
|
||||
group["type"] = json!("url-test");
|
||||
group["url"] = json!("http://probe.invalid/");
|
||||
group["interval"] = json!(1);
|
||||
group["timeout"] = json!(500);
|
||||
group["client_proxies"][0]["address"] = json!(available_address().to_string());
|
||||
let mut process = fixture.start().await;
|
||||
timeout(Duration::from_secs(5), async {
|
||||
loop {
|
||||
let result = shoes_test_support::wireguard::connect(
|
||||
fixture.entry.port(),
|
||||
"192.0.2.100:80".parse().unwrap(),
|
||||
)
|
||||
.await;
|
||||
if let Ok(mut stream) = result {
|
||||
check_flow(&mut stream, b'W').await;
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!fixture.directory.path().join("cache.db").exists());
|
||||
let west = fixture.outbound_mut()["client_proxies"][1].clone();
|
||||
*fixture.outbound_mut() = json!({
|
||||
"client_group":"outbound", "type":"round-robin", "client_proxies":[west]
|
||||
});
|
||||
fixture.write();
|
||||
process.signal(libc::SIGHUP);
|
||||
process.wait_for("Servers ready").await;
|
||||
let _flow = flow(fixture.entry, b'W').await;
|
||||
fixture.stop(&mut process).await;
|
||||
std::net::TcpListener::bind(fixture.entry).unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn selectors_change_real_traffic_and_restore_after_restart() {
|
||||
let mut fixture = Fixture::new().await;
|
||||
|
||||
Reference in New Issue
Block a user