feat: add client protocols, TLS capabilities, and platform validation (#1)

* chore: download latest official mihomo for interop tests

* test: capture N0 baseline and protocol feasibility probes

* docs: record N0 hybrid REALITY fork verification

* test: verify N0 controlled Hysteria2 auth and streams

Add an isolated Quinn adapter over existing DIRECT and SOCKS5 datagrams, bounded queues, protect rejection probes, and official latest peer fixtures. Record passing auth/TCP checks and the native Hysteria half-close failure separately. N0 remains incomplete; production sources and dependency locks are unchanged.

* test: validate N0 native QUIC hopping and XHTTP H3

Keep the probes isolated from production. Verify one-state Hysteria hopping, protected rebind rejection, Xray H3 entry and existing HY2 regressions. Set only the owned Linux peer MTU to avoid Initial-packet EMSGSIZE; retain native half-close failures as separate gates.

* fix: align XHTTP shutdown with Mihomo connection close

* test: verify N0 supplied-stream transport interfaces

Add isolated TLS, ordinary WebSocket and gRPC probes with owned I/O, bounded buffers and cancellation barriers.

Match Mihomo close behavior: preserve TLS/WS reads, distinguish clean WS underlay EOF from truncation, and close both gRPC directions. Use separate official SOCKS5 peers to avoid fixture loop detection.

Record 12 Debug/Release tests, 42 official Mihomo checks, dependency audits and Apple/Android compile checks. Keep production dependencies, third-party sources and remaining N0 gates unchanged.

* fix: upgrade h2 and retain completed responses after reset

* chore: use the current OneXray rustls fork origin

* chore: adopt published rustls 0.23.45 and official TLS binding

* chore: refresh N1 foundation and spike dependencies

* feat: validate next-protocol field and combination catalogs

* feat: share immutable TLS policies and bounded close semantics

* feat: add shared stream transports and decouple XUDP framing

* feat: add controlled datagram foundations and N1 acceptance harness

* chore: finalize N1 with verified x25519 dependency

* chore: publish N1 against the rustls release branch

* feat: add bounded Trojan request and datagram codecs

* feat: connect Trojan TLS TCP and UDP through the runtime

* feat: support Trojan WebSocket and duplex gRPC transports

* feat: enable Trojan and complete N2 consumer acceptance

* feat(vmess): add bounded AEAD header and body codecs

* fix(vmess): align close acceptance with Mihomo transports

* feat(vmess): add UDP codecs and isolated peer comparisons

* feat(vmess): complete public runtime and N3 acceptance

* docs: bind N3 acceptance and refresh validation guidance

* feat(vless): complete N4 transports, Vision and UDP support

Extract VLESS assembly, add plain/TLS stream transports, HTTPUpgrade, node-owned gRPC pooling, mTLS and classic REALITY, and implement Vision through public TLS record boundaries. Preserve XHTTP and the XUDP default while adding raw and packetaddr.

Complete 145 required groups and 39 field rows against isolated latest official peers, with 160 lifecycle/resource rounds, independent coverage and Apple/Android release builds. Preserve the three explicit WS/REALITY layered close comparisons and historical failures in the acceptance report. No third-party or dependency-lock changes.

* feat(vless): complete N5 XHTTP and sing-mux support

Add H1/H2/H3 request fields, independent download security and pools, bounded packet-up batching, and h2mux/smux/yamux over controlled transports.

Keep Mihomo-aligned close and path semantics, node-owned shutdown barriers, and isolated official-peer fixtures including the approved Caddy mTLS gateway.

Validate 416 required groups, 57 fields, 112 two-leg security checks, 240 lifecycle/resource cycles, independent coverage, and Apple/Android release builds. Record native-peer gaps and leave N6-N10, devices, CI, and release gates separate.

* feat: integrate boring TLS profiles and retire rustls fork

* test(tls): freeze selected fingerprint reference baseline

* docs(tls): record selected-profile library acceptance

* feat(tls): integrate selected client fingerprints and REALITY profiles

* feat(tls): complete selected fingerprint CF5 acceptance

* feat(hysteria2): complete N6 client support and acceptance

Implement Hysteria2 over official Quinn/h3/rustls and shared controlled IO, including TCP/UDP, bandwidth pacing, Salamander, port hopping, upstream snapshots and synchronous Stop.

Keep old-path receives active for the full bounded handover window, preserve independent reader wakeups, and require the exact nested TCP evidence assertions.

Validate one frozen N6 run: 36 required groups, 20 field rows, native Mihomo/Hysteria peers, H3 regression, 40 lifecycle rounds and Apple/Android production builds. Record independent coverage, artifact hashes and native-peer limitations.

* test(tls): record N7 hybrid REALITY prerequisite evidence

Preserve the locked classic-only API failure and distinguish the locally accepted boring hybrid extension from pending VCore S03/D16 integration. Record wrapper feasibility and publication boundaries without changing production configuration or dependencies.

* feat(vless): support explicitly required hybrid REALITY

* feat(crypto): wrap official BLAKE3 binary-context derivation

* feat(vless): implement authenticated Encryption wire transport

* feat(vless): enable authenticated Encryption across public transports

* build(tls): pin the authenticated ShadowTLS ClientHello hook

* feat(vless): add authenticated JLS transport security

Pin the separately approved boring JLS revision and expose strict main/download credentials under schema 24. Preserve native CertificateVerify and Finished, cancellation, and controlled IO ownership. Accept the JLS field slice with 122 isolated cases, 80 lifecycle/resource cycles, and Apple/Android release libraries; keep broader N7 gates separate.

* docs(n7): reconcile current JLS acceptance progress

* chore(vless): retire Restls while retaining JLS

* feat(vless): complete N7 static ECH and security combinations

Add static ECH for VLESS main and XHTTP download legs using existing TLS backends and official Rust HPKE. Keep Invoke v5 and advance the internal schema to revision 27.

Accept the narrowed N7 scope with 43 required groups, 11 fields, 301 isolated native cases, 240 lifecycle/resource rounds, and Apple/Android release builds. Record frozen inputs, 470 container cleanups, explicit upstream comparison gaps, and prior failed runs in docs/acceptance/next-protocols/N7.md.

* chore(protocols): retire WireGuard scope and foundations

* test: complete N9 protocol integration and resource gates

* build: add N10 production artifact evidence gates

* fix: restore Windows snapshot encoding and fail-closed delivery CI

* fix: use native ARM64 LLVM assembly and scoped delivery CI

* build: keep delivery lockfile bytes stable across platforms

* fix: preserve production guard lifecycle and isolate CI feature checks

* build: adopt released boring and registry shadowsocks

* ci: use release-oriented workflow names

* docs: consolidate contracts and streamline test tooling

Merge current protocol and platform contracts, remove milestone reports and obsolete spikes, and share core checks between local development and CI. Preserve all 884 executable case definitions and unique legacy assertions.

Validation: core Debug/Release, feature checks, Clippy, netstack, offline script tests, formatting and documentation links passed locally. Container interop smoke attempts failed during official peer downloads; business paths remain NOT RUN.

* test: replace milestone names with capability-based validation

Rename protocol and fingerprint suites, fixtures, and evidence identifiers by capability. Preserve acceptance cases and golden wire data while documenting reference, wire, and interoperability evidence separately.

* fix: address review findings and stabilize core CI

Flush accepted gRPC/H2 data through the supplied IO before reset, and wait for pending smux writes before FIN. Sanitize AnyTLS peer failures, reject XHTTP userinfo, and remove stale Android delivery artifacts.

Close container log capture even when inventory fails. Keep legacy host suites blocked before resource acquisition.

Separate Shadowsocks adapter codec checks from upstream strict server acceptance. Lock the random empty-request zero-padding failure with deterministic negative and positive fixtures, without changing the dependency or production behavior.

* test: isolate delivery fixture from Windows home lookup

* test: preserve UDP socket semantics during peer teardown

The memory peer treated its counterpart closing an mpsc channel as a local UDP BrokenPipe. That could terminate Quinn endpoint IO without waking a pending wait_idle, despite all four datagrams and CONNECTION_CLOSE having arrived.

Keep a closed peer link silent and late sends best-effort, add a deterministic fixture regression, and preserve the existing data and shutdown deadlines. Debug/Release, Clippy, and 300 consecutive regression iterations pass.

* fix: drain XHTTP uploads before releasing connection drivers

* fix: enforce lazy setup deadlines and safe delivery paths

Check the original deadline before accepting ready gRPC/H2 headers or sing-mux status, without limiting established reads. Reject symlinks and Windows reparse points in every delivery output path before filesystem mutations. Add deterministic memory and temporary-filesystem regressions.

* test: join XHTTP H3 peer tasks during cleanup

Quinn retains closed connections for three probe timeouts, so a delayed handshake can legitimately exceed the fixture's two-second wait_idle budget. Own, abort, and join the passive peer tasks after upload and client shutdown instead. Preserve upload integrity and client close bounds, and cover delayed handshakes in both streaming modes.
This commit is contained in:
yiguodev
2026-09-28 12:50:01 +08:00
committed by GitHub
parent 3872789f85
commit 608a8f255c
361 changed files with 79800 additions and 3888 deletions
+47 -2
View File
@@ -13,11 +13,12 @@ from pathlib import Path
CORE_DIR = Path(__file__).resolve().parents[3]
EXPECTED_IDENTITY = (
b"VCore;engine=rust;coreVersion=0.1.0;invokeApiVersion=5;configVersion=14"
b"VCore;engine=rust;coreVersion=0.1.0;invokeApiVersion=5;configVersion=27"
)
DEFAULT_FEATURES = (
"ffi,tun,inbound-http,inbound-socks5,outbound-anytls,"
"outbound-socks5,outbound-shadowsocks,outbound-vless"
"outbound-socks5,outbound-shadowsocks,outbound-trojan,outbound-vmess,outbound-vless,"
"outbound-hysteria2"
)
@@ -196,17 +197,34 @@ def build_android() -> None:
for target in targets:
abi, clang, cargo_name = _android_target(target, android_api)
linker = toolchain / "bin" / clang
cpp = toolchain / "bin" / (clang + "++")
archive = toolchain / "bin" / "llvm-ar"
if not linker.is_file():
raise RuntimeError(f"Android linker not found: {linker}")
if not archive.is_file():
raise RuntimeError(f"Android archiver not found: {archive}")
if not cpp.is_file():
raise RuntimeError(f"Android C++ compiler not found: {cpp}")
runtime_target = (
"arm-linux-androideabi" if target == "armv7-linux-androideabi" else target
)
cpp_runtime = (
toolchain / "sysroot/usr/lib" / runtime_target / "libc++_shared.so"
)
if not cpp_runtime.is_file():
raise RuntimeError(f"Android C++ runtime not found: {cpp_runtime}")
target_env = target.replace("-", "_")
env = base_env | {
f"CC_{target_env}": str(linker),
f"CXX_{target_env}": str(cpp),
f"AR_{target_env}": str(archive),
f"CARGO_TARGET_{cargo_name}_LINKER": str(linker),
f"CARGO_TARGET_{cargo_name}_AR": str(archive),
f"CMAKE_TOOLCHAIN_FILE_{target_env}": str(
CORE_DIR / "scripts/cmake/android.toolchain.cmake"
),
"VCORE_CMAKE_ANDROID_ABI": abi,
"VCORE_CMAKE_ANDROID_API": android_api,
}
_cargo_build(target, profile_flags, features, env)
artifact = CORE_DIR / "target" / target / profile_name / "libvcore.so"
@@ -214,6 +232,9 @@ def build_android() -> None:
destination = output / abi / "libvcore.so"
destination.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(artifact, destination)
# BoringSSL links the NDK shared C++ runtime. It is not supplied by
# Android itself; distribute the matching ABI/runtime alongside VCore.
shutil.copy2(cpp_runtime, destination.parent / cpp_runtime.name)
print(output)
@@ -368,6 +389,30 @@ def _windows_msvc_environment(architecture: str) -> dict[str, str]:
key, separator, value = line.partition("=")
if separator and key:
env[key] = value
if architecture == "arm64":
search_path = next(value for key, value in env.items() if key.upper() == "PATH")
compilers = {
name: shutil.which(name, path=search_path)
for name in ("clang-cl", "clang", "ninja")
}
if not all(compilers.values()):
raise RuntimeError(
"native Windows ARM64 requires LLVM clang-cl/clang and Ninja"
)
# Visual Studio's generator does not assemble BoringSSL's preprocessed
# .S inputs. Keep assembly enabled using LLVM + Ninja, still MSVC ABI,
# the selected Windows SDK, and the same static CRT. No source patch.
env.update(
{
"CC_aarch64_pc_windows_msvc": compilers["clang-cl"],
"CXX_aarch64_pc_windows_msvc": compilers["clang-cl"],
"VCORE_WINDOWS_ARM64_CLANG": compilers["clang"],
"CMAKE_GENERATOR_aarch64_pc_windows_msvc": "Ninja",
"CMAKE_TOOLCHAIN_FILE_aarch64_pc_windows_msvc": str(
CORE_DIR / "scripts/cmake/windows-arm64.toolchain.cmake"
),
}
)
return env
+128
View File
@@ -0,0 +1,128 @@
"""Explicit xcaddy-only exception for the isolated H3/mTLS test gateway."""
from __future__ import annotations
import os
import re
import shutil
import urllib.parse
import urllib.request
from pathlib import Path
from .native_release import PeerArtifact
from .protocol_inputs import redact, sha256
from .protocol_peers import run_command
def latest_tag(project):
"""Official latest redirect, not the GitHub API or a pinned old tag."""
if project not in ("caddy", "xcaddy"):
raise ValueError("unsupported gateway project")
url = f"https://github.com/caddyserver/{project}/releases/latest"
request = urllib.request.Request(
url, method="HEAD", headers={"User-Agent": "VCore-interop-scripts"}
)
with urllib.request.urlopen(request, timeout=30) as response:
final = urllib.parse.urlsplit(response.geturl())
prefix = f"/caddyserver/{project}/releases/tag/"
tag = final.path.removeprefix(prefix)
if (
final.scheme != "https"
or final.netloc != "github.com"
or final.query
or final.fragment
or not final.path.startswith(prefix)
or not re.fullmatch(r"v[0-9]+\.[0-9]+\.[0-9]+", tag)
):
raise RuntimeError("cannot resolve official stable gateway release")
return tag
def build_caddy(directory: Path):
directory.mkdir(parents=True, exist_ok=False)
xcaddy, go = shutil.which("xcaddy"), shutil.which("go")
if not xcaddy or not go:
raise RuntimeError("xcaddy and Go are required for the approved gateway build")
versions = {project: latest_tag(project) for project in ("xcaddy", "caddy")}
tool = run_command([xcaddy, "version"], timeout=10, limit=4096)
if (
tool.returncode
or not tool.cleanup
or tool.stdout.decode().split()[0] != versions["xcaddy"]
):
raise RuntimeError(
"installed xcaddy must match the official latest stable release"
)
compiler = run_command([go, "version"], timeout=10, limit=4096)
if compiler.returncode or not compiler.cleanup:
raise RuntimeError("cannot read Go build identity")
temporary = directory / "build-work"
temporary.mkdir()
binary = directory / "caddy"
environment = os.environ.copy()
# A fresh standard-module build, never a local source replacement or plugin.
for key in tuple(environment):
if key.startswith("XCADDY_") or key in ("CADDY_VERSION", "GOFLAGS", "GOWORK"):
environment.pop(key)
environment.update(
GOOS="linux",
GOARCH="arm64",
CGO_ENABLED="0",
GOWORK="off",
XCADDY_SKIP_CLEANUP="1",
XCADDY_PRINT_VERSION="0",
TMPDIR=str(temporary),
)
print("Building official latest Caddy with xcaddy for linux/arm64", flush=True)
built = run_command(
[xcaddy, "build", "latest", "--output", str(binary)],
env=environment,
cwd=directory,
timeout=600,
limit=4 * 1024 * 1024,
)
(directory / "build.log").write_text(redact(built.stdout.decode(errors="replace")))
if built.returncode or not built.cleanup or not binary.is_file():
raise RuntimeError("xcaddy gateway build failed")
metadata = run_command([go, "version", "-m", str(binary)], timeout=10, limit=65536)
info = metadata.stdout.decode(errors="replace")
expected = f"\tdep\tgithub.com/caddyserver/caddy/v2\t{versions['caddy']}\t"
if (
metadata.returncode
or not metadata.cleanup
or expected not in info
or "=>" in info
):
raise RuntimeError("gateway build is not the unmodified official latest Caddy")
# xcaddy deliberately uses cwd on macOS, TMPDIR on other hosts.
manifests = [
*directory.glob("buildenv_*/go.mod"),
*temporary.glob("buildenv_*/go.mod"),
]
if len(manifests) != 1:
raise RuntimeError("xcaddy build manifest is missing or ambiguous")
inputs = {}
for name in ("go.mod", "go.sum", "main.go"):
source = manifests[0].with_name(name)
shutil.copyfile(source, directory / name)
inputs[name] = sha256(source)
# Omit the host executable path from the retained module metadata.
(directory / "modules.txt").write_text("\n".join(info.splitlines()[1:]) + "\n")
return PeerArtifact(
binary,
dict(
kind="Caddy-xcaddy",
target="linux-arm64",
source_built=True,
source_url="https://github.com/caddyserver/caddy/releases/latest",
release=versions["caddy"],
xcaddy_version=tool.stdout.decode().strip(),
go_version=compiler.stdout.decode().strip(),
binary_sha256=sha256(binary),
build_seconds=built.seconds,
build_cleanup=built.cleanup,
inputs=inputs,
extra_plugins=[],
version=None,
),
)
+78 -21
View File
@@ -13,13 +13,9 @@ CRATES_IO_SOURCES = {
"registry+https://github.com/rust-lang/crates.io-index",
"registry+https://index.crates.io/",
}
RUSTLS_GIT_SOURCE_PREFIX = (
"git+https://github.com/OneVCore/rustls?branch=vcore/reality-0.23#"
)
SHADOWSOCKS_REVISION = "ab388c7466d21f979430e33cc9ef10e22fb05955"
SHADOWSOCKS_GIT_SOURCE = (
"git+https://github.com/shadowsocks/shadowsocks-rust.git"
f"?rev={SHADOWSOCKS_REVISION}#{SHADOWSOCKS_REVISION}"
BORING_REVISION = "d5a5d41850886aef5b269015130ee6d6eb2129ba"
BORING_GIT_SOURCE = (
f"git+https://github.com/OneXray/boring?branch=release#{BORING_REVISION}"
)
_C_SOURCE = r"""#include "vcore.h"
@@ -100,7 +96,7 @@ def _shadowsocks_aws_lc_errors(metadata: dict[str, Any]) -> list[str]:
errors: list[str] = []
allowed: dict[str, dict[str, Any]] = {}
expected = {
"shadowsocks": ("1.25.0", {SHADOWSOCKS_GIT_SOURCE}),
"shadowsocks": ("1.25.0", CRATES_IO_SOURCES),
"shadowsocks-crypto": ("0.8.0", CRATES_IO_SOURCES),
"aws-lc-rs": (None, CRATES_IO_SOURCES),
"aws-lc-sys": (None, CRATES_IO_SOURCES),
@@ -178,8 +174,8 @@ def _tls_dependency_errors(metadata: dict[str, Any]) -> list[str]:
return None
return matches[0]
rustls = require_single("rustls", "0.23.43")
tokio_rustls = require_single("tokio-rustls", "0.26.4")
rustls = require_single("rustls", "0.23.45")
tokio_rustls = require_single("tokio-rustls", "0.26.5")
ring = named("ring")
if len(ring) != 1:
@@ -195,12 +191,9 @@ def _tls_dependency_errors(metadata: dict[str, Any]) -> list[str]:
if rustls is not None:
rustls_source = rustls.get("source") or ""
revision = rustls_source.removeprefix(RUSTLS_GIT_SOURCE_PREFIX)
if len(revision) != 40 or any(
character not in "0123456789abcdef" for character in revision
):
if rustls_source not in CRATES_IO_SOURCES:
errors.append(
"rustls must come from the vcore/reality-0.23 GitHub branch; "
"rustls must come from crates.io; "
f"resolved source: {rustls_source or 'path'}"
)
@@ -226,8 +219,8 @@ def _tls_dependency_errors(metadata: dict[str, Any]) -> list[str]:
errors.append("rustls is missing from the resolved dependency graph")
else:
features = set(rustls_node["features"])
missing = {"reality", "ring"} - features
forbidden = {"aws_lc_rs", "fips"} & features
missing = {"ring"} - features
forbidden = {"reality", "aws_lc_rs", "fips"} & features
if missing:
errors.append(
f"rustls is missing required features: {', '.join(sorted(missing))}"
@@ -238,6 +231,63 @@ def _tls_dependency_errors(metadata: dict[str, Any]) -> list[str]:
f"{', '.join(sorted(forbidden))}"
)
native = {}
node_by_id = {node["id"]: node for node in nodes}
hpke = require_single("hpke", "0.14.1")
if hpke is not None:
if hpke.get("source") not in CRATES_IO_SOURCES:
errors.append("hpke must be the unmodified crates.io release")
hpke_node = node_by_id.get(hpke["id"])
if hpke_node is None or set(hpke_node.get("features", [])) != {
"alloc",
"aes",
"chacha",
"x25519",
"hkdfsha2",
}:
errors.append("hpke must use only the approved ECH algorithms")
for name, required in (
("boring", {"reality", "client-fingerprint", "shadow-tls-v3", "jls"}),
("boring-sys", {"reality", "shadow-tls-v3", "jls"}),
("tokio-boring", set()),
):
package = require_single(name, "5.2.0")
if package is None:
continue
native[name] = package
if package.get("source") != BORING_GIT_SOURCE:
errors.append(
f"{name} must use the approved boring release branch "
"and locked revision"
)
node = node_by_id.get(package["id"])
if node is None:
errors.append(f"{name} is missing from the resolved graph")
continue
features = set(node["features"])
if not required <= features:
errors.append(f"{name} is missing required TLS features")
if features & {
"fips",
"fips-precompiled",
"rpk",
"pq-experimental",
"restls",
}:
errors.append(f"{name} enables an unapproved native TLS mode")
for parent, children in (
("boring", {"boring-sys"}),
("tokio-boring", {"boring", "boring-sys"}),
):
if parent in native:
deps = {
dep["pkg"]
for dep in node_by_id.get(native[parent]["id"], {}).get("deps", [])
}
for child in children:
if child not in native or native[child]["id"] not in deps:
errors.append(f"{parent} must directly use the locked {child}")
errors.extend(_shadowsocks_aws_lc_errors(metadata))
return errors
@@ -273,12 +323,19 @@ def check_tls_dependencies() -> None:
rustls = next(
package for package in metadata["packages"] if package["name"] == "rustls"
)
revision = rustls["source"].rsplit("#", 1)[1]
print("TLS dependency check passed:")
print(f"- one OneVCore rustls 0.23.43 vcore/reality-0.23 @ {revision[:12]}")
print("- one official tokio-rustls 0.26.4")
print(f"- one official crates.io rustls {rustls['version']}")
print("- one official tokio-rustls 0.26.5")
print("- one official hpke 0.14.1 with the approved static ECH algorithms")
print(
"- one boring/boring-sys/tokio-boring 5.2.0 release branch "
f"@ {BORING_REVISION[:12]}"
)
print(f"- one registry ring {ring['version']} provider")
print("- no Watfaq or second rustls version; TLS uses ring only")
print(
"- unprofiled TLS/QUIC use rustls + ring; REALITY/named profiles use BoringSSL"
)
print("- no Watfaq, second rustls source or rustls REALITY backend")
if any(p["name"] == "aws-lc-rs" for p in metadata["packages"]):
print("- AWS-LC is restricted to the pinned official Shadowsocks 2022 chain")
else:
+160 -34
View File
@@ -8,7 +8,10 @@ from pathlib import Path
from .builds import build_android, build_apple, build_windows
from .checks import check_c_header, check_tls_dependencies
from .mihomo import run_mihomo_interop
from .mihomo_release import SUPPORTED_TARGETS, download_mihomo
from .protocol_catalogs import CATALOG_DIR, SUITES, check_protocol_catalogs
from .protocol_evidence import check_run
from .protocol_harness import run_protocol_interop
from .tun2socks import run_demo
@@ -21,41 +24,124 @@ def _parser() -> argparse.ArgumentParser:
build = commands.add_parser("build", help="build platform artifacts")
platforms = build.add_subparsers(dest="platform", required=True)
platforms.add_parser("apple", help="build LibVCore.xcframework on macOS")
platforms.add_parser("android", help="build Android libvcore.so artifacts")
platforms.add_parser("windows", help="build packaged Windows artifacts")
for name, description in (
("apple", "build LibVCore.xcframework on macOS"),
("android", "build Android libvcore.so artifacts"),
("windows", "build packaged Windows artifacts"),
):
command = platforms.add_parser(name, help=description)
command.add_argument(
"--delivery",
action="store_true",
help="record production artifact identity",
)
download = commands.add_parser("download", help="download official test peers")
downloads = download.add_subparsers(dest="download", required=True)
peer = downloads.add_parser("mihomo", help="download the latest stable mihomo")
peer.add_argument(
"--target", choices=SUPPORTED_TARGETS, help="default: host platform"
)
check = commands.add_parser("check", help="run repository checks")
checks = check.add_subparsers(dest="check", required=True)
core = checks.add_parser(
"core", help="run memory-only tests or feature/build checks"
)
core.add_argument(
"--profile", choices=("debug", "release", "features"), default="debug"
)
core.add_argument("--list", dest="list_only", action="store_true")
checks.add_parser("c-header", help="compile vcore.h as C and C++")
checks.add_parser("tls-dependencies", help="validate the locked TLS graph")
mihomo = checks.add_parser(
"mihomo-interop", help="run local protocol interoperability against mihomo"
delivery = checks.add_parser(
"platform-artifacts", help="verify production artifact evidence, not devices"
)
mihomo.add_argument("--binary", type=Path)
mihomo.add_argument(
"--container-binary",
type=Path,
help="run upstream/terminal peers in Apple Container (Linux ARM64 binary)",
)
mihomo.add_argument(
"--extended",
delivery.add_argument("--manifest", type=Path, action="append", required=True)
delivery.add_argument("--source-dir", type=Path)
delivery.add_argument(
"--complete",
action="store_true",
help="include repeated cross-protocol two-hop gates",
help="require every production platform group",
)
mihomo.add_argument(
"--soak-seconds",
type=int,
default=0,
help="requires --extended; full soak acceptance needs 1800 wall-clock seconds",
abi = checks.add_parser("platform-abi", help="link/load native production artifact")
abi.add_argument("--manifest", type=Path, required=True)
reality = checks.add_parser(
"reality-hybrid",
help="verify isolated hybrid REALITY, not the complete security suite",
)
reality.add_argument("--run-dir", type=Path, required=True)
reality.add_argument("--case", dest="identifiers", action="append")
xhttp = checks.add_parser(
"xhttp-peers",
help="probe isolated native XHTTP peer capabilities, not suite acceptance",
)
xhttp.add_argument("--run-dir", type=Path, required=True)
xhttp.add_argument(
"--identities-only",
action="store_true",
help="probe real download-leg client-identity enforcement only",
)
gateway = checks.add_parser(
"xhttp-gateway",
help="build xcaddy and test isolated H3/mTLS native topology",
)
gateway.add_argument("--run-dir", type=Path, required=True)
gateway.add_argument(
"--identities-only",
action="store_true",
help="small identity probes with native QUIC certificate-error observation",
)
coverage = checks.add_parser(
"protocol-coverage", help="validate executable catalogs or persisted evidence"
)
coverage_modes = coverage.add_mutually_exclusive_group(required=True)
coverage_modes.add_argument(
"--catalog-only",
action="store_true",
help="check declarations only, not implementation or behavior acceptance",
)
coverage_modes.add_argument(
"--run-dir", type=Path, help="validate a complete persisted suite run"
)
suites = [name.lower() for name in SUITES]
coverage.add_argument("--suite", choices=suites, default="foundations")
coverage.add_argument(
"--manifest",
type=Path,
help="explicit frozen executable manifest for evidence checks",
)
protocol = checks.add_parser(
"protocol-interop", help="run a protocol or integration capability suite"
)
protocol.add_argument(
"--suite", choices=suites, required=True, help="capability suite to run"
)
protocol.add_argument("--case", dest="identifiers", action="append")
protocol.add_argument(
"--protocol",
choices=[
"foundation",
"legacy",
"trojan",
"vmess",
"vless",
"hysteria2",
],
)
modes = protocol.add_mutually_exclusive_group()
modes.add_argument("--list", dest="list_only", action="store_true")
modes.add_argument("--preflight", dest="preflight_only", action="store_true")
protocol.add_argument(
"--run-dir", type=Path, help="fresh child directory of target/interop/runs"
)
demo = commands.add_parser("demo", help="run opt-in interoperability demos")
demos = demo.add_subparsers(dest="demo", required=True)
tun2socks = demos.add_parser(
"windows-tun2socks", help="run VCore TUN through an external Xray SOCKS inbound"
)
tun2socks.add_argument("config", nargs="?", type=Path)
tun2socks.add_argument("config", type=Path)
tun2socks.add_argument("--xray-source", type=Path, required=True)
return parser
@@ -63,29 +149,69 @@ def main(argv: Sequence[str] | None = None) -> int:
args = _parser().parse_args(argv)
try:
if args.command == "build":
if args.platform == "apple":
if args.delivery:
from .platform_delivery import build_delivery
build_delivery(args.platform)
elif args.platform == "apple":
build_apple()
elif args.platform == "android":
build_android()
else:
build_windows()
elif args.command == "download":
download_mihomo(args.target)
elif args.command == "check":
if args.check == "c-header":
if args.check == "core":
from .core_checks import run
run(args.profile, list_only=args.list_only)
elif args.check == "c-header":
check_c_header()
elif args.check == "mihomo-interop":
if args.extended or args.soak_seconds or args.container_binary:
run_mihomo_interop(
args.binary,
extended=args.extended,
soak_seconds=args.soak_seconds,
container_binary=args.container_binary,
)
elif args.check == "platform-artifacts":
from .platform_delivery import check_delivery
check_delivery(
args.manifest, source_dir=args.source_dir, complete=args.complete
)
elif args.check == "platform-abi":
from .platform_delivery import check_abi
check_abi(args.manifest.resolve())
elif args.check == "xhttp-peers":
from .protocol_xhttp_peers import main as xhttp_peers
return xhttp_peers(args.run_dir, identities_only=args.identities_only)
elif args.check == "xhttp-gateway":
from .protocol_xhttp_gateway import main as xhttp_gateway
return xhttp_gateway(args.run_dir, identities_only=args.identities_only)
elif args.check == "protocol-coverage":
if args.catalog_only:
check_protocol_catalogs(CATALOG_DIR)
else:
run_mihomo_interop(args.binary)
check_run(
args.run_dir.resolve(),
args.suite.upper(),
args.manifest,
)
elif args.check == "protocol-interop":
run_protocol_interop(
stage=args.suite.upper(),
identifiers=args.identifiers,
protocol=args.protocol,
list_only=args.list_only,
preflight_only=args.preflight_only,
run_dir=args.run_dir,
)
elif args.check == "reality-hybrid":
from .protocol_reality_hybrid import main as run_reality_hybrid
return run_reality_hybrid(args.run_dir, args.identifiers)
else:
check_tls_dependencies()
else:
run_demo(args.config)
run_demo(args.config, xray_source=args.xray_source)
except (OSError, RuntimeError, ValueError, subprocess.SubprocessError) as error:
print(f"vcore-scripts: {error}", file=sys.stderr)
return 1
@@ -0,0 +1,105 @@
"""Bounded one-way bandwidth origin, only inside the owned HYSTERIA2 container.
64-KiB records carry a sequence and deterministic payload. Upload progress is
timestamped at the actual origin, not inferred from the client's send buffer.
"""
import hashlib
import os
import select
import socket
import struct
import sys
import threading
import time
SIZE = 65536
END = (1 << 64) - 1
SLOTS = threading.BoundedSemaphore(4)
def exact(stream, size):
value = bytearray()
while len(value) < size:
part = stream.recv(size - len(value))
if not part:
raise ValueError("truncated bandwidth record")
value.extend(part)
return bytes(value)
def record(sequence):
return struct.pack("!Q", sequence) + bytes([sequence % 251]) * (SIZE - 8)
def serve(control):
try:
with control, socket.socket() as listener:
control.settimeout(90)
direction = exact(control, 1)
if direction not in (b"U", b"D"):
raise ValueError("invalid bandwidth direction")
listener.bind(("0.0.0.0", 0))
listener.listen(1)
listener.settimeout(15)
control.sendall(struct.pack("!H", listener.getsockname()[1]))
stream, _ = listener.accept()
with stream:
stream.settimeout(30)
stream.sendall(b"ready")
digest, sequence = hashlib.sha256(), 0
started = time.monotonic_ns()
if direction == b"U":
while True:
data = exact(stream, SIZE)
number = struct.unpack("!Q", data[:8])[0]
if number == END:
break
if data != record(sequence):
raise ValueError("upload sequence or payload mismatch")
digest.update(data)
sequence += 1
control.sendall(
b"P"
+ struct.pack(
"!QQ", sequence, time.monotonic_ns() - started
)
)
stream.sendall(b"done!")
else:
if exact(stream, 1) != b"S":
raise ValueError("missing start")
while True:
ready, _, _ = select.select([stream], [], [], 0)
if ready:
if exact(stream, 1) != b"X":
raise ValueError("invalid stop")
break
data = record(sequence)
stream.sendall(data)
digest.update(data)
sequence += 1
stream.sendall(record(END))
control.sendall(b"D" + struct.pack("!Q", sequence) + digest.digest())
except (OSError, ValueError) as error:
print(type(error).__name__, flush=True)
finally:
SLOTS.release()
def main():
if sys.platform != "linux" or os.environ.get("VCORE_ISOLATED_ORIGIN") != "1":
raise SystemExit("isolated container required")
with socket.socket() as listener:
listener.bind(("0.0.0.0", 24002))
listener.listen(4)
while True:
control, _ = listener.accept()
if SLOTS.acquire(blocking=False):
threading.Thread(target=serve, args=(control,), daemon=True).start()
else:
control.close()
if __name__ == "__main__":
main()
@@ -0,0 +1,74 @@
"""Read a real official client's close behavior; execute only inside its VM lab."""
import json
import os
import socket
import ssl
import struct
import sys
from pathlib import Path
def exact(io, length):
data = bytearray()
while len(data) < length:
part = io.recv(length - len(data))
if not part:
raise ValueError("truncated close control")
data.extend(part)
return bytes(data)
def main(proxy, origin, variant="plain"):
if sys.platform != "linux" or os.environ.get("VCORE_ISOLATED_ORIGIN") != "1":
raise RuntimeError("isolated close consumer required")
with socket.create_connection((origin, 24000), timeout=10) as control:
vision = variant == "vision-direct"
if variant not in ("plain", "vision-direct"):
raise ValueError("unknown close variant")
control.sendall(b"\x15" if vision else b"\x0b")
port = struct.unpack("!H", exact(control, 2))[0]
with socket.create_connection((proxy, 23002), timeout=10) as client:
client.sendall(b"\x05\x01\x00")
assert exact(client, 2) == b"\x05\x00"
client.sendall(
b"\x05\x01\x00\x01" + socket.inet_aton(origin) + struct.pack("!H", port)
)
header = exact(client, 4)
assert header[:3] == b"\x05\x00\x00"
exact(client, 6 if header[3] == 1 else 18)
io = client
if vision:
from tls_bio import TlsBio
context = ssl.create_default_context(
cafile=str(Path(__file__).with_name("cert.pem"))
)
context.minimum_version = ssl.TLSVersion.TLSv1_3
context.maximum_version = ssl.TLSVersion.TLSv1_3
io = TlsBio(client, context, server=False)
assert io.tls.version() == "TLSv1.3"
assert exact(control, 2) == b"A\x13"
assert exact(io, 5) == b"hello"
payload = b"Z" * 65536 if vision else b"ping"
io.sendall(payload)
assert exact(io, len(payload)) == payload
client.shutdown(socket.SHUT_WR)
tail = bytearray()
try:
while chunk := io.recv(1024):
tail.extend(chunk)
if len(tail) > 1024:
raise ValueError("close tail limit")
except ConnectionResetError:
pass
assert exact(control, 1 if vision else 2) == (b"D" if vision else b"AD")
print(
json.dumps(
{"tail_hex": tail.hex(), "terminated": True, "variant": variant}
)
)
if __name__ == "__main__":
main(*sys.argv[1:])
@@ -0,0 +1,56 @@
"""Read-only, bounded native qlog observer for the isolated identity diagnostic.
Return certificate failure categories only, never frames, keys, connection IDs,
request headers or peer addresses. Raw qlogs stay in the disposable container.
"""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
def summarize_trace(text):
failures = []
for line in text.splitlines(keepends=True):
if not line.endswith("\n"):
continue # The native writer can still be appending the last record.
row = json.loads(line.lstrip("\x1e"))
if row.get("name") != "transport:connection_closed":
continue
data = row.get("data", {})
code = data.get("connection_error", "")
if data.get("initiator") != "local" or not code.startswith("crypto_error_"):
continue
# RFC 9001 CRYPTO_ERROR = 0x100 + the RFC 8446 TLS alert number.
# quic-go's local crypto error deliberately leaves qlog.reason empty.
categories = {
"crypto_error_0x174": "certificate_required",
"crypto_error_0x12d": "certificate_expired",
"crypto_error_0x130": "unknown_ca",
}
category = categories.get(code)
if category:
failures.append({"tls_alert": code, "category": category})
return failures
def main():
files = sorted(Path("/data/qlog").glob("*.sqlog"))
if len(files) > 128 or sum(path.stat().st_size for path in files) > 16 * 1024**2:
raise ValueError("native identity trace exceeded diagnostic bound")
print(
json.dumps(
{
hashlib.sha256(path.name.encode()).hexdigest(): summarize_trace(
path.read_text()
)
for path in files
}
)
)
if __name__ == "__main__":
main()
@@ -0,0 +1,263 @@
"""Container-only OpenSSL cover sites, TCP origin, and transparent observation."""
import json
import os
import select
import socket
import ssl
import sys
import threading
import time
from pathlib import Path
ROOT = Path("/data/fixture")
EVENTS = Path("/data/events.jsonl")
LOCK = threading.Lock()
SLOTS = threading.BoundedSemaphore(32)
def emit(**event):
with LOCK, EVENTS.open("a") as output:
if EVENTS.stat().st_size > 8 * 1024 * 1024:
raise ValueError("observer event limit")
output.write(json.dumps(event) + "\n")
def exact(stream, size):
data = bytearray()
while len(data) < size:
part = stream.recv(size - len(data))
if not part:
raise EOFError()
data.extend(part)
return bytes(data)
def word(data, at):
return int.from_bytes(data[at : at + 2], "big")
def client_hello(stream):
wire, message = bytearray(), bytearray()
for _ in range(8):
head = exact(stream, 5)
size = word(head, 3)
if head[0] != 22 or not 1 <= size <= 16384:
raise ValueError("TLS record")
body = exact(stream, size)
wire.extend(head + body)
message.extend(body)
if len(message) > 65536:
raise ValueError("hello limit")
if len(message) >= 4:
expected = 4 + int.from_bytes(message[1:4], "big")
if message[0] != 1 or len(message) > expected:
raise ValueError("ClientHello length")
if len(message) == expected:
return bytes(wire), bytes(message)
raise ValueError("record count")
def extensions(message, client):
at = 39 + message[38]
if client:
at += 2 + word(message, at)
at += 1 + message[at]
else:
at += 3
end = at + 2 + word(message, at)
at += 2
if end != len(message):
raise ValueError("extension length")
while at < end:
kind, size = word(message, at), word(message, at + 2)
data = message[at + 4 : at + 4 + size]
if len(data) != size:
raise ValueError("truncated extension")
yield kind, data
at += 4 + size
def shares(message):
result = []
for kind, data in extensions(message, True):
if kind == 51:
at = 2
while at < len(data):
group, size = word(data, at), word(data, at + 2)
if group & 0x0F0F != 0x0A0A:
result.append([group, size])
at += 4 + size
if at != len(data):
raise ValueError("key share length")
return result
def selected_group(wire):
if len(wire) < 5 or len(wire) < 5 + word(wire, 3):
return None
if wire[0] != 22 or wire[5] != 2:
return -1
message = wire[5 : 5 + word(wire, 3)]
size = 4 + int.from_bytes(message[1:4], "big")
for kind, data in extensions(message[:size], False):
if kind == 51:
return word(data, 0)
return -1
def handle(stream, port, context):
try:
with stream:
stream.settimeout(5)
if port == 9000:
# Readiness opens a connection without application data.
received = 0
while part := stream.recv(16384):
if not received:
emit(port=port, event="origin-data")
received += len(part)
if received > 10 * 1024 * 1024:
raise ValueError("origin byte limit")
stream.sendall(part)
if received:
emit(port=port, event="origin-complete", bytes=received)
return
try:
raw, hello = client_hello(stream)
except EOFError:
return
emit(
port=port,
event="client-hello",
shares=shares(hello),
groups=[
word(data, at)
for kind, data in extensions(hello, True)
if kind == 10
for at in range(2, len(data), 2)
if word(data, at) & 0x0F0F != 0x0A0A
],
)
if context is None:
address = json.loads((ROOT / "routes.json").read_text())[str(port)]
with socket.create_connection(tuple(address), timeout=5) as upstream:
upstream.sendall(raw)
deadline, total = time.monotonic() + 240, len(raw)
flight, observed = bytearray(), False
while time.monotonic() < deadline:
ready, _, _ = select.select([stream, upstream], [], [], 0.2)
for source in ready:
data = source.recv(16384)
if not data:
if source is upstream:
emit(
port=port,
event="server-close",
bytes=len(flight),
)
return
total += len(data)
if total > 128 * 1024 * 1024:
raise ValueError("relay byte limit")
if source is upstream and not observed:
flight.extend(data)
if len(flight) > 65536:
raise ValueError("server flight limit")
group = selected_group(flight)
if group is not None:
emit(
port=port,
event="server-hello"
if group >= 0
else "tls-response",
group=group,
hrr=flight[11:43]
== bytes.fromhex(
"cf21ad74e59a6111be1d8c021e65b891"
"c2a211167abb8c5e079e09e2c8a8339c"
),
)
observed = True
(upstream if source is stream else stream).sendall(data)
raise TimeoutError("relay duration")
incoming, outgoing = ssl.MemoryBIO(), ssl.MemoryBIO()
tls = context.wrap_bio(incoming, outgoing, server_side=True)
incoming.write(raw)
for _ in range(32):
try:
tls.do_handshake()
stream.sendall(outgoing.read())
emit(port=port, event="cover-complete", version=tls.version())
return
except ssl.SSLWantReadError:
data = outgoing.read()
if data:
stream.sendall(data)
data = stream.recv(16384)
if not data:
return
incoming.write(data)
raise ValueError("cover iteration limit")
except ssl.SSLError as error:
emit(
port=port,
event="no-shared-group"
if error.reason == "NO_SUITABLE_KEY_SHARE"
else "closed",
)
except (OSError, ValueError, EOFError):
emit(port=port, event="closed")
finally:
SLOTS.release()
def serve(port, context):
with socket.socket(socket.AF_INET6) as listener:
listener.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
listener.bind(("::", port))
listener.listen(32)
while True:
stream, _ = listener.accept()
SLOTS.acquire()
threading.Thread(
target=handle, args=(stream, port, context), daemon=True
).start()
if __name__ == "__main__":
if sys.platform != "linux" or os.environ.get("VCORE_ISOLATED_ORIGIN") != "1":
raise SystemExit("isolated REALITY observer required")
EVENTS.touch()
contexts = {}
for port, group, version in [
(24431, "X25519MLKEM768", ssl.TLSVersion.TLSv1_3),
(24432, "X25519", ssl.TLSVersion.TLSv1_3),
(24433, "secp384r1", ssl.TLSVersion.TLSv1_3),
(24434, "X25519", ssl.TLSVersion.TLSv1_2),
]:
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(ROOT / "cert.pem", ROOT / "key.pem")
# Python's EC-NID setter rejects hybrid group names even with OpenSSL
# 3.5. Use the native default list for the hybrid cover, then verify the
# actual ServerHello/finished connection selected 4588 in the harness.
if group != "X25519MLKEM768":
context.set_ecdh_curve(group)
context.minimum_version = context.maximum_version = version
context.set_alpn_protocols(["h2", "http/1.1"])
contexts[port] = context
emit(event="native-tls", version=ssl.OPENSSL_VERSION)
threads = [
threading.Thread(target=serve, args=(port, context))
for port, context in [
*contexts.items(),
*(
(int(port), None)
for port in json.loads((ROOT / "routes.json").read_text())
),
]
]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
@@ -0,0 +1,48 @@
"""Fixture-only public OpenSSL MemoryBIO driver; never a TLS implementation."""
import ssl
class TlsBio:
def __init__(self, stream, context, *, server):
self.stream = stream
self.incoming = ssl.MemoryBIO()
self.outgoing = ssl.MemoryBIO()
self.tls = context.wrap_bio(
self.incoming,
self.outgoing,
server_side=server,
server_hostname=None if server else "localhost",
)
self.perform(self.tls.do_handshake)
def flush(self):
while self.outgoing.pending:
self.stream.sendall(self.outgoing.read())
def perform(self, operation, *args):
while True:
try:
result = operation(*args)
self.flush()
return result
except ssl.SSLWantWriteError:
self.flush()
except ssl.SSLWantReadError:
self.flush()
chunk = self.stream.recv(65536)
if chunk:
self.incoming.write(chunk)
else:
self.incoming.write_eof()
def recv(self, length):
try:
return self.perform(self.tls.read, length)
except (ssl.SSLEOFError, ssl.SSLZeroReturnError):
return b""
def sendall(self, data):
offset = 0
while offset < len(data):
offset += self.perform(self.tls.write, data[offset:])
@@ -0,0 +1,179 @@
"""Container-only TLS observer; never decodes or serves a proxy protocol."""
from __future__ import annotations
import base64
import contextlib
import json
import os
import socket
import ssl
import threading
import time
from pathlib import Path
def capture_client_hello(receive):
"""Read at most eight records and 64 KiB of ClientHello from injected IO."""
def exact(count):
result = bytearray()
while len(result) < count:
part = receive(count - len(result))
if not part:
raise EOFError("incomplete TLS record")
result.extend(part)
return bytes(result)
records, handshake, layout = bytearray(), bytearray(), []
for _ in range(8):
header = exact(5)
size = int.from_bytes(header[3:], "big")
if header[0] != 22 or not 1 <= size <= 18432:
raise ValueError("expected bounded handshake record")
payload = exact(size)
records.extend(header + payload)
handshake.extend(payload)
layout.append(
dict(type=header[0], version=int.from_bytes(header[1:3], "big"), bytes=size)
)
if len(handshake) >= 4:
hello_size = int.from_bytes(handshake[1:4], "big") + 4
if handshake[0] != 1 or hello_size > 65536:
raise ValueError("expected bounded ClientHello")
if len(handshake) >= hello_size:
return bytes(records), bytes(handshake[:hello_size]), layout
raise ValueError("ClientHello record limit")
def main():
if os.environ.get("VCORE_ISOLATED_TLS_OBSERVER") != "1":
raise RuntimeError("TLS observer requires an owned isolated container")
log = Path("/data/events.jsonl")
lock, stop = threading.Lock(), threading.Event()
deadline = time.monotonic() + 900
log.touch()
def emit(event):
with lock:
if log.stat().st_size > 8 * 1024 * 1024:
stop.set()
raise RuntimeError("observer output limit")
with log.open("a") as output:
output.write(json.dumps(event, separators=(",", ":")) + "\n")
contexts = {24430: None}
for port, version in (
(24431, ssl.TLSVersion.TLSv1_3),
(24432, ssl.TLSVersion.TLSv1_2),
):
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain("/data/fixture/cert.pem", "/data/fixture/key.pem")
context.minimum_version = context.maximum_version = version
context.set_alpn_protocols(["h2", "http/1.1"])
contexts[port] = context
def serve(port, context):
try:
with socket.socket() as listener:
listener.settimeout(0.5)
listener.bind(("0.0.0.0", port))
listener.listen(8)
sequence = 0
while not stop.is_set() and time.monotonic() < deadline:
try:
stream, _ = listener.accept()
except TimeoutError:
continue
sequence += 1
if sequence > 512:
raise RuntimeError("observer connection limit")
with stream:
stream.settimeout(5)
event = dict(port=port, sequence=sequence)
try:
raw, hello, layout = capture_client_hello(stream.recv)
except EOFError:
continue # The readiness connect carries no TLS bytes.
except (OSError, ValueError) as error:
emit(
event
| dict(
status="capture-error", reason=type(error).__name__
)
)
continue
event.update(
client_hello_b64=base64.b64encode(hello).decode(),
records_b64=base64.b64encode(raw).decode(),
records=layout,
)
if context is None:
emit(event | dict(status="captured-only"))
with contextlib.suppress(OSError):
stream.sendall(bytes.fromhex("15030300020228"))
continue
incoming, outgoing = ssl.MemoryBIO(), ssl.MemoryBIO()
tls = context.wrap_bio(incoming, outgoing, server_side=True)
incoming.write(raw)
try:
consumed = len(raw)
for _ in range(32):
try:
tls.do_handshake()
stream.sendall(outgoing.read())
emit(
event
| dict(
status="tls-handshake-complete",
version=tls.version(),
cipher=tls.cipher()[0],
alpn=tls.selected_alpn_protocol(),
resumed=tls.session_reused,
)
)
break
except ssl.SSLWantReadError:
stream.sendall(outgoing.read())
chunk = stream.recv(16384)
if not chunk:
raise EOFError() from None
consumed += len(chunk)
if consumed > 131072:
raise ValueError(
"TLS handshake input limit"
) from None
incoming.write(chunk)
else:
raise ValueError("TLS handshake iteration limit")
except (OSError, EOFError, ValueError) as error:
with contextlib.suppress(OSError):
stream.sendall(outgoing.read())
emit(
event
| dict(
status="tls-handshake-rejected",
reason=getattr(
error, "reason", type(error).__name__
),
)
)
except BaseException as error:
emit(dict(status="observer-error", reason=type(error).__name__))
stop.set()
workers = [
threading.Thread(target=serve, args=(port, ctx))
for port, ctx in contexts.items()
]
print(
json.dumps(dict(openssl=ssl.OPENSSL_VERSION, ports=list(contexts))), flush=True
)
for worker in workers:
worker.start()
for worker in workers:
worker.join()
if __name__ == "__main__":
main()
@@ -0,0 +1,286 @@
"""Bounded UDP echo origin, run only inside the owned fixture container.
A TCP observer allocates one UDP association with a one-byte address family.
The response starts with its two-byte UDP port. Each real datagram is reported
as length/source-port (two network-order u16s), then the exact payload. Echoes
are sent by this container, never by the host test driver.
"""
from __future__ import annotations
import contextlib
import os
import select
import socket
import ssl
import struct
import sys
import threading
import time
CAPACITY = 20000
INTEGRATION_PROFILE = os.environ.get("VCORE_ORIGIN_PROFILE") == "INTEGRATION"
# Explicit, still bounded pressure profile. Historical protocol fixtures keep
# their original limits; no production queue or socket policy is changed.
SLOTS = threading.BoundedSemaphore(96 if INTEGRATION_PROFILE else 8)
COVER_SLOTS = threading.BoundedSemaphore(8)
def receive_exact(stream, size):
data = bytearray()
while len(data) < size:
chunk = stream.recv(min(65536, size - len(data)))
if not chunk:
raise ValueError("truncated fixture request")
data.extend(chunk)
return bytes(data)
def serve_tcp(control, mode, ipv6=False):
"""10=bulk/server-first, 11=close differential, 12=identity probe.
Observer A proves an actual accept; D proves exact request bytes and normal
completion. Rejected identities must produce neither. No host-side server.
"""
with socket.socket(
socket.AF_INET6 if ipv6 else socket.AF_INET, socket.SOCK_STREAM
) as listener:
if ipv6:
listener.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
listener.bind(("::" if ipv6 else "0.0.0.0", 0))
listener.listen(8)
control.sendall(struct.pack("!H", listener.getsockname()[1]))
for _ in range(32):
ready, _, _ = select.select([control, listener], [], [], 30)
if not ready or control in ready:
return
stream, _ = listener.accept()
control.sendall(b"A")
if mode == 21:
# Generate the withheld tail with the real TLS engine before
# transport EOF. This tests FIN semantics without depending on
# OpenSSL accepting an application write after a ragged EOF.
from tls_bio import TlsBio
with stream:
stream.settimeout(15)
tls = TlsBio(
stream, tls_context(ssl.TLSVersion.TLSv1_3), server=True
)
assert tls.tls.version() == "TLSv1.3"
control.sendall(b"\x13")
tls.sendall(b"hello")
data = receive_exact(tls, 65536)
if data != b"Z" * 65536:
raise ValueError("TLS direct close data")
tls.sendall(data)
tail = b"native-after-upload-eof"
assert tls.tls.write(tail) == len(tail)
if stream.recv(1):
raise ValueError("unexpected data after TLS direct request")
with contextlib.suppress(OSError):
tls.flush()
control.sendall(b"D")
continue
if mode in (16, 18):
context = tls_context(
ssl.TLSVersion.TLSv1_3 if mode == 16 else ssl.TLSVersion.TLSv1_2
)
stream.settimeout(15)
stream = context.wrap_socket(stream, server_side=True)
control.sendall(b"\x13" if stream.version() == "TLSv1.3" else b"\x12")
with stream:
stream.settimeout(15)
if mode == 13: # bounded-lifetime echo; client-first and cancellation
while data := stream.recv(65536):
stream.sendall(data)
elif mode == 14: # measureDelay/HTTP entrypoint, not a proxy decoder
request = bytearray()
while not request.endswith(b"\r\n\r\n"):
request.extend(receive_exact(stream, 1))
if len(request) > 16384:
raise ValueError("HTTP origin header limit")
stream.sendall(
b"HTTP/1.1 200 OK\r\nConnection: close\r\n"
b"Content-Length: 0\r\n\r\n"
)
elif mode == 15: # deliberately withhold all handshake replies
while stream.recv(65536):
pass
elif mode == 12:
if receive_exact(stream, 15) != b"synthetic-probe":
raise ValueError("identity probe data")
stream.sendall(b"ok")
else:
stream.sendall(b"hello")
if mode in (10, 16, 18):
data = receive_exact(stream, 10 * 1024 * 1024)
if data != b"\x5a" * len(data):
raise ValueError("bulk data")
stream.sendall(data + b"trailer")
else:
data = receive_exact(stream, 4)
if data != b"ping":
raise ValueError("close probe data")
stream.sendall(data)
if stream.recv(1):
raise ValueError("unexpected data after request")
if mode == 11:
# Whole-close peers need not accept a tail.
with contextlib.suppress(OSError):
stream.sendall(b"native-after-upload-eof")
control.sendall(b"D")
def serve(control):
try:
with control:
control.settimeout(15)
control.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
family = control.recv(1)
if family and (family[0] & 0x7F) in (*range(10, 17), 18, 21):
serve_tcp(control, family[0] & 0x7F, bool(family[0] & 0x80))
return
if family not in (b"\x04", b"\x06", b"\x11", b"\x13", b"\x14"):
return
ipv6 = family == b"\x06"
with socket.socket(
socket.AF_INET6 if ipv6 else socket.AF_INET, socket.SOCK_DGRAM
) as udp:
if ipv6:
udp.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
udp.bind(("::" if ipv6 else "0.0.0.0", 0))
control.sendall(struct.pack("!H", udp.getsockname()[1]))
peer_ipv4 = (
socket.inet_ntop(socket.AF_INET, receive_exact(control, 4))
if family == b"\x13"
else None
)
seen = False
dns = family in (b"\x11", b"\x13")
dns_deadline = time.monotonic() + 240
for _ in range(100000 if INTEGRATION_PROFILE else 20000):
wait = (
min(30, max(0, dns_deadline - time.monotonic())) if dns else 30
)
if not wait:
return
ready, _, _ = select.select([control, udp], [], [], wait)
if control in ready:
return # EOF/cancel closes this association's UDP socket.
if not ready:
if dns:
# Runtime-owned DNS can be quiet during TCP or an
# IP-only UDP phase. Keep it until control closes,
# within the same bounded 240-second fixture lifetime.
continue
return
packet, source = udp.recvfrom(CAPACITY + 1)
if len(packet) > CAPACITY:
return
if family == b"\x14":
# A native origin blackhole, not a protocol peer. Report
# only first receipt; all handshake bytes are discarded.
if not seen:
control.sendall(b"A")
seen = True
continue
control.sendall(struct.pack("!HH", len(packet), source[1]) + packet)
response = (
dns_response(packet, control.getsockname()[0], peer_ipv4)
if family in (b"\x11", b"\x13")
else packet
)
if udp.sendto(response, source) != len(response):
return
except (OSError, ValueError):
# Client cancellation/failed association is recorded by the test driver.
pass
finally:
SLOTS.release()
def dns_response(packet, ipv4, peer_ipv4=None):
"""Only the synthetic business name is answerable; never system DNS."""
question = b"\x0dvcore-fixture\x04test\x00"
peer_question = b"\x04peer\x07fixture\x04test\x00"
if peer_ipv4 and packet[12 : 12 + len(peer_question)] == peer_question:
question, ipv4 = peer_question, peer_ipv4
if (
len(packet) < 12 + len(question) + 4
or packet[12 : 12 + len(question)] != question
):
raise ValueError("unexpected DNS question (including magic names)")
qtype = struct.unpack("!H", packet[-4:-2])[0]
answer = socket.inet_pton(socket.AF_INET, ipv4) if qtype == 1 else b""
response = bytearray(packet)
response[2:4] = b"\x81\x80"
response[6:8] = struct.pack("!H", int(bool(answer)))
if answer:
response.extend(b"\xc0\x0c\x00\x01\x00\x01\x00\x00\x00\x00\x00\x04" + answer)
return bytes(response)
def tls_context(version):
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.minimum_version = version
context.maximum_version = version
context.set_ecdh_curve("X25519")
context.load_cert_chain(
os.environ["VCORE_ORIGIN_CERT"], os.environ["VCORE_ORIGIN_KEY"]
)
context.set_alpn_protocols(["h2", "http/1.1"])
context.num_tickets = 0
return context
def serve_camouflage(stream, context):
try:
with stream:
stream.settimeout(5)
with context.wrap_socket(stream, server_side=True) as secured:
secured.recv(1)
except OSError:
pass
finally:
COVER_SLOTS.release()
def camouflage():
"""Container-only TLS 1.3 target for REALITY authentication."""
context = tls_context(ssl.TLSVersion.TLSv1_3)
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as listener:
listener.bind(("0.0.0.0", 24001))
listener.listen(8)
while True:
stream, _ = listener.accept()
# Independent XHTTP legs need concurrent cover handshakes. A quiet
# first leg must not hold the accept loop for its five-second bound.
if not COVER_SLOTS.acquire(blocking=False):
stream.close()
continue
threading.Thread(
target=serve_camouflage, args=(stream, context), daemon=True
).start()
def main():
if sys.platform != "linux" or os.environ.get("VCORE_ISOLATED_ORIGIN") != "1":
raise SystemExit(
"BLOCKED: this origin must be launched by the container harness"
)
if os.environ.get("VCORE_ORIGIN_CERT"):
threading.Thread(target=camouflage, daemon=True).start()
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as listener:
listener.bind(("0.0.0.0", 24000))
listener.listen(8)
while True:
control, _ = listener.accept()
if not SLOTS.acquire(blocking=False):
control.close()
continue
threading.Thread(target=serve, args=(control,), daemon=True).start()
if __name__ == "__main__":
main()
+168
View File
@@ -0,0 +1,168 @@
"""Memory-only checks shared by local development and CI.
Network consumers remain explicit container suites. Keep this allowlist independent
of historical milestone runners: compiling all targets is safe; executing them is not.
"""
from __future__ import annotations
import re
from .builds import CORE_DIR, DEFAULT_FEATURES
from .protocol_peers import run_command
FEATURE_TEST = (
"runtime::tests::integration_feature_admission_is_explicit_without_opening_sockets"
)
PROTOCOLS = ("socks5", "anytls", "shadowsocks", "trojan", "vmess", "vless", "hysteria2")
TARGETS = (
"ech_config",
"ech_tls",
"encryption_config",
"reality_config",
"jls_config",
"security_capabilities",
"retired_security",
"feature_foundations",
"limit_foundations",
"trojan",
"trojan_config",
"vmess_config",
"vless_config",
"vless_lifecycle",
"xhttp_config",
"xhttp_budget",
"h2_stream_regression",
"stream_foundations",
"stream_shutdown",
"xhttp_h3_shutdown",
"hysteria2_paths",
"hysteria2_config",
"shadowsocks_backpressure",
"hysteria2_packet_ids",
)
LIB_FILTERS = (
"security::",
"outbound::vless::encryption",
"config::",
"outbound::hysteria2::",
"outbound::shadowsocks::tests::official_tcp_",
"outbound::anytls::",
"transport::h2_write::tests::",
"transport::sing_mux::smux_driver::tests::",
"transport::sing_mux::tests::",
)
EXACT_LIB_TESTS = (
"outbound::connector::tests::authenticated_continuation_keeps_group_choice_but_has_a_new_io_deadline",
FEATURE_TEST,
"ffi::tests::runtime_thread_cannot_reenter_invoke",
"ffi::tests::null_invalid_utf8_and_oversized_input_return_json_failures",
"ffi::tests::same_instance_command_is_fail_fast",
"ffi::tests::android_protector_is_required_only_for_tun",
)
def commands(profile: str) -> list[list[str]]:
cargo = ["cargo", "test", "--locked", "--all-features"]
if profile in {"debug", "release"}:
if profile == "release":
cargo += ["--release"]
return [
cargo + [arg for target in TARGETS for arg in ("--test", target)],
*(cargo + ["--lib", name] for name in LIB_FILTERS),
*(cargo + ["--lib", name, "--", "--exact"] for name in EXACT_LIB_TESTS),
cargo
+ [
"--test",
"quic_datagram_foundations",
"pending_send_is_not_restarted_and_stop_cancels_without_waiting_for_writable",
"--",
"--exact",
],
]
if profile == "features":
features = ["outbound-" + protocol for protocol in PROTOCOLS]
return (
[
["cargo", "check", "--locked", "--no-default-features", "--lib"]
+ (["--features", name] if name else [])
for name in [
"",
*features,
"stream-transport",
"quic-transport",
"tun",
"ffi",
]
]
+ [
[
"cargo",
"test",
"--locked",
"--no-default-features",
"--features",
name,
"--lib",
FEATURE_TEST,
"--",
"--exact",
]
for name in features
]
+ [
[
"cargo",
"test",
"--locked",
"--no-default-features",
"--test",
"feature_foundations",
"--test",
"hysteria2_config",
],
["cargo", "check", "--locked", "--lib"],
cargo + ["--all-targets", "--no-run"],
[
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
],
]
)
raise ValueError("unknown core check profile")
def validate_execution(
command: list[str], output: str, returncode: int, cleanup: bool
) -> None:
if returncode != 0 or not cleanup:
raise RuntimeError("core check failed, timed out, or did not join its process")
if command[:2] != ["cargo", "test"] or "--no-run" in command:
return
summaries = re.findall(
r"test result: ok\. (\d+) passed; (\d+) failed; \d+ ignored;", output
)
expected = max(1, command.count("--test"))
if len(summaries) != expected or any(
int(passed) == 0 or int(failed) for passed, failed in summaries
):
raise RuntimeError(
"core check selected no tests or returned incomplete test results"
)
def run(profile: str, *, list_only: bool = False) -> None:
for command in commands(profile):
print(" ".join(command), flush=True)
if list_only:
continue
result = run_command(command, cwd=CORE_DIR, timeout=3600, limit=4 * 1024 * 1024)
output = result.stdout.decode("utf-8", errors="replace")
print(output, end="", flush=True)
validate_execution(command, output, result.returncode, result.cleanup)
+39 -46
View File
@@ -15,7 +15,7 @@ from contextlib import ExitStack
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import exclusive_run, reserve_port
from .mihomo_isolation import reserve_port
def _stop_peer(peer: subprocess.Popen) -> None:
@@ -41,32 +41,6 @@ def _wait_ready(peer, port: int, host: str = "127.0.0.1") -> None:
raise RuntimeError("mihomo did not become ready within 10 seconds")
def run_mihomo_interop(
binary: Path | None = None,
*,
extended: bool = False,
soak_seconds: int = 0,
container_binary: Path | None = None,
) -> None:
if not 0 <= soak_seconds <= 7200 or (soak_seconds and not extended):
raise ValueError("soak seconds must be 0..7200 and require --extended")
binary = (
binary
or Path(os.environ.get("VCORE_MIHOMO_BIN", CORE_DIR / "references/mihomo-test"))
).resolve()
if not binary.is_file():
raise RuntimeError(
"mihomo binary is missing; build the reference or pass --binary (NOT RUN)"
)
with exclusive_run():
_run_mihomo_interop(
binary,
extended=extended,
soak_seconds=soak_seconds,
container_binary=container_binary,
)
def _run_mihomo_interop(
binary: Path,
*,
@@ -320,25 +294,44 @@ def _run_mihomo_interop(
if containers:
environment.update(containers.environment())
try:
subprocess.run(
[
"cargo",
"test",
"--locked",
"--features",
"ffi",
"--test",
"mihomo_interop",
"--",
"--ignored",
"--nocapture",
"--test-threads=1",
],
cwd=CORE_DIR,
env=environment,
check=True,
timeout=300 + soak_seconds,
)
command = [
"cargo",
"test",
"--locked",
"--features",
"ffi,interop-test" if os.environ.get("VCORE_CASE_EVENTS") else "ffi",
"--test",
"mihomo_interop",
"public_client_inbounds_interoperate_with_mihomo_in_both_directions",
"--",
"--exact",
"--ignored",
"--nocapture",
"--test-threads=1",
]
if os.environ.get("VCORE_CASE_EVENTS"):
from .protocol_peers import run_command
completed = run_command(
command,
cwd=CORE_DIR,
env=environment,
timeout=300 + soak_seconds,
limit=4 * 1024 * 1024,
)
print(completed.stdout.decode("utf-8", errors="replace"), flush=True)
if completed.returncode != 0 or not completed.cleanup:
raise subprocess.CalledProcessError(
completed.returncode or 1, command
)
else:
subprocess.run(
command,
cwd=CORE_DIR,
env=environment,
check=True,
timeout=300 + soak_seconds,
)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired):
for index in range(len(configs)):
print(f"Generated fixture peer {index} diagnostics (last 4096 bytes):")
@@ -19,6 +19,9 @@ class PortReservation:
# IPv4-only peers take these ports; IPv6 guards remain owned until exit.
self.ipv4.close()
def release_ipv6(self) -> None:
self.ipv6.close()
def close(self) -> None:
self.ipv4.close()
self.ipv6.close()
+231
View File
@@ -0,0 +1,231 @@
"""Download the latest official Mihomo release for interop tests without an API."""
from __future__ import annotations
import gzip
import hashlib
import http.client
import io
import os
import platform
import re
import stat
import tempfile
import time
import urllib.parse
import urllib.request
import zipfile
import zlib
from contextlib import contextmanager
from pathlib import Path, PurePosixPath
from .builds import CORE_DIR
LATEST_VERSION_URL = (
"https://github.com/MetaCubeX/mihomo/releases/latest/download/version.txt"
)
RELEASE_URL = "https://github.com/MetaCubeX/mihomo/releases/download"
SUPPORTED_TARGETS = (
"darwin-arm64",
"darwin-amd64",
"linux-arm64",
"linux-amd64",
"windows-arm64",
"windows-amd64",
)
MAX_VERSION_BYTES = 128
MAX_ARCHIVE_BYTES = 128 * 1024 * 1024
MAX_BINARY_BYTES = 256 * 1024 * 1024
CHUNK_BYTES = 64 * 1024
TRANSFER_SECONDS = 180
def _target(target: str | None) -> str:
if target is None:
system = platform.system().lower()
machine = platform.machine().lower()
architecture = {
"arm64": "arm64",
"aarch64": "arm64",
"amd64": "amd64",
"x86_64": "amd64",
}.get(machine, machine)
target = f"{system}-{architecture}"
if target not in SUPPORTED_TARGETS:
raise RuntimeError(f"unsupported official mihomo target: {target}")
return target
def _release_tag(tag: str) -> str:
if (
not isinstance(tag, str)
or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}", tag) is None
):
raise RuntimeError("invalid official mihomo download tag")
return tag
def _copy_and_hash(source, output, limit: int) -> str:
digest = hashlib.sha256()
total = 0
deadline = time.monotonic() + TRANSFER_SECONDS
# read1 avoids waiting for a complete chunk before checking the deadline.
read = getattr(source, "read1", source.read)
while True:
if time.monotonic() > deadline:
raise RuntimeError("official mihomo transfer exceeded its time limit")
chunk = read(min(CHUNK_BYTES, limit - total + 1))
if not chunk:
break
total += len(chunk)
if total > limit:
raise RuntimeError("official mihomo payload exceeded its size limit")
digest.update(chunk)
output.write(chunk)
if not total:
raise RuntimeError("official mihomo payload is empty")
return digest.hexdigest()
def _https_response(response) -> None:
if urllib.parse.urlsplit(response.geturl()).scheme != "https":
raise RuntimeError("official mihomo download redirected outside HTTPS")
def latest_release() -> str:
"""Read the official latest/version.txt asset only to form download filenames."""
request = urllib.request.Request(
LATEST_VERSION_URL, headers={"User-Agent": "VCore-interop-scripts"}
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
_https_response(response)
contents = io.BytesIO()
_copy_and_hash(response, contents, MAX_VERSION_BYTES)
tag = contents.getvalue().decode("utf-8").strip()
except (OSError, ValueError, http.client.HTTPException) as error:
raise RuntimeError(
f"cannot read official mihomo latest version.txt: {error}"
) from error
return _release_tag(tag)
def _asset(tag: str, target: str) -> tuple[str, str]:
variant = "-v1" if target in {"linux-amd64", "windows-amd64"} else ""
suffix = "zip" if target.startswith("windows-") else "gz"
name = f"mihomo-{target}{variant}-{tag}.{suffix}"
return name, f"{RELEASE_URL}/{tag}/{name}"
@contextmanager
def _payload(archive: Path):
if archive.suffix == ".gz":
with gzip.open(archive, "rb") as payload:
yield payload
return
with zipfile.ZipFile(archive) as bundle:
files = []
for member in bundle.infolist():
name = member.filename
if (
PurePosixPath(name).is_absolute()
or ".." in PurePosixPath(name).parts
or "\\" in name
or ":" in name
):
raise RuntimeError("unsafe path in official mihomo ZIP")
if member.is_dir():
continue
mode = stat.S_IFMT(member.external_attr >> 16)
if (
mode not in {0, stat.S_IFREG}
or member.flag_bits & 1
or len(PurePosixPath(name).parts) != 1
or not name.endswith(".exe")
):
raise RuntimeError("unsafe executable entry in official mihomo ZIP")
files.append(member)
if len(files) != 1:
raise RuntimeError(
"official mihomo ZIP must contain exactly one executable"
)
if not 0 < files[0].file_size <= MAX_BINARY_BYTES:
raise RuntimeError("official mihomo executable exceeds its size limit")
with bundle.open(files[0]) as payload:
yield payload
def _report(
tag: str, asset: str, url: str, archive_digest: str, digest: str, binary: Path
):
print(f"Mihomo download release tag: {tag}", flush=True)
print(f"Official mihomo asset: {asset}", flush=True)
print(f"Official mihomo download URL: {url}", flush=True)
print(f"Mihomo archive SHA-256 (downloaded content): {archive_digest}", flush=True)
print(f"Mihomo binary SHA-256 (downloaded content): {digest}", flush=True)
print(f"Official mihomo binary: {binary}", flush=True)
def download_mihomo(
target: str | None = None,
*,
release: str | None = None,
directory: Path | None = None,
identity: dict | None = None,
) -> Path:
"""Download a fresh official archive and atomically publish its binary.
Callers can share a freshly read version.txt tag for multiple architectures.
Every call downloads again; failures neither replace nor return an old binary.
Hashes identify downloaded content, without claiming an upstream digest check.
The harness records the actual version by running the binary with -v.
"""
target = _target(target)
if release is None:
release = latest_release()
tag = _release_tag(release)
name, url = _asset(tag, target)
directory = (
directory
if directory is not None
else CORE_DIR / "target/interop/mihomo" / tag / target
)
binary = directory / ("mihomo.exe" if target.startswith("windows-") else "mihomo")
request = urllib.request.Request(
url, headers={"User-Agent": "VCore-interop-scripts"}
)
try:
directory.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix=".download-", dir=directory) as staging:
staged_archive = Path(staging) / name
staged_binary = Path(staging) / binary.name
with (
urllib.request.urlopen(request, timeout=30) as response,
staged_archive.open("wb") as output,
):
_https_response(response)
archive_digest = _copy_and_hash(response, output, MAX_ARCHIVE_BYTES)
with _payload(staged_archive) as source, staged_binary.open("wb") as output:
digest = _copy_and_hash(source, output, MAX_BINARY_BYTES)
staged_binary.chmod(0o755)
os.replace(staged_binary, binary)
except (
OSError,
EOFError,
http.client.HTTPException,
zipfile.BadZipFile,
zlib.error,
) as error:
raise RuntimeError(f"cannot download official mihomo {tag}: {error}") from error
_report(tag, name, url, archive_digest, digest, binary)
if identity is not None:
identity.update(
kind="M",
target=target,
release=tag,
asset=name,
source_url=url,
archive_sha256=archive_digest,
binary_sha256=digest,
)
return binary
+229
View File
@@ -0,0 +1,229 @@
"""Fresh official native peers. No API lookup, source build or stale fallback."""
from __future__ import annotations
import http.client
import os
import platform
import re
import stat
import subprocess
import tarfile
import tempfile
import urllib.request
import zipfile
from dataclasses import dataclass
from pathlib import Path, PurePosixPath
from .mihomo_release import (
MAX_ARCHIVE_BYTES,
MAX_BINARY_BYTES,
_copy_and_hash,
_https_response,
)
from .protocol_peers import run_command
@dataclass(frozen=True)
class PeerArtifact:
binary: Path
identity: dict
def download_native(
kind: str,
directory: Path,
target: str | None = None,
*,
defer_version: bool = False,
) -> PeerArtifact:
try:
return _download_native(kind, directory, target, defer_version)
except (
OSError,
ValueError,
zipfile.BadZipFile,
tarfile.TarError,
http.client.HTTPException,
subprocess.SubprocessError,
):
raise RuntimeError(
"official native peer download or version check failed"
) from None
def _download_native(
kind: str, directory: Path, target: str | None, defer_version: bool
) -> PeerArtifact:
"""Install only a named executable into this run's owned directory."""
if target is None:
architecture = {"aarch64": "arm64", "x86_64": "amd64"}.get(
platform.machine().lower(), platform.machine().lower()
)
target = f"{platform.system().lower()}-{architecture}"
assets = {
"SS": (
"shadowsocks/shadowsocks-rust",
"ssserver",
{"linux-arm64": "aarch64-unknown-linux-musl"},
),
"V2": (
"v2fly/v2ray-core",
"v2ray",
{
"darwin-arm64": "v2ray-macos-arm64-v8a.zip",
"darwin-amd64": "v2ray-macos-64.zip",
"linux-arm64": "v2ray-linux-arm64-v8a.zip",
"linux-amd64": "v2ray-linux-64.zip",
},
),
"XR": (
"XTLS/Xray-core",
"xray",
{
"darwin-arm64": "Xray-macos-arm64-v8a.zip",
"darwin-amd64": "Xray-macos-64.zip",
"linux-arm64": "Xray-linux-arm64-v8a.zip",
"linux-amd64": "Xray-linux-64.zip",
},
),
"H": (
"HyNetworks/hysteria",
"hysteria",
{
target: f"hysteria-{target}"
for target in [
"darwin-arm64",
"darwin-amd64",
"linux-arm64",
"linux-amd64",
]
},
),
}
if kind not in assets or target not in assets[kind][2]:
raise RuntimeError("unsupported official native peer target")
project, name, targets = assets[kind]
asset = targets[target]
if kind == "SS":
# SS assets include their tag; follow latest's HTTP redirect, not an API
# or a pinned version. The executable reports its own actual version.
request = urllib.request.Request(
f"https://github.com/{project}/releases/latest",
method="HEAD",
headers={"User-Agent": "VCore-interop-scripts"},
)
with urllib.request.urlopen(request, timeout=30) as response:
prefix = f"https://github.com/{project}/releases/tag/"
resolved = response.geturl()
if not resolved.startswith(prefix) or not re.fullmatch(
r"v\d+\.\d+\.\d+", resolved[len(prefix) :]
):
raise RuntimeError("invalid official SS latest stable redirect")
tag = resolved[len(prefix) :]
asset = f"shadowsocks-{tag}.{asset}.tar.xz"
url = f"https://github.com/{project}/releases/latest/download/{asset}"
directory.mkdir(parents=True, exist_ok=True)
binary = directory / name
with tempfile.TemporaryDirectory(prefix=".download-", dir=directory) as temporary:
archive, executable = Path(temporary) / "archive.zip", Path(temporary) / name
request = urllib.request.Request(
url, headers={"User-Agent": "VCore-interop-scripts"}
)
with (
urllib.request.urlopen(request, timeout=30) as response,
archive.open("wb") as output,
):
_https_response(response)
archive_hash = _copy_and_hash(response, output, MAX_ARCHIVE_BYTES)
if kind == "H":
with archive.open("rb") as source, executable.open("wb") as output:
binary_hash = _copy_and_hash(source, output, MAX_BINARY_BYTES)
elif kind == "SS":
binary_hash = _extract_tar(archive, executable, name)
else:
binary_hash = _extract_zip(archive, executable, name)
executable.chmod(0o755)
# Foreign executables are verified inside the owned VM before traffic;
# a deferred artifact is not yet a ready/verified peer.
version = None
if not defer_version:
result = run_command(
[str(executable), "--version" if kind == "SS" else "version"],
timeout=10,
limit=4096,
)
if result.returncode != 0 or not result.cleanup:
raise RuntimeError("official native peer version check failed")
version = result.stdout.decode("utf-8", errors="replace").strip()
if not version:
raise RuntimeError("invalid official native peer version output")
os.replace(executable, binary)
return PeerArtifact(
binary,
{
"kind": kind,
"target": target,
"source_url": url,
"archive_sha256": archive_hash,
"binary_sha256": binary_hash,
"version": version,
},
)
def _extract_zip(archive: Path, executable: Path, name: str) -> str:
with zipfile.ZipFile(archive) as bundle:
matches = []
for member in bundle.infolist():
path = PurePosixPath(member.filename)
mode = stat.S_IFMT(member.external_attr >> 16)
if (
path.is_absolute()
or ".." in path.parts
or "\\" in member.filename
or ":" in member.filename
or mode not in {0, stat.S_IFREG, stat.S_IFDIR}
or member.flag_bits & 1
):
raise RuntimeError("unsafe official native peer archive")
if member.filename == name:
matches.append(member)
if len(matches) != 1 or not 0 < matches[0].file_size <= MAX_BINARY_BYTES:
raise RuntimeError("invalid official native peer executable")
with bundle.open(matches[0]) as source, executable.open("wb") as output:
binary_hash = _copy_and_hash(source, output, MAX_BINARY_BYTES)
return binary_hash
def _extract_tar(archive: Path, executable: Path, name: str) -> str:
"""Stream a bounded official bundle; never extract archive paths or links."""
found, total, digest = 0, 0, None
with tarfile.open(archive, "r|xz") as bundle:
for index, entry in enumerate(bundle):
path = PurePosixPath(entry.name)
total += entry.size
if (
index >= 32
or total > MAX_BINARY_BYTES
or path.is_absolute()
or ".." in path.parts
or "\\" in entry.name
or ":" in entry.name
or not (entry.isfile() or entry.isdir())
):
raise RuntimeError("unsafe official native peer archive")
if entry.name != name:
continue
found += 1
if (
found != 1
or not entry.isfile()
or not 0 < entry.size <= MAX_BINARY_BYTES
):
raise RuntimeError("invalid official native peer executable")
with bundle.extractfile(entry) as source, executable.open("wb") as output:
digest = _copy_and_hash(source, output, MAX_BINARY_BYTES)
if found != 1:
raise RuntimeError("missing official native peer executable")
return digest
@@ -0,0 +1,495 @@
"""Production build evidence. This is not device or signed-release acceptance."""
from __future__ import annotations
import hashlib
import json
import os
import platform
import plistlib
import shutil
import stat
import subprocess
from datetime import UTC, datetime
from pathlib import Path, PurePosixPath
from . import builds
GROUPS = {"apple", "android", "windows-arm64", "windows-x64"}
APPLE_LIBRARIES = {
"ios-arm64": ("ios", None, {"arm64"}),
"ios-arm64_x86_64-simulator": ("ios", "simulator", {"arm64", "x86_64"}),
"macos-arm64_x86_64": ("macos", None, {"arm64", "x86_64"}),
}
def _sha(path: Path) -> str:
with path.open("rb") as stream:
return hashlib.file_digest(stream, "sha256").hexdigest()
def _output(argv: list[str], root: Path) -> str:
return subprocess.check_output(argv, cwd=root, text=True, timeout=60).strip()
def _source(root: Path) -> dict:
if _output(["git", "status", "--porcelain", "--untracked-files=normal"], root):
raise ValueError("delivery requires a clean committed source checkout")
return {
"commit": _output(["git", "rev-parse", "HEAD"], root),
"tree": _output(["git", "rev-parse", "HEAD^{tree}"], root),
"lockSha256": _sha(root / "Cargo.lock"),
}
def check_delivery(
manifests: list[Path], *, source_dir: Path | None = None, complete: bool = False
) -> None:
source = _source(source_dir or builds.CORE_DIR)
groups = set()
if not manifests:
raise ValueError("empty manifest set cannot pass")
for manifest in manifests:
record = json.loads(manifest.read_text(encoding="utf-8"))
if (
record.get("formatVersion") != 1
or record.get("profile") != "release"
or record.get("features") != builds.DEFAULT_FEATURES.split(",")
or record.get("buildIdentity") != builds.EXPECTED_IDENTITY.decode()
):
raise ValueError("incompatible production artifact metadata")
if record.get("source") != source:
raise ValueError("artifact source/lock identity does not match checkout")
group = record.get("group")
if group not in GROUPS or group in groups:
raise ValueError("duplicate delivery group")
groups.add(group)
toolchain = record.get("toolchain", {})
required = {"rustc", "cargo"} | (
{"ndk", "clang", "androidApi"}
if group == "android"
else {"xcode", "iphoneos", "iphonesimulator", "macosx"}
if group == "apple"
else {"msvc", "windowsSdk"}
)
if not all(
isinstance(toolchain.get(key), str) and toolchain[key] for key in required
):
raise ValueError("missing toolchain evidence")
rows = record.get("artifacts", [])
if not rows:
raise ValueError("empty artifact set cannot pass")
names = set()
for row in rows:
name = row["path"]
relative = PurePosixPath(name)
if (
relative.is_absolute()
or ".." in relative.parts
or "\\" in name
or str(relative) != name
or name in names
):
raise ValueError("invalid or duplicate artifact path")
names.add(name)
base = manifest.parent.resolve()
path = base / name
if not path.is_file() or any(
base.joinpath(*relative.parts[:index]).is_symlink()
for index in range(1, len(relative.parts) + 1)
):
raise ValueError(f"missing or non-regular artifact: {name}")
if path.stat().st_size != row["size"] or _sha(path) != row["sha256"]:
raise ValueError(f"artifact hash/size mismatch: {name}")
actual = {
p.relative_to(manifest.parent).as_posix()
for p in manifest.parent.rglob("*")
if p.is_file() and p != manifest
}
if actual != names:
raise ValueError("unrecorded or missing artifact files")
if record.get("group") == "android":
expected = {
f"{abi}/{name}"
for abi in ("arm64-v8a", "x86_64")
for name in ("libvcore.so", "libc++_shared.so")
}
if names != expected:
raise ValueError("Android delivery requires both ABIs and C++ runtimes")
for abi, machine in (("arm64-v8a", 183), ("x86_64", 62)):
for name in ("libvcore.so", "libc++_shared.so"):
path = manifest.parent / abi / name
with path.open("rb") as stream:
header = stream.read(20)
if (
header[:7] != b"\x7fELF\x02\x01\x01"
or len(header) != 20
or int.from_bytes(header[16:18], "little") != 3
or int.from_bytes(header[18:20], "little") != machine
):
raise ValueError(
f"wrong Android ELF architecture: {abi}/{name}"
)
if name == "libvcore.so":
builds._require_identity(path, "Android")
elif group == "apple":
expected = {"LibVCore.xcframework/Info.plist"} | {
f"LibVCore.xcframework/{identifier}/{name}"
for identifier in APPLE_LIBRARIES
for name in (
"libvcore.a",
"Headers/vcore.h",
"Headers/module.modulemap",
)
}
if names != expected:
raise ValueError("incomplete Apple XCFramework")
with (manifest.parent / "LibVCore.xcframework/Info.plist").open(
"rb"
) as stream:
libraries = plistlib.load(stream).get("AvailableLibraries", [])
if len(libraries) != 3:
raise ValueError("incomplete Apple platform slices")
identifiers = set()
for library in libraries:
identifier = library.get("LibraryIdentifier")
if identifier not in APPLE_LIBRARIES or identifier in identifiers:
raise ValueError("invalid Apple library identifier")
identifiers.add(identifier)
target_os, variant, architectures = APPLE_LIBRARIES[identifier]
if (
library.get("SupportedPlatform") != target_os
or library.get("SupportedPlatformVariant") != variant
or set(library.get("SupportedArchitectures", [])) != architectures
or library.get("LibraryPath") != "libvcore.a"
or library.get("HeadersPath") != "Headers"
):
raise ValueError("invalid Apple slice metadata")
path = (
manifest.parent / "LibVCore.xcframework" / identifier / "libvcore.a"
)
actual_archs = set(
_output(
["xcrun", "lipo", "-archs", str(path)], builds.CORE_DIR
).split()
)
if actual_archs != architectures:
raise ValueError("wrong Apple binary architecture")
builds._require_identity(path, "Apple")
else:
arch = group.removeprefix("windows-")
expected = {
"vcore.dll",
"vcore-windows-vpn-host.exe",
"vcore-windows-session-host.exe",
}
if names != expected | {"vcore-windows-artifacts.json"}:
raise ValueError("incomplete Windows artifact set")
if (
record.get("host", {}).get("os") != "Windows"
or record["host"].get("architecture") != arch
):
raise ValueError(
"Windows delivery requires native OS architecture evidence"
)
for name in expected:
builds._require_windows_architecture(manifest.parent / name, arch)
builds._require_identity(manifest.parent / "vcore.dll", "Windows")
package = json.loads(
(manifest.parent / "vcore-windows-artifacts.json").read_text()
)
if (
package.get("formatVersion") != 1
or package.get("architecture") != arch
or package.get("windowsPackageIntegrationRevision") != 3
or package.get("buildIdentity") != record["buildIdentity"]
or package.get("artifacts")
!= {
row["path"]: row["sha256"]
for row in rows
if row["path"] in expected
}
):
raise ValueError("Windows package integration identity mismatch")
print(
f"PASS {record['group']} artifact integrity (not device/release acceptance)"
)
if complete and groups != GROUPS:
raise ValueError(
"complete platform artifacts require Apple, Android, "
"native Windows ARM64 and x64"
)
if complete:
print(
"PASS production platform artifacts; "
"ABI, devices and release require separate evidence"
)
def build_delivery(platform_name: str) -> None:
if os.environ.get("VCORE_BUILD_PROFILE", "release") != "release":
raise ValueError("delivery requires the release profile")
if (
os.environ.get("VCORE_FEATURES", builds.DEFAULT_FEATURES)
!= builds.DEFAULT_FEATURES
):
raise ValueError("delivery requires the complete production feature set")
# Keep the recorded build command honest: existing development builds still
# allow customization, but acceptance must not inherit hidden overrides.
overrides = [
name
for name in os.environ
if (
name
in {
"RUSTFLAGS",
"CARGO_ENCODED_RUSTFLAGS",
"CARGO_TARGET_DIR",
"VCORE_ANDROID_OUTPUT_DIR",
"VCORE_APPLE_DIST_DIR",
"VCORE_ANDROID_TARGETS",
}
or name.startswith("CARGO_PROFILE_")
)
and os.environ[name]
]
if overrides:
raise ValueError(
"unsupported delivery build overrides: " + ", ".join(sorted(overrides))
)
source = _source(builds.CORE_DIR)
group = platform_name
architecture = platform.machine().lower()
toolchain = {
"rustc": _output(["rustc", "-Vv"], builds.CORE_DIR),
"cargo": _output(["cargo", "--version"], builds.CORE_DIR),
}
if platform_name == "windows":
if os.name != "nt":
raise ValueError("Windows delivery must run on native Windows")
architecture = builds._windows_architecture()
group += "-" + architecture
environment = builds._windows_msvc_environment(architecture)
# Windows env names are case insensitive, even when a subprocess returns
# a plain Python dict. Never persist the full environment (credentials).
normalized = {key.upper(): value for key, value in environment.items()}
toolchain["msvc"] = normalized.get("VCTOOLSVERSION", "").strip()
toolchain["windowsSdk"] = normalized.get("WINDOWSSDKVERSION", "").strip("\\ ")
toolchain["cmake"] = _output(["cmake", "--version"], builds.CORE_DIR)
toolchain["generator"] = "Ninja" if architecture == "arm64" else "Visual Studio"
if architecture == "arm64":
toolchain["clang"] = _output(
[normalized["VCORE_WINDOWS_ARM64_CLANG"], "--version"], builds.CORE_DIR
)
toolchain["assembly"] = "enabled"
elif platform_name == "apple":
toolchain["xcode"] = _output(["xcodebuild", "-version"], builds.CORE_DIR)
for sdk in ("iphoneos", "iphonesimulator", "macosx"):
toolchain[sdk] = _output(
["xcrun", "--sdk", sdk, "--show-sdk-version"], builds.CORE_DIR
)
toolchain["iosDeploymentTarget"] = os.environ.get(
"VCORE_IOS_DEPLOYMENT_TARGET", "13.0"
)
toolchain["macosDeploymentTarget"] = os.environ.get(
"VCORE_MACOS_DEPLOYMENT_TARGET", "10.15"
)
else:
android_home = Path(
os.environ.get("ANDROID_HOME", Path.home() / "Library/Android/sdk")
)
ndk = Path(
os.environ.get(
"ANDROID_NDK_HOME",
android_home
/ "ndk"
/ os.environ.get("VCORE_ANDROID_NDK_VERSION", "28.2.13676358"),
)
)
toolchain["ndk"] = (ndk / "source.properties").read_text().strip()
toolchain["clang"] = _output(
[str(builds._android_toolchain(ndk) / "bin/clang"), "--version"],
builds.CORE_DIR,
)
toolchain["androidApi"] = os.environ.get("VCORE_ANDROID_API", "24")
output = builds.CORE_DIR / "dist" / platform_name
if platform_name == "windows":
output /= architecture
# Check the lexical path before resolving, unlinking a manifest, or letting
# a builder clean output. CORE_DIR is the canonical, trusted checkout root;
# Windows junctions and other reparse points can redirect descendants too.
current = builds.CORE_DIR
for part in output.relative_to(builds.CORE_DIR).parts:
current /= part
try:
metadata = current.lstat()
except FileNotFoundError:
break
if stat.S_ISLNK(metadata.st_mode) or (
getattr(metadata, "st_file_attributes", 0)
& stat.FILE_ATTRIBUTE_REPARSE_POINT
):
raise ValueError(
"delivery output must not contain symlinks or reparse points"
)
manifest = output / "vcore-delivery.json"
if platform_name == "android":
# Development builds can leave additional ABIs in the same ignored
# directory. Never mix those artifacts into a fresh delivery manifest.
if output.exists():
shutil.rmtree(output)
else:
manifest.unlink(missing_ok=True)
started = datetime.now(UTC).isoformat()
{
"apple": builds.build_apple,
"android": builds.build_android,
"windows": builds.build_windows,
}[platform_name]()
if _source(builds.CORE_DIR) != source:
raise ValueError("source changed during delivery build")
record = {
"formatVersion": 1,
"group": group,
"profile": "release",
"source": source,
"features": builds.DEFAULT_FEATURES.split(","),
"buildIdentity": builds.EXPECTED_IDENTITY.decode(),
"host": {
"os": platform.system(),
"architecture": architecture,
"osVersion": platform.release(),
},
"toolchain": toolchain,
"startedUtc": started,
"finishedUtc": datetime.now(UTC).isoformat(),
"artifacts": [
{
"path": path.relative_to(output).as_posix(),
"size": path.stat().st_size,
"sha256": _sha(path),
}
for path in sorted(output.rglob("*"))
if path.is_file()
],
}
manifest.write_text(
json.dumps(record, indent=2, sort_keys=True) + "\n", encoding="utf-8"
)
try:
check_delivery([manifest])
except Exception:
manifest.unlink(missing_ok=True)
raise
print(manifest)
def check_abi(manifest: Path) -> None:
"""Compile a real C consumer; no replacement library and no protocol servers."""
# Commands below run from an isolated output directory, not the caller cwd.
manifest = manifest.resolve()
check_delivery([manifest])
record = json.loads(manifest.read_text(encoding="utf-8"))
group = record["group"]
root = builds.CORE_DIR
work = root / "target/platform-delivery/abi" / group
work.mkdir(parents=True, exist_ok=True)
(work / "result.json").unlink(missing_ok=True)
source = root / "scripts/fixtures/platform_abi.c"
binary = work / ("abi.exe" if os.name == "nt" else "abi")
environment = os.environ.copy()
if group == "apple" and platform.system() == "Darwin":
library = manifest.parent / "LibVCore.xcframework/macos-arm64_x86_64/libvcore.a"
command = [
"xcrun",
"clang",
"-Wall",
"-Wextra",
"-Werror",
"-I",
str(root / "include"),
str(source),
str(library),
"-lc++",
"-lresolv",
"-framework",
"Security",
"-framework",
"SystemConfiguration",
"-framework",
"CoreFoundation",
"-o",
str(binary),
]
execute = [str(binary)]
architecture = platform.machine()
elif group.startswith("windows-") and os.name == "nt":
architecture = builds._windows_architecture()
if group != "windows-" + architecture:
raise ValueError("ABI check cannot substitute emulation for native Windows")
environment = builds._windows_msvc_environment(architecture)
search_path = next(
value for key, value in environment.items() if key.upper() == "PATH"
)
# CreateProcess does not use env[PATH] to resolve the executable. Resolve
# cl from vcvars explicitly rather than requiring it in the parent PATH.
compiler = shutil.which("cl", path=search_path)
if not compiler:
raise ValueError(
"MSVC C compiler is unavailable in the selected environment"
)
command = [
compiler,
"/nologo",
"/std:c11",
"/W4",
"/WX",
"/MT",
"/I" + str(root / "include"),
str(source),
"/Fe:" + str(binary),
"/Fo:" + str(work / "abi.obj"),
]
execute = [str(binary), str(manifest.parent / "vcore.dll")]
else:
raise ValueError(
"native ABI runner requires a matching macOS or Windows artifact"
)
subprocess.run(command, cwd=work, env=environment, check=True, timeout=120)
subprocess.run(execute, cwd=work, env=environment, check=True, timeout=60)
if group.startswith("windows-"):
target = {"arm64": "aarch64-pc-windows-msvc", "x64": "x86_64-pc-windows-msvc"}[
architecture
]
base = [
"cargo",
"test",
"--locked",
"--release",
"--target",
target,
"--features",
"ffi",
]
for suffix in (
["--lib", "windows::snapshot::tests"],
["--test", "windows_session_startup"],
):
subprocess.run(
base + suffix, cwd=root, env=environment, check=True, timeout=1200
)
if _source(root) != record["source"]:
raise ValueError("source changed during ABI check")
evidence = {
"kind": "native-production-abi",
"group": group,
"source": record["source"],
"manifestSha256": _sha(manifest),
"architecture": architecture,
"iterations": 1000,
"invalidApiRejected": True,
"finishedUtc": datetime.now(UTC).isoformat(),
}
(work / "result.json").write_text(
json.dumps(evidence, indent=2) + "\n", encoding="utf-8"
)
@@ -0,0 +1,84 @@
"""Executable catalog integrity; production fields are documented in config.yaml."""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
CATALOG_DIR = Path(__file__).resolve().parents[3] / "tests" / "protocols"
PROTOCOLS = {"trojan", "vmess", "vless", "hysteria2"}
PEERS = {"M", "H", "XR", "V2"}
SUITES = {
"FOUNDATIONS": (
"schema",
"security",
"streams",
"datagrams",
"features",
"regression",
),
"TROJAN": ("codec", "configuration", "cancellation", "acceptance"),
"VMESS": ("identity", "codec", "udp", "close", "acceptance"),
"VLESS": ("tcp", "transports", "vision", "acceptance"),
"XHTTP": ("requests", "download", "mux", "http3", "acceptance"),
"HYSTERIA2": ("tcp", "udp", "bandwidth", "salamander", "hopping", "acceptance"),
"SECURITY": ("encryption", "reality", "ech", "jls", "acceptance"),
"INTEGRATION": ("routing", "lifecycle", "pressure", "regression"),
}
CATEGORIES = {
f"{suite}.{category}"
for suite, categories in SUITES.items()
for category in categories
}
# Stable evidence tags; field semantics live in docs/config.yaml, not a planning mirror.
FIELD_IDS = {
f"{prefix}{index:02}"
for prefix, count in {
"C": 6,
"T": 8,
"W": 7,
"G": 6,
"H": 6,
"TR": 2,
"VM": 8,
"VL": 6,
"X": 29,
"D": 32,
"S": 13,
"M": 7,
"HY": 8,
}.items()
for index in range(1, count + 1)
if not (
(prefix == "D" and 19 <= index <= 24) or (prefix == "S" and 6 <= index <= 11)
)
}
def _unique_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, value in pairs:
if key in result:
raise ValueError("protocol catalog: duplicate JSON key")
result[key] = value
return result
def check_protocol_catalogs(directory: Path = CATALOG_DIR) -> None:
from .protocol_evidence import check_limit_references, load_manifest
cases = load_manifest()
check_limit_references(cases, directory / "limits.json")
print(
json.dumps(
{
"kind": "executable-catalog-check",
"status": "VALID",
"behavior_status": "NOT RUN",
"cases": len(cases),
"fields": len(set().union(*(set(case["row_ids"]) for case in cases))),
},
sort_keys=True,
)
)
@@ -0,0 +1,234 @@
"""Owned Apple Container test peers; no native-server fallback or host publishing."""
from __future__ import annotations
import hashlib
import ipaddress
import json
import re
import socket
import time
import uuid
from contextlib import contextmanager
from contextvars import ContextVar
from pathlib import Path
from .protocol_inputs import redact
from .protocol_peers import OwnedProcess, run_command
NETWORK = "vcore-mihomo-interop"
IMAGE = "docker.io/library/python:3-alpine"
_FROZEN_IMAGE = ContextVar("vcore_frozen_container_image", default=None)
def _pull_image(log=None):
argv = ["container", "image", "pull", IMAGE]
result = run_command(argv, timeout=180)
if log is not None:
log.write_text(redact(result.stdout.decode(errors="replace")))
if result.returncode != 0 or not result.cleanup:
raise RuntimeError("isolated container image pull failed")
inspection = json.loads(command("image", "inspect", IMAGE))[0]
digest = inspection["configuration"]["descriptor"]["digest"]
if not isinstance(digest, str) or not re.fullmatch(r"sha256:[0-9a-f]{64}", digest):
raise RuntimeError("invalid isolated container image identity")
return dict(
tag=IMAGE,
digest=digest,
command=argv,
exit_code=0,
cleanup=True,
log=log.name if log is not None else None,
)
@contextmanager
def frozen_image(log):
"""Refresh once per owned run, never a persistent/offline fallback cache."""
if _FROZEN_IMAGE.get() is not None:
raise RuntimeError("container image scope is already frozen")
snapshot = _pull_image(log)
token = _FROZEN_IMAGE.set(snapshot["digest"])
try:
yield snapshot
finally:
_FROZEN_IMAGE.reset(token)
def command(*args, timeout=30):
result = run_command(["container", *args], timeout=timeout)
if result.returncode != 0 or not result.cleanup:
# CLI output may contain fixture paths/config; retain a bounded operation only.
raise RuntimeError(f"isolated container operation failed: {args[0]}")
return result.stdout.decode(errors="replace")
def listing():
return json.loads(command("list", "--all", "--format", "json"))
class ContainerLab:
def __init__(self, record, *, mtu=1280):
if type(mtu) is not int or mtu not in (1280, 1500):
raise ValueError("unsupported isolated guest MTU")
self.mtu = mtu
self.record = record
self.run_id = uuid.uuid4().hex[:12]
network = json.loads(command("network", "inspect", NETWORK))[0]
config = network["configuration"]
if (
config["mode"] != "hostOnly"
or config.get("labels", {}).get("purpose") != NETWORK
):
raise RuntimeError("owned host-only container network required")
self.v4 = ipaddress.ip_network(network["status"]["ipv4Subnet"])
self.v6 = ipaddress.ip_network(network["status"]["ipv6Subnet"])
digest = _FROZEN_IMAGE.get()
if digest is None:
digest = _pull_image()["digest"]
self.image = IMAGE.split(":")[0] + "@" + digest
record.update(
backend="Apple Container",
network_mode="hostOnly",
image_tag=IMAGE,
image_digest=digest,
run_id=self.run_id,
peers=[],
host_servers=False,
guest_mtu=mtu,
)
def start(self, stack, root: Path, role, argv, *, net_admin=False):
peer = ContainerPeer(self, root, role)
self.record["peers"].append(peer.record)
# Register before launching: CLI interruption/timeout can leave a live VM.
stack.callback(peer.stop)
peer.start(argv, net_admin=net_admin)
return peer
class ContainerPeer:
def __init__(self, lab, root, role):
self.lab, self.root = lab, root
self.name = f"vcore-interop-{lab.run_id}-{role}"
if len(self.name) > 63:
digest = hashlib.sha256(self.name.encode()).hexdigest()[:8]
self.name = self.name[:54] + "-" + digest
self.record = dict(role=role, name=self.name, joined=False, started=False)
self.capture = None
self.log = root / "peer.log"
def start(self, argv, *, net_admin=False):
if type(net_admin) is not bool:
raise ValueError("isolated capability choice must be boolean")
self.record["net_admin"] = net_admin
command(
"run",
"--detach",
"--name",
self.name,
"--label",
f"purpose={NETWORK}",
"--label",
f"vcore-run={self.lab.run_id}",
"--network",
f"{NETWORK},mtu={self.lab.mtu}",
"--cpus",
"1",
"--memory",
"256M",
"--arch",
"arm64",
*(["--cap-add", "CAP_NET_ADMIN"] if net_admin else ["--read-only"]),
"--no-dns",
"--tmpfs",
"/data",
"--mount",
f"type=bind,source={self.root},target=/data/fixture,readonly",
"--entrypoint",
"/bin/sh",
self.lab.image,
"-ec",
"for i in $(seq 1 600); do "
'if [ -f /data/fixture/ready ]; then exec "$@"; fi; '
"sleep 0.1; done; exit 1",
"fixture",
*argv,
timeout=45,
)
self.record["started"] = True
self.capture = OwnedProcess(
["container", "logs", "--follow", self.name],
self.log,
{},
limit=1024 * 1024,
).__enter__()
state = json.loads(command("inspect", self.name))[0]
self.record["guest_mtu"] = state["status"]["networks"][0]["mtu"]
if self.record["guest_mtu"] != self.lab.mtu:
raise RuntimeError("isolated guest MTU differs from requested value")
self.ipv4 = state["status"]["networks"][0]["ipv4Address"].split("/")[0]
if ipaddress.ip_address(self.ipv4) not in self.lab.v4:
raise RuntimeError("container address outside owned network")
# Wait for SLAAC/DAD before any business traffic. No host IPv6 binds needed.
query = (
"import pathlib,json; print(json.dumps(["
"[x[0],int(x[4],16)] for line in "
"pathlib.Path('/proc/net/if_inet6').read_text().splitlines() "
"if (x:=line.split())[5]=='eth0']))"
)
deadline = time.monotonic() + 15
while time.monotonic() < deadline:
rows = json.loads(command("exec", self.name, "python", "-c", query))
for address, flags in rows:
ip = ipaddress.IPv6Address(int(address, 16))
if ip in self.lab.v6 and not flags & (0x40 | 0x08):
self.ipv6 = str(ip)
return
time.sleep(0.1)
raise RuntimeError("container IPv6 readiness failed")
def release(self):
(self.root / "ready").touch()
def ensure_alive(self):
state = json.loads(command("inspect", self.name))[0]["status"]["state"]
if state != "running" or (self.capture and self.capture.overflow.is_set()):
raise RuntimeError("owned container peer exited or exceeded output bound")
def wait_tcp(self, port):
deadline = time.monotonic() + 15
while time.monotonic() < deadline:
self.ensure_alive()
try:
with socket.create_connection((self.ipv4, port), timeout=0.2):
self.record["ready"] = True
return
except OSError:
time.sleep(0.05)
raise TimeoutError("container service readiness failed")
def stop(self):
try:
owned = next((item for item in listing() if item["id"] == self.name), None)
if owned is not None:
labels = owned["configuration"].get("labels", {})
if (
labels.get("purpose") != NETWORK
or labels.get("vcore-run") != self.lab.run_id
):
raise RuntimeError("container ownership mismatch; refusing cleanup")
try:
if owned["status"]["state"] == "running":
command("stop", "--time", "5", self.name, timeout=15)
finally:
command("delete", "--force", self.name, timeout=15)
finally:
if self.capture is not None:
self.capture.__exit__(None, None, None)
self.record["log_cleanup"] = self.capture.record.get("joined", False)
self.record["joined"] = not any(
item["id"] == self.name for item in listing()
) and self.record.get("log_cleanup", True)
if not self.record["joined"]:
raise RuntimeError("owned container cleanup incomplete")
+226
View File
@@ -0,0 +1,226 @@
"""SECURITY static ECH consumers against official latest isolated peers."""
from __future__ import annotations
import base64
import os
import shutil
import struct
import textwrap
from pathlib import Path
from .protocol_peers import run_command
def catalog():
cases = {}
def add(mode, suffix, test):
cases[f"SECURITY-ECH-{mode.upper()}-{suffix}"] = ("M", mode, True, test)
modes = (
"tcp-tls",
"ws-tls",
"upgrade-tls",
"upgrade-fast-tls",
"grpc-tls",
) + tuple(
f"xhttp-{mode}{version}"
for mode in (
"stream-one",
"stream-up",
"packet-up",
"stream-up-download",
"packet-up-download",
)
for version in ("", "-h1")
)
for mode in modes:
add(mode, "BASE", "public_base")
add(mode, "CLOSE", "native_mihomo_close_alignment")
for mode in (
"tcp-tls",
"ws-tls",
"grpc-tls",
"xhttp-stream-up-download",
"xhttp-stream-up-download-h1",
):
add(mode, "REJECT", "native_ech_fail_closed")
for mode in modes:
if "download" in mode:
add(mode, "DOWNLOAD", "public_ech_download_identity")
for mode in ("tcp-mtls", "ws-mtls", "grpc-mtls"):
add(mode, "IDENTITY", "public_tls_identity_and_verification_name")
for suffix, test in (
("GRAPH", "runtime::public_graph"),
("ENTRYPOINTS", "runtime::public_entrypoints"),
("IPV6", "runtime::public_ipv6_and_gates"),
("UDP-ISOLATION", "runtime::public_udp_isolation"),
):
add("tcp-tls", suffix, test)
for mode in ("grpc-tls", "xhttp-stream-up-download"):
add(mode, "LIFE", "runtime::public_lifecycle")
add(mode, "OWNED", "runtime::owned_resources")
return cases
def material(directory: Path, key_id: int):
"""Generate synthetic keys with official OpenSSL, never implement crypto here."""
openssl = next(
(
path
for path in (
os.environ.get("OPENSSL_BIN"),
shutil.which("openssl3"),
"/opt/homebrew/opt/openssl@3/bin/openssl",
"/usr/local/opt/openssl@3/bin/openssl",
shutil.which("openssl"),
)
if path and Path(path).is_file()
),
None,
)
if openssl is None:
raise RuntimeError("OpenSSL with X25519 support is required")
private = directory / f"ech-{key_id}.der"
generated = run_command(
[
openssl,
"genpkey",
"-algorithm",
"X25519",
"-outform",
"DER",
"-out",
str(private),
],
timeout=15,
)
public = run_command(
[
openssl,
"pkey",
"-inform",
"DER",
"-in",
str(private),
"-pubout",
"-outform",
"DER",
],
timeout=15,
)
if any(item.returncode or not item.cleanup for item in (generated, public)):
raise RuntimeError("synthetic ECH material generation failed")
secret = private.read_bytes()
if len(secret) != 48 or len(public.stdout) != 44:
raise RuntimeError("unexpected synthetic X25519 DER encoding")
name = b"public.invalid"
contents = bytes([key_id]) + struct.pack("!HH", 32, 32) + public.stdout[-32:]
contents += struct.pack("!HHHBB", 4, 1, 1, 0, len(name)) + name + b"\0\0"
config = struct.pack("!HH", 0xFE0D, len(contents)) + contents
key_set = (
struct.pack("!H", 32) + secret[-32:] + struct.pack("!H", len(config)) + config
)
return base64.b64encode(struct.pack("!H", len(config)) + config).decode(), key_set
def configuration(node, peer, directory, mode, *, kind="M"):
primary, first_key = material(directory, 7)
download, second_key = material(directory, 9)
encoded = base64.b64encode(first_key + second_key).decode()
pem = (
"-----BEGIN ECH KEYS-----\n"
+ "\n".join(textwrap.wrap(encoded, 64))
+ "\n-----END ECH KEYS-----\n"
)
(directory / "ech-keys.pem").write_text(pem)
node["ech-opts"] = dict(enable=True, config=primary)
if mode.endswith("-h1"):
node["alpn"] = ["http/1.1"]
if kind == "M":
for listener in peer["listeners"]:
listener["ech-key"] = "/data/fixture/ech-keys.pem"
elif kind == "XR":
for inbound in peer["inbounds"]:
inbound["streamSettings"]["tlsSettings"]["echServerKeys"] = encoded
else:
raise ValueError("unsupported static ECH peer")
return dict(ech_download_config=download)
def wrong_key(options):
wire = bytearray(base64.b64decode(options["config"], validate=True))
wire[11:43] = bytes([7]) * 32
return dict(enable=True, config=base64.b64encode(wire).decode())
def legacy_gateway_command():
# Xray's dokodemo-door marks its inbound splice-capable even with TLS.
# Its default freedom response path then bypasses the TLS writer. Use
# the upstream runtime switch only for this synthetic TLS gateway.
return [
"env",
"XRAY_BUF_SPLICE=disable",
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
]
def legacy_gateway_config(upstream, tls_options):
"""Official Xray TLS ingress, no custom protocol server or V2Ray patches."""
return dict(
log={"loglevel": "warning"},
inbounds=[
dict(
listen="::",
port=23000,
protocol="dokodemo-door",
settings=dict(address=upstream, port=23000, network="tcp"),
streamSettings=dict(
network="tcp", security="tls", tlsSettings=tls_options
),
)
],
outbounds=[dict(protocol="freedom")],
)
def field_variants():
cases = {}
for version in ("h1", "h2", "h3"):
for action in ("reject-main", "reject-download", "replace", "clear"):
cases[f"{version}-ech-{action}"] = (
{
"mode": "stream-up",
"download-settings": {},
"_ech_action": action,
"_reject": action.startswith("reject-"),
},
version,
)
return cases
if __name__ == "__main__":
import argparse
from .mihomo_isolation import exclusive_run
from .protocol_vless_container import CLIENT_FINGERPRINTS, run
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
parser.add_argument("cases", nargs="*")
parser.add_argument("--client-fingerprint", choices=CLIENT_FINGERPRINTS)
args = parser.parse_args()
with exclusive_run():
raise SystemExit(
run(
args.output.resolve(),
args.cases or None,
ech=True,
client_fingerprint=args.client_fingerprint,
)
)
@@ -0,0 +1,347 @@
"""Encryption wire checks; not public YAML or complete security acceptance."""
from __future__ import annotations
import base64
import contextlib
import json
import os
import shutil
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .protocol_containers import ContainerLab, command
from .protocol_inputs import redact, same_source, source_identity
from .protocol_peers import run_command
def cases():
return {
f"{style}-{rtt}-{key}": (style, rtt, key)
for style in ("native", "xorpub", "random")
for rtt in ("1rtt", "0rtt")
for key in ("x25519", "mlkem", "mixed")
}
def encode(value):
return base64.urlsafe_b64encode(bytes.fromhex(value)).decode().rstrip("=")
def run(
output: Path,
selected=None,
*,
consumer=False,
chacha=False,
expiry=False,
padding="minimal",
supplied=None,
):
output = output.resolve()
required = cases()
if expiry:
required = {
name: value for name, value in required.items() if value[1] == "0rtt"
}
paddings = {
"minimal": ".100-35-35",
"default": "",
"fragmented": ".100-35-35.100-1-1.100-47-47",
"maximum": ".100-65553-65553",
}
selected = list(required) if selected is None else selected
if (
not output.is_relative_to(CORE_DIR / "target/interop/runs")
or not selected
or len(set(selected)) != len(selected)
or not set(selected) <= required.keys()
or padding not in paddings
or (consumer and (chacha or expiry))
):
raise ValueError("fresh run directory and known unique cases required")
output.mkdir(parents=True, exist_ok=False)
if expiry:
test = "native_ticket_expiry"
marker = "SECURITY-ENCRYPTION-EXPIRY-PASS full-resumed-expired-full-resumed"
elif consumer:
test = "public_tcp_roundtrip"
marker = (
"SECURITY-ENCRYPTION-CONSUMER-PASS rounds=4 bytes_per_direction=10485760"
)
else:
test = "native_encryption_roundtrip"
marker = "SECURITY-ENCRYPTION-WIRE-PASS rounds=4 bytes_per_direction=10485760"
report = dict(
stage="SECURITY.encryption-wire",
scope="public-config-and-outbound" if consumer else "native-wire-only",
cipher="chacha20-poly1305" if chacha else "cpu-selected",
ticket_expiry=expiry,
padding=padding,
complete_selection=set(selected) == set(required),
source=source_identity(),
required=list(required),
cases=[],
isolation={},
status="NOT RUN",
)
try:
if supplied is None:
identity = {}
binary = download_mihomo(
"linux-arm64", directory=output / "binaries", identity=identity
)
else:
binary, identity = supplied["M"]
report["peer"] = identity
built = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"encryption_wire",
"--no-run",
],
timeout=240,
)
(output / "build.log").write_text(redact(built.stdout.decode(errors="replace")))
if built.returncode or not built.cleanup:
raise RuntimeError("Encryption wire consumer build failed")
vectors = json.loads(
(CORE_DIR / "tests/protocols/encryption-crypto.json").read_text()
)
pairs = dict(
x25519=(
encode(vectors["x25519_public"]),
encode(vectors["x25519_private"]),
),
mlkem=(encode(vectors["mlkem_public"]), encode(vectors["mlkem_seed"])),
)
pairs["mixed"] = tuple(
".".join((pairs["x25519"][i], pairs["mlkem"][i], pairs["x25519"][i]))
for i in (0, 1)
)
lab = ContainerLab(report["isolation"], mtu=1500)
with (
tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary,
contextlib.ExitStack() as stack,
):
root = Path(temporary)
server_dir, origin_dir = root / "server", root / "origin"
server_dir.mkdir()
origin_dir.mkdir()
shutil.copy2(binary, server_dir / "peer")
shutil.copy2(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
origin = lab.start(
stack,
origin_dir,
"security-e-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
server = lab.start(
stack,
server_dir,
"security-e-mihomo",
[
"/data/fixture/peer",
"-d",
"/data/mihomo",
"-f",
"/data/fixture/config.json",
],
)
listeners = []
for index, name in enumerate(selected):
style, _, key = required[name]
lifetime = "2-3s" if expiry else "600s"
listeners.append(
dict(
name=f"encryption-{index}",
type="vless",
listen="::",
port=23000 + index,
users=[
dict(
username="fixture",
uuid="07070707-0707-0707-0707-070707070707",
)
],
decryption=(
f"mlkem768x25519plus.{style}.{lifetime}."
f"{pairs[key][1]}{paddings[padding]}"
),
)
)
# A distinct native handler has no tickets from the primary.
# It exercises genuine unknown-ticket rejection without
# patching the peer, restarting it, or replaying business data.
listeners.append(
{
**listeners[-1],
"name": f"encryption-fresh-{index}",
"port": 23100 + index,
}
)
(server_dir / "config.json").write_text(
json.dumps(
dict(
ipv6=True,
**{"log-level": "silent"},
listeners=listeners,
rules=["MATCH,DIRECT"],
hosts={"vcore-fixture.test": origin.ipv4},
)
)
)
origin.release()
origin.wait_tcp(24000)
server.release()
for index in range(len(selected)):
server.wait_tcp(23000 + index)
server.wait_tcp(23100 + index)
identity["version"] = command(
"exec", server.name, "/data/fixture/peer", "-v"
).strip()
if (
command("exec", server.name, "sha256sum", "/data/fixture/peer").split()[
0
]
!= identity["binary_sha256"]
):
raise RuntimeError("Mihomo binary identity mismatch")
for index, name in enumerate(selected):
style, rtt, key = required[name]
encryption = (
f"mlkem768x25519plus.{style}.{rtt}."
f"{pairs[key][0]}{paddings[padding]}"
)
fixture = root / "input.json"
fixture.write_text(
json.dumps(
dict(
isolation="containers",
encryption=encryption,
force_chacha=chacha,
ticket_expiry=expiry,
node=dict(
name="edge",
type="vless",
server=server.ipv4,
port=23000 + index,
uuid="07070707-0707-0707-0707-070707070707",
udp=True,
encryption=encryption,
),
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
server_ipv4=server.ipv4,
server_ipv6=server.ipv6,
port=23000 + index,
reject_port=23100 + index,
)
)
)
argv = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"encryption_wire",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
result = run_command(
argv,
env={**os.environ, "VCORE_ENCRYPTION_FIXTURE": str(fixture)},
timeout=150,
)
text = result.stdout.decode(errors="replace")
(output / f"{name}.log").write_text(redact(text))
passed = (
result.returncode == 0
and result.cleanup
and marker in text
and "1 passed; 0 failed; 0 ignored" in text
)
report["cases"].append(
dict(
case_id=name,
command=argv,
returncode=result.returncode,
cleanup=result.cleanup,
status="PASS" if passed else "FAIL",
)
)
print(f"{name}: {'PASS' if passed else 'FAIL'}", flush=True)
if not passed:
raise RuntimeError("Encryption wire gate failed")
origin.ensure_alive()
server.ensure_alive()
report["status"] = "PASS"
except KeyboardInterrupt:
report["status"] = "INTERRUPTED"
except Exception as error:
report["status"] = "FAIL"
report["error"] = redact(str(error))
finally:
report["source_after"] = source_identity()
report["source_unchanged"] = same_source(
report["source"], report["source_after"]
)
report["cleanup"] = all(
peer.get("joined", False) for peer in report["isolation"].get("peers", [])
)
if not report["source_unchanged"] or not report["cleanup"]:
report["status"] = "FAIL"
(output / "encryption-results.json").write_text(
json.dumps(report, indent=2) + "\n"
)
return 0 if report["status"] == "PASS" else 1
if __name__ == "__main__":
import argparse
import sys
parser = argparse.ArgumentParser()
parser.add_argument("output", type=Path)
parser.add_argument("cases", nargs="*")
parser.add_argument("--consumer", action="store_true")
parser.add_argument("--chacha", action="store_true")
parser.add_argument("--expiry", action="store_true")
parser.add_argument(
"--padding",
choices=("minimal", "default", "fragmented", "maximum"),
default="minimal",
)
args = parser.parse_args()
with exclusive_run():
sys.exit(
run(
args.output,
args.cases or None,
consumer=args.consumer,
chacha=args.chacha,
expiry=args.expiry,
padding=args.padding,
)
)
@@ -0,0 +1,103 @@
"""Public Encryption consumers using the existing isolated VLESS lab."""
from __future__ import annotations
import json
from pathlib import Path
from .builds import CORE_DIR
from .protocol_encryption import cases, encode
def configuration(profile, node, peer):
style, rtt, key = cases()[profile]
vectors = json.loads(
(CORE_DIR / "tests/protocols/encryption-crypto.json").read_text()
)
pairs = {
"x25519": (encode(vectors["x25519_public"]), encode(vectors["x25519_private"])),
"mlkem": (encode(vectors["mlkem_public"]), encode(vectors["mlkem_seed"])),
}
pairs["mixed"] = tuple(
".".join((pairs["x25519"][i], pairs["mlkem"][i], pairs["x25519"][i]))
for i in (0, 1)
)
# Nontrivial fragments/gaps exercise early-data continuation, not merely
# the smallest one-flight padding used by the wire/replay lab.
padding = "100-35-35.100-1-1.100-47-47"
node["encryption"] = f"mlkem768x25519plus.{style}.{rtt}.{pairs[key][0]}.{padding}"
for listener in peer["listeners"]:
listener["decryption"] = (
f"mlkem768x25519plus.{style}.600s.{pairs[key][1]}.{padding}"
)
def catalog():
required = {}
modes = (
"tcp",
"tcp-tls",
"tcp-reality",
"ws",
"ws-tls",
"ws-ed-1",
"ws-ed-2048",
"upgrade",
"upgrade-fast",
"grpc",
"grpc-tls",
"xhttp-stream-one",
"xhttp-stream-up",
"xhttp-packet-up",
"xhttp-stream-up-download",
"xhttp-packet-up-download",
)
for mode in modes:
tls = mode.endswith(("-tls", "-reality")) or mode.startswith("xhttp-")
for suffix, test in (
("BASE", "public_base"),
("CLOSE", "native_mihomo_close_alignment"),
):
required[f"SECURITY-E-{mode.upper()}-{suffix}"] = ("M", mode, tls, test)
for mode in ("tcp-tls", "grpc-tls"):
for suffix, test in (
("GRAPH", "runtime::public_graph"),
("IPV6", "runtime::public_ipv6_and_gates"),
("ENTRYPOINTS", "runtime::public_entrypoints"),
("LIFE", "runtime::public_lifecycle"),
("OWNED", "runtime::owned_resources"),
("UDP-ISOLATION", "runtime::public_udp_isolation"),
):
required[f"SECURITY-E-{mode.upper()}-{suffix}"] = ("M", mode, True, test)
for mode in ("vision-encryption", "vision-tls", "vision-reality"):
for suffix, test in (
("BASE", "public_base"),
("INNER-TLS", "native_vision_inner_tls"),
("UDP", "native_vision_xudp"),
("CLOSE", "native_mihomo_close_alignment"),
("DIRECT-CLOSE", "native_vision_direct_close_alignment"),
):
required[f"SECURITY-E-{mode.upper()}-{suffix}"] = (
"M",
mode,
mode != "vision-encryption",
test,
)
return required
if __name__ == "__main__":
import argparse
from .mihomo_isolation import exclusive_run
from .protocol_vless_container import run
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
parser.add_argument("profile", choices=cases())
parser.add_argument("cases", nargs="*")
args = parser.parse_args()
with exclusive_run():
raise SystemExit(
run(args.output.resolve(), args.cases or None, encryption=args.profile)
)
@@ -0,0 +1,600 @@
"""Executable requirements and fail-closed structured result checks."""
from __future__ import annotations
import hashlib
import json
import re
from pathlib import Path
from .protocol_catalogs import (
CATALOG_DIR,
CATEGORIES,
FIELD_IDS,
SUITES,
_unique_object,
)
FOUNDATIONS_REQUIRED_IDS = {
"FOUNDATIONS-SCHEMA",
"FOUNDATIONS-LIMITS",
"FOUNDATIONS-STREAM",
"FOUNDATIONS-DATAGRAM",
"FOUNDATIONS-QUIC",
"FOUNDATIONS-RESOLUTION",
"FOUNDATIONS-RESOURCES",
"FOUNDATIONS-SECURITY",
"FOUNDATIONS-XUDP",
"FOUNDATIONS-REGRESSION",
"FOUNDATIONS-M-WS",
"FOUNDATIONS-M-WSS",
"FOUNDATIONS-M-GRPC",
"FOUNDATIONS-M-GRPC-TLS",
"FOUNDATIONS-M-WS-ED",
"FOUNDATIONS-V2-HTTP",
"FOUNDATIONS-V2-H2",
"FOUNDATIONS-V2-WS-HEADER",
"FOUNDATIONS-V2-WS-PATH",
"FOUNDATIONS-FEATURES",
"FOUNDATIONS-SCRIPTS",
}
HEX = re.compile(r"[0-9a-f]{64}")
RESOURCE_KINDS = {
"task",
"socket",
"session",
"association",
"reassembly",
"pool",
"waiter",
"handshake",
}
def idle_resources(snapshot) -> bool:
if not isinstance(snapshot, dict) or not isinstance(snapshot.get("counts"), list):
return False
counts = snapshot["counts"]
return (
len(counts) == len(RESOURCE_KINDS)
and all(isinstance(count, dict) for count in counts)
and {count.get("kind") for count in counts} == RESOURCE_KINDS
and all(
type(count.get("current")) is int
and count["current"] == 0
and type(count.get("peak")) is int
and count["peak"] >= 0
for count in counts
)
)
def read_events(path: Path) -> list[dict]:
if path.stat().st_size > 4 * 1024 * 1024:
raise ValueError("case events exceeded bound")
events = [
json.loads(line, object_pairs_hook=_unique_object)
for line in path.read_text().splitlines()
]
if not all(isinstance(event, dict) for event in events):
raise ValueError("malformed structured case event")
return events
def read_json(path: Path):
if path.stat().st_size > 4 * 1024 * 1024:
raise ValueError("protocol evidence exceeds bounded document size")
return json.loads(
path.read_text(encoding="utf-8"), object_pairs_hook=_unique_object
)
def executable_cases() -> list[dict]:
"""Single source: retained foundation fixtures plus current protocol catalogs."""
from .protocol_hysteria2_catalog import definitions as hysteria2
from .protocol_integration_catalog import definitions as integration
from .protocol_security_catalog import definitions as security
from .protocol_vless_acceptance import definitions as vless
from .protocol_vmess_acceptance import definitions as vmess
from .protocol_xhttp_acceptance import definitions as xhttp
foundation = read_json(CATALOG_DIR / "foundation-cases.json")["cases"]
return foundation + [
case
for define in (vmess, vless, xhttp, hysteria2, security, integration)
for case in define()
]
def load_manifest(path: Path | None = None) -> list[dict]:
expected = executable_cases()
data = (
read_json(path)
if path is not None
else {
"schema_version": 1,
"kind": "executable-case-manifest",
"cases": expected,
}
)
if (
not isinstance(data, dict)
or type(data.get("schema_version")) is not int
or data.get("schema_version") != 1
or data.get("kind") != "executable-case-manifest"
):
raise ValueError("unsupported executable case manifest")
cases = data.get("cases")
if not isinstance(cases, list) or not cases:
raise ValueError("empty executable case manifest")
seen = set()
for case in cases:
required = {
"case_id",
"stage",
"substage",
"required",
"row_ids",
"protocol",
"network",
"security",
"udp_codec",
"field_values",
"outer_family",
"inner_family",
"target_type",
"upstream_graph",
"peer_kind",
"peer_config",
"gap_source",
"expected_observation",
"required_evidence",
"prerequisites",
"runner",
"timeout_seconds",
}
if not isinstance(case, dict) or not required <= case.keys():
raise ValueError("incomplete executable case metadata")
identifier = case["case_id"]
if (
not isinstance(identifier, str)
or not re.fullmatch(r"[A-Z0-9-]+", identifier)
or identifier in seen
):
raise ValueError("invalid or duplicate executable case ID")
seen.add(identifier)
if (
case["stage"] not in SUITES
or case["substage"] not in CATEGORIES
or not case["substage"].startswith(case["stage"] + ".")
):
raise ValueError("invalid case ownership")
if (
type(case["required"]) is not bool
or not isinstance(case["row_ids"], list)
or len(set(case["row_ids"])) != len(case["row_ids"])
or not set(case["row_ids"]) <= FIELD_IDS
):
raise ValueError("invalid required flag or field references")
if case["peer_kind"] not in {"M", "H", "XR", "V2", "SS", "unit"}:
raise ValueError("unknown peer kind")
for key in ["expected_observation", "required_evidence", "prerequisites"]:
items = case[key]
if (
not isinstance(items, list)
or not items
or not all(isinstance(item, str) and item for item in items)
or len(items) != len(set(items))
):
raise ValueError("empty or duplicate executable observations")
if (
not isinstance(case["field_values"], dict)
or not isinstance(case["peer_config"], dict)
or type(case["timeout_seconds"]) is not int
or not 1 <= case["timeout_seconds"] <= 3600
):
raise ValueError("invalid case bounds or configuration reference")
if {
case["case_id"]
for case in cases
if case["stage"] == "FOUNDATIONS" and case["required"]
} != FOUNDATIONS_REQUIRED_IDS:
raise ValueError("FOUNDATIONS required set was removed, renamed or downgraded")
from .protocol_trojan_acceptance import REQUIRED_IDS
if {
case["case_id"]
for case in cases
if case["stage"] == "TROJAN" and case["required"]
} != REQUIRED_IDS:
raise ValueError("TROJAN required set was removed, renamed or downgraded")
if cases != expected:
raise ValueError("manifest differs from the current executable catalog")
return cases
def check_limit_references(
cases: list[dict], path: Path = CATALOG_DIR / "limits.json"
) -> None:
limits = read_json(path)
identifiers = {case["case_id"] for case in cases}
if (
limits.get("schema_version") != 1
or limits.get("kind") != "runtime-limit-registry"
):
raise ValueError("unsupported runtime limit registry")
entries = limits.get("limits", [])
names = [entry.get("id") for entry in entries]
if not entries or len(names) != len(set(names)):
raise ValueError("empty or duplicate runtime limits")
for entry in entries:
if (
not entry.get("boundary_cases")
or not set(entry["boundary_cases"]) <= identifiers
):
raise ValueError("runtime limit has no executable boundary case")
def select_cases(
cases: list[dict],
*,
stage: str,
identifiers: list[str] | None = None,
protocol: str | None = None,
) -> list[dict]:
if identifiers and (
len(identifiers) != len(set(identifiers))
or not set(identifiers) <= {case["case_id"] for case in cases}
):
raise ValueError("duplicate or unknown requested case")
selected = [
case
for case in cases
if case["stage"] == stage
and case["required"]
and (not identifiers or case["case_id"] in identifiers)
and (protocol is None or case["protocol"] == protocol)
]
if not selected:
raise ValueError("selection has no executable required cases; stage is NOT RUN")
if identifiers and {case["case_id"] for case in selected} != set(identifiers):
raise ValueError("case selection conflicts with stage or protocol")
return selected
def validate_results(required: list[dict], results: list[dict]) -> None:
if not required:
raise ValueError("empty required result set")
expected = {case["case_id"]: case for case in required}
identifiers = [result.get("case_id") for result in results]
if len(identifiers) != len(set(identifiers)) or set(identifiers) != set(expected):
raise ValueError("missing, duplicate or unknown case result")
for result in results:
case = expected[result["case_id"]]
if (
result.get("status") != "PASS"
or result.get("cleanup") is not True
or type(result.get("command_exit_code")) is not int
or result["command_exit_code"] != 0
):
raise ValueError(
"required case failed, was blocked/not run, or did not clean up"
)
observations = result.get("assertions", {})
if (
not isinstance(observations, dict)
or set(observations) != set(case["expected_observation"])
or not all(value is True for value in observations.values())
):
raise ValueError("missing or false required behavioral assertion")
evidence = result.get("evidence", [])
if (
not isinstance(evidence, list)
or len(evidence) != len(set(evidence))
or set(evidence) != set(case["required_evidence"])
):
raise ValueError(
"incomplete behavior evidence; CFG alone is not acceptance"
)
if (
result.get("row_ids") != case["row_ids"]
or result.get("peer_kind") != case["peer_kind"]
or result.get("scope")
!= (
"protocol-consumer"
if case["stage"]
in {
"TROJAN",
"VMESS",
"VLESS",
"XHTTP",
"HYSTERIA2",
"SECURITY",
"INTEGRATION",
}
else "foundation-only"
)
):
raise ValueError("case mapping or evidence scope differs from manifest")
def rust_results(case: dict, events: list[dict], returncode: int) -> dict:
observed = [
event
for event in events
if event.get("suite") == case.get("event_suite", case["case_id"])
]
names = case["expected_observation"]
assertions = {name: False for name in names}
unknown = any(
event.get("assertion") not in assertions or event.get("schema_version") != 1
for event in observed
)
for name in names:
entries = [event for event in observed if event.get("assertion") == name]
assertions[name] = len(entries) == 2 and [
entry.get("status") for entry in entries
] == ["BEGIN", "PASS"]
if assertions[name] and "resources" in case["required_evidence"]:
assertions[name] = idle_resources(entries[-1].get("resources"))
result = new_result(case)
result.update(assertions=assertions, command_exit_code=returncode, cleanup=True)
result["status"] = (
"PASS"
if not unknown and returncode == 0 and all(assertions.values())
else "FAIL"
)
if result["status"] == "PASS":
result["evidence"] = [
item for item in case["required_evidence"] if item != "peer-identity"
]
return result
def new_result(case: dict) -> dict:
return {
"case_id": case["case_id"],
"row_ids": case["row_ids"],
"peer_kind": case["peer_kind"],
"scope": "protocol-consumer"
if case["stage"]
in {"TROJAN", "VMESS", "VLESS", "XHTTP", "HYSTERIA2", "SECURITY", "INTEGRATION"}
else "foundation-only",
"status": "NOT RUN",
"assertions": {},
"evidence": [],
"cleanup": False,
"command_exit_code": None,
}
def artifact(run_dir: Path, relative: str, digest: str) -> Path:
path = run_dir / relative
if (
Path(relative).is_absolute()
or ".." in Path(relative).parts
or path.is_symlink()
or not path.resolve().is_relative_to(run_dir.resolve())
or not HEX.fullmatch(digest)
):
raise ValueError("unsafe evidence path or invalid hash")
with path.open("rb") as stream:
actual = hashlib.file_digest(stream, "sha256").hexdigest()
if actual != digest:
raise ValueError("evidence content hash mismatch")
return path
def check_run(run_dir: Path, stage: str, manifest: Path | None = None) -> None:
if manifest is None and (run_dir / "manifest.json").is_file():
manifest = run_dir / "manifest.json"
cases = load_manifest(manifest)
check_limit_references(cases)
required = select_cases(cases, stage=stage)
run = read_json(run_dir / "run.json")
result = read_json(run_dir / "cases.json")
peers = read_json(run_dir / "peers.json")
if (
run.get("stage") != stage
or run.get("mode") != "execute"
or run.get("cleanup") is not True
or run.get("source_unchanged") is not True
or run.get("selected_cases") != [case["case_id"] for case in required]
):
raise ValueError(
"partial/preflight run or changing source cannot sign off the stage"
)
for key in ["source_tree_sha256", "lock_sha256", "dirty_patch_sha256"]:
if not isinstance(run.get(key), str) or not HEX.fullmatch(run[key]):
raise ValueError("missing immutable input identity")
validate_results(required, result)
artifacts = run.get("artifacts", [])
paths = [entry["path"] for entry in artifacts]
if len(paths) != len(set(paths)):
raise ValueError("duplicate evidence artifact")
if stage in {
"TROJAN",
"VMESS",
"VLESS",
"XHTTP",
"HYSTERIA2",
"SECURITY",
"INTEGRATION",
}:
if stage == "TROJAN":
from .protocol_trojan_acceptance import check
elif stage == "VMESS":
from .protocol_vmess_acceptance import check
elif stage == "VLESS":
from .protocol_vless_acceptance import check
elif stage == "XHTTP":
from .protocol_xhttp_acceptance import check
elif stage == "HYSTERIA2":
from .protocol_hysteria2_acceptance import check
elif stage == "SECURITY":
from .protocol_security_acceptance import check
else:
from .protocol_integration_acceptance import check
for evidence in artifacts:
artifact(run_dir, evidence["path"], evidence["sha256"])
check(run_dir, required, run, result, peers, paths)
return
necessary = {
"cases.json",
"peers.json",
"resources.jsonl",
"summary.md",
"rust-events.jsonl",
"legacy-events.jsonl",
"native/stream-cases.json",
"script-tests.json",
}
if not necessary <= set(paths):
raise ValueError("missing persisted behavior artifacts")
for evidence in artifacts:
artifact(run_dir, evidence["path"], evidence["sha256"])
commands = run.get("commands", [])
command_names = [command.get("name") for command in commands]
required_commands = {
"rust-foundations",
"native-probe-build",
"legacy-mihomo",
"offline-scripts",
"feature-default",
"feature-minimal",
"feature-outbound-trojan",
"feature-outbound-vmess",
"feature-outbound-hysteria2",
"feature-stream-transport",
"feature-quic-transport",
}
if (
len(command_names) != len(set(command_names))
or set(command_names) != required_commands
or any(
type(command.get("exit_code")) is not int
or command["exit_code"] != 0
or command.get("cleanup") is not True
for command in commands
)
):
raise ValueError("missing, failed or unjoined acceptance command")
for command in commands:
if command.get("log") and command["log"] not in paths:
raise ValueError("command log has no content identity")
rust_events = read_events(run_dir / "rust-events.jsonl")
legacy_events = read_events(run_dir / "legacy-events.jsonl")
native = read_json(run_dir / "native/stream-cases.json")
native_cases = [case for case in required if case["runner"] == "native-stream"]
if [item.get("case_id") for item in native.get("cases", [])] != [
case["case_id"] for case in native_cases
]:
raise ValueError("missing, duplicate or unknown native case evidence")
# Use exactly the same structured evaluators as execution, never console text.
from .protocol_harness import SCRIPT_OBSERVATIONS, _native_results
recalculated = _native_results(native_cases, native)
for case in required:
if case["runner"] in {"rust", "mihomo-legacy"}:
calculated = rust_results(
case,
legacy_events if case["runner"] == "mihomo-legacy" else rust_events,
0,
)
if case["runner"] == "mihomo-legacy" and calculated["status"] == "PASS":
calculated["evidence"] = case["required_evidence"]
recalculated.append(calculated)
actual = {item["case_id"]: item for item in result}
for calculated in recalculated:
for key in ["status", "assertions", "evidence", "cleanup", "command_exit_code"]:
if actual[calculated["case_id"]].get(key) != calculated[key]:
raise ValueError(
"case result disagrees with original structured evidence"
)
script = read_json(run_dir / "script-tests.json")
scripts = script.get("cases", [])
tests = {entry["test"]: entry["status"] for entry in scripts}
if (
not scripts
or len(tests) != len(scripts)
or len(tests) != script.get("tests_run")
or any(status != "PASS" for status in tests.values())
or any(
tests.get(identifier) != "PASS"
for identifier in SCRIPT_OBSERVATIONS.values()
)
):
raise ValueError("missing or failed offline harness failure-path proof")
resources = read_events(run_dir / "resources.jsonl")
if not resources or not all(
idle_resources(event.get("resources")) for event in resources
):
raise ValueError("missing, nonidle or malformed owned resource evidence")
resource_passes = [
event
for event in rust_events
if event.get("suite") == "FOUNDATIONS-RESOURCES"
and event.get("status") == "PASS"
]
for event in resource_passes:
checkpoints = event.get("checkpoints", [])
by_phase = {point["phase"]: point["resources"] for point in checkpoints}
if (
len(checkpoints) != len(by_phase)
or not {"baseline", "after-stop"} <= by_phase.keys()
or not idle_resources(by_phase["baseline"])
or not idle_resources(by_phase["after-stop"])
):
raise ValueError("resource ownership has no baseline/Stop evidence")
if "remain_quiet" in event["assertion"] and (
by_phase.get("quiet") != by_phase["after-stop"]
or event.get("seconds", 0) < 5
):
raise ValueError("resource quiet-window evidence missing")
for case in required:
if "peer-identity" in case["required_evidence"]:
matches = [
peer
for peer in peers
if peer.get("kind") == case["peer_kind"]
and peer.get("status") == "READY"
]
if (
len(matches) != 1
or not matches[0].get("version")
or not HEX.fullmatch(matches[0].get("binary_sha256", ""))
or not HEX.fullmatch(matches[0].get("archive_sha256", ""))
or not matches[0]
.get("source_url", "")
.startswith("https://github.com/")
):
raise ValueError("missing official peer identity")
if case["runner"] == "native-stream":
native_peers = [
peer
for peer in native.get("peers", [])
if peer.get("kind") == case["peer_kind"]
]
if (
len(native_peers) != 1
or native_peers[0].get("binary_sha256")
!= matches[0]["binary_sha256"]
):
raise ValueError("case used a different peer from preflight")
if case["runner"] == "mihomo-legacy":
peer = matches[0]
if (
peer.get("container_status") != "READY"
or not HEX.fullmatch(
peer.get("container_artifact", {}).get("binary_sha256", "")
)
or peer["container_artifact"].get("release") != peer.get("release")
):
raise ValueError(
"legacy peer did not use the same official release snapshot"
)
print(
f"{stage}: PASS ({len(required)} required foundation cases); "
"production field consumers remain NOT RUN"
)
@@ -0,0 +1,177 @@
"""Named-profile VCore integration, reusing the owned container wire harness."""
import argparse
import hashlib
import json
import sys
import time
from datetime import UTC, datetime
from pathlib import Path
from .mihomo_isolation import exclusive_run
from .protocol_inputs import same_source, source_identity
from .protocol_vless_container import CLIENT_FINGERPRINTS, run
SELECTED_TIMEOUT_SECONDS = 3600
CASES = [
"FINGERPRINT-ANYTLS",
*[
f"VLESS-REGRESSION-{protocol}-{network}"
for protocol in ("TROJAN", "VMESS")
for network in ("TCP", "WS", "GRPC")
],
*[
f"VLESS-{mode}-BASE"
for mode in (
"TCP-TLS",
"WS-TLS",
"WS-ED-1-TLS",
"WS-ED-2048-TLS",
"WS-HEADER-TLS",
"WS-PATH-TLS",
"UPGRADE-TLS",
"UPGRADE-FAST-TLS",
"HTTP-TLS",
"H2-TLS",
"GRPC-TLS",
"TCP-REALITY",
"WS-REALITY",
"UPGRADE-REALITY",
"UPGRADE-FAST-REALITY",
"GRPC-REALITY",
"XHTTP-STREAM-ONE",
"XHTTP-STREAM-UP",
"XHTTP-PACKET-UP",
"XHTTP-STREAM-ONE-REALITY",
"XHTTP-STREAM-UP-DOWNLOAD-REALITY",
"VISION-TLS",
"VISION-REALITY",
)
],
"VLESS-VISION-TLS-INNER-TLS",
"FINGERPRINT-VMESS-HTTP-TLS",
"FINGERPRINT-VMESS-H2-TLS",
"VLESS-VISION-REALITY-INNER-TLS",
"VLESS-TCP-REALITY-NEG",
"VLESS-TCP-TLS-NEG",
"VLESS-GRPC-MTLS-IDENTITY",
"VLESS-WS-ALPN-TLS-NEG",
"VLESS-TCP-TLS-CLOSE",
"VLESS-TCP-REALITY-CLOSE",
"VLESS-GRPC-REALITY-CLOSE",
"VLESS-VISION-REALITY-CLOSE",
"VLESS-GRPC-TLS-LIFE",
"VLESS-GRPC-TLS-OWNED",
"VLESS-VISION-REALITY-LIFE",
"VLESS-VISION-REALITY-OWNED",
]
def run_selected(output, profile):
from .builds import CORE_DIR
from .protocol_xhttp_fields import run as run_xhttp
from .protocol_xhttp_fields import selected_profile_variants
if (
profile not in CLIENT_FINGERPRINTS
or output.parent != CORE_DIR / "target/interop/runs"
):
raise ValueError("use a supported profile and fresh target/interop/runs child")
output.mkdir(parents=True, exist_ok=False)
started = time.monotonic()
report = dict(
stage="FINGERPRINT-INTEROP",
scope="selected-transports",
profile=profile,
source=source_identity(),
status="NOT RUN",
parts=[],
started_utc=datetime.now(UTC).isoformat(),
timeout_seconds=SELECTED_TIMEOUT_SECONDS,
)
try:
for phase in ("transports", "xhttp"):
if phase == "transports":
result = run(output / phase, CASES, client_fingerprint=profile)
path = output / phase / "vless-results.json"
else:
selected = [
f"{version}-{mode}-headers"
for version in ("h1", "h2")
for mode in ("stream-one", "stream-up", "packet-up")
]
selected.extend(selected_profile_variants(profile))
jobs = {
name: [
dict(
case_id=f"FINGERPRINT-{profile}-{name}",
test="security::native_xhttp_security"
if name.endswith("reject-pin")
else "public_base",
)
]
for name in selected
}
result = run_xhttp(
output / phase, selected, jobs=jobs, client_fingerprint=profile
)
path = output / phase / "xhttp-fields-results.json"
part = json.loads(path.read_text())
report["parts"].append(
dict(
phase=phase,
report=str(path.relative_to(output)),
sha256=hashlib.sha256(path.read_bytes()).hexdigest(),
status=part["status"],
cleanup=part["cleanup"],
source_unchanged=part["source_unchanged"],
cases=len(part["cases"]),
)
)
if result:
raise RuntimeError(f"{phase} gate failed")
report["status"] = "PASS"
except (OSError, ValueError, RuntimeError) as error:
report.update(status="FAIL", reason=str(error))
finally:
report["finished_utc"] = datetime.now(UTC).isoformat()
report["seconds"] = round(time.monotonic() - started, 3)
report["source_after"] = source_identity()
# Staging an already captured untracked file changes the diagnostic
# Git patch, not the exhaustive path/content source-tree digest.
report["source_unchanged"] = same_source(
report["source"], report["source_after"]
)
if not report["source_unchanged"]:
report["status"] = "FAIL"
if report["seconds"] > SELECTED_TIMEOUT_SECONDS:
report.update(
status="FAIL", reason="selected transport suite deadline exceeded"
)
(output / "fingerprint-results.json").write_text(
json.dumps(report, indent=2) + "\n"
)
return 0 if report["status"] == "PASS" else 1
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
parser.add_argument("cases", nargs="*")
parser.add_argument(
"--client-fingerprint", choices=CLIENT_FINGERPRINTS, default="chrome120"
)
args = parser.parse_args(argv)
with exclusive_run():
if not args.cases:
return run_selected(args.output.resolve(), args.client_fingerprint)
return run(
args.output.resolve(),
args.cases or CASES,
client_fingerprint=args.client_fingerprint,
)
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,551 @@
"""Selected-v1 reference ClientHello acquisition; not VCore interoperability."""
from __future__ import annotations
import argparse
import base64
import contextlib
import copy
import json
import secrets
import shutil
import signal
import sys
import tempfile
import time
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .protocol_containers import ContainerLab, command, listing
from .protocol_fixtures import certificates
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .tls_client_hello import is_grease, parse_client_hello, validate_capture
PROFILES = ("chrome", "chrome120", "firefox", "firefox120", "safari", "safari16")
TEMPLATES = {
"chrome": "Chrome133",
"chrome120": "Chrome120",
"firefox": "Firefox120",
"firefox120": "Firefox120",
"safari": "Safari16.0",
"safari16": "Safari16.0",
}
SNIS = ("fingerprint.test", "f" * 63 + ".fingerprint.test")
PORTS = {
"tcp": 24430,
"ws": 24430,
"grpc": 24430,
"reality": 24430,
"tls13": 24431,
"tls12": 24432,
}
def reference_cases() -> list[dict]:
"""The public reference-case catalog; a listing does not start a peer."""
cases = []
for profile in (*PROFILES, "none"):
contexts = ("tcp",) if profile == "none" else ("tcp", "ws", "grpc", "reality")
for context in contexts:
for sni in SNIS:
for attempt in range(2):
cases.append(
dict(
id=f"FINGERPRINT-REFERENCE-{profile}-{context}-"
f"{len(sni)}-{attempt}",
profile=profile,
template=TEMPLATES.get(profile),
context=context,
sni=sni,
sni_length=len(sni),
attempt=attempt,
)
)
for profile in ("chrome", "chrome120", "firefox", "safari"):
for context in ("tls12", "tls13"):
for attempt in range(2):
cases.append(
dict(
id=f"FINGERPRINT-REFERENCE-{profile}-{context}-16-{attempt}",
profile=profile,
template=TEMPLATES[profile],
context=context,
sni=SNIS[0],
sni_length=16,
attempt=attempt,
)
)
return cases
def _reference_shape(hello, case, *, expected_alpn=None, allow_psk=False):
"""Apply only source-defined reference variations, checking before masking."""
shape = copy.deepcopy(hello)
shape.pop("raw_sha256")
size = shape.pop("bytes")
padding = next((e for e in shape["extensions"] if e["type"] == 21), None)
psk = next((e for e in shape["extensions"] if e["type"] == 41), None)
if psk and (not allow_psk or shape["extensions"][-1] != psk):
raise ValueError("unexpected PSK or invalid last-extension position")
unpadded = size - (padding["bytes"] + 4 if padding else 0)
needs_padding = (
case["template"] in {"Chrome120", "Safari16.0"} and 255 < unpadded < 512
)
if bool(padding) != needs_padding:
raise ValueError("reference padding presence differs from template")
if padding:
expected = max(1, 512 - unpadded - 4)
if (
padding["bytes"] != expected
or padding["payload_hex"] != "00" * expected
or shape["extensions"][-2 if psk else -1] != padding
):
raise ValueError("reference padding contents, position or size mismatch")
shape["extensions"].remove(padding)
groups = next(e["values"] for e in shape["extensions"] if e["type"] == 10)
shares = next(e["shares"] for e in shape["extensions"] if e["type"] == 51)
if any(s["group"] not in groups for s in shares):
raise ValueError("reference key share/group correlation mismatch")
for row in shape["extensions"]:
if row["type"] == 0:
if row["names"] != [dict(type=0, name=case["sni"])]:
raise ValueError("reference SNI mismatch")
row["names"], row["bytes"] = "synthetic-sni", "sni-dependent"
elif row["type"] == 16:
expected = expected_alpn or (
["http/1.1"] if case["context"] == "ws" else ["h2", "http/1.1"]
)
if row["protocols"] != expected:
raise ValueError("reference transport ALPN mismatch")
row["protocols"], row["bytes"] = "transport-alpn", "alpn-dependent"
elif row["type"] == 65037:
firefox = case["template"] == "Firefox120"
if (
(row["hello_type"], row["kdf"], row["enc_bytes"]) != (0, 1, 32)
or row["aead"] not in ({1, 3} if firefox else {1})
or row["payload_bytes"]
not in ({239} if firefox else {144, 176, 208, 240})
or row["bytes"] != 42 + row["payload_bytes"]
):
raise ValueError(
"reference ECH GREASE differs from source-defined variants"
)
row["aead"], row["payload_bytes"], row["bytes"] = (
"checked-variant",
"checked-variant",
"checked-variant",
)
for key in ("values",):
if key in row:
row[key] = ["GREASE" if is_grease(v) else v for v in row[key]]
if row["type"] == 51:
for share in row["shares"]:
if is_grease(share["group"]):
share["group"] = "GREASE"
if is_grease(row["type"]):
row["type"] = "GREASE"
shape["ciphers"] = ["GREASE" if is_grease(v) else v for v in shape["ciphers"]]
if case["template"].startswith("Chrome"):
rows = shape["extensions"]
if rows[0] != dict(type="GREASE", bytes=0, payload_hex="") or rows[
-2 if psk else -1
] != dict(type="GREASE", bytes=1, payload_hex="00"):
raise ValueError("reference Chrome GREASE positions changed")
shape["extensions"] = [
rows[0],
*sorted(rows[1 : -2 if psk else -1], key=lambda e: e["type"]),
*rows[-2 if psk else -1 :],
]
return shape
def check_reference_report(report: dict) -> dict:
"""Verify reference scope, raw observations and source-defined variations."""
if (
report.get("scope") != "selected-v1"
or report.get("status") != "CAPTURED"
or report.get("selection") != "full"
or report.get("cleanup") is not True
or report.get("source_unchanged") is not True
):
raise ValueError("incomplete or unclean selected-profile reference run")
if report.get("source") != report.get("source_after"):
raise ValueError("reference source changed during capture")
catalog = {case["id"]: case for case in reference_cases()}
cases = report["cases"]
if len(cases) != len(catalog) or {c["id"] for c in cases} != catalog.keys():
raise ValueError("missing or duplicate reference cases")
fixtures = json.loads(
(CORE_DIR / "tests/fingerprints/mihomo-selected-v1.json").read_text()
)
baselines = {}
for sample in fixtures["samples"]:
case = catalog[sample["case_id"]]
baselines[(sample["profile"], sample["context"])] = _reference_shape(
validate_capture(sample), case
)
observed, tls_handshakes = 0, 0
for row in cases:
case = catalog[row["id"]]
if (
any(row.get(key) != value for key, value in case.items())
or row["status"] != "CAPTURED"
or not 1 <= len(row["observations"]) <= 4
):
raise ValueError("reference case metadata or observations changed")
for event in row["observations"]:
hello = validate_capture(event)
expected = (
"tls-handshake-complete"
if case["context"] in {"tls12", "tls13"}
else "captured-only"
)
if event["status"] != expected or event["port"] != PORTS[case["context"]]:
raise ValueError("reference observation has wrong outcome")
if expected == "tls-handshake-complete":
version = "TLSv1.2" if case["context"] == "tls12" else "TLSv1.3"
if event["version"] != version:
raise ValueError("reference negotiated wrong TLS version")
tls_handshakes += 1
if case["profile"] != "none":
profile = {"firefox120": "firefox", "safari16": "safari"}.get(
case["profile"], case["profile"]
)
context = "reality" if case["context"] == "reality" else "tcp"
if _reference_shape(hello, case) != baselines[(profile, context)]:
raise ValueError(f"reference structure changed: {case['id']}")
observed += 1
return dict(
scope="selected-v1",
stage="FINGERPRINT-REFERENCE",
status="BASELINE VERIFIED",
cases=len(cases),
observations=observed,
tls_only_handshakes=tls_handshakes,
business_interoperability="NOT TESTED",
production_support="UNCHANGED",
)
def _name(case):
return f"{case['profile']}-{case['context']}-{case['sni_length']}"
def _node(case, server):
context = case["context"]
node = dict(
name=_name(case),
type="vless",
server=server,
port=PORTS[context],
uuid="07070707-0707-0707-0707-070707070707",
tls=True,
servername=case["sni"],
network=context if context in {"ws", "grpc"} else "tcp",
alpn=["h2", "http/1.1"],
**{"client-fingerprint": case["profile"]},
)
if context == "reality":
# Public RFC 7748 test vector; observer never authenticates REALITY.
public = bytes.fromhex(
"de9edb7d7b7dc1b4d35b61c2ece435373f8343c85b78674dadfc7e146f882b4f"
)
node["reality-opts"] = {
"public-key": base64.urlsafe_b64encode(public).decode().rstrip("="),
"short-id": "01020304",
}
else:
node["skip-cert-verify"] = True
if context == "ws":
node["ws-opts"] = {"path": "/capture"}
elif context == "grpc":
node["grpc-opts"] = {"grpc-service-name": "capture"}
return node
def _probe(client, case, target, secret):
query = urllib.parse.urlencode(dict(url=f"http://{target}:80/", timeout=3000))
request = urllib.request.Request(
f"http://{client}:25000/proxies/{_name(case)}/delay?{query}",
headers={"Authorization": "Bearer " + secret},
)
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
try:
response = opener.open(request, timeout=5)
except urllib.error.HTTPError as error:
response = error
with response:
body = response.read(4097)
if len(body) > 4096:
raise ValueError("reference controller response limit")
# This observer cannot serve VLESS or a URL: delay failure is expected.
return dict(delay_http_status=response.status, business_result="NOT TESTED")
def _events(observer, after):
query = (
"import json,pathlib,sys; p=pathlib.Path('/data/events.jsonl'); "
"assert p.stat().st_size <= 8*1024*1024; "
"print(json.dumps(p.read_text().splitlines()[int(sys.argv[1]):]))"
)
lines = json.loads(
command("exec", observer.name, "python", "-c", query, str(after))
)
return [json.loads(line) for line in lines]
def run_reference(output: Path, selected=None) -> int:
catalog = {case["id"]: case for case in reference_cases()}
ids = list(catalog) if selected is None else selected
if not ids or len(set(ids)) != len(ids) or not set(ids) <= catalog.keys():
raise ValueError("invalid reference capture selection")
output = output.resolve()
if output.parent != CORE_DIR / "target/interop/runs" or output.exists():
raise ValueError("fresh in-repository reference run directory required")
output.mkdir(parents=True)
report = dict(
scope="selected-v1",
stage="FINGERPRINT-REFERENCE",
kind="reference-clienthello-only",
status="NOT RUN",
production_support="UNCHANGED",
business_interoperability="NOT TESTED",
selection="full" if set(ids) == catalog.keys() else "partial",
source=source_identity(),
cases=[],
cleanup=False,
source_unchanged=False,
)
lab = None
try:
with exclusive_run(), contextlib.ExitStack() as stack:
identity = {}
binary = download_mihomo(
"linux-arm64", directory=output / "official-mihomo", identity=identity
)
report["mihomo"] = identity
build = run_command(["go", "version", "-m", str(binary)], timeout=10)
if build.returncode or not build.cleanup:
raise RuntimeError("official binary build identity query failed")
(output / "mihomo-build-info.txt").write_bytes(build.stdout)
report["utls_build_info"] = [
line.strip()
for line in build.stdout.decode().splitlines()
if "utls" in line or "vcs.revision" in line
]
if not any(
"github.com/metacubex/utls\tv1.8.7\t" in line
for line in report["utls_build_info"]
):
raise RuntimeError(
"reference uTLS revision changed; recheck selected template mapping"
)
root = Path(
stack.enter_context(
tempfile.TemporaryDirectory(prefix="private-", dir=output)
)
)
observer_root, client_root = root / "observer", root / "client"
observer_root.mkdir()
client_root.mkdir()
certificates(observer_root)
shutil.copy2(
Path(__file__).with_name("container_tls_observer.py"),
observer_root / "observer.py",
)
shutil.copy2(binary, client_root / "mihomo")
lab = ContainerLab(report.setdefault("isolation", {}), mtu=1500)
observer = lab.start(
stack,
observer_root,
"fingerprint-reference-observer",
[
"env",
"VCORE_ISOLATED_TLS_OBSERVER=1",
"python",
"-B",
"-u",
"/data/fixture/observer.py",
],
)
observer.release()
for port in sorted(set(PORTS.values())):
observer.wait_tcp(port)
secret = secrets.token_hex(24)
nodes = {
_name(catalog[case]): _node(catalog[case], observer.ipv4)
for case in ids
}
config = dict(
proxies=list(nodes.values()),
rules=["MATCH,REJECT"],
secret=secret,
**{"log-level": "warning", "external-controller": "0.0.0.0:25000"},
)
(client_root / "config.json").write_text(json.dumps(config))
client = lab.start(
stack,
client_root,
"fingerprint-reference-mihomo",
[
"/data/fixture/mihomo",
"-d",
"/data/mihomo",
"-f",
"/data/fixture/config.json",
],
)
def retain_logs():
for role, peer in (("observer", observer), ("client", client)):
if peer.log.exists():
text = peer.log.read_text(errors="replace")[-16384:]
for value in (secret, observer.ipv4, client.ipv4):
text = text.replace(value, "<fixture>")
(output / f"{role}.log").write_text(redact(text))
stack.callback(retain_logs)
identity["version"] = command(
"exec", client.name, "/data/fixture/mihomo", "-v"
).strip()
digest = command(
"exec", client.name, "sha256sum", "/data/fixture/mihomo"
).split()[0]
if digest != identity["binary_sha256"]:
raise RuntimeError("container official binary identity mismatch")
report["observer_tls"] = command(
"exec",
observer.name,
"python",
"-c",
"import ssl; print(ssl.OPENSSL_VERSION)",
).strip()
client.release()
client.wait_tcp(25000)
observed_count = 0
for case_id in ids:
case = catalog[case_id]
row = dict(case, status="NOT RUN", observations=[])
report["cases"].append(row)
row["controller"] = _probe(client.ipv4, case, observer.ipv4, secret)
deadline = time.monotonic() + 6
while True:
events = _events(observer, observed_count)
if events or time.monotonic() >= deadline:
break
time.sleep(0.05)
if not 1 <= len(events) <= 4:
raise RuntimeError(
f"reference observation count mismatch: {case_id}"
)
observed_count += len(events)
expected = (
"tls-handshake-complete"
if case["context"] in {"tls12", "tls13"}
else "captured-only"
)
for event in events:
row["observations"].append(event)
if "client_hello_b64" not in event:
raise RuntimeError(
f"reference observer did not capture a hello: {case_id}"
)
hello = parse_client_hello(
base64.b64decode(event["client_hello_b64"], validate=True)
)
event["parsed"] = hello
sni = next(
(e["names"] for e in hello["extensions"] if e["type"] == 0), []
)
if (
event["status"] != expected
or event["port"] != PORTS[case["context"]]
or sni != [dict(type=0, name=case["sni"])]
):
raise RuntimeError(
f"reference TLS observation mismatch: {case_id}"
)
row["status"] = "CAPTURED"
print(
json.dumps(
dict(
case_id=case_id,
status=row["status"],
observations=len(events),
)
),
flush=True,
)
client.ensure_alive()
observer.ensure_alive()
report["status"] = "CAPTURED"
except BaseException as error:
report["status"] = "FAIL"
report["error"] = redact(f"{type(error).__name__}: {error}")
raise
finally:
report["source_after"] = source_identity()
report["source_unchanged"] = report["source_after"] == report["source"]
if lab is not None:
report["cleanup"] = not any(
item["configuration"].get("labels", {}).get("vcore-run") == lab.run_id
for item in listing()
) and all(peer.get("joined") for peer in report["isolation"]["peers"])
if report["status"] == "CAPTURED" and not (
report["source_unchanged"] and report["cleanup"]
):
report["status"] = "FAIL"
(output / "reference-results.json").write_text(
json.dumps(report, indent=2) + "\n"
)
print(
json.dumps(
{key: report[key] for key in ("status", "cleanup", "source_unchanged")}
),
flush=True,
)
return 0 if report["status"] == "CAPTURED" else 1
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--list", action="store_true")
mode.add_argument("--run-dir", type=Path)
mode.add_argument("--check-run", type=Path)
parser.add_argument("--case", action="append")
args = parser.parse_args(argv)
if args.check_run:
if args.case:
parser.error("--case requires --run-dir")
report = args.check_run / "reference-results.json"
if report.stat().st_size > 8 * 1024 * 1024:
raise ValueError("reference report size limit")
print(
json.dumps(check_reference_report(json.loads(report.read_text())), indent=2)
)
return 0
if args.list:
if args.case:
parser.error("--case requires --run-dir")
print(json.dumps(reference_cases(), indent=2))
return 0
def stop_signal(_signum, _frame):
raise KeyboardInterrupt()
previous = signal.signal(signal.SIGTERM, stop_signal)
try:
return run_reference(args.run_dir, args.case)
finally:
signal.signal(signal.SIGTERM, previous)
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,259 @@
"""Public-config TLS wire checks over memory IO; no network observer."""
from __future__ import annotations
import argparse
import base64
import copy
import io
import json
import os
from pathlib import Path
from .builds import CORE_DIR
from .container_tls_observer import capture_client_hello
from .protocol_fingerprint_reference import PROFILES, SNIS, TEMPLATES, _reference_shape
from .protocol_inputs import redact, same_source, source_identity
from .protocol_peers import run_command
from .tls_client_hello import parse_client_hello, validate_capture
CONTEXTS = ("tcp", "ws", "grpc", "reality", "vision", "xhttp-h1", "xhttp-h2")
def expected_shape(sample, context):
"""Transform an independent baseline only for documented caller policies."""
case = dict(template=sample["template"], sni=SNIS[0], context=sample["context"])
shape = _reference_shape(validate_capture(sample), case)
tls13 = context in {"vision", "xhttp-h1", "xhttp-h2"}
if tls13:
# Standard Vision/XHTTP require TLS1.3 before authentication. These
# TLS1.2-only fields are omitted by native version policy, not hidden
# from actual captures. REALITY instead enforces TLS1.3 natively while
# retaining the browser offer, exactly as the reference does.
shape["ciphers"] = [
c for c in shape["ciphers"] if c in {"GREASE", 4865, 4866, 4867}
]
shape["extensions"] = [
e for e in shape["extensions"] if e["type"] not in {11, 23, 35, 65281}
]
if context in {"ws", "xhttp-h1"}:
shape["extensions"] = [
e for e in shape["extensions"] if e["type"] not in {17513, 17613}
]
for extension in shape["extensions"]:
if extension["type"] == 43:
extension["values"] = [
v
for v in extension["values"]
if v == "GREASE" or v >= (772 if tls13 else 771)
]
extension["bytes"] = 1 + 2 * len(extension["values"])
return shape
def check_captures(rows):
wanted = {
(p, c, s, a) for p in PROFILES for c in CONTEXTS for s in SNIS for a in range(2)
}
actual = [(r["profile"], r["context"], r["sni"], r["attempt"]) for r in rows]
if len(actual) != len(wanted) or set(actual) != wanted:
raise ValueError("incomplete or duplicate public-config wire matrix")
samples = json.loads(
(CORE_DIR / "tests/fingerprints/mihomo-selected-v1.json").read_text()
)["samples"]
baselines = {(s["profile"], s["context"]): s for s in samples}
results = []
for row in rows:
profile, context = row["profile"], row["context"]
canonical = {"firefox120": "firefox", "safari16": "safari"}.get(
profile, profile
)
reference = baselines[(canonical, "reality" if context == "reality" else "tcp")]
encoded = row["records_b64"]
if len(encoded) > 128 * 1024:
raise ValueError("wire capture size limit")
wire = base64.b64decode(encoded, validate=True)
records, raw, layout = capture_client_hello(io.BytesIO(wire).read)
if records != wire or any(
r["type"] != 22 or r["version"] != 769 for r in layout
):
raise ValueError("unexpected initial TLS record envelope")
hello = parse_client_hello(raw)
alpn = (
["http/1.1"]
if context in {"ws", "xhttp-h1"}
else ["h2"]
if context == "xhttp-h2"
else ["h2", "http/1.1"]
)
shape = _reference_shape(
hello, dict(row, template=TEMPLATES[profile]), expected_alpn=alpn
)
expected = expected_shape(reference, context)
if shape != expected:
differences = {
k: dict(expected=expected.get(k), observed=shape.get(k))
for k in expected.keys() | shape.keys()
if expected.get(k) != shape.get(k)
}
raise ValueError(
f"{profile}/{context}/{len(row['sni'])}/{row['attempt']}: "
f"{json.dumps(differences)}"
)
results.append(
dict(
copy.deepcopy(row),
client_hello_b64=base64.b64encode(raw).decode(),
records=layout,
parsed=hello,
status="SHAPE VERIFIED",
)
)
return results
def check_warm_captures(rows):
wanted = {
(p, v, phase)
for p in ("chrome", "chrome120", "firefox", "safari")
for v in ("TLSv1_2", "TLSv1_3")
for phase in ("cold", "warm", "after-hint")
}
actual = [(r["profile"], r["version"], r["phase"]) for r in rows]
if len(actual) != len(wanted) or set(actual) != wanted:
raise ValueError("incomplete or duplicate warm/expiry matrix")
samples = json.loads(
(CORE_DIR / "tests/fingerprints/mihomo-selected-v1.json").read_text()
)["samples"]
baselines = {s["profile"]: s for s in samples if s["context"] == "tcp"}
results = []
for row in rows:
profile, version, phase = row["profile"], row["version"], row["phase"]
stateful_id = profile == "safari" and version == "TLSv1_2"
resumed = phase == "warm" or (phase == "after-hint" and stateful_id)
if row["resumed"] != resumed:
raise ValueError("unexpected real peer resumption/expiry outcome")
wire = base64.b64decode(row["records_b64"], validate=True)
records, raw, layout = capture_client_hello(io.BytesIO(wire).read)
if wire[len(records) :] not in (b"", b"\x14\x03\x03\x00\x01\x01"):
raise ValueError("unexpected warm first flight beyond ClientHello/CCS")
hello = parse_client_hello(raw)
shape = _reference_shape(
hello,
dict(template=TEMPLATES[profile], sni="fixture.invalid", context="tcp"),
allow_psk=True,
)
expected = expected_shape(baselines[profile], "tcp")
if resumed and not stateful_id:
length = row["prior_ticket_bytes"]
if not isinstance(length, int) or not 1 <= length <= 4096:
raise ValueError("independent peer did not issue a bounded ticket")
if version == "TLSv1_3":
expected["extensions"].append(
dict(type=41, bytes=length + 43, identities=[length], binders=[32])
)
else:
ticket = next(e for e in expected["extensions"] if e["type"] == 35)
ticket.update(bytes=length, ticket_bytes=length)
if shape != expected:
differences = {
k: dict(expected=expected.get(k), observed=shape.get(k))
for k in expected.keys() | shape.keys()
if expected.get(k) != shape.get(k)
}
raise ValueError(
f"warm {profile}/{version}/{phase}: {json.dumps(differences)}"
)
results.append(
dict(
row,
records_b64=base64.b64encode(records).decode(),
client_hello_b64=base64.b64encode(raw).decode(),
records=layout,
parsed=hello,
status="SHAPE VERIFIED",
)
)
return results
def run(output):
if output.parent != CORE_DIR / "target/interop/runs":
raise ValueError("use a fresh directory directly under target/interop/runs")
output.mkdir(exist_ok=False)
report = dict(
stage="FINGERPRINT-WIRE",
scope="memory-clienthello-structure",
source=source_identity(),
status="NOT RUN",
)
try:
result = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"fingerprint_capture",
"--",
"--nocapture",
],
timeout=180,
env=dict(
os.environ, VCORE_FINGERPRINT_CAPTURE=str(output / "captures.json")
),
)
(output / "command.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
report["cleanup"] = result.cleanup
if result.returncode or not result.cleanup:
raise RuntimeError("memory capture command failed")
report["cases"] = check_captures(
json.loads((output / "captures.json").read_text())
)
warm = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--lib",
"security::tls::tests::named_wire",
"--",
"--nocapture",
],
timeout=180,
env=dict(
os.environ, VCORE_FINGERPRINT_WARM_CAPTURE=str(output / "warm.json")
),
)
(output / "warm-command.log").write_text(
redact(warm.stdout.decode(errors="replace"))
)
report["cleanup"] = report["cleanup"] and warm.cleanup
if warm.returncode or not warm.cleanup:
raise RuntimeError("warm/expiry/version gate failed")
report["warm_cases"] = check_warm_captures(
json.loads((output / "warm.json").read_text())
)
report["status"] = "PASS"
except Exception as error:
report.update(status="FAIL", error=redact(str(error)))
finally:
report["source_after"] = source_identity()
report["source_unchanged"] = same_source(
report["source"], report["source_after"]
)
if not report["source_unchanged"]:
report["status"] = "FAIL"
(output / "shape-results.json").write_text(json.dumps(report, indent=2) + "\n")
print(report["status"], report.get("error", ""))
return 0 if report["status"] == "PASS" else 1
if __name__ == "__main__":
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
raise SystemExit(run(parser.parse_args().output.resolve()))
@@ -0,0 +1,181 @@
"""Shared synthetic certificates and peer configuration; never starts a listener."""
from __future__ import annotations
import hashlib
from pathlib import Path
from .protocol_peers import run_command
def certificates(directory: Path):
cert, key = directory / "cert.pem", directory / "key.pem"
generated = run_command(
[
"openssl",
"req",
"-x509",
"-newkey",
"rsa:2048",
"-nodes",
"-days",
"2",
"-subj",
"/CN=localhost",
"-addext",
"subjectAltName=DNS:localhost",
"-keyout",
str(key),
"-out",
str(cert),
],
timeout=20,
limit=65536,
)
der = run_command(
["openssl", "x509", "-in", str(cert), "-outform", "DER"],
timeout=10,
limit=65536,
)
if (
generated.returncode != 0
or der.returncode != 0
or not generated.cleanup
or not der.cleanup
):
raise RuntimeError("synthetic certificate generation failed")
return cert, key, hashlib.sha256(der.stdout).hexdigest()
def certificate_chain(directory):
"""Owned synthetic CA + leaf; root pin still verifies the leaf name."""
root = directory / "root.pem"
root_key = directory / "root-key.pem"
key = directory / "key.pem"
csr = directory / "leaf.csr"
cert = directory / "cert.pem"
extensions = directory / "extensions.cnf"
extensions.write_text(
"basicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth\nsubjectAltName=DNS:localhost\n"
)
commands = [
[
"openssl",
"req",
"-x509",
"-newkey",
"rsa:2048",
"-nodes",
"-days",
"2",
"-subj",
"/CN=trojan-root",
"-addext",
"basicConstraints=critical,CA:TRUE",
"-keyout",
str(root_key),
"-out",
str(root),
],
[
"openssl",
"req",
"-new",
"-newkey",
"rsa:2048",
"-nodes",
"-subj",
"/CN=localhost",
"-keyout",
str(key),
"-out",
str(csr),
],
[
"openssl",
"x509",
"-req",
"-in",
str(csr),
"-CA",
str(root),
"-CAkey",
str(root_key),
"-set_serial",
"2",
"-days",
"2",
"-extfile",
str(extensions),
"-out",
str(cert),
],
["openssl", "x509", "-in", str(root), "-outform", "DER"],
]
for command in commands:
result = run_command(command, timeout=20, limit=65536)
if result.returncode or not result.cleanup:
raise RuntimeError("synthetic chain generation failed")
cert.write_bytes(cert.read_bytes() + root.read_bytes())
return cert, key, hashlib.sha256(result.stdout).hexdigest()
def trojan_peer_config(kind, mode, port, password, cert, key):
host = "::1" if mode.endswith("-ipv6") else "127.0.0.1"
mode = mode.removesuffix("-ipv6").removesuffix("-ca")
if kind == "M":
listener = {
"name": "trojan",
"type": "trojan",
"listen": host,
"port": port,
"users": [{"username": "fixture", "password": password}],
"certificate": str(cert),
"private-key": str(key),
}
if mode.startswith("ws"):
listener["ws-path"] = "cover.example/trojan-ws"
if mode.startswith("grpc"):
listener["grpc-service-name"] = "trojan-grpc"
if mode == "ws-alpn":
listener["grpc-service-name"] = "unrelated-service"
return {
"mode": "rule",
"log-level": "silent",
"ipv6": True,
"hosts": {"vcore-fixture.test": "127.0.0.1"},
"listeners": [listener],
"rules": ["MATCH,DIRECT"],
}
stream = {
"network": "tcp",
"security": "tls",
"tlsSettings": {
"certificates": [{"certificateFile": str(cert), "keyFile": str(key)}]
},
}
if mode.startswith("ws"):
stream.update(network="ws", wsSettings={"path": "/trojan-ws/"})
stream["tlsSettings"]["alpn"] = ["http/1.1"]
if kind == "V2":
stream["wsSettings"].update(
maxEarlyData=2048,
earlyDataHeaderName="x-vcore-ed" if mode == "ws-header" else "",
)
if mode == "grpc":
stream.update(network="grpc", grpcSettings={"serviceName": "trojan-grpc"})
stream["tlsSettings"]["alpn"] = ["h2"]
return {
"log": {"loglevel": "none"},
"dns": {"hosts": {"vcore-fixture.test": "127.0.0.1"}},
"inbounds": [
{
"listen": "127.0.0.1",
"port": port,
"protocol": "trojan",
"settings": {"clients": [{"password": password}]},
"streamSettings": stream,
}
],
"outbounds": [{"protocol": "freedom", "settings": {"domainStrategy": "UseIP"}}],
}
@@ -0,0 +1,649 @@
"""Unified foundation and implemented protocol-consumer acceptance runner."""
from __future__ import annotations
import contextlib
import json
import os
import signal
import sys
import tempfile
import time
from datetime import UTC, datetime
from pathlib import Path
from .builds import CORE_DIR
from .mihomo import _run_mihomo_interop
from .mihomo_isolation import exclusive_run
from .protocol_catalogs import CATALOG_DIR, check_protocol_catalogs
from .protocol_evidence import (
check_limit_references,
check_run,
idle_resources,
load_manifest,
new_result,
read_events,
read_json,
rust_results,
select_cases,
validate_results,
)
from .protocol_inputs import redact, run_identity, sha256, source_identity
from .protocol_peers import run_command
from .protocol_preflight import preflight
from .protocol_streams import run_streams
@contextlib.contextmanager
def deadline(seconds: float):
if not hasattr(signal, "SIGALRM"):
raise RuntimeError("whole-suite watchdog requires a supported Unix test host")
previous = signal.getsignal(signal.SIGALRM)
def expired(*_):
raise TimeoutError("protocol suite deadline expired")
signal.signal(signal.SIGALRM, expired)
previous_timer = signal.getitimer(signal.ITIMER_REAL)
started = time.monotonic()
signal.setitimer(
signal.ITIMER_REAL,
min(seconds, previous_timer[0]) if previous_timer[0] > 0 else seconds,
)
try:
yield
finally:
remaining = (
max(0.001, previous_timer[0] - (time.monotonic() - started))
if previous_timer[0] > 0
else 0
)
signal.setitimer(signal.ITIMER_REAL, remaining, previous_timer[1])
signal.signal(signal.SIGALRM, previous)
def _events(path: Path) -> list[dict]:
if not path.exists():
return []
return read_events(path)
def _command(
run,
output: Path,
name: str,
command: list[str],
seconds: int,
*,
events: Path | None = None,
):
env = dict(os.environ)
env.pop("VCORE_CASE_EVENTS", None)
if events:
env["VCORE_CASE_EVENTS"] = str(events)
print(f"{run.get('stage', 'FOUNDATIONS')}: {name}", flush=True)
result = run_command(
command, cwd=CORE_DIR, env=env, timeout=seconds, limit=4 * 1024 * 1024
)
log = output / f"{name}.log"
log.write_text(redact(result.stdout.decode("utf-8", errors="replace")))
run["commands"].append(
{
"name": name,
"command": [redact(str(part)) for part in command],
"exit_code": result.returncode,
"seconds": result.seconds,
"cleanup": result.cleanup,
"reason": result.reason,
"log": log.name,
}
)
return result
def _features(case, run, output):
started = time.monotonic()
results = []
names = [
None,
"outbound-trojan",
"outbound-vmess",
"outbound-hysteria2",
"stream-transport",
"quic-transport",
]
for name in names:
command = ["cargo", "check", "--locked", "--no-default-features", "--lib"]
if name:
command.extend(["--features", name])
results.append(
_command(
run,
output,
"feature-" + (name or "minimal"),
command,
max(1, int(case["timeout_seconds"] - (time.monotonic() - started))),
)
)
results.append(
_command(
run,
output,
"feature-default",
["cargo", "test", "--locked", "--test", "feature_foundations"],
max(1, int(case["timeout_seconds"] - (time.monotonic() - started))),
)
)
cargo = __import__("tomllib").loads((CORE_DIR / "Cargo.toml").read_text())
for name in names[1:]:
pending = [name]
enabled = set()
while pending:
item = pending.pop()
if item not in enabled:
enabled.add(item)
pending.extend(cargo["features"].get(item, []))
if enabled & {"outbound-vless", "tun"}:
raise RuntimeError(
"foundation feature silently enables an old protocol or TUN"
)
passed = all(item.returncode == 0 and item.cleanup for item in results)
result = new_result(case)
result.update(
status="PASS" if passed else "FAIL",
assertions=dict.fromkeys(case["expected_observation"], passed),
evidence=case["required_evidence"] if passed else [],
cleanup=all(item.cleanup for item in results),
command_exit_code=0 if passed else 1,
)
return result
SCRIPT_OBSERVATIONS = {
"offline-success": (
"test_protocol_peers.OwnedPeerTest."
"test_success_is_bounded_and_records_exit_and_cleanup"
),
"missing-duplicate-unknown-case": (
"test_protocol_evidence.ProtocolEvidenceTest."
"test_missing_duplicate_unknown_and_empty_required_cases_fail"
),
"download-failure": (
"test_native_release.NativeReleaseTest."
"test_failed_download_never_uses_old_binary_or_leaks_network_details"
),
"timeout": (
"test_protocol_peers.OwnedPeerTest."
"test_timeout_and_output_overflow_fail_and_join"
),
"sigint": (
"test_protocol_peers.OwnedPeerTest."
"test_real_sigint_joins_nested_peer_without_stopping_unrelated_peer"
),
"partial-start": (
"test_protocol_peers.OwnedPeerTest."
"test_partial_start_and_permission_failure_do_not_leave_a_process"
),
"permission": (
"test_protocol_peers.OwnedPeerTest."
"test_partial_start_and_permission_failure_do_not_leave_a_process"
),
"peer-exit": ("test_protocol_peers.OwnedPeerTest.test_peer_exit_is_not_readiness"),
"cleanup-failure": (
"test_protocol_peers.OwnedPeerTest.test_cleanup_failure_is_explicit"
),
"redaction": (
"test_native_release.NativeReleaseTest."
"test_failed_download_never_uses_old_binary_or_leaks_network_details"
),
}
def _scripts(case, run, output):
path = output / "script-tests.json"
command = _command(
run,
output,
"offline-scripts",
[sys.executable, "-m", "vcore_scripts.protocol_script_tests", str(path)],
case["timeout_seconds"],
)
data = read_json(path)
tests = {entry["test"]: entry["status"] for entry in data["cases"]}
assertions = {
name: tests.get(identifier) == "PASS"
for name, identifier in SCRIPT_OBSERVATIONS.items()
}
passed = (
command.returncode == 0
and all(assertions.values())
and len(tests) == len(data["cases"]) == data["tests_run"]
)
result = new_result(case)
result.update(
status="PASS" if passed else "FAIL",
assertions=assertions,
evidence=case["required_evidence"] if passed else [],
cleanup=command.cleanup,
command_exit_code=command.returncode,
)
return result
def _legacy(case, run, output, artifacts):
result = new_result(case)
if "M" not in artifacts or "M-container" not in artifacts:
result.update(
status="BLOCKED",
reason="M native/container prerequisites unavailable",
cleanup=True,
)
return result
path = output / "legacy-events.jsonl"
previous = os.environ.get("VCORE_CASE_EVENTS")
os.environ["VCORE_CASE_EVENTS"] = str(path)
started = time.monotonic()
code = 1
cleaned = False
try:
# The existing orchestrator owns exact peers/containers through ExitStack.
# SIGINT/timeout unwinds it in this process, not by orphaning a child CLI.
with (
(output / "legacy.log").open("w") as log,
contextlib.redirect_stdout(log),
deadline(case["timeout_seconds"]),
):
_run_mihomo_interop(
artifacts["M"].binary,
extended=True,
soak_seconds=0,
container_binary=artifacts["M-container"].binary,
)
code = 0
cleaned = True
except Exception as error:
result.update(status="FAIL", reason=type(error).__name__)
finally:
if previous is None:
os.environ.pop("VCORE_CASE_EVENTS", None)
else:
os.environ["VCORE_CASE_EVENTS"] = previous
run["commands"].append(
{
"name": "legacy-mihomo",
"command": ["check", "mihomo-interop", "--container", "--extended"],
"exit_code": code,
"seconds": round(time.monotonic() - started, 3),
"cleanup": cleaned,
}
)
if (output / "legacy.log").exists():
log = (output / "legacy.log").read_text()
(output / "legacy.log").write_text(redact(log))
result = rust_results(case, _events(path), code)
result["cleanup"] = cleaned
if result["status"] == "PASS":
result["evidence"] = case["required_evidence"]
return result
def _native_results(cases, report):
actual = {case["case_id"]: case for case in report.get("cases", [])}
results = []
for case in cases:
result = new_result(case)
source = actual.get(case["case_id"], {})
event = source.get("rust_event", {})
assertions = {
"server_first": event.get("server_first") is True,
"payload_65536": event.get("payload_bytes") == 65536,
"trailer_exact": event.get("tail_bytes") == 14,
"origin_complete": source.get("origin")
== {"finished": True, "accepted": 1, "received": 65536, "failed": False},
"protect_once": event.get("protect_calls") == 1,
"driver_joined": event.get("driver_joined") is True,
"resources_idle": event.get("resources_idle") is True
and idle_resources(event.get("resources")),
}
passed = (
source.get("status") == "PASS"
and all(assertions.values())
and source.get("cleanup", {}).get("joined") is True
and report.get("cleanup") is True
)
if case["peer_kind"] == "M":
cleanup = report.get("abnormal_cleanup", {})
passed = passed and cleanup == {
"passed": True,
"kind": "M",
"failure_injected": True,
"joined": True,
"port_rebound": True,
}
result.update(
status="PASS" if passed else source.get("status", "NOT RUN"),
assertions=assertions,
evidence=case["required_evidence"] if passed else [],
cleanup=source.get("cleanup", {}).get("joined") is True
and report.get("cleanup") is True,
command_exit_code=source.get("command_exit_code"),
seconds=source.get("seconds"),
)
if not passed and result["status"] == "PASS":
result["status"] = "FAIL"
results.append(result)
return results
def _execute(selected, run, output, records):
rust = [case for case in selected if case["runner"] == "rust"]
if rust:
path = output / "rust-events.jsonl"
result = _command(
run,
output,
"rust-foundations",
["cargo", "test", "--locked", "--all-features", "--all-targets"],
max(case["timeout_seconds"] for case in rust),
events=path,
)
events = _events(path)
for case in rust:
records[case["case_id"]] = rust_results(case, events, result.returncode)
records[case["case_id"]]["cleanup"] = result.cleanup
kinds = {case["peer_kind"] for case in selected if case["peer_kind"] != "unit"}
artifacts, peers = preflight(
output / "binaries",
kinds,
container=any(case["runner"] == "mihomo-legacy" for case in selected),
)
(output / "peers.json").write_text(json.dumps(peers, indent=2) + "\n")
native = [case for case in selected if case["runner"] == "native-stream"]
if native:
build = _command(
run,
output,
"native-probe-build",
[
"cargo",
"build",
"--locked",
"--all-features",
"--example",
"protocol-stream-probe",
],
300,
)
if build.returncode == 0:
report = {}
try:
report = run_streams(
output / "native",
[case["case_id"] for case in native],
artifacts=artifacts,
)
except (OSError, ValueError, RuntimeError):
if (output / "native/stream-cases.json").exists():
report = read_json(output / "native/stream-cases.json")
for result in _native_results(native, report):
records[result["case_id"]] = result
for case in selected:
if case["runner"] == "features":
records[case["case_id"]] = _features(case, run, output)
elif case["runner"] == "scripts":
records[case["case_id"]] = _scripts(case, run, output)
elif case["runner"] == "mihomo-legacy":
records[case["case_id"]] = _legacy(case, run, output, artifacts)
def run_protocol_interop(
*,
stage: str,
identifiers=None,
protocol=None,
list_only=False,
preflight_only=False,
run_dir: Path | None = None,
) -> Path | None:
selected = select_cases(
load_manifest(), stage=stage, identifiers=identifiers, protocol=protocol
)
check_limit_references(load_manifest())
if list_only:
for case in selected:
print(
f"{case['case_id']}\t{case['substage']}\t{case['peer_kind']}\t{case['protocol']}"
)
return None
if stage in {"FOUNDATIONS", "TROJAN"}:
raise RuntimeError(
"legacy server orchestration is not fully containerized; "
"use check core or a current container suite"
)
check_protocol_catalogs(CATALOG_DIR)
root = CORE_DIR / "target/interop/runs"
root.mkdir(parents=True, exist_ok=True)
if run_dir is None:
output = Path(tempfile.mkdtemp(prefix=f"{stage.lower()}-", dir=root))
else:
output = run_dir.resolve()
if not output.is_relative_to(root.resolve()):
raise ValueError(
"run directory must be a fresh child of target/interop/runs"
)
output.mkdir(parents=True, exist_ok=False)
print(f"Protocol evidence: {output.relative_to(CORE_DIR.resolve())}", flush=True)
(output / "manifest.json").write_text(
json.dumps(
{
"schema_version": 1,
"kind": "executable-case-manifest",
"cases": load_manifest(),
},
indent=2,
)
+ "\n"
)
run = {
"schema_version": 1,
"stage": stage,
"mode": "preflight" if preflight_only else "execute",
"selected_cases": [case["case_id"] for case in selected],
"commands": [],
"cleanup": False,
"source_unchanged": False,
}
records = {case["case_id"]: new_result(case) for case in selected}
(output / "peers.json").write_text("[]\n")
error = None
try:
run.update(run_identity(stage, selected, preflight_only))
with exclusive_run(), deadline(run["suite_timeout_seconds"]):
if preflight_only and stage == "INTEGRATION":
from .protocol_integration_acceptance import (
preflight as integration_preflight,
)
integration_preflight(output)
elif preflight_only and stage == "SECURITY":
_, peers = preflight(
output / "binaries", {"M", "XR", "V2"}, container=True
)
(output / "peers.json").write_text(json.dumps(peers, indent=2) + "\n")
if any(
peer["status"] != "READY" or peer.get("container_status") != "READY"
for peer in peers
):
raise RuntimeError("SECURITY native prerequisites are BLOCKED")
elif preflight_only and stage == "HYSTERIA2":
from .protocol_hysteria2_acceptance import preflight as preflight_hy2
preflight_hy2(output)
elif preflight_only and stage == "XHTTP":
from .protocol_xhttp_acceptance import preflight as preflight_xhttp
preflight_xhttp(output)
elif preflight_only and stage in {"VMESS", "VLESS"}:
if stage == "VMESS":
from .protocol_vmess_container import run as preflight_vmess
else:
from .protocol_vless_container import run as preflight_vmess
protocol_name = "vmess" if stage == "VMESS" else "vless"
native = [
case["case_id"]
for case in selected
if case["runner"] == "native-" + protocol_name
]
if preflight_vmess(output / "native", native, preflight_only=True):
raise RuntimeError("container protocol preflight failed")
peers = read_json(output / f"native/{protocol_name}-results.json")[
"peers"
]
(output / "peers.json").write_text(json.dumps(peers, indent=2) + "\n")
elif preflight_only:
_, peers = preflight(
output / "binaries", {"M", "V2", "H", "XR"}, container=True
)
(output / "peers.json").write_text(json.dumps(peers, indent=2) + "\n")
if any(
peer["status"] != "READY"
or peer.get("container_status", "READY") != "READY"
for peer in peers
):
raise RuntimeError(
"some independent peer prerequisites are BLOCKED"
)
else:
if stage == "TROJAN":
from .protocol_trojan_acceptance import execute
execute(selected, run, output, records)
elif stage == "VMESS":
from .protocol_vmess_acceptance import execute
execute(selected, run, output, records)
elif stage == "VLESS":
from .protocol_vless_acceptance import execute
execute(selected, run, output, records)
elif stage == "XHTTP":
from .protocol_xhttp_acceptance import execute
execute(selected, run, output, records)
elif stage == "HYSTERIA2":
from .protocol_hysteria2_acceptance import execute
execute(selected, run, output, records)
elif stage == "SECURITY":
from .protocol_security_acceptance import execute
execute(selected, run, output, records)
elif stage == "INTEGRATION":
from .protocol_integration_acceptance import execute
execute(selected, run, output, records)
else:
_execute(selected, run, output, records)
except BaseException as caught:
error = caught
run["failure_kind"] = type(caught).__name__
finally:
results = list(records.values())
run["finished_utc"] = datetime.now(UTC).isoformat()
try:
current = source_identity()
run["source_unchanged"] = current == {key: run.get(key) for key in current}
except (OSError, RuntimeError, ValueError):
run["source_unchanged"] = False
run["identity_failure"] = True
run["cleanup"] = error is None and (
preflight_only or all(result["cleanup"] for result in results)
)
(output / "cases.json").write_text(json.dumps(results, indent=2) + "\n")
resources = []
try:
for path in output.rglob("*-events.jsonl"):
for event in _events(path):
if event.get("resources") is not None:
resources.append(event)
if (output / "native/stream-cases.json").exists():
for case in read_json(output / "native/stream-cases.json")["cases"]:
if case.get("rust_event", {}).get("resources") is not None:
resources.append(
{
"case_id": case["case_id"],
"phase": "after-stop",
"resources": case["rust_event"]["resources"],
}
)
except (OSError, ValueError, KeyError, TypeError) as caught:
error = error or caught
run.update(cleanup=False, failure_kind=type(error).__name__)
(output / "resources.jsonl").write_text(
"".join(json.dumps(item) + "\n" for item in resources)
)
status = (
"PREFLIGHT"
if preflight_only
else "PASS"
if error is None
and run["source_unchanged"]
and all(result["status"] == "PASS" for result in results)
else "FAIL"
)
(output / "summary.md").write_text(
f"# {stage}: {status}\n\n"
+ (
"Trojan protocol consumer evidence; other protocols remain NOT RUN.\n\n"
if stage == "TROJAN"
else "VMess protocol consumer evidence with "
"container-only servers and origins.\n\n"
if stage == "VMESS"
else "VLESS consumer evidence with container-only peers; "
"XHTTP and security remain separate suites.\n\n"
if stage == "VLESS"
else "VLESS XHTTP/sing-mux consumer evidence; "
"container-only native peers, security remains a separate suite.\n\n"
if stage == "XHTTP"
else "Hysteria2 consumer evidence with container-only peers; "
"native H server limits and platform/device gates remain explicit.\n\n"
if stage == "HYSTERIA2"
else "Selected VLESS security consumers; static ECH only, "
"container-only peers; integration, platforms and devices "
"remain separate.\n\n"
if stage == "SECURITY"
else "Seven-protocol integration; container-only peers. "
"Partial selections do not sign off lifecycle or soak gates.\n\n"
if stage == "INTEGRATION"
else "Foundation-only evidence; "
"production protocol field consumers remain NOT RUN.\n\n"
)
+ "\n".join(
f"- {result['case_id']}: {result['status']}" for result in results
)
+ "\n"
)
run["artifacts"] = [
{"path": str(path.relative_to(output)), "sha256": sha256(path)}
for path in sorted(output.rglob("*"))
if path.is_file()
and path.suffix in {".json", ".jsonl", ".log", ".md"}
and "binaries" not in path.parts
and path.name != "run.json"
]
(output / "run.json").write_text(json.dumps(run, indent=2) + "\n")
if error is not None:
raise RuntimeError(
f"protocol run incomplete ({type(error).__name__}); "
f"see {output.relative_to(CORE_DIR.resolve())}"
) from None
if not preflight_only:
if not run["source_unchanged"]:
raise RuntimeError(
"source changed during the run; evidence cannot sign off this input"
)
validate_results(selected, list(records.values()))
if not identifiers and protocol is None:
check_run(output, stage)
return output
@@ -0,0 +1,480 @@
"""HYSTERIA2 incremental native gates; all servers/origins stay in owned containers."""
from __future__ import annotations
import contextlib
import copy
import hashlib
import json
import os
import shutil
import ssl
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_release import download_mihomo
from .protocol_containers import ContainerLab, command
from .protocol_evidence import read_events
from .protocol_fixtures import certificate_chain, certificates
from .protocol_hysteria2_catalog import events_pass
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .protocol_vless_security import client_identities
def security_cases(base, identities, leaf_pin):
cases = []
def case(name, changes=None, *, absent=(), success=True):
node = copy.deepcopy(base)
node.update(changes or {})
for field in absent:
node.pop(field, None)
cases.append(dict(id=name, node=node, success=success))
case("root-pin")
case("leaf-pin", {"fingerprint": leaf_pin})
case("root-pin-wrong-name", {"sni": "wrong.fixture.test"}, success=False)
case("leaf-pin-is-trust", {"fingerprint": leaf_pin, "sni": "wrong.fixture.test"})
case("unknown-ca", absent=("fingerprint",), success=False)
case("explicit-skip", {"skip-cert-verify": True}, absent=("fingerprint",))
case("wrong-pin", {"fingerprint": "00" * 32}, success=False)
case(
"skip-does-not-override-pin",
{"fingerprint": "00" * 32, "skip-cert-verify": True},
success=False,
)
case("wrong-password", {"password": "not-the-password"}, success=False)
case("empty-password", {"port": 23005}, absent=("password",))
case("raw-password", {"port": 23006, "password": " untrimmed credential "})
case("empty-alpn-default", {"alpn": []})
case("custom-alpn", {"port": 23007, "alpn": ["h3-fixture"]})
case("wrong-alpn", {"alpn": ["wrong-fixture"]}, success=False)
case("mtls-valid", {"port": 23008, **identities["valid"]})
case("mtls-absent", {"port": 23008}, success=False)
for identity in ("expired", "wrong-ca"):
case("mtls-" + identity, {"port": 23008, **identities[identity]}, success=False)
obfs = {"obfs": "salamander", "obfs-password": "independent-obfs-fixture"}
case("salamander", {"port": 23009, **obfs})
case("salamander-client-only", obfs, success=False)
case("salamander-server-only", {"port": 23009}, success=False)
case(
"salamander-wrong-key",
{"port": 23009, **obfs, "obfs-password": "wrong-obfs-key"},
success=False,
)
case(
"salamander-wrong-auth",
{"port": 23009, **obfs, "password": "wrong-auth"},
success=False,
)
return cases
def run(output: Path, test="hysteria2_tcp_base", *, obfs=False, supplied=None):
if test not in {
"hysteria2_tcp_base",
"hysteria2_client_first",
"hysteria2_udp_base",
"native_bandwidth_matrix",
"native_mihomo_close_alignment",
"native_security_matrix",
"native_owned_lifecycle",
"native_deadline_and_udp_budget",
"runtime::public_lifecycle",
"runtime::public_ipv6_and_gates",
"runtime::public_entrypoints",
"hysteria2::public_udp_boundaries",
"hysteria2::public_concrete_upstream",
"hysteria2::public_graph_and_hop_snapshot",
}:
raise ValueError("unknown HYSTERIA2 test")
output = output.resolve()
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="HYSTERIA2",
scope="incremental-native",
source=source_identity(),
status="NOT RUN",
isolation={},
peers={},
cases=[],
obfs=obfs,
)
try:
if supplied is None:
identity = {}
binary = download_mihomo(
"linux-arm64", directory=output / "binaries", identity=identity
)
else:
binary, identity = supplied
if (
hashlib.sha256(binary.read_bytes()).hexdigest()
!= identity["binary_sha256"]
):
raise RuntimeError("supplied native peer identity mismatch")
report["peers"]["M"] = identity
lab = ContainerLab(report["isolation"], mtu=1500)
with tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary:
root = Path(temporary)
origin_dir, server_dir = root / "origin", root / "server"
origin_dir.mkdir()
server_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
shutil.copyfile(
Path(__file__).with_name("container_close_client.py"),
origin_dir / "close.py",
)
shutil.copy2(binary, server_dir / "peer")
cert, key, pin = (
certificate_chain if test == "native_security_matrix" else certificates
)(server_dir)
with contextlib.ExitStack() as stack:
origin = lab.start(
stack,
origin_dir,
"hysteria2-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
server = lab.start(
stack,
server_dir,
"hysteria2-mihomo",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
identity["version"] = command(
"exec", server.name, "/data/fixture/peer", "-v"
).strip()
digest = command(
"exec", server.name, "sha256sum", "/data/fixture/peer"
).split()[0]
if digest != identity["binary_sha256"]:
raise RuntimeError("native peer identity mismatch")
upstream = server
if test in {
"hysteria2::public_concrete_upstream",
"hysteria2::public_graph_and_hop_snapshot",
}:
# A separate native instance prevents its own outer UDP
# source tuple from tripping Mihomo's loopback detector.
hop_dir = root / "hop"
hop_dir.mkdir()
shutil.copy2(binary, hop_dir / "peer")
(hop_dir / "config.json").write_text(
json.dumps(
{
"socks-port": 23001,
"allow-lan": True,
"bind-address": "*",
"ipv6": True,
"log-level": "silent",
"rules": ["MATCH,DIRECT"],
}
)
)
upstream = lab.start(
stack,
hop_dir,
"hysteria2-hop",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
upstream.release()
upstream.wait_tcp(23001)
password = "synthetic-hysteria2-fixture"
(server_dir / "config.json").write_text(
json.dumps(
{
"socks-port": 23001,
"allow-lan": True,
"bind-address": "*",
"ipv6": True,
"log-level": "silent",
"hosts": {"vcore-fixture.test": origin.ipv4},
"listeners": [
{
"name": "hysteria2",
"type": "hysteria2",
"listen": "::",
"port": 23000,
"users": {"fixture": password},
"certificate": f"/data/fixture/{cert.name}",
"private-key": f"/data/fixture/{key.name}",
}
],
"rules": ["MATCH,DIRECT"],
}
)
)
fixture = root / "input.json"
bandwidth_peer = None
if test == "native_bandwidth_matrix":
bandwidth_dir = root / "bandwidth"
bandwidth_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_bandwidth_origin.py"),
bandwidth_dir / "origin.py",
)
bandwidth_peer = lab.start(
stack,
bandwidth_dir,
"hysteria2-bandwidth",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
bandwidth_peer.release()
bandwidth_peer.wait_tcp(24002)
peer_config = json.loads((server_dir / "config.json").read_text())
baseline = peer_config["listeners"][0]
for port, options in [
(23002, {"down": "1 Mbps"}),
(23003, {"up": "1 Mbps"}),
(23004, {"ignore-client-bandwidth": True}),
]:
peer_config["listeners"].append(
dict(
baseline, name=f"hysteria2-{port}", port=port, **options
)
)
(server_dir / "config.json").write_text(json.dumps(peer_config))
fixture.write_text(
json.dumps(
dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
server_ipv6=server.ipv6,
data_dir=str(root / "data"),
hop=dict(
name="hop",
type="socks5",
server=upstream.ipv4,
port=23001,
udp=True,
),
bandwidth_control=f"{bandwidth_peer.ipv4}:24002"
if bandwidth_peer
else None,
bandwidth_ipv4=bandwidth_peer.ipv4
if bandwidth_peer
else None,
node=dict(
name="peer",
type="hysteria2",
server=server.ipv4,
port=23000,
password=password,
udp=True,
sni="localhost",
fingerprint=pin,
alpn=["h3"],
),
)
)
)
if test == "native_security_matrix":
identities = client_identities(server_dir)
peer_config = json.loads((server_dir / "config.json").read_text())
baseline = peer_config["listeners"][0]
for port, options in [
(23005, {"users": {"fixture": ""}}),
(23006, {"users": {"fixture": " untrimmed credential "}}),
(23007, {"alpn": ["h3-fixture"]}),
(
23008,
{
"client-auth-type": "require-and-verify",
"client-auth-cert": "/data/fixture/root.pem",
},
),
(
23009,
{
"obfs": "salamander",
"obfs-password": "independent-obfs-fixture",
},
),
]:
peer_config["listeners"].append(
dict(
baseline, name=f"hysteria2-{port}", port=port, **options
)
)
(server_dir / "config.json").write_text(json.dumps(peer_config))
value = json.loads(fixture.read_text())
leaf = (
cert.read_text().split("-----END CERTIFICATE-----", 1)[0]
+ "-----END CERTIFICATE-----\n"
)
leaf_pin = hashlib.sha256(
ssl.PEM_cert_to_DER_cert(leaf)
).hexdigest()
value["security"] = security_cases(
value["node"], identities, leaf_pin
)
value["invalid_identities"] = [
{
"certificate": identities["valid"]["certificate"],
"private-key": identities["expired"]["private-key"],
},
{
"certificate": "bad PEM",
"private-key": identities["valid"]["private-key"],
},
]
fixture.write_text(json.dumps(value))
if obfs:
peer_config = json.loads((server_dir / "config.json").read_text())
for listener in peer_config["listeners"]:
listener.update(
obfs="salamander",
**{"obfs-password": "independent-obfs-fixture"},
)
(server_dir / "config.json").write_text(json.dumps(peer_config))
value = json.loads(fixture.read_text())
value["node"].update(
obfs="salamander",
**{"obfs-password": "independent-obfs-fixture"},
)
fixture.write_text(json.dumps(value))
origin.release()
server.release()
origin.wait_tcp(24000)
server.wait_tcp(23001)
if test == "native_mihomo_close_alignment":
from .protocol_xhttp_fields import close_reference
value = json.loads(fixture.read_text())
value["close_reference"] = close_reference(
lab, stack, root, "hy2", value["node"], origin, binary, output
)
fixture.write_text(json.dumps(value))
events = output / f"{test}-events.jsonl"
result = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"hysteria2_native"
if test.startswith("native_")
else "vless_public",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
],
cwd=CORE_DIR,
timeout=600 if test == "native_bandwidth_matrix" else 240,
limit=4 * 1024 * 1024,
env=dict(
os.environ,
VCORE_VLESS_INPUT=str(fixture),
VCORE_CASE_EVENTS=str(events),
VCORE_PROTOCOL_STAGE="HYSTERIA2",
VCORE_BANDWIDTH_OBSERVATIONS=str(output / "bandwidth.jsonl"),
),
)
(output / f"{test}.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
observed = read_events(events) if events.exists() else []
passed = (
result.returncode == 0
and result.cleanup
and events_pass(observed, test)
)
report["cases"].append(
dict(
case_id=test,
status="PASS" if passed else "FAIL",
exit_code=result.returncode,
seconds=result.seconds,
command_cleanup=result.cleanup,
command=test_command(test),
)
)
report["status"] = "PASS" if passed else "FAIL"
for role, peer in (("origin", origin), ("server", server)):
peer.ensure_alive()
if peer.log.exists():
(output / f"{role}.log").write_text(
redact(peer.log.read_text(errors="replace")[-65536:])
)
except (OSError, RuntimeError, ValueError) as error:
report.update(
status="FAIL",
reason=str(error)
if isinstance(error, RuntimeError)
else type(error).__name__,
)
finally:
report["cleanup"] = all(
peer["joined"] for peer in report["isolation"].get("peers", [])
)
report["source_unchanged"] = (
source_identity()["source_tree_sha256"]
== report["source"]["source_tree_sha256"]
)
if not report["source_unchanged"] or not report["cleanup"]:
report["status"] = "FAIL"
(output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
return report
def test_command(test):
return [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"hysteria2_native" if test.startswith("native_") else "vless_public",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
if __name__ == "__main__":
import sys
result = run(
Path(sys.argv[1]),
sys.argv[2] if len(sys.argv) > 2 else "hysteria2_tcp_base",
obfs="--salamander" in sys.argv[3:],
)
print(json.dumps({key: result[key] for key in ("status", "cleanup", "cases")}))
raise SystemExit(0 if result["status"] == "PASS" and result["cleanup"] else 1)
@@ -0,0 +1,670 @@
"""Hysteria2 execution and fail-closed reconstruction of persisted evidence."""
from __future__ import annotations
import contextlib
import json
import sys
from .builds import DEFAULT_FEATURES
from .protocol_evidence import HEX, new_result, read_events, read_json
from .protocol_hysteria2_catalog import (
CFG,
FIELDS,
GATES,
H3,
UNIT,
H,
M,
assertions_pass,
definitions,
events_pass,
)
from .protocol_inputs import redact
def commands(identifier):
cargo = ["cargo", "test", "--locked", "--all-features"]
if identifier in {"HYSTERIA2-CFG", "HYSTERIA2-UNIT", "HYSTERIA2-RELEASE"}:
if identifier == "HYSTERIA2-RELEASE":
cargo += ["--release"]
result = []
if identifier != "HYSTERIA2-UNIT":
result.append(cargo + ["--test", "hysteria2_config"])
if identifier != "HYSTERIA2-CFG":
result += [
cargo + ["--lib", "outbound::hysteria2::"],
cargo
+ [
"--lib",
"outbound::connector::tests::authenticated_continuation_keeps_group_choice_but_has_a_new_io_deadline",
"--",
"--exact",
],
cargo + ["--test", "hysteria2_paths"],
]
return result
if identifier == "HYSTERIA2-REGRESSION":
# Every chosen test uses memory IO/config, never a host server.
targets = [
"feature_foundations",
"limit_foundations",
"trojan",
"trojan_config",
"vmess_config",
"vless_config",
"vless_lifecycle",
"xhttp_config",
"xhttp_budget",
"h2_stream_regression",
"stream_foundations",
]
return [
cargo + sum((["--test", name] for name in targets), []),
cargo
+ [
"--test",
"quic_datagram_foundations",
"pending_send_is_not_restarted_and_stop_cancels_without_waiting_for_writable",
"--",
"--exact",
],
]
if identifier == "HYSTERIA2-QUALITY":
from .protocol_vmess_acceptance import commands as previous
return previous("VMESS-QUALITY")[:7] + [
[
"cargo",
"clippy",
"--locked",
"--all-features",
"--all-targets",
"--",
"-D",
"warnings",
],
[
"cargo",
"test",
"--locked",
"--manifest-path",
"crates/vcore-netstack/Cargo.toml",
"--all-targets",
],
[
"cargo",
"clippy",
"--locked",
"--manifest-path",
"crates/vcore-netstack/Cargo.toml",
"--all-targets",
"--",
"-D",
"warnings",
],
]
if identifier == "HYSTERIA2-PLATFORMS":
return [
["vcore-scripts", "build", "apple"],
["vcore-scripts", "build", "android"],
]
if identifier == "HYSTERIA2-FEATURES":
checks = [
["cargo", "check", "--locked", "--no-default-features", "--lib"]
+ (["--features", feature] if feature else [])
for feature in (
"",
"outbound-hysteria2",
"quic-transport",
"outbound-vless",
"tun",
)
]
return checks + [
[
"cargo",
"test",
"--locked",
"--test",
"feature_foundations",
"--test",
"hysteria2_config",
],
[
"cargo",
"test",
"--locked",
"--no-default-features",
"--test",
"hysteria2_config",
],
[
"cargo",
"test",
"--locked",
"--no-default-features",
"--features",
"outbound-hysteria2",
"--test",
"hysteria2_config",
],
[
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
],
]
if identifier == "HYSTERIA2-SCRIPTS":
return [
[
sys.executable,
"-m",
"vcore_scripts.protocol_script_tests",
"{output}/script-tests.json",
]
]
return []
def gate_result(case, records, events, script=None):
identifier = case["case_id"]
good = bool(records) and all(
r.get("exit_code") == 0 and r.get("cleanup") is True for r in records
)
if identifier in {"HYSTERIA2-CFG", "HYSTERIA2-UNIT", "HYSTERIA2-RELEASE"}:
expected = {}
if identifier != "HYSTERIA2-UNIT":
expected.update({("HYSTERIA2-CFG", name): 1 for name in CFG})
if identifier != "HYSTERIA2-CFG":
expected.update({("HYSTERIA2-UNIT", name): 1 for name in UNIT})
good &= assertions_pass(events, expected)
if identifier == "HYSTERIA2-SCRIPTS":
required = {
"test_protocol_hysteria2.Hysteria2AcceptanceTest." + name
for name in (
"test_catalog_covers_all_fields_and_eight_hop_combinations",
"test_partial_unknown_duplicate_and_wrong_suite_events_fail",
"test_native_identity_cleanup_or_missing_results_fail",
"test_bandwidth_and_hop_counters_cannot_be_forged_as_empty_success",
)
}
samples = (script or {}).get("cases", [])
names = [s.get("test") for s in samples]
good &= (
bool(samples)
and len(names) == len(set(names)) == (script or {}).get("tests_run")
and required <= set(names)
and all(s.get("status") == "PASS" for s in samples)
)
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions=dict.fromkeys(case["expected_observation"], bool(good)),
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=bool(records) and all(r.get("cleanup") is True for r in records),
)
return result
def bandwidth_pass(samples):
expected = [
(d, u, down, port, controller)
for d in ("U", "D")
for u, down, port, controller in (
(0, 0, 23000, 0),
*(
(rate, 100, 23000, rate * 125000) if d == "U" else (0, rate, 23000, 0)
for rate in (1, 2)
),
(2, 100, 23002, 125000) if d == "U" else (0, 2, 23003, 0),
)
] + [("U", 2, 0, 23000, 0), ("U", 2, 2, 23004, 0)]
if len(samples) != len(expected):
return False
for sample, key in zip(samples, expected, strict=True):
if (
tuple(
sample.get(k)
for k in (
"direction",
"up_mbps",
"down_mbps",
"listener_port",
"controller_bps",
)
)
!= key
):
return False
buckets = sample.get("buckets", [])
if (
len(buckets) != 30
or any(type(v) is not int or v < 0 for v in buckets)
or sample.get("payload_bps", 0) <= 0
or sample["payload_bps"] != sum(buckets) / 30
or sample.get("udp_tx_bytes", 0) <= 0
or sample.get("udp_rx_bytes", 0) <= 0
):
return False
for offset in (0, 4):
rates = [samples[offset + i]["payload_bps"] for i in range(4)]
if (
rates[0] < 750000
or rates[1] > 125000 * 1.1
or rates[2] > 250000 * 1.1
or rates[2] < rates[1] * 1.5
or rates[3] > 125000 * 1.1
):
return False
return True
def hopping_pass(value):
return (
value.get("seconds", 0) >= 60
and value.get("protected_sockets", 0) >= 9
and value.get("tcp_bytes_per_direction", 0) >= 10 * 1024 * 1024
and HEX.fullmatch(value.get("tcp_sha256", "")) is not None
and value.get("udp_packets") == 1500
and value.get("udp_target_kinds") == 3
and len(value.get("udp_maximums", [])) == 3
and all(1200 < n < 4096 for n in value["udp_maximums"])
and value.get("socket_peak") == 2
)
def native_result(case, report, directory):
from .protocol_hysteria2 import test_command
identifier = case["case_id"]
test = M[identifier][0] if identifier in M else H[identifier]["test"]
records = report.get("cases", [])
events = directory / (
f"{test}-events.jsonl" if identifier in M else "hysteria2-events.jsonl"
)
observed = read_events(events) if events.exists() else []
isolation = report.get("isolation", {})
peers = isolation.get("peers", [])
clean = (
report.get("cleanup") is True
and bool(peers)
and all(p.get("joined") is True and p.get("started") is True for p in peers)
)
good = (
clean
and report.get("status") == "PASS"
and report.get("source_unchanged") is True
and isolation.get("network_mode") == "hostOnly"
and isolation.get("host_servers") is False
and isolation.get("guest_mtu") == 1500
and bool(isolation.get("image_digest"))
)
good &= (
len(records) == 1
and all(
r.get("case_id") == test
and r.get("status") == "PASS"
and r.get("exit_code") == 0
and r.get("command_cleanup") is True
and r.get("command") == test_command(test)
for r in records
)
and events_pass(observed, test)
)
if identifier in M:
good &= report.get("obfs") is M[identifier][1]
identity = report.get("peers", {}).get("M", {})
project = "MetaCubeX/mihomo"
else:
identity = report.get("peer", {})
project = "HyNetworks/hysteria"
good &= (
all(
report.get(k) is H[identifier][k]
for k in ("ipv6", "obfs", "random_interval")
)
and report.get("auth_connections") == 1
)
counts = report.get("entry_port_packets", {})
good &= (
set(counts) == {"p23010", "p23011", "p23012"}
and all(type(v) is int and v >= 0 for v in counts.values())
and sum(v > 0 for v in counts.values())
>= (2 if test == "native_hopping" else 1)
)
if test == "native_hopping":
good &= (directory / "hopping.json").exists() and hopping_pass(
read_json(directory / "hopping.json")
)
good &= (
bool(identity.get("version"))
and HEX.fullmatch(identity.get("binary_sha256", "")) is not None
and identity.get("source_url", "").startswith(
f"https://github.com/{project}/releases/"
)
)
if test == "native_bandwidth_matrix":
path = directory / "bandwidth.jsonl"
good &= path.exists() and bandwidth_pass(read_events(path))
if test == "native_mihomo_close_alignment":
path = directory / "hy2-close-reference.json"
reference = read_json(path) if path.exists() else {}
good &= reference == {"tail_hex": "", "terminated": True, "scope": "same-mode"}
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions={test: bool(good)},
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=clean,
)
return result
def fields_report(cases, results):
status = {r["case_id"]: r["status"] for r in results}
rows = []
for field in sorted(FIELDS):
native = [
c["case_id"]
for c in cases
if field in c["row_ids"] and c["case_id"] in M | H
]
good = (
status.get("HYSTERIA2-CFG") == "PASS"
and bool(native)
and all(status.get(i) == "PASS" for i in native)
)
rows.append(
dict(
row_id=field,
configuration="HYSTERIA2-CFG",
behavior_cases=native,
status="PASS" if good else "NOT RUN",
)
)
return dict(stage="HYSTERIA2", scope="hysteria2-consumer", fields=rows)
def prepare(output, kinds):
from .protocol_containers import ContainerLab
from .protocol_hysteria2_hop import native_binary, packages
from .protocol_xhttp_acceptance import prepare_peers
supplied = prepare_peers(output, kinds - {"H"})
if "H" in kinds:
root = output / "binaries/H"
root.mkdir(parents=True)
binary, identity = native_binary(root)
record = {}
try:
lab = ContainerLab(record, mtu=1500)
with contextlib.ExitStack() as stack:
apks = packages(lab, stack, root / "apks")
supplied["H"] = binary, identity, apks, record["firewall_packages"]
finally:
(output / "package-preparation.json").write_text(
json.dumps(record, indent=2) + "\n"
)
return supplied
def h3_jobs():
return {
name: [dict(case_id=name, test="public_base")]
for name in (
"h3-stream-one-headers",
"h3-stream-up-headers",
"h3-packet-up-headers",
)
}
def h3_result(case, report, directory):
from .protocol_xhttp_fields import public_events_pass
jobs = h3_jobs()
records = report.get("cases", [])
isolation = report.get("isolation", {})
owned = isolation.get("peers", [])
good = (
report.get("status") == "PASS"
and report.get("cleanup") is True
and report.get("source_unchanged") is True
and {r.get("case_id") for r in records} == set(jobs)
and len(records) == len(jobs)
and isolation.get("network_mode") == "hostOnly"
and isolation.get("host_servers") is False
and isolation.get("guest_mtu") == 1500
and bool(isolation.get("image_digest"))
and bool(owned)
and all(p.get("started") is True and p.get("joined") is True for p in owned)
and set(report.get("peers", {})) == {"M", "XR"}
)
for record in records:
path = directory / (record["case_id"] + "-events.jsonl")
good &= (
record.get("exit_code") == 0
and record.get("command_cleanup") is True
and record.get("status") == "PASS"
and record.get("assertion") == "public_base"
and record.get("command")
== [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_public",
"public_base",
"--",
"--exact",
"--ignored",
"--nocapture",
]
and path.exists()
and public_events_pass(read_events(path), "public_base")
)
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions={case["expected_observation"][0]: bool(good)},
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=report.get("cleanup") is True,
)
return result
def execute(cases, run, output, results):
from .protocol_harness import _command
from .protocol_hysteria2 import run as run_m
from .protocol_hysteria2_hop import run as run_h
supplied = {}
try:
for case in cases:
identifier = case["case_id"]
if identifier not in GATES:
continue
records, events = [], []
for index, argv in enumerate(commands(identifier)):
argv = [v.replace("{output}", str(output)) for v in argv]
name = f"{identifier}-{index}"
path = output / f"{name}-events.jsonl"
_command(run, output, name, argv, case["timeout_seconds"], events=path)
records.append(run["commands"][-1])
if path.exists():
events += read_events(path)
script = (
read_json(output / "script-tests.json")
if identifier == "HYSTERIA2-SCRIPTS"
else None
)
results[identifier] = gate_result(case, records, events, script)
if results[identifier]["status"] != "PASS":
raise RuntimeError(f"HYSTERIA2 local gate failed: {identifier}")
kinds = {"M" for c in cases if c["case_id"] in M} | {
"H" for c in cases if c["case_id"] in H
}
if any(c["case_id"] == H3 for c in cases):
kinds |= {"M", "XR"}
supplied = prepare(output, kinds)
for case in cases:
identifier = case["case_id"]
directory = output / identifier
if identifier in M:
test, obfs, _ = M[identifier]
report = run_m(directory, test, obfs=obfs, supplied=supplied["M"])
result = native_result(case, report, directory)
elif identifier in H:
report = run_h(directory, **H[identifier], supplied=supplied["H"])
result = native_result(case, report, directory)
elif identifier == H3:
from .protocol_xhttp_fields import run as run_h3
jobs = h3_jobs()
run_h3(
directory,
list(jobs),
jobs=jobs,
supplied={k: supplied[k] for k in ("M", "XR")},
)
report = read_json(directory / "xhttp-fields-results.json")
result = h3_result(case, report, directory)
else:
continue
results[identifier] = result
print(f"{identifier}: {result['status']}", flush=True)
if result["status"] != "PASS":
raise RuntimeError(f"HYSTERIA2 native gate failed: {identifier}")
finally:
(output / "peers.json").write_text(
json.dumps(
{kind: artifact[1] for kind, artifact in supplied.items()}, indent=2
)
+ "\n"
)
(output / "fields.json").write_text(
json.dumps(fields_report(cases, list(results.values())), indent=2) + "\n"
)
def check(run_dir, cases, run, results, peers, paths):
if (
cases != definitions()
or set(peers) != {"M", "H", "XR"}
or not {
"fields.json",
"script-tests.json",
"package-preparation.json",
}
<= set(paths)
):
raise ValueError("incomplete HYSTERIA2 stage artifacts or altered manifest")
actual = {r["case_id"]: r for r in results}
records = run.get("commands", [])
expected_names = {
f"{c['case_id']}-{i}"
for c in cases
for i, _ in enumerate(commands(c["case_id"]))
}
if (
len(records) != len(expected_names)
or {r.get("name") for r in records} != expected_names
):
raise ValueError("missing, duplicate or extra HYSTERIA2 validation command")
source = {
k: run[k]
for k in (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
}
for case in cases:
identifier = case["case_id"]
if identifier in GATES:
selected, events = [], []
for i, argv in enumerate(commands(identifier)):
record = next(r for r in records if r["name"] == f"{identifier}-{i}")
if (
record["command"]
!= [redact(v.replace("{output}", str(run_dir))) for v in argv]
or record["log"] not in paths
):
raise ValueError("HYSTERIA2 validation command/log changed")
selected.append(record)
path = run_dir / f"{identifier}-{i}-events.jsonl"
if path.exists():
events += read_events(path)
recalculated = gate_result(
case,
selected,
events,
read_json(run_dir / "script-tests.json")
if identifier == "HYSTERIA2-SCRIPTS"
else None,
)
else:
directory = run_dir / identifier
report = read_json(
directory
/ ("xhttp-fields-results.json" if identifier == H3 else "report.json")
)
if report.get("source") != source:
raise ValueError("HYSTERIA2 native source differs from stage input")
recalculated = (
h3_result(case, report, directory)
if identifier == H3
else native_result(case, report, directory)
)
if identifier in H:
if report.get("peer") != peers.get("H"):
raise ValueError("H native identity changed")
preparation = read_json(run_dir / "package-preparation.json")
if report.get("isolation", {}).get(
"firewall_packages"
) != preparation.get("firewall_packages"):
raise ValueError("H native firewall package identity changed")
elif set(report.get("peers", {})) != (
{"M", "XR"} if identifier == H3 else {"M"}
):
raise ValueError("missing or extra native peer identity")
elif any(
identity != peers.get(kind)
for kind, identity in report.get("peers", {}).items()
):
raise ValueError("native identity changed between reports")
if recalculated != actual[identifier] or recalculated["status"] != "PASS":
raise ValueError("HYSTERIA2 results differ from structured evidence")
preparation = read_json(run_dir / "package-preparation.json")
if (
not preparation.get("firewall_packages")
or not preparation.get("peers")
or any(p.get("joined") is not True for p in preparation["peers"])
):
raise ValueError("native firewall preparation identity/cleanup missing")
fields = fields_report(cases, results)
if read_json(run_dir / "fields.json") != fields or any(
f["status"] != "PASS" for f in fields["fields"]
):
raise ValueError("missing HYSTERIA2 field behavior coverage")
print(f"HYSTERIA2: PASS ({len(cases)} required cases; {len(FIELDS)} field rows)")
def preflight(output):
# The normal native gates prove readiness as well as protocol behavior;
# this command deliberately never fabricates a protocol PASS.
supplied = prepare(output, {"M", "H", "XR"})
(output / "peers.json").write_text(
json.dumps({k: v[1] for k, v in supplied.items()}, indent=2) + "\n"
)
@@ -0,0 +1,329 @@
"""Frozen HYSTERIA2 requirements and exact structured assertion identities."""
from __future__ import annotations
from collections import Counter
from .protocol_evidence import idle_resources
FIELDS = {
*(f"C{i:02}" for i in range(1, 7)),
"T01",
"T03",
"T04",
"T05",
"T07",
"T08",
*(f"HY{i:02}" for i in range(1, 9)),
}
CFG = [
"hysteria2_configuration_follows_its_protocol_feature",
"hysteria2_accepts_the_approved_tls_bandwidth_obfs_and_hopping_configuration",
"hysteria2_fields_are_strict_normalized_and_credentials_are_not_trimmed",
]
UNIT = [
"tcp_wire_matches_the_protocol_and_preserves_server_first_bytes",
"udp_wire_round_trips_at_budget_and_bounds_malformed_fragments",
"udp_reassembly_isolates_sources_duplicates_counts_bytes_and_expiry",
"salamander_matches_independent_blake2b256_vector_and_rejects_short_packets",
"pacer_limits_actual_bytes_after_one_bounded_burst",
"bandwidth_negotiation_obeys_auto_and_the_lower_positive_limit",
"application_eof_closes_both_directions_without_consuming_a_tail",
"shutdown_wakes_an_already_pending_logical_reader",
"a_client_first_write_does_not_steal_a_pending_read_wakeup",
"a_new_path_packet_does_not_end_the_old_receive_window",
"authenticated_continuation_keeps_group_choice_but_has_a_new_io_deadline",
"controlled_path_maps_only_its_authorized_source_and_stops_without_replay",
]
SECURITY = (
"root-pin",
"leaf-pin",
"root-pin-wrong-name",
"leaf-pin-is-trust",
"unknown-ca",
"explicit-skip",
"wrong-pin",
"skip-does-not-override-pin",
"wrong-password",
"empty-password",
"raw-password",
"empty-alpn-default",
"custom-alpn",
"wrong-alpn",
"mtls-valid",
"mtls-absent",
"mtls-expired",
"mtls-wrong-ca",
"salamander",
"salamander-client-only",
"salamander-server-only",
"salamander-wrong-key",
"salamander-wrong-auth",
)
M = {
"HYSTERIA2-TCP": (
"hysteria2_tcp_base",
False,
["C01", "C02", "C03", "C04", "T01", "T03", "HY01"],
),
"HYSTERIA2-UDP": ("hysteria2_udp_base", False, ["C05", "HY06"]),
"HYSTERIA2-TCP-SALAMANDER": ("hysteria2_tcp_base", True, ["HY02", "HY03"]),
"HYSTERIA2-UDP-SALAMANDER": ("hysteria2_udp_base", True, ["HY02", "HY03", "HY06"]),
"HYSTERIA2-SECURITY": (
"native_security_matrix",
False,
["T01", "T03", "T04", "T05", "T07", "T08", "HY01", "HY02", "HY03"],
),
"HYSTERIA2-BANDWIDTH": ("native_bandwidth_matrix", False, ["HY04", "HY05"]),
"HYSTERIA2-CLOSE": ("native_mihomo_close_alignment", False, []),
"HYSTERIA2-CLOSE-SALAMANDER": (
"native_mihomo_close_alignment",
True,
["HY02", "HY03"],
),
"HYSTERIA2-OWNED": ("native_owned_lifecycle", False, []),
"HYSTERIA2-DEADLINE": ("native_deadline_and_udp_budget", False, ["HY06"]),
"HYSTERIA2-LIFE": ("runtime::public_lifecycle", False, []),
"HYSTERIA2-ENTRYPOINTS": ("runtime::public_entrypoints", False, ["C05"]),
"HYSTERIA2-IPV6-GATES": ("runtime::public_ipv6_and_gates", False, ["C03", "C05"]),
"HYSTERIA2-UDP-BOUNDARIES": ("hysteria2::public_udp_boundaries", False, ["HY06"]),
"HYSTERIA2-UPSTREAM": ("hysteria2::public_concrete_upstream", False, ["C06"]),
"HYSTERIA2-GRAPH": (
"hysteria2::public_graph_and_hop_snapshot",
False,
["C06", "HY07", "HY08"],
),
}
H = {
"-".join(
(
"HYSTERIA2-HOP",
"V6" if v6 else "V4",
"OBFS" if obfs else "PLAIN",
"RANDOM" if random else "FIXED",
)
): dict(test="native_hopping", ipv6=v6, obfs=obfs, random_interval=random)
for v6 in (False, True)
for obfs in (False, True)
for random in (False, True)
}
H.update(
{
"HYSTERIA2-UDP-DISABLED": dict(
test="native_udp_disabled", ipv6=False, obfs=False, random_interval=False
),
"HYSTERIA2-HOP-PROTECT": dict(
test="native_hop_protect_rejection",
ipv6=False,
obfs=False,
random_interval=False,
),
"HYSTERIA2-HOP-STOP": dict(
test="native_stop_during_hop", ipv6=True, obfs=True, random_interval=False
),
}
)
GATES = {
"HYSTERIA2-CFG",
"HYSTERIA2-UNIT",
"HYSTERIA2-RELEASE",
"HYSTERIA2-REGRESSION",
"HYSTERIA2-QUALITY",
"HYSTERIA2-FEATURES",
"HYSTERIA2-SCRIPTS",
"HYSTERIA2-PLATFORMS",
}
H3 = "HYSTERIA2-H3-REGRESSION"
def expected_events(test):
public = {
"hysteria2_tcp_base": ("HYSTERIA2-BASE", "tcp_ipv4_ipv6_domain_and_measure"),
"hysteria2_udp_base": ("HYSTERIA2-BASE", "udp_ipv4_ipv6_domain_fragmentation"),
"hysteria2_client_first": ("HYSTERIA2-BASE", "client_first"),
}
if test in public:
expected = {public[test]: 1}
if test == "hysteria2_tcp_base":
expected["HYSTERIA2-BASE", "tcp_10mib_both_directions"] = 3
return expected
if test.startswith("runtime::"):
result = {("HYSTERIA2-PUBLIC", test): 1}
if test == "runtime::public_lifecycle":
result["HYSTERIA2-LIFE", "stop_and_remain_quiet"] = 20
return result
if test.startswith("hysteria2::"):
return {("HYSTERIA2-PUBLIC", test.removeprefix("hysteria2::")): 1}
if test == "native_security_matrix":
return {
("HYSTERIA2-SECURITY", test): 1,
**{("HYSTERIA2-SECURITY-CASE", name): 1 for name in SECURITY},
}
return {
(
{
"native_bandwidth_matrix": "HYSTERIA2-BANDWIDTH",
"native_owned_lifecycle": "HYSTERIA2-LIFE",
"native_hopping": "HYSTERIA2-HOP",
"native_udp_disabled": "HYSTERIA2-UDP-DISABLED",
"native_hop_protect_rejection": "HYSTERIA2-HOP",
"native_stop_during_hop": "HYSTERIA2-HOP",
"native_deadline_and_udp_budget": "HYSTERIA2-NATIVE",
"native_mihomo_close_alignment": "HYSTERIA2-NATIVE",
}[test],
test,
): 20 if test == "native_owned_lifecycle" else 1
}
def assertions_pass(events, expected):
if not events or any(
e.get("schema_version") != 1 or e.get("status") not in {"BEGIN", "PASS"}
for e in events
):
return False
actual = Counter((e.get("suite"), e.get("assertion")) for e in events)
if actual != Counter({key: count * 2 for key, count in expected.items()}):
return False
for key, count in expected.items():
selected = [e for e in events if (e.get("suite"), e.get("assertion")) == key]
if [e["status"] for e in selected] != ["BEGIN", "PASS"] * count:
return False
for event in selected[1::2]:
if event.get("resources") is not None and not idle_resources(
event["resources"]
):
return False
return True
def events_pass(events, test):
if not assertions_pass(events, expected_events(test)):
return False
passed = [e for e in events if e["status"] == "PASS"]
if test in {
"native_hopping",
"native_hop_protect_rejection",
"native_stop_during_hop",
"native_owned_lifecycle",
"native_deadline_and_udp_budget",
"native_security_matrix",
}:
owned = [e for e in passed if e["assertion"] == test]
if any(not idle_resources(e.get("resources")) for e in owned):
return False
if test == "native_owned_lifecycle" and any(
[p["phase"] for p in e["checkpoints"]]
!= ["baseline", "after-stop", "quiet"]
or any(not idle_resources(p.get("resources")) for p in e["checkpoints"])
or e["seconds"] < 5
for e in owned
):
return False
return True
def definitions():
output = []
for identifier in sorted(GATES | set(M) | set(H) | {H3}):
kind, test, rows, options = "unit", identifier, [], {}
if identifier in M:
test, obfs, rows = M[identifier]
kind, options = "M", dict(obfs=obfs)
elif identifier in H:
options = H[identifier]
kind, test = "H", options["test"]
rows = ["C04", "HY07", "HY08"] if test != "native_udp_disabled" else ["C05"]
elif identifier == H3:
kind, test = "XR", "shared-quic-h3-native-regression"
elif identifier == "HYSTERIA2-CFG":
rows = sorted(FIELDS)
native = kind != "unit"
substage = (
"HYSTERIA2.hopping"
if kind == "H" and test != "native_udp_disabled"
else "HYSTERIA2.bandwidth"
if identifier == "HYSTERIA2-BANDWIDTH"
else "HYSTERIA2.udp"
if "UDP" in identifier
else "HYSTERIA2.tcp"
if identifier in M
else "HYSTERIA2.acceptance"
)
expected = (
CFG
if identifier == "HYSTERIA2-CFG"
else UNIT
if identifier == "HYSTERIA2-UNIT"
else CFG + UNIT
if identifier == "HYSTERIA2-RELEASE"
else [test]
)
output.append(
dict(
case_id=identifier,
stage="HYSTERIA2",
substage=substage,
required=True,
row_ids=sorted(rows),
protocol="hysteria2" if identifier != H3 else "vless",
network="quic",
security="tls13",
udp_codec="hysteria2-datagram" if identifier != H3 else "vless",
field_values=options,
outer_family="IPv6"
if options.get("ipv6")
else "IPv4/IPv6"
if identifier == "HYSTERIA2-IPV6-GATES"
else "IPv4",
inner_family="IPv4/IPv6/domain",
target_type="isolated synthetic origin"
if native
else "memory-only/build",
upstream_graph="direct/concrete/nested-select"
if identifier in {"HYSTERIA2-GRAPH", "HYSTERIA2-UPSTREAM"}
else "direct",
peer_kind=kind,
peer_config=dict(
test=test, **{k: v for k, v in options.items() if k != "test"}
),
gap_source=dict(
reason=(
"Mihomo HY2 listener cannot redirect one QUIC state "
"across a port set or disable authenticated business "
"UDP; official Hysteria provides these gates"
),
source="https://github.com/HyNetworks/hysteria/blob/e1366b173ccf5706e1e4630fe8aa654a4b574085/core/server/server.go",
)
if kind == "H"
else dict(
reason=(
"XHTTP H3 shared-adapter regression uses the XHTTP "
"official Xray handler"
),
source="https://github.com/XTLS/Xray-core",
)
if kind == "XR"
else None,
expected_observation=expected,
required_evidence=["structured-assertions", "command", "cleanup"]
+ (["peer-identity", "isolated-origins"] if native else []),
prerequisites=[
"owned-host-only-container-network",
"official-latest-peer",
]
if native
else ["rust-toolchain"],
runner="native-hysteria2" if kind in {"M", "H"} else "hysteria2-gate",
timeout_seconds=3600
if identifier == "HYSTERIA2-PLATFORMS"
else 900
if identifier == H3
else 1200
if not native
else 600
if identifier == "HYSTERIA2-BANDWIDTH"
else 300,
)
)
return output
@@ -0,0 +1,440 @@
"""Official Hysteria, with redirection confined to an owned Linux container."""
from __future__ import annotations
import contextlib
import hashlib
import json
import os
import shutil
import tempfile
import time
import urllib.request
from pathlib import Path
from .builds import CORE_DIR
from .protocol_containers import NETWORK, ContainerLab, ContainerPeer, command
from .protocol_evidence import read_events
from .protocol_fixtures import certificates
from .protocol_hysteria2_catalog import events_pass
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
def guest(peer, *argv, timeout=30):
# Container CLI can exit zero even when its guest command fails.
result = run_command(
[
"container",
"exec",
peer.name,
"/bin/sh",
"-c",
'"$@" 2>&1\ncode=$?\nprintf "\\nVCORE_STATUS=%s\\n" "$code"',
"fixture",
*argv,
],
timeout=timeout,
)
output = result.stdout.decode(errors="replace")
if result.returncode != 0 or not result.cleanup:
raise RuntimeError(
f"isolated guest {argv[0]} failed after {result.seconds:.1f}s "
f"(exit={result.returncode}, cleanup={result.cleanup}): "
f"{redact(output[-2048:])}"
)
body, marker, code = output.rpartition("\nVCORE_STATUS=")
if not marker or code.strip() != "0":
raise RuntimeError(
f"isolated guest operation failed: {argv[0]}: {redact(body[-2048:])}"
)
return body
def native_binary(output):
url = "https://github.com/HyNetworks/hysteria/releases/latest/download/hysteria-linux-arm64"
path = output / "hysteria"
digest, total, deadline = hashlib.sha256(), 0, time.monotonic() + 90
request = urllib.request.Request(url, headers={"User-Agent": "VCore-interop"})
with (
urllib.request.urlopen(request, timeout=30) as response,
path.open("xb") as target,
):
if not response.geturl().startswith("https://"):
raise RuntimeError("native download redirected outside HTTPS")
while chunk := response.read(1024 * 1024):
total += len(chunk)
if total > 128 * 1024 * 1024 or time.monotonic() > deadline:
raise RuntimeError("native binary download budget exceeded")
target.write(chunk)
digest.update(chunk)
path.chmod(0o755)
return path, {"source_url": url, "binary_sha256": digest.hexdigest()}
def packages(lab, stack, root):
"""Only package preparation has external egress; no server runs there."""
root.mkdir()
peer = ContainerPeer(lab, root, "hysteria2-packages")
lab.record["peers"].append(peer.record)
stack.callback(peer.stop)
command(
"run",
"--detach",
"--name",
peer.name,
"--label",
f"purpose={NETWORK}",
"--label",
f"vcore-run={lab.run_id}",
"--network",
"default",
"--arch",
"arm64",
"--cpus",
"1",
"--memory",
"256M",
"--mount",
f"type=bind,source={root},target=/packages",
"--entrypoint",
"/bin/sleep",
lab.image,
"360",
timeout=45,
)
peer.record.update(
started=True, role="package-preparation-only", network_mode="default"
)
guest(peer, "apk", "update", timeout=60)
guest(
peer,
"apk",
"fetch",
"--recursive",
"--output",
"/packages",
"nftables",
timeout=240,
)
found = sorted(root.glob("*.apk"))
if not found:
raise RuntimeError("missing native firewall packages")
lab.record["firewall_packages"] = {
path.name: hashlib.sha256(path.read_bytes()).hexdigest() for path in found
}
peer.stop()
return found
def run(
output: Path,
*,
ipv6=False,
obfs=False,
random_interval=False,
test="native_hopping",
supplied=None,
):
if test not in {
"native_hopping",
"native_udp_disabled",
"native_hop_protect_rejection",
"native_stop_during_hop",
}:
raise ValueError("unknown native H gate")
output = output.resolve()
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="HYSTERIA2",
scope="incremental-native-H",
source=source_identity(),
status="NOT RUN",
isolation={},
cases=[],
ipv6=ipv6,
obfs=obfs,
random_interval=random_interval,
)
try:
if supplied is None:
binary, report["peer"] = native_binary(output)
else:
binary, report["peer"], _, _ = supplied
if (
hashlib.sha256(binary.read_bytes()).hexdigest()
!= report["peer"]["binary_sha256"]
):
raise RuntimeError("supplied H artifact identity mismatch")
lab = ContainerLab(report["isolation"], mtu=1500)
with (
tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary,
contextlib.ExitStack() as stack,
):
root = Path(temporary)
root.chmod(0o700)
if supplied is None:
apks = packages(lab, stack, root / "apks")
else:
_, _, apks, digests = supplied
if {
p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in apks
} != digests:
raise RuntimeError("supplied firewall package identity mismatch")
report["isolation"]["firewall_packages"] = digests
origin_dir, server_dir = root / "origin", root / "server"
origin_dir.mkdir()
server_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
shutil.copy2(binary, server_dir / "hysteria")
(server_dir / "apks").mkdir()
for path in apks:
shutil.copy2(path, server_dir / "apks" / path.name)
cert, key, pin = certificates(server_dir)
origin = lab.start(
stack,
origin_dir,
"hysteria2-h-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
server = lab.start(
stack,
server_dir,
"hysteria2-h-peer",
[
"env",
"HYSTERIA_FIREWALL_BACKEND=nftables",
"/data/fixture/hysteria",
"server",
"--disable-update-check",
"--log-format",
"json",
"-c",
"/data/fixture/config.json",
],
net_admin=True,
)
guest(
server,
"apk",
"add",
"--no-network",
*[f"/data/fixture/apks/{path.name}" for path in apks],
timeout=45,
)
report["peer"]["version"] = guest(
server, "/data/fixture/hysteria", "version"
).strip()
if (
guest(server, "sha256sum", "/data/fixture/hysteria").split()[0]
!= report["peer"]["binary_sha256"]
):
raise RuntimeError("native binary identity mismatch")
report["isolation"]["nftables"] = guest(server, "nft", "--version").strip()
(server_dir / "hosts").write_text(
"127.0.0.1 localhost\n::1 localhost\n"
f"{origin.ipv4} vcore-fixture.test\n"
)
guest(server, "cp", "/data/fixture/hosts", "/etc/hosts")
# Pre-DNAT observations: native Hysteria owns its separate redirect table.
rules = (
"table inet vcore_hysteria2 { chain observe { "
"type filter hook prerouting priority -150; policy accept; "
)
for port in range(23010, 23013):
rules += f'udp dport {port} counter comment "p{port}"; '
rules += "}\n}\n"
(server_dir / "observe.nft").write_text(rules)
guest(server, "nft", "-f", "/data/fixture/observe.nft")
password = "synthetic-hysteria2-fixture"
config = dict(
# Native Hysteria deliberately treats [::] as IPv6-only.
listen=f"[{server.ipv6}]:23010-23012"
if ipv6
else f"{server.ipv4}:23010-23012",
tls=dict(
cert=f"/data/fixture/{cert.name}", key=f"/data/fixture/{key.name}"
),
auth=dict(type="password", password=password),
trafficStats=dict(listen="0.0.0.0:24003"),
)
node = dict(
name="peer",
type="hysteria2",
server=server.ipv6 if ipv6 else server.ipv4,
ports="23010-23012",
**{"hop-interval": "5-7" if random_interval else 5},
password=password,
udp=True,
sni="localhost",
fingerprint=pin,
)
if obfs:
config["obfs"] = dict(
type="salamander",
salamander=dict(password="independent-obfs-fixture"),
)
node.update(
obfs="salamander", **{"obfs-password": "independent-obfs-fixture"}
)
if test == "native_udp_disabled":
config["disableUDP"] = True
(server_dir / "config.json").write_text(json.dumps(config))
fixture = root / "input.json"
fixture.write_text(
json.dumps(
dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
node=node,
)
)
)
origin.release()
server.release()
origin.wait_tcp(24000)
server.wait_tcp(24003)
result = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"hysteria2_native",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
],
cwd=CORE_DIR,
timeout=180,
limit=4 * 1024 * 1024,
env=dict(
os.environ,
VCORE_VLESS_INPUT=str(fixture),
VCORE_CASE_EVENTS=str(output / "hysteria2-events.jsonl"),
VCORE_HOP_OBSERVATIONS=str(output / "hopping.json"),
),
)
(output / "test.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
raw_log = command("logs", server.name)
connected = sum(
'"msg":"client connected"' in line for line in raw_log.splitlines()
)
report["auth_connections"] = connected
(output / "server.log").write_text(
redact(raw_log.replace(password, "<fixture-auth>"))
)
rules = json.loads(
guest(server, "nft", "-j", "list", "table", "inet", "vcore_hysteria2")
)
counts = {}
for entry in rules["nftables"]:
if "rule" in entry:
rule = entry["rule"]
counts[rule["comment"]] = next(
item["counter"]["packets"]
for item in rule["expr"]
if "counter" in item
)
report["entry_port_packets"] = counts
passed = (
result.returncode == 0
and result.cleanup
and (output / "hysteria2-events.jsonl").exists()
and events_pass(read_events(output / "hysteria2-events.jsonl"), test)
and connected == 1
and sum(value > 0 for value in counts.values())
>= (2 if test == "native_hopping" else 1)
)
report["cases"].append(
dict(
case_id=test,
exit_code=result.returncode,
seconds=result.seconds,
command_cleanup=result.cleanup,
command=[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"hysteria2_native",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
],
status="PASS" if passed else "FAIL",
)
)
report["status"] = "PASS" if passed else "FAIL"
except (OSError, RuntimeError, ValueError) as error:
report.update(
status="FAIL",
reason=str(error)
if isinstance(error, RuntimeError)
else type(error).__name__,
)
finally:
report["cleanup"] = all(
peer["joined"] for peer in report["isolation"].get("peers", [])
)
report["source_unchanged"] = (
source_identity()["source_tree_sha256"]
== report["source"]["source_tree_sha256"]
)
if not report["source_unchanged"] or not report["cleanup"]:
report["status"] = "FAIL"
(output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
return report
if __name__ == "__main__":
import sys
result = run(
Path(sys.argv[1]),
ipv6="--ipv6" in sys.argv,
obfs="--salamander" in sys.argv,
random_interval="--random" in sys.argv,
test=(
sys.argv[sys.argv.index("--test") + 1]
if "--test" in sys.argv
else "native_udp_disabled"
if "--udp-disabled" in sys.argv
else "native_hopping"
),
)
print(
json.dumps(
{
key: result.get(key)
for key in (
"status",
"reason",
"cleanup",
"cases",
"auth_connections",
"entry_port_packets",
)
}
)
)
raise SystemExit(0 if result["status"] == "PASS" and result["cleanup"] else 1)
@@ -0,0 +1,133 @@
"""Reproducible, nonsensitive input identity for one local protocol run."""
from __future__ import annotations
import hashlib
import platform
import re
import sys
import tomllib
from datetime import UTC, datetime
from pathlib import Path
from .builds import CORE_DIR
from .protocol_peers import run_command
CODE_PATHS = [
"Cargo.toml",
"Cargo.lock",
"src",
"crates",
"tests",
"scripts",
"include",
]
def sha256(path: Path) -> str:
with path.open("rb") as stream:
return hashlib.file_digest(stream, "sha256").hexdigest()
def command_output(command: list[str]) -> bytes:
result = run_command(command, cwd=CORE_DIR, timeout=20, limit=8 * 1024 * 1024)
if result.returncode != 0:
raise RuntimeError("cannot record protocol input identity")
return result.stdout
def source_identity() -> dict:
paths = (
command_output(
["git", "ls-files", "-co", "--exclude-standard", "-z", "--", *CODE_PATHS]
)
.decode()
.split("\0")
)
digest = hashlib.sha256()
for name in sorted(set(paths) - {""}):
path = CORE_DIR / name
if not path.exists():
continue
if path.is_symlink() or not path.is_file():
raise ValueError("non-file source input")
digest.update(name.encode() + b"\0" + bytes.fromhex(sha256(path)))
patch = command_output(["git", "diff", "--binary", "HEAD", "--", *CODE_PATHS])
return {
"parent_commit": command_output(["git", "rev-parse", "HEAD"]).decode().strip(),
"source_tree_sha256": digest.hexdigest(),
"dirty_patch_sha256": hashlib.sha256(patch).hexdigest(),
"lock_sha256": sha256(CORE_DIR / "Cargo.lock"),
}
def same_source(before: dict, after: dict) -> bool:
"""Compare actual inputs, not a patch changed merely by staging new files."""
return all(
before[key] == after[key]
for key in ("parent_commit", "source_tree_sha256", "lock_sha256")
)
def run_identity(stage: str, selected: list[dict], preflight: bool) -> dict:
record = source_identity()
lock = tomllib.loads((CORE_DIR / "Cargo.lock").read_text())
tls = [
{key: package[key] for key in ["name", "version", "source"]}
for package in lock["package"]
if package["name"] in {"rustls", "tokio-rustls", "ring"}
]
text = (CORE_DIR / "src/lib.rs").read_text()
record.update(
schema_version=1,
stage=stage,
mode="preflight" if preflight else "execute",
started_utc=datetime.now(UTC).isoformat(),
os=platform.system(),
os_version=platform.release(),
architecture=platform.machine(),
python=platform.python_version(),
rustls=tls,
invoke_api_version=int(re.search(r"INVOKE_API_VERSION: u32 = (\d+)", text)[1]),
config_version=int(re.search(r"CONFIG_VERSION: u8 = (\d+)", text)[1]),
features=["all-features", "independent-feature-smokes"],
selected_cases=[case["case_id"] for case in selected],
commands=[],
cleanup=False,
source_unchanged=False,
artifacts=[],
suite_timeout_seconds=sum(case["timeout_seconds"] for case in selected) + 1800,
)
record["rustc"] = command_output(["rustc", "-Vv"]).decode().strip()
record["cargo"] = command_output(["cargo", "--version"]).decode().strip()
if sys.platform == "darwin":
record["xcode"] = command_output(["xcodebuild", "-version"]).decode().strip()
record["sdk"] = command_output(["xcrun", "--show-sdk-version"]).decode().strip()
return record
def redact(text: str) -> str:
text = text.replace(str(CORE_DIR), "<VCore>")
text = re.sub(
r"(?i)(authorization\s*[:=]\s*)(?:Bearer|Basic)\s+[^\s,}]+",
r"\1<redacted>",
text,
)
text = re.sub(
r"(?i)-----BEGIN .*?PRIVATE KEY-----.*?-----END .*?PRIVATE KEY-----",
"<private-key>",
text,
flags=re.S,
)
text = re.sub(
r"[0-9a-fA-F]{8}(?:-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}", "<synthetic-id>", text
)
text = re.sub(
r"(?i)((?:password|authorization|private[-_]key|token)\s*[:=]\s*)([^\s,}]+)",
r"\1<redacted>",
text,
)
text = re.sub(
r"/(?:Users|home|private/var|var/folders)/[^\s\"']+", "<host-path>", text
)
return text
@@ -0,0 +1,357 @@
"""INTEGRATION execution and independent reconstruction of raw integration evidence."""
from __future__ import annotations
import json
from .protocol_evidence import read_events, read_json
from .protocol_hysteria2_catalog import assertions_pass
from .protocol_integration_catalog import LOCAL, PAIRS, definitions
from .protocol_integration_checks import envelope, runtime_pass
from .protocol_integration_local import commands, passed
from .protocol_integration_shared import hops_pass, shared_pass
from .protocol_security_acceptance import result
def pair_pass(identifier, record, directory):
from .protocol_integration_native import rust_command
first, last = PAIRS[identifier]
targets = 2 if last == "trojan" else 3
events = read_events(directory / (identifier + "-events.jsonl"))
observation = read_json(directory / (identifier + "-observations.json"))
expected = [
dict(
outer_ipv6=v6,
nested_select=group,
tcp_target_kinds=3,
tcp_bytes_each_direction=31457280,
udp_target_kinds=targets,
udp_packets=targets * 400,
udp_sizes=[1, 64, 512, 1200],
server_first=True,
client_first=True,
ports_rebound=True,
)
for v6 in (False, True)
for group in (False, True)
]
return (
record.get("case_id") == identifier
and record.get("command") == rust_command()
and type(record.get("exit_code")) is int
and record["exit_code"] == 0
and record.get("command_cleanup") is True
and observation
== dict(
first=first,
last=last,
paths=expected,
domain_native=[
dict(
outer_ipv6=v6,
nested_select=group,
peer="XR",
udp_packets=400,
target="domain",
)
for v6 in (False, True)
for group in (False, True)
]
if last == "trojan"
else [],
budgets=[
dict(
transmit=tx,
receive=rx,
roundtrip_size=min(tx, rx),
packets=100,
oversize_rejected_before_origin=True,
)
for tx, rx in ((128, 128), (512, 256))
],
routed_udp=dict(
upstream_business_udp_disabled=True,
carrier_tcp=True,
carrier_udp=True,
leaf_business_udp_rejected=True,
),
carrier_capability=dict(
upstream_datagrams="rejected",
tcp_allowed=last != "hysteria2",
udp_allowed=last not in {"ss", "socks5", "hysteria2"},
no_bypass=True,
),
)
and assertions_pass(
events,
{
("INTEGRATION-PAIR", "ordered_pair"): 1,
("INTEGRATION-BASE", "tcp_10mib_both_directions"): 12,
("INTEGRATION-PAIR", "directional_budget"): 1,
("INTEGRATION-PAIR", "routed_udp_permission"): 1,
("INTEGRATION-PAIR", "carrier_capability"): 1,
**(
{("INTEGRATION-PAIR", "native_domain_terminal"): 1}
if last == "trojan"
else {}
),
},
)
)
def execute(cases, run, output, results):
from .protocol_containers import frozen_image
from .protocol_harness import _command
from .protocol_hysteria2_acceptance import prepare
from .protocol_integration_native import run as run_pairs
from .protocol_integration_shared import hops_run, shared_run
from .protocol_integration_suite import CONSUMERS
from .protocol_integration_suite import run as run_suite
by_id = {c["case_id"]: c for c in cases}
supplied, peers = {}, {}
source = {
k: run[k]
for k in (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
}
def save(identifier, good, cleanup=True):
results[identifier] = result(by_id[identifier], good, cleanup)
print(identifier + ": " + results[identifier]["status"], flush=True)
if not good:
raise RuntimeError("INTEGRATION gate failed: " + identifier)
try:
for identifier in sorted(by_id.keys() & LOCAL):
records, events = [], []
for i, argv in enumerate(commands(identifier)):
name = f"{identifier}-{i}"
path = output / (name + "-events.jsonl")
_command(
run,
output,
name,
[v.replace("{output}", str(output)) for v in argv],
3600,
events=path,
)
records.append(run["commands"][-1])
if path.exists():
events += read_events(path)
if records[-1]["exit_code"] != 0 or not records[-1]["cleanup"]:
save(identifier, False, records[-1]["cleanup"])
script = (
read_json(output / "script-tests.json")
if identifier == "INTEGRATION-SCRIPTS"
else None
)
save(identifier, passed(identifier, records, events, script))
if set(by_id) <= LOCAL:
return
with frozen_image(output / "container-image-pull.log") as snapshot:
run["container_image"] = snapshot
kinds = {"M"}
if "INTEGRATION-SHARED" in by_id:
kinds |= {"XR", "V2"}
if any(PAIRS[i][1] == "trojan" for i in by_id.keys() & PAIRS.keys()):
kinds.add("XR")
if "INTEGRATION-HY2-HOP" in by_id:
kinds.add("H")
supplied = prepare(output, kinds)
peers = {k: v[1] for k, v in supplied.items()}
digest = snapshot["digest"]
# Each group is rerun from this source/image/peer scope. Historical
# reports are never imported into results.
selected = sorted(by_id.keys() & PAIRS.keys())
if selected:
directory = output / "pairs"
report = run_pairs(
directory,
selected,
supplied=supplied["M"],
domain_peer=supplied.get("XR"),
)
if [r["case_id"] for r in report["cases"]] != selected:
raise RuntimeError("INTEGRATION ordered-pair selection differs")
for record in report["cases"]:
identifier = record["case_id"]
save(
identifier,
envelope(report, peers, source, digest)
and pair_pass(identifier, record, directory),
report["cleanup"],
)
selected = sorted(
by_id.keys() & CONSUMERS.keys(),
key=lambda k: (k == "INTEGRATION-SOAK", k),
)
if selected:
directory = output / "runtime"
report = run_suite(directory, selected, supplied=supplied["M"])
peers.update(report["peers"])
if [r["case_id"] for r in report["cases"]] != selected:
raise RuntimeError("INTEGRATION runtime selection differs")
for record in report["cases"]:
identifier = record["case_id"]
save(
identifier,
envelope(report, peers, source, digest)
and runtime_pass(identifier, record, directory),
report["cleanup"],
)
if "INTEGRATION-HY2-HOP" in by_id:
hops_run(output / "hop", supplied["H"])
save(
"INTEGRATION-HY2-HOP",
hops_pass(output / "hop", peers, source, digest),
)
if "INTEGRATION-SHARED" in by_id:
shared_run(
output / "shared",
{k: v for k, v in supplied.items() if k in {"M", "XR", "V2"}},
)
save(
"INTEGRATION-SHARED",
shared_pass(output / "shared", peers, source, digest),
)
finally:
(output / "peers.json").write_text(json.dumps(peers, indent=2) + "\n")
def check(run_dir, cases, run, results, peers, paths):
from .protocol_containers import IMAGE
from .protocol_evidence import HEX
from .protocol_inputs import redact
from .protocol_integration_suite import CONSUMERS
observed_paths = {
str(p.relative_to(run_dir))
for p in run_dir.rglob("*")
if p.is_file()
and p.suffix in {".json", ".jsonl", ".log", ".md"}
and "binaries" not in p.parts
and p.name != "run.json"
}
if set(paths) != observed_paths:
raise ValueError("unhashed or missing INTEGRATION evidence artifact")
snapshot = run.get("container_image", {})
digest = snapshot.get("digest", "")
if not (
digest.startswith("sha256:")
and HEX.fullmatch(digest[7:])
and snapshot
== dict(
tag=IMAGE,
digest=digest,
command=["container", "image", "pull", IMAGE],
exit_code=0,
cleanup=True,
log="container-image-pull.log",
)
and snapshot["log"] in paths
and cases == definitions()
and set(peers) == {"M", "XR", "V2", "H", "SS"}
and {"script-tests.json", "package-preparation.json"} <= set(paths)
):
raise ValueError("incomplete INTEGRATION scope, image or peer identities")
source = {
k: run[k]
for k in (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
}
records = run.get("commands", [])
names = {
f"{identifier}-{i}"
for identifier in LOCAL
for i, _ in enumerate(commands(identifier))
}
if len(records) != len(names) or {r.get("name") for r in records} != names:
raise ValueError("missing, duplicate or extra INTEGRATION local command")
pair_report = read_json(run_dir / "pairs/integration-native.json")
runtime_report = read_json(run_dir / "runtime/integration-suite.json")
expected_runtime = sorted(
{c["case_id"] for c in cases} & CONSUMERS.keys(),
key=lambda k: (k == "INTEGRATION-SOAK", k),
)
if (
[r.get("case_id") for r in pair_report.get("cases", [])] != sorted(PAIRS)
or [r.get("case_id") for r in runtime_report.get("cases", [])]
!= expected_runtime
or set(pair_report.get("peers", {})) != {"M", "XR"}
or set(runtime_report.get("peers", {})) != {"M", "SS"}
or runtime_report.get("ssserver_policy")
!= dict(
outbound_udp_allow_fragmentation=True,
inbound_udp_allow_fragmentation=True,
guest_mtu=1500,
)
):
raise ValueError(
"missing INTEGRATION ordered pairs, runtime gates or native SS policy"
)
actual = {r["case_id"]: r for r in results}
for case in cases:
identifier = case["case_id"]
if identifier in LOCAL:
selected, events = [], []
for i, argv in enumerate(commands(identifier)):
record = next(r for r in records if r["name"] == f"{identifier}-{i}")
if (
record.get("command")
!= [redact(v.replace("{output}", str(run_dir))) for v in argv]
or record.get("log") not in paths
):
raise ValueError("INTEGRATION command or log changed")
selected.append(record)
path = run_dir / f"{identifier}-{i}-events.jsonl"
if path.exists():
events += read_events(path)
good = passed(
identifier,
selected,
events,
read_json(run_dir / "script-tests.json")
if identifier == "INTEGRATION-SCRIPTS"
else None,
)
elif identifier in PAIRS or identifier in CONSUMERS:
pair = identifier in PAIRS
report = pair_report if pair else runtime_report
record = next(r for r in report["cases"] if r["case_id"] == identifier)
directory = run_dir / ("pairs" if pair else "runtime")
good = envelope(report, peers, source, digest) and (
pair_pass if pair else runtime_pass
)(identifier, record, directory)
elif identifier == "INTEGRATION-HY2-HOP":
good = hops_pass(run_dir / "hop", peers, source, digest)
else:
good = shared_pass(run_dir / "shared", peers, source, digest)
if not good or result(case, good, True) != actual[identifier]:
raise ValueError("INTEGRATION raw evidence does not pass: " + identifier)
print(
f"INTEGRATION: PASS ({len(cases)} required gates; 49 pairs, "
"100 lifetimes, 100 rebuilds, 1800-second soak)"
)
def preflight(output):
from .protocol_containers import frozen_image
from .protocol_hysteria2_acceptance import prepare
# Downloads and container package setup only; never reports behavior PASS.
with frozen_image(output / "container-image-pull.log"):
supplied = prepare(output, {"M", "XR", "V2", "H"})
(output / "peers.json").write_text(
json.dumps({k: v[1] for k, v in supplied.items()}, indent=2) + "\n"
)
@@ -0,0 +1,78 @@
"""Cross-protocol integration requirements and descriptive coverage categories."""
from __future__ import annotations
PROTOCOLS = ("socks5", "anytls", "ss", "trojan", "vmess", "vless", "hysteria2")
PAIRS = {
f"INTEGRATION-PAIR-{first.upper()}-{last.upper()}": (first, last)
for first in PROTOCOLS
for last in PROTOCOLS
}
GATES = {
"INTEGRATION-ENTRYPOINTS": "routing",
"INTEGRATION-GRAPH": "routing",
"INTEGRATION-DNS-MEASURE": "routing",
"INTEGRATION-SS-ALGORITHMS": "routing",
"INTEGRATION-SS-EIH": "routing",
"INTEGRATION-LIFECYCLE": "lifecycle",
"INTEGRATION-REBUILD": "lifecycle",
"INTEGRATION-FAILURES": "lifecycle",
"INTEGRATION-SOAK": "pressure",
"INTEGRATION-HY2-HOP": "pressure",
"INTEGRATION-DEBUG": "regression",
"INTEGRATION-RELEASE": "regression",
"INTEGRATION-QUALITY": "regression",
"INTEGRATION-FEATURES": "regression",
"INTEGRATION-SCRIPTS": "regression",
"INTEGRATION-SHARED": "regression",
}
LOCAL = {
"INTEGRATION-DEBUG",
"INTEGRATION-RELEASE",
"INTEGRATION-QUALITY",
"INTEGRATION-FEATURES",
"INTEGRATION-SCRIPTS",
}
def definitions():
result = []
for identifier in sorted(PAIRS.keys() | GATES.keys()):
pair = PAIRS.get(identifier)
local = identifier in LOCAL
result.append(
dict(
case_id=identifier,
stage="INTEGRATION",
substage=f"INTEGRATION.{GATES.get(identifier, 'routing')}",
required=True,
row_ids=["C06"] if pair else [],
protocol="integration",
network="seven-protocol-graph",
security="per-frozen-consumer",
udp_codec="per-protocol",
field_values={"first": pair[0], "last": pair[1]} if pair else {},
outer_family="IPv4/IPv6",
inner_family="IPv4/IPv6/domain",
target_type="memory-only/build" if local else "isolated origin",
upstream_graph="concrete/nested-select" if pair else "per-gate",
peer_kind="unit"
if local
else "H"
if identifier == "INTEGRATION-HY2-HOP"
else "SS"
if identifier == "INTEGRATION-SS-EIH"
else "M",
peer_config={"group": identifier},
gap_source=None,
expected_observation=[identifier],
required_evidence=["command", "structured-assertions", "cleanup"]
+ ([] if local else ["peer-identity", "isolated-origins"]),
prerequisites=["rust-toolchain"]
if local
else ["owned-host-only-container-network", "official-latest-peer"],
runner="integration-pair" if pair else "integration-gate",
timeout_seconds=3600 if not pair else 300,
)
)
return result
@@ -0,0 +1,189 @@
"""Raw runtime observations and native isolation, independent of run status."""
from __future__ import annotations
from .protocol_evidence import HEX, idle_resources, read_events, read_json
from .protocol_hysteria2_catalog import assertions_pass
from .protocol_integration_catalog import PROTOCOLS
from .protocol_integration_metrics import lifetimes, rebuild, soak
from .protocol_integration_native import rust_command
from .protocol_integration_suite import CONSUMERS
def envelope(report, peers, source, digest, *, single=None):
isolation = report.get("isolation", {})
identities = {single: report.get("peer", {})} if single else report.get("peers", {})
owned = isolation.get("peers", [])
urls = {
"M": "MetaCubeX/mihomo",
"XR": "XTLS/Xray-core",
"V2": "v2fly/v2ray-core",
"SS": "shadowsocks/shadowsocks-rust",
"H": "HyNetworks/hysteria",
}
return bool(
report.get("status") == "PASS"
and report.get("cleanup") is True
and report.get("source_unchanged") is True
and report.get("source") == source
and isolation.get("network_mode") == "hostOnly"
and isolation.get("host_servers") is False
and isolation.get("guest_mtu") == 1500
and isolation.get("image_digest") == digest
and digest.startswith("sha256:")
and HEX.fullmatch(digest[7:])
and owned
and all(p.get("started") is True and p.get("joined") is True for p in owned)
and identities
and all(
kind in urls
and identity == peers.get(kind)
and identity.get("version")
and HEX.fullmatch(identity.get("binary_sha256", ""))
and (kind == "H" or HEX.fullmatch(identity.get("archive_sha256", "")))
and identity.get("source_url", "").startswith(
f"https://github.com/{urls[kind]}/releases/"
)
for kind, identity in identities.items()
)
)
def runtime_pass(identifier, record, directory):
if not (
record.get("case_id") == identifier
and record.get("command") == rust_command(CONSUMERS[identifier])
and type(record.get("exit_code")) is int
and record["exit_code"] == 0
and record.get("command_cleanup") is True
):
return False
value = read_json(directory / (identifier + "-observations.json"))
events = read_events(directory / (identifier + "-events.jsonl"))
protocols = list(PROTOCOLS)
expected = {(identifier, p): 1 for p in protocols}
if identifier == "INTEGRATION-ENTRYPOINTS":
good = value == dict(
entrypoints=[
dict(
protocol=p,
http_forward=True,
http_connect=True,
tun_tcp=True,
tun_udp=True,
tun_dns=True,
)
for p in protocols
]
)
elif identifier == "INTEGRATION-GRAPH":
good = value == dict(
protocols=protocols,
nested_select=True,
old_transport_snapshot=True,
cold_reject=True,
direct=True,
unselected_cycle=True,
no_fallback=True,
)
elif identifier == "INTEGRATION-DNS-MEASURE":
good = value == dict(
protocols=protocols,
controlled_dns_via_proxy=True,
ip_rule_consumer=True,
node_measure=True,
group_measure_rejected=True,
)
elif identifier == "INTEGRATION-FAILURES":
expected = {
(suite, p): 1
for suite in (
identifier,
"INTEGRATION-FAILURES-AUTH",
"INTEGRATION-FAILURES-SOURCE",
)
for p in protocols
}
good = value == dict(
protect_rejected=protocols,
auth_rejected=protocols,
certificate_pin_rejected=[
"anytls",
"trojan",
"vmess",
"vless",
"hysteria2",
],
source_and_oversize_isolated=protocols,
cancel_preserves_sibling=protocols,
no_origin_bytes=True,
stop_idle=True,
)
good &= all(
idle_resources(e.get("resources"))
for e in events
if e.get("status") == "PASS"
and e.get("suite") != "INTEGRATION-FAILURES-AUTH"
)
elif identifier in {"INTEGRATION-SS-ALGORITHMS", "INTEGRATION-SS-EIH"}:
eih = identifier == "INTEGRATION-SS-EIH"
ciphers = ["2022-blake3-aes-128-gcm", "2022-blake3-aes-256-gcm"] + (
[] if eih else ["2022-blake3-chacha20-poly1305"]
)
expected = {(identifier, c): 1 for c in ciphers} | {
("INTEGRATION-BASE", "tcp_10mib_both_directions"): 8 if eih else 9
}
good = value == (
dict(
native_terminal_eih=[
dict(
cipher=c,
identity_depth=1,
outer_families=2,
tcp_targets=2,
tcp_bytes_each_direction=41943040,
udp_targets=2,
udp_packets=2000,
wrong_identity_rejected=True,
wrong_user_rejected=True,
)
for c in ciphers
],
arbitrary_relay_claim=False,
)
if eih
else dict(
algorithms=[
dict(
cipher=c,
tcp_targets=3,
tcp_bytes_each_direction=31457280,
udp_targets=3,
udp_packets=1500,
wrong_key_rejected=True,
)
for c in ciphers
]
)
)
elif identifier == "INTEGRATION-LIFECYCLE":
expected = {(identifier, "stop_and_remain_quiet"): 100}
good = lifetimes(value, events)
else:
name = (
"forty_flows_same_session"
if identifier == "INTEGRATION-REBUILD"
else "mixed_forty_flows"
)
expected = {(identifier, name): 1}
passes = [e for e in events if e.get("status") == "PASS"]
good = len(passes) == 1 and (
rebuild(value, passes[0])
if identifier == "INTEGRATION-REBUILD"
else soak(
value,
passes[0],
read_events(directory / (identifier + "-observations.json.jsonl")),
)
)
return bool(good and assertions_pass(events, expected))
@@ -0,0 +1,98 @@
"""Explicit memory-only/build commands; never run historical host-server suites."""
from __future__ import annotations
import sys
from .core_checks import commands as core_commands
from .protocol_hysteria2_acceptance import commands as hysteria2_commands
def commands(identifier):
if identifier in {
"INTEGRATION-DEBUG",
"INTEGRATION-RELEASE",
"INTEGRATION-FEATURES",
}:
return core_commands(identifier.removeprefix("INTEGRATION-").lower())
if identifier == "INTEGRATION-QUALITY":
return hysteria2_commands("HYSTERIA2-QUALITY") + [
["vcore-scripts", "check", "protocol-coverage", "--catalog-only"],
["vcore-scripts", "build", "apple"],
["vcore-scripts", "build", "android"],
]
if identifier == "INTEGRATION-SCRIPTS":
return [
[
sys.executable,
"-m",
"vcore_scripts.protocol_script_tests",
"{output}/script-tests.json",
]
]
return []
def passed(identifier, records, events, script=None):
from .protocol_hysteria2_catalog import CFG, UNIT, assertions_pass
from .protocol_integration_catalog import PROTOCOLS
from .protocol_security_units import TESTS
good = bool(records) and all(
type(r.get("exit_code")) is int
and r["exit_code"] == 0
and r.get("cleanup") is True
for r in records
)
if identifier in {"INTEGRATION-DEBUG", "INTEGRATION-RELEASE"}:
expected = {
("SECURITY-CONFIG-TLS", name): 1
for names in TESTS.values()
for name in names
}
expected.update({("HYSTERIA2-CFG", name): 1 for name in CFG})
expected.update({("HYSTERIA2-UNIT", name): 1 for name in UNIT})
expected.update({("INTEGRATION-FEATURE", name): 1 for name in PROTOCOLS})
expected[("INTEGRATION-ADAPTER", "growing_caller_buffer")] = 1
expected[("INTEGRATION-ADAPTER", "server_first_buffered_upstream")] = 1
expected[("INTEGRATION-ADAPTER", "hysteria2_fragment_id_reuse")] = 1
selected = [
e
for e in events
if e.get("suite")
in {
"SECURITY-CONFIG-TLS",
"HYSTERIA2-CFG",
"HYSTERIA2-UNIT",
"INTEGRATION-FEATURE",
"INTEGRATION-ADAPTER",
}
]
good &= assertions_pass(selected, expected) and all(
e.get("status") in {"BEGIN", "PASS"} for e in events
)
if identifier == "INTEGRATION-FEATURES":
good &= assertions_pass(
[e for e in events if e.get("suite") == "INTEGRATION-FEATURE"],
{("INTEGRATION-FEATURE", p): 7 for p in PROTOCOLS},
)
if identifier == "INTEGRATION-SCRIPTS":
rows = (script or {}).get("cases", [])
names = [r.get("test") for r in rows]
required = {
"test_protocol_integration.IntegrationAcceptanceTest." + name
for name in (
"test_integration_consumer_results_keep_the_manifest_scope",
"test_pair_evidence_requires_each_real_path_and_original_assertions",
"test_list_contains_all_ordered_pairs_and_remaining_gates",
"test_pressure_evidence_is_recomputed_not_inferred",
"test_native_source_identity_and_cleanup_are_required",
)
}
good &= (
bool(rows)
and len(names) == len(set(names)) == (script or {}).get("tests_run")
and required <= set(names)
and all(r.get("status") == "PASS" for r in rows)
)
return bool(good)
@@ -0,0 +1,220 @@
"""Recompute lifecycle/pressure gates from bounded, persisted observations."""
from __future__ import annotations
from math import isfinite
from statistics import median
from .protocol_evidence import RESOURCE_KINDS, idle_resources
NEW = ["trojan", "vmess", "vless", "hysteria2"]
QUEUES = {
"socks_udp": 16,
"hysteria2_udp": 32,
"quic_incoming": 32,
"quic_outgoing": 32,
}
def integer(value, minimum=0):
return type(value) is int and value >= minimum
def resources(value, *, active=False):
rows = value.get("counts", [])
if len(rows) != 8 or {r.get("kind") for r in rows} != RESOURCE_KINDS:
return False
if not all(
integer(r.get("current"))
and integer(r.get("peak"))
and r["current"] <= r["peak"]
for r in rows
):
return False
counts = {r["kind"]: r["current"] for r in rows}
return not active or all(
counts[k] > 0 for k in ("task", "socket", "session", "association")
)
def sample(value, *, active=True):
queues = value.get("queues", [])
return (
integer(value.get("heap_in_use"), 1)
and integer(value.get("rss_kib"), 1)
and integer(value.get("fd"), 1)
and resources(value.get("resources", {}), active=active)
and len(queues) == 4
and {q.get("kind"): q.get("capacity") for q in queues} == QUEUES
and all(
integer(q.get("peak"), 1) and q["peak"] <= q["capacity"] for q in queues
)
)
def lifetimes(value, events):
cycles = value.get("cycles", [])
passes = [e for e in events if e.get("status") == "PASS"]
if value.get("count") != 100 or len(cycles) != 100 or len(passes) != 100:
return False
for index, (row, event) in enumerate(zip(cycles, passes, strict=True)):
stop = row.get("after_stop", {})
checkpoints = event.get("checkpoints", [])
mode = index % 5
retained = {0: 0, 1: 10, 2: 1, 3: 0, 4: 8}[mode]
if not (
row.get("cycle") == index
and row.get("protocol") == NEW[(index // 5) % 4]
and row.get("mode") == mode
and integer(row.get("baseline_fd"), 1)
and row.get("retained_fixture_fd") == retained
and integer(row.get("after_stop_fd"), 1)
and row["after_stop_fd"] <= row["baseline_fd"] + retained
and integer(row.get("final_fd"), 1)
and row["final_fd"] <= row["baseline_fd"]
and integer(row.get("stop_ms"))
and row["stop_ms"] < 5000
and isfinite(row.get("quiet_seconds", 0))
and row.get("quiet_seconds", 0) >= 5
and row.get("ports_rebound") is True
and idle_resources(stop)
and row.get("quiet") == stop == event.get("resources")
and len(checkpoints) == 4
and [p.get("phase") for p in checkpoints]
== ["baseline", "active", "after-stop", "quiet"]
and idle_resources(checkpoints[0].get("resources"))
and checkpoints[1].get("resources") == row.get("active")
and resources(row.get("active", {}))
and checkpoints[2].get("resources")
== stop
== checkpoints[3].get("resources")
):
return False
if mode in (1, 4) and not resources(row["active"], active=True):
return False
return True
def stopped(value, event):
cold, baseline = value.get("cold_fd"), value.get("baseline_fd")
row = value.get("stopped", {})
after, quiet = row.get("after_stop", {}), row.get("quiet", {})
return (
integer(cold, 1)
and integer(baseline, 1)
and 0 <= baseline - cold <= 2
and integer(row.get("stop_ms"))
and row["stop_ms"] < 5000
and isfinite(row.get("quiet_seconds", 0))
and row.get("quiet_seconds", 0) >= 5
and row.get("ports_rebound") is True
and sample(after, active=False)
and sample(quiet, active=False)
and idle_resources(after.get("resources"))
and quiet.get("resources") == after.get("resources") == event.get("resources")
and after["fd"] <= baseline
and quiet["fd"] <= baseline
)
def setup(values):
return (
isinstance(values, list)
and len(values) == 40
and all(integer(v, 1) and v < 10_000_000 for v in values)
)
def rebuild(value, event):
rows = value.get("cycles", [])
return (
value.get("count") == 100
and value.get("same_running_session") is True
and value.get("protocols") == NEW
and len(rows) == 100
and stopped(value, event)
and all(
row.get("generation") == i
and row.get("tcp") == row.get("udp") == 20
and row.get("per_protocol_tcp") == row.get("per_protocol_udp") == 5
and sample(row.get("active", {}))
and sample(row.get("after_clients", {}), active=False)
and setup(row.get("setup_us"))
and row.get("fault") == "owned-peer-connections-closed"
and row.get("new_clients") is True
for i, row in enumerate(rows)
)
)
def soak(value, event, curve):
points, faults, switches = (
value.get("samples", []),
value.get("faults", []),
value.get("switches", []),
)
if not (
value.get("requested_seconds") == 1800
and 1800 <= value.get("seconds", 0) < 1860
and value.get("protocols") == NEW
and value.get("tcp") == value.get("udp") == 20
and value.get("per_protocol_tcp") == value.get("per_protocol_udp") == 5
and integer(value.get("waves"), 1)
and value.get("normal_verified_bytes") == value["waves"] * 20 * 2 * 2 * 257
and all(
value.get(k) == 0
for k in (
"normal_corruption",
"normal_misdirection",
"normal_unexpected_loss",
)
)
and len(points) >= 25
and points == curve
and 29 <= len(faults) <= 30
and len(switches) >= 1799
and all(i <= t < i + 5 for i, t in enumerate(switches, 1))
and all(
60 * i <= row.get("start", 0) < 60 * i + 5
and row["start"] <= row.get("end", 0) < row["start"] + 10
and row.get("old_tcp_closed")
== row.get("new_tcp")
== row.get("new_udp")
== 20
and row.get("replay") is False
for i, row in enumerate(faults, 1)
)
and all(
sample(p) and 300 + 60 * i <= p.get("seconds", 0) < 310 + 60 * i
for i, p in enumerate(points)
)
and stopped(value, event)
and sample(value.get("active_end", {}))
and setup(value.get("first_setup_us"))
and setup(value.get("last_setup_us"))
):
return False
first_setup, last_setup = (
int(median(value[k])) for k in ("first_setup_us", "last_setup_us")
)
early, late = points[:10], points[-10:]
first, last = (
int(median(p["heap_in_use"] for p in rows)) for rows in (early, late)
)
allowed = max(1024 * 1024, first // 20)
def counts(rows, kind):
return max(
next(r["current"] for r in p["resources"]["counts"] if r["kind"] == kind)
for p in rows
)
return (
first_setup == value.get("first_setup_median_us")
and last_setup == value.get("last_setup_median_us") <= 2 * first_setup
and value.get("heap")
== dict(early_median=first, late_median=last, allowed_growth=allowed)
and last <= first + allowed
and all(counts(late, kind) <= counts(early, kind) for kind in RESOURCE_KINDS)
and max(p["fd"] for p in late) <= max(p["fd"] for p in early)
)
@@ -0,0 +1,305 @@
"""Seven-protocol integration with owned official peers and container origins."""
from __future__ import annotations
import base64
import contextlib
import json
import os
import shutil
import sys
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .native_release import download_native
from .protocol_containers import ContainerLab, command, frozen_image
from .protocol_fixtures import certificates
from .protocol_inputs import redact, sha256, source_identity
from .protocol_integration_catalog import PAIRS
from .protocol_peers import run_command
def peer_configuration(protocol, server, origin, directory, *, certificate=None):
cert, key, pin = certificate or certificates(directory)
listener = dict(name="integration", type=protocol, listen="::", port=23000)
node = dict(name="peer", type=protocol, server=server, port=23000, udp=True)
password = "synthetic-integration-fixture"
identity = "07070707-0707-0707-0707-070707070707"
if protocol == "socks5":
listener.update(
type="socks", udp=True, users=[dict(username="fixture", password=password)]
)
node.update(username="fixture", password=password)
elif protocol == "ss":
password = base64.b64encode(bytes([7]) * 16).decode()
cipher = "2022-blake3-aes-128-gcm"
listener.update(type="shadowsocks", udp=True, cipher=cipher, password=password)
node.update(cipher=cipher, password=password)
elif protocol in {"trojan", "anytls", "hysteria2"}:
node.update(password=password, sni="localhost", fingerprint=pin)
listener["users"] = (
[dict(username="fixture", password=password)]
if protocol == "trojan"
else {"fixture": password}
)
else:
node.update(uuid=identity, tls=True, servername="localhost", fingerprint=pin)
listener["users"] = [dict(username="fixture", uuid=identity)]
if protocol == "vmess":
node.update(cipher="auto", alterId=0)
listener["users"][0]["alterId"] = 0
node["packet-encoding"] = "xudp"
if protocol not in {"socks5", "ss"}:
listener.update(
certificate=f"/data/fixture/{cert.name}",
**{"private-key": f"/data/fixture/{key.name}"},
)
return node, {
"socks-port": 23999,
"allow-lan": True,
"bind-address": "*",
"ipv6": True,
"log-level": "silent",
"hosts": {"vcore-fixture.test": origin},
"listeners": [listener],
"rules": ["MATCH,DIRECT"],
}
def rust_command(consumer="ordered_pair"):
return [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_public",
f"integration::{consumer}",
"--",
"--exact",
"--ignored",
"--nocapture",
]
def run(
output: Path, selected, *, supplied=None, consumer="ordered_pair", domain_peer=None
):
if (
not selected
or len(selected) != len(set(selected))
or not set(selected) <= PAIRS.keys()
):
raise ValueError("invalid INTEGRATION pair selection")
output = output.resolve()
if not output.is_relative_to((CORE_DIR / "target/interop/runs").resolve()):
raise ValueError(
"INTEGRATION evidence must stay inside the owned run directory"
)
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="INTEGRATION",
scope="ordered-pair-subset" if consumer == "ordered_pair" else "diagnostic",
source=source_identity(),
status="NOT RUN",
cases=[],
peers={},
isolation={},
)
try:
if supplied is None:
identity = {}
binary = download_mihomo(
"linux-arm64", directory=output / "binaries", identity=identity
)
else:
binary, identity = supplied
if sha256(binary) != identity["binary_sha256"]:
raise RuntimeError("INTEGRATION native peer identity mismatch")
report["peers"]["M"] = identity
if any(PAIRS[key][1] == "trojan" for key in selected):
if domain_peer is None:
artifact = download_native(
"XR", output / "binary-xray", "linux-arm64", defer_version=True
)
domain_peer = (artifact.binary, artifact.identity)
if sha256(domain_peer[0]) != domain_peer[1]["binary_sha256"]:
raise RuntimeError("INTEGRATION domain terminal identity mismatch")
report["peers"]["XR"] = domain_peer[1]
lab = ContainerLab(report["isolation"], mtu=1500)
for index, identifier in enumerate(selected):
first_kind, last_kind = PAIRS[identifier]
print(f"INTEGRATION: {identifier}", flush=True)
with tempfile.TemporaryDirectory(
prefix="private-", dir=output
) as temporary:
root = Path(temporary)
with contextlib.ExitStack() as stack:
origin_dir = root / "origin"
origin_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
origin = lab.start(
stack,
origin_dir,
f"{index}-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
origin.release()
origin.wait_tcp(24000)
fixture = dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
data_dir=str(root / "core"),
)
for role, protocol in (("first", first_kind), ("last", last_kind)):
directory = root / role
directory.mkdir()
shutil.copy2(binary, directory / "peer")
peer = lab.start(
stack,
directory,
f"{index}-{role}",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
identity["version"] = command(
"exec", peer.name, "/data/fixture/peer", "-v"
).strip()
if (
command(
"exec", peer.name, "sha256sum", "/data/fixture/peer"
).split()[0]
!= identity["binary_sha256"]
):
raise RuntimeError(
"INTEGRATION container binary differs from download"
)
node, config = peer_configuration(
protocol, peer.ipv4, origin.ipv4, directory
)
(directory / "config.json").write_text(json.dumps(config))
fixture[role], fixture[role + "_ipv6"] = node, peer.ipv6
peer.release()
peer.wait_tcp(23999)
if last_kind == "trojan":
from .protocol_fixtures import trojan_peer_config as peer_config
directory = root / "domain"
directory.mkdir()
shutil.copy2(domain_peer[0], directory / "peer")
peer = lab.start(
stack,
directory,
f"{index}-domain",
[
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
],
)
domain_peer[1]["version"] = command(
"exec", peer.name, "/data/fixture/peer", "version"
).strip()
if (
command(
"exec", peer.name, "sha256sum", "/data/fixture/peer"
).split()[0]
!= domain_peer[1]["binary_sha256"]
):
raise RuntimeError(
"INTEGRATION domain container binary identity mismatch"
)
certificate = certificates(directory)
node, _ = peer_configuration(
"trojan",
peer.ipv4,
origin.ipv4,
directory,
certificate=certificate,
)
cert, key, _ = certificate
config = peer_config(
"XR",
"tcp",
23000,
node["password"],
f"/data/fixture/{cert.name}",
f"/data/fixture/{key.name}",
)
config["inbounds"][0]["listen"] = "::"
config["dns"]["hosts"] = {"vcore-fixture.test": origin.ipv4}
(directory / "config.json").write_text(json.dumps(config))
fixture["domain_last"], fixture["domain_last_ipv6"] = (
node,
peer.ipv6,
)
peer.release()
peer.wait_tcp(23000)
path = root / "fixture.json"
path.write_text(json.dumps(fixture))
events = output / (identifier + "-events.jsonl")
observations = output / (identifier + "-observations.json")
env = dict(
os.environ,
VCORE_VLESS_INPUT=str(path),
VCORE_PROTOCOL_STAGE="INTEGRATION",
VCORE_CASE_EVENTS=str(events),
VCORE_INTEGRATION_OBSERVATIONS=str(observations),
)
argv = rust_command(consumer)
result = run_command(argv, cwd=CORE_DIR, env=env, timeout=300)
(output / (identifier + ".log")).write_text(
redact(result.stdout.decode(errors="replace"))
)
report["cases"].append(
dict(
case_id=identifier,
command=argv,
exit_code=result.returncode,
command_cleanup=result.cleanup,
seconds=result.seconds,
)
)
if result.returncode != 0 or not result.cleanup:
raise RuntimeError("INTEGRATION ordered-pair consumer failed")
report["status"] = "PASS"
except BaseException as error:
report.update(status="FAIL", failure_kind=type(error).__name__)
raise
finally:
report["source_unchanged"] = source_identity() == report["source"]
report["cleanup"] = all(
peer.get("joined") is True for peer in report["isolation"].get("peers", [])
)
if not report["cleanup"] or not report["source_unchanged"]:
report["status"] = "FAIL"
(output / "integration-native.json").write_text(
json.dumps(report, indent=2) + "\n"
)
return report
if __name__ == "__main__":
output = Path(sys.argv[1]).resolve()
with exclusive_run(), frozen_image(None):
run(output, sys.argv[2:] or list(PAIRS))
@@ -0,0 +1,91 @@
"""Frozen strongly-coupled TLS/transport and uninterrupted hopping regressions."""
from __future__ import annotations
from .protocol_evidence import read_json
from .protocol_integration_checks import envelope
from .protocol_security_acceptance import CHECKERS, REPORTS
from .protocol_security_catalog import groups
SHARED = (
"SECURITY-SHARED-NONE",
"SECURITY-SHARED-CHROME",
"SECURITY-SHARED-XHTTP",
"SECURITY-HYBRID-RETAINED",
"SECURITY-JLS-RETAINED",
"SECURITY-ECH-CHROME",
"SECURITY-ECH-ENCRYPTION-NATIVE-0RTT-MIXED",
"SECURITY-JLS-ENCRYPTION-NATIVE-0RTT-MIXED",
"SECURITY-HYBRID-ENCRYPTION-NATIVE-0RTT-MIXED",
)
HOPS = (
"HYSTERIA2-HOP-V4-PLAIN-FIXED",
"HYSTERIA2-HOP-V4-OBFS-RANDOM",
"HYSTERIA2-HOP-V6-PLAIN-RANDOM",
"HYSTERIA2-HOP-V6-OBFS-FIXED",
)
def shared_run(directory, supplied):
from .protocol_reality_hybrid import run as hybrid
from .protocol_vless_container import run as vless
from .protocol_xhttp_fields import run as xhttp
catalog = groups()
for name in SHARED:
print("INTEGRATION shared: " + name, flush=True)
group = catalog[name]
options = {k: v for k, v in group.items() if k not in {"rows", "runner"}}
if group["runner"] == "xhttp":
options["selected"] = list(options["jobs"])
{"vless": vless, "xhttp": xhttp, "hybrid": hybrid}[group["runner"]](
directory / name, supplied=supplied, **options
)
def shared_pass(directory, peers, source, digest):
catalog = groups()
for name in SHARED:
group = catalog[name]
root = directory / name
report = read_json(root / REPORTS[group["runner"]])
if not envelope(report, peers, source, digest) or not CHECKERS[group["runner"]](
group, report, root
):
return False
return True
def hops_run(directory, supplied):
from .protocol_hysteria2_catalog import H
from .protocol_hysteria2_hop import run
for name in HOPS:
print("INTEGRATION uninterrupted: " + name, flush=True)
run(directory / name, supplied=supplied, **H[name])
def hops_pass(directory, peers, source, digest):
from .protocol_hysteria2_acceptance import native_result
from .protocol_hysteria2_catalog import definitions
catalog = {c["case_id"]: c for c in definitions()}
prep = read_json(directory.parent / "package-preparation.json")
if not (
prep.get("firewall_packages")
and prep.get("peers")
and all(p.get("joined") is True for p in prep["peers"])
and prep.get("image_digest") == digest
):
return False
for name in HOPS:
root = directory / name
report = read_json(root / "report.json")
if not (
envelope(report, peers, source, digest, single="H")
and native_result(catalog[name], report, root)["status"] == "PASS"
and report["isolation"].get("firewall_packages")
== prep["firewall_packages"]
):
return False
return True
@@ -0,0 +1,351 @@
"""INTEGRATION public-runtime gates against seven listeners in an owned native peer."""
from __future__ import annotations
import base64
import contextlib
import json
import os
import re
import shutil
import sys
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .native_release import download_native
from .protocol_containers import ContainerLab, command, frozen_image
from .protocol_fixtures import certificates
from .protocol_inputs import redact, sha256, source_identity
from .protocol_integration_catalog import PROTOCOLS
from .protocol_integration_native import peer_configuration, rust_command
from .protocol_peers import run_command
CONSUMERS = {
"INTEGRATION-ENTRYPOINTS": "entrypoints",
"INTEGRATION-GRAPH": "graph",
"INTEGRATION-DNS-MEASURE": "dns_measure",
"INTEGRATION-RESOURCE-TRACER": "resource_tracer",
"INTEGRATION-LIFECYCLE": "lifecycle::public_lifetimes",
"INTEGRATION-LIFECYCLE-TRACER": "lifecycle::lifecycle_tracer",
"INTEGRATION-REBUILD": "pressure::rebuild",
"INTEGRATION-REBUILD-TRACER": "pressure::rebuild_tracer",
"INTEGRATION-SOAK": "pressure::soak",
"INTEGRATION-SOAK-TRACER": "pressure::soak_tracer",
"INTEGRATION-SS-ALGORITHMS": "shadowsocks::algorithms",
"INTEGRATION-SS-EIH": "shadowsocks::eih",
"INTEGRATION-FAILURES": "connector::protect_failure",
}
def run(output: Path, selected, *, supplied=None):
if (
not selected
or len(selected) != len(set(selected))
or not set(selected) <= CONSUMERS.keys()
):
raise ValueError("invalid INTEGRATION public gate selection")
output = output.resolve()
if not output.is_relative_to((CORE_DIR / "target/interop/runs").resolve()):
raise ValueError(
"INTEGRATION evidence must stay inside the owned run directory"
)
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="INTEGRATION",
scope="public-runtime-subset",
source=source_identity(),
status="NOT RUN",
cases=[],
peers={},
isolation={},
)
try:
if supplied is None:
identity = {}
binary = download_mihomo(
"linux-arm64", directory=output / "binaries", identity=identity
)
else:
binary, identity = supplied
if sha256(binary) != identity["binary_sha256"]:
raise RuntimeError("INTEGRATION native peer identity mismatch")
report["peers"]["M"] = identity
lab = ContainerLab(report["isolation"], mtu=1500)
with tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary:
root = Path(temporary)
with contextlib.ExitStack() as stack:
origin_dir = root / "origin"
origin_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
origin = lab.start(
stack,
origin_dir,
"origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"VCORE_ORIGIN_PROFILE=INTEGRATION",
"python",
"-B",
"/data/fixture/origin.py",
],
)
origin.release()
origin.wait_tcp(24000)
upstream_dir = root / "upstream"
upstream_dir.mkdir()
shutil.copy2(binary, upstream_dir / "peer")
upstream = lab.start(
stack,
upstream_dir,
"upstream",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
hop, upstream_config = peer_configuration(
"socks5", upstream.ipv4, origin.ipv4, upstream_dir
)
hop["name"] = "hop"
(upstream_dir / "config.json").write_text(json.dumps(upstream_config))
upstream.release()
upstream.wait_tcp(23999)
if (
command(
"exec", upstream.name, "sha256sum", "/data/fixture/peer"
).split()[0]
!= identity["binary_sha256"]
):
raise RuntimeError(
"INTEGRATION upstream binary differs from download"
)
directory = root / "native"
directory.mkdir()
shutil.copy2(binary, directory / "peer")
peer = lab.start(
stack,
directory,
"native",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
identity["version"] = command(
"exec", peer.name, "/data/fixture/peer", "-v"
).strip()
if (
command(
"exec", peer.name, "sha256sum", "/data/fixture/peer"
).split()[0]
!= identity["binary_sha256"]
):
raise RuntimeError(
"INTEGRATION container binary differs from download"
)
certificate = certificates(directory)
nodes, listeners = {}, []
config = None
for index, protocol in enumerate(PROTOCOLS):
node, current = peer_configuration(
protocol,
peer.ipv4,
origin.ipv4,
directory,
certificate=certificate,
)
port = 23000 + index
node["port"] = port
current["listeners"][0].update(name=protocol, port=port)
nodes[protocol] = node
listeners.extend(current["listeners"])
config = current
ss_nodes = [nodes["ss"].copy()]
for index, cipher in enumerate(
("2022-blake3-aes-256-gcm", "2022-blake3-chacha20-poly1305")
):
node = nodes["ss"].copy()
node.update(
port=23010 + index,
cipher=cipher,
password=base64.b64encode(bytes([7]) * 32).decode(),
)
listeners.append(
dict(
name=cipher,
type="shadowsocks",
listen="::",
port=node["port"],
cipher=cipher,
password=node["password"],
udp=True,
)
)
ss_nodes.append(node)
config.update(
listeners=listeners,
**{
"external-controller": "0.0.0.0:23998",
"secret": "synthetic-integration-control",
},
)
(directory / "config.json").write_text(json.dumps(config))
peer.release()
peer.wait_tcp(23999)
fixture = dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
nodes=nodes,
ss_nodes=ss_nodes,
hop=hop,
server_ipv6=peer.ipv6,
peer_controller=f"{peer.ipv4}:23998",
data_dir=str(root / "core"),
)
if "INTEGRATION-SS-EIH" in selected:
artifact = download_native(
"SS", output / "binaries-ss", "linux-arm64", defer_version=True
)
ss_dir = root / "ssserver"
ss_dir.mkdir()
shutil.copy2(artifact.binary, ss_dir / "ssserver")
ss = lab.start(
stack,
ss_dir,
"ssserver",
["/data/fixture/ssserver", "-c", "/data/fixture/config.json"],
)
artifact.identity["version"] = command(
"exec", ss.name, "/data/fixture/ssserver", "--version"
).strip()
if (
command(
"exec", ss.name, "sha256sum", "/data/fixture/ssserver"
).split()[0]
!= artifact.identity["binary_sha256"]
):
raise RuntimeError("INTEGRATION SS binary identity mismatch")
report["peers"]["SS"] = artifact.identity
servers, eih_nodes = [], []
for index, length in enumerate((16, 32)):
cipher = f"2022-blake3-aes-{length * 8}-gcm"
identity_key, user_key = [
base64.b64encode(bytes([v]) * length).decode()
for v in (9, 8)
]
port = 23020 + index
servers.append(
dict(
server="::",
server_port=port,
method=cipher,
password=identity_key,
users=[dict(name="fixture", password=user_key)],
mode="tcp_and_udp",
)
)
eih_nodes.append(
dict(
name="peer",
type="ss",
server=ss.ipv4,
port=port,
cipher=cipher,
password=f"{identity_key}:{user_key}",
udp=True,
)
)
(ss_dir / "config.json").write_text(
json.dumps(
dict(
servers=servers,
outbound_udp_allow_fragmentation=True,
inbound_udp_allow_fragmentation=True,
)
)
)
report["ssserver_policy"] = dict(
outbound_udp_allow_fragmentation=True,
inbound_udp_allow_fragmentation=True,
guest_mtu=1500,
)
ss.release()
ss.wait_tcp(23020)
fixture["eih_nodes"] = eih_nodes
fixture["eih_ipv6"] = ss.ipv6
path = root / "fixture.json"
path.write_text(json.dumps(fixture))
for identifier in selected:
print(f"INTEGRATION: {identifier}", flush=True)
env = dict(
os.environ,
VCORE_VLESS_INPUT=str(path),
VCORE_PROTOCOL_STAGE="INTEGRATION",
VCORE_CASE_EVENTS=str(output / (identifier + "-events.jsonl")),
VCORE_INTEGRATION_OBSERVATIONS=str(
output / (identifier + "-observations.json")
),
)
argv = rust_command(CONSUMERS[identifier])
result = run_command(argv, cwd=CORE_DIR, env=env, timeout=3600)
(output / (identifier + ".log")).write_text(
redact(result.stdout.decode(errors="replace"))
)
report["cases"].append(
dict(
case_id=identifier,
command=argv,
exit_code=result.returncode,
command_cleanup=result.cleanup,
seconds=result.seconds,
)
)
if "INTEGRATION-SS-EIH" in selected:
# The follow-log owner buffers while live; snapshot via
# the CLI so a short failed run retains real diagnostics.
text = command("logs", ss.name)
for server in servers:
for key in (
server["password"],
server["users"][0]["password"],
):
text = text.replace(key, "<synthetic-key>")
text = re.sub(r"(?:\d{1,3}\.){3}\d{1,3}", "<fixture-ip>", text)
(output / "ssserver.log").write_text(redact(text))
if result.returncode != 0 or not result.cleanup:
raise RuntimeError("INTEGRATION public-runtime consumer failed")
report["status"] = "PASS"
except BaseException as error:
report.update(status="FAIL", failure_kind=type(error).__name__)
raise
finally:
report["source_unchanged"] = source_identity() == report["source"]
report["cleanup"] = all(
peer.get("joined") is True for peer in report["isolation"].get("peers", [])
)
if not report["cleanup"] or not report["source_unchanged"]:
report["status"] = "FAIL"
(output / "integration-suite.json").write_text(
json.dumps(report, indent=2) + "\n"
)
return report
if __name__ == "__main__":
with exclusive_run(), frozen_image(None):
run(Path(sys.argv[1]), sys.argv[2:] or list(CONSUMERS))
+104
View File
@@ -0,0 +1,104 @@
"""JLS public consumers against official latest Mihomo in isolated containers."""
from __future__ import annotations
from pathlib import Path
def catalog():
cases = {}
def add(mode, suffix, test):
cases[f"SECURITY-JLS-{mode.removesuffix('-tls').upper()}-{suffix}"] = (
"M",
mode,
True,
test,
)
modes = (
"tcp-tls",
"ws-tls",
"upgrade-tls",
"upgrade-fast-tls",
"grpc-tls",
) + tuple(
f"xhttp-{mode}{version}"
for mode in (
"stream-one",
"stream-up",
"packet-up",
"stream-up-download",
"packet-up-download",
)
for version in ("", "-h1")
)
for mode in modes:
add(mode, "BASE", "public_base")
add(mode, "CLOSE", "native_mihomo_close_alignment")
for mode in (
"tcp-tls",
"ws-tls",
"grpc-tls",
"xhttp-stream-up-download",
"xhttp-stream-up-download-h1",
):
add(mode, "AUTH", "native_jls_fail_closed")
for mode in modes:
if "download" in mode:
add(mode, "IDENTITY", "public_jls_download_identity")
for mode in ("tcp-tls", "grpc-tls"):
for suffix, test in (
("GRAPH", "runtime::public_graph"),
("IPV6", "runtime::public_ipv6_and_gates"),
("ENTRYPOINTS", "runtime::public_entrypoints"),
("UDP-ISOLATION", "runtime::public_udp_isolation"),
):
add(mode, suffix, test)
for mode in ("grpc-tls", "xhttp-stream-up-download"):
add(mode, "LIFE", "runtime::public_lifecycle")
add(mode, "OWNED", "runtime::owned_resources")
return cases
def configuration(node, peer, origin, mode):
credentials = dict(username="synthetic-jls-user", password="synthetic-jls-password")
download = dict(
username="synthetic-download-user", password="synthetic-download-password"
)
node.pop("fingerprint", None)
node["jls-opts"] = credentials
if mode.endswith("-h1"):
node["alpn"] = ["http/1.1"]
for listener in peer["listeners"]:
listener.pop("certificate", None)
listener.pop("private-key", None)
listener["jls-config"] = dict(
enable=True,
sni="localhost",
dest=f"{origin}:24001",
users=[credentials, download],
)
return dict(jls_download_credentials=download)
if __name__ == "__main__":
import argparse
from .mihomo_isolation import exclusive_run
from .protocol_vless_container import CLIENT_FINGERPRINTS, run
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
parser.add_argument("cases", nargs="*")
parser.add_argument("--client-fingerprint", choices=CLIENT_FINGERPRINTS)
args = parser.parse_args()
with exclusive_run():
raise SystemExit(
run(
args.output.resolve(),
args.cases or None,
jls=True,
client_fingerprint=args.client_fingerprint,
)
)
+170
View File
@@ -0,0 +1,170 @@
"""Owned native-peer processes. Stop is a wait barrier, never a global prune."""
from __future__ import annotations
import os
import signal
import socket
import subprocess
import tempfile
import threading
import time
from dataclasses import dataclass
from pathlib import Path
from .builds import CORE_DIR
class OwnedProcess:
def __init__(
self,
command: list[str],
log: Path,
record: dict,
*,
env=None,
cwd=None,
limit=1024 * 1024,
):
self.command, self.log_path, self.record = command, log, record
self.process = None
self.log = None
self.env, self.cwd, self.limit = env, cwd, limit
self.reader = None
self.overflow = threading.Event()
def __enter__(self):
self.record.update(started=False, joined=False)
self.log = self.log_path.open("wb")
try:
self.process = subprocess.Popen(
self.command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
start_new_session=os.name != "nt",
stdin=subprocess.DEVNULL,
env=self.env,
cwd=self.cwd,
)
except BaseException:
self.log.close()
self.record["joined"] = True
raise
self.record["started"] = True
self.reader = threading.Thread(target=self._read, name="vcore-owned-output")
self.reader.start()
return self
def _read(self):
stored = 0
try:
while chunk := self.process.stdout.read(8192):
remaining = self.limit - stored
self.log.write(chunk[:remaining])
stored += min(len(chunk), remaining)
if len(chunk) > remaining:
self.overflow.set()
self._signal(signal.SIGKILL)
except OSError:
self.overflow.set()
finally:
self.process.stdout.close()
def _signal(self, value):
try:
if os.name == "nt":
if value == signal.SIGKILL:
self.process.kill()
else:
self.process.terminate()
else:
os.killpg(self.process.pid, value)
except ProcessLookupError:
pass
def _terminate(self):
if self.process.poll() is None:
self._signal(signal.SIGTERM)
try:
self.process.wait(timeout=5)
except subprocess.TimeoutExpired:
self._signal(signal.SIGKILL)
self.process.wait(timeout=5)
def _finish_reader(self):
if self.reader is not None:
self.reader.join(timeout=5)
if self.reader.is_alive():
self._signal(signal.SIGKILL)
self.reader.join(timeout=5)
if self.reader.is_alive():
raise RuntimeError("owned output reader did not stop")
if self.log is not None and not self.log.closed:
self.log.close()
def ensure_alive(self):
if self.overflow.is_set():
raise RuntimeError("native peer output exceeded its bound")
if self.process.poll() is not None:
self.record["unexpected_exit"] = self.process.returncode
raise RuntimeError("native peer exited before case completion")
def wait_tcp(self, port: int, seconds: float = 10, *, host: str = "127.0.0.1"):
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
self.ensure_alive()
try:
with socket.create_connection((host, port), 0.1):
self.record["ready"] = True
return
except OSError:
time.sleep(0.02)
raise TimeoutError("native peer readiness timeout")
def __exit__(self, *_):
try:
self._terminate()
self._finish_reader()
self.record.update(joined=True, exit_code=self.process.returncode)
finally:
if self.reader is None or not self.reader.is_alive():
self.log.close()
@dataclass(frozen=True)
class CommandResult:
returncode: int
stdout: bytes
cleanup: bool
seconds: float
reason: str | None
def run_command(
command: list[str], *, timeout: float, env=None, cwd=None, limit=1024 * 1024
) -> CommandResult:
"""Bounded capture with process-group cleanup even on timeout or SIGINT."""
started = time.monotonic()
record = {}
reason = None
root = CORE_DIR / "target/interop/commands"
root.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="command-", dir=root) as directory:
log = Path(directory) / "private.log"
with OwnedProcess(command, log, record, env=env, cwd=cwd, limit=limit) as owner:
try:
code = owner.process.wait(timeout=timeout)
except subprocess.TimeoutExpired:
code = 124
reason = "timeout"
if owner.overflow.is_set():
code = 125
reason = "output-limit"
output = log.read_bytes()
return CommandResult(
code,
output,
record.get("joined") is True,
round(time.monotonic() - started, 3),
reason,
)
@@ -0,0 +1,67 @@
"""Independent native prerequisites. Artifact readiness is not wire acceptance."""
from __future__ import annotations
from pathlib import Path
from .mihomo_release import download_mihomo, latest_release
from .native_release import PeerArtifact, download_native
from .protocol_catalogs import PEERS
from .protocol_peers import run_command
def preflight(directory: Path, kinds: set[str], *, container: bool = False):
if not kinds <= PEERS:
raise ValueError("unsupported native peer kind")
directory.mkdir(parents=True, exist_ok=True)
artifacts, records = {}, []
for kind in sorted(kinds):
record = {
"kind": kind,
"status": "BLOCKED",
"scope": "preflight-only",
"wire_acceptance": "NOT RUN",
}
records.append(record)
try:
if kind == "M":
identity = {}
release = latest_release()
binary = download_mihomo(
release=release, directory=directory / "M", identity=identity
)
version = run_command([str(binary), "-v"], timeout=10, limit=4096)
if version.returncode != 0 or not version.stdout.strip():
raise RuntimeError("official peer version query failed")
identity["version"] = version.stdout.decode(
"utf-8", errors="replace"
).strip()
artifact = PeerArtifact(binary, identity)
artifacts[kind] = artifact
record.update(identity, status="READY")
if container:
record["container_status"] = "BLOCKED"
status = run_command(
["container", "system", "status"], timeout=10, limit=4096
)
if status.returncode != 0:
raise RuntimeError("Apple Container is not running")
container_identity = {}
binary = download_mihomo(
"linux-arm64",
release=release,
directory=directory / "M-container",
identity=container_identity,
)
artifacts["M-container"] = PeerArtifact(binary, container_identity)
record.update(
container_status="READY", container_artifact=container_identity
)
else:
artifact = download_native(kind, directory / kind)
artifacts[kind] = artifact
record.update(artifact.identity, status="READY")
except (OSError, RuntimeError, ValueError) as error:
# Never expose URLs with userinfo, paths, config bodies or child output.
record["reason"] = type(error).__name__
return artifacts, records
@@ -0,0 +1,536 @@
"""Hybrid REALITY checks; not complete security suite acceptance."""
from __future__ import annotations
import contextlib
import copy
import hashlib
import json
import os
import shutil
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .protocol_containers import ContainerLab, command
from .protocol_evidence import read_events
from .protocol_fixtures import certificates
from .protocol_inputs import redact, same_source, source_identity
from .protocol_peers import run_command
from .protocol_vless_peers import configuration
from .protocol_vless_public import events_pass
MODES = {
"tcp": ("tcp-reality", None),
"ws": ("ws-reality", None),
"upgrade": ("upgrade-reality", None),
"upgrade-fast": ("upgrade-fast-reality", None),
"grpc": ("grpc-reality", None),
"vision": ("vision-reality", None),
**{
f"{version}-{mode}": (
f"xhttp-{mode}{'-download' if mode != 'stream-one' else ''}-reality",
alpn,
)
for version, alpn in (("h1", "http/1.1"), ("h2", "h2"))
for mode in ("stream-one", "stream-up", "packet-up")
},
}
def cases():
result = {}
for profile in ("none", "chrome"):
for mode in MODES:
result[f"{profile}-{mode}-base"] = (profile, mode, "public_base", "hybrid")
for variant in ("main-classic", "download-classic"):
for version in ("h1", "h2"):
result[f"{profile}-{version}-{variant}"] = (
profile,
f"{version}-stream-up",
"public_base",
variant,
)
for mode in ("tcp", "h1-stream-up", "h2-stream-up"):
result[f"{profile}-{mode}-security"] = (
profile,
mode,
"native_hybrid_fail_closed",
"hybrid",
)
result[f"{profile}-vision-inner-tls"] = (
profile,
"vision",
"native_vision_inner_tls",
"hybrid",
)
for test in (
"runtime::public_graph",
"runtime::public_ipv6_and_gates",
"runtime::public_entrypoints",
):
result[f"{profile}-tcp-{test.split('::')[-1]}"] = (
profile,
"tcp",
test,
"hybrid",
)
for mode in ("tcp", "grpc", "vision", "h2-stream-up"):
for suffix, test in (
("life", "runtime::public_lifecycle"),
("owned", "runtime::owned_resources"),
):
result[f"chrome-{mode}-{suffix}"] = ("chrome", mode, test, "hybrid")
return result
def observed_events(observer):
raw = command(
"exec",
observer.name,
"python",
"-c",
"from pathlib import Path; p=Path('/data/events.jsonl'); "
"assert p.stat().st_size <= 8388608; print(p.read_text(),end='')",
)
return [json.loads(line) for line in raw.splitlines()]
def wire_pass(events, expectations):
for port, (hybrid, profile) in expectations.items():
shares = (
[[4588, 1216], [29, 32]]
if hybrid and profile == "chrome"
else [[4588, 1216]]
if hybrid
else [[29, 32]]
)
hellos = [
e
for e in events
if e.get("port") == port and e.get("event") == "client-hello"
]
selections = [
e
for e in events
if e.get("port") == port and e.get("event") == "server-hello"
]
if (
not hellos
or not selections
or not all(e["shares"] == shares for e in hellos)
):
return False
if not all(e["group"] == (4588 if hybrid else 29) for e in selections):
return False
return True
def negative_wire_pass(events, profile, front=25000, legs=1):
for offset, cover, group in ((2, 24432, 29), (3, 24433, 24)):
observed = [e for e in events if e.get("port") == front + offset]
hellos = [e for e in observed if e.get("event") == "client-hello"]
replies = [e for e in observed if e.get("event") == "server-hello"]
if len(hellos) != legs:
return False
if profile == "chrome":
if (
len(replies) != legs
or not all(
e.get("group") == group and e.get("hrr") is (offset == 3)
for e in replies
)
or not all(
e.get("shares") == [[4588, 1216], [29, 32]]
and e.get("groups") == [4588, 29, 23, 24]
for e in hellos
)
):
return False
else:
# A hybrid-only offer has no common group with these peers. A
# missing ServerHello alone is not proof: require the cover's
# native no-shared-group error and an empty, closed server flight.
rejected = [
e
for e in events
if e.get("port") == cover and e.get("event") == "no-shared-group"
]
closed = [
e
for e in observed
if e.get("event") == "server-close" and e.get("bytes") == 0
]
if (
replies
or len(rejected) != legs
or len(closed) != legs
or not all(
e.get("shares") == [[4588, 1216]] and e.get("groups") == [4588]
for e in hellos
)
):
return False
return True
def run(output: Path, selected=None, *, encryption=None, supplied=None):
output = output.resolve()
if not output.is_relative_to(CORE_DIR / "target/interop/runs"):
raise ValueError("use a fresh child of target/interop/runs")
required = cases()
selected = list(required) if selected is None else selected
if (
not selected
or len(set(selected)) != len(selected)
or not set(selected) <= required.keys()
):
raise ValueError("invalid SECURITY.reality selection")
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="SECURITY.reality",
scope="S03/D16-production",
encryption_profile=encryption,
complete_selection=set(selected) == set(required),
required=list(required),
source=source_identity(),
cases=[],
isolation={},
peers={},
status="NOT RUN",
)
try:
if supplied is None:
identity = {}
binary = download_mihomo(
"linux-arm64", directory=output / "binaries", identity=identity
)
else:
binary, identity = supplied["M"]
report["peers"]["M"] = identity
built = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_public",
"--test",
"vless_native",
"--no-run",
],
timeout=240,
)
(output / "build.log").write_text(redact(built.stdout.decode(errors="replace")))
if built.returncode or not built.cleanup:
raise RuntimeError("SECURITY.reality consumer build failed")
lab = ContainerLab(report["isolation"], mtu=1500)
with (
tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary,
contextlib.ExitStack() as stack,
):
root = Path(temporary)
origin_dir, server_dir, observer_dir, hop_dir = [
root / p for p in ("origin", "server", "observer", "hop")
]
for directory in (origin_dir, server_dir, observer_dir, hop_dir):
directory.mkdir()
shutil.copy2(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
shutil.copy2(
Path(__file__).with_name("container_reality_hybrid.py"),
observer_dir / "observer.py",
)
shutil.copy2(binary, server_dir / "peer")
shutil.copy2(binary, hop_dir / "peer")
cert, key, pin = certificates(origin_dir)
certificates(observer_dir)
origin = lab.start(
stack,
origin_dir,
"security-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
f"VCORE_ORIGIN_CERT=/data/fixture/{cert.name}",
f"VCORE_ORIGIN_KEY=/data/fixture/{key.name}",
"python",
"-B",
"/data/fixture/origin.py",
],
)
observer = lab.start(
stack,
observer_dir,
"security-observer",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/observer.py",
],
)
server = lab.start(
stack,
server_dir,
"security-mihomo",
[
"/data/fixture/peer",
"-d",
"/data/mihomo",
"-f",
"/data/fixture/config.json",
],
)
hop = lab.start(
stack,
hop_dir,
"security-upstream",
[
"/data/fixture/peer",
"-d",
"/data/mihomo",
"-f",
"/data/fixture/config.json",
],
)
listeners, routes, nodes, ports = [], {}, {}, {}
for index, (name, (mode, alpn)) in enumerate(MODES.items()):
node, config = configuration(
mode,
observer.ipv4,
origin.ipv4,
Path("/data/fixture/cert.pem"),
Path("/data/fixture/key.pem"),
)
backend, front = 23010 + index * 10, 25000 + index * 10
listener = config["listeners"][0]
if encryption:
from .protocol_encryption_public import configuration as encrypt
encrypt(encryption, node, config)
listener.update(name=name, port=backend)
listener["reality-config"]["dest"] = f"{observer.ipv4}:24431"
listeners.append(listener)
routes[str(front)] = routes[str(front + 1)] = [server.ipv4, backend]
if name == "tcp":
for offset, cover in ((2, 24432), (3, 24433)):
negative = copy.deepcopy(listener)
negative.update(name=f"reject-{cover}", port=backend + offset)
negative["reality-config"]["dest"] = f"{observer.ipv4}:{cover}"
listeners.append(negative)
routes[str(front + offset)] = [server.ipv4, backend + offset]
node.update(name="peer", port=front)
node["reality-opts"]["support-x25519mlkem768"] = True
if alpn:
node["alpn"] = [alpn]
if "download-settings" in node.get("xhttp-opts", {}):
node["xhttp-opts"]["download-settings"] = {
"server": observer.ipv4,
"port": front + 1,
}
nodes[name], ports[name] = node, front
(observer_dir / "routes.json").write_text(json.dumps(routes))
(server_dir / "config.json").write_text(
json.dumps(
{
"ipv6": True,
"log-level": "silent",
"listeners": listeners,
"hosts": {"vcore-fixture.test": origin.ipv4},
"rules": ["MATCH,DIRECT"],
}
)
)
(hop_dir / "config.json").write_text(
json.dumps(
{
"socks-port": 23001,
"allow-lan": True,
"bind-address": "*",
"log-level": "silent",
"ipv6": True,
"hosts": {"peer.fixture.test": observer.ipv4},
"rules": ["MATCH,DIRECT"],
}
)
)
for peer, port in (
(origin, 24000),
(observer, 24431),
(server, 23010),
(hop, 23001),
):
peer.release()
peer.wait_tcp(port)
identity["version"] = command(
"exec", server.name, "/data/fixture/peer", "-v"
).strip()
if (
command("exec", server.name, "sha256sum", "/data/fixture/peer").split()[
0
]
!= identity["binary_sha256"]
):
raise RuntimeError("native peer identity mismatch")
report["openssl"] = command(
"exec",
observer.name,
"python",
"-c",
"import ssl; print(ssl.OPENSSL_VERSION)",
).strip()
for case in selected:
profile, mode, test, variant = required[case]
node = copy.deepcopy(nodes[mode])
node["client-fingerprint"] = profile
expectations = {ports[mode]: (True, profile)}
if node.get("xhttp-opts", {}).get("download-settings") is not None:
expectations[ports[mode] + 1] = (True, profile)
if variant != "hybrid":
down = node["xhttp-opts"]["download-settings"]
down["reality-opts"] = copy.deepcopy(node["reality-opts"])
if variant == "main-classic":
node["reality-opts"]["support-x25519mlkem768"] = False
expectations[ports[mode]] = (False, profile)
else:
down["reality-opts"]["support-x25519mlkem768"] = False
expectations[ports[mode] + 1] = (False, profile)
f = dict(
isolation="containers",
node=node,
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
server_ipv6=observer.ipv6,
origin_pin=pin,
data_dir=str(root / "data"),
peer_kind="M",
vision_probe=mode == "vision",
hop=dict(
name="hop", type="socks5", server=hop.ipv4, port=23001, udp=True
),
downgrade_port=ports["tcp"] + 2,
hrr_port=ports["tcp"] + 3,
ordinary_tls_port=24431,
tls12_port=24434,
)
fixture = root / "input.json"
fixture.write_text(json.dumps(f))
before = len(observed_events(observer))
events_path = output / f"{case}-events.jsonl"
test_command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_native" if test.startswith("native_") else "vless_public",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
print(case, flush=True)
result = run_command(
test_command,
timeout=240,
env=dict(
os.environ,
VCORE_VLESS_INPUT=str(fixture),
VCORE_CASE_EVENTS=str(events_path),
VCORE_PROTOCOL_STAGE="SECURITY",
),
)
(output / f"{case}.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
observed = observed_events(observer)[before:]
wire_path = output / f"{case}-wire.json"
wire_path.write_text(json.dumps(observed, indent=2) + "\n")
events = read_events(events_path) if events_path.exists() else []
remapped = [
dict(e, suite=e.get("suite", "").replace("SECURITY-", "VLESS-", 1))
for e in events
]
assertions = (
remapped
== [
dict(
schema_version=1,
suite="VLESS-WIRE",
assertion=test,
status=status,
)
for status in ("BEGIN", "PASS")
]
if test.startswith("native_")
else events_pass(remapped, test, MODES[mode][0])
)
# Security matrices deliberately contain rejected peers; their
# strict per-attempt assertions require errors, never timeouts.
wire_valid = wire_pass(observed, expectations)
if test == "native_hybrid_fail_closed":
wire_valid &= negative_wire_pass(
observed, profile, ports["tcp"], len(expectations)
)
passed = (
result.returncode == 0
and result.cleanup
and assertions
and wire_valid
)
report["cases"].append(
dict(
id=case,
test=test,
variant=variant,
profile=profile,
status="PASS" if passed else "FAIL",
exit_code=result.returncode,
command_cleanup=result.cleanup,
seconds=result.seconds,
command=test_command,
wire_sha256=hashlib.sha256(wire_path.read_bytes()).hexdigest(),
assertions=assertions,
wire_valid=wire_valid,
)
)
print(f"{case}: {report['cases'][-1]['status']}", flush=True)
for peer in (origin, observer, server, hop):
peer.ensure_alive()
if not passed:
raise RuntimeError(f"SECURITY.reality case failed: {case}")
report["status"] = "PASS"
except KeyboardInterrupt:
report.update(status="INTERRUPTED", reason="user interruption")
except (OSError, RuntimeError, ValueError, KeyError) as error:
report.update(
status="FAIL",
reason=str(error)
if isinstance(error, RuntimeError)
else type(error).__name__,
)
finally:
report["cleanup"] = all(
p["joined"] for p in report["isolation"].get("peers", [])
)
report["source_unchanged"] = same_source(report["source"], source_identity())
if not report["cleanup"] or not report["source_unchanged"]:
report["status"] = "FAIL"
(output / "reality-results.json").write_text(
json.dumps(report, indent=2) + "\n"
)
return 0 if report["status"] == "PASS" else 1
def main(output, selected=None):
with exclusive_run():
return run(output, selected)
@@ -0,0 +1,68 @@
"""Structured unittest results for the protocol harness; no stdout PASS parsing."""
from __future__ import annotations
import contextlib
import io
import json
import sys
import unittest
from pathlib import Path
from .builds import CORE_DIR
class Result(unittest.TestResult):
def __init__(self):
super().__init__()
self.cases = []
def addSuccess(self, test):
super().addSuccess(test)
self.cases.append({"test": test.id(), "status": "PASS"})
def addFailure(self, test, error):
super().addFailure(test, error)
self.cases.append({"test": test.id(), "status": "FAIL", "reason": "assertion"})
def addError(self, test, error):
super().addError(test, error)
self.cases.append(
{"test": test.id(), "status": "FAIL", "reason": error[0].__name__}
)
def addSkip(self, test, reason):
super().addSkip(test, reason)
self.cases.append({"test": test.id(), "status": "NOT RUN"})
class BoundedOutput(io.TextIOBase):
def __init__(self):
self.size = 0
def write(self, text):
self.size += len(text)
if self.size > 128 * 1024:
raise RuntimeError("script test output exceeded bound")
return len(text)
def execute(path: Path) -> bool:
result = Result()
try:
suite = unittest.defaultTestLoader.discover(str(CORE_DIR / "scripts/tests"))
with (
contextlib.redirect_stdout(BoundedOutput()),
contextlib.redirect_stderr(BoundedOutput()),
):
suite.run(result)
finally:
path.write_text(
json.dumps({"tests_run": result.testsRun, "cases": result.cases}, indent=2)
+ "\n"
)
return result.testsRun > 0 and result.wasSuccessful() and not result.skipped
if __name__ == "__main__":
raise SystemExit(0 if execute(Path(sys.argv[1])) else 1)
@@ -0,0 +1,662 @@
"""Execute security cases and reconstruct results from commands and native events."""
from __future__ import annotations
import json
import sys
from contextlib import ExitStack
from .protocol_evidence import HEX, new_result, read_events, read_json
from .protocol_inputs import redact
from .protocol_security_catalog import FIELDS, GATES, definitions, groups
def commands(identifier):
from .protocol_hysteria2_acceptance import commands as previous
cargo = ["cargo", "test", "--locked", "--all-features"]
targets = [
"ech_config",
"ech_tls",
"encryption_config",
"reality_config",
"jls_config",
]
if identifier in {"SECURITY-CFG", "SECURITY-RELEASE"}:
if identifier == "SECURITY-RELEASE":
cargo += ["--release"]
return [
cargo + sum((["--test", name] for name in targets), []),
cargo + ["--lib", "security::"],
cargo + ["--lib", "outbound::vless::encryption"],
cargo + ["--lib", "config::"],
]
if identifier == "SECURITY-FEATURES":
return previous("HYSTERIA2-FEATURES") + [
[
"cargo",
"test",
"--locked",
"--no-default-features",
"--features",
"outbound-vless",
"--test",
"ech_config",
"--test",
"ech_tls",
],
[
"cargo",
"test",
"--locked",
"--all-features",
"--all-targets",
"--no-run",
],
]
if identifier == "SECURITY-SCRIPTS":
return [
[
sys.executable,
"-m",
"vcore_scripts.protocol_script_tests",
"{output}/script-tests.json",
]
]
return previous(identifier.replace("SECURITY-", "HYSTERIA2-"))
def gate_result(case, records, events, script=None):
from .protocol_hysteria2_catalog import assertions_pass
from .protocol_security_units import TESTS
identifier = case["case_id"]
good = bool(records) and all(
r.get("exit_code") == 0 and r.get("cleanup") is True for r in records
)
if identifier in {"SECURITY-CFG", "SECURITY-RELEASE"}:
# Config and TLS tests each have an exact BEGIN/PASS pair. Library
# regressions run separately without these assertion identities.
selected = [e for e in events if e.get("suite") == "SECURITY-CONFIG-TLS"]
good &= assertions_pass(
selected,
{
("SECURITY-CONFIG-TLS", name): 1
for names in TESTS.values()
for name in names
},
)
good &= all(e.get("status") in {"BEGIN", "PASS"} for e in events)
if identifier == "SECURITY-SCRIPTS":
samples = (script or {}).get("cases", [])
names = [s.get("test") for s in samples]
required = {
"test_protocol_security.SecurityAcceptanceTest." + name
for name in (
"test_frozen_scope_and_compositions",
"test_native_missing_duplicate_wrong_events_fail",
"test_source_cleanup_and_identity_are_not_inferred",
"test_empty_gate_and_missing_script_proof_fail",
"test_legacy_ech_gateway_keeps_response_inside_tls",
"test_close_reference_cannot_hide_a_different_official_profile",
)
}
required |= {
"test_protocol_containers.ContainerTests." + name
for name in (
"test_frozen_latest_image_is_single_run_owned",
"test_image_refresh_failure_never_falls_back_to_a_cached_image",
)
}
good &= (
bool(samples)
and len(names) == len(set(names)) == (script or {}).get("tests_run")
and required <= set(names)
and all(s.get("status") == "PASS" for s in samples)
)
return result(
case, good, bool(records) and all(r.get("cleanup") is True for r in records)
)
def result(case, good, cleanup):
record = new_result(case)
record.update(
status="PASS" if good else "FAIL",
assertions={case["case_id"]: bool(good)},
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=cleanup,
)
return record
def native_envelope(report, peers=None, source=None, *, image_digest=None):
isolation = report.get("isolation", {})
owned = isolation.get("peers", [])
identities = report.get("peers", {})
if "peer" in report:
identities = {"M": report["peer"]}
good = (
report.get("status") == "PASS"
and report.get("cleanup") is True
and report.get("source_unchanged") is True
and isolation.get("network_mode") == "hostOnly"
and isolation.get("host_servers") is False
and isolation.get("guest_mtu") == 1500
and isolation.get("image_digest", "").startswith("sha256:")
and bool(HEX.fullmatch(isolation.get("image_digest", "")[7:]))
and bool(owned)
and all(p.get("started") is True and p.get("joined") is True for p in owned)
and bool(identities)
)
for kind, identity in identities.items():
good &= (
bool(identity.get("version"))
and bool(HEX.fullmatch(identity.get("binary_sha256", "")))
and bool(HEX.fullmatch(identity.get("archive_sha256", "")))
and identity.get("source_url", "").startswith(
{
"M": "https://github.com/MetaCubeX/mihomo/releases/",
"XR": "https://github.com/XTLS/Xray-core/releases/",
"V2": "https://github.com/v2fly/v2ray-core/releases/",
}.get(kind, "invalid-source")
)
)
if peers is not None:
good &= identity == peers.get(kind)
if source is not None:
good &= report.get("source") == source
if image_digest is not None:
good &= isolation.get("image_digest") == image_digest
return bool(good)
def rust_command(target, test, *, fields=False):
return [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
target,
test,
"--",
*(["--exact", "--ignored"] if fields else ["--ignored", "--exact"]),
"--nocapture",
]
def vless_catalog(group):
if group.get("ech"):
from .protocol_ech import catalog
return catalog()
if group.get("jls"):
from .protocol_jls import catalog
return catalog()
from .protocol_vless_container import ALL_CASES
return ALL_CASES | {
"FINGERPRINT-ANYTLS": ("M", "anytls", True, "public_legacy_regression")
}
def close_reference_pass(group, mode, directory):
profile = group.get("client_fingerprint")
expected_profile, scope = profile, "same-mode"
if mode == "grpc-tls":
if group.get("jls") and profile in (None, "", "none"):
expected_profile, scope = "chrome", "jls-grpc-chrome-baseline"
elif group.get("ech") and profile in ("safari", "safari16"):
expected_profile, scope = "chrome", "ech-safari-chrome-baseline"
try:
reference = read_json(directory / f"{mode}-tls-close-reference.json")
except (OSError, ValueError):
return False
return (
isinstance(reference, dict)
and reference.get("scope") == scope
and reference.get("client_fingerprint") == expected_profile
and reference.get("dut_client_fingerprint") == profile
and reference.get("terminated") is True
)
def vless_pass(group, report, directory):
from .protocol_vless_public import events_pass
catalog = vless_catalog(group)
expected = group["selected"]
records = report.get("cases", [])
if len(records) != len(expected) or {r.get("case_id") for r in records} != set(
expected
):
return False
if any(report.get(k) != group.get(k, False) for k in ("jls", "ech")):
return False
if report.get("encryption_profile") != group.get("encryption") or report.get(
"client_fingerprint"
) != group.get("client_fingerprint"):
return False
stage = (
"SECURITY"
if group.get("ech") or group.get("jls") or group.get("encryption")
else "VLESS"
)
for record in records:
name = record["case_id"]
kind, mode, _, test = catalog[name]
native = test.startswith("native_")
events = read_events(directory / (name + "-events.jsonl"))
if not (
record.get("status") == "PASS"
and record.get("exit_code") == 0
and record.get("command_cleanup") is True
and record.get("cleanup") is True
and record.get("peer_kind") == kind
and record.get("command")
== rust_command("vless_native" if native else "vless_public", test)
):
return False
if native:
if test == "native_mihomo_close_alignment" and not close_reference_pass(
group, mode, directory
):
return False
if events != [
dict(
schema_version=1,
suite=stage + "-WIRE",
assertion=test,
status=status,
)
for status in ("BEGIN", "PASS")
]:
return False
elif not events_pass(events, test, mode, stage=stage):
return False
return True
def xhttp_pass(group, report, directory):
from .protocol_xhttp_fields import PUBLIC_TESTS, events_pass, public_events_pass
jobs = group["jobs"]
expected = {
entry["case_id"]: (variant, entry["test"])
for variant, entries in jobs.items()
for entry in entries
}
records = report.get("cases", [])
if len(records) != len(expected) or {r.get("case_id") for r in records} != set(
expected
):
return False
if (
report.get("ech") != group.get("ech", False)
or report.get("jls") != group.get("jls", False)
or report.get("encryption_profile") != group.get("encryption")
):
return False
for record in records:
variant, test = expected[record["case_id"]]
events = read_events(directory / (record["case_id"] + "-events.jsonl"))
if not (
record.get("variant") == variant
and record.get("assertion") == test
and record.get("status") == "PASS"
and record.get("exit_code") == 0
and record.get("command_cleanup") is True
and record.get("command")
== rust_command(
"vless_public" if test in PUBLIC_TESTS else "xhttp_native",
test,
fields=True,
)
):
return False
if not (
public_events_pass(events, test)
if test in PUBLIC_TESTS
else events_pass(events, test, owned=test.startswith("lifecycle::"))
):
return False
return True
def hybrid_pass(group, report, directory):
from .protocol_reality_hybrid import MODES, cases, negative_wire_pass, wire_pass
from .protocol_vless_public import events_pass
catalog = cases()
records = report.get("cases", [])
selected = group["selected"]
if (
len(records) != len(selected)
or {r.get("id") for r in records} != set(selected)
or report.get("encryption_profile") != group.get("encryption")
):
return False
ports = {mode: 25000 + i * 10 for i, mode in enumerate(MODES)}
for record in records:
name = record["id"]
profile, mode, test, variant = catalog[name]
expected = {ports[mode]: (True, profile)}
if mode.startswith(("h1-", "h2-")) and not mode.endswith("stream-one"):
expected[ports[mode] + 1] = (True, profile)
if variant != "hybrid":
expected[ports[mode] + (variant == "download-classic")] = (False, profile)
wire = read_json(directory / (name + "-wire.json"))
events = read_events(directory / (name + "-events.jsonl"))
native = test.startswith("native_")
good = (
record.get("status") == "PASS"
and record.get("exit_code") == 0
and record.get("command_cleanup") is True
and (record.get("test"), record.get("variant"), record.get("profile"))
== (test, variant, profile)
and record.get("command")
== rust_command("vless_native" if native else "vless_public", test)
and wire_pass(wire, expected)
)
if test == "native_hybrid_fail_closed":
good &= negative_wire_pass(wire, profile, ports["tcp"], len(expected))
good &= (
events
== [
dict(
schema_version=1,
suite="SECURITY-WIRE",
assertion=test,
status=status,
)
for status in ("BEGIN", "PASS")
]
if native
else events_pass(events, test, MODES[mode][0], stage="SECURITY")
)
if not good:
return False
return True
def encryption_pass(group, report, directory):
from .protocol_encryption import cases
expected = [
key
for key, (_, rtt, _) in cases().items()
if not group.get("expiry") or rtt == "0rtt"
]
records = report.get("cases", [])
test = (
"native_ticket_expiry" if group.get("expiry") else "native_encryption_roundtrip"
)
marker = (
"SECURITY-ENCRYPTION-EXPIRY-PASS full-resumed-expired-full-resumed"
if group.get("expiry")
else "SECURITY-ENCRYPTION-WIRE-PASS rounds=4 bytes_per_direction=10485760"
)
if (
len(records) != len(expected)
or {r.get("case_id") for r in records} != set(expected)
or report.get("ticket_expiry") != group.get("expiry", False)
or report.get("cipher")
!= ("chacha20-poly1305" if group.get("chacha") else "cpu-selected")
or report.get("padding") != "minimal"
):
return False
for record in records:
log = (directory / (record["case_id"] + ".log")).read_text()
if not (
record.get("status") == "PASS"
and record.get("returncode") == 0
and record.get("cleanup") is True
and marker in log
and "1 passed; 0 failed; 0 ignored" in log
and record.get("command") == rust_command("encryption_wire", test)
):
return False
return True
REPORTS = {
"vless": "vless-results.json",
"xhttp": "xhttp-fields-results.json",
"hybrid": "reality-results.json",
"encryption": "encryption-results.json",
}
CHECKERS = {
"vless": vless_pass,
"xhttp": xhttp_pass,
"hybrid": hybrid_pass,
"encryption": encryption_pass,
}
def native_result(
case, group, report, directory, *, peers=None, source=None, image_digest=None
):
good = native_envelope(report, peers, source, image_digest=image_digest)
good &= CHECKERS[group["runner"]](group, report, directory)
return result(case, good, report.get("cleanup") is True)
def fields_report(cases, results):
by_id = {r["case_id"]: r for r in results}
rows = []
for field in sorted(FIELDS):
owners = [c["case_id"] for c in cases if field in c["row_ids"]]
cfg = [c for c in owners if c in {"SECURITY-CFG", "SECURITY-RELEASE"}]
wire = [c for c in owners if c not in GATES]
good = (
bool(cfg)
and bool(wire)
and all(by_id.get(c, {}).get("status") == "PASS" for c in owners)
)
rows.append(
dict(
id=field,
cfg_cases=cfg,
wire_cases=wire,
status="PASS" if good else "NOT RUN",
)
)
return dict(
stage="SECURITY", scope="selected-vless-security-consumers", fields=rows
)
def execute(cases, run, output, results):
from .protocol_containers import frozen_image
from .protocol_encryption import run as run_encryption
from .protocol_harness import _command
from .protocol_reality_hybrid import run as run_hybrid
from .protocol_vless_container import run as run_vless
from .protocol_xhttp_acceptance import prepare_peers
from .protocol_xhttp_fields import run as run_xhttp
supplied = {}
image_scope = ExitStack()
catalog = groups()
source = {
k: run[k]
for k in (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
}
try:
for case in cases:
identifier = case["case_id"]
if identifier not in GATES:
continue
records, events = [], []
for i, argv in enumerate(commands(identifier)):
name = f"{identifier}-{i}"
path = output / (name + "-events.jsonl")
_command(
run,
output,
name,
[v.replace("{output}", str(output)) for v in argv],
case["timeout_seconds"],
events=path,
)
records.append(run["commands"][-1])
if path.exists():
events += read_events(path)
script = (
read_json(output / "script-tests.json")
if identifier == "SECURITY-SCRIPTS"
else None
)
results[identifier] = gate_result(case, records, events, script)
print(identifier + ": " + results[identifier]["status"], flush=True)
if results[identifier]["status"] != "PASS":
raise RuntimeError("SECURITY local gate failed: " + identifier)
if any(c["case_id"] in catalog for c in cases):
supplied = prepare_peers(output, {"M", "XR", "V2"})
run["container_image"] = image_scope.enter_context(
frozen_image(output / "container-image-pull.log")
)
runners = {
"vless": run_vless,
"xhttp": run_xhttp,
"hybrid": run_hybrid,
"encryption": run_encryption,
}
# Retained combinations first, followed by the longer new ECH matrix.
# Every required group still runs in this same frozen source/image scope.
for case in sorted(
cases,
key=lambda c: (c["case_id"].startswith("SECURITY-ECH-"), c["case_id"]),
):
identifier = case["case_id"]
if identifier in GATES:
continue
group = catalog[identifier]
kwargs = {k: v for k, v in group.items() if k not in {"rows", "runner"}}
if group["runner"] == "xhttp":
kwargs["selected"] = list(kwargs["jobs"])
directory = output / identifier
runners[group["runner"]](directory, supplied=supplied, **kwargs)
report = read_json(directory / REPORTS[group["runner"]])
results[identifier] = native_result(
case,
group,
report,
directory,
peers={k: v[1] for k, v in supplied.items()},
source=source,
image_digest=run["container_image"]["digest"],
)
print(identifier + ": " + results[identifier]["status"], flush=True)
if results[identifier]["status"] != "PASS":
raise RuntimeError("SECURITY native gate failed: " + identifier)
finally:
image_scope.close()
(output / "peers.json").write_text(
json.dumps({k: v[1] for k, v in supplied.items()}, indent=2) + "\n"
)
(output / "fields.json").write_text(
json.dumps(fields_report(cases, list(results.values())), indent=2) + "\n"
)
def check(run_dir, cases, run, results, peers, paths):
from .protocol_containers import IMAGE
snapshot = run.get("container_image", {})
digest = snapshot.get("digest", "")
if (
not isinstance(digest, str)
or not digest.startswith("sha256:")
or not HEX.fullmatch(digest[7:])
or snapshot
!= dict(
tag=IMAGE,
digest=digest,
command=["container", "image", "pull", IMAGE],
exit_code=0,
cleanup=True,
log="container-image-pull.log",
)
or snapshot["log"] not in paths
):
raise ValueError("missing or altered SECURITY frozen container image")
if (
cases != definitions()
or set(peers) != {"M", "XR", "V2"}
or not {"fields.json", "script-tests.json"} <= set(paths)
):
raise ValueError("incomplete SECURITY stage artifacts or altered manifest")
records = run.get("commands", [])
names = {
f"{c['case_id']}-{i}"
for c in cases
for i, _ in enumerate(commands(c["case_id"]))
if c["case_id"] in GATES
}
if len(records) != len(names) or {r.get("name") for r in records} != names:
raise ValueError("missing, duplicate or extra SECURITY gate command")
actual = {r["case_id"]: r for r in results}
source = {
k: run[k]
for k in (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
}
catalog = groups()
for case in cases:
identifier = case["case_id"]
if identifier in GATES:
selected, events = [], []
for i, argv in enumerate(commands(identifier)):
record = next(r for r in records if r["name"] == f"{identifier}-{i}")
if (
record["command"]
!= [redact(v.replace("{output}", str(run_dir))) for v in argv]
or record["log"] not in paths
):
raise ValueError("SECURITY gate command or log changed")
selected.append(record)
path = run_dir / f"{identifier}-{i}-events.jsonl"
if path.exists():
events += read_events(path)
recalculated = gate_result(
case,
selected,
events,
read_json(run_dir / "script-tests.json")
if identifier == "SECURITY-SCRIPTS"
else None,
)
else:
group = catalog[identifier]
directory = run_dir / identifier
report = read_json(directory / REPORTS[group["runner"]])
recalculated = native_result(
case,
group,
report,
directory,
peers=peers,
source=source,
image_digest=digest,
)
if recalculated != actual[identifier] or recalculated["status"] != "PASS":
raise ValueError("SECURITY results differ from independent evidence")
fields = fields_report(cases, results)
if read_json(run_dir / "fields.json") != fields or any(
f["status"] != "PASS" for f in fields["fields"]
):
raise ValueError("missing SECURITY field behavior coverage")
print(f"SECURITY: PASS ({len(cases)} required groups; {len(FIELDS)} field rows)")
@@ -0,0 +1,331 @@
"""Selected security requirements, independent of runtime results.
This is an explicit covering set, not a Cartesian-product support claim. Encryption,
hybrid REALITY and JLS have dedicated reports; this suite exercises their critical
consumers together with static ECH and composition checks.
"""
from __future__ import annotations
FIELDS = {"VL06", "S03", "S04", "S05", "S12", "S13", "D16", "D17", "D18", "D25", "D26"}
ECH_FIELDS = ["S04", "S05", "D17", "D18"]
JLS_FIELDS = ["S12", "S13", "D25", "D26"]
HYBRID_FIELDS = ["S03", "D16"]
GATES = {
"SECURITY-CFG",
"SECURITY-RELEASE",
"SECURITY-REGRESSION",
"SECURITY-QUALITY",
"SECURITY-FEATURES",
"SECURITY-SCRIPTS",
"SECURITY-PLATFORMS",
}
PROFILES = ("chrome", "chrome120", "firefox", "safari")
ENCRYPTION = tuple(
f"{style}-{rtt}-mixed"
for style in ("native", "xorpub", "random")
for rtt in ("1rtt", "0rtt")
)
def groups():
from .protocol_ech import catalog as ech_catalog
from .protocol_jls import catalog as jls_catalog
result = {}
def add(name, runner, rows, **options):
result[name] = dict(runner=runner, rows=rows, **options)
add(
"SECURITY-ECH-STANDARD",
"vless",
ECH_FIELDS,
selected=list(ech_catalog()),
ech=True,
)
named = [
"SECURITY-ECH-TCP-TLS-BASE",
"SECURITY-ECH-TCP-TLS-REJECT",
"SECURITY-ECH-WS-TLS-BASE",
"SECURITY-ECH-GRPC-TLS-BASE",
"SECURITY-ECH-GRPC-TLS-CLOSE",
"SECURITY-ECH-TCP-MTLS-IDENTITY",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-BASE",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-REJECT",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-DOWNLOAD",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-H1-BASE",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-H1-DOWNLOAD",
]
for profile in PROFILES:
add(
"SECURITY-ECH-" + profile.upper(),
"vless",
ECH_FIELDS,
selected=named,
ech=True,
client_fingerprint=profile,
)
add(
"SECURITY-JLS-RETAINED",
"vless",
JLS_FIELDS,
jls=True,
selected=[
key
for key, (_, mode, _, test) in jls_catalog().items()
if mode
in {
"tcp-tls",
"grpc-tls",
"xhttp-stream-up-download",
"xhttp-stream-up-download-h1",
}
and test
in {
"public_base",
"native_jls_fail_closed",
"public_jls_download_identity",
"runtime::owned_resources",
"runtime::public_lifecycle",
"native_mihomo_close_alignment",
}
],
)
add(
"SECURITY-HYBRID-RETAINED",
"hybrid",
HYBRID_FIELDS,
selected=[
f"{profile}-{mode}-{test}"
for profile in ("none", "chrome")
for mode in ("tcp", "h1-stream-up", "h2-stream-up")
for test in ("base", "security")
]
+ [
"chrome-vision-inner-tls",
"chrome-h2-stream-up-owned",
"chrome-h2-stream-up-life",
"chrome-h1-main-classic",
"chrome-h2-download-classic",
],
)
for profile in ENCRYPTION:
suffix = profile.upper()
add(
"SECURITY-ECH-ENCRYPTION-" + suffix,
"vless",
["VL06", *ECH_FIELDS],
ech=True,
encryption=profile,
client_fingerprint="chrome",
selected=[
"SECURITY-ECH-TCP-TLS-BASE",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-BASE",
"SECURITY-ECH-XHTTP-STREAM-UP-DOWNLOAD-H1-BASE",
],
)
add(
"SECURITY-JLS-ENCRYPTION-" + suffix,
"vless",
["VL06", *JLS_FIELDS],
jls=True,
encryption=profile,
selected=[
"SECURITY-JLS-TCP-BASE",
"SECURITY-JLS-XHTTP-STREAM-UP-DOWNLOAD-BASE",
],
)
add(
"SECURITY-HYBRID-ENCRYPTION-" + suffix,
"hybrid",
["VL06", *HYBRID_FIELDS],
encryption=profile,
selected=[
"chrome-tcp-base",
"chrome-vision-base",
"chrome-h2-stream-up-base",
],
)
def fields(name, rows, specs, **options):
jobs = {}
for variant, tests in specs:
jobs[variant] = [
dict(case_id=variant + "-" + str(i), test=test)
for i, test in enumerate(tests)
]
add(name, "xhttp", rows, jobs=jobs, **options)
specs = [
(f"h3-{mode}-headers", ["public_base"])
for mode in ("stream-one", "stream-up", "packet-up")
]
specs += [
(
f"h3-ech-{action}",
["security::native_xhttp_security"]
if action.startswith("reject")
else ["public_base"],
)
for action in ("reject-main", "reject-download", "replace", "clear")
]
specs += [
(
"h3-stream-up-reuse",
[
"runtime::public_graph",
"runtime::public_entrypoints",
"runtime::public_ipv6_and_gates",
"runtime::public_udp_isolation",
"runtime::public_lifecycle",
"lifecycle::native_xhttp_owned_resources",
],
),
("h3-keepalive", ["keepalive::native_h3_keepalive_observes_each_leg_and_stop"]),
]
fields("SECURITY-ECH-H3", ECH_FIELDS, specs, ech=True)
for security in ("ech", "jls"):
versions = ("h1", "h2", "h3") if security == "ech" else ("h1", "h2")
specs = [
(f"{version}-mux-{mux}-padded{only}", ["public_base"])
for version in versions
for mux in ("h2mux", "smux", "yamux")
for only in ("", "-only-tcp")
]
fields(
"SECURITY-" + security.upper() + "-MUX",
ECH_FIELDS if security == "ech" else JLS_FIELDS,
specs,
**{security: True},
encryption="native-0rtt-mixed",
)
fields(
"SECURITY-ECH-NATIVE-TRANSPORT",
ECH_FIELDS,
[
("outer-" + transport + "-tls", ["public_base"])
for transport in ("http", "h2", "ws-header", "ws-path")
],
ech=True,
encryption="native-0rtt-mixed",
)
fields(
"SECURITY-ECH-H3-ENCRYPTION",
["VL06", *ECH_FIELDS],
[
(f"h3-{mode}-headers", ["public_base"])
for mode in ("stream-one", "stream-up", "packet-up")
]
+ [("h3-ech-replace", ["public_base"])],
ech=True,
encryption="native-0rtt-mixed",
)
add("SECURITY-ENCRYPTION-WIRE", "encryption", ["VL06"])
add("SECURITY-ENCRYPTION-CHACHA", "encryption", ["VL06"], chacha=True)
add("SECURITY-ENCRYPTION-EXPIRY", "encryption", ["VL06"], expiry=True)
for profile in ("none", "chrome"):
add(
"SECURITY-SHARED-" + profile.upper(),
"vless",
[],
client_fingerprint=profile,
selected=[
"VLESS-TCP-TLS-BASE",
"VLESS-TCP-TLS-NEG",
"VLESS-TCP-REALITY-BASE",
"VLESS-TCP-REALITY-NEG",
"VLESS-VISION-REALITY-BASE",
"VLESS-GRPC-TLS-BASE",
"VLESS-HTTP-TLS-BASE",
"VLESS-H2-TLS-BASE",
"VLESS-REGRESSION-TROJAN-TCP",
"VLESS-REGRESSION-VMESS-GRPC",
"FINGERPRINT-ANYTLS",
],
)
fields(
"SECURITY-SHARED-XHTTP",
[],
[
("h1-stream-up-split-reuse", ["public_base"]),
("h2-stream-up-split-reuse", ["public_base"]),
("h3-stream-up-split-reuse", ["public_base"]),
("h2r-stream-up-split-reuse", ["public_base"]),
("h3-mux-yamux-padded", ["public_base"]),
],
)
return result
def definitions():
native = groups()
result = []
for identifier in sorted(GATES | native.keys()):
group = native.get(identifier, {})
runner = group.get("runner", "gate")
rows = sorted(
FIELDS
if identifier in {"SECURITY-CFG", "SECURITY-RELEASE"}
else group.get("rows", [])
)
options = {k: v for k, v in group.items() if k not in {"rows", "runner"}}
kind = (
"unit"
if not group
else (
"XR" if "H3" in identifier or "NATIVE-TRANSPORT" in identifier else "M"
)
)
result.append(
dict(
case_id=identifier,
stage="SECURITY",
substage="SECURITY.acceptance",
required=True,
row_ids=rows,
protocol="vless",
network="explicit-selected-compositions",
security="static-ech/encryption/hybrid-reality/jls",
udp_codec="raw/xudp/packetaddr/sing-mux",
field_values=options,
outer_family="IPv4/IPv6",
inner_family="IPv4/IPv6/domain",
target_type=(
"memory-only/build"
if kind == "unit"
else "isolated synthetic origin"
),
upstream_graph="direct/concrete/nested-select",
peer_kind=kind,
peer_config=dict(runner=runner, **options),
gap_source=dict(
reason="Mihomo v1.19.31 Safari ECH fails before business data. "
"Only the gRPC close reference uses Chrome with the same ECH "
"listener; VCore remains Safari, not a same-profile comparison.",
source="https://github.com/MetaCubeX/utls/blob/v1.8.7/u_handshake_client.go",
)
if identifier == "SECURITY-ECH-SAFARI"
else None
if kind != "XR"
else dict(
reason="Mihomo has no H3 or legacy HTTP/H2/extended WS listener; "
"official Xray/V2Ray terminate that transport, with "
"Mihomo VLESS for packetaddr, Encryption and sing-mux.",
source="https://github.com/XTLS/Xray-core",
),
expected_observation=[identifier],
required_evidence=["command", "cleanup", "structured-assertions"]
+ ([] if kind == "unit" else ["peer-identity", "isolated-origins"]),
prerequisites=["rust-toolchain"]
if kind == "unit"
else ["owned-host-only-container-network", "official-latest-peer"],
runner="vless-security-" + runner,
timeout_seconds=3600,
)
)
return result
@@ -0,0 +1,42 @@
# Exact configuration and in-memory TLS assertion identities for SECURITY.
TESTS = {
"ech_config": [
"static_ech_builds_both_tls_backends_without_dns_or_io",
"static_ech_is_strict_and_never_enables_dynamic_lookup",
"unsupported_ech_configs_fail_at_parse_before_a_client_can_emit_inner_sni",
"ech_selection_skips_unknown_or_mandatory_configs_and_keeps_exact_supported_bytes",
"download_ech_inherits_replaces_or_clears_as_a_whole",
"ech_is_standard_tls_only_with_explicit_download_clear_and_dns_inner_name",
],
"ech_tls": [
"rejected_or_wrong_key_ech_never_reaches_application_data_even_with_a_matching_pin",
"client_identity_is_sent_only_after_ech_acceptance",
"cancelled_ech_emits_only_public_sni_and_releases_io_for_every_backend",
"each_backend_and_hpke_suite_requires_real_ech_acceptance_before_data",
],
"encryption_config": [
"public_encryption_accepts_all_six_modes_without_exposing_key_material",
"vision_encryption_is_independent_of_outer_tls_but_keeps_transport_and_udp_limits",
],
"reality_config": [
"public_reality_accepts_explicit_hybrid_without_replacing_the_fingerprint",
"public_reality_rejects_profiles_without_the_required_hybrid_share",
"download_inherits_or_replaces_the_entire_reality_object_without_leaf_merging",
"hybrid_flag_is_a_strict_nonnullable_boolean_on_both_legs",
"classic_default_and_explicit_false_preserve_all_existing_profiles",
"inherited_profile_is_validated_after_download_reality_replacement",
"hybrid_reality_cannot_be_combined_with_h3_or_plaintext_on_either_leg",
],
"jls_config": [
"public_jls_builds_an_authenticated_security_client_without_exposing_credentials",
"independent_download_inherits_jls_identity_and_applies_its_own_tls_fields",
"download_jls_credentials_are_replaced_as_a_whole_or_explicitly_cleared",
"malformed_or_partial_credentials_never_inherit_a_missing_secret",
"invalid_jls_objects_do_not_echo_credentials_in_public_configuration_errors",
"jls_rejects_incompatible_security_at_configuration_time",
"download_security_switch_requires_explicit_clearing_of_inherited_identity",
"selected_profiles_build_jls_without_changing_its_identity",
"cancelled_jls_handshakes_release_supplied_io_and_never_reuse_randoms",
"jls_security_client_rejects_ordinary_tls_before_application_data",
],
}
@@ -0,0 +1,310 @@
"""Transport-only native cases; synthetic VLESS is not public YAML support."""
from __future__ import annotations
import contextlib
import json
import socket
import socketserver
import subprocess
import tempfile
import threading
import time
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import reserve_port
from .protocol_fixtures import certificates
from .protocol_peers import OwnedProcess, run_command
from .protocol_preflight import preflight
CLIENT_ID = "b831381d-6324-4d53-ad4f-8cda48b30811" # Public synthetic identity.
GREETING, TRAILER = b"FOUNDATIONS-server-first\n", b"FOUNDATIONS-half-close\n"
STREAM_CASES = {
"FOUNDATIONS-M-WS": ("M", "ws"),
"FOUNDATIONS-M-WSS": ("M", "wss"),
"FOUNDATIONS-M-GRPC": ("M", "grpc"),
"FOUNDATIONS-M-GRPC-TLS": ("M", "grpc-tls"),
"FOUNDATIONS-M-WS-ED": ("M", "ws-header"),
"FOUNDATIONS-V2-HTTP": ("V2", "http"),
"FOUNDATIONS-V2-H2": ("V2", "h2"),
"FOUNDATIONS-V2-WS-HEADER": ("V2", "ws-header"),
"FOUNDATIONS-V2-WS-PATH": ("V2", "ws-path"),
}
def abnormal_cleanup(binary: Path, directory: Path, cert: Path, key: Path) -> dict:
"""A real native peer must be joined even when its calling case unwinds."""
class InjectedFailure(Exception):
pass
record = {}
with contextlib.ExitStack() as stack:
port, reservation = reserve_port(stack)
config = directory / "cleanup-peer.json"
config.write_text(json.dumps(peer_config("M", "ws", port, cert, key)))
reservation.release_ipv4()
owner = OwnedProcess(
[str(binary), "-d", str(directory), "-f", str(config)],
directory / "cleanup-peer.log",
record,
)
try:
with owner:
owner.wait_tcp(port)
raise InjectedFailure
except InjectedFailure:
pass
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as rebind:
rebind.bind(("127.0.0.1", port))
if not record.get("joined") or owner.process.poll() is None:
raise RuntimeError("native abnormal cleanup failed")
return {
"passed": True,
"kind": "M",
"failure_injected": True,
"joined": True,
"port_rebound": True,
}
class Echo(socketserver.BaseRequestHandler):
def handle(self):
self.server.accepted += 1
self.request.settimeout(15)
try:
self.request.sendall(GREETING)
while data := self.request.recv(8192):
self.server.received += len(data)
self.request.sendall(data)
if self.server.received >= 65536:
break
self.request.sendall(TRAILER)
except OSError:
self.server.failed = True
finally:
self.server.finished.set()
@contextlib.contextmanager
def origin():
with socketserver.TCPServer(("127.0.0.1", 0), Echo) as server:
server.accepted, server.received, server.failed = 0, 0, False
server.finished = threading.Event()
thread = threading.Thread(target=server.serve_forever)
thread.start()
try:
yield server
finally:
server.shutdown()
thread.join(timeout=20)
if thread.is_alive():
raise RuntimeError("origin cleanup failed")
def peer_config(kind, mode, port, cert, key):
encrypted = mode in {"wss", "grpc-tls", "h2"}
if kind == "M":
listener = {
"name": "foundations-stream",
"type": "vless",
"listen": "127.0.0.1",
"port": port,
"users": [{"uuid": CLIENT_ID}],
}
if encrypted:
listener.update(certificate=str(cert), **{"private-key": str(key)})
else:
listener["allow-insecure"] = True # Explicit loopback-only plaintext.
if mode.startswith("ws"):
listener["ws-path"] = "/foundations-ws"
if mode.startswith("grpc"):
listener["grpc-service-name"] = "foundations-grpc"
return {
"mode": "rule",
"log-level": "silent",
"allow-lan": False,
"bind-address": "127.0.0.1",
"listeners": [listener],
"rules": ["MATCH,DIRECT"],
}
stream = {"network": "tcp", "security": "tls" if encrypted else "none"}
if mode == "http":
stream["tcpSettings"] = {
"header": {"type": "http", "request": {"path": ["/foundations-http"]}}
}
elif mode == "h2":
stream.update(
network="http",
httpSettings={"host": ["localhost"], "path": "/foundations-h2"},
)
else:
stream.update(
network="ws",
wsSettings={
"path": "/foundations-ws",
"maxEarlyData": 2048,
"earlyDataHeaderName": "x-vcore-ed" if mode == "ws-header" else "",
},
)
if encrypted:
stream["tlsSettings"] = {
"alpn": ["h2"],
"certificates": [{"certificateFile": str(cert), "keyFile": str(key)}],
}
return {
"log": {"loglevel": "none"},
"inbounds": [
{
"listen": "127.0.0.1",
"port": port,
"protocol": "vless",
"settings": {"clients": [{"id": CLIENT_ID}], "decryption": "none"},
"streamSettings": stream,
}
],
"outbounds": [{"protocol": "freedom", "settings": {}}],
}
def run_streams(output: Path, selected: list[str] | None = None, *, artifacts=None):
selected = list(STREAM_CASES) if selected is None else selected
if (
not selected
or len(set(selected)) != len(selected)
or not set(selected) <= STREAM_CASES.keys()
):
raise RuntimeError("invalid native stream case selection")
output.mkdir(parents=True, exist_ok=False)
report = {
"scope": "FOUNDATIONS-shared-transport-only",
"cases": [],
"peers": [],
"cleanup": False,
}
try:
kinds = {STREAM_CASES[case][0] for case in selected}
if artifacts is None:
artifacts, report["preflight"] = preflight(output / "binaries", kinds)
peers = {kind: item.binary for kind, item in artifacts.items() if kind in kinds}
report["peers"] = [
item.identity for kind, item in artifacts.items() if kind in kinds
]
with tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary:
temporary = Path(temporary)
cert, key, pin = certificates(temporary)
if "M" in peers:
report["abnormal_cleanup"] = abnormal_cleanup(
peers["M"], temporary, cert, key
)
for case_id in selected:
kind, mode = STREAM_CASES[case_id]
record = {
"case_id": case_id,
"peer_kind": kind,
"mode": mode,
"status": "NOT RUN",
"cleanup": {},
}
report["cases"].append(record)
if kind not in peers:
record.update(
status="BLOCKED",
reason="required peer unavailable",
cleanup={"joined": True, "started": False},
)
continue
started = time.monotonic()
try:
with contextlib.ExitStack() as stack:
port, reservation = reserve_port(stack)
config = temporary / "peer.json"
config.write_text(
json.dumps(peer_config(kind, mode, port, cert, key))
)
command = (
[str(peers[kind]), "-d", str(temporary), "-f", str(config)]
if kind == "M"
else [str(peers[kind]), "run", "-c", str(config)]
)
reservation.release_ipv4()
peer = stack.enter_context(
OwnedProcess(
command, temporary / "peer.log", record["cleanup"]
)
)
peer.wait_tcp(port)
with origin() as echo:
fixture = {
"mode": mode,
"server": f"127.0.0.1:{port}",
"target": f"127.0.0.1:{echo.server_address[1]}",
"pin": pin,
"ed_header": "sec-websocket-protocol"
if kind == "M"
else "x-vcore-ed",
}
probe_config = temporary / "probe.json"
probe_config.write_text(json.dumps(fixture))
completed = run_command(
[
str(
CORE_DIR
/ "target/debug/examples/protocol-stream-probe"
),
str(probe_config),
],
timeout=20,
limit=65536,
)
result = json.loads(completed.stdout)
record["rust_event"] = result
record["command_exit_code"] = completed.returncode
record["origin"] = {
"finished": echo.finished.wait(3),
"accepted": echo.accepted,
"received": echo.received,
"failed": echo.failed,
}
peer.ensure_alive()
record["status"] = (
"PASS"
if completed.returncode == 0
and result["outcome"] == "pass"
and result["driver_joined"]
and result["protect_calls"] == 1
and result.get("resources_idle") is True
and result.get("server_first") is True
and result.get("payload_bytes") == 65536
and result.get("tail_bytes") == len(TRAILER)
and completed.cleanup
and record["origin"]
== {
"finished": True,
"accepted": 1,
"received": 65536,
"failed": False,
}
else "FAIL"
)
except (
OSError,
ValueError,
RuntimeError,
subprocess.SubprocessError,
) as error:
record.update(status="FAIL", reason=type(error).__name__)
finally:
record["seconds"] = round(time.monotonic() - started, 3)
if not record["cleanup"].get("joined"):
record["status"] = "FAIL"
print(json.dumps(record), flush=True)
report["cleanup"] = True
finally:
(output / "stream-cases.json").write_text(json.dumps(report, indent=2) + "\n")
if not report["cleanup"] or any(
case["status"] != "PASS" for case in report["cases"]
):
raise RuntimeError("native stream cases did not pass; see structured report")
return report
@@ -0,0 +1,363 @@
"""TROJAN public Trojan consumer acceptance against owned official native peers."""
from __future__ import annotations
import contextlib
import json
import os
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_isolation import reserve_port
from .protocol_evidence import idle_resources, read_events
from .protocol_fixtures import certificate_chain, certificates
from .protocol_fixtures import trojan_peer_config as peer_config
from .protocol_inputs import redact, source_identity
from .protocol_peers import OwnedProcess, run_command
from .protocol_preflight import preflight
CASES = {
"TROJAN-M-TCP": ("M", "tcp", "public_trojan_native_base"),
"TROJAN-M-WS": ("M", "ws", "public_trojan_native_base"),
"TROJAN-M-GRPC": ("M", "grpc", "public_trojan_native_base"),
"TROJAN-M-WS-ED-1": ("M", "ws-ed-1", "public_trojan_native_base"),
"TROJAN-M-WS-ED-2048": ("M", "ws-ed-2048", "public_trojan_native_base"),
"TROJAN-M-WS-POLICY": (
"M",
"ws",
"public_trojan_native_policy_and_group_snapshots",
),
"TROJAN-M-GRPC-POLICY": (
"M",
"grpc",
"public_trojan_native_policy_and_group_snapshots",
),
"TROJAN-M-WS-NEGATIVE": ("M", "ws", "public_trojan_native_transport_negative"),
"TROJAN-M-GRPC-NEGATIVE": ("M", "grpc", "public_trojan_native_transport_negative"),
"TROJAN-M-ALPN-NEGATIVE": (
"M",
"ws-alpn",
"public_trojan_native_transport_negative",
),
"TROJAN-V2-WS-HEADER": (
"V2",
"ws-header",
"public_trojan_native_extended_early_data",
),
"TROJAN-V2-WS-PATH": ("V2", "ws-path", "public_trojan_native_extended_early_data"),
"TROJAN-M-TCP-POLICY": (
"M",
"tcp",
"public_trojan_native_policy_and_group_snapshots",
),
"TROJAN-XR-UDP-DOMAIN": ("XR", "tcp", "public_trojan_native_udp_domain"),
"TROJAN-XR-WS-UDP-DOMAIN": ("XR", "ws", "public_trojan_native_udp_domain"),
"TROJAN-XR-GRPC-UDP-DOMAIN": ("XR", "grpc", "public_trojan_native_udp_domain"),
}
for _mode in ("tcp", "ws", "grpc"):
for _suffix, _test in (
("LIFE", "runtime::public_trojan_native_lifecycle"),
("ENTRYPOINTS", "runtime::public_trojan_native_entrypoints"),
("UDP-ISOLATION", "runtime::public_trojan_native_udp_isolation_and_limit"),
):
CASES[f"TROJAN-M-{_mode.upper()}-{_suffix}"] = ("M", _mode, _test)
CASES[f"TROJAN-M-{_mode.upper()}-IPV6"] = (
"M",
_mode + "-ipv6",
"public_trojan_native_base",
)
CASES[f"TROJAN-M-{_mode.upper()}-OWNED"] = (
"M",
_mode,
"trojan_native_owned_resources",
)
CASES[f"TROJAN-M-{_mode.upper()}-CERTIFICATE"] = (
"M",
_mode + "-ca",
"runtime::public_trojan_native_certificate_names",
)
CASES["TROJAN-M-GRPC-CUSTOM"] = ("M", "grpc-custom", "public_trojan_native_base")
def native_events_pass(events, test):
suites = {"TROJAN-NATIVE"}
if test == "runtime::public_trojan_native_lifecycle":
suites.add("TROJAN-LIFE-CYCLE")
elif test == "trojan_native_owned_resources":
suites.add("TROJAN-OWNED-CYCLE")
main = [event for event in events if event.get("suite") == "TROJAN-NATIVE"]
if (
len(main) != 2
or any(
event.get("assertion") != test or event.get("schema_version") != 1
for event in main
)
or [event.get("status") for event in main] != ["BEGIN", "PASS"]
):
return False
if test in {
"runtime::public_trojan_native_lifecycle",
"trojan_native_owned_resources",
}:
suite = (
"TROJAN-OWNED-CYCLE"
if test == "trojan_native_owned_resources"
else "TROJAN-LIFE-CYCLE"
)
cycles = [event for event in events if event.get("suite") == suite]
if len(cycles) != 40:
return False
for begin, end in zip(cycles[::2], cycles[1::2], strict=True):
if (
begin.get("status") != "BEGIN"
or end.get("status") != "PASS"
or end.get("seconds", 0) < 5
or any(
event.get("assertion") != "stop_and_remain_quiet"
or event.get("schema_version") != 1
for event in (begin, end)
)
):
return False
if suite == "TROJAN-OWNED-CYCLE":
points = end.get("checkpoints", [])
phases = {point["phase"]: point["resources"] for point in points}
if (
len(points) != 3
or set(phases) != {"baseline", "after-stop", "quiet"}
or not all(idle_resources(value) for value in phases.values())
or phases["after-stop"] != phases["quiet"]
or not idle_resources(end.get("resources"))
):
return False
return all(
event.get("status") in {"BEGIN", "PASS"}
and event.get("schema_version") == 1
and event.get("suite") in suites
for event in events
)
def run(output: Path, selected=None, *, artifacts=None):
selected = list(CASES) if selected is None else selected
if (
not selected
or len(selected) != len(set(selected))
or not set(selected) <= CASES.keys()
):
raise ValueError("invalid TROJAN case selection")
output.mkdir(parents=True, exist_ok=False)
report = {
"stage": "TROJAN",
"scope": "protocol-consumer",
"source": source_identity(),
"selected_cases": selected,
"cases": [],
"cleanup": False,
}
try:
if artifacts is None:
artifacts, report["preflight"] = preflight(
output / "binaries", {CASES[case][0] for case in selected}
)
for case_id in selected:
kind, mode, test = CASES[case_id]
record = {
"case_id": case_id,
"mode": mode,
"cleanup": {},
"status": "NOT RUN",
}
report["cases"].append(record)
if kind not in artifacts:
record.update(
status="BLOCKED", reason="required official native peer unavailable"
)
continue
record["peer"] = artifacts[kind].identity
print(f"TROJAN: {case_id}", flush=True)
with (
tempfile.TemporaryDirectory(prefix="private-", dir=output) as directory,
contextlib.ExitStack() as stack,
):
directory = Path(directory)
cert, key, pin = (
certificate_chain if mode.endswith("-ca") else certificates
)(directory)
port, reservation = reserve_port(stack)
password = " synthetic TROJAN 密码 "
config = peer_config(kind, mode, port, password, cert, key)
path = directory / "peer.json"
path.write_text(json.dumps(config))
node = {
"name": "peer",
"type": "trojan",
"server": "127.0.0.1",
"port": port,
"password": password,
"udp": True,
"sni": "localhost",
"fingerprint": pin,
}
if mode.startswith("ws"):
node.update(
network="ws",
**{
"ws-opts": {
"path": "/trojan-ws?q=1",
"headers": {
"Host": "cover.example:443",
"X-TROJAN": "fixture",
},
}
},
)
if mode.startswith("grpc"):
node.update(
network="grpc",
**{"grpc-opts": {"grpc-service-name": "trojan-grpc"}},
)
if mode.startswith("ws") and kind != "M":
node["ws-opts"]["path"] = "/trojan-ws/"
if mode in {"ws-ed-1", "ws-ed-2048", "ws-header", "ws-path"}:
node["ws-opts"]["max-early-data"] = 1 if mode == "ws-ed-1" else 2048
if mode in {"ws-header", "ws-path"}:
node["ws-opts"]["early-data-header-name"] = (
"x-vcore-ed" if mode == "ws-header" else ""
)
if mode == "ws-alpn":
node["alpn"] = ["h2", "http/1.1"]
if mode == "grpc-custom":
node["grpc-opts"]["grpc-service-name"] = "/trojan-grpc/Tun"
if mode.endswith("-ipv6"):
node["server"] = "::1"
command = (
[str(artifacts[kind].binary), "-d", str(directory), "-f", str(path)]
if kind == "M"
else [str(artifacts[kind].binary), "run", "-c", str(path)]
)
if mode.endswith("-ipv6"):
reservation.release_ipv6()
else:
reservation.release_ipv4()
with OwnedProcess(
command, directory / "peer.log", record["cleanup"]
) as peer:
peer.wait_tcp(port, host=node["server"])
hop = None
if test == "public_trojan_native_policy_and_group_snapshots":
# Two independently owned peers, not one process proxying
# back into its own loopback detector.
hop_dir = directory / "hop"
hop_dir.mkdir()
hop_cert, hop_key, hop_pin = certificates(hop_dir)
hop_port, hop_reservation = reserve_port(stack)
hop_path = hop_dir / "peer.json"
hop_path.write_text(
json.dumps(
peer_config(
kind, mode, hop_port, password, hop_cert, hop_key
)
)
)
record["upstream_cleanup"] = {}
hop_reservation.release_ipv4()
hop_peer = stack.enter_context(
OwnedProcess(
[
str(artifacts[kind].binary),
"-d",
str(hop_dir),
"-f",
str(hop_path),
],
hop_dir / "peer.log",
record["upstream_cleanup"],
)
)
hop_peer.wait_tcp(hop_port)
hop = dict(node, name="hop", port=hop_port, fingerprint=hop_pin)
env = dict(
os.environ,
VCORE_CASE_EVENTS=str(output / f"{case_id}-events.jsonl"),
VCORE_TROJAN_FIXTURE=json.dumps(
{
"node": node,
"hop": hop,
"mode": mode,
"udp_payload_max": 2048
if kind == "V2"
else 8166
if kind == "XR"
else 8192,
"data_dir": str(directory / "core"),
}
),
)
target = (
"trojan_lifecycle"
if test == "trojan_native_owned_resources"
else "mihomo_interop"
)
test_name = (
test if target == "trojan_lifecycle" else f"trojan::{test}"
)
command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
target,
test_name,
"--",
"--ignored",
"--exact",
"--nocapture",
]
result = run_command(
command,
cwd=CORE_DIR,
env=env,
timeout=180,
limit=4 * 1024 * 1024,
)
(output / f"{case_id}.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
record.update(
status="PASS"
if result.returncode == 0
and result.cleanup
and native_events_pass(
read_events(output / f"{case_id}-events.jsonl")
if (output / f"{case_id}-events.jsonl").exists()
else [],
test,
)
else "FAIL",
exit_code=result.returncode,
command_cleanup=result.cleanup,
seconds=result.seconds,
command=command,
)
peer.ensure_alive()
report["cleanup"] = all(
case["cleanup"].get("joined")
and case.get("upstream_cleanup", {"joined": True}).get("joined")
for case in report["cases"]
)
report["source_unchanged"] = (
source_identity()["source_tree_sha256"]
== report["source"]["source_tree_sha256"]
)
return (
0
if report["cleanup"]
and report["source_unchanged"]
and all(case["status"] == "PASS" for case in report["cases"])
else 1
)
finally:
(output / "trojan-results.json").write_text(json.dumps(report, indent=2) + "\n")
@@ -0,0 +1,329 @@
"""Trojan stage execution and independent, fail-closed persisted evidence checks."""
from __future__ import annotations
import json
from .protocol_evidence import HEX, new_result, read_events, read_json, rust_results
from .protocol_preflight import preflight
from .protocol_trojan import native_events_pass
from .protocol_trojan import run as run_native
# Frozen independently of cases.json: removing or downgrading a case is not signoff.
REQUIRED_IDS = {
"TROJAN-CODEC",
"TROJAN-CFG",
"TROJAN-CANCEL",
"TROJAN-M-TCP",
"TROJAN-M-WS",
"TROJAN-M-GRPC",
"TROJAN-M-WS-ED-1",
"TROJAN-M-WS-ED-2048",
"TROJAN-M-WS-POLICY",
"TROJAN-M-GRPC-POLICY",
"TROJAN-M-WS-NEGATIVE",
"TROJAN-M-GRPC-NEGATIVE",
"TROJAN-M-ALPN-NEGATIVE",
"TROJAN-V2-WS-HEADER",
"TROJAN-V2-WS-PATH",
"TROJAN-M-TCP-POLICY",
"TROJAN-XR-UDP-DOMAIN",
"TROJAN-XR-WS-UDP-DOMAIN",
"TROJAN-XR-GRPC-UDP-DOMAIN",
"TROJAN-M-TCP-LIFE",
"TROJAN-M-TCP-ENTRYPOINTS",
"TROJAN-M-TCP-UDP-ISOLATION",
"TROJAN-M-TCP-IPV6",
"TROJAN-M-TCP-OWNED",
"TROJAN-M-TCP-CERTIFICATE",
"TROJAN-M-WS-LIFE",
"TROJAN-M-WS-ENTRYPOINTS",
"TROJAN-M-WS-UDP-ISOLATION",
"TROJAN-M-WS-IPV6",
"TROJAN-M-WS-OWNED",
"TROJAN-M-WS-CERTIFICATE",
"TROJAN-M-GRPC-LIFE",
"TROJAN-M-GRPC-ENTRYPOINTS",
"TROJAN-M-GRPC-UDP-ISOLATION",
"TROJAN-M-GRPC-IPV6",
"TROJAN-M-GRPC-OWNED",
"TROJAN-M-GRPC-CERTIFICATE",
"TROJAN-M-GRPC-CUSTOM",
"TROJAN-FEATURES",
"TROJAN-SCRIPTS",
"TROJAN-REGRESSION",
}
FIELD_IDS = {
"C01",
"C02",
"C03",
"C04",
"C05",
"C06",
"T01",
"T03",
"T04",
"T05",
"W01",
"W02",
"W03",
"W04",
"W05",
"G01",
"TR01",
"TR02",
}
def native_results(cases, report, directory):
records = report.get("cases", [])
if [entry.get("case_id") for entry in records] != [
case["case_id"] for case in cases
]:
raise ValueError("missing, duplicate or reordered Trojan native evidence")
results = []
for case, record in zip(cases, records, strict=True):
path = directory / (case["case_id"] + "-events.jsonl")
events = read_events(path) if path.exists() else []
observed = native_events_pass(events, case["peer_config"]["test"])
clean = (
record.get("command_cleanup") is True
and record.get("cleanup", {}).get("joined") is True
and record.get("upstream_cleanup", {"joined": True}).get("joined") is True
)
passed = (
observed
and clean
and record.get("exit_code") == 0
and record.get("status") == "PASS"
and record.get("mode") == case["peer_config"]["mode"]
and report.get("source_unchanged") is True
)
result = new_result(case)
result.update(
status="PASS" if passed else "FAIL",
assertions={name: observed for name in case["expected_observation"]},
evidence=case["required_evidence"] if passed else [],
command_exit_code=record.get("exit_code"),
cleanup=clean,
)
results.append(result)
return results
def fields_report(cases, results):
statuses = {result["case_id"]: result["status"] for result in results}
complete = {case["case_id"] for case in cases} == REQUIRED_IDS
return {
"schema_version": 1,
"stage": "TROJAN",
"protocol": "trojan",
"scope": "protocol-consumer; other protocols remain NOT RUN",
"fields": [
{
"row_id": row,
"status": "PASS"
if complete
and owners
and all(statuses.get(case_id) == "PASS" for case_id in owners)
else "NOT RUN",
"required_cases": owners,
}
for row in sorted(FIELD_IDS)
for owners in [
[case["case_id"] for case in cases if row in case["row_ids"]]
]
],
"peer_limitations": [
"Mihomo v1.19.31 domain UDP FAIL is retained; "
"Xray TCP/WS/gRPC supplies separate domain evidence.",
"V2Ray ED supplement UDP path is bounded by its 2048-byte return buffer; "
"8192-byte truncation is retained as peer failure.",
"Xray domain UDP return frame has 8192 total bytes, "
"leaving 8166 payload for the controlled domain; "
"larger peer failure retained.",
],
}
def execute(cases, run, output, records):
from .protocol_harness import _command, _events, _features, _legacy, _scripts
rust = [case for case in cases if case["runner"] == "rust"]
if rust:
command = _command(
run,
output,
"rust-consumer",
["cargo", "test", "--locked", "--all-features", "--all-targets"],
600,
events=output / "rust-events.jsonl",
)
events = _events(output / "rust-events.jsonl")
for case in rust:
result = rust_results(case, events, command.returncode)
result["cleanup"] = command.cleanup
records[case["case_id"]] = result
kinds = {case["peer_kind"] for case in cases if case["peer_kind"] != "unit"}
artifacts, peers = preflight(
output / "binaries",
kinds,
container=any(case["runner"] == "mihomo-legacy" for case in cases),
)
(output / "peers.json").write_text(json.dumps(peers, indent=2) + "\n")
native = [case for case in cases if case["runner"] == "native-trojan"]
if native:
directory = output / "native"
run_native(directory, [case["case_id"] for case in native], artifacts=artifacts)
report = read_json(directory / "trojan-results.json")
for result in native_results(native, report, directory):
records[result["case_id"]] = result
for case in cases:
if case["runner"] == "features":
records[case["case_id"]] = _features(case, run, output)
elif case["runner"] == "scripts":
records[case["case_id"]] = _scripts(case, run, output)
elif case["runner"] == "mihomo-legacy":
records[case["case_id"]] = _legacy(case, run, output, artifacts)
(output / "fields.json").write_text(
json.dumps(fields_report(cases, list(records.values())), indent=2) + "\n"
)
def check(run_dir, required, run, results, peers, paths):
from .protocol_harness import SCRIPT_OBSERVATIONS
necessary = {
"cases.json",
"peers.json",
"resources.jsonl",
"summary.md",
"rust-events.jsonl",
"legacy-events.jsonl",
"script-tests.json",
"native/trojan-results.json",
"fields.json",
}
if not necessary <= set(paths):
raise ValueError("missing Trojan acceptance artifacts")
expected_commands = {
"rust-consumer",
"legacy-mihomo",
"offline-scripts",
"feature-default",
"feature-minimal",
"feature-outbound-trojan",
"feature-outbound-vmess",
"feature-outbound-hysteria2",
"feature-stream-transport",
"feature-quic-transport",
}
commands = run.get("commands", [])
if (
len(commands) != len(expected_commands)
or {command.get("name") for command in commands} != expected_commands
or any(
type(command.get("exit_code")) is not int
or command["exit_code"] != 0
or command.get("cleanup") is not True
or (command.get("log") and command["log"] not in paths)
for command in commands
)
):
raise ValueError("missing, failed or unjoined Trojan acceptance command")
native = [case for case in required if case["runner"] == "native-trojan"]
report = read_json(run_dir / "native/trojan-results.json")
if (
report.get("source_unchanged") is not True
or report.get("cleanup") is not True
or report.get("source", {}).get("source_tree_sha256")
!= run["source_tree_sha256"]
or report.get("source", {}).get("lock_sha256") != run["lock_sha256"]
):
raise ValueError("Trojan peer tests used another source input")
for record, case in zip(report.get("cases", []), native, strict=True):
for suffix in (".log", "-events.jsonl"):
if "native/" + case["case_id"] + suffix not in paths:
raise ValueError("native log/events missing immutable identity")
identity = [peer for peer in peers if peer.get("kind") == case["peer_kind"]]
if (
len(identity) != 1
or identity[0].get("status") != "READY"
or not identity[0].get("version")
or not HEX.fullmatch(identity[0].get("binary_sha256", ""))
or not HEX.fullmatch(identity[0].get("archive_sha256", ""))
or not identity[0].get("source_url", "").startswith("https://github.com/")
or any(
record.get("peer", {}).get(key) != identity[0].get(key)
for key in ("version", "binary_sha256", "archive_sha256", "source_url")
)
):
raise ValueError("Trojan used a peer different from official preflight")
test = case["peer_config"]["test"]
target = (
"trojan_lifecycle"
if test == "trojan_native_owned_resources"
else "mihomo_interop"
)
name = test if target == "trojan_lifecycle" else "trojan::" + test
if record.get("command") != [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
target,
name,
"--",
"--ignored",
"--exact",
"--nocapture",
]:
raise ValueError("Trojan case did not execute its declared public consumer")
calculated = native_results(native, report, run_dir / "native")
rust = read_events(run_dir / "rust-events.jsonl")
legacy = read_events(run_dir / "legacy-events.jsonl")
for case in required:
if case["runner"] in {"rust", "mihomo-legacy"}:
entry = rust_results(
case, legacy if case["runner"] == "mihomo-legacy" else rust, 0
)
if case["runner"] == "mihomo-legacy" and entry["status"] == "PASS":
entry["evidence"] = case["required_evidence"]
calculated.append(entry)
actual = {entry["case_id"]: entry for entry in results}
for entry in calculated:
if actual[entry["case_id"]] != entry:
raise ValueError(
"Trojan result disagrees with original structured assertions"
)
script = read_json(run_dir / "script-tests.json")
tests = {entry["test"]: entry["status"] for entry in script.get("cases", [])}
if (
not tests
or len(tests) != len(script.get("cases", []))
or len(tests) != script.get("tests_run")
or any(status != "PASS" for status in tests.values())
or any(tests.get(name) != "PASS" for name in SCRIPT_OBSERVATIONS.values())
or not any("test_protocol_trojan." in name for name in tests)
):
raise ValueError("Trojan harness failure paths were not exercised")
mihomo = next(peer for peer in peers if peer["kind"] == "M")
if (
mihomo.get("container_status") != "READY"
or not HEX.fullmatch(
mihomo.get("container_artifact", {}).get("binary_sha256", "")
)
or mihomo["container_artifact"].get("release") != mihomo.get("release")
):
raise ValueError("old regression did not use the same official Mihomo snapshot")
field_report = fields_report(required, results)
if read_json(run_dir / "fields.json") != field_report or any(
row["status"] != "PASS" for row in field_report["fields"]
):
raise ValueError("incomplete or inconsistent Trojan field report")
print(
f"TROJAN: PASS ({len(required)} required cases, "
f"{len(FIELD_IDS)} Trojan fields); "
"other protocols remain NOT RUN"
)
@@ -0,0 +1,27 @@
"""VLESS development mode selection, using the same container fixtures as acceptance.
This entry does not execute local/build gates or sign off the complete stage.
"""
from pathlib import Path
from .protocol_vless_container import run as run_cases
from .protocol_vless_peers import MODES
from .protocol_vless_public import CASES
def run(output: Path, modes=None):
modes = MODES if modes is None else modes
if not modes or not set(modes) <= set(MODES) or len(set(modes)) != len(modes):
raise ValueError("invalid VLESS development mode")
selected = [case for case, (_, mode, _, _) in CASES.items() if mode in modes]
return run_cases(output, selected)
if __name__ == "__main__":
import sys
from .mihomo_isolation import exclusive_run
with exclusive_run():
sys.exit(run(Path(sys.argv[1]).resolve(), sys.argv[2:] or None))
@@ -0,0 +1,652 @@
"""VLESS container-only public consumer acceptance and independent coverage."""
from __future__ import annotations
import json
import sys
from .builds import DEFAULT_FEATURES
from .protocol_evidence import HEX, idle_resources, new_result, read_events, read_json
from .protocol_inputs import redact
from .protocol_vless_container import run as native_run
from .protocol_vless_public import PUBLIC, WIRE, events_pass
from .protocol_vmess_acceptance import OBSERVATIONS as PREVIOUS_OBSERVATIONS
from .protocol_vmess_acceptance import SCRIPT_ASSERTIONS
NATIVE = WIRE | PUBLIC
FIELD_IDS = {
*(f"C{i:02}" for i in range(1, 7)),
*(f"T{i:02}" for i in range(2, 9)),
*(f"W{i:02}" for i in range(1, 8)),
*(f"G{i:02}" for i in range(1, 7)),
*(f"H{i:02}" for i in range(1, 7)),
*(f"VL{i:02}" for i in range(1, 6)),
"S01",
"S02",
}
GATES = {
"VLESS-CFG",
"VLESS-CODEC",
"VLESS-TRANSPORT",
"VLESS-VISION",
"VLESS-REGRESSION",
"VLESS-RELEASE",
"VLESS-FEATURES",
"VLESS-SCRIPTS",
"VLESS-QUALITY",
}
REQUIRED_IDS = set(NATIVE) | GATES
OBSERVATIONS = {
"VLESS-CFG": [
"inline_client_identity_is_validated_and_redacted_before_io",
"vision_requires_tcp_tls13_and_xudp_before_any_io",
"default_tcp_and_three_udp_encodings_are_accepted_without_io",
"tcp_and_existing_xhttp_reject_mismatched_and_future_options",
"stream_transports_and_explicit_tls_policy_have_strict_public_fields",
"extended_ws_and_grpc_fields_are_scoped_and_bounded",
],
"VLESS-CODEC": [
"stopping_before_first_udp_send_wakes_receive_and_releases_io",
"raw_and_packetaddr_consume_wire_and_keep_cancelled_receive_progress",
"udp_cancelled_send_and_bad_response_header_close_owned_io",
"tcp_bad_response_header_poisoning_and_absolute_response_deadline",
"vless_all_handshakes_keep_the_original_deadline_and_join_cancelled_io",
"vless_expired_deadline_and_protect_failure_never_fall_back",
],
"VLESS-TRANSPORT": [
"http_camouflage_shutdown_releases_both_directions_without_waiting_for_peer_eof",
"grpc_server_first_flushes_vless_request_without_an_application_write",
"http_upgrade_consumes_early_prefix_and_requires_valid_101_even_fast_open",
"grpc_idle_ping_is_observable_disabled_by_zero_and_joined_at_stop",
"grpc_pool_matches_both_threshold_policies_and_keeps_other_streams_alive",
],
"VLESS-VISION": [
"vision_fragmented_headers_content_padding_and_cancelled_reads_keep_raw_tail",
"vision_invalid_uuid_command_and_truncation_poison_the_stream",
"vision_bounded_partial_writes_flush_and_close_preserve_all_plaintext",
"fragmented_server_hello_distinguishes_tls12_tls13_and_non_tls",
"record_boundary_preserves_buffered_plaintext_and_coalesced_raw_tail_and_flush_order",
],
}
OBSERVATIONS["VLESS-REGRESSION"] = (
PREVIOUS_OBSERVATIONS["VMESS-REGRESSION"]
+ PREVIOUS_OBSERVATIONS["VMESS-CFG"]
+ PREVIOUS_OBSERVATIONS["VMESS-CODEC"]
+ PREVIOUS_OBSERVATIONS["VMESS-CANCEL"]
+ [
"all_two_hop_protocol_combinations_build_as_connector_graphs",
"direct_download_constructor_reuses_or_requires_the_precise_prepared_endpoint",
"split_node_over_a_proxy_automatically_reuses_its_parent_for_both_legs",
]
)
OBSERVATIONS["VLESS-RELEASE"] = sum(
(
OBSERVATIONS[key]
for key in ("VLESS-CFG", "VLESS-CODEC", "VLESS-TRANSPORT", "VLESS-VISION")
),
[],
)
def rows(identifier):
if identifier == "VLESS-CFG":
return sorted(FIELD_IDS)
if identifier == "VLESS-CODEC":
return ["VL01", "VL02", "VL04"]
if identifier == "VLESS-TRANSPORT":
return [*(f"G{i:02}" for i in range(1, 7)), "W06", "W07"]
if identifier == "VLESS-VISION":
return ["VL03", "VL04", "VL05", "S01", "S02"]
if identifier in GATES or identifier.startswith("VLESS-REGRESSION"):
return []
_, mode, tls, test = NATIVE[identifier]
result = {"VL01", "VL02", "VL04", "VL05"}
if identifier in PUBLIC:
result |= {*(f"C{i:02}" for i in range(1, 6))}
if "graph" in test or "selection" in test:
result.add("C06")
if mode.endswith("-reality"):
result |= {"S01", "S02", "T02"}
elif tls:
result |= {"T02", "T03", "T04", "T05"}
if mode.endswith("-mtls"):
result |= {"T06", "T07", "T08"}
if mode.startswith("vision-"):
result.add("VL03")
if mode.startswith(("ws", "upgrade")):
result |= {*(f"W{i:02}" for i in range(1, 6))}
if mode.startswith("upgrade"):
result |= {"W06", "W07"}
if mode.startswith("grpc"):
result |= {*(f"G{i:02}" for i in range(1, 7))}
if mode.startswith("http"):
result |= {"H01", "H02", "H03", "H04"}
if mode.startswith("h2"):
result |= {"H05", "H06"}
return sorted(result)
def definitions():
cases = []
for identifier in sorted(REQUIRED_IDS):
kind, mode, tls, test = NATIVE.get(
identifier, ("unit", "memory-or-build", False, identifier)
)
native = identifier in NATIVE
substage = (
"VLESS.vision"
if mode.startswith("vision-") or identifier == "VLESS-VISION"
else "VLESS.tcp"
if mode in {"tcp", "tcp-tls"} or identifier == "VLESS-CODEC"
else "VLESS.transports"
if mode.startswith(("ws", "grpc", "http", "h2", "upgrade"))
or identifier == "VLESS-TRANSPORT"
else "VLESS.acceptance"
)
cases.append(
dict(
case_id=identifier,
stage="VLESS",
substage=substage,
required=True,
row_ids=rows(identifier),
protocol="vless",
network=mode,
security="tls" if tls else "plain",
udp_codec="raw/xudp/packetaddr"
if "udp" in test or test == "public_base"
else "per-assertion",
field_values={
"network": mode,
"tls": tls,
"scope": "protocol-consumer",
"close_reference": "ws-standard-tls-baseline"
if test == "native_ws_reality_close_boundary"
else "same-mode"
if test == "native_mihomo_close_alignment"
else "not-applicable",
},
outer_family="IPv4/IPv6" if "ipv6" in test else "IPv4",
inner_family="IPv4/IPv6/domain"
if "base" in test or "udp_boundaries" in test
else "per-assertion",
target_type="isolated synthetic origin" if native else "memory-only",
upstream_graph="direct/concrete/nested-select"
if "graph" in test
else "per-assertion",
peer_kind=kind,
peer_config=dict(mode=mode, tls=tls, test=test),
gap_source={
"reason": "Mihomo WS client ignores RealityOpts; "
"REALITY listener data "
"is native-tested, but close uses the separate standard-TLS WS "
"adapter reference plus independent REALITY close cases, not a "
"same-combination client differential. Original failures retained.",
"source": "https://github.com/MetaCubeX/mihomo/blob/ab405bad5beeeac8b003bb01f60f134f6df54471/adapter/outbound/vless.go",
}
if test == "native_ws_reality_close_boundary"
else {
"reason": "Mihomo VLESS listener has no HTTP header/legacy H2 "
"or custom ED field; official V2Ray supplements only these paths",
"source": "https://github.com/MetaCubeX/mihomo/blob/v1.19.31/listener/inbound/vless.go",
}
if kind == "V2"
else None,
expected_observation=[test]
if native
else SCRIPT_ASSERTIONS
if identifier == "VLESS-SCRIPTS"
else OBSERVATIONS.get(identifier, [identifier]),
required_evidence=["structured-assertions", "command", "cleanup"]
+ (["peer-identity", "isolated-origins"] if native else []),
prerequisites=[
"owned-host-only-container-network",
"official-latest-peer",
]
if native
else ["rust-toolchain"],
runner="native-vless" if native else "vless-gate",
timeout_seconds=240 if native else 1200,
)
)
return cases
def commands(identifier):
cargo = ["cargo", "test", "--locked", "--all-features"]
if identifier == "VLESS-CFG":
return [cargo + ["--test", "vless_config"], cargo + ["--lib", "config::"]]
if identifier == "VLESS-CODEC":
return [cargo + ["--test", "vless_codec", "--test", "vless_lifecycle"]]
if identifier == "VLESS-TRANSPORT":
return [cargo + ["--test", "vless_transports", "--test", "grpc_pool"]]
if identifier == "VLESS-VISION":
return [cargo + ["--lib", "vision"]]
if identifier == "VLESS-RELEASE":
return [
cargo
+ [
"--release",
"--test",
"vless_config",
"--test",
"vless_codec",
"--test",
"vless_lifecycle",
"--test",
"vless_transports",
"--test",
"grpc_pool",
],
cargo + ["--release", "--lib", "vision"],
]
if identifier == "VLESS-REGRESSION":
from .protocol_vmess_acceptance import commands as previous
return (
previous("VMESS-REGRESSION")
+ previous("VMESS-CFG")
+ previous("VMESS-CODEC")
+ previous("VMESS-CANCEL")
+ [
cargo
+ ["--lib", "outbound::vless::outbound::connector_composition_tests::"]
]
+ [
["cargo", "test", "--locked", "--no-default-features"]
+ features
+ [
"--test",
"feature_foundations",
"vless_yaml_follows_its_own_feature",
"--",
"--exact",
]
for features in ([], ["--features", "outbound-trojan"])
]
)
if identifier == "VLESS-QUALITY":
from .protocol_vmess_acceptance import commands as previous
return previous("VMESS-QUALITY")
return []
def gate_result(case, records, events):
result = new_result(case)
good = bool(records) and all(
r.get("exit_code") == 0 and r.get("cleanup") is True for r in records
)
identifier = case["case_id"]
assertions = {}
if identifier in OBSERVATIONS:
for name in case["expected_observation"]:
observed = [e for e in events if e.get("assertion") == name]
assertions[name] = len(observed) == 2 and [
e.get("status") for e in observed
] == ["BEGIN", "PASS"]
good &= (
bool(assertions)
and all(assertions.values())
and all(
e.get("schema_version") == 1 and e.get("status") in {"BEGIN", "PASS"}
for e in events
)
)
else:
assertions = dict.fromkeys(case["expected_observation"], good)
result.update(
status="PASS" if good else "FAIL",
assertions=assertions,
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=all(r.get("cleanup") is True for r in records),
)
return result
def native_results(cases, report, directory):
records = report.get("cases", [])
expected = {case["case_id"] for case in cases}
if len(records) != len(expected) or {r.get("case_id") for r in records} != expected:
raise ValueError("missing, duplicate or unknown VLESS native case")
by_id = {record["case_id"]: record for record in records}
output = []
for case in cases:
record = by_id[case["case_id"]]
path = directory / (case["case_id"] + "-events.jsonl")
observed = read_events(path) if path.exists() else []
test = NATIVE[case["case_id"]][3]
observed_ok = (
events_pass(observed, test, NATIVE[case["case_id"]][1])
if case["case_id"] in PUBLIC
else observed
== [
dict(
schema_version=1, suite="VLESS-WIRE", assertion=test, status=status
)
for status in ("BEGIN", "PASS")
]
)
clean = (
record.get("command_cleanup") is True
and record.get("cleanup") is True
and report.get("cleanup") is True
)
expected_command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_public" if case["case_id"] in PUBLIC else "vless_native",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
good = (
observed_ok
and clean
and report.get("source_unchanged") is True
and record.get("exit_code") == 0
and record.get("status") == "PASS"
and record.get("command") == expected_command
and record.get("peer_kind") == case["peer_kind"]
)
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions={test: observed_ok},
evidence=case["required_evidence"] if good else [],
command_exit_code=record.get("exit_code"),
cleanup=clean,
)
output.append(result)
return output
def fields_report(cases, results):
statuses = {result["case_id"]: result["status"] for result in results}
complete = set(statuses) == REQUIRED_IDS and all(
value == "PASS" for value in statuses.values()
)
fields = []
for row in sorted(FIELD_IDS):
owners = [c["case_id"] for c in cases if row in c["row_ids"]]
fields.append(
dict(
row_id=row,
required_cases=owners,
status="PASS" if complete and len(owners) > 1 else "NOT RUN",
)
)
return dict(schema_version=1, stage="VLESS", protocol="vless", fields=fields)
def execute(cases, run, output, results):
from .protocol_harness import _command, _features, _scripts
for case in cases:
identifier = case["case_id"]
if identifier in NATIVE:
continue
if identifier == "VLESS-FEATURES":
result = _features(case, run, output)
independent = _command(
run,
output,
"feature-outbound-vless",
[
"cargo",
"check",
"--locked",
"--no-default-features",
"--lib",
"--features",
"outbound-vless",
],
1200,
)
if independent.returncode or not independent.cleanup:
result.update(
status="FAIL",
command_exit_code=independent.returncode,
cleanup=independent.cleanup,
)
build = _command(
run,
output,
"production-build",
[
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
],
1200,
)
if build.returncode or not build.cleanup:
result.update(
status="FAIL",
command_exit_code=build.returncode,
cleanup=build.cleanup,
)
results[identifier] = result
elif identifier == "VLESS-SCRIPTS":
results[identifier] = _scripts(case, run, output)
else:
events = []
start = len(run["commands"])
for index, argv in enumerate(commands(identifier)):
path = output / f"{identifier}-{index}-events.jsonl"
_command(run, output, f"{identifier}-{index}", argv, 1200, events=path)
if path.exists():
events.extend(read_events(path))
results[identifier] = gate_result(case, run["commands"][start:], events)
native = [case for case in cases if case["case_id"] in NATIVE]
if native:
directory = output / "native"
native_run(directory, [case["case_id"] for case in native])
report = read_json(directory / "vless-results.json")
(output / "peers.json").write_text(json.dumps(report["peers"], indent=2) + "\n")
for result in native_results(native, report, directory):
results[result["case_id"]] = result
(output / "fields.json").write_text(
json.dumps(fields_report(cases, list(results.values())), indent=2) + "\n"
)
def required_command_names():
# The shared feature smoke includes VMess; VLESS separately adds VLESS.
return {
f"{identifier}-{i}"
for identifier in GATES
for i in range(len(commands(identifier)))
} | {
"production-build",
"offline-scripts",
"feature-default",
"feature-minimal",
"feature-outbound-vless",
*(
"feature-" + name
for name in (
"outbound-trojan",
"outbound-vmess",
"outbound-hysteria2",
"stream-transport",
"quic-transport",
)
),
}
def check(run_dir, cases, run, results, peers, paths):
from .protocol_harness import SCRIPT_OBSERVATIONS
needed = {
"native/vless-results.json",
"fields.json",
"script-tests.json",
"resources.jsonl",
"peers.json",
"cases.json",
"summary.md",
}
if not needed <= set(paths):
raise ValueError("missing VLESS acceptance artifacts")
expected = {c["case_id"]: c for c in definitions()}
if len(cases) != len(expected) or any(
c != expected.get(c["case_id"]) for c in cases
):
raise ValueError("VLESS frozen case metadata changed")
report = read_json(run_dir / "native/vless-results.json")
source_keys = (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
if (
report.get("source") != {key: run.get(key) for key in source_keys}
or report.get("status") != "PASS"
):
raise ValueError("VLESS peer evidence input mismatch")
isolation = report.get("isolation", {})
if (
isolation.get("host_servers") is not False
or isolation.get("network_mode") != "hostOnly"
or not isolation.get("peers")
or not all(
p.get("joined") is True and p.get("started") is True
for p in isolation["peers"]
)
):
raise ValueError("VLESS peers not isolated or cleaned")
if peers != report.get("peers") or set(peers) != {"M", "V2"}:
raise ValueError("missing VLESS official peers")
for kind, peer in peers.items():
project = "MetaCubeX/mihomo" if kind == "M" else "v2fly/v2ray-core"
if (
peer.get("kind") != kind
or peer.get("target") != "linux-arm64"
or not peer.get("version")
or not peer.get("source_url", "").startswith(
f"https://github.com/{project}/releases/"
)
or not all(
HEX.fullmatch(peer.get(key, ""))
for key in ("archive_sha256", "binary_sha256")
)
):
raise ValueError("invalid official VLESS peer identity")
actual = {r["case_id"]: r for r in results}
recalculated = native_results(
[c for c in cases if c["case_id"] in NATIVE], report, run_dir / "native"
)
for case in cases:
identifier = case["case_id"]
argv = commands(identifier)
if not argv:
continue
records = []
events = []
for i, command in enumerate(argv):
matching = [r for r in run["commands"] if r["name"] == f"{identifier}-{i}"]
if len(matching) != 1 or matching[0]["command"] != [
redact(part) for part in command
]:
raise ValueError("missing VLESS gate command")
records.extend(matching)
path = run_dir / f"{identifier}-{i}-events.jsonl"
if path.exists():
events.extend(read_events(path))
recalculated.append(gate_result(case, records, events))
for item in recalculated:
if item != actual.get(item["case_id"]):
raise ValueError("VLESS results disagree with structured observations")
names = [c["name"] for c in run["commands"]]
needed_commands = required_command_names()
if (
len(names) != len(set(names))
or set(names) != needed_commands
or any(
c.get("exit_code") != 0
or c.get("cleanup") is not True
or c.get("log") not in paths
for c in run["commands"]
)
):
raise ValueError("missing or failed VLESS validation command")
for record in run["commands"]:
name = record["name"]
if name == "feature-default":
expected_command = [
"cargo",
"test",
"--locked",
"--test",
"feature_foundations",
]
elif name.startswith("feature-"):
expected_command = [
"cargo",
"check",
"--locked",
"--no-default-features",
"--lib",
]
if name != "feature-minimal":
expected_command += ["--features", name.removeprefix("feature-")]
elif name == "production-build":
expected_command = [
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
]
elif name == "offline-scripts":
expected_command = [
sys.executable,
"-m",
"vcore_scripts.protocol_script_tests",
str(run_dir / "script-tests.json"),
]
else:
continue
if record["command"] != [redact(part) for part in expected_command]:
raise ValueError("VLESS feature or script command changed")
script = read_json(run_dir / "script-tests.json")
tests = {t["test"]: t["status"] for t in script["cases"]}
if (
len(tests) != script["tests_run"]
or any(tests.get(name) != "PASS" for name in SCRIPT_OBSERVATIONS.values())
or any(status != "PASS" for status in tests.values())
):
raise ValueError("missing harness failure-path tests")
fields = fields_report(cases, results)
if read_json(run_dir / "fields.json") != fields or any(
f["status"] != "PASS" for f in fields["fields"]
):
raise ValueError("incomplete VLESS field coverage")
resources = read_events(run_dir / "resources.jsonl")
owned = [
e
for e in resources
if e.get("suite") == "VLESS-OWNED" and e.get("status") == "PASS"
]
if len(owned) != 80 or not all(idle_resources(e.get("resources")) for e in owned):
raise ValueError("missing 20-round owned resource evidence per selected mode")
print(
f"VLESS: PASS ({len(cases)} required cases; {len(FIELD_IDS)} VLESS field rows)"
)
@@ -0,0 +1,759 @@
"""VLESS wire cases against official latest peers and entirely isolated origins."""
from __future__ import annotations
import contextlib
import json
import os
import shutil
import ssl
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_release import download_mihomo
from .native_release import PeerArtifact, download_native
from .protocol_containers import ContainerLab, command
from .protocol_evidence import read_events
from .protocol_fixtures import certificates
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .protocol_vless_peers import configuration
from .protocol_vless_public import PUBLIC as PUBLIC_CASES
from .protocol_vless_public import WIRE as CASES
from .protocol_vless_public import events_pass
from .protocol_vless_security import client_identities
from .protocol_vmess import peer_config as legacy_peer
from .protocol_vmess_public import node_config as legacy_node
ALL_CASES = CASES | PUBLIC_CASES
CLOSE_TESTS = {
"native_mihomo_close_alignment",
"native_ws_reality_close_boundary",
"native_vision_direct_close_alignment",
}
CLIENT_FINGERPRINTS = (
"none",
"chrome",
"chrome120",
"firefox",
"firefox120",
"safari",
"safari16",
)
def close_reference(mode, node, config, certificate, private_key, pin):
reference = dict(node, name="reference")
if reference.get("network") == "ws":
options = dict(reference.get("ws-opts", {}))
if options.get("max-early-data", 0) > 0:
# VCore defaults ED to this header; Mihomo's raw option defaults
# to path ED. Explicit empty/custom values must stay unchanged.
options.setdefault("early-data-header-name", "Sec-WebSocket-Protocol")
reference["ws-opts"] = options
scope = "same-mode"
if mode.endswith("-reality") and reference.get("network") == "ws":
# Mihomo's WS client never consumes RealityOpts. Its REALITY listener
# does support WS, so preserve VCore's real peer and compare the WS
# close layer with a separate TLS entrance. Never call this same-mode.
baseline = json.loads(json.dumps(config["listeners"][0]))
baseline.pop("reality-config")
baseline.update(
name="vless-ws-close-baseline",
port=23003,
certificate=str(certificate),
**{"private-key": str(private_key)},
)
config["listeners"].append(baseline)
reference.pop("reality-opts")
reference.update(port=23003, fingerprint=pin)
scope = "ws-standard-tls-baseline"
elif mode.endswith("-reality") and reference.get("client-fingerprint") in (
None,
"",
"none",
):
reference["client-fingerprint"] = "chrome"
elif (
reference.get("jls-opts")
and reference.get("network") == "grpc"
and reference.get("client-fingerprint") in (None, "", "none")
):
# Official Mihomo v1.19.31's gRPC ALPN reader does not recognize the
# ordinary jls-tls ConnectionState type. uTLS JLS does work. Keep the
# VCore node unchanged and label this different-profile close baseline.
reference["client-fingerprint"] = "chrome"
scope = "jls-grpc-chrome-baseline"
elif (
reference.get("network") == "grpc"
and reference.get("ech-opts", {}).get("enable") is True
and reference.get("client-fingerprint") in ("safari", "safari16")
):
# Official Mihomo v1.19.31/uTLS rejects Safari ECH before business
# data with "malformed outer client hello". Compare the same ECH
# listener's transport close using Chrome; the VCore node stays Safari.
reference["client-fingerprint"] = "chrome"
scope = "ech-safari-chrome-baseline"
return reference, scope
def run(
output: Path,
selected=None,
*,
preflight_only=False,
client_fingerprint=None,
encryption=None,
jls=False,
ech=False,
supplied=None,
):
if client_fingerprint is not None and client_fingerprint not in CLIENT_FINGERPRINTS:
raise ValueError("unsupported named client profile")
public_cases = PUBLIC_CASES | (
{
"FINGERPRINT-ANYTLS": ("M", "anytls", True, "public_legacy_regression"),
**{
f"FINGERPRINT-VMESS-{network.upper()}-TLS": (
"V2",
f"vmess-{network}",
True,
"public_legacy_tcp",
)
for network in ("http", "h2")
},
}
if client_fingerprint
else {}
)
all_cases = CASES | public_cases
if encryption is not None:
from .protocol_encryption import cases as encryption_profiles
from .protocol_encryption_public import catalog
if encryption not in encryption_profiles():
raise ValueError("unsupported Encryption fixture")
all_cases = catalog()
public_cases = {
key: value
for key, value in all_cases.items()
if not value[3].startswith("native_")
}
if jls:
from .protocol_jls import catalog
all_cases = catalog()
public_cases = {
key: value
for key, value in all_cases.items()
if not value[3].startswith("native_")
}
if ech:
if jls:
raise ValueError("ECH and JLS are mutually exclusive")
from .protocol_ech import catalog
all_cases = catalog()
public_cases = {
key: value
for key, value in all_cases.items()
if not value[3].startswith("native_")
}
selected = list(all_cases) if selected is None else selected
if (
not selected
or len(set(selected)) != len(selected)
or not set(selected) <= all_cases.keys()
):
raise ValueError("invalid VLESS native selection")
if client_fingerprint and any(not all_cases[case][2] for case in selected):
raise ValueError("named client profiles require TLS cases")
output.mkdir(parents=True, exist_ok=False)
if preflight_only:
# One listener/origin group per official implementation, not a traffic run.
kinds = {all_cases[case][0] for case in selected}
selected = [
next(case for case in selected if all_cases[case][0] == kind)
for kind in sorted(kinds)
]
report = dict(
stage="SECURITY.ech"
if ech
else "SECURITY.jls"
if jls
else "SECURITY.encryption"
if encryption
else "FINGERPRINT-INTEROP"
if client_fingerprint
else "VLESS",
jls=jls,
ech=ech,
encryption_profile=encryption,
client_fingerprint=client_fingerprint,
scope="container-wire-and-public-consumer",
source=source_identity(),
status="NOT RUN",
phase="preflight",
cases=[],
peers={},
isolation={},
)
try:
artifacts = {}
for kind in sorted({all_cases[case][0] for case in selected} | {"M"}):
directory = output / "binaries" / kind
if supplied is not None:
artifacts[kind] = PeerArtifact(*supplied[kind])
elif kind == "M":
identity = {}
binary = download_mihomo(
"linux-arm64", directory=directory, identity=identity
)
artifacts[kind] = PeerArtifact(binary, identity)
else:
artifacts[kind] = download_native(
kind, directory, "linux-arm64", defer_version=True
)
report["peers"][kind] = artifacts[kind].identity
lab = ContainerLab(report["isolation"], mtu=1500)
report["phase"] = "build"
built = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_native",
"--test",
"vless_public",
"--no-run",
],
timeout=180,
)
(output / "build.log").write_text(redact(built.stdout.decode(errors="replace")))
if built.returncode != 0 or not built.cleanup:
raise RuntimeError("container wire build failed")
# Reuse a peer only within one transport; every case gets fresh origin
# controls/ports and sessions. One downloaded release snapshot per run.
groups = {}
for case in selected:
kind, mode, encrypted, test = all_cases[case]
groups.setdefault((kind, mode, encrypted), []).append((case, test))
for (kind, mode, encrypted), cases in groups.items():
report["phase"] = "peer-start"
with tempfile.TemporaryDirectory(
prefix="private-", dir=output
) as temporary:
root = Path(temporary)
with contextlib.ExitStack() as stack:
origin_dir, server_dir = root / "origin", root / "server"
origin_dir.mkdir()
server_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
shutil.copyfile(
Path(__file__).with_name("container_close_client.py"),
origin_dir / "close.py",
)
shutil.copyfile(
Path(__file__).with_name("container_tls_bio.py"),
origin_dir / "tls_bio.py",
)
artifact = artifacts[kind]
shutil.copy2(artifact.binary, server_dir / "peer")
tag = f"{mode}-{'tls' if encrypted else 'plain'}"
origin_cert, origin_key, origin_pin = certificates(origin_dir)
origin = lab.start(
stack,
origin_dir,
tag + "-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
f"VCORE_ORIGIN_CERT=/data/fixture/{origin_cert.name}",
f"VCORE_ORIGIN_KEY=/data/fixture/{origin_key.name}",
"python",
"-B",
"/data/fixture/origin.py",
],
)
argv = (
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
]
if kind == "M"
else [
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
]
)
server = lab.start(stack, server_dir, tag + "-server", argv)
def retain_logs(origin=origin, server=server, tag=tag):
for role, peer in (("origin", origin), ("server", server)):
if peer.log.exists():
(output / f"{tag}-{role}.log").write_text(
redact(
# Capture before cleanup: live log
# following may buffer short messages.
command("logs", peer.name)[-65536:]
)
)
stack.callback(retain_logs)
version = command(
"exec",
server.name,
"/data/fixture/peer",
"-v" if kind == "M" else "version",
).strip()
digest = command(
"exec", server.name, "sha256sum", "/data/fixture/peer"
).split()[0]
if not version or digest != artifact.identity["binary_sha256"]:
raise RuntimeError("container peer identity mismatch")
artifact.identity["version"] = version
# Non-leaf pin still exercises actual certificate name checks.
from .protocol_fixtures import certificate_chain
cert, key, pin = (
certificate_chain(server_dir)
if encrypted
else certificates(server_dir)
)
identities = {}
if mode == "anytls":
node = dict(
name="peer",
type="anytls",
server=server.ipv4,
port=23000,
password="fixture",
sni="localhost",
udp=True,
fingerprint=pin,
)
config = {
"listeners": [
{
"name": "anytls",
"type": "anytls",
"listen": "::",
"port": 23000,
"users": {"fixture": "fixture"},
"certificate": f"/data/fixture/{cert.name}",
"private-key": f"/data/fixture/{key.name}",
}
],
"ipv6": True,
"log-level": "silent",
"hosts": {"vcore-fixture.test": origin.ipv4},
"rules": ["MATCH,DIRECT"],
}
elif mode.startswith(("vmess-", "trojan-")):
legacy_mode = mode.removeprefix("vmess-")
node = legacy_node(legacy_mode, True, server.ipv4, pin)
config = legacy_peer(
kind,
legacy_mode,
True,
23000,
Path("/data/fixture") / cert.name,
Path("/data/fixture") / key.name,
)
if kind == "M":
config["hosts"] = {"vcore-fixture.test": origin.ipv4}
config["listeners"][0]["listen"] = "::"
else:
config["dns"]["hosts"] = {"vcore-fixture.test": origin.ipv4}
config["inbounds"][0]["listen"] = "::"
else:
node, config = configuration(
mode.removesuffix("-h1") if jls or ech else mode,
server.ipv4,
origin.ipv4,
Path("/data/fixture") / cert.name,
Path("/data/fixture") / key.name,
)
node["name"] = "peer"
if encrypted and not mode.endswith("-reality"):
node["fingerprint"] = pin
if mode.endswith("-mtls"):
identities = client_identities(server_dir)
node.update(identities["valid"])
config["listeners"][0].update(
{
"client-auth-type": "require-and-verify",
"client-auth-cert": "/data/fixture/root.pem",
}
)
if client_fingerprint:
node["client-fingerprint"] = client_fingerprint
extra_fixture = {}
if ech:
from .protocol_ech import configuration as ech_config
extra_fixture = ech_config(node, config, server_dir, mode)
if jls:
from .protocol_jls import configuration as jls_config
extra_fixture = jls_config(node, config, origin.ipv4, mode)
if encryption:
from .protocol_encryption_public import (
configuration as encrypted_config,
)
encrypted_config(encryption, node, config)
if mode == "vision-encryption":
node.pop("tls", None)
node.pop("servername", None)
for listener in config["listeners"]:
listener.pop("certificate", None)
listener.pop("private-key", None)
listener["allow-insecure"] = True
reference_node = None
reference_scope = None
if any(test in CLOSE_TESTS for _, test in cases):
reference_node, reference_scope = close_reference(
mode,
node,
config,
Path("/data/fixture") / cert.name,
Path("/data/fixture") / key.name,
pin,
)
(server_dir / "config.json").write_text(json.dumps(config))
upstream = None
hop = None
if any(case in public_cases for case, _ in cases):
upstream_dir = root / "upstream"
upstream_dir.mkdir()
shutil.copy2(artifacts["M"].binary, upstream_dir / "peer")
upstream = lab.start(
stack,
upstream_dir,
tag + "-upstream",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
(upstream_dir / "config.json").write_text(
json.dumps(
{
"socks-port": 23001,
"allow-lan": True,
"bind-address": "*",
"log-level": "silent",
"ipv6": True,
"hosts": {"peer.fixture.test": server.ipv4},
"rules": ["MATCH,DIRECT"],
}
)
)
hop = dict(
name="hop",
type="socks5",
server=upstream.ipv4,
port=23001,
udp=True,
)
upstream.release()
upstream.wait_tcp(23001)
fixture = root / "input.json"
fixture.write_text(
json.dumps(
dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
server_ipv6=server.ipv6,
data_dir=str(root / "data"),
node=node,
vision_probe=mode.startswith("vision-"),
flow_reject_port=23004
if mode.startswith("vision-")
else None,
origin_pin=origin_pin,
origin_root_der=list(
ssl.PEM_cert_to_DER_cert(origin_cert.read_text())
),
identities=identities,
root_der=list(
ssl.PEM_cert_to_DER_cert(
(
server_dir / "root.pem"
if encrypted
else cert
).read_text()
)
),
hop=hop,
peer_kind=kind,
**extra_fixture,
)
)
)
origin.release()
server.release()
origin.wait_tcp(24000)
if mode.endswith("-reality") or jls:
origin.wait_tcp(24001)
server.wait_tcp(23000)
comparison = None
if reference_node is not None:
comparison_dir = root / "comparison"
comparison_dir.mkdir()
shutil.copy2(artifacts["M"].binary, comparison_dir / "peer")
(comparison_dir / "config.json").write_text(
json.dumps(
{
"socks-port": 23002,
"allow-lan": True,
"bind-address": "*",
"log-level": "warning",
"ipv6": True,
"proxies": [reference_node],
"rules": ["MATCH,reference"],
}
)
)
comparison = lab.start(
stack,
comparison_dir,
tag + "-comparison",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
comparison.release()
comparison.wait_tcp(23002)
stack.callback(
lambda peer=comparison, tag=tag: (
output / f"{tag}-comparison.log"
).write_text(redact(command("logs", peer.name)[-65536:]))
)
observed_close = run_command(
[
"container",
"exec",
origin.name,
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/close.py",
comparison.ipv4,
origin.ipv4,
],
timeout=30,
)
(output / f"{tag}-close-command.log").write_text(
redact(observed_close.stdout.decode(errors="replace"))
)
if observed_close.returncode or not observed_close.cleanup:
raise RuntimeError("official close comparison failed")
reference = json.loads(observed_close.stdout)
reference["scope"] = reference_scope
reference["client_fingerprint"] = reference_node.get(
"client-fingerprint"
)
reference["dut_client_fingerprint"] = node.get(
"client-fingerprint"
)
(output / f"{tag}-close-reference.json").write_text(
json.dumps(reference) + "\n"
)
data = json.loads(fixture.read_text())
data["close_reference"] = reference
if any(
test == "native_vision_direct_close_alignment"
for _, test in cases
):
direct = run_command(
[
"container",
"exec",
origin.name,
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/close.py",
comparison.ipv4,
origin.ipv4,
"vision-direct",
],
timeout=30,
)
(output / f"{tag}-direct-close-command.log").write_text(
redact(direct.stdout.decode(errors="replace"))
)
if direct.returncode or not direct.cleanup:
raise RuntimeError(
"official direct close comparison failed"
)
direct_reference = json.loads(direct.stdout)
direct_reference["scope"] = reference_scope
(output / f"{tag}-direct-close-reference.json").write_text(
json.dumps(direct_reference) + "\n"
)
data["direct_close_reference"] = direct_reference
fixture.write_text(json.dumps(data))
report["phase"] = "execute"
for case, test in cases:
if preflight_only:
continue
print(case, flush=True)
record = dict(case_id=case, status="NOT RUN", peer_kind=kind)
report["cases"].append(record)
events = output / f"{case}-events.jsonl"
test_command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_public" if case in public_cases else "vless_native",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
result = run_command(
test_command,
cwd=CORE_DIR,
timeout=240,
limit=4 * 1024 * 1024,
env=dict(
os.environ,
VCORE_VLESS_PEER=f"{server.ipv4}:23000",
VCORE_VLESS_TRANSPORT=json.dumps(
dict(
mode=mode,
tls=encrypted,
pin=pin,
peer_kind=kind,
udp_path_limit=15000,
)
),
VCORE_VLESS_ORIGIN_V4=origin.ipv4,
VCORE_VLESS_INPUT=str(fixture),
VCORE_CASE_EVENTS=str(events),
VCORE_PROTOCOL_STAGE="SECURITY"
if encryption or jls or ech
else "VLESS",
),
)
(output / f"{case}.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
observed = read_events(events) if events.exists() else []
record.update(
command=test_command,
exit_code=result.returncode,
seconds=result.seconds,
command_cleanup=result.cleanup,
status="PASS"
if result.returncode == 0
and result.cleanup
and (
events_pass(
observed,
test,
mode,
stage="SECURITY"
if encryption or jls or ech
else "VLESS",
)
if case in public_cases
else observed
== [
dict(
schema_version=1,
suite="SECURITY-WIRE"
if encryption or jls or ech
else "VLESS-WIRE",
assertion=test,
status=status,
)
for status in ("BEGIN", "PASS")
]
)
else "FAIL",
)
server.ensure_alive()
origin.ensure_alive()
print(f"{case}: {record['status']}", flush=True)
for case, _ in cases:
if preflight_only:
continue
next(item for item in report["cases"] if item["case_id"] == case)[
"cleanup"
] = all(
peer.record["joined"]
for peer in [server, origin] + ([upstream] if upstream else [])
)
report["status"] = (
"READY"
if preflight_only
else "PASS"
if (preflight_only or len(report["cases"]) == len(selected))
and all(case["status"] == "PASS" for case in report["cases"])
else "FAIL"
)
except KeyboardInterrupt:
report.update(status="INTERRUPTED", reason="user interruption")
except (OSError, RuntimeError, ValueError, KeyError) as error:
report.update(
status="BLOCKED" if report["phase"] == "preflight" else "FAIL",
reason=str(error)
if isinstance(error, RuntimeError)
else type(error).__name__,
)
finally:
report["cleanup"] = all(
peer["joined"] for peer in report["isolation"].get("peers", [])
)
report["source_unchanged"] = (
source_identity()["source_tree_sha256"]
== report["source"]["source_tree_sha256"]
)
if report["status"] in {"PASS", "READY"} and not (
report["cleanup"] and report["source_unchanged"]
):
report["status"] = "FAIL"
(output / "vless-results.json").write_text(json.dumps(report, indent=2) + "\n")
return (
0
if report["status"] in {"PASS", "READY"}
else (130 if report["status"] == "INTERRUPTED" else 1)
)
if __name__ == "__main__":
import sys
from .mihomo_isolation import exclusive_run
with exclusive_run():
sys.exit(run(Path(sys.argv[1]).resolve(), sys.argv[2:] or None))
@@ -0,0 +1,219 @@
"""VLESS isolated native fixtures. No host service or reference checkout is used."""
from pathlib import Path
MODES = [
mode + suffix
for mode in (
"tcp",
"ws",
"ws-ed-1",
"ws-ed-2048",
"upgrade",
"upgrade-fast",
"grpc",
"http",
"h2",
"ws-header",
"ws-path",
)
for suffix in ("", "-tls")
] + [
"xhttp-stream-one",
"xhttp-stream-up",
"xhttp-packet-up",
"vision-tls",
"vision-reality",
"tcp-reality",
"ws-reality",
"grpc-reality",
"upgrade-reality",
"upgrade-fast-reality",
"tcp-mtls",
"ws-mtls",
"grpc-mtls",
"xhttp-stream-one-reality",
"xhttp-stream-up-reality",
"xhttp-packet-up-reality",
"xhttp-stream-up-download",
"xhttp-packet-up-download",
"xhttp-stream-up-download-reality",
"xhttp-packet-up-download-reality",
"grpc-path-tls",
]
def peer_kind(mode):
base = mode.removesuffix("-tls").removesuffix("-reality").removesuffix("-mtls")
return "V2" if base in {"http", "h2", "ws-header", "ws-path"} else "M"
def configuration(mode, server, origin, cert: Path, key: Path):
encrypted = mode.endswith(("-tls", "-reality", "-mtls")) or mode.startswith(
("xhttp-", "vision-")
)
base = mode.removesuffix("-tls").removesuffix("-reality").removesuffix("-mtls")
separate_download = base.endswith("-download")
base = base.removesuffix("-download")
kind = peer_kind(mode)
identity = "07070707-0707-0707-0707-070707070707"
node = dict(
name="edge", type="vless", server=server, port=23000, uuid=identity, udp=True
)
if encrypted:
node.update(tls=True, servername="localhost")
if base.startswith("xhttp-"):
node.update(
network="xhttp",
**{
"xhttp-opts": dict(
path="/vless", host="localhost", mode=base.removeprefix("xhttp-")
)
},
)
if separate_download:
node["xhttp-opts"]["download-settings"] = {}
elif base.startswith(("ws", "upgrade")):
ws = dict(path="/vless-ws", headers={"Host": "localhost"})
if base in {"ws-header", "ws-path"}:
ws.update(
path="/vless-ws/",
**{
"max-early-data": 2048,
"early-data-header-name": "X-Vcore-Ed"
if base == "ws-header"
else "",
},
)
elif base in {"ws-ed-1", "ws-ed-2048"}:
ws["max-early-data"] = int(base.removeprefix("ws-ed-"))
elif base.startswith("upgrade"):
ws.update(
**{
"v2ray-http-upgrade": True,
"v2ray-http-upgrade-fast-open": base == "upgrade-fast",
"max-early-data": 1,
}
)
node.update(network="ws", **{"ws-opts": ws})
elif base in {"grpc", "grpc-path"}:
node.update(
network="grpc",
**{
"grpc-opts": dict(
**{
"grpc-service-name": "/vless-grpc/Tun"
if base == "grpc-path"
else "vless-grpc",
"grpc-user-agent": "VCore-VLESS",
"ping-interval": 1,
}
)
},
)
elif base == "http":
node.update(
network="http",
**{
"http-opts": dict(
method="POST", path=["/vless-http"], headers={"Host": ["localhost"]}
)
},
)
elif base == "h2":
node.update(
network="h2", **{"h2-opts": dict(host=["localhost"], path="/vless-h2")}
)
if kind == "M":
listener = dict(
name="vless",
type="vless",
listen="::",
port=23000,
users=[dict(username="fixture", uuid=identity)],
)
if mode.startswith("vision-"):
listener["users"][0]["flow"] = "xtls-rprx-vision"
node["flow"] = "xtls-rprx-vision"
if encrypted:
listener.update(certificate=str(cert), **{"private-key": str(key)})
else:
listener["allow-insecure"] = True
if mode.endswith("-reality"):
from .mihomo_extended import PRIVATE_KEY, PUBLIC_KEY, SHORT_ID
listener.pop("certificate", None)
listener.pop("private-key", None)
listener["reality-config"] = {
"dest": f"{origin}:24001",
"private-key": PRIVATE_KEY,
"short-id": [SHORT_ID],
"server-names": ["localhost"],
}
node["reality-opts"] = {"public-key": PUBLIC_KEY, "short-id": SHORT_ID}
if node.get("network") == "ws":
listener["ws-path"] = "/vless-ws"
if base in {"grpc", "grpc-path"}:
listener["grpc-service-name"] = "vless-grpc"
if base == "ws-alpn":
node["alpn"] = ["h2", "http/1.1"]
listener["grpc-service-name"] = "vless-alpn"
if base.startswith("xhttp-"):
listener["xhttp-config"] = dict(path="/vless", mode="auto")
listeners = [listener]
if mode.startswith("vision-"):
# Mihomo permits an explicit empty flow even for a Vision user.
# Rejection requires a Vision request against an ordinary user.
listeners.append(
dict(
listener,
name="vless-flow-rejection",
port=23004,
users=[dict(user, flow="") for user in listener["users"]],
)
)
return node, dict(
ipv6=True,
mode="rule",
**{"log-level": "silent"},
hosts={"vcore-fixture.test": origin},
listeners=listeners,
rules=["MATCH,DIRECT"],
)
stream = dict(network="tcp", security="tls" if encrypted else "none")
if base == "http":
stream["tcpSettings"] = dict(
header=dict(type="http", request=dict(path=["/vless-http"]))
)
elif base == "h2":
stream.update(
network="http", httpSettings=dict(host=["localhost"], path="/vless-h2")
)
else:
stream.update(
network="ws",
wsSettings=dict(
path="/vless-ws/",
maxEarlyData=2048,
earlyDataHeaderName="X-Vcore-Ed" if base == "ws-header" else "",
),
)
if encrypted:
stream["tlsSettings"] = dict(
alpn=["h2"] if base == "h2" else ["http/1.1"],
certificates=[dict(certificateFile=str(cert), keyFile=str(key))],
)
return node, dict(
log={"loglevel": "none"},
dns={"hosts": {"vcore-fixture.test": origin}},
inbounds=[
dict(
listen="::",
port=23000,
protocol="vless",
settings=dict(clients=[dict(id=identity)], decryption="none"),
streamSettings=stream,
)
],
outbounds=[dict(protocol="freedom", settings={"domainStrategy": "UseIP"})],
)
@@ -0,0 +1,154 @@
"""Frozen VLESS native/public consumer requirements; no result-derived catalog."""
from .protocol_evidence import idle_resources
from .protocol_vless_peers import MODES, peer_kind
from .protocol_vmess_public import pairs
PUBLIC = {}
WIRE = {}
def add(mode, suffix, test, *, wire=False):
kind = peer_kind(mode)
encrypted = mode.endswith(("-tls", "-reality", "-mtls")) or mode.startswith(
("xhttp-", "vision-")
)
(WIRE if wire else PUBLIC)[f"VLESS-{mode.upper()}-{suffix}"] = (
kind,
mode,
encrypted,
test,
)
for mode in MODES:
add(mode, "BASE", "public_base")
add("ws-alpn-tls", "NEG", "public_alpn_rejection")
for mode in ("tcp", "ws", "grpc", "http", "h2"):
for suffix in ("", "-tls"):
add(mode + suffix, "NEG", "public_negative")
for suffix, test in (
("GRAPH", "runtime::public_graph"),
("IPV6", "runtime::public_ipv6_and_gates"),
("ENTRYPOINTS", "runtime::public_entrypoints"),
):
add(mode + "-tls", suffix, test)
for mode in (
"upgrade-tls",
"upgrade-fast-tls",
"upgrade-reality",
"upgrade-fast-reality",
"tcp-reality",
"ws-reality",
"grpc-reality",
"vision-tls",
"vision-reality",
"xhttp-stream-one",
"xhttp-stream-up",
"xhttp-packet-up",
):
add(mode, "NEG", "public_negative")
for mode in ("tcp-tls", "grpc-tls", "vision-tls", "vision-reality"):
add(mode, "LIFE", "runtime::public_lifecycle")
add(mode, "OWNED", "runtime::owned_resources")
for mode in ("tcp-tls", "grpc-tls", "vision-tls"):
add(mode, "UDP-ISOLATION", "runtime::public_udp_isolation")
for mode in ("vision-tls", "vision-reality"):
add(mode, "INNER-TLS", "native_vision_inner_tls", wire=True)
add(mode, "GRAPH", "runtime::public_graph")
add(mode, "ENTRYPOINTS", "runtime::public_entrypoints")
add(mode, "IPV6", "runtime::public_ipv6_and_gates")
for mode in ("tcp", "grpc-tls", "http", "h2-tls"):
add(mode, "UDP-BOUNDARIES", "native_three_udp_encodings", wire=True)
for mode in ("tcp-mtls", "ws-mtls", "grpc-mtls"):
add(mode, "IDENTITY", "public_tls_identity_and_verification_name")
add(
"grpc-tls",
"POOL-SELECTION",
"runtime::grpc_pool_keeps_physical_selection_until_new_transport",
)
for protocol in ("vmess", "trojan"):
for network in ("tcp", "ws", "grpc"):
PUBLIC[f"VLESS-REGRESSION-{protocol.upper()}-{network.upper()}"] = (
"M",
f"{protocol}-{network}",
True,
"public_legacy_regression",
)
for mode in (
"tcp",
"tcp-tls",
"tcp-reality",
"ws-reality",
"grpc-reality",
"ws",
"ws-tls",
"grpc",
"grpc-tls",
"http",
"http-tls",
"h2",
"h2-tls",
"upgrade-fast-tls",
"upgrade-reality",
"upgrade-fast-reality",
"vision-tls",
"vision-reality",
"xhttp-stream-one",
"xhttp-stream-up",
"xhttp-packet-up",
):
add(
mode,
"CLOSE",
"native_ws_reality_close_boundary"
if mode in {"ws-reality", "upgrade-reality", "upgrade-fast-reality"}
else "native_mihomo_close_alignment",
wire=True,
)
add("grpc-tls", "POOL-COUNTS", "native_grpc_pool_thresholds", wire=True)
CASES = WIRE | PUBLIC
def events_pass(events, test, mode, *, stage="VLESS"):
main = [event for event in events if event.get("suite") == f"{stage}-PUBLIC"]
if len(main) != 2 or not pairs(main, f"{stage}-PUBLIC", test, 1):
return False
if any(
event.get("schema_version") != 1 or event.get("status") not in {"BEGIN", "PASS"}
for event in events
):
return False
if test == "public_base" and not (
pairs(events, f"{stage}-BASE", "tcp_10mib_both_directions", 3)
and pairs(
events,
f"{stage}-BASE",
"udp_each_codec_and_family",
1 if mode.startswith("vision-") else 3,
)
):
return False
if test in {"runtime::public_lifecycle", "runtime::owned_resources"}:
suite = (
f"{stage}-LIFE" if test.endswith("public_lifecycle") else f"{stage}-OWNED"
)
if not pairs(events, suite, "stop_and_remain_quiet", 20):
return False
for event in [
e for e in events if e.get("suite") == suite and e["status"] == "PASS"
]:
if event.get("seconds", 0) < 5:
return False
if suite == f"{stage}-OWNED":
points = event.get("checkpoints", [])
phases = {p["phase"]: p["resources"] for p in points}
if (
len(points) != 3
or set(phases) != {"baseline", "after-stop", "quiet"}
or not all(idle_resources(r) for r in phases.values())
or phases["after-stop"] != phases["quiet"]
or not idle_resources(event.get("resources"))
):
return False
return True
@@ -0,0 +1,94 @@
"""Ephemeral public-test identities. Keys never leave the private fixture."""
from pathlib import Path
from .protocol_fixtures import certificates
from .protocol_peers import run_command
def client_identities(directory: Path):
result = {}
extensions = directory / "client-ext.cnf"
extensions.write_text(
"basicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature\nextendedKeyUsage=clientAuth\n"
)
(directory / "index.txt").write_text("")
(directory / "serial.txt").write_text("10\n")
ca = directory / "client-ca.cnf"
ca.write_text(
"[ca]\ndefault_ca=fixture\n[fixture]\n"
f"database={directory / 'index.txt'}\nserial={directory / 'serial.txt'}\n"
f"new_certs_dir={directory}\ncertificate={directory / 'root.pem'}\n"
f"private_key={directory / 'root-key.pem'}\n"
"default_md=sha256\npolicy=names\n[names]\ncommonName=supplied\n"
)
for name, days in (("valid", "2"), ("expired", "2")):
key, csr, cert = (
directory / f"client-{name}.{suffix}" for suffix in ("key", "csr", "pem")
)
for argv in (
[
"openssl",
"req",
"-new",
"-newkey",
"rsa:2048",
"-nodes",
"-subj",
"/CN=synthetic-client",
"-keyout",
str(key),
"-out",
str(csr),
],
[
"openssl",
"x509",
"-req",
"-in",
str(csr),
"-CA",
str(directory / "root.pem"),
"-CAkey",
str(directory / "root-key.pem"),
"-set_serial",
"3" if name == "valid" else "4",
"-days",
days,
"-extfile",
str(extensions),
"-out",
str(cert),
],
):
if name == "expired" and argv[1] == "x509":
argv = [
"openssl",
"ca",
"-batch",
"-notext",
"-config",
str(ca),
"-in",
str(csr),
"-out",
str(cert),
"-extfile",
str(extensions),
"-startdate",
"20000101000000Z",
"-enddate",
"20010101000000Z",
]
command = run_command(argv, timeout=20, limit=65536)
if command.returncode or not command.cleanup:
raise RuntimeError("synthetic client identity generation failed")
result[name] = {"certificate": cert.read_text(), "private-key": key.read_text()}
wrong = directory / "wrong-ca"
wrong.mkdir()
cert, key, _ = certificates(wrong)
result["wrong-ca"] = {
"certificate": cert.read_text(),
"private-key": key.read_text(),
}
return result
+132
View File
@@ -0,0 +1,132 @@
"""VMESS wire gate; all peers and origins run in isolated containers."""
from __future__ import annotations
import sys
from pathlib import Path
from .mihomo_isolation import exclusive_run
CASES = {
"VMESS-M-IDENTITY": ("M", "tcp", False, "native_identity_time_replay_rejection")
}
CASES["VMESS-M-RAW-UDP"] = ("M", "tcp", False, "native_raw_udp_boundaries")
CASES["VMESS-M-ENCODED-UDP"] = ("M", "tcp", False, "native_encoded_udp_boundaries")
for _mode in ("tcp", "ws", "grpc", "http", "h2"):
for _tls in (False, True):
_kind = "V2" if _mode in {"http", "h2"} else "M"
CASES[f"VMESS-{_kind}-{_mode.upper()}-{'TLS' if _tls else 'PLAIN'}"] = (
_kind,
_mode,
_tls,
"native_cipher_matrix",
)
CASES[f"VMESS-{_kind}-{_mode.upper()}-{'TLS' if _tls else 'PLAIN'}-CLOSE"] = (
_kind,
_mode,
_tls,
"native_mihomo_close_alignment",
)
if _mode != "tcp" or _tls:
for _encoding in ("raw", "encoded"):
CASES[
f"VMESS-{_kind}-{_mode.upper()}-{'TLS' if _tls else 'PLAIN'}"
f"-{_encoding.upper()}-UDP"
] = (
_kind,
_mode,
_tls,
f"native_{_encoding}_udp_boundaries",
)
def udp_path_limit():
# Both packet sender and origin are now Linux VM services.
return 15000
def peer_config(kind, mode, encrypted, port, cert, key):
if mode.startswith("trojan-"):
config = peer_config(
kind, mode.removeprefix("trojan-"), encrypted, port, cert, key
)
config["listeners"][0].update(
type="trojan",
users=[dict(username="fixture", password="synthetic-regression-only")],
)
return config
variant = mode
mode = mode.split("-")[0]
identity = "07070707-0707-0707-0707-070707070707"
if kind == "M":
listener = dict(
name="vmess",
type="vmess",
listen="127.0.0.1",
port=port,
users=[dict(username="fixture", uuid=identity, alterId=0)],
)
if encrypted:
listener.update(certificate=str(cert), **{"private-key": str(key)})
if mode == "ws":
listener["ws-path"] = "/vmess-ws"
if mode == "grpc":
listener["grpc-service-name"] = "vmess-grpc"
if variant == "ws-alpn":
listener["grpc-service-name"] = "vmess-alpn"
return {
"mode": "rule",
"log-level": "silent",
"ipv6": True,
"hosts": {"vcore-fixture.test": "127.0.0.1"},
"listeners": [listener],
"rules": ["MATCH,DIRECT"],
}
stream = dict(network="tcp", security="tls" if encrypted else "none")
if mode == "http":
stream["tcpSettings"] = {
"header": {"type": "http", "request": {"path": ["/vmess-http"]}}
}
elif mode == "h2":
stream.update(
network="http", httpSettings=dict(host=["localhost"], path="/vmess-h2")
)
elif mode == "ws":
stream.update(
network="ws",
wsSettings={
"path": "/vmess-ws/",
"maxEarlyData": 2048,
"earlyDataHeaderName": "X-Vcore-Ed" if variant == "ws-header" else "",
},
)
if encrypted:
stream["tlsSettings"] = {
"alpn": ["h2"] if mode == "h2" else ["http/1.1"],
"certificates": [{"certificateFile": str(cert), "keyFile": str(key)}],
}
return {
"log": {"loglevel": "none"},
"dns": {"hosts": {"vcore-fixture.test": "127.0.0.1"}},
"inbounds": [
{
"listen": "127.0.0.1",
"port": port,
"protocol": "vmess",
"settings": {"clients": [{"id": identity, "alterId": 0}]},
"streamSettings": stream,
}
],
"outbounds": [{"protocol": "freedom", "settings": {"domainStrategy": "UseIP"}}],
}
def run(output: Path, selected=None):
from .protocol_vmess_container import run as run_container
return run_container(output, selected)
if __name__ == "__main__":
with exclusive_run():
sys.exit(run(Path(sys.argv[1]).resolve(), sys.argv[2:] or None))
@@ -0,0 +1,691 @@
"""VMESS container-only execution and independently recomputed stage acceptance."""
from __future__ import annotations
import json
import sys
from .builds import DEFAULT_FEATURES
from .protocol_evidence import HEX, idle_resources, new_result, read_events, read_json
from .protocol_inputs import redact
from .protocol_vmess import CASES as WIRE
from .protocol_vmess_container import run as native_run
from .protocol_vmess_public import CASES as PUBLIC
from .protocol_vmess_public import events_pass
NATIVE = WIRE | PUBLIC
FIELD_IDS = {
*(f"C{i:02}" for i in range(1, 7)),
*(f"T{i:02}" for i in range(2, 6)),
*(f"W{i:02}" for i in range(1, 6)),
"G01",
*(f"H{i:02}" for i in range(1, 7)),
*(f"VM{i:02}" for i in range(1, 9)),
}
GATES = {
"VMESS-CFG",
"VMESS-CODEC",
"VMESS-CANCEL",
"VMESS-REGRESSION",
"VMESS-RELEASE",
"VMESS-FEATURES",
"VMESS-SCRIPTS",
"VMESS-QUALITY",
}
SCRIPT_ASSERTIONS = [
"offline-success",
"missing-duplicate-unknown-case",
"download-failure",
"timeout",
"sigint",
"partial-start",
"permission",
"peer-exit",
"cleanup-failure",
"redaction",
]
REQUIRED_IDS = set(NATIVE) | GATES
# Frozen names, not a regex over test output and not read from the manifest.
OBSERVATIONS = {
"VMESS-CFG": [
"vmess_default_node_and_explicit_cipher_aliases_are_accepted_without_io",
"vmess_transport_and_security_combinations_are_strict",
"vmess_field_boundaries_and_normalized_transport_values",
"vmess_normalized_websocket_fields_preserve_headers_and_early_data_order",
],
"VMESS-CODEC": [
"xudp_cancelled_send_and_bad_frame_release_io_and_cannot_resume",
"xudp_zero_global_id_omits_the_optional_extension_like_mihomo",
"packetaddr_has_ip_only_address_first_wire_and_rejects_truncation",
"vmess_aead_requests_are_fresh_bounded_and_redacted",
"vmess_response_authentication_cannot_be_skipped_by_none_cipher",
"vmess_whole_close_wakes_reader_and_releases_io_before_response",
"xudp_explicit_budget_rejects_max_plus_one_before_writing",
"independently_published_nested_hmac_vectors",
"chunks_preserve_boundaries_masks_padding_and_authenticated_eof",
"tags_lengths_and_nonce_exhaustion_fail_closed",
"auto_uses_detected_hardware_not_architecture_name",
],
"VMESS-CANCEL": [
"vmess_all_handshakes_keep_the_original_deadline_and_join_cancelled_io",
"vmess_expired_deadline_and_protect_failure_never_fall_back",
],
"VMESS-REGRESSION": [
"shared_xudp_receives_frames_without_a_vless_response_header",
"first_frame_matches_xray_mux_wire_format",
"followup_frame_carries_each_datagrams_destination",
"destination_codec_round_trips_all_address_families",
"normal_and_error_end_frames_are_distinguishable",
"oversized_response_is_rejected_before_payload_read",
"fragmented_receive_survives_cancellation_and_resumes",
"keepalive_data_is_consumed_without_becoming_an_udp_response",
"full_wire_payload_remains_available_to_proxy_inbounds",
"request_matches_the_official_sha224_and_socks_address_wire_format",
"authentication_and_requests_are_strict_and_never_debug_credentials",
"invalid_truncated_and_oversized_frames_fail_closed_without_payload_leaks",
"datagram_limits_preserve_messages_and_drain_over_budget_responses",
"datagram_sends_one_frame_and_receives_fragmented_consecutive_frames",
"cancelled_receive_preserves_partial_frames_and_does_not_block_send",
"cancelled_partial_send_poison_closes_without_replaying_or_waiting",
"trojan_tcp_configuration_and_node_graph_accept_the_approved_fields",
"trojan_tcp_defaults_preserve_credentials_and_address_policy",
"trojan_invalid_configuration_is_rejected_without_exposing_credentials",
"trojan_ws_and_grpc_configuration_applies_transport_specific_defaults",
"trojan_transport_boundaries_fail_before_runtime_io",
"transport_options_reject_ambiguous_headers_before_io",
"http_first_header_preserves_prefix_raw_continuation_and_half_close_tail",
"legacy_h2_keeps_unframed_bytes_server_first_and_owned_whole_close",
"websocket_rejects_invalid_upgrade_responses_and_bounded_header_overflows",
"websocket_early_data_and_remaining_frames_keep_the_original_byte_order",
"setup_deadline_releases_supplied_io_in_stream_adapters",
"cancelled_setup_releases_supplied_io_without_a_detached_task",
"ws_rejects_oversized_and_truncated_frames_instead_of_reporting_eof",
"ws_accepts_mihomo_clean_underlay_eof_only_at_complete_message_boundaries",
"dropping_an_established_ws_releases_its_only_io_owner",
"grpc_rejects_oversized_truncated_and_invalid_records",
"grpc_large_writes_obey_small_http2_windows_and_keep_byte_integrity",
"ws_partial_writes_empty_frames_ping_and_half_close_preserve_every_byte",
"grpc_handles_response_after_upload_fragmented_records_and_owned_stop",
"websocket_supplied_io_preserves_server_first_and_partial_writes",
"xhttp_keeps_complete_response_when_peer_resets_after_end_stream",
"xhttp_does_not_hide_reset_before_end_stream",
"feature_skeletons_do_not_open_unimplemented_yaml_or_measurement_protocols",
"future_fields_and_over_limit_documents_remain_rejected",
"registry_matches_live_production_constants_and_has_owned_boundary_cases",
"webpki_rejects_untrusted_wrong_name_and_expired_unless_explicitly_skipped",
"leaf_pin_is_trust_but_nonleaf_pin_checks_chain_name_and_expiry",
"ticket_storage_obeys_exact_node_budget_and_consumes_each_ticket_once",
"every_certificate_policy_verifies_tls12_and_tls13_handshake_signatures",
"vless_keeps_webpki_tls13_and_required_h2_after_anytls_connections",
"alpn_and_tls_resumption_are_isolated_between_node_policies",
"explicit_verification_name_does_not_change_sni_or_allow_skip_to_override_it",
"mutual_tls_identity_is_required_verified_and_not_shared_between_clients",
"tls_options_reject_invalid_alpn_identity_and_budget_before_using_a_stream",
"certificate_rejection_delivers_no_business_bytes_and_diagnostics_are_redacted",
"tls_close_write_sends_notify_without_closing_the_supplied_transport",
"tls_close_notify_flush_has_a_five_second_bound",
"cancelling_tls_handshake_releases_the_caller_supplied_stream",
],
"VMESS-RELEASE": [
"vmess_default_node_and_explicit_cipher_aliases_are_accepted_without_io",
"vmess_transport_and_security_combinations_are_strict",
"vmess_field_boundaries_and_normalized_transport_values",
"vmess_normalized_websocket_fields_preserve_headers_and_early_data_order",
"xudp_cancelled_send_and_bad_frame_release_io_and_cannot_resume",
"xudp_zero_global_id_omits_the_optional_extension_like_mihomo",
"packetaddr_has_ip_only_address_first_wire_and_rejects_truncation",
"vmess_aead_requests_are_fresh_bounded_and_redacted",
"vmess_response_authentication_cannot_be_skipped_by_none_cipher",
"vmess_whole_close_wakes_reader_and_releases_io_before_response",
"xudp_explicit_budget_rejects_max_plus_one_before_writing",
"independently_published_nested_hmac_vectors",
"chunks_preserve_boundaries_masks_padding_and_authenticated_eof",
"tags_lengths_and_nonce_exhaustion_fail_closed",
"auto_uses_detected_hardware_not_architecture_name",
"vmess_all_handshakes_keep_the_original_deadline_and_join_cancelled_io",
"vmess_expired_deadline_and_protect_failure_never_fall_back",
],
}
def rows(identifier):
if identifier == "VMESS-CFG":
return sorted(FIELD_IDS)
if identifier == "VMESS-CODEC":
return ["VM01", "VM03", "VM06", "VM07", "VM08"]
if identifier in GATES or identifier.startswith("VMESS-REGRESSION"):
return []
_, mode, tls, test = NATIVE[identifier]
result = {"VM04", "VM05"}
if identifier in WIRE:
if test == "native_identity_time_replay_rejection":
result |= {"VM01", "VM02"}
elif test == "native_cipher_matrix":
result |= {"VM03", "VM07", "VM08"}
elif test.endswith("udp_boundaries"):
result.add("VM06")
else:
result |= {"C01", "C02", "C03", "C04", "VM01", "VM02"}
if test == "public_body_options":
result |= {"VM03", "VM07", "VM08"}
if "graph" in test:
result.add("C06")
if "base" in test or "udp" in test or "gates" in test:
result |= {"C05", "VM06"}
if tls:
result |= {"T02", "T03", "T05"}
if test == "public_negative":
result.add("T04")
if mode.startswith("ws"):
result |= {"W01", "W02", "W03"}
if mode != "ws":
result |= {"W04", "W05"}
if mode.startswith("grpc"):
result.add("G01")
if mode == "http":
result |= {"H01", "H02", "H03", "H04"}
if mode == "h2":
result |= {"H05", "H06"}
return sorted(result)
def definitions():
cases = []
for identifier in sorted(REQUIRED_IDS):
kind, mode, tls, test = NATIVE.get(
identifier, ("unit", "memory-or-build", False, identifier)
)
native = identifier in NATIVE
substage = "VMESS.acceptance"
if identifier in WIRE:
substage = (
"VMESS.identity"
if "identity" in test
else "VMESS.codec"
if "cipher" in test
else "VMESS.udp"
if "udp" in test
else "VMESS.close"
)
elif identifier == "VMESS-CODEC":
substage = "VMESS.codec"
cases.append(
dict(
case_id=identifier,
stage="VMESS",
substage=substage,
required=True,
row_ids=rows(identifier),
protocol="vmess",
network=mode,
security="tls" if tls else "plain",
udp_codec="raw/xudp/packetaddr"
if "udp" in test or test == "public_base"
else "per-assertion",
field_values={
"network": mode,
"tls": tls,
"scope": "protocol-consumer",
},
outer_family="IPv4/IPv6" if "ipv6" in test else "IPv4",
inner_family="IPv4/IPv6/domain"
if "base" in test or "udp_boundaries" in test
else "per-assertion",
target_type="isolated synthetic origin" if native else "memory-only",
upstream_graph="direct/concrete/nested-select"
if "graph" in test
else "per-assertion",
peer_kind=kind,
peer_config=dict(mode=mode, tls=tls, test=test),
gap_source={
"reason": "Mihomo VMess listener has no HTTP header/legacy H2 "
"or custom ED field; official V2Ray supplements only these paths",
"source": "https://github.com/MetaCubeX/mihomo/blob/v1.19.31/listener/inbound/vmess.go",
}
if kind == "V2"
else None,
expected_observation=[test]
if native
else SCRIPT_ASSERTIONS
if identifier == "VMESS-SCRIPTS"
else OBSERVATIONS.get(identifier, [identifier]),
required_evidence=["structured-assertions", "command", "cleanup"]
+ (["peer-identity", "isolated-origins"] if native else []),
prerequisites=[
"owned-host-only-container-network",
"official-latest-peer",
]
if native
else ["rust-toolchain"],
runner="native-vmess" if native else "vmess-gate",
timeout_seconds=240 if native else 1200,
)
)
return cases
def commands(identifier):
cargo = ["cargo", "test", "--locked", "--all-features"]
if identifier == "VMESS-CFG":
return [cargo + ["--test", "vmess_config"], cargo + ["--lib", "config::"]]
if identifier == "VMESS-CODEC":
return [cargo + ["--test", "vmess_codec"], cargo + ["--lib", "outbound::vmess"]]
if identifier == "VMESS-CANCEL":
return [cargo + ["--test", "vmess_lifecycle"]]
if identifier == "VMESS-RELEASE":
return [
cargo
+ [
"--release",
"--test",
"vmess_config",
"--test",
"vmess_codec",
"--test",
"vmess_lifecycle",
],
cargo + ["--release", "--lib", "outbound::vmess"],
]
if identifier == "VMESS-REGRESSION":
return [
cargo + ["--lib", "xudp::"],
cargo + ["--lib", "security::tls::tests::"],
cargo
+ [
"--test",
"trojan",
"--test",
"trojan_config",
"--test",
"stream_foundations",
"--test",
"h2_stream_regression",
"--test",
"feature_foundations",
"--test",
"limit_foundations",
],
cargo
+ [
"--lib",
"inbound::listen::tests::socks_udp_send_buffer_covers_the_declared_wire_limit",
"--",
"--exact",
],
]
if identifier == "VMESS-QUALITY":
return [
["cargo", "fmt", "--all", "--", "--check"],
[
"cargo",
"clippy",
"--locked",
"--all-features",
"--lib",
"--bins",
"--",
"-D",
"warnings",
],
["vcore-scripts", "check", "c-header"],
["vcore-scripts", "check", "tls-dependencies"],
["ruff", "check", "scripts"],
["ruff", "format", "--check", "scripts"],
["git", "diff", "--check"],
["vcore-scripts", "build", "apple"],
["vcore-scripts", "build", "android"],
]
return []
def gate_result(case, records, events):
result = new_result(case)
good = bool(records) and all(
r.get("exit_code") == 0 and r.get("cleanup") is True for r in records
)
identifier = case["case_id"]
assertions = {}
if identifier in OBSERVATIONS:
for name in case["expected_observation"]:
observed = [e for e in events if e.get("assertion") == name]
assertions[name] = len(observed) == 2 and [
e.get("status") for e in observed
] == ["BEGIN", "PASS"]
good &= (
bool(assertions)
and all(assertions.values())
and all(
e.get("schema_version") == 1 and e.get("status") in {"BEGIN", "PASS"}
for e in events
)
)
else:
assertions = dict.fromkeys(case["expected_observation"], good)
result.update(
status="PASS" if good else "FAIL",
assertions=assertions,
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=all(r.get("cleanup") is True for r in records),
)
return result
def native_results(cases, report, directory):
records = report.get("cases", [])
expected = {case["case_id"] for case in cases}
if len(records) != len(expected) or {r.get("case_id") for r in records} != expected:
raise ValueError("missing, duplicate or unknown VMess native case")
by_id = {record["case_id"]: record for record in records}
output = []
for case in cases:
record = by_id[case["case_id"]]
path = directory / (case["case_id"] + "-events.jsonl")
observed = read_events(path) if path.exists() else []
test = NATIVE[case["case_id"]][3]
observed_ok = (
events_pass(observed, test)
if case["case_id"] in PUBLIC
else observed
== [
dict(
schema_version=1, suite="VMESS-WIRE", assertion=test, status=status
)
for status in ("BEGIN", "PASS")
]
)
clean = (
record.get("command_cleanup") is True
and record.get("cleanup") is True
and report.get("cleanup") is True
)
expected_command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vmess_public" if case["case_id"] in PUBLIC else "vmess_native",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
good = (
observed_ok
and clean
and report.get("source_unchanged") is True
and record.get("exit_code") == 0
and record.get("status") == "PASS"
and record.get("command") == expected_command
and record.get("peer_kind") == case["peer_kind"]
)
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions={test: observed_ok},
evidence=case["required_evidence"] if good else [],
command_exit_code=record.get("exit_code"),
cleanup=clean,
)
output.append(result)
return output
def fields_report(cases, results):
statuses = {result["case_id"]: result["status"] for result in results}
complete = set(statuses) == REQUIRED_IDS and all(
value == "PASS" for value in statuses.values()
)
fields = []
for row in sorted(FIELD_IDS):
owners = [c["case_id"] for c in cases if row in c["row_ids"]]
fields.append(
dict(
row_id=row,
required_cases=owners,
status="PASS" if complete and len(owners) > 1 else "NOT RUN",
)
)
return dict(schema_version=1, stage="VMESS", protocol="vmess", fields=fields)
def execute(cases, run, output, results):
from .protocol_harness import _command, _features, _scripts
for case in cases:
identifier = case["case_id"]
if identifier in NATIVE:
continue
if identifier == "VMESS-FEATURES":
result = _features(case, run, output)
build = _command(
run,
output,
"production-build",
[
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
],
1200,
)
if build.returncode or not build.cleanup:
result.update(
status="FAIL",
command_exit_code=build.returncode,
cleanup=build.cleanup,
)
results[identifier] = result
elif identifier == "VMESS-SCRIPTS":
results[identifier] = _scripts(case, run, output)
else:
events = []
start = len(run["commands"])
for index, argv in enumerate(commands(identifier)):
path = output / f"{identifier}-{index}-events.jsonl"
_command(run, output, f"{identifier}-{index}", argv, 1200, events=path)
if path.exists():
events.extend(read_events(path))
results[identifier] = gate_result(case, run["commands"][start:], events)
native = [case for case in cases if case["case_id"] in NATIVE]
if native:
directory = output / "native"
native_run(directory, [case["case_id"] for case in native])
report = read_json(directory / "vmess-results.json")
(output / "peers.json").write_text(json.dumps(report["peers"], indent=2) + "\n")
for result in native_results(native, report, directory):
results[result["case_id"]] = result
(output / "fields.json").write_text(
json.dumps(fields_report(cases, list(results.values())), indent=2) + "\n"
)
def check(run_dir, cases, run, results, peers, paths):
from .protocol_harness import SCRIPT_OBSERVATIONS
needed = {
"native/vmess-results.json",
"fields.json",
"script-tests.json",
"resources.jsonl",
"peers.json",
"cases.json",
"summary.md",
}
if not needed <= set(paths):
raise ValueError("missing VMESS acceptance artifacts")
expected = {c["case_id"]: c for c in definitions()}
if len(cases) != len(expected) or any(
c != expected.get(c["case_id"]) for c in cases
):
raise ValueError("VMess frozen case metadata changed")
report = read_json(run_dir / "native/vmess-results.json")
source_keys = (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
if (
report.get("source") != {key: run.get(key) for key in source_keys}
or report.get("status") != "PASS"
):
raise ValueError("VMess peer evidence input mismatch")
isolation = report.get("isolation", {})
if (
isolation.get("host_servers") is not False
or isolation.get("network_mode") != "hostOnly"
or not isolation.get("peers")
or not all(
p.get("joined") is True and p.get("started") is True
for p in isolation["peers"]
)
):
raise ValueError("VMess peers not isolated or cleaned")
if peers != report.get("peers") or set(peers) != {"M", "V2"}:
raise ValueError("missing VMess official peers")
for kind, peer in peers.items():
project = "MetaCubeX/mihomo" if kind == "M" else "v2fly/v2ray-core"
if (
peer.get("kind") != kind
or peer.get("target") != "linux-arm64"
or not peer.get("version")
or not peer.get("source_url", "").startswith(
f"https://github.com/{project}/releases/"
)
or not all(
HEX.fullmatch(peer.get(key, ""))
for key in ("archive_sha256", "binary_sha256")
)
):
raise ValueError("invalid official VMess peer identity")
actual = {r["case_id"]: r for r in results}
recalculated = native_results(
[c for c in cases if c["case_id"] in NATIVE], report, run_dir / "native"
)
for case in cases:
identifier = case["case_id"]
argv = commands(identifier)
if not argv:
continue
records = []
events = []
for i, command in enumerate(argv):
matching = [r for r in run["commands"] if r["name"] == f"{identifier}-{i}"]
if len(matching) != 1 or matching[0]["command"] != [
redact(part) for part in command
]:
raise ValueError("missing VMess gate command")
records.extend(matching)
path = run_dir / f"{identifier}-{i}-events.jsonl"
if path.exists():
events.extend(read_events(path))
recalculated.append(gate_result(case, records, events))
for item in recalculated:
if item != actual.get(item["case_id"]):
raise ValueError("VMess results disagree with structured observations")
names = [c["name"] for c in run["commands"]]
needed_commands = {
f"{identifier}-{i}"
for identifier in GATES
for i in range(len(commands(identifier)))
} | {
"production-build",
"offline-scripts",
"feature-default",
"feature-minimal",
*(
"feature-" + name
for name in (
"outbound-trojan",
"outbound-vmess",
"outbound-hysteria2",
"stream-transport",
"quic-transport",
)
),
}
if (
len(names) != len(set(names))
or set(names) != needed_commands
or any(
c.get("exit_code") != 0
or c.get("cleanup") is not True
or c.get("log") not in paths
for c in run["commands"]
)
):
raise ValueError("missing or failed VMess validation command")
for record in run["commands"]:
name = record["name"]
if name == "feature-default":
expected_command = [
"cargo",
"test",
"--locked",
"--test",
"feature_foundations",
]
elif name.startswith("feature-"):
expected_command = [
"cargo",
"check",
"--locked",
"--no-default-features",
"--lib",
]
if name != "feature-minimal":
expected_command += ["--features", name.removeprefix("feature-")]
elif name == "production-build":
expected_command = [
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
]
elif name == "offline-scripts":
expected_command = [
sys.executable,
"-m",
"vcore_scripts.protocol_script_tests",
str(run_dir / "script-tests.json"),
]
else:
continue
if record["command"] != [redact(part) for part in expected_command]:
raise ValueError("VMess feature or script command changed")
script = read_json(run_dir / "script-tests.json")
tests = {t["test"]: t["status"] for t in script["cases"]}
if (
len(tests) != script["tests_run"]
or any(tests.get(name) != "PASS" for name in SCRIPT_OBSERVATIONS.values())
or any(status != "PASS" for status in tests.values())
):
raise ValueError("missing harness failure-path tests")
fields = fields_report(cases, results)
if read_json(run_dir / "fields.json") != fields or any(
f["status"] != "PASS" for f in fields["fields"]
):
raise ValueError("incomplete VMess field coverage")
resources = read_events(run_dir / "resources.jsonl")
owned = [
e
for e in resources
if e.get("suite") == "VMESS-OWNED" and e.get("status") == "PASS"
]
if len(owned) != 40 or not all(idle_resources(e.get("resources")) for e in owned):
raise ValueError("missing 20-round owned resource evidence per selected mode")
print(
f"VMESS: PASS ({len(cases)} required cases; {len(FIELD_IDS)} VMess field rows)"
)
@@ -0,0 +1,352 @@
"""VMESS wire cases against official latest peers and entirely isolated origins."""
from __future__ import annotations
import contextlib
import json
import os
import shutil
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_release import download_mihomo
from .native_release import PeerArtifact, download_native
from .protocol_containers import ContainerLab, command
from .protocol_evidence import read_events
from .protocol_fixtures import certificates
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .protocol_vmess import CASES, peer_config
from .protocol_vmess_public import CASES as PUBLIC_CASES
from .protocol_vmess_public import events_pass, node_config
ALL_CASES = CASES | PUBLIC_CASES
def run(output: Path, selected=None, *, preflight_only=False):
selected = list(ALL_CASES) if selected is None else selected
if (
not selected
or len(set(selected)) != len(selected)
or not set(selected) <= ALL_CASES.keys()
):
raise ValueError("invalid VMESS native selection")
output.mkdir(parents=True, exist_ok=False)
if preflight_only:
# One listener/origin group per official implementation, not a traffic run.
kinds = {ALL_CASES[case][0] for case in selected}
selected = [
next(case for case in selected if ALL_CASES[case][0] == kind)
for kind in sorted(kinds)
]
report = dict(
stage="VMESS",
scope="container-wire-and-public-consumer",
source=source_identity(),
status="NOT RUN",
phase="preflight",
cases=[],
peers={},
isolation={},
)
try:
artifacts = {}
for kind in sorted({ALL_CASES[case][0] for case in selected} | {"M"}):
directory = output / "binaries" / kind
if kind == "M":
identity = {}
binary = download_mihomo(
"linux-arm64", directory=directory, identity=identity
)
artifacts[kind] = PeerArtifact(binary, identity)
else:
artifacts[kind] = download_native(
kind, directory, "linux-arm64", defer_version=True
)
report["peers"][kind] = artifacts[kind].identity
lab = ContainerLab(report["isolation"])
report["phase"] = "build"
built = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vmess_native",
"--test",
"vmess_public",
"--no-run",
],
timeout=180,
)
(output / "build.log").write_text(redact(built.stdout.decode(errors="replace")))
if built.returncode != 0 or not built.cleanup:
raise RuntimeError("container wire build failed")
# Reuse a peer only within one transport; every case gets fresh origin
# controls/ports and sessions. One downloaded release snapshot per run.
groups = {}
for case in selected:
kind, mode, encrypted, test = ALL_CASES[case]
groups.setdefault((kind, mode, encrypted), []).append((case, test))
for (kind, mode, encrypted), cases in groups.items():
report["phase"] = "peer-start"
with tempfile.TemporaryDirectory(
prefix="private-", dir=output
) as temporary:
root = Path(temporary)
with contextlib.ExitStack() as stack:
origin_dir, server_dir = root / "origin", root / "server"
origin_dir.mkdir()
server_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
artifact = artifacts[kind]
shutil.copy2(artifact.binary, server_dir / "peer")
tag = f"{mode}-{'tls' if encrypted else 'plain'}"
origin = lab.start(
stack,
origin_dir,
tag + "-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
argv = (
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
]
if kind == "M"
else [
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
]
)
server = lab.start(stack, server_dir, tag + "-server", argv)
version = command(
"exec",
server.name,
"/data/fixture/peer",
"-v" if kind == "M" else "version",
).strip()
digest = command(
"exec", server.name, "sha256sum", "/data/fixture/peer"
).split()[0]
if not version or digest != artifact.identity["binary_sha256"]:
raise RuntimeError("container peer identity mismatch")
artifact.identity["version"] = version
# Non-leaf pin still exercises actual certificate name checks.
from .protocol_fixtures import certificate_chain
cert, key, pin = (
certificate_chain(server_dir)
if encrypted
else certificates(server_dir)
)
config = peer_config(
kind,
mode,
encrypted,
23000,
Path("/data/fixture") / cert.name,
Path("/data/fixture") / key.name,
)
if kind == "M":
config["hosts"] = {"vcore-fixture.test": origin.ipv4}
config["listeners"][0]["listen"] = "::"
else:
config["dns"]["hosts"] = {"vcore-fixture.test": origin.ipv4}
config["inbounds"][0]["listen"] = "::"
(server_dir / "config.json").write_text(json.dumps(config))
upstream = None
node = node_config(mode, encrypted, server.ipv4, pin)
hop = None
if any(case in PUBLIC_CASES for case, _ in cases):
upstream_dir = root / "upstream"
upstream_dir.mkdir()
shutil.copy2(artifacts["M"].binary, upstream_dir / "peer")
upstream = lab.start(
stack,
upstream_dir,
tag + "-upstream",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
(upstream_dir / "config.json").write_text(
json.dumps(
{
"socks-port": 23001,
"allow-lan": True,
"bind-address": "*",
"log-level": "silent",
"ipv6": True,
"hosts": {"peer.fixture.test": server.ipv4},
"rules": ["MATCH,DIRECT"],
}
)
)
hop = dict(
name="hop",
type="socks5",
server=upstream.ipv4,
port=23001,
udp=True,
)
upstream.release()
upstream.wait_tcp(23001)
fixture = root / "input.json"
fixture.write_text(
json.dumps(
dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
server_ipv6=server.ipv6,
data_dir=str(root / "data"),
node=node,
hop=hop,
peer_kind=kind,
)
)
)
origin.release()
server.release()
origin.wait_tcp(24000)
server.wait_tcp(23000)
report["phase"] = "execute"
for case, test in cases:
if preflight_only:
continue
print(case, flush=True)
record = dict(case_id=case, status="NOT RUN", peer_kind=kind)
report["cases"].append(record)
events = output / f"{case}-events.jsonl"
test_command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vmess_public" if case in PUBLIC_CASES else "vmess_native",
test,
"--",
"--ignored",
"--exact",
"--nocapture",
]
result = run_command(
test_command,
cwd=CORE_DIR,
timeout=240,
limit=4 * 1024 * 1024,
env=dict(
os.environ,
VCORE_VMESS_PEER=f"{server.ipv4}:23000",
VCORE_VMESS_TRANSPORT=json.dumps(
dict(
mode=mode,
tls=encrypted,
pin=pin,
peer_kind=kind,
udp_path_limit=15000,
)
),
VCORE_VMESS_ORIGIN_V4=origin.ipv4,
VCORE_VMESS_AB_INPUT=str(fixture),
VCORE_CASE_EVENTS=str(events),
),
)
(output / f"{case}.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
observed = read_events(events) if events.exists() else []
record.update(
command=test_command,
exit_code=result.returncode,
seconds=result.seconds,
command_cleanup=result.cleanup,
status="PASS"
if result.returncode == 0
and result.cleanup
and (
events_pass(observed, test)
if case in PUBLIC_CASES
else observed
== [
dict(
schema_version=1,
suite="VMESS-WIRE",
assertion=test,
status=status,
)
for status in ("BEGIN", "PASS")
]
)
else "FAIL",
)
server.ensure_alive()
origin.ensure_alive()
print(f"{case}: {record['status']}", flush=True)
for case, _ in cases:
if preflight_only:
continue
next(item for item in report["cases"] if item["case_id"] == case)[
"cleanup"
] = all(
peer.record["joined"]
for peer in [server, origin] + ([upstream] if upstream else [])
)
report["status"] = (
"READY"
if preflight_only
else "PASS"
if (preflight_only or len(report["cases"]) == len(selected))
and all(case["status"] == "PASS" for case in report["cases"])
else "FAIL"
)
except KeyboardInterrupt:
report.update(status="INTERRUPTED", reason="user interruption")
except (OSError, RuntimeError, ValueError, KeyError) as error:
report.update(
status="BLOCKED" if report["phase"] == "preflight" else "FAIL",
reason=str(error)
if isinstance(error, RuntimeError)
else type(error).__name__,
)
finally:
report["cleanup"] = all(
peer["joined"] for peer in report["isolation"].get("peers", [])
)
report["source_unchanged"] = (
source_identity()["source_tree_sha256"]
== report["source"]["source_tree_sha256"]
)
if report["status"] in {"PASS", "READY"} and not (
report["cleanup"] and report["source_unchanged"]
):
report["status"] = "FAIL"
(output / "vmess-results.json").write_text(json.dumps(report, indent=2) + "\n")
return (
0
if report["status"] in {"PASS", "READY"}
else (130 if report["status"] == "INTERRUPTED" else 1)
)
@@ -0,0 +1,153 @@
"""Required public-consumer cases, independent of the persisted manifest."""
from .protocol_evidence import idle_resources
CASES = {
"VMESS-PUBLIC-UDP-FIRST": ("M", "tcp", False, "public_udp_first_response"),
"VMESS-PUBLIC-BODY-OPTIONS": ("M", "tcp", False, "public_body_options"),
}
CASES["VMESS-PUBLIC-WS-ALPN"] = ("M", "ws-alpn", True, "public_alpn_rejection")
for mode in ("tcp", "ws", "grpc", "http", "h2"):
kind = "V2" if mode in {"http", "h2"} else "M"
for tls in (False, True):
label = f"VMESS-PUBLIC-{mode.upper()}-{'TLS' if tls else 'PLAIN'}"
for suffix, test in (
("BASE", "public_base"),
("NEG", "public_negative"),
("GRAPH", "runtime::public_graph"),
("IPV6", "runtime::public_ipv6_and_gates"),
("ENTRYPOINTS", "runtime::public_entrypoints"),
):
CASES[label + "-" + suffix] = (kind, mode, tls, test)
for mode in ("tcp", "grpc"):
for suffix, test in (
("LIFE", "runtime::public_lifecycle"),
("OWNED", "runtime::owned_resources"),
):
CASES[f"VMESS-PUBLIC-{mode.upper()}-{suffix}"] = ("M", mode, True, test)
for mode in ("tcp", "ws", "grpc"):
CASES[f"VMESS-REGRESSION-TROJAN-{mode.upper()}"] = (
"M",
"trojan-" + mode,
True,
"public_legacy_regression",
)
CASES[f"VMESS-PUBLIC-{mode.upper()}-UDP-ISOLATION"] = (
"M",
mode,
True,
"runtime::public_udp_isolation",
)
for mode in ("ws-ed-1", "ws-ed-2048", "ws-header", "ws-path", "grpc-custom"):
kind = "V2" if mode in {"ws-header", "ws-path"} else "M"
CASES[f"VMESS-PUBLIC-{mode.upper()}"] = (kind, mode, True, "public_base")
def node_config(mode, tls, host, pin):
if mode.startswith("trojan-"):
node = node_config(mode.removeprefix("trojan-"), tls, host, pin)
node.update(
type="trojan",
password="synthetic-regression-only",
sni=node.pop("servername"),
)
for key in ("uuid", "tls"):
node.pop(key)
return node
network = mode.split("-")[0]
node = dict(
name="peer",
type="vmess",
server=host,
port=23000,
uuid="07070707-0707-0707-0707-070707070707",
udp=True,
network=network,
tls=tls,
)
if tls:
node.update(servername="localhost", fingerprint=pin)
node["alpn"] = ["h2" if network in {"grpc", "h2"} else "http/1.1"]
if network == "ws":
node["ws-opts"] = {
"path": "/vmess-ws",
"headers": {"Host": "localhost", "X-Fixture": "exact-value"},
}
if mode == "ws-alpn":
node["alpn"] = ["h2", "http/1.1"]
if mode.startswith("ws-ed"):
node["ws-opts"]["max-early-data"] = int(mode.split("-")[-1])
elif mode in {"ws-header", "ws-path"}:
node["ws-opts"].update(
{
"path": "/vmess-ws/",
"max-early-data": 2048,
"early-data-header-name": "X-Vcore-Ed"
if mode == "ws-header"
else "",
}
)
elif network == "grpc":
node["grpc-opts"] = {
"grpc-service-name": "/vmess-grpc/Tun"
if mode == "grpc-custom"
else "vmess-grpc"
}
elif network == "http":
node["http-opts"] = {
"method": "POST",
"path": ["/vmess-http"],
"headers": {"Host": ["localhost"], "X-Fixture": ["exact-value"]},
}
elif network == "h2":
node["h2-opts"] = {"host": ["localhost"], "path": "/vmess-h2"}
return node
def pairs(events, suite, assertion, count):
selected = [
e for e in events if e.get("suite") == suite and e.get("assertion") == assertion
]
return len(selected) == count * 2 and all(
e.get("status") == ("BEGIN" if i % 2 == 0 else "PASS")
for i, e in enumerate(selected)
)
def events_pass(events, test):
main = [event for event in events if event.get("suite") == "VMESS-PUBLIC"]
if len(main) != 2 or not pairs(main, "VMESS-PUBLIC", test, 1):
return False
if any(
event.get("schema_version") != 1 or event.get("status") not in {"BEGIN", "PASS"}
for event in events
):
return False
if test == "public_base" and not (
pairs(events, "VMESS-BASE", "tcp_10mib_both_directions", 3)
and pairs(events, "VMESS-BASE", "udp_each_codec_and_family", 3)
):
return False
if test in {"runtime::public_lifecycle", "runtime::owned_resources"}:
suite = "VMESS-LIFE" if test.endswith("public_lifecycle") else "VMESS-OWNED"
if not pairs(events, suite, "stop_and_remain_quiet", 20):
return False
for end in [
e for e in events if e.get("suite") == suite and e["status"] == "PASS"
]:
if end.get("seconds", 0) < 5:
return False
if suite == "VMESS-OWNED":
checkpoints = end.get("checkpoints", [])
points = {p["phase"]: p["resources"] for p in checkpoints}
if (
len(checkpoints) != 3
or set(points) != {"baseline", "after-stop", "quiet"}
or not all(idle_resources(p) for p in points.values())
or points["after-stop"] != points["quiet"]
or not idle_resources(end.get("resources"))
):
return False
return test != "public_body_options" or pairs(
events, "VMESS-BODY", "config_controls_aead_body", 14
)
@@ -0,0 +1,88 @@
"""Container-only matched UDP diagnostic, not VMESS stage acceptance."""
from __future__ import annotations
import argparse
import itertools
import re
import sys
from pathlib import Path
from .mihomo_isolation import exclusive_run
def options(codecs):
return [
dict(codec=codec, cipher=cipher, padding=padding, length=length)
for codec, cipher, padding, length in itertools.product(
codecs,
["none", "auto", "aes-128-gcm", "chacha20-poly1305"],
[False, True],
[False, True],
)
if cipher != "none" or not (padding or length)
]
def warning_summary(path):
lines = path.read_text(errors="replace").splitlines()
loopbacks = []
for line in lines:
if "reject loopback connection" in line:
endpoints = re.findall(
r"(?:\[[0-9a-fA-F:]+\]|(?:[a-zA-Z0-9_-]+\.)+[a-zA-Z0-9_-]+):(\d+)\b",
line,
)
loopbacks.append(dict(ports=[int(port) for port in endpoints]))
return dict(lines=len(lines), loopback_rejections=loopbacks)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
parser.add_argument(
"--modes",
nargs="+",
choices=["tcp", "ws", "grpc"],
default=["tcp", "ws", "grpc"],
)
parser.add_argument(
"--tls", nargs="+", choices=["plain", "tls"], default=["plain", "tls"]
)
parser.add_argument(
"--codecs",
nargs="+",
choices=["raw", "xudp", "packetaddr"],
default=["raw", "xudp", "packetaddr"],
)
parser.add_argument(
"--families",
nargs="+",
choices=["ipv4", "ipv6", "domain"],
default=["ipv4", "ipv6", "domain"],
)
parser.add_argument("--rounds", type=int, choices=range(1, 21), default=2)
parser.add_argument("--packets", type=int, choices=range(1, 101), default=100)
parser.add_argument("--sizes", nargs="+", type=int)
parser.add_argument("--include-boundary", action="store_true")
parser.add_argument("--collision-probe", action="store_true")
parser.add_argument("--socket-probe", action="store_true")
parser.add_argument("--nat-reuse-probe", action="store_true")
args = parser.parse_args()
if args.collision_probe or args.socket_probe:
parser.error("BLOCKED: archived host-server probes require container migration")
if args.sizes and any(size < 1 or size > 15000 for size in args.sizes):
parser.error("sizes must be between 1 and the 15000-byte fixture budget")
args.tls = [value == "tls" for value in args.tls]
if args.nat_reuse_probe:
args.modes, args.tls, args.codecs = ["tcp"], [False], ["raw", "xudp"]
args.rounds, args.packets, args.sizes, args.families = 1, 1, [1], ["ipv4"]
args.include_boundary = False
from .protocol_vmess_udp_container import run as container_run
with exclusive_run():
return container_run(args)
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,382 @@
"""Container-only VMESS UDP differential; not public-runtime or stage acceptance."""
from __future__ import annotations
import contextlib
import itertools
import json
import os
import shutil
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .mihomo_release import download_mihomo
from .protocol_containers import ContainerLab, command
from .protocol_fixtures import certificates
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .protocol_vmess import peer_config
from .protocol_vmess_udp_ab import options, warning_summary
def client_config(matrix, mode, encrypted, server, origin, client, pin):
listeners, nodes, selected = [], [], []
for index, option in enumerate(matrix):
port = 25000 + index
selected.append(dict(option, socks_port=port, socks_host=client))
node = dict(
name=f"edge-{index}",
type="vmess",
server=server,
port=23000,
uuid="07070707-0707-0707-0707-070707070707",
alterId=0,
cipher=option["cipher"],
network=mode,
tls=encrypted,
udp=True,
)
node.update(
{
"global-padding": option["padding"],
"authenticated-length": option["length"],
}
)
if option["codec"] != "raw":
node["packet-encoding"] = option["codec"]
if encrypted:
node.update(servername="localhost", fingerprint=pin)
if mode == "ws":
node["ws-opts"] = dict(path="/vmess-ws", headers={"Host": "localhost"})
if mode == "grpc":
node["grpc-opts"] = {"grpc-service-name": "vmess-grpc"}
nodes.append(node)
listeners.append(
dict(
name=f"fixture-{index}",
type="socks",
listen=client,
port=port,
udp=True,
proxy=node["name"],
)
)
return {
"mode": "rule",
"log-level": "warning",
"ipv6": True,
"hosts": {"vcore-fixture.test": origin},
"listeners": listeners,
"proxies": nodes,
"rules": ["MATCH,REJECT"],
}, selected
def run(args):
output = args.output.resolve()
output.mkdir(parents=True, exist_ok=False)
matrix = options(args.codecs)
sizes = args.sizes or [1, 64, 512, 1200]
boundary = args.include_boundary or args.sizes is None
report = dict(
scope="container-matched-mihomo-vcore-udp-diagnostic",
status="NOT RUN",
phase="preflight",
source=source_identity(),
cases=[],
isolation={},
peer={},
cleanup=False,
rounds=args.rounds,
packets_per_size=args.packets,
sizes=sizes,
include_boundary=boundary,
boundary_policy="raw/XUDP: 15000; packetaddr: 15000 minus 7/19 address bytes",
families=args.families,
options=matrix,
nat_reuse_probe=args.nat_reuse_probe,
fixture_source_policy="distinct SOCKS UDP sockets retained until transport end",
notes=[
"Separate owned VMs: protocol server, official client ingress, UDP origin.",
"Host runs VCore and test driver only; no published host/server ports.",
"Same body flags, families, sizes, packet count; order alternates.",
"Send, origin observation and reply each retain their 1s deadline.",
"Origin now echoes autonomously and reports exact received bytes via TCP.",
"Failed associations stop without retries; loopback protection unchanged.",
"Virtual IPv6 and wire diagnostic do not prove device or VMESS acceptance.",
],
)
try:
binary = download_mihomo(
"linux-arm64", directory=output / "binary", identity=report["peer"]
)
lab = ContainerLab(report["isolation"])
report["phase"] = "build"
build = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vmess_native",
"--no-run",
],
cwd=CORE_DIR,
timeout=180,
)
(output / "build.log").write_text(redact(build.stdout.decode(errors="replace")))
if build.returncode != 0 or not build.cleanup:
raise RuntimeError("diagnostic build failed")
for mode, encrypted in itertools.product(args.modes, args.tls):
name = f"{mode}-{'tls' if encrypted else 'plain'}"
record = dict(name=name, status="NOT RUN")
report["cases"].append(record)
report["phase"] = "peer-start"
print(f"UDP-CONTAINERS: {name}", flush=True)
with tempfile.TemporaryDirectory(
prefix="private-", dir=output
) as temporary:
root = Path(temporary)
with contextlib.ExitStack() as stack:
directories = {}
peers = {}
for role in ["origin", "server", "client"]:
directory = root / role
directory.mkdir()
directories[role] = directory
if role == "origin":
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
directory / "origin.py",
)
argv = [
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
]
else:
shutil.copy2(binary, directory / "mihomo")
argv = [
"/data/fixture/mihomo",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
]
peers[role] = lab.start(
stack, directory, f"{name}-{role}", argv
)
origin, server, client = (
peers[key] for key in ["origin", "server", "client"]
)
version = command(
"exec", server.name, "/data/fixture/mihomo", "-v"
).strip()
digest = command(
"exec", server.name, "sha256sum", "/data/fixture/mihomo"
).split()[0]
if digest != report["peer"]["binary_sha256"]:
raise RuntimeError(
"container peer identity differs from download"
)
report["peer"]["version"] = version
record["peer_version"] = version
record["origin_python"] = command(
"exec", origin.name, "python", "--version"
).strip()
cert, key, pin = certificates(directories["server"])
config = peer_config(
"M",
mode,
encrypted,
23000,
Path("/data/fixture") / cert.name,
Path("/data/fixture") / key.name,
)
config["log-level"] = "warning"
config["hosts"] = {"vcore-fixture.test": origin.ipv4}
config["listeners"][0]["listen"] = server.ipv4
(directories["server"] / "config.json").write_text(
json.dumps(config)
)
config, selected = client_config(
matrix,
mode,
encrypted,
server.ipv4,
origin.ipv4,
client.ipv4,
pin,
)
(directories["client"] / "config.json").write_text(
json.dumps(config)
)
for peer in peers.values():
peer.release()
origin.wait_tcp(24000)
server.wait_tcp(23000)
for option in selected:
client.wait_tcp(option["socks_port"])
report["phase"] = "execute"
fixture = root / "input.json"
fixture.write_text(
json.dumps(
dict(
isolation="containers",
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
options=selected,
rounds=args.rounds,
packets_per_size=args.packets,
sizes=sizes,
include_boundary=boundary,
families=args.families,
)
)
)
events = output / f"{name}.jsonl"
test_command = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vmess_native",
"udp_ab::socks_nat_source_reuse"
if args.nat_reuse_probe
else "udp_ab::matched_clients",
"--",
"--ignored",
"--exact",
"--nocapture",
]
execution = run_command(
test_command,
cwd=CORE_DIR,
timeout=600,
limit=4 * 1024 * 1024,
env=dict(
os.environ,
VCORE_VMESS_PEER=f"{server.ipv4}:23000",
VCORE_VMESS_TRANSPORT=json.dumps(
dict(mode=mode, tls=encrypted, pin=pin)
),
VCORE_VMESS_ORIGIN_V4=origin.ipv4,
VCORE_VMESS_AB_INPUT=str(fixture),
VCORE_VMESS_AB_EVENTS=str(events),
VCORE_VMESS_AB_PRIVATE=str(root),
),
)
(output / f"{name}.log").write_text(
redact(execution.stdout.decode(errors="replace"))
)
observations = (
[json.loads(line) for line in events.read_text().splitlines()]
if events.exists()
else []
)
expected = (
3
if args.nat_reuse_probe
else args.rounds * len(matrix) * len(args.families) * 2
)
record.update(
command=test_command,
exit_code=execution.returncode,
seconds=execution.seconds,
command_cleanup=execution.cleanup,
expected_associations=expected,
observed_associations=len(observations),
failures=[
item for item in observations if item["status"] != "PASS"
],
clients={
kind: dict(
associations=sum(
item["client"] == kind for item in observations
),
failed=sum(
item["client"] == kind and item["status"] != "PASS"
for item in observations
),
completed_packets=sum(
item["completed"]
for item in observations
if item["client"] == kind
),
)
for kind in ["vcore", "mihomo"]
},
)
for peer in peers.values():
peer.ensure_alive()
# Stop/delete VMs and join bounded log readers before inspection.
for role in ["server", "client"]:
record[f"{role}_warnings"] = warning_summary(
directories[role] / "peer.log"
)
record["cleanup"] = all(
peer.record["joined"] for peer in peers.values()
)
expected_outcome = (
[item["status"] for item in observations]
== ["PASS", "REPRODUCED", "PASS"]
if args.nat_reuse_probe
else not record["failures"]
)
record["status"] = (
("REPRODUCED" if args.nat_reuse_probe else "PASS")
if (
execution.returncode == 0
and execution.cleanup
and record["cleanup"]
and len(observations) == expected
and expected_outcome
and all(
item.get("driver_joined", True) for item in observations
)
)
else "FAIL"
)
print(
json.dumps(
{key: record[key] for key in ["name", "status", "clients"]}
),
flush=True,
)
expected_status = "REPRODUCED" if args.nat_reuse_probe else "PASS"
report["status"] = (
expected_status
if report["cases"]
and all(case["status"] == expected_status for case in report["cases"])
else "FAIL"
)
except KeyboardInterrupt:
report.update(status="INTERRUPTED", reason="user interruption")
except (OSError, RuntimeError, ValueError, KeyError) as error:
status = "BLOCKED" if report["phase"] == "preflight" else "FAIL"
report.update(status=status, reason=type(error).__name__)
print(f"UDP-CONTAINERS {status}: {type(error).__name__}", flush=True)
finally:
report["cleanup"] = all(
peer["joined"] for peer in report["isolation"].get("peers", [])
)
report["source_unchanged"] = (
source_identity()["source_tree_sha256"]
== report["source"]["source_tree_sha256"]
)
if report["status"] in {"PASS", "REPRODUCED"} and not (
report["cleanup"] and report["source_unchanged"]
):
report["status"] = "FAIL"
(output / "udp-ab.json").write_text(json.dumps(report, indent=2) + "\n")
return (
0
if report["status"] == "PASS"
else (130 if report["status"] == "INTERRUPTED" else 1)
)
@@ -0,0 +1,660 @@
"""XHTTP execution and independently recomputed, fail-closed stage evidence."""
from __future__ import annotations
import json
import sys
from .builds import DEFAULT_FEATURES
from .protocol_evidence import HEX, idle_resources, new_result, read_events, read_json
from .protocol_inputs import redact
from .protocol_vless_acceptance import commands as previous_commands
from .protocol_xhttp_catalog import (
FIELD_IDS,
GATES,
NATIVE,
OBSERVATIONS,
REPRESENTATIVES,
SECURITY,
UNIT_TESTS,
definitions,
)
from .protocol_xhttp_fields import PUBLIC_TESTS, events_pass, public_events_pass
def test_command(test):
return [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"vless_public" if test in PUBLIC_TESTS else "xhttp_native",
test,
"--",
"--exact",
"--ignored",
"--nocapture",
]
def commands(identifier):
cargo = ["cargo", "test", "--locked", "--all-features"]
if identifier in {"XHTTP-CFG", "XHTTP-UNIT", "XHTTP-RELEASE"}:
names = list(UNIT_TESTS)
if identifier == "XHTTP-CFG":
names = names[:2]
elif identifier == "XHTTP-UNIT":
names = names[2:]
flags = ["--release"] if identifier == "XHTTP-RELEASE" else []
selected = sum((["--test", name] for name in names), [])
result = [cargo + flags + selected]
if identifier != "XHTTP-CFG":
result += [
cargo + flags + ["--lib", "transport::xhttp::tests::"],
cargo + flags + ["--lib", "transport::sing_mux::"],
cargo + flags + ["--test", "vless_lifecycle"],
]
return result
if identifier == "XHTTP-REGRESSION":
return sum(
(
previous_commands("VLESS-" + part)
for part in ("REGRESSION", "CODEC", "TRANSPORT", "VISION")
),
[],
)
if identifier == "XHTTP-QUALITY":
return previous_commands("VLESS-QUALITY") + [
[
"cargo",
"test",
"--locked",
"--manifest-path",
"crates/vcore-netstack/Cargo.toml",
"--all-targets",
],
[
"cargo",
"clippy",
"--locked",
"--manifest-path",
"crates/vcore-netstack/Cargo.toml",
"--all-targets",
"--",
"-D",
"warnings",
],
]
return []
def gate_result(case, records, events):
result = new_result(case)
good = bool(records) and all(
r.get("exit_code") == 0 and r.get("cleanup") is True for r in records
)
assertions = {}
for name in case["expected_observation"]:
observed = [e for e in events if e.get("assertion") == name]
assertions[name] = len(observed) == 2 and [
e.get("status") for e in observed
] == ["BEGIN", "PASS"]
if case["case_id"] not in OBSERVATIONS:
assertions = dict.fromkeys(case["expected_observation"], good)
else:
good &= all(assertions.values()) and all(
e.get("schema_version") == 1 and e.get("status") in {"BEGIN", "PASS"}
for e in events
)
if case["case_id"] != "XHTTP-REGRESSION":
unit = [e for e in events if e.get("suite") == "XHTTP-UNIT"]
good &= len(unit) == 2 * len(assertions) and all(
e["assertion"] in assertions for e in unit
)
result.update(
status="PASS" if good else "FAIL",
assertions=assertions,
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=bool(records) and all(r.get("cleanup") is True for r in records),
)
return result
def native_results(cases, report, directory, *, allow_partial=False):
records = report.get("cases", [])
expected = {case["case_id"] for case in cases}
actual = {r.get("case_id") for r in records}
if (
len(records) != len(actual)
or not actual <= expected
or (not allow_partial and actual != expected)
):
raise ValueError("missing, duplicate or unknown XHTTP native case")
by_id = {record["case_id"]: record for record in records}
output = []
for case in cases:
identifier = case["case_id"]
if identifier not in by_id:
continue
record = by_id[identifier]
variant, test = NATIVE[identifier]
path = directory / f"{identifier}-events.jsonl"
observed = read_events(path) if path.exists() else []
observed_ok = (
public_events_pass(observed, test)
if test in PUBLIC_TESTS
else events_pass(observed, test, owned=test.startswith("lifecycle::"))
)
clean = record.get("command_cleanup") is True and report.get("cleanup") is True
good = (
observed_ok
and clean
and report.get("source_unchanged") is True
and record.get("status") == "PASS"
and record.get("exit_code") == 0
and record.get("command") == test_command(test)
and record.get("assertion") == test
and record.get("variant") == variant
)
if test.startswith("close::"):
path = directory / f"{variant}-close-reference.json"
reference = read_json(path) if path.exists() else {}
good &= (
reference.get("scope") == "same-mode"
and reference.get("terminated") is True
and reference.get("tail_hex") == ""
)
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions={test: observed_ok},
evidence=case["required_evidence"] if good else [],
command_exit_code=record.get("exit_code"),
cleanup=clean,
)
output.append(result)
return output
def security_results(cases, report, directory, *, allow_partial=False):
from .protocol_xhttp_security import security_cases
expected = {}
identity = {
k: {"certificate": "fixture", "private-key": "fixture"}
for k in ("valid", "absent", "expired", "wrong-ca")
}
for case in cases:
version = SECURITY[case["case_id"]]
base = dict(
alpn=["http/1.1" if version == "h1" else version],
certificate="fixture",
**{"private-key": "fixture", "xhttp-opts": {}},
)
expected[version] = {
f"{version}-{name}": failure
for name, _, _, failure in security_cases(
base, identity, identity["valid"], "2001:db8::1"
)
}
records = report.get("cases", [])
ids = set().union(*(set(e) for e in expected.values()))
actual = {r.get("case_id") for r in records}
if (
len(records) != len(actual)
or not actual <= ids
or (not allow_partial and actual != ids)
):
raise ValueError("missing, duplicate or unknown XHTTP security case")
results = []
by_id = {r["case_id"]: r for r in records}
for case in cases:
version = SECURITY[case["case_id"]]
if allow_partial and not actual.intersection(expected[version]):
continue
clean = report.get("cleanup") is True and report.get("owned_remaining") == []
good = clean and report.get("source_unchanged") is True
for name, failure in expected[version].items():
record = by_id.get(name, {})
path = directory / f"{name}-events.jsonl"
events = read_events(path) if path.exists() else []
good &= (
events_pass(events, "security::native_xhttp_security")
and record.get("exit_code") == 0
and record.get("command_cleanup") is True
and record.get("command")
== test_command("security::native_xhttp_security")
and record.get("status") == "PASS"
)
if version == "h3" and failure:
category = {
"absent": "certificate_required",
"expired": "certificate_expired",
"wrong-ca": "unknown_ca",
}[failure]
good &= record.get("identity_enforced") is True and any(
e.get("category") == category
for e in record.get("certificate_errors", [])
)
result = new_result(case)
result.update(
status="PASS" if good else "FAIL",
assertions={case["case_id"]: good},
evidence=case["required_evidence"] if good else [],
command_exit_code=0 if good else 1,
cleanup=clean,
)
results.append(result)
return results
def fields_report(cases, results):
statuses = {r["case_id"]: r["status"] for r in results}
complete = set(statuses) == {c["case_id"] for c in definitions()} and all(
s == "PASS" for s in statuses.values()
)
fields = []
for row in sorted(FIELD_IDS):
owners = [c["case_id"] for c in cases if row in c["row_ids"]]
behavior = [c for c in owners if c in NATIVE or c in SECURITY]
fields.append(
dict(
row_id=row,
required_cases=owners,
status="PASS"
if complete and behavior and "XHTTP-CFG" in owners
else "NOT RUN",
)
)
return dict(schema_version=1, stage="XHTTP", protocol="vless", fields=fields)
def prepare_peers(output, kinds):
from .caddy_build import build_caddy
from .mihomo_release import download_mihomo
from .native_release import download_native
supplied = {}
for kind in sorted(kinds):
directory = output / "binaries" / kind
if kind == "M":
identity = {}
binary = download_mihomo(
"linux-arm64", directory=directory, identity=identity
)
supplied[kind] = binary, identity
else:
artifact = (
build_caddy(directory)
if kind == "Caddy"
else download_native(kind, directory, "linux-arm64", defer_version=True)
)
supplied[kind] = artifact.binary, artifact.identity
return supplied
def preflight(output):
"""Only verify the XHTTP native artifacts inside an owned isolated guest."""
import shutil
import tempfile
from contextlib import ExitStack
from .protocol_containers import ContainerLab, command
from .protocol_inputs import CORE_DIR
supplied = prepare_peers(output, {"M", "XR", "V2", "Caddy"})
isolation = {}
lab = ContainerLab(isolation, mtu=1500)
try:
with tempfile.TemporaryDirectory(
prefix="xhttp-preflight-", dir=CORE_DIR / "target"
) as tmp:
from pathlib import Path
root = Path(tmp)
for kind, (binary, _) in supplied.items():
shutil.copy2(binary, root / kind)
with ExitStack() as stack:
peer = lab.start(
stack,
root,
"preflight",
["python", "-c", "import time; time.sleep(600)"],
)
peer.release()
for kind, (_, identity) in supplied.items():
binary = "/data/fixture/" + kind
version = command(
"exec", peer.name, binary, "-v" if kind == "M" else "version"
).strip()
digest = command("exec", peer.name, "sha256sum", binary).split()[0]
if not version or digest != identity["binary_sha256"]:
raise RuntimeError("isolated XHTTP artifact identity mismatch")
identity["version"] = version
finally:
(output / "peers.json").write_text(
json.dumps({k: i for k, (_, i) in supplied.items()}, indent=2) + "\n"
)
(output / "preflight.json").write_text(
json.dumps(
dict(
scope="preflight-only",
wire_acceptance="NOT RUN",
isolation=isolation,
),
indent=2,
)
+ "\n"
)
def execute(cases, run, output, results):
from .protocol_harness import _command, _features, _scripts
from .protocol_xhttp_fields import run as fields_run
from .protocol_xhttp_security import run as security_run
for case in cases:
identifier = case["case_id"]
if identifier not in GATES:
continue
if identifier == "XHTTP-FEATURES":
result = _features(case, run, output)
for name, command in (
(
"feature-outbound-vless",
[
"cargo",
"check",
"--locked",
"--no-default-features",
"--lib",
"--features",
"outbound-vless",
],
),
(
"production-build",
[
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
],
),
):
observed = _command(run, output, name, command, 1200)
if observed.returncode or not observed.cleanup:
result.update(
status="FAIL",
command_exit_code=observed.returncode,
cleanup=observed.cleanup,
)
results[identifier] = result
elif identifier == "XHTTP-SCRIPTS":
results[identifier] = _scripts(case, run, output)
else:
start = len(run["commands"])
events = []
for i, command in enumerate(commands(identifier)):
path = output / f"{identifier}-{i}-events.jsonl"
observed = _command(
run, output, f"{identifier}-{i}", command, 1200, events=path
)
if path.exists():
events.extend(read_events(path))
if observed.returncode or not observed.cleanup:
break
results[identifier] = gate_result(case, run["commands"][start:], events)
if results[identifier]["status"] != "PASS":
raise RuntimeError(f"{identifier} failed; later cases remain NOT RUN")
native = [c for c in cases if c["case_id"] in NATIVE]
security = [c for c in cases if c["case_id"] in SECURITY]
kinds = {c["peer_kind"] for c in native + security}
if native or security:
kinds.add("M")
if any(c["peer_kind"] == "XR" for c in security):
kinds.add("Caddy")
supplied = prepare_peers(output, kinds)
try:
if native:
jobs = {}
for case in native:
identifier = case["case_id"]
variant, test = NATIVE[identifier]
jobs.setdefault(variant, []).append(dict(case_id=identifier, test=test))
directory = output / "native"
# Exercise native-gap transports before the long Mihomo matrix.
# Ordering changes neither the frozen case set nor its evidence.
ordered = sorted(
jobs, key=lambda name: (not name.startswith(("h3-", "outer-")), name)
)
fields_run(directory, ordered, jobs=jobs, supplied=supplied)
report = read_json(directory / "xhttp-fields-results.json")
for result in native_results(native, report, directory, allow_partial=True):
results[result["case_id"]] = result
if report["status"] != "PASS":
raise RuntimeError("native XHTTP cases failed")
if security:
directory = output / "security"
security_run(
directory, [SECURITY[c["case_id"]] for c in security], supplied=supplied
)
report = read_json(directory / "xhttp-security-results.json")
for result in security_results(
security, report, directory, allow_partial=True
):
results[result["case_id"]] = result
if report["status"] != "PASS":
raise RuntimeError("native XHTTP security failed")
finally:
(output / "peers.json").write_text(
json.dumps({k: identity for k, (_, identity) in supplied.items()}, indent=2)
+ "\n"
)
(output / "fields.json").write_text(
json.dumps(fields_report(cases, list(results.values())), indent=2) + "\n"
)
def required_command_names():
return {
f"{identifier}-{i}"
for identifier in GATES
for i in range(len(commands(identifier)))
} | {
"production-build",
"offline-scripts",
"feature-default",
"feature-minimal",
"feature-outbound-vless",
*(
"feature-" + name
for name in (
"outbound-trojan",
"outbound-vmess",
"outbound-hysteria2",
"stream-transport",
"quic-transport",
)
),
}
def check(run_dir, cases, run, results, peers, paths):
from .protocol_harness import SCRIPT_OBSERVATIONS
needed = {
"native/xhttp-fields-results.json",
"security/xhttp-security-results.json",
"fields.json",
"script-tests.json",
"resources.jsonl",
"peers.json",
"cases.json",
"summary.md",
}
if not needed <= set(paths) or cases != definitions():
raise ValueError("missing XHTTP artifacts or altered frozen metadata")
recalculated = []
source_keys = (
"parent_commit",
"source_tree_sha256",
"dirty_patch_sha256",
"lock_sha256",
)
for name, file, evaluator, wanted in (
("native", "xhttp-fields-results.json", native_results, NATIVE),
("security", "xhttp-security-results.json", security_results, SECURITY),
):
report = read_json(run_dir / name / file)
if (
report.get("source") != {key: run.get(key) for key in source_keys}
or report.get("status") != "PASS"
):
raise ValueError("XHTTP native source identity mismatch")
isolation = report.get("isolation", {})
if (
isolation.get("host_servers") is not False
or isolation.get("network_mode") != "hostOnly"
or not isolation.get("peers")
or any(
p.get("joined") is not True or p.get("started") is not True
for p in isolation["peers"]
)
):
raise ValueError("XHTTP native isolation or cleanup missing")
if any(
identity != peers.get(kind)
for kind, identity in report.get("peers", {}).items()
):
raise ValueError("XHTTP peer identity changed between reports")
recalculated += evaluator(
[c for c in cases if c["case_id"] in wanted], report, run_dir / name
)
if set(peers) != {"M", "XR", "V2", "Caddy"}:
raise ValueError("XHTTP required official peers missing")
for kind, peer in peers.items():
project = {
"M": "MetaCubeX/mihomo",
"XR": "XTLS/Xray-core",
"V2": "v2fly/v2ray-core",
"Caddy": "caddyserver/caddy",
}[kind]
if (
peer.get("target") != "linux-arm64"
or not peer.get("version")
or not peer.get("source_url", "").startswith(
f"https://github.com/{project}/releases/"
)
or not HEX.fullmatch(peer.get("binary_sha256", ""))
):
raise ValueError("invalid official XHTTP peer identity")
if kind != "Caddy" and not HEX.fullmatch(peer.get("archive_sha256", "")):
raise ValueError("missing native download digest")
if kind == "Caddy" and (
peer.get("source_built") is not True
or peer.get("build_cleanup") is not True
or peer.get("extra_plugins") != []
or set(peer.get("inputs", {})) != {"go.mod", "go.sum", "main.go"}
or any(not HEX.fullmatch(value) for value in peer["inputs"].values())
):
raise ValueError("gateway is outside the approved xcaddy build exception")
records = run.get("commands", [])
if (
len(records) != len(required_command_names())
or {r.get("name") for r in records} != required_command_names()
or any(
r.get("exit_code") != 0
or r.get("cleanup") is not True
or r.get("log") not in paths
for r in records
)
):
raise ValueError("missing, duplicate or failed XHTTP command")
for case in cases:
identifier = case["case_id"]
expected_commands = commands(identifier)
if not expected_commands:
continue
selected, events = [], []
for i, argv in enumerate(expected_commands):
record = next(r for r in records if r["name"] == f"{identifier}-{i}")
if record["command"] != [redact(part) for part in argv]:
raise ValueError("XHTTP validation command changed")
selected.append(record)
path = run_dir / f"{identifier}-{i}-events.jsonl"
if path.exists():
events.extend(read_events(path))
recalculated.append(gate_result(case, selected, events))
actual = {r["case_id"]: r for r in results}
if any(r != actual.get(r["case_id"]) for r in recalculated):
raise ValueError("XHTTP results disagree with structured evidence")
for record in records:
name = record["name"]
if name == "feature-default":
argv = ["cargo", "test", "--locked", "--test", "feature_foundations"]
elif name.startswith("feature-"):
argv = ["cargo", "check", "--locked", "--no-default-features", "--lib"]
if name != "feature-minimal":
argv += ["--features", name.removeprefix("feature-")]
elif name == "production-build":
argv = [
"cargo",
"build",
"--locked",
"--release",
"--no-default-features",
"--features",
DEFAULT_FEATURES,
"--lib",
]
elif name == "offline-scripts":
argv = [
sys.executable,
"-m",
"vcore_scripts.protocol_script_tests",
str(run_dir / "script-tests.json"),
]
else:
continue
if record["command"] != [redact(part) for part in argv]:
raise ValueError("XHTTP feature or script command changed")
script = read_json(run_dir / "script-tests.json")
tests = {t["test"]: t["status"] for t in script["cases"]}
required_tests = set(SCRIPT_OBSERVATIONS.values()) | {
"test_protocol_xhttp.XhttpAcceptanceTest." + name
for name in (
"test_frozen_catalog_has_all_rows_modes_and_mux_wires",
"test_unknown_duplicate_wrong_suite_and_partial_events_fail",
"test_unit_gate_requires_every_frozen_assertion_and_successful_command",
"test_native_missing_and_duplicate_records_are_not_success",
)
}
if (
len(tests) != script["tests_run"]
or not required_tests <= set(tests)
or any(v != "PASS" for v in tests.values())
):
raise ValueError("XHTTP offline failure-path proof missing")
fields = fields_report(cases, results)
if read_json(run_dir / "fields.json") != fields or any(
f["status"] != "PASS" for f in fields["fields"]
):
raise ValueError("incomplete XHTTP field behavior coverage")
resources = read_events(run_dir / "resources.jsonl")
owned = [
e
for e in resources
if e.get("suite") == "XHTTP-OWNED" and e.get("status") == "PASS"
]
if len(owned) != 20 * len(REPRESENTATIVES) or any(
not idle_resources(e.get("resources")) for e in owned
):
raise ValueError("missing XHTTP twenty-round owned resource proof")
print(f"XHTTP: PASS ({len(cases)} required cases; {len(FIELD_IDS)} field rows)")
@@ -0,0 +1,394 @@
"""Frozen XHTTP cases. Independent tuning is pairwise; coupled modes remain explicit."""
from __future__ import annotations
import hashlib
from .protocol_vless_acceptance import OBSERVATIONS as PREVIOUS
from .protocol_vmess_acceptance import SCRIPT_ASSERTIONS
from .protocol_xhttp_fields import native_kind, variants
UNIT_TESTS = {
"xhttp_config": [
"xhttp_fields_reject_wrong_types_null_and_every_range_boundary_before_io",
"empty_http_authority_falls_back_to_each_legs_authentication_name",
"download_security_changes_do_not_silently_discard_inherited_certificate_policy",
"h3_requires_exclusive_alpn_and_standard_tls_on_each_leg",
"custom_headers_inherit_replace_clear_and_do_not_leak_values",
"conflicting_request_fields_are_rejected_on_both_legs_before_io",
"http_version_selects_h1_only_for_its_single_alpn_and_supports_plaintext",
"download_certificate_policy_inherits_and_explicit_false_or_empty_clears",
"download_mtls_identity_is_inherited_or_replaced_and_cleared_as_a_pair",
"download_reality_object_replaces_inherits_or_clears_without_merging_keys",
"reuse_object_presence_and_download_whole_object_replacement_are_preserved",
],
"sing_mux_config": [
"sing_mux_accepts_three_protocols_and_rejects_invalid_or_ignored_options"
],
"xhttp_requests": [
"packet_up_coalesces_small_writes_and_flushes_without_more_caller_io",
"packet_up_backpressures_bounded_batches_and_stop_cancels_the_timer",
"first_vless_read_never_acknowledges_or_replays_a_concurrent_packet_write",
"response_codes_follow_mihomo_streaming_and_packet_rules",
"streaming_content_type_can_be_disabled_without_disabling_padding",
"custom_semantic_headers_are_allowed_when_no_generated_field_overwrites_them",
"padding_uses_the_configured_http_location_and_encoding",
"packet_metadata_and_upload_methods_are_sent_in_each_supported_location",
"packet_upload_can_move_payload_into_bounded_header_or_cookie_chunks",
"http1_stream_one_is_chunked_duplex_and_shutdown_closes_both_directions",
"http1_packet_up_shares_a_session_across_two_connections",
"http1_response_body_can_arrive_after_the_response_headers",
"custom_request_headers_reach_the_http_peer_without_changing_body",
],
"xhttp_reuse": [
"enabled_reuse_shares_physical_h2_without_one_close_killing_its_sibling",
"reuse_thresholds_expand_and_retire_without_closing_active_sessions",
"expired_transport_is_not_assigned_again_while_old_session_stays_live",
"pooled_packet_posts_do_not_each_consume_a_transport_lease",
"h2_keepalive_uses_default_explicit_and_disabled_idle_periods",
"download_uses_its_own_reuse_counters_and_keeps_shared_sessions_alive",
"h1_packet_upload_reuses_its_idle_connection_but_reopens_the_cancelled_get",
"stopping_during_handshake_prevents_late_driver_admission",
"stop_wakes_a_handshake_even_when_the_peer_never_responds",
],
"sing_mux": [
"yamux_replacing_dropped_streams_at_capacity_keeps_the_sibling_alive",
"sing_mux_scheduling_uses_the_selected_branch_not_a_global_stream_quota",
"h2mux_sends_idle_ping_and_retires_a_peer_that_never_acknowledges",
"only_tcp_preserves_all_three_vless_udp_wire_commands",
"h2mux_reuses_physical_vless_and_cancels_only_one_logical_stream",
"yamux_reuses_physical_vless_and_cancels_only_one_logical_stream",
"smux_reuses_physical_vless_and_cancels_only_one_logical_stream",
"sing_mux_udp_preserves_addresses_and_cancellation_safe_partial_frames",
],
"xhttp_budget": [
"h3_rejects_incapable_or_small_budget_upstreams_before_any_datagram"
],
}
DRIVER_TESTS = ["malformed_smux_headers_close_the_owned_driver_before_reading_a_body"]
FIELD_IDS = {
*(f"X{i:02}" for i in range(1, 30)),
*(f"D{i:02}" for i in range(1, 16)),
*(f"D{i:02}" for i in range(27, 33)),
*(f"M{i:02}" for i in range(1, 8)),
}
OBSERVATIONS = {
"XHTTP-CFG": UNIT_TESTS["xhttp_config"] + UNIT_TESTS["sing_mux_config"],
"XHTTP-UNIT": sum(
(
UNIT_TESTS[n]
for n in ("xhttp_requests", "xhttp_reuse", "sing_mux", "xhttp_budget")
),
[],
)
+ DRIVER_TESTS,
"XHTTP-REGRESSION": PREVIOUS["VLESS-REGRESSION"]
+ PREVIOUS["VLESS-CODEC"]
+ PREVIOUS["VLESS-TRANSPORT"]
+ PREVIOUS["VLESS-VISION"],
}
OBSERVATIONS["XHTTP-RELEASE"] = OBSERVATIONS["XHTTP-CFG"] + OBSERVATIONS["XHTTP-UNIT"]
GATES = {
"XHTTP-CFG",
"XHTTP-UNIT",
"XHTTP-REGRESSION",
"XHTTP-RELEASE",
"XHTTP-QUALITY",
"XHTTP-FEATURES",
"XHTTP-SCRIPTS",
}
SECURITY = {f"XHTTP-SECURITY-{v.upper()}": v for v in ("h1", "h2", "h3")}
NATIVE = {}
VERSIONS = ("h1", "h2", "h1c", "h2c", "h1r", "h2r", "h3")
ALL_VARIANTS = variants()
def add(variant, suffix, test):
if variant not in ALL_VARIANTS:
raise ValueError("missing frozen XHTTP variant")
# Case-sensitive table names need distinct stable identifiers.
digest = hashlib.sha256(variant.encode()).hexdigest()[:8].upper()
identifier = f"XHTTP-{variant.upper()}-{suffix}-{digest}"
if identifier in NATIVE:
raise ValueError("duplicate XHTTP requirement")
NATIVE[identifier] = (variant, test)
for version in VERSIONS:
for mode in ("stream-one", "stream-up", "packet-up"):
variant = f"{version}-{mode}-headers"
add(variant, "BASE", "public_base")
add(variant, "CLOSE", "close::native_mihomo_xhttp_close")
for mode in ("stream-up", "packet-up"):
variant = f"{version}-{mode}-split-reuse"
add(variant, "BASE", "public_base")
add(variant, "CLOSE", "close::native_mihomo_xhttp_close")
for suffix in ("auto", "auto-download"):
add(f"{version}-{suffix}", "AUTO", "native_xhttp_request_fields")
if version.endswith("r"):
for suffix in (
"download-reality-replacement",
"reject-download-short-id",
"reject-download-public-key",
):
add(
f"{version}-{suffix}",
"SECURITY",
"security::native_xhttp_security"
if suffix.startswith("reject")
else "native_xhttp_request_fields",
)
# Every finite request enum is consumed on H1, H2 and H3. Security is orthogonal
# here; every plain/TLS/REALITY + mode + download branch has a public BASE above.
for version in ("h1", "h2", "h3"):
for name, (options, v) in ALL_VARIANTS.items():
if v != version or "_mux" in options or "_keepalive" in options:
continue
suffix = name.removeprefix(version + "-")
if suffix.startswith(
("padding-", "metadata-", "payload-", "session-", "post-", "reject-")
) or suffix in (
"stream-up-no-grpc",
"stream-one-no-grpc",
"body-auto",
):
add(
name,
"FIELDS",
"security::native_xhttp_security"
if options.get("_reject")
else "native_xhttp_request_fields",
)
add(
"h3-keepalive",
"KEEPALIVE",
"keepalive::native_h3_keepalive_observes_each_leg_and_stop",
)
# All three wire protocols and both only-tcp branches for every outer transport.
# Padding parity alternates between branches; H2 additionally expands all padding
# values. Padding and the outer framing are independent, tested separately above.
for outer in (
*VERSIONS,
*(
f"outer-{m}"
for m in (
"tcp",
"tcp-tls",
"tcp-reality",
"ws",
"ws-tls",
"ws-reality",
"grpc",
"grpc-tls",
"grpc-reality",
"http",
"http-tls",
"h2",
"h2-tls",
"ws-header",
"ws-header-tls",
"ws-path",
"ws-path-tls",
)
),
):
for index, protocol in enumerate(("h2mux", "smux", "yamux")):
for only in (False, True):
paddings = (
(False, True) if outer == "h2" else ((index + int(only)) % 2 == 0,)
)
for padding in paddings:
variant = (
f"{outer}-mux-{protocol}-{'padded' if padding else 'plain'}"
+ ("-only-tcp" if only else "")
)
add(variant, "BASE", "public_base")
for version in ("h1", "h2", "h3"):
for protocol in ("h2mux", "smux", "yamux"):
add(
f"{version}-mux-{protocol}-padding-required",
"REJECT",
"security::native_xhttp_security",
)
REPRESENTATIVES = (
"h1-packet-up-split-reuse",
"h2-stream-up-split-reuse",
"h3-packet-up-split-reuse",
"h2-mux-h2mux-padded",
"h2-mux-smux-padded",
"h2-mux-yamux-padded",
)
for variant in REPRESENTATIVES:
for suffix, test in (
("GRAPH", "runtime::public_graph"),
("IPV6", "runtime::public_ipv6_and_gates"),
("ENTRYPOINTS", "runtime::public_entrypoints"),
("LIFE", "runtime::public_lifecycle"),
("ISOLATION", "runtime::public_udp_isolation"),
("OWNED", "lifecycle::native_xhttp_owned_resources"),
("NEG", "public_negative"),
):
add(variant, suffix, test)
for variant in (
"h1-packet-up-reuse",
"h2-stream-up-reuse",
"h3-packet-up-reuse",
"h2-mux-h2mux-padded",
"h2-mux-smux-padded",
"h2-mux-yamux-padded",
):
add(
variant,
"POOL-SELECTION",
"runtime::grpc_pool_keeps_physical_selection_until_new_transport",
)
X_NAMES = (
"path",
"host",
"mode",
"headers",
"no-grpc-header",
"x-padding-bytes",
"x-padding-obfs-mode",
"x-padding-key",
"x-padding-header",
"x-padding-placement",
"x-padding-method",
"uplink-http-method",
"session-placement",
"session-key",
"session-table",
"session-length",
"seq-placement",
"seq-key",
"uplink-data-placement",
"uplink-data-key",
"uplink-chunk-size",
"sc-max-each-post-bytes",
"sc-min-posts-interval-ms",
)
REUSE_ROWS = {
*(f"X{i:02}" for i in range(24, 30)),
*(f"D{i:02}" for i in range(27, 33)),
}
def rows(identifier):
if identifier in {"XHTTP-CFG", "XHTTP-UNIT"}:
return sorted(FIELD_IDS)
if identifier in GATES:
return []
if identifier in SECURITY:
return [f"D{i:02}" for i in range(1, 14)]
variant, test = NATIVE[identifier]
options, version = ALL_VARIANTS[variant]
if version.startswith("outer-"):
return [f"M{i:02}" for i in range(1, 8)]
result = {"X01", "X02", "X03", "X06"}
result |= {f"X{i:02}" for i, name in enumerate(X_NAMES, 1) if name in options}
if "download-settings" in options:
result |= {"D01", "D02", "D03", "D04", "D05", "D06", "D07", "D08"}
if version.endswith("r"):
result |= {"D14", "D15"}
if "reuse-settings" in options or "reuse" in variant:
result |= REUSE_ROWS
if "_mux" in options:
result |= {f"M{i:02}" for i in range(1, 8)}
if options.get("_keepalive"):
result |= {"X29", "D32"}
return sorted(result)
def definitions():
result = []
for identifier in sorted(GATES | set(SECURITY) | set(NATIVE)):
variant, test = NATIVE.get(
identifier, (SECURITY.get(identifier, "memory-or-build"), identifier)
)
options, version = ALL_VARIANTS.get(variant, ({}, variant))
native = identifier not in GATES
kind = native_kind(version) if native else "unit"
if identifier in SECURITY:
kind = "XR" if version == "h3" else "M"
encrypted = not version.endswith("c") and (
not version.startswith("outer-") or version.endswith(("-tls", "-reality"))
)
gap = (
{
"reason": "Xray H3 forwards to the official Mihomo VLESS decoder "
"for packetaddr/sing-mux. mTLS uses the approved unmodified "
"xcaddy/Caddy gateway to one Xray handler.",
"source": "https://github.com/XTLS/Xray-core/tree/v26.3.27/transport/internet/splithttp",
}
if kind == "XR"
else {
"reason": "Official V2Ray supplies HTTP/H2/custom-ED framing; "
"sing-mux is decoded by official Mihomo, not V2Ray CommandMux.",
"source": "https://github.com/MetaCubeX/mihomo/blob/v1.19.31/listener/inbound/vless.go",
}
if kind == "V2"
else None
)
result.append(
dict(
case_id=identifier,
stage="XHTTP",
substage="XHTTP.http3"
if version == "h3"
else "XHTTP.mux"
if "_mux" in options
else "XHTTP.download"
if identifier in SECURITY or "download-settings" in options
else "XHTTP.requests"
if native
else "XHTTP.acceptance",
required=True,
row_ids=rows(identifier),
protocol="vless",
network=variant,
security="tls-or-reality" if encrypted else "plain",
udp_codec="raw/xudp/packetaddr/sing-mux",
field_values=dict(
scope="protocol-consumer", variant=variant, **options
),
outer_family="IPv4/IPv6",
inner_family="IPv4/IPv6/domain",
target_type="isolated synthetic origin"
if native
else "memory-only/build",
upstream_graph="direct/concrete/nested-select"
if "graph" in test
else "per-assertion",
peer_kind=kind,
peer_config=dict(variant=variant, test=test),
gap_source=gap,
expected_observation=OBSERVATIONS.get(
identifier,
SCRIPT_ASSERTIONS if identifier == "XHTTP-SCRIPTS" else [test],
),
required_evidence=["structured-assertions", "command", "cleanup"]
+ (["official-peer-identity", "isolated-origins"] if native else []),
prerequisites=[
"owned-host-only-container-network",
"official-latest-peer",
]
if native
else ["rust-toolchain"],
runner="native-xhttp"
if identifier in NATIVE
else "xhttp-security"
if identifier in SECURITY
else "xhttp-gate",
timeout_seconds=3600
if identifier in GATES or identifier in SECURITY
else 300,
)
)
return result
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,564 @@
"""Isolated xcaddy H3/mTLS -> one native Xray XHTTP handler probe.
Native comparator evidence only: this never claims VCore XHTTP acceptance.
"""
from __future__ import annotations
import contextlib
import hashlib
import ipaddress
import json
import select
import shutil
import socket
import struct
import tempfile
import time
from pathlib import Path
from .builds import CORE_DIR
from .caddy_build import build_caddy
from .container_close_client import exact
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .native_release import PeerArtifact, download_native
from .protocol_containers import ContainerLab, command, listing
from .protocol_fixtures import certificate_chain
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .protocol_vless_security import client_identities
from .protocol_xhttp_peers import client_config, exercise, peer_config, socks
def sanitized_gateway_log(text):
"""Keep native protocol evidence, not HTTP headers, identities or destinations."""
entries = []
events = {
"aborting with incomplete response": "response_canceled",
"strict SNI-Host enforcement": "strict_sni_host",
"admin endpoint disabled": "admin_disabled",
"unable to create folder for config autosave": "autosave_failed",
"unable to get instance ID": "storage_id_failed",
}
for line in text.splitlines():
try:
row = json.loads(line)
except ValueError:
entries.append({"event": "non_json_log", "omitted": True})
continue
message = row.get("msg", "")
entry = {
"level": row.get("level"),
"event": next(
(name for pattern, name in events.items() if pattern in message),
"other_native_event",
),
}
request = row.get("request", {})
if request:
entry.update(
http_version=request.get("proto"),
alpn=request.get("tls", {}).get("proto"),
client_certificate_present=bool(
request.get("tls", {}).get("client_common_name")
),
)
entries.append(entry)
return "".join(json.dumps(entry) + "\n" for entry in entries)
def gateway_config(upstream):
address = ipaddress.ip_address(upstream)
authority = f"[{address}]" if address.version == 6 else str(address)
return {
"admin": {"disabled": True},
"storage": {"module": "file_system", "root": "/data/caddy"},
"logging": {"logs": {"default": {"level": "WARN"}}},
"apps": {
"tls": {
"certificates": {
"load_files": [
{
"certificate": "/data/fixture/cert.pem",
"key": "/data/fixture/key.pem",
}
]
}
},
"http": {
"servers": {
"fixture": {
"listen": [":23000"],
"protocols": ["h3"],
"automatic_https": {"disable": True},
"tls_connection_policies": [
{
"client_authentication": {
"mode": "require_and_verify",
"ca": {
"provider": "file",
"pem_files": ["/data/fixture/root.pem"],
},
}
}
],
"routes": [
{
"handle": [
{
"handler": "reverse_proxy",
"upstreams": [{"dial": f"{authority}:23000"}],
"transport": {
"protocol": "http",
"versions": ["h2c"],
"compression": False,
},
}
]
}
],
}
}
},
},
}
def cases(*, identities_only=False):
for mode in ("packet-up", "stream-up", "stream-one"):
for split in (False, True) if mode != "stream-one" else (False,):
prefix = mode + ("-download" if split else "")
checks = (
("echo",)
if identities_only
else ("echo", "bulk-server-first", "close", "udp")
)
for check in checks:
yield dict(id=f"{prefix}-{check}", mode=mode, split=split, check=check)
if split or identities_only:
for leg in ("upload", "download") if split else ("shared",):
for identity in ("absent", "expired", "wrong-ca"):
yield dict(
id=f"{prefix}-{leg}-{identity}",
mode=mode,
split=split,
check="reject",
leg=leg,
identity=identity,
)
def native_config():
config = peer_config("XR", "unused", "unused")
stream = config["inbounds"][0]["streamSettings"]
stream["security"] = "none"
del stream["tlsSettings"]
# This ONE handler owns the upload and download session table. Caddy does
# not decode XHTTP or VLESS; h2c avoids H1 full-duplex/buffering differences.
return config
def stream_check(proxy, port, origin, *, bulk):
with socket.create_connection((origin, 24000), timeout=15) as control:
control.sendall(b"\x0a" if bulk else b"\x0b")
target = struct.unpack("!H", exact(control, 2))[0]
stream, _ = socks(proxy, port, 1, origin, target)
with stream:
stream.settimeout(15)
if exact(stream, 5) != b"hello":
raise ValueError("missing native server-first data")
size = 10 * 1024 * 1024 if bulk else 4
payload = b"\x5a" * size if bulk else b"ping"
stream.sendall(payload)
observed = hashlib.sha256()
remaining = size
while remaining:
data = exact(stream, min(65536, remaining))
observed.update(data)
remaining -= len(data)
expected = hashlib.sha256(payload).hexdigest()
if observed.hexdigest() != expected:
raise ValueError("gateway data mismatch")
if bulk and exact(stream, 7) != b"trailer":
raise ValueError("missing complete native reply")
stream.settimeout(5)
started = time.monotonic()
stream.shutdown(socket.SHUT_WR)
tail = bytearray()
try:
while data := stream.recv(1024):
tail.extend(data)
if len(tail) > 1024:
raise ValueError("gateway close exceeded response bound")
except ConnectionResetError:
pass
control.settimeout(5)
if exact(control, 2) != b"AD":
raise ValueError("native origin did not finish")
return dict(
bytes_each_way=size,
sha256=expected,
server_first=True,
closed=True,
close_seconds=round(time.monotonic() - started, 3),
tail_after_upload_eof=len(tail),
origin_finished=True,
)
def rejected_identity(proxy, port, origin):
with socket.create_connection((origin, 24000), timeout=5) as control:
control.sendall(b"\x0c")
target = struct.unpack("!H", exact(control, 2))[0]
connected = False
outcome = "closed"
try:
stream, _ = socks(proxy, port, 1, origin, target)
with stream:
stream.sendall(b"synthetic-probe")
connected = exact(stream, 2) == b"ok"
except TimeoutError:
outcome = "timeout" # Never infer rejection from a blackhole.
except (OSError, ValueError):
pass
ready, _, _ = select.select([control], [], [], 0.3)
return dict(
rejected=not connected and not ready and outcome == "closed",
data_exchanged=connected,
origin_accepted=bool(ready),
client_outcome=outcome,
quiet_seconds=0.3,
)
def run(output: Path, *, identities_only=False):
output = output.resolve()
if output.parent != (CORE_DIR / "target/interop/runs").resolve():
raise ValueError("gateway output must be a fresh target/interop/runs child")
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="XHTTP-PEERS",
purpose="XHTTP-xcaddy-H3-mTLS-prerequisite",
vcore_acceptance=False,
identities_only=identities_only,
source=source_identity(),
status="NOT RUN",
phase="build",
cases=[],
peers={},
isolation={},
topology=(
"Mihomo-H3-client -> Caddy-H3-mTLS -> one-Xray-h2c-XHTTP-handler -> origin"
),
)
try:
built = build_caddy(output / "binaries/Caddy")
report["phase"] = "download"
identity = {}
mihomo = PeerArtifact(
download_mihomo(
"linux-arm64",
directory=output / "binaries/M",
identity=identity,
),
identity,
)
xray = download_native(
"XR", output / "binaries/XR", "linux-arm64", defer_version=True
)
artifacts = {"Caddy": built, "M": mihomo, "XR": xray}
report["peers"] = {
kind: artifact.identity for kind, artifact in artifacts.items()
}
lab = ContainerLab(report["isolation"], mtu=1500)
report["phase"] = "start"
with tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary:
root = Path(temporary)
directories = {
role: root / role for role in ("origin", "gateway", "server", "client")
}
for directory in directories.values():
directory.mkdir()
origin_dir, gateway_dir, server_dir, client_dir = directories.values()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
shutil.copyfile(
Path(__file__).with_name("container_quic_observer.py"),
gateway_dir / "observe.py",
)
for role, artifact in (
("gateway", built),
("server", xray),
("client", mihomo),
):
shutil.copyfile(artifact.binary, directories[role] / "peer")
(directories[role] / "peer").chmod(0o755)
_, _, pin = certificate_chain(gateway_dir)
identities = client_identities(gateway_dir)
identities["absent"] = {"certificate": "", "private-key": ""}
second = root / "second-identity"
second.mkdir()
for name in ("root.pem", "root-key.pem"):
shutil.copyfile(gateway_dir / name, second / name)
download_identity = client_identities(second)["valid"]
(server_dir / "config.json").write_text(json.dumps(native_config()))
with contextlib.ExitStack() as stack:
origin = lab.start(
stack,
origin_dir,
"xhttp-gateway-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
server = lab.start(
stack,
server_dir,
"xhttp-gateway-xray",
[
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
],
)
(gateway_dir / "config.json").write_text(
json.dumps(gateway_config(server.ipv4))
)
gateway = lab.start(
stack,
gateway_dir,
"xhttp-gateway-caddy",
[
"env",
"XDG_CONFIG_HOME=/data/config",
"XDG_DATA_HOME=/data/share",
*(["QLOGDIR=/data/qlog"] if identities_only else []),
"/data/fixture/peer",
"run",
"--config",
"/data/fixture/config.json",
],
)
selected = list(cases(identities_only=identities_only))
config = dict(
ipv6=True,
**{"log-level": "warning"},
proxies=[],
listeners=[],
rules=["MATCH,REJECT"],
)
for index, case in enumerate(selected):
candidate = client_config(
gateway.ipv4, pin, "h3", case["mode"], download=case["split"]
)
node, listener = candidate["proxies"][0], candidate["listeners"][0]
node.update(identities["valid"])
node["xhttp-opts"]["host"] = "localhost"
if case["split"]:
node["xhttp-opts"]["download-settings"].update(
download_identity
)
if case["check"] == "reject":
target = (
node
if case["leg"] in ("upload", "shared")
else node["xhttp-opts"]["download-settings"]
)
target.update(identities[case["identity"]])
node["name"] = listener["name"] = listener["proxy"] = case["id"]
listener["port"] = 23100 + index
config["proxies"].append(node)
config["listeners"].append(listener)
(client_dir / "config.json").write_text(json.dumps(config))
client = lab.start(
stack,
client_dir,
"xhttp-gateway-mihomo",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
def preserve_logs():
# Snapshot before deletion, independent of log-follow buffering.
for role, peer in (
("gateway", gateway),
("server", server),
("client", client),
):
(output / f"{role}.log").write_text(
sanitized_gateway_log(command("logs", peer.name))
if role == "gateway"
else redact(command("logs", peer.name))
)
stack.callback(preserve_logs)
for kind, peer in (("Caddy", gateway), ("XR", server), ("M", client)):
version = command(
"exec",
peer.name,
"/data/fixture/peer",
"-v" if kind == "M" else "version",
).strip()
digest = command(
"exec", peer.name, "sha256sum", "/data/fixture/peer"
).split()[0]
if not version or digest != report["peers"][kind]["binary_sha256"]:
raise RuntimeError("container binary identity mismatch")
if kind == "Caddy" and not version.startswith(
report["peers"][kind]["release"] + " "
):
raise RuntimeError(
"container Caddy version differs from latest source build"
)
report["peers"][kind]["version"] = version
validation = run_command(
[
"container",
"exec",
gateway.name,
"env",
"XDG_CONFIG_HOME=/data/config",
"XDG_DATA_HOME=/data/share",
"/data/fixture/peer",
"validate",
"--config",
"/data/fixture/config.json",
],
timeout=30,
limit=65536,
)
(output / "gateway-validation.log").write_text(
redact(validation.stdout.decode(errors="replace"))
)
if validation.returncode or not validation.cleanup:
raise RuntimeError("native Caddy configuration validation failed")
for peer in (origin, server, gateway, client):
peer.release()
origin.wait_tcp(24000)
server.wait_tcp(23000)
client.wait_tcp(23100)
report["phase"] = "traffic"
def trace_snapshot():
return json.loads(
command(
"exec",
gateway.name,
"python",
"-B",
"/data/fixture/observe.py",
)
)
for index, case in enumerate(selected):
result = dict(case, status="FAIL")
before = trace_snapshot() if identities_only else {}
started = time.monotonic()
try:
check = case["check"]
args = (client.ipv4, 23100 + index, origin.ipv4)
observation = (
rejected_identity(*args)
if check == "reject"
else stream_check(*args, bulk=check == "bulk-server-first")
if check in ("bulk-server-first", "close")
else exercise(*args, udp=check == "udp")
)
result.update(observation, status="PASS")
if check == "reject" and not observation["rejected"]:
result.update(
status="FAIL",
failure_kind=(
"TimeoutError"
if observation["client_outcome"] == "timeout"
else "OriginAccepted"
),
)
except (OSError, ValueError) as error:
result["failure_kind"] = type(error).__name__
if identities_only:
after = trace_snapshot()
result["gateway_trace_files"] = len(after)
result["gateway_new_trace_files"] = len(
after.keys() - before.keys()
)
evidence = [
event
for key, events in after.items()
if key not in before
for event in events
]
result["gateway_certificate_errors"] = evidence
if case["check"] == "reject":
expected = {
"absent": "certificate_required",
"expired": "certificate_expired",
"wrong-ca": "unknown_ca",
}[case["identity"]]
result["identity_enforced"] = any(
event["category"] == expected for event in evidence
) and (
result.get("origin_accepted") is False
and result.get("data_exchanged") is False
)
if not result["identity_enforced"]:
result.update(status="FAIL", evidence_missing=True)
result["seconds"] = round(time.monotonic() - started, 3)
report["cases"].append(result)
print(case["id"], result["status"], flush=True)
(output / "result.json").write_text(
json.dumps(report, indent=2) + "\n"
)
for peer in (origin, server, gateway, client):
peer.ensure_alive()
report["status"] = (
"PASS"
if all(case["status"] == "PASS" for case in report["cases"])
else "FAIL"
)
report["phase"] = "complete"
except BaseException as error:
report.update(status="FAIL", failure_kind=type(error).__name__)
raise
finally:
report["source_unchanged"] = report["source"] == source_identity()
report["cleanup"] = all(
peer.get("joined") for peer in report["isolation"].get("peers", [])
)
run_id = report["isolation"].get("run_id")
report["owned_remaining"] = (
[
item["id"]
for item in listing()
if item["configuration"].get("labels", {}).get("vcore-run") == run_id
]
if run_id
else []
)
(output / "result.json").write_text(json.dumps(report, indent=2) + "\n")
return (
0
if report["status"] == "PASS"
and report["source_unchanged"]
and report["cleanup"]
and not report["owned_remaining"]
else 1
)
def main(output, *, identities_only=False):
with exclusive_run():
return run(output, identities_only=identities_only)
@@ -0,0 +1,524 @@
"""XHTTP-PEERS native XHTTP/mux capability probes, not VCore XHTTP acceptance.
Official clients, servers and origins run in owned containers. The host only
drives SOCKS clients and observes the origin. Direct and layered native decoder
topologies have separate results; a layered pass never overwrites a direct fail.
"""
from __future__ import annotations
import contextlib
import json
import shutil
import socket
import struct
import tempfile
import time
from pathlib import Path
from .builds import CORE_DIR
from .container_close_client import exact
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .native_release import PeerArtifact, download_native
from .protocol_containers import ContainerLab, command, listing
from .protocol_fixtures import certificate_chain, certificates
from .protocol_inputs import redact, source_identity
from .protocol_vless_security import client_identities
CLIENT_ID = "07070707-0707-0707-0707-070707070707"
def peer_config(kind, cert, key, *, decoder=None, plain=False):
if kind == "M":
listener = dict(
name="xhttp",
type="vless",
listen="::",
port=23000,
users=[dict(uuid=CLIENT_ID)],
**{"xhttp-config": dict(path="/xhttp", mode="auto")},
)
if plain:
listener["allow-insecure"] = True
else:
listener.update(certificate=cert, **{"private-key": key})
return dict(
ipv6=True,
**{"log-level": "warning"},
listeners=[listener],
rules=["MATCH,DIRECT"],
)
inbound = dict(
listen="::",
port=23000,
protocol="vless",
settings=dict(clients=[dict(id=CLIENT_ID)], decryption="none"),
streamSettings=dict(
network="xhttp",
security="tls",
tlsSettings=dict(
alpn=["h3"], certificates=[dict(certificateFile=cert, keyFile=key)]
),
xhttpSettings=dict(path="/xhttp", mode="auto"),
),
)
if decoder:
# The same Xray XHTTP handler owns both legs. It forwards the resulting
# byte stream to a real Mihomo VLESS decoder, without parsing its bytes.
inbound.update(
protocol="dokodemo-door",
settings=dict(address=decoder, port=23001, network="tcp"),
)
return dict(
log=dict(loglevel="warning"),
inbounds=[inbound],
outbounds=[dict(protocol="freedom")],
)
def client_config(
server, pin, version, mode, *, download=False, mux=None, codec="xudp", plain=False
):
node = dict(
name="probe",
type="vless",
server=server,
port=23000,
uuid=CLIENT_ID,
tls=not plain,
udp=True,
network="xhttp",
servername="localhost",
alpn=["http/1.1" if version == "h1" else version],
**{"packet-encoding": codec, "xhttp-opts": dict(path="/xhttp", mode=mode)},
)
if not plain:
node["fingerprint"] = pin
if download:
node["xhttp-opts"]["download-settings"] = {}
if mux:
node["smux"] = dict(enabled=True, protocol=mux)
return dict(
ipv6=True,
**{"log-level": "warning"},
proxies=[node],
listeners=[
dict(
name="consumer",
type="socks",
listen="::",
port=23002,
udp=True,
proxy="probe",
)
],
rules=["MATCH,REJECT"],
)
def identity_peer_config(kind, cert, key, ca):
config = peer_config(kind, cert, key)
if kind == "M":
config["listeners"][0].update(
{"client-auth-type": "require-and-verify", "client-auth-cert": ca}
)
else:
# Xray's published verify usage adds RootCAs; it is NOT client auth.
# This probe checks whether a client identity is actually enforced,
# without inventing an unsupported client-auth option for that peer.
tls = config["inbounds"][0]["streamSettings"]["tlsSettings"]
tls["certificates"].append(dict(certificateFile=ca, usage="verify"))
return config
def variants():
for mode in ("packet-up", "stream-up", "stream-one"):
yield dict(name=mode, mode=mode)
if mode != "stream-one":
yield dict(name=mode + "-download", mode=mode, download=True)
yield dict(name="packetaddr", mode="packet-up", codec="packetaddr")
for mux in ("h2mux", "smux", "yamux"):
yield dict(name=mux, mode="packet-up", mux=mux)
def socks(proxy, port, command_id, target, target_port):
connection = socket.create_connection((proxy, port), timeout=5)
try:
connection.sendall(b"\x05\x01\x00")
if exact(connection, 2) != b"\x05\x00":
raise ValueError("SOCKS greeting rejected")
connection.sendall(
bytes([5, command_id, 0, 1])
+ socket.inet_aton(target)
+ struct.pack("!H", target_port)
)
reply = exact(connection, 4)
if reply[:3] != b"\x05\x00\x00" or reply[3] != 1:
raise ValueError("SOCKS request rejected")
address = socket.inet_ntoa(exact(connection, 4))
relay_port = struct.unpack("!H", exact(connection, 2))[0]
return connection, (proxy if address == "0.0.0.0" else address, relay_port)
except BaseException:
connection.close()
raise
def exercise(proxy, port, origin, *, udp=False):
"""One small positive data probe; no retry, no full stage coverage claim."""
payload = b"xhttp-native-decoder-probe" * 3
with socket.create_connection((origin, 24000), timeout=5) as control:
control.sendall(b"\x04" if udp else b"\x0d")
target_port = struct.unpack("!H", exact(control, 2))[0]
connection, relay = socks(
proxy,
port,
3 if udp else 1,
"0.0.0.0" if udp else origin,
0 if udp else target_port,
)
with connection:
if not udp:
connection.sendall(payload)
if (
exact(connection, len(payload)) != payload
or exact(control, 1) != b"A"
):
raise ValueError("native TCP mismatch")
else:
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as datagram:
datagram.settimeout(5)
datagram.bind(("0.0.0.0", 0))
address = (
b"\x01"
+ socket.inet_aton(origin)
+ struct.pack("!H", target_port)
)
datagram.sendto(b"\x00\x00\x00" + address + payload, relay)
reply, source = datagram.recvfrom(2048)
if source != relay or reply != b"\x00\x00\x00" + address + payload:
raise ValueError("native UDP mismatch")
observed = exact(control, 4)
length = struct.unpack("!H", observed[:2])[0]
if length != len(payload) or exact(control, length) != payload:
raise ValueError("native origin mismatch")
return dict(bytes=len(payload), exact_echo=True, origin_observed=True)
def run(output: Path, *, identities_only=False):
output = output.resolve()
if output.parent != (CORE_DIR / "target/interop/runs").resolve():
raise ValueError(
"native probe output must be a fresh target/interop/runs child"
)
output.mkdir(parents=True, exist_ok=False)
report = dict(
stage="XHTTP-PEERS",
purpose="XHTTP-H3-client-identity-prerequisite"
if identities_only
else "XHTTP-native-prerequisites",
vcore_acceptance=False,
source=source_identity(),
status="NOT RUN",
cases=[],
peers={},
isolation={},
)
try:
artifacts = {}
for kind in ("M", "XR"):
directory = output / "binaries" / kind
if kind == "M":
identity = {}
artifacts[kind] = PeerArtifact(
download_mihomo(
"linux-arm64", directory=directory, identity=identity
),
identity,
)
else:
artifacts[kind] = download_native(
kind, directory, "linux-arm64", defer_version=True
)
report["peers"][kind] = artifacts[kind].identity
# Go QUIC peers send 1280-byte Initial UDP payloads. A 1280-byte guest
# L3 interface cannot carry those plus IP/UDP headers. This changes only
# these disposable guests, never the shared network or host interface.
lab = ContainerLab(report["isolation"], mtu=1500)
with (
tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary,
contextlib.ExitStack() as stack,
):
root = Path(temporary)
origin_dir = root / "origin"
origin_dir.mkdir()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
origin = lab.start(
stack,
origin_dir,
"xhttp-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
origin.release()
origin.wait_tcp(24000)
decoder_dir = root / "decoder"
decoder_dir.mkdir()
shutil.copy2(artifacts["M"].binary, decoder_dir / "peer")
(decoder_dir / "config.json").write_text(
json.dumps(
dict(
ipv6=True,
**{"log-level": "warning"},
listeners=[
dict(
name="decoder",
type="vless",
listen="::",
port=23001,
users=[dict(uuid=CLIENT_ID)],
**{"allow-insecure": True},
)
],
rules=["MATCH,DIRECT"],
)
)
)
decoder = lab.start(
stack,
decoder_dir,
"xhttp-decoder",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
decoder.release()
decoder.wait_tcp(23001)
groups = (
("M", "h1", True, False),
("M", "h2", True, False),
("M", "h1", False, False),
("M", "h2", False, False),
("XR", "h3", False, False),
("XR", "h3", False, True),
)
if identities_only:
groups = (("M", "h2", False, False), ("XR", "h3", False, False))
for kind, version, plain, layered in groups:
security = "plain" if plain else "tls"
topology = "layered" if layered else "direct"
label = f"{kind}-{version}-{security}-{topology}"
with contextlib.ExitStack() as group:
server_dir, client_dir = (
root / (label + "-server"),
root / (label + "-client"),
)
server_dir.mkdir()
client_dir.mkdir()
def preserve_logs(
server_dir=server_dir, client_dir=client_dir, label=label
):
for role, directory in (
("server", server_dir),
("client", client_dir),
):
log = directory / "peer.log"
if log.exists():
(output / f"{label}-{role}.log").write_text(
redact(log.read_text(errors="replace"))
)
# Runs after peer stop/join, before temporary keys/configs
# are removed. No credentials or configs are copied out.
group.callback(preserve_logs)
shutil.copy2(artifacts[kind].binary, server_dir / "peer")
shutil.copy2(artifacts["M"].binary, client_dir / "peer")
cert, key, pin = (
certificate_chain(server_dir)
if identities_only
else certificates(server_dir)
)
config = peer_config(
kind,
"/data/fixture/" + cert.name,
"/data/fixture/" + key.name,
decoder=decoder.ipv4 if layered else None,
plain=plain,
)
identities = {}
if identities_only:
identities = client_identities(server_dir)
identities["absent"] = {"certificate": "", "private-key": ""}
config = identity_peer_config(
kind,
"/data/fixture/" + cert.name,
"/data/fixture/" + key.name,
"/data/fixture/root.pem",
)
(server_dir / "config.json").write_text(json.dumps(config))
argv = (
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
]
if kind == "M"
else [
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
]
)
server = lab.start(group, server_dir, label + "-server", argv)
version_text = command(
"exec",
server.name,
"/data/fixture/peer",
"-v" if kind == "M" else "version",
).strip()
report["peers"][kind]["version"] = version_text
# One immutable client config gives every variant its own
# listener, node and pool; no API mutation or NAT-key reuse.
clients = dict(
ipv6=True,
**{"log-level": "warning"},
proxies=[],
listeners=[],
rules=["MATCH,REJECT"],
)
selected = list(variants())
if identities_only:
selected = [
dict(name=name, mode="packet-up", download=True)
for name in ("valid", "absent", "expired", "wrong-ca")
]
for index, variant in enumerate(selected):
kwargs = {k: v for k, v in variant.items() if k != "name"}
client = client_config(
server.ipv4, pin, version, plain=plain, **kwargs
)
node, listener = client["proxies"][0], client["listeners"][0]
if identities_only:
node.update(identities["valid"])
node["xhttp-opts"]["download-settings"].update(
identities[variant["name"]]
)
node["name"] = listener["name"] = listener["proxy"] = variant[
"name"
]
listener["port"] = 23100 + index
clients["proxies"].append(node)
clients["listeners"].append(listener)
(client_dir / "config.json").write_text(json.dumps(clients))
client = lab.start(
group,
client_dir,
label + "-client",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
],
)
server.release()
client.release()
if kind == "M":
server.wait_tcp(23000)
client.wait_tcp(23100)
for index, variant in enumerate(selected):
for udp in (False,) if identities_only else (False, True):
business = "udp" if udp else "tcp"
case = dict(
id=f"{label}-{variant['name']}-{business}",
topology="Xray-XHTTP-to-Mihomo-VLESS"
if layered
else "single-native-handler",
status="FAIL",
)
started = time.monotonic()
should_connect = (
not identities_only or variant["name"] == "valid"
)
case["expected_data_connected"] = should_connect
try:
case.update(
exercise(
client.ipv4, 23100 + index, origin.ipv4, udp=udp
),
data_connected=True,
status="PASS" if should_connect else "FAIL",
)
if not should_connect:
case["failure_kind"] = "ClientIdentityNotEnforced"
except (OSError, ValueError) as error:
case["failure_kind"] = type(error).__name__
case["data_connected"] = False
# A rejected exchange alone is only a peer
# prerequisite observation, not an XHTTP negative
# gate proving zero business bytes/resources.
if not should_connect:
case["status"] = "PASS"
case["seconds"] = round(time.monotonic() - started, 3)
report["cases"].append(case)
print(case["id"], case["status"], flush=True)
(output / "result.json").write_text(
json.dumps(report, indent=2) + "\n"
)
server.ensure_alive()
client.ensure_alive()
report["status"] = (
"PASS"
if all(case["status"] == "PASS" for case in report["cases"])
else "FAIL"
)
except BaseException as error:
report["status"] = "FAIL"
report["failure_kind"] = type(error).__name__
raise
finally:
report["source_unchanged"] = report["source"] == source_identity()
report["cleanup"] = all(
peer.get("joined") for peer in report["isolation"].get("peers", [])
)
run_id = report["isolation"].get("run_id")
report["owned_remaining"] = (
[
item["id"]
for item in listing()
if item["configuration"].get("labels", {}).get("vcore-run") == run_id
]
if run_id
else []
)
(output / "result.json").write_text(json.dumps(report, indent=2) + "\n")
return (
0
if report["status"] == "PASS"
and report["source_unchanged"]
and report["cleanup"]
and not report["owned_remaining"]
else 1
)
def main(output, *, identities_only=False):
with exclusive_run():
return run(output, identities_only=identities_only)
@@ -0,0 +1,494 @@
"""XHTTP VCore upload/download security against owned native handlers.
H1/H2 use one Mihomo listener with two ports. H3 uses the approved xcaddy
gateway (two bindings, one Xray h2c handler). Neither topology invents a decoder.
"""
from __future__ import annotations
import contextlib
import copy
import json
import os
import shutil
import ssl
import tempfile
from pathlib import Path
from .builds import CORE_DIR
from .caddy_build import build_caddy
from .mihomo_isolation import exclusive_run
from .mihomo_release import download_mihomo
from .native_release import download_native
from .protocol_containers import ContainerLab, command, listing
from .protocol_evidence import read_events
from .protocol_fixtures import certificate_chain
from .protocol_inputs import redact, source_identity
from .protocol_peers import run_command
from .protocol_vless_security import client_identities
from .protocol_xhttp_fields import events_pass
from .protocol_xhttp_gateway import gateway_config, sanitized_gateway_log
from .protocol_xhttp_peers import CLIENT_ID, peer_config
def security_cases(base, identities, download_identity, server_ipv6):
"""Frozen cases, selected before observing results. No retry or skip-on-error."""
for mode in ("packet-up", "stream-up", "stream-one"):
node = copy.deepcopy(base)
options = node["xhttp-opts"]
options["mode"] = mode
if mode == "packet-up":
options["sc-min-posts-interval-ms"] = "1"
yield f"{mode}-shared", node, False, None
for identity in ("absent", "expired", "wrong-ca"):
invalid = copy.deepcopy(node)
invalid.update(identities[identity])
if identity == "absent":
del invalid["certificate"], invalid["private-key"]
yield f"{mode}-shared-{identity}", invalid, True, identity
if mode == "stream-one":
continue
options["download-settings"] = {}
yield f"{mode}-inherit", copy.deepcopy(node), False, None
download = options["download-settings"]
download.update(download_identity)
download.update(
server=server_ipv6,
port=23002,
tls=True,
servername="localhost",
host="localhost",
path="/xhttp/",
headers={"X-Download": "fixture"},
**{"reuse-settings": {}, "fingerprint": "", "name-cert-verify": ""},
)
yield f"{mode}-replace-dualstack-port", copy.deepcopy(node), False, None
download["server"] = "peer.fixture.test"
yield f"{mode}-download-domain", copy.deepcopy(node), False, None
node["servername"] = "sni.fixture.test"
node["name-cert-verify"] = "localhost"
if base["alpn"] != ["h3"]:
# One native handler accepts both ALPNs; the download has its own policy.
download["alpn"] = ["http/1.1"] if base["alpn"] == ["h2"] else ["h2"]
yield f"{mode}-name-clear-alpn", copy.deepcopy(node), False, None
node["servername"] = "localhost"
del node["name-cert-verify"]
for leg in ("upload", "download"):
for identity in ("absent", "expired", "wrong-ca"):
invalid = copy.deepcopy(node)
target = (
invalid
if leg == "upload"
else invalid["xhttp-opts"]["download-settings"]
)
target.update(identities[identity])
if identity == "absent" and leg == "upload":
del target["certificate"], target["private-key"]
yield f"{mode}-{leg}-{identity}", invalid, True, identity
for key, value in (
("fingerprint", "00" * 32),
("name-cert-verify", "wrong.fixture.test"),
):
invalid = copy.deepcopy(node)
invalid["xhttp-opts"]["download-settings"][key] = value
yield f"{mode}-download-wrong-{key}", invalid, True, None
invalid = copy.deepcopy(node)
invalid["skip-cert-verify"] = True
invalid["xhttp-opts"]["download-settings"].update(
{"skip-cert-verify": False, "servername": "wrong.fixture.test"}
)
yield f"{mode}-download-false-restores-verify", invalid, True, None
def run(output: Path, versions: list[str], *, supplied: dict | None = None):
if (
not versions
or len(versions) != len(set(versions))
or not set(versions) <= {"h1", "h2", "h3"}
):
raise ValueError("select unique H1/H2/H3 security versions")
if (
not output.is_relative_to(CORE_DIR / "target/interop/runs")
or output == CORE_DIR / "target/interop/runs"
):
raise ValueError("security output must be a fresh target/interop/runs child")
output.mkdir(exist_ok=False)
report = dict(
stage="XHTTP",
scope="security-development-only",
source=source_identity(),
status="NOT RUN",
cases=[],
peers={},
isolation={},
)
try:
artifacts = {}
if supplied is not None:
for kind in ({"M"} if any(v != "h3" for v in versions) else set()) | (
{"XR", "Caddy"} if "h3" in versions else set()
):
artifacts[kind], report["peers"][kind] = supplied[kind]
elif any(v != "h3" for v in versions):
identity = {}
artifacts["M"] = download_mihomo(
"linux-arm64", directory=output / "binaries/M", identity=identity
)
report["peers"]["M"] = identity
if "h3" in versions and supplied is None:
for kind, artifact in (
("Caddy", build_caddy(output / "binaries/Caddy")),
(
"XR",
download_native(
"XR", output / "binaries/XR", "linux-arm64", defer_version=True
),
),
):
artifacts[kind] = artifact.binary
report["peers"][kind] = artifact.identity
lab = ContainerLab(report["isolation"], mtu=1500)
build = run_command(
[
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"xhttp_native",
"--no-run",
],
timeout=180,
)
(output / "build.log").write_text(redact(build.stdout.decode(errors="replace")))
if build.returncode or not build.cleanup:
raise RuntimeError("native security driver build failed")
groups = [
(version, negative)
for version in versions
for negative in ((False, True) if version == "h3" else (None,))
]
for version, negative in groups:
with (
tempfile.TemporaryDirectory(prefix="private-", dir=output) as temporary,
contextlib.ExitStack() as stack,
):
root = Path(temporary)
directories = {
role: root / role for role in ("origin", "server", "gateway")
}
for directory in directories.values():
directory.mkdir()
origin_dir, server_dir, gateway_dir = directories.values()
shutil.copyfile(
Path(__file__).with_name("container_udp_origin.py"),
origin_dir / "origin.py",
)
origin = lab.start(
stack,
origin_dir,
version + "-origin",
[
"env",
"VCORE_ISOLATED_ORIGIN=1",
"python",
"-B",
"/data/fixture/origin.py",
],
)
cert_dir = gateway_dir if version == "h3" else server_dir
cert, key, pin = certificate_chain(cert_dir)
identities = client_identities(cert_dir)
identities["absent"] = {"certificate": "", "private-key": ""}
second = root / "second"
second.mkdir()
for filename in ("root.pem", "root-key.pem"):
shutil.copyfile(cert_dir / filename, second / filename)
download_identity = client_identities(second)["valid"]
kind = "XR" if version == "h3" else "M"
shutil.copy2(artifacts[kind], server_dir / "peer")
config = peer_config(
kind, f"/data/fixture/{cert.name}", f"/data/fixture/{key.name}"
)
if kind == "M":
config["hosts"] = {"vcore-fixture.test": origin.ipv4}
config["listeners"][0].update(
{
"port": "23000,23002",
"client-auth-type": "require-and-verify",
"client-auth-cert": "/data/fixture/root.pem",
}
)
else:
stream = config["inbounds"][0]["streamSettings"]
stream["security"] = "none"
del stream["tlsSettings"]
config["dns"] = {"hosts": {"vcore-fixture.test": origin.ipv4}}
(server_dir / "config.json").write_text(json.dumps(config))
server = lab.start(
stack,
server_dir,
version + "-server",
[
"/data/fixture/peer",
"-d",
"/data",
"-f",
"/data/fixture/config.json",
]
if kind == "M"
else [
"/data/fixture/peer",
"run",
"-c",
"/data/fixture/config.json",
],
)
peers = [(kind, server)]
gateway = None
edge = server
if version == "h3":
shutil.copy2(artifacts["Caddy"], gateway_dir / "peer")
shutil.copyfile(
Path(__file__).with_name("container_quic_observer.py"),
gateway_dir / "observe.py",
)
gateway = lab.start(
stack,
gateway_dir,
version + "-gateway",
[
"env",
"XDG_CONFIG_HOME=/data/config",
"XDG_DATA_HOME=/data/share",
*(["QLOGDIR=/data/qlog"] if negative else []),
"/data/fixture/peer",
"run",
"--config",
"/data/fixture/config.json",
],
)
conf = gateway_config(server.ipv4)
conf["apps"]["http"]["servers"]["fixture"]["listen"].append(
":23002"
)
(gateway_dir / "config.json").write_text(json.dumps(conf))
peers.append(("Caddy", gateway))
edge = gateway
for implementation, peer in peers:
identity = report["peers"][implementation]
actual = command(
"exec",
peer.name,
"/data/fixture/peer",
"-v" if implementation == "M" else "version",
).strip()
digest = command(
"exec", peer.name, "sha256sum", "/data/fixture/peer"
).split()[0]
if not actual or digest != identity["binary_sha256"]:
raise RuntimeError("native security binary identity mismatch")
identity["version"] = actual
def preserve_logs(peers=peers, version=version, negative=negative):
for implementation, peer in peers:
text = command("logs", peer.name)[-65536:]
(
output / f"{version}-{negative}-{implementation}.log"
).write_text(
sanitized_gateway_log(text)
if implementation == "Caddy"
else redact(text)
)
stack.callback(preserve_logs)
origin.release()
server.release()
if gateway:
gateway.release()
origin.wait_tcp(24000)
server.wait_tcp(23000)
base = dict(
name="peer",
type="vless",
server=edge.ipv4,
port=23000,
uuid=CLIENT_ID,
udp=True,
tls=True,
servername="localhost",
fingerprint=pin,
alpn=["http/1.1" if version == "h1" else version],
network="xhttp",
**identities["valid"],
**{
"xhttp-opts": {
"path": "/xhttp",
"host": "localhost",
"headers": {"X-Upload": "fixture"},
}
},
)
selected = [
case
for case in security_cases(
base, identities, download_identity, edge.ipv6
)
if negative is None or case[2] == negative
]
def trace_snapshot(gateway=gateway, negative=negative):
return (
json.loads(
command(
"exec",
gateway.name,
"python",
"-B",
"/data/fixture/observe.py",
)
)
if gateway and negative
else {}
)
for name, node, reject, expected_identity in selected:
case_id = f"{version}-{name}"
print(case_id, flush=True)
fixture = root / "input.json"
fixture.write_text(
json.dumps(
dict(
isolation="containers",
node=node,
reject=reject,
server_ipv4=edge.ipv4,
origin_control=f"{origin.ipv4}:24000",
origin_ipv4=origin.ipv4,
origin_ipv6=origin.ipv6,
root_der=list(
ssl.PEM_cert_to_DER_cert(
(cert_dir / "root.pem").read_text()
)
),
)
)
)
events = output / f"{case_id}-events.jsonl"
assertion = "security::native_xhttp_security"
cmd = [
"cargo",
"test",
"--locked",
"--all-features",
"--test",
"xhttp_native",
assertion,
"--",
"--exact",
"--ignored",
"--nocapture",
]
before = trace_snapshot()
result = run_command(
cmd,
timeout=140,
env=dict(
os.environ,
VCORE_XHTTP_INPUT=str(fixture),
VCORE_CASE_EVENTS=str(events),
),
)
(output / f"{case_id}.log").write_text(
redact(result.stdout.decode(errors="replace"))
)
observation = read_events(events) if events.exists() else []
passed = (
result.returncode == 0
and result.cleanup
and events_pass(observation, assertion)
)
row = dict(
case_id=case_id,
status="PASS" if passed else "FAIL",
command=cmd,
exit_code=result.returncode,
seconds=result.seconds,
command_cleanup=result.cleanup,
topology="Caddy H3 -> one Xray h2c handler"
if gateway
else "one Mihomo handler, two ports",
)
if gateway and negative:
after = trace_snapshot()
# QUIC trace files contain only lifecycle records at this
# level; retain categories, never identities or packets.
evidence = [
event
for key, records in after.items()
if key not in before
for event in records
]
row["certificate_errors"] = evidence
if expected_identity:
expected = {
"absent": "certificate_required",
"expired": "certificate_expired",
"wrong-ca": "unknown_ca",
}[expected_identity]
row["identity_enforced"] = any(
e["category"] == expected for e in evidence
)
if not row["identity_enforced"]:
row["status"] = "FAIL"
report["cases"].append(row)
print(case_id, row["status"], flush=True)
if row["status"] != "PASS":
raise RuntimeError("native security case failed")
for _, peer in peers:
peer.ensure_alive()
report["status"] = "PASS"
except (OSError, ValueError, RuntimeError) as error:
report.update(
status="FAIL",
reason=str(error)
if isinstance(error, RuntimeError)
else type(error).__name__,
)
finally:
report["cleanup"] = all(
p.get("joined") for p in report["isolation"].get("peers", [])
)
report["source_unchanged"] = report["source"] == source_identity()
run_id = report["isolation"].get("run_id")
report["owned_remaining"] = (
[
p["id"]
for p in listing()
if p["configuration"].get("labels", {}).get("vcore-run") == run_id
]
if run_id
else []
)
if (
not report["cleanup"]
or not report["source_unchanged"]
or report["owned_remaining"]
):
report["status"] = "FAIL"
(output / "xhttp-security-results.json").write_text(
json.dumps(report, indent=2) + "\n"
)
return 0 if report["status"] == "PASS" else 1
if __name__ == "__main__":
import argparse
import sys
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("output", type=Path)
parser.add_argument("versions", nargs="+", choices=("h1", "h2", "h3"))
args = parser.parse_args()
with exclusive_run():
sys.exit(run(args.output.resolve(), args.versions))
@@ -0,0 +1,186 @@
"""Bounded, offline TLS ClientHello inspection, not a TLS implementation."""
from __future__ import annotations
import base64
import hashlib
import io
from .container_tls_observer import capture_client_hello
class Cursor:
def __init__(self, data: bytes):
self.data, self.at = data, 0
def take(self, size: int) -> bytes:
if size < 0 or self.at + size > len(self.data):
raise ValueError("truncated ClientHello")
result = self.data[self.at : self.at + size]
self.at += size
return result
def number(self, size: int) -> int:
return int.from_bytes(self.take(size), "big")
def vector(self, size: int) -> bytes:
return self.take(self.number(size))
def done(self):
if self.at != len(self.data):
raise ValueError("unparsed ClientHello data")
def _codes(data: bytes) -> list[int]:
if len(data) % 2:
raise ValueError("odd TLS uint16 vector")
return [int.from_bytes(data[i : i + 2], "big") for i in range(0, len(data), 2)]
def is_grease(value: int) -> bool:
return value & 0x0F0F == 0x0A0A and value >> 8 == value & 0xFF
def _shape(hello: dict, shuffle_extensions: bool) -> dict:
def normalize(value, key=""):
if isinstance(value, dict):
return {k: normalize(v, k) for k, v in value.items() if k != "raw_sha256"}
if isinstance(value, list):
return [normalize(v, key) for v in value]
if key in {"ciphers", "values", "type", "group"} and isinstance(value, int):
return "GREASE" if is_grease(value) else value
return value
result = normalize(hello)
groups = next((e["values"] for e in hello["extensions"] if e["type"] == 10), [])
shares = next((e["shares"] for e in hello["extensions"] if e["type"] == 51), [])
result["grease_share_group_links"] = [
[i for i, group in enumerate(groups) if group == share["group"]]
for share in shares
if is_grease(share["group"])
]
if shuffle_extensions:
# Preserve GREASE/padding/PSK positions, sort ONLY the shuffled slots.
rows = result["extensions"]
slots = [
i for i, row in enumerate(rows) if row["type"] not in {"GREASE", 21, 41}
]
ordered = sorted((rows[i] for i in slots), key=lambda row: row["type"])
for index, row in zip(slots, ordered, strict=True):
rows[index] = row
return result
def compare_client_hellos(
expected: bytes, observed: bytes, *, shuffle_extensions: bool = False
) -> list[str]:
"""Return differing structural fields; no blanket JA3/ECH/PSK exemption.
Random values are normalized, not their widths or positions. A caller may
separately classify an observed ECH/padding length variant using its pinned
template; it is intentionally still reported here.
"""
left = _shape(parse_client_hello(expected), shuffle_extensions)
right = _shape(parse_client_hello(observed), shuffle_extensions)
return [
key
for key in sorted(left.keys() | right.keys())
if left.get(key) != right.get(key)
]
def validate_capture(record: dict) -> dict:
"""Recompute the retained observation from raw records, not PASS text."""
if any(
len(record[key]) > 256 * 1024 for key in ("client_hello_b64", "records_b64")
):
raise ValueError("capture size limit")
wire = base64.b64decode(record["records_b64"], validate=True)
raw = base64.b64decode(record["client_hello_b64"], validate=True)
records, handshake, layout = capture_client_hello(io.BytesIO(wire).read)
parsed = parse_client_hello(raw)
if records != wire or handshake != raw or layout != record["records"]:
raise ValueError("TLS records differ from retained ClientHello")
if parsed != record["parsed"]:
raise ValueError("ClientHello evidence differs from raw capture")
return parsed
def parse_client_hello(data: bytes) -> dict:
"""Observe an entire handshake message, including its four-byte envelope."""
if len(data) > 65536:
raise ValueError("ClientHello size limit")
cursor = Cursor(data)
if cursor.number(1) != 1 or cursor.number(3) != len(data) - 4:
raise ValueError("invalid ClientHello envelope")
result = {"bytes": len(data), "legacy_version": cursor.number(2)}
cursor.take(32)
result["session_id_bytes"] = len(cursor.vector(1))
result["ciphers"] = _codes(cursor.vector(2))
result["compression_methods"] = list(cursor.vector(1))
extensions = Cursor(cursor.vector(2))
cursor.done()
result["extensions"] = []
seen = set()
while extensions.at < len(extensions.data):
kind, payload = extensions.number(2), extensions.vector(2)
if kind in seen or len(seen) >= 128:
raise ValueError("duplicate or excessive ClientHello extensions")
seen.add(kind)
row = {"type": kind, "bytes": len(payload)}
value = Cursor(payload)
if kind in (10, 13, 34, 50):
row["values"] = _codes(value.vector(2))
elif kind in (27, 43):
row["values"] = _codes(value.vector(1))
elif kind == 0:
names = Cursor(value.vector(2))
row["names"] = []
while names.at < len(names.data):
row["names"].append(
{"type": names.number(1), "name": names.vector(2).decode("ascii")}
)
elif kind in (16, 17513, 17613):
protocols = Cursor(value.vector(2))
row["protocols"] = []
while protocols.at < len(protocols.data):
row["protocols"].append(protocols.vector(1).decode("ascii"))
elif kind == 51:
shares = Cursor(value.vector(2))
row["shares"] = []
while shares.at < len(shares.data):
group, key = shares.number(2), shares.vector(2)
share = {"group": group, "bytes": len(key)}
if is_grease(group):
share["grease_data"] = key.hex()
elif group in (23, 24, 25) and key:
share["point_format"] = key[0]
row["shares"].append(share)
elif kind == 65037:
row["hello_type"] = value.number(1)
if row["hello_type"] != 0:
raise ValueError("expected outer ECH structure")
row["kdf"], row["aead"] = value.number(2), value.number(2)
value.take(1) # Random GREASE config_id remains in the raw evidence.
row["enc_bytes"] = len(value.vector(2))
row["payload_bytes"] = len(value.vector(2))
elif kind == 35:
row["ticket_bytes"] = len(value.take(len(payload)))
elif kind == 41:
identities = Cursor(value.vector(2))
row["identities"] = []
while identities.at < len(identities.data):
row["identities"].append(len(identities.vector(2)))
identities.take(4) # Obfuscated age is time-dependent.
binders = Cursor(value.vector(2))
row["binders"] = []
while binders.at < len(binders.data):
row["binders"].append(len(binders.vector(1)))
if not row["identities"] or len(row["identities"]) != len(row["binders"]):
raise ValueError("invalid PSK identity/binder counts")
else:
row["payload_hex"] = value.take(len(payload)).hex()
value.done()
result["extensions"].append(row)
result["raw_sha256"] = hashlib.sha256(data).hexdigest()
return result
+6 -7
View File
@@ -20,8 +20,6 @@ import time
from pathlib import Path
from urllib.parse import urlparse
ROOT = Path(__file__).resolve().parents[3]
XRAY_SOURCE = ROOT / "references" / "Xray-core"
PACKAGE_NAME = "VCore.UwpDemo.Dev"
SOCKS_PORT = 19080
TUN_IPV4 = "192.168.8.1"
@@ -328,11 +326,12 @@ rules:
"""
def run_demo(source_config: Path | None = None) -> None:
def run_demo(source_config: Path, *, xray_source: Path) -> None:
if os.name != "nt":
raise RuntimeError("This demo requires Windows")
source_config = source_config or Path.home() / "lib" / "xray" / "config.json"
if not source_config.is_file() or not (XRAY_SOURCE / "go.mod").is_file():
source_config = source_config.resolve()
xray_source = xray_source.resolve()
if not source_config.is_file() or not (xray_source / "go.mod").is_file():
raise RuntimeError("Xray source checkout or source config is missing")
package = package_info()
@@ -356,13 +355,13 @@ def run_demo(source_config: Path | None = None) -> None:
str(xray_exe),
"./main",
],
cwd=XRAY_SOURCE,
cwd=xray_source,
env=env,
check=True,
)
revision = subprocess.run(
["git", "rev-parse", "--short", "HEAD"],
cwd=XRAY_SOURCE,
cwd=xray_source,
check=True,
capture_output=True,
text=True,