feat: restore existing self-host realtime connections through normalized legacy gateway URLs and an authenticated API-origin WebSocket proxy, preserve ticket-bound client IP limits and direct retry fallback, and remember workspace mailbox tag filters across navigation with deterministic default account grouping

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Matthew Meszaros
2026-10-08 06:57:48 +00:00
co-authored by Devin AI
parent e45cac4a08
commit b4ef37fe82
28 changed files with 619 additions and 43 deletions
+2
View File
@@ -532,6 +532,8 @@ A completed sign-in counts as a confirmation for the first five minutes of the s
`GET /auth/config` also carries `websocket_url` and `app_url` (both strings, each omitted when the instance has none). They are the realtime gateway a developer client connects to and the dashboard origin a client sends someone to. Both are served here because on a self-hosted instance the host layout is whatever the operator chose, and there is no other way to discover it: the [CLI](/api/cli/) reads them for `warmbly events tail` and `warmbly browse`.
`POST /getaway` is session-only and mints a ten-minute WebSocket ticket. Its `url` remains the direct gateway; the optional `proxy_path` is resolved against `/v1` for browser connections. `GET /realtime/socket/websocket` upgrades a connection using that ticket, preserving the configured realtime service's origin checks and connection limits. It accepts no ordinary access tokens, API keys or OAuth tokens. See [Realtime WebSocket](/api/realtime/).
Alongside them, `api_url` is this API's own public base (for a copyable example that names the right server), and `brand` is who the deployment says it is: `name`, and `website_url`, `website_label`, `terms_url`, `privacy_url` and `support_email`, each omitted when unset. On a self-hosted instance that configured no `EMAIL_BRAND_*` only `name` is present, and a client should render no link at all rather than substituting one of its own. See [configuration](/development/configuration/).
`GET /auth/config` also carries `gmail_oauth_connect` (boolean): whether a new Gmail mailbox may be connected with Google sign-in. Both `BOX_GOOGLE_CLIENT_ID` and `BOX_GOOGLE_CLIENT_SECRET` are required. With them set, the capability defaults to `true`; explicit `BOX_GOOGLE_OAUTH_CONNECT=false` or an invalid value disables it. When `false`, offer the app-password route (`POST /emails/onboarding/smtp-imap` against Gmail's servers) rather than start an OAuth round trip that returns `mailbox_gmail_oauth_disabled`. The dashboard additionally requires its deployment-local [frontend opt-in](/development/configuration/#google-sign-in-per-dashboard-deployment) to show Google sign-in. Existing Google sign-in mailboxes are unaffected either way.
+2
View File
@@ -27,6 +27,8 @@ Three token types are accepted:
Send API keys and OAuth access tokens in the `x-warmbly-token` header. Putting one in the URL as `token` still works but is deprecated: URLs end up in proxy and access logs, and these credentials outlive any session, so it will stop working in a later API version. The 10-minute connection token belongs in the query string. Browsers cannot set headers on a WebSocket, so connect with an API key or OAuth token from a server-side client.
`POST /v1/getaway` continues returning the direct gateway `url` and `expires_in`. It also returns `proxy_path` when a gateway is configured. The dashboard resolves that path against its versioned API base and uses the API origin for its WebSocket, so a self-host does not depend on realtime CSP discovery succeeding during dashboard startup. Failed connection attempts alternate with the direct gateway for older proxy configurations. The proxy accepts only short-lived WebSocket tickets, not access tokens, API keys or OAuth credentials; developer clients continue connecting directly. It preserves the dashboard Origin and realtime authorization and connection limits. The backend signs a ticket-bound client-address proof so per-IP limits do not treat every proxied user as the same client.
A credential stops working on the socket as soon as it does on the REST API: a revoked key or token, a holder banned from signing in, or a suspended OAuth app is refused at the handshake. Signing out, signing out everywhere, changing or resetting a password, and an account ban also close every socket that user has open; the dashboard reconnects on its own with a fresh connection token if its session is still valid.
In short, both developer auth methods work here: an API key with `REALTIME_SUBSCRIBE`, or an OAuth access token with the realtime scope. They authenticate through the same gate, so everything below applies to either.
@@ -337,6 +337,14 @@ A worker's command topic is named after the node id issued when it joined, so th
Migrations are embedded in the backend binary and applied on boot. There is no separate migration step, and a standalone `/app/migrate` binary ships in the image for the cases where you want one.
## Realtime upgrade compatibility
Existing `WEBSOCKET_URL` values ending at the host, `/socket`, or `/socket/websocket` remain supported, including custom path prefixes, ports and query parameters. HTTP(S) forms are converted to WS(S). Both discovery and ticket responses use the same normalization.
The dashboard can connect through `/v1/realtime/socket/websocket` on the API origin. No new environment variable, CSP allowlist entry or data migration is required. The API forwards only to the configured realtime gateway and keeps the browser Origin intact. Legacy Docker installs advertising a loopback gateway can fall back to the existing `realtime` service on the same port when the API container cannot reach its own loopback address.
Upgrade the backend, realtime and dashboard images together to the same release; the realtime update preserves per-client IP limits for API-proxied connections. Existing direct clients and older dashboard/backend combinations retain the direct URL. A custom reverse proxy should allow WebSocket upgrades on the API route as well as the direct realtime route; direct connections remain a retry fallback. Keep `CHECK_ORIGIN_HOSTS` and the shared `AUTH_SECRET`/`JWT_SECRET` consistent. A retry label does not prove recovery: **Live** means the socket is open, while **Reconnecting** means it is still retrying.
## Cache
| Variable | What it does | Default | Restart needed |
+3 -1
View File
@@ -11,7 +11,9 @@ New mailboxes default to diagnostic participation **off**. The organization-auth
## Filtering accounts
Use the **Tags** dropdown above the mailbox list to show accounts with a particular tag. The selection is kept in the page URL, so refreshing the page or opening a bookmark keeps the filter. Choose **All accounts** to clear it. A tag that no longer exists in the current workspace falls back to **All accounts**.
Use the **Tags** dropdown above the mailbox list to show accounts with a particular tag. The selection is kept in the page URL and remembered for your user and workspace in the browser tab, so refreshing, opening a bookmark, or visiting another page and returning keeps the filter. Choose **All accounts** to clear it. A tag that no longer exists in the current workspace falls back to **All accounts**.
Without an explicit column sort, **All accounts** groups the loaded mailboxes by workspace tag order, then by email address, with untagged accounts last. A mailbox with several tags belongs to its first tag in that order and appears only once. Selecting a column sort takes priority over grouping.
## Connecting an account
+1 -4
View File
@@ -128,10 +128,7 @@ func (h *Handler) AuthConfig(c *gin.Context) {
providers := h.AuthService.FederatedProviders()
registration := h.AuthService.RegistrationMode(c.Request.Context())
websocketURL := config.NormalizeWebsocketURL(h.WebsocketURI)
if websocketURL == "" {
websocketURL = config.WebsocketURL()
}
websocketURL := h.websocketEndpoint()
c.JSON(http.StatusOK, DeploymentAuthConfig{
Captcha: config.CaptchaProvider() != "none",
+101 -4
View File
@@ -1,13 +1,21 @@
package handler
import (
"context"
"net"
"net/http"
"net/http/httputil"
"net/url"
"os"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
"github.com/warmbly/warmbly/internal/api/middleware"
"github.com/warmbly/warmbly/internal/app/socket"
"github.com/warmbly/warmbly/internal/app/token"
"github.com/warmbly/warmbly/internal/config"
"github.com/warmbly/warmbly/internal/errx"
)
@@ -26,8 +34,9 @@ func (h *Handler) GenerateWebsocket(c *gin.Context) {
}
wsURL := token
if h.WebsocketURI != "" {
u, err := url.Parse(h.WebsocketURI)
endpoint := h.websocketEndpoint()
if endpoint != "" {
u, err := url.Parse(endpoint)
if err != nil {
errx.Handle(c, errx.InternalError())
return
@@ -39,8 +48,96 @@ func (h *Handler) GenerateWebsocket(c *gin.Context) {
wsURL = u.String()
}
c.JSON(http.StatusOK, gin.H{
response := gin.H{
"url": wsURL,
"expires_in": socket.SocketTTL.Seconds(),
})
}
if endpoint != "" {
response["proxy_path"] = "/realtime/socket/websocket?token=" + url.QueryEscape(token)
}
c.JSON(http.StatusOK, response)
}
func (h *Handler) websocketEndpoint() string {
if h.WebsocketURI != "" {
return config.NormalizeWebsocketURL(h.WebsocketURI)
}
return config.WebsocketURL()
}
// Browser connections use the API origin already permitted by the dashboard CSP.
func (h *Handler) ProxyWebsocket(c *gin.Context) {
endpoint := h.websocketEndpoint()
if endpoint == "" {
c.Status(http.StatusServiceUnavailable)
return
}
if !strings.EqualFold(c.GetHeader("Upgrade"), "websocket") {
c.Status(http.StatusBadRequest)
return
}
claims, xerr := h.TokenService.VerifyTokenFor(token.PurposeWebSocket, c.Query("token"))
if xerr != nil {
c.Status(http.StatusUnauthorized)
return
}
proof, err := h.TokenService.GenerateWebsocketProxyProof(claims, c.ClientIP())
if err != nil {
c.Status(http.StatusInternalServerError)
return
}
target, err := url.Parse(endpoint)
if err != nil {
c.Status(http.StatusServiceUnavailable)
return
}
target.Scheme = strings.Replace(target.Scheme, "ws", "http", 1)
proxy := &httputil.ReverseProxy{
Transport: websocketProxyTransport(target),
Rewrite: func(request *httputil.ProxyRequest) {
request.SetURL(target)
request.Out.URL.Path = target.Path
request.Out.URL.RawPath = target.RawPath
query := target.Query()
query.Set("token", c.Query("token"))
query.Set("vsn", c.DefaultQuery("vsn", "1.0.0"))
request.Out.URL.RawQuery = query.Encode()
request.Out.Header.Del("Authorization")
request.Out.Header.Del("Cookie")
request.Out.Header.Del("X-Warmbly-Token")
request.Out.Header.Set("X-Warmbly-Proxy-Proof", proof)
},
ErrorHandler: func(w http.ResponseWriter, _ *http.Request, _ error) {
w.WriteHeader(http.StatusBadGateway)
},
}
proxy.ServeHTTP(c.Writer, c.Request)
}
func websocketProxyTransport(target *url.URL) http.RoundTripper {
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.ResponseHeaderTimeout = 10 * time.Second
// Old compose files advertise localhost to browsers, not to the API container.
host := target.Hostname()
ip := net.ParseIP(host)
if host == "localhost" || (ip != nil && ip.IsLoopback()) {
if _, err := os.Stat("/.dockerenv"); err == nil {
dialer := &net.Dialer{Timeout: 5 * time.Second, KeepAlive: 30 * time.Second}
transport.DialContext = func(ctx context.Context, network, address string) (net.Conn, error) {
connection, err := dialer.DialContext(ctx, network, address)
if err == nil {
return connection, nil
}
port := target.Port()
if port == "" {
port = "80"
if target.Scheme == "https" {
port = "443"
}
}
return dialer.DialContext(ctx, network, net.JoinHostPort("realtime", port))
}
}
}
return transport
}
+124
View File
@@ -0,0 +1,124 @@
package handler
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
"github.com/gorilla/websocket"
"github.com/warmbly/warmbly/internal/app/token"
"github.com/warmbly/warmbly/internal/errx"
)
type socketTicketStub struct{ ticket string }
func (s socketTicketStub) GenerateWebsocketToken(context.Context, uuid.UUID) (string, *errx.Error) {
return s.ticket, nil
}
func TestSocketTicketNormalizesLegacyURL(t *testing.T) {
gin.SetMode(gin.TestMode)
for _, endpoint := range []string{"wss://realtime.test", "wss://realtime.test/socket", "wss://realtime.test/socket/websocket/"} {
h := &Handler{WebsocketURI: endpoint, SocketService: socketTicketStub{ticket: "short-lived"}}
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/getaway", nil)
c.Set("user_id", uuid.New().String())
h.GenerateWebsocket(c)
var response map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &response); err != nil {
t.Fatal(err)
}
if response["url"] != "wss://realtime.test/socket/websocket?token=short-lived" {
t.Fatalf("legacy response: %s", w.Body)
}
if response["proxy_path"] != "/realtime/socket/websocket?token=short-lived" {
t.Fatalf("missing API-origin path: %s", w.Body)
}
}
}
func TestSocketProxyPreservesHandshakeAndRejectsOtherCredentials(t *testing.T) {
gin.SetMode(gin.TestMode)
service := token.NewService(nil, nil, nil, nil, "test-secret-at-least-32-characters-long")
userID, ticketID := uuid.New(), uuid.New()
now := time.Now()
ticket, err := service.GenerateTokenFor(token.PurposeWebSocket, userID, ticketID, "", "nonce", now, now.Add(time.Minute))
if err != nil {
t.Fatal(err)
}
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/prefix/socket/websocket" || r.URL.Query().Get("vsn") != "1.0.0" || r.URL.Query().Get("tenant") != "legacy" || r.URL.Query().Get("target") != "" {
t.Errorf("wrong upstream route: %s", r.URL.Path)
}
if r.Header.Get("Authorization") != "" || r.Header.Get("Cookie") != "" || r.Header.Get("X-Warmbly-Token") != "" {
t.Error("forwarded unrelated credentials")
}
proof, xerr := service.VerifyTokenFor(token.PurposeWebSocketProxy, r.Header.Get("X-Warmbly-Proxy-Proof"))
if xerr != nil || proof.SessionID != ticketID || proof.ClientIP != "127.0.0.1" {
t.Error("missing authenticated peer identity")
}
upgrader := websocket.Upgrader{CheckOrigin: func(r *http.Request) bool { return r.Header.Get("Origin") == "https://dashboard.test" }}
connection, err := upgrader.Upgrade(w, r, nil)
if err != nil {
return
}
defer connection.Close()
kind, frame, err := connection.ReadMessage()
if err == nil {
_ = connection.WriteMessage(kind, frame)
}
}))
defer upstream.Close()
h := &Handler{WebsocketURI: upstream.URL + "/prefix/socket?tenant=legacy", TokenService: service}
router := gin.New()
_ = router.SetTrustedProxies(nil)
router.GET("/v1/realtime/socket/websocket", h.ProxyWebsocket)
api := httptest.NewServer(router)
defer api.Close()
base := strings.Replace(api.URL, "http", "ws", 1) + "/v1/realtime/socket/websocket?vsn=1.0.0&target=https://untrusted.test&token="
headers := http.Header{"Origin": {"https://dashboard.test"}, "Authorization": {"Bearer unrelated"}, "Cookie": {"session=unrelated"}, "X-Warmbly-Token": {"wmbly_unrelated"}, "X-Warmbly-Proxy-Proof": {"forged"}}
connection, handshake, err := websocket.DefaultDialer.Dial(base+url.QueryEscape(ticket), headers)
if handshake != nil {
defer handshake.Body.Close()
}
if err != nil {
t.Fatal(err)
}
defer connection.Close()
frame := []byte(`{"topic":"phoenix","event":"heartbeat","payload":{},"ref":"1"}`)
if err := connection.WriteMessage(websocket.TextMessage, frame); err != nil {
t.Fatal(err)
}
_, echoed, err := connection.ReadMessage()
if err != nil || string(echoed) != string(frame) {
t.Fatalf("frame relay failed: %v", err)
}
for _, purpose := range []string{token.PurposeAccess, token.PurposePasswordReset, token.PurposeWebSocketProxy} {
invalid, err := service.GenerateTokenFor(purpose, userID, ticketID, "", "nonce", now, now.Add(time.Minute))
if err != nil {
t.Fatal(err)
}
_, response, err := websocket.DefaultDialer.Dial(base+url.QueryEscape(invalid), headers)
if response != nil {
_ = response.Body.Close()
}
if err == nil || response == nil || response.StatusCode != http.StatusUnauthorized {
t.Fatalf("accepted %s credential", purpose)
}
}
_, response, err := websocket.DefaultDialer.Dial(base+url.QueryEscape(ticket), http.Header{"Origin": {"https://untrusted.test"}})
if response != nil {
_ = response.Body.Close()
}
if err == nil || response == nil || response.StatusCode != http.StatusForbidden {
t.Fatal("lost upstream origin validation")
}
}
+1
View File
@@ -320,6 +320,7 @@ func Run(
// (health, signed webhooks, OAuth bouncers, worker enroll, the internal API,
// and /admin) are NOT versioned and stay at their bare paths.
v1 := r.Group("/v1")
v1.GET("/realtime/socket/websocket", m.PublicIPRateLimitMiddleware(), h.ProxyWebsocket)
v1.POST("/dashboard-image/public", m.PublicIPRateLimitMiddleware(), h.PublicDashboardImage)
v1.POST("/dashboard-image", m.AuthMiddleware(), m.RateLimitMiddleware(models.RateLimitRead), h.DashboardImage)
+8 -7
View File
@@ -30,13 +30,14 @@ const (
// each token is separately bound to a nonce in Redis or a row in `sessions`;
// the realtime service checks neither, and had nothing else to go on.
const (
PurposeAccess = "access"
PurposeRefresh = "refresh"
PurposeWebSocket = "ws"
PurposeLoginCode = "login"
PurposeRegistration = "registration"
PurposePasswordReset = "reset"
PurposeTwoFAPending = "2fa"
PurposeAccess = "access"
PurposeRefresh = "refresh"
PurposeWebSocket = "ws"
PurposeWebSocketProxy = "ws_proxy"
PurposeLoginCode = "login"
PurposeRegistration = "registration"
PurposePasswordReset = "reset"
PurposeTwoFAPending = "2fa"
// PurposeSSOLink is a federated sign-in parked on an existing password
// account until that password is presented; it opens nothing else.
PurposeSSOLink = "sso_link"
+15 -1
View File
@@ -23,7 +23,8 @@ type TokenClaims struct {
// Purpose names what this token may be spent on. See the Purpose*
// constants: one signing key issues all of them, so this is what keeps a
// password-reset token from being accepted as a session.
Purpose string `json:"purpose,omitempty"`
Purpose string `json:"purpose,omitempty"`
ClientIP string `json:"client_ip,omitempty"`
jwt.RegisteredClaims
}
@@ -71,6 +72,19 @@ func (s *tokenService) VerifyToken(tokenStr string) (*TokenClaims, *errx.Error)
return claims, nil
}
// The proof is bound to one ticket and is never accepted as a socket credential.
func (s *tokenService) GenerateWebsocketProxyProof(ticket *TokenClaims, clientIP string) (string, error) {
now := time.Now()
claims := TokenClaims{
UserID: ticket.UserID, SessionID: ticket.SessionID, Nonce: ticket.Nonce,
Purpose: PurposeWebSocketProxy, ClientIP: clientIP,
RegisteredClaims: jwt.RegisteredClaims{
IssuedAt: jwt.NewNumericDate(now), ExpiresAt: jwt.NewNumericDate(now.Add(30 * time.Second)),
},
}
return jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString([]byte(s.AuthSecret))
}
func (s *tokenService) GenerateSession(ctx context.Context, userID uuid.UUID, email, ipaddr, userAgent, authProvider string) (*models.Token, *errx.Error) {
return s.generateSession(ctx, userID, email, ipaddr, userAgent, authProvider, nil, false, true)
}
+1
View File
@@ -21,6 +21,7 @@ func TestTokenPurposeIsEnforced(t *testing.T) {
PurposeAccess,
PurposeRefresh,
PurposeWebSocket,
PurposeWebSocketProxy,
PurposeLoginCode,
PurposeRegistration,
PurposePasswordReset,
+1
View File
@@ -19,6 +19,7 @@ type TokenService interface {
// constants). Every verifier requires the purpose it expects, so a token
// minted for one flow cannot be spent on another.
GenerateTokenFor(purpose string, userID, sessionID uuid.UUID, email, nonce string, issuedAt, expiresAt time.Time) (string, error)
GenerateWebsocketProxyProof(ticket *TokenClaims, clientIP string) (string, error)
VerifyToken(tokenStr string) (*TokenClaims, *errx.Error)
// VerifyTokenFor also requires the token to have been minted for this
// purpose, so one flow's token cannot be spent on another.
+25 -13
View File
@@ -115,23 +115,35 @@ func WebsocketURL() string {
}
func NormalizeWebsocketURL(value string) string {
v := strings.TrimRight(strings.TrimSpace(value), "/")
v := strings.TrimSpace(value)
if v == "" {
return ""
}
// The variable is written three ways in the wild: a bare host, the Phoenix
// socket mount (".../socket"), and the full transport endpoint. Clients
// dial what this returns, so all three normalise to the last one. Matching
// on a "/socket" substring instead of the suffix left ".../socket"
// untouched, which is not a websocket endpoint.
switch {
case strings.HasSuffix(v, "/socket/websocket"):
case strings.HasSuffix(v, "/socket"):
v += "/websocket"
default:
v += "/socket/websocket"
u, err := url.Parse(v)
if err != nil || u.Host == "" || u.User != nil {
return ""
}
return v
switch u.Scheme {
case "https":
u.Scheme = "wss"
case "http":
u.Scheme = "ws"
case "ws", "wss":
default:
return ""
}
u.Path = strings.TrimRight(u.Path, "/")
// Normalize the path, not the query string on a legacy reverse-proxy URL.
switch {
case strings.HasSuffix(u.Path, "/socket/websocket"):
case strings.HasSuffix(u.Path, "/socket"):
u.Path += "/websocket"
default:
u.Path += "/socket/websocket"
}
u.RawPath = ""
u.Fragment = ""
return u.String()
}
func GetPasswordResetURL(sessionToken, origin string) string {
+13
View File
@@ -2,6 +2,19 @@ package config
import "testing"
func TestNormalizeWebsocketURLWithPathsAndQueries(t *testing.T) {
for input, want := range map[string]string{
"https://realtime.test/socket?tenant=old": "wss://realtime.test/socket/websocket?tenant=old",
"http://localhost:4000/proxy/socket/websocket/?tenant=old": "ws://localhost:4000/proxy/socket/websocket?tenant=old",
"wss://realtime.test?tenant=old": "wss://realtime.test/socket/websocket?tenant=old",
"file:///etc/passwd": "", "wss://user:password@realtime.test": "", "://bad": "",
} {
if got := NormalizeWebsocketURL(input); got != want {
t.Errorf("NormalizeWebsocketURL(%q) = %q, want %q", input, got, want)
}
}
}
func TestNormalizeWebsocketURLPreservesLegacyConfiguration(t *testing.T) {
for _, input := range []string{"", " wss://realtime.test/ ", "wss://realtime.test/socket", "wss://realtime.test/socket/websocket/"} {
want := "wss://realtime.test/socket/websocket"
+37
View File
@@ -79,6 +79,43 @@ defmodule Realtime.Auth do
{:error, :verification_error}
end
# Only the backend's ticket-bound proof can replace the socket peer address.
def proxy_ip(proof, ticket) when is_binary(proof) and is_binary(ticket) do
secret = Application.get_env(:realtime, :jwt_secret)
now = System.system_time(:second)
with {true, %JOSE.JWT{fields: claims}, _} <-
JOSE.JWT.verify_strict(jwk(secret), ["HS256"], proof),
{true, %JOSE.JWT{fields: credential}, _} <-
JOSE.JWT.verify_strict(jwk(secret), ["HS256"], ticket),
{:ok, user_id} <- validate_claims(credential),
%{
"purpose" => "ws_proxy",
"sub" => ^user_id,
"exp" => exp,
"iat" => iat,
"sid" => sid,
"nonce" => nonce,
"client_ip" => ip
} <- claims,
true <-
is_integer(exp) and is_integer(iat) and exp > now and iat <= now + 5 and
exp - iat <= 30,
true <-
is_binary(sid) and is_binary(nonce) and sid == credential["sid"] and
nonce == credential["nonce"],
true <- is_binary(ip),
{:ok, address} <- :inet.parse_address(String.to_charlist(ip)) do
{:ok, address |> :inet.ntoa() |> to_string()}
else
_ -> :error
end
rescue
_ -> :error
end
def proxy_ip(_, _), do: :error
@doc """
Verify an API key.
"""
@@ -73,7 +73,7 @@ defmodule RealtimeWeb.UserSocket do
defp present?(value), do: is_binary(value) and value != ""
defp authenticate(token, socket, connect_info) do
ip = get_ip(connect_info)
ip = client_ip(token, connect_info)
with {:ok, user_id, auth_type} <- Auth.verify_token(token, ip: ip),
limits <- Subscription.get_limits(user_id),
@@ -166,6 +166,22 @@ defmodule RealtimeWeb.UserSocket do
end
end
def client_ip(ticket, connect_info) do
proof =
connect_info
|> Map.get(:x_headers, [])
|> Enum.find_value(fn
{"x-warmbly-proxy-proof", value} -> value
_ -> nil
end)
with {:ok, ip} <- Auth.proxy_ip(proof, ticket) do
ip
else
_ -> get_ip(connect_info)
end
end
defp get_ip(%{peer_data: %{address: address}}) do
address |> :inet.ntoa() |> to_string()
end
+61
View File
@@ -0,0 +1,61 @@
defmodule Realtime.ProxyIPTest do
use ExUnit.Case, async: true
alias Realtime.Auth
alias RealtimeWeb.UserSocket
defp sign(claims, secret \\ "test_jwt_secret") do
JOSE.JWT.sign(JOSE.JWK.from_oct(secret), %{"alg" => "HS256"}, claims)
|> JOSE.JWS.compact()
|> elem(1)
end
test "only a signed proof bound to the authenticated ticket can preserve the client IP" do
now = System.system_time(:second)
claims = %{
"sub" => "user",
"sid" => "ticket",
"nonce" => "nonce",
"purpose" => "ws",
"exp" => now + 600
}
ticket = sign(claims)
proof =
Map.merge(claims, %{
"purpose" => "ws_proxy",
"iat" => now,
"exp" => now + 30,
"client_ip" => "192.0.2.10"
})
assert Auth.proxy_ip(sign(proof), ticket) == {:ok, "192.0.2.10"}
assert UserSocket.client_ip(ticket, %{
peer_data: %{address: {127, 0, 0, 1}},
x_headers: [{"x-warmbly-proxy-proof", sign(proof)}]
}) == "192.0.2.10"
assert {:error, :wrong_token_purpose} = Auth.verify_jwt(sign(proof))
for invalid <- [
Map.put(proof, "sid", "different"),
Map.put(proof, "nonce", "different"),
Map.put(proof, "sub", "another-user"),
Map.put(proof, "exp", now - 1),
Map.put(proof, "client_ip", "not-an-ip"),
Map.put(proof, "purpose", "ws")
] do
assert Auth.proxy_ip(sign(invalid), ticket) == :error
end
assert Auth.proxy_ip(sign(proof, "wrong-key"), ticket) == :error
assert Auth.proxy_ip(sign(proof), "wmbly_key") == :error
assert UserSocket.client_ip(ticket, %{
peer_data: %{address: {127, 0, 0, 1}},
x_headers: [{"x-warmbly-proxy-proof", "forged"}, {"x-forwarded-for", "192.0.2.99"}]
}) == "127.0.0.1"
end
end
+1 -1
View File
@@ -79,7 +79,7 @@ if [ -f "$CSP_DIR/csp-template.txt" ]; then
:
else
realtime=""
printf 'Realtime CSP discovery unavailable; using explicit CSP origins. Restart the dashboard after the backend is ready, or set WARMBLY_CSP_CONNECT_ORIGINS for offline deployments.\n' >&2
printf 'Realtime CSP discovery unavailable; the dashboard can use the API-origin socket route. Explicit CSP origins are needed only for direct gateway connections.\n' >&2
fi
fi
if [ -n "$realtime" ]; then
+8 -5
View File
@@ -1,6 +1,9 @@
import { RiFireLine, RiMoreLine } from "@remixicon/react";
import React, { useEffect, useMemo, useRef } from "react";
import { useSearchParam, useSearchParams } from "@/hooks/useSearchParams";
import { useSearchParams } from "@/hooks/useSearchParams";
import useMailboxTagFilter from "@/hooks/useMailboxTagFilter";
import groupMailboxes from "@/lib/groupMailboxes";
import { useAppStore } from "@/stores/useAppStore";
import toast from "react-hot-toast/headless";
import { useQueryClient } from "@tanstack/react-query";
import useEmails from "@/lib/api/hooks/app/emails/useEmails";
@@ -128,8 +131,8 @@ export default function AddressesPage() {
const canView = usePermission("MANAGE_EMAILS");
const [query, setQuery] = React.useState<string>("");
const [selectedTag, setTag] = useSearchParam("tag");
const tag = p.user.tags.some((t) => t.id === selectedTag) ? selectedTag : "";
const organizationID = useAppStore((state) => state.currentOrganization?.id ?? "personal");
const [tag, setTag] = useMailboxTagFilter(`${p.user.id}:${organizationID}`, p.user.tags);
const emailsData = useEmails({ query, tag });
const [selected, setSelected] = React.useState<string[]>([]);
const [view, setView] = React.useState<string>("");
@@ -365,7 +368,7 @@ export default function AddressesPage() {
const sortedEmails = useMemo(() => {
const list = emailsData.emails ?? [];
const col = sort && MAILBOX_COLUMNS.find((c) => c.id === sort.by);
if (!sort || !col?.sortValue) return list;
if (!sort || !col?.sortValue) return tag ? list : groupMailboxes(list, p.user.tags);
const value = col.sortValue;
const dir = sort.reverse ? 1 : -1;
return [...list].sort((a, b) => {
@@ -374,7 +377,7 @@ export default function AddressesPage() {
if (x === y) return 0;
return (x > y ? 1 : -1) * dir;
});
}, [emailsData.emails, sort, statusById]);
}, [emailsData.emails, sort, statusById, tag, p.user.tags]);
if (!canView) {
return <NoAccess feature="email accounts" permissionLabel="Manage mailboxes" />;
+6 -3
View File
@@ -626,7 +626,7 @@ export default function SocketProvider({
}
try {
const urlData = await getSocket();
const urlData = await getSocket(reconnectAttemptRef.current % 2 === 0);
// Phoenix vsn=1.0.0 — our sendRaw / joinChannel paths emit the
// V1 object format ({topic, event, payload, ref}), not the
// V2 array format. Sending vsn=2.0.0 made the realtime server
@@ -640,6 +640,7 @@ export default function SocketProvider({
wsRef.current.onopen = (ev) => {
setIsConnected(true);
reconnectAttemptRef.current = 0;
setReconnectAttempt(0);
startHeartbeat();
rejoinChannels();
@@ -684,7 +685,8 @@ export default function SocketProvider({
// ±25% jitter so clients don't reconnect in lockstep after an outage.
const delay = Math.round(base * (0.75 + Math.random() * 0.5));
reconnectTimerRef.current = setTimeout(() => {
setReconnectAttempt((a) => a + 1);
reconnectAttemptRef.current += 1;
setReconnectAttempt(reconnectAttemptRef.current);
connect();
}, delay);
}
@@ -742,7 +744,8 @@ export default function SocketProvider({
const base = RECONNECT_SCHEDULE[Math.min(attempt, RECONNECT_SCHEDULE.length - 1)];
const delay = Math.round(base * (0.75 + Math.random() * 0.5));
reconnectTimerRef.current = setTimeout(() => {
setReconnectAttempt((a) => a + 1);
reconnectAttemptRef.current += 1;
setReconnectAttempt(reconnectAttemptRef.current);
connect();
}, delay);
}
+15
View File
@@ -4,6 +4,7 @@ import React, { useState } from 'react'
import SocketProvider from './SocketProvider'
import { useChannel, useChannelEvent } from './context/socket'
import { installFakeSocket, freezeJitter, type SocketEnv } from './socketTestHarness'
import getSocket from '@/lib/api/client/app/socket/getSocket'
vi.mock('@/lib/api/client/app/socket/getSocket', () => ({
default: vi.fn(async () => ({ url: 'ws://localhost:4000/socket/websocket?token=test' })),
@@ -35,6 +36,7 @@ function Panel({ topic = TOPIC, onEvent }: { topic?: string; onEvent?: () => voi
}
beforeEach(() => {
vi.mocked(getSocket).mockClear()
vi.useFakeTimers()
freezeJitter(0)
env = installFakeSocket()
@@ -45,6 +47,19 @@ afterEach(() => {
vi.restoreAllMocks()
})
it('retries the direct gateway if an existing API proxy cannot upgrade WebSockets', async () => {
await mount(<Panel />)
expect(getSocket).toHaveBeenLastCalledWith(true)
await act(async () => {
const socket = env.instances[0]
socket.readyState = 3
socket.onclose?.({ wasClean: false })
})
await tick(1000)
expect(getSocket).toHaveBeenLastCalledWith(false)
expect(env.instances).toHaveLength(2)
})
describe('channel subscription lifecycle', () => {
it('joins once and stays joined across unrelated re-renders', async () => {
// Regression: the provider's context value changed identity on every
@@ -0,0 +1,72 @@
import { act, fireEvent, render, screen, waitFor } from "@testing-library/react";
import { createMemoryHistory, createRootRoute, createRoute, createRouter, Outlet, RouterProvider } from "@tanstack/react-router";
import { beforeEach, describe, expect, it } from "vitest";
import { parseSearch, stringifySearch } from "@/lib/routerSearch";
import useMailboxTagFilter from "./useMailboxTagFilter";
let workspace = "workspace";
let tags: readonly { id: string }[] | undefined;
function Filters() {
const [tag, setTag] = useMailboxTagFilter(`user:${workspace}`, tags);
return <><output aria-label="Tag">{tag || "All accounts"}</output>
<button onClick={() => setTag("sending")}>Sending</button>
<button onClick={() => setTag("")}>All accounts</button></>;
}
async function open(href = "/app/emails") {
const root = createRootRoute({ component: Outlet });
const emails = createRoute({ getParentRoute: () => root, path: "/app/emails", component: Filters });
const tasks = createRoute({ getParentRoute: () => root, path: "/app/tasks", component: () => <div>Tasks</div> });
const router = createRouter({ routeTree: root.addChildren([emails, tasks]), history: createMemoryHistory({ initialEntries: [href] }), parseSearch, stringifySearch });
await act(() => router.load());
return { router, ...render(<RouterProvider router={router} />) };
}
describe("remembered mailbox tag", () => {
beforeEach(() => {
sessionStorage.clear();
workspace = "workspace";
tags = [{ id: "sending" }];
});
it("waits for profile tags and keeps other workspaces isolated", async () => {
tags = undefined;
sessionStorage.setItem("warmbly:mailbox-tag:user:workspace", "sending");
const page = await open();
expect(screen.getByLabelText("Tag")).toHaveTextContent("sending");
tags = [{ id: "sending" }];
page.rerender(<RouterProvider router={page.router} />);
expect(screen.getByLabelText("Tag")).toHaveTextContent("sending");
page.unmount();
workspace = "another-workspace";
await open();
expect(screen.getByLabelText("Tag")).toHaveTextContent("All accounts");
expect(sessionStorage.getItem("warmbly:mailbox-tag:user:workspace")).toBe("sending");
});
it("survives navigating to Tasks and returning through a queryless Accounts link", async () => {
const page = await open();
fireEvent.click(screen.getByRole("button", { name: "Sending" }));
await waitFor(() => expect(page.router.state.location.searchStr).toBe("?tag=sending"));
await act(() => page.router.navigate({ href: "/app/tasks" }));
await act(() => page.router.navigate({ to: "/app/emails", search: { mailbox: "mailbox", tab: "settings" } }));
await waitFor(() => expect(screen.getByLabelText("Tag")).toHaveTextContent("sending"));
expect(page.router.state.location.searchStr).toContain("mailbox=mailbox");
expect(page.router.state.location.searchStr).toContain("tab=settings");
const href = page.router.state.location.href;
page.unmount();
await open(href);
expect(screen.getByLabelText("Tag")).toHaveTextContent("sending");
});
it("keeps All accounts after clearing and falls back for deleted or other-workspace tags", async () => {
sessionStorage.setItem("warmbly:mailbox-tag:other:workspace", "sending");
const page = await open("/app/emails?tag=sending");
fireEvent.click(screen.getByRole("button", { name: "All accounts" }));
await waitFor(() => expect(page.router.state.location.searchStr).toBe(""));
await act(() => page.router.navigate({ href: "/app/tasks" }));
await act(() => page.router.navigate({ to: "/app/emails" }));
expect(screen.getByLabelText("Tag")).toHaveTextContent("All accounts");
page.unmount();
await open("/app/emails?tag=deleted");
await waitFor(() => expect(screen.getByLabelText("Tag")).toHaveTextContent("All accounts"));
});
});
+30
View File
@@ -0,0 +1,30 @@
import { useCallback, useEffect, useState } from "react";
import { useSearchParam, useSearchParams } from "./useSearchParams";
export default function useMailboxTagFilter(owner: string, validTags: readonly { id: string }[] | undefined) {
const key = `warmbly:mailbox-tag:${owner}`;
const [params] = useSearchParams();
const [urlTag, setURLTag] = useSearchParam("tag");
const read = useCallback(() => {
try { return sessionStorage.getItem(key) ?? ""; } catch { return ""; }
}, [key]);
const [memory, setMemory] = useState(() => ({ key, value: read() }));
const remembered = memory.key === key ? memory.value : read();
const selected = params.has("tag") ? urlTag : remembered;
const tag = validTags === undefined || validTags.some((candidate) => candidate.id === selected) ? selected : "";
const remember = useCallback((value: string) => {
setMemory({ key, value });
try { sessionStorage.setItem(key, value); } catch { /* Storage may be disabled. */ }
}, [key]);
useEffect(() => {
remember(tag);
if (tag !== urlTag) setURLTag(tag);
}, [tag, urlTag, remember, setURLTag]);
const setTag = useCallback((value: string) => {
remember(value);
setURLTag(value);
}, [remember, setURLTag]);
return [tag, setTag] as const;
}
@@ -0,0 +1,22 @@
import { describe, expect, it } from "vitest";
import { socketURL } from "./getSocket";
describe("realtime upgrade routing", () => {
const ticket = { url: "wss://old.realtime.test/socket/websocket?token=ticket", expires_in: 600 };
it("keeps direct URLs from old backends", () => {
expect(socketURL(ticket, "https://api.test/v1")).toBe(ticket.url);
});
it.each(["https://api.test/v1", "https://api.test/v1/"])("uses the API CSP origin with %s", (base) => {
expect(socketURL({ ...ticket, proxy_path: "/realtime/socket/websocket?token=ticket" }, base))
.toBe("wss://api.test/v1/realtime/socket/websocket?token=ticket");
});
it("preserves plain HTTP ports and same-origin API deployments", () => {
expect(socketURL({ ...ticket, proxy_path: "/realtime/socket/websocket?token=ticket" }, "http://localhost:8080/v1"))
.toBe("ws://localhost:8080/v1/realtime/socket/websocket?token=ticket");
expect(socketURL({ ...ticket, proxy_path: "/realtime/socket/websocket?token=ticket" }, "/v1"))
.toContain("/v1/realtime/socket/websocket?token=ticket");
});
it("refuses another origin supplied as a proxy path", () => {
expect(() => socketURL({ ...ticket, proxy_path: "https://untrusted.test/socket" }, "https://api.test/v1")).toThrow();
});
});
+16 -3
View File
@@ -1,10 +1,23 @@
import Request from "../../Request";
import type GetSocket from "@/lib/api/models/app/socket/GetSocket";
import { API_BASE_URL } from "@/lib/information";
export default async function getSocket(): Promise<GetSocket> {
return await Request<GetSocket>({
export function socketURL(ticket: GetSocket, base = API_BASE_URL): string {
if (!ticket.proxy_path) return ticket.url;
const api = new URL(base.replace(/\/$/, "") + "/", window.location.origin);
const url = new URL(ticket.proxy_path.replace(/^\//, ""), api);
if (url.origin !== api.origin || !["http:", "https:"].includes(url.protocol)) {
throw new Error("Invalid realtime proxy URL");
}
url.protocol = url.protocol === "https:" ? "wss:" : "ws:";
return url.toString();
}
export default async function getSocket(useProxy = true): Promise<GetSocket> {
const ticket = await Request<GetSocket>({
method: "POST",
url: `/getaway`,
authorization: true,
})
});
return { ...ticket, url: useProxy ? socketURL(ticket) : ticket.url };
}
@@ -1,4 +1,5 @@
export default interface GetSocket {
url: string;
proxy_path?: string;
expires_in: number;
}
+19
View File
@@ -0,0 +1,19 @@
import { describe, expect, it } from "vitest";
import type Inbox from "./api/models/app/emails/Inbox";
import type Tag from "./api/models/app/Tag";
import groupMailboxes from "./groupMailboxes";
describe("default mailbox tag groups", () => {
const tags = [{ id: "sending", title: "Sending", position: 0 }, { id: "receiving", title: "Receiving", position: 1 }] as Tag[];
const mailbox = (id: string, assigned: string[]) => ({ id, email: `${id}@test.example`, tags: assigned }) as Inbox;
it("keeps shared tags together with untagged mailboxes last without mutating cached data", () => {
const rows = [mailbox("z", []), mailbox("c", ["receiving"]), mailbox("b", ["sending"]), mailbox("a", ["sending"])];
expect(groupMailboxes(rows, tags).map((row) => row.id)).toEqual(["a", "b", "c", "z"]);
expect(rows.map((row) => row.id)).toEqual(["z", "c", "b", "a"]);
});
it("assigns overlapping tags to the first tag in the workspace order and ignores assignment order", () => {
const rows = [mailbox("b", ["receiving"]), mailbox("a", ["receiving", "sending"]), mailbox("c", ["deleted"])];
expect(groupMailboxes(rows, tags).map((row) => row.id)).toEqual(["a", "b", "c"]);
expect(groupMailboxes(rows, [...tags].reverse()).map((row) => row.id)).toEqual(["a", "b", "c"]);
});
});
+9
View File
@@ -0,0 +1,9 @@
import type Inbox from "./api/models/app/emails/Inbox";
import type Tag from "./api/models/app/Tag";
export default function groupMailboxes(mailboxes: readonly Inbox[], tags: readonly Tag[]): Inbox[] {
const ordered = [...tags].sort((a, b) => a.position - b.position || a.title.localeCompare(b.title) || a.id.localeCompare(b.id));
const rank = new Map(ordered.map((tag, index) => [tag.id, index]));
const group = (mailbox: Inbox) => Math.min(...mailbox.tags.map((id) => rank.get(id) ?? ordered.length), ordered.length);
return [...mailboxes].sort((a, b) => group(a) - group(b) || a.email.localeCompare(b.email) || a.id.localeCompare(b.id));
}