feat: Embed Noto Sans SC subset for reliable CJK rendering

.gitignore

@@ -65,6 +65,8 @@ docs/.docusaurus
 
 /desktop/ui/i18n.slint
 /desktop/ui/version.slint
+/desktop/assets/fonts/source/
+/desktop/assets/fonts/.tools/
 /dev
 /bak/
 .idea/

Makefile

@@ -28,6 +28,15 @@ orbien:
 desktop-dev:
 	cargo run -p orbien-desktop
 
+desktop-font-subset:
+	python3 scripts/gen-desktop-font-subset.py
+
+desktop-font-subset-tiny:
+	python3 scripts/gen-desktop-font-subset.py --regular-only
+
+desktop-font-subset-full:
+	python3 scripts/gen-desktop-font-subset.py --full
+
 desktop-build:
 	cargo build --release -p orbien-desktop
 

Makefile.md

@@ -2,19 +2,22 @@
 
 Run from the repository root. Build artifacts go to `target/`; packages go to `dist/`.
 
-| Command | Description |
-|---------|-------------|
-| `make web` | Build the dashboard UI into `server/assets` |
-| `make build` | Debug-build `orbien-server` and `orbien` |
-| `make release` | Run `web`, then release-build server and client |
-| `make orbien-server` | Run `web`, then release-build the server |
-| `make orbien` | Release-build the client |
-| `make desktop-dev` | Run the desktop app (dev) |
-| `make desktop-build` | Release-build the desktop app |
-| `make desktop-app` / `make desktop-dmg` | Package macOS `.app` / `.dmg` |
-| `make desktop-windows` | Package Windows exe / zip (run on Windows) |
-| `make desktop-deb` | Package Linux `.deb` (run on Debian/Ubuntu) |
-| `make package` | Run `release`, then copy binaries and `conf/` to `dist/` |
-| `make test` | Run workspace tests |
-| `make fmt` | Format Rust code |
-| `make clean` | Clean `target/` and frontend install/build outputs |
+| Command                                 | Description                                              |
+|-----------------------------------------|----------------------------------------------------------|
+| `make web`                              | Build the dashboard UI into `server/assets`              |
+| `make build`                            | Debug-build `orbien-server` and `orbien`                 |
+| `make release`                          | Run `web`, then release-build server and client          |
+| `make orbien-server`                    | Run `web`, then release-build the server                 |
+| `make orbien`                           | Release-build the client                                 |
+| `make desktop-dev`                      | Run the desktop app (dev)                                |
+| `make desktop-font-subset`              | Rebuild embedded fonts (i18n + GB2312 L1, ~1.8 MB)       |
+| `make desktop-font-subset-tiny`         | Regular-only fonts (~0.9 MB)                             |
+| `make desktop-font-subset-full`         | Full SC regional fonts (~16 MB)                          |
+| `make desktop-build`                    | Release-build the desktop app                            |
+| `make desktop-app` / `make desktop-dmg` | Package macOS `.app` / `.dmg`                            |
+| `make desktop-windows`                  | Package Windows exe / zip (run on Windows)               |
+| `make desktop-deb`                      | Package Linux `.deb` (run on Debian/Ubuntu)              |
+| `make package`                          | Run `release`, then copy binaries and `conf/` to `dist/` |
+| `make test`                             | Run workspace tests                                      |
+| `make fmt`                              | Format Rust code                                         |
+| `make clean`                            | Clean `target/` and frontend install/build outputs       |

desktop/assets/app-icons/README.md

@@ -1,9 +0,0 @@
-# App icons
-
-| File | Used by |
-|------|---------|
-| `icon.icns` | macOS `.app` / `.dmg` (`pack-desktop-macos.sh`) |
-| `icon.ico` | Windows exe (`desktop/build.rs`) |
-| `128x128.png` | Window icon (`ui/app.slint`) + Linux deb |
-| `64x64.png` | Linux deb |
-| `32x32.png` | Linux deb |

desktop/assets/fonts/OFL.txt

@@ -0,0 +1,93 @@
+Copyright 2014-2021 Adobe (http://www.adobe.com/), with Reserved Font Name 'Source'
+
+This Font Software is licensed under the SIL Open Font License, Version 1.1.
+This license is copied below, and is also available with a FAQ at:
+https://scripts.sil.org/OFL
+
+
+-----------------------------------------------------------
+SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
+-----------------------------------------------------------
+
+PREAMBLE
+The goals of the Open Font License (OFL) are to stimulate worldwide
+development of collaborative font projects, to support the font creation
+efforts of academic and linguistic communities, and to provide a free and
+open framework in which fonts may be shared and improved in partnership
+with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and
+redistributed freely as long as they are not sold by themselves. The
+fonts, including any derivative works, can be bundled, embedded, 
+redistributed and/or sold with any software provided that any reserved
+names are not used by derivative works. The fonts and derivatives,
+however, cannot be released under any other type of license. The
+requirement for fonts to remain under this license does not apply
+to any document created using the fonts or their derivatives.
+
+DEFINITIONS
+"Font Software" refers to the set of files released by the Copyright
+Holder(s) under this license and clearly marked as such. This may
+include source files, build scripts and documentation.
+
+"Reserved Font Name" refers to any names specified as such after the
+copyright statement(s).
+
+"Original Version" refers to the collection of Font Software components as
+distributed by the Copyright Holder(s).
+
+"Modified Version" refers to any derivative made by adding to, deleting,
+or substituting -- in part or in whole -- any of the components of the
+Original Version, by changing formats or by porting the Font Software to a
+new environment.
+
+"Author" refers to any designer, engineer, programmer, technical
+writer or other person who contributed to the Font Software.
+
+PERMISSION & CONDITIONS
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of the Font Software, to use, study, copy, merge, embed, modify,
+redistribute, and sell modified and unmodified copies of the Font
+Software, subject to the following conditions:
+
+1) Neither the Font Software nor any of its individual components,
+in Original or Modified Versions, may be sold by itself.
+
+2) Original or Modified Versions of the Font Software may be bundled,
+redistributed and/or sold with any software, provided that each copy
+contains the above copyright notice and this license. These can be
+included either as stand-alone text files, human-readable headers or
+in the appropriate machine-readable metadata fields within text or
+binary files as long as those fields can be easily viewed by the user.
+
+3) No Modified Version of the Font Software may use the Reserved Font
+Name(s) unless explicit written permission is granted by the corresponding
+Copyright Holder. This restriction only applies to the primary font name as
+presented to the users.
+
+4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
+Software shall not be used to promote, endorse or advertise any
+Modified Version, except to acknowledge the contribution(s) of the
+Copyright Holder(s) and the Author(s) or with their explicit written
+permission.
+
+5) The Font Software, modified or unmodified, in part or in whole,
+must be distributed entirely under this license, and must not be
+distributed under any other license. The requirement for fonts to
+remain under this license does not apply to any document created
+using the Font Software.
+
+TERMINATION
+This license becomes null and void if any of the above conditions are
+not met.
+
+DISCLAIMER
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
+OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
+OTHER DEALINGS IN THE FONT SOFTWARE.

desktop/build.rs

@@ -25,8 +25,14 @@ fn main() {
     }
 
     generate_slint_version(&version);
-    generate_slint_i18n();
-    slint_build::compile("ui/app.slint").expect("slint compile failed");
+    let zh = generate_slint_i18n();
+    verify_font_assets(&zh);
+    slint_build::compile_with_config(
+        "ui/app.slint",
+        slint_build::CompilerConfiguration::new()
+            .embed_resources(slint_build::EmbedResourcesKind::EmbedFiles),
+    )
+    .expect("slint compile failed");
 }
 
 fn generate_slint_version(version: &str) {
@@ -57,7 +63,7 @@ fn embed_macos_plist(version: &str) {
     );
 }
 
-fn generate_slint_i18n() {
+fn generate_slint_i18n() -> BTreeMap<String, String> {
     let manifest = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
     let zh_path = manifest.join("i18n/zh_CN.properties");
     let en_path = manifest.join("i18n/en_US.properties");
@@ -78,6 +84,46 @@ fn generate_slint_i18n() {
 
     let out = manifest.join("ui/i18n.slint");
     fs::write(&out, render_slint(&keys, &zh, &en)).expect("write ui/i18n.slint");
+    zh
+}
+
+fn verify_font_assets(zh: &BTreeMap<String, String>) {
+    let manifest = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
+    let fonts = manifest.join("assets/fonts");
+    for name in [
+        "NotoSansSC-Regular.subset.otf",
+        "NotoSansSC-Bold.subset.otf",
+        "OFL.txt",
+        "subset-manifest.json",
+    ] {
+        let path = fonts.join(name);
+        assert!(
+            path.is_file(),
+            "missing font asset {} — run: make desktop-font-subset",
+            path.display()
+        );
+        println!("cargo:rerun-if-changed={}", path.display());
+    }
+
+    let meta =
+        fs::read_to_string(fonts.join("subset-manifest.json")).expect("read subset-manifest.json");
+    if meta.contains("\"i18n-gb2312") {
+        for (key, value) in zh {
+            for ch in value.chars().filter(|c| needs_embedded_cjk_coverage(*c)) {
+                let needle = format!("\"{ch}\"");
+                assert!(
+                    meta.contains(&needle),
+                    "zh_CN key '{key}' uses '{ch}' missing from font subset; \
+                     run: make desktop-font-subset"
+                );
+            }
+        }
+    }
+}
+
+fn needs_embedded_cjk_coverage(ch: char) -> bool {
+    matches!(ch as u32, 0x3000..=0x303F | 0x4E00..=0x9FFF | 0xFF00..=0xFFEF)
+        || matches!(ch, '…' | '·' | '—')
 }
 
 fn parse_properties(raw: &str) -> BTreeMap<String, String> {

desktop/ui/app.slint

@@ -1,3 +1,5 @@
+import "../assets/fonts/NotoSansSC-Regular.subset.otf";
+import "../assets/fonts/NotoSansSC-Bold.subset.otf";
 import { Palette } from "std-widgets.slint";
 import { Theme } from "theme.slint";
 import { Sidebar, AppPage } from "sidebar.slint";
@@ -14,6 +16,8 @@ import { AppMeta } from "version.slint";
 export { AppPage, TunnelRow, LogLine, Tr, AboutDialog, AppMeta, Theme }
 export component AppWindow inherits Window {
     title: "Orbien Desktop";
+    default-font-family: "Noto Sans SC";
+    default-font-size: 13px;
     icon: @image-url("../assets/app-icons/128x128.png");
     preferred-width: 960px;
     preferred-height: 640px;

desktop/ui/pages/logger.slint

@@ -140,7 +140,7 @@ component LogRow inherits Rectangle {
                 text: root.line.pre;
                 color: root.body-color;
                 font-size: 12px;
-                font-family: "Menlo, Monaco, Consolas, monospace";
+                font-family: "Cascadia Mono, Consolas, Menlo, Monaco, monospace";
                 vertical-alignment: center;
                 overflow: elide;
                 horizontal-stretch: 0;
@@ -159,7 +159,7 @@ component LogRow inherits Rectangle {
                         color: #fef3c7;
                         font-size: 12px;
                         font-weight: 700;
-                        font-family: "Menlo, Monaco, Consolas, monospace";
+                        font-family: "Cascadia Mono, Consolas, Menlo, Monaco, monospace";
                         vertical-alignment: center;
                     }
                 }
@@ -169,7 +169,7 @@ component LogRow inherits Rectangle {
                 text: root.line.post;
                 color: root.body-color;
                 font-size: 12px;
-                font-family: "Menlo, Monaco, Consolas, monospace";
+                font-family: "Cascadia Mono, Consolas, Menlo, Monaco, monospace";
                 vertical-alignment: center;
                 overflow: elide;
                 horizontal-stretch: 1;
@@ -182,7 +182,7 @@ component LogRow inherits Rectangle {
             text: root.line.text;
             color: root.line.level == 0 ? root.body-color : root.tag-color;
             font-size: 12px;
-            font-family: "Menlo, Monaco, Consolas, monospace";
+            font-family: "Cascadia Mono, Consolas, Menlo, Monaco, monospace";
             vertical-alignment: center;
             overflow: elide;
             horizontal-stretch: 1;

desktop/ui/pages/tunnel/fields.slint

@@ -14,7 +14,6 @@ export component FormField inherits Rectangle {
         x: 0;
         y: 0;
         width: parent.width;
-        height: 18px;
         text: root.label;
         color: Theme.text-secondary;
         font-size: 13px;
@@ -47,7 +46,6 @@ export component FormPathField inherits Rectangle {
         x: 0;
         y: 0;
         width: parent.width;
-        height: 18px;
         text: root.label;
         color: Theme.text-secondary;
         font-size: 13px;
@@ -94,7 +92,6 @@ export component FormCombo inherits Rectangle {
         x: 0;
         y: 0;
         width: parent.width;
-        height: 18px;
         text: root.label;
         color: Theme.text-secondary;
         font-size: 13px;

scripts/gen-desktop-font-subset.py

@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+"""Build embedded Noto Sans SC subsets for orbien-desktop.
+
+  python3 scripts/gen-desktop-font-subset.py              # i18n + GB2312 L1 (~1.8 MB)
+  python3 scripts/gen-desktop-font-subset.py --regular-only
+  python3 scripts/gen-desktop-font-subset.py --full       # full SC regional (~16 MB)
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import tempfile
+import urllib.request
+import zipfile
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+DESKTOP = ROOT / "desktop"
+FONTS = DESKTOP / "assets" / "fonts"
+SOURCE = FONTS / "source"
+TOOLS = FONTS / ".tools"
+I18N_ZH = DESKTOP / "i18n" / "zh_CN.properties"
+MANIFEST = FONTS / "subset-manifest.json"
+
+CDN = "https://cdn.jsdelivr.net/gh/notofonts/noto-cjk@main/Sans/SubsetOTF/SC"
+SRC_REGULAR = f"{CDN}/NotoSansSC-Regular.otf"
+SRC_BOLD = f"{CDN}/NotoSansSC-Bold.otf"
+OFL_URL = "https://cdn.jsdelivr.net/gh/google/fonts@main/ofl/notosanssc/OFL.txt"
+FONTTOOLS_WHL = (
+    "https://files.pythonhosted.org/packages/py3/f/fonttools/"
+    "fonttools-4.59.0-py3-none-any.whl"
+)
+
+OUT_REGULAR = "NotoSansSC-Regular.subset.otf"
+OUT_BOLD = "NotoSansSC-Bold.subset.otf"
+FAMILY = "Noto Sans SC"
+
+
+def download(url: str, dest: Path) -> None:
+    dest.parent.mkdir(parents=True, exist_ok=True)
+    print(f"fetch {dest.name}")
+    with urllib.request.urlopen(url, timeout=180) as resp:
+        dest.write_bytes(resp.read())
+
+
+def ensure_fonttools() -> None:
+    try:
+        import fontTools
+
+        return
+    except ImportError:
+        pass
+
+    TOOLS.mkdir(parents=True, exist_ok=True)
+    if not (TOOLS / "fontTools").is_dir():
+        whl = TOOLS / "fonttools.whl"
+        if not whl.exists():
+            download(FONTTOOLS_WHL, whl)
+        with zipfile.ZipFile(whl) as zf:
+            zf.extractall(TOOLS)
+    sys.path.insert(0, str(TOOLS))
+    import fontTools
+
+
+def i18n_chars() -> set[str]:
+    chars: set[str] = set()
+    for line in I18N_ZH.read_text(encoding="utf-8").splitlines():
+        line = line.strip()
+        if not line or line.startswith("#") or "=" not in line:
+            continue
+        value = line.split("=", 1)[1].strip().replace("\\n", "\n").replace("\\t", "\t")
+        chars.update(value)
+    return chars
+
+
+def gb2312_level1() -> set[str]:
+    out: set[str] = set()
+    for zone in range(0xB0, 0xD8):
+        for point in range(0xA1, 0xFF):
+            try:
+                ch = bytes([zone, point]).decode("gb2312")
+            except UnicodeDecodeError:
+                continue
+            if len(ch) == 1:
+                out.add(ch)
+    return out
+
+
+def target_chars() -> set[str]:
+    chars = set(chr(c) for c in range(0x20, 0x7F))
+    chars.update("，、。（）…·—「」『』【】《》！？：；")
+    return chars | i18n_chars() | gb2312_level1()
+
+
+def write_manifest(mode: str, chars: set[str], files: dict[str, int]) -> None:
+    listed = sorted(c for c in chars if ord(c) > 0x7F)
+    MANIFEST.write_text(
+        json.dumps(
+            {
+                "font_family": FAMILY,
+                "license": "SIL Open Font License 1.1 (see OFL.txt)",
+                "mode": mode,
+                "char_count": len(chars),
+                "files": files,
+                "chars": listed,
+            },
+            indent=2,
+            ensure_ascii=False,
+        )
+        + "\n",
+        encoding="utf-8",
+    )
+
+
+def ensure_sources() -> tuple[Path, Path]:
+    SOURCE.mkdir(parents=True, exist_ok=True)
+    regular = SOURCE / "NotoSansSC-Regular.otf"
+    bold = SOURCE / "NotoSansSC-Bold.otf"
+    if not regular.exists():
+        download(SRC_REGULAR, regular)
+    if not bold.exists():
+        download(SRC_BOLD, bold)
+    return regular, bold
+
+
+def subset(src: Path, dest: Path, unicodes_file: Path) -> int:
+    from fontTools import subset as ftsubset
+
+    ftsubset.main(
+        [
+            str(src),
+            f"--output-file={dest}",
+            f"--unicodes-file={unicodes_file}",
+            "--layout-features=kern,liga",
+            "--name-IDs=*",
+            "--name-legacy",
+            "--name-languages=*",
+            "--notdef-outline",
+            "--recommended-glyphs",
+            "--recalc-bounds",
+        ]
+    )
+    return dest.stat().st_size
+
+
+def build_subset(regular_only: bool) -> None:
+    ensure_fonttools()
+    regular_src, bold_src = ensure_sources()
+    chars = target_chars()
+    print(f"subsetting {len(chars)} codepoints")
+
+    with tempfile.NamedTemporaryFile("w", encoding="utf-8", suffix=".txt", delete=False) as f:
+        for c in sorted(chars, key=ord):
+            f.write(f"U+{ord(c):04X}\n")
+        unicodes = Path(f.name)
+
+    files: dict[str, int] = {}
+    try:
+        regular_out = FONTS / OUT_REGULAR
+        bold_out = FONTS / OUT_BOLD
+        files[OUT_REGULAR] = subset(regular_src, regular_out, unicodes)
+        print(f"  {OUT_REGULAR}: {files[OUT_REGULAR] / 1024:.0f} KB")
+
+        if regular_only:
+            bold_out.write_bytes(regular_out.read_bytes())
+            files[OUT_BOLD] = bold_out.stat().st_size
+            mode = "i18n-gb2312-level1-regular-only"
+            print(f"  {OUT_BOLD}: copy of Regular")
+        else:
+            files[OUT_BOLD] = subset(bold_src, bold_out, unicodes)
+            mode = "i18n-gb2312-level1"
+            print(f"  {OUT_BOLD}: {files[OUT_BOLD] / 1024:.0f} KB")
+    finally:
+        unicodes.unlink(missing_ok=True)
+
+    ofl = FONTS / "OFL.txt"
+    if not ofl.exists():
+        download(OFL_URL, ofl)
+
+    write_manifest(mode, chars, files)
+    print(f"total {sum(files.values()) / 1024 / 1024:.2f} MB")
+
+
+def build_full() -> None:
+    files: dict[str, int] = {}
+    for name, url in [(OUT_REGULAR, SRC_REGULAR), (OUT_BOLD, SRC_BOLD)]:
+        dest = FONTS / name
+        download(url, dest)
+        files[name] = dest.stat().st_size
+    download(OFL_URL, FONTS / "OFL.txt")
+    write_manifest("noto-cjk-sc-regional", i18n_chars(), files)
+
+
+def main() -> None:
+    p = argparse.ArgumentParser(description=__doc__)
+    p.add_argument("--full", action="store_true", help="full SC regional (~16 MB)")
+    p.add_argument("--regular-only", action="store_true", help="one weight only (~0.9 MB)")
+    args = p.parse_args()
+
+    if args.full:
+        build_full()
+    else:
+        build_subset(regular_only=args.regular_only)
+    print(f"wrote {MANIFEST}")
+
+
+if __name__ == "__main__":
+    main()