fix(mobile): reject unpaired manifest surrogates

2026-10-09 00:02:39 +00:00 · 2026-08-09 18:34:59 -04:00
parent b055ed1303
commit 976a9bca46
8 changed files with 110 additions and 22 deletions
@@ -2667,4 +2667,7 @@ and diff hygiene pass. A fresh production RNW build remains
 | 2026-07-28 | Finding  | Native staged-asset writes validated hashes only after append, so an asset or ancestor replaced by a symlink could receive bytes outside the cache. Activation writers likewise trusted a previously verified host tree.                                                                                                                                                                                                                |
 | 2026-07-28 | Complete | Both stores now require an unlinked cache descendant before staged or activation writes. Mirrored faults preserve external bytes for linked assets and host trees, and prove atomic activation replacement removes an in-cache file link without changing its target. iOS native faults and Android Debug unit/Release Kotlin gates pass.                                                                                               |
 | 2026-07-28 | Complete | Post-write-boundary validation passes the 569-file mobile suite with 3,378 tests and 2 expected skips, mobile/mobile-web typechecks and lints, reliability, max-lines, focused formatting, diff hygiene, and unchanged `b17ead7a…` package verification.                                                                                                                                                                                |
+| 2026-07-28 | Finding  | Native exact-JSON scanners accepted syntactically escaped lone UTF-16 surrogates before handing strings to Foundation and `org.json`, whose Unicode handling can diverge.                                                                                                                                                                                                                                                               |
+| 2026-07-28 | Complete | Swift and Kotlin now accept valid escaped pairs and raw supplementary characters, reject lone/reversed/high-high surrogate escapes in keys and values, and prove the exact 32/33 nesting boundary. Both native fault suites and Android Release Kotlin compilation pass.                                                                                                                                                                |
+| 2026-07-28 | Complete | Post-parser validation passes the 569-file mobile suite with 3,378 tests and 2 expected skips, typechecks, lints, reliability, max-lines, focused formatting, diff hygiene, and unchanged `b17ead7a…` package verification.                                                                                                                                                                                                             |
 | 2026-07-28 | Next     | Complete the remaining gated cutover cleanup, then execute the physical-device, topology, security, performance, packaged-release, and App Store gates.                                                                                                                                                                                                                                                                                 |
@@ -531,7 +531,9 @@ manifest, activation, or executable asset bytes are consumed. Primary and
 canonical manifests plus activation metadata also pass a bounded exact JSON
 grammar before native parsing. Duplicate decoded keys, trailing tokens,
 malformed scalars, and nesting beyond 32 levels fail consistently on both
-platforms. Cache mutation faults additionally require every destructive
+platforms. Escaped Unicode surrogate pairs and raw supplementary characters
+pass, while unpaired surrogate escapes in keys or values fail before platform
+JSON parsing. Cache mutation faults additionally require every destructive
 Android store path to delete only lexical descendants without following
 symlinks; quota traversal ignores linked trees. Direct, nested, and
 live-stage-replacement links plus host-subtree and dangling host links preserve
@@ -2844,6 +2844,12 @@ error. Activation uses an atomic sibling replacement, so an in-cache activation
 file link is replaced rather than followed and its external target remains
 unchanged.

+Persisted manifest and activation parsing also applies one bounded exact JSON
+grammar before Foundation or `org.json`. It rejects duplicate decoded keys,
+trailing tokens, malformed numbers/escapes, nesting beyond 32 levels, and
+unpaired Unicode surrogate escapes, avoiding platform-specific coercion or
+Unicode behavior before schema validation.
+
 ### Native-shell rollback

 A defect in the asset origin, credential broker, native bridge, audio/picker
@@ -241,6 +241,14 @@ in-cache link rather than modifying its external target.
 The 569-file mobile suite, mobile/mobile-web typechecks and lints, reliability,
 max-lines, focused formatting, diff hygiene, and unchanged `b17ead7a…` package
 verification also pass after the write-boundary repair.
+The exact native JSON grammar now validates Unicode surrogate pairing before
+Foundation or `org.json` parsing. Escaped pairs and raw supplementary
+characters pass, while lone, reversed, or high/high surrogate escapes fail in
+keys and values. Mirrored corpora also prove the exact 32-level acceptance and
+33-level rejection boundary.
+The 569-file mobile suite, typechecks, lints, reliability, max-lines, focused
+formatting, diff hygiene, and unchanged `b17ead7a…` package verification remain
+green after the parser repair.
 The remaining security work below is release-app corpus testing, fuzzing,
 cross-scope races, privacy/authorization audit, and independent review.

@@ -303,9 +311,10 @@ cross-scope races, privacy/authorization audit, and independent review.
      symlink traversal without touching external sentinels; Android quota
      accounting ignores linked external bytes. Staged-asset and activation
      writes also reject linked parents, while atomic activation replacement
-      preserves an external file behind an in-cache link. A fresh exact-app
-      rerun, further generated mutation, concurrent cache mutation, and the
-      other listed boundaries remain.
+      preserves an external file behind an in-cache link. Exact JSON now rejects
+      unpaired Unicode surrogate escapes and has explicit depth-edge coverage.
+      A fresh exact-app rerun, further generated mutation, concurrent cache
+      mutation, and the other listed boundaries remain.
 - [ ] Attempt cross-host, cross-build, cross-workspace, cross-session, replay,
      reconnect, process-loss, and host-removal races.
 - [ ] Verify no credential or privileged host identity reaches URLs, DOM state,
@@ -72,13 +72,23 @@ private class MobileWebExactJsonParser(
      val character = value[index++]
      if (character == '"') return value.substring(start, index)
      if (character.code < 0x20) return null
+      if (character.isHighSurrogate()) {
+        if (index >= value.length || !value[index].isLowSurrogate()) return null
+        index += 1
+        continue
+      }
+      if (character.isLowSurrogate()) return null
      if (character != '\\') continue
      if (index >= value.length) return null
      val escaped = value[index++]
      if (escaped == 'u') {
-        if (index + 4 > value.length) return null
-        repeat(4) {
-          if (!value[index++].isHexDigit()) return null
+        val codeUnit = parseUnicodeCodeUnit() ?: return null
+        if (codeUnit in 0xD800..0xDBFF) {
+          if (!consume('\\') || !consume('u')) return null
+          val lowSurrogate = parseUnicodeCodeUnit() ?: return null
+          if (lowSurrogate !in 0xDC00..0xDFFF) return null
+        } else if (codeUnit in 0xDC00..0xDFFF) {
+          return null
        }
      } else if (escaped !in "\"\\/bfnrt") {
        return null
@@ -87,6 +97,16 @@ private class MobileWebExactJsonParser(
    return null
  }

+  private fun parseUnicodeCodeUnit(): Int? {
+    if (index + 4 > value.length) return null
+    var codeUnit = 0
+    repeat(4) {
+      val digit = value[index++].hexValue() ?: return null
+      codeUnit = (codeUnit shl 4) or digit
+    }
+    return codeUnit
+  }
+
  private fun parseNumber(): Boolean {
    val start = index
    consume('-')
@@ -137,8 +157,16 @@ private class MobileWebExactJsonParser(
    null
  }

-  private fun Char.isHexDigit(): Boolean =
-    isJsonDigit() || this in 'A'..'F' || this in 'a'..'f'
+  private fun Char.hexValue(): Int? = when (this) {
+    in '0'..'9' -> code - '0'.code
+    in 'A'..'F' -> code - 'A'.code + 10
+    in 'a'..'f' -> code - 'a'.code + 10
+    else -> null
+  }

  private fun Char.isJsonDigit(): Boolean = this in '0'..'9'
+
+  private fun Char.isHighSurrogate(): Boolean = code in 0xD800..0xDBFF
+
+  private fun Char.isLowSurrogate(): Boolean = code in 0xDC00..0xDFFF
 }
@@ -10,11 +10,13 @@ class MobileWebExactJsonTest {
    val valid = listOf(
      "{}",
      """{"a":1,"b":[true,false,null,{"c":"\u0063"}]}""",
-      """{"a":-1.25e+2}"""
+      """{"a":-1.25e+2}""",
+      """{"emoji":"\uD83D\uDE00"}""",
+      """{"emoji":"😀"}""",
+      nestedObject(32)
    )
    valid.forEach { assertTrue(it, isExactMobileWebJsonDocument(it)) }

-    val deeplyNested = """{"a":""".repeat(34) + "0" + "}".repeat(34)
    val invalid = listOf(
      "",
      """{"a":1} trailing""",
@@ -27,8 +29,15 @@ class MobileWebExactJsonTest {
      """{"a":1٢}""",
      """{"a":1,}""",
      """{"a":"\x"}""",
-      deeplyNested
+      """{"a":"\uD800"}""",
+      """{"a":"\uDC00"}""",
+      """{"a":"\uD800\uD800"}""",
+      """{"\uD800":1}""",
+      nestedObject(33)
    )
    invalid.forEach { assertFalse(it, isExactMobileWebJsonDocument(it)) }
  }
+
+  private fun nestedObject(depth: Int): String =
+    """{"a":""".repeat(depth) + "0" + "}".repeat(depth)
 }
@@ -6,13 +6,12 @@ enum MobileWebExactJsonTests {
      "{}",
      #"{"a":1,"b":[true,false,null,{"c":"\u0063"}]}"#,
      #"{"a":-1.25e+2}"#,
+      #"{"emoji":"\uD83D\uDE00"}"#,
+      #"{"emoji":"😀"}"#,
+      nestedObject(depth: 32),
    ]
    precondition(valid.allSatisfy(isExactMobileWebJsonDocument))

-    let deeplyNested =
-      String(repeating: #"{"a":"#, count: 34)
-      + "0"
-      + String(repeating: "}", count: 34)
    let invalid = [
      "",
      #"{"a":1} trailing"#,
@@ -25,8 +24,18 @@ enum MobileWebExactJsonTests {
      #"{"a":1٢}"#,
      #"{"a":1,}"#,
      #"{"a":"\x"}"#,
-      deeplyNested,
+      #"{"a":"\uD800"}"#,
+      #"{"a":"\uDC00"}"#,
+      #"{"a":"\uD800\uD800"}"#,
+      #"{"\uD800":1}"#,
+      nestedObject(depth: 33),
    ]
    precondition(invalid.allSatisfy { !isExactMobileWebJsonDocument($0) })
  }
+
+  private static func nestedObject(depth: Int) -> String {
+    String(repeating: #"{"a":"#, count: depth)
+      + "0"
+      + String(repeating: "}", count: depth)
+  }
 }
@@ -88,11 +88,19 @@ private struct MobileWebExactJsonParser {
      let escaped = bytes[index]
      index += 1
      if escaped == 0x75 {
-        guard index + 4 <= bytes.count else { return nil }
-        for byte in bytes[index..<(index + 4)] where !isHexDigit(byte) {
+        guard let codeUnit = parseUnicodeCodeUnit() else { return nil }
+        if (0xD800...0xDBFF).contains(codeUnit) {
+          guard
+            consumeByte(0x5C),
+            consumeByte(0x75),
+            let lowSurrogate = parseUnicodeCodeUnit(),
+            (0xDC00...0xDFFF).contains(lowSurrogate)
+          else {
+            return nil
+          }
+        } else if (0xDC00...0xDFFF).contains(codeUnit) {
          return nil
        }
-        index += 4
      } else if ![0x22, 0x5C, 0x2F, 0x62, 0x66, 0x6E, 0x72, 0x74].contains(escaped) {
        return nil
      }
@@ -100,6 +108,17 @@ private struct MobileWebExactJsonParser {
    return nil
  }

+  private mutating func parseUnicodeCodeUnit() -> UInt16? {
+    guard index + 4 <= bytes.count else { return nil }
+    var codeUnit: UInt16 = 0
+    for byte in bytes[index..<(index + 4)] {
+      guard let digit = hexValue(byte) else { return nil }
+      codeUnit = (codeUnit << 4) | UInt16(digit)
+    }
+    index += 4
+    return codeUnit
+  }
+
  private mutating func parseNumber() -> Bool {
    let start = index
    _ = consumeByte(0x2D)
@@ -171,7 +190,10 @@ private struct MobileWebExactJsonParser {
    (0x31...0x39).contains(byte)
  }

-  private func isHexDigit(_ byte: UInt8) -> Bool {
-    isDigit(byte) || (0x41...0x46).contains(byte) || (0x61...0x66).contains(byte)
+  private func hexValue(_ byte: UInt8) -> UInt8? {
+    if (0x30...0x39).contains(byte) { return byte - 0x30 }
+    if (0x41...0x46).contains(byte) { return byte - 0x41 + 10 }
+    if (0x61...0x66).contains(byte) { return byte - 0x61 + 10 }
+    return nil
  }
 }