chore(relay): bound the shadow health gate and apply a pending backend update on resume (#21865)

* fix(relay): bound the same-cap shadow gate and apply a resumed backend update

Two findings both adversarial reviews of tonight's merged set agree on.

The report-only shadow health gate (#21849) had `continue-on-error: true` but
no step timeout. That bounds the step's contribution to the job outcome, not
its clock. Its reads are serialised, and a failure that answers nothing slowly
— an expired credential, a project-wide Logging 429 storm — makes every read
cost its full 3 x 60 s retry budget, so the cost scales with the roll window:
roughly 8S + 2 reads for S ten-minute sub-windows. A 40-minute window is about
34 reads, or 108 minutes, against the job's `timeout-minutes: 75`. A cancelled
job cannot be absorbed by continue-on-error, fires the failure-gated cleanup
isolation on an already-restored cell, and stops the strict next-cell chain.

Give the step `timeout-minutes: 5` and the artifact upload `timeout-minutes: 2`.
A timed-out step is a failed step, which continue-on-error covers, so the job
stays green. Inside the script, stop reading after an overall four-minute
deadline and report the remaining checks unverified, so the normal outcome is a
written verdict rather than a killed process; the step timeout is then only for
a hung process. The census test pins both timeouts and that the deadline leaves
the step time to write its verdict.

The resume branch (#21860) accepted `changes == 0` with a non-empty
`backendUpdate` as complete and applied nothing, so a resumed cell silently
kept the 300-second drain and no request logging behind a green resume. That
shape means the template and MIG are converged and only this cell's reviewed
backend update is left, so apply the saved resume plan — the validator has
already bounded it to this cell's backend and neither attribute restarts an
instance — then continue as converged. Template-and-MIG drift still applies
nothing, which is what a resume means, and a stranded cell's explicit MIG
replace is unchanged.

Claude-Session: relay-same-cap-gate-timeout-and-resume

* fix(relay): raise the shadow gate bounds clear of a healthy gate's read time

A healthy gate is already minutes of serial reads on the 2-vcpu runner, so a
four-minute deadline would report unverified tails on ordinary days and stop
the shadow roll measuring the comparison it exists for. Raise both together:
the step to eight minutes and the script's own deadline to seven, keeping the
census pin that the deadline leaves the step room to write its verdict. The
job budget is unaffected: a ~14-minute cell plus eight is well inside 75.

Claude-Session: relay-same-cap-gate-timeout-and-resume

.github/workflows/cloud-deploy-relay-production-same-cap-job.yml

@@ -626,7 +626,7 @@ jobs:
                     | select(.change.actions | any(. != "no-op" and . != "read"))
                     | .address] | join(","))'
             echo 'requiring reviewed rollback-image drift'
-            terraform -chdir=infra/terraform show -json \
+            RESUME_REVIEW="$(terraform -chdir=infra/terraform show -json \
               "${RUNNER_TEMP}/relay-same-cap-resume.tfplan" \
               | node dev/scripts/validate-relay-capacity-plan.mjs \
                 --mode same-cap-cell --cell-id "${TARGET_CELL_ID}" \
@@ -638,9 +638,23 @@ jobs:
                 --rehome-director-service-account "${DIRECTOR_RUNTIME_SERVICE_ACCOUNT}" \
                 --rehome-audience https://relay.onorca.dev/v1/admin/host-drain \
                 --regional-rehome-protocol "${DESIRED_REHOME_PROTOCOL}" \
-                "${POOL_ARGUMENTS[@]}" \
-              | jq -e '.changes == 2
-                  or (.changes == 0 and ((.backendUpdate // []) | length) > 0)' >/dev/null
+                "${POOL_ARGUMENTS[@]}")"
+            echo "${RESUME_REVIEW}"
+            jq -e '.changes == 2
+                or (.changes == 0 and ((.backendUpdate // []) | length) > 0)' \
+              <<< "${RESUME_REVIEW}" >/dev/null
+            # changes == 0 here means the template and MIG are converged and this cell's
+            # reviewed backend update is the only thing left, so the resume is not complete:
+            # apply it, or the cell silently keeps the 300-second drain and no request
+            # logging and the operator reads that as a finished roll. The plan holds nothing
+            # else (the validator bounded it to this cell's backend, and the template and MIG
+            # are no-ops in it), and neither attribute restarts an instance, so the
+            # incarnation check downstream stays honest. Template-and-MIG drift still applies
+            # nothing, which is what a resume means.
+            if test "$(jq -er '.changes' <<< "${RESUME_REVIEW}")" = 0; then
+              terraform -chdir=infra/terraform apply -auto-approve \
+                "${RUNNER_TEMP}/relay-same-cap-resume.tfplan"
+            fi
           fi
           gcloud compute instance-groups managed wait-until "${MIG_NAME}" --stable \
             --project "${GCP_PROJECT_ID}" --zone "${TARGET_ZONE}" --timeout 900
@@ -829,6 +843,15 @@ jobs:
         id: shadow-gate
         if: ${{ inputs.mode != 'verify' }}
         continue-on-error: true
+        # continue-on-error bounds this step's contribution to the job outcome, not its clock,
+        # and its reads are serialised. A timed-out step is a failed step, which continue-on-error
+        # absorbs; without this bound a Logging 429 storm or an expired credential makes every
+        # read cost its full retry budget and can push the job past timeout-minutes, and a
+        # cancelled job takes the whole wave with it. A healthy gate is already minutes of
+        # serial reads, so both bounds sit above that: the script settles at seven minutes and
+        # reaching this eight is the pathological case. Eight on top of a ~14-minute cell still
+        # leaves the job's 75 minutes intact.
+        timeout-minutes: 8
         env:
           DRAIN_STARTED_AT: ${{ steps.drain.outputs.drain-started-at }}
           APPLY_STARTED_AT: ${{ steps.apply.outputs.apply-started-at }}
@@ -853,6 +876,8 @@ jobs:
       - name: Publish the shadow health gate verdict
         if: ${{ inputs.mode != 'verify' }}
         continue-on-error: true
+        # One small JSON file; a retrying upload must not spend the wave's remaining minutes either.
+        timeout-minutes: 2
         uses: actions/upload-artifact@v4
         with:
           name: relay-same-cap-shadow-gate-${{ inputs.target-cell-id }}-${{ github.run_id }}.json