~0x64616E69656C
nxTLS · EXP 04 / QUIC Reload · 2026-10-09

QUIC reload.
The fix belongs upstream.

Can nginx keep existing HTTP/3 connections alive while admitting new ones during a graceful reload? An independent before/after test of the merged QUIC BPF change, using the same pinned BoringSSL build and exact nginx commits.

Upstream validated, now pinned.
Original differential: 13/32 before versus 32/32 after. Official nginx 1.31.7 release regression: 48/48 fresh H3 connections and 4/4 established transfers passed. No nxTLS-specific reload patch.
October 9 Git differential · October 18 official 1.31.7 release smoke SIGHUP only · loopback H3 · independent validation
Independent experiment · 9 October 2026. Results were obtained on the nxTLS Debian build host from an isolated HTTP/3 test harness. The compared nginx trees were staged from exact upstream Git revisions. At the time, nxTLS still pinned nginx 1.31.6, and the vendor source was not modified for this experiment. The October 18 release validation above is a later, separate result.
Release follow-up: The sections below preserve the original October 9 Git-commit differential. On October 18, nxTLS updated its upstream pin to the official nginx 1.31.7 release and ran fresh SIGHUP regressions. These are separate tests and must not be combined into one sample count.

Official nginx 1.31.7 release — 18 October 2026

The unpatched official nginx 1.31.7 release, statically linked with pinned BoringSSL 594a44291086dcc7cef06cabec4c0b47afd674c0, was exercised with the existing isolated loopback harness. Each run used four workers, quic_bpf on, genuine curl --http3-only connections and SIGHUP reloads.

Release regressionBefore reloadFresh post-reload H3Established transfers
IPv4 · one reload16/1616/161/1
IPv6 · one reload16/1616/161/1
IPv4 · two stacked reloads8/88/8 at g1, 8/8 at g22/2

Total: 40/40 initial controls, 48/48 fresh post-reload responses and 4/4 established transfers completed with correct generation markers and response body integrity. This corroborates the earlier upstream merge-differential on the newly pinned release, without an nxTLS-specific QUIC reload patch.

Binary upgrades (USR2), BPF-map cleanup/lifetime, migration, adverse networks and non-loopback topologies remain untested. Canonical provenance: docs/UPSTREAM_REFRESH_2026-10-18.md. The original test below correctly refers to 1.31.7 development commits and the then-current 1.31.6 pin.

At a glance

Under a four-worker quic_bpf on configuration, the pre-merge nginx revision failed 19 of 32 fresh HTTP/3 requests after a reload. The merged revision accepted all 32. Additional IPv6 and multi-reload runs also passed.

IPv4 · before merge
13/32
Fresh H3 connections after reload
IPv4 · after merge
32/32
Fresh H3 connections after reload
IPv6 · after merge
32/32
Fresh H3 connections after reload
Stacked · after merge
96/96
Fresh H3 connections over three reloads
4 nginx workersLinux eBPF / reuseportHTTP/3 onlyreal curl / ngtcp2private loopback fixture

Question & mechanism

Graceful nginx reload creates a short interval in which two worker generations coexist. For HTTP/3, old workers must continue servicing their established QUIC connections, while fresh QUIC connections must reach workers that still accept new sessions.

Without suitable QUIC-aware UDP socket selection, the initial packets of a new connection can be routed to a worker that is already draining. The visible symptom in the older baseline was a fresh client that never completed an HTTP/3 response. This experiment does not trace individual kernel socket-selection decisions; the routing mechanism is the hypothesis and upstream change being evaluated.

Required behavior during the overlap
existing connection
Keep the old worker

Previously established QUIC connection IDs continue to reach their original owner.

g0 → g0   ✓
new connection
Use the new worker

A fresh QUIC connection receives an HTTP/3 response tagged with the new generation.

new Initial → g1   ✓

Exact build provenance

The principal differential uses the first parent of nginx merge commit 63af971a as the control, and the merge commit itself as the candidate. Both builds identify themselves as nginx 1.31.7 development snapshots, were built with identical flags, and link against the same pinned BoringSSL archives.

ComponentBeforeAfter
nginx Git revisionce6e7d6c12c1f21c0180805d031bb8042b19658a63af971a1ec5c6d5aac8660184c389e4a02633f0
nginx reported version1.31.7 (development)1.31.7 (development)
nginx executable SHA-256720844bd85c9c19b2f23b8f4dec447b43f7af2fba97c1dd5c8dd8bdd0902db8a13c13e1925233bc500412508232977eba4daef638c0206f22b1260f675de8076
TLS libraryBoringSSL b75f405cde1cc3c9fb811be155eecabe38f379bb (static; same built archives)
ToolchainDebian Testing / amd64 · Clang 21.1.8 · LLD · libc++
Kernel7.2.8+deb14-amd64
HTTP/3 clientcurl 8.22.0 · ngtcp2 1.24.0 · nghttp3 1.17.0
nginx modules--with-http_ssl_module · --with-http_v3_module · --with-debug

The different binary hashes are recorded for traceability. Build paths/prefixes also differ; the isolated source delta is established by the Git parent/merge pair, not by comparing the binary hashes alone.

Methodology

The harness launches its own nginx master with a loopback-only UDP listener, ephemeral certificate and separate PID/configuration. It does not send signals to a production nginx instance. The test is run with the privileges required to initialize quic_bpf on.

  1. Start nginx with four workers and issue 32 fresh curl --http3-only requests. Require HTTP 200, HTTP/3, complete body, and expected response generation g0.
  2. Begin a long-running HTTP/3 transfer on g0 and confirm it has started receiving data.
  3. Validate the next configuration, switch the response header to X-NxTLS-Generation: g1, send SIGHUP and observe the new worker PIDs.
  4. Open 32 fresh H3 connections during the overlapping worker generations. Each must return g1, a valid body and HTTP/3.
  5. Verify that the original slow transfer still reports g0 and completes with the expected full-body SHA-256.

In the principal A/B test the slow response was 4 MiB, rate-limited client-side to 64 KiB/s, and ran for approximately 63.9 seconds. Separate follow-ups exercised IPv6 and three successive IPv4 reloads with overlapping older transfers.

The fixture sets ssl_early_data off and quic_retry off. This isolates normal QUIC reload behavior rather than testing Early Data, Retry tokens or migration.

Measured results

ScenarioBeforeAfter mergeExisting H3 transfer
IPv4 · initial requests32/3232/32—
IPv4 · fresh requests after one SIGHUP13/32 (19 failed)32/32PASS / PASS
IPv6 · fresh requests after one SIGHUPNot tested32/32After: PASS (15.95 s)
IPv4 · three stacked SIGHUP reloadsNot tested96/96 (32 per reload)After: 3/3 PASS

In the matched IPv4 comparison, the slow transfer completed on both builds: 63.9058 s before and 63.9057 s after. This narrows the measured difference to the successful admission of new connections during the reload. The pre-merge 13/32 result was also observed in an earlier independent attempt.

The stacked after-merge run recorded 32/32 successes in each of reloads 1, 2 and 3, and full completion of slow_before_g1, slow_before_g2 and slow_before_g3.

Observed: The merged upstream change removed the reproduced failure for fresh HTTP/3 requests under these conditions. This is focused functional evidence, not a claim that every QUIC reload or network topology is covered.

Interpretation

The comparison is deliberately narrow: same kernel, same BoringSSL commit, same Clang toolchain, same HTTP/3 client, same test fixture, and the nginx source change introduced by a single merge commit. The before revision consistently fails a substantial fraction of new H3 requests; the merged revision passes every fresh request in the tested IPv4, IPv6 and stacked cases.

Existing connections already completed in the matched BPF-enabled before test. The most important observed improvement is therefore not a faster response or a new protocol feature: it is correct connection admission across worker generations.

Keep the behavioral proof. Let upstream own the implementation.

Limits & project decision

  • Scope: SIGHUP configuration reload, not USR2 binary upgrade or two-master upgrade lifecycles.
  • Network: isolated IPv4/IPv6 loopback listeners; no shaped WAN loss, congestion, reordering or external routing.
  • Kernel resources: the harness does not inspect long-term BPF map/socket cleanup or leak behavior.
  • QUIC features: migration, Retry, Early Data and independent client implementations were not covered.
  • Statistical reading: 32/32 or 96/96 proves success for those finite runs, not universal absence of regressions.
  • Evidence publication: this page records the observed aggregates and provenance. Raw runner output may include local test keys and is not published wholesale.
nxTLS boundary: On October 9 this experiment did not change the then-current nginx 1.31.6 pin or introduce a project-specific QUIC patch. The separate, verified October 18 refresh updated nxTLS to official nginx 1.31.7. Neither event decides production release readiness.

Upstream references

nginx issue #1323 — Full HTTP3/QUIC Reload Support ↗

nginx PR #1756 — QUIC: support BPF reload and binary upgrade ↗

Merged upstream commit 63af971a ↗

nginx QUIC and HTTP/3 documentation ↗

Local reproduction harness: experiments/quic-reload/run.py and compare.py. The experiment harness was committed to the nxTLS repository as de21428. Raw local run artifacts are not published in this HTML report.

← EXP 04 summary