Pipeline NFC writes to avoid delayed ACK and RTT stalls

Send each AIO write as one buffer with TCP_NODELAY, and keep four
IOs in flight so FastLZ and multi-chunk writes are not serialized.

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Lucian Petrut
2026-09-07 13:51:24 +00:00
co-authored by Cursor
parent 33b6a3967b
commit d0a6de03d2
3 changed files with 60 additions and 28 deletions
+8 -5
View File
@@ -67,7 +67,9 @@ sends type `0` and raw extra (same as an uncompressed write).
Sector bytes follow the 44-byte payload and are **not** counted in AIO
`size`. VDDK sends header + payload + data in one `write()`. The
replacement may split that into two `sendall`s; TCP does not care.
replacement does the same (`sendall` of those bytes together) and sets
`TCP_NODELAY` on the NFC socket so a small FastLZ extra is not delayed
behind Nagle / delayed ACK.
The server replies with a type-7 header and a 44-byte payload for that
`opId`. There is no extra data on the write reply (unlike reads).
@@ -76,10 +78,11 @@ A 1-sector VDDK write was 572 bytes on the wire: 16 + 44 + 512.
## Client-side split
`NfcAioInitSession` advertises a 64 KiB buffer. VDDK splits writes
larger than that into 64 KiB chunks (VDDK programming guide). The
Python client does the same: several IO requests of at most
`NFC_AIO_BUFFER_SIZE` bytes, each with its own `opId`.
`NfcAioInitSession` advertises a 64 KiB buffer and count 4. VDDK splits
writes larger than 64 KiB into 64 KiB chunks (VDDK programming guide)
and keeps several IOs in flight. The Python client does the same: IO
requests of at most `NFC_AIO_BUFFER_SIZE` bytes, up to
`NFC_AIO_BUFFER_COUNT` outstanding `opId`s before waiting for a reply.
## Python replacement
+3 -2
View File
@@ -248,8 +248,9 @@ the ticket switched from `NfcGetVmFiles` to `NfcRandomAccessOpenDisk`
`NfcRandomAccessOpenDisk`). Integration tests create a temporary empty
10 GiB VM for the run so writes cannot land on other lab disks.
The Python client splits writes larger than 64 KiB; it does not send a
single oversized write the way VDDK sends an oversized read. Details:
The Python client splits writes larger than 64 KiB into AIO chunks and
keeps up to four in flight (`NfcAioInitSession` buffer count). Header
and extra go in one `sendall`, with `TCP_NODELAY`. Details:
`docs/nfc_write.md`. Proof: write then read in
`tests/integration/test_nfc_read_write.py` and the VDDK cross-check in
`tests/integration/test_crosscheck.py`.