Lucian Petrut
2099591245
VDDK always decompresses the chunks it retrieves. However, some callers may be interested in the compressed chunks, which may be forwarded to another service involved in the backup process. We'll add a read flag (skip_decompression). If set, the read operation will return a "ReadResult" container, containing a list of fragments and their compressed / decompressed lengths. If the returned buffer size matches the AIO buffer size (which now becomes configurable), at most one fragment will be returned. While at it, we're adding perf tests that check various AIO buffer sizes, cross checking against VDDK.
126 lines
5.9 KiB
Markdown
126 lines
5.9 KiB
Markdown
# VDDK NFC disk write
|
||
|
||
This document records how OpenVixDiskLib (`NfcDisk.write` in
|
||
`openvixdisklib/nfc_open.py`) implements `VixDiskLib_Write` over NFC AIO.
|
||
The request layout matches the captured `VixDiskLib_Read` IO message in
|
||
`docs/nfc_read.md` (write requests use the same fragment fields as
|
||
read replies). Open flags and the IO direction field were taken
|
||
from a `strace` of VDDK 8 writing one sector to a temporary 10 GiB
|
||
disk (`docs/reverse_engineering_procedure.md`).
|
||
|
||
The public `VixDiskLib_Write` prototype is in `.vddk/vixDiskLib.h`:
|
||
|
||
```
|
||
VixError VixDiskLib_Write(VixDiskLibHandle diskHandle,
|
||
VixDiskLibSectorType startSector,
|
||
VixDiskLibSectorType numSectors,
|
||
const uint8 *writeBuffer);
|
||
```
|
||
|
||
`ConnectEx(..., Bool readOnly, ...)` with `readOnly=FALSE` and `Open`
|
||
without `VIXDISKLIB_FLAG_OPEN_READ_ONLY` (that flag is `1 << 2` in the
|
||
same header) is what produces the writable NFC open below. The ticket
|
||
must be `NfcRandomAccessOpenDisk` (`docs/nfc_auth.md`); flags `0x1a`
|
||
on a `NfcGetVmFiles` ticket are rejected as `VIX_E_FILE_READ_ONLY`.
|
||
|
||
## Mapping from VDDK
|
||
|
||
Units on the wire are **bytes**, as for reads:
|
||
|
||
```
|
||
offset = startSector * sectorSize
|
||
length = numSectors * sectorSize
|
||
```
|
||
|
||
| VDDK call | Wire effect |
|
||
| ------------------------------------------- | ------------------------------------------------ |
|
||
| Open without `VIXDISKLIB_FLAG_OPEN_READ_ONLY` | `OPEN_FILE` flags `0x1a` |
|
||
| Open with `VIXDISKLIB_FLAG_OPEN_READ_ONLY` | `OPEN_FILE` flags `0x1e` (read-only) |
|
||
| `VixDiskLib_Write(h, start, n, buf)` | IO opcode `0`, then `n * 512` data bytes |
|
||
| `VixDiskLib_Read(h, start, n, buf)` | IO opcode `1` |
|
||
|
||
`0x1e` vs `0x1a` is bit `0x04`, the same value as
|
||
`VIXDISKLIB_FLAG_OPEN_READ_ONLY`. Writable opens clear that bit.
|
||
|
||
VDDK also issues several `DDB_GET` queries and a type-10
|
||
`GET_FILE_INFO` (`longContentID`) before the first write. They are not
|
||
required to write or read sectors.
|
||
|
||
## Request (44 bytes + data)
|
||
|
||
Little-endian, after the usual 16-byte AIO header
|
||
(`magic 0xA100DA7A`, type 7, size 44, one `opId` per
|
||
`VixDiskLib_Write`):
|
||
|
||
| Offset | Type | `Write(start, n)` |
|
||
| ------ | -------- | --------------------------------------------------------- |
|
||
| 0 | `uint64` | File handle from `OPEN_FILE` |
|
||
| 8 | `uint64` | `0` (`NFC_AIO_IO_WRITE`; read uses `1`) |
|
||
| 16 | `uint64` | Byte offset of the **whole** write |
|
||
| 24 | `uint32` | Total byte length |
|
||
| 28 | `uint32` | Byte offset of this fragment (`0`, `65536`, …) |
|
||
| 32 | `uint32` | This fragment’s uncompressed length |
|
||
| 36 | `uint32` | Extra size (same as 32, or FastLZ packed size) |
|
||
| 40 | `uint32` | `0` |
|
||
|
||
This is the same 44-byte layout as a **read reply** fragment
|
||
(`docs/nfc_read.md`): writes stream request fragments, reads stream
|
||
reply fragments. A single-fragment write (≤ 64 KiB) still looks like a
|
||
`uint64` length at offset 24 because the fragment offset is 0.
|
||
|
||
FASTLZ writes use the same header. The opcode `uint64` high half is
|
||
`2`, offset 36 is the compressed size, and FastLZ bytes follow instead
|
||
of raw sectors. If compression does not shrink the fragment, VDDK
|
||
sends type `0` and raw extra. Each fragment is compressed on its own;
|
||
a 32 MiB FastLZ write is 512 independent FastLZ extras, not one.
|
||
|
||
Sector bytes follow the 44-byte payload and are **not** counted in AIO
|
||
`size`. OpenVixDiskLib sends header + payload + extra in one
|
||
`sendall` and sets `TCP_NODELAY` on the NFC socket so a small FastLZ
|
||
extra is not delayed behind Nagle / delayed ACK. Captured VDDK often
|
||
uses two `write()`s (`60` then `65536`) for a 64 KiB fragment and
|
||
coalesces only a 512-byte tail (`572` = 16 + 44 + 512).
|
||
|
||
A 1-sector VDDK write was 572 bytes on the wire: 16 + 44 + 512.
|
||
|
||
## Fragments and the single reply
|
||
|
||
`OPEN_SESSION` advertises the AIO buffer size (VDDK default 64 KiB;
|
||
`BufSizeIn64KB` can raise it). Extra per type-7 message is at most
|
||
that size. VDDK does **not** issue a new `opId` per
|
||
chunk, and it does **not** coalesce separate `VixDiskLib_Write` calls
|
||
(eight 8 KiB writes stayed eight IOs). One public write becomes N
|
||
client type-7 messages with the **same** `opId`, then **one** 44-byte
|
||
reply (no extra) after the last fragment:
|
||
|
||
```
|
||
C: type=7 opId=14 size=44 total=66048 dest=0 chunk=65536 + 65536 data
|
||
C: type=7 opId=14 size=44 total=66048 dest=65536 chunk=512 + 512 data
|
||
S: type=7 opId=14 size=44 total=66048 dest=0 chunk=66048
|
||
```
|
||
|
||
A 32 MiB write is 512 client fragments and one ACK. OpenVixDiskLib
|
||
does the same. An earlier attempt that used a distinct `opId` per
|
||
64 KiB chunk and a sliding window of 4–512 outstanding IOs was waiting
|
||
for one reply per chunk; raising the window did not match VDDK
|
||
throughput because VDDK pays one RTT per `Write`, not per fragment.
|
||
|
||
`NfcAioFlushCoalescedWrites` is server-side (`nfcAioServer.c`), not a
|
||
client merge of API writes. Buffer count is
|
||
`vixDiskLib.nfcAio.Session.BufCount` (OPEN_SESSION offset 8); size is
|
||
`BufSizeIn64KB` (offset 4, in bytes). `docs/nfc_open.md`.
|
||
|
||
## OpenVixDiskLib
|
||
|
||
`NfcDisk.write(start_sector, num_sectors, data)` in
|
||
`openvixdisklib/nfc_open.py`. `open_disk(..., read_only=False)` selects
|
||
flags `0x1a`. The drop-in handle exposes the same shape as VDDK:
|
||
`connect(read_only=False)`, `open` without
|
||
`VIXDISKLIB_FLAG_OPEN_READ_ONLY`, then `write`.
|
||
|
||
Integration tests create an empty 10 GiB disk, write known patterns,
|
||
and read them back (`tests/integration/test_nfc_read_write.py`,
|
||
`tests/integration/test_openvixdisklib.py`). Cross-check tests write
|
||
with VDDK and with OpenVixDiskLib and read with both
|
||
(`tests/integration/test_crosscheck.py`).
|