Update the docs to use "OpenVixDiskLib" naming

This commit is contained in:
Lucian Petrut
2026-09-10 09:36:47 +00:00
parent e3948d2a45
commit 0f45cdf4c1
7 changed files with 50 additions and 47 deletions
+10 -7
View File
@@ -1,8 +1,11 @@
# openvixdisklib # OpenVixDiskLib
A Python replacement for VMware VDDK's `vixDiskLib` NBD path. It reads OpenVixDiskLib is an open-source Python replacement for VMware VDDK's
and writes VMDK contents over vSphere NFC without the proprietary VDDK `VixDiskLib` NBD path. It reads and writes VMDK contents over vSphere
SDK. NFC without the proprietary VDDK SDK.
The Python package is `openvixdisklib` (lowercase, following usual
Python naming).
VIM login and inventory use [pyVmomi](https://github.com/vmware/pyvmomi). VIM login and inventory use [pyVmomi](https://github.com/vmware/pyvmomi).
The NFC ticket, ESXi authd handshake, and disk I/O were reverse-engineered The NFC ticket, ESXi authd handshake, and disk I/O were reverse-engineered
@@ -68,13 +71,13 @@ VDDK-shaped handle.
| `openvixdisklib/nfc_open.py` | Classic NFC handshake, AIO open, sector read/write | | `openvixdisklib/nfc_open.py` | Classic NFC handshake, AIO open, sector read/write |
| `openvixdisklib/fastlz.py` | FastLZ NFC adapter (pip `pyfastlz`) | | `openvixdisklib/fastlz.py` | FastLZ NFC adapter (pip `pyfastlz`) |
| `tests/integration/` | Live pytest suite against a lab vCenter | | `tests/integration/` | Live pytest suite against a lab vCenter |
| `tests/perf/` | Throughput comparison of openvixdisklib vs VDDK | | `tests/perf/` | Throughput comparison of OpenVixDiskLib vs VDDK |
| `tests/stress/` | Repeated connect/open/close leak check | | `tests/stress/` | Repeated connect/open/close leak check |
| `tests/integration/vixdisklib.py` | Native VDDK wrapper used only to cross-check | | `tests/integration/vixdisklib.py` | Native VDDK wrapper used only to cross-check |
| `docs/` | Protocol notes and reverse-engineering steps | | `docs/` | Protocol notes and reverse-engineering steps |
VDDK shared libraries, if present for cross-check, belong in `.vddk/` VDDK shared libraries, if present for cross-check, belong in `.vddk/`
(gitignored). They are not required to use `openvixdisklib`. (gitignored). They are not required to use OpenVixDiskLib.
## Tests ## Tests
@@ -107,7 +110,7 @@ Some tests are marked as ``slow`` and skipped unless you pass ``--runslow``:
tox -e integration -- --runslow tox -e integration -- --runslow
``` ```
Compare write/read throughput of openvixdisklib and native VDDK Compare write/read throughput of OpenVixDiskLib and native VDDK
(`64KiB`, 129-sector, and `32MiB` transfers; `nbdssl` and `nbd`; (`64KiB`, 129-sector, and `32MiB` transfers; `nbdssl` and `nbd`;
plain and FastLZ): plain and FastLZ):
+9 -9
View File
@@ -1,7 +1,7 @@
# VDDK NFC authentication # VDDK NFC authentication
This document records how VMware VDDK authenticates for NBD/NFC disk This document records how VMware VDDK authenticates for NBD/NFC disk
access, and how the Python replacement in `openvixdisklib/nfc_auth.py` access, and how OpenVixDiskLib (`openvixdisklib/nfc_auth.py`)
reproduces that path. Findings come from VDDK 8.0.2 libraries reproduces that path. Findings come from VDDK 8.0.2 libraries
(`libvixDiskLib`, `libvddkVimAccess`, `libvim-types`), live SOAP calls (`libvixDiskLib`, `libvddkVimAccess`, `libvim-types`), live SOAP calls
against vCenter against vCenter
@@ -54,7 +54,7 @@ rather than crafting SOAP.
- SOAPAction: `"urn:vim25/8.0.1.0"` (negotiated) - SOAPAction: `"urn:vim25/8.0.1.0"` (negotiated)
VDDK logs this as `Connected to VIM Server` / `Authenticating user` / VDDK logs this as `Connected to VIM Server` / `Authenticating user` /
`Logged in!`. The Python replacement keeps that `ServiceInstance` and `Logged in!`. OpenVixDiskLib keeps that `ServiceInstance` and
its stub for the ticket call. its stub for the ticket call.
Direct ESXi login is the same SOAP login against hostd, but the NFC Direct ESXi login is the same SOAP login against hostd, but the NFC
@@ -76,10 +76,10 @@ ships. vCenter still implements it:
`ServiceManager.QueryServiceList` does **not** list NFC. The moref is `ServiceManager.QueryServiceList` does **not** list NFC. The moref is
hardcoded in VDDK as `nfcService` (vCenter) or `ha-nfc` (ESXi). hardcoded in VDDK as `nfcService` (vCenter) or `ha-nfc` (ESXi).
`openvixdisklib/nfc_auth.py` registers the missing type with OpenVixDiskLib (`openvixdisklib/nfc_auth.py`) registers the missing
`pyVmomi.VmomiSupport.CreateManagedType` and invokes it on the existing type with `pyVmomi.VmomiSupport.CreateManagedType` and invokes it on
SmartConnect stub, so serialization, cookies, and `HostServiceTicket` the existing SmartConnect stub, so serialization, cookies, and
stay in pyVmomi. `HostServiceTicket` stay in pyVmomi.
### Methods VDDK actually calls ### Methods VDDK actually calls
@@ -253,7 +253,7 @@ are for local ESXi credentials. With a vCenter ticket:
argument is not what VDDK sends. The SHA-1 value is for verifying the argument is not what VDDK sends. The SHA-1 value is for verifying the
TLS certificate, not for the `THUMBPRINT_SHA2` command. TLS certificate, not for the `THUMBPRINT_SHA2` command.
## Python replacement ## OpenVixDiskLib
| Piece | Module | Reuses pyVmomi? | | Piece | Module | Reuses pyVmomi? |
| -------------------- | ---------------------------------------- | ---------------------------------- | | -------------------- | ---------------------------------------- | ---------------------------------- |
@@ -282,6 +282,6 @@ and asserts an established TLS socket on `ticket.host:ticket.port`.
Authentication stops at `200 Connect ha-nfc` (NBD) or `200 Connect Authentication stops at `200 Connect ha-nfc` (NBD) or `200 Connect
ha-nfcssl` (NBDSSL). Opening the VMDK and reading or writing sectors is ha-nfcssl` (NBDSSL). Opening the VMDK and reading or writing sectors is
documented in `docs/nfc_open.md` and implemented in documented in `docs/nfc_open.md` and implemented in OpenVixDiskLib
`openvixdisklib/nfc_open.py`. The datastore path is consumed there (`openvixdisklib/nfc_open.py`). The datastore path is consumed there
(and, for writes, as `diskDeviceKey` on the ticket). (and, for writes, as `diskDeviceKey` on the ticket).
+3 -3
View File
@@ -1,8 +1,8 @@
# VDDK NFC disk open # VDDK NFC disk open
This document records how VMware VDDK opens a VMDK over NBD/NFC after This document records how VMware VDDK opens a VMDK over NBD/NFC after
the authd handshake in `docs/nfc_auth.md`, and how the authd handshake in `docs/nfc_auth.md`, and how OpenVixDiskLib
`openvixdisklib/nfc_open.py` reproduces that path. Findings come from (`openvixdisklib/nfc_open.py`) reproduces that path. Findings come from
VDDK 8.0.2 verbose logs VDDK 8.0.2 verbose logs
(`vixDiskLib.nfc.LogLevel=4`) plus an `LD_PRELOAD` intercept of (`vixDiskLib.nfc.LogLevel=4`) plus an `LD_PRELOAD` intercept of
`write` / `read` on the ESXi:902 file descriptor. Capture method: `write` / `read` on the ESXi:902 file descriptor. Capture method:
@@ -203,7 +203,7 @@ and `docs/nfc_write.md`. `NfcDisk.read` / `NfcDisk.write` match
`CLOSE_FILE` (handle as `uint64`), `CLOSE_SESSION` (`uint32` 0), then `CLOSE_FILE` (handle as `uint64`), `CLOSE_SESSION` (`uint32` 0), then
classic type 4 `NFC_SESSION_COMPLETE`. classic type 4 `NFC_SESSION_COMPLETE`.
## Python replacement ## OpenVixDiskLib
| Piece | Module | | Piece | Module |
| ----------------------------- | ----------------------------------------------- | | ----------------------------- | ----------------------------------------------- |
+7 -6
View File
@@ -1,9 +1,10 @@
# VDDK NFC disk read # VDDK NFC disk read
This document records how VMware VDDK reads VMDK sectors over NBD/NFC This document records how VMware VDDK reads VMDK sectors over NBD/NFC
after the open in `docs/nfc_open.md`, and how `NfcDisk.read` in after the open in `docs/nfc_open.md`, and how OpenVixDiskLib
`openvixdisklib/nfc_open.py` reproduces `VixDiskLib_Read`. Capture (`NfcDisk.read` in `openvixdisklib/nfc_open.py`) reproduces
method: `docs/reverse_engineering_procedure.md`. `VixDiskLib_Read`. Capture method:
`docs/reverse_engineering_procedure.md`.
## Mapping from VDDK ## Mapping from VDDK
@@ -51,8 +52,8 @@ Little-endian, after the usual 16-byte AIO header
An earlier guess that offset 36 was `NFC_DISK` (`2`) was wrong: a An earlier guess that offset 36 was `NFC_DISK` (`2`) was wrong: a
1-sector VDDK read puts `512` in both `uint32` length fields. A Python 1-sector VDDK read puts `512` in both `uint32` length fields. A Python
read that sent `(512, 2, 0)` still worked for one sector; the read that sent `(512, 2, 0)` still worked for one sector; OpenVixDiskLib
replacement now matches VDDK. now matches VDDK.
`VIXDISKLIB_FLAG_OPEN_COMPRESSION_FASTLZ` does not change OPEN_FILE `VIXDISKLIB_FLAG_OPEN_COMPRESSION_FASTLZ` does not change OPEN_FILE
flags. The IO opcode at offset 8 is a `uint64`: low 32 bits are still flags. The IO opcode at offset 8 is a `uint64`: low 32 bits are still
@@ -121,7 +122,7 @@ back. An unwritten region is zeros.
Writes use the same 44-byte IO payload with opcode `2`; see Writes use the same 44-byte IO payload with opcode `2`; see
`docs/nfc_write.md`. `docs/nfc_write.md`.
## Python replacement ## OpenVixDiskLib
`NfcDisk.read(start_sector, num_sectors)` in `NfcDisk.read(start_sector, num_sectors)` in
`openvixdisklib/nfc_open.py`. Run: `openvixdisklib/nfc_open.py`. Run:
+6 -6
View File
@@ -1,7 +1,7 @@
# VDDK NFC disk write # VDDK NFC disk write
This document records how `NfcDisk.write` in This document records how OpenVixDiskLib (`NfcDisk.write` in
`openvixdisklib/nfc_open.py` implements `VixDiskLib_Write` over NFC AIO. `openvixdisklib/nfc_open.py`) implements `VixDiskLib_Write` over NFC AIO.
The request layout matches the captured `VixDiskLib_Read` IO message in The request layout matches the captured `VixDiskLib_Read` IO message in
`docs/nfc_read.md` (write requests use the same fragment fields as `docs/nfc_read.md` (write requests use the same fragment fields as
read replies). Open flags and the IO direction field were taken read replies). Open flags and the IO direction field were taken
@@ -75,7 +75,7 @@ sends type `0` and raw extra. Each fragment is compressed on its own;
a 32 MiB FastLZ write is 512 independent FastLZ extras, not one. a 32 MiB FastLZ write is 512 independent FastLZ extras, not one.
Sector bytes follow the 44-byte payload and are **not** counted in AIO Sector bytes follow the 44-byte payload and are **not** counted in AIO
`size`. The replacement sends header + payload + extra in one `size`. OpenVixDiskLib sends header + payload + extra in one
`sendall` and sets `TCP_NODELAY` on the NFC socket so a small FastLZ `sendall` and sets `TCP_NODELAY` on the NFC socket so a small FastLZ
extra is not delayed behind Nagle / delayed ACK. Captured VDDK often extra is not delayed behind Nagle / delayed ACK. Captured VDDK often
uses two `write()`s (`60` then `65536`) for a 64 KiB fragment and uses two `write()`s (`60` then `65536`) for a 64 KiB fragment and
@@ -98,7 +98,7 @@ C: type=7 opId=14 size=44 total=66048 dest=65536 chunk=512 + 512 data
S: type=7 opId=14 size=44 total=66048 dest=0 chunk=66048 S: type=7 opId=14 size=44 total=66048 dest=0 chunk=66048
``` ```
A 32 MiB write is 512 client fragments and one ACK. The Python client A 32 MiB write is 512 client fragments and one ACK. OpenVixDiskLib
does the same. An earlier attempt that used a distinct `opId` per does the same. An earlier attempt that used a distinct `opId` per
64 KiB chunk and a sliding window of 4–512 outstanding IOs was waiting 64 KiB chunk and a sliding window of 4–512 outstanding IOs was waiting
for one reply per chunk; raising the window did not match VDDK for one reply per chunk; raising the window did not match VDDK
@@ -109,7 +109,7 @@ client merge of API writes. OPEN_SESSION is 16 zero bytes both ways, so
the logged AIO buffer count of 4 is a VDDK client default the logged AIO buffer count of 4 is a VDDK client default
(`vixDiskLib.nfcAio.Session.BufCount`), not a server cap. (`vixDiskLib.nfcAio.Session.BufCount`), not a server cap.
## Python replacement ## OpenVixDiskLib
`NfcDisk.write(start_sector, num_sectors, data)` in `NfcDisk.write(start_sector, num_sectors, data)` in
`openvixdisklib/nfc_open.py`. `open_disk(..., read_only=False)` selects `openvixdisklib/nfc_open.py`. `open_disk(..., read_only=False)` selects
@@ -120,5 +120,5 @@ flags `0x1a`. The drop-in handle exposes the same shape as VDDK:
Integration tests create an empty 10 GiB disk, write known patterns, Integration tests create an empty 10 GiB disk, write known patterns,
and read them back (`tests/integration/test_nfc_read_write.py`, and read them back (`tests/integration/test_nfc_read_write.py`,
`tests/integration/test_openvixdisklib.py`). Cross-check tests write `tests/integration/test_openvixdisklib.py`). Cross-check tests write
with VDDK and with the replacement and read with both with VDDK and with OpenVixDiskLib and read with both
(`tests/integration/test_crosscheck.py`). (`tests/integration/test_crosscheck.py`).
+11 -11
View File
@@ -1,7 +1,7 @@
# Reverse-engineering procedure # Reverse-engineering procedure
This is the working method used to replace VDDK’s NBD path with a Python This is the working method used to replace VDDK’s NBD path with
implementation. Protocol details live in `docs/nfc_auth.md`, OpenVixDiskLib. Protocol details live in `docs/nfc_auth.md`,
`docs/nfc_open.md`, `docs/nfc_read.md`, and `docs/nfc_write.md`. `docs/nfc_open.md`, `docs/nfc_read.md`, and `docs/nfc_write.md`.
The capture tool is described in `docs/ssl_hook.md`. The capture tool is described in `docs/ssl_hook.md`.
This file is the **sequence of steps**, including dead ends, so later This file is the **sequence of steps**, including dead ends, so later
@@ -42,7 +42,7 @@ wrong wire command until the intercept existed.
| Pickled `LabEnv` | `/tmp/vddk-write-wire-lab.pkl` during hooked captures only | | Pickled `LabEnv` | `/tmp/vddk-write-wire-lab.pkl` during hooked captures only |
Always set `LD_LIBRARY_PATH` to `.vddk/` so VDDK uses its own Always set `LD_LIBRARY_PATH` to `.vddk/` so VDDK uses its own
`libssl.so.3`. Unset `LD_PRELOAD` before running the Python replacement; `libssl.so.3`. Unset `LD_PRELOAD` before running OpenVixDiskLib;
a leftover `write` hook will crash pyVmomi’s TLS. a leftover `write` hook will crash pyVmomi’s TLS.
## Tools ## Tools
@@ -57,7 +57,7 @@ names is in this table.
| `strings -a` on `.vddk/*.so` | Candidate tokens (`SESSION`, `NfcGetVmFiles`, `NFC_AIO_MSG_*`) | Not command order, spacing, or replies | | `strings -a` on `.vddk/*.so` | Candidate tokens (`SESSION`, `NfcGetVmFiles`, `NFC_AIO_MSG_*`) | Not command order, spacing, or replies |
| `nm -D` / `objdump -T` | Which library imports `SSL_write` vs `write`; exported APIs | Not wire layout | | `nm -D` / `objdump -T` | Which library imports `SSL_write` vs `write`; exported APIs | Not wire layout |
| VDDK `vixDiskLib.nfc.LogLevel=4` | Function names and AIO `opId` / `type` / `size` to label a frame | Not magic numbers, path placement, or `BANNER \r\n` | | VDDK `vixDiskLib.nfc.LogLevel=4` | Function names and AIO `opId` / `type` / `size` to label a frame | Not magic numbers, path placement, or `BANNER \r\n` |
| `LD_PRELOAD` SSL / `write` hook | Plaintext of SOAP, authd, and (after PROXY) NFC on fd 902 | Must not stay on the replacement process; `docs/ssl_hook.md` | | `LD_PRELOAD` SSL / `write` hook | Plaintext of SOAP, authd, and (after PROXY) NFC on fd 902 | Must not stay on the OpenVixDiskLib process; `docs/ssl_hook.md` |
| `strace -f -x` on `write` / `send*` | First writable NFC capture without rebuilding the hook (Step 10) | Noisy; TLS still opaque; `-s` truncates large extras | | `strace -f -x` on `write` / `send*` | First writable NFC capture without rebuilding the hook (Step 10) | Noisy; TLS still opaque; `-s` truncates large extras |
| `pickle` of `LabEnv` | Create the temp VM unhooked, then load it under the hook | `/tmp` only; never commit pickles (lab host and credentials) | | `pickle` of `LabEnv` | Create the temp VM unhooked, then load it under the hook | `/tmp` only; never commit pickles (lab host and credentials) |
| ctypes drivers in `docs/probing_samples/` | Repeatable `ConnectEx` / `Open` / `Read` / `Write` under capture | Not library code | | ctypes drivers in `docs/probing_samples/` | Repeatable `ConnectEx` / `Open` / `Read` / `Write` under capture | Not library code |
@@ -172,7 +172,7 @@ parameter names and which moref vCenter accepts:
ticket. `GetVmFiles` plus `OPEN_FILE` flags `0x1a` fails with ticket. `GetVmFiles` plus `OPEN_FILE` flags `0x1a` fails with
`VIX_E_FILE_READ_ONLY` (`0x0b`). `VIX_E_FILE_READ_ONLY` (`0x0b`).
The replacement registers those methods and calls them through pyVmomi. OpenVixDiskLib registers those methods and calls them through pyVmomi.
It does not ship a hand-rolled SOAP client for login or tickets. It does not ship a hand-rolled SOAP client for login or tickets.
## Step 6 — Probe authd; record dead ends ## Step 6 — Probe authd; record dead ends
@@ -230,10 +230,10 @@ Python must **dup the authd fd** and send NFC as raw TCP.
`SSLSocket.send` would encrypt; `unwrap()` would `SSL_shutdown`. VDDK `SSLSocket.send` would encrypt; `unwrap()` would `SSL_shutdown`. VDDK
does neither. does neither.
`openvixdisklib/nfc_open.py` replays handshake + AIO `OPEN_SESSION` / OpenVixDiskLib (`openvixdisklib/nfc_open.py`) replays handshake + AIO
sockopts / resource pool / `OPEN_FILE`. VDDK’s extra `DDB_GET` keys `OPEN_SESSION` / sockopts / resource pool / `OPEN_FILE`. VDDK’s extra
were omitted once a file handle was enough to read. Proof of open: `DDB_GET` keys were omitted once a file handle was enough to read. Proof
`tests/integration/test_nfc_open.py`. of open: `tests/integration/test_nfc_open.py`.
Do not copy every VDDK message. Copy what the server requires for the Do not copy every VDDK message. Copy what the server requires for the
Python API you are replacing. Python API you are replacing.
@@ -395,8 +395,8 @@ After a stage works:
| `docs/ssl_hook.md` | Capture tool only | | `docs/ssl_hook.md` | Capture tool only |
| `docs/reverse_engineering_procedure.md` | This procedure (update when the method changes) | | `docs/reverse_engineering_procedure.md` | This procedure (update when the method changes) |
Keep the hook and ctypes driver under `/tmp`. They are not part of the Keep the hook and ctypes driver under `/tmp`. They are not part of
replacement library. OpenVixDiskLib.
## Next stages (same procedure) ## Next stages (same procedure)
+4 -5
View File
@@ -9,8 +9,7 @@ This project used a small `LD_PRELOAD` library (`sslhook.c`, built to
`VixDiskLib_ConnectEx` / `VixDiskLib_Open`. The authd sequence in `VixDiskLib_ConnectEx` / `VixDiskLib_Open`. The authd sequence in
`docs/nfc_auth.md` was recovered from that log, not from VDDK source. `docs/nfc_auth.md` was recovered from that log, not from VDDK source.
The hook is a reverse-engineering aid. It is not part of the Python The hook is a reverse-engineering aid. It is not part of OpenVixDiskLib.
NFC client.
## Why not tcpdump or VDDK logs ## Why not tcpdump or VDDK logs
@@ -124,8 +123,8 @@ Details that only the hex dump made obvious:
- `SESSION` has no reply; waiting for a line after it looks like a hang. - `SESSION` has no reply; waiting for a line after it looks like a hang.
- Ticket `sessionId` is the UUID string on the `SESSION` line. - Ticket `sessionId` is the UUID string on the `SESSION` line.
Those facts are written up in `docs/nfc_auth.md`. The Python client in Those facts are written up in `docs/nfc_auth.md`. OpenVixDiskLib
`openvixdisklib/nfc_auth.py` replays this sequence; it does not use (`openvixdisklib/nfc_auth.py`) replays this sequence; it does not use
the hook. the hook.
After `200 Connect`, NFC is **not** on `SSL_write`. VDDK uses After `200 Connect`, NFC is **not** on `SSL_write`. VDDK uses
@@ -142,5 +141,5 @@ skipped.
`SSL *`. `SSL *`.
- It does not decode TLS handshakes, certificates, or SOAP envelopes; - It does not decode TLS handshakes, certificates, or SOAP envelopes;
that is done offline on the hex log. that is done offline on the hex log.
- It must not ship in a production VDDK replacement. Keep it out of - It must not ship in OpenVixDiskLib. Keep it out of
the library path used by `openvixdisklib/nfc_auth.py`. the library path used by `openvixdisklib/nfc_auth.py`.