Lucian Petrut
2099591245
VDDK always decompresses the chunks it retrieves. However, some callers may be interested in the compressed chunks, which may be forwarded to another service involved in the backup process. We'll add a read flag (skip_decompression). If set, the read operation will return a "ReadResult" container, containing a list of fragments and their compressed / decompressed lengths. If the returned buffer size matches the AIO buffer size (which now becomes configurable), at most one fragment will be returned. While at it, we're adding perf tests that check various AIO buffer sizes, cross checking against VDDK.
OpenVixDiskLib
OpenVixDiskLib is an open-source Python replacement for VMware VDDK's
VixDiskLib NBD path. It reads and writes VMDK contents over vSphere
NFC without the proprietary VDDK SDK.
AI tools (Cursor + Grok 4.6) have been heavily used to reverse engineer the NBD and NFC protocols, obtaining a working VDDK replacement in a few hours and comprehensive testing in a matter of days.
The Python package is openvixdisklib (lowercase, following usual
Python naming).
VIM login and inventory use pyVmomi.
The NFC ticket, ESXi authd handshake, and disk I/O were reverse-engineered
from VDDK 8 NBD traffic; see docs/.
Status
Implemented against vCenter 8 / ESXi 8. Default transport is nbdssl
(nbd is still available):
VixDiskLib_ConnectEx(UID credentials)VixDiskLib_Open(datastore path, read-only or read-write)VixDiskLib_Read(optionalskip_decompressionpacks FastLZ extras)VixDiskLib_Write
Not implemented: compression open flags other than FastLZ, CBT /
allocated-block queries, disk geometry (DDB_GET), encrypted disks,
and direct ESXi ha-nfc without vCenter vpxa-nfc.
Requires Python 3.10 or later.
Install
python3 -m venv .venv
.venv/bin/pip install -e .
Usage
from openvixdisklib import nfc_auth
from openvixdisklib import openvixdisklib as vixdisklib
handle = vixdisklib.VixDiskLibHandle(
vixdisklib_compatibility_version="8.0")
buf = vixdisklib.get_buffer(vixdisklib.VIXDISKLIB_SECTOR_SIZE)
thumbprint = nfc_auth.get_ssl_cert_thumbprint("vcenter.example.com")
with handle.connect(
server_name="vcenter.example.com",
thumbprint=thumbprint,
username="[email protected]",
password="secret",
vmx_spec="moref=vm-1234",
transport_modes="nbdssl",
read_only=False) as conn:
with handle.open(conn, "[datastore] vm/vm.vmdk", flags=0) as disk:
handle.write(disk, 0, 1, buf)
handle.read(disk, 0, 1, buf)
Lower-level NFC helpers live in openvixdisklib.nfc_auth and
openvixdisklib.nfc_open if you need the ticket or socket without the
VDDK-shaped handle.
Layout
| Path | Role |
|---|---|
openvixdisklib/openvixdisklib.py |
Drop-in handle (connect / open / read / write) |
openvixdisklib/nfc_auth.py |
VIM login, NFC ticket, authd on 902 |
openvixdisklib/nfc_open.py |
Classic NFC handshake, AIO open, sector read/write |
openvixdisklib/fastlz.py |
FastLZ NFC adapter (pip pyfastlz) |
tests/integration/ |
Live pytest suite against a lab vCenter |
tests/perf/ |
Throughput comparison of OpenVixDiskLib vs VDDK |
tests/stress/ |
Repeated connect/open/close leak check |
tests/integration/vixdisklib.py |
Native VDDK wrapper used only to cross-check |
docs/ |
Protocol notes and reverse-engineering steps |
VDDK shared libraries, if present for cross-check, belong in .vddk/
(gitignored). They are not required to use OpenVixDiskLib.
Tests
Lab connection settings live in .test_config.yaml at the repo root
(gitignored). Copy:
host: vcenter.example.com
port: 443
username: [email protected]
password: secret
allow_untrusted: true
datacenter: Datacenter
datastore: datastore0
A session-scoped pytest fixture creates an empty VM with a 10 GiB thin disk on that datastore and tears it down when the session ends. Tests write known patterns and read them back.
tox -e integration
# or
.venv/bin/pytest tests/integration
Some tests are marked as slow and skipped unless you pass --runslow:
tox -e integration -- --runslow
Compare write/read throughput of OpenVixDiskLib and native VDDK
(64KiB, 129-sector, and 32MiB transfers; nbdssl and nbd;
plain, FastLZ, and OpenVixDiskLib FastLZ skip_decompression;
AIO sessions 64 KiB×1, 1 MiB×1, 2 MiB×1, and 2 MiB×4).
tox -e perf
Repeat connect / open / write-read one sector / close /
disconnect 200 times (one process, sequential, like many VMs) and
assert the fd count does not grow:
tox -e stress
VDDK cross-check tests skip when libvixDiskLib is not loadable from
.vddk. tox -e integration sets LD_LIBRARY_PATH to that directory
and clears LD_PRELOAD. For a direct pytest run, do the same.
Lint and typecheck: tox -e pep8, tox -e mypy.
Documentation
| Document | Contents |
|---|---|
docs/nfc_auth.md |
Ticket SOAP and authd handshake |
docs/nfc_open.md |
Classic NFC and AIO open |
docs/nfc_read.md |
AIO IO / VixDiskLib_Read |
docs/nfc_write.md |
AIO IO / VixDiskLib_Write |
docs/ssl_hook.md |
TLS intercept used for capture |
docs/reverse_engineering_procedure.md |
How the protocol was recovered |