commit 4ddf0c015d3031c669727654d3efc2b50542765b Author: Lucian Petrut Date: Mon Sep 7 11:41:01 2026 +0000 Add openvixdisklib as an open NBD replacement for VMware VDDK. VDDK is no longer publicly distributed, so this library reverse-engineers the vSphere NFC path and exposes ConnectEx, Open, Read, and Write without the proprietary SDK. Co-authored-by: Cursor diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..7a6162c --- /dev/null +++ b/.gitignore @@ -0,0 +1,52 @@ +# These are some examples of commonly ignored file patterns. +# You should customize this list as applicable to your project. +# Learn more about .gitignore: +# https://www.atlassian.com/git/tutorials/saving-changes/gitignore + +# Compiled Python bytecode +*.py[cod] + +# Log files +*.log + +# JetBrains IDE +.idea/ + +# Unit test reports +TEST*.xml + +# Generated by MacOS +.DS_Store + +# Generated by Windows +Thumbs.db + +# Applications +*.app +*.exe +*.war + +.egg* +*.swo +*.swp +AUTHORS +ChangeLog +coriolis_provider_libvirt.egg-info/* +*__pycache__* +.coverage +.tox +.venv +.stestr +coverage.xml +cover/* +MANIFEST +nosetests.xml +coverage.html + +# Avoid republishing VDDK due to licensing constraints. +# Copy the VDDK files to .vddk for reverse engineering and cross-check +# purposes, which is safe from a licensing point of view. +.vddk + +# Lab credentials and VM/snapshot names for integration tests. +.test_config.yaml diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..caed4dc --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,68 @@ +# AI agent guidelines + +## Overview + +- This is a test project meant to bypass/replace VDDK, which is no longer + publicly available. +- The end goal is to have a Python library that can be used as a VDDK replacement + to retrieve VMware disk contents. +- Integration tests under `tests/integration/` are a good starting point for + interacting with the VMware NBD / NFC APIs. They inherit lab credentials and + VM/disk settings from `tests.integration.base.TestBase`. The base + class creates a temporary empty VM for the run. We can make use of + them to reverse engineer the VMware protocol, for example making various + calls, capturing the request and replies and then trying to determine the + structures used by the protocol. +- tcpdump may be used to intercept the communication with ESXI +- if deemed helpful, we may write a simple service that impersonates ESXI, + capturing the information sent by VDDK +- we should reuse pyVmomi for any operation that it supports. It's publicly + available and safe to use. +- `docs/` contains various documents that describe the reverse engineered + vmware APIs and resulting modules. +- Use `docs/reverse_engineering_procedure.md` to best describe the steps that + were undertaken to reverse engineer the Vmware APIs. + + +## Architecture + +- The project uses Python and must be Python 3.12 compatible. +- Library code lives in the `openvixdisklib` package (`nfc_auth`, `nfc_open`, + `openvixdisklib`). +- The `.vddk` dir contains the VDDK libraries and their dependencies, including + `libvixDiskLib`. These files shouldn't be included in git commits due to + licensing constrains. +- `tests/integration/vixdisklib.py` is a Python wrapper on top of + `libvixDiskLib`, used to cross-check the replacement against native VDDK. +- Integration tests live under `tests/integration/`, use the unittest + framework, and inherit `tests.integration.base.TestBase`. Lab vCenter + credentials, datacenter, and datastore come from repo-root + `.test_config.yaml` (gitignored; sample in `README.md`). Each test + class shares a temporary empty VM with a 10 GiB disk created in + `TestBase.setUpClass` and destroyed in `tearDownClass`. Run them with + `tox -e integration` or + `.venv/bin/python -m unittest discover -s tests/integration`. + + +## Other rules + +- AI agents should ignore folders that start with a dot, e.g. .mypy_cache, .ruff_cache, .tox +- AI agents may use the `.venv/` virtual env, it is expected to have + all project dependencies, including the `pyVmomi` vmware client +- AI agents should not generate unit or integration tests unless asked to. +- When modifying Markdown tables, the columns should be properly aligned. +- If an agent regenerates a file, avoid appending the new content, but instead + replace the file contents. We don't want duplicate definitions. +- Empty __init__.py files should not contain license headers. +- Use Linux style line endings. +- All public methods should include docstrings. Subclasses may reuse the ones + from the parent class. +- Avoid defining new methods for trivial checks such as `server.power_status == "RUNNING"`, + make the checks inline. +- Avoid removing inline comments that are still applicable. +- Agents should use type hints when the argument type can be determined. +- When writing unit tests, assert_has_calls is preferred instead of checking + the call cound and call parameters separately. +- When writing unit tests, mock decorators are preferred instead of context + managers. +- If a folder or file under this directory is inaccessible, ask for permissions. diff --git a/README.md b/README.md new file mode 100644 index 0000000..33da552 --- /dev/null +++ b/README.md @@ -0,0 +1,113 @@ +# openvixdisklib + +A Python replacement for VMware VDDK's `vixDiskLib` NBD path. It reads +and writes VMDK contents over vSphere NFC without the proprietary VDDK +SDK. + +VIM login and inventory use [pyVmomi](https://github.com/vmware/pyvmomi). +The NFC ticket, ESXi authd handshake, and disk I/O were reverse-engineered +from VDDK 8 NBD traffic; see `docs/`. + +## Status + +Implemented against vCenter 8 / ESXi 8, transport `nbd`: + +- `VixDiskLib_ConnectEx` (UID credentials) +- `VixDiskLib_Open` (datastore path, read-only or read-write) +- `VixDiskLib_Read` +- `VixDiskLib_Write` + +Not implemented: compression open flags, CBT / allocated-block +queries, disk geometry (`DDB_GET`), encrypted disks, and direct ESXi +`ha-nfc` without vCenter `vpxa-nfc`. + +## Install + +```bash +python3.12 -m venv .venv +.venv/bin/pip install -e . +``` + +## Usage + +```python +from openvixdisklib import nfc_auth +from openvixdisklib import openvixdisklib as vixdisklib + +handle = vixdisklib.VixDiskLibHandle( + vixdisklib_compatibility_version="8.0") +buf = vixdisklib.get_buffer(vixdisklib.VIXDISKLIB_SECTOR_SIZE) +thumbprint = nfc_auth.get_ssl_cert_thumbprint("vcenter.example.com") + +with handle.connect( + server_name="vcenter.example.com", + thumbprint=thumbprint, + username="administrator@vsphere.local", + password="secret", + vmx_spec="moref=vm-1234", + transport_modes="nbd", + read_only=False) as conn: + with handle.open(conn, "[datastore] vm/vm.vmdk", flags=0) as disk: + handle.write(disk, 0, 1, buf) + handle.read(disk, 0, 1, buf) +``` + +Lower-level NFC helpers live in `openvixdisklib.nfc_auth` and +`openvixdisklib.nfc_open` if you need the ticket or socket without the +VDDK-shaped handle. + +## Layout + +| Path | Role | +| ---------------------------------- | ------------------------------------------------------ | +| `openvixdisklib/openvixdisklib.py` | Drop-in handle (`connect` / `open` / `read` / `write`) | +| `openvixdisklib/nfc_auth.py` | VIM login, NFC ticket, authd on 902 | +| `openvixdisklib/nfc_open.py` | Classic NFC handshake, AIO open, sector read/write | +| `tests/integration/` | Live unittest suite against a lab vCenter | +| `tests/integration/vixdisklib.py` | Native VDDK wrapper used only to cross-check | +| `docs/` | Protocol notes and reverse-engineering steps | + +VDDK shared libraries, if present for cross-check, belong in `.vddk/` +(gitignored). They are not required to use `openvixdisklib`. + +## Tests + +Lab connection settings live in `.test_config.yaml` at the repo root +(gitignored). Copy: + +```yaml +host: vcenter.example.com +port: 443 +username: administrator@vsphere.local +password: secret +allow_untrusted: true +datacenter: Datacenter +datastore: datastore0 +``` + +`TestBase.setUpClass` creates an empty VM with a 10 GiB thin disk on +that datastore and tears it down in `tearDownClass`. Tests write known +patterns and read them back. + +```bash +tox -e integration +# or +.venv/bin/python -m unittest discover -s tests/integration +``` + +VDDK cross-check tests skip when `libvixDiskLib` is not loadable from +`.vddk`. `tox -e integration` sets `LD_LIBRARY_PATH` to that directory +and clears `LD_PRELOAD`. For a direct unittest run, do the same. + +Lint and typecheck: `tox -e pep8`, `tox -e mypy`. + +## Documentation + +| Document | Contents | +| --------------------------------------- | -------------------------------- | +| `docs/nfc_auth.md` | Ticket SOAP and authd handshake | +| `docs/nfc_open.md` | Classic NFC and AIO open | +| `docs/nfc_read.md` | AIO IO / `VixDiskLib_Read` | +| `docs/nfc_write.md` | AIO IO / `VixDiskLib_Write` | +| `docs/ssl_hook.md` | TLS intercept used for capture | +| `docs/reverse_engineering_procedure.md` | How the protocol was recovered | diff --git a/docs/nfc_auth.md b/docs/nfc_auth.md new file mode 100644 index 0000000..1ac8c37 --- /dev/null +++ b/docs/nfc_auth.md @@ -0,0 +1,265 @@ +# VDDK NFC authentication + +This document records how VMware VDDK authenticates for NBD/NFC disk +access, and how the Python replacement in `openvixdisklib/nfc_auth.py` +reproduces that path. Findings come from VDDK 8.0.2 libraries +(`libvixDiskLib`, `libvddkVimAccess`, `libvim-types`), live SOAP calls +against vCenter +8.0.1, and a TLS intercept of `VixDiskLib_ConnectEx` / `VixDiskLib_Open`. +The steps used to obtain those findings are in +`docs/reverse_engineering_procedure.md`. + +The goal of this stage is authentication only: a logged-in VIM session +plus an authd TLS socket that has completed `200 Connect`. Opening a +VMDK and reading sectors is `docs/nfc_open.md`. + +## Mapping from VDDK + +The VDDK wrapper in `tests/integration/vixdisklib.py` calls +`VixDiskLib_ConnectEx` with UID credentials and `VixDiskLib_Open` on a +datastore path. VDDK does **not** send the vCenter username and +password to ESXi port 902. It: + +1. Logs into vCenter over HTTPS 443 (SOAP / `urn:vim25`). +2. Asks vCenter for a one-time NFC ticket. +3. Connects to the ESXi **authd** daemon on TCP 902, upgrades to TLS, + and presents that ticket. + +| VDDK call | What actually happens | +| --------------------------------- | ---------------------------------------------------------- | +| `VixDiskLib_InitEx` | Load plugins, SSL, logging | +| `VixDiskLib_ConnectEx` | SOAP `SessionManager.Login` to vCenter | +| `VixDiskLib_Open` (read-only) | `NfcGetVmFiles` ticket, then authd handshake, then NFC I/O | +| `VixDiskLib_Open` (read-write) | `NfcRandomAccessOpenDisk` ticket (disk key + host) | +| `transport_modes="nbd"` | NBD over NFC (`vpxa-nfc://...@esxi:902`) | +| `vmxSpec=moref=vm-13098` | VM managed object used as the ticket target | +| `snapshot_ref` | Not consumed by the ticket call itself | +| `VIXDISKLIB_CRED_UID` | Username/password for VIM only | + +Lab topology used for capture: + +- vCenter: `10.8.1.199` (VirtualCenter 8.0.1) +- VM: `vm-13098` on host `host-13001` (`10.8.1.250`) +- NFC service moref on vCenter: `nfcService` +- Authd: `10.8.1.250:902` + +## Stage 1: VIM login + +This is a public pyVmomi operation. Reuse `pyVim.connect.SmartConnect` +rather than crafting SOAP. + +- Endpoint: `https://:443/sdk` +- Cookie: `vmware_soap_session` +- SOAPAction: `"urn:vim25/8.0.1.0"` (negotiated) + +VDDK logs this as `Connected to VIM Server` / `Authenticating user` / +`Logged in!`. The Python replacement keeps that `ServiceInstance` and +its stub for the ticket call. + +Direct ESXi login is the same SOAP login against hostd, but the NFC +moref and service name differ (`ha-nfc` instead of `nfcService` / +`vpxa-nfc`). The lab path is vCenter-mediated. + +## Stage 2: NFC ticket + +### Why this is not public pyVmomi + +`vim.NfcService` is omitted from the public vim25 WSDL that pyVmomi +ships. vCenter still implements it: + +- Version document: `GET /sdk/nfcServiceVersions.xml` → namespace + `urn:nfc`, version `7.0.3.2` +- Methods also accept `urn:vim25` (that is what VDDK uses) +- Well-known moref on this vCenter: `nfcService` + +`ServiceManager.QueryServiceList` does **not** list NFC. The moref is +hardcoded in VDDK as `nfcService` (vCenter) or `ha-nfc` (ESXi). + +`openvixdisklib/nfc_auth.py` registers the missing type with +`pyVmomi.VmomiSupport.CreateManagedType` and invokes it on the existing +SmartConnect stub, so serialization, cookies, and `HostServiceTicket` +stay in pyVmomi. + +### Methods VDDK actually calls + +Intercepted SOAP for a **read-only** `VixDiskLib_Open` of a datastore +path: + +```xml + + <_this type="NfcService">nfcService + vm-13098 + +``` + +No disk path, snapshot, or host is in this request. The path +(`[datastore0] ...-000007.vmdk`) is used later on the NFC channel. + +A `GetVmFiles` ticket is **not** writable. Opening the same path with +NFC flags `0x1a` returns AIO error `0x0b` (`VIX_E_FILE_READ_ONLY`). +Writable `ConnectEx(readOnly=FALSE)` uses a disk-scoped ticket instead. + +`libvim-types.so` maps vmodl `randomAccessOpen` to WSDL +`NfcRandomAccessOpenDisk` (same arguments as the read-only sibling): + +```xml + + <_this type="NfcService">nfcService + vm-13098 + 2000 + host-13001 + +``` + +A disk-scoped **read** ticket also works and returns the same +`HostServiceTicket` type: + +```xml + + <_this type="NfcService">nfcService + vm-13098 + 2000 + host-13001 + +``` + +`diskDeviceKey` is `VirtualDisk.key` from `vm.config.hardware.device` +(2000 for Hard disk 1). The replacement resolves it from the datastore +path when `open` is given a VMDK rather than a key. + +### Return value: `vim.HostServiceTicket` + +Public pyVmomi type. Example from this lab: + +| Field | Example | Role | +| ---------------- | -------------------------------------- | ----------------------------------------- | +| `host` | `10.8.1.250` | ESXi management / NFC address | +| `port` | `902` | authd TCP port | +| `sslThumbprint` | `BE:22:58:...:76:29` | SHA-1 of the ESXi TLS cert | +| `service` | `vpxa-nfc` | authd `PROXY` argument | +| `serviceVersion` | `1.1` | NFC hosted by hostd (ESX 3.0+ convention) | +| `sessionId` | `52cdebc5-b7ee-359a-1dec-76f0bc105ac5` | One-time authd `SESSION` token | + +Tickets are single-use. Calling `GetVmFiles` twice issues two tickets; +only the one presented to authd is consumed. + +### Other NfcService methods seen in VDDK + +WSDL names are prefixed with `Nfc`. The vmodl names (from +`libvim-types.so`) include: + +| WSDL name | Parameters (observed / from C++) | Notes | +| ----------------------------- | -------------------------------------- | ------------------------------ | +| `NfcGetVmFiles` | `vm` | VDDK read-only Open path | +| `NfcRandomAccessOpenReadonly` | `vm`, `diskDeviceKey`, `hostForAccess` | Disk-scoped read ticket | +| `NfcRandomAccessOpenDisk` | `vm`, `diskDeviceKey`, `hostForAccess` | Disk-scoped read-write ticket | +| `NfcGetServerNfcLibVersion` | `hostForAccess` | Lab returned `11` | +| `NfcFileManagement` | requires `ds` (datastore) | File copy, not NBD | +| `NfcSystemManagement` | host moref | Not used for disk open | + +`NfcGetServerNfcLibVersion` without `hostForAccess` fails with +`A specified parameter was not correct: hostForAccess`. Using moref +`ha-nfc` on vCenter fails with `ManagedObjectNotFound`; `nfcService` +is the correct vCenter object. + +## Stage 3: authd handshake (TCP 902) + +authd is the VMware Authentication Daemon. Plaintext banner from ESXi +8: + +``` +220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported, NFCSSL supported/t, SHA256 supported +``` + +SSL is required. Sending commands before `wrap_socket` closes the +connection. After TLS there is **no** `USER` / `PASS` when the client +holds a vCenter NFC ticket. + +### Sequence captured from VDDK + +VDDK log line immediately before the socket: + +``` +Using proxy/session authentication, sessionId=..., useSSL=0 +Plain-text connection is deprecated; use SSL to connect to NFC server +``` + +`useSSL=0` does **not** mean skip TLS on 902. It means skip a second +NFCSSL wrap after authd TLS (`THUMBPRINT_SHA2 PlainText`). The +management channel is still TLS. + +Intercepted writes/reads after the TLS handshake: + +``` +C -> SESSION \r\n +C -> BANNER \r\n +S -> 220 VMware Authentication Daemon Version 1.10: ...\r\n +C -> THUMBPRINT_SHA2 PlainText\r\n +S -> 200 \r\n +C -> PROXY vpxa-nfc\r\n +S -> 200 Connect ha-nfc\r\n +``` + +Notes: + +- `SESSION` does not get a reply of its own. Waiting for a line after + `SESSION` looks like a hang. +- `BANNER` is the 7-byte command `BANNER` plus a trailing space. That + space is part of the token; authd strips spaces when matching some + commands, so `THUMBPRINT_SHA2 ` is parsed as one + token and returns `501 Invalid arguments`. `PlainText` has no extra + spaces/colons and is the argument VDDK sends. +- `PROXY` uses `ticket.service` (`vpxa-nfc` via vCenter). The success + line names the host-side NFC endpoint (`ha-nfc`). +- After `200 Connect`, the socket speaks binary NFC (not documented + here). + +### Commands that are not used for this ticket type + +authd also implements FTP-style `USER` / `PASS` (and `XPAS`). Those +are for local ESXi credentials. With a vCenter ticket: + +| Attempt | Result | +| -------------------------------------------- | --------------------------------------- | +| `USER` / `PASS` (vCenter account) | `530 Login incorrect` | +| `USER *` / `PASS ` | `530 Login incorrect` | +| `USER ` / `PASS ` | `530 Login incorrect` | +| `SESSIONID ` | `530 Please login with USER and PASS` | +| `CONNECT_VPXA ` (after TLS) | `530 Please login with USER and PASS` | +| `SESSION ` then wait for a reply | No line until `BANNER` / `PROXY` follow | + +`THUMBPRINT` / `THUMBPRINT_SHA2` with the SHA-1 ticket thumbprint as +argument is not what VDDK sends. The SHA-1 value is for verifying the +TLS certificate, not for the `THUMBPRINT_SHA2` command. + +## Python replacement + +| Piece | Module | Reuses pyVmomi? | +| -------------------- | ---------------------------------------- | ---------------------------------- | +| VIM login | `openvixdisklib.nfc_auth.connect_vim` | Yes — `SmartConnect` | +| VM / host lookup | `vim.VirtualMachine` | Yes | +| `HostServiceTicket` | return type of ticket call | Yes — public data object | +| NFC ticket | `openvixdisklib.nfc_auth.get_nfc_ticket` | Same stub; type registered locally | +| authd TLS + commands | `openvixdisklib.nfc_auth.connect_authd` | No public API | +| End-to-end | `openvixdisklib.nfc_auth.authenticate` | `NfcAuthSession` | + +Management SHA-1 thumbprints are read with +`openvixdisklib.nfc_auth.get_ssl_cert_thumbprint` (stdlib `ssl` and +`hashlib`; no pyOpenSSL). Integration tests call that instead of +hard-coding the lab certificate. + +Run: + +```bash +.venv/bin/python -m unittest tests.integration.test_nfc_auth +``` + +The test completes VIM login and the authd handshake (`200 Connect`) +and asserts an established TLS socket on `ticket.host:ticket.port`. + +## What comes after authentication + +Authentication stops at `200 Connect ha-nfc`. Opening the VMDK and +reading or writing sectors is documented in `docs/nfc_open.md` and +implemented in `openvixdisklib/nfc_open.py`. The datastore path is +consumed there (and, for writes, as `diskDeviceKey` on the ticket). diff --git a/docs/nfc_open.md b/docs/nfc_open.md new file mode 100644 index 0000000..76324df --- /dev/null +++ b/docs/nfc_open.md @@ -0,0 +1,226 @@ +# VDDK NFC disk open + +This document records how VMware VDDK opens a VMDK over NBD/NFC after +the authd handshake in `docs/nfc_auth.md`, and how +`openvixdisklib/nfc_open.py` reproduces that path. Findings come from +VDDK 8.0.2 verbose logs +(`vixDiskLib.nfc.LogLevel=4`) plus an `LD_PRELOAD` intercept of +`write` / `read` on the ESXi:902 file descriptor. Capture method: +`docs/reverse_engineering_procedure.md`. + +Authentication is already done: VIM login, NFC ticket (`NfcGetVmFiles` +for read-only, `NfcRandomAccessOpenDisk` for write), TLS to authd, +`SESSION` / `BANNER` / `THUMBPRINT_SHA2 PlainText` / `PROXY`. This +stage starts at `200 Connect ha-nfc` and ends with an open file handle +that can read and write sectors. Flags `0x1a` require the writable +ticket; the same flags on a `GetVmFiles` ticket fail with +`VIX_E_FILE_READ_ONLY`. + +## Mapping from VDDK + +| VDDK call / log | Wire effect | +| ---------------------------------------------------- | -------------------------------------------------------- | +| `VixDiskLib_Open` | Ticket + authd (see `nfc_auth.md`), then this protocol | +| `NBD_ClientOpen` `vpxa-nfc://[ds] path.vmdk@esxi:902` | Datastore path is the NFC open argument, not the ticket | +| `useSSL=0` | NFC bytes are raw TCP, not `SSL_write` | +| `NfcProcessSessionParams` flags `0x3` | Classic 264-byte session messages | +| `SendConnectionDataMsg` payloadInfo 4 and 7 | Client name `vddk` (4) and opId `nbdmode` (7) | +| Server version 11 | Classic version message; 11 on this ESXi 8 lab | +| `NfcAio_OpenSession` | AIO framing after the classic handshake | +| `NfcUtil_PrintFileInfoOpenFlag` `NFC_DISK` `0x1e` | `NFC_AIO_MSG_OPEN_FILE` (read-only) | +| Open without `VIXDISKLIB_FLAG_OPEN_READ_ONLY` | `OPEN_FILE` flags `0x1a` (read-write) | +| `VixDiskLib_Read` / `VixDiskLib_Write` | `NFC_AIO_MSG_IO` + sector bytes | + +`snapshot_ref` is still not on the wire. Integration tests pass the +flat VMDK created with the temporary lab VM. + +## After PROXY: plaintext on the TLS fd + +`THUMBPRINT_SHA2 PlainText` tells authd not to wrap NFC in a second +TLS session. VDDK logs `useSSL=0` and “Plain-text connection is +deprecated”. + +On the wire that means: + +1. Authd commands stay inside the original TLS session (`SSL_write` / + `SSL_read`). +2. After `200 Connect ha-nfc`, VDDK calls `write(SSL_get_fd(ssl), …)` + and `read` on that same descriptor. Those buffers are NFC, not TLS + records (`0x17 0x03 …`). +3. ESXi’s `ha-nfc` side does the same: replies are plaintext NFC. + +An SSL hook that only interposes `SSL_write` / `SSL_read` therefore +goes silent after PROXY. Interposing `write` / `read` and filtering +`getpeername` port 902 shows the frames. + +Python must not use `SSLSocket.send` for this stage: that would +`SSL_write` and encrypt bytes the server now reads as NFC. +`nfc_open.takeover_authd_socket` dups `SSL_get_fd` and uses a raw +`socket.socket`. `unwrap()` / `SSL_shutdown` is not used; VDDK does +not send `close_notify` before NFC. + +## Classic 264-byte messages + +Before AIO, both peers send a **fixed 264-byte** struct, little-endian: + +| Offset | Type | Meaning | +| ------ | --------- | -------------------------------------------- | +| 0 | `uint32` | Message type | +| 4 | remainder | Type-specific fields, zero-padded to 264 | + +Types seen in this Open (names from `libvixDiskLib` strings matched to +the first `uint32`): + +| Type | Name (inferred) | Body | +| ---- | ---------------------- | ------------------------------------------------- | +| 43 | `NFC_HANDSHAKE` | ASCII `PlainText` at offset 4 | +| 33 | `NFC_SESSION_PARAMS` | zeros | +| 36 | session-params reply | `uint32` 1 at offset 16 | +| 51 | version | `uint32` protocol version (11) at offset 4 | +| 54 | `NFC_CONNECTION_DATA` | `uint32` nameLen, `uint32` opIdLen | +| 55 | session features | `uint32` `0x3` (interruption \| switch) | +| 52 | `NFC_AIO_SESSION_OPEN` | zeros | +| 4 | `NFC_SESSION_COMPLETE` | zeros (sent on close) | + +After type 54, VDDK writes the two connection-data payloads as **raw +strings**, not 264-byte frames: `vddk` then `nbdmode`. Lengths 4 and 7 +are the `payloadInfo` values in the VDDK log. + +Handshake order (client → server unless noted): + +``` +C: 43 PlainText +C: 33 +S: 36 +C: 51 version=11 +S: 51 version=11 +C: 54 nameLen=4 opIdLen=7 +C: "vddk" +C: "nbdmode" +C: 55 features=3 +C: 52 +S: 52 +``` + +Server version 11 is what this lab returned. VDDK logs that connection +info requires version ≥ 3. + +## AIO framing + +Once type 52 has been acknowledged, I/O uses a 16-byte header: + +``` +uint32 magic # 0xA100DA7A, wire bytes 7a da 00 a1 +uint32 type # NfcAioSendMessage "type =" +uint32 size # payload bytes that follow the header +uint32 opId # monotonic, starting at 0 +``` + +Then `size` bytes of payload. Variable-length extras (VMDK path, DDB +key name, read data) are **separate** `write`/`read` calls after that +payload, not counted in `size`. + +The server echoes the same header (`magic`, `type`, `size`, `opId`) +and a payload of `size` bytes. + +Magic mismatch is the `invalid msg hdr magic` string in VDDK. Type 1 +is `NFC_AIO_MSG_ERROR`. + +AIO types used for Open / Read / Close, correlated with the consecutive +`NFC_AIO_MSG_*` string table and VDDK logs: + +| Type | Name | Payload size | Extra on the wire | +| ---- | -------------------- | ------------ | --------------------------------------------- | +| 2 | `OPEN_SESSION` | 16 | | +| 9 | `SET_SOCK_OPTS` | 12 | | +| 22 | `SET_RES_POOL` | 4 | | +| 4 | `OPEN_FILE` | 60 | path string | +| 11 | `DDB_GET` | 16 | key name (VDDK only) | +| 7 | `IO` | 44 | sector bytes (read reply / write request) | +| 5 | `CLOSE_FILE` | 8 | | +| 3 | `CLOSE_SESSION` | 4 | | + +`opId` increases by one per client message. Replies reuse the request +`opId`. + +VDDK Open also issues several `DDB_GET` queries (`resumeConsolidateSector`, +`isDigest`, `iofilters`, …). The server answered “key is not found” +(16 zero bytes) on this unencrypted disk. They are not required to +obtain a file handle or to read sector 0. + +### OPEN_SESSION / sockopts / resource pool + +VDDK sends 16 zero bytes (`OPEN_SESSION`), 12 zero bytes +(`SET_SOCK_OPTS`; server returns send/recv buffer sizes), then +`uint32` 1 (`SET_RES_POOL`, log: “Setting Resource Pool(1)”). + +### OPEN_FILE + +60-byte payload, little-endian: + +| Offset | Type | Value on a VDDK open | +| ------ | -------- | -------------------------------------------------------------- | +| 0 | `uint32` | Path length in bytes | +| 4 | `uint32` | 0 | +| 8 | `uint32` | 0 | +| 12 | `uint32` | 0 | +| 16 | `uint32` | `2` (`NFC_DISK`) | +| 20 | `uint32` | `0x0000001e` (read-only) or `0x1a` (read-write) | +| 24 | 36 bytes | zeros | + +Immediately afterwards the client writes the path, no NUL terminator +(for example `[datastore0] ovdl-test-…/ovdl-test-….vmdk`). + +Reply payload (60 bytes), fields that matter: + +| Offset | Type | Meaning | +| ------ | -------- | ------------------------------- | +| 8 | `uint64` | File handle (opaque, per open) | +| 16 | `uint32` | File type (`2` = `NFC_DISK`) | +| 20 | `uint32` | Flags echoed (`0x1e` or `0x1a`) | +| 36 | `uint32` | Sector size (`512` on this VM) | + +Later AIO messages pass that handle as a `uint64`. + +### IO (read / write) + +Sector reads and writes are `NFC_AIO_MSG_IO` (type 7). Request layout, +read fragments, and write extras are documented in `docs/nfc_read.md` +and `docs/nfc_write.md`. `NfcDisk.read` / `NfcDisk.write` match +`VixDiskLib_Read` / `VixDiskLib_Write`. + +### Close + +`CLOSE_FILE` (handle as `uint64`), `CLOSE_SESSION` (`uint32` 0), then +classic type 4 `NFC_SESSION_COMPLETE`. + +## Python replacement + +| Piece | Module | +| ----------------------------- | ----------------------------------------------- | +| VIM + authd | `openvixdisklib.nfc_auth.authenticate` | +| Dup fd, skip TLS for NFC | `openvixdisklib.nfc_open.takeover_authd_socket` | +| Handshake + AIO + OPEN_FILE | `openvixdisklib.nfc_open.open_disk` | +| Sector read / write / close | `openvixdisklib.nfc_open.NfcDisk` | + +Run: + +```bash +.venv/bin/python -m unittest tests.integration.test_nfc_open +``` + +The test opens the temporary lab VMDK, asserts an opaque handle and +`sector_size=512`, writes sector 0, and reads it back. Multi-sector +I/O: `docs/nfc_read.md`, `docs/nfc_write.md`, and +`tests/integration/test_nfc_read_write.py`. + +## What is still VDDK-only + +- `DDB_GET` / geometry / compression / encryption keys +- `NFC_DELTA_DISK`, change-block tracking +- Host-switch (`NFC_AIO_SWITCH_HOST_*`) and a second NFCSSL wrap + (`useSSL=1`, not what VDDK NBD used here) +- Direct ESXi `ha-nfc` without vCenter `vpxa-nfc` + +Reads after open are in `docs/nfc_read.md`. Writes are in +`docs/nfc_write.md`. diff --git a/docs/nfc_read.md b/docs/nfc_read.md new file mode 100644 index 0000000..8afccca --- /dev/null +++ b/docs/nfc_read.md @@ -0,0 +1,120 @@ +# VDDK NFC disk read + +This document records how VMware VDDK reads VMDK sectors over NBD/NFC +after the open in `docs/nfc_open.md`, and how `NfcDisk.read` in +`openvixdisklib/nfc_open.py` reproduces `VixDiskLib_Read`. Capture +method: `docs/reverse_engineering_procedure.md`. + +## Mapping from VDDK + +`VixDiskLib_Read(handle, startSector, numSectors, buf)` becomes one +`NFC_AIO_MSG_IO` (type 7) on the NFC socket. Units on the wire are +**bytes**, not sectors: + +``` +offset = startSector * sectorSize +length = numSectors * sectorSize +``` + +`sectorSize` is 512 from the `OPEN_FILE` reply on this lab disk. + +| VDDK call | Wire effect | +| --------------------------------- | ------------------------------------------------ | +| `VixDiskLib_Read(h, 0, 1, buf)` | IO offset 0, length 512, one 512-byte fragment | +| `VixDiskLib_Read(h, 1, 1, buf)` | IO offset 512, length 512 | +| `VixDiskLib_Read(h, 0, 128, buf)` | IO length 65536 (AIO buffer size), one fragment | +| `VixDiskLib_Read(h, 0, 129, buf)` | One request of 66048; **two** reply fragments | + +VDDK does **not** split a `Read` larger than 64 KiB into multiple +requests. The client sends one AIO message; the server answers with +one or more same-`opId` replies, each carrying at most +`NFC_AIO_BUFFER_SIZE` (65536) data bytes. `NfcAioInitSession` logged +that buffer size and count 4 during open. + +Sparse regions are still transferred as zeros. A read of 8 sectors at +LBA 8 on this disk was 4096 zero bytes on the wire, not a skip. + +## Request (44 bytes) + +Little-endian, after the usual 16-byte AIO header +(`magic 0xA100DA7A`, type 7, size 44, monotonic `opId`): + +| Offset | Type | VDDK `Read(start, n)` | +| ------ | -------- | ---------------------------------------------- | +| 0 | `uint64` | File handle from `OPEN_FILE` | +| 8 | `uint64` | `1` (`NFC_AIO_IO_READ`; write uses `0`) | +| 16 | `uint64` | Byte offset | +| 24 | `uint64` | Byte length | +| 32 | `uint32` | Byte length (same value) | +| 36 | `uint32` | Byte length (same value) | +| 40 | `uint32` | `0` (flags; uncompressed in this capture) | + +An earlier guess that offset 36 was `NFC_DISK` (`2`) was wrong: a +1-sector VDDK read puts `512` in both `uint32` length fields. A Python +read that sent `(512, 2, 0)` still worked for one sector; the +replacement now matches VDDK. + +## Reply + +Each fragment is: 16-byte AIO header (same `type` and `opId`) + 44-byte +payload + `chunkLength` data bytes. + +Reply payload (handle is zeroed; lengths describe this fragment): + +| Offset | Type | Meaning | +| ------ | -------- | ----------------------------------------------- | +| 0 | `uint64` | `0` | +| 8 | `uint64` | `1` (read) | +| 16 | `uint64` | Byte offset of the **request** | +| 24 | `uint32` | Total request length | +| 28 | `uint32` | Fragment index (`0`, `1`, …) | +| 32 | `uint32` | This fragment’s byte length | +| 36 | `uint32` | Same as offset 32 | +| 40 | `uint32` | `0` | + +When there is a single fragment, offsets 24–31 look like a `uint64` +length (index is 0). The 129-sector capture shows why they are two +`uint32`s: fragment 0 has `(66048, 0)` then chunk 65536; fragment 1 +has `(66048, 1)` then chunk 512. + +Read loop: receive fragments with that `opId` until the concatenated +data length equals the request. Use the `uint32` at payload offset 32 +as the extra-data size for that fragment. Do not treat extra data as +part of AIO `size` (that field stays 44). + +129-sector example (one client request, two server fragments): + +``` +C: type=7 opId=18 size=44 offset=0 length=66048 +S: type=7 opId=18 size=44 index=0 chunk=65536 + 65536 data +S: type=7 opId=18 size=44 index=1 chunk=512 + 512 data +``` + +## Lab check + +Integration tests create an empty 10 GiB thin disk, write a repeating +pattern at each captured range (including 129 sectors), and read it +back. An unwritten region is zeros. + +Writes use the same 44-byte IO payload with opcode `2`; see +`docs/nfc_write.md`. + +## Python replacement + +`NfcDisk.read(start_sector, num_sectors)` in +`openvixdisklib/nfc_open.py`. Run: + +```bash +.venv/bin/python -m unittest tests.integration.test_nfc_read_write +``` + +The integration test writes and then reads the captured VDDK ranges +(including a 129-sector transfer that must assemble two read +fragments). + +## What is still VDDK-only + +- Compression flags on the last `uint32` +- `VixDiskLib_ReadAsync` (same IO messages, different client threading) +- `VixDiskLib_QueryAllocatedBlocks` / allocation bitmaps +- `VixDiskLib_GetInfo` capacity (not required to read a known range) diff --git a/docs/nfc_write.md b/docs/nfc_write.md new file mode 100644 index 0000000..f8ab3ec --- /dev/null +++ b/docs/nfc_write.md @@ -0,0 +1,91 @@ +# VDDK NFC disk write + +This document records how `NfcDisk.write` in +`openvixdisklib/nfc_open.py` implements `VixDiskLib_Write` over NFC AIO. +The request layout matches the captured `VixDiskLib_Read` IO message in +`docs/nfc_read.md`. Open flags and the IO direction field were taken +from a `strace` of VDDK 8 writing one sector to a temporary 10 GiB +disk (`docs/reverse_engineering_procedure.md`). + +The public `VixDiskLib_Write` prototype is in `.vddk/vixDiskLib.h`: + +``` +VixError VixDiskLib_Write(VixDiskLibHandle diskHandle, + VixDiskLibSectorType startSector, + VixDiskLibSectorType numSectors, + const uint8 *writeBuffer); +``` + +`ConnectEx(..., Bool readOnly, ...)` with `readOnly=FALSE` and `Open` +without `VIXDISKLIB_FLAG_OPEN_READ_ONLY` (that flag is `1 << 2` in the +same header) is what produces the writable NFC open below. The ticket +must be `NfcRandomAccessOpenDisk` (`docs/nfc_auth.md`); flags `0x1a` +on a `NfcGetVmFiles` ticket are rejected as `VIX_E_FILE_READ_ONLY`. + +## Mapping from VDDK + +Units on the wire are **bytes**, as for reads: + +``` +offset = startSector * sectorSize +length = numSectors * sectorSize +``` + +| VDDK call | Wire effect | +| ------------------------------------------- | ------------------------------------------------ | +| Open without `VIXDISKLIB_FLAG_OPEN_READ_ONLY` | `OPEN_FILE` flags `0x1a` | +| Open with `VIXDISKLIB_FLAG_OPEN_READ_ONLY` | `OPEN_FILE` flags `0x1e` (read-only) | +| `VixDiskLib_Write(h, start, n, buf)` | IO opcode `0`, then `n * 512` data bytes | +| `VixDiskLib_Read(h, start, n, buf)` | IO opcode `1` | + +`0x1e` vs `0x1a` is bit `0x04`, the same value as +`VIXDISKLIB_FLAG_OPEN_READ_ONLY`. Writable opens clear that bit. + +VDDK also issues several `DDB_GET` queries and a type-10 +`GET_FILE_INFO` (`longContentID`) before the first write. They are not +required to write or read sectors. + +## Request (44 bytes + data) + +Little-endian, after the usual 16-byte AIO header +(`magic 0xA100DA7A`, type 7, size 44, monotonic `opId`): + +| Offset | Type | `Write(start, n)` | +| ------ | -------- | -------------------------------------------------- | +| 0 | `uint64` | File handle from `OPEN_FILE` | +| 8 | `uint64` | `0` (`NFC_AIO_IO_WRITE`; read uses `1`) | +| 16 | `uint64` | Byte offset | +| 24 | `uint64` | Byte length | +| 32 | `uint32` | Byte length (same value) | +| 36 | `uint32` | Byte length (same value) | +| 40 | `uint32` | `0` | + +Sector bytes follow the 44-byte payload and are **not** counted in AIO +`size`. VDDK sends header + payload + data in one `write()`. The +replacement may split that into two `sendall`s; TCP does not care. + +The server replies with a type-7 header and a 44-byte payload for that +`opId`. There is no extra data on the write reply (unlike reads). + +A 1-sector VDDK write was 572 bytes on the wire: 16 + 44 + 512. + +## Client-side split + +`NfcAioInitSession` advertises a 64 KiB buffer. VDDK splits writes +larger than that into 64 KiB chunks (VDDK programming guide). The +Python client does the same: several IO requests of at most +`NFC_AIO_BUFFER_SIZE` bytes, each with its own `opId`. + +## Python replacement + +`NfcDisk.write(start_sector, num_sectors, data)` in +`openvixdisklib/nfc_open.py`. `open_disk(..., read_only=False)` selects +flags `0x1a`. The drop-in handle exposes the same shape as VDDK: +`connect(read_only=False)`, `open` without +`VIXDISKLIB_FLAG_OPEN_READ_ONLY`, then `write`. + +Integration tests create an empty 10 GiB disk, write known patterns, +and read them back (`tests/integration/test_nfc_read_write.py`, +`tests/integration/test_openvixdisklib.py`). Cross-check tests write +with VDDK and with the replacement and read with both +(`tests/integration/test_crosscheck.py`). diff --git a/docs/reverse_engineering_procedure.md b/docs/reverse_engineering_procedure.md new file mode 100644 index 0000000..b948eb0 --- /dev/null +++ b/docs/reverse_engineering_procedure.md @@ -0,0 +1,281 @@ +# Reverse-engineering procedure + +This is the working method used to replace VDDK’s NBD path with Python. +Protocol details live in `docs/nfc_auth.md`, `docs/nfc_open.md`, +`docs/nfc_read.md`, and `docs/nfc_write.md`. The capture tool is +described in `docs/ssl_hook.md`. +This file is the **sequence of steps**, including dead ends, so later +NFC work can follow the same loop instead of rediscovering it. + +Scope so far: `VixDiskLib_ConnectEx` + `VixDiskLib_Open` + +`VixDiskLib_Read` + `VixDiskLib_Write` against lab vCenter 8.0.1 / +ESXi 8, transport `nbd`. Driver: `tests/integration/` (`TestBase` +creates a temporary empty VM with a 10 GiB disk in `setUpClass` and +destroys it in `tearDownClass`). + +Rule from `AGENTS.md`: reuse pyVmomi for every public VIM operation. +Only reimplement what pyVmomi does not expose. + +## Loop + +Each unknown stage (ticket SOAP, authd, NFC binary) went through: + +1. **Name it** from VDDK logs and `strings` on the bundled libraries. +2. **See it** on the wire (or prove that tcpdump cannot). +3. **Replay** the smallest working subset in Python against the lab. +4. **Write** findings into a protocol doc and keep the hook out of the + library path. + +Do not skip (2). Log lines such as `SESSIONID` or `useSSL=0` named the +wrong wire command until the intercept existed. + +## Lab and artifacts + +| Item | Where / value | +| ----------------------- | -------------------------------------------------------------------- | +| VDDK 8.0.2 | `.vddk/` (`libvixDiskLib`, `libvddkVimAccess`, `libvim-types`) | +| pyVmomi | `.venv` | +| Known-good VDDK client | `tests/integration/test_vddk.py` / `tests/integration/vixdisklib.py` | +| Verbose NFC logs | `vixDiskLib.nfc.LogLevel=4` in a temp VDDK config | +| ctypes Open+Read driver | `/tmp/vddk_open_trace.py` (not in the library) | +| SSL / `write` hook | `/tmp/sslhook.c` → `/tmp/sslhook.so` | + +Always set `LD_LIBRARY_PATH` to `.vddk/` so VDDK uses its own +`libssl.so.3`. Unset `LD_PRELOAD` before running the Python replacement; +a leftover `write` hook will crash pyVmomi’s TLS. + +## Step 1 — Map the public VDDK calls + +`tests/integration/test_vddk.py` is the specification of what +“success” looks like: login, open the temporary VM’s VMDK, write a +known pattern, read it back. + +Turn on VDDK verbose logging around `InitEx` / `ConnectEx` / `Open`. The +logs split the work that the Python API hides: + +- `ConnectEx` → VIM login only. +- `Open` → NFC ticket, authd, NFC handshake, AIO open, then I/O. +- `transport_modes=nbd` → URL form `vpxa-nfc://[ds] path.vmdk@esxi:902`. +- `snapshot_ref` does not appear on the ticket SOAP call. + +That mapping is the table at the top of `docs/nfc_auth.md`. It tells you +which stage to reverse next and which arguments belong there (VM moref +on the ticket, VMDK path on NFC `OPEN_FILE`). + +## Step 2 — Strings and pyVmomi before any capture + +`strings -a` on `.vddk/*.so` produced candidate tokens before a single +packet was decoded: + +- SOAP: `NfcService`, `NfcGetVmFiles`, `nfcService`, `ha-nfc`, + `HostServiceTicket`. +- authd: `SESSION`, `BANNER`, `THUMBPRINT_SHA2`, `PROXY`, `USER`, + `PASS`, `SSL Required`. +- NFC: `NFC_HANDSHAKE`, `NFC_CONNECTION_DATA`, `NFC_AIO_MSG_*`, + `NFC_DISK`. + +Then check whether pyVmomi already has the type: + +```python +from pyVmomi import vim +hasattr(vim, "NfcService") # False +hasattr(vim, "HostServiceTicket") # True +``` + +`ServiceManager.QueryServiceList` on the live vCenter does **not** list +NFC. `GET /sdk/nfcServiceVersions.xml` does (`urn:nfc` 7.0.3.2). The +moref is hardcoded in VDDK (`nfcService` on vCenter, `ha-nfc` on ESXi). + +Anything public (`SmartConnect`, `vim.VirtualMachine`, +`HostServiceTicket`) stays in pyVmomi. Missing managed types are +registered with `CreateManagedType` on the same SOAP stub so cookies +and serialization are not reimplemented. + +## Step 3 — Confirm tcpdump is the wrong tool for TLS stages + +tcpdump on 443 and 902 shows TLS records only. That is enough to prove +“something talks to vCenter then to ESXi:902”, and not enough for SOAP +bodies, authd lines, or NFC headers. + +VDDK logs name functions and AIO `opId` / `type` / `size`. They do not +give magic numbers, path placement, or command spacing (`BANNER \r\n`). + +An ESXi-impersonating service was considered (`AGENTS.md`) and not +needed: the lab answers VDDK, so capturing the real client is simpler +than simulating the server. + +## Step 4 — Interpose OpenSSL (authd and SOAP) + +VDDK 8.0.2 still calls `SSL_write` / `SSL_read`. A small `LD_PRELOAD` +library logs those buffers as hex, tagged with the `SSL *` pointer. + +Run a minimal ctypes program (`InitEx`, `ConnectEx`, `Open`, `Read`) +under: + +```bash +export LD_LIBRARY_PATH=…/.vddk +export LD_PRELOAD=/tmp/sslhook.so +export SSLHOOK_LOG=/tmp/sslhook-open.log +python /tmp/vddk_open_trace.py +``` + +Parse offline: + +1. Concatenate adjacent same-direction records (authd `SSL_read` is + often one byte). +2. Split by `SSL *`. vCenter HTTPS contains `POST /sdk` and SOAP. + ESXi:902 contains `SESSION` / `PROXY`. +3. An early hook without the pointer mixed both streams; always tag. + +The vCenter stream identified the ticket as `NfcGetVmFiles` with moref +`nfcService` and `xmlns="urn:vim25"` (not a guess from strings alone). +The ESXi stream gave the authd command order, including the trailing +space on `BANNER` and `THUMBPRINT_SHA2 PlainText`. + +Hook implementation notes: `docs/ssl_hook.md`. + +## Step 5 — Probe SOAP, then replay only what VDDK sends + +With a SmartConnect session, raw SOAP posts were used to learn +parameter names and which moref vCenter accepts: + +- `ha-nfc` on vCenter → `ManagedObjectNotFound`. +- `NfcGetServerNfcLibVersion` without `hostForAccess` → invalid + argument; with a host moref → `11`. +- `NfcGetVmFiles(vm)` → `HostServiceTicket` (VDDK read-only Open). +- `NfcRandomAccessOpenReadonly(vm, diskDeviceKey, host)` → same ticket + type, disk-scoped read. +- `NfcRandomAccessOpenDisk(vm, diskDeviceKey, host)` → writable + ticket. `GetVmFiles` plus `OPEN_FILE` flags `0x1a` fails with + `VIX_E_FILE_READ_ONLY` (`0x0b`). + +The replacement registers those methods and calls them through pyVmomi. +It does not ship a hand-rolled SOAP client for login or tickets. + +## Step 6 — Probe authd; record dead ends + +Plaintext banner (`220 … SSL Required`), then `ssl.wrap_socket`. +Commands before TLS drop the connection. + +vCenter UID/password are **not** sent to port 902. Attempts that failed +and must not be retried for this ticket type: + +| Attempt | Result | +| ------------------------------------- | ------------------------------------- | +| `USER` / `PASS` (vCenter account) | `530 Login incorrect` | +| `USER` / `PASS` with `sessionId` | `530 Login incorrect` | +| `SESSIONID ` | `530 Please login with USER and PASS` | +| `CONNECT_VPXA` after TLS | `530 Please login with USER and PASS` | +| Wait for a reply after `SESSION` | Hang until `BANNER` / `PROXY` follow | +| `THUMBPRINT_SHA2` with SHA-1 digest | `501 Invalid arguments` | + +The working sequence is in `docs/nfc_auth.md`. `useSSL=0` in the VDDK +log means skip a **second** NFCSSL wrap, not skip TLS on 902. + +Replay: `openvixdisklib/nfc_auth.py` / +`tests/integration/test_nfc_auth.py`. Stop at `200 Connect`. + +## Step 7 — NFC binary: extend the hook to `write` / `read` + +After `PROXY`, `SSL_write` on the ESXi `SSL *` goes silent. VDDK logs +`useSSL=0` / “plain-text connection is deprecated” and then NFC +function names. The bytes are `write(SSL_get_fd(ssl), …)` / `read` on +peer port 902. + +The hook was extended to those syscalls, filtered with `getpeername` +port 902, and mutex-locked (VDDK is multi-threaded). Skip TLS records +(`16 03` / `17 03`) left over from the authd phase. + +Correlate each frame with the verbose log line that has the same +`type` and `size` (`NfcAioSendMessage: opId = … type = … size = …`). +Name the types from the consecutive `NFC_AIO_MSG_*` string table in +`libvixDiskLib.so`. Lengths 4 and 7 on the connection-data message are +the ASCII strings `vddk` and `nbdmode` sent in the next two writes. + +Classic NFC uses a 264-byte padded struct; AIO uses a 16-byte header +(`magic 0xA100DA7A`) plus payload; path / DDB key / sector data are +extra writes not included in `size`. + +## Step 8 — Replay the smallest subset, then compare to VDDK + +Python must **dup the authd fd** and send NFC as raw TCP. +`SSLSocket.send` would encrypt; `unwrap()` would `SSL_shutdown`. VDDK +does neither. + +`openvixdisklib/nfc_open.py` replays handshake + AIO `OPEN_SESSION` / +sockopts / resource pool / `OPEN_FILE`. VDDK’s extra `DDB_GET` keys +were omitted once a file handle was enough to read. Proof of open: +`tests/integration/test_nfc_open.py`. + +Do not copy every VDDK message. Copy what the server requires for the +Python API you are replacing. + +## Step 9 — Vary `VixDiskLib_Read` until IO fields stop moving + +A single-sector read is not enough to decode `NFC_AIO_MSG_IO`. Drive +VDDK with several `(startSector, numSectors)` pairs in one Open +(including `n=128` = 64 KiB and `n=129`) under the `write`/`read` hook. + +What that comparison showed: + +- Wire units are bytes (`offset = start * 512`, `length = n * 512`). +- Request size stays 44; data is extra after the payload. +- VDDK sends **one** request even when `length > 65536`. The server + replies with several type-7 messages that share `opId`, each with a + chunk length at payload offset 32 (max 65536). +- Treating offset 36 as `NFC_DISK` (`2`) was a 1-sector coincidence; + VDDK repeats the byte length there. +- Zeros on the wire are real transferred zeros, not a sparse skip. + +Replay: `NfcDisk.read` loops on fragments until `length` bytes arrive. +Proof: `tests/integration/test_nfc_read_write.py` writes a known pattern +(including a 129-sector read that must assemble two fragments) and +checks the bytes that came back. + +## Step 10 — Writes from the same IO message + +`VixDiskLib_Write` uses the same 44-byte `NFC_AIO_MSG_IO` layout as +read. The direction field at offset 8 is `0` instead of `1`, and the +sector bytes are sent after the payload (like the path on +`OPEN_FILE`). Writable `OPEN_FILE` flags are `0x1a` (the captured +read-only flags `0x1e` with bit `0x04` cleared, matching +`VIXDISKLIB_FLAG_OPEN_READ_ONLY` in `.vddk/vixDiskLib.h`). + +Writable `OPEN_FILE` still failed with `VIX_E_FILE_READ_ONLY` until +the ticket switched from `NfcGetVmFiles` to `NfcRandomAccessOpenDisk` +(`libvim-types.so`: vmodl `randomAccessOpen` ↔ WSDL +`NfcRandomAccessOpenDisk`). Integration tests create a temporary empty +10 GiB VM for the run so writes cannot land on other lab disks. + +The Python client splits writes larger than 64 KiB; it does not send a +single oversized write the way VDDK sends an oversized read. Details: +`docs/nfc_write.md`. Proof: write then read in +`tests/integration/test_nfc_read_write.py` and the VDDK cross-check in +`tests/integration/test_crosscheck.py`. + +## What to write down + +After a stage works: + +| Document | Contents | +| --------------------------------------- | --------------------------------------------- | +| `docs/nfc_auth.md` | Ticket SOAP + authd wire format | +| `docs/nfc_open.md` | Classic NFC + AIO open | +| `docs/nfc_read.md` | AIO IO / `VixDiskLib_Read` | +| `docs/nfc_write.md` | AIO IO / `VixDiskLib_Write` | +| `docs/ssl_hook.md` | Capture tool only | +| `docs/reverse_engineering_procedure.md` | This procedure (update when the method changes) | + +Keep the hook and ctypes driver under `/tmp`. They are not part of the +replacement library. + +## Next stages (same procedure) + +Not yet reversed, same loop as above: + +- `DDB_GET` / disk geometry, compression, encrypted disks +- `NFC_DELTA_DISK`, CBT / `QueryAllocatedBlocks` +- `VixDiskLib_GetInfo` capacity +- Host-switch AIO messages +- `useSSL=1` (second NFCSSL wrap) +- Direct ESXi `ha-nfc` without vCenter `vpxa-nfc` diff --git a/docs/ssl_hook.md b/docs/ssl_hook.md new file mode 100644 index 0000000..bb2edc2 --- /dev/null +++ b/docs/ssl_hook.md @@ -0,0 +1,145 @@ +# SSL hook for VDDK protocol capture + +VDDK’s NBD path is TLS end to end: SOAP to vCenter on 443, then authd/NFC +to ESXi on 902. Packet captures on those ports are ciphertext, so they +cannot show command names, tickets, or NFC frames. + +This project used a small `LD_PRELOAD` library (`sslhook.c`, built to +`sslhook.so`) to log OpenSSL plaintext while a ctypes wrapper ran +`VixDiskLib_ConnectEx` / `VixDiskLib_Open`. The authd sequence in +`docs/nfc_auth.md` was recovered from that log, not from VDDK source. + +The hook is a reverse-engineering aid. It is not part of the Python +NFC client. + +## Why not tcpdump or VDDK logs + +| Approach | What it shows | Gap | +| -------------------------------- | -------------------------------------------------- | ------------------------------------------------ | +| tcpdump on 443 / 902 | TLS records | No SOAP bodies, no authd lines, no NFC frames | +| `vixDiskLib.nfc.LogLevel=4` | Function names, `opId` / `type` / `size` | Not the bytes on the wire | +| Strings in `libvixDiskLib.so` | Command tokens (`SESSION`, `PROXY`, `BANNER`) | Not order, spacing, or replies | +| SSL hook on `SSL_write`/`read` | Exact buffers before encrypt / after decrypt | Must split connections and reassemble 1-byte I/O | + +VDDK logs were still useful to *name* AIO message types after the hex +dump showed `type` and `size`. The hook supplied the actual framing. + +## How `LD_PRELOAD` interposition works + +The hook exports `SSL_write` and `SSL_read` with OpenSSL’s signatures. +When the process starts with `LD_PRELOAD=/path/sslhook.so`, the dynamic +linker binds VDDK’s calls to those symbols instead of `libssl`. + +Each wrapper: + +1. Resolves the real OpenSSL function with `dlsym(RTLD_NEXT, ...)`. +2. Logs the plaintext buffer. +3. Calls the real function so the session is unchanged. + +``` +VixDiskLib --> SSL_write (hook) --> log hex --> SSL_write (libssl) +VixDiskLib <-- SSL_read (hook) <-- log hex <-- SSL_read (libssl) +``` + +`SSL_write` logs **before** encryption. `SSL_read` calls OpenSSL first, +then logs `n` decrypted bytes when `n > 0`. + +## Implementation notes + +The working copy lived under `/tmp` during capture (`/tmp/sslhook.c`). +Behavior that mattered for parsing: + +- Log path from `SSLHOOK_LOG`, default `/tmp/sslhook-open.log`. +- Unbuffered writes (`_IONBF`) so a crash still leaves a complete file. +- Each record tagged with the `SSL *` pointer so vCenter HTTPS and + ESXi:902 are separable. An earlier version omitted the pointer and + mixed both streams into one timeline. +- Payload stored as hex, not mixed ASCII, so binary NFC frames stay + unambiguous. + +Record layout: + +``` +==== W 0x7f8a1234 46 ==== +53455353494f4e2035326364656263352d...0d0a +``` + +| Field | Meaning | +| ------- | ---------------------------------------------------- | +| `W`/`R` | Write (plaintext to encrypt) or read (decrypted) | +| `%p` | `SSL *` for this socket | +| length | Byte count of this OpenSSL call | +| hex | Buffer contents | + +`SSL_read` is often **one byte per call**. A 220 banner is therefore +dozens of `R 1` records. Adjacent records with the same `SSL *` and +direction must be concatenated before parsing lines or NFC headers. + +OpenSSL 3 also has `SSL_write_ex` / `SSL_read_ex`. This VDDK 8.0.2 +build still used `SSL_write` / `SSL_read`, so those two symbols were +enough. If a later library switches APIs, the hook would need matching +wrappers. + +## How it was used for authd + +A minimal ctypes program loaded `libvixDiskLib.so`, called +`VixDiskLib_InitEx`, `ConnectEx` (UID to vCenter, `nbd`), and `Open` on +the lab VMDK. The process was started as: + +```bash +export LD_LIBRARY_PATH=/home/ubuntu/workspace/vmware_nbd_tests/.vddk +export LD_PRELOAD=/tmp/sslhook.so +export SSLHOOK_LOG=/tmp/sslhook-open.log +python /tmp/vddk_open_trace.py +``` + +`LD_LIBRARY_PATH` is required so VDDK uses its bundled `libssl.so.3`. +`LD_PRELOAD` still interposes that copy. + +After the run, records were grouped by `SSL *`. The ESXi connection is +the one whose writes contain `SESSION ` and `PROXY `. Concatenating +that stream after the TLS handshake produced: + +``` +C -> SESSION \r\n +C -> BANNER \r\n +S -> 220 VMware Authentication Daemon Version 1.10: ...\r\n +C -> THUMBPRINT_SHA2 PlainText\r\n +S -> 200 \r\n +C -> PROXY vpxa-nfc\r\n +S -> 200 Connect ha-nfc\r\n +``` + +The same log also showed the SOAP `NfcGetVmFiles` body on the vCenter +`SSL *` (`xmlns="urn:vim25"`, moref `nfcService`). That is how the +ticket call was identified as `NfcGetVmFiles` rather than guessing +from `libvim-types` strings alone. + +Details that only the hex dump made obvious: + +- `BANNER` includes a trailing space (`BANNER \r\n`). +- `THUMBPRINT_SHA2` argument is the literal `PlainText`, not the + ticket SHA-1 thumbprint. +- `SESSION` has no reply; waiting for a line after it looks like a hang. +- Ticket `sessionId` is the UUID string on the `SESSION` line. + +Those facts are written up in `docs/nfc_auth.md`. The Python client in +`openvixdisklib/nfc_auth.py` replays this sequence; it does not use +the hook. + +After `200 Connect`, NFC is **not** on `SSL_write`. VDDK uses +`write`/`read` on `SSL_get_fd` (`useSSL=0`). A later hook that also +interposed those syscalls, filtered to peer port 902, recovered the +264-byte handshake and AIO frames in `docs/nfc_open.md`. TLS record +bytes (`16 03` / `17 03`) on that fd are the authd phase and must be +skipped. + +## Limits + +- The hook sees every OpenSSL client in the process (VDDK and, if the + same interpreter is used, anything else linked to OpenSSL). Filter by + `SSL *`. +- It does not decode TLS handshakes, certificates, or SOAP envelopes; + that is done offline on the hex log. +- It must not ship in a production VDDK replacement. Keep it out of + the library path used by `openvixdisklib/nfc_auth.py`. diff --git a/openvixdisklib/__init__.py b/openvixdisklib/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/openvixdisklib/nfc_auth.py b/openvixdisklib/nfc_auth.py new file mode 100644 index 0000000..c4d2bdd --- /dev/null +++ b/openvixdisklib/nfc_auth.py @@ -0,0 +1,371 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""VDDK-compatible vSphere NFC authentication. + +VixDiskLib_ConnectEx / Open authenticate in two stages: + +1. SOAP login to vCenter (or ESXi) and an internal NfcService call that + returns a one-time vim.HostServiceTicket. +2. A TLS session to the ESXi authd daemon on TCP 902, completed with the + ticket's sessionId and service name. + +pyVim / pyVmomi are used for every public VIM operation (login, inventory, +HostServiceTicket). NfcService is not in the public WSDL, so it is registered +with pyVmomi's type system and invoked through the same SOAP stub. +""" + +from __future__ import annotations + +import hashlib +import socket +import ssl +from typing import Optional + +from pyVim.connect import Disconnect, SmartConnect +from pyVmomi import vim +from pyVmomi.VmomiSupport import CreateManagedType, F_OPTIONAL, GetVmodlType + +NFC_SERVICE_MOID = "nfcService" +AUTHD_DEFAULT_PORT = 902 +_NFC_TYPES_REGISTERED = False + + +def _ssl_client_context(verify: bool = True) -> ssl.SSLContext: + """Return a client TLS context built with public ``ssl`` APIs.""" + context = ssl.create_default_context() + if not verify: + context.check_hostname = False + context.verify_mode = ssl.CERT_NONE + return context + + +def _register_nfc_types() -> None: + """Register internal vim.NfcService methods on the pyVmomi type map.""" + global _NFC_TYPES_REGISTERED + if _NFC_TYPES_REGISTERED: + return + try: + GetVmodlType("vim.NfcService") + _NFC_TYPES_REGISTERED = True + return + except Exception: + pass + + CreateManagedType( + "vim.NfcService", + "NfcService", + "vmodl.ManagedObject", + "vim.version.version1", + [], + [ + ("getVmFiles", "NfcGetVmFiles", "vim.version.version1", + (("vm", "vim.VirtualMachine", "vim.version.version1", 0, None),), + (0, "vim.HostServiceTicket", "vim.HostServiceTicket"), None, None), + ("randomAccessOpen", "NfcRandomAccessOpenDisk", + "vim.version.version1", + (("vm", "vim.VirtualMachine", "vim.version.version1", 0, None), + ("diskDeviceKey", "int", "vim.version.version1", 0, None), + ("hostForAccess", "vim.HostSystem", "vim.version.version1", + F_OPTIONAL, None),), + (0, "vim.HostServiceTicket", "vim.HostServiceTicket"), None, None), + ("randomAccessOpenReadonly", "NfcRandomAccessOpenReadonly", + "vim.version.version1", + (("vm", "vim.VirtualMachine", "vim.version.version1", 0, None), + ("diskDeviceKey", "int", "vim.version.version1", 0, None), + ("hostForAccess", "vim.HostSystem", "vim.version.version1", + F_OPTIONAL, None),), + (0, "vim.HostServiceTicket", "vim.HostServiceTicket"), None, None), + ("getServerNfcLibVersion", "NfcGetServerNfcLibVersion", + "vim.version.version1", + (("hostForAccess", "vim.HostSystem", "vim.version.version1", + 0, None),), + (0, "int", "int"), None, None), + ], + ) + _NFC_TYPES_REGISTERED = True + + +def nfc_service(si: vim.ServiceInstance) -> vim.NfcService: + """Return the vCenter/ESXi NfcService managed object on ``si``'s SOAP stub. + + Args: + si: An authenticated ServiceInstance from pyVim.connect.SmartConnect. + """ + _register_nfc_types() + nfc_cls = GetVmodlType("vim.NfcService") + return nfc_cls(NFC_SERVICE_MOID, si._stub) + + +def connect_vim( + host: str, + username: str, + password: str, + port: int = 443, + thumbprint: Optional[str] = None, + allow_untrusted: bool = False) -> vim.ServiceInstance: + """Login to vCenter or ESXi using pyVim.connect.SmartConnect. + + Args: + host: vCenter or ESXi hostname/IP. + username: VIM user name. + password: VIM password. + port: HTTPS port, usually 443. + thumbprint: Optional SHA-1 SSL thumbprint of the management endpoint. + allow_untrusted: If True, skip certificate validation. + """ + ssl_context = None + if allow_untrusted: + ssl_context = _ssl_client_context(verify=False) + return SmartConnect( + host=host, + user=username, + pwd=password, + port=port, + thumbprint=thumbprint, + sslContext=ssl_context, + disableSslCertValidation=allow_untrusted) + + +def _virtual_disk_key(vm: vim.VirtualMachine, disk_path: str) -> int: + """Return the VirtualDisk device key whose backing path is ``disk_path``.""" + for device in vm.config.hardware.device: + if isinstance(device, vim.vm.device.VirtualDisk): + backing = getattr(device, "backing", None) + file_name = getattr(backing, "fileName", None) + if file_name == disk_path: + return device.key + raise ValueError( + f"VMDK path {disk_path!r} is not attached to {vm._moId}") + + +def get_nfc_ticket( + si: vim.ServiceInstance, + vm: vim.VirtualMachine, + disk_device_key: Optional[int] = None, + host_for_access: Optional[vim.HostSystem] = None, + read_only: bool = True, + disk_path: Optional[str] = None) -> vim.HostServiceTicket: + """Return a one-time NFC HostServiceTicket for ``vm``. + + Matches VDDK: ``NfcGetVmFiles`` when only the VM is known (read-only), + ``NfcRandomAccessOpenReadonly`` / ``NfcRandomAccessOpenDisk`` when a + virtual disk device key (or datastore path) is supplied. + + Args: + si: Authenticated ServiceInstance. + vm: Target virtual machine. + disk_device_key: Optional VirtualDisk.device key (for example 2000). + host_for_access: Host that should serve NFC; defaults to the VM's host. + read_only: When False, request a writable ticket (needs a disk). + disk_path: Datastore path used to resolve ``disk_device_key``. + """ + nfc = nfc_service(si) + if read_only and disk_device_key is None and disk_path is None: + return nfc.GetVmFiles(vm) + if disk_device_key is None: + if disk_path is None: + raise ValueError( + "writable NFC tickets need disk_path or disk_device_key") + disk_device_key = _virtual_disk_key(vm, disk_path) + if host_for_access is None: + host_for_access = vm.runtime.host + if read_only: + return nfc.RandomAccessOpenReadonly( + vm, disk_device_key, host_for_access) + return nfc.RandomAccessOpen(vm, disk_device_key, host_for_access) + + +def _format_thumbprint(digest: bytes) -> str: + return ":".join(f"{byte:02X}" for byte in digest) + + +def _sha1_thumbprint(der_cert: bytes) -> str: + return _format_thumbprint(hashlib.sha1(der_cert).digest()) + + +def _normalize_thumbprint(thumbprint: str) -> str: + return thumbprint.replace(":", "").replace(" ", "").upper() + + +def get_ssl_cert_thumbprint( + host: str, + port: int = 443, + digest_algorithm: str = "sha1", + ssl_context: Optional[ssl.SSLContext] = None, + timeout: float = 30.0) -> str: + """Return the TLS certificate thumbprint of ``host``:``port``. + + Reads the peer certificate in DER form and hashes it with ``hashlib``. + The result is colon-separated uppercase hex (for example + ``A5:AF:7D:…``), matching VDDK / pyVmomi SHA-1 thumbprints. + + Args: + host: Hostname or IP of the TLS server. + port: TLS port, usually 443. + digest_algorithm: Hash name accepted by ``hashlib.new``. Default + ``sha1`` is the format VDDK and pyVmomi expect. + ssl_context: Optional SSL context. When omitted, a default client + context is used with hostname checks and certificate + validation disabled so a self-signed management certificate + can still be read. + timeout: Connect timeout in seconds. + """ + if ssl_context is None: + ssl_context = _ssl_client_context(verify=False) + with socket.create_connection((host, port), timeout=timeout) as sock: + with ssl_context.wrap_socket( + sock, server_hostname=host) as ssock: + cert = ssock.getpeercert(binary_form=True) + if not cert: + raise ConnectionError( + f"no peer certificate from {host}:{port}") + return _format_thumbprint( + hashlib.new(digest_algorithm, cert).digest()) + + +def _readline(sock: socket.socket) -> str: + buf = b"" + while not buf.endswith(b"\n"): + chunk = sock.recv(1) + if not chunk: + raise ConnectionError("authd connection closed") + buf += chunk + if len(buf) > 4096: + raise ConnectionError("oversized authd response") + return buf.decode("ascii", "replace").rstrip("\r\n") + + +def _expect_code(line: str, code: str, what: str) -> str: + if not line.startswith(code): + raise ConnectionError(f"authd {what} failed: {line}") + return line[len(code):].lstrip() + + +def connect_authd( + ticket: vim.HostServiceTicket, + allow_untrusted: bool = False, + timeout: float = 30.0) -> ssl.SSLSocket: + """Complete the ESXi authd handshake using an NFC HostServiceTicket. + + Wire sequence captured from VDDK against authd on TCP 902: + + 1. Read the plaintext 220 banner, then wrap the socket with TLS. + 2. SESSION + 3. BANNER + 4. THUMBPRINT_SHA2 PlainText (NFC data stays on this TLS socket) + 5. PROXY (vpxa-nfc when connecting via vCenter) + + Args: + ticket: One-time ticket from get_nfc_ticket(). + allow_untrusted: If False, require the peer SHA-1 thumbprint to match + ticket.sslThumbprint. + timeout: Socket timeout in seconds. + """ + host = ticket.host + port = ticket.port or AUTHD_DEFAULT_PORT + raw = socket.create_connection((host, port), timeout=timeout) + try: + banner = _readline(raw) + if not banner.startswith("220"): + raise ConnectionError(f"unexpected authd banner: {banner}") + + ssl_context = _ssl_client_context(verify=False) + ssock = ssl_context.wrap_socket(raw, server_hostname=host) + except Exception: + raw.close() + raise + + try: + if not allow_untrusted and ticket.sslThumbprint: + peer = _sha1_thumbprint(ssock.getpeercert(True)) + if _normalize_thumbprint(peer) != _normalize_thumbprint( + ticket.sslThumbprint): + raise ConnectionError( + f"ESXi SSL thumbprint mismatch: got {peer}, " + f"expected {ticket.sslThumbprint}") + + ssock.sendall(f"SESSION {ticket.sessionId}\r\n".encode("ascii")) + # Trailing space is part of the BANNER command token used by authd. + ssock.sendall(b"BANNER \r\n") + _expect_code(_readline(ssock), "220", "BANNER") + + ssock.sendall(b"THUMBPRINT_SHA2 PlainText\r\n") + _expect_code(_readline(ssock), "200", "THUMBPRINT_SHA2") + + service = ticket.service or "vpxa-nfc" + ssock.sendall(f"PROXY {service}\r\n".encode("ascii")) + _expect_code(_readline(ssock), "200", "PROXY") + return ssock + except Exception: + ssock.close() + raise + + +class NfcAuthSession: + """Authenticated VIM session plus an authd/NFC TLS socket.""" + + def __init__( + self, + si: vim.ServiceInstance, + ticket: vim.HostServiceTicket, + authd_sock: ssl.SSLSocket) -> None: + self.si = si + self.ticket = ticket + self.authd_sock = authd_sock + + def close(self) -> None: + """Close the authd socket and logout of the VIM session.""" + try: + self.authd_sock.close() + finally: + Disconnect(self.si) + + def __enter__(self) -> "NfcAuthSession": + return self + + def __exit__(self, exc_type, exc, tb) -> None: + self.close() + + +def authenticate( + host: str, + username: str, + password: str, + vm_moref: str, + port: int = 443, + thumbprint: Optional[str] = None, + allow_untrusted: bool = False, + disk_device_key: Optional[int] = None, + disk_path: Optional[str] = None, + read_only: bool = True) -> NfcAuthSession: + """Login to vSphere and complete NFC authd authentication for a VM. + + Args: + host: vCenter or ESXi hostname/IP. + username: VIM user name. + password: VIM password. + vm_moref: Virtual machine managed object id (for example ``vm-13098``). + port: HTTPS port for VIM, usually 443. + thumbprint: Optional SHA-1 thumbprint of the management endpoint. + allow_untrusted: Skip TLS certificate checks when True. + disk_device_key: Optional VirtualDisk device key; when omitted with + ``read_only``, the VDDK ``NfcGetVmFiles`` ticket is used. + disk_path: Datastore path used to resolve ``disk_device_key``. + read_only: When False, request a writable ``NfcRandomAccessOpenDisk`` + ticket. + """ + si = connect_vim( + host, username, password, port=port, + thumbprint=thumbprint, allow_untrusted=allow_untrusted) + try: + vm = vim.VirtualMachine(vm_moref, si._stub) + ticket = get_nfc_ticket( + si, vm, disk_device_key=disk_device_key, + disk_path=disk_path, read_only=read_only) + authd_sock = connect_authd( + ticket, allow_untrusted=allow_untrusted) + except Exception: + Disconnect(si) + raise + return NfcAuthSession(si, ticket, authd_sock) diff --git a/openvixdisklib/nfc_open.py b/openvixdisklib/nfc_open.py new file mode 100644 index 0000000..b291913 --- /dev/null +++ b/openvixdisklib/nfc_open.py @@ -0,0 +1,425 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""VDDK-compatible NFC disk open, sector read, and sector write. + +After ``nfc_auth.connect_authd`` returns ``200 Connect``, VDDK stops using +``SSL_write`` on the authd socket. ``THUMBPRINT_SHA2 PlainText`` means the +NFC binary protocol runs as raw TCP on that same file descriptor +(``useSSL=0``). This module dups that fd and speaks: + +1. Classic 264-byte NFC messages (handshake, version, connection data, + AIO session open). +2. NFC AIO frames (16-byte header plus payload) to open a VMDK and read + or write sectors. + +pyVmomi is not involved here; the ticket and TLS authd handshake already +happened in ``nfc_auth``. +""" + +from __future__ import annotations + +import os +import socket +import ssl +import struct + +from openvixdisklib.nfc_auth import NfcAuthSession + +NFC_MSG_SIZE = 264 +NFC_AIO_MAGIC = 0xA100DA7A +NFC_AIO_HDR_SIZE = 16 +NFC_SECTOR_SIZE = 512 +NFC_PROTOCOL_VERSION = 11 +# Max data bytes in one AIO IO reply fragment (NfcAioInitSession buffer). +NFC_AIO_BUFFER_SIZE = 65536 + +# Classic NFC message types observed on the wire (uint32 at offset 0). +NFC_MSG_SESSION_COMPLETE = 4 +NFC_MSG_SESSION_PARAMS = 33 +NFC_MSG_SESSION_PARAMS_REPLY = 36 +NFC_MSG_HANDSHAKE = 43 +NFC_MSG_VERSION = 51 +NFC_MSG_AIO_SESSION_OPEN = 52 +NFC_MSG_CONNECTION_DATA = 54 +NFC_MSG_SESSION_FEATURES = 55 + +# SessionParams / feature bits from VDDK logs (interruption | switch). +NFC_SESSION_FEATURE_INTERRUPTION_SWITCH = 3 + +# AIO message types (NfcAioSendMessage "type = N"). +NFC_AIO_MSG_ERROR = 1 +NFC_AIO_MSG_OPEN_SESSION = 2 +NFC_AIO_MSG_CLOSE_SESSION = 3 +NFC_AIO_MSG_OPEN_FILE = 4 +NFC_AIO_MSG_CLOSE_FILE = 5 +NFC_AIO_MSG_IO = 7 +NFC_AIO_MSG_SET_SOCK_OPTS = 9 +NFC_AIO_MSG_DDB_GET = 11 +NFC_AIO_MSG_SET_RES_POOL = 22 + +# Open-file body: file type NFC_DISK. 0x1e is what VDDK sends for +# VIXDISKLIB_FLAG_OPEN_READ_ONLY; writable opens clear bit 0x04 (0x1a). +NFC_DISK = 2 +NFC_OPEN_FLAGS_READ_ONLY = 0x1E +NFC_OPEN_FLAGS_READ_WRITE = 0x1A + +NFC_AIO_IO_WRITE = 0 +NFC_AIO_IO_READ = 1 + + +class NfcProtocolError(ConnectionError): + """Raised when an NFC message is malformed or reports failure.""" + + +def takeover_authd_socket(ssock: ssl.SSLSocket) -> socket.socket: + """Return a raw socket on the authd TCP connection. + + VDDK writes NFC with ``write(SSL_get_fd(ssl), ...)`` after PROXY, so + those bytes are not TLS records. Duping the fd lets Python do the + same without ``SSLSocket.send`` re-encrypting, and without + ``SSL_shutdown``. + + Args: + ssock: The TLS socket from ``nfc_auth.connect_authd``. + """ + timeout = ssock.gettimeout() + raw = socket.socket( + family=ssock.family, + type=ssock.type, + proto=ssock.proto, + fileno=os.dup(ssock.fileno())) + raw.settimeout(timeout) + return raw + + +def _recvn(sock: socket.socket, size: int) -> bytes: + buf = bytearray() + while len(buf) < size: + chunk = sock.recv(size - len(buf)) + if not chunk: + raise NfcProtocolError( + f"NFC connection closed, needed {size} bytes, got {len(buf)}") + buf.extend(chunk) + return bytes(buf) + + +def _send_nfc_msg( + sock: socket.socket, msg_type: int, body: bytes = b"") -> None: + if len(body) > NFC_MSG_SIZE - 4: + raise ValueError("NFC classic message body too large") + frame = struct.pack(" tuple[int, bytes]: + frame = _recvn(sock, NFC_MSG_SIZE) + msg_type = struct.unpack_from(" bytes: + return struct.pack( + " tuple[int, int, int]: + magic, msg_type, size, op_id = struct.unpack_from(" None: + """Wrap an AIO session that already has ``path`` open. + + Args: + sock: Raw NFC socket after handshake. + path: Datastore path that was opened. + handle: Server file handle from OPEN_FILE. + sector_size: Sector size from the OPEN_FILE reply. + """ + self._sock = sock + self._op_id = 0 + self.path = path + self.handle = handle + self.sector_size = sector_size + self._closed = False + + def _next_op_id(self) -> int: + op_id = self._op_id + self._op_id += 1 + return op_id + + def _aio_roundtrip( + self, + msg_type: int, + payload: bytes, + extra: bytes = b"", + extra_recv: int = 0) -> bytes: + """Send one AIO request and return the reply payload (+ extra).""" + op_id = self._next_op_id() + self._sock.sendall( + _pack_aio_hdr(msg_type, len(payload), op_id) + payload) + if extra: + self._sock.sendall(extra) + rhdr = _recvn(self._sock, NFC_AIO_HDR_SIZE) + magic, rtype, rsize, rop = struct.unpack_from(" bytes: + """Read ``num_sectors`` starting at ``start_sector``. + + Matches ``VixDiskLib_Read``: one ``NFC_AIO_MSG_IO`` request in + byte units. If the length exceeds the AIO buffer (64 KiB) the + server replies with several same-``opId`` fragments. + + Args: + start_sector: Sector offset from the start of the disk. + num_sectors: Number of sectors to read. + """ + if num_sectors < 1: + raise ValueError("num_sectors must be at least 1") + length = num_sectors * self.sector_size + offset = start_sector * self.sector_size + payload = struct.pack( + " remaining: + raise NfcProtocolError( + f"AIO IO chunk length {chunk_len} invalid, " + f"remaining {remaining}") + data.extend(_recvn(self._sock, chunk_len)) + return bytes(data) + + def write( + self, + start_sector: int, + num_sectors: int, + data: bytes) -> None: + """Write ``num_sectors`` starting at ``start_sector``. + + Matches ``VixDiskLib_Write``: one ``NFC_AIO_MSG_IO`` request per + chunk in byte units, with sector bytes sent after the 44-byte + payload. Chunks larger than the AIO buffer (64 KiB) are split. + + Args: + start_sector: Sector offset from the start of the disk. + num_sectors: Number of sectors to write. + data: Bytes to write; length must be ``num_sectors * sector_size``. + """ + if num_sectors < 1: + raise ValueError("num_sectors must be at least 1") + length = num_sectors * self.sector_size + if len(data) != length: + raise ValueError( + f"write data is {len(data)} bytes, need {length}") + max_sectors = NFC_AIO_BUFFER_SIZE // self.sector_size + offset_sectors = start_sector + remaining = data + while remaining: + n_sectors = min(len(remaining) // self.sector_size, max_sectors) + chunk = remaining[:n_sectors * self.sector_size] + self._write_once(offset_sectors, n_sectors, chunk) + offset_sectors += n_sectors + remaining = remaining[n_sectors * self.sector_size:] + + def _write_once( + self, + start_sector: int, + num_sectors: int, + data: bytes) -> None: + length = num_sectors * self.sector_size + offset = start_sector * self.sector_size + payload = struct.pack( + " None: + """Close the VMDK, the AIO session, and the classic NFC session.""" + if self._closed: + return + self._closed = True + try: + self._aio_roundtrip( + NFC_AIO_MSG_CLOSE_FILE, struct.pack(" "NfcDisk": + return self + + def __exit__(self, exc_type, exc, tb) -> None: + self.close() + + +def _handshake( + sock: socket.socket, + client_name: str, + op_id: str, + version: int) -> None: + """Run the classic NFC session handshake used by VDDK NBD.""" + _send_nfc_msg(sock, NFC_MSG_HANDSHAKE, b"PlainText") + _send_nfc_msg(sock, NFC_MSG_SESSION_PARAMS) + reply_type, _ = _recv_nfc_msg(sock) + if reply_type != NFC_MSG_SESSION_PARAMS_REPLY: + raise NfcProtocolError( + f"expected session-params reply {NFC_MSG_SESSION_PARAMS_REPLY}, " + f"got {reply_type}") + + _send_nfc_msg(sock, NFC_MSG_VERSION, struct.pack(" None: + disk._aio_roundtrip( + NFC_AIO_MSG_OPEN_SESSION, bytes(16)) + disk._aio_roundtrip( + NFC_AIO_MSG_SET_SOCK_OPTS, bytes(12)) + disk._aio_roundtrip( + NFC_AIO_MSG_SET_RES_POOL, struct.pack(" tuple[int, int]: + if len(body) < 40: + raise NfcProtocolError(f"OPEN_FILE reply too short: {len(body)}") + handle, file_type, _flags = struct.unpack_from(" NfcDisk: + """Open ``disk_path`` over the authenticated authd socket. + + Matches VDDK ``VixDiskLib_Open`` of a datastore path after the NFC + ticket and authd PROXY handshake: session init, AIO open, then + ``NFC_AIO_MSG_OPEN_FILE`` with type ``NFC_DISK``. + + Args: + session: Result of ``nfc_auth.authenticate``. + disk_path: Datastore path, for example + ``[datastore0] vm/vm.vmdk``. + client_name: NFC client name; VDDK sends ``vddk``. + op_id: NFC operation id; VDDK NBD sends ``nbdmode``. + version: Client NFC protocol version (lab ESXi answered 11). + read_only: When True, open with VDDK's read-only NFC flags. + """ + sock = takeover_authd_socket(session.authd_sock) + try: + _handshake(sock, client_name, op_id, version) + disk = NfcDisk(sock, disk_path, handle=0, sector_size=NFC_SECTOR_SIZE) + _aio_prepare(disk) + path_b = disk_path.encode("utf-8") + open_flags = ( + NFC_OPEN_FLAGS_READ_ONLY if read_only + else NFC_OPEN_FLAGS_READ_WRITE) + open_body = struct.pack( + " str: + if not vmx_spec: + raise ValueError( + "vmx_spec is required (for example 'moref=vm-13098')") + if "=" in vmx_spec: + kind, value = vmx_spec.split("=", 1) + if kind.lower() != "moref" or not value: + raise ValueError(f"unsupported vmx_spec: {vmx_spec}") + return value + return vmx_spec + + +def _require_nbd(transport_modes: Optional[str]) -> None: + if transport_modes is None: + return + modes = [m for m in transport_modes.split(":") if m] + if "nbd" not in modes: + raise NotImplementedError( + f"only nbd transport is supported, got {transport_modes!r}") + + +class _Connection: + """VIM session plus the VM moref needed to issue an NFC ticket at Open.""" + + def __init__( + self, + si: vim.ServiceInstance, + vm_moref: str, + snapshot_ref: Optional[str], + thumbprint: Optional[str], + allow_untrusted: bool, + read_only: bool) -> None: + self.si = si + self.vm_moref = vm_moref + self.snapshot_ref = snapshot_ref + self.thumbprint = thumbprint + self.allow_untrusted = allow_untrusted + self.read_only = read_only + + +class _DiskHandle: + """Opened NFC disk plus the authd TLS socket it was taken from.""" + + def __init__( + self, + disk: nfc_open.NfcDisk, + authd_sock) -> None: + self.disk = disk + self.authd_sock = authd_sock + + +class VixDiskLibHandle: + """VDDK-compatible handle backed by pyVmomi and the NFC replacement.""" + + def __init__( + self, + config_path: Optional[str] = None, + vixdisklib_compatibility_version: Optional[str] = None) -> None: + """Accept the VDDK wrapper constructor; no native library is loaded. + + Args: + config_path: Ignored. VDDK used this for logging plugins. + vixdisklib_compatibility_version: Optional ``major.minor`` string + such as ``8.0``. Validated for form only. + """ + del config_path + target_versions = VIX_SUPPORTED_COMPATIBILITY_MODES + if vixdisklib_compatibility_version: + target_versions = [vixdisklib_compatibility_version] + LOG.debug("vixDiskLib versions targeted: %s", target_versions) + + version_used = None + for version in reversed(target_versions): + try: + major_ver, minor_ver = version.split(".") + int(major_ver) + int(minor_ver) + except ValueError as ex: + raise ValueError( + "Unsupported vixDiskLib version format '%s'. vixDiskLib " + "compatibility mode must be of the form " + "'$major.$minor'" % version) from ex + version_used = version + break + + if not version_used: + raise Exception( + "Could not initialize vixDiskLib with any of the following " + "versions: %s" % target_versions) + + LOG.info( + "Successfully initialized vixDiskLib with target version '%s'", + version_used) + + @classmethod + def get_vix_disklib_name(cls) -> str: + """Return the native library name; this replacement does not load it.""" + if os.name == "nt": + return "vixDiskLib.dll" + return "libvixDiskLib.so" + + def get_transport_modes(self) -> list[str]: + """Return the transport modes this replacement implements.""" + return ["nbd"] + + def get_transport_mode(self, disk_handle: _DiskHandle) -> str: + """Return the transport used for ``disk_handle``.""" + del disk_handle + return "nbd" + + @contextlib.contextmanager + def connect( + self, + server_name: str, + thumbprint: Optional[str], + username: str, + password: str, + vmx_spec: Optional[str] = None, + snapshot_ref: Optional[str] = None, + read_only: bool = True, + transport_modes: Optional[str] = None, + port: int = 443, + allow_untrusted: bool = False) -> Iterator[_Connection]: + """Login to vCenter/ESXi. Matches ``VixDiskLib_ConnectEx``. + + The NFC ticket and authd handshake are deferred to ``open``, as in + VDDK. Writable opens use ``NfcRandomAccessOpenDisk``; read-only + opens use ``NfcGetVmFiles``. ``snapshot_ref`` is accepted for API + compatibility and is not sent on the ticket SOAP call. + + Args: + server_name: vCenter or ESXi hostname/IP. + thumbprint: SHA-1 thumbprint of the management TLS certificate. + username: VIM user name. + password: VIM password. + vmx_spec: VM selector, ``moref=vm-…``. + snapshot_ref: Snapshot moref; unused on the NFC ticket. + read_only: When False, the disk may be opened for write. + transport_modes: ``nbd`` or a colon list that includes ``nbd``. + port: HTTPS port, usually 443. + allow_untrusted: Skip management TLS verification when True. + """ + LOG.debug("Connecting VixDiskLib: %s", server_name) + _require_nbd(transport_modes) + vm_moref = _parse_vm_moref(vmx_spec) + si = nfc_auth.connect_vim( + server_name, + username, + password, + port=port, + thumbprint=thumbprint, + allow_untrusted=allow_untrusted or not thumbprint) + conn = _Connection( + si, vm_moref, snapshot_ref, thumbprint, + allow_untrusted or not thumbprint, read_only) + try: + yield conn + finally: + self.disconnect(conn) + + @contextlib.contextmanager + def open( + self, + conn: _Connection, + disk_path: str, + flags: int = VIXDISKLIB_FLAG_OPEN_READ_ONLY) -> Iterator[_DiskHandle]: + """Open ``disk_path`` over NFC. Matches ``VixDiskLib_Open``. + + Args: + conn: Connection from ``connect``. + disk_path: Datastore path of the VMDK. + flags: Open flags. ``VIXDISKLIB_FLAG_OPEN_READ_ONLY`` opens + the disk read-only; omit it for write. Compression flags + are not implemented. + """ + LOG.debug("Openning VixDiskLib disk: %s", disk_path) + if flags & _COMPRESSION_FLAGS: + raise NotImplementedError( + "NBD compression open flags are not supported") + read_only = bool(flags & VIXDISKLIB_FLAG_OPEN_READ_ONLY) + if not read_only and conn.read_only: + raise NotImplementedError( + "ConnectEx was read-only; cannot open for write") + + vm = vim.VirtualMachine(conn.vm_moref, conn.si._stub) + ticket = nfc_auth.get_nfc_ticket( + conn.si, vm, read_only=read_only, disk_path=disk_path) + authd_sock = nfc_auth.connect_authd( + ticket, allow_untrusted=conn.allow_untrusted) + session = nfc_auth.NfcAuthSession(conn.si, ticket, authd_sock) + try: + disk = nfc_open.open_disk( + session, disk_path, read_only=read_only) + except Exception: + authd_sock.close() + raise + handle = _DiskHandle(disk, authd_sock) + try: + yield handle + finally: + self.close(handle) + + def read( + self, + disk_handle: _DiskHandle, + start_sector: int, + num_sectors: int, + buf: Union[ctypes.Array, bytearray, memoryview]) -> None: + """Read ``num_sectors`` from ``start_sector`` into ``buf``. + + Args: + disk_handle: Handle from ``open``. + start_sector: First sector to read. + num_sectors: Number of sectors to read. + buf: Destination buffer (``get_buffer`` or a writable bytes-like). + """ + data = disk_handle.disk.read(start_sector, num_sectors) + if isinstance(buf, (bytearray, memoryview)): + if len(buf) < len(data): + raise Exception( + f"read buffer is {len(buf)} bytes, need {len(data)}") + buf[:len(data)] = data + return + ctypes.memmove(buf, data, len(data)) + + def write( + self, + disk_handle: _DiskHandle, + start_sector: int, + num_sectors: int, + buf: Union[ctypes.Array, bytes, bytearray, memoryview]) -> None: + """Write ``num_sectors`` from ``buf`` starting at ``start_sector``. + + Args: + disk_handle: Handle from ``open``. + start_sector: First sector to write. + num_sectors: Number of sectors to write. + buf: Source buffer (``get_buffer`` or a bytes-like). + """ + length = num_sectors * VIXDISKLIB_SECTOR_SIZE + if isinstance(buf, (bytes, bytearray, memoryview)): + data = bytes(buf[:length]) + else: + data = buf.raw[:length] + disk_handle.disk.write(start_sector, num_sectors, data) + + def close(self, disk_handle: _DiskHandle) -> None: + """Close the VMDK and the authd socket used for NFC. + + Args: + disk_handle: Handle from ``open``. + """ + LOG.debug("Closing VixDiskLib disk handle: %s", disk_handle) + try: + disk_handle.disk.close() + finally: + try: + disk_handle.authd_sock.close() + except OSError: + pass + + def disconnect(self, conn: _Connection) -> None: + """Logout of the VIM session. + + Args: + conn: Connection from ``connect``. + """ + LOG.debug("Disconnecting VixDiskLib") + Disconnect(conn.si) + + def exit(self) -> None: + """No-op; there is no native VDDK library to tear down.""" + return diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..8d002bd --- /dev/null +++ b/requirements.txt @@ -0,0 +1,5 @@ +pbr +pyOpenSSL +pyVmomi>=7.0 +PyYAML +requests diff --git a/ruff.toml b/ruff.toml new file mode 100644 index 0000000..c64fc5c --- /dev/null +++ b/ruff.toml @@ -0,0 +1,11 @@ +line-length = 88 +target-version = "py312" + +[lint] +# UP031: Allow percent formatting for now. +# PIE790: Unnecessary `pass` statement - actually improves readability. +# RUF100: Unused blanket `noqa` directive - used by other linters. +# BLE001: we'll allow catching Exception for now. +# RUF015: enforces next() to be used when getting the first item of a list, +# may be enabled at a later time. +ignore = ["UP031", "PIE790", "RUF100", "BLE001", "RUF015"] diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 0000000..55a6a75 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,28 @@ +[metadata] +name = openvixdisklib +summary = + A reverse engineered replacement for the proprietary VMware VDDK vixdisklib + library. +description-file = README.md +author = Cloudbase Solutions SRL +author-email = info@cloudbasesolutions.com +home-page = http://cloudbase.it +classifier = + Environment :: VMware + Intended Audience :: Information Technology + Intended Audience :: System Administrators + Operating System :: OS Independent + Programming Language :: Python + Programming Language :: Python :: 3 + Programming Language :: Python :: 3.12 + +[files] +packages = + openvixdisklib + +[global] +setup-hooks = + pbr.hooks.setup_hook + +[wheel] +universal = 1 diff --git a/setup.py b/setup.py new file mode 100644 index 0000000..6740b8d --- /dev/null +++ b/setup.py @@ -0,0 +1,6 @@ +import setuptools + + +setuptools.setup( + setup_requires=['pbr>=1.8'], + pbr=True) diff --git a/test-requirements.txt b/test-requirements.txt new file mode 100644 index 0000000..8bcd6eb --- /dev/null +++ b/test-requirements.txt @@ -0,0 +1,4 @@ +coverage +discover +ddt +stestr diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/__init__.py b/tests/integration/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/base.py b/tests/integration/base.py new file mode 100644 index 0000000..532b9d8 --- /dev/null +++ b/tests/integration/base.py @@ -0,0 +1,336 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Test base classes for openvixdisklib integration tests.""" + +from __future__ import annotations + +import ctypes +import os +import time +import unittest +import uuid +from typing import Any, Optional + +import yaml +from pyVim.connect import Disconnect +from pyVmomi import vim + +from openvixdisklib import nfc_auth +from openvixdisklib.nfc_auth import NfcAuthSession + +_REPO_ROOT = os.path.abspath( + os.path.join(os.path.dirname(__file__), "..", "..")) +_CONFIG_PATH = os.path.join(_REPO_ROOT, ".test_config.yaml") +_CONFIG_KEYS = ( + "host", + "port", + "username", + "password", + "allow_untrusted", + "datacenter", + "datastore", +) +_VDDK_DIR = os.path.join(_REPO_ROOT, ".vddk") +_VDDK_LIB = os.path.join(_VDDK_DIR, "libvixDiskLib.so") +_DISK_CAPACITY_KB = 10 * 1024 * 1024 +_TASK_POLL_S = 0.5 +_TASK_TIMEOUT_S = 300 +_LAB_VM_PREFIX = "ovdl-test-" + + +class TestBase(unittest.TestCase): + """Shared lab vSphere settings for live NFC / VDDK integration tests.""" + + HOST: str + PORT: int + USERNAME: str + PASSWORD: str + ALLOW_UNTRUSTED: bool + DATACENTER: str + DATASTORE: str + THUMBPRINT: str + VM_MOREF: str + VMX_SPEC: str + DISK_PATH: str + SECTOR_SIZE = 512 + SECTOR_AT_1GB = (1024 * 1024 * 1024) // SECTOR_SIZE + VDDK_DIR = _VDDK_DIR + _lab_refcount = 0 + _lab_vm_moref: Optional[str] = None + _lab_vm_name: Optional[str] = None + + @classmethod + def setUpClass(cls) -> None: + """Prepare process environment and create a temporary lab VM.""" + super().setUpClass() + os.environ.pop("LD_PRELOAD", None) + cls._ensure_vddk_library_path() + TestBase._acquire_lab() + + @classmethod + def tearDownClass(cls) -> None: + """Release the temporary lab VM when the last test class finishes.""" + TestBase._release_lab() + super().tearDownClass() + + def setUp(self) -> None: + """Reset per-test state; subclasses may reuse this.""" + super().setUp() + + @classmethod + def _ensure_vddk_library_path(cls) -> None: + current = os.environ.get("LD_LIBRARY_PATH", "") + parts = [p for p in current.split(":") if p] + if cls.VDDK_DIR not in parts: + os.environ["LD_LIBRARY_PATH"] = ( + cls.VDDK_DIR if not current else f"{cls.VDDK_DIR}:{current}") + + @classmethod + def require_vddk(cls) -> None: + """Skip when ``libvixDiskLib`` cannot be loaded from ``.vddk``.""" + cls._ensure_vddk_library_path() + try: + ctypes.CDLL(_VDDK_LIB) + except OSError as exc: + raise unittest.SkipTest( + f"VDDK library not available at {_VDDK_LIB}: {exc}") from exc + + @classmethod + def _load_test_config(cls) -> None: + """Load lab settings from the repo-root ``.test_config.yaml``.""" + if not os.path.isfile(_CONFIG_PATH): + raise unittest.SkipTest( + "integration tests need .test_config.yaml in the repo " + "root; see README.md for a sample") + with open(_CONFIG_PATH, encoding="utf-8") as config_file: + data = yaml.safe_load(config_file) or {} + missing = [key for key in _CONFIG_KEYS if key not in data] + if missing: + raise RuntimeError( + f"{_CONFIG_PATH} is missing keys: {', '.join(missing)}") + TestBase.HOST = str(data["host"]) + TestBase.PORT = int(data["port"]) + TestBase.USERNAME = str(data["username"]) + TestBase.PASSWORD = str(data["password"]) + TestBase.ALLOW_UNTRUSTED = bool(data["allow_untrusted"]) + TestBase.DATACENTER = str(data["datacenter"]) + TestBase.DATASTORE = str(data["datastore"]) + + @classmethod + def _connect_vim(cls) -> vim.ServiceInstance: + return nfc_auth.connect_vim( + cls.HOST, + cls.USERNAME, + cls.PASSWORD, + port=cls.PORT, + thumbprint=cls.THUMBPRINT, + allow_untrusted=cls.ALLOW_UNTRUSTED) + + @classmethod + def _wait_for_task(cls, task: vim.Task) -> Any: + deadline = time.monotonic() + _TASK_TIMEOUT_S + while task.info.state in ( + vim.TaskInfo.State.running, vim.TaskInfo.State.queued): + if time.monotonic() > deadline: + raise TimeoutError( + f"timed out waiting for vSphere task {task}") + time.sleep(_TASK_POLL_S) + if task.info.state != vim.TaskInfo.State.success: + raise RuntimeError(f"vSphere task failed: {task.info.error}") + return task.info.result + + @classmethod + def _find_datacenter( + cls, content: vim.ServiceInstanceContent) -> vim.Datacenter: + matches = [ + entity for entity in content.rootFolder.childEntity + if isinstance(entity, vim.Datacenter) + and entity.name == cls.DATACENTER] + if not matches: + raise RuntimeError(f"datacenter {cls.DATACENTER!r} not found") + return matches[0] + + @classmethod + def _find_datastore(cls, datacenter: vim.Datacenter) -> vim.Datastore: + matches = [ + datastore for datastore in datacenter.datastore + if datastore.name == cls.DATASTORE] + if not matches: + raise RuntimeError( + f"datastore {cls.DATASTORE!r} not found in " + f"datacenter {cls.DATACENTER!r}") + return matches[0] + + @classmethod + def _bind_lab_fields(cls) -> None: + """Copy shared lab VM fields onto the active test class.""" + cls.HOST = TestBase.HOST + cls.PORT = TestBase.PORT + cls.USERNAME = TestBase.USERNAME + cls.PASSWORD = TestBase.PASSWORD + cls.ALLOW_UNTRUSTED = TestBase.ALLOW_UNTRUSTED + cls.DATACENTER = TestBase.DATACENTER + cls.DATASTORE = TestBase.DATASTORE + cls.THUMBPRINT = TestBase.THUMBPRINT + cls.VM_MOREF = TestBase.VM_MOREF + cls.VMX_SPEC = TestBase.VMX_SPEC + cls.DISK_PATH = TestBase.DISK_PATH + + @classmethod + def _acquire_lab(cls) -> None: + if TestBase._lab_refcount == 0: + TestBase._load_test_config() + TestBase.THUMBPRINT = nfc_auth.get_ssl_cert_thumbprint( + TestBase.HOST, TestBase.PORT) + TestBase._create_lab_vm() + TestBase._lab_refcount += 1 + cls._bind_lab_fields() + + @classmethod + def _release_lab(cls) -> None: + if TestBase._lab_refcount == 0: + return + TestBase._lab_refcount -= 1 + if TestBase._lab_refcount == 0: + cls._destroy_lab_vm() + + @classmethod + def _create_lab_vm(cls) -> None: + """Create an empty VM with a 10 GiB thin disk for I/O tests.""" + si = cls._connect_vim() + vm = None + try: + content = si.RetrieveContent() + datacenter = cls._find_datacenter(content) + datastore = cls._find_datastore(datacenter) + if not datastore.host: + raise RuntimeError( + f"datastore {cls.DATASTORE!r} is not mounted on any host") + host = datastore.host[0].key + pool = host.parent.resourcePool + vm_name = _LAB_VM_PREFIX + uuid.uuid4().hex[:12] + vm = cls._wait_for_task( + datacenter.vmFolder.CreateVM_Task( + config=cls._vm_config_spec(vm_name, datastore.name), + pool=pool, + host=host)) + TestBase._lab_vm_moref = vm._moId + TestBase._lab_vm_name = vm_name + TestBase.VM_MOREF = vm._moId + TestBase.VMX_SPEC = f"moref={vm._moId}" + disks = [ + device.backing.fileName + for device in vm.config.hardware.device + if isinstance(device, vim.vm.device.VirtualDisk)] + if not disks: + raise RuntimeError( + f"temporary VM {vm_name!r} has no virtual disks") + TestBase.DISK_PATH = disks[0] + except Exception: + if vm is not None: + try: + cls._wait_for_task(vm.Destroy_Task()) + except Exception: + pass + TestBase._lab_vm_moref = None + TestBase._lab_vm_name = None + raise + finally: + Disconnect(si) + + @classmethod + def _vm_config_spec( + cls, vm_name: str, datastore_name: str) -> vim.vm.ConfigSpec: + config = vim.vm.ConfigSpec() + config.name = vm_name + config.guestId = "otherGuest64" + config.memoryMB = 128 + config.numCPUs = 1 + config.files = vim.vm.FileInfo( + vmPathName=f"[{datastore_name}]") + + controller = vim.vm.device.ParaVirtualSCSIController() + controller.key = 1000 + controller.busNumber = 0 + controller.sharedBus = ( + vim.vm.device.VirtualSCSIController.Sharing.noSharing) + controller_spec = vim.vm.device.VirtualDeviceSpec() + controller_spec.operation = ( + vim.vm.device.VirtualDeviceSpec.Operation.add) + controller_spec.device = controller + + backing = vim.vm.device.VirtualDisk.FlatVer2BackingInfo() + backing.diskMode = "persistent" + backing.thinProvisioned = True + backing.fileName = f"[{datastore_name}]" + disk = vim.vm.device.VirtualDisk() + disk.key = 2000 + disk.controllerKey = 1000 + disk.unitNumber = 0 + disk.capacityInKB = _DISK_CAPACITY_KB + disk.backing = backing + disk_spec = vim.vm.device.VirtualDeviceSpec() + disk_spec.operation = vim.vm.device.VirtualDeviceSpec.Operation.add + disk_spec.fileOperation = ( + vim.vm.device.VirtualDeviceSpec.FileOperation.create) + disk_spec.device = disk + + config.deviceChange = [controller_spec, disk_spec] + return config + + @classmethod + def _destroy_lab_vm(cls) -> None: + """Power off and delete the temporary lab VM if it still exists.""" + moref = TestBase._lab_vm_moref + TestBase._lab_vm_moref = None + TestBase._lab_vm_name = None + if not moref: + return + si = cls._connect_vim() + try: + vm = vim.VirtualMachine(moref, si._stub) + try: + vm.Reload() + except Exception: + return + if vm.runtime.powerState == vim.VirtualMachinePowerState.poweredOn: + cls._wait_for_task(vm.PowerOffVM_Task()) + cls._wait_for_task(vm.Destroy_Task()) + finally: + Disconnect(si) + + def authenticate(self, read_only: bool = True) -> NfcAuthSession: + """Login to the lab vCenter and complete NFC authd for the temp VM.""" + return nfc_auth.authenticate( + host=self.HOST, + username=self.USERNAME, + password=self.PASSWORD, + vm_moref=self.VM_MOREF, + thumbprint=self.THUMBPRINT, + allow_untrusted=self.ALLOW_UNTRUSTED, + disk_path=None if read_only else self.DISK_PATH, + read_only=read_only) + + def vixdisklib_connect_kwargs( + self, extra: Optional[dict[str, Any]] = None) -> dict[str, Any]: + """Return common ``VixDiskLib_ConnectEx`` arguments for the temp VM.""" + kwargs: dict[str, Any] = { + "server_name": self.HOST, + "port": self.PORT, + "thumbprint": self.THUMBPRINT, + "username": self.USERNAME, + "password": self.PASSWORD, + "vmx_spec": self.VMX_SPEC, + "transport_modes": "nbd", + "read_only": False, + } + if extra: + kwargs.update(extra) + return kwargs + + def pattern_bytes(self, length: int, seed: bytes) -> bytes: + """Return ``length`` bytes by repeating ``seed``.""" + if not seed: + raise ValueError("seed must be non-empty") + return (seed * ((length // len(seed)) + 1))[:length] diff --git a/tests/integration/test_crosscheck.py b/tests/integration/test_crosscheck.py new file mode 100644 index 0000000..18156b6 --- /dev/null +++ b/tests/integration/test_crosscheck.py @@ -0,0 +1,94 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Compare writes and reads from VDDK with openvixdisklib.""" + +from typing import Any, Optional + +from openvixdisklib import openvixdisklib as open_vix +from tests.integration import vixdisklib +from tests.integration.base import TestBase + + +class CrosscheckTest(TestBase): + @classmethod + def setUpClass(cls) -> None: + """Skip when the bundled VDDK shared library is not present.""" + cls.require_vddk() + super().setUpClass() + + def _connect_extra(self, module: Any) -> Optional[dict[str, Any]]: + """Return extra ``connect`` kwargs needed by ``module``.""" + if module is open_vix: + return {"allow_untrusted": self.ALLOW_UNTRUSTED} + return None + + def _write_sectors( + self, + module: Any, + payloads: dict[int, bytes]) -> None: + """Write one sector at each index using a vixdisklib-compatible module.""" + handle = module.VixDiskLibHandle( + vixdisklib_compatibility_version="8.0", + config_path=None) + buf = module.get_buffer(self.SECTOR_SIZE) + kwargs = self.vixdisklib_connect_kwargs(self._connect_extra(module)) + with handle.connect(**kwargs) as conn: + with handle.open(conn, self.DISK_PATH, flags=0) as disk: + for start, data in payloads.items(): + buf[:self.SECTOR_SIZE] = data + handle.write(disk, start, 1, buf) + + def _read_sectors( + self, + module: Any, + sectors: tuple[int, ...]) -> dict[int, bytes]: + """Read one sector at each index using a vixdisklib-compatible module.""" + handle = module.VixDiskLibHandle( + vixdisklib_compatibility_version="8.0", + config_path=None) + buf = module.get_buffer(self.SECTOR_SIZE) + result: dict[int, bytes] = {} + kwargs = self.vixdisklib_connect_kwargs(self._connect_extra(module)) + with handle.connect(**kwargs) as conn: + with handle.open(conn, self.DISK_PATH, flags=0) as disk: + for start in sectors: + buf[:self.SECTOR_SIZE] = b"\xa5" * self.SECTOR_SIZE + handle.read(disk, start, 1, buf) + result[start] = buf.raw[:self.SECTOR_SIZE] + return result + + def _assert_both_read( + self, + sectors: tuple[int, ...], + expected: dict[int, bytes]) -> None: + vddk = self._read_sectors(vixdisklib, sectors) + replacement = self._read_sectors(open_vix, sectors) + for start in sectors: + self.assertEqual( + vddk[start], expected[start], + f"VDDK mismatch at sector {start}") + self.assertEqual( + replacement[start], expected[start], + f"openvixdisklib mismatch at sector {start}") + + def test_openvixdisklib_matches_vddk_sectors(self) -> None: + """Writes from either library must be visible to both readers.""" + sectors = (0, 1, self.SECTOR_AT_1GB) + vddk_payloads = { + 0: self.pattern_bytes(self.SECTOR_SIZE, b"XCHK-VDDK-S0"), + 1: self.pattern_bytes(self.SECTOR_SIZE, b"XCHK-VDDK-S1"), + self.SECTOR_AT_1GB: self.pattern_bytes( + self.SECTOR_SIZE, b"XCHK-VDDK-1G"), + } + self._write_sectors(vixdisklib, vddk_payloads) + self._assert_both_read(sectors, vddk_payloads) + + ovdl_payloads = { + 0: self.pattern_bytes(self.SECTOR_SIZE, b"XCHK-OVDL-S0"), + 1: self.pattern_bytes(self.SECTOR_SIZE, b"XCHK-OVDL-S1"), + self.SECTOR_AT_1GB: self.pattern_bytes( + self.SECTOR_SIZE, b"XCHK-OVDL-1G"), + } + self._write_sectors(open_vix, ovdl_payloads) + self._assert_both_read(sectors, ovdl_payloads) diff --git a/tests/integration/test_nfc_auth.py b/tests/integration/test_nfc_auth.py new file mode 100644 index 0000000..96dfef6 --- /dev/null +++ b/tests/integration/test_nfc_auth.py @@ -0,0 +1,18 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Exercise VDDK-compatible NFC authentication against the lab vCenter.""" + +from tests.integration.base import TestBase + + +class NfcAuthTest(TestBase): + def test_authd_handshake_completes(self) -> None: + """Complete VIM login and authd PROXY through ``200 Connect``.""" + with self.authenticate() as session: + ticket = session.ticket + self.assertTrue(ticket.host) + self.assertTrue(ticket.port) + self.assertTrue(ticket.sessionId) + self.assertTrue(session.authd_sock.version()) + self.assertTrue(session.authd_sock.cipher()) diff --git a/tests/integration/test_nfc_open.py b/tests/integration/test_nfc_open.py new file mode 100644 index 0000000..255e7d2 --- /dev/null +++ b/tests/integration/test_nfc_open.py @@ -0,0 +1,23 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Exercise NFC disk open and a one-sector write/read against the lab.""" + +from openvixdisklib import nfc_open +from tests.integration.base import TestBase + + +class NfcOpenTest(TestBase): + def test_open_disk_and_read_first_sector(self) -> None: + """Open the temp VMDK, write sector 0, and read it back.""" + expected = self.pattern_bytes(self.SECTOR_SIZE, b"NFC-OPEN-S0") + with self.authenticate(read_only=False) as session: + with nfc_open.open_disk( + session, self.DISK_PATH, read_only=False) as disk: + self.assertEqual(disk.path, self.DISK_PATH) + self.assertGreater(disk.handle, 0) + self.assertEqual(disk.sector_size, self.SECTOR_SIZE) + disk.write(0, 1, expected) + got = disk.read(0, 1) + self.assertIsNot(got, expected) + self.assertEqual(got, expected) diff --git a/tests/integration/test_nfc_read_write.py b/tests/integration/test_nfc_read_write.py new file mode 100644 index 0000000..427eaa1 --- /dev/null +++ b/tests/integration/test_nfc_read_write.py @@ -0,0 +1,54 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Exercise NFC sector writes and reads against the lab vCenter.""" + +from openvixdisklib import nfc_open +from tests.integration.base import TestBase + + +class NfcReadWriteTest(TestBase): + def test_sector_writes_and_reads(self) -> None: + """Write known patterns and read them back at several ranges.""" + ranges = [ + (0, 1), + (0, 2), + (1, 1), + (8, 8), + (0, 128), + (0, 129), + (256, 64), + ] + with self.authenticate(read_only=False) as session: + with nfc_open.open_disk( + session, self.DISK_PATH, read_only=False) as disk: + for start, n_sectors in ranges: + length = n_sectors * self.SECTOR_SIZE + seed = f"NFC-R{start}:{n_sectors}-".encode() + to_write = self.pattern_bytes(length, seed) + disk.write(start, n_sectors, to_write) + got = disk.read(start, n_sectors) + self.assertIsNot(got, to_write) + self.assertEqual(len(got), length) + self.assertEqual(got, to_write) + + two_seed = b"NFC-TWO-SECTOR" + two_to_write = self.pattern_bytes( + 2 * self.SECTOR_SIZE, two_seed) + disk.write(0, 2, two_to_write) + two_got = disk.read(0, 2) + self.assertIsNot(two_got, two_to_write) + self.assertEqual(two_got, two_to_write) + self.assertEqual( + disk.read(1, 1), two_to_write[self.SECTOR_SIZE:]) + + big_seed = b"NFC-129-SECTOR-WRITE" + big_to_write = self.pattern_bytes( + 129 * self.SECTOR_SIZE, big_seed) + disk.write(0, 129, big_to_write) + big_got = disk.read(0, 129) + self.assertIsNot(big_got, big_to_write) + self.assertEqual(big_got, big_to_write) + self.assertEqual( + big_got[self.SECTOR_SIZE:2 * self.SECTOR_SIZE], + big_to_write[self.SECTOR_SIZE:2 * self.SECTOR_SIZE]) diff --git a/tests/integration/test_openvixdisklib.py b/tests/integration/test_openvixdisklib.py new file mode 100644 index 0000000..b1b0f51 --- /dev/null +++ b/tests/integration/test_openvixdisklib.py @@ -0,0 +1,33 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Exercise the VDDK-compatible openvixdisklib handle against the lab.""" + +from openvixdisklib import openvixdisklib as vixdisklib +from tests.integration.base import TestBase + + +class OpenVixDiskLibTest(TestBase): + def test_write_and_read_sector_zero_and_one_gib(self) -> None: + """Write then read sector 0 and the sector at a 1 GiB offset.""" + handle = vixdisklib.VixDiskLibHandle( + vixdisklib_compatibility_version="8.0", + config_path=None) + write_buf = vixdisklib.get_buffer(self.SECTOR_SIZE) + read_buf = vixdisklib.get_buffer(self.SECTOR_SIZE) + connect_kwargs = self.vixdisklib_connect_kwargs({ + "allow_untrusted": self.ALLOW_UNTRUSTED, + }) + patterns = { + 0: self.pattern_bytes(self.SECTOR_SIZE, b"OVDL-S0"), + self.SECTOR_AT_1GB: self.pattern_bytes( + self.SECTOR_SIZE, b"OVDL-1GB"), + } + with handle.connect(**connect_kwargs) as conn: + with handle.open(conn, self.DISK_PATH, flags=0) as disk: + for start, expected in patterns.items(): + write_buf[:self.SECTOR_SIZE] = expected + handle.write(disk, start, 1, write_buf) + read_buf[:self.SECTOR_SIZE] = b"\xa5" * self.SECTOR_SIZE + handle.read(disk, start, 1, read_buf) + self.assertEqual(read_buf.raw[:self.SECTOR_SIZE], expected) diff --git a/tests/integration/test_vddk.py b/tests/integration/test_vddk.py new file mode 100644 index 0000000..466d7dc --- /dev/null +++ b/tests/integration/test_vddk.py @@ -0,0 +1,31 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Exercise native VDDK via tests.integration.vixdisklib against the lab.""" + +from tests.integration import vixdisklib +from tests.integration.base import TestBase + + +class VddkTest(TestBase): + @classmethod + def setUpClass(cls) -> None: + """Skip when the bundled VDDK shared library is not present.""" + cls.require_vddk() + super().setUpClass() + + def test_write_and_read_first_sector(self) -> None: + """Open the temp VMDK with VDDK, write sector 0, and read it back.""" + handle = vixdisklib.VixDiskLibHandle( + vixdisklib_compatibility_version="8.0", + config_path=None) + write_buf = vixdisklib.get_buffer(self.SECTOR_SIZE) + read_buf = vixdisklib.get_buffer(self.SECTOR_SIZE) + expected = self.pattern_bytes(self.SECTOR_SIZE, b"VDDK-S0") + write_buf[:self.SECTOR_SIZE] = expected + with handle.connect(**self.vixdisklib_connect_kwargs()) as conn: + with handle.open(conn, self.DISK_PATH, flags=0) as disk: + handle.write(disk, 0, 1, write_buf) + read_buf[:self.SECTOR_SIZE] = b"\xa5" * self.SECTOR_SIZE + handle.read(disk, 0, 1, read_buf) + self.assertEqual(read_buf.raw[:self.SECTOR_SIZE], expected) diff --git a/tests/integration/vix_disklib_errors.py b/tests/integration/vix_disklib_errors.py new file mode 100644 index 0000000..2f09e61 --- /dev/null +++ b/tests/integration/vix_disklib_errors.py @@ -0,0 +1,538 @@ +# Copyright 2019 Cloudbase Solutions Srl +# All Rights Reserved. +# Generated from vixDiskLib 6.7.0-8535999 from: +# $VIX_ROOT/doc/errors/errors.html + +VIX_OK = 0 +VIX_E_FAIL = 1 +VIX_E_OUT_OF_MEMORY = 2 +VIX_E_INVALID_ARG = 3 +VIX_E_FILE_NOT_FOUND = 4 +VIX_E_OBJECT_IS_BUSY = 5 +VIX_E_NOT_SUPPORTED = 6 +VIX_E_FILE_ERROR = 7 +VIX_E_DISK_FULL = 8 +VIX_E_INCORRECT_FILE_TYPE = 9 +VIX_E_CANCELLED = 10 +VIX_E_FILE_READ_ONLY = 11 +VIX_E_FILE_ALREADY_EXISTS = 12 +VIX_E_FILE_ACCESS_ERROR = 13 +VIX_E_REQUIRES_LARGE_FILES = 14 +VIX_E_FILE_ALREADY_LOCKED = 15 +VIX_E_VMDB = 16 +VIX_E_NOT_SUPPORTED_ON_REMOTE_OBJECT = 20 +VIX_E_FILE_TOO_BIG = 21 +VIX_E_FILE_NAME_INVALID = 22 +VIX_E_ALREADY_EXISTS = 23 +VIX_E_BUFFER_TOOSMALL = 24 +VIX_E_OBJECT_NOT_FOUND = 25 +VIX_E_HOST_NOT_CONNECTED = 26 +VIX_E_INVALID_UTF = 8 +VIX_E_OPERATION_ALREADY_IN_PROGRESS = 31 +VIX_E_UNFINISHED_JOB = 29 +VIX_E_NEED_KEY = 30 +VIX_E_LICENSE = 32 +VIX_E_VM_HOST_DISCONNECTED = 34 +VIX_E_AUTHENTICATION_FAIL = 35 +VIX_E_HOST_CONNECTION_LOST = 36 +VIX_E_DUPLICATE_NAME = 41 +VIX_E_ARGUMENT_TOO_BIG = 44 +VIX_E_INVALID_HANDLE = 1000 +VIX_E_NOT_SUPPORTED_ON_HANDLE_TYPE = 1001 +VIX_E_TOO_MANY_HANDLES = 1002 +VIX_E_NOT_FOUND = 2000 +VIX_E_TYPE_MISMATCH = 2001 +VIX_E_INVALID_XML = 2002 +VIX_E_TIMEOUT_WAITING_FOR_TOOLS = 3000 +VIX_E_UNRECOGNIZED_COMMAND = 3001 +VIX_E_OP_NOT_SUPPORTED_ON_GUEST = 3003 +VIX_E_PROGRAM_NOT_STARTED = 3004 +VIX_E_CANNOT_START_READ_ONLY_VM = 3005 +VIX_E_VM_NOT_RUNNING = 3006 +VIX_E_VM_IS_RUNNING = 3007 +VIX_E_CANNOT_CONNECT_TO_VM = 3008 +VIX_E_POWEROP_SCRIPTS_NOT_AVAILABLE = 3009 +VIX_E_NO_GUEST_OS_INSTALLED = 3010 +VIX_E_VM_INSUFFICIENT_HOST_MEMORY = 3011 +VIX_E_SUSPEND_ERROR = 3012 +VIX_E_VM_NOT_ENOUGH_CPUS = 3013 +VIX_E_HOST_USER_PERMISSIONS = 3014 +VIX_E_GUEST_USER_PERMISSIONS = 3015 +VIX_E_TOOLS_NOT_RUNNING = 3016 +VIX_E_GUEST_OPERATIONS_PROHIBITED = 3017 +VIX_E_ANON_GUEST_OPERATIONS_PROHIBITED = 3018 +VIX_E_ROOT_GUEST_OPERATIONS_PROHIBITED = 3019 +VIX_E_MISSING_ANON_GUEST_ACCOUNT = 3023 +VIX_E_CANNOT_AUTHENTICATE_WITH_GUEST = 3024 +VIX_E_UNRECOGNIZED_COMMAND_IN_GUEST = 3025 +VIX_E_CONSOLE_GUEST_OPERATIONS_PROHIBITED = 3026 +VIX_E_MUST_BE_CONSOLE_USER = 3027 +VIX_E_VMX_MSG_DIALOG_AND_NO_UI = 3028 +VIX_E_OPERATION_NOT_ALLOWED_FOR_LOGIN_TYPE = 3031 +VIX_E_LOGIN_TYPE_NOT_SUPPORTED = 3032 +VIX_E_EMPTY_PASSWORD_NOT_ALLOWED_IN_GUEST = 3033 +VIX_E_INTERACTIVE_SESSION_NOT_PRESENT = 3034 +VIX_E_INTERACTIVE_SESSION_USER_MISMATCH = 3035 +VIX_E_CANNOT_POWER_ON_VM = 3041 +VIX_E_NO_DISPLAY_SERVER = 3043 +VIX_E_TOO_MANY_LOGONS = 3046 +VIX_E_INVALID_AUTHENTICATION_SESSION = 3047 +VIX_E_VM_NOT_FOUND = 4000 +VIX_E_NOT_SUPPORTED_FOR_VM_VERSION = 4001 +VIX_E_CANNOT_READ_VM_CONFIG = 4002 +VIX_E_TEMPLATE_VM = 4003 +VIX_E_VM_ALREADY_LOADED = 4004 +VIX_E_VM_ALREADY_UP_TO_DATE = 4006 +VIX_E_VM_UNSUPPORTED_GUEST = 4011 +VIX_E_UNRECOGNIZED_PROPERTY = 6000 +VIX_E_INVALID_PROPERTY_VALUE = 6001 +VIX_E_READ_ONLY_PROPERTY = 6002 +VIX_E_MISSING_REQUIRED_PROPERTY = 6003 +VIX_E_INVALID_SERIALIZED_DATA = 6004 +VIX_E_PROPERTY_TYPE_MISMATCH = 6005 +VIX_E_BAD_VM_INDEX = 8000 +VIX_E_INVALID_MESSAGE_HEADER = 10000 +VIX_E_INVALID_MESSAGE_BODY = 10001 +VIX_E_SNAPSHOT_INVAL = 13000 +VIX_E_SNAPSHOT_DUMPER = 13001 +VIX_E_SNAPSHOT_DISKLIB = 13002 +VIX_E_SNAPSHOT_NOTFOUND = 13003 +VIX_E_SNAPSHOT_EXISTS = 13004 +VIX_E_SNAPSHOT_VERSION = 13005 +VIX_E_SNAPSHOT_NOPERM = 13006 +VIX_E_SNAPSHOT_CONFIG = 13007 +VIX_E_SNAPSHOT_NOCHANGE = 13008 +VIX_E_SNAPSHOT_CHECKPOINT = 13009 +VIX_E_SNAPSHOT_LOCKED = 13010 +VIX_E_SNAPSHOT_INCONSISTENT = 13011 +VIX_E_SNAPSHOT_NAMETOOLONG = 13012 +VIX_E_SNAPSHOT_VIXFILE = 13013 +VIX_E_SNAPSHOT_DISKLOCKED = 13014 +VIX_E_SNAPSHOT_DUPLICATEDDISK = 13015 +VIX_E_SNAPSHOT_INDEPENDENTDISK = 13016 +VIX_E_SNAPSHOT_NONUNIQUE_NAME = 13017 +VIX_E_SNAPSHOT_MEMORY_ON_INDEPENDENT_DISK = 13018 +VIX_E_SNAPSHOT_MAXSNAPSHOTS = 13019 +VIX_E_SNAPSHOT_MIN_FREE_SPACE = 13020 +VIX_E_SNAPSHOT_HIERARCHY_TOODEEP = 13021 +VIX_E_SNAPSHOT_NOT_REVERTABLE = 13024 +VIX_E_HOST_DISK_INVALID_VALUE = 14003 +VIX_E_HOST_DISK_SECTORSIZE = 14004 +VIX_E_HOST_FILE_ERROR_EOF = 14005 +VIX_E_HOST_NETBLKDEV_HANDSHAKE = 14006 +VIX_E_HOST_SOCKET_CREATION_ERROR = 14007 +VIX_E_HOST_SERVER_NOT_FOUND = 14008 +VIX_E_HOST_NETWORK_CONN_REFUSED = 14009 +VIX_E_HOST_TCP_SOCKET_ERROR = 14010 +VIX_E_HOST_TCP_CONN_LOST = 14011 +VIX_E_HOST_NBD_HASHFILE_VOLUME = 14012 +VIX_E_HOST_NBD_HASHFILE_INIT = 14013 +VIX_E_DISK_INVAL = 16000 +VIX_E_DISK_NOINIT = 16001 +VIX_E_DISK_NOIO = 16002 +VIX_E_DISK_PARTIALCHAIN = 16003 +VIX_E_DISK_NEEDSREPAIR = 16006 +VIX_E_DISK_OUTOFRANGE = 16007 +VIX_E_DISK_CID_MISMATCH = 16008 +VIX_E_DISK_CANTSHRINK = 16009 +VIX_E_DISK_PARTMISMATCH = 16010 +VIX_E_DISK_UNSUPPORTEDDISKVERSION = 16011 +VIX_E_DISK_OPENPARENT = 16012 +VIX_E_DISK_NOTSUPPORTED = 16013 +VIX_E_DISK_NEEDKEY = 16014 +VIX_E_DISK_NOKEYOVERRIDE = 16015 +VIX_E_DISK_NOTENCRYPTED = 16016 +VIX_E_DISK_NOKEY = 16017 +VIX_E_DISK_INVALIDPARTITIONTABLE = 16018 +VIX_E_DISK_NOTNORMAL = 16019 +VIX_E_DISK_NOTENCDESC = 16020 +VIX_E_DISK_NEEDVMFS = 16022 +VIX_E_DISK_RAWTOOBIG = 16024 +VIX_E_DISK_TOOMANYOPENFILES = 16027 +VIX_E_DISK_TOOMANYREDO = 16028 +VIX_E_DISK_RAWTOOSMALL = 16029 +VIX_E_DISK_INVALIDCHAIN = 16030 +VIX_E_DISK_KEY_NOTFOUND = 16052 +VIX_E_DISK_SUBSYSTEM_INIT_FAIL = 16053 +VIX_E_DISK_INVALID_CONNECTION = 16054 +VIX_E_DISK_ENCODING = 16061 +VIX_E_DISK_CANTREPAIR = 16062 +VIX_E_DISK_INVALIDDISK = 16063 +VIX_E_DISK_NOLICENSE = 16064 +VIX_E_DISK_NODEVICE = 16065 +VIX_E_DISK_UNSUPPORTEDDEVICE = 16066 +VIX_E_DISK_CAPACITY_MISMATCH = 16067 +VIX_E_DISK_PARENT_NOTALLOWED = 16068 +VIX_E_DISK_ATTACH_ROOTLINK = 16069 +VIX_E_CRYPTO_UNKNOWN_ALGORITHM = 17000 +VIX_E_CRYPTO_BAD_BUFFER_SIZE = 17001 +VIX_E_CRYPTO_INVALID_OPERATION = 17002 +VIX_E_CRYPTO_RANDOM_DEVICE = 17003 +VIX_E_CRYPTO_NEED_PASSWORD = 17004 +VIX_E_CRYPTO_BAD_PASSWORD = 17005 +VIX_E_CRYPTO_NOT_IN_DICTIONARY = 17006 +VIX_E_CRYPTO_NO_CRYPTO = 17007 +VIX_E_CRYPTO_ERROR = 17008 +VIX_E_CRYPTO_BAD_FORMAT = 17009 +VIX_E_CRYPTO_LOCKED = 17010 +VIX_E_CRYPTO_EMPTY = 17011 +VIX_E_CRYPTO_KEYSAFE_LOCATOR = 17012 +VIX_E_CANNOT_CONNECT_TO_HOST = 18000 +VIX_E_NOT_FOR_REMOTE_HOST = 18001 +VIX_E_INVALID_HOSTNAME_SPECIFICATION = 18002 +VIX_E_SCREEN_CAPTURE_ERROR = 19000 +VIX_E_SCREEN_CAPTURE_BAD_FORMAT = 19001 +VIX_E_SCREEN_CAPTURE_COMPRESSION_FAIL = 19002 +VIX_E_SCREEN_CAPTURE_LARGE_DATA = 19003 +VIX_E_GUEST_VOLUMES_NOT_FROZEN = 20000 +VIX_E_NOT_A_FILE = 20001 +VIX_E_NOT_A_DIRECTORY = 20002 +VIX_E_NO_SUCH_PROCESS = 20003 +VIX_E_FILE_NAME_TOO_LONG = 20004 +VIX_E_OPERATION_DISABLED = 20005 +VIX_E_TOOLS_INSTALL_NO_IMAGE = 21000 +VIX_E_TOOLS_INSTALL_IMAGE_INACCESIBLE = 21001 +VIX_E_TOOLS_INSTALL_NO_DEVICE = 21002 +VIX_E_TOOLS_INSTALL_DEVICE_NOT_CONNECTED = 21003 +VIX_E_TOOLS_INSTALL_CANCELLED = 21004 +VIX_E_TOOLS_INSTALL_INIT_FAILED = 21005 +VIX_E_TOOLS_INSTALL_AUTO_NOT_SUPPORTED = 21006 +VIX_E_TOOLS_INSTALL_GUEST_NOT_READY = 21007 +VIX_E_TOOLS_INSTALL_SIG_CHECK_FAILED = 21008 +VIX_E_TOOLS_INSTALL_ERROR = 21009 +VIX_E_TOOLS_INSTALL_ALREADY_UP_TO_DATE = 21010 +VIX_E_TOOLS_INSTALL_IN_PROGRESS = 21011 +VIX_E_TOOLS_INSTALL_IMAGE_COPY_FAILED = 21012 +VIX_E_WRAPPER_WORKSTATION_NOT_INSTALLED = 22001 +VIX_E_WRAPPER_VERSION_NOT_FOUND = 22002 +VIX_E_WRAPPER_SERVICEPROVIDER_NOT_FOUND = 22003 +VIX_E_WRAPPER_PLAYER_NOT_INSTALLED = 22004 +VIX_E_WRAPPER_RUNTIME_NOT_INSTALLED = 22005 +VIX_E_WRAPPER_MULTIPLE_SERVICEPROVIDERS = 22006 +VIX_E_MNTAPI_MOUNTPT_NOT_FOUND = 24000 +VIX_E_MNTAPI_MOUNTPT_IN_USE = 24001 +VIX_E_MNTAPI_DISK_NOT_FOUND = 24002 +VIX_E_MNTAPI_DISK_NOT_MOUNTED = 24003 +VIX_E_MNTAPI_DISK_IS_MOUNTED = 24004 +VIX_E_MNTAPI_DISK_NOT_SAFE = 24005 +VIX_E_MNTAPI_DISK_CANT_OPEN = 24006 +VIX_E_MNTAPI_CANT_READ_PARTS = 24007 +VIX_E_MNTAPI_UMOUNT_APP_NOT_FOUND = 24008 +VIX_E_MNTAPI_UMOUNT = 24009 +VIX_E_MNTAPI_NO_MOUNTABLE_PARTITONS = 24010 +VIX_E_MNTAPI_PARTITION_RANGE = 24011 +VIX_E_MNTAPI_PERM = 24012 +VIX_E_MNTAPI_DICT = 24013 +VIX_E_MNTAPI_DICT_LOCKED = 24014 +VIX_E_MNTAPI_OPEN_HANDLES = 24015 +VIX_E_MNTAPI_CANT_MAKE_VAR_DIR = 24016 +VIX_E_MNTAPI_NO_ROOT = 24017 +VIX_E_MNTAPI_LOOP_FAILED = 24018 +VIX_E_MNTAPI_DAEMON = 24019 +VIX_E_MNTAPI_INTERNAL = 24020 +VIX_E_MNTAPI_SYSTEM = 24021 +VIX_E_MNTAPI_NO_CONNECTION_DETAILS = 24022 +VIX_E_MNTAPI_INCOMPATIBLE_VERSION = 24300 +VIX_E_MNTAPI_OS_ERROR = 24301 +VIX_E_MNTAPI_DRIVE_LETTER_IN_USE = 24302 +VIX_E_MNTAPI_DRIVE_LETTER_ALREADY_ASSIGNED = 24303 +VIX_E_MNTAPI_VOLUME_NOT_MOUNTED = 24304 +VIX_E_MNTAPI_VOLUME_ALREADY_MOUNTED = 24305 +VIX_E_MNTAPI_FORMAT_FAILURE = 24306 +VIX_E_MNTAPI_NO_DRIVER = 24307 +VIX_E_MNTAPI_ALREADY_OPENED = 24308 +VIX_E_MNTAPI_ITEM_NOT_FOUND = 24309 +VIX_E_MNTAPI_UNSUPPROTED_BOOT_LOADER = 24310 +VIX_E_MNTAPI_UNSUPPROTED_OS = 24311 +VIX_E_MNTAPI_CODECONVERSION = 24312 +VIX_E_MNTAPI_REGWRITE_ERROR = 24313 +VIX_E_MNTAPI_UNSUPPORTED_FT_VOLUME = 24314 +VIX_E_MNTAPI_PARTITION_NOT_FOUND = 24315 +VIX_E_MNTAPI_PUTFILE_ERROR = 24316 +VIX_E_MNTAPI_GETFILE_ERROR = 24317 +VIX_E_MNTAPI_REG_NOT_OPENED = 24318 +VIX_E_MNTAPI_REGDELKEY_ERROR = 24319 +VIX_E_MNTAPI_CREATE_PARTITIONTABLE_ERROR = 24320 +VIX_E_MNTAPI_OPEN_FAILURE = 24321 +VIX_E_MNTAPI_VOLUME_NOT_WRITABLE = 24322 +VIX_E_ASYNC_MIXEDMODE_UNSUPPORTED = 26000 +VIX_E_NET_HTTP_UNSUPPORTED_PROTOCOL = 30001 +VIX_E_NET_HTTP_URL_MALFORMAT = 30003 +VIX_E_NET_HTTP_COULDNT_RESOLVE_PROXY = 30005 +VIX_E_NET_HTTP_COULDNT_RESOLVE_HOST = 30006 +VIX_E_NET_HTTP_COULDNT_CONNECT = 30007 +VIX_E_NET_HTTP_HTTP_RETURNED_ERROR = 30022 +VIX_E_NET_HTTP_OPERATION_TIMEDOUT = 30028 +VIX_E_NET_HTTP_SSL_CONNECT_ERROR = 30035 +VIX_E_NET_HTTP_TOO_MANY_REDIRECTS = 30047 +VIX_E_NET_HTTP_TRANSFER = 30200 +VIX_E_NET_HTTP_SSL_SECURITY = 30201 +VIX_E_NET_HTTP_GENERIC = 30202 + +VIX_ERROR_CODE_MAP = { + VIX_OK: "The operation was successful.", + VIX_E_FAIL: "Unknown error.", + VIX_E_OUT_OF_MEMORY: "Memory allocation failed. Out of memory.", + VIX_E_INVALID_ARG: "One of the parameters was invalid.", + VIX_E_FILE_NOT_FOUND: "A file was not found.", + VIX_E_OBJECT_IS_BUSY: "This function cannot be performed because the handle is executing another function.", + VIX_E_NOT_SUPPORTED: "The operation is not supported.", + VIX_E_FILE_ERROR: "A file access error occurred on the host or guest operating system.", + VIX_E_DISK_FULL: "An error occurred while writing a file; the disk is full. Data has not been saved. Free some disk space and try again.", + VIX_E_INCORRECT_FILE_TYPE: "An error occurred while accessing a file: wrong file type.", + VIX_E_CANCELLED: "The operation was canceled.", + VIX_E_FILE_READ_ONLY: "The file is write-protected.", + VIX_E_FILE_ALREADY_EXISTS: "The file already exists.", + VIX_E_FILE_ACCESS_ERROR: "You do not have access rights to this file.", + VIX_E_REQUIRES_LARGE_FILES: "The file system does not support large files.", + VIX_E_FILE_ALREADY_LOCKED: "The file is already in use.", + VIX_E_VMDB: "The system returned an error. Communication with the virtual machine might have been interrupted.", + VIX_E_NOT_SUPPORTED_ON_REMOTE_OBJECT: "The command is not supported on remote objects.", + VIX_E_FILE_TOO_BIG: "The file is too large for the file system.", + VIX_E_FILE_NAME_INVALID: "The file name is not valid.", + VIX_E_ALREADY_EXISTS: "Already exists.", + VIX_E_BUFFER_TOOSMALL: "Buffer is too small.", + VIX_E_OBJECT_NOT_FOUND: "The request refers to an object that does not exist.", + VIX_E_HOST_NOT_CONNECTED: "Unable to connect to the host.", + VIX_E_INVALID_UTF: "The string parameter has incorrect encoding.", + VIX_E_OPERATION_ALREADY_IN_PROGRESS: "The operation is already in progress.", + VIX_E_UNFINISHED_JOB: "The job has not finished.", + VIX_E_NEED_KEY: "A decryption key is required to perform the operation.", + VIX_E_LICENSE: "This operation is not supported with the current license.", + VIX_E_VM_HOST_DISCONNECTED: "Unable to communicate with the virtual machine's host because it is disconnected.", + VIX_E_AUTHENTICATION_FAIL: "Authentication for encrypted virtual machine failed.", + VIX_E_HOST_CONNECTION_LOST: "The connection to the host was lost.", + VIX_E_DUPLICATE_NAME: "Another object is using this name.", + VIX_E_ARGUMENT_TOO_BIG: "One of the specified arguments is too large.", + VIX_E_INVALID_HANDLE: "The handle is not a valid VIX object.", + VIX_E_NOT_SUPPORTED_ON_HANDLE_TYPE: "The operation is not supported on this type of handle.", + VIX_E_TOO_MANY_HANDLES: "Too many handles are open.", + VIX_E_NOT_FOUND: "Invalid file. A required section of the file is missing.", + VIX_E_TYPE_MISMATCH: "Invalid file. An object has the wrong type.", + VIX_E_INVALID_XML: "Invalid file. The contents might be corrupt.", + VIX_E_TIMEOUT_WAITING_FOR_TOOLS: "A timeout error occurred while waiting for .", + VIX_E_UNRECOGNIZED_COMMAND: "The command is not recognized by the virtual machine.", + VIX_E_OP_NOT_SUPPORTED_ON_GUEST: "The requested operation is not supported on this guest operating system.", + VIX_E_PROGRAM_NOT_STARTED: "A program could not run on the guest operating system.", + VIX_E_CANNOT_START_READ_ONLY_VM: "Cannot power on a read-only virtual machine.", + VIX_E_VM_NOT_RUNNING: "The virtual machine needs to be powered on.", + VIX_E_VM_IS_RUNNING: "The virtual machine should not be powered on. It is already running.", + VIX_E_CANNOT_CONNECT_TO_VM: "Cannot connect to the virtual machine.", + VIX_E_POWEROP_SCRIPTS_NOT_AVAILABLE: "Cannot execute scripts.", + VIX_E_NO_GUEST_OS_INSTALLED: "There is no operating system installed in the virtual machine.", + VIX_E_VM_INSUFFICIENT_HOST_MEMORY: "Not enough physical memory is available to power on this virtual machine.", + VIX_E_SUSPEND_ERROR: "An error occurred while suspending the virtual machine.", + VIX_E_VM_NOT_ENOUGH_CPUS: "This virtual machine is configured to run with 2 CPUs, but the host has only 1 CPU. The virtual machine cannot be powered on.", + VIX_E_HOST_USER_PERMISSIONS: "Insufficient permissions in the host operating system.", + VIX_E_GUEST_USER_PERMISSIONS: "Authentication failure or insufficient permissions in guest operating system.", + VIX_E_TOOLS_NOT_RUNNING: " are not running in the guest.", + VIX_E_GUEST_OPERATIONS_PROHIBITED: "Guest operations are not allowed on this virtual machine.", + VIX_E_ANON_GUEST_OPERATIONS_PROHIBITED: "Anonymous guest operations are not allowed on this virtual machine. You must call VixVM_LoginInGuest before performing guest operations.", + VIX_E_ROOT_GUEST_OPERATIONS_PROHIBITED: "Guest operations are not allowed for the administrative user on this virtual machine.", + VIX_E_MISSING_ANON_GUEST_ACCOUNT: "The virtual machine configuration must specify the guest account name to be used for anonymous guest operations.", + VIX_E_CANNOT_AUTHENTICATE_WITH_GUEST: "The virtual machine cannot authenticate users with guest.", + VIX_E_UNRECOGNIZED_COMMAND_IN_GUEST: "The command is not recognized by .", + VIX_E_CONSOLE_GUEST_OPERATIONS_PROHIBITED: "Guest operations are not allowed for console users on this virtual machine.", + VIX_E_MUST_BE_CONSOLE_USER: "Only the console user can run the command.", + VIX_E_VMX_MSG_DIALOG_AND_NO_UI: "The virtual machine is blocked waiting for a user operation.", + VIX_E_OPERATION_NOT_ALLOWED_FOR_LOGIN_TYPE: "The command is not allowed by this login type.", + VIX_E_LOGIN_TYPE_NOT_SUPPORTED: "This login type is not supported.", + VIX_E_EMPTY_PASSWORD_NOT_ALLOWED_IN_GUEST: "The guest OS does not support empty passwords.", + VIX_E_INTERACTIVE_SESSION_NOT_PRESENT: "The specified guest user must be logged in interactively to perform this operation.", + VIX_E_INTERACTIVE_SESSION_USER_MISMATCH: "The specified guest user does not match the user currently logged in interactively.", + VIX_E_CANNOT_POWER_ON_VM: "The virtual machine could not start.", + VIX_E_NO_DISPLAY_SERVER: "Cannot launch the UI because no display server is present in the current environment.", + VIX_E_TOO_MANY_LOGONS: "The supported number of active authentication sessions has been exceeded.", + VIX_E_INVALID_AUTHENTICATION_SESSION: "The authenticaton session provided does not exist.", + VIX_E_VM_NOT_FOUND: "The virtual machine cannot be found.", + VIX_E_NOT_SUPPORTED_FOR_VM_VERSION: "The operation is not supported for this virtual machine version.", + VIX_E_CANNOT_READ_VM_CONFIG: "Cannot read the virtual machine configuration file.", + VIX_E_TEMPLATE_VM: "Cannot perform this operation on a template virtual machine.", + VIX_E_VM_ALREADY_LOADED: "The virtual machine has already been loaded.", + VIX_E_VM_ALREADY_UP_TO_DATE: "The virtual machine is already up-to-date.", + VIX_E_VM_UNSUPPORTED_GUEST: "The specified guest operating system is not supported on the host that is the target of the operation.", + VIX_E_UNRECOGNIZED_PROPERTY: "Unrecognized handle property identifier.", + VIX_E_INVALID_PROPERTY_VALUE: "Invalid property value.", + VIX_E_READ_ONLY_PROPERTY: "Cannot change a read-only property.", + VIX_E_MISSING_REQUIRED_PROPERTY: "This handle is missing a required property.", + VIX_E_INVALID_SERIALIZED_DATA: "A serialized object is invalid and cannot be deserialized.", + VIX_E_PROPERTY_TYPE_MISMATCH: "The data provided does not match the property type.", + VIX_E_BAD_VM_INDEX: "The index parameter does not correspond to a result set.", + VIX_E_INVALID_MESSAGE_HEADER: "A message header was corrupted or has the incorrect version.", + VIX_E_INVALID_MESSAGE_BODY: "A message body was corrupted or is missing.", + VIX_E_SNAPSHOT_INVAL: "A snapshot-related error has occurred.", + VIX_E_SNAPSHOT_DUMPER: "Unable to open the snapshot file.", + VIX_E_SNAPSHOT_DISKLIB: "Disk error.", + VIX_E_SNAPSHOT_NOTFOUND: "The snapshot does not exist.", + VIX_E_SNAPSHOT_EXISTS: "The snapshot already exists.", + VIX_E_SNAPSHOT_VERSION: "Snapshots are not allowed on this virtual machine.", + VIX_E_SNAPSHOT_NOPERM: "Insufficient permissions.", + VIX_E_SNAPSHOT_CONFIG: "There is an error in the configuration file.", + VIX_E_SNAPSHOT_NOCHANGE: "The state of the virtual machine has not changed since the last snapshot operation.", + VIX_E_SNAPSHOT_CHECKPOINT: "Unable to save the snapshot file.", + VIX_E_SNAPSHOT_LOCKED: "A snapshot operation is already in progress.", + VIX_E_SNAPSHOT_INCONSISTENT: "The snapshot files are in an inconsistent state.", + VIX_E_SNAPSHOT_NAMETOOLONG: "The filename is too long.", + VIX_E_SNAPSHOT_VIXFILE: "Cannot snapshot all metadata files.", + VIX_E_SNAPSHOT_DISKLOCKED: "One or more of the disks are busy.", + VIX_E_SNAPSHOT_DUPLICATEDDISK: "The virtual disk is used multiple times.", + VIX_E_SNAPSHOT_INDEPENDENTDISK: "Cannot take snapshots of powered on virtual machines with independent disks.", + VIX_E_SNAPSHOT_NONUNIQUE_NAME: "The name does not uniquely identify one snapshot.", + VIX_E_SNAPSHOT_MEMORY_ON_INDEPENDENT_DISK: "Failed to take a memory snapshot because the virtual machine is configured with independent disks.", + VIX_E_SNAPSHOT_MAXSNAPSHOTS: "Exceeded the maximum number of permitted snapshots.", + VIX_E_SNAPSHOT_MIN_FREE_SPACE: "Available free space is less than the configured minimum free space.", + VIX_E_SNAPSHOT_HIERARCHY_TOODEEP: "Snapshot hierarchy is too deep.", + VIX_E_SNAPSHOT_NOT_REVERTABLE: "Cannot revert. The snapshot is .", + VIX_E_HOST_DISK_INVALID_VALUE: "The specified device is not a valid physical disk device.", + VIX_E_HOST_DISK_SECTORSIZE: "The disk sector size check failed.", + VIX_E_HOST_FILE_ERROR_EOF: "Read beyond the end of file.", + VIX_E_HOST_NETBLKDEV_HANDSHAKE: "Error in protocol.", + VIX_E_HOST_SOCKET_CREATION_ERROR: "Unable to create a socket.", + VIX_E_HOST_SERVER_NOT_FOUND: "The specified server could not be contacted.", + VIX_E_HOST_NETWORK_CONN_REFUSED: "The server refused connection.", + VIX_E_HOST_TCP_SOCKET_ERROR: "There was an error in communication.", + VIX_E_HOST_TCP_CONN_LOST: "The connection was lost.", + VIX_E_HOST_NBD_HASHFILE_VOLUME: "NBD_ERR_HASHFILE_VOLUME.", + VIX_E_HOST_NBD_HASHFILE_INIT: "NBD_ERR_HASHFILE_INIT.", + VIX_E_DISK_INVAL: "One of the parameters supplied is invalid.", + VIX_E_DISK_NOINIT: "The disk library has not been initialized.", + VIX_E_DISK_NOIO: "The called function requires the virtual disk to be opened for I/O.", + VIX_E_DISK_PARTIALCHAIN: "The called function cannot be performed on partial chains. Open the parent virtual disk.", + VIX_E_DISK_NEEDSREPAIR: "The specified virtual disk needs repair.", + VIX_E_DISK_OUTOFRANGE: "You have requested access to an area of the virtual disk that is out of bounds.", + VIX_E_DISK_CID_MISMATCH: "The parent virtual disk has been modified since the child was created. Parent virutal disk's content ID does not match with the parent content ID in the child.", + VIX_E_DISK_CANTSHRINK: "The specified virtual disk cannot be shrunk because it is not the parent disk.", + VIX_E_DISK_PARTMISMATCH: "The partition table on the physical disk has changed since the disk was created. Remove the physical disk from the virtual machine, then add it again.", + VIX_E_DISK_UNSUPPORTEDDISKVERSION: "The version of the virtual disk is newer than the version supported by this program.", + VIX_E_DISK_OPENPARENT: "The parent of this virtual disk could not be opened.", + VIX_E_DISK_NOTSUPPORTED: "The specified feature is not supported by this version.", + VIX_E_DISK_NEEDKEY: "One or more required keys were not provided.", + VIX_E_DISK_NOKEYOVERRIDE: "Will not create an unencrypted child of an encrypted disk without explicit request.", + VIX_E_DISK_NOTENCRYPTED: "Not an encrypted disk.", + VIX_E_DISK_NOKEY: "No keys were supplied for encrypting the disk.", + VIX_E_DISK_INVALIDPARTITIONTABLE: "The partition table is invalid.", + VIX_E_DISK_NOTNORMAL: "Only sparse extents with embedded descriptors can be encrypted.", + VIX_E_DISK_NOTENCDESC: "Not an encrypted descriptor file.", + VIX_E_DISK_NEEDVMFS: "The file system is not VMFS.", + VIX_E_DISK_RAWTOOBIG: "The physical disk is too big.", + VIX_E_DISK_TOOMANYOPENFILES: "The host's limit for open files has been exceeded.", + VIX_E_DISK_TOOMANYREDO: "Too many levels of redo logs.", + VIX_E_DISK_RAWTOOSMALL: "The physical disk is too small.", + VIX_E_DISK_INVALIDCHAIN: "Invalid disk chain: cannot mix hosted and managed style disks in the same chain.", + VIX_E_DISK_KEY_NOTFOUND: "The specified key is not found in the disk database.", + VIX_E_DISK_SUBSYSTEM_INIT_FAIL: "One or more required subsystems failed to initialize.", + VIX_E_DISK_INVALID_CONNECTION: "Invalid connection handle.", + VIX_E_DISK_ENCODING: "Disk encoding error.", + VIX_E_DISK_CANTREPAIR: "The disk is corrupted and unrepairable.", + VIX_E_DISK_INVALIDDISK: "The specified file is not a virtual disk.", + VIX_E_DISK_NOLICENSE: "The host is not licensed for this feature.", + VIX_E_DISK_NODEVICE: "The device does not exist.", + VIX_E_DISK_UNSUPPORTEDDEVICE: "The operation is not supported on this type of device.", + VIX_E_DISK_CAPACITY_MISMATCH: "The parent virtual disk's capacity is not the same as child's capacity.", + VIX_E_DISK_PARENT_NOTALLOWED: "Disk type cannot be allowed as parent.", + VIX_E_DISK_ATTACH_ROOTLINK: "Both parent and child virtual disks are root links.", + VIX_E_CRYPTO_UNKNOWN_ALGORITHM: "Security library error.", + VIX_E_CRYPTO_BAD_BUFFER_SIZE: "Security library error.", + VIX_E_CRYPTO_INVALID_OPERATION: "Security library error.", + VIX_E_CRYPTO_RANDOM_DEVICE: "Security library error.", + VIX_E_CRYPTO_NEED_PASSWORD: "A password is required for this operation.", + VIX_E_CRYPTO_BAD_PASSWORD: "Incorrect password.", + VIX_E_CRYPTO_NOT_IN_DICTIONARY: "Security library error.", + VIX_E_CRYPTO_NO_CRYPTO: "Security library error.", + VIX_E_CRYPTO_ERROR: "Security library error.", + VIX_E_CRYPTO_BAD_FORMAT: "Security library error.", + VIX_E_CRYPTO_LOCKED: "Security library error.", + VIX_E_CRYPTO_EMPTY: "Security library error.", + VIX_E_CRYPTO_KEYSAFE_LOCATOR: "Security library error.", + VIX_E_CANNOT_CONNECT_TO_HOST: "Cannot connect to the host.", + VIX_E_NOT_FOR_REMOTE_HOST: "Only a local host can support this feature.", + VIX_E_INVALID_HOSTNAME_SPECIFICATION: "Malformed hostname parameter. For the given service provider, the hostname must be a URL in the form https://:/sdk.", + VIX_E_SCREEN_CAPTURE_ERROR: "Could not capture screen.", + VIX_E_SCREEN_CAPTURE_BAD_FORMAT: "Requested unsupported format.", + VIX_E_SCREEN_CAPTURE_COMPRESSION_FAIL: "Could not compress the screen capture.", + VIX_E_SCREEN_CAPTURE_LARGE_DATA: "The screen capture data is larger than the maximum size.", + VIX_E_GUEST_VOLUMES_NOT_FROZEN: "The drives are not frozen.", + VIX_E_NOT_A_FILE: "The object is not a file.", + VIX_E_NOT_A_DIRECTORY: "The object is not a directory.", + VIX_E_NO_SUCH_PROCESS: "No such process.", + VIX_E_FILE_NAME_TOO_LONG: "File name too long.", + VIX_E_OPERATION_DISABLED: "The operation has been disabled by the guest operating system.", + VIX_E_TOOLS_INSTALL_NO_IMAGE: "No .", + VIX_E_TOOLS_INSTALL_IMAGE_INACCESIBLE: "The .", + VIX_E_TOOLS_INSTALL_NO_DEVICE: "The guest operating system does not have a device configured for the .", + VIX_E_TOOLS_INSTALL_DEVICE_NOT_CONNECTED: "The guest operating system device used for installation of .", + VIX_E_TOOLS_INSTALL_CANCELLED: "The .", + VIX_E_TOOLS_INSTALL_INIT_FAILED: "The .", + VIX_E_TOOLS_INSTALL_AUTO_NOT_SUPPORTED: "The .", + VIX_E_TOOLS_INSTALL_GUEST_NOT_READY: " are not running in the guest OS. Automatic upgrade is not possible.", + VIX_E_TOOLS_INSTALL_SIG_CHECK_FAILED: "The .", + VIX_E_TOOLS_INSTALL_ERROR: "The .", + VIX_E_TOOLS_INSTALL_ALREADY_UP_TO_DATE: " are already up to date.", + VIX_E_TOOLS_INSTALL_IN_PROGRESS: "A .", + VIX_E_TOOLS_INSTALL_IMAGE_COPY_FAILED: "Could not copy .", + VIX_E_WRAPPER_WORKSTATION_NOT_INSTALLED: "Service type VIX_SERVICEPROVIDER_VMWARE_WORKSTATION was specified but not installed.", + VIX_E_WRAPPER_VERSION_NOT_FOUND: "The specified version was not found.", + VIX_E_WRAPPER_SERVICEPROVIDER_NOT_FOUND: "The specified service provider was not found.", + VIX_E_WRAPPER_PLAYER_NOT_INSTALLED: "Service type VIX_SERVICEPROVIDER_VMWARE_PLAYER was specified but not installed.", + VIX_E_WRAPPER_RUNTIME_NOT_INSTALLED: "Cannot find support libraries; VIX appears to have not been installed.", + VIX_E_WRAPPER_MULTIPLE_SERVICEPROVIDERS: "Cannot connect with multiple service providers.", + VIX_E_MNTAPI_MOUNTPT_NOT_FOUND: "Could not find the specified mountpoint.", + VIX_E_MNTAPI_MOUNTPT_IN_USE: "The mountpoint is already in use.", + VIX_E_MNTAPI_DISK_NOT_FOUND: "Could not find the specified virtual disk.", + VIX_E_MNTAPI_DISK_NOT_MOUNTED: "The specified disk is not mounted.", + VIX_E_MNTAPI_DISK_IS_MOUNTED: "The specified disk is already mounted.", + VIX_E_MNTAPI_DISK_NOT_SAFE: "It is not safe to mount the virtual disk. It might be attached to a suspended or powered-on virtual machine, or it may be inside a snapshot chain.", + VIX_E_MNTAPI_DISK_CANT_OPEN: "Cannot open the virtual disk.", + VIX_E_MNTAPI_CANT_READ_PARTS: "Cannot read or parse the partition table on the virtual disk.", + VIX_E_MNTAPI_UMOUNT_APP_NOT_FOUND: "Could not find the umount application in a standard system directory such as /bin, /usr/bin, or /sbin.", + VIX_E_MNTAPI_UMOUNT: "The umount command failed.", + VIX_E_MNTAPI_NO_MOUNTABLE_PARTITONS: "The virtual disk does not have any partitions that the host system knows how to mount.", + VIX_E_MNTAPI_PARTITION_RANGE: "An invalid partition number was specified.", + VIX_E_MNTAPI_PERM: "Insufficient permissions to perform this operation.", + VIX_E_MNTAPI_DICT: "Error accessing metadata. You might not have sufficient permission to access this disk or the metadata may be corrupted.", + VIX_E_MNTAPI_DICT_LOCKED: "The metadata for this disk is locked. Check for other running virtual disk mounter applications.", + VIX_E_MNTAPI_OPEN_HANDLES: "Another process is performing an operation on this mounted virtual disk.", + VIX_E_MNTAPI_CANT_MAKE_VAR_DIR: "Cannot create directory '/var/run/vmware/fuse'.", + VIX_E_MNTAPI_NO_ROOT: "This application must be run setuid root.", + VIX_E_MNTAPI_LOOP_FAILED: "A loop device operation failed.", + VIX_E_MNTAPI_DAEMON: "The VMware fuse daemon failed to start.", + VIX_E_MNTAPI_INTERNAL: "An internal error has occurred. Contact VMware support.", + VIX_E_MNTAPI_SYSTEM: "A system call has failed.", + VIX_E_MNTAPI_NO_CONNECTION_DETAILS: "Unable to get vixDiskLib connection details.", + VIX_E_MNTAPI_INCOMPATIBLE_VERSION: "The product version number is lower than the expected version number.", + VIX_E_MNTAPI_OS_ERROR: "There was an operating system error.", + VIX_E_MNTAPI_DRIVE_LETTER_IN_USE: "The specified drive letter is already in use.", + VIX_E_MNTAPI_DRIVE_LETTER_ALREADY_ASSIGNED: "The specified drive letter is already assigned.", + VIX_E_MNTAPI_VOLUME_NOT_MOUNTED: "The specified volume is not mounted.", + VIX_E_MNTAPI_VOLUME_ALREADY_MOUNTED: "The specified volume is already mounted.", + VIX_E_MNTAPI_FORMAT_FAILURE: "Unable to format volume.", + VIX_E_MNTAPI_NO_DRIVER: "Driver not found.", + VIX_E_MNTAPI_ALREADY_OPENED: "A handle to the Volume or DiskSet is already open.", + VIX_E_MNTAPI_ITEM_NOT_FOUND: "Invalid file. A required section of the file is missing.", + VIX_E_MNTAPI_UNSUPPROTED_BOOT_LOADER: "Boot loader not supported.", + VIX_E_MNTAPI_UNSUPPROTED_OS: "The current operating system is not supported.", + VIX_E_MNTAPI_CODECONVERSION: "An error occurred while converting the string.", + VIX_E_MNTAPI_REGWRITE_ERROR: "There was an error writing to the registry.", + VIX_E_MNTAPI_UNSUPPORTED_FT_VOLUME: "Windows NT4 Fault Tolerant volume type is not supported.", + VIX_E_MNTAPI_PARTITION_NOT_FOUND: "The specified partition was not found.", + VIX_E_MNTAPI_PUTFILE_ERROR: "Putfile error.", + VIX_E_MNTAPI_GETFILE_ERROR: "Getfile error.", + VIX_E_MNTAPI_REG_NOT_OPENED: "Unable to open registry key.", + VIX_E_MNTAPI_REGDELKEY_ERROR: "There was an error deleting the registry key.", + VIX_E_MNTAPI_CREATE_PARTITIONTABLE_ERROR: "An error occurred while creating the partition table.", + VIX_E_MNTAPI_OPEN_FAILURE: "Failed to open DiskSet.", + VIX_E_MNTAPI_VOLUME_NOT_WRITABLE: "The volume is write-protected.", + VIX_E_ASYNC_MIXEDMODE_UNSUPPORTED: "Synchronous and asynchronous I/O on the same disk handle is not allowed.", + VIX_E_NET_HTTP_UNSUPPORTED_PROTOCOL: "The URL provided uses an unsupported protocol.", + VIX_E_NET_HTTP_URL_MALFORMAT: "The URL was not properly formatted.", + VIX_E_NET_HTTP_COULDNT_RESOLVE_PROXY: "Failed to resolve proxy.", + VIX_E_NET_HTTP_COULDNT_RESOLVE_HOST: "Failed to resolve host.", + VIX_E_NET_HTTP_COULDNT_CONNECT: "Failed to connect to host or proxy.", + VIX_E_NET_HTTP_HTTP_RETURNED_ERROR: "Server returned HTTP error code >= 400.", + VIX_E_NET_HTTP_OPERATION_TIMEDOUT: "Network operation timed out.", + VIX_E_NET_HTTP_SSL_CONNECT_ERROR: "A problem occurred during the SSL/TLS handshake.", + VIX_E_NET_HTTP_TOO_MANY_REDIRECTS: "Reached the maximum number of redirects.", + VIX_E_NET_HTTP_TRANSFER: "Failure sending/receiving network data.", + VIX_E_NET_HTTP_SSL_SECURITY: "An SSL error occurred.", + VIX_E_NET_HTTP_GENERIC: "A generic HTTP error occurred." +} diff --git a/tests/integration/vixdisklib.py b/tests/integration/vixdisklib.py new file mode 100755 index 0000000..4ae71a9 --- /dev/null +++ b/tests/integration/vixdisklib.py @@ -0,0 +1,324 @@ +# Copyright 2016 Cloudbase Solutions Srl +# All Rights Reserved. + +"""Python bindings for the VDDK vixdisklib library. Superseded by the +openvixdisklib, which avoids the proprietary VDDK SDK. + +This module is used by the integration tests in order to cross-check the +openvixdisklib library. ctypes layouts follow `.vddk/vixDiskLib.h`. +""" + +import contextlib +import ctypes +import logging +import os +import traceback + +from tests.integration import vix_disklib_errors + +LOG = logging.getLogger(__name__) + +VIXDISKLIB_VERSION_MAJOR = 8 +VIXDISKLIB_VERSION_MINOR = 0 + +VIXDISKLIB_SECTOR_SIZE = 512 + +VIXDISKLIB_CRED_UID = 1 + +VIXDISKLIB_FLAG_OPEN_UNBUFFERED = 1 +VIXDISKLIB_FLAG_OPEN_SINGLE_LINK = 2 +VIXDISKLIB_FLAG_OPEN_READ_ONLY = 4 + +# NBD compression flags +VIXDISKLIB_FLAG_OPEN_COMPRESSION_ZLIB = 16 +VIXDISKLIB_FLAG_OPEN_COMPRESSION_FASTLZ = 32 +VIXDISKLIB_FLAG_OPEN_COMPRESSION_SKIPZ = 64 + +VIX_SUPPORTED_COMPATIBILITY_MODES = [ + "6.0", "6.5", "6.7", "7.0", "8.0"] + + +class VixDiskLibUidPasswdCreds(ctypes.Structure): + _fields_ = [ + ("userName", ctypes.c_char_p), + ("password", ctypes.c_char_p), + ] + + +class VixDiskLibSessionIdCreds(ctypes.Structure): + _fields_ = [ + ("cookie", ctypes.c_char_p), + ("userName", ctypes.c_char_p), + ("key", ctypes.c_char_p), + ] + + +class VixDiskLibCreds(ctypes.Union): + _fields_ = [ + ("uid", VixDiskLibUidPasswdCreds), + ("sessionId", VixDiskLibSessionIdCreds), + ] + + +class VixDiskLibConnectParams(ctypes.Structure): + _fields_ = [ + ("vmxSpec", ctypes.c_char_p), + ("serverName", ctypes.c_char_p), + ("thumbPrint", ctypes.c_char_p), + # Note: this is 32bit on Windows + ("privateUse", ctypes.c_longlong), + ("credType", ctypes.c_uint32), + ("creds", VixDiskLibCreds), + ("port", ctypes.c_uint32), + ("nfcHostPort", ctypes.c_uint32), + ("vimApiVer", ctypes.c_char_p), + ] + + +class VixDiskLibConnection(ctypes.Structure): + _fields_ = [] + + +def get_buffer(size): + return ctypes.create_string_buffer(size) + + +class VixDiskLibHandle(object): + """ Class which acts as a proxy for vixDiskLib-related operations: + """ + def __init__( + self, config_path=None, vixdisklib_compatibility_version=None): + self._vix_disklib = ctypes.cdll.LoadLibrary( + self.get_vix_disklib_name()) + self._setup_vix_disklib() + + if config_path: + config_path = config_path.encode() + + target_versions = VIX_SUPPORTED_COMPATIBILITY_MODES + if vixdisklib_compatibility_version: + target_versions = [vixdisklib_compatibility_version] + LOG.debug("vixDiskLib versions targeted: %s", target_versions) + + # NOTE: iterate through all versions and try to initialize using each: + version_used = None + for version in reversed(target_versions): + major_ver = None + minor_ver = None + try: + major_ver, minor_ver = version.split(".") + major_ver = int(major_ver) + minor_ver = int(minor_ver) + except ValueError as ex: + raise ValueError( + "Unsupported vixDiskLib version format '%s'. vixDiskLib " + "compatibility mode must be of the form " + "'$major.$minor'" % version) from ex + + try: + self._check_err(self._vix_disklib.VixDiskLib_InitEx( + major_ver, minor_ver, None, None, None, None, config_path)) + version_used = version + break + except Exception: + LOG.debug( + "Failed to initialize vixDiskLib using compatibility " + "version '%s'. Trying next version. Error trace: %s", + version, traceback.format_exc()) + + if not version_used: + raise Exception( + "Could not initialize vixDiskLib with any of the following " + "versions: %s" % target_versions) + + LOG.info( + "Successfully initialized vixDiskLib with target version '%s'", + version_used) + + @classmethod + def get_vix_disklib_name(cls): + vixDiskLibName = None + if os.name == 'nt': + vixDiskLibName = 'vixDiskLib.dll' + else: + vixDiskLibName = 'libvixDiskLib.so' + return vixDiskLibName + + def _setup_vix_disklib(self): + self._vix_disklib.VixDiskLib_InitEx.argtypes = [ + ctypes.c_uint32, ctypes.c_uint32, ctypes.c_void_p, ctypes.c_void_p, + ctypes.c_void_p, ctypes.c_char_p, ctypes.c_char_p] + self._vix_disklib.VixDiskLib_InitEx.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_GetErrorText.argtypes = [ + ctypes.c_uint64, ctypes.c_char_p] + self._vix_disklib.VixDiskLib_GetErrorText.restype = ctypes.c_void_p + + self._vix_disklib.VixDiskLib_FreeErrorText.arg_types = [ + ctypes.c_char_p] + self._vix_disklib.VixDiskLib_FreeErrorText.restype = None + + self._vix_disklib.VixDiskLib_ListTransportModes.argtypes = [] + self._vix_disklib.VixDiskLib_ListTransportModes.restype = ( + ctypes.c_char_p) + + self._vix_disklib.VixDiskLib_GetTransportMode.argtypes = [ + ctypes.c_void_p] + self._vix_disklib.VixDiskLib_GetTransportMode.restype = ( + ctypes.c_char_p) + + self._vix_disklib.VixDiskLib_ConnectEx.argtypes = [ + ctypes.POINTER(VixDiskLibConnectParams), ctypes.c_char, + ctypes.c_char_p, ctypes.c_char_p, ctypes.POINTER(ctypes.c_void_p)] + self._vix_disklib.VixDiskLib_ConnectEx.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_Open.argtypes = [ + ctypes.c_void_p, ctypes.c_char_p, ctypes.c_uint32, + ctypes.POINTER(ctypes.c_void_p)] + self._vix_disklib.VixDiskLib_Open.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_Read.argtypes = [ + ctypes.c_void_p, ctypes.c_uint64, ctypes.c_uint64, ctypes.c_char_p] + self._vix_disklib.VixDiskLib_Read.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_Write.argtypes = [ + ctypes.c_void_p, ctypes.c_uint64, ctypes.c_uint64, ctypes.c_char_p] + self._vix_disklib.VixDiskLib_Write.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_GetMetadataKeys.argtypes = [ + ctypes.c_void_p, ctypes.c_char_p, ctypes.c_uint64, + ctypes.POINTER(ctypes.c_uint64)] + self._vix_disklib.VixDiskLib_GetMetadataKeys.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_ReadMetadata.argtypes = [ + ctypes.c_void_p, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_uint64, + ctypes.POINTER(ctypes.c_uint64)] + self._vix_disklib.VixDiskLib_ReadMetadata.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_Close.argtypes = [ctypes.c_void_p] + self._vix_disklib.VixDiskLib_Close.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_Disconnect.argtypes = [ctypes.c_void_p] + self._vix_disklib.VixDiskLib_Disconnect.restype = ctypes.c_uint64 + + self._vix_disklib.VixDiskLib_Exit.argtypes = [] + self._vix_disklib.VixDiskLib_Exit.restype = None + + def _check_err(self, err, allowed_values=[vix_disklib_errors.VIX_OK]): + if err not in allowed_values: + err_msg = self._vix_disklib.VixDiskLib_GetErrorText(err, None) + err_msg_copy = str(ctypes.cast( + err_msg, ctypes.c_char_p).value.decode()) + self._vix_disklib.VixDiskLib_FreeErrorText( + ctypes.cast(err_msg, ctypes.c_char_p)) + + msg = None + if err == vix_disklib_errors.VIX_E_OUT_OF_MEMORY: + msg = ( + "The ESXi host performing the CBT export ran out of RAM. " + "The host performing the export is automatically chosen " + "by vCenter, so enough RAM to run the export is required " + "on all hosts. To force the export from the specific host " + "the VM is on, create a Coriolis endpoint with the DNS " + "name/IP address of that host.") + if err == vix_disklib_errors.VIX_E_HOST_NETWORK_CONN_REFUSED: + msg = ( + "The ESXi host performing the CBT export refused " + "connection. The host is chosen automatically by vCenter, " + "so please ensure that the Coriolis deployment " + "can dial TCP/902 on all of the ESXi hosts of a vSphere, " + "and that DNS name resolution and firewalls are setup to " + "facilitate this. Alternatively, try connecting Coriolis " + "directly to the specific ESXi host which is running the " + "VM(s) to be migrated by creating a Coriolis endpoint " + "using the DNS name/IP address of the host itself.") + + if err == vix_disklib_errors.VIX_E_CANNOT_CONNECT_TO_HOST: + msg = ( + "Coriolis lost connection to the ESXi host performing the " + "CBT export. If the Coriolis Endpoint connects to a " + "vSphere host, please try connecting Coriolis to the ESXi " + "host directly. If problem persists, try re-enabling CBT " + "on the VM, or moving it to another ESXi host.") + + err_msg = err_msg_copy + if msg: + LOG.debug("Original vixDiskLib error message: %s", err_msg_copy) + err_msg = msg + + raise Exception(err_msg) + + def get_transport_modes(self): + transport_modes = self._vix_disklib.VixDiskLib_ListTransportModes() + return transport_modes.decode().split(':') + + def get_transport_mode(self, disk_handle): + t_mode = self._vix_disklib.VixDiskLib_GetTransportMode(disk_handle) + return t_mode.decode() + + @contextlib.contextmanager + def connect( + self, server_name, thumbprint, username, password, + vmx_spec=None, snapshot_ref=None, read_only=True, + transport_modes=None, port=443): + LOG.debug("Connecting VixDiskLib: %s", server_name) + + connectParams = VixDiskLibConnectParams() + + connectParams.serverName = server_name.encode() + if vmx_spec: + connectParams.vmxSpec = vmx_spec.encode() + if thumbprint: + connectParams.thumbPrint = thumbprint.encode() + + connectParams.credType = VIXDISKLIB_CRED_UID + connectParams.creds.uid.userName = username.encode() + connectParams.creds.uid.password = password.encode() + connectParams.port = port + + if transport_modes: + transport_modes = transport_modes.encode() + + if snapshot_ref: + snapshot_ref = snapshot_ref.encode() + + conn = ctypes.c_void_p() + self._check_err(self._vix_disklib.VixDiskLib_ConnectEx( + connectParams, read_only, snapshot_ref, transport_modes, + ctypes.byref(conn))) + try: + yield conn + finally: + self.disconnect(conn) + + @contextlib.contextmanager + def open(self, conn, disk_path, flags=VIXDISKLIB_FLAG_OPEN_READ_ONLY): + LOG.debug("Openning VixDiskLib disk: %s", disk_path) + + disk_handle = ctypes.c_void_p() + self._check_err(self._vix_disklib.VixDiskLib_Open( + conn, disk_path.encode(), flags, ctypes.byref(disk_handle))) + try: + yield disk_handle + finally: + self.close(disk_handle) + + def read(self, disk_handle, start_sector, num_sectors, buf): + self._check_err(self._vix_disklib.VixDiskLib_Read( + disk_handle, start_sector, num_sectors, buf)) + + def write(self, disk_handle, start_sector, num_sectors, buf): + """Write ``num_sectors`` from ``buf`` starting at ``start_sector``.""" + self._check_err(self._vix_disklib.VixDiskLib_Write( + disk_handle, start_sector, num_sectors, buf)) + + def close(self, disk_handle): + LOG.debug("Closing VixDiskLib disk handle: %s", disk_handle) + self._check_err(self._vix_disklib.VixDiskLib_Close(disk_handle)) + + def disconnect(self, conn): + LOG.debug("Disconnecting VixDiskLib") + self._check_err(self._vix_disklib.VixDiskLib_Disconnect(conn)) + + def exit(self): + self._vix_disklib.VixDiskLib_Exit() diff --git a/tests/unit/__init__.py b/tests/unit/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tox.ini b/tox.ini new file mode 100644 index 0000000..00a438c --- /dev/null +++ b/tox.ini @@ -0,0 +1,94 @@ +# Copyright 2026 Cloudbase Solutions Srl +# All Rights Reserved. + +[tox] +minversion = 4.0.2 +envlist = fmt,flake8,mypy,pep8,py3 +skipsdist = True + +[vars] +src_path = {toxinidir}/openvixdisklib +tst_path = {toxinidir}/tests +all_path = {[vars]src_path} {[vars]tst_path} +vddk_path = {toxinidir}/.vddk + +[testenv] +sitepackages = True +usedevelop = True +setenv = + VIRTUAL_ENV={envdir} + PYTHONDONTWRITEBYTECODE=1 +deps = + -r{toxinidir}/requirements.txt + -r{toxinidir}/test-requirements.txt +allowlist_externals = + bash + +[testenv:integration] +description = Run integration tests against a VMware environment. +sitepackages = True +usedevelop = True +setenv = + {[testenv]setenv} + LD_LIBRARY_PATH={[vars]vddk_path} + LD_PRELOAD= +commands = + stestr run --slowest --concurrency 1 \ + --test-path tests/integration/ {posargs} + +[testenv:integration-coverage] +description = Integration tests with coverage report. +sitepackages = True +usedevelop = True +setenv = + {[testenv:integration]setenv} +deps = + {[testenv]deps} + pytest + pytest-cov + pycobertura +commands = + pytest --cov=openvixdisklib --cov-report=term-missing \ + --cov-report=xml -p no:cacheprovider tests/integration/ {posargs} + pycobertura show --format html --source openvixdisklib \ + coverage.xml -o coverage.html + +[testenv:fmt] +description = Format the code based on the coding style standards. +deps = + ruff +commands = + ruff check --select I --fix {[vars]all_path} + ruff format {[vars]all_path} + +[testenv:pep8] +description = Verify the coding style (pep8). +deps = + ruff +commands = + ruff format --diff {[vars]all_path} + ruff check {[vars]all_path} + +[testenv:mypy] +description = Type checks (mypy). +deps = + mypy +commands = + mypy {[vars]all_path} + +[testenv:flake8] +sitepackages = True +deps = + flake8 +commands = flake8 {posargs} +allowlist_externals = + flake8 + +[flake8] +ignore = E125,E251,W503,W504,E305,E731,E117,W605,F632 +exclude = .venv,.git,.tox,dist,build,*.egg +# Same length as ruff. +max-line-length = 88 + +[stestr] +test_path = ./tests