justinandClaude Opus 5
8ac89c7f82
Two concurrent tool calls each found the token stale, each POSTed the password grant, and the appliance rejected one of the simultaneous grants. That call died with "Keycloak token failed HTTP 401" -- nothing to do with the operation it was performing. Hit while testing the guard: two concurrent zerto_guard_before_mutate calls, one came back 401. ensure_token now double-checks the cache under an asyncio.Lock, so N concurrent callers produce exactly one token request and the rest reuse the result. Measured against the lab ZVM with a cold client: 8 concurrent reads made 8 token POSTs before, 1 after. The 401-retry path had the same shape and was worse: every in-flight request that got a 401 set _token = None and re-authed independently, so one expiry became a thundering herd. Requests now capture a token generation, and _reauth re-fetches only if nothing else has already moved past it. Token fetch also gets a bounded retry for transient failures (5xx, network) with linear backoff. 401 and 403 are NOT retried: those are the credentials themselves, and hammering Keycloak can trip its brute-force lockout on a real service account. Per the Zerto API lessons, auth failures now name the cause Keycloak reported instead of a generic hint -- invalid_client means the client_id is wrong for this appliance (10.x zerto-client, 9.x may be zerto-api), invalid_grant means the username or password is. That is the first thing to check and it was previously guesswork. Also clears the two long-standing lint findings in this file (PIE810, E501) while it is open. ruff check now passes across src/ and tests/. pytest 48 passed (7 new, covering single-request concurrency, cache reuse, both generation branches, no-retry-on-bad-credentials, the invalid_client message, and transient retry). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_016yVfC5nvZowoLFnEGWhLGn
zerto-ai-rewind
PoC MCP that makes an agent pin a Zerto tagged checkpoint before it changes a VM, then pull a file back from that tag.
If the loop works, these tools are the delta to add to official ZVM MCP (ZVM.MCP, 10.9). This repo is one MCP process for the demo. It is not a second full ZVM catalog.
What it does
zerto_check_tool— before running anything against a VM:read_only(go),mutating(guard first), orunknown(ask the human whether to checkpoint).zerto_find_protection— VM name, hostname, or vmIdentifier to exactly one VM and every VPG. Zero or two-plus VMs: stop.zerto_create_tagged_checkpoint/zerto_guard_before_mutate— same tag on every protecting VPG, wait until listed. The name records which agent and what it is doing:ai:<agent> | <action> | vm=<vm> | change=<id> | <utc>.zerto_recover_file— FLR after a human setsconfirmed=true. Linux and Windows guest paths. Locally replicated VPGs only: FLR runs at the VPG's recovery site. Reports its own unmount;zerto_list_flr_sessions/zerto_end_flr_sessionfind and reap a mount orphaned by a crashed recovery.- Two catalogs —
mutating_tools(guard first) andread_only_tools(safe). Both cover Linux (ssh,ansible) and Windows (winrm,powershell,smb), and both are illustrative, not exhaustive. A tool in neither is unknown, not safe: ask the human.
Official ZVM MCP already has inventory and failover test. It does not insert tagged checkpoints or run FLR.
Setup
Python 3.12+.
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
cp config.example.json config.json
# edit zerto_url, username, password
Keycloak password-grant, client_id zerto-client on 10.x. Appliance certs are self-signed; verify_tls defaults to false.
stdio MCP (Claude Desktop, VS Code, Cursor, OpenCode):
{
"mcpServers": {
"zerto-rewind": {
"command": "zerto-rewind-mcp",
"env": {
"ZERTO_REWIND_CONFIG": "/absolute/path/to/config.json"
}
}
}
}
Copy skills/zerto-rewind/SKILL.md into the client's skill path.
pytest
Demo
Protected app VM. Agent is about to edit a guest config file.
- Guard: discover VPG set, insert tagged checkpoint, wait.
- Agent writes the bad config.
- Human confirms.
zerto_recover_filefrom that tag.
Git never had the file. RPO is the journal, not last night's backup.
Certified for this PoC
vSphere ZVM 10.x and ZCA on AWS/Azure, same REST paths. HVM is out (separate swagger). Failover Live is not a tool.
Tagged checkpoints cannot be inserted when the protected site is Azure or AWS (Zerto API). Point this server at the vSphere protected ZVM.
Not this product
Moholo Agent Rewind snapshots the agent's laptop tools. This server uses the Zerto journal as the snapshot store. Do not copy their file blobs.
Upstream
Ask Zerto engineering to add to ZVM.MCP: find-by-unique-VM-with-all-VPGs, tagged checkpoint insert that waits, FLR. Keep VPG settings CRUD where it already is.