Compare commits
1 Commits
main
..
24b9e0aa80
| Author | SHA1 | Date | |
|---|---|---|---|
 justin
|
24b9e0aa80 |
@@ -62,9 +62,11 @@ jobs:
|
||||
shell: pwsh
|
||||
run: ./scripts/build-installer.ps1 -VersionOverride ${{ steps.ver.outputs.version }}
|
||||
|
||||
# actions/upload-artifact@v4 is GitHub-only ("GHESNotSupportedError" on
|
||||
# Gitea). The release-creation step below attaches the .exe via Gitea's
|
||||
# API directly, which is the only place we actually need to surface it.
|
||||
- name: Upload installer artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: WebhookServer-Setup-${{ steps.ver.outputs.version }}
|
||||
path: dist/WebhookServer-Setup-*.exe
|
||||
|
||||
- name: Create Gitea release with installer attached
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<Project>
|
||||
|
||||
<PropertyGroup>
|
||||
<Version>0.1.5</Version>
|
||||
<Version>0.1.4</Version>
|
||||
<Authors>Justin Paul</Authors>
|
||||
<Company>Justin Paul</Company>
|
||||
<Product>Webhook Server</Product>
|
||||
@@ -9,7 +9,6 @@
|
||||
<PackageProjectUrl>https://jpaul.me</PackageProjectUrl>
|
||||
<RepositoryUrl>https://github.com/recklessop/webhook-server</RepositoryUrl>
|
||||
<RepositoryType>git</RepositoryType>
|
||||
<PackageLicenseExpression>MIT</PackageLicenseExpression>
|
||||
</PropertyGroup>
|
||||
|
||||
</Project>
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025-2026 Justin Paul
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -61,11 +61,11 @@ Everything you need to operate the server:
|
||||
|
||||
Recipes:
|
||||
|
||||
- [Zerto ZVMA (Kubernetes) pre/post → notify + VM health check](docs/recipes/zerto-zvma-pre-post.md) ← **canonical use case**
|
||||
- [Zerto failover post-script → DNS + service checks](docs/recipes/zerto-pre-post-scripts.md) ← **canonical use case**
|
||||
- [GitHub-style HMAC-signed webhook](docs/recipes/github-style-hmac.md)
|
||||
- [Pop UI on the user's desktop](docs/recipes/ui-on-desktop.md)
|
||||
|
||||
The Zerto ZVMA recipe ships ready-to-drop-in scripts: [`scripts/examples/zerto-zvma-send.ps1`](scripts/examples/zerto-zvma-send.ps1) (sender, runs inside the ZVMA `scripts-service` container) plus [`zerto-receiver-notify.ps1`](scripts/examples/zerto-receiver-notify.ps1) and [`zerto-receiver-vm-healthcheck.ps1`](scripts/examples/zerto-receiver-vm-healthcheck.ps1) (receivers, run on the Webhook Server host).
|
||||
A ready-to-drop-in Zerto-side script is included at [`scripts/examples/zerto-post-failover.ps1`](scripts/examples/zerto-post-failover.ps1).
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -88,4 +88,4 @@ powershell -ExecutionPolicy Bypass -File scripts\build-installer.ps1
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE). Use it for whatever you want, including commercial — just keep the copyright + license notice in copies and don't sue me when it eats your filesystem. No warranty, express or implied.
|
||||
TBD.
|
||||
|
||||
+3
-3
@@ -6,7 +6,7 @@ Webhook Server is a Windows service that runs a script (PowerShell, cmd, or any
|
||||
|
||||
1. [Concepts](concepts.md) — five-minute read on what a webhook is and how this server uses one
|
||||
2. [Installation](installation.md) — download, install, first endpoint
|
||||
3. [Recipe: Zerto ZVMA pre/post → notify + VM health check](recipes/zerto-zvma-pre-post.md) — the canonical reason this exists
|
||||
3. [Recipe: Zerto failover post-script → DNS + service checks](recipes/zerto-pre-post-scripts.md) — the canonical reason this exists
|
||||
|
||||
## Topical
|
||||
|
||||
@@ -19,11 +19,11 @@ Webhook Server is a Windows service that runs a script (PowerShell, cmd, or any
|
||||
|
||||
## Recipes (cookbook style)
|
||||
|
||||
- [Zerto ZVMA (Kubernetes) pre/post → notify + VM health check](recipes/zerto-zvma-pre-post.md) ← canonical use case
|
||||
- [Zerto failover post-script → DNS + service checks](recipes/zerto-pre-post-scripts.md) ← canonical use case
|
||||
- [GitHub-style HMAC-signed webhook](recipes/github-style-hmac.md)
|
||||
- [Pop UI on the user's desktop](recipes/ui-on-desktop.md)
|
||||
|
||||
The Zerto ZVMA recipe ships with [`zerto-zvma-send.ps1`](../scripts/examples/zerto-zvma-send.ps1) (sender, runs inside the ZVMA `scripts-service` container) plus [`zerto-receiver-notify.ps1`](../scripts/examples/zerto-receiver-notify.ps1) and [`zerto-receiver-vm-healthcheck.ps1`](../scripts/examples/zerto-receiver-vm-healthcheck.ps1) (receivers, run on the Webhook Server host).
|
||||
The flagship Zerto recipe also ships with a **ready-to-use Zerto-side post-script** at [`scripts/examples/zerto-post-failover.ps1`](../scripts/examples/zerto-post-failover.ps1).
|
||||
|
||||
## Reference
|
||||
|
||||
|
||||
@@ -76,7 +76,7 @@ The endpoint appears in the grid. Right-click it → **Copy URL**, paste into a
|
||||
{ "runId": "...", "exitCode": 0, "durationMs": 134, "stdout": "pong\r\n", ... }
|
||||
```
|
||||
|
||||
That's it. Real-world recipes start with [Zerto ZVMA pre/post → notify + VM health check](recipes/zerto-zvma-pre-post.md).
|
||||
That's it. Real-world recipes start with [Zerto pre/post scripts → AD / DNS update](recipes/zerto-pre-post-scripts.md).
|
||||
|
||||
## Silent / unattended install
|
||||
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
# Recipe: Zerto failover post-script → DNS update + service checks
|
||||
|
||||
This is the canonical reason Webhook Server exists.
|
||||
|
||||
When Zerto fails a VM over from production to DR, the VM boots fine — but **the things around it** often need attention: DNS records still point at the production IP, dependent services need to be checked, on-call needs a heads-up. Zerto pre/post scripts run on the **Zerto Virtual Manager**, not on a domain controller and not necessarily with admin rights to the things that need fixing. So you want a single webhook URL that the post-script hits, and a Windows host on the DR side that does the actual work with the right identity.
|
||||
|
||||
## What we're building
|
||||
|
||||
Zerto's post-recovery script (a one-shot PowerShell file pointing at curl) calls `http://webhook.dr.contoso.local:8080/hook/post-failover` with a JSON body identifying the VPG and operation. The Webhook Server, running on a DR-side Windows host as a gMSA with delegated AD/DNS rights, runs PowerShell that:
|
||||
|
||||
1. Updates DNS A records to point the failed-over hostnames at their DR IPs
|
||||
2. Waits for the failed-over VM to come up (ping + WinRM probe)
|
||||
3. Connects to the VM via PowerShell remoting and starts/checks critical services
|
||||
4. Sends a Teams notification with the result
|
||||
|
||||
The endpoint is **Async** so the Zerto script returns in milliseconds — no risk of timing out Zerto's failover sequence even if the actions take minutes. The script's full output ends up in the webhook log and (optionally) in an outbound callback.
|
||||
|
||||
## Why curl and not Invoke-WebRequest?
|
||||
|
||||
Zerto's PowerShell runner is intentionally minimal — many environments run an older Windows on the ZVM and don't have full PowerShell modules installed. `curl.exe` ships with Windows 10 1803+ and Server 2019+ and works without any modules. Plus, calling an HTTP endpoint with `curl.exe` doesn't depend on the version of `Invoke-WebRequest` shipped with the host's PowerShell.
|
||||
|
||||
## 1. The Zerto post-script (client side)
|
||||
|
||||
A ready-to-use script ships in this repo at [`scripts/examples/zerto-post-failover.ps1`](../../scripts/examples/zerto-post-failover.ps1). Copy it to the ZVM, edit `$WebhookUrl` and the bearer-token path at the top, and wire it into the VPG:
|
||||
|
||||
> **VPG settings → Recovery → Scripts → Post-Recovery Script**
|
||||
> Path: `C:\Scripts\zerto-post-failover.ps1`
|
||||
> Parameters: *(leave empty)*
|
||||
|
||||
The script is ~50 lines and only depends on `curl.exe` + a token file readable by the ZVM service account.
|
||||
|
||||
The flow:
|
||||
|
||||
```
|
||||
Zerto VPG failover starts
|
||||
|
|
||||
+-- VM is brought up at DR site
|
||||
|
|
||||
+-- Zerto post-script fires:
|
||||
| curl POST http://webhook.dr/hook/post-failover (async, returns 202 in ~50ms)
|
||||
|
|
||||
+-- Zerto sees success, finishes the failover and reports done
|
||||
|
|
||||
(meanwhile, on the webhook server)
|
||||
|
|
||||
running PowerShell for several minutes:
|
||||
- update DNS
|
||||
- wait for VM ready
|
||||
- check services on VM
|
||||
- notify Teams
|
||||
```
|
||||
|
||||
## 2. The server-side script (does the actual work)
|
||||
|
||||
Save this on the webhook host as `C:\Scripts\post-failover-handler.ps1`:
|
||||
|
||||
```powershell
|
||||
[CmdletBinding()]
|
||||
param()
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
$body = $input | ConvertFrom-Json
|
||||
|
||||
# ---------- environment specifics; edit for your site ----------
|
||||
$dnsServer = 'dc01.contoso.local'
|
||||
$forwardZone = 'contoso.local'
|
||||
$teamsWebhook = 'https://contoso.webhook.office.com/...'
|
||||
$drIpMap = @{
|
||||
'app01' = '10.42.10.11'
|
||||
'app02' = '10.42.10.12'
|
||||
'db01' = '10.42.10.21'
|
||||
}
|
||||
$serviceMap = @{
|
||||
'app01' = @('W3SVC','MyAppSvc')
|
||||
'app02' = @('W3SVC','MyAppSvc')
|
||||
'db01' = @('MSSQLSERVER','SQLAgent')
|
||||
}
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
# Default the VM list to "all VMs we know about" if the post-script didn't
|
||||
# tell us, so the same handler works without having to embed the VM list in
|
||||
# every Zerto post-script.
|
||||
$vms = if ($body.vms) { $body.vms } else { $drIpMap.Keys }
|
||||
|
||||
$summary = @()
|
||||
|
||||
foreach ($vm in $vms) {
|
||||
if (-not $drIpMap.ContainsKey($vm)) {
|
||||
$summary += "skip $vm (no DR IP mapping in handler)"
|
||||
continue
|
||||
}
|
||||
$ip = $drIpMap[$vm]
|
||||
|
||||
# 1. DNS - delete + re-add the A record
|
||||
try {
|
||||
$existing = Get-DnsServerResourceRecord -ZoneName $forwardZone -Name $vm `
|
||||
-RRType A -ComputerName $dnsServer -ErrorAction SilentlyContinue
|
||||
if ($existing) {
|
||||
Remove-DnsServerResourceRecord -ZoneName $forwardZone -Name $vm `
|
||||
-RRType A -RecordData $existing.RecordData.IPv4Address `
|
||||
-ComputerName $dnsServer -Force
|
||||
}
|
||||
Add-DnsServerResourceRecordA -ZoneName $forwardZone -Name $vm `
|
||||
-IPv4Address $ip -ComputerName $dnsServer -TimeToLive 00:05:00
|
||||
$summary += "dns $vm -> $ip"
|
||||
} catch {
|
||||
$summary += "DNS! $vm $($_.Exception.Message)"
|
||||
continue
|
||||
}
|
||||
|
||||
# 2. Wait for the VM to be reachable (up to 5 minutes)
|
||||
$deadline = (Get-Date).AddMinutes(5)
|
||||
$reachable = $false
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
if (Test-Connection -ComputerName $ip -Count 1 -Quiet -ErrorAction SilentlyContinue) {
|
||||
try {
|
||||
# Quick WinRM probe; succeeds when the VM has finished booting
|
||||
Invoke-Command -ComputerName $ip -ScriptBlock { $true } -ErrorAction Stop | Out-Null
|
||||
$reachable = $true
|
||||
break
|
||||
} catch { Start-Sleep -Seconds 10 }
|
||||
} else {
|
||||
Start-Sleep -Seconds 10
|
||||
}
|
||||
}
|
||||
if (-not $reachable) {
|
||||
$summary += "wait! $vm not reachable after 5 minutes"
|
||||
continue
|
||||
}
|
||||
|
||||
# 3. Check + start critical services on the VM
|
||||
if ($serviceMap.ContainsKey($vm)) {
|
||||
$svcReport = Invoke-Command -ComputerName $ip -ArgumentList @(,$serviceMap[$vm]) -ScriptBlock {
|
||||
param($services)
|
||||
$report = @()
|
||||
foreach ($s in $services) {
|
||||
$svc = Get-Service -Name $s -ErrorAction SilentlyContinue
|
||||
if (-not $svc) { $report += "$s : missing"; continue }
|
||||
if ($svc.Status -ne 'Running') {
|
||||
Start-Service $s
|
||||
Start-Sleep -Seconds 2
|
||||
$svc.Refresh()
|
||||
}
|
||||
$report += "$s : $($svc.Status)"
|
||||
}
|
||||
return $report
|
||||
}
|
||||
$summary += "svc $vm : $($svcReport -join ', ')"
|
||||
} else {
|
||||
$summary += "svc $vm (no services configured)"
|
||||
}
|
||||
}
|
||||
|
||||
# 4. Notify Teams
|
||||
$teamsBody = @{
|
||||
text = "Webhook post-failover for VPG **$($body.vpg)**:`n" + ($summary -join "`n")
|
||||
} | ConvertTo-Json
|
||||
try {
|
||||
Invoke-RestMethod -Uri $teamsWebhook -Method POST -ContentType 'application/json' -Body $teamsBody | Out-Null
|
||||
} catch {
|
||||
$summary += "teams! notification failed: $($_.Exception.Message)"
|
||||
}
|
||||
|
||||
# Return the summary so it shows up in the webhook log + outbound callback
|
||||
$summary -join "`n"
|
||||
```
|
||||
|
||||
Two things to call out:
|
||||
|
||||
- **PowerShell remoting to the VM** uses the gMSA's network identity (or whoever the service runs as). Make sure the gMSA / service account can `Invoke-Command` to the failed-over hosts — usually that means the account is a local admin on the target VMs, or you've configured constrained delegation.
|
||||
- **WinRM** must be enabled on the failed-over VMs for the remoting calls to work. `Enable-PSRemoting` is the simplest, but most prod environments configure WinRM via Group Policy.
|
||||
|
||||
## 3. Configure the endpoint in the GUI
|
||||
|
||||
**File → New endpoint:**
|
||||
|
||||
| Section | Setting | Value |
|
||||
|---|---|---|
|
||||
| Identity | Slug | `post-failover` |
|
||||
| Identity | Description | "Zerto post-recovery: DNS + service checks" |
|
||||
| Auth | Mode | **Bearer** |
|
||||
| Auth | Bearer secret | generate a 32-byte random string; copy it for the Zerto script's token file |
|
||||
| Allowed clients | (one per line) | `10.0.0.0/8` *(your ZVM's network)* |
|
||||
| Executor | Type | **Windows PowerShell** |
|
||||
| Executor | Script path | `C:\Scripts\post-failover-handler.ps1` |
|
||||
| Data passing | JSON body to stdin | ✓ |
|
||||
| Run as | Identity | **Service** if the service runs under a gMSA with the right rights, otherwise **SpecificUser** with a delegated account |
|
||||
| Response | Mode | **Async** ← critical: this is what makes the Zerto script non-blocking |
|
||||
| Response | Timeout (sec) | `600` *(this is the cap on the long-running handler script, not the Zerto-facing response)* |
|
||||
| Response | Fail on non-zero exit | unticked *(async hooks have no caller to receive a 502)* |
|
||||
|
||||
Save. Right-click the row → **Copy URL** to grab `http://webhook.dr.contoso.local:8080/hook/post-failover` and paste it into `$WebhookUrl` at the top of the Zerto-side script.
|
||||
|
||||
> **Why Bearer instead of HMAC?** Both work. Bearer is simpler — drop the token in a file on the ZVM that's readable by the ZVM service account and you're done. HMAC requires the Zerto-side script to compute a signature, which is doable but adds a few lines of code. Pick what fits your environment.
|
||||
|
||||
## 4. Wire up the bearer token
|
||||
|
||||
Place the bearer token in a file the ZVM service account can read (and nobody else):
|
||||
|
||||
```powershell
|
||||
# on the ZVM, from elevated PowerShell
|
||||
$token = (New-Guid).ToString('N') # or paste the value from the GUI
|
||||
$tokenPath = 'C:\ProgramData\Zerto\webhook-token.txt'
|
||||
$token | Out-File -LiteralPath $tokenPath -Encoding utf8 -NoNewline
|
||||
icacls $tokenPath /inheritance:r /grant 'NT SERVICE\Zerto Online Services:R' 'BUILTIN\Administrators:F' /T
|
||||
```
|
||||
|
||||
Adjust the service principal name to whatever Zerto runs as on your version. The script reads from this path automatically; no change needed in the script itself.
|
||||
|
||||
## 5. Test before going live
|
||||
|
||||
In a maintenance window, fire the webhook by hand:
|
||||
|
||||
```powershell
|
||||
# from any machine that can reach the webhook server
|
||||
$body = @{
|
||||
operation = 'test'
|
||||
vpg = 'SmokeTest'
|
||||
timestamp = (Get-Date).ToUniversalTime().ToString('o')
|
||||
} | ConvertTo-Json -Compress
|
||||
|
||||
curl.exe --silent --show-error --max-time 10 -X POST `
|
||||
-H "Authorization: Bearer paste-the-token" `
|
||||
-H "Content-Type: application/json" `
|
||||
-d $body `
|
||||
http://webhook.dr.contoso.local:8080/hook/post-failover
|
||||
```
|
||||
|
||||
You'll get back `{"runId":"…","accepted":true}` immediately. Open the Webhook Server GUI and watch the log panel — within 30 seconds or so you'll see lines for the run. Confirm DNS records updated, services on each VM ended in `Running`, and the Teams notification arrived.
|
||||
|
||||
## Variations
|
||||
|
||||
### Different actions for failover vs. failback
|
||||
|
||||
Pass an `operation` field in the body and branch on it. The Zerto-side script already sends `operation = 'failover'`. Add a separate post-failback script (or detect from `$env:ZertoOperationType`) that sends `operation = 'failback'` and have the handler revert DNS to production IPs.
|
||||
|
||||
### Per-VPG endpoints
|
||||
|
||||
If you want fine-grained access control or different actions per VPG, create one endpoint per VPG (`post-failover-app`, `post-failover-db`, …) and give each its own bearer token. The GUI handles dozens of endpoints fine.
|
||||
|
||||
### Audit trail to a SIEM
|
||||
|
||||
Each endpoint can have an outbound **Callback** URL. Configure it with your SIEM's HTTP collector + an HMAC secret, and every run produces a JSON record with runId, exit code, duration, stdout, and stderr — perfect for compliance.
|
||||
@@ -1,275 +0,0 @@
|
||||
# Recipe: Zerto ZVMA pre/post scripts → notify + VM health check
|
||||
|
||||
> This is the **canonical** Zerto recipe. It targets the **ZVMA on
|
||||
> Kubernetes** — the supported deployment — where pre/post scripts run
|
||||
> inside the in-cluster `scripts-service` container (Linux + pwsh 7). The
|
||||
> webhook-server side is a normal Windows service that does the
|
||||
> Windows-domain work the ZVMA container can't reach directly.
|
||||
|
||||
## What we're building
|
||||
|
||||
ZVMA's `scripts-service` pod runs your VPG pre/post scripts inside a Linux
|
||||
container. It exposes a small set of `Zerto*` environment variables, and we
|
||||
want to:
|
||||
|
||||
1. POST those variables to a Webhook Server endpoint at the start (pre) and
|
||||
end (post) of every VPG operation, and
|
||||
2. On the receiving Windows host, do something useful with them — at minimum
|
||||
a chat notification, and on `post` a quick health check of the VMs that
|
||||
just powered on.
|
||||
|
||||
The endpoints are **Async**, so the Zerto VPG sequence is never blocked by
|
||||
slow downstream actions (notifications, port probes, etc.).
|
||||
|
||||
```
|
||||
Zerto VPG operation starts
|
||||
|
|
||||
+-- ZVMA scripts-service container runs:
|
||||
| /app/scripts-files/zerto-zvma-send.ps1 -Phase pre
|
||||
| -> POST http://webhook.dr/hook/zerto-pre (async, returns 202)
|
||||
|
|
||||
+-- VMs come up at recovery site
|
||||
|
|
||||
+-- ZVMA scripts-service container runs:
|
||||
/app/scripts-files/zerto-zvma-send.ps1 -Phase post
|
||||
-> POST http://webhook.dr/hook/zerto-post (async, returns 202)
|
||||
|
||||
(meanwhile, on the webhook server)
|
||||
/hook/zerto-pre -> Slack/Teams notification ("Test failover starting...")
|
||||
/hook/zerto-post -> Slack/Teams notification + ping/port probe each VM,
|
||||
write a JSON report to disk, exit non-zero on failure.
|
||||
```
|
||||
|
||||
## What ZVMA exposes
|
||||
|
||||
Captured from a real Test failover; same set is present in pre and post:
|
||||
|
||||
| Variable | Example | Notes |
|
||||
|---|---|---|
|
||||
| `ZertoVPGName` | `ubuntu-2404-local` | The VPG that fired the script |
|
||||
| `ZertoInternalVpgName` | `ubuntu-2404-local` | Usually identical to `ZertoVPGName` |
|
||||
| `ZertoOperation` | `Test` | `Test` / `Failover` / `Move` / `FailoverBeforeCommit` / `FailoverDuringCommit` |
|
||||
| `ZertoForce` | `Yes` (pre) / `No` (post) | Set to `Yes` only during the pre phase when force mode is on; reset to `No` by post |
|
||||
| `VmDisplayNames` | `ubuntu-2404(1)(1)(1)` | Comma-separated for multi-VM VPGs; Test failovers add `(N)` suffixes |
|
||||
| `ZertoHypervisorManagerIP` | `192.168.50.20` | The vCenter / Hyper-V manager ZVMA is talking to |
|
||||
| `ZertoHypervisorManagerPort` | `443` | |
|
||||
| `ZertoOutputDir` | `/app/scripts-output` | Container-side output dir (written back to ZVMA via PVC) |
|
||||
| `ZertoWorkingDir` | `/app/scripts-files` | Where script files live in-container |
|
||||
|
||||
Branch on `ZertoOperation` to differentiate Test runs from real failovers.
|
||||
**`ZertoForce` is only meaningful during the pre phase** — capture it there
|
||||
if you need it later, because by post it's been reset.
|
||||
|
||||
## 1. The Zerto-side script (sender)
|
||||
|
||||
A ready-to-use script ships in this repo at
|
||||
[`scripts/examples/zerto-zvma-send.ps1`](../../scripts/examples/zerto-zvma-send.ps1).
|
||||
Place it where the `scripts-service` pod can read it — typically the
|
||||
`scripts-service-scripts-files-pvc`, mounted at `/app/scripts-files/` — and
|
||||
wire it into the VPG twice:
|
||||
|
||||
> **VPG settings → Recovery → Scripts → Pre-Recovery Script**
|
||||
> Path: `/app/scripts-files/zerto-zvma-send.ps1`
|
||||
> Parameters: `-Phase pre`
|
||||
>
|
||||
> **VPG settings → Recovery → Scripts → Post-Recovery Script**
|
||||
> Path: `/app/scripts-files/zerto-zvma-send.ps1`
|
||||
> Parameters: `-Phase post`
|
||||
|
||||
The default `$WebhookUrl` includes `{phase}` so one script + one URL config
|
||||
serves both phases — `http://webhook.dr/hook/zerto-{phase}` becomes
|
||||
`/hook/zerto-pre` and `/hook/zerto-post` automatically. Override with
|
||||
`-WebhookUrl` and `-Bearer` if you'd rather pass them per-VPG.
|
||||
|
||||
The script POSTs a single JSON object:
|
||||
|
||||
```json
|
||||
{
|
||||
"phase": "pre",
|
||||
"capturedAt": "2026-05-08T17:45:54Z",
|
||||
"host": "scripts-service-f9b6cb7-4xbxq",
|
||||
"zerto": {
|
||||
"vpgName": "ubuntu-2404-local",
|
||||
"internalVpgName": "ubuntu-2404-local",
|
||||
"operation": "Test",
|
||||
"force": "Yes",
|
||||
"vmDisplayNames": "ubuntu-2404(1)(1)(1)",
|
||||
"hypervisorManagerIP": "192.168.50.20",
|
||||
"hypervisorManagerPort": "443",
|
||||
"outputDir": "/app/scripts-output",
|
||||
"workingDir": "/app/scripts-files"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A webhook outage **does not fail the VPG** — the script catches and exits 0.
|
||||
Comment in the file shows how to flip that to strict mode if you'd rather a
|
||||
webhook outage abort the failover.
|
||||
|
||||
## 2. The webhook-server-side scripts (receivers)
|
||||
|
||||
Two examples ship in the repo. Both read the JSON body from stdin (the
|
||||
webhook server delivers the body to the script's stdin when **JSON body to
|
||||
stdin** is ticked on the endpoint).
|
||||
|
||||
### a. Slack/Teams notification — both phases
|
||||
|
||||
[`scripts/examples/zerto-receiver-notify.ps1`](../../scripts/examples/zerto-receiver-notify.ps1)
|
||||
posts a single-line summary to a Slack or Teams Incoming Webhook URL. It
|
||||
picks an icon based on `ZertoOperation`:
|
||||
|
||||
- `Test` → 🧪 — benign, expected
|
||||
- `Failover` → 🚨 — real production event
|
||||
- `Move` → 🚚 — planned migration
|
||||
|
||||
…and highlights `ZertoForce=Yes` on the **pre** message so you can see at
|
||||
a glance whether the operation was force-flagged.
|
||||
|
||||
Set the destination via `NOTIFY_URL` env var on the webhook host, or
|
||||
hardcode at the top of the script.
|
||||
|
||||
### b. Post-recovery VM health check — post phase only
|
||||
|
||||
[`scripts/examples/zerto-receiver-vm-healthcheck.ps1`](../../scripts/examples/zerto-receiver-vm-healthcheck.ps1)
|
||||
runs only on `phase=post` for operations that bring VMs up
|
||||
(`Test`/`Failover`/`Move`/`FailoverBeforeCommit`/`FailoverDuringCommit`).
|
||||
For each name in `VmDisplayNames` it:
|
||||
|
||||
1. Strips the trailing `(1)(1)(1)` suffix Zerto adds on Test failovers, so
|
||||
DNS resolution targets the actual hostname.
|
||||
2. Pings (`Test-Connection`).
|
||||
3. Probes a configurable TCP port (`-ProbePort`, default `3389` for RDP;
|
||||
use `22` for SSH or `443` for the web tier).
|
||||
4. Writes a JSON report to
|
||||
`C:\ProgramData\WebhookServer\zerto-healthchecks\<vpg>-<op>-<utcstamp>.json`.
|
||||
5. Exits non-zero if any VM failed either probe — which surfaces in the
|
||||
webhook server's run history (and outbound callback, if configured).
|
||||
|
||||
Bump the endpoint's **Timeout (sec)** to `120` when wiring this in, since
|
||||
network probes can take a while.
|
||||
|
||||
## 3. Configure the endpoints in the GUI
|
||||
|
||||
Two endpoints. Identical except for the slug, the script, and (for the
|
||||
healthcheck) the timeout.
|
||||
|
||||
### `zerto-pre`
|
||||
|
||||
| Section | Setting | Value |
|
||||
|---|---|---|
|
||||
| Identity | Slug | `zerto-pre` |
|
||||
| Identity | Description | "Zerto pre-recovery: chat notification" |
|
||||
| Auth | Mode | **Bearer** |
|
||||
| Auth | Bearer secret | generate a 32-byte random string; reuse for `zerto-post` |
|
||||
| Allowed clients | (one per line) | the IP of the K8s node running `scripts-service` (e.g. `192.168.50.30`) |
|
||||
| Executor | Type | **Windows PowerShell** (or PowerShell 7) |
|
||||
| Executor | Script path | `C:\scripts\zerto-receiver-notify.ps1` |
|
||||
| Data passing | JSON body to stdin | ✓ |
|
||||
| Run as | Identity | **Service** |
|
||||
| Response | Mode | **Async** |
|
||||
| Response | Timeout (sec) | `30` |
|
||||
| Response | Fail on non-zero exit | unticked *(async hooks have no caller to receive a 502)* |
|
||||
|
||||
### `zerto-post`
|
||||
|
||||
Same as above, except:
|
||||
|
||||
| Setting | Value |
|
||||
|---|---|
|
||||
| Slug | `zerto-post` |
|
||||
| Description | "Zerto post-recovery: notify + VM health check" |
|
||||
| Script path | a **wrapper** that calls both receiver scripts in turn (see below) |
|
||||
| Timeout (sec) | `120` |
|
||||
|
||||
Two receivers on one endpoint is easiest with a tiny wrapper that fans
|
||||
stdin out to both scripts:
|
||||
|
||||
```powershell
|
||||
# C:\scripts\zerto-post-fanout.ps1
|
||||
$body = [Console]::In.ReadToEnd()
|
||||
$body | & 'C:\scripts\zerto-receiver-notify.ps1'
|
||||
$body | & 'C:\scripts\zerto-receiver-vm-healthcheck.ps1'
|
||||
```
|
||||
|
||||
Or run the two as separate endpoints (`zerto-post-notify` and
|
||||
`zerto-post-healthcheck`) and have the Zerto-side script POST to both —
|
||||
either pattern is fine. The fanout wrapper keeps the Zerto config simpler.
|
||||
|
||||
## 4. Wire up the bearer token
|
||||
|
||||
On the ZVMA / scripts-service side, the easiest place to put the token is
|
||||
a Kubernetes Secret mounted into the pod, but the simplest approach for
|
||||
testing is to pass it as a parameter to the Zerto-side script:
|
||||
|
||||
> VPG settings → Pre-Recovery Script → Parameters:
|
||||
> `-Phase pre -Bearer <paste-token>`
|
||||
>
|
||||
> VPG settings → Post-Recovery Script → Parameters:
|
||||
> `-Phase post -Bearer <paste-token>`
|
||||
|
||||
For production, mount a Secret at a known path in the pod and have the
|
||||
sender script read from it (`Get-Content /run/secrets/webhook-token`).
|
||||
|
||||
## 5. Test before going live
|
||||
|
||||
Run a Test failover on a non-critical VPG. Watch:
|
||||
|
||||
- **Slack/Teams**: a `:test_tube: Zerto Test - phase: pre` message arrives,
|
||||
followed ~30s–several minutes later by a `:test_tube: Zerto Test - phase:
|
||||
post` message.
|
||||
- **Webhook Server GUI** → run history: two runs for `zerto-pre` /
|
||||
`zerto-post`, both green.
|
||||
- **`C:\ProgramData\WebhookServer\zerto-healthchecks\`**: a fresh JSON
|
||||
report named `<vpg>-Test-<utcstamp>.json` containing per-VM ping and port
|
||||
probe results.
|
||||
- **ZVMA**: the VPG operation completes successfully; nothing in the
|
||||
pre/post logs blocked on the webhook.
|
||||
|
||||
## Variations
|
||||
|
||||
### Branch on Test vs. real failover in the receivers
|
||||
|
||||
The notifier already styles the message differently. To do something only
|
||||
on a real failover (e.g. update DNS), guard with:
|
||||
|
||||
```powershell
|
||||
if ($p.zerto.operation -ne 'Test') {
|
||||
# do the destructive thing
|
||||
}
|
||||
```
|
||||
|
||||
A `ZertoOperation` of `Test` means "exercise — don't touch production
|
||||
dependencies." Always check it before doing anything that mutates real
|
||||
state.
|
||||
|
||||
### Capture `ZertoForce` from pre for use in post
|
||||
|
||||
`ZertoForce` is `Yes` only during the **pre** phase when force mode is on
|
||||
and is reset to `No` by the **post** phase. If your post-side logic needs
|
||||
to know the operation was force-flagged, save it during pre (e.g. write a
|
||||
small marker to the shared `ZertoOutputDir`) and read it back during post.
|
||||
|
||||
### Per-VPG endpoints
|
||||
|
||||
For fine-grained access control or different actions per VPG, create one
|
||||
endpoint per VPG (`zerto-pre-app01`, `zerto-post-app01`, …) with its own
|
||||
bearer token. Override `-WebhookUrl` and `-Bearer` on the Zerto side per
|
||||
VPG.
|
||||
|
||||
### Audit trail
|
||||
|
||||
Every endpoint can have an outbound **Callback** URL. Configure with your
|
||||
SIEM's HTTP collector + an HMAC secret, and every run produces a JSON
|
||||
record with runId, exit code, duration, stdout, and stderr — convenient
|
||||
for compliance.
|
||||
|
||||
## Security note
|
||||
|
||||
The ZVMA `scripts-service` pod runs your scripts inside a Linux container
|
||||
with broad reach into the management cluster — anything your script does
|
||||
runs with whatever ServiceAccount that pod uses. Treat the script content
|
||||
as privileged and make sure pre/post script edit rights are restricted to
|
||||
trusted operators. If you're unfamiliar with the pod's RBAC posture, check
|
||||
`Get-ChildItem Env:` from inside the container and look at
|
||||
`/var/run/secrets/kubernetes.io/serviceaccount/` — that token is what your
|
||||
scripts (and a malicious script) can use to talk to the K8s API.
|
||||
@@ -119,7 +119,7 @@ foreach ($ref in $issRefs) {
|
||||
Write-Host ""
|
||||
|
||||
Write-Host "--- runtime context ---" -ForegroundColor Cyan
|
||||
Write-Host " identity: $([Security.Principal.WindowsIdentity]::GetCurrent().Name)"
|
||||
Write-Host " whoami: $(whoami)"
|
||||
Write-Host " USERPROFILE: $env:USERPROFILE"
|
||||
Write-Host " APPDATA: $env:APPDATA"
|
||||
Write-Host " LOCALAPPDATA: $env:LOCALAPPDATA"
|
||||
@@ -142,31 +142,6 @@ try {
|
||||
Write-Host " PS location (post): $((Get-Location).Path)"
|
||||
Write-Host " .NET cwd (post): $([System.IO.Directory]::GetCurrentDirectory())"
|
||||
|
||||
# Sanity: compile a minimal .iss right next to ours BEFORE attempting the
|
||||
# real one. Minimal has no #defines, no [Code], no [Files], no compression
|
||||
# tweak - just the absolute floor of what ISCC will accept. If THIS fails
|
||||
# under the same SYSTEM context with the same identical exit/error, the
|
||||
# problem is environmental, not in our .iss content.
|
||||
$minIss = Join-Path $issDir "min-test.iss"
|
||||
@"
|
||||
[Setup]
|
||||
AppName=MinTest
|
||||
AppVersion=1.0
|
||||
AppId={{12345678-1234-1234-1234-123456789ABC}
|
||||
DefaultDirName={pf}\MinTest
|
||||
CreateAppDir=no
|
||||
Uninstallable=no
|
||||
OutputBaseFilename=mintest
|
||||
OutputDir=$dist
|
||||
"@ | Set-Content -Path $minIss -Encoding ascii
|
||||
Write-Host ""
|
||||
Write-Host "--- bisect step 1: minimal .iss ---" -ForegroundColor Cyan
|
||||
& $iscc (Split-Path $minIss -Leaf) *>&1 | ForEach-Object { Write-Host " $_" }
|
||||
$minExit = $LASTEXITCODE
|
||||
Write-Host " minimal exit: $minExit"
|
||||
Remove-Item $minIss -ErrorAction SilentlyContinue
|
||||
Write-Host ""
|
||||
|
||||
# Bake the version into a temp .iss and override OutputDir to an absolute
|
||||
# path so nothing in the build depends on cwd resolution.
|
||||
$tempIss = Join-Path $issDir "webhook-server.gen.iss"
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Server-side receiver for the env-dump webhook. Reads the JSON body from
|
||||
stdin and writes it to a timestamped file on disk.
|
||||
|
||||
.DESCRIPTION
|
||||
Configure a webhook endpoint like this:
|
||||
Executable: powershell.exe (or pwsh.exe)
|
||||
Arguments: -NoProfile -ExecutionPolicy Bypass -File C:\path\to\save-env-vars.ps1
|
||||
Data passing: [x] Stdin JSON
|
||||
Run As: Service (or any account that can write to $OutDir)
|
||||
|
||||
Output goes to C:\ProgramData\WebhookServer\env-dumps\<host>-<utcstamp>.json
|
||||
by default; override with -OutDir.
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string] $OutDir = 'C:\ProgramData\WebhookServer\env-dumps'
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
if (-not (Test-Path $OutDir)) {
|
||||
New-Item -ItemType Directory -Path $OutDir -Force | Out-Null
|
||||
}
|
||||
|
||||
$body = [Console]::In.ReadToEnd()
|
||||
if ([string]::IsNullOrWhiteSpace($body)) {
|
||||
Write-Error 'Empty request body on stdin.'
|
||||
exit 2
|
||||
}
|
||||
|
||||
# Parse so we can pull the host name for the filename, and to fail fast on
|
||||
# malformed JSON before writing it.
|
||||
$parsed = $body | ConvertFrom-Json
|
||||
$hostName = if ($parsed.host) { $parsed.host } else { 'unknown' }
|
||||
$safeHost = ($hostName -replace '[^A-Za-z0-9_.-]', '_')
|
||||
$stamp = (Get-Date).ToUniversalTime().ToString('yyyyMMddTHHmmssZ')
|
||||
$path = Join-Path $OutDir "$safeHost-$stamp.json"
|
||||
|
||||
# Persist the original body verbatim - keeps key ordering and avoids any
|
||||
# round-trip surprises from ConvertTo-Json.
|
||||
Set-Content -Path $path -Value $body -Encoding utf8
|
||||
|
||||
Write-Host "Saved $($body.Length) bytes to $path"
|
||||
@@ -1,68 +0,0 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Collects env vars from PowerShell and bash, packages them into a single
|
||||
JSON object, and POSTs the result to a Webhook Server endpoint.
|
||||
|
||||
.DESCRIPTION
|
||||
Output JSON shape:
|
||||
{
|
||||
"host": "<computername>",
|
||||
"capturedAt":"2026-05-08T12:34:56Z",
|
||||
"pwsh": { "VAR": "value", ... },
|
||||
"bash": { "VAR": "value", ... }
|
||||
}
|
||||
|
||||
Pair this with `save-env-vars.ps1` on the server side - configure an
|
||||
endpoint with StdinJson enabled and that script as the executable.
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string] $WebhookUrl = 'http://localhost:8080/hook/env-dump',
|
||||
[string] $Bearer = '',
|
||||
[string] $BashExe = 'bash'
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# --- pwsh env vars --------------------------------------------------------
|
||||
$pwshVars = [ordered]@{}
|
||||
Get-ChildItem Env: | Sort-Object Name | ForEach-Object {
|
||||
$pwshVars[$_.Name] = $_.Value
|
||||
}
|
||||
|
||||
# --- bash env vars --------------------------------------------------------
|
||||
$bashVars = [ordered]@{}
|
||||
$bashCmd = Get-Command $BashExe -ErrorAction SilentlyContinue
|
||||
if ($null -ne $bashCmd) {
|
||||
# `env -0` separates entries with NUL so values containing newlines stay intact.
|
||||
$raw = & $bashCmd.Source -c 'env -0' 2>$null
|
||||
if ($LASTEXITCODE -eq 0 -and $raw) {
|
||||
foreach ($entry in ($raw -split "`0")) {
|
||||
if ([string]::IsNullOrEmpty($entry)) { continue }
|
||||
$eq = $entry.IndexOf('=')
|
||||
if ($eq -lt 1) { continue }
|
||||
$bashVars[$entry.Substring(0, $eq)] = $entry.Substring($eq + 1)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
Write-Warning "bash not found on PATH (looked for '$BashExe'); 'bash' section will be empty."
|
||||
}
|
||||
|
||||
# --- assemble payload -----------------------------------------------------
|
||||
$payload = [ordered]@{
|
||||
host = $env:COMPUTERNAME
|
||||
capturedAt = (Get-Date).ToUniversalTime().ToString('o')
|
||||
pwsh = $pwshVars
|
||||
bash = $bashVars
|
||||
}
|
||||
|
||||
$json = $payload | ConvertTo-Json -Depth 5 -Compress
|
||||
|
||||
# --- POST -----------------------------------------------------------------
|
||||
$headers = @{ 'Content-Type' = 'application/json' }
|
||||
if ($Bearer) { $headers['Authorization'] = "Bearer $Bearer" }
|
||||
|
||||
Write-Host "POST $WebhookUrl ($($json.Length) bytes; pwsh=$($pwshVars.Count), bash=$($bashVars.Count))"
|
||||
$response = Invoke-RestMethod -Method Post -Uri $WebhookUrl -Headers $headers -Body $json
|
||||
$response | ConvertTo-Json -Depth 5
|
||||
@@ -0,0 +1,78 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Zerto post-failover script. Fires the on-prem Webhook Server which does
|
||||
the real work (DNS updates, service health checks, notifications).
|
||||
|
||||
.DESCRIPTION
|
||||
Designed to be dropped into a Zerto VPG's post-recovery script slot. The
|
||||
Zerto Virtual Manager's PowerShell runner has a limited module set and
|
||||
runs scripts synchronously, so this script:
|
||||
|
||||
- uses curl.exe (ships with Windows 10 1803+ / Server 2019+) instead
|
||||
of any module-dependent HTTP client;
|
||||
- calls an ASYNC webhook endpoint - the server returns 202 in
|
||||
milliseconds and runs the actual work in the background;
|
||||
- returns within seconds regardless of how long the post-failover
|
||||
actions take, so Zerto's failover sequence is never blocked.
|
||||
|
||||
Wire this into your VPG via the Zerto UI:
|
||||
VPG settings -> Recovery -> Scripts -> Post-Recovery Script
|
||||
Path: C:\path\to\zerto-post-failover.ps1
|
||||
Parameters: leave empty (we read from $env:ZertoVPGName)
|
||||
|
||||
.NOTES
|
||||
Configure $WebhookUrl and either:
|
||||
- paste the bearer token directly into $Bearer (simplest, but the
|
||||
token then lives in this file), or
|
||||
- point $BearerFile at a file readable only by the ZVM service
|
||||
account (better - same threat model as Zerto's own credential
|
||||
storage).
|
||||
#>
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# ----------------------------- CONFIGURE ---------------------------------
|
||||
$WebhookUrl = 'http://webhook.contoso.local:8080/hook/post-failover'
|
||||
$Bearer = '' # paste here, or use $BearerFile
|
||||
$BearerFile = 'C:\ProgramData\Zerto\webhook-token.txt' # one line: the token
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
if (-not $Bearer -and (Test-Path $BearerFile)) {
|
||||
$Bearer = (Get-Content -LiteralPath $BearerFile -TotalCount 1).Trim()
|
||||
}
|
||||
if (-not $Bearer) {
|
||||
throw "No bearer token. Set `$Bearer in this script or write the token to $BearerFile."
|
||||
}
|
||||
|
||||
# Compose the payload. Zerto exposes a few env vars; fall back gracefully.
|
||||
$payload = @{
|
||||
operation = 'failover'
|
||||
vpg = if ($env:ZertoVPGName) { $env:ZertoVPGName } else { 'unknown' }
|
||||
timestamp = (Get-Date).ToUniversalTime().ToString('o')
|
||||
} | ConvertTo-Json -Compress
|
||||
|
||||
# curl on Windows handles long / quoted JSON better via @file than via -d "...".
|
||||
$tempBody = Join-Path $env:TEMP ("zerto-webhook-{0}.json" -f ([guid]::NewGuid()))
|
||||
$payload | Out-File -FilePath $tempBody -Encoding utf8 -NoNewline
|
||||
|
||||
try {
|
||||
Write-Host "POST $WebhookUrl (vpg=$($env:ZertoVPGName))"
|
||||
& curl.exe `
|
||||
--silent --show-error --fail-with-body `
|
||||
--max-time 10 `
|
||||
-X POST `
|
||||
-H "Authorization: Bearer $Bearer" `
|
||||
-H "Content-Type: application/json" `
|
||||
-d "@$tempBody" `
|
||||
"$WebhookUrl"
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
# curl prints its own error to stderr; surface a non-zero exit so Zerto's
|
||||
# script log records the failure but we don't block the failover.
|
||||
Write-Warning "Webhook call failed with curl exit $LASTEXITCODE; continuing."
|
||||
} else {
|
||||
Write-Host "Webhook accepted (run id is in the response above)."
|
||||
}
|
||||
}
|
||||
finally {
|
||||
Remove-Item $tempBody -ErrorAction SilentlyContinue
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Webhook-server-side receiver: posts a Slack/Teams notification when a VPG
|
||||
fires its pre or post recovery script.
|
||||
|
||||
.DESCRIPTION
|
||||
Reads the JSON body from stdin (the payload sent by zerto-zvma-send.ps1),
|
||||
builds a phase-aware message, and posts it to an Incoming Webhook URL.
|
||||
|
||||
The message highlights:
|
||||
- VPG name + operation type (Test / Failover / Move / ...)
|
||||
- Whether ZertoForce was set (only relevant pre)
|
||||
- VM display names included in the run
|
||||
- Phase (pre vs post) so you can see the bracketing in chat
|
||||
|
||||
Wire up two endpoints:
|
||||
/hook/zerto-pre -> this script with -Phase pre (pass via args)
|
||||
/hook/zerto-post -> this script with -Phase post
|
||||
|
||||
Or one endpoint per phase, each pointing at this script. The script reads
|
||||
`phase` from the JSON body, so the -Phase param is optional.
|
||||
|
||||
.NOTES
|
||||
Compatible with:
|
||||
- Slack Incoming Webhooks (posts {"text": "..."})
|
||||
- Teams legacy connector "Incoming Webhook" (same body shape)
|
||||
- Discord webhooks (use ?wait=true for body, but text is "content" not
|
||||
"text" - tweak below)
|
||||
|
||||
Endpoint config:
|
||||
ExecutorType: WindowsPowerShell or PowerShell 7
|
||||
ScriptPath: C:\scripts\zerto-receiver-notify.ps1
|
||||
DataPassing: [x] Stdin JSON
|
||||
ResponseMode: async (we don't need to block the VPG on a chat post)
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string] $NotifyUrl = $env:NOTIFY_URL # set on the Webhook Server host, or hardcode below
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
if (-not $NotifyUrl) {
|
||||
# Fall back to a hardcoded URL if NOTIFY_URL env var isn't set.
|
||||
# Replace with your Slack/Teams Incoming Webhook URL.
|
||||
$NotifyUrl = 'https://hooks.slack.com/services/REPLACE/ME/HERE'
|
||||
}
|
||||
|
||||
$body = [Console]::In.ReadToEnd()
|
||||
if ([string]::IsNullOrWhiteSpace($body)) {
|
||||
Write-Error 'Empty stdin - expected JSON body from the webhook server.'
|
||||
exit 2
|
||||
}
|
||||
$p = $body | ConvertFrom-Json
|
||||
|
||||
$z = $p.zerto
|
||||
$phase = if ($p.phase) { $p.phase } else { 'unknown' }
|
||||
$op = if ($z.operation) { $z.operation } else { 'unknown' }
|
||||
|
||||
# Pick an icon based on operation. Test is benign; Failover/Move are real.
|
||||
$icon = switch ($op) {
|
||||
'Test' { ':test_tube:' }
|
||||
'Failover' { ':rotating_light:' }
|
||||
'Move' { ':truck:' }
|
||||
default { ':information_source:' }
|
||||
}
|
||||
|
||||
$forceTag = if ($phase -eq 'pre' -and $z.force -eq 'Yes') { ' *(FORCE)*' } else { '' }
|
||||
|
||||
$lines = @(
|
||||
"$icon *Zerto $op* - phase: ``$phase``$forceTag"
|
||||
"VPG: ``$($z.vpgName)``"
|
||||
"VMs: ``$($z.vmDisplayNames)``"
|
||||
"Hypervisor mgr: ``$($z.hypervisorManagerIP):$($z.hypervisorManagerPort)``"
|
||||
"Captured: $($p.capturedAt) (from $($p.host))"
|
||||
)
|
||||
$text = $lines -join "`n"
|
||||
|
||||
$payload = @{ text = $text } | ConvertTo-Json -Compress
|
||||
|
||||
try {
|
||||
Invoke-RestMethod -Method Post -Uri $NotifyUrl `
|
||||
-ContentType 'application/json' -Body $payload -TimeoutSec 10 | Out-Null
|
||||
Write-Host "[$phase] notified $op for VPG '$($z.vpgName)'"
|
||||
}
|
||||
catch {
|
||||
Write-Error "Notification post failed: $($_.Exception.Message)"
|
||||
exit 1
|
||||
}
|
||||
@@ -1,140 +0,0 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Webhook-server-side receiver: post-failover VM health check. Pings each
|
||||
VM in the VPG and probes a configurable TCP port; writes a per-run
|
||||
report to disk.
|
||||
|
||||
.DESCRIPTION
|
||||
Intended for the POST-recovery webhook only - on a Test or real Failover,
|
||||
once the VMs are powered on at the recovery site, we can spot-check that
|
||||
they responded to ICMP and that a known port is listening (RDP, SSH,
|
||||
HTTP, etc).
|
||||
|
||||
Skips itself entirely on the pre-recovery phase (nothing's running yet)
|
||||
and on $z.operation values that don't bring VMs up.
|
||||
|
||||
Wire up one endpoint:
|
||||
/hook/zerto-post -> this script
|
||||
DataPassing: [x] Stdin JSON
|
||||
ResponseMode: async
|
||||
|
||||
.NOTES
|
||||
VmDisplayNames is a comma-separated list for multi-VM VPGs; some Zerto
|
||||
versions wrap each name in parentheses (e.g. "vm1(1)(1)(1)") to disambig
|
||||
after Test failover. We strip the trailing parenthesised suffixes when
|
||||
resolving DNS so the recovered hostname is what we ping.
|
||||
|
||||
Endpoint config:
|
||||
ExecutorType: WindowsPowerShell or PowerShell 7
|
||||
ScriptPath: C:\scripts\zerto-receiver-vm-healthcheck.ps1
|
||||
DataPassing: [x] Stdin JSON
|
||||
ResponseMode: async
|
||||
TimeoutSeconds: 120 (this script does network I/O - bump from default)
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[int] $ProbePort = 3389, # RDP. Use 22 for Linux, 80/443 for web tier.
|
||||
[int] $PingTimeout = 2000, # ms
|
||||
[string] $ReportDir = 'C:\ProgramData\WebhookServer\zerto-healthchecks'
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# --- read + parse payload -------------------------------------------------
|
||||
$body = [Console]::In.ReadToEnd()
|
||||
if ([string]::IsNullOrWhiteSpace($body)) {
|
||||
Write-Error 'Empty stdin.'
|
||||
exit 2
|
||||
}
|
||||
$p = $body | ConvertFrom-Json
|
||||
|
||||
$z = $p.zerto
|
||||
$phase = $p.phase
|
||||
$op = $z.operation
|
||||
|
||||
# Skip if this isn't a post-phase run for an op that powers VMs on.
|
||||
if ($phase -ne 'post') {
|
||||
Write-Host "Phase '$phase' - nothing to check yet, skipping."
|
||||
exit 0
|
||||
}
|
||||
if ($op -notin @('Test','Failover','Move','FailoverBeforeCommit','FailoverDuringCommit')) {
|
||||
Write-Host "Operation '$op' doesn't bring VMs up; skipping."
|
||||
exit 0
|
||||
}
|
||||
|
||||
# --- parse VM list --------------------------------------------------------
|
||||
function Strip-ZertoSuffix {
|
||||
param([string] $name)
|
||||
# "ubuntu-2404(1)(1)(1)" -> "ubuntu-2404"
|
||||
return ($name -replace '(\([^)]*\))+\s*$','').Trim()
|
||||
}
|
||||
|
||||
$rawNames = ($z.vmDisplayNames -split '[,;]') | ForEach-Object { $_.Trim() } |
|
||||
Where-Object { $_ }
|
||||
if (-not $rawNames) {
|
||||
Write-Warning 'No VM display names in payload - nothing to check.'
|
||||
exit 0
|
||||
}
|
||||
|
||||
# --- run checks -----------------------------------------------------------
|
||||
$results = foreach ($raw in $rawNames) {
|
||||
$clean = Strip-ZertoSuffix $raw
|
||||
$pingOk = $false
|
||||
$portOk = $false
|
||||
$err = $null
|
||||
|
||||
try {
|
||||
$pingOk = (Test-Connection -ComputerName $clean -Count 1 -Quiet `
|
||||
-TimeoutSeconds ([math]::Max(1, [int]($PingTimeout / 1000))) `
|
||||
-ErrorAction Stop)
|
||||
} catch { $err = "ping: $($_.Exception.Message)" }
|
||||
|
||||
try {
|
||||
$portOk = (Test-NetConnection -ComputerName $clean -Port $ProbePort `
|
||||
-InformationLevel Quiet -WarningAction SilentlyContinue)
|
||||
} catch { $err = ($err, "port: $($_.Exception.Message)") -ne $null -join '; ' }
|
||||
|
||||
[pscustomobject]@{
|
||||
DisplayName = $raw
|
||||
Resolved = $clean
|
||||
PingOk = $pingOk
|
||||
PortOk = $portOk
|
||||
ProbePort = $ProbePort
|
||||
Error = $err
|
||||
}
|
||||
}
|
||||
|
||||
# --- write report ---------------------------------------------------------
|
||||
if (-not (Test-Path $ReportDir)) {
|
||||
New-Item -ItemType Directory -Path $ReportDir -Force | Out-Null
|
||||
}
|
||||
|
||||
$safeVpg = ($z.vpgName -replace '[^A-Za-z0-9_.-]','_')
|
||||
$stamp = (Get-Date).ToUniversalTime().ToString('yyyyMMddTHHmmssZ')
|
||||
$file = Join-Path $ReportDir "$safeVpg-$op-$stamp.json"
|
||||
|
||||
$report = [ordered]@{
|
||||
vpgName = $z.vpgName
|
||||
operation = $op
|
||||
phase = $phase
|
||||
capturedAt = $p.capturedAt
|
||||
completedAt = (Get-Date).ToUniversalTime().ToString('o')
|
||||
probePort = $ProbePort
|
||||
vms = $results
|
||||
summary = @{
|
||||
total = $results.Count
|
||||
pingFailures = ($results | Where-Object { -not $_.PingOk }).Count
|
||||
portFailures = ($results | Where-Object { -not $_.PortOk }).Count
|
||||
}
|
||||
}
|
||||
$report | ConvertTo-Json -Depth 5 | Set-Content -Path $file -Encoding utf8
|
||||
|
||||
# Console output goes back via the webhook callback (if configured) so the
|
||||
# Zerto-side script log shows a quick summary even though the call is async.
|
||||
$bad = $report.summary.pingFailures + $report.summary.portFailures
|
||||
Write-Host "[$op/$phase] $($z.vpgName): $($results.Count) VM(s), $bad issue(s). Report: $file"
|
||||
|
||||
# Exit non-zero if anything failed, so the webhook server's failOnNonZeroExit
|
||||
# turns this into a 502 for the caller (and shows up in the run history).
|
||||
if ($bad -gt 0) { exit 1 }
|
||||
@@ -1,74 +0,0 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Zerto pre/post script (ZVMA / Linux scripts-service edition). Reads the
|
||||
Zerto-injected environment variables and POSTs them to a Webhook Server
|
||||
endpoint as a structured JSON payload.
|
||||
|
||||
.DESCRIPTION
|
||||
Drop into a VPG's Recovery Scripts in the ZVM UI:
|
||||
VPG settings -> Recovery -> Scripts -> Pre / Post Recovery Script
|
||||
Path: /app/scripts-files/zerto-zvma-send.ps1
|
||||
Parameters: -Phase pre (or -Phase post on the post-recovery slot)
|
||||
|
||||
Configure $WebhookUrl + $Bearer (or use the -WebhookUrl / -Bearer params
|
||||
so one script file can serve multiple VPGs / endpoints).
|
||||
|
||||
Async by default - the call returns 202 in milliseconds and the actual
|
||||
work runs in the webhook server's background, so the VPG sequence is
|
||||
never blocked by slow downstream actions (DNS, notifications, etc.).
|
||||
|
||||
.NOTES
|
||||
The scripts-service container has pwsh 7 and curl available. This script
|
||||
uses Invoke-RestMethod to keep things native to PowerShell.
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory)]
|
||||
[ValidateSet('pre', 'post')]
|
||||
[string] $Phase,
|
||||
|
||||
[string] $WebhookUrl = 'http://192.168.50.250:8080/hook/zerto-{phase}',
|
||||
[string] $Bearer = '',
|
||||
[int] $TimeoutSec = 10
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Resolve {phase} placeholder so one URL template can route to /hook/zerto-pre
|
||||
# and /hook/zerto-post. Plain URLs without the token work too.
|
||||
$url = $WebhookUrl.Replace('{phase}', $Phase)
|
||||
|
||||
$payload = [ordered]@{
|
||||
phase = $Phase
|
||||
capturedAt = (Get-Date).ToUniversalTime().ToString('o')
|
||||
host = $env:HOSTNAME # scripts-service pod name
|
||||
zerto = [ordered]@{
|
||||
vpgName = $env:ZertoVPGName
|
||||
internalVpgName = $env:ZertoInternalVpgName
|
||||
operation = $env:ZertoOperation # Test / Failover / Move / ...
|
||||
force = $env:ZertoForce # only meaningful pre
|
||||
vmDisplayNames = $env:VmDisplayNames # comma-separated for multi-VM VPGs
|
||||
hypervisorManagerIP = $env:ZertoHypervisorManagerIP
|
||||
hypervisorManagerPort = $env:ZertoHypervisorManagerPort
|
||||
outputDir = $env:ZertoOutputDir
|
||||
workingDir = $env:ZertoWorkingDir
|
||||
}
|
||||
}
|
||||
|
||||
$body = $payload | ConvertTo-Json -Depth 4 -Compress
|
||||
|
||||
$headers = @{ 'Content-Type' = 'application/json' }
|
||||
if ($Bearer) { $headers['Authorization'] = "Bearer $Bearer" }
|
||||
|
||||
try {
|
||||
$resp = Invoke-RestMethod -Method Post -Uri $url -Headers $headers `
|
||||
-Body $body -TimeoutSec $TimeoutSec
|
||||
Write-Host "[$Phase] webhook accepted: $($resp | ConvertTo-Json -Compress)"
|
||||
}
|
||||
catch {
|
||||
# Pre/post failures should not block the VPG operation. Log loudly and exit 0
|
||||
# so Zerto's recovery sequence continues. Flip to `exit 1` if you want a
|
||||
# webhook outage to fail the failover.
|
||||
Write-Warning "[$Phase] webhook call failed: $($_.Exception.Message)"
|
||||
}
|
||||
@@ -65,7 +65,7 @@ $mapping.Add('runas-modes.md', 'Run-As-Modes')
|
||||
$mapping.Add('service-account-and-ad.md', 'Service-Account-and-AD')
|
||||
$mapping.Add('network-and-security.md', 'Network-and-Security')
|
||||
$mapping.Add('troubleshooting.md', 'Troubleshooting')
|
||||
$mapping.Add('recipes/zerto-zvma-pre-post.md', 'Recipe-Zerto-ZVMA')
|
||||
$mapping.Add('recipes/zerto-pre-post-scripts.md', 'Recipe-Zerto-Failover')
|
||||
$mapping.Add('recipes/github-style-hmac.md', 'Recipe-GitHub-HMAC')
|
||||
$mapping.Add('recipes/ui-on-desktop.md', 'Recipe-UI-on-Desktop')
|
||||
|
||||
@@ -94,7 +94,7 @@ function New-Sidebar() {
|
||||
}
|
||||
$lines += ""
|
||||
$lines += "## Recipes"
|
||||
foreach ($key in @('recipes/zerto-zvma-pre-post.md','recipes/github-style-hmac.md','recipes/ui-on-desktop.md')) {
|
||||
foreach ($key in @('recipes/zerto-pre-post-scripts.md','recipes/github-style-hmac.md','recipes/ui-on-desktop.md')) {
|
||||
$slug = $mapping[$key]
|
||||
$lines += "- [$($slug -replace '^Recipe-' -replace '-', ' ')]($slug)"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user