# Cluster layout upgrade

Morpheus 9.0.0 includes a new cluster layout for HVM Clusters, version 1.3, which succeeds version 1.2. The primary difference in cluster layout 1.3 is the replacement of Pacemaker as the resource manager for GFS2 datastores (now known as HPE Clustered Datastores). Rather than Pacemaker observing the reported state of Corosync and making fencing decisions, the Morpheus Agent becomes the high-availability control plane. This results in even greater reliability for HPE Clustered Datastores in HVM Clusters.

Morpheus 9.0.0+ also includes a one-click upgrade to take existing 1.2 HVM Clusters to cluster layout version 1.3. The rest of this section discusses pre-upgrade checks and describes the automated upgrade flow in detail to provide an understanding of how the upgrade is handled.

Pre-upgrade Requirements

- All GFS2 clusters must be in a healthy state and mounted on each host
- The Pacemaker cluster must be in a clean state
- Sufficient capacity must exist for workloads to migrate
- All backups are completed
- There is a maintenance window available for the upgrade to be performed

Running the Upgrade

Navigate to the host detail page (Infrastructure > Clusters > select cluster). Click Actions, then Update. From the Update drop-down, select "HVM Cluster 1.3 Upgrade." Some information about the upgrade process is shown. To begin, click Execute.

Upgrade Notes

The cluster layout upgrade occurs in two phases. If any portion of the first phase fails, the cluster is rolled back to its initial state. Once the first phase is successful, Pacemaker decommissioning begins and rollback is no longer possible.

**Phase 1**

- Agent upgrade on HVM Hosts
- Host is placed into maintenance mode
- Workloads are evacuated to other Hosts
- Pacemaker is unmanaged (but not yet stopped)
- Maintenance mode is exited
- This process continues to the next HVM Host

NOTE

If any part of this phase fails, the process is rolled back and the HVM Cluster is unchanged (except for the Host agent upgrade). If this phase is successful, Phase 2 will begin, at which point it is no longer possible to roll back to using Pacemaker or HVM cluster layouts below version 1.3.

**Phase 2**

- Pacemaker is stopped
- Pacemaker is masked
- HA ownership is transferred to the Morpheus Agent
- Corosync is revalidated

General Validation

Following the upgrade, administrators may wish to validate the cluster upgrade was successful and all components are in a happy state. If desired, validate the following:

- Check Morpheus Agent Health:

  ```
  ```
  morpheus-node-ctl status  
  systemctl status morpheus-morphd  
  morpheus-node-ctl version
  ```
  ```
- Validate Corosync

  ```
  ```
  corosync-quorumtool
  ```
  ```
- Verify DLM reports no fenced nodes

  ```
  ```
  dlm_tool ls  
  dlm_tool ls -n
  ```
  ```
- Check the GFS2 mount point

  ```
  ```
  mount | grep gfs2
  ```
  ```
- Verify if the pools are still active

  ```
  ```
  virsh pool-list --all  
  virsh pool-info {pool name}
  ```
  ```
