So you have a Cluster of Citrix XenServers and you want to upgrade them to a new version, for example to go from XenServer 6.0.2 to XenServer 6.2, or simply apply the latest Hotfixes. As this is a cluster that is being managed by CloudStack it is not as simple as using the Rolling Pool Upgrade feature in XenCenter – in fact this is the LAST thing you want to do, and WILL result in a broken Cluster.
This article walks you through the steps required to perform the upgrade, but as always you must test this yourself in your own test environment before attempting on a production system.
We need to change the default behaviour of CloudStack with respect to how it manages XenServer Clusters before continuing. Edit /etc/cloudstack/management/environment.properties and add the following line:
# vi /etc/cloudstack/management/environment.properties
Add > manage.xenserver.pool.master=false
Now restart the CloudStack Management Service
# service cloudstack-management restart
Repeat for all CloudStack Management servers
It is vital that you upgrade the XenServer Pool Master first before any of the Slaves. To do so you need to empty the Pool Master of all CloudStack VMs, and you do this by putting the Host into Maintenance Mode within CloudStack to trigger a live migration of all VMs to alternate Hosts (do not place the Host into Maintenance Mode using XenCenter as this will cause a new Master to be elected and we do not want that).
Next you need to ‘Unmanage’ the Cluster, as this prevents users from being able to interact (stop/start) VMs you will need to arrange a ‘Maintenance Window’ but only long enough to update the Pool Master. All Customer VMs will continue to run during the upgrade process unless you are using Local Storage, in which case VMs on the Hosts being upgraded will have to shut down. After ‘Unmanaging’ the Cluster, all Hosts will go into a ‘Disconnected’ state, this is expected and is not a cause for concern.
Now you can upgrade your Pool Master, either upgrading to a newer version, or simply applying XenServer Hotfixes as required. Once the Pool Master has been fully upgraded re-manage the Cluster and then wait for all of the Hosts in the Cluster to come back online within CloudStack.
Monitor the status of your NFS Storage via XenCenter and wait for all Volumes to reconnect on the upgraded Host. Once storage has reconnected and all Hosts are back online, take the Pool Master you just upgraded out of CloudStack Maintenance Mode.
Edit /etc/cloudstack/management/environment.properties and remove the following line which you added earlier:
# vi /etc/cloudstack/management/environment.properties
Delete > manage.xenserver.pool.master=false
Now restart the CloudStack Management Service
# service cloudstack-management restart
Repeat for all CloudStack Management servers
You can now upgrade each Slave by simply placing it into Maintenance Mode in CloudStack, apply the upgrade / Hotfixes and when completed, bringing out of Maintenance Mode before starting on the next Host.
About the Author
Geoff Higginbottom is CTO of ShapeBlue, the strategic cloud consultancy and an Apache CloudStack Committer. Geoff spends most of his time designing private & public cloud infrastructures for telco’s, ISP’s and enterprises based on CloudStack.
Steve is ShapeBlue’s COO and is responsible for all day-to-day administrative and operational functions of the business, including the consulting, programme management, and support functions.
Involved with CloudStack since 2012, Steve has led several large customer engagements including a number of major public and private cloud deployments; co-ordinated and developed worldwide teams and helped implement and deliver an enterprise grade support product.
Prior to ShapeBlue, Steve held senior technical, project and account management roles in enterprise IT outsourcing companies where he gained domain experience in the finance, professional services and defence markets.
Away from work, Steve is a father, keen guitarist, snowboarder and singer (not necessarily in that order).
Away from work, Steve is a music lover and semi-professional musician. Although he doesn’t speak at many technology conferences, he can sometimes be heard providing the evening entertainment.
