Introduction
ShapeBlue have been working on a new feature for Apache Cloudstack 4.11.1 that will allow users to bypass secondary storage with KVM. The feature introduces a new way to use templates and ISOs, allowing administrators to use them without being cached on secondary storage. Using this approach Cloudstack administrators will not have to worry about massive secondary storage, since it will be simple bypassed, there won’t be any template sitting there waiting. As well it’s bypassing the SSVM since the download task will not be carried on by the SSVM, but the KVM agent itself. This will enable administrators not to spare resources for SSVM, but to use them for commercial purposes. The usual process of virtual machine deployment will stay as before.
Overview
This feature adds a new field in the vm_template table which is called ‘direct_download’. The field will determine if template needs to be downloaded by SSVM (in case of ‘0’), or directly on the host when deploying the VM (in case of ‘1’). CloudStack administrators will have the option to set this field through the UI or API call as described in the following examples:
From the UI:
From Cloudmonkey:
register template zoneid=3e80c1e6-0710-4018-9062-194d6b3bab97 ostypeid=6f232c75-5370-11e8-afb9-06354a01076f hypervisor=KVM url=http://dl.openvm.eu/cloudstack/macchinina/x86_64/macchinina-kvm.qcow2.bz2 format=QCOW2 displaytext=TestMachina name=TestMachina directdownload=true
The same feature applies to ISOs as well – they don’t need to be cached on secondary storage but can be directly downloaded by the host. CloudStack admins have this option available on the API call when registering ISOs and through the UI form as well.
Whenever a VM deployment is started the template will be downloaded on primary storage. The feature actually checks if the template/ISO has been already downloaded on the pool, checking template_spool_ref table. If there’s an entry on the table matching its pool ID and the template ID, then it won’t be downloaded again. The same action applies if the running VM requires the template again (eg. when reinstalling ). Please note that due to the direct download nature of this feature, the uniqueness of the templates across primary storage pools is the responsibility of the CloudStack operator. CloudStack itself can’t detect if the files in a template download URL have changed or not.
Metalinks are also supported for this feature, and administrators can be more flexible in terms of managing their templates as they can set priorities and location preferences in the metalink file. Metalinks are effectively xml that provides URLs for downloading files. The duplicate download locations provide reliability in case one method fails. Some clients also achieve faster download speeds by allowing different chunks/segments of each file to be downloaded from multiple resources at the same time. Please see the following example:
As the example shows, CloudStack administrators can set location preference and priority, which will be considered upon VM deployment. The deployment logic itself introduces a retry mechanism in 2 cases of failures: VM deployment failure and template download failure.
VM deployment retry logic: this will initiate the deployment on a suitable host and will try to deploy it (which includes the template download itself). If the deployment fails for some reason it will retry the deployment on another suitable host.
Template download retry logic: this is part of the VM deployment and will try to download the template/iso directly by the host. If it fails for some reason (e.g. URL not available) it will iterate through the provided list of priority and location. Once download is completed it will execute the checksum validation (if provided), if that one fails it will download it again, until it has made three attempts. If all three attempts unsuccessful it will return a deployment failure and go back to VM Deployment logic.
Please see the following simplified picture of the deployment logic:
Since the download task has been delegated to the KVM agent instead of SSVM, this feature will be available only for KVM templates.
About the author
Boris Stoyanov is Software Engineer in testing at ShapeBlue, The Cloud Specialists. Bobby spends his time testing features for the Apache CloudStack Community and for our ShapeBlue clients.
Boris is a QA Engineer who found his ikigai in Apache CloudStack. Since 2016, he has been involved in testing CloudStack working for ShapeBlue. Boris is also usually driving forward a lot of community initiatives like the GSoC. In his free time, he enjoys hiking, boxing and other sports. Boris is a father of two young girls and spends most of his time on family activities.
