Apache CloudStack 4.18 is the latest release of the cloud management platform from the Apache Software Foundation and is a result of months of work from the development community. Apache CloudStack 4.18 is an LTS (Long Term Support) release so will be maintained for a period of 18 months after release.

As always, the release contains a myriad of small improvements and bug fixes but here we focus on the major new functionality in the release.

Several new features have been introduced to significantly enhance the platform’s capabilities. These include Edge Zones (which provide lightweight zones); Tungsten Fabric integration (offering support for open-source SDN without the requirement of a virtual router); native support for autoscaling (enabling the dynamic adjustment of the number of running instances based on load); Managed Userdata (allowing users to register and manage their own UserData scripts as a CloudStack resource); and Two-Factor Authentication Framework / T-OTP which reduce the risk of compromise of platform user accounts, improving the overall security of the platform.

All in all, these exciting new features make Apache CloudStack 4.18 an even more robust and flexible platform, offering enhanced capabilities to meet the evolving needs of modern businesses.

Edge Zones

Edge computing is a distributed computing paradigm that brings computation and data storage closer to the data source and end consumer. The presence of computing capabilities at the edge of the network reduces response times for applications while reducing connectivity costs.

Typical Edge environments are constrained in space, power and bandwidth making the deployment of full-fledged CloudStack zones, with multiple hypervisor hosts, storage arrays, and switching infrastructure, either cost prohibitive or technically unfeasible.

This new feature enables operators to leverage CloudStack in deploying lightweight Zones in edge locations. This is achieved by reducing the complexity of the Zone construct, with resources like shared storage, external switching, and System VMs not being required for Edge Zones.

SDN Integration – Tungsten Fabric

Tungsten Fabric is an open-source SDN project initially developed by Juniper and previously known as OpenContrail SDN. Tungsten Fabric solves tooling complexity with the simplicity of only one networking and security tool. Today, Tungsten Fabric is maintained by The Linux Foundation and is designed to support any cloud anywhere.

From 4.18, Apache CloudStack users can leverage Tungsten Fabric as a Network, adding this widely adopted SDN fabric to fill feature gaps in CloudStack, enhancing security and capabilities. Tungsten uses DPDK to offload network processing for higher performance whilst managing the entire networking lifecycle.

This integration enables CloudStack users to configure Static NAT, Port Forwarding, Firewall and Load Balancing without the overhead of a separate Virtual Router. In Apache CloudStack, 4.18 Tungsten Fabric is supported in KVM deployments.

Autoscaling

Autoscaling is a method used in cloud computing to dynamically adjust the number of running Instances based on load. Application load can change drastically throughout a given period and, as compute resources are limited and cost money even when they are idle, the idea is to increase the amount of available compute resources in order to withstand peaks of high activity and reduce it when activity subsides.

Typically, engineers must design oversized infrastructure based on historical peak application requirements, which drives up costs in computing resources. Given the inherent difficulties in predicting workload demands and the relatively large amount of time necessary to allocate more resources, the ability to autoscale workloads based on usage requirements dramatically decreases the costs associated with running idle infrastructure and guarantees business continuity in case of busy periods or burst requests.

Apache CloudStack has supported Autoscaling for some time, but it has previously relied on external devices to provide the underlying metrics. From 4.18, Autoscaling is supported natively with no requirement for external devices.

From Apache CloudStack 4.18, users can configure Autoscaling to dynamically scale up and down the number of Instances when the defined trigger conditions are met. When creating an Autoscale VM Group, users define the autoscaling limits based on the memory or CPU average, the public network received or transmitted data rate, and load balancer average connections. When certain defined thresholds are reached, CloudStack deploys new instances which are automatically associated with the relevant load balancing rule to seamlessly meet demand. Likewise, when peaks are below defined thresholds, CloudStack removes instances to save resources.

Here is the internal workflow of the decision-making process that triggers autoscaling in CloudStack:

Managed User Data

Introduced in Apache CloudStack 4.18, Managed UserData allows users to register and manage their own UserData scripts as a CloudStack resource. It can be associated with a Template or ISO image or at deployment time. The UserData scripts are stored on CloudStack, allowing users to leverage them whenever necessary, thus cutting overhead and increasing deployment agility.

Also, when registering a new UserData script, users can now define custom parameters. These parameters are used as variables in UserData scripts. When deploying an Instance and upon selecting the required UserData, users can pass values to these custom parameters. The keys/values referring to the UserData parameter map must be considered if using the API.

Two-Factor Authentication Framework

Two-Factor Authentication (2FA), is an authentication method where an user is authorised to login into system only after successfully presenting two pieces of data, or factors, something that the user knows plus something which the user possesses. In this case, the password being something that only the user knows, and the keys generated by an algorithm, like OTP for example, being something that only the user possesses.

When considering compliance and security regarding access to sensitive and critical computing resources, 2FA is an invaluable tool which ensures integrity when accessing the CloudStack portal. In this regard, 2FA guarantees that whoever is accessing the portal is really the authorized user.

2FA framework is part of CloudStack 4.18 and, when enabled, increases the level of security when users access the CloudStack UI. Operators can enable this feature globally or by domain or for specific users, applying compliance policies for access to the CloudStack UI. With the 2FA framework, plugins for other 2-factor authentication mechanisms can be created in the future to better adhere to companies’ or governmental institution’s security and compliance policies. There is also the option to make the use of 2FA authentication for users mandatory globally or per domain.

Support for Time-based OTP (TOTP) Authenticator

A one-time password (OTP) is a password that is valid only for one login session. OTP avoids several shortcomings with static password-based authentication. When OTP is associated with a static password it is a form of 2FA (two-factor authentication).

Time base OTP (TOTP) Authenticator is a temporary OTP generated by an algorithm that uses the current time and is impossible to be guessed, thus reducing the risk of accounts being compromised and eliminating the possibility for users to share credentials unduly. Any TOTP-based authenticators like Google, Microsoft and other authenticators can be used with CloudStack.

This new feature present in Apache CloudStack 4.18 provides a Time-based OTP (TOTP) Authenticator plugin, supporting random OTP for each transaction when authenticating a cloud user in the UI. The CloudStack administrator can enable the TOTP Authenticator plugin in Global Settings on a per-user basis, globally, or on a per-user account or domain basis.

Volume Encryption

Encrypting disk volumes protects information by converting raw data into unreadable code that cannot be deciphered. With full disk encryption (FDE), all the data in the disk volume is encrypted, differently from software encryption performed by software running inside the operating system, where only certain directories or files are encrypted.

Regarding the integrity and security of the data contained on an operating system’s disks, encryption at the hypervisor level proves to be the most efficient and secure option, ensuring that the data is completely inaccessible to third parties. While most volume encryption software implements different, incompatible, and undocumented formats, the Linux Unified Key Setup (LUKS) provides a vendor-independent platform, facilitating compatibility and interoperability between different hypervisors and systems, also ensuring the implementation of password management in a secure and documented manner.

Volume Encryption is part of CloudStack 4.18, supporting initially the KVM hypervisor. Both root and data volumes can be encrypted, using all supported storage types – currently, NFS, local storage, shared mount-point and ScaleIO.

Ceph Multi-Monitor support

Ceph is an open-source, software-defined storage solution which provides a highly scalable architecture with support for object, block and file storage. Ceph decouples data from physical storage hardware by utilizing software abstraction layers, providing both scalability and fault management capabilities. This makes Ceph ideal for use with Apache CloudStack as it can efficiently address large data storage needs when using the KVM hypervisor.

Prior to this improvement, Apache CloudStack supported only one Ceph monitor, which could lead to Instances getting “stuck” when this single Ceph monitor had any downtime. An alternative option was to use round-robin DNS-based balancing to achieve monitor redundancy, which although functional, added another point of failure in addition to increasing management complexity.

This new improvement is part of Apache CloudStack 4.18 and provides the option to add Ceph storage with multiple monitors, including IPv6 support. With this feature, if the primary monitor is unavailable, CloudStack connects to other monitors, therefore ensuring that Instances can continue to run seamlessly.

API-Driven Console Access

The Apache CloudStack Console Proxy (CPVM) is a type of system virtual machine that presents an Instance’s console view via the web UI (connecting to the VNC port made available through the hypervisor). It provides a proxy between the user’s browser and the Instance’s console, allowing for interaction with the user’s input devices (i.e. keyboard and mouse) and the Instance’s operating system through an encapsulated VNC connection.

Prior to Apache CloudStack 4.18, the instance’s console service wasn’t designed to integrate with third-party services in mind, leading to many operators resorting to a hack, extracting the redirect URL from the HTML response sent by the CloudStack management server to connect to the Instance.

Now, Apache CloudStack 4.18 introduces a new way to access an Instance’s console, extending it to easily integrate CloudStack with other services. The API response returns the console URL and a one-time token to avoid replay attacks so that the user can securely connect to the Instance’s console. The generated URL will be only used once per session, ensuring that other users intercepting the generated URL are not able to access the Instance’s console once the token has been authenticated on a session.

Console access security improvements

In addition to improvements in console access, Apache CloudStack 4.18 makes it possible for the cloud operator to allow for secure WebSocket traffic between the CPVM and end users. By default, unencrypted TCP port 8080 is used, and when enabled, TCP port 8443 is used instead (for SSL enabled CloudStack environments).

Visual Resource Metrics

The visualization of historical consumption data through a visual graph panel helps cloud users and operators to easier identify bottlenecks or over-supplying of the cloud resources, allowing resource owners to estimate and analyse the performance of their resources. In this respect, CloudStack already allows retrieving metrics data for various resources in general and Instances and their Volumes.

With Apache CloudStack 4.18, metrics for Instances and their Volumes can be visualized in the UI in multiple interactive graphs for CPU, memory, disk IO and network historical statistics. Some changes have been made in CloudStack’s metrics framework to persist historical metrics data for Instances and their Volumes in the CloudStack database. UI would allow filtering metrics data for the desired time period. The CloudStack management server collects these data from the underlying hypervisor and the availability of metrics for a particular resource will depend on the existing hypervisor metrics support in CloudStack. UI would allow filtering metrics data for the desired time period.

New global settings UI

With CloudStack 4.18, the Global Settings page has been revamped. The settings are now organized logically, and hierarchically, with all configurations with a parent displayed in a tree structure. This makes navigating and searching settings simpler and more intuitive. With this new view, users now have an enhanced experience with a list of predefined dropdowns, sliders for percentage values and a provision to define custom values for applicable settings. Users comfortable with the legacy view can still use it under the ‘All Settings’ Tab.

Configurable MTU for VR

CloudStack 4.18 introduces the capability for an end-user to set non-standard MTU sizes for the CloudStack virtual router’s private and public interfaces, thus allowing for the user of either jumbo frames (MTU larger than 1500 bytes) or limiting frames to less than 1500 bytes to avoid fragmentation.

Adaptive Affinity Groups

In hypervisor technologies affinity groups are utilized to either group Virtual Machine Instances together, ensuring they run from the same hypervisor host or to ensure that they do not run in the same hypervisor host (anti-affinity). Previously, CloudStack affinity groups would enforce affinity and anti-affinity groups in a strict fashion, with the deployment of new Instances failing if no appropriate hypervisor resources were present.

Apache CloudStack 4.18 introduces the concept of strict and non-strict affinity groups. Strict affinity groups work as previously, while non-strict affinity groups mean that the Instance deployer will make a best effort to respect the affinity (or non-affinity) of Instances but will not cause the deployment to fail in case the appropriate hypervisor resources are not present.

Custom DNS Servers for Networks

Apache CloudStack 4.18 introduces the capability to specify network specific DNS, this selection overrides the Zone specific DNS servers and allows for more flexibility regarding which DNS servers are used by CloudStack Instances.

Improved guest OS support framework

Prior to Apache CloudStack 4.18, users needed to use the API to change the type of Guest Operating Systems available for Instances. This new feature allows for users to manage the Guest OS type directly in the UI, including adding new ones based on their needs. This feature maintains the consistency between the registered Guest OS in CloudStack and the underlying hypervisors.

Support for Enterprise Linux 9

From Apache CloudStack 4.18, users can register Templates and create Instances for all EL9 derivations like Alma Linux 9, Rocky Linux 9, Centos 9 and RedHat 9.