Apache CloudStack 4.19 is the latest release of the cloud management platform from the Apache Software Foundation and is a result of months of work from the development community. Apache CloudStack 4.19 is an LTS (Long Term Support) release that will be maintained for 18 months after release.

As always, the release contains a myriad of small improvements and bug fixes but here we focus on the major new functionality in the release.

Several new features have been introduced to enhance the platform’s capabilities significantly. These include:

• VNF Appliances Support

Enables cloud operators to deploy their own VNF devices to extend their network capabilities with additional services such as routing, firewall, optimization WAN, IDS/IPS systems, load balancing, and more, while maintaining compliance with organization-wide security standards.

• KVM Import

Enables cloud administrators to import unmanaged KVM hypervisor infrastructure into CloudStack.

• Scheduled Instance Lifecycle Operations

Efficiently manage cloud resources, allowing users to automate their instantaneous state transitions with predefined hours.

• CloudStack DRS

Enhance cloud environment performance and resource utilization automating workload distribution and rebalancing workload between Hosts.

• OAuth2 Authentication

OAuth2 plugin which is UI-first and can perform Single-Sign-On with 3-party identity providers such as Google and GitHub.

• CloudStack Snapshot Copy

Empower operators to select and copy volume snapshots across zones, ensuring data recovery and continuity in disaster recovery scenarios.

• VMware to KVM Migration

Enables operators to convert VMware disks to KVM via virt-v2v, supporting migrations through API and UI from managed and unmanaged vCenter to CloudStack-managed KVM hosts.

• CloudStack Object Storage

Implement the Object Storage Framework in CloudStack, allowing operators to select external object store providers, thereby making them available to users as a first-class resource.

All in all, these exciting new features make Apache CloudStack 4.19 an even more robust and flexible platform, offering enhanced capabilities to meet the needs of modern organisations. Let’s embark on a detailed exploration of these new features one by one:

VNF Appliances Support

Traditionally, Network Functions (NF) were housed in dedicated hardware appliances, such as routers, firewalls, and load balancers. This hardware was expensive, inflexible, and difficult to manage. However, in recent years, there has been a shift towards Virtual Network Functions (VNFs), which are software-based network functions that can be deployed on virtualised infrastructure.

This new feature simplifies the process of provisioning, deploying, and managing VNF Appliances, making it easier for organizations to take advantage of the scalability and flexibility that VNFs offer.

VNF Appliances are similar to traditional CloudStack Instances which require a Template for deployment. The VNF Template introduced a new type called ‘VNF’, which has unique characteristics that are not found in standard Templates. These characteristics include the ability to specify the network NICs that need to be connected to the corresponding CloudStack Guest Networks during deployment, as well as storing access details and other information about the device.

The ability to deploy VNFs directly on CloudStack is a valuable addition for operators looking to consolidate their network management and improve the efficiency of their operations. VNFs can provide a more scalable and flexible approach to network service delivery and can also help reduce the cost of infrastructure at the expense of traditional physical NF Appliances.

KVM Import

Numerous organisations rely on traditional KVM virtualization infrastructures, which often pose challenges in managing and defining access policies as they lack the inherent benefits of a multi-tenant cloud.

Anticipating these challenges that many organizations face, CloudStack 4.19 introduces the “KVM Import” Feature, a valuable tool that simplifies the difficult process of transitioning to a modern technology asset management model. This feature provides administrators and operators with an efficient management layer to streamline the migration process to Apache CloudStack.

The KVM Import Feature builds upon the foundations established by the VMware Ingestion Feature since version 4.14, offering a simplified mechanism for seamlessly integrating existing KVM infrastructure into CloudStack. This feature empowers organizations to seamlessly transition their traditional KVM environments into a unified and modern cloud orchestration framework.

In essence, the KVM Import feature bridges the gap between standalone KVM infrastructure and orchestrated cloud management. It offers a structured and user-friendly approach to integrating existing KVM environments into CloudStack, ensuring that organizations can transition towards utilising a unified, efficient, and scalable cloud management platform and benefiting from the agility and large-scale management of KVM assets.

Scheduled Instance Lifecycle Operations

Organisations, while managing their cloud infrastructure, often encounter the need for a systematic, scheduled, and orderly start-up or shutdown of Instances, ensuring optimal resource utilization, adherence to operational timelines and respecting mutual functional dependencies.

The “Scheduled Instance Lifecycle Operations” Feature in Apache CloudStack 4.19 seeks to address this need, offering a structured and automated approach to managing the state of Instances, enabling operators to define, manage, and automate the Instance’s start, stop, and reboot based on a predefined schedule.

Operators can now specify per Instance actions, periodicity (in both UNIX cron and human-readable formats), and optionally, a time zone, providing a flexible and precise way to manage their cloud resources. The architecture, natively driven by CloudStack, employs hypervisor-agnostic code, utilizing the VM manager for orchestrating Instance state transitions and ensuring that the defined schedules are adhered to with accuracy and reliability.

The feature benefits for organisations provide the ability to ensure that Instances are only active when needed, optimising resource utilisation and minimizing operational costs. Still, it also ensures that any dependent applications or services are initiated in the correct order and with the required timing.

CloudStack DRS

Balancing workload distribution, ensuring optimal resource allocation, minimizing power consumption, and maintaining consistent application performance are complex tasks, especially in large-scale cloud environments. Manual management and rebalancing of workloads can lead to errors and inefficiencies, highlighting a need for an automated, intelligent solution that can balance workloads and ensure resource utilization across all instances and hypervisor clusters.

Introduced in Apache CloudStack 4.19, the “CloudStack DRS (Distributed Resource Scheduler)” Feature has two distinct algorithms for workload balancing between Hosts: “Balanced” and “Condensed”. Each of these algorithms has unique characteristics and is suitable for different types of workloads as described below:

Balanced

This algorithm is designed to distribute workloads evenly across all available hypervisors. Ideal for production environments, aims to maximize operational efficiency and stability by distributing Instances in a way that avoids overloads and underuses. Comparable to VMWare DRS, it’s the perfect choice for organizations looking for a balance between performance, reliability and efficiency.

Condensed

This algorithm is designed to focus on optimizing energy consumption and consolidating workloads into the lowest possible number of Hosts. This is particularly beneficial in non-critical environments or laboratories where energy conservation is a priority. By focusing workloads on a minimal set of hypervisors and leaving others idle, “Condensed” saves energy without compromising performance and availability in applicable scenarios.

The CloudStack DRS implementation brings a number of significant benefits to cloud computing environments. These include improving resource utilization, reducing manual effort in workload management, minimizing energy consumption and maintaining consistent performance across hosts. By automating the distribution and rebalancing of workloads, DRS ensures that resources are neither underutilized nor overburdened, optimizing the efficiency and overall performance of the cloud environment. In addition, the flexibility offered by the algorithms “Balanced” and “Condensed” allows a precise adaptation to the specific needs of each environment, whether focused on energy efficiency or high availability and performance.

OAuth2 Authentication

Organisations that manage diverse user bases and seek to ensure secure access to their environments often seek to implement standardized authentication protocols in their IT ecosystems. OAuth2, recognized for its wide adoption in token-based authentication, has become a widely used standard for ensuring secure access for users, ensuring the protection of their credentials, and granting access only to authorised entities.

The introduction of OAuth2 Authentication in CloudStack 4.19 represents a significant step towards improving security and user management. This functionality allows organisations to integrate CloudStack with their existing OAuth2 authentication providers, including support for popular platforms such as GitHub and Google OAuth2. In this way, a secure and consistent authentication mechanism is guaranteed, aligned with the compliance standards adopted in the IT and cloud environments of organizations. Integration with these authentication providers enhances security by utilising trusted and widely recognized authentication methods.

CloudStack Snapshot Copy

Relying solely on proprietary disaster recovery solutions can be a costly and restrictive approach. These solutions often come with high licensing fees, recurring maintenance costs, and the need for specialized hardware, which can significantly impact an organization’s budget and flexibility. Additionally, proprietary DR solutions may not be as seamlessly integrated with existing IT infrastructure or cloud platforms, potentially hindering the overall disaster recovery capabilities.

Introduced in CloudStack 4.19, the “CloudStack Snapshot Copy” Feature adds an important layer of data protection and application availability. It facilitates the replication of the Instance’s disk snapshots across multiple CloudStack Zones, providing a defence against system failures, data corruption, and other disastrous events.

The “Snapshot Copy” feature utilises the Secondary Storage VM to provide data replication, which can be employed in Disaster Recovery scenarios. This functionality is complemented by a user interface that displays a list of zones where a snapshot is present, as well as the status of the copy, similar to Template registration. With the disk snapshot copy in place on the Zones, users can create new Volumes or Instances. This ensures that data can be recovered, and Instances can be swiftly reinstalled in the event of a disaster, enhancing business resilience and continuity.

VMware to KVM Migration

Broadcom’s acquisition of VMware has indeed raised concerns among organizations that rely on the platform for their business operations. The change in ownership has introduced uncertainty regarding future pricing policies and potential changes to the platform’s features and functionality. This has prompted analysts from Gartner and Forrester to advise VMware users to consider developing a migration strategy to mitigate the potential risks associated with Broadcom’s ownership.

Apache CloudStack 4.19 introduces a seamless VMware to KVM migration path, enabling organizations to minimize or eliminate reliance on VMware products while preserving disk volumes, attributes, and networks. Apache CloudStack’s ability to manage multiple hypervisors is further enhanced with its new VMware to KVM migration feature, providing a simple and efficient path to convert VMware instances to KVM. This feature ensures the preservation of disk volumes and attributes like name, compute offering, and networks, allowing for modification as needed.

The seamless migration process, supported by both API and user interface (UI), enables organizations to transition between VMware and KVM with minimal disruption and operational overhead. CloudStack 4.19’s VMware to KVM migration capability paves the way for organizations to adopt open-source virtualization technologies without compromising on efficiency, data integrity, or licensing costs.

CloudStack Object Storage

Object stores are designed to efficiently store, manage, and access massive amounts of unstructured data, such as images, videos, logs, and backups. They offer several key benefits that make them attractive for organizations, including scalability, cost-effectiveness, durability, and high availability.

Introduced in CloudStack 4.19, the “CloudStack Object Storage Framework” Feature, offers a centralized and streamlined approach to managing object storage providers. CloudStack Object Storage Framework enables administrators to seamlessly integrate external Object Store providers that comply with S3 protocols, enhancing the storage capabilities of the CloudStack environment. In the initial release, the feature exclusively supports MinIO, delivering a robust and seamless storage solution while plans to expand support to additional object store providers in future versions.

Once an Object Store is integrated into CloudStack, users can seamlessly utilize it as a regular CloudStack resource, enabling them to create, manage and access buckets, set the quota sizes, enable encryption, versioning and locking, also configure security policies.

CloudStack Object Store enables uninterrupted use of Object Stores and allows users to refine their storage strategies by browsing their specific use cases without resorting to external Object Store providers, minimising latency and improving the experience of platform users.

 

Guest OS Management

In Apache CloudStack, the guest operating system (OS) is used when registering templates or ISOs, subsequently configuring the Instance created. From version 4.19, CloudStack has enhanced its guest OS support, allowing administrators to manage guest OS names and their corresponding hypervisor mappings directly from the UI, a feature previously exclusive to the API.

This enhancement ensures the smooth operation of Instances created in the underlying hypervisor by CloudStack, promoting efficient resource allocation and Instance performance. In addition, the user interface now has sections for “Guest OS” and “Guest OS Mapping”, offering a simplified path for managing the operating system and the hypervisor mappings. An important point of evolution when developing was the introduction of the “GuestOsMapper” class, designed to avoid hardcoding and possible future conflicts. Overall, these improvements in CloudStack 4.19 provide a more robust and user-friendly approach to guest operating system management.

 

User-specified Source NAT

In Apache CloudStack, managing network ranges and reassigning IP addresses to routers is a common task. However, until version 4.18, one aspect remained static: the Source NAT (Network Address Translation) address for Guest Networks.

With the introduction of CloudStack 4.19, this limitation has been addressed. The new version brings enhanced API functionalities, allowing users to specify the Source NAT address for Guest networks. This is achieved through the addition of the sourcenatipaddress parameter in the API. Furthermore, the user interface has been updated to support this feature, offering a seamless experience for users.

 

CloudStack Storage Browser

Introduced in Apache CloudStack 4.15, the Secondary Storage Management feature offers high-level management capabilities, such as migrating and balancing entire secondary storage pools. However, there’s a growing demand for more granular control over the artefacts stored within these pools.

This new feature addresses this need, aiming to provide root administrators with enhanced visibility and control over their storage assets. With the new CloudStack Storage Browser, administrators can delve into secondary storage to view Templates, ISOs, Volumes, and Volume Snapshots. They can also retrieve live data on actual Secondary Storage usage, identify stale or failed files, and even perform bulk operations on specific files, such as deletion or migration.

This feature ensures that both the physical files and database records are synchronised transactionally. Moreover, it incorporates robust validation mechanisms to handle potential issues, especially when dealing with linked snapshots, and provides clear warnings to administrators about the implications of their actions.

 

Safe CloudStack Shutdown

The Safe CloudStack Shutdown feature in Apache CloudStack aims to enhance system reliability by introducing a controlled shutdown procedure. Recognising the potential disruptions caused by lost jobs during abrupt shutdowns, this feature ensures that the management server ceases to accept new asynchronous jobs immediately upon a shutdown request. It then patiently waits for the completion of all ongoing tasks before safely shutting down. This approach preserves the integrity of the operations and also reduces the chances of unexpected issues for customers.

 

New CloudStack Dashboard

The New CloudStack Dashboard redefine the initial user experience by presenting a more intuitive and comprehensive interface. Recognising the dashboard as the primary touchpoint for users, this redesign prioritises clarity and functionality. It offers modules for managing instances, Kubernetes clusters, Networks, Volumes, and Projects, each displaying usage against set limits. The dashboard also introduces “Quick Actions” shortcuts for common tasks, an events section, and a “Get Help” module providing essential links or embedded content. Furthermore, operators have the flexibility to customise the dashboard and there’s a provision for a service health module.

 

Comments on Management Servers

Comments on Management Server feature allows Administrators to annotate specific management servers with comments, providing essential context or notes for their colleagues ensuring that any operator accessing the system has the necessary information about a particular management server right at their fingertips.

 

Auto-Disable-Enable KVM Host

Auto-Disable-Enable KVM Host Feature allows for an on-hypervisor customisable health check, which can be configured through a script path in agent.properties. When executed at intervals, a non-zero result from the script will disable the host for new allocations, trigger an alert, and potentially update its comments with the script’s output. Conversely, a zero result will re-enable the host. This is an opt-in feature, potentially set at a cluster scope and can be globally configured. Notably, while existing Instances remain unaffected and operational, the system will avoid deploying new Instances on a host deemed unhealthy.

This feature distinguishes itself from the HA feature; it doesn’t aim to fence or restart Instances elsewhere but serves as a guideline for scheduling, indicating if hosts are “Ready” for new workloads even if they are “Up” and connected.

 

Support for vSphere 8.0.0

Apache CloudStack has introduced support for vSphere 8, encompassing both vCenter and ESXi. This enhancement ensures seamless integration and compatibility between CloudStack and the latest vSphere version.

 

UI Improvements

CloudStack’s user interface is undergoing a series of enhancements to improve user experience and functionality. Key changes include renaming terminologies for clarity, such as replacing “VM” with “Instance” and ensuring consistent capitalization across object types like Templates, Networks, and Snapshots.

The interface will also see improved error messages, offering clearer and more user-friendly feedback. Navigation is being refined, with “VM snapshots” being moved under the “Compute” main menu and renamed to “Instance Snapshots”. Additionally, there’s a push to remove ambiguous terminologies, like the term “public”, and to provide clearer explanations and tooltips throughout the interface. Dialogues will be enhanced with relevant and grammatically correct information overlays. Furthermore, the UI will be updated to use Vue3, with the Ant Design library migrated to support this Vue version. These improvements aim to streamline the user experience, making CloudStack’s interface more intuitive and efficient.

 

Support for Userdata in Autoscale Groups

In the latest enhancements to CloudStack, the Autoscale Instance group now offers support for Instances equipped with userdata content, allowing users to build reusable libraries of UserData configurations  Building on the advancements where userdata was elevated to a primary resource, the Autoscale Instance group has been expanded to accommodate Instances with both userdata IDs and detailed userdata specifications.

 

Resource Catalogue

CloudStack now provides users with access to  “at a glance” information about zones, offerings, and other resources directly within the UI. This includes details such as zone names, zone IDs, compute offerings, network offerings, and more. The primary motivation behind this feature is to facilitate users who require resource names or IDs to parameterise external integrations, such as Terraform.