Introduction

In my previous post, I described the new ‘Open vSwitch with DPDK support’ on CloudStack for KVM hosts. There, I focused on describing the feature, as it was new to CloudStack, and also explained the necessary configuration on the KVM agents’ side to enable DPDK support.

DPDK (Data Plane Development Kit) (https://www.dpdk.org/) is a set of libraries and NIC drivers for fast package processing in userspace. Using DPDK along with OVS brings benefits to networking performance on VMs and networking appliances. DPDK support in CloudStack requires that the KVM hypervisor is running on DPDK compatible hardware.

In this post, I will describe the new functions which ShapeBlue has introduced for the CloudStack 4.13 LTS. With these new features, DPDK support is extended, allowing administrators to:

• Create service offerings with additional configurations. In particular, the DPDK required additional configurations can be included on service offerings
• Select the DPDK vHost User mode to use on each VM deployment, from these service offerings
• Perform live migrations of DPDK-enabled VMs between DPDK-enabled hosts

CloudStack new additions for DPDK support

In the first place, it is necessary to mention that DPDK support works along with additional VM configurations. Please ensure that the global setting ‘enable.additional.vm.configuration’ is turned on.

As a reminder from the previous post, DPDK support is enabled on VMs with additional configuration details/keys:

• ‘extraconfig-dpdk-numa’
• ‘extraconfig-dpdk-hugepages’

One of the new additions for DPDK support is the ability to select vHost user mode to use for DPDK by the administrator via service offerings. The vHost user mode describes a client / server model between Openvswitch along with DPDK and QEMU, in which one acts as a client while the other as a server. The server creates and manages the vHost user sockets, and the client connects to the sockets created by the server.

Additional configurations on service offerings

CloudStack allows VM XML additional configurations and DPDK vHost user mode to be stored on service offerings as details and be used on VM deployments from the service offering. Additional configurations and the DPDK vHost user mode for VM deployments must be passed as service offering details to ‘createServiceOffering’ API by the administrator.

For example, the following format is valid:

(cloudmonkey)> create serviceoffering name=NAME displaytext=TEXT domainid=DOMAIN hosttags=TAGS
serviceofferingdetails[0].key=DPDK-VHOSTUSER serviceofferingdetails[0].value=server
serviceofferingdetails[1].key=extraconfig-dpdk-numa serviceofferingdetails[1].value=NUMACONF
serviceofferingdetails[2].key=extraconfig-dpdk-hugepages serviceofferingdetails[2].value=HUGEPAGESCONF


Please note:

• Each additional configuration value must be URL UTF-8 encoded (NUMACONF and HUGEPAGESCONF in the example above).
• The DPDK vHost user mode key must be: “DPDK-VHOSTUSER”, and its possible values are “client” and “server”. Its value is passed to the KVM hypervisors. If it is not passed, then “server” mode is assumed. Please note this value must not be encoded.
• Additional configurations on VMs are additive to the additional configurations on service offerings.
• In case one or more additional configuration have the same name (or key), then the additional configurations on the VM take precedence over the additional configuration on the service offering.

On VM deployment, the DPDK vHost user mode is passed to the KVM host. Based on its value:

• When DPDK vHost user mode = “server”:
  • OVS with DPDK acts as the server, while QEMU acts as the client. This means that VM’s interfaces are created in ‘client’ mode.
  • The DPDK ports are created with type: ‘dpdkvhostuser’
• When DPDK vHost user mode = “client”:
  • OVS with DPDK acts as the client, and QEMU acts as the server.
  • If Openvswitch is restarted, then the sockets can reconnect to the existing sockets on the server, and standard connectivity can be resumed.
  • The DPDK ports are created with type: ‘dpdkvhostuserclient’

Live migrations of DPDK-enabled VMs

Another useful functionality of DPDK support is live migration between DPDK enabled hosts. This is possible by introducing a new host capability on DPDK enabled hosts (enablement has been described on the previous post). CloudStack uses the DPDK host capability to determine which hosts are DPDK enabled.

However, the management servers also need a mechanism to decide if a VM is DPDK-enabled, before allowing live migration to DPDK enabled hosts. The decision is made on the following criteria:

• A VM is running on a DPDK-enabled host.
• The VM possess the DPDK required configuration from VM details or service offering details.

This allows administrators to live migrate these VMs to suitable hosts.

Conclusion

As the previous post describes, DPDK support was initially introduced in CloudStack 4.12. This blog post covers the DPDK support extension for CloudStack 4.13 LTS, introducing more flexibility and improving its usage. As CloudStack recently started supporting DPDK, more additions to its support are expected to be added in future versions.

Future work may involve UI support for the previously described features. Please note that it is currently not possible to pass additional configuration to VMs or service offerings using the CloudStack UI, it is only available through the API.

For references, please check PRs:

• https://github.com/apache/cloudstack/pull/2839
• https://github.com/apache/cloudstack/pull/3153
• https://github.com/apache/cloudstack/pull/3365

About the author

Nicolas Vazquez is a Senior Software Engineer at ShapeBlue, the Cloud Specialists, and is a committer in the Apache CloudStack project. Nicolas spends his time designing and implementing features in Apache CloudStack.

Nicolas Vazquez

Nicolas Vazquez is a Senior Software Engineer at ShapeBlue and is a PMC member of the Apache CloudStack project. He spends his time designing and implementing features in Apache CloudStack and can be seen acting as a release manager also. Nicolas is based in Uruguay and is a father of a young girl. He is a fan of sports, enjoy playing tennis and football. In his free time, he also enjoys reading and listening to economic and politics materials.