heartbleed

How to Mitigate OpenSSL HeartBleed Vulnerability in Apache CloudStack

UPDATE: 09-Apr-2014 - The proper upgrade command is “apt-get install openssl libssl1.0.0″. If you’ve just updated openssl, please go back and update libssl as well.

UPDATE: 10-Apr-2014 - Added detailed verification steps / Apache CloudStack 4.0 – 4.1 are not vulnerable, they use older Debian/openssl.

Thanks to all involved for helping to put together and update this information

 

Earlier this week, a security vulnerability was disclosed in OpenSSL, one of the software libraries that Apache CloudStack uses to encrypt data sent over network connections. As the vulnerability has existed in OpenSSL since early 2012, System VMs in Apache CloudStack versions 4.0.0-incubating-4.3 are running software using vulnerable versions of OpenSSL. This includes CloudStack’s Virtual Router VMs, Console Proxy VMs, and Secondary Storage VMs.

The CloudStack community are actively working on creating updated System VM templates for each recent version of Apache CloudStack, and for each of the hypervisor platforms which Apache CloudStack supports. Due to testing and QA processes, this will take several days. In the meantime, a temporary workaround is available for currently running System VMs.

If you are running Apache CloudStack 4.0.0-incubating through the recent 4.3 release, the following steps will help ensure the security of your cloud infrastructure until an updated version of the System VM template is available:

Logon to each Secondary Storage VM, Console Proxy VM and Virtual Router and update openssl

XenServer & KVM

  1. Use the GUI to identify the Link Local IP and Host of the VM
  2. Connect to the Hypervisor Host using SSH
  3. From the Host, Connect to the VM using the following command, replacing n.n.n.n with the Link Local IP identified in step 1.
  4. On the System VM,When updating Secondary Storage VMs, run /etc/init.d/apache2 restart
    • run apt-get update
    • then run apt-get install openssl libssl1.0.0
    • If a dialog appears asking to restart programs, accept its request
  5. Log out of the System VM and host server
  6. Repeat for all Secondary Storage, Console Proxy and Virtual Router VMs

 

 VMware

  1. Use the GUI to identify the Management / Private IP of the VM
  2. SSH onto a CloudStack Management Server
  3. From the Management Server, connect to the VM using the following command, replacing n.n.n.n with the Managemnt/Private IP identified in step 1.
    • ​ssh -i /var/lib/cloud/management/.ssh/id_rsa -p 3922 root@n.n.n.n
  4. On the System VM,When updating Secondary Storage VMs, run /etc/init.d/apache2 restart
    • run apt-get update
    • then run apt-get install openssl libssl1.0.0
    • If a dialog appears asking to restart programs, accept its request
  5. Log out of the System VM and host server
  6. Repeat for all Secondary Storage, Console Proxy and Virtual Router VMs

 

Verification

On each System VM, you can test if it has non-vulnerable openssl packages installed by listing installed packages and looking at the installed versions of openssl and libssl. As in the example below, for a system to be non-vulnerable, the packages need to be at or above version 1.0.1e-2+deb7u6:

root@v-14-VM:~# dpkg -l|grep ssl
ii  libssl1.0.0:i386                1.0.1e-2+deb7u6                  i386         SSL shared libraries
ii  openssl                              1.0.1e-2+deb7u6                  i386         Secure Socket Layer (SSL) binary and related cryptographic tools

We realise that for larger installations where System VMs are being actively created and destroyed based on customer demand, this is a very rough stop-gap. The Apache CloudStack security team is actively working on a more permanent fix and will be releasing that to the community as soon as possible.

For Apache CloudStack installations that secure the web-based user-interface with SSL, these may also be vulnerable to HeartBleed, but that is outside the scope of this blog post. We recommend testing your installation with [1] to determine if you need to patch/upgrade the SSL library used by any web servers (or other SSL-based services) you use.

Information originally posted on https://blogs.apache.org/cloudstack/entry/how_to_mitigate_openssl_heartbleed

1: http://filippo.io/Heartbleed/

 

About the Author

Geoff Higginbottom is CTO of ShapeBlue, the strategic cloud consultancy and an Apache CloudStack Committer. Geoff spends most of his time designing private & public cloud infrastructures for telco’s, ISP’s and enterprises based on CloudStack.