Justin Burdine

Inventor - Creator - Dreamer

Adding a disk to a VMware VM in Cloudforms

Scenario

We are using an VMware provider with our Cloudforms installation, and we can successfully provision virtual machines using Native Clone provision type from fully configured VMware templates. The templates all have a single 30GB thin-provisioned hard drive.

Task

We would like all virtual machines provisioned from these templates to have a second 30GB hard drive added automatically during provisioning. The second drive should be created in the same VMware datastore as the first drive.

Methodology

Edit the VMProvision_VM state machine to add two new states to perform the task. We’ll add the second disk using the VMware API, using credentials stored for the provider. We can achieve this in a series of steps.

Step 1. Extend the State Machine

We’re going to extend the VM provisioning state machine by adding states, but we cannot do this to the state machine in the locked ManageIQ domain.

Copy the State Machine

The first thing that we must do is copy the ManageIQ/Infrastructure/VM/Provisioning/StateMachines/VMProvision_VM/Provision VM from Template (template) state machine instance into our own ACME domain so that we can edit the schema.

(insert How To copy a schema)

Edit the schema

Now we edit the schema of the copied class (see Editing the schema of the copied class).

ch22_ss
Figure 1. Editing the schema of the copied class
Add the new states

We add two more steps, AddDisk and StartVM to the bottom of the schema (see Adding two further states).

ch22_ss
Figure 2. Adding two further states
Adjust the sequence

Now we adjust the class schema sequence so that our new states come after PostProvision(see Adjusting the class schema sequence).

ch22_ss
Figure 3. Adjusting the class schema sequence

Step 2. Disable Auto-Power-On

We’re going to override the default behaviour of the VM provisioning workflow which is to auto-start a VM after provisioning. We do this because we want to add our new disk with the VM powered off, and then power on the VM ourselves afterwards.

Copy the method

We copy the /Infrastructure/VM/Provisioning/StateMachines/Methods/vmware_CustomizeRequest method from the ManageIQ domain (if we’re using ManageIQ) or from the RedHat domain (if we’re using CloudForms) into ours (see Copying the redhat_CustomizeRequest method into our own domain).

Note
The RedHat domain contains an enhanced version of redhat_CustomizeRequest. We must ensure that we copy and extend the correct version for our product.
ch22_ss
Figure 4. Copying the vmware_CustomizeRequest method into our own domain
Edit the method

We edit redhat_CustomizeRequest to set the options hash key :vm_auto_start to be false. We must do this after the line:

prov = $evm.root["miq_provision"]

The additional lines are as follows:

# Get provisioning object
prov = $evm.root["miq_provision"]

####  Add the following lines
# Set the autostart parameter to false so that RHEV won't start the VM directly
$evm.log(:info, "Setting vm_auto_start to false")
prov.set_option(:vm_auto_start, false)
####  End of additional lines

Step 3. Create Our New Instances and Methods

We’ll create a new namespace Integration/RedHat in our own domain, and create a simple one-field Methods class as we did in [writing-running-our-own-automation-scripts]. We add two new instances AddDisk and StartVM, and two new methods add_disk and start_vm to this class (see Adding two new instances and methods).

ch22_ss
Figure 5. Adding two new instances and methods

Next we’ll examine the interesting parts of the code in each of the methods.

add_disk

add_disk defines its own method call_rhev that handles the REST communication with the Red Hat Enterprise Virtualizaton Manager:

  def call_rhev(servername, username, password, action,
                ref=nil, body_type=:xml, body=nil)
    #
    # If ref is a url then use that one instead
    #
    unless ref.nil?
      url = ref if ref.include?('http')
    end
    url ||= "https://#{servername}#{ref}"

    params = {
      :method => action,
      :url => url,
      :user => username,
      :password => password,
      :headers => { :content_type=>body_type, :accept=>:xml },
      :verify_ssl => false
    }
    params[:payload] = body if body
    rest_response = RestClient::Request.new(params).execute
    #
    # RestClient raises an exception for us on any non-200 error
    #
    return rest_response
  end

In the main section of code we account for the fact that we’re allowing add_disk to be callable in either of two ways: from a button on a virtual machine in the WebUI, or as part of the VM provision workflow. (see [ways-of-entering-automate]). We first need to find out how add_diskhas been called, and retrieve the virtual machine service model object accordingly.

We also need to determine the new disk size. If add_disk has been called from a button, the new disk dize will have been passed as a service dialog element. If it’s called as part of a VM provisioning operation we’ll hardcode this as the NEW_DISK_SIZE constant (for this example it’s 30GB):

  case $evm.root['vmdb_object_type']
  when 'miq_provision'                  # called from a VM provision workflow
    vm = $evm.root['miq_provision'].destination
    disk_size_bytes = NEW_DISK_SIZE * 1024**3
  when 'vm'
    vm = $evm.root['vm']                # called from a button
    disk_size_bytes = $evm.root['dialog_disk_size_gb'].to_i * 1024**3
  end

We’re going to create the new disk on the same storage domain as the existing first disk, so we need to find the existing storage domain details:

  storage_id = vm.storage_id rescue nil
  #
  # Extract the RHEV-specific Storage Domain ID
  #
  unless storage_id.nil? || storage_id.blank?
    storage = $evm.vmdb('storage').find_by_id(storage_id)
    storage_domain_id = storage.ems_ref.match(/.*\/(\w.*)$/)[1]
  end

Next we extract the credentials of the RHEV Manager (from the ext_management_systemobject), as we’ll need to use these when we make the REST call. We also build our XML payload using the Nokogiri gem:

  unless storage_domain_id.nil?
    #
    # Extract the IP address and credentials for the RHEV provider
    #
    servername = vm.ext_management_system.ipaddress ||
                                            vm.ext_management_system.hostname
    username = vm.ext_management_system.authentication_userid
    password = vm.ext_management_system.authentication_password

    builder = Nokogiri::XML::Builder.new do |xml|
      xml.disk {
        xml.storage_domains {
          xml.storage_domain :id => storage_domain_id
        }
        xml.size disk_size_bytes
        xml.type 'system'
        xml.interface 'virtio'
        xml.format 'cow'
        xml.bootable 'false'
      }
    end

    body = builder.to_xml

We make the REST call to the RHEV Manager, and parse the response:

    $evm.log(:info,
              "Adding #{disk_size_bytes / 1024**3} GByte disk to VM: #{vm.name}")
    response = call_rhev(servername, username, password, :post, \
                                               "#{vm.ems_ref}/disks", :xml, body)
    #
    # Parse the response body XML
    #
    doc = Nokogiri::XML.parse(response.body)

The initial response back from the API contains some hrefs that we need to use, so we extract those:

    #
    # Pull out some reusable hrefs from the initial response
    #
    disk_href = doc.at_xpath("/disk")['href']
    creation_status_href = \
                       doc.at_xpath("/disk/link[@rel='creation_status']")['href']
    activate_href = doc.at_xpath("/disk/actions/link[@rel='activate']")['href']

We poll the API for the completion status:

Note
It’s not good practice to sleep in an Automate method. For simplicity in this example we’re handling the sleep → retry counter logic ourselves to avoid the possibility of sleeping forever. In a production environment we’d use the built-in state machine retry logic to handle this for us.
    #
    # Validate the creation_status (wait for up to a minute)
    #
    creation_status = doc.at_xpath("/disk/creation_status/state").text
    counter = 13
    while creation_status != "complete"
      counter -= 1
      if counter == 0
        raise "Timeout waiting for new disk creation_status to reach \
                              \"complete\": Creation Status = #{creation_status}"
      else
        sleep 5
        response = call_rhev(servername, username, password, :get,
                                                 creation_status_href, :xml, nil)
        doc = Nokogiri::XML.parse(response.body)
        creation_status = doc.at_xpath("/creation/status/state").text
      end
    end

If the disk has been attached to a powered-on VM (as it may have been if the method is called from a button), we would need to activate the disk in RHEV. If the VM is powered off when the disk is added, this stage is unnecessary:

    #
    # Disk has been created successfully,
    # now check its activation status and if necessary activate it
    #
    response = call_rhev(servername, username, password, :get,
                                                            disk_href, :xml, nil)
    doc = Nokogiri::XML.parse(response.body)
    if doc.at_xpath("/disk/active").text != "true"
      $evm.log(:info, "Activating disk")
      body = "<action/>"
      response = call_rhev(servername, username, password, :post,
                                                        activate_href, :xml, body)
    else
      $evm.log(:info, "New disk already active")
    end
  end
  #
  # Exit method
  #
  $evm.root['ae_result'] = 'ok'
  exit MIQ_OK
start_vm

The code for start_vm is as follows:

begin
  vm = $evm.root['miq_provision'].destination
  $evm.log(:info, "Current VM power state = #{vm.power_state}")
  unless vm.power_state == 'on'
    vm.start
    vm.refresh
    $evm.root['ae_result'] = 'retry'
    $evm.root['ae_retry_interval'] = '30.seconds'
  else
    $evm.root['ae_result'] = 'ok'
  end

rescue => err
  $evm.log(:error, "[#{err}]\n#{err.backtrace.join("\n")}")
  $evm.root['ae_result'] = 'error'
end

The full scripts are also available from here

Step 4. Add Our New Instances to the Copied State Machine

Now we edit our copied Provision VM from Template state machine instance to add the AddDisk and StartVM instance URIs to the appropriate steps (see Adding the instance URIs to the provisioning state machine).

ch22_ss
Figure 6. Adding the instance URIs to the provisioning state machine

Step 5. Provision a Virtual Machine

We’ll provision a VM to test this. We should see that the VM is not immediately started after creation, and suitable messages in automation.log show that our additional methods are working:

...<AEMethod add_disk> Adding 30GB disk to VM: rhel7srv006
...<AEMethod add_disk> Creation Status: pending
...<AEMethod add_disk> Creation Status: complete
...<AEMethod add_disk> New disk already active
...
...<AEMethod start_vm> Current VM power state = off
...<AEMethod start_vm> Current VM power state = unknown
...<AEMethod start_vm> Current VM power state = on

We can take a look at the number of disks in the virtual machine Details page in the ManageIQ WebUI (see VM details pane showing additional disk).

ch22_ss
Figure 7. VM details pane showing additional disk

Here we see the second disk attached to the virtual machine. Our modified VM provisioning workflow has been successful.

Summary

This chapter has shown how we can extend the provisioning state machine to add our own workflow stages. Although this has been a simple example, some kind of provisioning workflow extension is very common in practice. We see another example in [integrating-with-satellite-6-during-provisioning] where we extend the workflow to register our newly provisioned virtual machine with a Satellite 6 server.

The example has also shown the Integration functionality of CloudForms/ManageIQ, and how we can use API calls – in this case using the REST client – to extend our workflows into the wider enterprise.

start_vm-rb

Below is code that can be used as a method in Cloudforms to startup a VM after auto-provisioning a new disk.
See the How-To for steps how to implement this.

 
#----------------------------------------------------------------  
#  
# CFME Automate Method: start_vm  
#
# Author: Peter McGowan (Red Hat)  
#  
#----------------------------------------------------------------  
begin  
  vm = $evm.root['miq_provision'].destination  
  $evm.log(:info, "Current VM power state = #{vm.power_state}")  
  unless vm.power_state == 'on'  
  vm.start  
  vm.refresh  
  $evm.root['ae_result'] = 'retry'  
  $evm.root['ae_retry_interval'] = '30.seconds'  
  else  
  $evm.root['ae_result'] = 'ok'  
  end  
  
rescue => err  
  $evm.log(:error, "[#{err}]\n#{err.backtrace.join("\n")}")  
  $evm.root['ae_result'] = 'error'  
end

add_disk.rb

Below is code that can be used as a method in Cloudforms to add a disk to a VM either at provisioning time or via a CF UI button.
See the How-To for steps how to implement this.

 
#----------------------------------------------------------------  
#  
# CFME Automate Method: add_disk.rb  
#  
# Author: Peter McGowan (Red Hat)  
#  
#----------------------------------------------------------------  
  
case $evm.root['vmdb_object_type']  
when 'miq_provision'  
  prov = $evm.root['miq_provision']    
  ws_values = prov.options.fetch(:ws_values, {})  
  if ws_values.has_key?(:disk_size_gb)   
    size = ws_values[:disk_size_gb].to_i  
  else  
    size = 10  
    ws_values[:disk_size_gb] = 10  
    prov.set_option(:ws_values, ws_values)  
  end     
  new_disks = []  
  scsi_start_idx = 1  
  new_disks << {:bus => 0, :pos => scsi_start_idx, :sizeInMB => size.gigabytes / 1.megabyte, :backing => {:thinprovisioned => true}}  
  prov.set_option(:disk_scsi, new_disks) unless new_disks.blank?  
  $evm.log(:info, "Provisioning object  updated with <#{prov.get_option(:disk_scsi)}>")  
when 'vm'  
  vm = $evm.root['vm']  
  size = $evm.root['dialog_disk_size_gb'].to_i  
  $evm.log(:info, "AddDisk: Creating a new #{size}GB disk on Storage: #{vm.storage_name}")  
  # Get the vimVm object  
  vim_vm = vm.object_send('instance_eval', 'with_provider_object { | vimVm | return vimVm }')  
  vim_vm.addDisk("[#{vm.storage_name}]", size * 1024, 'label', 'summary', {dependent: true, thin_provisioned: true})  
end  
$evm.log(:info, "Processing add_disk...Complete")

Extending a partition on a Cloudforms appliance

If you are needing more space on your Cloudforms Appliance, below is a quick cheatsheet.

  • In your hypervisor, extend the virtual disk that the Cloudforms Appliance uses.
  • Inside the Cloudforms Appliance console run the following commands
    fdisk /dev/sda
    create a new partition (n)
    change the partition type to be Linux LVM (t 8e)
    write changes (w)
    pvcreate /dev/sdaX
    vgextend VG-CFME /dev/sdaX
    lvextend -L +20G /dev/mapper/VG--CFME-lv_var
    this command extends the LV of lv_var by 20 Gigabytes
    xfs_growfs /dev/mapper/VG--CFME-lv_var
    this extends the filesystem of lv_var
    

adding static routes to Mac OSX VPN connection

The other day I needed to setup a remote VPN connection back to my colocated environment and found that after setting up the VPN with l2tp I was unable to connect to any hosts. I quickly realized that I needed to setup static routes… so I dug around and found a great article by Rob Allen.

To solve this, you simply need to do the following on your Mac.

sudo touch /etc/ppp/ip-up
sudo chmod 755 /etc/ppp/ip-up
vi /etc/ppp/ip-up
the contents should look something like:
#!/bin/sh
/sbin/route add -net 172.16.33.0/24 -interface ppp0
/sbin/route add -net 10.0.21.0/24 -interface ppp0

test

function foo()
{
}

How Double-Take Works

How It Works

Double-Take Availability is a software offering from Vision Solutions.  While it enables you to do quite a few things from an IT operational perspective, like disaster recovery and automatic failover, etc, we primarily use it to migrate both physical (P2V) and virtual (V2V) servers.  It offers a way to provide my clients with minimal downtime while making it very easy to migrate from datacenters around the globe.

Double-Take continuously captures byte-level changes and replicates the data changes in real-time to any storage across at any location.  This continuous real-time synchronization allows us to setup migration jobs, let them synchronize and then schedule them in batches to be cutover.  We

Windows Requirements

  • Windows Server 2003 / 2008 Standard, Enterprise and Datacenter Editions (32-bit / 64-bit)
  • Windows Server 2012 Foundation, Essentials, Standard, Datacenter Editions

Hyper-V Requirements:

  • Microsoft .NET 3.5 SP1
  • VMs stored on a standard NTFS file system
  • TCP/IP with static IP addressing or reserved DHCP addressing for Hyper-V hosts

VMWare Requirements

  • VMware VirtualCenter 2.x or later
  • VMware ESX Server 3.x or later
  • VMware ESXi 5.0 or later

Using DoubleTake to migrate VMs

  • How Does DoubleTake Work?
  • Different types of Jobs
    • Full Server Migration
      • When to use
      • Pros/Cons
    • Full Server to ESX Migration
      • When to use
      • Pros/Cons
  • How to setup a Full Server Migration Job
  • How to setup a Full Server to ESX Migration Job

iPhone Video Routing Switch Controller

miranda_hbo-605x453
Recently I was asked to build an interface mechanism to make it easy to control a newly purchased video routing switcher. After a bit of tinkering in the lab I was able to come up with something fun, I present to you an iphone based video switcher. Attached is a video of the finished product which is an iphone web application that allows one to control the routing switcher from the familiar comfort of an iphone, itouch or ipad. The underlying technology is a linux LAMP server that has a bit of php code that issues commands to the serial port. The routing switcher is in turn hooked up to the linux servrer and responds to the given commands. The gui is based on iWebKit which is an amazing platform that I’ve used in the past to build out the iphone portion of workthefunk.com. To over simplify it it’s basically css that makes your web app work in safari and android browsers.

Video Multiviewers – DIY

I ran across a really cool video a while back that showed a live production coupled with the director’s talkback. The video was very cool, but I was once again reminded of how much I love video multiviewers.



I’ve built a few video production rooms in my time, but I’ve never had the good fortune of convincing anyone to buy a multiviewer.  Don’t know what I’m talking about?  Well a multiviewer is a device that takes a bunch of video inputs and then displays them up on a big screen tv.  The alternative is a bunch of physical monitors that end up generating a ton of heat and don’t lend themselves to being very flexible.  The advantage of the multi-viewer like the Miranda is to be able to configure your previews and inputs into custom positions for various productions.




This is HBOs new HD studio in new york, in which they utilized a Miranda Kalieido-X system to run their preview monitors.  The problem I have is that a miranda system is crazy expensive.  As an example a Kaleido 16 input dual output is in the $16k, I don’t have that kind of money… but MAN I really want one.  So, insert my favorite phrase, “maybe I can build one…”
The key to success is getting video input cards that can do hardware encoding.  Below is my current hardware list that I think might work

  • 4 x Blue Cherry PV-981 – Four input video card
    • this would give us a total of 16 video inputs
  • 1 x dual head video card (this would give us 2 outputs to send to two big screen tvs)
  • 1 x pc that can accommodate 4 pci-e slots + video card.
  • Magical software that will run/configure the windows inside the display. (I’ll supply the magic)
    • make it scalable by combining multiple pcs similar to what i described above and tie them back to a single config interface.

Now gotta do a bit of research to see if these cards can actually do what I’m wanting.  If I can get past that I believe this could be quite a viable solution.  Man if I could find multi channel HD cards!?!  That would be the holy grail!  Anyone?

 

© 2016 Justin Burdine

Theme by Anders NorenUp ↑