Citrix XenServer 5.5 Automated Live VM Backup to Windows CIFS Share

Up until now reliably backing up virtual machines from any of the free virtualisation hypervisors has been either expensive or impossible.

With the launch of Citrix XenServer 5.5 you can now not only take VM snapshots of live servers without shutting them down and taking them offline, you can also export these snapshots including the hard drive disk data to a template file.


This script has been superseded with a more advanced version, fixing several bugs including the snapshot disk space issue, and with additional functionality.

If you wish to use this backup script instead please follow the link -

http://blog.andyburton.co.uk/index.php/2009-11/updated-citrix-xenserver-5-5-automatic-vm-backup-scripts/


From the Citrix XenServer 5.5 Documentation:

Exporting and importing virtual machines

When you export a VM, a complete copy of the VM (including disk images) is stored as a single file on your local machine, with a .xva file extension. The VM export/import feature can be used in a number of different ways:

As a convenient backup facility for your VMs. An exported VM file can be used to recover an entire VM in the event of disaster.
As a way of quickly copying a VM, for example, a special-purpose server configuration that you use many times. You simply configure the VM the way you want it, export it, and then import it to create copies of your original VM.
As a simple method for moving a VM to another server.

However, the catch with these VM exports comes with step 1 – you must shut down the VM you wish to export. This is where the snapshots come in…

From the Citrix XenServer 5.5 Documentation:

Using VM snapshots

A virtual machine (VM) snapshot is a record of a running virtual machine at a point in time. When you take a snapshot of a VM, its storage information (the data on the hard drive) and metadata (configuration information) is also saved. Where necessary, I/O is temporarily halted while the snapshot is being taken to ensure that a self-consistent disk image can be captured. Unlike VM exports, snapshots can be created without first shutting down the VM. A snapshot is similar to a normal VM template but it contains all the storage and configuration information for the original VM, including networking information. Snapshots provide a fast way of creating templates that can be exported for backup purposes and then restored, or that can be used to quickly create new VMs.

If you combine snapshots and snapshot exporting you can see the functionality to take a live export exists as standard. You can easily take a VM snapshot and export it to a local disk using the XenCenter management tool, however you may wish to automatically backup your VMs…

Luckily Citrix XenServer 5.5 comes with a fantastic command line API which we can take advantage of.

Citrix XenServer itself is also essentially a customised linux system. This allows us to write a live VM backup bash script using the Citrix XenServer command line API (xe) which we can automate using the natively installed crontab (CRON).

With this in mind, it is just a case of merging it all together to write the batch script to backup the servers and automate it.

For my setup i wanted the hypervisors to backup to a seperate windows server 2008 storage server with a windows CIFS share (a typical windows shared folder).

The Citrix XenServer hyper-visor has the ability to mount a windows CIFS shared folder, acting as a local disk.

Mount Windows CIFS Shared Folder

Create the backup folder on the Windows server and make sure the user you want to backup as has write access. Then, assuming you know the backup folder address and login details, simply SSH into the hypervisor and run the following:


[root@vs-xs2 /]# mkdir /backup
[root@vs-xs2 /]# mount -t cifs "//192.168.0.20/VM Backup" -o username=username,password=password /backup

The above would mount the windows shared folder “VM Backup” on 192.168.0.20 to the /backup folder on the hypervisor, with the windows authentication details of username=username and password=password.

Reference: http://support.citrix.com/article/CTX121937

Live VM Backup/Export Script

SSH into the hypervisor and create the following script:

[root@vs-xs2 /]# nano /home/vm_backup

#!/bin/bash

# Settings

backup_dir="/backup/"
backup_ext=" VM.xva"
date=$(date +%Y-%m-%d_%H-%M-%S)

# VMs to backup

vm_backup_list=()
#vm_backup_list[0]="901d9d55-a61d-aff8-d0d1-7da25c83d827"
#vm_backup_list[1]="fe1f60cb-dcd6-46e0-8ed6-d0d8301baf04"

vm_backup_list_count=${#vm_backup_list[@]}

# Get VM list

vm_list_string=`xe vm-list is-control-domain=false`
IFS="
"
vm_list_array=($vm_list_string)
vm_list_count=${#vm_list_array[@]}

# Create arrays to use

vm_uuid_array=()
vm_label_array=()
vm_log=()


# Start Log

vm_log[${#vm_log[@]}]="Starting VM Backup: $date"
vm_log[${#vm_log[@]}]="-----------------------------"


# Get VMs to export

vm_log[${#vm_log[@]}]="Parsing VM list"

key=0
index=0

for line in ${vm_list_array[@]}; do

        if [ ${line:0:4} = "uuid" ]; then

                uuid=`expr "$line" : '.*: \(.*\)$'`
                label=`expr "${vm_list_array[key+1]}" : '.*: \(.*\)$'`

                vm_uuid_array[index]=$uuid
                vm_label_array[index]=$label

                vm_log[${#vm_log[@]}]="Added VM #$index: $uuid, $label"

                let "index = $index+1"

        fi

        let "key = $key+1"

done

vm_log[${#vm_log[@]}]="Done parsing VM list"


# Backup VMs

vm_log[${#vm_log[@]}]="Backup VMs"

key=0

for uuid in ${vm_uuid_array[@]}; do

        # Set VM backup state

        backup_vm=false

        # If the backup list is empty

        if [ $vm_backup_list_count = 0 ]; then

                # Backup all VMs

                backup_vm=true

        # Else check to see if the VM is to be backed up

        else

                for backup_uuid in ${vm_backup_list[@]}; do

                        if [ $uuid = $backup_uuid ]; then

                                backup_vm=true
                                break

                        fi

                done

        fi

        # If the VM is being backed up

        if [ $backup_vm = true ]; then

                # Log

                vm_log[${#vm_log[@]}]="VM: $uuid"

                # Label

                label=${vm_label_array[key]}

                # Create snapshot

                snapshot=`xe vm-snapshot vm=$uuid new-name-label=backup_$date`
                vm_log[${#vm_log[@]}]="Snapshot: $snapshot"

                # Set as VM not template

                snapshot_template=`xe template-param-set is-a-template=false uuid=$snapshot`
                vm_log[${#vm_log[@]}]="Set as VM"

                # Export

                snapshot_export=`xe vm-export vm=$snapshot filename="$backup_dir$label-$date$backup_ext"`
                vm_log[${#vm_log[@]}]="Export: $snapshot_export"

                # Delete snapshot

                snapshot_delete=`xe vm-uninstall uuid=$snapshot force=true`
                vm_log[${#vm_log[@]}]="Delete Snapshot: $snapshot_delete"

        # Else the VM isnt being backed up

        else

                # Log

                vm_log[${#vm_log[@]}]="VM: $uuid"
                vm_log[${#vm_log[@]}]="Ignoring Backup"

        fi

        # Increment Key

        let "key = $key+1"

done

vm_log[${#vm_log[@]}]="Export Complete"
vm_log[${#vm_log[@]}]="

"

# Logging

#echo ${vm_uuid_array[@]}
#echo ${vm_label_array[@]}

for log in ${vm_log[@]}; do
        echo $log
done

You can download the script directly here (instead of copy/pasting): http://www.andy-burton.co.uk/blog/wp-content/uploads/2009/10/xenserver-backup-02.txt

Press ctrl+x, y and then enter to save.

The above script will look up all of the virtual servers running on the hyper-visor, generate a snapshot for each, export the snapshot to the windows shared folder and then remove the snapshot.

It will also output a log of what happens, for confirmation and debugging purposes.

The saved filename will be in the format “VM Name-2009-08-24_02-00-00 VM.xva”. You can change this by editing the backup_ext=” VM.xva” and date=$(date +%Y-%m-%d_%H-%M-%S) lines at the top of the backup script.

If you wish for individual VMs to be backed up you can add their uuid (run xe vm-list on the hypervisor to see a list of all VM uuids) to the vm_backup_list array

e.g The following would only backup the 2 specified virtual machines, ignoring any others.

vm_backup_list[0]="901d9d55-a61d-aff8-d0d1-7da25c83d827"
vm_backup_list[1]="fe1f60cb-dcd6-46e0-8ed6-d0d8301baf04"

The 2 lines above are supplied as an example and are commented out using the # character in the code.

Reference: http://forums.citrix.com/message.jspa?messageID=1342058

Automate the VM Backup/Export Script

Now that you’ve got the script to backup all your VMs you might want this to happen automatically, say 2am in the morning each day when everything is quiet. Luckily the hypervisor, being linux based, comes with crontab pre-installed which allows us to schedule a script to run automatically whenever we choose.

Whilst logged into the hypervisor run the following:


[root@vs-xs2 /]# crontab -e
0 2 * * * /home/vm_backup >> /home/vm_backup.log 2>&1

The likelyhood is that the crontab file will be opened using vi, a more specialised editor than nano. To exit type :wq. If you need to exit without saving type :q!

You can tell the crontab to be opened using nano as default (which i prefer) by adding the following lines to /etc/bashrc


EDITOR=nano
export EDITOR

You will need to re-login for the change to be effective.

Back to the cronjob we have setup – this will automatically run the backup script we created at 2am every morning. It will also save the output of the script (the progress log) to /home/vm_backup.log

Just to explain a little about the cronjob timing for those that dont understand – the 5 stars indicate 5 different periods of time: hour, minute, day of the month, month, day of the week (0-6 sunday-monday).

Here are a few examples:


0 2 * * * - Will run at 2am every day
0 0 * * * - Will run at midnight every day
0 0 * * 0 - Will run at midnight every sunday
0 0 1 * * - Will run on the 1st day of each month
0 0,12 * * * - Will run every day at midnight and midday.

For more information on how to configure the cronjob to run at different times the following should help: http://www.adminschoice.com/docs/crontab.htm

Automatically Clearing Backups

And if you want to automatically delete backup files over a week old (7 days) on the windows server to stop your disks running out of space pretty quickly?…

… The following should help:


Forfiles -p "D:\VM Backup" -m *.* -d -7 -c "cmd /c del /q @path"

This will delete all files in the “D:\VM Backup” folder that are older than 7 days. You can change the folder path and age to delete files at by editing the command.

Just setup this command up as a scheduled task in windows.

To view which files would be deleted, rather than actually deleting them, just run the following from command line:


Forfiles -p "D:\VM Backup" -m *.* -d -14 -c "cmd /c Echo @path"

Reference: http://forums.devshed.com/showpost.php?p=2072796&postcount=17

All comments good/bad welcome.

109 thoughts on “Citrix XenServer 5.5 Automated Live VM Backup to Windows CIFS Share

  1. Pingback: How to backup XenServer VMs | Stephen Garriques

  2. I have to thank you for the job done on that script.

    It quickly saved my life when XenCenter wouldn’t want to perform exports. It makes XenServer much safer to use even with low end storage environments when I had no time to make research for developping my own script of that kind.

    Keep up the good job,
    Regards,

  3. Pingback: VMnerds blog » script Backup pour XenServer

  4. Just found this script and I can’t thank you enough!

    I noticed in one of the comments that at least one user was getting errors on the first ‘do’ loop. This problem is created by using a DOS format for txt file and can be fixed using the dos2unix program. Might be useful to put in the notes for the script.

    I’m also running XenServer 5.6 and there were no errors if anyone is interested in compatibility above 5.5. =)

  5. Hi Andy…

    your scripts running good on my citrix.

    i have little question.

    when i try to import xva as a “new” VM why ip configuration is back to default with no IP(s)?

  6. Pingback: leandrocaetano .info « Citrix XenServer Automated Live VM Backup

  7. This script works great. Andy has done absolutely amazing work on this. The only comments I have would be a short wish list for the script. One more thing, I will suggest to use xe vm-install based on the template, and not xe template-param-set is-a-template=false

  8. Hi Andy,

    Thank you for the great script! I have a small problem. The script runs every night and makes backups. The only thinig is that 1 VM doesn’t backup….how strange is that?

    I can’t figure out why.

    Any help is appreciated!

    With kind regards,
    Robert

  9. Pingback: Anonymous

  10. man thanks a ton for posting this. i was trying to clone a vm and the clone kept coming up as a template. but looking over your script, i was able to determine what i needed to do. thanks a ton for sharing

  11. Pingback: XEN BACKUP MADE EASY AND FREE! THANK YOU ANDY BURTON! « dwave's blogg

  12. Pingback: XEN LIVE BACKUP OF RUNNING VMS MADE EASY AND FREE! THANK YOU ANDY BURTON! « dwave's blogg

  13. Im getting the following with a syntax error but don’t know why. Any help appreciated.

    [root@ensxen01 home]# sh vm_backup
    expr: syntax error
    Starting
    VM
    Backup:
    2011-07-13_00-53-08
    —————————–
    Parsing
    VM
    list
    Added
    VM
    #0:
    ,
    Done
    parsing
    VM
    list
    Backup
    VMs
    Export
    Complete

  14. Pingback: What is the best design (software/hardware, backups) for two servers providing virtual machines in colo - Admins Goodies

  15. I’m having issue importing VMs created by this script – the error message I am getting is:
    msg: INTERNAL_ERROR: [ Stream_vdi.Invalid_checksum("Block Ref:956/00021836 checksum failed: original = edb8da5342bea8e87b3d5c4aa1ccf6dd1fe42a22; recomputed = 1528d7e52102dc20302514c09be21f28c4593dc5") ]

    Any ideas how to “ignore” it or why this happens?

  16. Hi Andy,
    Backup was completed successfully. But when I am trying to import I getting error>

    Error: This file couldn’t be imported.

    Is there anything I am missing?

    Thanks,
    Indresh

  17. Your script is a genuine life saver. Thank you very much, bro. It saved me a lot of time, not mentioning saving my a** :D
    Once again thank you very much for your work and for sharing it with us.

  18. Hi Andy,
    Thanks for the script. Only one concern.

    The script is taking the backup correctly. but can it delete the old one backup and create the new one every time.

  19. This is drving me nuts. The 6.0 documentation sya snapshots are "crash consistent". That means you cant reliably restore from them! I dont think they would have downgraded this mechanism in version 6 and your experts from 5.5 suggest it the same as in 6.0 on said:

    This is driving me nuts. The 6.0 documentation says snapshots are “crash consistent”. That means you cant reliably restore from them *unless you shutdown the VMs just like for vm-export! I dont think they would have downgraded this mechanism in version 6, and your experts from 5.5 suggest its the same as in 6.0. Why is everyone just ignoring this!! See http://www.backupcentral.com/mr-backup-blog-mainmenu-47/13-mr-backup-blog/293-crash-no-goo.html

  20. Hi Andy,

    Nice script and thanks for that. it works fine for me

    Can you please edit the script to automatically delete the backup taken before taking a new one?

  21. Hi, how can i mount my CIFS share at the startup, instead of typing the following command in console? mount -t cifs “//192.168.0.20/VM Backup” -o username=username,password=password /backup
    Thanx a Lot
    Have a wonderful day

  22. Hi can zou please tell me why i get Feb 27, 2013 1:41:27 AM Error: Importing VM from ‘D:\Backup\Windows_Server_2008_64-bit_1-2013-02-27_01-08-10.xva’ to ‘xenserv’ – This file could not be imported
    when i try to import from file

  23. Fantastic, really well designed solution.
    Seems to work (I have not tested recovery of a backup’d VM yet) perfectly also with XenServer 6.1.

  24. Pingback: Live VM Backup to Windows CIFS Share | Open Word

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>