VM Stuck in "Pending" State on XenServer

On a rare occassion, you may encounter a XenServer-based VM that is stuck in a “Pending” state. Recently I came across a VM that was stuck with a pending “Shutdown” task. Within XenCenter, the VM was “unresponsive” and the Shutdown and Reboot buttons were greyed out.

A couple of potential causes for this condition include:

1. Taken from the comments of the following post

I’ve since found that the most common reason my VM’s won’t shut down is that they’re using a storage repository (such as an ISO library) that is offline.

In other words, if the virtual cd of a VM is pointing at an ISO that’s on an offline storage repository, then that VM can’t be shut down! (bug)”

2. From Citrix:

Windows being Windows. At times, when a server executes shutdown, it hangs, spinning it’s wheels. XenServer waits for a confirmation from XenTools that the shutdown is proceeding before it completes the task. If there are problems during the Shutdown, XenServer does not consider the task completed leading to a loss of connectivity to the XenServer tools. If connectivity between XenServer and XenTools is lost during the shutdown process meaning XenServer never receives a proper signal from Xentools about the state of the shutdown, you get that “stuck” symptom and the only thing you can do is force-kill the task and down a Force Reboot.

In this case, I had to force-kill the task from the XenServer console CLI using the following commands:

1. xe task-list to view the Pending tasks

2. xe task-cancel force=true uuid=<UUID> to cancel a specific task

Attempting a shutdown or a reboot using the xe vm-reboot or xe vm-shutdown commands did nothing but create another pending task. xe-toolstack-restart and xe task-cancel uuid=<UUID> (without the force=true) had no effect.

Once I cancelled the Pending task affecting the VM, the VM rebooted and worked normally; I did not have to run xe-toolstack-restart after cancelling the task(s), but this instance may be the exception and not the rule.

4 Comments

Filed under Citrix

4 responses to “VM Stuck in "Pending" State on XenServer

  1. Anonymous

    Dave, great work! I've run into this problem just recently with several VM's and this works great!

  2. Anonymous

    How do you determine which uuid belongs to the vm. The vm has a different uuid than the task list uuid.

  3. Anonymous

    Thank you, just stumbled into this problem when we changed the IP of the virtual machine and forgot about it's mounted NFS-shares – after using the “task-cancel force” like you described we were able to reboot the machine 🙂 – Again, thank you very much

  4. Thank you very much, helped me a lot!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s