Allow me, once again, to vent my frustration on this blog. I am trying to upgrade VMware workstation from version 7.1.6 to 8.0.6 on Ubuntu 12.04. I am doing this so I can run windows 8 VMs.

There is no working upgrade path. You have to uninstall VMware workstation 7 and install VMware workstation 8 on Ubuntu. The command to uninstall is:

/usr/bin/vmware-installer -u vmware-workstation

but this starts a wizard which immediately fails with the message:

The VWMare Installer could not shut down all
running virtual machines. If you have ACE
VMs open, please shut them down or suspend
them now and press 'Retry' to continue.

Needless to say, I have no running VMs after a reboot. So, OK, lets shut down all VMware process and unload all kernel modules using:

/etc/init.d/vmware stop

Oh dear. That gives:

Stopping VMware services:
   VMware USB Arbitrator                     done
/etc/init.d/vmware: line 370: [: 18177068162212705687: integer expression expected
   VM communication interface socket family                           failed
   Virtual machine communication interface                            failed
   Virtual machine monitor                                            failed
   Blocking file system                                               failed
ERROR: Module vmnet is in use by [permanent]

Well, that doesn’t look good! Turning on some debugging leads to the following line in the vmmonUseCount() function:

   count=`/sbin/lsmod | awk 'BEGIN {n = 0} {if ($1 == "'"$driver"'") n = $3} END {print n}'`

which indeed when used with vmmon as $driver returns a silly number.

# count=`/sbin/lsmod | awk 'BEGIN {n = 0} {if ($1 == "'vmmon'") n = $3} END  {print n}'`
# echo $count
18177068162212705687

So, what is going on? Well, it’s not lsmod, because cat’ing /proc/modules gives a similar number, which means that the scripts is making bad assumptions. It thinks there are 18177068162212705687 VMware workstations are running. I think not. In fact, it can’t even know this since the number is so large, it’s bigger than the biggest integer that the shell test expression can handle, hence the error. See http://stackoverflow.com/questions/2965761/find-maximum-positive-integer-value-in-bourne-shell

My answer here is to comment out the block which begins with the integer -gt expression.

# /etc/init.d/vmware stop
Stopping VMware services:
   VMware USB Arbitrator                                               done
   VM communication interface socket family                           failed
   Virtual machine communication interface                            failed
   Virtual machine monitor                                            failed
   Blocking file system                                               failed
ERROR: Module vmnet is in use by [permanent]

Well, that gets rid of one thing. The rest will have to wait since it’s past my bedtime. Sigh.