KVM Success!

After a long elapsed period, I finally found a site which gave good instructions on how to move a VMware Windows 7 machine to KVM. That site is https://docs.google.com/document/d/1X8TaBP1v_rh8e2QXDGFmkmzoPB13Pvp2V0FQ630VCsU/edit. (Googled for qemu import qcow2).

After converting the image using qemu-img convert -f vmdk -O qcow2 Windows7-1.vmdk /VMbackups2/Windows7-1.qcow2 the instructions turned out to be very simple.

Result! I now have a Windows 7 desktop running under KVM on Centos 7! Next I will convert Windows 8.

In other news, yesterday I started a new job as a solutions architect for HSBC in London.

Centos 7 diary entry

As mentioned below, as I go off MacOS, I have been investing more (precious) time getting a Centos 7.0 desktop fully functional. Mail is up and running in the shape of Thunderbird, which I preferred over Evolution. Neither seem particularly customisable (particularly fonts) but Thunderbird seems to have the edge.

My next major requirement is virtualisation. I didn’t have any luck with VMware Player as a first attempt. It loaded some kernel modules and I pointed at my existing vmx files but the VMs didn’t boot and now the player just crashed with a couple of error lines in the USBArb log.

In any case, I wanted to use KVM on Linux and have reached the milestone of building a Centos 7 guest VM using VMM and qemu-kvm (after overcoming the obligatory hurdles, luckily relatively minor). I don’t have much faith in VMware and KVM working together although there’s no reason why they shouldn’t (VMware player is a type 2 hypervisor, or is it?) Things are complicated enough as it is without trying to run two hypervisors on the same machine!

Next experiment is to get my Windows desktops currently running under VMware workstation on Ubuntu into QEMU/KVM.

VMware Workstation to Hyper-V

I’ve been using Windows 8 for more than a year, but only in a virtual machine running in VMware workstation. Recently I have been using my laptop in a dual-boot configuration, which works but confuses the Windows 7 disk which always requires a disk check on booting. I finally have a licensed Windows 8.1 as my primary OS (still in dual boot) which I can use to migrate my lab from VMware Workstation to Hyper-V.

Now VMware workstation is a fantastic product and has Hyper-V is not quite up to the same level yet. However, Hyper-V has one big advantage on Windows 8.1: it is free with the operating system. so the task begins to migrate VMware to Hyper-V. I have several Windows Servers which form a lab for CBT nuggets as well as some Linux and Centos hosts and some legacy laptops which I have P2Vd.

The logical place to start seems to be to download and install the Microsoft Virtual Machine Converter 3.0. However, this is taking a wrong turn at Albuquerque. The converter downloads and installs fine. But it is a server product which converts machines from an ESXi server to a Hyper-V host running Windows Server.

The disk2vhd from sysinternals would seem like another option but no, it will not read vmdks that I can see. It might just be the case that I have to backup my VMs and restore them. A common result in search is http://www.starwindsoftware.com/converter. I will investigate this and report in the next post.

Frustration :-(

Allow me, once again, to vent my frustration on this blog. I am trying to upgrade VMware workstation from version 7.1.6 to 8.0.6 on Ubuntu 12.04. I am doing this so I can run windows 8 VMs.

There is no working upgrade path. You have to uninstall VMware workstation 7 and install VMware workstation 8 on Ubuntu. The command to uninstall is:

/usr/bin/vmware-installer -u vmware-workstation

but this starts a wizard which immediately fails with the message:

The VWMare Installer could not shut down all
running virtual machines. If you have ACE
VMs open, please shut them down or suspend
them now and press 'Retry' to continue.

Needless to say, I have no running VMs after a reboot. So, OK, lets shut down all VMware process and unload all kernel modules using:

/etc/init.d/vmware stop

Oh dear. That gives:

Stopping VMware services:
   VMware USB Arbitrator                     done
/etc/init.d/vmware: line 370: [: 18177068162212705687: integer expression expected
   VM communication interface socket family                           failed
   Virtual machine communication interface                            failed
   Virtual machine monitor                                            failed
   Blocking file system                                               failed
ERROR: Module vmnet is in use by [permanent]

Well, that doesn’t look good! Turning on some debugging leads to the following line in the vmmonUseCount() function:

   count=`/sbin/lsmod | awk 'BEGIN {n = 0} {if ($1 == "'"$driver"'") n = $3} END {print n}'`

which indeed when used with vmmon as $driver returns a silly number.

# count=`/sbin/lsmod | awk 'BEGIN {n = 0} {if ($1 == "'vmmon'") n = $3} END  {print n}'`
# echo $count

So, what is going on? Well, it’s not lsmod, because cat’ing /proc/modules gives a similar number, which means that the scripts is making bad assumptions. It thinks there are 18177068162212705687 VMware workstations are running. I think not. In fact, it can’t even know this since the number is so large, it’s bigger than the biggest integer that the shell test expression can handle, hence the error. See http://stackoverflow.com/questions/2965761/find-maximum-positive-integer-value-in-bourne-shell

My answer here is to comment out the block which begins with the integer -gt expression.

# /etc/init.d/vmware stop
Stopping VMware services:
   VMware USB Arbitrator                                               done
   VM communication interface socket family                           failed
   Virtual machine communication interface                            failed
   Virtual machine monitor                                            failed
   Blocking file system                                               failed
ERROR: Module vmnet is in use by [permanent]

Well, that gets rid of one thing. The rest will have to wait since it’s past my bedtime. Sigh.



Too Much Virtualisation

You can have too much of a good thing, VMs included. The scenario is that I am at home using a Windows 7 VM running under VMWare Workstation. From the VM, I am VPN’d into the office where I am connected to my physical laptop, on which I am running vCenter. I have the console of a Windows XP machine open from virtual center but there is no way to log in to it because Ctrl-Alt-Delete just affects the local VM. Ctrl-Alt-Ins the same.

Will post a solution when I find it.

London Solaris SIG February 2013

Last Thursdays talk, SmartOS in the cloud was interesting, not least because it was the first meeting I’ve been to where the presenter was giving the talk via Webex. It worked well. Perhaps one day Ben Rockwood’s descendant will be able to give a talk via tele-holo-vision, as if he were in the room.

The interest for me also came from looking at cloud technologies at the far end of the spectrum from the major vendors offerings, which is where all the professional services work is. Joyent does have a share of this market but I’m not sure if it is more than a Rizla worth. Perhaps I am doing them a disservice.

We were reminded that Illumos is the Opensource continuation of the OpenSolaris kernel, upon which SmartOS, OmniOS, OpenIndiana and Tribblix distros are built. These all have different uses. SmartOS is purpose built for virtualisation and is certainly not a general purpose operating system. It is lightweight, stateless (think esxi), has zones and KVM and essence of Solaris”. This cool sounding OS runs in RAM and is best booted from PXE.

We went into quite a lot of technical stuff, and Ben could have carried on for hours (another time). What struck me was the amount of open source software in the stack. Some of them I remember are Zabbix, chef, jabber, pingdom, openldap, bart. Of course, if you are a FTSE100 company you like to buy your software from other big companies. Personally I like open source software.

One question which I didn’t get the opportunity to ask was regarding the commercials. If you are selling a cloud service (which Joyent are), on the back of all this free software, are there any legal or licensing issues? Do you have to give you users loads of license files to tick? At one point, some open source software disallowed it’s use for commercial purposes. Perhaps I am out of date, or more likely Joyent themselves pay for it in some way.

Anyway, it looks good and definitely worth bearing in mind, and even labbing if time were no barrier.


Solaris for the cloud

I received the usual invite to the Solaris SIG, an event which I attended regularly but have missed for several months. This talk looks particularly interesting being titled “SmartOS: Solaris re-envisioned for the Cloud” to be given by Ben Rockwood.

Given Solaris’ standing in the server operating system league tables, which has been declining for years, thanks to Linux, this talk will be more for academic interest than any practical application. A niche product on a niche product, sad to say. My own experiences over the past five years have not brought me into contact with Solaris at any clients. The great technology it contains will continue to ensure it does not disappear completely and if it can keep bringing new ideas to the party, so much the better for IT in general.


I was so impressed with XenClient Enterprise I bought the company. I mean, of course, I downloaded the product (formerly known as NxTop). The first hurdle was to clear up a disk being used as an ESXi datastore. Vmotioning the disks worked fine to the QNAP which left an apparently empty datastore. ESXi 5.0 refused to unmount or remove it though saying it was “in use” and giving a VC: 637 or similar in the kernel log. Reboots didn’t help and even sshing on to the console and manually removing the .locker directory didn’t help. Didn’t do any harm though! The correct, geeky way to solve this problem would have been to dig through ESXi and find the cause. But I have kids with homework to do so I used the quicker tactic of simply physically removing the disk. ESXi still booted, no problem at all.

Now I’ve installed XenClient Express on that drive, but first the disk controller had to be put in AHCI mode, which the installer helpfully told me. I thought the change would prevent my other O/Ss from booting and VMware station to have a problem but it hasn’t affected them at all. Going for broke, I installed XenClient Enterprise over the top. Will have to wait and see how long the license lasts.

Now just have to persuade grub to give me the option of booting off either disk.


PoC is easy

Working for RedPixie on a VDI PoC at a client recently, we were asked “how have you guys managed to get all this VDI working so quickly when everyone else has tried and failed?”

The answer is that installing software out of the box in a greenfield proof of concept environment is easy. Whether it be vCloud Director or Citrix Streaming Server, installing it is half a days work. But you can’t do that and then say “I’ve installed a cloud for the client”.

All the technical challenges in these types of professional services engagements are to do with integrating the product into the existing environment. Environments that have evolved over decades and are embedded with processes built one way and resistant to be change. I like the analogy of changing the direction of a supertanker. Every client’s environment is different. Understanding it, and the culture, is as important as understanding the product.

What is an application?

Working for RedPixie, I have recently been working on streaming applications. Or, to use Numecent‘s phrase, “cloud paging”. I see a slow but steady increase in interest in these technologies, going hand in hand with renewed interest in VDI (after some initial disappointment). At the end of the day, all most users want to do is run their apps. I don’t think they care how they get their apps, so long as they are fast, reliable and accessible. I don’t think they care what operating system it runs on: Windows, MacOS, IOS, or a gaming console.

It is helpful to define what an application actually is, as opinions may vary. A program, certainly. But not all programs are applications. “Program” is the most general term and in IT we like to be a bit more specific and separate our programs into types.

Most ITers would agree there are a set of programs called “utilities”, which may or may not mean “middleware”. To me a utility boils down to something which has no lasting results. It does not “create”. A utility typically converts formats or moves existing things around. Examples of utilities would be calculators, printer drivers, clocks (Look in the “Utilities” folder on a Mac).

Middleware, to me, implies glue which allows systems to talk to one another: a format converter and communication program.

Databases are a clear subset of programs on their own.

Operating systems are a subset too but are themselves subdivided into onion rings of kernel, drivers, user interfaces and a million other “algorithms”.

Compilers and Editors? Have we reached applications yet? Well I think we’re getting close because to me an application is a program used to create something. The obvious examples are Word, Excel, PowerPoint, Photoshop and newer kids on the block like Prezi.

Games? Browsers? They don’t create anything, but are normally considered applications. Perhaps it’s not so easy to define after all. But then, perhaps we don’t need to try so long as words such as those above evolve to allow us to talk about groups of programs.

What got me started on this post was trying to find out all the “applications” I used, so that I could convert them into streamable packages. This in itself turns out to be non trivial! Where do you look on a PC…Programs and Features? That’s no good, it includes patches, hotfixes and well, programs. And what about the web-based apps I use? There is software which is used in large organisations that sniffs the network and identifies applications  in use by their network packet profile. Hardly surprising that an organisation can’t determine what apps it has if one person can’t.

In summary, “application” is a bad word. It’s too vague, there is no agreement on a definition. Where is the International Standards Organisation when you need an RFC on program types? (I’ll look shortly).

While I’m fairly confident that we are going to be streaming a lot more programs in the future, what we’re going to be streaming remains to be seen.