LOSUG – UKOUG September – and other things

Posted on by
Reply

LinkedIn helpfully tells everyone it is 99 days since I wrote a blog post. Thanks LinkedIn. A few non-IT projects have been in progress (and summer holidays as well).

On the IT side, I have recently managed to download ESXi5.0 and install it easily on a USB stick. I’ve imported guests from my datastore and got the appliance based virtual center running. All very easy, I am happy to report!

Touching on IT, I have built a VM and installed Magento e-commerce web shopping software for a related project.

For a client, I have written a few simple load generating scripts in perl to help testing a virtual environment. It’s not as easy as you think to generate memory load. Just assigning a large chunk of RAM doesn’t work as the host operating system (Linux in this case) notices the memory is not being accessed and pages it out over time. You can see it happening in a nice graph in virtualcenter. ESXi will probably try to do something clever too, even if the O/S doesn’t. To keep RAM in use, you need to continuously access it, which I did with random accesses into an array.

I made it along to the September OpenSolaris User Group meeting where Nick Todd gave a talk on the Solaris linker and Alastair Lumsden gave an update on the OpenIndiana project (which I am downloading now).

Nick’s talk was entitled “The Missing Link”. Apt, as we all tend to take that step for granted but there’s a decent amount of engineering in there. It was interesting to note that even in the days of card decks, you nearly always had to “bracket” your deck with pre- and post- instructions to tell the machine what to do with your deck. That concept lives on in the elf file format where executable code is prefixed by crt1 and crtn code.

There are two main aspects of linking: the link editor and the runtime linker and a set of Solaris commands to aid development and debugging, not least of which is “elfdump”.

This talk was fascinating, not least because it simply reminded us that this goes on and contained plently of tips and places for further reading. (I will insert links when I get them).

Alastair gave an update on OpenIndiana (1 year old!) and the upcoming 151a stable release. OpenIndiana is based on Solaris 11 express and Illumos but future releases will fork from Oracle and enable innovation and new features. KVM has been added and GCC will be used as a compiler. The combination of these various technologies: KVM, Qemu, Illumos, ZFS, Crossbow, Zones, Dtrace is a potent mix.

Not least the consideration that the source code is freely available and if you are serious about security there is no substitute for examining and compiling the code yourself. Particularly with recent hacks against the Linux kernel.

LOSUG – UKOUG June: Phil Harman – Virtualisation Fever – Where do you draw the line?

Posted on by
Reply

I have a theory that the harder it is to make notes at a talk, the better the talk is. Phil Harman’s talk “Virtualisation Fever – Where do you draw the line?” (a whimsical look at virtualisation) was indeed hard to make notes on and all the more interesting for that.

Phil’s inimitable style keeps you guessing what is coming next and we started with some Hammond organ music (were we in the right talk)? Virtualisation is not new, and it’s not just evident in IT. The B3 Hammond organ is a virtual instrument. It replicated the function of the telharmonium of 1893 which weighed 200 tons and played virtual organ music to people on hold on the phone. Organs themselves, which were introduced in churches hundreds of years ago, were themselves virtual choirs. Stops on organs give hints: Solo vox humana.

Moving forward you have the Roland VR-760 (part of the v series) which is a virtual Hammond organ and you can even get V-drums. There is even a virtual grand piano whose material parameters you can program from a PC. It’s an increasing virtual world and sometimes we can’t tell the difference. Have we gone mad?

We move to a demo of a virtual BBC Micro; great stuff. A program is loaded to calculate if a given number is prime, but it is a bit slow. Partly this is due to the emulator matching the speed of the beeb and partly due to the revelation that the beeb emulator is running in a JVM, in a Solaris zone, in Solaris, in VMware, on a Mac!

Even in a JVM the BBC emulator runs 20x faster than the original, but on a 2.4GHz processor, there is still a factor of 100 in speed missing. A factor caused by virtualisation layers.
We can speed up our program by running it in an obfuscated C interpreter and we can speed it up even more with compiled C code. A speedup from 50 minutes to 15 seconds.

This demonstrates two of Phil’s points: just because something is possible, doesn’t mean it is a good idea and the further away your program gets from the hardware, the more difficult it is to schedule, illustrated with an example of the introduction of threads and lightweight process scheduling in Solaris. Resources have to be scheduled at each level of the stack and the bottom layer can’t see what the layers above are trying to achieve via scheduling. Another way of saying if you want something to go fast, get it as close to the hardware as possible, I suppose.

All the power of modern hardware encourages laziness, inefficiency and waste. The speed of processors hides a multitude of bad programming. I find this very ironic given the argument that virtualisation is driven by goals of greater efficiency i.e. not having a large number of lightly used servers.

Some aspects of virtualisation came in for praise: zones, Ultrasparc T, VMware fusion, Cloud computing, immutable service containers and Exadata.

We have to question why, if a virtual solution is not more efficient, why are we doing it.

By the end of the talk we had headed into the land of Exadata and efficient methods for running databases. Not as whimsical as the start but a fascinating talk!

Thanks Phil, I wish I could have captured more than a fleeting impression of the talk.

Bitten again

Posted on by
Reply

This is another “note to self”. I’ve been bitten again by a vCloud Director problem that I first encountered last November. Laughably, when I was Googling for a solution I turned up the Vmware communities page where I posted the problem before solving it myself.

I’m not totally sure what triggers this but various “odd” symptoms eventually boil down to the error message: “The operation failed because VirtualCenter server is not connected. vCD, as noted on the communities post, is not good at spotting this error and will happily carry on until it needs to call the vsphere API to do something at which point it generates an error like this one:

One odd vCD error

A similar error might appear as follows:

To fix this, go to the “Manage & Monitor” pane of the System tab. Click on “”vCenters” under “vSphere Resources” and you will see the not connected error under the Status column. Simply right-click on the virtual center and choose “Reconnect vCenter”. That should fix all the odd problems.

VDI on demand – exp.1

Posted on by
Reply

My first test was pretty successful! I had exported from my virtualcenter my old Portege 7200 to an ovf file on my current laptop with VMware workstation. This I managed to import and run. Cool! The only fly in the ointment being the bug in ovftool which gives an error when parsing the ovf file. I got round this by upgrading to the latest version of ovftool and running it manually to convert the ovf to a vmx file and then importing that.

The next challenge is, of course to move the VM I used today back to my VC, to make it truly portable. Changing formats multiple times is not practical so I need to see what I can do with a USB stick, which harps back to my datastore on a USB stick search of last year (still not supported by VMware). Should still be able to use one to store the VM files though.

Next experiment.

Briforum Day 2

Posted on by
Reply

First of all, here’s a scan of the programme to help me remember what I went to:

The most thought-provoking session being Brian’s Myths and Realities (consumerisation of IT) which led on from his and Harry Labana’s talk the day before and the morning roundtable. I realised that a lot of the problems could be solved by my USB idea (see below).

I am currently working on a cloud project for a client. For me, the goal of cloud computing is to make IT another utility. You go to the office, fill the kettle from the tap, plug the kettle into the power socket and plug your USB sticks into the hypervisor which the IT department supplies along with a nice big screen and keyboard.

What you have on your USB sticks is virtual machine which contains your desktop: O/S, apps and even data (maybe another USB stick for that). The hypervisor loads and runs your VM, which is in a standard format and you log in to your familiar desktop and share your data using an internet connection, gmail and dropbox. Don’t forget, I’m assuming that in a few years time a pendrive will contain terabytes, but even today you could carry a pretty massive hard drive easily.

I already have an old Toshiba laptop running XP which had 128M and I P2V’d when I bought a new machine (which is also old now). I’m going to export that to a USB stick and do an experiment soonish to see how portable it is (using VMware products for starters).

Finally thanks to Jim Moyle (who gave a fascinating, nay shocking talk on Windows IOPs) and for inviting me to “Ye Olde Cheddar Cheese” in Fleet Street along with some of the other presenters and Techtarget folks where we ended up in the basement (surprise surprise) and from where I parted company very late as they headed off to another venue.

I trust everyone made their flights and I look forward to another meet-up.

 

Briforum Day One

Posted on by
Reply

Extra early start for me to get from the South East of London to Stamford Bridge for 8am but luckily made perfect connections with all the trains so got there in plenty of time.

I have to say first that the venue is not the best. Certainly not a patch on the Nemo centre in Amsterdam. This is largely due to the fact that the “Great Hall” is probably the most depressing space I have ever been in. It’s a long thin room and it’s ALL BLACK. I had to go out for half an hour at lunchtime just to get some daylight. Apart from that the rooms are “ok” but there is no lab either and consequently no computers to use. Old fashioned perhaps.

I was slightly disappointed by the breakouts. Ian Parker’s talk on Xperf and Ron Oglesby’s talk on VDI were the best for me but the others seemed lacking. Reflecting the state of the desktop industry, much of the breakout material, particularly those involving Brian, have been on crystal ball gazing at a point in time where Windows 7 is being rolled out and VDI is not being accepted as the ultimate solution for desktops. Of course, VDI isn’t the ultimate solution for desktops as there is a growing realisation that server virtualisation and desktop VDI are very different things.

Being a cloud guy with a server background I was disappointed that cloud didn’t get more of a mention. In many ways it is “implied” by the realisation that apps are evolving in the environment of the cloud and the desktop will ultimately become a human interface device for accessing new apps as traditional operating systems and brick based applications are replaced over time. Traditional operating systems and apps developed for them will be left as niche products in an increasingly complex world where more and more choice of apps is available.

Key is the fact that younger users have grown up with IT and they will not be restricted by the whims of old-school IT management. In the end I don’t think this is a battle that IT can win. On the one hand users have enough know-how to get and use the apps they want and on the other, business managers want to cut costs and take a more utility approach to IT. IAAS and PAAS. Implementation of public and private clouds will be the tipping point.

And what are Microsoft going to do with Skype?

Briforum 2011!

Posted on by
Reply

I’m looking forward to BriForum London Tuesday and Wednesday this week in Stamford Bridge.

If you are going and would like more information about RedPixie, our products, consultancy, the cloud or meet me for a chat, drop me an email at duncan.baillie@redpixie.com or duncan@p2vpro.com.

Can we make better use of USB flash storage?

Posted on by
Reply

I’m a big fan of USB flash drives. You can buy a 256G flash drive today which is remarkable given that until 2005 most personal computers were still shipping with 2.88MB floppy drives. The capacity will certainly grow and the cost will certainly come down. It is not too hard to imagine a flash device weighing a couple of grams with 1TB of capacity in the not-too-distant future.

The question is, what can we use them for? What existing IT problems can we solve with these devices and what novel applications may spring up?

A feature of modern life is almost everyone carries with them a mobile phone. In many cases this is a smartphone so it is not a stretch to say everyone carries a few Gig around with them. To most people that just means they can play better games or store more music. But there seems to be an opportunity there to combine mobility and massive storage to solve more general IT problems.

As I work for RedPixie, a technology company specialising in virtualisation, I have ample opportunity to get involved with leading edge IT. In an increasingly “virtual” and “cloudy” world, particularly with reference to remote working, moving data around quickly is key, particularly “personal” data. In a Windows world that would be referred to as the “profile”. I see that as the chunk of config which makes your computer usage personal. An organisation may well have 2000 identical computers for people to use in terms of hardware and operating system (even apps). But you can guarantee that after a very short usage period , no two computers will be the same just because of the way an individual configures settings to suit themselves.

This whole problem comes under the banner of “profile management” and it’s pretty big business with several products in the marketplace targetted at storing, moving, applying and changing profiles.

I have a theory that a portable mass storage device seems like just the ticket for storing large amounts of personal data. I posed this question to my learned colleagues over a beer one evening. I was told it was a question that only a Unix person would raise :-). The key point is that as much as possible needs to be as close as possible i.e. compute, data and profile. If your compute and data is on the other side of the Atlantic it doesn’t help you that you carry your profile with you, you still have a latency/bandwidth problem.

But, my point is, with a 1TB flash drive you take EVERYTHING with you: O/S, apps, data and profile. All you use at the remote location is the compute hardware and human interface devices. This strategy is made easier by virtualisation where the O/S you carry can be a VM running on a corporate virtual platform.

What do you do, I said after a couple of beers, if your base is on Mars? The maximum latency there is 20mins. You have to take it with you. Much hilarity ensued but that’s another story.

Baggage

Posted on by
Reply

A colleague asked me the other day how to increase a Linux filesystem on an Esxi guest. This procedure (I hope to post a solution in addition to the others on the web) is in no way simple. And it should be.

The reason that it is not simple is that Linux, and operating systems in general, are carrying so much baggage related to physical infrastructure. LVM, which is standard on Linux to manage disks comes from a lineage of managing large numbers of physical disks. It was not designed to cope with a situation where a machine has one disk and that disk can suddenly increase in size from 20G to 30G. This is a common scenario in a virtual world.

Even disk partitioning itself can be called into question. Take fdisk, which stands for “fixed disk”. Wikipedia informs us that this was introduced by IBM in 1983! Partitioning came along mainly in response to increasing disk sizes and the need to make one physical disk appear as multiple logical disks for convenience.

All this technology was designed to cope with changes in physical capabilities and because we take it for granted it is still baked in to our virtualisation stack. To increase a filesystem on a Linux VM after changing the disk size you need to delete the partition, re-create it (with fdisk); use lvm commands (pvresize, lvextend) to make use of the extra *physical* extents and then grow the filesystem. Jeez.

Now, where did I put my cloud operating system?