Ads by TechWords
Subscribe to our e-mail newsletters
For more info on a specific newsletter, click the title. Details will be displayed in a new window.
Computerworld Daily News (First Look and Wrap-Up)
Computerworld Blogs Newsletter
The Weekly Top 10
More E-Mail Newsletters 

The VMware blues

Would you believe a major software vendor released a patch for their flagship program that made it impossible to run their software? Believe it. That's exactly what happened with VMware.

On August 12th, users throughout the U.S. tried to turn on their ESX 3.5 Update 2 and ESXi Server 3.5 Update 2 hypervisors and ... were told that their VMware licenses had expired. That's always a great way to win friends and influence software license sales.

The problem, which came with Update 2 and appeared about two and a half weeks ago, was that VMware developers had left beta code in the ESX 3.5 and ESXi Update 2 patches that expired on August 12th. If you used the patch, like a good little system administrator, and then switched on your virtual servers on August 12th, you found that they were as dead as doornails.

How? How could any serious developer group do this? How could any quality assurance team let this get by them? This isn't rocket-science. It's barely computer science. Lots of beta and test programs have this kind of code in them. It's usually set off by itself so all you have to do is pull that one section out when the program is ready to go.

Heck, it's not even usually part of the real program. It's just a block of code you stick in place of the real licensing check routines.

VMware's new CEO Paul Maritz apologized to users in an open letter immediately after reports of the problem came in and the fix was released. If I'm a VMware user I'm not sure that's enough.

People rely on VMware to run their businesses. This isn't like not being able to get your Google e-mail for a few hours. That's a major pain in the rump, but it won't keep your business from operating. This, I have no doubt, actually kept some businesses from doing anything for about a day.

Can your business afford to be dark for a day? I don't think so.

Now, I have long thought that VMware was in trouble. The company is getting tag-teamed by Microsoft on one side with its Hyper-V in Server 2008 and Red Hat with KVM on the other. On top of that, you have OpenVZ, the open-source side of Parallels and Citrix with Xen lining up to get their turn at VMware.

After this, can anyone really believe VMware has much of a future? Ironically, I recently suggested that one of VMware's basic mistakes is not going open source. Whether or not you believe, as I do, that open source is one of the best ways to develop software, one thing is for certain: There's no way that this bad beta code would have made it into production machines if it had been open-sourced.

Oh well, too late now.

What People Are Saying

Rate this
Rated +6
474 Votes

No one ever did this before...

I can remember a certain Service Pack 1 which blue screened any system that it got onto. The solution was to rebuild the system. After waxing thousands of systems the SP was pulled from the website and released a second time a few weeks later.

And then again it happened just recently. Funny how "the" major software corporation of the world does this and no one said..."Well they are done now!"

Rate this
Rated +2
486 Votes

Although they need to be

Although they need to be publicly ridiculed for this error, its hardly the first time that this has happened with a commercial product.

Not too long ago, a major company (who's name I will not mention to avoid red faces) was horrified to get a stream of support calls form users of one of their products refusing to start with a message about an expired beta. As the earth rotated, calls followed the sun around the globe.

The issue was that beta releases of the product can optionally include a time-bomb to expire exactly 6 months after the build. The intent here is good. Its to prevent beta customers continuing to run beta software once the RTM release is available.

The (outsourced) build staff had not included the appropriate flag to suppress the timebomb when they did the RTM build.

The product in question doesn't use software licenses to limit its use. The company has chosen to make the product freely available, but users must by licenses if they want support, so having the product refuse to run like that was doubly embarrassing.

The final decision from the postmortem on this incident was to rip out the timebomb code completely, leaving beta users responsible for their own fate if they continue to run (unsupported) beta versions.

The real question here is how to manage beta code releases. This applies as much to FOSS as to commercial software. Significant time and effort can be expended trying to track down user problems, only to discover the source of the problem being bugs in beta software. Time wasted on such support issues is something that neither FOSS nor commercial support organizations want to have to deal with.

Rate this
Rated +4
478 Votes

Anti-VMware Bias

Steven,

You REALLY should have led this flame with a disclaimer that you are anti-proprietary software and already have a bias against VMware. As a SCON and Virtualization consultant, I can tell you VMware is doing just fine (thank you very much) and Microsoft is a blip on the horizon right now. Xen and OpenVZ discussion is usually met with a "who, what?" when discussed with clients.

I can also speak from experience when telling you that VMware is still miles ahead of the FOSS projects in terms of both basic functionality (FOSS virtualization still has some basic hiccups) and enterprise-grade services.

I agree that the licensing SNAFU was definitely a huge blunder and enterprises should probably beat their VMware reps up a little and expect more than an open apology. But to take this issue and stretch it into the end of VMware sounds more like wishful thinking than logic.

Rate this
Rated +6
484 Votes

Beneath the usual wild

Beneath the usual wild speculations and overblown conclusions we are used to from SVN, there is a good point here. FOSS is not a cure-all for bugs and goof-ups, but there is one thing I never run into with FOSS applications: licensing lock outs.

I can tolerate bugs in the code, and I can tolerate security flaws to a point, because all software has those things; but one thing that drives me up a wall is a program telling me I don't have a license to run it when I bloody well do. Think of all the time and money that is wasted in the proprietary world making sure that the customer has a right to run the code before it executes. Case in point, WGA. If you want to be a proprietary software company, well, that's your business. Don't make protecting your business model MY PROBLEM. If I've paid you for a license, your software had better not tell me I haven't; and if it thinks I haven't, it better give me the benefit of the doubt and run anyway.

Rate this
Rated -4
486 Votes

Say what??

The first line of this blog really made me jump... "Would you believe a major software vendor released a patch for their flagship program that made it impossible to run their software?"

After reading through the rest of the blog my only thought was that this was about the most uninformed, ill-conceived post I have read in a long, long time. After looking back at prior blogs from SJVN concerning VMware I would have to add "nasty" to the description. It quickly bacame obvious that he has already made up his mind about VMware and is now latching onto ANYTHING that he believes validates it, takes his information and "proof" public and presents in a very inflammatory way. Hmmm.... Sounds a lot like Bush in that respect.

Having been personally affected by this incident at work (4 hosts and 70 vm's), I can say that for the 2 days that it took to get the patch in place this whole thing almost rated as a non-event. Nothing stopped running, everything worked and the few vm's we needed to startup were done so with just a little workaround effort on our part.

Rate this
Rated -65
547 Votes

VMware is so much better

When did open source become about politics and not technical excellence?

I remember when people like Linux because it was similar to UNIX, not because it passed some ideological litmus test.

I run a hosting company. We rent dedicateds all over the place. We tried a Xen virtual server, it was chunky and slow when you first connected to it.

VMWare is a small miracle -- it works, its player is free, and there is a large amount of expertise and support around it.

KVM, virtualbox, and Xen all miss the point -- virtualization is about portability, performance, and cross-platform support.

All that being said, I think that vmware's problem is that the only people seriously using its product are old-school MSCE types trying to host production linux as a guest os -- a stupid move, as a toy for software developers to replicate their server environment at home, and for sales demos that don't have to rely on a customer's internet connection.

I think that 10 years from now we will see virtualization in much the same way we do now - a stupid computer trick with a few edge cases that is utterly irrelevant to production tasks.

A large telecom vendor in my city has decided to take a run at the dedicated hosting market -- their answer to cheap, powerful servers that can be leased month-to-month is to offer expensive 3 year contracts for a vmware nas farm. They want you to upload your vmware appliance to their clound and pay them $100/GB/mo.

Here's my question -- does anyone know how to debug a corrupted vmware virtual disk? I have one lying around here that a customer wanted fixed, it was installed by an idiot nephew, etc.

Rate this
Rated -77
567 Votes

Converting to the OSS model isn't a solution

Just look at the SNAFU with debian and SSL certs! Listen, I'm not bashing the OSS community here, but that debian vuln lasted 20 months, do you really expect us to believe that OSS will make the code get properly vetted? Sure, but only after someone on the outside spots a mistake and releases the details of the flaw, leaving thousands scrambling to find something that is CLOSED source.

I'm all for transparency, but let's be realistic. If a cracker wants to crack your code, he will. It doesn't matter if he has to decompile it first, thats part of the "fun" to them. The real challenge is getting enough SKILLED programmers involved in open source.

I'm more interested in whether the number of skilled programmers involved in (F)OSS is increasing, decreasing, or has stagnated. That would be much more useful, and might convince me that (F)OSS is going to truly be a viable alternative in my lifetime.

Rate this
Rated -3
505 Votes

kill switch

Mike,
the point is FLOSS code wouldn't include a kill switch.

B

Rate this
Rated +4
522 Votes

Im sorry... I have to

Im sorry... I have to disagree with your VMware has no future... No product is at feature parity with VMware, well one Virtual Iron.
When you are virtualizing thousands of servers Hyper-V, Xen and any of the others don't even come close.
Vmware screwed up. They know it, the world knows it. But they were very straight forward with their customers.
I use and like open source software. But the open source community should put up or shut up. When they can match VMware feature for feature they have an argument, but until then they are tilting at windmills.

Rate this
Rated +9
523 Votes

in defense of VMWare

As a developer, I *could* imagine how this would sneak into code, and it would be hard to test for.

In addition, we saw how signed certificates were rendered completely useless for 2 years on
Debian (aka Ubuntu) systems when someone
effectively converted 1024 bit keys into 15 bit keys.

The comments here would seem to be by someone who has not really worked in depth in the software field.