I seem to end up evaluating a lot of software. This requires me to constantly install all kinds of things on my system. It creates a huge clutter and I spend a lot of time during the install process, and if I don't like it, then removing everything I've done. Much of my evaluation tends away from the features of the software being evaluated and toward how difficult it is to install. I'm sure I miss good software which may have actually been a better choice, because of this startup cost.
With the advent of VM software like VMWare Player and VirtualBox, it would be much easier to sell someone like me your software, if you just provided an image that I could load into the VM and run. I'd be looking at the features almost immediately rather than fighting with which revision of whatever. The VM would take care of all of this for me.
Am I missing something, or should vendors and OSS start distributing VMs for their wares?
Most of my evaluations are for server side software installed on Linux, so OS licensing is not the issue.
VMs require that the operating system have a valid license key. For free operating systems this wouldn't be an issue, but if you're developing for something like Windows machines, each time they send out a demo version of their software, they're sending out a license key that they would have to pay for.
This would be incredibly expensive for most companies.
The only downside I would say IMHO is the size of the images, if say you have a 20 MB application, do you really want to download/transfer an entire OS just for that application.
I would say a better approach would be to have a ready to go VM and then you simply take a snapshot (on Virtual Box, I assume similar feature exist in other players)
Then simply install the applciation inside your sandbox environment, and then just Zap it when done (i.e. return to your Snapshot)
Darknight
This can be done for softwre that runs on open source platforms, and VMware have a library of images which do just this (though the images that are used for evaluating commercial software is generally for infrastructure-type things that have very, very complex installation requirements):
http://www.vmware.com/appliances/
However, if the software is for the Windows platform, you don't really have the opportunity to do this, as Microsoft's Windows licensing would prevent it. Unless, you're Microsoft, of course, in which case you can in fact do this - and MS has done this to permit easier evaluation of such software as Visual Studio, SQL, and many others:
http://technet.microsoft.com/en-us/bb738372.aspx?ppud=4
Novell has an appliance builder called Suse Studio that lets you pick the software you want, it builds out a VM with the software (and dependencies, etc) for you. You can then try out the VM, download it, etc.
Whether the software you're looking for is available or not is a different matter.
Disclaimer: I work for Novell (though not with the Suse team)
But yes, if you can deal with the OS licensing issues, or possibly host trial environments yourself, this is a very effective way for a vendor to demo their app. The problem is that all vendors don't always have the infrastructure (or lack the awareness) to do so.
Microsoft provides fully-provisioned VM's for time-limited trials of their software. So if you want to trial select Microsoft products in that manner, you can do that today.
There is no sign, though, that Microsoft will make this available to third party Windows software vendors.
In the SaaS (Software-as-a-Service) world, you can get fully-provisioned virtual servers that include Windows and your software of interest on a pay-as-you-go basis, based on both Linux and Windows. For example, see Amazon Web Services
For windows, you may be better off developing a portable application that runs from a usb key. That is how Embarcadero distribute All Access. I received a 4 gb usb key that contained multiple applications. Most could be run straight from the key without installation. I believe Embarcadero will be licensing the technology at some stage.
If you are using a programming language such as Delphi or C++ with little in the way of external dependencies, a portable application is straight forward to develop. For .net, it is much harder, but can be done with Mono, or something like Virtual Application Studio.
Related
We are looking for an automated testing software for our web application. We need to come up with a solution or software that our non-it staffs could write test cases as well as the developers.
For example I've run through some of them such as: SmartBear, National Instrument and IBM. Most of these guys are MS Windows based or commercial Linux distros which remove them from our list since we are all Debian based.
Any recommendation or guideline would be much appreciated.
Ps. We don't have any budget limit!
You're going to have a hard time getting tooling for non-technical testers to build test cases if you limit yourselves to Debian OS for developing and running the tests on. There's no reason you couldn't have a few Windows system to manage your test suites from -- those would run against your web site just fine, regardless of what stack it's hosted on. That would open you up to the tools you mentioned (and Telerik's Test Studio, the tool I help promote).
Those Windows systems could easily be run via whatever virtualization host you prefer, so you wouldn't even need physical systems to deal with that. You could easily share the same source control repository as your devs, too, since nearly every decent SCM has Windows clients.
If you're unwilling to consider having a few Windows boxes around for your testing, then you'll need to have a look at getting all your testers proficient in APIs and frameworks like WebDriver and Robot Framework. The Pages gem from Jeff Morgan (#chzy) in Ruby would be another option, as would Adam Goucher's Saunter (in Python).
Practical uses of virtualization in software development are about as diverse as the techniques to achieve it.
Whether running your favorite editor in a virtual machine, or using a system of containers to host various services, which use cases have proven worth the effort and boosted your productivity, and which ones were a waste of time ?
I'll edit my question to provide a summary of the answers given here.
Also it'd be interesting to read about about the virtualization paradigms employed too, as they have gotten quite numerous over the years.
Edit : I'd be particularly interested in hearing about how people virtualize "services" required during development, over the more obvious system virtualization scenarios mentioned so far, hence the title edit.
Summary of answers :
Development Environment
Allows encapsulation of a particular technology stack, particularly useful for build systems
Testing
Easy switching of OS-specific contexts
Easy mocking of networked workstations in a n-tier application context
We deploy our application into virtual instances at our host (Amazon EC2). It's amazing how easy that makes it to manage our test, QA and production environments.
Version upgrade? Just fire up a few new virtual servers, install the software to be tested/QA'd/used in production, verify the deployment went well, and throw away the old instances.
Need more capacity? Fire up new virtual servers and deploy the software.
Peak usage over? Just dispose of no-longer-needed virtual servers.
Virtualization is used mainly for various server uses where I work:
Web servers - If we create a new non-production environment, the servers for it tend to be virtual ones so there is a virtual dev server, virtual test server, etc.
Version control and QA applications - Quality Center and SVN are run on virtual servers. The SVN box also runs CC.Net for our CI here.
There may be other uses but those seem to be the big ones at the moment.
We're testing the way our application behaves on a new machine after every development iteration, by installing it onto multiple Windows virtual machines and testing the functionality. This way, we can avoid re-installing the operating system and we're able to test more often.
We needed to test the setup of a collaborative network application in which data produced on some of the nodes was shared amongst cooperating nodes on the network in a setup with ~30 machines, which was logistically (and otherwise) prohibitive to deploy and set up. The test runs could be long, up to 48 hours in some cases. It was also tedious to deploy changes based on the results of our tests because we'd have to go around to each workstation and make the appropriate changes, which was a manual and error-prone process involving several tired developers.
One approach we used with some success was to deploy stripped-down virtual machines containing the software to be tested to various people's PCs and run the software in a simulated data-production/sharing mode on those PCs as a background task in the virtual machine. They could continue working on their day-to-day tasks (which largely consisted of producing documentation, writing email, and/or surfing the web, as near as I could tell) while we could make more productive use of the spare CPU cycles without "harming" their PC configuration. Deployment (and re-deployment) of the software was simplified, since we could essentially just update one image and re-use it on all the PCs. This wasn't the entirety of our testing, but it did make that particular aspect a lot easier.
We put the development environments for older versions of the software in virtual machines. This is particularly useful for Delphi development, as not only do we use different units, but different versions of components. Using the VMs makes managing this much easier, and we can be sure that any updated exes or dlls we issue for older versions of our system are built against the right stuff. We don't waste time changing our compiler setups to point at the right shares, or de-installing and re-installing components. That's good for productivity.
It also means we don't have to keep an old dev machine set up and hanging around just-in-case. Dev machines can be re-purposed as test machines, and it's no longer a disaster if a critical old dev machine expires in a cloud of bits.
I am trying to get a better idea on this as so far I have had mixed answers in person.
I am a solo dev in a 5 man IT dept for a Health Care related business. My developer machine is running Win 7 RC1 (x64) but my users are all running Win XP Pro (x86). Is this a big deal? Whan kind of pitfalls should I be aware of? Is having a VM of the user image enough?
Should my environment completely mirror my end user's?
Your development environment doesn't need to mirror your user's environment, but your testing environment certainly should!
Having a VM of the users image for testing would probably be good enough.
First and foremost, as a developer, your machine will never look like your client's machine. Just accept that.
You will have tools and utilities installed that they won't have. That will fundamentally change the configuration of your machine from the outset. You have DLLs, applications, services, and possibly drivers installed that your users have never even heard of (and likely never will).
As far as the OS is concerned, Win7 and WinXP, despite claims to the contrary, are not the same animal. Don't believe the hype. Having said that, don't believe the anti-hype, either. Just be aware, as you well should, that any piece of software developed under one version of an OS is not guaranteed to behave the same way under another.
The short of it: Yes, it's important that your environment is different. Should you panic about it? No. Should you account for it in testing? Absolutely. As rigorously as possible.
Is this a big deal?
Yes, it is. You have an OS 2 generations ahead of the one the users have, including you are running a non-release version.
Whan kind of pitfalls should I be aware of?
Depends on what you are developing. Some libraries may be missing that you already have, then the versions may be different etc.
Should my environment completely mirror my end user's?
Not necessarily, but you definitely need to have a testing environment that corresponds to the one the users have.
If you were developing web applications, all that would not have been an issue (well, unless you used some fancy fonts that are not present in a clean OS by default).
Its unreasonable to develop on exactly the same type of system as your users. If nothing else, your life is made much easier by installing all sorts of developer tools your end users have no reason to install. I hear Visual Studio in particular likes to squirrel a number of potential dependencies onto a system.
However, you do need to test on a system more inline with that of your end users. If you have access to an image of such a system, your VM approach should be sufficient. If nothing else, you should try for a staged (or better yet, beta/trial) release so as to avoid pushing a completely broken app out the door.
In short, don't fret about the development environment but put some thought into your testing one!
It's effectively impossible for any two machines to be set up the same, so development and production environments will always be different. One advantage in them being VERY different is that you will be more aware of possible deployment problems.
The type of environment that you need to use for testing entirely depends on what you're developing.
If you're writing web applications, having a VM with the user's standard image should be more than enough (just make sure the VM contains all the browsers your users might be using). Web development is much easier in this respect (I'm also running Windows 7 and have a couple VMs to test various environments)
If you're writing a full desktop environment, you'll probably want to ask for a second computer that you can test on (even if just to test before a final release). I say that because of differences in hardware. Just imagine if something runs fine for you, but slows down the users computer so that everything else is unusable. Opposite of that, you might spend hours trying to make something faster in the VM whereas running that on a users computer might run just fine.
What are the key use cases for the use of virtualization -- that is, running one or more "virtual PCs" using software such as VMWare and Microsoft Virtual PC -- for software development?
Also -- are there other instances/uses of virtualization that aren't covered by my definition above (use of a tool like MS Virtual PC or VMWare), and that are useful to developers?
My impetus for asking is this StackOverflow comment by Metro Smurf asserting "You'll wonder how you ever developed without it!", regarding use of virtualization.
(Please include just one use case per response. Thanks!)
Application testing in multiple environments is one obvious use of virtualization that I'm aware of. Testing your application on other operating systems (without requiring additional physical computers to do so), as well as testing that involves software that generally only allows you to install a single version on a given machine (such as the Internet Explorer browser; running both IE6 and IE7 on the same machine is not an officially supported configuration), are good candidates for virtual machine usage.
If your build-server is running in a VM, you can make a snapshots of it for every software release in order to be 100% sure that you can recreate the build environment (in case you want to make patches to old releases, for example).
If you set up snapshots of your development environment (and back them up) it can be very easy to resume productivity if your computer breaks down. When your machine breaks down right before your release - and you can resume immediately with all your tools installed and configured, it can be a lifesaver.
The simplest case which applies to my current situation is that we have a complex client-server environment and with virtualization every developer can very quickly get a baseline set of operating systems to deploy their local build to and verify end to end functionality.
Locally you have your dev box, and N client boxes which get re-initialized as fresh OSes each time you want to try a build. Essentially it's the test environment equivalent of a 'make clean' where even the client workstation gets replaced with a new OS.
Quickly distributing environments between team members is a very nice use case to for virtualization especially if you have a lot of various components, tools, etc.. This can save you a ton of time with new hires, contractors, or other individuals who need an environment quickly.
Many presenters use a VM for presentations - it allows them to revert immediately to reset the presentation for the next day, transfer all presentation materials quickly between computers, and not have to show your attendees your messy My Documents folder.
Using virtualization for sales activities is also a great use case. You can take a snapshot at a particular time that you can save as your demo baseline. Then once you run through the demonstration and change the data, etc. you can restore back to your previous baseline for future demonstrations. You can also capture multiple baselines and pick and choose which baseline best fits the upcoming demo.
Test environments. If you have more than one setup that a system needs to be targeted for (e.g. Windows & Linux, XP & Vista) then a machine with lots of RAM and VMWare (or on of the others) is a good way to manage the environments.
Another is developing on one system and targeting another. For example, at one point I did some J2EE work on a workstation running Linux where the client was I.E. 5.5. A VM with Windows 2000 and IE 5.5 would let me test the application.
Reasons I use virtual machines for development.
Isolate different development environments.
Testing environments.
Easy recovery due to computer hardware failure/upgrade.
Ability to "roll-back" changes to your development environment if something corrupts it.
Currently, I am using VirtualBox for my VM setup. I used to use VirtualPC, but I REALLY hated not having any type of "snapshot" feature (like VMware and VirtualBox have).
We develop software for use in our SaaS application, our production environment has a large number of servers and their software environment needs to be absolutely predictable; we can't have ANYTHING installed extra, or absent from our development machines.
Moreover, our application requires a number of different server types in order to function properly (at least 7 last time I counted); mostly they can't be installed on the same (virtual) machine - at least, not without violating the "same software as production" requirement.
In order to have a consistent environment, it's necessary to use VMs. I don't know how anyone ever manages without them.
Snapshots and rollbacks are nice too, but I use them only occasionally (really useful during installation / upgrade tests).
Suppose you're developing a new version of your software, and checking that the upgrade from the previous version works correctly... how long does it take to do a test cycle without being able to rollback the box? Do you have to reinstall the OS then the old version? Can you guarantee that the uninstall really uninstalls everything?
Being able to test/retest your deployment process is a huge savings.
Developing Add-Ins for different versions of Microsoft Office (using Visual Studio Tools for Office).
My main work machine has Office 2007. When I work with Add-Ins for Office 2003 I use a virtual machine with Visual Studio and Office 2003.
I'm suprised that nobody has mentioned the VMware record/replay feature (awesome video demo) which is great for debugging.
I have a headless server running ESXi which runs various machines for building installers (so I don't have to give up processing power on my desktop), automated testing (server is faster than any desktop) and various test environments (about 20 different configurations) so that the support team can easily jump onto a configuration that closely matches a customers system.
When you have one really beefy server running VMs that can be shared between support, test and dev teams, you introduce huge cost savings. In all, we're running ~25 VMs on ESXi (dual-quad core Xeon 2.5G + 8Gb RAM) shared between 5-10 people, some of the developers use Virtual PC and then I use VMware Workstation on my desktop. All of the Mac users here use VMware Fusion as well
I am surprised that no one has mentioned the benefit of increased security by isolating, for example, the database server and web server in different VM's.
Some server applications can use VMs too. When one vm is not used much, the server can locate the resources to other vms.
Some sort of test environment: if you are debugging malware (either writing or developing a pill against it) it is not clever to use the real OS. The only possible disadvantage is that the viruses can detect that they are being run in the virtualization. :( One of the possibilities to do it is because the VM engines can emulate a finite set of hardware.
I recently build myself a semi beef up PC (Q9450, 8GB DDR2 1066, 1TB HDD, Dual 8600GT, Vista Ultimate and Dual 22' Monitors) and I'm evaluating whether i should develop on a VPC/VMWare session on top of Vista or not?
One benefit I can see is that I can run the same VM on my Vista laptop so my development environment is the same on any of my machines. I also plan on purchasing a MBP before the end of the year as well.
Found a couple of articles online that semi-help Here
Any other thoughts would be really appreciated?
For webdevelopment I like to have the serverpart separeted out into a VM. My current setup is a Macbook Pro with several Debian VM's inside. I like the isolation aspect of it. I can try new software on the servers and have the ability to revert them back if something is messed up.
I do the programming via network-share (samba) in Textmate on the host system.
Another advantage of a VM is having a clean installed base. I use my desktop and laptop for lots of things aside from development. You never know when a piece of software you install is going to conflict, or if the little tweaks and what not you play around with are going to trash your OS. Reinstalling/configuring all your tools so they are exactly the way you want them can take quite some time. If you have a backup of your Development VM Image you can mess up your PC as much as you want but still be able to code without downtime. It also allows you to run Win/Visual Studio/Etc on a box that you would otherwise prefer Linux or MacOS on.
You can also make multiple copies of the same Image and use each one for a separate project.
Being able to transition between a laptop/desktop/server/remote connection, and always be in the same environment is also very helpful.
One problem I found (at least when using VMWare Server) is that no matter how fast your machine is, the screen refresh rate is still around ~30hz. That makes for a slightly unpleasant experience after using it for a while.
Where I'm working at now I use a VM for all of my development because I don't have admin rights to my base copy of XP.
Pros:
I like using a VM's because it give you some flexibility - you can switch between machines - have programs running on both and have a cool environment to work on.
Cons:
You have to boot up multiple operating systems. This takes time, memory and resources.
Clipboard operations on VM's can be interesting at times. Sometimes copying to clipboard does not work or gets mixed up between VM's. (Using VMWare).
File operations can be interesting when you plug in USB drives and other external devices. VM's sometimes do not see the devices, sometimes it does.
If your VM image become corrupt - you can easily loose everything in it.... unless it is backed up.....
It's great for presenting development talks, you can revert to a snapshot and give the talk from the exact same starting point each time.
Bulk-up your RAM on your future MacBookPro if VMWare will be used. I haven't (yet) and the performance with several other (mac-side) apps open really starts to feel sluggish.
All the best.
I was doing some work with Visual Studio recently with a Windows XP vm on Linux and somehow the guys who made the vm (vmware) made the windows machine actually run faster. We did some time tests to make sure and it wasn't major, but a few things (autocomplete for example) really did pop up faster.
If you are on Windows, Virtual PC is pretty decent for development work. VMWare Virtual Server is not really designed for use as a desktop and you will get very tired of it with any prolonged use. Sun's VirtualBox is another option competing with Virtual PC. VMWare has a workstation product but it is not free.
Typically, I do development on the real desktop (non-virtual) and then deploy or test to virtual machines which I can snapshot and roll back easily.
For a long time, we were developing on very early versions of Visual Studio 2005 and the associated .Net bits that went along with it. To protect our real machines from the various problems associated with pre-release software, we did all of our development work inside virtual machines. It worked amazingly well. I've been considering moving back to that model as it makes upgrading the physical hardware a snap (not to mention making it easier to deal with hardware failures by just replacing the entire machine): you just copy the VM image over.
On my current machine (A Core2Duo with 4GB of RAM), the performance drop when running one VM is almost not noticeable. Running two VMs, however, is painful.
I also can't figure out how to get VMWare Server to work across two monitors well.
I wouldnt want to develop in a VM so much as test things in a VM. For instance, it might be nice to set up a couple VM's to emulate an n-tier architecture, or a client-server setup or finally simply to test code on multiple OSs
It depends what you are developing and in what language.
VM's tend to take a fairly hard hit on disk access, so compiling may slow down significantly, especially for large C/C++ projects. Not sure if this would be such an issue with .NET/Java.
If you are doing anything that is graphics intensive (3D, video, etc) then I would steer clear of a VM too.
I don't know if it is so useful as a development platform unless you are doing something that ties into software you don't want to have installed on your regular working machine or that needs to work around a certain event that you need to be able to reset on a regular basis. It can also be handy when you are working with code that risks crashing your computer as it will at least only crash your VM.
It is brilliant for testing different configurations and setups- working with installers and so on, that is where virtualisation really shines as far as I am concerned, being able to roll things back whenever you need to and run through stuff repeatedy is amazingly useful for identifying problems before your end users run into them.
While doing development at home, I have to VPN into my company to be able to use the collaborative tools that are on the intranet. I also have a desktop + laptop that are hooked together through Synergy.
The problem that I have is that our VPN software wants things to be so secure that it will force all network routing through the VPN gateway -- even if I'm using additional NICs to network my desktop and laptop through a separate private network. The end result is that I can't use Synergy between my desktop and laptop and VPN into my company at the same time.
The solution suggested to me by a co-worker was to setup a VM instance on my desktop and use that for all my VPN needs. Works like a charm!
Speaking from personal experience developing java in an Ubuntu VM on Windows 7, I've found this to be quite productive. Mainly because my local IT support on the ground supports Windows 7, so I can do things like access all the local file shares and printers in Windows, and then config my Ubuntu VM to my heart's content.
Huge productivity benefits around remote access and desktop sharing. Windows allowed me to very quickly and easily use tools like logmein.com and join.me to access my machine from home and to desktop share the VM with other people in the company (both work seamlessly with the VM in a nearly full screen window). Neither of these services are supported on Linux, and I wouldn't want to deal with all the associated VNC/X setup and network config on Ubuntu.
My machine is fairly beefy. Quad core, with 16Gb RAM - 8Gb for the VM. Java dev in the VM is pretty quick.