Related
At work, we (2 other developers and me) develop all of our code on a single internal machine (via network file sharing). This machine runs our development environment (NGINX, Apache, PHP, MySQL, Memcache, Gearman, etc), which is unruly to get installed on a non Linux environment.
We're getting a few more team members (one remote) and I am looking for a better way to manage a common development environment (our developers currently use Windows, Mac, and Linux).
How does your team create a common development platform? A few things I'm thinking about:
Same setup (a single machine where we write code), but make it external (maybe spin up a cloud server).
Force everyone to use Linux and replicate the environment on thier development machines.
Create a virtual machine that replicates the environment and develop inside a VM.
I'm curious what others are doing... Any thoughts on best practices?
In my experience, I've used virtual machines (VMWare) and it has worked pretty effectively in creating the same environment for large teams with many developers.
I use Linux as primary OS. I need some suggestions regarding how should I set up my desktop and development. I do work on mostly .Net and Drupal, but some time on other lamp products and C/C++, Qt. I'm also interested in mobile (android..) and embedded development.
Currently I install everything on my main OS, even I use it a little. I use VMs a little (for lamp server).
Should I use separate VM for each kind of development (like one for .Net/Mono, another C++, one for mobile and one for db only, one for xyz things etc)
Keep primary development environment on main os and move others in VM.
main os should be messed up
keep things easy to organize (must)
performance should be optimal (optimal settings for best performance of components)
I'm interested to know how others' are doing.
There are both pros and cons with VM's.
Pros:
portability: you can move image to
different server
easy backup (but lengthy)
replication (new member joins team)
Cons
performance
hardware requirements
size of backups (20-40 GB per VM ...)
management of backed up images (what is the difference is not obvious)
keeping all images up to date
(patching / Windows updates)
For your scenario, I would create base VM with core OS and shared components (Web server, database), replicated it and installed specific tools into separate VM. If you combine tools within VM, you may end up with same mess as in case of using base OS - the advantage is that it is much easier to get rid of it ;-)
Optimal performance != using VMs
if you need to use VMs anyway, then yes: it could be better to use a separate VM for each thing that need one, unless you need more than one at once
Now that OCI containers are stable and well supported, using those through docker, podman or other similar tool is an increasingly popular option.
They are isolated, but under the same kernel, so:
they are almost as portable as virtual machines,
like virtual machines they can have their own virtual IP addresses, so they can run services not visible from the outside and without occupying port on the host, but
they don't reserve any extra space on disk or in memory like virtual machines and
they are not slowed by any virtualization layers and
mounting directories from the host is easy and does not require any special support.
The usual approach is to have the checkout in the developer's normal home directory and mount it into containers for building, testing and running.
Also building in containers is now supported by Remote Development extension for Visual Studio Code
I seem to end up evaluating a lot of software. This requires me to constantly install all kinds of things on my system. It creates a huge clutter and I spend a lot of time during the install process, and if I don't like it, then removing everything I've done. Much of my evaluation tends away from the features of the software being evaluated and toward how difficult it is to install. I'm sure I miss good software which may have actually been a better choice, because of this startup cost.
With the advent of VM software like VMWare Player and VirtualBox, it would be much easier to sell someone like me your software, if you just provided an image that I could load into the VM and run. I'd be looking at the features almost immediately rather than fighting with which revision of whatever. The VM would take care of all of this for me.
Am I missing something, or should vendors and OSS start distributing VMs for their wares?
Most of my evaluations are for server side software installed on Linux, so OS licensing is not the issue.
VMs require that the operating system have a valid license key. For free operating systems this wouldn't be an issue, but if you're developing for something like Windows machines, each time they send out a demo version of their software, they're sending out a license key that they would have to pay for.
This would be incredibly expensive for most companies.
The only downside I would say IMHO is the size of the images, if say you have a 20 MB application, do you really want to download/transfer an entire OS just for that application.
I would say a better approach would be to have a ready to go VM and then you simply take a snapshot (on Virtual Box, I assume similar feature exist in other players)
Then simply install the applciation inside your sandbox environment, and then just Zap it when done (i.e. return to your Snapshot)
Darknight
This can be done for softwre that runs on open source platforms, and VMware have a library of images which do just this (though the images that are used for evaluating commercial software is generally for infrastructure-type things that have very, very complex installation requirements):
http://www.vmware.com/appliances/
However, if the software is for the Windows platform, you don't really have the opportunity to do this, as Microsoft's Windows licensing would prevent it. Unless, you're Microsoft, of course, in which case you can in fact do this - and MS has done this to permit easier evaluation of such software as Visual Studio, SQL, and many others:
http://technet.microsoft.com/en-us/bb738372.aspx?ppud=4
Novell has an appliance builder called Suse Studio that lets you pick the software you want, it builds out a VM with the software (and dependencies, etc) for you. You can then try out the VM, download it, etc.
Whether the software you're looking for is available or not is a different matter.
Disclaimer: I work for Novell (though not with the Suse team)
But yes, if you can deal with the OS licensing issues, or possibly host trial environments yourself, this is a very effective way for a vendor to demo their app. The problem is that all vendors don't always have the infrastructure (or lack the awareness) to do so.
Microsoft provides fully-provisioned VM's for time-limited trials of their software. So if you want to trial select Microsoft products in that manner, you can do that today.
There is no sign, though, that Microsoft will make this available to third party Windows software vendors.
In the SaaS (Software-as-a-Service) world, you can get fully-provisioned virtual servers that include Windows and your software of interest on a pay-as-you-go basis, based on both Linux and Windows. For example, see Amazon Web Services
For windows, you may be better off developing a portable application that runs from a usb key. That is how Embarcadero distribute All Access. I received a 4 gb usb key that contained multiple applications. Most could be run straight from the key without installation. I believe Embarcadero will be licensing the technology at some stage.
If you are using a programming language such as Delphi or C++ with little in the way of external dependencies, a portable application is straight forward to develop. For .net, it is much harder, but can be done with Mono, or something like Virtual Application Studio.
When testing our software on several different systems (98-XP-Vista-Seven-Linux-etc), I think that the best choice is to use virtualized systems.
What's your choice: VMware, Virtual Box or MS Virtual PC/Server? and why?
We use VMWare here at work. Really any VM software that supports snapshots (or some way of saving the state of the machine) will work well. Snapshots make it easier for testing installs and rolling back. It can also help if you program goes and modifies files for returning back to a known-good state.
Virtual Box is the way to go. It has snapshots and is platform independent (Good for Mac users who want to test on other OS's). And it is free.
If it's available, Hyper-V on Windows Server 2008 is a powerful and full-featured entry including snapshot trees and all the niceties you'd expect with a quality UI.
If you're planning on using the VM on your local dev machine so you can (e.g.) bring it home on your laptop to work from there, then the more client-oriented virtualization software is probably the way to go.
If you're planning on using the virtualization in a primarily professional environment, a number of Hyper V machines in a computer lab that you can remote into is a powerful paradigm that we've been using at my office for a few months now.
My own preference is to use a local VM (Virtual PC is the easiest one for me) as my development environment because I can bring my work laptop home and use the VM there also (I don't VPN into the office). I then use the lab's Hyper-V machines for tests, deployments, etc because they have a better story for taking and restoring snapshots.
Go VMware. My reason is simple: before VMware released VMWare player and VMware server (the virtualisation platform formerly known as VMware GSX), the market for VM hosts was limited and expensive.
When VMware released these for free, all the other manufacturers (yes, I'm looking at Microsoft here) had to follow suit, so if it wasn't for the beneficence of VMware, we'd still be looking at having to buy our VM host software.
So, support VMware for being the good guys.
Oh, and their enterprise products are the business, they work well with Linux, have some excellent memory-saving tricks (here's the tech details), multiple snapshots and snapshots off a base image, and have features such as VMotion (load spreading) that other products don't support nearly as well (if at all).
Microsoft's VirtualPC. It free and simple.
One bit of functionality that is nice is the differenced VHDD that makes it easy (and space wise cheep) to keep backing up/reverting the image
VMWare, that's what we use here. We have both the full blown ESX for virtual servers and the VMWare workstations for development / testing. ESX resource management is very good, and easy to configure.
I've used VMWare (when the company would pay for it), VMWare Server (when the company would not), VirtualBox (because it's free, decent, and supports snapshots), Parallels on the Mac (which I bought), and Xen.
All work fine.
My current workhorse is VirtualBox, largely because it's free, supports snapshots, and runs on the various host platforms I have to use.
VMWare works pretty well, but for high cpu server apps we have found that Microsoft's Hyper-V works better because it has better cpu reservation abilities.
The key is that the system has snapshots, so you can easily roll back to several states (most do) and we have found that both VMWare and Hyper-V have excellent API's allowing us to kick off our automated tests when a new build completes.
Microsoft Virtual PC for Microsoft OS's, Virtual Box for *nix.
Virtual PC seem to be slightly faster and more stable, but it does not support linux.
We might have used VMWare if it was free,but our company would not spend the money.
Virtual box is great. It does have some stability issues if you run it inside Mac OS X. if you need a single solution to run multiple OS's this would be the one.
Linux/OpenSolaris on top of Virtual Box on top of Linux.
What are the key use cases for the use of virtualization -- that is, running one or more "virtual PCs" using software such as VMWare and Microsoft Virtual PC -- for software development?
Also -- are there other instances/uses of virtualization that aren't covered by my definition above (use of a tool like MS Virtual PC or VMWare), and that are useful to developers?
My impetus for asking is this StackOverflow comment by Metro Smurf asserting "You'll wonder how you ever developed without it!", regarding use of virtualization.
(Please include just one use case per response. Thanks!)
Application testing in multiple environments is one obvious use of virtualization that I'm aware of. Testing your application on other operating systems (without requiring additional physical computers to do so), as well as testing that involves software that generally only allows you to install a single version on a given machine (such as the Internet Explorer browser; running both IE6 and IE7 on the same machine is not an officially supported configuration), are good candidates for virtual machine usage.
If your build-server is running in a VM, you can make a snapshots of it for every software release in order to be 100% sure that you can recreate the build environment (in case you want to make patches to old releases, for example).
If you set up snapshots of your development environment (and back them up) it can be very easy to resume productivity if your computer breaks down. When your machine breaks down right before your release - and you can resume immediately with all your tools installed and configured, it can be a lifesaver.
The simplest case which applies to my current situation is that we have a complex client-server environment and with virtualization every developer can very quickly get a baseline set of operating systems to deploy their local build to and verify end to end functionality.
Locally you have your dev box, and N client boxes which get re-initialized as fresh OSes each time you want to try a build. Essentially it's the test environment equivalent of a 'make clean' where even the client workstation gets replaced with a new OS.
Quickly distributing environments between team members is a very nice use case to for virtualization especially if you have a lot of various components, tools, etc.. This can save you a ton of time with new hires, contractors, or other individuals who need an environment quickly.
Many presenters use a VM for presentations - it allows them to revert immediately to reset the presentation for the next day, transfer all presentation materials quickly between computers, and not have to show your attendees your messy My Documents folder.
Using virtualization for sales activities is also a great use case. You can take a snapshot at a particular time that you can save as your demo baseline. Then once you run through the demonstration and change the data, etc. you can restore back to your previous baseline for future demonstrations. You can also capture multiple baselines and pick and choose which baseline best fits the upcoming demo.
Test environments. If you have more than one setup that a system needs to be targeted for (e.g. Windows & Linux, XP & Vista) then a machine with lots of RAM and VMWare (or on of the others) is a good way to manage the environments.
Another is developing on one system and targeting another. For example, at one point I did some J2EE work on a workstation running Linux where the client was I.E. 5.5. A VM with Windows 2000 and IE 5.5 would let me test the application.
Reasons I use virtual machines for development.
Isolate different development environments.
Testing environments.
Easy recovery due to computer hardware failure/upgrade.
Ability to "roll-back" changes to your development environment if something corrupts it.
Currently, I am using VirtualBox for my VM setup. I used to use VirtualPC, but I REALLY hated not having any type of "snapshot" feature (like VMware and VirtualBox have).
We develop software for use in our SaaS application, our production environment has a large number of servers and their software environment needs to be absolutely predictable; we can't have ANYTHING installed extra, or absent from our development machines.
Moreover, our application requires a number of different server types in order to function properly (at least 7 last time I counted); mostly they can't be installed on the same (virtual) machine - at least, not without violating the "same software as production" requirement.
In order to have a consistent environment, it's necessary to use VMs. I don't know how anyone ever manages without them.
Snapshots and rollbacks are nice too, but I use them only occasionally (really useful during installation / upgrade tests).
Suppose you're developing a new version of your software, and checking that the upgrade from the previous version works correctly... how long does it take to do a test cycle without being able to rollback the box? Do you have to reinstall the OS then the old version? Can you guarantee that the uninstall really uninstalls everything?
Being able to test/retest your deployment process is a huge savings.
Developing Add-Ins for different versions of Microsoft Office (using Visual Studio Tools for Office).
My main work machine has Office 2007. When I work with Add-Ins for Office 2003 I use a virtual machine with Visual Studio and Office 2003.
I'm suprised that nobody has mentioned the VMware record/replay feature (awesome video demo) which is great for debugging.
I have a headless server running ESXi which runs various machines for building installers (so I don't have to give up processing power on my desktop), automated testing (server is faster than any desktop) and various test environments (about 20 different configurations) so that the support team can easily jump onto a configuration that closely matches a customers system.
When you have one really beefy server running VMs that can be shared between support, test and dev teams, you introduce huge cost savings. In all, we're running ~25 VMs on ESXi (dual-quad core Xeon 2.5G + 8Gb RAM) shared between 5-10 people, some of the developers use Virtual PC and then I use VMware Workstation on my desktop. All of the Mac users here use VMware Fusion as well
I am surprised that no one has mentioned the benefit of increased security by isolating, for example, the database server and web server in different VM's.
Some server applications can use VMs too. When one vm is not used much, the server can locate the resources to other vms.
Some sort of test environment: if you are debugging malware (either writing or developing a pill against it) it is not clever to use the real OS. The only possible disadvantage is that the viruses can detect that they are being run in the virtualization. :( One of the possibilities to do it is because the VM engines can emulate a finite set of hardware.