I was studying VMWare's VSphere suite, cloud computing virtualization platform.
I could not figure out whether there's any relation between VMotion and VMFS in the suite?
VMotion enables the live migration of running virtual machines from
one physical server to another with zero down time.
VMFS is a clustered file system that leverages shared storage to allow multiple physical hosts to read and write to the same storage simultaneously.
Is there any relation between them?
No.
As you mention, VMFS is the file system we use by default on "block" shared storage (i.e. LUNs). This allows us to have the same LUN mounted for read/write on multiple ESXi hosts which is not allowed with many file systems.
vMotion is when we move a running VM from one ESXi host to another. We do this by copying the running memory state from one host to another. When then "stun" the VM for a short period of time and quickly move it's virtual NIC to the new server. The VM "starts" on the far side in the same state, thus it appears like the VM has always been running. That is to say we "move" the running VM even though we are actually just creating a new VM with exactly the same memory state and disk.
The only relationship is that if you have a VM whose VMDKs live in a Datastore which is shared across multiple ESXi hosts, the vMotion process doesn't have to copy the VMDK which makes the process much simpler and faster. Since VMFS is one way we can support shared storage, it is common to have VMDK's on VMFS based datastores (in this case 1 datastore = one VMFS formatted LUN). Since VMFS is our oldest shared storage technology, it's the most common and usually be best understood by our customers.
However, any shared storage will work just fine for vMotion, including VSAN, VVOL and NFS based shared storage.
Related
Is it feasible to have a virtual machine running an OS on a local disk (not the one with the guest system) and booting the real machine from that same disk?
I'd like to do it with both linux and Windows. Is it possible?
If you use separate partitions on the disk for the two OSes, then it works with no problem. But I think you mean that you want both instances of the OS to use the same partition, so the answer is no. There are many files in both Linux and Windows that are modified by the system while it is running. If two different instances of the OS are running and trying to update these files at the same time, it will result in chaos.
It would be possible to share the read-only parts of the system, and have separate copies of the writable parts, but that would be fairly tricky to set up. And it would result in two separate OSes on the disk, albeit with some shared files, so I don't think it really meets the premise of your question.
i'm working on an application that needs to be tested in a HPC cluster.
i'm thinking about using xcat as a resource manager.
i don't have much hardware resources, i have one HP desktop and MacBook laptop.
the question: is it possible to set up a virtual cluster (using virtualBox or KVM) on one hardware resource
thanks,
The short answer here is yes, depending on how much memory and disk you have available on your one machine. I've done this numerous times on a MacBook Pro with 8 GB of RAM.
The long answer is that there is absolutely nothing magical about an HPC cluster. All you need to test basic parallel applications in a simulated cluster environment are two or more VMs which meet these criteria:
Same OS, as identical as possible.
Passwordless authentication (ssh key based auth).
Same software stack in same location on all nodes (See #4 or use rsync).
At least one shared filesystem, e.g. NFS mounted $HOME
Shared network with name resolution configured (correct /etc/hosts on all nodes)
None of this requires job schedulers, provisioning tools or any complex networking. You can find many NFS setup howtos to help get one node set up to share $HOME to the others, this might be the most complicated part. VirtualBox does a good job of setting up local networking.
On top of this you can layer setting up a job scheduler like SLURM (highly recommended), provisioning tools like Warewulf or xCat, parallel filesystems across the VMs (BeeGFS is easy to set up and a great introduction), etc. I have had a full featured stateless cluster simulated on my Macbook Pro a number of times using tools from this list and VirtualBox VMs. It's a great way to learn about setting up an HPC cluster.
As a very beginner, I only know how to create VMs and install OS on these using Oracle VirtualBox. All the VMs created are dependent on the hardware resources (CPU, RAM etc.) of a single machine. If the machine goes down the VMs will go down. Need to know how VMs can be created using taking resources from different physical machines (manually or dynamically) to avoid failure of any VMs.
For example: There are 4 physical machines having 8 core and 16GB RAM each. Now, I want to create three VM having having 8 core and 16GB RAM taking from different physical machines. If one physical machine goes down, no VM will be down.
You can look up clustering solutions (e.g. VMware clusters, or Hyper-V failover clusters). In this model, if a physical host goes down, then the virtualization platform will power up the VMs on other hosts.
If you're looking for zero downtime, then VMware has something called Fault Tolerance in which a shadow copy of a VM is running on a different host and is continuously synchronized with the primary copy. If the primary host goes down, the shadow copy can take over with zero downtime (e.g. you don't have to boot from the shadow copy because it's already running). This feature, while cool, has a lot of real-world limitations in how it inter-operates with other features of VMware. For example, as of vSphere 6.0, you cannot do various kinds of migrations for such VMs, etc. I believe it also requires a more expensive license.
These solutions generally require some shared resources between the physical hosts (most notably storage). Otherwise they will not work (or at the very least, performance will greatly suffer).
I have to set up a relatively big system consisting of Virtual Machines, where I will need to run several different applications. The applications will be provided to me as black boxes, either in form of software to be installed by myself (on a new VM), or in the form of Virtual Machine containing already everything for an application.
My task is to set up a host server and estimate its general resources, which will be then distributed between all Virtual Machines in my system. Some of the applications are more demanding than the others, and I have also time deadlines, so it could happen that all the application need to be executed simultaneously.
For each application I have the resources description it needs (but no corresponding time and performance estimates), so that I know how many processors and processors cores I normally will need for a single app. But how should I do with all of them running simultaneously? Should I simply add together the requirements or is there some common formula for scaling of the host servers general CPUs, Memory and Storage resources?
And one more questions. Such a system with distribution of real physical resources between several VMs - is it already a cluster? Or not yet?
So I have a cloud virtual machine on google compute, does this mean by nature that it is highly available? If the VM is running on a single piece of hardware on GCE, if the piece of hardware breaks then the VM could go down. Is the VM running on some kind of RAID, but for servers? So if one of the machines goes down another machine will pick up and continue running the vm? Thanks.
The machine itself is not highly available. However, Google takes several steps to increase reliability:
Storage is replicated and independent of the physical machine the VM is running on (obviously not for local SSD). This means that even if the physical machine catches on fire, only the "runtime" state is lost but the attached disks are fine.
VMs can live-migrate. This is a setting you can control. If enabled, the VM will be migrated to a different physical machine on maintenance events. Live-migration can lead to brief performance degradation while memory etc. is synced to the other host but the machine is not shut down / restarted.
Even when the physical host suddenly dies, you can set your instance to restart automatically on a new machine. If you plan to use this mode, make sure your instance is able to cleanly boot to serving state without manual intervention.
If you need high availability, the best approach is to spread your instances among zones of the same region and using a network or HTTP(S) loadbalancer. These will automatically stop sending traffic to a machine in case it becomes unhealthy. Also see this short youtube video on Google's network architecture for more info.
For high availability of your application data, there are highly available options like Datastore for database-like usage and Cloud Storage for file-oriented data. Keep in mind that Cloud SQL also runs on a single instance/physical machine which means that you have to setup slaves/replicas to get high availability. However, you can also do that with your favorite DB system on plain Compute Engine instances if you are willing to maintain them yourself.