I started investigating how virtual machines are scheduled in KVM, and stumbled upon the vm_list, but I got stuck. So what is the purpose of the vm_list? Any relation to scheduling?
Following the basic cloud architecture which had VMs on the top of Hypervisor;
KVM ideally allocates and tracks resource of all the VMs .
The two structs mm_struct and vm_list are a part of kvm struct to maintain resource allocation to n number of VMs spawn by KVM
Hope it helped.
Cheers!!
Related
I think the major feature of libvirt is abstract the different target, but if I only use kvm/qemu, what I can get from libvirt ?
There is quite a large set of benefits of libvirt, besides just hypervisor abstraction. Rather than list them all here, check out the blog post I wrote about this a few years back.
https://www.berrange.com/posts/2011/06/07/what-benefits-does-libvirt-offer-to-developers-targetting-qemukvm/
Everything listed there is still relevant today, and we've added some benefits since then, in particular running QEMU in separate mount namespace to isolate QEMU from the host /dev nodes.
the use of this abstraction layer becomes very handy when you start to script or automate KVM deplyments but ...
The main added value in the KVM QEMU setup is the Live Migation of a Virtual Machine to an ohter Hardware with Hypervisor.
i'm working on an application that needs to be tested in a HPC cluster.
i'm thinking about using xcat as a resource manager.
i don't have much hardware resources, i have one HP desktop and MacBook laptop.
the question: is it possible to set up a virtual cluster (using virtualBox or KVM) on one hardware resource
thanks,
The short answer here is yes, depending on how much memory and disk you have available on your one machine. I've done this numerous times on a MacBook Pro with 8 GB of RAM.
The long answer is that there is absolutely nothing magical about an HPC cluster. All you need to test basic parallel applications in a simulated cluster environment are two or more VMs which meet these criteria:
Same OS, as identical as possible.
Passwordless authentication (ssh key based auth).
Same software stack in same location on all nodes (See #4 or use rsync).
At least one shared filesystem, e.g. NFS mounted $HOME
Shared network with name resolution configured (correct /etc/hosts on all nodes)
None of this requires job schedulers, provisioning tools or any complex networking. You can find many NFS setup howtos to help get one node set up to share $HOME to the others, this might be the most complicated part. VirtualBox does a good job of setting up local networking.
On top of this you can layer setting up a job scheduler like SLURM (highly recommended), provisioning tools like Warewulf or xCat, parallel filesystems across the VMs (BeeGFS is easy to set up and a great introduction), etc. I have had a full featured stateless cluster simulated on my Macbook Pro a number of times using tools from this list and VirtualBox VMs. It's a great way to learn about setting up an HPC cluster.
We have recently purchased the new server having 16 GB of RAM. We have created 5 virtual machine. Out of these 5 virtual machines three are windows VM and the remaining two are Linux VM.
We have been contsantly facing the poor response of the virtual machines and the network / infra team is not able to tell us the root cause of the problem and providing the solution.
Can you please let me know what could be the possible cause of the slow down.
Additionally, we want to have the audits conducted to see if the infrastructure / network for our company has been setup correctly. Can you please let me know what are the typical audit parameters we evalluate for the best performance of Network and SYstems resources.
Thanks for the help.
Memory (RAM) is being distributed among so many VMs, each VM has very less RAM available for its own use.
In this coming semester, I am starting some research on large-scale distributed computing with MPI. What I am looking for help with is the initial stages, specifically getting a solid development environment set up. Does anyone have any recommendations for good tools to use for this?
I am also curious as to whether there exists a kind of simulator that would allow be to write MPI and distribute it to virtual (rather than physical) nodes.
You could download a MPI library such as Open-MPI, MPICH, etc. and run it on a multi-core system (such as a recent desktop) with number of processes = number of cores. They would operate without a network interconnect (for instance, over shared memory). That should be enough to explore initially.
If you really want multiple nodes, you can experiment with multiple VMs with a VM network before actually moving on to a physical cluster. One of the VMs would have to be configured to act like a NFS server and the rest of the VMs could mount your home directories over NFS.
Depends on what is your favourite language. I dove into MPI using python and the pypar module. It lets you concentrate on MPI procedures without worrying too much about pointers and complicated c / c++ stuff. MPI on a single machine is programmed no differently from MPI on 100s. Getting cross machine setups is more about what MPI implementation and operating systems you use.
I'm looking into using virtual machines to host multiple OSes and I'm looking at the free solutions which there are a lot of them. I'm confused by what a hypervisor is and why are they different or better than a "standard" virtual machine. When I mean standard I going to use the benchmark virtual machine VMWare Server 2.0.
For a dual core system with 4 GB of ram that would be capable of running a max of 3 VMs. Which is the best choice? Hypervisor or non-hypervisor and why? I've already read the Wikipedia article but the technical details are over my head. I need a basic answer of what can these different VM flavors do for me.
My main question relates to how I would do testing on multiple environments. I am concerned about the isolation of OSes so I can test applications on multiple OSes at the same time. Also which flavor gives a closer experience of how a real machine operates?
I'm considering the following:
(hypervisor)
Xen
Hyper-V
(non-hypervisor)
VirtualBox
VMWare Server 2.0
Virtual PC 2007
*The classifications of the VMs I've listed may be incorrect.
The main difference is that Hyper-V doesn't run on top of the OS but instead along with the system it runs on top of a thin layer called hypervisor. Hypervisor is a computer hardware platform virtualization software that allows multiple operating systems to run on a host computer concurrently.
Many other virtualization solution uses other techniques like emulation. For more details see Wikipedia.
Disclaimer, everything below is (broadly) my opinion.
Its helpful to consider a virtual machine monitor (a hypervisor) as a very small microkernel. It has very few jobs beyond accessing the underlying hardware, such as monitoring of event channels and granting guest domains access to specific resources .. while enforcing some kind of scheduler.
All guest machines are completely oblivious of the others, the isolation is true. Guests do not share memory with the privileged guest (or each other). So, in this instance, you could (roughly) think of each guest (even the privileged one) as a process, as far as the VMM is concerned. Typically, the first guest gets extra privileges so that it can manage the rest. This is the ideal technology to use when virtual machines are put into production and exposed to the world.
Additionally, some guests can be patched to become aware of the hypervisor, significantly increasing their performance.
On the other hand we have things like VMWare and QEMU, which rely on the host kernel to give it access to bare metal and enough memory to exist. They assume that all guests need to be presented with a complete machine, the limits put on the process presenting these (more or less) become the limits of the virtual machine. I say more or less because device mapper QoS is not commonly implemented. This is the ideal solution for trying code in some other OS, or some other architecture. A lot of people will call QEMU, Simics or even sometimes VMWare (depending on the product) a 'simulator'.
For production roll outs I use Xen, for testing something I just cross compiled I use QEMU, Simics or VirtualBox.
If you are just testing / rolling new code on various operating systems and architectures, I highly recommend #2. If your need is introspection (i.e. watching guest memory change as bad programs run in a guest) ... I'd need more explanation before answering.
Benefits of Hypervisor:
Hypervisor separates virtual machines logically, assigning each its own slice of underlying computing power, memory, and storage, thus preventing the virtual machines from interfering with each other.