I've seen various docs regarding containerD metrics (usage) stats polling buy applicable on the Linux-based machines (https://www.ibm.com/docs/en/instana-observability/current?topic=technologies-monitoring-containerd)
Is there an equivalent thing on Windows OS hosts running containerD for creating windows containers?
Related
i'm working on an application that needs to be tested in a HPC cluster.
i'm thinking about using xcat as a resource manager.
i don't have much hardware resources, i have one HP desktop and MacBook laptop.
the question: is it possible to set up a virtual cluster (using virtualBox or KVM) on one hardware resource
thanks,
The short answer here is yes, depending on how much memory and disk you have available on your one machine. I've done this numerous times on a MacBook Pro with 8 GB of RAM.
The long answer is that there is absolutely nothing magical about an HPC cluster. All you need to test basic parallel applications in a simulated cluster environment are two or more VMs which meet these criteria:
Same OS, as identical as possible.
Passwordless authentication (ssh key based auth).
Same software stack in same location on all nodes (See #4 or use rsync).
At least one shared filesystem, e.g. NFS mounted $HOME
Shared network with name resolution configured (correct /etc/hosts on all nodes)
None of this requires job schedulers, provisioning tools or any complex networking. You can find many NFS setup howtos to help get one node set up to share $HOME to the others, this might be the most complicated part. VirtualBox does a good job of setting up local networking.
On top of this you can layer setting up a job scheduler like SLURM (highly recommended), provisioning tools like Warewulf or xCat, parallel filesystems across the VMs (BeeGFS is easy to set up and a great introduction), etc. I have had a full featured stateless cluster simulated on my Macbook Pro a number of times using tools from this list and VirtualBox VMs. It's a great way to learn about setting up an HPC cluster.
I am not able to understand this hierarchy. Host OS runs on bare metal. Where does libvirt reside ? where does the Hypervisor reside ? Is libvirt a necessary lib in the creation of a VM or is it just a better abstraction to the core kernel APIs that provide the the service of creation of VMs? what is the libvirt equivalent in windows that is used by Virtualbox ? Does Hypervisor have any role to play in the scheduling of host OS applications or is it just another process handled by the Host OS. These are a few questions that confuse me. Can anyone explain this in a straight manner ? Thanks!
Host OS runs on bare metal - Yes
where does the Hypervisor reside? - It is a software layer on Host OS
Where does libvirt reside? - It is another API layer on the Host to indirectly interface with the hypervisor.
Is libvirt a necessary lib in the creation of a VM or is it just a better abstraction to the core kernel APIs that provide the the service of creation of
VMs? - Standard abstraction
What is the libvirt equivalent in windows that is used by Virtualbox ? - Libvirt can be used to interact with VirtualBox. Libvirt is known to work as a client (not server) on Windows XP (32-bit), and Windows 7 (64-bit).
Does Hypervisor have any role to play in the scheduling of host OS applications or is it just another process handled by the Host OS. - Just another process
References: http://virtualbox.guru
Managing the infrastructure (private cloud or public cloud) at scale and ease is addressed by Apache Mesos, Apache CloudStack and OpenStack.
I have few questions in this regard and wanted to see if someone can throw light.
Any article(s) that compares and contrast the above?
Why run one over the other at all? (as I see from tutorials that one can run over the other)
It seems like CloudStack is centered around VMs (Hypervisor) and Mesos is centered around Scheduling and allocation of resources in side VMs for different co-existing software systems. Am I right in my conclusion?
If so, why does Mesos claims that it can manage bare metal boxes w/o even the need for Hypervisor? Is this for the reason for workloads that do not go well with VMs? (esp. LSM based systems such as SOLR-Lucene and HBase)
Is it with the choice of Linux VMs. Vs. Linux Container for resource allocation? IOW, Mesos is Linux container based framework, and CloudStack is Linux VMs based framework?
Azure and EC2 are optimized for running servers. Lots and lots of servers. Both platforms attempt to manage tons of things for you -- in Azure's case, it wants to manage even the target operating system.
However, I'd like to use such a service for a different reason: Testing.
I've got a ton of operating systems I need to support. My tests don't actually take that long, but running them on every platform is time consuming. I was going to just use a cloud service for this, thinking that these machines would be running for much less than an hour, and it wouldn't cost all that much.
The problem is that the major cloud services won't run client versions of Windows -- Windows Server only.
Is there a cloud service which would let me run every client and server version, and every service pack level, of Windows released starting with Windows 2000 SP4 to the present day?
Try CloudSigma, Defiantly can upload your own ISO's and run any x86 and 64bit OS you like on it. They have their in-house versions to get started but you can bring your own OS versions.
Based in Switzerland but they would have also the servers in the US, performance i've expected to quite good.
https://www.cloudsigma.com/
There is also a free trail on at the moment
https://cs.cloudsigma.com/accounts/signup/
The list of Open Virtualization Alliance members may have some candidates for you.
A search on the page for "operating system" suggests the following possibilities (in addition to the already-mentioned CloudSigma):
ElasticHosts
stepping stone GmbH (I'm less sure about this one)
Sublime IP
No, commercial cloud services like Azure and Amazon EC2 are themselves virtual, so you don't get a great deal of control over the operating system.
An option may be to consider renting a full physical server (colocated, or managed) and then use a battery of virtual machines to run the tests. Something like VMWare's snapshot feature sounds perfect: spin up a clean virtual machine, deploy the test code, then throw away changes to the disk once the tests have been completed.
Or, indeed, as #Stuart suggests - run the tests locally.
This definitely isn't something Azure offers - I think all of Azure's images are based near to Windows Server 2008 R2.
For EC2 you could set up images for Server 2003 through to 2008R2 - but nothing else. There are also some services out there to assist with this - e.g. VaasNet http://www.vaasnet.com/catalog
For testing the other Windows operating systems, I simply don't think there's a cloud service available to let you do this. I don't even think there are any cloud services where you can run "Virtual PC" type applications on top of the hosted operating system - as I think most of the virtualization APIs are disabled in the cloud environments (virtualization within virtualization not supported!)
Sorry to say this, but your best bet may be local test hardware running VirtualPC images.
It appears that the Xen Cloud Platform might do what you're after. This page ends with:
Guest Operating Systems: the XCP binary distribution is delivered with a wide range of Linux and Widnows guests. Check out the release notes for a complete list.
And their PDF document Xen Cloud Platform Virtual Machine Installation Guide (Release 0.1, Published October 2009) says that Windows 2000 Server has "No known issues."
(I don't have any affiliation with Xen)
In conjunction with the above, there is also a list of Xen VirtualPrivateServerProviders, several of which say they include Windows.
Buy time on an EC2 instance and use it to host VirtualBox VMs with VMs set up for each operating system you want to test for. Use a RDP client or VNC or some other means to control the guest OS. This forum post seems to point to that being possible. But yes it is not a cloud service itself and you would have todo some initial setup and configuration work yourself.
What is your preferred development environment ?
Native
WAMP/MAMP/LAMP (Apache, MySQL, PHP) on Windows/MacOS/Linux
Working copy local, SVN/CVS on server
IDE/Editor on the same system (Eclipse, Aptana, Zend...)
Virtual/Native (Server on VM)
LAMP on VirtualBox/VMware
working copy in the VM
IDE/Editor on host, access to the VM with Samba, FTP, SFTP (eventually mapping with tools like WebDrive)
Virtual (VM)
Complete development environment running in a VM (server, tools, IDE)
Host is only used for special tools not available on the OS running in the VM
All have pros and cons.
With BitNami stacks you can run the exact same XAMP environment locally or remotely (and make sure everybody on your team is running the exact same stack). It is free and works on Windows, Linux, Mac.
I like having the SVN repository somewhere on a web server.
It's reasonably secure (using Apache WebDAV), and it gives me a good chance of recovering quickly from any disasters that may befall my main development machine. I have the luxury of control over my own web server, but there are lots of cheap hosts that will do the job at low cost.
As regards VM or no VM:
Advantages of VM - very fast recovery from screwing up your development environment
Ability to try out different versions or upgrades quickly
If you have many systems running the VM host, ability to quickly move the whole environment
Can choose any Host
Disadvantages of VM - performance impact; extra setup complexity.
On balance, I go for "no VM" if all the tools are available on my host system, but I do use VM when I need to run a different OS (the host system is a Mac Pro, so if I need Visual Studio, I do it with Parallels).