I have a number of projects that use Vagrant and associated VMs. Some are in current development, others are either in maintenance or archived, and I do not need the latter on my laptop. 256GBs fill up fast.
What I'd like to do is move the archived and maintenance virtual machines to an external SSD hard drive, but still have them accessible via the Vagrant command line.
All google queries and searches have only turned up a permanent move to another drive, and nothing about some here and some there.
Is this possible?
You may try to use environment variables for change location setting
i think that may help.
use it in this way VAR="C:\some_location" vagrant up
VAGRANT_DOTFILE_PATH
VAGRANT_HOME
VAGRANT_DOTFILE_PATH can be set to change the directory where Vagrant stores VM-specific state, such as the VirtualBox VM UUID. By default, this is set to .vagrant. If you keep your Vagrantfile in a Dropbox folder in order to share the folder between your desktop and laptop (for example), Vagrant will overwrite the files in this directory with the details of the VM on the most recently-used host. To avoid this, you could set VAGRANT_DOTFILE_PATH to .vagrant-laptop and .vagrant-desktop on the respective machines. (Remember to update your .gitignore!)
Related
I need to be able to pass some parameters to my virtual machine during it's bootup so it sets itself properly. To do that I either have to bake the info into the image or somehow pass it as parameters to my qemu-kvm command. These parameters are just few, and if it was VMware, we would just pass it as ova paramas and when the VM launches we would call the ova-environment to get these params. But launching it from qemu-kvm I have no such options. I did some homework and found that I could use virtio-9p driver for sharing files across host and guest. Unfortuantely RHEL/Centos has decided not to support 9p.
With no option of rebuilding my RHEL kernel with the 9p options enabled, how do I solve my above problem? Either solution would work, which is, pass/share some kind of json file to the VM(pre-populated on the host), which will read this and do it's setup OR set some kind of "environment variables" which I can query from within the VM to get these params and continue with setup. Any pointers would help.
If your version of QEMU supports it, you could use its -fw_cfg option to pass information to the guest. If that guest is running a Linux kernel with CONFIG_FW_CFG_SYSFS enabled, you will be able to read out the information from sysfs. An example:
If you launch your VM like so:
qemu-system-x86_64 <OPTIONS> -fw_cfg name=opt/com.example.test,string=qwerty
From inside the guest, you can then get the value back from sysfs:
cat /sys/firmware/qemu_fw_cfg/by_name/opt/com.example.test/raw
There appears to be some driver for Windows as well, but I've never used it.
When you boot your guest with -kernel and -initrd you should be able to pass environment variables with -append.
The downside is that you have to keep track of your current kernel and initrd outside of your disk image.
Other possibilities could be a small prepared disk image (as you said) or via network/dhcp or a serial link into your guest or ... this really depends on your environment.
I was just searching to see if this situation had improved and came across this question. Apparently it has not improved.
What I do is output my variable data to a temp file (eg. /tmp/xxFoo). Usually I write text or a tar straight to that file then truncate it to a minimum size and 512 byte multiple like 64K otherwise the disk controller won't configure it. Then the VM starts with a raw drive as that file. After the VM is started the temp file is deleted. From within the guest you can read/cat the raw block device and get the variable data (in BSD use the c partition as the raw drive).
In Windows guests it's tricky to get to the data. In theory you can read \\.\PhysicalDriveN but I have not ever been able to get that to work. Cygwin can do it and it works like Linux. The other option is to make your temp file a partitioned and formatted image but that's a pain to create and update.
As far as sharing a folder I use Samba which works in just about anything. I usually use several instances of smbd running with different configurations.
One option is to create a ISO file and pass as parameter. This works for both host Win and Ubuntu and Guest Win and Ubuntu. You can read the mounted CD ROM inside the guest OS
>>qemu-system-x86_64 -drive file=c:/qemuiso/winlive1.qcow2,format=qcow2 -m 8G -drive file=c:\qemuiso\sample.iso,index=1,media=cdrom
On Guest Linux Mount CDROM in Ubuntu:-
>>blkid //to check if media is there
>>sudo mkdir /mnt/cdrom
>>sudo mount /dev/sr0 /mnt/cdrom //this step can also be put in crontab
>>cd /mnt/cdrom
I have an instance of OpenNebula with 2 nodes running KVM and local file store. This means no live migration as vm images are scp'd to each node, so there is also no option of failover or Live Migration.
I would like to implement NFS shared storage and move the VM's from the local FS datastore to the NFS shared storage datastore. OpenNebula supports migrating VM's between datastores, but only datastores of the same type i.e. 'ssh' to 'ssh' but not 'ssh' to 'shared'.
I am working on a method of achieving this, and would love some feedback as to why this is a good or a bad idea.
Thanks
OpenNebula doesn't currently support migrating VM's from one type of datastore to another different type of datastore. I have been working on a method that is working and want to document it here to get some feedback and opinions on the method.
A datastore type is identified primarily by the Transfer manager Driver 'TM_MAD' setting. This setting cannot be changed, either through Sunstone or through the cli. So we need a method to do just this. This is what i did. I started with a fresh install of OpenNebula 5.4.13 in one VM, and 2 VM nodes all running Debian 9 within VMware virtual machines (don't forget to check virtualisation for the VM CPU options).
NOTE: This is an experimental process so make sure you Backup everything first!
Steps
To migrate to a different store, there are a few steps we need to do. They are as follows:
Setup the NFS share exports,
Move the VM images to the NFS share and mount the datastore,
Change the datastore types,
Configure the nodes for NFS share.
Setup NFS Server
First thing we want to do is setup the NFS shares that we want to use. I'm using a single share for the base datastore folder, but you could use separate shares for each datastore ID from different NFS servers.
On the NFS Server create the datastore folder i.e. mkdir /share/one_datastore,
Add the datastore path to exports and export the new share exportfs -rav,
Confirm the share is available showmount -e localhost
Prepare to Migrate
Before we modify the datastores there are a few things to do first:
Shut down any running VM's and undeploy them. This saves the machines states and copies the images back to the image store,
Stop Sunstone and OpenNebula services systemctl stop opennebula && systemctl stop opennebula-sunstone.
Migrate Data
Shared storage shares the VM disk images so all the nodes can access the same data. So copy the VM data to the NFS share ready for mounting.
From the Sunstone frontend server confirm the NFS shares showmount -e [nfs-server],
Create a temp folder to mount the share in mkdir /mnt/datastore,
Temporarily mount the NFS folder mount [nfs-server]:/share/one_datastore /mnt/datastore,
Move the datastore folders to the share mv /var/lib/one/datastores/* /mnt/datastore/
OpenNebula datastore folders now live on the NFS server: ls /mnt/datastore should list folders 0, 1 and 2,
Mount the NFS share to replace the OpenNebula datastore folder mount [nfs-server]:/share/one_datastore /var/lib/one/datastores,
Confirm the folders are available ls /var/lib/one/datastores should list our 3 folders 0, 1 and 2,
Add the mount into /etc/fstab to persist the mount on boot.
OpenNebula frontend is now configured to access the datastore folders from the NFS share. Next we want to change the datastores type from ssh to shared.
Change Datastore Types
The data for the datastore configuration is stored in the OpenNebula database /var/lib/one/one.db. We can change the driver type by editing the datastore configuration data which then tells OpenNebula whiche drivers to use, and how to handle the datastore data. By default OpenNebula uses an sqlite database with the option of MySql. i'm using sqlite but the same works for MySql.
Open the OpenNebula database sqlite3 /var/lib/one/one.db,
View all tables with .tables. datastore_pool is the table we want to modify,
List all the records in the table select * from datastore_pool; will result in a screen-full of configuration data. Each record has an identifier oid which matches the datastore ID, like this (the first 0 is the datastore ID for the default SYSTEM database):
0|system|<DATASTORE><ID>0</ID><UID>0</UID><GID>0</GID><UNAME>oneadmin</UNAME><GNAME>oneadmin</GNAME><NAME>system</NAME><PERMISSIONS><OWNER_U>1</OWNER_U><OWNER_M>1</OWNER_M><OWNER_A>0</OWNER_A><GROUP_U>1</GROUP_U><GROUP_M>0</GROUP_M><GROUP_A>0</GROUP_A><OTHER_U>0</OTHER_U><OTHER_M>0</OTHER_M><OTHER_A>0</OTHER_A></PERMISSIONS><DS_MAD><![CDATA[-]]></DS_MAD><TM_MAD><![CDATA[ssh]]></TM_MAD><BASE_PATH><![CDATA[/var/lib/one//datastores/0]]></BASE_PATH><TYPE>1</TYPE><DISK_TYPE>0</DISK_TYPE><STATE>0</STATE><CLUSTERS><ID>0</ID></CLUSTERS><TOTAL_MB>0</TOTAL_MB><FREE_MB>0</FREE_MB><USED_MB>0</USED_MB><IMAGES></IMAGES><TEMPLATE><ALLOW_ORPHANS><![CDATA[NO]]></ALLOW_ORPHANS><DISK_TYPE><![CDATA[FILE]]></DISK_TYPE><DS_MIGRATE><![CDATA[YES]]></DS_MIGRATE><RESTRICTED_DIRS><![CDATA[/]]></RESTRICTED_DIRS><SAFE_DIRS><![CDATA[/var/tmp]]></SAFE_DIRS><SHARED><![CDATA[NO]]></SHARED><TM_MAD><![CDATA[ssh]]></TM_MAD><TYPE><![CDATA[SYSTEM_DS]]></TYPE></TEMPLATE></DATASTORE>|0|0|1|1|0
Now to change the datastore type. Grab the data from the 3rd column body
(You can run select body from datastore_pool where oid=0;) and copy to your favourite text editor (that's the chunk starting with <DATASTORE> and ending with </DATASTORE>). Find and replace:
Find: <TM_MAD><![CDATA[ssh]]></TM_MAD>
Replace with: <TM_MAD><![CDATA[shared]]></TM_MAD>
Find: <SHARED><![CDATA[NO]]></SHARED>
Replace with: <SHARED><![CDATA[YES]]></SHARED>
Now to update the SYSTEM datastore record. Run the following command on the database, replacing [datastore-config] with the text block you just modified update datastore_pool set body='[datastore-config]' where oid=0,
Update IMAGE datastore is a little different. There is no SHARED option, but we want to use either shared or qcow2 drivers. I used qcow2. So: select body from datastore_pool where oid=1;:
Find: <TM_MAD><![CDATA[ssh]]></TM_MAD>
Replace: <TM_MAD><![CDATA[qcow2]]></TM_MAD>
Update the record: update datastore_pool set body='[datastore-config]' where oid=1;,
Update the FILES datastore (oid=3) by replacing <TM_MAD><![CDATA[ssh]]></TM_MAD> with <TM_MAD><![CDATA[shared]]></TM_MAD> and update using the method above.
Now that the datastores have been updated to use the shared driver, lets start Sunstone and check that the datastores show up.
systemctl start opennebula && systemctl start opennebula-sunstone
Jump into Sunstone web and go to datastores. Opening each datastore to check whether SHARED is enabled, and the correct drivers show i.e. shared or qcow2.
~DONT DO ANYTHING YET~ Still need to configure the nodes!
Configure the Nodes
So because we stopped and undeployed the VMs, there shouldn't be any data in the node datastores. So we can just set up NFS shares to the datastores folder. Confirm the folders are empty first and make sure to take backups! This is an experimental process so be warned! Right, lets get onto it:
Check the contents of /var/lib/one/datastores. If you are mounting each datastore ID based folder to its own NFS share then you can do this instead of the entire datastore folder. Empty any folders with 0, 1 and 2 folders. otherwise remove all folders from the datastores folder,
If not already installed: apt-get install nfs-common,
Check for NFS shares: showmount -e [nfs-server],
Mount the nfs share to the datastore folder: mount [nfs-server]:/share/one_datastore /var/lib/one/datastores,
Confirm the mount i.e. df,
Edit /etc/fstab adding the mount so its mounted on next boot.
Restart your node to confirm the datastore nfs persists, and to give them a restart!
Repeat with all host nodes.
Test it Out
In Sunstone go to the Hosts TAB and check they are up and running. Next go and grab a VM and deploy it. It should deploy without any issues and start booting.
Once up and running i like to constantly ping the VM while testing live migration. So start ping (ping [vm-ip] -t in windows) and then in Sunstone open the VM and do a 'Live Migrate' to another node. Watch the ping and check the logs to make sure it succeeded. I found i had to refresh the display, and go to the hosts TAB to check the VM had migrated. After that it showed correctly but i think its a caching issue in my browser. After the Live Migration you should still see the ping rolling along, with maybe one failed ping in the results.
Conclusion
So that's the process i used to migrate from ssh local storage to shared storage. I'v tested it and it is working without any issues. However, if you do have any issues or have an opinion on this process please let me know. If there are any pitfalls with this i have overlooked please also let me know.
Ok, have fun with it. I'm off to try moving the shared storage over to some kind of shared cluster like Ceph or GlusterFS!
I am attempting to relocate my DocumentRoot (i.e. localhost) to a synchronised folder (such as Google Drive, Dropbox or Tresorit), but the attempt fails with a 403 error.
On Windows machines I can configure localhost to run from D:/GoogleDrive/SitesG folder; the local site runs perfectly.
On a Mac, however, localhost won't work when running out of a cloud-based storage folder such as Google Drive, Dropbox, Tresorit, etc.
Everything is fine when localhost is at Users/myname/Sites.
However, when I reconfigure the Mac to run from Users/myname/GoogleDrive/SitesG - e.g. by editing the httpd.conf, etc, files - localhost is blocked.
Clearly the problem is to do with permissions on the parent folder (e.g. the Google Drive or Dropbox or Tresorit folder). I can see that the permissions on the various folders are as follows.
drwxr-xr-x 32 myname staff 1024 30 Apr 02:23 Sites
drwxr-xr-x 22 myname staff 704 30 May 21:01 SitesG
drwx------# 61 myname staff 1952 30 May 17:47 GoogleDrive
So my question is: On a Mac (running HighSierra), is it possible to relocate the DocumentRoot to GoogleDrive? Or is there something instrinsic to GoogleDrive that prohibits localhost from being run a Google Drive folder?
Locating an Apache virtual host to a cloud-based storage folder will create many files/folders permissions problems.
Instead of relocating your documentRoot and changing a lot of settings and permissions, you should more easily, for each cloud stored project, create a symlink in your Users/myname/Sites folder, pointing to your GoogleDrive/Dropbox website folder.
Imagine you have a "websiteA" folder inside your Dropbox folder :
1) Go to your "Users/myname/Sites folder" and create such a symlink
cd ~/Sites
ln -s ~/Dropbox/websiteA websiteA
As you can check opening your ~/Sites folder in the Finder, you have created a folder with an arrow on it, pointing to the "websiteA" cloud-based folder.
2) Now, you just have to create a virtual host pointing to ~/Sites/websiteA.
You could, instead, globally change your ~/Sites folder to a symlink pointing to your cloud-based folder, but the project-by-project approch is more flexible as it will allow you to manage both local and cloud-based projects.
Many thanks to #DrFred for the solution above, which I'm confident would work though I have not had the chance to test it.
Here's the solution I devised before receiving any answers. It's very similar to Dr Fred's above, in that both solve the problem with symlinks. I add mine for completeness and extra detail.
As above, I develop on multiple devices (several Macs and Windows PCs, side by side), so my aim was to have a single localhost development folder that would synch almost instantly between different devices without the need to check files into/out of git and without running into the file permissions problems created when using Google Drive to synch code files.
The steps I used to achieve this aim were as follows.
Create a folder called ~/Users/myname/SitesNew on a Mac.
Create a symlink from that folder to an identically named folder in Dropbox on the same Mac. You will then have two identical folders on the Mac:
~/Users/myname/SitesNew <-- Real folder on Mac
~/Users/myname/Dropbox/SitesNew <-- Symbolic folder on Mac
Synchronise Dropbox on all devices (making sure to add the SitesNew folder if you are using selective synch on any device). The symlink folder will now appear as a real folder on Dropbox in the cloud and on the Windows PCs. In my case the new Windows PC folder was at:
D:/Dropbox/SitesNew <-- Real folder on Windows
Update the Apache httpd.conf files on the Mac to recognise localhost at ~/Users/myname/SitesNew.
Update the Apache httpd.conf on the Windows PC to recognise localhost at D:/Dropbox/SitesNew.
From now on, any localhost development work (edit, add, delete) on one device will synch with the localhost on the other, even across different operating systems.
Note 1: This solution works only with Dropbox but not with Google Drive, as Google Drive has problems with symlinks and also messes with permissions in a different way, especially on a Mac.
Note 2: If any files have previously been saved on Google Drive (e.g. originally my Windows sites folder was at D:\GoogleDrive\SitesOld), use chmod both (a) to determine the right values for the permissions (e.g. see https://chmod-calculator.com), and (b) to convert folders and files to the right values.
As in the topic.
I wonder since I cannot find this information anywhere and currently I am using a virtual machine (linux) on my vcenter which is cloned and then a special shell script is run on this freshly cloned machine to setup up environment and IP adresses etc.
Maybe I would be able to benefit from templates this way.
I think this will be helpful
https://www.robertparten.com/virtualization/vmware-difference-between-clone-and-template/
Few Differences in my opinion:-
Virtual machine is the running instance while Template is compact copy of VM ( with baseline and factory settings), which can be stored anywhere.
one need to deploy template to make running VM.
one can create copy from both VM and template but in VM you need to clone it and in case of template you need to deploy it.
moving between different setup is easy with template.
Rest are already mentioned in link provided.
But first you need to search on your own and still have doubts than only ask, that's how we all learn.
Happy Learning!
Looking at these two scenarios:
Create a template from your active VM, then deploy from the template.
Deploy from the active VM directly.
As far as I know, there will be no difference in the end result if you run these scenarios in the near future. You'll still have to run a script in order to get your IPs setup, etc.
So what's the difference?
If you mess stuff up with your active VM, change things around or whatever, you lose the ability to deploy from the (good) setup you had.
Once you make a template from your active VM, that configuration is saved as a file on the ESX (or the storage, not 100% sure) and can be re-deployed in the future.
Is there a way to include configuration settings in Apache based on if a directory exists? Basically I have a portable hard drive that I transport between work and home that has some stuff I'm developing on it. I only want the Apache config to load a particular virtual host if the folder exists.
Since Apache 2.4.34 you can now use <IfFile>...</IfFile> which will check to see if a file exists. There's more details on the <IfFile> page.
No, there seems to be no direct way to do this.
The only thing that might be a solution is the IfDefine directive. You can define defines using the -d parameter to when the server is started.
The parameter-name argument is a define as given on the httpd command line via -Dparameter-, at the time the server was started.
You might be able to check for the existence of a directory in a batch or bash file, and set the -d parameter accordingly.
Whether that is an option, will depend on how your server is started from the portable hard drive.
I've come up with a solution that seems to work for Linux and OS X, and it hinges on "mountpoints". It might be possible to emulate it within Windows, as well, but you would probably have to get creative with FUSE and/or Cygwin.
If you create an empty folder in your home directory, such as "/Users/username/ExtraVhosts", you can add an apache directive to "Include /Users/username/ExtraVhosts/*".
Then, when you insert your thumb drive, you can mount somewhere and then use mountpoint "binding" to cross-link the ExtraVhosts folder to a folder on the mobile device.
An OS X example:
I have a thumb drive called 'Cherrybomb'
When I insert it, it always gets mounted to /Volumes/Cherrybomb
I can then use bindfs (sudo port install bindfs) to mount a subfolder of it, like so:
sudo bindfs /Volumes/Cherrybomb/Projects/vhosts /Users/username/ExtraVhosts
Then I can restart apache to read in the updated configuration:
sudo /opt/local/apache2/bin/apachectl restart
At that point, it's just a matter of adding entries in /etc/hosts for server aliases to get picked up.
The linux equivalent would be using the "--bind" parameter of the mount command.
One caveat: This makes it difficult to quickly unmount the USB drive, since it is always marked as "in use" by apache. Here's a removal procedure:
Close all open files and terminal sessions that are using the drive (the present-working-directory in terminal can cause unmount issues)
Stop apache: sudo /opt/local/apache2/bin/apachectl stop
umount /Users/username/ExtraVhosts
Then you can either unmount it graphically or manually (umount /Volumes/Cherrybomb).
If your work and home machines mount the drive to different locations, you could have multiple vhosts folders - home_vhost, work_vhost, etc - and use that in the binding step.
I hope this helps someone out :)
If you point apache to the mountpoint only there shouldn't be an issue. Just don't point Directory directives to directories within the drive.
eg, if you mount /dev/somedisk /mnt/somevhost, the
/mnt/somevhost directory will be there whether or not you have the drive mounted and apache will start. Apache doesn't care if the directory is empty so a <Directory "/mnt/somevhost"/> won't cause server to not start if the drive isn't mounted.
Work with UNIX not against it :-p This solution should be sufficient for development.