Related
I need to be able to pass some parameters to my virtual machine during it's bootup so it sets itself properly. To do that I either have to bake the info into the image or somehow pass it as parameters to my qemu-kvm command. These parameters are just few, and if it was VMware, we would just pass it as ova paramas and when the VM launches we would call the ova-environment to get these params. But launching it from qemu-kvm I have no such options. I did some homework and found that I could use virtio-9p driver for sharing files across host and guest. Unfortuantely RHEL/Centos has decided not to support 9p.
With no option of rebuilding my RHEL kernel with the 9p options enabled, how do I solve my above problem? Either solution would work, which is, pass/share some kind of json file to the VM(pre-populated on the host), which will read this and do it's setup OR set some kind of "environment variables" which I can query from within the VM to get these params and continue with setup. Any pointers would help.
If your version of QEMU supports it, you could use its -fw_cfg option to pass information to the guest. If that guest is running a Linux kernel with CONFIG_FW_CFG_SYSFS enabled, you will be able to read out the information from sysfs. An example:
If you launch your VM like so:
qemu-system-x86_64 <OPTIONS> -fw_cfg name=opt/com.example.test,string=qwerty
From inside the guest, you can then get the value back from sysfs:
cat /sys/firmware/qemu_fw_cfg/by_name/opt/com.example.test/raw
There appears to be some driver for Windows as well, but I've never used it.
When you boot your guest with -kernel and -initrd you should be able to pass environment variables with -append.
The downside is that you have to keep track of your current kernel and initrd outside of your disk image.
Other possibilities could be a small prepared disk image (as you said) or via network/dhcp or a serial link into your guest or ... this really depends on your environment.
I was just searching to see if this situation had improved and came across this question. Apparently it has not improved.
What I do is output my variable data to a temp file (eg. /tmp/xxFoo). Usually I write text or a tar straight to that file then truncate it to a minimum size and 512 byte multiple like 64K otherwise the disk controller won't configure it. Then the VM starts with a raw drive as that file. After the VM is started the temp file is deleted. From within the guest you can read/cat the raw block device and get the variable data (in BSD use the c partition as the raw drive).
In Windows guests it's tricky to get to the data. In theory you can read \\.\PhysicalDriveN but I have not ever been able to get that to work. Cygwin can do it and it works like Linux. The other option is to make your temp file a partitioned and formatted image but that's a pain to create and update.
As far as sharing a folder I use Samba which works in just about anything. I usually use several instances of smbd running with different configurations.
One option is to create a ISO file and pass as parameter. This works for both host Win and Ubuntu and Guest Win and Ubuntu. You can read the mounted CD ROM inside the guest OS
>>qemu-system-x86_64 -drive file=c:/qemuiso/winlive1.qcow2,format=qcow2 -m 8G -drive file=c:\qemuiso\sample.iso,index=1,media=cdrom
On Guest Linux Mount CDROM in Ubuntu:-
>>blkid //to check if media is there
>>sudo mkdir /mnt/cdrom
>>sudo mount /dev/sr0 /mnt/cdrom //this step can also be put in crontab
>>cd /mnt/cdrom
I am doing an intensive computing project with a super old C program. The program requires a library called Sun Performance Library which is a commercial ware. Instead of purchasing the library by myself, I am running the program by logging onto a Solaris machine in our computer lab with the ssh command, while the working directory to store output data is still on my local Mac.
Now, a problem just occurred: the program uses large amount of disk space to save some intermediate results and the space on my local Mac is quickly filled (50 GB for each user prescribed by the administrator). These results are necessary for the next stage of computing and I cannot delete any of them before it finally produce the output data. Therefore, I have to move the working directory to an external hard drive in order to continue. Obviously,
cd /Volumes/VOLNAME
is not the correct way to do it because the remote machine will give me a prompt saying
/Volumes/VOLNAME: No such file or directory.
So, what is the correct way to do it?
sshfs recently added support for "slave mode" which allows you to do this. Assuming you have sshfs on Solaris (I'm not sure about this), the following command (ran from your Mac) will do what you want: dpipe /usr/lib/openssh/sftp-server = ssh SOLARISHOSTNAME sshfs MACHOSTNAME:/Volumes/VOLNAME MOUNTPOINT -o slave
This will result in the MOUNTPOINT directory on the server being mounted to your local external drive. Note that I'm not sure whether macOS has dpipe. If it doesn't, you can replace it with one of the equivalent solutions at How to make bidirectional pipe between two programs?. Also, if your SFTP server binary is somewhere else, substitute its path.
The common way to mount a remote volume in Solaris is via NFS, but that usually requires root permissions.
Another approach would be to make your application read its data from stdin and output its results to stdout, without using the file system directly. Then you could just redirect the data from/to your local machine through ssh. For instance:
ssh user#host </Volumes/VOLNAME/input.data >/Volumes/VOLNAME/output.data
I'm trying to setup a crontab to execute at set intervals. The crontab job is setup as part of my PHP-Slim application running on Apache. For some reason, it just doesn't add the job to the crontab, so when I run the command:
crontab -u daemon -l
It says 'no crontab for daemon' (daemon is the default Apache account). I did manage to get the cronjob manually added using another account (and it executes with no further issues) so it's most likely a permissions issue. What is the best way to troubleshoot this, without resorting to things like chmod 777 (it will be a production server so I need to careful with setting permissions and documenting them)?
Managed to find the answer just after posting.
I looked in the log file for cron:
cat /var/log/cron
Lots of (daemon) AUTH (crontab command not allowed) error messages. Some further googling lead me to look at /etc/cron/allow which doesn't exist, but /etc/cron.deny does, and the daemon account was listed there. Problem solved.
By default we do not allow the user daemon to run crontab jobs. If you want that user to run crontab jobs, you would need to modify /etc/cron.deny and remove the daemon user from there.
Hope it helps.
I've recently created a dropbox system using inotify, watching for files created in a particular directory. The directory I'm watching is mounted from an NFS server, and inotify is behaving differently than I'd expect. Consider the following scenario in which an inotify script is run on machine A, watching /some/nfs/dir/also/visible/to/B.
-Using machine A to create a file in /some/nfs/dir/also/visible/to/B, the script behaves as expected. Using machine B to carry out the same action, the script is not notified about a new file dropped in the directory.
-When the script is run on the NFS server, it gets notified when files are created from both machine A and machine B.
Is this a bug in the bug in the package I'm using to access inotofy, or is this expected behaviour?
inotify requires support from the kernel to work. When an application tracks a directory, it asks the kernel to inform it when those changes occur. When the change occurs, in addition to writing those changes to disk, the kernel also notifies the watching process.
On a remote NFS machine, the change is not visible to the kernel; it happens entirely remotely. NFS predates inotify and there is no network level support for it in NFS, or anything equivalent.
If you want to get around this, You can run a service on the storage server (since that kernel will always see changes to the filesystem) that brokers inotify requests for remote machines, and forward the data to the remote clients.
Edit: It seems odd to me that NFS should be blamed for its lack of support for inotify.
Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems in 1984, wikipedia article
However:
Inotify (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem. [...] It has been included in the mainline Linux kernel from release 2.6.13 (June 18, 2005 ) [...]. wikipedia article
It's hard to expect a portable network protocol/application to support a specific kernel feature developed for a different operating system and that appeared more than twenty years later. Even if it did include extensions for it, they would not be available or useful on other operating systems.
*emphasis mine in all cases
Another problem with this; Lets suppose we are not using a network at all, but rather, a local filesystem with good inotify support: ext3 (suppose its mounted at /mnt/foo). But instead of a real disk, the filesystem is mounted from a loopback device ; and the underlying file is in turn accessible at a different location in the vfs (say, /var/images/foo.img).
Now, you're not supposed to modify mounted ext3 filesystems, But it's still reasonably safe to do so if the change is to file contents instead of metadata.
So suppose a clever user modifies the file system image (/var/images/foo.img) in a hex editor, replacing a file's contents with some other data, while at the same time an inotify watch is observing the same file on the mounted filesystem.
There's no reasonable way one can arrange for inotify to always inform the watching process of this sort of change. Although there are probably some gyrations that could be take to make ext3 notice and honor the change, none of that would apply to, say, the xfs drtiver, which is otherwise quite similar.
Nor should it. You're cheating!. inotify can only inform you of changes that occured through the vfs at the actual mountpoint being watched. If the changes occured outside that VFS, because of a change to the underlying data, inotify can't help you and isn't designed to solve that problem.
Have you considered using a message queue for network notification?
To anyone who has come across this question in the search for an answer of why bind mounting on Docker will not detect file changes from host directory (for hot reloading of an app), it's because the propagation of file changes between host and container is not communicated to the container kernel.
Only changes from the container itself is communicated to the kernel. Solution for this is to have your live reload utility turn on "polling mode" instead of using fsnotify.
I found an SGI FAM using an supervisor daemon to monitor file modification. It supports NFS and you can see some description on wiki
I agree with SingleNegationElimination's explanation, and would like to add that iSCSI targets will work, since they alert the kernel.
So things on "real" file systems (relative to the system, that is) will trigger Inotify to alert. Like Rsync'ing, net-catting something into a mounted partition.
If you have to get notifications via inotify (or have to use inotify) you can make a cron to rsync -avz over to the file system. Drawbacks of course are that you are using real system hdd space.
I second #SingleNegationElimination.
Also, you can try notify-forwarder.
Machine A watches for local inotify events, then forwards them to Machine B (via UDP).
Machine B doesn't (can't?) replay the events, but fires an ATTRIB event for the changed file.
If you use vagrant, use vagrant-notify-forwarder.
the problem with notify-forwarder is that it does not trigger an inotify event. It uses utime to update the timestamp for the file on the remote system but inotify fails to see this.
AFAIK, the timestamp already gets updated when using an NFS mount. I have verified this myself between a Synology NAS NFS server and a Raspbian NFS mount (client).
Here's my solution / hack on the client:
#!/bin/bash
path=$1
firstmd5=`ls -laR $path | md5sum | awk ' { print $1 }'`
while true
do
lastmd5=`ls -laR $path | md5sum | awk ' { print $1 }'`
if [ $firstmd5 != $lastmd5 ]
then
firstmd5=$lastmd5
echo files changed
fi
sleep 1
done
Granted, this doesn't report on the specific file being changed, but does provide a general notification hook that something's changed.
It's annoying / kludgy but if I needed more details I would do some additional hacking to isolate the actual files changed.
improved the script with action on click and icon
#!/bin/bash
DAT=$(date +%Y%m%d)
CAM="cam1 "
CHEMIN=/mnt/cams/cam1/$DAT/
first="$CHEMIN"
if [ -d "$CHEMIN" ];then
first=`ls -1rt $CHEMIN | tail -n 1`
fi
echo $first
while true
do
if [ -d "$CHEMIN" ];then
last=`ls -1rt $CHEMIN | tail -n 1`
if [ $first != $last ]
then
first=$last
echo $last created
#notify-send -h string:desktop-entry:nautilus -c "transfer.complete" -u critical -i $PWD../QtVsPlayer.png $CAM $last"\n\r"$CHEMIN
reply=$(dunstify -a QtVsPlayer -A 'open,ouvrir' -i "QtVsPlayer" "$CAM $last"\n\r"$CHEMIN")
if [[ "$reply" == "open" ]]; then
QtVsPlayer -s $CHEMIN$last
fi
fi
fi
sleep 5m
done
Is there a way to include configuration settings in Apache based on if a directory exists? Basically I have a portable hard drive that I transport between work and home that has some stuff I'm developing on it. I only want the Apache config to load a particular virtual host if the folder exists.
Since Apache 2.4.34 you can now use <IfFile>...</IfFile> which will check to see if a file exists. There's more details on the <IfFile> page.
No, there seems to be no direct way to do this.
The only thing that might be a solution is the IfDefine directive. You can define defines using the -d parameter to when the server is started.
The parameter-name argument is a define as given on the httpd command line via -Dparameter-, at the time the server was started.
You might be able to check for the existence of a directory in a batch or bash file, and set the -d parameter accordingly.
Whether that is an option, will depend on how your server is started from the portable hard drive.
I've come up with a solution that seems to work for Linux and OS X, and it hinges on "mountpoints". It might be possible to emulate it within Windows, as well, but you would probably have to get creative with FUSE and/or Cygwin.
If you create an empty folder in your home directory, such as "/Users/username/ExtraVhosts", you can add an apache directive to "Include /Users/username/ExtraVhosts/*".
Then, when you insert your thumb drive, you can mount somewhere and then use mountpoint "binding" to cross-link the ExtraVhosts folder to a folder on the mobile device.
An OS X example:
I have a thumb drive called 'Cherrybomb'
When I insert it, it always gets mounted to /Volumes/Cherrybomb
I can then use bindfs (sudo port install bindfs) to mount a subfolder of it, like so:
sudo bindfs /Volumes/Cherrybomb/Projects/vhosts /Users/username/ExtraVhosts
Then I can restart apache to read in the updated configuration:
sudo /opt/local/apache2/bin/apachectl restart
At that point, it's just a matter of adding entries in /etc/hosts for server aliases to get picked up.
The linux equivalent would be using the "--bind" parameter of the mount command.
One caveat: This makes it difficult to quickly unmount the USB drive, since it is always marked as "in use" by apache. Here's a removal procedure:
Close all open files and terminal sessions that are using the drive (the present-working-directory in terminal can cause unmount issues)
Stop apache: sudo /opt/local/apache2/bin/apachectl stop
umount /Users/username/ExtraVhosts
Then you can either unmount it graphically or manually (umount /Volumes/Cherrybomb).
If your work and home machines mount the drive to different locations, you could have multiple vhosts folders - home_vhost, work_vhost, etc - and use that in the binding step.
I hope this helps someone out :)
If you point apache to the mountpoint only there shouldn't be an issue. Just don't point Directory directives to directories within the drive.
eg, if you mount /dev/somedisk /mnt/somevhost, the
/mnt/somevhost directory will be there whether or not you have the drive mounted and apache will start. Apache doesn't care if the directory is empty so a <Directory "/mnt/somevhost"/> won't cause server to not start if the drive isn't mounted.
Work with UNIX not against it :-p This solution should be sufficient for development.