On Linux, we can launch a shell script, which keeps writing lines into a file, and at the same time, another "tail -f" process keeps reading the latest tail of the file.
I know that when two processes are accessing the same file, some file locking mechanisms should be utilized to avoid conflict and undefined behaviour, just as in-memory mutexes. Does that mean all the Linux shell commands which involve file reading/writing already have that under their belts? Or do I misunderstand something and this kind of locking is not a necessitate?
Related
I use read command for some read files in the scenarios.
If the tests are running, the files can not be moved and deleted.
It seems that the file stream is not closed when it uses read command.
Can it close the files as declarative?
Environment:
Karate 0.9.2
We are also writing log files and in some cases this can cause issues, see: https://github.com/intuit/karate/issues/661#issuecomment-458127918
But I think keeping a file lock is expected. Also we are closing files whenever we read them, but maybe we have a bug. So if you can create a sample project to replicate your problem, that would be great: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I download Redis-x64-3.2.100.zip from Here, when i extract it i see two files redis.windows.conf and redis.windows-service.conf, i compare these two files they really similar, except in logfile and syslog-enabled and syslog-ident redis, and when i want install the Redis i should change configuration on redis.windows-service.conf, not redis.windows.conf.
so what is the diff between redis.windows.conf and redis.windows-service.conf?
redis.windows-service.conf is meant to be run as a service/daemon. This means that it is meant to be run in the background and managed by the OS (started on reboot, restarted if it crashes, etc).
And redis.windows.conf is meant to be run from the command line or a script and managed in user space.
The only substantive difference between the two, as you have pointed out, is that the service hooks into the windows syslog, which is a really good idea for services, but usually unnecessary for normal processes.
Whichever one you choose, you'll probably need to tweak it for your own purposes. They are just around as samples.
For our development purposes, we have a base template installation of ExpressionEngine which is duplicated whenever we start a new project. This process involves copying the directory contents and changing ownership via SSH, then synchronizing the databases via phpMyAdmin.
I was wondering if it might be possible to compile a batch script (or equivalent) to perform all these operations via a single terminal command, and if so, I would appreciate some basic foundation on which to build the script.
You mean something like this?
#!/bin/sh
cp -R /foo /bar
chmod 644 /bar/*
mysql < default.sql
Note that I'm just importing a default file to MySQL, not synchronizing two different sets of data, but it doesn't sound like that's what you meant anyway.
I have a data parsing utility in the form of a runnable JAR file. I also have an Apache server (Ubuntu 12.04) to which data files are uploaded. Is there anyway that I could launch said JAR file as a background process when a file is uploaded? (FYI: File access by multiple processes isn't a concern here; I've got file locking in place.)
Related idea: if the above isn't possible, I could always launch the aforementioned JAR file from a bash script. However, I'm still not sure how to do that via Apache. I'm quite a novice at using it effectively.
Edit: Just noticed this potential php solution. Apache folks: is this a good idea, or is there a better solution?
Maybe you can use File Alternation Monitor to achieve this. It can be configured as a background daemon which performs operations if the new file is spotted. If you want to avoid starting while the file is currently uploaded, wait approx. 5 minutes after the file change time and start processing your utility.
I use a similar technique for monitoring uploaded files on a Samba share and it works flawless.
When Jenkins invokes maven build, /tmp fills with 100s of surefire839014140451157473tmp, how to explicitly redirect to another directory during the build. For clover build it fills with 100s of grover53460334580.jar? Any idea to over come this?
And any body know exact step by step to create ramdisk so I could redirect surefire stuffs into that ramdisk ? Will it save write time to hard drive?
Thanks
Many programs respect the TMPDIR (and sometimes TMP) environment variables. Maybe Jenkins uses APIs that respect them? Try:
TMPDIR=/path/to/bigger/filesystem jenkins
when launching Jenkins. (Or however you start it -- does it run as a daemon and have a shell-script to launch it?)
There might be some performance benefit to using a RAM-based filesystem -- ext3, ext4, and similar journaled filesystems will order writes to disk, and even a quick fd=open(O_CREAT); unlink(fd); sequence will probably require both on-disk journal updates and directory updates. (Homework: test this.) A RAM-based filesystem won't perform the journaling, and might or might not write anything to disk (depending upon which one you pick).
There are two main choices: ramfs is a very simple window into the kernel's caching mechanism. There is no disk-based backing for your files at all, and no memory limits. You can fill all your memory with one of these very quickly, and suffer very horrible consequences. (Almost no programs handle out-of-disk well, and the OOM-killer can't free up any of this memory.) See the Linux kernel file Documentation/filesystems/ramfs-rootfs-initramfs.txt.
tmpfs is a slight modification of ramfs -- you can specify an upper limit on the space it can allocate (-o size) and the page cache can swap the data to the swap partitions or swap files -- which is an excellent bonus, as your memory might be significantly better used elsewhere, such as keeping your compiler, linker, source files, and object files in core. See the Linux kernel file Documentation/filesystems/tmpfs.txt.
Adding this line to your /etc/fstab will change /tmp globally:
tmpfs /tmp tmpfs defaults 0 0
(The default is to allow up to half your RAM to be used on the filesystem. Change the defaults if you need to.)
If you want to mount a tmpfs somewhere else, you can; maybe combine that with the TMPDIR environment variable from above or learn about the new shared-subtree features in Documentation/filesystems/sharedsubtree.txt or made easy via pam_namespace to make it visible only to your Jenkins and child processes.