I have a data parsing utility in the form of a runnable JAR file. I also have an Apache server (Ubuntu 12.04) to which data files are uploaded. Is there anyway that I could launch said JAR file as a background process when a file is uploaded? (FYI: File access by multiple processes isn't a concern here; I've got file locking in place.)
Related idea: if the above isn't possible, I could always launch the aforementioned JAR file from a bash script. However, I'm still not sure how to do that via Apache. I'm quite a novice at using it effectively.
Edit: Just noticed this potential php solution. Apache folks: is this a good idea, or is there a better solution?
Maybe you can use File Alternation Monitor to achieve this. It can be configured as a background daemon which performs operations if the new file is spotted. If you want to avoid starting while the file is currently uploaded, wait approx. 5 minutes after the file change time and start processing your utility.
I use a similar technique for monitoring uploaded files on a Samba share and it works flawless.
Related
I'd like to offline profile a YARN application. My application starts 10 containers. One of those starts consuming 100% CPU shortly before it crashes. I don't know which one is going to crash. And sadly, as this application starts multiple containers, they could potentially run on the same cluster hardware nodes. If I use a standard JProfiler config, I have to specify a filename for the savepoint. On my cluster, I can't use a relative path (relative to the working directory) as the folder is removed when the application crashes/finishes, so I need to specify an absolute path on the machine which then is the same for all containers overwriting their snapshots...
Is there a way to solve this issue? I'd like to offline profile all of my containers, let them write to a savepoint and pick the snapshot of the one which crashed.
Today, I found an old jprofiler-config.xml for JProfiler 7 on GitHub indicating that there are several more options available compared to the ones I see in my jprofiler-config.xml (JProfiler 11). Are you aware if there is any XSD describing the schema and what can be configured?
The "Save snapshot" action can add a number to the snapshot name to make it unique. It will check for existing files before writing the snapshot.
Since JProfiler 11, the config file only saves modified attributes and not all possible attributes. You can make changes in the GUI and observe the changes in the config file. There is no XSD for the config file format.
I am trying to set-up the testing of the repository using travis-ci.org and Docker. However, I couldn't find any manuals about what is the politics on memory usage.
To perform a set of tests (test.sh) I need a set of input files to run on, which are very big (up to 1 Gb, but average 500 Mb).
One idea is to wget directly in test.sh script, but for each test-run it would be not efficient to download the input file again and again.
The other idea is to create a separate dockerfile containing the test-files and mount it as a drive, but this would be not nice to push such a big dockerimage in the general register.
Is there a general prescription for such tests?
Have you considered using Travis File Cache?
You can write your test.sh script in a way so that it will only download a test file if it was not available on the local file system yet.
In your .travis.yml file, you specify which directories should be cached after a successful build. Travis will automatically restore that directory and files in it at the beginning of the next build. As your test.sh script will then notice the file exists already, it will simply skip the download and your build should be a little faster.
Note that how the Travis cache works is that it will create an archive file and put it on some cloud storage where it will need to download it later on as well. However, the assumption is that the network traffic will likely be inside that "cloud" and potentially in the same data center as well. This should still give you some benefits in terms of build time and lower use of resources in your own infrastructure.
how will I read Segment/index files which is stored inside a jar file using Apache Lucene. If files are kept in a folder then it works properly.But I have to read the file from a jar only.
There is no such functionality out of the box. To make this happen, you would need to implement o.a.l.store.Directory and feed it to the IndexReader.
Needless to say, performance for such a thing would suffer as effectively every change would have to be zipped/unzipped.
I am working with Rails 3.0.3 and Bluepill 0.0.51. I am trying to troubleshoot a situation where Bluepill is trying to start multiple instances of my ruby servers in some cases, but having trouble knowing where to start since the only files I have to look at are my ruby server .rb files and my .pill file in my rails app root folder. What are the configuration/other files that Bluepill creates on my system? Thanks so much in advance.
When you do run a successful pill recepie it can create different files based on your configuration. If you demonize the process it will create a .pid for each process that stores a value that represents the Process ID of that process so it can shut it down. It will also create a log file and sometimes files in sock. It uses /var/run/bluepill by default but if you run it in --no-privileged mode it will want you to specify the location, preferebly in your applications folder.
It doesn't create anything to configure but rather you configure everything in the pill file. It can be tricky getting everything to work but you have to keep trying. I realize this post is old so i hope your solved it :)
I recently modify the "include_path" var in my php.ini file. Before you ask, I restarted the apache service. The change work for every pages we access from a web browser.
The problem is the cron jobs doesn't seems to consider that change. When I do a phpinfo() inside the cron job, it uses the same php.ini file than the web server and it is the one I changed, but the value beside "include_path" is the old one.
So is there a way to "restart" crontab?
Or maybe there is another problem?
Several systems use a separate php.ini file for web and CLI. You will need to make changes in that one as well: How to find the php.ini file used by the command line?
The easiest way to find this file is to run this at the command line: php --ini which will result in output like this:
user#computer:~$ php --ini
Configuration File (php.ini) Path: /etc/php5/cli
Loaded Configuration File: /etc/php5/cli/php.ini
Scan for additional .ini files in: /etc/php5/cli/conf.d
What you see as "Loaded Configuration File:" is where you need to add your changes.
EDIT: Another option, is in your CRON script use set_include_path() to make the change at runtime.
PHP generally has two .ini files. One for in-webserver (SAPI) and one for command-line (CLI) usage. If you modified only the SAPI one, then anything running from CLI (e.g. cron jobs) will not see the change.
do a php -i at the command line to see where PHP is looking for its ini file while in that mode.