I need to start an external process (which is around 300MB large on its own) several times using System.Diagnostics.Process.
The only problem is: once the first instance starts, it generates temporary data in its base folder (where the application is located), so I can't just start another instance - it would corrupt the data of the first one and mess up everything.
I thought about temporarily copying the whole application folder programmatically, so that each instance has its own, but that doesn't feel right.
Could anybody help me out? Thanks in advance!
Try starting each copy in a different directory.
If the third-party app ignores the current directory, you could make a symlink to it in a different folder. I'm not necessarily recommending that, though.
Pass an argument to your external process that specifies the temp folder to use.
Related
Are there guidelines regarding how to share a Snakemake workflow among multiple users on the same data under Linux, or is the whole thing considered bad practice?
Let me explain in case it's not clear:
Suppose user A executes a workflow in directory dir/. Assume the workflow terminates successfully, and he/she then properly sets file/directory permissions recursively on all output and intermediate files and the .snakemake/ subdirectory for other users to read/write, of course.
User B subsequently navigates to dir/, adds input files to the workflow, then executes it. Can anything go wrong?
TL;DR: I'm asking about non-concurrent execution of the same workflow by distinct users on the same system, and on the same data on disk. Is Snakemake designed for such use cases?
It's possible to run snakemake --nolock which will prevent locking of the directory, so multiple runs can be made from inside the same directory. However, without lock, there's now an opening for errors due to concurrent runs trying to modify the same files. It's probably OK, if you are certain that this will be avoided, e.g. if you are in constant communication with another user about which files will be modified.
An alternative option is to create a third directory/path, and put all the data there. This way you can work from separate directories/path and avoid costly recomputes.
I would say that from the point of view of snakemake, and workflow management in general, it's ok for user B to add or update input files and re-run the pipeline. After all, one of the advantages of a workflow management system is to update results according to new input. The problem is that user A could find her results updated without being aware of it.
From the top of my head and without more detail this is what I would suggest. Make snakemake read the list of input files from a table (pandas comes in handy for this) or from some configuration file. Keep this sample sheet under version control (with git/github) together with the Snakefile and other source code.
When users update the working directory with new files, they will also need to update the sample sheet in order for snakemake to "see" the new input and other users will know about it via version control. I prefer this setup over dumping files in a directory and letting snakemake process whatever is in there.
This is an open ended question. I have noob understanding of databases but willing to learn whatever is required. Though I believe my problem could be done without learning a lot.
So, here goes the question:
I have large amount of files getting generated in mt projects(depending on the builds) and I need to archive them and also need to reproduce them according to buildNumber if requested by users. I don't expect these requests to be a lot. May be 1-2 requests a day.
For eg: 16GB data per build every week. Most of the files in weekly builds are duplicate. And I don't want to archive them again and again. I prefer to store them only once. There is one caveat that it can happen that the files relative location can change, even though content hasn't changed.
My approach is as follow: Create a hash from each file. Create the key-value pair as fileHash-actual file and store it. Store this information in some kind of manifest file for each build. So, I should be able to create the builds back with correct files/paths etc.
Can it ever happen that 2 different files will ever have the same hash? Can some database help to do it efficiently? I am currently thinking of dumping all files in one folder.
Thanks
I'm trying to copy a file from a local computer to a remote share but I get an exception saying "Can't find the specified file".
My.Computer.FileSystem.CopyFile("C:\filename.jpg", "\\focserver2\consultoria\teste\filename.jpg")
The remote shared folder has full control permissions on "Everyone".
What am I doing wrong? Or it's not possible to copy to network paths using FileSystem.CopyFile ?
Thanks.
João
The issue does not appear to be the destination but the source. I assume that the sample you showed above is NOT the real code and in the real code, you combine a couple values together to define the source. To help prevent issues, get in the habit of using the Path.Combine() method when concatenating strings for a file path. It's a life-saver.
The next most important thing is to learn how to debug your code by setting break-points and seeing what the values of your combined strings are before posting to websites. This is a good one to get you started. http://weblogs.asp.net/scottgu/debugging-tips-with-visual-studio-2010
Most likely the problem is that you are trying to access the root folder of the C:\ drive. This folder is generally locked down by the operating system, and even simple file operations don't work easily there.
Try copying the file from a subfolder e.g.
My.Computer.FileSystem.CopyFile("C:\Temp\filename.jpg", "\\focserver2\consultoria\teste\filename.jpg")
I have a program that uploads .txt or .rje files from a folder. Now when you put any other file format into the folder, like .jar, then the application crashes.
Now I cannot change the mechanics of the application, so I would like to know if there is a type of program/script that I can use that monitors the folder for any files that are non-txt/rje and then move them out of the folder once they are put there...
Is this possible using a script? (I do not want to use a .exe application to do this...not allowed to install 3rd party software onto the server this folder exists...)
Thank you
Your solution won't work as you have a race condition between the program doing the upload and the one doing the deletion. If upload runs first it still crashes.
The correct solution is to modify the upload program to cope with this scenario.
If that is not possible then the only safe work around would be to use a new folder to drop the files in, have a script run that constantly scans the folder and if a new file appears either move it to the processing folder or deletes it as appropriate.
(For the actual detection that's not my area of expertise but the simplest would be to have a bat file that just runs periodically (or even just runs once and loops with a wait, check, move, wait, check, move, etc) and processes everything in the folder when it runs).
what language would be useful to make a program that looks in a directory and saves the amount of files inside of it as a variable then have it check if the variable does not equal what it was originally when it counted then run a script. since i have a folder that adds a new file to the directory every time something is changed i want a script to run that just runs an exe.
I have no idea on where to get started, if i should make a vb.net app thats a timer that runs every so often and does this for me. please help!!!
Thank you so much. Sorry I included no code I'm not expecting code in return i just need ideas on how i would plan this or go about doing it.
You can look into this class for watching updates into a directory and taking action against various types of updates
http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx