I have a pipeline that needs to copy some files from a folder to a new one only if the files exists in the source folder.
This is my script line:
script:
- cp source_folder/file.txt dest_folder/ 2>/dev/null
I have also tried this:
script:
- test -f source_folder/file.txt && cp source_folder/file.txt dest_folder/ 2>/dev/null
but still fails if the file do not exists.
Cleaning up project directory and file based variables.
ERROR: Job failed: exit code 1
How can I check the file and copy it only if exists?
EDIT:
this command is executed on a server, the pipeline use ssh to log into
Check for the existence of the file (-f) and, in positive case, copy it.
script:
- |
files=(conf.yaml log.txt)
for file in $files; do
if [[ -f "source_folder/$file" ]]; then
cp source_folder/$file dest_folder
fi
done
Take a look at other answers for one-shot less-flexible statements.
Note: I haven't tested the script above, but I'm quite accustomed with Gitlab pipeline and bash.
I have a nextflow script that sources scripts located in the ./bin directory that is located where I invoke the nextflow script. When I run the workflow without a container, Nextflow can find these scripts and execute them. However, when I run Nextflow with a container the scripts cannot be found, despite that I attempted to add those scripts to the container file.
I guess that I am either not adding the executables to the container properly or that I am not referencing them properly in the Nextflow script or config file. Any help is appreciated.
Here is an example process:
process clean {
// Remove common contaminants from fastq(s) using tapioca script
// see: https://github.com/ncgr/tapioca
input:
path fastq_file from fastq_raw
val x from machine_name
output:
path 'out.fastq' into clean_out
stdout ch1
script:
"""
echo "Number of reads in $fastq_file"
grep -c "^#" $fastq_file
tap_contam_analysis --db ${dbdir}/phix174 --pct 80 ${fastq_file} > hits.txt
echo "PhiX filtering completed for ${fastq_file}"
"""
}
Note that the "tap_contam_analysis" script is a perl script located in ./bin from where I invoke the Nextflow script.
Here are the relevant parts of my Docker file. Note that I attempted to modify the $PATH in hopes that would fix the issue...no luck:
FROM ubuntu:18.04
ARG DEBIAN_FRONTEND=noninteractive
WORKDIR /usr/src
#Copy all the stuff for this Nextflow workflow (python and perl scripts)
COPY . .
ENV PATH=${PATH}:/usr/src/bin
Finally, here is my nextflow.config file:
process {
container = 'nf_se_demux_to_bam_bai_denovo'
}
The executables just need to be added to somewhere in your container's $PATH. Folks often like to use /usr/local/bin for this, but you can check other locations with:
docker run --rm ubuntu:18.04 bash -c 'echo $PATH'
Your Dockerfile, therefore, might look like this:
FROM ubuntu:18.04
ARG DEBIAN_FRONTEND=noninteractive
WORKDIR /usr/src
COPY ./bin /usr/local/bin
Hi I am looking to automate my file transfering to my Jailbroken iPhone over USB with a bash file. Which will launch the relay then do the file transfers
With this here I installed and successfully transfered files to my iPhone with cygwin but now I want to automate the file transfer.
First I need to start the relay with cygwin and those commands are required
cd pyusbmux/python-client/
chmod +x *
./tclrelay.py -t 22:2222
so I created a .sh file that does it but when I launch it cygwin gives me those errors
This is what should happen on the left and the result of the script on the right
How can I make cygwin open with thoses commands
In addition to be sure that tcpON.sh has proper line termination with d2u of dos2unix package:
d2u tcpON.sh
You should add a proper SHEBANG on the first line of your script
https://linuxize.com/post/bash-shebang/
#!/bin/bash
cd /cygdrive/e/Grez/Desktop
cd pyusbmux/python-client/
chmod +x *
./tclrelay.py -t 22:2222
You can use as base the Cygwin.bat and make a tcpON.bat batch file like:
C:
chdir c:\cygwin64\bin
bash --login /cygdrive/e/Grez/Desktop/tcpON.sh
Verify the proper cd command to be sure that you are always in the expected directory.
It is not the only way but probably the most flexible (IMHO)
I'm on a Windows machine using Git 2.7.2.windows.1 with MinGW 64.
I have a script in C:/path/to/scripts/myScript.sh.
How do I execute this script from my Git Bash instance?
It was possible to add it to the .bashrc file and then just execute the entire bashrc file.
But I want to add the script to a separate file and execute it from there.
Let's say you have a script script.sh. To run it (using Git Bash), you do the following: [a] Add a "sh-bang" line on the first line (e.g. #!/bin/bash) and then [b]:
# Use ./ (or any valid dir spec):
./script.sh
Note: chmod +x does nothing to a script's executability on Git Bash. It won't hurt to run it, but it won't accomplish anything either.
#!/usr/bin/env sh
this is how git bash knows a file is executable. chmod a+x does nothing in gitbash. (Note: any "she-bang" will work, e.g. #!/bin/bash, etc.)
If you wish to execute a script file from the git bash prompt on Windows, just precede the script file with sh
sh my_awesome_script.sh
if you are on Linux or ubuntu write ./file_name.sh
and you are on windows just write sh before file name like that sh file_name.sh
For Linux -> ./filename.sh
For Windows -> sh file_name.sh
If your running export command in your bash script the above-given solution may not export anything even if it will run the script. As an alternative for that, you can run your script using
. script.sh
Now if you try to echo your var it will be shown. Check my the result on my git bash
(coffeeapp) user (master *) capstone
$ . setup.sh
done
(coffeeapp) user (master *) capstone
$ echo $ALGORITHMS
[RS256]
(coffeeapp) user (master *) capstone
$
Check more detail in this question
I had a similar problem, but I was getting an error message
cannot execute binary file
I discovered that the filename contained non-ASCII characters. When those were fixed, the script ran fine with ./script.sh.
Once you're in the directory, just run it as ./myScript.sh
If by any chance you've changed the default open for .sh files to a text editor like I had, you can just "bash .\yourscript.sh", provided you have git bash installed and in path.
I was having two .sh scripts to start and stop the digital ocean servers that I wanted to run from the Windows 10. What I did is:
downloaded "Git for Windows" (from https://git-scm.com/download/win).
installed Git
to execute the .sh script just double-clicked the script file it started the execution of the script.
Now to run the script each time I just double-click the script
#!/bin/bash at the top of the file automatically makes the .sh file executable.
I agree the chmod does not do anything but the above line solves the problem.
you can either give the entire path in gitbash to execute it or add it in the PATH variable
export PATH=$PATH:/path/to/the/script
then you an run it from anywhere
I have a shell script which in turn calls sql file. Its a bash shell running in UNIX. Following is the main steps taken in script.
1) Generate Term file
2) Remove previous day's Term and Rpt file from utility directory.
3) Copy Term file from Run directory to Utility directory.
4) Run the sql file
5) Copy the output, RPT file from Utility to Run directory.
Here is the code snippet:
> RUN_DIR/nj.terms
if [[ -s RUN_DIR/nj.terms ]] then
rm -f /utl/nj.terms
rm -f /utl/nj.rpt
cp RUN_DIR/nj.terms utl
/bin/sqlplus USER PSWD #sql
cp utl/nj.RPT RUN_DIR
fi
I get following error from sql output as :
ORA-29283 - Invalid file operation.
Mostly this error is due to absence of Term file whenever the sql runs. Due this error RPT file will not be generated and cause failure in following copy command (cp utl/nj.RPT RUN_DIR).After the failure, when we checked the Term file ,it was present in Utl directory.
This error occurs randomly. Is there any chance system takes more time to copy the Term file to Utility directory and before completing it sql was run? It would be great if someone can help me in this situation.