Redirect stderr to stdout in C shell - io-redirection

When I run the following command in csh, I got nothing, but it works in bash.
Is there any equivalent in csh which can redirect the standard error to standard out?
somecommand 2>&1

The csh shell has never been known for its extensive ability to manipulate file handles in the redirection process.
You can redirect both standard output and error to a file with:
xxx >& filename
but that's not quite what you were after, redirecting standard error to the current standard output.
However, if your underlying operating system exposes the standard output of a process in the file system (as Linux does with /dev/stdout), you can use that method as follows:
xxx >& /dev/stdout
This will force both standard output and standard error to go to the same place as the current standard output, effectively what you have with the bash redirection, 2>&1.
Just keep in mind this isn't a csh feature. If you run on an operating system that doesn't expose standard output as a file, you can't use this method.
However, there is another method. You can combine the two streams into one if you send it to a pipeline with |&, then all you need to do is find a pipeline component that writes its standard input to its standard output. In case you're unaware of such a thing, that's exactly what cat does if you don't give it any arguments. Hence, you can achieve your ends in this specific case with:
xxx |& cat
Of course, there's also nothing stopping you from running bash (assuming it's on the system somewhere) within a csh script to give you the added capabilities. Then you can use the rich redirections of that shell for the more complex cases where csh may struggle.
Let's explore this in more detail. First, create an executable echo_err that will write a string to stderr:
#include <stdio.h>
int main (int argc, char *argv[]) {
fprintf (stderr, "stderr (%s)\n", (argc > 1) ? argv[1] : "?");
return 0;
}
Then a control script test.csh which will show it in action:
#!/usr/bin/csh
ps -ef ; echo ; echo $$ ; echo
echo 'stdout (csh)'
./echo_err csh
bash -c "( echo 'stdout (bash)' ; ./echo_err bash ) 2>&1"
The echo of the PID and ps are simply so you can ensure it's csh running this script. When you run this script with:
./test.csh >test.out 2>test.err
(the initial redirection is set up by bash before csh starts running the script), and examine the out/err files, you see:
test.out:
UID PID PPID TTY STIME COMMAND
pax 5708 5364 cons0 11:31:14 /usr/bin/ps
pax 5364 7364 cons0 11:31:13 /usr/bin/tcsh
pax 7364 1 cons0 10:44:30 /usr/bin/bash
5364
stdout (csh)
stdout (bash)
stderr (bash)
test.err:
stderr (csh)
You can see there that the test.csh process is running in the C shell, and that calling bash from within there gives you the full bash power of redirection.
The 2>&1 in the bash command quite easily lets you redirect standard error to the current standard output (as desired) without prior knowledge of where standard output is currently going.

I object the above answer and provide my own. csh DOES have this capability and here is how it's done:
xxx |& some_exec # will pipe merged output to your some_exec
or
xxx |& cat > filename
or if you just want it to merge streams (to stdout) and not redirect to a file or some_exec:
xxx |& tee /dev/null

As paxdiablo said you can use >& to redirect both stdout and stderr. However if you want them separated you can use the following:
(command > stdoutfile) >& stderrfile
...as indicated the above will redirect stdout to stdoutfile and stderr to stderrfile.

xxx >& filename
Or do this to see everything on the screen and have it go to your file:
xxx | & tee ./logfile

What about just
xxx >& /dev/stdout
???

I think this is the correct answer for csh.
xxx >/dev/stderr
Note most csh are really tcsh in modern environments:
rmockler> ls -latr /usr/bin/csh
lrwxrwxrwx 1 root root 9 2011-05-03 13:40 /usr/bin/csh -> /bin/tcsh
using a backtick embedded statement to portray this as follows:
echo "`echo 'standard out1'` `echo 'error out1' >/dev/stderr` `echo 'standard out2'`" | tee -a /tmp/test.txt ; cat /tmp/test.txt
if this works for you please bump up to 1. The other suggestions don't work for my csh environment.

Related

Applescript: Running terminal commands and Saving output

So, I'm trying to write a script to remove wireless networks and their associated keychain credentials.
tell application "Terminal"
activate
string mywifi
set mywifi to "test"
set mywifi to do script ("networksetup -listallhardwareports | grep -A 1 'Wi-Fi' | grep -v 'Hardware' | sed -e 's/'Device:\ '//g'")
do script "networksetup -removepreferredwirelessnetwork $mywifi NETWORK1"
do script "security delete-generic-password NETWORK1"
delay 2
#do script "networksetup -removepreferredwirelessnetwork $mywifi NETWORK2"
#do script "security delete-generic-password Network2"
delay2
#do script "networksetup -removepreferredwirelessnetwork $mywifi Network3"
#do script "security delete-generic-password Network3"
delay 2
#do script "networksetup -removepreferredwirelessnetwork $mywifi Network4"
#do script "security delete-generic-password Network4"
delay 2
#do script "networksetup -removepreferredwirelessnetwork $mywifi Network5"
#do script "security delete-generic-password Network5"
delay 2
end tell
quit
Where i'm running into trouble is setting that variable with the output of that command. The command runs in terminal, though whenever I attempt to compile it, the following error is thrown
Syntax Error: Expected """ but found unknown token
It finds this right after /device:\ ' between the \ '
I have not been able to figure out what is missing. If i add " between them it just drops the terminal to >
Straight up my first foray into applescript but not my first language. I think i've been staring at it too long.
This is not intended to be a complete answer, because I'm not too familiar with the security command, although I beleive you'll need to preface it with the sudo command to modify the keychain. There's lots of good info on the Internet covering security command. I read quite a bit but there's nothing in my keychain I want to remove so I'm not able to test in this respect.
The following covers the networksetup command, which you may also need to use sudo, in Terminal, but in AppleScript add with administrator privileges at the end of the do shell script command", if necessary when preferred wireless networks.
It looks like your trying to remove all preferred wireless networks and this can be done directly in Terminal with the following command:
networksetup -removeallpreferredwirelessnetworks $(networksetup -listallhardwareports | awk '/Wi-Fi/{getline; print $2}')
If you want to do the same thing in AppleScript, e.g:
do shell script "networksetup -removeallpreferredwirelessnetworks $(networksetup -listallhardwareports | awk '/Wi-Fi/{getline; print $2}')"
If you want to use it in a bash script, then use the line above with a bash shebang, e.g.:
#!/bin/bash
networksetup -removeallpreferredwirelessnetworks $(networksetup -listallhardwareports | awk '/Wi-Fi/{getline; print $2}')
Save it in a plain text file without an extension, and make it executable, e.g. chmod u+x filename where filename is whatever you saved it as. Then to use it in Terminal, ./filename or /path/to/filename, if it's not saved in a location defined within the PATH environment variable.
If your looking to do it just for specific networks, then instead of a line for each network, you can loop through a list. If you need help with that, let us know.

Run RapSearch-Program with Torque PBS and qsub

My problem is that I have a cluster-server with Torque PBS and want to use it to run a sequence-comparison with the program rapsearch.
The normal RapSearch command is:
./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32
Now I want to run it with 2 nodes on the cluster-server.
I've tried with: echo "./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32" | qsub -l nodes=2 but nothing happened.
Do you have any suggestions? Where I'm wrong? Help please.
Standard output (and error output) files are placed in your home directory by default; take a look. You are looking for a file named STDIN.e[numbers], it will contain the error message.
However, I see that you're using ./rapsearch but are not really being explicit about what directory you're in. Your problem is therefore probably a matter of changing directory into the directory that you submitted from. When your terminal is in the directory of the rapsearch executable, try echo "cd \$PBS_O_WORKDIR && ./rapsearch [arguments]" | qsub [arguments] to submit your job to the cluster.
Other tips:
You could add rapsearch to your path if you use it often. Then you can use it like a regular command anywhere. It's a matter of adding the line export PATH=/full/path/to/rapsearch/bin:$PATH to your .bashrc file.
Create a submission script for use with qsub. Here is a good example.

Shell redirection operation >|?

What is the difference between these two redirections?
[localhost ~]$ echo "something" > a_file.txt
[localhost ~]$ echo "something" >| a_file.txt
I can't seem to find any documentation about >| in the help.
>| overrides the noclobber option in the shell (set with $ set -o noclobber, indicates that files can not be written over).
Basically, with noclobber, you get an error if you try to overwrite an existing file using >:
$ ./program > existing_file.txt
bash: existing_file.txt: cannot overwrite existing file
$
Using >| will override that error and force the file to be written over:
$ ./program >| existing_file.txt
$
It's analogous to using the -f or --force option on many shell commands.
From the Bash Reference Manual Section "3.6.2 Redirecting Output":
If the redirection operator is >, and the noclobber option to the set builtin has been enabled, the redirection will fail if the file whose name results from the expansion of word exists and is a regular file. If the redirection operator is >|, or the redirection operator is > and the noclobber option is not enabled, the redirection is attempted even if the file named by word exists.
Searching for "bash noclobber" generally brings up articles that mention this somewhere. See this question on SuperUser, this section in O'Reilly's "Unix Power Tools", and this Wikipedia article on Clobbering for examples.

Redirect stderr through grep -v in LSF batch job

I'm using a library that generates a whole ton of output to stderr (and there is really no way to suppress the output directly in the code; it is ROOT's Minuit2 minimizer which is known for not having a way to suppress the output). I'm running batch jobs through the LSF system, and the error output files are so big that they exceed my disk quota. Erk.
When I run locally on a shell, I do:
python main.py 2> >( grep -v Minuit2 2>&1 )
to suppress the output, as is done here.
This works great, but unfortunately I can't seem to get that or any variation of it to work when running on LSF. I think this is due to LSF not spawning the necessary subshell, but it's not clear.
I run on batch by passing LSF a submit script. The relevant line is:
python main.py $INPUT_FILE
which works great, aside from the aforementioned problem of gigantic error files.
When I try changing that line to
python main.py $INPUT_FILE 2> >( grep -v Minuit2 2>&1 )
I end up with
./singleSubmit.sh: line 16: syntax error near unexpected token `>'
./singleSubmit.sh: line 16: `python $MAIN $1 2> >( grep -v Minuit2 2>&1 )'
in the error log file.
Any idea how I could accomplish what I want, or why this is not working?
Thanks a ton!
The syntax you're using works in bash, not in csh/tcsh. Try changing the first line of your submission script to
#!/bin/bash

How can I cat back exact formatting regardless of shell?

While trying to write a script, I found an interesting issue with cat today. If I do the following at the command line, everything works properly:
var=$(ssh user#server "cat /directory/myfile.sh")
echo $var > ~/newfile.sh
This works and I have a script file with all the proper formatting and can run it. However, if I do the EXACT same thing in a script:
#!/bin/sh
var=$(ssh user#server "cat /directory/myfile.sh")
echo $var > ~/newfile.sh
The file is mangled with carriage returns and weird formatting.
Does anyone know why this is happening? My goal is to ultimately cat a script from a server and run it locally on my machine.
EDIT
I now know that this is happening because of my invoking #!/bin/sh in my shell script. The command line works because I'm using zsh and it is preserving the formatting.
Is there a way to cat back the results regardless of the shell?
As you seem to have figured out, word splitting is off by default on zsh, but on in sh, bash, etc. You can prevent word splitting in all shells by quoting the variable:
echo "$var" > ~/newfile.sh
Note that echo appends a newline to its output by default, which you can suppress (on most echo implementations and builtins) with -n.