Hard-to-read-line
#daily export sunshine="~/logs/Sunshine-`date '+\%F'`" && export sunshineUrl="http://www.sunshine.net/main/search_results.asp?currency_id=1&min_price=&max_price=50000&country_id=241®ion_id=&Submit=Search" && mkdir -p $sunshine && cd $sunshine && wget --mirror -l 1 $sunshineUrl
Which mark do I need to have it on many lines?
#daily <SOME MARK HERE>
export sunshine="~/logs/Sunshine-`date '+\%F'`" && <SOME MARK HERE>
export sunshineUrl="http://www.sunshine.net/main/search_results.asp?currency_id=1&min_price=&max_price=50000&country_id=241®ion_id=&Submit=Search" && <SOME MARK HERE>
mkdir -p $sunshine && <SOME MARK HERE>
cd $sunshine && wget --mirror -l 1 $sunshineUrl
No success by appending \, //, \n or /n.
Might it be easier to take all the above and create a shell script from it ? It'll be more maintainable in the long run.
I believe that you can do it with a semicolon ; but I agree with Brian that you need to take the above and put it in a script and run from cron.
Related
The question is best explained through a small example:
Create an infrastructure of directories as follows:
rm -rf /tmp/work
mkdir -p /tmp/work/a/b/c
cd /tmp/work
ln -s a/b/c s
mkdir t
tree
This results in the following infrastructure:
There is a very simple way to check for the existence of a directory:
cd /tmp/work
if test -d s/../t; then echo EXISTS; else echo DOES NOT EXIST; fi
However, it seems that the '-d' test only checks for the physical path (a/b/t); The following clearly shows that the logical path/directory does exist:
cd s/../t
pwd
I am hoping for a simple test to get this situation checked...
Checking for similar questions in StackOverflow, I haven't found an answer that differentiates logical paths from physical paths...
I did a brute force (silly) attempt to get some more insight:
for o in -b -c -d -e -f -g -G -k -h -L -O -p -r -S -s -t -u -w -x; do
if test $o s/../t; then echo EXISTS; else echo DOES NOT EXIST; fi
done
As expected, all of the tests indicate failure...
The closest I get to a simple solution is the following test:
if (unset CDPATH; cd -L s/../t &>/dev/null); then echo EXISTS; else echo DOES NOT EXIST; fi
If using '-P' rather than '-L', then this syntax tests for physical paths (the equivalent of 'test -d').
Note that if we're sure that we're using builtins, then the syntax is just a bit simpler:
if (unset CDPATH; cd s/../t &>/dev/null); then echo EXISTS; else echo DOES NOT EXIST; fi
But there must be a better way...
I would like to modify one of my recent Bash aliases to forward errors. Here is the alias:
alias makecclip=
"make |& tee >(sed \"s,\x1B\[[0-9;]*[a-zA-Z],,g\" |
egrep \":[0-9]+:[0-9]+: error\" | cut -d : -f1,2,3 |
head -n 1 | xargs -0 echo -n | xclip -selection clipboard &&
xclip -selection clipboard -o)
This code displays the results of a C++ compilation, and then removes formatting and displays and adds to the clipboard the first error location (if there is any).
However, I would like to use this code like this:
makecclip && bin/someexecutablecreated
This though ruins the && operator, since it always runs bin/someexecutablecreated even when there is a compilation error present. How can I add modifications to the code to set the error flag, when the error list (the things saved to clipboard and echoed) is not empty?
You can address your issue by using the PIPESTATUS internal variable (this variable has other names in non-bash shells). This allows to have an history of exit statuses of commands passed by pipe.
You precised in the comments that you didn't use bash, but used zsh instead. As such, some of the syntax of my solution has to be changed, as they handle the PIPESTATUS variable differently.
In bash, you use ${PIPESTATUS[0]}, whereas you'll use ${pipestatus[1]} in zsh.
A first approach, using your existing alias, could be as follow :
makecclip && [ "${pipestatus[1]}" -eq "0" ] && echo "ok"
This runs the echo command only if "${pipestatus[1]}" is equal to 0 (no errors during make)
A more convenient solution would be to use a function instead of an alias for makecclip. In your ~/.bashrc file, you could write :
makecclip () {
make |& tee >(sed "s,\x1B\[[0-9;]*[a-zA-Z],,g" | egrep ":[0-9]+:[0-9]+: error" | cut -d : -f1,2,3 | head -n 1 | xargs -0 echo -n | xclip -selection clipboard && xclip -selection clipboard -o)
return "${pipestatus[1]}"
}
Now, makecclip && echo "ok" will work as expected.
Test cases :
#!/bin/zsh
#do not run this test if there is an existing makefile in your current directory
rm -f makefile
makecclip () {
make |& tee >(sed "s,\x1B\[[0-9;]*[a-zA-Z],,g" | egrep ":[0-9]+:[0-9]+: error" | cut -d : -f1,2,3 | head -n 1 | xargs -0 echo -n | xclip -selection clipboard && xclip -selection clipboard -o)
# this part is only present to check the pipestatus values during the tests.
# In the real function, I wrote 'return ${pipestatus[1]}' instead.
a=(${pipestatus[#]})
echo ${a[#]}
return ${a[1]}
}
echo "# no makefile"
makecclip && echo "ok"
echo -e "\n# empty makefile"
touch makefile
makecclip && echo "ok"
echo -e "\n# dummy makefile entry"
echo -e 'a:\n\t#echo "inside makefile"' > makefile
makecclip && echo "ok"
echo -e "\n# program with error makefile"
echo -e "int main(){error; return 0;}" > target.cc
echo -e 'a:\n\tgcc target.cc' > makefile
makecclip && echo "ok"
Output :
$ ./test.sh
# no makefile
make: *** No targets specified and no makefile found. Stop.
2 0
# empty makefile
make: *** No targets. Stop.
2 0
# dummy makefile entry
inside makefile
0 0
ok
# program with error
gcc target.cc
target.cc: In function ‘int main()’:
target.cc:1:12: error: ‘error’ was not declared in this scope
int main(){error; return 0;}
^
makefile:2: recipe for target 'a' failed
make: *** [a] Error 1
target.cc:1:12
2 0
Any recursive chown or chmod command on an s3fs mount takes a long time when you have a few directories (about 70) each with quite a few files.
Either of these commands are likely to take almost 24 hours. I have to do this or the Apache process cannot access these files/directories. The command on a normal mount takes about 20 seconds.
Mounting with:
/storage -o noatime -o allow_other -o use_cache=/s3fscache -o default_acl=public-read-write
In /etc/fuse.conf:
user_allow_other
Using latest version: 1.78
Any thoughts on how to do this faster?
After a while, I found it just better to parallel the processes in order to speed it up. Example:
find /s3fsmount/path/to/somewhere -print | xargs --max-args=1 --max-procs=100 chmod 777
It is still slow, but nowhere near as slow as it was.
Using aws cli may help.
what I do:
use aws cli to get the full file list of the target directory.
write a script to parallel execute chmod 777 to each file (with > /dev/null 2>&1 &)
then I found that the chmod jobs finished immediately, from ps -ef.
my PHP code:
<?php
$s3_dir = 'path/to/target/';
$s3fs_dir = '/mnt/s3-drive/' .$s3_dir;
echo 'Fetching file list...' . "\n\n";
sleep(1.5);
$cmd = 'aws s3 ls --recursive s3://<bucket_name>/' . $s3_dir;
exec($cmd, $output, $return);
$num = 0;
if ( is_array($output) ) {
foreach($output as $file_str) {
if ( $num>100 ) {
sleep(4);
$num=0;
}
$n = sscanf( $file_str, "%s\t%s\t%s\t". $s3_dir ."%s", $none1, $none2, $none3, $file );
$cmd = 'chmod 777 ' . $s3fs_dir . $file . ' > /dev/null 2>&1 &';
echo $cmd ."\n";
exec( $cmd );
$num+=1;
}
}
?>
For Change user
find /s3fsmount/path/to/somewher -print | xargs --max-args=1 --max-procs=100 sudo chown -R user:user
It work me..
I need to see which files have been added or removed between two streams. The most obvious way would be "git lsfiles" in each stream. Except this is not GIT and I do not see an analogous command. So for today:
for f in $(find * -type f);do
accurev stat "$f"
done | \
fgrep -v '(external)' | \
awk '{print $1}' > .list
If there is a better way, it should be clear and easy to find here:
http://www.accurev.com/download/docs/5.7.0_books/AccuRev_5_7_User_CLI.pdf
but it is not. Help? Thank you.
If you want to see the difference between two streams, run the following command: accurev diff -a -v "Stream1" -V "Stream2"
As the command line question has been answered, here's how to do the same via the AccuRev GUI.
Select one dynamic stream, workspace or snapshot.
Right click and select "Show Diff By Files"
Select a different dynamic stream, workspace or snapshot.
You'll be presented with a list of files different between the two choices, and yes you can mix-and-match between dynamic streams, workspaces and snapshots.
You can then select any file and select "Show Difference" to see differences between the two files.
Since neither of the two answers addressed the question, I eventually worked out a script to do what is really needed. "accurev lsfiles" is sorely needed.
#! /bin/bash
declare -r progpid=$$
declare -r progdir=$(cd $(dirname $0) >/dev/null && pwd)
declare -r prog=$(basename $0)
declare -r program="$progdir/$prog"
declare -r usage_text=' [ <directory> ... ]
If no directory is specified, "." is assumed'
die() {
echo "$prog error: $*"
exec 1>/dev/null 2>&1
kill -9 $progpid
exit 1
} 1>&2
usage() {
test $# -gt 0 && {
exec 1>&2
echo "$prog usage error: $*"
}
printf "USAGE: $prog %s\n" "$usage_text"
exit $#
}
init() {
shift_ct=$#
tmpd=$(mktemp -d ${TMPDIR:-/tmp}/ls-XXXXXX)
test -d "$tmpd" || die "mktemp -d does not work"
exec 4> ${tmpd}/files
trap "rm -rf '$tmpd'" EXIT
prune_pat=
while :
do
test $# -eq 0 && break
test -f "$1" && break
[[ "$1" =~ -.* ]] || break
case "X$1" in
X-p )
prune_pat+="${2}|"
shift 2 || usage "missing arg for '-p' option"
;;
X-p* )
prune_pat+="${1#-p}"
shift
;;
X-x* )
set -x
tput reset 1>&2
PS4='>${FUNCNAME:-lsf}> '
shift
;;
X-* )
usage "unknown option: $1"
;;
* )
break
;;
esac
done
(( shift_ct -= $# ))
test ${#prune_pat} -gt 0 && \
prune_pat='(^|/)('${prune_pat%|}')$'
}
chkdir() {
declare list=$(exec 2> /dev/null
for f in "$#"
do ls -d ${f}/* ${f}/.*
done | \
grep -v -E '.*/\.\.*$' )
for f in $(accurev stat ${list} | \
grep -v -F '(external)' | \
awk '{print $1}' | \
sed 's#^/*\./##')
do
test ${#prune_pat} -gt 0 && [[ $f =~ ${prune_pat} ]] && continue
if test -d "$f"
then chkdir "$f"
elif test -f "$f" -o -L "$f"
then echo "$f" 1>&4
fi
done
}
init ${1+"$#"}
(( shift_ct > 0 )) && shift ${shift_ct}
test $# -eq 0 && set -- .
chkdir "$#"
exec 4>&-
sort -u ${tmpd}/files
It is a bit over-the-top, but I have a boilerplate I always use for my scripts.
I am trying to write a C shell equivalent script for the bash script mentioned here.
This is what I have :
#! /bin/tcsh
set now=`date +%Y%m%d%H%M.%S`
if (( ! -f "./cache" ) || (-n "`find ./monme -newer ./cache`" ))
then
touch cache -t "$now"
echo "new files added" | mail -s "new build" myemail#myserver.com
endif
and this is the error I get
$ ./scr
if: Badly formed number.
$
This page mentions that "Numbers in the C-shell must be integers", so I tried
set now=`date +%Y%m%d%H%M`
but I get the same error still.
I cut down your script to this:
#! /bin/tcsh
if ( -n "`find ./monme -newer ./cache`" ) then
echo hello
endif
This gives the same error. I think the culprit is
-n "`find ./monme -newer ./cache`"
What is -n supposed to do? I think it wants a number, but gets something else...
Update: -n in bash means "length of string is non-zero". In my version of tcsh it is as easy to replace as to use == "" like this:
if (( ! -f "./cache" ) || ("`find ./monme -newer ./cache`" != ""))
then
touch cache -t "$now"
echo "new files added" | mail -s "new build" myemail#myserver.com
endif
Try that and see if it works.