I have thousands of files inside a directory I need to rename adding a prefix like "th_" so that files will be th_65461516846.jpg
but I can't due to the error "argument too long"
I have used this command
rename 's/^/th_/' *
thanks!
The xargs program is used to break command lines into multiple commands to avoid blowing the shell line length limit. In your case, you'd use:
ls | xargs rename 's/^/th_/'
Which repeatedly executes rename with a portion of the output of ls until the list of files is exhausted. Do be aware this idiom requires special attention if the file names have spaces or other funny characters in them (which I'm assuming isn't so based on your example).
This one did the job
for f in *; do mv "$f" "${f/9/th_}";done
or
for f in * ; do mv $f th_${f#} ; done
I don't know what differs between the 2 but in my case they both work.
Related
Long story short, OneDrive has taken all my files and renamed them to include the string "-DESKTOP-9EI0FN7" at the end of the file name, resulting in files such as:
myTextFile-DESKTOP-9EI0FN7.txt
myVideo-DESKTOP-9EI0FN7.mp4
So I'd like to write a batch script that finds all the files with that string in them, and renames them to remove the string, so:
myTextFile-DESKTOP-9EI0FN7.txt becomes myTextFile.txt
The problem is, I know nothing about writing batch files. Any advice?
Test out with this bad boy:
find . -type f -exec rename -n -e 's/(.*)\-DESKTOP\-9EI0FN7(.*)/$1$2/' {} \;
If the output satisfies you, remove the -n portion and it does actually apply the changes.
Good luck, sir!
After some slicing and dicing, I end up with a relatively small data set which I want to handle off-line. I end up writing this:
store foo into 'foo' using PigStorage('\t');
copyToLocal foo foo;
rm foo;
sh cat foo/part* | sort -k... -o foo.tsv;
sh rm -rf foo;
I would like to replace these 5 lines with a macro call, but it does not
look like I can - I get Unexpected character '|' when I do.
So, can I avoid repeating these 5 lines a few times in every script?
You have to enclose the shell command with quotes.
I don't remember the syntax exactly. Something like:
sh bash "your commands"
If this doesn't work for you, I think you can put your commands in a separate shell executable and invoke it from Pig.
I cannot:
The shell commands (used with Grunt) are not supported.
not even copyToLocal appears to be allowed.
I have a number of files created by a program on our selling system that are produced in a format like the following:
CRY_SKI_14_EDI.LIS
CRY_SUM_14_EDI.LIS
THO_SKI_14_EDI.LIS
THO_LAK_14_EDI.LIS
CRY_SKI_IE_14_EDI.LIS
These files differ in numbers depending on the split of our product to different brandings. Is it possible to rename them all so that they read like the following:
CRY_SKI_14_EDI_DEMO.LIS
CRY_SUM_14_EDI_DEMO.LIS
THO_SKI_14_EDI_DEMO.LIS
THO_LAK_14_EDI_DEMO.LIS
CRY_SKI_IE_14_EDI_DEMO.LIS
I need the files to be correctly named prior to their FTP as a hardcoded file could potentially not exist due to the brand not being on sale and terminate the FTP which would prevent the other files following it from being transmitted to our FTP server.
The OpenVMS rename command is more handy (imho) than the windows or unix variants, because it can bulk change chuncks of the full file name. Such as 'name', 'type' or (sub)directory.
For example:
$ rename *.dat *.old
That's great but it will not change within the chunks (components) like the name part requested here.
For that, the classic DCL approach is a quick loop, either parsing directory output (Yuck!) or using F$SEARCH. For example:
$loop:
$ file = f$search("*EDI.LIS")
$ if file .eqs. "" then exit
$ name = f$parse(file,,,"name","syntax_only") ! grab name component from full name
$ rename/log 'file' 'name'_demo ! rename 'fills in the blanks'
$ goto loop
Personally I use PERL one-liners for this kind of work.
(and I test with -le using 'print' instead of 'rename' first. :-)
$ perl -e "for (<*edi.lis>) { $old = $_; s/_edi/_edi_demo/; rename $old,$_}"
Enjoy!
Hein
Is it possible to use a Linux environment variable inside a .sql file? I'm using the copy/select query to write to an output file, and I'll like to put that directory in a variable. So I want to do something like:
COPY (SELECT * FROM a)
TO $outputdir/a.csv
Outputdir would be set in my environment. Is this possible?
You can store the result of a shell command inside a psql variable like this:
\set afile `echo "$outputdir/a.csv"`
COPY (SELECT * FROM a) TO :'afile';
Another (better in my opinion) solution is to use only psql variables, see this answer of mine about psql variables, which is similar to your example. A example for your case would be:
\set outputdir '/path/to/output'
\set afile :outputdir '/a.csv'
COPY (SELECT * FROM a) TO :'afile';
Note that, in the example, you need to set the variable inside the script file, but you can skip the first line if you set it when you call psql:
psql --set=outputdir="$outputdir" <conn parameters> -f /path/to/yourscript.sql
This appears to work for your use case, provided you single quote the output file name as I mentioned. It will escape any double quotes as well contained within the SQL.
psql -c "$(eval echo '"' $(<envvars.sql | sed 's/"/\\"/g') '"')"
Of course, note that if your file contains any dollar quoted variables, the shell is going to try to interpret as a variable, and your script will break, so you will need to escape any dollar signs you need preserved literally with a backslash.
See also the second snippet in the accepted answer to this question for a possibly more robust answer.
The accepted answer is correct for PostgreSQL running on Unix. Under Windows a different incantation is required for obtaining the value of the environment variable from the CMD shell and for avoiding the carriage return returned by the echo command.
\set afile `set /p=%outputdir%/a.csv`
COPY (SELECT * FROM a) TO :'afile';
I am trying to write a bourne-shell script that takes a directory as a parameter and look for images named ixxx.a and rename them to ixxx_a.img where "xxx means the extension number for exemple image files would be named i001.a , i002.a ,i003.a ...)
here what I tried
mv $1/f[0-9][0-9][0-9].a $1/f[0-9][0-9][0-9]_a.img
but it says that the DEST is not a directory.
Any help would be much appreciated. Thanks.
for i in $1/f[0-9][0-9][0-9].a; do
mv $i ${i%.a}_a.img
done
However, this does not consider blank spaces in the file/folder names. In this case you'd have to use while so you get one file name per line (see below for bonus). There are probably dozens of other ways, including rename.
find $1 -maxdepth 1 -type f -name "f[0-9][0-9][0-9].a"|while read i; do
mv "$i" "${i%.a}_a.img"
done
Edit: Perhaps I should explain what I did there. It's called string substitution and the main use cases are these for a variable var:
# Get first two characters
${var:0:2}
# Remove shortest rear-anchored pattern - this one would give the directory name of a file, for example
${var%/*}
# Remove longest rear-anchored pattern
${var%%/*}
# Remove shortest front-anchored pattern - this in particular removes a leading slash
${var#/}
# Remove longest front-anchored pattern - this would remove all but the base name of a file given its path
# Replace a by b
${var//a/b}
${var##*/}
For more see the man page.