This is much easier than what I'm making it...
I have a non-delimited input file that I want to grab three columns from and output them to a new layout. The input file is fixed length. I want to take the data from the following positions:
field1 - position 1 for 7 characters
field2 - position 13 for 50 characters
field3 - position 187 for 10 characters
I want my output file to be field1,field2,field3...comma delimited.
This is what I thought would work, but alas, it doesn't:
#echo off
setlocal EnableDelayedExpansion
for /F "tokens=*" %%A in (C:\mydata.txt) do (
set line=%%A
set var_id=!line:~0,7%!
set var_name=!line:~12,50%!
set var_amt=!line:~186,10%!
set ABC=!%var_id%%var_name%%var_amt%!
echo ABC
)
Related
quick overview: I got sqlite3 db which contains following structure and data
Id|Name|Value
1|SomeName1|SomeValue1
2|SomeName2|SomeValue2
3|SomeName3|SomeValue3
(continuation of SomeValue3 in here, after ENTER)
Problem is with iteration trough "Value" column, I'm using that code:
records=(`sqlite3 database.db "SELECT Value FROM Values"`)
for record in "${records[#]}"; do
echo $record
done
Problem is there should three values using that iteration, but it is showing four.
As result I received:
1 step of loop - SomeValue1
2 step of loop - SomeValue2
3 step of loop - SomeValue3
4 step of loop - (continuation of SomeValue3 in here, after ENTER)
it should end at third step and just show with line break up something like that:
3 step of loop - SomeValue3
(continuation of SomeValue3 in here, after ENTER)
Any suggestion how I can handle it with bash?
Thank you in advance!
Instead of relying on word splitting to populate an array with the result of a command, it's much more robust to use the readarray builtin, or read a result at a time with a loop. Examples of both follow, using sqlite3's ascii output mode, where rows are separated by the byte 0x1E and columns in the rows by 0x1F. This allows the literal newlines in your data to be easily accepted.
#!/usr/bin/env bash
# The -d argument to readarray and read changes the end-of-line character
# from newline to, in this case, ASCII Record Separator
# Uses the `%q` format specifier to avoid printing the newline
# literally for demonstration purposes.
echo "Example 1"
readarray -d $'\x1E' -t rows < <(sqlite3 -batch -noheader -ascii database.db 'SELECT value FROM "Values"')
for row in "${rows[#]}"; do
printf "Value: %q\n" "$row"
done
echo "Example 2 - multiple columns"
while IFS=$'\x1F' read -d $'\x1E' -ra row; do
printf "Rowid: %d Value: %q\n" "${row[0]}" "${row[1]}"
done < <(sqlite3 -batch -noheader -ascii database.db 'SELECT rowid, value FROM "Values"')
outputs
Example 1
Value: SomeValue1
Value: SomeValue2
Value: $'SomeValue2\nand more'
Example 2 - multiple columns
Rowid: 1 Value: SomeValue1
Rowid: 2 Value: SomeValue2
Rowid: 3 Value: $'SomeValue2\nand more'
See Don't Read Lines With for for more on why your approach is bad.
Since VALUES is a SQL keyword, when using it as a table name (Don't do that!) it has to be escaped by double quotes.
Your problem here is the IFS (internal field seperator) in Bash, which the for -loop counts as a new record.
Your best option is to remove the linefeed in the select statement from sqlite, e.g:
records=(`sqlite3 database.db "SELECT replace(Value, '\n', '') FROM Values"`)
for record in "${records[#]}"; do
echo $record
done
Alternatively, you could change the IFS in Bash - but you are relying on linefeed as a seperator between records.
I need a script which rename all files in all sub folders of script's root.
I did search and found something which works and I can modify (I am a little bit newbie/laic)
#echo off
chcp 65001
setlocal enabledelayedexpansion
set filename=image
set /a x=1
>#rename.txt (
for /r %CD% %%f in (*.jpg) do (
echo rename "%%f" "!filename!_!x!.jpg"
rename "%%f" "!filename!_!x!.jpg"
set /a x+=1
)
)
endlocal
pause
But I want this to rename files to random string.
I found a lots of scripts which can generate random strings, but I just can't make them to work inside the FOR brackets.
for now im stuck with
#echo off
chcp 65001
setlocal enabledelayedexpansion
set /a x=%RANDOM%/99
>#rename.txt (
for /r %CD% %%f in (*.jpg) do (
echo "%%f" renamed to "!x!.jpg"
rename "%%f" "!x!.jpg"
set /a x+=%RANDOM%/99
)
)
endlocal
Which works fine, but has his limitations.
Any help will be appreciated
#echo off
chcp 65001
setlocal enableextensions enabledelayedexpansion
set "alphabet=a b c d f g h i j k l m n p q r s t u v w x y z 0 1 2 3 4 5 6 7 8 9"
set "size=0"
for %%a in (%alphabet%) do (
set "a.!size!=%%a"
set /a "size+=1"
)
for /r %CD% %%f in (*.jpg) do (
set "k="
for /l %%a in (1 1 64) do (
set /a "r=!random! %% size"
for %%b in (!r!) do set "k=!k!!a.%%b!"
)
rename "%%f" "!k!.jpg"
)
endlocal
I want to add a header t my input strings. The header should be > directly followed by the string and the number after the string separated with a _
To add a header I used this awk '{print ">"$0;print}' However I dont kno how to add the number behind.
input:
CTTCTATGATGAATTTGATTGCATTGATCGTCTGACATGATAATGTATTT 2
AGAACGAAAGTCGGAGGTTCGAAGACGATC 14
TACCCTGTAGAACCGAANTTGT 1
TCCCTGTGGTCTAGTGGTTAGGATTCTGCGCTCTCACCGCCGCGGCCCGGG 2
GGGCCAGGATGAAACCTAATTTGAGTGGCCATCCATGGATGAGAAATGCGG 4
TAATACGGCCGGGTAATGATGGA 0
CCAGATGATGAACTTATTGACGGGCGGACAGAAACTGTGTGCTGATTGTCA 7240
CGCCCGATCTCGTCTGATCTCG 34
GCAGGGGTGGTTCAGTGGTAGAATTCTCGCC 3
output:
>CTTCTATGATGAATTTGATTGCATTGATCGTCTGACATGATAATGTATTT_2
CTTCTATGATGAATTTGATTGCATTGATCGTCTGACATGATAATGTATTT
>AGAACGAAAGTCGGAGGTTCGAAGACGATC_14
AGAACGAAAGTCGGAGGTTCGAAGACGATC
....
$ awk '{printf ">%s_%s\n %s\n",$1,$2,$1;}' file
>CTTCTATGATGAATTTGATTGCATTGATCGTCTGACATGATAATGTATTT_2
CTTCTATGATGAATTTGATTGCATTGATCGTCTGACATGATAATGTATTT
>AGAACGAAAGTCGGAGGTTCGAAGACGATC_14
AGAACGAAAGTCGGAGGTTCGAAGACGATC
>TACCCTGTAGAACCGAANTTGT_1
TACCCTGTAGAACCGAANTTGT
>TCCCTGTGGTCTAGTGGTTAGGATTCTGCGCTCTCACCGCCGCGGCCCGGG_2
TCCCTGTGGTCTAGTGGTTAGGATTCTGCGCTCTCACCGCCGCGGCCCGGG
>GGGCCAGGATGAAACCTAATTTGAGTGGCCATCCATGGATGAGAAATGCGG_4
GGGCCAGGATGAAACCTAATTTGAGTGGCCATCCATGGATGAGAAATGCGG
>TAATACGGCCGGGTAATGATGGA_0
TAATACGGCCGGGTAATGATGGA
>CCAGATGATGAACTTATTGACGGGCGGACAGAAACTGTGTGCTGATTGTCA_7240
CCAGATGATGAACTTATTGACGGGCGGACAGAAACTGTGTGCTGATTGTCA
>CGCCCGATCTCGTCTGATCTCG_34
CGCCCGATCTCGTCTGATCTCG
>GCAGGGGTGGTTCAGTGGTAGAATTCTCGCC_3
GCAGGGGTGGTTCAGTGGTAGAATTCTCGCC
How it works
The awk script consists of a single command:
printf ">%s_%s\n %s\n",$1,$2,$1
By default, awk splits up input lines into fields based on white space. So, For the first line for example, field 1 is CTTCTATGATGAATTTGATTGCATTGATCGTCTGACATGATAATGTATTT and field 2 is 2. The printf allows us to rearrange the input into the desired format. For each input line, two lines are written. The first one, with format >%s_%s\n writes > followed by field 1 followed by _ followed by field 2 followed by a newline character. The format for the second output line is%s\n which outputs a space followed by field one followed by a newline character.
I have a series of .txt files that I am trying to get into a database.
The 7th row "contains" the delimitation in dashes and spaces. Tools like excel try and space delimit based on the entire file. but rows after row 7 have spaces within the fields.
From a post I found I have managed to create a cmd.bat script to remove the first 6 rows and footer. Now I need advise on how to "delimit" the file using row7. deleting row7 afterward would be a bonus. 4 of the fields fluctuate in length so fixed field length is not and option. Would appreciate any help.
Report Number: ST-49 ENERGY RESOURCES CONSERVATION BOARD Page 1 of 1
Run Date: 01 Jan 2012 DAILY DRILLING ACTIVITY LIST
For the Notification Period 31 Dec 2011 To 01 Jan 2012
WELL ID WELL NAME LICENCE +----------------CONTRACTOR-----------------+ RIG ACTIVITY DATE FIELD CENTRE BA ID LICENSEE NEW PROJECTED ACTIVITY
BA ID NAME NUMBER TOTAL DEPTH TYPE
------------------- ----------------------------------- -------- ------ -------------------------------------- ------ ----------------------- -------------- ----- ------------------------------------- ------------- -----------
00/04-11-028-28W4/0 ECA CROSSE 4-11-28-28 0438271 0Y52 Precision Drilling Corporation 239 31 Dec 2011 10:30:00 PM Midnapore 0026 EnCana Corporation Drill To LD
00/10-21-038-24W4/0 ECA HAYNES 10-21-38-24 0439238 0Y52 Precision Drilling Corporation 154 31 Dec 2011 11:00:00 PM Red Deer 0026 EnCana Corporation Drill To LD
00/04-08-049-18W5/0 HUSKY MINEHEAD 4-8-49-18 0440874 0Y52 Precision Drilling Corporation 401 01 Jan 2012 01:00:00 PM Drayton Valley 0R46 Husky Oil Operations Limited Drill To LD
TOTAL - Drill To LD - Drilling to Licensed Depth = 15
TOTAL - Re-entry - Re-entry of an Abandoned Well = 0
TOTAL - Resumption - Resumption of Drilling of a Non-abandoned Well = 0
TOTAL - Set Surface - Drilling to Set Surface Casing Only = 2
TOTAL NUMBER OF WELLS LISTED = 17
*** END OF REPORT ***
try this (it might be slow, there is not to change):
#ECHO OFF &SETLOCAL ENABLEDELAYEDEXPANSION
SET "infile=file"
SET "outfile=out"
FOR /f %%a in ('^<"%infile%" find /c /v ""') do SET /a lline=%%a-9
FOR /f "usebackqskip=6delims=" %%a IN ("%infile%") DO IF NOT DEFINED line SET "line=%%a"
FOR %%a IN (%line%) DO (
SET /a counter+=1
FOR /f "delims=:" %%b IN ('(echo(%%~a^&echo(^)^|findstr /o $') DO SET /a $!counter!=%%b-2
)
(FOR /f "skip=6delims=" %%a IN ('findstr /n $ "%infile%"') DO IF %lline% gtr !cline! (
SET "line=%%a"
FOR /f "delims=:" %%b IN ("!line!") DO SET /a cline=%%b
SET "line=!line:*:=!"
SET /a cstart=0
SET "Newline="
FOR /l %%b IN (1,1,%counter%) DO (
CALL SET "token=%%line:~!cstart!,!$%%b!%%"
SET "flag=true"
SET /a tstop=!$%%b!-1
FOR /l %%c IN (!tstop!,-1,0) DO IF DEFINED flag IF NOT "!token:~%%c,1!"==" " (
SET /a tstop=%%c+1
FOR /f %%d IN ("!tstop!") DO SET "token=!token:~0,%%d!"
FOR /f "tokens=*" %%d IN ("!token!") DO SET "token=%%d"
SET "flag="
) ELSE (
IF %%c equ 0 SET "token="
)
IF %%b equ 1 (SET "Newline="!token!"") ELSE SET "Newline=!Newline!,"!token!""
SET /a cstart+=!$%%b!
)
ECHO(!Newline!
))>"%outfile%"
TYPE "%outfile%"
..output:
"00/04-11-028-28W4/0","ECA CROSSE 4-11-28-28","0438271","0Y52","Precision Drilling Corporation","239","31 Dec 2011 10:30:00 PM","Midnapore","0026","EnCana Corporation","","Drill To LD"
"00/10-21-038-24W4/0","ECA HAYNES 10-21-38-24","0439238","0Y52","Precision Drilling Corporation","154","31 Dec 2011 11:00:00 PM","Red Deer","0026","EnCana Corporation","","Drill To LD"
"00/04-08-049-18W5/0","HUSKY MINEHEAD 4-8-49-18","0440874","0Y52","Precision Drilling Corporation","401","01 Jan 2012 01:00:00 PM","Drayton Valley","0R46","Husky Oil Operations Limited","","Drill To LD"
This doesn't work with special characters like ^"!%*=~ in the infile.
You should better use awk for Windows and set FIELDWIDTHS in the BEGIN pattern. You might combine it with sed for Windows.
Try using qualifies, i.e. " or anything that can make your application or tool to identify that column start from here. and it will ignore the delimiter with in column.
sorry for the question I am a total beginner in programming and probably this is very easy for someone who knows what to do:
I need to create, into a txt file, a serie of links with incremental numbers, for example I want to keep a certain part "fixed" and a part has to change; for example imagine this:
http://user:password#website.com/members/sets/001/somethingfixed-001.zip
in the text the following lines should be:
http://user:password#website.com/members/sets/002/somethingfixed-002.zip
etc..
I think there must be a way to generate those where I could specify, create the link keep the fixed part equal in every line and then incrementally put 001, 002, 003, up to the point i need it.
it could be awesome if I could create a .bat file that would ask me: what is the initial link, what is the part to be fixed and what to change.
I'd do the program like this:
insert the first link, and I paste it.
then it asks me for what is the fixed part and I put:
.../members/sets/X/somethingfixed-Y.zip
what is the range for the variables X and Y?
starting number? and I put 001 and then it asks me what is the end number? and i put 150
and it generates the links in the txt file so that I can use them.
Hope it is clear.
Cheers
Example
This will do it for the first number X, I will leave some of it for you to develop and learn. :)
#echo off
setlocal EnableExtensions EnableDelayedExpansion
echo LeftXRight Gen
set /p "Left=Left = "
set /p "Right=Right = "
set /p "Start=Begin = "
set /p "End=End = "
>output.txt ( <nul set /p "=" )
for /L %%A in (%Start%, 1, %End%) do (
set "X=00%%A"
echo !Left!!X:~-3!!Right!>>output.txt
)
endlocal
Output
LeftXRight Gen
Left = http://user:password#website.com/members/sets/
Right = /somethingfixed.zip
Start = 1
End = 10
output.txt
http://user:password#website.com/members/sets/001/somethingfixed.zip
http://user:password#website.com/members/sets/002/somethingfixed.zip
http://user:password#website.com/members/sets/003/somethingfixed.zip
http://user:password#website.com/members/sets/004/somethingfixed.zip
http://user:password#website.com/members/sets/005/somethingfixed.zip
http://user:password#website.com/members/sets/006/somethingfixed.zip
http://user:password#website.com/members/sets/007/somethingfixed.zip
http://user:password#website.com/members/sets/008/somethingfixed.zip
http://user:password#website.com/members/sets/009/somethingfixed.zip
http://user:password#website.com/members/sets/010/somethingfixed.zip
Update
Added this complete solution to the question.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
echo LeftXMidXRight Gen
set /p "Left=Left = "
set /p "Middle=Middle = "
set /p "Right=Right = "
set /p "Start=Begin = "
set /p "End=End = "
>output.txt ( <nul set /p "=" )
for /L %%A in (%Start%, 1, %End%) do (
set "X=00%%A"
echo !Left!!X:~-3!!Middle!!X:~-3!!Right!>>output.txt
)
endlocal
Example
LeftXMidXRight Gen
Left = http://user:password#website.com/members/sets/
Middle = /somethingfixed-
Right = .zip
Begin = 1
End = 150
output.txt
http://user:password#website.com/members/sets/001/somethingfixed-001.zip
http://user:password#website.com/members/sets/002/somethingfixed-002.zip
...
http://user:password#website.com/members/sets/150/somethingfixed-150.zip
This gets near what you are after, with a couple of limitations..
It can only append to the end of the url.
1 and 2 digit numbers are not zero prefixed.
#echo off
set /p url="Enter URL: "
set /p start="Start Number: "
set /p end="End Number: "
for /L %%X in (%start%, 1, %end% ) do echo %url% %%X >> file.txt