I have a batch script that is writing all files inclusive of path and version number to a TMP file for a physical device. This batch script then writes any lines that demonstrate a file version number greater than a provided variable.
DIRECTORYX := DIRECTORY /NODATE /NOHEADING /NOTRAILING
DIR_OUTPUT_FILENAME = "TOPAS$UTILLOG:" + F$UNIQUE() + ".TMP"
DEVICE = $1$DGA1112
DIRECTORYX 'DEVICE':[000000...]*.*;* /NOSIZE /OUTPUT='DIR_OUTPUT_FILENAME'
At the same time I have files to which I am not interested in being reported back to me. In this case i want to remove any lines that potentially contain this filename (filename.ext;) fromt he TMP file so that the batch script can continue through it and only report files that i dont explicity want to ignore.
How would I go about reading the file while inside a loop using 'IGNORE_FILE' as the variable for the text string to match and remove the associated line of text so that when my batch script proceeds through the file it will not report files requested to be ignored.
Many Thanks for any help
Allright... Now I see where you coming from.
How would i write everything from the 7th , to end of line?
Well, you could just loop starting with i=7, or you could use a "quote" the string and use the quote as new separator. Here is an example with both, using a double-quote as almost natural second separator choice
$
$ READ SYS$INPUT BUFF
SHG101,$1$DGA1101:,25,15,10,5000,100,X.TMP,Y.DAT
$
$ excludes = ""
$ tmp = F$ELEMENT(7, ",", BUFF)
$ IF tmp.NES.","
$ THEN
$ i = 8
$ excludes = tmp
$exclude_loop:
$ tmp = F$ELEMENT(i, ",", BUFF)
$ IF tmp.NES.","
$ THEN
$ excludes = excludes + "," + tmp
$ i = i + 1
$ GOTO exclude_loop
$ ENDIF
$ excludes = "/EXCLUDE=(" + excludes + ")"
$ ENDIF
$
$ SHOW SYMB excludes
$
$! Using a different delimitor:
$ READ SYS$INPUT BUFF
SHG101,$1$DGA1101:,25,15,10,5000,100,"X.TMP,Y.DAT"
$
$ excludes = ""
$ tmp = F$ELEMENT(1, """", BUFF)
$ IF tmp.NES."," THEN excludes = "/EXCLUDE=(" + tmp + ")"
$
$ SHOW SYMB excludes
In the code we see:
Checking ''DEVICE' for high file versions (>= ;''HVERNO') - may take some time...>
I urge you to check out DFU
The big loop code "READ FH3 /END_OF_FILE=LABEL$_EOF_DIRLIST1 BUFF2..."
will simplify to:
dfu searc/versio=min=200'excludes'form="!AS"/out=tmp.tmp dka0:
This will run in seconds almost no matter how many files.
Toss the whole warning stuff, include it always checking for 32000 as it is (almost) free.
After executing the DFU command, create the 'fancy' output if tmp.tmp is not empty by creating your header, and appending tmp.tmp. Always delete tmp.tmp ($ CLOSE/DISP=DELETE )
Free advice...
Those 15 deep nested IF-THEN-ELSE-ENDIF to pick up a message looks terrible (to maintain)
Consider an array lookup ?!
Here is a worked out example:
$! prep for test
$ EL = p1
$ EL_DIAG = "FILE.DAT"
$ LOCAL_STATUS = "12345"
$
$! Code snippet to be tested
$
$ x = 'EL'
$ ! Fold multiple conditions into 1 message
$
$ if (EL .EQ. 7) .OR. (EL .EQ. 14) .OR. (EL .EQ. 21) .OR. (EL .EQ. 25) -
.OR. (EL .EQ. 29) .OR. (EL .EQ. 30) THEN x = "L1"
$
$ MSG_6 = "error opening " + EL_DIAG + " for WRITE (RM=" + LOCAL_STATUS + ")"
$ IDT_6 = "OPENIN"
$ MSG_L1 = "error reading from " + EL_DIAG + " (RM=" + LOCAL_STATUS + ")"
$ IDT_L1 = "READERR"
$ MSG_8 = "device name missing " + EL_DIAG
$ IDT_8 = "DNNF"
$
$ ! Pick up the required texts
$
$ IF F$TYPE(MSG_'x').EQS.""
$ THEN
$ WRITE SYS$OUTPUT "No message found for code: ", EL
$ EXIT 16
$ ENDIF
$
$ MSG = MSG_'x
$ IDTXT = IDT_'x
$
$ WRITE SYS$OUTPUT "MSG : ", MSG
$ WRITE SYS$OUTPUT "IDTXT: ", IDTXT
Cheers,
Hein
Both comments are a great start. Check them carefully.
Which OpenVMS version? Something from the last 2 decades?
Just grab and use DFU !
$ define/user sys$output nl:
$ mcr dfu searc/versio=min=200/excl=(*.dat,*.tmp)/form="!AS"/out=tmp.tmp dka0:
$ type tmp.tmp
BUNDY$DKA0:[SYS0.SYSMGR]OPERATOR.LOG;242
BUNDY$DKA0:[SYS0.SYSMGR]ACME$SERVER.LOG;241
BUNDY$DKA0:[SYS0.SYSMGR]LAN$ACP.LOG;241
You could also consider sticking F$SEARCH in a loop and parse out version and other interesting components to implement your excludes....
$
$ type SEARCH_HIGH_VERSION.COM
$ max = 200
$ old = ""
$ IF p1.EQS."" THEN EXIT 16 ! PARAM
$loop:
$ file = F$SEARC(p1)
$ IF file.EQS."" THEN EXIT 99018 ! NMF
$ IF file.EQS.old THEN EXIT 100164 ! Not wild
$ old = file
$ version = F$PARSE(file,,,"VERSION") - ";"
$ IF max.GE.'version' THEN GOTO loop
$ ! IF ... further string tests
$ WRITE SYS$OUTPUT file
$ GOTO LOOP
$
$ #SEARCH_HIGH_VERSION.COM *.*;*
SYS$SYSROOT:[SYSMGR]ACME$SERVER.LOG;241
SYS$SYSROOT:[SYSMGR]LAN$ACP.LOG;241
SYS$SYSROOT:[SYSMGR]OPERATOR.LOG;242
SYS$SYSROOT:[SYSMGR]TMP.TMP;304
SYS$SYSROOT:[SYSMGR]TMP.TMP;303
SYS$SYSROOT:[SYSMGR]TMP.TMP;302
SYS$SYSROOT:[SYSMGR]TMP.TMP;301
SYS$SYSROOT:[SYSMGR]TMP.TMP;300
%RMS-E-NMF, no more files found
$
$ #SEARCH_HIGH_VERSION.COM tmp.tmp
SYS$SYSROOT:[SYSMGR]TMP.TMP;304
%RMS-F-WLD, invalid wildcard operation
$
DFU will probably be 10* faster than DIR
DIR will be 10* faster that F$SEARCH, but you'll loose that in the processing.
Good luck!
Hein
Related
I have a script that extracts a file from a bash script combined with a binary file. It does so using the following GNU sed syntax
sed -n '/__DATA__/{n;:1;n;p;b1}' /tmp/combined.file > /tmp/binary.file
The files are assembled by cat'ing an ISO file to the end of a bash script. Which is then sent over the network to an embedded device and extracted on the device, piping the ISO file to a temporary dir and executing the bash script to install it.
However, on executing this I get a
sed: unterminated {
Am I missing something here? Is this task possible with BusyBox sed?
It tried the "Second attempt" below with OSX/BSD awk and it failed, just printing up til the first NUL character. So you can't do this job portably with awk or sed.
Here's what should work everywhere given that the POSIX standard says
the input file to tail can be any type
so the input to tail doesn't have to be a POSIX text file (no NULs) and we're exiting from awk before the first NUL is encountered in the input so they should both be happy:
$ tail -n +"$(awk '/^__DATA__$/{print NR+2; exit}' binary.bin)" binary.bin | cat -ev
ER^H^#^#^#M-^PM-^P^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#3��M-^Nռ^#|��f1�f1�fSfQ^FWM-^N�M-^N�R�^#|�^#^F�^#^A��K^F^#^#R�A��U1�0���^Sr^VM-^A�U�u^PM-^C�^At^Kf�^F�^F�B�^U�^B1�ZQ�^H�^S[^O��#PM-^C�?Q��SRP�^#|�^D^#f��^G�D^#^OM-^BM-^#^#f#M-^#�^B��fM-^A>#|��xpu ��{�D|^#^#�M-^C^#isolinux.bin missing or corrupt.^M$
f`f1�f^C^F�{f^S^V�{fRfP^FSj^Aj^PM-^I�f�6�{��^FM-^H�M-^H�M-^R�6�{M-^H�^H�A�^A^BM-^J^V�{�^SM-^Md^Pfa��^^^#Operating system load error.^M$
^��^NM-^J>b^D�^G�^P<$
u��^X���^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#L^D^#^#^#^#^#^#�K�6^#^#M-^#^#^A^#^#?�M-^K^#^#^#^#^#`^\^#^#�������<R^#^#^#^_^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#U�EFI PART^#^#^A^#\^#^#^#]3�.^#^#^#^#^A^#^#^#^#^#^#^#�_^\^#^#^#^#^##^#^#^#^#^#^#^#�_^\^#^#^#^#^#Uc�r^Oqc#M-^Rc^F�$LZ�^L^#^#^#^#^#^#^#�^#^#^#M-^#^#^#^#�t^]F^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#$
Second attempt:
Now that I have a better idea what you're trying to do (process a file consisting of POSIX text lines up to a point and then can contain NUL characters afterwards), try this:
$ cat -ev file
echo "I: Installation finished!"$
exit 0$
$
__DATA__$
$
foo^#bar^#etc
$ cat tst.awk
/^__DATA__$/ { n=NR + 1 }
n && (NR == n) { RS="\0"; ORS="" }
n && (NR > n) { print (c++ ? RS : "") $0 }
$ awk -f tst.awk file | cat -ev
foo^#bar^#etc
The above doesn't try to store any input lines containing NUL in memory, instead it reads \n-terminated text lines until it reaches the line after the one containing __DATA__ and then switches to reading NUL-terminated records into memory and printing NULs between them on output.
It's still undefined behavior per POSIX (see my comments below) but in theory it should work since it just relies on being able to set one variable (RS) to NUL rather than trying to store input strings that contain NULs. Also, setting RS to NUL has been a (flawed) workaround for awk scripts for years to be able to read a whole file into memory at once so being able to set RS to NUL should work in any modern awk.
Using the new sample you provided with the missing blank line after the __DATA__ line added:
$ cat -ev file
#!/bin/bash$
$
echo "I: Awesome Things happened here"$
exit 0$
$
__DATA__$
$
ER^H^#^#^#M-^PM-^P^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#3M-mM-zM-^NM-UM-<^#|M-{M-|f1M-[f1M-IfSfQ^FWM-^NM-]M-^NM-ERM->^#|M-?^#^FM-9^#^AM-sM-%M-jK^F^#^#RM-4AM-;M-*U1M-I0M-vM-yM-M^Sr^VM-^AM-{UM-*u^PM-^CM-a^At^KfM-G^FM-s^FM-4BM-k^UM-k^B1M-IZQM-4^HM-M^S[^OM-6M-F#PM-^CM-a?QM-wM-aSRPM-;^#|M-9^D^#fM-!M-0^GM-hD^#^OM-^BM-^#^#f#M-^#M-G^BM-bM-rfM-^A>#|M-{M-#xpu M-zM-<M-l{M-jD|^#^#M-hM-^C^#isolinux.bin missing or corrupt.^M$
f`f1M-Rf^C^FM-x{f^S^VM-|{fRfP^FSj^Aj^PM-^IM-ffM-w6M-h{M-#M-d^FM-^HM-aM-^HM-EM-^RM-v6M-n{M-^HM-F^HM-aAM-8^A^BM-^J^VM-r{M-M^SM-^Md^PfaM-CM-h^^^#Operating system load error.^M$
^M-,M-4^NM-^J>b^DM-3^GM-M^P<$
uM-qM-M^XM-tM-kM-}^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#L^D^#^#^#^#^#^#M-/KM-66^#^#M-^#^#^A^#^#?M-`M-^K^#^#^#^#^#`^\^#^#M-~M-^?M-^?M-oM-~M-^?M-^?<R^#^#^#^_^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#UM-*EFI PART^#^#^A^#\^#^#^#]3M-%.^#^#^#^#^A^#^#^#^#^#^#^#M-^?_^\^#^#^#^#^##^#^#^#^#^#^#^#M-J_^\^#^#^#^#^#UcM-)r^Oqc#M-^Rc^FM-2$LZM-p^L^#^#^#^#^#^#^#M-P^#^#^#M-^#^#^#^#M-{t^]F^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#$
.
$ awk -f tst.awk file | cat -ev
ER^H^#^#^#M-^PM-^P^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#3M-mM-zM-^NM-UM-<^#|M-{M-|f1M-[f1M-IfSfQ^FWM-^NM-]M-^NM-ERM->^#|M-?^#^FM-9^#^AM-sM-%M-jK^F^#^#RM-4AM-;M-*U1M-I0M-vM-yM-M^Sr^VM-^AM-{UM-*u^PM-^CM-a^At^KfM-G^FM-s^FM-4BM-k^UM-k^B1M-IZQM-4^HM-M^S[^OM-6M-F#PM-^CM-a?QM-wM-aSRPM-;^#|M-9^D^#fM-!M-0^GM-hD^#^OM-^BM-^#^#f#M-^#M-G^BM-bM-rfM-^A>#|M-{M-#xpu M-zM-<M-l{M-jD|^#^#M-hM-^C^#isolinux.bin missing or corrupt.^M$
f`f1M-Rf^C^FM-x{f^S^VM-|{fRfP^FSj^Aj^PM-^IM-ffM-w6M-h{M-#M-d^FM-^HM-aM-^HM-EM-^RM-v6M-n{M-^HM-F^HM-aAM-8^A^BM-^J^VM-r{M-M^SM-^Md^PfaM-CM-h^^^#Operating system load error.^M$
^M-,M-4^NM-^J>b^DM-3^GM-M^P<$
uM-qM-M^XM-tM-kM-}^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#L^D^#^#^#^#^#^#M-/KM-66^#^#M-^#^#^A^#^#?M-`M-^K^#^#^#^#^#`^\^#^#M-~M-^?M-^?M-oM-~M-^?M-^?<R^#^#^#^_^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#UM-*EFI PART^#^#^A^#\^#^#^#]3M-%.^#^#^#^#^A^#^#^#^#^#^#^#M-^?_^\^#^#^#^#^##^#^#^#^#^#^#^#M-J_^\^#^#^#^#^#UcM-)r^Oqc#M-^Rc^FM-2$LZM-p^L^#^#^#^#^#^#^#M-P^#^#^#M-^#^#^#^#M-{t^]F^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#$
Original answer:
Assuming this question is related to your previous question, this will work using any awk in any shell on every UNIX box:
$ awk '/^__DATA__$/{n=NR+1} n && NR>n' file
3<ED>M-^PM-^PM-^PM-^PM-^
When it finds __DATA__ it sets a variable n to the line number to start printing after and then when n is set prints every line for which the line number is greater than n.
The above was run against this input file from your previous question:
$ cat -ev file
echo "I: Installation finished!"$
exit 0$
$
__DATA__$
$
3<ED>M-^PM-^PM-^PM-^PM-^$
I am working on an operating system with limited utilities. Utilities like tail, head, and tac are not available! sed, awk, and Grep are available, but grep does not have the -m option for stopping after the first find. see the list of available options here.
My goal is to search for a line containing a string in a potentially large log.txt file, maybe ~100Mb from the end in reverse and print it. The trick is the operation has to be fast: no more than 3-4sec tops.
I tried using sed to reverse the contents of the file into another and then using awk and grep in a loop to search chunks of 10,000 lines, but the sed reverse was way too slow for anything beyond a few Mb
Something I tried.
self.sed_line_search = 10001
self.sed_cmd = "sed -e :a -e '$q;N;"+str(self.sed_line_search)+",$D;ba'"
self.awk_cmd = "awk '/Version/{print}'"
self.Command = self.sed_cmd + " " + LOGFILE_PATH + " | " + self.awk_cmd + "\n"
tries, max_tries = 1,5
while tries < max_tries:
ret = execute(self.Command)
if not ret:
self.sed_line_search += 10000
self.sed_cmd = "sed -e :a -e '$q;N;"+str(self.sed_line_search)+",$D;ba'"
self.Command = self.sed_cmd + " " + LOGFILE_PATH + " | " + self.awk_cmd + "\n"
tries += 1
With out knowing how to stop at the fist match without the grep -m 1 option, this slightly achieves that goal by only looking at a few thousand lines at a time. But, It does not search in reverse.
Not sure if it this you want. It search for all line with test and prints them in reveres.
cat file
dfsdf
test1
fsdfsdf
fdg
sfdgs
fdgsdf
gsfdg
sfdte
test2
dgsfdgsdf
fdgsfdg
sdfgs
df
test3
sfdgsfdg
awk '/test/ {a[++x]=$0} END {for (i=x;i>=1;i--) print a[i]}' file
test3
test2
test1
This might work for you (GNU sed):
sed -n '/regexp/h;$!b;x;p' file
Copy the line that matches regexp to the hold space and at the end of the file print the hold space.
IMHO the fastest you could do would be:
grep 'regexp' | sed -n '$p'
I have a file with options for a command I run. Whenever I run the command I want it to run with the options defined in the first line which is not commented out. I do this using this bash script:
while read run opt c; do
[[ $run == \#* ]] && continue
./submit.py $opt $run -c "$c"
break
done < to_submit.txt
The file to_submit.txt has entries like this:
#167993 options/optionfile.py long description
167995 options/other_optionfile.py other long description
...
After having run the submit script with the options in the last not commented out line, I want to comment out that line after the command ran successfully.
I can find the line number of the options I used adding this to the while loop:
line=$(grep -n "$run" to_submit.txt | grep "$opt" | grep "$c" | cut -f 1 -d ":")
But I'm not sure how to actually prepend a # to that line now. I could probably use head and tail to save the other lines and process that line separately and combine it all back into the file. But this sounds like it's to complicated, there must be an easier sed or awk solution to this.
$ awk '!f && sub(/^[^#]/,"#&"){f=1} 1' file
#167993 options/optionfile.py long description
#167995 options/other_optionfile.py other long description
...
To overwrite the contents of the original file:
awk '!f && sub(/^[^#]/,"#&"){f=1} 1' file > tmp && mv tmp file
just like with any other UNIX command.
Using GNU sed is probably simplest here:
sed '0,/^[^#]/ s//#&/' file
Add option -i if you want to update file in place.
'0,/^[^#]/ matches all lines up to and including the first one that doesn't start with #
s//#&/ then prepends # to that line.
Note that s//.../ (i.e., an empty regex) reuses the last matching regex in the range, which is /^[^#]/ in this case.
Note that the command doesn't work with BSD/OSX sed, unfortunately, because starting a range with 0 so as to allow the range endpoint to match the very first line also is not supported there. It is possible to make the command work with BSD/OSX sed, but it's more cumbersome.
If the input/output file is not very large, you can do it all in Bash:
optsfile=to_submit.txt
has_run_cmd=0
outputlines=()
while IFS= read -r inputline || [[ -n $inputline ]] ; do
read run opt c <<<"$inputline"
if (( has_run_cmd )) || [[ $run == \#* ]] ; then
outputlines+=( "$inputline" )
elif ./submit.py "$opt" "$run" -c "$c" ; then
has_run_cmd=1
outputlines+=( "#$inputline" )
else
exit $?
fi
done < "$optsfile"
(( has_run_cmd )) && printf '%s\n' "${outputlines[#]}" > "$optsfile"
The lines of the file are put in the outputlines array, with a hash prepended to the line that was used in the ./submit.py command. If the command runs successfully, the file is overwritten with the lines in outputlines.
After some searching around I found that
awk -v run="$run" -v opt="$opt" '{if($1 == run && $2 == opt) {print "#" $0} else print}' to_submit.txt > temp
mv -b -f temp to_submit.txt
seems to solve this (without needing to find the line number first, just comparing $ run and $opt). This assumes that the combination of run and opt is enough to identify a line and the comment is not needed (which happens to be true in my case). Not sure how the comment which is spanning multiple fields in awk would also be taken into account.
I'm currently writing a shell script that will be given a directory, then output an ls of that directory with the return code from a C program appended to each line. The C program only needs to be called for regular files.
The problem I'm having is that output from the C program is cluttering up the output from awk, and I can't get stdout to redirect to /dev/null inside of awk. I have no use for the output, I just need the return code. Speed is definitely a factor, so if you have a more efficient solution I'd be happy to hear it. Code follows:
directory=$1
ls -i --full-time $directory | awk '
{
rc = 0
if (substr($2,1,1) == "-") {
dbType=system("cprogram '$directory'/"$10)
}
print $0 " " rc
}
'
awk is not shell so you cant just use a shell variable inside an awk script, and in shell always quote your variables. Try this:
directory="$1"
ls -i --full-time "$directory" | awk -v dir="$directory" '
{
rc = 0
if (substr($2,1,1) == "-") {
rc = system("cprogram \"" dir "/" $10 "\" >/dev/null")
}
print $0, rc
}
'
Oh and, of course, don't actually do this. See http://mywiki.wooledge.org/ParsingLs.
I just spent a minute thinking about what your script is actually doing and rather than trying to use awk as a shell and parse the output of ls, it looks like the solution you REALLY want would be more like:
directory="$1"
find "$directory" -type f -maxdepth 1 -print |
while IFS= read -r dirFile
do
op=$(ls -i --full-time "$dirFile")
cprogram "$dirFile" >/dev/null
rc="$?"
printf "%s %s\n" "$op" "$rc"
done
and you could probably save a step by using the -printf arg for find to get whatever info you're currently using ls for.
Using DCL, i have a .txt file with 3 lines
Line 1 test.
Line 2 test.
Line 3 test.
I'm trying to very that each contains exactly what is expected. I'm currently using the f#extract function which will give me the output of line 1 but i cannot figure out how to verify line 2 and 3. What function can i use to make sure lines 2 and 3 are correct?
$ OPEN read_test test.dat
$ READ/END_OF_FILE=ender read_test cc
$ line1 = f$extract(0,15,cc)
$ if line1.nes."Line 1 test."
$ then
$ WRITE SYS$OUTPUT "FALSE"
$ endif
$ line2 = f$extract(??,??,cc) ! f$extract not possible for multiple lines?
$ if line2.nes."Line 2 test."
$ then
$ WRITE SYS$OUTPUT "FALSE"
$ endif
For exactly 3 line you might just want to do 3 reads and 3 compares...
$ READ READ/END_OF_FILE=ender read_test cc
$ if f$extract(0,15,cc).nes."Line 1 test." ...
$ READ READ/END_OF_FILE=ender read_test cc
$ if f$extract(0,15,cc).nes."Line 2 test." ...
$ READ READ/END_OF_FILE=ender read_test cc
$ if f$extract(0,15,cc).nes."Line 3 test." ...
Any more and you want to be in a loop as replied.
TO follow up on Chris's approach, you may want to first prepare an array of values and then loop reading and comparing as long as there are values.
Untested:
$ line_1 = "Line 1 test."
$ line_2 = "Line 2 test."
$ line_3 = "Line 3 test."
$ line_num = 1
$ReadNext:
$ READ/END_OF_FILE=ender read_test cc
$ if line_'line_num'.nes.cc then WRITE SYS$OUTPUT "Line ", line_num, " FALSE"
$ line_num = line_num + 1
$ if f$type(line_'line_num').NES."" then GOTO ReadNext
$ WRITE SYS$OUTPUT "All provided lines checked out TRUE"
$ GOTO end
$Ender:
$ WRITE SYS$OUTPUT "Ran out of lines too soon. FALSE"
$end:
$ close Read_Test
hth,
Hein.
Try this variation (untested, so may need a little debugging). Makes use of symbol substitution to keep track of which line you are up to.
$ OPEN read_test test.dat
$ line_num = 1
$ ReadNext:
$ READ/END_OF_FILE=ender read_test cc
$ line'line_num' = f$extract(0,15,cc)
$ if line'line_num'.nes."Line ''line_num' test."
$ then
$ WRITE SYS$OUTPUT "FALSE"
$ endif
$ goto ReadNext
$ !
$ Ender:
$ close Read_Test
$ write sys$output "line1: "+line1
$ write sys$output "line2: "+line2
$ write sys$output "line3: "+line3
$ exit