Related
As per Lua documentation, file:read("*l") reads next line skipping end of line.
Note:- "*l": reads the next line skipping the end of line, returning nil on end of file. This is the default format
Is this documentation right? Because file:read("*l") reads the current line,instead of next line or my understanding is wrong? Pretty confusing...
Lua manages files using the same model of the underlying C implementation (this model is used also by other programming languages and it is fairly common). If you are not familiar with this way of looking at files, the terminology could be unclear, indeed.
In this model a file is represented as a stream of bytes having a so called current position. The current position is a sort of conceptual pointer to the first byte in the file that will be read or written by the next I/O operation. When you open a file for reading, a new stream is set-up so that its current position is the beginning of the file, i.e. the current position "points" to the first byte in the file.
In Lua you manage streams through so-called file handles, which are a sort of intermediaries for the underlying streams. Any operation you perform using the handle is carried over to the corresponding stream.
Lua io.open opens a file, associates a C stream with it and returns a file handle that represents that stream:
local file_handle = io.open( "myfile.txt" ) -- file opened for reading
Therefore, if you perform any operation that reads some bytes (usually interpreted as characters, if you work with text files) those are read from the stream and for each byte read the current position of the stream advances by one, pointing each time to the next byte to be read.
Lua documentation implies this model. Thus when it says next line, it means that the input operation will read all characters in the stream starting from the current position until an end-of-line character is found.
Note that if you look at text files as a sequence of lines you could be misled, since you could think of a "current line" and a "next line". That would be an higher level model compared to the C model. There is no "current line" in C. In C text files are nothing more than a sequence of bytes where some special characters (end-of-line characters) undergo some special treatment (which is mostly implementation-dependent) and are used by some C standard functions as line terminators, i.e. as marks to detect when stop reading characters.
Another source of confusion for newbies or people coming from higher level languages is that in C, for an historical accident, bytes are handled as characters (the basic data type to handle single bytes is char, which is the smallest numeric type in C!). Therefore for people with a C background it is natural to think of bytes as characters and vice versa.
Although Lua is a much higher level language than C, its close relationship with C (it was designed to be easily interfaced with C code) makes it inherit part of this C "bytes-as-characters" approach. In fact, for example, Lua strings can hold arbitrary bytes and can be used to process raw binary data.
Like Lorenso said above, read starts at the current file position and reads from that position some portion of the file. How much of the file it reads depends on read instruction. For reference, in Lua 5.3:
"*all" : reads to the end of the file
"*line" : reads from the current position to the end of the line.
The end of the line is marked by a special character usually denoted
LfCr (Line feed, carriage return )
"*number" : reads a number, that is, it will read up to the end of what
it recognizes in the text as a number, stopping at, for example, a
comma ",".
num : reads a string with up to num characters
Here's an example that reads a file with a list of numbers into an array (a table), then returns the array. (Just change the "*number" to "*line" and it would read a file line by line):
function read_array(file)
local arr = {}
local handle = assert( io.open(file,"r") )
local value = handle:read("*number")
while value do
table.insert( arr, value )
value = handle:read("*number")
end
handle:close()
return arr
end
I have been tasked with writing a Fortran 95 program that will read character input from a file, and then (to start with) simply spit it back out again.
The tricky part is that these lines of input are of varying length (no maximum length given) and there can be any number of lines within the file.
I've used
do
read( 1, *, iostat = IO ) DNA ! reads to EOF -- GOOD!!
if ( IO < 0 ) exit ! if EOF is reached, exit do
I = I + 1
NumRec = I ! used later for total no. of records
allocate( Seq(I) )
Seq(I) = DNA
print*, I, Seq(I)
X = Len_Trim( Seq(I) ) ! length of individual sequence
print*, 'Sequence size: ', X
print*
end do
However, my initial statements list
character(100), dimension(:), allocatable :: Seq
character(100) DNA
and the appropriate integers etc.
I guess what I'm asking is if there is any way to NOT list the size of the character strings in the first instance. Say I've got a string of DNA that is 200+ characters, and then another that is only 25, is there a way that the program can just read what there is and not need to include all the additional blanks? Can this be done without needing to use len_trim, since it can't be referenced in the declaration statements?
To progressively read a record in Fortran 95, use non-advancing input. For example:
CHARACTER(10) :: buffer
INTEGER :: size
READ (unit, "(A)", ADVANCE='NO', SIZE=size, EOR=10, END=20) buffer
will read up to 10 characters worth (the length of buffer) each time it is called. The file position will only advance to the next record (the next line) once the entire record has been read by a series of one or more non-advancing reads.
Barring an end of file condition, the size variable will be defined with the actual number of characters read into buffer each time the read statement is executed.
The EOR and END and specifiers are used to control execution flow (execution will jump to the appropriately labelled statement) when end of record or end of file conditions occur respectively. You can also use an IOSTAT specifier to detect these conditions, but the particular negative values to use for the two conditions are processor dependent.
You can sum size within a particular record to work out the length of that particular record.
Wrap such a non-advancing read in a loop that appropriately detects for end of file and end of record and you have the incremental reading part.
In Fortran 95, the length specification for a local character variable must be a specification expression - essentially an expression that can be safely evaluated prior to the first executable statement of the scope that contains the variable's declaration. Constants represent the simplest case, but a specification expression in a procedure can involve dummy arguments of that procedure, amongst other things.
Reading the entire record of arbitrary length in is then a multi stage process:
Determine the length of the current record by using a series of incremental reads. These incremental reads for a particular record finish when the end of record condition occurs, at which time the file position will have moved to the next record.
Backspace the file back to the record of interest.
Call a procedure, passing the length of the current record as a dummy argument. Inside that procedure have an character variable whose length is given by the dummy argument.
Inside that called procedure, read the current record into that character variable using normal advancing input.
Carry out further processing on that character variable!
Note that each record ends up being read twice - once to determine its length, the second to actually read the data into the correctly "lengthed" character variable.
Alternative approaches exist that use allocatable (or automatic) character arrays of length one. The overall strategy is the same. Look at the code of the Get procedures in the common ISO_VARYING_STRING implementation for an example.
Fortran 2003 introduces deferred length character variables, which can have their length specified by an arbitrary expression in an allocate statement or, for allocatable variables, by the length of the right hand side in an assignment statement. This (in conjunction with other "allocatable" enhancements) allows the progressive read that determines the record length to also build the character variable that holds the contents of the record. Your supervisor needs to bring his Fortran environment up to date.
Here's a function for Fortran 2003, which sets an allocatable string (InLine) of exactly the length of the input string (optionally trimmed), or returns .false. if end of file
function ReadLine(aunit, InLine, trimmed) result(OK)
integer, intent(IN) :: aunit
character(LEN=:), allocatable, optional :: InLine
logical, intent(in), optional :: trimmed
integer, parameter :: line_buf_len= 1024*4
character(LEN=line_buf_len) :: InS
logical :: OK, set
integer status, size
OK = .false.
set = .true.
do
read (aunit,'(a)',advance='NO',iostat=status, size=size) InS
OK = .not. IS_IOSTAT_END(status)
if (.not. OK) return
if (present(InLine)) then
if (set) then
InLine = InS(1:size)
set=.false.
else
InLine = InLine // InS(1:size)
end if
end if
if (IS_IOSTAT_EOR(status)) exit
end do
if (present(trimmed) .and. present(InLine)) then
if (trimmed) InLine = trim(adjustl(InLine))
end if
end function ReadLine
For example to do something with all lines in a file with unit "aunit" do
character(LEN=:), allocatable :: InLine
do while (ReadLine(aunit, InLine))
[.. something with InLine]
end do
I have used the following. Let me know if it is better or worse than yours.
!::::::::::::::::::::: SUBROUTINE OR FUNCTION :::::::::::::::::::::::::::::::::::::::
!__________________ SUBROUTINE lineread(filno,cargout,ios) __________________________
subroutine lineread(filno,cargout,ios)
Use reallocate,ErrorMsg,SumStr1,ChCount
! this subroutine reads
! 1. following row in a file except a blank line or the line begins with a !#*
! 2. the part of the string until first !#*-sign is found or to end of string
!
! input Arguments:
! filno (integer) input file number
!
! output Arguments:
! cargout (character) output chArActer string, converted so that all unecessay spaces/tabs/control characters removed.
implicit none
integer,intent(in)::filno
character*(*),intent(out)::cargout
integer,intent(out)::ios
integer::nlen=0,i,ip,ich,isp,nsp,size
character*11,parameter::sep='=,;()[]{}*~'
character::ch,temp*100
character,pointer::crad(:)
nullify(crad)
cargout=''; nlen=0; isp=0; nsp=0; ich=-1; ios=0
Do While(ios/=-1) !The eof() isn't standard Fortran.
READ(filno,"(A)",ADVANCE='NO',SIZE=size,iostat=ios,ERR=9,END=9)ch ! start reading file
! read(filno,*,iostat=ios,err=9)ch;
if(size>0.and.ios>=0)then
ich=iachar(ch)
else
READ(filno,"(A)",ADVANCE='no',SIZE=size,iostat=ios,EOR=9); if(nlen>0)exit
end if
if(ich<=32)then ! tab(9) or space(32) character
if(nlen>0)then
if(isp==2)then
isp=0;
else
isp=1;
end if
eend if; cycle;
elseif(ich==33.or.ich==35.or.ich==38)then !if char is comment !# or continue sign &
READ(filno,"(A)",ADVANCE='yes',SIZE=size,iostat=ios,EOR=9)ch; if(nlen>0.and.ich/=38)exit;
else
ip=scan(ch,sep);
if(isp==1.and.ip==0)then; nlen=nlen+1; crad=>reallocate(crad,nlen); nsp=nsp+1; endif
nlen=nlen+1; crad=>reallocate(crad,nlen); crad(nlen)=ch;
isp=0; if(ip==1)isp=2;
end if
end do
9 if(size*ios>0)call ErrorMsg('Met error in reading file in [lineread]',-1)
! ios<0: Indicating an end-of-file or end-of-record condition occurred.
if(nlen==0)return
!write(6,'(a,l)')SumStr1(crad),eof(filno)
!do i=1,nlen-1; write(6,'(a,$)')crad(i:i); end do; if(nlen>0)write(6,'(a)')crad(i:i)
cargout=SumStr1(crad)
nsp=nsp+1; i=ChCount(SumStr1(crad),' ',',')+1;
if(len(cargout)<nlen)then
call ErrorMsg(SumStr1(crad)// " is too long!",-1)
!elseif(i/=nsp.and.nlen>=0)then
! call ErrorMsg(SumStr1(crad)// " has unrecognizable data number!",-1)
end if
end subroutine lineread
I'm using Fortran 90 to do this:
X = Len_Trim( Seq(I) ) ! length of individual sequence
write(*,'(a<X>)') Seq(I)(1:X)
You can simply declare Seq to be a large character string and then trim it as your write it out. I don't know how kosher this solution is but it certainly works for my purpose. I know that some compilers do not support "variable format expressions", but there are various workarounds to do the same thing almost as simply.
GNU Fortran variable expression workaround.
I process lots of output from Fortran programs, and it generally appears that the output from these programs has a leading space, i.e., character column 1 is blank. Consider a short program leading.f:
program leading
print *, "<-- What's that space for?"
end program leading
Has the output
<-- What's that space for?
(yes, there is one space character at the beginning). My question is contained in the program.
Back in the dinosaur era, when FORTRAN output usually went to a green-bar impact printer, certain characters in the first print column were often interpreted as control codes (line feeds, form feeds, etc). Many programmers learned to explicitly blank column 1 of their output, unless some special effect was intended -- and old habits die hard!
As has been noted in another answer here, and elsewhere, Fortran output had the concept of carriage control. For printers which used carriage control the first character being a blank was necessary for a new line to be started.
Fortran itself deleted carriage control concept in Fortran 2003, but for completeness we can see that Fortran still requires list-directed output to have (in most cases) this (default) leading blank (Fortran 2018, 13.10.4 p.13):
Except for new records created by explicit formatting within a defined output procedure or by continuation of delimited character sequences, each output record begins with a blank character.
Namelist formatting has a similar statement.
You can avoid having this leading blank by avoiding using list-directed output:
print '(A)', '<-- No space here'
end
Note that it isn't the print here, but the list-directed output, which is to blame. We see similar with write:
write (*,*) '<-- Space from the list-directed output'
end
Finally, if we are using internal files we still get a leading blank with list-directed output:
character(len=20) :: internal
write (internal, *) '<-- Leading blank here'
end
(If we then output this internal file with list-directed output we'll see two leading blanks.)
I am writing a FORTRAN code that uses data in a file made by a MD program. the data is a list of values but has breaks in the data for list updates in the form (# Neighbor list update .. 6527 indexes in list), These breaks are at random intervals so I can't just skip every x
I when I do my code it doesn't ignore these lines and randomly adds the value from the previous step.
1, 0.98510699999999995, 0.98510699999999995
2, 1.9654170000000000, 0.98031000000000001
3, 2.9427820000000002, 0.97736500000000004
4, 3.9186540000000001, 0.97587199999999996
4, 4.8945259999999999, 0.97587199999999996
5, 5.8697910000000002, 0.97526500000000005
note the double step 4 with an identical value from the true step 4
How would I go about skipping this line. Please find the sample code below
Open(Unit=10,File='prod._100.tup')
do i=1,50
Read(10,*,IOSTAT=ios)step,temp,kinetic,potential,total,pressure
If(IS_IOSTAT_END(ios)) Exit
test=test+temp
print*, step, test, temp
End Do
It is not clear to me what the "breaks" in the file are. Are they blank lines? If so, the following code should work:
use, intrinsic :: iso_fortran_env
character (len=200) :: line
Open(Unit=10,File='prod._100.tup')
read_loop: do
Read (10,'(A)',IOSTAT=ios) line
If(ios == iostat_end) exit read_loop
if (len_trim (line) == 0) then
write (*, *) "blank line"
cycle read_loop
end if
read (line, *) step,temp,kinetic,potential,total,pressure
test=test+temp
print*, step, test, temp
end do: read_loop
write (*, *) "total is", test
The above is not tested. The "len_trim" test is based on bad records being blank lines. If breaks are otherwise defined you will have to create a different test.
Try:
i=1
do while (i<=50)
Read(10,*,IOSTAT=ios)step,temp,kinetic,potential,total,pressure
If(IS_IOSTAT_END(ios)) Exit
IF(ios.ne.0) cycle
test=test+temp
i=i+1
enddo
When a bad record is read, ios is assigned a system dependent non-zero number (it is zero on success). Apparently you've written a function (IS_IOSTAT_END) to tell if you've reached the end of the file, but other error conditions can exist (for example, the read statement doesn't match the data). That will return a different non-zero ios than an end-file record, so you should just restart the loop at that point (e.g. cycle)
I assume you want to read exactly 50 lines from the file, so I changed your do loop to a do while, but if the number of records you read doesn't actually matter, then feel free to change it back.
do you know if the following statement is guaranteed to be true by one of the fortran 90/95/2003 standards?
"Suppose a read statement for a character variable is given a blank line (i.e., containing only white spaces and new line characters). If the format specifier is an asterisk (*), it continues to read the subsequent lines until a non-blank line is found. If the format specifier is '(A)', a blank string is substituted to the character variable."
For example, please look at the following minimal program and input file.
program code:
PROGRAM chk_read
INTEGER, PARAMETER :: MAXLEN=30
CHARACTER(len=MAXLEN) :: str1, str2
str1='minomonta'
read(*,*) str1
write(*,'(3A)') 'str1_start|', str1, '|str1_end'
str2='minomonta'
read(*,'(A)') str2
write(*,'(3A)') 'str2_start|', str2, '|str2_end'
END PROGRAM chk_read
input file:
----'input.dat' content is below this line----
yamanakako
kawaguchiko
----'input.dat' content is above this line----
Please note that there are four lines in 'input.dat' and the first and third lines are blank (contain only white spaces and new line characters). If I run the program as
$ ../chk_read < input.dat > output.dat
I get the following output
----'output.dat' content is below this line----
str1_start|yamanakako |str1_end
str2_start| |str2_end
----'output.dat' content is above this line----
The first read statement for the variable 'str1' seems to look at the first line of 'input.dat', find a blank line, move on to the second line, find the character value 'yamanakako', and store it in 'str1'.
In contrast, the second read statement for the variable 'str2' seems to be given the third line, which is blank, and store the blank line in 'str2', without moving on to the fourth line.
I tried compiling the program by Intel Fortran (ifort 12.0.4) and GNU Fortran (gfortran 4.5.0) and got the same result.
A little bit about a background of asking this question: I am writing a subroutine to read a data file that uses a blank line as a separator of data blocks. I want to make sure that the blank line, and only the blank line, is thrown away while reading the data. I also need to make it standard conforming and portable.
Thanks for your help.
From Fortran 2008 standard draft:
List-directed input/output allows data editing according to the type
of the list item instead of by a format specification. It also allows
data to be free-field, that is, separated by commas (or semicolons) or
blanks.
Then:
The characters in one or more list-directed records constitute a
sequence of values and value separators. The end of a record has the
same effect as a blank character, unless it is within a character
constant. Any sequence of two or more consecutive blanks is treated as
a single blank, unless it is within a character constant.
This implicitly states that in list-directed input, blank lines are treated as blanks until the next non-blank value.
When using a fmt='(A)' format descriptor when reading, blank lines are read into str. On the other side, fmt=*, which implies list-directed I/O in free-form, skips blank lines until it finds a non-blank character string. To test this, do something like:
PROGRAM chk_read
INTEGER :: cnt
INTEGER, PARAMETER :: MAXLEN=30
CHARACTER(len=MAXLEN) :: str
cnt=1
do
read(*,fmt='(A)',end=100)str
write(*,'(I1,3A)')cnt,' str_start|', str, '|str_end'
cnt=cnt+1
enddo
100 continue
END PROGRAM chk_read
$ cat input.dat
yamanakako
kawaguchiko
EOF
Running the program gives this output:
$ a.out < input.dat
1 str_start| |str_end
2 str_start| |str_end
3 str_start| |str_end
4 str_start|yamanakako |str_end
5 str_start| |str_end
6 str_start|kawaguchiko |str_end
On the other hand, if you use default input:
read(*,fmt=*,end=100)str
You end up with this output:
$ a.out < input.dat
1 str1_start|yamanakako |str1_end
2 str2_start|kawaguchiko |str2_end
This Part of the F2008 standard draft probably treats your problem:
10.10.3 List-directed input
7 When the next effective item is of type character, the input form
consists of a possibly delimited sequence of zero or more
rep-char s whose kind type parameter is implied by the kind of the
effective item. Character sequences may be continued from the end of
one record to the beginning of the next record, but the end of record
shall not occur between a doubled apostrophe in an
apostrophe-delimited character sequence, nor between a doubled quote
in a quote-delimited character sequence. The end of the record does
not cause a blank or any other character to become part of the
character sequence. The character sequence may be continued on as many
records as needed. The characters blank, comma, semicolon, and slash
may appear in default, ASCII, or ISO 10646 character sequences.