Related
Given an input that looks like the following:
123
456
789
42
23
1337
3117
I want to iterate over this file in whitespace-separated chunks in Fortran (any version is fine). For example, let's say I wanted to take the average of each chunk (e.g. mean(123, 456, 789) then mean(42, 23, 1337) then mean(31337)).
I've tried iterating through the file normally (e.g. READ), reading in each line as a string and then converting to an int and doing whatever math I want to do on each chunk. The trouble here is that Fortran "helpfully" ignores blank lines in my text file - so when I try and compare against the empty string to check for the blank line, I never actually get a .True. on that comparison.
I feel like I'm missing something basic here, since this is a typical functionality in every other modern language, I'd be surprised if Fortran didn't somehow have it.
If you're using so-called "list-directed" input (format = '*'), Fortran does special handling to spaces, commas, and blank lines.
To your point, there's a feature which is using the BLANK keyword with read
read(iunit,'(i10)',blank="ZERO",err=1,end=2) array
You can set:
blank="ZERO" will return a valid zero value if a blank is found;
blank="NULL" is the default behavior that skips blank/returns an error depending on the input format.
If all your input values are positive, you could use blank="ZERO" and then use the location of zero values to process your data.
EDIT as #vladimir-f has correctly pointed out, you not only have blanks in between lines, but also after the end of the numbers in most lines, so this strategy will not work.
You can instead load everything into an array, and process it afterwards:
program array_with_blanks
integer :: ierr,num,iunit
integer, allocatable :: array(:)
open(newunit=iunit,file='stackoverflow',form='formatted',iostat=ierr)
allocate(array(0))
do
read(iunit,'(i10)',iostat=ierr) num
if (is_iostat_end(ierr)) then
exit
else
array = [array,num]
endif
end do
close(iunit)
print *, array
end program
Just read each line as a character (but note Francescalus's comment on the format). Then read the character as an internal file.
program stuff
implicit none
integer io, n, value, sum
character (len=1000) line
n = 0
sum = 0
io = 0
open( 42, file="stuff.txt" )
do while( io == 0 )
read( 42, "( a )", iostat = io ) line
if ( io /= 0 .or. line == "" ) then
if ( n > 0 ) print *, ( sum + 0.0 ) / n
n = 0
sum = 0
else
read( line, * ) value
n = n + 1
sum = sum + value
end if
end do
close( 42 )
end program stuff
456.000000
467.333344
3117.00000
Not entirely sure why, but either awk is rounding off my variable x to 2dp OR the regex pattern to test if integer doesn't work properly.
My code:
awk 'BEGIN{for (i=972286200; i<=972384300; i=i+900) {x=((i/86400)-0.375); if(x ~ /^[0-9]+$/ ) {s=i} printf("%010d,%.3f,%010d\n",i,x,s)}}'
Code is trying to create a dataset that has columns like this:
i - Data incremented by 900
x - Test if data divides by a number
s - If the division IS integer, use i, if NOT use previous integer i
Currently produces:
0972286200,11252.938,0000000000
0972287100,11252.948,0000000000
0972288000,11252.958,0972288000
0972288900,11252.969,0972288900
0972289800,11252.979,0972289800
0972290700,11252.990,0972290700
0972291600,11253.000,0972291600
0972292500,11253.010,0972292500
0972293400,11253.021,0972293400
0972294300,11253.031,0972294300
0972295200,11253.042,0972295200
0972296100,11253.052,0972295200
0972297000,11253.062,0972295200
0972297900,11253.073,0972295200
...
...
0972372600,11253.938,0972295200
0972373500,11253.948,0972295200
0972374400,11253.958,0972374400
0972375300,11253.969,0972375300
0972376200,11253.979,0972376200
0972377100,11253.990,0972377100
0972378000,11254.000,0972378000
0972378900,11254.010,0972378900
0972379800,11254.021,0972379800
0972380700,11254.031,0972380700
0972381600,11254.042,0972381600
0972382500,11254.052,0972381600
0972383400,11254.062,0972381600
0972384300,11254.073,0972381600
I want it to produce:
0972286200,11252.938,0000000000
0972287100,11252.948,0000000000
0972288000,11252.958,0000000000
0972288900,11252.969,0000000000
0972289800,11252.979,0000000000
0972290700,11252.990,0000000000
0972291600,11253.000,0972291600
0972292500,11253.010,0972291600
0972293400,11253.021,0972291600
0972294300,11253.031,0972291600
0972295200,11253.042,0972291600
0972296100,11253.052,0972291600
0972297000,11253.062,0972291600
0972297900,11253.073,0972291600
...
...
0972372600,11253.938,0972291600
0972373500,11253.948,0972291600
0972374400,11253.958,0972291600
0972375300,11253.969,0972291600
0972376200,11253.979,0972291600
0972377100,11253.990,0972291600
0972378000,11254.000,0972378000
0972378900,11254.010,0972378000
0972379800,11254.021,0972378000
0972380700,11254.031,0972378000
0972381600,11254.042,0972378000
0972382500,11254.052,0972378000
0972383400,11254.062,0972378000
0972384300,11254.073,0972378000
do the int test with x==int(x) instead.
$ awk 'BEGIN {for(i=972286200; i<=972384300; i+=900)
{x=(i/86400)-0.375;
if(x==int(x)) s=i;
printf "%010d,%.3f,%010d\n",i,x,s}}'
also changed couple minor things.
Is there a way to plot a function based on values from a text file?
I know how to define a function in gnuplot and then plot it but that is not what I need.
I have a table with constants for functions that are updated regularly. When this update happens I want to be able to run a script that draws a figure with this new curve. Since there are quite few figures to draw I want to automate the procedure.
Here is an example table with constants:
location a b c
1 1 3 4
2
There are two ways I see to solve the problem but I do not know if and how they can be implemented.
I can then use awk to produce the string: f(x)=1(x)**2+3(x)+4, write it to a file and somehow make gnuplot read this new file and plot on a certain x range.
or use awk inside gnuplot something like f(x) = awk /1/ {print "f(x)="$2 etc., or use awk directly in the plot command.
I any case, I'm stuck and have not found a solution to this problem online, do you have any suggestions?
Another possibilty to have a somewhat generic version for this, you can do the following:
Assume, the parameters are stored in a file parameters.dat with the first line containing the variable names and all others the parameter sets, like
location a b c
1 1 3 4
The script file looks like this:
file = 'parameters.dat'
par_names = system('head -1 '.file)
par_cnt = words(par_names)
# which parameter set to choose
par_line_num = 2
# select the respective string
par_line = system(sprintf('head -%d ', par_line_num).file.' | tail -1')
par_string = ''
do for [i=1:par_cnt] {
eval(word(par_names, i).' = '.word(par_line, i))
}
f(x) = a*x**2 + b*x + c
plot f(x) title sprintf('location = %d', location)
This question (gnuplot store one number from data file into variable) had some hints for me in the first answer.
In my case I have a file which contains parameters for a parabola. I have saved the parameters in gnuplot variables. Then I plot the function containing the parameter variables for each timestep.
#!/usr/bin/gnuplot
datafile = "parabola.txt"
set terminal pngcairo size 1000,500
set xrange [-100:100]
set yrange [-100:100]
titletext(timepar, apar, cpar) = sprintf("In timestep %d we have parameter a = %f, parameter c = %f", timepar, apar, cpar)
do for [step=1:400] {
set output sprintf("parabola%04d.png", step)
# read parameters from file, where the first line is the header, thus the +1
a=system("awk '{ if (NR == " . step . "+1) printf \"%f\", $1}' " . datafile)
c=system("awk '{ if (NR == " . step . "+1) printf \"%f\", $2}' " . datafile)
# convert parameters to numeric format
a=a+0.
c=c+0.
set title titletext(step, a, c)
plot c+a*x**2
}
This gives a series of png files called parabola0001.png,
parabola0002.png,
parabola0003.png,
…, each showing a parabola with the parameters read from the file called parabola.txt. The title contains the parameters of the given time step.
For understanding the gnuplot system() function you have to know that:
stuff inside double quotes is not parsed by gnuplot
the dot is for concatenating strings in gnuplot
the double quotes for the awk printf command have to be escaped, to hide them from gnuplot parser
To test this gnuplot script, save it into a file with an arbitrary name, e.g. parabolaplot.gplot and make it executable (chmad a+x parabolaplot.gplot). The parabola.txt file can be created with
awk 'BEGIN {for (i=1; i<=1000; i++) printf "%f\t%f\n", i/200, i/100}' > parabola.txt
awk '/1/ {print "plot "$2"*x**2+"$3"*x+"$4}' | gnuplot -persist
Will select the line and plot it
This was/is another question about how to extract specific values into variables with gnuplot (maybe it would be worth to create a Wiki entry about this topic).
There is no need for using awk, you can do this simply with gnuplot only (hence platform-independent), even with gnuplot 4.6.0 (March 2012).
You can do a stats (check help stats) and assign the values to variables.
Data: SO15007620_Parameters.txt
location a b c
1 1 3 4
2 -1 2 3
3 2 1 -1
Script: (works with gnuplot 4.6.0, March 2012)
### read parameters from separate file into variables
reset
FILE = "SO15007620_Parameters.txt"
myLine = 1 # line index 0-based
stats FILE u (a=$2, b=$3, c=$4) every ::myLine::myLine nooutput
f(x) = a*x**2 + b*x + c
plot f(x) w l lc rgb "red" ti sprintf("f(x) = %gx^2 + %gx + %g", a,b,c)
### end of script
Result:
I'm using the following gnuplot commands to create a plot:
#!/bin/bash
gnuplot << 'EOF'
set term postscript portrait color enhanced
set output 'out.ps'
plot 'data_file' u 3:2 w points , '' u 3:2:($4!=-3.60 ? $1:'aaa') w labels
EOF
where data_file looks like this:
O4 -1.20 -0.33 -5.20
O9.5 -1.10 -0.30 -3.60
B0 -1.08 -0.30 -3.25
B0.5 -1.00 -0.28 -2.60
B1.5 -0.90 -0.25 -2.10
B2.5 -0.80 -0.22 -1.50
B3 -0.69 -0.20 -1.10
I want gnuplot to label all points with the strings found in column 1, except the one where column 4 is equal to -3.60 in which case I want the aaa string. What I'm getting is that the $4=-3.60 data point is being labeled correctly as aaa, but the rest are not being labeled at all.
Update: gnuplot has no problem showing numbers as labels using the conditional statement, ie: any column but 1 is correctly displayed as a label for each point respecting the conditions imposed. That is, this line displays column 2 (numbres) as point labels respecting the conditional statement:
plot 'data_file' u 3:2 w points , '' u 3:2:($4!=-3.60 ? $2:'aaa') w labels
Update 2: It also has no problem in plotting column 1 as point labels if I plot it as a whole, ie not using a conditional statement. That is, this line plots correctly all the point labels in column 1 (strings):
plot 'data_file' u 3:2 w points , '' u 3:2:1 w labels
So clearly the problem is in using the conditional statement together with the strings column. Any of these used separately works just fine.
In a more clean way maybe, this should work. It seems label can't display a computed number if it isn't turned in a string.
#!/bin/bash
gnuplot << 'EOF'
set term postscript portrait color enhanced
set output 'out.ps'
plot 'data_file' u 3:2 w points , '' u 3:2:($4!=-3.60 ? sprintf("%d",$1):'aaa') w labels
EOF
Is this what you want?
#!/bin/bash
gnuplot << 'EOF'
set term postscript portrait color enhanced
set output 'out.ps'
plot 'data_file' u 3:2 w points , \
'' u (($4 == -3.60)? 1/0 : $3):2:1 w labels
EOF
All I do here is set (x) points where the column 4 equals -3.6 to NaN (1/0). Since gnuplot ignores those points, life is good. I think the problem with your script is that you were filtering a column where gnuplot expects string input -- although I haven't played around with it enough to verify that. I just switched the filter to a column where gnuplot expects numbers (the x position) and it works just fine.
OSX v10.6.8 and Gnuplot v4.4
I have a data file with 8 columns. I would like to take the first value from the 6th column and make it the title. Here's what I have so far:
#m1 m2 q taua taue K avgPeriodRatio time
#1 2 3 4 5 6 7 8
K = #read in data here
graph(n) = sprintf("K=%.2e",n)
set term aqua enhanced font "Times-Roman,18"
plot file using 1:3 title graph(K)
And here is what the first few rows of my data file looks like:
1.00e-07 1.00e-07 1.00e+00 1.00e+05 1.00e+04 1.00e+01 1.310 12070.00
1.11e-06 1.00e-07 9.02e-02 1.00e+05 1.00e+04 1.00e+01 1.310 12070.00
2.12e-06 1.00e-07 4.72e-02 1.00e+05 1.00e+04 1.00e+01 1.310 12070.00
3.13e-06 1.00e-07 3.20e-02 1.00e+05 1.00e+04 1.00e+01 1.310 12090.00
I don't know how to correctly read in the data or if this is even the right way to go about this.
EDIT #1
Ok, thanks to mgilson I now have
#m1 m2 q taua taue K avgPeriodRatio time
#1 2 3 4 5 6 7 8
set term aqua enhanced font "Times-Roman,18"
K = "`head -1 datafile | awk '{print $6}'`"
print K+0
graph(n) = sprintf("K=%.2e",n)
plot file using 1:3 title graph(K)
but I get the error: Non-numeric string found where a numeric expression was expected
EDIT #2
file = "testPlot.txt"
K = "`head -1 file | awk '{print $6}'`"
K=K+0 #Cast K to a floating point number #this is line 9
graph(n) = sprintf("K=%.2e",n)
plot file using 1:3 title graph(K)
This gives the error--> head: file: No such file or directory
"testPlot.gnu", line 9: Non-numeric string found where a numeric expression was expected
You have a few options...
FIRST OPTION:
use columnheader
plot file using 1:3 title columnheader(6)
I haven't tested it, but this may prevent the first row from actually being plotted.
SECOND OPTION:
use an external utility to get the title:
TITLE="`head -1 datafile | awk '{print $6}'`"
plot 'datafile' using 1:3 title TITLE
If the variable is numeric, and you want to reformat it, in gnuplot, you can cast strings to a numeric type (integer/float) by adding 0 to them (e.g).
print "36.5"+0
Then you can format it with sprintf or gprintf as you're already doing.
It's weird that there is no float function. (int will work if you want to cast to an integer).
EDIT
The script below worked for me (when I pasted your example data into a file called "datafile"):
K = "`head -1 datafile | awk '{print $6}'`"
K=K+0 #Cast K to a floating point number
graph(n) = sprintf("K=%.2e",n)
plot "datafile" using 1:3 title graph(K)
EDIT 2 (addresses comments below)
To expand a variable in backtics, you'll need macros:
set macro
file="mydatafile.txt"
#THE ORDER OF QUOTES (' and ") IS CRUCIAL HERE.
cmd='"`head -1 ' . file . ' | awk ''{print $6}''`"'
# . is string concatenation. (this string has 3 pieces)
# to get a single quote inside a single quoted string
# you need to double. e.g. 'a''b' yields the string a'b
data=#cmd
To address your question 2, it is a good idea to familiarize yourself with shell utilities -- sed and awk can both do it. I'll show a combination of head/tail:
cmd='"`head -2 ' . file . ' | tail -1 | awk ''{print $6}''`"'
should work.
EDIT 3
I recently learned that in gnuplot, system is a function as well as a command. To do the above without all the backtic gymnastics,
data=system("head -1 " . file . " | awk '{print $6}'")
Wow, much better.
This is a very old question, but here's a nice way to get access to a single value anywhere in your data file and save it as a gnuplot-accessible variable:
set term unknown #This terminal will not attempt to plot anything
plot 'myfile.dat' index 0 every 1:1:0:0:0:0 u (var=$1):1
The index number allows you to address a particular dataset (separated by two carriage returns), while every allows you to specify a particular line.
The colon-separated numbers after every should be of the form 1:1:<line_number>:<block_number>:<line_number>:<block_number>, where the line number is the line with the the block (starting from 0), and the block number is the number of the block (separated by a single carriage return, again starting from 0). The first and second numbers say plot every 1 lines and every one data block, and the third and fourth say start from line <line_number> and block <block_number>. The fifth and sixth say where to stop. This allows you to select a single line anywhere in your data file.
The last part of the plot command assigns the value in a particular column (in this case, column 1) to your variable (var). There needs to be two values to a plot command, so I chose column 1 to plot against my variable assignment statement.
Here is a less 'awk'-ward solution which assigns the value from the first row and 6th column of the file 'Data.txt' to the variable x16.
set table
# Syntax: u 0:($0==RowIndex?(VariableName=$ColumnIndex):$ColumnIndex)
# RowIndex starts with 0, ColumnIndex starts with 1
# 'u' is an abbreviation for the 'using' modifier
plot 'Data.txt' u 0:($0==0?(x16=$6):$6)
unset table
A more general example for storing several values is given below:
# Load data from file to variable
# Gnuplot can only access the data via the "plot" command
set table
# Syntax: u 0:($0==RowIndex?(VariableName=$ColumnIndex):$ColumnIndex)
# RowIndex starts with 0, ColumnIndex starts with 1
# 'u' is an abbreviation for the 'using' modifier
# Example: Assign all values according to: xij = Data33[i,j]; i,j = 1,2,3
plot 'Data33.txt' u 0:($0==0?(x11=$1):$1),\
'' u 0:($0==0?(x12=$2):$2),\
'' u 0:($0==0?(x13=$3):$3),\
'' u 0:($0==1?(x21=$1):$1),\
'' u 0:($0==1?(x22=$2):$2),\
'' u 0:($0==1?(x23=$3):$3),\
'' u 0:($0==2?(x31=$1):$1),\
'' u 0:($0==2?(x32=$2):$2),\
'' u 0:($0==2?(x33=$3):$3)
unset table
print x11, x12, x13 # Data from first row
print x21, x22, x23 # Data from second row
print x31, x32, x33 # Data from third row