I am trying to write down a Fortran subroutine for my code in order to read a data from a file (which is a huge data set on itself).The data file contains the Location (nx0,ny0,nz0) and the field related to that location (Bx,By,Bz).
(Ex: lets say the range for nx0, ny0 and nz0 is from [-15,15].
so the number of rows will be 31*31*31=29791)
-15.00000 -15.00000 -15.00000 700.00000 -590.00000 100.00000
-15.00000 -15.00000 -14.00000 -110.00000 -570.00000 100.00000
-15.00000 -15.00000 -13.00000 -550.00000 -200.00000 100.00000
-15.00000 -15.00000 -12.00000 -540.00000 -230.00000 100.00000
-15.00000 -15.00000 -11.00000 -140.00000 -50.00000 100.00000
. . . . . .
. . . . . .
. . . . . .
15.00000 15.00000 15.00000 140.00000 50.00000 100.000
What I want to do is to look for a specific location within my file (xi,yi and zi) and read the field related to that location then use it for further analysis. Not only the related field to the target position itself but also the surrounding field of that location (Like the three other side of the square around the target point).
subroutine read_data(xi,yi,zi,Bxij,Byij)
real*8,intent(in) :: xi,yi,zi !,time
real*8,intent(out) :: Bxij(4),Byij(4) !,Bzij(4)
integer,parameter :: step = 1 ,cols = 6, rows = 29791 !!!15,000,000
real,dimension(rows) :: nx0,ny0,nz0,Bx,By,Bz
character*15 filein
character*35 path_file
path_file = '/home/mehdi/Desktop/'
filein= 'test-0001'
open(7,file=trim(path_file)//filein, status='old',action='read')
xi_1 = xi +step
yi_1 = yi +step
do i = 1,rows
read(7,*) nx0(i),ny0(i),nz0(i),Bx(i),By(i),Bz(i)
c
if ( xi == nx0(i) .and. yi == ny0(i) .and.
& zi == nz0(i)) then
Bxij(1) = Bx(i)
Byij(1) = By(i)
cycle
endif
c
if ( xi == nx0(i) .and. yi_1 == ny0(i) .and.
& zi == nz0(i)) then
Bxij(2) = Bx(i)
Byij(2) = By(i)
cycle
endif
c
if ( xi_1 == nx0(i) .and. yi == ny0(i) .and.
& zi == nz0(i)) then
Bxij(3) = Bx(i)
Byij(3) = By(i)
cycle
endif
c
if ( xi_1 == nx0(i) .and. yi_1 == ny0(i) .and.
& zi == nz0(i)) then
Bxij(4) = Bx(i)
Byij(4) = By(i)
exit
endif
c
close(7)
enddo
end
I have done it this way but it is too slow. One of the most important things for me is the speed (which even for this small fraction of data set is really time consuming).
I know this slow mode is for the needs to read the whole data set each time in order to look for the target points. This subroutine is called couple times within the code and for the further steps the code is going to do the same thing over and over again, so it is time consuming.
How can I make this code work more efficiently?
Before I begin this answer, let me reiterate what I said in the comments to your question:
Do not underestimate how much data you can put into a single array. Reading once, and then having everything in memory is still the fastest way possible.
But let's assume that the data really gets too big.
Your main issue seems to be that you have to re-read all the data from the beginning until you find the value you're looking for. That takes the time.
If you can calculate which line of the data file the value you are interested in is, it might help to convert the file into an unformatted direct access file.
Here is an example code for the conversion. It's using Fortran 2008 features, so if your compiler can't do it, you have to modify it:
program convert
use iso_fortran_env, only: real64
implicit none
integer, parameter :: reclength = 6*8 ! Six 8-byte values
integer :: ii, ios
integer :: u_in, u_out
real(kind=real64) :: pos(3), B(3)
open(newunit=u_in, file='data.txt', form='formatted', &
status='old', action='read', access='sequential')
open(newunit=u_out, file='data.bin', form='unformatted', &
status='new', action='write', access='direct', recl=reclength)
ii = 0
do
ii = ii + 1
read(u_in, *, iostat=ios) pos, B
if (ios /= 0) exit
write(u_out, rec=ii) pos, B
end do
close(u_out)
close(u_in)
end program convert
Once you have converted the data, you can read only the record you need, as long as you can calculate which one it is. I have assumed that just like in your example, the z-coordinate changes fastest and the x-coordinate changes slowest.
program read_txt
use iso_fortran_env, only: real64
implicit none
integer, parameter :: nx=601, ny=181, nz=61
real(kind=real64), parameter :: x_min=real(-nx/2, kind=real64)
real(kind=real64), parameter :: y_min=real(-ny/2, kind=real64)
real(kind=real64), parameter :: z_min=real(-nz/2, kind=real64)
real(kind=real64), parameter :: x_step = 1.0_real64
real(kind=real64), parameter :: y_step = 1.0_real64
real(kind=real64), parameter :: z_step = 1.0_real64
real(kind=real64) :: request(3), pos(3), B(3)
integer :: ios, u_in
integer :: ii, jj, kk, record
integer, parameter :: reclength = 6 * 8 ! Six 8-byte values
open(newunit=u_in, file='data.bin', access='direct', form='unformatted', &
status='old', action='read', recl=reclength)
mainloop : do
read(*, *, iostat=ios) request
if (ios /= 0) exit mainloop
write(*, '(A, 3F7.2)') 'searching for ', request
! Calculate record
ii = nint((request(1)-x_min)/x_step)
jj = nint((request(2)-y_min)/y_step)
kk = nint((request(3)-z_min)/z_step)
record = kk + jj * nz + ii * nz * ny + 1
read(u_in, rec=record, iostat=ios) pos, B
if (ios /= 0) then
print *, 'failure to read'
cycle mainloop
end if
write(*, '(2(A, 3F7.2))') "found pos: ", pos, " Bx, By, Bz: ", B
end do mainloop
close(u_in)
end program read_txt
Note that the unformatted is not compiler- and system independent. A file created on one computer or with a program compiled by one compiler might not be able to be read with another program or on another computer.
But if you have control over it, it might be a useful way to speed things up.
PS: I left the x, y, and z coordinates in the file so that you can check whether the values are actually what you wanted. Always good to verify these things.
Related
My goal is to find a way to call Fortran subroutine in Excel VBA (can be found on Prof Alan Genz. The program is MVNPACK) to compute the CDF of multivariate normal distribution. Ideally, I would like to be able to use a version of DLL compiled from that source code in a C# project as well in the future. However, I am not sure how to troubleshoot and proceed further. I typically code in Python, have some exposure in C, Java, etc., but never use Fortran and not too familiar with what's going on when one calls a function in a DLL. To the best of my knowledge, this computation is not that widely available, and compiling the Fortran source code is my best bet.
I have been closely following the example here about creating the DLL, and here about using that in Excel VBA, and been trying to mimic the result. Starting from the MVNPACK source code mentioned above, I figured that what I need is to pass the inputs to the subroutine MVNDST, and get the result back by passing the pointers as arguments to the subroutine. So the first thing I did was trying to modify the code based on what the examples did. My modified version MVNDSTC looks like this.
SUBROUTINE MVNDSTC( N, LOWERC, UPPERC, INFINC, CORRELC, MAXPTS,
& ABSEPS, RELEPS, ERRORC, VALUEC, INFORMC)
& bind(c)
use ISO_C_BINDING
implicit none
cGCC$ ATTRIBUTES STDCALL, DLLEXPORT :: MVNDSTC
EXTERNAL MVNDFN
integer(kind=c_long), value:: N, MAXPTS
real(kind=c_double), value:: ABSEPS, RELEPS
type(c_ptr), value:: LOWERC, UPPERC, INFINC, CORRELC
type(c_ptr), value:: ERRORC, VALUEC, INFORMC
real(kind=c_double), dimension(:), pointer:: LOWER, UPPER, CORREL
integer(kind=c_long), dimension(:), pointer:: INFIN
real(kind=c_double), dimension(:), pointer:: ERROR_OUT, VALUE_OUT
integer(kind=c_int), dimension(:), pointer:: INFORM_OUT
INTEGER NN
INTEGER INFORM, INFIS, IVLS
DOUBLE PRECISION ERROR, VALUE, E, D, MVNDNT, MVNDFN
COMMON /DKBLCK/IVLS
NN = (N - 1) * N / 2
call C_F_POINTER(LOWERC, LOWER, [N])
call C_F_POINTER(UPPERC, UPPER, [N])
call C_F_POINTER(INFINC, INFIN, [N])
call C_F_POINTER(CORRELC, CORREL, [NN])
call C_F_POINTER(ERRORC, ERROR_OUT, [1])
call C_F_POINTER(VALUEC, VALUE_OUT, [1])
call C_F_POINTER(INFORMC, INFORM_OUT, [1])
IF ( N .GT. 500 .OR. N .LT. 1 ) THEN
INFORM = 2
VALUE = 0
ERROR = 1
ELSE
INFORM = MVNDNT(N, CORREL, LOWER, UPPER, INFIN, INFIS, D, E)
IF ( N-INFIS .EQ. 0 ) THEN
VALUE = 1
ERROR = 0
ELSE IF ( N-INFIS .EQ. 1 ) THEN
VALUE = E - D
ERROR = 2D-16
ELSE
*
* Call the lattice rule integration subroutine
*
IVLS = 0
CALL DKBVRC( N-INFIS-1, IVLS, MAXPTS, MVNDFN,
& ABSEPS, RELEPS, ERROR, VALUE, INFORM )
ENDIF
ENDIF
VALUE_OUT(0) = VALUE
ERROR_OUT(0) = ERROR
INFORM_OUT(0) = INFORM
END
Then I created a small subroutine with the mvndstc declaration on top. The VBA code is as follows.
Private Declare PtrSafe Sub mvndstc Lib "C:\Users\poopa\Desktop\mvn\mvn_project\fortran-library.dll" _
(ByVal N As Integer, _
ByRef LOWER As Single, _
ByRef UPPER As Single, _
ByRef INFIN As Single, _
ByRef CORREL As Single, _
ByVal MAXPTS As Integer, _
ByVal ABSEPS As Double, _
ByVal RELEPS As Double, _
ByRef ERROR As Single, _
ByRef VALUE As Single, _
ByRef INFORM As Single)
Sub mvn_test()
Dim value_1(1 To 1) As Single ' Result of the function
Dim inform_1(1 To 1) As Single ' Information
Dim error_1(1 To 1) As Single ' Error estimate
Dim upper_1() As Single
Dim lower_1() As Single
Dim infin_1() As Single
Dim correl_1() As Single
Dim n_1 As Long, n_1_2 As Long, max_pts_1 As Long
n_1 = 5
ReDim lower_1(1 To n_1)
ReDim upper_1(1 To n_1)
ReDim infin_1(1 To n_1)
lower_1(1) = 0#
lower_1(2) = 0#
lower_1(3) = 1.7817
lower_1(4) = 0.14755
lower_1(5) = 0#
upper_1(1) = 0#
upper_1(2) = 1.5198
upper_1(3) = 0#
upper_1(4) = 0#
upper_1(5) = 1.5949
infin_1(1) = 1
infin_1(2) = 2
infin_1(3) = 1
infin_1(4) = 1
infin_1(5) = 0
n_1_2 = Int(n_1 / 2 * (n_1 - 1))
ReDim correl_1(1 To n_1_2)
correl_1(1) = -0.707107 ' 12
correl_1(2) = 0# ' 13
correl_1(3) = 0.5 ' 14
correl_1(4) = 0# ' 15
correl_1(5) = 0.5 ' 23
correl_1(6) = 0.5 ' 24
correl_1(7) = 0# ' 25
correl_1(8) = 0.5 ' 34
correl_1(9) = 0.5 ' 35
correl_1(10) = 0.5 ' 45
max_pts_1 = 625000
mvndstc n_1, lower_1(1), upper_1(1), infin_1(1), correl_1(1), max_pts_1, 0.00005, 0, error_1(1), value_1(1), inform_1(1)
Debug.Print "Value = " & (value_1(1))
Debug.Print "Error Est = " & (error_1(1))
Debug.Print "Inform = " & inform_1(1)
End Sub
Now my first attempt I did not modify ERROR, VALUE, INFORM parameters at all and simply declare then in Fortran as theirs respective primitive types. I can actually run the VBA subroutine, but I got all zeros for the results. So I was speculating that the program runs but perhaps I didn't get the result back properly and I should treat these three outputs as pointers with size of 1. That way I just keep whatever procedure exactly the same in Fortran and then if I put VALUE_OUT(0) = VALUE and so on, before the function ends I should be get the results just fine. Right now using the code I posted here, I can actually see the results printed out in VBA, still all zeros, but right after that Excel would immediately crash.
So I want to ask how do I proceed from here? What did I got wrong here? Is there any resource worth looking into?
Thanks in advance.
I fixed this yesterday, the problem is indeed about the data types. When I read the tutorial I was assuming Single is some kind of object type in VBA. Little did I know that Single is actually the single precision of Double! I tried debugging all this by letting the DLL print out all the values inside the function to a file.
In Lua, Is there any good way to read Only numbers from input like "[1,2,3,4]" if i know the number of the numbers
I did io.read("*n") for each, but that just returns nil values
local num = io.read()
for i = 1, num do
print(io.read("*n"))
end
for that code inputs are
4
[1,1,15,54]
outputs are
nil
nil
nil
nil
I expect
1
1
15
54
Should I just use string.find and sub with ',' ?
If you're sure that there are no spaces in the input, you can use this code:
local num = io.read()
for i = 1, num do
io.read(1)
print(io.read("*n"))
end
The first io.read(1) reads [, the next ones read the commas. The closing ] is left unread.
Reading the whole line and parsing it is more robust:
local num = io.read()
local lin = io.read()
local i=0
for w in lin:gmatch("%d+") do
i=i+1
print(i,w)
end
This question already has answers here:
Fortran function in a module not found by subroutine in the same module [duplicate]
(1 answer)
Why is this a function declared inside the module and then used somewhere else in the same module not seen by the linker?
(2 answers)
Closed 4 years ago.
I am trying to get familiarize with modules in fortran and for that I have created a simple code with module. The source code is attached.
MAIN PROGRAM
program main_prog
use mod_f_g_h
!
implicit none
! f,g,h,s are defined in mod_f_g_h
real :: x,y
!
! read x,y
print *, "enter x and y "
read*, x, y
!
! check
if( y == 0 ) then
print*, "y must be non-zero"
stop
end if
!
! print on screen
print*, ' x = ', x
print*, ' y = ', y
print*, 'x + y = ', f(x,y)
print*, 'x * y = ', g(x,y)
print*, 'x * x = ', s(x)
print*, 'y * y = ', s(y)
print*, 'x / y = ', h(x,y)
end program main_prog
MODULE
module mod_f_g_h
!
! use other modules
implicit none
!
contains
!
real function s(x)
implicit none
real :: x,g
s = g(x,x)
end function s
!
real function f(x,y)
implicit none
real :: x,y
f = x+y
end function f
!
real function g(x,y)
implicit none
real :: x,y
g = x*y
end function g
!
real function h(x,y)
implicit none
real :: x,y
h = x/y
end function h
end module mod_f_g_h
But when I try to compile and link the code, using
gfortran -c mod_f_g_h.f90
gfortran -c main.f90
gfortran *.o -o run.x
Got the following error (in last step)
mod_f_g_h.o: In function `__mod_f_g_h_MOD_s':
mod_f_g_h.f90:(.text+0xb6): undefined reference to `g_'
collect2: ld returned 1 exit status
I think, I have defined all the variables, moreover I am not linking any external library, so I don't know why this error is appearing? Any comment/idea?
It should read
real function s(x)
implicit none
real :: x
s = g(x,x)
end function s
inside the module.
Further explanation by VladimirF:
By real g you were declaring the function g inside s as an external real function distinct from that one in the module.
I have this code which shifts the alphabet by a certain amount. The size of the alphabet is 26. When I enter a larger size shift (for example 22) I get some weird characters displaying. I think I need to mod the ASCII alphabet to 26 to get it working but Im not quite sure which bit to mod.
Basically I need to wrap around the alphabet (once it reaches Z it goes back to letter A) Do I have to create a dictionary for the mod to work (like A = 0... Z = 26) or can I stick with using the normal ASCII table? Here is the code below:
Public Function encrypt(ByVal input As String) 'input is a variable within the funcion
Dim n as Integer
Dim i As Integer
n = key.Text Mod 26 'gets what is in the text box of 'key' and sets it as n
' the key is a multiple of 26 so 26 will = 0
'need to remove white spaces
While input.Contains(" ") 'when the input text contains a space
input = input.Replace(" ", "") 'replaces it with no space.
End While
For i = 1 To Len(input) 'find the length of the input
Mid(input, i, 1) = Chr(Asc(Mid(input, i, 1)) + n) 'chr returns the character associated with the specified character code
'
Next
encrypt = input
End Function
Look at this code:
For i = 1 To Len(input) 'find the length of the input
Mid(input, i, 1) = Chr(Asc(Mid(input, i, 1)) + n) 'chr returns the character associated with the specified character code
'
Next
String indexes are 0-based. Your first index is 0, not 1! Also, you are assigning to the result of a function call. You need to instead construct a new string.
You didn't say, but the way you used the Replace and Contains methods indicates .Net, and if that's the case, I would do it like this:
Public Function encrypt(ByVal key As Integer, ByVal input As String) As String 'Don't forget the return type on the function
key = key Mod 26
Return New String(input.Replace(" ", "").ToUpper().Select(Function(c) Chr(((Asc(c) + key - Asc("A"c)) Mod 26) + Asc("A"c))).ToArray())
End Function
Just like that, and it's almost a one-liner. I can see this works now by calling it this way:
Encrypt("C"c, "the quick brown fox jumps over the lazy dog")
Encrypt("D"c, "the quick brown fox jumps over the lazy dog")
The results:
BPMYCQKSJZWEVNWFRCUXMLWDMZBPMTIHGLWOA
CQNZDRLTKAXFWOXGSDVYNMXENACQNUJIHMXPB
Look for the results mapped for the word "lazy", and you will see that the 'a' wraps to 'z' and 'y' correctly, and that the 'D' key results are one letter off of the 'C' results.
First of all, I have been using this site as a reference through the entire scripting process and it has been wonderful. I appreciate how useful and knowledgeable everyone is here. With that in mind, I have a question regarding matching (pattern matching) in Lua. I am writing a script that essentially takes input from a file and imports it into a table. I am checking for specific MAC addresses in the file as the host I am querying.
if macFile then
local file = io.open(macFile)
if file then
for line in file:lines() do
local f = line
i, j = string.find ( f, "%x+" )
m = string.sub(f, i, j)
table.insert( macTable, m )
end
file:close()
end
This parses the file into a format I will use to query later. Once the table is built, I run a pattern matching sequence to try and match the MAC from the table by iterating the table and matching the pattern against the current iteration:
local output = {}
t = "00:00:00:00:00:00"
s = string.gsub( t, ":", "")
for key,value in next,macTable,nil do
a, p = string.find ( s, value )
matchFound = string.sub(s, a, p)
table.insert( output, matchFound )
end
This doesn't return any output although when I enter it line by line in a Lua prompt, it seems to work. The variables are being passed correctly I believe. Any suggestions?
If your macFile uses a structure like this:
012345678900
008967452301
000000000000
ffffffffffff
The following script should work:
macFile = "./macFile.txt"
macTable = {}
if macFile then
local hFile = io.open(macFile, "r")
if hFile then
for line in hFile:lines() do
local _,_, sMac = line:find("^(%x+)")
if sMac then
print("Mac address matched: "..sMac)
table.insert(macTable, sMac)
end
end
hFile:close()
end
end
local output = {}
t = "00:00:00:00:00:00"
s = string.gsub( t, ":", "")
for k,v in ipairs(macTable) do
if s == v then
print("Matched macTable address: "..v)
table.insert(output, v)
end
end
I am just leaving from work and can't have a deeper look at your problem right now, but the next two lines seem quite odd.
t = "00:00:00:00:00:00"
s = string.gsub( t, ":", "")
Basically you are removing all the ':' characters in the string t. Afterwards you end up with s being "000000000000". This is probably not what you want?