Veryreverie suggested an answer (https://stackoverflow.com/a/73366165/7462275) to merge two programs (a 2D and a 3D ones). The solution is to have only a 3D program and to set the third dimension to 1 for the the 2D case. I did not imagined this solution because I thought that there would be overhead by adding an useful third dimension. (Program is written in Fortran)
Then, I did many tests to compare 2D and 3D with third dimension = 1.
For example :
module m
implicit none
type :: int_3dim_t
integer :: x
integer :: y
integer :: z
end type int_3dim_t
end module m
program main
use m
implicit none
type(int_3dim_t) :: ii_curr, full_dim
real, dimension(:,:,:,:), allocatable :: myMatrix
real :: aaa
integer :: ll
open(unit=50,file='dim2.dat',status='old')
read(50,*) full_dim%x,full_dim%y
close(50)
full_dim%z = 1
allocate(myMatrix(1:8,1:full_dim%x,1:full_dim%y,1:full_dim%z))
associate (ii_x => ii_curr%x, ii_y => ii_curr%y, ii_z => ii_curr%z)
do ii_z = 1, full_dim%z
do ii_y = 1, full_dim%y
do ii_x = 1, full_dim%x
do ll=1,8
myMatrix(ll,ii_curr%x,ii_curr%y,ii_curr%z) = myMatrix(ll,ii_curr%x,ii_curr%y,ii_curr%z) + my_calc(ii_curr,ll)
enddo
enddo
enddo
enddo
end associate
print *,myMatrix(:,10,10,1)
contains
real function my_calc(ii,ll)
type(int_3dim_t), intent(in) :: ii
integer, intent(in) :: ll
my_calc = sqrt(real((ii%x/100.0)**2+(ii%y/100.0)**2))*ll
end function my_calc
end program main
The input file is 10000 10000 (another values do not change the conclusion), gfortran version is 12.1.1 and O2 optimisation flag is used.
(For 2D version, everything concerning the third dimension is removed)
And surprisingly, I did not found any difference in execution time.
So, I am looking for simple examples where this third dimension (set to 1) has an effect on the execution time. (Of course, with the correct fortran loop order : first indice in the innner loop and last indice in the outer loop)
I've written a Fortran function that calculates the moving average of a 1D array of numbers in a very straightforward way:
function moving_average(data, w)
implicit none
integer, intent(in) :: w
real(8), intent(in) :: data(:)
integer :: n, i
real(8) :: moving_average(size(data)-w+1)
n = w-1
do i=1, size(data)-n
moving_average(i) = mean(data(i:i+n))
end do
end function
Where the function mean is defined as:
real(8) function mean(data)
implicit none
real(8), dimension(:), intent(in) :: data
mean = sum(data)/size(data)
end function
When running the function moving_average on my laptop with a data set of 100000 numbers and a window width of 1000, it takes 0.1 seconds. However, the function running_mean in this post using numpy takes only 1 ms. Why is my algorithm so much slower?
Your algorithm is of the order O(n*m) with n the size of the moving average and m the size of the array.
Every time you compute a point in the array moving_average, you do the following steps:
extract a part of the array
compute the sum over that slice
divide by the constant n
However, moving_average(i) and moving_average(i+1) are related in the following way:
moving_average(i+i) = moving_average(i) + (data(i+n) - data(i-1))/n
When you use this, you can reduce the computational time from O(n*m) to O(m)
I'm having trouble when I pass a portion of an array of a derived type which has an allocatable component. A (probable) memory leaks seems to occur (unfortunatly valgrind is not yet available on mac 10.14/mojave, so I simply used a long do loop and looked the memory used with top).
Consider the following test program which reproduces my problem in a simpler context:
program test_leak
implicit none
type :: vint_t
integer, allocatable :: col(:)
end type vint_t
type(vint_t) :: row(4)
integer :: i, Ind(2)=[2,3]
do i = 1, 4
allocate(row(i)%col(i))
end do
row(1)%col(1) = 11
row(2)%col(1) = 21 ; row(2)%col(2) = 22
row(3)%col(1) = 31 ; row(3)%col(2) = 32 ; row(3)%col(3) = 33
row(4)%col(1) = 41 ; row(4)%col(2) = 42 ; row(4)%col(3) = 43 ; row(4)%col(4) = 44
do i = 1, 1000000
call do_nothing ( row(Ind) ) ! (version #A) with this version and with gfortran: memory grows with iter
!call do_nothing ( row(2:3) ) ! (version #B) but not with this version
if (mod(i,10000) == 0) then
print*,i ; read*
end if
end do
contains
subroutine do_nothing ( a )
type(vint_t), intent(in) :: a(:)
end subroutine do_nothing
end program test_leak
The problem occurs with gfortran (8.2) and not with ifort (19) nor with nagfor (6.2) (with nagfor and ifort the used memory is stable during the iterations and ca 400 Kb, while it reaches ca 30 Mb with gfortran!).
A final procedure doesn't solve the problem.
The problem disapears if the allocatable member "col" is replaced by an explicit array (say col(4)) or if the version #B is used.
The problem also occurs with an allocatable character component.
If I print "a" in the subroutine "do_nothing" both versions give the correct result.
Does anyone have some insight on this issue? Of course I can design the called subroutines by passing the array Ind (do_nothing(row, Ind)) but that would be somewhat a pity.
I have been banging my head against a wall all afternoon trying to figure out this problem, so I hope someone can help me.
I have an abstract base class, called base_model (say), which in Fortran2003 looks like:
type, abstract :: base_model
contains
procedure(initMe), pass(this), deferred :: init ! constructor
procedure(delMe), pass(this), deferred :: delete ! destructor
procedure(solveMe), pass(this), deferred :: solve
end type base_model
where, obviously, the abstract procedures initMe, delMe and solveMe are defined using an abstract interface block. I then have three derived classes, called model1, model2 and model3 (say):
type, extends(base_model) :: model1
double precision :: x,y
contains
procedure :: init => init_model1
procedure :: delete => delete_model1
procedure :: solve => solve_model1
end type model1
type, extends(base_model) :: model2
contains
procedure :: init => init_model2
procedure :: delete => delete_model2
procedure :: solve => solve_model2
end type model2
type, extends(base_model) :: model3
contains
procedure :: init => init_model3
procedure :: delete => delete_model3
procedure :: solve => solve_model3
end type model3
I then have a "controlling" object, called control (say), which extends an abstract base_control:
type, abstract :: base_control
class(base_model), allocatable :: m1
class(base_model), allocatable :: m2
class(base_model), allocatable :: m3
contains
procedure(initMe), pass(self), deferred :: init
procedure(delMe), pass(self), deferred :: delete
procedure(runMe), pass(self), deferred :: run
end type base_control
type, extends(base_control) :: control
contains
procedure :: init => init_control
procedure :: delete => delete_control
procedure :: run => run_control
end type control
The objects m1, m2 and m3 can be allocated into any of the models: model1, model2 or model3, and are "solved" in any particular order depending on which "control" is requested by the user.
The three allocatable objects (m1, m2 and m3) need to pass data between them. Given that they are members of a "controlling" object, I can define a "getter" for each model which then passes the required data into each model. However, the specific models are not known at compile time and therefore, the "control" object does not know what data to get and indeed, the models don't know what data to receive!
For example, if I allocate(model1::m1) (that is, allocate m1 to be of type model1) then it will contain two bits of data double precision :: x,y. Then if m2 is allocated to be of type model2 (allocate(model2::m2)), it could require x but if it is allocated to be of type model3 (allocate(model3::m2)) then it may require y from m1. Therefore, given that the "controlling" object cannot know what type m2 is allocated to be, how can it get the necessary data from m1 to pass into m2?
An additional complication is that the interactions between the models are in general, circular. That is, m1 requires data from m2, m2 requires data from m1 and so-on. Also, the data required is in general not only specific to the models but also variable in both type and quantity.
Unfortunately, the data x and y are not members of base_model and therefore, passing m1 into m2 as an argument would not work either.
So I have the following questions:
Is there a better way to design these objects so that I can pass data between them easily? Looking around on here, there have been some suggestions that the best thing to do is re-design the objects so that the interaction between them is not circular. However, that is kind of necessary here!
Do I have to write a "getter" for each piece of data that might be shared between the objects? That seems like a lot of coding (I have a lot of data which might be shared). However, that also seems rather complicated because the "getter" (specific to a piece of data) would also have to satisfy an abstract interface.
In higher level languages like Python, this would be easy as we could simply create a new data type as a composite of the models but that's not possible, as far as I am aware, in Fortran.
Thanks in advance. Any help is greatly appreciated.
Edit: Following the discussion with francescalus below, select type is an option. Indeed, in the simple example given above, select type would be a good choice. However, in my actual code, this would result in large, nested select types and so if there is a way to do it without using select type I would prefer it. Thanks to francescalus for pointing out my error regarding select type.
To answer your two questions:
Is there a better way to design?
I don't quite know why you have so many contraints in your design, but in short yes. You can use a context manager for your models. I recommend you check out this answer: Context class pattern
Do you have to write a getter method on every model?
Not quite, if you use a context strategy on this specific issue, the only thing you need to implement on every model is a setter method that will share the data between models.
I implemented a working solution for this scenario in Python. Code speaks louder than words. I've avoided using any special features of Python to provide you a clear understanding of how to use a context in this case.
from abc import ABC, abstractmethod
import random
class BaseModel(ABC):
def __init__(self, ctx):
super().__init__()
self.ctx = ctx
print("BaseModel initializing with context id:", ctx.getId())
#abstractmethod
def solveMe():
pass
class BaseControl(object):
# m1 - m3 could be replaced here with *args
def __init__(self, m1, m2, m3):
super().__init__()
self.models = [m1, m2, m3]
class Control(BaseControl):
def __init__(self, m1, m2, m3):
super().__init__(m1, m2, m3)
def run(self):
print("Now Solving..")
for m in self.models:
print("Class: {} reports value: {}".format(type(m).__name__, m.solveMe()))
class Model1(BaseModel):
def __init__(self, x, y, ctx):
super().__init__(ctx)
self.x = x
self.y = y
ctx.setVal("x", x)
ctx.setVal("y", y)
def solveMe(self):
return self.x * self.y
class Model2(BaseModel):
def __init__(self, z, ctx):
super().__init__(ctx)
self.z = z
ctx.setVal("z", z)
def solveMe(self):
return self.z * self.ctx.getVal("x")
class Model3(BaseModel):
def __init__(self, z, ctx):
super().__init__(ctx)
self.z = z
ctx.setVal("z", z)
def solveMe(self):
return self.z * self.ctx.getVal("y")
class Context(object):
def __init__(self):
self.modelData = {}
self.ctxId = random.getrandbits(32)
def getVal(self, key):
return self.modelData[key]
def setVal(self, key, val):
self.modelData[key] = val
def getId(self):
return self.ctxId
ctx = Context()
m1 = Model1(1,2, ctx)
m2 = Model2(4, ctx)
m3 = Model3(6, ctx)
# note that the order in the arguments to control defines behavior
control = Control(m1, m2, m3)
control.run()
Output
python context.py
BaseModel initializing with context id: 1236512420
BaseModel initializing with context id: 1236512420
BaseModel initializing with context id: 1236512420
Now Solving..
Class: Model1 reports value: 2
Class: Model2 reports value: 4
Class: Model3 reports value: 12
Explanation
In short, we create a context class which has a dictionary that can be shared across the different models. This implementation is very specific to the primitive data types you provided (I.E. x, y, z). If you need to compute the data before it is shared across the models you can still use this pattern by replacing the return of solveMe() with a deferred promise.
FWIW, below is a similar attempt to access fields of another objects based on key/values (*). For simplicity, the main program gets one integer from a child1_t object and sets a complex value to a child2_t object (both of which are extended types of parent_t).
parent.f90:
module parent_m
implicit none
type, abstract :: parent_t !(abstract is optional)
contains
procedure :: set
procedure :: get
procedure :: show
endtype
type composite_t
class(parent_t), allocatable :: pa, pb
endtype
contains
subroutine set( this, key, val ) ! key-based setter
class(parent_t), intent(inout) :: this
character(*), intent(in) :: key
integer, intent(in) :: val
endsubroutine
subroutine get( this, key, val ) ! key-based getter
class(parent_t), intent(in) :: this
character(*), intent(in) :: key
integer, intent(out) :: val
endsubroutine
subroutine show( this ) ! print contents
class(parent_t), intent(in) :: this
endsubroutine
end module
child.f90:
module child_m
use parent_m, only: parent_t
implicit none
type, extends(parent_t) :: child1_t
integer :: n1 = 777 ! some property
contains
procedure :: get => child1_get
procedure :: show => child1_show
endtype
type, extends(parent_t) :: child2_t
complex :: c2 = ( 0.0, 0.0 ) ! another property
contains
procedure :: set => child2_set
procedure :: show => child2_show
endtype
contains
subroutine child1_get( this, key, val )
class(child1_t), intent(in) :: this
character(*), intent(in) :: key
integer, intent(out) :: val
select case( key )
case ( "num", "n1" ) ; val = this % n1 ! get n1
case default ; stop "invalid key"
end select
end subroutine
subroutine child1_show( this )
class(child1_t), intent(in) :: this
print *, "[child1] ", this % n1
endsubroutine
subroutine child2_set( this, key, val )
class(child2_t), intent(inout) :: this
character(*), intent(in) :: key
integer, intent(in) :: val
select case( key )
case ( "coeff", "c2" ) ; this % c2 = cmplx( real(val), 0.0 ) ! set c2
case default ; stop "invalid key"
end select
end subroutine
subroutine child2_show( this )
class(child2_t), intent(in) :: this
print *, "[child2] ", this % c2
endsubroutine
end module
main.f90:
program main
use parent_m, only: composite_t
use child_m, only: child1_t, child2_t
implicit none
type(composite_t) :: c
integer itmp
allocate( child1_t :: c % pa )
allocate( child2_t :: c % pb )
print *, "initial state:"
call c % pa % show()
call c % pb % show()
call c % pa % get( "num", itmp ) ! get an integer value from pa
call c % pb % set( "coeff", itmp ) ! set a complex value to pb
print *, "modified state:"
call c % pa % show()
call c % pb % show()
end
Compile & results:
$ gfortran parent.f90 child.f90 main.f90
initial state:
[child1] 777
[child2] (0.00000000,0.00000000)
modified state:
[child1] 777
[child2] (777.000000,0.00000000)
Though the above code deals with "data transfer" for only integer and complex, other kind of data may be added similaly in the select case construct (without adding new getter/setter methods for each). Further, if necessary, we could overload get() and set() for different types of value (e.g. set_int() and set_real()) via generic keyword (in the parent_t type), and override them in the extended types. The same also holds for value of array type, maybe...
If the transfer (copy) of data between extended types are expensive (e.g., large arrays), I guess the getter could return pointers to them so that childX_t's can communicate in a tightly coupled way (without knowing their implementation).
(But I guess there may be a simpler approach than doing a thing like the above, including Point 1 in the question (e.g., re-design of the program itself). Also, if we have some dictionary type in hand, the use of dictionaries (as in the other answer) seems more appealing to me.)
I'm totally new to Fortran, and I'm trying to learn the language here:
http://www.fortrantutorial.com/files-precision/index.php. I have some basic experience with C and Python, but not much, like introduction class and such.
So in exercises 4.1, they ask me to input some numbers from a file, and check if these numbers are even or odd. Here is the code to input the numbers:
program readdata
implicit none
!reads data from a file called mydata.txt
real :: x,y,z
open(10,file='mydata.txt')
read(10,*) x,y,z
print *,x,y,z
end program readdata
The file mydata.txt contains some random numbers. And they can check if the number is even or odd by:
if (mod(num,2)>0) then……
My question is that: if this file have like 10, or 1000 numbers, do I have to manually assign every single one of them? Is there any other way for me to do quick calculation with mass numbers situation like that?
Every read also moves the read pointer forwards. So with every new read, a new line is read in from the file.
The easiest thing to do is to keep reading until the READ statement returns an error. Of course, you have to pass a variable for the READ to write its error into. Something like this:
program readdata
implicit none
real :: x, y, z
integer :: iounit, ios
open(newunit=iounit, file='mydata.txt', iostat=ios, action='READ')
if (ios /= 0) STOP 1
do
read(iounit, *, iostat=ios) x, y, z
if (ios /= 0) exit
print *, x, y, z
end do
close(iounit)
end program readdata
Update: If you're limited to Fortran 95, as OP suggested in his comments, here's what to change: Instead of
integer :: iounit, ios
open(newunit=iounit, ...)
you use
integer :: ios
integer, paramter :: iounit = 100
open(unit=iounit, ...)
All that's important is that iounit is a number, greater than 10, which is not used as a unit for any other read/write operation.
well a mass of numbers means that X, Y, and Z need to be more than a single number of each... so something like this should get you vectored towards a solution:
program readdata
implicit none
!reads data from a file called mydata.txt
INTEGER, PARAMETER :: max2process = 10000
INTEGER :: I
INTEGER :: K = 0
real , DIMENSION(max2process) :: x,y,z
LOGICAL, DIMENSION(max2process) :: ODD = .FALSE.
open(10,file='mydata.txt')
10 CONTINUE
X(:) = 0
Y(:) = 0
K(:) = 0
DO I = 1, max2process
K = K + I
read(10,*, EOF=99) x(I), y(I), z(I)
print *,x(I), y(I), z(I)
ENDDO
WRITE(*,22) I, MINVAL(X(1:I)), MAXVAL(X(1:I)
22 FORMAT(' MIN(X(1:',I5,')=',0PE12.5,' Max=',0PE12.5)
odd = .FALSE.
WHERE (MODULO(X, 2) /= 0)
ODD = .TRUE.
ENDWHERE
DO I = 1, 10
WRITE(*,88) I, X(I), odd(I)
88 FORMAT('x(', I2,')=',0PE12.5,' odd="',L1,'"')
EBDDO
WRITE(*,*) ' we did not hit the end of file... Go back and read some more'
GOTO 10
99 WRITE(*,*) 'END of file reached at ', (K-1)
end program readdata
Basically something like that, but I probably have a typo (LTR). If MODULO is ELEMENTAL then you can do it easily... And MODULO is ELEMENTAL, so you are in luck (which is common).
There is a https://rosettacode.org/wiki.Even_or_Odd#Fortran that looks nice. the functions could be further enhanced using ELEMENTAL FUNCTION.
that should give you enough to work it out.
or you could read the file to find the size, close and reopen it, and then allocate X, Y, Z and Odd... the example i gave was more of a streaming example. one should generally use a DO WHILE rather than a GOTO, but starting out a GOTO can be conceptually easier. (You need an EXIT or some DO WHILE (IOSTAT /= ) in order to break out)