Matrix Inversion in CBLAS/LAPACK vs Python - numpy

The matrix I am trying to invert is:
[ 1 0 1]
A = [ 2 0 1]
[-1 1 1]
The true inverse is:
[-1 1 0]
A^-1 = [-3 2 1]
[ 2 -1 0]
Using Python's numpy.linalg.inv, I get the correct answer. One of my routines for matrix inverse uses dgetri_, it is:
void compute_matrix_inverse_dbl( double* matrix,
int order,
double * inverse )
{
int N, lwork;
int success;
int *pivot;
double* workspace;
//===Allocate Space===//
pivot = malloc(order * order * order * sizeof(*pivot));
workspace = malloc(order * order * sizeof(*workspace));
//===Run Setup===//
N = order;
copy_array_dbl(matrix, order*order, inverse);
lwork = order*order;
//===Factor Matrix===//
dgetrf_(&N,&N,inverse,&N,pivot,&success);
//===Compute Inverse===//
dgetri_(&N, inverse, &N, pivot, workspace, &lwork, &success);
//===Clean Up===//
free(workspace);
free(pivot);
return;
}
Using this routine, I get:
[-1 1 +-e1 ]
A^-1 = [-3 2 1 ]
[ 2 -1 +-e2 ]
Where e1 and e2 and small numbers on the order of machine precision 1e-16. Now perhaps dgetri_ is not the best to use. However, when I invert using QR decomposition via zgeqrf_ and zungqr_, I get a similar answer. When I use dgesvd_ for inverse using SVD, I get a similar answer as well.
Python seems to use a routine called _umath_linalg.inv. So I have a few questions:
What does that routine do?
What CBLAS/LAPACK routine can I use to invert this matrix and get a result like CBLAS/LAPACK (such that the e1 and e2 get replaced by proper zeros)?

It seems that numpy.linalg.inv is a lite version of the scipy.linalg.inv as per the description:
This module is a lite version of the linalg.py module in SciPy which
contains high-level Python interface to the LAPACK library.
Looking at scipy.linalg.inv, it does a call to getrf, then getri.

Related

Domain error when using Nelder Mead algorithm in Julia

I am struggling with optimization in Julia.
I used to use Matlab but I am trying to work on Julia instead.
The following is the code I wrote.
using Optim
V = fill(1.0, (18,14,5))
agrid = range(-2, stop=20, length=18)
dgrid = range(0.01, stop=24, length=14)
#zgrid = [0.5; 0.75; 1.0; 1.25; 1.5]
zgrid = [0.7739832502827438; 0.8797631785217791; 1.0; 1.1366695315439874; 1.2920176239404275]
# function
function adj_utility(V,s_a,s_d,s_z,i_z,c_a,c_d)
consumption = s_z + 1.0125*s_a + (1-0.018)*s_d - c_a - c_d - 0.05*(1-0.018)*s_d
if consumption >= 0
return (1/(1-2)) * (( (consumption^0.88) * (c_d^(1-0.88)) )^(1-2))
end
if consumption < 0
return -99999999
end
end
# Optimization
i_a = 1
i_d = 3
i_z = 1
utility_adj(x) = -adj_utility(V,agrid[i_a],dgrid[i_d],zgrid[i_z],i_z,x[1],x[2])
result1 = optimize(utility_adj, [1.0, 1.0], NelderMead())
If I use zgrid = [0.5; 0.75; 1.0; 1.25; 1.5], then the code works.
However, if I use zgrid = [0.7739832502827438; 0.8797631785217791; 1.0; 1.1366695315439874; 1.2920176239404275], I got an error message "DomainError with -0.3781249999999996"
In the function, if the consumption is less than 0 then the value should be -9999999 so I am not sure why I am getting this message.
Any help would be appreciated.
Thank you.
Raising negative numbers to non-integer powers returns complex numbers, which is where your error is coming from.
julia> (-0.37)^(1-0.88)
ERROR: DomainError with -0.37:
Exponentiation yielding a complex result requires a complex argument.
Replace x^y with (x+0im)^y, Complex(x)^y, or similar.
Stacktrace:
[1] throw_exp_domainerror(::Float64) at ./math.jl:37
[2] ^(::Float64, ::Float64) at ./math.jl:888
[3] top-level scope at REPL[5]:1
You have a constraint that consumption must be strictly positive, but if you want consumption to be a real number you will need constraints that c_d is positive as well. You can either add this directly to your objective function as above, or you can use one of the constrained optimization algorithms in NLopt, which is available in Julia via the NLopt package.

calling a fortran dll from python using cffi with multidimensional arrays

I use a dll that contains differential equation solvers among other useful mathematical tools. Unfortunately, this dll is written in Fortran. My program is written in python 3.7 and I use spyder as an IDE.
I successfully called easy functions from the dll. However, I can't seem to get functions to work that require multidimensional arrays.
This is the online documentation to the function I am trying to call:
https://www.nag.co.uk/numeric/fl/nagdoc_fl26/html/f01/f01adf.html
The kernel dies without an error message if I execute the following code:
import numpy as np
import cffi as cf
ffi=cf.FFI()
lib=ffi.dlopen("C:\Windows\SysWOW64\DLL20DDS")
ffi.cdef("""void F01ADF (const int *n, double** a, const int *lda, int *ifail);""")
#Integer
nx = 4
n = ffi.new('const int*', nx)
lda = nx + 1
lda = ffi.new('const int*', lda)
ifail = 0
ifail = ffi.new('int*', ifail)
#matrix to be inversed
ax1 = np.array([5,7,6,5],dtype = float, order = 'F')
ax2 = np.array([7,10,8,7],dtype = float, order = 'F')
ax3 = np.array([6,8,10,9],dtype = float, order = 'F')
ax4 = np.array([5,7,9,10], dtype = float, order = 'F')
ax5 = np.array([0,0,0,0], dtype = float, order = 'F')
ax = (ax1,ax2,ax3,ax4,ax5)
#Array
zx = np.zeros(nx, dtype = float, order = 'F')
a = ffi.cast("double** ", zx.__array_interface__['data'][0])
for i in range(lda[0]):
a[i] = ffi.cast("double* ", ax[i].__array_interface__['data'][0])
lib.F01ADF(n, a, lda, ifail)
Since function with 1D arrays work I assume that the multidimensional array is the issues.
Any kind of help is greatly appreciated,
Thilo
Not having access to the dll you refer to complicates giving a definitive answer, however, the documentation of the dll and the provided Python script may be enough to diagnose the problem. There are at least two issues in your example:
The C header interface:
Your documentation link clearly states what the function's C header interface should look like. I'm not very well versed in C, Python's cffi or cdef, but the parameter declaration for a in your function interface seems wrong. The double** a (pointer to pointer to double) in your function interface should most likely be double a[] or double* a (pointer to double) as stated in the documentation.
Defining a 2d Numpy array with Fortran ordering:
Note that your Numpy arrays ax1..5 are one dimensional arrays, since the arrays only have one dimension order='F' and order='C' are equivalent in terms of memory layout and access. Thus, specifying order='F' here, probably does not have the intended effect (Fortran using column-major ordering for multi-dimensional arrays).
The variable ax is a tuple of Numpy arrays, not a 2d Numpy array, and will therefore have a very different representation in memory (which is of utmost importance when passing data to the Fortran dll) than a 2d array.
Towards a solution
My first step would be to correct the C header interface. Next, I would declare ax as a proper Numpy array with two dimensions, using Fortran ordering, and then cast it to the appropriate data type, as in this example:
#file: test.py
import numpy as np
import cffi as cf
ffi=cf.FFI()
lib=ffi.dlopen("./f01adf.dll")
ffi.cdef("""void f01adf_ (const int *n, double a[], const int *lda, int *ifail);""")
# integers
nx = 4
n = ffi.new('const int*', nx)
lda = nx + 1
lda = ffi.new('const int*', lda)
ifail = 0
ifail = ffi.new('int*', ifail)
# matrix to be inversed
ax = np.array([[5, 7, 6, 5],
[7, 10, 8, 7],
[6, 8, 10, 9],
[5, 7, 9, 10],
[0, 0, 0, 0]], dtype=float, order='F')
# operation on matrix using dll
print("BEFORE:")
print(ax.astype(int))
a = ffi.cast("double* ", ax.__array_interface__['data'][0])
lib.f01adf_(n, a, lda, ifail)
print("\nAFTER:")
print(ax.astype(int))
For testing purposes, consider the following Fortran subroutine that has the same interface as your actual dll as a substitute for your dll. It will simply add 10**(i-1) to the i'th column of input array a. This will allow checking that the interface between Python and Fortran works as intended, and that the intended elements of array a are operated on:
!file: f01adf.f90
Subroutine f01adf(n, a, lda, ifail)
Integer, Intent (In) :: n, lda
Integer, Intent (Inout) :: ifail
Real(Kind(1.d0)), Intent (Inout) :: a(lda,*)
Integer :: i
print *, "Fortran DLL says: Hello world!"
If ((n < 1) .or. (lda < n+1)) Then
! Input variables not conforming to requirements
ifail = 2
Else
! Input variables acceptable
ifail = 0
! add 10**(i-1) to the i'th column of 2d array 'a'
Do i = 1, n
a(:, i) = a(:, i) + 10**(i-1)
End Do
End If
End Subroutine
Compiling the Fortran code, and then running the suggested Python script, gives me the following output:
> gfortran -O3 -shared -fPIC -fcheck=all -Wall -Wextra -std=f2008 -o f01adf.dll f01adf.f90
> python test.py
BEFORE:
[[ 5 7 6 5]
[ 7 10 8 7]
[ 6 8 10 9]
[ 5 7 9 10]
[ 0 0 0 0]]
Fortran DLL says: Hello world!
AFTER:
[[ 6 17 106 1005]
[ 8 20 108 1007]
[ 7 18 110 1009]
[ 6 17 109 1010]
[ 1 10 100 1000]]

Quadratic Programming with a large number of variables using CVXOPT

I am new to CVXOPT. I have tried out the example quadratic program (with 2 variables) in CVXOPT documentation, and I am able to understand it. Now I need to solve a quadratic programming problem with a large number of variables (eg: 100 variables). How can I do this using CVXOPT?
The problem that I want to solve is shown below.
Minimize
Σ [ d(t) + x(t) ]²        ; t=1, ....., 100
such that,
0 <= x(t) <= 10
Σ x(t) = 600
Here, d(t) is known for t=(1, ...,100).
x(t) for t=(1, ...,100) are the decision variables.
 
Cheers !!! 
cvxpy may be a bit easier:
import numpy as np
import cvxpy as cvx
N = 100
d = np.random.uniform(-500,500,N)
x = cvx.Variable(N)
prob = cvx.Problem(cvx.Minimize(cvx.norm(x+d)), [x >= 0, x <= 10, sum(x) == 600])
prob.solve()
print(prob.status)
v = x.value
print(v)
This gives
optimal
[[ 3.65513295e-09]
[ 4.89791266e-09]
[ 3.05045765e-09]
[ 9.99999999e+00]
. . .
[ 1.00000000e+01]
[ 2.85640643e-09]
[ 5.42473434e-09]]
I don't need a solver for that:
0 <= x(t) <= 10
sum(x(t)) = 2000
with T=100 will be infeasible. With these bounds the sum will be between 0 and 1000.

EigenvalueDecomposition returns wrong eigenvectors

I'm using the apache.commons.math3 library to calculate eigenvectors of a 3x3 matrix, but the EigenDecomposition methods for calculating eigenvectors return wrong results: here's my code:
double[][] matrix = {
{1 ,3 ,2},
{1 ,4 ,3},
{2 ,1 ,0}
};
RealMatrix realMatrix = MatrixUtils.createRealMatrix(matrix);
EigenDecomposition decomposition = new EigenDecomposition(realMatrix);
for(int i = 0; i<3; i++){
RealVector eigenvector = decomposition.getEigenvector(i);
System.out.println(eigenvector.getEntry(0)+" "+eigenvector.getEntry(1)+" "+eigenvector.getEntry(2));
}
The printed results are:
-0.5760517243311052 -0.7536997812678066 -0.31638750072027233
0.22370947445236325 -0.6030287282098593 0.770086088626364
0.293925829450875 1.583437114738283 -2.642858652367182
while the correct ones should be
0.29050, -0.78307, 1
1.82072, 2.38220, 1
What is the problem? Is it a precision error? It seems to me impossible such a wrong result
If v is an eigenvector of a matrix, then a non-zero real multiple of v is also an eigenvector. The vector
(-0.5760517243311052 -0.7536997812678066 -0.31638750072027233)
is a multiple of
(1.82072, 2.38220, 1).
The difference is just that the first one has norm 1, whereas the second one has third component 1. Your library seems to choose the normalization by norm 1, which is better, since it is always possible.

computing cumulative distribution of a conditional probability distribution

I have a conditional probability of z for the given m, p(z|m), where the coefficients are chosen in order that integral over z in the limit of [0,1.5] and m in the range of [18:28] would be equal to one.
def p(z,m):
if (m<21.25):
E = { 'ft':0.55, 'alpha': 2.99, 'z0':0.191, 'km':0.089, 'kt':0.25 }
S = { 'ft':0.39, 'alpha': 2.15, 'z0':0.121, 'km':0.093, 'kt':-0.175 }
I={ 'ft':0.06, 'alpha': 1.77, 'z0':0.045, 'km':0.096, 'kt':-0.9196 }
Evalue=E['ft']*np.exp(-1*E['kt']*(m-18))*z**E['alpha']*np.exp(-1*(z/(E['z0']+E['km']*(m-18)))**E['alpha'])
Svalue=S['ft']*np.exp(-1*S['kt']*(m-18))*z**S['alpha']*np.exp(-1*(z/(S['z0']+S['km']*(m-18)))**S['alpha'])
Ivalue=I['ft']*np.exp(-1*I['kt']*(m-18))*z**I['alpha']*np.exp(-1*(z/(I['z0']+I['km']*(m-18)))**I['alpha'])
value=Evalue+Svalue+Ivalue
elif(m>=21.25):
E = { 'ft':0.25, 'alpha': 1.957, 'z0':0.321, 'km':0.196, 'kt':0.565 }
S = { 'ft':0.61, 'alpha': 1.598, 'z0':0.291, 'km':0.167, 'kt':0.155 }
I = { 'ft':0.14, 'alpha': 0.964, 'z0':0.170, 'km':0.129, 'kt':0.1759 }
Evalue=E['ft']*np.exp(-1*E['kt']*(m-18))*z**E['alpha']*np.exp(-1*(z/(E['z0']+E['km']*(m-18)))**E['alpha'])
Svalue=S['ft']*np.exp(-1*S['kt']*(m-18))*z**S['alpha']*np.exp(-1*(z/(S['z0']+S['km']*(m-18)))**S['alpha'])
Ivalue=I['ft']*np.exp(-1*I['kt']*(m-18))*z**I['alpha']*np.exp(-1*(z/(I['z0']+I['km']*(m-18)))**I['alpha'])
value=Evalue+Svalue+Ivalue
return value
I would like to draw a sample from this distribution, therefore I made a grid points in z and m plane to estimate the cumulative distribution, the cumulative integral over m reaches to one but the cumulative integral over z doesn't give me one in the edge. I don't know why it won't get converged to one?!!
grid_m = np.linspace(18, 28, 1000)
grid_z = np.linspace(0, 1.5, 1000)
dz = np.diff(grid_z[:2])
# get cdf on grid, use cumtrapz
prob_zgm=np.empty((grid_z.shape[0], grid_m.shape[0]),float)
for i in range(grid_z.shape[0]):
for j in range(grid_m.shape[0]):
prob_zgm[i,j]=p(grid_z[i],grid_m[j])
pr = np.column_stack((np.zeros(prob_zgm.shape[0]),prob_zgm))
dm = np.diff(grid_m[:2])
cdf_zgm = integrate.cumtrapz(pr, dx=dm, axis=1)
cdf = integrate.cumtrapz(pr, dx=dz, axis=0)
Which assumption might cause this inconsistency or I compute something wrongly?
Update: The cumulative distribution cdf_zgm is shown as
In the rest, in order to get the inverse of the probability, it is the approach I have used:
# fix bounds of cdf_zgm
cdf_zgm[:, 0] = 0
cdf_zgm[:, -1] = 1
#Interpolate the data using a linear spline to "grid_q" samples
grid_q = np.linspace(0, 1, 200)
grid_qm = np.empty((len(grid_m), len(grid_q)), float)
for i in range(len(grid_m)):
grid_qm[i] = interpolate.interp1d(cdf_zgm[i], grid_z)(grid_q)
# build 2d interpolation for z as function of (q,m)
z_interp = interpolate.interp2d(grid_q, grid_m, grid_qm)
#sample magnitude
ng=20000
r = dist_m.rvs(ng)
rvs_u = np.random.rand(ng)
rvs_z = np.asarray([z_interp(rvs_u[i], r[i]) for i in range(len(rvs_u))]).ravel()
Is it right approach to fix the boundaries of CDF to one?
I don't know what's wrong with that code. But here are a couple of different ideas to try:
(1) Just sum the array elements instead of trying to compute the numerical integrals. It is simpler that way. (Summing the array elements is essentially computing a rectangle rule approximation, which as it turns out, is actually more accurate than the trapezoidal rule.)
(2) Instead of trying to create a whole 2-d array at once, write a function which creates just a 1-d slice of p(z | m) for a given value of m. Then just sum those elements to get the cumulative probability.