I'm new here and I don't really know how to precisely ask my question. I have to prepare code that will proceed like x1 = x0 + t* e, which in practice looks like:
x1 = [0.5, 1] + [0, t]
x1 = [0.5, 1+t]
How should I declare t to make it work? I mean t has to remain here all the time, to make it possible to calculate the roots of a quadratic function a few steps further.
This would be hard to implement in a general purpose programming language because you need t to stay "symbolic" so you can do algebraic manipulations with it. You should look into implementing this in a Computer Algebra System (CAS) because those are specifically designed to handle symbolic computations. Implementing what you describe would be very quick and easy in a CAS.
There is well-known (and expensive proprietary) CAS software like Mathematica or Matlab. Or if you are working in C++ or python there is SymbolicC++ and SymPy that integrate well with each of them respectively. You can see Wikipedia for a list of CAS software.
Related
Is it possible to create a new feature out of two, or more than two existing features using a decision tree?
If so, how, and can it produce features with good information value that can better help the model?
The Decision Tree itself doesn't create the third variable. You would create the third variable yourself, a task commonly referred to as feature engineering. There are numerous and perhaps infinite possibilities, for example,
x3 = x1 + x2
x3 = x1 / x2 (as long as x2 can't be zero)
x3 = x1 * exp(x2)
...
As you explore this wonderful world of feature engineering, you may find that some types of combinations work better with decision trees than others... but in general there is no correct answer; just experiment.
Just a tip to get you started - Decision Trees naturally handle collinearity quite well, because as soon as 1 node is split on x, the variables that were collinear with x are suddenly less useful within the split. So transformations that are highly correlated with x1 or x2 directly might not help very much.
I'm writing a program to do the Newton Raphson Method for n variable (System of equation) using Datagridview. My problem is to determine the inverse for Jacobian Matrix. I've search in internet to find a solution but a real couldn't get it until now so if someone can help me I will real appreciate. Thanks in advance.
If you are asking for a recommendation of a library, that is explicitly off topic in Stack Overflow. However below I mention some algorithms that are commonly used; this may help you to find, or write, what you need. I would, though, not recommend writing something, unless you really want to, as it can be tricky to get these algorithms right. If you do decide to write something I'd recommend the QR method, as the easiest to write, though the theory is a little subtle.
First off do you really need to compute the inverse? If, for example, what you need to do is to compute
x = inv(J)*y
then it's faster and more accurate to treat this problem as
solve J*x = y for x
The methods below all factor J into other matrices, for which this solution can be done. A good package that implements the factorisation will also have the code to perform the solution.
If you do really really need the inverse often the best way is to solve, one column at a time
J*K = I for K, where I is the identity matrix
LU decomposition
This may well be the fastest of the algorithms described here but is also the least accurate. An important point is that the algorithm must include (partial) pivoting, or it will not work on all invertible matrices, for example it will fail on a rotation through 90 degrees.
What you get is a factorisation of J into:
J = P*L*U
where P is a permutation matrix,
L lower triangular,
U upper triangular
So having factorised, to solve for x we do three steps, each straightforward, and each can be done in place (ie all the x's can be the same variable)
Solve P*x1 = y for x1
Solve L*x2 = x1 for x2
Solve U*x = x2 for x
QR decomposition
This may be somewhat slower than LU but is more accurate. Conceptually this factorises J into
J = Q*R
Where Q is orthogonal and R upper triangular. However as it is usually implemented you in fact pass y as well as J to the routine, and it returns R (in J) and Q'*y (in the passed y), so to solve for x you just need to solve
R*x = y
which, given that R is upper triangular, is easy.
SVD (Singular value decomposition)
This is the most accurate, but also the slowest. Moreover unlike the others you can make progress even if J is singular (you can compute the 'generalised inverse' applied to y).
I recommend reading up on this, but advise against implementing it yourself.
Briefly you factorise J as
J = U*S*V'
where U and V are orthogonal and S diagonal.
There are, of course, many other ways of solving this problem. For example if your matrices are very large (dimension in the thousands) an it may, particularly if they are sparse (lots of zeroes), be faster to use an iterative method.
When I reading this paper http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?1976ApJ...209..214B&data_type=PDF_HIGH&whole_paper=YES&type=PRINTER&filetype=.pdf
I try to solve eq(49) numerically, it seems a fokker-planck equation, I find finite difference method doesn't work, it's unstable.
Does any one know how to solve it?
computational science stack exchange is where you can ask and hope for an answer. Or you could try its physics cousin. The equation, you quote, is integro-differential equation, fairly non-linear... Fokker-Plank looking equation. Definitely not the typical Fokker-Plank.
What you can try is to discretize the space part of the function g(x,t) using finite differences or finite-elements. After all, 0 < x < x_max and you have boundary conditions. You also have to discretize the corresponding integration. So maybe finite elements might be more appropriate? Finite elements means you can write g(x, t) as a series of a well chosen basis of compactly supported simple enough functions Bj(x) : j = 1...N in the interval [0, x_max]
g(x,t) = sum_j=1:N gj(t)*Bj(x)
That will turn your function into a (large) vector gj(t) = g(x_j, t), for j = 1, 1, ...., N. As a result, you will obtain a non-linear system of ODEs
dgj(t)/dt = Qj(g1(t), g2(t), ..., gN(t))
j = 1 ... N
After that use something like Runge-Kutta to integrate numerically the ODE system.
The newest version of Gurobi allows for the use of indicator constraints but I can't figure out from the user manual how to implement these with the Matlab API.
Unfortunately, indicator constraints are not supported by the Gurobi MATLAB and R interfaces. These interfaces use a matrix representation. For example, for a linear program in canonical form:
max ct x
Ax = b
x ≥ 0
The interface takes the matrix A and vectors b and c, and returns the optimal solution. Unfortunately, this means that high-level representations like piecewise linear functions or indicator constraints are beyond the scope of the MATLAB and R interfaces. They are available for Python, if that helps.
I have a function in Python:
def f(x):
return x[0]**3 + x[1]**2 + 7
# Actually more than this.
# No analytical expression
It's a scalar valued function of a vector.
How can I approximate the Jacobian and Hessian of this function in numpy or scipy numerically?
(Updated in late 2017 because there's been a lot of updates in this space.)
Your best bet is probably automatic differentiation. There are now many packages for this, because it's the standard approach in deep learning:
Autograd works transparently with most numpy code. It's pure-Python, requires almost no code changes for typical functions, and is reasonably fast.
There are many deep-learning-oriented libraries that can do this.
Some of the most popular are TensorFlow, PyTorch, Theano, Chainer, and MXNet. Each will require you to rewrite your function in their kind-of-like-numpy-but-needlessly-different API, and in return will give you GPU support and a bunch of deep learning-oriented features that you may or may not care about.
FuncDesigner is an older package I haven't used whose website is currently down.
Another option is to approximate it with finite differences, basically just evaluating (f(x + eps) - f(x - eps)) / (2 * eps) (but obviously with more effort put into it than that). This will probably be slower and less accurate than the other approaches, especially in moderately high dimensions, but is fully general and requires no code changes. numdifftools seems to be the standard Python package for this.
You could also attempt to find fully symbolic derivatives with SymPy, but this will be a relatively manual process.
Restricted to just SciPy, the most convenient way I found was scipy.misc.derivative, within the appropriate loops, with lambdas to curry the function of interest.