In Sage, there is a function 'append'. One example is as follows.
A=[]
for i in range(100):
if(i%2==0):
A.append(i)
In libgmp is there any such kind of function where I can store mpz_t values?
No. GMP just provides the primitives for mpz_t creation, deletion, and mathmatical operations. Support for data structures is provided by the language that is using libgmp. Sage uses Python as its language for gluing together other libraries and your example is just an example of Python code.
Related
How much cross compatibility is there between the different language APIs?
For example, is it possible to train and save a model in Python and run it in C/C++ or any other language?
I would try this myself however my skills in non-Python languages are very limited.
You can dump the model into a text file as like this:
model.get_booster().dump_model('xgb_model.txt')
Then you should parse the text dump and reproduce the prediction function in C++.
I have implemented this in a little library that I call FastForest, if you want to save some time and want to make sure you use a fast implementation:
https://github.com/guitargeek/XGBoost-FastForest
The mission of the library is to be:
Easy: deploying your xgboost model should be as painless as it can be
Fast: thanks to efficient structure-of-array data structures for storing the trees, this library goes very easy on your CPU and memory (it is about 3 to 5 times faster than xgboost in prediction)
Safe: the FastForest objects are immutable, and therefore they are an excellent choice in multithreading environments
Portable: FastForest has no dependency other than the C++ standard library
Here is a little usage example, loading the model you have dumped before and assuming the model requires 5 features:
std::vector<std::string> features{"f0", "f1", "f2", "f3", "f4"};
FastForest fastForest("xgb_model.txt", features);
std::vector<float> input{0.0, 0.2, 0.4, 0.6, 0.8};
float output = fastForest(input.data());
When you create the FastForest you have to tell it in which order you intend to pass the features, because the text file does not store the order of the features.
Also note that the FastForest does not do the logistic transformation for you, so in order to reproduce predict_proba() you need to apply the logistic transformation:
float proba = 1./(1. + std::exp(-output));
The treelite package(research paper, documentation) enables compilation of tree-based models, including XGBoost, to optimized C code, making inference much faster than with native model libraries.
You could consider dumping your model in a text file using
model.get_booster().dump_model('xgb_model.txt', with_stats=True)
then, after some parsing, you can easily reproduce the .predict() function in C/C++. For the rest I am not aware of native porting of xgboost to C
I'm trying to find the source code for TensorFlow's low level linear-algebra and matrix arithmetic operators for execution on CPU. For example, where is the actual implementation of tf.add() for execution on a CPU? As far as I know, most linear algebra operators are actually implemented by Eigen, but I'd like to know what Eigen functions specifically are being called.
I've tried tracing back from the high-level API, but this is difficult as there are a lot of steps between placing an operator on the graph, and the actual execution of the operator by the TF runtime.
The implementation is hidden behind some meta-template programming (not unusual for Eigen).
Each operation in TensorFlow is registered at some point. Add is registered here and here.
REGISTER3(BinaryOp, GPU, "Add", functor::add, float, Eigen::half, double);
The actual implementation of Operations is based on OpKernel. The Add operation is implemented in BinaryOp::Compute The class hierarchy would be BinaryOp : BinaryOpShared : OpKernel
In the case of adding two scalars, the entire implementation is just:
functor::BinaryFunctor<Device, Functor, 1>().Right(
eigen_device, out_flat, in0.template flat<Tin>(),
in1.template scalar<Tin>(), error_ptr);
where in0, in1 are the incoming Tensor-Scalars, Device is either GPU or CPU, and Functor is the operation itself. The other lines are just for performing the broadcasting.
Scroll down in this file and expanding the REGISTER3 macro explains how the arguments are passed from REGISTER3 to functor::BinaryFunctor<Device, Functor, ...>.
You cannot expect to see some loops as Eigen use Expressions to do Lazy Evaluation and Aliasing. The Eigen-"Call" is here:
https://github.com/tensorflow/tensorflow/blob/7a0def60d45c1841a4e79a0ddf6aa9d50bf551ac/tensorflow/core/kernels/cwise_ops.h#L693-L696
I'm trying to translate something from Python to Fortran because of speed limitations. (So I can then use f2py on it.)
The problem is that the code contains many NumPy functions that don't exist in Fortran 90. So my questions is: is there a Fortran library that implements at least some of the NumPy functionality in Fortran?
The functions that I have to use in the code are generally simple, so I could translate them by hand. However, I'm trying not to re-invent the wheel here, specially because I don't have that much experience in Fortran and I might not know some important caveats.
Anyway, here's a list of some of the functions that I need.
np.mean (with the axis parameter)
np.std (with the axis parameter)
np.roll (again with the axis parameter)
np.mgrid
np.max (again with axis parameter)
Anything is helpful at this point. I'm not counting on finding substitutes for all of them, but it would be very good if some of them, at least, already existed.
I find that the intrinsic list of procedures from gfortran is useful as a first reference here https://gcc.gnu.org/onlinedocs/gfortran/Intrinsic-Procedures.html#Intrinsic-Procedures
np.mean (with the axis parameter)
See sum. It has an axis parameter. In combination with size it can output the mean:
result = sum(data, dim=axis)/size(data, dim=axis)
Here, result has one less dimension than data.
np.std (with the axis parameter)
np.roll (again with the axis parameter)
np.mgrid
np.max (again with axis parameter)
See maxval, it has a dim argument.
I am not aware of a Fortran equivalent to NumPy. The standard-based array abilities of Fortran are such that a "base" library has not emerged. There are several initiatives though:
https://github.com/astrofrog/fortranlib "Collection of personal scientific routines in Fortran"
http://fortranwiki.org/ "The Fortran Wiki is an open venue for discussing all aspects of the Fortran programming language and scientific computing."
http://flibs.sourceforge.net/ "FLIBS - A collection of Fortran modules"
http://www.fortran90.org/ General resource for modern Fortran. Contains a "Python Fortran Rosetta Stone"
What is the equivalent of Python function tf.gradients(loss, [var]) in C++? Thanks!
The equivalent function in C++ is tensorflow::AddSymbolicGradients(). You will need to obtain a tensorflow::Graph object representing your graph to use this function. However, adding gradients in C++ is still experimental, so beware that this function signature is subject to change.
I have a function in Python:
def f(x):
return x[0]**3 + x[1]**2 + 7
# Actually more than this.
# No analytical expression
It's a scalar valued function of a vector.
How can I approximate the Jacobian and Hessian of this function in numpy or scipy numerically?
(Updated in late 2017 because there's been a lot of updates in this space.)
Your best bet is probably automatic differentiation. There are now many packages for this, because it's the standard approach in deep learning:
Autograd works transparently with most numpy code. It's pure-Python, requires almost no code changes for typical functions, and is reasonably fast.
There are many deep-learning-oriented libraries that can do this.
Some of the most popular are TensorFlow, PyTorch, Theano, Chainer, and MXNet. Each will require you to rewrite your function in their kind-of-like-numpy-but-needlessly-different API, and in return will give you GPU support and a bunch of deep learning-oriented features that you may or may not care about.
FuncDesigner is an older package I haven't used whose website is currently down.
Another option is to approximate it with finite differences, basically just evaluating (f(x + eps) - f(x - eps)) / (2 * eps) (but obviously with more effort put into it than that). This will probably be slower and less accurate than the other approaches, especially in moderately high dimensions, but is fully general and requires no code changes. numdifftools seems to be the standard Python package for this.
You could also attempt to find fully symbolic derivatives with SymPy, but this will be a relatively manual process.
Restricted to just SciPy, the most convenient way I found was scipy.misc.derivative, within the appropriate loops, with lambdas to curry the function of interest.