I'm converting a floating point number (or numpy array) to Pytorch tensor and it seems to be copying the inexact value to the tensor. The error comes in the 8th significant digit and afterwards. This is significant (no-pun intended) for my work as I deal with chaotic dynamics which is very sensitive towards the slight change in the initial conditions.
I'm already using torch.set_printoptions(precision=16) to print 16 significant digits.
np_x = state
print(np_x)
x = torch.tensor(np_x,requires_grad=True,dtype=torch.float32)
print(x.data[0])
and the output is :
0.7575408585008059
tensor(0.7575408816337585)
It would be helpful to know what is going wrong or how it could be resolved ?
Because you're using float32 dtype. If you convert these two numbers to binary, you will find they are actually the same. Strictly speaking, the most accurate representations of those two numbers in float32 format are the same.
0.7575408585008059
Most accurate representation = 7.57540881633758544921875E-1
0.7575408816337585
Most accurate representation = 7.57540881633758544921875E-1
Binary: 00111111 01000001 11101110 00110011
Related
I'm trying to understand the difference between numpy fft and rfft. I've read the doc, and it only says rfft is meant for real inputs.
I've tested their performance on a large real array and found out rfft is faster than fft by about a third. My question is: why is rfft fast? Thanks!
An RFFT has half the degrees of freedom on the input, and half the number of complex outputs, compared to an FFT. Thus the FFT computation tree can be pruned to remove those adds and multiplies not needed for the non-existent inputs and/or those unnecessary since there are a lesser number of independant output values that need to be computed.
This is because an FFT of a strictly real input (e.g. all the input value imaginary components zero) produces a complex conjugate mirrored result, where each half can be trivially derived from the other half.
at the risk of using the wrong SE...
My question concerns the rcond parameter in numpy.linalg.lstsq(a, b, rcond). I know it's used to define the cutoff for small singular values in the singular value decomposition when numpy computed the pseudo-inverse of a.
But why is it advantageous to set "small" singular values (values below the cutoff) to zero rather than just keeping them as small numbers?
PS: Admittedly, I don't know exactly how the pseudo-inverse is computed and exactly what role SVD plays in that.
Thanks!
To determine the rank of a system, you could need compare against zero, but as always with floating point arithmetic we should compare against some finite range. Hitting exactly 0 never happens when adding and subtracting messy numbers.
The (reciprocal) condition number allows you to specify such a limit in a way that is independent of units and scale in your matrix.
You may also opt of of this feature in lstsq by specifying rcond=None if you know for sure your problem can't run the risk of being under-determined.
The future default sounds like a wise choice from numpy
FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions
because if you hit this limit, you are just fitting against rounding errors of floating point arithmetic instead of the "true" mathematical system that you hope to model.
when trying Tensorflow intro i came across the following code
w=tf.Variable(.3,tf.float32)
b=tf.Variable(-.3,tf.float32)
while printing this values it gives following output
print(sess.run(w))
print(sess.run(b))
print(sess.run([w]))
print(sess.run([b]))
Output
-0.3
-0.3
[0.30000001]
[-0.30000001]
why while print as array it gives extra floating point precision?
Is there any documentation related this topic?
Here is a great resource to answer this question. To paraphrase the first paragraph on that web page :
TensorFlow isn't broken, it's doing floating point math. Computers can only natively store integers, so they need some way of representing decimal numbers. This representation comes with some degree of inaccuracy. That's why, more often than not, .3 == .30000001.
When I decrease the value of a coefficient in my code something stops working. Can I have a division by zero without an error message? Can this be solved by increasing the number of significant digits?
How can I increase the number of significant digits in numpy? Thank you
Numpy does not support arbitrary precision. see here. The scalar types they have are these.
Consider using fractions module or other library w arbitrary precision...
In numpy, I'm reading an ASCII file (see below) using np.genfromtxt()
0.085 102175 0.00025
0.094 103325 0.00030
raw = genfromtxt(fn)
When checking raw I get the following:
>>> raw[0,0]
0.085000000000000006
How do I prevent the artifact 6 at the end and where does it come from?
This is normal behaviour, and is due to the fundamental imprecision of floating point arithmetic. In other words, 0.085 cannot be represented exactly in floating point bits. For this reason, it's generally a good idea to assume a bit of noise in any numerical calculations.