Having problems to implement a linear interpolation in numpy that works for n-D arrays. I would like a simple solution working for 1D and also 3D arrays.
For the linear interpolation, I'm using np.interp, which works great for 1D:
x = [1, 2, 3, 4, 5]
xp = [1, 3, 5]
fp = [10, 30, 50]
np.interp(x, xp, fp)
Out: array([10., 20., 30., 40., 50.])
Now if my fp has multiple columns:
fp_multicol = np.array([10, 30, 50, 100, 300, 500]).reshape((2,3))
Out:
array([[ 10, 30, 50],
[100, 300, 500]])
That doesn't work directly with np.interp since it can only handle 1D arrays:
np.interp(x, xp, fp_multicol)
Traceback (most recent call last):
blablabla
ValueError: object too deep for desired array
So I came up with a for-loop in a list comprehension:
np.array([np.interp(x, xp, col) for col in fp_multicol])
array([[ 10., 20., 30., 40., 50.],
[100., 200., 300., 400., 500.]])
But then that doesn't work for 1D arrays...
np.array([np.interp(x, xp, col) for col in fp])
Traceback (most recent call last):
blablabla
ValueError: object of too small depth for desired array
Any quick and simple solution that would work for both?
I'd expect numpy to have such interpolation function but so far could not find any.
Related
I have several RaggedTensors that I want to concatenate; I am using Keras. Vanilla Tensorflow will happily concatenate them, so I tried the code:
card_feature = layers.concatenate([ragged1, ragged2, ragged3])
but it gave the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/timeroot/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 925, in __call__
return self._functional_construction_call(inputs, args, kwargs,
File "/home/timeroot/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1084, in _functional_construction_call
base_layer_utils.create_keras_history(inputs)
File "/home/timeroot/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 191, in create_keras_history
_, created_layers = _create_keras_history_helper(tensors, set(), [])
File "/home/timeroot/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 222, in _create_keras_history_helper
raise ValueError('Tensorflow ops that generate ragged or sparse tensor '
ValueError: Tensorflow ops that generate ragged or sparse tensor outputs are currently not supported by Keras automatic op wrapping. Please wrap these ops in a Lambda layer:
```
weights_mult = lambda x: tf.sparse.sparse_dense_matmul(x, weights)
output = tf.keras.layers.Lambda(weights_mult)(input)
```
so then I tried:
concat_lambda = lambda xs: tf.concat(xs, axis=2)
card_feature = layers.Lambda(concat_lambda)([ragged1, ragged2, ragged3])
but it gave the exact same error, even though I had wrapped it. Is this a bug / is there a workaround?
Code to concatenate 3 Ragged Tensors is shown below:
import tensorflow as tf
print(tf.__version__)
Ragged_Tensor1 = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []])
Ragged_Tensor2 = tf.ragged.constant([[5, 3]])
Ragged_Tensor3 = tf.ragged.constant([[6,7,8], [9,10]])
print(tf.concat([Ragged_Tensor1, Ragged_Tensor2, Ragged_Tensor3], axis=0))
Output is shown below:
2.3.0
<tf.RaggedTensor [[3, 1, 4, 1], [], [5, 9, 2], [6], [], [5, 3], [6, 7, 8], [9, 10]]>
But it looks like you are trying to concatenate Ragged Tensor Operations. Please share your complete code so that we can try to help you.
My data is numpy ndarray with shape(2,3,4) following this:
I've try to normalize 0-1 scale for each column through sklearn normalization.
from sklearn.preprocessing import normalize
x = np.array([[[1, 2, 3, 4],
[2, 2, 3, 4],
[3, 2, 3, 4]],
[[4, 2, 3, 4],
[5, 2, 3, 4],
[6, 2, 3, 4]]])
x.shape ==> ( 2,3,4)
x = normalize(x, norm='max', axis=0, )
However, I catch the error :
ValueError: Found array with dim 3. the normalize function expected <= 2.
How do I solve this problem?
Thank you.
It seems scikit-learn expects ndarrays with at most two dims. So, to solve it would be to reshape to 2D, feed it to normalize that gives us a 2D array, which could be reshaped back to original shape -
from sklearn.preprocessing import normalize
normalize(x.reshape(x.shape[0],-1), norm='max', axis=0).reshape(x.shape)
Alternatively, it's much simpler with NumPy that works fine with generic ndarrays -
x/np.linalg.norm(x, ord=np.inf, axis=0, keepdims=True)
In torch there is a module named, torch.Min, that can apply the min operation over different branches of a network.
nn.Min
is there something similar in tensorflow?. (noob in tensorflow)
I think you want tf.reduce_min. To find the min of [10, 3, 5, 4]:
import tensorflow as tf
def one_dim_graph():
input_tensor = tf.constant([10, 3, 5, 4])
tensor_min = tf.reduce_min(input_tensor)
return input_tensor, tensor_min
def run():
in_one, g_one = one_dim_graph()
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
out_one = sess.run([g_one])
print in_one
print out_one
run()
This will give you the min:
Tensor("Const_74:0", shape=(4,), dtype=int32)
[3]
You can also use this across multi-dimension tensors and reduce all dimensions or reduce along a certain dimension. So, for example, to find the min of:
[[1, 4, 3, 10],
[6, 5, 2, 12],
[9, 7, 8, 11]]
We can find:
the total min, [1] with tf.reduce_min(input_tensor)
the column-wise min [1, 4, 2, 10] with tf.reduce_min(input_tensor, reduction_indices=0)
the row-wise min [1, 2, 7] with tf.reduce_min(input_tensor, reduction_indices=1)
the new higher order functions within TF is detailed here:
https://www.tensorflow.org/versions/r0.8/api_docs/python/functional_ops.html#map_fn
In particular, the map function looks useful. Here is what they wrote for the tutorial:
elems = [1, 2, 3, 4, 5, 6]
squares = map_fn(lambda x: x * x, elems)
# squares == [1, 4, 9, 16, 25, 36]
Thus I created an empty python file:
import tensorflow as tf
elems = [1, 2, 3, 4, 5, 6]
squares = tf.map_fn(lambda x: x * x, elems)
Running this gives this error:
/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.pyc in make_tensor_proto(values, dtype, shape)
323 else:
324 if values is None:
--> 325 raise ValueError("None values not supported.")
326 # if dtype is provided, forces numpy array to be the type
327 # provided if possible.
ValueError: None values not supported.
Does anyone know what's going on? Thanks!
Edit: I'm using TensorFlow version 0.8.
Your code seems OK, but I was able to fix it using the numpy array something like this:
import tensorflow as tf
import numpy as np
elems = np.array([1, 2, 3, 4, 5, 6], dtype="float32")
squares = tf.map_fn(lambda x: x * x, elems)
sess = tf.Session()
sess.run(squares)
This outputs:
[ 1. 4. 9. 16. 25. 36.]
Why can't I use numpy.logaddexp.reduce?
In [46]: a = np.array([1,5, 3, 2])
In [47]: np.logaddexp.reduce(a)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-47-701e5f4017fe> in <module>()
----> 1 np.logaddexp.reduce(a)
TypeError: No loop matching the specified signature was found for ufunc logaddexp
Looks like the reduce function doesn't accept an integer array. Use a floating point array:
In [28]: a = np.array([1.0, 5.0, 3.0, 2.0])
In [29]: np.logaddexp.reduce(a)
Out[29]: 5.1851824526038124
or use the dtype argument:
In [34]: a = np.array([1, 5, 3, 2])
In [35]: np.logaddexp.reduce(a, dtype=np.float64)
Out[35]: 5.1851824526038124