tf.clip_by_norm(grad, 1.0) throws InvalidArgumentError Shapes must be equal rank, but are 2 and 1 - tensorflow

Question

can someone explain why tensorflow is giving me trouble when running the following code.
import tensorflow as tf
x = tf.keras.layers.Input(shape=(1,))
y = tf.keras.layers.Dense(1, activation=tf.nn.relu)(x)
loss = tf.losses.mean_squared_error(x,y)
grad = tf.gradients(loss, tf.trainable_variables())
# !!! GIVES ME TROUBLE !!!
clipped_grad = tf.clip_by_norm(grad, 1.0)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(y, feed_dict={x: [[1.0], [2.0], [3.0]]})
The error I get:
Traceback (most recent call last):
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\ops.py", line 1589, in _create_c_op
c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shapes must be equal rank, but are 2 and 1
From merging shape 0 with other shapes. for 'clip_by_norm/t' (op: 'Pack') with input shapes: [1,1], [1].
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/TObs/.PyCharmCE2018.1/config/scratches/scratch.py", line 11, in <module>
clipped_grad = tf.clip_by_norm(grad, 1.0)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\ops\clip_ops.py", line 140, in clip_by_norm
t = ops.convert_to_tensor(t, name="t")
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\ops.py", line 1011, in convert_to_tensor
as_ref=False)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\ops.py", line 1107, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\ops\array_ops.py", line 960, in _autopacking_conversion_function
return _autopacking_helper(v, inferred_dtype, name or "packed")
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\ops\array_ops.py", line 923, in _autopacking_helper
return gen_array_ops.pack(elems_as_tensors, name=scope)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 5532, in pack
"Pack", values=values, axis=axis, name=name)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\ops.py", line 3414, in create_op
op_def=op_def)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\ops.py", line 1756, in __init__
control_input_ops)
File "D:\Program Files\Python\Python_3_6_2\lib\site-packages\tensorflow\python\framework\ops.py", line 1592, in _create_c_op
raise ValueError(str(e))
ValueError: Shapes must be equal rank, but are 2 and 1
From merging shape 0 with other shapes. for 'clip_by_norm/t' (op: 'Pack') with input shapes: [1,1], [1].
Any thoughts? Im running on a Windows10 machine, with tensorflow-gpu 1.9.0, NVidia GTX 1080.
Help would be much appreciated :)
Cheers,
Tobs.

Answer 1

so after tinkering around I found out that one have to apply the tf.clip_by_norm seperatly for each value in the gradients tensor liek so:
clipped_gradients = [tf.clip_by_norm(g, grad_norm_clip) for g in tf.gradients(loss, tf.trainable_variables())]
I guess, thats the right way to do, right?
Cheers,
Tobs.