Problem with Logits using Densenet121 in Tensorflow 2.4 - tensorflow

I was trying to reproach the examples Transfer learning and fine-tuning adapting for my problem My colab with GPU. When I use a softmax in the last Dense Layers, this error not ocorrer. But with 'from_logits=True', this error ocorrer, my imagens are jpg and they are divided in folders:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-85-7ea61d5df8ec> in <module>()
----> 1 loss0, accuracy0, auc0, precision0, recall0 = model.evaluate(val_ds)
5 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in evaluate(self, x, y, batch_size, verbose, sample_weight, steps, callbacks, max_queue_size, workers, use_multiprocessing, return_dict)
1387 with trace.Trace('test', step_num=step, _r=1):
1388 callbacks.on_test_batch_begin(step)
-> 1389 tmp_logs = self.test_function(iterator)
1390 if data_handler.should_sync:
1391 context.async_wait()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
826 tracing_count = self.experimental_get_tracing_count()
827 with trace.Trace(self._name) as tm:
--> 828 result = self._call(*args, **kwds)
829 compiler = "xla" if self._experimental_compile else "nonXla"
830 new_tracing_count = self.experimental_get_tracing_count()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
893 # If we did not create any variables the trace we have is good enough.
894 return self._concrete_stateful_fn._call_flat(
--> 895 filtered_flat_args, self._concrete_stateful_fn.captured_inputs) # pylint: disable=protected-access
896
897 def fn_with_cond(inner_args, inner_kwds, inner_filtered_flat_args):
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
1917 # No tape is watching; skip to running the function.
1918 return self._build_call_outputs(self._inference_function.call(
-> 1919 ctx, args, cancellation_manager=cancellation_manager))
1920 forward_backward = self._select_forward_and_backward_functions(
1921 args,
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
558 inputs=args,
559 attrs=attrs,
--> 560 ctx=ctx)
561 else:
562 outputs = execute.execute_with_cancellation(
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (Dense121/dense_1/BiasAdd:0) = ] [[0.853173912 1.97515857 0.608713508...]...] [y (Cast_4/x:0) = ] [0]
[[{{node assert_greater_equal/Assert/AssertGuard/else/_1/assert_greater_equal/Assert/AssertGuard/Assert}}]]
(1) Invalid argument: assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (Dense121/dense_1/BiasAdd:0) = ] [[0.853173912 1.97515857 0.608713508...]...] [y (Cast_4/x:0) = ] [0]
[[{{node assert_greater_equal/Assert/AssertGuard/else/_1/assert_greater_equal/Assert/AssertGuard/Assert}}]]
[[assert_greater_equal_2/Assert/AssertGuard/branch_executed/_65/_167]]
0 successful operations.
0 derived errors ignored. [Op:__inference_test_function_61870]
Function call stack:
test_function -> test_function
I tried several things to solve and nothing

Related

How to fix TypeError: x and y must have the same dtype, got tf.int32 != tf.float32 with tensorflow using PyGad?

I have defined a new layer to be put as an input layer using Keras. The code is:
class layer(tensorflow.keras.layers.Layer):
def __init__(self):
super(layer, self).__init__()
H_init = tf.random_normal_initializer()
self.H = tf.Variable(
initial_value=H_init(shape=(1,), dtype="float32"),
trainable=True,
)
b_init = tf.zeros_initializer()
self.b = tf.Variable(
initial_value=b_init(shape=(1,), dtype="float32"), trainable=True
)
n_init = tf.zeros_initializer()
self.n = tf.Variable(
initial_value=n_init(shape=(1,), dtype="float32"), trainable=True
)
def call(self, z):
return self.H**2 / (1+(1/self.b)**(2/self.n)) *(1+((1+z)/self.b)**(2/self.n))
I intend to put this layer as input in my generation callback function, which is
`def callback_generation(ga_instance):
print("Generation"= {generation}".format(generation=ga_instance.generations_completed))
print("Fitness =
{fitness}".format(fitness=ga_instance.best_solution()[1]))
inputs = tensorflow.keras.Input(shape=(1,), name="inputs")
targets = tensorflow.keras.Input(shape=(1,), name="targets")
logits = tensorflow.keras.layers.Dense(15)(inputs)
predictions = layer(name="predictions")(logits, targets)
model = keras.Model(inputs=[inputs, targets],
outputs=predictions)
data = {
"inputs": z,
"targets": H_z,
}
model = tensorflow.keras.Sequential(inputs=input_layer,
outputs=output_layer)
weights_vector =
tensorflow.pygad.kerasga.model_weights_as_vector(model=model)
keras_ga = pygad.kerasga.KerasGA(model=model,
num_solutions=12)
model.summary()`
This gives me an error reporting that the tf__call() method takes 2 positional arguments but 3 were given.
This is the detailed error traceback:
TypeError Traceback (most
recent call last)
<ipython-input-101-e09c7d4f3228> in <module>
20 targets = tensorflow.keras.Input(shape=(1,),
name="targets")
21 logits = tensorflow.keras.layers.Dense(15)(inputs)
---> 22 predictions = layer(name="predictions")(logits,
targets)
23
24 model = keras.Model(inputs=[inputs, targets],
outputs=predictions)
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in
__call__(self, *args, **kwargs)
950 if _in_functional_construction_mode(self, inputs,
args, kwargs, input_list):
951 return self._functional_construction_call(inputs,
args, kwargs,
--> 952
input_list)
953
954 # Maintains info about the `Layer.call` stack.
~/anaconda3/lib/python3.7/site-
packages/tensorflow/python/keras/engine/base_layer.py in
_functional_construction_call(self, inputs, args, kwargs,
input_list)
1089 # Check input assumptions set after layer
building, e.g. input shape.
1090 outputs = self._keras_tensor_symbolic_call(
-> 1091 inputs, input_masks, args, kwargs)
1092
1093 if outputs is None:
~/anaconda3/lib/python3.7/site-
packages/tensorflow/python/keras/engine/base_layer.py in
_keras_tensor_symbolic_call(self, inputs, input_masks, args,
kwargs)
820 return
nest.map_structure(keras_tensor.KerasTensor, output_signature)
821 else:
--> 822 return self._infer_output_signature(inputs, args,
kwargs, input_masks)
823
824 def _infer_output_signature(self, inputs, args,
kwargs, input_masks):
~/anaconda3/lib/python3.7/site-
packages/tensorflow/python/keras/engine/base_layer.py in
_infer_output_signature(self, inputs, args, kwargs,
input_masks)
861 # TODO(kaftan): do we maybe_build here, or
have we already done it?
862 self._maybe_build(inputs)
--> 863 outputs = call_fn(inputs, *args, **kwargs)
864
865 self._handle_activity_regularization(inputs,
outputs)
~/anaconda3/lib/python3.7/site-
packages/tensorflow/python/autograph/impl/api.py in
wrapper(*args, **kwargs)
668 except Exception as e: # pylint:disable=broad-
except
669 if hasattr(e, 'ag_error_metadata'):
--> 670 raise e.ag_error_metadata.to_exception(e)
671 else:
672 raise
TypeError: in user code:
TypeError: tf__call() takes 2 positional arguments but 3 were given

TensorFlow GradienTape crashes with InvalidArgumentError when working on sparse Tensors

I am encountering strange behavior when trying to evaluate the derivatives of a result obtained by sparse tensor operations. If I blow up all sparse inputs to dense before operating on them, the following code works as expected (first part of the following code), but it crashes with InvalidArgumentError when I do the same with sparse tensors. In addition, I get while_loop Warnings as below. As in the actual problem of course more operations and much bigger and more tensors are involved, I essentially have to collect the entries of c in sparse mode. Can anyone make (more) sense of this behavior?
import tensorflow as tf
import numpy as np
a=tf.SparseTensor(indices=[[0,0],[1,1]],values=np.array([1,1],dtype=np.float32),dense_shape=(2,2))
b=tf.SparseTensor(indices=[[0,1],[1,0]],values=np.array([-1,-1],dtype=np.float32),dense_shape=(2,2))
#dense mode...
f1=tf.Variable([1,1],dtype=np.float32)
with tf.GradientTape() as gtape:
c=tf.sparse.to_dense(a)*f1[0]+tf.sparse.to_dense(b)*f1[1]
print(gtape.jacobian(c,f1)) #... works fine
#sparse mode...
f2=tf.Variable([1,1],dtype=np.float32)
with tf.GradientTape() as gtape:
c=tf.sparse.add(a*f2[0],b*f2[1],0)
c=tf.sparse.to_dense(c)
print(gtape.jacobian(c,f2)) #... InvalidArgumentError
#WARNING:tensorflow:Using a while_loop for converting SparseAddGrad
#WARNING:tensorflow:Using a while_loop for converting SparseTensorDenseAdd
#WARNING:tensorflow:Using a while_loop for converting SparseTensorDenseAdd
#---------------------------------------------------------------------------
#InvalidArgumentError Traceback (most recent call last)
#<ipython-input-10-d449761ef6b2> in <module>
# 12 c=tf.sparse.add(a*f2[0],b*f2[1],0)
# 13 c=tf.sparse.to_dense(c)
#---> 14 print(gtape.jacobian(c,f2)) #InvalidArgumentError
#c:\program files\python37\lib\site-packages\tensorflow\python\eager\backprop.py in jacobian(self, target, sources, unconnected_gradients, parallel_iterations, experimental_use_pfor)
# 1187 try:
# 1188 output = pfor_ops.pfor(loop_fn, target_size,
#-> 1189 parallel_iterations=parallel_iterations)
# 1190 except ValueError as err:
# 1191 six.reraise(
#c:\program files\python37\lib\site-packages\tensorflow\python\ops\parallel_for\control_flow_ops.py in pfor(loop_fn, iters, fallback_to_while_loop, parallel_iterations)
# 203 def_function.run_functions_eagerly(False)
# 204 f = def_function.function(f)
#--> 205 outputs = f()
# 206 if functions_run_eagerly is not None:
# 207 def_function.run_functions_eagerly(functions_run_eagerly)
#c:\program files\python37\lib\site-packages\tensorflow\python\eager\def_function.py in __call__(self, *args, **kwds)
# 826 tracing_count = self.experimental_get_tracing_count()
# 827 with trace.Trace(self._name) as tm:
#--> 828 result = self._call(*args, **kwds)
# 829 compiler = "xla" if self._experimental_compile else "nonXla"
# 830 new_tracing_count = self.experimental_get_tracing_count()
#c:\program files\python37\lib\site-packages\tensorflow\python\eager\def_function.py in _call(self, *args, **kwds)
# 893 # If we did not create any variables the trace we have is good enough.
# 894 return self._concrete_stateful_fn._call_flat(
#--> 895 filtered_flat_args, self._concrete_stateful_fn.captured_inputs) # pylint: disable=protected-access
# 896
# 897 def fn_with_cond(inner_args, inner_kwds, inner_filtered_flat_args):
#c:\program files\python37\lib\site-packages\tensorflow\python\eager\function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
# 1917 # No tape is watching; skip to running the function.
# 1918 return self._build_call_outputs(self._inference_function.call(
#-> 1919 ctx, args, cancellation_manager=cancellation_manager))
# 1920 forward_backward = self._select_forward_and_backward_functions(
# 1921 args,
#c:\program files\python37\lib\site-packages\tensorflow\python\eager\function.py in call(self, ctx, args, cancellation_manager)
# 558 inputs=args,
# 559 attrs=attrs,
#--> 560 ctx=ctx)
# 561 else:
# 562 outputs = execute.execute_with_cancellation(
#c:\program files\python37\lib\site-packages\tensorflow\python\eager\execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
# 58 ctx.ensure_initialized()
# 59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
#---> 60 inputs, attrs, num_outputs)
# 61 except core._NotOkStatusException as e:
# 62 if name is not None:
#InvalidArgumentError: Only tensors with ranks between 1 and 5 are currently supported. Tensor rank: 0
# [[{{node gradient_tape/SparseTensorDenseAdd_1/pfor/while/body/_56/gradient_tape/SparseTensorDenseAdd_1/pfor/while/SparseTensorDenseAdd}}]] [Op:__inference_f_6235]
#Function call stack:
#f

TypeError: Expected any non-tensor type, got a tensor instead

I Was following a post on 'Training a transformer model for a chatbot with TensorFlow 2.0'. I have encountered an error on my local machine although the code seems to work fine in colab. Below is the code snippet.
def encoder_layer(units, d_model, num_heads, dropout, name="encoder_layer"):
inputs = tf.keras.Input(shape=(None, d_model), name="inputs")
padding_mask = tf.keras.Input(shape=(1, 1, None), name="padding_mask")
attention = MultiHeadAttention(
d_model, num_heads, name="attention")({
'query': inputs,
'key': inputs,
'value': inputs,
'mask': padding_mask
})
attention = tf.keras.layers.Dropout(rate=dropout)(attention)
attention = tf.keras.layers.LayerNormalization(
epsilon=1e-6)(inputs + attention)
outputs = tf.keras.layers.Dense(units=units, activation='relu')(attention)
outputs = tf.keras.layers.Dense(units=d_model)(outputs)
outputs = tf.keras.layers.Dropout(rate=dropout)(outputs)
outputs = tf.keras.layers.LayerNormalization(
epsilon=1e-6)(attention + outputs)
return tf.keras.Model(
inputs=[inputs, padding_mask], outputs=outputs, name=name)
I called above function with the following function call;
sample_encoder_layer = encoder_layer(
units=512,
d_model=128,
num_heads=4,
dropout=0.3,
name="sample_encoder_layer")
Below is the traceback of the error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in _AssertCompatible(values, dtype)
323 try:
--> 324 fn(values)
325 except ValueError as e:
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in _check_not_tensor(values)
275 def _check_not_tensor(values):
--> 276 _ = [_check_failed(v) for v in nest.flatten(values)
277 if isinstance(v, ops.Tensor)]
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in <listcomp>(.0)
276 _ = [_check_failed(v) for v in nest.flatten(values)
--> 277 if isinstance(v, ops.Tensor)]
278 # pylint: enable=invalid-name
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in _check_failed(v)
247 # it is safe to use here.
--> 248 raise ValueError(v)
249
ValueError: Tensor("attention_1/Identity:0", shape=(None, None, 128), dtype=float32)
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-20-3fa05a9bbfda> in <module>
----> 1 sample_encoder_layer = encoder_layer(units=512, d_model=128, num_heads=4, dropout=0.3, name='sample_encoder_layer')
2
3 tf.keras.utils.plot_model(
4 sample_encoder_layer, to_file='encoder_layer.png', show_shapes=True)
<ipython-input-18-357ca53de1c0> in encoder_layer(units, d_model, num_heads, dropout, name)
10 'mask': padding_mask
11 })
---> 12 attention = tf.keras.layers.Dropout(rate=dropout)(attention)
13 attention = tf.keras.layers.LayerNormalization(
14 epsilon=1e-6)(inputs + attention)
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
920 not base_layer_utils.is_in_eager_or_tf_function()):
921 with auto_control_deps.AutomaticControlDependencies() as acd:
--> 922 outputs = call_fn(cast_inputs, *args, **kwargs)
923 # Wrap Tensors in `outputs` in `tf.identity` to avoid
924 # circular dependencies.
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py in call(self, inputs, training)
209 output = tf_utils.smart_cond(training,
210 dropped_inputs,
--> 211 lambda: array_ops.identity(inputs))
212 return output
213
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/keras/utils/tf_utils.py in smart_cond(pred, true_fn, false_fn, name)
63 pred, true_fn=true_fn, false_fn=false_fn, name=name)
64 return smart_module.smart_cond(
---> 65 pred, true_fn=true_fn, false_fn=false_fn, name=name)
66
67
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/smart_cond.py in smart_cond(pred, true_fn, false_fn, name)
57 else:
58 return control_flow_ops.cond(pred, true_fn=true_fn, false_fn=false_fn,
---> 59 name=name)
60
61
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
505 'in a future version' if date is None else ('after %s' % date),
506 instructions)
--> 507 return func(*args, **kwargs)
508
509 doc = _add_deprecated_arg_notice_to_docstring(
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py in cond(pred, true_fn, false_fn, strict, name, fn1, fn2)
1175 if (util.EnableControlFlowV2(ops.get_default_graph()) and
1176 not context.executing_eagerly()):
-> 1177 return cond_v2.cond_v2(pred, true_fn, false_fn, name)
1178
1179 # We needed to make true_fn/false_fn keyword arguments for
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/ops/cond_v2.py in cond_v2(pred, true_fn, false_fn, name)
82 true_name, collections=ops.get_default_graph()._collections), # pylint: disable=protected-access
83 add_control_dependencies=add_control_dependencies,
---> 84 op_return_value=pred)
85 false_graph = func_graph_module.func_graph_from_py_func(
86 false_name,
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
979 _, original_func = tf_decorator.unwrap(python_func)
980
--> 981 func_outputs = python_func(*func_args, **func_kwargs)
982
983 # invariant: `func_outputs` contains only Tensors, CompositeTensors,
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py in dropped_inputs()
205 noise_shape=self._get_noise_shape(inputs),
206 seed=self.seed,
--> 207 rate=self.rate)
208
209 output = tf_utils.smart_cond(training,
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
505 'in a future version' if date is None else ('after %s' % date),
506 instructions)
--> 507 return func(*args, **kwargs)
508
509 doc = _add_deprecated_arg_notice_to_docstring(
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py in dropout(x, keep_prob, noise_shape, seed, name, rate)
4341 raise ValueError("You must provide a rate to dropout.")
4342
-> 4343 return dropout_v2(x, rate, noise_shape=noise_shape, seed=seed, name=name)
4344
4345
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py in dropout_v2(x, rate, noise_shape, seed, name)
4422 raise ValueError("rate must be a scalar tensor or a float in the "
4423 "range [0, 1), got %g" % rate)
-> 4424 x = ops.convert_to_tensor(x, name="x")
4425 x_dtype = x.dtype
4426 if not x_dtype.is_floating:
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, dtype_hint, ctx, accepted_result_types)
1339
1340 if ret is None:
-> 1341 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
1342
1343 if ret is NotImplemented:
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py in _constant_tensor_conversion_function(v, dtype, name, as_ref)
319 as_ref=False):
320 _ = as_ref
--> 321 return constant(v, dtype=dtype, name=name)
322
323
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py in constant(value, dtype, shape, name)
260 """
261 return _constant_impl(value, dtype, shape, name, verify_shape=False,
--> 262 allow_broadcast=True)
263
264
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py in _constant_impl(value, dtype, shape, name, verify_shape, allow_broadcast)
298 tensor_util.make_tensor_proto(
299 value, dtype=dtype, shape=shape, verify_shape=verify_shape,
--> 300 allow_broadcast=allow_broadcast))
301 dtype_value = attr_value_pb2.AttrValue(type=tensor_value.tensor.dtype)
302 const_tensor = g._create_op_internal( # pylint: disable=protected-access
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape, allow_broadcast)
449 nparray = np.empty(shape, dtype=np_dt)
450 else:
--> 451 _AssertCompatible(values, dtype)
452 nparray = np.array(values, dtype=np_dt)
453 # check to them.
~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in _AssertCompatible(values, dtype)
326 [mismatch] = e.args
327 if dtype is None:
--> 328 raise TypeError("Expected any non-tensor type, got a tensor instead.")
329 else:
330 raise TypeError("Expected %s, got %s of type '%s' instead." %
TypeError: Expected any non-tensor type, got a tensor instead.
I had this error when I converted a function argument of int datatype to tf.constant . I resolved the issue in my case by undoing it. I faced this issue when I was converting TF1 codes to TF2.3.0 . Looking at your error trace I can see it's pointed to handling some constants in tf-chatbot. Kindly check how that constant is handled.
This is a fixed issue in TensorFlow 2.3.0 onwards. Can you upgrade your TensorFlow version?
pip install tensorflow==2.3.0
pip install --upgrade tensorflow

"TypeError: unsupported callable" when saving keras model using tensorlow

When saving a Keras model defined like this:
# define model
model = Sequential()
model.add(LSTM(200, activation='relu', input_shape=(n_steps_in, n_features)))
model.add(RepeatVector(n_steps_out))
model.add(LSTM(200, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(n_features)))
model.compile(optimizer='adam', loss='mse')
model.save(path)
I got this following message:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~/anaconda3/envs/topic_forecaster/lib/python3.7/inspect.py in getfullargspec(func)
1125 skip_bound_arg=False,
-> 1126 sigcls=Signature)
1127 except Exception as ex:
~/anaconda3/envs/topic_forecaster/lib/python3.7/inspect.py in _signature_from_callable(obj, follow_wrapper_chains, skip_bound_arg, sigcls)
2287 return _signature_from_builtin(sigcls, obj,
-> 2288 skip_bound_arg=skip_bound_arg)
2289
~/anaconda3/envs/topic_forecaster/lib/python3.7/inspect.py in _signature_from_builtin(cls, func, skip_bound_arg)
2111 if not s:
-> 2112 raise ValueError("no signature found for builtin {!r}".format(func))
2113
ValueError: no signature found for builtin <tensorflow.python.keras.saving.saved_model.save_impl.LayerCall object at 0x7f8c1f357190>
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
<ipython-input-9-3fe80778ab16> in <module>
3 path = Path.cwd().parent / 'models' / 'tpi'
4 Path(path).mkdir(parents=True, exist_ok=True)
----> 5 model.save(str(path))
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in save(self, filepath, overwrite, include_optimizer, save_format, signatures, options)
973 """
974 saving.save_model(self, filepath, overwrite, include_optimizer, save_format,
--> 975 signatures, options)
976
977 def save_weights(self, filepath, overwrite=True, save_format=None):
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/save.py in save_model(model, filepath, overwrite, include_optimizer, save_format, signatures, options)
113 else:
114 saved_model_save.save(model, filepath, overwrite, include_optimizer,
--> 115 signatures, options)
116
117
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save.py in save(model, filepath, overwrite, include_optimizer, signatures, options)
72 # default learning phase placeholder.
73 with K.learning_phase_scope(0):
---> 74 save_lib.save(model, filepath, signatures, options)
75
76 if not include_optimizer:
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/saved_model/save.py in save(obj, export_dir, signatures, options)
868 if signatures is None:
869 signatures = signature_serialization.find_function_to_export(
--> 870 checkpoint_graph_view)
871
872 signatures = signature_serialization.canonicalize_signatures(signatures)
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/saved_model/signature_serialization.py in find_function_to_export(saveable_view)
62 # If the user did not specify signatures, check the root object for a function
63 # that can be made into a signature.
---> 64 functions = saveable_view.list_functions(saveable_view.root)
65 signature = functions.get(DEFAULT_SIGNATURE_ATTR, None)
66 if signature is not None:
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/saved_model/save.py in list_functions(self, obj)
139 if obj_functions is None:
140 obj_functions = obj._list_functions_for_serialization( # pylint: disable=protected-access
--> 141 self._serialization_cache)
142 self._functions[obj] = obj_functions
143 return obj_functions
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py in _list_functions_for_serialization(self, serialization_cache)
2420 def _list_functions_for_serialization(self, serialization_cache):
2421 return (self._trackable_saved_model_saver
-> 2422 .list_functions_for_serialization(serialization_cache))
2423
2424
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/base_serialization.py in list_functions_for_serialization(self, serialization_cache)
89 `ConcreteFunction`.
90 """
---> 91 fns = self.functions_to_serialize(serialization_cache)
92
93 # The parent AutoTrackable class saves all user-defined tf.functions, and
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/layer_serialization.py in functions_to_serialize(self, serialization_cache)
77 def functions_to_serialize(self, serialization_cache):
78 return (self._get_serialized_attributes(
---> 79 serialization_cache).functions_to_serialize)
80
81 def _get_serialized_attributes(self, serialization_cache):
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/layer_serialization.py in _get_serialized_attributes(self, serialization_cache)
92
93 object_dict, function_dict = self._get_serialized_attributes_internal(
---> 94 serialization_cache)
95
96 serialized_attr.set_and_validate_objects(object_dict)
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/model_serialization.py in _get_serialized_attributes_internal(self, serialization_cache)
51 objects, functions = (
52 super(ModelSavedModelSaver, self)._get_serialized_attributes_internal(
---> 53 serialization_cache))
54 functions['_default_save_signature'] = default_signature
55 return objects, functions
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/layer_serialization.py in _get_serialized_attributes_internal(self, serialization_cache)
101 """Returns dictionary of serialized attributes."""
102 objects = save_impl.wrap_layer_objects(self.obj, serialization_cache)
--> 103 functions = save_impl.wrap_layer_functions(self.obj, serialization_cache)
104 # Attribute validator requires that the default save signature is added to
105 # function dict, even if the value is None.
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in wrap_layer_functions(layer, serialization_cache)
154 # Reset the losses of the layer and its children. The call function in each
155 # child layer is replaced with tf.functions.
--> 156 original_fns = _replace_child_layer_functions(layer, serialization_cache)
157 original_losses = _reset_layer_losses(layer)
158
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in _replace_child_layer_functions(layer, serialization_cache)
246 layer_fns = (
247 child_layer._trackable_saved_model_saver._get_serialized_attributes(
--> 248 serialization_cache).functions)
249 else:
250 layer_fns = (
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/layer_serialization.py in _get_serialized_attributes(self, serialization_cache)
92
93 object_dict, function_dict = self._get_serialized_attributes_internal(
---> 94 serialization_cache)
95
96 serialized_attr.set_and_validate_objects(object_dict)
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/layer_serialization.py in _get_serialized_attributes_internal(self, serialization_cache)
101 """Returns dictionary of serialized attributes."""
102 objects = save_impl.wrap_layer_objects(self.obj, serialization_cache)
--> 103 functions = save_impl.wrap_layer_functions(self.obj, serialization_cache)
104 # Attribute validator requires that the default save signature is added to
105 # function dict, even if the value is None.
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in wrap_layer_functions(layer, serialization_cache)
164 call_fn_with_losses = call_collection.add_function(
165 _wrap_call_and_conditional_losses(layer),
--> 166 '{}_layer_call_and_return_conditional_losses'.format(layer.name))
167 call_fn = call_collection.add_function(
168 _extract_outputs_from_fn(layer, call_fn_with_losses),
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in add_function(self, call_fn, name)
492 # Manually add traces for layers that have keyword arguments and have
493 # a fully defined input signature.
--> 494 self.add_trace(*self._input_signature)
495 return fn
496
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in add_trace(self, *args, **kwargs)
411 fn.get_concrete_function(*args, **kwargs)
412
--> 413 trace_with_training(True)
414 trace_with_training(False)
415 else:
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in trace_with_training(value, fn)
409 utils.set_training_arg(value, self._training_arg_index, args, kwargs)
410 with K.learning_phase_scope(value):
--> 411 fn.get_concrete_function(*args, **kwargs)
412
413 trace_with_training(True)
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in get_concrete_function(self, *args, **kwargs)
536 if not self.call_collection.tracing:
537 self.call_collection.add_trace(*args, **kwargs)
--> 538 return super(LayerCall, self).get_concrete_function(*args, **kwargs)
539
540
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in get_concrete_function(self, *args, **kwargs)
774 if self._stateful_fn is None:
775 initializer_map = object_identity.ObjectIdentityDictionary()
--> 776 self._initialize(args, kwargs, add_initializers_to=initializer_map)
777 self._initialize_uninitialized_variables(initializer_map)
778
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
406 self._concrete_stateful_fn = (
407 self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
--> 408 *args, **kwds))
409
410 def invalid_creator_scope(*unused_args, **unused_kwds):
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
1846 if self.input_signature:
1847 args, kwargs = None, None
-> 1848 graph_function, _, _ = self._maybe_define_function(args, kwargs)
1849 return graph_function
1850
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _maybe_define_function(self, args, kwargs)
2148 graph_function = self._function_cache.primary.get(cache_key, None)
2149 if graph_function is None:
-> 2150 graph_function = self._create_graph_function(args, kwargs)
2151 self._function_cache.primary[cache_key] = graph_function
2152 return graph_function, args, kwargs
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
2039 arg_names=arg_names,
2040 override_flat_arg_shapes=override_flat_arg_shapes,
-> 2041 capture_by_value=self._capture_by_value),
2042 self._function_attributes,
2043 # Tell the ConcreteFunction to clean up its graph once it goes out of
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
913 converted_func)
914
--> 915 func_outputs = python_func(*func_args, **func_kwargs)
916
917 # invariant: `func_outputs` contains only Tensors, CompositeTensors,
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in wrapped_fn(*args, **kwds)
356 # __wrapped__ allows AutoGraph to swap in a converted function. We give
357 # the function a weak reference to itself to avoid a reference cycle.
--> 358 return weak_wrapped_fn().__wrapped__(*args, **kwds)
359 weak_wrapped_fn = weakref.ref(wrapped_fn)
360
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in wrapper(*args, **kwargs)
513 layer, inputs=inputs, build_graph=False, training=training,
514 saving=True):
--> 515 ret = method(*args, **kwargs)
516 _restore_layer_losses(original_losses)
517 return ret
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/utils.py in wrap_with_training_arg(*args, **kwargs)
109 training,
110 lambda: replace_training_and_call(True),
--> 111 lambda: replace_training_and_call(False))
112
113 # Create arg spec for decorated function. If 'training' is not defined in the
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/utils/tf_utils.py in smart_cond(pred, true_fn, false_fn, name)
57 pred, true_fn=true_fn, false_fn=false_fn, name=name)
58 return smart_module.smart_cond(
---> 59 pred, true_fn=true_fn, false_fn=false_fn, name=name)
60
61
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/framework/smart_cond.py in smart_cond(pred, true_fn, false_fn, name)
52 if pred_value is not None:
53 if pred_value:
---> 54 return true_fn()
55 else:
56 return false_fn()
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/utils.py in <lambda>()
108 return tf_utils.smart_cond(
109 training,
--> 110 lambda: replace_training_and_call(True),
111 lambda: replace_training_and_call(False))
112
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/utils.py in replace_training_and_call(training)
104 def replace_training_and_call(training):
105 set_training_arg(training, training_arg_index, args, kwargs)
--> 106 return wrapped_call(*args, **kwargs)
107
108 return tf_utils.smart_cond(
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saved_model/save_impl.py in call_and_return_conditional_losses(inputs, *args, **kwargs)
555 layer_call = _get_layer_call_method(layer)
556 def call_and_return_conditional_losses(inputs, *args, **kwargs):
--> 557 return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs)
558 return _create_call_fn_decorator(layer, call_and_return_conditional_losses)
559
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/layers/wrappers.py in call(self, inputs, training, mask)
218 def call(self, inputs, training=None, mask=None):
219 kwargs = {}
--> 220 if generic_utils.has_arg(self.layer.call, 'training'):
221 kwargs['training'] = training
222
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/keras/utils/generic_utils.py in has_arg(fn, name, accept_all)
302 bool, whether `fn` accepts a `name` keyword argument.
303 """
--> 304 arg_spec = tf_inspect.getfullargspec(fn)
305 if accept_all and arg_spec.varkw is not None:
306 return True
~/anaconda3/envs/topic_forecaster/lib/python3.7/site-packages/tensorflow_core/python/util/tf_inspect.py in getfullargspec(obj)
255 if d.decorator_argspec is not None:
256 return _convert_maybe_argspec_to_fullargspec(d.decorator_argspec)
--> 257 return _getfullargspec(target)
258
259
~/anaconda3/envs/topic_forecaster/lib/python3.7/inspect.py in getfullargspec(func)
1130 # else. So to be fully backwards compatible, we catch all
1131 # possible exceptions here, and reraise a TypeError.
-> 1132 raise TypeError('unsupported callable') from ex
1133
1134 args = []
TypeError: unsupported callable
However, when saving a regular model without a TimeDistributed layer as below, it worked fine:
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10)
])
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
loss=loss_fn,
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test, verbose=2)
model.save('temp', save_format='tf')
By upgrading Tensorflow to 2.1 with CUDA10 using Anaconda as below, the problem is solved.
conda create -n tf-gpu-cuda10 tensorflow-gpu=2.1 cudatoolkit=10
conda activate tf-gpu-cuda10

ResNet model in Tensorflow Federated

I tried to customize the model in "Image classification" tutorial in Tensorflow Federated. (It originally used a sequential model)
I use Keras ResNet50 but when it began to train, there is always an error "Incompatible shapes"
Here are my codes:
NUM_CLIENTS = 4
NUM_EPOCHS = 10
BATCH_SIZE = 2
SHUFFLE_BUFFER = 5
def create_compiled_keras_model():
model = tf.keras.applications.resnet.ResNet50(include_top=False, weights='imagenet',
input_tensor=tf.keras.layers.Input(shape=(100,
300, 3)), pooling=None)
model.compile(
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
optimizer=tf.keras.optimizers.SGD(learning_rate=0.02),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
return model
def model_fn():
keras_model = create_compiled_keras_model()
return tff.learning.from_compiled_keras_model(keras_model, sample_batch)
iterative_process = tff.learning.build_federated_averaging_process(model_fn)
Error information:
enter image description here
I feel that the shape is incompatible because the epoch and clients information were somehow missing. Would be very thankful if someone could give me a hint.
Updates:
The Assertion error happened during tff.learning.build_federated_averaging_process
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-164-dac26193d9d8> in <module>()
----> 1 iterative_process = tff.learning.build_federated_averaging_process(model_fn)
2
3 # iterative_process = build_federated_averaging_process(model_fn)
13 frames
/usr/local/lib/python3.6/dist-packages/tensorflow_federated/python/learning/federated_averaging.py in build_federated_averaging_process(model_fn, server_optimizer_fn, client_weight_fn, stateful_delta_aggregate_fn, stateful_model_broadcast_fn)
165 return optimizer_utils.build_model_delta_optimizer_process(
166 model_fn, client_fed_avg, server_optimizer_fn,
--> 167 stateful_delta_aggregate_fn, stateful_model_broadcast_fn)
/usr/local/lib/python3.6/dist-packages/tensorflow_federated/python/learning/framework/optimizer_utils.py in build_model_delta_optimizer_process(model_fn, model_to_client_delta_fn, server_optimizer_fn, stateful_delta_aggregate_fn, stateful_model_broadcast_fn)
349 # still need this.
350 with tf.Graph().as_default():
--> 351 dummy_model_for_metadata = model_utils.enhance(model_fn())
352
353 # ===========================================================================
<ipython-input-159-b2763ace8e5b> in model_fn()
1 def model_fn():
2 keras_model = model
----> 3 return tff.learning.from_compiled_keras_model(keras_model, sample_batch)
/usr/local/lib/python3.6/dist-packages/tensorflow_federated/python/learning/keras_utils.py in from_compiled_keras_model(keras_model, dummy_batch)
211 # Model.test_on_batch() once before asking for metrics.
212 if isinstance(dummy_tensors, collections.Mapping):
--> 213 keras_model.test_on_batch(**dummy_tensors)
214 else:
215 keras_model.test_on_batch(*dummy_tensors)
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py in test_on_batch(self, x, y, sample_weight, reset_metrics)
1007 sample_weight=sample_weight,
1008 reset_metrics=reset_metrics,
-> 1009 standalone=True)
1010 outputs = (
1011 outputs['total_loss'] + outputs['output_losses'] + outputs['metrics'])
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2_utils.py in test_on_batch(model, x, y, sample_weight, reset_metrics, standalone)
503 y,
504 sample_weights=sample_weights,
--> 505 output_loss_metrics=model._output_loss_metrics)
506
507 if reset_metrics:
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py in __call__(self, *args, **kwds)
568 xla_context.Exit()
569 else:
--> 570 result = self._call(*args, **kwds)
571
572 if tracing_count == self._get_tracing_count():
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py in _call(self, *args, **kwds)
606 # In this case we have not created variables on the first call. So we can
607 # run the first trace but we should fail if variables are created.
--> 608 results = self._stateful_fn(*args, **kwds)
609 if self._created_variables:
610 raise ValueError("Creating variables on a non-first call to a function"
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py in __call__(self, *args, **kwargs)
2407 """Calls a graph function specialized to the inputs."""
2408 with self._lock:
-> 2409 graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
2410 return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
2411
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py in _maybe_define_function(self, args, kwargs)
2765
2766 self._function_cache.missed.add(call_context_key)
-> 2767 graph_function = self._create_graph_function(args, kwargs)
2768 self._function_cache.primary[cache_key] = graph_function
2769 return graph_function, args, kwargs
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
2655 arg_names=arg_names,
2656 override_flat_arg_shapes=override_flat_arg_shapes,
-> 2657 capture_by_value=self._capture_by_value),
2658 self._function_attributes,
2659 # Tell the ConcreteFunction to clean up its graph once it goes out of
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
979 _, original_func = tf_decorator.unwrap(python_func)
980
--> 981 func_outputs = python_func(*func_args, **func_kwargs)
982
983 # invariant: `func_outputs` contains only Tensors, CompositeTensors,
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py in wrapped_fn(*args, **kwds)
437 # __wrapped__ allows AutoGraph to swap in a converted function. We give
438 # the function a weak reference to itself to avoid a reference cycle.
--> 439 return weak_wrapped_fn().__wrapped__(*args, **kwds)
440 weak_wrapped_fn = weakref.ref(wrapped_fn)
441
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py in wrapper(*args, **kwargs)
966 except Exception as e: # pylint:disable=broad-except
967 if hasattr(e, "ag_error_metadata"):
--> 968 raise e.ag_error_metadata.to_exception(e)
969 else:
970 raise
AssertionError: in user code:
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_eager.py:345 test_on_batch *
with backend.eager_learning_phase_scope(0):
/usr/lib/python3.6/contextlib.py:81 __enter__
return next(self.gen)
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/backend.py:425 eager_learning_phase_scope
assert ops.executing_eagerly_outside_functions()
AssertionError:
Ah, I believe this issue is coming from mismatched expectations on sample_batch. TFF passes sample_batch to Keras, which calls a forward pass with this sample batch to initialize various attributes of the keras model. sample_batch should be either a sample from the literal data you are going to be feeding the model as on the server side, or a batch of fake data which matches the shape and type of the data you will be passing in.
An example of the former can be found here (this uses tf.data.Dataset), and there are several examples of the latter in test code, like here.
From what I see of the definition of the model, likely the x element of your sample_batch should be an ndarray of shape [2, 100, 300, 3] (where 2 is for the batch size, but technically this can be any nonzero dimension), and the y element should also match the expected y structure in the data you are using.
I hope this helps, just ping back if there are any problems!
One thing to note, that may be helpful in thinking about TFF--TFF is building a syntax tree representing the distributed computation you are defining via build_federated_averaging_process. This error actually occurs during construction of this object. TFF must trace the computation you pass it in order to know what structure to generate, and this is what is raising here. Actual training of the model happens when you call next on the returned IterativeProcess.
I have same problem:
if I execute this line
state, metrics = iterative_process.next(state, federated_train_data)
print('round 1, metrics={}'.format(metrics))
I find this error
InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Default MaxPoolingOp only supports NHWC on device type CPU
[[{{node StatefulPartitionedCall/StatefulPartitionedCall/sequential/vgg16/block1_pool/MaxPool}}]]
[[subcomputation/StatefulPartitionedCall_1/ReduceDataset]]
[[subcomputation/StatefulPartitionedCall_1/ReduceDataset/_140]]
(1) Invalid argument: Default MaxPoolingOp only supports NHWC on device type CPU
[[{{node StatefulPartitionedCall/StatefulPartitionedCall/sequential/vgg16/block1_pool/MaxPool}}]]
[[subcomputation/StatefulPartitionedCall_1/ReduceDataset]]
0 successful operations.
0 derived errors ignored.
knowin that I employe VGG16
have you any idea on this type of error