Evaluating the state value function when using the SAC agent of TF-Agents - tensorflow2.0

Question

The state value function v at states x is a quantity of interest of the Markov decision process (MDP) which I intend to solve. (My MDP is fully observable: observation = state.)
I use the SAC agent of TF-agents to learn action value function q(x,a) and policy π. Thus given a state x, the policy returns an approximately optimal action a = π(x) so that v(x) ≈ q(x,π(x)).
Problem description: How can one write q(x,π(x)) as a TF-Agents expression?
I can examine the problem already with the SAC tutorial https://www.tensorflow.org/agents/tutorials/7_SAC_minitaur_tutorial by adding the following lines to the end of the tutorial:
# Resetting the environment to obtain a TimeStep object
time_step = env.reset()
# An observation which respects the observation specs of env, corresponding to x above
observation = time_step.observation
# Calling the evaluation policy we obtain an action, this is essentially π(x) above
action = eval_policy.action(time_step).action
# I was expecting that the next line would return q(x,π(x))
critic_net((observation,action))
The reason for the last line was that the input_tensor_spec of a CriticNetwork was described as a tuple of (observation, action) in https://www.tensorflow.org/agents/api_docs/python/tf_agents/agents/ddpg/critic_network/CriticNetwork.
However instead critic_net((observation,action)) raises the following error:
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-32-8446b099696b> in <module>
----> 1 critic_net((observation,action))
2 frames
/usr/local/lib/python3.8/dist-packages/tf_agents/networks/network.py in __call__(self, inputs, *args, **kwargs)
425 normalized_kwargs.pop("network_state", None)
426
--> 427 outputs, new_state = super(Network, self).__call__(**normalized_kwargs) # pytype: disable=attribute-error # typed-keras
428
429 nest_utils.assert_matching_dtypes_and_inner_shapes(
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb
/usr/local/lib/python3.8/dist-packages/tf_agents/agents/ddpg/critic_network.py in call(***failed resolving arguments***)
166 actions = layer(actions, training=training)
167
--> 168 joint = tf.concat([observations, actions], 1)
169 for layer in self._joint_layers:
170 joint = layer(joint, training=training)
InvalidArgumentError: Exception encountered when calling layer 'CriticNetwork' (type CriticNetwork).
{{function_node __wrapped__ConcatV2_N_2_device_/job:localhost/replica:0/task:0/device:CPU:0}} ConcatOp : Dimension 0 in both shapes must be equal: shape[0] = [28,1] vs. shape[1] = [8,1] [Op:ConcatV2] name: concat
Call arguments received by layer 'CriticNetwork' (type CriticNetwork):
• inputs=('tf.Tensor(shape=(28,), dtype=float32)', 'tf.Tensor(shape=(8,), dtype=float32)')
• step_type=()
• network_state=()
• training=False
Can someone help me with the evaluation of the critic network?