Invalid argument: TypeError: an integer is required (got type NoneType) when closing Tensorflow - tensorflow

Question

I'm implementing the IMPALA framework https://github.com/deepmind/scalable_agent which uses tensorflows multiprocessing. Everything seems to work fine as the experiment ran for the specified number of steps... however I start to get a type error when tensorflow is closing processes.
Does someone know what could've contributed to this error and how to solve it? This seems to be something with tensorflow rather than the original code itself.
INFO:tensorflow:Closing all processes.
[750. 450. 25.125]
2020-01-13 17:01:00.624889: W tensorflow/core/framework/op_kernel.cc:1389] Invalid argument: TypeError: an integer is required (got type NoneType)
Traceback (most recent call last):
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 207, in __call__
ret = func(*args)
File "/home/haianh/anaconda3/envs/lab/scalable_agent/py_process.py", line 86, in py_call
result = self._out.recv()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
return self._recv(size)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
TypeError: an integer is required (got type NoneType)
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, TypeError: an integer is required (got type NoneType)
Traceback (most recent call last):
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 207, in __call__
ret = func(*args)
File "/home/haianh/anaconda3/envs/lab/scalable_agent/py_process.py", line 86, in py_call
result = self._out.recv()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
return self._recv(size)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
TypeError: an integer is required (got type NoneType)
[[{{node scan/while/flow_environment_step/step}}]]
INFO:tensorflow:All processes closed.
Traceback (most recent call last):
File "experiment.py", line 689, in <module>
tf.app.run()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "experiment.py", line 683, in main
train(action_set, level_names)
File "experiment.py", line 630, in train
session.run(enqueue_ops)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 788, in __exit__
self._close_internal(exception_type)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 826, in _close_internal
self._sess.close()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1082, in close
self._sess.close()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1244, in close
ignore_live_threads=True)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/six.py", line 696, in reraise
raise value
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/queue_runner_impl.py", line 257, in _run
enqueue_callable()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1257, in _single_operation_run
self._call_tf_sessionrun(None, {}, [], target_list, None)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: an integer is required (got type NoneType)
Traceback (most recent call last):
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 207, in __call__
ret = func(*args)
File "/home/haianh/anaconda3/envs/lab/scalable_agent/py_process.py", line 86, in py_call
result = self._out.recv()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
return self._recv(size)
File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
TypeError: an integer is required (got type NoneType)
[[{{node scan/while/flow_environment_step/step}}]]