vhost in rabbitmq is not starting - rabbitmq

Question

My rabbitmq was working fine. Suddenly, one of the vhosts is not starting up. On restarting rabbitmq server, it shows in admin UI, that error is starting up vhost, and all the queues are in down state. Here is the error when I restart the vhost. Please suggest. Our production message broker is down, need immediate help.
Rabbitmq ver. - 3.8.3
Erlang ver. - 22.3
Trying to restart vhost 'r_t' on node 'rabbit#myserver' ...
Error:
Failed to start vhost 'r_t' on node 'rabbit#myserver'Reason: {:shutdown, {:failed_to_start_child,
:rabbit_vhost_process, {:error, {{{:badarg, [{:erlang, :binary_to_term, [<<131, 104, 6, 100, 0, 13, 98,
97, 115, 105, 99, 95, 109, 101, 115, 115, 97, 103, 101, 104, 4, 100, 0, 8, 114, 101, 115, 111, 117, 114,
99, 101, 109, 0, ...>>], []}, {:rabbit_queue_index, :parse_pub_record_body, 2, [file:
'src/rabbit_queue_index.erl', line: 783]}, {:rabbit_queue_index, :"-segment_entries_foldr/3-fun-0-", 4,
[file: 'src/rabbit_queue_index.erl', line: 1111]}, {:array, :sparse_foldr_3, 6, [file: 'array.erl', line:
1847]}, {:array, :sparse_foldr_2, 8, [file: 'array.erl', line: 1836]}, {:rabbit_queue_index,
:scan_queue_segments, 3, [file: 'src/rabbit_queue_index.erl', line: 741]}, {:rabbit_queue_index,
:queue_index_walker_reader, 2, [file: 'src/rabbit_queue_index.erl', line: 728]}, {:rabbit_queue_index,
:"-queue_index_walker/1-fun-1-", 2, [file: 'src/rabbit_queue_index.erl', line: 710]}]}, {:gen_server2,
:call, [#PID<10691.1882.0>, :out, :infinity]}}, {:child, :undefined, :msg_store_persistent,
{:rabbit_msg_store, :start_link, [:msg_store_persistent,
'/var/lib/rabbitmq/mnesia/rabbit#1myserver/msg_stores/vhosts/1SLGRHB3T7STV1U1TEB4MR6QS', [],
{#Function<2.23124100/1 in :rabbit_queue_index>, {:start, [{:resource, "r_t", :queue,
"product.import_royn_se"}, {:resource, "r_t", :queue, "customer.import_ronin_es"}, {:resource, "r_t",
...}, {:resource, ...}, {...}, ...]}}]}, :transient, 30000, :worker, [:rabbit_msg_store]}}}}}

Answer 1

I got a workaround.
We just exported and saved vhost definitions for existing vhost which was not starting. The deleted this vhost. Created same vhost and imported the definition back. So, we got all queues with same features.

Answer 2

-> For dump all messages
from os import listdir
from os.path import isfile, join
import re
base_path = '/docker/rabbitmq-nfe/data/mnesia/rabbit#rabbitmq_nfe/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/queues'
list_queues = listdir(base_path)
#GET ALL QUEUES
for queue in list_queues:
dir_queue = '%s/%s' % (base_path, queue)
list_files = [f for f in listdir(dir_queue) if isfile(join(dir_queue, f)) and f not in ['.queue_name', 'journal.jif']]
list_files.sort()
name_queue = open('%s/.queue_name' % (dir_queue), 'r').read().split('QUEUE: ')[-1][:-1]
payload_queue = []
#GET ALL FILES QUEUES
for file in list_files:
path_file = '%s/%s' % (dir_queue, file)
binary_file = open(path_file, 'r')
string_file = binary_file.read()
string_file_decoded = string_file.decode('iso 8859-1')
#GET ALL PAYLOAD FILES QUEUES
list_payload_queue = ['{%s}' % f for f in re.split('\{(.*?)\}', string_file_decoded) if 'chave' in f and 'rede' in f]
for idx, payload in enumerate(list_payload_queue):
if payload.count('{') > 1:
list_payload_queue[idx] = ['{%s' % f for f in payload.split('{') if 'chave' in f and 'rede' in f][0]
payload_queue = payload_queue + list_payload_queue
#SAVE BACKUP QUEUES
print('FILA: %s ARQUIVOS: %s' % (name_queue, str(len(payload_queue))))
with open('/tmp/%s.log' % name_queue, 'w') as f:
for line in payload_queue:
f.write("%s\n" % line)