How to parse rabbitmq status output? - rabbitmq

Question

I installed RabbitMQ on Linux, it's a great piece of software.
When I run this command:
sudo rabbitmqctl status
I get a mess of output:
[{pid,18665},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","3.1.5"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.1.5"},
{webmachine,"webmachine","1.10.3-rmq3.1.5-gite9359c7"},
{mochiweb,"MochiMedia Web Server","2.7.0-rmq3.1.5-git680dba8"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.1.5"},
{rabbit,"RabbitMQ","3.1.5"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{inets,"INETS CXC 138 49","5.7.1"},
{xmerl,"XML parser","1.2.10"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{amqp_client,"RabbitMQ AMQP Client","3.1.5"},
{sasl,"SASL CXC 138 11","2.1.10"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [rq:1] [async-threads:30] [kernel-poll:true]\n"},
{memory,
[{total,179426464},
{connection_procs,300224},
{queue_procs,14434024},
{plugins,474968},
{other_proc,9607952},
{mnesia,89264},
{mgmt_db,1539936},
{msg_index,85175152},
{other_ets,29060560},
{binary,18243208},
{code,17504466},
{atom,1602617},
{other_system,1394093}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,1522479923},
{disk_free_limit,1000000000},
{disk_free,58396659712},
{file_descriptors,
[{total_limit,924},{total_used,17},{sockets_limit,829},{sockets_used,4}]},
{processes,[{limit,1048576},{used,233}]},
{run_queue,0},
{uptime,5169640}]
It looks like JSON, but it's not.
What data format is this? And how did you find out?
Closest thing I can find is this: http://erlang.org/doc/man/yecc.html

Answer 1

rather than querying the rabbitctrl process I suggest querying the REST api which will return JSON.
GET: http://localhost:15672/api/overview
Here is the documentation:
http://hg.rabbitmq.com/rabbitmq-management/raw-file/3646dee55e02/priv/www-api/help.html

Answer 2

rabbitmqctl has a --formatter flag to request JSON formatted output optionally available. For instance:
sudo rabbitmqctl status --formatter json | jq .disk_free_limit
50000000

Answer 3

The output format of rabbitmqctl is an Erlang term or also Erlang ETF (External Term Format).
You can use the python library erl_terms to transform the output into something usable by Python:
from erl_terms import decode
from os import getuid
from re import sub
from subprocess import check_output
check_command = ['/usr/sbin/rabbitmqctl', '-q', 'status']
if getuid() != 0:
check_command.insert(0, '/usr/bin/sudo')
status = check_output(check_command)
## Join into a single line string then add a period at the end to make it a valid erlang term
status = ''.join(status.splitlines()) + '.'
# Remove any literal \n's since the erlang_version item has one in it
status = sub('(?:\\\\n)+', '', status)
# Decode this into a python object
status = decode(status)
# And now let's find just mem_stat for mgmt_db
for item in status[0]:
if 'memory' in item:
for mem_stat in item[1]:
if 'mgmt_db' in mem_stat:
print mem_stat[1]

Answer 4

Here is a method I made in python for the purpose, which only requires PyYAML (available in pypi). First revision, might be sub-optimal or buggy, but works for me:
import re
import subprocess
import yaml
def fix_dicts(json_str_list, pos):
'''this recursive function puts all comma-separted values into square
brackets to make data look like normal 'key: value' dicts'''
quoted_string = False
value = True
value_pos = 0
commas = False
is_list = False
in_list = 0
while pos < len(json_str_list):
if not quoted_string:
if json_str_list[pos] == '{':
json_str_list, pos = fix_dicts(json_str_list, pos+1)
elif json_str_list[pos] == '"':
quoted_string = True
elif json_str_list[pos] == ':':
value = True
value_pos = pos + 1
elif json_str_list[pos] == '[':
if value and not commas:
is_list = True
in_list += 1
elif json_str_list[pos] == ']':
in_list -= 1
elif json_str_list[pos] == ',':
commas = True
if not in_list:
is_list = False
elif json_str_list[pos] == '}':
if not is_list and commas:
json_str_list = (json_str_list[:value_pos] + ['['] +
json_str_list[value_pos:pos] + [']'] +
json_str_list[pos:])
pos += 2
return json_str_list, pos
elif json_str_list[pos] == '"':
quoted_string = False
pos += 1
return json_str_list, pos
def squash_dicts(input_data):
# recursively converts [{a:1},{b:2},{c:3}...] into {a:1, b:2, c:3}'''
if type(input_data) is list:
for i in range(len(input_data)):
input_data[i] = squash_dicts(input_data[i])
if all([type(e) is dict for e in input_data]):
input_data = dict([(k,v) for e in input_data for k,v in e.items()])
elif type(input_data) is dict:
for k, v in input_data.items():
input_data[k] = squash_dicts(v)
return input_data
text = subprocess.check_output(['rabbitmqctl','status'])
text = text.splitlines()
text = text[1:] # skip the first line "Status of node..."
text = ''.join(text) # back into string for regex processing
# quote strings
bad_yaml = re.sub(r'([,{])([a-z_A-Z]+)([,}])', r'\1"\2"\3', text)
# change first element into a key - replacing ',' with ':'
bad_yaml = re.sub(r'({[^,]+),',r'\1:', bad_yaml)
bad_yaml_list = list(bad_yaml) # into a list for another fix
good_yaml, _ = fix_dicts(bad_yaml_list, 0)
status_list = yaml.load(''.join(good_yaml))
status_dict = squash_dicts(status_list)
# now we can use "status_dict" - it's an ordinary dict
print(yaml.safe_dump(status_dict, default_flow_style=False))

Answer 5

If anyone is interested in using the command line util instead of the UI plugin, you can use this open-source simple parser I wrote:
https://github.com/yuvaltir/RabbitMQNet
Example of use:
// assume file RmqStatus.txt includes the outputof the rabbitmqctl status command and that the node name is 'serverx'
var statusLines = File.ReadAllText("RmqStatus.txt");
RabbitStatusParser parser = new RabbitStatusParser();
var res = parser.ParseText(statusLines.Replace(Environment.NewLine,string.Empty));
var keyWord1 = "Status of node 'serverx#serverx'.memory.mgmt_db"
var mgmtDbMem = res[keyWord1];
var keyWord2 = "Status of node 'serverx#serverx'.file_descriptors.sockets_used"
var socketsUsed = res[keyWord2];