RabbitMQ with MQTT sends message on connect - rabbitmq

I'm using mqtt-launcher (https://github.com/jpmens/mqtt-launcher) to execute commands when a certain MQTT message with the payload "0" was received.
Here is the config
logfile = '/home/user/mqtt-launcher/logfile'
mqtt_broker = 'broker' # default: 'localhost'. If using TLS, this must be set to the domain name signed by$
mqtt_port = 1883 # default: 1883
mqtt_clientid = 'mqtt-launcher-1'
mqtt_username = ''
mqtt_password = ''
mqtt_tls = None # default: No TLS
topiclist = {
# topic payload value program & arguments
"channel/dostuff" : {
'0' : [
'/usr/bin/ssh',
'-i',
'/home/user/.ssh/privatekey',
'user#host',
'script.sh'
]
}
}
Everytime, I start the python script, the shell script is executed twice.
But I want it to execute only once if the MQTT message with the payload "0" is sent.
I made sure the queue which is implicitly created when subscribing was empty before by purging it, then starting mqtt-launcher but still the script is execute twice after the program connected.
When I run user#localhost:~$ mosquitto_sub -h broker -p 1883 -t 'channel/dostuff' -v -u 'user' -P 'mysecurepassword' I get channel/dostuff 0
I'm not familiar with mosquitto but I think that this means I receive a message, right?
I turned of the retain option, restarted openHAB and RabbitMQ, but still the message is sent. Here is the openHAB mqtt.cfg:
broker.url=tcp://broker:1883
broker.user=openhab
broker.pwd=mysecurepassword
broker.qos=1
broker.retain=false
broker.async=false

You have published a message with the payload 0 and the retained bit set.
This means that when ever a client subscribes to that topic the last message with the retained bit set will be delivered to that client.
You can clear the retained message by publishing a message with the retained bit set and a null payload to the same topic. You can do this with the mosquitto_pub command as follows:
mosquitto_pub -t "channel/dostuff" -u 'user' -P 'password' -r -n
You should make sure what ever you are using to publish the message normally is not setting the retained bit.

Related

Sending messages to durable queue using Rabbit perf test tool

How can I send messages to a durable queue using the Rabbit Perf Test tool? I am getting this following error:
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'durable' for queue 'myservice_q' in vhost 'myvhost': received 'false' but current is 'true', class-id=50, method-id=10)
A command I tried is:
bin/runjava com.rabbitmq.perf.PerfTest -h amqps://myrabbit.instance.com:5671/myvhost -u myservice_q -ad false -pmessages 10 -y0 -f mandatory -f persistent
I used the -f flag with both values persistent and mandatory and it did not work. I tried other flags offered by the tool and none worked. Can someone tell me how this can be done?
I was able to do it by using both the flags -f mandatory -f persistent. It worked.

RabbitMQ channel has no consumer

I am trying to send a message to RabbitMQ via MQTT, however when the connection is established, the channel shows that is has ... no consumers .... To my understanding, upon connecting via MQTT to RabbitMQ, the topic will default to the amq.topic, and a new queue will be created for these connections.
Oddly enough, this does happen when I connect using Mosquitto, however, other MQTT clients do not get their own queue.
So, as an example, connecting like so: mosquitto_sub -h rabbit-01.<hostname.com> -p 1883 -u <v_host>:<username> -P <pass> -t "#"
produces:
HOWEVER!
my connection from the other server pub/sub mqtt integration thing results in no new queue displaying. Here are the settings for that:
Any help would be greatly apprecieated!
Thanks!

Unexpected connection reset when bridging mosquitto broker with Azure IoT Hub

I am trying to create a bridge with Azure IoT Hub.
The configuration of mosquitto bridge is following:
connection Azure
address <my_hub_name>.azure-devices.net:8883
# error | information | notice | warning | all | debug
log_dest stdout
log_type all
## Auth Info
remote_username <my_hub_name>.azure-devices.net/<my_dev_name>/?api-version=2018-06-30
remote_password <SAS>
remote_clientid <my_dev_name>
bridge_cafile cert.pem
## some config
try_private false
cleansession true
start_type automatic
bridge_insecure false
bridge_protocol_version mqttv311
bridge_tls_version tlsv1.2
notifications false
## Topics
topic devices/<my_dev_name>/messages/events/# out
And I get the following output:
Opening ipv6 listen socket on port 1883.
Bridge local.<my_dev_name> doing local SUBSCRIBE on topic devices/<my_dev_name>/messages/events/#
Connecting bridge Azure (<my_hub_name>.azure-devices.net:8883)
Bridge <my_dev_name> sending CONNECT
Received CONNACK on connection local.<my_dev_name>.
Bridge local.<my_dev_name> sending UNSUBSCRIBE (Mid: 1, Topic: devices/<my_dev_name>/messages/events/#)
Received UNSUBACK from local.<my_dev_name>
Socket error on client local.<my_dev_name>, disconnecting.
Bridge local.<my_dev_name> doing local SUBSCRIBE on topic devices/<my_dev_name>/messages/events/#
Connecting bridge Azure (<my_hub_name>.azure-devices.net:8883)
Bridge <my_dev_name> sending CONNECT
Received CONNACK on connection local.<my_dev_name>.
Bridge local.<my_dev_name> sending UNSUBSCRIBE (Mid: 1, Topic: devices/<my_dev_name>/messages/events/#)
Received UNSUBACK from local.<my_dev_name>
Socket error on client local.<my_dev_name>, disconnecting.
...
Wireshark shows that for unknown reason Azure sends me a packet with FIN bit set, which results into connection reset.
At the same time mosquitto_pub is able to send a packet with same parameters (host, username, password). The utility mosquitto_sub also works without errors, but does not receive any of messages published by mosquitto_pub.
Spent several days trying to figure out the correct configuration without any success (including, of course, search on stackoverflow). I will be very appreciated for any hints.
As far as I know, nobody could connect to our IoT Hub before, so probably there maybe an issue with created hub. Unfortunately, learn.microsoft.com looks more like a quest rather than a help.
EDITED: corrected the topic parameter in configuration from "in" to "out".
As inferred from the comments.
The problem will be that the 2 bridges will have generated the same client id.
Client IDs need to be unique across all clients connected to the broker. If a second client connects with the same client id the spec says the broker must disconnect the fist one.
The disconnected client then tries to reconnect and this results in the other broker being kicked off, which then starts a feedback loop with each client kicking the other off as it reconnects.
Almost all connection errors were resolved by accurate setup of certificate and broker configuration. Except one and its description follows.
We create a bridge with above credentials (certificate, SAS login/password/clientid) - everything works fine.
The bridge is configured with following topic rule:
topic # both 0 /devices/ devices/<my_dev_name>/messages/events/
If I publish the following to my local broker:
mosquitto_pub \
-h 10.0.2.15 -p 1883 \
-t "/devices/foobar" \
-m "{\"foo\" : \"bar\"}" \
-d
everything is fine. But if I publish a bit more long topic
mosquitto_pub \
-h 10.0.2.15 -p 1883 \
-t "/devices/foo/foobar" \
-m "{\"foo\" : \"bar\"}" \
-d
I get connection reset error:
1607623721: Received PUBLISH from mosq-52yqOrz6C6riwvUUF3 (d0, q0, r0, m0, '/devices/foo/foobar1', ... (16 bytes))
1607623721: Sending PUBLISH to local.<my_dev_name> (d0, q0, r0, m0, 'devices/<my_dev_name>/messages/events/foo/foobar1', ... (16 bytes))
1607623721: Received DISCONNECT from mosq-52yqOrz6C6riwvUUF3
1607623721: Client mosq-52yqOrz6C6riwvUUF3 disconnected.
1607623721: Socket error on client local.<my_dev_name>, disconnecting.
I do not understand why Azure disconnects my bridge. I can make about a hundred messages to the topic /devices/foobar in 2-3 seconds without any complaints, but only one message to /devices/foo/foobar results into immediate socket error.
Probably, there is some restriction, but I can not find any of them, except the total maximum number of messages during one day.

Get the latest message in the queue

I want to fetch the last / latest message added in the queue, is there a specific option available in the rabbitmqadmin utility.
The following command is giving the first message in the queue,
./rabbitmqadmin get queue='log' -H localhost -P 15672 -u <username> -p <password> --vhost=logging count=1
Are you looking to consume or view the messages? I use this tool plumber to view the latest incoming messages without removing them from the queue. If you are looking to consume just the latest message you may have to write a script.
To read the latest inbound message and exit:
plumber read messages rabbitmq --address amqp://user#pass:127.0.0.1:5672 --exchange events --routing-key \#
To watch all messages as they come in:
plumber read messages RabbitMQ --address amqp://user#pass:127.0.0.1:5672 --exchange events --routing-key \# --follow
If you use plumber your queue must be set up on its own exchange you can not use the RabbitMQ default exchange see.

RabbitMQ creating queues and bindings from command line

If I have RabbitMQ installed on my machine, is there a way to create a message queue from the command line and bind it to a certain exchange without using a client?
I think it is not possible, but I want to be sure.
Summary:
Other answers are good alternatives to what was asked for. Below are commands you can use from the command line.
First, do all the necessary prep work, e.g. install rabbit, rabbitmqadmin, and rabbitctl. The idea is to use commands from rabbitmqctl and rabbitmqadmin. You can see some command examples: https://www.rabbitmq.com/management-cli.html
Example Commands/Setup:
The following commands should give you the majority if not all of what you need:
# Get the cli and make it available to use.
wget http://127.0.0.1:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin
mv rabbitmqadmin /etc/rabbitmq
Add a user and permissions
rabbitmqctl add_user testuser testpassword
rabbitmqctl set_user_tags testuser administrator
rabbitmqctl set_permissions -p / testuser ".*" ".*" ".*"
Make a virtual host and Set Permissions
rabbitmqctl add_vhost Some_Virtual_Host
rabbitmqctl set_permissions -p Some_Virtual_Host guest ".*" ".*" ".*"
Make an Exchange
./rabbitmqadmin declare exchange --vhost=Some_Virtual_Host name=some_exchange type=direct
Make a Queue
./rabbitmqadmin declare queue --vhost=Some_Virtual_Host name=some_outgoing_queue durable=true
Make a Binding
./rabbitmqadmin --vhost="Some_Virtual_Host" declare binding source="some_exchange" destination_type="queue" destination="some_incoming_queue" routing_key="some_routing_key"
Alternative Way to Bind with Python
The following is an alternative to command line binding, as I've had issues with it sometimes and found the following python code to be more reliable.
#!/usr/bin/env python
import pika
rabbitmq_host = "127.0.0.1"
rabbitmq_port = 5672
rabbitmq_virtual_host = "Some_Virtual_Host"
rabbitmq_send_exchange = "some_exchange"
rabbitmq_rcv_exchange = "some_exchange"
rabbitmq_rcv_queue = "some_incoming_queue"
rabbitmq_rcv_key = "some_routing_key"
outgoingRoutingKeys = ["outgoing_routing_key"]
outgoingQueues = ["some_outgoing_queue "]
# The binding area
credentials = pika.PlainCredentials(rabbitmq_user, rabbitmq_password)
connection = pika.BlockingConnection(pika.ConnectionParameters(rabbitmq_host, rabbitmq_port, rabbitmq_virtual_host, credentials))
channel = connection.channel()
channel.queue_bind(exchange=rabbitmq_rcv_exchange, queue=rabbitmq_rcv_queue, routing_key=rabbitmq_rcv_key)
for index in range(len(outgoingRoutingKeys)):
channel.queue_bind(exchange=rabbitmq_send_exchange, queue=outgoingQueues[index], routing_key=outgoingRoutingKeys[index])
The above can be run as part of a script using python. Notice I put the outgoing stuff into arrays, which will allow you to iterate through them. This should make things easy for deploys.
Last Thoughts
I think the above should get you moving in the right direction, use google if any specific commands don't make sense or read more with rabbitmqadmin help subcommands. I tried to use variables that explain themselves.
Install the RabbitMQ management plugin. It comes with a command line tool which you can use to configure all of your queues/exchanges/etc.
Create Exchange:
rabbitmqadmin -u {user} -p {password} -V {vhost} declare exchange name={name} type={type}
Create Queue:
rabbitmqadmin -u {user} -p {password} -V {vhost} declare queue name={name}
Bind Queue to Exchange:
rabbitmqadmin -u {user} -p {password} -V {vhost} declare binding source={Exchange} destination={queue}
Maybe a little late to the party but I've done so using CURL.
For queues:
curl -i -u RABBITUSER:RABBITPASSWORD -H "content-type:application/json" \
-XPUT -d'{"durable":true}' \
http://192.168.99.100:15672/api/queues/%2f/QUEUENAME
And for bindings
curl -i -u RABBITUSER:RABBITPASSWORD -H "content-type:application/json" \
-XPOST -d"{\"routing_key\":\"QUEUENAME\"}" \
http://192.168.99.100:15672/api/bindings/%2f/e/EXCHANGENAME/q/QUEUENAME
Note 192.168.99.100:15672 points to my RMQ Management
If you are using Linux Debian, there's a package called "amqp-tools". Install it with
apt-get install amqp-tools
You can then use command line such as amqp-publish to send messages to your queue
amqp-publish -e exchange_name -b "your message"
Then you can collect message(s) from the queue using
amqp-get -q queue_name
or
amqp-consume -q queue_name
There are also (command line) examples from rabbitmq-c package / library. After you build it, you can send messages through command line such as
amqp_sendstring localhost 5672 amq.direct test "hello world"
Have fun ...
rabbitmqctl, the provided command line interface, doesn't expose the ability to create a queue and bind it.
It, however, is quite trivial to do it with a quick script though, and the RabbitMQ getting started guide shows several examples of it, both on the publisher as well as the consumer side.
#do some work to connect
#do some work to open a channel
channel.queue_declare(queue='helloworld')
I'm glossing over connecting, but it's a literal one liner to create a queue. The operation is also idempotent, meaning you can include the statement in a script and be safe, knowing that it won't keep recreating the queue or blowing out an existing one of the same name.
Create RabbitMq Exchange, Queue and Bindings dynamically from CLI on Windows
I already had a RabbitMQ Server installed and running with multiple queue and exchange and now wanted to create it on the fly from command line. I know it is an old question but I thought giving out this information will be helpful.
Following is what I did:
Setup
Downloaded and installed Python 2.6.6-201008-24 Windows x86-64 MSI installer , any version of python,
Download RabbitMqAdmin: RabbitMq Web User Interface has a link Command Line which navigates to http://server-name:15672/cli/ (server-name: server on which rabbitmq is installed) alternatively,use the above url and save the file as rabbitmqadmin.exe in the python exe location
eg: C:\Python26
C:\Python26\python
C:\Python26\rabbitmqadmin.exe
Code:in a batch file used the below commands
Create exchange:
c:\python26\python.exe rabbitmqadmin.exe declare exchange name=*ExchangeName1* type=topic durable=true
Create queue:
c:\python26\python.exe rabbitmqadmin.exe declare queue name=*NameofQueue1* durable=true
Create binding:
c:\python26\python.exe rabbitmqadmin.exe declare binding source=ExchangeName1 destination_type=queue destination=*NameofQueue1* routing_key=*RoutingKey1*
by executing rabbitmqadmin.exe -help -subcommands it lists all the available commands
eg: c:\python26\python.exe rabbitmqadmin.exe -help -subcommands
For me, my RabbitMQ Management deal kept trying to redirect to the https version... everything in my setup is vanilla, I don't even have a config file... anyways, my work around was to manually create rabbitmqadmin.py in the sbin folder, then fill it with https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.8.1/bin/rabbitmqadmin
Then, make sure that python is in your PATH and run this to, for example, add an exchange:
python rabbitmqadmin.py declare exchange --vhost=/ name=CompletedMessageExchange type=direct
Here is a more minimal Python example, taken from the RabbitMQ Python tutorial.
First, install pika:
sudo easy_install pika
# (or use pip)
This is all you need to send a message to localhost:
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='test-queue')
channel.basic_publish(exchange='', routing_key='test-queue', body='Hello World!')
If any windows user looking for powershell based solution then there is the function I have written.
Function createQueue([string]$QueueName){
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("content-type", "application/json")
$headers.Add("Authorization", "Basic Z3Vlc3Q6Z3Vlc3Q=")
$body = "{
`n `"vhost`": `"/`",
`n `"name`": `"$QueueName`",
`n `"durable`": `"true`",
`n `"arguments`": {}
`n}"
# Write-Host $body
$url='http://localhost:15672/api/queues/%2f/'+$QueueName
# Write-Host $url
$response = Invoke-RestMethod $url -Method 'PUT' -Headers $headers -Body $body
$response | ConvertTo-Json
}
Save this into helper.ps1 file and include it into your script like this
$queueNames = 'my-queue-name'
. .\helper.ps1
createQueue($queueName)
Walkthrough to Create and delete a queue in RabbitMQ:
I couldn't find a commandline command to do it. Here is how I did it in code with java.
Rabbitmq-server version 3.3.5 on Ubuntu.
List the queues, no queues yet:
sudo rabbitmqctl list_queues
[sudo] password for eric:
Listing queues ...
...done.
Put this in CreateQueue.java
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
import java.util.*;
public class CreateQueue {
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-message-ttl", 60000);
channel.queueDeclare("kowalski", false, false, false, args);
channel.close();
connection.close();
}
}
Supply the jar file that came with your rabbitmq installation:
I'm using rabbitmq-client.jar version 0.9.1, use the one that comes with your version of rabbitmq.
Compile and run:
javac -cp .:rabbitmq-client.jar CreateQueue.java
java -cp .:rabbitmq-client.jar CreateQueue
It should finish without errors, check your queues now:
sudo rabbitmqctl list_queues
Listing queues ...
kowalski 0
...done.
the kowalski queue exists.
helps to bind the exchange while you're at it:
channel.queue_bind(queueName, exchange)
C-;