I am getting this error when I run celery beat -S redbeat.RedBeatScheduler.
beat raised exception : ConnectionError('Error -2 connecting to redis-sentinel:26379. Name or service not known.',)
How can I create a service_name and password in redis-sentinel
I am not trying to use redis as a message broker. I am using celery-redbeat to store celerybeat data in redis-sentinel cluster from this page.https://pypi.org/project/celery-redbeat/
and
from this configuration
redbeat_redis_url = 'redis-sentinel://redis-sentinel:26379/0'
redbeat_redis_options = {
'sentinels': [('192.168.1.1', 26379),
('192.168.1.2', 26379),
('192.168.1.3', 26379)],
'socket_timeout': 0.1,
}
I add 192.168.1.1:26379 instead of redis-sentinel:/26379 but when master node down in redis-sentinel cluster beat is down too.
redbeat_redis_url = 'redis-sentinel://192.168.1.1:26379/0'
redbeat_redis_options = {
'sentinels': [('192.168.1.2', 26379),
('192.168.1.3', 26379)],
'socket_timeout': 0.1,
}
Unless you have redis-sentinel in your /etc/hosts file it will not be able to resolve it to a correct IP address. You may try to replace redis-sentinel with an IP address of your Redis server. Furthermore, it does not look like a proper Redis Sentinel configuration. Redis Configuration section explains how to connect to Redis Sentinel, please read it.
I'm using mqtt-launcher (https://github.com/jpmens/mqtt-launcher) to execute commands when a certain MQTT message with the payload "0" was received.
Here is the config
logfile = '/home/user/mqtt-launcher/logfile'
mqtt_broker = 'broker' # default: 'localhost'. If using TLS, this must be set to the domain name signed by$
mqtt_port = 1883 # default: 1883
mqtt_clientid = 'mqtt-launcher-1'
mqtt_username = ''
mqtt_password = ''
mqtt_tls = None # default: No TLS
topiclist = {
# topic payload value program & arguments
"channel/dostuff" : {
'0' : [
'/usr/bin/ssh',
'-i',
'/home/user/.ssh/privatekey',
'user#host',
'script.sh'
]
}
}
Everytime, I start the python script, the shell script is executed twice.
But I want it to execute only once if the MQTT message with the payload "0" is sent.
I made sure the queue which is implicitly created when subscribing was empty before by purging it, then starting mqtt-launcher but still the script is execute twice after the program connected.
When I run user#localhost:~$ mosquitto_sub -h broker -p 1883 -t 'channel/dostuff' -v -u 'user' -P 'mysecurepassword' I get channel/dostuff 0
I'm not familiar with mosquitto but I think that this means I receive a message, right?
I turned of the retain option, restarted openHAB and RabbitMQ, but still the message is sent. Here is the openHAB mqtt.cfg:
broker.url=tcp://broker:1883
broker.user=openhab
broker.pwd=mysecurepassword
broker.qos=1
broker.retain=false
broker.async=false
You have published a message with the payload 0 and the retained bit set.
This means that when ever a client subscribes to that topic the last message with the retained bit set will be delivered to that client.
You can clear the retained message by publishing a message with the retained bit set and a null payload to the same topic. You can do this with the mosquitto_pub command as follows:
mosquitto_pub -t "channel/dostuff" -u 'user' -P 'password' -r -n
You should make sure what ever you are using to publish the message normally is not setting the retained bit.
There is clustered rabbitmq.
There is one connected consumer to it.
root#serwer1:~# rabbitmqctl list_consumers -p vhost1
Listing consumers ...
queue1.dlq <rabbit#serwer1.3.9529.109> amq.ctag-C5lFDLY7LZjnDdi1hjbAIA true 1000 []
...done.
There is connected with it channel:
root#serwer1:~# rabbitmqctl list_channels vhost pid state connection |grep rabbit#serwer1.3.9529.109
vhost1 <rabbit#serwer1.3.9529.109> running <rabbit#serwer1.3.9537.109>
But - there is no connected connection for it:
root#serwer1:~# rabbitmqctl list_connections vhost pid state |grep rabbit#serwer1.3.9529.109
root#serwer1:~#
How such situation is possible? How to fix it?
(rabbitmq 3.3.5 from debian jessie)
I have a simple redis cluster on my local machine that consists of:
master on port 3679
slave on port 6380
sentinel on port 26379
I am using ServiceStack.Redis to connect with no problems so far. Today I added a password to each of them using the requirepass 42 setting. I can connect to all of them using Redis Desktop Manager fine and everything works as expected.
Using the following code, I get an error when I attempt to connect. Removing the password works as expected.
var config = RedisConfiguration.Instance;
Func<string, string> hostFilter = host => string.IsNullOrEmpty(config.SecurityKey)
? $"{host}?db={config.Database}"
: $"{config.SecurityKey}#{host}?db={config.Database}";
var sentinelHosts = config.SentinelHosts.Select(hostFilter);
var sentinel = new RedisSentinel(sentinelHosts, config.ServiceName)
{
HostFilter = hostFilter,
RedisManagerFactory = (master, slaves) => new RedisManagerPool(master)
};
sentinel.OnFailover += manager => Logger?.Warn($"Redis fail over to {sentinel.GetMaster()}");
sentinel.Start()
This code throw a RedisException "No Redis Sentinels were available" with an inner exception of "unknown command 'AUTH'".
I am not clear if I am using the ServiceStack.Redis library improperly or my Redis cluster configuration is incorrect.
Can some one point me in the right direction?
You can use HostFilter to specify the password:
sentinel.HostFilter = host => $"{config.SecurityKey}#{host}?db={config.Database}";
But when using a password, it needs to be configured everywhere, i.e. in both Master and Slave configurations using:
requirepass password
masterauth password
The Redis Sentinels also need to be configured to use the same password so it can control the redis instances it's monitoring, which can be configured in your sentinel.conf with:
sentinel auth-pass mymaster pasword
The windows-password folder in the redis-config project shows an example of a password-protected Redis Sentinel configuration.
If I have RabbitMQ installed on my machine, is there a way to create a message queue from the command line and bind it to a certain exchange without using a client?
I think it is not possible, but I want to be sure.
Summary:
Other answers are good alternatives to what was asked for. Below are commands you can use from the command line.
First, do all the necessary prep work, e.g. install rabbit, rabbitmqadmin, and rabbitctl. The idea is to use commands from rabbitmqctl and rabbitmqadmin. You can see some command examples: https://www.rabbitmq.com/management-cli.html
Example Commands/Setup:
The following commands should give you the majority if not all of what you need:
# Get the cli and make it available to use.
wget http://127.0.0.1:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin
mv rabbitmqadmin /etc/rabbitmq
Add a user and permissions
rabbitmqctl add_user testuser testpassword
rabbitmqctl set_user_tags testuser administrator
rabbitmqctl set_permissions -p / testuser ".*" ".*" ".*"
Make a virtual host and Set Permissions
rabbitmqctl add_vhost Some_Virtual_Host
rabbitmqctl set_permissions -p Some_Virtual_Host guest ".*" ".*" ".*"
Make an Exchange
./rabbitmqadmin declare exchange --vhost=Some_Virtual_Host name=some_exchange type=direct
Make a Queue
./rabbitmqadmin declare queue --vhost=Some_Virtual_Host name=some_outgoing_queue durable=true
Make a Binding
./rabbitmqadmin --vhost="Some_Virtual_Host" declare binding source="some_exchange" destination_type="queue" destination="some_incoming_queue" routing_key="some_routing_key"
Alternative Way to Bind with Python
The following is an alternative to command line binding, as I've had issues with it sometimes and found the following python code to be more reliable.
#!/usr/bin/env python
import pika
rabbitmq_host = "127.0.0.1"
rabbitmq_port = 5672
rabbitmq_virtual_host = "Some_Virtual_Host"
rabbitmq_send_exchange = "some_exchange"
rabbitmq_rcv_exchange = "some_exchange"
rabbitmq_rcv_queue = "some_incoming_queue"
rabbitmq_rcv_key = "some_routing_key"
outgoingRoutingKeys = ["outgoing_routing_key"]
outgoingQueues = ["some_outgoing_queue "]
# The binding area
credentials = pika.PlainCredentials(rabbitmq_user, rabbitmq_password)
connection = pika.BlockingConnection(pika.ConnectionParameters(rabbitmq_host, rabbitmq_port, rabbitmq_virtual_host, credentials))
channel = connection.channel()
channel.queue_bind(exchange=rabbitmq_rcv_exchange, queue=rabbitmq_rcv_queue, routing_key=rabbitmq_rcv_key)
for index in range(len(outgoingRoutingKeys)):
channel.queue_bind(exchange=rabbitmq_send_exchange, queue=outgoingQueues[index], routing_key=outgoingRoutingKeys[index])
The above can be run as part of a script using python. Notice I put the outgoing stuff into arrays, which will allow you to iterate through them. This should make things easy for deploys.
Last Thoughts
I think the above should get you moving in the right direction, use google if any specific commands don't make sense or read more with rabbitmqadmin help subcommands. I tried to use variables that explain themselves.
Install the RabbitMQ management plugin. It comes with a command line tool which you can use to configure all of your queues/exchanges/etc.
Create Exchange:
rabbitmqadmin -u {user} -p {password} -V {vhost} declare exchange name={name} type={type}
Create Queue:
rabbitmqadmin -u {user} -p {password} -V {vhost} declare queue name={name}
Bind Queue to Exchange:
rabbitmqadmin -u {user} -p {password} -V {vhost} declare binding source={Exchange} destination={queue}
Maybe a little late to the party but I've done so using CURL.
For queues:
curl -i -u RABBITUSER:RABBITPASSWORD -H "content-type:application/json" \
-XPUT -d'{"durable":true}' \
http://192.168.99.100:15672/api/queues/%2f/QUEUENAME
And for bindings
curl -i -u RABBITUSER:RABBITPASSWORD -H "content-type:application/json" \
-XPOST -d"{\"routing_key\":\"QUEUENAME\"}" \
http://192.168.99.100:15672/api/bindings/%2f/e/EXCHANGENAME/q/QUEUENAME
Note 192.168.99.100:15672 points to my RMQ Management
If you are using Linux Debian, there's a package called "amqp-tools". Install it with
apt-get install amqp-tools
You can then use command line such as amqp-publish to send messages to your queue
amqp-publish -e exchange_name -b "your message"
Then you can collect message(s) from the queue using
amqp-get -q queue_name
or
amqp-consume -q queue_name
There are also (command line) examples from rabbitmq-c package / library. After you build it, you can send messages through command line such as
amqp_sendstring localhost 5672 amq.direct test "hello world"
Have fun ...
rabbitmqctl, the provided command line interface, doesn't expose the ability to create a queue and bind it.
It, however, is quite trivial to do it with a quick script though, and the RabbitMQ getting started guide shows several examples of it, both on the publisher as well as the consumer side.
#do some work to connect
#do some work to open a channel
channel.queue_declare(queue='helloworld')
I'm glossing over connecting, but it's a literal one liner to create a queue. The operation is also idempotent, meaning you can include the statement in a script and be safe, knowing that it won't keep recreating the queue or blowing out an existing one of the same name.
Create RabbitMq Exchange, Queue and Bindings dynamically from CLI on Windows
I already had a RabbitMQ Server installed and running with multiple queue and exchange and now wanted to create it on the fly from command line. I know it is an old question but I thought giving out this information will be helpful.
Following is what I did:
Setup
Downloaded and installed Python 2.6.6-201008-24 Windows x86-64 MSI installer , any version of python,
Download RabbitMqAdmin: RabbitMq Web User Interface has a link Command Line which navigates to http://server-name:15672/cli/ (server-name: server on which rabbitmq is installed) alternatively,use the above url and save the file as rabbitmqadmin.exe in the python exe location
eg: C:\Python26
C:\Python26\python
C:\Python26\rabbitmqadmin.exe
Code:in a batch file used the below commands
Create exchange:
c:\python26\python.exe rabbitmqadmin.exe declare exchange name=*ExchangeName1* type=topic durable=true
Create queue:
c:\python26\python.exe rabbitmqadmin.exe declare queue name=*NameofQueue1* durable=true
Create binding:
c:\python26\python.exe rabbitmqadmin.exe declare binding source=ExchangeName1 destination_type=queue destination=*NameofQueue1* routing_key=*RoutingKey1*
by executing rabbitmqadmin.exe -help -subcommands it lists all the available commands
eg: c:\python26\python.exe rabbitmqadmin.exe -help -subcommands
For me, my RabbitMQ Management deal kept trying to redirect to the https version... everything in my setup is vanilla, I don't even have a config file... anyways, my work around was to manually create rabbitmqadmin.py in the sbin folder, then fill it with https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.8.1/bin/rabbitmqadmin
Then, make sure that python is in your PATH and run this to, for example, add an exchange:
python rabbitmqadmin.py declare exchange --vhost=/ name=CompletedMessageExchange type=direct
Here is a more minimal Python example, taken from the RabbitMQ Python tutorial.
First, install pika:
sudo easy_install pika
# (or use pip)
This is all you need to send a message to localhost:
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='test-queue')
channel.basic_publish(exchange='', routing_key='test-queue', body='Hello World!')
If any windows user looking for powershell based solution then there is the function I have written.
Function createQueue([string]$QueueName){
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("content-type", "application/json")
$headers.Add("Authorization", "Basic Z3Vlc3Q6Z3Vlc3Q=")
$body = "{
`n `"vhost`": `"/`",
`n `"name`": `"$QueueName`",
`n `"durable`": `"true`",
`n `"arguments`": {}
`n}"
# Write-Host $body
$url='http://localhost:15672/api/queues/%2f/'+$QueueName
# Write-Host $url
$response = Invoke-RestMethod $url -Method 'PUT' -Headers $headers -Body $body
$response | ConvertTo-Json
}
Save this into helper.ps1 file and include it into your script like this
$queueNames = 'my-queue-name'
. .\helper.ps1
createQueue($queueName)
Walkthrough to Create and delete a queue in RabbitMQ:
I couldn't find a commandline command to do it. Here is how I did it in code with java.
Rabbitmq-server version 3.3.5 on Ubuntu.
List the queues, no queues yet:
sudo rabbitmqctl list_queues
[sudo] password for eric:
Listing queues ...
...done.
Put this in CreateQueue.java
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
import java.util.*;
public class CreateQueue {
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-message-ttl", 60000);
channel.queueDeclare("kowalski", false, false, false, args);
channel.close();
connection.close();
}
}
Supply the jar file that came with your rabbitmq installation:
I'm using rabbitmq-client.jar version 0.9.1, use the one that comes with your version of rabbitmq.
Compile and run:
javac -cp .:rabbitmq-client.jar CreateQueue.java
java -cp .:rabbitmq-client.jar CreateQueue
It should finish without errors, check your queues now:
sudo rabbitmqctl list_queues
Listing queues ...
kowalski 0
...done.
the kowalski queue exists.
helps to bind the exchange while you're at it:
channel.queue_bind(queueName, exchange)
C-;