Is there a way to copy all queues and exchanges from multiple virtual hosts into a new single virtual host in rabbitmq?
Related
We want to set up Redis 6.2 clustering behind a LB. There are only master nodes and there is no Redis Sentinel being used. Each cluster-enabled Redis instance is running on a different host with the same configuration (eg. all of them are configured with port 6379). Is this possible with some port configuration on the LB such that a unique port on an LB maps to a unique_ip:6379?
Our idea is to use a cluster-aware Redis client like Lettuce RedisClusterClient which would issue CLUSTER NODES/SLOTS commands or react to MOVED/ASK redirection. It would also take care of split up a pipeline into using separate connections based on the slot for a command
It seems like this is not possible to achieve if the same port is used on all Redis hosts. Using https://docs.redis.com/latest/rs/networking/cluster-lba-setup/ as a guide, the best we could manage was to configure each Redis with a unique port and set cluster-announce-ip as the virtual IP (points to LB) and then manually make sure that the same port is used on LB as the Redis host. With this, the CLUSTER SLOTS and MOVED responses from Redis hosts could be correctly acted upon by the client. But this complicates our setup when a new Redis host has to be added or removed
You can use Route 53 if you're on AWS to achieve this.
Create A setup like this:
Add all hosts(IP addresses) in Route 53 and set TTL to smaller values like 30 seconds or so. Route 53 will return one of these Redis IP addresses, using this endpoint Redis clients like Lettuce or Jedis will discover all the Redis nodes.
You can use any other DNS system as well, record type should be A.
I use RabbitMQ with its mqtt plugin. Also, there is a guest user who can reach multiple virtual hosts. For example, I want to publish an MQTT message directly to a virtual host (/cse-id-1) but it sends the message to the default one (/). What should I do to send the message to the specified virtual host while using MQTT?
There are several options for specifying the vhost when connecting the client, like prepending the name of the vhost followed by a colon to the username (format vhost:username), so in your case the username would be cse-id-1:guest.
See details and other options in the official documentation: https://www.rabbitmq.com/mqtt.html#virtual-hosts
When I set permissions to the rabbitmq user, there is output the vhost:
[root#ha-node1 my.cnf.d]# rabbitmqctl set_permissions openstack ".*" ".*" ".*"
Setting permissions for user "openstack" in vhost "/" ...
What is the meaning of the vhost when I set permission, and what function does it have?
In RabbitMQ virtual hosts are logical groups of entities, they are similar to virtual hosts in Apache or server blocks in Nginx.
Virtual hosts are created using rabbitmqctl or HTTP API and they provide logical grouping and separation of resources.
Every virtual host has a name. When an AMQP 0-9-1 client connects to RabbitMQ, it specifies a vhost name to connect to.
If authentication succeeds and the username provided was granted permissions to the vhost, connection is established.
Let me say this by giving you an analogy.
Vhosts are to Rabbit what virtual machines are to physical servers: Vhosts allow you to run data for multiple applications safely and securely by providing logical separation between instances.
This is useful for anything from separating multiple customers on the same Rabbit to avoiding naming collisions on queues and exchanges. Where otherwise you might have to run multiple Rabbits
Every RabbitMQ server has a ability to create virtual message brokers called virtual hosts (vhosts). Each one is essentially a mini-RabbitMQ server with its own queues, exchanges, and bindings ... etc, more important, with its own permissions.
For details information ref: https://livebook.manning.com/book/rabbitmq-in-action/chapter-2/
We are implementing scale-out for our SignalR app and trying to avoid a single point of failure in our cluster. Thus, more than one Redis message bus server is required.
The problem with implementing Redis Sentinel is that upon fail-over, a the client needs to connect to a new endpoint [address], which would require the SignalR application to be restarted (Redis endpoint defined in Application_Start()).
Not an option.
I'm trying to understand if/how Booksleeve will help, and would like some explain this.
The issue is that we can only have one single endpoint defined for message bus. A hardware solution is not currently an option.
Would the SignalR application connect to a Booksleeve wrapper, which maintains the list of master/slaves?
Another option using Azure Service Bus. However, the instructions for Wiring Up the Windows Azure Service Bus Provider indicate there are still problems with this:
Note, this web site is an ASP.NET site that runs in an Azure web role.
As of 1.0alpha2 there are some bugs in AzureWebSites due to which
ServiceBus Scale out scenarios do not work well. We are working on
resolving this for the future
I don't know the specifics of how SignalR does the connect, but: BookSleeve already offers some concessions towards failover nodes. In particular, the ConnectionUtils.Connect method takes a string (ideal for web.config configuration values etc), which can include multiple redis nodes, and BookSleeve will then try to locate the most appropriate node to connect to. If the nodes mentioned in the string are regular redis nodes, it will attempt to connect to a master, otherwise falling back to a slave (optionally promoting the slave in the process). If the nodes mentioned are sentinel nodes, it will ask sentinel to nominate a serer to connect to.
What BookSleeve doesn't offer at the moment is a redundant connection wrapper that will automatically reconnect. That is on the road-map, but isn't difficult to do in the calling code. I plan to add more support for this at the same time as implementing redis-cluster support.
But: all that is from a BookSleeve perspective - I can't comment on SignalR specifically.
BookSleeve 1.3.41.0 supports Redis sentinel. Deployment configuration we use: 1 master redis, 1 slave redis. Each box has sentinel (one for master, one for slave). Clients connect to sentinel first, sentinel then redirects them to active master.
This is how it is implemented in client code:
public class OwinStartup
{
public void Configuration(IAppBuilder app)
{
var config = new WebClientRedisScaleoutConfiguration();
GlobalHost.DependencyResolver.UseRedis(config);
app.MapSignalR();
}
}
public class WebClientRedisScaleoutConfiguration : RedisScaleoutConfiguration
{
public WebClientRedisScaleoutConfiguration()
: base(() => getRedisConnection(), WebUIConfiguration.Default.Redis.EventKey)
{ }
private static BookSleeve.RedisConnection _recentConnection;
private static BookSleeve.RedisConnection getRedisConnection()
{
var log = new TraceTextWriter();
var connection = BookSleeve.ConnectionUtils.Connect("sentinel1:23679,sentinel2:23679,serviceName=WebClient", log);
if (connection != null)
{
_recentConnection = connection;
return connection;
}
if (_recentConnection != null)
{
return _recentConnection;
}
// Cannot return null nor throw exception -- this will break reconnection cycle.
return new BookSleeve.RedisConnection(string.Empty);
}
}
Hot to configure redis.
Common steps
Download Redis for windows http://redis.io/download
Unzip to c:\redis
Master (only very first redis box, only one such config)
Create Redis service: execute command within redis directory redis-server --service-install redis.conf --service-name redis
Start Redis service
Ensure Redis is listing port 6379
Slave (other boxes)
Update redis.conf: add line slaveof masterAddr 6379 where masterAddr
is address where redis in master mode is running, 6379 is default
redis port.
Create Redis service: execute command within redis directory redis-server --service-install redis.conf --service-name redis
Start Redis service
Ensure Redis is listing port 6379
Sentinel (common for master and slave)
Create file redis-sentinel.conf with content:
port 26379
logfile "redis-sentinel1.log"
sentinel monitor WebClient masterAddr 6379 1
where masterAddr is address where redis in master mode is running,
6379 is default redis port, 1 is quorum (number of host that makes
decision is server down or not). WebClient is group name. You specify it in client code ConnectionUtils.Connect("...,serviceName=WebClient...")
Create redis sentinel service:execute command within redis directory redis-server --service-install redis-sentinel.conf --service-name redis-sentinel --sentinel
Start redis-sentinel service
We want to use a single Redis server for servers that span two subnets.
If we put Redis on just subnet A, the servers on B will have to go across a router to get to redis.
Our thought is to make the Redis server multi homed (multiple nics), attached to both subnets A and B.
1) Will this work?
2) Will Redis then attach to both IP's?
Thanks!
You can provide the bind address in the redis configuration file (bind parameter).
Now if you comment out the definition and do not provide a bind address, Redis will listen to its port on all the interfaces (i.e. it will listen to 0.0.0.0).
I did not try, but I would say a configuration with 2 addresses should work.