We going to use ServiceStack.RedisClient, but I was not able figure out how to define a password for sentinels and masters.
I've tried pwd#ipv4:port but no result.
Our code is:
var sentinelHosts = new[] { "node1:26379", "node2:26379", "node3:26379" };
var sentinel = new RedisSentinel(sentinelHosts, "mymaster");
var manager = sentinel.Start();
while (true)
{
using (var client = manager.GetClient())
{
try
{
Console.WriteLine(client.IncrementValue("MyTestKey"));
}
catch (RedisException ex)
{
Console.WriteLine($"Error {ex.Message}");
}
}
Thread.Sleep(1000);
}
You can specify to use passwords for connecting to masters / slaves with an Custom Redis Connection String, e.g:
sentinel.HostFilter = host => "pwd#{0}".Fmt(host);
Related
I have a project with ASP.NET Core integrated with Redis Sentinel.
Caching works very well with Sentinel but it doesn't work while getting all keys with GetServer(), and it wants me to give it a parameter ServiceName, I don't know how to find it??
There is a Master and 4 Slaves
--> appsettings.json
RedisConfiguration:ConnectionString --- > "127.0.0.1:6379,127.0.0.1:6380,127.0.0.1:6381,127.0.0.1:6382,127.0.0.1:6383"
--> RedisCacheManager.cs
Constructor :
public RedisCacheManager()
{
_connectionString = configuration.GetSection("RedisConfiguration:ConnectionString")?.Value;
var connectionStrings = _connectionString.Split(",");
_configurationOptions = new ConfigurationOptions()
{
EndPoints = { aa.ToString() },
AbortOnConnectFail = false,
AsyncTimeout = 10000,
ConnectTimeout = 10000,
KeepAlive = 180,
//ServiceName = ServiceName
};
foreach (var item in connectionStrings)
{
_configurationOptions.EndPoints.Add(item);
}
_client = ConnectionMultiplexer.Connect(_configurationOptions);
}
--> RemoveByPattern method in RedisCacheManager.cs
public void RemoveByPattern()
{
//THIS IS WHERE I USE SENTINEL CONNECT AND CONFIG OPTIONS....
//var aa = ConnectionMultiplexer.SentinelConnect(_configurationOptions);
//THIS IS ALSO WHERE I NEED SERVICE NAME... (FOUND THIS WHILE RESEARCHING)
//var settings = ConfigurationOptions.Parse("localhost,serviceName=mymaster");
// HERE I HAVE TO USE ONLY ONE ENDPOINT....
var server = ConnectionMultiplexer.Connect(settings).GetServer(_client.GetEndPoints().First());
var keys = server.Keys();
var values = keys.Where(x => x.ToString().Contains(pattern)).Select(c => (string)c);
List<string> listKeys = new();
listKeys.AddRange(values);
foreach (var key in listKeys)
{
await _client.GetDatabase().KeyDeleteAsync(key);
}
}
Producer count in the activemq web console shows 0 all the time, even if there are producers connected to the broker. I'm not sure why?
My producer code looks like this.
public boolean postMessage(List<? extends JMSMessageBean> messageList, String data, int messageCount)
throws JMSException {
String queueName = null;
MessageProducer producer = null;
Connection connection = null;
Session session = null;
try {
connection = pooledConnectionFactory.createConnection();
connection.setExceptionListener(this);
connection.start();
session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
int index = 0;
for (JMSMessageBean message : messageList) {
if (producer == null || !message.getQueueName().equals(queueName)) {
queueName = message.getQueueName();
producer = getQueueProducer(queueName, session);
}
Message _omessage = session.createObjectMessage(message);
_omessage.setStringProperty("MESSAGE_INDEX", messageCount + ":" + index);
_omessage.setIntProperty("RETRY_COUNT", 0);
_omessage.setJMSType(message.getJmsType());
if (data != null) {
_omessage.setStringProperty("RAW_DATA", data);
}
producer.send(_omessage);
index++;
}
} catch (JMSException e) {
logger.error("Exception while creating connection to jms broker", e);
} finally {
try {
if (null != session) {
session.close();
}
if (null != connection) {
connection.close();
}
if(null != producer) {
producer.close();
}
} catch (JMSException e) {
logger.error(e.getMessage(), e);
}
}
return true;
}
Am using a pooledconnectionfactory to create sessions, connections, and messageproducers. Everytime, someone has to post a message, a new connection is requested from the pooledconnectionfactory. and then
The ActiveMQ client often uses what they call "dynamic producers"-- a producer per message for non-transacted sessions. If you walked the JMS object lifecycle, you'd find there is little need to keep a producer object around in a non-transacted session-- which is different from the consumer object.
Look under the dynamicProducers list in JMX, and you'll catch them being created. You can also monitor the advisory topics to see them get created and destroyed.
Side note: your object close order in the finally is incorrect.. you should close objects in reverse order-- producer, session, connection.
I have a simple rabbit set up that is currently doing what I want...
It publishes messages based on the type of message. Each type gets its own queue.
When messages are published they sit on the queue even if there is no consumer to consume them (sit there forever if no consumer arrives).
When a consumer is there (there is only one!) it eats the messages.
If for some reason it cannot handle a message (eg it gets a sub message before the parent has arrived) it nacks the message back onto the queue.
If it sees the same message six times it nacks the message.
This all works, but currently after six attempts it drops the message.
What I would like is for the message to pass to a 'dead letter queue' and after some time (say 5 mins) re-queue that message at the end of the particular queue it came from.
I am definitely cargo cult programing and I don't quite understand all the exchange/queue/binding/routing keys and other arcania involved... hand holding is appreciated!
public void PublishEntity<T>(T message) where T : class, ISendable
{
logger.Info($"publishing {message.UniqueId}");
var factory = new ConnectionFactory
{
HostName = appSettings.RabbitHostName,
UserName = appSettings.RabbitUsername,
Password = appSettings.RabbitPassword
};
try
{
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
Console.WriteLine($"Setting up queues for: {typeof(T).Name}");
channel.QueueDeclare($"App_{typeof(T).Name}",
true,
false,
false,
null);
var json = JsonConvert.SerializeObject(message);
var body = Encoding.UTF8.GetBytes(json);
channel.TxSelect();
var properties = channel.CreateBasicProperties();
properties.Persistent = true;
properties.Headers = new Dictionary<string, object>
{
{ "Id", Guid.NewGuid().ToString() }
};
channel.BasicPublish("",
$"App_{typeof(T).Name}",
properties,
body);
Data.MarkAsSent(message);
channel.TxCommit();
}
}
}
ISendable just make sure the message has some properties used in Data.MarkAsSent(message); to mark in a database where we have got too.
The receiver has a similar lump of code to handle each type. As I say this is working.
What do I need to do to add the dead letter queue things?
My attempt like so created the dead letter queues, but nothing ever moves to them.
public void PublishEntity<T>(T message) where T : class, ISendable
{
logger.Info($"publishing {message.UniqueId}");
var factory = new ConnectionFactory
{
HostName = appSettings.RabbitHostName,
UserName = appSettings.RabbitUsername,
Password = appSettings.RabbitPassword
};
try
{
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
Console.WriteLine($"Setting up queues for: {typeof(T).Name}");
channel.ExchangeDeclare("App.Dead.Letter", "direct", true);
var args = new Dictionary<string, object>
{
{ "x-dead-letter-exchange", "App.Dead.Letter" },
{
"x-dead-letter-routing-key", $"DLQ.App_{typeof(T).Name}"
}
};
channel.QueueDeclare($"App_{typeof(T).Name}",
true,
false,
false,
args);
channel.QueueDeclare($"DLQ.App_{typeof(T).Name}",
true,
false,
false,
null);
var json = JsonConvert.SerializeObject(message);
var body = Encoding.UTF8.GetBytes(json);
channel.TxSelect();
var properties = channel.CreateBasicProperties();
properties.Persistent = true;
properties.Headers = new Dictionary<string, object>
{
{ "Id", Guid.NewGuid().ToString() }
};
channel.BasicPublish("",
$"App_{typeof(T).Name}",
properties,
body);
Data.MarkAsSent(message);
channel.TxCommit();
}
}
}
In my reciever I have this magic
catch (Exception ex)
{
var attemptsToHandle = MarkFailedToHandleMessage(logId, ex);
if (attemptsToHandle > 5)
{
//If we have seen this message many times then don't re-que.
channel.BasicNack(ea.DeliveryTag, false, false);
return;
}
// re-que so we can re-try later.
channel.BasicNack(ea.DeliveryTag, false, true);
return;
}
Phew... lot of code. Thanks if you have made it this far....
I am asking what are the glaring issues in my code to make things fall to the dead letter queue.
and what extra do I need to add so that things in the dlq will bounce back to the main queue after some time.
Additionally this sets up a dlq for each types queue... is this required or should there be a single queue to hold the errored messages?
So I think I got this working as I intend. Hard to test all this stuff though!
public void PublishEntity<T>(T message) where T : class, ISendable
{
logger.Info($"publishing {message.UniqueId}");
var factory = new ConnectionFactory
{
HostName = appSettings.RabbitHostName,
UserName = appSettings.RabbitUsername,
Password = appSettings.RabbitPassword
};
try
{
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
Console.WriteLine($"Setting up queues for: {typeof(T).Name}");
// Declair dead letter queue for this type
channel.ExchangeDeclare("App.Dead.Letter", "direct", true);
var queueArgs = new Dictionary<string, object>
{
{ "x-dead-letter-exchange", "App" },
{
"x-dead-letter-routing-key", $"App_{typeof(T).Name}"
}
,{ "x-message-ttl", 30000 }
};
channel.QueueDeclare($"DLQ.App_{typeof(T).Name}",
true,
false,
false,
queueArgs);
channel.QueueBind($"DLQ.App_{typeof(T).Name}", "App.Dead.Letter", $"DLQ.App_{typeof(T).Name}", null);
// declair queue for this type
channel.ExchangeDeclare("App", "direct", true);
var args = new Dictionary<string, object>
{
{ "x-dead-letter-exchange", "App.Dead.Letter" },
{
"x-dead-letter-routing-key", $"DLQ.App_{typeof(T).Name}"
}
};
channel.QueueDeclare($"App_{typeof(T).Name}",
true,
false,
false,
args);
channel.QueueBind($"App_{typeof(T).Name}", "App", $"App_{typeof(T).Name}", null);
I added and exchange for my main queues to live in and actually bound the queues to the exchanges. I still don't know why I need to do this as it was working without this extra complexity.. I guess some magic was hiding it from me before?
I'm trying to test the AutomaticRecoveryEnabled property of the RabbitMQ ConnectionFactory. I'm connecting to a RabbitMQ instance on a local VM and on the client I'm publishing messages in a loop. The problem is if I intentionally break the connection, the client just waits forever and doesn't time out. How do I set the time out value? RequestedConnectionTimeout doesn't appear to have any effect.
I'm using the RabbitMQ client 3.5.4
Rudimentary publish loop:
// Client is a wrapper around the RabbitMQ client
for (var i = 0; i < 1000; ++i)
{
// Publish sequentially numbered messages
client.Publish("routingkey", GetContent(i)));
Thread.Sleep(100);
}
The Publish method inside the wrapper:
public bool Publish(string routingKey, byte[] body)
{
try
{
using (var channel = _connection.CreateModel())
{
var basicProps = new BasicProperties
{
Persistent = true,
};
channel.ExchangeDeclare(_exchange, _exchangeType);
channel.BasicPublish(_exchange, routingKey, basicProps, body);
return true;
}
}
catch (Exception e)
{
_logger.Log(e);
}
return false;
}
The connection and connection factory:
_connectionFactory = new ConnectionFactory
{
UserName = _userName,
Password = _password,
HostName = _hostName,
Port = _port,
Protocol = Protocols.DefaultProtocol,
VirtualHost = _virtualHost,
// Doesn't seem to have any effect on broken connections
RequestedConnectionTimeout = 2000,
// The behaviour appears to be the same with or without these included
// AutomaticRecoveryEnabled = true,
// NetworkRecoveryInterval = TimeSpan.FromSeconds(10),
};
_connection = _connectionFactory.CreateConnection();
It appears this is a bug in version 3.5.4. Version 3.6.3 does not wait indefinitely.
How can I format the below Redis connection string:
Connection string:
myIP,keepAlive=180,ConnectRetry=30,ConnectTimeout=5000
I started writing a unit test but keep getting a input string was not in correct format error message
[TestFixtureSetUp]
private void Init()
{
var redisConnectionString = "myIP,keepAlive=180,ConnectRetry=30,ConnectTimeout=5000";
_clientsManager = new PooledRedisClientManager(redisConnectionString);
}
[Test]
public void CanConnectToRedis()
{
var readWrite = (RedisClient) _clientsManager.GetClient();
using (var redis = _clientsManager.GetClient())
{
var redisClient = redis;
}
}
See the connection string format on the ServiceStack.Redis home page:
redis://localhost:6379?ConnectTimeout=5000&IdleTimeOutSecs=180
Which can be used in any of the Redis Client Managers:
var redisManager = new RedisManagerPool(
"redis://localhost:6379?ConnectTimeout=5000&IdleTimeOutSecs=180");
using (var client = redisManager.GetClient())
{
client.Info.PrintDump();
}
The list of available configuratoin options are also listed on the homepage.