we get RedisConnectionException and need to retry on them.
We are using Spring Data Redis with lettuce client, we have a config like:
var lettuceClientConfigurationBuilder =
LettucePoolingClientConfiguration.builder()
.poolConfig(poolConfig).readFrom(MASTER);
var connectionFactory = new LettuceConnectionFactory(redisConfiguration,
lettuceClientConfigurationBuilder.build());
connectionFactory.setShareNativeConnection(false);
Do you have any advice to setup this retry in such a configuration?
Or is this something that we need to do in code or with an outside mechanism of retry like Spring's, Spring-Kafka's, etc?
Another question:
setShareNativeConnection(false) & having clientOptions with ping=true
do both mean extra round trips per client interaction?
Thanks
Related
In spring cloud gw redis rate limiter impl, there are headers infos like X-RateLimit-Remaining, X-RateLimit-Requested-Tokens, X-RateLimit-Burst-Capacity, X-RateLimit-Replenish-Rate. I dont want to give that information to clint due to some security issues.
There is a way to remove those headers on response even if request got 429?
Thanks
One potential solution to the problem is to configure special field for each instance of the RedisRateLimiter.
For example:
#Configuration
public class RateLimiterConfiguration {
public RateLimiterConfiguration(List<RedisRateLimiter> rateLimiters) {
rateLimiters.forEach(rateLimiter -> rateLimiter.setIncludeHeaders(false));
}
}
Currently I am able to see the streaming values exposed by the code below, but only one http client will receive the continuous stream of values, the others will not be able to.
The code, a modified version of the quarkus quickstart for kafka reactive streaming is:
#Path("/migrations")
public class StreamingResource {
private volatile Map<String, String> counterBySystemDate = new ConcurrentHashMap<>();
#Inject
#Channel("migrations")
Flowable<String> counters;
#GET
#Path("/stream")
#Produces(MediaType.SERVER_SENT_EVENTS) // denotes that server side events (SSE) will be produced
#SseElementType("text/plain") // denotes that the contained data, within this SSE, is just regular text/plain data
public Publisher<String> stream() {
Flowable<String> mainStream = counters.doOnNext(dateSystemToCount -> {
String key = dateSystemToCount.substring(0, dateSystemToCount.lastIndexOf("_"));
counterBySystemDate.put(key, dateSystemToCount);
});
return fromIterable(counterBySystemDate.values().stream().sorted().collect(Collectors.toList()))
.concatWith(mainStream)
.onBackpressureLatest();
}
}
Is it possible to make any modification that would allow multiple clients to consume the same data, in a broadcast fashion?
I guess this implies letting go of backpressure, because that would imply a state per consumer?
I saw that Observable is not accepted as a return type in the resteasy-rxjava2 for the Server Side Events media-tpe.
Please let me know any ideas,
Thank you
Please find the full code in Why in multiple connections to PricesResource Publisher, only one gets the stream?
I was currently using NMS to develop application based ActiveMQ(5.6).
We have several consumers(exe) trying to recieving massgaes from the same queue(not topic). While all the messages just all go to one consumer though I have make the consumer to sleep for seconds after recieving a message. By the way, we don't want the consumers recieving the same messages other consumers have recieved.
It is mentioned in the official website that we should set Prefetch Limit to decide how many messages can be streamed to a consumer at any point in time. And it can both be configured and coded.
One way I tried is to code using PrefetchPolicy class binding the ConnectionFactory class like bellow.
PrefetchPolicy poli = new PrefetchPolicy();
poli.QueuePrefetch = 0;
ConnectionFactory fac = new ConnectionFactory("activemq:tcp://Localhost:61616?jms.prefetchPolicy.queuePrefetch=1");
fac.PrefetchPolicy = poli;
using (IConnection con = fac.CreateConnection())
{
using (ISession se = con.CreateSession())
{
IDestination destination = SessionUtil.GetDestination(se, queue, DestinationType.Queue);
using (IMessageConsumer consumer = se.CreateConsumer(queue1))
{
con.Start();
while (true)
{
ITextMessage message = consumer.Receive() as ITextMessage;
Thread.Sleep(2000);
if (message != null)
{
Task.Factory.StartNew(() => extractAndSend(message.Text)); //do something
}
else
{
Console.WriteLine("No message received~");
}
}
}
}
}
But no matter what prefetch value I set the behavior of the consumers stay the same as before.
And I've tried the second way tying to get the result, namely configure the server conf file. I change the activemq.xml of the server like bellow.
" producerFlowControl="true" memoryLimit="5mb" />
" producerFlowControl="true" memoryLimit="5mb">
But though I've set the dispatchpolicy the messages still go to one consumer.
I want to know that:
Whether this behavior can be achieved by just configuring the server xml file to enable all the consumers recieve messages from one queue? If so, how to configure this and what is wrong with my configuration? If not, how can I use codes to achieve the goal?
Thanks.
Take a look at "Message Groups" feature.
I had the same problem. Only one consumer processed all messages. I found in my code I used group header during send:
request.Properties["NMSXGroupID"] = "cheese";
According to official docs:
Standard JMS header JMSXGroupID is used to define which message group
the message belongs to. The Message Group feature then ensures that
all messages for the same message group will be sent to the same JMS
consumer - while that consumer stays alive. As soon as the consumer
dies another will be chosen.
See full details at http://activemq.apache.org/message-groups.html
I need to remove/delete my topic subscriber. I found this http://activemq.apache.org/manage-durable-subscribers.html
However, it's not good enough for us. We want to control the timing of removing a subscriber, and no matter there are any message or not. Besides, our program is written by C#. So the best solution for us is NMS API.
Thanks.
Here are the code,
Apache.NMS.ActiveMQ.ConnectionFactory factory = new Apache.NMS.ActiveMQ.ConnectionFactory(m_brokerURI);
m_connection = factory.CreateConnection(username, password);
Apache.NMS.ActiveMQ.Connection con = (Apache.NMS.ActiveMQ.Connection)m_connection;
ISession session = m_connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
try
{
session.DeleteDurableConsumer(strQueueName);
}
catch (Exception ex)
{
// log the error message
}
Update
Our scenario is quite simple.
A client built a queue and subscribed a consumer on a topic.
the client side closed the connection.
delete the consumer on the server side(as the example code in the last update)
Here is the snapshot of activemq broker via jconsole:
jconsole snapshot
We would like to remove the subscriber “7B0FD84D-6A2A-4921-967F-92B215E22751” by following method,
But always got this error "javax.jms.InvalidDestinationException : No durable subscription exists for: 7B0FD84D-6A2A-4921-967F-92B215E22751"
strSubscriberName = “7B0FD84D-6A2A-4921-967F-92B215E22751”
session.DeleteDurableConsumer(strSubscriberName);
To delete a durable subscription from the NMS API you use the DeleteDurableConsumer method defined in ISession. You must call this method from a Connection that uses the same client Id as was used when the subscription was created and you pass the name of the subscription that is to be removed. The method will fail if there is an active subscriber though so be prepared for that exception.
In the sample code you don't set a Client Id on the connection. When working with durable subscriptions you must, must, MUST always use the same client Id and subscription name. So in you same you will get this error until you set the client Id to the same value as the connection that created the subscription in the first place.
I have a cxf JAX-WS client. I added the failover strategy. The question is how the client can recovery from the backup solution and use again the primary URL? Because now after the client will switch to secondary URL remains there, will not use the primary URL even if this become available again.
The code for the client part is:
JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
factory.setServiceClass(GatewayPort.class);
factory.setAddress(this.configFile.getPrimaryURL());
FailoverFeature feature = new FailoverFeature();
SequentialStrategy strategy = new SequentialStrategy();
List<String> addList = new ArrayList<String>();
addList.add(this.configFile.getSecondaryURL());
strategy.setAlternateAddresses(addList);
feature.setStrategy(strategy);
List<AbstractFeature> features = new ArrayList<AbstractFeature>();
features.add(feature);
factory.setFeatures(features);
this.serviceSoap = (GatewayPort)factory.create();
Client client = ClientProxy.getClient(this.serviceSoap);
if (client != null)
{
HTTPConduit conduit = (HTTPConduit)client.getConduit();
HTTPClientPolicy policy = new HTTPClientPolicy();
policy.setConnectionTimeout(this.configFile.getTimeout());
policy.setReceiveTimeout(this.configFile.getTimeout());
conduit.setClient(policy);
}
You may add the primary URL to the alternate addresses list instead of setting that to JaxWsProxyFactoryBean. This way, since you are using SequentialStrategy, the primary URL will be checked first for every service call, if it fails then secodary URL will be tried.
You might as well try an alternative CXF failover feture with failback.
https://github.com/jaceko/cxf-circuit-switcher