In case of resilience i am trying to simulate a database breakdown.
public void createParticipationCheckWorkflowsForThis(Integer numberOfAdvices) {
participationAdviceSource.getParticipationAdvicesByAMaximumLimitOf(numberOfAdvices) // i want to retry this
.subscribe(participationAdviceSender::sendAdviceToWorkflowEngine);
}
My test scenario specifies that on the first database call, which is using Spring-R2DBC i return a flux error and on its second call a correct result.
when(participationAdviceSource.getParticipationAdvicesByAMaximumLimitOf(anyInt()))
.thenReturn(Flux.error(new RuntimeException()))
.thenReturn(Flux.just(ParticipationAdvice.builder().participationId(1L).build()));
My specific question is how can i retry the invoker/producer, because the retry mechanism of reactive retries the subscribe not the producer
You can use retry or retryWhen along with the Retry API:
Flux.defer(() -> participationAdviceSource.getParticipationAdvicesByAMaximumLimitOf(numberOfAdvices))
.retryWhen(Retry.backoff(retryMaxAttempts, Duration.ofMillis(retryMinBackoff)).maxBackoff(Duration.ofMillis(retryMaxBackoff)))
A Flux.defer wrapper is required because retry and retryWhen work by re-subscribing to the Flux (if the Flux throws an error and the retry conditions are fulfilled).
Related
in my kafka streams app, I need to re-try processing a message whenever a particular type of exception is thrown in the processing logic.
Rather than wrapping my logic in the RetryTemplate (am using springboot), am considering just simply writing the message back into the input topic, my assumption is that this message will be added to the back of the log in the appropriate partition and it will eventually be re-processed.
Am aware that this would mess up the ordering and am okay with that.
My question is, would kafka streams have an issue when it encounters a message that was supposedly already processed in the past (am assuming kafka streams has a way of marking the messages it has processed especially when exactly is enabled)?
Here is an example of the code am considering for this solution.
val branches = streamsBuilder.stream(inputTopicName)
.mapValues { it -> myServiceObject.executeSomeLogic(it) }
.branch(
{ _, value -> value is successfulResult() }, // success
{ _, error -> error is errorResult() }, // exception was thrown
)
branches[0].to(outputTopicName)
branches[1].to(inputTopicName) //write them back to input as a way of retrying
I have a common rest controller:
private final KafkaReceiver<String, Domain> receiver;
#GetMapping(produces = MediaType.APPLICATION_STREAM_JSON_VALUE)
public Flux<Domain> produceFluxMessages() {
return receiver.receive().map(ConsumerRecord::value)
.timeout(Duration.ofSeconds(2));
}
What I am trying to achieve is to collect messages from Kafka topic for a certain period of time, and then just stop consuming and consider this flux completed. If I remove timeout and open this in a browser, I am getting messages forever, downloading never stops. And with this timeout consuming stops after 2 seconds, but I'm getting an exception:
java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 2000ms in 'map' (and no fallback has been configured)
Is there a way to successfully complete Flux after timeout?
There's multiple overloads of the timeout() method - you're using the standard one that throws an exception on timeout.
Instead, just use the overloaded timeout method to provide an empty default publisher to fallback to:
timeout(Duration.ofSeconds(2), Mono.empty())
(Note in a general case you could explicitly capture the TimeoutException and fallback to an empty publisher using onErrorResume(TimeoutException.class, e -> Mono.empty()), but that's much less preferable to using the above option where possible.)
There is a flow as per below scenario.
Initiating Party : PartyA
Responding Party : PartyB
Transaction 1: Input StateA - ContractA results in output StateB - ContractA. Participants are PartyA and PartyB
Transaction 2: Input StateB - ContractA and no output. Participants are PartyA and PartyB
Is this possible in Corda? Please do share an example with response. Thanks.
It sounds like you're getting two different error messages:
If you don't try and initiate a second flow-session to get the second signature, you get something like:
net.corda.core.flows.UnexpectedFlowEndException: Counterparty flow on
O=Mock Company 2, L=London, C=GB has completed without sending data
While if you do initiate a second flow-session to get the second signature, you get something like:
java.lang.IllegalStateException: Attempted to initiateFlow() twice in
the same InitiatingFlow
com.example.flow.ExampleFlow$Initiator#312d7fe4 for the same party
O=Mock Company 2, L=London, C=GB. This isn't supported in this version
of Corda. Alternatively you may initiate a new flow by calling
initiateFlow() in an #InitiatingFlow sub-flow.
In the first case, the error is caused by the fact that the counterparty's flow has already completed. You try and get around this by creating a second flow session, but each Initiating flow can only initiate a single flow-session with a given counterparty.
Instead, you simply need to modify the responder flow to sign twice. For example:
#InitiatedBy(Initiator::class)
class Acceptor(val otherPartyFlow: FlowSession) : FlowLogic<Unit>() {
#Suspendable
override fun call() {
val signTransactionFlow = object : SignTransactionFlow(otherPartyFlow) {
override fun checkTransaction(stx: SignedTransaction) = requireThat {
// Transaction checks...
}
}
subFlow(signTransactionFlow)
subFlow(signTransactionFlow)
}
}
yes it is possible .Please find the link to know more
https://docs.corda.net/key-concepts-transactions.html
I'm having trouble to identify a way to delay message level in SpringAMQP.
I call a Webservice if the service is not available or if it throws some exception I store all the requests into RabbitMQ queue and i keep retry the service call until it executes successfully. If the service keeps throwing an error or its not available the rabbitMQ listener keeps looping.( Meaning Listener retrieves the message and make service call if any error it re-queue the message)
I restricted the looping until X hours using MessagePostProcessor however i wanted to enable delay on message level and every time it tries to access the service. For example 1st try 3000ms delay and second time 6000ms so on until i try x number of time.
It would be great if you provide a few examples.
Could you please provide me some idea on this?
Well, it isn't possible the way you do that.
Message re-queuing is fully similar to transaction rallback, where the system returns to the state before an exception. So, definitely you can't modify a message to return to the queue.
Probably you have to take a look into Spring Retry project for the same reason and poll message from the queue only once and retries in memory until successful answer or retry policy exhausting. In the end you can just drop message from the queue or move it into DLQ.
See more info in the Reference Manual.
I added CustomeMessage delay exchange
#Bean
CustomExchange delayExchange() {
Map<String, Object> args = new HashMap<>();
args.put("x-delayed-type", "direct");
return new CustomExchange("delayed-exchange", "x-delayed-message", true, false, args);
}
Added MessagePostProcessor
if (message.getMessageProperties().getHeaders().get("x-delay") == null) {
message.getMessageProperties().setHeader("x-delay", 10000);
} else {
Integer integer = (Integer) message.getMessageProperties().getHeaders().get("x-delay");
if (integer < 60000) {
integer = integer + 10000;
message.getMessageProperties().setHeader("x-delay", integer);
}
}
First time it delays 30 seconds and adds 10seconds each time till it reaches 600 seconds.This should be configurable.
And finally send the message to
rabbitTemplate.convertAndSend("delayed-exchange", queueName,message, rabbitMQMessagePostProcessor);
I'd like to send a message to a RabbitMQ server and then wait for a reply message (on a "reply-to" queue). Of course, I don't want to wait forever in case the application processing these messages is down - there needs to be a timeout. It sounds like a very basic task, yet I can't find a way to do this. I've now run into this problem with Java API.
The RabbitMQ Java client library now supports a timeout argument to its QueueConsumer.nextDelivery() method.
For instance, the RPC tutorial uses the following code:
channel.basicPublish("", requestQueueName, props, message.getBytes());
while (true) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
if (delivery.getProperties().getCorrelationId().equals(corrId)) {
response = new String(delivery.getBody());
break;
}
}
Now, you can use consumer.nextDelivery(1000) to wait for maximum one second. If the timeout is reached, the method returns null.
channel.basicPublish("", requestQueueName, props, message.getBytes());
while (true) {
// Use a timeout of 1000 milliseconds
QueueingConsumer.Delivery delivery = consumer.nextDelivery(1000);
// Test if delivery is null, meaning the timeout was reached.
if (delivery != null &&
delivery.getProperties().getCorrelationId().equals(corrId)) {
response = new String(delivery.getBody());
break;
}
}
com.rabbitmq.client.QueueingConsumer has a nextDelivery(long timeout) method, which will do what you want. However, this has been deprecated.
Writing your own timeout isn't so hard, although it may be better to have an ongoing thread and a list of in-time identifiers, rather than adding and removing consumers and associated timeout threads all the time.
Edit to add: Noticed the date on this after replying!
There is similar question. Although it's answers doesn't use java, maybe you can get some hints.
Wait for a single RabbitMQ message with a timeout
I approached this problem using C# by creating an object to keep track of the response to a particular message. It sets up a unique reply queue for a message, and subscribes to it. If the response is not received in a specified timeframe, a countdown timer cancels the subscription, which deletes the queue. Separately, I have methods that can be synchronous from my main thread (uses a semaphore) or asynchronous (uses a callback) to utilize this functionality.
Basically, the implementation looks like this:
//Synchronous case:
//Throws TimeoutException if timeout happens
var msg = messageClient.SendAndWait(theMessage);
//Asynchronous case
//myCallback receives an exception message if there is a timeout
messageClient.SendAndCallback(theMessage, myCallback);