I'm trying to receive a message asynchronously from IBM MQ:
#JmsListener(destination = "queue", containerFactory = "Factory", id = "start")
public Mono<Void> requestProcess(Message message) {return Mono.just("").then();
}
Catch:
Caused by: org.springframework.jms.support.converter.MessageConversionException: Cannot convert object of type [reactor.core.publisher.MonoLift] to JMS message. Supported message payloads are: String, byte array, Map<String,?>, Serializable object.
If I switch method type to simple void it works as supposed to. How can I set listener to receive messages in non-blocking reactive way?
The conversion that is failing is the input to requestProcess.
#JmsListener has a found a publisher of type reactor.core.publisher.MonoLift, but requestProcess is expecting a Message, and it doesn't have a conversion.
The way round this would be to changed the input signature for requestProcess to
#JmsListener(destination = "queue", containerFactory = "Factory", id = "start")
public Mono<Void> requestProcess(MonoLift<?> publisher) {
...
}
and modify the method body accordingly.
Related
I am trying to handle gRPC errors properly (Java, Spring-boot app).
Basically, I need too transfer error details from gRPC server to client, but I find it hard to understand the proper usage of StreamObserver.onError();
The method doc says:
"Receives a terminating error from the stream. May only be called once
and if called it must be the last method called. In particular if an
exception is thrown by an implementation of onError no further calls
to any method are allowed."
What does this "no further calls are allowed" mean? In the app that I maintain, they call other gRPC methods and they get java.lang.IllegalStateException: call already closed which is just fine, as per documentation.
I am wondering - should I (the developer) terminate the current java method (which usus gRPC calls) after an error is received? Like for example throwing an exception to stop execution. Or it is expected tht gRPC is going to terminate the execution.. (something like throwing an exception from gRPC)
Basically how do I properly use onError() and what should I expect and handle if I call it?
I need an explanation of its usage and effects.
There are two StreamObserver instances involved. One is for the inbound direction, which is the StreamObserver instance you implement and pass to the gRPC library. This is the StreamObserver containing your logic for how to handle responses. The other is for the outbound direction, which is the StreamObserver instance that gRPC library returns to you when calling the RPC method. This is the StreamObserver that you use to send requests. Most of the time, these two StreamObservers are interacting with each other (e.g., in a fully duplexed streaming call, the response StreamObserver usually calls the request StreamObserver's onNext() method, this is how you achieve ping-pong behavior).
"no further calls are allowed" means you should not call any more onNext(), onComplete() and/or onError() on the outbound direction StreamObserver when the inbound StreamObserver's onError() method is invoked, even if your implementation for the inbound onError() throws an exception. Since the inbound StreamObserver is invoked asynchronously, it has nothing to do with your method that encloses the StreamObserver's implementation.
For example:
public class HelloWorld {
private final HelloWorldStub stub;
private StreamObserver<HelloRequest> requestObserver;
...
private void sendRequest(String message) {
requestObserver.onNext(HelloRequest.newBuilder.setMessage(message).build());
}
public void start() {
stub.helloWorld(new StreamObserver<HelloResponse> {
#Override
public void onNext(HelloResponse response) {
sendRequest("hello from client");
// Optionally you can call onCompleted() or onError() on
// the requestObserver to terminate the call.
}
#Override
public void onCompleted() {
// You should not call any method on requestObserver.
}
#Override
public void onError(Throwable error) {
// You should not call any method on requestObserver.
}
});
}
}
It has nothing to do with the start() method.
The doc is also mentioning that you should not do things like
try {
requestObserver.onCompleted();
} catch(RuntimeException e) {
requestObserver.onError();
}
It's mostly for user's own StreamObserver implementations. StreamObserver's returned by gRPC never throws.
I've extracted a template for GRPC streaming which sort of abstracts away a lot of the GRPC boilerplate that also addresses the the logic for onError. In the DechunkingStreamObserver
I use the following general pattern for GRPC streaming which is something along the lines of
META DATA DATA DATA META DATA DATA DATA
An example of where I would use it would be to take one form and transform it to another form.
message SavedFormMeta {
string id = 1;
}
message SavedFormChunk {
oneof type {
SavedFormMeta meta = 1;
bytes data = 2;
}
}
rpc saveFormDataStream(stream SavedFormChunk) returns (stream SavedFormChunk) {}
I use a flag that would track the inError state to prevent further processing and catch exceptions on the onNext and onComplete both of which I redirect to onError which forwards the error to the server side.
The code below pulls the GRPC semantics and takes lamdas that do the processing.
/**
* Dechunks a GRPC stream from the request and calls the consumer when a complete object is created. This stops
* further processing once an error has occurred.
*
* #param <T> entity type
* #param <R> GRPC chunk message type
* #param <S> GRPC message type for response streams
*/
class DechunkingStreamObserver<T, R, S> implements StreamObserver<R> {
/**
* This function takes the current entity state and the chunk and returns a copy of the combined result. Note the combiner may modify the existing data, but may cause unexpected behaviour.
*/
private final BiFunction<T, R, T> combiner;
/**
* A function that takes in the assembled object and the GRPC response observer.
*/
private final BiConsumer<T, StreamObserver<S>> consumer;
/**
* Predicate that returns true if it is a meta chunk indicating a start of a new object.
*/
private final Predicate<R> metaPredicate;
/**
* this function gets the meta chunk and supplies a new object.
*/
private final Function<R, T> objectSupplier;
/**
* GRPC response observer.
*/
private final StreamObserver<S> responseObserver;
/**
* Currently being processed entity.
*/
private T current = null;
/**
* In error state. Starts {#code false}, but once it is set to {#code true} it stops processing {#link #onNext(Object)}.
*/
private boolean inError = false;
/**
* #param metaPredicate predicate that returns true if it is a meta chunk indicating a start of a new object.
* #param objectSupplier this function gets the meta chunk and supplies a new object
* #param combiner this function takes the current entity state and the chunk and returns a copy of the combined result. Note the combiner may modify the existing data, but may cause unexpected behaviour.
* #param consumer a function that takes in the assembled object and the GRPC response observer.
* #param responseObserver GRPC response observer
*/
DechunkingStreamObserver(
final Predicate<R> metaPredicate,
final Function<R, T> objectSupplier,
final BiFunction<T, R, T> combiner,
final BiConsumer<T, StreamObserver<S>> consumer,
final StreamObserver<S> responseObserver) {
this.metaPredicate = metaPredicate;
this.objectSupplier = objectSupplier;
this.combiner = combiner;
this.consumer = consumer;
this.responseObserver = responseObserver;
}
#Override
public void onCompleted() {
if (inError) {
return;
}
try {
if (current != null) {
consumer.accept(current, responseObserver);
}
responseObserver.onCompleted();
} catch (final Exception e) {
onError(e);
}
}
#Override
public void onError(final Throwable throwable) {
responseObserver.onError(throwable);
inError = true;
}
#Override
public void onNext(final R chunk) {
if (inError) {
return;
}
try {
if (metaPredicate.test(chunk)) {
if (current != null) {
consumer.accept(current, responseObserver);
}
current = objectSupplier.apply(chunk);
} else {
current = combiner.apply(current, chunk);
}
} catch (final Exception e) {
onError(e);
}
}
}
I have 4 lamdas
Predicate<R> metaPredicate which takes in a chunk and returns whether the chunk is meta or not.
Function<R, T> objectSupplier which takes in a meta chunk and creates a new object that is used by your module.
BiFunction<T, R, T> combiner, which takes in a data chunk and the current object and returns a new object that contains the combination.
BiConsumer<T, StreamObserver<S>> consumer which will consume a completed object. It also passes in a stream observer in the case of sending new objects in response.
the only thing you want to do is mark as return after calling the responseObserver.onError(); like below. because there is nothing to do after sending the error.
if(condition){
responseObserver.onError(StatusProto.toStatusException(status));
//this is the required part
return;
}else{
responseObserver.onComplete(DATA);
}
I am trying to use delayed exchange plugin, but somehow its not working for me and message are received without delay.
I tried following things :
a) enabled rabbitmq_delayed_message_exchange successfully and restarted rabbitmq server on ubuntu-16.04.
b) Declaring exchange
Map<String,Object> props = new HashMap<String,Object>();
props.put("x-delayed-type", "direct");
this.automationExchange = new DirectExchange(exchangeName,true,false, props);
c) Pushing message as
DefaultClassMapper typeMapper = QueueUtils.classMapper;
typeMapper.setDefaultType(type);
Jackson2JsonMessageConverter converter = QueueUtils.converter;
converter.setClassMapper(typeMapper);
RabbitTemplate template = AMQPRabbitMQTemplate.getAMQPTemplate();
template.setMessageConverter(converter);
template.convertAndSend(routingKey, message, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message m) throws AmqpException {
m.getMessageProperties().setDelay(delayMiliSeconds);
m.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
return m;
}
});
Now when i am printing message
public void onMessage(Message message, Channel channel) throws Exception{
System.out.println(message.getMessageProperties().getDelay());
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}
It is printing null for getDelay, which ideally should be negative of set value as per https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq.
Please let me know if i am doing something wrong.
I am using 1.6.8.RELEASE version for spring-amqp and spring-rabbit.
In order to avoid unexpected propagation of headers from an inbound message to an outbound message, certain headers for inbound messages are provided by MessageProperties.getReceived... methods.
In this case, the header is in MessageProperties.getReceivedDelay().
You also need setDelayed(true) on automationExchange before declaring it with the admin.
I presume you have set the exchange as the default in the RabbitTemplate too.
I have a use case where i have to re-queue the message with updated properties , Messages are getting re queued but message properties are not getting updated
public class TestListener implements MessageListener{
#Override
public void onMessage(Message arg0) {
MessageProperties properties = arg0.getMessageProperties();
int count = properties.getMessageCount();
System.out.println(count);
properties.setMessageCount(++count);
throw new AmqpException("test");
}
But the value of count always prints its always as 0
You can't do that - the amqp protocol does not support sending data back when rejecting a message.
You have to republish the message yourself, e.g with a RabbitTemplate.send() call.
You should also not use a "system" property for your own purposes; use messageGetProperties().set("myHeader", count++).
I am implementing a MassTransit middleware in my receive end point to record the performance of handling the message, i want to get the message type from the PipeContext, how can i get it?
public async Task Send(T context, IPipe<T> next)
{
// I want to know the message type from here so that i can log it
using (_logger.BeginTimedOperation("Time for handling message", null, LogEventLevel.Debug))
{
await next.Send(context);
}
}
You would need to intercept at the ConsumeContext, which has a property for the message types from the serialization envelope.
Then, you can get the supported message types using:
IEnumerable<string> SupportedMessageTypes {get;}
That should get you what you need to log the message type with the duration.
So a filter along the lines of:
public class LogMessageTypeFilter :
IFilter<ConsumeContext>
{
}
Implement the send method, call next within the method, and then take action after the next pipe completes.
I am trying to understand how I would implement message framing with WCF. The goal is to create a server in WCF that can handle proprietary formats over Tcp. I can't use the net.Tcp binding because that is only for SOAP.
I need to write a custom channel that would receive messages in the following format
. An example message would be "5 abcde". In particular I am not sure how to do framing in my custom channel.
Here is some sample code
class CustomChannel: IDuplexSessionChannel
{
private class PendingRead
{
public NetworkStream Stream = null;
public byte[] Buffer = null;
public bool IsReading = false;
}
private CommunicationState state = CommunicationState.Closed;
private TcpClient tcpClient = null;
private MessageEncoder encoder = null;
private BufferManager bufferManager = null;
private TransportBindingElement bindingElement = null;
private Uri uri = null;
private PendingRead pendingRead;
public CustomChannel(Uri uri, TransportBindingElement bindingElement, MessageEncoderFactory encoderFactory, BufferManager bufferManager, TcpClient tcpClient)
{
this.uri = uri;
this.bindingElement = bindingElement;
this.tcpClient = tcpClient;
this.bufferManager = bufferManager;
state = CommunicationState.Created;
}
public IAsyncResult BeginTryReceive(TimeSpan timeout, AsyncCallback callback, object state)
{
if (this.state != CommunicationState.Opened) return null;
byte[] buffer = bufferManager.TakeBuffer(tcpClient.Available);
NetworkStream stream = tcpClient.GetStream();
pendingRead = new PendingRead { Stream = stream, Buffer = buffer, IsReading = true };
IAsyncResult result = stream.BeginRead(buffer, 0, buffer.Length, callback, state);
return result;
}
public bool EndTryReceive(IAsyncResult result, out Message message)
{
int byteCount = tcpClient.Client.EndReceive(result);
string content = Encoding.ASCII.GetString(pendingRead.buffer)
// framing logic here
Message.CreateMessage( ... )
}
}
So basically the first time around EndTryReceive could just get a piece of the message from the pending read buffer "5 ab". Then the second time around it could get the rest of the message. The problem is when EndTryReceive gets called the first time, I am forced to create a Message object, this means that there will be a partial Message going up the channel stack.
What I really want to do is to make sure that I have my full message "5 abcde" in the buffer, so that when I construct the message in EndTryReceive it is a full message.
Does anyone have any examples of how they are doing custom framing with WCF?
Thanks,
Vadim
Framing at the wire level is not something that the WCF channel model really cares about; it's pretty much up to you to handle it.
What I mean by this is that it is your responsibility to ensure that your transport channel returns "entire" messages on a receive (streaming changes that a bit, but only up to a point).
In your case, it seems you're translating receive operations on your channel directly into receive operations on the underlying socket, and that just won't do, because that won't give you a chance to enforce your own framing rules.
So really, a single receive operation on your channel might very well translate to more than one receive operation on the underlying socket, and that's fine (and you can still do all that async, so it doesn't need to affect that part).
So basically the question becomes: what's your protocol framing model look like? Wild guess here, but it looks like messages are length prefixed, with the length encoded as a decimal string? (looks annoying).
I think your best bet in that case would be to have your transport buffer incoming data (say, up to 64KB of data or whatever), and then on each receive operation check the buffer to see if it contains enough bytes to extract the length of the incoming message. If so, then either read as many bytes as necessary from the buffer, or flush the buffer and read as many bytes from the socket. You'll have to be careful as, depending on how your protocol works, I'm assuming you might end up reading partial messages before you actually need them.
I agree with the thomasr. You can find some basic inspiration in Microsoft Technology Sample "ChunkingChannel".