I want to receive multiple messages on one request from Spring AMQP RabbitTemplate (https://docs.spring.io/spring-amqp/api/org/springframework/amqp/rabbit/core/RabbitTemplate.html).
But I can't find the method working this way. Can I do that in Spring AMQP?
I don't want to use RabbitListener because it consumes all the time..
No, there is no such an API on the RabbitTemplate, as well as there is no way to send several message per single publish.
You may consider to use this one instead of receive():
/**
* Execute the callback with a channel and reliably close the channel afterwards.
* #param action the call back.
* #param <T> the return type.
* #return the result from the
* {#link ChannelCallback#doInRabbit(com.rabbitmq.client.Channel)}.
* #throws AmqpException if one occurs.
*/
#Nullable
<T> T execute(ChannelCallback<T> action) throws AmqpException;
So, you can call the Channel.basicGet() in a loop on the same opened channel.
However there is also this more higher API:
/**
* Invoke the callback and run all operations on the template argument in a dedicated
* thread-bound channel and reliably close the channel afterwards.
* #param action the call back.
* #param <T> the return type.
* #return the result from the
* {#link OperationsCallback#doInRabbit(RabbitOperations operations)}.
* #throws AmqpException if one occurs.
* #since 2.0
*/
#Nullable
default <T> T invoke(OperationsCallback<T> action) throws AmqpException {
So, you can call RabbitOperations.receive() in a loop.
Related
I am new to Kotlin, and while I am aware of the concept about ListenableFuture, and how we can add callbacks to it for checking for success or failure.
However, I am trying to use await (https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-guava/kotlinx.coroutines.guava/await.html) feature.
There is a library function I am calling and this is how it looks like:
#NonNull
ListenableFuture<Void> removeAsync(#NonNull String queryExpression,
#NonNull SearchSpec searchSpec);
/**
* #deprecated use {#link #removeAsync}
*
* #param queryExpression Query String to search.
* #param searchSpec Spec containing schemaTypes, namespaces and query expression
* indicates how document will be removed. All specific about how to
* scoring, ordering, snippeting and resulting will be ignored.
* #return The pending result of performing this operation.
*/
#NonNull
#Deprecated
default ListenableFuture<Void> remove(#NonNull String queryExpression,
#NonNull SearchSpec searchSpec) {
return removeAsync(queryExpression, searchSpec);
}
Now, I am calling this function like this:
suspend fun removeDocs() {
val removeQuery = SearchSpec.Builder().addFilterSchemas("MyDocument").build()
appSearchSession.removeAsync("", removeQuery).await()
}
Now, I don't understand how do I check for the success of this operation? What sort of error handling can I do in removeDocs function?
Coroutines allow to execute asynchronous operations using classic sequential code. Then we don't have to use callbacks or check for some flags in order to handle exceptions. Exceptions are handled directly - by throwing them.
If await() didn't throw an exception, that means the operation succeeded:
appSearchSession.removeAsync("", removeQuery).await()
// if we get here, remove succeeded
If you have to react somehow to the failed operation, use try...catch:
try {
appSearchSession.removeAsync("", removeQuery).await()
// remove succeeded
catch (e: Throwable) {
// remove failed
}
I'm using RabbitMQ and Spring Boot and I want to create all queues and exchanges declared when application starts.
I have one exchange and two queues binding to it. Also I have another queue without binding to any exchange.
The exchange declaration is like this:
#Bean
TopicExchange exchange() {
return new TopicExchange(name, false, false);
}
And the queues:
#Bean
Queue queue1() {
return new Queue(name, false);
}
#Bean
Binding bindingLogger(Queue queue1, TopicExchange exchange) {
return BindingBuilder.bind(queue1).to(exchange).with("routingKey");
}
And the queue without binding:
#Bean
Queue queue2() {
return new Queue(name, false);
}
Also I have used #Component tag in the classes.
I think this is ok because if I add a "dummy" #RabbitListener all queues and the exchange are created. Adding something like this:
#Component
public class DummyListener {
#RabbitListener(queues = {FAKE_QUEUE_NAME})
public void dummyMethod(Message message, Channel channel) {
// The code never will enter here because nobody are going to
// insert data into the queue.
// This method is only to create queues and exchange on init
}
}
But I think this is a dirty solution, is neccesary create a listener which never will be triggered and a queue which never will be used.
And, as I said before, the queues and exchange declarations works perfectly and are created when project start if this "dummy listener" is implemented.
So, how can I create the exchange and queues (if not exists) when start the application? Is there a more elegant way?
I've read about rabbitAdmin but I think this is to create a new queue at runtime (actually I don't know if I have to manage in a different way at start and at runtime)
Thanks in advance.
Those Declarables are populated into RabbitMQ broker when the connection is opened.
This really happens with the listener container starting from that #RabbitListener.
All the hard logic is done from the mentioned RabbitAdmin:
/**
* If {#link #setAutoStartup(boolean) autoStartup} is set to true, registers a callback on the
* {#link ConnectionFactory} to declare all exchanges and queues in the enclosing application context. If the
* callback fails then it may cause other clients of the connection factory to fail, but since only exchanges,
* queues and bindings are declared failure is not expected.
*
* #see InitializingBean#afterPropertiesSet()
* #see #initialize()
*/
#Override
public void afterPropertiesSet() {
Another point of connection is of course a RabbitTemplate when you produce the message into an exchange.
If you really are not going to do any consumption or production, you can consider to inject an AmqpAdmin into your service and call its initialize() when you need:
/**
* Declares all the exchanges, queues and bindings in the enclosing application context, if any. It should be safe
* (but unnecessary) to call this method more than once.
*/
#Override // NOSONAR complexity
public void initialize() {
However from here is the question: what is the point to have all those declarations in your application if you are not going to use them in the further logic. Looks like a mix of concerns and abuse of AMQP API. Better to have those entities declared outside of your application, e.g. using RabbitMQ Management console or command line util...
You can simply open the connection. If you are using Spring Boot, see this answer.
If you are not using Spring Boot, add a #Bean that implements SmartLifecycle and open the connection in start().
I am trying to handle gRPC errors properly (Java, Spring-boot app).
Basically, I need too transfer error details from gRPC server to client, but I find it hard to understand the proper usage of StreamObserver.onError();
The method doc says:
"Receives a terminating error from the stream. May only be called once
and if called it must be the last method called. In particular if an
exception is thrown by an implementation of onError no further calls
to any method are allowed."
What does this "no further calls are allowed" mean? In the app that I maintain, they call other gRPC methods and they get java.lang.IllegalStateException: call already closed which is just fine, as per documentation.
I am wondering - should I (the developer) terminate the current java method (which usus gRPC calls) after an error is received? Like for example throwing an exception to stop execution. Or it is expected tht gRPC is going to terminate the execution.. (something like throwing an exception from gRPC)
Basically how do I properly use onError() and what should I expect and handle if I call it?
I need an explanation of its usage and effects.
There are two StreamObserver instances involved. One is for the inbound direction, which is the StreamObserver instance you implement and pass to the gRPC library. This is the StreamObserver containing your logic for how to handle responses. The other is for the outbound direction, which is the StreamObserver instance that gRPC library returns to you when calling the RPC method. This is the StreamObserver that you use to send requests. Most of the time, these two StreamObservers are interacting with each other (e.g., in a fully duplexed streaming call, the response StreamObserver usually calls the request StreamObserver's onNext() method, this is how you achieve ping-pong behavior).
"no further calls are allowed" means you should not call any more onNext(), onComplete() and/or onError() on the outbound direction StreamObserver when the inbound StreamObserver's onError() method is invoked, even if your implementation for the inbound onError() throws an exception. Since the inbound StreamObserver is invoked asynchronously, it has nothing to do with your method that encloses the StreamObserver's implementation.
For example:
public class HelloWorld {
private final HelloWorldStub stub;
private StreamObserver<HelloRequest> requestObserver;
...
private void sendRequest(String message) {
requestObserver.onNext(HelloRequest.newBuilder.setMessage(message).build());
}
public void start() {
stub.helloWorld(new StreamObserver<HelloResponse> {
#Override
public void onNext(HelloResponse response) {
sendRequest("hello from client");
// Optionally you can call onCompleted() or onError() on
// the requestObserver to terminate the call.
}
#Override
public void onCompleted() {
// You should not call any method on requestObserver.
}
#Override
public void onError(Throwable error) {
// You should not call any method on requestObserver.
}
});
}
}
It has nothing to do with the start() method.
The doc is also mentioning that you should not do things like
try {
requestObserver.onCompleted();
} catch(RuntimeException e) {
requestObserver.onError();
}
It's mostly for user's own StreamObserver implementations. StreamObserver's returned by gRPC never throws.
I've extracted a template for GRPC streaming which sort of abstracts away a lot of the GRPC boilerplate that also addresses the the logic for onError. In the DechunkingStreamObserver
I use the following general pattern for GRPC streaming which is something along the lines of
META DATA DATA DATA META DATA DATA DATA
An example of where I would use it would be to take one form and transform it to another form.
message SavedFormMeta {
string id = 1;
}
message SavedFormChunk {
oneof type {
SavedFormMeta meta = 1;
bytes data = 2;
}
}
rpc saveFormDataStream(stream SavedFormChunk) returns (stream SavedFormChunk) {}
I use a flag that would track the inError state to prevent further processing and catch exceptions on the onNext and onComplete both of which I redirect to onError which forwards the error to the server side.
The code below pulls the GRPC semantics and takes lamdas that do the processing.
/**
* Dechunks a GRPC stream from the request and calls the consumer when a complete object is created. This stops
* further processing once an error has occurred.
*
* #param <T> entity type
* #param <R> GRPC chunk message type
* #param <S> GRPC message type for response streams
*/
class DechunkingStreamObserver<T, R, S> implements StreamObserver<R> {
/**
* This function takes the current entity state and the chunk and returns a copy of the combined result. Note the combiner may modify the existing data, but may cause unexpected behaviour.
*/
private final BiFunction<T, R, T> combiner;
/**
* A function that takes in the assembled object and the GRPC response observer.
*/
private final BiConsumer<T, StreamObserver<S>> consumer;
/**
* Predicate that returns true if it is a meta chunk indicating a start of a new object.
*/
private final Predicate<R> metaPredicate;
/**
* this function gets the meta chunk and supplies a new object.
*/
private final Function<R, T> objectSupplier;
/**
* GRPC response observer.
*/
private final StreamObserver<S> responseObserver;
/**
* Currently being processed entity.
*/
private T current = null;
/**
* In error state. Starts {#code false}, but once it is set to {#code true} it stops processing {#link #onNext(Object)}.
*/
private boolean inError = false;
/**
* #param metaPredicate predicate that returns true if it is a meta chunk indicating a start of a new object.
* #param objectSupplier this function gets the meta chunk and supplies a new object
* #param combiner this function takes the current entity state and the chunk and returns a copy of the combined result. Note the combiner may modify the existing data, but may cause unexpected behaviour.
* #param consumer a function that takes in the assembled object and the GRPC response observer.
* #param responseObserver GRPC response observer
*/
DechunkingStreamObserver(
final Predicate<R> metaPredicate,
final Function<R, T> objectSupplier,
final BiFunction<T, R, T> combiner,
final BiConsumer<T, StreamObserver<S>> consumer,
final StreamObserver<S> responseObserver) {
this.metaPredicate = metaPredicate;
this.objectSupplier = objectSupplier;
this.combiner = combiner;
this.consumer = consumer;
this.responseObserver = responseObserver;
}
#Override
public void onCompleted() {
if (inError) {
return;
}
try {
if (current != null) {
consumer.accept(current, responseObserver);
}
responseObserver.onCompleted();
} catch (final Exception e) {
onError(e);
}
}
#Override
public void onError(final Throwable throwable) {
responseObserver.onError(throwable);
inError = true;
}
#Override
public void onNext(final R chunk) {
if (inError) {
return;
}
try {
if (metaPredicate.test(chunk)) {
if (current != null) {
consumer.accept(current, responseObserver);
}
current = objectSupplier.apply(chunk);
} else {
current = combiner.apply(current, chunk);
}
} catch (final Exception e) {
onError(e);
}
}
}
I have 4 lamdas
Predicate<R> metaPredicate which takes in a chunk and returns whether the chunk is meta or not.
Function<R, T> objectSupplier which takes in a meta chunk and creates a new object that is used by your module.
BiFunction<T, R, T> combiner, which takes in a data chunk and the current object and returns a new object that contains the combination.
BiConsumer<T, StreamObserver<S>> consumer which will consume a completed object. It also passes in a stream observer in the case of sending new objects in response.
the only thing you want to do is mark as return after calling the responseObserver.onError(); like below. because there is nothing to do after sending the error.
if(condition){
responseObserver.onError(StatusProto.toStatusException(status));
//this is the required part
return;
}else{
responseObserver.onComplete(DATA);
}
I have an integration test for my Spring Integration config, which consumes messages from a JMS topic with durable subscription. For testing, I am using ActiveMQ instead of Tibco EMS.
The issue I have is that I have to delay sending the first message to the endpoint using a sleep call at the beginning of our test method. Otherwise the message is dropped.
If I remove the setting for durable subscription and selector, then the first message can be sent right away without delay.
I'd like to get rid of the sleep, which is unreliable. Is there a way to check if the endpoint is completely setup before I send the message?
Below is the configuration.
Thanks for your help!
<int-jms:message-driven-channel-adapter
id="myConsumer" connection-factory="myCachedConnectionFactory"
destination="myTopic" channel="myChannel" error-channel="errorChannel"
pub-sub-domain="true" subscription-durable="true"
durable-subscription-name="testDurable"
selector="..."
transaction-manager="emsTransactionManager" auto-startup="false"/>
If you are using a clean embedded activemq for the test, the durability of the subscription is irrelevant until the subscription is established. So you have no choice but to wait until that happens.
You could avoid the sleep by sending a series of startup messages and only start the real test when the last one is received.
EDIT
I forgot that there is a methodisRegisteredWithDestination() on the DefaultMessageListenerContainer.
Javadocs...
/**
* Return whether at least one consumer has entered a fixed registration with the
* target destination. This is particularly interesting for the pub-sub case where
* it might be important to have an actual consumer registered that is guaranteed
* not to miss any messages that are just about to be published.
* <p>This method may be polled after a {#link #start()} call, until asynchronous
* registration of consumers has happened which is when the method will start returning
* {#code true} – provided that the listener container ever actually establishes
* a fixed registration. It will then keep returning {#code true} until shutdown,
* since the container will hold on to at least one consumer registration thereafter.
* <p>Note that a listener container is not bound to having a fixed registration in
* the first place. It may also keep recreating consumers for every invoker execution.
* This particularly depends on the {#link #setCacheLevel cache level} setting:
* only {#link #CACHE_CONSUMER} will lead to a fixed registration.
*/
We use it in some channel tests, where we get the container using reflection and then poll the method until we are subscribed to the topic.
/**
* Blocks until the listener container has subscribed; if the container does not support
* this test, or the caching mode is incompatible, true is returned. Otherwise blocks
* until timeout milliseconds have passed, or the consumer has registered.
* #see DefaultMessageListenerContainer#isRegisteredWithDestination()
* #param timeout Timeout in milliseconds.
* #return True if a subscriber has connected or the container/attributes does not support
* the test. False if a valid container does not have a registered consumer within
* timeout milliseconds.
*/
private static boolean waitUntilRegisteredWithDestination(SubscribableJmsChannel channel, long timeout) {
AbstractMessageListenerContainer container =
(AbstractMessageListenerContainer) new DirectFieldAccessor(channel).getPropertyValue("container");
if (container instanceof DefaultMessageListenerContainer) {
DefaultMessageListenerContainer listenerContainer =
(DefaultMessageListenerContainer) container;
if (listenerContainer.getCacheLevel() != DefaultMessageListenerContainer.CACHE_CONSUMER) {
return true;
}
while (timeout > 0) {
if (listenerContainer.isRegisteredWithDestination()) {
return true;
}
try {
Thread.sleep(100);
} catch (InterruptedException e) { }
timeout -= 100;
}
return false;
}
return true;
}
What is the best way to unit test this code? I need to establish a consistent check for httpResponse which sendError() when condition is true. Thanks in advance!
Edit: Unfortunately, this filter is not with Spring MVC so my choice is limited.
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain filterchain) throws IOException, ServletException {
String ipAddress = request.getRemoteAddr();
if( SomeParameterCheckingFunction ((request)) ) {
logger.error("Error detected! Time: " + new Date().toString() + ", Originating IP: " + ipAddress);
if (response instanceof HttpServletResponse){
HttpServletResponse httpResponse = (HttpServletResponse) response;
httpResponse.sendError(HttpServletResponse.SC_FORBIDDEN,"You are not allowed to access the server!");
}
}
filterchain.doFilter(request, response);
}
For example when using the Mockito mock framework, the provided doFilter() method could be JUnit tested using below test case:
#Test
public void testDoFilter() throws IOException, ServletException {
// create the objects to be mocked
HttpServletRequest httpServletRequest = mock(HttpServletRequest.class);
HttpServletResponse httpServletResponse = mock(HttpServletResponse.class);
FilterChain filterChain = mock(FilterChain.class);
// mock the getRemoteAddr() response
when(httpServletRequest.getRemoteAddr()).thenReturn("198.252.206.16");
AccessFilter accessFilter = new AccessFilter();
accessFilter.doFilter(httpServletRequest, httpServletResponse,
filterChain);
// verify if a sendError() was performed with the expected values
verify(httpServletResponse).sendError(HttpServletResponse.SC_FORBIDDEN,
"You are not allowed to access the server!");
}
One option might be to pull the definition of SomeParameterCheckingFunction out of the filter itself (and consume it in the filter). Then, you could separately unit test that logic. But, I'm not sure this will be sufficient for you.
Another option might be to test the filter class using Mockito. This would involve some labor, mocking the request, response, filterchain, logging, and such. You should be able to verify the response.sendError() call afterwards something like so (assumes static import of Mockito.* methods):
//setup mock with concrete class for instanceof check
ServletResponse resp = mock(HttpServletResponse.class);
//define behavior for desired method with void return type. this appears optional for mocks.
//doNothing().when(resp).sendError(anyInt(), anyString());
//your test here, passing in resp
if(resp instanceof HttpServletResponse) resp.sendError(400, "blah");
//verify method was called
verify(resp).sendError(400, "blah");
There may or may not be a way to test your filter with HttpUnit, but that might not meet your desire for a proper unit test.
Updated Feb 2018: OpenBrace Limited has closed down, and its ObMimic product is no longer supported.
Here's an example of how you could test the "sendError" using my ObMimic library of out-of-container test-doubles for the Servlet API.
First, some notes on the given doFilter code:
"((request))" is presumably meant to just be "(request)".
The nature of "logger" isn't shown, so I've ignored that for now as irrelevant.
The filter as given will check non-HTTP requests but if they're rejected the usual log message is written but then sendError is skipped and the normal processing carried out instead. Depending on what you're trying to do, you might want to reject non-HTTP requests up-front, or not check them at all, or do some other type of rejection in place of "sendError". For now I'm assuming we're only interested in how to test "sendError" on HTTP requests.
The example test below has shown that even when sendError is called the processing still drops through to the normal "doFilter" call. Assuming this isn't intentional, the "sendError" should be followed by a "return", or "doFilter" should be in an "else" clause.
For the example code below:
I've used JUnit as the test framework, but it could just as well be TestNG or anything else.
For "SomeParameterCheckingFunction", I've just used one that returns true if the remote IP address is "1.1.1.1".
"sendError" produces an HTML response that displays the given message. For this example I just check that the response body contains the given message. If you instead wanted to check what parameter values were passed to "sendError", the "Professional Edition" of ObMimic has a "history" feature that would let you do that. Or you could do a "sendError" on a separate response instance and then check that the response bodies match exactly.
Anyway, here's the example code:
package com.openbrace.experiments.examplecode.stackoverflow13365536;
import static org.junit.Assert.*;
import com.openbrace.experiments.examplecode.stackoverflow13365536.YourFilter;
import com.openbrace.obmimic.mimic.servlet.FilterChainMimic;
import com.openbrace.obmimic.mimic.servlet.FilterConfigMimic;
import com.openbrace.obmimic.mimic.servlet.ServletMimic;
import com.openbrace.obmimic.mimic.servlet.http.HttpServletRequestMimic;
import com.openbrace.obmimic.mimic.servlet.http.HttpServletResponseMimic;
import com.openbrace.obmimic.support.servlet.EndPoint;
import org.hamcrest.CoreMatchers;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
/**
* Example tests for {#link YourFilter#doFilter(ServletRequest, ServletResponse,
* FilterChain)}.
*
* #author Mike Kaufman, OpenBrace Limited
*/
public class YourFilterTest {
/** The filter to be tested by this instance's test. */
private YourFilter filter;
/** The "mimic" request to be used in this instance's test. */
private HttpServletRequestMimic request;
/** The "mimic" response to be used in this instance's test. */
private HttpServletResponseMimic response;
/** The filter chain to be used in this instance's test. */
private FilterChainMimic filterChain;
/**
* Set up for this instance's test by creating an initialized filter and a
* request, response and filter chain for use in the test.
*
* #throws ServletException if the filter's init method throws such an
* exception.
*/
#Before
public void setUp() throws ServletException {
/*
* Note that for this example's simple filter and tests:
* - We don't need anything particular in the filter's FilterConfig.
* - We don't need any particular ServletContext, so we can leave
* ObMimic to use its default ServletContext throughout.
* - We don't need to retain any references to the filter's FilterConfig
* or ServletContext.
* - We can use a FilterChainMimic with default values as the filter
* chain. This has a ServletMimic as the next thing in the chain, and
* ServletMimic keeps a count of the calls to its service method, so
* we can use this to check whether the filter chain's doFilter has
* been invoked.
*/
filter = new YourFilter();
filter.init(new FilterConfigMimic());
request = new HttpServletRequestMimic();
response = new HttpServletResponseMimic();
filterChain = new FilterChainMimic();
}
/**
* Test the doFilter method with an example request for which
* SomeParameterCheckingFunction returns true (so that the FORBIDDEN
* response should result).
*
* #throws ServletException if the servlet throws such an exception.
* #throws IOException if the servlet throws such an exception.
*/
#Test
public void testYourFilterWithForbiddenRequest()
throws ServletException, IOException {
// Configure the request so that SomeParameterCheckingFunction will
// return true, which for purposes of this example is triggered by a
// particular "remote address".
request.getMimicState().setRemoteEndPoint(
new EndPoint(null, "1.1.1.1", null));
// Invoke the doFilter method.
filter.doFilter(request, response, filterChain);
// Check that the response is now FORBIDDEN, and that its HTML content
// does include the expected text message.
int responseCode = response.getMimicState().getHttpStatusCode();
String responseBody = response.getMimicState().getBodyContentAsString();
String expectedMessage = "You are not allowed to access the server!";
assertEquals("Response has incorrect status code",
HttpServletResponse.SC_FORBIDDEN, responseCode);
Assert.assertThat("FORBIDDEN response does not include expected message",
responseBody, CoreMatchers.containsString(expectedMessage));
// Check that the filter chain was not invoked. As we've used a
// FilterChainMimic with default values, its "target" is a ServletMimic,
// so we can just check if there have been any calls to that Servlet.
ServletMimic targetServlet
= (ServletMimic) filterChain.getMimicState().getTarget();
boolean filterChainInvoked
= targetServlet.getMimicState().getServiceCount() > 0;
assertFalse("FORBIDDEN response but filterChain.doFilter still called",
filterChainInvoked);
}
}
If you want to try it, there are full details of ObMimic and a free download at the ObMimic website.