I am new to Kafka and We are using Spring Web Flux in the application. We have a requirement to push two different messages to two different Topics say T1 and T2. Kafka Broker is the same.
We are using ReactiveKafkaProducerTemplate and it working fine.
#Bean
public ReactiveKafkaProducerTemplate<String, Object> reactiveKafkaProducerTemplate(
KafkaProperties properties) {
final Map<String, Object> props = properties.buildProducerProperties();
return new ReactiveKafkaProducerTemplate<String, Object>(SenderOptions.create(props));
}
Now we have requirement to compress only one Topic[T1] content alone as the message size is more on the Topic T1.
Do we have something like RoutingKafkaTemplate support in Reactive Kafka or Project Reactor where we can modify the Producer Config as per Topic needs?
No; there is no equivalent; you need to configure two templates with different producer configs.
Related
I have multiple httpClient instances in one app. I want to collect metrics by each httpClient with specific label or name. Is it possible with vert.x? I saw in documentation block of collecting httpClient metrics and I want to enrich each metric with specification for client.
You can provide a metrics name when creating the client:
HttpClientOptions options = new HttpClientOptions().setMetricsName("foo");
Actually, I have on premises spring boot application which consumes rabbitMQ messages using #RabbitListener and I have migrated the same application to azure which consumes AzureServiceBus messages using #JMSListener.
We are maintaining same code for both on premises and Azure . So, because of these two listeners, I'm planning to replicate the same consumer code in two different classes with same content with two different Listeners
consumer with JMSListener:
#JmsListener(destination = "${queue}", concurrency = "${threads}", containerFactory = "defaultContainer")
public Message processMessage(#Payload final String message) {
//do stuff with same content
}
consumer with RabbitListener:
#RabbitListener(queues = "${app.rabbitmq.queue}")
public Message processMessage(#Payload final String message) {
//do stuff with same content
}
Is there any possibility of avoiding the duplicate code in two classes ? How can we handle listeners on a fly with only one consumer? Can any one please suggest me out ?
You can add both annotations to the same method with the autoStartup property set according to which Spring profile is active.
For #RabbitListener there is an autoStartup property on the annotation itself but, in both cases, there are Spring Boot properties auto-startup to control whether the container starts or not.
I'm using RabbitMQ as message broker. I have a consumer and a producer. So far I have created a payload with few primitive attributes. The problem is, as long as I need more information from the payload in my consumer, I need to change payload so I can handle in the consumer: this task at the end may be heavy.
I was wondering if I could use some kind of a Map as payload or it is not recommended? Like this, I have a generic payload.
On the other hand, we have Serializable DTO in our application. I guess I could create a Payload containing DTO?
I'm trying to figure out the best way to do it :)
if you need extra information you could use message properties:
AMQP.BasicProperties.Builder builder =
new AMQP.BasicProperties().builder();
Map<String,Object> headerMap =
new HashMap<String, Object>();
headerMap.put("mykey1", myvalue1);
headerMap.put("mykey2", myvalue2);
builder.headers(headerMap);
channel.basicPublish("","myqueue",builder.build(),"message".getBytes());
Using headerMap you can add or remove info without modify your message
hope it helps
Is it possible to do Javaconfig i.e annotations in spring instead of xml to create client regions in Spring gemfire?
I need to plug in cache loader and cache writer also to the regions created...how is that possible to do?
I want to perform the client pool configuration as well..How is that possible?
There is a good example of this in the spring.io guides. However, GemFire APIs are factories, wrapped by Spring FactoryBeans in Spring Data Gemfire, so I find XML actually more straightforward for configuring Cache and Regions.
Regarding... "how can I create a client region in a distributed environment?"
In the same way the Spring IO guides demonstrate Regions defined in a peer cache on a GemFire Server, something similar to...
#Bean
public ClientRegionFactoryBean<Long, Customer> clientRegion(ClientCache clientCache) {
return new ClientRegionFactoryBean() {
{
setCache(clientCache);
setName("Customers");
setShortcut(ClientRegionShortcut.CACHING_PROXY); // Or just PROXY if the client is not required to store data, or perhaps another shortcut type.
...
}
}
}
Disclaimer, I did not test this code snippet, so it may need minor tweaking along with additional configuration as necessary by the application.
You of course, will defined a along with a Pool in your Spring config, or use the element abstraction on the client-side.
Is it possible to discover all ActiveMQ queues and topics via JNDI? It was possible to fetch them all with HornetQ using the "list" method. I would like to implement a JMS client for multiple brokers and don't like to pre-configure all queues in jndi.properties.
Properties props = new Properties();
props.setProperty("java.naming.factory.initial","org.apache.activemq.jndi.ActiveMQInitialContextFactory");
props.setProperty("java.naming.provider.url", "tcp://localhost:61616");
Context context = new InitialContext(props);
NamingEnumeration<NameClassPair> names = ctx.list(jndiPrefix);
The activemq initialContext factory implements a simple hashmap that is unrelated to the broker because the broker will create any destination on demand unless authorization prevents that behaviour.
You can use the dynamic contexts - dynamicQueues/FOO.BAR or dynamicTopics/FOO.BAR to access a destination named FOO.BAR without any additional configuration.
see: jndi-support in the doc1 for some more detail.