Kafka Error handling : Processor.output().send(message, kafkaTimeoutInMS) always returns true and its async - error-handling

Question

May be this issue is already reported and resolved .I didn't find the solution and any open issues which talk about this, so creating new one.
I am trying to handle error while publishing data to kafka topic.
With kafka spring steam we are pushing to kafka by using this
if (processor.output().send(messsage , kafkaTimeoutInMS) && acknowledgment != null)
{
LOGGER.debug("Acknowledgment provided");
LOGGER.info("Sending to Kafka successful");
acknowledgment.acknowledge();
}
else
{
LOGGER.error("Sending to Kafka failed", message);
}
Send() method always returns true, I tried stopping kafka manual while running in debug mode, but still it returns true. I have that read it is asynchronous.
I Tried setting
bindings: output: producer: sync: true
This didnt help.
But I see some error which I cant use in my logic to decide whether there is failure or success.
We are manually acknowledging hence we are only supposed to acknowledge when its sent to topic successfully and we need to log all failed messages.
Any suggestions?

Answer 1

I believe you've misinterpreted on how spring-cloud-stream works.
As a framework there is certain contract between the user and the framework and when it comes to messaging the acks, retries, DLQ and many more aspects are handled automatically to ensure the user doesn't have to be exposed to this manually (as you are trying to do).
Consider spending a little time and going through the user guide - https://docs.spring.io/spring-cloud-stream/docs/Fishtown.M3/reference/htmlsingle/
Also, here is the very basic example that will demonstrates a typical interaction of user(developer) with the framework. As you can see, all you're doing is implementing a simple handler which receives and returns a piece of data. The rest (the actual receive from Kafka and send to Kafka or any other messaging system) is handled by the framework provided binders.
#SpringBootApplication
#EnableBinding(Processor.class)
public class ProcessorApplication {
public static void main(String[] args) {
SpringApplication.run(ProcessorApplication.class);
}
#StreamListener(Processor.INPUT)
#SendTo(Processor.OUTPUT)
public String echo(String message) {
return message;
}
}