I am trying to configure SSL in Apache Flume JMS1.6.0. As part of Apache Flume JMS Source does not support SSL by nature.
Is Anyone implemented by enabling SSL in Apache Flume JMS1.6.0.
Is there any option in writing custom code to enable SSL for JMS Source
Currently, Flume JMS Source doesn't have this feature. Flume Avro source or HTTP source has this.
https://github.com/apache/flume/blob/trunk/flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
You can refer the above class and write a custom JMS Source.
Related
Would it be possible to create a server that publishes a client's request to Kafka's topic, and the Consumer that is subscribed to the topic will respond directly to the client?
client -> server -> kafka -> consumer -> client
I guess you are looking for an HTTP bridge, able to bridge the Apache Kafka protocol over HTTP.
I would suggest to take a look at the Strimzi HTTP - Kafka bridge here: https://github.com/strimzi/strimzi-kafka-bridge
It's open source and Apache 2.0 licensed. It's not just when you have Kafka running on Kubernetes via the Strimzi project but even if you have a bare metal Kafka installation. Details on the official documentation: https://strimzi.io/docs/bridge/latest/
Another one is the Confluent REST proxy here: https://github.com/confluentinc/kafka-rest
Still open source but with a custom Confluent Community License. Official documentation https://docs.confluent.io/platform/current/kafka-rest/index.html
I have a Kafka cluster that is running on K8S. I am using the confluent kafka image as and I have an EXTERNAL listeners that is working.
How can I add SSL encryption? Should I use an ingress? Where can I find good documentation?
Thank you
You have a manual way in this gist, which does not use the confluent image.
But for Confluent and its Helm chart (see "Confluent Operator: Getting Started with Apache Kafka and Kubernetes" from Rohit Bakhshi), you can follow:
"Encryption, authentication and external access for Confluent Kafka on Kubernetes" from Ryan Morris
Out of the box, the helm chart doesn’t support SSL configurations for encryption and authentication, or exposing the platform for access from outside the Kubernetes cluster.
To implement these requirements, there are a few modifications to the installation needed.
In summary, they are:
Generate some private keys/certificates for brokers and clients
Create Kubernetes Secrets to provide them within your cluster
Update the broker StatefulSet with your Secrets and SSL configuration
Expose each broker pod via an external service
I recommend using Strimzi kafka operator to deploy Kafka to Kubernetes. I'm using it in production for a year now.
It supports SSL, external load balancers, kafka exporter, etc
Strimzi Kafka Operator
I have a task to use Apache Flume to send messages to a Kafka topic. The caveat is that I have to specify a partition based on an IP address that will be in the message.
Is there a way to configure Apache Flume to do this dynamically, or do I have to implement a custom Producer plugin?
Thank you.
After some research, the only real custom way of configuring the producer is to roll your own.
I implemented the AbstractSink interface and put in my own hash algorithm to generate a partition to send the message to.
How to access the data packets routed to apache webserver in Mule ESB flow.
After I call the exposed webservice, the webservice sends two responses,
To mule server
Routes response XML as data packets to apache webserver.
Can someone help in how to proceed with second flow? Thanks.
Use a Java library that understands PCAP and create your own Mule connector around it.
Reference:
List of Java PCAP libraries: https://en.wikipedia.org/wiki/Pcap#Wrapper_libraries_for_libpcap.2FWinPcap
Mule connector development kit: https://developer.mulesoft.com/docs/display/current/Anypoint+Connector+DevKit
Does any open source ESB support RabbitMQ out of the box?
It seems that apache camel based ESB works with Apache Qpid non RabbitMQ.
As of version 2.12 Camel now has a rabbitmq component.
WSO2 ESB has support for RabbitMQ based AMQP Transport. This documentation explains on how to configure this RabbitMQ based AMQP transport. Some samples also included in that document itself to try out.