i am using activemq PooledConnectionFactory to create connection. I am creating threads and each thread would have its own connection, session and producer.
I have two queries:
1. Do i need to close connection,session, producer myself in code or pooledConnectionFactory would do it once the message sending is successful by producer.
2. creating connection for every thread (eventually for each message) would be a performance bottleneck. Is it possible to have only one connection with many sessions in it (or there should be one-to-one mapping between session and connection, I think I read this somewhere on activemq website)
Any help would be appreciated.
You need to use the code just as you would any other JMS Connection, Session, and Producer. There's not magic to detect when your thread is done with it, you need to close it which will return it to the pool. You can use only one Connection and take many sessions from it, but you need to close them so that they go back to the pool to be handed out to others on demand.
Related
Currently I am using Aerospike in my application.
I faced lots of timeout issues as shown below when I was creating new java client for each transaction and I was not closing it so number of connection ramp up dramatically.
Aerospike Error: (9) Client timeout: timeout=1000 iterations=1 failedNodes=0 failedConns=0
so to resolve this timeout issue,I didn't made any changes to client, read and write policy, I just created only one client, stored it's instance in some variable and used this same client for all transaction (get or put requests).
now I want to understand how moving from multiple client to one client resolved my timeout issue.
how these connection were not closing automatically.
The AerospikeClient constructor requests peers, partition maps and racks for all nodes in the cluster and initializes connection pools and async eventloops. This is an expensive process that is only meant to be performed once per cluster at application startup. AerospikeClient is thread-safe, so instances can be shared between threads.
If AerospikeClient close() is not called, connections residing in the pools (at least one connection pool per node) will not be closed. There are no finalize() methods in AerospikeClient.
The first transaction(s) usually need to create new connections. This adds to the latency and can cause timeouts.
The client does more than just the application's transactions. It also monitors the cluster for changes so that it can maintain one hop per transaction. Also, I believe when we initialize the client, we create an initial pool of sockets.
It is expected that most apps would only need one global client.
My app has multiple threads that publish messages to a single RabbitMQ cluster.
Reading the rabbit docs: i read the following:
For applications that use multiple threads/processes for processing, it is very common to open a new channel per thread/process and not share channels between them.
And I understand that instead of opening multiple connection (expensive)
it is better to open multiple channels.
But why not use a single channel to all threads?
What are the benefits of using multiple channels over a single channel?
AMQP has the concept of Channel to provide more flexibility over reliable TCP connections. Opening a TCP connection per message would be extremely expensive, so they came up with the idea of logical Channels within a connection.
It is not a good idea to use a Channel for all the threads because if anything fails in a particular thread and the Channel dies, the rest of the threads will throw the exception AlreadyClosedException. A channel can die for multiple reasons: for example for trying to declare something that is already declared with other parameters or trying to cancel a consumer which doesn't exist, publishing to an exchange that doesn't exist, etc...
My best advice would be to have an object that holds a Channel in a local variable and also implements ShutdownListener interface, so every time the channel fails, it is able to recover and create a new one from a connection. So I would say that the main benefit is failure tolerance and scalability, since if a Channel dies it won't affect the rest.
my existing code which uses BlockingQueue creates a list of BlockingQueue (like private List> queues;) in which I can put messages to process.
However due to persistence issue we plan to shift to activemq.
Can anyone please help me if we can get a list of activemq queue (in java program not from configuration file). I know that I can use createQueue on session to create a single instance of the queue but I want list of queue like done for BlockingQueue.
Any help would be much appreciated.
You can get a list of the available queues using DestinationSource from your connection.
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");
ActiveMQConnection connection = (ActiveMQConnection)connectionFactory.createConnection();
DestinationSource ds = connection.getDestinationSource();
Set<ActiveMQQueue> queues = ds.getQueues();
edit:
to create a queue take a look at ActiveMQ Hello world sample link What the code does there is creating a connection to an activeMQ-broker embedded in the jvm
// Create a ConnectionFactory
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost");
// Create a Connection
Connection connection = connectionFactory.createConnection();
connection.start();
// Create a Session
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Create the destination (Topic or Queue)
Destination destination = session.createQueue("TEST.FOO");
The thing that might not be obvious with above code is that the line:
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost");
will not only setup a connection to a broker, but embedded a broker inside the connection if there isn't one already. Explained at the bottom of this page
That feature can be turned of using (you need a broker, but if you want to set it up in other way):
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory("vm://localhost?create=false");
I really like ActiveMQ, but it offers a lot more than persistence, so things might seem a little overly complex when doing the simple things. But hope that will not scare you.
To create a list of queues, you have to create that list, then create each queue individually from the session object.
Queue q = session.createQueue("someQueueName")
This, however, does not really "create" a queue in that sense, since a queue is a persistent "thing" in the ActiveMQ process/server. This will only create a reference to an ActiveMQ queue given an identifier/name.
I'm not sure why you need ten queues right up. Typically, you have one queue per event type or use case (or similar), then use concurrent consumers to process in parallel.
But of course, you can always do somethings similar by a simple for loop, creating one queue at a time and attaching them to an arraylist. Note that you cannot get type safe queues with only Event objects in them.
You can send ObjectMessages with events though. Just create one:
Event event = createEvent(..); // Given "Event" is serializable (need to be able to persist it).
Message m = session.createObjectMessage(event);
// Send message as usual in ActiveMQ.
It might be that you need to rethink one or a few things in your code when converting from BlockingQueues to persistent ActiveMQ queues.
I have a server that at the minute that creates a new thread for each client connecting securely. If I use a thread pool this will mean that I will have a finite number of clients at once. However this means that I can not be listening on ports for all clients.
My idea is to have the client send a UDP packet with some ID linked to there connection so that they can re-establish the connect rather than lock up a thread for 10-60 seconds (server will keep the SSLsockets in memory). Is that a good way to solve the problem? - I don't see any security security vulnerabilities.
The server is java and the client is C++ not that effects the question.
Your question doesn't make sense. If the client wants to reconnect it should just open a new socket. You are positing at least one extra thread to listen to the UDP port and then ... what? It still has to use the thread pool to handle that client, if that is your self-imposed constraint, or else start a new thread, in which case you may as well not have had the thread pool constraint in the first place.
However this means I cannot be listening on ports for all clients.
No it doesn't. It just means that some clients will get delayed service while the thread pool is full, and a very few clients will get connection failure while the backlog queue is full. It doesn't impair your ability to listen for clients at all.
What if the only port you have say TCP/443 (HTTPS)? What if UDP is firewalled (very much possible)? In other words, you should NOT introduce UDP into this picture.
Even in thread-pool scenario, you can still know the difference between multiple clients who connected to the same server port.
Typical solution for this is to create set of sockets you are going to be watching for at once (in one thread) - in C/C++ it is typically done using select()/poll()/epoll(), and in Java you can use java.nio.
This way, if any client(s) have something to say to you as a server, your select loop will instantly notice that, serve these clients and go back to select(), which consumes very little (effectively 0) CPU usage.
This is an example how to do select loop in C and similar example in Java.
What is the scope of the following ActiveMQ objects in an application.
Connection (is it one per application?)
Session (one per application?)
Producers
Consumers
Am designing an application that will send a lot of messages and I need to know if I can use one of connection, session, producer, consumer in the application.
these depend on various settings, the concepts are summarized well on this page...
http://web.archive.org/web/20120704235809/http://fusesource.com/wiki/display/ProdInfo/Understanding+the+Threads+Allocated+in+ActiveMQ