In ActiveMQ while using blob messages we use this as broker
String broker1 = "tcp://localhost:7005?jms.blobTransferPolicy.UploadUrl=http://localhost:7005/fileserver/"
Can anybody explain what is UploadUrl and why we need to configure for blob messages(we don't need to configure for text messages). Why it doesn't allow tcp protocol?
So plain text messages are good and easy to use, but needs to be in memory at all times. It works well with a KBs of data, or even a few MB. However, sending very large files, such as initial data loads, large media files or BI data, is not nice to keep around in memory. There may still be a need to pass the message around, route/filter based on message properties, use transactions and similar.
Blob messages is an attempt to solve the need to pass around GBs of data through the semantics of messaging. The trade off is that you have to define a streaming based server somewhere that both sender and receiver can reach. It can be HTTP, FTP, a local file, WebDAV or similar. ActiveMQ comes with a HTTP based fileserver if you have no other file area around.
Related
Is it a good idea to send files with size about 1Mb through RabbitMQ? I want to send message in json format with binary fields corresponding to the files.
And how to do it properly using spring-amqp? Just by publishing object with next class?
class Message {
String field1;
byte[] fileField1;
byte[] fileField2;
}
I would suggest not only reading those links that were posted but also, doing some of your own experimentation. The thing I would be concerned about is performance at the service level and at the client level.
You might want to consider having a server host the files/data and allow rabbitmq just send the message to the consumer with the id of the message in it. So when your consumer gets the message, it sends an HTTP GET request to a service that requests the actual message payload. That way RabbitMQ stays lightweight. You can always add consumers and servers if you need.
That's my opinion without experimenting. You might find that it's still lighting fast with 1MB payloads. That's why I would say to experiment and find out for yourself.
Hope you find this helpful!
I am new to the formum and new to MSMQ.
I have been asked to do research on it, to see if it will help out business, but still not sure if and how it works. Here is a short Summary.
We have a service provider that will receive messages via a mobile phone, to which they will pass certain info(Such as the cell number, text in message, etc.) to a URL that we have given them, which is an app we created, which will then process the data and store into our Database etc etc.
However, as we at times receive between a few hundred to a few thousand at any given time(Spread out or at once) - We get timeouts.
What I would like to know, is it possible to get this info stored into a que using MSMQ, before it hits our URL(That we provided to our service provider), so that we can avoid timeouts ?
I hope this makes sense and that someone can help!
Thank you!
MSMQ is a transport protocol. It is designed to get data from A to B as fast and as reliably as possible. As it uses store-and-forward, it can be used as a data buffer at the sender. In your case, though, that's not important. I would recommend:
ISP sends MSMQ message over HTTP(S) to your server.
Message is stored on server's local queue.
Your app reads messages from queue.
Your app writes to database.
Goto 3
So you would be buffering the messages AFTER they reach your server.
The messages would also be buffered at the ISP in case of network outage between them and you.
I am making a chat app. There is a pic of yourself. You can send messages to other people and when they receive the message they receive your avatar too.
The trick is that images cost resources and they don't change that often. Is RabbitMQ suited for exchanging avatars ? Or do I need to design my own system ?
P.S : I'm new to RabbitMQ but Im experienced with writing apps, servers, ...
In brief:
Yes, technically, it may be used. But it would be better to send link to image that served separately, say, from CDN or just from your web-server.
Long answer:
RabbitMQ can handle large payload (I tested 2GB and it works like a charm) but it doesn't designed for that. RabbitMQ works best when queue fits in memory. Even if messages get dumped to disk (I guess this is your case if you want your messages be permanent and available after RabbitMQ restart, also don't forget to make queues also permanent) it always will be faster to keep message as small as you can (without any side performance degradation). Also, most apps (web or platform-specific) use caching, so downloading avatar image once and then just display it whenever you need would be definitely faster and resource-friendly way rather then send that binary every time.
What's the best way to store custom messages into the queue? I mean if I have a queue that can store different types of messages should I store them in binary format or json?
What do you think?
Windows Azure Storage Client Library provides overloads for binary and string that handle encoding for you. As such you can make use of any serialization mechanism you like, given that the serialized form is less than 64 KB.
Hence, the answer to your question actually depends on your specific scenario. Handling JSON data would be much easier, but if you have a specific need to send the data in another format, please consider such alternatives. For larger scenarios some users augment queue messages to simply point to blob or table storage as a more flexible and verbose option while using the queue messages to provide for reliable message delivery.
I'm using the web console against my AMQ 5.2 instance successfully, except for I cannot see the content of all of my messages.
If I send a test message using the web console, I can see the sample text content, but I believe the vendor app I am working with has binary or byte array message content.
Is there something I need to do to be able to view this raw data?
Thanks,
To my knowledge, it is not possible to inspect messages in the Admin Console. You can get some statistics (like how many messages have been sent etc.).
ActiveMQ does not unmarshal messages when receiving them (for performance reasons, unmarshalling is rather expensive).
Thus, if you want to have some way to inspect messages for their content, you can basically do 2 things:
Write a consumer which registers for all topics/queues, through which you can see messages' content. Drawback: if you're using queue-based interaction, your "real" consumers will not get all messages
Write an activeMQ plugin which looks at the messages. Have a look at ActiveMQ's Logger Plugin. Then write your own (you'll need the sources to compile it) and load it with ActiveMQ (see the documentation on how to configure ActiveMQ to load plugins). You want to override the send() method which is called whenever someone sends a message to the broker. There you get a reference to the message and can access its content.
Neither of the two messages provides a convenient viewing-mechanism though. You'll have to resort to standard out, or write your own web-based access.
hawtio now shows first 256 chars of messages. Don't know if that is enough for you. Use browse() method.