I want to transfer an image from a client to the server using RabbitMQ. However, from whatever I read, I understood that RabbitMQ can only transfer a stream of text. So how to transfer an image?
The amqp body is a buffer, you can send what you want.
In general, if you want to send a file, you have to read it and send the buffers.
You should send the file using more publish and not just one, then recreate the file on the consumer side.
You should avoid sending big buffer.
Related
In our team, we exchange messages via RabbitMQ between two systems. The messages are encoded in protobuf (v3). We use NServiceBus on the sending and receiving side. We use the RabbitMQ management UI for monitoring the error queue. In production, we noticed that it is not easy to make sense of the payload of the messages in the error queue, which are base64-encoded.
What is the easiest way to get human-readable access to the messages in the error queue? We have full control over the decisions in both systems, and also discussed a switch to JSON-encoded messages (instead of protobuf). But we are otherwise happy with our protobuf-based implementation. Which is already implemented after all.
Protobuf v3 supports formatting as json, once you have the data parsed as IMessage (the base type for in-memory protobuf objects).
So you can convert a single message to be human readable as follows:
Use the webUI GetMessage function to get the message as base64 then requeue it
Convert the message back to protobuf binary via Convert.FromBase64String
Parse it back to an IMessage via ProtoMessageTypeGoesHere.Parser.ParseFrom(binaryData)
You can then convert the parsed message to Json via ToString() or Google.Protobuf.JsonFormatter.
As long as your error queue isn't going to be disrupted by the re-queuing (e.g. resetting of timestamps or reprocessing), you should be able to do this for all messages in the queue.
I wouldn't recommend using the management UI for this. A simple script or html page with a stomp client would be a lot easier to use and more error-proof, in my opinion.
However, to answer your question: to simply decode the message and replace the text, a simple javascript solution will work fine.
$(".msg-payload").text(atob($(".msg-payload").text()))
This will simply select the message field on the queue page on the RabbitMQ management UI and replace it with the decoded value (that's the function atob).
To use this, you can either run it from the console or add it as a bookmark in your browser. Simply use the code prefixed with javascript:, like so:
javascript:$(".msg-payload").text(atob($(".msg-payload").text()))
In ActiveMQ while using blob messages we use this as broker
String broker1 = "tcp://localhost:7005?jms.blobTransferPolicy.UploadUrl=http://localhost:7005/fileserver/"
Can anybody explain what is UploadUrl and why we need to configure for blob messages(we don't need to configure for text messages). Why it doesn't allow tcp protocol?
So plain text messages are good and easy to use, but needs to be in memory at all times. It works well with a KBs of data, or even a few MB. However, sending very large files, such as initial data loads, large media files or BI data, is not nice to keep around in memory. There may still be a need to pass the message around, route/filter based on message properties, use transactions and similar.
Blob messages is an attempt to solve the need to pass around GBs of data through the semantics of messaging. The trade off is that you have to define a streaming based server somewhere that both sender and receiver can reach. It can be HTTP, FTP, a local file, WebDAV or similar. ActiveMQ comes with a HTTP based fileserver if you have no other file area around.
Is it a good idea to send files with size about 1Mb through RabbitMQ? I want to send message in json format with binary fields corresponding to the files.
And how to do it properly using spring-amqp? Just by publishing object with next class?
class Message {
String field1;
byte[] fileField1;
byte[] fileField2;
}
I would suggest not only reading those links that were posted but also, doing some of your own experimentation. The thing I would be concerned about is performance at the service level and at the client level.
You might want to consider having a server host the files/data and allow rabbitmq just send the message to the consumer with the id of the message in it. So when your consumer gets the message, it sends an HTTP GET request to a service that requests the actual message payload. That way RabbitMQ stays lightweight. You can always add consumers and servers if you need.
That's my opinion without experimenting. You might find that it's still lighting fast with 1MB payloads. That's why I would say to experiment and find out for yourself.
Hope you find this helpful!
I'm able to queue messages as strings on my queue via the RabbitMQ Manager. My problem is that my app will be using Protocol Buffers and I can't figure out how to queue the raw bytes of a protocol buffer. I'm able to en-queue and de-queue protocol buffer messages through my app but I want to be able to queue messages through the RabbitMQ Manager and de-queue via my app.
I found this tool called Protocol Buffers Workbench (http://blog.functionalfun.net/2010/10/getting-started-with-protocol-buffers.html) which will convert my message to raw bytes. I can't however figure out how to take the raw bytes and en-queue them through the RabbitMQ Manager.
Is there an easy way to do this?
I couldn't find an easy way to do this so I wrote my own program to do it for me:
https://github.com/thereallisa/proto_enqueue
I need to be able to send files like pdf or word via RabbitMQ (I think answer might be more generic meaning not specific to RabbitMQ but any broker I am however using RabbitMQ).
Is there are way to send them as mimes or do I need to convert to base64 and then send that.
Or am I going on a tangent here :( please help.
RabbitMQ doesn't look into the body of the message, you can send anything, any binary data.