I am working with ActiveMQ (5.15.2) and have the following question. Is there any way for me to impose a maximum size for messages? Can I set this limitation for a given queue/topic (so topic A has a 1k message limitation and topic B has a 5k message limitation)?
Also, is it possible to set this limitation for a given connection?
I have searched around and cant't find anything out-of-the-box. It seems like the only way would be to define custom plugins.
Is there any out of the box way that I am now aware of? And, if not, can anyone point me to what would be the best way to define such a limitation?
Thanks!
To set the max message size (32k in the example below) for all messages on a specific transport connector, set the wireformat.maxFrameSize value (in bytes)
<transportConnector .. uri="tcp://0.0.0.0:61616?wireformat.maxFrameSize=32768.."
ref: Configure transport wireformat settings
Related
Will the ActiveMQ REST interface support zipped XML messages when posting to topics? I'm not able to test this myself right now.
Generally speaking, ActiveMQ doesn't care about the content of the message. It's all just bytes. Assuming the message doesn't fail for some other indirectly related reason (e.g. it exceed the max frame size) then it should work.
I play with nsqd a little bit and met the prob mentioned in the title. But when I send message via single nsqd, there is no such prob. Does this mean message should always be sent via same nsqd?
Here is my project. https://github.com/hoozecn/nsqd-cluster
It's resolved by set a higher MaxInFlight value.
ref: https://github.com/nsqio/nsq/issues/1213
I am implementing an application using spring websockets, stompjs and sockjs. The messages that I am currently sending are working fine when the message size is less than the configured buffer size limit, however its hard for me to judge what the size would be as it would depend on the user selection and the message is then built on it.
My question here is
1) Who handles the message size limit, is it the message broker, right now I am using the default message broker that comes with spring, does it help if I move to ActiveMQ and do I need to do something on ActiveMQ's configuration to change this message size?
2) I was looking into partial messaging in spring websockets, but I couldnt find any demonstration of it. Can you please let me know?
I'm evaluating using NSQ, http://nsq.io/, for a specific project. The idea is to setup a data pipeline where each step is a job, and where the state ideally will be located in the message body.
Which got me to think about a potential maximal message size. I don't manage to find any documentation on the subject. Can it be any size? I guess it will affect performance if messages are big enough to not fit in memory.
As Aldo mentioned, the default maximum message size is around 1 megabyte.
This is easy enough to change with the max-msg-size switch for the NSQ daemon. Here is an example of how to use it:
nsqd --lookupd-tcp-address=127.0.0.1:4160 -max-msg-size=2097152
This tells the NSQ daemon (nsqd) that max message size should be 2 megabytes instead of the default.
Using this setting, you could indeed have a message that would fill up your memory, as you mentioned in a comment.
as far as I can tell from the documentation, it looks like it's 1MB by default.
I know it sounds like a lot but I managed to hit this limit.
What I'm doing now is to send an NSQ message with the minimum amount of data regarding the event, and get the full data on the other end when I handle it.
There is a built in limitation of 2 MB for the IBM WebSphere MQ JMS interface.
http://www-01.ibm.com/support/docview.wss?uid=swg21221260
Is there a way to bypass that limitation?
The limitation applied to WMQ versions distributed with WAS back at V5.1.1 many years ago. If this is the problem, upgrading to current version of WMQ will resolve it. The current version of WMQ is V7.0.1. V6.0.2 is also still current but will be out of service in September of 2012. V6 & V7 can send and receive messages up to 100MB but WMQ itself defaults to 4MB out of the box. It is necessary to tune parameters of the QMgr, queues and channels if messages larger than 4MB are required but JMS is not a limitation at modern versions.
The WMQ Java/JMS manuals do not specifically mention a maximum size because it is the same as the native WMQ max length of 100MB. However, the WMQ V6 Performance Report provides benchmarks for JMS messages up to 64MB.
Whatever is preventing you from putting a 3MB message isn't a limitation of WMQ's JMS implementation as regards message size. If you have checked MAXMSGL on all of the channels and queues and the QMgr then it's something less obvious but it is configuration.
This might sound arduous, but it is a solution:
Take your message content, convert it into a byte array.
Split the byte array into n sub arrays that are ~< 1.9 MB each.
Start a JMS transaction and send each sub array in a ByteMessage, incrementing the group count:
e.g.
message.setStringProperty("JMSXGroupID", groupId);
message.setIntProperty("JMSXGroupSeq", i);
On the receiver side, you implement a selector to get all the messages in the group as soon as you receive the first message. Retrieve all the messages in the group (hopefully you get them all), sort them correctly, re-create the big byte array, unmarshall it, and you're done.
Trivial really.....
Here's a better example.