I'm very new to Apache ActiveMQ and still in learning phase so please bear with my silly or with not very precise questions. I would like to know is there is any possibility of message lost if we keep enablebatch property to true?
As per my knowledge enabling batch processing is always helpful and enhance your performance too in terms of processing transaction but I would like to know negative side of this property which may lead to message lost issue. Also if you can let me know whether there are some additional properties too which may need to set if we enable batch as true ?
Will appreciate your response
Regards,
Deepak Hardasani
Don't try to pre-tune! It only leads to de-tuning. Most Enterprise message flows are sufficiently handled with defaults on modern server systems without jumping through a lot of hoops and adding complexity.
Build up a representative test case.. run it. If the numbers meet your requirements, move on to more useful tasks. If not, then worry about tuning.
Related
Not that I can find any by googling, but ... does anyone know of any open source code/development frameworks/test software/etc for the Multidrop Bus commonly used in vending machines?
In my opinion there isn't a free framework for the MDB, as this bus is only used by profit oriented companies and nobody would make his own code open source (me too).
But the MDB protocol itself isn't very complex, it's the error handling for the several devices that is a bit complicated, as it should be 100% safe.
And today it can be tricky to implement the 9bit serial layer, as this isn't standard, even many MCUs didn't support it any more.
Edit: How I would implement it today
Regard all specification, especially the timings/timeout (ex. NAK-Timeout of 5ms).
I would use state machines to collect the configuration data, setting the normal mode of operation, set settings and all other things.
In the first step(not later) plan to build at any state an error handling, what should happen if the communication get lost, or you got an unexpected answer?
I would also implement logging much as possible, as sometimes there will money get lost and you have to explain why.
this maybe will be off topic, but I am preparing for an exam in real time. And I have been browsing the book and Internet for an answer for a problem.
Basically I wonder if by adding additional test code if it may change the real time behavior for an embedded system, and or also if it will introduce new errors.
Anyone who might know the answer for this, or refer me to some reading material for it?
Your question is too general.. So I guess the default answer would be it depends.. But considering the possibilities as an exercise of logic and thought, yes it surely can!
There are many schemes available to guarantee the 'real-timeness' of an embedded system. For example, one can have a pre-emptive timer based ISR to service the real-time task.. In such a case, your test code could possibly not affect the 'real-timeness'.. But if the testing takes too long, and the context switches are not pre-emptive, you could get into trouble..
But again it depends on what you're testing and how you're testing. Your test code can possible mess with the timers, interrupts or the memory of system. The possibilities to mess up stuff if you're not careful are endless..
Having an OS underneath will prevent some errors, but again depending on how it works, you may be saved from bad 'test code'..
Yes, when you add code (test, diagnostic, statistic) it may change the real time behavior. It depends on the design, the implementation and the CPU power if it will actually change the behavior. You also have more lines of code and the probability for errors may increase. But I wouldn't say, "it will introduce errors", since it can introduce errors.
Yes it can. See How can adding data to a segment in flash memory screw up a program's timing? for an example of how even adding non-executable code can adjust timing enough to screw up a system.
Yea, changing your code base could totally change its timing. Consider if you dumped some debug output to a serial port, it takes time to call that function, format the data, and if the function is synchronous, then for it to wait for data to go out. This kinda stuff definitely changes system timing behavior.
Looking for experience with RabbitMQ, especially in HA configuration using Pacemaker and DRDB as recommended here: http://www.rabbitmq.com/pacemaker.html
The DRBD part in particular makes me nervous, so I'm hoping someone here has real-world experience to share.
Works most of the time. However you'll have to pay special attention to fencing (split brain), when dealing with DRBD. On a production system it's always a pain to have to fix this kind of issues manually.
We failed to run RabbitMQ in a master/slave (multi-state RA). We thought we'd enhance availability. We're back to a single instance now. If anyone else has experience with several RabbitMQ instances running concurrently and backing a master entity that would be great to share!
I find the lack of tools to debug Pacemaker when there are issues is a big hurdle to deploy to live systems... It's not always clear what Pacemaker is "thinking" or doing. hb_report is not sufficient unfortunately.
Hope this helps,
D.
We tried master/slave configuration as well, however it became difficult to maintain all instances up to date with no downtime. And trust me, you want to update RabbitMQ. There are always bugs popping up either in RabbitMQ itself or in Erlang.
We've are getting about 100 crashes per year without any meaningful explanation in the logs. The error log just has generic "error while starting" in it and that's pretty much it. Sometimes it won't start after the crash and most of those times, the only solution is to delete all the persistent messages from all instances, so that the queue state is synchronized across the cluster. Other times it would crash immediately after launching and only after multiple repeated attempts will it properly load. Meaning there is no added reliability what so ever when using master/slave. At least there was none in our case. (RabbitMQ 3.5.3, Erlang 18.0)
It works for production, but only if you keep a copy of the message somewhere in the logs or in the database, from where it can be quickly recovered after a major crash.
I have a web site I am building for a client. I now have a tester on the project with me.
I feel testers are needed. REALLY! I cannot test my own code. I also appreciate the value of a new set of eyes. But what desires reporting?
It is easy to say everything should be reported, but I don't have someone between me and the tester to filter out the unimportant requests. The tester does not know the system nor the target user well. She is assigning me tasks and not the project manager. I think this will change soon, but until it does, what do you recommend? There seems to be a believe that our users have NEVER used the interent before at all, and they are as dumb as rocks.
The problem I am having is that EVERYTHING the tester suggests is being accepted automattically and assigned to me.
I have many cases that make me drop my jaw and say "Really? Are you serious? This deserves to be a issue?"
Ex: Need to add text at top of page that says "* = Required" for required fields.
Have you ever felt this way? How did you deal with it?
For now, I am just doing as I am told, but I am making it clear I do not agree.
It sounds to me like your tester is doing the right thing. You can't assume any level of user expertise when testing an application. If a user can break something, they will.
You and your tester need to work out a severity scale. The outliers (those that anybody with Internet experience could probably work around/would never hit) would be considered low priority and sit on the back burner until you knock out the high priority items.
...never the less, those outliers should still be logged because they can definitely come back to bite you in the ass in the end.
You need to add priorities for your issues. This will allow you to do the important issues first, and cosmetic issues last. Here is example priorities from Jira:
Priority 1 - a reproducible crash; issue blocking any further testing or development of a specific feature; loss of user's persistent data; huge memory leak
Priority 2 - a major issue that must be fixed before the product is released; prevents users from using a feature; negatively affects partner; significant memory leak in frequently used functionality
Priority 3 - a minor issue that should be fixed before a product is released; does not prevent users from using a product; highly visible usability issue; small memory leak in rarely used functionality
Priority 4 - a purely cosmetic issue; doesn't affect functionality
Actually it sounds like your tester is doing the right thing (and the text for "* = required" is a very good idea).
In addition to the suggestions about prioritizing reports, I would suggest that you categorize the reports as to whether they refer to user experience or functionality.
You and the tester will never exactly agree on what "needs" to be reported. Just set the priority on issues correctly, and get on with fixing the high-priority stuff first.
One thing you absolutely do not want to do is to discourage the tester from filing bugs. That'll come back to bite you when something ships totally broken, and they say "I thought that was just how it worked".
Do make sure that you're communicating the development schedule and status properly, so they don't waste time testing features that aren't sufficiently complete.
I would report to the client what each change will cost in terms of time and money. Things that are legitimate bugs you'll probably need to fix on your own time (unless your contract says otherwise). Things that are design / subjective issues you should be able to assign a cost to. Let the client know what it is going to cost them and they can decide if they want to proceed or not.
Hopefully you've got some sort of a project specification that the client has signed off on so that you know when the project is complete and what sort of things are not included in the project scope. If not, you might have a bit of a fight on your hands. For changes that you think are outside of the project scope, you might need to compromise - maybe bill them at a cheaper rate or split the cost with them. If you're in that situation it's a good learning experience to get everything documented in the project specification so that there is no question about what falls outside of the project scope. I've been there - one experience like this is enough to teach you to put more work into your specifications.
Report everything and triage. After a bit of time, she'll start to understand what gets past triage and what doesn't. Humans can learn; teach.
as opposed to writing your own library.
We're working on a project here that will be a self-dividing server pool, if one section grows too heavy, the manager would divide it and put it on another machine as a separate process. It would also alert all connected clients this affects to connect to the new server.
I am curious about using ZeroMQ for inter-server and inter-process communication. My partner would prefer to roll his own. I'm looking to the community to answer this question.
I'm a fairly novice programmer myself and just learned about messaging queues. As i've googled and read, it seems everyone is using messaging queues for all sorts of things, but why? What makes them better than writing your own library? Why are they so common and why are there so many?
what makes them better than writing your own library?
When rolling out the first version of your app, probably nothing: your needs are well defined and you will develop a messaging system that will fit your needs: small feature list, small source code etc.
Those tools are very useful after the first release, when you actually have to extend your application and add more features to it.
Let me give you a few use cases:
your app will have to talk to a big endian machine (sparc/powerpc) from a little endian machine (x86, intel/amd). Your messaging system had some endian ordering assumption: go and fix it
you designed your app so it is not a binary protocol/messaging system and now it is very slow because you spend most of your time parsing it (the number of messages increased and parsing became a bottleneck): adapt it so it can transport binary/fixed encoding
at the beginning you had 3 machine inside a lan, no noticeable delays everything gets to every machine. your client/boss/pointy-haired-devil-boss shows up and tell you that you will install the app on WAN you do not manage - and then you start having connection failures, bad latency etc. you need to store message and retry sending them later on: go back to the code and plug this stuff in (and enjoy)
messages sent need to have replies, but not all of them: you send some parameters in and expect a spreadsheet as a result instead of just sending and acknowledges, go back to code and plug this stuff in (and enjoy.)
some messages are critical and there reception/sending needs proper backup/persistence/. Why you ask ? auditing purposes
And many other use cases that I forgot ...
You can implement it yourself, but do not spend much time doing so: you will probably replace it later on anyway.
That's very much like asking: why use a database when you can write your own?
The answer is that using a tool that has been around for a while and is well understood in lots of different use cases, pays off more and more over time and as your requirements evolve. This is especially true if more than one developer is involved in a project. Do you want to become support staff for a queueing system if you change to a new project? Using a tool prevents that from happening. It becomes someone else's problem.
Case in point: persistence. Writing a tool to store one message on disk is easy. Writing a persistor that scales and performs well and stably, in many different use cases, and is manageable, and cheap to support, is hard. If you want to see someone complaining about how hard it is then look at this: http://www.lshift.net/blog/2009/12/07/rabbitmq-at-the-skills-matter-functional-programming-exchange
Anyway, I hope this helps. By all means write your own tool. Many many people have done so. Whatever solves your problem, is good.
I'm considering using ZeroMQ myself - hence I stumbled across this question.
Let's assume for the moment that you have the ability to implement a message queuing system that meets all of your requirements. Why would you adopt ZeroMQ (or other third party library) over the roll-your-own approach? Simple - cost.
Let's assume for a moment that ZeroMQ already meets all of your requirements. All that needs to be done is integrating it into your build, read some doco and then start using it. That's got to be far less effort than rolling your own. Plus, the maintenance burden has been shifted to another company. Since ZeroMQ is free, it's like you've just grown your development team to include (part of) the ZeroMQ team.
If you ran a Software Development business, then I think that you would balance the cost/risk of using third party libraries against rolling your own, and in this case, using ZeroMQ would win hands down.
Perhaps you (or rather, your partner) suffer, as so many developers do, from the "Not Invented Here" syndrome? If so, adjust your attitude and reassess the use of ZeroMQ. Personally, I much prefer the benefits of Proudly Found Elsewhere attitude. I'm hoping I can proud of finding ZeroMQ... time will tell.
EDIT: I came across this video from the ZeroMQ developers that talks about why you should use ZeroMQ.
what makes them better than writing your own library?
Message queuing systems are transactional, which is conceptually easy to use as a client, but hard to get right as an implementor, especially considering persistent queues. You might think you can get away with writing a quick messaging library, but without transactions and persistence, you'd not have the full benefits of a messaging system.
Persistence in this context means that the messaging middleware keeps unhandled messages in permanent storage (on disk) in case the server goes down; after a restart, the messages can be handled and no retransmit is necessary (the sender does not even know there was a problem). Transactional means that you can read messages from different queues and write messages to different queues in a transactional manner, meaning that either all reads and writes succeed or (if one or more fail) none succeeds. This is not really much different from the transactionality known from interfacing with databases and has the same benefits (it simplifies error handling; without transactions, you would have to assure that each individual read/write succeeds, and if one or more fail, you have to roll back those changes that did succeed).
Before writing your own library, read the 0MQ Guide here: http://zguide.zeromq.org/page:all
Chances are that you will either decide to install RabbitMQ, or else you will make your library on top of ZeroMQ since they have already done all the hard parts.
If you have a little time give it a try and roll out your own implemntation! The learnings of this excercise will convince you about the wisdom of using an already tested library.