I am new to RabbitMq. We are evaluating it for production use with rails app, currently for background mailing system. RabbitMq tutorial presents bunny gem. On it's other gem list, there is also sneakers gem.
It would be useful to know, what are the difference between them, and which one is more useful to create a simple background mailing system ?
Bunny is your RabbitMQ connection wrapper. Send, receive, subscribe, etc.
Sneakers is a framework for a worker-centric approach. It uses Bunny to wrap its connections. You'd use Sneakers if you needed background processing in a Rails application or if you had different worker classes and wanted to standardize your approach.
Related
I have just started installing CKAN.
I was looking in the initial documentation to install CKAN and on the principal website of ckan.org, its impossible for me to find and know why Redis is necessary to run a version 2.7 or higher of CKAN.
For example:
Why do we need Redis for running CKAN?
In this question, it says that its necessary for the new system of asynchronous background jobs that is it used on CKAN. So what type of asynchronous job can do CKAN? and when CKAN can use this type of asynchronous background jobs?
Currently, CKAN core only provides the infrastructure for asynchronous background jobs but doesn't actually create such jobs on its own (this might change in future releases).
There are, however, CKAN extensions that use the system, for example ckanext-extractor.
Disclosure: I'm the author of ckanext-extractor.
I am developing applications and c#, I at the moment, I work on projects for Windows Platform only. However, I am planning to move into using C# mono to make my programs be able to work with linux, mac-os and windows.
One of the feature I am implementing in my program is the ability to communicate between them (ie. A Console type program that can communicate and interact with GUI Program by sending commands and receiving reply messages, logging messages, signals,.etc). Back in windows dotnet framework, I am looking at anonymous pipes, but now, I am checking if Mono.Unix.UnixPipes will do the job for me and will let me implement inter process messaging with very little to no adjustments at all under linux, mac-os and windows.
I am a little bit new to this kind of feature, and i am now reading into the documentations (however, class and objects documentations are not helping me so much yet). I am also browsing to some of the inter process messaging questions that are posted here in stackoverflow.
If anyone has a link to a tutorial kind of document or example on how to do this, it will be a great help. please help?
thank you.
I highly recommend running a mongodb (easly scalable from a dev boxes to hundreds of servers) and using the library https://github.com/dominionenterprises/mongo-queue-csharp on top of it for the messaging. It has the ability to query for messages which enables some really nice patterns. Also its compatible with mono ! Also has some other languages if needed down the road.
My only experience with SOA comes from working with NServiceBus. It's really well designed and has some nifty features that I really like:
Processing of messages is transactional (and taken care of for you), so it's "once and only once" (as opposed to "at least once")
Built-in saga support
Automatic retry support
Pub/sub support (but I assume all service/message busses support this)
All that stuff leaves me to concentrate exclusively on the app's logic, but I've only used it in .NET, and Windows is almost certainly a non-starter for the project I'm working on. So my question is...
What service bus technologies out there exist in the *nix ecosystem (including experiences running NSB on Mono) and how do they measure up to NServiceBus running on Windows?
Bonus: In addition to the features above, what languages can send/receive/process messages from the bus? Having the flexibility to write one service in language A and another in language B would be very advantageous for this project.
SonicESB / MQ
I can speak from experience with SonicESB on Linux. It's basically SonicMQ - a JMS compliant MOM - with a process mediation layer on top; that's the service bus.
Pros
Itinerary-based routing; messages contain their own workflow for what services to visit
Intra-container messaging; increases performance by routing messages within the ESB container, avoiding the broker entirely.
Web service endpoints; ESB service may be invoked over HTTP endpoints, bridging the gap between non-Java/JMS clients and the bus.
Good development tools in eclipse for developing and debugging esb services and processes in Java.
Administration tools aren't bad either.
Cons
Transactions don't span service boundaries (may have changed since version 7.5)
No message retry mechanism; we rolled our own using a utility service and message parameter manipulation (may have changed in 7.5)
Pricey; licensed by core, so expect to pay for it.
Client Support
In terms of what technologies can send and receive to/from SonicESB, anything Java JMS, that's for sure. Sonic also offers a library in .net/C#/VB for Windows technology, so you can send and receive messages from the bus. This was a huge plus for us since our development teams use RIA's built in C#.
You may want to check out MuleESB, an open source service bus.
Hope it helps,
It has been quite a while since I've seriously used NServiceBus (in fact, back then, it wasn't even called that - it was just replacement library Udi wrote, to an awful comm library we had to use on a joint project :) ) so I don't know about all NServiceBus features.
I would say, based on what I do know, that Apache Kafka (originally developed by LinkedIn) would be a good starting point. It has relatively strong ordering guarantees, supports publish/subscribe to pools of consumers and is supportive of workflow behaviours e.g. Samza which provides a streaming framework on top of it
Also regarding the client interfaces. You can find the current list here
Is it possible to implement a RabbitMQ RPC between Java (acting as a client) and a .NET application (server/worker) using the RabbitMQ .NET library?
It seems like it should be possible, as RabbitMQ broker handles the queues and the socket connections are established between party and the RabbitMQ broker.
Are there any practical considerations in doing so?
The short answer is yes. In fact, my company uses RabbitMQ for that very reason - it offers a platform-neutral (as close as possible anyway) way to communicate between different applications. So in theory, I could have Java applications running on Linux and .NET applications running on Windows, and everything works together just fine.
You will need to come up with a common serialization format for your messages; I recommend using JSON as there are JSON libraries in every major programming language.
If you have any trouble with the RabbitMQ .NET library, feel free to post more questions :-)
Many times, I get:
-Frozen, load goes to 5.0. Can't use my box.
-Just doesn't work.
Do following steps:
1.rabbitmq-plugins enable rabbitmq_management
2.service rabbitmq-server restart
3.browse to http://rabbitmq-server-ip:15672
4.login with
username: guest
password: guest
Dont forget to change your password later.
As sheki notes, rabbitmqctl is your first port of call for diagnostics, and for building monitoring on top of, but it's not suitable for actual monitoring directly being a manual command line.
I've found DataDog very good to monitor both the MQ details, plus the host platform in parallel. e.g. you can watch the queue levels and set alerts on queues backing-up, while also watching the CPU/memory/IO inflicted by these queue levels. It really helps to get ratios of resource usage, and the alerts are good. Having a uniform platform for both infrastructure and application level monitoring is surprisingly rare, but speeds up diagnoses of production issues hugely.
NewRelic is similar and also has a RabbitMQ plugin, although I've not used this plugin specifically, I've used NR for years and found it invaluable in diagnosing operational issues.
AppDynamics is another example. Similarly this allows you to drill down into your app from a high-level dashboard, and visually navigate from problems to causes. It's especially good with visualising the network of a distributed application across various services/servers. I've used this, for example, to find complex problems in .NET applications and SQL Server clusters using 3rd party Web Services (e.g. latency and its consequences to your app over chatty protocols). These things are very difficult to diagnose, especially for developers who are limited to checking their code. Diagnosing operational issues requires a much broader picture.
I gave up trying to even install and configure Nagios. I know it's the 'best' but it's the best of an old breed of self-configured beasts which we don't have time to manage. I didn't even get it going... and eventually turned to the more 'modern' cloud approach. Once you get over the trust factor, it's pretty liberating.
I'm using these APM platforms together* to aggregate data from:
Windows O/S level Event Logs/Services
Linux O/S level
AWS console level
RDS, EC2
Apache
MySQL
App integrations / custom NR plugins I've written
Rabbit MQ
*NewRelic can feed into Datadog! So if you are already using NR you don't need to install DD on those hosts as well.
Being able to view all these levels together gives you a view on the publishers, middleware, MQ servers, workers and front-end app - all in one dashboard.
I would highly recommend an approach like this, because just looking at one server alone leads you to a lot of head-scratching. Seeing an entire stack in one customisable dashboard is just so illuminating it takes most of the guesswork out of it.
Worried about installing these things? I found New Relic to be especially light-weight and unobtrusive. AppDynamics seemed to stress the host a bit more, but mostly that's because you had to run the visualisation tools on the host! (this may have changed). DataDog seems performant, but creates a lot of control panels/icons on the target host (perhaps just a visual impression).
To a four year old question - this answer probably wasn't available in 2011, but in 2015 these once 'startup' style APM services are just tens or hundred dollars a month for an unbelievably rich enterprise-level solution.
There are bunch of RabbitMQ monitoring plugins available for different monitoring systems like Nagios, Zabbix etc.
Look at http://www.rabbitmq.com/how.html#management
Using rabbitmqctl is the most straight forward solution to check the status of the node.
$ rabbitmqctl status
This should tell you the status of the RabbitMQ node.
If you have PRTG (or any probe system with a HTTP sensor check), you can check the server status described at the following page:
https://blog.cdemi.io/monitoring-rabbitmq-in-prtg/
In particular you have to
Enable Management Plugin
The rabbitmq-management plugin provides an HTTP-based API for management and monitoring of your RabbitMQ
server, along with a browser-based UI and a command line tool,
rabbitmqadmin. The management plugin is included in the RabbitMQ
distribution. To enable it, we need to run: rabbitmq-plugins enable
rabbitmq_management on the RabbitMQ nodes. For more details on the
Management plugin refer to RabbitMQ Documentation.
The web UI is located at: http://server-name:15672/ The HTTP API and
its documentation are both located at: http://server-name:15672/api/
Once done, you can check the overview of your server with the API:
http://server-name:15672/api/overview
Where you have a JSON with all details about the server, active connections, queues, etc.
This cmd will help you service rabbitmq-server status
OR try theseservice rabbitmq-server stop and service rabbitmq-server start then service rabbitmq-server status.