I have created extended events in SQL server 2012. Everything is working fine.
Now I am looking for if any events occur (example :deadlock), it should send mail to given mail id.
Is it possible in extended events?
There is a very interesting article about it, basically you need to:
Enable service broker on the database.
Create a service broker queue to receive the event notification messages.
Create a service broker service to deliver event notification messages.
Create a service broker route to route the event notification message to the service broker queue.
Create event notification on deadlock event to create messages and send them to the service broker service
Through service broker, a
stored procedure can be written that responds to deadlock events. Event notifications allow deadlock graphs to be
transformed, stored, and sent wherever they need to go.
Store the deadlock graph in a table.
Retrieve the cached plans associated with the deadlock in another table.
Email the deadlock graph to the DBA team.
You can find the article with the examples on this link:
http://sqlmag.com/site-files/sqlmag.com/files/archive/sqlmag.com/content/content/142603/wpd-sql-extevtandnotif-us-sw-01112012_1.pdf
Pages of reference:
9 - 13
Related
I'm working on a library for interacting with the Discord API. My current setup is:
A gateway, each handling x amounts of shards - so that I can spin up as many of these as I like to scale well. These gateways publish events received to a redis message queue.
A client, which subscribes to the message queue, and responds to events received.
However, there are some scenarios - working with message components - where I want a specific client to handle events related to that message. This client will then use the node.js event emitter to emit an event in itself which is then received by a 'collector' in my code.
Does anyone have any recommendations how I might stop other clients from picking up the event from the message queue, so that only this specific client picks it up? Is it possible for a subscriber to 'read' an event before it like accepts it? As then all clients could read an event to see if it like matches a list of events its waiting for?
Requirement
A system undergoes some state change, and multiple other parts of the system has to know this(lets call them observers) so that they can perform some actions based on the current state, the actions of the observers are important, if some of the observers are not online(not listening currently due to some trouble, but will be back soon), the message should not be discarded till all the observers gets the message.
Trying to accomplish this with pub/sub model, here are my findings, (please correct if this understanding is wrong) -
The publisher creates an event on specific topic, and multiple subscribers can consume the same message. This model either provides no delivery guarantee(in redis), or delivery is guaranteed once(with messaging queues), ie. when one of the consumer acknowledges a message, the message is discarded(rabbitmq).
Example
A new Person Profile entity gets created in DB
Now,
A background verification service has to know this to trigger the verification process.
Subscriptions service has to know this to add default subscriptions to the user.
Now both the tasks are important, unrelated and can run in parallel.
Now In Queue model, if subscription service is down for some reason, a BG verification process acknowledges the message, the message will be removed from the queue, or if it is fire and forget like most of pub/sub, the delivery is anyhow not guaranteed for both the services.
One more point is both the tasks are unrelated and need not be triggered one after other.
In short, my need is to make sure all the consumers gets the same message and they should be able to acknowledge them individually, the message should be evicted only after all the consumers acknowledged it either of the above approaches doesn't do this.
Anything I am missing here ? How should I approach this problem ?
This scenario is explicitly supported by RabbitMQ's model, which separates "exchanges" from "queues":
A publisher always sends a message to an "exchange", which is just a stateless routing address; it doesn't need to know what queue(s) the message should end up in
A consumer always reads messages from a "queue", which contains its own copy of messages, regardless of where they originated
Multiple consumers can subscribe to the same queue, and each message will be delivered to exactly one consumer
Crucially, an exchange can route the same message to multiple queues, and each will receive a copy of the message
The key thing to understand here is that while we talk about consumers "subscribing" to a queue, the "subscription" part of a "pub-sub" setup is actually the routing from the exchange to the queue.
So a RabbitMQ pub-sub system might look like this:
A new Person Profile entity gets created in DB
This event is published as a message to an "events" topic exchange with a routing key of "entity.profile.created"
The exchange routes copies of the message to multiple queues:
A "verification_service" queue has been bound to this exchange to receive a copy of all messages matching "entity.profile.#"
A "subscription_setup_service" queue has been bound to this exchange to receive a copy of all messages matching "entity.profile.created"
The consuming scripts don't know anything about this routing, they just know that messages will appear in the queue for events that are relevant to them:
The verification service picks up the copy of the message on the "verification_service" queue, processes, and acknowledges it
The subscription setup service picks up the copy of the message on the "subscription_setup_service" queue, processes, and acknowledges it
If there are multiple consuming scripts looking at the same queue, they'll share the messages on that queue between them, but still completely independent of any other queue.
Here's a screenshot from this interactive visualisation tool that shows this scenario:
As you mentioned it is not something that you can control with Redis Pub/Sub data structure.
But you can do it easily with Redis Streams.
Streams will allow you to post messages using the XADD command and then control which consumers are dealing with the message and acknowledge that message has been processed.
You can look at these sample application that provides (in Java) example about:
posting and consuming messages
create multiple consumer groups
manage exceptions
Links:
Getting Started with Redis Streams and Java
Redis Streams in Action ( Project that shows how to use ADD/ACK/PENDING/CLAIM and build an error proof streaming application with Redis Streams and SpringData )
I have a system where clients pull tasks from a queue (via amqp, using RabbitMQ as a message server). When a new client gets spawned, he pulls a task, tells the main server that he is the one in charge for that specific task and starts executing the long-time task. From the server side I want to be able to cancel the task on this client so he can move on to another one in case the task already causes errors.
Pushing a cancel task on the existing task queue is no option because I have to guarantee it arrives in time or even at the same client.
When the client gets spawned, is it a good approach that the client sets up a dedicated exchange and queue only for him where the server can send commands to?
The client won't have to set up a dedicated exchange - you can use the default topic exchange or set up your own topic exchange.
Then, when the client pulls a task it also creates its own exclusive queue and binds that queue to the topic exchange for a specific "cancel task X" routing key. Then, the client subscribes to this queue. If the task must be cancelled, the "server side" can publish a message to the topic exchange with the appropriate routing key.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I am thinking how to remove duplicacy for subscriber cluster in pubsub, for example:
There is a service called email, which should send welcome emails after user signing up. By using pub/sub, email service shall listen a event called "signedUp" which will be triggered each time user sign up. However, how about if I have 2 email services to balance? Without any special efforts, I think two welcome emails will be sent out. So how to solve this issue?
I prefer redis as pubsub server for simplicity, or rabbitmq if redis doesn't work out.
I don't think it is possible to do it in redis. But in rabbitmq, it can, let me explain below:
Rabbitmq has a separate stuff called 'exchange' from queue. So server publish a message to exchange, client can create queues to bind to the exchange. Therefore, instances from one services can create the same queue to bind with exchange, by doing that exchange will delivery message to the queue once and handled by only one instance once.
Account service:
channel.assertExchange(‘signedUp’, 'fanout')
channel.publish(ex, '', new Buffer(message)
Email service:
let queue = channmel.assertQueue(‘email’);
channel.bindQueue(queue, 'signedUp'); // bind this queue to exchange
ch.consume(queue, logMessage)
By given a queue name in email service, no matter how many email services started, the published message (signedUp in this case) will be handled by one and ONLY ONE email service.
Just started learning NServiceBus and trying to understand the concept.
When it talks about queues, are we talking about MSMQs on both publisher and subscriber?
So, if I have an application that generates a list of something (say, name of animals), then it dumps the list into publisher’s queue. The publisher polls the queue every minute and if there is something in the queue, it will publish to subscriber’s queue for further processing. Does this make sense?
Thanks.
The sequence of events for a publish is as follows:
The Publisher will start up(Windows Service)
A Subscriber will start up and place a message into the Publisher's input queue(MSMQ)
The Publisher will take that message, read the address of the Subscriber and place that into storage(subscription storage: memory, MSMQ, or RDBMS)
When it is time to publish and event, the Publisher will inspect the type of message and then read subscription storage to find Subscribers interested in that message
The Publisher will then send a message to each of the Subscribers found in subscription storage
The Subscriber receives the message in its input queue(MSMQ) and processes it
You can leverage other messaging platforms instead of MSMQ, but MSMQ is the default. There really is no polling done, all the endpoints are signaled when a message hits the queues.
MSMQ is a transport layer. It passes the messages around.
The application will publish something using a NServiceBus queue. If you configured it to use MSMQ, that's what it will use for its transport layer and this is what the subscribers will be looking at.
NServiceBus follows the publisher/subscriber model as you have correctly stated. However your confusion is based on the use of two queues. This is incorrect. The server (publisher) will maintain the queue which is interfaced via the MSMQ protocol and so your application would communicate directly with this possibly remotely or locally.
You would typically use a WCF service which would raise an event upon a new message being pushed onto the queue. Your application can then make use of this new message as desired. See the NServiceBus documentation for examples: http://www.nservicebus.com/ArchitecturalPrinciples.aspx