I've implemented a simple publisher/consumer set with MassTransit, and I want to have the consumers read the messages from the same queue. However, when I run it, I see a large portion of the messages sent to the error queue instead of being consumed. From the discussions I've seen (SO, Forum), this should be really really simple with RabbitMQ (just point to the same queue), but it's not working. Is there an additional configuration that should be set?
Here's My Publisher
public class YourMessage { public string Text { get; set; } }
public class Program
{
public static void Main()
{
Console.WriteLine("Publisher");
Bus.Initialize(sbc =>
{
sbc.UseRabbitMqRouting();
sbc.ReceiveFrom("rabbitmq://localhost/test_queue");
});
var x = Console.Read();
for (var i = 0; i <= 1000; i++)
{
Console.WriteLine("Message Number " + i);
Bus.Instance.Publish(new YourMessage { "Message Number " + i });
}
}
}
And My Consumer
public class YourMessage { public string Text { get; set; } }
public class Program
{
public static void Main()
{
Console.WriteLine("Consumer");
Bus.Initialize(sbc =>
{
sbc.UseRabbitMqRouting();
sbc.ReceiveFrom("rabbitmq://localhost/test_queue");
sbc.Subscribe(subs =>
{
var del = new Action<IConsumeContext<YourMessage>,YourMessage>((context, msg) =>
{
Console.WriteLine(msg.Text);
});
subs.Handler<YourMessage>(del);
});
});
while (true) { }
}
}
The receiver/consumer and publisher cannot be on the same queue. If you want competing consumers have multiple instances of the consumer running against the same queue.
We have documentation, but this section is currently lacking, so I understand your confusion: http://readthedocs.org/docs/masstransit/en/latest/configuration/gotchas.html#how-to-setup-a-competing-consumer If you succeed, help with the documentation would be wonderful.
So, it looks like the solution was to change the line in the publisher:
sbc.ReceiveFrom("rabbitmq://localhost/test_queue");
To something like:
sbc.ReceiveFrom("rabbitmq://localhost/test_queue_publisher");
This prevented the publishers from competing for messages they weren't configured to consume.
Just found an example on GitHub https://github.com/khalidabuhakmeh/MassTransit.ScaleOut ...
Related
I have an issue with MassTransit not sending messages with the following code - this is a port from our Azure Service Bus code which works fine. The examples in GitHub populate the Queue - Starbucks example, so my infrastructure is working.
Can anyone please suggest why this is not sending messages? I have created both the queue and exchange, and tried without.
The console app prints out the expected results.
Thanks in advance.
public class Program
{
static void Main()
{
IBusControl busControl = CreateBus();
TaskUtil.Await(() => busControl.StartAsync());
List<Task> tList = new List<Task>();
for (int i = 0; i < 10; i++)
{
var t = Send(busControl);
tList.Add(t);
}
Task.WaitAll(tList.ToArray());
Console.WriteLine("done!");
}
private static async Task Send(IBusControl busControl)
{
var endpoint = await busControl.GetSendEndpoint(new Uri("rabbitmq://localhost/test"));
Console.WriteLine("Sending");
await endpoint.Send(new SomethingHappenedMessage()
{
What = "Stuff",
When = DateTime.Now
});
Console.WriteLine("Sent");
}
static IBusControl CreateBus()
{
return Bus.Factory.CreateUsingRabbitMq(x => x.Host(new Uri("rabbitmq://localhost"), h =>
{
h.Username("guest");
h.Password("guest");
}));
}
}
public interface SomethingHappened
{
string What { get; }
DateTime When { get; }
}
public class SomethingHappenedMessage : SomethingHappened
{
public string What { get; set; }
public DateTime When { get; set; }
}
When you send messages with MassTransit using RabbitMQ, by default the bindings for the queue are not created. It is assumed that a receive endpoint in a service will create the queue and related bindings.
To ensure that the queue and bindings exist when sending a message, you can modify the endpoint address to include some additional query string parameters as shown below:
rabbitmq://localhost/vhost/exchange_name?bind=true&queue=queue_name
In the case of a receive endpoint, the exchange name and queue name are the same.
I want to implement following scenario using rebus. I am creating on sender application and one receiving application. There will be a class suppose
public class GetPersonRequest
{
public int Id { get; set; }
public string Name { get; set; }
}
public class GetPersonResponse
{
public int Id { get; set; }
public string Name { get; set; }
}
I will send this class object in queue with values. And want to display those value in receiver. How to achieve this?
SENDER code like this:
static void Main(string[] args)
{
GetPersonRequest objGetPersonRequest = new GetPersonRequest();
objGetPersonRequest.Id = 12;
objGetPersonRequest.Name = "Kumar";
using (var activator = new BuiltinHandlerActivator())
{
activator.Register(() => new PrintName());
var bus = Configure.With(activator)
.Logging(l => l.None())
.Transport(t => t.UseMsmq("rebus-application.input"))
.Routing(r => r.TypeBased().Map<GetPersonRequest>("rebus.application.output"))
.Start();
bus.Send(objGetPersonRequest);
Console.WriteLine("Press enter to quit");
Console.ReadLine();
}
RECEIVER Code like this in another console application:
static void Main(string[] args)
{
using (var activator = new BuiltinHandlerActivator())
{
activator.Register(() => new PrintName());
var bus = Configure.With(activator)
.Logging(l => l.None())
.Transport(t => t.UseMsmq("rebus-application.output"))
.Routing(r => r.TypeBased().Map<GetPersonResponse>("rebus-application.input"))
.Start();
Console.WriteLine("Press enter to quit");
Console.ReadLine();
}
}
class PrintName : IHandleMessages<GetPersonResponse>
{
public async Task Handle(GetPersonResponse objGetPersonResponse)
{
Console.WriteLine("RebusDetails Name is {0}", objGetPersonResponse.Name);
}
}
How to achieve this?
I suggest you take a look at the request/reply sample from the RebusSamples repository - it shows the configuration needed in order to do proper request/reply.
From quickly glancing over your code, I can see the following issues/misunderstandings:
Rebus methods are asynchronous, hence bus.Send(objGetPersonRequest) will execute on another thread and you will not know if it failed - always either await bus.Send(...) or bus.Send(...).Wait()
In many cases, only "clients" (*) should have endpoint mappings - in your case, you should map GetPersonRequest (or possibly the entire assembly containing it?) to rebus.application.output, and then do an await bus.Reply(new GetPersonResponse(...)) in the handler - this way, the "server"(*) will not have any dependencies
Moreover - this might be a detail, but I think it leads to a better understanding and easier communication over time:
There's no such thing as an "output queue" - all queues are the input queue of the endpoint that has it as its input queue - therefore, I would argue that the name rebus-application.output is misleading
I suggest you change your queue names to something that identifies each endpoint better, e.g. since your server seems to be capable of returning a person's details, you could call it masterdata, crm, etc., possibly suffixing .input if you e.g. want to have an error queue for each endpoint (e.g. masterdata.input and masterdata.error).
I hope that makes sense :)
(*) In my experience, it's beneficial to have a pretty clear distinction between client and server roles for your endpoints, where clients are endpoints with no (or very few) afferent couplings, which allows for them to easily be added/removed/changed, and servers are endpoints with more afferent couplings.
When you await bus.Reply(...) from a server, it allows for the sender to remain a client and not have its endpoint address configured anywhere but in its own configuration.
We've encountered a situation where MassTransit is losing messages if you create a publisher and consumer using the same endpoint name.
Note the code below; if I use a different endpoint name for either the consumer or publisher (e.g. "rabbitmq://localhost/mtlossPublised" for the publisher) then the message counts both published and consumed match; if I use the same endpoint name (as in the sample) then I get less messages consumed than published.
Is this expected behaviour? or am I doing something wrong, working sample code below.
using MassTransit;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace MTMessageLoss
{
class Program
{
static void Main(string[] args)
{
var consumerBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
var publisherBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
consumerBus.SubscribeConsumer(() => new MessageConsumer());
for (int i = 0; i < 10; i++)
publisherBus.Publish(new SimpleMessage() { CorrelationId = Guid.NewGuid(), Message = string.Format("This is message {0}", i) });
Console.WriteLine("Press ENTER Key to see how many you consumed");
Console.ReadLine();
Console.WriteLine("We consumed {0} simple messages. Press Enter to terminate the applicaion.", MessageConsumer.Count);
Console.ReadLine();
consumerBus.Dispose();
publisherBus.Dispose();
}
}
public interface ISimpleMessage : CorrelatedBy<Guid>
{
string Message { get; }
}
public class SimpleMessage : ISimpleMessage
{
public Guid CorrelationId { get; set; }
public string Message { get; set; }
}
public class MessageConsumer : Consumes<ISimpleMessage>.All
{
public static int Count = 0;
public void Consume(ISimpleMessage message)
{
System.Threading.Interlocked.Increment(ref Count);
}
}
}
Bottom line, every instance of a bus needs it's own queue to read from. Even if the bus only exists to publish messages. This is just a requirement of how MassTransit works.
http://masstransit.readthedocs.org/en/master/configuration/config_api.html#basic-options - see the warning.
We leave the behaviour as undefined when two bus instances share the same queue. Regardless, it's not a condition we support. Each bus instance may send meta data to other bus instances, and requires it's own endpoint. This was a much bigger deal with MSMQ, so maybe we could get this case to work on RabbitMQ - but it's not something we've spent much thought into at this point.
What's happening is that in giving the same Receiver Uri you're telling MT to load balance consumption on the two busses, however you've only one bus listening to the messages.
If you get it to keep track of which messages are received you'll see it's (nearly) every second one.
Having tweaked your sample code I get
We consumed 6 simple messages. Press Enter to terminate the applicaion.
Received 0
Received 3
Received 5
Received 6
Received 7
Received 8
Start a consumer on the other bus and you'll get them all
We consumed 10 simple messages. Press Enter to terminate the applicaion.
Received 0
Received 1
Received 2
Received 3
Received 4
Received 5
Received 6
Received 7
Received 8
Received 9
So yes, I'd say this is expected behaviour.
Here's the tweaked sample code with two subscribers
using MassTransit;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace MTMessageLoss
{
class Program
{
internal static bool[] msgReceived = new bool[10];
static void Main(string[] args)
{
var consumerBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
var publisherBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
publisherBus.SubscribeConsumer(() => new MessageConsumer());
consumerBus.SubscribeConsumer(() => new MessageConsumer());
for (int i = 0; i < 10; i++)
consumerBus.Publish(new SimpleMessage()
{CorrelationId = Guid.NewGuid(), MsgId = i});
Console.WriteLine("Press ENTER Key to see how many you consumed");
Console.ReadLine();
Console.WriteLine("We consumed {0} simple messages. Press Enter to terminate the applicaion.",
MessageConsumer.Count);
for (int i = 0; i < 10; i++)
if (msgReceived[i])
Console.WriteLine("Received {0}", i);
Console.ReadLine();
consumerBus.Dispose();
publisherBus.Dispose();
}
}
public interface ISimpleMessage : CorrelatedBy<Guid>
{
int MsgId { get; }
}
public class SimpleMessage : ISimpleMessage
{
public Guid CorrelationId { get; set; }
public int MsgId { get; set; }
}
public class MessageConsumer : Consumes<ISimpleMessage>.All
{
public static int Count = 0;
public void Consume(ISimpleMessage message)
{
Program.msgReceived[message.MsgId] = true;
System.Threading.Interlocked.Increment(ref Count);
}
}
}
I'm using NServiceBus and I need to know how many clients are subscribed to a specific message type (even better the names of the subscribers). I'm talking in a pub\sub scenario.
Is it possible to get this information in NServiceBus?
Thx
You can pull this right out of your subscription storage. Either a query to the database or a .GetAllMessages() on the queue will get you a count and the subscribers address. If you are looking to do this in code, you could write a handler for the subscription message and count them up that way.
I have used ISubscriptionStorage with success.
public class SubscribersForMessageHandler :
IHandleMessages<SubscribersForMessageRequest>
{
public ISubscriptionStorage Storage { get; set; }
public IBus Bus { get; set; }
public void Handle(SubscribersForMessageRequest message)
{
Bus.Reply<SubscribersForMessageResponse>(m=>
{
m.SagaId = message.SagaId;
m.MessageType = message.MessageType;
m.SubscriberEndpoints = GetSubscribersForMessage(message.MessageType);
});
}
private List<string> GetSubscribersForMessage(string type)
{
return Storage.GetSubscribersForMessage(
new List<string> { type }).ToList();
}
}
I have 2 different sagas (I mean saga types) that handle the same message.
public class AttachMessageToBugSaga : TpSaga<AttachMessageToBugSagaData>, IAmStartedByMessages<MessageIsNotAttached>, IHandleMessages<MessageAttachedToGeneralMessage>
{
public override void ConfigureHowToFindSaga()
{
ConfigureMapping<MessageAttachedToGeneralMessage>(
saga => saga.Id,
message => message.SagaId
);
}
public void Handle(MessageIsNotAttachedToBug message)
{
Send(new AttachMessageToGeneralCommand { MessageId = 66, GeneralId = 13 });
}
public void Handle(MessageAttachedToGeneralMessage message)
{
//do some stuf fhere
}
}
public class AttachMessageToBugSagaData : IContainSagaData
{
public Guid Id { get; set; }
public string Originator { get; set; }
public string OriginalMessageId { get; set; }
}
public class AttachMessageToRequestSaga : TpSaga<AttachMessageToRequestSagaData>, IAmStartedByMessages<MessageIsNotAttachedToRequest>, IHandleMessages<MessageAttachedToGeneralMessage>
{
public override void ConfigureHowToFindSaga()
{
ConfigureMapping<MessageAttachedToGeneralMessage>(
saga => saga.Id,
message => message.SagaId
);
}
public void Handle(MessageIsNotAttachedMessageToRequest message)
{
//do some stuff here
}
public void Handle(MessageAttachedToGeneralMessage message)
{
//do some stuff here
}
}
public class AttachMessageToRequestSagaData : IContainSagaData
{
public Guid Id { get; set; }
public string Originator { get; set; }
public string OriginalMessageId { get; set; }
}
When I run the sample I get an exception :
System.InvalidCastException: Unable to cast object of type 'MyCustomPlugin.AttachMessageToGeneralSagaData' to type 'MyCustomPlugin.AttachMessageToRequestSagaData'.
I understand why it happens, but I still need some workaround. I tried to implement my own IFindSagas class :
public class SagaFinder : IFindSagas<AttachMessageToGeneralSagaData>.Using<MessageAttachedToGeneralMessage>,
IFindSagas<AttachMessageToRequestSagaData>.Using<MessageAttachedToGeneralMessage>,
IFindSagas<AttachMessageToRequestSagaData>.Using<MessageIsNotAttachedToRequest>,
IFindSagas<AttachMessageToRequestSagaData>.Using<MessageIsNotAttachedToBug>
{
AttachMessageToGeneralSagaData IFindSagas<AttachMessageToGeneralSagaData>.Using<MessageAttachedToGeneralMessage>.FindBy(MessageAttachedToGeneralMessage message)
{
return ObjectFactory.GetInstance<AttachMessageToGeneralSagaData>();
}
AttachMessageToRequestSagaData IFindSagas<AttachMessageToRequestSagaData>.Using<MessageAttachedToGeneralMessage>.FindBy(MessageAttachedToGeneralMessage message)
{
return ObjectFactory.GetInstance<AttachMessageToRequestSagaData>();
}
public AttachMessageToRequestSagaData FindBy(MessageIsNotAttachedToRequest message)
{
return new AttachMessageToRequestSagaData();
}
public AttachMessageToRequestSagaData FindBy(MessageIsNotAttachedToBug message)
{
return new AttachMessageToRequestSagaData();
}
}
But I do not get into my finders for "MessageAttachedToGeneralMessage".
Please tell me if there is some other workaround, or how to make this example working.
I'm not sure that having more than one Saga within the same process boundary works very well - at least, I've had problems with it too. It's probably better (in general) to have Sagas separated into two different processes anyway, because otherwise it would cause a lot of locking and potentially deadlocks on your saga storage.
Is your message that is handled by 2 Sagas Sent or Published? If it's published (or can be made to), it would be easy to separate the Sagas into two separate assemblies. Just be sure to manually call Bus.Subscribe() for the message type in each Saga, since Sagas don't auto-subscribe to messages listed in the app.config.
If your message is Sent, and there's nothing you can do to change it, then create a central handler for your existing message type that either Publishes a second message type to go to both Sagas, or Sends two separate messages to each saga.
Finally (after digging into the source code) I've found the solution. It seems the only way is to implement my own SagaPersister, where I can do anything I want.
Default implementation in NserviceBus of InMemorySagaPersister has the following code :
T ISagaPersister.Get<T>(Guid sagaId)
{
ISagaEntity result;
data.TryGetValue(sagaId, out result);
return (T)result;
}
And exception occurs while casting .