I have an issue with MassTransit not sending messages with the following code - this is a port from our Azure Service Bus code which works fine. The examples in GitHub populate the Queue - Starbucks example, so my infrastructure is working.
Can anyone please suggest why this is not sending messages? I have created both the queue and exchange, and tried without.
The console app prints out the expected results.
Thanks in advance.
public class Program
{
static void Main()
{
IBusControl busControl = CreateBus();
TaskUtil.Await(() => busControl.StartAsync());
List<Task> tList = new List<Task>();
for (int i = 0; i < 10; i++)
{
var t = Send(busControl);
tList.Add(t);
}
Task.WaitAll(tList.ToArray());
Console.WriteLine("done!");
}
private static async Task Send(IBusControl busControl)
{
var endpoint = await busControl.GetSendEndpoint(new Uri("rabbitmq://localhost/test"));
Console.WriteLine("Sending");
await endpoint.Send(new SomethingHappenedMessage()
{
What = "Stuff",
When = DateTime.Now
});
Console.WriteLine("Sent");
}
static IBusControl CreateBus()
{
return Bus.Factory.CreateUsingRabbitMq(x => x.Host(new Uri("rabbitmq://localhost"), h =>
{
h.Username("guest");
h.Password("guest");
}));
}
}
public interface SomethingHappened
{
string What { get; }
DateTime When { get; }
}
public class SomethingHappenedMessage : SomethingHappened
{
public string What { get; set; }
public DateTime When { get; set; }
}
When you send messages with MassTransit using RabbitMQ, by default the bindings for the queue are not created. It is assumed that a receive endpoint in a service will create the queue and related bindings.
To ensure that the queue and bindings exist when sending a message, you can modify the endpoint address to include some additional query string parameters as shown below:
rabbitmq://localhost/vhost/exchange_name?bind=true&queue=queue_name
In the case of a receive endpoint, the exchange name and queue name are the same.
Related
I've got an Asp.Net 2.1 project that acts as a service host for consuming messages published by other processes/applications. I've setup/configured multiple consumers in the Startup class (Startup.cs) as below (only MassTransit portion is given here for brevity):
public void ConfigureServices(IServiceCollection services)
{
services.AddScoped<SendMessageConsumer>();
services.AddScoped<AnotherMessageConsumer>();
services.AddMassTransit(c =>
{
c.AddConsumer<SendMessageConsumer>();
c.AddConsumer<AnotherMessageConsumer>();
});
services.AddSingleton(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host("localhost", "/", h => { });
cfg.ReceiveEndpoint(host, "Queue-1", e =>
{
e.PrefetchCount = 16;
e.UseMessageRetry(x => x.Interval(2, 100));
e.LoadFrom(provider);
e.Consumer<SendMessageConsumer>();
EndpointConvention.Map<Message>(e.InputAddress);
});
cfg.ReceiveEndpoint(host, "Queue-2", e =>
{
e.PrefetchCount = 16;
e.UseMessageRetry(x => x.Interval(2, 100));
e.LoadFrom(provider);
e.Consumer<AnotherMessageConsumer>();
EndpointConvention.Map<AnotherMessage>(e.InputAddress);
});
}));
services.AddSingleton<IBus>(provider => provider.GetRequiredService<IBusControl>());
services.AddSingleton<IHostedService, BusService>();
}
Messages:
namespace MasstransitDemo.Models
{
public class Message
{
public string Value { get; set; }
}
public class AnotherMessage
{
public string Value { get; set; }
}
}
Consumers:
public class SendMessageConsumer : IConsumer<Message>
{
public Task Consume(ConsumeContext<Message> context)
{
Console.WriteLine($"Receive message value: {context.Message.Value}");
return Task.CompletedTask;
}
}
public class AnotherMessageConsumer : IConsumer<AnotherMessage>
{
public Task Consume(ConsumeContext<AnotherMessage> context)
{
Console.WriteLine($"Receive another message value: {context.Message.Value}");
return Task.CompletedTask;
}
}
This causes both messages to come through to each queue. See the resulting RabbitMq exchanges below:
How do I set it up to make SendMessageConsumer to receive only "Message" and AnotherMessageConsumer to receive "AnotherMessage"?
Thanks in advance.
You tell MassTransit about your consumer explicitly, but also load all consumers from the container, for each endpoint.
e.LoadFrom(provider);
e.Consumer<AnotherMessageConsumer>();
By doing this, you all both consumers to each endpoint with LoadFrom, plus one consumer in addition by Consumer<T>. So each of your endpoints gets three consumers, and you get both queues bound to both exchanges.
You don't need to use LoadFromContainer here. If your consumer has a dependency that needs to be resolved by the container, you can use this:
e.Consumer<AnotherMessageConsumer>(container);
It is not working for me. do you commented both e.loadfrom(provider). please share you code snippet to understand
services.AddSingleton(provider => Bus.Factory.CreateUsingRabbitMq(configurator =>
{
IRabbitMqHost rabbitMqHost=configurator.Host(_busConfiguration.RabbitMqUri, _busConfiguration.Port, _busConfiguration.Vhost,hostConfigurator =>
{
hostConfigurator.Username(_busConfiguration.UserName);
hostConfigurator.Password(_busConfiguration.Password);
});
configurator.ReceiveEndpoint(rabbitMqHost,_busConfiguration.GeneratePayLoadQueue, e =>
{
e.PrefetchCount = _busConfiguration.PrefetchCount;
//e.LoadFrom(provider);
e.Consumer<StagingConsumerService>(provider);
EndpointConvention.Map<StagingConsumer>(e.InputAddress);
});
configurator.ReceiveEndpoint(rabbitMqHost, _busConfiguration.CreateJournalQueue , e =>
{
e.PrefetchCount = _busConfiguration.PrefetchFinDocCount;
//e.LoadFrom(provider);
e.Consumer<FinDocConsumerService>();
EndpointConvention.Map<FinDocConsumer>(e.InputAddress);
});
}));
services.AddSingleton<IBus>(provider => provider.GetRequiredService<IBusControl>());
services.AddSingleton<IHostedService, BusService>();
We've encountered a situation where MassTransit is losing messages if you create a publisher and consumer using the same endpoint name.
Note the code below; if I use a different endpoint name for either the consumer or publisher (e.g. "rabbitmq://localhost/mtlossPublised" for the publisher) then the message counts both published and consumed match; if I use the same endpoint name (as in the sample) then I get less messages consumed than published.
Is this expected behaviour? or am I doing something wrong, working sample code below.
using MassTransit;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace MTMessageLoss
{
class Program
{
static void Main(string[] args)
{
var consumerBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
var publisherBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
consumerBus.SubscribeConsumer(() => new MessageConsumer());
for (int i = 0; i < 10; i++)
publisherBus.Publish(new SimpleMessage() { CorrelationId = Guid.NewGuid(), Message = string.Format("This is message {0}", i) });
Console.WriteLine("Press ENTER Key to see how many you consumed");
Console.ReadLine();
Console.WriteLine("We consumed {0} simple messages. Press Enter to terminate the applicaion.", MessageConsumer.Count);
Console.ReadLine();
consumerBus.Dispose();
publisherBus.Dispose();
}
}
public interface ISimpleMessage : CorrelatedBy<Guid>
{
string Message { get; }
}
public class SimpleMessage : ISimpleMessage
{
public Guid CorrelationId { get; set; }
public string Message { get; set; }
}
public class MessageConsumer : Consumes<ISimpleMessage>.All
{
public static int Count = 0;
public void Consume(ISimpleMessage message)
{
System.Threading.Interlocked.Increment(ref Count);
}
}
}
Bottom line, every instance of a bus needs it's own queue to read from. Even if the bus only exists to publish messages. This is just a requirement of how MassTransit works.
http://masstransit.readthedocs.org/en/master/configuration/config_api.html#basic-options - see the warning.
We leave the behaviour as undefined when two bus instances share the same queue. Regardless, it's not a condition we support. Each bus instance may send meta data to other bus instances, and requires it's own endpoint. This was a much bigger deal with MSMQ, so maybe we could get this case to work on RabbitMQ - but it's not something we've spent much thought into at this point.
What's happening is that in giving the same Receiver Uri you're telling MT to load balance consumption on the two busses, however you've only one bus listening to the messages.
If you get it to keep track of which messages are received you'll see it's (nearly) every second one.
Having tweaked your sample code I get
We consumed 6 simple messages. Press Enter to terminate the applicaion.
Received 0
Received 3
Received 5
Received 6
Received 7
Received 8
Start a consumer on the other bus and you'll get them all
We consumed 10 simple messages. Press Enter to terminate the applicaion.
Received 0
Received 1
Received 2
Received 3
Received 4
Received 5
Received 6
Received 7
Received 8
Received 9
So yes, I'd say this is expected behaviour.
Here's the tweaked sample code with two subscribers
using MassTransit;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace MTMessageLoss
{
class Program
{
internal static bool[] msgReceived = new bool[10];
static void Main(string[] args)
{
var consumerBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
var publisherBus = ServiceBusFactory.New(b =>
{
b.UseRabbitMq();
b.UseRabbitMqRouting();
b.ReceiveFrom("rabbitmq://localhost/mtloss");
});
publisherBus.SubscribeConsumer(() => new MessageConsumer());
consumerBus.SubscribeConsumer(() => new MessageConsumer());
for (int i = 0; i < 10; i++)
consumerBus.Publish(new SimpleMessage()
{CorrelationId = Guid.NewGuid(), MsgId = i});
Console.WriteLine("Press ENTER Key to see how many you consumed");
Console.ReadLine();
Console.WriteLine("We consumed {0} simple messages. Press Enter to terminate the applicaion.",
MessageConsumer.Count);
for (int i = 0; i < 10; i++)
if (msgReceived[i])
Console.WriteLine("Received {0}", i);
Console.ReadLine();
consumerBus.Dispose();
publisherBus.Dispose();
}
}
public interface ISimpleMessage : CorrelatedBy<Guid>
{
int MsgId { get; }
}
public class SimpleMessage : ISimpleMessage
{
public Guid CorrelationId { get; set; }
public int MsgId { get; set; }
}
public class MessageConsumer : Consumes<ISimpleMessage>.All
{
public static int Count = 0;
public void Consume(ISimpleMessage message)
{
Program.msgReceived[message.MsgId] = true;
System.Threading.Interlocked.Increment(ref Count);
}
}
}
I have the same issue as this question on MSDN, but I don't understand the solution because it is still not clear to me if Roman Kiss's solution will correctly replace an endpoint address while a single workflow instance being executed concurrently.
When internal Send activity is scheduled for execution by one thread with certain enpoint address, wouldn't this address be overridden by another thread that schedules same activity with different endpoint address? Correct me if I am mistaken, but I assume it would, because Send.Endpoint is a regular property as oppose to being InArgument<Endpoint> bound to whatever current workflow execution context is.
Can someone shed more light onto this?
UPDATE
I tested the solution provided by Roman Kiss, and it turns out that it is not working as expected in my scenario. I modified Execute method as follows:
protected override void Execute(NativeActivityContext context)
{
Thread.Sleep(Address.Get(context).EndsWith("1") ? 1000 : 0);
Body.Endpoint.Binding = GetBinding(Binding.Get(context));
Body.Endpoint.AddressUri = new Uri(Address.Get(context));
Thread.Sleep(Address.Get(context).EndsWith("1") ? 0 : 3000);
var address = Address.Get(context) + " => " + Body.Endpoint.AddressUri;
Console.WriteLine(address);
Thread.Sleep(10000);
context.ScheduleActivity(Body);
}
Ran this test:
static void Main(string[] args)
{
// Workflow1 is just a SendScope wrapped around by a Sequence with single Address input argument exposed
var workflow = new Workflow1();
Task.WaitAll(
Task.Run(() => WorkflowInvoker.Invoke(workflow, new Dictionary<string, object> { { "Address", #"http://localhost/1" } })),
Task.Run(() => WorkflowInvoker.Invoke(workflow, new Dictionary<string, object> { { "Address", #"http://localhost/2" } })));
Console.ReadLine();
}
The result I am getting is:
http://localhost/1 => http://localhost/1
http://localhost/2 => http://localhost/1
The question remains open: how do I assign endpoint address of my Send activity dynamically at runtime?
This will work as shown because a new Send activity is created by the factory and so when using the CacheMetadata method to setup that Send activity it is setting the binding properly on that instance of the activity.
Including Content Incase Link Dies
[ContentProperty("Body")]
public class SendScope : NativeActivity
{
[DefaultValue((string)null)]
[RequiredArgument]
public InArgument<string> Binding { get; set; }
[DefaultValue((string)null)]
[RequiredArgument]
public InArgument<string> Address { get; set; }
[Browsable(false)]
public Send Body { get; set; }
protected override void CacheMetadata(NativeActivityMetadata metadata)
{
if (this.Body == null || this.Body.EndpointAddress != null)
{
metadata.AddValidationError("Error ...");
return;
}
this.Body.Endpoint = new Endpoint()
{
AddressUri = new Uri("http://localhost/"),
Binding = new BasicHttpBinding(),
ServiceContractName = this.Body.ServiceContractName
};
metadata.AddChild(this.Body);
base.CacheMetadata(metadata);
}
protected override void Execute(NativeActivityContext context)
{
this.Body.Endpoint.Binding = GetBinding(this.Binding.Get(context));
this.Body.Endpoint.AddressUri = new Uri(this.Address.Get(context));
context.ScheduleActivity(Body);
}
private System.ServiceModel.Channels.Binding GetBinding(string binding)
{
if (binding == "basicHttpBinding")
return new BasicHttpBinding();
//else ... others bindings
return null;
}
}
public class SendScopeFactory : IActivityTemplateFactory
{
public Activity Create(DependencyObject target)
{
return new SendScope()
{
DisplayName = "SendScope",
Body = new Send()
{
Action = "*",
OperationName = "ProcessMessage",
ServiceContractName = "IGenericContract",
}
};
}
}
Create a custom native activity for setting Send.Endpoint property during the runtime based on your properties such as Binding, Address, Security, etc.
Create designer for this SendScope activity something simular like CorrelationScope
Create SendScopeFactory - see the above code snippet.
I've implemented a simple publisher/consumer set with MassTransit, and I want to have the consumers read the messages from the same queue. However, when I run it, I see a large portion of the messages sent to the error queue instead of being consumed. From the discussions I've seen (SO, Forum), this should be really really simple with RabbitMQ (just point to the same queue), but it's not working. Is there an additional configuration that should be set?
Here's My Publisher
public class YourMessage { public string Text { get; set; } }
public class Program
{
public static void Main()
{
Console.WriteLine("Publisher");
Bus.Initialize(sbc =>
{
sbc.UseRabbitMqRouting();
sbc.ReceiveFrom("rabbitmq://localhost/test_queue");
});
var x = Console.Read();
for (var i = 0; i <= 1000; i++)
{
Console.WriteLine("Message Number " + i);
Bus.Instance.Publish(new YourMessage { "Message Number " + i });
}
}
}
And My Consumer
public class YourMessage { public string Text { get; set; } }
public class Program
{
public static void Main()
{
Console.WriteLine("Consumer");
Bus.Initialize(sbc =>
{
sbc.UseRabbitMqRouting();
sbc.ReceiveFrom("rabbitmq://localhost/test_queue");
sbc.Subscribe(subs =>
{
var del = new Action<IConsumeContext<YourMessage>,YourMessage>((context, msg) =>
{
Console.WriteLine(msg.Text);
});
subs.Handler<YourMessage>(del);
});
});
while (true) { }
}
}
The receiver/consumer and publisher cannot be on the same queue. If you want competing consumers have multiple instances of the consumer running against the same queue.
We have documentation, but this section is currently lacking, so I understand your confusion: http://readthedocs.org/docs/masstransit/en/latest/configuration/gotchas.html#how-to-setup-a-competing-consumer If you succeed, help with the documentation would be wonderful.
So, it looks like the solution was to change the line in the publisher:
sbc.ReceiveFrom("rabbitmq://localhost/test_queue");
To something like:
sbc.ReceiveFrom("rabbitmq://localhost/test_queue_publisher");
This prevented the publishers from competing for messages they weren't configured to consume.
Just found an example on GitHub https://github.com/khalidabuhakmeh/MassTransit.ScaleOut ...
I'm using NServiceBus and I need to know how many clients are subscribed to a specific message type (even better the names of the subscribers). I'm talking in a pub\sub scenario.
Is it possible to get this information in NServiceBus?
Thx
You can pull this right out of your subscription storage. Either a query to the database or a .GetAllMessages() on the queue will get you a count and the subscribers address. If you are looking to do this in code, you could write a handler for the subscription message and count them up that way.
I have used ISubscriptionStorage with success.
public class SubscribersForMessageHandler :
IHandleMessages<SubscribersForMessageRequest>
{
public ISubscriptionStorage Storage { get; set; }
public IBus Bus { get; set; }
public void Handle(SubscribersForMessageRequest message)
{
Bus.Reply<SubscribersForMessageResponse>(m=>
{
m.SagaId = message.SagaId;
m.MessageType = message.MessageType;
m.SubscriberEndpoints = GetSubscribersForMessage(message.MessageType);
});
}
private List<string> GetSubscribersForMessage(string type)
{
return Storage.GetSubscribersForMessage(
new List<string> { type }).ToList();
}
}