NServiceBus: Trouble getting PubSub working with Distributor - nservicebus

I'm using NServiceBus 2.5 and trying to get a simplified version of the NServiceBus PubSub sample working with the distributor. The simplification:
1 Publisher
Distributor
1 Subscriber
All of this is on a single machine.
I would like to get this working before I move on to more complex things like multiple subscribers behind a single distributor, multiple machines, etc.
First I got the simplified pub sub example working without the distributor (i.e 1 pub and 1 sub - and I got that working fine).
As I understand it, the way it should work is:
The distributor defines its own control and data queues.
The publisher has it's control queue, but no other config refers to this queue. The publisher refers to the distributes data queue.
The subscriber refers to the distributors data and control queues.
I've not been able to get the simplified pub-sub distributor working with this config. I've done some debugging of the publisher and what I've found is that the publisher's Bus.Publish doesn't find any subscribers to publish to.
Any ideas what I'm doing wrong and what I need to do to get this working?
Here's configs for each of those:
Publisher:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="MsmqTransportConfig" type="NServiceBus.Config.MsmqTransportConfig, NServiceBus.Core" />
<section name="UnicastBusConfig" type="NServiceBus.Config.UnicastBusConfig, NServiceBus.Core" />
</configSections>
<MsmqTransportConfig InputQueue="MyPublisherInputQueue" ErrorQueue="error" NumberOfWorkerThreads="1" MaxRetries="5" />
<UnicastBusConfig>
<MessageEndpointMappings>
<add Messages="MyMessages" Endpoint="distributorDataBus" />
</MessageEndpointMappings>
</UnicastBusConfig>
</configuration>
Distributor
<add key="DataInputQueue" value="distributorDataBus"/>
<add key="ControlInputQueue" value="distributorControlBus"/>
<add key="ErrorQueue" value="error"/>
<add key="StorageQueue" value="distributorStorage"/>
<add key="NameSpace" value="http://www.UdiDahan.com"/>
<!-- relevant for a Serialization of "interfaces" or "xml" -->
<add key="Serialization" value="xml"/>
<!-- can be either "xml", or "binary" -->
</appSettings>
</configuration>
Subscriber
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="MsmqTransportConfig" type="NServiceBus.Config.MsmqTransportConfig, NServiceBus.Core" />
<section name="UnicastBusConfig" type="NServiceBus.Config.UnicastBusConfig, NServiceBus.Core" />
</configSections>
<MsmqTransportConfig
InputQueue="Subscriber1InputQueue_1"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"
/>
<UnicastBusConfig DistributorControlAddress="distributorControlBus" DistributorDataAddress="distributorDataBus">
<MessageEndpointMappings />
</UnicastBusConfig>
</configuration>

Ok, I finally got this working. The key clue came from here: http://tech.groups.yahoo.com/group/nservicebus/message/8525
Basically, the subscriber needs to be pointed to the publisher by adding this section:
<UnicastBusConfig DistributorControlAddress="distributorControlBus" DistributorDataAddress="distributorDataBus">
<MessageEndpointMappings>
<add Messages="MyMessages" Endpoint="MyPublisherInputQueue" />
</MessageEndpointMappings>
</UnicastBusConfig>
I initially thought that this is making the subscriber register directly with the publisher, but no. I tested that out with two different subscribers pointed to the same distributor and noted that only one of the subscribers gets any single event published.

Related

How to host the core project with IIS using mdf file

I have tried to host the core project with mdf file in the IIS server. The below webconfig file works fine in the localhost. But when I publish it to IIS, I got a 500-internal server error.
This is the webconfig:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.entit, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
</configSections>
<connectionStrings>
<add name="FileManagerConnection" connectionString="Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|App_Data\FileManager.mdf;Integrated Security=True;Connect Timeout=30" />
</connectionStrings>
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="1073741824"></requestLimits>
</requestFiltering>
</security>
<handlers>
<add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
</handlers>
<aspNetCore processPath="%LAUNCHER_PATH%" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" arguments="%LAUNCHER_ARGS%">
<environmentVariables />
</aspNetCore>
</system.webServer>
<entityFramework>
<defaultConnectionFactory type="System.Data.Entity.Infrastructure.SqlConnectionFactory, EntityFramework" />
<providers>
<provider invariantName="System.Data.SqlClient" type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
</providers>
</entityFramework>
</configuration>
LocalDB is a database instance provided by the Visual studio. In order to make it work in IIS properly, we should start the Visual Studio and change the application pool identity to the specific account that running the Visual studio. But this is not a common way to publish the database, we should add database immigration to the current project so that we publish the database to the remote database server.
Here is an example of database immigration from localdb to a remote SQL server.
How to transfer ASP.NET MVC Database from LocalDb to SQL Server?
Feel free to let me know if there is anything I can help with.
The .mdf files are SQL Server database files. No other application can understand, read, or update those files. So, I have installed the SQL server in my machine and then host the core project in IIS.

Where to configure NLog NHibernate bridge for logging sql configuration?

As a proof-of-concept I am using a console application wrapping a proxy of a WCF service. I execute a few calls through the proxy and write the results to the console. Underlying the WCF service is a Data Access Layer built on the Repository pattern using NHibernate as an ORM. I want to write the NHib-generated SQL to the console.
I have attempted to configure NLog using the NuGet package and guidance from here with no luck, but I suspect perhaps I don't properly understand where I need to perform these different configuration bits in my many-layered architecture. Here's an attempt at describing my layout:
Solution
|
-- WCF Service
|
-- DAL
| | (NHibernate is used here)
| -- hibernate.cfg.xml
| NLog.config
| App.config (r)
-- WCF Client (console app)
|
--App.config (c)
hibernate.cfg.xml contents:
<hibernate-configuration xmlns="urn:nhibernate-configuration-2.2">
<session-factory>
<property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
<property name="dialect">NHibernate.Dialect.MsSql2012Dialect</property>
<property name="connection.driver_class">NHibernate.Driver.SqlClientDriver</property>
<property name="show_sql">true</property>
<property name="connection.connection_string">Data Source=.\SQLEXPRESS;Initial Catalog=AdventureWorks2012;User ID=**;Password=********</property>
</session-factory>
</hibernate-configuration>
NLog.config contents:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="trace" type="Trace" />
</targets>
<rules>
<logger name="NHibernate.SQL" writeTo="trace" />
</rules>
</nlog>
App.config (r) contents:
<configuration>
<appSettings>
<add key="nhibernate-logger" value="NHibernate.NLogLoggerFactory, NHibernate.NLog" />
</appSettings>
</configuration>
App.config (c) contents:
<configuration>
<system.diagnostics>
<trace autoflush="true">
<listeners>
<add name="consoleListener" type="System.Diagnostics.ConsoleTraceListener" />
</listeners>
</trace>
</system.diagnostics>
<appSettings>
<add key="nhibernate-logger" value="NHibernate.NLogLoggerFactory, NHibernate.NLog" />
</appSettings>
</configuration>
My NUnit unit tests are displaying the SQL Statements, so the show-sql instruction in hibernate.cfg.xml is working correctly. If I insert a Trace.WriteLine() in my console app that shows up in the console, so (I expect) anything written to Trace ought to show up there. But I do not see any SQL in the console.
I'm assuming that I've placed some of my .config instructions in the wrong layers of my app, but where?

App.Config MessageEndpointMappings not working in NSB 3.3.8

In version 3.2.6 of NSB I had my endpoints mapped in my app config like this:
<UnicastBusConfig>
<MessageEndpointMappings>
<add Messages="Foo.Bar.Baz1, Foo.Interfaces" Endpoint="Foo.Bar.Monitor" />
<add Messages="Foo.Bar.Baz2, Foo.Interfaces" Endpoint="Foo.Bar.Monitor" />
<add Messages="Foo.Bar.Baz3, Foo.Interfaces" Endpoint="Foo.Bar.Monitor" />
</MessageEndpointMappings>
</UnicastBusConfig>
This worked fine. When I upgraded to NSB 3.3.8 I saw that the API for endpointmappings had changed, and that in particular, the Messages attribute was deprecated, so I tried to port my code to one of the new methods. None of them work. Here are the three different methods I tried:
<UnicastBusConfig>
<MessageEndpointMappings>
<add Assembly="Foo.Interfaces" Endpoint="Foo.Bar.Monitor" />
</MessageEndpointMappings>
</UnicastBusConfig>
<UnicastBusConfig>
<MessageEndpointMappings>
<add Assembly="Foo.Interfaces" Namespace="Foo.Bar" Endpoint="Foo.Bar.Monitor" />
</MessageEndpointMappings>
</UnicastBusConfig>
<UnicastBusConfig>
<MessageEndpointMappings>
<add Assembly="Foo.Interfaces" Type="Foo.Bar.Baz1" Endpoint="Foo.Bar.Monitor" />
<add Assembly="Foo.Interfaces" Type="Foo.Bar.Baz2" Endpoint="Foo.Bar.Monitor" />
<add Assembly="Foo.Interfaces" Type="Foo.Bar.Baz3" Endpoint="Foo.Bar.Monitor" />
</MessageEndpointMappings>
</UnicastBusConfig>
Using any of these methods, when I try to send a Baz1 I get an InvalidOperationException: No destination specified for message(s): Foo.Bar.Baz1.
It's worth noting that if I intentionally put in a typo in my type name (e.g. Foo.Bar.Bazzzzz1), NSB correctly faults during startup when it can't find the specified type, indicating that something must be happening to the ones that have no typos.
I can work around this issue by directly specifying my endpoint as a parameter to Send, but NSB recommends not doing that.
So my question is: Am I still doing something wrong in my configuration, or is this truly a bug in NSB 3.3.8?

Private NuGet Server: Request Entity Too Large

We have a internal NuGet server (ASP.net app using the NuGet.Server package) and we want to use it with Octopus to deploy packages. So the first thing you hit is that the packages are too large.
When you push a package larger than around 7 Meg you get:
Failed to process request. 'Request Entity Too Large'.
The remote server returned an error: (413) Request Entity Too Large..
Based on the documentation on Octopus, I updated the web.config file to have the changes.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<sectionGroup name="elmah">
<section name="security" requirePermission="false" type="Elmah.SecuritySectionHandler, Elmah"/>
<section name="errorLog" requirePermission="false" type="Elmah.ErrorLogSectionHandler, Elmah"/>
<section name="errorMail" requirePermission="false" type="Elmah.ErrorMailSectionHandler, Elmah"/>
<section name="errorFilter" requirePermission="false" type="Elmah.ErrorFilterSectionHandler, Elmah"/>
</sectionGroup>
</configSections>
<system.web>
<compilation debug="true" targetFramework="4.0"/>
<httpModules>
<add name="ErrorLog" type="Elmah.ErrorLogModule, Elmah"/>
<add name="ErrorMail" type="Elmah.ErrorMailModule, Elmah"/>
<add name="ErrorFilter" type="Elmah.ErrorFilterModule, Elmah"/>
</httpModules>
<httpRuntime maxRequestLength="419430400" executionTimeout="3600"/>
</system.web>
<system.webServer>
<validation validateIntegratedModeConfiguration="false"/>
<modules runAllManagedModulesForAllRequests="true">
<add name="ErrorLog" type="Elmah.ErrorLogModule, Elmah" preCondition="managedHandler"/>
<add name="ErrorMail" type="Elmah.ErrorMailModule, Elmah" preCondition="managedHandler"/>
<add name="ErrorFilter" type="Elmah.ErrorFilterModule, Elmah" preCondition="managedHandler"/>
</modules>
<staticContent>
<mimeMap fileExtension=".nupkg" mimeType="application/zip"/>
</staticContent>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="419430400"/>
</requestFiltering>
</security>
</system.webServer>
<elmah>
<security allowRemoteAccess="false"/>
<errorLog type="Elmah.XmlFileErrorLog, Elmah" logPath="~/App_Data"/>
</elmah>
<location path="elmah.axd" inheritInChildApplications="false">
<system.web>
<httpHandlers>
<add verb="POST,GET,HEAD" path="elmah.axd" type="Elmah.ErrorLogPageFactory, Elmah"/>
</httpHandlers>
</system.web>
<system.webServer>
<handlers>
<add name="ELMAH" verb="POST,GET,HEAD" path="elmah.axd" type="Elmah.ErrorLogPageFactory, Elmah" preCondition="integratedMode"/>
</handlers>
</system.webServer>
</location>
<appSettings>
<add key="apiKey" value="KeyHere"/>
<add key="packagesPath" value=""/>
</appSettings>
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>
</system.serviceModel>
</configuration>
That does not work. Other posts talk about running something like (IIS7):
appcmd.exe set config -section:system.webServer/serverRuntime /uploadReadAheadSize:"419430400" /commit:apphost
or (IIS6):
cscript adsutil.vbs set w3svc/1/uploadreadaheadsize 419430400
I tried both to no avail. Neither command returned an error, so I assume that the value '419430400' is correct for all of the calls (bytes vs. some other unit of size).
Anyone have any idea what I am missing?
I ended up just copying the package to a share on the web server, but I would really like the push command to work.
Not exactly answering the OP's question, but related to the topic, I was getting the (413) Request Entity Too Large error while using NuGet push to push to a local SymbolSource server - turned out I was submitting to a slightly incorrect URL, once I corrected the command to point to the base /NuGet/ URL, it ran just fine.
No idea why an incorrect URL results in the 413 error, but there you go. Hope this helps someone.
EDIT: based on comments below, you may have more luck just referencing the base http://www.myserver.com/ URL rather than including the /NuGet as well. Worth playing around a bit.
I know this is an old question, but today I was faced with the same error. It's worth noticing that I'm using TeamCity package building and publishing. Anyway, when I try to Publish my huge package (about 200 MB) I was blocked with this. The solution was simple:
Instead of publishing to http://mynugetserver/api/v2/, use: http://mynugetserver/
You'll have to set these guys to higher values:
system.web - httpRuntime - maxRequestLength to, say, 1048576
system.webserver - security - requestFiltering - requestLimits -
maxAllowedContentLength to, say 1073741824
Both values are in a different unit so the second one should be larger than the first.
Also, have a look at www.myget.org which I found great when working with Octopus Deploy.
Check your serverRuntime configuration.
The maxRequestEntityAllowed and uploadReadAheadSize attributes respectively configure limits for the maximum number of bytes allowed in the entity body of a request and the number of bytes a Web server will read into a buffer and pass to an ISAPI extension.
More details: http://www.iis.net/configreference/system.webserver/serverruntime
My guess is that you are using SSL and setting uploadReadAheadSize will solve the issue. Because during client renegotiation process,the request entity body must be preloaded using SSL preload. SSL preload will use the value of the uploadReadAheadSize property, which is used for ISAPI extensions.
Here are the defaults
<location path="Default Web Site">
<system.webServer>
<serverRuntime enabled="true"
uploadReadAheadSize="49152"
maxRequestEntityAllowed="4294967295" />
</system.webServer>
</location>
Based on #Keith and #Nubigetter's answers, I did some further research, because the behavior seemed really weird to me.
The answer is actually in the documentation for Nuget.Server (if you look very carefully), it's just not very obvious:
use http://mynugetserver/nuget for list/restore
use http://mynugetserver/ for push
I've raised this with the Nuget team here https://github.com/NuGet/NuGetGallery/issues/2903 because I regard this behavior as 'presenting opportunity for improvement'.
This is due to nginx limitations, the nuget server in Linux system use nginx as proxy and config file under:
/etc/nginx/conf.d/nuget.conf
server_name localhost;
root /var/www/public/;
client_max_body_size 200M;
change client_max_body_size 200M is working for me.
I had the same issue:
[Step 1/2] Publishing package to feed at http://localhost/OctopusDeploy/nuget/packages...
[Step 1/2] Failed to push to package feed at 'http://localhost/OctopusDeploy/nuget/packages/'.
[Step 1/2] (The server responded with: [413] Request Entity Too Large)
[Step 1/2] Process exited with code 1
but the Octopus Deploy service had stopped!
What worked for me was in this article:
http://blogs.blackmarble.co.uk/blogs/rfennell/post/2012/10/31/403-and-413-errors-when-publishing-to-a-local-Nuget-Server.aspx
"Important: This second error was a red herring, you don't need the /nuget on the end of the URL"

Forwarding Messages to Remote Endpoint in Nservicebus

In my message publisher configuration I have
<MsmqTransportConfig
InputQueue="EnformMessages"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"
/>
<UnicastBusConfig ForwardReceivedMessagesTo="testqueue#cgy1-web01">
<MessageEndpointMappings>
<!-- publishers don't need to set this for their own message types -->
</MessageEndpointMappings>
</UnicastBusConfig>
which I was hoping would copy the messages published to EnformMessages to a queue on a remote machine. No messages ever seem to be sent to the remote machine although messages are certainly being received locally. The remote listener's config file looks like
<MsmqTransportConfig
InputQueue="worker"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"
/>
<UnicastBusConfig>
<MessageEndpointMappings>
<add Messages="EnformMessages" Endpoint="testqueue" />
</MessageEndpointMappings>
</UnicastBusConfig>
I also tried using the distributor in the fashion described at http://www.candland.net/blog/2009/06/08/NServiceBusDistributorOverview.aspx. So my publisher configuration looked like
<MsmqTransportConfig
InputQueue="client"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"
/>
<UnicastBusConfig
DistributorControlAddress=""
DistributorDataAddress=""
ForwardReceivedMessagesTo="">
<MessageEndpointMappings>
<!-- publishers don't need to set this for their own message types -->
<add Messages="EnformMessages" Endpoint="distributordatabus#cgy1-web01" />
</MessageEndpointMappings>
</UnicastBusConfig>
Subscriber configuration like
<MsmqTransportConfig
InputQueue="EnformMessages"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"
/>
<UnicastBusConfig
DistributorControlAddress="distributorcontrolbus#cgy1-web01"
DistributorDataAddress="distributordatabus#cgy1-web01">
<MessageEndpointMappings>
<!--<add Messages="EnformMessages" Endpoint="EnformMessages" />-->
</MessageEndpointMappings>
</UnicastBusConfig>
and distributor like
<appSettings>
<add key="NumberOfWorkerThreads" value="1"/>
<add key="DataInputQueue" value="distributorDataBus"/>
<add key="ControlInputQueue" value="distributorControlBus"/>
<add key="ErrorQueue" value="error"/>
<add key="StorageQueue" value="distributorStorage"/>
<add key="NameSpace" value="http://www.UdiDahan.com"/>
<!-- relevant for a Serialization of "interfaces" or "xml" -->
<add key="Serialization" value="xml"/>
<!-- can be either "xml", or "binary" -->
</appSettings>
<MsmqTransportConfig
InputQueue="distributorControlBus"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"
/>
<UnicastBusConfig >
<MessageEndpointMappings >
<add Messages="EnformMessages" Endpoint="EnformMessages" />
</MessageEndpointMappings>
</UnicastBusConfig>
No messages seem to arrive. In fact nothing is printed out by the distributor at all. I added a logging section to the config file in the hopes it would produce some output but got nothing.
Nservicebus 2.0.0.768
In a pub/sub scenario, it is the subscriber that will be forwarding messages to an audit queue, not the publisher. Also, you've told your subscriber that its publisher is "testqueue" but you specified your publisher's input queue as "EnformMessages". These two queues need to match up.