NServiceBus 5 using AzureStoragePersistence results in "Failed to fetch timeouts from the timeout storage" on machines other then development machine - nservicebus

Question

I try to use Azure Table Storage for the persistence of timeout data and I experience an error on environments other than my local development machine.
My local machine is creating the timeout tables on Azure and is able to poll timeout data successfully. But, if I host the same software on premise on another server it failed to fetch the timeouts. I receive the following error:
2015-02-12 09:43:50,638 [10] WARN NServiceBus.Timeout.Hosting.Windows.TimeoutPersisterReceiver - Failed to fetch timeouts from the timeout storage
System.NullReferenceException: Object reference not set to an instance of an object.
at NServiceBus.Timeout.Hosting.Windows.TimeoutPersisterReceiver.Poll(Object obj) in c:\BuildAgent\work\1b05a2fea6e4cd32\src\NServiceBus.Core\Timeout\Hosting\Windows\TimeoutPersisterReceiver.cs:line 88
at System.Threading.Tasks.Task.Execute()
It seems that the TimeoutPersister is null at the point it wants to fetch data from it.
I host NServiceBus using the NServiceBus.Host. My endpoint configuration looks like this:
public class EndpointConfig : IConfigureThisEndpoint, AsA_Server
{
public void Customize(BusConfiguration configuration)
{
configuration.UsePersistence<AzureStoragePersistence>();
configuration.EndpointName("MyEndpoint");
configuration.UseTransport<RabbitMQTransport>()
.DisableCallbackReceiver();
configuration.DisableFeature<Sagas>();
configuration.ScaleOut().UseSingleBrokerQueue();();
}
}
And my app.config contains:
<connectionStrings>
<add name="NServiceBus/Transport" connectionString="host=myrabbitmqserver;virtualhost=myhost;username=me;password=secret" />
</connectionStrings>
<AzureTimeoutPersisterConfig ConnectionString="DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=myaccouuntkey;" TimeoutManagerDataTableName="TimeoutManager" TimeoutDataTableName="TimeoutData" />
Does anyone have any idea what I am doing wrong or can anyone point me in the right direction investigating what the problem can be?
Update 1
It seems that the NServiceBus.Azure assembly is not loaded on the other machines. So azure persistence features are not initialized resulting in NullReferenceException when using the TimeoutPersister.
Update 2 After some NServiceBus debugging I noticed that an exception was thrown when extracting the types from the NServiceBus.Azure.dll assembly. It is unable to load the referenced assembly Miscrosoft.Data.Services.Client.dll 5.6.0.0. This assembly is indeed not in the bin folder. The present version is 5.6.3.0. The NServiceBus.Azure NuGet package supports versions >= 5.6.0.0 < 6.0.0.0, but somehow it's still expecting version 5.6.0.0. It still feels weird that it is working on my development machine? Maybe there are some old versions of the Microsoft.Data.Services.Client.dll installed on my machine as part of the Azure SDK, which are found during the assembly loading.
Update 3
I indeed had somewhere at my system the older 5.6.0 version available. Downgrading the Microsoft.Data.xxx packages to version 5.6.0 solved the issue for now. Does anyone have the same issues using 5.6.3 versions and found a solution for that?
Update 4
Since 2015-02-13 a new version of NServiceBus.Azure is released and now it requires Microsoft.Data.Services.Client version 5.6.2.0. I am still not able to use the 5.6.3 version. Adding a assembly binding redirect will not help either.

Answer 1

The binding redirect must be added to the NServiceBus.Host.exe.config instead of the app.config. Pretty annoying because visual studio automatically updates the app.config.
Information from Yves Goeleven:
The reason is that default load behavior of the CLR is at the process level, which includes the host config, once the host is loaded we actively switch over to the appdomain level (using topshelf) from then on it uses the endpoint's config...
But if the CLR needs to resolve the reference prior to the switch to the appdomain, you will have to put the redirect at the host level (and at the endpoint level I guess)

Answer 2

It should work with 5.6.3 version. Try adding assembly bindingRedirect in the following way:
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="Microsoft.Data.Services.Client" publicKeyToken="31bf3856ad364e35" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-5.6.0.0" newVersion="5.6.0.0" />
</dependentAssembly>