we have a WCF service hosted inside IIS. Now there are loads of different client applications calling this service. WS-SecureConversion is used.
Now, the service diagnostic log shows warnings that security sessions are being aborted. Most likely this is because of clients that do not properly close the session.
More info: the problem were "pending" security sessions. Those are sessions that were never used, only opened. This is pretty annoying as you can have a maximum of 128 such pending sessions before your services starts barfing 500s.
This can be easily reproduced (see answer below). I was able to query 128 SessionInitiationMessageHandlers using WinDbg. So this might be a good measure to identify this scenario.
Still, a way to identify those "misbehaving" clients would be useful.
Regards,
Alex
Since client and server share nothing but messages going between them, there's not much you can really do.
On the server side, you could look at some bits of information being sent from the client - check out the OperationContext.Current property in your service method - see the MSDN documentation on OperationContext about details what exactly is provided.
So you might be able to log certain information to identify the "offending" clients.
Marc
Sweet....the best way to kill a WCF service with a secure conversion seems to be to do nothing.
ServicePointManager.ServerCertificateValidationCallback += delegate { return true; };
var client = new MyClient();
client.ClientCredentials.UserName.UserName = "user";
client.ClientCredentials.UserName.Password = "password";
while(true)
{
Console.WriteLine("OPEN");
var c = client.ChannelFactory.CreateChannel();
((ICommunicationObject)c).Open();
// If I comment the following line, the service throws a
// "Too busy, too many pending sessions" 500.
var request = new MyRequest { };
c.Do(request);
// Of course I did *not* comment this line
((ICommunicationObject)c).Close();
}
Meanwhile, this bug has been confirmed by MS but still remains in .NET 4.x even if MS says otherwise:
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=499859
Related
We recently switched our azure functions durable functions based app from a dedicated s1/standard app service plan to dynamic y1 plan to same money and now we are getting a common error:
"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."
this happens after about an hour of the app running. The exceptions comes from a svcutil generated wcf client. I'm fairly certain this is related to the limitation of socket connections from a consumption function app vs a "dedicated" app plan as described at https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale#service-limits but not totally convinced because i do NOT see the log message "Host thresholds exceeded: Connections" listed at https://learn.microsoft.com/en-us/azure/azure-functions/manage-connections#connection-limit
our client is actually a wrapper around a dozen wcf clients instantiated on our wrappers construction. the wrapper is registed with di as a singleton
builder.Services.AddSingleton<IWrapperClient, OurSoapClient>();
public OurSoapClient(
IMemoryCache memoryCache,
IOptions<Options> options,
ILogger<OurSoapClient> log
)
{
this.options = options.Value;
this.memoryCache = memoryCache;
this.log = log;
this.metaClient = new Meta.MetaWebServiceClient(
Meta.MetaWebServiceClient.EndpointConfiguration.MetaWebServicePort,
this.options.MetaHref
);
this.wmsClient = new Wms.WmsWebServiceClient(
Wms.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsStageItemsClient = new Wms.Stage.Items.WmsWebServiceClient(
Wms.Stage.Items.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsReceiptClient = new Wms.Stage.ExpectedReceipts.WmsWebServiceClient(
Wms.Stage.ExpectedReceipts.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsStageRmaClient = new Wms.Stage.Rma.WmsWebServiceClient(
Wms.Stage.Rma.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsStageShipmentsClient = new Wms.Stage.Shipments.WmsWebServiceClient(
Wms.Stage.Shipments.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsUpdateShipmentsClient = new Wms.Updates.ShippingResults.WmsWebServiceClient(
Wms.Updates.ShippingResults.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsUpdatesReceivingResultsClient = new Wms.Updates.ReceivingResults.WmsWebServiceClient(
Wms.Updates.ReceivingResults.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsUpdatesInventoryAdjustmentClient = new Wms.Updates.InventoryAdjustments.WmsWebServiceClient(
Wms.Updates.InventoryAdjustments.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsInboundOrderClient = new Wms.Inbound.CurrentAndHistory.WmsWebServiceClient(
Wms.Inbound.CurrentAndHistory.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsOutboundOrderClient = new Wms.Outbound.CurrentAndHistory.WmsWebServiceClient(
Wms.Outbound.CurrentAndHistory.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsInboundOrderDetailsClient = new Wms.Inbound.CurrentAndHistoryDetails.WmsWebServiceClient(
Wms.Inbound.CurrentAndHistoryDetails.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
this.wmsOutboundOrderDetailsClient = new Wms.Outbound.CurrentAndHistoryDetails.WmsWebServiceClient(
Wms.Outbound.CurrentAndHistoryDetails.WmsWebServiceClient.EndpointConfiguration.WmsWebServicePort,
this.options.WmsHref
);
}
switching back to standard app service plan seems to make this go away.
i'm fairly certain durable functions isn't a cause here, but just to be clear all the calls to the client happen from Orchestrator or Activity functions...we see the same failure errors in both function types.
One anecdote i've noticed repeated is the errors seem to occur just after a second OurWrapperClient is instantiated (which instantiates all the wcf clients again)...since it's a singleton this must be the azure functions control plane spinning up another instance of my app
so a couple of questions:
any idea how to prove this is max outbound connections related issue?
any suggestions for reasons why this becomes a problem
assuming this is related to WCF
what's the correct way to use wcf clients, should they be instantiated for each call with usings, or is it ok to instantiate them once per wrapper client as we have and then dispose them only once?
should we instantiate them as singletons with the DI and then inject them instead? This means DI would call Dispose on them i believe
is there any way to pass the http client to be used to the wcf client generated code? a lot of the azure functions best practices say to have a single injected http client for all your http I/O, but i don't see how to do that with WCF.
Using app insights i noticed that the takes about an hour thing corresponded to my app switching host instances around that time. eventually i started to see that on deploys it would fail right away..ie got a "bad" host. opened up a MS support case they remoted into a bad isntance and found they could not TCP ping from that host.
Each webspace you are assigned makes requests from a pool of IPs, i suspect my targets WAF was blocking some of these IPs for whatever reason. Switching to a new region which guaranteed a new webspace (they're assigned on created, but are region specific) made the problem go away.
did find https://github.com/dotnet/runtime/issues/35508 during this which seemed similar
I am trying to use HttpContext.Session in my ASP.NET Core Blazor Server application (as described in this MS Doc, I mean: all correctly set up in startup)
Here is the code part when I try to set a value:
var session = _contextAccessor.HttpContext?.Session;
if (session != null && session.IsAvailable)
{
session.Set(key, data);
await session.CommitAsync();
}
When this code called in Razor component's OnAfterRenderAsync the session.Set throws following exception:
The session cannot be established after the response has started.
I (probably) understand the message, but this renders the Session infrastructure pretty unusable: the application needs to access its state in every phase of the execution...
Question
Should I forget completely the DistributedSession infrastructure, and go for Cookies, or Browser SessionStorage? ...or is there a workaround here still utilizing HttpContext.Session? I would not want to just drop the distributed session infra for a way lower level implementation...
(just for the record: Browser's Session Storage is NOT across tabs, which is a pain)
Blazor is fundamentally incompatible with the concept of traditional server-side sessions, especially in the client-side or WebAssembly hosting model where there is no server-side to begin with. Even in the "server-side" hosting model, though, communication with the server is over websockets. There's only one initial request. Server-side sessions require a cookie which must be sent to the client when the session is established, which means the only point you could do that is on the first load. Afterwards, there's no further requests, and thus no opportunity to establish a session.
The docs give guidance on how to maintain state in a Blazor app. For the closest thing to traditional server-side sessions, you're looking at using the browser's sessionStorage.
Note: I know this answer is a little old, but I use sessions with WebSockets just fine, and I wanted to share my findings.
Answer
I think this Session.Set() error that you're describing is a bug, since Session.Get() works just fine even after the response has started, but Session.Set() doesn't. Regardless, the workaround (or "hack" if you will) includes making a throwaway call to Session.Set() to "prime" the session for future writing. Just find a line of code in your application where you KNOW the response hasn't sent, and insert a throwaway call to Session.Set() there. Then you will be able to make subsequent calls to Session.Set() with no error, including ones after the response has started, inside your OnInitializedAsync() method. You can check if the response is started by checking the property HttpContext.Response.HasStarted.
Try adding this app.Use() snippet into your Startup.cs Configure() method. Try to ensure the line is placed somewhere before app.UseRouting():
...
...
app.UseHttpsRedirection();
app.UseStaticFiles();
//begin Set() hack
app.Use(async delegate (HttpContext Context, Func<Task> Next)
{
//this throwaway session variable will "prime" the Set() method
//to allow it to be called after the response has started
var TempKey = Guid.NewGuid().ToString(); //create a random key
Context.Session.Set(TempKey, Array.Empty<byte>()); //set the throwaway session variable
Context.Session.Remove(TempKey); //remove the throwaway session variable
await Next(); //continue on with the request
});
//end Set() hack
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapBlazorHub();
endpoints.MapFallbackToPage("/_Host");
});
...
...
Background Info
The info I can share here is not Blazor specific, but will help you pinpoint what's happening in your setup, as I've come across the same error myself. The error occurs when BOTH of the following criteria are met simultaneously:
Criteria 1. A request is sent to the server with no session cookie, or the included session cookie is invalid/expired.
Criteria 2. The request in Criteria 1 makes a call to Session.Set() after the response has started. In other words, if the property HttpContext.Response.HasStarted is true, and Session.Set() is called, the exception will be thrown.
Important: If Criteria 1 is not met, then calling Session.Set() after the response has started will NOT cause the error.
That is why the error only seems to happen upon first load of a page--it's because often in first loads, there is no session cookie that the server can use (or the one that was provided is invalid or too old), and the server has to spin up a new session data store (I don't know why it has to spin up a new one for Set(), that's why I say I think this is a bug). If the server has to spin up a new session data store, it does so upon the first call to Session.Set(), and new session data stores cannot be spun up after the response has started. On the other hand, if the session cookie provided was a valid one, then no new data store needs to be spun up, and thus you can call Session.Set() anytime you want, including after the response has started.
What you need to do, is make a preliminary call to Session.Set() before the response gets started, so that the session data store gets spun up, and then your call to Session.Set() won't cause the error.
SessionStorege has more space than cookies.
Syncing (two ways!) the sessionStorage is impossible correctly
I think you are thinking that if it is on the browser, how can you access that in C#? Please see some examples. It actually read from the browser and transfers (use) on the server side.
sessionstorage and localstorage in blazor are encrypted. We do not need to do extra for encryption. The same applies for serialization.
I have a server/client WCF app with calls from client to server and callbacks from server to client.
I'm getting some kind of error when I try to do the callback.
But the info I get is useless, so they give you the tip to set the exception Detail In Faults to true to see more details.
I have this but I dont know how to include other behaviour since only 1 is allowed.
ServiceMetadataBehavior smb1 = host.Description.Behaviors.Find<ServiceMetadataBehavior>();
// If not, add one
if (smb1 == null)
smb1 = new ServiceMetadataBehavior();
smb1.HttpGetEnabled = true;
host.Description.Behaviors.Add(smb1);
I also want to let know the way I'm using the callbacks.
When the client calls the server, my function in the server does all the stuff I need and saves the channel:
callbacks = OperationContext.Current.GetCallbackChannel<IServiceCallbackContract>();
And then when I need to call the client from the server, I use that channel. BUT NOT IN THAT FUNCTION!
I thought this couldn't be done but I saw it in some examples.
So maybe I'm doing something wrong there too and you only can do the callback within the function in the server called from the client and which interface specifies the interface callback contract.
Finally got it working that way, just saved the channel and answered when I need in other function.
I do asynchronous requests in LoadState method of a certain Page. I use HttpClient to make a request and I expect the splashscreen to go away while I await the result.
If I am not connected to any networks, the splashscreen immediately goes away and I get a blank page because the request obviously didn't happen.
But if I am connected to a network but have connectivity issues (for example, I set a wrong IP address) it seems to start a request and just block.
My expectation was that the HttpClient would realize that it cannot send a request and either throw an exception or just return something.
I managed to solve the issue of blocking by setting a timeout of around 800 milliseconds, but now it doesn't work properly when the Internet connection is ok. Is this the best solution, should I be setting the timeout at all? What is the timeout that's appropriate which would enable me to differentiate between an indefinitely blocking call and a proper call that's just on a slower network?
I could perhaps check for Internet connectivity before each request, but that sounds like an unpredictable solution...
EDIT: Now, it's really interesting. I have tried again, and it blocks at this point:
var rd = await httpClient.SendAsync(requestMsg);
If I use Task.Run() as suggested in the comments and get a new Thread, then it's always fine.
BUT it's also fine without Task.Run() if there is no Internet access but the network access is not "Limited" (it says that the IPv4 connectivity is "Internet access" although I cannot open a single website in a browser and no data is returned from the web service. It just throws System.Net.Http.HttpRequestException which was something I was expecting in the first place) Only blocks when the network connection is Limited.
What if instead of setting a timeout, you checked the connection status using
public static bool IsConnected
{
get
{
return NetworkInformation.GetInternetConnectionProfile() != null;
}
}
This way if IsConnected, then you make the call; otherwise, ignore it.
I'm not sure if you are running this in App.xaml.cs? I've found requests made in that class can be fickle and it may be best to move the functionality to an extended splash screen to ensure the application makes it all the way through the activation process.
http://msdn.microsoft.com/en-us/library/windows/apps/xaml/Hh868191(v=win.10).aspx
I have a Dispatch MessageInspector which is deserializing a SAML Token contained in the SOAP message header.
To do the deserialization I am using a variation of the following code:
List<SecurityToken> tokens = new List<SecurityToken>();
tokens.Add(new X509SecurityToken(CertificateUtility.GetCertificate()));
SecurityTokenResolver outOfBandTokenResolver = SecurityTokenResolver.CreateDefaultSecurityTokenResolver(new ReadOnlyCollection<SecurityToken>(tokens), true);
SecurityToken token = WSSecurityTokenSerializer.DefaultInstance.ReadToken(xr, outOfBandTokenResolver);
The problem I am seeing is that the performance of the ReadToken call varies depending on the account that is running the windows service (in which the WCF service is hosted).
If the service is running as a windows domain account the elapsed time for the ReadToken call is virtually zero. When running as a local machine account the call takes between 200 and 1000 milliseconds.
Can anyone shed any light on what is going on here and why the account running this bit of code makes a difference as to its performance?
Thanks,
Martin
When the service is running under a local account that there is considerably more activity taking place, examples of this are :
Accessing and using C:\WINDOWS\system32\certcli.dll
Accessing and using C:\WINDOWS\system32\atl.dll
Attempting to access registry keys e.g.
HKLM\SYSTEM\CurrentControlSet\Services\CertSvc\Configuration
None of this extra activity appears to occur when running under a domain account.
A quick search on the internet for "certcli.dll domain user" brings up microsoft knowledge base article 948080 which sounds similar.
Unsure how to resolve this as ultimately a .Net method is being called (WSSecurityTokenSerializer.ReadToken) where you have little to no control over the internals.
This appears to also describe the same problem :
http://groups.google.com/group/microsoft.public.biztalk.general/browse_thread/thread/402a159810661bf6?pli=1