I have occasional exception when reading / storing objects with Amazon S3 from asp.net application.
Exception says:
Unable to read data from the transport connection: A blocking operation was interrupted by a call to WSACancelBlockingCall. A blocking operation was interrupted by a call to WSACancelBlockingCall
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
Any ideas about how I can locate what is reason of this error?
here is my read function:
public Image GetImage(string fullKey, out string errorMessage)
{
errorMessage = null;
try
{
GetObjectResponse response = s3Client.GetObject(new GetObjectRequest()
{
BucketName = BucketName,
Key = fullKey,
Timeout = ImportTimeout
});
return Image.FromStream(response.ResponseStream);
}
catch (Exception e)
{
errorMessage = e.Message;
Log.Current.WriteError(e);
}
return null;
}
s3Client is initialised in constructor of this class:
s3Client = AWSClientFactory.CreateAmazonS3Client(AWSAccessKey, AWSSecretKey);
According to aws documentation it is correct to wrap call to GetObject with using:
http://docs.amazonwebservices.com/sdkfornet/latest/apidocs/html/M_Amazon_S3_AmazonS3_GetObject.htm
This eliminates all exceptions related to GetObject method
Related
I am having a issue with connect to CloudAMQP. That is the code and error
copy from the cloudamqp guideline.
void Start()
{
// I replaced the url that I got from cloudamqp
private static readonly string _url = "amqp://guest:guest#localhost/%2f";
// Create a ConnectionFactory and set the Uri to the CloudAMQP url
// the connectionfactory is stateless and can safetly be a static resource in your app
var factory = new ConnectionFactory
{
uri = new Uri(url)
};
// create a connection and open a channel, dispose them when done
using var connection = factory.CreateConnection();
using var channel = connection.CreateModel();
// ensure that the queue exists before we publish to it
var queueName = "queue1";
bool durable = false;
bool exclusive = false;
bool autoDelete = true;
channel.QueueDeclare(queueName, durable, exclusive, autoDelete, null);
// read message from input
var message = "hello";
// the data put on the queue must be a byte array
var data = Encoding.UTF8.GetBytes(message);
// publish to the "default exchange", with the queue name as the routing key
var exchangeName = "";
var routingKey = queueName;
channel.BasicPublish(exchangeName, routingKey, null, data);
}
the issues is
SocketException: An existing connection was forcibly closed by the remote host.
System.Net.Sockets.Socket.Receive (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Net.Sockets.SocketFlags socketFlags) (at :0)
System.Net.Sockets.NetworkStream.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 size) (at :0)
Rethrow as IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
does any one know how to slove it?
We get an occasional TaskCanceledException on a call that, according to our logs, completes well inside the timeout we configure for the request. The first log entry is from the server. This is the last thing the method logs out before returning a JsonResult (MVC 4 Controller).
{
"TimeGenerated": "2021-03-19T12:08:48.882Z",
"CorrelationId": "b1568096-fdbd-46a7-8b69-58d0b33f458c",
"date_s": "2021-03-19",
"time_s": "07:08:37.9582",
"callsite_s": "...ImportDatasets",
"stacktrace_s": "",
"Level": "INFO",
"class_s": "...ReportConfigController",
"Message": "Some uninteresting message",
"exception_s": ""
}
In this case, the request took about 5 minutes to complete. Then 30 minutes later, our caller throws the task canceled exception from HttpClient.SendAsync:
{
"TimeGenerated": "2021-03-19T12:48:27.783Z",
"CorrelationId": "b1568096-fdbd-46a7-8b69-58d0b33f458c",
"date_s": "2021-03-19",
"time_s": "12:48:17.5354",
"callsite_s": "...AuthorizedApiAccessor+<CallApi>d__29.MoveNext",
"stacktrace_s": "TaskCanceledException
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)\r\n
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)\r\n
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)\r\n
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)\r\n
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)\r\n
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)\r\n
at ...AuthorizedApiAccessor.CallApi(String url, Object content, HttpMethod httpMethod, AuthenticationType authType, Boolean isCompressed)\r\nIOException
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)\r\n
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)\r\n
at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial)\r\n
at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)\r\n
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)\r\nSocketException",
"Level": "ERROR",
"class_s": "...AuthorizedApiAccessor",
"Message": "Nothing good",
"exception_s": "The operation was canceled."
}
Given that within the process that makes the request we block an async call (.Result -- hitting a brownfield caching implementation that doesn't support async), my first guess was that we had a deadlock as described by Stephen Cleary. But the caller is a dotnetcore 3.1 application, so that kind of deadlock is not possible.
Our usage of HttpClient is pretty standard, I think. This is the method that ultimately makes the call:
private async Task<string> CallApi(string url, object content, HttpMethod httpMethod, AuthenticationType authType, bool isCompressed)
{
try
{
var request = new HttpRequestMessage()
{
RequestUri = new Uri(url),
Method = httpMethod,
Content = GetContent(content, isCompressed)
};
AddRequestHeaders(request);
var httpClient = _httpClientFactory.CreateClient(HTTPCLIENT_NAME);
httpClient.Timeout = Timeout;
AddAuthenticationHeaders(httpClient, authType);
var resp = await httpClient.SendAsync(request);
var responseString = await (resp.Content?.ReadAsStringAsync() ?? Task.FromResult<string>(string.Empty));
if (!resp.IsSuccessStatusCode)
{
var message = $"{url}: {httpMethod}: {authType}: {isCompressed}: {responseString}";
if (resp.StatusCode == HttpStatusCode.Forbidden || resp.StatusCode == HttpStatusCode.Unauthorized)
{
throw new CustomException(message, ErrorType.AccessViolation);
}
if (resp.StatusCode == HttpStatusCode.NotFound)
{
throw new CustomException(message, ErrorType.NotFound);
}
throw new CustomException(message);
}
return responseString;
}
catch (CustomException) { throw; }
catch (Exception ex)
{
var message = "{Url}: {HttpVerb}: {AuthType}: {IsCompressed}: {Message}";
_logger.ErrorFormat(message, ex, url, httpMethod, authType, isCompressed, ex.Message);
throw;
}
}
We are at a loss for theories on this behavior. We've seen the task cancelation between 3-5 times a month within a few hundred successful requests, so it's intermittent, but far from rare.
Where else should we be looking for the source of what behaves like a deadlock?
update
Might be relevant to note that we are using the standard HttpClientHandler. The retry policies have been added recently, but we don't retry on a long-running POST, which is the scenario above.
builder.Services.AddHttpClient(AuthorizedApiAccessor.HTTPCLIENT_NAME)
.ConfigurePrimaryHttpMessageHandler(_ => new HttpClientHandler()
{
AutomaticDecompression = System.Net.DecompressionMethods.Deflate | System.Net.DecompressionMethods.GZip
})
.AddRetryPolicies(retryOptions);
I am trying to send a large file using FileStreamResult -
return new UnbufferedFileStreamResult(new FileStream(apiResponse.url, FileMode.Open, FileAccess.Read), "text/csv") { FileDownloadName = new FileInfo(apiResponse.url).Name };
The UnbufferedFileStreamResult is:
public class UnbufferedFileStreamResult : FileStreamResult {
public UnbufferedFileStreamResult(Stream fileStream, string contentType) : base(fileStream, contentType) {
}
public override void ExecuteResult(ActionContext context) {
context.HttpContext.DisableOutputBuffering();
base.ExecuteResult(context);
}
}
This seems to work for files up to a certain size, but if they get too big I get the following exception:
System.IO.IOException: Stream was too long.
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.MemoryStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Http.Extensions.StreamCopyOperation.CopyToAsync(Stream source, Stream destination, Nullable`1 count, Int32 bufferSize, CancellationToken cancel)
at Microsoft.AspNetCore.Mvc.Infrastructure.FileResultExecutorBase.WriteFileAsync(HttpContext context, Stream fileStream, RangeItemHeaderValue range, Int64 rangeLength)
at Microsoft.AspNetCore.Mvc.Infrastructure.FileStreamResultExecutor.ExecuteAsync(ActionContext context, FileStreamResult result)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultAsync(IActionResult result)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResultExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, O
So it appears that FileStreamResultExecutorBase is copying my file stream into a memory stream before it goes to the http output stream. My questions are 1) why is it doing this, 2) how can I prevent this behavior, and 3) are there implementations of results like FileStreamResult that can read directly from the input stream to the output stream without copying to memory first?
I have a WCF service application that is supposed to connect to another server X to retrieve data. When server X is configured with one-way SSL, the WCF service application is able to connect to it without problem. But when server X is configured with two-way SSL, the WCF service application will have problem connecting to it.
The WCF service application uses a self-signed certificate. This certificate was exported without private key into a .cer file, and imported into server X's trust store. The same exported certificate was used in the WCF service application's codes:
string URL = "https://11.205.112.49:5311/Ping";
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create(URL);
X509Certificate2 Certificate = new X509Certificate2("F:\\cert.cer");
Request.ClientCertificates.Add(Certificate);
Request.Method = "GET";
HttpWebResponse Response = (HttpWebResponse)Request.GetResponse();
The last line of the codes above will throw WebException with the following message:
The request was aborted: Could not create SSL/TLS secure channel.
The above error does not provide much details.
Help is appreciate.
Thanks in advance.
EDIT:
This is my entire program:
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.IO;
namespace TestTwoWaySSL
{
class Program
{
static void Main(string[] args)
{
HttpWebResponse Response = null;
try
{
string URL = "https://11.205.112.49:5311/Ping";
Console.WriteLine("URL: [{0}].", URL);
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create(URL);
X509Certificate2 Certificate = new X509Certificate2("F:\\cert.pfx", "password");
Request.ClientCertificates.Add(Certificate);
Request.Method = "GET";
Console.WriteLine("Sending request.");
Response = (HttpWebResponse)Request.GetResponse();
Console.WriteLine("Request sent.");
}
catch (WebException webException)
{
Console.WriteLine("Web exception occurred: [{0}].", webException.Message);
Console.WriteLine(webException.StackTrace);
Response = webException.Response as HttpWebResponse;
}
catch (Exception exception)
{
Console.WriteLine("Exception occurred: [{0}].", exception.Message);
Console.WriteLine(exception.StackTrace);
}
Console.WriteLine("Response status code: [{0}].", Response.StatusCode);
Console.WriteLine("Response \"Content-Type\" header value: [{0}].", Response.ContentType);
Console.WriteLine("Response \"Content-Length\" header value: [{0}].", Response.ContentLength);
String ResponseMessageBody = new StreamReader(Response.GetResponseStream()).ReadToEnd();
Console.WriteLine("Response message body: [{0}].", ResponseMessageBody);
}
}
}
When it reaches the line Request.GetResponse(), an AccessVioationException is thrown:
Unhandled Exception: System.AccessViolationException: Attempted to read or write
protected memory. This is often an indication that other memory is corrupt.
at System.Net.UnsafeNclNativeMethods.SafeNetHandles_SECURITY.AcquireCredentialsHandleW(String principal, String moduleName, Int32 usage, Void* logonID, SecureCredential& authData, Void* keyCallback, Void* keyArgument, SSPIHandle& handlePtr, Int64& timeStamp)
at System.Net.SafeFreeCredentials.AcquireCredentialsHandle(SecurDll dll, String package, CredentialUse intent, SecureCredential& authdata, SafeFreeCredentials& outCredential)
at System.Net.SSPISecureChannelType.AcquireCredentialsHandle(String moduleName, CredentialUse usage, SecureCredential& authdata, SafeFreeCredentials& outCredential)
at System.Net.SSPIWrapper.AcquireCredentialsHandle(SSPIInterface SecModule, String package, CredentialUse intent, SecureCredential scc)
at System.Net.Security.SecureChannel.AcquireCredentialsHandle(CredentialUse credUsage, SecureCredential& secureCredential)
at System.Net.Security.SecureChannel.AcquireClientCredentials(Byte[]& thumbPrint)
at System.Net.Security.SecureChannel.GenerateToken(Byte[] input, Int32 offset, Int32 count, Byte[]& output)
at System.Net.Security.SecureChannel.NextMessage(Byte[] incoming, Int32 offset, Int32 count)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.TlsStream.CallProcessAuthentication(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.ConnectStream.WriteHeaders(Boolean async)
at System.Net.HttpWebRequest.EndSubmitRequest()
at System.Net.HttpWebRequest.SetRequestSubmitDone(ConnectStream submitStream)
at System.Net.Connection.CompleteConnection(Boolean async, HttpWebRequest request)
at System.Net.Connection.CompleteStartConnection(Boolean async, HttpWebRequest httpWebRequest)
at System.Net.Connection.CompleteStartRequest(Boolean onSubmitThread, HttpWebRequest request, TriState needReConnect)
at System.Net.Connection.SubmitRequest(HttpWebRequest request, Boolean forcedsubmit)
at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connName)
at System.Net.HttpWebRequest.SubmitRequest(ServicePoint servicePoint)
at System.Net.HttpWebRequest.GetResponse()
at TestTwoWaySSL.Program.Main(String[] args)
The problem is that you are missing private key. You are using only the certificate.
Either use pkcs#12 when loading certificate into X509Certificate2 class
X509Certificate2 Certificate = new X509Certificate2("F:\\cert.p12", "p12 password");
or load certificate from store
X509Store store = new X509Store("MY",StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);
X509Certificate2Collection collection = (X509Certificate2Collection)store.Certificates;
X509Certificate2 certificate = collection.Find(...)[0];
store.Close();
I'm making a java NIO server which receive messages, each message have its size on the head of the message, that why I'm reading first into a buffer which have default size(44), then get the complete size from this buffer, and then create a new buffer which is supposed to get the rest of the message(body) and then use System.arrayCopy() to make an array with the complete message. this operations are working good, the problem is that the second buffer(body of the message)have the size but does not contain the right data.
plz here is my code if you something wrong :
public void getMessageByMessageSize(SelectionKey key) {
socket = (SocketChannel) key.channel();
int nBytes = 0;
byte[] message = null;
try {
nBytes = socket.read(headBuffer);
if (nBytes < 0) {
try {
key.channel().close();
key.cancel();
return;
} catch (IOException e) {
e.printStackTrace();
}
}
//size of the message body
int corpMessageSize = MessageUtils.getMessageSize(headBuffer)
- HEADER_SIZE;
ByteBuffer corpsBuffer = ByteBuffer.allocate(corpMessageSize);
headBuffer.flip();
nBytes += socket.read(corpsBuffer);
corpsBuffer.flip();
byte[] corp=corpsBuffer.array();
message = new byte[nBytes];
System.arraycopy(headBuffer.array(), 0, message, 0, HEADER_SIZE);
System.arraycopy(corpsBuffer.array(), 0, message, HEADER_SIZE,
nBytes - HEADER_SIZE);
System.out.println(nBytes);
headBuffer.clear();
corpsBuffer.clear();
} catch (IOException e) {
e.printStackTrace();
try {
key.channel().close();
key.cancel();
return;
} catch (IOException ex) {
e.printStackTrace();
}
}
this.worker.verifyConnection(this,message, key);
//this.worker.processData(this, socket, message, nBytes);
}
i have a simple client which send create a byte message, make its size in the head, and then send it.
thanks
nBytes += socket.read(corpsBuffer);
You are assuming that this reads the entire message and fills the buffer. Nothing in TCP/IP guarantees that. You have to loop. If you're in non-blocking mode, you have to re-select if you get a zero length read.