SqlDataReader is lost after posting request using RestSharp - restsharp

I'm writing a Windows service. It periodically connects to a SQL database and looks for new records. I can read the SQL database without an issue and it gets the data using a SqlDataReader. I then have a while loop that just does a while Read. Part way through the while loop I create a web request to send that data to a server side component. For that I am using RestSharp v107. When I call ExecuteAsync with the request, I can see the data being sent and it ends up on my server side without an issue. However, when the response comes back, it has wiped out my SqlDataReader so on the next loop I get "Exception: Invalid attempt to call Read when reader is closed." as the reader is null. Here is the code, truncated where necessary for brevity.
private async Task AddUpdateCustomers(SqlConnection poConn)
{
string sQuery = "select * from customers";
try
{
SqlCommand oCmd = new SqlCommand(sQuery, poConn);
poConn.Open();
using (SqlDataReader dataReader = oCmd.ExecuteReader())
{
if (dataReader.HasRows)
{
while (dataReader.Read())
{
int iRecordId = dataReader.GetInt32(dataReader.GetOrdinal("ID"));
Customer customer = new Customer()
{
CustomerId = dataReader.GetString(dataReader.GetOrdinal("CustomerCodeFinancialReference")),
Name = dataReader.GetString(dataReader.GetOrdinal("Name"))
// more fields here removed for brevity
};
RestRequest request = new RestRequest();
request.AddHeader("Profile", aCLIntacctClientServiceConfiguration.ProfileName);
request.AddHeader("Entity", aCLIntacctClientServiceConfiguration.EntityName);
request.AddHeader("CreateUpdate", dataReader.GetBoolean(dataReader.GetOrdinal("IsNew")) ? "create" : "update");
request.AddHeader("Type", "Customer");
request.AddHeader("CorrelationId", Guid.NewGuid().ToString());
string body = ObjectToString(customer);
request.AddStringBody(body, DataFormat.Json);
var options = new RestClientOptions(ClientURI)
{
ThrowOnAnyError = false,
Timeout = 30000
};
RestClient client = new RestClient(options);
RestResponse response = await client.ExecuteAsync(request);
if (!string.IsNullOrEmpty(response.Content))
{
try
{
CustomResponse customerResponseObject = JsonConvert.DeserializeObject<CustomResponse>(response.Content);
if (customerResponseObject.ExitCode == 0)
{
WriteBackToStagingTable(iRecordId, true, "");
}
else
{
string sMessages = "";
foreach (CustomResponseMessage message in customerResponseObject.Messages)
{
sMessages += $"Record ID '{message.RecordId}' failed with the following error: '{message.MessageValue}'" + Environment.NewLine;
}
WriteBackToStagingTable(iRecordId, false, sMessages);
}
}
catch (Exception ex)
{
WriteBackToStagingTable(iRecordId, false, ex.Message + Environment.NewLine + response.Content);
}
}
}
if (!dataReader.IsClosed)
{
dataReader.Close();
}
}
else
{
// No records read from db
}
}
}
catch (Exception ex)
{
HandleLogRequest("Exception: " + ex.Message);
}
}
I've tried using PostAsync but that didn't work and reading the documentation, it isn't what I want. I cannot see why after calling client.ExecuteAsync the dataReader is suddenly null. I'm using RestSharp v107 and so many examples on the web but they are earlier versions and there are a lot of differences.

Rather than pass the connection round as it wasn't particularly necessary, I moved the connection into each function and created it as necessary. With the SQL data reader, I didn't bother with the using block. I also changed it so that it was executing asynchronously.
SqlDataReader dataReader = await oCmd.ExecuteReaderAsync();
Once I'd done that, calling the rest client execute method, it returned and the data reader was still in scope.

Related

How to insert a large image into an SQL database

If I insert a small image, for example 1-2 kb, then it is successful, if I select an image of 70 kb or any other large one, then I always have an error.
I have tried both the stored procedure and directly pass an array of bytes - error.
Exception System.Data.SqlClient.SQLException: "A transport layer error occurred while receiving results from the server. (provider: TCP provider, error: 0 - The semaphore timeout is exceeded.)"
Everything is fine with SQL Server itself, images are loaded successfully through WinForms, there are no problems.
Button:
<RadzenUpload Accept="image/*" ChooseText="Select..." Url=#($"api/upload/single/{clientid}/1") Progress="#((args) => OnProgress(args, "Loading ..."))" />
Controller and void:
[HttpPost("single/{id}/{photoid}")]
public async Task<IActionResult> Single(IFormFile file, string id,string photoid)
{
try
{
IVZBlazor.Iservice.IPhotoService serv;
string fileName = Path.GetFileName(file.FileName);
string rootpath = System.IO.Path.Combine(System.IO.Directory.GetCurrentDirectory(), "wwwroot");
string path = rootpath + "\\Uploads";
using var fileStream = file.OpenReadStream();
long length = file.Length;
byte[] bytes = new byte[length];
fileStream.Read(bytes, 0, (int)file.Length);
using (FileStream stream = new FileStream(Path.Combine(path, fileName), FileMode.Create))
{
file.CopyTo(stream);
}
await Save(bytes, id);
return StatusCode(200);
}
catch (Exception ex)
{
return StatusCode(500, ex.Message);
}
}
private DynamicParameters SetParameter(byte[] oPhoto)
{
DynamicParameters parameters = new DynamicParameters();
parameters.Add("#photo", oPhoto);
return parameters;
}
public async Task<int> Save(byte[] photo, string clientid)
{
using (var connection = new SqlConnection(Startup.ConnectSQL))
{
await connection.OpenAsync();
var sqlStatement = #"UPDATE [dbo].[photo] SET [image1] =#photo WHERE [personid] ='" + clientid + "'";
int res = await connection.ExecuteAsync(sqlStatement, SetParameter(photo));
return res;
}
}

A possible object cycle was detected. in both System.Text.Json and Newtonsoft.Json

I have been pulling my hair out with this one.
I have a very simple test class that throws this error:
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
An unhandled exception has occurred while executing the request.
System.Text.Json.JsonException: A possible object cycle was detected. This can either be due to a cycle or if the object depth is larger than the maximum allowed depth of 32. Consider using ReferenceHandler.Preserve on JsonSerializerOptions to support cycles.
It doesn't seem to break much, as the put request is successful and the serialize is also successful.
EDIT
I have chased the serialize exception out if it was ever really there. I am starting to think it is a problem with typed HttpClient. It throws the exception that comes out on the console and in the response on Postman. However, it doesn't allow me to catch the exception in the code and the PUT call works. So the exception is happening after the PUT request and is handled before it returns control to my app.
I am going to try to use a standard HttpClientFactor instead of a typed client and see if that works. I know that the JSON exception is a red herring, but it is ugly and breaking the response.
Any suggestions would be welcome.
public virtual async Task<CouchResponse> Create(string id, string db, TObj info)
{
CouchResponse ret = new() { Reason = "Unknown and unExpected error", Ok = false };
HttpResponseMessage rc = null;
if (id is null)
{
return new CouchResponse() { Id = "missing", Ok = false, Rev = "missing" };
}
string url = $"{db}/1";
try
{
// login to Couchdb servwer
await CouchLogin();
try
{
//var jsonInfo = JsonUtils.Serialize<TestJson>(jTest);
var jsonInfo = JsonSerializer.Serialize<TObj>(info, options);
HttpContent content = new StringContent(jsonInfo, Encoding.UTF8,
"application/json");
rc = await client.PutAsync(url, content);
}
catch (Exception eNewton)
{
Console.WriteLine($"Json Exception: {eNewton.Message}");
}
if (rc is not null)
{
var str = await rc.Content.ReadAsStringAsync();
var ret = JsonSerializer.Deserialize<CouchResponse>(str,options);
rc.EnsureSuccessStatusCode();
}
return ret;
}
catch (Exception e)
{
Console.WriteLine(e.Message);
//return ret;
}
return ret;
}
Suggestions?
What a crazy bug. The diagnostic was very missing leading. Everything I was doing in the create method was correct.
What is missed was an await when I called the create method. This made it appear that the sendAsync was having the issue when it was really the controller trying to format the task return as a response. This caused the stack trace in the response message. Thanks for all the help.
Change this
var jsonSerializerSettings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
NullValueHandling = NullValueHandling.Ignore
};
To this
var jsonSerializerSettings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
NullValueHandling = NullValueHandling.Ignore,
MaxDepth = null,
};

Sql Dependency - Notification received before processing current request

I am currently using sql dependency notification to detect changes in a table and process them. I am having a problem where the notification gets called while its still in the middle of completing the first request which causes duplicate processing
private void ProcessData()
{
try
{
m_Guids = new List<Guid>();
using (SqlCommand command = new SqlCommand("SP_XXX_SELECT", m_sqlConn))
{
command.CommandType = CommandType.StoredProcedure;
command.Notification = null;
SqlDependency dependency = new SqlDependency(command);
dependency.OnChange += new OnChangeEventHandler(OnDependencyChange);
SqlDependency.Start(m_ConnectionString, m_QueueName);
if (m_sqlConn.State == ConnectionState.Closed)
{
m_sqlConn.Open();
}
using (SqlDataReader reader = command.ExecuteReader())
{
if (reader.HasRows)
{
while (reader.Read())
{
m_Guids.Add(reader.GetGuid(0));
}
}
}
Console.WriteLine(m_Guids.Count.ToString());
ProcessGuids();
}
}
}
catch (Exception ex)
{
//SendFailureEmail
}
}
private void OnDependencyChange(object sender, SqlNotificationEventArgs e)
{
SqlDependency dependency = sender as SqlDependency;
dependency.OnChange -= OnDependencyChange;
ProcessData();
}
public void OnStart()
{
SqlDependency.Stop(m_ConnectionString, m_QueueName);
SqlDependency.Start(m_ConnectionString, m_QueueName);
m_sqlConn = new SqlConnection(m_ConnectionString);
}
ProcessData method gets called again while its still in the middle of processing (processGuids) Should I subscribe to the event after processing all the data?
If I don't subscribe until processing is complete, what happens to the data that was changed during the process, which I believe doesn't get notified until next change happens?. What is the correct way of doing this or am I doing something wrong.
Thanks
SqlDependency.OnChange is called not only on data change.
In the OnDependencyChange you must check e.Type/e.Source/e.Info.
F.e., combination of {Type = Subscribe, Source = Statement, Info = Invalid} means "Statement not ready for notification, no notification started".
See Creating a Query for Notification for SQL statement requirements for notification. You must follow these requirements in SELECT statements in your SP.
Additional requirements for stored procedures are not well documented. Known restrictions for SP:
Use of SET NOCOUNT (ON and OFF) is prohibited.
Use of RETURN is prohibited.

force singleordefault to execute

Can anyone tell me why the following code does not work in an api controller (MVC4)? I get an error saying that the "Cannot access a disposed object". I found that in other cases there should be a call to ToList() so that it forces the query to be executed. But in my case, I'm using SingleOrDefault(), isn't this executed on the database right away?
public HttpResponseMessage GetPurchaseOrder(int POId)
{
PurchaseOrder po;
HttpResponseMessage resp;
using (PMDataDataContext database = new PMDataDataContext())
{
po = database.PurchaseOrders.SingleOrDefault(x => x.POId == POId);
if (po == null)
{
var message = string.Format("id = {0} not found", POId);
HttpError err = new HttpError(message);
resp = Request.CreateResponse(HttpStatusCode.NotFound, err);
}
else
{
resp = Request.CreateResponse(HttpStatusCode.OK, po);
}
}
return resp;
}
Yes, but if you have any lazy-loaded properties in PurchaseOrder, when Web API tries to serialize the object, EF will try to fetch the data from the database and for this the context will be needed but it will not be available.
Try it so that the context to database gets disposed right after the query. SingleOrDefault should send the query to the database immediately.
public HttpResponseMessage GetPurchaseOrder(int POId)
{
PurchaseOrder po;
HttpResponseMessage resp;
using (PMDataDataContext database = new PMDataDataContext())
{
po = database.PurchaseOrders.SingleOrDefault(x => x.POId == POId);
}
if (po == null)
{
var message = string.Format("id = {0} not found", POId);
HttpError err = new HttpError(message);
resp = Request.CreateResponse(HttpStatusCode.NotFound, err);
}
else
{
resp = Request.CreateResponse(HttpStatusCode.OK, po);
}
return resp;
}
Badri led me to solve my problem. When I placed a breakpoint in my code and followed in debug mode, it seemed that one of the associated objects needed the context to be fetched. The property PurchaseOrder.Item.ItemCategory was giving an ObjectDisposedException, when I was looking at the related properties.
I solved it by placing database.DeferredLoadingEnabled = false right before fetching the po using the SingleOrDefault call.

ProtocolError while calling HttpWebRequest.GetResponse()

I have a page containing links to some files.
I basically need to access the source of the page for parsing it then and obtaining all the hyperlinks to the files.
My code is something like this (some piece of code I've found in many places on the net ..):
"private static byte[] ReadImageFromUrl(string url)
{
var myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.Timeout = 10000;
WebResponse myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
List<byte> bytesList = new List<byte>();
using (var br = new BinaryReader(stream))
{
try
{
while (true)
{
var b = br.ReadByte();
bytesList.Add(b);
}
}
catch (Exception)
{}
br.Close();
}
myResp.Close();
return bytesList.ToArray();
}"
Now the problem is I get "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." when calling "myReq.GetResponse()" - examining the error I see that the status is 'ProtocolError'.
The response property of the WebException object contains some server error ..(although when opening it from the browser it opens correctly) ...also when I call this function with the url of one of my files I get the same ProtocolError status, but the 404 error ...
Please give any hint how could I solve it... or any other possibility of accomplishing this task.
Thanks !
My new code after using Fiddler is:
private static byte[] ReadFileFromUrl(string url)
{
var myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.Accept = const_AcceptHeader;
myReq.Headers.Set(const_AcceptLanguageHeaderName, const_AcceptLanguageHeader);
myReq.UserAgent = const_AcceptUserAgentHeader;
myReq.CookieContainer = new CookieContainer();
myReq.KeepAlive = true;
myReq.Timeout = Int32.Parse(ConfigSettings.RequestPageTimeout) * 1000;
WebResponse myResp = null;
List<byte> bytesList = null;
myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
bytesList = new List<byte>();
using (var br = new BinaryReader(stream))
{
try
{
while (true)
{
var b = br.ReadByte();
bytesList.Add(b);
}
}
catch (Exception ex)
{
throw;
}
br.Close();
}
return bytesList.ToArray();
}
All variables that start with const_ are taken from Fiddler.
Well, I solved that using Fiddler ... I passed to my request object the headers as I have seen them in Fiddler ...& it worked, no error