C# Elastic search connection pool implementation facing issue - nest

Error: "Invalid NEST response built from a unsuccesful low level call on HEAD:Audit exception in step 2 PingFailure:\r\nElasticsearch.Net.PipelineException: An error occurred trying to establish a connection with the specified node.\r\n "
private static ElasticClient GetESClient(Uri[] nodes, string username, string password)
{
var connectionPool = new SniffingConnectionPool(nodes);
var setting = new ConnectionSettings(connectionPool)
.SniffOnConnectionFault(false)
.SniffOnStartup(false)
.SniffLifeSpan(TimeSpan.FromMinutes(1)).DisableDirectStreaming()
.BasicAuthentication(username, password)
.MaximumRetries(5);
setting.DefaultIndex(ConfigurationManager.AppSettings["ElasticSearch_val"].ToString());
var esClient = new ElasticClient(setting);
return esClient;
}

Related

Asp.net core website intermittently refusing connections

I have an asp.net core 3.0 website. It has a controller that implements an HttpGet function that does some database stuff then returns a Json object (https://localhost:44356/api/runner/match).
I have a console application that uses an HttpClient to hit that url. I launch the site locally and then I launch my console app. About 50% of the time it works. The other 50% of the time I get:
HttpRequestException: No connection could be made because the target machine actively refused it.
SocketException: No connection could be made because the target machine actively refused it.
I'm trying to figure out why my console app's connection is being blocked. I don't know how to start debugging this. I tried to implement a retry on the request, but once I get the exception, I keep getting it. So I think it's something non-deterministic happening in my website, potentially related to SSL?
I'm able to hit the url in Chrome locally just fine.
How do I figure out what is blocking the connection from being made?
Is there any chance this is something IIS Express is doing?
Calling code in console app:
static async Task<List<Deck>> GetMatchData()
{
string baseUrl = "https://localhost:44356/api/runner/match";
using (HttpClient client = new HttpClient())
{
HttpResponseMessage res = null;
res = await client.GetAsync(baseUrl);
Controller function:
[HttpGet("match")]
public async Task<ActionResult> GetMatchup()
{
int count = db.Decks.Count();
Random r = new Random();
int d1 = r.Next(count) + 1; // database ids start at 1 for some reason
int d2 = r.Next(count - 1) + 1;
if (d1 == d2)
d2++;
List<Deck> result = new List<Deck>();
result.Add(await db.Decks.FindAsync(d1));
result.Add(await db.Decks.FindAsync(d2));
if (result[0] == null || result[1] == null)
{
return BadRequest();
}
return Ok(result);
}
try
HttpClient httpClient = new HttpClient();
//specify to use TLS 1.2 as default connection
System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
string baseUrl = "https://localhost:44356/api/runner/match";
httpClient.BaseAddress = new Uri(baseUrl );
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var getResult = await httpClient.GetAsync(baseUrl);

SSIS: Exception has been thrown by the target of an invocation

I have a SSIS package to perform incremental processing of the cube. In the sequence container in this package, we have a script task to count the number of rows.
and the code in script task is as follows :
public void Main()
{
string connectionString = "Data Source=localhost;PROVIDER=MSOLAP;Impersonation Level=Impersonate;Catalog=EcovaPlatform";
connectionString = connectionString.Replace("localhost", Dts.Variables["User::CubeServer1"].Value.ToString()).Replace("EcovaPlatform", Dts.Variables["User::CubeName1"].Value.ToString());
string queryString = "evaluate row(\"count\", countrows(BillDetail))";
AdomdConnection connection = new AdomdConnection(connectionString);
connection.Open();
AdomdCommand cmd = new AdomdCommand(queryString);
cmd.Connection = connection;
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
Dts.Variables["CubeBillDetailRowCount1"].Value = Convert.ToInt64(reader[0]);
}
}
Dts.TaskResult = (int)ScriptResults.Success;
But every once in a while, this script task(here I am assuming its the script task) fails. It throws out the following error:
Source Name: Cube Table RowCount
Error Code: 1
Error Description: Exception has been thrown by the target of an invocation.
Now I dont know how to fix this issue. So I am turning to you good folks, to guide me in the right direction.
If somebody wants to look at the XMLA script, I can post that as well.
Thanks.

Timeout using ServiceStack.Client

I have been using service stack via AJAX calls for some time without issue, but have recently created a quick winforms app which utilizes the service stack client (specifically JsonServiceClient).
However - I have hit a problem whereby I consistently get a timeout on a call which works successfully on the the first TWO attempts. It looks like either the service stack client is holding on to some resource, or I am using the client in the wrong way. It only occurs when running against a remote service (works every time on a local machine). Here is my code, and the exception:
var url = "http://www.TestServer.com/api";
var taskId = Guid.Parse("30fed418-214b-e411-80c1-22000a5b9fe5");
var email = "admin#example.com";
using (var client = new JsonServiceClient(url))
{
var result = client.Send(new Authenticate {UserName = "username", Password = "Password01", RememberMe = true});
client.Put(new AssignTask { AdminTaskId = taskId, Assignee = email });//Call #1 - works fine
client.Put(new AssignTask { AdminTaskId = taskId, Assignee = email });//Call #2 - works fine
try
{
client.Put(new AssignTask { AdminTaskId = taskId, Assignee = email });//Call #3 - works fine
}
catch (WebException ex)
{
//Times out every time
//at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
//at System.Net.HttpWebRequest.GetRequestStream()
//at ServiceStack.Net40PclExport.GetRequestStream(WebRequest webRequest)
//at ServiceStack.ServiceClientBase.<>c__DisplayClassa.<SendRequest>b__9(HttpWebRequest client)
//at ServiceStack.ServiceClientBase.PrepareWebRequest(String httpMethod, String requestUri, Object request, Action`1 sendRequestAction)
//at ServiceStack.ServiceClientBase.SendRequest(String httpMethod, String requestUri, Object request)
//at ServiceStack.ServiceClientBase.Send[TResponse](String httpMethod, String relativeOrAbsoluteUrl, Object request)
//at ServiceStack.ServiceClientBase.Put[TResponse](String relativeOrAbsoluteUrl, Object requestDto)
//at ServiceStack.ServiceClientBase.Put(Object requestDto)
//at SSClientIssue.Program.Main(String[] args) in c:\Users\David\Documents\Visual Studio 2013\Projects\SSClientIssue\SSClientIssue\Program.cs:line 27
throw;
}
}
After the timeout, I can close and reload the app (server stays up), and then get same behavior again (two successful calls). IIS logs show that the 3rd call does not make it to the server, so looks like a Client issue.
I have been looking at this for 8 hours and I think my eyes are starting to bleed...If anyone can help I will buy you a beer!
The issue is due to your ServiceClient requests not specifying a known response type.
Response types can either be marked on the Request DTO using the IReturn<T> marker (recommended):
public class GetAllAdminUsernamesRequest : IReturn<List<string>> { ... }
By adding this on the Request DTO, the ServiceClient is able to automatically infer and convert the response, e.g:
List<string> response = client.Get(new GetCurrentAdminUserAdminTasks());
Otherwise an alternative to specifying the Response on the Request DTO, is to specify it on the call-site, e.g:
List<string> response = client.Get<List<string>>(new GetCurrentAdminUserAdminTasks());
If you don't do this the Response is unknown so the ServiceClient will just return the underlying HttpWebResponse so you can inspect the response yourself.
HttpWebResponse tasks = client.Get(new GetCurrentAdminUserAdminTasks());
In order to be able to inspect and read from the HttpWebResponse the response cannot be disposed by the ServiceClient, so it's up to the call-site making the request to properly dispose of it, i.e:
using (HttpWebResponse tasks = client.Get(new GetCurrentAdminUserAdminTasks())) {}
using (HttpWebResponse adminUsers = client.Get(new GetAllAdminUsernames())) {}
try
{
using (client.Put(new AssignTask { AdminTaskId = taskId, Assignee = user })) {}
using (client.Put(new AssignTask { AdminTaskId = taskId, Assignee = user })) {}
using (client.Put(new AssignTask { AdminTaskId = taskId, Assignee = user })) {}
using (client.Put(new AssignTask { AdminTaskId = taskId, Assignee = user })) {}
}
...
Disposing of your WebResponses responses will resolve your issue.
If you don't do this the underlying WebRequest will throttle open connections and only let a limited number of simultaneous connections through at any one time, possibly as a safe-guard to prevent DDOS attacks. This is what keeps the underlying connections open and WebRequest to block, waiting for them to be released.

What uri pattern do I need to communicate with my PC from my handheld device?

As I was reminded here, I need to probably use "ppp_peer" to programmatically connect from my Compact Framework app to my Web API app running on my PC.
I have tried this (replacing an IPAddress with "ppp_peer"):
string uri = string.Format("http://ppp_peer:28642/api/FileTransfer/GetHHSetupUpdate?serialNum={0}&clientVersion={1}", serNum, clientVer);
...but I get, "NullReferenceException" in "Main" (prior to this I got "Unable to Connect to the Remote Server").
I have a breakpoint in the server code, but it doesn't reach that, so it must be somewhere in the client where this is occurring.
The client code in context is:
string uri = string.Format("http://ppp_peer:28642/api/FileTransfer/GetHHSetupUpdate?serialNum={0}&clientVersion={1}", serNum, clientVer);
RESTfulMethods.DownloadNewerVersionOfHHSetup(uri);
. . .
public static void DownloadNewerVersionOfHHSetup(string uri)
{
string dateElements = DateTime.Now.ToString("yyyyMMddHHmmssfff", CultureInfo.InvariantCulture);
var outputFileName = string.Format("HHSetup_{0}.exe", dateElements);
try
{
var webRequest = (HttpWebRequest)WebRequest.Create(uri);
var webResponse = (HttpWebResponse)webRequest.GetResponse();
string statusCode = webResponse.StatusCode.ToString();
if (statusCode == "NoContent")
{
MessageBox.Show("You already have the newest available version.");
}
else
{
var responseStream = webResponse.GetResponseStream();
using (Stream file = File.Create(outputFileName))
{
CopyStream(responseStream, file);
MessageBox.Show(string.Format("New version downloaded to {0}", outputFileName));
}
}
}
catch (WebException webex)
{
string msg = webex.Message;
string innerEx = webex.InnerException.ToString();
string status = webex.Status.ToString();
MessageBox.Show(string.Format("Message: {0}; Status: {1}; inner Ex: {2}", msg, status, innerEx));
}
}
Do I need an IPAddress in addition to ppp_peer, or is my formatting of the URI wrong, or...???
UPDATE
After the "NRE" I also see, ""...encountered a serious error and must shut down"
I changed the code from above to see just what ppp_peer is translated as:
IPAddress ipAd = Dns.Resolve("PPP_PEER").AddressList[0];
string IPAddr = ipAd.ToString();
MessageBox.Show(IPAddr);
string uri = string.Format("http://{0}:28642/api/FileTransfer/GetHHSetupUpdate?serialNum={1}&clientVersion={2}", IPAddr, serNum, clientVer);
The MessageBox call shows me "192.168.55.100" which is different from what I thought my PC's IPAddress was...???
I get the same with:
IPAddress ipAd = Dns.GetHostEntry("PPP_PEER").AddressList[0];
UPDATE 2
Using this instead (I got it from here [Get ip address of host pc from windows mobile when connected via ActiveSync):
IPAddress ipAd = Dns.GetHostEntry(Dns.GetHostName()).AddressList[0];
...the IP Address displayed is "one up" (192.168.55.101), and instead of an NRE, I get:
Message: Unable to connect to the remote server; Status: ConnectFailure; inner Ex: System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it at System.Net.Sockets.SocketConnectNoCheck(EndPoint remoteEP) ...
So it seems I'm doing all I can on the client end, and the server hears the knock, but is not opening the door - am I right?
BTW, out of curiosity I also added this code:
string hostName = Dns.GetHostName();
MessageBox.Show(string.Format("host name is {0}", hostName));
...and I see "WindowsCE"
UPDATE 3
According to this post by Andy Wiggly (the cat/bloke who wrote "MS .NET Compact Framework"), you do use "ppp_peer":
HttpWebRequest request = REST.CreateRequest(#"http://ppp_peer/DataServicesWebsite/NorthwindService.svc/Customers",
HttpMethods.GET, String.Empty, #"application/atom+xml", "", "");
The interestingest thing about this is the lack of a port assignment (":28642" or whatever); however, this style also gives me an NRE (yes, kind of like a Null Ready to Eat).
UPDATE 4
So what uri will it take to access the host machine from the handheld device?
I have tried all of the following permutations from the client/Compact Framework app, and none work:
IPAddress ipAd = Dns.GetHostEntry(Dns.GetHostName()).AddressList[0];
string IPAddr = ipAd.ToString();
//string uri = string.Format("http://ppp_peer/api/...
//string uri = string.Format("http://ppp_peer:28642/api...
//string uri = string.Format("http://PPP_PEER/api/...
string uri = string.Format("http://PPP_PEER:28642/api/...
//string uri = string.Format("http://{0}:28642/api/...
//string uri = string.Format("http://192.168.125.50:28642/api/...
//string uri = string.Format("http://Platypus:28642/api/...
RESTfulMethods.DownloadNewerVersionOfHHSetup(uri);
The error is happening somewhere in that client code (can't step through it, so I don't know exactly where), because I have a breakpoint on the last line shown, and it is never reached.
SERVER (Web API) code:
[Route("api/FileTransfer/GetUpdatedHHSetup")]
public HttpResponseMessage GetUpdate(string serialNum, string clientVersion)
{
return _fileTransfer.GetHHSetupUpdate(serialNum, clientVersion);
}
public HttpResponseMessage GetHHSetupUpdate(string serialNum, string clientVersion)
{
HttpResponseMessage result;
string filePath = GetAvailableUpdateForCustomer(serialNum); // <= breakpoint on this
line
I put some debug lines in DownloadNewerVersionOfHHSetup() so that it now looks like this:
public static void DownloadNewerVersionOfHHSetup(string uri)
{
MessageBox.Show("Made it into DownloadNewerVersionOfHHSetup");
string dateElements = DateTime.Now.ToString("yyyyMMddHHmmssfff",
CultureInfo.InvariantCulture);
var outputFileName = string.Format("HHSetup_{0}.exe", dateElements);
try
{
var webRequest = (HttpWebRequest)WebRequest.Create(uri);
MessageBox.Show("Made it into DownloadNewerVersionOfHHSetup #2");
var webResponse = (HttpWebResponse)webRequest.GetResponse();
MessageBox.Show("Made it into DownloadNewerVersionOfHHSetup #3");
. . .
I never see "#3", so it must be a problem inside the call to GetResponse(), but how can I find out exactly what? I get the NRE, then "...encountered a serious error and must shut down"
This is where it tries to call the server but, as mentioned, it never makes it to the server method being called...
UPDATE 5
It turns out that this now works:
http://192.168.125.50:28642/api/
...and the main reason that it does is because there was a mismatch between my routing attribute (GetUpdatedHHSetup) and what I was calling from the client (GetHHSetupUpdate). Once I aligned those planets, the NRE went away, and I got the expected result.
PPP_PEER is not needed in the uri/connection string. Using the host PC's IP Address (and port number for the server/Web API app) works now (after fixing the mismatch between the routing attribute and what the client was calling it).
I reckon using the machine name would work just as well, too.

Issue of multiple SQL notifications in ASP.Net web application on page refresh

I am facing an issue while using SQL Server Notifications. I am developing a web application in ASP.net where one of the page needs to be notified about new entries in one of the tables in a SQL Server database. I am using SQL Server Notification services along with Signal R to achieve this functionality.All seems to work fine with my web page getting updates about new data entries.
The problem arises when the page using notification is refreshed. I find the no of notification for single entry in database go up by the number of refreshes. So if I refresh the page thrice, I get 3 notifications for one entry. I am bit concerned if this would be a burden on server when the no of connected users increases. Also if there is an error while processing the request to update the page with new entry, the user gets multiple error messages with same text. I tried debugging my code and found out that the on change event of SqlDependency object used is fired multiple time with different IDs every time. Below is brief overview of what my code is doing to use notifications -
I am using SQL Server 2012 and enable_broker is set for the database.
In global.asax, I am using application_start and application_stop events to start and stop SqlDependency.
In page code, I am setting a new SqlDependency object on page load using a command object to monitor the exact data field of the table.
When onchange of SqlDependency object fires, I am notifying the UI using Signal R hub class. Then I remove the OnChange handler of the SqlDependency object, call for SqlDependency.Stop(connectionstring), set SqlDependency object to nothing, call for SqlDependency.Start(connectionstring) and finally set up the SqlDependency object again using the command object for updated data. This whole set to nothing-stop-start-reset object is to continue monitoring the data for changes.
The above steps work fine but when I refresh the page, those are repeated for the number of refreshes. I tried a lot of things by changing code and debugging but nothing seems to resolve the issue. Now I am wondering if it is some setting somewhere that I missed.
Please help me resolve this issue. Also let me know if any other information such as environment, coding details etc are required.
Regards,
Tanmay
This is probably caused by connection pooling. It reurns a notification for each connection open in the pool. You can cancel the pooling for this specific service by changing the Connection String property:
Pooling = False;
i have resolved the following problem by using the below code, its works me.
SingletonDbConnect.cs
public class SingletonDbConnect
{
private static SingletonDbConnect dbInstance;
private static string connString = ConfigurationManager.ConnectionStrings["MyConnection"].ConnectionString;
private readonly SqlConnection conn = new SqlConnection(connString);
private SingletonDbConnect()
{
}
public static SingletonDbConnect getDbInstance()
{
if (dbInstance == null)
{
dbInstance = new SingletonDbConnect();
}
return dbInstance;
}
public SqlConnection getDbConnection()
{
try
{
conn.Close();
conn.Open();
}
catch (SqlException e)
{
}
finally
{
}
return conn;
}
}
SqlDependencyEvent.cs
public class SqlDependencyEvent
{
internal static int PageLoadCounter = 0;
public void getEmailMessagesByEmailId(Guid emailid)
{
SingletonDbConnect conn = SingletonDbConnect.getDbInstance();
using (MembersController.command = new SqlCommand(SQL.emailmessagesbyaccount_sql(), conn.getDbConnection()))
{
MembersController.command.Notification = null;
if (MembersController.dependency == null)
{
MembersController.dependency = new SqlDependency(MembersController.command);
MembersController.dependency.OnChange += new OnChangeEventHandler(emailMessages_OnChange);
}
var reader = MembersController.command.ExecuteReader();
}
PageLoadCounter++;
}
private void emailMessages_OnChange(object sender, SqlNotificationEventArgs e)
{
if (e.Type == SqlNotificationType.Change)
{
if (MembersController.dependency != null)
{
MembersController.dependency.OnChange -= emailMessages_OnChange;
}
NotificationHub.EmailUpdateRecords();
SingletonDbConnect conn = SingletonDbConnect.getDbInstance();
using (MembersController.command = new SqlCommand(SQL.emailmessagesbyaccount_sql(), conn.getDbConnection()))
{
MembersController.command.Parameters.Add(new SqlParameter("#emailaccountid", defaultemailid));
MembersController.command.Notification = null;
MembersController.dependency = new SqlDependency(MembersController.command);
MembersController.dependency.OnChange += new OnChangeEventHandler(emailMessages_OnChange);
var reader = MembersController.command.ExecuteReader();
}
PageLoadCounter++;
}
}
}
MembersController.cs
public class MembersController : Controller
{
SingletonDbConnect conn = SingletonDbConnect.getDbInstance();
internal static SqlCommand command = null;
internal static SqlDependency dependency = null;
//
// GET: /Members/
public ActionResult Index()
{
SqlDependency.Stop(conn.getDbConnection().ConnectionString);
SqlDependency.Start(conn.getDbConnection().ConnectionString);
return View();
}
}
its resolved my problem and its working me, even we refresh page more than 1, but SqlDependency will call only once.
i used one of the MembersController for SqlDependency start and stop, its your own logic, you can use the same code in Global.ascx instead of MembersController.cs
i hope it will help you and resolve issue. ask me if you have still any problem thanks.