CrateDB cannot connect in HttpEndpoint to remote secure server - cratedb

I've been testing crateDB 3.3.4 for a few weeks now, and I've always been connecting through localhost (127.0.0.1) in http. I've been bulk importing data with HttpEndpoint on my localhost successfully.
I'm now testing on a cloud cluster in https. I manage to open Chrome and logon on the server, but I cannot manage to remote in C# via HttpEndpoint.
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("https://remoteServerUrl:4200/_sql");
httpWebRequest.Method = "POST";
httpWebRequest.Credentials = new NetworkCredential("username", "password");
httpWebRequest.Timeout = 600000;
httpWebRequest.ContentType = "application/json";
using(var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(request);
streamWriter.Flush();
}
HttpWebResponse response = (HttpWebResponse)httpWebRequest.GetResponse();
success = response.StatusCode == HttpStatusCode.OK;
When I try the following code, I always get "The underlying connection was closed: An unexpected error occurred on a send." and in the exception, I see "Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host."
Any Idea what I'm doing wrong here? I was doing the same thing before, on my localhost, but without specifying credentials, and it was working great.

Assuming from your code sample you are using password authentication. Permanent solution would indeed be using autorization header.
As described here https://crate.io/docs/crate/reference/en/latest/admin/auth/methods.html#password-authentication-method

It's not a permanent solution but for test purposes it works. I worked around the problem by adding the Authorization Header directly in my httpwebRequest.
httpWebRequest.Headers.Add(HttpRequestHeader.Authorization, "SomeBasicauthenticationToken");
To find the authentication token to pass, simply put the username and password in a basic authentication token generator like this one :
https://www.blitter.se/utils/basic-authentication-header-generator/
But if anyone has a permanent solution, I would still like to see what it is.

Related

Not able to connect from Windows Application to IdentityServer4 with SSL

I have a Windows application that is using the "password" grant type. It is able to authenticate to the Identityserver4 without SSL, but not with SSL. The problem is that it is giving an error:
The underlying connection was closed: An unexpected error occurred on a send
I tried it from postman, and it worked, but not from my Windows Application. Below is the code:
var tokenClient = new TokenClient($"{IdentityServer}/connect/token", Constants.ClientId, Constants.ClientSecret);
var tokenResponseTask = tokenClient.RequestResourceOwnerPasswordAsync(username, password, Constants.Scope);
tokenResponseTask.Wait();
return tokenResponseTask.Result;
Below also is another code the I tried, but it doesn't work:
TokenResponse tokenResponse;
string request = $"client_id={clientId}&client_secret={clientSecret}&grant_type={grantType}&scope={scope}&username={username}&password={password}";
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
byte[] result = client.UploadData(endpointUrl, "POST", Encoding.UTF8.GetBytes(request));
string resultJson = Encoding.UTF8.GetString(result, 0, result.Length);
tokenResponse = JsonConvert.DeserializeObject<TokenResponse>(resultJson);
}
Finally, I was able to find the solution under the following link:
Authentication failed because remote party has closed the transport stream
'The underlying connection was closed' error is often seen when the SSL handshake fails.
SSL handshake failure usually has something to do with the relevant SSL certificate and whether or not the certificate is trusted.
Check ...
Whether IdentityServer is configured to run under HTTP and HTTPS.
Your Windows Application is correctly configured for SSL.
Test some of the IdentityServer endpoints using a browser with the https protocol.
Hope this gets you going on a helpful investigation path.

how to refresh google access token with refresh token without SSL?

I have faced a problem to refresh google access token on server side.
RestTemplate restTemplate = new RestTemplate();
MultiValueMap<String, String> params = new LinkedMultiValueMap();
params.add("client_id", clientSecrets.getDetails().getClientId());
params.add("client_secret", clientSecrets.getDetails().getClientSecret());
params.add("refresh_token", this.refreshToken);
params.add("grant_type", "refresh_token");
String result = restTemplate.postForObject(requestUrl, params, String.class);
Response I got from google authentication server is just 403 status code.
and message is like this
{"error":"internal_failure","error_description":"SSL is required to perform this operation."}
Is that compulsory to use SSL on my server just in order to refresh access token ?
It's been tested on my local server and does not attach any SSL to it.
References for this code is from below URL.
https://developers.google.com/identity/protocols/OAuth2WebServer#offline
I solved this issue by adding 'https' protocol instead of 'http' protocol for google api
I had called like
http://www.googleapis.com/oauth2/v4/token
but for SSL call
https://www.googleapis.com/oauth2/v4/token
is required to refresh access token

HttpWebRequest to Okta result in Forbidden 403

I'm currently stumble on frustrating Forbidden 403 error when doing simple GET WebRequest to Okta. I've added my application url into Okta Admin > Security > API as CORS. Below is my code
string requestURL = "https://myokta.oktapreview.com/api/v1/users/me";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(requestURL);
request.Method = "GET";
request.Timeout = 120000;
request.ContentType = "application/json";
WebResponse response = request.GetResponse(); // exception Forbidden 403
I could copy paste the requestURL in internet browser and get the json response without any problem. And if I don't have session, it also return error code E0000005 - Invalid session.
If I cannot do it through WebRequest, any suggestion on which client to use in okta.core.dll ?
Please help me and any help would be very much appreciated. Thank you ...
Thanks for the info. There isn't any out-of-the-box solution for your use case, but I would strongly suggest that you switch to OpenID Connect. I have a .NET sample with a Single Page Application and a web API that demonstrates how you can deserialize the JWT token in the Web API to retrieve the user's identity. You can find at https://github.com/rlondner/okta-aspnet-spa_webapi-oidc/
When enabling CORS, please make sure that you type out the correct url.
For instance if you are running https locally, 'http://localhost:8080/' would be the correct url to enable. You also have to enable 'https://developer.okta.com' in Okta's developer console.

Remote service call through proxy

I'm trying to make a very simple service call from VS2012.
The service is on a domain outside a proxy and requires logon credentials.
I have set a refrence to the service in visuals studio. At that point i entered in the remote domian username and password and VS created all the proxy classes for me.
I then added this line to appconf file.
<system.net>
<defaultProxy enabled="true" useDefaultCredentials="true">
</defaultProxy>
</system.net>
Which i believe will allow me to get through our proxy using my own credentails
I then wronte this simple piece of code
private void GetData()
{
OASIS.OasisServiceSoapClient o = new OASIS.OasisServiceSoapClient();
o.ClientCredentials.UserName.UserName = #"OtherDimain\UserName";
o.ClientCredentials.UserName.Password = "Password";
var d = o.SelectOfficersAll();
}
and of course it didn't work and i got all the errors that everyone has posted on.
So first question is
do i need to add this
o.ClientCredentials.Windows.ClientCredential = System.Net.CredentialCache.DefaultNetworkCredentials;
because i did and still get that same stupid error
"The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'Negotiate,NTLM'."
and inner exception
"{"The remote server returned an error: (401) Unauthorized."}"
so am i getting through the proxy ?
Am i using my own credentials ?
Am i passing the right paramaters in to the Service Model ?
Some examples show the username and password properties in the code above are to impersonate the current job.
But i read these on the MSDN page as being the credentials you want to use on the remote serve. The Help topic is ambigious. And if i don't enter them here then how ?
I'm trying to do something so simple , yet can't seem to get past this point.
Ok thanks to my Colleague Sean. It seems that depending on wether you are calling a web service or a WCF services determines what you need to do.
So as a web service this works
OASISWeb.OasisService s = new OASISWeb.OasisService();
s.Credentials = new System.Net.NetworkCredential("Username", "Password", "Domain");
var d = s.SelectOfficersAll();
DataSet x = (DataSet)d;
if it's a WCF service then you need this
var service = new OasisTest2.ServiceReference1.OasisServiceSoapClient();
System.Net.WebRequest.DefaultWebProxy.Credentials = system.Net.CredentialCache.DefaultNetworkCredentials;
service.ClientCredentials.Windows.ClientCredential = new System.Net.NetworkCredential("Username", "Password", "Domain");
var result = service.SelectOfficersAll();
It seems that WebRequest is a global object and you need to set the DefaultWebProxy.Credentails on it.
How you are suppose to know that ? I never found any reference to it when i searched on how to connect to a WCF service on MSDN. Must be a secret. So keep it under your hat.

IIS 7 Serves GET request correctly to browser but throws timeout exception for API request

I am running a very simple Web application (Asp.Net MVC3) on Win 7 IIS.
I have a very simple HTTP GET API which returns hello world.
Calling:
http://localhost/helloworld
Returns:
Hello World!
This works perfectly over a browser.
But when I write an app which tries to pull this URL using a webclient, I get the following error:
{"Unable to read data from the transport connection: The connection was closed."}
My Code is as follows
WebClient web = new WebClient();
var response = web.DownloadString("http://localhost/helloworld");
My IIS Settings are as follows
What should I be looking at? I have been at this for hours and I have run out of options to try! Any help will be really appreciated!
Thanks.
I suspect it's because WebClient does not send some of the HTTP headers:
A WebClient instance does not send optional HTTP headers by default. If your request requires an optional header, you must add the header to the Headers collection. For example, to retain queries in the response, you must add a user-agent header. Also, servers may return 500 (Internal Server Error) if the user agent header is missing. http://msdn.microsoft.com/en-us/library/system.net.webclient(v=vs.80).aspx
Try using HttpWebRequest instead. http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx
I finally figured out what the issue was and instead of it being an IIS specific issue - which I was leaning towards, it turned out to be an issue with the code that I wrote.
Adding details here incase someone else runs into a similar problem.
I had the following method in my code which I was using to send the response of the request as a JSON object.
private void sendJsonResult(string result) {
Response.StatusCode = 200;
Response.Headers.Add("Content-Type", "application/json; charset=utf-8");
Response.Flush();
Response.Write(result);
Response.End();
Response.Close(); // <-- This is the problem statement
}
On digging around a bit, I found out that we should not be doing a Response.Close().
A better explanation of this is here.
Once I removed that line, it started working perfectly - both in my consuming app as well as the web browser, etc.
If you will read the link above, you will clearly understand why we should not be using a Response.Close() - so I will not go into that description. Learnt a new thing today.