I want to send a POST request to https server and get the response. Here is what I am doing in curl and it works well.
curl --key ./client.key --cert ./client.crt https://test-as.sgx.trustedservices.intel.com:443/attestation/sgx/v2/report -H 'Content-Type: application/json' --data '{"key": "value"}'
This is the code snippet I tried in Go.
url := "https://test-as.sgx.trustedservices.intel.com:443/attestation/sgx/v2/report"
pair, e := tls.LoadX509KeyPair("client.crt", "client.key")
if e != nil {
log.Fatal("LoadX509KeyPair:", e)
}
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
Certificates: []tls.Certificate{pair},
},
}}
resp, e := client.Post(url, "application/json", bytes.NewBufferString(payload))
The program is hanging at the last line, error message is
Post: dial tcp connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
I feel there is problem in my connection establish code, instead of the server's problem since server works perfectly with curl.
Firstly, never ever ever use InsecureSkipVerify: true no matter how convenient it may seem. Instead set something like:
tls.Config {
ServerName: "test-as.sgx.trustedservices.intel.com",
Certificates: []tls.Certificate{pair}
}
Second, initializing http.Transport - to pass your custom tls.Config - also zeros out all the other default http.Transport settings that come with the default http.Client.
Some of those zero defaults may force behavior you might not expect.
See here on how to restore some of those original defaults.
Related
I have an HTTPS endpoint that works fine with a cURL command like this:
curl -k -u xyz:pqr \
--data "grant_type=client_credentials" \
https://my-url
Now I am trying to use golang to call the same API endpoint like this:
data := url.Values{}
data.Set("grant_type", "client_credentials")
req, err := http.NewRequestWithContext(
ctx,
"POST",
"https://my-url",
strings.NewReader(data.Encode())
)
It fails with the error body:
{
"error_description":"grant_type is required",
"error":"invalid_request"
}
I am already passing grant_type as the last argument to the http.NewRequestWithContext() function.
What am I missing?
You are not translating the curl correctly. This tool might be helpful: https://mholt.github.io/curl-to-go/
Essentially you need to add user auth.
req.SetBasicAuth("xyz", "pqr")
// Might help to set the `Content-Type` too
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
-k on curl seems to be reading some curl arguments from a file too. You are going to want to make sure you translate those over as well. I cannot see what they are from your example.
So I am having difficulties sending a curl request to Hive. I want to take the json flow-file that I have created and send it as a command to Hive but I keep getting errors when I try to configure InvokeHTTP processor. For reference here is my workflow as it currently stands.
Replace Text-> Update Attributes - > InvokeHTTP->Put processor
I have tried mostly to get InvokeHTTP processor to work. The configurations that I have are:
1.HTTP Method: POST
2.Remote URL: ${https://hive-prod-1.sample_text/alert}
3.SSL Context Service: StandardSSLContext Service
4.Proxy Type: https
-Content-type: application/json
I then added a property
5.curl: curl-XPOST-H"Authorization: Bearer xWJbexxxxxxxx -H "Content-Type: application/json'
I am not sure if my configuration is incorrect or if there is another issue going on.
When I tried to use/configure ExecuteStreamCommand:
1.Command Arguments: curl-XPOST-H"Authorization xxxxx -H "Content-type: application/json
2.Command Path: application/json
Argument Delimiter: ;
Again, I am not sure if the configuration if correct for either of these processors or if it has something to do with a cert. When I run it I also get the error message 'java.lang.illegalstateexception: trustmanagerfactory =is not initialized.
It sounds as if you have not successfully/completely configured the SSLContextService which is required for InvokeHTTP when connecting to a service which uses TLS. Your Hive instance is protected with TLS, so you need to obtain the public certificate of the Hive instance (you can do this via a browser, using openssl s_client, etc.), load the public certificate into a Java Keystore (JKS) formatted truststore file as a trustedCertEntry, and then point the SSLContextService to that truststore file. For more information, look at the first section of Tomas Zezula's article on NiFi SSL configuration.
I'm using Go to perform HTTPS requests with a custom root CA. The root CA is the only certificate I have on my side.
My code looks like this:
// performRequest sets up the HTTPS Client we'll use for communication and handle the actual requesting to the external
// end point. It is used by the auth and collect adapters who set their response data up first.
func performRequest(rawData []byte, soapHeader string) (*http.Response, error) {
conf := config.GetConfig()
// Set up the certificate handler and the HTTP client.
certPool := x509.NewCertPool()
certPool.AppendCertsFromPEM(certificate)
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: certPool,
InsecureSkipVerify: false,
},
},
}
req, err := http.NewRequest(http.MethodPost, baseURL, bytes.NewBuffer(rawData))
if err != nil {
return nil, err
}
// Sets the SOAPAction and Content-Type headers to the request.
req.Header.Set("SOAPAction", soapHeader)
req.Header.Set("Content-Type", "text/xml; charset=UTF-8")
// Send request as our custom client, return response
return client.Do(req)
}
The error I get back is this:
2017/12/09 21:06:13 Post https://secure.site: x509: certificate is not valid for any names, but wanted to match secure.site
I've been unable to find out exactly what the cause is of this. When checking the SANs of the CA cert, I don't have secure.site in there (no names at all, as the error states), but I can't see how I've done this wrong.
What should I do to troubleshoot this?
You need to do two things:
add the CA certificate on the server side as well, the CA needs to be known by all parties.
generate and use a server certificate (with the hostname in the certificate) on the server. The server cert needs to be signed by the CA.
You can find an example of this at here (first google example)
Edit: to clarify, the error is due to the fact that you are trying to connect securely to a remote host. By default, the go client will look for a valid certificate returned by the server.
Valid means (among other things):
it is signed by a known CA
it contains the ip/dns of the server (the one you passed to http.NewRequest) in the CommonName or Subject Alternative Name: DNS/IP fields.
final edit:
The server certificate contained the correct Common Name set to the server hostname, but it also contained a Subject Alternative Name set to an email address.
As mentioned in https://groups.google.com/a/chromium.org/forum/#!topic/security-dev/IGT2fLJrAeo, Go now ignores the Common Name if is a SAN is found.
I am writing a TCP text protocol for a project that I am working on. One of the commands in the protocol is STARTTLS which should upgrade the connection to TLS and continue on. My code to upgrade the connection is similar to the answer in this question. The problem I am having is when I am upgrading the TLS connection, the tlsConn.Handshake will hang and never let go. There are some code samples below. Any help is greatly appreciated.
After the STARTTLS command is received...
// Init a new TLS connection. I need a *tls.Conn type
// so that I can do the Handshake()
s.Logf("++> Upgrading connection to TLS")
tlsConn := tls.Server(s.Conn, s.Server.TLSConfig)
s.Logf("++> Attempting TLS Handshake")
tlsConn.Handshake()
s.Logf("++> TLS Handshake Successful")
// Here is the trick. Since I do not need to access
// any of the TLS functions anymore,
// I can convert tlsConn back in to a net.Conn type
s.Conn = net.Conn(tlsConn)
s.Logf("++> Updating read/write buffers")
s.reader = textproto.NewReader(bufio.NewReader(s.Conn))
s.writer = textproto.NewWriter(bufio.NewWriter(s.Conn))
s.Printf("100 SUCCESS")
The client is currently upgrading the connection right after it sends the STARTTLS command like this...
c.conn = tls.Client(c.conn, clientTLSConfig)
The server *tls.Config looks like this...
// Load the key and certificate - paths are provided in flags.
cert, err := tls.LoadX509KeyPair(flagTLSCert, flagTLSKey)
if err != nil {
log.Fatal(err)
}
// Create the TLS config
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert},
ClientAuth: tls.VerifyClientCertIfGiven,
ServerName: fqdn(),
}
The client *tls.Config looks like this...
clientTLSConfig := &tls.Config{
InsecureSkipVerify: true,
}
Do you call c.conn.Handshake() or do something else to initiate the TLS handshake on the client side?
If the client does not initiate the handshake by sending TLS Client Hello, the server will sit forever waiting for it.
This is my best guess as you did not provide much of the client side code. Also checking with tcpdump would help to narrow the problem down (to server or client side).
I am testing Office 365 REST API using CURL following this link:
Contacts REST API in Office 365 APIs Preview
I can obtain correctly one contact using curl command in Windows like this:
curl --no-sessionid --insecure --basic --user "user#domain.com:password" -H "Accept: application/json" "https://outlook.office365.com/EWS/OData/Me/Contacts?$orderby=DisplayName+asc&$top=1"
And following documentation on this link if I try to create one contact using CURL with minimum required options for testing:
curl -X POST -d "{\"#odata.type\": \"#Microsoft.Exchange.Services.OData.Model.Contact\",\"GivenName\": \"TestContact\",\"EmailAddress1\": \"test#test.com\",\"BusinessPhone1\": \"123-456-7890\"}" https://outlook.office365.com/ews/odata/Me/Contacts --header "Content-Type:application/json" --insecure --verbose --user "user#domain.com:password"
I receive following error:
* About to connect() to outlook.office365.com port 443 (#0)
* Trying 157.56.250.178...
* connected
* Connected to outlook.office365.com (157.56.250.178) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
...
* SSL connection using ECDHE-RSA-AES256-SHA
* Server certificate:
...
* Server auth using Basic with user 'user#domain.com'
> POST /ews/odata/Me/Contacts HTTP/1.1
...
> Content-Length: 157
>
* upload completely sent off: 157 out of 157 bytes
< HTTP/1.1 400 Bad Request
...
< Content-Length: 82
<
{"error":{"code":"ErrorInvalidRequest","message":"Cannot read the request body."}}* Connection #0 to host outlook.office365.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
I have searched the internet and in stackoverflow but similar questions have no answer I'm looking for.
How could I create on contact using CURL?, I have tested it on Linux too but have the same results.
The following request works when I try it from Fiddler. Can you please try this out using CURL?
POST https://outlook.office365.com/ews/odata/Me/Contacts HTTP/1.1
Authorization: Basic <XXXX>
Content-Type: application/json
{
"GivenName" : "John",
"EmailAddresses" : [
{ "Address": "John#contoso.com", "Name" : "John" }
],
"BusinessPhones" : [
"123-456-7890"
]
}
I made a few changes to your request. You don't need to specify the OData.type as we infer that you are adding a Contact because you are sending a POST to Contacts collection. We need to fix our documentation as it lists the entity type as required. We have updated our namespace to Microsoft.OutlookServices and hence the type definitions have changed. To make the API easier to use, we have replaced EmailAddress1, EmailAddress2 etc. with a collection of EmailAddresses. Similarly, we have also changed BusinessPhones, HomePhones etc. to collections as well.
As I just explained in another post, the issues you are seeing are from some changes being rolled out to our preview APIs and our documentation is in the process of being updated. The current set of changes include versioning support, and this won't be an issue going forward.
Please let me know if you have any questions or need more info.
Thanks,
Venkat
Using following CURL command worked perfect:
curl -X POST -d "{\"GivenName\":\"John\",\"EmailAddresses\":[{\"Address\":\"John#contoso.com\",\"Name\":\"John\"}],\"BusinessPhones\":[\"123-456-7890\"]}" https://outlook.office365.com/ews/odata/Me/Contacts --header "Content-Type:application/json" --insecure --verbose --user "user#domain.com:password"
Now we can export contacts (this is not allowed from OWA) and import it using simple utilities like curl.
Thanks for your help Venkat.
Your original post showed the following namespace:
Microsoft.Exchange.Services.OData.Model.Contact
That has been changed to:
Microsoft.Office365.OutlookServices.Contact
Since the namespace was wrong, it couldn't read the request body.