i have used package like github.com/Azure/go-ntlmssp to authenticate to sharepoint 2010.
url, username, password := "http://www.some-website.com", "admin", "12345"
client := &http.Client{
Transport: ntlmssp.Negotiator{
RoundTripper:&http.Transport{},
},
}
req, _ := http.NewRequest("GET", url, nil)
req.SetBasicAuth(username, password)
res, _ := client.Do(req)
in response i am getting 401 Unauthorized
Now here the wierd thing is sometimes(after many trials and not consistent after that also) the authentication happens successfully.
And when the authentication fails in response header i can see Www-Authenticate:[NTLM].
Please suggest
Related
The route for login authenticate -- app.Handle("GET", "/v1/users/token", u.Token).
We can get name and password from request.BasicAuth.
func (u *User) Token(ctx context.Context, w http.ResponseWriter, r *http.Request, params map[string]string) error {
...
name, pass, ok := r.BasicAuth()
...
}
But how can I set name and pass from client web url?
Before a browser asks the user to supply basic auth credentials, you have to deny the request access (using status code 401 Unauthorized). You should set the header WWW-Authenticate to Basic realm="Your message".
Also see this article.
So in your code, when ok is false, you should deny that request:
func (u *User) Token(ctx context.Context, w http.ResponseWriter, r *http.Request, params map[string]string) error {
...
name, pass, ok := r.BasicAuth()
if !ok {
w.Header().Set("WWW-Authenticate", "Basic realm=\"Your message\"")
http.Error(w, "Must supply authentication credentials", http.StatusUnauthorized)
return
}
}
What I'm trying to do:
Build a package (later usage) that provides a method to execute a get-request to any page through a given socks5 proxy.
My problem:
When ever I try to request a page with SSL (https) I get the following error:
Error executing request Get https://www.xxxxxxx.com: socks connect tcp 83.234.8.214:4145->www.xxxxxxx.com:443: EOF
However requesting http://www.google.com is working fine. So there must be a problem with the SSL connection. Can't imagine why this isn't working as I'm not very experienced with SSL-connections. End of file makes no sense to me.
My current code:
func main() {
// public socks5 - worked when I created this question
proxy_addr := "83.234.8.214:4145"
// With this address I get the error
web_addr := "https://www.whatismyip.com"
// Requesting google works fine
//web_addr := "http://www.google.com"
dialer, err := proxy.SOCKS5("tcp", proxy_addr, nil, proxy.Direct)
handleError(err, "error creating dialer")
httpTransport := &http.Transport{}
httpClient := &http.Client{Transport: httpTransport}
httpTransport.DialTLS = dialer.Dial
req, err := http.NewRequest("GET", web_addr, nil)
handleError(err, "error creating request")
httpClient.Timeout = 5 * time.Second
resp, err := httpClient.Do(req)
handleError(err, "error executing request")
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
handleError(err, "error reading body")
fmt.Println(string(b))
}
func handleError(err error, msg string) {
if err != nil {
log.Fatal(err)
}
}
So what am I missing in here to deal with ssl-connections?
Thank you very much.
Edit 1:
In case someone would think this is an issue with whatismyip.com I've done some more tests:
https://www.google.com
EOF error
https://stackoverflow.com
EOF error
https://www.youtube.com/
EOF error
Connection between your program and your socks5 proxy goes not through SSL/TLS
So you should change line
httpTransport.DialTLS = dialer.Dial
to
httpTransport.Dial = dialer.Dial
I checked https://www.whatismyip.com and https://www.google.com.
URLs are downloaded fine.
For test I set up 3proxy service on my server, test your code with fixed line and check 3proxy logs.
All made requests was in proxy server logs.
If you need more help - please let me know, I'll help
Things to notice:
Socks5 proxies need to support SSL connections.
The code from the question won't work with this answer as the proxy (used in the code) isn't supporting SSL connections.
In my Go application I use a Web API, which accepts a POST request and returns a token if it is successful.
The URL is the following: "https://accounts.zoho.com/apiauthtoken/nb/create?SCOPE=xxxx&EMAIL_ID=xxxx&PASSWORD=xxxx"
If I send this in Postman or any other tool, I get the status code 200 with the expected body, but in my go code the same request returns error code 400.
url := "https://accounts.zoho.com/apiauthtoken/nb/create?SCOPE=xxxx&EMAIL_ID=xxxx&PASSWORD=xxxx"
req, err := http.NewRequest("POST", url, nil)
log.Println(err)
res, err := client.Do(req)
log.Println(err)
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
log.Println(err)
fmt.Println(res)
fmt.Println(string(body))
All errors are nil, but the res struct contains 400 Bad Request and the last output (of the body) is empty.
Output:
&{400 Bad Request 400 HTTP/1.1 1 1 map[Date:[Sat, 03 Mar 2018 16:48:35 GMT] Connection:[keep-alive] Set-Cookie:[a8c61fa0dc=5f224f2ab9d067cfce10fd0b7fae48bf; Path=/; Secure; HttpOnly] Server:[ZGS]] 0xc04225a080 -1 [chunked] false false map[] 0xc042034a00 0xc0421bc370}
I would appreciate any help, because I don't know where this problem comes from.
Url-encoded parameters should be provided in body parameter not as query string. So this can be a problem too. Set your paramters in body implementing io.Reader.
func NewRequest(method, url string, body io.Reader) (*Request, error)
Set the header Content-Type in your request before sending it using Client.Do.
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
Since postman is smart enough to set the headers for you. Thats why no error.
The whole code becomes
package main
import (
"log"
"net/http"
"net/url"
"io/ioutil"
"strings"
)
func main(){
url := "https://accounts.zoho.com/apiauthtoken/nb/create"
var data url.Values{}
data.Set("SCOPE","xxxx")
data.Add("EMAIL","xxxx")
data.Add("PASSWORD","xxxx")
req, err := http.NewRequest("POST", url, strings.NewReader(data.Encode()))
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
log.Println(err)
res, err := client.Do(req)
log.Println(err)
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
log.Println(err)
fmt.Println(res)
fmt.Println(string(body))
}
I was facing the similar issue. The reason for 401 simply means the invalid authorization token. Later I noticed that Postman executes the Pre-request script that updates the header every time. You can pull in the same pre-request script code in golang and it works.
Same question here, try to make a POST request with some QUERY params in url, which works fine in Postman but always got Http 400 in go debug with all the same params (Query params , post body ,headers and MD5 sign)
timeStr := time.Now().Format("2006-01-02 15:04:05")
serverUrlAll = serverUrlAll + "?token="+token+"&provider="+provide+"×tamp="+timeStr+"&signature="+signature
contentBody := strings.NewReader(signString)
request, err := http.NewRequest("POST",serverUrlAll , contentBody)
if err != nil {
return nil, err
}
request.Header.Set("Content-Type", "application/json")
Finally found the blank in query url will cause the error , remove the blank and it works in my case.
I am going to secure my golang application using keycloak, but keycloak itself does not support go language.
There are some go adaptor as an open project in github that has implemented openId connect protocol as a provider service, but they do not provide an example or documentation on how to integrate libraries with an application.
How can i interact with keycloak using golang?
As you have pointed out, there is no official keycloak adapter for golang.
But it is pretty straight forward to implement it. Here is a little walk through.
Keycloak server
For this example, I will use the official keycloak docker image to start the server.
The version used is 4.1.0.Final. I think this will work with older KeyCloak versions too though.
docker run -d -p 8080:8080 -e KEYCLOAK_USER=keycloak -e KEYCLOAK_PASSWORD=k --name keycloak jboss/keycloak:4.1.0.Final
After the server is up and running, you can open localhost:8080/auth in your browser, navigate to the administration console and login with username keycloak and k as the corresponding password.
I will not go through the complete process of creating a realm/clients/users. You can look this up under
https://www.keycloak.org/docs/latest/server_admin/index.html#admin-console
Here is just an outline for what I did to reproduce this example:
create a realm named demo
turn off the requirement of ssl for this realm (realmsettings -> login -> require ssl)
create a client named demo-client (change the "Access Type" to confidential)
create a user named demo with password demo (users -> add user). Make sure to activate and impersonate this user.
configure the demo-client to be confidential and use http://localhost:8181/demo/callback as a valid redirect URI.
The resulting keycloak.json (obtained from the installation tab) looks like this:
{
"realm": "demo",
"auth-server-url": "http://localhost:8080/auth",
"ssl-required": "none",
"resource": "demo-client",
"credentials": {
"secret": "cbfd6e04-a51c-4982-a25b-7aaba4f30c81"
},
"confidential-port": 0
}
Beware that your secret will be different though.
The Go Server
Let's go over to the go server. I use the github.com/coreos/go-oidc package for the heavy lifting:
package main
import (
"context"
"encoding/json"
"log"
"net/http"
"strings"
oidc "github.com/coreos/go-oidc"
"golang.org/x/oauth2"
)
func main() {
configURL := "http://localhost:8080/auth/realms/demo"
ctx := context.Background()
provider, err := oidc.NewProvider(ctx, configURL)
if err != nil {
panic(err)
}
clientID := "demo-client"
clientSecret := "cbfd6e04-a51c-4982-a25b-7aaba4f30c81"
redirectURL := "http://localhost:8181/demo/callback"
// Configure an OpenID Connect aware OAuth2 client.
oauth2Config := oauth2.Config{
ClientID: clientID,
ClientSecret: clientSecret,
RedirectURL: redirectURL,
// Discovery returns the OAuth2 endpoints.
Endpoint: provider.Endpoint(),
// "openid" is a required scope for OpenID Connect flows.
Scopes: []string{oidc.ScopeOpenID, "profile", "email"},
}
state := "somestate"
oidcConfig := &oidc.Config{
ClientID: clientID,
}
verifier := provider.Verifier(oidcConfig)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
rawAccessToken := r.Header.Get("Authorization")
if rawAccessToken == "" {
http.Redirect(w, r, oauth2Config.AuthCodeURL(state), http.StatusFound)
return
}
parts := strings.Split(rawAccessToken, " ")
if len(parts) != 2 {
w.WriteHeader(400)
return
}
_, err := verifier.Verify(ctx, parts[1])
if err != nil {
http.Redirect(w, r, oauth2Config.AuthCodeURL(state), http.StatusFound)
return
}
w.Write([]byte("hello world"))
})
http.HandleFunc("/demo/callback", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Get("state") != state {
http.Error(w, "state did not match", http.StatusBadRequest)
return
}
oauth2Token, err := oauth2Config.Exchange(ctx, r.URL.Query().Get("code"))
if err != nil {
http.Error(w, "Failed to exchange token: "+err.Error(), http.StatusInternalServerError)
return
}
rawIDToken, ok := oauth2Token.Extra("id_token").(string)
if !ok {
http.Error(w, "No id_token field in oauth2 token.", http.StatusInternalServerError)
return
}
idToken, err := verifier.Verify(ctx, rawIDToken)
if err != nil {
http.Error(w, "Failed to verify ID Token: "+err.Error(), http.StatusInternalServerError)
return
}
resp := struct {
OAuth2Token *oauth2.Token
IDTokenClaims *json.RawMessage // ID Token payload is just JSON.
}{oauth2Token, new(json.RawMessage)}
if err := idToken.Claims(&resp.IDTokenClaims); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
data, err := json.MarshalIndent(resp, "", " ")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Write(data)
})
log.Fatal(http.ListenAndServe("localhost:8181", nil))
}
This program starts a regular http server with two endpoints. The first one ("/") is your regular endpoint that handles
application logic. In this case, it only returns "hello world" to your client.
The second endpoint ("/demo/callback") is used as a callback for keycloak. This endpoint needs to be registered on your
keycloak server. Keycloak will issue a redirect to this callback URL upon successful user authentication. The redirect contains some additional query parameters. These parameters contain a code that can be used to obtain access/id tokens.
Verify your setup
In order to test this setup you can open a webbrowser and navitage to http://localhost:8181.
The request should reach your go server, which tries to authenticate you. Since you did not send a token, the go server
will redirecty you to keycloak to authenticate.
You should see the login screen of keycloak. Login with the demo user you have created for this realm (demo/demo).
If you have configured your keycloak correctly, it will authenticate you and redirect you to your go server callback.
The end result should be a json like this
{
"OAuth2Token": {
"access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJsc1hHR2VxSmx3UUZweWVYR0x6b2plZXBYSEhXUngtTHVJTVVLdDBmNmlnIn0.eyJqdGkiOiI5ZjAxNjM2OC1lYmEwLTRiZjMtYTU5Ni1kOGU1MzdmNTNlZGYiLCJleHAiOjE1MzIxNzM2NTIsIm5iZiI6MCwiaWF0IjoxNTMyMTczMzUyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZGVtbyIsImF1ZCI6ImRlbW8tY2xpZW50Iiwic3ViIjoiMzgzMzhjOGItYWQ3Zi00NjlmLTgzOTgtMTc5ODk1ODFiYTEyIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoiZGVtby1jbGllbnQiLCJhdXRoX3RpbWUiOjE1MzIxNzMzNTIsInNlc3Npb25fc3RhdGUiOiJjZTg2NWFkZC02N2I4LTQ5MDUtOGYwMy05YzE2MDNjMWJhMGQiLCJhY3IiOiIxIiwiYWxsb3dlZC1vcmlnaW5zIjpbXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsInByZWZlcnJlZF91c2VybmFtZSI6ImRlbW8iLCJlbWFpbCI6ImRlbW9AZGVtby5jb20ifQ.KERz8rBddxM9Qho3kgigX-fClWqbKY-3JcWT3JOQDoLa-prkorfa40BWlyf9ULVgjzT2d8FLJpqQIQYvucKU7Q7vFBVIjTGucUZaE7b6JGMea5H34A1i-MNm7L2CzDJ2GnBONhNwLKoftTSl0prbzwkzcVrps-JAZ6L2gssSa5hBBGJYBKAUfm1OIb57Jq0vzro3vLghZ4Ay7iNunwfcHUrxiFJfUjaU6PQwzrA5pnItOPuavJFUgso7-3JLtn3X9GQuyyZKrkDo6-gzU0JZmkQQzAXXgt43NxooryImuacwSB5xbIKY6qFkedldoOPehld1-oLv0Yy_FIwEad3uLw",
"token_type": "bearer",
"refresh_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJsc1hHR2VxSmx3UUZweWVYR0x6b2plZXBYSEhXUngtTHVJTVVLdDBmNmlnIn0.eyJqdGkiOiI0MjdmMTlhYy1jMTkzLTQ2YmQtYWFhNi0wY2Q1OTI5NmEwMGQiLCJleHAiOjE1MzIxNzUxNTIsIm5iZiI6MCwiaWF0IjoxNTMyMTczMzUyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZGVtbyIsImF1ZCI6ImRlbW8tY2xpZW50Iiwic3ViIjoiMzgzMzhjOGItYWQ3Zi00NjlmLTgzOTgtMTc5ODk1ODFiYTEyIiwidHlwIjoiUmVmcmVzaCIsImF6cCI6ImRlbW8tY2xpZW50IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiY2U4NjVhZGQtNjdiOC00OTA1LThmMDMtOWMxNjAzYzFiYTBkIiwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBwcm9maWxlIGVtYWlsIn0.FvvDW6ZSH8mlRR2zgaN1zesX14SmkCs9RrIVU4Jn1-SHVdKEA6YKur0-RUAFTObQDMLVhFFJ05AjGVGWpBrgVDcAwW2pI9saM-OHlyTJ3VfFoylgfzakVOIpbIDnHO12UaJrkOI9NWPAJdbBOzBHfsDhKbxhjg4ZX8SwlKr42rV4WWuSRcNu4_YDVO19SiXSCKXVldZ1_2S-qPvViq7VZfaoRLHuYyDvma_ByMsmib9JUkevJ8dxsYxVQ5FWaAfFanh1a1f8HxNRI-Cl180oPn1_Tqq_SYwxzBCw7Q_ENkMirwRS1a4cX9yMVEDW2uvKz2D-OiNAUK8d_ONuPEkTGQ",
"expiry": "2018-07-21T13:47:28.986686385+02:00"
},
"IDTokenClaims": {
"jti": "f4d56526-37d9-4d32-b99d-81090e92d3a7",
"exp": 1532173652,
"nbf": 0,
"iat": 1532173352,
"iss": "http://localhost:8080/auth/realms/demo",
"aud": "demo-client",
"sub": "38338c8b-ad7f-469f-8398-17989581ba12",
"typ": "ID",
"azp": "demo-client",
"auth_time": 1532173352,
"session_state": "ce865add-67b8-4905-8f03-9c1603c1ba0d",
"acr": "1",
"email_verified": true,
"preferred_username": "demo",
"email": "demo#demo.com"
}
}
You can copy your access token and use curl to verify if the server is able to accept your tokens:
# use your complete access token here
export TOKEN="eyJhbG..."
curl -H "Authorization: Bearer $TOKEN" localhost:8181
# output hello world
You can try it again after the token has expired - or temper with the token. In case you do it, you should get a redirect to
your keycloak server again.
There is also the gocloak library which provides lot's of functionality. The lib is in active development and allready in use in real world projects. So possible bugs & feature requests are being handled.
It provides administration features like "CreateUser","CreateGroup" etc. and also provides functions for Login, Token validation, etc.
For example creating a user is as easy as:
client := gocloak.NewClient("https://mycool.keycloak.instance")
token, err := client.LoginAdmin("user", "password", "realmName")
if err != nil {
panic("Something wrong with the credentials or url")
}
user := gocloak.User{
FirstName: "Bob",
LastName: "Uncle",
EMail: "something#really.wrong",
Enabled: true,
Username: "CoolGuy",
}
client.CreateUser(token.AccessToken, "realm", user)
if err != nil {
panic("Oh no!, failed to create user :(")
}
It does also supports Introspecting a Requesting Party Token
client := gocloak.NewClient(hostname)
token, err := client.LoginClient(clientid, clientSecret, realm)
if err != nil {
panic("Login failed:"+ err.Error())
}
rptResult, err := client.RetrospectToken(token.AccessToken, clientid, clientSecret, realm)
if err != nil {
panic("Inspection failed:"+ err.Error())
}
if !rptResult.Active {
panic("Token is not active")
}
permissions := rptResult.Permissions
//Do something with the permissions ;)
Also to handle easy authentication & token refresh when using echo there is another lib based on gocloak called gocloak-echo. This lib provides handler & middleware to help out, but is still in a more WIP state.
The library also provides decoding of accessTokens into custom claims
Disclosure: I am the (main) author of gocloak, so it's also a little advertising, but in general it answers the question. I had the same problem as the author and i decided to create my own lib (based on the lib of someone else, as stated in the readme on github).
I am trying to make a GET request against an HTTPS URL using proxy with username/password authorization (auth is required by the proxy not the website).
Here's what I do:
package main
import (
"crypto/tls"
"encoding/base64"
"fmt"
"net/http"
"net/http/httputil"
"net/url"
)
func main() {
ua := "Mozilla/5.0 (Windows NT 6.1"
basic := "Basic " + base64.StdEncoding.EncodeToString([]byte("username:mypass"))
req, err := http.NewRequest("GET", "https://api.ipify.org/", nil)
proxyUrl, _ := url.Parse("http://myproxy.com:9999")
fmt.Println(basic) // Basic dXNlcm5hbWU6bXlwYXNz
req.Header.Add("Proxy-Authorization", basic)
req.Header.Add("User-Agent", ua)
bb, _ := httputil.DumpRequest(req, false)
fmt.Println(string(bb))
/*
Get / HTTP/1.1
Host: api.ipify.org
Proxy-Authorization: Basic dXNlcm5hbWU6bXlwYXNz
User-Agent: Mozilla/5.0 (Windows NT 6.1
*/
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
Proxy: http.ProxyURL(proxyUrl),
},
}
resp, err := client.Do(req)
fmt.Println(err) // Proxy Authentication Required
fmt.Println(resp) // <nil>
}
The catch is that when I try to do a request to an HTTP (not HTTPS) site it goes fine, but if I make HTTPS request it fails (see above the message).
I tested the proxy with my browser (FireFox) and everything goes well, I looked for the headers through firebug and added everything relevant to the request (above). I've double-triple-checked the basic value and everything else but without any luck.
So does some one have any idea why this happens or at least how do I research the problem?
The last thing to add is that I can use a public HTTP proxy (that doesn't require any auth) in this case and problems seem to start when auth enters in this process (the error also suggests that).
P.S. Unfortunately I cannot share the proxy IP, port and username cause it is against their policy.
package main
import (
"crypto/tls"
"fmt"
"net/url"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://api.ipify.org/", nil)
proxyUrl, _ := url.Parse("http://username:password#127.0.0.1:9999")
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
Proxy: http.ProxyURL(proxyUrl),
},
}
_, err = client.Do(req)
fmt.Println(err)
}