Curl request with ntlm authentication fails if password is set - authentication

I try to fetch some data from a Microsoft Dynamics Nav WebService.
This service uses the NTML authentication.
If I open the webservice url in a browser and use the given credentials everything works fine.
For setting up the environment for the WebService Client, I used the command line to check whether everything is working fine, I was, at a specific point, unable to authenticate.
Thats the command I am using:
curl --ntlm -u "DOMAIN\USERNAME" -k -v "http://hostname:port/instance/Odata/Company('CompanyName')/Customer"
The command will prompt for the password.
I copy in the password and everything is doing fine.
But when I use this command, with the password already included, it stops working and the authentication fails:
curl --ntlm -u "DOMAIN\USERNAME:PASSWORD" -k -v "http://hostname:port/instance/Odata/Company('CompanyName')/Customer"
The password contains some special chars, so I tried to use the percent encoding, which had no effect at all.
It is very difficult to research this kind of issue. Searching for curl + ntlm authentication issues provides a lot of results, but nothing is related to this specific kind of issue.
Does anyone of you guys already had experience with this kind of issue?

I had a problem with authentication because of cookies. I solved this containing cookies in txt file and using exactly this file through all requests. For example, after login request I saved this cookies:
curl -X POST -u username:password https://mysite/login -c cookies.txt
And with next request I used this file like this:
curl -X POST -u username:password https://mysite/link -b cookies.txt
This solution worked for me, I don't know if your problem is similar, but, I think, you may try this.

I was struggling with similar issue for a long time and finally I found this curl bug report #1253 NTLM authentication fails when password contains special characters (british pound symbol £) .
NTLM authentication in cURL supports only ASCII characters in passwords! This is still the case in version 7.50.1 on Ubuntu but I tested this on many different distributions and it is always the same. This bug also will break curl_init() in PHP (tested on PHP7). The only way to solve that is to avoid non ASCII characters in NTLM authentication passwords.
If you are using Python then you are lucky. Apparently Python developers rewrote cURL implementation and it works with non ASCII characters if you use HttpNtlmAuth package.

Try with nltm flag.
Something like this:
curl -v --proxy-nltm -u 'username:password' youproxy.com:8080 someURL
from > curl --help
-x, --proxy [PROTOCOL://]HOST[:PORT] Use proxy on given port
--proxy-anyauth Pick "any" proxy authentication method (H)
--proxy-basic Use Basic authentication on the proxy (H)
--proxy-digest Use Digest authentication on the proxy (H)
--proxy-negotiate Use Negotiate authentication on the proxy (H)
--proxy-ntlm Use NTLM authentication on the proxy (H)

Related

How to get Bearer access token in Jmeter

I have problems to get Bearer access token in Jmeter. Problem's cause most probably is in
server authentication - do not really understand how to properly setup this in Jmeter.
curl consist of several parts:
curl -X POST -v https://xxxx.xxx.xx.lv/token -u
"d123c9e3-4e91-46db-931e-37e8a52b8c8d:0e7cb8ad50fe3686de05fcf46815abc0a9d7cd095c23f75446d933ccxyxy"
-d "grant_type=password" -d "username=xxxxxxx" -d "password=xxxxxx" -d "scope=scop"
I have tried to use HTTP AuthorizationManager, HTTP Request and HTTP Header Manager without any successful results. Any advices?
Thanks!
Tatjana
I think you should do something like:
HTTP Request sampler:
these protocol and Server Name bits can be moved to HTTP Request Defaults so you won't have to specify them for each and every HTTP Request sampler
HTTP Authorization Manager:
In general since JMeter 5.1 it's possible to create a test plan from curl command line, the option lives under "Tools -> Import from cURL" main menu entry:

Disable cert revocation check in unix/linux using curl command

I am using curl command to invoke a rest service. It is as follows:
{curl -X POST --ssl-no-revoke --cacert xyz.pem -K urls.txt -H "Authorization:Basic XYZ" -H "Content-Type:application/json" -d #data.json}
The above command is used to hit the service using one way SSL and basic authorization. The data to be passed is enclosed in data.json file and the urls to be hit are enclosed in urls.txt file.
The above command works perfectly in Windows but when executed from linux, it says:
{curl: option --ssl-no-revoke: is unknown
curl: try 'curl --help' or 'curl --manual' for more information}
I want to disable certificate revocation checks altogether. It looks like {--ssl-no-revoke} works on Windows but not Unix/Linux.
Would like to know if any alternative.
ssl-no-revoke is Windows Only. The only alternative I'm aware of is to have a valid certificate or not use SSL.
https://curl.haxx.se/docs/manpage.html
Using a valid certificate is not always a solution as revocation checks will fail with a valid certificate too when there is no Internet connection (for example, in the presence of a captive portal).
One way is to disable certificate checking altogether, i.e.:
curl --insecure https://www.example.com
Note that this will greatly reduce the security as self signed certificates will also be accepted as well as revoked ones!

Traefik basic auth does not work with HTTPS

I run the service in Docker Warm Mode with these labels:
- "traefik.docker.network=proxy"
- "traefik.backend=kibana"
- "traefik.frontend.entryPoints=https,http"
- "traefik.frontend.rule=Host:mydomain"
- "traefik.port=5601"
- "traefik.frontend.auth.basic=test:098f6bcd4621d373cade4e832627b4f6"
And have this problem using HTTPS
curl -u test:test https://my-domain.com
401 Unauthorized
With HTTP all is ok
curl -u test:test http://my-domain.com
Found
Using htpassword solved it for me. It seems Traefik uses the same algorithm to unhash the passwords.
apt install apache2-utils
htpasswd -nb your_username "your_password_here"
You will receive the according hash
your_username:khrglahfslgkha345346
Copy paste it to your .toml or your docker-compose script.
Use your password (not the hash) for the login on your frontend and everything will work fine.
I have recently found out you have to take care of the double dollar signs in the resulting hash. You have to escape the $ in different scenarios....
I found cause of problem, I deploy service as a stack with traefik variable "traefik.frontend.auth.basic=test:$$apr1$$EaOXV0L6$DQbzuXBeb6Y8jjI2ZbGsg/". But after deploy value of this variable looks like test:/.
After manually setting correct value - auth work fine.
Also I have tried deploy service with command docker service create and variable have correct value.

Setting up Shibboleth IdP with LDAP

I am attempting to use LDAP with Shibboleth. I am nearly there, but I cannot authenticate. I followed these instructions, and my login.config file contains the following:
ShibUserPassAuth {
edu.vt.middleware.ldap.jaas.LdapLoginModule required
host="ldap://localhost:10389" base="ou=users,ou=system"
ssl="false" userField="uid";
};
I have tried several variations of the above.
I can use TestShib to reach my IdP's login page, but always receive "Login failed. Double-check your username and password" when I attempt to authenticate.
This might be easier to debug myself if I could manage to get any authentication log messages, but I can't seem to get those working either.
Note that the following ldapsearch command works perfectly fine:
ldapsearch -h "ldap.example.com:10389" -w testpass -x -D "uid=testuser,ou=users,ou=system" -b "dc=example,dc=com" '(objectclass=*)'
If you could tell me what my problem might be, or at least how to enable logging, that would be very helpful.
I have this working locally. I have taken my working config and replaced my LDAP host and base path with yours from the question above.
ShibUserPassAuth {
edu.vt.middleware.ldap.jaas.LdapLoginModule required
ldapUrl="ldap://localhost:10389"
baseDn="ou=users,ou=system"
userFilter="uid={0}";
};

Gitlab API: How to generate the private token

This is what I tried:
curl http://git.ep.petrobras.com.br/api/v3/session --data-urlencode 'login=myUser&password=myPass'
Answer:
{"message":"401 Unauthorized"}
The problem is the data-urlencode CURL option. Since it's an HTTP POST you don't need to URL encode the data, and is actually encoding the & into & and causing your issue. Instead use the --data option.
curl http://git.ep.petrobras.com.br/api/v3/session --data 'login=myUser&password=myPass'
Also, be careful sending credentials over plain HTTP. It could be easily sniffed.
This is how:
$ curl http://git.ep.petrobras.com.br/api/v3/session/ --data-urlencode 'login=myUser' --data-urlencode 'password=myPass'
The solution pointed out by Steven doesn't work if your username or password contains characters that have to be urleencoded. The name=content format will urlencode the content part (the name part has to be urlencoded but login and password are fine).
To actually retrieve the private_token you can pipe the output of curl into jq like this:
$ curl [as above] | jq --raw-output .private_token
x_the_private_token_value_x
This way you can easily use it in a shell script.
Also, as Steven pointed out already, please use https instead so that your password is not transmitted in clear text across the wire.
Note: this workflow no longer works as of GitLab 8.6.0 as the default password has been removed.
Changelog: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG#L205
I only just noticed this and raised the issue. Leaving this note here to hopefully save someone else some time. Hopefully, this is a decision that will be reviewed and reverted.
Discussion/issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/1980