Small question regarding Spring Webflux WebClient please.
I have a netrc file at ~/.netrc
The content is something like:
machine example.com
login abc
password qwerty
I use this netrc file in my curl command:
curl --netrc --insecure http://example.com:9876/API/abc
And I am getting the correct response, very happy.
I just need to migrate this curl command into Spring Webflux WebClient.
Therefore, I went to write this, here is what I tried:
final var webClient = WebClient.create().mutate().defaultHeader("machine","example.com", "login", "abc", "password", "qwerty").clientConnector(new ReactorClientHttpConnector(HttpClient.create())).build();
final var response = webClient.post().uri("http://example.com:9876/API/abc").retrieve().bodyToMono(String.class).block();
Please note I put the content of the netrc file onto the WebClient header.
Unfortunately, I am getting 401 unauthorized.
The content of the netrc file, which I pasted onto the header, is for sure correct.
Therefore, I am having a hard time understanding why is it failing.
What did I do wrong please?
Thank you
Related
I am running a spring application and when doing GET requests on POSTMAN I get my API response, see below
However, when I run a curl command I get a 307 Redirect error
I tried to dig into but could not solve it. Could be a proxy issue ?
As pointed out by #pavelsaman, you're using different endpoints in Postman and your curl.
To avoid that and run the very same request in curl as in Postman, you can export your request from Postman as curl:
Click "Code", then select "cURL", copy the code, and paste it to your commandline:
You can find the documentation here: https://learning.postman.com/docs/sending-requests/generate-code-snippets/
I maintain a private repository but want to make one file publicly available.
GitHub documentation states that the CURL command below can retrieve a file:
curl -u username:token https://api.github.com/user
But I would like to provide access through a URL. E.g.
https://username:token#raw.githubusercontent.com/me/repo/master/README.md
This always return a 404. Am I missing something?
From "How can I download a single raw file from a private github repo using the command line?", you wouldneed to use a PAT (Personnal Access Token) without the username:
curl -s https://$TOKEN#raw.githubusercontent.com/....
But I would not recommend making that token visible in any way: it would give access to that file and the rest of the repository.
Putting that file in a separate location (be it a separate public repository, or any other online text storage service) would be safer.
For those of you wondering the "why" on 404 vs 401, it's basically a security measure on GitHub's part to output 404 instead of 401: https://docs.github.com/en/github-ae#latest/rest/overview/other-authentication-methods#basic-authentication
For those wondering why we get a 404 in the browser while cURL gives us a success response, you might've assumed that providing the username and password in the URL like https://username:password#somesite.com would pass the credentials along in the initial request. That is not the case - the browser steps in and sees if the page you are requesting returns a WWW-Authenticate response header, and only then does it send your credentials. In the case of your GitHub url, the resource doesn't send back a WWW-Authenticate. If it did return WWW-Authenticate, then you obviously wouldn't run into this problem.
And then there's cURL. cURL assumes Basic Authentication by default and automatically sets the Authorization header to your username and password (either from the url like my previous example, or set through CLI options like in your example), and it sends it regardless of whether or not the server returns a WWW-Authenticate response header.
Unfortunately for us, there's no way to force the browser to send it with the initial request. As to why GitHub doesn't send a WWW-Authenticate response header, it's probably because they don't want to promote the least secure way of authentication - they no longer allow account passwords to be sent this way, after all. However, they do realize its ease of use and have mitigated some of its weaker points by allowing users to use oAuth access token, GitHub App installation access token, or Personal Access Token in its place that can limit its scope of access. So really, it's the browser that is following standards, and GitHub allowing a form of Basic Authentication with some alterations, and cURL immediately passing our credentials into the Authorization header. I believe the below is what's happening behind your requests:
cURL sends a request along with Authorization header → GitHub: "Well, I didn't ask, but yeah, your creds check out" → GitHub: Authorized and redirects to resource
Browser sends request and waits for WWW-Authenticate response before handing credentials → GitHub: "Umm, you don't have permission to access this resource but I can't let you know whether it actually exists") → GitHub: Returns 404 (instead of 401 with WWW-Authenticate header) stopping the browser short from receiving the WWW-Authenticate header response and sending out an Authorization header with the credentials on hand.
I normally don't ask questions here because most of the times I can find answers. But at this moment I haven´t find one, and I am really stuck. I have a problem trying to get access token for the Cherwell api: http://13.88.176.216/CherwellAPI/Swagger/ui/index#!/Service/Service_Token I used postman to generate this code:
This is relevant to Cherwell Service Management's V8+ REST API.
Code that throws server run time exception:
string user = "myUser";
string password = "myPassword";
var client1 = new RestClient("http://13.88.176.216/cherwellapi/token?auth_mode=Internal");
client.Authenticator = new HttpBasicAuthenticator(user, password);
var request1 = new RestRequest(Method.POST);
request1.AddHeader("content-type", "application/x-www-form-urlencoded");
request1.AddHeader("cache-control", "no-cache");
request1.AddParameter("application/x-www-form-urlencoded", "grant_type=password&client_id=my_client_id&client_secret=my_client_secret", ParameterType.RequestBody);
IRestResponse response = client1.Execute(request1);
The thing is when I execute the same method from the swagger ui (http://13.88.176.216/CherwellAPI/Swagger/ui/index#!/Service/Service_Token) I can get the token without getting any error.
Details of the request in CURL:
Curl
curl -X POST
--header "Content-Type: application/x-www-form-urlencoded"
--header "Accept: application/json" -d "grant_type=password&client_id=my_client_id&client_secret=my_client_secret&username=my_user_name&password=my_password" "http://13.88.176.216/CherwellAPI/token?auth_mode=Internal"
Request URL
http://13.88.176.216/CherwellAPI/token?auth_mode=Internal
This is the response body from the swagger ui test, not my code:
{
"access_token": "the_acces_token",
"token_type": "bearer",
"expires_in": 1199,
"refresh_token": "the_refresh_token",
"as:client_id": "client_key",
"username": "user",
".issued": "date",
".expires": "other_date"
}
Any help will be appreciated.
Try including the username/password as part of your form encoded data.
Drop the authenticator section, it shouldn't be necessary for this part.
So,
request1.AddParameter("application/x-www-form-urlencoded", "grant_type=password&client_id=my_client_id&client_secret=my_client_secret&username=(yourusernamehere)&password=(yourpasswordhere)", ParameterType.RequestBody);
I actually just recorded a video on this not too long ago (using a browser rest client, not C#, but you get the picture), that should post to our youtube channel soon at https://youtube.com/beyond20llc - I can send this video to you if you'd like to see it before it reaches youtube.
The data I sent when I was authenticating for a token essentially looked like the following:
grant_type=password&
client_id=1234567890&
username=CSDAdmin&
password=CSDAdmin
(Of course, CSDAdmin being the default username/password on a fresh installation of Cherwell - if you're CSDAdmin account still has these credentials, change immediately as this is a well-known default pass).
Have you tried using the swagger code generation tool as documented in the Cherwell documentation?
Once you have generated the client code, you will have wrapper data structures for all Cherwell REST API requests and responses.
Using Swagger Code Gen
You will need to install Maven and the Java Development kit.
all. recently I learned wso2 api manager 1.4+ and see the publisher apis are open. As I tried to invoke the apis using the RESTClient in api store, the response show "{"message" : "AuthenticateError", "error" : true}". I follow the doc in here: http://docs.wso2.org/display/AM140/Publisher+APIs. I am not familiar with rest apis, I just invoke the GET method, using the URL: http://**:9763/publisher/site/blocks/listing/ajax/item-list.jag, and get the error above. did I miss something or use the wrong format?
then I use curl to test the apis, it works.
root#appfactory01:~# curl -b cookies http://*:9763/publisher/site/blocks/listing/ajax/item-list.jag?action=getAllAPIs
{"error" : false, "apis" : [{"provider" : "admin", "subs" : 0, "name" : "redmine", "thumb" : "/registry/resource/_system/governance/apimgt/applicationdata/icons/admin/redmine/1.0.0/icon", "version" : "1.0.0", "status" : "PUBLISHED"}]}
It is because when using curl you are setting cookies through -b cookies option, but when using RESTClient it's not used. That's why you get Authenticate Error. You can add the -v option to curl to see the exact request and response.
To access through RESTClient add the Cookie Header as follows. Get the value for JSESSIONID from the cookies file used with curl or by observing the output of curl with -v option.
Cookie:JSESSIONID=8BB4617ABC21786D90A450xxxxxxx
and use the following url with GET request.
http://localhost:9763/publisher/site/blocks/listing/ajax/item-list.jag?action=getAllAPIs
I'm trying to retrieve HTTP HEAD information from a S3 resource using Python. Unfortunately the response always returns a 403 error. I am using code that was suggested
here but unfortunately this does not work for S3 requests. Here is the code (host and key details omitted):
>>> import httplib
>>> conn = httplib.HTTPConnection("www.mybucket.us")
>>> conn.request("HEAD", "/mykey?myparamterers")
>>> res = conn.getresponse()
>>> res.status
>>> 403
The request also sends a signed expiration as part of the query string.
I have also tried using httplib2 but the HEAD REQUEST simply hangs.
If you are using a signed URL, the URL is signed by method (e.g. HEAD or GET). So you will need a different URL for each method.
403 tells you that the request sent was valid (whether it was correct is another story), but that you don't have permission to access the requested page for some reason. At least you know that a valid HTTP request is being sent.
Can you check the server log at all? That might help shed some light on the problem...
I'm not sure about the "HEAD" request either. Can you not use "GET" or "POST" and extract the header yourself? It might be that "HEAD" is not implemented by the library... I am not sure - the documentation I've been able to find by quick googling is woefully inadequate.
403 HTTP code means forbidden. Probably, site administrator disabled this method.
try telnet
telnet www.mybucket.us 80
HEAD http://www.mybucket.us/mykey?myparamterers
Host: www.mybucket.us
<ENTER>
<ENTER>
and watch for server response.
Alternatively, you could use conn.set_debuglevel(1) in python code.
A HEAD request on an AWS S3 pre-signed URL generated for GET will result in a 403 Forbidden response. You can, however, achieve the equivalent of a HEAD request on a pre-signed AWS S3 URL by doing a GET request with the Range header set as follows:
Range: bytes=0-0
here is a code snippet
import requests
response = requests.get(download_url, headers={"Range": "bytes=0-0"})
As shown in the Screen shot, response will have enpty body but will have Hearder info.
https://i.stack.imgur.com/9J99G.png