I am attempting to build a c# module to connect to the Twitter streaming API using OAuth (now the only option). I have got to the point where my module will successfully access api urls using GET, but everything I do to try and make a POST request fails with a 401.
I have checked my signature is correct by using the OAuth Tool tab on the page for my Twitter App, and fixing the values for nonce and timestamp in my code. I have curl for Windows set up and can verify that it works with the sample curl script generated by the OAuth tool (by the way, this needs some correction of the quotes to make it work for curl in Windows Cmd. Get rid of single quotes on values that don't need them, use double quotes on anything that needs to be quoted, and on the Authorization header, use double quotes and escape double quotes within the header with a backslash).
I have even gone to the length of running curl in trace mode and outputting the bytes I send in the post body from my c# code and I can verify that they are the same.
I am trying to access 'https://stream.twitter.com/1.1/statuses/filter.json' using 'track=twitter' as the post body. The headers are:
Accept: */*
User-Agent: curl/7.21.7(amd64-pc-win32) libcurl/7.21.7 OpenSSL/0.9.8rzlib/1.2.5
Content-Type: application/x-www-form-urlencoded
Host: stream.twitter.com
Content-Length: 13
Connection: Keep-Alive
Authorization: OAuth <the oauth stuff>
I can't inspect the packets being sent to check on the wire that the requests are identical as they are of course SSL encoded.
Any ideas?
I eventually got this to work. Things that might help you if you have this kind of problem which I discovered:
I had a problem initially because I created a new nonce every time the bit of code was accessed. This meant the nonce which was used in generating the signature key was different from the one in the header. Obviously fail.
I then ran into the above problem. What it was is that I was adding the OAuth header to my request AFTER I sent the request body. For some reason it seems to send the request as soon as you write to the request stream for a POST.
It was very useful in finding 2. that I found out how to use Fiddler to trace web requests from code. Essentially all you need to do is add this to your web.config:
<system.net>
<defaultProxy>
<proxy proxyaddress="http://127.0.0.1:8888" />
</defaultProxy>
</system.net>
As soon as I tried to read the HTTPS request, Fiddler prompted me to install bits so it could decrypt the request, which I did and then I could see the exact request going down the wire. I could compare this with what cURL was doing using
-x 127.0.0.1:8888
option.
However I then ran into a problem with my request timing out. Which bizarrely enough was caused by the fact that Fiddler was proxying the response. Once I took the above out of my web.config again it all worked. Halleluja!
Related
When i try to test api with localhost:[port] it gives the invalid character in header ["Host"] console error. I am using dotnet core webApi. I cross checked the CORS configuration from api end it is fine. The issue is on the Postman side.
Postman version: v8.7.0
I had the same error being reported for any forked or created Postman requests:
Error: Invalid character in header content ["Host"]
The request URL was using a global parameter:
{{BaseUri}}/some/sort/of/resource
In the console logs the following was reported (URLs redacted):
Request Headers
User-Agent: PostmanRuntime/7.29.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 9d14e81d-1e21-44a2-93ed-2758f0ad24fa
Host: my.url.co.uk↵
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Note the ↵ character at the end of the Host Header.
The global BaseUri parameter did not appear to have a line break at the end of it. However, completely deleting said parameter and recreating it seems to have fixed the issue.
I also had same incident and I was able to find the error by exporting "My Workspace" content and open it from notepad++. Then change the encoding to "ANSI" from notepad++ (Encoding=> ANSI). You will notice special characters as below,
This can happen when you copy the url and paste in Postman and then try to edit it.
If you are getting this URL from someone what you can ask is to provide exported json file from postman. Then import it to your workspace.
I thought the issue was in a variable I was using because the error was telling me there's an invalid character in my host https://localhost:4431 which is exactly the value of my variable.
I figured out the invalid character was actually not in my variable but in the rest of the URL in my request.
Turns out, when copying endpoint names from the Swagger of my API, I was also copying an invisible character %E2%80%8B. I saw it when checking the API's console RequestPath:/%E2%80%8BmyEndpoint
Removing this invisible character solved the issue
Taken from question comments. by Fidel Garcia
I created the request again via Add Request menu and it works. I'm not sure if it is a problem with the update and old requests. The old one is still failing.
===================================================
It also worked for me. I created new request with the same parameters and it worked.
I created a requested with and wrote the parameters in headers. After wrong requested I changed it to correct one (post request and parameters in body) and got the error. After creating new request with correct configuration (post request parameters in body) it worked correctly.
In my case:
I removed the authentication from Header then I re-enter the authentication credentials again.
In my case enter after param and path generate error. Exact reason could be found in postman console.
In my case this happened because I added an extra blanckspace at the end of an environment variable deffinition. That extra space was being taken into account in the route when making a request.
Be careful with those extra blanck spaces.
I've just started using Postman to test an API I am integrating to.
I have the following error that keeps showing up
Invalid CORS request
Note the following:
The API uses Bearer token authentication(OAuth2). I have this
working without a problem.
I do get the bearer token successfully, assign it to an Environment variable and then attempt to use it for the RESTful operations.
The problem is in the subsequent RESTful operation that uses the token.
When I use an old token (through a POST operation), it rightfully
tells me that it is expired and not authorized.
When I then generate a new one and try to run the restful call, it gives me that Invalid CORS request error.
Using cURL, I have no issues. But I am frustrated by Postman.
What I have found so far:
Using postman with Http POST requests - I don't get the part in bold
Just in case anybody else has this same problem, here is how to solve
it. Go to https://www.getpostman.com/docs/capture in your chrome
browser. Click on interceptor extension and then choose add to
chrome. Once it is added there is a new icon top right of both the
browser and postman that looks like a traffic light. In postman click
this and it turns green. Then add a header to every request going to
third light. Every header consists of the header name and a value.
Start typing over the header name and a list of allowed http headers
comes up. Choose "Origin". In the cell for value simply type the
full URL of your server. (Do not forget the 'http://' or 'https://').
What is the expected response to an invalid CORS request? - Best explanation I have seen so far on CORS errors.
The other material speaks about Access-Control-Allow-Method header, preflight requests
... and there is an illustrative Apache Tomcat flowchart of the CORS flow.
Here's the answer you found again:
Just in case anybody else has this same problem, here is how to solve it. Go to https://www.getpostman.com/docs/capture in your chrome browser. Click on interceptor extension and then choose add to chrome. Once it is added there is a new icon top right of both the browser and postman that looks like a traffic light. In postman click this and it turns green.
... With the bit in bold translated:
Then add a header to your request. The header Key should be "Origin" and the header Value should be the full URL of your server (Do not forget the http:// or https://).
Note that Chrome/Postman won't allow you to add a Header with a Key of Origin without the Interceptor plugin.
Also note that at least on my system the Interceptor icon no longer looks like a traffic light.
If your back-end service side code checks for origin of the request (just to avoid CORS attack) you may face this issues when testing your Rest API through postman.
How to Resolve this .?
You need to install a Chrome plugin called Postman Interceptor (https://chrome.google.com/webstore/detail/postman-interceptor/aicmkgpgakddgnaphhhpliifpcfhicfo?hl=en).
After successfully installing this plugin , in you Postman client you can see small icon called Postman Interceptor , you need to toggle it to turn it on.
Now you can add a Request header as below
RequestHeader Key "Origin"
RequestHeader Value "your application base URL"
Check this image
Now you should be able to over come CORS issues you are facing
Cheers !!
Just avoid using browser/chrome postman plugin. Use the desktop application instead!
Seems our server is seeing from a Postman manual HTTP POST that the orgin is invalid b/c its coming from Postman as "chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop"
Not sure why or how to resolve on client/Postman side. Seems our server is correclty rejecting it as is though and issuing a 403.
Value of "Origin" header set in Postman request should be allowed in API backend. For example, using Spring Boot for API should have next:
#Configuration
public class WebConfig implements WebMvcConfigurer {
#Value("${cors.allowedOrigins}")
private String allowedOrigins;
#Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**")
.allowedOrigins(allowedOrigins)
.allowedMethods("*")
.allowedHeaders("*");
}
}
where allowedOrigins is set using application.properties property cors.allowedOrigins having comma separated list of allowed origins, eg:
cors.allowedOrings=http://localhost:8080,http://example.com
and set 'Origin' value in Postman to any url from cors.allowedOrigins
I was getting this error when testing my APIs on the postman. Even after meticulously configuring my cors. So I used Insomnia instead of Postman and it works fine. I guess sometimes postman is the problem as it needs some extra effort.
You can try new version of PostMan. To me it works after upgraded postman version from 5.5.5 to 7.36.5
There is no authentication on server side so authentication should not be issue.
URL format: PUT
https://localhost/api/v1/protections?integrationKey=111&userKey=1111&group=111&category=foo
Payload:
{"action":"BLOCK"}
This is working fine in Postman.
In SOAP UI , I am giving input as under:
EndPoint: https://localhost
Resource: /api/v1/protections
Parameters:?integrationKey=111&userKey=1111&group=111&category=foo
in Media type, I am selecting "application/json"
and entering {"action": "BLOCK"} but getting "Wed Jan 20 16:25:27 PST 2016:DEBUG:Receiving response: HTTP/1.1 403 Forbidden
"
Is there any suggestion to get the output in SOAP UI.
Depending on the server where the rest is exposed service generates an HTTP 403, you should verify that server is and thus find the fastest response.
Also try making a GET request from the browser to see if you can answer correctly because problem lock your machine to the server.
As is https, it may be that you lack some certificate set SOAPUI. possibly Postman you use already has configured. Try to check this setting.
In my case, I missed the Header "User-Agent" and "accept". I put in Soap UI and Works.
In Postman, this headers it put automatically.
The Docker Registry API Documentation (here) states that you can do:
https://index.docker.io/v1/images/088b4505aa3adc3d35e79c031fa126b403200f02f51920fbd9b7c503e87c7a2c/ancestry
However this (with several image ids I have tried) only yields a 404. Why?
You are using the incorrect domain. Note from the documentation example:
GET /v1/images/088b4505aa3adc3d35e79c031fa126b403200f02f51920fbd9b7c503e87c7a2c/ancestry HTTP/1.1
Host: registry-1.docker.io
Accept: application/json
Content-Type: application/json
Cookie: (Cookie provided by the Registry)
You should be using registry-1.docker.io, not index.docker.io. So your url should be:
https://registry-1.docker.io/v1/images/088b4505aa3adc3d35e79c031fa126b403200f02f51920fbd9b7c503e87c7a2c/ancestry
When I run this I no longer get a 404 but instead get an authorization error. This is because you need to auth against index.docker.io and then run your api against registry-1.docker.io.
It's not well documented - you may find this discussion helpful:
https://forums.docker.com/t/registry-hub-api-is-buggy/732
from the discussion:
registry-1.docker.io is the docker-registry used for storing images,
this API is public, but requires authorization.
index.docker.io is the "index" now called RegistryHub, this tags
endpoint is not public, and might change at any time. I would not use
this one, since it is for internal use only, and may change without
warning.
I've recently started using soapUI 4.5.1 and I'm still not familiar with all the settings. I've set up a few web requests and all but one are working correctly. I'm trying to send content of type application/xml but I'm getting:
org.apache.http.client.ClientProtocolException caused by org.apache.http.ProtocolException: Content-Length header already present.
The same request always worked fine for me in 4.5.0. Content of request is something like this:
POST http://exampleHost.com/exampleRequest HTTP/1.1
Accept-Encoding: gzip,deflate
Accept: text/xml
Content-Type: application/xml
Content-Length: 456
Host: exampleHost.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
Followed by the xml.
I take it that content length is sent twice, but I don't know where, because I haven't set it anywhere. SoapUI is showing Additional HTTP Headers for this message as 0.
Any pointers would be great!
I got the error to go away (in SoapUI 4.5.1) by checking the Authenticate Preemptively flag in
Preferences -> HTTP Settings -> Authenticate Preemptively
I got in touch with SmartBear support, the problem seems to have been fixed in the latest nightly build available # http://soapui.org/Downloads/soapui-pro-nightly-builds.html.
I encountered the same problem using an authenticating development server that was requesting user credentials but actually accepted a blank or any other password. So I was leaving the password blank which worked fine in soapUI 4.5.0, but failed the way you described in 4.5.1. But I found that simply putting some text into the password appears to fix the problem.
Don't know if this relates to your case, but just in case it is useful.
I faced the same issue with Soap UI pro version 4.5.1.
Finally figured out that the issue was with proxy settings.
Resolution:
Adding the target server in Exclude List for Proxy Settings has
resolved the issue. This is the case even if the end point url is a
local host.
Preferences --> Proxy Settings ---> Exclude
can specify mutliple servers by comma seperated values
Research / Observation on my system:
Strangely the same test suite runs without an issue on other system
within same office. Must be something to do with the way systems are
configured.
Playing with Proxy settings in Internet Options has no
effect.
For my request, Proxy authentication was required. But when
i enable Proxy settings i get Http Client protocol exception with
duplicate content length error.
We can find this from the http log
once the request is sent...But we don't have an option to configure
it.
An interesting observation was that one of the content length
header was in Incoming Request and the other one was in Outgoing
Request...This shouldn't throw off the request though.
Another way to cause this error is to call the web service with the wrong password (I was told wrong honest) too many times and get your account locked.
As soon as the password was reset and the account unlocked the "org.apache.http.client.ClientProtocolException caused by org.apache.http.ProtocolException: Content-Length header already present" exception went away and the web service call worked as expected.