When calling rest API with http.NewRequest, the response body is garbled - api

I try to call an API with Go. When using Postman everything is OK. But if I use the Go code from Postman the response is garbled/unclear.
Down below the code I'm using:
func CallAPI() {
url := "https://url"
req, _ := http.NewRequest("GET", url, nil)
req.Header.Add("Authorization", "Bearer Token is normaly here")
req.Header.Add("User-Agent", "PostmanRuntime/7.19.0")
req.Header.Add("Accept", "Accept: application/json")
req.Header.Add("Cache-Control", "no-cache")
req.Header.Add("Postman-Token", "Postman token normaly here")
req.Header.Add("Host", "host normaly here")
req.Header.Add("Accept-Encoding", "gzip, deflate")
req.Header.Add("Connection", "keep-alive")
req.Header.Add("cache-control", "no-cache")
res, _ := http.DefaultClient.Do(req)
defer res.Body.Close()
body, _ := ioutil.ReadAll(res.Body)
fmt.Println(string(body))
}
The response I get when I use fmt.Println(string(body)) looks like below. I also tried other API's with this code and had the same result.
r�痱�
I also tried to unmarshal the json to a struct and did get the following error
Invalid character '\x1f' looking for beginning of value
I think it's something about decoding. But I don't know what.

You ask the server to send the content compressed (req.Header.Add("Accept-Encoding", "gzip, deflate")), and that's what you get: a gzip response, indicated by the response header: Content-Encoding:[gzip].
Remove that header (don't set Accept-Encoding request header), and you should get plain JSON response. Or decode the gzip response yourself.
Note that if you omit this header, the default transport will still request gzip encoding, but then it will also transparently decode it. Since you request it explicitly, transparent, automatic decoding does not happen. This is documented at Transport.DisableCompression field:
// DisableCompression, if true, prevents the Transport from
// requesting compression with an "Accept-Encoding: gzip"
// request header when the Request contains no existing
// Accept-Encoding value. If the Transport requests gzip on
// its own and gets a gzipped response, it's transparently
// decoded in the Response.Body. However, if the user
// explicitly requested gzip it is not automatically
// uncompressed.
DisableCompression bool

Related

Request header field authorization

I am trying to create "UUID" form this Api call
I am using axios and vue.js. Here's my source code.
generateUUID() {
// console.log("call generate uuid");
const headers = {
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Origin': '*'
}
axios.get('https://www.uuidtools.com/api/generate/v1', {
headers: headers
}).then(response => console.log(response))
},
But unfortunately I am getting
Access to XMLHttpRequest at 'https://www.uuidtools.com/api/generate/v1' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
I have tried searching google and followed many answers already to that question but got nothing.
Problem(s)
Access-Control-Allow-Headers and Access-Control-Allow-Origin are response headers, not request headers. Specifying them in a request is more than useless; it's actually counterproductive, here.
Why would you specify a content type for a GET request? GET requests are not supposed to have a body:
A payload within a GET request message has no defined semantics;
sending a payload body on a GET request might cause some existing
implementations to reject the request.
According to my tests, the resource that you're requesting (https://www.uuidtools.com/api/generate/v1) appears to be CORS-aware but expects only simple requests, not preflight requests. And because your request contains headers that are not CORS-safelisted, it gets preflighted by your browser, and the preflight access-control check fails.
Solution
Just send a simple GET request (without adding any headers):
fetch('https://www.uuidtools.com/api/generate/v1')
.then(r => r.json())
.then(arr => arr[0])
.then(console.log)
Console output
No CORS error, then:
]

Graphql post body "Must provide query string."

I use Express-graphql middleware.
I send the following request in the body line:
POST /graphql HTTP/1.1
Host: local:8083
Content-Type: application/graphql
Cache-Control: no-cache
Postman-Token: d71a7ea9-5502-d5fe-2e36-0ae49c635a29
{
testing {
pass(id: 1) {
idn
}
}
}
and have error
{
"errors": [
{
"message": "Must provide query string."
}
]
}
in graphql i can send update in URL.
URL string is too short. i must send update model like
mutation {
update(id: 2, x1: "zazaza", x2: "zazaza", x3: "zazaza" ...(more more fields)...) {
idn
}
}
I think its must be in request body. How can I send 'update' query or that I'm doing wrong?
Post request needs to manage headers info.
Using Http client - Content-Type: application/json
Using Postman client - Content-Type: application/graphql
but request body looks like string
{"query":"mutation{update(id:1,x1:\"zazaz\",x2:\"zazaz\"......){id x1 x2}}"}
If you are using graphql and want to test it using postman or any other Rest client do this.
In postman, select POST method and enter your URL and set Content-Type as application/graphql then pass your query in the body.
Example:
http://localhost:8080/graphql
Mehtod: POST
Content-Type: application/graphql
Body:
query{
FindAllGames{
_id
title
company
price
year
url
}
}
Thats it you will get the response.
Using Postman Version 7.2.2 I had a similar issue. This version of Postman supports Graphql out of the box. Changing the Content-type to application/json fixed it for me.
for me worked like as following:
In the body
In the Headers
Don't forget mark GraphQl [x] on Body settings
And how was quoted before changes the verb to POST.
This generally occurs when your 'express-graphql' doest receive any params. You need to added a json/applicaton parser in your application.
npm install body-parser
eg -
const bodyParser = require('body-parser');
app.use(bodyParser.json()); // application/json
go to the relevant web page and open "inspect" (by write click ->
inspect || Ctrl+Shift+I in chrome)
go to the network tab and copy the cURL command
open the postman ,then import -> raw text
paste the copied command
then,continue ->
Switch content type to JSON.
Like this
Check if you are using correct protocol in your Postman requests.
I used HTTP instead of HTTPS and this caused the same error.
Changes of content-type, raw or json instead of graphql type didn't help.

How to set Content-Type header in a http get response using C#'s HttpClient?

I am having problems in retrieving the contents of a http get request in the proper charset.
I tried several pieces of code, such as the following:
HttpClient h = new HttpClient();
//Content-Type: text/html; charset=UTF-8
//p.s. contents are in hebrew.
var resp = h.GetAsync("http://www.wnf.co.il");
var content = resp.Result.Content;
//remove the default Content-Type header
content.Headers.Remove("Content-Type");
content.Headers.Add("Content-Type", "text/html; charset=utf-8");
var res = content.ReadAsStringAsync();
var s = res.Result;
Console.WriteLine(s);
which still does not help, I still get the content in wrong encoding.
This post clarifies that setting the header's request headers charset will not help, it's the response's one that needs to be set. (Besides, you will get an error in trying to add
header "Content-Type" to a request Header.)
But I still could not end up with working retrieval of the content in the proper charset (utf-8).
What am I missing ?
I have been doing similar stuff with hebrew sites for a while, in comparing the response's header in Fiddler from this site and others where I do not have this problem - the only difference I see is indeed this Content-Type header in the response.
The issue is probably due to this bug:
https://connect.microsoft.com/VisualStudio/feedback/details/790174/system-net-http-httpcontent-readasstringasync-does-not-handle-imperfect-content-type-headers
The work-around is to get the response as a byte array and encode it yourself:
var bytes = await content.ReadAsByteArrayAsync();
var s = Encoding.UTF8.GetString(bytes, 0, bytes.Length);
As a side-note, is there a reason you're using .Result instead of await? You are blocking the current thread unnecessarily and setting yourself up for deadlocks.

Go - middleware blocks MIME type of each requests

I modiefied the middleware of this tutorial to check for all PUT and POST request the JSON MIME type.
But the middleware seems to respond everytime with "Mediatype Not Supported". I tried the curl command below where I set explicitly the right MIME type. I print which each request the client's Content-Type header field which is allways "text/plain; charset=utf-8".
The Middleware:
func EnforceJSON(h httprouter.Handle) httprouter.Handle {
return func(rw http.ResponseWriter, req *http.Request, ps httprouter.Params) {
// Check the existence of a request body
if req.ContentLength == 0 {
http.Error(rw, http.StatusText(400), http.StatusBadRequest)
return
}
// Check the MIME type
buf := new(bytes.Buffer)
buf.ReadFrom(req.Body)
// Prints "text/plain; charset=utf-8"
fmt.Println(http.DetectContentType(buf.Bytes()))
if http.DetectContentType(buf.Bytes()) != "application/json; charset=utf-8" {
http.Error(rw, http.StatusText(415), http.StatusUnsupportedMediaType)
return
}
h(rw, req, ps)
}
}
...
router.POST("/api/v1/users", EnforceJSON(CreateUser))
My curl command:
curl -H "Content-Type: application/json; charset=utf-8" \
-X POST \
-d '{"JSON": "Will be checked after the middleware accepted the MIME type."}' \
http://localhost:8080/api/v1/users
Alternativly I tried Postman but the result was the same.
Looking at the implementation of DetectContentType function they follow the rules described on "MIME Sniffing" guide (https://mimesniff.spec.whatwg.org/).
To determine the MIME Type by the content this specific function use the item "7. Determining the computed MIME type of a resource". That item doesn't have a determination for 'application/json' type and following these rules will result in a 'text/plain' MIME type.
This is visible on the function match on the DetectContentType implementation (https://golang.org/src/net/http/sniff.go) line 252.

Invalid 'HttpContent' instance provided. It does not have a 'multipart' content-type header with a 'boundary' parameter

I'm writing a web API that has a post method accepting files uploaded from UI.
public async Task<List<string>> PostAsync()
{
if (Request.Content.IsMimeMultipartContent("form-data"))
{
string uploadPath = HttpContext.Current.Server.MapPath("~/uploads");
var streamProvider = new MyStreamProvider(uploadPath);
await Request.Content.ReadAsMultipartAsync(streamProvider);
return streamProvider.FileData
.Select(file => new FileInfo(file.LocalFileName))
.Select(fi => "File uploaded as " + fi.FullName + " (" + fi.Length + " bytes)")
.ToList();
}
else
{
HttpResponseMessage response = Request.CreateResponse(HttpStatusCode.BadRequest, "Invalid Request!");
throw new HttpResponseException(response);
}
}
Then I post a request for the above action by postman.
I set the content-type header to multipart/form-data
but an error occurred during the execution of action.
here is the error message body :
"Invalid 'HttpContent' instance provided. It does not have a 'multipart' content-type header with a 'boundary' parameter.\r\nParameter name: content"
I went to the postman headers but I found that the request header content type was set to application-json.
You are looking on the response header which is json format and this is ok for you.
Your real problem is with the postman request, so just remove the 'Content-Type: multipart/form-data' entry from request header.
It's enough to upload a file as form-data and send the request.
Look what happen when you set the Content-Type manually vs. when you not:
Postman knows to set both the content type and boundary, since you set only the content type
First: Postman have a bug in handling file-based requests.
You can try adding this to your WebApiConfig.cs it worked for me:
GlobalConfiguration.Configuration.Formatters.XmlFormatter.SupportedMediaTypes.Clear();