play framework WS API always escape character ';' and '=' in URL - api

When I send a URL like abc.efg.com/query?para1=cat;para2=dog, play WS API always convert it to abc.efg.com/query?para1=cat%03Bpara2%03Ddog. Of course, there are http:// in the beginning in the URL. my code is as below.
val url= "http://abc.efg.com/query?para1=cat;para2=dog"
val response = WS.url(url).get()
When I use fidder or netmon to look at the data that sent to sever, I found play framework WS (2.1.5) always change to the URL above I mentioned. How do I tell WS not to convert?

The semicolon is an alternate syntax to ampersand (&) as a seperator for query parameters, so it must be encoded.
If you want multiple query parameters, you should use withQueryString.

Related

Getting illegal character when sending GET request in JMETER. Working fine in browser

Im getting illegal character in JMETER for GET request -
https://dev1/api/v1/query/job/?filter={%22job_manager_id%22:%22553f2350-12d3-4252-8fe0-39691019c495%22}
tried replacing %22 with "" but still getting illegal character.
Any solutions ?
I think problematic characters are { and }, they need to be percent-encoded
The options are in:
Tick "URL-encode" box next to the filter parameter in the HTTP Request sampler :
Use __urlencode() function in "Path" field like
see Apache JMeter Functions - An Introduction article to learn more about JMeter Functions concept
Or just hard-code the percent-encoded Path part like:
https://dev1/api/v1/query/job/?filter=%7B%22job_manager_id%22%3A%22553f2350-12d3-4252-8fe0-39691019c495%22%7D

Special characters parsing in Ktor

I am trying to parse some special characters through the api's with Ktor in a KMM project but it seems like Ktor doesn't support it ?.
For eg :- I am trying to parse "Saut� Pans" here "�" is a special character and Ktor throws an exception saying :- io.ktor.utils.io.charsets.MalformedInputException: Input length = 1. Now I have tried passing different headers for example passing different ContentType to UTF-8 or ISO-8859-2 but didn't help.
Whereas parsing the same api response with Gson() works for me.
Any solution or workaround so that for now so that I can parse such kind of special characters in my response ?
You could try out
install(ContentEncoding){
gzip()
}

How to properly encode special characters in a REST API url

EDIT: The NHTSA docs, as CBroe points out, say to replace an ampersand with an underscore. However, I'm also getting an error with forward slashes (albeit a different one, page not found, because it's decoding the slash), for example the make 'EXISS/SOONER':
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/EXISS%2FSOONER?format=json
And replacing the ampersand with an underscore no longer results in an error message, but in zero results returned, which should not be the case.
ORIGINAL POST:
I'm trying to download the content from the following URL:
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s&s?format=json
And the site returns the following error message:
Server Error in '/' Application.
A potentially dangerous Request.Path value was detected from the client (&).
The problem is the ampersand; a similar request for a different car make works:
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/toyota?format=json
I have verified from a different endpoint that S&S is a valid make for the API.
Based on stackoverflow answers, I've tried all the following without success:
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s%26s?format=json
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s%26amp;s?format=json
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s%26amp%3Bs?format=json

Bad request with base64 encoded parameter in WCF rest url on IIS 7

I have a rest wcf 4.0 service that takes in a base64 encoded string as a parameter. It works properly when I run the service in Visual Studio using casini, but gives an Error 400 message when I run the same service under IIS 7.5. The break point doesn't get hit. I guess its not a problem with the code because it works under casini with the same code and web.config.
What changes do I need to make to enable the service to function under IIS 7.5?
This is the URL
http://localhost/MyServices/MyServ.svc/Accept/eyJXb3JkY291bnQiOjMwLCJJbWFnZWNvdW50IjoxMCwiU2VsZWN0ZWRMYW5ndWFnZXMiOlt7ImlkIjoxLCJsYW5ndWFnZTEiOiJIaW5kaSIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoiTWFyYXRoIiwiR3JvdXBfTGFuZ3VhZ2VzX2lkIjpudWxsfSx7ImlkIjoxLCJsYW5ndWFnZTEiOiJGcmVuY2giLCJHcm91cF9MYW5ndWFnZXNfaWQiOm51bGx9LHsiaWQiOjEsImxhbmd1YWdlMSI6IkdFcm1hbiIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoiSXRhbGlhbiIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoic3BhbmlzaCIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH1dfQ==
Thanks.
The equal sign has a special meaning in the URL. It separates a parameter key from its value. The way you use the trailing equal signs, IIS will reject it.
Note that the Base 64 encoding uses additional characters that have a special meaning in a URL and cause problems (namely + and /). Therefore I recommend to use a modified Base 64 encoding that uses only URL safe characters.
Typically, the following characters are replaced:
+ with -,
/ with _, and
= with *.
Update:
The Javascript code you use for Base 64 encoding is easy to modify. Just replace this line:
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
with
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_*";
And
return (r ? enc.slice(0, r - 3) : enc) + '==='.slice(r || 3);
with:
return (r ? enc.slice(0, r - 3) : enc) + '***'.slice(r || 3);
Update 2:
I'm afraid the Base 64 URLs are never going to work with IIS. IIS still treats every part of an URL as a file or directory name. There are many problems with this implementation (just try COM2 or PRN as a URL segment).
You'll need to come up with a different URL scheme or request type. Either use a POST request to transmit the Base64 encoded part or use a URL like:
http://server/MyServices/MyServ.svc/Accept?data=eyJXb3JkY291bnQiOjMwLCJ...VsbH1dfQ**
But the second case will only work if the URL isn't too long. It wouldn't expect URLs of more than 1000 characters to work.

' character in string parameter of Odata Url

Hmmm this seems silly, so I feel silly. I have a custom operation on an Odata service which has a string parameter. It works fine when the string contains no special characters, but as soon as I include the ['] character I get:
<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"yes\"?>\r\n<error xmlns=\"http://schemas.microsoft.com/ado/2007/08/dataservices/metadata\">\r\n <code></code>\r\n <message xml:lang=\"en-US\">Bad Request - Error in query syntax.</message>\r\n</error>
I am invoking the service with the following code:
new DataServiceContext(new Uri(svcUri)).Execute<string>(new Uri(relativeOperationUrl, UriKind.Relative));
I've tried encoding like this [& apos;] (xml encoding) and like this [%27] (url encoding) and the things is still blowing up.
Try '' (like escaping it for SQL); see How to escape a single quote to be used in an OData query?.