Bad request with base64 encoded parameter in WCF rest url on IIS 7 - wcf

I have a rest wcf 4.0 service that takes in a base64 encoded string as a parameter. It works properly when I run the service in Visual Studio using casini, but gives an Error 400 message when I run the same service under IIS 7.5. The break point doesn't get hit. I guess its not a problem with the code because it works under casini with the same code and web.config.
What changes do I need to make to enable the service to function under IIS 7.5?
This is the URL
http://localhost/MyServices/MyServ.svc/Accept/eyJXb3JkY291bnQiOjMwLCJJbWFnZWNvdW50IjoxMCwiU2VsZWN0ZWRMYW5ndWFnZXMiOlt7ImlkIjoxLCJsYW5ndWFnZTEiOiJIaW5kaSIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoiTWFyYXRoIiwiR3JvdXBfTGFuZ3VhZ2VzX2lkIjpudWxsfSx7ImlkIjoxLCJsYW5ndWFnZTEiOiJGcmVuY2giLCJHcm91cF9MYW5ndWFnZXNfaWQiOm51bGx9LHsiaWQiOjEsImxhbmd1YWdlMSI6IkdFcm1hbiIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoiSXRhbGlhbiIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoic3BhbmlzaCIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH1dfQ==
Thanks.

The equal sign has a special meaning in the URL. It separates a parameter key from its value. The way you use the trailing equal signs, IIS will reject it.
Note that the Base 64 encoding uses additional characters that have a special meaning in a URL and cause problems (namely + and /). Therefore I recommend to use a modified Base 64 encoding that uses only URL safe characters.
Typically, the following characters are replaced:
+ with -,
/ with _, and
= with *.
Update:
The Javascript code you use for Base 64 encoding is easy to modify. Just replace this line:
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
with
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_*";
And
return (r ? enc.slice(0, r - 3) : enc) + '==='.slice(r || 3);
with:
return (r ? enc.slice(0, r - 3) : enc) + '***'.slice(r || 3);
Update 2:
I'm afraid the Base 64 URLs are never going to work with IIS. IIS still treats every part of an URL as a file or directory name. There are many problems with this implementation (just try COM2 or PRN as a URL segment).
You'll need to come up with a different URL scheme or request type. Either use a POST request to transmit the Base64 encoded part or use a URL like:
http://server/MyServices/MyServ.svc/Accept?data=eyJXb3JkY291bnQiOjMwLCJ...VsbH1dfQ**
But the second case will only work if the URL isn't too long. It wouldn't expect URLs of more than 1000 characters to work.

Related

Call to java script code returning the ASCII encoding for ':' separating key and value of returned object

I making an api request using karate where one of the api request params takes a filter condition (which is a java script object).
I am using a literal notation to create a java script object as shown below. This code is in a separate filter.js file.
function() {
var params = {
val1:[],
val2:[]
};
return params;
}
Now i call the above .js file in the karate scenario as below:
Scenario: Test
Given path 'filtertest/'
* param filter = call read('classpath:feature/common/filter/filter.js')
When method get
Ran the above and when i check the log, the api throws bad request error. I looked at the request url and there i can see that the ':' in the js file where I am assigning a value to a object key is replaced with %3A which i believe is the ASCII encoding for ':'. (the param with its values below)
?filter=%7B%22val1%22%3A%5B%5D%2C%22val2%22%3A%5B%5D
What I want is the ':' to come as it is from the .js call as the server side expects the filter param values as key value pairs.
Is there a way I can achieve this?
If your server cannot decode an encoded : it is a bug: https://www.w3schools.com/tags/ref_urlencode.asp
If you really need this - the workaround is to use the url keyword and build it manually, path and param will always encode.
Given url baseUrl + '/filtertest?filter=foo:bar'

HttpRequestMessage - Decoding 'parameter' - c# net40

I am integrating with a third party API via a PUT request (required). However the URL is not what I consider standard.
https://standard.com/services/U2FsdGVkX18qfVhkTn7JsiXWA8q4vco7vpU%2BmeKqiXKafdKxfJqsq1ELGFOHIilpoR7VXqUcg89yXiabxjjfVXiRmVqGOffsKw%2BKLp5OU6%2FJZaTyn6BYxi%2F10ndtWFLG2KgxRmm%2BxtgTAopUi6m7wWTnSoAlL8qoS%2F3UbiippOw%3D.
Note that the reservation number isn't in the body nor as a parameter, its just a / then the number.
They have encoded the reservation number (requested via another API call). The issue is I'm using HttpRequestMessage, this decodes the %2F to /. Meaning it fails. I have tried double encoding the special characters but this isn't returning a consistent response, sometimes it fails other times it works (using different reservation numbers, only the %2F is double encoded). Is there a way to stop HttpRequestMessage decoding part of the URL?
I have tried encoding/decoding but it always gets stripped out here (unless double encoded:
var path = "https://standard.com/services/U2FsdGVkX18qfVhkTn7JsiXWA8q4vco7vpU%2BmeKqiXKafdKxfJqsq1ELGFOHIilpoR7VXqUcg89yXiabxjjfVXiRmVqGOffsKw%2BKLp5OU6%2FJZaTyn6BYxi%2F10ndtWFLG2KgxRmm%2BxtgTAopUi6m7wWTnSoAlL8qoS%2F3UbiippOw%3D";
var requestMessage = new HttpRequestMessage(HttpMethod.PUT, path);
var requestMessage = (HttpWebRequest)WebRequest.Create(path);
var requestMessage = new HttpRequestMessage(method, new Uri(path));
Update: This was a design decision made by Microsoft:
https://connect.microsoft.com/VisualStudio/feedback/details/511010/erroneous-uri-parsing-for-encoded-reserved-characters-according-to-rfc-3986
The solution from their point of view is to add a setting in the
web.config. But this is a class library and therefore used by more
than one project. I don't want to alter that unless I really have to
This was a design decision made by Microsoft: https://connect.microsoft.com/VisualStudio/feedback/details/511010/erroneous-uri-parsing-for-encoded-reserved-characters-according-to-rfc-3986

URL encoded colon (%3A) resolves in 403

I'm hosting some rest APIs in an Apache server (a typical xampp package).
When I attack a resource with an http method (whichever: put, post, get...) and the URL contains an encoded colon %3A, the server sometimes replies with a 403 error. It seems to depend on the folder structure of the server. If there's an existing folder and your url attacks a resource that contains %3A in that folder, the server returns 403. When it doesn't contain %3A, it returns 404 like it should.
with a structure like this:
htdocs/apis/userContext
htdocs/apis/subscriptions
http://localhost/apis/userContext/users/tel%3A2032342349 Works (returns 404 not found, because users doesn't exist)
http://localhost/apis/userContext/tel%3A2032342349 Doesn't work (returns 403)
http://localhost/subscriptions/tel%3A2032342349 Doesn't work (returns 403)
http://localhost/nonexistingfolder/tel%3A2032342349 works (returns 404, becasue nonexistingfolder doesn't exist)
It's quite annoying since a lot of the values that are going to appear in the urls are telURIs and look like this
tel:+34678678678
so please don't tell me not to use colons there because that's simply impossible. How could I fix this?
I was able to find only one valid information about this (from October 2006) - here.
Apparently, this error doesn't appear on Linux, it happens on Windows platforms. According to the source, a short name, followed by colon (:) could be interpreted as a drive name (why is that and why file:// doesn't denote that - I don't know).
I had to solve this issue, but I could set the links myself (don't know if You have the luxury). My solution was to:
replace colons in URI by a character of choice (I chose the underscore) - urlencode($uri) → urlencode(strtr($uri, ":", "_"))
search for those URIs after applying the same replacement - SELECT ... FROM ... WHERE uri = REPLACE(##uri, ":", "_")

Problems Connecting to MtGox API 2 with Python

I am writing a trading program that I need to connect to MtGox (a bitcoin exchange) through the API v2. But I keep getting the following error:
URL: 1 https://data.mtgox.com/api/2/BTCUSD/money/bitcoin/address
HTTP Error 403: Forbidden.
Most of my script is a direct copy from here (that is a pastebin link). I just had to change it to work with Python 3.3.
I suspect that it has to do with the part of script where I use base64.b64encode. In my code, I have to encode my strings to utf-8 to use base64.b64encode:
url = self.__url_parts + '2/' + path
api2postdatatohash = (path + chr(0) + post_data).encode('utf-8') #new way to hash for API 2, includes path + NUL
ahmac = base64.b64encode(str(hmac.new(base64.b64decode(self.secret),api2postdatatohash,hashlib.sha512).digest()).encode('utf-8'))
# Create header for auth-requiring operations
header = {
"User-Agent": 'Arbitrater',
"Rest-Key": self.key,
"Rest-Sign": ahmac
}
However, with the other guy's script, he doesn't have too:
url = self.__url_parts + '2/' + path
api2postdatatohash = path + chr(0) + post_data #new way to hash for API 2, includes path + NUL
ahmac = base64.b64encode(str(hmac.new(base64.b64decode(self.secret),api2postdatatohash,hashlib.sha512).digest()))
# Create header for auth-requiring operations
header = {
"User-Agent": 'genBTC-bot',
"Rest-Key": self.key,
"Rest-Sign": ahmac
}
I'm wondering if that extra encoding is causing my header credentials to be incorrect. I think this is another Python 2 v. Python 3 problem. I don't know how the other guy got away without changing to utf-8, because the script won't run if you try to pass a string to b64encode or hmac. Do you guys see any problems with what I am doing? Is out code equivalent?
This line specifically seems to be the problem -
ahmac = base64.b64encode(str(hmac.new(base64.b64decode(self.secret),api2postdatatohash,hashlib.sha512).digest()).encode('utf-8'))
To clarify, hmac.new() creates an object to which you then call digest(). Digest returns a bytes object such as
b.digest()
b'\x92b\x129\xdf\t\xbaPPZ\x00.\x96\xf8%\xaa'
Now, when you call str on this, it turns to
b'\\x92b\\x129\\xdf\\t\\xbaPPZ\\x00.\\x96\\xf8%\\xaa'
So, see what happens there? The byte indicator is now part of the string itself, which you then call encode() on.
str(b.digest()).encode("utf-8")
b"b'\\x92b\\x129\\xdf\\t\\xbaPPZ\\x00.\\x96\\xf8%\\xaa'"
To fix this, as turning bytes into a string back into bytes was unnecessary anyhow(besides problematic), I believe this will work -
ahmac = base64.b64encode(hmac.new(base64.b64decode(self.secret),api2postdatatohash,hashlib.sha512).digest())
I believe you are likely to find help in a related question of mine although it deals with the WebSocket API:
Authenticated call to MtGox WebSocket API in Python 3
Also, the HTTP 403 error seems to indicate that there is something fundamentally wrong with the request. Even if you threw the wrong authentication info at the API you should have gotten an error message as a response and not a 403. My best guess is that you are using the wrong HTTP method so check if you are using the appropriate one (GET/POST).

Problems with Delphi Rest Client talking to C# Rest server

I am writing a REST WCF Service, and have it working for connections from a C# client, but am having issues with connecting via a Delphi 2009 client. The problem I have is specifically with a PUT request, which looks (for the moment) as below. It expects an XML request, containing the document's objects.
[WebInvoke(UriTemplate = "Document/{id}", Method = "PUT", RequestFormat=WebMessageFormat.Xml)]
public void UpdateDocument (string id, Document document)
{
Document doc = document;
// this should update or something!
Console.WriteLine(doc.Id);
}
When I try and call this via my Delphi client (as shown below), I get a 'Bad Request'. Oddly if I send an empty document, the request is received, but obviously has no data.
...
msg := '<?xml version="1.0"?>' +
'<Document>' +
'<Id>123456788888</Id>' +
'</Document>';
XMLDocument1.LoadFromXML(msg);
xmlStream := TMemoryStream.Create;
idHttp1.Request.ContentType := 'application/xml';
XMLDocument1.SaveToStream(xmlStream);
url := 'http://localhost:50435/service1/Document/12345678';
result := idHttp1.Put(url, xmlStream);
ShowMessage (result);
...
Any ideas, as I am a bit lost now!
Thanks
The mapping of the URI to server method is not correct. According to default mapping a PUT request will invoke AcceptDocument method, and a POST request will invoke UpdateDocument method.
http://docwiki.embarcadero.com/RADStudio/en/REST
Also passing a TStream as parameter is also possibility for difficulties. Is it possible for you to use JSON? I am not sure Delphi 2009 has support for JSON though.
Edit:
quote from Delphi documentation:
by default, a prefix of 'update' is assigned to any method invoked
with POST. Similarly, a prefix of 'cancel' is used for DELETE
requests, and a prefix of 'accept' is used for PUT requests. This
prefixing can be avoided by putting quotation marks around the method
name
Above quote applies to Delphi. Maybe also for WCF.
As per my comment above, installing Fiddler made me look at the actual page that the webservice outputs, this helpfully has example XML and json structures. I was missing the namespace for the object, so I guess the WCF end didn't know what to translate, changing the message to the following made it work:
msg := '<?xml version="1.0"?>' +
'<Document xmlns="http://schemas.datacontract.org/2004/07/Contracts.Contracts">' +
'<Id>123456788888</Id>' +
'</Document>';