URL encoded colon (%3A) resolves in 403 - apache

I'm hosting some rest APIs in an Apache server (a typical xampp package).
When I attack a resource with an http method (whichever: put, post, get...) and the URL contains an encoded colon %3A, the server sometimes replies with a 403 error. It seems to depend on the folder structure of the server. If there's an existing folder and your url attacks a resource that contains %3A in that folder, the server returns 403. When it doesn't contain %3A, it returns 404 like it should.
with a structure like this:
htdocs/apis/userContext
htdocs/apis/subscriptions
http://localhost/apis/userContext/users/tel%3A2032342349 Works (returns 404 not found, because users doesn't exist)
http://localhost/apis/userContext/tel%3A2032342349 Doesn't work (returns 403)
http://localhost/subscriptions/tel%3A2032342349 Doesn't work (returns 403)
http://localhost/nonexistingfolder/tel%3A2032342349 works (returns 404, becasue nonexistingfolder doesn't exist)
It's quite annoying since a lot of the values that are going to appear in the urls are telURIs and look like this
tel:+34678678678
so please don't tell me not to use colons there because that's simply impossible. How could I fix this?

I was able to find only one valid information about this (from October 2006) - here.
Apparently, this error doesn't appear on Linux, it happens on Windows platforms. According to the source, a short name, followed by colon (:) could be interpreted as a drive name (why is that and why file:// doesn't denote that - I don't know).
I had to solve this issue, but I could set the links myself (don't know if You have the luxury). My solution was to:
replace colons in URI by a character of choice (I chose the underscore) - urlencode($uri) → urlencode(strtr($uri, ":", "_"))
search for those URIs after applying the same replacement - SELECT ... FROM ... WHERE uri = REPLACE(##uri, ":", "_")

Related

How to escape out a backslash in API Request

I'm trying to hit the stock item WSE ("~entity/Default/20.200.001/StockItem/{keyvalue}") but when I have a InventoryCD with a slash in it (eg: "WD-ON/OFF") and I run it, I get the error Operation is not valid due to the current state of the object.
I tried using WebUtility.UrlEncode("WD-ON/OFF") to replace the / with %2f, but that doesn't seem to do anything. Per a long shot on the Aug forums, also tried to use "//" but that also does nothing.
The exact same code with a non-slash item works just fine.
Thoughts?
You should HTML encode your string
string myEncodedString = HttpUtility.HtmlEncode(filename);
It turns out that I'm not doing anything wrong. Acumatica support has sent this to Engineering, and there will be some future fix. For now, "don't do that" is their answer.

Getting illegal character when sending GET request in JMETER. Working fine in browser

Im getting illegal character in JMETER for GET request -
https://dev1/api/v1/query/job/?filter={%22job_manager_id%22:%22553f2350-12d3-4252-8fe0-39691019c495%22}
tried replacing %22 with "" but still getting illegal character.
Any solutions ?
I think problematic characters are { and }, they need to be percent-encoded
The options are in:
Tick "URL-encode" box next to the filter parameter in the HTTP Request sampler :
Use __urlencode() function in "Path" field like
see Apache JMeter Functions - An Introduction article to learn more about JMeter Functions concept
Or just hard-code the percent-encoded Path part like:
https://dev1/api/v1/query/job/?filter=%7B%22job_manager_id%22%3A%22553f2350-12d3-4252-8fe0-39691019c495%22%7D

Invalid username format in Azure database connection

I am unable to connect with database using psycopg2 module. This may be due to the # character needing escaping, but I have already tried this. Also, it could be related to my password having the "/" character, or the hostname having the "-" character, and I don't know how to escape it (I've already tried escaping using %2f and %2D).
This is the code I have tried with a edited password, so you can see the "special" characters it has, and also the "-" characters in the hostname. Most likely I am doing this wrong.
engine = create_engine('postgresql://username:asd/32foo/T123%40sql-is-a232#sql-is-a232.database.windows.net:5432/database')
The username here is: username,
The password here is: asd/32foo/T123,
The hostame (I think this is) here is: sql-is-a232,
The server here is: sql-is-a232.database.windows.net,
The database name here is: database.
Any help would be greatly appreciated!
Please reference this document: Database Urls:
As the URL is like any other URL, special characters such as those that may be used in the password need to be URL encoded. Below is an example of a URL that includes the password "kx%jj5/g":
postgresql+pg8000://dbuser:kx%25jj5%2Fg#pghost10/appdb
The encoding for the above password can be generated using urllib:
>>> import urllib.parse
>>> urllib.parse.quote_plus("kx%jj5/g")
'kx%25jj5%2Fg'
Examples for common connection styles follow below. For a full index of detailed information on all included dialects as well as links to third-party dialects, see Dialects.
Hope this helps.

How to properly encode special characters in a REST API url

EDIT: The NHTSA docs, as CBroe points out, say to replace an ampersand with an underscore. However, I'm also getting an error with forward slashes (albeit a different one, page not found, because it's decoding the slash), for example the make 'EXISS/SOONER':
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/EXISS%2FSOONER?format=json
And replacing the ampersand with an underscore no longer results in an error message, but in zero results returned, which should not be the case.
ORIGINAL POST:
I'm trying to download the content from the following URL:
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s&s?format=json
And the site returns the following error message:
Server Error in '/' Application.
A potentially dangerous Request.Path value was detected from the client (&).
The problem is the ampersand; a similar request for a different car make works:
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/toyota?format=json
I have verified from a different endpoint that S&S is a valid make for the API.
Based on stackoverflow answers, I've tried all the following without success:
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s%26s?format=json
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s%26amp;s?format=json
http://www.nhtsa.gov/webapi/api/Recalls/vehicle/modelyear/1997/make/s%26amp%3Bs?format=json

Bad request with base64 encoded parameter in WCF rest url on IIS 7

I have a rest wcf 4.0 service that takes in a base64 encoded string as a parameter. It works properly when I run the service in Visual Studio using casini, but gives an Error 400 message when I run the same service under IIS 7.5. The break point doesn't get hit. I guess its not a problem with the code because it works under casini with the same code and web.config.
What changes do I need to make to enable the service to function under IIS 7.5?
This is the URL
http://localhost/MyServices/MyServ.svc/Accept/eyJXb3JkY291bnQiOjMwLCJJbWFnZWNvdW50IjoxMCwiU2VsZWN0ZWRMYW5ndWFnZXMiOlt7ImlkIjoxLCJsYW5ndWFnZTEiOiJIaW5kaSIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoiTWFyYXRoIiwiR3JvdXBfTGFuZ3VhZ2VzX2lkIjpudWxsfSx7ImlkIjoxLCJsYW5ndWFnZTEiOiJGcmVuY2giLCJHcm91cF9MYW5ndWFnZXNfaWQiOm51bGx9LHsiaWQiOjEsImxhbmd1YWdlMSI6IkdFcm1hbiIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoiSXRhbGlhbiIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH0seyJpZCI6MSwibGFuZ3VhZ2UxIjoic3BhbmlzaCIsIkdyb3VwX0xhbmd1YWdlc19pZCI6bnVsbH1dfQ==
Thanks.
The equal sign has a special meaning in the URL. It separates a parameter key from its value. The way you use the trailing equal signs, IIS will reject it.
Note that the Base 64 encoding uses additional characters that have a special meaning in a URL and cause problems (namely + and /). Therefore I recommend to use a modified Base 64 encoding that uses only URL safe characters.
Typically, the following characters are replaced:
+ with -,
/ with _, and
= with *.
Update:
The Javascript code you use for Base 64 encoding is easy to modify. Just replace this line:
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
with
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_*";
And
return (r ? enc.slice(0, r - 3) : enc) + '==='.slice(r || 3);
with:
return (r ? enc.slice(0, r - 3) : enc) + '***'.slice(r || 3);
Update 2:
I'm afraid the Base 64 URLs are never going to work with IIS. IIS still treats every part of an URL as a file or directory name. There are many problems with this implementation (just try COM2 or PRN as a URL segment).
You'll need to come up with a different URL scheme or request type. Either use a POST request to transmit the Base64 encoded part or use a URL like:
http://server/MyServices/MyServ.svc/Accept?data=eyJXb3JkY291bnQiOjMwLCJ...VsbH1dfQ**
But the second case will only work if the URL isn't too long. It wouldn't expect URLs of more than 1000 characters to work.