No problems with: HTTP/HTTPS
When defanging/disarming URL schemes (e.g. with python-defang):
http becomes hXXp
https becomes hXXps
So, no problem here.
But what happens with: FTP/FTPS/FXP
But how can these schemes be properly defanged?
ftp becomes fXp
how do I know, if a given URL is defanged or if it's a real URL which just makes use of the File eXchange Protocol (fxp) instead of the normal File Transfer Protocol (ftp)?
ftps becomes what? fXps?
what is the "official defanged" version of ftps?!
fxp becomes what? fXxp?
what is the "official defanged" version of fxp?!
Alternative?
Is there something like a rule of thumb for defanging/disarming: just to make sure that a URL doesn't work anymore within a browser so that the client won't open a malicious URL accidentally?
Link's source indicates it only supports HTTP HTTPS and FTP. Not SFTP, FTPS or FXP. Although support seems as trivial to add by updating the PROTOCOL_TRANSLATIONS list in init.py
FXP:// SFTP:// and FTPS:// are not supported in modern browsers. At best clicking such a URL will show an external application launch dialog. Similar to what you get with a magnet link.
As a rule of thumb; if crippling URLs is the goal. I would replace ':' with something else. Changing the protocol name itself doesn't make the url invalid, just unlikely to be understood/exist. It will still be parsed by extensions, plugins, etc, which may be enough to trigger bad mojo. Changing the colon will render them to strings.
Related
Dig, wget, nslookup and curl commands work perfectly for a specific URL I have pointed to another server less than 24 hours ago.
Problem is, it just refuses to be resolved by the browser (Chrome, Safari and Firefox). The strangest part is that it is being successfully resolved by Postman (by testing the OPTIONS and the GET methods separately), but still doesn't return a proper response on the browser side of things.
DNS checks are returning positive, so this is when I started suspecting that the problem is actually within the headers of the HTTP protocol's requests which are sent - alongside the fact that different responses are being returned for the requests that don't include the default browser headers (being issued through the different command-line tools & Postman) and the ones who do (being issued by the browsers automatically or manually using the dev tools).
After fully flushing the current local system's DNS cache, including the browsers's and even trying another device on another network - I still get still no response on the browser.
Kept going, and attempted to verify that with a VPN (locally - which didn't work), and an online web proxy tool (which did work).
Finally, I extracted the router's default DNS server address, used nslookup to look up the URL again, this time specifically mentioning the desired DNS server (the one stated above), and after getting a successful response with the correct values, I am now pretty much sure the HTTP request is causing the problem.
The URL is hosted on Amazon S3 Static Hosting option, which I used many times before, and didn't have a problem with, with that exact same configuration. Looking up the recent changes/features that were possibly added, pointed out that I may need to explicitly set a CORS policy for the newly created bucket, on top of the usual public access policy that is needed.
After applying that as-well - it still doesn't seem to work.
As a quick change in direction that may possibly make some parts clearer about what's going on (and as I started to think that the browser might not be getting the correct Content-Type header in the response, which should be text/html header as its response, and therefore, possibly doesn't resolve the URL with the expected behavior), I went ahead and applied a 301 redirection on the S3 bucket, instead of the static files hosting, and again, it all works perfectly through the command line tools, but not through the browsers.
Anyway, the browser just doesn't seem to complete any of the requests being sent to the URL.
That might be the OPTIONS pre-flight request failing to respond correctly, and the browser just doesn't continue to issuing the GET request, or the URL is not being found by the DNS route the browser is taking, which is unclear to me currently if that is the option.
Any ideas? (besides the fact that sometimes it just takes longer time for some DNS servers that happen to be on the chosen route to update/refresh their cache, which doesn't appear to be affecting my local machine's DNS route specifically for this case. That, being said with caution, was verified by validating the different parts of DNS configuration and prioritization throughout the different possible parts on my system (Mac OS X), including the fact that the response gets back with the correct address successfully).
Found my answer here:
https://serverfault.com/questions/942030/aws-s3-static-hosting-how-to-debug-connection-timeout
As linked there, more details can be found here:
Non-Authoritative-Reason header field [HTTP]
Solution & Explanation: Because of the nature of the domain extension I have purchased (.dev extension) Chrome was silently using HTTPS because of the URL being part of Chrome's HTTP Strict Transport Security (HSTS), because all .dev domains should be using HTTPS only. Therefore, the issue was still showing up, even when explicitly typing http:// into the URL address bar.
This can be overridden by applying a CloudFront distribution with HTTPS support on top of the S3 Static Hosting, as usual (but still, it should be noted as HSTS listings can cause that for different cases, including this one as part of them, because of the .dev domain extension).
Useful Resources (for debugging purposes)
In addition to what is stated here:
https://gist.github.com/stollcri/7c09bafc97223481920e
You can issue a lookup query (and also add or delete your local set of HSTS listings) through the following Chrome's settings URL:
You can also check the current listings here: https://hstspreload.org/
I am currently hosting the contents of a site with ProviderA. I have a domain registered with ProviderB. I want users to access the contents (www.providerA.com/sub/content) by visiting www.providerB.com. A domain forward is easy enough and works as intended, however, unless I embed the site in a frame (which is a big no-no), the actual URL reads www.providerA.com/sub/content despite the user inputting www.providerB.com.
I really need a solution for this. A domain masking without the use of a frame. I'm sure this has been done before. An .htaccess domain rewrite?
Your help would be hugely appreciated! I'm going nuts trying to find a solution.
For Apache
Usual way: setup mod_proxy. The apache on providerB becomes a client to providerA's apache. It gets the content and sends it back to the client.
But looks like you only have .htaccess. So no proxy, you need full configuration access for that.
So you cannot, see: How to set up proxy in .htaccess
If you have PHP on providerB
Setup a proxy written in PHP. All requests to providerB are intercepted by that PHP proxy. It gets the content from providerA and sends it back. So it does the same thing as the Apache module. However, depending on the quality of the implementation, it might fail on some requests, types, sizes, timeouts, ...
Search for "php proxy" on the web, you will see a couple available on GitHub and others. YMMV as to how difficult it is to setup, and the reliability.
No PHP but some other server side language
Obviously that could be done in another language, I checked PHP because that is what I use the most.
The best solution would be to transfer the content to providerB :-)
I want to modify my application URL from //localhost:8080/monitor/index.html to just monitor , so that on putting monitor on browser, my application should open. Is there a way to achieve this, can someone suggest the configuration changes which will be required for this.
Can I map my short URL to the existing one may be somewhere in web.xml. I am not sure about the approach any suggestions will be great.
Thanks and regards
Deb
You're mixing up several different protocol layers in your question.
If you just enter nothing but "monitor" in the browser URL bar the browser is going to first lookup "monitor" in DNS and finding nothing it will then probably send a query to Google or your configured search engine. In the past browsers have taken other steps, such as appending ".com" and prepending "www." but I don't think modern browsers do that any more.
So far, your server is not even remotely involved.
If you're a large ISP user (TimeWarner, Comcast) and use their DNS it's also possible the ISP will intercept your failed DNS lookup and route the request to a "helpful" search page (i.e. SPAM) of their own.
At this point the request is still nowhere near your server.
I suppose you could mess with the /etc/hosts file on your local system to resolve "monitor" to the proper hostname, but that's an extremely brittle solution that has to be hard coded on each machine you want to have this "shortcut" link (and which breaks when the hostname changes).
You're much better off just setting up a web shortcut in your browser that points to the right place.
I'm trying to use Liferay for http and https
if I include in portal-ext.properties:
company.security.auth.requires.https=true
web.server.protocol=https
Will be working ok with https but in http is showing incorrect themes due is trying to load https://domain.com/theme
If I remove this two lines is working ok for http but not for https.
What can I do?
IMHO mixed mode, e.g. offering http as well as https never gives you what you expect: You expect security from https, but you always risk leaking session information, e.g. being vulnerable to session-hijacking attacks (ala Firesheep). My actual advice would be to go https only if you do https for security. Read on if that's not an option for you, but don't complain when you find information leaking (this is not dependent on Liferay, but for any web-based environment)
What is the exact problem that you have with the themes? (images/css through http?) Which version of Liferay are you using?
Before you specify more, you might want to configure your theme's "virtual path", this will rewrite all the URLs referring to your theme. It's typically used to serve static resources through a webserver or cdn, but it works with any kind of URL. Simply using a protocol-relative URL should work (I love this mostly unknown http feature):
Add this to your theme's liferay-look-and-feel.xml:
<look-and-feel>
<theme id="my" name="My Theme">
<virtual-path>//domain.com/myTheme</virtual-path>
</theme>
</look-and-feel>
note that the URL omits the protocol part, http: or https:, thus the browser will use the same protocol that the whole page is loaded with.
Edit: corrected the xml. Will investigate if there's a problem with protocol-relative URLs in themes.
Edit 2: Something is weird. It seems, virtual-path does not work like this, but I recall it did earlier. Do you add domain.com as cdn.host.http or cdn.host.https? (this would be concatenated)
On related stuff, please check if you're running Apache in front of your appserver. In this case you might forward some traffic for the portal (e.g. in the virtual host for http) but not forward the traffic in the https virtual host.
I have an ssl page that also downloads an avatar from a non-ssl site. Is there anything i can do to isolate that content so that the browser does not warn user of mixed content?
Just an idea - either:
try to use an ssl url on the avatar website, if necessary by editing whatever JS/PHP/... script they provide, or:
use your scripting language of choice to grab a copy of the avatar and store it on your server, then serve it from there.
There are a number of good security reasons for the browser to warn about this situation, and attempting to directly bypass it is only likely to set off more red flags.
Ninefingers' suggestions are good, and I would suggest a third option: you can proxy the content directly through your own server using a simple binary retrieve/transmit script, if it changes frequently and is unsuitable for caching.
If all the content you want to include from foreign sites comes from a specific server and path (i.e. http://other.guy/avatar/*) you could use mod_proxy to create a reverse proxy which makes https://your.site/avatar_proxy/{xyz} mirror http://other.guy/avatar/{xyz} .This will increase your bandwidth usage and probably slow things down.