.htaccess redirect facebook crawlers except privacy policy - apache

I have a SPA app with dynamic content for sharing on Facebook so I am redirecting Facebook crawlers to a nice static page using the following rule in htaccess:
<IfModule mod_rewrite.c>
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit.*$
RewriteRule ^(.*)$ https://sharing.mysite.tld/api/share/$1 [L]
This works great! But there is one problem... I can't make my app live because Facebook requires a link to privacy policy, terms and conditions etc - and these get redirected too!!
I need to ignore a certain URLs - anything requested in /docs/ - from the above rule EDIT: so that urls containing /docs/ are followed as normal (no redirect, just served normally). I can't get .htaccess to pick up on the ignore rule. I would have thought this would do it (with thanks to https://stackoverflow.com/a/1848579/4881971):
RewriteRule ^(docs)($|/) - [L]
so I would have thought my .htaccess file would look like this :
<IfModule mod_rewrite.c>
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit.*$
RewriteRule ^(docs)($|/) - [L]
RewriteRule ^(.*)$ https://sharing.mysite.tld/api/share/$1 [L]
but when I use Facebook Object Debugger on https://mysite.tld/docs/privacy I get a 404! It redirecting to https://sharing.mysite.tld/api/share/docs/privacy
How do I retain the rule but ignore requests from mysite.tld/docs/* ? Thanks.

Could you please try following, please make sure you clear your browser cache before testing your URLs. This considers your uri starts from docs.
<IfModule mod_rewrite.c>
RewriteEngine ON
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit.*$ [NC]
RewriteCond %{REQUEST_URI} ^/docs [NC]
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^(.*)$ https://sharing.mysite.tld/api/share/$1 [L]
In case you want to pass URLs where docs could come anywhere in uri(not from starting what 1st solution looks for), then try following Rules.
<IfModule mod_rewrite.c>
RewriteEngine ON
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit.*$ [NC]
RewriteCond %{REQUEST_URI} docs [NC]
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^(.*)$ https://sharing.mysite.tld/api/share/$1 [L]

Have it like this with a negated comdition:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit [NC]
RewriteRule %{THE_REQUEST} !\s/+docs [NC]
RewriteRule ^ https://sharing.mysite.tld/api/share%{REQUEST_URI} [L,R=301,NE]

Related

httaccess change mydomain.com/knowledgebase.php to mydomain.com

I checked many other similar questions but I can't come up with a .htaccess rule that would work properly.
I have a server containing a Knowledgebase system (knowledgebase.php). I don't want to show this knowledgebase.php in the URL, ever.
Examples what I want:
https://help.thecrypto.app should show same url (now it shows https://help.thecrypto.app/knowledgebase.php when you visit)
https://help.thecrypto.app/knowledgebase.php?article=1 should show URL https://help.thecrypto.app/?article=1
How can I do this?
I tried many options, including this:
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /knowledgebase.php?/$1 [L]
Could you please try following, written based on your shown samples. Please clear your browser cache before testing urls.
RewriteEngine ON
RewriteCond %{REQUEST_URI} ^/?$
RewritewriteRule ^ knowledgebase.php [L]
RewriteCond %{REQUEST_URI} ^/(knowledgebase) [NC]
RewriteCond %{QUERY_STRING} ^$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/?article=1 [L]
You can use this :
RewriteEngine on
#1) redirect "help.thecrypto.app/" to /knowledgebase.php
RewriteCond %{HTTP_HOST} ^help.thecrypto.app$ [NC]
RewriteRule ^knowledgebase\.php$ / [L,R]
#2) internally map knowledgebase.php to the root /
RewriteRule ^/?$ /knowledgebase.php [END]
This will serve /knowledgebase.php if you visit your site hompage / .

htaccess remove folder redirect

I have a problem removing folders from an url. I want that google / old links aren't broken. The old webpage had several sections with a structure like this
example.com/news/items/entry1.html
example.com/news/items/entry2.html
example.com/blog/items/foo.html
The new page has the urls like this:
example.com/news/entry1
example.com/news/entry2
example.com/blog/foo
Removing html was rather straight forward
<IfModule mod_rewrite.c>
RewriteEngine On
# Send would-be 404 requests to Craft
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !^/(favicon\.ico|apple-touch-icon.*\.png)$ [NC]
RewriteRule (.+) index.php [QSA,L]
RewriteCond %{THE_REQUEST} /([^.]+)\.html[\s?] [NC]
RewriteRule ^ /%1 [R=302,L,NE]
</IfModule>
The part I'm struggling with is removing the 'items' part. The rules I found only worked for request path like 'example.com/items/subfolder1/...'
Any help would be greatly appreciated.
To remove /items/ from your URLs you can use the following in your .htaccess file:
RewriteEngine On
RewriteRule ^(.*)/items /$1 [L,R=301]
So for example, this will take the URL: http://example.com/news/items/entry1 and turn it into http://example.com/news/entry1
Make sure you clear your cache before testing this.

Apache, url rewrite

I have the following .htaccess file inside my root directory:
RewriteEngine On
<If "%{SERVER_NAME} = 'example.com'">
# The actual condition is really long one so I replaced it for illustration.
RewriteCond %{HTTP_USER_AGENT} (iPhone|Blackberry|Android) [NC]
RewriteRule .* http://m.example.com%{REQUEST_URI} [R,L]
RewriteCond %{REQUEST_URI} ^/([a-z-]+)/$ [NC]
RewriteRule .* %{REQUEST_SCHEME}://%{SERVER_NAME}/?title=%1 [L]
</If>
<Else>
RewriteCond %{REQUEST_URI} ^/([a-z-]+)/$ [NC]
RewriteRule .* %{REQUEST_SCHEME}://%{SERVER_NAME}/?title=%1 [L]
DirectoryIndex /m/index.htm /m/index.php /m/index.html
</Else>
So as you can see its a pretty much simple way to redirect users of mobile devices to m.example.com and if that is the case I change the DirectoryIndex, few notes:
The reason I duplicate the same RewriteCond/RewriteRule is because I can't use them outside <If>/<Else> blocks.
The way I redirect(if/else) is not the question and I'm pretty sure not the best way, but if anyone have suggestions I'm all ears.
The problem:
If I access example.com/Article-Name on desktop browser there will be internal redirection to: example.com/?title=articleName but I won't see example.com/?title=articleName at my url bar as expected.
The exception:
If I access m.example.com/Article-Name my url bar is now m.example.com/?title=articleName.
I was expecting the same behavior as the desktop (internal redirection not redirection I can see on my url bar), I'm pretty sure its because the DirectoryIndex but I don't how to prove it or solve this issue, if anyone have a solution I will be very thankful.
The main issue with your code is that you're feeding it with an URL that is not the same as the current URL then, what happens is that it ignores the internal and make it an external redirect.
Another very important thing is that you forgot %{HTTP_USER_AGENT} to match against the mobile browsers.
Here is an idea to resolve your issue:
RewriteEngine On
# First we verify if its a mobile device and
# if its not already on the mobile subdomain
RewriteCond %{HTTP_HOST} !^m\.domain\.com$ [NC]
RewriteCond %{HTTP_USER_AGENT} (iPhone|Blackberry|Android) [NC]
RewriteRule ^ http://m.example.com%{REQUEST_URI} [R,L]
# here we deal with all the internal redirects for
# the mobile subdomain
RewriteRule ^m/index\.php$ - [L]
RewriteCond %{HTTP_HOST} ^m\.domain\.com$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)$ /m/index.php?title=$1 [L]
# here we deal with the normal website
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)$ /index.php?title=$1 [L]
Your doing internal redirects, if you want to redirect to a new URL use external ones
Replace [L] with [L,R=302]
or 301 if you want the result to be cached

htaccess redirect in joomla WITHOUT landing on index.php

I've added an redirect from an old domain to my new domain in the .htaccess file of my Joomla 3.x site, running on Apache.
This is the code I'm trying to get working;
RewriteCond %{HTTP_HOST} !newdomain\.co\.uk$ [NC]
RewriteRule ^(.*)$ http\:\/\/www\.newdomain\.co\.uk/$1 [R=301,L]
This succeeds in redirecting any incorrect domain to my new domain, but does NOT put me on the corresponding sub-page.
So, for example, I type in the following address ..;
http:\\old.com\calendar
... and I WANT to end up at;
http:\\newdomain.co.uk\calendar
But instead, I get directed to;
http:\\newdomain.co.uk\index.php
Can anyone help me get the redirect working to land me on the corresponding sub-page?
For the record, the Joomla site's SEO settings are as follows (and work as you would expect - site pages do not generally include index.php in the URL and correspond to the menu item alias);
Search Engine Friendly URLs - YES
Use URL rewriting - YES
Adds Suffix to URL - NO
Unicode Aliases - NO
The COMPLETE .htaccess file looks like this - I don't know if Joomla's .htaccess definitions are affecting the redirect that I've put in place;
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
RewriteRule .* index.php [F]
RewriteBase /
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteCond %{REQUEST_URI} !^/index\.php
RewriteCond %{REQUEST_URI} /component/|(/[^.]*|\.(php|html?|feed|pdf|vcf|raw))$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* index.php [L]
suPHP_ConfigPath /whatever/php.ini
RewriteCond %{HTTP_HOST} !newdomain\.co\.uk$ [NC]
RewriteRule ^(.*)$ http\:\/\/www\.newdomain\.co\.uk/$1 [R=301,L]
Header set X-UA-Compatible "IE=10"
Thanks in advance to anyone who can provide some insight!
You are rewriting a lot of stuff to index.php before already – so when your new Rules you added below that are evaluated, the (internal) request URI most likely is index.php already.
So move those new rules further up.

Rule exceptions in mod_rewrite

I have created page that every new user is redirected to upon visit, it sets a cookie that allows access to the rest of my site. I want to add exception to this rule for some sites and their bots:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_REFERER} http://www.site1.com [NC,OR]
RewriteCond %{HTTP_REFERER} http://www.site2.net [NC,OR]
RewriteCond %{HTTP_USER_AGENT} bot1 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} bot2 [NC]
RewriteRule .? - [S=1]
RewriteCond %{HTTP_COOKIE} !yes=1 [NC]
RewriteRule !^(script1.php|script2.php)$ script2.php [L]
</IfModule>
As you can see, if the cookie doesn't exist or !=1 every user is redirected to script2.php.
I wrote some exceptions, but sometimes it works, and sometimes it doesn't.
I've made an error somewhere, but i don't see it.
Can you help me with this? Thank you for you time.
I'd rather use [L] instead of [S=1].
Secondly, I would do a external redirect to script2.php, so it's not cached (incorrectly). So use [R,L] instead of just [L] on your last rule.