I run wiki like database website. Some people crawl (scrape) my website data regularly.
So I am trying to block these crawlers.
Is there a script to prevent after certain number of requests from same IP address?
Yes, there are modules like mod_security and mod_evasive in apache to prevent these kind of attack like scraping and DDOS.
You just need to install and configure it as you need for your server.
Check this link for apache and CentOS configuration.
Related
How to block IP automatically if number of requests exceed via WHM cPanel. I tried but did not found way. As we have lots of domain hosted, so we can not use Cloud flare.
Please suggest the way with cPanel WHM.
You can do the following in that case:
Install ConfigSecurity and Firewall and limit number of requests using PORTFLOOD directive in csf config. This setting will limit requests for the whole server.
Install Litespeed Webserver and edit per client throttle. This setting will limit request per domain hosted.
i want to ask something about dedicated server.
i have dedicated server and a cPanel website with heavy load, when i check the server load, all parameter didn't go up to 60% usage. but the apache work is high.
so i wonder if i can do this.
i buy dedicated server(DS) and install 2 cPanel on same DS. i know that cPanel need an IP to bind the license so i add 1 additional IP to my DS.
what i am trying to archieve here is to split workload in same website, and to split the traffic i use loadbalancer from CF.
so i have abc.com with 2 different IPs and use LoadBalancer to split the load.
here is why i need to do this
Server load relative low (under 80%)
Apache load relative high 3-10 req/s
There is a problem in your problem definition
What do you mean by Apache work?
if you want have more threads and processes of Apache httpd on the same server, you dont need to install two Cpanel instances, you could tune your Apache httpd worker configuration for a better performance and resource utilization.
you can even use litespeed or nginx web servers on cpanel.
I am using solr 4.10.3. Remote users access it through a url and run their specific queries like in search engines. I want to know IP address along with queries run by users. In solr log file, details of queries are there but no information about user ip. I am using httpd (apache) webserver.
Or
if I analyze apache webserver log, there I get informations about ips etc but no information about query. So How I can configure httpd configuration so that its log also include queries details.
Anyone of two will solve my problem.
I doubt that you're using Apache httpd to directly serve content from Solr (since it's usually served by either the bundled jetty or a different java application server (such as Tomcat) if you've installed it that way), but you can configure Apache httpd's log format through the LogFormat directive.
%q The query string (prepended with a ? if a query
string exists, otherwise an empty string).
The other option is to configure Solr's logging properly, see Configuring Logging, but depending on your setup (if you're proxying requests), that might not include the originating IP address (and just the address of the proxy server).
I am using web2py to run some application in the server, and i have also websites created with php running on the same server under apache. Is there a way to redirect requests for web2py to apache based on the domain name used.
I have little information about apache so i appreciate some explanation.
thanks.
Just use Name based Virtual Hosts for Apache. Besides, redirects based on some specific conditions can be implemented with mod_rewrite.
I want to add a javascript based chat system to my web app. I have read in many places that apache needs to forward(proxy) the requests to the xmpp chat server.
From what I have understood , I either need to add proxying support to apache server
OR use the jabberHTTPbind servelet.
I am trying the first thing. Add proxy support to apache. Now within Eclipse I am using Tomcat 7.0. I think this Tomcat comes with its own web server and is not using Apache. Is that true ?I am using Tomcat from http://tomcat.apache.org/download-70.cgi and added server in Eclipse by providing its extracted path.
I am not able to find any apache related folder structure eg(/etc/apache2/apache2.conf) within my Tomcat extracted directory.Also can't find httpd.conf anywhere .
Should I need to download apache separately and connect my tomcat to it.
I think adding proxy support to apache is required because javascript code won't be able to connect to any URL other than the current domain. so the question again is the same , where are all these apache2.conf and httpd.conf to add mod_proxy. and where can I do the virtual host mapping.
Any help to move ahead with be very much appreciated.
You need to install the Apache HTTPD server and enable reverse proxying. Read the doc on mod_proxy on how to do this. There are tons of resources on Stackoverflow as well; search for Apache reverse Proxy. The conf files you mention are part of the Apache HTTPD install.
javascript code won't be able to connect to any URL other than the current domain
That is generally true unless you enable CORS