This site or app is sending too much traffic to rawgit.com. Please contact its owner and ask them to use cdn.rawgit.com instead, which has no traffic limit.
I am getting this above notification on my website. Can anyone help me to resolve out this issue.How I can remove it.
Rawgit has two Domains for displaying your Github files:
cdn.rawgit.com is for when you request that site a lot (Updates take a while to show).
rawgit.com is for testing (Updates are visible immediately).
You need to use the cdn.rawgit.com link as you are sending to much traffic.
For example:
https://rawgit.com/Oisins/Mod/master/.project
becomes
https://cdn.rawgit.com/Oisins/Mod/master/.project
See http://rawgit.com/ for more info.
Related
Every time I visit a website that is using Cloudflare's Under-Attack-Mode, it shows me the usual text telling me to wait a few seconds until Cloudflare verified I am not a bot. Every time I reload the page it changes my current Ray ID.
What is the purpose of a Ray ID? Is it some kind of session ID?
It is a UID which can be used by the website operator (and Cloudflare support) to potentially debug issues. The ray id is actually returned in the headers of most requests through Cloudflare, just not as visibly as what you see in the case of I'm under attack mode.
I will try to explain both in simple words, Cloudfare and Ray ID.
Cloudfare:
This is a service sitting in front of servers tasked with the ability to handle requests and allow response depending upon multiple security parameters. The service provides website security, performance optimization and DDoS protection. The service mimics a reverse proxy.
When a request is sent to Cloudfare, it initially hits one of the edge servers (Content Delivery Network) strategically placed across the continent. This helps to fetch cached data at a rapid pace without the need to traverse to the origin server to fetch the content.
Ray ID:
Ray ID is a unique identifier generated by Cloudflare's edge servers, and is included in the response headers of each request processed by these edge servers and is used to help identify specific requests when troubleshooting issues. Other than issues, it helps to track the request, request specifier, topo location and auditing becomes easier.
Hope, it clears the doubt
I have a web shop on the very much traffic is, but this traffic is apparently generated by bots, which load pages, but never the pictures, javascript files or css files load.
I want to lock these connections with mod-security, but I find nowhere a rule with which I can do that.
First I had tried with the firewall the IP lock, but the always come with other addresses. Undertaking is pointless.
The user agent is different ... it can not be seen that the connection of a bot comes. The only thing that is noticeable is that they never load an image or a CSS file from the page.
The accesses are however not so much and fast, so the mod_evasive does not strike.
Obviously, it is a good idea to make the page so slow, the normal user quickly give up :-( I think a competitor has a ddos attack run ... who knows that already ...
Does anyone have the same problem or for this problem a mod-security rule, with which I could work?
Does somebody has any idea?
Best regards,
Holger
I'm getting constantly ban from a website, I set download_delay = 10 in scrapy, I tried a package fake_user_agent then I tried implementing tor and polipo, according to this site the config is ok. But after running 1/2 times again I got banned! Can anyone help me here ?
Note: scrapy-proxie I also want to try this but can't activate.
You should take a look at what the documentation says.
Here are some tips to keep in mind when dealing with these kinds of
sites:
rotate your user agent from a pool of well-known ones from browsers
(google around to get a list of them)
disable cookies (see
COOKIES_ENABLED) as some sites may use cookies to spot bot behaviour
use download delays (2 or higher). See DOWNLOAD_DELAY setting.
if
possible, use Google cache to fetch pages, instead of hitting the
sites directly use a pool of rotating IPs. For example, the free Tor
project or paid services like ProxyMesh
use a highly distributed
downloader that circumvents bans internally, so you can just focus on
parsing clean pages. One example of such downloaders is Crawlera
Use delay on clicks
Not tor - all connections from one address - bad, rotate proxy after several visits
And check this post - web scraping etiquette
I am only just starting out with iPhone application development and have been doing some research with regards at getting data into an app using information available via the web.
I understand that I can access web pages using the NSURL* classes. Does anyone know how I might request data from a page that requires user input?
I can understand accessing an actual page, but I am not sure if (or how) I might be able to initiate the request and get the appropriate data back into my app.
Any help / pointers is very much appreciated.
EDIT_001:
I was thinking that I would have to interact with the actual controls on the page, but after a little more investigation I have found that I can simply use the request HTML that the page generates.
gary
This question is really too broad ("how does http networking work"), and if you look around this site you will probably find several questions that will take you in the right direction. As a first stop, check out the ASIHTTPRequest framework. Many people use it, and it makes http networking really simple.
You could also have a look at these links for inspiration:
http://iphoneonrails.com/
http://metaskills.net/2010/2/12/synchronizing-core-data-with-rails-3-0-0-pre
Is there a trick to logging in to Apple Developer Connection? For the past two weeks, out of about 100 tries, I've been able to log in three times. Every other time, after a successful entry of my username and password, it takes me back to the login screen.
This happens to me on both my Macs, on Safari and Firefox, so I'm not hopeful of a solution. But I have a hard time believing that the situation is really this bad...
I am having the same problem, I have narrowed down to a problem with my ISP. Of course they will not acknowledge it, but the problem only arises when I attempt a login from home. I think they are probably using a caching proxy and something in the scheme used by apple to login->access the content makes the proxy believe it's only visiting content that is still valid. I am going slightly mad because of this.
This question and the related discussion clued me in to how to fix my problem with the same symptoms on developer.apple.com. In my case, I have multiple "teams," so after entering in my Apple ID, it takes me to a team selection page. After selecting a team, I'd just be redirected back to the login/Apple ID page.
Turns out, the login is done over HTTPS, while the team selection (and probably the bulk of other activities on developer.apple.com) are over HTTP. Our firewall load balances over a couple of Internet connections, and the HTTPS traffic was passing over a different interface than the HTTP. Evidently, this was confusing Apple's authentication mechanism. It also explains why I was occasionally able to get through -- sometimes all traffic would end up on the same interface.
Ultimately, my solution was to add a rule to the firewall to send all 17.0.0.0/8 traffic (Apple's legacy class A network) over the same interface.
Hopefully this helps someone else with a frustratingly endless login loop.