- Feb 20, 2012
0- by A2 Dev Team
CloudFlare, our content delivery network (CDN), can speed up your site by automatically delivering your site to your visitors in the quickest route possible via its network of worldwide servers. CloudFlare also has built in security that can block threats before they even reach your site.
Have you opted into CloudFlare yet? It’s easy! Just log into cPanel, click the CloudFlare icon and follow the prompts.
CloudFlare published a blog post giving some very neat insight into how it gets your new content crawled faster by search engines. Here’s a snippet:
“…Search engines don’t want to overburden sites when they crawl them. To minimize this, they cluster sites by IP address and, if they detect a problem on one site, they reduce the crawl rate on all sites sharing that IP. We discovered that what was happening was one site behind CloudFlare was having a problem with their origin and, because of that, the search engines would slow down their crawl “velocity” (the rate at which a site gets crawled)…”
It’s so cool how CloudFlare resolved this issue! If CloudFlare detects a problem on a site, they will automatically change that site’s CloudFlare IP address to isolate it from your other sites so it’s not “infected” by a single problem site. CloudFlare also worked with the major search engines to ensure that their IP ranges were listed in a special category within search crawlers. This means Google, Bing and Yahoo can crawl your sites at their maximum velocity because CloudFlare can handle this load without your site becoming “overburdened” by search engine crawlers!
