FAQ Database Discussion Community
apache,googlebot,ddos
We have Ubuntu 14.04 Apache servers that host thousands of websites in a shared hosting environment, and occasionally one or another websites get the attention of DDOS attackers. Question: What is the best apache module or method for detecting and banning IP addresses that are too active? Requirements: Do not...
seo,dotnetnuke,robots.txt,googlebot
I am using the EasyDNN News module for the blog, news articles, etc. on our DNN website. The core DNN sitemap does not include the articles generated by this module, but the module creates its own sitemap. For example: domain.com/blog/mid/1005/ctl/sitemap When I try to submit this sitemap to Google, it...
html,web-crawler,robots.txt,googlebot,noindex
From the information I have been able to find so far, <noindex> is supposed to achieve this, making a single section of a page hidden from search engine spiders. But then it also seems this is not obeyed by many browsers - so if that is the case, what markup...
php,seo,search-engine,googlebot
I am about to create an Online Shopping site for my one of the client. I have to make this site SEO Friendly and therefore I must have to understand few things before I proceed to make a custom CMS Based website. As I said I am going to make...
seo,googlebot
I was told to delete ".html" from web application links in order to be better seen by Google bot. So example.com/path/to/resource.html should be example.com/path/to/resource. I didn't find any document saying that format of the resource, placed in urls, has any influence as such. No matter if it's php, htm, html...
css,3d,seo,transform,googlebot
Google's made a big deal of "if you hide it, it won't count." However, google cache shows it doesn't understand a lot of things correctly. Maybe google sees something as hidden when it should show or shows when it should be hidden. So, I ask about CSS 3D transforms. Take...
seo,sitemap,googlebot,google-sitemap
I noticed that some results of searching on Google are not a single url but a single url with a two-column list of what I call 'important links' of this website. For example: If you open Google and search for "amazon.it", without the double quote, you got this: As you...
html,seo,meta,googlebot,nofollow
If i have a page ("dontFollowMe.html") with the meta tag: < meta name = "robots" content = "noindex, nofollow" / > ... and I link to that page ... Do I need to include the nofollow rel attribute to the a element? : <a href="dontFollowMe.html" rel="nofollow">sign in</a> Thanks...
mediawiki,user-agent,googlebot
My site was hacked and the useragent is sending googlebot the incorrect page. It looks like it is reading it the wrong language as a result of the extra code: dir="ltr" lang="ja" xml:lang="ja"> <head> <meta http-equiv="Content-Type" http-equiv="content-type" content="text/html; charset=UTF-8" /> I can't find where the code is in the files....
xml,seo,sitemap,googlebot
I'm intereset if i can have many sitemapindex like this: <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>https://domain.com/sitemap/destinatieTag.xml</loc> <lastmod>2015-02-01T05:00:34+02:00</lastmod> </sitemap> </sitemapindex> I mean 1 sitemapindex to refer athor sitemapindex , or what is maxium limit for a sitemap? Example if destinatieTag.xml is athor <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap>...
url,robots.txt,googlebot
I have couple of pages and URL which I do not want to be crawled by Google crawler. I know it can be done via robots.txt. I search Google and found this way we need to arrange the whole things in robots.txt for disallow crawler but I am not sure...