FAQ Database Discussion Community


How to customize DNN robots.txt to allow a module specific sitemap to be crawled by search engines?

seo,dotnetnuke,robots.txt,googlebot
I am using the EasyDNN News module for the blog, news articles, etc. on our DNN website. The core DNN sitemap does not include the articles generated by this module, but the module creates its own sitemap. For example: domain.com/blog/mid/1005/ctl/sitemap When I try to submit this sitemap to Google, it...

How to increse number of sitemapindex

xml,seo,sitemap,googlebot
I'm intereset if i can have many sitemapindex like this: <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>https://domain.com/sitemap/destinatieTag.xml</loc> <lastmod>2015-02-01T05:00:34+02:00</lastmod> </sitemap> </sitemapindex> I mean 1 sitemapindex to refer athor sitemapindex , or what is maxium limit for a sitemap? Example if destinatieTag.xml is athor <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap>...

How to evidence a particular page on Google SERP?

seo,sitemap,googlebot,google-sitemap
I noticed that some results of searching on Google are not a single url but a single url with a two-column list of what I call 'important links' of this website. For example: If you open Google and search for "amazon.it", without the double quote, you got this: As you...

What is the best way to ban DDOS attacks for Apache, on a shared hosting platform?

apache,googlebot,ddos
We have Ubuntu 14.04 Apache servers that host thousands of websites in a shared hosting environment, and occasionally one or another websites get the attention of DDOS attackers. Question: What is the best apache module or method for detecting and banning IP addresses that are too active? Requirements: Do not...

Does document format change behaviour of Google bot in terms of SEO? Like /path and /path.htm .html .php [closed]

seo,googlebot
I was told to delete ".html" from web application links in order to be better seen by Google bot. So example.com/path/to/resource.html should be example.com/path/to/resource. I didn't find any document saying that format of the resource, placed in urls, has any influence as such. No matter if it's php, htm, html...

googlebot reading site in incorrect language

mediawiki,user-agent,googlebot
My site was hacked and the useragent is sending googlebot the incorrect page. It looks like it is reading it the wrong language as a result of the extra code: dir="ltr" lang="ja" xml:lang="ja"> <head> <meta http-equiv="Content-Type" http-equiv="content-type" content="text/html; charset=UTF-8" /> I can't find where the code is in the files....

EHow to Disallow few list of URL crawled by google crawler using robots.txt

url,robots.txt,googlebot
I have couple of pages and URL which I do not want to be crawled by Google crawler. I know it can be done via robots.txt. I search Google and found this way we need to arrange the whole things in robots.txt for disallow crawler but I am not sure...

How does using CSS 3D Transforms affect SEO? [closed]

css,3d,seo,transform,googlebot
Google's made a big deal of "if you hide it, it won't count." However, google cache shows it doesn't understand a lot of things correctly. Maybe google sees something as hidden when it should show or shows when it should be hidden. So, I ask about CSS 3D transforms. Take...

how google crawls dynamic pages? [closed]

php,seo,search-engine,googlebot
I am about to create an Online Shopping site for my one of the client. I have to make this site SEO Friendly and therefore I must have to understand few things before I proceed to make a custom CMS Based website. As I said I am going to make...

Do I need to add nofollow rel attribute to links if the href page contains a robots meta tag containing noindex and nofollow?

html,seo,meta,googlebot,nofollow
If i have a page ("dontFollowMe.html") with the meta tag: < meta name = "robots" content = "noindex, nofollow" / > ... and I link to that page ... Do I need to include the nofollow rel attribute to the a element? : <a href="dontFollowMe.html" rel="nofollow">sign in</a> Thanks...

How to prevent search engines from indexing a span of text?

html,web-crawler,robots.txt,googlebot,noindex
From the information I have been able to find so far, <noindex> is supposed to achieve this, making a single section of a page hidden from search engine spiders. But then it also seems this is not obeyed by many browsers - so if that is the case, what markup...