sitemap,google-search,robots.txt , Robotstxt Google Searchresults
Robotstxt Google Searchresults
Question:
Tag: sitemap,google-search,robots.txt
My search results index all the sites on Google. I only want to index the main page and not the /nl/, /en/ and /fr/ How can I prevent this in my robots.txt?
I used
Disallow: /nl/
Disallow: /fr/
Disallow: /en/
But what with my sitemap with the URL www.eikhoeve.be/nl/sitemap.xml how can I allow this one for my Google Webmaster Tools?
My website
Answer:
You can use the Allow keyword to give access to a URL in a Disallowed directory.
Allow: /nl/sitemap.xml
Disallow: /nl/
Related:
django,sitemap
I've gone through the process, which is seemingly fairly straightforward of setting up an sitemap.xml file.. I'm getting 404 page when I go to www.example.com/sitemap.xml 'page not found' and a message (I've still got debug settings turned on..) along the lines of: " Django tried these URL patterns, in this...
seo,sitemap,googlebot,google-sitemap
I noticed that some results of searching on Google are not a single url but a single url with a two-column list of what I call 'important links' of this website. For example: If you open Google and search for "amazon.it", without the double quote, you got this: As you...
seo,google-search,backlink
I am a bit confused to if a product microsite/brochure site is a bad idea. Since back links are not a good thing. I've read a fair few articles but couldn't get to any conclusion. Basically I have this client how sells online hundreds of products on their main website....
seo,search-engine,sitemap
I have a web site with content from 2001 and I need to remake the sitemap. Question arises: if the old urls have already been indexed do I need to add them again (the same urls) to the sitemap even if not haven't changed? for example: the sitemap have this...
c#,asp.net,webforms,sitemap
I am finished with my website, or so I thought. I don't have a sitemap and by some miracle I've missed the entire concept of a sitemap, didn't even know it was a thing. Big shoutout to my CS teachers I guess. I've been reading up on it and it...
dynamics-crm,sitemap,dynamics-crm-2015
Is it possible that using the SiteMap to change the name of the Opportunity button that appears in the Sales navigation pane to any custom name instead of "Opportunities" ? Or this is a fixed name based on the name of the entity ? ...
seo,google-search,pagerank
I have just launched a new domain, www.nextlevelsmf.com and it's not showing for some keywords I would like it to. Can anyone give me some advice to help it rank better please? I'd like it to show on the first 2 pages for: Managed SMF hosting SMF host/hosting Managed SMF...
c#,sitemap,mvcsitemap,google-sitemap
I have sitemap format as below. I want to delete a complete node that I find loc. For example: Where a node has <loc>with a value of http://www.my.com/en/flight1. I want to delete the <url> node and his child I want to delete loc than lastmod than priority and than changefreq...
typo3,sitemap,typo3-6.2.x
I try to let typo3 generate a sitemap for all the news records. For that I tried the dd_googlesitemap_dmf extention. The dd_googlesitemap works (it creates a sitemap for all the typo3 pages - but not for extensions). I filled in the basic infos into the configuration and called the url...
html,seo,google-search
When I search keywords from Google, it shows all articles related to these words and It has separate tab called "News", where Google shows related news. How Goolge "Knows" that article from site is about News? I have opened source codes of multiple news websites and they has "itemprops" in...
web-crawler,sitemap,meta-tags,google-webmaster-tools,noindex
By accident, I had put <meta name="robots" content="noindex"> into lots of pages on my domain. I have now removed this meta-tag, but how can I get these pages to be re-indexed by Google? Any tip? I have tried re-submitting my sitemap.xml in Webmaster Tools, but I'm not sure if it...
sitemap,rich-snippets,google-rich-snippets
How can I get a snippet like in the picture below in google page results? I submitted a sitemap.xml in google webmaster tools 3 months ago, but there is no change until now? Do you know how I should proceed to get this Result? Or the name (keyword) of this...
html,vb.net,google-search
I'm trying to develop my coding skills, I was trying to design a sort of custom client for viewing google searches. I don't want to use the google API, mainly because I feel like this will cut out a lot of the interesting programming. I tried downloading the webpages, with...
.htaccess,redirect,seo,sitemap,google-webmaster-tools
I have a relatively new site that has just started to pick up a bit of traction in the SERP's. My problem is that I have published it and had it indexed with PHP URL extensions, as follows: www.example.com/page.php www.example.com/product.php And so on. Obviously it is a fairly easy matter...
sitemap,google-search,robots.txt
My search results index all the sites on Google. I only want to index the main page and not the /nl/, /en/ and /fr/ How can I prevent this in my robots.txt? I used Disallow: /nl/ Disallow: /fr/ Disallow: /en/ But what with my sitemap with the URL www.eikhoeve.be/nl/sitemap.xml how...
xml,magento,sitemap
Wondering if anyone can shed some light on this problem... I have my magento install at showcarsign.com and i'm also running a 2nd storefront with the install at showcarboards.com. I can generate a sitemap for showcarsign.com no problem (showcarsign.com/sitemap/sitemap.xml). But when I go to create the sitemap for showcarboards.com I...
php,xml,sitemap,priority,sitemap.xml
How can I make the root index in the sitemap a priority of 1.0 and the rest of the sub directory files a priority of 0.5? I'm using this php code to generate my sitemap on the fly when page is accessed. As it is right now, it gives a...
seo,google-search
When I Search for distance between bangalore to mumbai in Google, distancebetween.com comes up in the search results. I mean if I search for distance between any source to destination they have results for that. They have one dynamic page where user can enter source and destination and those inputs...
asp.net-mvc,routing,sitemap,mvcsitemap
I Want Create SiteMap For ASP.NET MVC Site. I Write This Code [XmlRoot("urlset", Namespace = "http://www.sitemaps.org/schemas/sitemap/0.9")] public class Sitemap { private ArrayList _map; public Sitemap() { _map = new ArrayList(); } [XmlElement("url")] public Location[] Locations { get { Location[] items = new Location[_map.Count]; _map.CopyTo(items); return items; } set { if...
sql,azure,sitemap,provider,sqlsitemapprovider
I'm trying to migrate my company website to a Azure Server but I'm getting problems with Sql SiteMap Provider. I'm always getting this error: Parser Error Message: Could not load type 'SqlSiteMapProvider'. Hope someone could help me how to solve this problem....
php,xml,parsing,sitemap
I'm using php to create a sitemap xml file for google submission but I'm getting an error in my code, which is: <?php $get_posts_sql = "SELECT * FROM posts ORDER BY added DESC"; $get_posts_res = mysqli_query($con, $get_posts_sql); while($post = mysqli_fetch_assoc($get_posts_res)){ $post_id = $post["id"]; $post_title = $post["title"]; $post_added = $post["added"]; $post_date...
nlp,search-engine,google-search,bing
I'm interested in training a question-answering system on top of user-generated search queries but so far it looks like such data is not made available. Are there some research centers or industry labs that have compiled corpora of search-engine queries?
php,arrays,object,simplexml,sitemap
I have and object array as displayed below. Im having trouble adding to this object array as i keep getting an error. Here is how it is at the moment: SimpleXMLElement Object ( [url] => Array ( [0] => SimpleXMLElement Object ( [loc] => http://jbsoftware.co.uk/ [lastmod] => 2015-02-02 [changefreq] =>...
python,django,sitemap
I'm trying to output a sitemap.xml file within Django and have been successfully doing so as per the following. <url> <loc>http://example.com/day/AAA</loc> </url> <url> <loc>http://example.com/day/AAA</loc> </url> <url> <loc>http://example.com/day/BBB</loc> </url> ...etc However, as you will see from my attached sitemaps file, I am trying to only return the distinct 'name' field as...
seo,sitemap,google-search
I created a sitemap for my website that contains the below code: <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"> <url> <loc>http://www.example.com/</loc> </url> <url> <loc>http://www.example.com/aboutus.html</loc> </url> <url>...
php,xml,symfony2,composer-php,sitemap
I have followed the presta sitemap bundle documentation and I am still having issues. I have this line in my composer.json file: "presta/sitemap-bundle": "dev-master" But I get this error: A typo in the package name The package is not available in a stable-enough version according to your minimum-stability setting On...
php,node.js,google-search,google-image-search
I want to make a search on google, using php or node.js... I not yet decided that, it depends about what answer for this question is easier to implement (the rest of what I want to do is easy in both languages). After make this consultation I want to process...
html,google-search,json-ld
We are currently using the Microdata format to expose data to search engines and we are looking at exposing more info to be able to support some more advanced Google Search features. As I'm working my way through the fields I'm finding I need information that we currently load asynchronously...
regex,sitemap,regex-negation,regex-greedy,regedit
I have two URLs with the patterns: 1.http://localhost:9001/f/ 2.http://localhost:9001/flight/ I have a site filter which redirects to the respective sites if the regex matches. I tried the following regex patterns for the 2 URLs above: http?://localhost[^/]/f[^flight]/.* http?://localhost[^/]/flight/.* Both URLS are getting redirected to the first site, as both URLs are...
android,eclipse,google-search,google-search-platform
I am developing the android application in which user gives the key to search that key has to be search in google and read the output of google (like google voice search). The google gives the multiple results as per the key but i need only first result among the...
javascript,html,ajax,seo,google-search
Have a look on this search where you can see that just my main page is indexed. But why does Google/Search engines not take arda-maps.org/about/ and the other subpages? Is my deep linking done in a wrong way? Do the search engines need more time? If they do need more...
typo3,sitemap,extbase
There are a lot of sitemap Generators for TYPO3 in the TER available. But none of them can handle Sites created by Extbase, which are not shown in the TYPO3 page tree. Edit Thanks to biesior, detailed informations: Unconventional I switch in TypoScript the GET Parameter for my Extbase extension[globalVar...
html,google-search
I hope this is the right place to ask my question: For example I had a webpage: http://something.com/extra/index.html Then from /extra/index.html I have created a new page (for example http://extra.com) and completely removed the extra folder from something.com But now, when I try to search in Google "something extra" it...
html,css,twitter-bootstrap,google-search
I am trying to add a Google search bar in the navbar, but i do not why it comes in a new line as shown below: This is the html that should brings the navbar and its components all in the same column. <div class='col-lg-12 col-md-12 col-sm-12 col-xs-12'> <nav class="navbar...
google-search,microdata,google-custom-search,structured-data
I have been using a Google Custom Search Engine to return useful results using standard X-Ray techniques and the more:pagemap:: techniques as outlined here https://developers.google.com/custom-search/docs/structured_search Does anyone know how, on a standard google search, I could apply the same idea to query the Structured Data on a page when x-ray...
javascript,ajax,seo,sitemap,shebang
My site is a Single Page Application and uses alot of javascript. I've got my server setup to generate the site the user sees with javascript so Google sees the same content. Google knows about this via the fragment meta tag and converts links with the shebang to escaped_fragment to...
xml,seo,sitemap,googlebot
I'm intereset if i can have many sitemapindex like this: <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>https://domain.com/sitemap/destinatieTag.xml</loc> <lastmod>2015-02-01T05:00:34+02:00</lastmod> </sitemap> </sitemapindex> I mean 1 sitemapindex to refer athor sitemapindex , or what is maxium limit for a sitemap? Example if destinatieTag.xml is athor <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap>...
xml,sitemap,google-sitemap
I have put together a XML sitemap for Google that attempts to explain that my website's five main pages each exist in one of eight languages, so the other languages have more or less identical content, just not in English. Here's a representative snippet of what I'm doing: <url> <loc>http://www.mywebsite.com/lang/french/</loc>...
google-chrome,google-search
Some will think is not related to programming but I think it is, because most of the time I am encounter programming issues I search in google to find solutions, or ways to do what I plan to do before I am writing it from scratch, lets face it we...
google-search
Can you use Google to search a site for a specific link? I dont know if it would be a relative or absolute link but its an internal link to that domain. Another solution to my issue (but not quite as good) would be to test if a page is...
javascript,html,search-engine,google-search
I would like to use a Google Custom Search Engine on my website. With Google's default implementation, you have to put Javascript on each page that has the search box. For privacy reasons, I would like to load that Javascript only for those users who actually use the search engine....
asp.net,asp.net-mvc,asp.net-mvc-5,sitemap,mvcsitemapprovider
I have donwloaded the Nuget package, MvcSiteMapProvider. In one project it only downloaded the razor views. In my other, I have both the Razor and ascx files. Also in that project it is default to using the ascx. I want it to use the razor pages instead. Does anyone know...
seo,sitemap,robots.txt
Having to manage a multilingual site, where users are redirected to a local version of site, like myBurger.com/en // for users from US, UK, etc.. myBurger.com/fr // for users from France, Swiss, etc... How should be organized the robots.txt file in pair with the sitemap? myBurger.com/robots.txt // with - Sitemap:...