c#,asp.net,webforms,sitemap , ASP.NET sitemap , how important?


ASP.NET sitemap , how important?

Question:

Tag: c#,asp.net,webforms,sitemap

I am finished with my website, or so I thought. I don't have a sitemap and by some miracle I've missed the entire concept of a sitemap, didn't even know it was a thing. Big shoutout to my CS teachers I guess.

I've been reading up on it and it seems quite complicated to generate sitemap dynamically, which I have to do since my page is basically just a big database where you search using parameters.

How important is this for Google search engine relevance etc? For example, if someone searches for my website on google like this "www.mySite.com food". If I have a category in my database called "food" and indeed if someone uses the search on my site they will find a food-category, but will Google know this? Will Google find "www.mySite.com/Find?Result=food"?

EDIT: Is it even correct to use Web.Sitemap that you can create in ASP.NET or should I use a file called sitemap.txt?


Answer:

Site Map Types

There are 3 different kinds of "site maps".

  1. A page that has links on it that assists users by giving them a complete outline of the site structure. This sometimes, but not always, follows the site hierarchy. Also, it is usually limited to major categories of the site. The detail pages are not normally listed.
  2. A system in ASP.NET that helps to build menus, breadcrumb trails, and other navigation features of the site. This is driven by a provider that by default uses .sitemap XML files.
  3. An XML document that provides links to the site's pages along with information about when the page was last updated, how important it is to update, etc. to assist with indexing the site in search engines.

A Brief History

If you look at the history, it makes parsing this information a bit less confusing. The above list is in chronological order. "Site map" was a term originally given to provide a user page that linked to the major sections of the web site. Microsoft decided to again use the term "site map" to refer to their navigation system for ASP.NET 2.0. Some years later, the major search engines teamed up to provide an XML specification for providing URL information about sites, and they unfortunately overloaded the term "site map" once again to describe them.

One of the consequences of this history is that ASP.NET has no built-in support for XML sitemaps, which it sounds like you are trying to make based on your question. This is primarily because Microsoft's implementation hasn't changed (much) since ASP.NET 2.0 and it was written before that specification came to fruition. So, using Microsoft's ASP.NET navigation system will do you no good for the purpose of generating an XML sitemap.

SEO Considerations

Do note however that all 3 of these items are important for SEO.

  1. A user site map helps with search engine indexing because it provides inner site links to the major areas that need to be indexed.
  2. Menus and breadcrumb trails are important to link all of your content pages to each other, which helps to identify relevance between different parts of the site.
  3. XML sitemaps provide search engines direct URL information and help to reduce search engine traffic on the site and to help ensure there are no important URLs that the search engine fails to consider for indexing.

So you may want to take each of these into consideration, although strictly speaking all of them are optional.

SEO is about first providing good content, and then doing the required steps so that content is accessible. Each of the above should be considered a step forward for SEO and if you really care about search engine placement, you should do all that is possible, including all 3 of these steps.

XML Sitemap Recommendation

Actually, the XML sitemap specification is pretty simple for small sites under 50,000 pages. So it generally doesn't take much effort to put them together. You may want to consider doing this yourself.

Most of the open source implementations are not very scalable because they all load the entire set of data into memory at once to generate the XML instead of streaming it. Also, most of them don't really account for the fact that you might have dynamic data in multiple tables that all contribute to the URLs of the web site. They usually leave it up to you to provide paging. Most of the existing implementations also do not provide the specialized content types offered by Google.

I have recently created an implementation that does all of these things. It has not officially been released, and it is currently part of the MvcSiteMapProvider project. We have plans to separate the XML sitemap functionality into a different assembly (and separate NuGet package) so it can be used without referencing MVC but for now MVC is a requirement (unless of course you want to grab all of the relevant types and compile them into a separate DLL yourself). An ASP.NET project with MVC in it will still run just fine, so this isn't that much of an issue for most projects.

There isn't any documentation at this point, but I have created a quick post showing how to wire it up in MVC, including a demo application.

For ASP.NET, you could either use the MVC implementation (if you install the MvcSiteMapProvider, it will install MVC into your project anyway), or you could just use an ASP.NET page.

In the second case, the configuration would look similar with a few exceptions.

  1. You don't need to use the XmlSitemapFeedRouteRegistrar class.
  2. You (of course) don't need to use the Controller, and you would replace that with an ASP.NET page, like this:

XmlSitemap.aspx

<%@ Page Language="C#" AutoEventWireup="true" CodeBehind="XmlSitemap.aspx.cs" Inherits="XmlSitemap" %>

XmlSitemap.aspx.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using MvcSiteMapProvider.IO;
using MvcSiteMapProvider.Xml.Sitemap;
using MvcSiteMapProvider.Xml.Sitemap.Configuration;

public partial class XmlSitemap : System.Web.UI.Page
{
    private HttpContextBase HttpContext
    {
        get { return new HttpContextWrapper(System.Web.HttpContext.Current); }
    }

    private int PageNumber
    {
        get
        {
            var pageString = HttpContext.Request.QueryString["page"];
            if (!string.IsNullOrEmpty(pageString))
            {
                int page;
                if (int.TryParse(pageString, out page))
                {
                    return page;
                }
            }
            return 0;
        }
    }

    private string FeedName
    {
        get
        {
            var feedName = HttpContext.Request.QueryString["feedName"];
            if (!string.IsNullOrEmpty(feedName))
            {
                return feedName;
            }
            return "default";
        }
    }

    protected void Page_Load(object sender, EventArgs e)
    {
        // Build feeds for XML sitemap
        var xmlSitemapFeedStrategy = new XmlSitemapFeedStrategyBuilder()
            .SetupPageNameTempates(templates => templates
                .WithDefaultFeedRoot("XmlSitemap.aspx?feedName={feedName}&page={page}")
                .WithDefaultFeedPaged("XmlSitemap.aspx?feedName={feedName}&page={page}")
                .WithNamedFeedRoot("XmlSitemap.aspx?feedName={feedName}&page={page}")
                .WithNamedFeedPaged("XmlSitemap.aspx?feedName={feedName}&page={page}"))
            .AddDefaultFeed()
            .AddNamedFeed("google", feed => feed.WithContent(c => c.Image().Video()))

            // Optional - add news feed (will be at ~/XmlSitemap.aspx?feedName=news)
            .AddNamedFeed("news", feed => feed.WithContent(c => c.News()))

            // Optional - add mobile feed (will be at ~/XmlSitemap.aspx?feedName=mobile)
            .AddNamedFeed("mobile", feed => feed.WithContent(c => c.Mobile()).WithMaximumPageSize(10000))
            .Create();

        var xmlSitemapFeed = xmlSitemapFeedStrategy.GetFeed(this.FeedName);

        if (xmlSitemapFeed != null)
        {
            var outputCompressor = new HttpResponseStreamCompressor();
            var response = HttpContext.Response;

            response.Clear();

            // Output content type
            response.ContentType = "text/xml";

            using (var stream = outputCompressor.Compress(HttpContext))
            {
                if (!xmlSitemapFeed.WritePage(this.PageNumber, stream))
                {
                    response.Clear();
                    //Return 404 not found
                    response.StatusCode = (int)System.Net.HttpStatusCode.NotFound;
                    response.StatusDescription = "Page Not Found";
                }
            }

            response.End();
        }
    }
}

The above configuration actually creates 4 different XML sitemap feeds, mostly to adhere to the specification of these specialized types.

  1. Default - /XmlSitemap.aspx - A sitemap that can be used with any search engine. This URL needs to be in your robots.txt file as per the XML sitemap specification.
  2. Google - /XmlSitemap.aspx?feedName=google - This sitemap includes the same URLs from Default, but it also includes the Image and Video specialized content. Since incorrectly coded crawlers may crash if they encounter any of Google's specialized types, I have separated them into a different feed. You would need to manually submit this URL to Google through their webmaster tools.
  3. News - /XmlSitemap.aspx?feedName=news - This is for Google's news sitemap functionality, which needs to be submitted per their specifications.
  4. Mobile - /XmlSitemap.aspx?feedname=mobile - This is for Googles mobile sitemap functionality, which needs to be submitted per their specifications.

Note that it is also possible to use routing in ASP.NET to make these URLs nicer. You just need to ensure that the name templates are specified above the same way that the URLs are generated by routes, putting the placeholders for {feedName} and {page} in the appropriate place in the URL.

Do be aware that according to the XML sitemap specification, an XML sitemap cannot contain any URLs for directories above its virtual directory. So, it is best to ensure your route does not contain any / characters (meaning it applies to the whole site, not a subdirectory).


Related:


deployment of a site asp.net and iis


c#,asp.net,iis
I know this is for some of you a stupid question but for me is a real problem. I have never deployed a site before What i have done so far: 1) publish the site from visual studio to a folder. 2) added to iis for testing everything works great...

System.net.http.formatting causing issues with Newtonsoft.json


c#,asp.net,asp.net-mvc,json.net
My Windows service is in the same solution as a MVC project. The MVC project uses a reference to SignalR Client which requires Newtonsoft.Json v6 + the Windows service uses System.Net.Http.Formatting, which requires Newtonsoft.Json version 4.5.0.0. I assumed this would not be a problem, as I could just use a...

How to Customize Visual Studio Setup


c#,visual-studio,setup-project
I have created a video chat application in c#. Now I wan to make a setup of it. I have created a setup using Visual studio's setup project but my client told me to customize the setup progress bar styles and other properties. i dont know how to do it....

Aligning StackPanel to top-center in Canvas


c#,wpf,xaml,canvas
I'm trying to create an application which is supposed to measure quick reaction performance of it's user. The application starts up in full-screen mode and resizes it's elements accordingly to the screen resolution. The project was strongly inspired by training_aim_csgo2 map. It's mostly done, but here is the problem: I...

SQL Server / C# : Filter for System.Date - results only entries at 00:00:00


c#,asp.net,sql-server,date,gridview-sorting
I have a connected SQL Server database in Visual Studio and am displaying its content in a grid. I created a dropdown menu with the column names as selectable options and a text field to filter for specific content, e.g., DropDown = "Start" - Textfield = 14.03.2015 = Filter Column...

Update list of items in c#


c#,linq,list,updates
I would like to know if you can suggest me an efficient way to update a list of items in c#. Here is a generic example: If CurrentList is [ {Id: 154, Name: "George", Salary: 10 000} {Id: 233, Name: "Alice", Salary: 10 000}] And NewList is [ {Id: 154,...

Difference between application and module pipelines in Nancy?


c#,asp.net,nancy
I have seen in the documentation of Nancy, sometimes these two are referred distinctively. And also is there a difference in the Before/After hooks of these two pipelines?...

C# XML: System.InvalidOperationException


c#,xml
I have been learning C#'s XML with a project however I keep getting the InvalidOperationException. I have put the code below XmlTextWriter writer = new XmlTextWriter(path, System.Text.Encoding.UTF8); writer.WriteStartDocument(true); writer.Formatting = Formatting.Indented; writer.Indentation = 4; writer.WriteStartElement("User Info"); writer.WriteStartElement("Name"); writer.WriteString(userName); writer.WriteEndElement(); writer.WriteStartElement("Tutor Name"); writer.WriteString(tutorName); writer.WriteEndElement();...

Is it possible to concactenate a DataBound value with a constant string in XAML DataBinding?


c#,xaml,windows-phone
To bind a value to a TextBlock we use the following syntax to display an <ItemName> property of a bounded object. <TextBlock Text="{Binding Path=ItemName}" /> But is there a syntax to use the above tag to concatenate the constant string 'Item' with the databounded property, in order display something like:...

Convert contents of an XmlNodeList to a new XmlDocument without looping


c#,xml,xpath,xmldocument,xmlnodelist
I have Xml that I filter using XPath (a query similar to this): XmlNodeList allItems = xDoc.SelectNodes("//Person[not(PersonID = following::Person/PersonID)]"); This filters all duplicates from my original Persons Xml. I want to create a new XmlDocument instance from the XmlNodeList generated above. At the minute, the only way I can see...

C# Code design / Seperate classes for each TabControl


c#,oop,architecture,software-design,code-design
My main problem is that my tool grows and grows and I start loosing the focus on the different parts of my code. The main-Form got a docked tabControl at fullsize. I got 5 different tabs with for really different functions. So I can say my tool is splitted into...

Access manager information from Active Directory


c#,asp.net,active-directory
Attach is the picture of active directory, which i got from my IT department. Now i want to get the manager information in C#. NOTE: I am able to get all information of user but there isn't any key of manager, but IT department just gave me above attached...

Catch concurrency exception in EF6 to change message to be more user friendly


c#,asp.net,.net,entity-framework,entity-framework-6
I am using EF6.1 and i would like to change the message to a more system specific message when the below exception is thrown. Store update, insert, or delete statement affected an unexpected number of rows (0) Now, my problem is i cannot seem to catch the exception? I have...

Error when building an XDocument


c#,xml,linq,xpath,linq-to-xml
Using the following example xml containing one duplicate: <Persons> <Person> <PersonID>7506</PersonID> <Forename>K</Forename> <Surname>Seddon</Surname> <ChosenName /> <MiddleName /> <LegalSurname /> <Gender>Male</Gender> </Person> <Person> <PersonID>6914</PersonID> <Forename>Clark</Forename> <Surname>Kent</Surname> <ChosenName>Clark</ChosenName> <MiddleName />...

Regex to remove `.` from a sub-string enclosed in square brackets


c#,.net,regex,string,replace
I have this regex in C#: \[.+?\] This regex extracts the sub-strings enclosed between square brackets. But before doing that I want to remove . inside these sub-strings. For example, the string hello,[how are yo.u?]There are [300.2] billion stars in [Milkyw.?ay]. should become hello,[how are you?]There are [3002] billion stars...

Marshal struct in struct from c# to c++


c#,c++,marshalling
I have the following structures in C# and C++. C++: struct TestA { char* iu; }; struct TestB { int cycle1; int cycle2; }; struct MainStruct { TestA test; TestB test2; }; C#: [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi, Pack = 1)] internal struct TestA { [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 36)] private string iu; public...

Convert Date Time to IST


c#
I want to convert the date time to "Indian Standard Time", so i used the following code :- public static TimeZoneInfo INDIAN_ZONE = TimeZoneInfo.FindSystemTimeZoneById("Indian Standard Time"); writer.WriteLine("{0} {1}", indianTime.ToLongTimeString(), indianTime.ToLongDateString()); The above code gives me error :- System.TimeZoneNotFoundException: The time zone ID 'Indian Standard Time' was not found on the...

Why is the task is not cancelled when I call CancellationTokenSource's Cancel method in async method?


c#,asynchronous,task,cancellationtokensource,cancellation-token
I created a small wrapper around CancellationToken and CancellationTokenSource. The problem I have is that the CancelAsync method of CancellationHelper doesn't work as expected. I'm experiencing the problem with the ItShouldThrowAExceptionButStallsInstead method. To cancel the running task, it calls await coordinator.CancelAsync();, but the task is not cancelled actually and doesn't...

How to declare var datatype in public scope in c#?


c#,linq
I write simple query with linq to sql : var query = (from p in behzad.GAPERTitles select new { p.id, p.gaptitle }).ToArray(); up code into the c# windows application ,windows form load event,and i want use up result into the button click event in this scope: private void button1_Click(object sender,...

Export data from table in Pervasive


c#,pervasive
I want to export data from table programatically. And i wonder if it's even possible? The picture is from Pervasive, that the db-server I'm using. Please assist! :) ...

Regex that allow void fractional part of number


c#,regex
@"[+-]?\d+(\.\d+)?" -this is a regex I have wrote for numbers it allows [+-] minus before the number digits before and digits after the point the question is how to change this to allow "not finished" values so that input of "5." - is fine too ?...

how can I add a column to IQueryable object and modify its values


c#,.net,linq,grid,devexpress
var packs = from r in new XPQuery<Roll>(session) select new { Number = r.number Selection = new bool() }; gcPack.DataSource = packs; I want to add another column to my grid control with: Selection = new bool(). It will be added to the grid but I can't change its...

How do I provide a collection of elements to a custom attached property?


c#,wpf,binding
I found a few examples online, and a few questions and answers here, but I just can't get it to work. I need a custom attached property that can take one or more target elements. For example... <ListView> <dd:MyDragDrop.DropBorders> <Binding ElementName="brdOne"/> <Binding ElementName="brdTwo"/> <Binding ElementName="brdThree"/> </dd:MyDragDrop.DropBorders> </ListView> I've also had...

Validate a field only if it is populated


c#,wpf,idataerrorinfo
I am having a problem with validating phone numbers. In our system we have two phone numbers which you can store. The problem I am having is that these are optional fields. So I want it to validate the phone number IF and only IF the user has tried to...

C# PCL HMACSHAX with BouncyCastle-PCL


c#,bouncycastle,portable-class-library
I want to implement this logic in portable C# class: static JsonWebToken() { HashAlgorithms = new Dictionary<JwtHashAlgorithm, Func<byte[], byte[], byte[]>> { { JwtHashAlgorithm.HS256, (key, value) => { using (var sha = new HMACSHA256(key)) { return sha.ComputeHash(value); } } }, { JwtHashAlgorithm.HS384, (key, value) => { using (var sha = new...

Visual Studio Assembly force-installs Target Framework


c#,.net,visual-studio-2013,.net-framework-version
I have this Assembly targeted at .NET 3.5. The code will work on later versions as well, but I like this to work on Windows XP. I mean, .NET is backwards compatible, right? I can run apps for .NET 3.5 on Windows 8.1. However, when I run my own assembly,...

Unconstrained type parameters casting


c#,.net,types,casting
My situation: interface ISomeInterface { void DoSmth<T>(T other); } class Base : ISomeInterface { public virtual void DoSmth<T>(T other){ // for example do nothing } } class Derived<T2> : Base { Action<T2> MyAction {get;set;} public override void DoSmth<T>(T other){ if(typeof(T2).IsAssignableFrom(typeof(T))) MyAction((T2) other); } } This gives me an error: Cannot...

How can I determine if an object of anonymous type is empty?


c#,.net
I am sure the answer to this is quite simple but I am trying to write an if statement (C# 5.0) to determine whether or not an anonymous type is empty or not. Here is a simplified version of my code: public void DoSomething(object attributes) { // This is the...

How to send Ctrl+S through SendKeys.Send() method to save a file(save as dialog)


c#,.net,windows,sendkeys
I need to save a file which is in an External application using SendKeys.Send() method. The keys needed to be sent are Ctrl+S. I wrote the below code, but its not working: SendKeys.SendWait("^%s?"); // to get the Save As dialog Thread.Sleep(5000); SetForegroundWindow(FindWindow(null, "Save As")); Thread.Sleep(5000); SendKeys.SendWait("xyz"); // Sending FileName ...

Collect strings after a foreach loop


c#,xml,foreach
Is it possible to collect the strings after a foreach loop? For example: StringCollection col = new StringCollection(); XmlNodeList skillNameNodeList=SkillXML.GetElementsByTagName("name"); foreach (XmlNode skillNameNode in skillNameNodeList) { skillsName=skillNameNode.Attributes["value"].Value; } col.Add(skillsName); //Return System.Collections.Specialized.StringCollection I want to collect each skillsName and put them in a collection or a list so that I can...

How to return result while applying Command query separation (CQS)


c#,design-patterns,cqrs,command-query-separation
I am separating my query and command on service side like this: public class ProductCommandService{ void AddProduct(Product product); } public interface ProductQueryService{ Product GetProduct(Guid id); Product[] GetAllProducts(); } Command Query Separation accepts that a method should change state or return a result. There is no problem. public class ProductController: ApiController{...

Foreign key in C#


c#,sql,sql-server,database
I have two tables, A and B, in a dataset in SQL Server; I have created a connection to the dataset in a c# project in visual studio. How can I create a foreign key ( A is the parent) between my two tables ? I want to create the...

while Inherit style in WPF it affect parent style


c#,xaml,styles,wpf-controls
In WPF i have a style for the control like below, <Style TargetType="local:CustomControl"> <Setter Property="Background" Value="Transparent" /> <Setter Property="BorderBrush" Value="Gray" /> <Setter Property="BorderThickness" Value="0,0,0,1" /> <Setter Property="Padding" Value="3,0,3,0" /> <Setter Property="IsTabStop" Value="False" /> <Setter Property="VerticalContentAlignment" Value="Center" /> </Style> Now i need to override customcontrol border for some other place like...

WCF service architecture query


asp.net,architecture,wcfserviceclient
I have an application that consists of a web application, and mutliple windows services, only one windows service is installed depending on what version of the backend sofware is used. Currently, Data is saved by the web app in a database, then the relevant service is installed and this picks...

Index was out of range. Must be non-negative or less than size of collection [duplicate]


c#
This question already has an answer here: What is an “index out of range” exception, and how do I fix it? [duplicate] 1 answer Trying to run a delete application in C#. If there is more than 10 files in a directory, delete the oldest file, and iterate again....

check if file is image


c#,asp.net,asp.net-mvc
I want to check if file is image. and then you will see a link where you can see the image. But the link only has to appear if file is link. I try it like this: if (!String.IsNullOrEmpty(item.FileName)) { var file = item.FileName; string[] formats = new string[] {...

How to make a website work only with https [duplicate]


asp.net,ssl,https
This question already has an answer here: How to force HTTPS using a web.config file 3 answers How do I make a website to work only with https? Is there any method to make my website work only if the protocol is https? For example let me say http://www.mywebsite.com,...

Multiple Threads searching on same folder at same time


c#,multithreading,file-search
Currently I have a .txt file of about 170,000 jpg file names and I read them all into a List (fileNames). I want to search ONE folder (this folder has sub-folders) to check if each file in fileNames exists in this folder and if it does, copy it to a...

Memory consumption when chaining string methods


c#,string,immutability,method-chaining
I know that string in C# is an immutable type. Is it true that when you chain string functions, every function instantiates a new string? If it is true, what is the best practice to do too many manipulations on a string using chaining methods?...

C# - Can't connect to remote MySQL server


c#,mysql
My problem is that I can't connect to my website remote MySQL server. I have read all answers in stackoverflow.com, but I can't find right answer. Here's my C# code: using System; using System.Collections.Generic; using System.Data; using System.Data.SqlClient; namespace ConsoleApplication3 { class Program { static void Main(string[] args) { SqlConnection...

Get object by attribute value [duplicate]


c#,reflection,custom-attributes,spring.net
This question already has an answer here: How enumerate all classes with custom class attribute? 4 answers I have a set of classes which implement a common interface and are annotated with a business domain attribute. By design, each class is annotated with different parametrization [Foo(Bar=1)] public class EntityA...

Load XML to list using LINQ [duplicate]


c#,xml,linq
This question already has an answer here: XDocument to List of object 1 answer I have following XML: <?xml version="1.0" encoding="utf-8"?> <start> <Current CurrentID="5"> <GeoLocations> <GeoLocation id="1" x="78492.61" y="-80973.03" z="-4403.297"/> <GeoLocation id="2" x="78323.57" y="-81994.98" z="-4385.707"/> <GeoLocation id="3" x="78250.57" y="-81994.98" z="-4385.707"/> </GeoLocations> <Vendors> <Vendor id = "1" x="123456" y="456789" z="0234324"/>...