Archive

Posts Tagged ‘C#’

Client side caching

4 October 2009 5 comments

One thing that gets neglected a lot these days is the optimisation of websites.  I never bothered with it much until fairly recently.

In this post I will cover client side caching.  That is when the page gets stored on the clients browser and only updated when that page actually changes.  This can all be set up in IIS, however we do not always have access to the server or more often have dynamic pages. 

Note: IIS7 is needed in order to access the Response headers so this only works with IIS7

HTTP 1.1 specifications sets out the rules by which servers and clients must abide.  Basically who can cache content, how long for and when it was cached.  The client will then tell the server what version of the content it has cached and the server will then either return a HTTP status 304 ("Not modified") or reprocess the request.

Firstly we will create an ETag.  This will uniquely identify each page.  The HTTP 1.1 specification expects double quotes around the ETag.  Also, SetETag method can also only be called once, a second call will cause an exception. If “cacheability” is set to “private”, the header will not be added and you will have to set it using the HttpRequest’s AppenHeader method.

If contents is cached, the client will send a conditional GET request, meaning the request will contain the If-Modified-Since header amongst others.  This corresponds to Last-Modified header for the response.  The HTTP 1.1 specification says that both the ETag and last modification date must be processed by the server if both is contained in the request.

Here is the code that I have.  First is for creating the ETag, which as you can see is a MD5 hash of the filename and last modified date.  This code might not be the most efficient so please review it before using it in a live environment.  I copied this code and translated it from a Delphi blog just to a state where it works.

private string GetFileETag(string fileName, DateTime modifyDate)
{
    string FileString;
    System.Text.Encoder StringEncoder;
    byte[] StringBytes;
    MD5CryptoServiceProvider MD5Enc;
    //use file name and modify date as the unique identifier
    FileString = fileName + modifyDate.ToString("d", CultureInfo.InvariantCulture);
    //get string bytes
    StringEncoder = Encoding.UTF8.GetEncoder();
    StringBytes = new byte[StringEncoder.GetByteCount(FileString.ToCharArray(), 0, FileString.Length, true)];
    StringEncoder.GetBytes(FileString.ToCharArray(), 0, FileString.Length, StringBytes, 0, true);
    //hash string using MD5 and return the hex-encoded hash
    MD5Enc = new MD5CryptoServiceProvider();
    return "\"" + BitConverter.ToString(MD5Enc.ComputeHash(StringBytes)).Replace("-", string.Empty) + "\"";
}

Here we check whether the ETag is in the request and then take the appropriate action.  Please note the hardcoded date, replace this with your last updated date for your page

//REPLACE THIS WITH ACTUAL DATE AND FILENAME
DateTime updated = DateTime.Parse("01/10/2009");
string filename = "Default.aspx";

DateTime modifyDate = new DateTime();
//see if we got a valid date
if (!DateTime.TryParse(Request.Headers["If-Modified-Since"], out modifyDate))
{
    modifyDate = DateTime.Now;
}
//get request's etag
string eTag = Request.Headers["If-None-Match"];
//check if we got an etag
if (string.IsNullOrEmpty(eTag))
{
    //get new etag
    eTag = GetFileETag(filename, updated);
}

//check if the file had been modified
if (!IsFileModified(filename, modifyDate, eTag))
{
    //no change, return 304
    Response.StatusCode = 304;
    Response.StatusDescription = "Not Modified";
    //set to 0 to prevent client waiting for data
    Response.AddHeader("Content-Length", "0");
    //has to be not Private
    Response.Cache.SetCacheability(HttpCacheability.Public);
    Response.Cache.SetLastModified(modifyDate);
    Response.Cache.SetETag(eTag);
    Response.End();
    return;
}
else
{
    //make sure the client caches it
    Response.Cache.SetAllowResponseInBrowserHistory(true);
    Response.Cache.SetCacheability(HttpCacheability.Public);
    Response.Cache.SetLastModified(modifyDate);
    Response.Cache.SetETag(eTag);
}

The above coded was added to my Page_Load event. The 304 should be returned before any content.

private bool IsFileModified(string filename, DateTime modifyDate, string eTag)
{
    bool fileDateModified;
    DateTime modifiedSince;
    TimeSpan modifyDiff;
    bool eTagChanged;

    //assume file has been modified unless we can determine otherwise
    fileDateModified = true;

    //Check If-Modified-Since request header, if it exists 
    if (!string.IsNullOrEmpty(Request.Headers["If-Modified-Since"]) 
        && DateTime.TryParse(Request.Headers["If-Modified-Since"], out modifiedSince))
    {
        fileDateModified = false;
        if (modifyDate > modifiedSince)
        {
            modifyDiff = modifyDate - modifiedSince;
            //ignore time difference of up to one seconds to compensate for date encoding
            fileDateModified = modifyDiff > TimeSpan.FromSeconds(1);
        }
    }

    //check the If-None-Match header, if it exists, this header is used by FireFox to validate entities based on the etag response header 
    eTagChanged = false;
    if (!string.IsNullOrEmpty(Request.Headers["If-None-Match"]))
    {
        eTagChanged = Request.Headers["If-None-Match"] != eTag;
    }
    return (eTagChanged || fileDateModified);
}

It might be a good idea to put this into a HttpHandler for your dynamic pages/images or maybe a HttpModule.

Note: IE does not return the caching headers when you are access http://localhost/, not IE7 anyway.  Took me about an hour to find the problem.

I hope this is clear enough.  If not, comment and I will try and answer all questions.

Domain name availability checker

25 August 2009 3 comments

I was puzzled about how they do domain name availability checks for some time now.  It was only when I actually needed it for a website that I am doing that I actually looked into this.  This blog entry will cover the basics of getting a domain checker up and running, using a web service to do the actual checking.  I will cover the Ajax part in a next post.

The theory behind it all is actually really simple;  connect to a whois server and do a check on that domain, then check the response for a specific string and voila, we know whether the domain had been registered or not.

First off  create a new ASP.NET project.  Then add a new web service, I called mine DomainService.asmx.

You now got a brand new service with a default HelloWorld method, so the first thing we need to do is replace that with our own method, in this case I’ll call it CheckDomain and pass the domain to check through as a string.

First thing I do is check whether we have some text passed in, if not, throw and exception so that the calling client can handle it.  You might want to do a check first to see whether it is someone else trying to use your service but I won’t cover that here to keep it simple.

Next we need to create a new TcpClient instance.  You should notice when you type the CaSe right, you will get a red accelerator, then click on it (or hit ctrl+. [full stop]) and you will see what appears in the image below, select the using System.Net.Sockets; option.  You can manually add it to the “uses” at the top of the page, but that involves effort :).

AcceleratorOpen

The next thing to do is create a byte[] of the domain and ASCII encode it.  We also need to add the new line and line feed characters to the end of that.

//convert to byte[] to send over stream    
byte[] byteDomain = Encoding.ASCII.GetBytes(domain + "\r\n"); 

Now that we have the domain in an ASCII byte array we can send it to the server, but first we need to create the connection.  Also, a stream is needed to write to the server using our TcpClient connection so we create this as well.  Now lets open the connection, get the stream, write our domain to it, read the response and then close the connection.  I also included some debugging text so we can see the response and make life a little bit easier.

//create connection 
tcpClient = new TcpClient("YOUR_WHOIS_SERVER", 43); 
//get stream
Stream objStream = tcpClient.GetStream(); 
//write to the stream
objStream.Write(byteDomain, 0, byteDomain.Length);
//get response
StreamReader objReader = new StreamReader(tcpClient.GetStream(), 
    Encoding.ASCII); 
//read the response 
response = objReader.ReadToEnd();
#if DEBUG 
    Debug.WriteLine("--[" + domain + "]--"); 
    Debug.WriteLine(response); 
    Debug.WriteLine("--------"); 
#endif 
//close connection 
tcpClient.Close(); 

We now have a response from the server if nothing had gone wrong.  The response can now be analysed for the string.  Note that this response string I am talking about is different for most servers.  Just check through your response and you will find a common string.

//string to return
string returnValue = string.Empty;
//see if it is registered or not
if (Regex.IsMatch(response, "No match for ", RegexOptions.IgnoreCase))
{
    //yes it is registered
    returnValue = "available";
}
else if (Regex.IsMatch(response, "Registrar:", RegexOptions.IgnoreCase))
{
    //no it is not registered
    returnValue = "registered";
}
else
{
    //maybe some error returned from the server, i.e. disconnected 
    throw new Exception("Unknown response from server");
}
return returnValue;

As you probably noticed by now, I love throwing exceptions.  This makes life an awful lot easier when something goes wrong.

Now let’s test this.  Running the webservice (http://localhost:1929/DomainService.asmx in my case) by hitting F5 in Visual Web Developer brings you to the domain page.  Click on the CheckDomain option and enter a test domain (microsoft.com).  If you used the same server as me then you would have gotten (are my tenses right?) an exception, just try a different domain since loads of microsoft.com.* domains exists.  Our company domain works perfect for me, it returns “registered”.

Here is my full source code for this web service

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Services;
using System.Net.Sockets;
using System.Text;
using System.IO;
#if DEBUG
    using System.Diagnostics;
using System.Text.RegularExpressions;
#endif

namespace DnsChecker
{
    /// <summary>
    /// Summary description for DomainService
    /// </summary>
    [WebService(Namespace = "http://tempuri.org/")]
    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
    [System.ComponentModel.ToolboxItem(false)]
    // To allow this Web Service to be called from script, using ASP.NET AJAX, uncomment the following line. 
    // [System.Web.Script.Services.ScriptService]
    public class DomainService : System.Web.Services.WebService
    {

        [WebMethod]
        public string CheckDomain(string domain)
        {
            //some error checking
            if (string.IsNullOrEmpty(domain))
                throw new ArgumentNullException(domain, "Domain was not specified");

            //will contain the response from the server
            string response = string.Empty;
            TcpClient tcpClient = null;
            try
            {
                //convert to byte[] to send over stream
                byte[] byteDomain = Encoding.ASCII.GetBytes(domain + "\r\n");
                //create connection
                tcpClient = new TcpClient("whois.crsnic.net", 43);//possible exception that extension is not in list
                //get stream
                Stream objStream = tcpClient.GetStream();
                //write to the stream
                objStream.Write(byteDomain, 0, byteDomain.Length);
                //get response
                StreamReader objReader = new StreamReader(tcpClient.GetStream(), Encoding.ASCII);
                //read the response
                response = objReader.ReadToEnd();

#if DEBUG
                Debug.WriteLine("--[" + domain + "]------------------");
                Debug.WriteLine(response);
                Debug.WriteLine("---------------------------------------");
#endif
            }
            catch (Exception ex)
            {
#if DEBUG
                Debug.WriteLine("--[" + domain + "]------------------");
                Debug.WriteLine("ERROR FOR " + domain + ex.Message);
                Debug.WriteLine("---------------------------------------");
#endif
                //maybe send some mail in case the whois server is down
                throw;//throw the error after writing the debug info so the client can deal with it
            }
            finally
            {
                //close connection
                tcpClient.Close();
            }

            //string to return
            string returnValue = string.Empty;
            //see if it is registered or not
            if (Regex.IsMatch(response, "No match for ", RegexOptions.IgnoreCase))
            {
                //yes it is registered
                returnValue = "available";
            }
            else if (Regex.IsMatch(response, "Registrar:", RegexOptions.IgnoreCase))
            {
                //no it is not registered
                returnValue = "registered";
            }
            else
            {
                //maybe some error returned from the server, i.e. disconnected 
                throw new Exception("Unknown response from server");
            }
            return returnValue;
        }
    }
}

There you have it, a skeleton web service that checks whether a domain is available or not. You should really update this code and make it safer.  In a next post I will connect this service up to a nice Ajax front end to dynamically check the domain as the user enters it, using jQuery.

This is not guaranteed to be the best way of doing things, but they work for me and others.  Your comments, good or bad, would be greatly appreciated as it helps and inspires me to improve.