Have Geek Brain

Data Access Layer - Static List Objects and Caching

I am developing a site using .net MVC

I have a data access layer that basically consists of static list objects that are created from data in my database.

The method that rebuilds this data first clears all objects in the list. Once they are empty, add data. Here is an example of one of the lists im using. its a method that generates all UK postcodes. there are about 50 methods in my application that return all kinds of information like cities, regions, members, emails, etc.

public static List<PostCode> AllPostCodes = new List<PostCode>();

  • when the rebuild method is called first, it clears the list.

    ListPostCodes.AllPostCodes.Clear ();

  • Then it reloads the data again by calling the GetAllPostCodes () method

    /// <summary>
/// static method that returns all the UK postcodes
/// </summary>
public static void GetAllPostCodes()
{
    using (fab_dataContextDataContext db = new fab_dataContextDataContext())
    {
        IQueryable AllPostcodeData = from data in db.PostCodeTables select data;

        IDbCommand cmd = db.GetCommand(AllPostcodeData);
        SqlDataAdapter adapter = new SqlDataAdapter();
        adapter.SelectCommand = (SqlCommand)cmd;
        DataSet dataSet = new DataSet();

        cmd.Connection.Open();
        adapter.FillSchema(dataSet, SchemaType.Source);
        adapter.Fill(dataSet);
        cmd.Connection.Close();

        // crete the objects
        foreach (DataRow row in dataSet.Tables[0].Rows)
        {
            PostCode postcode = new PostCode();
            postcode.ID = Convert.ToInt32(row["PostcodeID"]);
            postcode.Outcode = row["OutCode"].ToString();
            postcode.Latitude = Convert.ToDouble(row["Latitude"]);
            postcode.Longitude = Convert.ToDouble(row["Longitude"]);
            postcode.TownID = Convert.ToInt32(row["TownID"]);

            AllPostCodes.Add(postcode);
            postcode = null;
        }
    }
}

Rebuilding takes place every 1 hour. this ensures that every 1 hour the site has a fresh set of cached data.

the problem with ive was due to the fact that sometimes if during recovery the server gets hit with a request and an exception is thrown. The exception is "Index out of bounds of array". this is due to when the list is cleared.

ListPostCodes.AllPostCodes.Clear(); - // throws exception - although its not always in regard to this list.

Once this exception is thrown, all users will be affected. I have to restart the server to fix it.

I have 2 questions ...

  • If I use caching instead of static objects, does this help?
  • Is there a way I can say "while the rebuild is in progress, wait for it to complete until requests are accepted"

any help most appricaiated;)

truegilly

+2


a source to share


4 answers


1 If I use caching instead of static objects help?

Yes, everything you do is easier to do with the caching feature that is built into ASP.NET

Is there a way I can tell? rebuilding is in progress, waiting for it to complete before accepting requests "

The general template looks like this:



  • You are requesting data from the data layer

  • If the Datlayer sees that there is data in the cache, it serves data from the cache. If there is no data in the cache, the data is requested from the db and put into the cache. After that, it is provided to the client

There are rules (CacheDependency and Timeout) when clearing the cache.

The simplest solution is to stick to this pattern: this way the first request will go to the database, and the other requests will be sent from the cache. You trigger the update by implementing SQLCacheDependency

+1


a source


You have to make sure that your list is not being modified by one thread while other threads are trying to use it. This would be a problem even if you were using ASP.NET cache, since collections are simply not thread safe. One way to do this is to use SynchronizedCollection instead of List. Then, when accessing the collection, make sure to use the following code:

lock (synchronizedCollection.SyncRoot) {
    synchronizedCollection.Clear();
    etc...
}

You will also have to use a lock when you read the collection. If you list it, you should probably make a copy before doing this, as you don't want to block for a long time. For instance:



List<whatever> tempCollection;
lock (synchrnonizedCollection.SyncRoot) {
    tempCollection = new List<whatever>(synchronizedCollection);
}
//use temp collection to access cached data

Another option is to create a ThreadSafeList class that uses internal locking to make the list object itself non-thread safe.

0


a source


I agree with Tom, you will need to do sync to make this work. One thing that will improve performance is not clearing the list until you actually get the new values ​​from the database:

// Modify your function to return a new list instead of filling the existing one.
public static List<PostCode> GetAllPostCodes()
{
    List<PostCode> temp = new List<PostCode>();
    ...
    return temp;
}

And when you rebuild the data:

List<PostCode> temp = GetAllPostCodes();
AllPostCodes = temp;

This will ensure that your cached list is still valid while GetAllPostCodes () is executed. This also has the advantage that you can use a read-only list, which makes synchronization easier.

0


a source


In your case, you need to update the data every one hour.

1) IT must use a cache with an absolute expiration set to 1 hour, so it expires every 1 hour. Check the cache before using it by doing NULL check.If its NULL get data from DB and fill the cache.

2) With this approach, the disadvantage is that the data may be 1 hour old. Therefore, if you want to get the most recent data at all times, use SQLCacheDependency (PUSH). so whenever there is a change to the select u command, the cache will be updated from the database with the updated data.

0


a source






More articles:


All Articles