DAL, session, cache architecture
I am trying to create a Data Access Layer for my web application. Currently all data is stored in the session. When I'm done, the DAL will populate and return datatables. Is it ok to store the returned data in the session? Distributed / shared cache? Or just ping the database every time? Note: Usually the number of rows in datatable will be small <2000.
Additional Information:
Almost none of the data is shared. The parameters that are sent to SQL queries are user-selectable. The parameter values available to the user are based on who the user is. In most cases, two users cannot run the same SQL queries. However, the same user can run the same query more than once.
Additional information: Number of concurrent users ~ 50,000
Important information: In 99% of cases, neither of the two users will have the same data / requests, however, the same user can run the same request / receive the same data multiple times.
thanks
a source to share
Saving data in a session is not a good idea because:
- Each user receives a separate copy of the same data - a huge waste of server memory.
- IIS will recycle the session if you fill it with too much data.
I recommend storing data tables in Cache and also only populating each table when requested first, not all at the same time. This way, if IIS starts reclaiming cache space, your code will not be affected.
A very simple sample on demand example:
T GetCached<T>(string cacheKey, Func<T> getDirect) {
object value = HttpContext.Current.Cache.Item(cacheKey);
if(value == null) {
value = getDirect();
HttpContext.Current.Cache.Insert(cacheKey, value);
}
return (T) value;
}
EDIT: - Update question
Cache versus local session. Local session state is all or nothing. If it gets too complete, IIS will recycle everything in it. In contrast, cache items are packed individually when memory gets too low, so the problem is much less.
Cache vs Session state server - I don't have any data to back up, so please tell me if I'm wrong, but I would think that caching data independently in memory on each physical AppDomain would scale better than storing it in a shared service session state.
a source to share
Store search / dictionaries - and items that your application will require very often in the object Application
or Cache
; query the database for data that depends on the user's role.
- EDIT -
This is a response to your comment.
Typically, in any data-centric system, queries are performed around a fact table (or tables that are inevitable for a query); assuming you have a set of unavoidable tables, so you can use Cache.Insert()
:
- Load the inevitable tables when starting the application;
- Loading the most requested tables into Cache based on a table query;
- Query database for the smallest tables requested.
If you don't have performance issues, let SQL handle everything.
a source to share
The first thing I would say is that cache is optional everywhere. You have to use it wisely and very specific for data access bottlenecks.
I don't think it's a good idea to store 1000 different data with 2000 documents anywhere. If the queries are so dynamic that there is an exception in the same query in a short amount of time, then the cache does not seem like a good option.
And regarding the distributed cache option, I suggest you check out http://memcached.org . A distributed cache used by many large projects around the world.
I know Velocity is close, but so far I know it needs Windows Server 2008 and this is still something very new. Usually Microsoft products are good since version 2.0 :-)
a source to share