Client side caching issue?
I want to cache data on the client. What is the best algorithm / data structure that can be used?
Case 1. The data being stored requires extremely fast string lookups.
Case 2. The cache dataset can be large. I don't want to blow up client memory usage, nor do I want to make network and disk access calls, which slows down my client side processing time.
Solutions:
Case 1: I think the tree / Tries suffix provides you with a good solution in this case.
Case 2: Two issues to consider here:
- For storing big data with minimal memory consumption
- Do not make any network calls to access any data that is not available in the cache. The LRU caching model is one solution I can think of, but it doesn't stop me from bloating memory.
Is there a way to write the file and access it without compromising the data (security aspect)?
Let me know if any point is not clear.
EDIT: Josh, I know my requirements are unrealistic. To narrow down my requirement, I am looking for something that is stored using the LRU algorithm. It will be good if we can have a dynamic size configuration for this LRU with the maximum limit. This will reduce the number of calls going to the network / database and provide good performance.
If this LRU algorithm deals with compressed data that can be interpreted with little overhead (but less than a network call), it will be much better.
a source to share
Check out all the available caching frameworks / libraries - I found Ehcache very useful. Also, you can only keep some (most recent) in memory and disk fault tolerance under certain memory usage. Disk calls will still be much faster than network calls and you won't use up all of your memory.
a source to share
Unfortunately, I think your expectations are unrealistic.
Insufficient memory usage, but also no disk access calls means you have nowhere to store your data.
Also, to answer your question about security, there is no client side data store (assuming you are talking about a web application) that is "secure". You can encrypt it, but that will destroy your speed requirements and also require server side processing. Anything stored and sent from the client is suspicious.
Perhaps if you could describe the problem in more detail, we can offer some realistic solutions.
a source to share