Have Geek Brain

How can I pass a python object between two requests?

I want to process a python dict object in batches between two requests. I was wondering what is the best way to do it.

I want to do this because my dict is big and I couldn't do all the processing in 30 seconds.

thank

+2


source to share


1 answer


You can serialize your object ( pickle perhaps, although there may be more efficient and specific ways if the nature of the object is well -constrained) and store the serialized byte string in the data store and memcache (I do not recommend using memcache alone, because it can sometimes happen. that the cache has "blushed" between the two requests - in which case you definitely want to get your serialized byte string from the datastore!).

memcache

will be pick-up for you if you pass in the original object, but since you need a serialized string anyway to put it in the data store, I think it's better to do your own explicit serialization. Once you memcache.add

string it, the fact that the latter gets pickled (and then unopened when retrieved) doesn't really matter - the time and space overhead is really quite modest.



There are limitations to this approach: for example, you cannot cache more than 1MB per key, so if your object is truly huge, you need to split the serialized bytestring into multiple keys (and for more than a few such megabyte-slices, things get very cumbersome) ...

In addition, of course, the first and second requests must "match" the key to be used for sequentially storing and retrieving data, i. E. there should be an easy way to get this key without confusion (for example, it could be based on the name of the current user).

+4


source






More articles:


All Articles