Have Geek Brain

Handling very large object lists without swapping?

I have a class that can contain many small items in a list. Looks like:

public class Farm {

    private ArrayList<Horse> mHorses;
}

just wondering what happens if the mHorses array gets as crazy as 15,000 elements. I guess trying to write and read this from the datastore would be crazy because I would die in the serialization process.

It's important that I can get the entire array in one shot without swapping, and each Horse element can only have two string properties, so they are pretty lightweight:

public class Horse {
    private String mId;
    private String mName;
}

I don't need these horses indexed at all. Is it smart enough to just store the mHorse array as a raw text field and force my clients to deserialize? Sort of:

public class Farm {
    private Text mHorsesSerialized;
}

then whenever the client gets a Farm instance, it has to take the raw horses string and split it to restore the list, e.g .:

// GWT client perhaps
Farm farm = rpcCall.getMyFarm();
String horsesSerialized = farm.getHorses();
String[] horseBlocks = horsesSerialized.split(",");
for (int i = 0; i < horseBlocks.length; i++) {
    // .. continue deserializing the individual objects ...
}

Yes...

so hopefully it would be quick to read the Farm instance from the datastore and the serialization penalty is paid by the client,

thanks

+2


a source to share


2 answers


In general, it is not recommended to use lists unless your lists are short (not here!), Or need to be indexed (also not here). You also need to keep in mind that the maximum size for a serialized object is 1 MB, so whatever serialization mechanism you use must match 15,000 records per MB.



If they do fit, then yes, using your own serialization to the Blob field (and not the textbox unless you are using a text format like JSON) is your best option.

0


a source


Another consideration you might want to take into account is that you will make one big request from GWT and presumably use the response from that request to create your 15,000 horse user interface.

It won't be fast, and as long as it fights back from building a roster of your horses, the rest of the UI will be pretty much unusable.

What's more, the UI will be nearly impossible to navigate once it is displayed. What is this user interface for? A list of horses to choose from? Do you want to sift through 15,000 records to find the one they like?



Pagination is not only a way to relieve stress on the server, but also to relieve stress for the browser and the user.

You might want to look into some way so that users can search or filter your list, otherwise it will be nearly impossible for them to use them. And for that, you probably want to restructure your data model.

0


a source






More articles:


All Articles