Increase multi_index_container performance and erase performance

I have a boost multi_index_container declared below which is indexed with hash_unique id (unsigned long) and hash_non_unique transaction id (long). Inserting and retrieving items is fast, but when I remove items it is much slower. I expected this to be constant time when the key is hashed.

eg To erase items from a container

for 10000 elements, it takes about 2.53927016 seconds

for 15,000 elements, it takes about 7.37100068 seconds

for 20,000 elements, it takes about 21.391720757 seconds

Am I missing something or expected behavior?

class Session
{
  public:
    Session () {
      // increment unique id
      static unsigned long counter = 0;
      boost :: mutex :: scoped_lock guard (mx);
      counter ++;
      m_nId = counter;
    }

    unsigned long GetId () {
     return m_nId;
    }
    long GetTransactionHandle () {
    return m_nTransactionHandle;
    }
    ....
private:
  unsigned long m_nId;
  long m_nTransactionHandle;
  boost :: mutext mx;
  ....
};
typedef multi_index_container<
  Session*,
  indexed_by< 
    hashed_unique< mem_fun<Session,unsigned long,&Session::GetId> >,
    hashed_non_unique< mem_fun<Session,unsigned long,&Session::GetTransactionHandle> >
    >  //end indexed_by
  > SessionContainer;
typedef SessionContainer::nth_index<0>::type SessionById;


int main() {
  ...
   SessionContainer container;
   SessionById *pSessionIdView = &get<0>(container);
   unsigned counter = atoi(argv[1]);
   vector<Session*> vSes(counter);
   //insert
   for(unsigned i = 0; i < counter; i++) {
     Session *pSes = new Session();
     container.insert(pSes); 
     vSes.push_back(pSes);
   }
   timespec ts;
   lock_settime(CLOCK_PROCESS_CPUTIME_ID, &ts);
   //erase
   for(unsigned i = 0; i < counter; i++) {
      pSessionIdView->erase(vSes[i]->getId());
      delete vSes[i];
   }
   lock_gettime(CLOCK_PROCESS_CPUTIME_ID, &ts);
   std::cout << "Total time taken for erase:" << ts.tv_sec << "." << ts.tv_nsec << "\n";
   return (EXIST_SUCCESS);
}

      

+2


a source to share


3 answers


In your test code, what's the value for m_nTransactionHandle

gets the object Session

? Could this be the same value for all objects? If so, the erasure will take a long time, as the performance of hashed containers is poor when there are many equal elements. Try assigning different values m_nTransactionHandle

to your build to see if your test speeds up.



+3


a source


When erasing an item, performance is a function of all the indices that make up the container (basically, the item should be removed from each index, not just the index you are currently working with). Hashes hit hard when there are many equivalent elements, this is not the pattern they are designed to work against.



0


a source


I just found that the performance for hashed_non_unique versus hashed_unique for the second index is almost the same except for a minor overhead of checking for a duplicate.

The bottleneck was with boost :: object_pool. I don't know the internal implementation, but it seems to be a list in which it iterates over the list to find objects. See link for work results and source code.

http://joshitech.blogspot.com/2010/05/boost-object-pool-destroy-performance.html

0


a source







All Articles