How do I navigate using URLConnection?
My app needs some kind of web cleanup feature. I have a URL object that downloads all the data. But I need to clear a lot of pages and I create a lot of url objects, so I open a lot of connections. How do I optimize it so that I can have one connection and only navigate to other pages with it?
Greetings
a source to share
As far as I can tell, you should have a different one URLConnection
for each url (which makes sense, since the network connection should change as well). I seriously doubt that creating this object is your bottleneck; I suspect this is network time, but without a profile, it's hard to know for sure.
For a moderate amount of pages, I would consider a work queue (say using ExecutorService
). For a lot of pages, I might even have a look at the Java Map / Reduce version .
Edit: In order for Map / Reduce to be better than a simple work queue, you need to have multiple computers to clean up.
a source to share
You can use Apache HTTP Components , it has many features including a connection manager that supports concurrent access
a source to share