Using polling on a file-like object returned by urllib2.urlopen ()?
I ran into the error described at http://bugs.python.org/issue1327971 while trying to poll the file-like object returned by urllib2.urlopen ().
Unfortunately, being relatively new to Python, I cannot figure out how to work around the problem from the answers, as they seem to be mainly aimed at fixing the bug rather than cracking the code that makes it work.
Here's a distilled version of my code that is throwing the error:
import urllib2, select
if __name__ == "__main__":
p = select.poll()
url = "http://localhost/"
fd = urllib2.urlopen(url)
p.register(fd, select.POLLIN | select.POLLERR | select.POLLHUP | select.POLLNVAL)
result = p.poll()
for fd, event in result:
if event == select.POLLIN:
while 1:
buf = fd.read(4096)
if not buf:
break
print buf
And the error thrown when running on python 2.6:
Traceback (most recent call last):
File "/home/shab/py/test.py", line 9, in <module>
p.register(fd, select.POLLIN | select.POLLERR | select.POLLHUP | select.POLLNVAL)
File "/usr/lib/python2.6/socket.py", line 287, in fileno
return self._sock.fileno()
AttributeError: HTTPResponse instance has no attribute 'fileno'
Update: I don't want to modify system libraries.
a source to share
If you don't want to change your system libraries, you can also patch httplib
on the fly to match the patch in the bug report:
import httplib
@property
def http_fileno(self):
return self.fp.fileno
@http_fileno.setter
def http_fileno(self, value):
self.fp.fileno = value
httplib.HTTPResponse.fileno = http_fileno
# and now on with the previous code
# ...
Then you will get an error message fd.read(4096)
because what fd
is returned poll
is the file value of the raw file, not a file-like object. You probably need to use the original object to read the data, not the polled return value.
a source to share
It looks like you want to change urllib with this patch . Please be aware there is a reason this code has not been released. It has not been fully reviewed.
EDIT: Actually, I think you want to change httplib with a different patch .
a source to share