Ian Bicking: the old part of his blog

WSGIRemote now HTTPEncode

I renamed WSGIRemote to be HTTPEncode. The relation to WSGI was only minimal -- it happens to know about internal requests, and serves up content via a WSGI application. But more generally it's just about serializing and deserializing HTTP requests and responses, not unlike FormEncode.

There's some simple examples; basic client usage looks like:

from httpencode import GET
data = GET('http://slashdot.org', output='lxml')
all_ids = [el.attrib['id'] for el in data.xpath('//*[@id]')]

I.e., you give the output type (current supported are lxml, etree, cgi.FieldStorage and BeautifulSoup and python), and it does its best to coerce the output from the page to that type. As you can see, several of these convert XML and/or HTML, and the library doesn't assume there is only one suitable Python data type for this. python refers to basic Python data structures -- right now only json produces such structures (which aren't really a class of anything).

There's still a lot I want to do with it -- probably move to httplib2 and maybe like httplib2 use a class that is constructed with any app-specific options (like a default encoding). And I have to implement insecure formats (like Pickle) with the necessary options, and somehow figure out how to keep the function signatures under control.

Created 08 Jan