Jeff Hardy: Django, Zlib and easy_install on IronPython

Jeff Hardy is the creator of NWSGI, a .NET implementation of WSGI implemented as an ASP.NET HttpHandler for use with IIS 6 & 7. His goal is to be able to use Python web applications and frameworks like Django and Trac from IronPython.

He has been working on getting Django and easy_install working with IronPython. For easy_install one of the big hurdles is that the zlib module isn't available (even the version in FePy is missing a key part of the API that setuptools uses to unpack archives it downloads from PyPI). Getting setuptools working with IronPython will be a big win for using Python libraries and frameworks as more and more of them depend on being installed in this way.

He has written three blog entries on how far he has got with these issues.

• Django + IronPython

I've had a couple of people ask about a Django + IronPython "Getting Started". This has been on my TODO list for a while; I just haven't gotten around to it. I'll try to do something over the next couple of weeks.

A couple of tips to anyone trying in the meantime:

• Get the SQLite provider from FePy SVN; it's the only one that I know works
• If manage.py has problems, file (or vote on) bugs, and use Python for manage.py instead
• Set the session backend to use the caching system, and set your cache backed to be in-memory (you can try to use the ASP.NET backends I posted earlier, but no guarantees)
• It probably won't work 100%, but I did work my way through the tutorial – except the admin system

Jeff has also been working on getting the zlib module working with IronPython, a prerequisite for using easy_install:

• Solving the zlib problem

IronPython 2.0 does not include an implementation of the zlib module, which handles data compression. In CPython the zlib module is a C module, which it can't be used from IronPython. It is also required by several standard Python modules, include tarfile, gzip, and zipfile, all of which handle compressed archives. Thus, for setuptools - particularly, easy_install and zipimport (Eggs) - to work, there must be a fully working zlib module.

FePy includes a partial implementation built on .NET's DeflateStream, but it is missing support for the decompressobj API that is used by gzip and zipfile. Supporting this API using DeflateStream appears to be impossible, as it expects a few implementation details of the zlib implementation.

To get round this problem, Jeff has reimplemented zlib based on ComponentAce's zlib.net:

IronPython.Zlib is an implementation of the zlib module for IronPython using zlib.net. Both binary and source are available under the MS-PL license. To use it create a "DLLs" directory under your IronPython implementation and drop IronPython.Zlib.dll into it.

It passes most of the Python zlib tests and all of the gzip tests. There are a few issues with the CRC-32 implementation that the tests didn't catch but real-world usage did, but I haven't figured out how to fix them yet.

There are still some issues with setuptools (the zipfile module uses binascii.crc32, which is not yet implemented) that I'll detail in a later post, but it almost works.

Having got zlib working, Jeff moved onto easy_install:

• easy_install on IronPython, Part Deux

Getting easy_install working for IronPython will be a big win for the IronPython ecosystem, and as I mentioned last time, the lack of zlib is really the only thing holding it back. Well, IronPython.Zlib solves that problem, so what else is holding up easy_install?

The big thing that's missing for setuptools is zipimport support, which is used to implement the Egg packages.

Jeff then details other minor changes needed to get basic easy_install working with IronPython:

Once you get this far, easy_install can download, extract, and build a Python library (I used simplejson for my tests). Installation is still a problem because pkg_resources is looking for an Egg, which require zipimport, which was blanked out in the first step (it's almost poetic, really). Getting zipimport working should be the last major hurdle.