Jeff Hardy: Django, Zlib and easy_install on IronPython

By Michael Foord -
Jeff Hardy is the creator of NWSGI, a .NET implementation of WSGI implemented as an ASP.NET HttpHandler for use with IIS 6 & 7. His goal is to be able to use Python web applications and frameworks like Django and Trac from IronPython.

He has been working on getting Django and easy_install working with IronPython. For easy_install one of the big hurdles is that the zlib module isn't available (even the version in FePy is missing a key part of the API that setuptools uses to unpack archives it downloads from PyPI). Getting setuptools working with IronPython will be a big win for using Python libraries and frameworks as more and more of them depend on being installed in this way.

He has written three blog entries on how far he has got with these issues.
I've had a couple of people ask about a Django + IronPython "Getting Started". This has been on my TODO list for a while; I just haven't gotten around to it. I'll try to do something over the next couple of weeks.

A couple of tips to anyone trying in the meantime:
  • Get the SQLite provider from FePy SVN; it's the only one that I know works
  • If manage.py has problems, file (or vote on) bugs, and use Python for manage.py instead
  • Set the session backend to use the caching system, and set your cache backed to be in-memory (you can try to use the ASP.NET backends I posted earlier, but no guarantees)
  • It probably won't work 100%, but I did work my way through the tutorial – except the admin system
Jeff has also been working on getting the zlib module working with IronPython, a prerequisite for using easy_install:
IronPython 2.0 does not include an implementation of the zlib module, which handles data compression. In CPython the zlib module is a C module, which it can't be used from IronPython. It is also required by several standard Python modules, include tarfile, gzip, and zipfile, all of which handle compressed archives. Thus, for setuptools - particularly, easy_install and zipimport (Eggs) - to work, there must be a fully working zlib module.

FePy includes a partial implementation built on .NET's DeflateStream, but it is missing support for the decompressobj API that is used by gzip and zipfile. Supporting this API using DeflateStream appears to be impossible, as it expects a few implementation details of the zlib implementation.
To get round this problem, Jeff has reimplemented zlib based on ComponentAce's zlib.net:
IronPython.Zlib is an implementation of the zlib module for IronPython using zlib.net. Both binary and source are available under the MS-PL license. To use it create a "DLLs" directory under your IronPython implementation and drop IronPython.Zlib.dll into it.

It passes most of the Python zlib tests and all of the gzip tests. There are a few issues with the CRC-32 implementation that the tests didn't catch but real-world usage did, but I haven't figured out how to fix them yet.

There are still some issues with setuptools (the zipfile module uses binascii.crc32, which is not yet implemented) that I'll detail in a later post, but it almost works.
Having got zlib working, Jeff moved onto easy_install:
Getting easy_install working for IronPython will be a big win for the IronPython ecosystem, and as I mentioned last time, the lack of zlib is really the only thing holding it back. Well, IronPython.Zlib solves that problem, so what else is holding up easy_install?

The big thing that's missing for setuptools is zipimport support, which is used to implement the Egg packages.
Jeff then details other minor changes needed to get basic easy_install working with IronPython:
Once you get this far, easy_install can download, extract, and build a Python library (I used simplejson for my tests). Installation is still a problem because pkg_resources is looking for an Egg, which require zipimport, which was blanked out in the first step (it's almost poetic, really). Getting zipimport working should be the last major hurdle.

Comments

Post a Comment

Popular posts from this blog

Extending Abobe Flash Player and AIR with Python and Ruby

By Michael Foord -
An intriguing blog entry about an Adobe experimental project. It is a C/C++ cross- compiler for the flash player and AIR. This means that you can (will be able to?) cross-compile the Python / Ruby/ lots of other language runtimes and then run them in Flash. The blog entry contrasts this approach to the DLR which requires the re-implementation of dynamic languages - hence IronPython and IronRuby. The project is already beyond proof-of concept, they even have Quake III cross-compiled and running in Flash. Extending Adobe Flash Player and AIR with C/C++ Cross-Compiler Quake 3 has already been ported to managed code , but this is even more impressive. Whether the Python interpreter can be cross-compiled to run on top of another runtime (with completely different basic types), without changing the language semantics at all, has yet to be proved - but it is a very interesting project. (It also means that Python/Ruby will possibly be enabled only by additional optional downloads - but don...
Read more

IronPython Tools for Visual Studio CTP3

By Michael Foord -
At PyCon this year Dino Veihland announced IronPython Tools for Visual Studio, an extension to Visual Studio 2010 for working with IronPython. It features Python syntax highlighting, awesome auto-complete (intellisense) and a host of other features for working with IronPython code in Visual Studio. It can be used with the free Visual Studio shell and doesn't require you to own a full copy of Visual Studio. The third CTP (Community Technology Preview) has been made available for download. IronPython Tools for Visual Studio We are happy to announce a minor update to the IronPython Tools for Visual Studio.  IronPython Tools for Visual Studio (IPyTools) is a set of extensions available for Visual Studio 2010 which supports development of IronPython applications.  This release is our 3rd Community Technical Preview (CTP) and builds upon the previous two releases.  The release is a minor update which includes bug fixes and a number of small features.  You can download...
Read more

Further Adventures of the Debugger

By Michael Foord -
Harry Pierson (Microsoft IronPython PM) has been experimenting with writing an IronPython debugger ( ipydbg ) on top of the .NET MDbg APIs. When we last left him , he had implemented Just My Code (JMC) support allowing you to step into IronPython code. Since then he has made a lot of progress including a colourful REPL (interactive console) and solving some difficult problems around getting to real values behind boxed IronPython variables (needed for showing local variables). He has blogged extensively on what he has been up to (more recent posts last): Showing Source Code Colorful Console IronPython ConsoleColorMgr (the colorful console code pulled out into a separate IronPython module) A Little Hack...err...Cleanup Getting Local Variables Displaying Values Command Routing Getting Arguments REPL Console A lot of functionality is now present. In the command routing entry Harry outlines the seven commands supported by the debugger so far: Continue, Quit, Show Stack Trace, Show Locals, ...
Read more