Using Nose with IronPython

nose is a testing library for Python. nose is a popular alternative to unittest, which is part of the standard library, and boasts features like:

• Test autodiscovery
• Output capture
• Not restricted to class based tests
• Class and module level setUp and tearDown
• Assert introspection
• Test generators
• Plugin mechanism
  

Personally I don't find the object oriented way of organising tests a problem; test methods grouped in classes seems very logical to me. I also like assert methods and the richer error messages they provide. I usually end up defining all sorts of convenience assert methods myself that do the heavy lifting of my testing anyway. I'm sceptical of the magic that libraries like nose and py.test do to provide some of the same information with bare asserts - and that magic doesn't work with IronPython anyway.

The test discovery stuff is invaluable though. Whenever I create a new project I inevitably end up recreating some variation that does test discovery for the project with unittest. It looks like there is a strong move to get test discovery for unittest into the standard library. Unfortunately it is the sort of subject that tends to attract bike shed discussions so it may take a while! Unless someone is actively working on it I will implement something myself (for inclusion in the standard library or not).

Because of the magic that nose does, plus some abortive early experiments when IronPython was much younger, I've always assumed that nose doesn't work with IronPython. Thankfully Darrell Hawley has mostly proved me wrong. There is a big performance problem, but it does work:

• Running nose with IronPython

I just spent the past week at the 2009 Microsoft MVP Summit where I got a chance to talk to the some of the crew behind the dynamic languages at Microsoft. When I mentioned that I had gotten nose, a Python unit testing framework, working with IronPython, I was strongly encouraged to blog about it. Without further adieu, here's my code.

import sys
sys.path.append(r"C:\Python25\Lib\site-packages\nose-0.10.4-py2.5.egg")
sys.path.append(r"C:\python25\lib")
sys.path.append(r"C:\Python25\Lib\email")
sys.path.append(r"C:\Python25\Scripts")
import nose
nose.main()

I dropped this script into a directory with a couple of bogus unit tests and ran it with Python 2.5.4. Results were consistent with my expectations - successful tests passed and failing tests threw exceptions. When I ran the same suite against IronPython 2.0.0, I still received the correct results but the script took 1 minute and 20 seconds to complete. Compare that to less than a second (estimated) for CPython.

The root of the problem appears to be the large number of GeneratorExitExceptions being thrown. I'm not sure why they're being thrown, but I'll continue to investigate and report my findings.