Tuesday, November 24, 2009

Cheminformatics Tutorial Using Python and Silverlight

Try Python is an online interactive Python tutorial using IronPython and Silverlight. The source code is available and open source and the tutorial material is written in a subset of the ReStructured Text markup.

Noel Baoilleach has taken this a step further by creating his own version, including the Webel library library and a tutorial on using it with interactive examples.
Recently I introduced Webel, a Python cheminformatics module that runs entirely on web services. One of the advantages of such a module is that it can be used in places where it is difficult to install a traditional cheminformatics toolkit. Like in your browser.

After some little work, I present Try Python...with Cheminformatics. This adds Webel as well as a short tutorial that introduces many of its features. With a few more tutorials that cover SMILES, InChI and so on in more detail, this could be useful for teaching purposes as well as bridging the gap to having students develop their own Python scripts that use the CDK, OpenBabel or the RDKit.
 Personally I think the Cheminformatics version of Try Python would be improved by the removal of the Python tutorial parts (only part 6 is Cheminformatics specific - making it a bit harder to find!). Creating custom tutorials using Try Python as a base is *fairly* easy (I'm sure it could be easier): it could be a great way of creating online examples of working with .NET or Python libraries.... I hope more people try this out.

Sunday, November 22, 2009

Hello GTK# from IronPython and F#

GTK# is a managed interface to the GTK user interface library. GTK# is part of the Mono project, but it can also be used from .NET on Windows.

In this blog entry, another one from Steve Gilham, he illustrates using GTK# from both IronPython and C# by porting some C# examples.
A little post to record a short bit of spiking with GTK# as UI toolkit, porting the simple C# examples from here to my preferred .net languages. Neither of these explorations are totally novel -- there are examples in either language to Google, but not all of them recorded all the details of the slight rough spots that needed a little working (and they were often not at all recent, either).

For IronPython 2.6 latest with GTK# 2.12.9-2, running the programs as ipy Program.py.

The dynamic nature of the language means we can lose a lot of the declaration clutter. We just have to explicitly reference the main GTK# assembly (which is GAC'd by the GTK# installer), and away we go.

A Good Mix 32: Alpha Encoding Files, Embedding IronPython in Russian, IronSharePoint, IronRuby, World's Worst Paint Program

More IronPython and DLR related projects, articles and news from around the web.
 Steve Gilham shows how to generated encoded version strings for installers using IronPython:
When building installers the UpgradeVersion must have a unique property value that is an installer public property (upper-case alpha). So, what better way of adding uniqueness than making it have the form "product name + product version" with the version suitably encoded...

So, a script for turning a file version (4 x 16bit ints) encoded as a System.Version into a short alpha string, assuming that Major and Minor will be small, and that common approaches are to step Build, to use a stepped Build plus date-stamped Revision, or a timestamp Build and Revision.
An article on IronPython, and in particular using IronPython from C#, in Russian.
 A while ago Christian Glessner announced the start of the IronSharePoint project: showing how to develop SharePoint applications with IronPython. Well, he hasn't released anything but there is now code available for download from the Codeplex site. Could be worth exploring if you work with SharePoint...
 Since early 2007 John Lam has been working for Microsoft as program manager of IronRuby, the implementation of Ruby on the Dynamic Language Runtime. Up until almost eleven months ago that is. John quietly transferred to a 'secret' project within Microsoft, a project that has also claimed Curt Hagenlocher (IronCurt). As they are *close* to being able to announce the secret project John has made his departure from the IronRuby team official and handed over the reigns to Jimmy Schementi who was previously only in charge of the integration between dynamic languages and Silverlight. Now Jimmy is doing both.
 Dino Veihland presented on IronPython and the Dynamic Language Runtime at the recent PDC conference. Part of his talk featured the "World's Worst Paint Program", which from Dino's description includes both IronPython and IronRuby code. He has made this available for download from the IronPython Codeplex Site.
 This is built using Visual Studio 2010 Beta 2 and the IronPython CTP for that release - so you'll need to download both of those.  I've also included the Python and Ruby code that I used during the demonstration in a file called code.txt.  The paint program it's self is in the final form at the end of the demo.

Saturday, November 21, 2009

Face Detection with IronPython

Face detection in images is very cool (and perhaps a little bit scary), and with the help of Emgu CV it can be done from IronPython.
Emgu CV is a cross platform .Net wrapper to the Intel OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc. The wrapper can be compiled in Mono and run on Linux / Mac OS X.
This blog entry from Clark Updike shows how.
 A couple notes compared to the Emgu wiki version--I was able to get everything working without having to disturb the IronPython install (I didn't copy any files into the IronPython start directory). I simply linked used sys.path to add the locations where the dll's live. Also, I did explicit import for everything. It's a bit more work to figure out what needs to be imported, but it avoids namespace pollution issues. Also, the img.Draw line needed a few tweaks. I had to drop the [float] bit from the original. And with the original version, I was also getting an error on this same line, saying:

TypeError: expected MCvBox2D, got MCvAvgComp

I was able to work around this by passing in the ".rect" rectangle attribute to the Draw method (I'm guessing the api has evolved since the 1.4 Emgu version used for the original). Lastly, I was getting to objects detected--so I used parameters to the detector to only get the single face in the image (basically by telling it to use a larger minimum detection size, minSearchScale). Anyway, now it works the way I was hoping.

Databinding and WCF Services with IronPython 2.6

One of the important new features in IronPython 2.6 is the __clrtype__ metaclass.The __clrtype__ metaclass allows you to create a real .NET class that backs your Python classes. This is important because there are many .NET features that *require* a real .NET class: which includes databinding and implementing WCF services (Windows Communication Foundation).

The problem with __clrtype__ is that it requires dealing with low level details; namely building the class yourself from IL bytecode. Harry Pierson and Shri Borde have been working on a library (clrtype.py) to make this simpler.

Lukáš Čenovský has looked at this before but hit limitations with what clrtype made possible. In three new blog entries he demonstrates how to use __clrtype__ with databinding in WPF and Silverlight and to implement WCF services.
 INotifyPropertyChanged is important interface for building WPF or Silverlight applications using M-V-VM concept (MSDN article).

In simple language, you have a Model which provides access to your data (e.g in database, files, web, etc.). Then you have a ViewModel that access data in the Model via Model's interface and provides data to a View which is XAML file with UI layout. Linkage between ViewModel and View is done by binding that utilizes PropertyChanged event to properly update all UI elements.

I have found two examples how to implement INotifyPropertyChanged interface in IronPython. The first one uses __setattr__ hook. Personally, I don't like it - it is not clear and easily readable code. The second one is better because it uses properties. But you have to write self.OnPropertyChanged("my_property_name") for every property. Not ideal.

That's why I sit down and write a notify_property.
This article uses the updated clrtype module to apply the databinding technique to Silverlight. The cool thing is that I'm already using this technique for databinding with the Silverlight Toolkit DataGrid in my new job. No more stub C# proxy classes, yay!
 Even using __clrtype__ you still couldn't use WCF (for creating and consuming web services) from pure IronPython because it requires *creating* interfaces, which couldn't be done. With the latest improvements to clrtype it is now possible.
I wrote about implementing WCF service in IronPython a couple of weeks ago. Meanwhile I pushed Shri a little bit with the clrtype.py and he has implemented ClrInterface metaclass there so we can create the whole WCF service in IronPython now.

The IronPython interface implementation is straightforward.
Also switching from C# interface to IronPython interface is easy.

Friday, November 20, 2009

IronPython 2.6 Release Candidate 3

IronPython 2.6 is the up-and-coming version of IronPython targeting compatibility with Python 2.6. As well as the new features in Python 2.6, IronPython 2.6 has several important new features specific to IronPython. These include:
  • The __clrtype__ metaclass for data binding and .NET attribute support
  • Implementation of the ctypes module
  • Support for Python stack frames and sys.settrace, which means the pdb debugger works
  • Better performance through adaptive compilation
  • Faster startup
IronPython 2.6 Release Candidate 3 has just been released. The hope is that this will be the last release candidate before the final release:
We’re pleased to announce the third and hopefully final release candidate of IronPython 2.6. Release Candidate 3 only includes Silverlight-related changes pertaining to some incompatibilities between 2.6 RC1 and RC2. Those who utilize IronPython for non-Silverlight scenarios will happily find virtually no churn from RC2. We strongly encourage everyone interested in Silverlight to test out this release ASAP because we plan on releasing IronPython 2.6 final in a week if no major new regressions are detected.

    Two Articles: IronPython 2.0 and WPF Error

    Two more articles from Ibrahim Kivanc, the Turkish blogger who has written several articles on IronPython and Silverlight. Both of these articles are in English.
    IronPython 2.0 version now runs on DLR (Dynamic Language Runtime). DLR is a platform on .NET which is host Dynamicly typed languages on it. Now Dynamic Languages Communicate eachother and C#,VB, COM Objects, .NET Libraries.

    IronPython, with 2.0 version runs on DLR (Dynamic Language Runtime); it’s a platform like CLR architecture. It’s host for Dynamic Languages on .NET. With this architecture Dynamic Languages now faster then running on CLR and easily communicate with other .NET objects!
    In my opinion IronPython Studio is not stable enough for production use. It does have the advantage of being integrated in Visual Studio so some people can't resist trying it out. (You can read my write-up of IronPython Studio at: IronPython Tools and IDEs.)

    If you insist on trying out IronPython Studio there are various minor trials and tribulations for you to overcome; using the WPF designer is one of them. In this article Ibrahim explains how to jump over this particular hurdle and get the WPF designer working in IronPython Studio.

    Sunday, November 15, 2009

    Embedding IronPython in Silverlight - Importing

    Jimmy Schementi is the Microsoft program manager for the integration of dynamic languages (IronPython and IronRuby) and the Silverlight browser plugin. As well as dynamic languages, Silverlight applications can be written in .NET languages like C# and VB.NET - and these languages can embed IronPython.

    Although embedding IronPython in a C# Silverlight application is initially straightforward (although a bit more verbose than embedding using the normal .NET framework / Mono as the ScriptRuntime needs to be configured), it gets painful fast. This is especially true when the Python code you execute needs to import anything, which has basically been broken on Silverlight for quite some time.

    In his latest blog entry Jimmy looks at how embedding from Silverlight has got simpler recently, and also at fixing the import problems.
    I’ve heard plenty of times on the IronPython Mailing List that embedding IronPython in Silverlight is easy at first, but then you fall off a cliff when trying to import, be it from a .NET namespace, the Python built-ins, or even other Python files. Let me clear some things up, and fix some code in the process.

    Slight detour – Embedding in Silverlight 101
    First off, Michael Foord has an article on his website about Embedding IronPython in Silverlight, which still works. However, that’s a ton of complicated code, all having to do with creating a ScriptRuntime instance correctly configured for Silverlight. To make this much less verbose, DynamicEngine.CreateRuntime (in the Microsoft.Scripting.Silverlight namespace) will create a ScriptRuntime all prepped for use in Silverlight; this significantly reduces the boilerplate hosting code:

    A Good Mix 31: Texas Holdem, IronScheme, Indigo Cheminformatics, IronRuby and bridge xml

    More IronPython and DLR related projects, articles and news from around the web.
     Greg Bray has been experimenting with using Resolver One to write Texas Holdem calculating spreadsheets with IronPython. In this blog entry he shows the IronPython code for calculating hand equity:
    This year I have been working in my free time to create easy to use Texas Holdem poker spreadsheets based on IronPython using Resolver One. These spreadsheets can be used to calculate Win/Tie/Loss odds, but some people like to use hand equity instead since it represents a player’s overall stake in the pot. Equity of 1.0 or 100% means that they will win the entire pot, where as equity of 0.5 (50%) or 0.25 (25%) means that they will split the pot with other players. You can again use Monte Carlo analysis to run a number of trials before all of the board cards are dealt to estimate a player’s current equity in the hand. This means that if during 4 trials I would win 2 of them and split the pot with one other player on the other 2, then my equity would be (1.0+1.0+0.5+0.5)/4 =  3/4 or 75%.

    Calculating Win/Tie/Loss statistics on an independent player vs player basis is very simple, as the player’s pocket cards are evaluated against each opponent one at a time to show which opponent hands are stronger or weaker than your own. This does not take into account how the pocket cards of other opponents might change the odds, since in reality you usually play verse a table of opponents not just one opponent at a time. Changing the spreadsheet to work against the whole table would require changing the Monte Carlo simulation from a “row by row” analysis to a “table by table” analysis, where by you setup a table of opponents using a random hand for each player given the information that you have about their hand (AK, KK or QQ, Group1, etc..) and then generate a random 5 card hand based on the currently dealt cards.
    IronScheme is an implementation of R6RS Scheme for .NET and Mono implemented on the Dynamic Language Runtime. You can either use it for writing .NET applications or for embedding into other applications to provide a Scheme scripting environment:
    IronScheme provides 2 run modes: either via direct invocation on the commandline or via a REPL interactive environment.
    IronScheme can also used an embedded scripting library in your .NET programs.
    With 1.0 RC 1, IronScheme implements the entire R6RS Scheme standard, and only fail 4 of 8884 tests in the PLT R6RS test suite.
    IronScheme runs on MS .NET and Mono. Unfortunately, it does not seem to work under 64bit on Mono.
    Since the release of 1.0 Release Candidate 1 leppie has also written a blog entry demonstrating the use of the new dynamic keyword in C# 4.0 with IronScheme: IronScheme and C# 4’s new dynamic keyword
    Noel Baoilleach demonstrates using the open source Indigo cheminformatics toolkit from IronPython.
    Charlie Zhu has already written an example application using C#. Rather than wait for CPython bindings, I installed IronPython and used it to access Indigo's .NET libraries (Dingo, in this case) to do a SMILES to png conversion.
    A new release of IronRuby, IronPython's red-headed cousin that is still on it's path to 1.0. Release Notes:
    The IronRuby team is pleased to announce version 0.9.2! As IronRuby approaches 1.0, these 0.9.x releases contain important bug fixes and enhancements that IronRuby users request; this release fixes/closes 44 bugs. For more information see the "Bugs fixed" section, and the CHANGELOG.

    This release is the first to give IronRuby a MSI-based installer for Windows. Try it out and let us know if it does not install on your system.
    Bridge is a "General purpose XML library for CPython and IronPython". It provides a common interface (Document, Element, Attribute, Comment and PI) for working with XML whether you are using CPython or IronPython.

    Saturday, November 14, 2009

    IronPython at PyCon 2010

    PyCon 2010, the annual international Python conference is coming soon (February 17th 2010 in Atlanta US). The schedule of talks is now up, and as usual there are several IronPython related talks.
    • 67. IronPython Tooling
     By Dino Veihland, core IronPython developer:
    One of the most popular requests for the IronPython team is tooling support. During this talk I’ll show you some of the existing tools available to help create IronPython applications. I’ll also look at the latest IronPython features which can help you debug, profile and improve your applications. I’ll also compare and contrast these with the solutions available for CPython that you may already be familiar with.
    • 71. Python in the Browser
    By Jimmy Schementi (Microsoft program manager for IronPython / IronRuby Silverlight integration):
    You write your server code in Python because you want to. You write your browser code in JavaScript because you have to. But with IronPython and Silverlight, you can write your browser code in Python and leave JavaScript behind. This talk provides an overview of Silverlight for Python developers, including out of browser applications as well as the new “just text” model where Python code is embedded directly in HTML pages. Even if you develop on Linux or Mac, you can still take advantage of Silverlight in the browser.
    As well as these talks the conference starts with a Virtual Machine (VM) summit, with attendees from the major implementations of Python plus those implementing other popular dynamic languages and Virtual Machines for dynamic languages.

    Dave Fugate (IronPython tester and infrastructure guy) is also planning a Testing IronPython poster session.

    After the summits are the tutorials and I will be repeating the IronPython tutorial I gave last year. If you're interested in getting some hands on experience of developing with IronPython then come along.

    For those interested in alternative implementations of Python you will also appreciate these talks:
    • 148. Pynie: Python 3 on Parrot
    By Allison Randal of the Parrot Foundation:
    Pynie is an implementation of Python 3 on Parrot. The goal of Pynie is to duplicate the pure-Python behavior of CPython 3.x, and perhaps eventually a C API compatibility layer. Parrot provides a set of compiler tools and core functionality common to many dynamic languages, so the core code of Pynie is lightweight. Pynie allows Python 3 libraries to be shared between multiple languages. This talk is an overview of the features Pynie currently supports and the work left to be done, together with an introduction to the internals of Pynie.
    • 13. How and why Python is being used to by the Military to model real-world battlefield scenarios
    By Mr. Eric Silverman (ALATEC Inc. / US Army):
    Leveraging the power of Python, military analyst and software developers out at White Sands Missile Range in New Mexico have converted a closed-form model, which was developed for scripted Cold War scenarios, into a more sophisticated and adaptive system. COMBATXXI is a premium wargaming model, which now is poised to answer questions that have a direct effect on military operations currently in theater, in large part because of Python.
    • 65. Extending Java Applications with Jython
    By Frank Wierzbicki (core Jython developer):
    Jython is an implementation of Python for the JVM. This talk covers the options available for integrating Jython and Java code, especially Java calling into Jython, since this is the trickier case. This talk will also cover some Java specific deployment options, such as deploying your Jython application as a jar file. Jar files allow you to package Java/Jython applications into a single archive. In many environments a user can click on a jar file to launch the contained application.
    • 83. The speed of PyPy
    By Mr. Maciej Fijalkowski (PyPy core developer):
    The first part of the talk will cover PyPy's speed achievements resulting from the last year's work on the Just-In-Time Compiler. I'll present and discuss a number of benchmarks and compare against other Python-speed projects. I'll also cover the basics of how the JIT works and what sort of programs it can greatly speedup (and which ones it can't).
    • 101. How to write cross-interpreter Python programs
    By Mr. Maciej Fijalkowski (PyPy core developer):
    This talk will cover basics about writing cross-interpreter
    python programs. What to do and most of all what not to do. This will help you if you want at some point in time to
    run program on for example Java platform or faster python
    interpreter, but also if you want to keep it running between CPython releases.
    • 189. The Ring of Python
    By Holger Krekel (PyPy and py.test developer):
    CPython 2.5/2.6/3.1, Jython, IronPython, PyPy, StacklessPython, UnladenSwallow, Cython ... what do we make of all these cool interpreter projects and versions? Where does competition help and where does it hamper?

    This talk is for stirring discussion and thoughts on making the most out of wealth/curse of interpreters. I am also going to showcase and discuss 'execnet' one of my own related projects.

    A Good Mix 30: Visual Studio 2010, DevDays, Detecting 64 bit, Silverlight and Django

    More IronPython and DLR related projects, articles and news from around the web.
     Visual Studio 2010 doesn't come with IronPython support out of the box, but it does have extensive APIs for writing your own extensions. Jeff Hardy has taken up the challenge and has written a set of IronPython extensions for Visual Studio 2010 with syntax highlighting, regions etc. The project has binaries available: "Just double click the .vsix file to install."
    At the end of October I attended the StackOverflow DevDays in Cambridge UK and spoke on Python and IronPython. I demonstrated .NET integration with IronPython by creating a simple Windows Forms application at the interactive interpreter. This was followed by going through Peter Norvig's Python Spell Checker as an example of concise Python code. Thanks to Ned Batchelder for the slides of his Boston presentation, which served as inspiration.

    The links above are three reports from the day, the first two positive. My favourite quote being "It was a huge surprise that the Knuth-packing beardo in the front row turned out to be a Windows developer, demoing on IronPython". But like Jeff Atwood said in his closing presentation; if someone doesn't hate what you do then you're doing it wrong. The third reporter didn't enjoy my presentation "I wasn't too keen on this presentation. There were statements like "Dynamic Programming makes it easy to test" and "Dynamic Programming means I type less" that I didn't agree with and there was no follow up to back it up". (See the comments on that entry for my reply.)
    A Japanese blog entry showing IronPython code (using WMI) for detecting whether your code is running on a 32bit operating system or a 64bit one.
    Yet another release from the mysterious Japanese IronPython and Windows Forms project that we've covered before.

     'xap' files are the way you serve Silverlight applications. django-xapper is  a project to build and serve xap files from Django, making it easier to integrate Python on the server with Python on the client for projects using IronPython and Silverlight.
    Xapper is a Django app to help facilitate the development of other Django apps that include Silverlight apps. In particular it is geared towards Silverlight apps implemented in IronPython or other dynamic CLR languages.

    Xapper serves a similar function as Chiron, which is now distributed with the IronPython Silverlight binaries. Both these apps can dynamically generate Silverlight packages (.xap files) from a directory of files and serve it over HTTP upon request. This means that all you need to do to pick up the changes you've made to your Silverlight app is reload the page in your browser, making development of Silverlight apps much nicer. The main difference between Chiron and Xapper is that Xapper is oriented towards integrating with Django, while Chiron is a standalone web server oriented only towards development of the client-side app.

    Thursday, November 12, 2009

    Using Solver Foundation and plug-in solvers in IronPython

    Microsoft Solver Foundation is a set of mathematical programming tools that can be used from .NET languages like C#, F# and IronPython:
    Solver Foundation Services (SFS) can automatically analyze models and determine which solver is most appropriate. If you are an advanced modeler, you can choose specific solvers and solver attributes. While solving the models, SFS manages all threading, many-core, synchronization, scheduling, and model execution issues. When finished, SFS produces reports about solver behavior and results, and provides additional information about solutions, including sensitivity. Finally, SFS allows LINQ data binding of model parameters and delivery of results in multiple formats.
    Lengning Liu has a blog entry on using the solver foundation from IronPython, including the magic incantations in app.config required to use plug in solvers.
    Solver Foundation provides an easy-to-use and flexible plug-in infrastructure as an addition to its already rich set of solvers. Users can choose their favorite plugin solvers to solve the model, and yet leverage all features provided in Solver Foundation Services (SFS).

    The information such as assembly name and solver class name can usually be found from the plug-in solver provider.

    The config file then will co-exist with the compiled exe file of the user app. When, inside the app code, a Solve call is issued, SFS will pick a solver based on the model kind, directive instances passed in, and the solver registration.

    The app code in this case can be written in any .NET compliant language such as C# or VB.NET.

    IronPython is “an implementation of the Python programming language running under .NET …” So it is possible to use Solver Foundation, and with plug-in solvers, in IronPython code. However, one difficulty is that Python is an interpreted language. Therefore we cannot compile Python code to an exe file. So how do we register our plug-in solvers?

    The trick is to register the plug-in solvers inside ipy.exe.config (or ipy64.exe.config). These two files must be put under the same folder with ipy.exe and ipy64.exe. We assume IronPython 2.6 RC2 in the following discussion.

    Scripting Your .Net Applications with IronPython

    One of the major use cases for IronPython is embedding in .NET applications to provide user scripting. The hosting APIs make it easy to experiment with embedding IronPython.

    Chris Umbel has written up a blog entry demonstrating the IronPython 2.6 hosting API and giving an example of exposing an API to Python scripts from a C# applications.

    At several points in my .Net development career I've had the need to make an application I wrote scriptable. Sometimes it was to provide easy product extension to customers or lower level information workers. Sometimes it was to ease maintenance of very fine grained logic that has the capacity to change frequently or unpredictably. But every time I found it to be one of the more interesting facets of the project at hand.
    Early in .Net's history this was made easy by using Visual Studio for Applications (VSA) which allowed you to host arbitrary C# or VB.Net code within the executing AppDomain. Unfortunately VSA was plagued with resource leak problems and was therefore impractical in most enterprise situations. VSA was eventually deprecated.

    One of the many alternatives is to perform dynamic, on-the-fly code compilation. While certainly quite manageable it was a bit more complex and much akin to cutting down a sapling with a chainsaw.

    Another option that came along later is Visual Studio Tools for Applications which brought the Visual Studio IDE to the scripter.

    My favorite avenue, however, is to host a Dynamic Language Runtime (DLR) and use a language like IronPython. Not only is it disgustingly simple to implement from a plumbing point of view but Python itself seems like a natural fit due to it's simplicity. IronRuby's another wonderful choice but I'll stick to IronPython for the scope of this post.

    Tuesday, November 10, 2009

    A WCF Service from IronPython

    Windows Communication Foundation (WCF) is part of .NET 3 and makes creating and interacting with web services from C# substantially easier. WCF makes extensive use of .NET attributes, making it difficult to use from IronPython versions before 2.6 without wrapping the use of attributes in some C#.

    IronPython 2.6 brings in support for attributes through the __clrtype__ metaclass. It doesn't yet allow you to create a .NET interface from pure Python code (although you can implement existing interfaces just by inheriting from them), so *some* C# is still needed for full use of WCF with IronPython 2.6. The dynamic languages team are looking at ways that __clrtype__ could be extended to allow the creation of interfaces in a future release.

    In his latest blog post Lukáš Čenovský demonstrates how to create a WCF service from C# and then shows the same code from IronPython, along with the minimal C# still needed.
    Until IronPython 2.6, it was not possible to create WCF service host in pure IronPython. The closest way was to create stub in C# and subclass it in IronPython or create the whole service in C# and run it from IronPython. It is now much simpler with IronPython 2.6 although you still have to write a little C# code.

    Getting Started with IronPython and Embedding IronPython

    Charlie Calvert is Community Program Manager for the C# group at Microsoft. He has posted a couple of blog entries on using IronPython. The first shows you how to get started with IronPython on Windows, including setting up your path to easily execute Python scripts with ipy.exe. The second entry shows how to execute a Python file from C# when embedding the IronPython engine.
    I recently spent some time getting IronPython up and running on my system; I will review what I learned in this post.

    Hosted inside an interpreter, Python belongs to the same family of scripting tools as VBScript, JavaScript, Perl and Ruby. You can fairly compare Python to a general purpose language such as C#. Developers praise this loosely typed, dynamic language for its ease of use and rapid development capabilities. IronPython is Microsoft’s free version of the open source Python language.

    IronPython can be hosted inside a C# program as a scripting language. Nonetheless, Python is a powerful standalone language frequently used as the glue in web based applications. Before I came to work for Microsoft I enjoyed using it as part of my daily toolkit.

    In this post I explore the simple steps needed to run IronPython as a standalone tool from the command prompt. In other posts I will show how to call Python from inside a C# application.
    IronPython is a scripting language hosted on the .NET platform. This posts shows how you can use the Dynamic Language Runtime (DLR) and the new C# 4.0 dynamic keyword to call an IronPython script from a C# program.

    Before going any further, it might be helpful to take a moment to explore the architecture that makes this technology possible. The key building block is the DLR, which sits on top of the .NET 4.0 CLR and provides the tooling for adding dynamic programs such as Python to the family of .NET languages. C# 4.0 adds the dynamic keyword and some related technologies to integrate support for dynamic binding into the language. The end result is that developers can arbitrary Python scripts directly from C# or other .NET language

    Monday, November 09, 2009

    Performance of IronPython vs. C#

    The Loose XAML blog has posted an interesting article profiling some IronPython code and the semantically identical C# and C code (for generating the fibonnaci series). He comes to the unsurprising conclusion that C# is substantially faster, but digs deeper into the results to try and work out why. An interesting discussion ensues in the comments.
     I’ve been using IronPython for a while, and every now and then I do a cursory check to see how it’s performs against the equivalent C# code. C# is generally a touch faster at most logic, but IronPython is faster at dynamic invocation of methods than reflection in C#. In general it’s a wash…over the course of an application you dabble in things that different languages are better optimized to handle. When performance is a wash, you get the freedom to choose entirely based on the features of each language, which is nice.

    This was going well, until I had a need to do some recursion. I found that the recursive IronPython code was executing extremely slowly – code that was taking milliseconds in C# was taking minutes in IronPython. That’s not a wash anymore…it’s a few orders of magnitude. So to really figure out what’s going on with these recursive calls, I made two implementations of calculating Fibonacci numbers – one in C# and one in IronPython.
    Of course performance is one of the tradeoffs you make when choosing between a dynamically typed language and a statically typed one. If your application is not computationally bound, but is network, database or I/O bound then the performance of your programming language *may* be more-or-less irrelevant. On the other hand, if a 'slower' language is more flexible and expressive, or you can develop faster with it, then you may be able to spend more time optimising your algorithms and still have acceptable performance.

    This has been my practical experience at Resolver Systems where we have always been open to the idea of moving core parts of the application into C# but have basically always been able to gain our targetted performance improvements by refactoring and optimising without having to rewrite in another language. With the extra code rewriting in C# would require (and studies show that bugs are pretty much proportional to lines of code irrespective of the language you use), and how much extra work C# is to test than Python,  keeping as much code as possible in Python is something that the development team appreciates.

    Talking to ActiveDirectory from (Iron)Python

    Active directory is one of the standard (and important) parts of a Windows network. Working with active directory is therefore a common task for a Windows system administrator. Naturally you can work with active directory from Python, but there is also good support for active directory in the .NET framework that can be accessed from IronPython (including from IronPython on Mono). This is something that Brendan McAdams has been exploring on the Evil Monkey Labs blog and has created a utility module to make it even easier:
     We're building a new intranet system at work, and I've been toying with a few things that the Windows admin asked for. Namely, since the secretaries here will update the intranet data to add people's Work & Emergency contact numbers, AIM handles, email addresses, etc. that we find a way to keep it all in sync with ActiveDirectory. Thereby keeping all the Outlooks and Blackberries up to date with the latest contact information.

    This seemed like a fairly reasonable request, presuming we could figure out how to do it and since I've been using Mono and IronPython a lot more lately, I figured there would be a way to accomplish it. Most of the information I found online was either really old and/or crappy docs for doing it in C#, or more commonly using PowerShell or VBScript. So, I managed to poke around and sort out how to get IronPython on Mono (IronPython 2.6RC + Mono to find and update our users.
    The end result is that I can now, from IronPython, find and update valid information on ActiveDirectory entries to reflect the latest and greatest information. One thing to note, the MS .Net ActiveDirectory APIs (System.DirectoryServices, which is mirrored in Mono) do something that confused and annoyed me. There are a limited set of 'valid' attribute keys for a user object in Active Directory (Which is really just LDAP, in case you didn't know). The DirectoryEntry object has a Properties attribute, which contains a hashmap of these values.

    The object will not allow you to set an "Invalid" key (see this list for valid keys). But if you call .Properties.Keys you only get back the Properties that have values set. So, it doesn't appear to be possible to actually ask What keys are valid and do some introspective programming. I have written a wrapper class to make the DirectoryEntry properties look a bit more pythonic (but disabled support for multi-value attributes for now) - at some point in the near future i'll likely add in a "valid value" filter.

    The end result is, if I want to find my own user in ActiveDirectory by my name, I can do the following from the IronPython console...

    Sunday, November 08, 2009

    FDO Toolbox (Geospatial Data Tool) IronPython Scripting

    FDO Toolbox is an open source "multi-purpose geospatial tool to create, analyse, process and manage spatial data. It is written in C# and uses the Feature Data Objects (FDO) API". Going on from his success at the Free and Open Source Software for Geospatial conference in Sydney, the main developer has announced a new version that includes IronPython scripting support for extending the toolbox.
    Here's a new release of FDO Toolbox, fresh from the FOSS4G 2009 conference. This was the exact build I was demonstrating during the presentation, which went better than I expected.

    The new main features include:

    IronPython Scripting

    With the introduction of a IronPython scripting engine, FDO Toolbox now has a 2nd extensibility point. You can now customise and drive FDO Toolbox through python scripts. I will be posting some example scripts in the near future to demonstrate what can be done with a scripting engine.

    The side-effect of introducing IronPython is that the minimum .net Framework requirement is now 2.0 SP1. The installer has been updated to check for this version. If you have .net Framework 3.5 (or 3.5 SP1) installed, you will already have this installed.

    Shortly after this release came a bugfix FDO Toolbox 0.8.8 release. His Foss4G conference presentation was recorded and you can watch it on his blog:

    A Good Mix 29: Small PIL for IronPython, iDalogue, Book Sales, Umbraco and PDC

    More IronPython and DLR related projects, articles and news from around the web.
     One of the disadvantages of developing with IronPython, particularly for using existing libraries and codebases, is that C extensions don't work. One possibility is to use Ironclad, a C extension compatibility layer by Resolver Systems. Another one is to find or create a compatible API using .NET libraries. An important C extension in the Python world, virtually ubiquitous where images are used and manipulated, is the Python Imaging Library. All sorts of other projects (Django and Reportlab for example) use PIL, causing problems for using them from IronPython.

    A Japanese blogger has *started* to solve this problem by creating an IronPython project called "small pil" implementing part of the PIL API for IronPython.
     i-Dialogue is a "Customer Experience Management" application by Cubic Compass. It is relevant to us because they use IronPython as a scripting language. They have just announced changes coming to "Dialogue Script" in December:
    The following announcement just went out to Dialogue Developers today. Dialogue Script is a Domain Specific Language (DSL) that runs on the IronPython Dynamic Language Runtime (DLR).

    As an i-Dialogue Developer working with Dialogue Script, I wanted to share a couple changes in the next release:
    • Dialogue Script will be upgraded to run on the newer IronPython v2.6 engine
    • JScript will no longer be supported
    The upgrade from IronPython from v2.0 to v2.6 will result in a tremendous performance improvement and provide access to a broader set of features and 3rd party libraries.
     IronPython in Action has now been on sale for about eight months. I've recently had the second quarter sales figures, which represent four months of sales:
    The first quarter started in January (strange that) but IronPython in Action only went on sale in March, meaning that the first quarter only included one month's sales. That month included all the preorders from places like Amazon, plus all the people who paid for the book during the Early Access Program, so they're surprisingly high.

    Thankfully the sales were enough to pay off the advance entirely and even yield an extra $2700 odd dollars, of which almost $2000 was held in reserve in case of returns. Sales in the first quarter were 2300 print books and 651 ebook sales.
     Umbraco is a .NET CMS built on ASP.NET; it supports scripting and extending with IronPython with the IronPython page type. This short blog entry shows how to use the Node API to access information from Umbraco pages:
    I was just working with the IronPython page type in umbraco and needed to get a property from the page I was on. This can be done by accessing the Node API found in umbraco.presentation.nodeFactory. In order to be able to pull a value you will need pull in that part of the API.
    Microsoft Professional Developers Conference 2009 (PDC) is November 17th - 19th in Los Angeles. On the Tuesday Dino Veihland (core IronPython developer) is giving a talk on dynamic languages and the DLR:
    The Dynamic Language Runtime (DLR) brings the power of dynamic languages to .NET. It provides the plumbing for IronPython and IronRuby, a shared lanaguage hosting API, and also enables interoperability with static languages like C# and Visual Basic. Come hear how you can leverage these technologies in your own applications, and learn why dynamic languages deserve a spot in your toolbox!

    Saturday, November 07, 2009

    QCon Presentation: Real World IronPython

    In March I spoke at the  QCon London 2009 conference on "Real World IronPython". InfoQ has just put up a video of the presentation, including the slides and a demo of "Resolver One", the highly programmable spreadsheet system created by Resolver Systems with IronPython.
    Michael Foord discusses IronPython, the DLR, dynamic languages on .Net, static vs. dynamic typing, Visual Studio integration, Resolver One, Intellipad, Crack.net, embedding IronPython, the ScriptEngine, error handling, dynamic operations, functions as delegates, and the C# 4.0 dynamic keyword.

    Michael Foord works full time with IronPython for Resolver Systems; creating a highly programmable spreadsheet called Resolver One. He has been using IronPython since about version 0.7, and has been developing with Python since 2002. He blogs and writes about Python and IronPython far more than is healthy for one individual and in 2008 was made the first Microsoft MVP for dynamic languages.

    About the conference

    QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community. QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.

    A Good Mix 28: NTornado, WPF, Testing in Italian, More Benchmarking, and adodbapi

    Yet another collection of IronPython and DLR related articles, projects and blog entries from the past few weeks.
    NTornado is an IronPython version of the Tornando web server. The Tornado Web Server is the open source version of the non-blocking web server that power FriendFeed and now part of the Facebook's open source initiative. This server is coded in Python and with strong emphasis on operating systems with epoll support.

    NTornado is a port of Tornado to IronPython using asynchronous high-performance sockets in .NET. To run the demos (requires IronPython 2.6):
    > ipy -X:Frames "demo file name".py
    I always find myself needing a control template so I can customize one of the WPF controls. I used to fire up Expression Blend to get it, and then realized I could write a little IronPython code to do it. Paste this code into the IronPython 2.0 or 2.6 console to see it work!
     A translation of James McCaffrey's article on testing .NET code with IronPython into Italian, for the MSDN magazine.
    Pyevolve aims to be a  "complete genetic algorithm framework written in pure python". The Pyevolve blog has benchmarked some genetic algorithms code running on all the major Python implementations: CPython 2.6 & 2.5, Unladen Swallow, IronPython, PyPy and Jython. IronPython 2.6 comes out slower than CPython (and Unladen Swallow) and faster than PyPy and Jython.
    adodbapi is a Python module for working with Microsoft ADO for connecting to databases and datasources. adodbapi is compatible with the Python DB-API 2.0 specification, meaning that it can be used by any DB-API compatible code. Thanks to the work of Vernon Cole, adodbapi now passes all tests when run on both CPython and IronPython.