Thursday, December 31, 2009

A Good Mix 34: Silverlight Logging, WPF and NotifyIcon, more Python and Ruby and pickling Python books

Another collection of IronPython and DLR related articles from around the web. A fine way to end 2009.
A nascent project to Port Log(4|5)J from Java to C# with the goal of usefulness in Silverlight, especially for IronPython.
 Two Japanese blog entries, both by sasakima-nao. As with previous entries the code examples are very readable. The first is a simple WPF picture viewer (nice penguins) and the second shows how to create a NotifyIcon and ContextMenu in the taskbar (with Windows Forms classes).
This blog entry is in Russian, but I think there are enough code examples for it to be useful for those of us who don't speak Russian. As I've mentioned before the promise of the Dynamic Language Runtime is that dynamic languages can interoperate and share libraries. This is exactly what this blog entry shows: using the Ruby soap/wsdlDriver from Python.

The cool thing is that this is done with a helper / wrapper library, that looks like it could be used to expose virtually any Ruby module / class that can be accessed through IronRuby to IronPython. Using his ruby module, the code looks like this:
from ruby import _import_, get_class
Soap = get_class('SOAP::WSDLDriverFactory')
client = Soap('http://localhost1/Service1.asmx?WSDL').create_rpc_driver()
Marcus McElhaney has discovered IronPython and likes it. The reasons he gives are:
1. I can very easy use everything I know about the .Net Framework, VB.Net, and  C#
2. I can run IronPython in Visual Studio
3. I can reference ESRI's  ArcObjects libraries in  Visual Studio and IronPython
What's even more gratifying is that he has been exploring IronPython through IronPython in Action and likes that too. A slight misspelling gives rise to my favourite quote about IronPython in Action:
IronPython In Action is probably the best Python book I have ever pickled up because it also explains a lot of not just Python but also about .Net.
If any book deserved to be pickled up, this is it... Have a great 2010.

Wednesday, December 30, 2009

Does Microsoft take Dynamic Languages Seriously?

My belief is that the answer to the question in the title of this entry is an emphatic yes. Microsoft have poured a lot of money into IronPython, IronRuby and the Dynamic Language Runtime and have demonstrated a consistent commitment since the inception of IronPython. What they haven't done is build full support into their premier development tool, Visual Studio. The reason for this is that it is a very difficult problem. Visual Studio is built around statically typed languages. Features like intellisense, refactoring and code navigation all rely on having type information which is absent in languages like Python and Ruby. (The way they are implemented in Visual Studio requires that information I mean.)

What Microsoft have done is provide example integration in the form of IronPython Studio, which frankly sucks. Many important features are fragile, broken or missing altogether. Good IDEs like PyDev and Wing do provide these features, so it is definitely possible - it just requires a lot of work from scratch.

Microsoft have however added intellisense for Javascript, particularly for jQuery, into Visual Studio. Javascript is of course a dynamic language, but without having tried the integration I can't tell you how well it compares to using a good Python IDE. I believe that Microsoft would like to add support for dynamic languages to Visual Studio and that it will come eventually. Really they have a lot to lose by not doing it.

Now they have a good platform for dynamic developers to use, which they give away free, but the tools they charge for don't really support them. This puts them in a position of either bringing developers to their platform but not being able to make any money, or having their existing developers stay with their platform but losing dependence on the toolset. Neither option looks much good for Microsoft.

(NOTE: for those who can't wait for official Microsoft support and have Visual Studio 2010, try Jeff Hardy's IronPython for Visual Studio Extensions.)

Anyway, K. Scott Allen sees things a little differently and I understand his frustration:
Consider this …

IronPython got underway in July of 2004. Five years later it appears IronPython is still not a candidate to be a first class language in the .NET framework and tools. You can vote on this issue.

Microsoft first released IronRuby at Mix in 2007. Nearly three years later it appears IronRuby is still not a candidate to be a first class language in the .NET framework and tools. You can vote on this issue.

A first class language is deployed when the full .NET framework is installed. It’s as easy to find as csc.exe. It’s not a language you have to ask the IT department to install separately. It’s not a language that requires you to jump out of Visual Studio to edit or run.

Most of all, a first class language doesn’t require justification to higher powers. A first class language is pre-certified and stamped with a seal of approval. It’s as easy to use in the locked-down big corporate setting as the company paper shredder.
Of course here we are talking about Microsoft as if it was a single entity with a single intention. The reality of course is that Microsoft is a huge company with many divisions and even more individuals working there. In all likelihood the vast majority of Microsoft employees have never heard of IronPython. The relevant division is the programming languages group, which includes Visual Studio development, and where the majority of employees who are developers probably have heard of IronPython if not actually used it... Adding official dynamic languages support to Visual Studio would require substantial investment of time and effort, so even if everyone in this department was determined to add support it would still be dependent on forces from other parts of the company who have other needs and priorities...

Ironclad 2.6 and Spare Batteries for IronPython

Ironclad is a compatibility layer that allows you to use Python C extensions with IronPython. Ironclad is open source and development has been funded by Resolver Systems and it is integrated into Resolver One to allow you to use Numpy within Resolver One spreadsheets.

Ironclad works by implementing the Python C API in a combination of C#, C and Python. Although Ironclad only works on 32 bit Windows at the moment the implementation has been done in such a way that porting it to run on other platforms (with Mono) and 64 bit would be relatively easy. Patches welcomed!

Recent development has changed the implementation to use gcc-xml to access and transform the Python C source code. By reusing as much of the original implementation as possible it minimizes the amount that needs to be 'hand-coded'. It leaves only a (moderately) small core that would need to be reimplemented if Jython, PyPy (or other implementations) wanted to reuse Ironclad. The C# would need to be re-coded in Java or RPython, using the platform's native FFI instead of PInvoke on .NET. The advantage of reusing Ironclad is that difficult problems, like handling the Global Interpreter Lock and matching reference counting to different garbage collection strategies, are already solved (well, for some value of solved...).

Anyway, that is all by way of introduction. William Reade, core developer of Ironclad, has just announced Ironclad 2.6 RC. This is a release candidate of Ironclad targeting IronPython 2.6.
I'm very happy to announce the latest release (candidate) of Ironclad, the 120-proof home-brewed CPython compatibility layer, now available for IronPython 2.6!

No longer need .NET pythonistas toil thanklessly without the benefits of bz2, csv, numpy and scipy: with a simple 'import ironclad', (most parts of) the above packages -- and many more -- will transparently Just Work.

Get the package from:

...and get support from:

...or just ask me directly.

I'm very keen to hear your experiences, both positive and negative; I haven't been able to test it on as many machines as I have in the past, so your feedback is especially important this time round*.


* I'd be especially grateful if someone with a newish multicore machine would run the numpy and scipy test scripts (included in the source distrbution) a few times to check for consistent results and absence of weird crashes; if someone volunteers, I'll help however I can.
William has recently started blogging. I recommend browsing the few entries he has already posted, particularly this rant on static typing and this post on .NET marshalling, but his latest is of particular interest:
As we all know, Python comes with batteries included in the form of a rich standard library; and, on top of this, there are many awesome and liberally-licensed packages just an easy_install away.

IronPython, of course, includes *most* of the CPython standard library, but if you're a heavy user you might have noticed a few minor holes: in the course of my work on Ironclad, I certainly have. Happily for you I can vaguely remember what I did in the course of bodging them closed with cow manure and chewing gum; here then, for your edification and delectation, is my personal recipe for a delicious reduced-hassle IronPython install, with access to the best and brightest offered by CPython, on win32.

Tuesday, December 29, 2009

A Good Mix 33: Embedding Python and Ruby, Profiling IronPython, News on JScript, ctypes and DeviantArt

More IronPython and DLR related projects, articles and news from around the web.
A  nice example of embedding both IronPython and IronRuby in a single C# project. As an added bonus the project is a Silverlight project so you can add both Python and Ruby scripting to applications that run in the browser.
IronPython 2.6 has useful new hooks for profiling and debugging IronPython code. Unfortunately most 'standard' .NET tools don't know how to use these, and if you attempt to profile IronPython code (particularly in an embedded environment) you have to work hard to get useful information about performance of your Python code. It's nice to see a new (and open source) tool that is designed to work with IronPython:
SlimTune is a free profiler and performance analysis/tuning tool for .NET based applications, including C#, VB.NET, IronPython, and more. It provides many powerful features, such as remote profiling, real time results, multiple plugin-based visualizations, and much more. The source code is available under the terms of the MIT License.

SlimTune is currently in the prototyping phase of development, but a preview release is available for testing and feedback is welcome. Both x86 and x64 targets are supported, but only sampling based profiling is available in the release.
When IronPython 2 and the Dynamic Language Runtime were announced one of the three Microsoft developed languages that ran on the DLR was JScript. Managed JScript was an implementation of ECMAScript (otherwise known as Javascript) and touted as a useful bridge for porting 'traditional-ajax' applications to run on Silverlight. Unfortunately as the DLR evolved JScript languished and there was no official word on its fate.

In this post on the DLR Codeplex forum Bill Chiles (DLR Program Manager) gives the bad news:
The DLR JScript was experimental for informing the design of the DLR (expression trees, interop, callsites, hosting, etc.).  The JS we released with asp futures and the Silverlight dynamic sdk became very old and unserviceable as the DLR continued evolving for release in CLR 4.0.  Unfortunately, there are no plans at this time to develop and release a DLR-hostable JScript.
A Japanese blog entry showing example code using the new implementation of ctypes in IronPython 2.6. ctypes is the Python FFI and in IronPython it is built on top of PInvoke, the .NET FFI. ctypes is used for calling into native code, like the Win32 API. Calls SHGetFileInfoW, which "Retrieves information about an object in the file system, such as a file, folder, directory, or drive root."
No idea what this has to do with the implementation of Python for .NET, but IronPython is the username of someone on the alternative community art site deviantART. Some pretty nice computer generated art, and who knows - maybe they were created from Python...

Scripting Applications with IronPython (ADAM, Revit, AutoCAD & Postsharp)

IronPython and the Dynamic Language Runtime make it almost ridiculously easy to add scripting to .NET applications.

In recent weeks several examples of using IronPython to add scripting or interactive shells to .NET applications have been posted by the .NET community:
This post was heavily inspired by the code presented by my old friend Albert Szilvasy during his excellent AU class on using .NET 4.0 with AutoCAD.
In this post we’ll take Albert’s technique and implement a command-line interface for querying and executing IronPython script. This approach could also be adapted to work with other DLR languages such as IronRuby, of course.

Here’s the updated C# code which now not only implements PYLOAD functionality, but also a PYEXEC command.
 Making an application scriptable (particularly in a static language) has historically been difficult. With the advent of the DLR (Dynamic Language Runtime) on the .NET platform it becomes almost trivial to add scripting support to any application. For a recent project I needed the ability to add scripting hooks throughout the application and coupling the DLR with PostSharp AOP attributes made this effort pretty straightforward. Here’s how it was done.
According to wikipedia: "Autodesk Revit is Building Information Modeling software for Microsoft Windows, currently developed by Autodesk, which allows the user to design with parametric modeling and drafting elements.  Building Information Modeling is a Computer Aided Design (CAD) paradigm that allows for intelligent, 3D and parametric object-based design."

Jeremy Tavik blogs about working with the Revit API, and in this post he discusses a Python Shell for Revit that created by Daren Thomas with IronPython:
Daren Thomas of the Professur für Gebäudetechnik, Institut für Hochbautechnik at the technical university ETH Zürich has published a Python Shell for Revit. It was implemented using IronPython and is used to automate the running of daily tests of a building energy analysis package. Hosting IronPython via a Revit plug-in is a now solved problem. Daren's intention is to continue publishing samples of what you can do with it pretty regularly in the future.

The source code is available under the GNU General Public License v3 from a Subversion repository on It is documented, though there is currently little other stand-alone documentation. One would probably want to look for the IExternalApplication entry point and go from there. What the application does is:

Add a button to the Revit ribbon to start the python shell.
Execute a python script entered in a text box, with output going to a separate window.
Daren describes one of the main scripts like this: it "will open up an interactive interpreter loop known as a REPL that will let you explore the Revit API. And by explore, I mean type a statement, hit enter, see the results, carry on. It doesn't get easier as this!"
IronPython is an implementation of the Python language for the .NET framework, using the new Dynamic Language Runtime (DLR) as language services layer on top of the Common Language Runtime (CLR). The DLR's reusable hosting model makes it extremely suitable to add scripting capabilities to existing products.
In this post we'll create an ADAM command line utility that will allow us to run IronPython scripts against an ADAM application.
Our command line tool will expose a single command, RunScript, that will take in ADAM connection information and a path to a Python file to execute:

Monday, December 28, 2009

PythonSilverScripting: Silverlight apps in the browser with Python

Tarn Barford is an Australian blogger and programmer who uses IronPython. He's created a Google App Engine site for experimenting with IronPython in the browser (through Silverlight).
PythonSilverScripting lowers the barriers to building Silverlight applications so anyone from high school kids to seasoned programmers can have fun writing Silverlight applications in Python from within a browser!

Below is a simple script that creates a Silverlight application, sets the background blue and adds a TextBlock element with the text "Hello World".
He's also blogged about the new site, including his future plans for it:
PythonSilverScripting is based on crazy idea I have that it should be possible to make Silverlight applications in Python on the web. No tools, no SDKs.. just a browser (and obviously the Silverlight browser plug-in).
I've played around a bit with Silverlight and Dynamic Languages here and there and thought it was probably possible. When I got a chance I looked into it and found I was able to build working XAPs by zipping up plain text IronPython code, the IronPython DLR assemblies, an xml application manifest file and all the additional resources such XAML, images and more source.
All I needed was a web server that could zip files. Google App Engine with Python is good fun and I was able to get this working prototype together in an evening. I'd like to build the features to support building bigger projects (more source files, XAML documents, images and assemblies). On the other hand I'd also like to build features to share, discuss and rate scripts. We'll see how it goes.

Why IronPython Podcast and Best of MSDN Ebook

I recently wrote an article for the UK MSDN (Microsoft Developer Network) newsletter called Why IronPython? This article made it into the collection of the thirteen best technical articles of 2009:

You can download these articles as a free ebook in XPS or PDF format, or read it online:
The UK MSDN Flash developer newsletter contains great short technical articles written by UK developers both inside Microsoft and in the broader developer community. This eBook pulls together these great articles in one place. There are thirteen articles in this second edition covering Python, Inversion of Control, Behavior Driven Development, Silverlight and more.
The MSDN Flash newsletter is run by Eric Nelson. He also has a podcast and we recorded an episode together about Python, IronPython, PyCon and various other topics:
A great chat with Michael Foord, author of IronPython in Action, on why a C# or Visual Basic .NET developer should look at also investing time in learning and using IronPython. Michael wrote an article on IronPython for the November 18th 2009 UK MSDN Flash newsletter:

“One of the new features in .NET 4.0 is the dynamic keyword, building on the Dynamic Language Runtime. A major reason for dynamic is to enable easier interaction with dynamic languages like IronPython. But if you're a dyed in the wool C# or VB.NET programmer why should you be interested in IronPython? 

IronPython is a .NET implementation of the popular open source programming language Python. Python is an expressive language that is easy to learn and supports several different programming styles; interactive, scripting, procedural, functional, object oriented and metaprogramming. But what can you do with IronPython that isn't already easy with your existing tools? …

Tuesday, December 22, 2009

Executing IronPython Code from IronRuby

The Dynamic Language Runtime is a framework for writing dynamic languages that run on the .NET framework, with the two "Microsoft sponsored" languages being IronPython and IronRuby. There is also IronScheme, a community project hosted on Codeplex.

The promise of the DLR is not just that it makes implementing dynamic languages possible, but also that through the DLR .NET languages can interoperate. This includes IronPython and IronRuby (etc) interacting with C#, F# and VB.NET (the supported and statically typed Microsoft .NET languages) but also the reverse (statically typed languages interoperating with dynamically typed languages) and dynamically typed languages interoperating amongst themselves. All very incestuous.

As far as I know this is still unique amongst the modern polyglot runtimes (.NET and Mono, the JVM, LLVM, Parrot and so on).

Whilst IronRuby in particular has been changing very rapidly (IronRuby has only recently reached 1.0 RC 1) it has been hard to get versions of both languages built on the same version of the DLR. However recent builds have tended to include compatible versions of the other language, making it a bit easier.

IronShay (Shay Friedman) has written a blog entry demonstrating how IronPython and IronRuby can share code and use each others libraries:
 One of the advantages of the Dynamic Language Runtime (DLR) is the fact that it makes sharing code between the languages that are written on top of it (and on top of the CLR as well). Therefore, it is possible to share code between IronPython and IronRuby (and any other DLR language as well like IronScheme).

This means that IronPython libraries can be used from IronRuby code and vice versa. Ruby on Rails in Python? Django in Ruby? feels like the end of days, isn’t it? perhaps we should really start preparing to year 2012

In this post I’ll show you how to run simple IronPython code from IronRuby so you can take it and do whatever your imagination guides you to.

Monday, December 21, 2009

Why IronPython?

This is a short article I wrote for the UK MSDN Flash newsletter (a Microsoft newsletter for developers). Unfortunately the online versions of these newsletters aren't being updated at the moment; so instead of linking to it I'm reproducing it here.

Why IronPython?

One of the new features in .NET 4.0 is the dynamic keyword, building on the Dynamic Language Runtime. A major reason for dynamic is to enable easier interaction with dynamic languages like IronPython. But if you're a dyed in the wool C# or VB.NET programmer why should you be interested in IronPython?

Much of the discussion here applies to other dynamic languages, including IronRuby, but Python is my particular area of expertise.

IronPython is a .NET implementation of the popular open source programming language Python. Python is an expressive language that is easy to learn and supports several different programming styles; interactive, scripting, procedural, functional, object oriented and metaprogramming. But what can you do with IronPython that isn't already easy with your existing tools?

The first entry in the list of programming styles is 'interactive'. The IronPython distribution includes ipy.exe, the executable for running scripts or programs that also doubles as an interactive interpreter. When you run ipy.exe you can enter Python code that is evaluated immediately and the result returned. It is a powerful tool for exploring assemblies and learning how to use new frameworks and classes by working with live objects.

The second reason to use IronPython is also the second programming style in the list; scripting. Python makes an excellent tool for generating XML from templates, automating build tasks and a host of other everyday operations. Because scripts can be executed without compilation experimentation is simple and fast. Python often creeps into businesses as a scripting language, but beware it spreads!

One of the big use cases for IronPython is for embedding in applications. Potential uses include user scripting, adding a live console for debugging, creating domain specific languages where rules can be added or modified at runtime or even building hybrid applications using several languages. Python has several features, like the ability to customise attribute access, that make it particularly suited to the creation of lightweight DSLs. IronPython has been designed with these uses in mind and has a straightforward hosting API.

There are many areas where dynamic languages are fundamentally different from statically typed languages, a topic that rouses strong opinions. Here are a few features of IronPython that make it easy to develop with:
  • No type declarations
  • First class and higher order functions
  • No need for generics, flexible container types instead
  • Protocols and duck-typing instead of compiler enforced interfaces
  • First class types and namespaces that can be modified at runtime
  • Easier to test than statically typed languages
  • Easy introspection (reflection without the pain)
  • Problems like covariance, contravariance and casting just disappear
The best way to learn how to get the best from IronPython is my book IronPython in Action. I've also written a series of articles aimed at .NET developers to help get you started including Introduction to IronPython, Python for .NET Programmers and Tools and IDEs for IronPython.

Happy experimenting!

Michael Foord

Tuesday, December 15, 2009

IronPython gets a long overdue website!

Thanks to Jimmy Schementi (the new IronPython PM, Ruby lover and all round good guy) IronPython has a new website. And as said best on Reddit, it doesn't look like Microsoft made it. The site has links to documentation, community resources and online IronPython examples using Silverlight (including Try Python).
CodePlex will continue to be the tool we use for project management and releases, but all end-user information will be on Also, the .com domain still points at the super-old site, but will redirect to this site shortly.

The most notably addition is the .NET Integration Documentation, a thorough set of examples and descriptions of using IronPython with .NET. Considering this is IronPython’s main purpose, it’s amazing we got away with having hardly no documentation for this long … I guess the .NET integration is just that intuitive :) Anyway, please give it a read and let us know if you have any suggestions.

WCF Service in pure IronPython with config file

In previous blog entries Lukáš Čenovský looked into Databinding and WCF Services with IronPython 2.6. This uses the new __clrtype__ feature in IronPython 2.6 to interact with .NET features that previously couldn't be done with IronPython.

In this follow up blog entry Lukáš shows how to saved the IronPython interface in an assembly with a config file.
I was wrong when I wrote in the last post that the IronPython service cannot be saved into an assembly. It can. Which opens a way to use .config file to configure the service.
The interface is the same as in the previous version. The only difference in the service to the previous version is in the ServiceHost initialization - we omit the service configuration parameters because they are in the .config file. I also changed the clr namespace.

Sunday, December 13, 2009

NWSGI 2 RC 2 and IronPython Extensions for Visual Studio 2010

Jeff Hardy is a big cheese in the IronPython world. One of his older projects is NWSGI, an implementation of the Python WSGI (Web Server Gateway Interface) specification for .NET. This allows Python WSGI applications to run on IronPython and be served by IIS.

One of his newer projects is an extension of Visual Studio 2010 (still in beta) to provide Python syntax highlighting and IronPython support.

Jeff has recently announced progress in both projects.
The final release candidate of NWSGI 2.0 is now available. Except for version numbers, this is what will become NWSGI 2.0 as soon as IronPython 2.6 is released.

NWSGI 2.0 Release Candidate 2 includes all of the features of NWSGI 2.0 Release Candidate 1, and adds support configuring tracing, lightweight scopes, and the profiler. It is built against IronPython 2.6 RC3.

The following issues are fixed in this release:
 Lately, I’ve been experimenting with the new editor APIs in Visual Studio 2010 to build some simple extensions to make Python editing a little better. Helpfully, IronPython exposes most of the complicated infrastructure (the tokenizer) to make the job easy. The documentation for the new editor APIs is still a bit weak; hopefully that will improve by the final release. In the meantime, the examples are the best source of information. There also a series of blog posts by an Mike Feingold about the NDjango editor for VS2010 that form a great end-to-end example.

Download & Installation
Download IronPython Extensions for Visual Studio 2010 (and the source code; Ms-PL as always) from my BitBucket repository or from the Visual Studio Gallery (direct link). For direct downloads, just run (double-click) the .vsix file to install it into Visual Studio. Otherwise, you can use the extension manager (search for "IronPython") and install it directly from Visual Studio.

The extensions provide three features so far: syntax highlighting, brace matching, and outlining.

In the future I'm planning on adding project support, templates, and (most difficult of all) IntelliSense.

IronPython 2.6 Final Released

IronPython 2.6 is finally out! IronPython 2.6 is a new version of IronPython targeting compatibility with Python 2.6. As well as the new language and library features that come with Python 2.6, IronPython 2.6 has new features for improved performance, general Python compatibility and better .NET integration. It's a great release; congratulations and thanks to the IronPython team.
 IronPython 2.6 final is identical to release candidate 3. Features new to IronPython in version 2.6 include:
  • The __clrtype__ metaclass for data binding and .NET attribute support
  • Implementation of the ctypes module
  • Support for Python stack frames and sys.settrace, which means the pdb debugger works (and better debugging from the .NET side)
  • Better performance through adaptive compilation
  • Faster startup
There are several changes to the samples in IronPython 2.6. These include two new samples:
Not only will this sample show you how to utilize Word's spell checking functionality from IronPython; it'll also make your own Python code better! You see, this sample processes Python files looking for code comments that contain misspellings. Once these mistakes have been identified Word is utilized again to present you with the correct spellings. All of this is tied together with an elegant, user-friendly interface constructed from .NET's Windows Presentation Framework.
The ClrType sample shows how to define a class in IronPython with required CLR members. Normally, a class definition in Python does not map directly to a unique CLR type. This is because the semantics of classes are different between Python and the CLR. For example, in Python it is possible to change the base types just by assigning to the bases attribute on the type object. However, the same is not possible with CLR classes. Hence, IronPython implements Python classes without mapping them directly to CLR types. Using the ClrType sample you’ll be able to define CLR fields, properties, class-level attributes, and declare strongly-typed methods.
Pyc, the IronPython compiler, is no longer available as a sample as "it was incorporated directly into IronPython releases as a tool".

The full IronPython 2.6 release notes:
We’re proud to announce the release of IronPython 2.6 final. This is a major release with improvements across all areas of IronPython. Significant changes include: 
  • Updating the language and standard library to match CPython 2.6
  • Improved .NET integration
  • Updating to the latest version of the DLR
  • Adding previously missing CPython features and fixing bugs
  • Performance improvements in particular startup time improvements
Python 2.6 support includes a large number of new features which include support for the new bytes and byte array types (PEP 3112), decorators for classes (PEP 3129), advanced string formatting (PEP 3101) which will feel familiar to .NET users and integrates nicely with IFormattable, print as a function (PEP 3105), Abstract Base Classes (PEP 3119), support for binary literals, along with lots of other minor features and changes.

IronPython also continues to improve .NET integration in this release. We’ve added the __clrtype__ method to types which enables deep integration via meta-classes with the .NET type system. There are also a number of smaller changes such as supporting IDisposable in for loops matching the behavior of other .NET languages. This release also includes the latest version of the DLR and fixes a number of issues related to cross-language dynamic interop. Not to be left out there’s improvements in Silverlight integration by supporting Python code in HTML script tags.

We’ve also continued to improve Python compatibility by adding missing features and fixing bugs. In this release we’ve added support for the ctypes module which provides interop with native code in a compatible way across Python implementations. We’ve also implemented sys.settrace to provide support for the pdb module and other Python debuggers. This release also changes how we support sys.getframe: a fully working version is now available by a command line option; when not enabled sys.getframe doesn’t exist at all. This release also fixes over 400 bugs removing a large number of smaller incompatibilities.

As always we’ve also continued to improve performance, and this release primarily focuses on improving startup time. The IronPython installer now pre-compiles (ngens) IronPython during installation on both 32-bit and 64-bit platforms. Modules are now interpreted initially and hot functions are compiled for faster import times. A number of other code paths that used to involve runtime code generation have been optimized to be contained within the pre-compiled IronPython binaries. We’ve also made a number of smaller changes which improve performance in other areas such as adding constant folding.

Tuesday, December 01, 2009

Gestalt 1.0 and the Gestalt Widget Pack

Gestalt is a project by Mix Online to use Silverlight and the Dynamic Language Runtime to allow you to script web pages with Python or Ruby instead of with Javascript.
DLR.js is a library released by a collaborative effort between the Dynamic Language Runtime team and MIX Online Labs. This JavaScript library allows you to write Ruby, Python & XAML code in your (X)HTML pages. It enables you to build richer and more powerful web applications by marrying the benefits of expressive languages, modern compilers, AJAX & RIAs with the write » save » refresh development model of the web.
 Gestalt has reached a significant milestone with the release of version 1.0 and a widget pack. It has matured to the point where you can do really crazy things like run Rails *in your browser*. Probably not specifically useful, but it shows what it is capable of:
A few months ago, we released Gestalt beta as a MIX Online lab.  Gestalt began as an exploration—a way to bring Ruby and Python to the web browser.  Today, we’re delighted to announce that Gestalt has been updated to version 1.0. It's now part of IronRuby 1.0 and IronPython.

What’s new in Gestalt 1.0? Gestalt Beta -- plus more!
  • Supports bundling of Ruby and Python files using each language’s convention.  You can now (for example) package and run Rails on your client!
  • Allows you to dynamically load assemblies (only downloads what you need)
  • Integrated with the Dynamic Language Runtime
    Simplified and streamlined model for creating Gestalt apps.
  • No jQuery dependency.
  • Support for remote references to Gestalt
  • Visual Studio debugging support
  • A browser console for debugging and manipulating Gestalt
  • Rich error reporting
  • Built-in support for loading external assemblies and content
  • Support for running multiple instances of Gestalt on the same page
They also have a longer blog entry discussing the thinking behind Gestalt and what it makes possible. Perhaps worth drawing out from this article is the following: "By adding a <script>
tag to the top of your HTML file, you can automatically enable use of HTML5-compatible video and audio tag syntax.
Last week, we discussed the fact that Microsoft invests heavily in both HTML5 and Silverlight, two technologies that other companies would have you believe are mortally opposed.  Our commitment to both was underscored this week by our announcements about IE9 and Silverlight 4.

When we launched Gestalt beta less than 4 months ago, our goal was to demonstrate a really simple idea: that a proprietary plugin like Silverlight complements and advances the standards-based web.  With today’s launch of Gestalt 1.0 and the first few widgets in the Gestalt Widgets Pack, I’d like to drill deeper into this underlying philosophy of Gestalt.

Tuesday, November 24, 2009

Cheminformatics Tutorial Using Python and Silverlight

Try Python is an online interactive Python tutorial using IronPython and Silverlight. The source code is available and open source and the tutorial material is written in a subset of the ReStructured Text markup.

Noel Baoilleach has taken this a step further by creating his own version, including the Webel library library and a tutorial on using it with interactive examples.
Recently I introduced Webel, a Python cheminformatics module that runs entirely on web services. One of the advantages of such a module is that it can be used in places where it is difficult to install a traditional cheminformatics toolkit. Like in your browser.

After some little work, I present Try Python...with Cheminformatics. This adds Webel as well as a short tutorial that introduces many of its features. With a few more tutorials that cover SMILES, InChI and so on in more detail, this could be useful for teaching purposes as well as bridging the gap to having students develop their own Python scripts that use the CDK, OpenBabel or the RDKit.
 Personally I think the Cheminformatics version of Try Python would be improved by the removal of the Python tutorial parts (only part 6 is Cheminformatics specific - making it a bit harder to find!). Creating custom tutorials using Try Python as a base is *fairly* easy (I'm sure it could be easier): it could be a great way of creating online examples of working with .NET or Python libraries.... I hope more people try this out.

Sunday, November 22, 2009

Hello GTK# from IronPython and F#

GTK# is a managed interface to the GTK user interface library. GTK# is part of the Mono project, but it can also be used from .NET on Windows.

In this blog entry, another one from Steve Gilham, he illustrates using GTK# from both IronPython and C# by porting some C# examples.
A little post to record a short bit of spiking with GTK# as UI toolkit, porting the simple C# examples from here to my preferred .net languages. Neither of these explorations are totally novel -- there are examples in either language to Google, but not all of them recorded all the details of the slight rough spots that needed a little working (and they were often not at all recent, either).

For IronPython 2.6 latest with GTK# 2.12.9-2, running the programs as ipy

The dynamic nature of the language means we can lose a lot of the declaration clutter. We just have to explicitly reference the main GTK# assembly (which is GAC'd by the GTK# installer), and away we go.

A Good Mix 32: Alpha Encoding Files, Embedding IronPython in Russian, IronSharePoint, IronRuby, World's Worst Paint Program

More IronPython and DLR related projects, articles and news from around the web.
 Steve Gilham shows how to generated encoded version strings for installers using IronPython:
When building installers the UpgradeVersion must have a unique property value that is an installer public property (upper-case alpha). So, what better way of adding uniqueness than making it have the form "product name + product version" with the version suitably encoded...

So, a script for turning a file version (4 x 16bit ints) encoded as a System.Version into a short alpha string, assuming that Major and Minor will be small, and that common approaches are to step Build, to use a stepped Build plus date-stamped Revision, or a timestamp Build and Revision.
An article on IronPython, and in particular using IronPython from C#, in Russian.
 A while ago Christian Glessner announced the start of the IronSharePoint project: showing how to develop SharePoint applications with IronPython. Well, he hasn't released anything but there is now code available for download from the Codeplex site. Could be worth exploring if you work with SharePoint...
 Since early 2007 John Lam has been working for Microsoft as program manager of IronRuby, the implementation of Ruby on the Dynamic Language Runtime. Up until almost eleven months ago that is. John quietly transferred to a 'secret' project within Microsoft, a project that has also claimed Curt Hagenlocher (IronCurt). As they are *close* to being able to announce the secret project John has made his departure from the IronRuby team official and handed over the reigns to Jimmy Schementi who was previously only in charge of the integration between dynamic languages and Silverlight. Now Jimmy is doing both.
 Dino Veihland presented on IronPython and the Dynamic Language Runtime at the recent PDC conference. Part of his talk featured the "World's Worst Paint Program", which from Dino's description includes both IronPython and IronRuby code. He has made this available for download from the IronPython Codeplex Site.
 This is built using Visual Studio 2010 Beta 2 and the IronPython CTP for that release - so you'll need to download both of those.  I've also included the Python and Ruby code that I used during the demonstration in a file called code.txt.  The paint program it's self is in the final form at the end of the demo.

Saturday, November 21, 2009

Face Detection with IronPython

Face detection in images is very cool (and perhaps a little bit scary), and with the help of Emgu CV it can be done from IronPython.
Emgu CV is a cross platform .Net wrapper to the Intel OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc. The wrapper can be compiled in Mono and run on Linux / Mac OS X.
This blog entry from Clark Updike shows how.
 A couple notes compared to the Emgu wiki version--I was able to get everything working without having to disturb the IronPython install (I didn't copy any files into the IronPython start directory). I simply linked used sys.path to add the locations where the dll's live. Also, I did explicit import for everything. It's a bit more work to figure out what needs to be imported, but it avoids namespace pollution issues. Also, the img.Draw line needed a few tweaks. I had to drop the [float] bit from the original. And with the original version, I was also getting an error on this same line, saying:

TypeError: expected MCvBox2D, got MCvAvgComp

I was able to work around this by passing in the ".rect" rectangle attribute to the Draw method (I'm guessing the api has evolved since the 1.4 Emgu version used for the original). Lastly, I was getting to objects detected--so I used parameters to the detector to only get the single face in the image (basically by telling it to use a larger minimum detection size, minSearchScale). Anyway, now it works the way I was hoping.

Databinding and WCF Services with IronPython 2.6

One of the important new features in IronPython 2.6 is the __clrtype__ metaclass.The __clrtype__ metaclass allows you to create a real .NET class that backs your Python classes. This is important because there are many .NET features that *require* a real .NET class: which includes databinding and implementing WCF services (Windows Communication Foundation).

The problem with __clrtype__ is that it requires dealing with low level details; namely building the class yourself from IL bytecode. Harry Pierson and Shri Borde have been working on a library ( to make this simpler.

Lukáš Čenovský has looked at this before but hit limitations with what clrtype made possible. In three new blog entries he demonstrates how to use __clrtype__ with databinding in WPF and Silverlight and to implement WCF services.
 INotifyPropertyChanged is important interface for building WPF or Silverlight applications using M-V-VM concept (MSDN article).

In simple language, you have a Model which provides access to your data (e.g in database, files, web, etc.). Then you have a ViewModel that access data in the Model via Model's interface and provides data to a View which is XAML file with UI layout. Linkage between ViewModel and View is done by binding that utilizes PropertyChanged event to properly update all UI elements.

I have found two examples how to implement INotifyPropertyChanged interface in IronPython. The first one uses __setattr__ hook. Personally, I don't like it - it is not clear and easily readable code. The second one is better because it uses properties. But you have to write self.OnPropertyChanged("my_property_name") for every property. Not ideal.

That's why I sit down and write a notify_property.
This article uses the updated clrtype module to apply the databinding technique to Silverlight. The cool thing is that I'm already using this technique for databinding with the Silverlight Toolkit DataGrid in my new job. No more stub C# proxy classes, yay!
 Even using __clrtype__ you still couldn't use WCF (for creating and consuming web services) from pure IronPython because it requires *creating* interfaces, which couldn't be done. With the latest improvements to clrtype it is now possible.
I wrote about implementing WCF service in IronPython a couple of weeks ago. Meanwhile I pushed Shri a little bit with the and he has implemented ClrInterface metaclass there so we can create the whole WCF service in IronPython now.

The IronPython interface implementation is straightforward.
Also switching from C# interface to IronPython interface is easy.

Friday, November 20, 2009

IronPython 2.6 Release Candidate 3

IronPython 2.6 is the up-and-coming version of IronPython targeting compatibility with Python 2.6. As well as the new features in Python 2.6, IronPython 2.6 has several important new features specific to IronPython. These include:
  • The __clrtype__ metaclass for data binding and .NET attribute support
  • Implementation of the ctypes module
  • Support for Python stack frames and sys.settrace, which means the pdb debugger works
  • Better performance through adaptive compilation
  • Faster startup
IronPython 2.6 Release Candidate 3 has just been released. The hope is that this will be the last release candidate before the final release:
We’re pleased to announce the third and hopefully final release candidate of IronPython 2.6. Release Candidate 3 only includes Silverlight-related changes pertaining to some incompatibilities between 2.6 RC1 and RC2. Those who utilize IronPython for non-Silverlight scenarios will happily find virtually no churn from RC2. We strongly encourage everyone interested in Silverlight to test out this release ASAP because we plan on releasing IronPython 2.6 final in a week if no major new regressions are detected.

    Two Articles: IronPython 2.0 and WPF Error

    Two more articles from Ibrahim Kivanc, the Turkish blogger who has written several articles on IronPython and Silverlight. Both of these articles are in English.
    IronPython 2.0 version now runs on DLR (Dynamic Language Runtime). DLR is a platform on .NET which is host Dynamicly typed languages on it. Now Dynamic Languages Communicate eachother and C#,VB, COM Objects, .NET Libraries.

    IronPython, with 2.0 version runs on DLR (Dynamic Language Runtime); it’s a platform like CLR architecture. It’s host for Dynamic Languages on .NET. With this architecture Dynamic Languages now faster then running on CLR and easily communicate with other .NET objects!
    In my opinion IronPython Studio is not stable enough for production use. It does have the advantage of being integrated in Visual Studio so some people can't resist trying it out. (You can read my write-up of IronPython Studio at: IronPython Tools and IDEs.)

    If you insist on trying out IronPython Studio there are various minor trials and tribulations for you to overcome; using the WPF designer is one of them. In this article Ibrahim explains how to jump over this particular hurdle and get the WPF designer working in IronPython Studio.

    Sunday, November 15, 2009

    Embedding IronPython in Silverlight - Importing

    Jimmy Schementi is the Microsoft program manager for the integration of dynamic languages (IronPython and IronRuby) and the Silverlight browser plugin. As well as dynamic languages, Silverlight applications can be written in .NET languages like C# and VB.NET - and these languages can embed IronPython.

    Although embedding IronPython in a C# Silverlight application is initially straightforward (although a bit more verbose than embedding using the normal .NET framework / Mono as the ScriptRuntime needs to be configured), it gets painful fast. This is especially true when the Python code you execute needs to import anything, which has basically been broken on Silverlight for quite some time.

    In his latest blog entry Jimmy looks at how embedding from Silverlight has got simpler recently, and also at fixing the import problems.
    I’ve heard plenty of times on the IronPython Mailing List that embedding IronPython in Silverlight is easy at first, but then you fall off a cliff when trying to import, be it from a .NET namespace, the Python built-ins, or even other Python files. Let me clear some things up, and fix some code in the process.

    Slight detour – Embedding in Silverlight 101
    First off, Michael Foord has an article on his website about Embedding IronPython in Silverlight, which still works. However, that’s a ton of complicated code, all having to do with creating a ScriptRuntime instance correctly configured for Silverlight. To make this much less verbose, DynamicEngine.CreateRuntime (in the Microsoft.Scripting.Silverlight namespace) will create a ScriptRuntime all prepped for use in Silverlight; this significantly reduces the boilerplate hosting code:

    A Good Mix 31: Texas Holdem, IronScheme, Indigo Cheminformatics, IronRuby and bridge xml

    More IronPython and DLR related projects, articles and news from around the web.
     Greg Bray has been experimenting with using Resolver One to write Texas Holdem calculating spreadsheets with IronPython. In this blog entry he shows the IronPython code for calculating hand equity:
    This year I have been working in my free time to create easy to use Texas Holdem poker spreadsheets based on IronPython using Resolver One. These spreadsheets can be used to calculate Win/Tie/Loss odds, but some people like to use hand equity instead since it represents a player’s overall stake in the pot. Equity of 1.0 or 100% means that they will win the entire pot, where as equity of 0.5 (50%) or 0.25 (25%) means that they will split the pot with other players. You can again use Monte Carlo analysis to run a number of trials before all of the board cards are dealt to estimate a player’s current equity in the hand. This means that if during 4 trials I would win 2 of them and split the pot with one other player on the other 2, then my equity would be (1.0+1.0+0.5+0.5)/4 =  3/4 or 75%.

    Calculating Win/Tie/Loss statistics on an independent player vs player basis is very simple, as the player’s pocket cards are evaluated against each opponent one at a time to show which opponent hands are stronger or weaker than your own. This does not take into account how the pocket cards of other opponents might change the odds, since in reality you usually play verse a table of opponents not just one opponent at a time. Changing the spreadsheet to work against the whole table would require changing the Monte Carlo simulation from a “row by row” analysis to a “table by table” analysis, where by you setup a table of opponents using a random hand for each player given the information that you have about their hand (AK, KK or QQ, Group1, etc..) and then generate a random 5 card hand based on the currently dealt cards.
    IronScheme is an implementation of R6RS Scheme for .NET and Mono implemented on the Dynamic Language Runtime. You can either use it for writing .NET applications or for embedding into other applications to provide a Scheme scripting environment:
    IronScheme provides 2 run modes: either via direct invocation on the commandline or via a REPL interactive environment.
    IronScheme can also used an embedded scripting library in your .NET programs.
    With 1.0 RC 1, IronScheme implements the entire R6RS Scheme standard, and only fail 4 of 8884 tests in the PLT R6RS test suite.
    IronScheme runs on MS .NET and Mono. Unfortunately, it does not seem to work under 64bit on Mono.
    Since the release of 1.0 Release Candidate 1 leppie has also written a blog entry demonstrating the use of the new dynamic keyword in C# 4.0 with IronScheme: IronScheme and C# 4’s new dynamic keyword
    Noel Baoilleach demonstrates using the open source Indigo cheminformatics toolkit from IronPython.
    Charlie Zhu has already written an example application using C#. Rather than wait for CPython bindings, I installed IronPython and used it to access Indigo's .NET libraries (Dingo, in this case) to do a SMILES to png conversion.
    A new release of IronRuby, IronPython's red-headed cousin that is still on it's path to 1.0. Release Notes:
    The IronRuby team is pleased to announce version 0.9.2! As IronRuby approaches 1.0, these 0.9.x releases contain important bug fixes and enhancements that IronRuby users request; this release fixes/closes 44 bugs. For more information see the "Bugs fixed" section, and the CHANGELOG.

    This release is the first to give IronRuby a MSI-based installer for Windows. Try it out and let us know if it does not install on your system.
    Bridge is a "General purpose XML library for CPython and IronPython". It provides a common interface (Document, Element, Attribute, Comment and PI) for working with XML whether you are using CPython or IronPython.

    Saturday, November 14, 2009

    IronPython at PyCon 2010

    PyCon 2010, the annual international Python conference is coming soon (February 17th 2010 in Atlanta US). The schedule of talks is now up, and as usual there are several IronPython related talks.
    • 67. IronPython Tooling
     By Dino Veihland, core IronPython developer:
    One of the most popular requests for the IronPython team is tooling support. During this talk I’ll show you some of the existing tools available to help create IronPython applications. I’ll also look at the latest IronPython features which can help you debug, profile and improve your applications. I’ll also compare and contrast these with the solutions available for CPython that you may already be familiar with.
    • 71. Python in the Browser
    By Jimmy Schementi (Microsoft program manager for IronPython / IronRuby Silverlight integration):
    You write your server code in Python because you want to. You write your browser code in JavaScript because you have to. But with IronPython and Silverlight, you can write your browser code in Python and leave JavaScript behind. This talk provides an overview of Silverlight for Python developers, including out of browser applications as well as the new “just text” model where Python code is embedded directly in HTML pages. Even if you develop on Linux or Mac, you can still take advantage of Silverlight in the browser.
    As well as these talks the conference starts with a Virtual Machine (VM) summit, with attendees from the major implementations of Python plus those implementing other popular dynamic languages and Virtual Machines for dynamic languages.

    Dave Fugate (IronPython tester and infrastructure guy) is also planning a Testing IronPython poster session.

    After the summits are the tutorials and I will be repeating the IronPython tutorial I gave last year. If you're interested in getting some hands on experience of developing with IronPython then come along.

    For those interested in alternative implementations of Python you will also appreciate these talks:
    • 148. Pynie: Python 3 on Parrot
    By Allison Randal of the Parrot Foundation:
    Pynie is an implementation of Python 3 on Parrot. The goal of Pynie is to duplicate the pure-Python behavior of CPython 3.x, and perhaps eventually a C API compatibility layer. Parrot provides a set of compiler tools and core functionality common to many dynamic languages, so the core code of Pynie is lightweight. Pynie allows Python 3 libraries to be shared between multiple languages. This talk is an overview of the features Pynie currently supports and the work left to be done, together with an introduction to the internals of Pynie.
    • 13. How and why Python is being used to by the Military to model real-world battlefield scenarios
    By Mr. Eric Silverman (ALATEC Inc. / US Army):
    Leveraging the power of Python, military analyst and software developers out at White Sands Missile Range in New Mexico have converted a closed-form model, which was developed for scripted Cold War scenarios, into a more sophisticated and adaptive system. COMBATXXI is a premium wargaming model, which now is poised to answer questions that have a direct effect on military operations currently in theater, in large part because of Python.
    • 65. Extending Java Applications with Jython
    By Frank Wierzbicki (core Jython developer):
    Jython is an implementation of Python for the JVM. This talk covers the options available for integrating Jython and Java code, especially Java calling into Jython, since this is the trickier case. This talk will also cover some Java specific deployment options, such as deploying your Jython application as a jar file. Jar files allow you to package Java/Jython applications into a single archive. In many environments a user can click on a jar file to launch the contained application.
    • 83. The speed of PyPy
    By Mr. Maciej Fijalkowski (PyPy core developer):
    The first part of the talk will cover PyPy's speed achievements resulting from the last year's work on the Just-In-Time Compiler. I'll present and discuss a number of benchmarks and compare against other Python-speed projects. I'll also cover the basics of how the JIT works and what sort of programs it can greatly speedup (and which ones it can't).
    • 101. How to write cross-interpreter Python programs
    By Mr. Maciej Fijalkowski (PyPy core developer):
    This talk will cover basics about writing cross-interpreter
    python programs. What to do and most of all what not to do. This will help you if you want at some point in time to
    run program on for example Java platform or faster python
    interpreter, but also if you want to keep it running between CPython releases.
    • 189. The Ring of Python
    By Holger Krekel (PyPy and py.test developer):
    CPython 2.5/2.6/3.1, Jython, IronPython, PyPy, StacklessPython, UnladenSwallow, Cython ... what do we make of all these cool interpreter projects and versions? Where does competition help and where does it hamper?

    This talk is for stirring discussion and thoughts on making the most out of wealth/curse of interpreters. I am also going to showcase and discuss 'execnet' one of my own related projects.

    A Good Mix 30: Visual Studio 2010, DevDays, Detecting 64 bit, Silverlight and Django

    More IronPython and DLR related projects, articles and news from around the web.
     Visual Studio 2010 doesn't come with IronPython support out of the box, but it does have extensive APIs for writing your own extensions. Jeff Hardy has taken up the challenge and has written a set of IronPython extensions for Visual Studio 2010 with syntax highlighting, regions etc. The project has binaries available: "Just double click the .vsix file to install."
    At the end of October I attended the StackOverflow DevDays in Cambridge UK and spoke on Python and IronPython. I demonstrated .NET integration with IronPython by creating a simple Windows Forms application at the interactive interpreter. This was followed by going through Peter Norvig's Python Spell Checker as an example of concise Python code. Thanks to Ned Batchelder for the slides of his Boston presentation, which served as inspiration.

    The links above are three reports from the day, the first two positive. My favourite quote being "It was a huge surprise that the Knuth-packing beardo in the front row turned out to be a Windows developer, demoing on IronPython". But like Jeff Atwood said in his closing presentation; if someone doesn't hate what you do then you're doing it wrong. The third reporter didn't enjoy my presentation "I wasn't too keen on this presentation. There were statements like "Dynamic Programming makes it easy to test" and "Dynamic Programming means I type less" that I didn't agree with and there was no follow up to back it up". (See the comments on that entry for my reply.)
    A Japanese blog entry showing IronPython code (using WMI) for detecting whether your code is running on a 32bit operating system or a 64bit one.
    Yet another release from the mysterious Japanese IronPython and Windows Forms project that we've covered before.

     'xap' files are the way you serve Silverlight applications. django-xapper is  a project to build and serve xap files from Django, making it easier to integrate Python on the server with Python on the client for projects using IronPython and Silverlight.
    Xapper is a Django app to help facilitate the development of other Django apps that include Silverlight apps. In particular it is geared towards Silverlight apps implemented in IronPython or other dynamic CLR languages.

    Xapper serves a similar function as Chiron, which is now distributed with the IronPython Silverlight binaries. Both these apps can dynamically generate Silverlight packages (.xap files) from a directory of files and serve it over HTTP upon request. This means that all you need to do to pick up the changes you've made to your Silverlight app is reload the page in your browser, making development of Silverlight apps much nicer. The main difference between Chiron and Xapper is that Xapper is oriented towards integrating with Django, while Chiron is a standalone web server oriented only towards development of the client-side app.

    Thursday, November 12, 2009

    Using Solver Foundation and plug-in solvers in IronPython

    Microsoft Solver Foundation is a set of mathematical programming tools that can be used from .NET languages like C#, F# and IronPython:
    Solver Foundation Services (SFS) can automatically analyze models and determine which solver is most appropriate. If you are an advanced modeler, you can choose specific solvers and solver attributes. While solving the models, SFS manages all threading, many-core, synchronization, scheduling, and model execution issues. When finished, SFS produces reports about solver behavior and results, and provides additional information about solutions, including sensitivity. Finally, SFS allows LINQ data binding of model parameters and delivery of results in multiple formats.
    Lengning Liu has a blog entry on using the solver foundation from IronPython, including the magic incantations in app.config required to use plug in solvers.
    Solver Foundation provides an easy-to-use and flexible plug-in infrastructure as an addition to its already rich set of solvers. Users can choose their favorite plugin solvers to solve the model, and yet leverage all features provided in Solver Foundation Services (SFS).

    The information such as assembly name and solver class name can usually be found from the plug-in solver provider.

    The config file then will co-exist with the compiled exe file of the user app. When, inside the app code, a Solve call is issued, SFS will pick a solver based on the model kind, directive instances passed in, and the solver registration.

    The app code in this case can be written in any .NET compliant language such as C# or VB.NET.

    IronPython is “an implementation of the Python programming language running under .NET …” So it is possible to use Solver Foundation, and with plug-in solvers, in IronPython code. However, one difficulty is that Python is an interpreted language. Therefore we cannot compile Python code to an exe file. So how do we register our plug-in solvers?

    The trick is to register the plug-in solvers inside ipy.exe.config (or ipy64.exe.config). These two files must be put under the same folder with ipy.exe and ipy64.exe. We assume IronPython 2.6 RC2 in the following discussion.

    Scripting Your .Net Applications with IronPython

    One of the major use cases for IronPython is embedding in .NET applications to provide user scripting. The hosting APIs make it easy to experiment with embedding IronPython.

    Chris Umbel has written up a blog entry demonstrating the IronPython 2.6 hosting API and giving an example of exposing an API to Python scripts from a C# applications.

    At several points in my .Net development career I've had the need to make an application I wrote scriptable. Sometimes it was to provide easy product extension to customers or lower level information workers. Sometimes it was to ease maintenance of very fine grained logic that has the capacity to change frequently or unpredictably. But every time I found it to be one of the more interesting facets of the project at hand.
    Early in .Net's history this was made easy by using Visual Studio for Applications (VSA) which allowed you to host arbitrary C# or VB.Net code within the executing AppDomain. Unfortunately VSA was plagued with resource leak problems and was therefore impractical in most enterprise situations. VSA was eventually deprecated.

    One of the many alternatives is to perform dynamic, on-the-fly code compilation. While certainly quite manageable it was a bit more complex and much akin to cutting down a sapling with a chainsaw.

    Another option that came along later is Visual Studio Tools for Applications which brought the Visual Studio IDE to the scripter.

    My favorite avenue, however, is to host a Dynamic Language Runtime (DLR) and use a language like IronPython. Not only is it disgustingly simple to implement from a plumbing point of view but Python itself seems like a natural fit due to it's simplicity. IronRuby's another wonderful choice but I'll stick to IronPython for the scope of this post.

    Tuesday, November 10, 2009

    A WCF Service from IronPython

    Windows Communication Foundation (WCF) is part of .NET 3 and makes creating and interacting with web services from C# substantially easier. WCF makes extensive use of .NET attributes, making it difficult to use from IronPython versions before 2.6 without wrapping the use of attributes in some C#.

    IronPython 2.6 brings in support for attributes through the __clrtype__ metaclass. It doesn't yet allow you to create a .NET interface from pure Python code (although you can implement existing interfaces just by inheriting from them), so *some* C# is still needed for full use of WCF with IronPython 2.6. The dynamic languages team are looking at ways that __clrtype__ could be extended to allow the creation of interfaces in a future release.

    In his latest blog post Lukáš Čenovský demonstrates how to create a WCF service from C# and then shows the same code from IronPython, along with the minimal C# still needed.
    Until IronPython 2.6, it was not possible to create WCF service host in pure IronPython. The closest way was to create stub in C# and subclass it in IronPython or create the whole service in C# and run it from IronPython. It is now much simpler with IronPython 2.6 although you still have to write a little C# code.

    Getting Started with IronPython and Embedding IronPython

    Charlie Calvert is Community Program Manager for the C# group at Microsoft. He has posted a couple of blog entries on using IronPython. The first shows you how to get started with IronPython on Windows, including setting up your path to easily execute Python scripts with ipy.exe. The second entry shows how to execute a Python file from C# when embedding the IronPython engine.
    I recently spent some time getting IronPython up and running on my system; I will review what I learned in this post.

    Hosted inside an interpreter, Python belongs to the same family of scripting tools as VBScript, JavaScript, Perl and Ruby. You can fairly compare Python to a general purpose language such as C#. Developers praise this loosely typed, dynamic language for its ease of use and rapid development capabilities. IronPython is Microsoft’s free version of the open source Python language.

    IronPython can be hosted inside a C# program as a scripting language. Nonetheless, Python is a powerful standalone language frequently used as the glue in web based applications. Before I came to work for Microsoft I enjoyed using it as part of my daily toolkit.

    In this post I explore the simple steps needed to run IronPython as a standalone tool from the command prompt. In other posts I will show how to call Python from inside a C# application.
    IronPython is a scripting language hosted on the .NET platform. This posts shows how you can use the Dynamic Language Runtime (DLR) and the new C# 4.0 dynamic keyword to call an IronPython script from a C# program.

    Before going any further, it might be helpful to take a moment to explore the architecture that makes this technology possible. The key building block is the DLR, which sits on top of the .NET 4.0 CLR and provides the tooling for adding dynamic programs such as Python to the family of .NET languages. C# 4.0 adds the dynamic keyword and some related technologies to integrate support for dynamic binding into the language. The end result is that developers can arbitrary Python scripts directly from C# or other .NET language

    Monday, November 09, 2009

    Performance of IronPython vs. C#

    The Loose XAML blog has posted an interesting article profiling some IronPython code and the semantically identical C# and C code (for generating the fibonnaci series). He comes to the unsurprising conclusion that C# is substantially faster, but digs deeper into the results to try and work out why. An interesting discussion ensues in the comments.
     I’ve been using IronPython for a while, and every now and then I do a cursory check to see how it’s performs against the equivalent C# code. C# is generally a touch faster at most logic, but IronPython is faster at dynamic invocation of methods than reflection in C#. In general it’s a wash…over the course of an application you dabble in things that different languages are better optimized to handle. When performance is a wash, you get the freedom to choose entirely based on the features of each language, which is nice.

    This was going well, until I had a need to do some recursion. I found that the recursive IronPython code was executing extremely slowly – code that was taking milliseconds in C# was taking minutes in IronPython. That’s not a wash anymore…it’s a few orders of magnitude. So to really figure out what’s going on with these recursive calls, I made two implementations of calculating Fibonacci numbers – one in C# and one in IronPython.
    Of course performance is one of the tradeoffs you make when choosing between a dynamically typed language and a statically typed one. If your application is not computationally bound, but is network, database or I/O bound then the performance of your programming language *may* be more-or-less irrelevant. On the other hand, if a 'slower' language is more flexible and expressive, or you can develop faster with it, then you may be able to spend more time optimising your algorithms and still have acceptable performance.

    This has been my practical experience at Resolver Systems where we have always been open to the idea of moving core parts of the application into C# but have basically always been able to gain our targetted performance improvements by refactoring and optimising without having to rewrite in another language. With the extra code rewriting in C# would require (and studies show that bugs are pretty much proportional to lines of code irrespective of the language you use), and how much extra work C# is to test than Python,  keeping as much code as possible in Python is something that the development team appreciates.

    Talking to ActiveDirectory from (Iron)Python

    Active directory is one of the standard (and important) parts of a Windows network. Working with active directory is therefore a common task for a Windows system administrator. Naturally you can work with active directory from Python, but there is also good support for active directory in the .NET framework that can be accessed from IronPython (including from IronPython on Mono). This is something that Brendan McAdams has been exploring on the Evil Monkey Labs blog and has created a utility module to make it even easier:
     We're building a new intranet system at work, and I've been toying with a few things that the Windows admin asked for. Namely, since the secretaries here will update the intranet data to add people's Work & Emergency contact numbers, AIM handles, email addresses, etc. that we find a way to keep it all in sync with ActiveDirectory. Thereby keeping all the Outlooks and Blackberries up to date with the latest contact information.

    This seemed like a fairly reasonable request, presuming we could figure out how to do it and since I've been using Mono and IronPython a lot more lately, I figured there would be a way to accomplish it. Most of the information I found online was either really old and/or crappy docs for doing it in C#, or more commonly using PowerShell or VBScript. So, I managed to poke around and sort out how to get IronPython on Mono (IronPython 2.6RC + Mono to find and update our users.
    The end result is that I can now, from IronPython, find and update valid information on ActiveDirectory entries to reflect the latest and greatest information. One thing to note, the MS .Net ActiveDirectory APIs (System.DirectoryServices, which is mirrored in Mono) do something that confused and annoyed me. There are a limited set of 'valid' attribute keys for a user object in Active Directory (Which is really just LDAP, in case you didn't know). The DirectoryEntry object has a Properties attribute, which contains a hashmap of these values.

    The object will not allow you to set an "Invalid" key (see this list for valid keys). But if you call .Properties.Keys you only get back the Properties that have values set. So, it doesn't appear to be possible to actually ask What keys are valid and do some introspective programming. I have written a wrapper class to make the DirectoryEntry properties look a bit more pythonic (but disabled support for multi-value attributes for now) - at some point in the near future i'll likely add in a "valid value" filter.

    The end result is, if I want to find my own user in ActiveDirectory by my name, I can do the following from the IronPython console...