Saturday, January 31, 2009

IronPython at PyCon 2009

The talk schedule has gone up for for PyCon 2009 in Chicago. There are three talks on IronPython, plus a tutorial on the Wednesday before the conference.

PyCon is the international Python conference, and the highlight of the Python year.

The conference dates are:
Jim Hugunin is one of the invited speakers, so his talk doesn't yet appear on the schedule - but his usual talk is "IronPython the Road Ahead".

The other two IronPython talks are:
  • IronPython Implementation, by Dino Veihland (core IronPython developer)
IronPython is an implementation of Python running on .NET. This talk will provide an overview of the IronPython internals. The talk will start with a high-level walk through of the IronPython architecture and source layout. From there we’ll drill into details such as method dispatch, how the .NET and Python type systems interrelate, multi-runtime support, and how IronPython uses the DLR. Finally we’ll bring it all together and show you how you can do Python aware interop between .NET and IronPython. Whether you’re just curious about language implementations, planning on hosting IronPython in your own app, or just want to write a new built-in module for IronPython you’ll find something interesting in this talk. Knowledge of C# would be helpful but is not required.
As Python grows in popularity, IronPython has started making more waves. What is IronPython and why should regular Python programmers be familiar with it? This session will introduce the open source .NET implementation of Python known as IronPython without using Windows. Come see FePy (IronPython community edition) in action via Mono -- a cross-platform open source implementation of the .NET framework.
In the last year Sarah has spoken regularly on IronPython, and she has blogged about PyCon: PyCon Talks Posted!

Jonathan Hartley and I are giving a half day tutorial on Application Development with IronPython.

If you're a Python developer, this tutorial will get you up to scratch on developing with IronPython on .NET or Mono (Windows, Mac OS X and Linux will all be supported). We'll cover getting started, .NET interoperation and use some of the common .NET APIs like Windows Forms, databases, XML, threading, network operations and so on. You will be coding and it should be great fun - no previous IronPython experience needed!

My blog entry has all the details, plus a description of my talk on testing desktop applications:
The tutorials, whichever you choose, are really an awesome opportunity to get training from some well known members of the Python community.

Using F# Code from IronPython

IronPython isn't the only new language going mainstream on the .NET framework. In fact the dynamic languages team (responsible for IronRuby, IronPython and the Dynamic Language Runtime) are part of a larger team called the GO team - Growth Opportunities.

One of these languages is F#, a functional language created by Don Syme and team and based on Ocaml. Because it runs on the .NET framework (or Mono) F# has the great advantage over many interesting functional languages in being of immediate practical use! It also lends itself to polyglot programming - algorithms and problems particularly suited to being solved in a functional language can easily be mixed with code written in C# or IronPython. F# is already beloved by statisticians who where previously working in C#. Like Python it has a powerful interactive shell.

Luis Diego Fallas has a blog entry on using F# code from IronPython:
F# has its own built-in types like lists and tuples. Although Python has similarly named types they have different implementations. Luis shows adding references to F# assemblies and using F# types, functions and operators.
We can call F# functions to process values generated by fibo. For example the following IronPython code prints the first 15 squared Fibonacci numbers.

for n in Seq.map[int,int]( FuncConvert.ToFastFunc[int,int](lambda x: x*x) , Seq.take[int](15,fibo())):

print n

Thursday, January 29, 2009

A Simple IronPython ControllerFactory for ASP.NET MVC

The .NET answer to the wonderful world of web frameworks like Django and Rails is ASP.NET MVC (which reached RC1 in the last couple of days).

There are various ways that dynamic languages can be used with ASP.NET MVC, and the integration is improving. This entry on Code Voyeur demonstrates how to configure controllers with IronPython, using a PyControllerFactory:
The default behavior for ASP.NET MVC is to wire up a controller instance to the controller value found in a request's RouteData. This mapping is desirable for its simplicity, but limited in its ability to provide an extensible configuration model. Managing properties on controllers would likely require either a custom configuration section or extensive use of AppSettings.

An alternative approach would be to use an IoC container, such as Spring.NET. This approach simply manages controller instances as standard Spring container objects. An MVC contrib project already provides support for a Spring Controller factory (among others). This article will present an alternative approach in which controller instances are configured in an IronPython script that lives alongside the MVC application.

Ironclad 0.8 Released: Python C Extensions from IronPython

Ironclad is a project by Resolver Systems to allow the use of Python C extensions from IronPython. Ironclad provides a compatibility layer for the Python C API, allowing extensions to be imported from IronPython. The goal is to allow the use of Numpy within Resolver One spreadsheets, but Ironclad is open source and available under the MIT license.

The core developer of Ironclad, William Reade, has just announced the release of Ironclad 0.8. This is an important release as many extension libraries are now usable with Ironclad. In fact now you can run through the whole Numpy tutorial using the IronPython interactive interpreter! As usual, sources and windows binaries are available from the project page:
I'm delighted to announce the release of Ironclad v0.8 -- the all-singing, all-dancing CPython API compatibility layer for IronPython -- available now from the Ironclad Project Page. Notable improvements over the last release include:
  • Ironclad is now a neatly self-contained package -- just copy to your site-packages and 'import ironclad'.
  • No more silly requirement to call ironclad.shutdown() when you're finished.
  • A few performance improvements.
  • Over 900 NumPy tests now pass: in fact, almost all the tests from the core, fft, lib, linalg, ma, oldnumeric and random subpackages.
  • Over half the .pyds distributed with CPython 2.5 now import cleanly; some of them appear to actually work, including _hashlib and _elementtree.
Ironclad grows more stable and mature with every release, and I urge IronPython users to try it out and share their impressions: feedback, whether positive or negative, is always welcomed.
If you're interested in the implementation details, William Reade has posted an explanation of how Ironclad manages object lifecycles over on my techie blog:

Expression evaluator in 15 minutes with Irony & Dlr

One use for IronPython in .NET applications is as an embedded engine for expression evaluation:
Evaluation of a string expression to a value is a common programming task. Almost any college course in informatics includes this excercise. It involves implementing some fundamental algorithms and structures such as: recursive-descent parser, regexp matching, traversal algorithms, syntax tree, hash-tables, etc. And for sure it should be done on some pure programming language such as ANSI C or Pascal.

.NET Framework brought a lot of in-box algorithms and structures which simplifies the usual live of software developer. However it is still a good excercise to evaluate a string expression, say 2+(4*5)^12.

But fortunately there are some new tools that facilitate producing language parsers and their interpreters/compilers.
Piter has used the DLR, along with a C# parser library called Irony (which he has also used to create Script.NET), to create a string expression evaluator:
And it took aproximately 15 minutes! Wow!

Irony already had examples of parsers for arithmetic expressions and I’ve used them. The next step is to evaluate parsed syntax tree. For these purpose one may use very convenient Expression class which comes as a part of DLR under Microsoft.Linq.Expressions namespace.

So, please take a look at the results. Bin+Src.

This console application will accept a single line valid string expression on natural numbers. It supports following operations: +,-,*,/,** (Pow).

Saturday, January 24, 2009

IronPython: Old Wine in a New Bottle

MC Press Online has a new article on IronPython by Joydip Kanjilal:
The article is an introduction to using IronPython and interacting with the .NET framework:
Learn how to write simple programs in IronPython and access .NET libraries with IronPython code.

The advent of the IronPython programming language in the managed environment of the CLR provides a lot of benefits, namely these:
  • Support for dynamic typing
  • Support for procedural, functional, object-oriented programming
  • Support for rapid development
  • Support for embedded scripting
  • Support for seamless extendibility
  • Support for accessing .NET managed libraries
  • Support for accessing .NET objects from IronPython script using reflection
The article covers installing IronPython, the interactive interpreter, using .NET assemblies, plus examples with WPF and XML. There are a couple of minor issues with the article: indentation in the code examples is missing and after showing installing IronPython 2.0 it seems to use the IronPython 1 hosting API. Despite these issues it is still a great introduction to IronPython.

Craig Murphy Reviews IronPython In Action

Craig Murphy is a .NET developer who helps organise a lot of community events in the UK and Scotland. He has read an early access version of IronPython in Action, and has posted a detailed review:
During 2008, dynamic languages picked up a lot of momentum. Indeed, in IronPython In Action, Foord and Muirhead describe this momentum somewhat amusingly: “dynamic languages are trendy; all the alpha-geeks are using them!” With what is the largest known IronPython codebase forming a large part of their careers, the authors are well-placed to provide us with accurate and authorative information coupled with practical code examples.
Fortunately Craig liked the book (and so he should!) and says some nice things:
You should buy this book if you wish to understand where the Dynamic Language Runtime fits into your .NET toolkit. If you are looking for better ways to test your code, dynamic languages and duck typing lend themselves to mock objects: you should buy this book to discover the gems of chapter 7. If you are searching for the Holy Grail that is an extensible application architecture, you need this book for chapter 15 alone.
Chapter 15 is one of my favorite chapters (it was fun to write). This chapter is on embedding IronPython in C# and VB.NET applications:
The ability to run Python scripts from within your CLR-based .NET application is frankly astounding, the authors present us with everything we need in order to achieve this in just a few lines of code. Similarly, the ability to treat Python code as part of your application’s “plug in” architecture makes extending and customising your applications without altering the primary codebase so much easier. Given that such functionality, extensible application architectures, are becoming commonplace, this chapter serves us well. It’s examples are to-the-point and are supported by sensible quantities of explanatory narrative.

Friday, January 23, 2009

Dynamic Language Testing in Silverlight

Jimmy Schementi maintains the Silverlight and DLR (agDLR) integration, which is now hosted on github and open to community contributions. Unfortunately being a browser plugin, Silverlight is hard to test - or at least was:
getting the infrastructure to run tests has taken some time, but I just committed a Silverlight spec runner for bacon (a little Ruby spec framework) and a bunch of specs for Microsoft.Scripting.Silverlight!

So, the tests are written in Ruby, but they test the C# code in Microsoft.Scripting.Silverlight ... pretty cool. This is a deviation from my common belief of "test in the language you write the code in", but so what ... I never cared for that way of thinking =)
This test system is reusable by Silverlight apps. Jimmy has a video of it in action on his blog entry and explains the problems with the old system.
To get tests out to the world, the only sane thing would be to head back to the drawing board, which is what I've done with test/runner. Hopefully by using existing open source projects, a testing paradigm most people understand, and a very small test runner, this testing solution for AgDLR will be kept small and friendly for developers.

So, now that there's a sane way to verify changes to AgDLR don't break, please fork AgDLR and send me pull requests! Next post will probably be about adding continuous integration to AgDLR, so stay tuned.

Thursday, January 22, 2009

RIA Applications for Python and Ruby Developers - with Silverlight 2

A while ago Cal Schrotenboer contacted me about an article on dynamic languages and Silverlight. He used chapter 13 of IronPython in Action for research, and its nice to finally see the article published:
If you're a Python or Ruby fan, you may feel you're missing out on all the latest developments in Rich Internet Applications. With support for Python and Ruby in Microsoft® Silverlight™ 2, dynamic language developers now have easy access to the latest RIA technologies.

Moreover, in my conversations with dynamic language developers, the rapid development cycle and the ability to see changes immediately without the need to recompile are the most commonly stated reasons for their preference for dynamic languages. This article will focus specifically on dynamic language support in the recently released Silverlight 2. This topic should be of potential interest to three categories of developers:

  1. Python or Ruby developers who would like to be able to use Python or Ruby in the browser.
  2. Classic C# and Visual Basic.NET developers who, based on my informal and non-scientific survey, most likely are not aware of the advantages that they might reap from using one of the dynamic languages in the right circumstances to supplement their normal development.
  3. Developers of any stripe who want to offer scripting capabilities to end users via a web application.

Currently, dynamic language support by Microsoft extends to three languages: Python, Ruby and JavaScript. Of these three Microsoft supported dynamic languages, the most advanced at this point is IronPython. Version 2.0, RC1 was released in October 2008. By contrast, IronRuby is still in Alpha for Release 1.0. (I am advised that for open source languages—which are continuously updated and not officially supported by Microsoft—there is not quite the same degree of significance associated with the terms Alpha and Beta.) The third language is Managed JScript, a Microsoft version of JavaScript.

Another Selection

Another selection of blog entries and articles about IronPython and the Dynamic Language Runtime that have appeared over the last month or so.
A blogger called mithrandir explains the problems he has with the source code for the DLR. It basically boils down to two things:
  • There seems to be a lot of hard coded references to the main DLR languages IronPython & IronRuby, potentially making it hard to implement other languages.
  • There are a lot of compiler directives (#ifdefs) for the Silverlight version
Bill Chiles (DLR PM) responds to this (particularly the first point) in the comments, so it really boils down to mithrandrir not liking compiler directives...
What looks like a great article on hosting IronPython 2.0 from C#, explaining the core classes and replete with diagrams and code examples. Unfortunately (for me) the article is in Chinese - although as usual the code examples are easy to follow. For those who only speak English (shame on us) there are other sources of useful information.
In December I linked to the November update of the Cobra programming language, so it is only fitting that in January I link to the December update. Cobra is a Python inspired .NET language with first class support for unit tests and contracts. The December update includes bug fixes along with improvements to the language, libraries and documentation.
A short InfoQ article on the final release of the Microsoft Robotics Studio 2008.
RDS includes a .NET-based RESTful runtime consisting of two components: Concurrency and Coordination Runtime (CCR) and Decentralized Software Services (DSS). With RDS, robotics applications can be developed in Microsoft Visual Studio and Microsoft Visual Studio Express using C# or VB.NET, as well as scripting languages such as Microsoft Iron Python.
One interesting feature of the robotics studio is that it works just as well with simulated robots as it does with real robots, including simulated environments. This looks like an interesting presentation: Controlling Simulated Robots Using Iron Python.
Another DevHawk blog entry (Harry Pierson), this one on creating PowerShell aliases for searching sets of folders for a given executable. PowerShell is a Microsoft shell scripting environment / language that allows you to pipe objects between commands. This entry deserves a mention on IronPython-URLs purely because Harry includes aliases for working with IronPython and Chiron (the Silverlight development tool for dynamic languages).
A short blog entry from a .NET developer extolling the virtues of the Dynamic Language Runtime. The announcement that the DLR will be part of Common Language Runtime in .NET 4.0, along with the changes to C# 4.0 and Visual Basic 10 (particularly the dynamic keyword and IDynamicObject interface), has made a lot of waves in the .NET community. Jim Hugunin believes this is part of blurring the distinction between statically and dynamically typed languages. This blogger (Kevin Hazzard) also welcomes it:
On the other end of the spectrum is IronPython 2.0, Microsoft's first DLR-based implementation of the Python programming language. I just love Python. It's so clean and simple that almost anyone can learn and use it effectively after studying it for an hour or two. If I could send a note to myself in the past to change one thing about my career (besides the winning lottery numbers for the past decade) it would be to encourage my younger self to learn Python back in the mid-1990s. [...]

Learning Python made me fall in love with Ruby and LISP and F#, too. I'm a real polyglot now and being able to understand how these other languages solve problems helps me design better in the language I use every day: C#. By the way, the term polyglot from its Greek roots literally means many tongues. I'm excited about the DLR because it's enabling Python and Ruby on my favorite platform. But the DLR is so much more important than that. It's the Language of Languages. The DLR defines the primitives that must exist on the boundaries of our languages to make them interoperate. In the same way that COM unified the dispatch model in the 1990s through IUnknown and IDispatch, the DLR will define the call semantics and dispatch models for the .NET Framework for the 2010s through IDynamicObject and DynamicMetaObject.

Pretty soon, we'll see a flurry of languages appear on top of the DLR. A bunch of them are already in development. But we'll also see a wave of DLR object binders appear that have no significant language support on the platform, too. The effects will be profound. Imagine a Java RMI bridge written as a DLR object binder. Using the dynamic type in C# 4.0, you could make calls to remote Java objects from .NET as if they were just part of the framework. Or imagine a type binder that exposes RESTful services as a dynamic object model. There's really no end to the possibilities here. The key to all of these scenarios is the enablement of dynamic services and that sounds a lot like what Web 2.0 was supposed to be.

Tuesday, January 20, 2009

IronPython Reflector Addin

Reflector is an extremely well known .NET tool (free but not open source and owned by Redgate) for exploring .NET assemblies. It can show you all the types in assemblies, their members and the relationships between them. In addition to this, it can disassemble the IL bytecode back into .NET languages like C# and VB.NET - allowing you to examine the implementation.

Reflector has a plugin mechanism and lots of interesting plugins. A new one has appeared on Codeplex called IP reflector, which promises to be able to decompile the bytecode generated by IronPython back into Python code (IronPython code that has been compiled by the Pyc.py compiler):
There is a release available on codeplex - version 1.0.0 Beta. No word on whether it is based on IronPython 1 or 2.

Silvershell 0.6.1 Released

There is a new release of Silvershell, version 0.6.1. As well as fixing several bugs this release includes the sourcecode!

Silvershell is a Python interactive interpreter with intellisense and lots of other features. It can run on the Windows desktop with WPF or in the browser with Silverlight:
Dan Eloff announced the release on the IronPython mailing list:
There's an important new bugfix release. I put a lot of irritating flaws in the "intellisense" to rest, including a major issue with member completion that crept into 0.6.0 at the last minute. Had I known there was going to be such a response I think I would have tested it more thoroughly.

As promised all source code is in this release. Please notify me of remaining bugs, either via email, or via the issue tracker on Google Code. Patches are most welcome.

Monday, January 19, 2009

Debugging IronPython in Visual Studio

Ryan Brady has posted a great blog entry with step-by-step instructions on how to debug IronPython code with the Visual Studio debugger (unfortunately it requires Visual Studio Professional and so doesn't work with Visual Studio Express - although it is possible that you could get round it by building IronPython from source as part of your solution):
The steps he lists are (although if you go directly to his entry it has more detailed instructions along with pretty pictures):
  • Create a Python file to debug.
  • From Visual Studio, open the IronPython executable (ipy.exe). If it doesn’t do so for you, add the ipy.exe as an existing project.
  • From the Solution Explorer, right-click on Properties and set the Command Arguments property. Use “-D” to tell ipy.exe to create debugging output and add the full path to the script you made before.
  • Click Ok.
  • Open the script you created earlier in Visual Studio and place breakpoints as desired.
  • Run the script in debug mode (F5).
He also uses the "-i" option to "ipy.exe" to open an interactive session when you break into your Python script.

Jeff Hardy: Django, Zlib and easy_install on IronPython

Jeff Hardy is the creator of NWSGI, a .NET implementation of WSGI implemented as an ASP.NET HttpHandler for use with IIS 6 & 7. His goal is to be able to use Python web applications and frameworks like Django and Trac from IronPython.

He has been working on getting Django and easy_install working with IronPython. For easy_install one of the big hurdles is that the zlib module isn't available (even the version in FePy is missing a key part of the API that setuptools uses to unpack archives it downloads from PyPI). Getting setuptools working with IronPython will be a big win for using Python libraries and frameworks as more and more of them depend on being installed in this way.

He has written three blog entries on how far he has got with these issues.
I've had a couple of people ask about a Django + IronPython "Getting Started". This has been on my TODO list for a while; I just haven't gotten around to it. I'll try to do something over the next couple of weeks.

A couple of tips to anyone trying in the meantime:
  • Get the SQLite provider from FePy SVN; it's the only one that I know works
  • If manage.py has problems, file (or vote on) bugs, and use Python for manage.py instead
  • Set the session backend to use the caching system, and set your cache backed to be in-memory (you can try to use the ASP.NET backends I posted earlier, but no guarantees)
  • It probably won't work 100%, but I did work my way through the tutorial – except the admin system
Jeff has also been working on getting the zlib module working with IronPython, a prerequisite for using easy_install:
IronPython 2.0 does not include an implementation of the zlib module, which handles data compression. In CPython the zlib module is a C module, which it can't be used from IronPython. It is also required by several standard Python modules, include tarfile, gzip, and zipfile, all of which handle compressed archives. Thus, for setuptools - particularly, easy_install and zipimport (Eggs) - to work, there must be a fully working zlib module.

FePy includes a partial implementation built on .NET's DeflateStream, but it is missing support for the decompressobj API that is used by gzip and zipfile. Supporting this API using DeflateStream appears to be impossible, as it expects a few implementation details of the zlib implementation.
To get round this problem, Jeff has reimplemented zlib based on ComponentAce's zlib.net:
IronPython.Zlib is an implementation of the zlib module for IronPython using zlib.net. Both binary and source are available under the MS-PL license. To use it create a "DLLs" directory under your IronPython implementation and drop IronPython.Zlib.dll into it.

It passes most of the Python zlib tests and all of the gzip tests. There are a few issues with the CRC-32 implementation that the tests didn't catch but real-world usage did, but I haven't figured out how to fix them yet.

There are still some issues with setuptools (the zipfile module uses binascii.crc32, which is not yet implemented) that I'll detail in a later post, but it almost works.
Having got zlib working, Jeff moved onto easy_install:
Getting easy_install working for IronPython will be a big win for the IronPython ecosystem, and as I mentioned last time, the lack of zlib is really the only thing holding it back. Well, IronPython.Zlib solves that problem, so what else is holding up easy_install?

The big thing that's missing for setuptools is zipimport support, which is used to implement the Egg packages.
Jeff then details other minor changes needed to get basic easy_install working with IronPython:
Once you get this far, easy_install can download, extract, and build a Python library (I used simplejson for my tests). Installation is still a problem because pkg_resources is looking for an Egg, which require zipimport, which was blanked out in the first step (it's almost poetic, really). Getting zipimport working should be the last major hurdle.

Saturday, January 17, 2009

Davy's IronPython Editor

One thing that isn't clear when working with IronPython (as if it was clear for Python) is which IDE to use. The Visual Studio 2008 integration is done through IronPython Studio. This has impressive elements - but also plenty of rough edges and in my opinion isn't stable enough for regular use. IronPython Studio is built on IronPython 1 and seems to be little more than abandonware from the company that produced it.

It is not necessary to use IronPython Studio to debug IronPython code in Visual Studio, however if you want features you're used to in a Python IDE (autocomplete, go to definition, call tips, integrated interactive interpreter and so on) then you probably want to look elsewhere. I use the capable Wing IDE, but a custom solution for IronPython would be an interesting project.

This is where Davy Mitchell steps in with a new project called Davy's IronPython editor.

The screenshot below shows the editor pane (implemented with the Windows Forms SyntaxBox component) and the class browser.


You can download Davy's IronPython Editor from its Google code project page:
The project intentions is to create a editor along the lines of SPE or DrPython for the IronPython world. DIE is easy to deploy (zero install) and will run from a USB key etc.

This project is targeted at IronPython 2.0 and is developed on the Windows Platform. Currently it uses the SyntaxBox component (LGPL) for the text editing component.

IronPython and data binding plus the PyBinding markup extension

XAML is a markup language used to describe user interfaces for the Windows Presentation Foundation (WPF) UI library. PyBinding is a library created by Pixel-in-Gene allowing you to use IronPython for custom expressions in data binding situations: "essentially a DSL for creating Bindings". A second blog entry discusses a detail of the WPF framework that makes this less useful than it could be in some situations.
Many of us have felt the need to have custom expressions inside of Bindings. The need has ranged from applying mathematical operators to calling methods on bound objects. For simple needs you can probably create a custom IValueConverter or a IMultiValueConverter that can do the job, but when you want to use arbitrary expressions, something more elaborate is needed. The ScriptConverter class that I blogged about earlier provides the underpinning for the PyBinding MarkupExtension. The MarkupExtension provides a simplified way of using the ScriptConverter by automatically setting up the Bindings used in the Python statements. Thus the PyBinding extension provides you the following:
  • Allows use of IronPython for scripting arbitrary expressions
  • Each expression will be converted to a Binding or a MultiBinding depending on the number of subexpressions
  • A syntax for specifying the sub-expression. Essentially a DSL for creating Bindings
Rob Oakes has also been thinking about IronPython and data binding. He's written a blog entry with great coverage of the subject. It doesn't show much code, but there is promise of a follow up entry.
In Windows Forms programming, much of the data binding occurs within the Visual Basic or C# code. WPF, however, is substantially different in that much of the syntax required for data binding appears alongside the XAML that defines the user interface. While it is possible to provide the syntax necessary for data binding from either XAML or code, it is easier to do some things from one location rather than the other. As my language of choice when working with WPF is Python, it should be mentioned that it is sometimes necessary to do things from code rather than from XAML. Defining how data should be presented in the XAML while managing the data source from the Python appears to work well.

Friday, January 16, 2009

IronPython and Python 2.6

Now that Python 2.6 is out of the door the IronPython team are working on IronPython support for 2.6 features. Useful new features in Python 2.6 (or at least the ones I particularly like) include:
  • with statement available without future import
  • New syntax for exception handling
  • Class decorators
  • hasattr not discarding all exceptions
  • Being able to provide keyword arguments after *args
  • Getter, setter and deleter on properties (damn useful)
  • __dir__ support
In fact the last one is already available - it is analogous to the method GetMemberNames that you can implement on .NET objects to provide dynamic Python behavior for classes implemented in C#. Because it was basically already there, the IronPython team built it into IronPython 2 anyway.

Now that IronPython 2 is also out of the door, the team have been working on the 2.1 branch and implementing Python 2.6 features. Of course having IronPython 2.1 tracking Python 2.6 is a recipe for confusion, so Dave Fugate asked whether the new branch should be renamed IronPython 2.6:
I think the name for the next major release of IronPython might still be up in the air so to speak, but our developers have indeed started working on CPython 2.6 features. If you build the latest source drop from CodePlex, you should now be able to pass the '-X:Python26' flag to ipy.exe.

Speaking of the next major release, do people prefer "IronPython 2.6" implying that we're targeting CPython 2.6 features or "IronPython 2.1" following the pattern we established with IP 1.1? My gut feeling is that IP 2.6 would cause less confusion...
Personally I agree that "IronPython 2.6" would be less confusing, and it is also the pattern followed by Jython for its version numbers.

Yesterday Dave Fugate posted this email to the IronPython mailing list:
Some of you may have noticed that IronPython bugs on CodePlex previously assigned to the “2.1” release are now assigned to “2.6”. We’ve simply renamed the 2.1 release of IronPython to 2.6 to reflect upon the fact that the next major release of IronPython will be largely compatible with CPython 2.6. All details about IP 2.1 that were previously discussed on Harry Pierson’s blog still apply to IP 2.6. We’re doing this simple name change to help eliminate confusion about IronPython’s compatibility with various CPython and Jython releases.
Great news.

Dave also has another recent blog entry responding to another blog claiming that Sun hiring Jython developers and Microsoft developing IronPython is an attack on Python and free software:

IronPython - Nightly Builds

As well as the 'signed-and-supported' releases of IronPython, source code drops of the latest sources are available from the public facing repository. The same is also true of the Dynamic Language Runtime project.

The scripts that pushed from the Microsoft internal repositories to the public ones were broken over Christmas, but on Wednesday new drops arrived with some nice performance improvements.

The source code drops include the 2.0 branch (the stable branch that will shortly be released as 2.0.1) and head (IronPython_Main) that includes deeper changes to the DLR, with even more dramatic performance gains in some areas. Until recently this was referred to as the 2.1 branch; more on that shortly.

To get working binaries from the source code releases it needs to be compiled with the .NET 3.5 compiler. This doesn't require Visual Studio (the compiler is part of the framework) and the resulting binaries will run on .NET 2.0 (SP1). It does require some command line magic, and as I can never remember the exact invocation I blogged it a while back.

Even that isn't now necessary. In his further adventures with Azure (the Microsoft cloud platform), Harry Pierson has created a web application that delivers compiled binaries from the source drops:
Given how often we ship source, we’re thinking of extending the the time between binary drops. Binary releases have to be signed and there’s a fairly arduous process we have to go thru in order to get each binary release out the door.

However, there’s something nice and convenient about downloading a pre-compiled binary release. So I spent my Christmas vacation building a script to download and build IronPython nightly builds. Once built, I compress the binaries and upload them to Azure blob storage. Finally, I built a *very* simple cloud app for users to view and download available nightly builds. As an extra benefit, I’m also providing nightly builds of the DLR.
He's also written a post explaining the technical details behind the nightly build application. As you might expect, most of it is built in IronPython.
The one part I didn’t write in IPy was the Azure cloud web app, which I wrote in C#. Jon Udell’s been investigating getting IPy to run in Azure, but I just wanted something quick and dirty so I decided to use C#.

As for the IronPython parts, for the most part I’m using external tools for downloading, building and compressing. I use the Source Control RSS Feed to discover recent source code changesets, CodePlex Client to download source from CodePlex, MSBuild to build the binaries, 7-zip to compress the binaries and the StorageClient library sample to upload the compressed binaries up to Azure blob storage.

For building and compressing, I’m literally shelling out to MSBuild and 7-Zip via os.system.

Uploading to Azure was very straightforward because of the StorageClient library. Here’s the code to create a blob container object (creating the actual blob container if it doesn’t already exist) and to upload a file to a container.
There are several interesting snippets of information and code in the post.

Thursday, January 15, 2009

Scripting Second Life with realXtend and IronPython

realXtend is an "Open source platform for interconnected virtual worlds".

The server can be scripted with Python using IronPython:
realXtend adds Python script support for the server using Ironpython. The script uses a different kind of system compared to LSL scripts which use the inventory system. This is because the scripts are more powerful and can also access the server machine's other resources. The scripts are stored in the server's ScriptEngines\PythonScript subfolders, not to database. By default this folder contains the RXCore folder which contains the core implementation of the script engine so this folder shouldn't be modified. Samples folder contains some example scripts.
Apparently realXtend can also be used to do some pretty funky things with Second Life scripting:
One of the most annoying aspects of supplying parameters to scripted objects in Second Life is the need to use notecards. Notecards have a variety of annoying restrictions, and LSL lacks decent parsing tools for dealing with parameters on notecards. In contrast, RealXtend has a remarkable, but up till now undocumented facility for supplying configuration data to models in any format you like with virtually no size restrictions on the data. When combined with your favorite object serialization library in Python, a fantastic world of possibilities for initializing and persisting object data appears.
Even better, the blog entry use ConfigObj and has plenty of code, screenshots and a video!

It looks like Peter's blog has other entries showing example Python code for use with realXtend (like this one), so it could be worth an explore.

Wednesday, January 14, 2009

IronPython in Action News and Summaries in Italian

IronPython in Action inches ever closer to publication. Several people have been reading and reviewing the early access version of the ebook. To cover news about the book and its accompanying sourcecode, plus online reviews, I've started an IronPython in Action news blog:
(The blog is run with the excellent new Python blog engine: Zine.)

We've just had the latest deadlines from Manning, and the latest blog entry sets out the schedule - if all goes well the book should go into publication in the middle of next month.

Another of the blog entries notes what Lawrence Oluyede has been up to with the book. Whilst reading he has been posting summaries of each chapter in Italian. So far he is up to chapter 12 (and by what I can make out from Google translate he is enjoying the book).
Lawrence assures me that when he has finished the book he will post a review in English!

Tuesday, January 13, 2009

Resolver One 1.3.1 and the $25000 Spreadsheet Challenge

Resolver Systems has just announced the release of version 1.3.1 of Resolver One, the IronPython spreadsheet. This is purely a bugfix release, with a few usability fixes and documentation improvements.

Resolver One 1.3 was a major release that I didn't cover on this blog. It had several major new features:
  • Row and column level formulae
  • Improved startup time
  • An option to trigger a recalculation, reloading all imports
  • Various API and usability improvements, plus bugfixes
We're currently working on the next major release of Resolver One, version 1.4. The major features in this include moving to IronPython 2 and preliminary support for using Numpy in your spreadsheets through Ironclad.

The spreadsheet challenge is a competition run by Resolver Systems. We're offering $25000 in prize money for the best spreadsheets created with Resolver One.
We're so confident about the revolutionary potential of Resolver One that we've set up the $25,000 Resolver One Challenge. Every month between now and May, we will be giving away $2,000 for the best spreadsheet we receive. And in late May, we'll be handing over $15,000 for the best of the best. Let your imagination run wild.
To go along with the announcement of the competition, Jonathan Hartley created a brand new Resolver One tutorial taking you through all its major and special features (with examples to follow).
Oh, and in other Resolver related news, Resolver Systems is currently being featured on the front page of the Daily WTF!

Monday, January 12, 2009

Reusing Import Symbols to Avoid Performance Hits

Philipp Sumi has been looking at performance optimizations when using IronPython hosted from C#.
Lets do an artificial sample: The snippet below just increments a variable by one. As expected, it executes blazingly fast - after compilation, it executes a few thousand times without getting over a single millisecond:

value = value + 1

Now look at the following example:

import clr
clr.AddReference('System.Xml')
from System.Xml import *
value = value + 1

Basically, this snippet performs the same logic (incrementing the ‘value’ variable), but it contains an import for the System.Xml namespace. It’s not necessary, but it still needs to be compiled. Executing this (compiled!) script 4000 times takes over 5 seconds!
From a hosting point of view, the extra time is spent populating the namespace (the ScriptScope) in which the code is executed. Philipp shows how to reuse imports by creating new scopes from a pre-populated SymbolDictionary.

Announcing esedb, a dbm interface to the Esent database engine

Laurion Burchall has just posted the following message to the IronPython mailing list:

I've added the esedb module (esedb.zip) to the release page of the ManagedEsent CodePlex project. This is still under development and I'm interested in comments, feedback and suggestions on how to do things better in Python.

What is Esent?

Esent is the Windows database engine. It is used by the Active Directory and several smaller applications on Windows clients (e.g. Windows Update). The same code is also used by Microsoft Exchange as their database engine. The unmanaged API is documented on MSDN and there is a managed interop layer on code plex.

What is esedb?

The esedb module uses the Esent interop layer to provide a persisted key/data store in a dbm-like way. The zip file contains the managed interop DLL and the Python module. The code uses the esent DLL which is part of Windows. This code will work on Windows XP up (it won't work on Windows 2000). That does mean that this module will only work on Windows.

>>> x = esedb.open('test.db', mode='n')
>>> x['a'] = 'somedata'
>>> x['a']
'somedata'
>>> x.has_key('b')
False
>>> x['b'] = 'somemoredata'
>>> x['c'] = 'deleteme'
>>> del x['c']
>>> x.keys()
['a', 'b']
>>> x.values()
['somedata', 'somemoredata']
>>> x.close()

Sunday, January 11, 2009

IronShay: Getting Started with Dynamic Languages

Shay Friedman works at a startup called ActionBase, and has a blog "chatting about IronRuby, IronPython, the DLR, Office and more".

He already has several entries about IronRuby, the new dynamic keyword in C# and presentations that he has done. One particularly useful entry is a collection of resources on dynamic languages on the .NET framework:
For IronPython he lists:

Resources
Recommended Blogs
He also lists resources for IronRuby, IronScheme and the DLR.

Saturday, January 10, 2009

DevHawk on LiveFX

Harry Pierson has been playing with Windows Azure and the Live Framework.
In this first post he uses the Live Framework SDK, showing IronPython code to do the following:
I retrieve the user’s ID and password using raw_input (Python’s equivalent to Console.ReadLine). Finally, I connect to the user’s LiveOperatingEnvironment and pull their name, email address, personal status message and the number of contacts they have.
In the second post he accesses LiveFX via the raw HTTP interface rather than the .NET assemblies. He again shows IronPython code using WebRequest - with XmlReader to handle XML and uhm... simpler techniques for handling JSON.
In the third post on LiveFX rewrites his scripts to use the LiveFX Python module by Ori Amiga the Live Framework group Program Manager.

Friday, January 09, 2009

Combining Dynamic Languages and Static Languages

The UK MSDN Flash developer newsletter features an article on the Dynamic Language Runtime by Mark Bloodworth:
Traditionally, there has been tension between dynamic languages and static languages. Each language has its distinct strengths and weaknesses, which means a choice: constrain yourself to one language or face the pain of language interoperability.

The Dynamic Language Runtime (DLR) solves this problem. The Common Language Runtime (CLR) provides a unified platform for statically typed languages. The DLR extends the CLR to provide a unified platform for dynamic languages on the .NET platform. This means that there is a common, unified platform for static and dynamic languages. By having one platform, which means one object world, the pain of language interoperability is history.

In addition to dynamic languages such as IronPython and IronRuby, C# and VB will also, in the 4.0 release of the .NET platform, be able to take advantage of the DLR. In C# this capability is achieved through a new type: dynamic. Declaring a dynamic type in C# 4.0 tells the compiler that method resolution is to be performed at runtime. There’s a DynamicObject abstract class (which implements the IDynamicObject interface) that allows you to create your own dynamic objects in your chosen .NET language. In Visual Basic 10, the latebinder will be updated to recognise objects that implement the IDynamicObject interface, which will allow developers to take full advantage of dynamic languages from VB.

To talk to languages and platforms the DLR uses binders. As you would expect, there’s a binder that allows the DLR to call .NET. Binders are also being built to allow calling Javascript (in Silverlight), Python, Ruby and COM. This model removes the n squared problem of language interoperability and replaces it with a standard, well-defined and well-understood platform.

To find out more about the DLR, the following sessions from PDC2008 are a great place to start.

TL10 Deep Dive: Dynamic Languages in .NET – Jim Hugunin

TL16 The Future of C# - Anders Hejlsberg

To see what you can do with the DLR today, download the latest version of IronPython, which comes with the DLR. Then write some ASP .NET with IronPython or call Python from C#.

To experiment with the dynamic capabilities of C# 4.0, download the CTP of Visual Studio 10 and .NET 4.0. There’s a CTP of IronPython 2.0 for Visual Studio 10 that comes with a Dynamic Programming in C# walkthrough.

Thursday, January 08, 2009

Two IronPythonic Spreadsheets

Jon Udell is at it again; this time looking at different ways to wrangle data with IronPython.
I should get a life, I know, but I can’t help myself, one of my favorite pastimes is figuring out new ways to wrangle information. One of the reasons that IronPython had me at hello is that, my fondness for the Python programming language notwithstanding, IronPython sits in an interesting place: on Windows, side by side with Office, where a lot of information gets wrangled — particularly in spreadsheets.
He has been looking at two very different ways of working with spreadsheets, and both of them have IronPython integration. The first of these is a tool called Inference.NET, which is part of the Inference suite that integrates statistical and analytical tools into Microsoft Excel. Inference.NET allows you to use the "general-purpose scripting capabilities of the dynamic .NET languages, including IronPython and IronRuby".

The second is Resolver One by Resolver Systems (the company I work for). Resolver One is a programmable spreadsheet that is actually written in IronPython and exposes the core object model of the spreadsheet to user code:
With Inference, the IronPython engine is loosely coupled to the Office apps. That buys you the full fidelity of the applications, but costs you Pythonic impedance.

With Resolver One there is no impedance. The application and your data are made of Pythonic stuff. You give up a ton of affordances in order to get that unification, but it enables some really interesting things.
He particularly likes one of our new features: row and column level formulae.
This is a pretty handy idea all by itself. Instead of putting a formula into the first row of a column and then copying it down, you put it into the column header where it applies to the whole column automatically.
The rest of the post demonstrates how column level formulae work, and how he uses them in his crime analysis spreadsheet to transform data using a pipeline, ending up with XML.

Wednesday, January 07, 2009

Extending your C# application with IronPython

Bernie Almosni looks at extending C# applications with IronPython. He shows how to use the DLR hosting API to create a 'rules engine' for the application - and example IronPython code for adding new rules.
I think that in almost every enterprise application project at some point you’ll come to the conclusion that you need extension points in the system. It can be a Rule Engine, a Workflow of some sort or a pluggable User Interface. So now you want to give someone else the ability to extend or modify the application’s behavior with a minimal effort and time. Of course it’s possible to achieve this with custom tools and code generators that compile and load at runtime, but these solutions are complex and time consuming. Utilizing the DLR as an embedded scripting engine is easy and for my opinion quite elegant.

Show Me the Code

As promised, a small collection of links showing people doing useful, interesting or just plain odd things with IronPython.
This first post definitely falls into the odd category. Steve Gilham continues his experiments using the Swing Java UI toolkit from IronPython. This time he creates Python code that runs unmodified on the JVM (with Jython) or the CLR (with IronPython).

He also attempts the same thing with Ruby, using JRuby and IronRuby. It's hard to tell whether there are fundamental problems with his approach on IronRuby or whether it just needs to mature.
Apparently Infer.NET is: "a .NET framework for machine learning. It provides state-of-the-art message-passing algorithms and statistical routines for performing Bayesian inference. It has applications in a wide variety of domains, including information retrieval, bioinformatics, epidemiology, vision, and many others."

Larry O'Brien shows some very simple example code using Infer.NET from IronPython to create an InferenceEngine and predict the probabilities of results when tossing coins.
I needed to find some files without restoring to the desktop search UI – this IronPython script implements a very simple search that uses System.Data.OleDb* to query the Windows Desktop Search catalog.
This simple recipe can be pasted and used from the interactive interpreter. The blog entry includes references if you want to dig deeper.
Inspired by Dive into Python, David Seruyange explores using the Python standard library module sgmllib from IronPython (for processing HTML - pushing data into a database).
IronPython is quickly becoming my scripting language of choice. I really like it. It’s easy to work with and I am getting work done with it. And, in spite of my best efforts, I am even starting to learn Python (a major 2009 goal).
JP Hamilton is using IronPython for build scripts and creating tools. This blog entry shows a script that checks out files from Team Foundation Server source code control. He has created a utility module "tfs.py" that can be used from the console.

Tuesday, January 06, 2009

A New Year Selection

Another selection of interesting links related to IronPython and the Dynamic Language Runtime.
Notable in that this is the first blog entry in a long while from Seo Sanghyeon, maintainer of FePy the community distribution of IronPython.

The blog entry focuses on the differences between how CPython (the reference implementation) and IronPython do encoding name normalisation in the codecs module.
So Python normalizes encoding names received from codecs.lookup. How exactly this is done isn't really specified.

But did you know that the following code work with CPython? (I don't recommend this!!!)

import codecs
codecs.lookup('utf!!!8')

Yes, those are three exclamation marks. I'm not kidding...
Jimmy Schementi is part of the dynamic languages team at Microsoft and maintains the integration with Silverlight. This blog entry is really a presentation that he gave in December, but it has lots of interesting parts. Most of the code examples he gives are in Ruby, but he also has a nice high level overview of how the DLR itself (the magic behind both IronPython and IronRuby) works.
Ted Neward gazes into his crystal ball for the coming year(s). One of his predictions has to do with the consequences of the DLR moving into core .NET (becoming a part of the Common Language Runtime in .NET 4).
The IronPython and IronRuby teams will find some serious versioning issues as they try to manage the DLR versioning story between themselves and the CLR as a whole. An initial hack will result, which will be codified into a standard practice when .NET 4.0 ships. Then the next release of IPy or IRb will have to try and slip around its restrictions in 2010/2011. By 2012, IPy and IRb will have to be shipping as part of Visual Studio just to put the releases back into lockstep with one another (and the rest of the .NET universe).
The Coding Geekette has been counting down the subjects in preparation for the Codemash - happening January 7-9th in Ohio. In this post Sarah outlines how Python, Jython and IronPython will be represented. Plus:
This year, I hope to get my IronPython resources section together here on CodingGeekette.com and blog about it a bit more. I will be talking about IronPython at CodeMash, PyCon, and quite possibly PyOhio. If other people want to hear me talk on it, just send me an email at sarah at codinggeekette dot com
This is a 'first impressions' post of a Python programmer trying IronPython. He mentions Ironclad, shows how to load assemblies and import from them, and also has handy hints for using the Python logging module and working with the Windows registry. He also talks about turning IronPython programs into binary executables with the Pyc compiler.

This collection of posts hasn't included much actual code, so the next one will be more focused on practical examples with IronPython.

John Udell in the Cloud with IronPython

John Udell is a well known blogger with an interest in Python, who now works for Microsoft. Recently he has been exploring various aspects of working with Microsoft technologies with Python, including several posts on using the new cloud computing platform Azure.

His first post doesn't show much code:
But what happens behind the scenes is quite interesting to me. The URL hits a deployment in the Azure cloud where I’m hosting an IronPython runtime. Then it invokes that runtime on a file that contains this little Python program:

hello = "Hello world"

Finally, it gets back an object representing the result of that program, extracts the value of the hello variable, and pops it into the textbox.

This is the proof of concept I’ve been looking for.
He updates this post with a status report:
I’m exploring the viability of Python as a way of programming the newly-announced Microsoft cloud platform, Azure. Partly that’s because I love Python, but mainly it’s because I believe that the culture surrounding Python and other open source dynamic languages can fruitfully cross-pollinate with the culture that infuses Microsoft’s platforms.
It isn't as straightforward as he'd hoped - both because of the lack of C extensions used by standard library modules and because of the Azure security policy.

Jon does however turn his proof of concept into "a service in the Azure cloud that aggregates calendar events from Eventful.com and from a diverse set of iCalendar feeds".
Jon explores the problems of working with Azure from IronPython - Azure runs in a 'medium trust' environment, something that only partly works with IronPython. He also talks about the blurring of the lines between static and dynamic languages:
Along the way, I’ve been recalling something IronPython’s creator, Jim Hugunin, said at the Professional Developers Conference back in October. Jim’s talk followed one by Anders Hejlsberg, the creator of C#. Anders showed an experimental future version of C# that makes use of the Dynamic Language Runtime which supports IronPython and IronRuby on .NET. The effect was to create an island of dynamic typing within C#’s otherwise statically-typed world. We all appreciated the delicious irony of a static type called ‘dynamic’.

Jim might have sounded a bit wistful when he said: “I’m not sure what a dynamic language is any more.” But I think this blurring of boundaries is a wonderful thing. Many smart people I deeply respect value the static typing of C#. Some of the same smart people, and many different ones, value the dynamic typing in languages like Ruby and Python. If I can leverage the union of what all of those smart people find valuable, I’ll happily do so.
His more recent posts track the development of this project, including how his experience of Python development is affecting the C# he writes.

Consuming Extension Methods in IronPython

Extension methods are a feature of .NET 3 that allows you to add methods to existing types. These are effectively static methods, but are called as if they were instance methods on the extended type.

Saveen Reddy has a blog entry on using these methods from IronPython.
  • Consuming Extension Methods in IronPython
    One of my projects implemented in C# makes frequent use of extension methods. Recently I started using IronPython to script that project and what I learned is that consuming those extension methods in C# is straightforward but with IronPython some extra work is involved.
It turns out to be harder than it should be...

Monday, January 05, 2009

Cleanup WSUS - Remove Computers No Longer in the Domain

Another practical example of using IronPython for system administration, this one using WSUS (Windows Server Update Status).
One thing I love about WSUS is the ability to monitor the presence of clients. It gives me a good approximation of the last time a computer was on the network. I often use this information to help me clean missing computers out of Active Directory.

But what about when a computer is removed from the domain before it is removed from WSUS? Rather than manually checking, I wrote an IronPython script that compares the list of computers in Active Directory with the computers on WSUS. When I run this script, it lists computers that should be removed from WSUS, and deletes them for me (after prompting).
Requires WSUS 3.0 API, specifically Microsoft.UpdateServices.Administration.dll. As long as that library is available, the script doesn't need to be run on the server itself.

DevHawk: IronPython and Linq to XML

Continuing his IronPython and Microsoft Technologies series, Harry Pierson has posted four articles on working with XML and Linq from IronPython.
Linq stands for Language Integrated Query and is part of .NET 3. It adds first class support for querying data sources to C# and VB.NET. Underlying this are libraries that can be used from any .NET language (whether it has syntactic support for Linq or not.) In its basic form, "Linq over objects" is remarkably similar to Python list comprehensions and generator expressions.

IronPython doesn't have syntactic support for Linq, but you can use the libraries. Part 2 of this series does exactly this:
LINQ to objects works just fine from IronPython, with a few caveats. First, IronPython doesn’t have extension methods, so you can’t chain calls together sequentially like you can in C#. So instead of collection.Where(…).Select(…), you have to write Select(Where(collection, …), …). Second, all the LINQ methods are generic, so you have to use the verbose list syntax (for example: Single[object] or Select[object,object]). Since Python doesn’t care about the generic types, I wrote a bunch of simple helper functions around the common LINQ methods that just use object as the generic type.
As for the motivation for this, part 1 explains:
There are lots of songs available for Rock Band - 461 currently available between on-disc and downloadable tracks – with more added every week. Frankly, there’s lots of music on that list that I don’t recognize. Luckily, I’m also a Zune Pass subscriber, so I can go out and download all the Rock Band tracks and listen to them on my Zune. But who has time to manually search for 461 songs? Not me. So I wrote a little Python app to download the list of Rock Band songs and save it as a Zune playlist.
Part 3 uses the XDocument.Load (another part of Linq) to read XML, and part 4 uses the XmlWriter to write out the playlist.

IronPython (well the DLR as a whole) shares an important element with Linq. You construct Linq queries with lambda expressions that filter the result. For example (from this blog entry):

static void UseLINQ()
{

var names = new List {
new GenderedName { Name="Bob", Gender=Gender.Boy }
, new GenderedName { Name="Sally", Gender=Gender.Girl }
, new GenderedName { Name="Jack", Gender=Gender.Boy }
, new GenderedName { Name="Sarah", Gender=Gender.Girl }
, new GenderedName { Name="Philbert", Gender=Gender.Boy }
};

var boyNames = names.Where((n) => n.Gender == Gender.Boy).Select((n) => new { n.Name });

foreach (var name in boyNames)
Console.WriteLine("{0}", name.Name)
}
The lambda actually becomes an expression tree which is effectively the AST. This expression tree can be compiled to query the data provider - unsurprisingly Linq to SQL compiles to SQL queries.

The Dynamic Language Runtime uses expression trees as the AST representation for dynamic languages.

You can also represent the same expression with first class Linq syntax (where and select keywords):
var boyNames = from n in names
where n.Gender == Gender.Boy
select new { n.Name }
Introducing this capability into IronPython without changing the language is a challenge for the IronPython team. It may not be possible, but adding new keywords would draw criticism from those worried that Microsoft have entered the 'extend' phase of embrace, extend & extinguish in their new found enthusiasm for dynamic languages!

Saturday, January 03, 2009

Rob Oakes: Learning IronPython

Rob Oakes has set about learning Python through IronPython, and has written a series of articles that both document his progress and serve as a tutorial for others following in his footsteps.

He starts with a confession and a manifesto:
There are certain times for confessions, and now seems like such a time so … here goes: My name is Rob, and I’m an addict. I am addicted to Matlab. If I didn’t have Matlab, my productivity would go to near-zero. For this reason, like any good junkie, I tolerate its expensive fees, obnoxious quirks and serious limitations. That is, until recently.

Some recent events, however, have forced me to start looking for alternatives. [...] I have made a serious decision: I am going to learn Python!
And why Python?
  1. Like Matlab, it is a dynamic (or scripting) language.
  2. Like Matlab, it has an enormous number of existing libraries and functions available. Indeed, after a bit of surface scratching, I’ve come to the conclusion that it might have even more libraries and functions available than Matlab does. Put differently: Python comes with batteries included.
  3. Python has some seriously beautiful syntax.
Even better: "I have started reading the book. Chapters 1 to 3 of IronPython in Action are great. The first three chapters start with a fantastic summary of Python and .Net".

His second article introduces the projects he will work on to learn IronPython, including a Windows Podcast aggregator based on the Windows Presentation Foundation (WPF) GUI framework.
Part three is more code focused and is an introduction to using WPF from IronPython, including the basic user interface components like, loading XAML at runtime, and using doing work in a background thread with the BackGroundWorker class (updating a progress bar as it goes). He made good use of the IronPython in Action examples and the IronPython Cookbook, which is nice to see.
Part 4 is about .NET interoperation, extending IronPython and creating a download manager using the Background Intelligent Transfer Service (BITS).
Part 5 puts what he has learned so far into practise, in the form of a rudimentary download manager. Again, this entry has useful example WPF code.
Part 6 makes the download manager more functional, with more example IronPython and XAML. The download manager is starting to look a bit more impressive.
Part 7 summarises what Rob has learned on his journey:
His major points are:
  • Set goals and measure progress
  • Follow schedules while maintaining flexibility
  • Work on something real
  • Try things out
  • Limit the number of dependencies
  • Find high quality reference and documentation
Under the "Try things out" heading, he makes this note about the Python interactive interpreter (REPL):
Unlike compiled languages, Python and other scripting languages ship with an interactive interpreter that begs to be used. When trying out the various .Net classes, I tried to apply a piece advice found in IronPython in Action.

For those new to dynamic languages, the interactive interpreter will … be a pleasant surprise. Far from being a toy, the interactive interpreter is a fantastic tool for experimenting with classes and objects at the console. You can instantiate and explore … classes live, using introspection and the built-in ‘dir’ and ‘help’ commands to see what methods and attributes are available to you. As well as experimenting with objects, you can also try out language features to see how they work … (IronPython in Action, Chapter 1).

I found the interpreter to be a fundamental part of my experimentation. As you can see from the screen shot at left, the interpreter became one of my most used programs. it even passed Microsoft Word, which is the first program I open after turning on my computer.

I often found myself transcribing interpreter commands into my code, once I had figured out how to accomplish a particularly difficult set of manipulations. In many ways, the interpreter became my de-facto composition board and I was able to avoid the write-compile-run loop equivalent which often sucks up my time when working in other languages.
These articles were only the first part, the download manager, of his podcast aggregator project. Before moving onto the next part (the 'library') he lists some of the tutorials and resources he found useful whilst learning to use WPF from IronPython:
I'm sure we'll see more good things from Rob as he continues his adventures.