Saturday, February 28, 2009

Blogpingr goes GUI

Blogpingr is a Python command line tool for pinging xml-rpc services when you write a new blog entry. Norman Kosmal has been playing with IronPython, so he decided to use it to give blogpingr a GUI:
Now diving into IronPython I decided to give blogpingr a GUI. I am lazy, just so you know and using a shell to execute the programm got me bored. Also this probably is a good oportunity to check out how you can use window forms from within IronPython.

Of course credit goes to Borislav Gizdov`s for coming up with blogpingr in the first place. I just hooked his code into IronPython structures.

I still got to add threads to this programm so the methods won`t lockdown the textbox on execution. Maybe I will add some inplace editing functions to add, edit or remove rpc servers from the .conf file. Once that is down I will release the source on this blog. So stay tuned for updates.

Writing Xml with IronPython, XmlWriter and the 'with' statement

One of the best things about IronPython 2 is that it is based on Python 2.5 rather than 2.4. Amongst other goodies this means that the with statement is available. The with statement is the equivalent of C#'s using statement, although the Python context management protocol methods give a finer degree of control over exception handling.

The with statement makes resource initialization and clean up simpler, and it will be nice to start using it throughout the Resolver One code base. It can often be used to replace the use of decorators or explicit try finally blocks, and many objects in the Python standard library have the necessary context manager methods to be used with "with".

Luis Diego Fallas has written a great blog entry demonstrating 'with' with the XML classes provided by System.Xml:
He creates a small wrapper class that provides the __enter__ and __exit__ methods used by the context mangement protocol, so that he can use with and the XmlWriter for the creation of SVG files.

Friday, February 27, 2009

Apress: Pro IronPython

IronPython in Action is on its way to the printer, and when it is finally available in real paper in about 4 weeks (final ebook available any day now) it will be the first English book on IronPython.

Various other publishers have been asking around the IronPython community for a while now (including O'Reilly which is ironic given that after promising to come back to me I never heard anything more from them when I first pitched them 2 1/2 years ago - ha! I've been very happy with Manning though and so glad things went the way they did), and finally Apress have announced an IronPython book. In the interests of balance (despite managing to make this blog entry entirely about my book so far) here are the details:
IronPython represents a unique direction for developers interested in working with dynamic languages within the .NET Framework. Whether you’re looking to develop applications from scratch or add functionality and maintainability to an existing application, IronPython opens many doors while providing a high–speed, high–performance language that integrates tightly with other .NET languages.

Learn to create applications using the benefits of a dynamically typed language.
Discover how to leverage the power of IronPython to improve existing applications.
Explore interacting with other .NET languages by invoking the common language runtime.

What you’ll learn
  • Write IronPython console and forms applications.
  • Integrate IronPython scripts into existing applications to increase their functionality.
  • Implement complex data patterns and interact with databases.
  • Use the common language runtime to take advantage of .NET’s existing framework within IronPython scripts.
  • Understand how IronPython can be brought in to solve a variety of programming tasks through extensive exercises and recipes.
  • Avoid the common pitfalls of working with a dynamically typed language.
Who is this book for?
This book is designed for .NET developers interested in expanding their IronPython skills and learning how to make use of them to improve their daily programming practices. It will introduce core concepts of IronPython programming using a .NET–centric approach. Additionally, it will speak to Python developers looking to expand into their skills into the .NET world. Experience with a .NET language is expected. Previous Python experience is not necessary.

Author Information
Alan Harris
Alan Harris is a web and application developer living in Arlington, VA. He has worked at more than a few organizations using .NET to create enterprise solutions since the glory days of version 1.1, and can still browse to some of the oldest ones. Aside from working at his desk, he spends most of his time studying Krav Maga and writing music.
According to Amazon the book will be out in June, in which case we'll beat it by a few months - but unless the book has been in progress for a while it is likely that it will be out later. I've not heard of Alan before and I'll definitely be reading the book when it comes out.

Thursday, February 26, 2009

An IronPython Silverlight Bonanza

Silverlight 2 is the Microsoft browser plugin that includes a cut down version of the .NET runtime called the CoreCLR. Silverlight 2 runs on the IE, Firefox and Safari browsers on Windows and Mac OS X (Intel only). The Dynamic Language Runtime can run inside Silverlight, making it possible to natively script the browser with Python.

Meanwhile, the Mono guys have been busy creating a Mono version of Silverlight 2 for Linux called Moonlight.

Although Moonlight 2 hasn't yet been released, they are making great progress with it. The latest development version (build instructions here) is capable of running some of my IronPython demos:
On my Silverlight pages I have a series of articles and simple demos of developing Silverlight applications with IronPython. This includes my Web IDE which is a great tool for interactively exploring the Silverlight APIs and comes with several different demos preloaded. Due to the minor trials of completing IronPython in Action I've only belatedly updated my articles and demos to work with the final release of Silverlight 2:
I'm not the only one experimenting with IronPython and Silverlight however. Steve Gilham got round to converting one of my demos to Silverlight 2 final before I did, and this blog entry compares it to a similar Java applet written using Jython:
Of IronPython and Silverlight he concludes: "now both products are at a stable release, it starts to make sense to actually use the technology".

Steve Gilham also blogs on his Tinesware blog, and has posted an example Silverlight app called astroclock written in IronPython:
Astroclock allows you to set your latitude and longitude and tells you the local time, with a nice animated clock face. It includes a 'permalink' feature that generates a URL to the clock, set to the specific coordinates it is currently set to. He has also developed a version of the clock in F# (a .NET functional programming language): Astroclock in F#

ArtyProg has also posted some example IronPython code for Silverlight, this one a port of a (very short) demo written by Scott Guthrie from C# to Python:
If you crave more detailed information rather than just demos then you'll be interested in this article from Marvin Killing:
There are quite a lot of articles about how to use System.Net.Sockets from inside of Silverlight 2.0 with C#. However, I've not found any info on how to do it with IronPython. Here's what I've found out so you can hopefully avoid some of the pitfalls I've encountered:

Prerequisites:
  • Silverlight 2.0
  • Silverlight Dynamic Languages SDK to develop a Silverlight application with Python
  • Python 2.5 or 2.6 to run the TCP server
  • Visual Studio 2008 to compile the policy server
What we're about to create:
  • A Silverlight app acting as a client (written in IronPython)
  • A TCP server to accept client connections (written in Python)
  • A policy server (written in C#)

Wednesday, February 25, 2009

web2py - a Python web framework that supports IronPython

At PyCon 200 Dino Viehland (core IronPython developer) demonstrated Django running on IronPython. This was very cool, but required minor modifications to Django 0.9.6. Since then Jeff Hardy has been working on getting the latest version of Django working with IronPython, but it isn't yet ready for the prime time.

Meanwhile, web2py is a Python web framework (yes, another one) created by Massimo Di Pierro as part of a course he teaches on Python. Amongst other things, it is being used to run the PyCon 2009 registration system.

The latest release of web2py is the recently announced 1.56 (56!?):
Included in the announcement is this:
- Runs on Jython (although without db drivers)
- Runs on IronPython (although without CSV, db drivers and internal web-server)
As far as I know web2py is the first Python web framework to include IronPython support out of the box.

Python is the easiest language in which to do interesting things

Larry O'Brien is a .NET blogger who is a fan of IronPython and Resolver One. Larry is also a writer for the SD Times technology website. One of his recent articles is on Python, IronPython and Resolver One (amongst other things). It starts well: "Python is the easiest language in which to do interesting things".
As a .NET developer he naturally goes onto talk about IronPython and then discusses IronPython in Action.
For fellow dilettante Python programmers, I recommend the book “IronPython in Action” by Michael Foord and Christian Muirhead. Soon to be published by Manning (I reviewed a preprint), the book is particularly strong in providing simple-but-not-simplistic illustrations and tables that clarify behind-the-scenes structural elements. Visual Studio screenshots may be a little more common than I’d like, but for those new to the VS environment, these may be welcome.

The cheek, there's maybe four or five screenshots of Visual Studio in total - making it one every hundred pages or so! They're mainly showing the IronPython integration and the designers, but if that is his worst criticism of the book then we've not done too badly. Anyway, he continues:

A common challenge for books involving technologies ported to new environments is balancing viewpoints. A strength of the Foord/Muirhead book is just such a balance, providing “Pythonic” topics such as test-driven development, mocks and metaprogramming, along with clear discussions of .NET’s CLR structure, Windows Presentation Foundation, and even programming PowerShell with Python.

Larry is also a fan of Resolver One, the Python powered spreadsheet created by Resolver Systems. I've been working with Resolver Systems for nearly three years, and it is how I got involved in IronPython, so it's nice to hear people say good things about it:

I can’t praise enough the combination of spreadsheet and programming models embodied in Resolver One: I truly believe that this is one of the best ways to do exploratory programming. Although it’s been 15 years since I’ve worked with fuzzy logic, I was able to put together the not-entirely-rudimentary building blocks of a fuzzy manifold editor (a tool for visualizing the response curve of a fuzzy system) in a matter of hours.

Saturday, February 21, 2009

Venturing into IronPython: SQLite and IronPython Studio

Mark Garringer has been venturing into IronPython and posted a couple of blog entries on his experiences.

Mark has been trying to learn Python for a while, and as he is a .NET programmer he found it easier to get into IronPython. His first experiment is a Windows Forms SQLite Administrative UI using System.Data.SQLite. The blog entry shows the code:
The need for this arises because of the lack of built in encryption in SQLite. There are hooks for creating your own, which is what the people who make the ADO.NET Provider, System.Data.SQLite, did. What this means is that the standard SQLite3 command line utility won't open a sqlite database encrypted using the System.Data.SQLite provider. From what I understand, if I REALLY wanted to, I could get the Provider's source and make it so that the SQLite3 library was NOT compiled into the data provider then I could get the library and probably get an existing UI to use it and all would be fine. But where's the fun in that??

Pretty much all this does currently is execute a query against a database. But thats all the functionality I've needed so far so it's all good. This is like 20 minutes of my time but should improve as time passes.
In this entry he also wonders about which IDE to use. IronPython Studio provides intellisense (autocompletion) for IronPython code, but amongst its deficiencies is the fact that it firmly depends on IronPython 1; making it hard to use with IronPython 2.

Another thing IronPython Studio does for you is automatically add references to assemblies you're using. This means that projects created with IronPython Studio can't be run 'out of the box' with ipy.exe. In his second entry Mark looks at what you need to do to your code to run it with the IronPython console.
Note that if you use the IronPython Studio windows forms or WPF designers you will also need to add imports between your project files.

The Problem Solver: IronPython in Action

Maurice De Beijer (the Problem Solver) is a Microsoft MVP, and one of the organisers of the Software Developer Network events in Holland. He has also reviewed a pre-release copy of IronPython in Action.
At the start of the review he explains his interest in Python:
So why am I interested in a book about IronPython? Well I used to do some Python programming a number of years ago and found the language kind of nice to work with. Working with dynamic languages like IronPython relieves you from a lot of the code you have to write just to keep the compiler happy.
The rest of the review presents a quick summary of the four parts of the book, with commentary. And his conclusion:
This book is a good read for all .NET developers who want to know about the new trend towards the dynamic languages and IronPython in particular. You get a good explanation what IronPython can do for you and how to go about solving every day programming problems. With the current push towards dynamic behavior, even in languages like C#, every experienced .NET developer should read this book!

Friday, February 20, 2009

Ironclad 0.8.1 Released

Following hot on the heels of the 0.8 release, William has managed another update to Ironclad. Ironclad is an open source project by Resolver Systems which reimplements the Python C API to allow you to use Python C extensions from IronPython.

He posted the following announcement:
I'm fairly pleased to announce the release of Ironclad v0.8.1; it's not an enormous technical leap above v0.8, but it does now enables you to import and use SciPy and Matplotlib with IronPython on Win32 (with some restrictions; see project page). Downloads of the source and binaries, plus more details, are available at the Ironclad Google Code Project Page -- please do play with it, and let me know if you have any problems.
It's a fairly momentous release; the reason that William is only fairly pleased maybe because the release has been eclipsed by him becoming a father! Congratulations William. The list of C extensions that have now been tried with Ironclad has now been extended to:
  • numpy 1.2 (over 900 tests pass; memory-mapped files, Unicode data, and numpy.distutils don't work).
  • scipy 0.7 (over 800 tests pass; some parts are extremely slow, while others are pretty fast)
  • matplotlib 0.98 (ps backend can produce simple plots; pdf/svg backends should work as well, given a compatible zlib)
  • bz2 from Python 2.5 (well tested)
  • _hashlib, _elementtree, _socket, _ssl, winsound from Python 2.5 (import cleanly; not otherwise tested to any degree of confidence
Ironclad works with IronPython 2 and targets CPython 2.5 on 32-bit Windows; efforts to support other platforms are underway. If you're keen to see Ironclad work with Mono or on 64bit systems then contributions are welcomed!

Ironclad was included in the 1.4 Beta release of Resolver One (our IronPython powered spreadsheet) where it provides the capability of using numpy in the spreadsheet grid and in your spreadsheet user code. We have a 5 minute screencast showing how this works: NumPy in Resolver.

Although a lot of Ironclad is written in C# and uses both the IronPython API and the .NET FFI (Foreign Function Interface - on .NET this is P/Invoke and is what allows you to call into C), there is a core that could be reusable. This is mainly the dll bootstrap (the part that pretends to be the Python C API - which includes some assembly!). Once Ironclad is stable the intention is to factor this out so that other implementations of Python, like Jython and PyPy, only have to reimplement parts of Ironclad to also be able to use Python C extensions.

YAAS 2 (Another Yet Another Awesome Selection)

Another blog entry accreted from only the finest selection of IronPython related blog postings from the last few weeks:
This entry is notable because Dino Viehland (core IronPython developer) has finally written another blog post! This post is on writing DLR binders, a core part of creating dynamic languages that use the Dynamic Language Runtime. DLR binders define the semantics of your language, and Dino explains them along with some C# example code implementing integer addition:
There’s a few key pieces you need to understand here. First you’re returning an expression tree back to the DLR to tell the DLR what to do. The DLR will compile this expression tree and run it. You’re also returning a set of restrictions. These will be evaluated by the DLR to see if it can re-use the same code for other objects in the future. You’re unlimited in what you can do with restrictions as they can be arbitrary Expression trees. But here I’m simply restricting based upon the arguments .NET types which is one of the most common restrictions. Both the expression tree and restrictions get packaged up in a DynamicMetaObject which is conveniently the same type of object you receive as your arguments.

Finally there’s the hash identity which is how the DLR knows if it can share rules or not amongst different binders. The base DLR binders override GetHashCode/Equals to hash on the operation, member name, etc depending on the binder type so I just return this here.
A three page article on Developer Fusion about how programming languages on the .NET platform are evolving in .NET 4.0 (which includes C# 4.0 and Visual Basic 10). A big part of this is the introduction of the DLR to the .NET framework to provide dynamic features to both C# and VB.NET.

Perhaps the most exciting new feature in the upcoming .NET 4.0 release is the Dynamic Language Runtime (DLR). In as much as the Common Language Runtime (CLR) provides a common platform for statically typed languages like VB.NET and C#, the Dynamic Language Runtime provides a common platform for dynamically typed languages like JavaScript, Ruby, Python, and even legacy COM components. It represents a major leap forward in language interoperability for the .NET Framework, providing an abstraction of language operations, shared memory space to avoid marshalling data back and forth between processes, a common set of language features like garbage collection, and the plumbing to convert different representations of data from one language to another.

At a high level, you can think of the Dynamic Language Runtime as having three layers:

  • .NET Language Integration
  • DLR Core Components
  • Language Binders

The first layer, .NET Language Integration, simply represents the need for .NET languages to have a notion of what the DLR is and how to use it. For the most part, you won't even notice this aspect of the DLR because most of the .NET languages had a natural integration point. IronRuby and IronPython are both dynamically typed languages, so the DLR fit right in. VB.NET has always supported the notion of late binding on the Object type, so the DLR incorporated nicely into late binding resolution. C#, however, has no notion of late binding and needed an additional static type for dynamic language support. It's called the dynamic type, and we'll talk about it in more detail a bit later.

The second layer is the Dynamic Language Runtime itself, which consists of three core components: Expression Trees, Dynamic Dispatch, and Call Site Caching. An Expression Tree is a representation of code in the form of a tree, which helps abstract languages into a consistent format on which the DLR can operate. Once dynamic code is in a tree representation, the DLR can look at the tree and generate CLR code from that tree for actual execution. Parsing dynamic code into an expression tree and then building the CLR is an expensive operation, so the DLR employs a performance technique known as Call Site Caching to avoid having to "recompile" the dynamic code each time it's called. Dynamic Dispatch ensures that appropriate Language Binders are used for dynamic invocations.

Language Binders, which make up the third layer, are language-specific implementations of certain operations the Dynamic Language Runtime needs to understand about each language that wishes to participate in the DLR.

Of course, the DLR is far more detailed than this brief overview can provide. For more information on all of its intricacies, please watch Jim Hugunin's PDC talk on Dynamic Languages in .NET

The article also covers the new language features in C# 4 and VB.NET 10, the focus on concurrent programming and F# (a functional programming language for .NET based on Ocaml) becoming a 'first class' .NET programming language.
Boo is another .NET programming language: a Python inspired statically typed language with support for macros and duck typing. There are a whole host of new features "huge improvements all over the board" to quote the release notes, including new macros, pattern matching and generic extension methods. It looks like Boo can also now be used for programming Silverlight!
Whilst we're on the subject of Python inspired .NET languages, there is another Cobra update. Just a few improvements this time, but the language seems to be maturing nicely. It has survived quite a while - and for any new programming language surviving beyond a few months is a major achievement. Cobra is also a Python inspired .NET programming language, with static and dynamic binding and first class support for unit tests and contracts. Like Boo it runs on both .NET and Mono.
In a blog simply titled 'Code' there is some example IronPython code! It is an example of using the MySQL database (System.Data and MySql.Data.MySqlClient). The example reads data from a database and writes it to the console (although why use Console.WriteLine instead of print I have no idea).
In the last selection we read Darren Hawley's note to self number 6. Note 7 is als on IronPython - and this one is on naming Python modules and more exultation of SciTE as an IronPython editor.

Visio from IronPython and Powershell, Parts II & III

Saveen Reddy posted recently on automating Visio to draw diagrams from the command line. He has followed this up with two more posts. Both posts use the Visio automation API from IronPython and Powershell, and have plenty of screenshots to show you the results of the interactive code examples he presents.
This post discusses using data to generate your diagrams; specifically tabular, hierarchical data, and directed graphs:
Draw some simple shapes

>>> vi.Draw.Rectangle( 0, 0, 1,1 )
>>> vi.Draw.Oval( 2, 2, 3,3 )
>>> vi.Draw.Line( 4, 4, 5,5 )

Loading data from a CSV file exported from Excel.

The interactive shell extensively uses System.Data.DataTable to store tabular data

>>> data = ( ('Hello',1) , ('World',2) )
>>> datatable = ToDataTable( data )
>>> vi.Draw.Table( datatable )

create a CSV file in Excel

And then load it as a DataTable and let Visio Draw it

>>> datatable = vi.Data.ImportCSV( r"D:\saveenr\data1.csv" )
>>> vi.Draw.Table( datatable )

Of course, you can load an XLSX file. In this case, you’ll have to identify the name of the worksheet also…

>>> datatable = vi.Data.ImportExcelWorksheet( r"d:\\data1.xlsx" , "Sheet1" )
>>> vi.Draw.Table( datatable )

Drawing Hierarchical Data

Let’s draw a tree from a set of directory paths

vi.Page.New()
items=List[System.String]()
items.Add( r'c:' )
items.Add( r'c:\windows' )
items.Add( r'c:\windows\system' )
items.Add( r'c:\windows\system32' )
items.Add( r'c:\windows\tasks' )
items.Add( r'c:\program files' )
items.Add( r'c:\program files\office live' )
items.Add( r'c:\baz' )
dir = Isotope.Drawing.CardinalDirection.Down
tree = vi.Data.PathsToTree( items )
vi.Draw.Tree( tree.Root , dir )
vi.Page.ResizeToFitContents()
vi.Zoom.ToPage()

Automatic Layout Directed Graphs

Technical Note: the AutoLayoutDrawing class uses Microsoft’s Automatic Graph Layout library

d= VisioDOM.AutoLayout.AutoLayoutDrawing()
s1= d.AddShape('A')
s2= d.AddShape('B')
s3= d.AddShape('C')
c1 = d.Connect('c1',s1,s2)
c2 = d.Connect('c2',s1,s3)
r = VisioDOM.AutoLayout.AutoLayoutRenderer()
r.RenderToVisio( d , vi.visapp , True)
vi.Zoom.ToPage()

This was a simple example, you can draw much more complicated diagrams using the autolayout feature.
This post demonstrates using Visio to draw some shapes, and then automatically generating Excel spreadsheets from the data. This data can also be exported as a CSV or XML file.

He also connects the shapes and then programmatically inspects the relationships between the objects:
In this diagram:
  • Sheet.1 is connected to Sheet.2
  • Sheet.1 is connected to Sheet.3
  • Sheet.2 is connected to Sheet.3
  • Sheet.2 is connected to Sheet.4
Let’s discover this programmatically

>>> pairs = vi.Connect.GetConnectedShapePairs()
>>> for pair in pairs:
>>> print pair.ConnectedShape0.Name, "is connected to", pair.ConnectedShape1.Name, “by way of”, pair.ConnectingShape.Name

This will print …

Sheet.1 is connected to Sheet.2 by way of Dynamic connector
Sheet.2 is connected to Sheet.3 by way of Dynamic connector.6
Sheet.4 is connected to Sheet.2 by way of Dynamic connector.7
Sheet.3 is connected to Sheet.1 by way of Dynamic connector.8

Using this information and looking up of the line endpoints for the connectors, you can identify create a directed graph. (QED)

Tuesday, February 17, 2009

Consuming Extension Methods in IronPython Parts II & III

A while ago Saveen Reddy posted an entry on using extension methods from IronPython.

Extension methods allow you to extend types and interfaces in C# by defining new methods in a *different* assembly. IronPython can only see these extension methods if they are decorated with the ExtensionType attribute.

This is fine if you have access to the source of the extension methods to and can recompile with this attribute. In part II Saveen shows how you can make the extension methods visible to IronPython without having to modify the assemblies containing the extensions - by creating a third assembly:
Part III does some weirder magic using the CodeDom. It allows you to activate the extension methods from inside IronPython without having to create the extra assembly used in Part II.
  • The Isotope.IronPythonUtil is an assembly that contains a method CreateIronPythonExtensionMethodAssembly()
  • CreateIronPythonExtensionMethodAssembly() loads an assembly from a DLL
  • This loads all the assemblies to which that first assembly refers
  • It finds all the extension methods in the first assembly
  • builds a CodeDOM of all the ExtensionType attributes
  • builds a placeholder class (see previous post about the need for this)
  • and generates a DLL
  • The original code generated a “C#” file and used CodeDOM to compile it to an DLL. This one generates a CodeDOM manually so it is faster to create the Dll.
  • The source code is included as an attachment

IronScheme 1.0 Beta 2 Released

There are three Microsoft developed languages that use the Dynamic Language Runtime: IronRuby, IronPython and a closed source language that only runs on Silverlight - Managed JScript.

IronScheme is a 'community-developed' language that also runs on .NET through the DLR. IronScheme includes a Visual Studio plugin (for VS 2008 SP1) and IronScheme 1.0 Beta 2 has just been released.

IronScheme aims to be a R6RS conforming Scheme implementation.
New in the 1.0 Beta 2 release:
  • Added a new LINQ library (ironscheme linq2) which implements all methods found under System.Linq.Enumerable. The new version also uses deferred execution
  • Added a few more libraries
  • Added (ironscheme unsafe) library
  • Many bug fixes
  • Added P/Invoke functionality, and a small FFI layer
  • Many compiler optimizations added (TCE, lambda body hoisting, assignment elimination), more coming for beta 3!
  • Rewrote fixnums and flonums in Scheme. Uses unsafe features.
  • Memory optimizations (previous versions had many objects that could not be GC'd), much less GC's needed now
A codeproject article takes you through installing and playing with the new release - including calling .NET libraries from Scheme.
Personally I'd like to play around with calling into Python from Scheme and vice-versa...

Monday, February 16, 2009

Make Your Application Extendable Using the DLR

IronShay has posted an article on extending applications by hosting the Dynamic Language Runtime. He shows code for adding Python or Ruby scripting to an example Windows Forms application written in C#.
It’s very common for applications to have a way to extend them. Extensibility comes in various ways and have multiple names too – plug-ins, add-ins, addons, etc. It seems, though, that one kind of extensibility was left to the very few – application macros.

The concept is very simple – You don’t need to create a special Dll, implement a specific interface and register it somehow, you don’t even have to install an IDE. All you have to do is to open the relevant window, write some code and voila – you’ve extended the application.

This sample is very simple but it took me about 20 minutes to get it done. It’s just magic!

IronPython and CodeDOM: Dynamically Compiling C# Files

Harry Pierson (DevHawk) has an interesting article on dynamically compiling C# from IronPython using CodeDom. His motivation for this is to allow him to use extension methods without tying himself into a dependency on a specific version of the DLR.
Of course, I could have simply re-compiled the assembly against the new bits, but that would mean every time I moved to a new version of IronPython, I’d have to recompile. Worse, it would limit my ability to run multiple versions of IronPython on my machine at once. I currently have three – count ‘em, *three* – copies of IronPython installed: 2.0 RTM, nightly build version 46242, and an internal version without the mangled namespaces of our public CodePlex releases. Having to manage multiple copies of my extension assembly would get annoying very quickly.

Instead of adding a reference to the compiled assembly, what if I could add a reference to a C# file directly? Kinda like how adding references to Python files works, but for statically compiled C#. That would let me write code like the following, which falls back to adding a reference to the C# file directly if adding a reference to the compiled assembly fails.
If you're interested in this technique, then you may be interested in my article on a similar subject...

Sunday, February 15, 2009

Bridge, Kamaelia and XMPP on IronPython

Kamaelia is a Python concurrency library that uses Python coroutines (generators). Despite being pure Python it is powerful enough for streaming audio and video. Bugs in IronPython 1 prevented Kamaelia running on IronPython, but that doesn't seem to be the case for IronPython 2...

Sylvain Hellegouarch has a Python library called headstock for communicating over XMPP:
For a while IronPython had severe shortcomings that prevented it running simple Kamaelia applications. Today I was able to run a simplechat demo using a vanilla IP2 on Windows with only one single modification to the logging module (thanks Seo). To be honest I didn’t expect it to go through :)

The chat demo is simple enough but means more complex examples using XMPP PubSub will work as well (they are all based on the same framework).

Now this isn’t production ready or anything. For instance the TLS support is broken (hopefully something easy enough to fix) so you won’t be able to connect to Google Talk for now.

Moreover I’m not sure the code is that fast considering how I had to simulate an incremental XML parser atop System.Xml (this allows for a XML stream to be parsed without requiring the full document or even fragment to be read first).

This is a great news for me because it means I’ll be able to move ahead with more work using IronPython 2.

Introduction to WPF with IronPython

Windows Presentation Foundation (WPF) is the successor to Windows Forms; a powerful user interface library for .NET. Where Windows Forms is built on GDI/GDI+ for drawing the GUI, WPF is built on DirectX. This means that much of the work can actually be done on the GPU (the graphics card), and so despite being richer and easier to theme WPF can perform better than Windows Forms.

Mark Bloodworth has been experimenting with WPF and IronPython:
Having scanned a few pages, I decided to create a very simple app that included databinding and an event handler. Armed with Notepad++, XamlPad and a command prompt I set out.

First step was a little C#. I created a class as detailed here that adds a little dynamic-ness to WPF.
WPF is also used by Silverlight - which was originally called WPF/E - and is one of the few Microsoft projects to go from a boring codename to an interesting final release name; WPF itself was originally codenamed Avalon. WPF has a new set of controls, although still nothing like as many as Windows Forms, and uses an XML dialect called XAML for describing user interfaces.

WPF was new in .NET 3.0, which added several new libraries to .NET 2.0 and is the base version of .NET that comes with Vista. It isn't a new version of the CLR (Common Language Runtime), but instead extends .NET 2.0. Both .NET 3.5 and .NET 2.0 SP1 have been released since .NET 3.0. Although several of the 3.0 libraries are being brought into Mono (see the Olive project), the Mono team have said in the past that they have no interest in porting WPF to run on Mono.

As the Silverlight UI is based on WPF and Moonlight (the Mono Silverlight implementation) is well on the way (Moonlight 1 is released and Moonlight 2 is progressing well) perhaps this decision will be revisited.

Saturday, February 14, 2009

Visio/IronPython/Powershell – How to draw nice diagrams from the command line

Saveen Reddy has an automation project for working with Visio - a technical drawing package from Microsoft. The automation project allows you write extensions, plus use Visio from the command line: Visio Automatic Extensions.

He's posted before on this project, for example his hello world in C#, F# and IronPython article. This entry provides lots more examples (with huge screenshots), this time on using the Powershell and IronPython interactive interpreters to drive Visio:
Demo 1 – Shape formatting

Python
>>> vi.Start()

Powershell
vis> $vi.Start()

This will launch a new instance of Visio 2007 that is bound to the Interactive session. When Visio launches this way a new doc will be created with a single empty page. It will also load the Basic Shapes stencil – the interactive session depends on this stencil being loaded.

Draw some shapes in visio. Then select them all.

The selection is important – most of the VisioInteractive commands work on the active selection.

in the interactive shell, set the fill foreground color to red

Python: vi.Fill.ForegroundColor = 0xff0000

Powershell: $vi.Fill.ForegroundColor = 0xff0000

Python on Android with Mono and the DLR

Koushik Dutta has been working on porting Mono to the Android mobile platform, including a bridge that allows you to call into the Java APIs from C# and other Mono supported languages.

Of course through the Dynamic Language Runtime, other Mono supported languages includes IronPython and IronRuby:
After copying the necessary DLLs over to the phone, I was able to run the simple ipy.exe console application that comes with the DLR source code. The neat thing I discovered while perusing through the documentation is that IronRuby and IronPython are actually compiled into CIL byte code and thus eventually native code! Anyhow, as you can see, IronPython is working dandily on the phone.

Friday, February 13, 2009

IronPython 2.0.1 Released

IronPython 2.0.1 is a minor update to IronPython 2.0 which in turn is a CPython 2.5 compatible release running on the .NET platform.

The top priority for this release was improving upon performance while retaining backwards compatibility with IronPython 2.0. One of many notable areas we’ve improved upon is that float-integer comparisons are now 74% faster than they were in 2.0. A full report documenting changes in interpreter performance from 2.0 to 2.0.1 can be found at: Comparing IronPython 2 and 2.0.1 Performance

A special thanks goes out to Resolver Systems for helping us in identifying areas needing performance improvements.

You can download the new release from:
In addition to numerous bug fixes in our IronPython 2.6 branch that were backported to 2.0.1, we also fixed the following CodePlex bugs specifically for this release:
  • 20632: can't write a __len__ returning a uint
  • 20492: TupleExpression.IsExpandable is internal, should be public
  • 20605: Compiling with pyc and PySerial module
  • 20616: wrong TypeError message when invoking str.join: implicit parameter 'self' not counted
  • 20623: InitializeModule needs to add refs to mscorlib/System

ArcGIS (Geographical Information System) from IronPython

ArcGIS is one of the market leaders in the massive Geographical Information System software industry. Alex Willmer has been using the ArcGIS APIs from IronPython and written up the results of some of his experimenting:

On the desktop ArcMap is used to create map documents (.mxd file), whilst ArcCatalog is used to manage data sources. ArcGIS Server can (amongst other things) serve a map document, as a service for web client, Google Earth or remote ArcMap users.

ArcGIS may be automated to an extent, through an interface known as ArcGIS Geoprocessing. But this covers only some cases, delving deeper provides much greater opportunities.

ArcGIS is built on a COM object library named ArcObjects. Native ArcGIS files, such as an ArcSDE connection (.sde file) are the in memory COM object, serialised to disk as binary. It is difficult to edit or create such files in an automated fashion, without calling ArcObjects.

So, like the Java code CreateSDEConnFile.py calls ArcObjects directly. It can produce an ArcSDE connection file, suitable for ArcCatalog. It works by calling the .NET bindings, through Interop assemblies. Anything that can be done through VBA, or C# should be possible through IronPython.

IronPython in Action Reviewed by Lawrence Oluyede

Lawrence Oluyede is a Python developer who used to do C# development and has also used IronPython. He has been reviewing IronPython in Action, and has posted summaries of all fifteen chapters in Italian as he has read them. You can read all of his summaries from here.

He has now written a review in English. This also summarises the chapters, but is not as long as the Italian entries!

Recently I’ve had the opportunity to read Michael Foord’s upcoming book: IronPython in Action and let me say that I found it extremely interesting, and I suggest it to everybody needing (or wanting) to work with .NET from a dynamic perspective or to .NET developers interested in the world out there.
The book covers a lot of ground, sometimes deeply, sometimes just in the surface but it seems to talk about everything there’s to know to make that ground solid enough to build something lasting.
From the Python introduction to IronPython extension, from IronPython embedding to XML, Visual Studio, testing, metaprogramming, Windows Presentation Foundation (the .NET new UI), system administration, ASP.NET, databases, Silverlight and much more.

His conclusions:

I personally found the book very enlightening and there’s much of interesting stuff in there for both Python and .NET developers. I guess it’s the first book on IronPython and anyway it’s a must have for everyone who needs to work on .NET.
It shows there are some unpolished parts with the integration and some differences between CPython and IronPython behavior but it can only become better in the future.

Tuesday, February 03, 2009

Ironclad Screencast and NumPy in Resolver One

Following on from the release of Ironclad 0.8 it is now possible to run through the NumPy tutorial at the IronPython interactive interpreter.

Resolver Systems also has exciting news about using NumPy inside the spreadsheet. We have a four minute screencast showing off both of these things:
The screencast demonstrates using NumPy arrays of up to a million numbers inside the spreadsheet grid, with impressive performance despite being early days in the integration. For two dimensional arrays the grid is a natural representation; and multi-dimensional arrays can be sliced through the user interface.

The spreadsheet part of the screencast uses the new beta version of Resolver One, which is built on IronPython 2:
New features in version 1.4 include:
  • We’ve moved over to IronPython 2.0. This brings you some immediate advantages, including bugfixes to the core scripting language, new supported core libraries like urllib2, and support for Python 2.5 syntax in your user code.
  • Alpha support for numpy, Python’s flagship number-crunching library. This feature is very much at an “early access” level - but it’s certainly ready enough for you to try and to tell us what we need to add to make it ready for your needs.
  • The first steps towards what we call “model-side scripting”. In Resolver One 1.4, you can set cell’s formulae from your button click handlers. While this is a small change in itself, it has big consequences - and should be a big help for people trying to write CRUD applications in Resolver One.
  • A major upgrade to our support for statistical calculations, with 24 new statistics functions, from AVEDEV to VAPR.

Sunday, February 01, 2009

YAAS (Yet Another Awesome Selection)

I've nearly caught up with my backlog, but in the meantime there have been several selections full of smaller entries and snippets. Here's the first one.
Two new pages on the IronPython cookbook showing how to use COM interop to automate Microsoft Word.
I've featured various non-English articles on IronPython before, but I think this is the first one in Russian. This article shows how to use IronPython to provide macros for C# .NET applications. The example shows Python scripts adding menu items to a notepad style app.

I'm not sure of all the details, this is the best google translate could manage for part of the article:
Create your tongue + interpretor to him with an opportunity. NET interoperability - non-trivial task, leave it for the enthusiasts.
The code is all shown though, so it should be easy to figure it out.
Steve Gilham continues his adventures of experimenting with Scala, Jython, and IronPython all using the same codebase...
in my quest for a cross-VM (JVM and CLR), single code-base, polyglot language stack, which doesn't involve any more Java than can be helped.

To summarise where we are, and what we have discovered to date --
  • Python can call Scala and Java happily
  • Java calling into Python is funky and platform dependent; doubly so for Scala
  • Scala cannot both call into java.* classes and compile on .net
  • Low-level Scala pre-defs differ in the Byte type (signed in Java/J#/Scala-JVM, unsigned in Scala-msil)
  • Ruby's predilection for Pascal-case namespaces (fudged in JRuby to match Java's lower-case convention) would require a lot of plumbing around in C# to mate IronRuby to J# or Scala-msil (as the rest of .net uses Pascal-casing).
A very simple example showing how you reference .NET assemblies from IronPython, and import and use classes from the namespaces in the assemblies.
Darren Hawley leaves himself a note on using the pyc.py IronPython compiler and reccomends SciTE as an IronPython IDE rather than the IronPython Studio.
Those of you interested in how dynamic languages are influencing the .NET framework may be interested in this article on the IDynamicObject interface. Specifically it shows how to implement the GetMember method to provide dynamic behavior for objects declared as dynamic. GetMember is the equivalent of __getattr__ in Python.
OpenBabel (a .NET library described as "The Open Source Chemistry Toolbox") already supports IronPython (example) and Noel O'Blog is looking at explicitly supporting Linux and Mac OS X through Mono.