In the latest release of Resolver One we improved startup time by about 20-30%. Most of the startup time in Resolver One is spent in importing Python modules, which is considerably more expensive in IronPython 1 than it is in CPython (even when compiled in binary form). We achieved most of our speedup by delaying the creation of certain objects until they are needed (if ever), and other standard techniques for performance improvement.
There is good news for IronPython 2. Binary compilation is much more efficient as we can compile multiple packages into a single binary and then ngen (pre-JIT), making imports faster.
Whilst exploring how to improve our startup time Kamil Dworakowski experimented with a system for performing parallel imports on multiple threads.
IronPython doesn't have the same import lock as CPython, so you need to ensure that parallel imports don't pull in the same modules simultaneously or some of the imports may fail. To get round this Kamil created a modified modulefinder that analyses a codebase and generates a dependency graph for all the imports (done at 'compile time' not runtime). So long as you start at the leaves of the graph you know you are safe. You can then configure how many threads the parallel importer should use in its threadpool to do the imports.
On a multi-core machine this yields substantial improvements for import time. Unfortuately there are a few thread safe issues for importing in IronPython 1, leading to intermittent crashes when starting Resolver One. We weren't able to use Kamil's code for Resolver One 1.3, but the specific problems we encountered are fixed in IronPython 2 so we might be able to use it in our next release...
Even though we haven't yet been able to take advantage of Kamil's work, others have. Dan Eloff, who is building a Silverlight gaming platform with IronPython and C#, has used it. He had this to say on the IronPython mailing list:
Thanks to the excellent work of Kamil and the help of Jimmy I now have importing being done in parallel in 4 threads in Silverlight.
I had to restructure my code a little to reduce circular imports (not
a bad thing to do anyway) and I had to hand-tweak the dependency graph to deal with conditional imports and other difficult to track
dependencies.
The result was better than I could have hoped for. I'm seeing 43%
faster loading on my dual-core development pc. I used to have blank screen at start that made you wonder if the browser has frozen. But now the browser is responsive and I have a nice working progress bar. But that's not the best news.
The kicker is that the benefit is even larger on slow single core
machines with dialup internet. I posted a while back on this list that
IE does not immediately send an asynchronous request. It waits until its UI thread is idle (which doesn't happen if your importing modules like crazy on it.) So the net effect was the application loads (about 28 seconds on a P4 2ghz) and then it sends the request, waits for the response (which is big in my case), and processes it - all in order.
So if that takes another 30 seconds, you have nearly 1 minute of load time. With the UI thread mostly idle now, this happens in parallel to the importing, and the overall load time drops by a whopping 45%.
In the words of my generation - w00t!
Saturday, November 29, 2008
Friday, November 28, 2008
WSGI on .NET and in the Cloud
Jeff Hardy has been working on getting Django to run on IronPython, or at least that is part of what he has been working on. This includes his project NWSGI:
NWSGI is a .NET implementation of the Python WSGI specification using IronPython. It is implemented as an ASP.NET HttpHandler. The main goal is to provide an easy path for running Python web applications/frameworks (such as Django, Trac, etc.) on IIS7.
He recently released a new version of NWSGI:
This release has some possibly breaking changes and a whole host of new features. There is also an installer available along with the zip archive.
New features in this release:
Using NWSGI to run Django on IronPython opens up some interesting possibilities, such as using ASP.NET's caching, session, or authentication systems for Django. I imagine any ASP.NET shop spent time configuring their ASP.NET providers for those systems, and I thought it would be interesting to see if that effort could be used for Django apps as well, and it certainly looks like they can.
NWSGI is a .NET implementation of the Python WSGI specification using IronPython. It is implemented as an ASP.NET HttpHandler. The main goal is to provide an easy path for running Python web applications/frameworks (such as Django, Trac, etc.) on IIS7.
He recently released a new version of NWSGI:
This release has some possibly breaking changes and a whole host of new features. There is also an installer available along with the zip archive.
New features in this release:
- The major new feature is IIS7 management integration, including a UI editor. It makes configuring NWSGI a breeze from the command line (appcmd) or from the UI. The IIS7 integration is only available through the installer.
- There is one minor feature addition: wsgi.file_wrapper is now supported for faster file transfers. It uses HttpResponse.TransmitFile to send the file directly.
The changes are in the configuration system to support the IIS7 management system. Unfortunately, IIS7 doesn't use System.Configuration; instead, it has its own assembly, Microsoft.Web.Administration. This caused some complications for configuration. There are now three possible usage scenarios (IIS6, IIS7 xcopy, and IIS7 installed) that require slightly different web.config settings; I will detail these in an upcoming post.
As for using Django with IronPython and NWSGI, Jeff has some interesting ideas (and demo code to go with it) on ASP.NET features that might actually be useful for Django:Using NWSGI to run Django on IronPython opens up some interesting possibilities, such as using ASP.NET's caching, session, or authentication systems for Django. I imagine any ASP.NET shop spent time configuring their ASP.NET providers for those systems, and I thought it would be interesting to see if that effort could be used for Django apps as well, and it certainly looks like they can.
A demo version of django-aspnet is now available...
Azure is Microsoft's new cloud computing platform. If you're interested in using IronPython and NWSGI on Azure, Jeff has been looking at that too:
Architecturally (as far as NWSGI is concerned), Azure is just IIS7 running in medium trust. You have to use xcopy deployment, which makes me glad I went to the effort of supporting it.NWSGI 0.6 actually supports medium trust. I only discovered after the release that the error I was hitting was a bug in Cassini; it works fine in medium trust on IIS7. With that hurdle out of the way, there's no reason NWSGI shouldn't work on Azure.
And, what do you know, it works just fine: NWSGI "Hello, World" for Windows Azure. You'll need the Azure SDK and the Visual Studio tools to run it.
Two Source Code Repositories for IronPython
Dave Fugate is an IronPython tester. In the blog entrry he talks about how source code from the Microsoft internal development repositories is synced to the TFS repository on Codeplex.
In particular he addresses the issue as to why all the commits to the public repository have the astonishingly helpful checkin message "Latest sources migrated to CodePlex TFS", rather than actually telling you anything useful about what has changed.
He also explains how their processes have changed, enabling them to do code syncs to the public repository more often. (Something that is very useful for us at Resolver Systems as we port Resolver One to IronPython 2. We are using the new source drops daily so that we can pick up problems as quickly as possible.)
In particular he addresses the issue as to why all the commits to the public repository have the astonishingly helpful checkin message "Latest sources migrated to CodePlex TFS", rather than actually telling you anything useful about what has changed.
He also explains how their processes have changed, enabling them to do code syncs to the public repository more often. (Something that is very useful for us at Resolver Systems as we port Resolver One to IronPython 2. We are using the new source drops daily so that we can pick up problems as quickly as possible.)
Thursday, November 27, 2008
Ironclad 0.7 Released
Following swiftly on heels of the IronPython 2 RC 2 and DLR 0.9 is a new release of Ironclad. Ironclad is an Open Source project by Resolver Systems, and is an implementation of the Python C API in C#. It allows you to use Python C extensions from IronPython, and our goal is to be able to use Numpy within Resolver One spreadsheets.
The latest release is version 0.7 and includes several major advances since the last release.
Major advances in the release include:
C:\compile\ironclad-v0.7-bin\build>"C:\Program Files\IronPython 2.0\ipy.exe"
IronPython 2.0 (2.0.0.0) on .NET 2.0.50727.1433
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append(r'C:\Python25\Lib\site-packages')
>>> import ironclad
>>> import numpy
Detected numpy import
faking out modules: nosetester, parser, mmap, urllib2, ctypes
>>> numpy.__version__
'1.2.1'
>>> a = numpy.array([[1, 2, 3], [4, 5, 6]])
>>> a.shape
(2, 3)
>>> a[1, 1]
5
>>> a = numpy.arange(60).reshape(3, 4, 5)
>>> a
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]]])
>>> a[..., 3]
array([[ 3, 8, 13, 18],
[23, 28, 33, 38],
[43, 48, 53, 58]])
>>>
So much progress has been made in this release that Giles Thomas hints that "we’re now seriously considering having it as an option (with an explicit note that it’s not ready for production use) in the next release of Resolver One".
The latest release is version 0.7 and includes several major advances since the last release.
Major advances in the release include:
- Huge chunks of numpy can now be imported and used
- Lots of the numpy tests now pass (from core, fft, lib, linalg and random subpackages) - the distribution includes a test runner that runs the tests we have enabled so far
- Major performance improvements (several orders of magnitude faster in some areas)
- Ironclad works with IronPython 2 RC 2 and it no longer creates a separate Python engine, nor does it break the importing of .NET libraries!
C:\compile\ironclad-v0.7-bin\build>"C:\Program Files\IronPython 2.0\ipy.exe"
IronPython 2.0 (2.0.0.0) on .NET 2.0.50727.1433
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append(r'C:\Python25\Lib\site-packages')
>>> import ironclad
>>> import numpy
Detected numpy import
faking out modules: nosetester, parser, mmap, urllib2, ctypes
>>> numpy.__version__
'1.2.1'
>>> a = numpy.array([[1, 2, 3], [4, 5, 6]])
>>> a.shape
(2, 3)
>>> a[1, 1]
5
>>> a = numpy.arange(60).reshape(3, 4, 5)
>>> a
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]]])
>>> a[..., 3]
array([[ 3, 8, 13, 18],
[23, 28, 33, 38],
[43, 48, 53, 58]])
>>>
So much progress has been made in this release that Giles Thomas hints that "we’re now seriously considering having it as an option (with an explicit note that it’s not ready for production use) in the next release of Resolver One".
IronPython 2.0 RC 2 and DLR 0.9 Released
IronPython 2.0 Release Candidate 2 has just been released. This fixes several bugs identified in RC1 (several of these reported by Resolver Systems as we port Resolver One to run on IronPython 2).
Barring the discovery of any major new bugs, the intention is for IronPython 2 final to be released within 2 weeks!
Bugs fixed in this release include:
In the announcement on the IronPython mailing list, Bill Chiles (DLR program manager) had this to say about the Dynamic Language Runtime:
IronPython Program Manager is excited about the release, and describes it as Early Christmas from Iron Languages and DLR :
Tomorrow may be Thanksgiving, but the Microsoft DevDiv dynamic language teams are trying to make it feel like Christmas with three separate pre-holiday releases.
IronRuby 1.0 Alpha 2
There’s been zero blog traffic on this, just a notice on the IronRuby mailing list. As per said notice, “Notable features” include “the inclusion of iirb.bat, igem.bat, irails.bat, irake.bat”.
Seshadri (one of the DLR testers) promises regular binaries and weekly source drops in his blog entry on the DLR release.
Barring the discovery of any major new bugs, the intention is for IronPython 2 final to be released within 2 weeks!
Bugs fixed in this release include:
- 19350: Missing restrictions on index operation against user defined type
- 11733: time.timezone has different semantic than in cpython
- 19675: subclasses of float have trouble with __int__ and __str__
- 19656: Module Name Lookup Broken
- 19678: in operator calls __getitem__ on class that has __len__ and __iter__ defined
- 19665: operator.isSequenceType(NewStyleClass) broken
- 19130: One hour error in IPY implementation of time.mktime and time.gmtime
In the announcement on the IronPython mailing list, Bill Chiles (DLR program manager) had this to say about the Dynamic Language Runtime:
The DLR project provides one-stop shopping for folks who want to:
- implement a language on .NET using the DLR
- add dynamic features to their existing language like C#’s ‘dynamic’
- add scripting to their applications.
- create .NET libraries with dynamic objects
This Codeplex project includes the DLR sources as well as sources for IronPython, IronRuby, and samples (such as ToyScript). This is a classic open source version 0.9 release with high quality code through much of the system. We’re shipping some of this in the .NET 4.0 release coming up. Some parts are still open to design changes as we solidify those parts for post .NET 4.0, and we’ll continue to add more docs and samples. We will have coordinated releases or sync points across IronPython and IronRuby sites for major releases so that if you’re not pulling from the DLR project, you can still see which sources or releases are consistent across our projects.
Jim Hugunin did a great talk at PDC 08 on DLR architecture, concepts, and coding with the .NET 4.0 functionality (http://channel9.msdn.com/pdc2008/TL10/ ). In his talk, he also outlines what’s in .NET 4.0:
- fast dynamic dispatch with polymorphic inline caching
- dynamic object interoperability across languages and libraries (including C# ‘dynamic’ consuming IronPython and IronRuby objects naturally, including any language whose objects participate in the DLR’s dynamic object protocol)
- support for library authors to easily make their model objects consumable with nice looking and lightweight code in languages that support the DLR dynamic object protocol (for example, with C#’s ‘dynamic’, you can write xml.Customer.Name instead of xml.GetChild(“Customer”).GetChild(“Name”)
- ability to mix binding logic from various languages and library objects in a single dynamic call site cache
- Expression Trees v2 with support for control flow, assignments, etc.
- COM IDispatch binding as DLR dynamic objects
The following highlights the support shipping only on Codeplex until a future .NET release:
- common hosting model for languages built on or supporting the DLR hosting model
- helpers such as a default .NET binder, complex numbers, tuples
- more Expression Tree support, such as globals access in hosted scenarios and iterator/generator functions.
All the code is available for what the DLR Team is shipping in .NET 4.0. Furthermore, we’re releasing all the code for LINQ Expression Trees v1 since the DLR merged its ASTs with LINQ trees. Some of you have noticed this code has been on the IronPython site for a couple of months now. We will continue indefinitely shipping all of our code open source through .NET 4.0 shipping and future releases of .NET as we move more and more of the DLR into .NET (for example, the common hosting APIs and language implementer APIs). We have no plan in place at this time for source take back from the community.
It is worth noting that the sources and binaries in the .zip files are circa mid-October, and the daily source tree pushes have several small but pervasive cleanups to the API (name changes), perf improvements, and so on. We’re busy here solidifying final design changes for the DLR parts going into .NET 4.0. We wanted to have our release match IronPython 2.0, and they had to lock down sources before we did. The documents in the release speak to the latest sources. Some of the documents are our internal working specs, but in general we prefer you know where we’re planning to end up with the code. There are no significant cognitive changes in the APIs, so you’ll easily map from the docs to the actual code.
For a consistent IronPython release, go to www.codeplex.com/ironpython and get their v2.0 RC2. We won’t normally have consistent release across all projects for RCs, betas, etc., and we don’t have one for IronRuby today. When we RTW IPy v2 and DLR v0.9 shortly, we’ll have consistent releases for all three projects.
If you want to read some documents without downloading sources or bits, go to http://www.codeplex.com/dlr/Wiki/View.aspx?title=Docs%20and%20specs :
- dlr-overview.doc
- sites-binders-dyn-objs-spec.doc
- expr-tree-spec.doc
- dlr-spec-hosting.doc
There is a discussion list at http://www.codeplex.com/dlr/Thread/List.aspx (click on the “get email notifications” link). All mail there goes to dlr@microsoft.com which has all the folks working on the DLR and our languages on it.
IronPython Program Manager is excited about the release, and describes it as Early Christmas from Iron Languages and DLR :
Tomorrow may be Thanksgiving, but the Microsoft DevDiv dynamic language teams are trying to make it feel like Christmas with three separate pre-holiday releases.
So there you go, new versions of IronPython and IronRuby plus a whole new DLR CodePlex project to boot. Enjoy.
He notes that along with the IronPython and DLR releases, there has also been a new release of IronRuby:IronRuby 1.0 Alpha 2
There’s been zero blog traffic on this, just a notice on the IronRuby mailing list. As per said notice, “Notable features” include “the inclusion of iirb.bat, igem.bat, irails.bat, irake.bat”.
Seshadri (one of the DLR testers) promises regular binaries and weekly source drops in his blog entry on the DLR release.
GUI Automation: Building the Framework (3)
After a hiatus, the GUI Automation blog continues the series on the automated testing of Windows GUI applications. This entry uses the Win32 API to interact with a list box. As usual source code is provided for download:
In the entry Lukas makes an important point about any testing framework:
We do not test the list box component. We test the application.
When testing an application you don't need to test the functionality provided by the operating system or the frameworks you're using; you test your application logic.
Example code for selecting an item in a list box from the application under test (is SUT for "System Under Test" a commonly used acronym?):
In the entry Lukas makes an important point about any testing framework:
We do not test the list box component. We test the application.
When testing an application you don't need to test the functionality provided by the operating system or the frameworks you're using; you test your application logic.
Example code for selecting an item in a list box from the application under test (is SUT for "System Under Test" a commonly used acronym?):
def Select(self, aItem):
""" select an item from list box
@aItem - string with item to select
"""
if aItem not in self.items:
raise Exception("Item '%s' not in list box" % aItem)
self.guiat.Activate()
self._CheckVisibility()
pos = self.location
# click on the first item to focus the list box
Win32API.MouseClick(pos[0] + (pos[2]/2), pos[1] + 3)
# send Home to be sure we are on the first item
# (we could be scrolled down a little)
Win32API.SendKey(Win32API.VK_HOME)
# simulate pressing down arrow until we find the item
# we should find it because it is among self.items
while self.value != aItem:
Win32API.SendKey(Win32API.VK_DOWN)
Friday, November 21, 2008
A Whole Bunch of Stuff: Part II
More IronPython and DLR related stuff collected over the last few weeks.
This looks like a very interesting blog entry, and the only reason it is here rather than having a post to itself is that it is in Spanish and so I can't tell you much about what it says... It is by Jacobnix and about using IronPython with Mandriva Linux and Mono.
Two patches to Mono (by Bill Holmes) allowing you to compile and use recent versions of IronPython 2. This is great news as the version of IronPython 2 included with Mono (from FePy) is 2.0 alpha 5 - which is pretty out of date now.
Two new recipes in the IronPython Cookbook (both by Davy Mitchell I believe).
This StackOverflow question yielded several answers, including this example IronPython code for working with the Microsoft SQLServer database:
import clr
clr.AddReference('System.Data')
from System.Data.SqlClient import SqlConnection, SqlParameter
conn_string = 'data source=; initial catalog=; trusted_connection=True'
connection = SqlConnection(conn_string)
connection.Open()
command = connection.CreateCommand()
command.CommandText = 'select id, name from people where group_id = @group_id'
command.Parameters.Add(SqlParameter('group_id', 23))
reader = command.ExecuteReader()
while reader.Read():
print reader['id'], reader['name']
connection.Close()
This blog entry is in Japanese, so I have no idea what it is about (I would really like to know)! All I know is that if you download the example application and run it with IronPython it complains about you not being logged in to the IRC channel and giant WPF teddy bears float down your screen! The program is called 'irc-wankumer.py' which is intriguing.
A French firm called LeeBeLLuL have a large web application written almost entirely in IronPython. One of their developers recently posted to the mailing list:
"I developed an application with LeeBeLLuL that communicates with Google App Engine in SOAP. I mainly use the DataStore, process are done on the PC. I think GAE is an effective platform with good response time, whose implementation is easy with SOAP."
Again, I have no idea what is actually going on - but the page linked to has quite a few examples of working with XML (presumably SOAP as they also have a big WSDL definition) from IronPython.
You've just missed my colleague speaking at the Oredev developer conference in Sweden: "Pumping Iron: Dynamic Languages on .NET". I'm mainly posting this link because his bio is fun and you can see from his photo how much weight he has gained recently...
Jonathan and I have a proposal to do an IronPython tutorial at PyCon 2009.
A Whole Bunch of Stuff: Part I
My backlog includes a lot of interesting snippets and blog entries that probably don't warrant a whole entry to themselves. There's a lot of good stuff in them, so I'm splitting them out into two separate entries.
A blog entry on using the IronPython 1 hosting API to call functions - including embedding it in ASP.NET pages.
This happened a while ago, but I missed it. Windows High Performance Computing Operating System is out (HPC Server 2008) - and it includes IronPython 'out of the box'!
"To tempt Unix users and coders in scientific programming languages, there is a built-in POSIX shell, support for IronPython, IronRuby and Fortran. Microsoft is also developing new tools such as the F# programming language and .NET Parallel Extensions."
"Much of the legacy code that developers may need to support is Linux-based, he says. Mendillo notes that support for Iron Ruby, Iron Python, Fortan and a built-in Posix-compliant shell should help many developers bridge to non-Windows environments."
I noted recently that the Intellipad tool, part of the Oslo framework, is scriptable with IronPython. Jon Flanders has a blog entry showing how to write new commands for IPad using IronPython.
One of the things I didn't mention in the recent entry on Silverlight 2 is that a version for Windows Mobile has been planned. The version of .NET targetting Windows Mobile (and the XBox 360) is the Compact Framework - and this doesn't include the 'Reflection.Emit' (code generation and introspection) APIs needed by the Dynamic Language Runtime. This unfortunately means that Silverlight 2 for Windows Mobile won't be able to run applications written in IronRuby or IronPython.
As the Dynamic Language Runtime forms the basis of both the IronPython and IronRuby projects (and currently slightly out of sync versions of the DLR are included in the downloads for both of these projects - making it hard to create projects that target both languages), and the DLR will be part of .NET 4, it really needs its own home. That home is a project page on the Codeplex Open Source project hosting site. This page is little more than a placeholder currently, but it notes:
"The Dynamic Language Runtime enables language developers to more easily create dynamic languages for the .NET platform. In addition to being a pluggable back-end for dynamic language compilers, the DLR provides language interop for dynamic operations on objects. The DLR has common hosting APIs for using dynamic languages as libraries or for scripting in your .NET applications. This project provides one stop shopping for the DLR components, our open source implementations of IronPython and IronRuby, documentation, and samples."
"This project is currently under construction, and we're targeting our first release of DLR v0.9 in November. We will be releasing simultaneously with IronPython v2.0. You can see all the DLR code in the IronPython Project's latest RC release and weekly source pushes. Our code is also available on IronRuby's site with daily source pushes."
Thursday, November 20, 2008
The Microsoft Dynamic Languages Team Blogroll
The dynamic languages team at Microsoft are relatively small, but have an extremely high proportion of bloggers! (The obvious conclusion is that dynamic language enthusiasts tend to be better communicators...)
Oleg Tkachenko lists them on his blog:
For the record, they are:
Oleg Tkachenko lists them on his blog:
For the record, they are:
- John Lam – IronRuby PM
- Harry Pierson – IronPython PM
- Jimmy Schementi – PM for Silverlight integration
- Shri Borde – dev lead for the team
- Dave Remy – PM lead fro the team
- Oleg Tkachenko – Visual Studio integration
- Dave Fugate – IronPython tester
- Curt Hagenlocher - IronPython/IronRuby dev
- Jim Hugunin – architect for the team, DLR and Visual Studio languages
- Tomas Matousek – IronRuby dev
- Srivatsn Narayanan – IronPython/IronRuby tester
- Jim Deville – IronRuby tester
- Dino Viehland – IronPython dev
Wednesday, November 19, 2008
Silverlight 2 and Dynamic Languages
Well my backlog is logged so far back that I haven't even blogged about the final release of Silverlight 2.
In case you've been living under a rock for the last year, Silverlight 2 is a browser plugin from Microsoft. It is similar to Flash (aimed at games, media streaming and rich internet applications) and is cross-platform (Mac OS X and Windows - the officially blessed Linux port Moonlight by the Mono guys is making good progress though) and cross browser (IE 7+, Safari & Firefox 2+). Unlike Flash it can be programmed with a choice of languages, and through the Dynamic Language Runtime it can be programmed in Python, Ruby and Javascript.
Silverlight 2 final is now out, and according to Scott Guthrie has now been installed on over 100 million consumer computers.
Naturally Jimmy Schementi, who maintains the Dynamic Language Support for Silverlight, released an updated version of the Silverlight Dynamic Languages SDK (sucky name - more on this in a bit):
In preparation for my talk at the PyWorks conference I updated my IronPython Web IDE (tool for experimenting with the Silverlight APIs from Python in the browser) and Try Python (interactive Python interpreter in the browser) for Silverlight 2:
Of course if you're interested in building internet applications with Silverlight then you will want the extended controls that come as part of the Visual Studio Tools for Silverlight 2 (this works with Visual Studio Professional or the free Visual Web Developer Express). The assemblies that come with the tools include the data grid, extra controls like the date picker and various other useful APIs.
Even better is the Silverlight Toolkit. This is by Microsoft, but Open Source (living on Codeplex and with Unit Tests). As it is a separate project it can have a separate release cycle, including experimental components and being updated more frequently than Silverlight itself.
The toolkit includes charting components plus new controls covering styling, layout, and user input.
Since these releases Jimmy Schementi has been far from idle. His latest blog entries track what he has been up to:
Though, for certain scenarios, running scripts in a VB/C# application would be useful. For example, a shopping application that has a bunch of business rules, like "when someone has three items in their cart that all have to do with cooking, give them 10% off." These type of rules can change all the time, and traditionally you'd either store the rules in a database and implement a engine to understand the rules, or hand-code them yourself and have to redeploy the system every time you want to change them. Or, you could save yourself the hassle and store the rules as Python or Ruby code, and then host the DLR in your application to run the code. Want to update the rules? Just update the code, nothing more.
Embedding an IronRuby REPL (interactive interpreter) in a Silverlight application.
This is the good one! Jimmy posted this email to the IronRuby and IronPython mailing lists:
First and foremost, I want to thank anyone who has used the bits on http://codeplex.com/sdlsdk, and accepting my bullshit version of open-source. While getting monthly binaries/sources is nice, it should be about working on the project together ... not just me throwing stuff over the wall to you. That's changing, now ...
Oh, and remember me complaining about the crappy "sdlsdk" name ... well, I'm trying to get rid of that acronym ...
http://github.com/jschementi/agdlr
Above the public repository for the DLR integration in Silverlight. The following post explains what's in there, what's not, what's git, and how to contribute: http://blog.jimmy.schementi.com/2008/11/agdlr-silverlight-dlr-open-source.html
My first order of collaboration is this simple new feature, "console=true": http://blog.jimmy.schementi.com/2008/11/repls-in-silverlight.html. If you like this, please feel free to look at what's been done, and if you want to fix something that doesn't yet work correctly, I won't stop you.
Also, as I mentioned in a previous mail, I want to make the filesystem->XAP/isolatedstorage metaphor stronger, so feel free to experiment with that as well. Over the next week I'll get some website-presence/wiki/etc, and we can run this project up and running. There are still some hurdles I need to clear with getting contributed code back into our internal codebase, and shipping on Codeplex, but there are no problems with keeping things on GitHub for now.
Let me know if there are any question. I know I've been a bit silent on the Silverlight front, but take this as me making it up to you.
The resulting discussion also revealed where the Silverlight development tool Chiron got its name from:
Yep, Ag is Silver … made pretty obvious by my little logo for it.
As far as Chiron. It's a planetoid between Saturn and Uranus. The port that Chiron.exe runs on by default, 2060, is Chiron's "object" number. It was derived from the Cassini ASP.NET Web server that Dmitry Robsman wrote. Cassini was a probe mission to explore the moons of Saturn, and Chiron was initially thought to be a moon of Saturn.
Plus this from Michael Letterle:
More importantly, it's also the name of one of Jonathan Coulton's songs: "Chiron Beta Prime".
Because of this I knew how to pronounce the name :)
In case you've been living under a rock for the last year, Silverlight 2 is a browser plugin from Microsoft. It is similar to Flash (aimed at games, media streaming and rich internet applications) and is cross-platform (Mac OS X and Windows - the officially blessed Linux port Moonlight by the Mono guys is making good progress though) and cross browser (IE 7+, Safari & Firefox 2+). Unlike Flash it can be programmed with a choice of languages, and through the Dynamic Language Runtime it can be programmed in Python, Ruby and Javascript.
Silverlight 2 final is now out, and according to Scott Guthrie has now been installed on over 100 million consumer computers.
Naturally Jimmy Schementi, who maintains the Dynamic Language Support for Silverlight, released an updated version of the Silverlight Dynamic Languages SDK (sucky name - more on this in a bit):
In preparation for my talk at the PyWorks conference I updated my IronPython Web IDE (tool for experimenting with the Silverlight APIs from Python in the browser) and Try Python (interactive Python interpreter in the browser) for Silverlight 2:
Of course if you're interested in building internet applications with Silverlight then you will want the extended controls that come as part of the Visual Studio Tools for Silverlight 2 (this works with Visual Studio Professional or the free Visual Web Developer Express). The assemblies that come with the tools include the data grid, extra controls like the date picker and various other useful APIs.
Even better is the Silverlight Toolkit. This is by Microsoft, but Open Source (living on Codeplex and with Unit Tests). As it is a separate project it can have a separate release cycle, including experimental components and being updated more frequently than Silverlight itself.
The toolkit includes charting components plus new controls covering styling, layout, and user input.
Since these releases Jimmy Schementi has been far from idle. His latest blog entries track what he has been up to:
Though, for certain scenarios, running scripts in a VB/C# application would be useful. For example, a shopping application that has a bunch of business rules, like "when someone has three items in their cart that all have to do with cooking, give them 10% off." These type of rules can change all the time, and traditionally you'd either store the rules in a database and implement a engine to understand the rules, or hand-code them yourself and have to redeploy the system every time you want to change them. Or, you could save yourself the hassle and store the rules as Python or Ruby code, and then host the DLR in your application to run the code. Want to update the rules? Just update the code, nothing more.
Embedding an IronRuby REPL (interactive interpreter) in a Silverlight application.
This is the good one! Jimmy posted this email to the IronRuby and IronPython mailing lists:
First and foremost, I want to thank anyone who has used the bits on http://codeplex.com/sdlsdk, and accepting my bullshit version of open-source. While getting monthly binaries/sources is nice, it should be about working on the project together ... not just me throwing stuff over the wall to you. That's changing, now ...
Oh, and remember me complaining about the crappy "sdlsdk" name ... well, I'm trying to get rid of that acronym ...
http://github.com/jschementi/agdlr
Above the public repository for the DLR integration in Silverlight. The following post explains what's in there, what's not, what's git, and how to contribute: http://blog.jimmy.schementi.com/2008/11/agdlr-silverlight-dlr-open-source.html
My first order of collaboration is this simple new feature, "console=true": http://blog.jimmy.schementi.com/2008/11/repls-in-silverlight.html. If you like this, please feel free to look at what's been done, and if you want to fix something that doesn't yet work correctly, I won't stop you.
Also, as I mentioned in a previous mail, I want to make the filesystem->XAP/isolatedstorage metaphor stronger, so feel free to experiment with that as well. Over the next week I'll get some website-presence/wiki/etc, and we can run this project up and running. There are still some hurdles I need to clear with getting contributed code back into our internal codebase, and shipping on Codeplex, but there are no problems with keeping things on GitHub for now.
Let me know if there are any question. I know I've been a bit silent on the Silverlight front, but take this as me making it up to you.
The resulting discussion also revealed where the Silverlight development tool Chiron got its name from:
Yep, Ag is Silver … made pretty obvious by my little logo for it.
As far as Chiron. It's a planetoid between Saturn and Uranus. The port that Chiron.exe runs on by default, 2060, is Chiron's "object" number. It was derived from the Cassini ASP.NET Web server that Dmitry Robsman wrote. Cassini was a probe mission to explore the moons of Saturn, and Chiron was initially thought to be a moon of Saturn.
Plus this from Michael Letterle:
More importantly, it's also the name of one of Jonathan Coulton's songs: "Chiron Beta Prime".
Because of this I knew how to pronounce the name :)
Tuesday, November 11, 2008
Visio Automation: Three Hello World Samples – C#, F#, and IronPython
Saveen Reddy has posted examples of automating Visio from three different languages - C#, F#, IronPython. The code samples are short, and remarkably similar. The examples all do the same thing: launch Visio 2007, create a new doc, and draw a rectangle with the text “Hello World”.
The IronPython is:
import sys
import clr
import System
clr.AddReference("Microsoft.Office.Interop.Visio")
import Microsoft.Office.Interop.Visio
IVisio = Microsoft.Office.Interop.Visio
visapp = IVisio.ApplicationClass()
doc = visapp.Documents.Add("")
page = visapp.ActivePage
shape = page.DrawRectangle(1, 1, 5, 4)
shape.Text = "Hello World"
The IronPython is:
import sys
import clr
import System
clr.AddReference("Microsoft.Office.Interop.Visio")
import Microsoft.Office.Interop.Visio
IVisio = Microsoft.Office.Interop.Visio
visapp = IVisio.ApplicationClass()
doc = visapp.Documents.Add("")
page = visapp.ActivePage
shape = page.DrawRectangle(1, 1, 5, 4)
shape.Text = "Hello World"
i-Dialogue Now Supports Python
i-Dialogue now supports scripting with Python through IronPython. iDialogue is an SaS (Software-as-Service) system integrated with the Salesforce CRM. They call it a 'Customer Experience Management' system (marketing tools).
Mike Leach was new to Python, and he likes it:
After only a few days of playing with Python, I see now what others have been raving about. Web developers will really enjoy using Python with Dialogue Script:
Mike Leach was new to Python, and he likes it:
After only a few days of playing with Python, I see now what others have been raving about. Web developers will really enjoy using Python with Dialogue Script:
- Cleaner code. Easier to read and manage
- No need for thick IDEs. Too often, an IDE like Eclipse or Visual Studio stands between you and the desired solution. Dialogue Script development is 100% browser-based (yes, it even works in Chrome!)
- Agile business rules management. Work side-by-side with business users and apply business rules directly in web pages
- Dynamic typing
Monday, November 10, 2008
Mixing Static and Dynamic Languages
Kevin Hazzard presented at the Philly Code Camp about mixing DLR (Dynamic Language Runtime) based languages with statically typed languages like C#.
"The gist of this presentation is that it's possible to mix the Dynamic Language Runtime (DLR) into statically-typed, early-bound languages like C# to make them much more flexible. In this talk, I demonstrated how a ShoppingCart being filled with Products can adjust discount rates based on marketing rules written in an external Domain Specific Language (DSL). In this case, my DSL was really just Python. I chose to use Python because the syntax is so simple and clean. It's so light, it doesn't get in the way. It's not a real DSL, of course, but by injecting .NET objects into a ScriptScope on a ScriptRuntime (all DLR hosting terms), the Python syntax acting on those injected types looks an awful lot like a language for managing product discounts."
Slides and code examples are linked to from the blog entry.
UPDATE: Kevin has done a 'November Update' to this presentation, for an upcoming Raleigh Code Camp talk. The new blog entry has screenshots and updated code to download:
I like a couple of comments on this entry:
"Python code is pretty easy to read isn't it."
"One of the most often asked questions I get when talking about IronPython is, "How do I convince my boss to let me write that application in Python?" My answer is, "You don't have to write the whole thing in Python. With the fantastically rich hosting APIs in the .NET Dynamic Language Runtime (DLR), you can write the portions that need dynamism in Python and write the rest in C# or VB.NET."
"The gist of this presentation is that it's possible to mix the Dynamic Language Runtime (DLR) into statically-typed, early-bound languages like C# to make them much more flexible. In this talk, I demonstrated how a ShoppingCart being filled with Products can adjust discount rates based on marketing rules written in an external Domain Specific Language (DSL). In this case, my DSL was really just Python. I chose to use Python because the syntax is so simple and clean. It's so light, it doesn't get in the way. It's not a real DSL, of course, but by injecting .NET objects into a ScriptScope on a ScriptRuntime (all DLR hosting terms), the Python syntax acting on those injected types looks an awful lot like a language for managing product discounts."
Slides and code examples are linked to from the blog entry.
UPDATE: Kevin has done a 'November Update' to this presentation, for an upcoming Raleigh Code Camp talk. The new blog entry has screenshots and updated code to download:
I like a couple of comments on this entry:
"Python code is pretty easy to read isn't it."
"One of the most often asked questions I get when talking about IronPython is, "How do I convince my boss to let me write that application in Python?" My answer is, "You don't have to write the whole thing in Python. With the fantastically rich hosting APIs in the .NET Dynamic Language Runtime (DLR), you can write the portions that need dynamism in Python and write the rest in C# or VB.NET."
GeckoFX (Firefox) WebBrowser Control from IronPython
A Japanese blogger shows off how to use the GeckoFX (Firefox) Windows Forms WebBrowser control from IronPython. This allows you to embed webpages in Windows Forms applications using the Gecko rendering engine.
The example code uses the winforms sample that comes with IronPython:
import sys
sys.path.append("(IronPythonTutorial")
import winforms
from System.Windows.Forms import *
f = Form()
f.Text = "GeckoFX"
f.Show()
import clr
clr.AddReference("Skybound.Gecko.dll")
from Skybound.Gecko import *
Xpcom.Initialize("(Firefox 3")
browser = GeckoWebBrowser()
browser.Parent = f
browser.Dock = DockStyle.Fill
browser.Navigate("about:robots")
The example code uses the winforms sample that comes with IronPython:
import sys
sys.path.append("(IronPythonTutorial")
import winforms
from System.Windows.Forms import *
f = Form()
f.Text = "GeckoFX"
f.Show()
import clr
clr.AddReference("Skybound.Gecko.dll")
from Skybound.Gecko import *
Xpcom.Initialize("(Firefox 3")
browser = GeckoWebBrowser()
browser.Parent = f
browser.Dock = DockStyle.Fill
browser.Navigate("about:robots")
Uploading a File with IronPython
An example of uploading a file to a website from IronPython:
This is an example of a class in IronPython that can upload a file to a web site. This is a port from a codeplex example (except without cookies...but the addition should be trivial). It uses the WebRequest class.
Points to note are: 1) the parameter query_string is of type NameValueCollection, 2) the parameter url must be a valid URI (e.g. www.somewebsite.com/somepage), 3) the parameter content_type can be 'multipart/form-data'.
This is an example of a class in IronPython that can upload a file to a web site. This is a port from a codeplex example (except without cookies...but the addition should be trivial). It uses the WebRequest class.
Points to note are: 1) the parameter query_string is of type NameValueCollection, 2) the parameter url must be a valid URI (e.g. www.somewebsite.com/somepage), 3) the parameter content_type can be 'multipart/form-data'.
The Chemistry Development Kit and IronPython
The Scientific C# blog looks at using another "mature open source cheminformatics api" from C# and IronPython: The java Chemistry Development Kit.
As it is a Java library, using it from .NET is a novel idea... The blog entry shows you how to compile the library for .NET using IKVM, an implementation of Java for .NET.
The command line magic to compile CDK for .NET is:
ikvmc -assembly:cdk_dotnet -target:library yourcdkjar.jar
This produces a .NET assembly. An example of using the library from IronPython:
import clr
clr.AddReference("cdk_dotnet.dll")
clr.AddReference("IKVM.OpenJDK.ClassLibrary.dll")
from org.openscience.cdk import Molecule
from org.openscience.cdk.smiles import SmilesParser
#import the whole package for brevity
from org.openscience.cdk.qsar.descriptors.molecular import *
from org.openscience.cdk.qsar.result import DoubleResult
tpsa = TPSADescriptor()
logP = XLogPDescriptor()
smiles = SmilesParser()
mol = smiles.parseSmiles("N=CCC=O")
dr = tpsa.calculate(mol).getValue()
tpsaVal = dr.doubleValue()
dr = logP.calculate(mol).getValue()
logPVal = dr.doubleValue()
print logPVal,tpsaVal
the output is:
3.181 40.92
As it is a Java library, using it from .NET is a novel idea... The blog entry shows you how to compile the library for .NET using IKVM, an implementation of Java for .NET.
The command line magic to compile CDK for .NET is:
ikvmc -assembly:cdk_dotnet -target:library yourcdkjar.jar
This produces a .NET assembly. An example of using the library from IronPython:
import clr
clr.AddReference("cdk_dotnet.dll")
clr.AddReference("IKVM.OpenJDK.ClassLibrary.dll")
from org.openscience.cdk import Molecule
from org.openscience.cdk.smiles import SmilesParser
#import the whole package for brevity
from org.openscience.cdk.qsar.descriptors.molecular import *
from org.openscience.cdk.qsar.result import DoubleResult
tpsa = TPSADescriptor()
logP = XLogPDescriptor()
smiles = SmilesParser()
mol = smiles.parseSmiles("N=CCC=O")
dr = tpsa.calculate(mol).getValue()
tpsaVal = dr.doubleValue()
dr = logP.calculate(mol).getValue()
logPVal = dr.doubleValue()
print logPVal,tpsaVal
the output is:
3.181 40.92
Sunday, November 09, 2008
Why Use IronPython
My boss has a blog! Even better, he's actually writing entries occasionally. He recently answered a question on the Joel on Software discussion board about whether or not a new company should consider using IronPython:
He cites the advantages of IronPython as:
Another interesting post is his description of how we're transitioning development at Resolver Systems away from being a single team into two. This is due both to growth in the development team and the need for us to do development work for some of our larger customers. The entry includes an example Resolver One spreadsheet that uses an evolutionary algorithm to do scheduling to spread developers across the two teams over time:
The evolutionary algorithm is mainly implemented in IronPython user code in the spreadsheet, using the spreadsheet interface for data entry and displaying the results.
He cites the advantages of IronPython as:
- All of the .NET libraries are available.
- UIs look nice. I’ve never seen a pure traditional Python application that looked good, no matter how advanced its functionality.
- We use a bunch of third-party components - for example, Syncfusion’s Essential Grid - without any problems.
- Reasonably decent multithreading using the .NET libraries - CPython, the normal Python implementation, has the problem of the Global Interpreter Lock, an implementation choice that makes multithreading dodgy at best.
- We can build our GUI in Visual Studio, and then generate C# classes for each dialog, and then subclass them from IronPython to add behaviour. (We never need to look at the generated code.)
- When things go wrong, the CLR debugger works well enough - it’s not perfect, but we’ve never lost a significant amount of time for want of anything better.
Another interesting post is his description of how we're transitioning development at Resolver Systems away from being a single team into two. This is due both to growth in the development team and the need for us to do development work for some of our larger customers. The entry includes an example Resolver One spreadsheet that uses an evolutionary algorithm to do scheduling to spread developers across the two teams over time:
The evolutionary algorithm is mainly implemented in IronPython user code in the spreadsheet, using the spreadsheet interface for data entry and displaying the results.
Functional Testing of Desktop Applications
I've published series of articles on the functional testing of GUI applications. It is based on a talk that I gave at PyCon UK.
The user interface toolkit used for the examples is Windows Forms driven from IronPython. The examples are available for download, and test a small program called "MultiDoc", the example application from chapters 4-7 of IronPython in Action.
The techniques and testing patterns used in the articles are not tied to IronPython at all. The topics and principles discussed are relevant to the testing of all desktop applications, whichever framework you are using (almost...). Topics covered include:
The user interface toolkit used for the examples is Windows Forms driven from IronPython. The examples are available for download, and test a small program called "MultiDoc", the example application from chapters 4-7 of IronPython in Action.
The techniques and testing patterns used in the articles are not tied to IronPython at all. The topics and principles discussed are relevant to the testing of all desktop applications, whichever framework you are using (almost...). Topics covered include:
- The why and how of functional testing - including the processes and infrastructure you need around them
- Basic principles of practical testing
- Common problems and ways to overcome them
Subscribe to:
Posts (Atom)