Sunday, August 31, 2008
Ironclad is a project to get Python C extensions working with IronPython. Although the project was created with a cross-platform implementation in mind, it is currently Windows only. Seo Sanghyeon has just posted the start of a patch for Mono compatibility to the Ironclad mailing list. He is hoping that the Mono port will pass the Ironclad test suite within a week.
More of a script than a project, but still cool. It receives Unix syslog packets and store them as Windows eventlog - in 44 lines of IronPython.
Joseph Jude has started an IronPython project called 'mysecs' to track how much time he spends in different applications. It is a desktop and web web applications to "measure/analyse time spent in a day":
ryPDF Report Tools
Ryan wrote a Python PDF report generation tool in Python (and then lost the source code!). He needed something similar to use from VB.NET, so he rewrote it in IronPython.
This project has been around for a while, but I've never posted about it. It is a "Rich Browser Platform" that provides an extensible "tabbed web browser by default and plugin architecture". There are no releases made but you can checkout from the subversion repository. Although the project is written in C#, the '0.3' code supports IronPython.
You can get it at http://sourceforge.net/projects/adodbapi.
Saturday, August 30, 2008
The home page has screenshots and more details:
FX Composer User Guide (PDF):
"Scripting in FX Composer is typically accomplished using the Python scripting language. The implementation of Python used is IronPython 1 from Microsoft. IronPython supports all of the standard python syntax & library, with the additional advantage that it is implemented in a .NET language. This means that python scripting in FX Composer can call the rest of the FX Composer engine without additional work. It is important to note that none of the core functionality of FX Composer requires Python. The scripting is implemented entirely using a plugin (FXComposer.UI.Python), and in theory any scripting language that could communicate with the FX Composer engine could be used."
The user guide contains details of the FXComposer namespaces and examples of scripting with Python (including hooking into editor functionality like Undo/Redo and creating graphics objects like a torus).
Vista comes with .NET 3 as standard - but IronPython itself only depends on .NET 2 (with Visual Studio 2008 you can set the version of .NET you are targeting when you compile). Specifically, IronPython 2 depends on .NET 2 SP1,which is actually more recent than .NET 3 - Service Pack 1 came out in November 2007.
At Resolver Systems we are selling Resolver One into financial organisations, some of whom are technologically conservative, so it is possible that the requirement for .NET 2.0 SP1 will affect our ability to move to IronPython 2. As IronPython 2 is still in beta it is something we haven't yet evaluated (although we have ported Resolver One to IronPython 2 to test there would be no major problems when we do make the change).
On the other hand, the .NET 2 service pack did fix some bugs on the Common Language Runtime (which is why IronPython depends on it). For example, it means that running IronPython code in an AppDomain with reduced privileges now works.
From the article:
As a concrete example, the following measurements were taken from a single 32-bit Vista machine under .NET 3.5 and .NET 3.5 SP1:
- float comparisons were 37% faster under .NET 3.5 SP1
- nested for loops were 30% faster under .NET 3.5 SP1
- method invocations slowed down 26% under .NET 3.5 SP1
The moral of this story is to always include every machine setup detail you can think of when reporting performance results. Some things you might not expect to impact performance actually do.
Friday, August 29, 2008
IronPython in Action of course). Personally I feel ambivalent about ASP.NET. Its 'page-by-page' model seems to be fine for simple forms, but not a very good way of creating applications. ASP.NET MVC looks worlds better (Rails and Django even more so of course).
On the other hand ASP.NET is massively used (possibly the most widely used web development framework), and people aren't going to switch over to ASP.NET MVC in any great proportion for probably years to come. If they're going to use ASP.NET then they might as well at least use IronPython.
Although ASP.NET standard is still 'supported' (the MVC framework is still in beta), and probably will be for some time to come, Microsoft have been focusing their dynamic language integration on MVC and have been 'quiet' about development of the current IronPython integration into ASP. If more people are vocal about using it this could encourage them to be a bit more public about their plans.
I had an informal meeting with Drew and, what do you know, the Visual Studio Languages Unit has a team called the “Growth Opportunities” (GO) team that is responsible for IronRuby, IronPython and emerging languages. They needed a Lead Program Manager. I took the position and now I am working on a truly great team with folks like John Lam (the face of IronRuby), Harry Pierson (DevHawk blogger, IronPython PM), Jimmy Schmenti (Dynamic languages PM, wrote the Silverline plugin), Tomas Matousek (most awesome IronRuby developer), Dino Viehland (most awesome IronPython developer), Jim Deville (IronRuby QA), Curt Hagenlocher (both IronPython and IronRuby developer), Jim Hugunin (Dynamic Language Runtime and IronPython Architect), Bill Chiles (Dynamic Language Runtime Program Manager) and many, many more."
Thursday, August 28, 2008
In my last post, I posted an implementation of the ViewModel or M-V-VM pattern for use in Silverlight applications. This pattern allows you to decouple your view presentation logic and data from the view, thereby facilitating independent development/design of your app as well as easier unit testing of your application code."
Wednesday, August 27, 2008
"setuptools is a critical piece of the Python ecosystem: it extends distutils, but more importantly it provides easy_install, which is used by almost every Python library and application to handle installation. Making easy_install work in IronPython would make it far easier to download and install Python libraries."
"The good news is that it's not too hard to get going; there are only a couple of bugs that need to be worked around. The bad news is that it can't work with any of the files that are downloaded, as the zlib module is shot."
I wonder if zlib would work with Ironclad? The Bz2 module already works with Ironclad.
William Reade, the main developer of Ironclad, has just announced the release of version 0.5.
- Ironclad 0.5 Announcement
- Ironclad Project Homepage
- Download Ironclad Source and Windows Binaries
- Ironclad Mailing List
There is still a lot of work to do, but this is great progress.
From the announcement:
Our original goal for 0.5 was to import numpy, from IronPython, and do something with it. With one monstrous caveat, we have reached that goal; the problem is that you need to comment out line 532 of numpy.core.numerictypes.py before it'll import. That line reads "_unicodesize = array('u','U1').itemsize", and fails trying to construct a unicode array; however, since _unicodesize appears to be unreferenced, we feel reasonably comfortable ignoring it for now.
Once numpy is imported, you can create integer arrays and do a few things with them; of course, the number of things you can do is still dwarfed by the number of things you can't do. You can add, subtract and multiply integer arrays, and raise them to integer powers... and that's about it.
Ironclad targets IronPython 2 and although the underlying techniques should be cross-platform it currently only builds on Windows. Mono / Linux hackers welcomed!
UPDATE: It turns out that _unicodesize is no longer used, and in response to us asking what it is for it has now been removed...
This blog entry is about the different implementations of Python, which parts of Python are specification and which are implementation details and what it means to talk about the performance of Python in the light of the different implementations.
UPDATE: It turns out that documentation for __reversed__ was added to the Python 2.6 docs in January. However it (currently) incorrectly states that it is new in 2.6 when it was in fact new in Python 2.4.
Tuesday, August 26, 2008
This fourth entry creates a user interface with a stack panel, listbox and button. It creates the UI at runtime by reading in the XAML (an XML dialect for describing WPF UIs) and then hooking up the event handlers.
- Introduction to XAML and WPF with IronPython : Part Four
- Blog entry with links to all four parts of the tutorial
The article is an introduction to Python and IronPython, including the history of IronPython and some examples of Python. It is aimed at C# and VB.NET developers.
On block structure by indentation (lightweight syntax - nice):
"Python’s significant whitespace is one of those things developers tend to either love or hate. However, the use of significant whitespace does appear to becoming more popular. Both Boo and Cobra, two open source .NET languages, use a Python-inspired syntax, including significant whitespace. Furthermore, Microsoft’s new F# language includes an optional lightweight syntax mode that makes indentation significant, just like Python."
On dynamic typing:
"In any system of significant scale, there are essentially an infinite number of ways the application can be wrong. Type safe languages eliminate one of this infinite number of ways. The overwhelming majority of errors and bugs you might make when developing your system won’t be caught by a static type language compiler. If they could, then any application that compiled would automatically be bug free! You need some other mechanism to catch these other errors, typically automated unit tests. So while it’s true that static types do provide a safety net, it’s just not a very big one."
His initial blog entry introduces himself and what he does (plus IronPython related topics he intends to blog about):
Every checkin triggers a benchmark run that takes over six hours (running a combination of PyBench, PyStone and custom microbenchmarks) on custom OS install.
As IronPython 2 has reduced Code DOM support, the Windows Forms designer and code conversion (C# / VB.NET to Python) is currently disabled. Using the forms designer to create forms in C# and then using them from IronPython is easy though.
As IronPython 2 now supports static compilation, SharpDevelop can compile IronPython projects into executables or assemblies. SharpDevelop also has an IronPython console.
Monday, August 25, 2008
One of the basic classes in the hosting API is the ScriptScope. These represent module namespaces, and where name lookups from inside the Python engine are done. As well as creating and manipulating standard namespaces, they can be created with a 'backing store' that allows you to customize name lookup.
The DLR Hosting API blog shows how to use this technique to chain scopes and create a 'pseudo heirarchy'.
Saturday, August 16, 2008
subclasses the VB.NET class, and he can then pull this class out of the IronPython execution scope because he knows what type it is.
This is one of the ways of solving the 'type problem' when working with dynamic languages from statically typed languages like C# and VB.NET. Using dynamic operations exposed by the IronPython runtime you can then create instance of this Python class. It's actually easier than I made it sound; Maurice's example is very clear.
In the next tutorial, we'll look at some dynamic features of Python to reduce code and decrease the time it takes to create a form. You can download both the IronPython and C# source codes below."
Thursday, August 14, 2008
This is done using the .NET 'Reflection.Emit' API to generate the IL (and the lack of this API in .NET Compact Framework is why we can't run IronPython code on the XBox 360 or Windows Mobile devices). It also means that you can use standard .NET debugging tools like 'windbg.exe' to introspect and debug your IronPython code in memory.
In his latest blog entry Curt Hagenlocher takes us through some of the techniques to do just this using a 'noninvasive debugging' technique:
"There are a lot of compilers out there and the number is increasing. This has been very interesting in the last few years because of the development frameworks, which has removed the borders between machine architectures. One of those frameworks is the .NET framework, which is Microsofts own competitor to other frameworks like the Java Platform. The .NET framework was designed to support multiple languages and let them co-exist as equal languages on the same platform sharing the same class and runtime library. There has been a lot of talk about dynamic languages on the CLR (the runtime engine of the .NET framework)."
UPDATE: One of the original Phalanger devs was Tomas Matousek who is now one of the main IronRuby developers. (Thanks to Curt Hagenlocher.)
Tuesday, August 12, 2008
In this tutorial we will be creating a sql table based class for a simple form. It will demonstrate how to create a class and reference it from a form."
"That's it for this tutorial. The next tutorial will go into something practical - creating and updating live data on a form. The code files can be downloaded below. Remember to change the connection string to point to an AdventureWorks database."
Monday, August 11, 2008
There are three new pages on the IronPython Cookbook.
Resolver Systems guys and a few other folk who've made contributions to the IronPython community. If there is someone missing who you think ought to be there, please add them.
a collection of utilities for programmatically controlling SQL 2005 SQL Server Integration Services (SSIS) packages with IronPython".
This simple routine uses the Windows Shell to get MP3 file information from the tags. Due to the differences between Vista and XP different magic numbers are required to get the tags."
The PyCon UK talk schedule is now up and there is an IronPython Tutorial wiki page:
IronPython in Action.
Between us we will have a Linux laptop, a Mac laptop and a Windows laptop and it is our intention to support you through the tutorial whichever operating system you run (so long as you have a recent version of Mono installed).
UPDATE: The wiki page also include Menno's notes on installing Mono 1.9 alongside an older version for Linux users.
The wiki page lists the pre-requisites for the tutorial (what you will need installed), the topics we intend to cover and has space for you to list any topics you are interested in that we haven't included.
_winreg module plus some performance improvements.
Notable changes / bugfixes in this release include:
- nt.unlink will now throw an exception if the file doesn’t exist, as it does in CPython
- The signature of IronPython.Runtime.Operations.Ops.Id() has changed and it now returns an object instead of long. This maps to the id() function in Python. Note that there is no change for Python caused by this and only code which is directly calling Ops.Id from a statically typed language like C# or VB will be affected
- Tuple hashing improvements - if you use tuples as dictionary keys you'll see a performance improvement from this
- Backport fix for compiled regular expressions - more performance improvements - compiled regexes weren't actually compiled previously
- Event handlers can cause circular references and leak memory - Nice to see this fixed, no need to manually unhook event handlers when disposing of UI objects now (hopefully)
The following CodePlex Work Items were also closed:
- 16368 1.1.2: nt.access is missing
- 15105 1.1.2: endpos is zero in IronPython 1.1.1
- 16337 1.1.2: Trivial: Implement float.__lt__(float)
- 16338 1.1.2: Using lambda in class definition will add
- 16342 1.1.2: calling base class __call__ invokes constructor instead
- 16343 1.1.2: problem with __slots__ and __init__ in new-style classes
- 16347 1.1.2: Trivial: popen shouldn't open new window
- 16348 1.1.2: Removes the inexistent file did not throw OSError in IP
- 16350 1.1.2: int() doesn't convert representable longs to int
- 16351 1.1.2: dict.update doesn't take keyword arguments - differs from CPython
- 16353 1.1.2: Trivial: int('0x20', 16) fails to parse, long too
- 16355 1.1.2: unpacking single element tuples in for-statement, listcomp and generator
- 16356 1.1.2: socket.getnameinfo(...) broken under Vista
- 16360 1.1.2: Class with slots and getattr not compatible
- 16363 1.1.2: Can't call method w/ nullable as 1st argument w/ greater than 5 arguments
- 16366 1.1.2: PyCF_DONT_IMPLY_DEDENT support in compile
- 16749 1.1.2 (Trivial): Modifier of PythonEngine.DefaultCompilerContext(..)
Sunday, August 10, 2008
Here are two recent blog entries on IronPython in both Japanese and Chinese. They are both about the new features in 2.0 Beta 4. The first is about the new COM interop features, whilst the second shows off the COM interop (this one has understandable code examples of working with Excel) along with static compilation and an obligatory mention of the inclusion of the Python standard library.
Saturday, August 09, 2008
A new blog has started, called ASP.NET IronPython Resource, which has started a tutorial series on using IronPython with ASP.NET. There are currently two entries. The first gets you started creating new sites with either Visual Studio 2005 or 2008. It creates a "Hello World" application using the visual designer for the webform, and shows the IronPython code behind.
The second entry is a tutorial that shows connecting to a database with databinding to a gridview. The example project is available for download.
Friday, August 08, 2008
Most Python web frameworks speak WSGI (pronounced wizz-gee), including the best known Python web framework Django (which is approaching a full 1.0 release). Jeff Hardy has been developing NWSGI by running Django on IronPython with the Cassini web server.
It seems like Django has become the standard 'compatibility-test' for Python implementations. Both Jython and PyPy have been working on improved Django compatibility performance, and Dino Viehland demonstrated Django on IronPython at this year's PyCon in Chicago.
Jeff has started a blog tracking the progress of Django with NWSGI, including any necessary patches. In the first two entries he describes getting the Django admin panel to run and in the third he talks about databases with IronPython and Django (both SQLite and MS SQL Server):
The fourth blog entry is the announcement of NWSGI 0.4, with a few more details:
The DLR Hosting (and related stuff...) blog has posted some notes on the ScriptScope (and how to use them with the runtime and engine):
Thursday, August 07, 2008
UPDATE: The new 'pyc.py' sample demonstrating static compilation for IronPython 2 is now available for download.
This feature is curently used by Resolver Systems to do binary only distributions of Resolver One. It was initially missing from IronPython 2, and for a while it looked like it wouldn't make it back again - but thankfully it has now returned. This makes the transition for Resolver One from IronPython 1 to IronPython 2 (once it out of beta) an easier one - although the Resolver One codebase is heavily optimised for IronPython 1 and the performance profile of IronPython 2 is quite different, so it will still involve a lot of work.
Srivatsn is on the dynamic language team as a tester, and he also manages some of the releases. In this blog entry he looks at the new static compilation feature; how to use it and what it does.
This is great news, and is the result of Harry Pierson pressing the Microsoft lawyers into submission so that they can make a 'dual-license' release - IronPython under the
Microsoft Public License and the Python standard library under the Python license.
This opens the way to patches to the standard library for IronPython compatibility. FePy (the IronPython Community Edition) has included several of these patches for some time.
Harry has more 'lawyer-time' ahead of him - so that IronPython can accept community contributions and submit patches back to core Python - but this is a great step.
At the moment the entire standard library is included, including modules that can never work because they rely on C extensions. In his latest blog entry Harry talks about the new 'batteries-included' approach for IronPython, and how it might develop.
NWSGI is a .NET implementation of the Python WSGI specification using IronPython. It is implemented as an ASP.NET HttpHandler. The main goal is to provide an easy path for running Python web applications/frameworks (such as Django, Trac, etc.) on IIS7.
This version brings improvements to configuration and performance.
IronPython itself has improved dramatically (kudos to the team for a fantastic job) and is now capable of running parts of Django, although some workarounds are still required. I can make it through the tutorial, and once I get some testing with 2.0b4 done I'll post patches as well.
I'm going to start testing against some other applications as well, and post the results on my blog as well as the compatibility pages on ironpython.info.
Wednesday, August 06, 2008
This is an important release, not just because of the number of new features and bugfixes, but because with it come changes to the way IronPython is packaged and what it is packaged with...
As well as the normal binary / source releases, IronPython 2b4 comes with an msi installer that also includes the Python 2.5 standard library. This is something the community has wanted for a long time, and opens the way for the Microsoft Dynamic Languages team to officially support some of the patches needed to enable some standard library modules to work with IronPython. It also makes IronPython a more 'complete' (and installable) version of Python.
UPDATE: The msi bug has now been fixed.
Significant new features in this release include:
- Static compilation of python files is now supported again and the pyc.py sample has been updated to use the newer version.
- IDispatch-based cominterop is now the default mode as opposed to generating interop assemblies for interacting with COM components. This is a change that might break some existing scripts. One can switch back to the old behavior by setting the environment variable COREDLR_PreferComInteropAssembly. Read this blog for more details on this feature.
- Apart from the Silverlight binaries and Chiron, the templates and tools to create a barebones app are included as well. This is in the Silverlight\Script folder.
- Performance degrades that were reported in Beta 3 have been fixed.
Bugs fixed (45 on codeplex plus 5 reported internally):
- nt.popen(...) breaks where ... consists of cmd.exe commands
- globals().fromkeys(...) broken
- globals().Values enumerator broken
- Trivial: nt.tempnam("", None) broken
- Trivial: Remove exceptions.OverflowWarning
- New-style class dicts (dictproxy) cannot be used for C# methods with IDictionary parameters
- Negative: ipyw.exe run w/o args crashes
- 'dictproxy' object has no attribute 'copy'
- class.__dict__ (dictproxy) doesn't support .get
- repr() of a UserList containing another userList is wrong
- NaN == NaN
- DBNull should implement __nonzero__ that always returns false.
- IP implementation of _sha512 has extra module attributes
- Dir on textbox raises exception
- re.compile(...).match(...) does not permit certain integer pos/endpos keyword parameter values
- re.compile(...).match(...).groupdict() broken WRT to unnamed groups
- re.match(..., ...).regs is nearly always incorrect
- datetime.datetime.fromtimestamp() is not POSIX timestamp
- Trivial: Charset name
- datetime.datetime.isoformat() has no parameter
- nt.fstat(...) broken
- Python25: __hash__ returning longs broken on old style classes WRT CPython 2.5
- Trivial: IronPython converts -0x80000000 to a long (should be int)
- -X:TabCompletion broken in IP2.0A1 WRT simple variable expansion
- import socket
- TypeError: staticmethod is not callable when overriding __new__ in old+new derived type
- using the PropertyGrid class - SelectedObject issue
- new.classobj fails with "TypeError: __bases__ items must be classes (got type)"
- dict(...) broken on new style class dictionaries (dictproxy)
- IronPython should not implement weakref.proxy.__eq__
- nt.stat(someDirectory) is broken
- Trivial: Implement 'file(...).mode'
- Trivial: os.chmod followed by os.unlink fails
- os.umask does not exist
- Regular Expression incompatibility
- co_filename in code objects is None
- Trivial: help(clr.AddReference) not helpful
- tb_line member of Traceback objects is incorrect
- Valid regex under CPython will fail with IronPython
- time.timezone has different semantic than in cpython
- IronPython 2.0B2 cannot iterate over an IEnumerable returned from a C# method using yield
- Implement os.system (nt.system)
- cStringIO 1-arg .read() and .readlines() broken
- Trivial: id should return int or long, not Int64
- misc string related bugs from c-python25 test
- Implement warnings for -Qwarn and -Qwarnall
- StackOverflowException when exec "ao==7" for Ironpython, but Cpython can work
- Helper stubs shown in Python stack traces
Tuesday, August 05, 2008
He has already covered the basics, how to access a winforms application that we want to test, run it in a separate thread, and simulate user input. In this entry he takes a more methodical look at some of the components involved in testing a winforms application and starts to build up a testing framework (source available):
Monday, August 04, 2008
- IronPython at Resolver Systems (Jonathan's Blog entry)
- IronPython at Resolver Systems: Python Learns New Tricks
Resolver Systems is a two year-old start-up based on the outskirts of London's financial district. Our first product, Resolver One, is a desktop and web-accessible spreadsheet aimed primarily at the financial services market. It is written entirely in IronPython, and directly exposes end-users to Python, both as expressions in cell formulae, and as an embedded scripting language which provides programmatic access to spreadsheet content.
Resolver One launched in Spring of 2008, by which time the team had grown to fourteen people, and the codebase to 40k lines of product and 140k lines of tests.
From the outset, it was clear that it would be good to use an existing language as our embedded scripting language, rather than inventing one. A dynamic language would confer appropriate type semantics on the spreadsheet formulae...
As the core Python developers will be maintaining Python 2.X and 3.X in tandem for a while, whilst the community transitions over to Python 3, I asked if IronPython would do the same - or whether IronPython 2 would be left behind once work on IronPython 3 begins. Another commentor, Francois, was keen to know whether the Visual Studio integration would require users to purchase Visual Studio - or whether they would be able to use the Express version or the VSx Shell as is the case with IronPythonStudio.
Harry answers both these questions in his roadmap clarifications blog entry:
Sunday, August 03, 2008
The only magic here is in some dynamic code generation that my ProxyForWsdl class does by downloading the WSDL contract and building classes for services, operations and data contracts.
Python scripts making use of this magic can be embedded into C# applications. The blog entry shows screenshots and example scripts, plus an example that embeds IronPython in C#. All the code is downloadable.
should be doing is writing the whole application in IronPython...
Saturday, August 02, 2008
Not only that, but the IronPython interactive interpreter makes it very easy to manually test C# assemblies - creating instances and calling methods of your .NET classes.
James McCaffrey demonstrates IronPython for 'ad-hoc' testing of some classes from a poker library. He emphasises the value of Python for automated testing.
Friday, August 01, 2008
In previous versions of IronPython trying to doing this without the Primary Interop Assemblies available would try to generate them on the fly (which could take several minutes). But with IronPython 2.0 BEta 4 it should work fine through IDispatch. Shri provides some details of how it works in practise, how it works under the hood, and some of the known issues (including how to turn it off if it causes any problems!).
He has two blog entries already. The first explores stack overflow in IronPython (which is a catchable error in Python), and how it maps to the underlying Common Language Runtime (where it is generally an unrecoverable error). By default IronPython doesn't catch stack overflows (sys.recursionlimit is set to MaxInt), because there is a performance overhead in monitoring the frame depth to catch the exception. Curt explores the issues around this for IronPython.
This is part of what it means to be dynamic. Take the power and use it wisely.