Tuesday, September 30, 2008
He posts some example code with tests using the unittest module:
This example should illustrate that we can write some unit tests for our Python code fairly easily and quickly. It's nice to have a built-in framework for unit testing. There is also a "doctest" module available for doing more like system testing."
"I had the idea of having a server which will act as a centralized data repository - laptop/desktop tool, mobile tool should send data to the server and data will be analyzed in the server."
"Next steps? Incorporate with Google Charts to provide some neat analytics reports, so that it really becomes an useful tool."
Friday, September 26, 2008
- Python list comprehensions can do the same as LINQ for simple cases, but LINQ is much more powerful, and it supports deferred execution, while list comprehensions are evaluated greedily. (Ed: generator expressions are evaluated lazily in Python)
- Every description of the python syntax I ever see emphasizes the fact you don't need to put in braces. Pity they don't spend more time telling you that you have to put in colons, that would actually be useful knowledge. This really bit me when I learnt boo, which is syntactically very similar.
- IronPython 1.0 targets CPython 2.4. This sounds fine until you realize that this was released in 2004. A fair bit has happened since then, not limited to the introduction of an inline if syntax. (Ed: IronPython 2 is Python 2.5 which is still the stable current version of Python)
- While we're on the subject, the inline if (defaultValue if condition else exceptionValue) is actually quite cool.
- The fairly lax approach to types means I don't need to call .Cast
- Tabs are the devil as far as indents in Boo and Python are concerned. I highly recommend using something that shows tabs explicitly, and then eliminating the lot.
- Instance methods that explicitly take self feels awkward to me.
- Static Methods that require an incantation such as "ToAbsoluteUrl = staticmethod(ToAbsoluteUrl)" also feels pretty awkward. (Ed: staticmethod can be used as a decorator which is much nicer)
- The casting to delegate isn't quite as slick as it is in C#. Passing in "spiderTracker.FoundUrl" instead of "lambda url: spiderTracker.FoundUrl(url)" results in an extremely unhelpful runtime error.
- The lambda syntax is pretty elegant, but so is C#3's. Indeed, C#3 seems to have the edge.
- Python's regular expressions are powerful, but not quite as good as .NET's. In particular search does what you'd expect match to do
- findall doesn't return matches. Rather, it returns the value of the first group of the matches. This is often actually what you wanted, but it's a bit peculiar. The semantics of "Matches" in .NET are much easier to understand.
- It did seem rather slow compared to the C# version. There are, of course, way too many variables here to make a final judgement, but it was disappointing.
My conclusions from this are:
- If it's not tested it doesn't work. And when you change the environment under which your code runs you need to retest.
- The unit tests paid off again, since they allowed me to find many problems in places I didn't expect.
- I need to do more to flush out these hidden assumptions. One way is to add a randomized test that runs over longer periods and plays with some variables. I could easily randomize the order in which I go over the checks, inject random errors (the system is supposed to be self healing) or delays in some strategic points, etc. The not-so-easy part is verifying the system behaved correctly, and being able to reproduce problems once they surface.
- Would be really good if I had code coverage tools. Don't think there's anything available for IPy :-(
- This happens every time I port non-trivial code. Need to stop being surprised :-)
Thursday, September 25, 2008
This question sparks a good discussion and is relevant to many other Python projects and applications:
One of the topics on my mind recently, as we consider refactoring the core container, is how we ensure that when we introduce internal dependencies, such as possibly the Amara libraries for XML processing, what secondary constraints we might be introducing."
"For example, Amara represents a very pythonic and therefore simple to read and use approach to developing XML processing, such as our very own XML application configuration parsers. So from that perspective, things are very attractive. But then you ask the question, "can I run this on another python platform, such as IronPython and Jython?" and things get more constraining."
"However, my question is "is this important, in this case?" It's a general question for the Spring Python community really. Should we attempt to support the execution of SP applications on other platforms than CPython?"
"My feelings right now are that it is unlikely that a Spring Python application developer would expect to take their entire application as is and expect to run it on a different platform, such as .NET or Jython in the JVM. If I was going to do that, I would want to take advantage of the bits and pieces that are there in the .NET and Java worlds, and that would mean a port. It's also unlikely that I would necessarily get a business requirement to shift platforms like that... but I'm open to arguments there."
"My take is that Spring.NET provides Spring for the .NET community. Spring Java, well, it does the same for the Java community. Spring Python is doing the same for the Python community. The programming model and approach is portable, but it's not expected that the actual codebase is. Think of this as the concepts are the same, but exactly how you implement them would take full advantage of the platform you are on."
"So is this fair? This is where I need the community to chime in. Are my assumptions here wrong? Is there a defined need, special to the Python community, to come up with a framework that is truly portable at the implementation level, or is the goal really to create a framework that helps us use Spring concepts in the Python world. It's a BIG question, because it informs many of the core containers development choices, hence I thought it was time to get it out there for everyone to have their say."
Starting with two new pages on the IronPython Cookbook:
Steve Gilham has been doing some odd things. Notably creating Java libraries in Java and Scala and then running them on .NET using the IKVM cross-compiler. I haven't been tracking them on this blog because most of the experiments have only been tangentially related to IronPython, but his latest entry is interesting. Here he is writing Scala (a very statically typed language for the Java Virtual Machine) code and using it from IronPython:
why?... it's still cool though.
UPDATE: In a new post Steve attempts to answer my question as to why he is doing this (and corrects my assumption that he is using IKVM to cross-compile Scala): Why am I doing this sort of thing?
Mitch Barnett has produced a promotional video (complete with rousing soundtrack) for his web based IronPython / DLR IDE the Global System Builder:
"I’m impressed that C# and IronPython can interact like this. The possibilities for extending applications with runtime dynamic code are intriguing - worth looking into."
Wednesday, September 24, 2008
NWSGI is a .NET implementation of the Python Web Server Gateway Interface specification. It is implemented as an ASP.NET HttpHandler with the goal of allowing you to run Python web applications / frameworks (such as Django, Trac etc) with the IIS 7 web server.
This latest release is version 0.5 and has a couple of new features and breaking changes. Jeff Hardy announces and explains the release on his blog:
"I'm working on building an IIS7 admin interface for all of this; it's a bit of a mess right now because the advanced UI tricks aren't really documented that well. I'm hoping to have the IIS7 admin and the installer ready by the time IronPython goes 2.0."
yet Controllers or Models.
There are samples plus documentation available and Jimmy's blog entry walks through creating an IronPython webforms site with Visual Studio.
Phil Haack, who is now PM for the ASP.NET Dynamic Language Support along with ASP.NET MVC, has also blogged about the new release:
At least one Python blogger is excited by the news. Alcides Fonseca says:
Tuesday, September 23, 2008
def f(*args, **keywargs):
The arguments are collected as a tuple and a dictionary. C# can collect an arbitrary number of arguments to a method (no messy functions in this language!) as an array (with the params keyword) - but it has no concept of keyword arguments let alone collecting them as a dictionary.
In this blog entry Srivatsn (IronPython tester) introduces the 'ParamDictionaryAttribute' that is part of the Dynamic Language Runtime ane explains how to use it to write C# classes that can take arbitrary number of keyword arguments:
"I wanted to explore this scripting for Perenthia because I want to be able to add dynamic functions or scripts to objects. I wrote this sample to test out using C# as base classes for IronPython classes and passing operations back and forth between the two. As such the source is kind of loose but the concepts are demonstrated.
What I have is a basic framework of C# classes that define an Actor (any object in the game), a Client (a connected player), a Game (the logic of the game) and a Server (for handling HTTP communication, etc.). The Actor, Client and Game classes also server as the base classes for IronPython classes or Creatures, NPCs, Players and Rooms."
In his blog entry he shows some of the C# classes and how they are used from IronPython.
Monday, September 22, 2008
At the heart of the Dynamic Language Runtime are expression trees - the Abstract Syntax Trees of DLR languages. Another part of the .NET framework (at least since .NET 3.5) is LINQ - Language Integrated Query. LINQ makes querying a first class language concept. In its most basic form (over objects) it resembles Python list comprehensions and generator expressions. With the right providers you can also use LINQ to query XML and databases - with your LINQ expressions being 'compiled' to SQL or an appropriate query language.
LINQ expression trees represent query expressions. In IronPython 2.0 Beta 4 the DLR expression trees merged with LINQ expression trees and were moved into the same namespace.
Unfortunately this caused name collisions all over the shop and made it virtually impossible to use IronPython or the DLR in projects that also used .NET 3.5 (particularly for ASP.NET projects). To resolve this problem the names have been moved again.
Harry Pierson explains the issues and the solutions:
summarises it more succintly:
"The recent beta release of the DLR allows the usage of LINQ and other CSharp 3.0 features in an application that hosts the DLR."
The first change is that a significant portion of the hosting API has changed - and for some (but not all) use cases got a lot simpler.
Embedding Dynamic Language Runtime engines (what we are discussing here is the DLR hosting API) uses the App.Config file to configure the languages they intend to use. This part is probably more complicated than before. The DLR Hosting blog has a self-contained Visual Studio 2008 sample demonstrating how it works:
Python. You can create a ScriptRuntime (one of the core hosting classes) without using the config system.
The DLR Hosting blog shows the basics:
- DLR Hosting API : How to create a ScriptRuntime object without using App.Config files using language specific hosting API
Sunday, September 21, 2008
another blog entry, he comments on working with Windows Forms from IronPython:
"One thing I did notice whilst playing with IronPython and System.Windows.Forms was how relevant any S.W.F. experience you already have is likely to be. Yes, you’ll be writing a lot less code, but apart from the fact you’re using a much nicer language, the way that the UI works is obviously the same - which for some reason wasn’t as obvious to me as it should have been."
If you've ever wondered what the dynamic language team members look like, Dave has some photos of them from a lunch meeting with the 'brain-trust':
on the Board of Directors of INETA (International .NET Association) which works to coordinate a huge community of .NET user groups".
INETA is in touch with a large community of .NET developers:
"It appears INETA has great momentum, with greater than 250 active user groups and over 100,000 active developers in North America, and a reach of approximately 1.2 million developers world-wide."
"One question we asked was how important Mike saw Visual Studio Integration for the Iron languages. We get feedback at times that it is more important to have a lightweight, quick to bring up, quick to close down, editor for script editing than full VS integration. Others tell us, make sure that we integrate well with VS now since this is their primary code editing tool. Mike’s response was that VS integration is “a big deal” for adoption by the mainstream .NET developer and consequently a must have."
Saturday, September 20, 2008
"In short, I didn’t come to a conclusion.
But I did have some fun. I played with the thought that using python’s native dictionary initialization syntax would be nice as a configuration file syntax (e.g. ini files).
So I whipped a sample project that would embed both Boo and IronPython and looked at the differences with usage. So far I’m liking IronPython better.
This example is ridiculously simple, but you can go ahead and tinker with the engines, its a good timekiller."
The IronPython issue tracker is hosted on Codeplex. The tracker is triaged weekly to review new bugs and Dave explains the different factors involved in assigning bugs to releases.
Getting Your IronPython Bugs Fixed Quickly
- Use the bug template when reporting on CodePlex
- Emphasize if the bug blocks an application
- Email the list as well
- Provide the simplest repro possible
Friday, September 19, 2008
IronPython Desktop Application will pass userid, password to AppEngine Application, which in turn will validate with Google Accounts Authentication service. If authenticated, AppEngine application will proceed; else it will pass the error back to the desktop application.
This is an IronPython example of using .NET classes to load a XML document, query it with a simple XPath expression, and then using the results to transfer data from an Oracle to a SQL Server database.
Thursday, September 18, 2008
Resolver One is a spreadsheet development environment created by Resolver Systems and built on IronPython. Version 1.2 was released not long ago.
This includes several major new features, including one called 'RunWorkbook' that greatly helps with our goal of making spreadsheet system design modular, maintainable and resusable.
Part of our plan for Resolver One includes being able to use numpy (a C extension). At PyCon UK William Reade demonstrated working with numpy arrays containing one million items from inside Resolver One. This was using an experimental version of Resolver One ported to run on IronPython 2, along with Ironclad - our open source project that allows you to use Python C extensions from IronPython.
The Resolver Systems blog includes links to the slides for Giles Thomas's PyCon UK talk and William's lightning talk showing numpy with Resolver One.
From the announcement by William:
I'm very happy to announce the release of Ironclad v0.6, which targets the recent release of IronPython 2.0b5. It's now possible to import the current release candidate for NumPy 1.2, and some of it actually works. Yay!
If you're interested, please download it and try it out; I'm very keen to hear everyone's experiences, both with NumPy and with other C extensions. I'm fully prepared for the fact that most of these will take the form of extremely short narratives along the lines of "I couldn't import foo", or "I did X, and then Y, and then it gave me the following error", but the project is now mature enough that it's *possible* that someone will say "I imported bar, and it kinda worked". That would, of course, make me tremendously happy, so I feel it's worth a try. In the absence of outside feedback, I will continue to work on NumPy; I get the impression that matrices would be widely useful, so they're my next target.
If you're interested in using numpy from IronPython, then download the latest release and tell us which bits don't work!
IronPython 2 Beta 5 is intended to be the last beta before the release of IronPython 2 final (although there will probably be at least one release candidate) - so if you want bugs fixing now is the time to test and report them!
IronPython.Hosting.Python' that makes a lot of the common patterns for embedding IronPython a lot simpler.
It also includes some namespace moves to avoid clashing with .NET 3.5.
Over 100 codeplex work items (plus other bugs reported internally at Microsoft) were closed between beta 4 and beta 5. As this number includes items closed as 'duplicate' or 'by design' the actual number of bugs fixed is not reported... For the full list see the release notes, and if you're embedding IronPython then get ready to change your code again - but hopefully for the last time for a while.
Wednesday, September 10, 2008
At the same time a new convenience class has been introduced making it much simpler to work with an embedded IronPython engine.
The new spec gives all the details and the blog entry demonstrates initializing an 'arbitrary' language through XML configuration.
Tuesday, September 09, 2008
Scott Yang liked harry Pierson't talk at TechEd:
The final show today was the highlight — “Pumping Iron: Dynamic Languages on .NET” where Harry Pierson talked about the benefit of dynamic typing languages and why they are useful to .NET users. Great presentation, and great arguments on why the lack of compile time checking is NOT really an issue. IronPython was used in the code example, which is something I am a bit familiar with.
Jamie Penney's notes on the same talk:
WEB305: Pumping Iron: Dynamic Languages on .Net - Harry Pierson
- Tradeoff between Type Safety and Flexibility
- Rails ActiveRecord example - AR adds the column names of a table as properties on the model class at run time.
- Real products are being shipped on IronPython - Resolver One
- Dynamic languages are very productive - good for initial work on greenfield projects.
- Dynamic languages are not as fast as Static languages, so optimise by replacing slow parts of Iron(Ruby/Python) with C#. This is easy in the Iron* languages.
- Dynamic Languages are easy to teach and learn.
- Short on Ceremony - You don't have to add extra stuff like package management or class creation to python if you don't want to.
- Formatting and whitespace matters. All Python code looks the same, so it is easy to start working with someone else's code.
- Dynamic languages are powerful.
- AOP is easy in Python - it is possible to pass functions around as they are first level citizens, or you can use decorators to wrap functions.
- You can create new types at runtime - example given was an XML to Python loader, which created Python objects from any given XML document.
- Iron* languages are first class .Net citizens.
- Interop with other .Net code. Easy to use C# or VB.Net code from Iron* languages, but it is harder to go the other way. Standard .Net assemblies just work in Iron.
- Iron languages are embeddable by using the DLR.
- Microsoft are working on integration with Visual Studio currently, due in 2009.
- IronRuby and IronPython are both true Open Source.
A report on dynamic languages from day 2 of the Microsoft Technology Summit. This is from a talk given by John Lam:
Dynamic Languages @ Microsoft
John Lam, Senior Program Manager for Visual Studio Managed Languages discussed the Dynamic Language Runtime. In response to his own rhetorical question asking why Microsoft would support dynamic and open source languages on the .NET runtime, he listed the following reasons:
- increased opportunity to get Microsoft technology used
- clear trend to build stuff on OSS
- developers want freedom to create stuff
- developers want freedom to see the source code
The Microsoft Permissive License got changed to Microsoft Public License(MSPL), and got OSI approval.
The Dynamic Language Runtime(DLR) was refactored from the IronPython implementation. Common code and services that were needed for dynamic languages was extracted into a reusable framework.
Because the DLR is distributed as MSPL, anyone can port to other platforms. So assemblies can run on Mono without changes.
Microsoft is hoping to have Rails running on the DLR by RailsConf 08. John demoed Django today, but acknowledged having to make a few changes to the Django source to do so. He also admitted that both IronRuby and IronPython have performance issues right now.
During the tutorial we will work through creating a windows forms based Twitter client called Twatter. There are more details on my blog, along with a screenshot of it running on the Mac and a screenshot of the example application for the early chapters of IronPython in Action.
Thursday, September 04, 2008
- Windows Forms
- Web services and network access
- Handling XML