Pumping Iron: Dynamic Languages on .NET
The esteemed Mr Tartley recently stepped into the shoes of Harry Pierson (Microsoft IronPython PM) and spoke about dynamic languages on the .NET framework at the Øredev conference in Malmö, Sweden.
He has posted the slides and notes online, based on Harry's presentation but delivered with Jonathan's irrepressible style (and with the audience's reaction inline).
The talk is an exploration of the differences between static and dynamically typed languages, plus some of the specific language features of Python and IronPython.
My favourite part is where he discusses the cost of compile time checking. Those who advocate the benefits of statically typed languages often discuss what losing compile time checks costs, what they don't talk about so often is what keeping them costs:
"So let’s examine what we’re really talking about losing when we have no compile-time type checks. As we’ve noted, type safety really does catch and eliminate a whole class of errors, namely whenever you perform an operation that expects one data type, and you give it another. However, and importantly, this is not remotely sufficient to have any confidence that a program actually works, just because it compiles. There are many other classes of error that can and do commonly happen.
Jay Fields has a marvellous blog post about this, in which he hypothesises a function performing integer addition. Now, static type checking can confirm that all uses of your function pass it integers, and that the return value is always an integer. Obviously this is not sufficient to assure the function behaves correctly. Nowadays, when we want to be sure about the behaviour of something like this, we write unit tests.
Out of interest, how many people here are using unit tests? [almost every hand goes up] Alright, excellent, of course you do, you’re all professional, and it’s become mandatory, right, because they are so great. So absolutely everybody wants to be using unit tests, no matter if you’re using a static language or a dynamic language. And your unit tests get to be very detailed about the runtime behaviour of your functions, so they don’t simply check that your function is returning an integer, they are testing that 1+1 actually returns 2. Now, 2 is an integer, right? So once these tests are in place, the static type check is now redundant. It is providing no value, and can safely be removed.
Cue sharp intake of breath from everyone in the room currently working on compilers.
Interestingly, when I first started working with Python, I hadn’t used any dynamic language before that, and I was concerned about this myself. I imagined development could be inundated by multiplying bugs which we otherwise would have caught cheaply, at compile time. And this has an element of truth in it - we do find that we get some bugs which normally would have been flagged by the compiler. But it turns out, in practice, these errors are only a very small proportion of all the bugs that any application contains. And also, they are also amongst the most superficial. We passed in function parameters in the wrong order. We supplied a string when we should have used an integer. These are the sort of bugs which are generally trivially easy to locate, identify and fix. They are not the sort of errors that require hours to track down, or make you belatedly realise that you need to redesign the whole project. They are often simple typos, revealed immediately, with a line number, in stack trace that happens the second you run your application. In short, the actual cost of losing type safety is, surprisingly, actually negligible in practice.
People find this hard to accept. It surprised me. So much effort is expended maintaining type safety, it’s such a pervasive feature of almost everything we do in a static language, that it seems inconceivable we could live without it. But interestingly, once we notice how small the actual benefits of type safety are, the other thing you start to notice almost immediately afterwards, is that type safety also comes with a bunch of costs. And these costs are always overlooked, because type safety is so pervasive that it has become invisible, nobody even thinks about it when they have to jump through hoops to maintain it. It’s only when you consider not doing, by using a language that doesn’t enforce it, that you start to realise how huge the costs actually are.
These costs come in two forms, both the direct costs of having to jump through hoops, and also indirect ones, where the restrictions introduced in the name of type safety will fence off ways of doing things that are simply made impossible in a static language."
UPDATE: Comments from the reddit thread that I thought worth posting.
He has posted the slides and notes online, based on Harry's presentation but delivered with Jonathan's irrepressible style (and with the audience's reaction inline).
The talk is an exploration of the differences between static and dynamically typed languages, plus some of the specific language features of Python and IronPython.
My favourite part is where he discusses the cost of compile time checking. Those who advocate the benefits of statically typed languages often discuss what losing compile time checks costs, what they don't talk about so often is what keeping them costs:
"So let’s examine what we’re really talking about losing when we have no compile-time type checks. As we’ve noted, type safety really does catch and eliminate a whole class of errors, namely whenever you perform an operation that expects one data type, and you give it another. However, and importantly, this is not remotely sufficient to have any confidence that a program actually works, just because it compiles. There are many other classes of error that can and do commonly happen.
Jay Fields has a marvellous blog post about this, in which he hypothesises a function performing integer addition. Now, static type checking can confirm that all uses of your function pass it integers, and that the return value is always an integer. Obviously this is not sufficient to assure the function behaves correctly. Nowadays, when we want to be sure about the behaviour of something like this, we write unit tests.
Out of interest, how many people here are using unit tests? [almost every hand goes up] Alright, excellent, of course you do, you’re all professional, and it’s become mandatory, right, because they are so great. So absolutely everybody wants to be using unit tests, no matter if you’re using a static language or a dynamic language. And your unit tests get to be very detailed about the runtime behaviour of your functions, so they don’t simply check that your function is returning an integer, they are testing that 1+1 actually returns 2. Now, 2 is an integer, right? So once these tests are in place, the static type check is now redundant. It is providing no value, and can safely be removed.
Cue sharp intake of breath from everyone in the room currently working on compilers.
Interestingly, when I first started working with Python, I hadn’t used any dynamic language before that, and I was concerned about this myself. I imagined development could be inundated by multiplying bugs which we otherwise would have caught cheaply, at compile time. And this has an element of truth in it - we do find that we get some bugs which normally would have been flagged by the compiler. But it turns out, in practice, these errors are only a very small proportion of all the bugs that any application contains. And also, they are also amongst the most superficial. We passed in function parameters in the wrong order. We supplied a string when we should have used an integer. These are the sort of bugs which are generally trivially easy to locate, identify and fix. They are not the sort of errors that require hours to track down, or make you belatedly realise that you need to redesign the whole project. They are often simple typos, revealed immediately, with a line number, in stack trace that happens the second you run your application. In short, the actual cost of losing type safety is, surprisingly, actually negligible in practice.
People find this hard to accept. It surprised me. So much effort is expended maintaining type safety, it’s such a pervasive feature of almost everything we do in a static language, that it seems inconceivable we could live without it. But interestingly, once we notice how small the actual benefits of type safety are, the other thing you start to notice almost immediately afterwards, is that type safety also comes with a bunch of costs. And these costs are always overlooked, because type safety is so pervasive that it has become invisible, nobody even thinks about it when they have to jump through hoops to maintain it. It’s only when you consider not doing, by using a language that doesn’t enforce it, that you start to realise how huge the costs actually are.
These costs come in two forms, both the direct costs of having to jump through hoops, and also indirect ones, where the restrictions introduced in the name of type safety will fence off ways of doing things that are simply made impossible in a static language."
UPDATE: Comments from the reddit thread that I thought worth posting.
"your unit tests get to be very detailed about the runtime behaviour of your functions... So once these tests are in place, the static type check is now redundant. It is providing no value, and can safely be removed"My response:
The last statement is simply false: If you remove the type checker, you actually have to add more tests. You cannot "safely remove" the type checker just because you have unit tests in a statically typed language.
Actually your statement is false for another reason - duck typing.
When you test in a dynamic language what you largely test for is that the code performs the correct operations on your objects - and so what type they are is irrelevant so long as they are used in the correct way. This is where the added flexibility of dynamic languages come from.
If your code performs the correct operations then it must behave correctly when passed the correct types (and of course you test that it is called in the correct way - but again not necessarily specifically caring about types).
Programming in dynamic languages is simply a different mindset to programming in statically typed languages - which is why the two camps disagree so much. Caring about type safety is not something that matters a great deal to dynamic language programmers, what matters is the way their objects are used.
Hi Fuzzyman,
ReplyDeleteUnfortunately you come across as too much of a dynamic language advocate in this otherwise well-written article to shift any of the static advocates methinks.
I was thinking that what might persuade is much more testimony from noted static language programmers that things don't fall apart when using a dynamic language and tests.
- Paddy.
P.S. Have a happy new year.
I'm sure you're right. However all I'm doing is quoting the original article by Jonathan - who in turn 'borrowed' most of the material from Harry Pierson.
ReplyDeleteBoth of them are dynamic typing enthusiasts who previously almost exclusively used statically typed programming languages. Hopefully those who click through will find a bit more of a balanced approach. :-)