Programming, Software and Code

Bright Blue Yonder

19 May 2010
legacy  —  Parrot
Yesterday I released 2.4, "Sulfur Crest". It's a cooler, more succinct name than "Sulfur-Crested Cockatoo". Every time I do a Parrot release I find myself searching through Wikipedia to look for interesting Parrot names (and colorful pictures too!).

I picked an interesting quote for the release announcement. Amid the broken grammar and colorful imagery was a key point that I wanted to express: Parrot is getting smarter.

If you had told me two years ago, or even a year ago, that Parrot would have the features it has now, or that it would be on the verge of adding the features that we are currently planning, I might have been incredulous. Parrot has matured a lot recently, and I really think all that effort is going to start paying off.

Parrot had a lot of features when I first joined the project. Many of these were first drafts and prototypes which have since been ripped out or reimplemented. This is a normal, natural, healthy part of the software development process, and nobody should be surprised or upset about it. Pop quiz: Name for me one other dynamic language virtual machine, open-source or otherwise, that aims to support the same number of programming languages and runtime environments that Parrot aims to support? Smalltalk comes to mind as a virtual machine for a dynamic language, but that VM backend was never intended to support anything besides Smalltalk. At least, not well. .NET supports a few languages, but is certainly not dynamic.

There's a great rule in programming that we always throw the first one away. The first draft is never the final manuscript, the first stab in the dark never hits the target. The original Parrot designers and developers didn't have a whole hell of a lot of prior art to copy from. They made some guesses, but there was no way they could have gotten everything right the first time, no way they could have forseen all the problems that the project would run into over time. But that lack of information and that uncertainty can't hold people back. You grit your teeth, you write a prototype, and you resolve to yourself that you always throw the first one away.

Let's take JIT for an example. In a basic JIT system, you have a snippet of code with a runtime cost of E. A JIT attempts to compile that code into a piece of machine code with a runtime:

OE < E

Where O is some sort of optimization factor. Of course, there is some kind of overhead H, in compiling the code at runtime:

H + OE < E

H involves constructing the machine code and performing optimizations. The more optimizations we perform, the larger H gets, but the smaller O can get. If E is large enough and if we tune H properly, eventually we cross a threshold and the left side of the inequality becomes smaller than the right side of the equation. At this point, JIT is a net performance win for the application. Simple, right?

Anyway, I'll skip the rest of the theory lesson. The point is this: Parrot's original JIT didn't really have anything a modern compiler system would recognize as "optimization passes", so it wasn't really tunable. It also didn't readily support code generation on any platform besides x86. These things weren't going to be easy (or, even possible) to add either. When the burden of maintenance became too high, and when it was clear that the old prototype system was doing more harm than good, we ripped it out.

There were a number of such systems in early Parrot, prototype code that got Parrot up to the first plateau but needed to be redesigned and reimplemented for Parrot to move up to the next one.  There are several projects either going on right now, or about to get started to work on these systems. When you look at the list, you realize that while it's ambitious, with the current team and the current state of the VM it is entirely plausible that they will all succeed. A quick overview:

Allison, Bacek and chromatic are talking about several ideas for implementing new, more efficient GC algorithms to replace our first GC prototype.

Plobsing has been hard at work redoing our freeze/thaw serialization system and, along with NotFound and others is working to fix several long-standing bugs in the PBC system.

Nat Tuck is preparing a GSoC project to implement a new threading system to replace the first prototype version of that system.

Tyler Curtis is preparing a GSoC project to implement a first prototype of a PAST optimization system.

Daniel Arbelo is preparing his GSoC project to implement a first prototype of the NFG string normalization form.

Muhd Khairul Syamil Hashim Is putting together a GSoC project to implement an instrumentation framework for Parrot so we can get some high-quality analysis and debugging tools for Parrot.

John Harrison is preparing his GSoC project to develop a new NCI frame builder with LLVM, which will replace the old prototype frame builder and (hopefully) lay the groundwork to start replacing our prototype JIT system.

...and there are even other projects that I can't think of off the top of my head right now too.

If you had told me a year or two ago that all these projects would be on the table, or that we would have such high chances of them all succeeding, I would have called you crazy. But after all the work we've put in between then and now, and considering the high caliber of our development team, I'm feeling pretty confident that we will be successful and Parrot is going to gain some of these awesome new features. Cross  your fingers!

This entry was originally posted on Blogger and was automatically converted. There may be some broken links and other errors due to the conversion. Please let me know about any serious problems.