Parrot as a Mature Platform
19 Sep 2010Yesterday I met with Jim Keenan, fellow neophyte on the Parrot Foundation board. We got together for an informal get-together yesterday at a Barnes and Nobel, and had a highly productive little chat about Parrot: the foundation, the culture, the community, and the software. Obviously no "official" business happened at this little meetup, but we did get to know each other better and discussed a number of things. In this post, and others in the coming days, I'm going to talk about some of the points that came up in this meeting.
I've said many times, on this blog and elsewhere, that I don't think Parrot is currently a mature platform. It is certainly not suitable for use in a professional, production environment. I mentioned this sentiment to Jim, and he asked for some clarification. What did I mean by that?
Let me illustrate with a few examples.
You're employed as a system designer, and are preparing sketches for your companies next-generation software product. The success or failure of your design will have extreme effects on your current position, and maybe even on your career in the long term. In short, the design needs to be solid, flexible, expandible, robust, and all sorts of other good things. So here's the question: Do you choose Parrot as the basis for your new system, right now? If not, why?
Thought about it? Let me share a few answers with you, in the form of more questions. Name me one other mature, production-quality application VM or language interpreter which does not support threading. This isn't an empty set, but it's not a particularly large list either. Name me one other application VM that does not currently sport an aggressive, modern JIT.
Here's another example. Go to Google Scholar and search for papers and patents involving virtual machines. What percentage of the resulting papers use the Java Hotspot VM as the basis for their research? The .NET CLI? Smalltalk? Now take a closer look and count up how many results are based on Parrot? How many papers even mention Parrot in the footnotes?
You're a graduate student pursuing a PhD in CompSci. You want to spend the next three years of your life researching some new feature on virtual machines, the results of which will have major effects on your career and maybe will even influence whether you graduate or not. Do you choose to implement and study your fancy new feature on Java Hotspot, or on Parrot? Why?
Here's another example from a different direction. You work as the president of a large company which is reasonably sympathetic to the goals of the Parrot project. A Parrot contributor comes forward with a grant proposal to implement an exciting new feature. Do you cut a grant a sizable grant to support that developer through the production of that feature on Parrot, or do you spend your money elsewhere? In other words, do you expect to see a return on your investment, and do you expect that money spent on Parrot as it currently is will be well and efficiently spent? If you were doing your research and read over the long-term Parrot design, and if you were looking at the current state of the Parrot community and the community leadership, would you feel comfortable and confident to invest in it? Why or why not?
Turning that same example around a little bit, pretend that you're me. As a foundation board member, I'm going to take that grant request from the developer, put together all the necessary paperwork and approach a philanthropist with it. What do you think are the odds that I would get laughed out of the room and told never to come back?
Even though it's been 10 years since the start of it, Parrot really is a young project. Our long-term designs and goals are lacking. We have some extremely talented, enthusiastic, and energetic contributors, but we don't always do a great job of organizing and motivating them. There are plenty of areas where we do pretty well, but I can't think of a single aspect of the project or the community that we couldn't tune and improve. How much better do you want Parrot to be?
I want to be very clear about one thing here: I am not being insulting or disparaging about Parrot. It is not an insult to say that Parrot is not ready for enterprise-level production deployment. It is not disparaging to say that Parrot isn't a sure bet to make when careers and livelihoods are on the line. What we do need is honest self-assessment, and to use that as a basis for making long term plans and goals.
Starting from that honest self-assessment, we can start asking the important questions. Where do we go from here?
Hypothetically, we approach the Python Foundation and say "in 5 years, we want Parrot to be at a level where your premier, standard Python interpreter implementation could be implemented on top of it. What would you need to see in order to comfortably make that kind of decision in favor of Parrot?" Ask the same question of the Ruby, PHP, Perl 6, and Perl 5 interpreters. What does Parrot need to do in order to convince the leaders, architects, and developers of these projects that Parrot is a modern, competitive and even a desirable platform on which to build the next generation of their software? How long do people reasonably think it would take for Parrot to get into that condition? 1 year? 3 years? 5 years? There are no wrong answers, but the more honest we are with ourselves, the more certain we can be laying out a comprehensive roadmap to get us there.
In the coming days I'll address some of these issues. The point of this post was to get people thinking, dreaming, and doing both big.
This entry was originally posted on Blogger and was automatically converted. There may be some broken links and other errors due to the conversion. Please let me know about any serious problems.
Comments
chromatic
9/20/2010 12:14:23 AM
wsppan
9/20/2010 4:59:29 AM
Whiteknight
9/20/2010 7:59:44 AM
chromatic
9/20/2010 11:20:28 AM
Whiteknight
9/20/2010 11:34:43 AM
muixirt
9/20/2010 5:56:24 PM
Paul Topping
9/20/2010 6:52:51 PM
Tom Legrady
10/4/2010 10:35:51 AM
I can name lots of VMs which fit your "Woe is Parrot!" criteria (Python, Perl 5, Ruby, PHP). Consider also the first few years of the JVM.
remember Java 1.0? Companies started using that in a production environment.
Good points, both. In terms of the threading criteria, Python, Perl5 and Ruby all do have some threading support, of varying quality. PHP is the odd man out in this regard.
In terms of JIT, you are correct that none of these entries have it. I was drawing a (perhaps ficticious) line between "application VM" and "interpreter", so I'll definitely rescind that part of the post.
In terms of Java 1.0, you're right that it didn't have either of these features really. However, the state-of-the-art has significantly changed since 1999. In 2010, Java 1.0 would hardly qualify as a "mature" VM.
I could also make a point that things like Perl5, Python, Ruby, and PHP were all introduced at a time when the state-of-the-art didn't include robust, scalable threading and aggressive JIT, and have since been grandfathered in to enterprise use despite those omissions. For a new technology to gain significant market share starting in 2010 and the years ahead, we can't scrape by with the same limitations as were acceptable last century.
I've used so-called "enterprise software". Much of it was slow, buggy, resource hungry, undocumented, difficult to install, ugly, inscrutable, self-contradictory, and generally unusable without handholding from consultants and escalated support. That colors my perception of "mature software", however unfairly.
I think this means you pick your poison.
I'll stick with something that improves every month (or whatever its periodicity, as long as said period is at least twice a year) and which I can improve myself over something where my only hope of getting my work done involves filing support requests and documenting a leaning tower of workarounds.
(False dilemma? Not really; I don't use Parrot where "Needs a JIT" is a hard requirement.)
I understand what you're saying. Maybe the word "enterprise" is carrying connotations that I don't intend. What I mean to say (and maybe you can suggest better terminology) is the kind of software that a company would be ready and willing to invest real money. Any software is a gamble, but if a company is going to use a piece of software they want to minimize the odds of failure as much as possible.
Quick quiz: You're working on a high-volume web application (say, social networking for example). You have 10 servers now, each of which sports 16 processor cores and you have more hardware on the way. Do you use Parrot to implement the backend, which needs to scale easily on this hardware and beyond, or not?
Next question: You're developing a web application on a shoestring budget. You have hardware with lower specs than is recommended for the job. Do you use PHP with APC to cut processing requirements in half, or do you use Parrot? Right now.
Your comment really outlines the point I am trying to make: Parrot is growing. It's evolving. It's improving every month. My thesis is three-fold: One, Parrot isn't nearly as competitive now in the dynamic language runtime market as it could (and maybe should) be. Two, we can increase our rate of improvement with focused objectives and more developer motivation. Three, if we do that I project Parrot to be among the top-tier of dynamic language runtime environments within the next 1-5 years. Do you disagree with any of these things specifically?
What sort of things can be done with Parrot 2.8.0 in a productive manner? And what sort of things could be done with 200h (or 1000h) additional developer time?
I'm a Parrot outsider. I don't know all of its goals and all of its parts. I do know compiler and interpreter and computer language technology. I also have experience with the kind of considerations a company makes when selecting software tools that integrate so tightly into a product as typically Parrot would.
I think the problem with Parrot is that it attempts to do so much that outsiders can't see the use cases for which it promises to provide a solid solution. It is ok for Parrot to have such flexibility at its base but this will always be fuzzy at its edges. The Parrot Project should draw rings around the solid stuff and leave the fuzzy stuff on the pheriphery. In short, draw a stronger more visible boundary between the solid stuff that addresses well-defined use cases. Put the rest under Research Projects.
As far as looking into the future is concerned, my hope is that within 3 years Parrot will be a fast, efficient and flexible basis for implementing languages. Within 5 years, I hope that Parrot will have changed the concept of 'up-to-date VM', just as JIT implementations of Java once did.