Programming, Software and Code

Lots of IO work to do

26 May 2009
legacy  —  Parrot  —  IO
It's no secret that I've been planning pretty aggressively to implement an asynchronous IO subsystem for Parrot. However, this is really just one part of a larger group of projects that the IO system needs.

Today in the weekly Parrotsketch meeting, a lot of IO-related issues were raised. Several people mentioned performance issues, which seems to be a hot button issue in Parrot-world and for good reason. The way the IO system API is currently implemented is as a thin wrapper layer around PCCINVOKE method calls for the FileHandle and related PMCs. This implementation works decently, and was necessary for a time to support performing IO operations on a wide array of hypothetical IO PMCs, but there is a huge overhead involved in calling all those PMC methods. In fact, it was this very issue that prompted the current round of PCC refactors. Instead of having IO PMCs related simply by sharing a common set of methods, it's probably much better for these PMCs to be related together by inheritance and to use a common set of C functions to access their common VTABLEs and attributes directly.

So Infinoid and I have been putting together a plan tonight and I'm pretty excited about it. Here are some of the main points:
  1. We're refactoring the PMC hiearchy. IO types will all derive from a common Handle type that will have a few common attributes, vtables, and methods.
  2. We're going to refactor FileHandle to subclass from Handle
  3. We're going to fix Socket so that it subclasses Handle, and not FileHandle
  4. We're going to use roles and the VTABLE_does to determine the capabilities of each type of IO PMC. Roles like "read", "write", "file", "socket" and "pipe" will determine capabilities. Then we can also have subroles, like "connection" and "connectionless" for different types of sockets, etc.
  5. Try to fix attributes, where possible, to be inheritable so we can subclass all these things from PIR reliably
  6. Change the IO API to use C functions instead of PCCINVOKE methods to perform operations. We sacrifice a little bit of encapsulation for big performance gains, but I think it's a worthwhile change.
After this, Infinoid is really looking to get an IO multiplexing "Select" PMC put together, and I'm looking at getting AIO in place. I was planning to do something like Select as part of the AIO work, but I'm not sure that my idea is the best for general purpose use. I'm still considering using an "IOListener" that will check IO events as part of the scheduler, but that could be the asynchronous equivalent of a manually-polled Select PMC. Maybe we need both, maybe we only need one. We'll see how it goes.

My goal is to get this cleanup work done within 1-2 weeks, and with a great coder like Infinoid in my team I think it's completely possible. After that, I have the ambitious goal of getting AIO implemented, or at least a working subset of it, by the 1.3 release (that I am the release manager for). Certainly by 1.5 Parrot is going to have a world-class IO system available, which is perfect timing because that's when a few of my other tasklist items become unblocked.

This entry was originally posted on Blogger and was automatically converted. There may be some broken links and other errors due to the conversion. Please let me know about any serious problems.