Ruby’s Array>>shift is shifty

Here is a simple Ruby expression:

	(a = [1, 2, 3]).shift

The value of a will be [2, 3] which is exactly what you’d expect. The thing is, there is a bug here.

The Pick Axe book describes the shift function as:

Returns the first element of arr and removes it (shifting all other elements down by one)

This accurately describes the effect but is a bit misleading. What the implementation actually does (1.8.4) is to increment the C pointer to the first of the array by 1, and return the old value. This difference may be subtle, but it is significant. In particular the 1.8.4 implementation _does not_ overwrite the now inaccessible former first element, there is still a reference to the object and it will not be collected. If that object is big and the array long-lived this might pose a significant memory leak.

Sounds a little unlikely? Well, the Mutex class uses shift to manage the waiting thread list. This means that there are Thread instances in there, and they can be big. This has already created serious problems for Mongrel.

In the near term, you can simply do something like:

	a[0] = nil
	a.shift

A fix would be to modify array.c to look like (please excuse the formatting, sigh):

VALUE
rb_ary_shift(ary)
    VALUE ary;
{
    VALUE top;

    rb_ary_modify_check(ary);
    if (RARRAY(ary)->len == 0) return Qnil;
    top = RARRAY(ary)->ptr[0];
    RARRAY(ary)->ptr[0] = Qnil; /* < <<<ptr++;   /* shift ptr */
    RARRAY(ary)->len--;

    return top;
}