Delphi Native Closures, Part II

A few days back, I made a post on the difficulties of implementing closures in a non-GC language.

Craig Stuntz made a comment:

Barry, I don't see how this problem is any different than in operator overloading, which is already supported in Win32, albeit in very limited conditions. So there's a fourth possibility not mentioned in your list, which is to only support it for functions with argument data types such as strings and value types which can be simply cleaned up. It's far from ideal, but given the OO example not unprecedented.

I agree with Craig re possibilities, there are more ways to slice it - but some slices are bigger than others! In my previous post, I cut things into two basic slices: how to free the delegate, and which captured variables to free. Of the two, the first one is about 90% (or more, FinalizeRecord in the Delphi RTL can do the second job) of the problem.

I don't agree with the comparison to operator overloading. Implementing closures is far more complex than operator overloading, because closures aren't limited to traditional Standard Pascal downward funargs; they can live long after the creating frame goes away. That isn't the case for temporaries created through support for operator overloading. The bulk of the work would be in putting the stack frame on the heap and refcounting it, and in implementing reference counting on every Delphi Win32 method pointer. I could then see people (ab)using COM interfaces to get deterministic destruction, effectively the same as the notional 'finalization' section I described.

Lest one think that one could get away without refcounting by limiting closure support to downward funargs, well, that would be a problem: closures gain a lot of power through composability. By composability in this context, I mean the ability to write a function that takes a function and returns a function, where the returned function is a function of the argument function, and thus relies on closure or some other way of tracking the passed in argument.

I should make that concrete. Consider this predicate, in C#:

public static Predicate<T> Not<T>(Predicate<T> pred)
{
    return delegate(T value) { return !pred(value); };
}

This snippet can't be implemented with just downward funargs because the stack frame of Not disappears when Not returns.

This is the heart of the problem: not only a transformation of stack into heap (of the lexically closed & captured variables etc.), but implementing refcounting everywhere a method pointer exists. Whether or not simple types like string and record and interface (and thus 'simple types' is open ended) are supported or not, is relatively trivial.

Another thing this brings up: the (ab)use of COM interfaces to implement simplify lifetime management of plain old Delphi objects, something which effectively would add the power of a 'finalization' section, without being explicit about it. It gets at another issue in my earlier post: the divergence of .NET and Win32 forms of Delphi. How important is it that the two are the same? Because interfaces don't work in a similar way at all in .NET, they're completely different. They're so different they probably deserve different names.