Delphi Closures / Anonymous Delegates

Fredrik Haglund's post today reminded me of a post I made in the newsgroups about the difficulty of implementing anonymous delegates or closures in Delphi.

I'm a big fan of closures, as I've written about them in the past. I hope that Delphi gets some form of anonymous delegate, closure or lambda or some similar feature, while also being less verbose than local procedures and functions. Indeed, with LINQ coming along in the next version of C# and its corresponding .NET version, there'll be a requirement to integrate with it. LINQ pretty much needs lambdas, as well as expression trees, for good integration.

The problem comes in implementing closures in native code. It would be unfortunate if Delphi, the language, diverged further between the two main platforms it supports, .NET and Win32. Closures gain much (if not most) of their power by capturing local variables and arguments. If any captured local variables refer to object instances whose lifetime is logically associated with the lifetime of the stack frame which creates the closures, how does one ensure that those objects will get freed?

For example, in C# 2.0, one can write:

using System;

static class App
{
    static Action<string> CreatePrefixer(string prefix)
    {
        return delegate(string value)
        {
            Console.WriteLine("{0}: {1}", prefix, value);
        };
    }

    static void Main()
    {
        Action<string> prefixer = CreatePrefixer("foo");
        prefixer("bar");
        prefixer("baz");
    }
}

This keeps things simple by only using native .NET types, but hopefully it can illustrate the problem. By calling CreatePrefixer(), one creates an argument value (i.e. similar to a local - I'd have made it a local, but I wanted to keep the sample simple) in the stack frame associated with the CreatePrefixer() invocation, called "prefix". This value is captured by the created anonymous delegate, which is returned. That means that "prefix" must live on beyond the lifetime of the stack frame. So, the questions come:

1. How do you free the returned delegate?
   • Manually? Delphi users aren't used to freeing procedures or functions of object.
   • Refcounted? Is there a type system difference between existing method pointers and these new delegates (confusing) or would all method pointers now be refcounted (wasteful)? What if you got a cycle? A cycle involving method pointers would be extremely hard to spot since the graph depends on the runtime execution flow graph, much harder than a cycle in data structures, which is part of the statically declared data structure graph (assuming no funny stuff with typecasting).
2. How do you free the captured variables?
   • How do you know which ones the stack frame has ownership semantics over? In Native Delphi, strings have the nice property of being reference counted, so the solution is trivial for that datatype, so long as we are informed when the delegate is no longer needed. But what if it's an arbitrary user-defined datatype?
   • Should the anonymous delegate automagically create code to call TObject.Free?
   • Should there be a "destructor" or "finalization" block in the inline delegate definition? And won't this code be ignored for .NET execution?

For that last option, consider something like:

program Test;

type
  TActionOfString = procedure(const s: string) of object;

function CreatePrefixer(const prefix: string): TActionOfString;
begin
  Result := procedure(const value: string)
    begin
      Writeln(prefix, ': ', value);
    finalization
      // An optional section, any finalization to execute
      // when the anonymous delegate ultimately goes out of scope.
    end;
end;

var
  prefixer: TActionOfString;
begin
  prefixer := CreatePrefixer('foo');
  prefixer('bar');
  prefixer('baz');
end.

These two problems are why this construct is (to the best of my knowledge) typically restricted to garbage collected languages. Should Native Delphi have an optional compile mode that simultaneously turns on this language feature, and links in a conservative GC similar to the Boehm collector (which is currently used by Mono)?

Two other alternatives are possible: only support closures in the .NET compiler, but not in the native compiler; or don't support them at all.