Tiburon: fun with generics and anonymous methods

Update: I've added more code to the end of this entry.

The next release, Delphi 2009 (codename Tiburón) has some new language features: generics and anonymous methods in particular. I've decided to write up some of my experiments, playing with the new tools to see what can be done with them.

First, though, a disclaimer: I am of course working with the development tip for both compiler and RTL, so some details mentioned here (RTL in particular) may change before final release. However, the basics of generics and anonymous methods are pretty solid and should not change, so any gaps should be fillable if needs be.

Here's a generic method reference:

type
  TFunc<T> = reference to function: T;

This guy (a "method reference") can refer to any global function, method or anonymous method which takes no parameters and returns a value. It's a managed type, like strings and interfaces, because it can contain state that needs to be freed when the last reference to it goes out of scope. Here's a trivial example of a method reference in use with an anonymous method:

{$APPTYPE CONSOLE}

uses
  SysUtils;

function MakeCounter(Start, Increment: Integer): TFunc<Integer>;
begin
  Result := function: Integer
  begin
    Result := Start;
    Inc(Start, Increment);
  end;
end;

procedure WriteList(const Source: TFunc<Integer>; Count: Integer);
begin
  while Count > 1 do
  begin
    Write(Source, ', ');
    Dec(Count);
  end;
  Writeln(Source);
end;

procedure DoIt;
var
  evenNumbers: TFunc<Integer>;
  decades: TFunc<Integer>;
begin
  evenNumbers := MakeCounter(0, 2);
  decades := MakeCounter(0, 10);

  WriteList(evenNumbers, 10);
  WriteList(decades, 10);
end;

begin
  try
    DoIt;
  except
    on E:Exception do
      Writeln(E.Classname, ': ', E.Message);
  end;
end.

The code should be pretty easy to follow. It prints out the first 10 odd numbers, and the first 10 numbers evenly divisible by 10, starting at 0. Here's the exact output:

0, 2, 4, 6, 8, 10, 12, 14, 16, 18
0, 10, 20, 30, 40, 50, 60, 70, 80, 90

Notice how the value returned from MakeCounter has captured its parameters, and that these parameters have become internal state of the return value. This action is called variable capture, and an anonymous method that performs variable capture is a closure. TFunc<T> is useful enough that it is predefined in SysUtils; there are similar versions taking up to 4 arguments, and there are also TProc method references, also taking up to 4 arguments.

This example suggests to me an easy way of creating enumerators in Delphi. Rather than having to define a separate enumerator class, and implement the GetCurrent and MoveNext functions, as well as a property that reads from GetCurrent, we can use an anonymous method that simply returns each value in sequence. To signal termination, we'll use a special exception. With these tools, we can write a counter generator that has an upper bound too:

{$APPTYPE CONSOLE}

uses
  SysUtils;

type
  EEnumerationFinished = class(Exception)
    constructor Create;
  end;
  
  TSimpleEnumerator<T> = class
  private
    FSource: TFunc<T>;
    FCurrent: T;
    FFinished: Boolean;
    function GetCurrent: T;
  public
    constructor Create(const ASource: TFunc<T>);
    function MoveNext: Boolean;
    property Current: T read GetCurrent;
  end;

{ EEnumerationFinished }

constructor EEnumerationFinished.Create;
begin
  inherited Create('Enumeration finished');
end;
  
{ TSimpleEnumerator<T> }

constructor TSimpleEnumerator<T>.Create(const ASource: TFunc<T>);
begin
  FSource := ASource;
end;

function TSimpleEnumerator<T>.GetCurrent: T;
begin
  Result := FCurrent;
end;

function TSimpleEnumerator<T>.MoveNext: Boolean;
begin
  if FFinished then
    Exit(False);
  try
    FCurrent := FSource;
  except
    on e: EEnumerationFinished do
    begin
      FFinished := True;
      Exit(False);
    end;
  end;
  Exit(True);
end;

function MakeCounter(Start, Increment, Count: Integer): TSimpleEnumerator<Integer>;
begin
  Result := TSimpleEnumerator<Integer>.Create(function: Integer
  begin
    if Count <= 0 then
      raise EEnumerationFinished.Create;
    Result := Start;
    Inc(Start, Increment);
    Dec(Count);
  end);
end;

function ToString(e: TSimpleEnumerator<Integer>): string;
var
  sb: TStringBuilder;
begin
  sb := TStringBuilder.Create;
  try
    while e.MoveNext do
      sb.Append(e.Current).Append(', ');
    if sb.Length > 2 then
      sb.Length := sb.Length - 2;
    Result := sb.ToString;
  finally
    sb.Free;
  end;
end;

procedure DoIt;
var
  oddNums: TSimpleEnumerator<Integer>;
begin
  oddNums := MakeCounter(1, 2, 20);
  try
    Writeln('Odd numbers: ', ToString(oddNums));
    Writeln('And again: ', ToString(oddNums));
  finally
    oddNums.Free;
  end;
end;

begin
  try
    DoIt;
  except
    on E:Exception do
      Writeln(E.Classname, ': ', E.Message);
  end;
end.

Unfortunately, this has several drawbacks. First, the for-in syntax can't be used, because that expects a collection of some kind that has a method called GetEnumerator, whereas we only have the enumerator itself. Secondly, the enumeration can only be performed once. You can see this in the output:

Odd numbers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39
And again: 

We can fix both of these problems, however, by adding another layer of indirection. Rather than working with TFunc<T>, we'll work with TFunc<TFunc<T>>; and we'll write a TSimpleEnumerable<T> that returns our enumerator.

Now, inside one of the new units, Generics.Collections, are a couple of new base classes, TEnumerator<T> and TEnumerable<T>. I'll use those base classes too:

{$APPTYPE CONSOLE}

uses
  SysUtils, Generics.Collections;

type
  EEnumerationFinished = class(Exception)
    constructor Create;
  end;

  TSimpleEnumerable<T> = class(TEnumerable<T>)
  private
    FSourceMaker: TFunc<TFunc<T>>;
  public
    type
      TEnumerator = class(TEnumerator<T>)
      private
        FSource: TFunc<T>;
        FCurrent: T;
        FFinished: Boolean;
        constructor Create(const ASource: TFunc<T>);
        function GetCurrent: T;
      protected
        function DoGetCurrent: T; override;
        function DoMoveNext: Boolean; override;
      public
        function MoveNext: Boolean;
        property Current: T read GetCurrent;
      end;
  protected
    function DoGetEnumerator: TEnumerator<T>; override;
  public
    constructor Create(const ASourceMaker: TFunc<TFunc<T>>);
    function GetEnumerator: TEnumerator;
  end;
  
{ EEnumerationFinished }

constructor EEnumerationFinished.Create;
begin
  inherited Create('Enumeration finished');
end;
  
{ TSimpleEnumerable<T>.TEnumerator }

constructor TSimpleEnumerable<T>.TEnumerator.Create(const ASource: TFunc<T>);
begin
  FSource := ASource;
end;

function TSimpleEnumerable<T>.TEnumerator.DoGetCurrent: T;
begin
  Result := GetCurrent;
end;

function TSimpleEnumerable<T>.TEnumerator.DoMoveNext: Boolean;
begin
  Result := MoveNext;
end;

function TSimpleEnumerable<T>.TEnumerator.GetCurrent: T;
begin
  Result := FCurrent;
end;

function TSimpleEnumerable<T>.TEnumerator.MoveNext: Boolean;
begin
  if FFinished then
    Exit(False);
  try
    FCurrent := FSource;
  except
    on e: EEnumerationFinished do
    begin
      FFinished := True;
      FCurrent := Default(T); // don't keep alive if refcounted
      Exit(False);
    end;
  end;
  Result := True;
end;

{ TSimpleEnumerable<T> }

constructor TSimpleEnumerable<T>.Create(const ASourceMaker: TFunc<TFunc<T>>);
begin
  FSourceMaker := ASourceMaker;
end;

function TSimpleEnumerable<T>.DoGetEnumerator: TEnumerator<T>;
begin
  Result := GetEnumerator;
end;

function TSimpleEnumerable<T>.GetEnumerator: TEnumerator;
begin
  Result := TEnumerator.Create(FSourceMaker());
end;

function MakeCounter(Start, Increment, Count: Integer): TSimpleEnumerable<Integer>;
begin
  Result := TSimpleEnumerable<Integer>.Create(function: TFunc<Integer>
  var
    myStart, myIncrement, myCount: Integer;
  begin
    myStart := Start;
    myIncrement := Increment;
    myCount := Count;
    
    Result := function: Integer
    begin
      if myCount <= 0 then
        raise EEnumerationFinished.Create;
      Result := myStart;
      Inc(myStart, myIncrement);
      Dec(myCount);
    end;
  end);
end;

function ToString(Collection: TEnumerable<Integer>): string;
var
  sb: TStringBuilder;
  item: Integer;
begin
  sb := TStringBuilder.Create;
  try
    for item in Collection do
      sb.Append(item).Append(', ');
    if sb.Length > 2 then
      sb.Length := sb.Length - 2;
    Result := sb.ToString;
  finally
    sb.Free;
  end;
end;

procedure DoIt;
var
  oddNums: TEnumerable<Integer>;
begin
  oddNums := MakeCounter(1, 2, 20);
  try
    Writeln('Odd numbers: ', ToString(oddNums));
    Writeln('And again: ', ToString(oddNums));
  finally
    oddNums.Free;
  end;
end;

begin
  try
    DoIt;
  except
    on E:Exception do
      Writeln(E.Classname, ': ', E.Message);
  end;
end.

This did require a little extra complexity - making a copy of the state in the outer anonymous method of MakeCounter - but it also now performs as expected:

Odd numbers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39
And again: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39

So there we have it: an easy way to create enumerators and simple collection wrappers using a combination of generics and anonymous methods. There's more that can be built on this foundation, but I'll leave that until later.

Update:

Joylon in the comments prompts me to show the alternative - i.e. how to write the above without using anonymous methods. Here's a basic enumerator which does the same thing as the above - i.e. generate an integer sequence of a specified length and stride:

type
  TEnumerator = class
  private
    FCurrent: Integer;
    FCount: Integer;
    FIncrement: Integer;
    function GetCurrent: Integer;
  public
    constructor Create(Start, Increment, Count: Integer);
    function MoveNext: Boolean;
    property Current: Integer read GetCurrent;
  end;
  
{ TEnumerator }

constructor TEnumerator.Create(Start, Increment, Count: Integer);
begin
  FCurrent := Start;
  FCount := Count;
  FIncrement := Increment;
end;

function TEnumerator.GetCurrent: Integer;
begin
  Result := FCurrent;
end;

function TEnumerator.MoveNext: Boolean;
begin
  if FCount <= 0 then
    Exit(False);
  Inc(FCurrent, FIncrement);
  Dec(FCount);
  Result := True;
end;

At 35 lines, it is quite a bit (3x) more verbose than the directly equivalent MakeCounter function (11 lines; this isn't one that supports resetting or recreating the enumerator, but the translation above doesn't either):

function MakeCounter(Start, Increment, Count: Integer): TSimpleEnumerator<Integer>;
begin
  Result := TSimpleEnumerator<Integer>.Create(function: Integer
  begin
    if Count <= 0 then
      raise EEnumerationFinished.Create;
    Result := Start;
    Inc(Start, Increment);
    Dec(Count);
  end);
end;

However, even then, I wouldn't necessarily recommend using the above technique to implement all enumerators in general - the fact that exceptions are used to signal termination will cause some performance loss and, more importantly, quite a bit of annoyance when running in a debugger.

Rather, I intend, over the next few posts in this blog, to use anonymous methods in all sorts of ways so that you, the reader, might see opportunities for their creative use in your own code.