The implementation of iterators in C# and its consequences (part 1) Raymond Chen

Like
anonymous methods,
iterators in C# are very complex syntactic sugar.
You could do it all yourself (after all, you did have to do
it all yourself in earlier versions of C#),
but the compiler transformation makes for much greater convenience.

 

The idea behind iterators is that they take a function with
yield return
statements
(and possible some yield break statements)
and convert it into a state machine.
When you yield return, the state of the function is
recorded, and execution resumes from that state the next time the
iterator is called upon to produce another object.

 

Here’s the basic idea:
All the local variables of the iterator (treating iterator parameters
as pre-initialized local variables, including the hidden this
parameter)
become member variables of a helper class.
The helper class also has an internal state member that keeps
track of where execution left off and an internal current
member that holds the object most recently enumerated.

 

class MyClass {

 int limit = 0;

 public MyClass(int limit) { this.limit = limit; }


 public IEnumerable<int> CountFrom(int start)

 {

  for (int i = start; i <= limit; i++) {

   yield return i;

  }

 }

}

 

The CountFrom method produces an integer
enumerator that spits out the integers starting at start
and continuing up to and including limit.
The compiler internally converts this enumerator into
something like this:

 

class MyClass_Enumerator : IEnumerable<int> {

  int state$0 = 0;// internal member

  int current$0;  // internal member

  MyClass this$0; // implicit parameter to CountFrom

  int start;      // explicit parameter to CountFrom

  int i;          // local variable of CountFrom


  public int Current {

   get { return current$0; }

  }

  public bool MoveNext()

  {

   switch (state$0) {

   case 0: goto resume$0;

   case 1: goto resume$1;

   case 2: return false;

   }

 resume$0:;

   for (i = start; i <= this$0.limit; i++) {

    current$0 = i;

    state$0 = 1;

    return true;

 resume$1:;

   }

   state$0 = 2;

   return false;

  }

  … other bookkeeping, not important here …

 }

 public IEnumerable<int> CountFrom(int start)

 {

  MyClass_Enumerator e = new MyClass_Enumerator();

  e.this$0 = this;

  e.start = start;

  return e;

 }

 

用dnSpy反编译上面的代码，同时在配置中

 

 得到如下代码，是一个状态机

    // Token: 0x02000005 RID: 5
    internal class MyClass
    {
        // Token: 0x06000006 RID: 6 RVA: 0x000020C9 File Offset: 0x000002C9
        public MyClass(int limit)
        {
            this.limit = limit;
        }

        // Token: 0x06000007 RID: 7 RVA: 0x000020E1 File Offset: 0x000002E1
        public IEnumerable<int> CountFrom(int start)
        {
            MyClass.<CountFrom>d__2 <CountFrom>d__ = new MyClass.<CountFrom>d__2(-2);
            <CountFrom>d__.<>4__this = this;
            <CountFrom>d__.<>3__start = start;
            return <CountFrom>d__;
        }

        // Token: 0x04000001 RID: 1
        private int limit = 0;

        // Token: 0x02000006 RID: 6
        [CompilerGenerated]
        private sealed class <CountFrom>d__2 : IEnumerable<int>, IEnumerable, IEnumerator<int>, IDisposable, IEnumerator
        {
            // Token: 0x06000008 RID: 8 RVA: 0x000020F8 File Offset: 0x000002F8
            [DebuggerHidden]
            public <CountFrom>d__2(int <>1__state)
            {
                this.<>1__state = <>1__state;
                this.<>l__initialThreadId = Environment.CurrentManagedThreadId;
            }

            // Token: 0x06000009 RID: 9 RVA: 0x00002113 File Offset: 0x00000313
            [DebuggerHidden]
            void IDisposable.Dispose()
            {
            }

            // Token: 0x0600000A RID: 10 RVA: 0x00002118 File Offset: 0x00000318
            bool IEnumerator.MoveNext()
            {
                int num = this.<>1__state;
                if (num != 0)
                {
                    if (num != 1)
                    {
                        return false;
                    }
                    this.<>1__state = -1;
                    int num2 = this.<i>5__1;
                    this.<i>5__1 = num2 + 1;
                }
                else
                {
                    this.<>1__state = -1;
                    this.<i>5__1 = this.start;
                }
                if (this.<i>5__1 > this.<>4__this.limit)
                {
                    return false;
                }
                this.<>2__current = this.<i>5__1;
                this.<>1__state = 1;
                return true;
            }

            // Token: 0x17000001 RID: 1
            // (get) Token: 0x0600000B RID: 11 RVA: 0x0000219C File Offset: 0x0000039C
            int IEnumerator<int>.Current
            {
                [DebuggerHidden]
                get
                {
                    return this.<>2__current;
                }
            }

            // Token: 0x0600000C RID: 12 RVA: 0x000021A4 File Offset: 0x000003A4
            [DebuggerHidden]
            void IEnumerator.Reset()
            {
                throw new NotSupportedException();
            }

            // Token: 0x17000002 RID: 2
            // (get) Token: 0x0600000D RID: 13 RVA: 0x000021AB File Offset: 0x000003AB
            object IEnumerator.Current
            {
                [DebuggerHidden]
                get
                {
                    return this.<>2__current;
                }
            }

            // Token: 0x0600000E RID: 14 RVA: 0x000021B8 File Offset: 0x000003B8
            [DebuggerHidden]
            IEnumerator<int> IEnumerable<int>.GetEnumerator()
            {
                MyClass.<CountFrom>d__2 <CountFrom>d__;
                if (this.<>1__state == -2 && this.<>l__initialThreadId == Environment.CurrentManagedThreadId)
                {
                    this.<>1__state = 0;
                    <CountFrom>d__ = this;
                }
                else
                {
                    <CountFrom>d__ = new MyClass.<CountFrom>d__2(0);
                    <CountFrom>d__.<>4__this = this.<>4__this;
                }
                <CountFrom>d__.start = this.<>3__start;
                return <CountFrom>d__;
            }

            // Token: 0x0600000F RID: 15 RVA: 0x00002207 File Offset: 0x00000407
            [DebuggerHidden]
            IEnumerator IEnumerable.GetEnumerator()
            {
                return this.System.Collections.Generic.IEnumerable<System.Int32>.GetEnumerator();
            }

            // Token: 0x04000002 RID: 2
            private int <>1__state;

            // Token: 0x04000003 RID: 3
            private int <>2__current;

            // Token: 0x04000004 RID: 4
            private int <>l__initialThreadId;

            // Token: 0x04000005 RID: 5
            private int start;

            // Token: 0x04000006 RID: 6
            public int <>3__start;

            // Token: 0x04000007 RID: 7
            public MyClass <>4__this;

            // Token: 0x04000008 RID: 8
            private int <i>5__1;
        }
    }

 

The enumerator class is auto-generated by the compiler
and, as promised, it contains two internal members for the
state and current object,
plus a member for each parameter
(including the hidden this parameter),
plus a member for each local variable.
The Current property merely returns the current object.
All the real work happens in MoveNext.

 

To generate the MoveNext method, the compiler
takes the code you write and performs a few transformations.
First, all the references to variables and parameters need to
be adjusted since the code moved to a helper class.

 

 

Notice that this transformation is quite different from
the enumeration model we built based on coroutines and fibers.
The C# method is far more efficient in terms of memory usage
since it doesn’t consume an entire stack (typically a megabyte in size)
like the fiber approach does.
Instead it just borrows the stack of the caller,
and anything that it needs to save across calls to MoveNext
are stored in a helper object (which goes on the heap rather than the stack).
This fake-out is normally quite effective—most
people don’t even realize that it’s happening—but there are places
where the difference is significant, and we’ll see that shortly.

 

作者:Chuck Lu    GitHub