Ping class as expected holds an unmanaged resource viewed via reflector. In most cases, as i know, the class implements an explicit IDisposable.Dispose method. And Ping does it the same way as others, like EventWaitHandle. But unlike EventWaitHandle, it also inherits Component, an implementation class, which implements an implicit Dispose from IDisposable. As an outsider, i am surprising to compose my code like this:
Ping p = new Ping();
//...
p.Dispose();
As the Component class follows standard IDisposable pattern. There is a protected virtual Dispose(bool) method inside it. An interesting observation is the Ping doesn't override that protected method and implment the explicit IDisposable.Dispose as follow:
//from reflector (.net 2.0)
void IDisposable.Dispose()
{
  InternalDispose();
}

The InternalDispose() just closes everything and returns.

So if anyone acts like me, call Ping.Dispose() will get unexpected result.

I just got a reply from MSFT (Tratcher, a System.Net developer in Windows Core Networking) that "Your observations are correct.  This has been fixed in .Net 4.0 Beta 2, which was released this week.  Take a look and let us know what you think."

At last, I just got my beta 2 installed today. It looks pretty good. :)