TCMalloc源码学习(四)(小内存块释放)
pagemap_和pagemap_cache_
PageHeap有两个map,pagemap_记录某一内存页对应哪一个span,显然可能多页对应一个span,pagemap_cache_记录某一内存页对应哪一个SizeClass。
在TCMalloc源码分析(一)中有提到过pagemap_所占内存的问题,假设32位系统4GB可用内存,若pagemap_使用数组实现需要占用4MB的内存(假设一页4KB),仿佛还可以接受,但如果是64位系统呢?所以实际上TCMalloc使用了radix-tree树实现了
pagemap_(64位系统使用三层radix-tree TCMalloc_PageMap2,32位使用两层 TCMalloc_PageMap3)。
radix-tree其实是一棵多叉树,原理是这样:比如三层,会把对应的key的二进制位分成三部分(High,Medium,Low),依次来生成树的三层,最后一层是叶子节点保存key对应的value。
节点都是插入key的过程动态生成的,不像数组实现一开始就要很大一块内存,radix-tree是随着TCMalloc内存分配的增多而增大,而且因为通常都是相邻的页分配出去,也就是页号的二进位比较相似,所以可以共用high和medium节点,这样可以减缓随着页内存分配的增多radix-tree所耗内存的增速。
之前说 pagemap_是PageId ----> Span的映射,再show一下这个图
pagemap_cache_是PageId -----> SizeClass的映射,用的数据结构是一个压缩的哈希表PackedCache,实现上就是用了一个数组,然后用key做hash插入到对应的entry,但是却内部有不少细节:PackedCache有三个位数kHashbits,kValuebits,kKeybits。数组的长度是1<<kHashBits,hash函数直接是用key对长度取模(key & ( 1 << kHashBits - 1));kValueBits和kKeyBits代表value和key分别占多少位;有一个问题,如果kKeybits大于了kHashbits,那就有可能多个key映射到同一个entry,为了区分开,每一个entry放的值的数值二进制位是部分key位和所有value位的: | kKeybits - kHashbits | kValuebits | 。
entry的类型在64位系统上是64位,32位系统上16位,在更新每一个entry时候不存在只更新到部分数位的情况,所以可以无锁访问pagemap_cache_。
PageId -----> SizeClass的映射能用到压缩的哈希表,也是因为SizeClass是一个比较小的值,如果纯用作Value的话比如Value可能是16位或者64位,会浪费一些数位,剩余的数位也可以利用起来做key。
小内存释放
1.求出要释放的内存指针在哪一内存页上;
const PageID p = reinterpret_cast<uintptr_t >(ptr) >> kPageShift;
2.用PageID在pagemap_cache_找对应的SizeClass;
size_t cl = Static:: pageheap()->GetSizeClassIfCached (p);
3.如果cache中没有对应的PageID,用在pagemap_中找对应的Span,然后得到SizeClass;
span = Static ::pageheap()-> GetDescriptor(p );
cl = span ->sizeclass;
4.取到对应线程的ThreadCache,用参数cl调用Deallocate;
heap->Deallocate (ptr, cl);
在本地线程释放
函数ThreadCache:: Deallocate 的实现:
inline void ThreadCache:: Deallocate(void * ptr, size_t cl ) { FreeList* list = &list_ [cl]; size_ += Static::sizemap ()->ByteSizeForClass( cl); ssize_t size_headroom = max_size_ - size_ - 1; // This catches back-to-back frees of allocs in the same size // class. A more comprehensive (and expensive) test would be to walk // the entire freelist. But this might be enough to find some bugs. ASSERT( ptr != list ->Next()); list-> Push(ptr ); ssize_t list_headroom = static_cast<ssize_t >(list-> max_length()) - list ->length(); // There are two relatively uncommon things that require further work. // In the common case we're done, and in that case we need a single branch // because of the bitwise-or trick that follows. if (( list_headroom | size_headroom ) < 0) { if (list_headroom < 0) { ListTooLong(list , cl); } if (size_ >= max_size_) Scavenge(); } }
主要是找到对应SizeClass的FreeList,把回收的内存插入空闲链表头部。ThreadCache有两个指标,一个是每一条FreeList都有的max_length限制,一个是总共使用的内存max_size_限制,如果超过了需要调整。ListTooLong就是当前FreeList长度超过max_length的调整:
void ThreadCache ::ListTooLong( FreeList* list , size_t cl) { const int batch_size = Static:: sizemap()->num_objects_to_move (cl); ReleaseToCentralCache( list, cl , batch_size); // If the list is too long, we need to transfer some number of // objects to the central cache. Ideally, we would transfer // num_objects_to_move, so the code below tries to make max_length // converge on num_objects_to_move. if ( list->max_length () < batch_size) { // Slow start the max_length so we don't overreserve. list->set_max_length (list-> max_length() + 1); } else if (list ->max_length() > batch_size) { // If we consistently go over max_length, shrink max_length. If we don't // shrink it, some amount of memory will always stay in this freelist. list->set_length_overages (list-> length_overages() + 1); if (list ->length_overages() > kMaxOverages) { ASSERT(list ->max_length() > batch_size); list->set_max_length (list-> max_length() - batch_size ); list->set_length_overages (0); } } }
当长度超过上限的时候,移回部分空闲对象到Central Cache中去,ReleaseToCentralCache实现不贴了,无非就是从线程FreeList弹出指定个内存对象插入到对应CentralFreeList中去。
在Centreal Cache中释放
对应从CentrealCache中分配内存的RemoveRange接口,把内存收回到CentrealCache中的接口是InsertRange,InsertRange的实现也是先判断转移缓存(TCEntry)中是否还有空间放置回收内存,有就放到转移缓存然后就返回了,这个在对内存频繁分配释放的时候比较高效。若没有地方放了就要转到其对应的Span了,ReleaseListToSpans调用如下:
void CentralFreeList ::ReleaseListToSpans( void* start ) { while ( start) { void *next = SLL_Next( start); ReleaseToSpans(start ); start = next ; } }
就是一个一个内存对象调用ReleaseToSpans 释放,ReleaseToSpans 如下:
void CentralFreeList ::ReleaseToSpans( void* object ) { Span* span = MapObjectToSpan (object); ASSERT( span != NULL ); ASSERT( span->refcount > 0); // If span is empty, move it to non-empty list if ( span->objects == NULL) { tcmalloc::DLL_Remove (span); tcmalloc::DLL_Prepend (&nonempty_, span); Event(span , 'N', 0); } // The following check is expensive, so it is disabled by default if ( false) { // Check that object does not occur in list int got = 0; for (void * p = span->objects ; p != NULL; p = *((void**) p)) { ASSERT(p != object); got++; } ASSERT(got + span-> refcount == ( span->length <<kPageShift) / Static::sizemap ()->ByteSizeForClass( span->sizeclass )); } counter_++; span-> refcount--; if ( span->refcount == 0) { Event(span , '#', 0); counter_ -= ((span ->length<< kPageShift) / Static::sizemap ()->ByteSizeForClass( span->sizeclass )); tcmalloc::DLL_Remove (span); -- num_spans_; // Release central list lock while operating on pageheap lock_.Unlock (); { SpinLockHolder h(Static ::pageheap_lock()); Static::pageheap ()->Delete( span); } lock_.Lock (); } else { *( reinterpret_cast<void **>(object)) = span->objects ; span->objects = object; } }
过程简单描述如下:
1.判断这个Span所标识的对象是不是之前已经分配完了,若是就要把他从CentralFreeList的empty_ Spans List列表中挪出到nonempty_ Spans List中,因为我要把返回的内存对象给这个Span了
;
2.递减Span的引用计数,如果已经没有人在引用了就要把Span标识的所有内存返还给PageHeap了。
内存在PageHeap中的释放
与在PageHeap中分配内存的New对应,释放内存是Delete,Delete主要是取消Span与某个SizeClass关联和取消这个Span正在使用的状态标记为ON_NORMAL_FREELIST,即将要放入normal list中,之后就是调用MergeIntoFreeList,即和邻近的空闲内存合并放入空闲链表中。MergeIntoFreeList的实现如下:
void PageHeap ::MergeIntoFreeList( Span* span ) { ASSERT( span->location != Span:: IN_USE); // Coalesce -- we guarantee that "p" != 0, so no bounds checking // necessary. We do not bother resetting the stale pagemap // entries for the pieces we are merging together because we only // care about the pagemap entries for the boundaries. // // Note that only similar spans are merged together. For example, // we do not coalesce "returned" spans with "normal" spans. const PageID p = span-> start; const Length n = span-> length; Span* prev = GetDescriptor (p-1); if ( prev != NULL && prev-> location == span ->location) { // Merge preceding span into this span ASSERT(prev ->start + prev->length == p); const Length len = prev->length ; RemoveFromFreeList(prev ); DeleteSpan(prev ); span->start -= len; span->length += len; pagemap_.set (span-> start, span ); Event(span , 'L', len); } Span* next = GetDescriptor (p+ n); if ( next != NULL && next-> location == span ->location) { // Merge next span into this span ASSERT(next ->start == p+n ); const Length len = next->length ; RemoveFromFreeList(next ); DeleteSpan(next ); span->length += len; pagemap_.set (span-> start + span ->length - 1, span); Event(span , 'R', len); } PrependToFreeList( span); }
MergeIntoFreeList就是取目标Span标识的内存邻近的页对应的Span出来,判断如果能和目标Span合并就合并之,之后才插入到normal free list中去。
回到Delete,MergeIntoFreeList返回后,IncrementalScavenge调用有可能触发把一些空闲内存释放回系统的操作,释放的策略是这样:有一个scavenge_counter_计数,每次Delete调用都会降低其值,若降为0才真正去释放给系统。可以调整scavenge_counter_的值来控制释放给系统的频率,IncrementalScavenge代码如下:
void PageHeap ::IncrementalScavenge( Length n ) { // Fast path; not yet time to release memory scavenge_counter_ -= n; if ( scavenge_counter_ >= 0) return ; // Not yet time to scavenge const double rate = FLAGS_tcmalloc_release_rate; if ( rate <= 1e-6) { // Tiny release rate means that releasing is disabled. scavenge_counter_ = kDefaultReleaseDelay ; return; } Length released_pages = ReleaseAtLeastNPages (1); if ( released_pages == 0) { // Nothing to scavenge, delay for a while. scavenge_counter_ = kDefaultReleaseDelay ; } else { // Compute how long to wait until we return memory. // FLAGS_tcmalloc_release_rate==1 means wait for 1000 pages // after releasing one page. const double mult = 1000.0 / rate; double wait = mult * static_cast<double >(released_pages); if (wait > kMaxReleaseDelay) { // Avoid overflow and bound to reasonable range. wait = kMaxReleaseDelay ; } scavenge_counter_ = static_cast <int64_t>( wait); } }
又是ReleaseAtLeastNPages 调用,在TCMalloc源码分析(三)中有详细分析这个调用,记住windows上内存是不还给系统的,细节不在复述了。
再回到ThreadCache
之前说到ListTooLong返还内存给Central Cache后,调整了max_length,主要是怕链表后面的空闲内存一直在本地线程中,自己不用也不释放给其他线程用。
ListTooLong是调整单个FreeList的长度,Scavenge则是在整个ThreadCache使用的内存上来考虑,当前使用的内存大于一个上限后就会被调用,Scavenge代码如下:
void ThreadCache ::Scavenge() { // If the low-water mark for the free list is L, it means we would // not have had to allocate anything from the central cache even if // we had reduced the free list size by L. We aim to get closer to // that situation by dropping L/2 nodes from the free list. This // may not release much memory, but if so we will call scavenge again // pretty soon and the low-water marks will be high on that call. //int64 start = CycleClock::Now(); for ( int cl = 0; cl < kNumClasses; cl ++) { FreeList* list = &list_[ cl]; const int lowmark = list->lowwatermark (); if (lowmark > 0) { const int drop = ( lowmark > 1) ? lowmark /2 : 1; ReleaseToCentralCache(list , cl, drop); // Shrink the max length if it isn't used. Only shrink down to // batch_size -- if the thread was active enough to get the max_length // above batch_size, it will likely be that active again. If // max_length shinks below batch_size, the thread will have to // go through the slow-start behavior again. The slow-start is useful // mainly for threads that stay relatively idle for their entire // lifetime. const int batch_size = Static::sizemap ()->num_objects_to_move( cl); if (list ->max_length() > batch_size) { list->set_max_length ( max<int >(list-> max_length() - batch_size , batch_size)); } } list->clear_lowwatermark (); } IncreaseCacheLimit(); }
整个过程就是遍历所有FreeList进行逐一释放,每一个FreeList有一个lowwatermark L,代表上次回收内存后FreeList的长度,每次回收时释放 L/2个object,下次回收时L就表示自从上次回收后一直没有用过的内存,那就把他还给Central Cache吧。这就是这种用历史记录预测未来内存使用情况的策略。
最后就是IncreaseCacheLimit调用,实现为锁住后调用IncreaseCacheLimitLocked,IncreaseCacheLimitLocked的代码如下:
void ThreadCache ::IncreaseCacheLimitLocked() { if ( unclaimed_cache_space_ > 0) { // Possibly make unclaimed_cache_space_ negative. unclaimed_cache_space_ -= kStealAmount ; max_size_ += kStealAmount ; return; } // Don't hold pageheap_lock too long. Try to steal from 10 other // threads before giving up. The i < 10 condition also prevents an // infinite loop in case none of the existing thread heaps are // suitable places to steal from. for ( int i = 0; i < 10; ++ i, next_memory_steal_ = next_memory_steal_-> next_) { // Reached the end of the linked list. Start at the beginning. if (next_memory_steal_ == NULL) { ASSERT(thread_heaps_ != NULL); next_memory_steal_ = thread_heaps_ ; } if (next_memory_steal_ == this || next_memory_steal_->max_size_ <= kMinThreadCacheSize) { continue; } next_memory_steal_->max_size_ -= kStealAmount; max_size_ += kStealAmount ; next_memory_steal_ = next_memory_steal_ ->next_; return; } }
kStealAmount是在ThreadCache被强制Scavenge后,max_size_应该从unclaimed_cache_space_或者其他线程偷取的字节数,这样就可以使得下次 Scavenge被延迟避免频繁Scavenge。这个过程其实是在表达这样的意思:这次是我花时间把自己的内存返还给Central Cache了,下次轮到其他线程去做了。因为这个过程是在多个线程之间调整他们所能够拥有的内存上限,所以当然要用到锁了。
总结:
释放过程不像其他malloc-free实现,在内存头几个字节保存了size,而是直接算出内存所在页号,借助pagemap_cache_和pagemap_索引其应回收到的位置。内存在ThreadCache,CentralFreeList,PageHeap之间层层回收,优先回收在本地,延迟回收到下层,其间有空闲内存合并,启发式的回收策略,多个线程互相调整回收频率,为达到内存在不同线程间有效利用,高效回收。