早就听说过Memcached独特的内存管理方式,写着篇文章的目的就是了解Memcached的内存管理,学习其源代码.
1.什么是Slab Allocator
memcached默认情况下采用了名为Slab Allocator的机制分配、管理内存，Slab Allocator的基本原理是按照预先规定的大小，将分配的内存分割成特定长度的块，以期望完全解决内存碎片问题。而且，slab allocator还有重复使用已分配的内存的目的。 也就是说，分配到的内存不会释放，而是重复利用。
2.Slab Allocation的主要术语
Page 分配给Slab的内存空间,默认是1MB,分配给Slab之后根据slab的大小切分成chunk
Chunk 用于缓存记录的内存空间
Slab Class 特定大小的chunk的组
3.Slab初始化
在Memcached启动时候会调用slab的初始化代码(详见memcached.c中main函数调用slabs_init函数).
slabs_init函数声明:
1
2
3
4
5
6
7
|
/** Init the subsystem. 1st argument is the limit on no. of bytes to allocate,
0 if no limit. 2nd argument is the growth factor; each slab will use a chunk
size equal to the previous slab's chunk size times this factor.
3rd argument specifies if the slab allocator should allocate all memory
up front (if true), or allocate memory in chunks as it is needed (if false)
*/
void slabs_init(const size_t limit, const double factor, const bool prealloc);
|
其中limit表示memcached最大使用内存;factor表示slab中chunk size的增长因子,slab中chunk size的大小等于前一个slab的chunk size乘以factor;
memcached.c中main函数调用slabs_init函数:
1
|
slabs_init(settings.maxbytes, settings.factor, preallocate);
|
其中settings.maxbytes默认值为64M,启动memcached使用选项-m设置;settings.factor默认为1.25,启动memcached时候使用-f设置;preallocate指的是启动memcached的时候默认为每种类型slab预先分配一个page的内存,默认是false;
1
2
3
4
5
|
settings.maxbytes = 64 * 1024 * 1024; /* default is 64MB */
...
settings.factor = 1.25;
...
preallocate = false
|
slabs_init函数实现:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
|
/**
* Determines the chunk sizes and initializes the slab class descriptors
* accordingly.
*/
void slabs_init(const size_t limit, const double factor, const bool prealloc) {
int i = POWER_SMALLEST - 1;
//真实占用大小=对象大小+48
unsigned int size = sizeof(item) + settings.chunk_size;
mem_limit = limit;
//开启预分配,则首先将limit大小(默认64M)的内存全部申请
if (prealloc) {
/* Allocate everything in a big chunk with malloc */
mem_base = malloc(mem_limit);
if (mem_base != NULL) {
mem_current = mem_base;
mem_avail = mem_limit;
} else {
fprintf(stderr, "Warning: Failed to allocate requested memory in"
" one large chunk.\nWill allocate in smaller chunks\n");
}
}
//清空所有的slab
memset(slabclass, 0, sizeof(slabclass));
while (++i < POWER_LARGEST && size <= settings.item_size_max / factor) {
/* Make sure items are always n-byte aligned */
if (size % CHUNK_ALIGN_BYTES)
size += CHUNK_ALIGN_BYTES - (size % CHUNK_ALIGN_BYTES);
slabclass[i].size = size;
slabclass[i].perslab = settings.item_size_max / slabclass[i].size;
size *= factor;
if (settings.verbose > 1) {
fprintf(stderr, "slab class %3d: chunk size %9u perslab %7u\n",
i, slabclass[i].size, slabclass[i].perslab);
}
}
//最大chunksize的一个slab,chunksize为settings.item_size_max(默认1M)
power_largest = i;
slabclass
|
