mysql内存管理
1 内存管理结构
mysql有自己的内存申请和释放机制
mysql层有mem_root
innodb层有mem_heap,mem_pool,buf_pool
它们的结构图如下
2 mem_root
mem_root是mysql层的动态内存管理
typedef struct st_used_mem { /* struct for once_alloc (block) */ struct st_used_mem *next; /* Next block in use */ unsigned int left; /* memory left in block */ unsigned int size; /* size of block */ } USED_MEM; typedef struct st_mem_root { USED_MEM *free; /* blocks with free memory in it */ USED_MEM *used; /* blocks almost without free memory */ USED_MEM *pre_alloc; /* preallocated block */ /* if block have less memory it will be put in 'used' list */ size_t min_malloc; size_t block_size; /* initial block size */ unsigned int block_num; /* allocated blocks counter */ /* first free block in queue test counter (if it exceed MAX_BLOCK_USAGE_BEFORE_DROP block will be dropped in 'used' list) */ unsigned int first_block_usage; void (*error_handler)(void); } MEM_ROOT;
mem_root主要由两个链表组成free和used。
free：存放有空闲的block，其中的block可能部分被使用。
used：存放已使用的block，其中block可能有少量空闲.
block_size：初始块大小,后面分配的块可能比block_size大。
pre_alloc：一直指向初始化时分配的block，改block开始在free链表中，随着内存的不断申请，改block可能会存在于used链表中
2.1 init_alloc_root
初始化一个block放入free链表，pre_alloc指向这个链表
2.2 alloc_root
首先从free链表中查找，如果有空间合适的block，则直接使用，若该block剩余空间小于min_malloc，则会放入used链表头部。如果没有空间合适的block，则新分配block放入free链表尾部。
2.3 reset_root_defaults
1没有初始化的mem_root不需要reset
满足if (!mem_root->pre_alloc || mem_root->pre_alloc->size != size)才reset
2 只释放完全没有使用的空闲块。
2.4 free_root
MY_MARK_BLOCKS_FREE:释放所有块
MY_KEEP_PREALLOC：只保留pre_alloc块
3 mem_pool
内存池。innodb早期自己实现的一套伙伴算法内存管理系统，由参数innodb_use_sys_malloc 控制开启和关闭，默认关闭，即直接从操作系统申请和释放内存.参见http://docs.oracle.com/cd/E17952_01/refman-5.5-en/innodb-performance-use_sys_malloc.html
mem_pool_t
/** Data structure for a memory pool. The space is allocated using the buddy algorithm, where free list i contains areas of size 2 to power i. */ struct mem_pool_t{ byte* buf; /*!< memory pool */ ulint size; /*!< memory common pool size */ ulint reserved; /*!< amount of currently allocated memory */ ib_mutex_t mutex; /*!< mutex protecting this struct */ UT_LIST_BASE_NODE_T(mem_area_t) free_list[64]; /*!< lists of free memory areas: an area is put to the list whose number is the 2-logarithm of the area size */ }; /** Memory area header */ struct mem_area_t{ ulint size_and_free; /*!< memory area size is obtained by anding with ~MEM_AREA_FREE; area in a free list if ANDing with MEM_AREA_FREE results in nonzero */ UT_LIST_NODE_T(mem_area_t) free_list; /*!< free list node */ };
buf:一片连续的内存区域
mem_area_t：内存块, size_and_free标记内存块的大小和是否已使用。
free_list[64]： 64个桶，每个桶编号为0,1,2,3,…,i;每个桶依次存放2i大小的内存块。
3.1初始化
mem_pool_create
在服务启动时调用一次
如初始化一个19字节的内存池
19=100011=24+21+20
3.2 mem_area_alloc
void* mem_area_alloc( /*===========*/ ulint* psize, /*!< in: requested size in bytes; for optimum space usage, the size should be a power of 2 minus MEM_AREA_EXTRA_SIZE; out: allocated size in bytes (greater than or equal to the requested size) */ mem_pool_t* pool)
从pool中申请内存块，psize<=2i,i为满足此条件的最小值。当第i个中存在空闲时，直接取出。当第i个中不存在空闲块时，从i+1个桶中切割一半到第i个桶中，这是一个递归的过程。具体实现在mem_pool_fill_free_list；
以申请大小为3的内存为例，（为了说明方便，这里不考虑块中MEM_AREA_EXTRA_SIZE的空间）
申请的得到大小为4的块，这是内存池中的结构如下
3.3 mem_pool_free
释放一个块到pool中，将块放入对应大小的桶i中，如果桶i中存在自己的伙伴（和自己相邻的内存块，可能在左边也可能在右边），则合并放入i+1的桶中，此过程递归，直到没有可以合并的伙伴。
找自己伙伴的函数如下
mem_area_t* mem_area_get_buddy( /*===============*/ mem_area_t* area, /*!< in: memory area */ ulint size, /*!< in: memory area size */ mem_pool_t* pool) /*!< in: memory pool */ { mem_area_t* buddy; ut_ad(size != 0); if (((((byte*) area) - pool->buf) % (2 * size)) == 0) { /* The buddy is in a higher address */ buddy = (mem_area_t*)(((byte*) area) + size); if ((((byte*) buddy) - pool->buf) + size > pool->size) { /* The buddy is not wholly contained in the pool: there is no buddy */ buddy = NULL; } } else { /* The buddy is in a lower address; NOTE that area cannot be at the pool lower end, because then we would end up to the upper branch in this if-clause: the remainder would be 0 */ buddy = (mem_area_t*)(((byte*) area) - size); } return(buddy); }
找x的伙伴：
1 首先确定x的伙伴在高位还是在低位，x左边的块大小都>=x的大小，且x左边的块大小=2i*sizeof(x) (i>0);。
如果x的伙伴是y,则(c-a)%(2*sizeof(x))=sizeof(x) != 0
如果x的伙伴是z,则(c-a)%(2*sizeof(x))= 0
需要说明的是：在桶中的块都是空闲块。
以释放刚申请得的大小为4的块为例，释放后，内存结构如下，回到了初始状态。
3.4 mem_pool_free
在服务关闭时调用一次
释放buf和mem_pool结构即可
mem_pool_free( /*==========*/ mem_pool_t* pool) /*!< in, own: memory pool */ { ut_free(pool->buf); ut_free(pool); }
4 buf_pool
块/页缓存池。
4.1buf_pool_init
Innodb_buffer_pool_instances指定了bp的个数，innodb_buffer_pool_size指定所有bp的总大小。
buf_pool_init在服务启动时调用一次
bp初始化主要是申请内存，并初始化free链表。每个buf_page_t的大小为innodb_page_size.
UT_LIST_BASE_NODE_T(buf_page_t) free;
4.2 buf_block_alloc
从free链表头部摘出
不放入LRU链表
4.3 buf_block_free
放回free链表头部
4.4 buffer_pool_free
在服务关闭时调用一次
5 mem_heap
/** The info structure stored at the beginning of a heap block */ struct mem_block_info_t { ulint magic_n;/* magic number for debugging */ char file_name[8];/* file name where the mem heap was created */ ulint line; /*!< line number where the mem heap was created */ UT_LIST_BASE_NODE_T(mem_block_t) base; /* In the first block in the the list this is the base node of the list of blocks; in subsequent blocks this is undefined */ UT_LIST_NODE_T(mem_block_t) list; /* This contains pointers to next and prev in the list. The first block allocated to the heap is also the first block in this list, though it also contains the base node of the list. */ ulint len; /*!< physical length of this block in bytes */ ulint total_size; /*!< physical length in bytes of all blocks in the heap. This is defined only in the base node and is set to ULINT_UNDEFINED in others. */ ulint type; /*!< type of heap: MEM_HEAP_DYNAMIC, or MEM_HEAP_BUF possibly ORed to MEM_HEAP_BTR_SEARCH */ ulint free; /*!< offset in bytes of the first free position for user data in the block */ ulint start; /*!< the value of the struct field 'free' at the creation of the block */ #ifndef UNIV_HOTBACKUP void* free_block; /* if the MEM_HEAP_BTR_SEARCH bit is set in type, and this is the heap root, this can contain an allocated buffer frame, which can be appended as a free block to the heap, if we need more space; otherwise, this is NULL */ void* buf_block; /* if this block has been allocated from the buffer pool, this contains the buf_block_t handle; otherwise, this is NULL */ #endif /* !UNIV_HOTBACKUP */ #ifdef MEM_PERIODIC_CHECK UT_LIST_NODE_T(mem_block_t) mem_block_list; /* List of all mem blocks allocated; protected by the mem_comm_pool mutex */ #endif };
innodb绝大多数内存申请和释放都是在mem_heap上进行的
mem_heap实际上是一个内存块链表，内存块大小依次增长，至少是两倍的增长。
5.1 mem_heap_create
初始内存块链表，根据参数初始化内存块，此时base链表只有一个内存块
if (heap && heap->magic_n != MEM_BLOCK_MAGIC_N) { mem_analyze_corruption(heap); } /* In dynamic allocation, calculate the size: block header + data. */ len = MEM_BLOCK_HEADER_SIZE + MEM_SPACE_NEEDED(n); #ifndef UNIV_HOTBACKUP if (type == MEM_HEAP_DYNAMIC || len < UNIV_PAGE_SIZE / 2) { ut_ad(type == MEM_HEAP_DYNAMIC || n <= MEM_MAX_ALLOC_IN_BUF); block = static_cast<mem_block_t*>( mem_area_alloc(&len, mem_comm_pool)); } else { len = UNIV_PAGE_SIZE; if ((type & MEM_HEAP_BTR_SEARCH) && heap) { /* We cannot allocate the block from the buffer pool, but must get the free block from the heap header free block field */ buf_block = static_cast<buf_block_t*>(heap->free_block); heap->free_block = NULL; if (UNIV_UNLIKELY(!buf_block)) { return(NULL); } } else { buf_block = buf_block_alloc(NULL); } block = (mem_block_t*) buf_block->frame; } ut_ad(block); block->buf_block = buf_block; block->free_block = NULL; #else /* !UNIV_HOTBACKUP */ len = MEM_BLOCK_HEADER_SIZE + MEM_SPACE_NEEDED(n); block = ut_malloc(len); ut_ad(block); #endif /* !UNIV_HOTBACKUP */
以上代码可以看出
Mem_heap的内存来源可以有三种
1 mem_pool 的mem_area_alloc
2 buffer_pool的buf_block_alloc
3 ut_malloc ->malloc
5.2 mem_heap_alloc
从base链表最后一个内存块分配内存，不够则新分配一个较大内存块，放入base链表的最后。
5.3 mem_heap_free_heap_top
void mem_heap_free_heap_top( /*===================*/ mem_heap_t* heap, /*!< in: heap from which to free */ byte* old_top)/*!< in: pointer to old top of heap */
释放base链表从old_top到最后的内存块，old_top不释放
5.4 mem_heap_free_heap_top
释放base链表最后一个内存块
5.5 mem_heap_free
释放base链表所有内存块
