Linux进程管理 (2)CFS调度器
关键词：
目录：
根据进程的特性可以将进程划分为：交互式进程、批处理进程、实时进程。
O(N)调度器从就绪队列中比较所有进程的优先级，然后选择一个最高优先级的进程作为下一个调度进程。每个进程都一个固定时间片，当进程时间片用完之后，调度器会选择下一个调度进程，当所有进程都运行一遍后再重新分配时间片。调度器选择下一个调度进程前需要遍历整个就绪队列，花费O(N)时间。
O(1)调度器优化了选择下一个进程的时间，它为每个CPU维护一组进程优先级队列，每个优先级一个队列，这样在选择下一个进程时，只需查询优先级队列相应的位图即可知道哪个队列中有酒须进程，查询时间为常数O(1)。
Linux定义了5种调度器类，分别对应stop、deadline、realtime、cfs、idle，他们通过next串联起来。
const struct sched_class stop_sched_class = { .next = &dl_sched_class, ... }; const struct sched_class dl_sched_class = { .next = &rt_sched_class, ... }; const struct sched_class rt_sched_class = { .next = &fair_sched_class, ... }; const struct sched_class fair_sched_class = { .next = &idle_sched_class, ... } const struct sched_class idle_sched_class = { /* .next is NULL */ ... }; /* * Scheduling policies */ #define SCHED_NORMAL 0 #define SCHED_FIFO 1 #define SCHED_RR 2 #define SCHED_BATCH 3 /* SCHED_ISO: reserved but not implemented yet */ #define SCHED_IDLE 5 #define SCHED_DEADLINE 6
同时定义了6中调度策略，3中调度实体，他们之间的关系如下表。
| 调度器类 | 调度策略 | 调度实体 | 优先级 |
| stop_sched_class | |||
| dl_sched_class | SCHED_DEADLINE | sched_dl_entity | (, 0) |
| rt_sched_class |
SCHED_FIFO SCHED_RR |
sched_rt_entity | [0, 100) |
| fair_sched_class |
SCHED_NORMAL SCHED_BATCH |
sched_entity | [100, ) |
| idle_sched_class | SCHED_IDLE |
实时调度类相关参考《实时调度类分析，以及FIFO和RR对比实验》。
1. 权重计算
1.1 计算优先级
计算优先级之前，首先要明白struct task_struct中各个关于优先级成员的含义。
struct task_struct { ... int prio, static_prio, normal_prio; unsigned int rt_priority; ... unsigned int policy; ... };
prio：保存进程动态优先级，系统根据prio选择调度类，有些情况需要暂时提高进程优先级。
static_prio：静态优先级，在进程启动时分配。内核不保存nice值，通过PRIO_TO_NICE根据task_struct->static_prio计算得到。这个值可以通过nice/renice或者setpriority()修改。
normal_prio：是基于static_prio和调度策略计算出来的优先级，在创建进程时会继承父进程normal_prio。对普通进程来说，normal_prio等于static_prio；对实时进程，会根据rt_priority重新计算normal_prio。
rt_priority：实时进程的优先级，和进程设置参数sched_param.sched_priority等价。
nice/renice系统调用可以改变static_prio值。
rt_priority在普通进程中等于0，实时进程中范围是1~99。
normal_prio在普通进程中等于static_prio；在实时进程中normal_prio=99-rt_priority。
获取normal_prio的函数是normal_prio()
static inline int __normal_prio(struct task_struct *p) { return p->static_prio; } static inline int normal_prio(struct task_struct *p) { int prio; if (task_has_dl_policy(p)) prio = MAX_DL_PRIO-1;-----------------------------------对于DEADLINE类进程来说固定值为-1。 else if (task_has_rt_policy(p)) prio = MAX_RT_PRIO-1 - p->rt_priority;------------------对于实时进程来说，normal_prio=100-1-rt_priority else prio = __normal_prio(p);--------------------------------对普通进程来说normal_prio=static_prio return prio; }
prio在普通进程中和static_prio相等；在实时进程中prio和rt_priority存在prio+rt_priority=99关系。
获取prio的函数是effective_prio()。
static int effective_prio(struct task_struct *p) { p->normal_prio = normal_prio(p); /* * If we are RT tasks or we were boosted to RT priority, * keep the priority unchanged. Otherwise, update priority * to the normal priority: */ if (!rt_prio(p->prio))-------------------即prio大于99的情况，此时为普通进程，prio=normal_prio=static_prio。 return p->normal_prio; return p->prio; }
普通进程：static_prio=prio=normal_prio；rt_priority=0。
实时进程：prio=normal_prio=99-rt_priority；rt_priority=sched_param.sched_priority，rt_priority=[1, 99]；static_prio保持默认值不改变。
static_prio和nice之间的关系
内核使用0~139数值表示优先级，数值越低优先级越高。其中0~99给实时进程使用，100~139给普通进程(SCHED_NORMAL/SCHED_BATCH)使用。
用户空间nice传递的变量映射到普通进程优先级，即100~139。
关于nice和prio之间的转换，内核提供NICE_TO_PRIO和PRIO_TO_NICE两个宏。
#define MAX_USER_RT_PRIO 100 #define MAX_RT_PRIO MAX_USER_RT_PRIO #define MAX_PRIO (MAX_RT_PRIO + NICE_WIDTH) #define DEFAULT_PRIO (MAX_RT_PRIO + NICE_WIDTH / 2) /* * Convert user-nice values [ -20 ... 0 ... 19 ] * to static priority [ MAX_RT_PRIO..MAX_PRIO-1 ], * and back. */ #define NICE_TO_PRIO(nice) ((nice) + DEFAULT_PRIO) #define PRIO_TO_NICE(prio) ((prio) - DEFAULT_PRIO) /* * 'User priority' is the nice value converted to something we * can work with better when scaling various scheduler parameters, * it's a [ 0 ... 39 ] range. */ #define USER_PRIO(p) ((p)-MAX_RT_PRIO) #define TASK_USER_PRIO(p) USER_PRIO((p)->static_prio) #define MAX_USER_PRIO (USER_PRIO(MAX_PRIO))
1.2 计算权重
内核中使用struct load_weight数据结构来记录调度实体的权重信息。
权重信息是根据优先级来计算的，通过task_struct->se.load来获取进程的权重信息。
因为权重仅适用于普通进程，普通进程的nice对应范围是-20~19。
struct task_struct { ... struct sched_entity se; ... };
struct sched_entity { struct load_weight load; /* for load-balancing */ ... }; struct load_weight { unsigned long weight;----------------调度实体的权重 u32 inv_weight;----------------------inverse weight，是全中一个中间计算结果。 };
set_load_weight()设置进程的权重值，通过task_struct->static_prio从prio_to_weight[]和prio_to_wmult[]获取。
static void set_load_weight(struct task_struct *p) { int prio = p->static_prio - MAX_RT_PRIO;---------------------权重值取决于static_prio，减去100而不是120，对应了下面数组下标。 struct load_weight *load = &p->se.load; /* * SCHED_IDLE tasks get minimal weight: */ if (p->policy == SCHED_IDLE) { load->weight = scale_load(WEIGHT_IDLEPRIO);-------------IDLE调度策略进程使用固定优先级权重，取最低普通优先级权重的1/5。 load->inv_weight = WMULT_IDLEPRIO;----------------------取最低普通优先级反转权重的5倍。 return; } load->weight = scale_load(prio_to_weight[prio]); load->inv_weight = prio_to_wmult[prio]; }
nice从-20~19，共40个等级，nice值越高优先级越低。
进程每提高一个优先级，则增加10%CPU时间，同时另一个进程减少10%时间，他们之间的关系从原来的1:1变成了1.1:0.9=1.22。
因此相同优先级之间的关系使用系统1.25来表示。
假设A和B进程nice都为0，权重都是1024.
A的nice变为1，B不变。那么B获得55%运行时间，A获得45%运行时间。A的权重就变成了A/(A+1024)=9/(9+11)，A=1024*9/11=838。
但是Linux并不是严格按照1.22系数来计算的，而是近似1.25。
A的权重值就变成了1024/1.25≈820。
prio_to_weight[]以nice-0为基准权重1024，然后将nice从-20~19预先计算出。set_load_weight()就可以通过优先级得到进程对应的权重。
prio_to_wmult[]为了方便计算vruntime而预先计算结果。
inv_weight=232/weight
static const int prio_to_weight[40] = { /* -20 */ 88761, 71755, 56483, 46273, 36291, /* -15 */ 29154, 23254, 18705, 14949, 11916, /* -10 */ 9548, 7620, 6100, 4904, 3906, /* -5 */ 3121, 2501, 1991, 1586, 1277, /* 0 */ 1024, 820, 655, 526, 423, /* 5 */ 335, 272, 215, 172, 137, /* 10 */ 110, 87, 70, 56, 45, /* 15 */ 36, 29, 23, 18, 15, }; static const u32 prio_to_wmult[40] = { /* -20 */ 48388, 59856, 76040, 92818, 118348, /* -15 */ 147320, 184698, 229616, 287308, 360437, /* -10 */ 449829, 563644, 704093, 875809, 1099582, /* -5 */ 1376151, 1717300, 2157191, 2708050, 3363326, /* 0 */ 4194304, 5237765, 6557202, 8165337, 10153587, /* 5 */ 12820798, 15790321, 19976592, 24970740, 31350126, /* 10 */ 39045157, 49367440, 61356676, 76695844, 95443717, /* 15 */ 119304647, 148102320, 186737708, 238609294, 286331153, };
1.2.1 优先级和权重关系实验
下面设计一下CPU intensive的进程，然后设置不同优先级，再使用top查看他们实际得到的CPU执行事件。
这样就可以验证他们的优先级和权重关系。
首先需要将这些进程固定到一个CPU上，然后调整优先级。
#define _GNU_SOURCE #include <stdio.h> #include <sched.h> #include <stdlib.h> #include <unistd.h> #include <sys/resource.h> int main(void) { int i, pid; cpu_set_t mask; //Set CPU affinity. CPU_ZERO(&mask); CPU_SET(0, &mask); if(sched_setaffinity(0, sizeof(cpu_set_t), &mask) == -1) { exit(EXIT_FAILURE); } pid = getpid(); setpriority(PRIO_PROCESS, pid, -20); while(1) { } return 0; }
1.2.1.1 -20和-19关系
理论上-20和-19的CPU占比应该是88761:71755=1.24:1=55.3/44.7。
来看一下实际执行效果，符合预期。
通过kernelshark查看一下他们之间的关系，两个进程之间是有规律的。
-20执行了2+3tick，-19执行了2+2tick，两者之间的比例也接近1.25。符合预期。
1.2.1.2 -20、-19、-18三者关系呢？
这三者之间的比例关系应该是88761:71755:56483=1.57:1.27:1。
实际结果是41.2:32.9:25.9=1.59:1.27:1，符合预期。
1.2.1.3 -20、-19、-18、0四者关系呢？
88761:71755:56483:1024=1.57:1.27:1:0.018
实际结果是40.8:33.0:25.8:0.4=1.58:1.28:1:0.016，基本符合预期。
为什么不以nice-0为基准呢？首先在-20、-19、-18存在的情况下，nice-0的误差显得特别大，另一个系统还存在其它很多nice-0的进程。
1.3 计算vrtime
CFS中所谓的Fair是vrtime的，而不是实际时间的平等。
CFS调度器抛弃以前固定时间片和固定调度周期的算法，而采用进程权重值的比重来量化和计算实际运行时间。
引入虚拟时钟概念，每个进程虚拟时间是实际运行时间相对Nice值为0的权重比例值。
Nice值小的进程，优先级高且权重大，其虚拟时钟比真实时钟跑得慢，所以可以获得更多的实际运行时间。
反之，Nice值大的进程，优先级低权重小，获得的实际运行时间也更少。
CFS选择虚拟时钟跑得慢的进程，而不是实际运行时间运行的少的进程。
vruntime=delta_exec*nice_0_weight/weight
vruntime表示进程的虚拟运行时间，delta_exec表示进程实际运行时间，nice_0_weight表示nice为0权重值，weight表示该进程的权重值，可以通过prio_to_weight[]获取。
vruntime=delta_exec*nice_0_weight*232/weight>>32
其中232/weight可以用inv_weight来表示，其中inv_weight可以从prio_to_wmult[]中获取。
vruntime=delta_exec*nice_0_weight*inv_weight>>32
calc_delta_fair()是计算虚拟时间的函数，其返回值是虚拟时间。
__calc_delta()是计算vruntime的核心，delta_exec是进程实际运行时间，weight是nice_0_weight，lw是对应进程的load_weight，里面包含了其inv_weight值。
static inline u64 calc_delta_fair(u64 delta, struct sched_entity *se) { if (unlikely(se->load.weight != NICE_0_LOAD))----如果当前进程权重是NICE_0_WEIGHT，虚拟时间就是delta，不需要__calc_delta()计算。 delta = __calc_delta(delta, NICE_0_LOAD, &se->load); return delta; } static u64 __calc_delta(u64 delta_exec, unsigned long weight, struct load_weight *lw) { u64 fact = scale_load_down(weight);--------------fact等于weight。 int shift = WMULT_SHIFT;-------------------------WMULT_SHIFT等于32 __update_inv_weight(lw);-------------------------更新load_weight->inv_weight，一般情况下已经设置，不需要进行操作。 if (unlikely(fact >> 32)) {----------------------一般fact>>32为0，所以跳过 while (fact >> 32) { fact >>= 1; shift--; } } /* hint to use a 32x32->64 mul */ fact = (u64)(u32)fact * lw->inv_weight;----------此处相当于nice_0_weight*inv_weight while (fact >> 32) { fact >>= 1; shift--; } return mul_u64_u32_shr(delta_exec, fact, shift);----此处相当于delta_exec*(nice_0_weight*inv_weight)>>32。 }
优先级越低的进程inv_weight值越大，其它nice_0_weight和位置都是一样的。
所以相同的delta_exec情况下，优先级越低vruntime越大。
cfs总是在红黑树中选择vrunime最小的进程进行调度，优先级高的进程在相同实际运行时间的情况下vruntime最小，所以总会被优先选择。但是随着vruntime的增长，优先级低的进程也有机会运行。
1.4 负载计算
内核中计算CPU负载的方法是PELT(Per-Entity Load Tracing)，不仅考虑进程权重，而且跟踪每个调度实体的负载情况。
sched_entity结构中有一个struct sched_avg用于描述进程的负载。
runnable_sum：表示该调度实体在就绪队列里(sched_entity->on_rq==1)可运行状态的总时间。包括两部分，一是正在运行的时间，即running时间；二是在就绪队列中等待的时间。进程进入就绪队列时(调用enqueue_entity())，on_rq被置为1，但该进程因为睡眠等原因退出就绪队列时(调用dequeue_entity())，on_rq会被清0，因此runnable_sum就是统计进程在就绪队列的时间。
runnable_period：可以理解为该调度实体在系统中的总时间，period是指一个周期period为1024us。当一个进程fork出来后，无论是否在就绪队列中，runnable_period一直在递增。
runnable_avg_sum：考虑历史数据对负载的影响，采用衰减系统来计算平均复杂，调度实体在就绪队列里可运行状态下总的衰减累加时间。
runnable_avg_period：调度实体在系统中总的衰减累加时间。
last_runnable_update：最近更新load的时间点，用于计算时间间隔。
load_avg_contrib：进程平均负载的贡献度。
struct sched_avg { /* * These sums represent an infinite geometric series and so are bound * above by 1024/(1-y). Thus we only need a u32 to store them for all * choices of y < 1-2^(-32)*1024. */ u32 runnable_avg_sum, runnable_avg_period; u64 last_runnable_update; s64 decay_count; unsigned long load_avg_contrib; }; struct sched_entity { ... #ifdef CONFIG_SMP /* Per entity load average tracking */ struct sched_avg avg; #endif }
1.4.1 衰减因子
将1024us时间跨度算成一个周期，period简称PI。
一个PI周期内对系统负载的贡献除了权重外，还有PI周期内可运行的时间，包括运行时间或等待CPU时间。
一个理想的计算方式是：统计多个实际的PI周期，并使用一个衰减系数来计算过去的PI周期对付贼的贡献。
Li是一个调度实体在第i个周期内的负载贡献。那么一个调度实体负载综合计算公式如下：
L=L0+L1*y+L2*y2+L3*y3...+L32*y32+...
调度实体的负载需要考虑时间因素，不能只考虑当前负载，还要考虑其在过去一段时间表现。
一般认为过去第32个周期的负载减半，所以y32=0.5，得出衰减因子y=0.978左右。
同时内核不需要数组来存放过去PI个周期负载贡献，只需要用过去周期贡献总和乘以衰减系数y，并加上当前时间点的负载L0即可。
下表对衰减因子乘以232，计算完成后再右移32位。如下，就将原来衰减因子的浮点元算转换成乘法和移位操作。
L*yn = (L*yn*232)>>32 = (L*(0.978)n*232)>>32 = L*runnable_avg_yN_inv[n]>>32
runnable_avg_yN_inv[n]是计算第n个周期的衰减值，在实际使用中需要计算n个周期的负载累积贡献值。
runnable_avg_yN_sum[n] = 1024*(y + y2 + y3 + ... + yn)
取1024是因为一个周期是1024微秒。
下面两个数组虽然都是计算负载累计，但是runnable_avg_yN_inv[]使计算某一个周期的贡献值，runnable_avg_yN_sum[n]是计算n个周期的贡献值。
runnable_avg_yN_inv[]共32个成员，runnable_avg_yN_sum[]共33个成员。
static const u32 runnable_avg_yN_inv[] = { 0xffffffff, 0xfa83b2da, 0xf5257d14, 0xefe4b99a, 0xeac0c6e6, 0xe5b906e6, 0xe0ccdeeb, 0xdbfbb796, 0xd744fcc9, 0xd2a81d91, 0xce248c14, 0xc9b9bd85, 0xc5672a10, 0xc12c4cc9, 0xbd08a39e, 0xb8fbaf46, 0xb504f333, 0xb123f581, 0xad583ee9, 0xa9a15ab4, 0xa5fed6a9, 0xa2704302, 0x9ef5325f, 0x9b8d39b9, 0x9837f050, 0x94f4efa8, 0x91c3d373, 0x8ea4398a, 0x8b95c1e3, 0x88980e80, 0x85aac367, 0x82cd8698, }; static const u32 runnable_avg_yN_sum[] = { 0, 1002, 1982, 2941, 3880, 4798, 5697, 6576, 7437, 8279, 9103, 9909,10698,11470,12226,12966,13690,14398,15091,15769,16433,17082, 17718,18340,18949,19545,20128,20698,21256,21802,22336,22859,23371, };
这两个参数分别对应decay_load()和__compute_runnable_contrib()。
decay_load()根据一个load值和周期序号n，返回衰减后的load值。
__compute_runnable_contrib()只有一个参数过去的periods周期数目，返回累计衰减load值。
1.4.2 update_entity_load_avg()
update_entity_load_avg()主要更新struct sched_avg结构体成员，其中__update_entity_runnable_avg()更新了last_runnable_update、runnable_avg_sum和runnable_avg_period三个数据；
__update_entity_load_avg_contrib()更新了load_avg_contrib；最后同时更新了cfs_rq->runnable_load_avg。
static inline void update_entity_load_avg(struct sched_entity *se, int update_cfs_rq) { struct cfs_rq *cfs_rq = cfs_rq_of(se); long contrib_delta; u64 now; /* * For a group entity we need to use their owned cfs_rq_clock_task() in * case they are the parent of a throttled hierarchy. */ if (entity_is_task(se)) now = cfs_rq_clock_task(cfs_rq); else now = cfs_rq_clock_task(group_cfs_rq(se)); if (!__update_entity_runnable_avg(now, &se->avg, se->on_rq))-------更新sched_avg的三个参数：last_runnable_update、runnable_avg_sum、runnable_avg_period。如果上次更新到本次不足1024
us，不做衰减计算，不计算负载贡献度。 return; contrib_delta = __update_entity_load_avg_contrib(se);--------------计算本次更新贡献度，更新到load_avg_contrib中。 if (!update_cfs_rq) return; if (se->on_rq) cfs_rq->runnable_load_avg += contrib_delta;--------------------累加到cfs_rq->runnable_laod_avg中 else subtract_blocked_load_contrib(cfs_rq, -contrib_delta); } static __always_inline int __update_entity_runnable_avg(u64 now, struct sched_avg *sa, int runnable)--------------------------------runnable表示该进程是否在就绪队列上接受调度 { u64 delta, periods; u32 runnable_contrib; int delta_w, decayed = 0; delta = now - sa->last_runnable_update;------------------------------上次更新负载到本次更新的间隔，单位是ns。 /* * This should only happen when time goes backwards, which it * unfortunately does during sched clock init when we swap over to TSC. */ if ((s64)delta < 0) { sa->last_runnable_update = now; return 0; } /* * Use 1024ns as the unit of measurement since it's a reasonable * approximation of 1us and fast to compute. */ delta >>= 10;--------------------------------------------------------delta单位变成近似1微秒 if (!delta) return 0; sa->last_runnable_update = now; /* delta_w is the amount already accumulated against our next period */ delta_w = sa->runnable_avg_period % 1024;----------------------------runnable_avg_period是上一次更新时的总周期数，delta_w是上一次周周期数不能凑成一个周期的剩余时间，单位是微秒。 if (delta + delta_w >= 1024) {---------------------------------------如果时间大于一个周期，就需要进行衰减计算。 /* period roll-over */ decayed = 1; /* * Now that we know we're crossing a period boundary, figure * out how much from delta we need to complete the current * period and accrue it. */ delta_w = 1024 - delta_w; if (runnable) sa->runnable_avg_sum += delta_w; sa->runnable_avg_period += delta_w; delta -= delta_w; /* Figure out how many additional periods this update spans */ periods = delta / 1024;---------------------------------------------本次更新和上次更新之间经历的周期数periods delta %= 1024; sa->runnable_avg_sum = decay_load(sa->runnable_avg_sum, periods + 1);-------------------------------------分别计算第periods+1个周期的runnable_avg_sum和runnable_avg_period的衰减。 sa->runnable_avg_period = decay_load(sa->runnable_avg_period, periods + 1); /* Efficiently calculate \sum (1..n_period) 1024*y^i */ runnable_contrib = __compute_runnable_contrib(periods);-------------得到过去periods个周期的累计衰减。 if (runnable) sa->runnable_avg_sum += runnable_contrib; sa->runnable_avg_period += runnable_contrib; } /* Remainder of delta accrued against u_0` */---------------------------不能凑成完成周期的部分直接进行相加。 if (runnable) sa->runnable_avg_sum += delta; sa->runnable_avg_period += delta; return decayed;---------------------------------------------------------decayed表示是否进行了衰减计算 } static __always_inline u64 decay_load(u64 val, u64 n)----------------------val表示n个周期前的负载值，n表示第n个周期。返回结果为val*yn，变成查表(val*runnable_avg_yN_inv[n])>>32。 { unsigned int local_n; if (!n) return val;---------------------------------------------------------n=0：表示当前周期，不衰减。 else if (unlikely(n > LOAD_AVG_PERIOD * 63))----------------------------n>=2016：LOAD_AVG_PERIOD=32，因此n超过2016就认为衰减值变为0。 return 0; /* after bounds checking we can collapse to 32-bit */ local_n = n; /* * As y^PERIOD = 1/2, we can combine * y^n = 1/2^(n/PERIOD) * y^(n%PERIOD) * With a look-up table which covers y^n (n<PERIOD) * * To achieve constant time decay_load. */ if (unlikely(local_n >= LOAD_AVG_PERIOD)) {-----------------------------32=<n<2016：每32个周期衰减1/2，即val右移一位。剩下周期数存入local_n。 val >>= local_n / LOAD_AVG_PERIOD; local_n %= LOAD_AVG_PERIOD; } val *= runnable_avg_yN_inv[local_n];------------------------------------0<n<32：根据local_n查表得到衰减值 /* We don't use SRR here since we always want to round down. */ return val >> 32;-------------------------------------------------------最终结果右移32位，归一化。 } static u32 __compute_runnable_contrib(u64 n) { u32 contrib = 0; if (likely(n <= LOAD_AVG_PERIOD))--------------------------------------n<=32：直接查表得到结果。 return runnable_avg_yN_sum[n]; else if (unlikely(n >= LOAD_AVG_MAX_N))--------------------------------n>=345：直接取最大值47742，这个值也是一共345个周期的累计衰减。 return LOAD_AVG_MAX; /* Compute \Sum k^n combining precomputed values for k^i, \Sum k^j */ do {-------------------------------------------------------------------以LOAD_AVG_PERIOD为步长，计算过去n/32个32周期的累计衰减 contrib /= 2; /* y^LOAD_AVG_PERIOD = 1/2 */ contrib += runnable_avg_yN_sum[LOAD_AVG_PERIOD];-------------------都取n=32的情况 n -= LOAD_AVG_PERIOD; } while (n > LOAD_AVG_PERIOD); contrib = decay_load(contrib, n);-------------------------------------还需经过n过周期衰减，因此经过decay_load()得到过去“n/32个32周期”的最终累计衰减。 return contrib + runnable_avg_yN_sum[n];------------------------------不能凑成32周期单独计算并和contrib累加得到最终的结果。 } static long __update_entity_load_avg_contrib(struct sched_entity *se) { long old_contrib = se->avg.load_avg_contrib; if (entity_is_task(se)) { __update_task_entity_contrib(se); } else { __update_tg_runnable_avg(&se->avg, group_cfs_rq(se)); __update_group_entity_contrib(se); } return se->avg.load_avg_contrib - old_contrib; } static inline void __update_task_entity_contrib(struct sched_entity *se) { u32 contrib; /* avoid overflowing a 32-bit type w/ SCHED_LOAD_SCALE */ contrib = se->avg.runnable_avg_sum * scale_load_down(se->load.weight); contrib /= (se->avg.runnable_avg_period + 1); se->avg.load_avg_contrib = scale_load(contrib);-------------------------更新sched_avg->load_avg_contrib }
load_avg_contrib = (runnable_avg_sum*weight)/runnable_avg_period
可见一个调度实体的平均负载和以下3个因素相关：
- 调度实体的权重weight
- 调度实体可运行状态下的总衰减累加时间runnable_avg_sum
- 调度实体在调度器中总衰减累加时间runnable_avg_period
runnable_avg_sum越接近runnable_avg_period，则平均负载越大，表示调度实体一直在占用CPU。
2. 进程创建
2.1 sched_entity、rq、cfs_rq
struct sched_entity内嵌在task_struct中，称为调度实体，描述进程作为一个调度实体参与调度的所需要的所有信息。
struct sched_entity { struct load_weight load; /* for load-balancing */----------------调度实体的权重。 struct rb_node run_node;--------------------------------------------表示调度实体在红黑树中的节点 struct list_head group_node; unsigned int on_rq;-------------------------------------------------表示该调度实体是否在就绪队列中接受调度 u64 exec_start; u64 sum_exec_runtime; u64 vruntime;---------------------------------------------------表示本调度实体的虚拟运行时间 u64 prev_sum_exec_runtime; u64 nr_migrations; #ifdef CONFIG_SCHEDSTATS struct sched_statistics statistics; #endif #ifdef CONFIG_FAIR_GROUP_SCHED int depth; struct sched_entity *parent; /* rq on which this entity is (to be) queued: */ struct cfs_rq *cfs_rq; /* rq "owned" by this entity/group: */ struct cfs_rq *my_q;-------------------------------------------------如果my_q不为null表示当前调度实体是调度组，而不是单个进程。 #endif #ifdef CONFIG_SMP /* Per-entity load-tracking */ struct sched_avg avg;-----------------------------------------------------表示调度实体平均负载信息。 #endif };
strcut sched_entity是per-task的，struct rq是per-cpu的。
系统中每个CPU就有一个struct rq数据结构，this_rq()可以获取当前CPU的就绪队列struct rq。
struct rq是描述CPU的通用就绪队列，rq数据结构记录了一个就绪队列所需要的全部信息，包括一个cfs就绪队列数据结构strct cfs_rq、一个实时调度器就绪队列数据结构struct rt_rq和一个deadline就绪队列数据结构structdl_rq。
struct rq { /* runqueue lock: */ raw_spinlock_t lock; /* * nr_running and cpu_load should be in the same cacheline because * remote CPUs use both these fields when doing load calculation. */ unsigned int nr_running;-------------------------------------运行进程个数 #ifdef CONFIG_NUMA_BALANCING unsigned int nr_numa_running; unsigned int nr_preferred_running; #endif #define CPU_LOAD_IDX_MAX 5 unsigned long cpu_load[CPU_LOAD_IDX_MAX]; unsigned long last_load_update_tick; #ifdef CONFIG_NO_HZ_COMMON u64 nohz_stamp; unsigned long nohz_flags; #endif #ifdef CONFIG_NO_HZ_FULL unsigned long last_sched_tick; #endif /* capture load from *all* tasks on this cpu: */ struct load_weight load;---------------------------------------就绪队列权重。 unsigned long nr_load_updates; u64 nr_switches; struct cfs_rq cfs;---------------------------------------------cfs就绪队列 struct rt_rq rt;-----------------------------------------------rt就绪队列 struct dl_rq dl;-----------------------------------------------deadline就绪队列 #ifdef CONFIG_FAIR_GROUP_SCHED /* list of leaf cfs_rq on this cpu: */ struct list_head leaf_cfs_rq_list; struct sched_avg avg; #endif /* CONFIG_FAIR_GROUP_SCHED */ /* * This is part of a global counter where only the total sum * over all CPUs matters. A task can increase this counter on * one CPU and if it got migrated afterwards it may decrease * it on another CPU. Always updated under the runqueue lock: */ unsigned long nr_uninterruptible; struct task_struct *curr, *idle, *stop; unsigned long next_balance; struct mm_struct *prev_mm; unsigned int clock_skip_update; u64 clock; u64 clock_task; atomic_t nr_iowait; #ifdef CONFIG_SMP struct root_domain *rd; struct sched_domain *sd; unsigned long cpu_capacity; unsigned char idle_balance; /* For active balancing */ int post_schedule; int active_balance; int push_cpu; struct cpu_stop_work active_balance_work; /* cpu of this runqueue: */ int cpu; int online; struct list_head cfs_tasks; u64 rt_avg; u64 age_stamp; u64 idle_stamp; u64 avg_idle; /* This is used to determine avg_idle's max value */ u64 max_idle_balance_cost; #endif #ifdef CONFIG_IRQ_TIME_ACCOUNTING u64 prev_irq_time; #endif #ifdef CONFIG_PARAVIRT u64 prev_steal_time; #endif #ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING u64 prev_steal_time_rq; #endif /* calc_load related fields */ unsigned long calc_load_update; long calc_load_active; #ifdef CONFIG_SCHED_HRTICK #ifdef CONFIG_SMP int hrtick_csd_pending; struct call_single_data hrtick_csd; #endif struct hrtimer hrtick_timer; #endif #ifdef CONFIG_SCHEDSTATS /* latency stats */ struct sched_info rq_sched_info; unsigned long long rq_cpu_time; /* could above be rq->cfs_rq.exec_clock + rq->rt_rq.rt_runtime ? */ /* sys_sched_yield() stats */ unsigned int yld_count; /* schedule() stats */ unsigned int sched_count; unsigned int sched_goidle; /* try_to_wake_up() stats */ unsigned int ttwu_count; unsigned int ttwu_local; #endif #ifdef CONFIG_SMP struct llist_head wake_list; #endif #ifdef CONFIG_CPU_IDLE /* Must be inspected within a rcu lock section */ struct cpuidle_state *idle_state; #endif }; struct cfs_rq { struct load_weight load;------------------------------------cfs就绪队列的权重 unsigned int nr_running, h_nr_running; u64 exec_clock; u64 min_vruntime;-------------------------------------------跟踪该就绪队列红黑树中最小的vruntime值。 #ifndef CONFIG_64BIT u64 min_vruntime_copy; #endif struct rb_root tasks_timeline;------------------------------运行队列红黑树根。 struct rb_node *rb_leftmost;--------------------------------红黑树最左边节点，也即为最小vruntime时间的节点，单进程选择下一个进程来运行时，就选择这个。 /* * 'curr' points to currently running entity on this cfs_rq. * It is set to NULL otherwise (i.e when none are currently running). */ struct sched_entity *curr, *next, *last, *skip; #ifdef CONFIG_SCHED_DEBUG unsigned int nr_spread_over; #endif #ifdef CONFIG_SMP /* * CFS Load tracking * Under CFS, load is tracked on a per-entity basis and aggregated up. * This allows for the description of both thread and group usage (in * the FAIR_GROUP_SCHED case). */ unsigned long runnable_load_avg, blocked_load_avg;----------runnable_load_avg跟踪该就绪队列中总平均负载。 atomic64_t decay_counter; u64 last_decay; atomic_long_t removed_load; #ifdef CONFIG_FAIR_GROUP_SCHED /* Required to track per-cpu representation of a task_group */ u32 tg_runnable_contrib; unsigned long tg_load_contrib; /* * h_load = weight * f(tg) * * Where f(tg) is the recursive weight fraction assigned to * this group. */ unsigned long h_load; u64 last_h_load_update; struct sched_entity *h_load_next; #endif /* CONFIG_FAIR_GROUP_SCHED */ #endif /* CONFIG_SMP */ #ifdef CONFIG_FAIR_GROUP_SCHED struct rq *rq; /* cpu runqueue to which this cfs_rq is attached */----本cfs_rq附着的struct rq /* * leaf cfs_rqs are those that hold tasks (lowest schedulable entity in * a hierarchy). Non-leaf lrqs hold other higher schedulable entities * (like users, containers etc.) * * leaf_cfs_rq_list ties together list of leaf cfs_rq's in a cpu. This * list is used during load balance. */ int on_list; struct list_head leaf_cfs_rq_list; struct task_group *tg; /* group that "owns" this runqueue */----------组调度数据结构 #ifdef CONFIG_CFS_BANDWIDTH int runtime_enabled; u64 runtime_expires; s64 runtime_remaining; u64 throttled_clock, throttled_clock_task; u64 throttled_clock_task_time; int throttled, throttle_count; struct list_head throttled_list; #endif /* CONFIG_CFS_BANDWIDTH */ #endif /* CONFIG_FAIR_GROUP_SCHED */ };
通过task_struct可以找到对应的cfs_rq，struct task_struct通过task_thread_info()找到thread_info；
通过struct thread_info得到cpu，通过cpu_rq()找到对应CPU的struct rq，进而找到对应的struct cfs_rq。
#define task_thread_info(task) ((struct thread_info *)(task)->stack) static inline unsigned int task_cpu(const struct task_struct *p) { return task_thread_info(p)->cpu; } DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); #define cpu_rq(cpu) (&per_cpu(runqueues, (cpu))) #define this_rq() this_cpu_ptr(&runqueues)--------------------当前CPU的struct rq #define task_rq(p) cpu_rq(task_cpu(p))----------------------指定CPU的struct rq static inline struct cfs_rq *task_cfs_rq(struct task_struct *p) { return &task_rq(p)->cfs; }
2.2 fair_sched_class
struct sched_class是调度类操作方法，CFS调度器的调度类fair_sched_class定义了CFS相关操作方法。
这些方法的具体介绍会在下面一一介绍。
const struct sched_class fair_sched_class = { .next = &idle_sched_class, .enqueue_task = enqueue_task_fair, .dequeue_task = dequeue_task_fair, .yield_task = yield_task_fair, .yield_to_task = yield_to_task_fair, .check_preempt_curr = check_preempt_wakeup, .pick_next_task = pick_next_task_fair, .put_prev_task = put_prev_task_fair, #ifdef CONFIG_SMP .select_task_rq = select_task_rq_fair, .migrate_task_rq = migrate_task_rq_fair, .rq_online = rq_online_fair, .rq_offline = rq_offline_fair, .task_waking = task_waking_fair, #endif .set_curr_task = set_curr_task_fair, .task_tick = task_tick_fair, .task_fork = task_fork_fair, .prio_changed = prio_changed_fair, .switched_from = switched_from_fair, .switched_to = switched_to_fair, .get_rr_interval = get_rr_interval_fair, .update_curr = update_curr_fair, #ifdef CONFIG_FAIR_GROUP_SCHED .task_move_group = task_move_group_fair, #endif };
2.3 进程创建
进程创建由do_fork()函数来完成，do_fork-->copy_process参与了进程调度相关初始化。
copy_process()
sched_fork()
__sched_fork()
fair_sched_class->task_fork()->task_fork_fair()
__set_task_cpu()
update_curr()
place_entity()
wake_up_new_task()
activate_task()
enqueue_task
fair_sched_class->enqueue_task-->enqueue_task_fair()
2.3.1 sched_fork()
sched_fork()调用__sched_fork()对struct task_struct进行初始化，
int sched_fork(unsigned long clone_flags, struct task_struct *p) { unsigned long flags; int cpu = get_cpu();--------------------------------------------------禁止任务抢占并且获取cpu序号 __sched_fork(clone_flags, p); p->state = TASK_RUNNING;----------------------------------------------此时并没有真正运行，还没有加入到调度器 p->prio = current->normal_prio; /* * Revert to default priority/policy on fork if requested. */ if (unlikely(p->sched_reset_on_fork)) {-------------------------------如果sched_reset_on_fork为true，重置policy、static_prio、prio、weight、inv_weight等。 if (task_has_dl_policy(p) || task_has_rt_policy(p)) { p->policy = SCHED_NORMAL; p->static_prio = NICE_TO_PRIO(0); p->rt_priority = 0; } else if (PRIO_TO_NICE(p->static_prio) < 0) p->static_prio = NICE_TO_PRIO(0); p->prio = p->normal_prio = __normal_prio(p); set_load_weight(p); p->sched_reset_on_fork = 0; } if (dl_prio(p->prio)) { put_cpu(); return -EAGAIN; } else if (rt_prio(p->prio)) { p->sched_class = &rt_sched_class; } else { p->sched_class = &fair_sched_class;-------------------------------根据task_struct->prio选择调度器类， } if (p->sched_class->task_fork) p->sched_class->task_fork(p);-------------------------------------调用调度器类的task_fork方法，cfs对应task_fork_fair()。 raw_spin_lock_irqsave(&p->pi_lock, flags); set_task_cpu(p, cpu);-------------------------------------------------将p指定到cpu上运行，如果task_struct->stack->cpu和当前所在cpu不一致，需要将cpu相关设置到新CPU上。 raw_spin_unlock_irqrestore(&p->pi_lock, flags); #if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) if (likely(sched_info_on())) memset(&p->sched_info, 0, sizeof(p->sched_info)); #endif #if defined(CONFIG_SMP) p->on_cpu = 0; #endif init_task_preempt_count(p);-------------------------------------------初始化preempt_count #ifdef CONFIG_SMP plist_node_init(&p->pushable_tasks, MAX_PRIO); RB_CLEAR_NODE(&p->pushable_dl_tasks); #endif put_cpu();------------------------------------------------------------启用任务抢占 return 0; }
__sched_fork()对task_struct数据结构进行初始值设定。
static void __sched_fork(unsigned long clone_flags, struct task_struct *p) { p->on_rq = 0; p->se.on_rq = 0; p->se.exec_start = 0; p->se.sum_exec_runtime = 0; p->se.prev_sum_exec_runtime = 0; p->se.nr_migrations = 0; p->se.vruntime = 0; #ifdef CONFIG_SMP p->se.avg.decay_count = 0; #endif INIT_LIST_HEAD(&p->se.group_node); #ifdef CONFIG_SCHEDSTATS memset(&p->se.statistics, 0, sizeof(p->se.statistics)); #endif RB_CLEAR_NODE(&p->dl.rb_node); init_dl_task_timer(&p->dl); __dl_clear_params(p); INIT_LIST_HEAD(&p->rt.run_list); #ifdef CONFIG_PREEMPT_NOTIFIERS INIT_HLIST_HEAD(&p->preempt_notifiers); #endif... }
task_fork_fair()参数是新创建的进程，
static void task_fork_fair(struct task_struct *p) { struct cfs_rq *cfs_rq; struct sched_entity *se = &p->se, *curr; int this_cpu = smp_processor_id();--------------------------获取当前cpu id struct rq *rq = this_rq(); unsigned long flags; raw_spin_lock_irqsave(&rq->lock, flags); update_rq_clock(rq); cfs_rq = task_cfs_rq(current);------------------------------获取当前进程所在cpu的cfs_rq curr = cfs_rq->curr; /* * Not only the cpu but also the task_group of the parent might have * been changed after parent->se.parent,cfs_rq were copied to * child->se.parent,cfs_rq. So call __set_task_cpu() to make those * of child point to valid ones. */ rcu_read_lock(); __set_task_cpu(p, this_cpu);--------------------------------将进程p和当前CUP绑定，p->wake_cpu在后续唤醒该进程时会用到这个成员。 rcu_read_unlock(); update_curr(cfs_rq);----------------------------------------更新当前调度实体的cfs_rq->curr信息 if (curr) se->vruntime = curr->vruntime; place_entity(cfs_rq, se, 1);--------------------------------cfs_rq是父进程对应的cfs就绪队列，se对应的是进程p调度实体，initial为1。 if (sysctl_sched_child_runs_first && curr && entity_before(curr, se)) { /* * Upon rescheduling, sched_class::put_prev_task() will place * 'current' within the tree based on its new key value. */ swap(curr->vruntime, se->vruntime); resched_curr(rq); } se->vruntime -= cfs_rq->min_vruntime; raw_spin_unlock_irqrestore(&rq->lock, flags); }
set_task_cpu()将进程和指定的cpu绑定。
update_curr()是cfs调度器核心函数，主要更新cfs_rq->curr，即当前调度实体。
主要更新了调度实体的vruntime、sum_exec_runtime、exec_start等等。
void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
{
if (task_cpu(p) != new_cpu) {
if (p->sched_class->migrate_task_rq)
p->sched_class->migrate_task_rq(p);
p->se.nr_migrations++;
perf_event_task_migrate(p);
}
__set_task_cpu(p, new_cpu);
}
static inline void __set_task_cpu(struct task_struct *p, unsigned int cpu) { set_task_rq(p, cpu); #ifdef CONFIG_SMP /* * After ->cpu is set up to a new value, task_rq_lock(p, ...) can be * successfuly executed on another CPU. We must ensure that updates of * per-task data have been completed by this moment. */ smp_wmb(); task_thread_info(p)->cpu = cpu; p->wake_cpu = cpu; #endif } static void update_curr(struct cfs_rq *cfs_rq) { struct sched_entity *curr = cfs_rq->curr;----------------------------------curr指向父进程调度实体。 u64 now = rq_clock_task(rq_of(cfs_rq));------------------------------------获取当前就绪队列保存的rq->clock_task值，该变量在每次时钟tick到来时更新。 u64 delta_exec; if (unlikely(!curr)) return; delta_exec = now - curr->exec_start;----------------------------------------delta_exec计算该进程从上次调用update_curr()函数到现在的时间差。 if (unlikely((s64)delta_exec <= 0)) return; curr->exec_start = now; schedstat_set(curr->statistics.exec_max, max(delta_exec, curr->statistics.exec_max)); curr->sum_exec_runtime += delta_exec;---------------------------------------sum_exec_runtime直接加上delta_exec。 schedstat_add(cfs_rq, exec_clock, delta_exec); curr->vruntime += calc_delta_fair(delta_exec, curr);------------------------根据delta_exec和进程curr->load计算该进程的虚拟事件curr->vruntime。 update_min_vruntime(cfs_rq);------------------------------------------------更新当前cfs_rq->min_vruntime if (entity_is_task(curr)) {-------------------------------------------------如果curr->my_q为null，那么当前调度实体是进程 struct task_struct *curtask = task_of(curr); trace_sched_stat_runtime(curtask, delta_exec, curr->vruntime); cpuacct_charge(curtask, delta_exec); account_group_exec_runtime(curtask, delta_exec); } account_cfs_rq_runtime(cfs_rq, delta_exec); }
place_entity()参数cfs_rq是se对应进程的父进程对应的cfs就绪队列，se是新进程调度实体，initial为1。
place_entity()考虑当前se所在cfs_rq总体权重，然后更新se->vruntime。
static void place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) { u64 vruntime = cfs_rq->min_vruntime;-----------------------------是单步递增的，用于跟踪整个cfs就绪队列中红黑树里最小的vruntime值。 if (initial && sched_feat(START_DEBIT))--------------------------如果当前进程用于fork新进程，那么这里会对新进程的vruntime做一些惩罚，因为新创建了一个新进程导致cfs运行队列权重发生了变化。 vruntime += sched_vslice(cfs_rq, se);------------------------sched_vslice()计算得到虚拟时间作为惩罚值，累加到vruntime。 /* sleeps up to a single latency don't count. */ if (!initial) { unsigned long thresh = sysctl_sched_latency; if (sched_feat(GENTLE_FAIR_SLEEPERS)) thresh >>= 1; vruntime -= thresh; } se->vruntime = max_vruntime(se->vruntime, vruntime);--------------取se->vruntime和惩罚后的vruntime的最大值，方式vruntime回退。 } static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se) { return calc_delta_fair(sched_slice(cfs_rq, se), se);---------------根据sched_slice()计算得到的执行时间和se中的权重，计算出虚拟时间。 } static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se) { u64 slice = __sched_period(cfs_rq->nr_running + !se->on_rq);-------根据运行中进程数目计算就绪队列调度周期长度。 for_each_sched_entity(se) {----------------------------------------遍历当前se所在就绪队列上所有的调度实体。 struct load_weight *load; struct load_weight lw; cfs_rq = cfs_rq_of(se);----------------------------------------通过sched_entity找到其所在的cfs_rq，进而获得cfs_rq->load。 load = &cfs_rq->load; if (unlikely(!se->on_rq)) { lw = cfs_rq->load; update_load_add(&lw, se->load.weight); load = &lw; } slice = __calc_delta(slice, se->load.weight, load);------------根据当前进程的权重来计算在cfs就绪队列总权重中可以瓜分的调度时间。 } return slice; }
unsigned int sysctl_sched_latency = 6000000ULL;
static unsigned int sched_nr_latency = 8;
unsigned int sysctl_sched_min_granularity = 750000ULL;
static u64 __sched_period(unsigned long nr_running)------------------计算CFS就绪对列中的一个调度周期的长度，可以理解为一个调度周期的时间片，根据当前运行的进程数目来计算。 { u64 period = sysctl_sched_latency;---------------------------------cfs默认调度时间片6ms unsigned long nr_latency = sched_nr_latency;-----------------------运行中的最大进程数目阈值 if (unlikely(nr_running > nr_latency)) {---------------------------如果运行中的进程数目大于8，按照每个进程最小的调度延时0.75ms计时，乘以进程数目来计算调度周期时间片。 period = sysctl_sched_min_granularity; period *= nr_running; } return period; }
2.3.2 wake_up_new_task()
void wake_up_new_task(struct task_struct *p) { unsigned long flags; struct rq *rq; raw_spin_lock_irqsave(&p->pi_lock, flags); #ifdef CONFIG_SMP set_task_cpu(p, select_task_rq(p, task_cpu(p), SD_BALANCE_FORK, 0));------------重新选择CPU，有可能cpus_allowed在fork中被改变，或者之前前选择的CPU被关闭了。 #endif /* Initialize new task's runnable average */ init_task_runnable_average(p); rq = __task_rq_lock(p); activate_task(rq, p, 0);--------------------------------------------------------最终调用到enqueue_task_fair()将进程p添加到cfs就绪队列中。 p->on_rq = TASK_ON_RQ_QUEUED; trace_sched_wakeup_new(p, true); check_preempt_curr(rq, p, WF_FORK);---------------------------------------------检查是否有进程可以抢占当前正在运行的进程。 #ifdef CONFIG_SMP if (p->sched_class->task_woken) p->sched_class->task_woken(rq, p); #endif task_rq_unlock(rq, p, &flags); } void activate_task(struct rq *rq, struct task_struct *p, int flags) { if (task_contributes_to_load(p)) rq->nr_uninterruptible--; enqueue_task(rq, p, flags); } static void enqueue_task(struct rq *rq, struct task_struct *p, int flags) { update_rq_clock(rq); sched_info_queued(rq, p); p->sched_class->enqueue_task(rq, p, flags); }
enqueue_task_fair()把新进程p放入cfs就绪队列rq中。
static void enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) { struct cfs_rq *cfs_rq; struct sched_entity *se = &p->se; for_each_sched_entity(se) {--------------------------对于没有定义CONFIG_FAIR_GROUP_SCHED的情况，只有一次结束for循环，即只有se一个调度实体。 if (se->on_rq) break; cfs_rq = cfs_rq_of(se); enqueue_entity(cfs_rq, se, flags);---------------把调度实体se添加到cfs_rq就绪队列中。 if (cfs_rq_throttled(cfs_rq)) break; cfs_rq->h_nr_running++; flags = ENQUEUE_WAKEUP; } for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se); cfs_rq->h_nr_running++; if (cfs_rq_throttled(cfs_rq)) break; update_cfs_shares(cfs_rq); update_entity_load_avg(se, 1);----------------------------------------更新该调度实体的负载load_avg_contrib和就绪队列负载runnable_load_avg。 } if (!se) { update_rq_runnable_avg(rq, rq->nr_running); add_nr_running(rq, 1); } hrtick_update(rq); } static void enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { /* * Update the normalized vruntime before updating min_vruntime * through calling update_curr(). */ if (!(flags & ENQUEUE_WAKEUP) || (flags & ENQUEUE_WAKING)) se->vruntime += cfs_rq->min_vruntime; /* * Update run-time statistics of the 'current'. */ update_curr(cfs_rq);-------------------------------------------------------更新当前进程的vruntime和该cfs就绪队列的min_vruntime。 enqueue_entity_load_avg(cfs_rq, se, flags & ENQUEUE_WAKEUP);---------------计算调度实体se的load_avg_contrib，然后添加到整个cfs就绪队列总平局负载cfs_rq->runnable_load_avg中。 account_entity_enqueue(cfs_rq, se); update_cfs_shares(cfs_rq); if (flags & ENQUEUE_WAKEUP) {-----------------------------------------------处理刚被唤醒的进程。 place_entity(cfs_rq, se, 0);--------------------------------------------对唤醒进程有一定补偿，最多可以补偿一个调度周期的一般，即vruntime减去半个调度周期时间。 enqueue_sleeper(cfs_rq, se); } update_stats_enqueue(cfs_rq, se); check_spread(cfs_rq, se); if (se != cfs_rq->curr) __enqueue_entity(cfs_rq, se);-------------------------------------------把调度实体se加入到cfs就绪队列的红黑树中。 se->on_rq = 1;--------------------------------------------------------------表示该调度实体已经在cfs就绪队列中。 if (cfs_rq->nr_running == 1) { list_add_leaf_cfs_rq(cfs_rq); check_enqueue_throttle(cfs_rq); } }
check_preempt_curr()用于检查是否有新进程抢占当前进程。
void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags) { const struct sched_class *class; if (p->sched_class == rq->curr->sched_class) { rq->curr->sched_class->check_preempt_curr(rq, p, flags); } else { for_each_class(class) { if (class == rq->curr->sched_class) break; if (class == p->sched_class) { resched_curr(rq); break; } } } /* * A queue event has occurred, and we're going to schedule. In * this case, we can save a useless back to back clock update. */ if (task_on_rq_queued(rq->curr) && test_tsk_need_resched(rq->curr)) rq_clock_skip_update(rq, true); } static void check_preempt_wakeup(struct rq *rq, struct task_struct *p, int wake_flags) { struct task_struct *curr = rq->curr; struct sched_entity *se = &curr->se, *pse = &p->se; struct cfs_rq *cfs_rq = task_cfs_rq(curr); int scale = cfs_rq->nr_running >= sched_nr_latency; int next_buddy_marked = 0; if (unlikely(se == pse)) return; /* * This is possible from callers such as attach_tasks(), in which we * unconditionally check_prempt_curr() after an enqueue (which may have * lead to a throttle). This both saves work and prevents false * next-buddy nomination below. */ if (unlikely(throttled_hierarchy(cfs_rq_of(pse)))) return; if (sched_feat(NEXT_BUDDY) && scale && !(wake_flags & WF_FORK)) { set_next_buddy(pse); next_buddy_marked = 1; } /* * We can come here with TIF_NEED_RESCHED already set from new task * wake up path. * * Note: this also catches the edge-case of curr being in a throttled * group (e.g. via set_curr_task), since update_curr() (in the * enqueue of curr) will have resulted in resched being set. This * prevents us from potentially nominating it as a false LAST_BUDDY * below. */ if (test_tsk_need_resched(curr)) return; /* Idle tasks are by definition preempted by non-idle tasks. */ if (unlikely(curr->policy == SCHED_IDLE) && likely(p->policy != SCHED_IDLE)) goto preempt; /* * Batch and idle tasks do not preempt non-idle tasks (their preemption * is driven by the tick): */ if (unlikely(p->policy != SCHED_NORMAL) || !sched_feat(WAKEUP_PREEMPTION)) return; find_matching_se(&se, &pse); update_curr(cfs_rq_of(se)); BUG_ON(!pse); if (wakeup_preempt_entity(se, pse) == 1) { /* * Bias pick_next to pick the sched entity that is * triggering this preemption. */ if (!next_buddy_marked) set_next_buddy(pse); goto preempt; } return; preempt: resched_curr(rq); /* * Only set the backward buddy when the current task is still * on the rq. This can happen when a wakeup gets interleaved * with schedule on the ->pre_schedule() or idle_balance() * point, either of which can * drop the rq lock. * * Also, during early boot the idle thread is in the fair class, * for obvious reasons its a bad idea to schedule back to it. */ if (unlikely(!se->on_rq || curr == rq->idle)) return; if (sched_feat(LAST_BUDDY) && scale && entity_is_task(se)) set_last_buddy(se); }
3. 进程调度
__schedule()是调度器的核心函数，其作用是让调度器选择和切换到一个合适进程运行。调度轨迹如下：
__schedule()
->pick_next_task()
->pick_next_task_fair()
->context_switch()
->switch_mm()
->cpu_v7_switch_mm()
->switch_to()
->__switch_to
3.1 进程调度时机
调度的时机分为如下3种：
1. 阻塞操作：互斥量(mutex)、信号量(semaphore)、等待队列(waitqueue)等。
2. 在中断返回前和系统调用返回用户空间时，去检查TIF_NEED_RESCHED标志位以判断是否需要调度。
3. 将要被唤醒的进程不会马上调用schedule()要求被调度，而是会被添加到cfs就绪队列中，并且设置TIF_NEED_RESCHED标志位。那么唤醒进程什么时候被调度呢？这要根据内核是否具有可抢占功能(CONFIG_PREEMPT=y)分两种情况。
3.1 如果内核可抢占，则：
- 如果唤醒动作发生在系统调用或者异常处理上下文中，在下一次调用preempt_enable()时会检查是否需要抢占调度。
- 如果唤醒动作发生在硬中断处理上下文中，硬件中断处理返回前夕(不管中断发生点在内核空间还是用户空间)会检查是否要抢占当前进程。
3.2 如果内核不可抢占，则：
- 当前进程调用cond_resched()时会检查是否要调度。
- 主动调度用schedule()。
- 系统调用或者异常处理返回用户空间时。
- 中断处理完成返回用户空间时(只有中断发生点在用户空间才会检查)。
3.2 preempt_schedule()
asmlinkage __visible void __sched notrace preempt_schedule(void)
{
/*
* If there is a non-zero preempt_count or interrupts are disabled,
* we do not want to preempt the current task. Just return..
*/
if (likely(!preemptible()))
return;
preempt_schedule_common();
}
static void __sched notrace preempt_schedule_common(void)
{
do {
__preempt_count_add(PREEMPT_ACTIVE);
__schedule();
__preempt_count_sub(PREEMPT_ACTIVE);
/*
* Check again in case we missed a preemption opportunity
* between schedule and now.
*/
barrier();
} while (need_resched());
}
3.3 __schedule()
__schedule()函数调用pick_next_task()让进程调度器从就绪队列中选择一个最合适的进程next，然后context_switch()切换到next进程运行。
static void __sched __schedule(void) { struct task_struct *prev, *next; unsigned long *switch_count; struct rq *rq; int cpu; preempt_disable(); cpu = smp_processor_id(); rq = cpu_rq(cpu); rcu_note_context_switch(); prev = rq->curr; schedule_debug(prev); if (sched_feat(HRTICK)) hrtick_clear(rq); smp_mb__before_spinlock(); raw_spin_lock_irq(&rq->lock); rq->clock_skip_update <<= 1; /* promote REQ to ACT */ switch_count = &prev->nivcsw; if (prev->state && !(preempt_count() & PREEMPT_ACTIVE)) {--------------当前进程状态不处于TASK_RUNNING状态， if (unlikely(signal_pending_state(prev->state, prev))) { prev->state = TASK_RUNNING; } else { deactivate_task(rq, prev, DEQUEUE_SLEEP); prev->on_rq = 0; if (prev->flags & PF_WQ_WORKER) { struct task_struct *to_wakeup; to_wakeup = wq_worker_sleeping(prev, cpu); if (to_wakeup) try_to_wake_up_local(to_wakeup); } } switch_count = &prev->nvcsw; } if (task_on_rq_queued(prev)) update_rq_clock(rq); next = pick_next_task(rq, prev);---------------------------------------调用pick_next_task_fair()从就绪队列rq上选择合适的进程返回给next。 clear_tsk_need_resched(prev); clear_preempt_need_resched(); rq->clock_skip_update = 0; if (likely(prev != next)) {--------------------------------------------如果待切入的进程next和待切出的进程next不等，那么调用context_switch()进行上下文切换。 rq->nr_switches++; rq->curr = next; ++*switch_count; rq = context_switch(rq, prev, next); /* unlocks the rq */ cpu = cpu_of(rq); } else raw_spin_unlock_irq(&rq->lock); post_schedule(rq); sched_preempt_enable_no_resched(); }
下面重点分析选择待切入函数pick_next_task()和进行切换函数context_switch()两部分。
3.3.1 pick_next_task()
pick_next_task()是对调度类中pick_next_task()方法的包裹，这里主要对应cfs调度策略的pick_next_task_fair()。
/* * Pick up the highest-prio task: */ static inline struct task_struct * pick_next_task(struct rq *rq, struct task_struct *prev) { const struct sched_class *class = &fair_sched_class; struct task_struct *p; /* * Optimization: we know that if all tasks are in * the fair class we can call that function directly: */ if (likely(prev->sched_class == class && rq->nr_running == rq->cfs.h_nr_running)) {------------------------如果当前进程prev的调度类是cfs，并且该CPU就绪队列中进程数量等于cfs就绪队列中进程数量。说明该CPU就绪队列中只有普通进程没有其它调度类进程。 p = fair_sched_class.pick_next_task(rq, prev); if (unlikely(p == RETRY_TASK)) goto again; /* assumes fair_sched_class->next == idle_sched_class */ if (unlikely(!p)) p = idle_sched_class.pick_next_task(rq, prev); return p; } again: for_each_class(class) {--------------------------------------------------其它情况就需要遍历整个调度类，优先级为stop->deadline->realtime->cfs->idle。从这里也可以看出不同调度策略的优先级。 p = class->pick_next_task(rq, prev); if (p) { if (unlikely(p == RETRY_TASK)) goto again; return p; } } BUG(); /* the idle class will always have a runnable task */ } static struct task_struct * pick_next_task_fair(struct rq *rq, struct task_struct *prev) { struct cfs_rq *cfs_rq = &rq->cfs; struct sched_entity *se; struct task_struct *p; int new_tasks; again: #ifdef CONFIG_FAIR_GROUP_SCHED ... #endif if (!cfs_rq->nr_running)--------------------------------如果cfs就绪队列上没有进程，那么选择idle进程。 goto idle; put_prev_task(rq, prev); do { se = pick_next_entity(cfs_rq, NULL);----------------选择cfs就绪队列中的红黑树最左边进程。 set_next_entity(cfs_rq, se); cfs_rq = group_cfs_rq(se);--------------------------如果定义CONFIG_FAIR_GROUP_SCHED，需要遍历cfs_rq->rq上的就绪队列。如果没定义，则返回NULL。 } while (cfs_rq); p = task_of(se); if (hrtick_enabled(rq)) hrtick_start_fair(rq, p); return p; idle: new_tasks = idle_balance(rq); /* * Because idle_balance() releases (and re-acquires) rq->lock, it is * possible for any higher priority task to appear. In that case we * must re-start the pick_next_entity() loop. */ if (new_tasks < 0) return RETRY_TASK; if (new_tasks > 0) goto again; return NULL; }
在没有定义CONFIG_FAIR_GROUP_SCHED的情况下，pick_next_entity()参数curr为NULL。表示pick_next_entity()优先获取cfs_rq->rb_leftmost结点。
set_next_entity()将cfs_rq->curr指向se，并且更行se的exec_start和prev_sum_exec_runtime。
static struct sched_entity * pick_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *curr) { struct sched_entity *left = __pick_first_entity(cfs_rq); struct sched_entity *se; /* * If curr is set we have to see if its left of the leftmost entity * still in the tree, provided there was anything in the tree at all. */ if (!left || (curr && entity_before(curr, left)))-----------------如果left不存在，left指向curr；或者left存在，curr不为NULL且curr的vruntime小于left的，那么left指向curr。 left = curr; se = left; /* ideally we run the leftmost entity */---------------在curr为NULL情况下，se即cfs_rq的最左侧节点。 ... /* * Prefer last buddy, try to return the CPU to a preempted task. */ if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1)----如果cfs_rq->last存在，且其vruntime小于left的。那么更新se为cfs_rq->last。 se = cfs_rq->last; /* * Someone really wants this to run. If it's not unfair, run it. */ if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1)----类似于cfs_rq->next，如果cfs_rq->next小于left的vruntime，那么更新se为cfs_rq->next。 se = cfs_rq->next; clear_buddies(cfs_rq, se); return se; } static void set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) { /* 'current' is not kept within the tree. */ if (se->on_rq) {-----------------------------------------------------如果当前调度实体在就绪队列，则移除。 /* * Any task has to be enqueued before it get to execute on * a CPU. So account for the time it spent waiting on the * runqueue. */ update_stats_wait_end(cfs_rq, se); __dequeue_entity(cfs_rq, se); } update_stats_curr_start(cfs_rq, se); cfs_rq->curr = se; #ifdef CONFIG_SCHEDSTATS /* * Track our maximum slice length, if the CPU's load is at * least twice that of our own weight (i.e. dont track it * when there are only lesser-weight tasks around): */ if (rq_of(cfs_rq)->load.weight >= 2*se->load.weight) { se->statistics.slice_max = max(se->statistics.slice_max, se->sum_exec_runtime - se->prev_sum_exec_runtime); } #endif se->prev_sum_exec_runtime = se->sum_exec_runtime; }
3.3.2 context_switch()
context_switch()共3个参数，其中rq表示进程切换所在的就绪队列，prev将要被换出的进程，next将要被换入执行的进程。
/* * context_switch - switch to the new MM and the new thread's register state. */ static inline struct rq * context_switch(struct rq *rq, struct task_struct *prev, struct task_struct *next) { struct mm_struct *mm, *oldmm; prepare_task_switch(rq, prev, next);-----------和finish_task_switch()成对操作，其中next->on_cpu置1。 mm = next->mm; oldmm = prev->active_mm; /* * For paravirt, this is coupled with an exit in switch_to to * combine the page table reload and the switch backend into * one hypercall. */ arch_start_context_switch(prev); if (!mm) {-------------------------------------对于内核线程来说是没有进程地址空间的 next->active_mm = oldmm;-------------------因为进程调度的需要，需要借用一个进程的地址空间，因此有了active_mm成员。为什么不用prev->mm呢？因为prev也可能是内核线程。 atomic_inc(&oldmm->mm_count); enter_lazy_tlb(oldmm, next); } else switch_mm(oldmm, mm, next);----------------对普通进程，需要调用switch_mm()函数做一些进程地址空间切换的处理。 if (!prev->mm) {-------------------------------对于prev是内核线程情况，prev->active_mm为NULL，rq->prev_mm记录prev->active_mm。 prev->active_mm = NULL; rq->prev_mm = oldmm; } /* * Since the runqueue lock will be released by the next * task (which is an invalid locking op but in the case * of the scheduler it's an obvious special-case), so we * do an early lockdep release here: */ spin_release(&rq->lock.dep_map, 1, _THIS_IP_); context_tracking_task_switch(prev, next); /* Here we just switch the register state and the stack. */ switch_to(prev, next, prev);-------------------切换进程，从prev进程切换到next进程来运行。该函数完成时，CPU运行next进程，prev进程被调度出去，俗称“睡眠”。 barrier(); return finish_task_switch(prev);---------------进程切换后的清理工作，prev->on_cpu置0，递减old_mm->mm_count，由next处理prev进程残局。 }
switch_mm()和switch_to()都是体系结构密切相关函数。
switch_mm()把新进程页表基地址设置到页目录表基地址寄存器中。
switch_mm()首先把当前CPU设置到下一个进程的cpumask位图中，然后调用check_and_switch_context()来完成ARM体系结构相关的硬件设置，例如flush TLB。
/* * This is the actual mm switch as far as the scheduler * is concerned. No registers are touched. We avoid * calling the CPU specific function when the mm hasn't * actually changed. */ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next, struct task_struct *tsk) { #ifdef CONFIG_MMU unsigned int cpu = smp_processor_id(); /* * __sync_icache_dcache doesn't broadcast the I-cache invalidation, * so check for possible thread migration and invalidate the I-cache * if we're new to this CPU. */ if (cache_ops_need_broadcast() && !cpumask_empty(mm_cpumask(next)) && !cpumask_test_cpu(cpu, mm_cpumask(next))) __flush_icache_all(); if (!cpumask_test_and_set_cpu(cpu, mm_cpumask(next)) || prev != next) { check_and_switch_context(next, tsk); if (cache_is_vivt()) cpumask_clear_cpu(cpu, mm_cpumask(prev)); } #endif }
switch_to()最终调用__switch_to()汇编函数。
__switch_to()包含三个参数，r0是移出进程(prev)的task_struct结构，r1是移出进程(task_thread_info(prev))的thread_info结构，r2是移入进程(task_thread_info(next))的thread_info结构。
这里把prev进程相关寄存器上下文保存到该进程的thread_info->cpu_context结构体中，然后再把next进程thread_info->cpu_context结构体中的值设置到物理CPU寄存器中，从而实现进程堆栈的切换。
#define switch_to(prev,next,last) \ do { \ last = __switch_to(prev,task_thread_info(prev), task_thread_info(next)); \ } while (0) /* * Register switch for ARMv3 and ARMv4 processors * r0 = previous task_struct, r1 = previous thread_info, r2 = next thread_info * previous and next are guaranteed not to be the same. */ ENTRY(__switch_to) UNWIND(.fnstart ) UNWIND(.cantunwind ) add ip, r1, #TI_CPU_SAVE ARM( stmia ip!, {r4 - sl, fp, sp, lr} ) @ Store most regs on stack THUMB( stmia ip!, {r4 - sl, fp} ) @ Store most regs on stack THUMB( str sp, [ip], #4 ) THUMB( str lr, [ip], #4 ) ldr r4, [r2, #TI_TP_VALUE] ldr r5, [r2, #TI_TP_VALUE + 4] #ifdef CONFIG_CPU_USE_DOMAINS ldr r6, [r2, #TI_CPU_DOMAIN] #endif switch_tls r1, r4, r5, r3, r7 #if defined(CONFIG_CC_STACKPROTECTOR) && !defined(CONFIG_SMP) ldr r7, [r2, #TI_TASK] ldr r8, =__stack_chk_guard ldr r7, [r7, #TSK_STACK_CANARY] #endif #ifdef CONFIG_CPU_USE_DOMAINS mcr p15, 0, r6, c3, c0, 0 @ Set domain register #endif mov r5, r0 add r4, r2, #TI_CPU_SAVE ldr r0, =thread_notify_head mov r1, #THREAD_NOTIFY_SWITCH bl atomic_notifier_call_chain #if defined(CONFIG_CC_STACKPROTECTOR) && !defined(CONFIG_SMP) str r7, [r8] #endif THUMB( mov ip, r4 ) mov r0, r5 ARM( ldmia r4, {r4 - sl, fp, sp, pc} ) @ Load all regs saved previously THUMB( ldmia ip!, {r4 - sl, fp} ) @ Load all regs saved previously THUMB( ldr sp, [ip], #4 ) THUMB( ldr pc, [ip] ) UNWIND(.fnend ) ENDPROC(__switch_to)
3.4 调度实体sched_entity红黑树操作
cfs使用红黑树来管理调度实体，红黑树的键值为sched_entity->vruntime。
__enqueue_entity()用于将调度实体se键入到cfs_rq运行队列上，具体是加入到cfs_rq->tasks_timeline的红黑树上。
/* * Enqueue an entity into the rb-tree: */ static void __enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct rb_node **link = &cfs_rq->tasks_timeline.rb_node;----------------------取当前cfs_rq->tasks_timeline树上的第一个节点，注意不一定是最左侧节点。 struct rb_node *parent = NULL; struct sched_entity *entry; int leftmost = 1; /* * Find the right place in the rbtree: */ while (*link) {---------------------------------------------------------------从第一个节点开始遍历当前cfs_rq红黑树，知道找到空的插入节点。 parent = *link; entry = rb_entry(parent, struct sched_entity, run_node);------------------通过parent找到其对应的调度实体 /* * We dont care about collisions. Nodes with * the same key stay together. */ if (entity_before(se, entry)) {-------------------------------------------如果se->vruntime < entry->vruntime则条件成立，插入点指向entry对应的左节点。 link = &parent->rb_left; } else {------------------------------------------------------------------否则插入点指向entry对应的右节点，则leftmost为0。 link = &parent->rb_right; leftmost = 0; } } /* * Maintain a cache of leftmost tree entries (it is frequently * used): */ if (leftmost)----------------------------------------------------------------如果新插入的节点为最左侧节点，那么需要改变cfs_rq->rb_leftmost。 cfs_rq->rb_leftmost = &se->run_node; rb_link_node(&se->run_node, parent, link);-----------------------------------将link指向se->run_node rb_insert_color(&se->run_node, &cfs_rq->tasks_timeline);---------------------在将se->run_node插入后，进行平衡调整。 } static void __dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) { if (cfs_rq->rb_leftmost == &se->run_node) {---------------------------------如果待删除的节点是cfs_rq->rb_leftmose，那么还需要更新cfs_rq->rb_leftmost，然后再删除。 struct rb_node *next_node; next_node = rb_next(&se->run_node); cfs_rq->rb_leftmost = next_node; } rb_erase(&se->run_node, &cfs_rq->tasks_timeline);---------------------------从cfs_rq->tasks_timeline删除节点se->run_node。 } struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq)-----------------获取cfs_rq->rb_leftmost对应的调度实体。 { struct rb_node *left = cfs_rq->rb_leftmost; if (!left) return NULL; return rb_entry(left, struct sched_entity, run_node); } static struct sched_entity *__pick_next_entity(struct sched_entity *se)----------获取当前调度实体右侧的调度实体。 { struct rb_node *next = rb_next(&se->run_node); if (!next) return NULL; return rb_entry(next, struct sched_entity, run_node); } struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq)------------------获取cfs_rq最右侧的调度实体。 { struct rb_node *last = rb_last(&cfs_rq->tasks_timeline);--------------------rb_last在cfs_rq->tasks_timeline不停遍历右节点，直到最后一个。 if (!last) return NULL; return rb_entry(last, struct sched_entity, run_node); }
static inline int entity_before(struct sched_entity *a,
struct sched_entity *b)
{
return (s64)(a->vruntime - b->vruntime) < 0;----------------------------比较调度实体a->vruntime和b->vruntime，如果a before b返回true。
}
4. schedule tick
时钟分为周期性时钟和单次触发时钟，通过clockevents_register_device()进行注册。
广播和非广播时钟的区别在于设备的clock_event_device->cpumask设置。
clockevnets_register_device()
->tick_check_new_device()
->tick_setup_device()
->tick_setup_periodic()-----------------------------如果tick_device->mode定义为TICKDEV_MODE_PERIODIC，则注册为周期性时钟。
->tick_set_periodic_handler()
->tick_handle_periodic()------------------------周期性时钟
->tick_handle_periodic_broadcast()---------周期性广播时钟
->tick_setup_oneshot()-----------------------------如果tick_device->mode定义为TICKDEV_MODE_ONESHOT，则为单次触发时钟。
tick_set_periodic_handler()将struct clock_event_device的event_handler设置为tick_handle_periodic()。
上面是时钟的注册，时钟是由中断驱动的，在中断的处理函数中会调用到clock_event_device->event_handler()。
对于周期性时钟对应函数为tick_handle_periodic()-->tick_periodic()-->update_process_times()-->scheduler_tick()。
/* * This function gets called by the timer code, with HZ frequency. * We call it with interrupts disabled. */ void scheduler_tick(void) { int cpu = smp_processor_id(); struct rq *rq = cpu_rq(cpu); struct task_struct *curr = rq->curr; sched_clock_tick(); raw_spin_lock(&rq->lock); update_rq_clock(rq);--------------------------更新当前CPU就绪队列rq中的时钟计数clock和clock_task。 curr->sched_class->task_tick(rq, curr, 0);----对应调度类方法task_tick，cfs调度类对应task_tick_fair()，用于处理时钟tick到来时与调度器相关的事情。 update_cpu_load_active(rq);-------------------更新运行队列中的cpu_load[] raw_spin_unlock(&rq->lock); perf_event_task_tick(); #ifdef CONFIG_SMP rq->idle_balance = idle_cpu(cpu); trigger_load_balance(rq); #endif rq_last_tick_reset(rq); }
task_tick_fair()是cfs调度类task_tick()对应函数，首先调用entity_tick()检查是否需要调度，然后调用update_rq_runnable_avg更新该就绪队列的统计信息。
/* * scheduler tick hitting a task of our scheduling class: */ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued) { struct cfs_rq *cfs_rq; struct sched_entity *se = &curr->se; for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se);-----------------------由sched_entity找到对应task_struct，进而找到所在的就绪队列，再找到cfs_rq。 entity_tick(cfs_rq, se, queued);--------------除了更新se和cfs_rq的统计信息之外，调用check_preempt_tick()检查是否需要调度。 } if (numabalancing_enabled) task_tick_numa(rq, curr); update_rq_runnable_avg(rq, 1); }
static void entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) { /* * Update run-time statistics of the 'current'. */ update_curr(cfs_rq);------------------------------------更新当前进程的vruntime、exec_start等和就绪队列cfs_rq的min_vruntime等。 /* * Ensure that runnable average is periodically updated. */ update_entity_load_avg(curr, 1);------------------------更新curr调度实体的sched_avg参数load_avg_contrib等。 update_cfs_rq_blocked_load(cfs_rq, 1); update_cfs_shares(cfs_rq); ... if (cfs_rq->nr_running > 1) check_preempt_tick(cfs_rq, curr);------------------如果当前就绪队列运行中进程数nr_running大于1，check_preempt_tick()进行检查当前进程是否需要让出CPU。 }
/* * Preempt the current task with a newly woken task if needed: */ static void check_preempt_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr) { unsigned long ideal_runtime, delta_exec; struct sched_entity *se; s64 delta; ideal_runtime = sched_slice(cfs_rq, curr);----------------------------该进程根据权重在一个调度周期里分到的实际运行时间，和sched_vslice()得到的虚拟运行时间区别。 delta_exec = curr->sum_exec_runtime - curr->prev_sum_exec_runtime;----delta_exec是该进程已经运行的实际时间 if (delta_exec > ideal_runtime) {-------------------------------------如果实际运行时间超过了理论分配运行时间，那么该进程需要被调度出去，设置该进程thread_info中TIF_NEED_RESCHED标志位。 resched_curr(rq_of(cfs_rq)); /* * The current task ran long enough, ensure it doesn't get * re-elected due to buddy favours. */ clear_buddies(cfs_rq, curr); return; } if (delta_exec < sysctl_sched_min_granularity)------------------------如果进程实际运行时间小于sysctl_sched_min_granularity(0.75ms)，那么同样不需要调度。 return; se = __pick_first_entity(cfs_rq);-------------------------------------选择当前cfs_rq就绪队列最左侧调度实体。 delta = curr->vruntime - se->vruntime; if (delta < 0)--------------------------------------------------------如果当前curr->vruntime小于最左侧调度实体vruntime，同样不需要调度。 return; if (delta > ideal_runtime)--------------------------------------------这里为什么要这么比？delta是虚拟事件差值，ideal_runtime是实际时间差值。 resched_curr(rq_of(cfs_rq)); }
5. 组调度
CFS调度器的调度粒度是进程，在某些场景下希望调度粒度是组。
组与组之间的关系是公平的，组内的调度实体又是公平的。组调度就是解决这方面的应用需求。
CFS调度器定义一个数据结构来抽象组调度struct task_group。
/* task group related information */ struct task_group { struct cgroup_subsys_state css; #ifdef CONFIG_FAIR_GROUP_SCHED /* schedulable entities of this group on each cpu */ struct sched_entity **se; /* runqueue "owned" by this group on each cpu */ struct cfs_rq **cfs_rq; unsigned long shares; #ifdef CONFIG_SMP atomic_long_t load_avg; atomic_t runnable_avg; #endif #endif #ifdef CONFIG_RT_GROUP_SCHED struct sched_rt_entity **rt_se; struct rt_rq **rt_rq; struct rt_bandwidth rt_bandwidth; #endif struct rcu_head rcu; struct list_head list; struct task_group *parent; struct list_head siblings; struct list_head children; #ifdef CONFIG_SCHED_AUTOGROUP struct autogroup *autogroup; #endif struct cfs_bandwidth cfs_bandwidth; }
5.1 创建组调度
组调度属于cgroup架构中的cpu子系统，在系统配置时需要打开CONFIG_CGROUP_SCHED和CONFIG_FAIR_GROUP_SCHED。
创建一个组调度的接口是sched_create_group()。
/* allocate runqueue etc for a new task group */ struct task_group *sched_create_group(struct task_group *parent)-----------parent指上一级的组调度节点，系统中有一个组调度的根root_task_group。 { struct task_group *tg; tg = kzalloc(sizeof(*tg), GFP_KERNEL);---------------------------------分配task_group数据结构 if (!tg) return ERR_PTR(-ENOMEM); if (!alloc_fair_sched_group(tg, parent))-------------------------------创建cfs调度器需要的组调度数据结构 goto err; if (!alloc_rt_sched_group(tg, parent))---------------------------------创建rt调度器需要的组调度数据结构 goto err; return tg; err: free_sched_group(tg); return ERR_PTR(-ENOMEM); }
alloc_fair_sched_group()创建cfs调度器需要的组调度数据结构。
int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent) { struct cfs_rq *cfs_rq; struct sched_entity *se; int i; tg->cfs_rq = kzalloc(sizeof(cfs_rq) * nr_cpu_ids, GFP_KERNEL);--分配NR_CPUS个cfs_rq数据结构，存放到指针数组中，这里数据结构不是struct cfs_rq。 if (!tg->cfs_rq) goto err; tg->se = kzalloc(sizeof(se) * nr_cpu_ids, GFP_KERNEL);----------分配NR_CPUS个se数据结构，注意这里不是struct sched_entity。 if (!tg->se) goto err; tg->shares = NICE_0_LOAD;---------------------------------------调度组的权重初始化为NICE值为0的权重。 init_cfs_bandwidth(tg_cfs_bandwidth(tg)); for_each_possible_cpu(i) {--------------------------------------遍历系统中所有possible CPU，为每个CPU分配一个struct cfs_rq调度队列和struct sched_entity调度实体。 cfs_rq = kzalloc_node(sizeof(struct cfs_rq),----------------之前分配的是指针数组，这里为每个CPU分配struct cfs_rq和struct sched_entity数据结构。 GFP_KERNEL, cpu_to_node(i)); if (!cfs_rq) goto err; se = kzalloc_node(sizeof(struct sched_entity), GFP_KERNEL, cpu_to_node(i)); if (!se) goto err_free_rq; init_cfs_rq(cfs_rq);----------------------------------------初始化cfs_rq就绪队列中的tasks_timeline和min_vruntime等信息。 init_tg_cfs_entry(tg, cfs_rq, se, i, parent->se[i]);--------构建组调度结构的关键函数。 } return 1; err_free_rq: kfree(cfs_rq); err: return 0; }
init_cfs_rq()初始化cfs_rq的tasks_timeline红黑树、min_vruntime。
init_tg_cfs_entry()初始化构建组调度结构的关键函数,，将rg和cfs_rq关联，。
void init_cfs_rq(struct cfs_rq *cfs_rq) { cfs_rq->tasks_timeline = RB_ROOT; cfs_rq->min_vruntime = (u64)(-(1LL << 20)); #ifndef CONFIG_64BIT cfs_rq->min_vruntime_copy = cfs_rq->min_vruntime; #endif #ifdef CONFIG_SMP atomic64_set(&cfs_rq->decay_counter, 1); atomic_long_set(&cfs_rq->removed_load, 0); #endif } void init_tg_cfs_entry(struct task_group *tg, struct cfs_rq *cfs_rq, struct sched_entity *se, int cpu, struct sched_entity *parent) { struct rq *rq = cpu_rq(cpu); cfs_rq->tg = tg; cfs_rq->rq = rq; init_cfs_rq_runtime(cfs_rq); tg->cfs_rq[cpu] = cfs_rq;-----------------------------将alloc_fair_sched_group()分配的指针数组和对应的数据结构关联上。 tg->se[cpu] = se; /* se could be NULL for root_task_group */ if (!se) return; if (!parent) { se->cfs_rq = &rq->cfs; se->depth = 0; } else { se->cfs_rq = parent->my_q; se->depth = parent->depth + 1; } se->my_q = cfs_rq;------------------------------------针对组调度中实体才有的my_q。 /* guarantee group entities always have weight */ update_load_set(&se->load, NICE_0_LOAD); se->parent = parent; }
5.1.1 双核task_group、cfs_rq、sched_entity、task_struct关系图
5.2 将进程加入组调度
通过调用cpu_cgrp_subsys的接口函数cpu_cgroup_attach()将今晨加入到组调度中。
struct cgroup_subsys cpu_cgrp_subsys = { ... .attach = cpu_cgroup_attach, .exit = cpu_cgroup_exit, .legacy_cftypes = cpu_files, .early_init = 1, }; static void cpu_cgroup_attach(struct cgroup_subsys_state *css, struct cgroup_taskset *tset) { struct task_struct *task; cgroup_taskset_for_each(task, tset)----------------遍历tset包含的进程链表。 sched_move_task(task);-------------------------将task进程迁移到组调度中。 } void sched_move_task(struct task_struct *tsk) { struct task_group *tg; int queued, running; unsigned long flags; struct rq *rq; rq = task_rq_lock(tsk, &flags); running = task_current(rq, tsk);--------------------------判断进程tsk是否正在运行 queued = task_on_rq_queued(tsk);--------------------------判断进程tsk是否在就绪队列里，tsk->on_rq等于TASK_ON_RQ_QUEUED表示该进程在就绪队列中。 if (queued) dequeue_task(rq, tsk, 0);-----------------------------如果进程在就绪队列中，那么要让该进程暂时先退出就绪队列。 if (unlikely(running))------------------------------------如果该进程在在运行中，刚才已经调用dequeue_task()把进程退出就绪队列，现在只能继续加回到就绪队列中。 put_prev_task(rq, tsk); /* * All callers are synchronized by task_rq_lock(); we do not use RCU * which is pointless here. Thus, we pass "true" to task_css_check() * to prevent lockdep warnings. */ tg = container_of(task_css_check(tsk, cpu_cgrp_id, true), struct task_group, css); tg = autogroup_task_group(tsk, tg); tsk->sched_task_group = tg; #ifdef CONFIG_FAIR_GROUP_SCHED if (tsk->sched_class->task_move_group) tsk->sched_class->task_move_group(tsk, queued); else #endif set_task_rq(tsk, task_cpu(tsk));---------------------将tsk对应的调度实体的cfs_rq、parent和当前CPU对应的cfs_rq、se关联起来。 if (unlikely(running)) tsk->sched_class->set_curr_task(rq); if (queued) enqueue_task(rq, tsk, 0); task_rq_unlock(rq, tsk, &flags); } static void task_move_group_fair(struct task_struct *p, int queued) { struct sched_entity *se = &p->se; struct cfs_rq *cfs_rq; if (!queued && (!se->sum_exec_runtime || p->state == TASK_WAKING)) queued = 1; if (!queued) se->vruntime -= cfs_rq_of(se)->min_vruntime; set_task_rq(p, task_cpu(p)); se->depth = se->parent ? se->parent->depth + 1 : 0; if (!queued) { cfs_rq = cfs_rq_of(se); se->vruntime += cfs_rq->min_vruntime; #ifdef CONFIG_SMP se->avg.decay_count = atomic64_read(&cfs_rq->decay_counter); cfs_rq->blocked_load_avg += se->avg.load_avg_contrib; #endif } } static inline void set_task_rq(struct task_struct *p, unsigned int cpu) { #if defined(CONFIG_FAIR_GROUP_SCHED) || defined(CONFIG_RT_GROUP_SCHED) struct task_group *tg = task_group(p);----------获取当前进程对应的task_group。 #endif #ifdef CONFIG_FAIR_GROUP_SCHED p->se.cfs_rq = tg->cfs_rq[cpu];-----------------设置调度实体的cfs_rq和parent。 p->se.parent = tg->se[cpu]; #endif... } static void enqueue_task(struct rq *rq, struct task_struct *p, int flags) { update_rq_clock(rq); sched_info_queued(rq, p); p->sched_class->enqueue_task(rq, p, flags); } static void enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) { struct cfs_rq *cfs_rq; struct sched_entity *se = &p->se; for_each_sched_entity(se) {------------------------------在打开CONFIG_FAIR_GROUP_SCHED之后，需要遍历进程调度实体和它的上一级调度实体。第一次遍历是p->se，第二滴遍历是对应组调度实体tg->se[]。 if (se->on_rq) break; cfs_rq = cfs_rq_of(se); enqueue_entity(cfs_rq, se, flags); /* * end evaluation on encountering a throttled cfs_rq * * note: in the case of encountering a throttled cfs_rq we will * post the final h_nr_running increment below. */ if (cfs_rq_throttled(cfs_rq)) break; cfs_rq->h_nr_running++; flags = ENQUEUE_WAKEUP; } for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se); cfs_rq->h_nr_running++; if (cfs_rq_throttled(cfs_rq)) break; update_cfs_shares(cfs_rq); update_entity_load_avg(se, 1); } if (!se) { update_rq_runnable_avg(rq, rq->nr_running); add_nr_running(rq, 1); } hrtick_update(rq); } static void set_curr_task_fair(struct rq *rq) { struct sched_entity *se = &rq->curr->se; for_each_sched_entity(se) { struct cfs_rq *cfs_rq = cfs_rq_of(se); set_next_entity(cfs_rq, se); /* ensure bandwidth has been allocated on our new cfs_rq */ account_cfs_rq_runtime(cfs_rq, 0); } }
组调度基本策略如下：
- 在创建组调度tg时，tg为每个CPU同时创建组调度内部使用的cfs_rq就绪队列。
- 组调度作为一个调度实体加入到系统的cfs就绪队列rq->cfs_rq中。
- 进程加入到一个组中后，就脱离了系统的cfs就绪队列，并且加入到组调度的cfs就绪队列tg->cfs_rq[]中。
- 在选择下一个进程时，从系统的cfs就绪队列开始，如果选中的调度实体是组调度tg，那么还需要继续遍历tg中的就绪队里，从中选择一个进程来运行。
5.3 组调度相关实验
6. PELT算法改进
PELT(Per-Entity Load Tracking)算法中有一个重要的变量runnable_load_avg，用于描述就绪队列基于可运行状态的总衰减累加时间(runnable time)和权重计算出来的平均负载。
在Linux 4.0中，一次只更新一个调度实体的负载，而没有更新cfs_rq所有调度实体的负载变化情况。
Linux 4.3做出了优化，在每次更新平均负载时会更新整个cfs_rq的平均负载。
struct cfs_rq中增加了struct sched_avg，并且struct sched_avg也做出了改变。
原来load_avg_contrib变成了load_avg，它是计算调度实体基于可运行时间的平均负载，并且考虑CPU频率因素。
util_avg是计算调度实体基于执行时间内的平均负载。对于就绪队列来说，这两个成员包括运行时间和阻塞时间。
struct sched_avg { /* * These sums represent an infinite geometric series and so are bound * above by 1024/(1-y). Thus we only need a u32 to store them for all * choices of y < 1-2^(-32)*1024. */ u32 runnable_avg_sum, runnable_avg_period; u64 last_runnable_update; s64 decay_count; unsigned long load_avg_contrib; }; /* * The load_avg/util_avg accumulates an infinite geometric series. * 1) load_avg factors the amount of time that a sched_entity is * runnable on a rq into its weight. For cfs_rq, it is the aggregated * such weights of all runnable and blocked sched_entities. * 2) util_avg factors frequency scaling into the amount of time * that a sched_entity is running on a CPU, in the range [0..SCHED_LOAD_SCALE]. * For cfs_rq, it is the aggregated such times of all runnable and * blocked sched_entities. * The 64 bit load_sum can: * 1) for cfs_rq, afford 4353082796 (=2^64/47742/88761) entities with * the highest weight (=88761) always runnable, we should not overflow * 2) for entity, support any load.weight always runnable */ struct sched_avg { u64 last_update_time, load_sum; u32 util_sum, period_contrib; unsigned long load_avg, util_avg; };
7. 小结
联系方式:arnoldlu@qq.com
