clocksource 出错cu: rcu_sched kthread timer wakeup didn't happen for 89211 jiffies

 

 

[  467.024565][    C2] rcu: Stack dump where RCU GP kthread last ran:
[  467.031360][    C2] Task dump for CPU 2:
[  467.035629][    C2] task:swapper/2       state:R  running task     stack:    0 pid:    0 ppid:     1 flags:0x00000000
[  467.047295][    C2] Call Trace:
[  467.050733][    C2] [<ffffffff800057fe>] walk_stackframe+0x0/0x8a
[  467.057541][    C2] [<ffffffff80038310>] sched_show_task+0x1a0/0x1d6
[  467.064594][    C2] [<ffffffff806475ba>] rcu_check_gp_kthread_starvation+0x10e/0x136
[  467.073218][    C2] [<ffffffff80078930>] rcu_sched_clock_irq+0x7fa/0x8ba
[  467.080731][    C2] [<ffffffff80092216>] tick_sched_handle.isra.18+0x70/0x74
[  467.088559][    C2] [<ffffffff80081f48>] update_process_times+0xbe/0xf4
[  467.096082][    C2] [<ffffffff80092280>] tick_sched_timer+0x66/0xb2
[  467.103181][    C2] [<ffffffff80082e5c>] __hrtimer_run_queues+0xb8/0x23c
[  467.110732][    C2] [<ffffffff80083680>] hrtimer_interrupt+0xe4/0x230
[  556.246690][    C0] watchdog: BUG: soft lockup - CPU#0 stuck for 84s! [swapper/0:0]
[  556.246870][    C2] [<ffffffff804f5a62>] riscv_timer_interrupt+0x3e/0x4c
[  556.247298][    C2] [<ffffffff8006b18e>] handle_percpu_devid_irq+0x94/0x208
[  556.248496][    C3] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[  556.248793][    C3]     (detected by 3, t=178875 jiffies, g=345, q=6)
[  556.249099][    C3] rcu: All QSes seen, last rcu_sched kthread activity 178611 (4296117347-4295938736), jiffies_till_next_fqs=1, root ->qsmask 0x0
[  556.249689][    C3] rcu: rcu_sched kthread timer wakeup didn't happen for 178610 jiffies! g345 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200
[  556.250241][    C3] rcu:     Possible timer handling issue on cpu=2 timer-softirq=342
[  556.250536][    C3] rcu: rcu_sched kthread starved for 178611 jiffies! g345 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 ->cpu=2
[  556.251063][    C3] rcu:     Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior.
[  556.251416][    C3] rcu: RCU grace-period kthread stack dump:
[  556.251604][    C3] task:rcu_sched       state:R stack:    0 pid:   12 ppid:     2 flags:0x00000000
[  556.252094][    C3] Call Trace:
[  556.252189][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  556.252686][    C3] [<ffffffff80652590>] schedule+0x68/0xe6
[  556.253060][    C3] [<ffffffff80657cf6>] schedule_timeout+0x18e/0x306
[  556.253528][    C3] [<ffffffff8007fefa>] timer_update_keys+0x52/0x56
[  556.254046][    C3] [<ffffffff8007708e>] rcu_gp_kthread+0x6b8/0x7da
[  556.254521][    C3] [<ffffffff8002ff4e>] kthreadd+0xc/0x184
[  556.254897][    C3] [<ffffffff80030004>] kthreadd+0xc2/0x184
[  556.255246][    C0] Modules linked in:
[  556.255304][    C3] [<ffffffff800769d2>] rcu_gp_cleanup+0x406/0x40a
[  556.255398][    C0] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G             L    5.14.12 #4 e15603ebd1614130c59080c5aa6e15fd2a8054f0
[  556.255752][    C3] [<ffffffff8002fd28>] kthread+0x12a/0x148
[  556.256166][    C3] [<ffffffff8002fbfa>] set_kthread_struct+0x54/0x58
[  556.256574][    C3] [<ffffffff8000396e>] ret_from_syscall_rejected+0x8/0xc
[  556.257186][    C3] rcu: Stack dump where RCU GP kthread last ran:
[  556.257392][    C3] Task dump for CPU 2:
[  556.257539][    C3] task:swapper/2       state:R  running task     stack:    0 pid:    0 ppid:     1 flags:0x00000000
[  556.258129][    C3] Call Trace:
[  556.258219][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  556.258657][    C3] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  556.259055][    C3] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  556.259612][    C3] watchdog: BUG: soft lockup - CPU#3 stuck for 84s! [swapper/3:0]
[  556.259964][    C3] Modules linked in:
[  556.260100][    C3] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G             L    5.14.12 #4 e15603ebd1614130c59080c5aa6e15fd2a8054f0
[  556.260358][    C3] Hardware name: sifive,hifive-unleashed-a00 (DT)
[  556.260461][    C3] epc : arch_cpu_idle+0x1e/0x24
[  556.260746][    C3]  ra : default_idle_call.part.19+0x2e/0xfa
[  556.261122][    C3] epc : ffffffff80003b28 ra : ffffffff80658904 sp : ffffffe080053f80
[  556.261475][    C3]  gp : ffffffff81d5ced8 tp : ffffffe080032a00 t0 : 0000000000000019
[  556.261819][    C3]  t1 : ffffffff80651126 t2 : 0000000000000001 s0 : ffffffe080053f90
[  556.262161][    C3]  s1 : ffffffff81d5f208 a0 : 0000000000000000 a1 : ffffffe080053f00
[  556.262276][    C2] [<ffffffff80065752>] handle_domain_irq+0x6c/0xa4
[  556.262509][    C3]  a2 : 0000000000000001 a3 : 000000000000266e a4 : 0000000000000000
[  556.262842][    C3]  a5 : 0000000000000000 a6 : 0000000cff96f17a a7 : 0000000000001000
[  556.263172][    C3]  s2 : 0000000000000003 s3 : 0000000000000008 s4 : 0000000000000007
[  556.263491][    C3]  s5 : ffffffff81d5f3c0 s6 : 0000000000000004 s7 : 0000000000000000
[  556.263821][    C3]  s8 : 0000000000002000 s9 : 0000000080039cb0 s10: 0000000000000000
[  556.264150][    C3]  s11: 0000000000000000 t3 : ffffffe0fc5e1400 t4 : 0000000000000000
[  556.264483][    C3]  t5 : ffffffff81c1a4e0 t6 : 0000000000000040
[  556.264734][    C3] status: 0000000200000120 badaddr: 0000000000000000 cause: 8000000000000005
[  556.265106][    C3] [<ffffffff80003b28>] arch_cpu_idle+0x1e/0x24
[  556.265373][    C3] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  556.265644][    C3] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  556.269632][    C0] Hardware name: sifive,hifive-unleashed-a00 (DT)
[  556.269744][    C0] epc : arch_cpu_idle+0x1e/0x24
[  556.276673][    C2] [<ffffffff80042e18>] can_migrate_task+0xaa/0x1f4
[  466.804763][    C0]  ra : default_idle_call.part.19+0x2e/0xfa
[  466.805162][    C0] epc : ffffffff80003b28 ra : ffffffff80658904 sp : ffffffff81c03f00
[  466.805517][    C0]  gp : ffffffff81d5ced8 tp : ffffffff81c0c140 t0 : ffffffe08000fae8
[  466.818777][    C2] [<ffffffff80003b24>] arch_cpu_idle+0x1a/0x24
[  466.819173][    C2] [<ffffffff8000396e>] ret_from_syscall_rejected+0x8/0xc
[  466.831120][    C0]  t1 : ffffffff80651126 t2 : 0000003fffba74e8 s0 : ffffffff81c03f10
[  466.831479][    C0]  s1 : ffffffff81d5f208 a0 : 0000000000000000 a1 : ffffffff81c03e80
[  466.839184][    C2] [<ffffffff80003b24>] arch_cpu_idle+0x1a/0x24
[  466.839560][    C2] [<ffffffff80658900>] default_idle_call.part.19+0x2a/0xfa
[  466.850562][    C0]  a2 : 0000000000000001 a3 : 00000000000054aa a4 : 0000000000000000
[  466.850896][    C0]  a5 : 0000000000000000 a6 : 0000000cff55c1fa a7 : ffffffe0fc5c5400
[  466.860864][    C2] [<ffffffff80651122>] rcu_eqs_enter.constprop.93+0x52/0x70
[  466.866836][    C0]  s2 : 0000000000000000 s3 : 0000000000000001 s4 : 0000000000000007
[  466.867166][    C0]  s5 : ffffffff81d5f3c0 s6 : ffffffe0fc5ed040 s7 : ffffffff81d5f040
[  466.876651][    C2] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  466.879774][    C0]  s8 : 0000000000000000 s9 : ffffffff80a00108 s10: 0000000000000000
[  466.880116][    C0]  s11: 0000000000000000 t3 : 0000000000000001 t4 : 0000000000000400
[  466.886110][    C2] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  466.891985][    C0]  t5 : 0000000000000000 t6 : 0000000000000000
[  466.892233][    C0] status: 0000000200000120 badaddr: 0000000000000000 cause: 8000000000000005
[  556.249085][    C3] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[  556.254518][    C0] [<ffffffff80003b28>] arch_cpu_idle+0x1e/0x24
[  556.254799][    C0] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  556.255074][    C0] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  556.262995][    C3] rcu:     0-...0: (1 GPs behind) idle=4ae/0/0x1 softirq=401/401 fqs=43 
[  556.271434][    C0] [<ffffffff808013d8>] start_kernel+0x9e2/0xa18
[  556.277702][    C3]     (detected by 3, t=89348 jiffies, g=349, q=5)
[  556.278004][    C3] Task dump for CPU 0:
[  556.278150][    C3] task:swapper/0       state:R  running task     stack:    0 pid:    0 ppid:     0 flags:0x00000008
[  467.060210][    C3] Call Trace:
[  467.063646][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  467.070405][    C3] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  467.076737][    C3] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  467.083781][    C3] [<ffffffff808013d8>] start_kernel+0x9e2/0xa18
[  556.247671][    C3] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[  556.255053][    C3] rcu:     0-...!: (0 ticks this GP) idle=4cc/0/0x0 softirq=404/404 fqs=0  (false positive?)
[  556.265987][    C3] rcu:     1-...!: (0 ticks this GP) idle=5c4/0/0x0 softirq=570/570 fqs=0  (false positive?)
[  556.276920][    C3] rcu:     2-...!: (0 ticks this GP) idle=608/0/0x0 softirq=422/422 fqs=0  (false positive?)
[  466.809361][    C3]     (detected by 3, t=89160 jiffies, g=353, q=4)
[  466.816025][    C3] Task dump for CPU 0:
[  466.820294][    C3] task:swapper/0       state:R  running task     stack:    0 pid:    0 ppid:     0 flags:0x00000000
[  466.831987][    C3] Call Trace:
[  466.835413][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  466.842191][    C3] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  466.848506][    C3] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  466.855553][    C3] [<ffffffff808013d8>] start_kernel+0x9e2/0xa18
[  466.862393][    C3] Task dump for CPU 1:
[  466.866664][    C3] task:swapper/1       state:R  running task     stack:    0 pid:    0 ppid:     1 flags:0x00000000
[  466.878336][    C3] Call Trace:
[  466.881766][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  466.888512][    C3] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  466.894831][    C3] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  466.901877][    C3] Task dump for CPU 2:
[  466.906165][    C3] task:swapper/2       state:R  running task     stack:    0 pid:    0 ppid:     1 flags:0x00000000
[  466.917841][    C3] Call Trace:
[  466.921264][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  466.928013][    C3] [<ffffffff8004156e>] do_idle+0x14c/0x152
[  466.934328][    C3] [<ffffffff80041700>] cpu_startup_entry+0x32/0x38
[  466.941379][    C3] rcu: rcu_sched kthread timer wakeup didn't happen for 89211 jiffies! g353 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
[  466.954526][    C3] rcu:     Possible timer handling issue on cpu=2 timer-softirq=386
[  466.962880][    C3] rcu: rcu_sched kthread starved for 89212 jiffies! g353 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=2
[  466.974704][    C3] rcu:     Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior.
[  466.985338][    C3] rcu: RCU grace-period kthread stack dump:
[  466.991540][    C3] task:rcu_sched       state:I stack:    0 pid:   12 ppid:     2 flags:0x00000000
[  467.001666][    C3] Call Trace:
[  467.005081][    C3] [<ffffffff8065218c>] __schedule+0x21c/0x5b8
[  467.011841][    C3] [<ffffffff80652590>] schedule+0x68/0xe6
[  467.018147][    C3] [<ffffffff80657cf6>] schedule_timeout+0x18e/0x306
[  467.025307][    C3] [<ffffffff80076972>] rcu_gp_cleanup+0x3a6/0x40a
[  467.032441][    C3] [<ffffffff8007fefa>] timer_update_keys+0x52/0x56
[  467.039600][    C3] [<ffffffff8007708e>] rcu_gp_kthread+0x6b8/0x7da
[  467.046734][    C3] [<ffffffff8002ff4e>] kthreadd+0xc/0x184
[  467.053039][    C3] [<ffffffff80030004>] kthreadd+0xc2/0x184
[  467.059366][    C3] [<ffffffff800769d2>] rcu_gp_cleanup+0x406/0x40a
[  467.066486][    C3] [<ffffffff8002fd28>] kthread+0x12a/0x148
[  467.072813][    C3] [<ffffffff8002fbfa>] set_kthread_struct+0x54/0x58
[  467.079952][    C3] [<ffffffff8000396e>] ret_from_syscall_rejected+0x8/0xc

 

 

这是因为clocksource_mmio_init的clocksource的 counter 的最大值有问题

clocksource_mmio_init(clkevt->value, name,
rate, 200, 32, clocksource_mmio_readl_down);