Kernel hard LOCKUP caused by timer 'tvec_base->lock' because 'timer_jiffies' is behind.

Solution In Progress - Updated -

Issue

  • Kernel detected hard LOCKUP caused by tvec_base->lock due to timer_jiffies is behind.
crash>  log
[4017171.269765] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 35
[4017171.269801] CPU: 35 PID: 0 Comm: swapper/35 Not tainted 3.10.0-327.36.1.el7.x86_64 #1
[4017171.269830] Hardware name: NEC Express5800/R120g-1M [N8100-2393Y]/MS-S0901-2, BIOS 5.0.4013 06/06/2016
[4017171.269864]  ffffffff81868b88 194570e2a87dc518 ffff881fffdc5af0 ffffffff81636301
[4017171.269900]  ffff881fffdc5b70 ffffffff8162fb90 0000000000000010 ffff881fffdc5b80
[4017171.269936]  ffff881fffdc5b20 194570e2a87dc518 0000000000000000 0000000000000023
[4017171.269980] Call Trace:
[4017171.269992]  <NMI>  [<ffffffff81636301>] dump_stack+0x19/0x1b
[4017171.270026]  [<ffffffff8162fb90>] panic+0xd8/0x1e7
[4017171.270050]  [<ffffffff8111b920>] ? restart_watchdog_hrtimer+0x50/0x50
[4017171.270076]  [<ffffffff8111b9e2>] watchdog_overflow_callback+0xc2/0xd0
[4017171.270103]  [<ffffffff8115f1d1>] __perf_event_overflow+0xa1/0x250
[4017171.270128]  [<ffffffff8115fca4>] perf_event_overflow+0x14/0x20
[4017171.270153]  [<ffffffff810325e8>] intel_pmu_handle_irq+0x1e8/0x470
[4017171.270179]  [<ffffffff8163fd4b>] perf_event_nmi_handler+0x2b/0x50
[4017171.270204]  [<ffffffff8163f499>] nmi_handle.isra.0+0x69/0xb0
[4017171.270227]  [<ffffffff8163f5b0>] do_nmi+0xd0/0x340
[4017171.270255]  [<ffffffff8163e8d3>] end_repeat_nmi+0x1e/0x2e
[4017171.270279]  [<ffffffff8163dd67>] ? _raw_spin_lock_irqsave+0x47/0x60
[4017171.270305]  [<ffffffff8163dd67>] ? _raw_spin_lock_irqsave+0x47/0x60
[4017171.270331]  [<ffffffff8163dd67>] ? _raw_spin_lock_irqsave+0x47/0x60
[4017171.270355]  <<EOE>>  <IRQ>  [<ffffffff8108c29b>] lock_timer_base.isra.33+0x2b/0x50
[4017171.270392]  [<ffffffff8108e4d4>] mod_timer+0x84/0x240
[4017171.270416]  [<ffffffff81175eee>] test_clear_page_writeback+0x15e/0x190
[4017171.270442]  [<ffffffff81169190>] end_page_writeback+0x20/0x60
[4017171.270490]  [<ffffffffa02ac5a4>] ext4_finish_bio+0x164/0x200 [ext4]
[4017171.270521]  [<ffffffffa02acac3>] ext4_end_bio+0x53/0x110 [ext4]
[4017171.270548]  [<ffffffff812c7635>] ? blk_run_queue+0x35/0x40
[4017171.270574]  [<ffffffff812191e4>] bio_endio+0x64/0xa0
[4017171.270595]  [<ffffffff812caa80>] blk_update_request+0x90/0x370
[4017171.270623]  [<ffffffff814211d3>] scsi_end_request+0x33/0x1d0
[4017171.270647]  [<ffffffff810c3627>] ? enqueue_entity+0x237/0x8b0
[4017171.270672]  [<ffffffff814214d3>] scsi_io_completion+0x103/0x600
[4017171.270696]  [<ffffffff81416815>] scsi_finish_command+0xd5/0x130
[4017171.270721]  [<ffffffff814209aa>] scsi_softirq_done+0x12a/0x150
[4017171.270745]  [<ffffffff812d18e0>] blk_done_softirq+0x90/0xc0
[4017171.270769]  [<ffffffff81084baf>] __do_softirq+0xef/0x280
[4017171.270792]  [<ffffffff8164831c>] call_softirq+0x1c/0x30
[4017171.270816]  [<ffffffff81016fc5>] do_softirq+0x65/0xa0
[4017171.270837]  [<ffffffff81084f45>] irq_exit+0x115/0x120
[4017171.270859]  [<ffffffff81648eb8>] do_IRQ+0x58/0xf0
[4017171.270879]  [<ffffffff8163e1ed>] common_interrupt+0x6d/0x6d
[4017171.270901]  <EOI>  [<ffffffff814d4ac2>] ? cpuidle_enter_state+0x52/0xc0
[4017171.270931]  [<ffffffff814d4c09>] cpuidle_idle_call+0xd9/0x210
[4017171.270957]  [<ffffffff8101e4ee>] arch_cpu_idle+0xe/0x30
[4017171.270980]  [<ffffffff810d64a5>] cpu_startup_entry+0x245/0x290
[4017171.271005]  [<ffffffff8104768a>] start_secondary+0x1ba/0x230

Environment

  • Red Hat Enterprise Linux 7
  • kernel-3.10.0-327.36.1.el7.x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content