Kernel panic occurred in third party [fp_gpl] module's code path due to soft lockup
Issue
Kernel panic occurred in third party [fp_gpl] module's code path due to soft lockup.
CPUS: 8
DATE: Tue Jun 25 21:55:27 2019
UPTIME: 20 days, 15:13:36
LOAD AVERAGE: 4.15, 2.62, 2.36
TASKS: 1094
NODENAME: host-name
RELEASE: 3.10.0-693.11.1.el7.x86_64
VERSION: #1 SMP Fri Oct 27 05:39:05 EDT 2017
MACHINE: x86_64 (2094 Mhz)
MEMORY: 64 GB
PANIC: "Kernel panic - not syncing: softlockup: hung tasks"
PID: 2092
COMMAND: "in:imjournal"
TASK: ffff880ff26faf70 [THREAD_INFO: ffff880ff1b98000]
CPU: 3
STATE: TASK_RUNNING (PANIC)
crash> log
[..]
[1782915.503215] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [in:imjournal:2092]
[1782915.504215] Modules linked in: iscsi_target_mod target_core_mod ip6_tables iptable_filter drbg ansi_cprng dm_crypt ext4 mbcache jbd2 fp_rcfs(POE) gfs2 dlm fp_fe(POE) fp_audit(POE) fp_delegation(POE) fp_ac(POE) fp_proc(POE) fp_core(POE) fp_syscall(POE) fp_gpl(OE) hipsec(OE) schook(OE) vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr sg shpchp vmw_vmci nfit i2c_piix4 libnvdimm parport_pc parport nfsd nfs_acl lockd grace binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix ahci libahci drm crct10dif_pclmul crct10dif_common
[1782915.504376] libata crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
[1782915.504402] CPU: 3 PID: 2092 Comm: in:imjournal Tainted: P W OE ------------ 3.10.0-693.11.1.el7.x86_64 #1
[1782915.504404] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
[1782915.504405] task: ffff880ff26faf70 ti: ffff880ff1b98000 task.ti: ffff880ff1b98000
[1782915.504407] RIP: 0010:[<ffffffffc0508464>] [<ffffffffc0508464>] __PK_rawVnodeGetVfsmount+0x18/0x104 [fp_gpl]
[1782915.504420] RSP: 0018:ffff880ff1b9b9d0 EFLAGS: 00000287
[1782915.504421] RAX: ffff880ff89af800 RBX: ffff88065df7a038 RCX: ffff880765d10d80
[1782915.504422] RDX: ffff880ff1b9ba48 RSI: ffff880ffcae6540 RDI: ffff880765d10dd0
[1782915.504423] RBP: ffff880ff1b9b9e0 R08: ffff880ff9344300 R09: ffff880ff9344320
[1782915.504424] R10: ffff880ffa6b0b40 R11: 0001f972cd5f2d11 R12: ffff880ff1b9b960
[1782915.504425] R13: 00000000b158c6b7 R14: 0000000000000000 R15: ffffffff81328cf2
[1782915.504426] FS: 00007f5491991700(0000) GS:ffff88103fcc0000(0000) knlGS:0000000000000000
[1782915.504428] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1782915.504428] CR2: 00007f54960fd000 CR3: 0000000ff2f04000 CR4: 00000000000407e0
[1782915.504469] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1782915.504501] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1782915.504503] Stack:
[1782915.504503] ffffffff8120b1e7 ffff880ffb568944 ffff880ff1b9ba28 ffffffffc0508618
[1782915.504505] ffff880ff9344320 ffff880ffa6b0b40 00000000b158c6b7 ffff880ff1b9bab0
[1782915.504507] ffff880fe2eaece8 ffff88003b7789c0 ffff880ffcae6540 ffff880ff1b9ba80
[1782915.504509] Call Trace:
[1782915.504520] [<ffffffff8120b1e7>] ? path_get+0x27/0x30
[1782915.504526] [<ffffffffc0508618>] PK_rawVnodeGetVfsmount+0xc8/0xd2 [fp_gpl]
[1782915.504545] [<ffffffffc06f794d>] PK_vfscallHookGetParams+0x4e5/0x1590 [fp_core]
[1782915.504551] [<ffffffffc06a4145>] PK_feVfscallIoPrevRename+0x287/0x510 [fp_fe]
[1782915.504554] [<ffffffffc06933bf>] PK_feVfscallPrevOperations+0x2f/0x46 [fp_fe]
[1782915.504563] [<ffffffffc06fb698>] PK_vfscallHookStackPrevOperation+0x9c/0xfc [fp_core]
[1782915.504572] [<ffffffffc06fd31d>] PK_vfscalliRename2+0x71/0x10c [fp_core]
[1782915.504686] [<ffffffffc02a6d60>] ? xfs_vn_follow_link+0xa0/0xa0 [xfs]
[1782915.504701] [<ffffffffc02a6d60>] ? xfs_vn_follow_link+0xa0/0xa0 [xfs]
[1782915.504787] [<ffffffffc05513f7>] PK_vfscallStub_iRename2+0x69/0xc0 [fp_syscall]
[1782915.504829] [<ffffffffc0551472>] PK_wrapper0x00PK_vfscallStub_iRename2+0x24/0x26 [fp_syscall]
[1782915.504831] [<ffffffff812115c0>] vfs_rename+0x5a0/0x930
[1782915.504872] [<ffffffffc055144e>] ? PK_vfscallStub_iRename2+0xc0/0xc0 [fp_syscall]
[1782915.504874] [<ffffffff812128c3>] SYSC_renameat2+0x503/0x5a0
[1782915.504882] [<ffffffffc06f021a>] ? PK_readUnlock+0xe/0x10 [fp_core]
[1782915.504891] [<ffffffffc04f0937>] ? hip_sys_rename+0x37/0xe0 [hipsec]
[1782915.504898] [<ffffffffc06f01fa>] ? PK_writeUnlock+0xe/0x10 [fp_core]
[1782915.504904] [<ffffffffc0665bbe>] ? PK_pextRele+0xa2/0xa8 [fp_proc]
[1782915.504911] [<ffffffffc06f0f2e>] ? PK_moduleMsgSend.part.13+0x20/0x24 [fp_core]
[1782915.504918] [<ffffffffc06f0f6d>] ? PK_moduleMsgSend+0x3b/0x40 [fp_core]
[1782915.504921] [<ffffffffc06a38cd>] ? PK_feSyscallIoPrevRename+0x113/0x54c [fp_fe]
[1782915.504924] [<ffffffff812137d0>] ? SyS_renameat+0x20/0x20
[1782915.504926] [<ffffffff812137ae>] SyS_renameat2+0xe/0x10
[1782915.504927] [<ffffffff812137ee>] SyS_rename+0x1e/0x20
[1782915.504936] [<ffffffffc0496a31>] schook_stub_rename+0x91/0xa0 [schook]
[1782915.504938] [<ffffffffc04969a0>] ? schook_stub_chmod+0xa0/0xa0 [schook]
[1782915.504946] [<ffffffffc0702e2b>] PK_sysRename+0x177/0x1ae [fp_core]
[1782915.504968] [<ffffffffc0540062>] PK_stubRename+0x40/0x6e [fp_syscall]
[1782915.504973] [<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
[1782915.504974] Code: fe 48 8b 3d 1f 6f 00 00 48 89 e5 e8 eb fe ff ff 5d c3 90 66 66 66 66 90 55 48 89 e5 53 48 83 ec 08 48 89 fb 4c 8d 47 e0 eb 20 90 <48> 8b 4b 30 48 8d 43 30 48 8d 7b e0 48 39 c1 0f 84 89 00 00 00
Environment
- Red Hat Enterprise Linux 7
- softlockup_panic is enabled
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.