Lustre assertion caused a message "(mdc_request.c:1162:mdc_page_locate()) ASSERTION( *start <= *hash ) failed: start = 0x5860000009d,end = 0x5b00000873,hash = 0x5b00000873" to be output, resulting in a kernel panic

Solution Unverified - Updated -

Issue

  • Kernel panic with the following error messages and call trace output:
[2536719.185457] LustreError: Skipped 143 previous similar messages
[2536719.185796] LustreError: 2173:0:(osc_request.c:2703:brw_interpret()) G2DPS-OST0012-osc-ffff8ac1e1994800: too many resent retries for object: 30064772096:1424895, rc = -11.
[2536719.186311] LustreError: 2173:0:(osc_request.c:2703:brw_interpret()) Skipped 13 previous similar messages
[2537182.107760] LustreError: 1781106:0:(mdc_request.c:1162:mdc_page_locate()) ASSERTION( *start <= *hash ) failed: start = 0x5860000009d,end = 0x5b00000873,hash = 0x5b00000873
[2537182.109229] LustreError: 1781106:0:(mdc_request.c:1162:mdc_page_locate()) LBUG
[2537182.109516] Pid: 1781106, comm: java 4.18.0-477.27.1.el8_8.x86_64 #1 SMP Thu Aug 31 10:29:22 EDT 2023
[2537182.109518] Call Trace TBD:
[2537182.109649] [<0>] libcfs_call_trace+0x6f/0xa0 [libcfs]
[2537182.109690] [<0>] lbug_with_loc+0x3f/0x70 [libcfs]
[2537182.109704] [<0>] mdc_read_page+0x987/0x9a0 [mdc]
[2537182.109727] [<0>] lmv_read_page+0x2ab/0x2d0 [lmv]
[2537182.109745] [<0>] ll_get_dir_page+0x102/0x170 [lustre]
[2537182.109821] [<0>] ll_dir_read+0x314/0x4b0 [lustre]
[2537182.109856] [<0>] ll_iterate+0x15f/0x680 [lustre]
[2537182.109890] [<0>] iterate_dir+0x144/0x1a0
[2537182.109960] [<0>] ksys_getdents64+0x9c/0x130
[2537182.109963] [<0>] __x64_sys_getdents64+0x16/0x20
[2537182.109965] [<0>] do_syscall_64+0x5b/0x1b0
[2537182.109977] [<0>] entry_SYSCALL_64_after_hwframe+0x61/0xc6
[2537182.110000] Kernel panic - not syncing: LBUG
[2537182.110263] CPU: 11 PID: 1781106 Comm: java Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-477.27.1.el8_8.x86_64 #1
[2537182.110796] Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.15.0-2.module+el8.6.0+14757+c25ee005 04/01/2014
[2537182.111076] Call Trace:
[2537182.111339]  dump_stack+0x41/0x60
[2537182.111657]  panic+0xe7/0x2ac
[2537182.111926]  ? entry_SYSCALL_64_after_hwframe+0x61/0xc6
[2537182.112183]  lbug_with_loc.cold.8+0x18/0x18 [libcfs]
[2537182.112455]  mdc_read_page+0x987/0x9a0 [mdc]
[2537182.112723]  ? ldlm_pool_recalc+0x59/0x160 [ptlrpc]
[2537182.113097]  lmv_read_page+0x2ab/0x2d0 [lmv]
[2537182.113349]  ll_get_dir_page+0x102/0x170 [lustre]
[2537182.113696]  ? ll_md_need_convert+0x160/0x160 [lustre]
[2537182.113972]  ll_dir_read+0x314/0x4b0 [lustre]
[2537182.114240]  ll_iterate+0x15f/0x680 [lustre]
[2537182.114513]  iterate_dir+0x144/0x1a0
[2537182.114740]  ksys_getdents64+0x9c/0x130
[2537182.114965]  ? iterate_dir+0x1a0/0x1a0
[2537182.115186]  __x64_sys_getdents64+0x16/0x20
[2537182.115407]  do_syscall_64+0x5b/0x1b0
[2537182.115660]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[2537182.115881] RIP: 0033:0x7fdf6318141b
[2537182.116102] Code: 00 00 48 83 c4 08 5b 5d c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 8b 47 20 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa b8 d9 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 39 9a 2c 00 f7 d8
[2537182.116571] RSP: 002b:00007fdf63eb8388 EFLAGS: 00000246 ORIG_RAX: 00000000000000d9
[2537182.116801] RAX: ffffffffffffffda RBX: 0000000000000030 RCX: 00007fdf6318141b
[2537182.117028] RDX: 0000000000008000 RSI: 00007fde9ebadd20 RDI: 0000000000007903
[2537182.117283] RBP: 00007fde9ebadd20 R08: 0000000000000007 R09: 000000035bc2a848
[2537182.117529] R10: 00007fdf4d353801 R11: 0000000000000246 R12: 00007fdf63eb83f0
[2537182.117764] R13: 00007fde9ebadcf0 R14: 00007fdf63eb83e8 R15: 0000000000000000

Environment

  • Red Hat Enterprise Linux 8
  • The 3rd party packages:
    • lustre-client-2.14.0_ddn134-1.el8.x86_64
    • kmod-lustre-client-2.14.0_ddn134-1.el8.x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content