Overcloud compute nodes reject connection and logs show "kernel: nf_conntrack: table full, dropping packet"

Issue

• We detected that a compute node was rejecting connections due to excessive connections that were having 2 instances on the platform which caused the services of other instances to be affected.

• The following errors are seen in /var/log/messages:

Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet
Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet
Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet
Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet
Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet
Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet
Jul 10 17:39:27 overcloud-compute-0 kernel: nf_conntrack: table full, dropping packet

• The following nf_conntrack_max values are set :

[root@overcloud-compute-0 ~]# cat /proc/sys/net/netfilter/nf_conntrack_max
262144
[root@overcloud-compute-0 ~]# cat /proc/sys/net/netfilter/nf_conntrack_count
262144