linux many open file问题排查

发表于 | 更新于 | 分类于 | 阅读次数：
| 字数： 4.1k | 时长 ≈ 4 分钟

lsof查询many open file等问题

1、netstat显示的tcp连接数正常

1
2
3
4
5
[imhtp@im2-wx-kf2 ~]$ netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
TIME_WAIT    8481
CLOSE_WAIT   322
FIN_WAIT2    43
ESTABLISHED      144

2、ss -s显示大量的closed连接

1
2
3
4
5
6
7
8
9
10
11
[imhtp@im2-wx-kf2 ~]$ ss -s
Total: 707 (kernel 766)
TCP:   8483 (estab 130, closed 7910, orphaned 0, synrecv 0, timewait 7896/0), ports 685

Transport Total     IP        IPv6
*     766       -         -
RAW   0         0         0
UDP   10        6         4
TCP   573       13        560
INET      583       19        564
FRAG      0         0         0

上面信息说明存在socket fd泄漏，那么用lsof命令检查系统sock的文件句柄。closed 7910，很多socket是处于closed状态。

3、lsof | grep sock

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[imhtp@im2-wx-kf2 ~]$ lsof | grep sock
java       1029   imhtp   36u     sock                0,6        0t0   7121949 can't identify protocol
java       1773   imhtp   36u     sock                0,6        0t0     13282 can't identify protocol
java       2747   imhtp   36u     sock                0,6        0t0 129765023 can't identify protocol
java       7070   imhtp   36u     sock                0,6        0t0   9465828 can't identify protocol
java       7681   imhtp   36u     sock                0,6        0t0 268478565 can't identify protocol
java       7681   imhtp  192u     unix 0xffff88004ac89080        0t0 268479492 socket
java      12545   imhtp   38u     sock                0,6        0t0 187396984 can't identify protocol
java      12545   imhtp  190u     unix 0xffff880233f7b3c0        0t0 187400250 socket
java      14216   imhtp   36u     sock                0,6        0t0 106323858 can't identify protocol
java      14216   imhtp  130u     unix 0xffff880235e539c0        0t0 106324660 socket
java      16615   imhtp   36u     sock                0,6        0t0  34219267 can't identify protocol
java      16615   imhtp  145u     unix 0xffff88023924e080        0t0  34219975 socket
java      16870   imhtp   38u     sock                0,6        0t0 198357475 can't identify protocol
java      16870   imhtp   56u     unix 0xffff8802392c7c80        0t0 198358088 socket
java      16870   imhtp   65u     unix 0xffff8802392c7c80        0t0 198358088 socket
java      16870   imhtp   66u     unix 0xffff8802392c7c80        0t0 198358088 socket
java      17021   imhtp   36u     sock                0,6        0t0 213679956 can't identify protocol
java      19405   imhtp   36u     sock                0,6        0t0 177291674 can't identify protocol
java      22165   imhtp   36u     sock                0,6        0t0 271476844 can't identify protocol
java      22165   imhtp   40u     unix 0xffff88004aea26c0        0t0 271477354 socket
java      30738   imhtp   36u     sock                0,6        0t0 269955359 can't identify protocol
java      30738   imhtp   41u     sock                0,6        0t0 271839241 can't identify protocol
java      30738   imhtp   42u     sock                0,6        0t0 271839255 can't identify protocol
java      30738   imhtp   43u     sock                0,6        0t0 271839242 can't identify protocol
java      30738   imhtp   44u     sock                0,6        0t0 271839253 can't identify protocol
java      30738   imhtp   46u     sock                0,6        0t0 271839257 can't identify protocol
java      30738   imhtp   47u     sock                0,6        0t0 271839256 can't identify protocol
java      30738   imhtp   51u     sock                0,6        0t0 271839258 can't identify protocol
java      30738   imhtp   53u     sock                0,6        0t0 271839259 can't identify protocol
java      30738   imhtp   60u     sock                0,6        0t0 271839248 can't identify protocol
java      30738   imhtp   61u     sock                0,6        0t0 271839260 can't identify protocol
java      30738   imhtp  124u     unix 0xffff88004aea2cc0        0t0 270139880 socket
java      31781   imhtp   36u     sock                0,6        0t0 207958180 can't identify protocol

可以发现，Name列的值为“an’t identify protocol”，socket找不到打开的文件。

  • 本文作者： forever杨
  • 本文链接： https://blog.yl-online.top/posts/5618a78c.html
  • 版权声明： 本博客所有文章除特别声明外，均采用 BY-NC-SA 许可协议。如果文章内容对你有用，请记录到你的笔记中。本博客站点随时会停止服务，请不要收藏、转载！