Linux

“ps”佔用了 99% 的 CPU

  • February 23, 2021

我有一個 EC2 t2.micro (Linux AMI) 實例。幾天后,我注意到 RAM 被 “ps” 使用了 99%。

發生這種情況時:

$ free -m
             total        used        free      shared  buff/cache   available
Mem:            983         561          79           0         342         257
Swap:             0           0           0

熱門節目

$ top
top - 05:58:52 up 17:28,  2 users,  load average: 1.00, 0.78, 0.39
Tasks:  95 total,   1 running,  58 sleeping,   0 stopped,   0 zombie
%Cpu(s):100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1006940 total,    80816 free,   575132 used,   350992 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   263376 avail Mem

 PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
14386 ec2-user  20   0  307096 269360   4192 S 99.6 26.8   7:15.88 ps
9019 ec2-user  20   0  771132 128884  47792 S  0.4 12.8   1:18.75 python
   1 root      20   0   45624   5372   3936 S  0.0  0.5   0:07.01 systemd
   2 root      20   0       0      0      0 S  0.0  0.0   0:00.05 kthreadd
   4 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 kworker/0:0H
   6 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 mm_percpu_wq
   7 root      20   0       0      0      0 S  0.0  0.0   0:00.52 ksoftirqd/0
   8 root      20   0       0      0      0 I  0.0  0.0   0:03.64 rcu_sched
   9 root      20   0       0      0      0 I  0.0  0.0   0:00.00 rcu_bh
  10 root      rt   0       0      0      0 S  0.0  0.0   0:00.00 migration/0
  11 root      rt   0       0      0      0 S  0.0  0.0   0:00.15 watchdog/0
  12 root      20   0       0      0      0 S  0.0  0.0   0:00.00 cpuhp/0
  14 root      20   0       0      0      0 S  0.0  0.0   0:00.00 kdevtmpfs
  15 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 netns
  16 root      20   0       0      0      0 I  0.0  0.0   0:00.33 kworker/u30:1
  21 root      20   0       0      0      0 S  0.0  0.0   0:00.00 xenbus
  22 root      20   0       0      0      0 S  0.0  0.0   0:00.06 xenwatch
 172 root      20   0       0      0      0 S  0.0  0.0   0:00.01 khungtaskd
 173 root      20   0       0      0      0 S  0.0  0.0   0:00.00 oom_reaper
 174 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 writeback
 176 root      20   0       0      0      0 S  0.0  0.0   0:00.00 kcompactd0
 177 root      25   5       0      0      0 S  0.0  0.0   0:00.00 ksmd
 178 root      39  19       0      0      0 S  0.0  0.0   0:00.00 khugepaged

一旦我殺死 ps 程序,系統就會進入正常狀態。

$ free -m
             total        used        free      shared  buff/cache   available
Mem:            983         294         343           0         345         524
Swap:             0           0           0
[ec2-user@ip-10-0-0-86 scenAI]$ top
top - 06:05:36 up 17:35,  1 user,  load average: 0.00, 0.20, 0.25
Tasks:  88 total,   1 running,  51 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1006940 total,   351520 free,   301488 used,   353932 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   537044 avail Mem

 PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
9017 ec2-user  20   0  231592  21596   9392 S  0.3  2.1   1:32.00 uvicorn
9019 ec2-user  20   0  771132 128884  47792 S  0.3 12.8   1:19.29 python
   1 root      20   0   45624   5372   3936 S  0.0  0.5   0:07.03 systemd
   2 root      20   0       0      0      0 S  0.0  0.0   0:00.05 kthreadd

雖然我沒有直接執行“ps”命令,但隨機呼叫似乎正在呼叫它,一旦腳本失敗,檢查“top”會顯示“ps”以 99% CPU 執行。

以前也ps -ef|grep曾經工作,但現在給出ps: unrecognized option: e ps: unrecognized option: f

** 編輯 ** 甚至 ps -ef|grep ec2-user 掛起:

[root@xxxxxx ec2-user]# ps -ef|grep ec2-user
ec2-user  9017  0.1  1.8 231592 18276 ?        S    Feb22   1:41 /home/ec2-user/FastAPI/apienv/bin/python /home/ec2-user
/FastAPI/apienv/bin/uvicorn -.......
ec2-user  9018  0.0  0.9 157884  9124 ?        S    Feb22   0:00 /home/ec2-user/FastAPI/apienv/bin/python -c from multiprocessing.semaphore_tracker import main;main(4)
ec2-user  9019  0.1 10.9 771132 110460 ?       S    Feb22   1:27 /home/ec2-user/FastAPI/apienv/bin/python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=5, pipe_handle=7) --multiprocessing-fork
root     13290  0.0  0.6 154688  6860 ?        Ss   05:26   0:00 sshd: ec2-user [priv]
ec2-user 13308  0.0  0.3 154824  3460 ?        S    05:26   0:00 sshd: ec2-user@pts/1
ec2-user 13309  0.0  0.6 129176  6320 pts/1    Ss   05:26   0:00 -bash
ec2-user 15102  0.0  7.4 444216 75372 ?        Sl   05:59   0:00 /usr/lib64/R/bin/exec/R --slave --no-restore --file=xxxxxxx
root     16096  0.0  0.0 123540   952 pts/1    S+   07:44   0:00 grep --color=auto ec2-user
ps: unrecognized option: e
ps: unrecognized option: f
^C

(是否 ctrl+c 退出)當時頂部顯示:

# top
top - 07:50:05 up 19:19,  1 user,  load average: 1.00, 0.69, 0.32
Tasks:  93 total,   1 running,  55 sleeping,   0 stopped,   0 zombie
%Cpu(s): 99.7 us,  0.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1006940 total,   258552 free,   532292 used,   216096 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   327024 avail Mem

 PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
16102 root      20   0  307088   4932   3992 S 99.9  0.5   5:44.08 ps
9019 ec2-user  20   0  771132 110460  29368 S  0.3 11.0   1:27.82 python
   1 root      20   0   45624   4516   3080 S  0.0  0.4   0:07.09 systemd

感謝任何指針。

問候,托馬斯

這可能是一個加密礦工,你被黑了。你沒有恢復,你現在 100% 受到威脅,你不能信任這個系統。保存數據、擦除實例、從頭開始安裝並正確保護您的系統。

欲了解更多資訊,請諮詢:

https://security.stackexchange.com/questions/39231/how-do-i-deal-with-a-compromised-server

引用自:https://unix.stackexchange.com/questions/635940