linux D状态进程长时间无动静,主动中断释放,D原因未知

4b(某服务器) 训 33x34 att-10s 内存22G,显存16G,确认当时我就跑了这个gpu进程

出于安全考虑,人名均为化名

# tmux 切到该进程,无动静有半天了
105/105 - 1s - loss: 2.1323e-04 - val_loss: 1.8251e-04
Create new model time:  0.616448163986206
# ctrl+c 中断后资源释放

诊断

内存占用

500G 占 4.6%,不拉满都认为正常

$ ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%mem | head -n 6                                                     
  PID  PPID %MEM %CPU CMD                  
23420 23236  4.6 28.9 python run_ippo.py    

显存占用

驱动问题,在gpu还是cpu跑这个比较玄学,占用那么大显存,原则是上有点问题的

$ gpustat
node04                      Wed Apr 24 16:37:52 2024  550.54.14
[0] NVIDIA GeForce RTX 3090 | 86°C,  90 % | 18273 / 24576 MB | zhangsan(15684M) iing(258M) lisi(1110M) lisi(1110M) kde(27M) kde(56M)
[1] NVIDIA GeForce RTX 3090 | 62°C,  88 % |  2891 / 24576 MB | zhangsan(398M) iing(258M) lisi(1110M) lisi(1110M)
[2] NVIDIA GeForce RTX 3090 | 64°C,  90 % |  2795 / 24576 MB | zhangsan(398M) iing(258M) lisi(1062M) lisi(1062M)
[3] NVIDIA GeForce RTX 3090 | 60°C,  48 % |  2795 / 24576 MB | zhangsan(398M) iing(258M) lisi(1062M) lisi(1062M)

进程

查找进程号

$ top | grep D                                                                                                   
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
23420 zhangsan  20   0 70.386g 0.023t 628168 D  21.1  4.6   1774:45 python                                                                      
23420 zhangsan  20   0 70.386g 0.023t 628168 D  23.8  4.6   1774:45 python

$ ps aux | grep 23420                                                                                            
zhangsan 23420 28.9  4.6 73769468 24412352 pts/2 Rl+ 4月20 1774:37 python run_ippo.py

网上查,D状态表示进程处于不可中断的睡眠状态,可能是等待磁盘I/O、等待硬件设备等原因。推测是磁盘,但不排除其他原因。比较服务器前面内存、显存都没拉满。确实有内存、显存拉满引发的异常,不过异常会直接中断程序了((ResourceExhaustedError: OOM))

lsof

本想通过 lsof 查看是哪个设备导致的等待,不过发现太多无关文件,不太好定位。最后没有弄明白原因,直接中断任务重新开始了,下面是 lsof 的部分输出。

$ lsof -p 23420
lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing
      Output information may be incomplete.                                
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/224300d3543ff23234fbef0b17256d871e1348d57b9ede8abd3466a2f94b658e/merged
      Output information may be incomplete.                                                                                                     
lsof: WARNING: can't stat() nsfs file system /run/docker/netns/fbea593b3cbe
      Output information may be incomplete.                                                                                                     
lsof: WARNING: can't stat() tmpfs file system /var/lib/kubelet/pods/aa094d10-9ac1-414d-a1bb-5c9dc61329f8/volumes/kubernetes.io~projected/kube-api-access-
qf8k2                                                      
      Output information may be incomplete.
lsof: WARNING: can't stat() tmpfs file system /var/lib/kubelet/pods/230d1279-8261-442b-8e39-88ee70eca235/volumes/kubernetes.io~projected/kube-api-access-
gqj8b                                      
      Output information may be incomplete.
# 省略相似内容
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/5dafb56dd2d21069586611b78b3b30724673ac20a70371663aa85e24736564ea/merged
      Output information may be incomplete.
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/a780ee78727452ab11c8f97aac63d16d64bd8825f9c2683c5f02a1c7c91d95f7/merged
      Output information may be incomplete.
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/39b1b56a3ebc7f12c6b5c9c04668a554696fa07874044f26d18ee952f0ab7887/merged
      Output information may be incomplete.
lsof: WARNING: can't stat() tmpfs file system /var/lib/docker/containers/630fc34395f9e010119f76e5084ecd116931165bd33ae724986a4f65e9ba5321/mounts/shm
      Output information may be incomplete.
lsof: WARNING: can't stat() nsfs file system /run/docker/netns/711c1e5bb2b4
      Output information may be incomplete.
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/90a63e0149c44191dc1e4d6a7c3b4aa5c57e857c6ec890443ec1b214e225cb8d/merged
      Output information may be incomplete.
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/3c33df5756ea8c0d2ff1579ac57e157a034970ce43fef498ed9a0d49ff9acd40/merged
      Output information may be incomplete.
lsof: WARNING: can't stat() tmpfs file system /run/snapd/ns
      Output information may be incomplete.
lsof: WARNING: can't stat() nsfs file system /run/snapd/ns/gnome-characters.mnt
      Output information may be incomplete.
COMMAND   PID     USER   FD      TYPE             DEVICE  SIZE/OFF       NODE NAME
python  23420 zhangsan  cwd       DIR               0,54      4096 1121965694 /home/zhangsan/tsc/ippo (192.0.0.2:/home)
python  23420 zhangsan  rtd       DIR                8,2      4096          2 /
python  23420 zhangsan  txt       REG               0,54  10668400 1126705228 /home/zhangsan/miniconda3/envs/36/bin/python3.6 (192.0.0.2:/home)
python  23420 zhangsan  mem       CHR              195,0                  710 /dev/nvidia0
python  23420 zhangsan  mem       CHR            195,255                  684 /dev/nvidiactl
python  23420 zhangsan  mem       CHR              507,0                  765 /dev/nvidia-uvm
python  23420 zhangsan  mem       CHR              195,1                  724 /dev/nvidia1
python  23420 zhangsan  mem       CHR              195,2                  736 /dev/nvidia2
python  23420 zhangsan  mem       CHR              195,3                  748 /dev/nvidia3
python  23420 zhangsan  DEL       REG                0,1             44746876 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44717481 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44717482 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44717484 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44710541 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44710542 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44710544 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44680112 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44680113 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44680115 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44699287 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44699288 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44699290 /dev/zero
python  23420 zhangsan  mem       REG                8,2 233295448    9973030 /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcusparse.so.11.2.0.275
python  23420 zhangsan  mem       REG                8,2 190512040   11149081 /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcusolver.so.10.3.0.89
python  23420 zhangsan  mem       REG                8,2  75343512    9973023 /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcurand.so.10.2.2.74
python  23420 zhangsan  mem       REG                8,2 235963680    9973019 /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcufft.so.10.3.0.74
python  23420 zhangsan  mem       REG                8,2 222330208    9973014 /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcublasLt.so.11.2.1.74
python  23420 zhangsan  mem       REG                8,2 120427896    9973016 /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcublas.so.11.2.1.74
python  23420 zhangsan  DEL       REG                0,1             44717483 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44710543 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44680114 /dev/zero
python  23420 zhangsan  DEL       REG                0,1             44699289 /dev/zero
python  23420 zhangsan  mem       REG                8,2    162360    9971979 /usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.0
python  23420 zhangsan  mem       REG                8,2  28392536    9961911 /usr/lib/x86_64-linux-gnu/libcuda.so.550.54.14
python  23420 zhangsan  mem       REG               0,54    516208 1126707768 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/cityflow.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   5230584 1131025253 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/core/kernels/libtfkernel_sobol_op.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   3993072 1131025420 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/lite/python/interpreter_wrapper/_pywrap_tensorflow_interpreter_wrapper.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    114576 1131023533 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_toco_api.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    948952 1131025400 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/lite/experimental/microfrontend/python/ops/_audio_microfrontend_op.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    269424 1131023517 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_python_op_gen.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1440544 1131023539 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_mlir.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1299144 1131023549 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_debug_events_writer.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    102224 1131023534 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_util_port.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    101848 1131023518 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_stacktrace_handler.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1268768 1131023522 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_quantize_training.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1732752 1131023514 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tfprof.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1374032 1131023519 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tf_cluster.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1291752 1131023521 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tf_optimizer.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    117504 1130978688 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5l.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    129752 1130978702 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5o.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     47224 1130978686 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5pl.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    212792 1130978689 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     67968 1130978705 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5i.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    187072 1130978680 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5g.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    142280 1130978699 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5f.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     92856 1130978693 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5ds.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    146224 1130978685 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5d.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     55424 1130978701 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/_proxy.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     63816 1130978681 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5ac.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    388584 1130978695 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5p.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    125680 1130978707 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5s.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    158544 1130978691 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5a.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     58200 1130978684 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5z.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     60024 1130978700 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/utils.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    549128 1130978697 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5t.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     76376 1130978696 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5r.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     97096 1130978682 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/_conv.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    150912 1130978703 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/_objects.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    257640 1130978683 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/defs.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    125880 1130978698 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/h5.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     87848 1130978713 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/.libs/libz-a147dcb0.so.1.2.3 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     37712 1130978710 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/.libs/libaec-2147abcd.so.0.0.4 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     16792 1130978711 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/.libs/libsz-1c7dd0cf.so.2.0.1 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    186152 1130978712 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/.libs/libhdf5_hl-db841637.so.100.1.1 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   4187280 1130978709 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/.libs/libhdf5-beac1db3.so.103.0.0 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     64432 1130978694 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/h5py/_errors.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    108000 1126705858 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/testing.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    105528 1126705854 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/json.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    585160 1126705954 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/parsers.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    342312 1126705924 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/reduction.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1235240 1126705979 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/groupby.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    190704 1126705867 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/window/indexers.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    479360 1126705940 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/window/aggregations.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    282776 1126705907 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/internals.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    239328 1126705891 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/writers.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    303816 1126705913 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/reshape.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    940432 1126705971 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/sparse.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   2822696 1126705990 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/join.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    618728 1126705957 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/index.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    245144 1126705895 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/ops.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    200056 1126705874 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/hashing.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    195632 1126705870 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslib.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    629976 1126705962 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/lib.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1662408 1126705987 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/algos.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     71416 1126705842 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/ops_dispatch.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    254128 1126705899 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/vectorized.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    473064 1126705935 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/period.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     82296 1126705846 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/properties.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    446000 1126705927 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/strptime.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    298608 1126705910 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/fields.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    515648 1126705949 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/timestamps.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    499016 1126705945 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/timedeltas.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1121952 1126705976 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/offsets.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    471072 1126705932 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/parsing.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     94368 1126705851 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/ccalendar.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    327512 1126705920 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/tzconversion.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    258960 1126705902 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/timezones.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     64240 1126705839 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/np_datetime.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    233344 1126705881 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/nattype.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    314160 1126705916 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/conversion.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    149544 1126705862 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/dtypes.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    227112 1126705878 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/missing.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    664024 1126705967 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/hashtable.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1387144 1126705984 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/interval.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    953992 1122630212 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/unicodedata.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    678000 1124448346 /home/zhangsan/miniconda3/envs/36/lib/libssl.so.1.1 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1273328 1131023515 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_device_lib.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    247808 1122630200 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1330080 1131024939 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/data/experimental/service/_pywrap_server_lib.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    101648 1131023545 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_py_func.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    102968 1131023542 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensor_float_32_execution.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1324296 1131023526 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_parallel_device.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1863296 1131023537 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_checkpoint_reader.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1370672 1131023547 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_record_io.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   5410256 1131023974 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/profiler/internal/_pywrap_profiler.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1388720 1131023553 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_file_io.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG                8,2     27112   22807808 /lib/x86_64-linux-gnu/libuuid.so.1.3.0
python  23420 zhangsan  mem       REG               0,54     76856 1122630171 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/mmap.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     46408 1122630157 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_multiprocessing.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    150616 1122630189 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/array.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1309088 1131023530 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_events_writer.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     57520 1126705836 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/indexing.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     55448 1126705833 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/pandas/_libs/tslibs/base.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    115960 1122630182 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_multibytecodec.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    293696 1122630203 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1267872 1131023532 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_op_def_registry.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    114944 1131025063 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/framework/fast_tensor_util.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1297720 1131023977 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/profiler/internal/_pywrap_traceme.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    324376 1131023551 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_dtypes.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    101608 1131023516 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_bfloat16.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    442600 1131023538 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_py_exception_registry.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    136912 1131023536 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_utils.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    170912 1130977549 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/wrapt/_wrappers.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    942888 1131023552 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_tf_stack.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   1982240 1131023529 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tf_session.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   2169280 1131023531 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tfe.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   2548904 1130977155 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/google/protobuf/pyext/_message.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     14408 1130977142 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/google/protobuf/internal/_api_implementation.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     49648 1122630162 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/fcntl.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    120848 1122630184 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_csv.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    246568 1122630199 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_socket.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     47816 1122630160 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/termios.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG                8,2    540184    9973007 /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudart.so.11.1.74
python  23420 zhangsan  mem       REG               0,54  17981480 1121835729 /home/zhangsan/miniconda3/envs/36/lib/libstdc++.so.6.0.29 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54  39413472 1130978951 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/libtensorflow_framework.so.2 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54 699303912 1131023525 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     27080 1122630150 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54  14093170 1126707657 /home/zhangsan/miniconda3/envs/36/lib/libmkl_vml_avx512.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54  61346160 1126707807 /home/zhangsan/miniconda3/envs/36/lib/libmkl_avx512.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54  13049502 1126707649 /home/zhangsan/miniconda3/envs/36/lib/libmkl_intel_lp64.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54  59196279 1126707774 /home/zhangsan/miniconda3/envs/36/lib/libmkl_intel_thread.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54  74028346 1126707880 /home/zhangsan/miniconda3/envs/36/lib/libmkl_core.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    852152 1126707122 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_generator.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     71392 1126707085 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_sfc64.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     87704 1126707089 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_pcg64.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    105928 1126707095 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_philox.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    119672 1126707098 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_mt19937.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    386080 1126707115 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_bounded_integers.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     56928 1122630166 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_random.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     33904 1122630152 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_bisect.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    386992 1122630207 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_sha3.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    184568 1122630194 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_blake2.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   3270744 1126703647 /home/zhangsan/miniconda3/envs/36/lib/libcrypto.so.1.1 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     82352 1122630173 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     80224 1122630172 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/binascii.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    268848 1126707112 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/_common.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    219432 1126707109 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/bit_generator.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    732064 1126707119 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/random/mtrand.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     92688 1126707092 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/fft/_pocketfft_internal.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    818728 1122630211 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    213984 1122560279 /home/zhangsan/miniconda3/envs/36/lib/liblzma.so.5.4.5 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    101544 1122630180 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    135152 1122630187 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    124744 1121835323 /home/zhangsan/miniconda3/envs/36/lib/libz.so.1.2.13 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    108232 1122630181 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/zlib.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    195856 1126707106 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/linalg/_umath_linalg.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     26568 1126707076 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/linalg/lapack_lite.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     74256 1122630170 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/select.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     58344 1122630167 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_posixsubprocess.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    164680 1126707102 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/core/_multiarray_tests.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   3723760 1126707129 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    146456 1126705782 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/mkl/_py_mkl_service.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    475272 1121785837 /home/zhangsan/miniconda3/envs/36/lib/libgcc_s.so.1 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG                8,2     31680   22811535 /lib/x86_64-linux-gnu/librt-2.27.so
python  23420 zhangsan  mem       REG               0,54    428160 1122630209 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_pickle.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   2624896 1126705661 /home/zhangsan/miniconda3/envs/36/lib/libiomp5.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54   7100733 1126707601 /home/zhangsan/miniconda3/envs/36/lib/libmkl_rt.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG                8,2   4795856    9970368 /usr/lib/locale/locale-archive
python  23420 zhangsan  mem       REG                8,2   2030928   22806611 /lib/x86_64-linux-gnu/libc-2.27.so
python  23420 zhangsan  mem       REG                8,2    144976   22811475 /lib/x86_64-linux-gnu/libpthread-2.27.so
python  23420 zhangsan  mem       REG                8,2   1700792   22806615 /lib/x86_64-linux-gnu/libm-2.27.so
python  23420 zhangsan  mem       REG                8,2     10592   22811537 /lib/x86_64-linux-gnu/libutil-2.27.so
python  23420 zhangsan  mem       REG                8,2     14560   22806614 /lib/x86_64-linux-gnu/libdl-2.27.so
python  23420 zhangsan  mem       REG                8,2    179152   22806544 /lib/x86_64-linux-gnu/ld-2.27.so
python  23420 zhangsan  mem       REG               0,54     41520 1122630155 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/grp.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     17008 1126705779 /home/zhangsan/miniconda3/envs/36/lib/python3.6/site-packages/mkl/_mklinit.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    151576 1122630190 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_struct.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    414664 1122630208 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    311720 1122630206 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_datetime.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54    122440 1122630185 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/math.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     50664 1124448844 /home/zhangsan/miniconda3/envs/36/lib/libffi.so.7.1.0 (192.0.0.2:/home)
python  23420 zhangsan  mem       REG               0,54     47376 1122630159 /home/zhangsan/miniconda3/envs/36/lib/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so (192.0.0.2:/home)
python  23420 zhangsan  mem       REG                8,2     26376   10229122 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
python  23420 zhangsan    0u      CHR              136,2       0t0          5 /dev/pts/2
python  23420 zhangsan    1u      CHR              136,2       0t0          5 /dev/pts/2
python  23420 zhangsan    2u      CHR              136,2       0t0          5 /dev/pts/2
python  23420 zhangsan    3r      CHR                1,3       0t0          6 /dev/null
python  23420 zhangsan    4w     FIFO               0,13       0t0   44719496 pipe
python  23420 zhangsan    5u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan    6r     FIFO               0,13       0t0   44675950 pipe
python  23420 zhangsan    7w     FIFO               0,13       0t0   44675950 pipe
python  23420 zhangsan    8r     FIFO               0,13       0t0   44675951 pipe
python  23420 zhangsan    9w     FIFO               0,13       0t0   44675951 pipe
python  23420 zhangsan   10u      CHR            195,255       0t0        684 /dev/nvidiactl
python  23420 zhangsan   11u      CHR              507,0       0t0        765 /dev/nvidia-uvm
python  23420 zhangsan   12u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   13u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   14u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   15u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   16u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   17u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   18u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   19u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   20u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   21u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   22u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   23u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   24u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   25u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   26u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   27u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   28u     unix 0x0000000000000000       0t0   44674030 @cuda-uvmfd-4026531836-23420@ type=SEQPACKET
python  23420 zhangsan   29u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   30u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   31u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   32u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   33u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   34u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   35u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   36u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   37u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   38u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   39u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   40u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   41u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   42u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   43u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   44u      CHR              195,0       0t0        710 /dev/nvidia0
python  23420 zhangsan   45r     FIFO               0,13       0t0   44699293 pipe
python  23420 zhangsan   46w     FIFO               0,13       0t0   44699293 pipe
python  23420 zhangsan   47u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   48u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   49u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   50u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   51u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   52u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   53u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   54u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   55u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   56u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   57u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   58u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   59u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   60u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   61u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   62u      CHR              195,1       0t0        724 /dev/nvidia1
python  23420 zhangsan   63u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   64u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   65u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   66u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   67u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   68u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   69u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   70u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   71u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   72u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   73u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   74u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   75u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   76u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   77u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   78u      CHR              195,2       0t0        736 /dev/nvidia2
python  23420 zhangsan   79u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   80u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   81u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   82u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   83u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   84u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   85u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   86u  a_inode               0,14         0      11836 [eventfd]
python  23420 zhangsan   87u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   88u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   89u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   90u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   91u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   92u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   93u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   94u      CHR              195,3       0t0        748 /dev/nvidia3
python  23420 zhangsan   95r      REG                0,5         0   44674025 /proc/23420/cmdline
posted @ 2024-04-24 17:04  沙滩炒花蛤  阅读(25)  评论(0编辑  收藏  举报