Linux Notes | Linux常用命令行笔记

[ show all running processes ]

(1) ps -aux | less

'ps' means: Process Status

The -a option tells ps to list the processes of all users on the system rather than just those of the current user, with the exception of group leaders and processes not associated with a terminal. A group leader is the first member of a group of related processes.

The -u option tells ps to provide detailed information about each process. The -x option adds to the list processes that have no controlling terminal, such as daemons, which are programs that are launched during booting (i.e., computer startup) and run unobtrusively in the background until they are activated by a particular event or condition.

As the list of processes can be quite long and occupy more than a single screen, the output of ps -aux can be piped (i.e., transferred) to the less command, which lets it be viewed one screenful at a time. The output can be advanced one screen forward by pressing the SPACE bar and one screen backward by pressing the b key.

(2) top

the command-line will show a process monitor. The meaning of each column is as follows:

• Process ID
• User
• Priority
• Nice level
• Virtual memory used by process
• Resident memory used by a process
• Shareable memory
• CPU used by process as a percentage
• Memory used by process as a percentage
• Time process has been running
• Command

Some options:

• -h - Show the current version
• -c - This toggles the command column between showing command and program name
• -d - Specify the delay time between refreshing the screen
• -o - Sorts by the named field
• -p - Only show processes with specified process IDs
• -u - Show only processes by the specified user
• -i - Do not show idle tasks

General operations:

Start top, then use Ctrl+s to stop refreshing and check certain tasks, press c to toggle the display between showing command and prgrame name. Use Ctrl+q to quit freezing mode.

 

[ create a new env with conda ]

conda create --name {ENV_NAME} python={PYTHON_VERSION}

There are 2 variables in the command line, ENV_NAME and PYTHON_VERSION. Decide them to your needs.

 

[ Python Virtual Environment ]

 python -m venv

-m: specify the mod. Here we set mod to be "venv" to manage virtual environments.

 

[ show the usage of the Nvidia GPUs]

(1) nvidia-smi 

Some useful option:

-l : Output the status frequently, default parameter is 5. You can make it show every 10 seconds by nvidia-smi -l 10.

-i : Choose to show a certain GPU by this option.

-f : redirect the output to other files.

To get more information officially, visit this doc: http://developer.download.nvidia.com/compute/DCGM/docs/nvidia-smi-367.38.pdf

(2) gpustat

gpustat is another tool to show the status of GPU. Using with watch command is a good way to frequently show sensor data.

gpustat [options]

options:

• --color : Force colored output (even when stdout is not a tty)
• --no-color : Suppress colored output
• -u, --show-user : Display username of the process owner
• -c, --show-cmd : Display the process name
• -p, --show-pid : Display PID of the process
• -P, --show-power : Display GPU power usage and/or limit (draw or draw,limit)
• --watch, -i, --interval : Run in watch mode (equivalent to watch gpustat) if given. Denotes interval between updates.

 

 

[ check GPU information ]

sudo nvidia-smi -q -d INFO1, INFOR,2, ...

-q, --query
Display GPU or Unit info. Displayed info includes all data listed in
the (GPU ATTRIBUTES) or (UNIT ATTRIBUTES) sections of this document.
Some devices and/or environments don't support all possible informa-
tion. Any unsupported data is indicated by a "N/A" in the output. By
default information for all available GPUs or Units is displayed. Use
the -i option to restrict the output to a single GPU or Unit.

-d TYPE, --display=TYPE
Display only selected information: MEMORY, UTILIZATION, ECC, TEMPERA-
TURE, POWER, CLOCK, COMPUTE, PIDS, PERFORMANCE, SUPPORTED_CLOCKS,
PAGE_RETIREMENT, ACCOUNTING Flags can be combined with comma e.g.
"MEMORY,ECC". Sampling data with max, min and avg is also returned for
POWER, UTILIZATION and CLOCK display types. Doesn't work with
-u/--unit or -x/--xml-format flags.

 

[ watch ]

a built-in command of Linux

watch [options] {COMM}

options:

-n：指定指令执行的间隔时间（秒）；
-d：高亮显示指令输出信息不同之处；
-t：不显示标题。

eg. watch -n 1 -d gpustat

 

[ show size of directories ]

du -h --max-depth=1

-h: human understandable

--max-depth: maximam recurrent depth

 

[ show disk usage]

df -h

-h: human understandable

 

df -Th

-T: show the type of filesystem
-h: human readable

 

[ output the line number of a file ]

wc [options] file

wc means word count, the output without any option consists of three numbers which are the numbers of lines, words and bytes.

 

[ download files and install ]

wget

frequently used option: wget -O {NEW_NAME} {FILE_URL}

 

[ stop a process forcibly ]

kill [options]

options:

-a：当处理当前进程时，不限制命令名和进程号的对应关系；
-l <信息编号>：若不加<信息编号>选项，则-l参数会列出全部的信息名称；
-p：指定kill 命令只打印相关进程的进程号，而不发送任何信号；
-s <信息名称或编号>：指定要送出的信息；
-u：指定用户。

-s signals:

HUP     1    终端断线
INT     2    中断（同 Ctrl + C）
QUIT    3    退出（同 Ctrl + \）
TERM   15    终止
KILL    9    强制终止
CONT   18    继续（与STOP相反， fg/bg命令）
STOP   19    暂停（同 Ctrl + Z）

ps -aus to search for pid, then kill it by signal 9.

 

[ 进程杀死后显存占用解除 ]

sudo fuser /dev/nvidia*

查看占用nvidia显卡的进程pid

ps -aux | less 

查看对应进程的状态、所有者、任务等信息，防止误杀

kill -9 pid

杀死僵尸进程

 

[ get the information about the hardware ]

CPU

cpu型号：cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c

cpu个数：cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l

cpu核心数：cat /proc/cpuinfo | grep "cpu cores"| uniq

cpu总线程数：cat /proc/cpuinfo | grep 'processor' | sort -u | wc -l

内存

内存情况：free -h

硬盘

硬盘分区情况：lsblk

lsblk -e7: list all drives exclude loop devices, which has the device No. of 7

lsblk -o (output column) {COLUMN_NAME}

eg. lsblk -o NAME,ROTA 

输出名称和旋转式磁盘两列信息。通过这个命令，可以找SSD。

硬盘分区使用情况：df -h

硬盘容量占用情况：du -h

查看某一个文件或文件夹的大小： du -sh {DIR_NAME}

等价于 du -s -h {DIR_NAME}

-s: summarize, -h: human-readable

统计文件个数：du {DIR_NAME} 或者 du {DIR_NAME} | wc -l

网卡

网卡型号：lspci | grep -i 'eth'

显卡

显卡使用情况：nvidia-smi

 

[ show image file information via command-line ]

concise version: identify {FILE_NAME}

detailed version: identify -verbose {FILE_NAME}

 

[ Disk format ]

Once you get a new disk, you'd better format it to make it more compatible with your system.

First, make sure you have unmounted the disk you want to format: sudo umount -l <device>

Then, format: sudo mkfs -t <FS format, eg, ext4> <device>

Check the information of disk filesystem: lsblk -f

The output would be like:

mke2fs 1.46.5 (30-Dec-2021)
Found a gpt partition table in /dev/sda
Proceed anyway? (y,N) y
Creating filesystem with 976754646 4k blocks and 244195328 inodes
Filesystem UUID: 197a34a0-c155-49a1-a6db-fc3f7d34da2c
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

　　

 

[ 硬盘挂载 ]

重启服务器之后，原来的一块硬盘根目录路径依然存在，但是里面的数据无法访问到，这是因为这块硬盘在服务器重启之后没有挂载。

sudo mount /dev/sdb /media/data/

命令行很简单，但是要找到是哪块物理硬盘掉了，对应挂到哪个挂载点，需要经验积累。

涉及到的命令： lsblk（查看硬盘挂载情况，查看ssd），mount（挂载）。

*挂载时遇到mount: unknown filesystem type 'LVM2_member'：使用逻辑卷Logic Volume的名称挂载即可。

使用 sudo lvdisplay 查看逻辑卷路径 LV path，而后使用 mount {LV_path} {MOUNT_POINT}完成挂载。 

 

[ auto mount disks ]

(refer to: https://www.wangsu.com/document/878/evs-quickstart-format-file-system-linux-auto-mount)

Two ways to set auto mount:

Method 1. 在/etc/rc.local内添加挂载命令

采用/etc/rc.local内添加挂载命令，如果输入命令错误，云服务器重启时不会影响操作系统正常运行。

注：该方法通过盘符进行自动挂载，云硬盘进行挂载卸载操作、云服务器硬重启时盘符会产生改变或者漂移，建议只有一块数据盘（vdb）时采用该方法设置自动挂载。

操作步骤

1. 打开vi /etc/rc.local文件，配置开机自动挂载

2. 执行命令chmod +x /etc/rc.d/rc.local 赋值执行权限

3. 执行命令reboot重启云服务器

4. 执行df –h命令查看磁盘挂载情况，重启后/dev/vdb1仍然挂载在/data目录下

Method 2. 修改配置文件/etc/fstab

注：如果配置文件信息有误，重启云服务器时会进入维护模式，需要修改配置信息正确才能正常进入操作系统。
通过将信息写入etc/fstab中进行自动化挂载云硬盘操作时，建议不要使用盘符以及分区id，建议使用文件系统的UUID，因为当云硬盘涉及到挂载和卸载操作时盘符会产生改变或者漂移。

1. 运行命令 cp /etc/fstab /etc/fstab.bak，备份etc/fstab

2. 运行命令sudo blkid查看文件系统的UUID，复制需要设置开机挂载的文件系统UUID及文件系统类型。这里/dev/vdb1的UUID为468f89f6-32b7-432f-bd98-34d6fd8ad375，文件系统类型为ext4。注意：需要sudo才能看到系统内的所有文件系统。

3. 运行命令echo UUID=468f89f6-32b7-432f-bd98-34d6fd8ad375 {MOUNTING PATH} ext4 defaults 0 0 >> /etc/fstab，向/etc/fstab写入文件系统的信息 （如果系统权限不允许以echo写入，直接vi打开后编辑fstab。

4. 运行命令cat /etc/fstab查看新分区信息

 

[ 创建用户、分配超级用户权限 ]

创建用户： adduser {USER_NAME}，然后根据提示输入密码，之后回车选择default即可。

删除用户：先退出登录的用户，然后userdel {USER_NAME}

若需要同时删除用户目录，userdel --remove-home {USER_NAME}

给超级权限：vim /etc/sudoers

在User privilege specification中加入：{USER_NAME} ALL=(ALL:ALL) ALL即可。

 

[ difference b/t adduser and useradd ]

在使用adduser命令时,它会添加这个用户名,并创建和用户名名称相同的组名,并把这个用户名添加到自己的组里去,并在/home目录想创建和用户名同名的目录,并拷贝/etc/skel目录下的内容到/home/用户名/的目录下,并提示输入密码,并提示填写相关这个用户名的信息。

在使用命令useradd时，它会添加这个用户名，并创建和用户名相同的组名，但它并不在/home目录下创建基于用户名的目录,也不提示创建新的密码。也就是说使用useradd mongo 创建出来的用户,将是默认的"三无"用户,无家目录,无密码,无系统shell,换句话说,它创建的是系统用户,无法用它来登陆系统.

 

[ delete user ]

deluser {USERNAME} --remove-home --backup --backup-to /media/hdd/backup

 

[ 添加已知IP的主机名 ]

/etc/hosts 文件中记录了已知的ip及对应的主机名，在文件中的第一部分添加ip和主机名即可。

 

[ Change file ownership and modification priority ]

1. chown <username>:<group name> <filename>

-R: operate recursively

2. chmod <priority code> <filename>

-R: operate recursively

priority code: 777 = rwxrwxrwx, 750 = rwxr-x---

 

[Change host name]

Two files need to be changed: (1) /etc/hostname, (2) /etc/hosts

Replace all old names in these two files with the new one

 

[ Set up SSH keys for Ubuntu server connection] 

1. Create key pair: ssh-keygen

# the tool will ask for the path to save and a passphrase, keep default if no special need
# Remember the path where the key pair is saved to

2. copy the public key to the server: ssh-copy-id username@remote_host

3. if using the ssh command-line to connect, then the ssh will automatically search the .ssh folder for the private key

　　If the SSH client is used, assign the private key in the client settings

 

 [ edit linux welcom message (MOTD) ]

When SSH connection established successfully, system will execute MOTD (Messages Of The Day).

MOTD are files stored at /etc/update-mote.d/

Generall operation to modify MOTD is to add a new file under the folder and chmod +x to set the file executable. The files are executed in the order of the filename prefixes.

 

To make different color output:

RED='\033[0;31m'
NC='\033[0m' # No Color
echo -e "I ${RED}love${NC} Stack Overflow"

 

[ Shell / Bash / sh ]

Shell is a kind of software which provides an interface for users to operate the core of OS using cammand lines.

Common Shells:

• Bourne Shell（/usr/bin/sh或/bin/sh）
• Bourne Again Shell（/bin/bash）
• C Shell（/usr/bin/csh）
• K Shell（/usr/bin/ksh）
• Shell for Root（/sbin/sh）

bash is the most common Shell for modern Linux systems.

 

[ Group management ]

0. groups

查看群组信息

groups: 单独使用，直接输出自己的群组信息

groups <username>: 输出指定用户的群组信息

1. add / delete group

sudo groupadd groupname

sudo groupdel groupname

2. gpasswd: 

gpasswd 是 Linux 下工作组文件 /etc/group 和 /etc/gshadow 管理工具，用于将一个用户添加到组或者从组中删除。

语法

gpasswd [可选项] 组名

可选项参数：

• -a：添加用户到组；
• -d：从组删除用户；
• -A：指定管理员；
• -M：指定组成员和-A的用途差不多；
• -r：删除密码；
• -R：限制用户登入组，只有组中的成员才可以用newgrp加入该组。

Examples

gpasswd -a <username> <groupname>: 添加用戶到指定組
gpasswd -d <username> <groupname>: 从指定组移除用户

3. usermode

usermod可用来修改用户帐号的各项设定。

语法

usermod [-LU][-a <附加>] [-c <备注>][-d <登入目录>][-e <有效期限>][-f <缓冲天数>][-g <群组>][-G <群组>][-l <帐号名称>][-s <shell>][-u <uid>][用户帐号]

参数说明：

• -a: appending，若不使用此选项，指定用户到群组之后，会清除掉原来的群组信息
• -c<备注> 　修改用户帐号的备注文字。
• -d登入目录> 　修改用户登入时的目录。
• -e<有效期限> 　修改帐号的有效期限。
• -f<缓冲天数> 　修改在密码过期后多少天即关闭该帐号。
• -g<群组> 　修改用户所属的群组。
• -G<群组> 　修改用户所属的附加群组。
• -l<帐号名称> 　修改用户帐号名称。
• -L 　锁定用户密码，使密码无效。
• -s<shell> 　修改用户登入后所使用的shell。
• -u<uid> 　修改用户ID。
• -U 　解除密码锁定。

Examples

sudo usermod -a -G group1,group2 username: 将group1和group2添加为用户的附加群组，注意要使用 -a 选项！否则会清楚原有群组信息
sudo usermod -g groupname username: 改变用户的主要群组（primary group）