容器基础(五): 实现一个简单容器sdocker

在前面几部分的基础上, 我们更新一下代码，实现一个简单容器 sdocker.

 

sdocker目录构成

linux: # tree
.
├── Makefile
├── cpu-test.c       # 由cpu.c重命名
├── memory-test.cpp  # 由memory.cpp重命名
├── resource.c       # 新增用于资源管理
├── resource.h
├── sdocker_exec.c   # 模拟 docker exec
└── sdocker_run.c    # 由namespace.c更新而来, 模拟 docker run

0 directories, 7 files
linux: #

 

修改点

1. 更新rootfs:
  a) 拷贝cpu-test/memory-test到rootfs/bin目录下, 对应的依赖库也按照先前文章介绍的方法拷贝到rootfs对应路径，方便后续进行cpu和memory测试;
  b) rootfs增加拷贝top命令, 方便查看cpu使用率;
  c) 增加rootfs/root/.bashrc, 用于设置容器的提示符, 内容如下所示:
    linux:~ # cat /var/lib/sdocker/rootfs/root/.bashrc
    export PS1="\h:\w $ "
    linux:~ #

  sdocker启动后的终端提示符如下所示(主机名@容器id: 路径)：
  sdocker@11041:/ $ 

2. 增加sdocker_run程序:
  a) 增加cgroup资源管理文件resource.c, 可以设置memory.limit_in_bytes, memory.swappiness, cpu.cfs_period_us, cpu.cfs_quota_us.
  资源管理会在启动容器进程后在/sys/fs/cgroup/{cpu,memory}下建立目录名为容器id的目录, 程序结束后删除.
  b) 增加信号量和共享内存, 用于主进程和容器进程间传递数据(比如容器id和rootfs路径);
  c) 把proc, sys, dev, dev/pts这些都挂载到容器;
  d) 默认启动的容器mount-propagation type是MS_SHARED, 容器进程mount操作在主机可见, 为了让容器mount操作主机不可见, 设置容器的propagation type为MS_PRIVATE;

3. 增加sdocker_exec程序, 通过注入namespace方式实现类似docker exec功能

 

基础验证

########### 启动容器, 默认执行bash ###########
linux: # ./sdocker_run -h

Usage:  sdocker [OPTIONS] IMAGE

Start a container using bash default

Options:
-c, --cpu-test               start cpu test with high cpu program
-m, --memory-test            start memory test with high memory program
-n, --hostname string        docker hostname, default is sdocker
    --memory-limit bytes     Memory limit, file: memory.limit_in_bytes
    --memory-swappiness int  Tune container memory swappiness (0 to 100) , file: memory.swappiness
    --cpu-period int         Limit CPU CFS (Completely Fair Scheduler) period, file: cpu.cfs_period_us
    --cpu-quota int          Limit CPU CFS (Completely Fair Scheduler) quota, file: cpu.cfs_quota_us
-h, --help                   show help

linux: # ./sdocker_run /var/lib/sdocker/rootfs/
[+] start container with id 11275
sdocker@11275:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
0            1     0  0 07:47 pts/0    00:00:00 /bin/bash
0            2     1  0 07:47 pts/0    00:00:00 ps -ef
sdocker@11275:/ $

########### 启动另一个终端查看 ###########
linux: # ls /sys/fs/cgroup/{cpu,memory}
/sys/fs/cgroup/cpu:
11275                  cpu.cfs_period_us  cpu.shares     cpuacct.usage_percpu
cgroup.clone_children  cpu.cfs_quota_us   cpu.stat       notify_on_release
cgroup.procs           cpu.rt_period_us   cpuacct.stat   release_agent
cgroup.sane_behavior   cpu.rt_runtime_us  cpuacct.usage  tasks

/sys/fs/cgroup/memory:
11275                  memory.low_limit_in_bytes        memory.swappiness
cgroup.clone_children  memory.max_usage_in_bytes        memory.usage_in_bytes
cgroup.event_control   memory.move_charge_at_immigrate  memory.use_hierarchy
cgroup.procs           memory.numa_stat                 notify_on_release
cgroup.sane_behavior   memory.oom_control               release_agent
memory.failcnt         memory.pressure_level            tasks
memory.force_empty     memory.soft_limit_in_bytes
memory.limit_in_bytes  memory.stat
linux: # ./sdocker_exec 11275 /var/lib/sdocker/rootfs/
sdocker@11275:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
0            1     0  0 07:47 pts/0    00:00:00 /bin/bash
0            3     0  0 07:48 pts/1    00:00:00 bash
0            4     3  0 07:48 pts/1    00:00:00 ps -ef
sdocker@11275:/ $ exit
exit
linux: #

 

无资源限制测试cpu-test

########### 启动容器执行cpu-test, 理论cpu占用要到100% ###########
linux: # ./sdocker_run -c /var/lib/sdocker/rootfs/
[+] start container with id 11293


########### 启动另一个终端查看 ###########
linux: # ./sdocker_exec 11293 /var/lib/sdocker/rootfs/
sdocker@11293:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
0            1     0 99 07:54 pts/0    00:00:22 /bin/cpu-test
0            2     0  0 07:54 pts/1    00:00:00 bash
0            3     2  0 07:54 pts/1    00:00:00 ps -ef
$sdocker@11293:/ $ top
top - 07:57:04 up 1 day, 27 min,  0 users,  load average: 0.96, 0.47, 0.19
Tasks:   3 total,   2 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.1 us,  0.0 sy,  0.0 ni, 99.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  32689888 total,  1003220 used, 31686668 free,     2204 buffers
KiB Swap: 33551356 total,        0 used, 33551356 free.   500328 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S   %CPU  %MEM     TIME+ COMMAND
    1 0         20   0    4052    172    136 R 100.00 0.001   3:03.84 cpu-test
    2 0         20   0   11576   2916   2548 S  0.000 0.009   0:00.00 bash
    4 0         20   0   11348   1924   1712 R  0.000 0.006   0:00.00 top
sdocker@11293:/ $ exit
exit
linux: #

 

有资源限制测试cpu-test, 限制cpu使用率20%

########### 启动容器执行cpu-test, 设置cpu占用率不超过20% ###########
linux: # ./sdocker_run -c --cpu-quota 20000 /var/lib/sdocker/rootfs/
[+] start container with id 11362


########### 启动另一个终端查看 ###########
linux: # ./sdocker_exec 11362 /var/lib/sdocker/rootfs/
sdocker@11362:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
0            1     0 20 08:02 pts/0    00:00:03 /bin/cpu-test
0            2     0  0 08:03 pts/1    00:00:00 bash
0            3     2  0 08:03 pts/1    00:00:00 ps -ef
sdocker@11362:/ $ top
top - 08:03:26 up 1 day, 33 min,  0 users,  load average: 0.31, 0.66, 0.40
Tasks:   3 total,   2 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.2 us,  0.0 sy,  0.0 ni, 98.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  32689888 total,  1004588 used, 31685300 free,     2204 buffers
KiB Swap: 33551356 total,        0 used, 33551356 free.   500372 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S   %CPU  %MEM     TIME+ COMMAND
    1 0         20   0    4052    180    140 R 20.333 0.001   0:06.46 cpu-test
    2 0         20   0   11576   2856   2492 S  0.000 0.009   0:00.00 bash
    4 0         20   0   11320   2100   1800 R  0.000 0.006   0:00.00 top
sdocker@11362:/ $ exit
exit
linux: #

 

有资源限制测试memory-test, 设置内存使用上限10M, 并关闭 swap

########### 启动容器执行memory-test, 设置memory使用不超过10M, 一段时间后容器触发oom退出 ###########
linux: # ./sdocker_run --memory-test --memory-limit 10485760  --memory-swappiness 0 /var/lib/sdocker/rootfs/
[+] start container with id 14673
[+] malloc size: 1048576
[+] malloc size: 2097152
[+] malloc size: 3145728
[+] malloc size: 4194304
[+] malloc size: 5242880
[+] malloc size: 6291456
[+] malloc size: 7340032
[+] malloc size: 8388608
[+] malloc size: 9437184
linux: # ps -ef | grep sdocker | grep -v grep
linux: #


########### 启动另一个终端查看, 一段时间后容器退出 ###########
linux: # ./sdocker_exec 14673 /var/lib/sdocker/rootfs/
sdocker@14673:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
0            1     0  0 00:52 pts/0    00:00:00 /bin/memory-test
0            2     0  0 00:52 pts/1    00:00:00 bash
0            3     2  0 00:52 pts/1    00:00:00 ps -ef
sdocker@14673:/ $