initProcess + runc init
docker: Error response from daemon: OCI runtime create failed: rpc error: code = Internal desc = Could not run process: container_linux.go:349: starting container process caused "process_linux.go:289: mount bind /run/sockets failed caused \"no such file or directory\"": unknown.
docker: Error response from daemon: OCI runtime create failed: rpc error: code = Internal desc = Could not run process: container_linux.go:349: starting container process caused "process_linux.go:299:
mount bind /run/sockets failed /run/kata-containers/shared/containers/2181a65982bcdfa06619d4fc76ba5b0d2b30df326e3a472e7638df9502ab8ada/rootfs , /run/sockets/xxx not exist caused \"\"": unknown.
docker: Error response from daemon: OCI runtime create failed: rpc error: code = Internal desc = Could not run process: container_linux.go:351: mount bind /run/sockets failed /run/kata-containers/shared/containers/ad6f3c13cf9f0fb5028678e8200177baebdf4797f0d624aa0b117a607bc4e477/rootfs , /run/sockets/xxx not exist caused "": unknown.
RunC 执
行流程与 cgroup 的应用
Container 的创建过程由 factory 调用 create 方法实现，在创建 factory 对象时指定了NewCgroupsManage func，在 factory 创建 container 时调用 func 为容器配置了fs.Manager对象。调用过程 runc create 命令创建容器开始: startContainer() => createContainer() => loadFactory() => libcontainer.New()
libcontainer/factory_linux.go:131
func New(root string, options ...func(*LinuxFactory) error) (Factory, error) {
//...
l := &LinuxFactory{
Root: root,
InitPath: "/proc/self/exe",
InitArgs: []string{os.Args[0], "init"},
Validator: validate.New(),
CriuPath: "criu",
}
Cgroupfs(l) //为LinuxFactory配置NewCgroupsManage实现func
//...
return l, nil
}
初始化配置LinuxFactory对象的NewCgroupsManage的func赋值，func将根据参数配置返回一个fs.Manager对象
libcontainer/factory_linux.go:65
// Cgroupfs is an options func to configure a LinuxFactory to return containers
// that use the native cgroups filesystem implementation to create and manage
// cgroups.
func Cgroupfs(l *LinuxFactory) error {
l.NewCgroupsManager = func(config *configs.Cgroup, paths map[string]string) cgroups.Manager {
return &fs.Manager{
Cgroups: config,
Paths: paths,
}
}
return nil
}
创建 Container 容器对象，返回 linuxContainer 结构。LinuxFactory.NewCgroupsManager() 调用根据全局 config 赋值并返回 Cgroup Manager 对象 fs.Manger
libcontainer/factory_linux.go:188
func (l *LinuxFactory) Create(id string, config *configs.Config) (Container, error) {
//...
c := &linuxContainer{
id: id,
root: containerRoot,
config: config,
initPath: l.InitPath,
initArgs: l.InitArgs,
criuPath: l.CriuPath,
newuidmapPath: l.NewuidmapPath,
newgidmapPath: l.NewgidmapPath,
cgroupManager: l.NewCgroupsManager(config.Cgroups, nil), //为容器指定fs.Manager
}
//...
return c, nil
}从容器的执行流程来看，此时已完成 container 对象的创建，接下来startContainer()中已创建的 runner 对象 run() 方法执行，容器进入运行阶段。执行流程runc run命令：runner.run() => newProcess() => runner.container.Run(process) => linuxContainer.strat() => linuxContainer.newParentProcess(process) => =>linuxContainer.commandTemplate() => linuxContaine.newInitProcess() =>parent.start() => initProcess.start() 。
Parent.start() 执行其实则是 runc init 命令的执行,其基本的执行流程(详细请参考《 RunC 源码通读指南之 Run 》)：
- parentproces 创建runc init子进程，中间会被 /runc/libcontainer/nsenter 劫持(c代码部分preamble)，使 runc init 子进程位于容器配置指定的各个 namespace 内
- parentProcess 用init管道将容器配置信息传输给runc init进程，runc init再据此进行容器的初始化操作。初始化完成之后，再向另一个管道exec.fifo进行写操作，进入阻塞状态
]
func (p *initProcess) start() error { defer p.messageSockPair.parent.Close() err := p.cmd.Start() p.process.ops = p // close the write-side of the pipes (controlled by child) p.messageSockPair.child.Close() p.logFilePair.child.Close() if err != nil { p.process.ops = nil return newSystemErrorWithCause(err, "starting init process command") } // Do this before syncing with child so that no children can escape the // cgroup. We don't need to worry about not doing this and not being root // because we'd be using the rootless cgroup manager in that case. if err := p.manager.Apply(p.pid()); err != nil { return newSystemErrorWithCause(err, "applying cgroup configuration for process") } if p.intelRdtManager != nil { if err := p.intelRdtManager.Apply(p.pid()); err != nil { return newSystemErrorWithCause(err, "applying Intel RDT configuration for process") } } defer func() { if err != nil { // TODO: should not be the responsibility to call here p.manager.Destroy() if p.intelRdtManager != nil { p.intelRdtManager.Destroy() } } }() if _, err := io.Copy(p.messageSockPair.parent, p.bootstrapData); err != nil { return newSystemErrorWithCause(err, "copying bootstrap data to pipe") }
Parent.start() 执行其实则是 runc init 命令的执行,其基本的执行流程(详细请参考《 RunC 源码通读指南之 Run 》)：
- parentproces 创建runc init子进程，中间会被 /runc/libcontainer/nsenter 劫持(c代码部分preamble)，使 runc init 子进程位于容器配置指定的各个 namespace 内
- parentProcess 用init管道将容器配置信息传输给runc init进程，runc init再据此进行容器的初始化操作。初始化完成之后，再向另一个管道exec.fifo进行写操作，进入阻塞状态
InitProcess.start()执行过程中对cgroup 资源组的配置与应用工作
libcontainer/process_linux.go:282
func (p *initProcess) start() error {
defer p.messageSockPair.parent.Close()
// 当前执行空间进程称为bootstrap进程
// 启动了 cmd，即启动了 runc init 命令,创建 runc init 子进程
// 同时也激活了C代码nsenter模块的执行（为了 namespace 的设置 clone 了三个进程parent、child、init）
// C 代码执行后返回 go 代码部分,最后的 init 子进程为了好区分此处命名为" nsInit "（即配置了Namespace的init）
// runc init go代码为容器初始化其它部分(网络、rootfs、路由、主机名、console、安全等)
err := p.cmd.Start() // runc init
//...
// 为进程 runc init 应用 Cgroup （p.cmd.Process.Pid()）
if err := p.manager.Apply(p.pid()); err != nil {
return newSystemErrorWithCause(err, "applying cgroup configuration for process")
}
//...
// messageSockPair 管道写入 bootstrapData
if _, err := io.Copy(p.messageSockPair.parent, p.bootstrapData); err != nil {
return newSystemErrorWithCause(err, "copying bootstrap data to pipe")
}
// 获取 nsInit pid
childPid, err := p.getChildPid()
if err != nil {
return newSystemErrorWithCause(err, "getting the final child's pid from pipe")
}
//...
// 为 nsInit 进程应用 Cgroup
if err := p.manager.Apply(childPid); err != nil {
return newSystemErrorWithCause(err, "applying cgroup configuration for process")
}
// 为 child 进程应用 intel RDT
if p.intelRdtManager != nil {
if err := p.intelRdtManager.Apply(childPid); err != nil {
return newSystemErrorWithCause(err, "applying Intel RDT configuration for process")
}
}
//...
// 解析runc init子进程的所有同步消息，当io.EOF返回
ierr := parseSync(p.messageSockPair.parent, func(sync *syncT) error {
switch sync.Type {
case procReady:
// prestart hooks 启动前执行勾子
if !p.config.Config.Namespaces.Contains(configs.NEWNS) {
// 根据全局配置设置Cgroup
if err := p.manager.Set(p.config.Config); err != nil {
return newSystemErrorWithCause(err, "setting cgroup config for ready process")
}
//...
// 运行执行前勾子
for i, hook := range p.config.Config.Hooks.Prestart {
if err := hook.Run(s); err != nil {
return newSystemErrorWithCausef(err, "running prestart hook %d", i)
}
}
}
}
// 与子进程 runC init 同步
if err := writeSync(p.messageSockPair.parent, procRun); err != nil {
return newSystemErrorWithCause(err, "writing syncT 'run'")
}
sentRun = true
case procHooks